2024-09-11T22:31:58.746 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T22:31:58.772 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T22:31:58.859 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901430 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901430' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi152.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPF/yIggPWKqVJsdmWuRnjeyry1Utd006wJ/blVQsK7hMmewWJfRNmYeJpefnGgsRjYEHXkpzveYZN+rByZpjP4= smithi191.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHlraAsxblYEwXvpFSNmaA+wMbxrMYVyr9OCA6PwBrKK8smhZJdnrzjZmBMmpZk/r6MN1NjGrjYJoRI6TQ6fxgA= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOWb1FELX+VPkND/4VNUWbWmEWo2ruvIcTjQ6zfNW/8dxjkXZlOjoni4d91D8qNelhIOfLfcpeh5wOEVIZs066k= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T22:31:58.859 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-11T22:31:58.860 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-11T22:31:58.860 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T22:31:58.861 INFO:teuthology.task.internal:Checking packages... 2024-09-11T22:31:58.883 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-11T22:31:58.883 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T22:31:58.883 INFO:teuthology.packaging:ref: None 2024-09-11T22:31:58.884 INFO:teuthology.packaging:tag: None 2024-09-11T22:31:58.884 INFO:teuthology.packaging:branch: squid 2024-09-11T22:31:58.884 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:31:58.884 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-11T22:31:59.122 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-11T22:31:59.123 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T22:31:59.129 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T22:31:59.129 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T22:31:59.143 INFO:teuthology.task.internal:Saving configuration 2024-09-11T22:31:59.164 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T22:31:59.170 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T22:31:59.196 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901430', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 22:26:36.501961', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPF/yIggPWKqVJsdmWuRnjeyry1Utd006wJ/blVQsK7hMmewWJfRNmYeJpefnGgsRjYEHXkpzveYZN+rByZpjP4='} 2024-09-11T22:31:59.220 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901430', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 22:26:36.501148', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHlraAsxblYEwXvpFSNmaA+wMbxrMYVyr9OCA6PwBrKK8smhZJdnrzjZmBMmpZk/r6MN1NjGrjYJoRI6TQ6fxgA='} 2024-09-11T22:31:59.239 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901430', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 22:26:36.500116', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOWb1FELX+VPkND/4VNUWbWmEWo2ruvIcTjQ6zfNW/8dxjkXZlOjoni4d91D8qNelhIOfLfcpeh5wOEVIZs066k='} 2024-09-11T22:31:59.240 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T22:31:59.246 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-11T22:31:59.246 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-11T22:31:59.246 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-11T22:31:59.246 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T22:31:59.330 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe59d613be0>, signals=[15]) 2024-09-11T22:31:59.330 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T22:31:59.336 INFO:teuthology.task.internal:Opening connections... 2024-09-11T22:31:59.336 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi152.front.sepia.ceph.com 2024-09-11T22:31:59.338 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:31:59.415 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2024-09-11T22:31:59.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:31:59.491 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-09-11T22:31:59.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:31:59.572 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T22:31:59.578 DEBUG:teuthology.orchestra.run.smithi152:> uname -m 2024-09-11T22:31:59.597 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2024-09-11T22:31:59.597 DEBUG:teuthology.orchestra.run.smithi152:> cat /etc/os-release 2024-09-11T22:31:59.656 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="CentOS Stream" 2024-09-11T22:31:59.656 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="9" 2024-09-11T22:31:59.656 INFO:teuthology.orchestra.run.smithi152.stdout:ID="centos" 2024-09-11T22:31:59.656 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="rhel fedora" 2024-09-11T22:31:59.656 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="9" 2024-09-11T22:31:59.657 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el9" 2024-09-11T22:31:59.657 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T22:31:59.657 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2024-09-11T22:31:59.657 INFO:teuthology.orchestra.run.smithi152.stdout:LOGO="fedora-logo-icon" 2024-09-11T22:31:59.657 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T22:31:59.658 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://centos.org/" 2024-09-11T22:31:59.658 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T22:31:59.658 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T22:31:59.658 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T22:31:59.659 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2024-09-11T22:31:59.837 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2024-09-11T22:31:59.857 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2024-09-11T22:31:59.857 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-09-11T22:31:59.914 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2024-09-11T22:31:59.914 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="9" 2024-09-11T22:31:59.915 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2024-09-11T22:31:59.915 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2024-09-11T22:31:59.915 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="9" 2024-09-11T22:31:59.915 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el9" 2024-09-11T22:31:59.915 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T22:31:59.915 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2024-09-11T22:31:59.915 INFO:teuthology.orchestra.run.smithi191.stdout:LOGO="fedora-logo-icon" 2024-09-11T22:31:59.915 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T22:31:59.915 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2024-09-11T22:31:59.915 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T22:31:59.915 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T22:31:59.915 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T22:31:59.916 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-09-11T22:31:59.943 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-09-11T22:31:59.962 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-09-11T22:31:59.962 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-09-11T22:32:00.020 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-09-11T22:32:00.020 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-09-11T22:32:00.020 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-09-11T22:32:00.021 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-09-11T22:32:00.021 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-09-11T22:32:00.021 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-09-11T22:32:00.021 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T22:32:00.021 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-09-11T22:32:00.021 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-09-11T22:32:00.021 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T22:32:00.021 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-09-11T22:32:00.021 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T22:32:00.021 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T22:32:00.021 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T22:32:00.022 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-09-11T22:32:00.043 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T22:32:00.053 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T22:32:00.088 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T22:32:00.089 DEBUG:teuthology.orchestra.run.smithi152:> test '!' -e /home/ubuntu/cephtest 2024-09-11T22:32:00.091 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2024-09-11T22:32:00.093 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-09-11T22:32:00.110 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T22:32:00.116 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T22:32:00.116 DEBUG:teuthology.orchestra.run.smithi152:> test -z $(ls -A /var/lib/ceph) 2024-09-11T22:32:00.149 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2024-09-11T22:32:00.152 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-09-11T22:32:00.240 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T22:32:00.317 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T22:32:00.331 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-11T22:32:00.331 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-11T22:32:00.331 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-11T22:32:00.332 DEBUG:teuthology.orchestra.run.smithi152:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T22:32:00.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:32:00.349 DEBUG:teuthology.orchestra.run.smithi152:> uname -r 2024-09-11T22:32:00.409 INFO:teuthology.orchestra.run.smithi152.stdout:5.14.0-503.el9.x86_64 2024-09-11T22:32:00.410 INFO:teuthology.task.kernel:Running kernel on smithi152: 5.14.0-503.el9.x86_64 2024-09-11T22:32:00.410 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum install -y kernel 2024-09-11T22:32:03.082 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-11T22:32:15.856 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - AppStream 1.7 MB/s | 20 MB 00:11 2024-09-11T22:32:22.598 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.5 MB 00:02 2024-09-11T22:32:23.990 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - Extras packages 99 kB/s | 19 kB 00:00 2024-09-11T22:32:24.778 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-11T22:32:30.209 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-11T22:32:32.224 INFO:teuthology.orchestra.run.smithi152.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T22:32:32.224 INFO:teuthology.orchestra.run.smithi152.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T22:32:32.323 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-11T22:32:32.329 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-11T22:32:32.329 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2024-09-11T22:32:32.329 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-11T22:32:32.329 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-11T22:32:32.329 INFO:teuthology.orchestra.run.smithi152.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T22:32:32.329 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-09-11T22:32:32.330 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T22:32:32.330 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T22:32:32.330 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T22:32:32.330 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:32:32.330 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-11T22:32:32.330 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2024-09-11T22:32:32.330 INFO:teuthology.orchestra.run.smithi152.stdout:Install 4 Packages 2024-09-11T22:32:32.330 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:32:32.331 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 79 M 2024-09-11T22:32:32.331 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 126 M 2024-09-11T22:32:32.331 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-11T22:32:32.452 INFO:teuthology.orchestra.run.smithi152.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 254 kB/s | 23 kB 00:00 2024-09-11T22:32:32.952 INFO:teuthology.orchestra.run.smithi152.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-11T22:32:33.278 INFO:teuthology.orchestra.run.smithi152.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-11T22:32:33.678 INFO:teuthology.orchestra.run.smithi152.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 23 MB/s | 29 MB 00:01 2024-09-11T22:32:33.678 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-11T22:32:33.678 INFO:teuthology.orchestra.run.smithi152.stdout:Total 58 MB/s | 79 MB 00:01 2024-09-11T22:32:34.355 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-11T22:32:34.704 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-11T22:32:34.704 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-11T22:32:35.695 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-11T22:32:35.695 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-11T22:32:38.189 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-11T22:32:38.863 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T22:32:38.946 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T22:32:41.746 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T22:32:41.892 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T22:32:46.737 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T22:32:46.843 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T22:32:51.695 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T22:33:29.901 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T22:33:29.941 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T22:33:30.286 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T22:33:30.287 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T22:33:30.287 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T22:33:30.287 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T22:33:31.181 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T22:33:31.181 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:33:31.181 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-11T22:33:31.181 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T22:33:31.181 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T22:33:31.181 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T22:33:31.181 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T22:33:31.181 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:33:31.182 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-11T22:33:31.557 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T22:33:31.635 INFO:teuthology.orchestra.run.smithi152.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T22:33:31.635 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T22:33:31.635 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T22:33:31.636 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T22:33:31.636 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T22:33:31.636 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-11T22:33:31.636 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi152.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T22:33:31.636 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T22:33:31.696 INFO:teuthology.orchestra.run.smithi152.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T22:33:31.696 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T22:33:31.696 DEBUG:teuthology.orchestra.run.smithi152:> sudo rpm -qi grub2-tools 2024-09-11T22:33:31.728 INFO:teuthology.orchestra.run.smithi152.stdout:Name : grub2-tools 2024-09-11T22:33:31.728 INFO:teuthology.orchestra.run.smithi152.stdout:Epoch : 1 2024-09-11T22:33:31.728 INFO:teuthology.orchestra.run.smithi152.stdout:Version : 2.06 2024-09-11T22:33:31.728 INFO:teuthology.orchestra.run.smithi152.stdout:Release : 82.el9 2024-09-11T22:33:31.729 INFO:teuthology.orchestra.run.smithi152.stdout:Architecture: x86_64 2024-09-11T22:33:31.729 INFO:teuthology.orchestra.run.smithi152.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T22:33:31.729 INFO:teuthology.orchestra.run.smithi152.stdout:Group : Unspecified 2024-09-11T22:33:31.729 INFO:teuthology.orchestra.run.smithi152.stdout:Size : 8274670 2024-09-11T22:33:31.729 INFO:teuthology.orchestra.run.smithi152.stdout:License : GPLv3+ 2024-09-11T22:33:31.729 INFO:teuthology.orchestra.run.smithi152.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T22:33:31.729 INFO:teuthology.orchestra.run.smithi152.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T22:33:31.729 INFO:teuthology.orchestra.run.smithi152.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T22:33:31.729 INFO:teuthology.orchestra.run.smithi152.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T22:33:31.729 INFO:teuthology.orchestra.run.smithi152.stdout:Packager : builder@centos.org 2024-09-11T22:33:31.729 INFO:teuthology.orchestra.run.smithi152.stdout:Vendor : CentOS 2024-09-11T22:33:31.729 INFO:teuthology.orchestra.run.smithi152.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T22:33:31.729 INFO:teuthology.orchestra.run.smithi152.stdout:Summary : Support tools for GRUB. 2024-09-11T22:33:31.729 INFO:teuthology.orchestra.run.smithi152.stdout:Description : 2024-09-11T22:33:31.730 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:33:31.730 INFO:teuthology.orchestra.run.smithi152.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T22:33:31.730 INFO:teuthology.orchestra.run.smithi152.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T22:33:31.730 INFO:teuthology.orchestra.run.smithi152.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T22:33:31.730 INFO:teuthology.orchestra.run.smithi152.stdout:hardware devices. 2024-09-11T22:33:31.730 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:33:31.730 INFO:teuthology.orchestra.run.smithi152.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T22:33:31.731 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T22:33:31.731 INFO:teuthology.task.kernel:Updating grub on smithi152 to boot 5.14.0-505.el9.x86_64 2024-09-11T22:33:31.731 DEBUG:teuthology.orchestra.run.smithi152:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T22:33:32.469 INFO:teuthology.orchestra.run.smithi152.stderr:Generating grub configuration file ... 2024-09-11T22:33:33.866 INFO:teuthology.orchestra.run.smithi152.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T22:33:33.892 INFO:teuthology.orchestra.run.smithi152.stderr:done 2024-09-11T22:33:33.894 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2024-09-11T22:33:33.911 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.cjs2X1J1OE 2024-09-11T22:33:33.911 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.cjs2X1J1OE 2024-09-11T22:33:33.983 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.cjs2X1J1OE 2024-09-11T22:33:34.124 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.cjs2X1J1OE is 6KB 2024-09-11T22:33:34.173 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.cjs2X1J1OE 2024-09-11T22:33:34.188 DEBUG:teuthology.orchestra.run.smithi152:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T22:33:34.256 INFO:teuthology.orchestra.run.smithi152.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T22:33:34.256 INFO:teuthology.orchestra.run.smithi152.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T22:33:34.256 INFO:teuthology.orchestra.run.smithi152.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T22:33:34.256 INFO:teuthology.orchestra.run.smithi152.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T22:33:34.257 DEBUG:teuthology.orchestra.run.smithi152:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T22:33:34.421 DEBUG:teuthology.orchestra.run.smithi152:> sudo shutdown -r now 2024-09-11T22:34:04.424 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-11T22:34:04.424 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-09-11T22:34:04.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:34:22.909 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-09-11T22:34:31.910 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-09-11T22:34:31.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:34:34.973 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2024-09-11T22:34:46.973 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-09-11T22:34:46.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:35:46.977 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T22:36:01.977 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-09-11T22:36:01.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:36:02.375 DEBUG:teuthology.orchestra.run.smithi152:> true 2024-09-11T22:36:02.756 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-09-11T22:36:02.756 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-11T22:36:02.756 DEBUG:teuthology.orchestra.run.smithi152:> uname -r 2024-09-11T22:36:02.772 INFO:teuthology.orchestra.run.smithi152.stdout:5.14.0-505.el9.x86_64 2024-09-11T22:36:02.773 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T22:36:02.773 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T22:36:02.773 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T22:36:03.774 DEBUG:teuthology.parallel:result is None 2024-09-11T22:36:03.774 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T22:36:03.781 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T22:36:03.782 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T22:36:03.784 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T22:36:03.787 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T22:36:03.805 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T22:36:03.811 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T22:36:03.822 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T22:36:03.823 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T22:36:03.843 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T22:36:03.846 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T22:36:03.955 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T22:36:03.961 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T22:36:03.961 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T22:36:03.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:36:03.978 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T22:36:03.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:36:03.996 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T22:36:03.999 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T22:36:04.021 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T22:36:04.061 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T22:36:04.082 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T22:36:04.099 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T22:36:04.100 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T22:36:04.118 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T22:36:04.120 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T22:36:04.122 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T22:36:04.129 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T22:36:04.129 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T22:36:04.131 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T22:36:04.134 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T22:36:04.161 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T22:36:04.171 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T22:36:04.171 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T22:36:04.201 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T22:36:04.203 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T22:36:04.220 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T22:36:04.304 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T22:36:04.354 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T22:36:04.438 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T22:36:04.479 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:36:04.479 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T22:36:04.545 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T22:36:04.592 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T22:36:04.643 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T22:36:04.726 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T22:36:04.772 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-11T22:36:04.772 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T22:36:04.838 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T22:36:04.883 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T22:36:04.938 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T22:36:05.020 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T22:36:05.067 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-11T22:36:05.068 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T22:36:05.136 DEBUG:teuthology.orchestra.run.smithi152:> sudo service rsyslog restart 2024-09-11T22:36:05.139 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2024-09-11T22:36:05.142 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-09-11T22:36:05.178 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T22:36:05.193 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T22:36:05.222 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T22:36:05.658 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T22:36:05.665 INFO:teuthology.task.internal:Starting timer... 2024-09-11T22:36:05.665 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T22:36:05.679 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T22:36:05.687 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-11T22:36:05.687 DEBUG:teuthology.orchestra.run.smithi152:> sudo service auditd rotate 2024-09-11T22:36:05.763 INFO:teuthology.orchestra.run.smithi152.stdout:Rotating logs: 2024-09-11T22:36:05.765 DEBUG:teuthology.orchestra.run.smithi191:> sudo service auditd rotate 2024-09-11T22:36:05.835 INFO:teuthology.orchestra.run.smithi191.stdout:Rotating logs: 2024-09-11T22:36:05.837 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-09-11T22:36:05.896 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-09-11T22:36:05.898 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T22:36:05.898 DEBUG:teuthology.orchestra.run.smithi152:> /usr/sbin/getenforce 2024-09-11T22:36:05.919 INFO:teuthology.orchestra.run.smithi152.stdout:Permissive 2024-09-11T22:36:05.919 DEBUG:teuthology.orchestra.run.smithi191:> /usr/sbin/getenforce 2024-09-11T22:36:05.947 INFO:teuthology.orchestra.run.smithi191.stdout:Permissive 2024-09-11T22:36:05.947 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-09-11T22:36:05.976 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-09-11T22:36:05.977 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi152.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi191.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-09-11T22:36:05.977 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T22:36:06.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:36:06.005 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T22:36:06.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:36:06.033 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T22:36:06.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:36:06.061 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T22:36:06.061 DEBUG:teuthology.orchestra.run.smithi152:> sudo /usr/sbin/setenforce permissive 2024-09-11T22:36:06.097 DEBUG:teuthology.orchestra.run.smithi191:> sudo /usr/sbin/setenforce permissive 2024-09-11T22:36:06.126 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-09-11T22:36:06.169 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T22:36:06.179 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T22:36:06.192 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T22:36:06.193 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi152.front.sepia.ceph.com,smithi191.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T22:40:09.646 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi152.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-09-11T22:40:09.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-09-11T22:40:09.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:40:09.771 DEBUG:teuthology.orchestra.run.smithi152:> true 2024-09-11T22:40:09.849 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-09-11T22:40:09.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-11T22:40:09.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:40:09.955 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-09-11T22:40:10.038 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-09-11T22:40:10.038 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-11T22:40:10.039 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:40:10.139 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-09-11T22:40:10.222 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-09-11T22:40:10.223 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T22:40:10.234 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T22:40:10.234 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T22:40:10.235 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T22:40:10.237 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T22:40:10.237 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T22:40:10.240 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T22:40:10.240 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T22:40:10.266 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T22:40:10.271 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T22:40:10.281 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T22:40:10.287 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T22:40:10.295 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T22:40:10.306 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: ntpd: command not found 2024-09-11T22:40:10.311 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T22:40:10.314 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: ntpd: command not found 2024-09-11T22:40:10.317 INFO:teuthology.orchestra.run.smithi152.stdout:506 Cannot talk to daemon 2024-09-11T22:40:10.325 INFO:teuthology.orchestra.run.smithi191.stdout:506 Cannot talk to daemon 2024-09-11T22:40:10.331 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T22:40:10.337 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-09-11T22:40:10.340 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T22:40:10.346 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T22:40:10.349 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-09-11T22:40:10.354 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T22:40:10.364 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T22:40:10.378 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T22:40:10.399 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2024-09-11T22:40:10.402 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T22:40:10.403 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-09-11T22:40:10.403 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:40:10.403 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:40:10.403 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:40:10.403 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:40:10.416 INFO:teuthology.orchestra.run.smithi191.stderr:bash: line 1: ntpq: command not found 2024-09-11T22:40:10.419 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T22:40:10.420 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-09-11T22:40:10.420 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:40:10.420 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:40:10.420 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:40:10.420 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:40:10.433 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-09-11T22:40:10.438 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T22:40:10.438 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-09-11T22:40:10.438 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:40:10.438 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:40:10.438 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:40:10.438 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:40:10.439 INFO:teuthology.run_tasks:Running task install... 2024-09-11T22:40:10.447 DEBUG:teuthology.task.install:project ceph 2024-09-11T22:40:10.448 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-11T22:40:10.448 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-11T22:40:10.448 INFO:teuthology.task.install:Using flavor: default 2024-09-11T22:40:10.455 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T22:40:10.455 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-11T22:40:10.455 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T22:40:10.456 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:40:10.458 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T22:40:10.458 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:40:10.459 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T22:40:10.459 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:40:10.611 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T22:40:10.611 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T22:40:10.628 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T22:40:10.628 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T22:40:10.644 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T22:40:10.644 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T22:40:10.747 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T22:40:10.747 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:40:10.747 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T22:40:10.752 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T22:40:10.752 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-11T22:40:10.752 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T22:40:10.768 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T22:40:10.768 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-11T22:40:10.768 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T22:40:10.773 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T22:40:10.773 DEBUG:teuthology.orchestra.run.smithi152:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T22:40:10.778 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T22:40:10.778 DEBUG:teuthology.orchestra.run.smithi191:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T22:40:10.794 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T22:40:10.794 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T22:40:10.839 DEBUG:teuthology.orchestra.run.smithi152:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T22:40:10.844 DEBUG:teuthology.orchestra.run.smithi191:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T22:40:10.861 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T22:40:10.914 DEBUG:teuthology.orchestra.run.smithi152:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T22:40:10.918 DEBUG:teuthology.orchestra.run.smithi191:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T22:40:10.937 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T22:40:10.979 INFO:teuthology.orchestra.run.smithi152.stdout:check_obsoletes = 1 2024-09-11T22:40:10.980 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum clean all 2024-09-11T22:40:10.982 INFO:teuthology.orchestra.run.smithi191.stdout:check_obsoletes = 1 2024-09-11T22:40:10.984 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum clean all 2024-09-11T22:40:11.002 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-09-11T22:40:11.003 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-09-11T22:40:11.210 INFO:teuthology.orchestra.run.smithi191.stdout:45 files removed 2024-09-11T22:40:11.216 INFO:teuthology.orchestra.run.smithi152.stdout:45 files removed 2024-09-11T22:40:11.229 INFO:teuthology.orchestra.run.smithi204.stdout:45 files removed 2024-09-11T22:40:11.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T22:40:11.242 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T22:40:11.257 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T22:40:11.793 INFO:teuthology.orchestra.run.smithi191.stdout:ceph packages for x86_64 302 kB/s | 84 kB 00:00 2024-09-11T22:40:11.802 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 313 kB/s | 84 kB 00:00 2024-09-11T22:40:11.811 INFO:teuthology.orchestra.run.smithi152.stdout:ceph packages for x86_64 296 kB/s | 84 kB 00:00 2024-09-11T22:40:12.069 INFO:teuthology.orchestra.run.smithi191.stdout:ceph noarch packages 50 kB/s | 12 kB 00:00 2024-09-11T22:40:12.077 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 50 kB/s | 12 kB 00:00 2024-09-11T22:40:12.095 INFO:teuthology.orchestra.run.smithi152.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-11T22:40:12.336 INFO:teuthology.orchestra.run.smithi191.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-11T22:40:12.337 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-09-11T22:40:12.371 INFO:teuthology.orchestra.run.smithi152.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-11T22:40:12.919 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-11T22:40:12.979 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-11T22:40:13.904 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - BaseOS 5.5 MB/s | 8.2 MB 00:01 2024-09-11T22:40:14.947 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-11T22:40:16.080 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-09-11T22:40:18.753 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - AppStream 4.3 MB/s | 20 MB 00:04 2024-09-11T22:40:20.623 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.5 MB 00:00 2024-09-11T22:40:21.990 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - Extras packages 125 kB/s | 19 kB 00:00 2024-09-11T22:40:22.847 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.5 MB 00:04 2024-09-11T22:40:23.194 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-11T22:40:24.004 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.5 MB 00:01 2024-09-11T22:40:24.256 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 102 kB/s | 19 kB 00:00 2024-09-11T22:40:25.194 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-11T22:40:25.421 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - Extras packages 91 kB/s | 19 kB 00:00 2024-09-11T22:40:26.259 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-11T22:40:28.608 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-09-11T22:40:30.675 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-11T22:40:30.945 INFO:teuthology.orchestra.run.smithi152.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:40:30.946 INFO:teuthology.orchestra.run.smithi152.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:40:30.960 INFO:teuthology.orchestra.run.smithi152.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T22:40:30.961 INFO:teuthology.orchestra.run.smithi152.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T22:40:30.961 INFO:teuthology.orchestra.run.smithi152.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T22:40:30.962 INFO:teuthology.orchestra.run.smithi152.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T22:40:30.964 INFO:teuthology.orchestra.run.smithi152.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T22:40:30.967 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T22:40:30.968 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T22:40:31.032 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-09-11T22:40:31.047 INFO:teuthology.orchestra.run.smithi152.stdout:====================================================================================== 2024-09-11T22:40:31.047 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-09-11T22:40:31.047 INFO:teuthology.orchestra.run.smithi152.stdout:====================================================================================== 2024-09-11T22:40:31.047 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-09-11T22:40:31.047 INFO:teuthology.orchestra.run.smithi152.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T22:40:31.047 INFO:teuthology.orchestra.run.smithi152.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T22:40:31.048 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T22:40:31.048 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T22:40:31.048 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T22:40:31.048 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T22:40:31.048 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T22:40:31.048 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T22:40:31.048 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T22:40:31.048 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T22:40:31.048 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T22:40:31.048 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T22:40:31.048 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T22:40:31.048 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T22:40:31.049 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T22:40:31.049 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T22:40:31.049 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T22:40:31.049 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T22:40:31.049 INFO:teuthology.orchestra.run.smithi152.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T22:40:31.049 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T22:40:31.049 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T22:40:31.049 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T22:40:31.049 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T22:40:31.049 INFO:teuthology.orchestra.run.smithi152.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T22:40:31.049 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T22:40:31.049 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T22:40:31.050 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T22:40:31.050 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T22:40:31.050 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T22:40:31.050 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T22:40:31.050 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T22:40:31.050 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T22:40:31.050 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T22:40:31.050 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T22:40:31.050 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T22:40:31.050 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T22:40:31.050 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T22:40:31.050 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrading: 2024-09-11T22:40:31.050 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T22:40:31.051 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T22:40:31.051 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T22:40:31.051 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T22:40:31.051 INFO:teuthology.orchestra.run.smithi152.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T22:40:31.051 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T22:40:31.051 INFO:teuthology.orchestra.run.smithi152.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T22:40:31.051 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T22:40:31.051 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-09-11T22:40:31.051 INFO:teuthology.orchestra.run.smithi152.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T22:40:31.051 INFO:teuthology.orchestra.run.smithi152.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T22:40:31.051 INFO:teuthology.orchestra.run.smithi152.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T22:40:31.051 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T22:40:31.051 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T22:40:31.051 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T22:40:31.052 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T22:40:31.052 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T22:40:31.052 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T22:40:31.052 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T22:40:31.052 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T22:40:31.052 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T22:40:31.052 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T22:40:31.052 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T22:40:31.052 INFO:teuthology.orchestra.run.smithi152.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T22:40:31.052 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T22:40:31.052 INFO:teuthology.orchestra.run.smithi152.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T22:40:31.052 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T22:40:31.052 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T22:40:31.053 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T22:40:31.053 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T22:40:31.053 INFO:teuthology.orchestra.run.smithi152.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T22:40:31.053 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T22:40:31.053 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T22:40:31.053 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T22:40:31.053 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T22:40:31.053 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T22:40:31.053 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T22:40:31.053 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T22:40:31.053 INFO:teuthology.orchestra.run.smithi152.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T22:40:31.053 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T22:40:31.054 INFO:teuthology.orchestra.run.smithi152.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T22:40:31.054 INFO:teuthology.orchestra.run.smithi152.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T22:40:31.054 INFO:teuthology.orchestra.run.smithi152.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T22:40:31.054 INFO:teuthology.orchestra.run.smithi152.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T22:40:31.054 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T22:40:31.054 INFO:teuthology.orchestra.run.smithi152.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T22:40:31.054 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T22:40:31.054 INFO:teuthology.orchestra.run.smithi152.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T22:40:31.054 INFO:teuthology.orchestra.run.smithi152.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T22:40:31.054 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T22:40:31.054 INFO:teuthology.orchestra.run.smithi152.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T22:40:31.054 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T22:40:31.054 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T22:40:31.054 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T22:40:31.055 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T22:40:31.055 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T22:40:31.055 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T22:40:31.055 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T22:40:31.055 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T22:40:31.055 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T22:40:31.055 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T22:40:31.055 INFO:teuthology.orchestra.run.smithi152.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T22:40:31.055 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T22:40:31.055 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T22:40:31.055 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T22:40:31.055 INFO:teuthology.orchestra.run.smithi152.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T22:40:31.055 INFO:teuthology.orchestra.run.smithi152.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T22:40:31.056 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T22:40:31.056 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T22:40:31.056 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T22:40:31.056 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T22:40:31.056 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T22:40:31.056 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T22:40:31.056 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T22:40:31.056 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T22:40:31.056 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T22:40:31.056 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T22:40:31.056 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T22:40:31.056 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T22:40:31.056 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T22:40:31.057 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T22:40:31.057 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T22:40:31.057 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T22:40:31.057 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T22:40:31.057 INFO:teuthology.orchestra.run.smithi152.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T22:40:31.057 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T22:40:31.057 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T22:40:31.057 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T22:40:31.057 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T22:40:31.057 INFO:teuthology.orchestra.run.smithi152.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T22:40:31.057 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T22:40:31.057 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T22:40:31.057 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T22:40:31.058 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T22:40:31.058 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T22:40:31.058 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T22:40:31.058 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T22:40:31.058 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T22:40:31.058 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T22:40:31.058 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T22:40:31.058 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T22:40:31.058 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T22:40:31.058 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T22:40:31.058 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T22:40:31.058 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T22:40:31.058 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T22:40:31.059 INFO:teuthology.orchestra.run.smithi152.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T22:40:31.059 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T22:40:31.059 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T22:40:31.059 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T22:40:31.059 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T22:40:31.059 INFO:teuthology.orchestra.run.smithi152.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T22:40:31.059 INFO:teuthology.orchestra.run.smithi152.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T22:40:31.059 INFO:teuthology.orchestra.run.smithi152.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T22:40:31.059 INFO:teuthology.orchestra.run.smithi152.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T22:40:31.059 INFO:teuthology.orchestra.run.smithi152.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T22:40:31.059 INFO:teuthology.orchestra.run.smithi152.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T22:40:31.059 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T22:40:31.059 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T22:40:31.060 INFO:teuthology.orchestra.run.smithi152.stdout:Installing weak dependencies: 2024-09-11T22:40:31.060 INFO:teuthology.orchestra.run.smithi152.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T22:40:31.060 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:40:31.060 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-09-11T22:40:31.060 INFO:teuthology.orchestra.run.smithi152.stdout:====================================================================================== 2024-09-11T22:40:31.060 INFO:teuthology.orchestra.run.smithi152.stdout:Install 148 Packages 2024-09-11T22:40:31.060 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrade 8 Packages 2024-09-11T22:40:31.060 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:40:31.061 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 226 M 2024-09-11T22:40:31.061 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-09-11T22:40:31.368 INFO:teuthology.orchestra.run.smithi152.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 49 kB/s | 6.5 kB 00:00 2024-09-11T22:40:31.593 INFO:teuthology.orchestra.run.smithi152.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.9 MB/s | 897 kB 00:00 2024-09-11T22:40:31.685 INFO:teuthology.orchestra.run.smithi152.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.5 MB/s | 145 kB 00:00 2024-09-11T22:40:31.740 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-09-11T22:40:31.760 INFO:teuthology.orchestra.run.smithi152.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-11T22:40:31.894 INFO:teuthology.orchestra.run.smithi152.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 2.4 MB 00:00 2024-09-11T22:40:31.927 INFO:teuthology.orchestra.run.smithi152.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 9.1 MB/s | 1.5 MB 00:00 2024-09-11T22:40:32.319 INFO:teuthology.orchestra.run.smithi152.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 4.7 MB 00:00 2024-09-11T22:40:32.528 INFO:teuthology.orchestra.run.smithi152.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 16 MB/s | 21 MB 00:01 2024-09-11T22:40:32.578 INFO:teuthology.orchestra.run.smithi152.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 506 kB/s | 25 kB 00:00 2024-09-11T22:40:32.935 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:40:32.935 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:40:32.947 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T22:40:32.948 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T22:40:32.949 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T22:40:32.950 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T22:40:32.951 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T22:40:32.953 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T22:40:32.954 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T22:40:33.003 INFO:teuthology.orchestra.run.smithi152.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 16 MB/s | 17 MB 00:01 2024-09-11T22:40:33.017 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-09-11T22:40:33.030 INFO:teuthology.orchestra.run.smithi204.stdout:====================================================================================== 2024-09-11T22:40:33.030 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-09-11T22:40:33.030 INFO:teuthology.orchestra.run.smithi204.stdout:====================================================================================== 2024-09-11T22:40:33.030 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-09-11T22:40:33.031 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T22:40:33.031 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T22:40:33.031 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T22:40:33.031 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T22:40:33.031 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T22:40:33.031 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T22:40:33.031 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T22:40:33.031 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T22:40:33.031 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T22:40:33.031 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T22:40:33.031 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T22:40:33.032 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T22:40:33.032 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T22:40:33.032 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T22:40:33.032 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T22:40:33.032 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T22:40:33.032 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T22:40:33.032 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T22:40:33.032 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T22:40:33.032 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T22:40:33.032 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T22:40:33.032 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T22:40:33.032 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T22:40:33.033 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T22:40:33.033 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T22:40:33.033 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T22:40:33.033 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T22:40:33.033 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T22:40:33.033 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T22:40:33.033 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T22:40:33.033 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T22:40:33.033 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T22:40:33.033 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T22:40:33.033 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T22:40:33.033 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T22:40:33.033 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T22:40:33.034 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T22:40:33.034 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-09-11T22:40:33.034 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T22:40:33.034 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T22:40:33.034 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T22:40:33.034 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T22:40:33.034 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T22:40:33.034 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T22:40:33.034 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T22:40:33.034 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T22:40:33.034 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-09-11T22:40:33.034 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T22:40:33.034 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T22:40:33.035 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T22:40:33.035 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T22:40:33.035 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T22:40:33.035 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T22:40:33.035 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T22:40:33.035 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T22:40:33.035 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T22:40:33.035 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T22:40:33.035 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T22:40:33.035 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T22:40:33.035 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T22:40:33.035 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T22:40:33.035 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T22:40:33.035 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T22:40:33.036 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T22:40:33.036 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T22:40:33.036 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T22:40:33.036 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T22:40:33.036 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T22:40:33.036 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T22:40:33.036 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T22:40:33.036 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T22:40:33.036 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T22:40:33.036 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T22:40:33.036 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T22:40:33.036 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T22:40:33.037 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T22:40:33.037 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T22:40:33.037 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T22:40:33.037 INFO:teuthology.orchestra.run.smithi204.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T22:40:33.037 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T22:40:33.037 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T22:40:33.037 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T22:40:33.037 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T22:40:33.037 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T22:40:33.037 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T22:40:33.037 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T22:40:33.037 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T22:40:33.037 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T22:40:33.038 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T22:40:33.038 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T22:40:33.038 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T22:40:33.038 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T22:40:33.038 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T22:40:33.038 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T22:40:33.038 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T22:40:33.038 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T22:40:33.038 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T22:40:33.038 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T22:40:33.038 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T22:40:33.038 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T22:40:33.039 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T22:40:33.039 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T22:40:33.039 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T22:40:33.039 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T22:40:33.039 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T22:40:33.039 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T22:40:33.039 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T22:40:33.039 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T22:40:33.039 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T22:40:33.039 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T22:40:33.039 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T22:40:33.040 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T22:40:33.040 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T22:40:33.040 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T22:40:33.040 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T22:40:33.040 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T22:40:33.040 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T22:40:33.040 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T22:40:33.040 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T22:40:33.040 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T22:40:33.040 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T22:40:33.040 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T22:40:33.040 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T22:40:33.040 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T22:40:33.041 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T22:40:33.041 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T22:40:33.041 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T22:40:33.041 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T22:40:33.041 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T22:40:33.041 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T22:40:33.041 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T22:40:33.041 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T22:40:33.041 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T22:40:33.041 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T22:40:33.041 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T22:40:33.041 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T22:40:33.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T22:40:33.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T22:40:33.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T22:40:33.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T22:40:33.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T22:40:33.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T22:40:33.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T22:40:33.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T22:40:33.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T22:40:33.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T22:40:33.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T22:40:33.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T22:40:33.042 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T22:40:33.042 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T22:40:33.043 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T22:40:33.043 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T22:40:33.044 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T22:40:33.044 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T22:40:33.044 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T22:40:33.044 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T22:40:33.045 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T22:40:33.045 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-09-11T22:40:33.045 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T22:40:33.045 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:40:33.045 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-09-11T22:40:33.045 INFO:teuthology.orchestra.run.smithi204.stdout:====================================================================================== 2024-09-11T22:40:33.045 INFO:teuthology.orchestra.run.smithi204.stdout:Install 148 Packages 2024-09-11T22:40:33.045 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 8 Packages 2024-09-11T22:40:33.045 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:40:33.046 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 226 M 2024-09-11T22:40:33.046 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-09-11T22:40:33.120 INFO:teuthology.orchestra.run.smithi152.stdout:(11/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 13 MB/s | 11 MB 00:00 2024-09-11T22:40:33.145 INFO:teuthology.orchestra.run.smithi152.stdout:(12/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.6 MB/s | 228 kB 00:00 2024-09-11T22:40:33.171 INFO:teuthology.orchestra.run.smithi152.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 540 kB/s | 27 kB 00:00 2024-09-11T22:40:33.196 INFO:teuthology.orchestra.run.smithi152.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 630 kB/s | 32 kB 00:00 2024-09-11T22:40:33.249 INFO:teuthology.orchestra.run.smithi152.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 3.2 MB/s | 163 kB 00:00 2024-09-11T22:40:33.280 INFO:teuthology.orchestra.run.smithi152.stdout:(16/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 6.8 MB/s | 752 kB 00:00 2024-09-11T22:40:33.305 INFO:teuthology.orchestra.run.smithi152.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 127 kB 00:00 2024-09-11T22:40:33.347 INFO:teuthology.orchestra.run.smithi152.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 7.4 MB/s | 506 kB 00:00 2024-09-11T22:40:33.405 INFO:teuthology.orchestra.run.smithi152.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 778 kB/s | 45 kB 00:00 2024-09-11T22:40:33.464 INFO:teuthology.orchestra.run.smithi152.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.3 MB/s | 136 kB 00:00 2024-09-11T22:40:33.531 INFO:teuthology.orchestra.run.smithi152.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.4 MB/s | 162 kB 00:00 2024-09-11T22:40:33.615 INFO:teuthology.orchestra.run.smithi152.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 321 kB 00:00 2024-09-11T22:40:33.698 INFO:teuthology.orchestra.run.smithi152.stdout:(23/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-11T22:40:33.701 INFO:teuthology.orchestra.run.smithi204.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 44 kB/s | 6.5 kB 00:00 2024-09-11T22:40:33.732 INFO:teuthology.orchestra.run.smithi152.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 2.5 MB/s | 301 kB 00:00 2024-09-11T22:40:33.757 INFO:teuthology.orchestra.run.smithi152.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.7 MB/s | 100 kB 00:00 2024-09-11T22:40:33.782 INFO:teuthology.orchestra.run.smithi152.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-11T22:40:33.900 INFO:teuthology.orchestra.run.smithi152.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 1.4 MB/s | 171 kB 00:00 2024-09-11T22:40:34.017 INFO:teuthology.orchestra.run.smithi152.stdout:(28/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-11T22:40:34.033 INFO:teuthology.orchestra.run.smithi191.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:40:34.034 INFO:teuthology.orchestra.run.smithi191.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:40:34.042 INFO:teuthology.orchestra.run.smithi152.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 225 kB/s | 32 kB 00:00 2024-09-11T22:40:34.046 INFO:teuthology.orchestra.run.smithi191.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T22:40:34.046 INFO:teuthology.orchestra.run.smithi191.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T22:40:34.047 INFO:teuthology.orchestra.run.smithi191.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T22:40:34.048 INFO:teuthology.orchestra.run.smithi191.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T22:40:34.050 INFO:teuthology.orchestra.run.smithi191.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T22:40:34.052 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T22:40:34.053 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T22:40:34.092 INFO:teuthology.orchestra.run.smithi152.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.9 MB/s | 146 kB 00:00 2024-09-11T22:40:34.117 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-09-11T22:40:34.126 INFO:teuthology.orchestra.run.smithi204.stdout:(2/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 9.4 MB/s | 5.4 MB 00:00 2024-09-11T22:40:34.134 INFO:teuthology.orchestra.run.smithi191.stdout:====================================================================================== 2024-09-11T22:40:34.134 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2024-09-11T22:40:34.134 INFO:teuthology.orchestra.run.smithi191.stdout:====================================================================================== 2024-09-11T22:40:34.134 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-09-11T22:40:34.134 INFO:teuthology.orchestra.run.smithi191.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T22:40:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T22:40:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T22:40:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T22:40:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T22:40:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T22:40:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T22:40:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T22:40:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T22:40:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T22:40:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T22:40:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T22:40:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T22:40:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T22:40:34.135 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T22:40:34.136 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T22:40:34.136 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T22:40:34.136 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T22:40:34.136 INFO:teuthology.orchestra.run.smithi191.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T22:40:34.136 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T22:40:34.136 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T22:40:34.136 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T22:40:34.136 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T22:40:34.136 INFO:teuthology.orchestra.run.smithi191.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T22:40:34.136 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T22:40:34.136 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T22:40:34.136 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T22:40:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T22:40:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T22:40:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T22:40:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T22:40:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T22:40:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T22:40:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T22:40:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T22:40:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T22:40:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T22:40:34.137 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrading: 2024-09-11T22:40:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T22:40:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T22:40:34.137 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T22:40:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T22:40:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T22:40:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T22:40:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T22:40:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T22:40:34.138 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-09-11T22:40:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T22:40:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T22:40:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T22:40:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T22:40:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T22:40:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T22:40:34.138 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T22:40:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T22:40:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T22:40:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T22:40:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T22:40:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T22:40:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T22:40:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T22:40:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T22:40:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T22:40:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T22:40:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T22:40:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T22:40:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T22:40:34.139 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T22:40:34.140 INFO:teuthology.orchestra.run.smithi191.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T22:40:34.140 INFO:teuthology.orchestra.run.smithi191.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T22:40:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T22:40:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T22:40:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T22:40:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T22:40:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T22:40:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T22:40:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T22:40:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T22:40:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T22:40:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T22:40:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T22:40:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T22:40:34.141 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T22:40:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T22:40:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T22:40:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T22:40:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T22:40:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T22:40:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T22:40:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T22:40:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T22:40:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T22:40:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T22:40:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T22:40:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T22:40:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T22:40:34.142 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T22:40:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T22:40:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T22:40:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T22:40:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T22:40:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T22:40:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T22:40:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T22:40:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T22:40:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T22:40:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T22:40:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T22:40:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T22:40:34.143 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T22:40:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T22:40:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T22:40:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T22:40:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T22:40:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T22:40:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T22:40:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T22:40:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T22:40:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T22:40:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T22:40:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T22:40:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T22:40:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T22:40:34.144 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T22:40:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T22:40:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T22:40:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T22:40:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T22:40:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T22:40:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T22:40:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T22:40:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T22:40:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T22:40:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T22:40:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T22:40:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T22:40:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T22:40:34.145 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T22:40:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T22:40:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T22:40:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T22:40:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T22:40:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T22:40:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T22:40:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T22:40:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T22:40:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T22:40:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T22:40:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T22:40:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T22:40:34.146 INFO:teuthology.orchestra.run.smithi191.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T22:40:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T22:40:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T22:40:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T22:40:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T22:40:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T22:40:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T22:40:34.147 INFO:teuthology.orchestra.run.smithi191.stdout:Installing weak dependencies: 2024-09-11T22:40:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T22:40:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:40:34.147 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-09-11T22:40:34.147 INFO:teuthology.orchestra.run.smithi191.stdout:====================================================================================== 2024-09-11T22:40:34.147 INFO:teuthology.orchestra.run.smithi191.stdout:Install 148 Packages 2024-09-11T22:40:34.147 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrade 8 Packages 2024-09-11T22:40:34.147 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:40:34.148 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 226 M 2024-09-11T22:40:34.148 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-09-11T22:40:34.270 INFO:teuthology.orchestra.run.smithi204.stdout:(3/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.5 MB/s | 897 kB 00:00 2024-09-11T22:40:34.310 INFO:teuthology.orchestra.run.smithi204.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 790 kB/s | 145 kB 00:00 2024-09-11T22:40:34.326 INFO:teuthology.orchestra.run.smithi152.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 13 MB/s | 3.6 MB 00:00 2024-09-11T22:40:34.434 INFO:teuthology.orchestra.run.smithi152.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.2 MB/s | 249 kB 00:00 2024-09-11T22:40:34.486 INFO:teuthology.orchestra.run.smithi204.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 8.7 MB/s | 1.5 MB 00:00 2024-09-11T22:40:34.526 INFO:teuthology.orchestra.run.smithi152.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 539 kB/s | 49 kB 00:00 2024-09-11T22:40:34.635 INFO:teuthology.orchestra.run.smithi152.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 152 kB/s | 16 kB 00:00 2024-09-11T22:40:34.827 INFO:teuthology.orchestra.run.smithi152.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 10 MB/s | 7.4 MB 00:00 2024-09-11T22:40:34.844 INFO:teuthology.orchestra.run.smithi204.stdout:(6/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-11T22:40:34.952 INFO:teuthology.orchestra.run.smithi152.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 6.0 MB/s | 765 kB 00:00 2024-09-11T22:40:34.958 INFO:teuthology.orchestra.run.smithi191.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 33 kB/s | 6.5 kB 00:00 2024-09-11T22:40:35.011 INFO:teuthology.orchestra.run.smithi204.stdout:(7/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 3.2 MB/s | 2.4 MB 00:00 2024-09-11T22:40:35.019 INFO:teuthology.orchestra.run.smithi152.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 274 kB/s | 18 kB 00:00 2024-09-11T22:40:35.350 INFO:teuthology.orchestra.run.smithi191.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.2 MB/s | 897 kB 00:00 2024-09-11T22:40:35.416 INFO:teuthology.orchestra.run.smithi191.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 2.1 MB/s | 145 kB 00:00 2024-09-11T22:40:35.445 INFO:teuthology.orchestra.run.smithi204.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 21 MB 00:01 2024-09-11T22:40:35.469 INFO:teuthology.orchestra.run.smithi152.stdout:(38/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 17 MB/s | 49 MB 00:02 2024-09-11T22:40:35.495 INFO:teuthology.orchestra.run.smithi152.stdout:(39/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 332 kB/s | 285 kB 00:00 2024-09-11T22:40:35.495 INFO:teuthology.orchestra.run.smithi204.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 505 kB/s | 25 kB 00:00 2024-09-11T22:40:36.000 INFO:teuthology.orchestra.run.smithi191.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 4.1 MB/s | 2.4 MB 00:00 2024-09-11T22:40:36.028 INFO:teuthology.orchestra.run.smithi204.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 10 MB/s | 11 MB 00:01 2024-09-11T22:40:36.095 INFO:teuthology.orchestra.run.smithi152.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 74 kB/s | 80 kB 00:01 2024-09-11T22:40:36.142 INFO:teuthology.orchestra.run.smithi191.stdout:(5/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 3.9 MB/s | 5.4 MB 00:01 2024-09-11T22:40:36.212 INFO:teuthology.orchestra.run.smithi204.stdout:(11/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 12 MB/s | 17 MB 00:01 2024-09-11T22:40:36.217 INFO:teuthology.orchestra.run.smithi191.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 7.0 MB/s | 1.5 MB 00:00 2024-09-11T22:40:36.228 INFO:teuthology.orchestra.run.smithi152.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 260 kB/s | 191 kB 00:00 2024-09-11T22:40:36.237 INFO:teuthology.orchestra.run.smithi204.stdout:(12/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.1 MB/s | 228 kB 00:00 2024-09-11T22:40:36.262 INFO:teuthology.orchestra.run.smithi204.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 540 kB/s | 27 kB 00:00 2024-09-11T22:40:36.288 INFO:teuthology.orchestra.run.smithi204.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 631 kB/s | 32 kB 00:00 2024-09-11T22:40:36.347 INFO:teuthology.orchestra.run.smithi204.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 8.8 MB/s | 752 kB 00:00 2024-09-11T22:40:36.451 INFO:teuthology.orchestra.run.smithi152.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 898 kB/s | 801 kB 00:00 2024-09-11T22:40:36.452 INFO:teuthology.orchestra.run.smithi204.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.9 MB/s | 163 kB 00:00 2024-09-11T22:40:36.452 INFO:teuthology.orchestra.run.smithi204.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 127 kB 00:00 2024-09-11T22:40:36.457 INFO:teuthology.orchestra.run.smithi204.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.9 MB/s | 506 kB 00:00 2024-09-11T22:40:36.541 INFO:teuthology.orchestra.run.smithi204.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 542 kB/s | 45 kB 00:00 2024-09-11T22:40:36.614 INFO:teuthology.orchestra.run.smithi191.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 10 MB/s | 4.7 MB 00:00 2024-09-11T22:40:36.617 INFO:teuthology.orchestra.run.smithi204.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.8 MB/s | 136 kB 00:00 2024-09-11T22:40:36.699 INFO:teuthology.orchestra.run.smithi204.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 162 kB 00:00 2024-09-11T22:40:36.805 INFO:teuthology.orchestra.run.smithi204.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.1 MB/s | 321 kB 00:00 2024-09-11T22:40:36.875 INFO:teuthology.orchestra.run.smithi152.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 330 kB/s | 253 kB 00:00 2024-09-11T22:40:36.972 INFO:teuthology.orchestra.run.smithi204.stdout:(23/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 10 MB/s | 5.3 MB 00:00 2024-09-11T22:40:36.995 INFO:teuthology.orchestra.run.smithi204.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 1.6 MB/s | 301 kB 00:00 2024-09-11T22:40:37.048 INFO:teuthology.orchestra.run.smithi204.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.2 MB/s | 100 kB 00:00 2024-09-11T22:40:37.078 INFO:teuthology.orchestra.run.smithi204.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.0 MB/s | 85 kB 00:00 2024-09-11T22:40:37.141 INFO:teuthology.orchestra.run.smithi152.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 142 kB/s | 106 kB 00:00 2024-09-11T22:40:37.194 INFO:teuthology.orchestra.run.smithi204.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 1.7 MB/s | 171 kB 00:00 2024-09-11T22:40:37.200 INFO:teuthology.orchestra.run.smithi152.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-11T22:40:37.263 INFO:teuthology.orchestra.run.smithi204.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 346 kB/s | 32 kB 00:00 2024-09-11T22:40:37.368 INFO:teuthology.orchestra.run.smithi204.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.3 MB/s | 146 kB 00:00 2024-09-11T22:40:37.443 INFO:teuthology.orchestra.run.smithi204.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 7.6 MB/s | 3.1 MB 00:00 2024-09-11T22:40:37.646 INFO:teuthology.orchestra.run.smithi152.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 173 kB/s | 135 kB 00:00 2024-09-11T22:40:37.676 INFO:teuthology.orchestra.run.smithi204.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 12 MB/s | 3.6 MB 00:00 2024-09-11T22:40:37.685 INFO:teuthology.orchestra.run.smithi191.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 7.3 MB/s | 21 MB 00:02 2024-09-11T22:40:37.735 INFO:teuthology.orchestra.run.smithi204.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.2 MB/s | 249 kB 00:00 2024-09-11T22:40:37.793 INFO:teuthology.orchestra.run.smithi204.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 845 kB/s | 49 kB 00:00 2024-09-11T22:40:37.821 INFO:teuthology.orchestra.run.smithi152.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 178 kB/s | 125 kB 00:00 2024-09-11T22:40:37.826 INFO:teuthology.orchestra.run.smithi191.stdout:(9/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9. 8.7 MB/s | 11 MB 00:01 2024-09-11T22:40:37.851 INFO:teuthology.orchestra.run.smithi191.stdout:(10/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9 151 kB/s | 25 kB 00:00 2024-09-11T22:40:37.919 INFO:teuthology.orchestra.run.smithi204.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 16 MB/s | 7.4 MB 00:00 2024-09-11T22:40:37.930 INFO:teuthology.orchestra.run.smithi152.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 283 kB/s | 215 kB 00:00 2024-09-11T22:40:37.944 INFO:teuthology.orchestra.run.smithi204.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 109 kB/s | 16 kB 00:00 2024-09-11T22:40:37.969 INFO:teuthology.orchestra.run.smithi204.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.6 MB/s | 285 kB 00:00 2024-09-11T22:40:38.003 INFO:teuthology.orchestra.run.smithi204.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 13 MB/s | 765 kB 00:00 2024-09-11T22:40:38.028 INFO:teuthology.orchestra.run.smithi204.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 312 kB/s | 18 kB 00:00 2024-09-11T22:40:38.035 INFO:teuthology.orchestra.run.smithi191.stdout:(11/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 9.3 MB/s | 17 MB 00:01 2024-09-11T22:40:38.069 INFO:teuthology.orchestra.run.smithi191.stdout:(12/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.0 MB/s | 228 kB 00:00 2024-09-11T22:40:38.094 INFO:teuthology.orchestra.run.smithi191.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 462 kB/s | 27 kB 00:00 2024-09-11T22:40:38.120 INFO:teuthology.orchestra.run.smithi191.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 630 kB/s | 32 kB 00:00 2024-09-11T22:40:38.170 INFO:teuthology.orchestra.run.smithi191.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 9.8 MB/s | 752 kB 00:00 2024-09-11T22:40:38.195 INFO:teuthology.orchestra.run.smithi191.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.1 MB/s | 163 kB 00:00 2024-09-11T22:40:38.220 INFO:teuthology.orchestra.run.smithi191.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.5 MB/s | 127 kB 00:00 2024-09-11T22:40:38.279 INFO:teuthology.orchestra.run.smithi191.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.9 MB/s | 506 kB 00:00 2024-09-11T22:40:38.337 INFO:teuthology.orchestra.run.smithi191.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 778 kB/s | 45 kB 00:00 2024-09-11T22:40:38.403 INFO:teuthology.orchestra.run.smithi204.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 199 kB/s | 80 kB 00:00 2024-09-11T22:40:38.404 INFO:teuthology.orchestra.run.smithi191.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.0 MB/s | 136 kB 00:00 2024-09-11T22:40:38.471 INFO:teuthology.orchestra.run.smithi191.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.4 MB/s | 162 kB 00:00 2024-09-11T22:40:38.538 INFO:teuthology.orchestra.run.smithi191.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 4.7 MB/s | 321 kB 00:00 2024-09-11T22:40:38.555 INFO:teuthology.orchestra.run.smithi152.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 759 kB/s | 689 kB 00:00 2024-09-11T22:40:38.622 INFO:teuthology.orchestra.run.smithi191.stdout:(23/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 5.3 MB 00:00 2024-09-11T22:40:38.647 INFO:teuthology.orchestra.run.smithi152.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 149 kB/s | 107 kB 00:00 2024-09-11T22:40:38.648 INFO:teuthology.orchestra.run.smithi191.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 2.7 MB/s | 301 kB 00:00 2024-09-11T22:40:38.673 INFO:teuthology.orchestra.run.smithi191.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-11T22:40:38.698 INFO:teuthology.orchestra.run.smithi191.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-11T22:40:38.748 INFO:teuthology.orchestra.run.smithi191.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-11T22:40:38.798 INFO:teuthology.orchestra.run.smithi191.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 640 kB/s | 32 kB 00:00 2024-09-11T22:40:38.864 INFO:teuthology.orchestra.run.smithi152.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 905 kB/s | 943 kB 00:01 2024-09-11T22:40:38.887 INFO:teuthology.orchestra.run.smithi204.stdout:(40/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 14 MB/s | 49 MB 00:03 2024-09-11T22:40:38.891 INFO:teuthology.orchestra.run.smithi191.stdout:(29/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 14 MB/s | 3.1 MB 00:00 2024-09-11T22:40:38.912 INFO:teuthology.orchestra.run.smithi204.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 376 kB/s | 191 kB 00:00 2024-09-11T22:40:38.916 INFO:teuthology.orchestra.run.smithi191.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.2 MB/s | 146 kB 00:00 2024-09-11T22:40:39.114 INFO:teuthology.orchestra.run.smithi152.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 39 kB/s | 22 kB 00:00 2024-09-11T22:40:39.133 INFO:teuthology.orchestra.run.smithi191.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 15 MB/s | 3.6 MB 00:00 2024-09-11T22:40:39.188 INFO:teuthology.orchestra.run.smithi204.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 691 kB/s | 801 kB 00:01 2024-09-11T22:40:39.192 INFO:teuthology.orchestra.run.smithi191.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.2 MB/s | 249 kB 00:00 2024-09-11T22:40:39.251 INFO:teuthology.orchestra.run.smithi191.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 848 kB/s | 49 kB 00:00 2024-09-11T22:40:39.296 INFO:teuthology.orchestra.run.smithi204.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 982 kB/s | 106 kB 00:00 2024-09-11T22:40:39.384 INFO:teuthology.orchestra.run.smithi191.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 16 MB/s | 7.4 MB 00:00 2024-09-11T22:40:39.388 INFO:teuthology.orchestra.run.smithi204.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 2.6 MB/s | 1.2 MB 00:00 2024-09-11T22:40:39.409 INFO:teuthology.orchestra.run.smithi191.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 104 kB/s | 16 kB 00:00 2024-09-11T22:40:39.413 INFO:teuthology.orchestra.run.smithi204.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-11T22:40:39.435 INFO:teuthology.orchestra.run.smithi191.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.6 MB/s | 285 kB 00:00 2024-09-11T22:40:39.485 INFO:teuthology.orchestra.run.smithi191.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 9.9 MB/s | 765 kB 00:00 2024-09-11T22:40:39.497 INFO:teuthology.orchestra.run.smithi204.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-11T22:40:39.497 INFO:teuthology.orchestra.run.smithi152.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 52 kB/s | 33 kB 00:00 2024-09-11T22:40:39.510 INFO:teuthology.orchestra.run.smithi191.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 243 kB/s | 18 kB 00:00 2024-09-11T22:40:39.531 INFO:teuthology.orchestra.run.smithi204.stdout:(47/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 394 kB/s | 253 kB 00:00 2024-09-11T22:40:39.531 INFO:teuthology.orchestra.run.smithi152.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 355 kB/s | 313 kB 00:00 2024-09-11T22:40:39.581 INFO:teuthology.orchestra.run.smithi204.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.3 MB/s | 215 kB 00:00 2024-09-11T22:40:39.627 INFO:teuthology.orchestra.run.smithi191.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 562 kB/s | 80 kB 00:00 2024-09-11T22:40:39.665 INFO:teuthology.orchestra.run.smithi204.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-09-11T22:40:39.698 INFO:teuthology.orchestra.run.smithi204.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 915 kB/s | 107 kB 00:00 2024-09-11T22:40:39.702 INFO:teuthology.orchestra.run.smithi191.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-11T22:40:39.732 INFO:teuthology.orchestra.run.smithi204.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 4.6 MB/s | 943 kB 00:00 2024-09-11T22:40:39.736 INFO:teuthology.orchestra.run.smithi191.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-11T22:40:39.757 INFO:teuthology.orchestra.run.smithi204.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 236 kB/s | 22 kB 00:00 2024-09-11T22:40:39.761 INFO:teuthology.orchestra.run.smithi191.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-11T22:40:39.792 INFO:teuthology.orchestra.run.smithi204.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-09-11T22:40:39.812 INFO:teuthology.orchestra.run.smithi191.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-09-11T22:40:39.816 INFO:teuthology.orchestra.run.smithi204.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 391 kB/s | 33 kB 00:00 2024-09-11T22:40:39.846 INFO:teuthology.orchestra.run.smithi191.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-11T22:40:39.879 INFO:teuthology.orchestra.run.smithi191.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-11T22:40:39.883 INFO:teuthology.orchestra.run.smithi204.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-09-11T22:40:39.905 INFO:teuthology.orchestra.run.smithi191.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-11T22:40:39.908 INFO:teuthology.orchestra.run.smithi204.stdout:(56/156): flexiblas-openblas-openmp-3.0.4-8.el9 155 kB/s | 18 kB 00:00 2024-09-11T22:40:39.938 INFO:teuthology.orchestra.run.smithi191.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-11T22:40:39.942 INFO:teuthology.orchestra.run.smithi204.stdout:(57/156): libaio-devel-0.3.111-13.el9.x86_64.rp 88 kB/s | 11 kB 00:00 2024-09-11T22:40:39.978 INFO:teuthology.orchestra.run.smithi204.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-11T22:40:39.995 INFO:teuthology.orchestra.run.smithi204.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 521 kB/s | 45 kB 00:00 2024-09-11T22:40:40.023 INFO:teuthology.orchestra.run.smithi152.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 18 kB 00:00 2024-09-11T22:40:40.037 INFO:teuthology.orchestra.run.smithi204.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.8 MB/s | 662 kB 00:00 2024-09-11T22:40:40.065 INFO:teuthology.orchestra.run.smithi152.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 21 kB/s | 11 kB 00:00 2024-09-11T22:40:40.071 INFO:teuthology.orchestra.run.smithi204.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 6.2 MB/s | 581 kB 00:00 2024-09-11T22:40:40.088 INFO:teuthology.orchestra.run.smithi204.stdout:(62/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 265 kB/s | 24 kB 00:00 2024-09-11T22:40:40.105 INFO:teuthology.orchestra.run.smithi204.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 706 kB/s | 47 kB 00:00 2024-09-11T22:40:40.147 INFO:teuthology.orchestra.run.smithi204.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-11T22:40:40.197 INFO:teuthology.orchestra.run.smithi204.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-11T22:40:40.222 INFO:teuthology.orchestra.run.smithi204.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 328 kB/s | 38 kB 00:00 2024-09-11T22:40:40.306 INFO:teuthology.orchestra.run.smithi204.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 31 MB/s | 4.9 MB 00:00 2024-09-11T22:40:40.316 INFO:teuthology.orchestra.run.smithi152.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.5 MB/s | 3.0 MB 00:01 2024-09-11T22:40:40.348 INFO:teuthology.orchestra.run.smithi204.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-11T22:40:40.382 INFO:teuthology.orchestra.run.smithi204.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-11T22:40:40.447 INFO:teuthology.orchestra.run.smithi191.stdout:(48/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 19 MB/s | 49 MB 00:02 2024-09-11T22:40:40.448 INFO:teuthology.orchestra.run.smithi204.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-09-11T22:40:40.465 INFO:teuthology.orchestra.run.smithi204.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-11T22:40:40.530 INFO:teuthology.orchestra.run.smithi191.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 689 kB 00:00 2024-09-11T22:40:40.557 INFO:teuthology.orchestra.run.smithi204.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-09-11T22:40:40.573 INFO:teuthology.orchestra.run.smithi191.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 1.5 MB/s | 943 kB 00:00 2024-09-11T22:40:40.582 INFO:teuthology.orchestra.run.smithi204.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 748 kB/s | 87 kB 00:00 2024-09-11T22:40:40.598 INFO:teuthology.orchestra.run.smithi191.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 711 kB/s | 107 kB 00:00 2024-09-11T22:40:40.616 INFO:teuthology.orchestra.run.smithi204.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.0 MB/s | 1.0 MB 00:00 2024-09-11T22:40:40.623 INFO:teuthology.orchestra.run.smithi191.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 235 kB/s | 22 kB 00:00 2024-09-11T22:40:40.640 INFO:teuthology.orchestra.run.smithi191.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 783 kB/s | 33 kB 00:00 2024-09-11T22:40:40.658 INFO:teuthology.orchestra.run.smithi204.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 814 kB/s | 61 kB 00:00 2024-09-11T22:40:40.665 INFO:teuthology.orchestra.run.smithi191.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-09-11T22:40:40.691 INFO:teuthology.orchestra.run.smithi191.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 361 kB/s | 18 kB 00:00 2024-09-11T22:40:40.699 INFO:teuthology.orchestra.run.smithi152.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-09-11T22:40:40.717 INFO:teuthology.orchestra.run.smithi191.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-11T22:40:40.741 INFO:teuthology.orchestra.run.smithi191.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-11T22:40:40.742 INFO:teuthology.orchestra.run.smithi204.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 6.0 MB 00:00 2024-09-11T22:40:40.766 INFO:teuthology.orchestra.run.smithi191.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-09-11T22:40:40.767 INFO:teuthology.orchestra.run.smithi204.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.6 MB/s | 245 kB 00:00 2024-09-11T22:40:40.793 INFO:teuthology.orchestra.run.smithi204.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-09-11T22:40:40.818 INFO:teuthology.orchestra.run.smithi204.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 340 kB/s | 26 kB 00:00 2024-09-11T22:40:40.868 INFO:teuthology.orchestra.run.smithi204.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-11T22:40:40.874 INFO:teuthology.orchestra.run.smithi152.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 188 kB/s | 160 kB 00:00 2024-09-11T22:40:40.883 INFO:teuthology.orchestra.run.smithi191.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-09-11T22:40:40.893 INFO:teuthology.orchestra.run.smithi204.stdout:(81/156): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-09-11T22:40:40.925 INFO:teuthology.orchestra.run.smithi191.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 591 kB/s | 24 kB 00:00 2024-09-11T22:40:40.927 INFO:teuthology.orchestra.run.smithi204.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 318 kB/s | 35 kB 00:00 2024-09-11T22:40:40.959 INFO:teuthology.orchestra.run.smithi191.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 3.0 MB/s | 581 kB 00:00 2024-09-11T22:40:40.960 INFO:teuthology.orchestra.run.smithi204.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.6 MB/s | 250 kB 00:00 2024-09-11T22:40:40.984 INFO:teuthology.orchestra.run.smithi191.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-09-11T22:40:41.003 INFO:teuthology.orchestra.run.smithi204.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-09-11T22:40:41.028 INFO:teuthology.orchestra.run.smithi204.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-11T22:40:41.051 INFO:teuthology.orchestra.run.smithi191.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-11T22:40:41.076 INFO:teuthology.orchestra.run.smithi191.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-09-11T22:40:41.101 INFO:teuthology.orchestra.run.smithi191.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-11T22:40:41.103 INFO:teuthology.orchestra.run.smithi204.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 5.1 MB 00:00 2024-09-11T22:40:41.128 INFO:teuthology.orchestra.run.smithi204.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 337 kB/s | 42 kB 00:00 2024-09-11T22:40:41.152 INFO:teuthology.orchestra.run.smithi191.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.7 MB/s | 3.0 MB 00:00 2024-09-11T22:40:41.153 INFO:teuthology.orchestra.run.smithi204.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-09-11T22:40:41.179 INFO:teuthology.orchestra.run.smithi204.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-11T22:40:41.200 INFO:teuthology.orchestra.run.smithi152.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 763 kB/s | 662 kB 00:00 2024-09-11T22:40:41.204 INFO:teuthology.orchestra.run.smithi204.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-11T22:40:41.229 INFO:teuthology.orchestra.run.smithi204.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-09-11T22:40:41.244 INFO:teuthology.orchestra.run.smithi191.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-11T22:40:41.254 INFO:teuthology.orchestra.run.smithi204.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-09-11T22:40:41.313 INFO:teuthology.orchestra.run.smithi204.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 498 kB/s | 42 kB 00:00 2024-09-11T22:40:41.319 INFO:teuthology.orchestra.run.smithi191.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 20 MB/s | 4.9 MB 00:00 2024-09-11T22:40:41.339 INFO:teuthology.orchestra.run.smithi204.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 528 kB/s | 44 kB 00:00 2024-09-11T22:40:41.352 INFO:teuthology.orchestra.run.smithi191.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 8.2 MB/s | 910 kB 00:00 2024-09-11T22:40:41.377 INFO:teuthology.orchestra.run.smithi191.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-11T22:40:41.380 INFO:teuthology.orchestra.run.smithi204.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-11T22:40:41.406 INFO:teuthology.orchestra.run.smithi204.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 537 kB/s | 36 kB 00:00 2024-09-11T22:40:41.448 INFO:teuthology.orchestra.run.smithi204.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-09-11T22:40:41.450 INFO:teuthology.orchestra.run.smithi152.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 43 kB/s | 24 kB 00:00 2024-09-11T22:40:41.453 INFO:teuthology.orchestra.run.smithi191.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-11T22:40:41.481 INFO:teuthology.orchestra.run.smithi204.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 4.0 MB/s | 304 kB 00:00 2024-09-11T22:40:41.495 INFO:teuthology.orchestra.run.smithi191.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-09-11T22:40:41.515 INFO:teuthology.orchestra.run.smithi204.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 863 kB/s | 58 kB 00:00 2024-09-11T22:40:41.529 INFO:teuthology.orchestra.run.smithi191.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-11T22:40:41.562 INFO:teuthology.orchestra.run.smithi191.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-11T22:40:41.600 INFO:teuthology.orchestra.run.smithi152.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 645 kB/s | 581 kB 00:00 2024-09-11T22:40:41.663 INFO:teuthology.orchestra.run.smithi191.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-09-11T22:40:41.682 INFO:teuthology.orchestra.run.smithi204.stdout:(100/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-11T22:40:41.688 INFO:teuthology.orchestra.run.smithi191.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-09-11T22:40:41.699 INFO:teuthology.orchestra.run.smithi204.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 347 kB/s | 64 kB 00:00 2024-09-11T22:40:41.713 INFO:teuthology.orchestra.run.smithi191.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-11T22:40:41.724 INFO:teuthology.orchestra.run.smithi204.stdout:(102/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 395 kB/s | 96 kB 00:00 2024-09-11T22:40:41.739 INFO:teuthology.orchestra.run.smithi191.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-11T22:40:41.758 INFO:teuthology.orchestra.run.smithi204.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-09-11T22:40:41.764 INFO:teuthology.orchestra.run.smithi191.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-11T22:40:41.789 INFO:teuthology.orchestra.run.smithi191.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-11T22:40:41.808 INFO:teuthology.orchestra.run.smithi204.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-11T22:40:41.814 INFO:teuthology.orchestra.run.smithi191.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-11T22:40:41.844 INFO:teuthology.orchestra.run.smithi204.stdout:(105/156): lua-devel-5.4.4-4.el9.x86_64.rpm 154 kB/s | 22 kB 00:00 2024-09-11T22:40:41.858 INFO:teuthology.orchestra.run.smithi152.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 72 kB/s | 47 kB 00:00 2024-09-11T22:40:41.875 INFO:teuthology.orchestra.run.smithi204.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 4.5 MB/s | 309 kB 00:00 2024-09-11T22:40:41.901 INFO:teuthology.orchestra.run.smithi204.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 346 kB/s | 19 kB 00:00 2024-09-11T22:40:41.926 INFO:teuthology.orchestra.run.smithi204.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-11T22:40:41.940 INFO:teuthology.orchestra.run.smithi191.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-11T22:40:41.951 INFO:teuthology.orchestra.run.smithi204.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-11T22:40:41.965 INFO:teuthology.orchestra.run.smithi191.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-09-11T22:40:41.976 INFO:teuthology.orchestra.run.smithi204.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-11T22:40:41.990 INFO:teuthology.orchestra.run.smithi191.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-11T22:40:42.010 INFO:teuthology.orchestra.run.smithi204.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-11T22:40:42.024 INFO:teuthology.orchestra.run.smithi191.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-11T22:40:42.049 INFO:teuthology.orchestra.run.smithi191.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 723 kB/s | 42 kB 00:00 2024-09-11T22:40:42.060 INFO:teuthology.orchestra.run.smithi204.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-11T22:40:42.074 INFO:teuthology.orchestra.run.smithi191.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-11T22:40:42.094 INFO:teuthology.orchestra.run.smithi204.stdout:(113/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-09-11T22:40:42.099 INFO:teuthology.orchestra.run.smithi191.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-11T22:40:42.125 INFO:teuthology.orchestra.run.smithi191.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-11T22:40:42.136 INFO:teuthology.orchestra.run.smithi204.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-09-11T22:40:42.150 INFO:teuthology.orchestra.run.smithi191.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-11T22:40:42.153 INFO:teuthology.orchestra.run.smithi204.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 2.2 MB/s | 32 kB 00:00 2024-09-11T22:40:42.170 INFO:teuthology.orchestra.run.smithi204.stdout:(116/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 575 kB/s | 43 kB 00:00 2024-09-11T22:40:42.175 INFO:teuthology.orchestra.run.smithi191.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-11T22:40:42.187 INFO:teuthology.orchestra.run.smithi204.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 422 kB/s | 14 kB 00:00 2024-09-11T22:40:42.220 INFO:teuthology.orchestra.run.smithi204.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-11T22:40:42.254 INFO:teuthology.orchestra.run.smithi204.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-11T22:40:42.259 INFO:teuthology.orchestra.run.smithi191.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-11T22:40:42.284 INFO:teuthology.orchestra.run.smithi191.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-09-11T22:40:42.288 INFO:teuthology.orchestra.run.smithi204.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 6.5 MB/s | 220 kB 00:00 2024-09-11T22:40:42.309 INFO:teuthology.orchestra.run.smithi191.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-09-11T22:40:42.321 INFO:teuthology.orchestra.run.smithi204.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-09-11T22:40:42.325 INFO:teuthology.orchestra.run.smithi152.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 337 kB/s | 292 kB 00:00 2024-09-11T22:40:42.343 INFO:teuthology.orchestra.run.smithi191.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-11T22:40:42.347 INFO:teuthology.orchestra.run.smithi204.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-11T22:40:42.368 INFO:teuthology.orchestra.run.smithi191.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-09-11T22:40:42.372 INFO:teuthology.orchestra.run.smithi204.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-11T22:40:42.393 INFO:teuthology.orchestra.run.smithi191.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-11T22:40:42.397 INFO:teuthology.orchestra.run.smithi204.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 705 kB/s | 18 kB 00:00 2024-09-11T22:40:42.419 INFO:teuthology.orchestra.run.smithi191.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-11T22:40:42.422 INFO:teuthology.orchestra.run.smithi204.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 933 kB/s | 23 kB 00:00 2024-09-11T22:40:42.434 INFO:teuthology.orchestra.run.smithi152.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 226 kB/s | 188 kB 00:00 2024-09-11T22:40:42.444 INFO:teuthology.orchestra.run.smithi191.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-11T22:40:42.448 INFO:teuthology.orchestra.run.smithi204.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-11T22:40:42.459 INFO:teuthology.orchestra.run.smithi152.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 65 kB/s | 38 kB 00:00 2024-09-11T22:40:42.469 INFO:teuthology.orchestra.run.smithi191.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-11T22:40:42.473 INFO:teuthology.orchestra.run.smithi204.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 791 kB/s | 20 kB 00:00 2024-09-11T22:40:42.495 INFO:teuthology.orchestra.run.smithi191.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-11T22:40:42.520 INFO:teuthology.orchestra.run.smithi191.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-09-11T22:40:42.541 INFO:teuthology.orchestra.run.smithi204.stdout:(128/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.7 MB/s | 4.4 MB 00:00 2024-09-11T22:40:42.565 INFO:teuthology.orchestra.run.smithi204.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T22:40:42.591 INFO:teuthology.orchestra.run.smithi204.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T22:40:42.616 INFO:teuthology.orchestra.run.smithi204.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-11T22:40:42.621 INFO:teuthology.orchestra.run.smithi191.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-09-11T22:40:42.650 INFO:teuthology.orchestra.run.smithi204.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-11T22:40:42.676 INFO:teuthology.orchestra.run.smithi204.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-11T22:40:42.709 INFO:teuthology.orchestra.run.smithi204.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 4.3 MB/s | 1.0 MB 00:00 2024-09-11T22:40:42.734 INFO:teuthology.orchestra.run.smithi204.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-11T22:40:42.759 INFO:teuthology.orchestra.run.smithi204.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-11T22:40:42.784 INFO:teuthology.orchestra.run.smithi204.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-11T22:40:42.796 INFO:teuthology.orchestra.run.smithi191.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-11T22:40:42.829 INFO:teuthology.orchestra.run.smithi191.stdout:(105/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-09-11T22:40:42.835 INFO:teuthology.orchestra.run.smithi204.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.7 MB/s | 2.0 MB 00:00 2024-09-11T22:40:42.860 INFO:teuthology.orchestra.run.smithi204.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 786 kB/s | 59 kB 00:00 2024-09-11T22:40:42.880 INFO:teuthology.orchestra.run.smithi191.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-09-11T22:40:42.894 INFO:teuthology.orchestra.run.smithi204.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-09-11T22:40:42.905 INFO:teuthology.orchestra.run.smithi191.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 285 kB/s | 19 kB 00:00 2024-09-11T22:40:42.919 INFO:teuthology.orchestra.run.smithi204.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-11T22:40:42.930 INFO:teuthology.orchestra.run.smithi191.stdout:(108/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.3 MB/s | 309 kB 00:00 2024-09-11T22:40:42.945 INFO:teuthology.orchestra.run.smithi204.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 887 kB/s | 74 kB 00:00 2024-09-11T22:40:42.955 INFO:teuthology.orchestra.run.smithi191.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 518 kB/s | 26 kB 00:00 2024-09-11T22:40:42.970 INFO:teuthology.orchestra.run.smithi204.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-11T22:40:42.981 INFO:teuthology.orchestra.run.smithi191.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 967 kB/s | 49 kB 00:00 2024-09-11T22:40:42.995 INFO:teuthology.orchestra.run.smithi204.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-11T22:40:43.006 INFO:teuthology.orchestra.run.smithi191.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-11T22:40:43.020 INFO:teuthology.orchestra.run.smithi204.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-11T22:40:43.031 INFO:teuthology.orchestra.run.smithi191.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-11T22:40:43.046 INFO:teuthology.orchestra.run.smithi204.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-11T22:40:43.071 INFO:teuthology.orchestra.run.smithi204.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-11T22:40:43.073 INFO:teuthology.orchestra.run.smithi191.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-11T22:40:43.099 INFO:teuthology.orchestra.run.smithi191.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-11T22:40:43.129 INFO:teuthology.orchestra.run.smithi204.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-11T22:40:43.140 INFO:teuthology.orchestra.run.smithi191.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-09-11T22:40:43.166 INFO:teuthology.orchestra.run.smithi191.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 481 kB/s | 32 kB 00:00 2024-09-11T22:40:43.191 INFO:teuthology.orchestra.run.smithi191.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-11T22:40:43.217 INFO:teuthology.orchestra.run.smithi191.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-11T22:40:43.250 INFO:teuthology.orchestra.run.smithi191.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-11T22:40:43.276 INFO:teuthology.orchestra.run.smithi191.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-11T22:40:43.309 INFO:teuthology.orchestra.run.smithi191.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-11T22:40:43.322 INFO:teuthology.orchestra.run.smithi204.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-11T22:40:43.351 INFO:teuthology.orchestra.run.smithi191.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-09-11T22:40:43.368 INFO:teuthology.orchestra.run.smithi191.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 639 kB/s | 11 kB 00:00 2024-09-11T22:40:43.385 INFO:teuthology.orchestra.run.smithi191.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-11T22:40:43.402 INFO:teuthology.orchestra.run.smithi191.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-11T22:40:43.427 INFO:teuthology.orchestra.run.smithi191.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 778 kB/s | 19 kB 00:00 2024-09-11T22:40:43.434 INFO:teuthology.orchestra.run.smithi152.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:00 2024-09-11T22:40:43.463 INFO:teuthology.orchestra.run.smithi204.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 8.1 MB/s | 3.2 MB 00:00 2024-09-11T22:40:43.502 INFO:teuthology.orchestra.run.smithi152.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.2 MB/s | 4.9 MB 00:01 2024-09-11T22:40:43.519 INFO:teuthology.orchestra.run.smithi191.stdout:(127/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.9 MB/s | 4.4 MB 00:00 2024-09-11T22:40:43.544 INFO:teuthology.orchestra.run.smithi191.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 168 kB/s | 20 kB 00:00 2024-09-11T22:40:43.570 INFO:teuthology.orchestra.run.smithi191.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T22:40:43.603 INFO:teuthology.orchestra.run.smithi191.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-11T22:40:43.637 INFO:teuthology.orchestra.run.smithi191.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-11T22:40:43.693 INFO:teuthology.orchestra.run.smithi152.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-09-11T22:40:43.696 INFO:teuthology.orchestra.run.smithi191.stdout:(132/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.9 MB/s | 2.0 MB 00:00 2024-09-11T22:40:43.721 INFO:teuthology.orchestra.run.smithi191.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-11T22:40:43.746 INFO:teuthology.orchestra.run.smithi191.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-11T22:40:43.747 INFO:teuthology.orchestra.run.smithi204.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 144 kB/s | 41 kB 00:00 2024-09-11T22:40:43.772 INFO:teuthology.orchestra.run.smithi191.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-11T22:40:43.797 INFO:teuthology.orchestra.run.smithi191.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-11T22:40:43.831 INFO:teuthology.orchestra.run.smithi191.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-09-11T22:40:43.864 INFO:teuthology.orchestra.run.smithi191.stdout:(138/156): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-09-11T22:40:43.890 INFO:teuthology.orchestra.run.smithi191.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-11T22:40:43.915 INFO:teuthology.orchestra.run.smithi191.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-11T22:40:43.940 INFO:teuthology.orchestra.run.smithi191.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-11T22:40:43.974 INFO:teuthology.orchestra.run.smithi191.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 887 kB/s | 74 kB 00:00 2024-09-11T22:40:43.999 INFO:teuthology.orchestra.run.smithi191.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-11T22:40:44.032 INFO:teuthology.orchestra.run.smithi191.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-11T22:40:44.058 INFO:teuthology.orchestra.run.smithi191.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-09-11T22:40:44.083 INFO:teuthology.orchestra.run.smithi191.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-11T22:40:44.117 INFO:teuthology.orchestra.run.smithi191.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-11T22:40:44.155 INFO:teuthology.orchestra.run.smithi204.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 609 kB/s | 249 kB 00:00 2024-09-11T22:40:44.167 INFO:teuthology.orchestra.run.smithi191.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-11T22:40:44.260 INFO:teuthology.orchestra.run.smithi152.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 181 kB/s | 137 kB 00:00 2024-09-11T22:40:44.342 INFO:teuthology.orchestra.run.smithi191.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 13 MB/s | 3.4 MB 00:00 2024-09-11T22:40:44.381 INFO:teuthology.orchestra.run.smithi204.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 1.7 MB/s | 1.8 MB 00:01 2024-09-11T22:40:44.385 INFO:teuthology.orchestra.run.smithi152.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 957 kB/s | 910 kB 00:00 2024-09-11T22:40:44.392 INFO:teuthology.orchestra.run.smithi191.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 10 MB/s | 2.3 MB 00:00 2024-09-11T22:40:44.434 INFO:teuthology.orchestra.run.smithi191.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 981 kB/s | 41 kB 00:00 2024-09-11T22:40:44.485 INFO:teuthology.orchestra.run.smithi191.stdout:(152/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 8.6 MB/s | 3.2 MB 00:00 2024-09-11T22:40:44.510 INFO:teuthology.orchestra.run.smithi191.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 3.2 MB/s | 249 kB 00:00 2024-09-11T22:40:44.560 INFO:teuthology.orchestra.run.smithi191.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 8.3 MB/s | 1.8 MB 00:00 2024-09-11T22:40:44.611 INFO:teuthology.orchestra.run.smithi191.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-09-11T22:40:44.681 INFO:teuthology.orchestra.run.smithi204.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 8.0 MB/s | 2.4 MB 00:00 2024-09-11T22:40:44.727 INFO:teuthology.orchestra.run.smithi191.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 5.6 MB/s | 1.4 MB 00:00 2024-09-11T22:40:44.733 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-09-11T22:40:44.733 INFO:teuthology.orchestra.run.smithi191.stdout:Total 21 MB/s | 226 MB 00:10 2024-09-11T22:40:44.756 INFO:teuthology.orchestra.run.smithi204.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 1.4 MB 00:00 2024-09-11T22:40:44.890 INFO:teuthology.orchestra.run.smithi204.stdout:(156/156): NetworkManager-1.49.90-1.el9.x86_64. 1.3 MB/s | 2.3 MB 00:01 2024-09-11T22:40:44.896 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-09-11T22:40:44.896 INFO:teuthology.orchestra.run.smithi204.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-11T22:40:44.961 INFO:teuthology.orchestra.run.smithi152.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 4.6 MB/s | 5.8 MB 00:01 2024-09-11T22:40:45.144 INFO:teuthology.orchestra.run.smithi152.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 115 kB/s | 87 kB 00:00 2024-09-11T22:40:45.270 INFO:teuthology.orchestra.run.smithi152.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:01 2024-09-11T22:40:45.819 INFO:teuthology.orchestra.run.smithi152.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 91 kB/s | 61 kB 00:00 2024-09-11T22:40:45.974 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-09-11T22:40:46.045 INFO:teuthology.orchestra.run.smithi152.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 316 kB/s | 245 kB 00:00 2024-09-11T22:40:46.128 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-09-11T22:40:46.128 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-09-11T22:40:46.130 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-09-11T22:40:46.220 INFO:teuthology.orchestra.run.smithi152.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.7 MB/s | 6.0 MB 00:01 2024-09-11T22:40:46.282 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-09-11T22:40:46.283 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-09-11T22:40:46.720 INFO:teuthology.orchestra.run.smithi152.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 276 kB/s | 248 kB 00:00 2024-09-11T22:40:46.737 INFO:teuthology.orchestra.run.smithi152.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 37 kB/s | 26 kB 00:00 2024-09-11T22:40:46.755 INFO:teuthology.orchestra.run.smithi152.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 36 kB/s | 19 kB 00:00 2024-09-11T22:40:47.354 INFO:teuthology.orchestra.run.smithi152.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 56 kB/s | 35 kB 00:00 2024-09-11T22:40:47.430 INFO:teuthology.orchestra.run.smithi152.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 243 kB/s | 172 kB 00:00 2024-09-11T22:40:47.880 INFO:teuthology.orchestra.run.smithi152.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-09-11T22:40:47.935 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-09-11T22:40:47.936 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-09-11T22:40:48.101 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-09-11T22:40:48.102 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-09-11T22:40:48.239 INFO:teuthology.orchestra.run.smithi152.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 284 kB/s | 250 kB 00:00 2024-09-11T22:40:48.305 INFO:teuthology.orchestra.run.smithi152.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 254 kB/s | 222 kB 00:00 2024-09-11T22:40:48.622 INFO:teuthology.orchestra.run.smithi152.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 104 kB/s | 77 kB 00:00 2024-09-11T22:40:48.859 INFO:teuthology.orchestra.run.smithi152.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 69 kB/s | 42 kB 00:00 2024-09-11T22:40:49.107 INFO:teuthology.orchestra.run.smithi152.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 336 kB/s | 269 kB 00:00 2024-09-11T22:40:49.365 INFO:teuthology.orchestra.run.smithi152.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 214 kB/s | 159 kB 00:00 2024-09-11T22:40:49.740 INFO:teuthology.orchestra.run.smithi152.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 318 kB/s | 279 kB 00:00 2024-09-11T22:40:49.765 INFO:teuthology.orchestra.run.smithi152.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 78 kB/s | 51 kB 00:00 2024-09-11T22:40:50.041 INFO:teuthology.orchestra.run.smithi152.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 79 kB/s | 54 kB 00:00 2024-09-11T22:40:50.390 INFO:teuthology.orchestra.run.smithi152.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-09-11T22:40:50.666 INFO:teuthology.orchestra.run.smithi152.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 71 kB/s | 44 kB 00:00 2024-09-11T22:40:51.291 INFO:teuthology.orchestra.run.smithi152.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-11T22:40:51.316 INFO:teuthology.orchestra.run.smithi152.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 55 kB/s | 36 kB 00:00 2024-09-11T22:40:51.342 INFO:teuthology.orchestra.run.smithi152.stdout:(97/156): qatlib-24.02.0-1.el9.x86_64.rpm 232 kB/s | 221 kB 00:00 2024-09-11T22:40:51.984 INFO:teuthology.orchestra.run.smithi152.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 69 kB/s | 47 kB 00:00 2024-09-11T22:40:52.017 INFO:teuthology.orchestra.run.smithi152.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 86 kB/s | 58 kB 00:00 2024-09-11T22:40:52.117 INFO:teuthology.orchestra.run.smithi152.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 380 kB/s | 304 kB 00:00 2024-09-11T22:40:52.709 INFO:teuthology.orchestra.run.smithi152.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 132 kB/s | 96 kB 00:00 2024-09-11T22:40:52.734 INFO:teuthology.orchestra.run.smithi152.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 89 kB/s | 64 kB 00:00 2024-09-11T22:40:52.759 INFO:teuthology.orchestra.run.smithi152.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 70 kB/s | 45 kB 00:00 2024-09-11T22:40:52.810 INFO:teuthology.orchestra.run.smithi152.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-11T22:40:52.852 INFO:teuthology.orchestra.run.smithi152.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-09-11T22:40:52.877 INFO:teuthology.orchestra.run.smithi152.stdout:(106/156): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-09-11T22:40:52.902 INFO:teuthology.orchestra.run.smithi152.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 386 kB/s | 19 kB 00:00 2024-09-11T22:40:52.936 INFO:teuthology.orchestra.run.smithi152.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-11T22:40:52.961 INFO:teuthology.orchestra.run.smithi152.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-11T22:40:52.986 INFO:teuthology.orchestra.run.smithi152.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-11T22:40:53.012 INFO:teuthology.orchestra.run.smithi152.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-11T22:40:53.062 INFO:teuthology.orchestra.run.smithi152.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-11T22:40:53.104 INFO:teuthology.orchestra.run.smithi152.stdout:(113/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-09-11T22:40:53.146 INFO:teuthology.orchestra.run.smithi152.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-09-11T22:40:53.171 INFO:teuthology.orchestra.run.smithi152.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.4 MB/s | 32 kB 00:00 2024-09-11T22:40:53.196 INFO:teuthology.orchestra.run.smithi152.stdout:(116/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 472 kB/s | 43 kB 00:00 2024-09-11T22:40:53.222 INFO:teuthology.orchestra.run.smithi152.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-11T22:40:53.256 INFO:teuthology.orchestra.run.smithi152.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-11T22:40:53.289 INFO:teuthology.orchestra.run.smithi152.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-11T22:40:53.323 INFO:teuthology.orchestra.run.smithi152.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 6.5 MB/s | 220 kB 00:00 2024-09-11T22:40:53.348 INFO:teuthology.orchestra.run.smithi152.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-09-11T22:40:53.382 INFO:teuthology.orchestra.run.smithi152.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-11T22:40:53.407 INFO:teuthology.orchestra.run.smithi152.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-11T22:40:53.449 INFO:teuthology.orchestra.run.smithi152.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 427 kB/s | 18 kB 00:00 2024-09-11T22:40:53.474 INFO:teuthology.orchestra.run.smithi152.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-09-11T22:40:53.499 INFO:teuthology.orchestra.run.smithi152.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 784 kB/s | 19 kB 00:00 2024-09-11T22:40:53.525 INFO:teuthology.orchestra.run.smithi152.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-11T22:40:53.600 INFO:teuthology.orchestra.run.smithi152.stdout:(128/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-09-11T22:40:53.625 INFO:teuthology.orchestra.run.smithi152.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T22:40:53.650 INFO:teuthology.orchestra.run.smithi152.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T22:40:53.684 INFO:teuthology.orchestra.run.smithi152.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-11T22:40:53.709 INFO:teuthology.orchestra.run.smithi152.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 987 kB/s | 58 kB 00:00 2024-09-11T22:40:53.751 INFO:teuthology.orchestra.run.smithi152.stdout:(133/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.8 MB/s | 2.0 MB 00:00 2024-09-11T22:40:53.777 INFO:teuthology.orchestra.run.smithi152.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-09-11T22:40:53.801 INFO:teuthology.orchestra.run.smithi152.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-11T22:40:53.827 INFO:teuthology.orchestra.run.smithi152.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-11T22:40:53.844 INFO:teuthology.orchestra.run.smithi152.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 735 kB/s | 31 kB 00:00 2024-09-11T22:40:53.861 INFO:teuthology.orchestra.run.smithi152.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-11T22:40:53.886 INFO:teuthology.orchestra.run.smithi152.stdout:(139/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-09-11T22:40:53.903 INFO:teuthology.orchestra.run.smithi152.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 854 kB/s | 36 kB 00:00 2024-09-11T22:40:53.936 INFO:teuthology.orchestra.run.smithi152.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-09-11T22:40:53.961 INFO:teuthology.orchestra.run.smithi152.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-11T22:40:53.987 INFO:teuthology.orchestra.run.smithi152.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-11T22:40:54.012 INFO:teuthology.orchestra.run.smithi152.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-11T22:40:54.037 INFO:teuthology.orchestra.run.smithi152.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-11T22:40:54.086 INFO:teuthology.orchestra.run.smithi152.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-11T22:40:54.111 INFO:teuthology.orchestra.run.smithi152.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-11T22:40:54.162 INFO:teuthology.orchestra.run.smithi152.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-11T22:40:54.261 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-09-11T22:40:54.332 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-09-11T22:40:54.364 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T22:40:54.387 INFO:teuthology.orchestra.run.smithi152.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-11T22:40:54.403 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T22:40:54.445 INFO:teuthology.orchestra.run.smithi152.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 9.5 MB/s | 3.2 MB 00:00 2024-09-11T22:40:54.452 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T22:40:54.490 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T22:40:54.497 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T22:40:54.532 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T22:40:54.552 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T22:40:54.565 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T22:40:54.586 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T22:40:54.612 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T22:40:54.625 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T22:40:54.663 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T22:40:55.049 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T22:40:55.071 INFO:teuthology.orchestra.run.smithi152.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 65 kB/s | 41 kB 00:00 2024-09-11T22:40:55.079 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T22:40:55.175 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:40:55.187 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:40:55.296 INFO:teuthology.orchestra.run.smithi152.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 2.0 MB/s | 2.3 MB 00:01 2024-09-11T22:40:55.297 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:40:55.309 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:40:55.334 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:40:55.339 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:40:55.426 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:40:55.431 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:40:55.459 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:40:55.463 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:40:55.496 INFO:teuthology.orchestra.run.smithi152.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 1.6 MB/s | 1.8 MB 00:01 2024-09-11T22:40:55.614 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T22:40:55.615 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T22:40:55.665 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T22:40:55.669 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T22:40:55.693 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:40:55.696 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:40:55.784 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:40:55.797 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:40:55.814 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:40:55.819 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:40:55.861 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:40:55.882 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:40:55.890 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:40:55.903 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:40:55.914 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:40:55.923 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:40:55.930 INFO:teuthology.orchestra.run.smithi152.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 290 kB/s | 249 kB 00:00 2024-09-11T22:40:56.084 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:40:56.113 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:40:56.226 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:40:56.268 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:40:56.314 INFO:teuthology.orchestra.run.smithi152.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.3 MB/s | 1.4 MB 00:01 2024-09-11T22:40:56.366 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T22:40:56.389 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T22:40:56.469 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:40:56.492 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:40:56.538 INFO:teuthology.orchestra.run.smithi152.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-09-11T22:40:56.546 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-11T22:40:56.546 INFO:teuthology.orchestra.run.smithi152.stdout:Total 8.9 MB/s | 226 MB 00:25 2024-09-11T22:40:56.548 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:40:56.553 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:40:56.760 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:40:56.773 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:40:56.873 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T22:40:56.887 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T22:40:56.971 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T22:40:56.982 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T22:40:56.999 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T22:40:57.018 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T22:40:57.034 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T22:40:57.052 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T22:40:57.107 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T22:40:57.126 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T22:40:57.151 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T22:40:57.178 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T22:40:57.183 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T22:40:57.218 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T22:40:57.227 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T22:40:57.259 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T22:40:57.262 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T22:40:57.287 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T22:40:57.294 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T22:40:57.322 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T22:40:57.356 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T22:40:57.393 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T22:40:57.418 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T22:40:57.446 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T22:40:57.456 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T22:40:57.475 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T22:40:57.502 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T22:40:57.510 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T22:40:57.587 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T22:40:57.588 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T22:40:57.621 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T22:40:57.622 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T22:40:57.656 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T22:40:57.664 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T22:40:57.685 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T22:40:57.693 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T22:40:57.714 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-09-11T22:40:57.757 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T22:40:57.764 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T22:40:57.808 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T22:40:57.820 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T22:40:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-09-11T22:40:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-09-11T22:40:57.883 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T22:40:57.896 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T22:40:58.018 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T22:40:58.032 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T22:40:58.077 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T22:40:58.089 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T22:40:58.609 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T22:40:58.628 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T22:40:58.691 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T22:40:58.711 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T22:40:58.727 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T22:40:58.739 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T22:40:58.782 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T22:40:58.786 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T22:40:58.820 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T22:40:58.824 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T22:40:58.862 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T22:40:58.875 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T22:40:59.011 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T22:40:59.034 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T22:40:59.073 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T22:40:59.103 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T22:40:59.128 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T22:40:59.159 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T22:40:59.202 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T22:40:59.232 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T22:40:59.232 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T22:40:59.262 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T22:40:59.268 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T22:40:59.298 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T22:40:59.316 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T22:40:59.346 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T22:40:59.360 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T22:40:59.390 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T22:40:59.403 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T22:40:59.433 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T22:40:59.445 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T22:40:59.475 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T22:40:59.584 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T22:40:59.613 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T22:40:59.630 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T22:40:59.660 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T22:40:59.666 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T22:40:59.701 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T22:40:59.770 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T22:40:59.806 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T22:40:59.808 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T22:40:59.834 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T22:40:59.848 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T22:40:59.873 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T22:40:59.882 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T22:40:59.898 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T22:40:59.917 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T22:40:59.942 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T22:40:59.975 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T22:41:00.018 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T22:41:00.101 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T22:41:00.131 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T22:41:00.144 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T22:41:00.158 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T22:41:00.168 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-09-11T22:41:00.168 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-09-11T22:41:00.193 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T22:41:00.234 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T22:41:00.316 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T22:41:00.344 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T22:41:00.349 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T22:41:00.386 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T22:41:00.426 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T22:41:00.466 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T22:41:00.471 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T22:41:00.495 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T22:41:00.505 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T22:41:00.530 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T22:41:00.539 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T22:41:00.563 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T22:41:00.581 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T22:41:00.615 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T22:41:00.620 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T22:41:00.679 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T22:41:00.706 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T22:41:00.771 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T22:41:00.771 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T22:41:00.801 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T22:41:00.820 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T22:41:00.851 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T22:41:00.906 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T22:41:00.960 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T22:41:00.963 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T22:41:01.027 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T22:41:01.712 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T22:41:01.758 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T22:41:01.764 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T22:41:01.791 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T22:41:01.815 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T22:41:01.849 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T22:41:01.894 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T22:41:01.947 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T22:41:02.057 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T22:41:02.058 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T22:41:02.058 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T22:41:02.060 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T22:41:02.089 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T22:41:02.111 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T22:41:02.159 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T22:41:02.195 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T22:41:02.295 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T22:41:02.325 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T22:41:02.378 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T22:41:02.403 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T22:41:02.407 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T22:41:02.485 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T22:41:03.064 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T22:41:03.148 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T22:41:03.174 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T22:41:03.264 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T22:41:04.444 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T22:41:04.520 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T22:41:04.521 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T22:41:04.549 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T22:41:04.582 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T22:41:04.619 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T22:41:04.648 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T22:41:04.680 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T22:41:04.892 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T22:41:04.929 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:41:04.993 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T22:41:05.012 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:41:05.042 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T22:41:05.044 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:41:05.078 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T22:41:05.134 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:41:05.173 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T22:41:05.209 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T22:41:05.556 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T22:41:05.582 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:41:05.645 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:41:05.675 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T22:41:05.688 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T22:41:05.871 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:41:05.926 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:41:05.956 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T22:41:06.519 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-09-11T22:41:06.634 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T22:41:06.687 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T22:41:06.751 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T22:41:06.784 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T22:41:06.810 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T22:41:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T22:41:07.275 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T22:41:07.327 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:41:07.450 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:41:07.478 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:41:07.570 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:41:07.600 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:41:07.774 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T22:41:07.838 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T22:41:07.874 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:41:07.978 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:41:08.000 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:41:08.073 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:41:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:41:08.114 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:41:08.120 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:41:08.271 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:41:08.311 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:41:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:41:08.406 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:41:08.513 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:41:08.624 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T22:41:08.728 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:41:08.809 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:41:08.859 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:41:08.906 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:41:08.954 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:41:08.992 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:41:08.994 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:41:08.994 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T22:41:08.994 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T22:41:08.994 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:41:09.041 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:41:09.047 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:41:09.077 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:41:09.077 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T22:41:09.077 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T22:41:09.077 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:41:09.110 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:41:09.161 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T22:41:09.252 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T22:41:09.295 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T22:41:09.340 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T22:41:09.413 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T22:41:09.466 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T22:41:09.498 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T22:41:09.550 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T22:41:09.583 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T22:41:09.610 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T22:41:09.688 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T22:41:09.740 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T22:41:09.771 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T22:41:09.822 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T22:41:09.907 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T22:41:09.959 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T22:41:10.002 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T22:41:10.031 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T22:41:10.099 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T22:41:10.145 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T22:41:10.228 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T22:41:10.360 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T22:41:10.430 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T22:41:10.965 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T22:41:11.042 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T22:41:11.072 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T22:41:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T22:41:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T22:41:11.206 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T22:41:11.377 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T22:41:11.431 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T22:41:11.485 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T22:41:11.566 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T22:41:11.607 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T22:41:11.643 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T22:41:11.689 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T22:41:11.726 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T22:41:11.769 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T22:41:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T22:41:11.946 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T22:41:11.995 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T22:41:12.045 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T22:41:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T22:41:12.188 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T22:41:12.228 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T22:41:12.252 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T22:41:12.280 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T22:41:12.384 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T22:41:12.514 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T22:41:12.546 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T22:41:12.573 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T22:41:12.611 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T22:41:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T22:41:12.686 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T22:41:12.717 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T22:41:12.751 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T22:41:12.784 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T22:41:12.835 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T22:41:12.869 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T22:41:12.955 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T22:41:13.008 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T22:41:13.039 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T22:41:13.141 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T22:41:13.189 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T22:41:13.926 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T22:41:14.220 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T22:41:14.262 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T22:41:14.360 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T22:41:14.408 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T22:41:14.462 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T22:41:14.547 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T22:41:14.589 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T22:41:14.783 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T22:41:14.812 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T22:41:14.898 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T22:41:15.551 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T22:41:15.667 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T22:41:16.917 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T22:41:16.998 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T22:41:17.025 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T22:41:17.059 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T22:41:17.383 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T22:41:17.430 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:41:17.521 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:41:17.552 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T22:41:17.587 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T22:41:18.065 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T22:41:18.091 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:41:18.162 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:41:18.184 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T22:41:20.443 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:41:20.481 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:41:20.841 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:41:20.842 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-09-11T22:41:20.842 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-09-11T22:41:20.842 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-09-11T22:41:20.842 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-09-11T22:41:20.842 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-09-11T22:41:20.842 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-09-11T22:41:20.842 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-09-11T22:41:20.842 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:41:21.016 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:41:21.028 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:41:21.032 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:41:21.032 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:21.032 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T22:41:21.032 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:41:21.055 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:41:21.055 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /sys 2024-09-11T22:41:21.055 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /proc 2024-09-11T22:41:21.055 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /mnt 2024-09-11T22:41:21.055 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /var/tmp 2024-09-11T22:41:21.055 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /home 2024-09-11T22:41:21.055 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /root 2024-09-11T22:41:21.055 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /tmp 2024-09-11T22:41:21.055 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:41:21.059 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:41:21.152 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:41:21.152 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T22:41:21.152 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T22:41:21.152 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:41:21.193 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:41:21.235 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:41:21.252 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:41:21.252 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:21.252 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T22:41:21.252 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:41:22.502 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:41:22.530 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:41:22.530 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:22.530 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T22:41:22.530 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:41:22.530 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:41:22.530 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:41:22.746 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:41:22.761 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:41:22.773 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:41:22.773 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:22.773 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T22:41:22.773 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:41:22.773 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:41:22.774 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:41:22.784 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:41:22.785 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:22.785 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T22:41:22.785 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:41:22.785 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:41:22.785 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:41:22.997 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:41:23.021 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:41:23.021 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:23.021 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T22:41:23.021 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:41:23.021 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:41:23.021 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:41:23.220 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:41:23.244 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:41:23.244 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:23.244 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T22:41:23.244 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:41:23.244 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:41:23.244 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:41:23.350 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T22:41:23.442 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T22:41:23.456 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:41:23.479 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:41:23.479 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:23.480 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T22:41:23.480 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:41:23.480 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:41:23.480 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:41:23.505 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T22:41:23.594 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T22:41:23.659 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T22:41:23.703 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T22:41:23.715 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T22:41:23.755 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T22:41:23.834 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T22:41:23.900 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T22:41:23.923 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T22:41:23.956 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T22:41:24.034 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T22:41:24.066 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T22:41:24.113 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T22:41:24.122 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T22:41:24.145 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T22:41:24.173 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T22:41:24.271 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T22:41:24.315 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T22:41:24.347 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T22:41:24.375 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T22:41:24.624 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:41:24.655 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:41:24.686 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T22:41:24.749 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T22:41:24.767 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:41:24.777 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:41:24.798 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:41:24.830 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T22:41:24.892 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T22:41:24.921 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:41:25.831 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:41:25.879 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:41:25.972 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:41:26.006 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:41:26.846 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:41:26.880 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:41:26.982 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:41:26.985 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:41:27.024 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:41:27.128 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:41:27.151 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T22:41:27.182 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:41:27.206 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:41:27.206 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:27.207 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T22:41:27.207 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:41:27.207 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:41:27.207 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:41:27.266 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:41:27.279 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:41:27.295 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T22:41:27.325 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:41:27.350 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:41:27.350 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:27.350 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T22:41:27.350 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:41:27.350 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:41:27.350 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:41:27.376 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T22:41:27.401 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:41:27.414 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:41:27.423 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T22:41:27.513 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T22:41:27.566 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T22:41:28.345 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T22:41:28.369 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:41:28.393 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:41:28.393 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:28.393 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T22:41:28.393 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:41:28.393 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:41:28.394 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:41:28.436 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:41:28.461 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:41:28.461 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:28.462 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T22:41:28.462 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:41:28.499 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T22:41:28.512 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:41:28.536 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:41:28.536 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:28.536 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T22:41:28.536 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:41:28.537 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:41:28.537 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:41:28.537 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:41:28.561 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:41:28.561 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:28.562 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T22:41:28.562 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:41:28.562 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:41:28.562 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:41:28.604 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:41:28.627 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:41:28.628 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:28.628 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T22:41:28.628 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:41:28.680 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:41:28.703 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:41:28.704 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:28.704 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T22:41:28.704 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:41:28.704 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:41:28.704 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:41:28.837 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:41:28.861 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:41:28.861 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:28.861 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T22:41:28.862 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:41:28.862 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:41:28.862 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:41:29.014 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:41:29.038 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:41:29.038 INFO:teuthology.orchestra.run.smithi191.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:29.038 INFO:teuthology.orchestra.run.smithi191.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T22:41:29.038 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:41:29.038 INFO:teuthology.orchestra.run.smithi191.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:41:29.038 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:41:33.270 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:41:33.271 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /sys 2024-09-11T22:41:33.271 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /proc 2024-09-11T22:41:33.271 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /mnt 2024-09-11T22:41:33.271 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /var/tmp 2024-09-11T22:41:33.271 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /home 2024-09-11T22:41:33.271 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /root 2024-09-11T22:41:33.271 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /tmp 2024-09-11T22:41:33.271 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:41:33.425 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T22:41:33.443 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:41:33.458 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:41:33.459 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:33.459 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T22:41:33.459 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:41:33.466 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T22:41:33.508 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T22:41:33.539 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T22:41:33.566 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T22:41:33.609 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T22:41:33.651 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T22:41:33.692 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T22:41:33.723 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T22:41:33.751 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T22:41:33.758 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T22:41:33.814 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T22:41:33.844 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T22:41:33.874 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T22:41:33.928 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T22:41:33.948 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T22:41:33.997 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T22:41:34.028 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T22:41:34.057 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T22:41:34.103 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T22:41:34.519 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:41:34.559 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:41:34.581 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:41:34.741 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:41:34.796 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:41:34.905 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:41:34.937 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:41:34.969 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:41:34.970 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:34.970 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T22:41:34.970 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:41:34.970 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:41:34.970 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:41:35.205 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:41:35.234 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:41:35.234 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:35.234 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T22:41:35.234 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:41:35.235 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:41:35.235 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:41:35.464 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T22:41:35.548 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T22:41:35.581 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T22:41:35.646 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T22:41:35.673 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:41:35.675 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T22:41:35.696 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:41:35.696 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:35.696 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T22:41:35.696 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:41:35.697 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:41:35.697 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:41:35.717 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T22:41:35.739 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T22:41:35.743 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T22:41:35.769 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T22:41:35.769 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:41:35.780 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T22:41:35.813 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T22:41:35.860 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T22:41:35.861 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:41:35.861 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:41:35.862 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T22:41:35.892 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T22:41:35.917 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T22:41:35.923 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T22:41:35.929 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:41:35.952 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T22:41:35.952 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:41:35.954 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T22:41:35.959 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:41:35.960 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:41:36.053 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:41:36.053 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:41:36.110 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T22:41:36.128 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:41:36.147 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:41:36.148 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:41:36.153 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T22:41:36.159 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:41:36.159 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:41:36.165 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T22:41:36.232 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:41:36.242 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T22:41:36.295 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:41:36.338 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:41:36.338 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:41:36.339 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T22:41:36.423 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:41:36.471 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T22:41:36.485 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:41:36.524 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T22:41:36.555 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T22:41:36.583 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T22:41:37.025 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:41:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:41:37.096 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T22:41:37.159 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T22:41:37.188 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:41:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:41:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:41:39.247 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:41:39.290 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:41:39.394 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:41:39.551 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T22:41:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:41:39.608 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:41:39.608 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:39.608 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T22:41:39.608 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:41:39.608 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:41:39.608 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:41:39.651 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:41:39.662 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:41:39.760 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T22:41:39.810 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T22:41:40.735 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T22:41:40.770 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:41:40.793 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:41:40.794 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:40.794 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T22:41:40.794 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:41:40.794 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:41:40.794 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:41:40.846 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:41:40.868 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:41:40.868 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:40.869 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T22:41:40.869 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:41:40.913 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:41:40.935 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:41:40.936 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:40.936 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T22:41:40.936 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:41:40.936 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:41:40.936 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:41:41.238 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:41:41.261 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:41:41.262 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:41:41.262 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T22:41:41.262 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:41:41.262 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:41:41.262 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:41:42.012 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:41:42.012 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T22:41:42.013 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T22:41:42.013 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T22:41:42.013 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T22:41:42.013 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T22:41:42.013 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T22:41:42.013 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T22:41:42.013 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:41:42.013 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:41:42.013 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:41:42.013 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T22:41:42.013 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T22:41:42.013 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:41:42.013 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T22:41:42.014 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T22:41:42.014 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T22:41:42.014 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T22:41:42.014 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T22:41:42.014 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T22:41:42.014 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T22:41:42.014 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T22:41:42.014 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T22:41:42.014 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T22:41:42.014 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T22:41:42.014 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T22:41:42.014 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T22:41:42.014 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T22:41:42.015 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T22:41:42.015 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T22:41:42.015 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T22:41:42.028 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T22:41:42.028 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T22:41:42.028 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T22:41:42.028 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T22:41:42.028 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T22:41:42.029 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T22:41:42.029 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T22:41:42.029 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T22:41:42.029 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T22:41:42.029 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T22:41:42.029 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T22:41:42.029 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T22:41:42.029 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T22:41:42.029 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T22:41:42.029 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T22:41:42.029 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T22:41:42.029 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T22:41:42.029 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T22:41:42.029 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T22:41:42.030 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T22:41:42.030 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T22:41:42.030 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T22:41:42.030 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T22:41:42.030 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T22:41:42.030 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T22:41:42.030 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T22:41:42.030 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T22:41:42.030 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T22:41:42.030 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T22:41:42.030 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T22:41:42.030 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T22:41:42.030 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T22:41:42.031 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T22:41:42.031 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T22:41:42.031 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T22:41:42.031 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T22:41:42.031 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T22:41:42.031 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T22:41:42.031 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T22:41:42.031 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T22:41:42.031 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T22:41:42.031 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T22:41:42.031 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T22:41:42.031 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T22:41:42.031 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T22:41:42.032 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T22:41:42.032 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T22:41:42.032 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T22:41:42.032 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T22:41:42.032 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T22:41:42.032 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T22:41:42.032 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T22:41:42.032 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T22:41:42.032 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T22:41:42.032 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T22:41:42.032 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T22:41:42.032 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T22:41:42.032 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T22:41:42.032 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T22:41:42.033 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T22:41:42.033 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T22:41:42.033 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T22:41:42.033 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T22:41:42.033 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T22:41:42.033 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T22:41:42.033 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T22:41:42.033 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T22:41:42.033 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T22:41:42.033 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T22:41:42.033 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T22:41:42.033 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T22:41:42.033 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T22:41:42.034 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T22:41:42.034 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T22:41:42.034 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T22:41:42.034 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T22:41:42.034 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T22:41:42.034 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T22:41:42.034 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T22:41:42.034 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T22:41:42.034 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T22:41:42.034 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T22:41:42.034 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T22:41:42.034 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T22:41:42.034 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T22:41:42.034 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T22:41:42.035 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T22:41:42.035 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T22:41:42.035 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T22:41:42.035 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T22:41:42.035 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T22:41:42.035 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T22:41:42.035 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T22:41:42.035 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T22:41:42.035 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T22:41:42.035 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T22:41:42.035 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T22:41:42.035 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T22:41:42.035 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T22:41:42.035 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T22:41:42.036 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T22:41:42.036 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T22:41:42.036 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T22:41:42.036 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T22:41:42.036 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T22:41:42.036 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T22:41:42.036 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T22:41:42.036 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T22:41:42.036 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T22:41:42.036 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T22:41:42.036 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T22:41:42.036 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T22:41:42.036 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T22:41:42.037 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T22:41:42.037 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T22:41:42.037 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T22:41:42.037 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T22:41:42.037 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T22:41:42.037 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T22:41:42.037 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T22:41:42.037 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T22:41:42.037 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T22:41:42.037 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T22:41:42.037 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T22:41:42.037 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T22:41:42.037 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T22:41:42.037 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T22:41:42.038 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T22:41:42.038 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T22:41:42.038 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T22:41:42.038 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T22:41:42.038 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T22:41:42.038 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T22:41:42.335 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:41:42.335 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T22:41:42.335 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T22:41:42.335 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T22:41:42.335 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T22:41:42.335 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T22:41:42.335 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T22:41:42.335 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T22:41:42.336 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:41:42.336 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:41:42.336 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:41:42.336 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T22:41:42.336 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T22:41:42.336 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:41:42.336 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T22:41:42.336 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T22:41:42.336 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T22:41:42.337 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T22:41:42.337 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T22:41:42.337 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T22:41:42.337 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T22:41:42.337 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T22:41:42.337 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T22:41:42.337 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T22:41:42.337 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T22:41:42.337 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T22:41:42.337 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T22:41:42.337 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T22:41:42.337 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T22:41:42.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T22:41:42.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T22:41:42.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T22:41:42.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T22:41:42.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T22:41:42.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T22:41:42.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T22:41:42.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T22:41:42.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T22:41:42.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T22:41:42.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T22:41:42.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T22:41:42.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T22:41:42.338 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T22:41:42.339 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T22:41:42.339 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T22:41:42.339 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T22:41:42.339 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T22:41:42.339 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T22:41:42.339 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T22:41:42.339 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T22:41:42.339 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T22:41:42.339 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T22:41:42.339 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T22:41:42.339 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T22:41:42.339 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T22:41:42.339 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T22:41:42.340 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T22:41:42.340 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T22:41:42.340 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T22:41:42.340 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T22:41:42.340 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T22:41:42.340 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T22:41:42.340 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T22:41:42.340 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T22:41:42.340 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T22:41:42.340 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T22:41:42.340 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T22:41:42.340 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T22:41:42.341 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T22:41:42.341 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T22:41:42.341 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T22:41:42.341 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T22:41:42.341 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T22:41:42.341 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T22:41:42.341 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T22:41:42.341 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T22:41:42.341 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T22:41:42.341 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T22:41:42.341 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T22:41:42.341 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T22:41:42.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T22:41:42.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T22:41:42.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T22:41:42.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T22:41:42.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T22:41:42.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T22:41:42.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T22:41:42.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T22:41:42.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T22:41:42.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T22:41:42.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T22:41:42.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T22:41:42.342 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T22:41:42.343 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T22:41:42.343 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T22:41:42.343 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T22:41:42.343 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T22:41:42.343 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T22:41:42.343 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T22:41:42.343 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T22:41:42.343 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T22:41:42.343 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T22:41:42.343 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T22:41:42.343 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T22:41:42.343 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T22:41:42.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T22:41:42.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T22:41:42.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T22:41:42.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T22:41:42.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T22:41:42.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T22:41:42.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T22:41:42.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T22:41:42.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T22:41:42.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T22:41:42.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T22:41:42.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T22:41:42.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T22:41:42.344 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T22:41:42.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T22:41:42.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T22:41:42.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T22:41:42.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T22:41:42.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T22:41:42.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T22:41:42.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T22:41:42.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T22:41:42.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T22:41:42.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T22:41:42.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T22:41:42.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T22:41:42.345 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T22:41:42.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T22:41:42.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T22:41:42.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T22:41:42.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T22:41:42.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T22:41:42.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T22:41:42.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T22:41:42.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T22:41:42.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T22:41:42.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T22:41:42.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T22:41:42.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T22:41:42.346 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T22:41:42.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T22:41:42.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T22:41:42.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T22:41:42.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T22:41:42.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T22:41:42.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T22:41:42.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T22:41:42.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T22:41:42.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T22:41:42.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T22:41:42.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T22:41:42.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T22:41:42.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T22:41:42.347 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T22:41:42.348 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T22:41:42.348 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T22:41:42.348 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T22:41:42.348 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T22:41:42.348 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T22:41:44.695 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:41:44.695 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:41:44.695 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-09-11T22:41:44.695 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T22:41:44.695 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T22:41:44.696 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T22:41:44.696 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T22:41:44.696 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.696 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.696 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T22:41:44.696 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T22:41:44.696 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-09-11T22:41:44.696 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T22:41:44.696 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T22:41:44.696 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T22:41:44.696 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T22:41:44.696 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.697 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.697 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.697 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.697 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.697 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.697 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.697 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.697 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.697 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.697 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.697 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.697 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.697 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.698 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.698 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.698 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.698 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.698 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.698 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.698 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.698 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.698 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.698 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.698 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T22:41:44.698 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T22:41:44.698 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T22:41:44.699 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T22:41:44.699 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T22:41:44.699 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T22:41:44.699 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T22:41:44.699 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T22:41:44.699 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T22:41:44.699 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T22:41:44.699 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T22:41:44.699 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.699 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.699 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.699 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T22:41:44.699 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T22:41:44.699 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T22:41:44.700 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T22:41:44.700 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T22:41:44.700 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.700 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.700 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T22:41:44.700 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.700 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T22:41:44.700 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T22:41:44.700 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T22:41:44.700 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T22:41:44.700 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T22:41:44.700 INFO:teuthology.orchestra.run.smithi204.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T22:41:44.701 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T22:41:44.701 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T22:41:44.701 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T22:41:44.701 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T22:41:44.701 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T22:41:44.701 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T22:41:44.701 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T22:41:44.701 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T22:41:44.701 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T22:41:44.701 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T22:41:44.701 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T22:41:44.701 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T22:41:44.701 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T22:41:44.701 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T22:41:44.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T22:41:44.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T22:41:44.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T22:41:44.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T22:41:44.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T22:41:44.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T22:41:44.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T22:41:44.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T22:41:44.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T22:41:44.702 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T22:41:44.703 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T22:41:44.703 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T22:41:44.703 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T22:41:44.703 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T22:41:44.703 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T22:41:44.703 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T22:41:44.703 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T22:41:44.703 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T22:41:44.703 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T22:41:44.703 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T22:41:44.703 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T22:41:44.703 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T22:41:44.703 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T22:41:44.704 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T22:41:44.704 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T22:41:44.704 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T22:41:44.704 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T22:41:44.704 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T22:41:44.704 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T22:41:44.704 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T22:41:44.704 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T22:41:44.704 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T22:41:44.704 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T22:41:44.704 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T22:41:44.704 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T22:41:44.704 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T22:41:44.705 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T22:41:44.705 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T22:41:44.705 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T22:41:44.705 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T22:41:44.705 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T22:41:44.705 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T22:41:44.705 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T22:41:44.705 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.705 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.705 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T22:41:44.705 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T22:41:44.705 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T22:41:44.705 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.706 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T22:41:44.706 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T22:41:44.706 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T22:41:44.706 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T22:41:44.706 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T22:41:44.706 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T22:41:44.706 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T22:41:44.706 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T22:41:44.706 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T22:41:44.706 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T22:41:44.706 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T22:41:44.706 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T22:41:44.706 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T22:41:44.706 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T22:41:44.707 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T22:41:44.707 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.707 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.707 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.707 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T22:41:44.707 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T22:41:44.707 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T22:41:44.707 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T22:41:44.707 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T22:41:44.707 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T22:41:44.708 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T22:41:44.708 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:41:44.708 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-09-11T22:41:44.841 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:41:44.841 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:41:44.841 INFO:teuthology.orchestra.run.smithi191.stdout:Upgraded: 2024-09-11T22:41:44.841 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T22:41:44.841 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T22:41:44.842 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T22:41:44.842 INFO:teuthology.orchestra.run.smithi191.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T22:41:44.842 INFO:teuthology.orchestra.run.smithi191.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.842 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.842 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T22:41:44.842 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T22:41:44.842 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-09-11T22:41:44.842 INFO:teuthology.orchestra.run.smithi191.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T22:41:44.842 INFO:teuthology.orchestra.run.smithi191.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T22:41:44.842 INFO:teuthology.orchestra.run.smithi191.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T22:41:44.842 INFO:teuthology.orchestra.run.smithi191.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T22:41:44.842 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.843 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.843 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.843 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.843 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.843 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.843 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.843 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.843 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.843 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.843 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.843 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.843 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.843 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.843 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T22:41:44.844 INFO:teuthology.orchestra.run.smithi191.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T22:41:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T22:41:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T22:41:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T22:41:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T22:41:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T22:41:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T22:41:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T22:41:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T22:41:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T22:41:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.845 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T22:41:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T22:41:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T22:41:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T22:41:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T22:41:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T22:41:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T22:41:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T22:41:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T22:41:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T22:41:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T22:41:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T22:41:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T22:41:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T22:41:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T22:41:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T22:41:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T22:41:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T22:41:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T22:41:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T22:41:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T22:41:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T22:41:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T22:41:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T22:41:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T22:41:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T22:41:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T22:41:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T22:41:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T22:41:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T22:41:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T22:41:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T22:41:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T22:41:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T22:41:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T22:41:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T22:41:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T22:41:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T22:41:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T22:41:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T22:41:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T22:41:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T22:41:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T22:41:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T22:41:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T22:41:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T22:41:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T22:41:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T22:41:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T22:41:44.851 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T22:41:44.851 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T22:41:44.851 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T22:41:44.851 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T22:41:44.851 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T22:41:44.851 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T22:41:44.852 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T22:41:44.852 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T22:41:44.852 INFO:teuthology.orchestra.run.smithi191.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T22:41:44.852 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T22:41:44.852 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T22:41:44.852 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T22:41:44.852 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T22:41:44.852 INFO:teuthology.orchestra.run.smithi191.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T22:41:44.852 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T22:41:44.852 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T22:41:44.853 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T22:41:44.853 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T22:41:44.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T22:41:44.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T22:41:44.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T22:41:44.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T22:41:44.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T22:41:44.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T22:41:44.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T22:41:44.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T22:41:44.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T22:41:44.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T22:41:44.854 INFO:teuthology.orchestra.run.smithi191.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T22:41:44.855 INFO:teuthology.orchestra.run.smithi191.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T22:41:44.855 INFO:teuthology.orchestra.run.smithi191.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T22:41:44.855 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T22:41:44.855 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T22:41:44.855 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T22:41:44.855 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T22:41:44.855 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T22:41:44.855 INFO:teuthology.orchestra.run.smithi191.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T22:41:44.855 INFO:teuthology.orchestra.run.smithi191.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T22:41:44.856 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.856 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.856 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:44.856 INFO:teuthology.orchestra.run.smithi191.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T22:41:44.856 INFO:teuthology.orchestra.run.smithi191.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T22:41:44.856 INFO:teuthology.orchestra.run.smithi191.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T22:41:44.856 INFO:teuthology.orchestra.run.smithi191.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T22:41:44.856 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T22:41:44.856 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T22:41:44.857 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T22:41:44.857 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:41:44.857 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-09-11T22:41:45.155 DEBUG:teuthology.parallel:result is None 2024-09-11T22:41:45.279 DEBUG:teuthology.parallel:result is None 2024-09-11T22:41:45.834 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T22:41:45.875 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T22:41:45.917 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T22:41:45.956 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T22:41:45.992 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T22:41:46.197 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T22:41:46.239 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T22:41:46.269 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T22:41:46.299 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T22:41:46.345 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T22:41:46.947 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:41:46.992 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:41:47.013 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:41:47.854 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T22:41:47.946 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T22:41:48.001 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T22:41:48.103 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T22:41:48.141 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T22:41:48.171 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T22:41:48.200 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T22:41:48.200 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:41:48.285 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:41:48.285 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:41:48.377 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:41:48.402 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T22:41:48.411 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:41:48.412 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:41:48.595 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:41:48.595 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:41:48.679 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:41:48.744 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:41:54.841 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:41:54.841 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T22:41:54.841 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T22:41:54.841 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T22:41:54.841 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T22:41:54.841 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T22:41:54.841 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T22:41:54.842 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T22:41:54.842 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:41:54.842 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:41:54.842 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:41:54.842 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T22:41:54.842 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T22:41:54.842 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:41:54.842 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T22:41:54.844 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T22:41:54.844 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T22:41:54.844 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T22:41:54.844 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T22:41:54.856 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T22:41:54.856 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T22:41:54.856 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T22:41:54.856 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T22:41:54.856 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T22:41:54.867 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T22:41:54.867 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T22:41:54.867 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T22:41:54.867 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T22:41:54.867 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T22:41:54.867 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T22:41:57.382 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:41:57.382 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:41:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:Upgraded: 2024-09-11T22:41:57.383 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T22:41:57.383 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T22:41:57.383 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T22:41:57.383 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T22:41:57.383 INFO:teuthology.orchestra.run.smithi152.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.383 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.383 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T22:41:57.383 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T22:41:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-09-11T22:41:57.383 INFO:teuthology.orchestra.run.smithi152.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T22:41:57.383 INFO:teuthology.orchestra.run.smithi152.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T22:41:57.384 INFO:teuthology.orchestra.run.smithi152.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T22:41:57.384 INFO:teuthology.orchestra.run.smithi152.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T22:41:57.384 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.384 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.384 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.384 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.384 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:57.384 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.384 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.384 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.384 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:57.384 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:57.385 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:57.385 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:57.385 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:57.385 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.385 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.385 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:57.385 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.385 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.385 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.385 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:57.385 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:57.385 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.386 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.386 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:41:57.386 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T22:41:57.386 INFO:teuthology.orchestra.run.smithi152.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T22:41:57.386 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T22:41:57.386 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T22:41:57.386 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T22:41:57.386 INFO:teuthology.orchestra.run.smithi152.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T22:41:57.386 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T22:41:57.386 INFO:teuthology.orchestra.run.smithi152.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T22:41:57.386 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T22:41:57.386 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T22:41:57.386 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T22:41:57.387 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.387 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.387 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.387 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T22:41:57.387 INFO:teuthology.orchestra.run.smithi152.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T22:41:57.387 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T22:41:57.387 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T22:41:57.387 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T22:41:57.387 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.387 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.387 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T22:41:57.387 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.388 INFO:teuthology.orchestra.run.smithi152.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T22:41:57.388 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T22:41:57.388 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T22:41:57.388 INFO:teuthology.orchestra.run.smithi152.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T22:41:57.388 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T22:41:57.388 INFO:teuthology.orchestra.run.smithi152.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T22:41:57.388 INFO:teuthology.orchestra.run.smithi152.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T22:41:57.388 INFO:teuthology.orchestra.run.smithi152.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T22:41:57.388 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T22:41:57.388 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T22:41:57.388 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T22:41:57.388 INFO:teuthology.orchestra.run.smithi152.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T22:41:57.388 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T22:41:57.389 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T22:41:57.389 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T22:41:57.389 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T22:41:57.389 INFO:teuthology.orchestra.run.smithi152.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T22:41:57.389 INFO:teuthology.orchestra.run.smithi152.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T22:41:57.389 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T22:41:57.389 INFO:teuthology.orchestra.run.smithi152.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T22:41:57.389 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T22:41:57.389 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T22:41:57.389 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T22:41:57.389 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.389 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.390 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.390 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T22:41:57.390 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T22:41:57.390 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T22:41:57.390 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T22:41:57.390 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T22:41:57.390 INFO:teuthology.orchestra.run.smithi152.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T22:41:57.390 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T22:41:57.390 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T22:41:57.390 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T22:41:57.390 INFO:teuthology.orchestra.run.smithi152.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T22:41:57.390 INFO:teuthology.orchestra.run.smithi152.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T22:41:57.390 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T22:41:57.390 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T22:41:57.391 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T22:41:57.391 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T22:41:57.391 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T22:41:57.391 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T22:41:57.391 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T22:41:57.391 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T22:41:57.391 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T22:41:57.391 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T22:41:57.391 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T22:41:57.391 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T22:41:57.391 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T22:41:57.391 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T22:41:57.391 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T22:41:57.392 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T22:41:57.392 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T22:41:57.392 INFO:teuthology.orchestra.run.smithi152.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T22:41:57.392 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T22:41:57.392 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T22:41:57.392 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T22:41:57.392 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T22:41:57.392 INFO:teuthology.orchestra.run.smithi152.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T22:41:57.392 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T22:41:57.392 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T22:41:57.392 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T22:41:57.392 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T22:41:57.392 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T22:41:57.393 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T22:41:57.393 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.393 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.393 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T22:41:57.393 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T22:41:57.393 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T22:41:57.393 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.393 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T22:41:57.393 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T22:41:57.393 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T22:41:57.393 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T22:41:57.393 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T22:41:57.393 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T22:41:57.394 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T22:41:57.394 INFO:teuthology.orchestra.run.smithi152.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T22:41:57.394 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T22:41:57.394 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T22:41:57.394 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T22:41:57.394 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T22:41:57.394 INFO:teuthology.orchestra.run.smithi152.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T22:41:57.394 INFO:teuthology.orchestra.run.smithi152.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T22:41:57.394 INFO:teuthology.orchestra.run.smithi152.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T22:41:57.394 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.394 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.394 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:41:57.394 INFO:teuthology.orchestra.run.smithi152.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T22:41:57.394 INFO:teuthology.orchestra.run.smithi152.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T22:41:57.395 INFO:teuthology.orchestra.run.smithi152.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T22:41:57.395 INFO:teuthology.orchestra.run.smithi152.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T22:41:57.395 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T22:41:57.395 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T22:41:57.395 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T22:41:57.395 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:41:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-09-11T22:41:57.836 DEBUG:teuthology.parallel:result is None 2024-09-11T22:41:57.837 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:41:58.000 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T22:41:58.022 INFO:teuthology.orchestra.run.smithi152.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T22:41:58.023 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T22:41:58.023 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T22:41:58.025 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:41:58.178 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T22:41:58.205 INFO:teuthology.orchestra.run.smithi191.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T22:41:58.205 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T22:41:58.205 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T22:41:58.208 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:41:58.363 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T22:41:58.390 INFO:teuthology.orchestra.run.smithi204.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T22:41:58.390 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T22:41:58.390 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T22:41:58.392 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T22:41:58.393 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:41:58.393 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T22:41:58.423 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-11T22:41:58.423 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T22:41:58.456 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-11T22:41:58.456 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T22:41:58.489 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T22:41:58.489 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:41:58.489 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T22:41:58.517 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T22:41:58.584 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-11T22:41:58.584 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T22:41:58.613 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T22:41:58.680 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-11T22:41:58.680 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T22:41:58.712 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T22:41:58.783 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T22:41:58.783 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:41:58.783 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T22:41:58.809 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T22:41:58.876 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-11T22:41:58.876 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T22:41:58.903 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T22:41:58.969 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-11T22:41:58.969 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T22:41:58.995 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T22:41:59.061 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T22:41:59.061 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:41:59.061 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T22:41:59.088 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T22:41:59.154 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-11T22:41:59.154 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T22:41:59.181 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T22:41:59.246 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-11T22:41:59.246 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T22:41:59.271 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T22:41:59.340 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T22:41:59.442 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-11T22:41:59.443 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:41:59.443 INFO:tasks.cephadm:Cluster fsid is 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:41:59.443 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T22:41:59.443 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.152', 'mon.b': '172.21.15.191', 'mon.c': '172.21.15.204'} 2024-09-11T22:41:59.444 INFO:tasks.cephadm:First mon is mon.a on smithi152 2024-09-11T22:41:59.444 INFO:tasks.cephadm:First mgr is x 2024-09-11T22:41:59.444 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T22:41:59.444 DEBUG:teuthology.orchestra.run.smithi152:> sudo hostname $(hostname -s) 2024-09-11T22:41:59.472 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname $(hostname -s) 2024-09-11T22:41:59.498 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname $(hostname -s) 2024-09-11T22:41:59.524 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-11T22:41:59.525 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:41:59.676 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-11T22:41:59.823 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:41:59.825 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-11T22:41:59.825 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-11T22:41:59.825 DEBUG:teuthology.orchestra.run.smithi152:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T22:42:00.226 INFO:teuthology.orchestra.run.smithi152.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 22:42 /home/ubuntu/cephtest/cephadm 2024-09-11T22:42:00.226 DEBUG:teuthology.orchestra.run.smithi191:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T22:42:00.624 INFO:teuthology.orchestra.run.smithi191.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 22:42 /home/ubuntu/cephtest/cephadm 2024-09-11T22:42:00.624 DEBUG:teuthology.orchestra.run.smithi204:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T22:42:00.980 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 22:42 /home/ubuntu/cephtest/cephadm 2024-09-11T22:42:00.980 DEBUG:teuthology.orchestra.run.smithi152:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T22:42:01.001 DEBUG:teuthology.orchestra.run.smithi191:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T22:42:01.021 DEBUG:teuthology.orchestra.run.smithi204:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T22:42:01.058 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-11T22:42:01.058 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T22:42:01.061 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T22:42:01.063 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T22:42:02.597 INFO:teuthology.orchestra.run.smithi204.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T22:42:02.662 INFO:teuthology.orchestra.run.smithi191.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T22:42:02.748 INFO:teuthology.orchestra.run.smithi152.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T22:42:35.046 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-11T22:42:35.046 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T22:42:35.047 INFO:teuthology.orchestra.run.smithi204.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T22:42:35.047 INFO:teuthology.orchestra.run.smithi204.stdout: "repo_digests": [ 2024-09-11T22:42:35.047 INFO:teuthology.orchestra.run.smithi204.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T22:42:35.047 INFO:teuthology.orchestra.run.smithi204.stdout: ] 2024-09-11T22:42:35.047 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-11T22:42:48.264 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-11T22:42:48.265 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T22:42:48.265 INFO:teuthology.orchestra.run.smithi191.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T22:42:48.265 INFO:teuthology.orchestra.run.smithi191.stdout: "repo_digests": [ 2024-09-11T22:42:48.265 INFO:teuthology.orchestra.run.smithi191.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T22:42:48.265 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-09-11T22:42:48.265 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-11T22:42:50.542 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:42:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T22:42:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T22:42:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: "repo_digests": [ 2024-09-11T22:42:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T22:42:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-11T22:42:50.542 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:42:50.568 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /etc/ceph 2024-09-11T22:42:50.622 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph 2024-09-11T22:42:50.655 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph 2024-09-11T22:42:50.689 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 777 /etc/ceph 2024-09-11T22:42:50.714 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /etc/ceph 2024-09-11T22:42:50.742 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /etc/ceph 2024-09-11T22:42:50.770 INFO:tasks.cephadm:Writing seed config... 2024-09-11T22:42:50.771 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-11T22:42:50.772 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-11T22:42:50.772 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-11T22:42:50.772 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-11T22:42:50.772 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-11T22:42:50.772 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-11T22:42:50.772 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-11T22:42:50.772 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-11T22:42:50.772 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-11T22:42:50.772 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-11T22:42:50.772 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-11T22:42:50.772 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-11T22:42:50.773 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-11T22:42:50.773 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-11T22:42:50.773 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-11T22:42:50.773 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-11T22:42:50.773 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-11T22:42:50.773 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-11T22:42:50.773 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-11T22:42:50.773 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-11T22:42:50.773 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-11T22:42:50.773 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-11T22:42:50.773 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-11T22:42:50.773 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T22:42:50.773 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T22:42:50.774 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T22:42:50.774 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T22:42:50.774 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T22:42:50.774 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-11T22:42:50.774 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-11T22:42:50.774 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-11T22:42:50.774 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-11T22:42:50.774 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-11T22:42:50.774 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-11T22:42:50.774 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-11T22:42:50.774 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-11T22:42:50.774 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T22:42:50.774 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T22:42:50.774 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-11T22:42:50.775 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-11T22:42:50.775 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-11T22:42:50.775 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-11T22:42:50.775 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-11T22:42:50.775 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-11T22:42:50.775 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-11T22:42:50.775 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T22:42:50.775 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-11T22:42:50.776 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:42:50.776 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T22:42:50.791 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0e4e82bc-708f-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-11T22:42:50.791 DEBUG:teuthology.orchestra.run.smithi152:mon.a> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T22:42:50.834 DEBUG:teuthology.orchestra.run.smithi152:mgr.x> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T22:42:50.877 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T22:42:50.877 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.152 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:42:51.084 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-09-11T22:42:51.084 INFO:teuthology.orchestra.run.smithi152.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', '0e4e82bc-708f-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.152', '--skip-admin-label'] 2024-09-11T22:42:51.084 INFO:teuthology.orchestra.run.smithi152.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T22:42:51.085 INFO:teuthology.orchestra.run.smithi152.stdout:Verifying podman|docker is present... 2024-09-11T22:42:51.109 INFO:teuthology.orchestra.run.smithi152.stdout:/bin/podman: stdout 5.2.2 2024-09-11T22:42:51.109 INFO:teuthology.orchestra.run.smithi152.stdout:Verifying lvm2 is present... 2024-09-11T22:42:51.109 INFO:teuthology.orchestra.run.smithi152.stdout:Verifying time synchronization is in place... 2024-09-11T22:42:51.117 INFO:teuthology.orchestra.run.smithi152.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T22:42:51.117 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T22:42:51.123 INFO:teuthology.orchestra.run.smithi152.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T22:42:51.123 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stdout inactive 2024-09-11T22:42:51.130 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stdout enabled 2024-09-11T22:42:51.136 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stdout active 2024-09-11T22:42:51.136 INFO:teuthology.orchestra.run.smithi152.stdout:Unit chronyd.service is enabled and running 2024-09-11T22:42:51.136 INFO:teuthology.orchestra.run.smithi152.stdout:Repeating the final host check... 2024-09-11T22:42:51.163 INFO:teuthology.orchestra.run.smithi152.stdout:/bin/podman: stdout 5.2.2 2024-09-11T22:42:51.163 INFO:teuthology.orchestra.run.smithi152.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T22:42:51.164 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl is present 2024-09-11T22:42:51.164 INFO:teuthology.orchestra.run.smithi152.stdout:lvcreate is present 2024-09-11T22:42:51.170 INFO:teuthology.orchestra.run.smithi152.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T22:42:51.170 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T22:42:51.176 INFO:teuthology.orchestra.run.smithi152.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T22:42:51.176 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stdout inactive 2024-09-11T22:42:51.183 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stdout enabled 2024-09-11T22:42:51.190 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stdout active 2024-09-11T22:42:51.190 INFO:teuthology.orchestra.run.smithi152.stdout:Unit chronyd.service is enabled and running 2024-09-11T22:42:51.190 INFO:teuthology.orchestra.run.smithi152.stdout:Host looks OK 2024-09-11T22:42:51.190 INFO:teuthology.orchestra.run.smithi152.stdout:Cluster fsid: 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:42:51.190 INFO:teuthology.orchestra.run.smithi152.stdout:Acquiring lock 140179902718976 on /run/cephadm/0e4e82bc-708f-11ef-bcea-c7b262605968.lock 2024-09-11T22:42:51.190 INFO:teuthology.orchestra.run.smithi152.stdout:Lock 140179902718976 acquired on /run/cephadm/0e4e82bc-708f-11ef-bcea-c7b262605968.lock 2024-09-11T22:42:51.191 INFO:teuthology.orchestra.run.smithi152.stdout:Verifying IP 172.21.15.152 port 3300 ... 2024-09-11T22:42:51.191 INFO:teuthology.orchestra.run.smithi152.stdout:Verifying IP 172.21.15.152 port 6789 ... 2024-09-11T22:42:51.191 INFO:teuthology.orchestra.run.smithi152.stdout:Base mon IP(s) is [172.21.15.152:3300, 172.21.15.152:6789], mon addrv is [v2:172.21.15.152:3300,v1:172.21.15.152:6789] 2024-09-11T22:42:51.194 INFO:teuthology.orchestra.run.smithi152.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.152 metric 100 2024-09-11T22:42:51.194 INFO:teuthology.orchestra.run.smithi152.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.152 metric 100 2024-09-11T22:42:51.197 INFO:teuthology.orchestra.run.smithi152.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T22:42:51.197 INFO:teuthology.orchestra.run.smithi152.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-11T22:42:51.197 INFO:teuthology.orchestra.run.smithi152.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-11T22:42:51.200 INFO:teuthology.orchestra.run.smithi152.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T22:42:51.200 INFO:teuthology.orchestra.run.smithi152.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T22:42:51.200 INFO:teuthology.orchestra.run.smithi152.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T22:42:51.200 INFO:teuthology.orchestra.run.smithi152.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-11T22:42:51.200 INFO:teuthology.orchestra.run.smithi152.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:6de3/64 scope link noprefixroute 2024-09-11T22:42:51.200 INFO:teuthology.orchestra.run.smithi152.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T22:42:51.201 INFO:teuthology.orchestra.run.smithi152.stdout:Mon IP `172.21.15.152` is in CIDR network `172.21.0.0/20` 2024-09-11T22:42:51.201 INFO:teuthology.orchestra.run.smithi152.stdout:Mon IP `172.21.15.152` is in CIDR network `172.21.0.0/20` 2024-09-11T22:42:51.201 INFO:teuthology.orchestra.run.smithi152.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-11T22:42:51.201 INFO:teuthology.orchestra.run.smithi152.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T22:42:51.202 INFO:teuthology.orchestra.run.smithi152.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T22:42:51.704 INFO:teuthology.orchestra.run.smithi152.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-11T22:42:51.704 INFO:teuthology.orchestra.run.smithi152.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T22:42:51.704 INFO:teuthology.orchestra.run.smithi152.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T22:42:51.704 INFO:teuthology.orchestra.run.smithi152.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-11T22:42:51.704 INFO:teuthology.orchestra.run.smithi152.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T22:42:51.704 INFO:teuthology.orchestra.run.smithi152.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-11T22:42:51.704 INFO:teuthology.orchestra.run.smithi152.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T22:42:52.700 INFO:teuthology.orchestra.run.smithi152.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-11T22:42:52.701 INFO:teuthology.orchestra.run.smithi152.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-11T22:42:52.701 INFO:teuthology.orchestra.run.smithi152.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T22:42:53.689 INFO:teuthology.orchestra.run.smithi152.stdout:stat: stdout 167 167 2024-09-11T22:42:53.690 INFO:teuthology.orchestra.run.smithi152.stdout:Creating initial keys... 2024-09-11T22:42:54.904 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph-authtool: stdout AQDuHOJmW/peABAANNcQCRloFBRumK7v+qJCRQ== 2024-09-11T22:42:55.918 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph-authtool: stdout AQDvHOJmhRomDRAAUp6nCm/NZaW0uyPFVQJkpg== 2024-09-11T22:42:56.983 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph-authtool: stdout AQDwHOJm17WYERAAYosGw5LpBL7j3jWOn7V5Cg== 2024-09-11T22:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:Creating initial monmap... 2024-09-11T22:42:58.057 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T22:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-11T22:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T22:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:monmaptool for a [v2:172.21.15.152:3300,v1:172.21.15.152:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T22:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:setting min_mon_release = quincy 2024-09-11T22:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/monmaptool: set fsid to 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T22:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:Creating mon... 2024-09-11T22:42:58.996 INFO:teuthology.orchestra.run.smithi152.stdout:create mon.a on 2024-09-11T22:42:59.200 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T22:42:59.353 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T22:42:59.532 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0e4e82bc-708f-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-0e4e82bc-708f-11ef-bcea-c7b262605968.target. 2024-09-11T22:42:59.532 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0e4e82bc-708f-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-0e4e82bc-708f-11ef-bcea-c7b262605968.target. 2024-09-11T22:42:59.770 INFO:teuthology.orchestra.run.smithi152.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mon.a 2024-09-11T22:42:59.771 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T22:42:59.932 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0e4e82bc-708f-11ef-bcea-c7b262605968.target.wants/ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@.service. 2024-09-11T22:43:00.413 INFO:teuthology.orchestra.run.smithi152.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T22:43:00.413 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stdout disabled 2024-09-11T22:43:00.420 INFO:teuthology.orchestra.run.smithi152.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T22:43:00.420 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stdout inactive 2024-09-11T22:43:00.420 INFO:teuthology.orchestra.run.smithi152.stdout:firewalld.service is not enabled 2024-09-11T22:43:00.421 INFO:teuthology.orchestra.run.smithi152.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T22:43:00.421 INFO:teuthology.orchestra.run.smithi152.stdout:Waiting for mon to start... 2024-09-11T22:43:00.421 INFO:teuthology.orchestra.run.smithi152.stdout:Waiting for mon... 2024-09-11T22:43:00.544 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:00 smithi152 systemd[1]: Started Ceph mon.a for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:43:01.489 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:01 smithi152 ceph-mon[30812]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T22:43:01.489 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:01 smithi152 ceph-mon[30812]: monmap epoch 1 2024-09-11T22:43:01.489 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:01 smithi152 ceph-mon[30812]: fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:43:01.489 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:01 smithi152 ceph-mon[30812]: last_changed 2024-09-11T22:42:57.315021+0000 2024-09-11T22:43:01.489 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:01 smithi152 ceph-mon[30812]: created 2024-09-11T22:42:57.315021+0000 2024-09-11T22:43:01.490 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:01 smithi152 ceph-mon[30812]: min_mon_release 19 (squid) 2024-09-11T22:43:01.490 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:01 smithi152 ceph-mon[30812]: election_strategy: 1 2024-09-11T22:43:01.490 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:01 smithi152 ceph-mon[30812]: 0: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.a 2024-09-11T22:43:01.490 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:01 smithi152 ceph-mon[30812]: fsmap 2024-09-11T22:43:01.490 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:01 smithi152 ceph-mon[30812]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T22:43:01.490 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:01 smithi152 ceph-mon[30812]: mgrmap e1: no daemons active 2024-09-11T22:43:01.490 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:01 smithi152 ceph-mon[30812]: from='client.? 172.21.15.152:0/3383976122' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T22:43:01.496 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T22:43:01.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout id: 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:43:01.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T22:43:01.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout 2024-09-11T22:43:01.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout services: 2024-09-11T22:43:01.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.551176s) 2024-09-11T22:43:01.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T22:43:01.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T22:43:01.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout 2024-09-11T22:43:01.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout data: 2024-09-11T22:43:01.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T22:43:01.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T22:43:01.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T22:43:01.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T22:43:01.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout 2024-09-11T22:43:01.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.861+0000 7f6bdafb9640 1 Processor -- start 2024-09-11T22:43:01.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.861+0000 7f6bdafb9640 1 -- start start 2024-09-11T22:43:01.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.862+0000 7f6bdafb9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd4107620 0x7f6bd4107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:01.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.862+0000 7f6bdafb9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd4107ff0 con 0x7f6bd4107620 2024-09-11T22:43:01.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.862+0000 7f6bd8d55640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd4107620 0x7f6bd4107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:01.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.862+0000 7f6bd8d55640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd4107620 0x7f6bd4107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41586/0 (socket says 172.21.15.152:41586) 2024-09-11T22:43:01.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.862+0000 7f6bd8d55640 1 -- 172.21.15.152:0/1825944702 learned_addr learned my addr 172.21.15.152:0/1825944702 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:01.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.862+0000 7f6bd8d55640 1 -- 172.21.15.152:0/1825944702 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd4108820 con 0x7f6bd4107620 2024-09-11T22:43:01.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.864+0000 7f6bd8d55640 1 --2- 172.21.15.152:0/1825944702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd4107620 0x7f6bd4107a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009820 tx=0x7f6bc402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a2157059d4268f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:01.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.864+0000 7f6bd37fe640 1 -- 172.21.15.152:0/1825944702 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bc4037070 con 0x7f6bd4107620 2024-09-11T22:43:01.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.864+0000 7f6bd37fe640 1 -- 172.21.15.152:0/1825944702 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc4032440 con 0x7f6bd4107620 2024-09-11T22:43:01.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.864+0000 7f6bd37fe640 1 -- 172.21.15.152:0/1825944702 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bc4032740 con 0x7f6bd4107620 2024-09-11T22:43:01.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.865+0000 7f6bdafb9640 1 -- 172.21.15.152:0/1825944702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd4107620 msgr2=0x7f6bd4107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:01.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.865+0000 7f6bdafb9640 1 --2- 172.21.15.152:0/1825944702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd4107620 0x7f6bd4107a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009820 tx=0x7f6bc402a270 comp rx=0 tx=0).stop 2024-09-11T22:43:01.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.865+0000 7f6bdafb9640 1 -- 172.21.15.152:0/1825944702 shutdown_connections 2024-09-11T22:43:01.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.865+0000 7f6bdafb9640 1 --2- 172.21.15.152:0/1825944702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd4107620 0x7f6bd4107a20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:01.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.865+0000 7f6bdafb9640 1 -- 172.21.15.152:0/1825944702 >> 172.21.15.152:0/1825944702 conn(0x7f6bd4102db0 msgr2=0x7f6bd41051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:01.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.865+0000 7f6bdafb9640 1 -- 172.21.15.152:0/1825944702 shutdown_connections 2024-09-11T22:43:01.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.865+0000 7f6bdafb9640 1 -- 172.21.15.152:0/1825944702 wait complete. 2024-09-11T22:43:01.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.866+0000 7f6bdafb9640 1 Processor -- start 2024-09-11T22:43:01.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.866+0000 7f6bdafb9640 1 -- start start 2024-09-11T22:43:01.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.866+0000 7f6bdafb9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd4107620 0x7f6bd419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:01.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.866+0000 7f6bdafb9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd4108dc0 con 0x7f6bd4107620 2024-09-11T22:43:01.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.866+0000 7f6bd8d55640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd4107620 0x7f6bd419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:01.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.866+0000 7f6bd8d55640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd4107620 0x7f6bd419adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41598/0 (socket says 172.21.15.152:41598) 2024-09-11T22:43:01.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.866+0000 7f6bd8d55640 1 -- 172.21.15.152:0/3383976122 learned_addr learned my addr 172.21.15.152:0/3383976122 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:01.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.866+0000 7f6bd8d55640 1 -- 172.21.15.152:0/3383976122 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd419b300 con 0x7f6bd4107620 2024-09-11T22:43:01.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.866+0000 7f6bd8d55640 1 --2- 172.21.15.152:0/3383976122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd4107620 0x7f6bd419adc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40090b0 tx=0x7f6bc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:01.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.866+0000 7f6bd1ffb640 1 -- 172.21.15.152:0/3383976122 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bc4041070 con 0x7f6bd4107620 2024-09-11T22:43:01.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.867+0000 7f6bd1ffb640 1 -- 172.21.15.152:0/3383976122 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc403c7a0 con 0x7f6bd4107620 2024-09-11T22:43:01.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.867+0000 7f6bdafb9640 1 -- 172.21.15.152:0/3383976122 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd419b590 con 0x7f6bd4107620 2024-09-11T22:43:01.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.867+0000 7f6bd1ffb640 1 -- 172.21.15.152:0/3383976122 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bc4037040 con 0x7f6bd4107620 2024-09-11T22:43:01.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.867+0000 7f6bdafb9640 1 -- 172.21.15.152:0/3383976122 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd419e280 con 0x7f6bd4107620 2024-09-11T22:43:01.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.867+0000 7f6bd1ffb640 1 -- 172.21.15.152:0/3383976122 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6bc4044070 con 0x7f6bd4107620 2024-09-11T22:43:01.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.867+0000 7f6bd1ffb640 1 -- 172.21.15.152:0/3383976122 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6bc403d3f0 con 0x7f6bd4107620 2024-09-11T22:43:01.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.868+0000 7f6bdafb9640 1 -- 172.21.15.152:0/3383976122 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd4109410 con 0x7f6bd4107620 2024-09-11T22:43:01.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.870+0000 7f6bd1ffb640 1 -- 172.21.15.152:0/3383976122 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6bc4046450 con 0x7f6bd4107620 2024-09-11T22:43:01.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.928+0000 7f6bdafb9640 1 -- 172.21.15.152:0/3383976122 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f6bd419e7d0 con 0x7f6bd4107620 2024-09-11T22:43:01.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.929+0000 7f6bd1ffb640 1 -- 172.21.15.152:0/3383976122 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f6bc4046630 con 0x7f6bd4107620 2024-09-11T22:43:01.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.929+0000 7f6bdafb9640 1 -- 172.21.15.152:0/3383976122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd4107620 msgr2=0x7f6bd419adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:01.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.929+0000 7f6bdafb9640 1 --2- 172.21.15.152:0/3383976122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd4107620 0x7f6bd419adc0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40090b0 tx=0x7f6bc40046c0 comp rx=0 tx=0).stop 2024-09-11T22:43:01.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.929+0000 7f6bdafb9640 1 -- 172.21.15.152:0/3383976122 shutdown_connections 2024-09-11T22:43:01.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.929+0000 7f6bdafb9640 1 --2- 172.21.15.152:0/3383976122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd4107620 0x7f6bd419adc0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:01.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.929+0000 7f6bdafb9640 1 -- 172.21.15.152:0/3383976122 >> 172.21.15.152:0/3383976122 conn(0x7f6bd4102db0 msgr2=0x7f6bd418f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:01.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.929+0000 7f6bdafb9640 1 -- 172.21.15.152:0/3383976122 shutdown_connections 2024-09-11T22:43:01.502 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:00.930+0000 7f6bdafb9640 1 -- 172.21.15.152:0/3383976122 wait complete. 2024-09-11T22:43:01.502 INFO:teuthology.orchestra.run.smithi152.stdout:mon is available 2024-09-11T22:43:01.502 INFO:teuthology.orchestra.run.smithi152.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T22:43:02.488 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:02 smithi152 ceph-mon[30812]: from='client.? 172.21.15.152:0/2232791780' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T22:43:02.489 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:02 smithi152 ceph-mon[30812]: from='client.? 172.21.15.152:0/2232791780' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T22:43:02.496 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout 2024-09-11T22:43:02.496 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T22:43:02.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout fsid = 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:43:02.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.152:3300,v1:172.21.15.152:6789] 2024-09-11T22:43:02.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T22:43:02.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T22:43:02.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T22:43:02.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T22:43:02.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout 2024-09-11T22:43:02.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T22:43:02.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T22:43:02.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout 2024-09-11T22:43:02.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T22:43:02.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T22:43:02.497 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T22:43:02.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T22:43:02.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T22:43:02.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T22:43:02.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.894+0000 7f9f85347640 1 Processor -- start 2024-09-11T22:43:02.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.894+0000 7f9f85347640 1 -- start start 2024-09-11T22:43:02.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.895+0000 7f9f85347640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f80108080 0x7f9f8010a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:02.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.895+0000 7f9f85347640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8007be30 con 0x7f9f80108080 2024-09-11T22:43:02.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.895+0000 7f9f7effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f80108080 0x7f9f8010a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:02.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.895+0000 7f9f7effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f80108080 0x7f9f8010a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41612/0 (socket says 172.21.15.152:41612) 2024-09-11T22:43:02.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.895+0000 7f9f7effd640 1 -- 172.21.15.152:0/2986017172 learned_addr learned my addr 172.21.15.152:0/2986017172 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:02.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.895+0000 7f9f7effd640 1 -- 172.21.15.152:0/2986017172 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8010a9b0 con 0x7f9f80108080 2024-09-11T22:43:02.498 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.896+0000 7f9f7effd640 1 --2- 172.21.15.152:0/2986017172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f80108080 0x7f9f8010a470 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009820 tx=0x7f9f7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afe85ec8c1c1fcc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:02.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.896+0000 7f9f7dffb640 1 -- 172.21.15.152:0/2986017172 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f70037070 con 0x7f9f80108080 2024-09-11T22:43:02.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.896+0000 7f9f7dffb640 1 -- 172.21.15.152:0/2986017172 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f70032440 con 0x7f9f80108080 2024-09-11T22:43:02.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.896+0000 7f9f7dffb640 1 -- 172.21.15.152:0/2986017172 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f70032740 con 0x7f9f80108080 2024-09-11T22:43:02.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.896+0000 7f9f85347640 1 -- 172.21.15.152:0/2986017172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f80108080 msgr2=0x7f9f8010a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:02.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.896+0000 7f9f85347640 1 --2- 172.21.15.152:0/2986017172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f80108080 0x7f9f8010a470 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009820 tx=0x7f9f7002a270 comp rx=0 tx=0).stop 2024-09-11T22:43:02.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.896+0000 7f9f85347640 1 -- 172.21.15.152:0/2986017172 shutdown_connections 2024-09-11T22:43:02.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.896+0000 7f9f85347640 1 --2- 172.21.15.152:0/2986017172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f80108080 0x7f9f8010a470 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:02.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.896+0000 7f9f85347640 1 -- 172.21.15.152:0/2986017172 >> 172.21.15.152:0/2986017172 conn(0x7f9f80101ca0 msgr2=0x7f9f801040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:02.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.897+0000 7f9f85347640 1 -- 172.21.15.152:0/2986017172 shutdown_connections 2024-09-11T22:43:02.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.897+0000 7f9f85347640 1 -- 172.21.15.152:0/2986017172 wait complete. 2024-09-11T22:43:02.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.897+0000 7f9f85347640 1 Processor -- start 2024-09-11T22:43:02.499 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.897+0000 7f9f85347640 1 -- start start 2024-09-11T22:43:02.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.897+0000 7f9f85347640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f80108080 0x7f9f80107900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:02.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.897+0000 7f9f85347640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8010ae90 con 0x7f9f80108080 2024-09-11T22:43:02.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.898+0000 7f9f7effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f80108080 0x7f9f80107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:02.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.898+0000 7f9f7effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f80108080 0x7f9f80107900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41622/0 (socket says 172.21.15.152:41622) 2024-09-11T22:43:02.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.898+0000 7f9f7effd640 1 -- 172.21.15.152:0/2232791780 learned_addr learned my addr 172.21.15.152:0/2232791780 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:02.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.898+0000 7f9f7effd640 1 -- 172.21.15.152:0/2232791780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f80107e40 con 0x7f9f80108080 2024-09-11T22:43:02.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.898+0000 7f9f7effd640 1 --2- 172.21.15.152:0/2232791780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f80108080 0x7f9f80107900 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9f700090b0 tx=0x7f9f700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:02.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.898+0000 7f9f67fff640 1 -- 172.21.15.152:0/2232791780 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f70041070 con 0x7f9f80108080 2024-09-11T22:43:02.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.898+0000 7f9f67fff640 1 -- 172.21.15.152:0/2232791780 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f7003c7a0 con 0x7f9f80108080 2024-09-11T22:43:02.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.898+0000 7f9f67fff640 1 -- 172.21.15.152:0/2232791780 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f70037040 con 0x7f9f80108080 2024-09-11T22:43:02.500 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.898+0000 7f9f85347640 1 -- 172.21.15.152:0/2232791780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f80106030 con 0x7f9f80108080 2024-09-11T22:43:02.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.898+0000 7f9f85347640 1 -- 172.21.15.152:0/2232791780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f80106570 con 0x7f9f80108080 2024-09-11T22:43:02.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.898+0000 7f9f67fff640 1 -- 172.21.15.152:0/2232791780 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9f70044070 con 0x7f9f80108080 2024-09-11T22:43:02.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.898+0000 7f9f67fff640 1 -- 172.21.15.152:0/2232791780 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9f70046dc0 con 0x7f9f80108080 2024-09-11T22:43:02.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.899+0000 7f9f85347640 1 -- 172.21.15.152:0/2232791780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f801084e0 con 0x7f9f80108080 2024-09-11T22:43:02.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.901+0000 7f9f67fff640 1 -- 172.21.15.152:0/2232791780 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9f70046450 con 0x7f9f80108080 2024-09-11T22:43:02.501 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.957+0000 7f9f85347640 1 -- 172.21.15.152:0/2232791780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f9f801946f0 con 0x7f9f80108080 2024-09-11T22:43:02.502 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.960+0000 7f9f67fff640 1 -- 172.21.15.152:0/2232791780 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f9f7003caa0 con 0x7f9f80108080 2024-09-11T22:43:02.502 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.961+0000 7f9f67fff640 1 -- 172.21.15.152:0/2232791780 <== mon.0 v2:172.21.15.152:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9f70032440 con 0x7f9f80108080 2024-09-11T22:43:02.502 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.962+0000 7f9f85347640 1 -- 172.21.15.152:0/2232791780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f80108080 msgr2=0x7f9f80107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:02.502 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.962+0000 7f9f85347640 1 --2- 172.21.15.152:0/2232791780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f80108080 0x7f9f80107900 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9f700090b0 tx=0x7f9f700046c0 comp rx=0 tx=0).stop 2024-09-11T22:43:02.502 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.962+0000 7f9f85347640 1 -- 172.21.15.152:0/2232791780 shutdown_connections 2024-09-11T22:43:02.502 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.962+0000 7f9f85347640 1 --2- 172.21.15.152:0/2232791780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9f80108080 0x7f9f80107900 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:02.502 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.962+0000 7f9f85347640 1 -- 172.21.15.152:0/2232791780 >> 172.21.15.152:0/2232791780 conn(0x7f9f80101ca0 msgr2=0x7f9f80102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:02.502 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.962+0000 7f9f85347640 1 -- 172.21.15.152:0/2232791780 shutdown_connections 2024-09-11T22:43:02.502 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:01.962+0000 7f9f85347640 1 -- 172.21.15.152:0/2232791780 wait complete. 2024-09-11T22:43:02.502 INFO:teuthology.orchestra.run.smithi152.stdout:Generating new minimal ceph.conf... 2024-09-11T22:43:03.578 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:03 smithi152 ceph-mon[30812]: from='client.? 172.21.15.152:0/2042753828' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:43:03.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.901+0000 7f8b7db0c640 1 Processor -- start 2024-09-11T22:43:03.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.901+0000 7f8b7db0c640 1 -- start start 2024-09-11T22:43:03.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.901+0000 7f8b7db0c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b78107620 0x7f8b78107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:03.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.901+0000 7f8b7db0c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8b78107ff0 con 0x7f8b78107620 2024-09-11T22:43:03.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.901+0000 7f8b777fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b78107620 0x7f8b78107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:03.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.901+0000 7f8b777fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b78107620 0x7f8b78107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41630/0 (socket says 172.21.15.152:41630) 2024-09-11T22:43:03.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.901+0000 7f8b777fe640 1 -- 172.21.15.152:0/492707287 learned_addr learned my addr 172.21.15.152:0/492707287 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:03.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.902+0000 7f8b777fe640 1 -- 172.21.15.152:0/492707287 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b78108820 con 0x7f8b78107620 2024-09-11T22:43:03.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.902+0000 7f8b777fe640 1 --2- 172.21.15.152:0/492707287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b78107620 0x7f8b78107a20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009e80 tx=0x7f8b6802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c5deef9649c866 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:03.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.902+0000 7f8b767fc640 1 -- 172.21.15.152:0/492707287 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b68036070 con 0x7f8b78107620 2024-09-11T22:43:03.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.903+0000 7f8b767fc640 1 -- 172.21.15.152:0/492707287 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8b680318c0 con 0x7f8b78107620 2024-09-11T22:43:03.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.903+0000 7f8b767fc640 1 -- 172.21.15.152:0/492707287 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b68007ab0 con 0x7f8b78107620 2024-09-11T22:43:03.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.903+0000 7f8b7db0c640 1 -- 172.21.15.152:0/492707287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b78107620 msgr2=0x7f8b78107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:03.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.903+0000 7f8b7db0c640 1 --2- 172.21.15.152:0/492707287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b78107620 0x7f8b78107a20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009e80 tx=0x7f8b6802a7a0 comp rx=0 tx=0).stop 2024-09-11T22:43:03.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.903+0000 7f8b7db0c640 1 -- 172.21.15.152:0/492707287 shutdown_connections 2024-09-11T22:43:03.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.903+0000 7f8b7db0c640 1 --2- 172.21.15.152:0/492707287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b78107620 0x7f8b78107a20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:03.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.903+0000 7f8b7db0c640 1 -- 172.21.15.152:0/492707287 >> 172.21.15.152:0/492707287 conn(0x7f8b78102db0 msgr2=0x7f8b781051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:03.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.903+0000 7f8b7db0c640 1 -- 172.21.15.152:0/492707287 shutdown_connections 2024-09-11T22:43:03.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.903+0000 7f8b7db0c640 1 -- 172.21.15.152:0/492707287 wait complete. 2024-09-11T22:43:03.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.904+0000 7f8b7db0c640 1 Processor -- start 2024-09-11T22:43:03.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.904+0000 7f8b7db0c640 1 -- start start 2024-09-11T22:43:03.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.904+0000 7f8b7db0c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b78107620 0x7f8b7819b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:03.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.904+0000 7f8b7db0c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7810bbd0 con 0x7f8b78107620 2024-09-11T22:43:03.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.904+0000 7f8b777fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b78107620 0x7f8b7819b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:03.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.904+0000 7f8b777fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b78107620 0x7f8b7819b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41632/0 (socket says 172.21.15.152:41632) 2024-09-11T22:43:03.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.904+0000 7f8b777fe640 1 -- 172.21.15.152:0/2042753828 learned_addr learned my addr 172.21.15.152:0/2042753828 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:03.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.904+0000 7f8b777fe640 1 -- 172.21.15.152:0/2042753828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7819bb90 con 0x7f8b78107620 2024-09-11T22:43:03.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.905+0000 7f8b777fe640 1 --2- 172.21.15.152:0/2042753828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b78107620 0x7f8b7819b650 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8b68007ce0 tx=0x7f8b68031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:03.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.905+0000 7f8b74ff9640 1 -- 172.21.15.152:0/2042753828 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b68036050 con 0x7f8b78107620 2024-09-11T22:43:03.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.905+0000 7f8b74ff9640 1 -- 172.21.15.152:0/2042753828 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8b68042070 con 0x7f8b78107620 2024-09-11T22:43:03.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.905+0000 7f8b7db0c640 1 -- 172.21.15.152:0/2042753828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7819be20 con 0x7f8b78107620 2024-09-11T22:43:03.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.905+0000 7f8b74ff9640 1 -- 172.21.15.152:0/2042753828 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b6803d8a0 con 0x7f8b78107620 2024-09-11T22:43:03.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.905+0000 7f8b7db0c640 1 -- 172.21.15.152:0/2042753828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b7819eb10 con 0x7f8b78107620 2024-09-11T22:43:03.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.905+0000 7f8b74ff9640 1 -- 172.21.15.152:0/2042753828 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8b6803dba0 con 0x7f8b78107620 2024-09-11T22:43:03.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.905+0000 7f8b74ff9640 1 -- 172.21.15.152:0/2042753828 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8b68007e00 con 0x7f8b78107620 2024-09-11T22:43:03.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.906+0000 7f8b7db0c640 1 -- 172.21.15.152:0/2042753828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b7810bd50 con 0x7f8b78107620 2024-09-11T22:43:03.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.908+0000 7f8b74ff9640 1 -- 172.21.15.152:0/2042753828 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f8b68007310 con 0x7f8b78107620 2024-09-11T22:43:03.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.965+0000 7f8b7db0c640 1 -- 172.21.15.152:0/2042753828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f8b78107aa0 con 0x7f8b78107620 2024-09-11T22:43:03.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.965+0000 7f8b74ff9640 1 -- 172.21.15.152:0/2042753828 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f8b78107aa0 con 0x7f8b78107620 2024-09-11T22:43:03.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.966+0000 7f8b7db0c640 1 -- 172.21.15.152:0/2042753828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b78107620 msgr2=0x7f8b7819b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:03.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.966+0000 7f8b7db0c640 1 --2- 172.21.15.152:0/2042753828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b78107620 0x7f8b7819b650 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8b68007ce0 tx=0x7f8b68031ca0 comp rx=0 tx=0).stop 2024-09-11T22:43:03.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.966+0000 7f8b7db0c640 1 -- 172.21.15.152:0/2042753828 shutdown_connections 2024-09-11T22:43:03.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.966+0000 7f8b7db0c640 1 --2- 172.21.15.152:0/2042753828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b78107620 0x7f8b7819b650 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:03.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.966+0000 7f8b7db0c640 1 -- 172.21.15.152:0/2042753828 >> 172.21.15.152:0/2042753828 conn(0x7f8b78102db0 msgr2=0x7f8b78103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:03.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.966+0000 7f8b7db0c640 1 -- 172.21.15.152:0/2042753828 shutdown_connections 2024-09-11T22:43:03.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:02.966+0000 7f8b7db0c640 1 -- 172.21.15.152:0/2042753828 wait complete. 2024-09-11T22:43:03.588 INFO:teuthology.orchestra.run.smithi152.stdout:Restarting the monitor... 2024-09-11T22:43:03.832 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:03 smithi152 systemd[1]: Stopping Ceph mon.a for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T22:43:04.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:03 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-a[30808]: 2024-09-11T22:43:03.831+0000 7f8f17594640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:43:04.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:03 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-a[30808]: 2024-09-11T22:43:03.831+0000 7f8f17594640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T22:43:04.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:03 smithi152 podman[31054]: 2024-09-11 22:43:03.923829804 +0000 UTC m=+0.182237824 container died a2e3b98f8298513789768dbcb23cfca1ef73d10c4d76b5d706980cd313b7f535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-11T22:43:04.433 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:04 smithi152 podman[31054]: 2024-09-11 22:43:04.222778774 +0000 UTC m=+0.481186793 container cleanup a2e3b98f8298513789768dbcb23cfca1ef73d10c4d76b5d706980cd313b7f535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-11T22:43:04.433 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:04 smithi152 bash[31054]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-a 2024-09-11T22:43:04.773 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:04 smithi152 podman[31066]: 2024-09-11 22:43:04.432365352 +0000 UTC m=+0.508055900 container remove a2e3b98f8298513789768dbcb23cfca1ef73d10c4d76b5d706980cd313b7f535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-11T22:43:04.774 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:04 smithi152 systemd[1]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T22:43:04.774 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:04 smithi152 systemd[1]: Stopped Ceph mon.a for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:43:04.774 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:04 smithi152 systemd[1]: Starting Ceph mon.a for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T22:43:05.024 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:04 smithi152 podman[31161]: 2024-09-11 22:43:04.675870546 +0000 UTC m=+0.016118066 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:43:05.024 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:04 smithi152 podman[31161]: 2024-09-11 22:43:04.774812941 +0000 UTC m=+0.115060446 container create 316177fd5605dab938c8f1eb25560984f1f6f171aed29416c26038ec7f66c86d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T22:43:05.024 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:04 smithi152 podman[31161]: 2024-09-11 22:43:04.994144301 +0000 UTC m=+0.334391807 container init 316177fd5605dab938c8f1eb25560984f1f6f171aed29416c26038ec7f66c86d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3) 2024-09-11T22:43:05.024 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:04 smithi152 podman[31161]: 2024-09-11 22:43:04.997405129 +0000 UTC m=+0.337652636 container start 316177fd5605dab938c8f1eb25560984f1f6f171aed29416c26038ec7f66c86d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T22:43:05.024 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T22:43:05.025 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-11T22:43:05.025 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: pidfile_write: ignore empty --pid-file 2024-09-11T22:43:05.025 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: load: jerasure load: lrc 2024-09-11T22:43:05.025 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: RocksDB version: 7.9.2 2024-09-11T22:43:05.025 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Git sha 0 2024-09-11T22:43:05.025 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-11T22:43:05.025 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: DB SUMMARY 2024-09-11T22:43:05.026 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: DB Session ID: 6ZP3BGM7X24JBEEE97PT 2024-09-11T22:43:05.026 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: CURRENT file: CURRENT 2024-09-11T22:43:05.026 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T22:43:05.026 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-11T22:43:05.026 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T22:43:05.026 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105298 ; 2024-09-11T22:43:05.026 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.error_if_exists: 0 2024-09-11T22:43:05.029 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.create_if_missing: 0 2024-09-11T22:43:05.029 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.paranoid_checks: 1 2024-09-11T22:43:05.029 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-11T22:43:05.029 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T22:43:05.029 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-11T22:43:05.030 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.env: 0x55d02d5e4940 2024-09-11T22:43:05.030 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.fs: PosixFileSystem 2024-09-11T22:43:05.030 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.info_log: 0x55d02fd656e0 2024-09-11T22:43:05.030 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T22:43:05.030 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.statistics: (nil) 2024-09-11T22:43:05.030 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.use_fsync: 0 2024-09-11T22:43:05.030 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_log_file_size: 0 2024-09-11T22:43:05.030 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T22:43:05.031 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T22:43:05.031 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T22:43:05.031 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T22:43:05.031 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.allow_fallocate: 1 2024-09-11T22:43:05.031 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T22:43:05.031 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T22:43:05.032 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.use_direct_reads: 0 2024-09-11T22:43:05.032 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T22:43:05.032 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T22:43:05.032 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.db_log_dir: 2024-09-11T22:43:05.032 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.wal_dir: 2024-09-11T22:43:05.032 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T22:43:05.032 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T22:43:05.032 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T22:43:05.032 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T22:43:05.033 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T22:43:05.033 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T22:43:05.033 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T22:43:05.033 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T22:43:05.033 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.write_buffer_manager: 0x55d02fd74dc0 2024-09-11T22:43:05.033 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T22:43:05.033 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T22:43:05.033 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T22:43:05.033 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T22:43:05.034 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T22:43:05.034 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T22:43:05.034 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T22:43:05.034 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T22:43:05.034 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.unordered_write: 0 2024-09-11T22:43:05.034 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T22:43:05.034 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T22:43:05.035 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T22:43:05.035 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T22:43:05.035 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.row_cache: None 2024-09-11T22:43:05.035 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.wal_filter: None 2024-09-11T22:43:05.035 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T22:43:05.035 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T22:43:05.035 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.two_write_queues: 0 2024-09-11T22:43:05.035 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T22:43:05.035 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.wal_compression: 0 2024-09-11T22:43:05.035 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.atomic_flush: 0 2024-09-11T22:43:05.035 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T22:43:05.035 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T22:43:05.036 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T22:43:05.036 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.log_readahead_size: 0 2024-09-11T22:43:05.036 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T22:43:05.036 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T22:43:05.036 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T22:43:05.036 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T22:43:05.036 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T22:43:05.036 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T22:43:05.036 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-11T22:43:05.036 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_background_jobs: 2 2024-09-11T22:43:05.036 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_background_compactions: -1 2024-09-11T22:43:05.036 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_subcompactions: 1 2024-09-11T22:43:05.037 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T22:43:05.037 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T22:43:05.037 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T22:43:05.037 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T22:43:05.037 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T22:43:05.037 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T22:43:05.037 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T22:43:05.037 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T22:43:05.037 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_open_files: -1 2024-09-11T22:43:05.037 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T22:43:05.037 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T22:43:05.038 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T22:43:05.038 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T22:43:05.038 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_background_flushes: -1 2024-09-11T22:43:05.038 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Compression algorithms supported: 2024-09-11T22:43:05.038 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: kZSTD supported: 0 2024-09-11T22:43:05.038 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: kXpressCompression supported: 0 2024-09-11T22:43:05.038 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: kBZip2Compression supported: 0 2024-09-11T22:43:05.038 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T22:43:05.038 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: kLZ4Compression supported: 1 2024-09-11T22:43:05.038 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: kZlibCompression supported: 1 2024-09-11T22:43:05.038 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T22:43:05.038 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: kSnappyCompression supported: 1 2024-09-11T22:43:05.038 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T22:43:05.039 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-11T22:43:05.039 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-11T22:43:05.039 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-11T22:43:05.039 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T22:43:05.039 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.merge_operator: 2024-09-11T22:43:05.039 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compaction_filter: None 2024-09-11T22:43:05.039 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T22:43:05.039 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T22:43:05.039 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T22:43:05.039 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T22:43:05.039 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d02fd65320) 2024-09-11T22:43:05.039 INFO:journalctl@ceph.mon.a.smithi152.stdout: cache_index_and_filter_blocks: 1 2024-09-11T22:43:05.039 INFO:journalctl@ceph.mon.a.smithi152.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T22:43:05.039 INFO:journalctl@ceph.mon.a.smithi152.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T22:43:05.040 INFO:journalctl@ceph.mon.a.smithi152.stdout: pin_top_level_index_and_filter: 1 2024-09-11T22:43:05.040 INFO:journalctl@ceph.mon.a.smithi152.stdout: index_type: 0 2024-09-11T22:43:05.040 INFO:journalctl@ceph.mon.a.smithi152.stdout: data_block_index_type: 0 2024-09-11T22:43:05.040 INFO:journalctl@ceph.mon.a.smithi152.stdout: index_shortening: 1 2024-09-11T22:43:05.040 INFO:journalctl@ceph.mon.a.smithi152.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T22:43:05.040 INFO:journalctl@ceph.mon.a.smithi152.stdout: checksum: 4 2024-09-11T22:43:05.040 INFO:journalctl@ceph.mon.a.smithi152.stdout: no_block_cache: 0 2024-09-11T22:43:05.040 INFO:journalctl@ceph.mon.a.smithi152.stdout: block_cache: 0x55d02fd591f0 2024-09-11T22:43:05.040 INFO:journalctl@ceph.mon.a.smithi152.stdout: block_cache_name: BinnedLRUCache 2024-09-11T22:43:05.040 INFO:journalctl@ceph.mon.a.smithi152.stdout: block_cache_options: 2024-09-11T22:43:05.040 INFO:journalctl@ceph.mon.a.smithi152.stdout: capacity : 536870912 2024-09-11T22:43:05.040 INFO:journalctl@ceph.mon.a.smithi152.stdout: num_shard_bits : 4 2024-09-11T22:43:05.041 INFO:journalctl@ceph.mon.a.smithi152.stdout: strict_capacity_limit : 0 2024-09-11T22:43:05.041 INFO:journalctl@ceph.mon.a.smithi152.stdout: high_pri_pool_ratio: 0.000 2024-09-11T22:43:05.041 INFO:journalctl@ceph.mon.a.smithi152.stdout: block_cache_compressed: (nil) 2024-09-11T22:43:05.041 INFO:journalctl@ceph.mon.a.smithi152.stdout: persistent_cache: (nil) 2024-09-11T22:43:05.041 INFO:journalctl@ceph.mon.a.smithi152.stdout: block_size: 4096 2024-09-11T22:43:05.041 INFO:journalctl@ceph.mon.a.smithi152.stdout: block_size_deviation: 10 2024-09-11T22:43:05.041 INFO:journalctl@ceph.mon.a.smithi152.stdout: block_restart_interval: 16 2024-09-11T22:43:05.041 INFO:journalctl@ceph.mon.a.smithi152.stdout: index_block_restart_interval: 1 2024-09-11T22:43:05.041 INFO:journalctl@ceph.mon.a.smithi152.stdout: metadata_block_size: 4096 2024-09-11T22:43:05.041 INFO:journalctl@ceph.mon.a.smithi152.stdout: partition_filters: 0 2024-09-11T22:43:05.041 INFO:journalctl@ceph.mon.a.smithi152.stdout: use_delta_encoding: 1 2024-09-11T22:43:05.041 INFO:journalctl@ceph.mon.a.smithi152.stdout: filter_policy: bloomfilter 2024-09-11T22:43:05.042 INFO:journalctl@ceph.mon.a.smithi152.stdout: whole_key_filtering: 1 2024-09-11T22:43:05.042 INFO:journalctl@ceph.mon.a.smithi152.stdout: verify_compression: 0 2024-09-11T22:43:05.042 INFO:journalctl@ceph.mon.a.smithi152.stdout: read_amp_bytes_per_bit: 0 2024-09-11T22:43:05.042 INFO:journalctl@ceph.mon.a.smithi152.stdout: format_version: 5 2024-09-11T22:43:05.042 INFO:journalctl@ceph.mon.a.smithi152.stdout: enable_index_compression: 1 2024-09-11T22:43:05.042 INFO:journalctl@ceph.mon.a.smithi152.stdout: block_align: 0 2024-09-11T22:43:05.042 INFO:journalctl@ceph.mon.a.smithi152.stdout: max_auto_readahead_size: 262144 2024-09-11T22:43:05.042 INFO:journalctl@ceph.mon.a.smithi152.stdout: prepopulate_block_cache: 0 2024-09-11T22:43:05.042 INFO:journalctl@ceph.mon.a.smithi152.stdout: initial_auto_readahead_size: 8192 2024-09-11T22:43:05.042 INFO:journalctl@ceph.mon.a.smithi152.stdout: num_file_reads_for_auto_readahead: 2 2024-09-11T22:43:05.042 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T22:43:05.042 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T22:43:05.042 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compression: NoCompression 2024-09-11T22:43:05.042 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T22:43:05.043 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T22:43:05.043 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T22:43:05.043 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.num_levels: 7 2024-09-11T22:43:05.043 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T22:43:05.043 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T22:43:05.043 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T22:43:05.043 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T22:43:05.043 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T22:43:05.043 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T22:43:05.043 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T22:43:05.043 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T22:43:05.043 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T22:43:05.044 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T22:43:05.044 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-11T22:43:05.044 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-11T22:43:05.044 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T22:43:05.044 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T22:43:05.044 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T22:43:05.044 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T22:43:05.044 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T22:43:05.044 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-11T22:43:05.044 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T22:43:05.044 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T22:43:05.044 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-11T22:43:05.044 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T22:43:05.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T22:43:05.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T22:43:05.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T22:43:05.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T22:43:05.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T22:43:05.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T22:43:05.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T22:43:05.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T22:43:05.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T22:43:05.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T22:43:05.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T22:43:05.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T22:43:05.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T22:43:05.046 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T22:43:05.046 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T22:43:05.046 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T22:43:05.046 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-11T22:43:05.046 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.arena_block_size: 1048576 2024-09-11T22:43:05.046 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T22:43:05.047 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T22:43:05.047 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T22:43:05.047 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T22:43:05.047 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T22:43:05.047 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T22:43:05.047 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T22:43:05.047 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T22:43:05.047 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T22:43:05.047 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T22:43:05.048 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T22:43:05.048 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T22:43:05.048 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T22:43:05.048 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T22:43:05.048 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.inplace_update_support: 0 2024-09-11T22:43:05.048 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T22:43:05.048 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T22:43:05.048 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T22:43:05.048 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T22:43:05.048 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.bloom_locality: 0 2024-09-11T22:43:05.048 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.max_successive_merges: 0 2024-09-11T22:43:05.048 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T22:43:05.048 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T22:43:05.049 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T22:43:05.049 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T22:43:05.049 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.ttl: 2592000 2024-09-11T22:43:05.049 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T22:43:05.049 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-11T22:43:05.049 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-11T22:43:05.049 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.enable_blob_files: false 2024-09-11T22:43:05.049 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.min_blob_size: 0 2024-09-11T22:43:05.049 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T22:43:05.049 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T22:43:05.049 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T22:43:05.049 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T22:43:05.050 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-11T22:43:05.050 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-11T22:43:05.050 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.blob_file_starting_level: 0 2024-09-11T22:43:05.050 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-11T22:43:05.050 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-11T22:43:05.050 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-11T22:43:05.050 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 51208c5e-9e8a-4abf-9c84-302bc08478f9 2024-09-11T22:43:05.050 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726094585022992, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-11T22:43:05.050 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-11T22:43:05.050 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726094585024068, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100365, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98250, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14290, "raw_average_key_size": 49, "raw_value_size": 90523, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726094585, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "51208c5e-9e8a-4abf-9c84-302bc08478f9", "db_session_id": "6ZP3BGM7X24JBEEE97PT", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-11T22:43:05.050 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726094585024227, "job": 1, "event": "recovery_finished"} 2024-09-11T22:43:05.050 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-11T22:43:05.051 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T22:43:05.051 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d02fd86e00 2024-09-11T22:43:05.051 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: DB pointer 0x55d02fe82000 2024-09-11T22:43:05.051 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-11T22:43:05.051 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-11T22:43:05.051 INFO:journalctl@ceph.mon.a.smithi152.stdout: ** DB Stats ** 2024-09-11T22:43:05.051 INFO:journalctl@ceph.mon.a.smithi152.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T22:43:05.051 INFO:journalctl@ceph.mon.a.smithi152.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T22:43:05.051 INFO:journalctl@ceph.mon.a.smithi152.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T22:43:05.051 INFO:journalctl@ceph.mon.a.smithi152.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T22:43:05.051 INFO:journalctl@ceph.mon.a.smithi152.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T22:43:05.051 INFO:journalctl@ceph.mon.a.smithi152.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T22:43:05.051 INFO:journalctl@ceph.mon.a.smithi152.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T22:43:05.051 INFO:journalctl@ceph.mon.a.smithi152.stdout: 2024-09-11T22:43:05.052 INFO:journalctl@ceph.mon.a.smithi152.stdout: ** Compaction Stats [default] ** 2024-09-11T22:43:05.052 INFO:journalctl@ceph.mon.a.smithi152.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T22:43:05.052 INFO:journalctl@ceph.mon.a.smithi152.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-11T22:43:05.052 INFO:journalctl@ceph.mon.a.smithi152.stdout: L0 2/0 99.87 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T22:43:05.052 INFO:journalctl@ceph.mon.a.smithi152.stdout: Sum 2/0 99.87 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T22:43:05.052 INFO:journalctl@ceph.mon.a.smithi152.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 133.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T22:43:05.052 INFO:journalctl@ceph.mon.a.smithi152.stdout: 2024-09-11T22:43:05.052 INFO:journalctl@ceph.mon.a.smithi152.stdout: ** Compaction Stats [default] ** 2024-09-11T22:43:05.052 INFO:journalctl@ceph.mon.a.smithi152.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T22:43:05.052 INFO:journalctl@ceph.mon.a.smithi152.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T22:43:05.052 INFO:journalctl@ceph.mon.a.smithi152.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 133.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T22:43:05.052 INFO:journalctl@ceph.mon.a.smithi152.stdout: 2024-09-11T22:43:05.053 INFO:journalctl@ceph.mon.a.smithi152.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-11T22:43:05.053 INFO:journalctl@ceph.mon.a.smithi152.stdout: 2024-09-11T22:43:05.053 INFO:journalctl@ceph.mon.a.smithi152.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T22:43:05.053 INFO:journalctl@ceph.mon.a.smithi152.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T22:43:05.053 INFO:journalctl@ceph.mon.a.smithi152.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T22:43:05.053 INFO:journalctl@ceph.mon.a.smithi152.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T22:43:05.053 INFO:journalctl@ceph.mon.a.smithi152.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T22:43:05.053 INFO:journalctl@ceph.mon.a.smithi152.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T22:43:05.053 INFO:journalctl@ceph.mon.a.smithi152.stdout: Cumulative compaction: 0.00 GB write, 24.91 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T22:43:05.053 INFO:journalctl@ceph.mon.a.smithi152.stdout: Interval compaction: 0.00 GB write, 24.91 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T22:43:05.053 INFO:journalctl@ceph.mon.a.smithi152.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T22:43:05.053 INFO:journalctl@ceph.mon.a.smithi152.stdout: Block cache BinnedLRUCache@0x55d02fd591f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-11T22:43:05.054 INFO:journalctl@ceph.mon.a.smithi152.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-11T22:43:05.054 INFO:journalctl@ceph.mon.a.smithi152.stdout: 2024-09-11T22:43:05.054 INFO:journalctl@ceph.mon.a.smithi152.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T22:43:05.054 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: starting mon.a rank 0 at public addrs [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] at bind addrs [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:43:05.054 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: mon.a@-1(???) e1 preinit fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:43:05.054 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: mon.a@-1(???).mds e1 new map 2024-09-11T22:43:05.054 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: mon.a@-1(???).mds e1 print_map 2024-09-11T22:43:05.054 INFO:journalctl@ceph.mon.a.smithi152.stdout: e1 2024-09-11T22:43:05.054 INFO:journalctl@ceph.mon.a.smithi152.stdout: btime 2024-09-11T22:43:00:379132+0000 2024-09-11T22:43:05.054 INFO:journalctl@ceph.mon.a.smithi152.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T22:43:05.054 INFO:journalctl@ceph.mon.a.smithi152.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T22:43:05.055 INFO:journalctl@ceph.mon.a.smithi152.stdout: legacy client fscid: -1 2024-09-11T22:43:05.055 INFO:journalctl@ceph.mon.a.smithi152.stdout: 2024-09-11T22:43:05.055 INFO:journalctl@ceph.mon.a.smithi152.stdout: No filesystems configured 2024-09-11T22:43:05.100 INFO:teuthology.orchestra.run.smithi152.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T22:43:05.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T22:43:05.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T22:43:05.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T22:43:05.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T22:43:05.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T22:43:05.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T22:43:05.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: monmap epoch 1 2024-09-11T22:43:05.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:43:05.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: last_changed 2024-09-11T22:42:57.315021+0000 2024-09-11T22:43:05.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: created 2024-09-11T22:42:57.315021+0000 2024-09-11T22:43:05.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: min_mon_release 19 (squid) 2024-09-11T22:43:05.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: election_strategy: 1 2024-09-11T22:43:05.340 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: 0: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.a 2024-09-11T22:43:05.340 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: fsmap 2024-09-11T22:43:05.340 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T22:43:05.340 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 ceph-mon[31181]: mgrmap e1: no daemons active 2024-09-11T22:43:05.340 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 bash[31161]: 316177fd5605dab938c8f1eb25560984f1f6f171aed29416c26038ec7f66c86d 2024-09-11T22:43:05.340 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:05 smithi152 systemd[1]: Started Ceph mon.a for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:43:06.167 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.527+0000 7f03d7a8b640 1 Processor -- start 2024-09-11T22:43:06.167 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.528+0000 7f03d7a8b640 1 -- start start 2024-09-11T22:43:06.167 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.528+0000 7f03d7a8b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d0106510 0x7f03d0106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:06.167 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.528+0000 7f03d7a8b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f03d0106ee0 con 0x7f03d0106510 2024-09-11T22:43:06.167 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.528+0000 7f03d5827640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d0106510 0x7f03d0106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:06.167 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.528+0000 7f03d5827640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d0106510 0x7f03d0106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41638/0 (socket says 172.21.15.152:41638) 2024-09-11T22:43:06.167 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.528+0000 7f03d5827640 1 -- 172.21.15.152:0/3298692742 learned_addr learned my addr 172.21.15.152:0/3298692742 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:06.167 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.528+0000 7f03d5827640 1 -- 172.21.15.152:0/3298692742 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d01076c0 con 0x7f03d0106510 2024-09-11T22:43:06.167 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.531+0000 7f03d5827640 1 --2- 172.21.15.152:0/3298692742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d0106510 0x7f03d0106910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009820 tx=0x7f03c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7994729f1b928590 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:06.168 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.531+0000 7f03d4825640 1 -- 172.21.15.152:0/3298692742 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03c4037070 con 0x7f03d0106510 2024-09-11T22:43:06.168 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.531+0000 7f03d4825640 1 -- 172.21.15.152:0/3298692742 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03c4032900 con 0x7f03d0106510 2024-09-11T22:43:06.168 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.531+0000 7f03d4825640 1 -- 172.21.15.152:0/3298692742 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03c4030850 con 0x7f03d0106510 2024-09-11T22:43:06.168 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.531+0000 7f03d7a8b640 1 -- 172.21.15.152:0/3298692742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d0106510 msgr2=0x7f03d0106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:06.168 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.531+0000 7f03d7a8b640 1 --2- 172.21.15.152:0/3298692742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d0106510 0x7f03d0106910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009820 tx=0x7f03c402a270 comp rx=0 tx=0).stop 2024-09-11T22:43:06.168 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.531+0000 7f03d7a8b640 1 -- 172.21.15.152:0/3298692742 shutdown_connections 2024-09-11T22:43:06.168 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.531+0000 7f03d7a8b640 1 --2- 172.21.15.152:0/3298692742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d0106510 0x7f03d0106910 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:06.168 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.531+0000 7f03d7a8b640 1 -- 172.21.15.152:0/3298692742 >> 172.21.15.152:0/3298692742 conn(0x7f03d0101ca0 msgr2=0x7f03d01040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:06.168 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.532+0000 7f03d7a8b640 1 -- 172.21.15.152:0/3298692742 shutdown_connections 2024-09-11T22:43:06.168 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.532+0000 7f03d7a8b640 1 -- 172.21.15.152:0/3298692742 wait complete. 2024-09-11T22:43:06.168 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.532+0000 7f03d7a8b640 1 Processor -- start 2024-09-11T22:43:06.168 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.532+0000 7f03d7a8b640 1 -- start start 2024-09-11T22:43:06.169 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.532+0000 7f03d7a8b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d0106510 0x7f03d019b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:06.169 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.532+0000 7f03d7a8b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f03d010aa70 con 0x7f03d0106510 2024-09-11T22:43:06.169 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.532+0000 7f03d5827640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d0106510 0x7f03d019b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:06.169 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.532+0000 7f03d5827640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d0106510 0x7f03d019b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41652/0 (socket says 172.21.15.152:41652) 2024-09-11T22:43:06.169 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.533+0000 7f03d5827640 1 -- 172.21.15.152:0/65090330 learned_addr learned my addr 172.21.15.152:0/65090330 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:06.169 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.533+0000 7f03d5827640 1 -- 172.21.15.152:0/65090330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d019bb70 con 0x7f03d0106510 2024-09-11T22:43:06.169 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.533+0000 7f03d5827640 1 --2- 172.21.15.152:0/65090330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d0106510 0x7f03d019b630 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009940 tx=0x7f03c40326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:06.169 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.533+0000 7f03c2ffd640 1 -- 172.21.15.152:0/65090330 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03c4042070 con 0x7f03d0106510 2024-09-11T22:43:06.169 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.533+0000 7f03c2ffd640 1 -- 172.21.15.152:0/65090330 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03c4032be0 con 0x7f03d0106510 2024-09-11T22:43:06.169 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.533+0000 7f03d7a8b640 1 -- 172.21.15.152:0/65090330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d019be00 con 0x7f03d0106510 2024-09-11T22:43:06.170 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.533+0000 7f03c2ffd640 1 -- 172.21.15.152:0/65090330 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f03c4037040 con 0x7f03d0106510 2024-09-11T22:43:06.170 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.533+0000 7f03d7a8b640 1 -- 172.21.15.152:0/65090330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d019eaf0 con 0x7f03d0106510 2024-09-11T22:43:06.170 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.533+0000 7f03c2ffd640 1 -- 172.21.15.152:0/65090330 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f03c4045070 con 0x7f03d0106510 2024-09-11T22:43:06.170 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.533+0000 7f03c2ffd640 1 -- 172.21.15.152:0/65090330 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f03c40479f0 con 0x7f03d0106510 2024-09-11T22:43:06.170 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.534+0000 7f03d7a8b640 1 -- 172.21.15.152:0/65090330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d010abf0 con 0x7f03d0106510 2024-09-11T22:43:06.170 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.536+0000 7f03c2ffd640 1 -- 172.21.15.152:0/65090330 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f03c4032d90 con 0x7f03d0106510 2024-09-11T22:43:06.170 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.593+0000 7f03d7a8b640 1 -- 172.21.15.152:0/65090330 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f03d0106990 con 0x7f03d0106510 2024-09-11T22:43:06.170 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.597+0000 7f03c2ffd640 1 -- 172.21.15.152:0/65090330 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f03c403d430 con 0x7f03d0106510 2024-09-11T22:43:06.170 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.598+0000 7f03d7a8b640 1 -- 172.21.15.152:0/65090330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d0106510 msgr2=0x7f03d019b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:06.170 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.598+0000 7f03d7a8b640 1 --2- 172.21.15.152:0/65090330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d0106510 0x7f03d019b630 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009940 tx=0x7f03c40326e0 comp rx=0 tx=0).stop 2024-09-11T22:43:06.170 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.598+0000 7f03d7a8b640 1 -- 172.21.15.152:0/65090330 shutdown_connections 2024-09-11T22:43:06.170 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.598+0000 7f03d7a8b640 1 --2- 172.21.15.152:0/65090330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03d0106510 0x7f03d019b630 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:06.171 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.598+0000 7f03d7a8b640 1 -- 172.21.15.152:0/65090330 >> 172.21.15.152:0/65090330 conn(0x7f03d0101ca0 msgr2=0x7f03d0102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:06.171 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.598+0000 7f03d7a8b640 1 -- 172.21.15.152:0/65090330 shutdown_connections 2024-09-11T22:43:06.171 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:05.598+0000 7f03d7a8b640 1 -- 172.21.15.152:0/65090330 wait complete. 2024-09-11T22:43:06.171 INFO:teuthology.orchestra.run.smithi152.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T22:43:06.184 INFO:teuthology.orchestra.run.smithi152.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:43:06.184 INFO:teuthology.orchestra.run.smithi152.stdout:Creating mgr... 2024-09-11T22:43:06.185 INFO:teuthology.orchestra.run.smithi152.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T22:43:06.185 INFO:teuthology.orchestra.run.smithi152.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T22:43:06.426 INFO:teuthology.orchestra.run.smithi152.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mgr.x 2024-09-11T22:43:06.426 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-11T22:43:06.573 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0e4e82bc-708f-11ef-bcea-c7b262605968.target.wants/ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@.service. 2024-09-11T22:43:06.804 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:06 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/65090330' entity='client.admin' 2024-09-11T22:43:07.073 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:07 smithi152 podman[31392]: 2024-09-11 22:43:07.008683102 +0000 UTC m=+0.316810462 container start 4c789e59e2f40e1ae10d44d44f69ea04a13e05589bafdd6bd523f85cebcfe6e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T22:43:07.073 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:07 smithi152 bash[31392]: 4c789e59e2f40e1ae10d44d44f69ea04a13e05589bafdd6bd523f85cebcfe6e0 2024-09-11T22:43:07.087 INFO:teuthology.orchestra.run.smithi152.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T22:43:07.087 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stdout disabled 2024-09-11T22:43:07.094 INFO:teuthology.orchestra.run.smithi152.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T22:43:07.094 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stdout inactive 2024-09-11T22:43:07.094 INFO:teuthology.orchestra.run.smithi152.stdout:firewalld.service is not enabled 2024-09-11T22:43:07.094 INFO:teuthology.orchestra.run.smithi152.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T22:43:07.103 INFO:teuthology.orchestra.run.smithi152.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T22:43:07.103 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stdout disabled 2024-09-11T22:43:07.110 INFO:teuthology.orchestra.run.smithi152.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T22:43:07.110 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stdout inactive 2024-09-11T22:43:07.110 INFO:teuthology.orchestra.run.smithi152.stdout:firewalld.service is not enabled 2024-09-11T22:43:07.110 INFO:teuthology.orchestra.run.smithi152.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T22:43:07.111 INFO:teuthology.orchestra.run.smithi152.stdout:Waiting for mgr to start... 2024-09-11T22:43:07.111 INFO:teuthology.orchestra.run.smithi152.stdout:Waiting for mgr... 2024-09-11T22:43:07.339 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:07 smithi152 systemd[1]: Started Ceph mgr.x for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:43:07.339 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:07 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:07.132+0000 7f15c9758040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:43:07.339 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:07 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:07.189+0000 7f15c9758040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:43:07.340 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:07 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:07.338+0000 7f15c9758040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:43:07.615 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:07 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:07.409+0000 7f15c9758040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:43:07.615 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:07 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:07.562+0000 7f15c9758040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:43:07.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:07 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2792606201' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T22:43:07.873 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:07 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:07.714+0000 7f15c9758040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:43:08.250 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout 2024-09-11T22:43:08.250 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:43:08.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:43:08.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T22:43:08.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T22:43:08.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T22:43:08.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T22:43:08.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:08.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T22:43:08.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T22:43:08.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout 0 2024-09-11T22:43:08.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:43:08.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T22:43:08.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T22:43:08.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:43:08.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-11T22:43:08.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T22:43:08.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:43:08.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T22:43:08.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T22:43:08.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:08.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T22:43:08.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:43:08.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T22:43:08.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T22:43:08.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T22:43:08.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T22:43:08.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T22:43:08.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T22:43:08.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:08.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T22:43:08.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T22:43:08.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T22:43:08.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T22:43:08.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T22:43:08.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T22:43:08.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T22:43:08.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T22:43:08.254 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T22:43:08.254 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:08.254 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T22:43:08.254 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:43:08.254 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T22:43:00:379132+0000", 2024-09-11T22:43:08.254 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T22:43:08.254 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T22:43:08.254 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:08.254 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T22:43:08.254 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T22:43:08.254 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T22:43:08.254 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T22:43:08.254 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T22:43:08.254 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T22:43:08.255 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T22:43:08.255 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:43:08.255 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T22:43:08.255 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:08.255 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T22:43:08.255 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:43:08.255 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T22:43:00.379772+0000", 2024-09-11T22:43:08.255 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T22:43:08.255 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:08.255 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T22:43:08.255 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:43:08.255 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.546+0000 7fe6f1b69640 1 Processor -- start 2024-09-11T22:43:08.255 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.546+0000 7fe6f1b69640 1 -- start start 2024-09-11T22:43:08.255 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.546+0000 7fe6f1b69640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6ec107630 0x7fe6ec107a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:08.256 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.546+0000 7fe6f1b69640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec108000 con 0x7fe6ec107630 2024-09-11T22:43:08.256 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.546+0000 7fe6eb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6ec107630 0x7fe6ec107a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:08.256 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.546+0000 7fe6eb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6ec107630 0x7fe6ec107a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41672/0 (socket says 172.21.15.152:41672) 2024-09-11T22:43:08.256 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.546+0000 7fe6eb7fe640 1 -- 172.21.15.152:0/3399375179 learned_addr learned my addr 172.21.15.152:0/3399375179 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:08.256 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.547+0000 7fe6eb7fe640 1 -- 172.21.15.152:0/3399375179 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ec1087e0 con 0x7fe6ec107630 2024-09-11T22:43:08.256 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.547+0000 7fe6eb7fe640 1 --2- 172.21.15.152:0/3399375179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6ec107630 0x7fe6ec107a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc009820 tx=0x7fe6dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f93d655878c536 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:08.256 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.548+0000 7fe6ea7fc640 1 -- 172.21.15.152:0/3399375179 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6dc037070 con 0x7fe6ec107630 2024-09-11T22:43:08.256 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.548+0000 7fe6ea7fc640 1 -- 172.21.15.152:0/3399375179 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe6dc032900 con 0x7fe6ec107630 2024-09-11T22:43:08.256 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.548+0000 7fe6ea7fc640 1 -- 172.21.15.152:0/3399375179 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6dc030850 con 0x7fe6ec107630 2024-09-11T22:43:08.256 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.548+0000 7fe6f1b69640 1 -- 172.21.15.152:0/3399375179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6ec107630 msgr2=0x7fe6ec107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:08.257 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.548+0000 7fe6f1b69640 1 --2- 172.21.15.152:0/3399375179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6ec107630 0x7fe6ec107a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc009820 tx=0x7fe6dc02a270 comp rx=0 tx=0).stop 2024-09-11T22:43:08.257 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.548+0000 7fe6f1b69640 1 -- 172.21.15.152:0/3399375179 shutdown_connections 2024-09-11T22:43:08.257 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.548+0000 7fe6f1b69640 1 --2- 172.21.15.152:0/3399375179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6ec107630 0x7fe6ec107a30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:08.257 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.548+0000 7fe6f1b69640 1 -- 172.21.15.152:0/3399375179 >> 172.21.15.152:0/3399375179 conn(0x7fe6ec102da0 msgr2=0x7fe6ec105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:08.257 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.548+0000 7fe6f1b69640 1 -- 172.21.15.152:0/3399375179 shutdown_connections 2024-09-11T22:43:08.257 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.548+0000 7fe6f1b69640 1 -- 172.21.15.152:0/3399375179 wait complete. 2024-09-11T22:43:08.257 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.549+0000 7fe6f1b69640 1 Processor -- start 2024-09-11T22:43:08.257 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.549+0000 7fe6f1b69640 1 -- start start 2024-09-11T22:43:08.257 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.549+0000 7fe6f1b69640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6ec107630 0x7fe6ec07db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:08.258 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.549+0000 7fe6f1b69640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec10bb90 con 0x7fe6ec107630 2024-09-11T22:43:08.258 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.549+0000 7fe6eb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6ec107630 0x7fe6ec07db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:08.258 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.549+0000 7fe6eb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6ec107630 0x7fe6ec07db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41674/0 (socket says 172.21.15.152:41674) 2024-09-11T22:43:08.258 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.549+0000 7fe6eb7fe640 1 -- 172.21.15.152:0/2792606201 learned_addr learned my addr 172.21.15.152:0/2792606201 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:08.258 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.550+0000 7fe6eb7fe640 1 -- 172.21.15.152:0/2792606201 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ec07e080 con 0x7fe6ec107630 2024-09-11T22:43:08.258 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.550+0000 7fe6eb7fe640 1 --2- 172.21.15.152:0/2792606201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6ec107630 0x7fe6ec07db40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0097f0 tx=0x7fe6dc030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:08.258 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.550+0000 7fe6e8ff9640 1 -- 172.21.15.152:0/2792606201 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6dc042070 con 0x7fe6ec107630 2024-09-11T22:43:08.258 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.550+0000 7fe6e8ff9640 1 -- 172.21.15.152:0/2792606201 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe6dc032ca0 con 0x7fe6ec107630 2024-09-11T22:43:08.258 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.550+0000 7fe6e8ff9640 1 -- 172.21.15.152:0/2792606201 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6dc037040 con 0x7fe6ec107630 2024-09-11T22:43:08.258 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.550+0000 7fe6f1b69640 1 -- 172.21.15.152:0/2792606201 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6ec07c270 con 0x7fe6ec107630 2024-09-11T22:43:08.258 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.550+0000 7fe6f1b69640 1 -- 172.21.15.152:0/2792606201 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6ec07c7b0 con 0x7fe6ec107630 2024-09-11T22:43:08.258 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.551+0000 7fe6e8ff9640 1 -- 172.21.15.152:0/2792606201 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe6dc045070 con 0x7fe6ec107630 2024-09-11T22:43:08.259 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.551+0000 7fe6e8ff9640 1 -- 172.21.15.152:0/2792606201 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe6dc047a30 con 0x7fe6ec107630 2024-09-11T22:43:08.259 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.551+0000 7fe6f1b69640 1 -- 172.21.15.152:0/2792606201 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6ec10bd10 con 0x7fe6ec107630 2024-09-11T22:43:08.259 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.553+0000 7fe6e8ff9640 1 -- 172.21.15.152:0/2792606201 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe6dc03da60 con 0x7fe6ec107630 2024-09-11T22:43:08.259 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.614+0000 7fe6f1b69640 1 -- 172.21.15.152:0/2792606201 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe6ec07ca30 con 0x7fe6ec107630 2024-09-11T22:43:08.259 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.615+0000 7fe6e8ff9640 1 -- 172.21.15.152:0/2792606201 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fe6ec07ca30 con 0x7fe6ec107630 2024-09-11T22:43:08.259 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.616+0000 7fe6f1b69640 1 -- 172.21.15.152:0/2792606201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6ec107630 msgr2=0x7fe6ec07db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:08.259 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.616+0000 7fe6f1b69640 1 --2- 172.21.15.152:0/2792606201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6ec107630 0x7fe6ec07db40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0097f0 tx=0x7fe6dc030ec0 comp rx=0 tx=0).stop 2024-09-11T22:43:08.259 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.616+0000 7fe6f1b69640 1 -- 172.21.15.152:0/2792606201 shutdown_connections 2024-09-11T22:43:08.259 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.616+0000 7fe6f1b69640 1 --2- 172.21.15.152:0/2792606201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6ec107630 0x7fe6ec07db40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:08.259 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.616+0000 7fe6f1b69640 1 -- 172.21.15.152:0/2792606201 >> 172.21.15.152:0/2792606201 conn(0x7fe6ec102da0 msgr2=0x7fe6ec103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:08.259 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.616+0000 7fe6f1b69640 1 -- 172.21.15.152:0/2792606201 shutdown_connections 2024-09-11T22:43:08.259 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:07.616+0000 7fe6f1b69640 1 -- 172.21.15.152:0/2792606201 wait complete. 2024-09-11T22:43:08.259 INFO:teuthology.orchestra.run.smithi152.stdout:mgr not available, waiting (1/15)... 2024-09-11T22:43:08.284 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:08 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:08.032+0000 7f15c9758040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:43:08.285 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:08 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:08.094+0000 7f15c9758040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:43:08.285 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:08 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:08.281+0000 7f15c9758040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:43:09.622 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:09 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:09.156+0000 7f15c9758040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:43:09.622 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:09 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:09.226+0000 7f15c9758040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:43:10.198 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:09 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:09.941+0000 7f15c9758040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:43:10.199 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:10 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:10.056+0000 7f15c9758040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:43:10.199 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:10 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:10.197+0000 7f15c9758040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:43:10.466 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:10 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:10.260+0000 7f15c9758040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:43:10.467 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:10 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:10.414+0000 7f15c9758040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:43:10.724 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:10 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:10.486+0000 7f15c9758040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:43:10.724 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:10 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:10.546+0000 7f15c9758040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:43:10.724 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:10 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:10.723+0000 7f15c9758040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:43:11.003 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:10 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2315671542' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T22:43:11.003 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:10 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:10.792+0000 7f15c9758040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:43:11.003 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:10 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:10.859+0000 7f15c9758040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:43:11.003 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:10 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:10.942+0000 7f15c9758040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:43:11.003 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:11 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:11.002+0000 7f15c9758040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:43:11.328 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:11 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:11.063+0000 7f15c9758040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:43:11.332 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout 2024-09-11T22:43:11.332 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:43:11.332 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:43:11.332 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T22:43:11.333 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T22:43:11.333 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T22:43:11.333 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T22:43:11.333 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:11.333 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T22:43:11.333 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T22:43:11.333 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout 0 2024-09-11T22:43:11.333 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:43:11.333 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T22:43:11.333 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T22:43:11.333 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:43:11.333 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-11T22:43:11.333 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T22:43:11.333 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:43:11.334 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T22:43:11.334 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T22:43:11.334 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:11.334 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T22:43:11.334 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:43:11.334 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T22:43:11.334 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T22:43:11.334 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T22:43:11.342 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T22:43:11.342 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T22:43:11.342 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T22:43:11.342 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:11.342 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T22:43:11.342 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T22:43:11.342 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T22:43:11.342 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T22:43:11.342 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T22:43:11.342 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T22:43:11.342 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T22:43:11.343 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T22:43:11.343 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T22:43:11.343 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:11.343 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T22:43:11.343 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:43:11.343 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T22:43:00:379132+0000", 2024-09-11T22:43:11.343 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T22:43:11.343 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T22:43:11.343 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:11.343 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T22:43:11.343 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T22:43:11.343 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T22:43:11.343 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T22:43:11.343 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T22:43:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T22:43:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T22:43:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:43:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T22:43:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T22:43:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:43:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T22:43:00.379772+0000", 2024-09-11T22:43:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T22:43:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T22:43:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:43:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.675+0000 7fe95e969640 1 Processor -- start 2024-09-11T22:43:11.344 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.675+0000 7fe95e969640 1 -- start start 2024-09-11T22:43:11.345 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.675+0000 7fe95e969640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe958106510 0x7fe958106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:11.345 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.675+0000 7fe95e969640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe958106ee0 con 0x7fe958106510 2024-09-11T22:43:11.345 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.676+0000 7fe957fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe958106510 0x7fe958106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:11.345 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.676+0000 7fe957fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe958106510 0x7fe958106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43114/0 (socket says 172.21.15.152:43114) 2024-09-11T22:43:11.345 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.676+0000 7fe957fff640 1 -- 172.21.15.152:0/3469298407 learned_addr learned my addr 172.21.15.152:0/3469298407 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:11.345 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.676+0000 7fe957fff640 1 -- 172.21.15.152:0/3469298407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9581076c0 con 0x7fe958106510 2024-09-11T22:43:11.345 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.677+0000 7fe957fff640 1 --2- 172.21.15.152:0/3469298407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe958106510 0x7fe958106910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe948009820 tx=0x7fe94802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972284a259dcebb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:11.345 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.677+0000 7fe956ffd640 1 -- 172.21.15.152:0/3469298407 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe948037070 con 0x7fe958106510 2024-09-11T22:43:11.345 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.677+0000 7fe956ffd640 1 -- 172.21.15.152:0/3469298407 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe948032900 con 0x7fe958106510 2024-09-11T22:43:11.345 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.677+0000 7fe956ffd640 1 -- 172.21.15.152:0/3469298407 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe948030850 con 0x7fe958106510 2024-09-11T22:43:11.345 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.678+0000 7fe95e969640 1 -- 172.21.15.152:0/3469298407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe958106510 msgr2=0x7fe958106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:11.345 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.678+0000 7fe95e969640 1 --2- 172.21.15.152:0/3469298407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe958106510 0x7fe958106910 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe948009820 tx=0x7fe94802a270 comp rx=0 tx=0).stop 2024-09-11T22:43:11.345 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.678+0000 7fe95e969640 1 -- 172.21.15.152:0/3469298407 shutdown_connections 2024-09-11T22:43:11.346 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.678+0000 7fe95e969640 1 --2- 172.21.15.152:0/3469298407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe958106510 0x7fe958106910 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:11.346 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.678+0000 7fe95e969640 1 -- 172.21.15.152:0/3469298407 >> 172.21.15.152:0/3469298407 conn(0x7fe958101ca0 msgr2=0x7fe9581040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:11.346 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.678+0000 7fe95e969640 1 -- 172.21.15.152:0/3469298407 shutdown_connections 2024-09-11T22:43:11.346 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.678+0000 7fe95e969640 1 -- 172.21.15.152:0/3469298407 wait complete. 2024-09-11T22:43:11.346 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.678+0000 7fe95e969640 1 Processor -- start 2024-09-11T22:43:11.346 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.678+0000 7fe95e969640 1 -- start start 2024-09-11T22:43:11.346 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.679+0000 7fe95e969640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe958106510 0x7fe95819b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:11.346 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.679+0000 7fe95e969640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe95810aa70 con 0x7fe958106510 2024-09-11T22:43:11.346 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.679+0000 7fe957fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe958106510 0x7fe95819b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:11.346 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.679+0000 7fe957fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe958106510 0x7fe95819b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43120/0 (socket says 172.21.15.152:43120) 2024-09-11T22:43:11.346 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.679+0000 7fe957fff640 1 -- 172.21.15.152:0/2315671542 learned_addr learned my addr 172.21.15.152:0/2315671542 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:11.346 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.679+0000 7fe957fff640 1 -- 172.21.15.152:0/2315671542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95819bb70 con 0x7fe958106510 2024-09-11T22:43:11.347 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.679+0000 7fe957fff640 1 --2- 172.21.15.152:0/2315671542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe958106510 0x7fe95819b630 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe948009940 tx=0x7fe9480326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:11.347 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.679+0000 7fe9557fa640 1 -- 172.21.15.152:0/2315671542 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe948042070 con 0x7fe958106510 2024-09-11T22:43:11.347 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.679+0000 7fe9557fa640 1 -- 172.21.15.152:0/2315671542 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe948032be0 con 0x7fe958106510 2024-09-11T22:43:11.347 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.679+0000 7fe95e969640 1 -- 172.21.15.152:0/2315671542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe95819be00 con 0x7fe958106510 2024-09-11T22:43:11.347 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.679+0000 7fe9557fa640 1 -- 172.21.15.152:0/2315671542 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe948037040 con 0x7fe958106510 2024-09-11T22:43:11.347 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.680+0000 7fe95e969640 1 -- 172.21.15.152:0/2315671542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe95819eaf0 con 0x7fe958106510 2024-09-11T22:43:11.347 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.680+0000 7fe9557fa640 1 -- 172.21.15.152:0/2315671542 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe948045070 con 0x7fe958106510 2024-09-11T22:43:11.347 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.680+0000 7fe9557fa640 1 -- 172.21.15.152:0/2315671542 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe948047a80 con 0x7fe958106510 2024-09-11T22:43:11.347 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.680+0000 7fe95e969640 1 -- 172.21.15.152:0/2315671542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe95810abf0 con 0x7fe958106510 2024-09-11T22:43:11.347 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.683+0000 7fe9557fa640 1 -- 172.21.15.152:0/2315671542 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe948032d90 con 0x7fe958106510 2024-09-11T22:43:11.347 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.758+0000 7fe95e969640 1 -- 172.21.15.152:0/2315671542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe958195590 con 0x7fe958106510 2024-09-11T22:43:11.347 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.758+0000 7fe9557fa640 1 -- 172.21.15.152:0/2315671542 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fe94803d430 con 0x7fe958106510 2024-09-11T22:43:11.347 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.759+0000 7fe95e969640 1 -- 172.21.15.152:0/2315671542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe958106510 msgr2=0x7fe95819b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:11.348 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.759+0000 7fe95e969640 1 --2- 172.21.15.152:0/2315671542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe958106510 0x7fe95819b630 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe948009940 tx=0x7fe9480326e0 comp rx=0 tx=0).stop 2024-09-11T22:43:11.348 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.760+0000 7fe95e969640 1 -- 172.21.15.152:0/2315671542 shutdown_connections 2024-09-11T22:43:11.348 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.760+0000 7fe95e969640 1 --2- 172.21.15.152:0/2315671542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe958106510 0x7fe95819b630 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:11.348 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.760+0000 7fe95e969640 1 -- 172.21.15.152:0/2315671542 >> 172.21.15.152:0/2315671542 conn(0x7fe958101ca0 msgr2=0x7fe958102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:11.348 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.760+0000 7fe95e969640 1 -- 172.21.15.152:0/2315671542 shutdown_connections 2024-09-11T22:43:11.348 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:10.760+0000 7fe95e969640 1 -- 172.21.15.152:0/2315671542 wait complete. 2024-09-11T22:43:11.348 INFO:teuthology.orchestra.run.smithi152.stdout:mgr not available, waiting (2/15)... 2024-09-11T22:43:12.122 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:11 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:11.848+0000 7f15c9758040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:43:12.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:11 smithi152 ceph-mon[31181]: Activating manager daemon x 2024-09-11T22:43:12.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:11 smithi152 ceph-mon[31181]: mgrmap e2: x(active, starting, since 0.00296349s) 2024-09-11T22:43:12.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:11 smithi152 ceph-mon[31181]: from='mgr.14100 172.21.15.152:0/488802770' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T22:43:12.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:11 smithi152 ceph-mon[31181]: from='mgr.14100 172.21.15.152:0/488802770' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T22:43:12.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:11 smithi152 ceph-mon[31181]: from='mgr.14100 172.21.15.152:0/488802770' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T22:43:12.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:11 smithi152 ceph-mon[31181]: from='mgr.14100 172.21.15.152:0/488802770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:43:12.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:11 smithi152 ceph-mon[31181]: from='mgr.14100 172.21.15.152:0/488802770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T22:43:12.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:11 smithi152 ceph-mon[31181]: Manager daemon x is now available 2024-09-11T22:43:12.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:11 smithi152 ceph-mon[31181]: from='mgr.14100 172.21.15.152:0/488802770' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T22:43:12.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:11 smithi152 ceph-mon[31181]: from='mgr.14100 172.21.15.152:0/488802770' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T22:43:12.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:11 smithi152 ceph-mon[31181]: from='mgr.14100 172.21.15.152:0/488802770' entity='mgr.x' 2024-09-11T22:43:12.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:11 smithi152 ceph-mon[31181]: from='mgr.14100 172.21.15.152:0/488802770' entity='mgr.x' 2024-09-11T22:43:12.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:11 smithi152 ceph-mon[31181]: from='mgr.14100 172.21.15.152:0/488802770' entity='mgr.x' 2024-09-11T22:43:13.976 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:13 smithi152 ceph-mon[31181]: mgrmap e3: x(active, since 1.00721s) 2024-09-11T22:43:14.584 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout 2024-09-11T22:43:14.584 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:43:14.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:43:14.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T22:43:14.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T22:43:14.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T22:43:14.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T22:43:14.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:14.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T22:43:14.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T22:43:14.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout 0 2024-09-11T22:43:14.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:43:14.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T22:43:14.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T22:43:14.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:43:14.585 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-11T22:43:14.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T22:43:14.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:43:14.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T22:43:14.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T22:43:14.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:14.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T22:43:14.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:43:14.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T22:43:14.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T22:43:14.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T22:43:14.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T22:43:14.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T22:43:14.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T22:43:14.586 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:14.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T22:43:14.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T22:43:14.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T22:43:14.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T22:43:14.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T22:43:14.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T22:43:14.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T22:43:14.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T22:43:14.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T22:43:14.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:14.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T22:43:14.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:43:14.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T22:43:00:379132+0000", 2024-09-11T22:43:14.587 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T22:43:14.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T22:43:14.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:14.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T22:43:14.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T22:43:14.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T22:43:14.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T22:43:14.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T22:43:14.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T22:43:14.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T22:43:14.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:43:14.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T22:43:14.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:14.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T22:43:14.588 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:43:14.589 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T22:43:00.379772+0000", 2024-09-11T22:43:14.589 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T22:43:14.589 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:43:14.589 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T22:43:14.589 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:43:14.589 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.720+0000 7f0c2bff0640 1 Processor -- start 2024-09-11T22:43:14.589 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.720+0000 7f0c2bff0640 1 -- start start 2024-09-11T22:43:14.589 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.720+0000 7f0c2bff0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c2407de70 0x7f0c2407c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:14.589 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.720+0000 7f0c2bff0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2407c8a0 con 0x7f0c2407de70 2024-09-11T22:43:14.589 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.720+0000 7f0c29d8c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c2407de70 0x7f0c2407c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:14.589 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.720+0000 7f0c29d8c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c2407de70 0x7f0c2407c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43198/0 (socket says 172.21.15.152:43198) 2024-09-11T22:43:14.589 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.720+0000 7f0c29d8c640 1 -- 172.21.15.152:0/2832892847 learned_addr learned my addr 172.21.15.152:0/2832892847 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:14.590 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.721+0000 7f0c29d8c640 1 -- 172.21.15.152:0/2832892847 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2407ca20 con 0x7f0c2407de70 2024-09-11T22:43:14.590 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.721+0000 7f0c29d8c640 1 --2- 172.21.15.152:0/2832892847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c2407de70 0x7f0c2407c2d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009820 tx=0x7f0c1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c06359fa75173d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:14.590 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.721+0000 7f0c28d8a640 1 -- 172.21.15.152:0/2832892847 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c14037070 con 0x7f0c2407de70 2024-09-11T22:43:14.590 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.721+0000 7f0c28d8a640 1 -- 172.21.15.152:0/2832892847 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0c14032900 con 0x7f0c2407de70 2024-09-11T22:43:14.590 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.721+0000 7f0c28d8a640 1 -- 172.21.15.152:0/2832892847 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c14030850 con 0x7f0c2407de70 2024-09-11T22:43:14.590 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.722+0000 7f0c2bff0640 1 -- 172.21.15.152:0/2832892847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c2407de70 msgr2=0x7f0c2407c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:14.590 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.722+0000 7f0c2bff0640 1 --2- 172.21.15.152:0/2832892847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c2407de70 0x7f0c2407c2d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009820 tx=0x7f0c1402a270 comp rx=0 tx=0).stop 2024-09-11T22:43:14.590 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.722+0000 7f0c2bff0640 1 -- 172.21.15.152:0/2832892847 shutdown_connections 2024-09-11T22:43:14.590 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.722+0000 7f0c2bff0640 1 --2- 172.21.15.152:0/2832892847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c2407de70 0x7f0c2407c2d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:14.590 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.722+0000 7f0c2bff0640 1 -- 172.21.15.152:0/2832892847 >> 172.21.15.152:0/2832892847 conn(0x7f0c24102db0 msgr2=0x7f0c241051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:14.590 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.722+0000 7f0c2bff0640 1 -- 172.21.15.152:0/2832892847 shutdown_connections 2024-09-11T22:43:14.590 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.722+0000 7f0c2bff0640 1 -- 172.21.15.152:0/2832892847 wait complete. 2024-09-11T22:43:14.590 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.723+0000 7f0c2bff0640 1 Processor -- start 2024-09-11T22:43:14.590 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.723+0000 7f0c2bff0640 1 -- start start 2024-09-11T22:43:14.591 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.723+0000 7f0c2bff0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c2407de70 0x7f0c2419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:14.591 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.723+0000 7f0c2bff0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2410dc90 con 0x7f0c2407de70 2024-09-11T22:43:14.591 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.723+0000 7f0c29d8c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c2407de70 0x7f0c2419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:14.591 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.723+0000 7f0c29d8c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c2407de70 0x7f0c2419d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43204/0 (socket says 172.21.15.152:43204) 2024-09-11T22:43:14.591 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.723+0000 7f0c29d8c640 1 -- 172.21.15.152:0/3962937864 learned_addr learned my addr 172.21.15.152:0/3962937864 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:14.591 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.723+0000 7f0c29d8c640 1 -- 172.21.15.152:0/3962937864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2419ddb0 con 0x7f0c2407de70 2024-09-11T22:43:14.591 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.723+0000 7f0c29d8c640 1 --2- 172.21.15.152:0/3962937864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c2407de70 0x7f0c2419d870 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0c14031000 tx=0x7f0c140326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:14.591 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.724+0000 7f0c1affd640 1 -- 172.21.15.152:0/3962937864 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c14042070 con 0x7f0c2407de70 2024-09-11T22:43:14.591 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.724+0000 7f0c1affd640 1 -- 172.21.15.152:0/3962937864 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0c14032be0 con 0x7f0c2407de70 2024-09-11T22:43:14.591 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.724+0000 7f0c2bff0640 1 -- 172.21.15.152:0/3962937864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2419e040 con 0x7f0c2407de70 2024-09-11T22:43:14.592 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.724+0000 7f0c1affd640 1 -- 172.21.15.152:0/3962937864 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c14037040 con 0x7f0c2407de70 2024-09-11T22:43:14.592 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.724+0000 7f0c2bff0640 1 -- 172.21.15.152:0/3962937864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c241a0d30 con 0x7f0c2407de70 2024-09-11T22:43:14.592 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.725+0000 7f0c1affd640 1 -- 172.21.15.152:0/3962937864 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f0c14045070 con 0x7f0c2407de70 2024-09-11T22:43:14.592 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.725+0000 7f0c2bff0640 1 -- 172.21.15.152:0/3962937864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c2407c350 con 0x7f0c2407de70 2024-09-11T22:43:14.592 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.725+0000 7f0c1affd640 1 --2- 172.21.15.152:0/3962937864 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f0c10038e10 0x7f0c1003b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:14.592 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.725+0000 7f0c1affd640 1 -- 172.21.15.152:0/3962937864 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0c1406bd30 con 0x7f0c2407de70 2024-09-11T22:43:14.592 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.725+0000 7f0c2958b640 1 --2- 172.21.15.152:0/3962937864 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f0c10038e10 0x7f0c1003b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:14.592 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.725+0000 7f0c2958b640 1 --2- 172.21.15.152:0/3962937864 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f0c10038e10 0x7f0c1003b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0c200098c0 tx=0x7f0c20006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:14.592 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.730+0000 7f0c1affd640 1 -- 172.21.15.152:0/3962937864 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c1402f2a0 con 0x7f0c2407de70 2024-09-11T22:43:14.592 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.859+0000 7f0c1affd640 1 -- 172.21.15.152:0/3962937864 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f0c140a11b0 con 0x7f0c2407de70 2024-09-11T22:43:14.592 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.964+0000 7f0c2bff0640 1 -- 172.21.15.152:0/3962937864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0c241a0f00 con 0x7f0c2407de70 2024-09-11T22:43:14.592 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.965+0000 7f0c1affd640 1 -- 172.21.15.152:0/3962937864 <== mon.0 v2:172.21.15.152:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f0c14042210 con 0x7f0c2407de70 2024-09-11T22:43:14.592 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.967+0000 7f0c2bff0640 1 -- 172.21.15.152:0/3962937864 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f0c10038e10 msgr2=0x7f0c1003b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:14.593 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.967+0000 7f0c2bff0640 1 --2- 172.21.15.152:0/3962937864 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f0c10038e10 0x7f0c1003b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0c200098c0 tx=0x7f0c20006db0 comp rx=0 tx=0).stop 2024-09-11T22:43:14.593 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.967+0000 7f0c2bff0640 1 -- 172.21.15.152:0/3962937864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c2407de70 msgr2=0x7f0c2419d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:14.593 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.967+0000 7f0c2bff0640 1 --2- 172.21.15.152:0/3962937864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c2407de70 0x7f0c2419d870 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0c14031000 tx=0x7f0c140326e0 comp rx=0 tx=0).stop 2024-09-11T22:43:14.593 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.967+0000 7f0c2bff0640 1 -- 172.21.15.152:0/3962937864 shutdown_connections 2024-09-11T22:43:14.593 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.967+0000 7f0c2bff0640 1 --2- 172.21.15.152:0/3962937864 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f0c10038e10 0x7f0c1003b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:14.593 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.967+0000 7f0c2bff0640 1 --2- 172.21.15.152:0/3962937864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c2407de70 0x7f0c2419d870 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:14.593 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.967+0000 7f0c2bff0640 1 -- 172.21.15.152:0/3962937864 >> 172.21.15.152:0/3962937864 conn(0x7f0c24102db0 msgr2=0x7f0c241039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:14.593 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.967+0000 7f0c2bff0640 1 -- 172.21.15.152:0/3962937864 shutdown_connections 2024-09-11T22:43:14.593 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:13.967+0000 7f0c2bff0640 1 -- 172.21.15.152:0/3962937864 wait complete. 2024-09-11T22:43:14.593 INFO:teuthology.orchestra.run.smithi152.stdout:mgr is available 2024-09-11T22:43:15.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:14 smithi152 ceph-mon[31181]: mgrmap e4: x(active, since 2s) 2024-09-11T22:43:15.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:14 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3962937864' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T22:43:18.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:18 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/348033059' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T22:43:24.248 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout 2024-09-11T22:43:24.248 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T22:43:24.248 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout fsid = 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:43:24.248 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.152:3300,v1:172.21.15.152:6789] 2024-09-11T22:43:24.248 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T22:43:24.248 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T22:43:24.248 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T22:43:24.249 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T22:43:24.249 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout 2024-09-11T22:43:24.249 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T22:43:24.249 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T22:43:24.249 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout 2024-09-11T22:43:24.249 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T22:43:24.249 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T22:43:24.249 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T22:43:24.249 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T22:43:24.249 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T22:43:24.249 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T22:43:24.249 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.808+0000 7f8bfefc4640 1 Processor -- start 2024-09-11T22:43:24.249 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.808+0000 7f8bfefc4640 1 -- start start 2024-09-11T22:43:24.250 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.808+0000 7f8bfefc4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf8108080 0x7f8bf810a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:24.250 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.808+0000 7f8bfefc4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf807be30 con 0x7f8bf8108080 2024-09-11T22:43:24.250 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.808+0000 7f8bfcd60640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf8108080 0x7f8bf810a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:24.250 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.808+0000 7f8bfcd60640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf8108080 0x7f8bf810a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43218/0 (socket says 172.21.15.152:43218) 2024-09-11T22:43:24.250 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.808+0000 7f8bfcd60640 1 -- 172.21.15.152:0/2836609859 learned_addr learned my addr 172.21.15.152:0/2836609859 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:24.250 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.808+0000 7f8bfcd60640 1 -- 172.21.15.152:0/2836609859 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf810a9b0 con 0x7f8bf8108080 2024-09-11T22:43:24.250 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.809+0000 7f8bfcd60640 1 --2- 172.21.15.152:0/2836609859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf8108080 0x7f8bf810a470 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009820 tx=0x7f8bec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad88a229149ffd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:24.250 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.809+0000 7f8bf77fe640 1 -- 172.21.15.152:0/2836609859 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bec037070 con 0x7f8bf8108080 2024-09-11T22:43:24.250 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.809+0000 7f8bf77fe640 1 -- 172.21.15.152:0/2836609859 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8bec032900 con 0x7f8bf8108080 2024-09-11T22:43:24.250 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.809+0000 7f8bf77fe640 1 -- 172.21.15.152:0/2836609859 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bec030850 con 0x7f8bf8108080 2024-09-11T22:43:24.250 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.810+0000 7f8bfefc4640 1 -- 172.21.15.152:0/2836609859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf8108080 msgr2=0x7f8bf810a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:24.250 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.810+0000 7f8bfefc4640 1 --2- 172.21.15.152:0/2836609859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf8108080 0x7f8bf810a470 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009820 tx=0x7f8bec02a270 comp rx=0 tx=0).stop 2024-09-11T22:43:24.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.810+0000 7f8bfefc4640 1 -- 172.21.15.152:0/2836609859 shutdown_connections 2024-09-11T22:43:24.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.810+0000 7f8bfefc4640 1 --2- 172.21.15.152:0/2836609859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf8108080 0x7f8bf810a470 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:24.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.810+0000 7f8bfefc4640 1 -- 172.21.15.152:0/2836609859 >> 172.21.15.152:0/2836609859 conn(0x7f8bf8101ca0 msgr2=0x7f8bf81040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:24.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.810+0000 7f8bfefc4640 1 -- 172.21.15.152:0/2836609859 shutdown_connections 2024-09-11T22:43:24.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.810+0000 7f8bfefc4640 1 -- 172.21.15.152:0/2836609859 wait complete. 2024-09-11T22:43:24.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.811+0000 7f8bfefc4640 1 Processor -- start 2024-09-11T22:43:24.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.811+0000 7f8bfefc4640 1 -- start start 2024-09-11T22:43:24.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.811+0000 7f8bfefc4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf8108080 0x7f8bf819b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:24.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.811+0000 7f8bfefc4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8bf810b9f0 con 0x7f8bf8108080 2024-09-11T22:43:24.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.811+0000 7f8bfcd60640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf8108080 0x7f8bf819b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:24.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.811+0000 7f8bfcd60640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf8108080 0x7f8bf819b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43222/0 (socket says 172.21.15.152:43222) 2024-09-11T22:43:24.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.811+0000 7f8bfcd60640 1 -- 172.21.15.152:0/348033059 learned_addr learned my addr 172.21.15.152:0/348033059 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:24.251 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.812+0000 7f8bfcd60640 1 -- 172.21.15.152:0/348033059 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bf819bb70 con 0x7f8bf8108080 2024-09-11T22:43:24.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.812+0000 7f8bfcd60640 1 --2- 172.21.15.152:0/348033059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf8108080 0x7f8bf819b630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8bec031000 tx=0x7f8bec0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:24.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.812+0000 7f8bf5ffb640 1 -- 172.21.15.152:0/348033059 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bec042070 con 0x7f8bf8108080 2024-09-11T22:43:24.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.812+0000 7f8bf5ffb640 1 -- 172.21.15.152:0/348033059 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8bec032be0 con 0x7f8bf8108080 2024-09-11T22:43:24.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.812+0000 7f8bfefc4640 1 -- 172.21.15.152:0/348033059 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bf819be00 con 0x7f8bf8108080 2024-09-11T22:43:24.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.812+0000 7f8bf5ffb640 1 -- 172.21.15.152:0/348033059 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bec037040 con 0x7f8bf8108080 2024-09-11T22:43:24.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.812+0000 7f8bfefc4640 1 -- 172.21.15.152:0/348033059 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bf819eaf0 con 0x7f8bf8108080 2024-09-11T22:43:24.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.813+0000 7f8bf5ffb640 1 -- 172.21.15.152:0/348033059 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f8bec045070 con 0x7f8bf8108080 2024-09-11T22:43:24.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.813+0000 7f8bf5ffb640 1 --2- 172.21.15.152:0/348033059 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f8be8038f70 0x7f8be803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:24.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.813+0000 7f8bfefc4640 1 -- 172.21.15.152:0/348033059 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bf81084e0 con 0x7f8bf8108080 2024-09-11T22:43:24.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.813+0000 7f8bf5ffb640 1 -- 172.21.15.152:0/348033059 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8bec06cf80 con 0x7f8bf8108080 2024-09-11T22:43:24.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.814+0000 7f8bf7fff640 1 --2- 172.21.15.152:0/348033059 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f8be8038f70 0x7f8be803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:24.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.814+0000 7f8bf7fff640 1 --2- 172.21.15.152:0/348033059 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f8be8038f70 0x7f8be803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8be40098c0 tx=0x7f8be4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:24.252 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.818+0000 7f8bf5ffb640 1 -- 172.21.15.152:0/348033059 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8bec042210 con 0x7f8bf8108080 2024-09-11T22:43:24.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.991+0000 7f8bfefc4640 1 -- 172.21.15.152:0/348033059 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f8bf819ecc0 con 0x7f8bf8108080 2024-09-11T22:43:24.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.992+0000 7f8bf5ffb640 1 -- 172.21.15.152:0/348033059 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f8bec032d90 con 0x7f8bf8108080 2024-09-11T22:43:24.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.995+0000 7f8bfefc4640 1 -- 172.21.15.152:0/348033059 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f8be8038f70 msgr2=0x7f8be803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:24.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.995+0000 7f8bfefc4640 1 --2- 172.21.15.152:0/348033059 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f8be8038f70 0x7f8be803b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8be40098c0 tx=0x7f8be4006db0 comp rx=0 tx=0).stop 2024-09-11T22:43:24.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.995+0000 7f8bfefc4640 1 -- 172.21.15.152:0/348033059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf8108080 msgr2=0x7f8bf819b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:24.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.995+0000 7f8bfefc4640 1 --2- 172.21.15.152:0/348033059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf8108080 0x7f8bf819b630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8bec031000 tx=0x7f8bec0326e0 comp rx=0 tx=0).stop 2024-09-11T22:43:24.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.995+0000 7f8bfefc4640 1 -- 172.21.15.152:0/348033059 shutdown_connections 2024-09-11T22:43:24.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.995+0000 7f8bfefc4640 1 --2- 172.21.15.152:0/348033059 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f8be8038f70 0x7f8be803b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:24.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.995+0000 7f8bfefc4640 1 --2- 172.21.15.152:0/348033059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8bf8108080 0x7f8bf819b630 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:24.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.995+0000 7f8bfefc4640 1 -- 172.21.15.152:0/348033059 >> 172.21.15.152:0/348033059 conn(0x7f8bf8101ca0 msgr2=0x7f8bf8102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:24.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.995+0000 7f8bfefc4640 1 -- 172.21.15.152:0/348033059 shutdown_connections 2024-09-11T22:43:24.253 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:17.995+0000 7f8bfefc4640 1 -- 172.21.15.152:0/348033059 wait complete. 2024-09-11T22:43:24.253 INFO:teuthology.orchestra.run.smithi152.stdout:Enabling cephadm module... 2024-09-11T22:43:27.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:27 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3198005034' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T22:43:27.373 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:27 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: ignoring --setuser ceph since I am not root 2024-09-11T22:43:27.373 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:27 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: ignoring --setgroup ceph since I am not root 2024-09-11T22:43:27.373 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:27 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:27.264+0000 7f1f9d138040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:43:27.373 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:27 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:27.320+0000 7f1f9d138040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:43:27.749 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:27 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:27.470+0000 7f1f9d138040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:43:27.749 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:27 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:27.536+0000 7f1f9d138040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:43:27.749 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:27 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:27.682+0000 7f1f9d138040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:43:27.760 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.853+0000 7f8c522f9640 1 Processor -- start 2024-09-11T22:43:27.760 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.853+0000 7f8c522f9640 1 -- start start 2024-09-11T22:43:27.760 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.853+0000 7f8c522f9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c4c1080d0 0x7f8c4c10a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:27.760 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.853+0000 7f8c522f9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c07be40 con 0x7f8c4c1080d0 2024-09-11T22:43:27.760 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.853+0000 7f8c4bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c4c1080d0 0x7f8c4c10a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:27.760 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.853+0000 7f8c4bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c4c1080d0 0x7f8c4c10a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50166/0 (socket says 172.21.15.152:50166) 2024-09-11T22:43:27.761 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.853+0000 7f8c4bfff640 1 -- 172.21.15.152:0/3487577278 learned_addr learned my addr 172.21.15.152:0/3487577278 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:27.761 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.854+0000 7f8c4bfff640 1 -- 172.21.15.152:0/3487577278 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4c10aa00 con 0x7f8c4c1080d0 2024-09-11T22:43:27.761 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.854+0000 7f8c4bfff640 1 --2- 172.21.15.152:0/3487577278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c4c1080d0 0x7f8c4c10a4c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009820 tx=0x7f8c3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3245b620348e8fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:27.761 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.855+0000 7f8c4affd640 1 -- 172.21.15.152:0/3487577278 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c3c037070 con 0x7f8c4c1080d0 2024-09-11T22:43:27.761 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.855+0000 7f8c4affd640 1 -- 172.21.15.152:0/3487577278 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8c3c032900 con 0x7f8c4c1080d0 2024-09-11T22:43:27.761 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.855+0000 7f8c4affd640 1 -- 172.21.15.152:0/3487577278 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c3c030850 con 0x7f8c4c1080d0 2024-09-11T22:43:27.761 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.855+0000 7f8c522f9640 1 -- 172.21.15.152:0/3487577278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c4c1080d0 msgr2=0x7f8c4c10a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:27.761 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.855+0000 7f8c522f9640 1 --2- 172.21.15.152:0/3487577278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c4c1080d0 0x7f8c4c10a4c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009820 tx=0x7f8c3c02a270 comp rx=0 tx=0).stop 2024-09-11T22:43:27.761 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.855+0000 7f8c522f9640 1 -- 172.21.15.152:0/3487577278 shutdown_connections 2024-09-11T22:43:27.761 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.855+0000 7f8c522f9640 1 --2- 172.21.15.152:0/3487577278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c4c1080d0 0x7f8c4c10a4c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:27.761 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.855+0000 7f8c522f9640 1 -- 172.21.15.152:0/3487577278 >> 172.21.15.152:0/3487577278 conn(0x7f8c4c101cb0 msgr2=0x7f8c4c1040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:27.761 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.855+0000 7f8c522f9640 1 -- 172.21.15.152:0/3487577278 shutdown_connections 2024-09-11T22:43:27.761 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.855+0000 7f8c522f9640 1 -- 172.21.15.152:0/3487577278 wait complete. 2024-09-11T22:43:27.762 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.856+0000 7f8c522f9640 1 Processor -- start 2024-09-11T22:43:27.762 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.856+0000 7f8c522f9640 1 -- start start 2024-09-11T22:43:27.762 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.856+0000 7f8c522f9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c4c1080d0 0x7f8c4c19b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:27.762 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.856+0000 7f8c522f9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c10ba40 con 0x7f8c4c1080d0 2024-09-11T22:43:27.762 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.857+0000 7f8c4bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c4c1080d0 0x7f8c4c19b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:27.762 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.857+0000 7f8c4bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c4c1080d0 0x7f8c4c19b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50180/0 (socket says 172.21.15.152:50180) 2024-09-11T22:43:27.762 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.857+0000 7f8c4bfff640 1 -- 172.21.15.152:0/3198005034 learned_addr learned my addr 172.21.15.152:0/3198005034 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:27.762 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.857+0000 7f8c4bfff640 1 -- 172.21.15.152:0/3198005034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4c19bb60 con 0x7f8c4c1080d0 2024-09-11T22:43:27.762 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.857+0000 7f8c4bfff640 1 --2- 172.21.15.152:0/3198005034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c4c1080d0 0x7f8c4c19b620 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c031000 tx=0x7f8c3c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:27.762 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.857+0000 7f8c497fa640 1 -- 172.21.15.152:0/3198005034 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c3c042070 con 0x7f8c4c1080d0 2024-09-11T22:43:27.762 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.857+0000 7f8c497fa640 1 -- 172.21.15.152:0/3198005034 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8c3c032be0 con 0x7f8c4c1080d0 2024-09-11T22:43:27.762 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.857+0000 7f8c522f9640 1 -- 172.21.15.152:0/3198005034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c4c19bdf0 con 0x7f8c4c1080d0 2024-09-11T22:43:27.762 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.857+0000 7f8c497fa640 1 -- 172.21.15.152:0/3198005034 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c3c037040 con 0x7f8c4c1080d0 2024-09-11T22:43:27.763 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.857+0000 7f8c522f9640 1 -- 172.21.15.152:0/3198005034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c4c19eae0 con 0x7f8c4c1080d0 2024-09-11T22:43:27.763 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.858+0000 7f8c497fa640 1 -- 172.21.15.152:0/3198005034 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f8c3c045070 con 0x7f8c4c1080d0 2024-09-11T22:43:27.763 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.858+0000 7f8c522f9640 1 -- 172.21.15.152:0/3198005034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c4c108530 con 0x7f8c4c1080d0 2024-09-11T22:43:27.763 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.858+0000 7f8c497fa640 1 --2- 172.21.15.152:0/3198005034 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f8c34038f20 0x7f8c3403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:27.763 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.858+0000 7f8c497fa640 1 -- 172.21.15.152:0/3198005034 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8c3c06c030 con 0x7f8c4c1080d0 2024-09-11T22:43:27.763 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.859+0000 7f8c4b7fe640 1 --2- 172.21.15.152:0/3198005034 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f8c34038f20 0x7f8c3403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:27.763 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.859+0000 7f8c4b7fe640 1 --2- 172.21.15.152:0/3198005034 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f8c34038f20 0x7f8c3403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8c400098c0 tx=0x7f8c40006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:27.763 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:26.863+0000 7f8c497fa640 1 -- 172.21.15.152:0/3198005034 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c3c03a380 con 0x7f8c4c1080d0 2024-09-11T22:43:27.763 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:27.074+0000 7f8c522f9640 1 -- 172.21.15.152:0/3198005034 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f8c4c19ecb0 con 0x7f8c4c1080d0 2024-09-11T22:43:27.763 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:27.126+0000 7f8c497fa640 1 -- 172.21.15.152:0/3198005034 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f8c3c04c070 con 0x7f8c4c1080d0 2024-09-11T22:43:27.763 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:27.127+0000 7f8c497fa640 1 -- 172.21.15.152:0/3198005034 <== mon.0 v2:172.21.15.152:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f8c3c047440 con 0x7f8c4c1080d0 2024-09-11T22:43:27.763 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:27.128+0000 7f8c522f9640 1 -- 172.21.15.152:0/3198005034 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f8c34038f20 msgr2=0x7f8c3403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:27.763 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:27.129+0000 7f8c522f9640 1 --2- 172.21.15.152:0/3198005034 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f8c34038f20 0x7f8c3403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8c400098c0 tx=0x7f8c40006db0 comp rx=0 tx=0).stop 2024-09-11T22:43:27.764 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:27.129+0000 7f8c522f9640 1 -- 172.21.15.152:0/3198005034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c4c1080d0 msgr2=0x7f8c4c19b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:27.764 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:27.129+0000 7f8c522f9640 1 --2- 172.21.15.152:0/3198005034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c4c1080d0 0x7f8c4c19b620 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c031000 tx=0x7f8c3c0326e0 comp rx=0 tx=0).stop 2024-09-11T22:43:27.764 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:27.129+0000 7f8c522f9640 1 -- 172.21.15.152:0/3198005034 shutdown_connections 2024-09-11T22:43:27.764 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:27.129+0000 7f8c522f9640 1 --2- 172.21.15.152:0/3198005034 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f8c34038f20 0x7f8c3403b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:27.764 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:27.129+0000 7f8c522f9640 1 --2- 172.21.15.152:0/3198005034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c4c1080d0 0x7f8c4c19b620 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:27.764 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:27.129+0000 7f8c522f9640 1 -- 172.21.15.152:0/3198005034 >> 172.21.15.152:0/3198005034 conn(0x7f8c4c101cb0 msgr2=0x7f8c4c1028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:27.764 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:27.129+0000 7f8c522f9640 1 -- 172.21.15.152:0/3198005034 shutdown_connections 2024-09-11T22:43:27.764 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:27.129+0000 7f8c522f9640 1 -- 172.21.15.152:0/3198005034 wait complete. 2024-09-11T22:43:28.077 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:27 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:27.827+0000 7f1f9d138040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:43:28.363 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:28 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3198005034' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T22:43:28.363 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:28 smithi152 ceph-mon[31181]: mgrmap e5: x(active, since 15s) 2024-09-11T22:43:28.363 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:28 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:28.150+0000 7f1f9d138040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:43:28.363 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:28 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:28.216+0000 7f1f9d138040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:43:28.622 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:28 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:28.408+0000 7f1f9d138040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:43:29.381 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:29 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2661519058' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T22:43:29.381 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:29 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:29.294+0000 7f1f9d138040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:43:29.381 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:29 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:29.361+0000 7f1f9d138040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:43:29.632 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:43:29.632 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T22:43:29.632 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T22:43:29.632 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T22:43:29.632 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T22:43:29.632 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:43:29.633 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.155+0000 7f94ee6e8640 1 Processor -- start 2024-09-11T22:43:29.633 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.155+0000 7f94ee6e8640 1 -- start start 2024-09-11T22:43:29.633 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.155+0000 7f94ee6e8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94e8108100 0x7f94e810a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:29.633 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.155+0000 7f94ee6e8640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f94e807bdb0 con 0x7f94e8108100 2024-09-11T22:43:29.633 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.156+0000 7f94e7fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94e8108100 0x7f94e810a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:29.633 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.156+0000 7f94e7fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94e8108100 0x7f94e810a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50212/0 (socket says 172.21.15.152:50212) 2024-09-11T22:43:29.633 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.156+0000 7f94e7fff640 1 -- 172.21.15.152:0/1171671076 learned_addr learned my addr 172.21.15.152:0/1171671076 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:29.633 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.156+0000 7f94e7fff640 1 -- 172.21.15.152:0/1171671076 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e810aa30 con 0x7f94e8108100 2024-09-11T22:43:29.633 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.157+0000 7f94e7fff640 1 --2- 172.21.15.152:0/1171671076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94e8108100 0x7f94e810a4f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009820 tx=0x7f94d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28431ccffb5b4af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:29.633 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.157+0000 7f94e6ffd640 1 -- 172.21.15.152:0/1171671076 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94d8037070 con 0x7f94e8108100 2024-09-11T22:43:29.633 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.157+0000 7f94e6ffd640 1 -- 172.21.15.152:0/1171671076 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f94d8032900 con 0x7f94e8108100 2024-09-11T22:43:29.633 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.157+0000 7f94e6ffd640 1 -- 172.21.15.152:0/1171671076 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94d8030850 con 0x7f94e8108100 2024-09-11T22:43:29.634 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.157+0000 7f94ee6e8640 1 -- 172.21.15.152:0/1171671076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94e8108100 msgr2=0x7f94e810a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:29.634 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.157+0000 7f94ee6e8640 1 --2- 172.21.15.152:0/1171671076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94e8108100 0x7f94e810a4f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009820 tx=0x7f94d802a270 comp rx=0 tx=0).stop 2024-09-11T22:43:29.634 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.157+0000 7f94ee6e8640 1 -- 172.21.15.152:0/1171671076 shutdown_connections 2024-09-11T22:43:29.634 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.157+0000 7f94ee6e8640 1 --2- 172.21.15.152:0/1171671076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94e8108100 0x7f94e810a4f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:29.634 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.157+0000 7f94ee6e8640 1 -- 172.21.15.152:0/1171671076 >> 172.21.15.152:0/1171671076 conn(0x7f94e8101ca0 msgr2=0x7f94e81040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:29.634 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.158+0000 7f94ee6e8640 1 -- 172.21.15.152:0/1171671076 shutdown_connections 2024-09-11T22:43:29.634 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.158+0000 7f94ee6e8640 1 -- 172.21.15.152:0/1171671076 wait complete. 2024-09-11T22:43:29.634 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.158+0000 7f94ee6e8640 1 Processor -- start 2024-09-11T22:43:29.634 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.158+0000 7f94ee6e8640 1 -- start start 2024-09-11T22:43:29.634 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.158+0000 7f94ee6e8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94e8108100 0x7f94e819b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:29.634 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.158+0000 7f94ee6e8640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f94e810b9b0 con 0x7f94e8108100 2024-09-11T22:43:29.634 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.159+0000 7f94e7fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94e8108100 0x7f94e819b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:29.634 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.159+0000 7f94e7fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94e8108100 0x7f94e819b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50222/0 (socket says 172.21.15.152:50222) 2024-09-11T22:43:29.635 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.159+0000 7f94e7fff640 1 -- 172.21.15.152:0/2661519058 learned_addr learned my addr 172.21.15.152:0/2661519058 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:29.635 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.159+0000 7f94e7fff640 1 -- 172.21.15.152:0/2661519058 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e819bb70 con 0x7f94e8108100 2024-09-11T22:43:29.635 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.159+0000 7f94e7fff640 1 --2- 172.21.15.152:0/2661519058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94e8108100 0x7f94e819b630 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f94d8031000 tx=0x7f94d80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:29.635 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.159+0000 7f94e57fa640 1 -- 172.21.15.152:0/2661519058 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94d8042070 con 0x7f94e8108100 2024-09-11T22:43:29.635 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.159+0000 7f94ee6e8640 1 -- 172.21.15.152:0/2661519058 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e819be00 con 0x7f94e8108100 2024-09-11T22:43:29.635 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.159+0000 7f94e57fa640 1 -- 172.21.15.152:0/2661519058 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f94d8032be0 con 0x7f94e8108100 2024-09-11T22:43:29.635 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.159+0000 7f94e57fa640 1 -- 172.21.15.152:0/2661519058 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94d8037040 con 0x7f94e8108100 2024-09-11T22:43:29.635 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.159+0000 7f94ee6e8640 1 -- 172.21.15.152:0/2661519058 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e819eaf0 con 0x7f94e8108100 2024-09-11T22:43:29.635 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.160+0000 7f94ee6e8640 1 -- 172.21.15.152:0/2661519058 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94e8108560 con 0x7f94e8108100 2024-09-11T22:43:29.635 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.160+0000 7f94e57fa640 1 -- 172.21.15.152:0/2661519058 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f94d8045070 con 0x7f94e8108100 2024-09-11T22:43:29.635 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.160+0000 7f94e57fa640 1 --2- 172.21.15.152:0/2661519058 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f94d0038f70 0x7f94d003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:29.635 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.161+0000 7f94e57fa640 1 -- 172.21.15.152:0/2661519058 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f94d806bfe0 con 0x7f94e8108100 2024-09-11T22:43:29.636 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.161+0000 7f94e77fe640 1 -- 172.21.15.152:0/2661519058 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f94d0038f70 msgr2=0x7f94d003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/2200357575 2024-09-11T22:43:29.636 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.161+0000 7f94e77fe640 1 --2- 172.21.15.152:0/2661519058 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f94d0038f70 0x7f94d003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:43:29.636 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.170+0000 7f94e57fa640 1 -- 172.21.15.152:0/2661519058 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f94d8042210 con 0x7f94e8108100 2024-09-11T22:43:29.636 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.362+0000 7f94e77fe640 1 -- 172.21.15.152:0/2661519058 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f94d0038f70 msgr2=0x7f94d003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/2200357575 2024-09-11T22:43:29.636 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.362+0000 7f94e77fe640 1 --2- 172.21.15.152:0/2661519058 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f94d0038f70 0x7f94d003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:43:29.636 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.383+0000 7f94ee6e8640 1 -- 172.21.15.152:0/2661519058 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f94e819f0c0 con 0x7f94e8108100 2024-09-11T22:43:29.636 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.383+0000 7f94e57fa640 1 -- 172.21.15.152:0/2661519058 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f94d8032d90 con 0x7f94e8108100 2024-09-11T22:43:29.636 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.385+0000 7f94ee6e8640 1 -- 172.21.15.152:0/2661519058 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f94d0038f70 msgr2=0x7f94d003b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:43:29.636 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.385+0000 7f94ee6e8640 1 --2- 172.21.15.152:0/2661519058 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f94d0038f70 0x7f94d003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:29.636 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.385+0000 7f94ee6e8640 1 -- 172.21.15.152:0/2661519058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94e8108100 msgr2=0x7f94e819b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:29.636 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.385+0000 7f94ee6e8640 1 --2- 172.21.15.152:0/2661519058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94e8108100 0x7f94e819b630 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f94d8031000 tx=0x7f94d80326e0 comp rx=0 tx=0).stop 2024-09-11T22:43:29.636 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.386+0000 7f94ee6e8640 1 -- 172.21.15.152:0/2661519058 shutdown_connections 2024-09-11T22:43:29.636 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.386+0000 7f94ee6e8640 1 --2- 172.21.15.152:0/2661519058 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7f94d0038f70 0x7f94d003b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:29.637 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.386+0000 7f94ee6e8640 1 --2- 172.21.15.152:0/2661519058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94e8108100 0x7f94e819b630 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:29.637 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.386+0000 7f94ee6e8640 1 -- 172.21.15.152:0/2661519058 >> 172.21.15.152:0/2661519058 conn(0x7f94e8101ca0 msgr2=0x7f94e8102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:29.637 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.386+0000 7f94ee6e8640 1 -- 172.21.15.152:0/2661519058 shutdown_connections 2024-09-11T22:43:29.637 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:28.386+0000 7f94ee6e8640 1 -- 172.21.15.152:0/2661519058 wait complete. 2024-09-11T22:43:29.637 INFO:teuthology.orchestra.run.smithi152.stdout:Waiting for the mgr to restart... 2024-09-11T22:43:29.637 INFO:teuthology.orchestra.run.smithi152.stdout:Waiting for mgr epoch 5... 2024-09-11T22:43:30.354 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:30 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:30.097+0000 7f1f9d138040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:43:30.354 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:30 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:30.211+0000 7f1f9d138040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:43:30.622 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:30 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:30.353+0000 7f1f9d138040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:43:30.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:30 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:30.413+0000 7f1f9d138040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:43:30.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:30 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:30.560+0000 7f1f9d138040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:43:30.913 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:30 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:30.627+0000 7f1f9d138040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:43:30.913 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:30 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:30.687+0000 7f1f9d138040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:43:30.913 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:30 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:30.844+0000 7f1f9d138040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:43:30.913 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:30 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:30.912+0000 7f1f9d138040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:43:31.177 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:30 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:30.978+0000 7f1f9d138040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:43:31.177 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:31 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:31.060+0000 7f1f9d138040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:43:31.177 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:31 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:31.118+0000 7f1f9d138040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:43:31.177 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:31 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:31.176+0000 7f1f9d138040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:43:32.372 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:31 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:31.968+0000 7f1f9d138040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:43:32.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:32 smithi152 ceph-mon[31181]: Active manager daemon x restarted 2024-09-11T22:43:32.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:32 smithi152 ceph-mon[31181]: Activating manager daemon x 2024-09-11T22:43:32.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:32 smithi152 ceph-mon[31181]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T22:43:32.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:32 smithi152 ceph-mon[31181]: mgrmap e6: x(active, starting, since 0.00415445s) 2024-09-11T22:43:32.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:32 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:43:32.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:32 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T22:43:32.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:32 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T22:43:32.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:32 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T22:43:32.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:32 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T22:43:32.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:32 smithi152 ceph-mon[31181]: Manager daemon x is now available 2024-09-11T22:43:32.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:32 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' 2024-09-11T22:43:32.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:32 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' 2024-09-11T22:43:32.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:32 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:43:32.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:32 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:43:32.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:32 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T22:43:32.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:32 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T22:43:33.296 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-mon[31181]: Found migration_current of "None". Setting to last migration. 2024-09-11T22:43:33.296 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' 2024-09-11T22:43:33.296 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' 2024-09-11T22:43:33.296 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-mon[31181]: mgrmap e7: x(active, since 1.00591s) 2024-09-11T22:43:33.548 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:33.358+0000 7f1f04492640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:22:43:33] ENGINE Error in HTTPServer.serve 2024-09-11T22:43:33.548 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: Traceback (most recent call last): 2024-09-11T22:43:33.548 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:43:33.548 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: self._connections.run(self.expiration_interval) 2024-09-11T22:43:33.548 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:43:33.548 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: self._run(expiration_interval) 2024-09-11T22:43:33.548 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:43:33.548 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:43:33.549 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:43:33.549 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:43:33.549 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:43:33.549 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: s = self.context.wrap_socket( 2024-09-11T22:43:33.549 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:43:33.549 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: return self.sslsocket_class._create( 2024-09-11T22:43:33.549 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:43:33.549 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: self.do_handshake() 2024-09-11T22:43:33.549 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:43:33.549 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: self._sslobj.do_handshake() 2024-09-11T22:43:33.549 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:43:33.550 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:33.558 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:43:33.558 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T22:43:33.558 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T22:43:33.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:43:33.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.063+0000 7fac227ae640 1 Processor -- start 2024-09-11T22:43:33.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.063+0000 7fac227ae640 1 -- start start 2024-09-11T22:43:33.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.063+0000 7fac227ae640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1080b0 0x7fac1c10a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:33.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.063+0000 7fac227ae640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c07be40 con 0x7fac1c1080b0 2024-09-11T22:43:33.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.063+0000 7fac1bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1080b0 0x7fac1c10a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:33.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.063+0000 7fac1bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1080b0 0x7fac1c10a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56288/0 (socket says 172.21.15.152:56288) 2024-09-11T22:43:33.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.063+0000 7fac1bfff640 1 -- 172.21.15.152:0/2151225580 learned_addr learned my addr 172.21.15.152:0/2151225580 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:33.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.063+0000 7fac1bfff640 1 -- 172.21.15.152:0/2151225580 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c10a9e0 con 0x7fac1c1080b0 2024-09-11T22:43:33.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.064+0000 7fac1bfff640 1 --2- 172.21.15.152:0/2151225580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1080b0 0x7fac1c10a4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009820 tx=0x7fac0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64595c8ebc36f30 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:33.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.065+0000 7fac1affd640 1 -- 172.21.15.152:0/2151225580 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac0c037070 con 0x7fac1c1080b0 2024-09-11T22:43:33.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.065+0000 7fac1affd640 1 -- 172.21.15.152:0/2151225580 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac0c032900 con 0x7fac1c1080b0 2024-09-11T22:43:33.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.065+0000 7fac1affd640 1 -- 172.21.15.152:0/2151225580 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac0c030850 con 0x7fac1c1080b0 2024-09-11T22:43:33.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.065+0000 7fac227ae640 1 -- 172.21.15.152:0/2151225580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1080b0 msgr2=0x7fac1c10a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:33.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.065+0000 7fac227ae640 1 --2- 172.21.15.152:0/2151225580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1080b0 0x7fac1c10a4a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009820 tx=0x7fac0c02a270 comp rx=0 tx=0).stop 2024-09-11T22:43:33.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.065+0000 7fac227ae640 1 -- 172.21.15.152:0/2151225580 shutdown_connections 2024-09-11T22:43:33.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.065+0000 7fac227ae640 1 --2- 172.21.15.152:0/2151225580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1080b0 0x7fac1c10a4a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:33.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.065+0000 7fac227ae640 1 -- 172.21.15.152:0/2151225580 >> 172.21.15.152:0/2151225580 conn(0x7fac1c101d10 msgr2=0x7fac1c104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:33.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.065+0000 7fac227ae640 1 -- 172.21.15.152:0/2151225580 shutdown_connections 2024-09-11T22:43:33.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.065+0000 7fac227ae640 1 -- 172.21.15.152:0/2151225580 wait complete. 2024-09-11T22:43:33.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.066+0000 7fac227ae640 1 Processor -- start 2024-09-11T22:43:33.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.066+0000 7fac227ae640 1 -- start start 2024-09-11T22:43:33.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.066+0000 7fac227ae640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1080b0 0x7fac1c19b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:33.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.066+0000 7fac227ae640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c10ba20 con 0x7fac1c1080b0 2024-09-11T22:43:33.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.066+0000 7fac1bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1080b0 0x7fac1c19b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:33.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.066+0000 7fac1bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1080b0 0x7fac1c19b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56300/0 (socket says 172.21.15.152:56300) 2024-09-11T22:43:33.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.066+0000 7fac1bfff640 1 -- 172.21.15.152:0/2530677813 learned_addr learned my addr 172.21.15.152:0/2530677813 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:33.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.067+0000 7fac1bfff640 1 -- 172.21.15.152:0/2530677813 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c19bb10 con 0x7fac1c1080b0 2024-09-11T22:43:33.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.067+0000 7fac1bfff640 1 --2- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1080b0 0x7fac1c19b5d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fac0c032ce0 tx=0x7fac0c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:33.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.067+0000 7fac197fa640 1 -- 172.21.15.152:0/2530677813 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac0c042070 con 0x7fac1c1080b0 2024-09-11T22:43:33.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.067+0000 7fac197fa640 1 -- 172.21.15.152:0/2530677813 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac0c0308c0 con 0x7fac1c1080b0 2024-09-11T22:43:33.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.067+0000 7fac227ae640 1 -- 172.21.15.152:0/2530677813 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1c19bda0 con 0x7fac1c1080b0 2024-09-11T22:43:33.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.067+0000 7fac197fa640 1 -- 172.21.15.152:0/2530677813 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac0c037040 con 0x7fac1c1080b0 2024-09-11T22:43:33.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.067+0000 7fac227ae640 1 -- 172.21.15.152:0/2530677813 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac1c19ea90 con 0x7fac1c1080b0 2024-09-11T22:43:33.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.068+0000 7fac197fa640 1 -- 172.21.15.152:0/2530677813 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fac0c045070 con 0x7fac1c1080b0 2024-09-11T22:43:33.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.068+0000 7fac197fa640 1 --2- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7fac04038f70 0x7fac0403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:33.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.068+0000 7fac227ae640 1 -- 172.21.15.152:0/2530677813 --> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fac1c108510 con 0x7fac04038f70 2024-09-11T22:43:33.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.068+0000 7fac197fa640 1 -- 172.21.15.152:0/2530677813 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fac0c06d060 con 0x7fac1c1080b0 2024-09-11T22:43:33.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.068+0000 7fac1b7fe640 1 -- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7fac04038f70 msgr2=0x7fac0403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/2200357575 2024-09-11T22:43:33.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.068+0000 7fac1b7fe640 1 --2- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7fac04038f70 0x7fac0403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:43:33.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.269+0000 7fac1b7fe640 1 -- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7fac04038f70 msgr2=0x7fac0403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/2200357575 2024-09-11T22:43:33.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.269+0000 7fac1b7fe640 1 --2- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7fac04038f70 0x7fac0403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:43:33.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.670+0000 7fac1b7fe640 1 -- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7fac04038f70 msgr2=0x7fac0403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/2200357575 2024-09-11T22:43:33.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:30.670+0000 7fac1b7fe640 1 --2- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7fac04038f70 0x7fac0403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T22:43:33.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:31.471+0000 7fac1b7fe640 1 -- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7fac04038f70 msgr2=0x7fac0403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/2200357575 2024-09-11T22:43:33.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:31.471+0000 7fac1b7fe640 1 --2- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7fac04038f70 0x7fac0403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T22:43:33.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:31.972+0000 7fac197fa640 1 -- 172.21.15.152:0/2530677813 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fac0c06bb60 con 0x7fac1c1080b0 2024-09-11T22:43:33.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:31.973+0000 7fac197fa640 1 -- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7fac04038f70 msgr2=0x7fac0403b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:43:33.563 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:31.973+0000 7fac197fa640 1 --2- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7fac04038f70 0x7fac0403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:33.563 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.973+0000 7fac197fa640 1 -- 172.21.15.152:0/2530677813 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fac0c03c7e0 con 0x7fac1c1080b0 2024-09-11T22:43:33.563 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.974+0000 7fac197fa640 1 --2- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fac0403c6b0 0x7fac0403eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:33.563 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.974+0000 7fac197fa640 1 -- 172.21.15.152:0/2530677813 --> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fac1c108510 con 0x7fac0403c6b0 2024-09-11T22:43:33.563 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.974+0000 7fac1b7fe640 1 --2- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fac0403c6b0 0x7fac0403eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:33.563 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.975+0000 7fac1b7fe640 1 --2- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fac0403c6b0 0x7fac0403eaa0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fac10003980 tx=0x7fac10006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:33.563 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.976+0000 7fac197fa640 1 -- 172.21.15.152:0/2530677813 <== mgr.14118 v2:172.21.15.152:6800/1868213162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fac1c108510 con 0x7fac0403c6b0 2024-09-11T22:43:33.563 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.982+0000 7fac227ae640 1 -- 172.21.15.152:0/2530677813 --> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fac1c102fe0 con 0x7fac0403c6b0 2024-09-11T22:43:33.563 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.983+0000 7fac197fa640 1 -- 172.21.15.152:0/2530677813 <== mgr.14118 v2:172.21.15.152:6800/1868213162 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fac1c102fe0 con 0x7fac0403c6b0 2024-09-11T22:43:33.563 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.983+0000 7fac227ae640 1 -- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fac0403c6b0 msgr2=0x7fac0403eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:33.563 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.983+0000 7fac227ae640 1 --2- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fac0403c6b0 0x7fac0403eaa0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fac10003980 tx=0x7fac10006e10 comp rx=0 tx=0).stop 2024-09-11T22:43:33.563 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.983+0000 7fac227ae640 1 -- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1080b0 msgr2=0x7fac1c19b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:33.563 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.983+0000 7fac227ae640 1 --2- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1080b0 0x7fac1c19b5d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fac0c032ce0 tx=0x7fac0c032d10 comp rx=0 tx=0).stop 2024-09-11T22:43:33.564 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.983+0000 7fac227ae640 1 -- 172.21.15.152:0/2530677813 shutdown_connections 2024-09-11T22:43:33.564 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.983+0000 7fac227ae640 1 --2- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fac0403c6b0 0x7fac0403eaa0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:33.564 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.983+0000 7fac227ae640 1 --2- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:6800/2200357575,v1:172.21.15.152:6801/2200357575] conn(0x7fac04038f70 0x7fac0403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:33.564 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.983+0000 7fac227ae640 1 --2- 172.21.15.152:0/2530677813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1080b0 0x7fac1c19b5d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:33.564 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.983+0000 7fac227ae640 1 -- 172.21.15.152:0/2530677813 >> 172.21.15.152:0/2530677813 conn(0x7fac1c101d10 msgr2=0x7fac1c1028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:33.564 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.983+0000 7fac227ae640 1 -- 172.21.15.152:0/2530677813 shutdown_connections 2024-09-11T22:43:33.564 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:32.983+0000 7fac227ae640 1 -- 172.21.15.152:0/2530677813 wait complete. 2024-09-11T22:43:33.564 INFO:teuthology.orchestra.run.smithi152.stdout:mgr epoch 5 is available 2024-09-11T22:43:33.564 INFO:teuthology.orchestra.run.smithi152.stdout:Setting orchestrator backend to cephadm... 2024-09-11T22:43:34.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:34 smithi152 ceph-mon[31181]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T22:43:34.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:34 smithi152 ceph-mon[31181]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T22:43:34.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:34 smithi152 ceph-mon[31181]: [11/Sep/2024:22:43:33] ENGINE Bus STARTING 2024-09-11T22:43:34.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:34 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:43:34.754 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.936+0000 7fc068eb3640 1 Processor -- start 2024-09-11T22:43:34.754 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.936+0000 7fc068eb3640 1 -- start start 2024-09-11T22:43:34.754 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.936+0000 7fc068eb3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc064107620 0x7fc064107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:34.754 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.936+0000 7fc068eb3640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc064107ff0 con 0x7fc064107620 2024-09-11T22:43:34.754 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.936+0000 7fc06259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc064107620 0x7fc064107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:34.754 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.936+0000 7fc06259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc064107620 0x7fc064107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56376/0 (socket says 172.21.15.152:56376) 2024-09-11T22:43:34.754 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.936+0000 7fc06259c640 1 -- 172.21.15.152:0/66616275 learned_addr learned my addr 172.21.15.152:0/66616275 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:34.754 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.937+0000 7fc06259c640 1 -- 172.21.15.152:0/66616275 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc064108820 con 0x7fc064107620 2024-09-11T22:43:34.755 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.937+0000 7fc06259c640 1 --2- 172.21.15.152:0/66616275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc064107620 0x7fc064107a20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc054009820 tx=0x7fc05402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa51c935359e5cde server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:34.755 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.937+0000 7fc06159a640 1 -- 172.21.15.152:0/66616275 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc054037070 con 0x7fc064107620 2024-09-11T22:43:34.755 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.938+0000 7fc06159a640 1 -- 172.21.15.152:0/66616275 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc054032900 con 0x7fc064107620 2024-09-11T22:43:34.755 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.938+0000 7fc06159a640 1 -- 172.21.15.152:0/66616275 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc054030850 con 0x7fc064107620 2024-09-11T22:43:34.755 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.938+0000 7fc068eb3640 1 -- 172.21.15.152:0/66616275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc064107620 msgr2=0x7fc064107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:34.755 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.938+0000 7fc068eb3640 1 --2- 172.21.15.152:0/66616275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc064107620 0x7fc064107a20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc054009820 tx=0x7fc05402a270 comp rx=0 tx=0).stop 2024-09-11T22:43:34.755 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.938+0000 7fc068eb3640 1 -- 172.21.15.152:0/66616275 shutdown_connections 2024-09-11T22:43:34.755 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.938+0000 7fc068eb3640 1 --2- 172.21.15.152:0/66616275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc064107620 0x7fc064107a20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:34.755 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.938+0000 7fc068eb3640 1 -- 172.21.15.152:0/66616275 >> 172.21.15.152:0/66616275 conn(0x7fc064102db0 msgr2=0x7fc0641051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:34.755 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.938+0000 7fc068eb3640 1 -- 172.21.15.152:0/66616275 shutdown_connections 2024-09-11T22:43:34.755 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.938+0000 7fc068eb3640 1 -- 172.21.15.152:0/66616275 wait complete. 2024-09-11T22:43:34.755 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.939+0000 7fc068eb3640 1 Processor -- start 2024-09-11T22:43:34.756 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.939+0000 7fc068eb3640 1 -- start start 2024-09-11T22:43:34.756 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.939+0000 7fc068eb3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc064107620 0x7fc06419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:34.756 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.939+0000 7fc068eb3640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc06410bbd0 con 0x7fc064107620 2024-09-11T22:43:34.756 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.939+0000 7fc06259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc064107620 0x7fc06419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:34.756 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.939+0000 7fc06259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc064107620 0x7fc06419b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56382/0 (socket says 172.21.15.152:56382) 2024-09-11T22:43:34.756 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.939+0000 7fc06259c640 1 -- 172.21.15.152:0/2532318339 learned_addr learned my addr 172.21.15.152:0/2532318339 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:34.756 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.939+0000 7fc06259c640 1 -- 172.21.15.152:0/2532318339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06419bc00 con 0x7fc064107620 2024-09-11T22:43:34.756 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.940+0000 7fc06259c640 1 --2- 172.21.15.152:0/2532318339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc064107620 0x7fc06419b6c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc054009940 tx=0x7fc0540326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:34.756 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.940+0000 7fc05b7fe640 1 -- 172.21.15.152:0/2532318339 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc054042070 con 0x7fc064107620 2024-09-11T22:43:34.756 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.940+0000 7fc05b7fe640 1 -- 172.21.15.152:0/2532318339 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc054032be0 con 0x7fc064107620 2024-09-11T22:43:34.756 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.940+0000 7fc05b7fe640 1 -- 172.21.15.152:0/2532318339 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc054037040 con 0x7fc064107620 2024-09-11T22:43:34.756 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.940+0000 7fc068eb3640 1 -- 172.21.15.152:0/2532318339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06419be90 con 0x7fc064107620 2024-09-11T22:43:34.756 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.940+0000 7fc068eb3640 1 -- 172.21.15.152:0/2532318339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06419eb80 con 0x7fc064107620 2024-09-11T22:43:34.757 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.941+0000 7fc05b7fe640 1 -- 172.21.15.152:0/2532318339 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fc054045070 con 0x7fc064107620 2024-09-11T22:43:34.757 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.941+0000 7fc05b7fe640 1 --2- 172.21.15.152:0/2532318339 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fc050038e60 0x7fc05003b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:34.757 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.941+0000 7fc05b7fe640 1 -- 172.21.15.152:0/2532318339 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc05406bd30 con 0x7fc064107620 2024-09-11T22:43:34.757 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.941+0000 7fc068eb3640 1 -- 172.21.15.152:0/2532318339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc06410bd50 con 0x7fc064107620 2024-09-11T22:43:34.757 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.941+0000 7fc061d9b640 1 --2- 172.21.15.152:0/2532318339 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fc050038e60 0x7fc05003b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:34.757 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.941+0000 7fc061d9b640 1 --2- 172.21.15.152:0/2532318339 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fc050038e60 0x7fc05003b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0098c0 tx=0x7fc04c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:34.757 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:33.946+0000 7fc05b7fe640 1 -- 172.21.15.152:0/2532318339 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc05404b170 con 0x7fc064107620 2024-09-11T22:43:34.757 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:34.028+0000 7fc05b7fe640 1 -- 172.21.15.152:0/2532318339 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc054068080 con 0x7fc064107620 2024-09-11T22:43:34.758 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:34.134+0000 7fc068eb3640 1 -- 172.21.15.152:0/2532318339 --> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fc064104090 con 0x7fc050038e60 2024-09-11T22:43:34.758 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:34.139+0000 7fc05b7fe640 1 -- 172.21.15.152:0/2532318339 <== mgr.14118 v2:172.21.15.152:6800/1868213162 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc064104090 con 0x7fc050038e60 2024-09-11T22:43:34.758 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:34.142+0000 7fc068eb3640 1 -- 172.21.15.152:0/2532318339 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fc050038e60 msgr2=0x7fc05003b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:34.758 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:34.142+0000 7fc068eb3640 1 --2- 172.21.15.152:0/2532318339 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fc050038e60 0x7fc05003b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0098c0 tx=0x7fc04c006db0 comp rx=0 tx=0).stop 2024-09-11T22:43:34.758 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:34.142+0000 7fc068eb3640 1 -- 172.21.15.152:0/2532318339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc064107620 msgr2=0x7fc06419b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:34.758 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:34.142+0000 7fc068eb3640 1 --2- 172.21.15.152:0/2532318339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc064107620 0x7fc06419b6c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc054009940 tx=0x7fc0540326e0 comp rx=0 tx=0).stop 2024-09-11T22:43:34.758 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:34.142+0000 7fc068eb3640 1 -- 172.21.15.152:0/2532318339 shutdown_connections 2024-09-11T22:43:34.758 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:34.142+0000 7fc068eb3640 1 --2- 172.21.15.152:0/2532318339 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fc050038e60 0x7fc05003b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:34.758 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:34.142+0000 7fc068eb3640 1 --2- 172.21.15.152:0/2532318339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc064107620 0x7fc06419b6c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:34.758 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:34.142+0000 7fc068eb3640 1 -- 172.21.15.152:0/2532318339 >> 172.21.15.152:0/2532318339 conn(0x7fc064102db0 msgr2=0x7fc064103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:34.758 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:34.142+0000 7fc068eb3640 1 -- 172.21.15.152:0/2532318339 shutdown_connections 2024-09-11T22:43:34.759 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:34.142+0000 7fc068eb3640 1 -- 172.21.15.152:0/2532318339 wait complete. 2024-09-11T22:43:35.349 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:35 smithi152 ceph-mon[31181]: [11/Sep/2024:22:43:33] ENGINE Serving on http://172.21.15.152:8765 2024-09-11T22:43:35.349 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:35 smithi152 ceph-mon[31181]: [11/Sep/2024:22:43:33] ENGINE Serving on https://172.21.15.152:7150 2024-09-11T22:43:35.349 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:35 smithi152 ceph-mon[31181]: [11/Sep/2024:22:43:33] ENGINE Bus STARTED 2024-09-11T22:43:35.349 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:35 smithi152 ceph-mon[31181]: [11/Sep/2024:22:43:33] ENGINE Error in HTTPServer.serve 2024-09-11T22:43:35.349 INFO:journalctl@ceph.mon.a.smithi152.stdout: Traceback (most recent call last): 2024-09-11T22:43:35.349 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:43:35.349 INFO:journalctl@ceph.mon.a.smithi152.stdout: self._connections.run(self.expiration_interval) 2024-09-11T22:43:35.349 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:43:35.349 INFO:journalctl@ceph.mon.a.smithi152.stdout: self._run(expiration_interval) 2024-09-11T22:43:35.349 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:43:35.350 INFO:journalctl@ceph.mon.a.smithi152.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:43:35.350 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:43:35.350 INFO:journalctl@ceph.mon.a.smithi152.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:43:35.350 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:43:35.350 INFO:journalctl@ceph.mon.a.smithi152.stdout: s = self.context.wrap_socket( 2024-09-11T22:43:35.350 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:43:35.350 INFO:journalctl@ceph.mon.a.smithi152.stdout: return self.sslsocket_class._create( 2024-09-11T22:43:35.350 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:43:35.350 INFO:journalctl@ceph.mon.a.smithi152.stdout: self.do_handshake() 2024-09-11T22:43:35.350 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:43:35.350 INFO:journalctl@ceph.mon.a.smithi152.stdout: self._sslobj.do_handshake() 2024-09-11T22:43:35.350 INFO:journalctl@ceph.mon.a.smithi152.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:43:35.351 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:35 smithi152 ceph-mon[31181]: mgrmap e8: x(active, since 2s) 2024-09-11T22:43:35.351 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:35 smithi152 ceph-mon[31181]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:43:35.351 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:35 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' 2024-09-11T22:43:35.351 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:35 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:43:35.926 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T22:43:35.927 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.169+0000 7fad40d42640 1 Processor -- start 2024-09-11T22:43:35.927 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.169+0000 7fad40d42640 1 -- start start 2024-09-11T22:43:35.927 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.169+0000 7fad40d42640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106500 0x7fad3c106900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:35.927 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.169+0000 7fad40d42640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c106ed0 con 0x7fad3c106500 2024-09-11T22:43:35.927 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.170+0000 7fad3a59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106500 0x7fad3c106900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:35.927 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.170+0000 7fad3a59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106500 0x7fad3c106900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56386/0 (socket says 172.21.15.152:56386) 2024-09-11T22:43:35.927 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.170+0000 7fad3a59c640 1 -- 172.21.15.152:0/1301081852 learned_addr learned my addr 172.21.15.152:0/1301081852 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:35.927 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.170+0000 7fad3a59c640 1 -- 172.21.15.152:0/1301081852 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c107700 con 0x7fad3c106500 2024-09-11T22:43:35.927 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.171+0000 7fad3a59c640 1 --2- 172.21.15.152:0/1301081852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106500 0x7fad3c106900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009820 tx=0x7fad2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01b66b28d7c5ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:35.927 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.171+0000 7fad3959a640 1 -- 172.21.15.152:0/1301081852 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad2c037070 con 0x7fad3c106500 2024-09-11T22:43:35.927 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.171+0000 7fad3959a640 1 -- 172.21.15.152:0/1301081852 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fad2c032900 con 0x7fad3c106500 2024-09-11T22:43:35.928 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.171+0000 7fad3959a640 1 -- 172.21.15.152:0/1301081852 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad2c030850 con 0x7fad3c106500 2024-09-11T22:43:35.928 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.171+0000 7fad40d42640 1 -- 172.21.15.152:0/1301081852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106500 msgr2=0x7fad3c106900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:35.928 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.171+0000 7fad40d42640 1 --2- 172.21.15.152:0/1301081852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106500 0x7fad3c106900 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009820 tx=0x7fad2c02a270 comp rx=0 tx=0).stop 2024-09-11T22:43:35.928 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.171+0000 7fad40d42640 1 -- 172.21.15.152:0/1301081852 shutdown_connections 2024-09-11T22:43:35.928 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.171+0000 7fad40d42640 1 --2- 172.21.15.152:0/1301081852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106500 0x7fad3c106900 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:35.928 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.171+0000 7fad40d42640 1 -- 172.21.15.152:0/1301081852 >> 172.21.15.152:0/1301081852 conn(0x7fad3c101d10 msgr2=0x7fad3c104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:35.928 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.172+0000 7fad40d42640 1 -- 172.21.15.152:0/1301081852 shutdown_connections 2024-09-11T22:43:35.928 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.172+0000 7fad40d42640 1 -- 172.21.15.152:0/1301081852 wait complete. 2024-09-11T22:43:35.928 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.172+0000 7fad40d42640 1 Processor -- start 2024-09-11T22:43:35.928 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.172+0000 7fad40d42640 1 -- start start 2024-09-11T22:43:35.928 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.172+0000 7fad40d42640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106500 0x7fad3c191730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:35.928 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.173+0000 7fad40d42640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c10aab0 con 0x7fad3c106500 2024-09-11T22:43:35.928 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.173+0000 7fad3a59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106500 0x7fad3c191730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:35.929 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.173+0000 7fad3a59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106500 0x7fad3c191730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56396/0 (socket says 172.21.15.152:56396) 2024-09-11T22:43:35.929 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.173+0000 7fad3a59c640 1 -- 172.21.15.152:0/2490439695 learned_addr learned my addr 172.21.15.152:0/2490439695 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:35.929 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.173+0000 7fad3a59c640 1 -- 172.21.15.152:0/2490439695 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c191c70 con 0x7fad3c106500 2024-09-11T22:43:35.929 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.173+0000 7fad3a59c640 1 --2- 172.21.15.152:0/2490439695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106500 0x7fad3c191730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009940 tx=0x7fad2c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:35.929 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.173+0000 7fad2b7fe640 1 -- 172.21.15.152:0/2490439695 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad2c042070 con 0x7fad3c106500 2024-09-11T22:43:35.929 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.173+0000 7fad2b7fe640 1 -- 172.21.15.152:0/2490439695 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fad2c0308c0 con 0x7fad3c106500 2024-09-11T22:43:35.929 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.173+0000 7fad40d42640 1 -- 172.21.15.152:0/2490439695 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad3c18fe60 con 0x7fad3c106500 2024-09-11T22:43:35.929 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.173+0000 7fad2b7fe640 1 -- 172.21.15.152:0/2490439695 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad2c037040 con 0x7fad3c106500 2024-09-11T22:43:35.929 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.173+0000 7fad40d42640 1 -- 172.21.15.152:0/2490439695 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad3c1903a0 con 0x7fad3c106500 2024-09-11T22:43:35.929 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.174+0000 7fad2b7fe640 1 -- 172.21.15.152:0/2490439695 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fad2c045070 con 0x7fad3c106500 2024-09-11T22:43:35.929 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.174+0000 7fad40d42640 1 -- 172.21.15.152:0/2490439695 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad3c10ac30 con 0x7fad3c106500 2024-09-11T22:43:35.929 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.174+0000 7fad2b7fe640 1 --2- 172.21.15.152:0/2490439695 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fad24038f70 0x7fad2403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:35.929 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.174+0000 7fad2b7fe640 1 -- 172.21.15.152:0/2490439695 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fad2c06c140 con 0x7fad3c106500 2024-09-11T22:43:35.930 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.175+0000 7fad39d9b640 1 --2- 172.21.15.152:0/2490439695 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fad24038f70 0x7fad2403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:35.930 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.175+0000 7fad39d9b640 1 --2- 172.21.15.152:0/2490439695 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fad24038f70 0x7fad2403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fad300098c0 tx=0x7fad30006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:35.930 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.179+0000 7fad2b7fe640 1 -- 172.21.15.152:0/2490439695 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad2c06ba90 con 0x7fad3c106500 2024-09-11T22:43:35.930 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.347+0000 7fad40d42640 1 -- 172.21.15.152:0/2490439695 --> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fad3c190c30 con 0x7fad24038f70 2024-09-11T22:43:35.930 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.348+0000 7fad2b7fe640 1 -- 172.21.15.152:0/2490439695 <== mgr.14118 v2:172.21.15.152:6800/1868213162 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fad3c190c30 con 0x7fad24038f70 2024-09-11T22:43:35.930 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.351+0000 7fad40d42640 1 -- 172.21.15.152:0/2490439695 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fad24038f70 msgr2=0x7fad2403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:35.930 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.351+0000 7fad40d42640 1 --2- 172.21.15.152:0/2490439695 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fad24038f70 0x7fad2403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fad300098c0 tx=0x7fad30006db0 comp rx=0 tx=0).stop 2024-09-11T22:43:35.930 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.351+0000 7fad40d42640 1 -- 172.21.15.152:0/2490439695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106500 msgr2=0x7fad3c191730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:35.930 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.351+0000 7fad40d42640 1 --2- 172.21.15.152:0/2490439695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106500 0x7fad3c191730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009940 tx=0x7fad2c032ce0 comp rx=0 tx=0).stop 2024-09-11T22:43:35.930 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.351+0000 7fad40d42640 1 -- 172.21.15.152:0/2490439695 shutdown_connections 2024-09-11T22:43:35.930 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.351+0000 7fad40d42640 1 --2- 172.21.15.152:0/2490439695 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fad24038f70 0x7fad2403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:35.930 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.351+0000 7fad40d42640 1 --2- 172.21.15.152:0/2490439695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad3c106500 0x7fad3c191730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:35.931 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.351+0000 7fad40d42640 1 -- 172.21.15.152:0/2490439695 >> 172.21.15.152:0/2490439695 conn(0x7fad3c101d10 msgr2=0x7fad3c102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:35.931 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.351+0000 7fad40d42640 1 -- 172.21.15.152:0/2490439695 shutdown_connections 2024-09-11T22:43:35.931 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:35.351+0000 7fad40d42640 1 -- 172.21.15.152:0/2490439695 wait complete. 2024-09-11T22:43:35.931 INFO:teuthology.orchestra.run.smithi152.stdout:Generating ssh key... 2024-09-11T22:43:36.331 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-mon[31181]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:43:36.993 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: Generating public/private rsa key pair. 2024-09-11T22:43:36.993 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: Your identification has been saved in /tmp/tmpg9nas1oi/key 2024-09-11T22:43:36.993 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: Your public key has been saved in /tmp/tmpg9nas1oi/key.pub 2024-09-11T22:43:36.993 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: The key fingerprint is: 2024-09-11T22:43:36.993 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: SHA256:0TN0MR53WgSodZ6pD8663GdT4RtZqRJluJ3nTT/OtbY ceph-0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:43:36.993 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: The key's randomart image is: 2024-09-11T22:43:36.993 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: +---[RSA 3072]----+ 2024-09-11T22:43:36.993 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: | . *ooo+| 2024-09-11T22:43:36.993 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: | o +o*.+ | 2024-09-11T22:43:36.993 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: | . +o*ooo.| 2024-09-11T22:43:36.994 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: | ..= o++o| 2024-09-11T22:43:36.994 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: | S ..=.*| 2024-09-11T22:43:36.994 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: | .o. *=| 2024-09-11T22:43:36.994 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: | o.oo.*| 2024-09-11T22:43:36.994 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: | . .o =* | 2024-09-11T22:43:36.994 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: | +o.o.E.| 2024-09-11T22:43:36.994 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: +----[SHA256]-----+ 2024-09-11T22:43:37.334 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.341+0000 7f3c9c24b640 1 Processor -- start 2024-09-11T22:43:37.335 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.341+0000 7f3c9c24b640 1 -- start start 2024-09-11T22:43:37.335 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.341+0000 7f3c9c24b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c9407de50 0x7f3c9407c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:37.335 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.341+0000 7f3c9c24b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9407c880 con 0x7f3c9407de50 2024-09-11T22:43:37.335 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.341+0000 7f3c99fe7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c9407de50 0x7f3c9407c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:37.335 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.341+0000 7f3c99fe7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c9407de50 0x7f3c9407c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56408/0 (socket says 172.21.15.152:56408) 2024-09-11T22:43:37.335 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.341+0000 7f3c99fe7640 1 -- 172.21.15.152:0/3447329262 learned_addr learned my addr 172.21.15.152:0/3447329262 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:37.335 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.341+0000 7f3c99fe7640 1 -- 172.21.15.152:0/3447329262 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9407ca00 con 0x7f3c9407de50 2024-09-11T22:43:37.335 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.342+0000 7f3c99fe7640 1 --2- 172.21.15.152:0/3447329262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c9407de50 0x7f3c9407c2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009820 tx=0x7f3c9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9d75dcb8d7499c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:37.335 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.342+0000 7f3c98fe5640 1 -- 172.21.15.152:0/3447329262 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c90037070 con 0x7f3c9407de50 2024-09-11T22:43:37.335 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.342+0000 7f3c98fe5640 1 -- 172.21.15.152:0/3447329262 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3c90032900 con 0x7f3c9407de50 2024-09-11T22:43:37.335 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.342+0000 7f3c98fe5640 1 -- 172.21.15.152:0/3447329262 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c90030850 con 0x7f3c9407de50 2024-09-11T22:43:37.335 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.343+0000 7f3c9c24b640 1 -- 172.21.15.152:0/3447329262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c9407de50 msgr2=0x7f3c9407c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:37.335 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.343+0000 7f3c9c24b640 1 --2- 172.21.15.152:0/3447329262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c9407de50 0x7f3c9407c2b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009820 tx=0x7f3c9002a270 comp rx=0 tx=0).stop 2024-09-11T22:43:37.336 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.343+0000 7f3c9c24b640 1 -- 172.21.15.152:0/3447329262 shutdown_connections 2024-09-11T22:43:37.336 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.343+0000 7f3c9c24b640 1 --2- 172.21.15.152:0/3447329262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c9407de50 0x7f3c9407c2b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:37.336 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.343+0000 7f3c9c24b640 1 -- 172.21.15.152:0/3447329262 >> 172.21.15.152:0/3447329262 conn(0x7f3c94102db0 msgr2=0x7f3c941051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:37.336 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.343+0000 7f3c9c24b640 1 -- 172.21.15.152:0/3447329262 shutdown_connections 2024-09-11T22:43:37.336 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.343+0000 7f3c9c24b640 1 -- 172.21.15.152:0/3447329262 wait complete. 2024-09-11T22:43:37.336 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.344+0000 7f3c9c24b640 1 Processor -- start 2024-09-11T22:43:37.336 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.344+0000 7f3c9c24b640 1 -- start start 2024-09-11T22:43:37.336 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.344+0000 7f3c9c24b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c9407de50 0x7f3c9419d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:37.336 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.344+0000 7f3c9c24b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9410dc50 con 0x7f3c9407de50 2024-09-11T22:43:37.336 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.344+0000 7f3c99fe7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c9407de50 0x7f3c9419d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:37.336 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.344+0000 7f3c99fe7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c9407de50 0x7f3c9419d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56414/0 (socket says 172.21.15.152:56414) 2024-09-11T22:43:37.336 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.344+0000 7f3c99fe7640 1 -- 172.21.15.152:0/1948048542 learned_addr learned my addr 172.21.15.152:0/1948048542 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:37.336 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.344+0000 7f3c99fe7640 1 -- 172.21.15.152:0/1948048542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9419dda0 con 0x7f3c9407de50 2024-09-11T22:43:37.337 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.344+0000 7f3c99fe7640 1 --2- 172.21.15.152:0/1948048542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c9407de50 0x7f3c9419d860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3c90031000 tx=0x7f3c900326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:37.337 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.345+0000 7f3c8affd640 1 -- 172.21.15.152:0/1948048542 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c90042070 con 0x7f3c9407de50 2024-09-11T22:43:37.337 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.345+0000 7f3c8affd640 1 -- 172.21.15.152:0/1948048542 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3c90032be0 con 0x7f3c9407de50 2024-09-11T22:43:37.337 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.345+0000 7f3c9c24b640 1 -- 172.21.15.152:0/1948048542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9419e030 con 0x7f3c9407de50 2024-09-11T22:43:37.337 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.345+0000 7f3c8affd640 1 -- 172.21.15.152:0/1948048542 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c90037040 con 0x7f3c9407de50 2024-09-11T22:43:37.337 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.345+0000 7f3c9c24b640 1 -- 172.21.15.152:0/1948048542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c941a0d20 con 0x7f3c9407de50 2024-09-11T22:43:37.337 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.345+0000 7f3c8affd640 1 -- 172.21.15.152:0/1948048542 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3c90045070 con 0x7f3c9407de50 2024-09-11T22:43:37.337 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.346+0000 7f3c9c24b640 1 -- 172.21.15.152:0/1948048542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c9407c330 con 0x7f3c9407de50 2024-09-11T22:43:37.337 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.346+0000 7f3c8affd640 1 --2- 172.21.15.152:0/1948048542 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f3c80038fc0 0x7f3c8003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:37.337 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.346+0000 7f3c8affd640 1 -- 172.21.15.152:0/1948048542 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3c9006bfc0 con 0x7f3c9407de50 2024-09-11T22:43:37.337 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.346+0000 7f3c997e6640 1 --2- 172.21.15.152:0/1948048542 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f3c80038fc0 0x7f3c8003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:37.337 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.346+0000 7f3c997e6640 1 --2- 172.21.15.152:0/1948048542 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f3c80038fc0 0x7f3c8003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3c840098c0 tx=0x7f3c84006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:37.337 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.350+0000 7f3c8affd640 1 -- 172.21.15.152:0/1948048542 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c90042210 con 0x7f3c9407de50 2024-09-11T22:43:37.338 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.517+0000 7f3c9c24b640 1 -- 172.21.15.152:0/1948048542 --> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f3c941040c0 con 0x7f3c80038fc0 2024-09-11T22:43:37.338 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.682+0000 7f3c8affd640 1 -- 172.21.15.152:0/1948048542 <== mgr.14118 v2:172.21.15.152:6800/1868213162 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3c941040c0 con 0x7f3c80038fc0 2024-09-11T22:43:37.338 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.684+0000 7f3c9c24b640 1 -- 172.21.15.152:0/1948048542 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f3c80038fc0 msgr2=0x7f3c8003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:37.338 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.684+0000 7f3c9c24b640 1 --2- 172.21.15.152:0/1948048542 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f3c80038fc0 0x7f3c8003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3c840098c0 tx=0x7f3c84006db0 comp rx=0 tx=0).stop 2024-09-11T22:43:37.338 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.684+0000 7f3c9c24b640 1 -- 172.21.15.152:0/1948048542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c9407de50 msgr2=0x7f3c9419d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:37.338 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.684+0000 7f3c9c24b640 1 --2- 172.21.15.152:0/1948048542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c9407de50 0x7f3c9419d860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3c90031000 tx=0x7f3c900326e0 comp rx=0 tx=0).stop 2024-09-11T22:43:37.338 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.684+0000 7f3c9c24b640 1 -- 172.21.15.152:0/1948048542 shutdown_connections 2024-09-11T22:43:37.338 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.684+0000 7f3c9c24b640 1 --2- 172.21.15.152:0/1948048542 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f3c80038fc0 0x7f3c8003b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:37.338 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.684+0000 7f3c9c24b640 1 --2- 172.21.15.152:0/1948048542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c9407de50 0x7f3c9419d860 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:37.338 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.684+0000 7f3c9c24b640 1 -- 172.21.15.152:0/1948048542 >> 172.21.15.152:0/1948048542 conn(0x7f3c94102db0 msgr2=0x7f3c941039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:37.338 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.684+0000 7f3c9c24b640 1 -- 172.21.15.152:0/1948048542 shutdown_connections 2024-09-11T22:43:37.338 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:36.684+0000 7f3c9c24b640 1 -- 172.21.15.152:0/1948048542 wait complete. 2024-09-11T22:43:37.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:37 smithi152 ceph-mon[31181]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:43:37.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:37 smithi152 ceph-mon[31181]: Generating ssh key... 2024-09-11T22:43:37.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:37 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' 2024-09-11T22:43:37.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:37 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' 2024-09-11T22:43:38.558 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFgLqTslnLmsR6QRRT/mWeoZoK2ggG3HNAriMiMo3k2vv/qL8kQ8gf2LdPGmF7ijtuBVFtgPjwtUHSIR/nnc9/bSbueLmlkfADVyVo5hG6UPRIK4uObXLBBSK1yCdrA0kDqQn1/AQCMcJvc49jhDZhVNKp6Cy11SSovCMnnKnggCmB9PTWL6R+rALluxMyUTWtQ7+GbiH7nQmEGhAmLGj6gnSjHtW2qt/uiSzvGTRfiop2mmPQgV0JJXcNT57uh8ovMJQrzkNkK1Ls05wUy7QzqO4p/ZCR5yn6jclJlr3xD2NXWWt6R+VOAjDrvLDqGusTSeDvzHpm5PIW1NqjgIn0pVE4fzkrbP+EdU50zlzmsEtWeqBIwRvCx4xizkfGiGxfIEjiwiSHMK46CEIY7Sq6y0pEtP0Z1ECjJ275jRaldk+QLH/7Go9ykyO00qNZlHbGXRLtmItiJy4VjqbceOu5TTv0wsFzFJo7taQoD4SGWTA2yz76vTFCCKRPt722YD8= ceph-0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:43:38.558 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.753+0000 7f2fb559e640 1 Processor -- start 2024-09-11T22:43:38.558 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.753+0000 7f2fb559e640 1 -- start start 2024-09-11T22:43:38.558 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.754+0000 7f2fb559e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb007de20 0x7f2fb007c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:38.558 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.754+0000 7f2fb559e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb007c810 con 0x7f2fb007de20 2024-09-11T22:43:38.558 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.754+0000 7f2faffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb007de20 0x7f2fb007c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:38.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.754+0000 7f2faffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb007de20 0x7f2fb007c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56422/0 (socket says 172.21.15.152:56422) 2024-09-11T22:43:38.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.754+0000 7f2faffff640 1 -- 172.21.15.152:0/2938872703 learned_addr learned my addr 172.21.15.152:0/2938872703 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:38.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.754+0000 7f2faffff640 1 -- 172.21.15.152:0/2938872703 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb007c990 con 0x7f2fb007de20 2024-09-11T22:43:38.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.755+0000 7f2faffff640 1 --2- 172.21.15.152:0/2938872703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb007de20 0x7f2fb007c2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009820 tx=0x7f2fa002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cb3ddd956af5e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:38.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.755+0000 7f2faeffd640 1 -- 172.21.15.152:0/2938872703 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fa0037070 con 0x7f2fb007de20 2024-09-11T22:43:38.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.755+0000 7f2faeffd640 1 -- 172.21.15.152:0/2938872703 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2fa0032900 con 0x7f2fb007de20 2024-09-11T22:43:38.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.755+0000 7f2faeffd640 1 -- 172.21.15.152:0/2938872703 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fa00308e0 con 0x7f2fb007de20 2024-09-11T22:43:38.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.755+0000 7f2fb559e640 1 -- 172.21.15.152:0/2938872703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb007de20 msgr2=0x7f2fb007c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:38.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.755+0000 7f2fb559e640 1 --2- 172.21.15.152:0/2938872703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb007de20 0x7f2fb007c2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0009820 tx=0x7f2fa002a300 comp rx=0 tx=0).stop 2024-09-11T22:43:38.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.755+0000 7f2fb559e640 1 -- 172.21.15.152:0/2938872703 shutdown_connections 2024-09-11T22:43:38.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.755+0000 7f2fb559e640 1 --2- 172.21.15.152:0/2938872703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb007de20 0x7f2fb007c2d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:38.559 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.755+0000 7f2fb559e640 1 -- 172.21.15.152:0/2938872703 >> 172.21.15.152:0/2938872703 conn(0x7f2fb0102c00 msgr2=0x7f2fb0105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:38.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.755+0000 7f2fb559e640 1 -- 172.21.15.152:0/2938872703 shutdown_connections 2024-09-11T22:43:38.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.755+0000 7f2fb559e640 1 -- 172.21.15.152:0/2938872703 wait complete. 2024-09-11T22:43:38.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.756+0000 7f2fb559e640 1 Processor -- start 2024-09-11T22:43:38.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.756+0000 7f2fb559e640 1 -- start start 2024-09-11T22:43:38.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.757+0000 7f2fb559e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb007de20 0x7f2fb019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:38.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.757+0000 7f2fb559e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb010da80 con 0x7f2fb007de20 2024-09-11T22:43:38.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.757+0000 7f2faffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb007de20 0x7f2fb019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:38.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.757+0000 7f2faffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb007de20 0x7f2fb019d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56430/0 (socket says 172.21.15.152:56430) 2024-09-11T22:43:38.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.757+0000 7f2faffff640 1 -- 172.21.15.152:0/754337871 learned_addr learned my addr 172.21.15.152:0/754337871 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:38.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.757+0000 7f2faffff640 1 -- 172.21.15.152:0/754337871 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb019db40 con 0x7f2fb007de20 2024-09-11T22:43:38.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.757+0000 7f2faffff640 1 --2- 172.21.15.152:0/754337871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb007de20 0x7f2fb019d600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0030610 tx=0x7f2fa0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:38.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.757+0000 7f2fad7fa640 1 -- 172.21.15.152:0/754337871 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fa0037040 con 0x7f2fb007de20 2024-09-11T22:43:38.560 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.757+0000 7f2fad7fa640 1 -- 172.21.15.152:0/754337871 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2fa0040070 con 0x7f2fb007de20 2024-09-11T22:43:38.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.757+0000 7f2fad7fa640 1 -- 172.21.15.152:0/754337871 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fa003a390 con 0x7f2fb007de20 2024-09-11T22:43:38.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.757+0000 7f2fb559e640 1 -- 172.21.15.152:0/754337871 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb019ddd0 con 0x7f2fb007de20 2024-09-11T22:43:38.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.757+0000 7f2fb559e640 1 -- 172.21.15.152:0/754337871 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb01a0ac0 con 0x7f2fb007de20 2024-09-11T22:43:38.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.758+0000 7f2fad7fa640 1 -- 172.21.15.152:0/754337871 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2fa003a690 con 0x7f2fb007de20 2024-09-11T22:43:38.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.758+0000 7f2fb559e640 1 -- 172.21.15.152:0/754337871 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb007c350 con 0x7f2fb007de20 2024-09-11T22:43:38.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.758+0000 7f2fad7fa640 1 --2- 172.21.15.152:0/754337871 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f2f98038f70 0x7f2f9803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:38.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.759+0000 7f2fad7fa640 1 -- 172.21.15.152:0/754337871 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2fa006c290 con 0x7f2fb007de20 2024-09-11T22:43:38.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.759+0000 7f2faf7fe640 1 --2- 172.21.15.152:0/754337871 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f2f98038f70 0x7f2f9803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:38.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.759+0000 7f2faf7fe640 1 --2- 172.21.15.152:0/754337871 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f2f98038f70 0x7f2f9803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40098c0 tx=0x7f2fa4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:38.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.763+0000 7f2fad7fa640 1 -- 172.21.15.152:0/754337871 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2fa0041270 con 0x7f2fb007de20 2024-09-11T22:43:38.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.932+0000 7f2fb559e640 1 -- 172.21.15.152:0/754337871 --> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f2fb0103ed0 con 0x7f2f98038f70 2024-09-11T22:43:38.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.933+0000 7f2fad7fa640 1 -- 172.21.15.152:0/754337871 <== mgr.14118 v2:172.21.15.152:6800/1868213162 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f2fb0103ed0 con 0x7f2f98038f70 2024-09-11T22:43:38.561 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.935+0000 7f2fb559e640 1 -- 172.21.15.152:0/754337871 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f2f98038f70 msgr2=0x7f2f9803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:38.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.935+0000 7f2fb559e640 1 --2- 172.21.15.152:0/754337871 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f2f98038f70 0x7f2f9803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40098c0 tx=0x7f2fa4006db0 comp rx=0 tx=0).stop 2024-09-11T22:43:38.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.935+0000 7f2fb559e640 1 -- 172.21.15.152:0/754337871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb007de20 msgr2=0x7f2fb019d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:38.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.935+0000 7f2fb559e640 1 --2- 172.21.15.152:0/754337871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb007de20 0x7f2fb019d600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0030610 tx=0x7f2fa0032670 comp rx=0 tx=0).stop 2024-09-11T22:43:38.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.935+0000 7f2fb559e640 1 -- 172.21.15.152:0/754337871 shutdown_connections 2024-09-11T22:43:38.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.935+0000 7f2fb559e640 1 --2- 172.21.15.152:0/754337871 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f2f98038f70 0x7f2f9803b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:38.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.935+0000 7f2fb559e640 1 --2- 172.21.15.152:0/754337871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2fb007de20 0x7f2fb019d600 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:38.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.935+0000 7f2fb559e640 1 -- 172.21.15.152:0/754337871 >> 172.21.15.152:0/754337871 conn(0x7f2fb0102c00 msgr2=0x7f2fb01037c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:38.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.936+0000 7f2fb559e640 1 -- 172.21.15.152:0/754337871 shutdown_connections 2024-09-11T22:43:38.562 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:37.936+0000 7f2fb559e640 1 -- 172.21.15.152:0/754337871 wait complete. 2024-09-11T22:43:38.562 INFO:teuthology.orchestra.run.smithi152.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T22:43:38.562 INFO:teuthology.orchestra.run.smithi152.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T22:43:38.563 INFO:teuthology.orchestra.run.smithi152.stdout:Adding host smithi152... 2024-09-11T22:43:38.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:38 smithi152 ceph-mon[31181]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:43:39.956 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:39 smithi152 ceph-mon[31181]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi152", "addr": "172.21.15.152", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:43:40.916 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:40 smithi152 ceph-mon[31181]: Deploying cephadm binary to smithi152 2024-09-11T22:43:42.949 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout Added host 'smithi152' with addr '172.21.15.152' 2024-09-11T22:43:42.950 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.970+0000 7f732c8a5640 1 Processor -- start 2024-09-11T22:43:42.950 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.970+0000 7f732c8a5640 1 -- start start 2024-09-11T22:43:42.950 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.970+0000 7f732c8a5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7328106510 0x7f7328106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:42.950 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.970+0000 7f732c8a5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7328106ee0 con 0x7f7328106510 2024-09-11T22:43:42.950 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.971+0000 7f732659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7328106510 0x7f7328106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:42.950 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.971+0000 7f732659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7328106510 0x7f7328106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40556/0 (socket says 172.21.15.152:40556) 2024-09-11T22:43:42.950 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.971+0000 7f732659c640 1 -- 172.21.15.152:0/1098204675 learned_addr learned my addr 172.21.15.152:0/1098204675 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:42.950 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.971+0000 7f732659c640 1 -- 172.21.15.152:0/1098204675 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73281076c0 con 0x7f7328106510 2024-09-11T22:43:42.950 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.972+0000 7f732659c640 1 --2- 172.21.15.152:0/1098204675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7328106510 0x7f7328106910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7318009820 tx=0x7f731802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e5c4bbca19d70a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:42.950 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.972+0000 7f732559a640 1 -- 172.21.15.152:0/1098204675 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7318037070 con 0x7f7328106510 2024-09-11T22:43:42.950 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.972+0000 7f732559a640 1 -- 172.21.15.152:0/1098204675 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7318032900 con 0x7f7328106510 2024-09-11T22:43:42.950 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.972+0000 7f732559a640 1 -- 172.21.15.152:0/1098204675 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7318030850 con 0x7f7328106510 2024-09-11T22:43:42.951 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.972+0000 7f732c8a5640 1 -- 172.21.15.152:0/1098204675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7328106510 msgr2=0x7f7328106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:42.951 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.972+0000 7f732c8a5640 1 --2- 172.21.15.152:0/1098204675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7328106510 0x7f7328106910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7318009820 tx=0x7f731802a270 comp rx=0 tx=0).stop 2024-09-11T22:43:42.951 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.973+0000 7f732c8a5640 1 -- 172.21.15.152:0/1098204675 shutdown_connections 2024-09-11T22:43:42.951 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.973+0000 7f732c8a5640 1 --2- 172.21.15.152:0/1098204675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7328106510 0x7f7328106910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:42.951 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.973+0000 7f732c8a5640 1 -- 172.21.15.152:0/1098204675 >> 172.21.15.152:0/1098204675 conn(0x7f7328101ca0 msgr2=0x7f73281040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:42.951 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.973+0000 7f732c8a5640 1 -- 172.21.15.152:0/1098204675 shutdown_connections 2024-09-11T22:43:42.952 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.973+0000 7f732c8a5640 1 -- 172.21.15.152:0/1098204675 wait complete. 2024-09-11T22:43:42.952 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.973+0000 7f732c8a5640 1 Processor -- start 2024-09-11T22:43:42.952 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.973+0000 7f732c8a5640 1 -- start start 2024-09-11T22:43:42.952 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.973+0000 7f732c8a5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7328106510 0x7f732807ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:42.952 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.973+0000 7f732c8a5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f732810aa70 con 0x7f7328106510 2024-09-11T22:43:42.952 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.974+0000 7f732659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7328106510 0x7f732807ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:42.952 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.974+0000 7f732659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7328106510 0x7f732807ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40570/0 (socket says 172.21.15.152:40570) 2024-09-11T22:43:42.952 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.974+0000 7f732659c640 1 -- 172.21.15.152:0/1623142922 learned_addr learned my addr 172.21.15.152:0/1623142922 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:42.952 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.974+0000 7f732659c640 1 -- 172.21.15.152:0/1623142922 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732819e3b0 con 0x7f7328106510 2024-09-11T22:43:42.952 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.974+0000 7f732659c640 1 --2- 172.21.15.152:0/1623142922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7328106510 0x7f732807ba20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7318009940 tx=0x7f73180326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:42.952 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.974+0000 7f73177fe640 1 -- 172.21.15.152:0/1623142922 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7318042070 con 0x7f7328106510 2024-09-11T22:43:42.952 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.974+0000 7f73177fe640 1 -- 172.21.15.152:0/1623142922 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7318032be0 con 0x7f7328106510 2024-09-11T22:43:42.953 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.974+0000 7f732c8a5640 1 -- 172.21.15.152:0/1623142922 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732819e580 con 0x7f7328106510 2024-09-11T22:43:42.953 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.974+0000 7f73177fe640 1 -- 172.21.15.152:0/1623142922 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7318037040 con 0x7f7328106510 2024-09-11T22:43:42.953 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.974+0000 7f732c8a5640 1 -- 172.21.15.152:0/1623142922 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f732819e920 con 0x7f7328106510 2024-09-11T22:43:42.953 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.975+0000 7f73177fe640 1 -- 172.21.15.152:0/1623142922 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7318045070 con 0x7f7328106510 2024-09-11T22:43:42.953 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.975+0000 7f732c8a5640 1 -- 172.21.15.152:0/1623142922 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7328106990 con 0x7f7328106510 2024-09-11T22:43:42.953 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.975+0000 7f73177fe640 1 --2- 172.21.15.152:0/1623142922 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f7310038f70 0x7f731003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:42.953 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.975+0000 7f73177fe640 1 -- 172.21.15.152:0/1623142922 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f731806c150 con 0x7f7328106510 2024-09-11T22:43:42.953 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.975+0000 7f7325d9b640 1 --2- 172.21.15.152:0/1623142922 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f7310038f70 0x7f731003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:42.953 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.976+0000 7f7325d9b640 1 --2- 172.21.15.152:0/1623142922 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f7310038f70 0x7f731003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f731c0098c0 tx=0x7f731c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:42.953 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:38.980+0000 7f73177fe640 1 -- 172.21.15.152:0/1623142922 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7318045e10 con 0x7f7328106510 2024-09-11T22:43:42.953 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:39.150+0000 7f732c8a5640 1 -- 172.21.15.152:0/1623142922 --> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi152", "addr": "172.21.15.152", "target": ["mon-mgr", ""]}) -- 0x7f7328102f40 con 0x7f7310038f70 2024-09-11T22:43:42.953 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:42.360+0000 7f73177fe640 1 -- 172.21.15.152:0/1623142922 <== mgr.14118 v2:172.21.15.152:6800/1868213162 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f7328102f40 con 0x7f7310038f70 2024-09-11T22:43:42.953 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:42.362+0000 7f732c8a5640 1 -- 172.21.15.152:0/1623142922 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f7310038f70 msgr2=0x7f731003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:42.954 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:42.363+0000 7f732c8a5640 1 --2- 172.21.15.152:0/1623142922 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f7310038f70 0x7f731003b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f731c0098c0 tx=0x7f731c006db0 comp rx=0 tx=0).stop 2024-09-11T22:43:42.954 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:42.363+0000 7f732c8a5640 1 -- 172.21.15.152:0/1623142922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7328106510 msgr2=0x7f732807ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:42.954 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:42.363+0000 7f732c8a5640 1 --2- 172.21.15.152:0/1623142922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7328106510 0x7f732807ba20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7318009940 tx=0x7f73180326e0 comp rx=0 tx=0).stop 2024-09-11T22:43:42.954 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:42.363+0000 7f732c8a5640 1 -- 172.21.15.152:0/1623142922 shutdown_connections 2024-09-11T22:43:42.954 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:42.363+0000 7f732c8a5640 1 --2- 172.21.15.152:0/1623142922 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f7310038f70 0x7f731003b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:42.954 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:42.363+0000 7f732c8a5640 1 --2- 172.21.15.152:0/1623142922 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7328106510 0x7f732807ba20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:42.954 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:42.363+0000 7f732c8a5640 1 -- 172.21.15.152:0/1623142922 >> 172.21.15.152:0/1623142922 conn(0x7f7328101ca0 msgr2=0x7f7328102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:42.954 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:42.363+0000 7f732c8a5640 1 -- 172.21.15.152:0/1623142922 shutdown_connections 2024-09-11T22:43:42.954 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:42.363+0000 7f732c8a5640 1 -- 172.21.15.152:0/1623142922 wait complete. 2024-09-11T22:43:42.954 INFO:teuthology.orchestra.run.smithi152.stdout:Deploying unmanaged mon service... 2024-09-11T22:43:43.564 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:43 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' 2024-09-11T22:43:43.564 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:43 smithi152 ceph-mon[31181]: Added host smithi152 2024-09-11T22:43:43.564 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:43 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:43:44.394 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T22:43:44.394 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.366+0000 7f6943c76640 1 Processor -- start 2024-09-11T22:43:44.394 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.367+0000 7f6943c76640 1 -- start start 2024-09-11T22:43:44.394 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.367+0000 7f6943c76640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c07de50 0x7f693c07c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:44.394 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.367+0000 7f6943c76640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f693c07c880 con 0x7f693c07de50 2024-09-11T22:43:44.394 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.367+0000 7f6941a12640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c07de50 0x7f693c07c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:44.394 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.367+0000 7f6941a12640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c07de50 0x7f693c07c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40576/0 (socket says 172.21.15.152:40576) 2024-09-11T22:43:44.394 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.367+0000 7f6941a12640 1 -- 172.21.15.152:0/557443448 learned_addr learned my addr 172.21.15.152:0/557443448 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:44.395 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.367+0000 7f6941a12640 1 -- 172.21.15.152:0/557443448 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c07ca00 con 0x7f693c07de50 2024-09-11T22:43:44.395 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.368+0000 7f6941a12640 1 --2- 172.21.15.152:0/557443448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c07de50 0x7f693c07c2b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6938009820 tx=0x7f693802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1651e9de603538 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:44.395 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.368+0000 7f6940a10640 1 -- 172.21.15.152:0/557443448 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6938037070 con 0x7f693c07de50 2024-09-11T22:43:44.395 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.368+0000 7f6940a10640 1 -- 172.21.15.152:0/557443448 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6938032900 con 0x7f693c07de50 2024-09-11T22:43:44.395 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.368+0000 7f6940a10640 1 -- 172.21.15.152:0/557443448 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6938030850 con 0x7f693c07de50 2024-09-11T22:43:44.395 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.369+0000 7f6943c76640 1 -- 172.21.15.152:0/557443448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c07de50 msgr2=0x7f693c07c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:44.395 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.369+0000 7f6943c76640 1 --2- 172.21.15.152:0/557443448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c07de50 0x7f693c07c2b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6938009820 tx=0x7f693802a270 comp rx=0 tx=0).stop 2024-09-11T22:43:44.395 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.369+0000 7f6943c76640 1 -- 172.21.15.152:0/557443448 shutdown_connections 2024-09-11T22:43:44.395 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.369+0000 7f6943c76640 1 --2- 172.21.15.152:0/557443448 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c07de50 0x7f693c07c2b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:44.395 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.369+0000 7f6943c76640 1 -- 172.21.15.152:0/557443448 >> 172.21.15.152:0/557443448 conn(0x7f693c102db0 msgr2=0x7f693c1051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:44.395 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.369+0000 7f6943c76640 1 -- 172.21.15.152:0/557443448 shutdown_connections 2024-09-11T22:43:44.395 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.369+0000 7f6943c76640 1 -- 172.21.15.152:0/557443448 wait complete. 2024-09-11T22:43:44.396 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.370+0000 7f6943c76640 1 Processor -- start 2024-09-11T22:43:44.396 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.370+0000 7f6943c76640 1 -- start start 2024-09-11T22:43:44.396 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.370+0000 7f6943c76640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c07de50 0x7f693c19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:44.396 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.370+0000 7f6943c76640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f693c10dc50 con 0x7f693c07de50 2024-09-11T22:43:44.396 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.370+0000 7f6941a12640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c07de50 0x7f693c19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:44.396 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.370+0000 7f6941a12640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c07de50 0x7f693c19d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40578/0 (socket says 172.21.15.152:40578) 2024-09-11T22:43:44.396 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.370+0000 7f6941a12640 1 -- 172.21.15.152:0/1332169169 learned_addr learned my addr 172.21.15.152:0/1332169169 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:44.396 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.370+0000 7f6941a12640 1 -- 172.21.15.152:0/1332169169 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c19dd50 con 0x7f693c07de50 2024-09-11T22:43:44.396 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.370+0000 7f6941a12640 1 --2- 172.21.15.152:0/1332169169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c07de50 0x7f693c19d810 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6938031000 tx=0x7f69380326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:44.396 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.370+0000 7f6932ffd640 1 -- 172.21.15.152:0/1332169169 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6938042070 con 0x7f693c07de50 2024-09-11T22:43:44.396 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.371+0000 7f6932ffd640 1 -- 172.21.15.152:0/1332169169 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6938032be0 con 0x7f693c07de50 2024-09-11T22:43:44.397 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.371+0000 7f6943c76640 1 -- 172.21.15.152:0/1332169169 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c19dfe0 con 0x7f693c07de50 2024-09-11T22:43:44.397 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.371+0000 7f6932ffd640 1 -- 172.21.15.152:0/1332169169 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6938037040 con 0x7f693c07de50 2024-09-11T22:43:44.397 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.371+0000 7f6943c76640 1 -- 172.21.15.152:0/1332169169 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c1a0cd0 con 0x7f693c07de50 2024-09-11T22:43:44.397 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.371+0000 7f6932ffd640 1 -- 172.21.15.152:0/1332169169 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6938045070 con 0x7f693c07de50 2024-09-11T22:43:44.397 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.372+0000 7f6932ffd640 1 --2- 172.21.15.152:0/1332169169 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f6928038f70 0x7f692803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:44.397 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.372+0000 7f6943c76640 1 -- 172.21.15.152:0/1332169169 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693c07c330 con 0x7f693c07de50 2024-09-11T22:43:44.397 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.372+0000 7f6932ffd640 1 -- 172.21.15.152:0/1332169169 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f693806bfa0 con 0x7f693c07de50 2024-09-11T22:43:44.397 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.372+0000 7f6941211640 1 --2- 172.21.15.152:0/1332169169 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f6928038f70 0x7f692803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:44.397 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.372+0000 7f6941211640 1 --2- 172.21.15.152:0/1332169169 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f6928038f70 0x7f692803b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f692c0098c0 tx=0x7f692c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:44.397 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.377+0000 7f6932ffd640 1 -- 172.21.15.152:0/1332169169 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f693802f2a0 con 0x7f693c07de50 2024-09-11T22:43:44.397 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.563+0000 7f6943c76640 1 -- 172.21.15.152:0/1332169169 --> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f693c104100 con 0x7f6928038f70 2024-09-11T22:43:44.397 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.568+0000 7f6932ffd640 1 -- 172.21.15.152:0/1332169169 <== mgr.14118 v2:172.21.15.152:6800/1868213162 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f693c104100 con 0x7f6928038f70 2024-09-11T22:43:44.398 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.570+0000 7f6943c76640 1 -- 172.21.15.152:0/1332169169 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f6928038f70 msgr2=0x7f692803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:44.398 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.570+0000 7f6943c76640 1 --2- 172.21.15.152:0/1332169169 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f6928038f70 0x7f692803b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f692c0098c0 tx=0x7f692c006db0 comp rx=0 tx=0).stop 2024-09-11T22:43:44.398 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.570+0000 7f6943c76640 1 -- 172.21.15.152:0/1332169169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c07de50 msgr2=0x7f693c19d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:44.398 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.570+0000 7f6943c76640 1 --2- 172.21.15.152:0/1332169169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c07de50 0x7f693c19d810 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6938031000 tx=0x7f69380326e0 comp rx=0 tx=0).stop 2024-09-11T22:43:44.398 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.570+0000 7f6943c76640 1 -- 172.21.15.152:0/1332169169 shutdown_connections 2024-09-11T22:43:44.398 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.570+0000 7f6943c76640 1 --2- 172.21.15.152:0/1332169169 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f6928038f70 0x7f692803b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:44.398 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.570+0000 7f6943c76640 1 --2- 172.21.15.152:0/1332169169 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f693c07de50 0x7f693c19d810 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:44.398 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.570+0000 7f6943c76640 1 -- 172.21.15.152:0/1332169169 >> 172.21.15.152:0/1332169169 conn(0x7f693c102db0 msgr2=0x7f693c1039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:44.398 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.570+0000 7f6943c76640 1 -- 172.21.15.152:0/1332169169 shutdown_connections 2024-09-11T22:43:44.398 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:43.571+0000 7f6943c76640 1 -- 172.21.15.152:0/1332169169 wait complete. 2024-09-11T22:43:44.398 INFO:teuthology.orchestra.run.smithi152.stdout:Deploying unmanaged mgr service... 2024-09-11T22:43:44.722 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:44 smithi152 ceph-mon[31181]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:43:44.722 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:44 smithi152 ceph-mon[31181]: Saving service mon spec with placement count:5 2024-09-11T22:43:44.722 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:44 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' 2024-09-11T22:43:45.743 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T22:43:45.744 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.803+0000 7f180d6e4640 1 Processor -- start 2024-09-11T22:43:45.744 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.803+0000 7f180d6e4640 1 -- start start 2024-09-11T22:43:45.744 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.804+0000 7f180d6e4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1808107620 0x7f1808107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:45.744 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.804+0000 7f180d6e4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1808107ff0 con 0x7f1808107620 2024-09-11T22:43:45.744 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.804+0000 7f1806ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1808107620 0x7f1808107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:45.744 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.804+0000 7f1806ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1808107620 0x7f1808107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40594/0 (socket says 172.21.15.152:40594) 2024-09-11T22:43:45.744 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.804+0000 7f1806ffd640 1 -- 172.21.15.152:0/2073923443 learned_addr learned my addr 172.21.15.152:0/2073923443 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:45.744 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.804+0000 7f1806ffd640 1 -- 172.21.15.152:0/2073923443 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1808108820 con 0x7f1808107620 2024-09-11T22:43:45.744 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.805+0000 7f1806ffd640 1 --2- 172.21.15.152:0/2073923443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1808107620 0x7f1808107a20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009820 tx=0x7f17f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897b8d45f70400cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:45.744 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.805+0000 7f1805ffb640 1 -- 172.21.15.152:0/2073923443 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17f8037070 con 0x7f1808107620 2024-09-11T22:43:45.744 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.805+0000 7f1805ffb640 1 -- 172.21.15.152:0/2073923443 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f17f8032900 con 0x7f1808107620 2024-09-11T22:43:45.745 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.805+0000 7f1805ffb640 1 -- 172.21.15.152:0/2073923443 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17f8030850 con 0x7f1808107620 2024-09-11T22:43:45.745 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.805+0000 7f180d6e4640 1 -- 172.21.15.152:0/2073923443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1808107620 msgr2=0x7f1808107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:45.745 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.805+0000 7f180d6e4640 1 --2- 172.21.15.152:0/2073923443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1808107620 0x7f1808107a20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009820 tx=0x7f17f802a270 comp rx=0 tx=0).stop 2024-09-11T22:43:45.745 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.805+0000 7f180d6e4640 1 -- 172.21.15.152:0/2073923443 shutdown_connections 2024-09-11T22:43:45.745 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.805+0000 7f180d6e4640 1 --2- 172.21.15.152:0/2073923443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1808107620 0x7f1808107a20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:45.745 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.805+0000 7f180d6e4640 1 -- 172.21.15.152:0/2073923443 >> 172.21.15.152:0/2073923443 conn(0x7f1808102db0 msgr2=0x7f18081051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:45.745 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.806+0000 7f180d6e4640 1 -- 172.21.15.152:0/2073923443 shutdown_connections 2024-09-11T22:43:45.745 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.806+0000 7f180d6e4640 1 -- 172.21.15.152:0/2073923443 wait complete. 2024-09-11T22:43:45.745 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.806+0000 7f180d6e4640 1 Processor -- start 2024-09-11T22:43:45.745 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.806+0000 7f180d6e4640 1 -- start start 2024-09-11T22:43:45.745 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.806+0000 7f180d6e4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1808107620 0x7f180819b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:45.745 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.806+0000 7f180d6e4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f180810bbd0 con 0x7f1808107620 2024-09-11T22:43:45.746 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.807+0000 7f1806ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1808107620 0x7f180819b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:45.746 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.807+0000 7f1806ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1808107620 0x7f180819b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40598/0 (socket says 172.21.15.152:40598) 2024-09-11T22:43:45.746 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.807+0000 7f1806ffd640 1 -- 172.21.15.152:0/3913178410 learned_addr learned my addr 172.21.15.152:0/3913178410 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:45.746 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.807+0000 7f1806ffd640 1 -- 172.21.15.152:0/3913178410 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180819bc00 con 0x7f1808107620 2024-09-11T22:43:45.746 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.807+0000 7f1806ffd640 1 --2- 172.21.15.152:0/3913178410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1808107620 0x7f180819b6c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009940 tx=0x7f17f80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:45.746 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.807+0000 7f17effff640 1 -- 172.21.15.152:0/3913178410 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17f8042070 con 0x7f1808107620 2024-09-11T22:43:45.746 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.807+0000 7f17effff640 1 -- 172.21.15.152:0/3913178410 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f17f8032be0 con 0x7f1808107620 2024-09-11T22:43:45.746 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.807+0000 7f180d6e4640 1 -- 172.21.15.152:0/3913178410 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f180819be90 con 0x7f1808107620 2024-09-11T22:43:45.746 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.807+0000 7f17effff640 1 -- 172.21.15.152:0/3913178410 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17f8037040 con 0x7f1808107620 2024-09-11T22:43:45.746 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.807+0000 7f180d6e4640 1 -- 172.21.15.152:0/3913178410 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f180819eb80 con 0x7f1808107620 2024-09-11T22:43:45.746 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.808+0000 7f17effff640 1 -- 172.21.15.152:0/3913178410 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f17f8045070 con 0x7f1808107620 2024-09-11T22:43:45.746 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.808+0000 7f17effff640 1 --2- 172.21.15.152:0/3913178410 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f17f0038fc0 0x7f17f003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:45.746 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.808+0000 7f17effff640 1 -- 172.21.15.152:0/3913178410 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f17f806c010 con 0x7f1808107620 2024-09-11T22:43:45.746 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.808+0000 7f18067fc640 1 --2- 172.21.15.152:0/3913178410 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f17f0038fc0 0x7f17f003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:45.747 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.808+0000 7f180d6e4640 1 -- 172.21.15.152:0/3913178410 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f180810bd50 con 0x7f1808107620 2024-09-11T22:43:45.747 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.809+0000 7f18067fc640 1 --2- 172.21.15.152:0/3913178410 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f17f0038fc0 0x7f17f003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0098c0 tx=0x7f17fc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:45.747 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:44.813+0000 7f17effff640 1 -- 172.21.15.152:0/3913178410 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17f8046e00 con 0x7f1808107620 2024-09-11T22:43:45.747 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:45.008+0000 7f180d6e4640 1 -- 172.21.15.152:0/3913178410 --> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1808104090 con 0x7f17f0038fc0 2024-09-11T22:43:45.747 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:45.013+0000 7f17effff640 1 -- 172.21.15.152:0/3913178410 <== mgr.14118 v2:172.21.15.152:6800/1868213162 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1808104090 con 0x7f17f0038fc0 2024-09-11T22:43:45.747 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:45.015+0000 7f180d6e4640 1 -- 172.21.15.152:0/3913178410 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f17f0038fc0 msgr2=0x7f17f003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:45.747 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:45.015+0000 7f180d6e4640 1 --2- 172.21.15.152:0/3913178410 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f17f0038fc0 0x7f17f003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0098c0 tx=0x7f17fc006db0 comp rx=0 tx=0).stop 2024-09-11T22:43:45.747 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:45.015+0000 7f180d6e4640 1 -- 172.21.15.152:0/3913178410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1808107620 msgr2=0x7f180819b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:45.747 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:45.015+0000 7f180d6e4640 1 --2- 172.21.15.152:0/3913178410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1808107620 0x7f180819b6c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f17f8009940 tx=0x7f17f80326e0 comp rx=0 tx=0).stop 2024-09-11T22:43:45.747 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:45.015+0000 7f180d6e4640 1 -- 172.21.15.152:0/3913178410 shutdown_connections 2024-09-11T22:43:45.747 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:45.015+0000 7f180d6e4640 1 --2- 172.21.15.152:0/3913178410 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f17f0038fc0 0x7f17f003b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:45.747 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:45.015+0000 7f180d6e4640 1 --2- 172.21.15.152:0/3913178410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1808107620 0x7f180819b6c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:45.747 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:45.015+0000 7f180d6e4640 1 -- 172.21.15.152:0/3913178410 >> 172.21.15.152:0/3913178410 conn(0x7f1808102db0 msgr2=0x7f1808103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:45.747 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:45.015+0000 7f180d6e4640 1 -- 172.21.15.152:0/3913178410 shutdown_connections 2024-09-11T22:43:45.747 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:45.015+0000 7f180d6e4640 1 -- 172.21.15.152:0/3913178410 wait complete. 2024-09-11T22:43:46.257 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:46 smithi152 ceph-mon[31181]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:43:46.257 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:46 smithi152 ceph-mon[31181]: Saving service mgr spec with placement count:2 2024-09-11T22:43:46.257 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:46 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' 2024-09-11T22:43:47.136 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.118+0000 7f76c3535640 1 Processor -- start 2024-09-11T22:43:47.136 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.118+0000 7f76c3535640 1 -- start start 2024-09-11T22:43:47.136 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.118+0000 7f76c3535640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76bc07deb0 0x7f76bc07c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:47.137 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.118+0000 7f76c3535640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc07c8e0 con 0x7f76bc07deb0 2024-09-11T22:43:47.137 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.119+0000 7f76c12d1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76bc07deb0 0x7f76bc07c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:47.137 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.119+0000 7f76c12d1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76bc07deb0 0x7f76bc07c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40608/0 (socket says 172.21.15.152:40608) 2024-09-11T22:43:47.137 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.119+0000 7f76c12d1640 1 -- 172.21.15.152:0/4012403951 learned_addr learned my addr 172.21.15.152:0/4012403951 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:47.137 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.119+0000 7f76c12d1640 1 -- 172.21.15.152:0/4012403951 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76bc07ca60 con 0x7f76bc07deb0 2024-09-11T22:43:47.137 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.120+0000 7f76c12d1640 1 --2- 172.21.15.152:0/4012403951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76bc07deb0 0x7f76bc07c310 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009820 tx=0x7f76b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53653473e519e1ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:47.137 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.120+0000 7f76b3fff640 1 -- 172.21.15.152:0/4012403951 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76b8037070 con 0x7f76bc07deb0 2024-09-11T22:43:47.137 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.120+0000 7f76b3fff640 1 -- 172.21.15.152:0/4012403951 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f76b8032900 con 0x7f76bc07deb0 2024-09-11T22:43:47.137 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.120+0000 7f76b3fff640 1 -- 172.21.15.152:0/4012403951 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76b8030850 con 0x7f76bc07deb0 2024-09-11T22:43:47.137 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.120+0000 7f76c3535640 1 -- 172.21.15.152:0/4012403951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76bc07deb0 msgr2=0x7f76bc07c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:47.137 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.120+0000 7f76c3535640 1 --2- 172.21.15.152:0/4012403951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76bc07deb0 0x7f76bc07c310 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009820 tx=0x7f76b802a270 comp rx=0 tx=0).stop 2024-09-11T22:43:47.137 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.120+0000 7f76c3535640 1 -- 172.21.15.152:0/4012403951 shutdown_connections 2024-09-11T22:43:47.137 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.120+0000 7f76c3535640 1 --2- 172.21.15.152:0/4012403951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76bc07deb0 0x7f76bc07c310 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:47.138 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.120+0000 7f76c3535640 1 -- 172.21.15.152:0/4012403951 >> 172.21.15.152:0/4012403951 conn(0x7f76bc102db0 msgr2=0x7f76bc1051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:47.138 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.120+0000 7f76c3535640 1 -- 172.21.15.152:0/4012403951 shutdown_connections 2024-09-11T22:43:47.138 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.120+0000 7f76c3535640 1 -- 172.21.15.152:0/4012403951 wait complete. 2024-09-11T22:43:47.138 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.121+0000 7f76c3535640 1 Processor -- start 2024-09-11T22:43:47.138 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.121+0000 7f76c3535640 1 -- start start 2024-09-11T22:43:47.138 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.121+0000 7f76c3535640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76bc07deb0 0x7f76bc19d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:47.138 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.121+0000 7f76c3535640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc10dc50 con 0x7f76bc07deb0 2024-09-11T22:43:47.138 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.121+0000 7f76c12d1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76bc07deb0 0x7f76bc19d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:47.138 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.121+0000 7f76c12d1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76bc07deb0 0x7f76bc19d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40616/0 (socket says 172.21.15.152:40616) 2024-09-11T22:43:47.138 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.121+0000 7f76c12d1640 1 -- 172.21.15.152:0/4091307139 learned_addr learned my addr 172.21.15.152:0/4091307139 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:47.138 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.122+0000 7f76c12d1640 1 -- 172.21.15.152:0/4091307139 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76bc19ddc0 con 0x7f76bc07deb0 2024-09-11T22:43:47.138 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.122+0000 7f76c12d1640 1 --2- 172.21.15.152:0/4091307139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76bc07deb0 0x7f76bc19d880 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009940 tx=0x7f76b80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:47.139 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.122+0000 7f76b27fc640 1 -- 172.21.15.152:0/4091307139 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76b8042070 con 0x7f76bc07deb0 2024-09-11T22:43:47.139 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.122+0000 7f76c3535640 1 -- 172.21.15.152:0/4091307139 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76bc19e050 con 0x7f76bc07deb0 2024-09-11T22:43:47.139 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.122+0000 7f76b27fc640 1 -- 172.21.15.152:0/4091307139 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f76b8032be0 con 0x7f76bc07deb0 2024-09-11T22:43:47.139 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.122+0000 7f76b27fc640 1 -- 172.21.15.152:0/4091307139 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76b8037040 con 0x7f76bc07deb0 2024-09-11T22:43:47.139 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.122+0000 7f76c3535640 1 -- 172.21.15.152:0/4091307139 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76bc1a0d40 con 0x7f76bc07deb0 2024-09-11T22:43:47.139 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.123+0000 7f76b27fc640 1 -- 172.21.15.152:0/4091307139 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f76b8045070 con 0x7f76bc07deb0 2024-09-11T22:43:47.139 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.123+0000 7f76b27fc640 1 --2- 172.21.15.152:0/4091307139 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f76a8038fc0 0x7f76a803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:47.139 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.123+0000 7f76b27fc640 1 -- 172.21.15.152:0/4091307139 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f76b806bef0 con 0x7f76bc07deb0 2024-09-11T22:43:47.139 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.123+0000 7f76c3535640 1 -- 172.21.15.152:0/4091307139 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76bc07c390 con 0x7f76bc07deb0 2024-09-11T22:43:47.139 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.123+0000 7f76c0ad0640 1 --2- 172.21.15.152:0/4091307139 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f76a8038fc0 0x7f76a803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:47.139 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.124+0000 7f76c0ad0640 1 --2- 172.21.15.152:0/4091307139 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f76a8038fc0 0x7f76a803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0098c0 tx=0x7f76ac006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:47.139 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.128+0000 7f76b27fc640 1 -- 172.21.15.152:0/4091307139 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76b8045360 con 0x7f76bc07deb0 2024-09-11T22:43:47.139 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.297+0000 7f76c3535640 1 -- 172.21.15.152:0/4091307139 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f76bc1a12b0 con 0x7f76bc07deb0 2024-09-11T22:43:47.140 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.300+0000 7f76b27fc640 1 -- 172.21.15.152:0/4091307139 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f76b803ae50 con 0x7f76bc07deb0 2024-09-11T22:43:47.140 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.303+0000 7f76c3535640 1 -- 172.21.15.152:0/4091307139 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f76a8038fc0 msgr2=0x7f76a803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:47.140 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.303+0000 7f76c3535640 1 --2- 172.21.15.152:0/4091307139 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f76a8038fc0 0x7f76a803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f76ac0098c0 tx=0x7f76ac006db0 comp rx=0 tx=0).stop 2024-09-11T22:43:47.140 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.303+0000 7f76c3535640 1 -- 172.21.15.152:0/4091307139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76bc07deb0 msgr2=0x7f76bc19d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:47.140 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.303+0000 7f76c3535640 1 --2- 172.21.15.152:0/4091307139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76bc07deb0 0x7f76bc19d880 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009940 tx=0x7f76b80326e0 comp rx=0 tx=0).stop 2024-09-11T22:43:47.140 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.303+0000 7f76c3535640 1 -- 172.21.15.152:0/4091307139 shutdown_connections 2024-09-11T22:43:47.140 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.303+0000 7f76c3535640 1 --2- 172.21.15.152:0/4091307139 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f76a8038fc0 0x7f76a803b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:47.140 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.303+0000 7f76c3535640 1 --2- 172.21.15.152:0/4091307139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76bc07deb0 0x7f76bc19d880 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:47.140 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.303+0000 7f76c3535640 1 -- 172.21.15.152:0/4091307139 >> 172.21.15.152:0/4091307139 conn(0x7f76bc102db0 msgr2=0x7f76bc1039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:47.140 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.303+0000 7f76c3535640 1 -- 172.21.15.152:0/4091307139 shutdown_connections 2024-09-11T22:43:47.140 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:46.303+0000 7f76c3535640 1 -- 172.21.15.152:0/4091307139 wait complete. 2024-09-11T22:43:47.448 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:47 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4091307139' entity='client.admin' 2024-09-11T22:43:48.529 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.527+0000 7f6bc3988640 1 Processor -- start 2024-09-11T22:43:48.529 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.528+0000 7f6bc3988640 1 -- start start 2024-09-11T22:43:48.529 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.528+0000 7f6bc3988640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc07de70 0x7f6bbc07c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:48.529 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.528+0000 7f6bc3988640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc07c8a0 con 0x7f6bbc07de70 2024-09-11T22:43:48.529 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.528+0000 7f6bc1724640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc07de70 0x7f6bbc07c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:48.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.528+0000 7f6bc1724640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc07de70 0x7f6bbc07c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40632/0 (socket says 172.21.15.152:40632) 2024-09-11T22:43:48.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.528+0000 7f6bc1724640 1 -- 172.21.15.152:0/271578780 learned_addr learned my addr 172.21.15.152:0/271578780 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:48.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.528+0000 7f6bc1724640 1 -- 172.21.15.152:0/271578780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc07ca20 con 0x7f6bbc07de70 2024-09-11T22:43:48.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.529+0000 7f6bc1724640 1 --2- 172.21.15.152:0/271578780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc07de70 0x7f6bbc07c2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009820 tx=0x7f6bac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154fe4fb0b44f911 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:48.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.529+0000 7f6bb3fff640 1 -- 172.21.15.152:0/271578780 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bac037070 con 0x7f6bbc07de70 2024-09-11T22:43:48.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.529+0000 7f6bb3fff640 1 -- 172.21.15.152:0/271578780 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6bac032900 con 0x7f6bbc07de70 2024-09-11T22:43:48.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.529+0000 7f6bb3fff640 1 -- 172.21.15.152:0/271578780 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bac030850 con 0x7f6bbc07de70 2024-09-11T22:43:48.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.530+0000 7f6bc3988640 1 -- 172.21.15.152:0/271578780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc07de70 msgr2=0x7f6bbc07c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:48.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.530+0000 7f6bc3988640 1 --2- 172.21.15.152:0/271578780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc07de70 0x7f6bbc07c2d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009820 tx=0x7f6bac02a270 comp rx=0 tx=0).stop 2024-09-11T22:43:48.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.530+0000 7f6bc3988640 1 -- 172.21.15.152:0/271578780 shutdown_connections 2024-09-11T22:43:48.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.530+0000 7f6bc3988640 1 --2- 172.21.15.152:0/271578780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc07de70 0x7f6bbc07c2d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:48.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.530+0000 7f6bc3988640 1 -- 172.21.15.152:0/271578780 >> 172.21.15.152:0/271578780 conn(0x7f6bbc102db0 msgr2=0x7f6bbc1051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:48.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.530+0000 7f6bc3988640 1 -- 172.21.15.152:0/271578780 shutdown_connections 2024-09-11T22:43:48.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.530+0000 7f6bc3988640 1 -- 172.21.15.152:0/271578780 wait complete. 2024-09-11T22:43:48.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.530+0000 7f6bc3988640 1 Processor -- start 2024-09-11T22:43:48.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.530+0000 7f6bc3988640 1 -- start start 2024-09-11T22:43:48.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.531+0000 7f6bc3988640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc07de70 0x7f6bbc19d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:48.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.531+0000 7f6bc3988640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc10dc90 con 0x7f6bbc07de70 2024-09-11T22:43:48.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.531+0000 7f6bc1724640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc07de70 0x7f6bbc19d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:48.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.531+0000 7f6bc1724640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc07de70 0x7f6bbc19d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40640/0 (socket says 172.21.15.152:40640) 2024-09-11T22:43:48.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.531+0000 7f6bc1724640 1 -- 172.21.15.152:0/1638070781 learned_addr learned my addr 172.21.15.152:0/1638070781 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:48.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.531+0000 7f6bc1724640 1 -- 172.21.15.152:0/1638070781 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc19dd60 con 0x7f6bbc07de70 2024-09-11T22:43:48.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.531+0000 7f6bc1724640 1 --2- 172.21.15.152:0/1638070781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc07de70 0x7f6bbc19d820 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0097f0 tx=0x7f6bac030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:48.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.531+0000 7f6bb27fc640 1 -- 172.21.15.152:0/1638070781 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bac042070 con 0x7f6bbc07de70 2024-09-11T22:43:48.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.531+0000 7f6bb27fc640 1 -- 172.21.15.152:0/1638070781 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6bac032ca0 con 0x7f6bbc07de70 2024-09-11T22:43:48.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.531+0000 7f6bc3988640 1 -- 172.21.15.152:0/1638070781 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bbc19dff0 con 0x7f6bbc07de70 2024-09-11T22:43:48.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.531+0000 7f6bb27fc640 1 -- 172.21.15.152:0/1638070781 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bac037040 con 0x7f6bbc07de70 2024-09-11T22:43:48.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.531+0000 7f6bc3988640 1 -- 172.21.15.152:0/1638070781 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bbc1a0ce0 con 0x7f6bbc07de70 2024-09-11T22:43:48.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.532+0000 7f6bb27fc640 1 -- 172.21.15.152:0/1638070781 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6bac045070 con 0x7f6bbc07de70 2024-09-11T22:43:48.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.532+0000 7f6bb27fc640 1 --2- 172.21.15.152:0/1638070781 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f6ba8038fc0 0x7f6ba803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:48.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.532+0000 7f6bb27fc640 1 -- 172.21.15.152:0/1638070781 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6bac03db40 con 0x7f6bbc07de70 2024-09-11T22:43:48.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.533+0000 7f6bc0f23640 1 --2- 172.21.15.152:0/1638070781 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f6ba8038fc0 0x7f6ba803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:48.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.533+0000 7f6bc3988640 1 -- 172.21.15.152:0/1638070781 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bbc07c350 con 0x7f6bbc07de70 2024-09-11T22:43:48.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.533+0000 7f6bc0f23640 1 --2- 172.21.15.152:0/1638070781 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f6ba8038fc0 0x7f6ba803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80098c0 tx=0x7f6bb8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:48.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.537+0000 7f6bb27fc640 1 -- 172.21.15.152:0/1638070781 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6bac045520 con 0x7f6bbc07de70 2024-09-11T22:43:48.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.705+0000 7f6bc3988640 1 -- 172.21.15.152:0/1638070781 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f6bbc1a1200 con 0x7f6bbc07de70 2024-09-11T22:43:48.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.708+0000 7f6bb27fc640 1 -- 172.21.15.152:0/1638070781 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v7)=0 v7) ==== 130+0+0 (secure 0 0 0) 0x7f6bac03add0 con 0x7f6bbc07de70 2024-09-11T22:43:48.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.710+0000 7f6bc3988640 1 -- 172.21.15.152:0/1638070781 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f6ba8038fc0 msgr2=0x7f6ba803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:48.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.710+0000 7f6bc3988640 1 --2- 172.21.15.152:0/1638070781 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f6ba8038fc0 0x7f6ba803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80098c0 tx=0x7f6bb8006db0 comp rx=0 tx=0).stop 2024-09-11T22:43:48.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.710+0000 7f6bc3988640 1 -- 172.21.15.152:0/1638070781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc07de70 msgr2=0x7f6bbc19d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:48.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.710+0000 7f6bc3988640 1 --2- 172.21.15.152:0/1638070781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc07de70 0x7f6bbc19d820 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0097f0 tx=0x7f6bac030ec0 comp rx=0 tx=0).stop 2024-09-11T22:43:48.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.710+0000 7f6bc3988640 1 -- 172.21.15.152:0/1638070781 shutdown_connections 2024-09-11T22:43:48.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.710+0000 7f6bc3988640 1 --2- 172.21.15.152:0/1638070781 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f6ba8038fc0 0x7f6ba803b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:48.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.710+0000 7f6bc3988640 1 --2- 172.21.15.152:0/1638070781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bbc07de70 0x7f6bbc19d820 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:48.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.710+0000 7f6bc3988640 1 -- 172.21.15.152:0/1638070781 >> 172.21.15.152:0/1638070781 conn(0x7f6bbc102db0 msgr2=0x7f6bbc103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:48.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.710+0000 7f6bc3988640 1 -- 172.21.15.152:0/1638070781 shutdown_connections 2024-09-11T22:43:48.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:47.710+0000 7f6bc3988640 1 -- 172.21.15.152:0/1638070781 wait complete. 2024-09-11T22:43:48.533 INFO:teuthology.orchestra.run.smithi152.stdout:Enabling the dashboard module... 2024-09-11T22:43:48.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:48 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1638070781' entity='client.admin' 2024-09-11T22:43:48.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:48 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' 2024-09-11T22:43:50.109 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:49 smithi152 ceph-mon[31181]: from='mgr.14118 172.21.15.152:0/1551276623' entity='mgr.x' 2024-09-11T22:43:50.110 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:49 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3585272220' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T22:43:50.110 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:49 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: ignoring --setuser ceph since I am not root 2024-09-11T22:43:50.110 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:49 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: ignoring --setgroup ceph since I am not root 2024-09-11T22:43:50.110 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:49 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:49.944+0000 7f579894d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:43:50.110 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:50.004+0000 7f579894d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:43:50.372 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:50.163+0000 7f579894d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:43:50.372 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:50.232+0000 7f579894d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:43:50.476 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.971+0000 7fddb6f9e640 1 Processor -- start 2024-09-11T22:43:50.476 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.971+0000 7fddb6f9e640 1 -- start start 2024-09-11T22:43:50.476 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.971+0000 7fddb6f9e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddb0107470 0x7fddb0107870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:50.476 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.971+0000 7fddb6f9e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fddb0107e40 con 0x7fddb0107470 2024-09-11T22:43:50.476 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.971+0000 7fddb5f9c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddb0107470 0x7fddb0107870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:50.476 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.971+0000 7fddb5f9c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddb0107470 0x7fddb0107870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44232/0 (socket says 172.21.15.152:44232) 2024-09-11T22:43:50.476 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.971+0000 7fddb5f9c640 1 -- 172.21.15.152:0/2830704724 learned_addr learned my addr 172.21.15.152:0/2830704724 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:50.477 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.971+0000 7fddb5f9c640 1 -- 172.21.15.152:0/2830704724 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb0108620 con 0x7fddb0107470 2024-09-11T22:43:50.477 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.972+0000 7fddb5f9c640 1 --2- 172.21.15.152:0/2830704724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddb0107470 0x7fddb0107870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fddac009820 tx=0x7fddac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6337254266033917 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:50.477 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.972+0000 7fddb4f9a640 1 -- 172.21.15.152:0/2830704724 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddac037070 con 0x7fddb0107470 2024-09-11T22:43:50.477 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.972+0000 7fddb4f9a640 1 -- 172.21.15.152:0/2830704724 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddac032920 con 0x7fddb0107470 2024-09-11T22:43:50.477 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.972+0000 7fddb4f9a640 1 -- 172.21.15.152:0/2830704724 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddac030810 con 0x7fddb0107470 2024-09-11T22:43:50.477 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.973+0000 7fddb6f9e640 1 -- 172.21.15.152:0/2830704724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddb0107470 msgr2=0x7fddb0107870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:50.477 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.973+0000 7fddb6f9e640 1 --2- 172.21.15.152:0/2830704724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddb0107470 0x7fddb0107870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fddac009820 tx=0x7fddac02a270 comp rx=0 tx=0).stop 2024-09-11T22:43:50.477 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.973+0000 7fddb6f9e640 1 -- 172.21.15.152:0/2830704724 shutdown_connections 2024-09-11T22:43:50.477 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.973+0000 7fddb6f9e640 1 --2- 172.21.15.152:0/2830704724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddb0107470 0x7fddb0107870 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:50.477 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.973+0000 7fddb6f9e640 1 -- 172.21.15.152:0/2830704724 >> 172.21.15.152:0/2830704724 conn(0x7fddb0102c00 msgr2=0x7fddb0105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:50.477 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.973+0000 7fddb6f9e640 1 -- 172.21.15.152:0/2830704724 shutdown_connections 2024-09-11T22:43:50.477 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.973+0000 7fddb6f9e640 1 -- 172.21.15.152:0/2830704724 wait complete. 2024-09-11T22:43:50.477 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.974+0000 7fddb6f9e640 1 Processor -- start 2024-09-11T22:43:50.478 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.974+0000 7fddb6f9e640 1 -- start start 2024-09-11T22:43:50.478 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.974+0000 7fddb6f9e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddb0107470 0x7fddb019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:50.478 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.974+0000 7fddb6f9e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fddb010b9d0 con 0x7fddb0107470 2024-09-11T22:43:50.478 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.974+0000 7fddb5f9c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddb0107470 0x7fddb019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:50.478 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.974+0000 7fddb5f9c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddb0107470 0x7fddb019b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44244/0 (socket says 172.21.15.152:44244) 2024-09-11T22:43:50.478 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.974+0000 7fddb5f9c640 1 -- 172.21.15.152:0/3585272220 learned_addr learned my addr 172.21.15.152:0/3585272220 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:50.478 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.974+0000 7fddb5f9c640 1 -- 172.21.15.152:0/3585272220 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb019b960 con 0x7fddb0107470 2024-09-11T22:43:50.478 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.975+0000 7fddb5f9c640 1 --2- 172.21.15.152:0/3585272220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddb0107470 0x7fddb019b420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fddac030560 tx=0x7fddac032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:50.478 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.975+0000 7fdda6ffd640 1 -- 172.21.15.152:0/3585272220 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddac037040 con 0x7fddb0107470 2024-09-11T22:43:50.478 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.975+0000 7fddb6f9e640 1 -- 172.21.15.152:0/3585272220 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb019bbf0 con 0x7fddb0107470 2024-09-11T22:43:50.478 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.975+0000 7fdda6ffd640 1 -- 172.21.15.152:0/3585272220 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddac040070 con 0x7fddb0107470 2024-09-11T22:43:50.478 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.975+0000 7fdda6ffd640 1 -- 172.21.15.152:0/3585272220 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddac03a740 con 0x7fddb0107470 2024-09-11T22:43:50.478 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.975+0000 7fddb6f9e640 1 -- 172.21.15.152:0/3585272220 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb019e8e0 con 0x7fddb0107470 2024-09-11T22:43:50.479 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.976+0000 7fdda6ffd640 1 -- 172.21.15.152:0/3585272220 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fddac03a8e0 con 0x7fddb0107470 2024-09-11T22:43:50.479 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.976+0000 7fdda6ffd640 1 --2- 172.21.15.152:0/3585272220 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fdd9c038fc0 0x7fdd9c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:50.479 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.976+0000 7fddb6f9e640 1 -- 172.21.15.152:0/3585272220 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb010bb50 con 0x7fddb0107470 2024-09-11T22:43:50.479 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.976+0000 7fdda6ffd640 1 -- 172.21.15.152:0/3585272220 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fddac06c560 con 0x7fddb0107470 2024-09-11T22:43:50.479 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.976+0000 7fddb579b640 1 --2- 172.21.15.152:0/3585272220 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fdd9c038fc0 0x7fdd9c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:50.479 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.977+0000 7fddb579b640 1 --2- 172.21.15.152:0/3585272220 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fdd9c038fc0 0x7fdd9c03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdda00098c0 tx=0x7fdda0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:50.479 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:48.981+0000 7fdda6ffd640 1 -- 172.21.15.152:0/3585272220 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fddac044c70 con 0x7fddb0107470 2024-09-11T22:43:50.479 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:49.220+0000 7fddb6f9e640 1 -- 172.21.15.152:0/3585272220 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fddb019eab0 con 0x7fddb0107470 2024-09-11T22:43:50.479 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:49.788+0000 7fdda6ffd640 1 -- 172.21.15.152:0/3585272220 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fddac041270 con 0x7fddb0107470 2024-09-11T22:43:50.479 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:49.790+0000 7fdda6ffd640 1 -- 172.21.15.152:0/3585272220 <== mon.0 v2:172.21.15.152:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fddac045980 con 0x7fddb0107470 2024-09-11T22:43:50.480 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:49.791+0000 7fddb6f9e640 1 -- 172.21.15.152:0/3585272220 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fdd9c038fc0 msgr2=0x7fdd9c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:50.480 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:49.791+0000 7fddb6f9e640 1 --2- 172.21.15.152:0/3585272220 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fdd9c038fc0 0x7fdd9c03b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdda00098c0 tx=0x7fdda0006db0 comp rx=0 tx=0).stop 2024-09-11T22:43:50.480 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:49.791+0000 7fddb6f9e640 1 -- 172.21.15.152:0/3585272220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddb0107470 msgr2=0x7fddb019b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:50.480 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:49.791+0000 7fddb6f9e640 1 --2- 172.21.15.152:0/3585272220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddb0107470 0x7fddb019b420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fddac030560 tx=0x7fddac032670 comp rx=0 tx=0).stop 2024-09-11T22:43:50.480 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:49.791+0000 7fddb6f9e640 1 -- 172.21.15.152:0/3585272220 shutdown_connections 2024-09-11T22:43:50.480 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:49.791+0000 7fddb6f9e640 1 --2- 172.21.15.152:0/3585272220 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fdd9c038fc0 0x7fdd9c03b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:50.480 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:49.791+0000 7fddb6f9e640 1 --2- 172.21.15.152:0/3585272220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fddb0107470 0x7fddb019b420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:50.480 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:49.791+0000 7fddb6f9e640 1 -- 172.21.15.152:0/3585272220 >> 172.21.15.152:0/3585272220 conn(0x7fddb0102c00 msgr2=0x7fddb0103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:50.480 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:49.792+0000 7fddb6f9e640 1 -- 172.21.15.152:0/3585272220 shutdown_connections 2024-09-11T22:43:50.480 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:49.792+0000 7fddb6f9e640 1 -- 172.21.15.152:0/3585272220 wait complete. 2024-09-11T22:43:50.713 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:50.387+0000 7f579894d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:43:50.714 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:50.539+0000 7f579894d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:43:51.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:50 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3585272220' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T22:43:51.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:50 smithi152 ceph-mon[31181]: mgrmap e9: x(active, since 17s) 2024-09-11T22:43:51.123 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:50.866+0000 7f579894d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:43:51.123 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:50.932+0000 7f579894d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:43:51.482 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:51 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:51.128+0000 7f579894d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:43:51.775 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:43:51.775 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T22:43:51.775 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T22:43:51.775 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T22:43:51.775 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T22:43:51.775 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:43:51.775 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.916+0000 7fd19ffc0640 1 Processor -- start 2024-09-11T22:43:51.775 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.917+0000 7fd19ffc0640 1 -- start start 2024-09-11T22:43:51.775 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.917+0000 7fd19ffc0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1981080e0 0x7fd19810a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:51.776 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.917+0000 7fd19ffc0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd19807bdd0 con 0x7fd1981080e0 2024-09-11T22:43:51.776 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.917+0000 7fd19dd5c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1981080e0 0x7fd19810a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:51.776 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.917+0000 7fd19dd5c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1981080e0 0x7fd19810a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44254/0 (socket says 172.21.15.152:44254) 2024-09-11T22:43:51.776 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.917+0000 7fd19dd5c640 1 -- 172.21.15.152:0/2355096196 learned_addr learned my addr 172.21.15.152:0/2355096196 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:51.776 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.917+0000 7fd19dd5c640 1 -- 172.21.15.152:0/2355096196 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19810aa10 con 0x7fd1981080e0 2024-09-11T22:43:51.776 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.918+0000 7fd19dd5c640 1 --2- 172.21.15.152:0/2355096196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1981080e0 0x7fd19810a4d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd194009820 tx=0x7fd19402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4517713ef7bb46ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:51.776 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.918+0000 7fd19cd5a640 1 -- 172.21.15.152:0/2355096196 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd194037070 con 0x7fd1981080e0 2024-09-11T22:43:51.776 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.918+0000 7fd19cd5a640 1 -- 172.21.15.152:0/2355096196 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd194032920 con 0x7fd1981080e0 2024-09-11T22:43:51.776 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.918+0000 7fd19cd5a640 1 -- 172.21.15.152:0/2355096196 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd194030810 con 0x7fd1981080e0 2024-09-11T22:43:51.776 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.919+0000 7fd19ffc0640 1 -- 172.21.15.152:0/2355096196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1981080e0 msgr2=0x7fd19810a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:51.776 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.919+0000 7fd19ffc0640 1 --2- 172.21.15.152:0/2355096196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1981080e0 0x7fd19810a4d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd194009820 tx=0x7fd19402a270 comp rx=0 tx=0).stop 2024-09-11T22:43:51.776 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.919+0000 7fd19ffc0640 1 -- 172.21.15.152:0/2355096196 shutdown_connections 2024-09-11T22:43:51.777 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.919+0000 7fd19ffc0640 1 --2- 172.21.15.152:0/2355096196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1981080e0 0x7fd19810a4d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:51.777 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.919+0000 7fd19ffc0640 1 -- 172.21.15.152:0/2355096196 >> 172.21.15.152:0/2355096196 conn(0x7fd198101ca0 msgr2=0x7fd1981040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:51.777 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.919+0000 7fd19ffc0640 1 -- 172.21.15.152:0/2355096196 shutdown_connections 2024-09-11T22:43:51.777 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.919+0000 7fd19ffc0640 1 -- 172.21.15.152:0/2355096196 wait complete. 2024-09-11T22:43:51.777 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.920+0000 7fd19ffc0640 1 Processor -- start 2024-09-11T22:43:51.777 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.920+0000 7fd19ffc0640 1 -- start start 2024-09-11T22:43:51.777 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.920+0000 7fd19ffc0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1981080e0 0x7fd198107960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:51.777 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.920+0000 7fd19ffc0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd19810ba30 con 0x7fd1981080e0 2024-09-11T22:43:51.778 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.920+0000 7fd19dd5c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1981080e0 0x7fd198107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:51.778 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.920+0000 7fd19dd5c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1981080e0 0x7fd198107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44258/0 (socket says 172.21.15.152:44258) 2024-09-11T22:43:51.778 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.920+0000 7fd19dd5c640 1 -- 172.21.15.152:0/1432840990 learned_addr learned my addr 172.21.15.152:0/1432840990 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:51.778 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.921+0000 7fd19dd5c640 1 -- 172.21.15.152:0/1432840990 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd198107ea0 con 0x7fd1981080e0 2024-09-11T22:43:51.778 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.921+0000 7fd19dd5c640 1 --2- 172.21.15.152:0/1432840990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1981080e0 0x7fd198107960 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd194030420 tx=0x7fd194032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:51.778 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.921+0000 7fd18effd640 1 -- 172.21.15.152:0/1432840990 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd194037040 con 0x7fd1981080e0 2024-09-11T22:43:51.778 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.921+0000 7fd18effd640 1 -- 172.21.15.152:0/1432840990 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd194040070 con 0x7fd1981080e0 2024-09-11T22:43:51.778 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.921+0000 7fd19ffc0640 1 -- 172.21.15.152:0/1432840990 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd198106090 con 0x7fd1981080e0 2024-09-11T22:43:51.778 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.922+0000 7fd18effd640 1 -- 172.21.15.152:0/1432840990 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd19403a6e0 con 0x7fd1981080e0 2024-09-11T22:43:51.778 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.922+0000 7fd19ffc0640 1 -- 172.21.15.152:0/1432840990 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1981065d0 con 0x7fd1981080e0 2024-09-11T22:43:51.778 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.922+0000 7fd18effd640 1 -- 172.21.15.152:0/1432840990 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fd19403a9e0 con 0x7fd1981080e0 2024-09-11T22:43:51.778 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.922+0000 7fd18effd640 1 --2- 172.21.15.152:0/1432840990 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fd184038fc0 0x7fd18403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:51.778 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.922+0000 7fd18effd640 1 -- 172.21.15.152:0/1432840990 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd194071580 con 0x7fd1981080e0 2024-09-11T22:43:51.779 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.922+0000 7fd19d55b640 1 -- 172.21.15.152:0/1432840990 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fd184038fc0 msgr2=0x7fd18403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/1868213162 2024-09-11T22:43:51.779 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.922+0000 7fd19d55b640 1 --2- 172.21.15.152:0/1432840990 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fd184038fc0 0x7fd18403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:43:51.779 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.923+0000 7fd19ffc0640 1 -- 172.21.15.152:0/1432840990 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd198108540 con 0x7fd1981080e0 2024-09-11T22:43:51.779 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:50.928+0000 7fd18effd640 1 -- 172.21.15.152:0/1432840990 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd194039cf0 con 0x7fd1981080e0 2024-09-11T22:43:51.779 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:51.123+0000 7fd19d55b640 1 -- 172.21.15.152:0/1432840990 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fd184038fc0 msgr2=0x7fd18403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/1868213162 2024-09-11T22:43:51.779 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:51.123+0000 7fd19d55b640 1 --2- 172.21.15.152:0/1432840990 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fd184038fc0 0x7fd18403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:43:51.779 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:51.162+0000 7fd19ffc0640 1 -- 172.21.15.152:0/1432840990 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd198106c90 con 0x7fd1981080e0 2024-09-11T22:43:51.779 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:51.163+0000 7fd18effd640 1 -- 172.21.15.152:0/1432840990 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fd19403acd0 con 0x7fd1981080e0 2024-09-11T22:43:51.779 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:51.165+0000 7fd19ffc0640 1 -- 172.21.15.152:0/1432840990 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fd184038fc0 msgr2=0x7fd18403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:43:51.779 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:51.165+0000 7fd19ffc0640 1 --2- 172.21.15.152:0/1432840990 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fd184038fc0 0x7fd18403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:51.779 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:51.165+0000 7fd19ffc0640 1 -- 172.21.15.152:0/1432840990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1981080e0 msgr2=0x7fd198107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:51.779 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:51.165+0000 7fd19ffc0640 1 --2- 172.21.15.152:0/1432840990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1981080e0 0x7fd198107960 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd194030420 tx=0x7fd194032670 comp rx=0 tx=0).stop 2024-09-11T22:43:51.780 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:51.165+0000 7fd19ffc0640 1 -- 172.21.15.152:0/1432840990 shutdown_connections 2024-09-11T22:43:51.780 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:51.165+0000 7fd19ffc0640 1 --2- 172.21.15.152:0/1432840990 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7fd184038fc0 0x7fd18403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:51.780 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:51.165+0000 7fd19ffc0640 1 --2- 172.21.15.152:0/1432840990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1981080e0 0x7fd198107960 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:51.780 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:51.165+0000 7fd19ffc0640 1 -- 172.21.15.152:0/1432840990 >> 172.21.15.152:0/1432840990 conn(0x7fd198101ca0 msgr2=0x7fd198102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:51.780 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:51.165+0000 7fd19ffc0640 1 -- 172.21.15.152:0/1432840990 shutdown_connections 2024-09-11T22:43:51.780 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:51.165+0000 7fd19ffc0640 1 -- 172.21.15.152:0/1432840990 wait complete. 2024-09-11T22:43:51.780 INFO:teuthology.orchestra.run.smithi152.stdout:Waiting for the mgr to restart... 2024-09-11T22:43:51.780 INFO:teuthology.orchestra.run.smithi152.stdout:Waiting for mgr epoch 9... 2024-09-11T22:43:52.032 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:51 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1432840990' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T22:43:52.372 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:52 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:52.029+0000 7f579894d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:43:52.372 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:52 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:52.100+0000 7f579894d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:43:53.073 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:52 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:52.815+0000 7f579894d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:43:53.073 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:52 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:52.931+0000 7f579894d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:43:53.349 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:53.071+0000 7f579894d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:43:53.350 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:53.133+0000 7f579894d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:43:53.350 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:53.280+0000 7f579894d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:43:53.350 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:53.348+0000 7f579894d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:43:53.622 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:53.408+0000 7f579894d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:43:53.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:53.566+0000 7f579894d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:43:53.897 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:53.631+0000 7f579894d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:43:53.897 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:53.698+0000 7f579894d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:43:53.897 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:53.779+0000 7f579894d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:43:53.897 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:53.837+0000 7f579894d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:43:53.897 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:53.895+0000 7f579894d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:43:55.122 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:54.676+0000 7f579894d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:43:55.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:54 smithi152 ceph-mon[31181]: Active manager daemon x restarted 2024-09-11T22:43:55.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:54 smithi152 ceph-mon[31181]: Activating manager daemon x 2024-09-11T22:43:55.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:54 smithi152 ceph-mon[31181]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T22:43:55.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:54 smithi152 ceph-mon[31181]: mgrmap e10: x(active, starting, since 0.00332355s) 2024-09-11T22:43:55.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:43:55.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T22:43:55.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T22:43:55.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T22:43:55.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T22:43:55.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:54 smithi152 ceph-mon[31181]: Manager daemon x is now available 2024-09-11T22:43:55.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:43:55.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T22:43:55.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T22:43:56.236 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:55.931+0000 7f5703c61640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:22:43:55] ENGINE Error in HTTPServer.serve 2024-09-11T22:43:56.237 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: Traceback (most recent call last): 2024-09-11T22:43:56.237 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:43:56.237 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: self._connections.run(self.expiration_interval) 2024-09-11T22:43:56.237 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:43:56.237 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: self._run(expiration_interval) 2024-09-11T22:43:56.237 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:43:56.237 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:43:56.237 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:43:56.237 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:43:56.237 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:43:56.237 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: s = self.context.wrap_socket( 2024-09-11T22:43:56.238 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:43:56.238 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: return self.sslsocket_class._create( 2024-09-11T22:43:56.238 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:43:56.238 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: self.do_handshake() 2024-09-11T22:43:56.238 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:43:56.238 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: self._sslobj.do_handshake() 2024-09-11T22:43:56.238 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:43:56.238 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:43:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:43:56.521 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:43:56.528 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T22:43:56.528 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T22:43:56.528 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:43:56.528 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.222+0000 7f7308c41640 1 Processor -- start 2024-09-11T22:43:56.529 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.222+0000 7f7308c41640 1 -- start start 2024-09-11T22:43:56.529 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.222+0000 7f7308c41640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7304106500 0x7f7304106900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:56.529 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.222+0000 7f7308c41640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7304106ed0 con 0x7f7304106500 2024-09-11T22:43:56.529 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.223+0000 7f730259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7304106500 0x7f7304106900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:56.529 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.223+0000 7f730259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7304106500 0x7f7304106900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44268/0 (socket says 172.21.15.152:44268) 2024-09-11T22:43:56.529 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.223+0000 7f730259c640 1 -- 172.21.15.152:0/1543454135 learned_addr learned my addr 172.21.15.152:0/1543454135 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:56.529 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.223+0000 7f730259c640 1 -- 172.21.15.152:0/1543454135 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7304107700 con 0x7f7304106500 2024-09-11T22:43:56.529 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.224+0000 7f730259c640 1 --2- 172.21.15.152:0/1543454135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7304106500 0x7f7304106900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009820 tx=0x7f72f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7c576ae6b0dc14 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:56.529 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.224+0000 7f730159a640 1 -- 172.21.15.152:0/1543454135 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72f4037070 con 0x7f7304106500 2024-09-11T22:43:56.529 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.224+0000 7f730159a640 1 -- 172.21.15.152:0/1543454135 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f4032920 con 0x7f7304106500 2024-09-11T22:43:56.529 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.224+0000 7f730159a640 1 -- 172.21.15.152:0/1543454135 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72f4030810 con 0x7f7304106500 2024-09-11T22:43:56.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.224+0000 7f7308c41640 1 -- 172.21.15.152:0/1543454135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7304106500 msgr2=0x7f7304106900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:56.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.224+0000 7f7308c41640 1 --2- 172.21.15.152:0/1543454135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7304106500 0x7f7304106900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009820 tx=0x7f72f402a270 comp rx=0 tx=0).stop 2024-09-11T22:43:56.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.225+0000 7f7308c41640 1 -- 172.21.15.152:0/1543454135 shutdown_connections 2024-09-11T22:43:56.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.225+0000 7f7308c41640 1 --2- 172.21.15.152:0/1543454135 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7304106500 0x7f7304106900 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:56.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.225+0000 7f7308c41640 1 -- 172.21.15.152:0/1543454135 >> 172.21.15.152:0/1543454135 conn(0x7f7304101d10 msgr2=0x7f7304104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:56.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.225+0000 7f7308c41640 1 -- 172.21.15.152:0/1543454135 shutdown_connections 2024-09-11T22:43:56.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.225+0000 7f7308c41640 1 -- 172.21.15.152:0/1543454135 wait complete. 2024-09-11T22:43:56.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.225+0000 7f7308c41640 1 Processor -- start 2024-09-11T22:43:56.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.225+0000 7f7308c41640 1 -- start start 2024-09-11T22:43:56.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.225+0000 7f7308c41640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7304106500 0x7f730407ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:56.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.225+0000 7f7308c41640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f730410aab0 con 0x7f7304106500 2024-09-11T22:43:56.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.226+0000 7f730259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7304106500 0x7f730407ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:56.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.226+0000 7f730259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7304106500 0x7f730407ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44280/0 (socket says 172.21.15.152:44280) 2024-09-11T22:43:56.530 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.226+0000 7f730259c640 1 -- 172.21.15.152:0/2962960195 learned_addr learned my addr 172.21.15.152:0/2962960195 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:56.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.226+0000 7f730259c640 1 -- 172.21.15.152:0/2962960195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730419c160 con 0x7f7304106500 2024-09-11T22:43:56.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.226+0000 7f730259c640 1 --2- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7304106500 0x7f730407ba20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f72f4030560 tx=0x7f72f4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:56.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.226+0000 7f72f37fe640 1 -- 172.21.15.152:0/2962960195 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72f4037040 con 0x7f7304106500 2024-09-11T22:43:56.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.226+0000 7f72f37fe640 1 -- 172.21.15.152:0/2962960195 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f4040070 con 0x7f7304106500 2024-09-11T22:43:56.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.226+0000 7f7308c41640 1 -- 172.21.15.152:0/2962960195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73041a0770 con 0x7f7304106500 2024-09-11T22:43:56.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.226+0000 7f72f37fe640 1 -- 172.21.15.152:0/2962960195 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72f403a740 con 0x7f7304106500 2024-09-11T22:43:56.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.226+0000 7f7308c41640 1 -- 172.21.15.152:0/2962960195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73041a0b10 con 0x7f7304106500 2024-09-11T22:43:56.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.227+0000 7f72f37fe640 1 -- 172.21.15.152:0/2962960195 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f72f403a9b0 con 0x7f7304106500 2024-09-11T22:43:56.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.227+0000 7f72f37fe640 1 --2- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f72ec038fc0 0x7f72ec03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:56.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.227+0000 7f7308c41640 1 -- 172.21.15.152:0/2962960195 --> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7304106980 con 0x7f72ec038fc0 2024-09-11T22:43:56.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.227+0000 7f72f37fe640 1 -- 172.21.15.152:0/2962960195 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f72f406d670 con 0x7f7304106500 2024-09-11T22:43:56.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.227+0000 7f7301d9b640 1 -- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f72ec038fc0 msgr2=0x7f72ec03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/1868213162 2024-09-11T22:43:56.531 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.227+0000 7f7301d9b640 1 --2- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f72ec038fc0 0x7f72ec03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:43:56.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.428+0000 7f7301d9b640 1 -- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f72ec038fc0 msgr2=0x7f72ec03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/1868213162 2024-09-11T22:43:56.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.428+0000 7f7301d9b640 1 --2- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f72ec038fc0 0x7f72ec03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:43:56.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.829+0000 7f7301d9b640 1 -- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f72ec038fc0 msgr2=0x7f72ec03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/1868213162 2024-09-11T22:43:56.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:52.829+0000 7f7301d9b640 1 --2- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f72ec038fc0 0x7f72ec03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T22:43:56.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:53.630+0000 7f7301d9b640 1 -- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f72ec038fc0 msgr2=0x7f72ec03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/1868213162 2024-09-11T22:43:56.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:53.630+0000 7f7301d9b640 1 --2- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f72ec038fc0 0x7f72ec03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T22:43:56.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:54.679+0000 7f72f37fe640 1 -- 172.21.15.152:0/2962960195 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f72f406c120 con 0x7f7304106500 2024-09-11T22:43:56.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:54.680+0000 7f72f37fe640 1 -- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f72ec038fc0 msgr2=0x7f72ec03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:43:56.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:54.680+0000 7f72f37fe640 1 --2- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f72ec038fc0 0x7f72ec03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:56.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.681+0000 7f72f37fe640 1 -- 172.21.15.152:0/2962960195 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f72f403aca0 con 0x7f7304106500 2024-09-11T22:43:56.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.681+0000 7f72f37fe640 1 --2- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f72ec03c890 0x7f72ec03ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:56.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.681+0000 7f72f37fe640 1 -- 172.21.15.152:0/2962960195 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7304106980 con 0x7f72ec03c890 2024-09-11T22:43:56.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.681+0000 7f7301d9b640 1 --2- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f72ec03c890 0x7f72ec03ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:56.532 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.682+0000 7f7301d9b640 1 --2- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f72ec03c890 0x7f72ec03ec80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f72f8003980 tx=0x7f72f8006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:56.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.683+0000 7f72f37fe640 1 -- 172.21.15.152:0/2962960195 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f7304106980 con 0x7f72ec03c890 2024-09-11T22:43:56.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.690+0000 7f7308c41640 1 -- 172.21.15.152:0/2962960195 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f730410ac30 con 0x7f72ec03c890 2024-09-11T22:43:56.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.690+0000 7f72f37fe640 1 -- 172.21.15.152:0/2962960195 <== mgr.14150 v2:172.21.15.152:6800/3438861948 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f730410ac30 con 0x7f72ec03c890 2024-09-11T22:43:56.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.690+0000 7f7308c41640 1 -- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f72ec03c890 msgr2=0x7f72ec03ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:56.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.690+0000 7f7308c41640 1 --2- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f72ec03c890 0x7f72ec03ec80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f72f8003980 tx=0x7f72f8006e10 comp rx=0 tx=0).stop 2024-09-11T22:43:56.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.690+0000 7f7308c41640 1 -- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7304106500 msgr2=0x7f730407ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:56.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.690+0000 7f7308c41640 1 --2- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7304106500 0x7f730407ba20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f72f4030560 tx=0x7f72f4032670 comp rx=0 tx=0).stop 2024-09-11T22:43:56.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.690+0000 7f7308c41640 1 -- 172.21.15.152:0/2962960195 shutdown_connections 2024-09-11T22:43:56.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.690+0000 7f7308c41640 1 --2- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f72ec03c890 0x7f72ec03ec80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:56.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.690+0000 7f7308c41640 1 --2- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:6800/1868213162,v1:172.21.15.152:6801/1868213162] conn(0x7f72ec038fc0 0x7f72ec03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:56.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.690+0000 7f7308c41640 1 --2- 172.21.15.152:0/2962960195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7304106500 0x7f730407ba20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:56.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.690+0000 7f7308c41640 1 -- 172.21.15.152:0/2962960195 >> 172.21.15.152:0/2962960195 conn(0x7f7304101d10 msgr2=0x7f7304102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:56.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.691+0000 7f7308c41640 1 -- 172.21.15.152:0/2962960195 shutdown_connections 2024-09-11T22:43:56.533 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:55.691+0000 7f7308c41640 1 -- 172.21.15.152:0/2962960195 wait complete. 2024-09-11T22:43:56.534 INFO:teuthology.orchestra.run.smithi152.stdout:mgr epoch 9 is available 2024-09-11T22:43:56.534 INFO:teuthology.orchestra.run.smithi152.stdout:Generating a dashboard self-signed certificate... 2024-09-11T22:43:56.786 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:56 smithi152 ceph-mon[31181]: mgrmap e11: x(active, since 1.00585s) 2024-09-11T22:43:56.786 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:56 smithi152 ceph-mon[31181]: [11/Sep/2024:22:43:55] ENGINE Bus STARTING 2024-09-11T22:43:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T22:43:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.016+0000 7eff9283d640 1 Processor -- start 2024-09-11T22:43:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.016+0000 7eff9283d640 1 -- start start 2024-09-11T22:43:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.016+0000 7eff9283d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff8c108100 0x7eff8c10a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.016+0000 7eff9283d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c07bdb0 con 0x7eff8c108100 2024-09-11T22:43:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.017+0000 7eff8bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff8c108100 0x7eff8c10a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.017+0000 7eff8bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff8c108100 0x7eff8c10a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44356/0 (socket says 172.21.15.152:44356) 2024-09-11T22:43:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.017+0000 7eff8bfff640 1 -- 172.21.15.152:0/2229019760 learned_addr learned my addr 172.21.15.152:0/2229019760 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.017+0000 7eff8bfff640 1 -- 172.21.15.152:0/2229019760 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c10aa30 con 0x7eff8c108100 2024-09-11T22:43:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.018+0000 7eff8bfff640 1 --2- 172.21.15.152:0/2229019760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff8c108100 0x7eff8c10a4f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009820 tx=0x7eff7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2386f0125210169 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.018+0000 7eff8affd640 1 -- 172.21.15.152:0/2229019760 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff7c037070 con 0x7eff8c108100 2024-09-11T22:43:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.018+0000 7eff8affd640 1 -- 172.21.15.152:0/2229019760 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff7c032920 con 0x7eff8c108100 2024-09-11T22:43:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.018+0000 7eff8affd640 1 -- 172.21.15.152:0/2229019760 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff7c030810 con 0x7eff8c108100 2024-09-11T22:43:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.018+0000 7eff9283d640 1 -- 172.21.15.152:0/2229019760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff8c108100 msgr2=0x7eff8c10a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.018+0000 7eff9283d640 1 --2- 172.21.15.152:0/2229019760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff8c108100 0x7eff8c10a4f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009820 tx=0x7eff7c02a270 comp rx=0 tx=0).stop 2024-09-11T22:43:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.018+0000 7eff9283d640 1 -- 172.21.15.152:0/2229019760 shutdown_connections 2024-09-11T22:43:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.018+0000 7eff9283d640 1 --2- 172.21.15.152:0/2229019760 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff8c108100 0x7eff8c10a4f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.018+0000 7eff9283d640 1 -- 172.21.15.152:0/2229019760 >> 172.21.15.152:0/2229019760 conn(0x7eff8c101ca0 msgr2=0x7eff8c1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.018+0000 7eff9283d640 1 -- 172.21.15.152:0/2229019760 shutdown_connections 2024-09-11T22:43:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.018+0000 7eff9283d640 1 -- 172.21.15.152:0/2229019760 wait complete. 2024-09-11T22:43:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.019+0000 7eff9283d640 1 Processor -- start 2024-09-11T22:43:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.019+0000 7eff9283d640 1 -- start start 2024-09-11T22:43:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.019+0000 7eff9283d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff8c108100 0x7eff8c107980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.019+0000 7eff9283d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c10b9f0 con 0x7eff8c108100 2024-09-11T22:43:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.019+0000 7eff8bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff8c108100 0x7eff8c107980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.019+0000 7eff8bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff8c108100 0x7eff8c107980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44358/0 (socket says 172.21.15.152:44358) 2024-09-11T22:43:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.019+0000 7eff8bfff640 1 -- 172.21.15.152:0/763261071 learned_addr learned my addr 172.21.15.152:0/763261071 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.020+0000 7eff8bfff640 1 -- 172.21.15.152:0/763261071 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c107ec0 con 0x7eff8c108100 2024-09-11T22:43:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.020+0000 7eff8bfff640 1 --2- 172.21.15.152:0/763261071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff8c108100 0x7eff8c107980 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7eff7c030420 tx=0x7eff7c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.020+0000 7eff897fa640 1 -- 172.21.15.152:0/763261071 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff7c037040 con 0x7eff8c108100 2024-09-11T22:43:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.020+0000 7eff897fa640 1 -- 172.21.15.152:0/763261071 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff7c040070 con 0x7eff8c108100 2024-09-11T22:43:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.020+0000 7eff9283d640 1 -- 172.21.15.152:0/763261071 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8c1060b0 con 0x7eff8c108100 2024-09-11T22:43:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.020+0000 7eff897fa640 1 -- 172.21.15.152:0/763261071 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff7c03a6e0 con 0x7eff8c108100 2024-09-11T22:43:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.020+0000 7eff9283d640 1 -- 172.21.15.152:0/763261071 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8c1065f0 con 0x7eff8c108100 2024-09-11T22:43:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.021+0000 7eff897fa640 1 -- 172.21.15.152:0/763261071 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7eff7c03bb20 con 0x7eff8c108100 2024-09-11T22:43:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.021+0000 7eff9283d640 1 -- 172.21.15.152:0/763261071 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff8c108560 con 0x7eff8c108100 2024-09-11T22:43:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.021+0000 7eff897fa640 1 --2- 172.21.15.152:0/763261071 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7eff78039020 0x7eff7803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.021+0000 7eff897fa640 1 -- 172.21.15.152:0/763261071 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7eff7c06c770 con 0x7eff8c108100 2024-09-11T22:43:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.022+0000 7eff8b7fe640 1 --2- 172.21.15.152:0/763261071 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7eff78039020 0x7eff7803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.022+0000 7eff8b7fe640 1 --2- 172.21.15.152:0/763261071 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7eff78039020 0x7eff7803b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7eff740098c0 tx=0x7eff74006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.026+0000 7eff897fa640 1 -- 172.21.15.152:0/763261071 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7eff7c04a200 con 0x7eff8c108100 2024-09-11T22:43:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.208+0000 7eff9283d640 1 -- 172.21.15.152:0/763261071 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7eff8c106d40 con 0x7eff78039020 2024-09-11T22:43:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.274+0000 7eff897fa640 1 -- 172.21.15.152:0/763261071 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7eff8c106d40 con 0x7eff78039020 2024-09-11T22:43:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.277+0000 7eff9283d640 1 -- 172.21.15.152:0/763261071 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7eff78039020 msgr2=0x7eff7803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.277+0000 7eff9283d640 1 --2- 172.21.15.152:0/763261071 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7eff78039020 0x7eff7803b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7eff740098c0 tx=0x7eff74006db0 comp rx=0 tx=0).stop 2024-09-11T22:43:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.277+0000 7eff9283d640 1 -- 172.21.15.152:0/763261071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff8c108100 msgr2=0x7eff8c107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.277+0000 7eff9283d640 1 --2- 172.21.15.152:0/763261071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff8c108100 0x7eff8c107980 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7eff7c030420 tx=0x7eff7c032670 comp rx=0 tx=0).stop 2024-09-11T22:43:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.277+0000 7eff9283d640 1 -- 172.21.15.152:0/763261071 shutdown_connections 2024-09-11T22:43:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.277+0000 7eff9283d640 1 --2- 172.21.15.152:0/763261071 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7eff78039020 0x7eff7803b4e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.277+0000 7eff9283d640 1 --2- 172.21.15.152:0/763261071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff8c108100 0x7eff8c107980 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.277+0000 7eff9283d640 1 -- 172.21.15.152:0/763261071 >> 172.21.15.152:0/763261071 conn(0x7eff8c101ca0 msgr2=0x7eff8c102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.278+0000 7eff9283d640 1 -- 172.21.15.152:0/763261071 shutdown_connections 2024-09-11T22:43:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:57.278+0000 7eff9283d640 1 -- 172.21.15.152:0/763261071 wait complete. 2024-09-11T22:43:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:Creating initial admin user... 2024-09-11T22:43:58.010 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:57 smithi152 ceph-mon[31181]: [11/Sep/2024:22:43:55] ENGINE Serving on https://172.21.15.152:7150 2024-09-11T22:43:58.010 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:57 smithi152 ceph-mon[31181]: [11/Sep/2024:22:43:55] ENGINE Error in HTTPServer.serve 2024-09-11T22:43:58.011 INFO:journalctl@ceph.mon.a.smithi152.stdout: Traceback (most recent call last): 2024-09-11T22:43:58.011 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:43:58.011 INFO:journalctl@ceph.mon.a.smithi152.stdout: self._connections.run(self.expiration_interval) 2024-09-11T22:43:58.011 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:43:58.011 INFO:journalctl@ceph.mon.a.smithi152.stdout: self._run(expiration_interval) 2024-09-11T22:43:58.011 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:43:58.011 INFO:journalctl@ceph.mon.a.smithi152.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:43:58.011 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:43:58.011 INFO:journalctl@ceph.mon.a.smithi152.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:43:58.011 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:43:58.011 INFO:journalctl@ceph.mon.a.smithi152.stdout: s = self.context.wrap_socket( 2024-09-11T22:43:58.011 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:43:58.011 INFO:journalctl@ceph.mon.a.smithi152.stdout: return self.sslsocket_class._create( 2024-09-11T22:43:58.012 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:43:58.012 INFO:journalctl@ceph.mon.a.smithi152.stdout: self.do_handshake() 2024-09-11T22:43:58.012 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:43:58.012 INFO:journalctl@ceph.mon.a.smithi152.stdout: self._sslobj.do_handshake() 2024-09-11T22:43:58.012 INFO:journalctl@ceph.mon.a.smithi152.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:43:58.012 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:57 smithi152 ceph-mon[31181]: [11/Sep/2024:22:43:56] ENGINE Serving on http://172.21.15.152:8765 2024-09-11T22:43:58.012 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:57 smithi152 ceph-mon[31181]: [11/Sep/2024:22:43:56] ENGINE Bus STARTED 2024-09-11T22:43:58.012 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:57 smithi152 ceph-mon[31181]: mgrmap e12: x(active, since 2s) 2024-09-11T22:43:58.012 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:57 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:43:58.012 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:57 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:43:58.012 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:57 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:43:58.969 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:58 smithi152 ceph-mon[31181]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:43:58.970 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:58 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:43:59.568 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$mq7Bvz2NQvIZhUMDz21xS.m.sQ4hGceegwORe4haqBBC9LOcr8.Vi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726094638, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T22:43:59.568 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.293+0000 7f7f4f0cc640 1 Processor -- start 2024-09-11T22:43:59.568 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.293+0000 7f7f4f0cc640 1 -- start start 2024-09-11T22:43:59.568 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.293+0000 7f7f4f0cc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f48107620 0x7f7f48107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:59.568 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.293+0000 7f7f4f0cc640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7f48107ff0 con 0x7f7f48107620 2024-09-11T22:43:59.568 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.293+0000 7f7f4ce68640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f48107620 0x7f7f48107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:59.568 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.293+0000 7f7f4ce68640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f48107620 0x7f7f48107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44370/0 (socket says 172.21.15.152:44370) 2024-09-11T22:43:59.568 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.293+0000 7f7f4ce68640 1 -- 172.21.15.152:0/1510097179 learned_addr learned my addr 172.21.15.152:0/1510097179 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:59.569 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.293+0000 7f7f4ce68640 1 -- 172.21.15.152:0/1510097179 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f48108820 con 0x7f7f48107620 2024-09-11T22:43:59.569 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.294+0000 7f7f4ce68640 1 --2- 172.21.15.152:0/1510097179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f48107620 0x7f7f48107a20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009820 tx=0x7f7f3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9dd37648d5853b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:59.569 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.295+0000 7f7f477fe640 1 -- 172.21.15.152:0/1510097179 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c037070 con 0x7f7f48107620 2024-09-11T22:43:59.569 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.295+0000 7f7f477fe640 1 -- 172.21.15.152:0/1510097179 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f3c032920 con 0x7f7f48107620 2024-09-11T22:43:59.569 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.295+0000 7f7f477fe640 1 -- 172.21.15.152:0/1510097179 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c030810 con 0x7f7f48107620 2024-09-11T22:43:59.569 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.296+0000 7f7f4f0cc640 1 -- 172.21.15.152:0/1510097179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f48107620 msgr2=0x7f7f48107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:59.569 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.296+0000 7f7f4f0cc640 1 --2- 172.21.15.152:0/1510097179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f48107620 0x7f7f48107a20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009820 tx=0x7f7f3c02a270 comp rx=0 tx=0).stop 2024-09-11T22:43:59.569 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.296+0000 7f7f4f0cc640 1 -- 172.21.15.152:0/1510097179 shutdown_connections 2024-09-11T22:43:59.569 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.296+0000 7f7f4f0cc640 1 --2- 172.21.15.152:0/1510097179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f48107620 0x7f7f48107a20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:59.569 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.296+0000 7f7f4f0cc640 1 -- 172.21.15.152:0/1510097179 >> 172.21.15.152:0/1510097179 conn(0x7f7f48102db0 msgr2=0x7f7f481051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:59.569 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.296+0000 7f7f4f0cc640 1 -- 172.21.15.152:0/1510097179 shutdown_connections 2024-09-11T22:43:59.569 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.296+0000 7f7f4f0cc640 1 -- 172.21.15.152:0/1510097179 wait complete. 2024-09-11T22:43:59.570 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.296+0000 7f7f4f0cc640 1 Processor -- start 2024-09-11T22:43:59.570 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.296+0000 7f7f4f0cc640 1 -- start start 2024-09-11T22:43:59.570 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.296+0000 7f7f4f0cc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f48107620 0x7f7f4819b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:59.570 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.296+0000 7f7f4f0cc640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4810bbd0 con 0x7f7f48107620 2024-09-11T22:43:59.570 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.297+0000 7f7f4ce68640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f48107620 0x7f7f4819b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:59.570 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.297+0000 7f7f4ce68640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f48107620 0x7f7f4819b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:44374/0 (socket says 172.21.15.152:44374) 2024-09-11T22:43:59.570 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.297+0000 7f7f4ce68640 1 -- 172.21.15.152:0/2766455952 learned_addr learned my addr 172.21.15.152:0/2766455952 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:43:59.570 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.297+0000 7f7f4ce68640 1 -- 172.21.15.152:0/2766455952 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4819bbd0 con 0x7f7f48107620 2024-09-11T22:43:59.570 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.297+0000 7f7f4ce68640 1 --2- 172.21.15.152:0/2766455952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f48107620 0x7f7f4819b690 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0097f0 tx=0x7f7f3c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:59.570 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.297+0000 7f7f45ffb640 1 -- 172.21.15.152:0/2766455952 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c037040 con 0x7f7f48107620 2024-09-11T22:43:59.570 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.297+0000 7f7f45ffb640 1 -- 172.21.15.152:0/2766455952 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f3c030c40 con 0x7f7f48107620 2024-09-11T22:43:59.570 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.297+0000 7f7f45ffb640 1 -- 172.21.15.152:0/2766455952 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c03a5b0 con 0x7f7f48107620 2024-09-11T22:43:59.570 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.297+0000 7f7f4f0cc640 1 -- 172.21.15.152:0/2766455952 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4819be60 con 0x7f7f48107620 2024-09-11T22:43:59.570 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.297+0000 7f7f4f0cc640 1 -- 172.21.15.152:0/2766455952 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4819eb50 con 0x7f7f48107620 2024-09-11T22:43:59.571 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.298+0000 7f7f45ffb640 1 -- 172.21.15.152:0/2766455952 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f7f3c03a7f0 con 0x7f7f48107620 2024-09-11T22:43:59.571 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.298+0000 7f7f45ffb640 1 --2- 172.21.15.152:0/2766455952 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f7f38038fc0 0x7f7f3803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:43:59.571 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.298+0000 7f7f45ffb640 1 -- 172.21.15.152:0/2766455952 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7f3c06c4f0 con 0x7f7f48107620 2024-09-11T22:43:59.571 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.298+0000 7f7f4f0cc640 1 -- 172.21.15.152:0/2766455952 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f4810bd50 con 0x7f7f48107620 2024-09-11T22:43:59.571 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.299+0000 7f7f47fff640 1 --2- 172.21.15.152:0/2766455952 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f7f38038fc0 0x7f7f3803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:43:59.571 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.299+0000 7f7f47fff640 1 --2- 172.21.15.152:0/2766455952 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f7f38038fc0 0x7f7f3803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7f340098c0 tx=0x7f7f34006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:43:59.571 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.303+0000 7f7f45ffb640 1 -- 172.21.15.152:0/2766455952 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f3c0371e0 con 0x7f7f48107620 2024-09-11T22:43:59.571 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.500+0000 7f7f4f0cc640 1 -- 172.21.15.152:0/2766455952 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f7f4819ed20 con 0x7f7f38038fc0 2024-09-11T22:43:59.571 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.732+0000 7f7f45ffb640 1 -- 172.21.15.152:0/2766455952 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f7f4819ed20 con 0x7f7f38038fc0 2024-09-11T22:43:59.571 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.734+0000 7f7f4f0cc640 1 -- 172.21.15.152:0/2766455952 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f7f38038fc0 msgr2=0x7f7f3803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:59.571 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.734+0000 7f7f4f0cc640 1 --2- 172.21.15.152:0/2766455952 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f7f38038fc0 0x7f7f3803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7f340098c0 tx=0x7f7f34006db0 comp rx=0 tx=0).stop 2024-09-11T22:43:59.571 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.734+0000 7f7f4f0cc640 1 -- 172.21.15.152:0/2766455952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f48107620 msgr2=0x7f7f4819b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:43:59.571 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.734+0000 7f7f4f0cc640 1 --2- 172.21.15.152:0/2766455952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f48107620 0x7f7f4819b690 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0097f0 tx=0x7f7f3c0327b0 comp rx=0 tx=0).stop 2024-09-11T22:43:59.571 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.734+0000 7f7f4f0cc640 1 -- 172.21.15.152:0/2766455952 shutdown_connections 2024-09-11T22:43:59.572 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.734+0000 7f7f4f0cc640 1 --2- 172.21.15.152:0/2766455952 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f7f38038fc0 0x7f7f3803b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:59.572 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.734+0000 7f7f4f0cc640 1 --2- 172.21.15.152:0/2766455952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f48107620 0x7f7f4819b690 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:43:59.572 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.734+0000 7f7f4f0cc640 1 -- 172.21.15.152:0/2766455952 >> 172.21.15.152:0/2766455952 conn(0x7f7f48102db0 msgr2=0x7f7f48103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:43:59.572 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.734+0000 7f7f4f0cc640 1 -- 172.21.15.152:0/2766455952 shutdown_connections 2024-09-11T22:43:59.572 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:58.734+0000 7f7f4f0cc640 1 -- 172.21.15.152:0/2766455952 wait complete. 2024-09-11T22:43:59.572 INFO:teuthology.orchestra.run.smithi152.stdout:Fetching dashboard port number... 2024-09-11T22:43:59.871 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:59 smithi152 ceph-mon[31181]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:43:59.871 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:43:59 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:00.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:00 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2238573980' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T22:44:01.006 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T22:44:01.011 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.950+0000 7fd1de938640 1 Processor -- start 2024-09-11T22:44:01.012 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.950+0000 7fd1de938640 1 -- start start 2024-09-11T22:44:01.012 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.950+0000 7fd1de938640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d8108080 0x7fd1d810a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:01.012 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.950+0000 7fd1de938640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d807be30 con 0x7fd1d8108080 2024-09-11T22:44:01.012 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.950+0000 7fd1d7fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d8108080 0x7fd1d810a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:01.012 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.950+0000 7fd1d7fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d8108080 0x7fd1d810a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59992/0 (socket says 172.21.15.152:59992) 2024-09-11T22:44:01.012 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.950+0000 7fd1d7fff640 1 -- 172.21.15.152:0/605314855 learned_addr learned my addr 172.21.15.152:0/605314855 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:44:01.012 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.951+0000 7fd1d7fff640 1 -- 172.21.15.152:0/605314855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d810a9b0 con 0x7fd1d8108080 2024-09-11T22:44:01.012 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.951+0000 7fd1d7fff640 1 --2- 172.21.15.152:0/605314855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d8108080 0x7fd1d810a470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009820 tx=0x7fd1c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbd9bcc3589c248 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:01.012 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.951+0000 7fd1d6ffd640 1 -- 172.21.15.152:0/605314855 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1c8037070 con 0x7fd1d8108080 2024-09-11T22:44:01.012 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.952+0000 7fd1d6ffd640 1 -- 172.21.15.152:0/605314855 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c8032920 con 0x7fd1d8108080 2024-09-11T22:44:01.012 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.952+0000 7fd1d6ffd640 1 -- 172.21.15.152:0/605314855 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1c8030810 con 0x7fd1d8108080 2024-09-11T22:44:01.012 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.952+0000 7fd1de938640 1 -- 172.21.15.152:0/605314855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d8108080 msgr2=0x7fd1d810a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:01.013 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.952+0000 7fd1de938640 1 --2- 172.21.15.152:0/605314855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d8108080 0x7fd1d810a470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009820 tx=0x7fd1c802a270 comp rx=0 tx=0).stop 2024-09-11T22:44:01.013 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.952+0000 7fd1de938640 1 -- 172.21.15.152:0/605314855 shutdown_connections 2024-09-11T22:44:01.013 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.952+0000 7fd1de938640 1 --2- 172.21.15.152:0/605314855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d8108080 0x7fd1d810a470 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:01.013 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.952+0000 7fd1de938640 1 -- 172.21.15.152:0/605314855 >> 172.21.15.152:0/605314855 conn(0x7fd1d8101ca0 msgr2=0x7fd1d81040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:01.013 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.952+0000 7fd1de938640 1 -- 172.21.15.152:0/605314855 shutdown_connections 2024-09-11T22:44:01.013 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.952+0000 7fd1de938640 1 -- 172.21.15.152:0/605314855 wait complete. 2024-09-11T22:44:01.013 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.953+0000 7fd1de938640 1 Processor -- start 2024-09-11T22:44:01.013 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.953+0000 7fd1de938640 1 -- start start 2024-09-11T22:44:01.013 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.953+0000 7fd1de938640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d8108080 0x7fd1d8107900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:01.013 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.953+0000 7fd1de938640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d810ba30 con 0x7fd1d8108080 2024-09-11T22:44:01.013 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.953+0000 7fd1d7fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d8108080 0x7fd1d8107900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:01.013 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.953+0000 7fd1d7fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d8108080 0x7fd1d8107900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60002/0 (socket says 172.21.15.152:60002) 2024-09-11T22:44:01.013 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.953+0000 7fd1d7fff640 1 -- 172.21.15.152:0/2238573980 learned_addr learned my addr 172.21.15.152:0/2238573980 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:44:01.014 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.954+0000 7fd1d7fff640 1 -- 172.21.15.152:0/2238573980 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d8107e40 con 0x7fd1d8108080 2024-09-11T22:44:01.014 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.954+0000 7fd1d7fff640 1 --2- 172.21.15.152:0/2238573980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d8108080 0x7fd1d8107900 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8030420 tx=0x7fd1c8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:01.014 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.954+0000 7fd1d57fa640 1 -- 172.21.15.152:0/2238573980 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1c8037040 con 0x7fd1d8108080 2024-09-11T22:44:01.014 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.954+0000 7fd1de938640 1 -- 172.21.15.152:0/2238573980 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d8106030 con 0x7fd1d8108080 2024-09-11T22:44:01.014 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.954+0000 7fd1d57fa640 1 -- 172.21.15.152:0/2238573980 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c8040070 con 0x7fd1d8108080 2024-09-11T22:44:01.014 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.954+0000 7fd1d57fa640 1 -- 172.21.15.152:0/2238573980 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1c803a6e0 con 0x7fd1d8108080 2024-09-11T22:44:01.014 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.954+0000 7fd1de938640 1 -- 172.21.15.152:0/2238573980 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d8106570 con 0x7fd1d8108080 2024-09-11T22:44:01.014 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.955+0000 7fd1d57fa640 1 -- 172.21.15.152:0/2238573980 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fd1c803a9e0 con 0x7fd1d8108080 2024-09-11T22:44:01.014 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.955+0000 7fd1d57fa640 1 --2- 172.21.15.152:0/2238573980 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd1c0038fc0 0x7fd1c003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:01.014 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.955+0000 7fd1d57fa640 1 -- 172.21.15.152:0/2238573980 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd1c806c7b0 con 0x7fd1d8108080 2024-09-11T22:44:01.014 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.955+0000 7fd1de938640 1 -- 172.21.15.152:0/2238573980 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d81084e0 con 0x7fd1d8108080 2024-09-11T22:44:01.014 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.955+0000 7fd1d77fe640 1 --2- 172.21.15.152:0/2238573980 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd1c0038fc0 0x7fd1c003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:01.014 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.956+0000 7fd1d77fe640 1 --2- 172.21.15.152:0/2238573980 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd1c0038fc0 0x7fd1c003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0098c0 tx=0x7fd1cc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:01.015 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:43:59.960+0000 7fd1d57fa640 1 -- 172.21.15.152:0/2238573980 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1c8031560 con 0x7fd1d8108080 2024-09-11T22:44:01.015 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:00.131+0000 7fd1de938640 1 -- 172.21.15.152:0/2238573980 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fd1d8106c90 con 0x7fd1d8108080 2024-09-11T22:44:01.015 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:00.131+0000 7fd1d57fa640 1 -- 172.21.15.152:0/2238573980 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fd1c802ed90 con 0x7fd1d8108080 2024-09-11T22:44:01.015 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:00.133+0000 7fd1de938640 1 -- 172.21.15.152:0/2238573980 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd1c0038fc0 msgr2=0x7fd1c003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:01.015 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:00.133+0000 7fd1de938640 1 --2- 172.21.15.152:0/2238573980 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd1c0038fc0 0x7fd1c003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0098c0 tx=0x7fd1cc006db0 comp rx=0 tx=0).stop 2024-09-11T22:44:01.015 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:00.133+0000 7fd1de938640 1 -- 172.21.15.152:0/2238573980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d8108080 msgr2=0x7fd1d8107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:01.015 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:00.133+0000 7fd1de938640 1 --2- 172.21.15.152:0/2238573980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d8108080 0x7fd1d8107900 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8030420 tx=0x7fd1c8032670 comp rx=0 tx=0).stop 2024-09-11T22:44:01.015 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:00.134+0000 7fd1de938640 1 -- 172.21.15.152:0/2238573980 shutdown_connections 2024-09-11T22:44:01.015 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:00.134+0000 7fd1de938640 1 --2- 172.21.15.152:0/2238573980 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd1c0038fc0 0x7fd1c003b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:01.015 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:00.134+0000 7fd1de938640 1 --2- 172.21.15.152:0/2238573980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d8108080 0x7fd1d8107900 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:01.015 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:00.134+0000 7fd1de938640 1 -- 172.21.15.152:0/2238573980 >> 172.21.15.152:0/2238573980 conn(0x7fd1d8101ca0 msgr2=0x7fd1d8102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:01.016 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:00.134+0000 7fd1de938640 1 -- 172.21.15.152:0/2238573980 shutdown_connections 2024-09-11T22:44:01.016 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:00.134+0000 7fd1de938640 1 -- 172.21.15.152:0/2238573980 wait complete. 2024-09-11T22:44:01.020 INFO:teuthology.orchestra.run.smithi152.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T22:44:01.020 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stdout disabled 2024-09-11T22:44:01.027 INFO:teuthology.orchestra.run.smithi152.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T22:44:01.027 INFO:teuthology.orchestra.run.smithi152.stdout:systemctl: stdout inactive 2024-09-11T22:44:01.027 INFO:teuthology.orchestra.run.smithi152.stdout:firewalld.service is not enabled 2024-09-11T22:44:01.027 INFO:teuthology.orchestra.run.smithi152.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T22:44:01.029 INFO:teuthology.orchestra.run.smithi152.stdout:Ceph Dashboard is now available at: 2024-09-11T22:44:01.029 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:44:01.029 INFO:teuthology.orchestra.run.smithi152.stdout: URL: https://smithi152.front.sepia.ceph.com:8443/ 2024-09-11T22:44:01.029 INFO:teuthology.orchestra.run.smithi152.stdout: User: admin 2024-09-11T22:44:01.029 INFO:teuthology.orchestra.run.smithi152.stdout: Password: 71lv6thyb1 2024-09-11T22:44:01.030 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:44:01.030 INFO:teuthology.orchestra.run.smithi152.stdout:Saving cluster configuration to /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config directory 2024-09-11T22:44:01.940 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:01 smithi152 ceph-mon[31181]: mgrmap e13: x(active, since 6s) 2024-09-11T22:44:02.855 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.928+0000 7fa43c87a640 1 Processor -- start 2024-09-11T22:44:02.855 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.928+0000 7fa43c87a640 1 -- start start 2024-09-11T22:44:02.855 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.928+0000 7fa43c87a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4381080e0 0x7fa43810a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:02.856 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.928+0000 7fa43c87a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa43807be20 con 0x7fa4381080e0 2024-09-11T22:44:02.856 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.928+0000 7fa43659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4381080e0 0x7fa43810a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:02.856 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.928+0000 7fa43659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4381080e0 0x7fa43810a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60008/0 (socket says 172.21.15.152:60008) 2024-09-11T22:44:02.856 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.928+0000 7fa43659c640 1 -- 172.21.15.152:0/2454252535 learned_addr learned my addr 172.21.15.152:0/2454252535 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:44:02.856 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.929+0000 7fa43659c640 1 -- 172.21.15.152:0/2454252535 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43810aa10 con 0x7fa4381080e0 2024-09-11T22:44:02.856 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.929+0000 7fa43659c640 1 --2- 172.21.15.152:0/2454252535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4381080e0 0x7fa43810a4d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa430009e80 tx=0x7fa43002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f879faecbeada085 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:02.856 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.929+0000 7fa43559a640 1 -- 172.21.15.152:0/2454252535 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa430036070 con 0x7fa4381080e0 2024-09-11T22:44:02.856 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.930+0000 7fa43559a640 1 -- 172.21.15.152:0/2454252535 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4300318e0 con 0x7fa4381080e0 2024-09-11T22:44:02.856 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.930+0000 7fa43559a640 1 -- 172.21.15.152:0/2454252535 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa430007a00 con 0x7fa4381080e0 2024-09-11T22:44:02.856 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.930+0000 7fa43c87a640 1 -- 172.21.15.152:0/2454252535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4381080e0 msgr2=0x7fa43810a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:02.856 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.930+0000 7fa43c87a640 1 --2- 172.21.15.152:0/2454252535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4381080e0 0x7fa43810a4d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa430009e80 tx=0x7fa43002a7a0 comp rx=0 tx=0).stop 2024-09-11T22:44:02.856 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.930+0000 7fa43c87a640 1 -- 172.21.15.152:0/2454252535 shutdown_connections 2024-09-11T22:44:02.857 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.930+0000 7fa43c87a640 1 --2- 172.21.15.152:0/2454252535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4381080e0 0x7fa43810a4d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:02.857 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.930+0000 7fa43c87a640 1 -- 172.21.15.152:0/2454252535 >> 172.21.15.152:0/2454252535 conn(0x7fa438101c80 msgr2=0x7fa4381040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:02.857 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.930+0000 7fa43c87a640 1 -- 172.21.15.152:0/2454252535 shutdown_connections 2024-09-11T22:44:02.857 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.930+0000 7fa43c87a640 1 -- 172.21.15.152:0/2454252535 wait complete. 2024-09-11T22:44:02.857 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.931+0000 7fa43c87a640 1 Processor -- start 2024-09-11T22:44:02.857 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.931+0000 7fa43c87a640 1 -- start start 2024-09-11T22:44:02.857 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.931+0000 7fa43c87a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4381080e0 0x7fa43819b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:02.857 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.931+0000 7fa43c87a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa43810ba90 con 0x7fa4381080e0 2024-09-11T22:44:02.857 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.931+0000 7fa43659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4381080e0 0x7fa43819b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:02.857 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.931+0000 7fa43659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4381080e0 0x7fa43819b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60022/0 (socket says 172.21.15.152:60022) 2024-09-11T22:44:02.857 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.931+0000 7fa43659c640 1 -- 172.21.15.152:0/1885871938 learned_addr learned my addr 172.21.15.152:0/1885871938 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:44:02.857 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.931+0000 7fa43659c640 1 -- 172.21.15.152:0/1885871938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43819bbb0 con 0x7fa4381080e0 2024-09-11T22:44:02.857 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.932+0000 7fa43659c640 1 --2- 172.21.15.152:0/1885871938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4381080e0 0x7fa43819b670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa430031cf0 tx=0x7fa430031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:02.857 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.932+0000 7fa4277fe640 1 -- 172.21.15.152:0/1885871938 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa430036050 con 0x7fa4381080e0 2024-09-11T22:44:02.858 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.932+0000 7fa4277fe640 1 -- 172.21.15.152:0/1885871938 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa430042070 con 0x7fa4381080e0 2024-09-11T22:44:02.858 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.932+0000 7fa43c87a640 1 -- 172.21.15.152:0/1885871938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43819be40 con 0x7fa4381080e0 2024-09-11T22:44:02.858 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.932+0000 7fa4277fe640 1 -- 172.21.15.152:0/1885871938 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa43003d9e0 con 0x7fa4381080e0 2024-09-11T22:44:02.858 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.932+0000 7fa43c87a640 1 -- 172.21.15.152:0/1885871938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43819eb30 con 0x7fa4381080e0 2024-09-11T22:44:02.858 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.933+0000 7fa4277fe640 1 -- 172.21.15.152:0/1885871938 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fa43003dc90 con 0x7fa4381080e0 2024-09-11T22:44:02.858 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.933+0000 7fa4277fe640 1 --2- 172.21.15.152:0/1885871938 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa4200390c0 0x7fa42003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:02.858 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.933+0000 7fa4277fe640 1 -- 172.21.15.152:0/1885871938 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa43006d5d0 con 0x7fa4381080e0 2024-09-11T22:44:02.858 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.933+0000 7fa43c87a640 1 -- 172.21.15.152:0/1885871938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa438108540 con 0x7fa4381080e0 2024-09-11T22:44:02.858 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.933+0000 7fa435d9b640 1 --2- 172.21.15.152:0/1885871938 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa4200390c0 0x7fa42003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:02.858 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.934+0000 7fa435d9b640 1 --2- 172.21.15.152:0/1885871938 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa4200390c0 0x7fa42003b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0098c0 tx=0x7fa42c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:02.858 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:01.938+0000 7fa4277fe640 1 -- 172.21.15.152:0/1885871938 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa43004c200 con 0x7fa4381080e0 2024-09-11T22:44:02.858 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:02.193+0000 7fa43c87a640 1 -- 172.21.15.152:0/1885871938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fa43819efc0 con 0x7fa4381080e0 2024-09-11T22:44:02.858 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:02.195+0000 7fa4277fe640 1 -- 172.21.15.152:0/1885871938 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) ==== 153+0+0 (secure 0 0 0) 0x7fa43003bd60 con 0x7fa4381080e0 2024-09-11T22:44:02.858 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T22:44:02.859 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:02.198+0000 7fa43c87a640 1 -- 172.21.15.152:0/1885871938 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa4200390c0 msgr2=0x7fa42003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:02.859 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:02.198+0000 7fa43c87a640 1 --2- 172.21.15.152:0/1885871938 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa4200390c0 0x7fa42003b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0098c0 tx=0x7fa42c006db0 comp rx=0 tx=0).stop 2024-09-11T22:44:02.859 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:02.198+0000 7fa43c87a640 1 -- 172.21.15.152:0/1885871938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4381080e0 msgr2=0x7fa43819b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:02.859 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:02.198+0000 7fa43c87a640 1 --2- 172.21.15.152:0/1885871938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4381080e0 0x7fa43819b670 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa430031cf0 tx=0x7fa430031d20 comp rx=0 tx=0).stop 2024-09-11T22:44:02.859 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:02.198+0000 7fa43c87a640 1 -- 172.21.15.152:0/1885871938 shutdown_connections 2024-09-11T22:44:02.859 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:02.198+0000 7fa43c87a640 1 --2- 172.21.15.152:0/1885871938 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa4200390c0 0x7fa42003b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:02.859 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:02.198+0000 7fa43c87a640 1 --2- 172.21.15.152:0/1885871938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa4381080e0 0x7fa43819b670 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:02.859 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:02.198+0000 7fa43c87a640 1 -- 172.21.15.152:0/1885871938 >> 172.21.15.152:0/1885871938 conn(0x7fa438101c80 msgr2=0x7fa4381028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:02.859 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:02.198+0000 7fa43c87a640 1 -- 172.21.15.152:0/1885871938 shutdown_connections 2024-09-11T22:44:02.859 INFO:teuthology.orchestra.run.smithi152.stdout:/usr/bin/ceph: stderr 2024-09-11T22:44:02.198+0000 7fa43c87a640 1 -- 172.21.15.152:0/1885871938 wait complete. 2024-09-11T22:44:02.859 INFO:teuthology.orchestra.run.smithi152.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T22:44:02.859 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:44:02.859 INFO:teuthology.orchestra.run.smithi152.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:44:02.859 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:44:02.860 INFO:teuthology.orchestra.run.smithi152.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T22:44:02.860 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:44:02.860 INFO:teuthology.orchestra.run.smithi152.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-11T22:44:02.860 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:44:02.860 INFO:teuthology.orchestra.run.smithi152.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T22:44:02.860 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:44:02.860 INFO:teuthology.orchestra.run.smithi152.stdout: ceph telemetry on 2024-09-11T22:44:02.860 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:44:02.860 INFO:teuthology.orchestra.run.smithi152.stdout:For more information see: 2024-09-11T22:44:02.860 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:44:02.860 INFO:teuthology.orchestra.run.smithi152.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-11T22:44:02.860 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:44:02.860 INFO:teuthology.orchestra.run.smithi152.stdout:Bootstrap complete. 2024-09-11T22:44:02.896 INFO:tasks.cephadm:Fetching config... 2024-09-11T22:44:02.896 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:44:02.896 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T22:44:02.913 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T22:44:02.913 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:44:02.913 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T22:44:02.969 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T22:44:02.969 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:44:02.969 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T22:44:03.034 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T22:44:03.034 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:44:03.034 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T22:44:03.089 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T22:44:03.090 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFgLqTslnLmsR6QRRT/mWeoZoK2ggG3HNAriMiMo3k2vv/qL8kQ8gf2LdPGmF7ijtuBVFtgPjwtUHSIR/nnc9/bSbueLmlkfADVyVo5hG6UPRIK4uObXLBBSK1yCdrA0kDqQn1/AQCMcJvc49jhDZhVNKp6Cy11SSovCMnnKnggCmB9PTWL6R+rALluxMyUTWtQ7+GbiH7nQmEGhAmLGj6gnSjHtW2qt/uiSzvGTRfiop2mmPQgV0JJXcNT57uh8ovMJQrzkNkK1Ls05wUy7QzqO4p/ZCR5yn6jclJlr3xD2NXWWt6R+VOAjDrvLDqGusTSeDvzHpm5PIW1NqjgIn0pVE4fzkrbP+EdU50zlzmsEtWeqBIwRvCx4xizkfGiGxfIEjiwiSHMK46CEIY7Sq6y0pEtP0Z1ECjJ275jRaldk+QLH/7Go9ykyO00qNZlHbGXRLtmItiJy4VjqbceOu5TTv0wsFzFJo7taQoD4SGWTA2yz76vTFCCKRPt722YD8= ceph-0e4e82bc-708f-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T22:44:03.163 INFO:teuthology.orchestra.run.smithi152.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFgLqTslnLmsR6QRRT/mWeoZoK2ggG3HNAriMiMo3k2vv/qL8kQ8gf2LdPGmF7ijtuBVFtgPjwtUHSIR/nnc9/bSbueLmlkfADVyVo5hG6UPRIK4uObXLBBSK1yCdrA0kDqQn1/AQCMcJvc49jhDZhVNKp6Cy11SSovCMnnKnggCmB9PTWL6R+rALluxMyUTWtQ7+GbiH7nQmEGhAmLGj6gnSjHtW2qt/uiSzvGTRfiop2mmPQgV0JJXcNT57uh8ovMJQrzkNkK1Ls05wUy7QzqO4p/ZCR5yn6jclJlr3xD2NXWWt6R+VOAjDrvLDqGusTSeDvzHpm5PIW1NqjgIn0pVE4fzkrbP+EdU50zlzmsEtWeqBIwRvCx4xizkfGiGxfIEjiwiSHMK46CEIY7Sq6y0pEtP0Z1ECjJ275jRaldk+QLH/7Go9ykyO00qNZlHbGXRLtmItiJy4VjqbceOu5TTv0wsFzFJo7taQoD4SGWTA2yz76vTFCCKRPt722YD8= ceph-0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:44:03.174 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFgLqTslnLmsR6QRRT/mWeoZoK2ggG3HNAriMiMo3k2vv/qL8kQ8gf2LdPGmF7ijtuBVFtgPjwtUHSIR/nnc9/bSbueLmlkfADVyVo5hG6UPRIK4uObXLBBSK1yCdrA0kDqQn1/AQCMcJvc49jhDZhVNKp6Cy11SSovCMnnKnggCmB9PTWL6R+rALluxMyUTWtQ7+GbiH7nQmEGhAmLGj6gnSjHtW2qt/uiSzvGTRfiop2mmPQgV0JJXcNT57uh8ovMJQrzkNkK1Ls05wUy7QzqO4p/ZCR5yn6jclJlr3xD2NXWWt6R+VOAjDrvLDqGusTSeDvzHpm5PIW1NqjgIn0pVE4fzkrbP+EdU50zlzmsEtWeqBIwRvCx4xizkfGiGxfIEjiwiSHMK46CEIY7Sq6y0pEtP0Z1ECjJ275jRaldk+QLH/7Go9ykyO00qNZlHbGXRLtmItiJy4VjqbceOu5TTv0wsFzFJo7taQoD4SGWTA2yz76vTFCCKRPt722YD8= ceph-0e4e82bc-708f-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T22:44:03.216 INFO:teuthology.orchestra.run.smithi191.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFgLqTslnLmsR6QRRT/mWeoZoK2ggG3HNAriMiMo3k2vv/qL8kQ8gf2LdPGmF7ijtuBVFtgPjwtUHSIR/nnc9/bSbueLmlkfADVyVo5hG6UPRIK4uObXLBBSK1yCdrA0kDqQn1/AQCMcJvc49jhDZhVNKp6Cy11SSovCMnnKnggCmB9PTWL6R+rALluxMyUTWtQ7+GbiH7nQmEGhAmLGj6gnSjHtW2qt/uiSzvGTRfiop2mmPQgV0JJXcNT57uh8ovMJQrzkNkK1Ls05wUy7QzqO4p/ZCR5yn6jclJlr3xD2NXWWt6R+VOAjDrvLDqGusTSeDvzHpm5PIW1NqjgIn0pVE4fzkrbP+EdU50zlzmsEtWeqBIwRvCx4xizkfGiGxfIEjiwiSHMK46CEIY7Sq6y0pEtP0Z1ECjJ275jRaldk+QLH/7Go9ykyO00qNZlHbGXRLtmItiJy4VjqbceOu5TTv0wsFzFJo7taQoD4SGWTA2yz76vTFCCKRPt722YD8= ceph-0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:44:03.228 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFgLqTslnLmsR6QRRT/mWeoZoK2ggG3HNAriMiMo3k2vv/qL8kQ8gf2LdPGmF7ijtuBVFtgPjwtUHSIR/nnc9/bSbueLmlkfADVyVo5hG6UPRIK4uObXLBBSK1yCdrA0kDqQn1/AQCMcJvc49jhDZhVNKp6Cy11SSovCMnnKnggCmB9PTWL6R+rALluxMyUTWtQ7+GbiH7nQmEGhAmLGj6gnSjHtW2qt/uiSzvGTRfiop2mmPQgV0JJXcNT57uh8ovMJQrzkNkK1Ls05wUy7QzqO4p/ZCR5yn6jclJlr3xD2NXWWt6R+VOAjDrvLDqGusTSeDvzHpm5PIW1NqjgIn0pVE4fzkrbP+EdU50zlzmsEtWeqBIwRvCx4xizkfGiGxfIEjiwiSHMK46CEIY7Sq6y0pEtP0Z1ECjJ275jRaldk+QLH/7Go9ykyO00qNZlHbGXRLtmItiJy4VjqbceOu5TTv0wsFzFJo7taQoD4SGWTA2yz76vTFCCKRPt722YD8= ceph-0e4e82bc-708f-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T22:44:03.269 INFO:teuthology.orchestra.run.smithi204.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFgLqTslnLmsR6QRRT/mWeoZoK2ggG3HNAriMiMo3k2vv/qL8kQ8gf2LdPGmF7ijtuBVFtgPjwtUHSIR/nnc9/bSbueLmlkfADVyVo5hG6UPRIK4uObXLBBSK1yCdrA0kDqQn1/AQCMcJvc49jhDZhVNKp6Cy11SSovCMnnKnggCmB9PTWL6R+rALluxMyUTWtQ7+GbiH7nQmEGhAmLGj6gnSjHtW2qt/uiSzvGTRfiop2mmPQgV0JJXcNT57uh8ovMJQrzkNkK1Ls05wUy7QzqO4p/ZCR5yn6jclJlr3xD2NXWWt6R+VOAjDrvLDqGusTSeDvzHpm5PIW1NqjgIn0pVE4fzkrbP+EdU50zlzmsEtWeqBIwRvCx4xizkfGiGxfIEjiwiSHMK46CEIY7Sq6y0pEtP0Z1ECjJ275jRaldk+QLH/7Go9ykyO00qNZlHbGXRLtmItiJy4VjqbceOu5TTv0wsFzFJo7taQoD4SGWTA2yz76vTFCCKRPt722YD8= ceph-0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:44:03.282 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T22:44:03.442 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:03 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1885871938' entity='client.admin' 2024-09-11T22:44:03.537 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:44:04.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.029+0000 7fc926c53640 1 -- 172.21.15.152:0/870656291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc920102160 msgr2=0x7fc920104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.029+0000 7fc926c53640 1 --2- 172.21.15.152:0/870656291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc920102160 0x7fc920104550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc914009880 tx=0x7fc91402a5d0 comp rx=0 tx=0).stop 2024-09-11T22:44:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.029+0000 7fc926c53640 1 -- 172.21.15.152:0/870656291 shutdown_connections 2024-09-11T22:44:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.029+0000 7fc926c53640 1 --2- 172.21.15.152:0/870656291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc920102160 0x7fc920104550 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.029+0000 7fc926c53640 1 -- 172.21.15.152:0/870656291 >> 172.21.15.152:0/870656291 conn(0x7fc9200fbca0 msgr2=0x7fc9200fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.029+0000 7fc926c53640 1 -- 172.21.15.152:0/870656291 shutdown_connections 2024-09-11T22:44:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.029+0000 7fc926c53640 1 -- 172.21.15.152:0/870656291 wait complete. 2024-09-11T22:44:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.030+0000 7fc926c53640 1 Processor -- start 2024-09-11T22:44:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.030+0000 7fc926c53640 1 -- start start 2024-09-11T22:44:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.030+0000 7fc926c53640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc920102160 0x7fc920198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.030+0000 7fc926c53640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc920107730 con 0x7fc920102160 2024-09-11T22:44:04.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.030+0000 7fc9249ef640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc920102160 0x7fc920198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:04.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.030+0000 7fc9249ef640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc920102160 0x7fc920198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60042/0 (socket says 172.21.15.152:60042) 2024-09-11T22:44:04.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.030+0000 7fc9249ef640 1 -- 172.21.15.152:0/986378796 learned_addr learned my addr 172.21.15.152:0/986378796 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:44:04.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.030+0000 7fc9249ef640 1 -- 172.21.15.152:0/986378796 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc920199500 con 0x7fc920102160 2024-09-11T22:44:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.030+0000 7fc9249ef640 1 --2- 172.21.15.152:0/986378796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc920102160 0x7fc920198fc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc914004f60 tx=0x7fc9140048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.031+0000 7fc91dffb640 1 -- 172.21.15.152:0/986378796 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc91403f070 con 0x7fc920102160 2024-09-11T22:44:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.031+0000 7fc91dffb640 1 -- 172.21.15.152:0/986378796 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc914003ad0 con 0x7fc920102160 2024-09-11T22:44:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.031+0000 7fc926c53640 1 -- 172.21.15.152:0/986378796 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc920199790 con 0x7fc920102160 2024-09-11T22:44:04.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.031+0000 7fc91dffb640 1 -- 172.21.15.152:0/986378796 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc914035030 con 0x7fc920102160 2024-09-11T22:44:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.031+0000 7fc926c53640 1 -- 172.21.15.152:0/986378796 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc92019c480 con 0x7fc920102160 2024-09-11T22:44:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.032+0000 7fc91dffb640 1 -- 172.21.15.152:0/986378796 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fc914042070 con 0x7fc920102160 2024-09-11T22:44:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.032+0000 7fc91dffb640 1 --2- 172.21.15.152:0/986378796 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc910039060 0x7fc91003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.032+0000 7fc926c53640 1 -- 172.21.15.152:0/986378796 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc920107eb0 con 0x7fc920102160 2024-09-11T22:44:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.032+0000 7fc91dffb640 1 -- 172.21.15.152:0/986378796 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc91406cde0 con 0x7fc920102160 2024-09-11T22:44:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.032+0000 7fc91ffff640 1 --2- 172.21.15.152:0/986378796 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc910039060 0x7fc91003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:04.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.032+0000 7fc91ffff640 1 --2- 172.21.15.152:0/986378796 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc910039060 0x7fc91003b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0098c0 tx=0x7fc90c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:04.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.037+0000 7fc91dffb640 1 -- 172.21.15.152:0/986378796 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc914039350 con 0x7fc920102160 2024-09-11T22:44:04.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.208+0000 7fc926c53640 1 -- 172.21.15.152:0/986378796 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fc9201025c0 con 0x7fc920102160 2024-09-11T22:44:04.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.210+0000 7fc91dffb640 1 -- 172.21.15.152:0/986378796 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fc9201025c0 con 0x7fc920102160 2024-09-11T22:44:04.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.212+0000 7fc926c53640 1 -- 172.21.15.152:0/986378796 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc910039060 msgr2=0x7fc91003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:04.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.212+0000 7fc926c53640 1 --2- 172.21.15.152:0/986378796 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc910039060 0x7fc91003b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0098c0 tx=0x7fc90c006db0 comp rx=0 tx=0).stop 2024-09-11T22:44:04.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.212+0000 7fc926c53640 1 -- 172.21.15.152:0/986378796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc920102160 msgr2=0x7fc920198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:04.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.212+0000 7fc926c53640 1 --2- 172.21.15.152:0/986378796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc920102160 0x7fc920198fc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc914004f60 tx=0x7fc9140048f0 comp rx=0 tx=0).stop 2024-09-11T22:44:04.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.212+0000 7fc926c53640 1 -- 172.21.15.152:0/986378796 shutdown_connections 2024-09-11T22:44:04.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.212+0000 7fc926c53640 1 --2- 172.21.15.152:0/986378796 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc910039060 0x7fc91003b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:04.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.212+0000 7fc926c53640 1 --2- 172.21.15.152:0/986378796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc920102160 0x7fc920198fc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:04.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.212+0000 7fc926c53640 1 -- 172.21.15.152:0/986378796 >> 172.21.15.152:0/986378796 conn(0x7fc9200fbca0 msgr2=0x7fc9200fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:04.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.212+0000 7fc926c53640 1 -- 172.21.15.152:0/986378796 shutdown_connections 2024-09-11T22:44:04.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:04.213+0000 7fc926c53640 1 -- 172.21.15.152:0/986378796 wait complete. 2024-09-11T22:44:05.333 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T22:44:05.334 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T22:44:05.487 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:05 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/986378796' entity='client.admin' 2024-09-11T22:44:05.487 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:05 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:05.487 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:05 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:05.487 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:05 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi152", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:44:05.487 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:05 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:05.487 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:05 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:44:05.487 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:05 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:05.581 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:44:06.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.025+0000 7fbe7d336640 1 -- 172.21.15.152:0/931770558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe781020c0 msgr2=0x7fbe781044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:06.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.025+0000 7fbe7d336640 1 --2- 172.21.15.152:0/931770558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe781020c0 0x7fbe781044b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009c20 tx=0x7fbe6802aa90 comp rx=0 tx=0).stop 2024-09-11T22:44:06.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.025+0000 7fbe7d336640 1 -- 172.21.15.152:0/931770558 shutdown_connections 2024-09-11T22:44:06.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.025+0000 7fbe7d336640 1 --2- 172.21.15.152:0/931770558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe781020c0 0x7fbe781044b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:06.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.025+0000 7fbe7d336640 1 -- 172.21.15.152:0/931770558 >> 172.21.15.152:0/931770558 conn(0x7fbe780fbca0 msgr2=0x7fbe780fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:06.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.026+0000 7fbe7d336640 1 -- 172.21.15.152:0/931770558 shutdown_connections 2024-09-11T22:44:06.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.026+0000 7fbe7d336640 1 -- 172.21.15.152:0/931770558 wait complete. 2024-09-11T22:44:06.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.026+0000 7fbe7d336640 1 Processor -- start 2024-09-11T22:44:06.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.026+0000 7fbe7d336640 1 -- start start 2024-09-11T22:44:06.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.026+0000 7fbe7d336640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe781020c0 0x7fbe78196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:06.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.026+0000 7fbe7d336640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbe781076d0 con 0x7fbe781020c0 2024-09-11T22:44:06.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.027+0000 7fbe76ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe781020c0 0x7fbe78196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:06.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.027+0000 7fbe76ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe781020c0 0x7fbe78196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60070/0 (socket says 172.21.15.152:60070) 2024-09-11T22:44:06.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.027+0000 7fbe76ffd640 1 -- 172.21.15.152:0/2433920766 learned_addr learned my addr 172.21.15.152:0/2433920766 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:44:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.027+0000 7fbe76ffd640 1 -- 172.21.15.152:0/2433920766 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe78197290 con 0x7fbe781020c0 2024-09-11T22:44:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.027+0000 7fbe76ffd640 1 --2- 172.21.15.152:0/2433920766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe781020c0 0x7fbe78196d50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbe68006010 tx=0x7fbe6802e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.027+0000 7fbe5ffff640 1 -- 172.21.15.152:0/2433920766 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe68047070 con 0x7fbe781020c0 2024-09-11T22:44:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.027+0000 7fbe5ffff640 1 -- 172.21.15.152:0/2433920766 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe68003e50 con 0x7fbe781020c0 2024-09-11T22:44:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.027+0000 7fbe7d336640 1 -- 172.21.15.152:0/2433920766 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe78197580 con 0x7fbe781020c0 2024-09-11T22:44:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.027+0000 7fbe5ffff640 1 -- 172.21.15.152:0/2433920766 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe68033050 con 0x7fbe781020c0 2024-09-11T22:44:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.027+0000 7fbe7d336640 1 -- 172.21.15.152:0/2433920766 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe7819a210 con 0x7fbe781020c0 2024-09-11T22:44:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.028+0000 7fbe5ffff640 1 -- 172.21.15.152:0/2433920766 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fbe6804a070 con 0x7fbe781020c0 2024-09-11T22:44:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.028+0000 7fbe5ffff640 1 --2- 172.21.15.152:0/2433920766 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbe60039060 0x7fbe6003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.028+0000 7fbe5ffff640 1 -- 172.21.15.152:0/2433920766 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbe6803e030 con 0x7fbe781020c0 2024-09-11T22:44:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.028+0000 7fbe7d336640 1 -- 172.21.15.152:0/2433920766 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe78107e50 con 0x7fbe781020c0 2024-09-11T22:44:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.028+0000 7fbe767fc640 1 --2- 172.21.15.152:0/2433920766 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbe60039060 0x7fbe6003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:06.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.029+0000 7fbe767fc640 1 --2- 172.21.15.152:0/2433920766 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbe60039060 0x7fbe6003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0098c0 tx=0x7fbe6c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:06.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.033+0000 7fbe5ffff640 1 -- 172.21.15.152:0/2433920766 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe68051080 con 0x7fbe781020c0 2024-09-11T22:44:06.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.203+0000 7fbe7d336640 1 -- 172.21.15.152:0/2433920766 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fbe7802d8c0 con 0x7fbe60039060 2024-09-11T22:44:06.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.206+0000 7fbe5ffff640 1 -- 172.21.15.152:0/2433920766 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbe7802d8c0 con 0x7fbe60039060 2024-09-11T22:44:06.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.208+0000 7fbe7d336640 1 -- 172.21.15.152:0/2433920766 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbe60039060 msgr2=0x7fbe6003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:06.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.208+0000 7fbe7d336640 1 --2- 172.21.15.152:0/2433920766 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbe60039060 0x7fbe6003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0098c0 tx=0x7fbe6c006db0 comp rx=0 tx=0).stop 2024-09-11T22:44:06.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.208+0000 7fbe7d336640 1 -- 172.21.15.152:0/2433920766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe781020c0 msgr2=0x7fbe78196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:06.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.208+0000 7fbe7d336640 1 --2- 172.21.15.152:0/2433920766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe781020c0 0x7fbe78196d50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbe68006010 tx=0x7fbe6802e3c0 comp rx=0 tx=0).stop 2024-09-11T22:44:06.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.209+0000 7fbe7d336640 1 -- 172.21.15.152:0/2433920766 shutdown_connections 2024-09-11T22:44:06.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.209+0000 7fbe7d336640 1 --2- 172.21.15.152:0/2433920766 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbe60039060 0x7fbe6003b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:06.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.209+0000 7fbe7d336640 1 --2- 172.21.15.152:0/2433920766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe781020c0 0x7fbe78196d50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:06.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.209+0000 7fbe7d336640 1 -- 172.21.15.152:0/2433920766 >> 172.21.15.152:0/2433920766 conn(0x7fbe780fbca0 msgr2=0x7fbe780fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:06.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.209+0000 7fbe7d336640 1 -- 172.21.15.152:0/2433920766 shutdown_connections 2024-09-11T22:44:06.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:06.209+0000 7fbe7d336640 1 -- 172.21.15.152:0/2433920766 wait complete. 2024-09-11T22:44:07.004 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi191 2024-09-11T22:44:07.005 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-11T22:44:07.005 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/etc/ceph/ceph.conf 2024-09-11T22:44:07.023 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-11T22:44:07.023 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:44:07.081 INFO:tasks.cephadm:Adding host smithi191 to orchestrator... 2024-09-11T22:44:07.082 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch host add smithi191 2024-09-11T22:44:07.324 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:44:07.502 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:07 smithi152 ceph-mon[31181]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:44:07.502 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:07 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:07.502 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:07 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:44:07.502 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:07 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:07.502 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:07 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:44:07.502 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:07 smithi152 ceph-mon[31181]: Updating smithi152:/etc/ceph/ceph.conf 2024-09-11T22:44:07.502 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:07 smithi152 ceph-mon[31181]: Updating smithi152:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:44:07.502 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:07 smithi152 ceph-mon[31181]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:44:07.502 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:07 smithi152 ceph-mon[31181]: Updating smithi152:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:44:07.502 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:07 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:07.502 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:07 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:07.503 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:07 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:07.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.808+0000 7f4b1137c640 1 -- 172.21.15.152:0/2540694089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b0c100990 msgr2=0x7f4b0c100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:07.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.808+0000 7f4b1137c640 1 --2- 172.21.15.152:0/2540694089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b0c100990 0x7f4b0c100d70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009880 tx=0x7f4afc02a5d0 comp rx=0 tx=0).stop 2024-09-11T22:44:07.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.808+0000 7f4b1137c640 1 -- 172.21.15.152:0/2540694089 shutdown_connections 2024-09-11T22:44:07.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.808+0000 7f4b1137c640 1 --2- 172.21.15.152:0/2540694089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b0c100990 0x7f4b0c100d70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:07.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.808+0000 7f4b1137c640 1 -- 172.21.15.152:0/2540694089 >> 172.21.15.152:0/2540694089 conn(0x7f4b0c076250 msgr2=0x7f4b0c076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:07.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.808+0000 7f4b1137c640 1 -- 172.21.15.152:0/2540694089 shutdown_connections 2024-09-11T22:44:07.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.808+0000 7f4b1137c640 1 -- 172.21.15.152:0/2540694089 wait complete. 2024-09-11T22:44:07.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.809+0000 7f4b1137c640 1 Processor -- start 2024-09-11T22:44:07.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.809+0000 7f4b1137c640 1 -- start start 2024-09-11T22:44:07.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.809+0000 7f4b1137c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b0c100990 0x7f4b0c196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:07.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.809+0000 7f4b1137c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c10ab90 con 0x7f4b0c100990 2024-09-11T22:44:07.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.809+0000 7f4b0affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b0c100990 0x7f4b0c196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:07.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.809+0000 7f4b0affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b0c100990 0x7f4b0c196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60082/0 (socket says 172.21.15.152:60082) 2024-09-11T22:44:07.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.809+0000 7f4b0affd640 1 -- 172.21.15.152:0/824164708 learned_addr learned my addr 172.21.15.152:0/824164708 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:44:07.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.810+0000 7f4b0affd640 1 -- 172.21.15.152:0/824164708 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0c196670 con 0x7f4b0c100990 2024-09-11T22:44:07.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.810+0000 7f4b0affd640 1 --2- 172.21.15.152:0/824164708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b0c100990 0x7f4b0c196130 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009230 tx=0x7f4afc003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:07.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.810+0000 7f4b08ff9640 1 -- 172.21.15.152:0/824164708 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4afc03f070 con 0x7f4b0c100990 2024-09-11T22:44:07.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.810+0000 7f4b08ff9640 1 -- 172.21.15.152:0/824164708 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4afc03bc80 con 0x7f4b0c100990 2024-09-11T22:44:07.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.810+0000 7f4b1137c640 1 -- 172.21.15.152:0/824164708 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0c196960 con 0x7f4b0c100990 2024-09-11T22:44:07.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.810+0000 7f4b08ff9640 1 -- 172.21.15.152:0/824164708 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4afc035030 con 0x7f4b0c100990 2024-09-11T22:44:07.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.810+0000 7f4b1137c640 1 -- 172.21.15.152:0/824164708 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b0c19aa00 con 0x7f4b0c100990 2024-09-11T22:44:07.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.811+0000 7f4b08ff9640 1 -- 172.21.15.152:0/824164708 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f4afc042070 con 0x7f4b0c100990 2024-09-11T22:44:07.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.811+0000 7f4b1137c640 1 -- 172.21.15.152:0/824164708 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b0c101c40 con 0x7f4b0c100990 2024-09-11T22:44:07.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.811+0000 7f4b08ff9640 1 --2- 172.21.15.152:0/824164708 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4af80390c0 0x7f4af803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:07.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.811+0000 7f4b08ff9640 1 -- 172.21.15.152:0/824164708 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4afc06cb20 con 0x7f4b0c100990 2024-09-11T22:44:07.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.811+0000 7f4b03fff640 1 --2- 172.21.15.152:0/824164708 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4af80390c0 0x7f4af803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:07.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.812+0000 7f4b03fff640 1 --2- 172.21.15.152:0/824164708 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4af80390c0 0x7f4af803b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4af40098c0 tx=0x7f4af4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:07.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.816+0000 7f4b08ff9640 1 -- 172.21.15.152:0/824164708 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4afc030c30 con 0x7f4b0c100990 2024-09-11T22:44:07.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:07.985+0000 7f4b1137c640 1 -- 172.21.15.152:0/824164708 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}) -- 0x7f4b0c100df0 con 0x7f4af80390c0 2024-09-11T22:44:09.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:08 smithi152 ceph-mon[31181]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:44:10.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:09 smithi152 ceph-mon[31181]: Deploying cephadm binary to smithi191 2024-09-11T22:44:11.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:11.149+0000 7f4b08ff9640 1 -- 172.21.15.152:0/824164708 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f4b0c100df0 con 0x7f4af80390c0 2024-09-11T22:44:11.151 INFO:teuthology.orchestra.run.smithi152.stdout:Added host 'smithi191' with addr '172.21.15.191' 2024-09-11T22:44:11.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:11.152+0000 7f4b1137c640 1 -- 172.21.15.152:0/824164708 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4af80390c0 msgr2=0x7f4af803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:11.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:11.152+0000 7f4b1137c640 1 --2- 172.21.15.152:0/824164708 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4af80390c0 0x7f4af803b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4af40098c0 tx=0x7f4af4006db0 comp rx=0 tx=0).stop 2024-09-11T22:44:11.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:11.152+0000 7f4b1137c640 1 -- 172.21.15.152:0/824164708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b0c100990 msgr2=0x7f4b0c196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:11.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:11.152+0000 7f4b1137c640 1 --2- 172.21.15.152:0/824164708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b0c100990 0x7f4b0c196130 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009230 tx=0x7f4afc003d40 comp rx=0 tx=0).stop 2024-09-11T22:44:11.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:11.152+0000 7f4b1137c640 1 -- 172.21.15.152:0/824164708 shutdown_connections 2024-09-11T22:44:11.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:11.152+0000 7f4b1137c640 1 --2- 172.21.15.152:0/824164708 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4af80390c0 0x7f4af803b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:11.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:11.152+0000 7f4b1137c640 1 --2- 172.21.15.152:0/824164708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b0c100990 0x7f4b0c196130 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:11.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:11.152+0000 7f4b1137c640 1 -- 172.21.15.152:0/824164708 >> 172.21.15.152:0/824164708 conn(0x7f4b0c076250 msgr2=0x7f4b0c1020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:11.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:11.152+0000 7f4b1137c640 1 -- 172.21.15.152:0/824164708 shutdown_connections 2024-09-11T22:44:11.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:11.152+0000 7f4b1137c640 1 -- 172.21.15.152:0/824164708 wait complete. 2024-09-11T22:44:11.819 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T22:44:12.056 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:44:12.436 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:12 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:12.436 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:12 smithi152 ceph-mon[31181]: Added host smithi191 2024-09-11T22:44:12.436 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:12 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:44:12.436 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:12 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:12.436 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:12 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:12.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.515+0000 7fbc3a8aa640 1 -- 172.21.15.152:0/3626961495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc34106590 msgr2=0x7fbc34106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:12.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.515+0000 7fbc3a8aa640 1 --2- 172.21.15.152:0/3626961495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc34106590 0x7fbc34106970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009880 tx=0x7fbc2402a5d0 comp rx=0 tx=0).stop 2024-09-11T22:44:12.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.515+0000 7fbc3a8aa640 1 -- 172.21.15.152:0/3626961495 shutdown_connections 2024-09-11T22:44:12.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.515+0000 7fbc3a8aa640 1 --2- 172.21.15.152:0/3626961495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc34106590 0x7fbc34106970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.515+0000 7fbc3a8aa640 1 -- 172.21.15.152:0/3626961495 >> 172.21.15.152:0/3626961495 conn(0x7fbc340fc1a0 msgr2=0x7fbc340fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.515+0000 7fbc3a8aa640 1 -- 172.21.15.152:0/3626961495 shutdown_connections 2024-09-11T22:44:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.515+0000 7fbc3a8aa640 1 -- 172.21.15.152:0/3626961495 wait complete. 2024-09-11T22:44:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.516+0000 7fbc3a8aa640 1 Processor -- start 2024-09-11T22:44:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.516+0000 7fbc3a8aa640 1 -- start start 2024-09-11T22:44:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.516+0000 7fbc3a8aa640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc34106590 0x7fbc34102150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.516+0000 7fbc3a8aa640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3410abc0 con 0x7fbc34106590 2024-09-11T22:44:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.516+0000 7fbc33fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc34106590 0x7fbc34102150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.516+0000 7fbc33fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc34106590 0x7fbc34102150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35042/0 (socket says 172.21.15.152:35042) 2024-09-11T22:44:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.516+0000 7fbc33fff640 1 -- 172.21.15.152:0/3665316195 learned_addr learned my addr 172.21.15.152:0/3665316195 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:44:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.516+0000 7fbc33fff640 1 -- 172.21.15.152:0/3665316195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc34104040 con 0x7fbc34106590 2024-09-11T22:44:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.517+0000 7fbc33fff640 1 --2- 172.21.15.152:0/3665316195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc34106590 0x7fbc34102150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009230 tx=0x7fbc24003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.517+0000 7fbc317fa640 1 -- 172.21.15.152:0/3665316195 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc2403f070 con 0x7fbc34106590 2024-09-11T22:44:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.517+0000 7fbc317fa640 1 -- 172.21.15.152:0/3665316195 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2403bc80 con 0x7fbc34106590 2024-09-11T22:44:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.517+0000 7fbc3a8aa640 1 -- 172.21.15.152:0/3665316195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc34102690 con 0x7fbc34106590 2024-09-11T22:44:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.517+0000 7fbc317fa640 1 -- 172.21.15.152:0/3665316195 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc24035030 con 0x7fbc34106590 2024-09-11T22:44:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.517+0000 7fbc3a8aa640 1 -- 172.21.15.152:0/3665316195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc34102b50 con 0x7fbc34106590 2024-09-11T22:44:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.518+0000 7fbc317fa640 1 -- 172.21.15.152:0/3665316195 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fbc24042070 con 0x7fbc34106590 2024-09-11T22:44:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.518+0000 7fbc3a8aa640 1 -- 172.21.15.152:0/3665316195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc34101c00 con 0x7fbc34106590 2024-09-11T22:44:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.518+0000 7fbc317fa640 1 --2- 172.21.15.152:0/3665316195 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbc1c039060 0x7fbc1c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:12.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.518+0000 7fbc317fa640 1 -- 172.21.15.152:0/3665316195 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbc2406ccd0 con 0x7fbc34106590 2024-09-11T22:44:12.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.518+0000 7fbc337fe640 1 --2- 172.21.15.152:0/3665316195 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbc1c039060 0x7fbc1c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:12.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.519+0000 7fbc337fe640 1 --2- 172.21.15.152:0/3665316195 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbc1c039060 0x7fbc1c03b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbc280098c0 tx=0x7fbc28006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:12.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.523+0000 7fbc317fa640 1 -- 172.21.15.152:0/3665316195 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc2404c350 con 0x7fbc34106590 2024-09-11T22:44:12.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.694+0000 7fbc3a8aa640 1 -- 172.21.15.152:0/3665316195 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbc341036f0 con 0x7fbc1c039060 2024-09-11T22:44:12.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.695+0000 7fbc317fa640 1 -- 172.21.15.152:0/3665316195 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7fbc341036f0 con 0x7fbc1c039060 2024-09-11T22:44:12.697 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:44:12.697 INFO:teuthology.orchestra.run.smithi152.stdout:[{"addr": "172.21.15.152", "hostname": "smithi152", "labels": [], "status": ""}, {"addr": "172.21.15.191", "hostname": "smithi191", "labels": [], "status": ""}] 2024-09-11T22:44:12.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.698+0000 7fbc3a8aa640 1 -- 172.21.15.152:0/3665316195 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbc1c039060 msgr2=0x7fbc1c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:12.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.698+0000 7fbc3a8aa640 1 --2- 172.21.15.152:0/3665316195 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbc1c039060 0x7fbc1c03b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbc280098c0 tx=0x7fbc28006db0 comp rx=0 tx=0).stop 2024-09-11T22:44:12.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.698+0000 7fbc3a8aa640 1 -- 172.21.15.152:0/3665316195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc34106590 msgr2=0x7fbc34102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:12.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.698+0000 7fbc3a8aa640 1 --2- 172.21.15.152:0/3665316195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc34106590 0x7fbc34102150 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009230 tx=0x7fbc24003d40 comp rx=0 tx=0).stop 2024-09-11T22:44:12.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.698+0000 7fbc3a8aa640 1 -- 172.21.15.152:0/3665316195 shutdown_connections 2024-09-11T22:44:12.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.698+0000 7fbc3a8aa640 1 --2- 172.21.15.152:0/3665316195 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbc1c039060 0x7fbc1c03b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:12.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.698+0000 7fbc3a8aa640 1 --2- 172.21.15.152:0/3665316195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc34106590 0x7fbc34102150 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:12.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.698+0000 7fbc3a8aa640 1 -- 172.21.15.152:0/3665316195 >> 172.21.15.152:0/3665316195 conn(0x7fbc340fc1a0 msgr2=0x7fbc340fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:12.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.698+0000 7fbc3a8aa640 1 -- 172.21.15.152:0/3665316195 shutdown_connections 2024-09-11T22:44:12.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:12.698+0000 7fbc3a8aa640 1 -- 172.21.15.152:0/3665316195 wait complete. 2024-09-11T22:44:13.343 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi204 2024-09-11T22:44:13.343 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-11T22:44:13.343 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.conf 2024-09-11T22:44:13.363 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-11T22:44:13.364 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:44:13.422 INFO:tasks.cephadm:Adding host smithi204 to orchestrator... 2024-09-11T22:44:13.422 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch host add smithi204 2024-09-11T22:44:13.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:13.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:13 smithi152 ceph-mon[31181]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:44:13.662 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:44:14.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.116+0000 7fd49b18a640 1 -- 172.21.15.152:0/1552164424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd494106590 msgr2=0x7fd494106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:14.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.116+0000 7fd49b18a640 1 --2- 172.21.15.152:0/1552164424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd494106590 0x7fd494106970 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd488009880 tx=0x7fd48802a5d0 comp rx=0 tx=0).stop 2024-09-11T22:44:14.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.116+0000 7fd49b18a640 1 -- 172.21.15.152:0/1552164424 shutdown_connections 2024-09-11T22:44:14.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.116+0000 7fd49b18a640 1 --2- 172.21.15.152:0/1552164424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd494106590 0x7fd494106970 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:14.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.116+0000 7fd49b18a640 1 -- 172.21.15.152:0/1552164424 >> 172.21.15.152:0/1552164424 conn(0x7fd4940fc1a0 msgr2=0x7fd4940fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:14.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.116+0000 7fd49b18a640 1 -- 172.21.15.152:0/1552164424 shutdown_connections 2024-09-11T22:44:14.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.116+0000 7fd49b18a640 1 -- 172.21.15.152:0/1552164424 wait complete. 2024-09-11T22:44:14.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.117+0000 7fd49b18a640 1 Processor -- start 2024-09-11T22:44:14.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.117+0000 7fd49b18a640 1 -- start start 2024-09-11T22:44:14.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.117+0000 7fd49b18a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd494106590 0x7fd494197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:14.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.117+0000 7fd49b18a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd49410abc0 con 0x7fd494106590 2024-09-11T22:44:14.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.117+0000 7fd498f26640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd494106590 0x7fd494197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:14.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.117+0000 7fd498f26640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd494106590 0x7fd494197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35060/0 (socket says 172.21.15.152:35060) 2024-09-11T22:44:14.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.117+0000 7fd498f26640 1 -- 172.21.15.152:0/2448421151 learned_addr learned my addr 172.21.15.152:0/2448421151 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:44:14.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.118+0000 7fd498f26640 1 -- 172.21.15.152:0/2448421151 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd494197640 con 0x7fd494106590 2024-09-11T22:44:14.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.118+0000 7fd498f26640 1 --2- 172.21.15.152:0/2448421151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd494106590 0x7fd494197100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd488009230 tx=0x7fd488003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:14.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.118+0000 7fd491ffb640 1 -- 172.21.15.152:0/2448421151 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd48803f070 con 0x7fd494106590 2024-09-11T22:44:14.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.118+0000 7fd491ffb640 1 -- 172.21.15.152:0/2448421151 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd48803bc80 con 0x7fd494106590 2024-09-11T22:44:14.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.118+0000 7fd49b18a640 1 -- 172.21.15.152:0/2448421151 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4941978d0 con 0x7fd494106590 2024-09-11T22:44:14.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.118+0000 7fd491ffb640 1 -- 172.21.15.152:0/2448421151 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd488035030 con 0x7fd494106590 2024-09-11T22:44:14.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.118+0000 7fd49b18a640 1 -- 172.21.15.152:0/2448421151 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd49419b9d0 con 0x7fd494106590 2024-09-11T22:44:14.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.119+0000 7fd491ffb640 1 -- 172.21.15.152:0/2448421151 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fd488042070 con 0x7fd494106590 2024-09-11T22:44:14.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.119+0000 7fd49b18a640 1 -- 172.21.15.152:0/2448421151 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd494101c00 con 0x7fd494106590 2024-09-11T22:44:14.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.119+0000 7fd491ffb640 1 --2- 172.21.15.152:0/2448421151 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd4840390c0 0x7fd48403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:14.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.119+0000 7fd491ffb640 1 -- 172.21.15.152:0/2448421151 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd48806cca0 con 0x7fd494106590 2024-09-11T22:44:14.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.119+0000 7fd493fff640 1 --2- 172.21.15.152:0/2448421151 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd4840390c0 0x7fd48403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:14.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.120+0000 7fd493fff640 1 --2- 172.21.15.152:0/2448421151 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd4840390c0 0x7fd48403b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd4800098c0 tx=0x7fd480006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:14.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.124+0000 7fd491ffb640 1 -- 172.21.15.152:0/2448421151 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd4880314c0 con 0x7fd494106590 2024-09-11T22:44:14.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:14.292+0000 7fd49b18a640 1 -- 172.21.15.152:0/2448421151 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}) -- 0x7fd4941069f0 con 0x7fd4840390c0 2024-09-11T22:44:15.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:15 smithi152 ceph-mon[31181]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:44:15.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:15 smithi152 ceph-mon[31181]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:15.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:15 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:15.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:15 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:15.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:15 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:15.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:15 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:15.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:15 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:44:15.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:15 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:15.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:15 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:44:17.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:16 smithi152 ceph-mon[31181]: Deploying cephadm binary to smithi204 2024-09-11T22:44:17.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:16 smithi152 ceph-mon[31181]: Updating smithi191:/etc/ceph/ceph.conf 2024-09-11T22:44:17.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:16 smithi152 ceph-mon[31181]: Updating smithi191:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:44:17.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:16 smithi152 ceph-mon[31181]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:44:17.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:17.424+0000 7fd491ffb640 1 -- 172.21.15.152:0/2448421151 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd4941069f0 con 0x7fd4840390c0 2024-09-11T22:44:17.425 INFO:teuthology.orchestra.run.smithi152.stdout:Added host 'smithi204' with addr '172.21.15.204' 2024-09-11T22:44:17.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:17.427+0000 7fd49b18a640 1 -- 172.21.15.152:0/2448421151 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd4840390c0 msgr2=0x7fd48403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:17.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:17.427+0000 7fd49b18a640 1 --2- 172.21.15.152:0/2448421151 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd4840390c0 0x7fd48403b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd4800098c0 tx=0x7fd480006db0 comp rx=0 tx=0).stop 2024-09-11T22:44:17.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:17.427+0000 7fd49b18a640 1 -- 172.21.15.152:0/2448421151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd494106590 msgr2=0x7fd494197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:17.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:17.427+0000 7fd49b18a640 1 --2- 172.21.15.152:0/2448421151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd494106590 0x7fd494197100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd488009230 tx=0x7fd488003d40 comp rx=0 tx=0).stop 2024-09-11T22:44:17.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:17.427+0000 7fd49b18a640 1 -- 172.21.15.152:0/2448421151 shutdown_connections 2024-09-11T22:44:17.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:17.427+0000 7fd49b18a640 1 --2- 172.21.15.152:0/2448421151 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd4840390c0 0x7fd48403b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:17.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:17.427+0000 7fd49b18a640 1 --2- 172.21.15.152:0/2448421151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd494106590 0x7fd494197100 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:17.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:17.427+0000 7fd49b18a640 1 -- 172.21.15.152:0/2448421151 >> 172.21.15.152:0/2448421151 conn(0x7fd4940fc1a0 msgr2=0x7fd494069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:17.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:17.427+0000 7fd49b18a640 1 -- 172.21.15.152:0/2448421151 shutdown_connections 2024-09-11T22:44:17.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:17.427+0000 7fd49b18a640 1 -- 172.21.15.152:0/2448421151 wait complete. 2024-09-11T22:44:17.989 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:17 smithi152 ceph-mon[31181]: Updating smithi191:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:44:17.990 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:17 smithi152 ceph-mon[31181]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:17.990 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:17.990 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:17.990 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:17.990 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:17.990 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:44:18.016 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T22:44:18.255 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:44:18.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.689+0000 7f537afb4640 1 -- 172.21.15.152:0/965692436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5374100990 msgr2=0x7f5374100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:18.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.689+0000 7f537afb4640 1 --2- 172.21.15.152:0/965692436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5374100990 0x7f5374100d70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5364009c20 tx=0x7f536402aa90 comp rx=0 tx=0).stop 2024-09-11T22:44:18.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.689+0000 7f537afb4640 1 -- 172.21.15.152:0/965692436 shutdown_connections 2024-09-11T22:44:18.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.689+0000 7f537afb4640 1 --2- 172.21.15.152:0/965692436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5374100990 0x7f5374100d70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:18.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.689+0000 7f537afb4640 1 -- 172.21.15.152:0/965692436 >> 172.21.15.152:0/965692436 conn(0x7f5374076250 msgr2=0x7f5374076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:18.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.689+0000 7f537afb4640 1 -- 172.21.15.152:0/965692436 shutdown_connections 2024-09-11T22:44:18.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.689+0000 7f537afb4640 1 -- 172.21.15.152:0/965692436 wait complete. 2024-09-11T22:44:18.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.689+0000 7f537afb4640 1 Processor -- start 2024-09-11T22:44:18.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.689+0000 7f537afb4640 1 -- start start 2024-09-11T22:44:18.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.690+0000 7f537afb4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5374100990 0x7f53741970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:18.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.690+0000 7f537afb4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f537410ab90 con 0x7f5374100990 2024-09-11T22:44:18.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.690+0000 7f5378d50640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5374100990 0x7f53741970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:18.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.690+0000 7f5378d50640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5374100990 0x7f53741970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59182/0 (socket says 172.21.15.152:59182) 2024-09-11T22:44:18.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.690+0000 7f5378d50640 1 -- 172.21.15.152:0/3579022012 learned_addr learned my addr 172.21.15.152:0/3579022012 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:44:18.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.690+0000 7f5378d50640 1 -- 172.21.15.152:0/3579022012 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5374197620 con 0x7f5374100990 2024-09-11T22:44:18.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.690+0000 7f5378d50640 1 --2- 172.21.15.152:0/3579022012 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5374100990 0x7f53741970e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5364005e30 tx=0x7f536402fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:18.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.690+0000 7f5371ffb640 1 -- 172.21.15.152:0/3579022012 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5364047070 con 0x7f5374100990 2024-09-11T22:44:18.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.690+0000 7f5371ffb640 1 -- 172.21.15.152:0/3579022012 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5364003df0 con 0x7f5374100990 2024-09-11T22:44:18.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.690+0000 7f537afb4640 1 -- 172.21.15.152:0/3579022012 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5374197910 con 0x7f5374100990 2024-09-11T22:44:18.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.690+0000 7f5371ffb640 1 -- 172.21.15.152:0/3579022012 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5364033050 con 0x7f5374100990 2024-09-11T22:44:18.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.691+0000 7f537afb4640 1 -- 172.21.15.152:0/3579022012 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537419b9b0 con 0x7f5374100990 2024-09-11T22:44:18.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.691+0000 7f5371ffb640 1 -- 172.21.15.152:0/3579022012 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f536404a070 con 0x7f5374100990 2024-09-11T22:44:18.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.692+0000 7f5371ffb640 1 --2- 172.21.15.152:0/3579022012 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f535c039060 0x7f535c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:18.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.692+0000 7f537afb4640 1 -- 172.21.15.152:0/3579022012 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5374101c40 con 0x7f5374100990 2024-09-11T22:44:18.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.692+0000 7f5371ffb640 1 -- 172.21.15.152:0/3579022012 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f536403e030 con 0x7f5374100990 2024-09-11T22:44:18.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.692+0000 7f5373fff640 1 --2- 172.21.15.152:0/3579022012 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f535c039060 0x7f535c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:18.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.692+0000 7f5373fff640 1 --2- 172.21.15.152:0/3579022012 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f535c039060 0x7f535c03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f53680098c0 tx=0x7f5368006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:18.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.697+0000 7f5371ffb640 1 -- 172.21.15.152:0/3579022012 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53640373b0 con 0x7f5374100990 2024-09-11T22:44:18.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.868+0000 7f537afb4640 1 -- 172.21.15.152:0/3579022012 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5374198440 con 0x7f535c039060 2024-09-11T22:44:18.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.869+0000 7f5371ffb640 1 -- 172.21.15.152:0/3579022012 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f5374198440 con 0x7f535c039060 2024-09-11T22:44:18.869 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:44:18.870 INFO:teuthology.orchestra.run.smithi152.stdout:[{"addr": "172.21.15.152", "hostname": "smithi152", "labels": [], "status": ""}, {"addr": "172.21.15.191", "hostname": "smithi191", "labels": [], "status": ""}, {"addr": "172.21.15.204", "hostname": "smithi204", "labels": [], "status": ""}] 2024-09-11T22:44:18.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.871+0000 7f537afb4640 1 -- 172.21.15.152:0/3579022012 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f535c039060 msgr2=0x7f535c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:18.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.871+0000 7f537afb4640 1 --2- 172.21.15.152:0/3579022012 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f535c039060 0x7f535c03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f53680098c0 tx=0x7f5368006db0 comp rx=0 tx=0).stop 2024-09-11T22:44:18.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.871+0000 7f537afb4640 1 -- 172.21.15.152:0/3579022012 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5374100990 msgr2=0x7f53741970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:18.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.871+0000 7f537afb4640 1 --2- 172.21.15.152:0/3579022012 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5374100990 0x7f53741970e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5364005e30 tx=0x7f536402fc10 comp rx=0 tx=0).stop 2024-09-11T22:44:18.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.871+0000 7f537afb4640 1 -- 172.21.15.152:0/3579022012 shutdown_connections 2024-09-11T22:44:18.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.871+0000 7f537afb4640 1 --2- 172.21.15.152:0/3579022012 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f535c039060 0x7f535c03b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:18.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.871+0000 7f537afb4640 1 --2- 172.21.15.152:0/3579022012 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5374100990 0x7f53741970e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:18.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.871+0000 7f537afb4640 1 -- 172.21.15.152:0/3579022012 >> 172.21.15.152:0/3579022012 conn(0x7f5374076250 msgr2=0x7f53740696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:18.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.872+0000 7f537afb4640 1 -- 172.21.15.152:0/3579022012 shutdown_connections 2024-09-11T22:44:18.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:18.872+0000 7f537afb4640 1 -- 172.21.15.152:0/3579022012 wait complete. 2024-09-11T22:44:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:18 smithi152 ceph-mon[31181]: Added host smithi204 2024-09-11T22:44:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:19.535 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T22:44:19.536 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T22:44:19.778 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:44:20.070 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:19 smithi152 ceph-mon[31181]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:20.070 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:19 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:20.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.292+0000 7f1509e7b640 1 -- 172.21.15.152:0/213200196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1504100990 msgr2=0x7f1504100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:20.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.292+0000 7f1509e7b640 1 --2- 172.21.15.152:0/213200196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1504100990 0x7f1504100d70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009880 tx=0x7f14f402a5d0 comp rx=0 tx=0).stop 2024-09-11T22:44:20.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.292+0000 7f1509e7b640 1 -- 172.21.15.152:0/213200196 shutdown_connections 2024-09-11T22:44:20.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.292+0000 7f1509e7b640 1 --2- 172.21.15.152:0/213200196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1504100990 0x7f1504100d70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:20.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.292+0000 7f1509e7b640 1 -- 172.21.15.152:0/213200196 >> 172.21.15.152:0/213200196 conn(0x7f1504076250 msgr2=0x7f1504076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:20.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.292+0000 7f1509e7b640 1 -- 172.21.15.152:0/213200196 shutdown_connections 2024-09-11T22:44:20.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.292+0000 7f1509e7b640 1 -- 172.21.15.152:0/213200196 wait complete. 2024-09-11T22:44:20.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.293+0000 7f1509e7b640 1 Processor -- start 2024-09-11T22:44:20.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.293+0000 7f1509e7b640 1 -- start start 2024-09-11T22:44:20.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.293+0000 7f1509e7b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1504100990 0x7f15041970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:20.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.293+0000 7f1509e7b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f150410ab90 con 0x7f1504100990 2024-09-11T22:44:20.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.293+0000 7f15037fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1504100990 0x7f15041970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:20.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.293+0000 7f15037fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1504100990 0x7f15041970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59194/0 (socket says 172.21.15.152:59194) 2024-09-11T22:44:20.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.293+0000 7f15037fe640 1 -- 172.21.15.152:0/4225721794 learned_addr learned my addr 172.21.15.152:0/4225721794 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:44:20.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.293+0000 7f15037fe640 1 -- 172.21.15.152:0/4225721794 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1504197620 con 0x7f1504100990 2024-09-11T22:44:20.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.294+0000 7f15037fe640 1 --2- 172.21.15.152:0/4225721794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1504100990 0x7f15041970e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009230 tx=0x7f14f4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:20.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.294+0000 7f15017fa640 1 -- 172.21.15.152:0/4225721794 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14f403f070 con 0x7f1504100990 2024-09-11T22:44:20.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.294+0000 7f1509e7b640 1 -- 172.21.15.152:0/4225721794 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1504197910 con 0x7f1504100990 2024-09-11T22:44:20.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.294+0000 7f15017fa640 1 -- 172.21.15.152:0/4225721794 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14f403bc80 con 0x7f1504100990 2024-09-11T22:44:20.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.294+0000 7f1509e7b640 1 -- 172.21.15.152:0/4225721794 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150419b9b0 con 0x7f1504100990 2024-09-11T22:44:20.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.294+0000 7f15017fa640 1 -- 172.21.15.152:0/4225721794 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14f4035030 con 0x7f1504100990 2024-09-11T22:44:20.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.295+0000 7f15017fa640 1 -- 172.21.15.152:0/4225721794 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f14f4042070 con 0x7f1504100990 2024-09-11T22:44:20.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.295+0000 7f1509e7b640 1 -- 172.21.15.152:0/4225721794 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1504101c40 con 0x7f1504100990 2024-09-11T22:44:20.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.295+0000 7f15017fa640 1 --2- 172.21.15.152:0/4225721794 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f14f0039060 0x7f14f003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:20.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.295+0000 7f15017fa640 1 -- 172.21.15.152:0/4225721794 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f14f406ccc0 con 0x7f1504100990 2024-09-11T22:44:20.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.295+0000 7f14fbfff640 1 --2- 172.21.15.152:0/4225721794 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f14f0039060 0x7f14f003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:20.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.296+0000 7f14fbfff640 1 --2- 172.21.15.152:0/4225721794 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f14f0039060 0x7f14f003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0098c0 tx=0x7f14ec006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:20.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.300+0000 7f15017fa640 1 -- 172.21.15.152:0/4225721794 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f14f40372d0 con 0x7f1504100990 2024-09-11T22:44:20.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.473+0000 7f1509e7b640 1 -- 172.21.15.152:0/4225721794 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1504198440 con 0x7f1504100990 2024-09-11T22:44:20.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.773+0000 7f15017fa640 1 -- 172.21.15.152:0/4225721794 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f14f4031570 con 0x7f1504100990 2024-09-11T22:44:20.774 INFO:teuthology.orchestra.run.smithi152.stderr:adjusted tunables profile to default 2024-09-11T22:44:20.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.776+0000 7f1509e7b640 1 -- 172.21.15.152:0/4225721794 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f14f0039060 msgr2=0x7f14f003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:20.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.776+0000 7f1509e7b640 1 --2- 172.21.15.152:0/4225721794 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f14f0039060 0x7f14f003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0098c0 tx=0x7f14ec006db0 comp rx=0 tx=0).stop 2024-09-11T22:44:20.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.776+0000 7f1509e7b640 1 -- 172.21.15.152:0/4225721794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1504100990 msgr2=0x7f15041970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:20.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.776+0000 7f1509e7b640 1 --2- 172.21.15.152:0/4225721794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1504100990 0x7f15041970e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009230 tx=0x7f14f4003d40 comp rx=0 tx=0).stop 2024-09-11T22:44:20.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.776+0000 7f1509e7b640 1 -- 172.21.15.152:0/4225721794 shutdown_connections 2024-09-11T22:44:20.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.776+0000 7f1509e7b640 1 --2- 172.21.15.152:0/4225721794 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f14f0039060 0x7f14f003b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:20.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.776+0000 7f1509e7b640 1 --2- 172.21.15.152:0/4225721794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1504100990 0x7f15041970e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:20.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.776+0000 7f1509e7b640 1 -- 172.21.15.152:0/4225721794 >> 172.21.15.152:0/4225721794 conn(0x7f1504076250 msgr2=0x7f15040696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:20.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.776+0000 7f1509e7b640 1 -- 172.21.15.152:0/4225721794 shutdown_connections 2024-09-11T22:44:20.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:20.776+0000 7f1509e7b640 1 -- 172.21.15.152:0/4225721794 wait complete. 2024-09-11T22:44:21.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:20 smithi152 ceph-mon[31181]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:44:21.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:20 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4225721794' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T22:44:21.431 INFO:tasks.cephadm:Adding mon.a on smithi152 2024-09-11T22:44:21.432 INFO:tasks.cephadm:Adding mon.b on smithi191 2024-09-11T22:44:21.432 INFO:tasks.cephadm:Adding mon.c on smithi204 2024-09-11T22:44:21.432 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi152:172.21.15.152=a;smithi191:172.21.15.191=b;smithi204:172.21.15.204=c' 2024-09-11T22:44:21.681 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T22:44:21.728 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T22:44:22.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:21 smithi152 ceph-mon[31181]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:21 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4225721794' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T22:44:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:21 smithi152 ceph-mon[31181]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:44:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:44:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:44:22.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.195+0000 7f37419c8640 1 -- 172.21.15.204:0/1218982614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f373c100530 msgr2=0x7f373c100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:22.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.195+0000 7f37419c8640 1 --2- 172.21.15.204:0/1218982614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f373c100530 0x7f373c100930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f372c009880 tx=0x7f372c02a5d0 comp rx=0 tx=0).stop 2024-09-11T22:44:22.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.195+0000 7f37419c8640 1 -- 172.21.15.204:0/1218982614 shutdown_connections 2024-09-11T22:44:22.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.195+0000 7f37419c8640 1 --2- 172.21.15.204:0/1218982614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f373c100530 0x7f373c100930 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:22.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.195+0000 7f37419c8640 1 -- 172.21.15.204:0/1218982614 >> 172.21.15.204:0/1218982614 conn(0x7f373c0fbca0 msgr2=0x7f373c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:22.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.195+0000 7f37419c8640 1 -- 172.21.15.204:0/1218982614 shutdown_connections 2024-09-11T22:44:22.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.195+0000 7f37419c8640 1 -- 172.21.15.204:0/1218982614 wait complete. 2024-09-11T22:44:22.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.196+0000 7f37419c8640 1 Processor -- start 2024-09-11T22:44:22.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.196+0000 7f37419c8640 1 -- start start 2024-09-11T22:44:22.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.196+0000 7f37419c8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f373c100530 0x7f373c06a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:22.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.196+0000 7f37419c8640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f373c106640 con 0x7f373c100530 2024-09-11T22:44:22.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.196+0000 7f373affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f373c100530 0x7f373c06a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:22.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.197+0000 7f373affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f373c100530 0x7f373c06a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.204:59368/0 (socket says 172.21.15.204:59368) 2024-09-11T22:44:22.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.197+0000 7f373affd640 1 -- 172.21.15.204:0/3295953297 learned_addr learned my addr 172.21.15.204:0/3295953297 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-11T22:44:22.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.197+0000 7f373affd640 1 -- 172.21.15.204:0/3295953297 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373c199b10 con 0x7f373c100530 2024-09-11T22:44:22.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.197+0000 7f373affd640 1 --2- 172.21.15.204:0/3295953297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f373c100530 0x7f373c06a140 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f372c004f60 tx=0x7f372c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:22.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.197+0000 7f37409c6640 1 -- 172.21.15.204:0/3295953297 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f372c03f070 con 0x7f373c100530 2024-09-11T22:44:22.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.197+0000 7f37419c8640 1 -- 172.21.15.204:0/3295953297 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373c199ce0 con 0x7f373c100530 2024-09-11T22:44:22.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.197+0000 7f37409c6640 1 -- 172.21.15.204:0/3295953297 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f372c004cc0 con 0x7f373c100530 2024-09-11T22:44:22.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.197+0000 7f37409c6640 1 -- 172.21.15.204:0/3295953297 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f372c035030 con 0x7f373c100530 2024-09-11T22:44:22.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.197+0000 7f37419c8640 1 -- 172.21.15.204:0/3295953297 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f373c19a080 con 0x7f373c100530 2024-09-11T22:44:22.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.198+0000 7f37409c6640 1 -- 172.21.15.204:0/3295953297 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f372c042070 con 0x7f373c100530 2024-09-11T22:44:22.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.198+0000 7f37409c6640 1 --2- 172.21.15.204:0/3295953297 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3728039060 0x7f372803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:22.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.198+0000 7f37409c6640 1 -- 172.21.15.204:0/3295953297 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f372c06ce40 con 0x7f373c100530 2024-09-11T22:44:22.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.198+0000 7f37419c8640 1 -- 172.21.15.204:0/3295953297 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f373c106dc0 con 0x7f373c100530 2024-09-11T22:44:22.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.199+0000 7f3733fff640 1 --2- 172.21.15.204:0/3295953297 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3728039060 0x7f372803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:22.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.199+0000 7f3733fff640 1 --2- 172.21.15.204:0/3295953297 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3728039060 0x7f372803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f37240098c0 tx=0x7f3724006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:22.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.203+0000 7f37409c6640 1 -- 172.21.15.204:0/3295953297 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f372c0433b0 con 0x7f373c100530 2024-09-11T22:44:22.395 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.393+0000 7f37419c8640 1 -- 172.21.15.204:0/3295953297 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi152:172.21.15.152=a;smithi191:172.21.15.191=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}) -- 0x7f373c02d9b0 con 0x7f3728039060 2024-09-11T22:44:22.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.398+0000 7f37409c6640 1 -- 172.21.15.204:0/3295953297 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f373c02d9b0 con 0x7f3728039060 2024-09-11T22:44:22.400 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mon update... 2024-09-11T22:44:22.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.400+0000 7f37419c8640 1 -- 172.21.15.204:0/3295953297 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3728039060 msgr2=0x7f372803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:22.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.400+0000 7f37419c8640 1 --2- 172.21.15.204:0/3295953297 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3728039060 0x7f372803b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f37240098c0 tx=0x7f3724006db0 comp rx=0 tx=0).stop 2024-09-11T22:44:22.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.400+0000 7f37419c8640 1 -- 172.21.15.204:0/3295953297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f373c100530 msgr2=0x7f373c06a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:22.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.400+0000 7f37419c8640 1 --2- 172.21.15.204:0/3295953297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f373c100530 0x7f373c06a140 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f372c004f60 tx=0x7f372c0048f0 comp rx=0 tx=0).stop 2024-09-11T22:44:22.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.400+0000 7f37419c8640 1 -- 172.21.15.204:0/3295953297 shutdown_connections 2024-09-11T22:44:22.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.400+0000 7f37419c8640 1 --2- 172.21.15.204:0/3295953297 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3728039060 0x7f372803b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:22.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.400+0000 7f37419c8640 1 --2- 172.21.15.204:0/3295953297 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f373c100530 0x7f373c06a140 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:22.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.400+0000 7f37419c8640 1 -- 172.21.15.204:0/3295953297 >> 172.21.15.204:0/3295953297 conn(0x7f373c0fbca0 msgr2=0x7f373c0fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:22.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.400+0000 7f37419c8640 1 -- 172.21.15.204:0/3295953297 shutdown_connections 2024-09-11T22:44:22.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:22.400+0000 7f37419c8640 1 -- 172.21.15.204:0/3295953297 wait complete. 2024-09-11T22:44:22.982 DEBUG:teuthology.orchestra.run.smithi191:mon.b> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T22:44:22.984 DEBUG:teuthology.orchestra.run.smithi204:mon.c> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T22:44:22.987 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T22:44:22.987 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T22:44:23.117 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:22 smithi152 ceph-mon[31181]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-11T22:44:23.117 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:22 smithi152 ceph-mon[31181]: Updating smithi204:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:44:23.117 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:22 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:23.117 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:22 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:23.117 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:22 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:23.117 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:22 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:23.117 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:22 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:23.117 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:22 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:23.117 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:22 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:23.117 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:22 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:23.118 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:22 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:44:23.118 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:22 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:44:23.118 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:22 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:23.256 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:44:23.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.686+0000 7f07fd400640 1 -- 172.21.15.204:0/850820414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8107660 msgr2=0x7f07f8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:23.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.686+0000 7f07fd400640 1 --2- 172.21.15.204:0/850820414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8107660 0x7f07f8107a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009880 tx=0x7f07e002a5d0 comp rx=0 tx=0).stop 2024-09-11T22:44:23.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.686+0000 7f07fd400640 1 -- 172.21.15.204:0/850820414 shutdown_connections 2024-09-11T22:44:23.688 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.686+0000 7f07fd400640 1 --2- 172.21.15.204:0/850820414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8107660 0x7f07f8107a40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:23.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.686+0000 7f07fd400640 1 -- 172.21.15.204:0/850820414 >> 172.21.15.204:0/850820414 conn(0x7f07f80fd2b0 msgr2=0x7f07f80ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:23.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.686+0000 7f07fd400640 1 -- 172.21.15.204:0/850820414 shutdown_connections 2024-09-11T22:44:23.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.686+0000 7f07fd400640 1 -- 172.21.15.204:0/850820414 wait complete. 2024-09-11T22:44:23.690 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.687+0000 7f07fd400640 1 Processor -- start 2024-09-11T22:44:23.690 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.687+0000 7f07fd400640 1 -- start start 2024-09-11T22:44:23.690 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.687+0000 7f07fd400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8107660 0x7f07f81970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:23.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.687+0000 7f07fd400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f07f810bc70 con 0x7f07f8107660 2024-09-11T22:44:23.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.688+0000 7f07f6ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8107660 0x7f07f81970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:23.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.688+0000 7f07f6ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8107660 0x7f07f81970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.204:59382/0 (socket says 172.21.15.204:59382) 2024-09-11T22:44:23.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.688+0000 7f07f6ffd640 1 -- 172.21.15.204:0/3081904542 learned_addr learned my addr 172.21.15.204:0/3081904542 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-11T22:44:23.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.688+0000 7f07f6ffd640 1 -- 172.21.15.204:0/3081904542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f81975e0 con 0x7f07f8107660 2024-09-11T22:44:23.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.688+0000 7f07f6ffd640 1 --2- 172.21.15.204:0/3081904542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8107660 0x7f07f81970a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009230 tx=0x7f07e0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:23.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.688+0000 7f07effff640 1 -- 172.21.15.204:0/3081904542 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07e003f070 con 0x7f07f8107660 2024-09-11T22:44:23.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.688+0000 7f07fd400640 1 -- 172.21.15.204:0/3081904542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f81978d0 con 0x7f07f8107660 2024-09-11T22:44:23.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.688+0000 7f07effff640 1 -- 172.21.15.204:0/3081904542 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07e003bc80 con 0x7f07f8107660 2024-09-11T22:44:23.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.688+0000 7f07effff640 1 -- 172.21.15.204:0/3081904542 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07e0035030 con 0x7f07f8107660 2024-09-11T22:44:23.691 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.688+0000 7f07fd400640 1 -- 172.21.15.204:0/3081904542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f819b970 con 0x7f07f8107660 2024-09-11T22:44:23.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.689+0000 7f07effff640 1 -- 172.21.15.204:0/3081904542 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f07e0042070 con 0x7f07f8107660 2024-09-11T22:44:23.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.689+0000 7f07fd400640 1 -- 172.21.15.204:0/3081904542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f8102d10 con 0x7f07f8107660 2024-09-11T22:44:23.692 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.689+0000 7f07effff640 1 --2- 172.21.15.204:0/3081904542 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f07e4039060 0x7f07e403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:23.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.689+0000 7f07effff640 1 -- 172.21.15.204:0/3081904542 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f07e006cf30 con 0x7f07f8107660 2024-09-11T22:44:23.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.690+0000 7f07f67fc640 1 --2- 172.21.15.204:0/3081904542 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f07e4039060 0x7f07e403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:23.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.690+0000 7f07f67fc640 1 --2- 172.21.15.204:0/3081904542 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f07e4039060 0x7f07e403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f07e80098c0 tx=0x7f07e8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:23.696 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.695+0000 7f07effff640 1 -- 172.21.15.204:0/3081904542 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f07e004c350 con 0x7f07f8107660 2024-09-11T22:44:23.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.927+0000 7f07fd400640 1 -- 172.21.15.204:0/3081904542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f07f8107af0 con 0x7f07f8107660 2024-09-11T22:44:23.929 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.928+0000 7f07effff640 1 -- 172.21.15.204:0/3081904542 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f07f8107af0 con 0x7f07f8107660 2024-09-11T22:44:23.930 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:44:23.930 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":1,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","modified":"2024-09-11T22:42:57.315021Z","created":"2024-09-11T22:42:57.315021Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T22:44:23.930 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 1 2024-09-11T22:44:23.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.930+0000 7f07fd400640 1 -- 172.21.15.204:0/3081904542 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f07e4039060 msgr2=0x7f07e403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:23.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.930+0000 7f07fd400640 1 --2- 172.21.15.204:0/3081904542 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f07e4039060 0x7f07e403b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f07e80098c0 tx=0x7f07e8006db0 comp rx=0 tx=0).stop 2024-09-11T22:44:23.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.930+0000 7f07fd400640 1 -- 172.21.15.204:0/3081904542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8107660 msgr2=0x7f07f81970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:23.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.930+0000 7f07fd400640 1 --2- 172.21.15.204:0/3081904542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8107660 0x7f07f81970a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009230 tx=0x7f07e0003d40 comp rx=0 tx=0).stop 2024-09-11T22:44:23.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.930+0000 7f07fd400640 1 -- 172.21.15.204:0/3081904542 shutdown_connections 2024-09-11T22:44:23.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.930+0000 7f07fd400640 1 --2- 172.21.15.204:0/3081904542 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f07e4039060 0x7f07e403b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:23.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.930+0000 7f07fd400640 1 --2- 172.21.15.204:0/3081904542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8107660 0x7f07f81970a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:23.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.930+0000 7f07fd400640 1 -- 172.21.15.204:0/3081904542 >> 172.21.15.204:0/3081904542 conn(0x7f07f80fd2b0 msgr2=0x7f07f80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:23.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.930+0000 7f07fd400640 1 -- 172.21.15.204:0/3081904542 shutdown_connections 2024-09-11T22:44:23.932 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:23.930+0000 7f07fd400640 1 -- 172.21.15.204:0/3081904542 wait complete. 2024-09-11T22:44:24.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:23 smithi152 ceph-mon[31181]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:44:24.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:23 smithi152 ceph-mon[31181]: Updating smithi204:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:44:24.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:23 smithi152 ceph-mon[31181]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi152:172.21.15.152=a;smithi191:172.21.15.191=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:44:24.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:23 smithi152 ceph-mon[31181]: Saving service mon spec with placement smithi152:172.21.15.152=a;smithi191:172.21.15.191=b;smithi204:172.21.15.204=c;count:3 2024-09-11T22:44:24.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:23 smithi152 ceph-mon[31181]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T22:44:24.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:23 smithi152 ceph-mon[31181]: Reconfiguring daemon mon.a on smithi152 2024-09-11T22:44:24.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:23 smithi152 ceph-mon[31181]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:25.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:24 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:25.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:24 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:25.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:24 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:44:25.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:24 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:25.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:24 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:44:25.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:24 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:25.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:24 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:44:25.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:24 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:25.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:24 smithi152 ceph-mon[31181]: Deploying daemon mon.c on smithi204 2024-09-11T22:44:25.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:24 smithi152 ceph-mon[31181]: from='client.? 172.21.15.204:0/3081904542' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:44:25.488 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T22:44:25.489 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T22:44:26.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:25 smithi152 ceph-mon[31181]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:26.124 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:44:27.044 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.018+0000 7fdd169d1640 1 -- 172.21.15.204:0/435264574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd10100990 msgr2=0x7fdd10100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:27.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.018+0000 7fdd169d1640 1 --2- 172.21.15.204:0/435264574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd10100990 0x7fdd10100d70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009880 tx=0x7fdd0002a5d0 comp rx=0 tx=0).stop 2024-09-11T22:44:27.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.018+0000 7fdd169d1640 1 -- 172.21.15.204:0/435264574 shutdown_connections 2024-09-11T22:44:27.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.018+0000 7fdd169d1640 1 --2- 172.21.15.204:0/435264574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd10100990 0x7fdd10100d70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:27.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.018+0000 7fdd169d1640 1 -- 172.21.15.204:0/435264574 >> 172.21.15.204:0/435264574 conn(0x7fdd10076250 msgr2=0x7fdd10076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:27.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.018+0000 7fdd169d1640 1 -- 172.21.15.204:0/435264574 shutdown_connections 2024-09-11T22:44:27.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.018+0000 7fdd169d1640 1 -- 172.21.15.204:0/435264574 wait complete. 2024-09-11T22:44:27.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.019+0000 7fdd169d1640 1 Processor -- start 2024-09-11T22:44:27.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.019+0000 7fdd169d1640 1 -- start start 2024-09-11T22:44:27.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.019+0000 7fdd169d1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd10100990 0x7fdd101970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:27.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.019+0000 7fdd169d1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1010ab90 con 0x7fdd10100990 2024-09-11T22:44:27.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.019+0000 7fdd0ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd10100990 0x7fdd101970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:27.045 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.019+0000 7fdd0ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd10100990 0x7fdd101970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.204:59410/0 (socket says 172.21.15.204:59410) 2024-09-11T22:44:27.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.019+0000 7fdd0ffff640 1 -- 172.21.15.204:0/308640541 learned_addr learned my addr 172.21.15.204:0/308640541 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-11T22:44:27.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.020+0000 7fdd0ffff640 1 -- 172.21.15.204:0/308640541 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd10197620 con 0x7fdd10100990 2024-09-11T22:44:27.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.020+0000 7fdd0ffff640 1 --2- 172.21.15.204:0/308640541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd10100990 0x7fdd101970e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009230 tx=0x7fdd00003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:27.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.020+0000 7fdd0d7fa640 1 -- 172.21.15.204:0/308640541 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd0003f070 con 0x7fdd10100990 2024-09-11T22:44:27.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.020+0000 7fdd0d7fa640 1 -- 172.21.15.204:0/308640541 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd0003bc80 con 0x7fdd10100990 2024-09-11T22:44:27.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.020+0000 7fdd169d1640 1 -- 172.21.15.204:0/308640541 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd10197910 con 0x7fdd10100990 2024-09-11T22:44:27.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.020+0000 7fdd0d7fa640 1 -- 172.21.15.204:0/308640541 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd00035030 con 0x7fdd10100990 2024-09-11T22:44:27.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.020+0000 7fdd169d1640 1 -- 172.21.15.204:0/308640541 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd1019b9b0 con 0x7fdd10100990 2024-09-11T22:44:27.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.021+0000 7fdd0d7fa640 1 -- 172.21.15.204:0/308640541 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fdd00042070 con 0x7fdd10100990 2024-09-11T22:44:27.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.021+0000 7fdd169d1640 1 -- 172.21.15.204:0/308640541 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd10101c40 con 0x7fdd10100990 2024-09-11T22:44:27.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.021+0000 7fdd0d7fa640 1 --2- 172.21.15.204:0/308640541 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fdcf8039060 0x7fdcf803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:27.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.021+0000 7fdd0d7fa640 1 -- 172.21.15.204:0/308640541 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdd0006cd40 con 0x7fdd10100990 2024-09-11T22:44:27.046 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.022+0000 7fdd0f7fe640 1 --2- 172.21.15.204:0/308640541 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fdcf8039060 0x7fdcf803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:27.047 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.022+0000 7fdd0f7fe640 1 --2- 172.21.15.204:0/308640541 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fdcf8039060 0x7fdcf803b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdd040098c0 tx=0x7fdd04006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:27.047 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.026+0000 7fdd0d7fa640 1 -- 172.21.15.204:0/308640541 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdd0004c350 con 0x7fdd10100990 2024-09-11T22:44:27.273 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.272+0000 7fdd169d1640 1 -- 172.21.15.204:0/308640541 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fdd10198440 con 0x7fdd10100990 2024-09-11T22:44:27.273 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.272+0000 7fdd0d7fa640 1 -- 172.21.15.204:0/308640541 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fdd000314c0 con 0x7fdd10100990 2024-09-11T22:44:27.274 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:44:27.274 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":1,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","modified":"2024-09-11T22:42:57.315021Z","created":"2024-09-11T22:42:57.315021Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T22:44:27.274 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 1 2024-09-11T22:44:27.275 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.274+0000 7fdd169d1640 1 -- 172.21.15.204:0/308640541 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fdcf8039060 msgr2=0x7fdcf803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:27.275 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.274+0000 7fdd169d1640 1 --2- 172.21.15.204:0/308640541 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fdcf8039060 0x7fdcf803b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdd040098c0 tx=0x7fdd04006db0 comp rx=0 tx=0).stop 2024-09-11T22:44:27.275 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.274+0000 7fdd169d1640 1 -- 172.21.15.204:0/308640541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd10100990 msgr2=0x7fdd101970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:27.275 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.274+0000 7fdd169d1640 1 --2- 172.21.15.204:0/308640541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd10100990 0x7fdd101970e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009230 tx=0x7fdd00003d40 comp rx=0 tx=0).stop 2024-09-11T22:44:27.276 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.275+0000 7fdd169d1640 1 -- 172.21.15.204:0/308640541 shutdown_connections 2024-09-11T22:44:27.276 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.275+0000 7fdd169d1640 1 --2- 172.21.15.204:0/308640541 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fdcf8039060 0x7fdcf803b520 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:27.276 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.275+0000 7fdd169d1640 1 --2- 172.21.15.204:0/308640541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd10100990 0x7fdd101970e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:27.276 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.275+0000 7fdd169d1640 1 -- 172.21.15.204:0/308640541 >> 172.21.15.204:0/308640541 conn(0x7fdd10076250 msgr2=0x7fdd100696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:27.276 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.275+0000 7fdd169d1640 1 -- 172.21.15.204:0/308640541 shutdown_connections 2024-09-11T22:44:27.276 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:27.275+0000 7fdd169d1640 1 -- 172.21.15.204:0/308640541 wait complete. 2024-09-11T22:44:28.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:27 smithi152 ceph-mon[31181]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:28.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:27 smithi152 ceph-mon[31181]: from='client.? 172.21.15.204:0/308640541' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:44:28.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:28 smithi204 systemd[1]: Started Ceph mon.c for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:44:28.915 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T22:44:28.915 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T22:44:29.185 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:44:33.651 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:44:33.651 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:33.651 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: mon.a calling monitor election 2024-09-11T22:44:33.652 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:33.652 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:33.652 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:33.652 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: mon.c calling monitor election 2024-09-11T22:44:33.652 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:33.652 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:33.652 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:33.652 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:33.652 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:33.652 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T22:44:33.652 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: monmap epoch 2 2024-09-11T22:44:33.652 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:44:33.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: last_changed 2024-09-11T22:44:28.323756+0000 2024-09-11T22:44:33.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: created 2024-09-11T22:42:57.315021+0000 2024-09-11T22:44:33.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: min_mon_release 19 (squid) 2024-09-11T22:44:33.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: election_strategy: 1 2024-09-11T22:44:33.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: 0: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.a 2024-09-11T22:44:33.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-11T22:44:33.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: fsmap 2024-09-11T22:44:33.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:44:33.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: mgrmap e13: x(active, since 38s) 2024-09-11T22:44:33.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: overall HEALTH_OK 2024-09-11T22:44:33.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:33.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:33.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:33.653 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:44:33.654 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:33 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:33.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:44:33.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:33.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: mon.a calling monitor election 2024-09-11T22:44:33.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:33.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:33.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:33.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: mon.c calling monitor election 2024-09-11T22:44:33.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:33.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:33.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:33.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:33.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:33.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T22:44:33.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: monmap epoch 2 2024-09-11T22:44:33.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:44:33.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: last_changed 2024-09-11T22:44:28.323756+0000 2024-09-11T22:44:33.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: created 2024-09-11T22:42:57.315021+0000 2024-09-11T22:44:33.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: min_mon_release 19 (squid) 2024-09-11T22:44:33.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: election_strategy: 1 2024-09-11T22:44:33.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: 0: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.a 2024-09-11T22:44:33.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-11T22:44:33.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: fsmap 2024-09-11T22:44:33.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:44:33.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: mgrmap e13: x(active, since 38s) 2024-09-11T22:44:33.875 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: overall HEALTH_OK 2024-09-11T22:44:33.875 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:33.875 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:33.875 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:33.875 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:44:33.875 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:33 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:34.092 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.090+0000 7fcc11a02640 1 -- 172.21.15.204:0/4176925756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbf8003530 msgr2=0x7fcbf80059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:34.092 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.090+0000 7fcc11a02640 1 --2- 172.21.15.204:0/4176925756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbf8003530 0x7fcbf80059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c1016a0 tx=0x7fcbfc02c490 comp rx=0 tx=0).stop 2024-09-11T22:44:34.092 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.091+0000 7fcc11a02640 1 -- 172.21.15.204:0/4176925756 shutdown_connections 2024-09-11T22:44:34.092 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.091+0000 7fcc11a02640 1 --2- 172.21.15.204:0/4176925756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbf8003530 0x7fcbf80059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:34.092 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.091+0000 7fcc11a02640 1 -- 172.21.15.204:0/4176925756 >> 172.21.15.204:0/4176925756 conn(0x7fcc0c0fc220 msgr2=0x7fcc0c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:34.093 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.091+0000 7fcc11a02640 1 -- 172.21.15.204:0/4176925756 shutdown_connections 2024-09-11T22:44:34.093 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.091+0000 7fcc11a02640 1 -- 172.21.15.204:0/4176925756 wait complete. 2024-09-11T22:44:34.093 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.091+0000 7fcc11a02640 1 Processor -- start 2024-09-11T22:44:34.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.092+0000 7fcc11a02640 1 -- start start 2024-09-11T22:44:34.094 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.092+0000 7fcc11a02640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbf8003530 0x7fcc0c1021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:34.095 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.092+0000 7fcc11a02640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c102730 0x7fcc0c102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:34.095 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.092+0000 7fcc11a02640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c108670 con 0x7fcbf8003530 2024-09-11T22:44:34.095 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.092+0000 7fcc11a02640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c1084f0 con 0x7fcc0c102730 2024-09-11T22:44:34.095 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.092+0000 7fcc03fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c102730 0x7fcc0c102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:34.095 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.092+0000 7fcc0affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbf8003530 0x7fcc0c1021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:34.095 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.092+0000 7fcc0affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbf8003530 0x7fcc0c1021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.204:39734/0 (socket says 172.21.15.204:39734) 2024-09-11T22:44:34.095 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.092+0000 7fcc03fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c102730 0x7fcc0c102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:55870/0 (socket says 172.21.15.204:55870) 2024-09-11T22:44:34.095 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.093+0000 7fcc03fff640 1 -- 172.21.15.204:0/3488998334 learned_addr learned my addr 172.21.15.204:0/3488998334 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-11T22:44:34.095 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.093+0000 7fcc03fff640 1 -- 172.21.15.204:0/3488998334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c102730 msgr2=0x7fcc0c102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T22:44:34.095 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.093+0000 7fcc03fff640 1 -- 172.21.15.204:0/3488998334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c102730 msgr2=0x7fcc0c102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T22:44:34.095 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.093+0000 7fcc03fff640 1 --2- 172.21.15.204:0/3488998334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c102730 0x7fcc0c102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T22:44:34.095 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.093+0000 7fcc03fff640 1 --2- 172.21.15.204:0/3488998334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c102730 0x7fcc0c102b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:44:34.096 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.093+0000 7fcc0affd640 1 -- 172.21.15.204:0/3488998334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c102730 msgr2=0x7fcc0c102b90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:44:34.096 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.093+0000 7fcc0affd640 1 --2- 172.21.15.204:0/3488998334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c102730 0x7fcc0c102b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:34.096 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.093+0000 7fcc0affd640 1 -- 172.21.15.204:0/3488998334 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c1040e0 con 0x7fcbf8003530 2024-09-11T22:44:34.096 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.093+0000 7fcc0affd640 1 --2- 172.21.15.204:0/3488998334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbf8003530 0x7fcc0c1021f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc036040 tx=0x7fcbfc033680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:34.096 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.094+0000 7fcc10a00640 1 -- 172.21.15.204:0/3488998334 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fcbfc032920 con 0x7fcbf8003530 2024-09-11T22:44:34.096 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.094+0000 7fcc10a00640 1 -- 172.21.15.204:0/3488998334 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbfc041070 con 0x7fcbf8003530 2024-09-11T22:44:34.096 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.094+0000 7fcc11a02640 1 -- 172.21.15.204:0/3488998334 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0c1a34c0 con 0x7fcbf8003530 2024-09-11T22:44:34.096 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.094+0000 7fcc10a00640 1 -- 172.21.15.204:0/3488998334 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fcbfc004970 con 0x7fcbf8003530 2024-09-11T22:44:34.096 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.094+0000 7fcc11a02640 1 -- 172.21.15.204:0/3488998334 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc0c1a3a50 con 0x7fcbf8003530 2024-09-11T22:44:34.097 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.095+0000 7fcc10a00640 1 -- 172.21.15.204:0/3488998334 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fcbfc004c20 con 0x7fcbf8003530 2024-09-11T22:44:34.097 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.095+0000 7fcc10a00640 1 --2- 172.21.15.204:0/3488998334 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcbec038e50 0x7fcbec03b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:34.097 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.095+0000 7fcc11a02640 1 -- 172.21.15.204:0/3488998334 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc0c101ca0 con 0x7fcbf8003530 2024-09-11T22:44:34.097 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.095+0000 7fcc10a00640 1 -- 172.21.15.204:0/3488998334 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcbfc06e2c0 con 0x7fcbf8003530 2024-09-11T22:44:34.097 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.095+0000 7fcc03fff640 1 --2- 172.21.15.204:0/3488998334 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcbec038e50 0x7fcbec03b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:34.097 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.096+0000 7fcc03fff640 1 --2- 172.21.15.204:0/3488998334 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcbec038e50 0x7fcbec03b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009100 tx=0x7fcbf4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:34.101 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.100+0000 7fcc10a00640 1 -- 172.21.15.204:0/3488998334 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcbfc03a550 con 0x7fcbf8003530 2024-09-11T22:44:34.335 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.334+0000 7fcc11a02640 1 -- 172.21.15.204:0/3488998334 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fcc0c1a3d30 con 0x7fcbf8003530 2024-09-11T22:44:34.335 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.334+0000 7fcc10a00640 1 -- 172.21.15.204:0/3488998334 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fcbfc03a2f0 con 0x7fcbf8003530 2024-09-11T22:44:34.336 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:44:34.336 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","modified":"2024-09-11T22:44:28.323756Z","created":"2024-09-11T22:42:57.315021Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T22:44:34.336 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-09-11T22:44:34.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.336+0000 7fcc11a02640 1 -- 172.21.15.204:0/3488998334 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcbec038e50 msgr2=0x7fcbec03b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:34.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.336+0000 7fcc11a02640 1 --2- 172.21.15.204:0/3488998334 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcbec038e50 0x7fcbec03b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009100 tx=0x7fcbf4008040 comp rx=0 tx=0).stop 2024-09-11T22:44:34.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.336+0000 7fcc11a02640 1 -- 172.21.15.204:0/3488998334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbf8003530 msgr2=0x7fcc0c1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:34.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.336+0000 7fcc11a02640 1 --2- 172.21.15.204:0/3488998334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbf8003530 0x7fcc0c1021f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc036040 tx=0x7fcbfc033680 comp rx=0 tx=0).stop 2024-09-11T22:44:34.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.336+0000 7fcc11a02640 1 -- 172.21.15.204:0/3488998334 shutdown_connections 2024-09-11T22:44:34.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.337+0000 7fcc11a02640 1 --2- 172.21.15.204:0/3488998334 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcbec038e50 0x7fcbec03b310 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:34.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.337+0000 7fcc11a02640 1 --2- 172.21.15.204:0/3488998334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c102730 0x7fcc0c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:34.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.337+0000 7fcc11a02640 1 --2- 172.21.15.204:0/3488998334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbf8003530 0x7fcc0c1021f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:34.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.337+0000 7fcc11a02640 1 -- 172.21.15.204:0/3488998334 >> 172.21.15.204:0/3488998334 conn(0x7fcc0c0fc220 msgr2=0x7fcc0c06a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:34.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.337+0000 7fcc11a02640 1 -- 172.21.15.204:0/3488998334 shutdown_connections 2024-09-11T22:44:34.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:34.337+0000 7fcc11a02640 1 -- 172.21.15.204:0/3488998334 wait complete. 2024-09-11T22:44:34.616 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:34 smithi204 ceph-mon[35454]: Deploying daemon mon.b on smithi191 2024-09-11T22:44:34.617 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:34.617 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:34 smithi204 ceph-mon[35454]: from='client.? 172.21.15.204:0/3488998334' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:44:34.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:34 smithi152 ceph-mon[31181]: Deploying daemon mon.b on smithi191 2024-09-11T22:44:34.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:34.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:34 smithi152 ceph-mon[31181]: from='client.? 172.21.15.204:0/3488998334' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:44:35.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:35 smithi204 ceph-mon[35454]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:35.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:35 smithi152 ceph-mon[31181]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:35.921 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T22:44:35.921 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T22:44:36.175 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:44:36.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.599+0000 7fe33c2e9640 1 -- 172.21.15.204:0/1428178820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3341028e0 msgr2=0x7fe334102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:36.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.599+0000 7fe33c2e9640 1 --2- 172.21.15.204:0/1428178820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3341028e0 0x7fe334102cc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe330009880 tx=0x7fe33002a670 comp rx=0 tx=0).stop 2024-09-11T22:44:36.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.600+0000 7fe33c2e9640 1 -- 172.21.15.204:0/1428178820 shutdown_connections 2024-09-11T22:44:36.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.600+0000 7fe33c2e9640 1 --2- 172.21.15.204:0/1428178820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3341028e0 0x7fe334102cc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:36.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.600+0000 7fe33c2e9640 1 -- 172.21.15.204:0/1428178820 >> 172.21.15.204:0/1428178820 conn(0x7fe3340fc220 msgr2=0x7fe3340fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:36.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.600+0000 7fe33c2e9640 1 -- 172.21.15.204:0/1428178820 shutdown_connections 2024-09-11T22:44:36.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.600+0000 7fe33c2e9640 1 -- 172.21.15.204:0/1428178820 wait complete. 2024-09-11T22:44:36.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.600+0000 7fe33c2e9640 1 Processor -- start 2024-09-11T22:44:36.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.600+0000 7fe33c2e9640 1 -- start start 2024-09-11T22:44:36.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.600+0000 7fe33c2e9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3341028e0 0x7fe334197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:36.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.600+0000 7fe33c2e9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3341979d0 0x7fe33419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:36.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.600+0000 7fe33c2e9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe33410bd70 con 0x7fe3341028e0 2024-09-11T22:44:36.630 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.600+0000 7fe33c2e9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe33410bbf0 con 0x7fe3341979d0 2024-09-11T22:44:36.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe339884640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3341979d0 0x7fe33419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:36.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe33a085640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3341028e0 0x7fe334197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:36.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe339884640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3341979d0 0x7fe33419bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:55884/0 (socket says 172.21.15.204:55884) 2024-09-11T22:44:36.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe33a085640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3341028e0 0x7fe334197490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.204:39744/0 (socket says 172.21.15.204:39744) 2024-09-11T22:44:36.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe339884640 1 -- 172.21.15.204:0/3565243234 learned_addr learned my addr 172.21.15.204:0/3565243234 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-11T22:44:36.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe339884640 1 -- 172.21.15.204:0/3565243234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3341979d0 msgr2=0x7fe33419bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T22:44:36.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe339884640 1 -- 172.21.15.204:0/3565243234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3341979d0 msgr2=0x7fe33419bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T22:44:36.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe339884640 1 --2- 172.21.15.204:0/3565243234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3341979d0 0x7fe33419bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T22:44:36.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe339884640 1 --2- 172.21.15.204:0/3565243234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3341979d0 0x7fe33419bd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:44:36.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe33a085640 1 -- 172.21.15.204:0/3565243234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3341979d0 msgr2=0x7fe33419bd60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:44:36.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe33a085640 1 --2- 172.21.15.204:0/3565243234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3341979d0 0x7fe33419bd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:36.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe33a085640 1 -- 172.21.15.204:0/3565243234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe334197ff0 con 0x7fe3341028e0 2024-09-11T22:44:36.631 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe33a085640 1 --2- 172.21.15.204:0/3565243234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3341028e0 0x7fe334197490 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe330005ab0 tx=0x7fe330032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:36.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe32b7fe640 1 -- 172.21.15.204:0/3565243234 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe330035070 con 0x7fe3341028e0 2024-09-11T22:44:36.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe32b7fe640 1 -- 172.21.15.204:0/3565243234 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe330032e40 con 0x7fe3341028e0 2024-09-11T22:44:36.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe33c2e9640 1 -- 172.21.15.204:0/3565243234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33419c2a0 con 0x7fe3341028e0 2024-09-11T22:44:36.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe32b7fe640 1 -- 172.21.15.204:0/3565243234 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe33003bc10 con 0x7fe3341028e0 2024-09-11T22:44:36.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.601+0000 7fe33c2e9640 1 -- 172.21.15.204:0/3565243234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33419c780 con 0x7fe3341028e0 2024-09-11T22:44:36.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.602+0000 7fe33c2e9640 1 -- 172.21.15.204:0/3565243234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe33406ac50 con 0x7fe3341028e0 2024-09-11T22:44:36.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.602+0000 7fe32b7fe640 1 -- 172.21.15.204:0/3565243234 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fe330042030 con 0x7fe3341028e0 2024-09-11T22:44:36.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.602+0000 7fe32b7fe640 1 --2- 172.21.15.204:0/3565243234 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe320038e00 0x7fe32003b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:36.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.602+0000 7fe32b7fe640 1 -- 172.21.15.204:0/3565243234 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe330040070 con 0x7fe3341028e0 2024-09-11T22:44:36.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.603+0000 7fe339884640 1 --2- 172.21.15.204:0/3565243234 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe320038e00 0x7fe32003b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:36.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.603+0000 7fe339884640 1 --2- 172.21.15.204:0/3565243234 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe320038e00 0x7fe32003b2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe324009100 tx=0x7fe324008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:36.632 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.608+0000 7fe32b7fe640 1 -- 172.21.15.204:0/3565243234 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe33004a260 con 0x7fe3341028e0 2024-09-11T22:44:36.844 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.843+0000 7fe33c2e9640 1 -- 172.21.15.204:0/3565243234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe33419ca30 con 0x7fe3341028e0 2024-09-11T22:44:36.845 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.844+0000 7fe32b7fe640 1 -- 172.21.15.204:0/3565243234 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fe33003bdb0 con 0x7fe3341028e0 2024-09-11T22:44:36.845 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:44:36.845 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","modified":"2024-09-11T22:44:28.323756Z","created":"2024-09-11T22:42:57.315021Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T22:44:36.845 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-09-11T22:44:36.847 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.846+0000 7fe33c2e9640 1 -- 172.21.15.204:0/3565243234 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe320038e00 msgr2=0x7fe32003b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:36.847 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.846+0000 7fe33c2e9640 1 --2- 172.21.15.204:0/3565243234 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe320038e00 0x7fe32003b2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe324009100 tx=0x7fe324008040 comp rx=0 tx=0).stop 2024-09-11T22:44:36.847 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.846+0000 7fe33c2e9640 1 -- 172.21.15.204:0/3565243234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3341028e0 msgr2=0x7fe334197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:36.847 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.846+0000 7fe33c2e9640 1 --2- 172.21.15.204:0/3565243234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3341028e0 0x7fe334197490 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe330005ab0 tx=0x7fe330032c50 comp rx=0 tx=0).stop 2024-09-11T22:44:36.847 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.846+0000 7fe33c2e9640 1 -- 172.21.15.204:0/3565243234 shutdown_connections 2024-09-11T22:44:36.847 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.846+0000 7fe33c2e9640 1 --2- 172.21.15.204:0/3565243234 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe320038e00 0x7fe32003b2c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:36.848 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.846+0000 7fe33c2e9640 1 --2- 172.21.15.204:0/3565243234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3341979d0 0x7fe33419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:36.848 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.846+0000 7fe33c2e9640 1 --2- 172.21.15.204:0/3565243234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3341028e0 0x7fe334197490 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:36.848 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.846+0000 7fe33c2e9640 1 -- 172.21.15.204:0/3565243234 >> 172.21.15.204:0/3565243234 conn(0x7fe3340fc220 msgr2=0x7fe3341056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:36.848 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.846+0000 7fe33c2e9640 1 -- 172.21.15.204:0/3565243234 shutdown_connections 2024-09-11T22:44:36.848 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:36.846+0000 7fe33c2e9640 1 -- 172.21.15.204:0/3565243234 wait complete. 2024-09-11T22:44:37.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:37 smithi191 systemd[1]: Started Ceph mon.b for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:44:38.443 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T22:44:38.443 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T22:44:38.689 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:44:42.126 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.125+0000 7f7d3320b640 1 -- 172.21.15.204:0/244481618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d1c003530 msgr2=0x7f7d1c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:42.126 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.125+0000 7f7d3320b640 1 --2- 172.21.15.204:0/244481618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d1c003530 0x7f7d1c0059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c102af0 tx=0x7f7d2002bce0 comp rx=0 tx=0).stop 2024-09-11T22:44:42.126 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.125+0000 7f7d3320b640 1 -- 172.21.15.204:0/244481618 shutdown_connections 2024-09-11T22:44:42.126 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.125+0000 7f7d3320b640 1 --2- 172.21.15.204:0/244481618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d1c003530 0x7f7d1c0059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:42.126 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.125+0000 7f7d3320b640 1 -- 172.21.15.204:0/244481618 >> 172.21.15.204:0/244481618 conn(0x7f7d2c0fd320 msgr2=0x7f7d2c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:42.126 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.125+0000 7f7d3320b640 1 -- 172.21.15.204:0/244481618 shutdown_connections 2024-09-11T22:44:42.127 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.125+0000 7f7d3320b640 1 -- 172.21.15.204:0/244481618 wait complete. 2024-09-11T22:44:42.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.126+0000 7f7d3320b640 1 Processor -- start 2024-09-11T22:44:42.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.126+0000 7f7d3320b640 1 -- start start 2024-09-11T22:44:42.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.126+0000 7f7d3320b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d1c003530 0x7f7d2c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:42.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.126+0000 7f7d3320b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c199ae0 0x7f7d2c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:42.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.126+0000 7f7d3320b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d2c19e3b0 0x7f7d2c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:42.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.126+0000 7f7d3320b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c109720 con 0x7f7d1c003530 2024-09-11T22:44:42.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.126+0000 7f7d3320b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1095a0 con 0x7f7d2c19e3b0 2024-09-11T22:44:42.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.126+0000 7f7d3320b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1098a0 con 0x7f7d2c199ae0 2024-09-11T22:44:42.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d30fa7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d1c003530 0x7f7d2c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:42.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d2bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c199ae0 0x7f7d2c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:42.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d30fa7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d1c003530 0x7f7d2c1995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.204:38712/0 (socket says 172.21.15.204:38712) 2024-09-11T22:44:42.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d30fa7640 1 -- 172.21.15.204:0/2016927683 learned_addr learned my addr 172.21.15.204:0/2016927683 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-11T22:44:42.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d2bfff640 1 -- 172.21.15.204:0/2016927683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c199ae0 msgr2=0x7f7d2c19de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T22:44:42.129 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d2bfff640 1 -- 172.21.15.204:0/2016927683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c199ae0 msgr2=0x7f7d2c19de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T22:44:42.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d2bfff640 1 --2- 172.21.15.204:0/2016927683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c199ae0 0x7f7d2c19de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T22:44:42.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d2bfff640 1 --2- 172.21.15.204:0/2016927683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c199ae0 0x7f7d2c19de70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:44:42.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d317a8640 1 --2- 172.21.15.204:0/2016927683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d2c19e3b0 0x7f7d2c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:42.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d30fa7640 1 -- 172.21.15.204:0/2016927683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c199ae0 msgr2=0x7f7d2c19de70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:44:42.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d30fa7640 1 --2- 172.21.15.204:0/2016927683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c199ae0 0x7f7d2c19de70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:42.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d30fa7640 1 -- 172.21.15.204:0/2016927683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d2c19e3b0 msgr2=0x7f7d2c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:42.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d30fa7640 1 --2- 172.21.15.204:0/2016927683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d2c19e3b0 0x7f7d2c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:42.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d30fa7640 1 -- 172.21.15.204:0/2016927683 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2c1a6120 con 0x7f7d1c003530 2024-09-11T22:44:42.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d30fa7640 1 --2- 172.21.15.204:0/2016927683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d1c003530 0x7f7d2c1995a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7d2002c2d0 tx=0x7f7d20005430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:42.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d29ffb640 1 -- 172.21.15.204:0/2016927683 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2002cc50 con 0x7f7d1c003530 2024-09-11T22:44:42.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d29ffb640 1 -- 172.21.15.204:0/2016927683 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2002cdf0 con 0x7f7d1c003530 2024-09-11T22:44:42.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d3320b640 1 -- 172.21.15.204:0/2016927683 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2c1a62f0 con 0x7f7d1c003530 2024-09-11T22:44:42.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.127+0000 7f7d29ffb640 1 -- 172.21.15.204:0/2016927683 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20004b10 con 0x7f7d1c003530 2024-09-11T22:44:42.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.128+0000 7f7d3320b640 1 -- 172.21.15.204:0/2016927683 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2c1a6850 con 0x7f7d1c003530 2024-09-11T22:44:42.131 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.129+0000 7f7d29ffb640 1 -- 172.21.15.204:0/2016927683 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f7d20004cb0 con 0x7f7d1c003530 2024-09-11T22:44:42.131 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.129+0000 7f7d29ffb640 1 --2- 172.21.15.204:0/2016927683 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f7d100394f0 0x7f7d1003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:42.131 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.129+0000 7f7d3320b640 1 -- 172.21.15.204:0/2016927683 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d2c10a8d0 con 0x7f7d1c003530 2024-09-11T22:44:42.132 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.129+0000 7f7d29ffb640 1 -- 172.21.15.204:0/2016927683 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7d2006d4a0 con 0x7f7d1c003530 2024-09-11T22:44:42.132 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.129+0000 7f7d2bfff640 1 --2- 172.21.15.204:0/2016927683 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f7d100394f0 0x7f7d1003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:42.132 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.130+0000 7f7d2bfff640 1 --2- 172.21.15.204:0/2016927683 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f7d100394f0 0x7f7d1003b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d18006fd0 tx=0x7f7d18008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:42.135 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.134+0000 7f7d29ffb640 1 -- 172.21.15.204:0/2016927683 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d200403a0 con 0x7f7d1c003530 2024-09-11T22:44:42.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.376+0000 7f7d3320b640 1 -- 172.21.15.204:0/2016927683 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7d2c1068e0 con 0x7f7d1c003530 2024-09-11T22:44:42.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.377+0000 7f7d29ffb640 1 -- 172.21.15.204:0/2016927683 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7f7d200472a0 con 0x7f7d1c003530 2024-09-11T22:44:42.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:44:42.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: mon.a calling monitor election 2024-09-11T22:44:42.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: mon.c calling monitor election 2024-09-11T22:44:42.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:42.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:42.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: mon.b calling monitor election 2024-09-11T22:44:42.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:42.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T22:44:42.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: monmap epoch 3 2024-09-11T22:44:42.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:44:42.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: last_changed 2024-09-11T22:44:37.076243+0000 2024-09-11T22:44:42.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: created 2024-09-11T22:42:57.315021+0000 2024-09-11T22:44:42.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: min_mon_release 19 (squid) 2024-09-11T22:44:42.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: election_strategy: 1 2024-09-11T22:44:42.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: 0: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.a 2024-09-11T22:44:42.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-11T22:44:42.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: 2: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.b 2024-09-11T22:44:42.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: fsmap 2024-09-11T22:44:42.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:44:42.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: mgrmap e13: x(active, since 47s) 2024-09-11T22:44:42.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: overall HEALTH_OK 2024-09-11T22:44:42.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:42.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:42.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:42.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:42.381 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:44:42.381 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":3,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","modified":"2024-09-11T22:44:37.076243Z","created":"2024-09-11T22:42:57.315021Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:3300","nonce":0},{"type":"v1","addr":"172.21.15.191:6789","nonce":0}]},"addr":"172.21.15.191:6789/0","public_addr":"172.21.15.191:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-11T22:44:42.381 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 3 2024-09-11T22:44:42.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.379+0000 7f7d3320b640 1 -- 172.21.15.204:0/2016927683 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f7d100394f0 msgr2=0x7f7d1003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:42.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.379+0000 7f7d3320b640 1 --2- 172.21.15.204:0/2016927683 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f7d100394f0 0x7f7d1003b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d18006fd0 tx=0x7f7d18008040 comp rx=0 tx=0).stop 2024-09-11T22:44:42.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.379+0000 7f7d3320b640 1 -- 172.21.15.204:0/2016927683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d1c003530 msgr2=0x7f7d2c1995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:42.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.379+0000 7f7d3320b640 1 --2- 172.21.15.204:0/2016927683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d1c003530 0x7f7d2c1995a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7d2002c2d0 tx=0x7f7d20005430 comp rx=0 tx=0).stop 2024-09-11T22:44:42.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.379+0000 7f7d3320b640 1 -- 172.21.15.204:0/2016927683 shutdown_connections 2024-09-11T22:44:42.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.379+0000 7f7d3320b640 1 --2- 172.21.15.204:0/2016927683 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f7d100394f0 0x7f7d1003b9b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:42.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.379+0000 7f7d3320b640 1 --2- 172.21.15.204:0/2016927683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d2c19e3b0 0x7f7d2c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:42.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.379+0000 7f7d3320b640 1 --2- 172.21.15.204:0/2016927683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c199ae0 0x7f7d2c19de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:42.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.379+0000 7f7d3320b640 1 --2- 172.21.15.204:0/2016927683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d1c003530 0x7f7d2c1995a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:42.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.379+0000 7f7d3320b640 1 -- 172.21.15.204:0/2016927683 >> 172.21.15.204:0/2016927683 conn(0x7f7d2c0fd320 msgr2=0x7f7d2c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:42.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.379+0000 7f7d3320b640 1 -- 172.21.15.204:0/2016927683 shutdown_connections 2024-09-11T22:44:42.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:42.379+0000 7f7d3320b640 1 -- 172.21.15.204:0/2016927683 wait complete. 2024-09-11T22:44:42.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:44:42.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: mon.a calling monitor election 2024-09-11T22:44:42.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: mon.c calling monitor election 2024-09-11T22:44:42.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:42.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:42.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: mon.b calling monitor election 2024-09-11T22:44:42.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:42.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T22:44:42.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: monmap epoch 3 2024-09-11T22:44:42.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:44:42.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: last_changed 2024-09-11T22:44:37.076243+0000 2024-09-11T22:44:42.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: created 2024-09-11T22:42:57.315021+0000 2024-09-11T22:44:42.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: min_mon_release 19 (squid) 2024-09-11T22:44:42.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: election_strategy: 1 2024-09-11T22:44:42.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: 0: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.a 2024-09-11T22:44:42.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-11T22:44:42.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: 2: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.b 2024-09-11T22:44:42.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: fsmap 2024-09-11T22:44:42.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:44:42.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: mgrmap e13: x(active, since 47s) 2024-09-11T22:44:42.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: overall HEALTH_OK 2024-09-11T22:44:42.540 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:42.540 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:42.540 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:42.540 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:42.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:44:42.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: mon.a calling monitor election 2024-09-11T22:44:42.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: mon.c calling monitor election 2024-09-11T22:44:42.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:44:42.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:42.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: mon.b calling monitor election 2024-09-11T22:44:42.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:42.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:42.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T22:44:42.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: monmap epoch 3 2024-09-11T22:44:42.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:44:42.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: last_changed 2024-09-11T22:44:37.076243+0000 2024-09-11T22:44:42.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: created 2024-09-11T22:42:57.315021+0000 2024-09-11T22:44:42.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: min_mon_release 19 (squid) 2024-09-11T22:44:42.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: election_strategy: 1 2024-09-11T22:44:42.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: 0: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.a 2024-09-11T22:44:42.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: 1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.c 2024-09-11T22:44:42.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: 2: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.b 2024-09-11T22:44:42.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: fsmap 2024-09-11T22:44:42.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:44:42.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: mgrmap e13: x(active, since 47s) 2024-09-11T22:44:42.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: overall HEALTH_OK 2024-09-11T22:44:42.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:42.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:42.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:42.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:42.950 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T22:44:42.950 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T22:44:43.206 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:44:43.244 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:44:43.244 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:43 smithi204 ceph-mon[35454]: from='client.? 172.21.15.204:0/2016927683' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:44:43.244 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:43 smithi204 ceph-mon[35454]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:43.245 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:43.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:44:43.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:43 smithi152 ceph-mon[31181]: from='client.? 172.21.15.204:0/2016927683' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:44:43.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:43 smithi152 ceph-mon[31181]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:43.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:43.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:44:43.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:43 smithi191 ceph-mon[35211]: from='client.? 172.21.15.204:0/2016927683' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:44:43.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:43 smithi191 ceph-mon[35211]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:43.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:44:43.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.675+0000 7ff782b95640 1 -- 172.21.15.152:0/3063613268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff77c106650 msgr2=0x7ff77c106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:43.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.675+0000 7ff782b95640 1 --2- 172.21.15.152:0/3063613268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff77c106650 0x7ff77c106a30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff770009880 tx=0x7ff77002a5d0 comp rx=0 tx=0).stop 2024-09-11T22:44:43.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.675+0000 7ff782b95640 1 -- 172.21.15.152:0/3063613268 shutdown_connections 2024-09-11T22:44:43.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.675+0000 7ff782b95640 1 --2- 172.21.15.152:0/3063613268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff77c106650 0x7ff77c106a30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:43.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.675+0000 7ff782b95640 1 -- 172.21.15.152:0/3063613268 >> 172.21.15.152:0/3063613268 conn(0x7ff77c0fc280 msgr2=0x7ff77c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:43.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.675+0000 7ff782b95640 1 -- 172.21.15.152:0/3063613268 shutdown_connections 2024-09-11T22:44:43.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.675+0000 7ff782b95640 1 -- 172.21.15.152:0/3063613268 wait complete. 2024-09-11T22:44:43.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.675+0000 7ff782b95640 1 Processor -- start 2024-09-11T22:44:43.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff782b95640 1 -- start start 2024-09-11T22:44:43.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff782b95640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff77c106650 0x7ff77c1901e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:43.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff782b95640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff77c194140 0x7ff77c190720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:43.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff782b95640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff77c190dc0 0x7ff77c1a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:43.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff782b95640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c10ac80 con 0x7ff77c106650 2024-09-11T22:44:43.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff780931640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff77c106650 0x7ff77c1901e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:43.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff780931640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff77c106650 0x7ff77c1901e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39892/0 (socket says 172.21.15.152:39892) 2024-09-11T22:44:43.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff780931640 1 -- 172.21.15.152:0/2070078117 learned_addr learned my addr 172.21.15.152:0/2070078117 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:44:43.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff782b95640 1 -- 172.21.15.152:0/2070078117 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c10ab00 con 0x7ff77c190dc0 2024-09-11T22:44:43.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff782b95640 1 -- 172.21.15.152:0/2070078117 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c10ae00 con 0x7ff77c194140 2024-09-11T22:44:43.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff77bfff640 1 --2- 172.21.15.152:0/2070078117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff77c194140 0x7ff77c190720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:43.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff781132640 1 --2- 172.21.15.152:0/2070078117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff77c190dc0 0x7ff77c1a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:43.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff780931640 1 -- 172.21.15.152:0/2070078117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff77c194140 msgr2=0x7ff77c190720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:43.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff780931640 1 --2- 172.21.15.152:0/2070078117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff77c194140 0x7ff77c190720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:43.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff780931640 1 -- 172.21.15.152:0/2070078117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff77c190dc0 msgr2=0x7ff77c1a9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:43.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff780931640 1 --2- 172.21.15.152:0/2070078117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff77c190dc0 0x7ff77c1a9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:43.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.676+0000 7ff780931640 1 -- 172.21.15.152:0/2070078117 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c1aa4c0 con 0x7ff77c106650 2024-09-11T22:44:43.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.677+0000 7ff780931640 1 --2- 172.21.15.152:0/2070078117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff77c106650 0x7ff77c1901e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff770006010 tx=0x7ff770003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:43.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.677+0000 7ff779ffb640 1 -- 172.21.15.152:0/2070078117 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77003f070 con 0x7ff77c106650 2024-09-11T22:44:43.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.677+0000 7ff779ffb640 1 -- 172.21.15.152:0/2070078117 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff77003b460 con 0x7ff77c106650 2024-09-11T22:44:43.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.677+0000 7ff782b95640 1 -- 172.21.15.152:0/2070078117 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77c1aa750 con 0x7ff77c106650 2024-09-11T22:44:43.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.677+0000 7ff779ffb640 1 -- 172.21.15.152:0/2070078117 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff770035030 con 0x7ff77c106650 2024-09-11T22:44:43.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.677+0000 7ff782b95640 1 -- 172.21.15.152:0/2070078117 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77c1aad00 con 0x7ff77c106650 2024-09-11T22:44:43.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.678+0000 7ff779ffb640 1 -- 172.21.15.152:0/2070078117 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7ff770042070 con 0x7ff77c106650 2024-09-11T22:44:43.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.678+0000 7ff782b95640 1 -- 172.21.15.152:0/2070078117 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff77c101cc0 con 0x7ff77c106650 2024-09-11T22:44:43.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.679+0000 7ff779ffb640 1 --2- 172.21.15.152:0/2070078117 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff76c0394f0 0x7ff76c03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:43.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.679+0000 7ff779ffb640 1 -- 172.21.15.152:0/2070078117 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff77006cbd0 con 0x7ff77c106650 2024-09-11T22:44:43.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.679+0000 7ff77bfff640 1 --2- 172.21.15.152:0/2070078117 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff76c0394f0 0x7ff76c03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:43.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.679+0000 7ff77bfff640 1 --2- 172.21.15.152:0/2070078117 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff76c0394f0 0x7ff76c03b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff77c1919a0 tx=0x7ff768009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:43.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.684+0000 7ff779ffb640 1 -- 172.21.15.152:0/2070078117 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff770039510 con 0x7ff77c106650 2024-09-11T22:44:43.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.861+0000 7ff782b95640 1 -- 172.21.15.152:0/2070078117 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff77c02d7c0 con 0x7ff77c106650 2024-09-11T22:44:43.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.861+0000 7ff779ffb640 1 -- 172.21.15.152:0/2070078117 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7ff770039770 con 0x7ff77c106650 2024-09-11T22:44:43.862 INFO:teuthology.orchestra.run.smithi152.stdout:# minimal ceph.conf for 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:44:43.862 INFO:teuthology.orchestra.run.smithi152.stdout:[global] 2024-09-11T22:44:43.862 INFO:teuthology.orchestra.run.smithi152.stdout: fsid = 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:44:43.862 INFO:teuthology.orchestra.run.smithi152.stdout: mon_host = [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-09-11T22:44:43.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.863+0000 7ff782b95640 1 -- 172.21.15.152:0/2070078117 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff76c0394f0 msgr2=0x7ff76c03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:43.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.863+0000 7ff782b95640 1 --2- 172.21.15.152:0/2070078117 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff76c0394f0 0x7ff76c03b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff77c1919a0 tx=0x7ff768009190 comp rx=0 tx=0).stop 2024-09-11T22:44:43.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.863+0000 7ff782b95640 1 -- 172.21.15.152:0/2070078117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff77c106650 msgr2=0x7ff77c1901e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:43.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.863+0000 7ff782b95640 1 --2- 172.21.15.152:0/2070078117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff77c106650 0x7ff77c1901e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff770006010 tx=0x7ff770003c00 comp rx=0 tx=0).stop 2024-09-11T22:44:43.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.864+0000 7ff782b95640 1 -- 172.21.15.152:0/2070078117 shutdown_connections 2024-09-11T22:44:43.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.864+0000 7ff782b95640 1 --2- 172.21.15.152:0/2070078117 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff76c0394f0 0x7ff76c03b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:43.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.864+0000 7ff782b95640 1 --2- 172.21.15.152:0/2070078117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff77c190dc0 0x7ff77c1a9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:43.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.864+0000 7ff782b95640 1 --2- 172.21.15.152:0/2070078117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff77c194140 0x7ff77c190720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:43.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.864+0000 7ff782b95640 1 --2- 172.21.15.152:0/2070078117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff77c106650 0x7ff77c1901e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:43.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.864+0000 7ff782b95640 1 -- 172.21.15.152:0/2070078117 >> 172.21.15.152:0/2070078117 conn(0x7ff77c0fc280 msgr2=0x7ff77c102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:43.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.864+0000 7ff782b95640 1 -- 172.21.15.152:0/2070078117 shutdown_connections 2024-09-11T22:44:43.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:43.864+0000 7ff782b95640 1 -- 172.21.15.152:0/2070078117 wait complete. 2024-09-11T22:44:44.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:44.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:44.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:44.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:44.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:44.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:44:44.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:44 smithi191 ceph-mon[35211]: Updating smithi152:/etc/ceph/ceph.conf 2024-09-11T22:44:44.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:44 smithi191 ceph-mon[35211]: Updating smithi191:/etc/ceph/ceph.conf 2024-09-11T22:44:44.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:44 smithi191 ceph-mon[35211]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-11T22:44:44.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:44 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2070078117' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:44.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:44.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:44.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:44.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:44.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:44.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:44:44.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:44 smithi204 ceph-mon[35454]: Updating smithi152:/etc/ceph/ceph.conf 2024-09-11T22:44:44.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:44 smithi204 ceph-mon[35454]: Updating smithi191:/etc/ceph/ceph.conf 2024-09-11T22:44:44.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:44 smithi204 ceph-mon[35454]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-11T22:44:44.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:44 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2070078117' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:44.562 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:44.562 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:44.562 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:44.562 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:44.562 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:44.562 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:44:44.562 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:44 smithi152 ceph-mon[31181]: Updating smithi152:/etc/ceph/ceph.conf 2024-09-11T22:44:44.562 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:44 smithi152 ceph-mon[31181]: Updating smithi191:/etc/ceph/ceph.conf 2024-09-11T22:44:44.562 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:44 smithi152 ceph-mon[31181]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-11T22:44:44.562 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:44 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2070078117' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:44.594 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T22:44:44.594 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:44:44.594 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T22:44:44.623 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:44:44.623 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:44:44.688 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-11T22:44:44.688 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T22:44:44.717 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-11T22:44:44.717 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:44:44.783 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-11T22:44:44.784 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T22:44:44.812 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-11T22:44:44.812 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:44:44.878 INFO:tasks.cephadm:Adding mgr.x on smithi152 2024-09-11T22:44:44.878 INFO:tasks.cephadm:Adding mgr.y on smithi191 2024-09-11T22:44:44.878 INFO:tasks.cephadm:Adding mgr.z on smithi204 2024-09-11T22:44:44.879 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi152=x;smithi191=y;smithi204=z' 2024-09-11T22:44:45.154 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:44:45.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: Updating smithi191:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:44:45.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: Updating smithi204:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:44:45.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: Updating smithi152:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:44:45.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: Reconfiguring mon.a (monmap changed)... 2024-09-11T22:44:45.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:44:45.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:44:45.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:45.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: Reconfiguring daemon mon.a on smithi152 2024-09-11T22:44:45.514 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:45.515 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: Updating smithi191:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:44:45.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: Updating smithi204:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:44:45.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: Updating smithi152:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:44:45.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: Reconfiguring mon.a (monmap changed)... 2024-09-11T22:44:45.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:44:45.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:44:45.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:45.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: Reconfiguring daemon mon.a on smithi152 2024-09-11T22:44:45.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:45.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.594+0000 7f8a9f05e640 1 -- 172.21.15.204:0/2784339699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a98101710 msgr2=0x7f8a98101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:45.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.594+0000 7f8a9f05e640 1 --2- 172.21.15.204:0/2784339699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a98101710 0x7f8a98101b10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009880 tx=0x7f8a8c02a5d0 comp rx=0 tx=0).stop 2024-09-11T22:44:45.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.594+0000 7f8a9f05e640 1 -- 172.21.15.204:0/2784339699 shutdown_connections 2024-09-11T22:44:45.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.594+0000 7f8a9f05e640 1 --2- 172.21.15.204:0/2784339699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a98101710 0x7f8a98101b10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:45.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.594+0000 7f8a9f05e640 1 -- 172.21.15.204:0/2784339699 >> 172.21.15.204:0/2784339699 conn(0x7f8a980fcec0 msgr2=0x7f8a980ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:45.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.594+0000 7f8a9f05e640 1 -- 172.21.15.204:0/2784339699 shutdown_connections 2024-09-11T22:44:45.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.594+0000 7f8a9f05e640 1 -- 172.21.15.204:0/2784339699 wait complete. 2024-09-11T22:44:45.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.595+0000 7f8a9f05e640 1 Processor -- start 2024-09-11T22:44:45.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.595+0000 7f8a9f05e640 1 -- start start 2024-09-11T22:44:45.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.595+0000 7f8a9f05e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a98101710 0x7f8a98196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:45.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.595+0000 7f8a9f05e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a98197510 0x7f8a9819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:45.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.595+0000 7f8a9f05e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a98197990 0x7f8a9819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:45.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.595+0000 7f8a9f05e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8a98107800 con 0x7f8a98197510 2024-09-11T22:44:45.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.595+0000 7f8a9f05e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8a98107680 con 0x7f8a98101710 2024-09-11T22:44:45.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.595+0000 7f8a9f05e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8a98107980 con 0x7f8a98197990 2024-09-11T22:44:45.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.595+0000 7f8a9d5fb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a98197990 0x7f8a9819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:45.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.595+0000 7f8a9d5fb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a98197990 0x7f8a9819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:58586/0 (socket says 172.21.15.204:58586) 2024-09-11T22:44:45.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.595+0000 7f8a9d5fb640 1 -- 172.21.15.204:0/2783252563 learned_addr learned my addr 172.21.15.204:0/2783252563 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-11T22:44:45.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.595+0000 7f8a97fff640 1 --2- 172.21.15.204:0/2783252563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a98197510 0x7f8a9819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:45.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.595+0000 7f8a9cdfa640 1 --2- 172.21.15.204:0/2783252563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a98101710 0x7f8a98196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:45.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.595+0000 7f8a9d5fb640 1 -- 172.21.15.204:0/2783252563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a98101710 msgr2=0x7f8a98196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:45.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.595+0000 7f8a9d5fb640 1 --2- 172.21.15.204:0/2783252563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a98101710 0x7f8a98196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:45.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.595+0000 7f8a9d5fb640 1 -- 172.21.15.204:0/2783252563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a98197510 msgr2=0x7f8a9819c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:45.607 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.596+0000 7f8a9d5fb640 1 --2- 172.21.15.204:0/2783252563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a98197510 0x7f8a9819c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:45.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.596+0000 7f8a9d5fb640 1 -- 172.21.15.204:0/2783252563 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9819f3d0 con 0x7f8a98197990 2024-09-11T22:44:45.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.596+0000 7f8a97fff640 1 --2- 172.21.15.204:0/2783252563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a98197510 0x7f8a9819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:44:45.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.596+0000 7f8a9d5fb640 1 --2- 172.21.15.204:0/2783252563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a98197990 0x7f8a9819eb10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000ba50 tx=0x7f8a9000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:45.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.596+0000 7f8a95ffb640 1 -- 172.21.15.204:0/2783252563 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90019070 con 0x7f8a98197990 2024-09-11T22:44:45.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.596+0000 7f8a9f05e640 1 -- 172.21.15.204:0/2783252563 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a9819f6c0 con 0x7f8a98197990 2024-09-11T22:44:45.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.596+0000 7f8a95ffb640 1 -- 172.21.15.204:0/2783252563 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9000cbe0 con 0x7f8a98197990 2024-09-11T22:44:45.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.596+0000 7f8a95ffb640 1 -- 172.21.15.204:0/2783252563 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a900076a0 con 0x7f8a98197990 2024-09-11T22:44:45.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.596+0000 7f8a9f05e640 1 -- 172.21.15.204:0/2783252563 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a9819fc30 con 0x7f8a98197990 2024-09-11T22:44:45.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.597+0000 7f8a95ffb640 1 -- 172.21.15.204:0/2783252563 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f8a9001c070 con 0x7f8a98197990 2024-09-11T22:44:45.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.597+0000 7f8a95ffb640 1 --2- 172.21.15.204:0/2783252563 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8a880394f0 0x7f8a8803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:45.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.597+0000 7f8a95ffb640 1 -- 172.21.15.204:0/2783252563 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8a9004baf0 con 0x7f8a98197990 2024-09-11T22:44:45.608 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.597+0000 7f8a9f05e640 1 -- 172.21.15.204:0/2783252563 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a98101b90 con 0x7f8a98197990 2024-09-11T22:44:45.609 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.597+0000 7f8a9cdfa640 1 --2- 172.21.15.204:0/2783252563 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8a880394f0 0x7f8a8803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:45.609 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.598+0000 7f8a9cdfa640 1 --2- 172.21.15.204:0/2783252563 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8a880394f0 0x7f8a8803b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c000c00 tx=0x7f8a8c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:45.609 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.602+0000 7f8a95ffb640 1 -- 172.21.15.204:0/2783252563 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a9000e0a0 con 0x7f8a98197990 2024-09-11T22:44:45.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: Updating smithi191:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:44:45.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: Updating smithi204:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:44:45.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: Updating smithi152:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:44:45.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: Reconfiguring mon.a (monmap changed)... 2024-09-11T22:44:45.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:44:45.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:44:45.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:45.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: Reconfiguring daemon mon.a on smithi152 2024-09-11T22:44:45.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:45.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:45.779 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.778+0000 7f8a9f05e640 1 -- 172.21.15.204:0/2783252563 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi152=x;smithi191=y;smithi204=z", "target": ["mon-mgr", ""]}) -- 0x7f8a980626c0 con 0x7f8a880394f0 2024-09-11T22:44:45.783 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.783+0000 7f8a95ffb640 1 -- 172.21.15.204:0/2783252563 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8a980626c0 con 0x7f8a880394f0 2024-09-11T22:44:45.783 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mgr update... 2024-09-11T22:44:45.786 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.785+0000 7f8a9f05e640 1 -- 172.21.15.204:0/2783252563 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8a880394f0 msgr2=0x7f8a8803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:45.786 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.785+0000 7f8a9f05e640 1 --2- 172.21.15.204:0/2783252563 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8a880394f0 0x7f8a8803b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c000c00 tx=0x7f8a8c031400 comp rx=0 tx=0).stop 2024-09-11T22:44:45.786 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.785+0000 7f8a9f05e640 1 -- 172.21.15.204:0/2783252563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a98197990 msgr2=0x7f8a9819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:45.786 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.785+0000 7f8a9f05e640 1 --2- 172.21.15.204:0/2783252563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a98197990 0x7f8a9819eb10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000ba50 tx=0x7f8a9000bf10 comp rx=0 tx=0).stop 2024-09-11T22:44:45.787 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.785+0000 7f8a9f05e640 1 -- 172.21.15.204:0/2783252563 shutdown_connections 2024-09-11T22:44:45.787 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.785+0000 7f8a9f05e640 1 --2- 172.21.15.204:0/2783252563 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8a880394f0 0x7f8a8803b9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:45.787 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.785+0000 7f8a9f05e640 1 --2- 172.21.15.204:0/2783252563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a98197990 0x7f8a9819eb10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:45.787 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.785+0000 7f8a9f05e640 1 --2- 172.21.15.204:0/2783252563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a98197510 0x7f8a9819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:45.787 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.785+0000 7f8a9f05e640 1 --2- 172.21.15.204:0/2783252563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a98101710 0x7f8a98196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:45.787 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.785+0000 7f8a9f05e640 1 -- 172.21.15.204:0/2783252563 >> 172.21.15.204:0/2783252563 conn(0x7f8a980fcec0 msgr2=0x7f8a980fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:45.787 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.785+0000 7f8a9f05e640 1 -- 172.21.15.204:0/2783252563 shutdown_connections 2024-09-11T22:44:45.787 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:44:45.785+0000 7f8a9f05e640 1 -- 172.21.15.204:0/2783252563 wait complete. 2024-09-11T22:44:46.311 DEBUG:teuthology.orchestra.run.smithi191:mgr.y> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T22:44:46.314 DEBUG:teuthology.orchestra.run.smithi204:mgr.z> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mgr.z.service 2024-09-11T22:44:46.316 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T22:44:46.317 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:44:46.317 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T22:44:46.334 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T22:44:46.334 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_1 2024-09-11T22:44:46.390 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T22:44:46.390 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:44:46.390 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-11T22:44:46.390 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:44:46.391 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:44:46.391 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-09-11 22:44:03.779838150 +0000 2024-09-11T22:44:46.391 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-09-11 22:39:31.195885627 +0000 2024-09-11T22:44:46.391 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-09-11 22:39:31.195885627 +0000 2024-09-11T22:44:46.391 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-09-11 22:39:31.195885627 +0000 2024-09-11T22:44:46.391 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T22:44:46.455 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-09-11T22:44:46.455 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-09-11T22:44:46.455 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000198135 s, 2.6 MB/s 2024-09-11T22:44:46.456 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T22:44:46.513 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_2 2024-09-11T22:44:46.569 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T22:44:46.569 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:44:46.569 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-11T22:44:46.570 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:44:46.570 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:44:46.570 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-09-11 22:44:03.813837732 +0000 2024-09-11T22:44:46.570 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-09-11 22:39:31.536881581 +0000 2024-09-11T22:44:46.570 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-09-11 22:39:31.536881581 +0000 2024-09-11T22:44:46.570 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-09-11 22:39:31.536881581 +0000 2024-09-11T22:44:46.570 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T22:44:46.633 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-09-11T22:44:46.633 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-09-11T22:44:46.633 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.00020144 s, 2.5 MB/s 2024-09-11T22:44:46.634 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T22:44:46.691 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_3 2024-09-11T22:44:46.746 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T22:44:46.747 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:44:46.747 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-11T22:44:46.747 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:44:46.747 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:44:46.747 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-09-11 22:44:03.847837314 +0000 2024-09-11T22:44:46.747 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-09-11 22:39:31.854877806 +0000 2024-09-11T22:44:46.747 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-09-11 22:39:31.854877806 +0000 2024-09-11T22:44:46.747 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-09-11 22:39:31.854877806 +0000 2024-09-11T22:44:46.747 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T22:44:46.812 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-09-11T22:44:46.812 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-09-11T22:44:46.812 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000221107 s, 2.3 MB/s 2024-09-11T22:44:46.813 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T22:44:46.869 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_4 2024-09-11T22:44:46.924 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T22:44:46.924 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:44:46.925 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-11T22:44:46.925 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:44:46.925 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:44:46.925 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-09-11 22:44:03.878836934 +0000 2024-09-11T22:44:46.925 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-09-11 22:39:32.206873626 +0000 2024-09-11T22:44:46.925 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-09-11 22:39:32.206873626 +0000 2024-09-11T22:44:46.925 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-09-11 22:39:32.206873626 +0000 2024-09-11T22:44:46.925 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T22:44:46.987 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-09-11T22:44:46.987 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-09-11T22:44:46.987 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000178044 s, 2.9 MB/s 2024-09-11T22:44:46.988 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T22:44:47.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:46 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:47.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:46 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:47.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:46 smithi191 ceph-mon[35211]: Reconfiguring mon.b (monmap changed)... 2024-09-11T22:44:47.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:46 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:44:47.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:46 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:44:47.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:46 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:47.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:46 smithi191 ceph-mon[35211]: Reconfiguring daemon mon.b on smithi191 2024-09-11T22:44:47.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:46 smithi191 ceph-mon[35211]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi152=x;smithi191=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:44:47.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:46 smithi191 ceph-mon[35211]: Saving service mgr spec with placement smithi152=x;smithi191=y;smithi204=z;count:3 2024-09-11T22:44:47.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:46 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:47.044 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-09-11T22:44:47.045 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T22:44:47.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:46 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:47.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:46 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:47.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:46 smithi204 ceph-mon[35454]: Reconfiguring mon.b (monmap changed)... 2024-09-11T22:44:47.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:46 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:44:47.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:46 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:44:47.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:46 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:47.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:46 smithi204 ceph-mon[35454]: Reconfiguring daemon mon.b on smithi191 2024-09-11T22:44:47.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:46 smithi204 ceph-mon[35454]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi152=x;smithi191=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:44:47.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:46 smithi204 ceph-mon[35454]: Saving service mgr spec with placement smithi152=x;smithi191=y;smithi204=z;count:3 2024-09-11T22:44:47.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:46 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:47.061 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T22:44:47.061 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_1 2024-09-11T22:44:47.117 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T22:44:47.117 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:44:47.117 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-09-11T22:44:47.118 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:44:47.118 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:44:47.118 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-09-11 22:44:36.521339013 +0000 2024-09-11T22:44:47.118 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-09-11 22:39:48.886189901 +0000 2024-09-11T22:44:47.118 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-09-11 22:39:48.886189901 +0000 2024-09-11T22:44:47.118 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-09-11 22:39:48.886189901 +0000 2024-09-11T22:44:47.118 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T22:44:47.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:46 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:47.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:46 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:47.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:46 smithi152 ceph-mon[31181]: Reconfiguring mon.b (monmap changed)... 2024-09-11T22:44:47.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:46 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:44:47.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:46 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:44:47.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:46 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:47.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:46 smithi152 ceph-mon[31181]: Reconfiguring daemon mon.b on smithi191 2024-09-11T22:44:47.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:46 smithi152 ceph-mon[31181]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi152=x;smithi191=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:44:47.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:46 smithi152 ceph-mon[31181]: Saving service mgr spec with placement smithi152=x;smithi191=y;smithi204=z;count:3 2024-09-11T22:44:47.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:46 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:47.186 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-09-11T22:44:47.186 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-09-11T22:44:47.186 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000277219 s, 1.8 MB/s 2024-09-11T22:44:47.188 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T22:44:47.244 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_2 2024-09-11T22:44:47.300 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T22:44:47.300 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:44:47.300 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-11T22:44:47.300 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:44:47.300 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:44:47.300 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-09-11 22:44:36.521339013 +0000 2024-09-11T22:44:47.300 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-09-11 22:39:49.291182709 +0000 2024-09-11T22:44:47.300 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-09-11 22:39:49.291182709 +0000 2024-09-11T22:44:47.300 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-09-11 22:39:49.291182709 +0000 2024-09-11T22:44:47.301 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T22:44:47.363 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-09-11T22:44:47.363 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-09-11T22:44:47.363 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000195652 s, 2.6 MB/s 2024-09-11T22:44:47.364 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T22:44:47.421 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_3 2024-09-11T22:44:47.476 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T22:44:47.476 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:44:47.476 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-11T22:44:47.476 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:44:47.476 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:44:47.476 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-09-11 22:44:36.522338996 +0000 2024-09-11T22:44:47.476 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-09-11 22:39:49.624176795 +0000 2024-09-11T22:44:47.476 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-09-11 22:39:49.624176795 +0000 2024-09-11T22:44:47.476 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-09-11 22:39:49.624176795 +0000 2024-09-11T22:44:47.477 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T22:44:47.539 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-09-11T22:44:47.539 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-09-11T22:44:47.539 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000229299 s, 2.2 MB/s 2024-09-11T22:44:47.540 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T22:44:47.597 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_4 2024-09-11T22:44:47.652 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T22:44:47.652 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:44:47.652 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-11T22:44:47.652 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:44:47.652 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:44:47.652 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-09-11 22:44:36.522338996 +0000 2024-09-11T22:44:47.652 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-09-11 22:39:49.945171095 +0000 2024-09-11T22:44:47.653 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-09-11 22:39:49.945171095 +0000 2024-09-11T22:44:47.653 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-09-11 22:39:49.945171095 +0000 2024-09-11T22:44:47.653 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T22:44:47.716 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-09-11T22:44:47.716 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-09-11T22:44:47.716 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000275114 s, 1.9 MB/s 2024-09-11T22:44:47.717 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T22:44:47.773 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-09-11T22:44:47.773 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T22:44:47.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:47 smithi191 ceph-mon[35211]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:47.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:47.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:47.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:44:47.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:44:47.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:47.790 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T22:44:47.790 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-09-11T22:44:47.848 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T22:44:47.848 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:44:47.848 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-11T22:44:47.848 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:44:47.848 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:44:47.848 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-11 22:44:27.424793262 +0000 2024-09-11T22:44:47.848 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-11 22:39:46.390071298 +0000 2024-09-11T22:44:47.848 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-11 22:39:46.390071298 +0000 2024-09-11T22:44:47.848 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-11 22:39:46.390071298 +0000 2024-09-11T22:44:47.849 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T22:44:47.853 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:47 smithi204 ceph-mon[35454]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:47.853 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:47.853 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:47.854 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:44:47.854 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:44:47.854 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:47.879 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-11T22:44:47.879 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-11T22:44:47.879 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000215749 s, 2.4 MB/s 2024-09-11T22:44:47.880 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T22:44:47.938 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-09-11T22:44:47.993 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T22:44:47.994 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:44:47.994 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-11T22:44:47.994 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:44:47.994 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:44:47.994 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-11 22:44:27.424793262 +0000 2024-09-11T22:44:47.994 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-11 22:39:46.728066971 +0000 2024-09-11T22:44:47.994 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-11 22:39:46.728066971 +0000 2024-09-11T22:44:47.994 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-11 22:39:46.728066971 +0000 2024-09-11T22:44:47.994 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T22:44:48.056 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-11T22:44:48.056 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-11T22:44:48.057 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000234554 s, 2.2 MB/s 2024-09-11T22:44:48.058 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T22:44:48.113 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-09-11T22:44:48.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:47 smithi152 ceph-mon[31181]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:48.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:48.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:48.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:44:48.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:44:48.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:48.169 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T22:44:48.169 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:44:48.169 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-09-11T22:44:48.169 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:44:48.169 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:44:48.169 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-11 22:44:27.424793262 +0000 2024-09-11T22:44:48.169 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-11 22:39:47.059062732 +0000 2024-09-11T22:44:48.169 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-11 22:39:47.059062732 +0000 2024-09-11T22:44:48.170 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-11 22:39:47.059062732 +0000 2024-09-11T22:44:48.170 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T22:44:48.232 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-11T22:44:48.232 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-11T22:44:48.232 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000173698 s, 2.9 MB/s 2024-09-11T22:44:48.233 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T22:44:48.289 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-09-11T22:44:48.345 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T22:44:48.345 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:44:48.345 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-11T22:44:48.345 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:44:48.345 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:44:48.346 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-09-11 22:44:27.425793250 +0000 2024-09-11T22:44:48.346 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-09-11 22:39:47.381058609 +0000 2024-09-11T22:44:48.346 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-09-11 22:39:47.381058609 +0000 2024-09-11T22:44:48.346 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-09-11 22:39:47.381058609 +0000 2024-09-11T22:44:48.346 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T22:44:48.412 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-09-11T22:44:48.412 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-09-11T22:44:48.412 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000247586 s, 2.1 MB/s 2024-09-11T22:44:48.413 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T22:44:48.469 INFO:tasks.cephadm:Deploying osd.0 on smithi152 with /dev/vg_nvme/lv_4... 2024-09-11T22:44:48.470 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T22:44:48.718 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:44:48.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:48 smithi204 ceph-mon[35454]: Reconfiguring mon.c (monmap changed)... 2024-09-11T22:44:48.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:48 smithi204 ceph-mon[35454]: Reconfiguring daemon mon.c on smithi204 2024-09-11T22:44:48.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:48 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:48.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:48 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:48.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:48 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:44:48.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:48 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:48.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:48 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:44:48.865 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:48 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:48.866 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:48 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:44:48.866 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:48 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:44:48.866 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:48 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:44:48.866 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:48 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:49.082 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:48 smithi152 ceph-mon[31181]: Reconfiguring mon.c (monmap changed)... 2024-09-11T22:44:49.082 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:48 smithi152 ceph-mon[31181]: Reconfiguring daemon mon.c on smithi204 2024-09-11T22:44:49.082 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:48 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:49.082 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:48 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:49.082 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:48 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:44:49.082 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:48 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:49.082 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:48 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:44:49.082 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:48 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:49.083 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:48 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:44:49.083 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:48 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:44:49.083 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:48 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:44:49.083 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:48 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:49.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:48 smithi191 ceph-mon[35211]: Reconfiguring mon.c (monmap changed)... 2024-09-11T22:44:49.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:48 smithi191 ceph-mon[35211]: Reconfiguring daemon mon.c on smithi204 2024-09-11T22:44:49.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:48 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:49.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:48 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:49.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:48 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:44:49.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:48 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:49.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:48 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:44:49.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:48 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:49.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:48 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:44:49.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:48 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:44:49.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:48 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:44:49.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:48 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:49.889 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:49 smithi204 ceph-mon[35454]: Deploying daemon mgr.z on smithi204 2024-09-11T22:44:49.889 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:49 smithi204 ceph-mon[35454]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:50.104 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:49 smithi152 ceph-mon[31181]: Deploying daemon mgr.z on smithi204 2024-09-11T22:44:50.104 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:49 smithi152 ceph-mon[31181]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:50.240 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:50 smithi204 systemd[1]: Starting Ceph mgr.z for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T22:44:50.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:49 smithi191 ceph-mon[35211]: Deploying daemon mgr.z on smithi204 2024-09-11T22:44:50.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:49 smithi191 ceph-mon[35211]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:50.546 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:50 smithi204 podman[37243]: 2024-09-11 22:44:50.352885991 +0000 UTC m=+0.015834643 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:44:50.546 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:50 smithi204 podman[37243]: 2024-09-11 22:44:50.458122098 +0000 UTC m=+0.121070736 container create 88afd8e36c63c6f556ee7456dbf9e7c54b82de1c55ea6aeae0f91842a6a6fd85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:44:50.863 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:50 smithi204 podman[37243]: 2024-09-11 22:44:50.669089627 +0000 UTC m=+0.332038267 container init 88afd8e36c63c6f556ee7456dbf9e7c54b82de1c55ea6aeae0f91842a6a6fd85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:44:50.864 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:50 smithi204 podman[37243]: 2024-09-11 22:44:50.672055631 +0000 UTC m=+0.335004283 container start 88afd8e36c63c6f556ee7456dbf9e7c54b82de1c55ea6aeae0f91842a6a6fd85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:44:50.864 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:50 smithi204 ceph-mgr[37262]: -- 172.21.15.204:0/831917975 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cd078590e0 con 0x55cd07837400 2024-09-11T22:44:50.864 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:50 smithi204 bash[37243]: 88afd8e36c63c6f556ee7456dbf9e7c54b82de1c55ea6aeae0f91842a6a6fd85 2024-09-11T22:44:50.864 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:50 smithi204 systemd[1]: Started Ceph mgr.z for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:44:50.864 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:50 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:50.803+0000 7f5c37796040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:44:50.864 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:50 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:50.862+0000 7f5c37796040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:44:51.222 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:51.010+0000 7f5c37796040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:44:51.222 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:51.075+0000 7f5c37796040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:44:51.222 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:51.221+0000 7f5c37796040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:44:51.230 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:44:51.255 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi152:vg_nvme/lv_4 2024-09-11T22:44:51.497 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:44:51.546 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:51.359+0000 7f5c37796040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:44:51.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:51 smithi152 ceph-mon[31181]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:51.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:51.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:51.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:51.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:44:51.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:44:51.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:44:51.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:51.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:51 smithi152 ceph-mon[31181]: Deploying daemon mgr.y on smithi191 2024-09-11T22:44:51.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.966+0000 7f639be9a640 1 -- 172.21.15.152:0/407058590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6394101cd0 msgr2=0x7f6394102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:51.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.966+0000 7f639be9a640 1 --2- 172.21.15.152:0/407058590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6394101cd0 0x7f6394102150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6384009900 tx=0x7f638402a410 comp rx=0 tx=0).stop 2024-09-11T22:44:51.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.966+0000 7f639be9a640 1 -- 172.21.15.152:0/407058590 shutdown_connections 2024-09-11T22:44:51.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.966+0000 7f639be9a640 1 --2- 172.21.15.152:0/407058590 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6394102690 0x7f6394108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:51.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.966+0000 7f639be9a640 1 --2- 172.21.15.152:0/407058590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6394101cd0 0x7f6394102150 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:51.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.966+0000 7f639be9a640 1 --2- 172.21.15.152:0/407058590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6394100ad0 0x7f6394100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:51.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.966+0000 7f639be9a640 1 -- 172.21.15.152:0/407058590 >> 172.21.15.152:0/407058590 conn(0x7f63940fc280 msgr2=0x7f63940fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:44:51.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.966+0000 7f639be9a640 1 -- 172.21.15.152:0/407058590 shutdown_connections 2024-09-11T22:44:51.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.966+0000 7f639be9a640 1 -- 172.21.15.152:0/407058590 wait complete. 2024-09-11T22:44:51.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.966+0000 7f639be9a640 1 Processor -- start 2024-09-11T22:44:51.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.966+0000 7f639be9a640 1 -- start start 2024-09-11T22:44:51.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639be9a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6394100ad0 0x7f6394197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:51.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639be9a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6394101cd0 0x7f6394198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:51.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639be9a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6394102690 0x7f639419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:51.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639be9a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f639410d880 con 0x7f6394102690 2024-09-11T22:44:51.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639be9a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f639410d700 con 0x7f6394100ad0 2024-09-11T22:44:51.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639be9a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f639410da00 con 0x7f6394101cd0 2024-09-11T22:44:51.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639a437640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6394102690 0x7f639419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:51.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639a437640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6394102690 0x7f639419f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60422/0 (socket says 172.21.15.152:60422) 2024-09-11T22:44:51.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639a437640 1 -- 172.21.15.152:0/1059037384 learned_addr learned my addr 172.21.15.152:0/1059037384 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:44:51.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f6399c36640 1 --2- 172.21.15.152:0/1059037384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6394100ad0 0x7f6394197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:51.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f6399435640 1 --2- 172.21.15.152:0/1059037384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6394101cd0 0x7f6394198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639a437640 1 -- 172.21.15.152:0/1059037384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6394101cd0 msgr2=0x7f6394198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639a437640 1 --2- 172.21.15.152:0/1059037384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6394101cd0 0x7f6394198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639a437640 1 -- 172.21.15.152:0/1059037384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6394100ad0 msgr2=0x7f6394197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:44:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639a437640 1 --2- 172.21.15.152:0/1059037384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6394100ad0 0x7f6394197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:44:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639a437640 1 -- 172.21.15.152:0/1059037384 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639419fb30 con 0x7f6394102690 2024-09-11T22:44:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639a437640 1 --2- 172.21.15.152:0/1059037384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6394102690 0x7f639419f4c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f638c00eb10 tx=0x7f638c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f638affd640 1 -- 172.21.15.152:0/1059037384 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c004b20 con 0x7f6394102690 2024-09-11T22:44:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639be9a640 1 -- 172.21.15.152:0/1059037384 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639419fe20 con 0x7f6394102690 2024-09-11T22:44:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f638affd640 1 -- 172.21.15.152:0/1059037384 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f638c004cc0 con 0x7f6394102690 2024-09-11T22:44:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f638affd640 1 -- 172.21.15.152:0/1059037384 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c010760 con 0x7f6394102690 2024-09-11T22:44:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.967+0000 7f639be9a640 1 -- 172.21.15.152:0/1059037384 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63941a0280 con 0x7f6394102690 2024-09-11T22:44:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.968+0000 7f638affd640 1 -- 172.21.15.152:0/1059037384 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f638c010900 con 0x7f6394102690 2024-09-11T22:44:51.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.968+0000 7f638affd640 1 --2- 172.21.15.152:0/1059037384 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f63800392c0 0x7f638003b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:44:51.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.968+0000 7f639be9a640 1 -- 172.21.15.152:0/1059037384 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6394107f20 con 0x7f6394102690 2024-09-11T22:44:51.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.969+0000 7f638affd640 1 -- 172.21.15.152:0/1059037384 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f638c04c9a0 con 0x7f6394102690 2024-09-11T22:44:51.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.969+0000 7f6399c36640 1 --2- 172.21.15.152:0/1059037384 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f63800392c0 0x7f638003b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:44:51.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.969+0000 7f6399c36640 1 --2- 172.21.15.152:0/1059037384 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f63800392c0 0x7f638003b780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6390005fd0 tx=0x7f639000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:44:51.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:51.973+0000 7f638affd640 1 -- 172.21.15.152:0/1059037384 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f638c0203a0 con 0x7f6394102690 2024-09-11T22:44:52.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:51 smithi191 ceph-mon[35211]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:52.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:52.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:52.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:52.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:44:52.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:44:52.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:44:52.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:52.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:51 smithi191 ceph-mon[35211]: Deploying daemon mgr.y on smithi191 2024-09-11T22:44:52.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:51 smithi204 ceph-mon[35454]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:52.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:52.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:52.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:52.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:44:52.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:44:52.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:44:52.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:52.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:51 smithi204 ceph-mon[35454]: Deploying daemon mgr.y on smithi191 2024-09-11T22:44:52.047 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:51.673+0000 7f5c37796040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:44:52.047 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:51.733+0000 7f5c37796040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:44:52.047 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:51.918+0000 7f5c37796040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:44:52.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:52.150+0000 7f639be9a640 1 -- 172.21.15.152:0/1059037384 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6394062630 con 0x7f63800392c0 2024-09-11T22:44:52.787 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:52 smithi191 systemd[1]: Starting Ceph mgr.y for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T22:44:53.087 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:52 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:44:53.087 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:52 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:44:53.087 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:52 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:53.087 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:52 smithi191 podman[36557]: 2024-09-11 22:44:52.729191081 +0000 UTC m=+0.031073063 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:44:53.087 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:52 smithi191 podman[36557]: 2024-09-11 22:44:52.83993819 +0000 UTC m=+0.141820158 container create d4400e87ae3bf26dee913ec779b7c9bd0fb1293617f19c0d3e1fa966aa153907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T22:44:53.087 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:53 smithi191 podman[36557]: 2024-09-11 22:44:53.059003916 +0000 UTC m=+0.360885889 container init d4400e87ae3bf26dee913ec779b7c9bd0fb1293617f19c0d3e1fa966aa153907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-11T22:44:53.087 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:53 smithi191 podman[36557]: 2024-09-11 22:44:53.061819179 +0000 UTC m=+0.363701148 container start d4400e87ae3bf26dee913ec779b7c9bd0fb1293617f19c0d3e1fa966aa153907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:44:53.087 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:53 smithi191 ceph-mgr[36576]: -- 172.21.15.191:0/880682303 <== mon.2 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55589de250e0 con 0x55589de03000 2024-09-11T22:44:53.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:52 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:44:53.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:52 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:44:53.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:52 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:53.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:52 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:44:53.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:52 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:44:53.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:52 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:53.296 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:52 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:52.795+0000 7f5c37796040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:44:53.296 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:52 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:52.863+0000 7f5c37796040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:44:53.400 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:53 smithi191 bash[36557]: d4400e87ae3bf26dee913ec779b7c9bd0fb1293617f19c0d3e1fa966aa153907 2024-09-11T22:44:53.400 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:53 smithi191 systemd[1]: Started Ceph mgr.y for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:44:53.400 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:53.182+0000 7fed19b1d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:44:53.400 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:53.243+0000 7fed19b1d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:44:53.400 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:53.399+0000 7fed19b1d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:44:53.778 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:53.467+0000 7fed19b1d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:44:53.779 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:53.626+0000 7fed19b1d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:44:53.779 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:53.777+0000 7fed19b1d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:44:53.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:53 smithi204 ceph-mon[35454]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:44:53.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:53 smithi204 ceph-mon[35454]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:53.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:53 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:53.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:53 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:53.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:53 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:53.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:53 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:53.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:53 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:44:53.884 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:53 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:53.607+0000 7f5c37796040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:44:53.884 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:53 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:53.737+0000 7f5c37796040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:44:53.884 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:53 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:53.882+0000 7f5c37796040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:44:54.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:53 smithi191 ceph-mon[35211]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:44:54.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:53 smithi191 ceph-mon[35211]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:54.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:53 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:54.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:53 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:54.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:53 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:54.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:53 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:54.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:53 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:44:54.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:53 smithi152 ceph-mon[31181]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:44:54.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:53 smithi152 ceph-mon[31181]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:54.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:53 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:54.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:53 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:54.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:53 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:54.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:53 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:54.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:53 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:44:54.155 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:53 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:53.946+0000 7f5c37796040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:44:54.155 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:54 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:54.103+0000 7f5c37796040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:44:54.349 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:54 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:54.101+0000 7fed19b1d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:44:54.349 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:54 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:54.162+0000 7fed19b1d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:44:54.470 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:54 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:54.173+0000 7f5c37796040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:44:54.470 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:54 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:54.234+0000 7f5c37796040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:44:54.470 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:54 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:54.402+0000 7f5c37796040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:44:54.470 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:54 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:54.469+0000 7f5c37796040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:44:54.742 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:54 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:54.538+0000 7f5c37796040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:44:54.742 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:54 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:54.622+0000 7f5c37796040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:44:54.742 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:54 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:54.681+0000 7f5c37796040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:44:54.742 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:54 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:54.741+0000 7f5c37796040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:44:54.787 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:54 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:54.348+0000 7fed19b1d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:44:55.537 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:55.233+0000 7fed19b1d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:44:55.537 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:55.299+0000 7fed19b1d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:44:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:44:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T22:44:55.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:44:55.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:44:55.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:55.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: Reconfiguring daemon mgr.x on smithi152 2024-09-11T22:44:55.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/953549849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cd917b5-77f6-4a4a-8fd0-4ebe3034feac"}]: dispatch 2024-09-11T22:44:55.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cd917b5-77f6-4a4a-8fd0-4ebe3034feac"}]: dispatch 2024-09-11T22:44:55.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cd917b5-77f6-4a4a-8fd0-4ebe3034feac"}]': finished 2024-09-11T22:44:55.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T22:44:55.540 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:44:55.540 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:55.540 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.540 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:55 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2092182195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:44:55.796 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:44:55.523+0000 7f5c37796040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:44:55.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:55.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:44:55.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T22:44:55.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:44:55.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:44:55.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:55.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: Reconfiguring daemon mgr.x on smithi152 2024-09-11T22:44:55.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/953549849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cd917b5-77f6-4a4a-8fd0-4ebe3034feac"}]: dispatch 2024-09-11T22:44:55.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cd917b5-77f6-4a4a-8fd0-4ebe3034feac"}]: dispatch 2024-09-11T22:44:55.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cd917b5-77f6-4a4a-8fd0-4ebe3034feac"}]': finished 2024-09-11T22:44:55.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T22:44:55.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:44:55.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:55.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.799 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:55 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2092182195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:44:55.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:55.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:44:55.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T22:44:55.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:44:55.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:44:55.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:44:55.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: Reconfiguring daemon mgr.x on smithi152 2024-09-11T22:44:55.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/953549849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cd917b5-77f6-4a4a-8fd0-4ebe3034feac"}]: dispatch 2024-09-11T22:44:55.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3cd917b5-77f6-4a4a-8fd0-4ebe3034feac"}]: dispatch 2024-09-11T22:44:55.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3cd917b5-77f6-4a4a-8fd0-4ebe3034feac"}]': finished 2024-09-11T22:44:55.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T22:44:55.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:44:55.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:55.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:44:55.875 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:55 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2092182195' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:44:56.261 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:56.005+0000 7fed19b1d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:44:56.261 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:56.119+0000 7fed19b1d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:44:56.261 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:56.260+0000 7fed19b1d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:44:56.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:56.421+0000 7f638affd640 1 -- 172.21.15.152:0/1059037384 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7f638c00ad10 con 0x7f6394102690 2024-09-11T22:44:56.535 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:56 smithi191 ceph-mon[35211]: Standby manager daemon z started 2024-09-11T22:44:56.535 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:56 smithi191 ceph-mon[35211]: from='mgr.? 172.21.15.204:0/3783196148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T22:44:56.535 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:56 smithi191 ceph-mon[35211]: from='mgr.? 172.21.15.204:0/3783196148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:44:56.535 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:56 smithi191 ceph-mon[35211]: from='mgr.? 172.21.15.204:0/3783196148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T22:44:56.535 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:56 smithi191 ceph-mon[35211]: from='mgr.? 172.21.15.204:0/3783196148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:44:56.536 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:56.320+0000 7fed19b1d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:44:56.536 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:56.468+0000 7fed19b1d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:44:56.536 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:56.533+0000 7fed19b1d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:44:56.788 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:56.592+0000 7fed19b1d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:44:56.788 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:56.747+0000 7fed19b1d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:44:56.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:56 smithi204 ceph-mon[35454]: Standby manager daemon z started 2024-09-11T22:44:56.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:56 smithi204 ceph-mon[35454]: from='mgr.? 172.21.15.204:0/3783196148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T22:44:56.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:56 smithi204 ceph-mon[35454]: from='mgr.? 172.21.15.204:0/3783196148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:44:56.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:56 smithi204 ceph-mon[35454]: from='mgr.? 172.21.15.204:0/3783196148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T22:44:56.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:56 smithi204 ceph-mon[35454]: from='mgr.? 172.21.15.204:0/3783196148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:44:56.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:56 smithi152 ceph-mon[31181]: Standby manager daemon z started 2024-09-11T22:44:56.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:56 smithi152 ceph-mon[31181]: from='mgr.? 172.21.15.204:0/3783196148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T22:44:56.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:56 smithi152 ceph-mon[31181]: from='mgr.? 172.21.15.204:0/3783196148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:44:56.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:56 smithi152 ceph-mon[31181]: from='mgr.? 172.21.15.204:0/3783196148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T22:44:56.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:56 smithi152 ceph-mon[31181]: from='mgr.? 172.21.15.204:0/3783196148' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:44:57.076 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:56.812+0000 7fed19b1d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:44:57.076 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:56.877+0000 7fed19b1d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:44:57.076 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:56.958+0000 7fed19b1d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:44:57.076 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:57 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:57.016+0000 7fed19b1d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:44:57.077 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:57 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:57.074+0000 7fed19b1d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:44:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:57 smithi191 ceph-mon[35211]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-11T22:44:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:57 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T22:44:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:57 smithi191 ceph-mon[35211]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:57.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:57 smithi204 ceph-mon[35454]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-11T22:44:57.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:57 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T22:44:57.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:57 smithi204 ceph-mon[35454]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:57.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:57 smithi152 ceph-mon[31181]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-11T22:44:57.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:57 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T22:44:57.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:57 smithi152 ceph-mon[31181]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:58.287 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:44:57 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:44:57.850+0000 7fed19b1d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:44:58.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:44:58.432+0000 7f638affd640 1 -- 172.21.15.152:0/1059037384 <== mon.0 v2:172.21.15.152:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f638c01f340 con 0x7f6394102690 2024-09-11T22:44:58.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:58 smithi191 ceph-mon[35211]: Standby manager daemon y started 2024-09-11T22:44:58.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:58 smithi191 ceph-mon[35211]: from='mgr.? 172.21.15.191:0/1045630610' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T22:44:58.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:58 smithi191 ceph-mon[35211]: from='mgr.? 172.21.15.191:0/1045630610' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:44:58.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:58 smithi191 ceph-mon[35211]: from='mgr.? 172.21.15.191:0/1045630610' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T22:44:58.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:58 smithi191 ceph-mon[35211]: from='mgr.? 172.21.15.191:0/1045630610' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:44:58.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:58 smithi204 ceph-mon[35454]: Standby manager daemon y started 2024-09-11T22:44:58.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:58 smithi204 ceph-mon[35454]: from='mgr.? 172.21.15.191:0/1045630610' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T22:44:58.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:58 smithi204 ceph-mon[35454]: from='mgr.? 172.21.15.191:0/1045630610' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:44:58.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:58 smithi204 ceph-mon[35454]: from='mgr.? 172.21.15.191:0/1045630610' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T22:44:58.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:58 smithi204 ceph-mon[35454]: from='mgr.? 172.21.15.191:0/1045630610' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:44:58.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:58 smithi152 ceph-mon[31181]: Standby manager daemon y started 2024-09-11T22:44:58.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:58 smithi152 ceph-mon[31181]: from='mgr.? 172.21.15.191:0/1045630610' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T22:44:58.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:58 smithi152 ceph-mon[31181]: from='mgr.? 172.21.15.191:0/1045630610' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:44:58.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:58 smithi152 ceph-mon[31181]: from='mgr.? 172.21.15.191:0/1045630610' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T22:44:58.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:58 smithi152 ceph-mon[31181]: from='mgr.? 172.21.15.191:0/1045630610' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:44:59.702 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:59 smithi152 ceph-mon[31181]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-11T22:44:59.702 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:59 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T22:44:59.702 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:44:59 smithi152 ceph-mon[31181]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:59.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:59 smithi191 ceph-mon[35211]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-11T22:44:59.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:59 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T22:44:59.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:44:59 smithi191 ceph-mon[35211]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:44:59.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:59 smithi204 ceph-mon[35454]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-11T22:44:59.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:59 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T22:44:59.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:44:59 smithi204 ceph-mon[35454]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:02.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:01 smithi191 ceph-mon[35211]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:02.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:01 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:02.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:01 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:02.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:01 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:45:02.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:01 smithi204 ceph-mon[35454]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:02.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:01 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:02.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:01 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:02.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:01 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:45:02.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:01 smithi152 ceph-mon[31181]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:02.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:01 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:02.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:01 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:02.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:01 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:45:02.865 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:02 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:02.865 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:02 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:45:02.865 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:02 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:03.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:02 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:03.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:02 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:45:03.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:02 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:03.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:02 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:03.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:02 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:45:03.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:02 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:04.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:03 smithi191 ceph-mon[35211]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:04.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:03 smithi204 ceph-mon[35454]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:04.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:03 smithi152 ceph-mon[31181]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:04.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:04 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T22:45:04.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:04 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:04.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:04 smithi152 ceph-mon[31181]: Deploying daemon osd.0 on smithi152 2024-09-11T22:45:05.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:04 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T22:45:05.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:04 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:05.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:04 smithi191 ceph-mon[35211]: Deploying daemon osd.0 on smithi152 2024-09-11T22:45:05.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:04 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T22:45:05.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:04 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:05.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:04 smithi204 ceph-mon[35454]: Deploying daemon osd.0 on smithi152 2024-09-11T22:45:05.789 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:05 smithi152 ceph-mon[31181]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:06.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:05 smithi191 ceph-mon[35211]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:06.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:05 smithi204 ceph-mon[35454]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:08.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:07 smithi191 ceph-mon[35211]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:08.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:07 smithi204 ceph-mon[35454]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:08.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:07 smithi152 ceph-mon[31181]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:10.013 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:09 smithi152 ceph-mon[31181]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:10.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:09 smithi191 ceph-mon[35211]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:09 smithi204 ceph-mon[35454]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:10.982 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:10 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:45:10.982 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:10 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:10.982 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:10 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:11.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:10 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:45:11.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:10 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:11.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:10 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:11.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:10 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:45:11.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:10 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:11.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:10 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:11.988 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:11 smithi152 ceph-mon[31181]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:12.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:11 smithi191 ceph-mon[35211]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:11 smithi204 ceph-mon[35454]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:14.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:13 smithi191 ceph-mon[35211]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:14.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:13 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:14.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:13 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:14.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:13 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:14.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:13 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:45:14.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:13 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:14.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:13 smithi204 ceph-mon[35454]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:14.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:13 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:14.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:13 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:14.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:13 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:14.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:13 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:45:14.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:13 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:14.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:13 smithi152 ceph-mon[31181]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:14.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:14.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:14.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:14.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:45:14.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:14.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:45:14.495+0000 7f638affd640 1 -- 172.21.15.152:0/1059037384 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6394062630 con 0x7f63800392c0 2024-09-11T22:45:14.497 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 0 on host 'smithi152' 2024-09-11T22:45:14.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:45:14.497+0000 7f639be9a640 1 -- 172.21.15.152:0/1059037384 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f63800392c0 msgr2=0x7f638003b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:45:14.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:45:14.497+0000 7f639be9a640 1 --2- 172.21.15.152:0/1059037384 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f63800392c0 0x7f638003b780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6390005fd0 tx=0x7f639000b410 comp rx=0 tx=0).stop 2024-09-11T22:45:14.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:45:14.498+0000 7f639be9a640 1 -- 172.21.15.152:0/1059037384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6394102690 msgr2=0x7f639419f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:45:14.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:45:14.498+0000 7f639be9a640 1 --2- 172.21.15.152:0/1059037384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6394102690 0x7f639419f4c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f638c00eb10 tx=0x7f638c00c2d0 comp rx=0 tx=0).stop 2024-09-11T22:45:14.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:45:14.498+0000 7f639be9a640 1 -- 172.21.15.152:0/1059037384 shutdown_connections 2024-09-11T22:45:14.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:45:14.498+0000 7f639be9a640 1 --2- 172.21.15.152:0/1059037384 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f63800392c0 0x7f638003b780 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:14.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:45:14.498+0000 7f639be9a640 1 --2- 172.21.15.152:0/1059037384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6394102690 0x7f639419f4c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:14.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:45:14.498+0000 7f639be9a640 1 --2- 172.21.15.152:0/1059037384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6394101cd0 0x7f6394198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:14.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:45:14.498+0000 7f639be9a640 1 --2- 172.21.15.152:0/1059037384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6394100ad0 0x7f6394197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:14.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:45:14.498+0000 7f639be9a640 1 -- 172.21.15.152:0/1059037384 >> 172.21.15.152:0/1059037384 conn(0x7f63940fc280 msgr2=0x7f63941050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:45:14.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:45:14.498+0000 7f639be9a640 1 -- 172.21.15.152:0/1059037384 shutdown_connections 2024-09-11T22:45:14.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:45:14.498+0000 7f639be9a640 1 -- 172.21.15.152:0/1059037384 wait complete. 2024-09-11T22:45:15.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:14 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:45:15.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:14 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:15.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:14 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:15.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:14 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:45:15.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:14 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:15.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:14 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:15.109 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:14 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:45:15.110 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:14 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:15.110 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:14 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:15.136 DEBUG:teuthology.orchestra.run.smithi152:osd.0> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T22:45:15.139 INFO:tasks.cephadm:Deploying osd.1 on smithi191 with /dev/vg_nvme/lv_4... 2024-09-11T22:45:15.139 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T22:45:15.381 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:45:16.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:15 smithi191 ceph-mon[35211]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:16.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:15 smithi204 ceph-mon[35454]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:16.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:15 smithi152 ceph-mon[31181]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:16.550 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 22:45:16 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-0[42707]: 2024-09-11T22:45:16.297+0000 7f76b8726640 -1 osd.0 0 log_to_monitors true 2024-09-11T22:45:16.820 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:16 smithi152 ceph-mon[31181]: from='osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T22:45:16.956 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:16 smithi191 ceph-mon[35211]: from='osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T22:45:17.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:16 smithi204 ceph-mon[35454]: from='osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T22:45:17.662 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:45:17.695 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi191:vg_nvme/lv_4 2024-09-11T22:45:17.959 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:45:18.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:17 smithi191 ceph-mon[35211]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:18.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:17 smithi191 ceph-mon[35211]: from='osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T22:45:18.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:17 smithi191 ceph-mon[35211]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T22:45:18.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:17 smithi191 ceph-mon[35211]: from='osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-09-11T22:45:18.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:45:18.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:17 smithi204 ceph-mon[35454]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:18.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:17 smithi204 ceph-mon[35454]: from='osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T22:45:18.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:17 smithi204 ceph-mon[35454]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T22:45:18.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:17 smithi204 ceph-mon[35454]: from='osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-09-11T22:45:18.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:45:18.092 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:17 smithi152 ceph-mon[31181]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:18.093 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:17 smithi152 ceph-mon[31181]: from='osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T22:45:18.093 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:17 smithi152 ceph-mon[31181]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T22:45:18.093 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:17 smithi152 ceph-mon[31181]: from='osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-09-11T22:45:18.093 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:45:18.372 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-0[42707]: 2024-09-11T22:45:18.248+0000 7f76b46a9640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T22:45:18.372 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-0[42707]: 2024-09-11T22:45:18.254+0000 7f76ac4c9640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:45:18.471 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.462+0000 7f6acde00640 1 -- 172.21.15.191:0/4130054681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ac8100b10 msgr2=0x7f6ac8100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:45:18.471 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.462+0000 7f6acde00640 1 --2- 172.21.15.191:0/4130054681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ac8100b10 0x7f6ac8100f10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8009900 tx=0x7f6ab802a410 comp rx=0 tx=0).stop 2024-09-11T22:45:18.471 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.462+0000 7f6acde00640 1 -- 172.21.15.191:0/4130054681 shutdown_connections 2024-09-11T22:45:18.471 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.462+0000 7f6acde00640 1 --2- 172.21.15.191:0/4130054681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac81026d0 0x7f6ac8108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:18.471 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.462+0000 7f6acde00640 1 --2- 172.21.15.191:0/4130054681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac8101d10 0x7f6ac8102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:18.471 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.462+0000 7f6acde00640 1 --2- 172.21.15.191:0/4130054681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ac8100b10 0x7f6ac8100f10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:18.472 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.462+0000 7f6acde00640 1 -- 172.21.15.191:0/4130054681 >> 172.21.15.191:0/4130054681 conn(0x7f6ac80fc2a0 msgr2=0x7f6ac80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:45:18.472 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.462+0000 7f6acde00640 1 -- 172.21.15.191:0/4130054681 shutdown_connections 2024-09-11T22:45:18.472 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.462+0000 7f6acde00640 1 -- 172.21.15.191:0/4130054681 wait complete. 2024-09-11T22:45:18.472 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.463+0000 7f6acde00640 1 Processor -- start 2024-09-11T22:45:18.472 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.463+0000 7f6acde00640 1 -- start start 2024-09-11T22:45:18.472 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.463+0000 7f6acde00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8100b10 0x7f6ac8104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:45:18.472 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.463+0000 7f6acde00640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ac8101d10 0x7f6ac81030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:45:18.472 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.463+0000 7f6acde00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac81026d0 0x7f6ac8103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:45:18.472 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.463+0000 7f6acde00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac810c790 con 0x7f6ac8100b10 2024-09-11T22:45:18.472 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.463+0000 7f6acde00640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac810c610 con 0x7f6ac8101d10 2024-09-11T22:45:18.472 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.463+0000 7f6acde00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6ac810c910 con 0x7f6ac81026d0 2024-09-11T22:45:18.472 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.463+0000 7f6abffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ac8101d10 0x7f6ac81030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:45:18.472 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.463+0000 7f6abffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ac8101d10 0x7f6ac81030f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:43176/0 (socket says 172.21.15.191:43176) 2024-09-11T22:45:18.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.463+0000 7f6abffff640 1 -- 172.21.15.191:0/3211411302 learned_addr learned my addr 172.21.15.191:0/3211411302 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-11T22:45:18.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.463+0000 7f6ac77fe640 1 --2- 172.21.15.191:0/3211411302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8100b10 0x7f6ac8104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:45:18.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.463+0000 7f6ac7fff640 1 --2- 172.21.15.191:0/3211411302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac81026d0 0x7f6ac8103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:45:18.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.464+0000 7f6abffff640 1 -- 172.21.15.191:0/3211411302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac81026d0 msgr2=0x7f6ac8103630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:45:18.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.464+0000 7f6abffff640 1 --2- 172.21.15.191:0/3211411302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac81026d0 0x7f6ac8103630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:18.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.464+0000 7f6abffff640 1 -- 172.21.15.191:0/3211411302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8100b10 msgr2=0x7f6ac8104aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:45:18.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.464+0000 7f6abffff640 1 --2- 172.21.15.191:0/3211411302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8100b10 0x7f6ac8104aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:18.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.464+0000 7f6abffff640 1 -- 172.21.15.191:0/3211411302 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ac8103ef0 con 0x7f6ac8101d10 2024-09-11T22:45:18.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.464+0000 7f6ac7fff640 1 --2- 172.21.15.191:0/3211411302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac81026d0 0x7f6ac8103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:45:18.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.464+0000 7f6ac77fe640 1 --2- 172.21.15.191:0/3211411302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8100b10 0x7f6ac8104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:45:18.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.464+0000 7f6abffff640 1 --2- 172.21.15.191:0/3211411302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ac8101d10 0x7f6ac81030f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000d770 tx=0x7f6ab000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:45:18.473 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.464+0000 7f6ac57fa640 1 -- 172.21.15.191:0/3211411302 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0017070 con 0x7f6ac8101d10 2024-09-11T22:45:18.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.464+0000 7f6ac57fa640 1 -- 172.21.15.191:0/3211411302 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ab0005500 con 0x7f6ac8101d10 2024-09-11T22:45:18.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.464+0000 7f6ac57fa640 1 -- 172.21.15.191:0/3211411302 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0004e00 con 0x7f6ac8101d10 2024-09-11T22:45:18.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.464+0000 7f6acde00640 1 -- 172.21.15.191:0/3211411302 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ac81a20d0 con 0x7f6ac8101d10 2024-09-11T22:45:18.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.464+0000 7f6acde00640 1 -- 172.21.15.191:0/3211411302 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ac81a2590 con 0x7f6ac8101d10 2024-09-11T22:45:18.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.465+0000 7f6acde00640 1 -- 172.21.15.191:0/3211411302 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ac81a27c0 con 0x7f6ac8101d10 2024-09-11T22:45:18.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.466+0000 7f6ac57fa640 1 -- 172.21.15.191:0/3211411302 <== mon.2 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f6ab0004fa0 con 0x7f6ac8101d10 2024-09-11T22:45:18.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.467+0000 7f6ac57fa640 1 --2- 172.21.15.191:0/3211411302 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f6ab40a2af0 0x7f6ab40a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:45:18.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.467+0000 7f6ac57fa640 1 -- 172.21.15.191:0/3211411302 <== mon.2 v2:172.21.15.191:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f6ab00cd260 con 0x7f6ac8101d10 2024-09-11T22:45:18.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.467+0000 7f6ac77fe640 1 --2- 172.21.15.191:0/3211411302 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f6ab40a2af0 0x7f6ab40a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:45:18.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.467+0000 7f6ac77fe640 1 --2- 172.21.15.191:0/3211411302 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f6ab40a2af0 0x7f6ab40a4fb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8005ab0 tx=0x7f6ab8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:45:18.474 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.470+0000 7f6ac57fa640 1 -- 172.21.15.191:0/3211411302 <== mon.2 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ab009bd50 con 0x7f6ac8101d10 2024-09-11T22:45:18.644 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:18.644+0000 7f6acde00640 1 -- 172.21.15.191:0/3211411302 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6ac806af10 con 0x7f6ab40a2af0 2024-09-11T22:45:19.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:18 smithi191 ceph-mon[35211]: from='osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-09-11T22:45:19.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:18 smithi191 ceph-mon[35211]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T22:45:19.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:18 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:45:19.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:18 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:45:19.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:18 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:19.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:18 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:19.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:18 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:45:19.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:18 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:19.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:18 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:19.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:18 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:45:19.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:18 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:19.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:18 smithi191 ceph-mon[35211]: from='osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007]' entity='osd.0' 2024-09-11T22:45:19.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:18 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:45:19.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:18 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:45:19.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:18 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:19.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:18 smithi204 ceph-mon[35454]: from='osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-09-11T22:45:19.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:18 smithi204 ceph-mon[35454]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T22:45:19.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:18 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:45:19.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:18 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:45:19.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:18 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:19.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:18 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:19.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:18 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:45:19.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:18 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:19.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:18 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:19.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:18 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:45:19.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:18 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:19.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:18 smithi204 ceph-mon[35454]: from='osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007]' entity='osd.0' 2024-09-11T22:45:19.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:18 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:45:19.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:18 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:45:19.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:18 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-mon[31181]: from='osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-09-11T22:45:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-mon[31181]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T22:45:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:45:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:45:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:45:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:19.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:45:19.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:19.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-mon[31181]: from='osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007]' entity='osd.0' 2024-09-11T22:45:19.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:45:19.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:45:19.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:20.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:19 smithi204 ceph-mon[35454]: purged_snaps scrub starts 2024-09-11T22:45:20.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:19 smithi204 ceph-mon[35454]: purged_snaps scrub ok 2024-09-11T22:45:20.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:19 smithi204 ceph-mon[35454]: Detected new or changed devices on smithi152 2024-09-11T22:45:20.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:19 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi152 to 17088M 2024-09-11T22:45:20.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:19 smithi204 ceph-mon[35454]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:45:20.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:19 smithi204 ceph-mon[35454]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:20.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:19 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:45:20.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:19 smithi204 ceph-mon[35454]: osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007] boot 2024-09-11T22:45:20.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:19 smithi204 ceph-mon[35454]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T22:45:20.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:19 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:45:20.058 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:19 smithi191 ceph-mon[35211]: purged_snaps scrub starts 2024-09-11T22:45:20.058 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:19 smithi191 ceph-mon[35211]: purged_snaps scrub ok 2024-09-11T22:45:20.058 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:19 smithi191 ceph-mon[35211]: Detected new or changed devices on smithi152 2024-09-11T22:45:20.058 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:19 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi152 to 17088M 2024-09-11T22:45:20.058 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:19 smithi191 ceph-mon[35211]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:45:20.058 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:19 smithi191 ceph-mon[35211]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:20.059 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:19 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:45:20.059 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:19 smithi191 ceph-mon[35211]: osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007] boot 2024-09-11T22:45:20.059 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:19 smithi191 ceph-mon[35211]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T22:45:20.059 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:19 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:45:20.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:19 smithi152 ceph-mon[31181]: purged_snaps scrub starts 2024-09-11T22:45:20.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:19 smithi152 ceph-mon[31181]: purged_snaps scrub ok 2024-09-11T22:45:20.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:19 smithi152 ceph-mon[31181]: Detected new or changed devices on smithi152 2024-09-11T22:45:20.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:19 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi152 to 17088M 2024-09-11T22:45:20.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:19 smithi152 ceph-mon[31181]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:45:20.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:19 smithi152 ceph-mon[31181]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:45:20.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:19 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:45:20.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:19 smithi152 ceph-mon[31181]: osd.0 [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007] boot 2024-09-11T22:45:20.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:19 smithi152 ceph-mon[31181]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T22:45:20.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:19 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:45:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:21 smithi152 ceph-mon[31181]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:21 smithi152 ceph-mon[31181]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T22:45:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:21 smithi152 ceph-mon[31181]: from='client.? 172.21.15.191:0/2440931590' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae589bc4-77b1-4522-9591-b0ce2a5432dc"}]: dispatch 2024-09-11T22:45:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:21 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae589bc4-77b1-4522-9591-b0ce2a5432dc"}]: dispatch 2024-09-11T22:45:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:21 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae589bc4-77b1-4522-9591-b0ce2a5432dc"}]': finished 2024-09-11T22:45:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:21 smithi152 ceph-mon[31181]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T22:45:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:45:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:21 smithi152 ceph-mon[31181]: from='client.? 172.21.15.191:0/3950471357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:45:22.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:21 smithi191 ceph-mon[35211]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:22.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:21 smithi191 ceph-mon[35211]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T22:45:22.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:21 smithi191 ceph-mon[35211]: from='client.? 172.21.15.191:0/2440931590' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae589bc4-77b1-4522-9591-b0ce2a5432dc"}]: dispatch 2024-09-11T22:45:22.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:21 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae589bc4-77b1-4522-9591-b0ce2a5432dc"}]: dispatch 2024-09-11T22:45:22.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:21 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae589bc4-77b1-4522-9591-b0ce2a5432dc"}]': finished 2024-09-11T22:45:22.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:21 smithi191 ceph-mon[35211]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T22:45:22.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:21 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:45:22.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:21 smithi191 ceph-mon[35211]: from='client.? 172.21.15.191:0/3950471357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:45:22.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:21 smithi204 ceph-mon[35454]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:22.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:21 smithi204 ceph-mon[35454]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T22:45:22.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:21 smithi204 ceph-mon[35454]: from='client.? 172.21.15.191:0/2440931590' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae589bc4-77b1-4522-9591-b0ce2a5432dc"}]: dispatch 2024-09-11T22:45:22.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:21 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae589bc4-77b1-4522-9591-b0ce2a5432dc"}]: dispatch 2024-09-11T22:45:22.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:21 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae589bc4-77b1-4522-9591-b0ce2a5432dc"}]': finished 2024-09-11T22:45:22.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:21 smithi204 ceph-mon[35454]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T22:45:22.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:21 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:45:22.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:21 smithi204 ceph-mon[35454]: from='client.? 172.21.15.191:0/3950471357' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:45:24.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:23 smithi152 ceph-mon[31181]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:24.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:23 smithi191 ceph-mon[35211]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:24.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:23 smithi204 ceph-mon[35454]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:25.916 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:25 smithi191 ceph-mon[35211]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:26.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:25 smithi152 ceph-mon[31181]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:26.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:25 smithi204 ceph-mon[35454]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:27.960 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:27 smithi191 ceph-mon[35211]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:28.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:27 smithi152 ceph-mon[31181]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:28.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:27 smithi204 ceph-mon[35454]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:29.953 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:29 smithi191 ceph-mon[35211]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:29.953 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:29 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T22:45:29.953 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:29 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:30.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:29 smithi152 ceph-mon[31181]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:30.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:29 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T22:45:30.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:29 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:30.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:29 smithi204 ceph-mon[35454]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:30.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:29 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T22:45:30.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:29 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:30.874 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:30 smithi191 ceph-mon[35211]: Deploying daemon osd.1 on smithi191 2024-09-11T22:45:31.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:30 smithi152 ceph-mon[31181]: Deploying daemon osd.1 on smithi191 2024-09-11T22:45:31.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:30 smithi204 ceph-mon[35454]: Deploying daemon osd.1 on smithi191 2024-09-11T22:45:32.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:31 smithi191 ceph-mon[35211]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:32.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:31 smithi152 ceph-mon[31181]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:32.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:31 smithi204 ceph-mon[35454]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:34.098 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:33 smithi204 ceph-mon[35454]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:34.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:33 smithi152 ceph-mon[31181]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:34.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:33 smithi191 ceph-mon[35211]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:36.090 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:35 smithi191 ceph-mon[35211]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:36.090 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:35 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:45:36.091 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:35 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:36.091 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:35 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:36.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:35 smithi152 ceph-mon[31181]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:36.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:35 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:45:36.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:35 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:36.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:35 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:36.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:35 smithi204 ceph-mon[35454]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:36.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:35 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:45:36.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:35 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:36.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:35 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:38.022 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:37 smithi191 ceph-mon[35211]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:38.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:37 smithi152 ceph-mon[31181]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:38.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:37 smithi204 ceph-mon[35454]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:39.433 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:39 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:39.434 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:39 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:39.434 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:39 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:39.434 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:39 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:45:39.434 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:39 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:39.434 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:39 smithi191 ceph-mon[35211]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:39.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:39 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:39.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:39 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:39.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:39 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:39.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:39 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:45:39.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:39 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:39.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:39 smithi204 ceph-mon[35454]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:39.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:39 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:39.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:39 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:39.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:39 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:39.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:39 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:45:39.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:39 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:39.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:39 smithi152 ceph-mon[31181]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:39.749 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:39.748+0000 7f6ac57fa640 1 -- 172.21.15.191:0/3211411302 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6ac806af10 con 0x7f6ab40a2af0 2024-09-11T22:45:39.749 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 1 on host 'smithi191' 2024-09-11T22:45:39.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:39.750+0000 7f6acde00640 1 -- 172.21.15.191:0/3211411302 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f6ab40a2af0 msgr2=0x7f6ab40a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:45:39.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:39.750+0000 7f6acde00640 1 --2- 172.21.15.191:0/3211411302 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f6ab40a2af0 0x7f6ab40a4fb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8005ab0 tx=0x7f6ab8000bc0 comp rx=0 tx=0).stop 2024-09-11T22:45:39.751 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:39.750+0000 7f6acde00640 1 -- 172.21.15.191:0/3211411302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ac8101d10 msgr2=0x7f6ac81030f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:45:39.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:39.750+0000 7f6acde00640 1 --2- 172.21.15.191:0/3211411302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ac8101d10 0x7f6ac81030f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000d770 tx=0x7f6ab000dc30 comp rx=0 tx=0).stop 2024-09-11T22:45:39.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:39.750+0000 7f6acde00640 1 -- 172.21.15.191:0/3211411302 shutdown_connections 2024-09-11T22:45:39.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:39.750+0000 7f6acde00640 1 --2- 172.21.15.191:0/3211411302 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f6ab40a2af0 0x7f6ab40a4fb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:39.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:39.750+0000 7f6acde00640 1 --2- 172.21.15.191:0/3211411302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ac81026d0 0x7f6ac8103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:39.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:39.750+0000 7f6acde00640 1 --2- 172.21.15.191:0/3211411302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ac8101d10 0x7f6ac81030f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:39.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:39.750+0000 7f6acde00640 1 --2- 172.21.15.191:0/3211411302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8100b10 0x7f6ac8104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:39.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:39.750+0000 7f6acde00640 1 -- 172.21.15.191:0/3211411302 >> 172.21.15.191:0/3211411302 conn(0x7f6ac80fc2a0 msgr2=0x7f6ac806a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:45:39.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:39.751+0000 7f6acde00640 1 -- 172.21.15.191:0/3211411302 shutdown_connections 2024-09-11T22:45:39.752 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:45:39.751+0000 7f6acde00640 1 -- 172.21.15.191:0/3211411302 wait complete. 2024-09-11T22:45:40.340 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:40 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:45:40.340 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:40 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:40.340 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:40 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:40.363 DEBUG:teuthology.orchestra.run.smithi191:osd.1> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T22:45:40.366 INFO:tasks.cephadm:Deploying osd.2 on smithi204 with /dev/vg_nvme/lv_4... 2024-09-11T22:45:40.366 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T22:45:40.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:40 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:45:40.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:40 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:40.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:40 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:40.615 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:45:40.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:40 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:45:40.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:40 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:40.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:40 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:41.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:41 smithi204 ceph-mon[35454]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:41.411 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:41 smithi191 ceph-mon[35211]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:41.411 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 22:45:41 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-1[39672]: 2024-09-11T22:45:41.214+0000 7fb62beb6640 -1 osd.1 0 log_to_monitors true 2024-09-11T22:45:41.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:41 smithi152 ceph-mon[31181]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:42.453 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:42 smithi204 ceph-mon[35454]: from='osd.1 [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:45:42.453 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:42 smithi204 ceph-mon[35454]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:45:42.467 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:42 smithi191 ceph-mon[35211]: from='osd.1 [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:45:42.467 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:42 smithi191 ceph-mon[35211]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:45:42.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:42 smithi152 ceph-mon[31181]: from='osd.1 [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:45:42.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:42 smithi152 ceph-mon[31181]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:45:42.831 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:45:42.858 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_4 2024-09-11T22:45:43.109 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:45:43.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:43 smithi204 ceph-mon[35454]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T22:45:43.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:43 smithi204 ceph-mon[35454]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T22:45:43.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:45:43.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:43 smithi204 ceph-mon[35454]: from='osd.1 [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:45:43.465 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:43 smithi204 ceph-mon[35454]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:45:43.466 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:43 smithi204 ceph-mon[35454]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:43.466 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:43.466 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:43.466 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:45:43.466 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:43.466 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:43.467 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:45:43.467 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:43.537 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 22:45:43 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-1[39672]: 2024-09-11T22:45:43.216+0000 7fb627e39640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T22:45:43.537 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 22:45:43 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-1[39672]: 2024-09-11T22:45:43.219+0000 7fb623c61640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:45:43.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:43 smithi191 ceph-mon[35211]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T22:45:43.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:43 smithi191 ceph-mon[35211]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T22:45:43.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:45:43.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:43 smithi191 ceph-mon[35211]: from='osd.1 [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:45:43.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:43 smithi191 ceph-mon[35211]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:45:43.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:43 smithi191 ceph-mon[35211]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:43.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:43.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:43.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:45:43.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:43.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:43.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:45:43.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:43.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.589+0000 7f8691ed4640 1 -- 172.21.15.204:0/3566707153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c107940 msgr2=0x7f868c109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:45:43.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.589+0000 7f8691ed4640 1 --2- 172.21.15.204:0/3566707153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c107940 0x7f868c109d30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8684009ca0 tx=0x7f868402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:45:43.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.589+0000 7f8691ed4640 1 -- 172.21.15.204:0/3566707153 shutdown_connections 2024-09-11T22:45:43.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.589+0000 7f8691ed4640 1 --2- 172.21.15.204:0/3566707153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c107940 0x7f868c109d30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:43.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.589+0000 7f8691ed4640 1 --2- 172.21.15.204:0/3566707153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f868c105010 0x7f868c107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:43.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.589+0000 7f8691ed4640 1 --2- 172.21.15.204:0/3566707153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f868c1026e0 0x7f868c104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:43.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.589+0000 7f8691ed4640 1 -- 172.21.15.204:0/3566707153 >> 172.21.15.204:0/3566707153 conn(0x7f868c0fc280 msgr2=0x7f868c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:45:43.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.589+0000 7f8691ed4640 1 -- 172.21.15.204:0/3566707153 shutdown_connections 2024-09-11T22:45:43.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.589+0000 7f8691ed4640 1 -- 172.21.15.204:0/3566707153 wait complete. 2024-09-11T22:45:43.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f8691ed4640 1 Processor -- start 2024-09-11T22:45:43.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f8691ed4640 1 -- start start 2024-09-11T22:45:43.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f8691ed4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c1026e0 0x7f868c197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:45:43.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f8691ed4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f868c105010 0x7f868c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:45:43.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f8691ed4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f868c107940 0x7f868c19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:45:43.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f8691ed4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f868c10d860 con 0x7f868c107940 2024-09-11T22:45:43.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f8691ed4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f868c10d6e0 con 0x7f868c105010 2024-09-11T22:45:43.595 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f8691ed4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f868c10d9e0 con 0x7f868c1026e0 2024-09-11T22:45:43.595 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f868b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c1026e0 0x7f868c197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:45:43.595 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f868b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c1026e0 0x7f868c197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:47708/0 (socket says 172.21.15.204:47708) 2024-09-11T22:45:43.595 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f868b7fe640 1 -- 172.21.15.204:0/491038075 learned_addr learned my addr 172.21.15.204:0/491038075 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-11T22:45:43.595 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f8683fff640 1 --2- 172.21.15.204:0/491038075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f868c105010 0x7f868c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:45:43.595 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f868b7fe640 1 -- 172.21.15.204:0/491038075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f868c105010 msgr2=0x7f868c1982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:45:43.595 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f868b7fe640 1 --2- 172.21.15.204:0/491038075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f868c105010 0x7f868c1982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:43.595 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f868bfff640 1 --2- 172.21.15.204:0/491038075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f868c107940 0x7f868c19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:45:43.595 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f868b7fe640 1 -- 172.21.15.204:0/491038075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f868c107940 msgr2=0x7f868c19f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:45:43.595 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f868b7fe640 1 --2- 172.21.15.204:0/491038075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f868c107940 0x7f868c19f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:45:43.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f868b7fe640 1 -- 172.21.15.204:0/491038075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c19f9d0 con 0x7f868c1026e0 2024-09-11T22:45:43.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.590+0000 7f868b7fe640 1 --2- 172.21.15.204:0/491038075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c1026e0 0x7f868c197db0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f867c00c920 tx=0x7f867c00cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:45:43.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.591+0000 7f86897fa640 1 -- 172.21.15.204:0/491038075 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c007bb0 con 0x7f868c1026e0 2024-09-11T22:45:43.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.591+0000 7f86897fa640 1 -- 172.21.15.204:0/491038075 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f867c007d50 con 0x7f868c1026e0 2024-09-11T22:45:43.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.591+0000 7f8691ed4640 1 -- 172.21.15.204:0/491038075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868c19fcc0 con 0x7f868c1026e0 2024-09-11T22:45:43.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.591+0000 7f86897fa640 1 -- 172.21.15.204:0/491038075 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c0187d0 con 0x7f868c1026e0 2024-09-11T22:45:43.596 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.591+0000 7f8691ed4640 1 -- 172.21.15.204:0/491038075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f868c1a01a0 con 0x7f868c1026e0 2024-09-11T22:45:43.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.592+0000 7f8691ed4640 1 -- 172.21.15.204:0/491038075 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f868c0fdd70 con 0x7f868c1026e0 2024-09-11T22:45:43.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.593+0000 7f86897fa640 1 -- 172.21.15.204:0/491038075 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f867c00f450 con 0x7f868c1026e0 2024-09-11T22:45:43.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.594+0000 7f86897fa640 1 --2- 172.21.15.204:0/491038075 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f86780a2b70 0x7f86780a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:45:43.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.594+0000 7f86897fa640 1 -- 172.21.15.204:0/491038075 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f867c0cda10 con 0x7f868c1026e0 2024-09-11T22:45:43.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.594+0000 7f8683fff640 1 --2- 172.21.15.204:0/491038075 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f86780a2b70 0x7f86780a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:45:43.597 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.594+0000 7f8683fff640 1 --2- 172.21.15.204:0/491038075 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f86780a2b70 0x7f86780a5030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f868c06aa50 tx=0x7f867400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:45:43.598 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.597+0000 7f86897fa640 1 -- 172.21.15.204:0/491038075 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f867c014030 con 0x7f868c1026e0 2024-09-11T22:45:43.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:43 smithi152 ceph-mon[31181]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T22:45:43.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:43 smithi152 ceph-mon[31181]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T22:45:43.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:45:43.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:43 smithi152 ceph-mon[31181]: from='osd.1 [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:45:43.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:43 smithi152 ceph-mon[31181]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:45:43.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:43 smithi152 ceph-mon[31181]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:43.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:43.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:43.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:45:43.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:43.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:43.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:45:43.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:45:43.771 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:45:43.770+0000 7f8691ed4640 1 -- 172.21.15.204:0/491038075 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f868c062630 con 0x7f86780a2b70 2024-09-11T22:45:44.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:44 smithi204 ceph-mon[35454]: Detected new or changed devices on smithi191 2024-09-11T22:45:44.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:44 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi191 to 17088M 2024-09-11T22:45:44.451 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:44 smithi204 ceph-mon[35454]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-11T22:45:44.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:44 smithi204 ceph-mon[35454]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T22:45:44.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:45:44.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:45:44.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:44 smithi204 ceph-mon[35454]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:45:44.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:45:44.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:45:44.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:44.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:44 smithi191 ceph-mon[35211]: Detected new or changed devices on smithi191 2024-09-11T22:45:44.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:44 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi191 to 17088M 2024-09-11T22:45:44.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:44 smithi191 ceph-mon[35211]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-11T22:45:44.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:44 smithi191 ceph-mon[35211]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T22:45:44.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:45:44.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:45:44.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:44 smithi191 ceph-mon[35211]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:45:44.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:45:44.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:45:44.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:44.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:44 smithi152 ceph-mon[31181]: Detected new or changed devices on smithi191 2024-09-11T22:45:44.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:44 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi191 to 17088M 2024-09-11T22:45:44.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:44 smithi152 ceph-mon[31181]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-11T22:45:44.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:44 smithi152 ceph-mon[31181]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T22:45:44.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:45:44.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:45:44.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:44 smithi152 ceph-mon[31181]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:45:44.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:45:44.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:45:44.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:45.345 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:45 smithi204 ceph-mon[35454]: purged_snaps scrub starts 2024-09-11T22:45:45.346 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:45 smithi204 ceph-mon[35454]: purged_snaps scrub ok 2024-09-11T22:45:45.346 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:45 smithi204 ceph-mon[35454]: osd.1 [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155] boot 2024-09-11T22:45:45.346 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:45 smithi204 ceph-mon[35454]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T22:45:45.346 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:45:45.346 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:45 smithi204 ceph-mon[35454]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:45.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:45 smithi191 ceph-mon[35211]: purged_snaps scrub starts 2024-09-11T22:45:45.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:45 smithi191 ceph-mon[35211]: purged_snaps scrub ok 2024-09-11T22:45:45.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:45 smithi191 ceph-mon[35211]: osd.1 [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155] boot 2024-09-11T22:45:45.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:45 smithi191 ceph-mon[35211]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T22:45:45.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:45:45.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:45 smithi191 ceph-mon[35211]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:45.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:45 smithi152 ceph-mon[31181]: purged_snaps scrub starts 2024-09-11T22:45:45.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:45 smithi152 ceph-mon[31181]: purged_snaps scrub ok 2024-09-11T22:45:45.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:45 smithi152 ceph-mon[31181]: osd.1 [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155] boot 2024-09-11T22:45:45.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:45 smithi152 ceph-mon[31181]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T22:45:45.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:45:45.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:45 smithi152 ceph-mon[31181]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:45:46.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:46 smithi191 ceph-mon[35211]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T22:45:46.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:46 smithi191 ceph-mon[35211]: from='client.? 172.21.15.204:0/441608946' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b35ffd0c-d084-43dd-a947-15132d78d59c"}]: dispatch 2024-09-11T22:45:46.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:46 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b35ffd0c-d084-43dd-a947-15132d78d59c"}]: dispatch 2024-09-11T22:45:46.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:46 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b35ffd0c-d084-43dd-a947-15132d78d59c"}]': finished 2024-09-11T22:45:46.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:46 smithi191 ceph-mon[35211]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T22:45:46.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:46 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:45:46.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:46 smithi204 ceph-mon[35454]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T22:45:46.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:46 smithi204 ceph-mon[35454]: from='client.? 172.21.15.204:0/441608946' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b35ffd0c-d084-43dd-a947-15132d78d59c"}]: dispatch 2024-09-11T22:45:46.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:46 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b35ffd0c-d084-43dd-a947-15132d78d59c"}]: dispatch 2024-09-11T22:45:46.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:46 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b35ffd0c-d084-43dd-a947-15132d78d59c"}]': finished 2024-09-11T22:45:46.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:46 smithi204 ceph-mon[35454]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T22:45:46.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:46 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:45:46.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:46 smithi152 ceph-mon[31181]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T22:45:46.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:46 smithi152 ceph-mon[31181]: from='client.? 172.21.15.204:0/441608946' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b35ffd0c-d084-43dd-a947-15132d78d59c"}]: dispatch 2024-09-11T22:45:46.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:46 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b35ffd0c-d084-43dd-a947-15132d78d59c"}]: dispatch 2024-09-11T22:45:46.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:46 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b35ffd0c-d084-43dd-a947-15132d78d59c"}]': finished 2024-09-11T22:45:46.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:46 smithi152 ceph-mon[31181]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T22:45:46.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:46 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:45:47.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:47 smithi191 ceph-mon[35211]: from='client.? 172.21.15.204:0/2765862064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:45:47.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:47 smithi191 ceph-mon[35211]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:47.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:47 smithi204 ceph-mon[35454]: from='client.? 172.21.15.204:0/2765862064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:45:47.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:47 smithi204 ceph-mon[35454]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:47.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:47 smithi152 ceph-mon[31181]: from='client.? 172.21.15.204:0/2765862064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:45:47.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:47 smithi152 ceph-mon[31181]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:50.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:49 smithi191 ceph-mon[35211]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:50.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:49 smithi204 ceph-mon[35454]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:50.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:49 smithi152 ceph-mon[31181]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:51.994 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:51 smithi204 ceph-mon[35454]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:52.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:51 smithi191 ceph-mon[35211]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:52.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:51 smithi152 ceph-mon[31181]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:53.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:53 smithi204 ceph-mon[35454]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:54.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:53 smithi191 ceph-mon[35211]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:54.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:53 smithi152 ceph-mon[31181]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:55.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:54 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T22:45:55.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:54 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:55.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:54 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T22:45:55.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:54 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:55.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T22:45:55.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:45:55.978 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:55 smithi204 ceph-mon[35454]: Deploying daemon osd.2 on smithi204 2024-09-11T22:45:55.978 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:55 smithi204 ceph-mon[35454]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:56.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:55 smithi191 ceph-mon[35211]: Deploying daemon osd.2 on smithi204 2024-09-11T22:45:56.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:55 smithi191 ceph-mon[35211]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:56.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:55 smithi152 ceph-mon[31181]: Deploying daemon osd.2 on smithi204 2024-09-11T22:45:56.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:55 smithi152 ceph-mon[31181]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:58.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:57 smithi191 ceph-mon[35211]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:58.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:57 smithi204 ceph-mon[35454]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:58.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:57 smithi152 ceph-mon[31181]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:45:59.814 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:45:59 smithi204 ceph-mon[35454]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:00.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:45:59 smithi191 ceph-mon[35211]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:00.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:45:59 smithi152 ceph-mon[31181]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:01.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:01.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:01.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:01.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:01.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:01.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:01.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:01.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:01.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:02.008 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:01 smithi204 ceph-mon[35454]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:02.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:01 smithi191 ceph-mon[35211]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:02.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:01 smithi152 ceph-mon[31181]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:03.839 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:03 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:03.840 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:03 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:03.840 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:03 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:03.840 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:03 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:46:03.840 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:03 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:03.840 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:03 smithi191 ceph-mon[35211]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:03.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:03 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:03.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:03 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:03.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:03 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:03.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:03 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:46:03.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:03 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:03.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:03 smithi152 ceph-mon[31181]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:04.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:03 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:04.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:03 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:04.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:03 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:04.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:03 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:46:04.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:03 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:04.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:03 smithi204 ceph-mon[35454]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:04.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:46:04.339+0000 7f86897fa640 1 -- 172.21.15.204:0/491038075 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f868c062630 con 0x7f86780a2b70 2024-09-11T22:46:04.340 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 2 on host 'smithi204' 2024-09-11T22:46:04.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:46:04.341+0000 7f8691ed4640 1 -- 172.21.15.204:0/491038075 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f86780a2b70 msgr2=0x7f86780a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:46:04.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:46:04.341+0000 7f8691ed4640 1 --2- 172.21.15.204:0/491038075 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f86780a2b70 0x7f86780a5030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f868c06aa50 tx=0x7f867400a2d0 comp rx=0 tx=0).stop 2024-09-11T22:46:04.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:46:04.341+0000 7f8691ed4640 1 -- 172.21.15.204:0/491038075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c1026e0 msgr2=0x7f868c197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:46:04.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:46:04.341+0000 7f8691ed4640 1 --2- 172.21.15.204:0/491038075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c1026e0 0x7f868c197db0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f867c00c920 tx=0x7f867c00cd20 comp rx=0 tx=0).stop 2024-09-11T22:46:04.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:46:04.341+0000 7f8691ed4640 1 -- 172.21.15.204:0/491038075 shutdown_connections 2024-09-11T22:46:04.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:46:04.341+0000 7f8691ed4640 1 --2- 172.21.15.204:0/491038075 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f86780a2b70 0x7f86780a5030 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:04.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:46:04.341+0000 7f8691ed4640 1 --2- 172.21.15.204:0/491038075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f868c107940 0x7f868c19f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:04.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:46:04.341+0000 7f8691ed4640 1 --2- 172.21.15.204:0/491038075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f868c105010 0x7f868c1982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:04.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:46:04.341+0000 7f8691ed4640 1 --2- 172.21.15.204:0/491038075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f868c1026e0 0x7f868c197db0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:04.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:46:04.341+0000 7f8691ed4640 1 -- 172.21.15.204:0/491038075 >> 172.21.15.204:0/491038075 conn(0x7f868c0fc280 msgr2=0x7f868c102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:46:04.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:46:04.341+0000 7f8691ed4640 1 -- 172.21.15.204:0/491038075 shutdown_connections 2024-09-11T22:46:04.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:46:04.341+0000 7f8691ed4640 1 -- 172.21.15.204:0/491038075 wait complete. 2024-09-11T22:46:04.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:04 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:04.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:04 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:04.644 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:04 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:04.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:04 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:04.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:04 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:04.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:04 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:04.997 DEBUG:teuthology.orchestra.run.smithi204:osd.2> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T22:46:04.999 INFO:tasks.cephadm:Deploying osd.3 on smithi152 with /dev/vg_nvme/lv_3... 2024-09-11T22:46:04.999 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T22:46:05.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:04 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:05.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:04 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:05.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:04 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:05.242 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:46:05.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:05 smithi152 ceph-mon[31181]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:06.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:05 smithi191 ceph-mon[35211]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:06.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:05 smithi204 ceph-mon[35454]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:06.398 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 22:46:06 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-2[40411]: 2024-09-11T22:46:06.181+0000 7f60018e7640 -1 osd.2 0 log_to_monitors true 2024-09-11T22:46:06.740 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:06 smithi152 ceph-mon[31181]: from='osd.2 [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:46:06.741 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:06 smithi152 ceph-mon[31181]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:46:06.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:06 smithi204 ceph-mon[35454]: from='osd.2 [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:46:06.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:06 smithi204 ceph-mon[35454]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:46:07.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:06 smithi191 ceph-mon[35211]: from='osd.2 [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:46:07.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:06 smithi191 ceph-mon[35211]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:46:07.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:07 smithi204 ceph-mon[35454]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T22:46:07.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:07 smithi204 ceph-mon[35454]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T22:46:07.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:07 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:46:07.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:07 smithi204 ceph-mon[35454]: from='osd.2 [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:46:07.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:07 smithi204 ceph-mon[35454]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:46:07.751 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:07 smithi204 ceph-mon[35454]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:07.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:07 smithi152 ceph-mon[31181]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T22:46:07.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:07 smithi152 ceph-mon[31181]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T22:46:07.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:07 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:46:07.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:07 smithi152 ceph-mon[31181]: from='osd.2 [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:46:07.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:07 smithi152 ceph-mon[31181]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:46:07.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:07 smithi152 ceph-mon[31181]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:07.951 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:46:07.985 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi152:vg_nvme/lv_3 2024-09-11T22:46:08.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:07 smithi191 ceph-mon[35211]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T22:46:08.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:07 smithi191 ceph-mon[35211]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T22:46:08.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:07 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:46:08.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:07 smithi191 ceph-mon[35211]: from='osd.2 [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:46:08.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:07 smithi191 ceph-mon[35211]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:46:08.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:07 smithi191 ceph-mon[35211]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:08.233 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:46:08.296 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 22:46:08 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-2[40411]: 2024-09-11T22:46:08.016+0000 7f5ffd86a640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T22:46:08.296 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 22:46:08 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-2[40411]: 2024-09-11T22:46:08.022+0000 7f5ff568a640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:46:08.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.724+0000 7f2c733e5640 1 -- 172.21.15.152:0/3181773664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c6c102f70 msgr2=0x7f2c6c1033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:46:08.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.724+0000 7f2c733e5640 1 --2- 172.21.15.152:0/3181773664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c6c102f70 0x7f2c6c1033f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c009930 tx=0x7f2c5c02a420 comp rx=0 tx=0).stop 2024-09-11T22:46:08.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.724+0000 7f2c733e5640 1 -- 172.21.15.152:0/3181773664 shutdown_connections 2024-09-11T22:46:08.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.724+0000 7f2c733e5640 1 --2- 172.21.15.152:0/3181773664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c6c103930 0x7f2c6c10a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:08.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.724+0000 7f2c733e5640 1 --2- 172.21.15.152:0/3181773664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c6c102f70 0x7f2c6c1033f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:08.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.724+0000 7f2c733e5640 1 --2- 172.21.15.152:0/3181773664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c6c0ffef0 0x7f2c6c1002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:08.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.724+0000 7f2c733e5640 1 -- 172.21.15.152:0/3181773664 >> 172.21.15.152:0/3181773664 conn(0x7f2c6c0fba40 msgr2=0x7f2c6c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:46:08.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.724+0000 7f2c733e5640 1 -- 172.21.15.152:0/3181773664 shutdown_connections 2024-09-11T22:46:08.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.725+0000 7f2c733e5640 1 -- 172.21.15.152:0/3181773664 wait complete. 2024-09-11T22:46:08.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.725+0000 7f2c733e5640 1 Processor -- start 2024-09-11T22:46:08.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.725+0000 7f2c733e5640 1 -- start start 2024-09-11T22:46:08.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.725+0000 7f2c733e5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c6c0ffef0 0x7f2c6c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:46:08.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.725+0000 7f2c733e5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c6c102f70 0x7f2c6c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:46:08.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.725+0000 7f2c733e5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c6c103930 0x7f2c6c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:46:08.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.725+0000 7f2c733e5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6c10d9a0 con 0x7f2c6c0ffef0 2024-09-11T22:46:08.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.725+0000 7f2c733e5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6c10d820 con 0x7f2c6c103930 2024-09-11T22:46:08.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.725+0000 7f2c733e5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6c10db20 con 0x7f2c6c102f70 2024-09-11T22:46:08.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.726+0000 7f2c71181640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c6c0ffef0 0x7f2c6c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:46:08.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.726+0000 7f2c71181640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c6c0ffef0 0x7f2c6c198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46068/0 (socket says 172.21.15.152:46068) 2024-09-11T22:46:08.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.726+0000 7f2c71181640 1 -- 172.21.15.152:0/486905097 learned_addr learned my addr 172.21.15.152:0/486905097 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:46:08.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.726+0000 7f2c70980640 1 --2- 172.21.15.152:0/486905097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c6c102f70 0x7f2c6c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:46:08.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.726+0000 7f2c71982640 1 --2- 172.21.15.152:0/486905097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c6c103930 0x7f2c6c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:46:08.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.726+0000 7f2c71181640 1 -- 172.21.15.152:0/486905097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c6c102f70 msgr2=0x7f2c6c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:46:08.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.726+0000 7f2c71181640 1 --2- 172.21.15.152:0/486905097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c6c102f70 0x7f2c6c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:08.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.726+0000 7f2c71181640 1 -- 172.21.15.152:0/486905097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c6c103930 msgr2=0x7f2c6c19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:46:08.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.726+0000 7f2c71181640 1 --2- 172.21.15.152:0/486905097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c6c103930 0x7f2c6c19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:08.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.726+0000 7f2c71181640 1 -- 172.21.15.152:0/486905097 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c6c19fd50 con 0x7f2c6c0ffef0 2024-09-11T22:46:08.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.726+0000 7f2c71181640 1 --2- 172.21.15.152:0/486905097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c6c0ffef0 0x7f2c6c198120 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800b450 tx=0x7f2c6800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:46:08.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.726+0000 7f2c627fc640 1 -- 172.21.15.152:0/486905097 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68015020 con 0x7f2c6c0ffef0 2024-09-11T22:46:08.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.726+0000 7f2c627fc640 1 -- 172.21.15.152:0/486905097 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c68011d70 con 0x7f2c6c0ffef0 2024-09-11T22:46:08.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.726+0000 7f2c733e5640 1 -- 172.21.15.152:0/486905097 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c6c19ffe0 con 0x7f2c6c0ffef0 2024-09-11T22:46:08.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.726+0000 7f2c627fc640 1 -- 172.21.15.152:0/486905097 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6800f340 con 0x7f2c6c0ffef0 2024-09-11T22:46:08.730 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.726+0000 7f2c733e5640 1 -- 172.21.15.152:0/486905097 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c6c1a0560 con 0x7f2c6c0ffef0 2024-09-11T22:46:08.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.727+0000 7f2c733e5640 1 -- 172.21.15.152:0/486905097 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c6c1092b0 con 0x7f2c6c0ffef0 2024-09-11T22:46:08.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.728+0000 7f2c627fc640 1 -- 172.21.15.152:0/486905097 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f2c68018020 con 0x7f2c6c0ffef0 2024-09-11T22:46:08.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.729+0000 7f2c627fc640 1 --2- 172.21.15.152:0/486905097 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2c580a2ba0 0x7f2c580a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:46:08.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.729+0000 7f2c70980640 1 --2- 172.21.15.152:0/486905097 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2c580a2ba0 0x7f2c580a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:46:08.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.729+0000 7f2c627fc640 1 -- 172.21.15.152:0/486905097 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f2c680cd1d0 con 0x7f2c6c0ffef0 2024-09-11T22:46:08.731 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.730+0000 7f2c70980640 1 --2- 172.21.15.152:0/486905097 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2c580a2ba0 0x7f2c580a5060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c006010 tx=0x7f2c5c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:46:08.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.734+0000 7f2c627fc640 1 -- 172.21.15.152:0/486905097 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c6801f020 con 0x7f2c6c0ffef0 2024-09-11T22:46:08.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:08 smithi152 ceph-mon[31181]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-11T22:46:08.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:08 smithi152 ceph-mon[31181]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T22:46:08.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:08 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:46:08.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:08 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:46:08.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:08 smithi152 ceph-mon[31181]: Detected new or changed devices on smithi204 2024-09-11T22:46:08.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:08 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:08.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:08 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:08.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:08 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:46:08.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:08 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi204 to 17088M 2024-09-11T22:46:08.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:08 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:08.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:08 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:08.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:08 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:46:08.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:08 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:08.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:08 smithi152 ceph-mon[31181]: from='osd.2 ' entity='osd.2' 2024-09-11T22:46:08.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:08.909+0000 7f2c733e5640 1 -- 172.21.15.152:0/486905097 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2c6c062630 con 0x7f2c580a2ba0 2024-09-11T22:46:09.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:08 smithi191 ceph-mon[35211]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-11T22:46:09.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:08 smithi191 ceph-mon[35211]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T22:46:09.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:08 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:46:09.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:08 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:46:09.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:08 smithi191 ceph-mon[35211]: Detected new or changed devices on smithi204 2024-09-11T22:46:09.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:08 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:09.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:08 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:09.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:08 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:46:09.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:08 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi204 to 17088M 2024-09-11T22:46:09.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:08 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:09.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:08 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:09.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:08 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:46:09.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:08 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:09.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:08 smithi191 ceph-mon[35211]: from='osd.2 ' entity='osd.2' 2024-09-11T22:46:09.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:08 smithi204 ceph-mon[35454]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-11T22:46:09.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:08 smithi204 ceph-mon[35454]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T22:46:09.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:08 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:46:09.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:08 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:46:09.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:08 smithi204 ceph-mon[35454]: Detected new or changed devices on smithi204 2024-09-11T22:46:09.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:08 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:09.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:08 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:09.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:08 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:46:09.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:08 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi204 to 17088M 2024-09-11T22:46:09.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:08 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:09.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:08 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:09.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:08 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:46:09.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:08 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:09.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:08 smithi204 ceph-mon[35454]: from='osd.2 ' entity='osd.2' 2024-09-11T22:46:09.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:09 smithi152 ceph-mon[31181]: purged_snaps scrub starts 2024-09-11T22:46:09.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:09 smithi152 ceph-mon[31181]: purged_snaps scrub ok 2024-09-11T22:46:09.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:46:09.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:09 smithi152 ceph-mon[31181]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:09.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:46:09.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:46:09.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:09.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:09 smithi152 ceph-mon[31181]: osd.2 [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211] boot 2024-09-11T22:46:09.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:09 smithi152 ceph-mon[31181]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T22:46:09.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:46:10.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:09 smithi191 ceph-mon[35211]: purged_snaps scrub starts 2024-09-11T22:46:10.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:09 smithi191 ceph-mon[35211]: purged_snaps scrub ok 2024-09-11T22:46:10.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:46:10.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:09 smithi191 ceph-mon[35211]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:10.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:46:10.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:46:10.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:10.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:09 smithi191 ceph-mon[35211]: osd.2 [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211] boot 2024-09-11T22:46:10.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:09 smithi191 ceph-mon[35211]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T22:46:10.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:46:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:09 smithi204 ceph-mon[35454]: purged_snaps scrub starts 2024-09-11T22:46:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:09 smithi204 ceph-mon[35454]: purged_snaps scrub ok 2024-09-11T22:46:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:46:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:09 smithi204 ceph-mon[35454]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:46:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:46:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:46:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:09 smithi204 ceph-mon[35454]: osd.2 [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211] boot 2024-09-11T22:46:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:09 smithi204 ceph-mon[35454]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T22:46:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:46:10.728 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:10 smithi152 ceph-mon[31181]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:46:10.729 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:10 smithi152 ceph-mon[31181]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T22:46:11.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:10 smithi191 ceph-mon[35211]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:46:11.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:10 smithi191 ceph-mon[35211]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T22:46:11.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:10 smithi204 ceph-mon[35454]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:46:11.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:10 smithi204 ceph-mon[35454]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T22:46:11.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:11 smithi152 ceph-mon[31181]: pgmap v77: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:11.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:11 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:46:11.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:11 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4161104425' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d66ba7e8-4ab7-449e-9460-2f44ffdf987f"}]: dispatch 2024-09-11T22:46:11.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:11 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T22:46:11.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:11 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4161104425' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d66ba7e8-4ab7-449e-9460-2f44ffdf987f"}]': finished 2024-09-11T22:46:11.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:11 smithi152 ceph-mon[31181]: osdmap e20: 4 total, 3 up, 4 in 2024-09-11T22:46:11.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:11 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:11.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:11 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:46:12.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:11 smithi191 ceph-mon[35211]: pgmap v77: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:12.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:11 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:46:12.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:11 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/4161104425' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d66ba7e8-4ab7-449e-9460-2f44ffdf987f"}]: dispatch 2024-09-11T22:46:12.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:11 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T22:46:12.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:11 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/4161104425' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d66ba7e8-4ab7-449e-9460-2f44ffdf987f"}]': finished 2024-09-11T22:46:12.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:11 smithi191 ceph-mon[35211]: osdmap e20: 4 total, 3 up, 4 in 2024-09-11T22:46:12.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:11 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:12.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:11 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:46:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:11 smithi204 ceph-mon[35454]: pgmap v77: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:11 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:46:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:11 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/4161104425' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d66ba7e8-4ab7-449e-9460-2f44ffdf987f"}]: dispatch 2024-09-11T22:46:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:11 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T22:46:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:11 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/4161104425' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d66ba7e8-4ab7-449e-9460-2f44ffdf987f"}]': finished 2024-09-11T22:46:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:11 smithi204 ceph-mon[35454]: osdmap e20: 4 total, 3 up, 4 in 2024-09-11T22:46:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:11 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:12.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:11 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:46:12.872 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 22:46:12 smithi152 sudo[47616]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:46:12.873 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 22:46:12 smithi152 sudo[47616]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:46:12.873 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 22:46:12 smithi152 sudo[47616]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:46:12.873 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 22:46:12 smithi152 sudo[47616]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:46:12.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:12 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3899228396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:46:12.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:12 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:46:12.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:12 smithi152 ceph-mon[31181]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T22:46:12.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:12 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:13.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:12 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3899228396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:46:13.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:12 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:46:13.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:12 smithi191 ceph-mon[35211]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T22:46:13.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:12 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:13.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:12 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3899228396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:46:13.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:12 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:46:13.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:12 smithi204 ceph-mon[35454]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T22:46:13.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:12 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:13.372 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 22:46:13 smithi152 sudo[47621]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:46:13.372 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 22:46:13 smithi152 sudo[47621]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:46:13.373 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 22:46:13 smithi152 sudo[47621]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:46:13.373 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 22:46:13 smithi152 sudo[47621]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:46:13.548 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 22:46:13 smithi191 sudo[43818]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:46:13.548 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 22:46:13 smithi191 sudo[43818]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:46:13.548 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 22:46:13 smithi191 sudo[43818]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:46:13.548 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 22:46:13 smithi191 sudo[43818]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:46:13.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:13.630+0000 7f2c627fc640 1 -- 172.21.15.152:0/486905097 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2c6800f050 con 0x7f2c6c0ffef0 2024-09-11T22:46:14.037 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 22:46:13 smithi191 sudo[43821]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:46:14.037 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 22:46:13 smithi191 sudo[43821]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:46:14.037 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 22:46:13 smithi191 sudo[43821]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:46:14.037 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 22:46:13 smithi191 sudo[43821]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:46:14.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:13 smithi191 ceph-mon[35211]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:14.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:13 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:14.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:13 smithi191 ceph-mon[35211]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T22:46:14.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:13 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:14.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:13 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:14.046 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 22:46:13 smithi204 sudo[44207]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:46:14.046 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 22:46:13 smithi204 sudo[44207]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:46:14.046 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 22:46:13 smithi204 sudo[44207]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:46:14.046 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 22:46:13 smithi204 sudo[44207]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:46:14.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:13 smithi204 ceph-mon[35454]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:14.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:13 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:14.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:13 smithi204 ceph-mon[35454]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T22:46:14.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:13 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:14.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:13 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:14.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:13 smithi152 ceph-mon[31181]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:14.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:14.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:13 smithi152 ceph-mon[31181]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T22:46:14.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:14.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:14.548 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 22:46:14 smithi204 sudo[44210]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:46:14.549 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 22:46:14 smithi204 sudo[44210]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:46:14.549 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 22:46:14 smithi204 sudo[44210]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:46:14.549 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 22:46:14 smithi204 sudo[44210]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:46:14.600 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:14 smithi152 sudo[47997]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:46:14.600 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:14 smithi152 sudo[47997]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:46:14.600 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:14 smithi152 sudo[47997]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:46:14.600 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:14 smithi152 sudo[47997]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:46:14.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:14 smithi152 sudo[48183]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:46:14.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:14 smithi152 sudo[48183]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:46:14.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:14 smithi152 sudo[48183]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:46:14.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:14 smithi152 sudo[48183]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:46:15.101 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:14 smithi204 sudo[44213]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:46:15.101 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:14 smithi204 sudo[44213]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:46:15.101 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:14 smithi204 sudo[44213]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:46:15.101 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:14 smithi204 sudo[44213]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:46:15.366 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 ceph-mon[35211]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T22:46:15.366 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:15.366 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 ceph-mon[35211]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:46:15.366 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 ceph-mon[35211]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:46:15.366 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:46:15.366 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:46:15.366 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:46:15.366 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:46:15.366 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:46:15.367 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:46:15.367 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 sudo[43824]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:46:15.367 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 sudo[43824]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:46:15.367 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 sudo[43824]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:46:15.367 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 sudo[43824]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:46:15.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:15 smithi152 ceph-mon[31181]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T22:46:15.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:15 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:15.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:15 smithi152 ceph-mon[31181]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:46:15.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:15 smithi152 ceph-mon[31181]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:46:15.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:15 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:46:15.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:15 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:46:15.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:15 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:46:15.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:15 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:46:15.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:15 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:46:15.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:15 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:46:15.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:15 smithi204 sudo[44216]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:46:15.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:15 smithi204 sudo[44216]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:46:15.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:15 smithi204 sudo[44216]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:46:15.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:15 smithi204 sudo[44216]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:46:15.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:15 smithi204 ceph-mon[35454]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T22:46:15.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:15 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:15.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:15 smithi204 ceph-mon[35454]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:46:15.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:15 smithi204 ceph-mon[35454]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:46:15.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:15 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:46:15.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:15 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:46:15.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:15 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:46:15.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:15 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:46:15.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:15 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:46:15.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:15 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:46:15.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 sudo[43827]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:46:15.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 sudo[43827]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:46:15.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 sudo[43827]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:46:15.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:15 smithi191 sudo[43827]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:46:16.367 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:16 smithi204 ceph-mon[35454]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:46:16.368 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:16 smithi204 ceph-mon[35454]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:16.368 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:16 smithi204 ceph-mon[35454]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:46:16.368 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:16 smithi204 ceph-mon[35454]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:46:16.368 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:16 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:46:16.368 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:16 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:46:16.368 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:16 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:46:16.368 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:16 smithi204 ceph-mon[35454]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:46:16.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:16 smithi152 ceph-mon[31181]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:46:16.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:16 smithi152 ceph-mon[31181]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:16.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:16 smithi152 ceph-mon[31181]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:46:16.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:16 smithi152 ceph-mon[31181]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:46:16.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:16 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:46:16.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:16 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:46:16.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:16 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:46:16.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:16 smithi152 ceph-mon[31181]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:46:16.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:16 smithi191 ceph-mon[35211]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:46:16.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:16 smithi191 ceph-mon[35211]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:16.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:16 smithi191 ceph-mon[35211]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:46:16.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:16 smithi191 ceph-mon[35211]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:46:16.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:16 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:46:16.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:16 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:46:16.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:16 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:46:16.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:16 smithi191 ceph-mon[35211]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:46:17.236 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:17 smithi152 ceph-mon[31181]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:17.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:17 smithi191 ceph-mon[35211]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:17.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:17 smithi204 ceph-mon[35454]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:20.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:19 smithi191 ceph-mon[35211]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:20.043 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:19 smithi152 ceph-mon[31181]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:20.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:19 smithi204 ceph-mon[35454]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:21.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:20 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T22:46:21.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:20 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:21.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:20 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T22:46:21.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:20 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:21.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:20 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T22:46:21.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:20 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:22.032 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:21 smithi152 ceph-mon[31181]: Deploying daemon osd.3 on smithi152 2024-09-11T22:46:22.032 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:21 smithi152 ceph-mon[31181]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-11T22:46:22.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:21 smithi191 ceph-mon[35211]: Deploying daemon osd.3 on smithi152 2024-09-11T22:46:22.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:21 smithi191 ceph-mon[35211]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-11T22:46:22.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:21 smithi204 ceph-mon[35454]: Deploying daemon osd.3 on smithi152 2024-09-11T22:46:22.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:21 smithi204 ceph-mon[35454]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-11T22:46:23.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:23 smithi152 ceph-mon[31181]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T22:46:24.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:23 smithi191 ceph-mon[35211]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T22:46:24.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:23 smithi204 ceph-mon[35454]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T22:46:26.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:25 smithi191 ceph-mon[35211]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T22:46:26.044 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:25 smithi152 ceph-mon[31181]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T22:46:26.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:25 smithi204 ceph-mon[35454]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T22:46:27.026 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:26 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:27.027 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:26 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:27.027 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:26 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:27.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:26 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:27.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:26 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:27.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:26 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:27.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:26 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:27.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:26 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:27.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:26 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:27.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:27 smithi152 ceph-mon[31181]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T22:46:28.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:27 smithi191 ceph-mon[35211]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T22:46:28.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:27 smithi204 ceph-mon[35454]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T22:46:29.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:29 smithi152 ceph-mon[31181]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T22:46:29.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:29.929+0000 7f2c627fc640 1 -- 172.21.15.152:0/486905097 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2c6c062630 con 0x7f2c580a2ba0 2024-09-11T22:46:29.929 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 3 on host 'smithi152' 2024-09-11T22:46:29.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:29.931+0000 7f2c733e5640 1 -- 172.21.15.152:0/486905097 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2c580a2ba0 msgr2=0x7f2c580a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:46:29.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:29.931+0000 7f2c733e5640 1 --2- 172.21.15.152:0/486905097 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2c580a2ba0 0x7f2c580a5060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c006010 tx=0x7f2c5c0315b0 comp rx=0 tx=0).stop 2024-09-11T22:46:29.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:29.931+0000 7f2c733e5640 1 -- 172.21.15.152:0/486905097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c6c0ffef0 msgr2=0x7f2c6c198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:46:29.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:29.931+0000 7f2c733e5640 1 --2- 172.21.15.152:0/486905097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c6c0ffef0 0x7f2c6c198120 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800b450 tx=0x7f2c6800b910 comp rx=0 tx=0).stop 2024-09-11T22:46:29.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:29.931+0000 7f2c733e5640 1 -- 172.21.15.152:0/486905097 shutdown_connections 2024-09-11T22:46:29.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:29.931+0000 7f2c733e5640 1 --2- 172.21.15.152:0/486905097 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2c580a2ba0 0x7f2c580a5060 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:29.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:29.931+0000 7f2c733e5640 1 --2- 172.21.15.152:0/486905097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c6c103930 0x7f2c6c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:29.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:29.931+0000 7f2c733e5640 1 --2- 172.21.15.152:0/486905097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c6c102f70 0x7f2c6c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:29.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:29.931+0000 7f2c733e5640 1 --2- 172.21.15.152:0/486905097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c6c0ffef0 0x7f2c6c198120 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:29.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:29.931+0000 7f2c733e5640 1 -- 172.21.15.152:0/486905097 >> 172.21.15.152:0/486905097 conn(0x7f2c6c0fba40 msgr2=0x7f2c6c106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:46:29.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:29.931+0000 7f2c733e5640 1 -- 172.21.15.152:0/486905097 shutdown_connections 2024-09-11T22:46:29.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:46:29.931+0000 7f2c733e5640 1 -- 172.21.15.152:0/486905097 wait complete. 2024-09-11T22:46:30.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:29 smithi191 ceph-mon[35211]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T22:46:30.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:29 smithi204 ceph-mon[35454]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T22:46:30.632 DEBUG:teuthology.orchestra.run.smithi152:osd.3> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T22:46:30.635 INFO:tasks.cephadm:Deploying osd.4 on smithi191 with /dev/vg_nvme/lv_3... 2024-09-11T22:46:30.635 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T22:46:30.890 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:46:31.176 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:30 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:31.176 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:30 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:31.176 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:30 smithi191 ceph-mon[35211]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T22:46:31.176 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:30 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:31.176 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:30 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:31.259 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:30 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:31.260 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:30 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:31.260 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:30 smithi152 ceph-mon[31181]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T22:46:31.260 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:30 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:31.260 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:30 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:31.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:30 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:31.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:30 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:31.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:30 smithi204 ceph-mon[35454]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T22:46:31.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:30 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:31.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:30 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:31.623 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 22:46:31 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3[49580]: 2024-09-11T22:46:31.495+0000 7ff0eb16a640 -1 osd.3 0 log_to_monitors true 2024-09-11T22:46:32.181 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:31 smithi152 ceph-mon[31181]: from='osd.3 [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T22:46:32.216 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:31 smithi191 ceph-mon[35211]: from='osd.3 [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T22:46:32.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:31 smithi204 ceph-mon[35454]: from='osd.3 [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T22:46:33.155 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:32 smithi191 ceph-mon[35211]: from='osd.3 [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T22:46:33.155 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:32 smithi191 ceph-mon[35211]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T22:46:33.155 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:32 smithi191 ceph-mon[35211]: from='osd.3 [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-09-11T22:46:33.155 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:32 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:33.155 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:32 smithi191 ceph-mon[35211]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:33.289 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 22:46:32 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3[49580]: 2024-09-11T22:46:32.941+0000 7ff0e70ed640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T22:46:33.289 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 22:46:32 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3[49580]: 2024-09-11T22:46:32.946+0000 7ff0e2f15640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:46:33.290 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:32 smithi152 ceph-mon[31181]: from='osd.3 [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T22:46:33.290 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:32 smithi152 ceph-mon[31181]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T22:46:33.290 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:32 smithi152 ceph-mon[31181]: from='osd.3 [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-09-11T22:46:33.290 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:32 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:33.290 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:32 smithi152 ceph-mon[31181]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:33.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:32 smithi204 ceph-mon[35454]: from='osd.3 [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T22:46:33.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:32 smithi204 ceph-mon[35454]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T22:46:33.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:32 smithi204 ceph-mon[35454]: from='osd.3 [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-09-11T22:46:33.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:32 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:33.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:32 smithi204 ceph-mon[35454]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:33.458 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:46:33.484 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi191:vg_nvme/lv_3 2024-09-11T22:46:33.737 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:46:34.150 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:33 smithi191 ceph-mon[35211]: from='osd.3 [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-09-11T22:46:34.150 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:33 smithi191 ceph-mon[35211]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T22:46:34.150 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:33 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:34.151 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:33 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:34.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:33 smithi204 ceph-mon[35454]: from='osd.3 [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-09-11T22:46:34.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:33 smithi204 ceph-mon[35454]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T22:46:34.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:33 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:34.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:33 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:34.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.235+0000 7f0b9101a640 1 -- 172.21.15.191:0/2798977739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b8c107960 msgr2=0x7f0b8c109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:46:34.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.235+0000 7f0b9101a640 1 --2- 172.21.15.191:0/2798977739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b8c107960 0x7f0b8c109d50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009ca0 tx=0x7f0b8402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:46:34.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.236+0000 7f0b9101a640 1 -- 172.21.15.191:0/2798977739 shutdown_connections 2024-09-11T22:46:34.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.236+0000 7f0b9101a640 1 --2- 172.21.15.191:0/2798977739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b8c107960 0x7f0b8c109d50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:34.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.236+0000 7f0b9101a640 1 --2- 172.21.15.191:0/2798977739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b8c105030 0x7f0b8c107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:34.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.236+0000 7f0b9101a640 1 --2- 172.21.15.191:0/2798977739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b8c102700 0x7f0b8c104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:34.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.236+0000 7f0b9101a640 1 -- 172.21.15.191:0/2798977739 >> 172.21.15.191:0/2798977739 conn(0x7f0b8c0fc280 msgr2=0x7f0b8c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:46:34.259 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.236+0000 7f0b9101a640 1 -- 172.21.15.191:0/2798977739 shutdown_connections 2024-09-11T22:46:34.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.236+0000 7f0b9101a640 1 -- 172.21.15.191:0/2798977739 wait complete. 2024-09-11T22:46:34.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.236+0000 7f0b9101a640 1 Processor -- start 2024-09-11T22:46:34.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.236+0000 7f0b9101a640 1 -- start start 2024-09-11T22:46:34.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b9101a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b8c102700 0x7f0b8c102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:46:34.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b9101a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b8c105030 0x7f0b8c1006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:46:34.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b9101a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b8c107960 0x7f0b8c100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:46:34.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b9101a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c10d860 con 0x7f0b8c105030 2024-09-11T22:46:34.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b9101a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c10d6e0 con 0x7f0b8c102700 2024-09-11T22:46:34.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b9101a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c10d9e0 con 0x7f0b8c107960 2024-09-11T22:46:34.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b8ad9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b8c102700 0x7f0b8c102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:46:34.260 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b8ad9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b8c102700 0x7f0b8c102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:50898/0 (socket says 172.21.15.191:50898) 2024-09-11T22:46:34.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b8ad9d640 1 -- 172.21.15.191:0/4266809294 learned_addr learned my addr 172.21.15.191:0/4266809294 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-11T22:46:34.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b8b59e640 1 --2- 172.21.15.191:0/4266809294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b8c107960 0x7f0b8c100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:46:34.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b8a59c640 1 --2- 172.21.15.191:0/4266809294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b8c105030 0x7f0b8c1006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:46:34.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b8ad9d640 1 -- 172.21.15.191:0/4266809294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b8c107960 msgr2=0x7f0b8c100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:46:34.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b8ad9d640 1 --2- 172.21.15.191:0/4266809294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b8c107960 0x7f0b8c100bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:34.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b8ad9d640 1 -- 172.21.15.191:0/4266809294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b8c105030 msgr2=0x7f0b8c1006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:46:34.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b8ad9d640 1 --2- 172.21.15.191:0/4266809294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b8c105030 0x7f0b8c1006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:34.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b8ad9d640 1 -- 172.21.15.191:0/4266809294 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c1014b0 con 0x7f0b8c102700 2024-09-11T22:46:34.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b8a59c640 1 --2- 172.21.15.191:0/4266809294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b8c105030 0x7f0b8c1006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:46:34.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.237+0000 7f0b8ad9d640 1 --2- 172.21.15.191:0/4266809294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b8c102700 0x7f0b8c102060 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000e9f0 tx=0x7f0b8000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:46:34.261 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.238+0000 7f0b73fff640 1 -- 172.21.15.191:0/4266809294 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8000cd80 con 0x7f0b8c102700 2024-09-11T22:46:34.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.238+0000 7f0b73fff640 1 -- 172.21.15.191:0/4266809294 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b80010910 con 0x7f0b8c102700 2024-09-11T22:46:34.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.238+0000 7f0b73fff640 1 -- 172.21.15.191:0/4266809294 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80018990 con 0x7f0b8c102700 2024-09-11T22:46:34.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.238+0000 7f0b9101a640 1 -- 172.21.15.191:0/4266809294 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8c1a1fe0 con 0x7f0b8c102700 2024-09-11T22:46:34.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.238+0000 7f0b9101a640 1 -- 172.21.15.191:0/4266809294 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b8c1a2440 con 0x7f0b8c102700 2024-09-11T22:46:34.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.239+0000 7f0b9101a640 1 -- 172.21.15.191:0/4266809294 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b8c0fdd70 con 0x7f0b8c102700 2024-09-11T22:46:34.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.240+0000 7f0b73fff640 1 -- 172.21.15.191:0/4266809294 <== mon.2 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0b80010ac0 con 0x7f0b8c102700 2024-09-11T22:46:34.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.241+0000 7f0b73fff640 1 --2- 172.21.15.191:0/4266809294 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0b740a2b80 0x7f0b740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:46:34.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.241+0000 7f0b73fff640 1 -- 172.21.15.191:0/4266809294 <== mon.2 v2:172.21.15.191:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7f0b800ce520 con 0x7f0b8c102700 2024-09-11T22:46:34.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.241+0000 7f0b8a59c640 1 --2- 172.21.15.191:0/4266809294 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0b740a2b80 0x7f0b740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:46:34.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.241+0000 7f0b8a59c640 1 --2- 172.21.15.191:0/4266809294 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0b740a2b80 0x7f0b740a5040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c101de0 tx=0x7f0b7c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:46:34.262 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.244+0000 7f0b73fff640 1 -- 172.21.15.191:0/4266809294 <== mon.2 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b8009c030 con 0x7f0b8c102700 2024-09-11T22:46:34.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:33 smithi152 ceph-mon[31181]: from='osd.3 [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-09-11T22:46:34.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:33 smithi152 ceph-mon[31181]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T22:46:34.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:33 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:34.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:33 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:34.446 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:34.445+0000 7f0b9101a640 1 -- 172.21.15.191:0/4266809294 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0b8c062630 con 0x7f0b740a2b80 2024-09-11T22:46:35.112 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: purged_snaps scrub starts 2024-09-11T22:46:35.112 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: purged_snaps scrub ok 2024-09-11T22:46:35.112 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:35.112 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: osd.3 [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181] boot 2024-09-11T22:46:35.112 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T22:46:35.112 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:35.112 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: Detected new or changed devices on smithi152 2024-09-11T22:46:35.113 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:35.113 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:35.113 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:46:35.113 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:46:35.113 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi152 to 8544M 2024-09-11T22:46:35.113 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:35.113 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:35.113 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:46:35.113 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:35.113 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:35.113 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:35.113 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:46:35.114 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:35.114 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:46:35.114 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:46:35.114 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:46:35.114 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:35.114 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:34 smithi191 ceph-mon[35211]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:35.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: purged_snaps scrub starts 2024-09-11T22:46:35.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: purged_snaps scrub ok 2024-09-11T22:46:35.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:35.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: osd.3 [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181] boot 2024-09-11T22:46:35.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T22:46:35.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:35.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: Detected new or changed devices on smithi152 2024-09-11T22:46:35.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:35.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:35.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:46:35.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:46:35.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi152 to 8544M 2024-09-11T22:46:35.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:35.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:35.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:46:35.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:35.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:35.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:35.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:46:35.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:35.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:46:35.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:46:35.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:46:35.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:35.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:34 smithi204 ceph-mon[35454]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: purged_snaps scrub starts 2024-09-11T22:46:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: purged_snaps scrub ok 2024-09-11T22:46:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: osd.3 [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181] boot 2024-09-11T22:46:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T22:46:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:46:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: Detected new or changed devices on smithi152 2024-09-11T22:46:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:46:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:46:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi152 to 8544M 2024-09-11T22:46:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:46:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:46:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:46:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:46:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:46:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:34 smithi152 ceph-mon[31181]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:46:36.164 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:35 smithi191 ceph-mon[35211]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T22:46:36.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:35 smithi204 ceph-mon[35454]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T22:46:36.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:35 smithi152 ceph-mon[31181]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T22:46:37.229 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:36 smithi191 ceph-mon[35211]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T22:46:37.229 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:36 smithi191 ceph-mon[35211]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:46:37.229 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:36 smithi191 ceph-mon[35211]: from='client.? 172.21.15.191:0/557348203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1b56468-6cc7-4d29-988e-9d281118ee36"}]: dispatch 2024-09-11T22:46:37.229 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:36 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1b56468-6cc7-4d29-988e-9d281118ee36"}]: dispatch 2024-09-11T22:46:37.229 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:36 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1b56468-6cc7-4d29-988e-9d281118ee36"}]': finished 2024-09-11T22:46:37.229 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:36 smithi191 ceph-mon[35211]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T22:46:37.229 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:36 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:46:37.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:36 smithi204 ceph-mon[35454]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T22:46:37.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:36 smithi204 ceph-mon[35454]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:46:37.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:36 smithi204 ceph-mon[35454]: from='client.? 172.21.15.191:0/557348203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1b56468-6cc7-4d29-988e-9d281118ee36"}]: dispatch 2024-09-11T22:46:37.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:36 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1b56468-6cc7-4d29-988e-9d281118ee36"}]: dispatch 2024-09-11T22:46:37.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:36 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1b56468-6cc7-4d29-988e-9d281118ee36"}]': finished 2024-09-11T22:46:37.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:36 smithi204 ceph-mon[35454]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T22:46:37.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:36 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:46:37.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:36 smithi152 ceph-mon[31181]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T22:46:37.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:36 smithi152 ceph-mon[31181]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:46:37.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:36 smithi152 ceph-mon[31181]: from='client.? 172.21.15.191:0/557348203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1b56468-6cc7-4d29-988e-9d281118ee36"}]: dispatch 2024-09-11T22:46:37.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:36 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1b56468-6cc7-4d29-988e-9d281118ee36"}]: dispatch 2024-09-11T22:46:37.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:36 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1b56468-6cc7-4d29-988e-9d281118ee36"}]': finished 2024-09-11T22:46:37.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:36 smithi152 ceph-mon[31181]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T22:46:37.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:36 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:46:38.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:37 smithi191 ceph-mon[35211]: from='client.? 172.21.15.191:0/2720788747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:46:38.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:37 smithi191 ceph-mon[35211]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T22:46:38.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:37 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:46:38.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:37 smithi204 ceph-mon[35454]: from='client.? 172.21.15.191:0/2720788747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:46:38.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:37 smithi204 ceph-mon[35454]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T22:46:38.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:37 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:46:38.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:37 smithi152 ceph-mon[31181]: from='client.? 172.21.15.191:0/2720788747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:46:38.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:37 smithi152 ceph-mon[31181]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T22:46:38.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:37 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:46:39.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:38 smithi191 ceph-mon[35211]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:46:39.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:38 smithi204 ceph-mon[35454]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:46:39.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:38 smithi152 ceph-mon[31181]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:46:42.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:41 smithi191 ceph-mon[35211]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-11T22:46:42.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:41 smithi204 ceph-mon[35454]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-11T22:46:42.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:41 smithi152 ceph-mon[31181]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-11T22:46:44.015 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:43 smithi191 ceph-mon[35211]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-11T22:46:44.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:43 smithi204 ceph-mon[35454]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-11T22:46:44.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:43 smithi152 ceph-mon[31181]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-11T22:46:45.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:45 smithi191 ceph-mon[35211]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T22:46:45.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T22:46:45.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:46.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:45 smithi204 ceph-mon[35454]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T22:46:46.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T22:46:46.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:46.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:45 smithi152 ceph-mon[31181]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T22:46:46.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T22:46:46.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:47.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:46 smithi191 ceph-mon[35211]: Deploying daemon osd.4 on smithi191 2024-09-11T22:46:47.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:46 smithi204 ceph-mon[35454]: Deploying daemon osd.4 on smithi191 2024-09-11T22:46:47.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:46 smithi152 ceph-mon[31181]: Deploying daemon osd.4 on smithi191 2024-09-11T22:46:48.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:47 smithi191 ceph-mon[35211]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T22:46:48.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:47 smithi204 ceph-mon[35454]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T22:46:48.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:47 smithi152 ceph-mon[31181]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T22:46:50.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:49 smithi191 ceph-mon[35211]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-11T22:46:50.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:49 smithi204 ceph-mon[35454]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-11T22:46:50.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:49 smithi152 ceph-mon[31181]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-11T22:46:52.034 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:51 smithi191 ceph-mon[35211]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T22:46:52.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:51 smithi204 ceph-mon[35454]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T22:46:52.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:51 smithi152 ceph-mon[31181]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T22:46:53.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:52 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:53.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:52 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:53.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:52 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:53.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:52 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:53.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:52 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:53.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:52 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:53.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:52 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:53.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:52 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:53.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:52 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:53.852 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:53 smithi191 ceph-mon[35211]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:46:54.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:53 smithi204 ceph-mon[35454]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:46:54.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:53 smithi152 ceph-mon[31181]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:46:56.094 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:55 smithi191 ceph-mon[35211]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:46:56.094 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:56.094 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:56.094 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:56.094 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:46:56.094 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:56.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:55 smithi152 ceph-mon[31181]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:46:56.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:56.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:56.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:56.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:46:56.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:56.140 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:56.138+0000 7f0b73fff640 1 -- 172.21.15.191:0/4266809294 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0b8c062630 con 0x7f0b740a2b80 2024-09-11T22:46:56.141 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 4 on host 'smithi191' 2024-09-11T22:46:56.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:56.141+0000 7f0b9101a640 1 -- 172.21.15.191:0/4266809294 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0b740a2b80 msgr2=0x7f0b740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:46:56.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:56.141+0000 7f0b9101a640 1 --2- 172.21.15.191:0/4266809294 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0b740a2b80 0x7f0b740a5040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c101de0 tx=0x7f0b7c00b390 comp rx=0 tx=0).stop 2024-09-11T22:46:56.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:56.141+0000 7f0b9101a640 1 -- 172.21.15.191:0/4266809294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b8c102700 msgr2=0x7f0b8c102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:46:56.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:56.141+0000 7f0b9101a640 1 --2- 172.21.15.191:0/4266809294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b8c102700 0x7f0b8c102060 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000e9f0 tx=0x7f0b8000eeb0 comp rx=0 tx=0).stop 2024-09-11T22:46:56.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:56.141+0000 7f0b9101a640 1 -- 172.21.15.191:0/4266809294 shutdown_connections 2024-09-11T22:46:56.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:56.141+0000 7f0b9101a640 1 --2- 172.21.15.191:0/4266809294 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0b740a2b80 0x7f0b740a5040 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:56.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:56.141+0000 7f0b9101a640 1 --2- 172.21.15.191:0/4266809294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b8c107960 0x7f0b8c100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:56.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:56.141+0000 7f0b9101a640 1 --2- 172.21.15.191:0/4266809294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b8c105030 0x7f0b8c1006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:56.143 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:56.141+0000 7f0b9101a640 1 --2- 172.21.15.191:0/4266809294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b8c102700 0x7f0b8c102060 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:46:56.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:56.141+0000 7f0b9101a640 1 -- 172.21.15.191:0/4266809294 >> 172.21.15.191:0/4266809294 conn(0x7f0b8c0fc280 msgr2=0x7f0b8c102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:46:56.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:56.141+0000 7f0b9101a640 1 -- 172.21.15.191:0/4266809294 shutdown_connections 2024-09-11T22:46:56.144 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:46:56.141+0000 7f0b9101a640 1 -- 172.21.15.191:0/4266809294 wait complete. 2024-09-11T22:46:56.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:55 smithi204 ceph-mon[35454]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:46:56.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:56.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:56.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:56.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:46:56.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:56.881 DEBUG:teuthology.orchestra.run.smithi191:osd.4> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T22:46:56.884 INFO:tasks.cephadm:Deploying osd.5 on smithi204 with /dev/vg_nvme/lv_3... 2024-09-11T22:46:56.884 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T22:46:57.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:56 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:57.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:56 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:57.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:56 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:57.136 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:46:57.212 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:56 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:57.212 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:56 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:57.212 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:56 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:57.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:56 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:46:57.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:56 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:57.250 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:56 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:58.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:57 smithi152 ceph-mon[31181]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:46:58.137 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:57 smithi191 ceph-mon[35211]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:46:58.137 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 22:46:57 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4[46568]: 2024-09-11T22:46:57.929+0000 7fc0cf5f5640 -1 osd.4 0 log_to_monitors true 2024-09-11T22:46:58.214 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:57 smithi204 ceph-mon[35454]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:46:59.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:58 smithi152 ceph-mon[31181]: from='osd.4 [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:46:59.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:58 smithi152 ceph-mon[31181]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:46:59.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:58 smithi191 ceph-mon[35211]: from='osd.4 [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:46:59.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:58 smithi191 ceph-mon[35211]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:46:59.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:58 smithi204 ceph-mon[35454]: from='osd.4 [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:46:59.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:58 smithi204 ceph-mon[35454]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:46:59.667 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:46:59.694 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_3 2024-09-11T22:46:59.949 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:46:59.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:59 smithi204 ceph-mon[35454]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:46:59.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:59 smithi204 ceph-mon[35454]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T22:46:59.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:59 smithi204 ceph-mon[35454]: from='osd.4 [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:46:59.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:59 smithi204 ceph-mon[35454]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T22:46:59.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:59 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:46:59.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:59 smithi204 ceph-mon[35454]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:46:59.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:59 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:59.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:59 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:59.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:59 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:46:59.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:59 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:46:59.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:59 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:46:59.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:59 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:46:59.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:59 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:46:59.975 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:46:59 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:00.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:59 smithi191 ceph-mon[35211]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:47:00.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:59 smithi191 ceph-mon[35211]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T22:47:00.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:59 smithi191 ceph-mon[35211]: from='osd.4 [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:47:00.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:59 smithi191 ceph-mon[35211]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T22:47:00.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:59 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:47:00.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:59 smithi191 ceph-mon[35211]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:47:00.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:59 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:00.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:59 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:00.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:59 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:00.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:59 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:00.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:59 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:00.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:59 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:00.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:59 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:47:00.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:46:59 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:00.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:59 smithi152 ceph-mon[31181]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:47:00.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:59 smithi152 ceph-mon[31181]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T22:47:00.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:59 smithi152 ceph-mon[31181]: from='osd.4 [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:47:00.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:59 smithi152 ceph-mon[31181]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T22:47:00.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:59 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:47:00.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:59 smithi152 ceph-mon[31181]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:47:00.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:59 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:00.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:59 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:00.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:59 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:00.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:59 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:00.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:59 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:00.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:59 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:00.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:59 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:47:00.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:46:59 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:00.401 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.377+0000 7f82e04de640 1 -- 172.21.15.204:0/3958761621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d8102690 msgr2=0x7f82d8108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:47:00.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.377+0000 7f82e04de640 1 --2- 172.21.15.204:0/3958761621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d8102690 0x7f82d8108f20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009ca0 tx=0x7f82d002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:47:00.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.377+0000 7f82e04de640 1 -- 172.21.15.204:0/3958761621 shutdown_connections 2024-09-11T22:47:00.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.377+0000 7f82e04de640 1 --2- 172.21.15.204:0/3958761621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d8102690 0x7f82d8108f20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:00.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.377+0000 7f82e04de640 1 --2- 172.21.15.204:0/3958761621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d8101cd0 0x7f82d8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:00.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.377+0000 7f82e04de640 1 --2- 172.21.15.204:0/3958761621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82d8100ad0 0x7f82d8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:00.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.377+0000 7f82e04de640 1 -- 172.21.15.204:0/3958761621 >> 172.21.15.204:0/3958761621 conn(0x7f82d80fc280 msgr2=0x7f82d80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:47:00.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.377+0000 7f82e04de640 1 -- 172.21.15.204:0/3958761621 shutdown_connections 2024-09-11T22:47:00.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.377+0000 7f82e04de640 1 -- 172.21.15.204:0/3958761621 wait complete. 2024-09-11T22:47:00.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.377+0000 7f82e04de640 1 Processor -- start 2024-09-11T22:47:00.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.377+0000 7f82e04de640 1 -- start start 2024-09-11T22:47:00.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82e04de640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d8100ad0 0x7f82d8104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:47:00.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82e04de640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82d8101cd0 0x7f82d81030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:47:00.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82e04de640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d8102690 0x7f82d81035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:47:00.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82e04de640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f82d810c750 con 0x7f82d8100ad0 2024-09-11T22:47:00.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82e04de640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f82d810c5d0 con 0x7f82d8101cd0 2024-09-11T22:47:00.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82e04de640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82d810c8d0 con 0x7f82d8102690 2024-09-11T22:47:00.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82dea7b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d8102690 0x7f82d81035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:47:00.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82dda79640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82d8101cd0 0x7f82d81030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:47:00.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82dea7b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d8102690 0x7f82d81035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:51476/0 (socket says 172.21.15.204:51476) 2024-09-11T22:47:00.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82dea7b640 1 -- 172.21.15.204:0/3902790565 learned_addr learned my addr 172.21.15.204:0/3902790565 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-11T22:47:00.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82de27a640 1 --2- 172.21.15.204:0/3902790565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d8100ad0 0x7f82d8104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:47:00.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82dea7b640 1 -- 172.21.15.204:0/3902790565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82d8101cd0 msgr2=0x7f82d81030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:47:00.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82dea7b640 1 --2- 172.21.15.204:0/3902790565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82d8101cd0 0x7f82d81030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:00.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82dea7b640 1 -- 172.21.15.204:0/3902790565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d8100ad0 msgr2=0x7f82d8104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:47:00.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82dea7b640 1 --2- 172.21.15.204:0/3902790565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d8100ad0 0x7f82d8104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:00.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82dea7b640 1 -- 172.21.15.204:0/3902790565 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d8103eb0 con 0x7f82d8102690 2024-09-11T22:47:00.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82dda79640 1 --2- 172.21.15.204:0/3902790565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82d8101cd0 0x7f82d81030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:47:00.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82de27a640 1 --2- 172.21.15.204:0/3902790565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d8100ad0 0x7f82d8104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:47:00.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.378+0000 7f82dea7b640 1 --2- 172.21.15.204:0/3902790565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d8102690 0x7f82d81035f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f82d0031ae0 tx=0x7f82d0031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:47:00.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.379+0000 7f82cf7fe640 1 -- 172.21.15.204:0/3902790565 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0003b70 con 0x7f82d8102690 2024-09-11T22:47:00.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.379+0000 7f82e04de640 1 -- 172.21.15.204:0/3902790565 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d81a1ff0 con 0x7f82d8102690 2024-09-11T22:47:00.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.379+0000 7f82cf7fe640 1 -- 172.21.15.204:0/3902790565 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d0003d10 con 0x7f82d8102690 2024-09-11T22:47:00.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.379+0000 7f82cf7fe640 1 -- 172.21.15.204:0/3902790565 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d002fc40 con 0x7f82d8102690 2024-09-11T22:47:00.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.379+0000 7f82e04de640 1 -- 172.21.15.204:0/3902790565 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d81a24f0 con 0x7f82d8102690 2024-09-11T22:47:00.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.380+0000 7f82e04de640 1 -- 172.21.15.204:0/3902790565 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d81a2720 con 0x7f82d8102690 2024-09-11T22:47:00.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.381+0000 7f82cf7fe640 1 -- 172.21.15.204:0/3902790565 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f82d0004050 con 0x7f82d8102690 2024-09-11T22:47:00.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.382+0000 7f82cf7fe640 1 --2- 172.21.15.204:0/3902790565 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f82c40a2b80 0x7f82c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:47:00.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.382+0000 7f82cf7fe640 1 -- 172.21.15.204:0/3902790565 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7f82d00ef1f0 con 0x7f82d8102690 2024-09-11T22:47:00.405 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.382+0000 7f82de27a640 1 --2- 172.21.15.204:0/3902790565 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f82c40a2b80 0x7f82c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:47:00.405 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.383+0000 7f82de27a640 1 --2- 172.21.15.204:0/3902790565 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f82c40a2b80 0x7f82c40a5040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f82d8101b30 tx=0x7f82c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:47:00.405 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.385+0000 7f82cf7fe640 1 -- 172.21.15.204:0/3902790565 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82d00bd2e0 con 0x7f82d8102690 2024-09-11T22:47:00.569 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:00.567+0000 7f82e04de640 1 -- 172.21.15.204:0/3902790565 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f82d81a28f0 con 0x7f82c40a2b80 2024-09-11T22:47:00.787 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 22:47:00 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4[46568]: 2024-09-11T22:47:00.299+0000 7fc0cb578640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T22:47:00.787 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 22:47:00 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4[46568]: 2024-09-11T22:47:00.304+0000 7fc0c3398640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:47:01.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:00 smithi204 ceph-mon[35454]: Detected new or changed devices on smithi191 2024-09-11T22:47:01.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:00 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi191 to 8544M 2024-09-11T22:47:01.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:00 smithi204 ceph-mon[35454]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-11T22:47:01.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:00 smithi204 ceph-mon[35454]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T22:47:01.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:47:01.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:47:01.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:00 smithi204 ceph-mon[35454]: from='osd.4 ' entity='osd.4' 2024-09-11T22:47:01.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:47:01.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:47:01.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:01.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:00 smithi191 ceph-mon[35211]: Detected new or changed devices on smithi191 2024-09-11T22:47:01.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:00 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi191 to 8544M 2024-09-11T22:47:01.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:00 smithi191 ceph-mon[35211]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-11T22:47:01.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:00 smithi191 ceph-mon[35211]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T22:47:01.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:47:01.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:47:01.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:00 smithi191 ceph-mon[35211]: from='osd.4 ' entity='osd.4' 2024-09-11T22:47:01.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:47:01.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:47:01.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:01.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:00 smithi152 ceph-mon[31181]: Detected new or changed devices on smithi191 2024-09-11T22:47:01.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:00 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi191 to 8544M 2024-09-11T22:47:01.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:00 smithi152 ceph-mon[31181]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-11T22:47:01.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:00 smithi152 ceph-mon[31181]: osdmap e31: 5 total, 4 up, 5 in 2024-09-11T22:47:01.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:47:01.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:47:01.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:00 smithi152 ceph-mon[31181]: from='osd.4 ' entity='osd.4' 2024-09-11T22:47:01.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:47:01.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:47:01.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:02.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:01 smithi204 ceph-mon[35454]: purged_snaps scrub starts 2024-09-11T22:47:02.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:01 smithi204 ceph-mon[35454]: purged_snaps scrub ok 2024-09-11T22:47:02.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:01 smithi204 ceph-mon[35454]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:47:02.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:01 smithi204 ceph-mon[35454]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:47:02.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:01 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:47:02.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:01 smithi204 ceph-mon[35454]: osd.4 [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563] boot 2024-09-11T22:47:02.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:01 smithi204 ceph-mon[35454]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T22:47:02.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:01 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:47:02.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:01 smithi191 ceph-mon[35211]: purged_snaps scrub starts 2024-09-11T22:47:02.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:01 smithi191 ceph-mon[35211]: purged_snaps scrub ok 2024-09-11T22:47:02.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:01 smithi191 ceph-mon[35211]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:47:02.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:01 smithi191 ceph-mon[35211]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:47:02.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:01 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:47:02.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:01 smithi191 ceph-mon[35211]: osd.4 [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563] boot 2024-09-11T22:47:02.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:01 smithi191 ceph-mon[35211]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T22:47:02.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:01 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:47:02.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:01 smithi152 ceph-mon[31181]: purged_snaps scrub starts 2024-09-11T22:47:02.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:01 smithi152 ceph-mon[31181]: purged_snaps scrub ok 2024-09-11T22:47:02.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:01 smithi152 ceph-mon[31181]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:47:02.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:01 smithi152 ceph-mon[31181]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:47:02.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:01 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:47:02.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:01 smithi152 ceph-mon[31181]: osd.4 [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563] boot 2024-09-11T22:47:02.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:01 smithi152 ceph-mon[31181]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T22:47:02.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:01 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:47:03.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:03 smithi204 ceph-mon[35454]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T22:47:03.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:03 smithi204 ceph-mon[35454]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:03.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:03 smithi204 ceph-mon[35454]: from='client.? 172.21.15.204:0/3592265842' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89dabd3e-77e3-4294-9648-b112ef694e0a"}]: dispatch 2024-09-11T22:47:03.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:03 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89dabd3e-77e3-4294-9648-b112ef694e0a"}]: dispatch 2024-09-11T22:47:03.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:03 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89dabd3e-77e3-4294-9648-b112ef694e0a"}]': finished 2024-09-11T22:47:03.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:03 smithi204 ceph-mon[35454]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T22:47:03.541 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:03 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:03.621 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:03 smithi152 ceph-mon[31181]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T22:47:03.621 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:03 smithi152 ceph-mon[31181]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:03.621 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:03 smithi152 ceph-mon[31181]: from='client.? 172.21.15.204:0/3592265842' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89dabd3e-77e3-4294-9648-b112ef694e0a"}]: dispatch 2024-09-11T22:47:03.621 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:03 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89dabd3e-77e3-4294-9648-b112ef694e0a"}]: dispatch 2024-09-11T22:47:03.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:03 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89dabd3e-77e3-4294-9648-b112ef694e0a"}]': finished 2024-09-11T22:47:03.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:03 smithi152 ceph-mon[31181]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T22:47:03.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:03 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:03.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:03 smithi191 ceph-mon[35211]: osdmap e33: 5 total, 5 up, 5 in 2024-09-11T22:47:03.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:03 smithi191 ceph-mon[35211]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:03.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:03 smithi191 ceph-mon[35211]: from='client.? 172.21.15.204:0/3592265842' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89dabd3e-77e3-4294-9648-b112ef694e0a"}]: dispatch 2024-09-11T22:47:03.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:03 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89dabd3e-77e3-4294-9648-b112ef694e0a"}]: dispatch 2024-09-11T22:47:03.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:03 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89dabd3e-77e3-4294-9648-b112ef694e0a"}]': finished 2024-09-11T22:47:03.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:03 smithi191 ceph-mon[35211]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T22:47:03.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:03 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:04.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:04 smithi152 ceph-mon[31181]: from='client.? 172.21.15.204:0/31304861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:47:04.697 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:04 smithi204 ceph-mon[35454]: from='client.? 172.21.15.204:0/31304861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:47:04.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:04 smithi191 ceph-mon[35211]: from='client.? 172.21.15.204:0/31304861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:47:05.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:05 smithi152 ceph-mon[31181]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:05.708 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:05 smithi204 ceph-mon[35454]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:05.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:05 smithi191 ceph-mon[35211]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:08.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:07 smithi191 ceph-mon[35211]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:08.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:07 smithi204 ceph-mon[35454]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:08.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:07 smithi152 ceph-mon[31181]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:10.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:09 smithi191 ceph-mon[35211]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:09 smithi204 ceph-mon[35454]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:10.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:09 smithi152 ceph-mon[31181]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:12.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:11 smithi191 ceph-mon[35211]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:12.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:11 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T22:47:12.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:11 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:11 smithi204 ceph-mon[35454]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:11 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T22:47:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:11 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:12.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:11 smithi152 ceph-mon[31181]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:12.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:11 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T22:47:12.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:11 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:13.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:12 smithi191 ceph-mon[35211]: Deploying daemon osd.5 on smithi204 2024-09-11T22:47:13.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:12 smithi204 ceph-mon[35454]: Deploying daemon osd.5 on smithi204 2024-09-11T22:47:13.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:12 smithi152 ceph-mon[31181]: Deploying daemon osd.5 on smithi204 2024-09-11T22:47:14.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:13 smithi191 ceph-mon[35211]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:14.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:13 smithi204 ceph-mon[35454]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:14.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:13 smithi152 ceph-mon[31181]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:16.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:15 smithi191 ceph-mon[35211]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:16.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:15 smithi204 ceph-mon[35454]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:16.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:15 smithi152 ceph-mon[31181]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:18.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:17 smithi204 ceph-mon[35454]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:18.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:47:18.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:18.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:18.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:17 smithi152 ceph-mon[31181]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:18.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:47:18.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:18.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:18.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:17 smithi191 ceph-mon[35211]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:18.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:47:18.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:18.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:20.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:19 smithi204 ceph-mon[35454]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:20.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:19 smithi152 ceph-mon[31181]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:20.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:19 smithi191 ceph-mon[35211]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:21.426 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:21 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:21.426 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:21 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:21.426 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:21 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:21.426 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:21 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:47:21.426 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:21 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:21.426 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:21 smithi204 ceph-mon[35454]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:21.469 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 5 on host 'smithi204' 2024-09-11T22:47:21.470 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:21.468+0000 7f82cf7fe640 1 -- 172.21.15.204:0/3902790565 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f82d81a28f0 con 0x7f82c40a2b80 2024-09-11T22:47:21.472 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:21.471+0000 7f82e04de640 1 -- 172.21.15.204:0/3902790565 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f82c40a2b80 msgr2=0x7f82c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:47:21.472 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:21.471+0000 7f82e04de640 1 --2- 172.21.15.204:0/3902790565 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f82c40a2b80 0x7f82c40a5040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f82d8101b30 tx=0x7f82c8005f90 comp rx=0 tx=0).stop 2024-09-11T22:47:21.472 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:21.471+0000 7f82e04de640 1 -- 172.21.15.204:0/3902790565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d8102690 msgr2=0x7f82d81035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:47:21.472 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:21.471+0000 7f82e04de640 1 --2- 172.21.15.204:0/3902790565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d8102690 0x7f82d81035f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f82d0031ae0 tx=0x7f82d0031ee0 comp rx=0 tx=0).stop 2024-09-11T22:47:21.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:21.471+0000 7f82e04de640 1 -- 172.21.15.204:0/3902790565 shutdown_connections 2024-09-11T22:47:21.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:21.471+0000 7f82e04de640 1 --2- 172.21.15.204:0/3902790565 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f82c40a2b80 0x7f82c40a5040 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:21.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:21.471+0000 7f82e04de640 1 --2- 172.21.15.204:0/3902790565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82d8102690 0x7f82d81035f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:21.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:21.471+0000 7f82e04de640 1 --2- 172.21.15.204:0/3902790565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82d8101cd0 0x7f82d81030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:21.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:21.471+0000 7f82e04de640 1 --2- 172.21.15.204:0/3902790565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82d8100ad0 0x7f82d8104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:21.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:21.471+0000 7f82e04de640 1 -- 172.21.15.204:0/3902790565 >> 172.21.15.204:0/3902790565 conn(0x7f82d80fc280 msgr2=0x7f82d80fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:47:21.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:21.471+0000 7f82e04de640 1 -- 172.21.15.204:0/3902790565 shutdown_connections 2024-09-11T22:47:21.473 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:47:21.471+0000 7f82e04de640 1 -- 172.21.15.204:0/3902790565 wait complete. 2024-09-11T22:47:21.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:21 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:21.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:21 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:21.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:21 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:21.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:21 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:47:21.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:21 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:21.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:21 smithi191 ceph-mon[35211]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:21.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:21.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:21.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:21.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:47:21.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:21.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:21 smithi152 ceph-mon[31181]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:22.493 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:22 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:47:22.493 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:22 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:22.493 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:22 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:22.517 DEBUG:teuthology.orchestra.run.smithi204:osd.5> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T22:47:22.520 INFO:tasks.cephadm:Deploying osd.6 on smithi152 with /dev/vg_nvme/lv_2... 2024-09-11T22:47:22.520 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T22:47:22.535 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:22 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:47:22.535 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:22 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:22.535 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:22 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:22.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:22 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:47:22.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:22 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:22.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:22 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:22.780 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:47:23.373 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:23 smithi204 ceph-mon[35454]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:23.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:23 smithi191 ceph-mon[35211]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:23.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:23 smithi152 ceph-mon[31181]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:23.658 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 22:47:23 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T22:47:23.371+0000 7fa7f334f640 -1 osd.5 0 log_to_monitors true 2024-09-11T22:47:24.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:24 smithi152 ceph-mon[31181]: from='osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T22:47:24.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:24 smithi191 ceph-mon[35211]: from='osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T22:47:24.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:24 smithi204 ceph-mon[35454]: from='osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T22:47:25.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:25 smithi191 ceph-mon[35211]: from='osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T22:47:25.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:25 smithi191 ceph-mon[35211]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T22:47:25.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:25 smithi191 ceph-mon[35211]: from='osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:47:25.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:25 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:25.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:25 smithi191 ceph-mon[35211]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:25.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:25 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:25.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:25 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:25.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:25 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:25.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:25 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:25.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:25 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:25.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:25 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:25.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:25 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:47:25.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:25 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:25.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:25 smithi204 ceph-mon[35454]: from='osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T22:47:25.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:25 smithi204 ceph-mon[35454]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T22:47:25.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:25 smithi204 ceph-mon[35454]: from='osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:47:25.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:25 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:25.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:25 smithi204 ceph-mon[35454]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:25.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:25 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:25.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:25 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:25.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:25 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:25.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:25 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:25.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:25 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:25.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:25 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:25.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:25 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:47:25.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:25 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:25.555 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:25 smithi152 ceph-mon[31181]: from='osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T22:47:25.555 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:25 smithi152 ceph-mon[31181]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T22:47:25.555 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:25 smithi152 ceph-mon[31181]: from='osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:47:25.555 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:25 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:25.555 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:25 smithi152 ceph-mon[31181]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:25.555 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:25 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:25.555 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:25 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:25.555 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:25 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:25.555 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:25 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:25.555 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:25 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:25.555 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:25 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:25.556 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:25 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:47:25.556 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:25 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:25.558 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:47:25.584 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi152:vg_nvme/lv_2 2024-09-11T22:47:25.869 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:47:26.046 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 22:47:25 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T22:47:25.630+0000 7fa7ef2d2640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T22:47:26.046 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 22:47:25 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T22:47:25.637+0000 7fa7e70f2640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:47:26.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.320+0000 7fd682177640 1 -- 172.21.15.152:0/2868198167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd67c102690 msgr2=0x7fd67c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:47:26.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.320+0000 7fd682177640 1 --2- 172.21.15.152:0/2868198167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd67c102690 0x7fd67c108f20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd674009ca0 tx=0x7fd67402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:47:26.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.320+0000 7fd682177640 1 -- 172.21.15.152:0/2868198167 shutdown_connections 2024-09-11T22:47:26.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.320+0000 7fd682177640 1 --2- 172.21.15.152:0/2868198167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd67c102690 0x7fd67c108f20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:26.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.320+0000 7fd682177640 1 --2- 172.21.15.152:0/2868198167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd67c101cd0 0x7fd67c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:26.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.320+0000 7fd682177640 1 --2- 172.21.15.152:0/2868198167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd67c100ad0 0x7fd67c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:26.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.320+0000 7fd682177640 1 -- 172.21.15.152:0/2868198167 >> 172.21.15.152:0/2868198167 conn(0x7fd67c0fc280 msgr2=0x7fd67c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:47:26.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.320+0000 7fd682177640 1 -- 172.21.15.152:0/2868198167 shutdown_connections 2024-09-11T22:47:26.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.320+0000 7fd682177640 1 -- 172.21.15.152:0/2868198167 wait complete. 2024-09-11T22:47:26.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd682177640 1 Processor -- start 2024-09-11T22:47:26.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd682177640 1 -- start start 2024-09-11T22:47:26.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd682177640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd67c100ad0 0x7fd67c104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:47:26.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd682177640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd67c101cd0 0x7fd67c1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:47:26.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd682177640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd67c102690 0x7fd67c1035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:47:26.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd682177640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c10c800 con 0x7fd67c100ad0 2024-09-11T22:47:26.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd682177640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c10c680 con 0x7fd67c102690 2024-09-11T22:47:26.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd682177640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd67c10c980 con 0x7fd67c101cd0 2024-09-11T22:47:26.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd67b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd67c100ad0 0x7fd67c104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:47:26.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd67b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd67c100ad0 0x7fd67c104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47030/0 (socket says 172.21.15.152:47030) 2024-09-11T22:47:26.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd67b7fe640 1 -- 172.21.15.152:0/1254190745 learned_addr learned my addr 172.21.15.152:0/1254190745 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:47:26.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd673fff640 1 --2- 172.21.15.152:0/1254190745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd67c101cd0 0x7fd67c1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:47:26.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd67bfff640 1 --2- 172.21.15.152:0/1254190745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd67c102690 0x7fd67c1035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:47:26.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd67b7fe640 1 -- 172.21.15.152:0/1254190745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd67c101cd0 msgr2=0x7fd67c1030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:47:26.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd67b7fe640 1 --2- 172.21.15.152:0/1254190745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd67c101cd0 0x7fd67c1030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:26.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd67b7fe640 1 -- 172.21.15.152:0/1254190745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd67c102690 msgr2=0x7fd67c1035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:47:26.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd67b7fe640 1 --2- 172.21.15.152:0/1254190745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd67c102690 0x7fd67c1035f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:26.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd67b7fe640 1 -- 172.21.15.152:0/1254190745 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67c103eb0 con 0x7fd67c100ad0 2024-09-11T22:47:26.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd67bfff640 1 --2- 172.21.15.152:0/1254190745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd67c102690 0x7fd67c1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:47:26.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.321+0000 7fd67b7fe640 1 --2- 172.21.15.152:0/1254190745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd67c100ad0 0x7fd67c104a60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00b450 tx=0x7fd66c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:47:26.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.322+0000 7fd6797fa640 1 -- 172.21.15.152:0/1254190745 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c015020 con 0x7fd67c100ad0 2024-09-11T22:47:26.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.322+0000 7fd682177640 1 -- 172.21.15.152:0/1254190745 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67c1a20e0 con 0x7fd67c100ad0 2024-09-11T22:47:26.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.322+0000 7fd6797fa640 1 -- 172.21.15.152:0/1254190745 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd66c011d70 con 0x7fd67c100ad0 2024-09-11T22:47:26.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.322+0000 7fd6797fa640 1 -- 172.21.15.152:0/1254190745 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c00f340 con 0x7fd67c100ad0 2024-09-11T22:47:26.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.322+0000 7fd682177640 1 -- 172.21.15.152:0/1254190745 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67c1a26c0 con 0x7fd67c100ad0 2024-09-11T22:47:26.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.323+0000 7fd682177640 1 -- 172.21.15.152:0/1254190745 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67c1a2950 con 0x7fd67c100ad0 2024-09-11T22:47:26.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.324+0000 7fd6797fa640 1 -- 172.21.15.152:0/1254190745 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd66c018020 con 0x7fd67c100ad0 2024-09-11T22:47:26.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.324+0000 7fd6797fa640 1 --2- 172.21.15.152:0/1254190745 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd6680a2b80 0x7fd6680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:47:26.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.325+0000 7fd673fff640 1 --2- 172.21.15.152:0/1254190745 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd6680a2b80 0x7fd6680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:47:26.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.325+0000 7fd6797fa640 1 -- 172.21.15.152:0/1254190745 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4566+0+0 (secure 0 0 0) 0x7fd66c0cdc20 con 0x7fd67c100ad0 2024-09-11T22:47:26.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.325+0000 7fd673fff640 1 --2- 172.21.15.152:0/1254190745 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd6680a2b80 0x7fd6680a5040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd67c1047e0 tx=0x7fd66400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:47:26.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.329+0000 7fd6797fa640 1 -- 172.21.15.152:0/1254190745 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd66c01f020 con 0x7fd67c100ad0 2024-09-11T22:47:26.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:26.504+0000 7fd682177640 1 -- 172.21.15.152:0/1254190745 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd67c0fe4a0 con 0x7fd6680a2b80 2024-09-11T22:47:26.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:26 smithi191 ceph-mon[35211]: Detected new or changed devices on smithi204 2024-09-11T22:47:26.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:26 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi204 to 8544M 2024-09-11T22:47:26.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:26 smithi191 ceph-mon[35211]: from='osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-11T22:47:26.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:26 smithi191 ceph-mon[35211]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T22:47:26.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:26 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:26.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:26 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:26.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:26 smithi191 ceph-mon[35211]: from='osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158]' entity='osd.5' 2024-09-11T22:47:26.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:26 smithi204 ceph-mon[35454]: Detected new or changed devices on smithi204 2024-09-11T22:47:26.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:26 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi204 to 8544M 2024-09-11T22:47:26.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:26 smithi204 ceph-mon[35454]: from='osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-11T22:47:26.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:26 smithi204 ceph-mon[35454]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T22:47:26.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:26 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:26.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:26 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:26.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:26 smithi204 ceph-mon[35454]: from='osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158]' entity='osd.5' 2024-09-11T22:47:26.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:26 smithi152 ceph-mon[31181]: Detected new or changed devices on smithi204 2024-09-11T22:47:26.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:26 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi204 to 8544M 2024-09-11T22:47:26.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:26 smithi152 ceph-mon[31181]: from='osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-11T22:47:26.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:26 smithi152 ceph-mon[31181]: osdmap e36: 6 total, 5 up, 6 in 2024-09-11T22:47:26.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:26 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:26.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:26 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:26.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:26 smithi152 ceph-mon[31181]: from='osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158]' entity='osd.5' 2024-09-11T22:47:27.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:27 smithi191 ceph-mon[35211]: purged_snaps scrub starts 2024-09-11T22:47:27.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:27 smithi191 ceph-mon[35211]: purged_snaps scrub ok 2024-09-11T22:47:27.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:27 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:27.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:27 smithi191 ceph-mon[35211]: osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158] boot 2024-09-11T22:47:27.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:27 smithi191 ceph-mon[35211]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T22:47:27.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:27 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:27.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:27 smithi191 ceph-mon[35211]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:47:27.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:27 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:47:27.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:27 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:47:27.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:27 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:27.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:27 smithi191 ceph-mon[35211]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:27.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:27 smithi204 ceph-mon[35454]: purged_snaps scrub starts 2024-09-11T22:47:27.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:27 smithi204 ceph-mon[35454]: purged_snaps scrub ok 2024-09-11T22:47:27.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:27 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:27.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:27 smithi204 ceph-mon[35454]: osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158] boot 2024-09-11T22:47:27.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:27 smithi204 ceph-mon[35454]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T22:47:27.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:27 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:27.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:27 smithi204 ceph-mon[35454]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:47:27.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:27 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:47:27.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:27 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:47:27.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:27 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:27.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:27 smithi204 ceph-mon[35454]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:27.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:27 smithi152 ceph-mon[31181]: purged_snaps scrub starts 2024-09-11T22:47:27.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:27 smithi152 ceph-mon[31181]: purged_snaps scrub ok 2024-09-11T22:47:27.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:27 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:27.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:27 smithi152 ceph-mon[31181]: osd.5 [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158] boot 2024-09-11T22:47:27.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:27 smithi152 ceph-mon[31181]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T22:47:27.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:27 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:47:27.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:27 smithi152 ceph-mon[31181]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:47:27.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:27 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:47:27.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:27 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:47:27.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:27 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:27.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:27 smithi152 ceph-mon[31181]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:47:28.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:28 smithi191 ceph-mon[35211]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T22:47:28.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:28 smithi204 ceph-mon[35454]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T22:47:28.597 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:28 smithi152 ceph-mon[31181]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T22:47:29.503 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:29 smithi152 ceph-mon[31181]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T22:47:29.503 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:29 smithi152 ceph-mon[31181]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:29.504 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:29 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/979499584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a0fded-b668-4aef-a68f-613cccc83483"}]: dispatch 2024-09-11T22:47:29.504 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:29 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/979499584' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46a0fded-b668-4aef-a68f-613cccc83483"}]': finished 2024-09-11T22:47:29.504 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:29 smithi152 ceph-mon[31181]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T22:47:29.504 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:29 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:29.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:29 smithi204 ceph-mon[35454]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T22:47:29.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:29 smithi204 ceph-mon[35454]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:29.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:29 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/979499584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a0fded-b668-4aef-a68f-613cccc83483"}]: dispatch 2024-09-11T22:47:29.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:29 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/979499584' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46a0fded-b668-4aef-a68f-613cccc83483"}]': finished 2024-09-11T22:47:29.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:29 smithi204 ceph-mon[35454]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T22:47:29.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:29 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:29.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:29 smithi191 ceph-mon[35211]: osdmap e39: 6 total, 6 up, 6 in 2024-09-11T22:47:29.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:29 smithi191 ceph-mon[35211]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:29.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:29 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/979499584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46a0fded-b668-4aef-a68f-613cccc83483"}]: dispatch 2024-09-11T22:47:29.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:29 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/979499584' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46a0fded-b668-4aef-a68f-613cccc83483"}]': finished 2024-09-11T22:47:29.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:29 smithi191 ceph-mon[35211]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T22:47:29.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:29 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:30.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:30 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3757579806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:47:30.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:30 smithi204 ceph-mon[35454]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T22:47:30.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:30 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:30.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:30 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3757579806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:47:30.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:30 smithi152 ceph-mon[31181]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T22:47:30.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:30 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:30.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:30 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3757579806' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:47:30.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:30 smithi191 ceph-mon[35211]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T22:47:30.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:30 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:31.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:31 smithi152 ceph-mon[31181]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:47:31.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:31 smithi191 ceph-mon[35211]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:47:31.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:31 smithi204 ceph-mon[35454]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:47:34.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:33 smithi191 ceph-mon[35211]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-11T22:47:34.048 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:33 smithi204 ceph-mon[35454]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-11T22:47:34.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:33 smithi152 ceph-mon[31181]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-11T22:47:36.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:35 smithi191 ceph-mon[35211]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-11T22:47:36.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:35 smithi204 ceph-mon[35454]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-11T22:47:36.120 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:35 smithi152 ceph-mon[31181]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-11T22:47:38.012 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:37 smithi152 ceph-mon[31181]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T22:47:38.012 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:37 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T22:47:38.012 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:37 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:38.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:37 smithi191 ceph-mon[35211]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T22:47:38.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:37 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T22:47:38.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:37 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:38.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:37 smithi204 ceph-mon[35454]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T22:47:38.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:37 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T22:47:38.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:37 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:39.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:38 smithi191 ceph-mon[35211]: Deploying daemon osd.6 on smithi152 2024-09-11T22:47:39.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:38 smithi204 ceph-mon[35454]: Deploying daemon osd.6 on smithi152 2024-09-11T22:47:39.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:38 smithi152 ceph-mon[31181]: Deploying daemon osd.6 on smithi152 2024-09-11T22:47:40.038 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:39 smithi152 ceph-mon[31181]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T22:47:40.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:39 smithi204 ceph-mon[35454]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T22:47:40.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:39 smithi191 ceph-mon[35211]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T22:47:42.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:41 smithi204 ceph-mon[35454]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-11T22:47:42.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:41 smithi152 ceph-mon[31181]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-11T22:47:42.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:41 smithi191 ceph-mon[35211]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-11T22:47:44.044 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:43 smithi152 ceph-mon[31181]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:44.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:47:44.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:44.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:44.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:43 smithi204 ceph-mon[35454]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:44.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:47:44.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:44.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:44.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:43 smithi191 ceph-mon[35211]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:44.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:47:44.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:44.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:45.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:45 smithi152 ceph-mon[31181]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:46.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:45 smithi191 ceph-mon[35211]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:46.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:45 smithi204 ceph-mon[35454]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:47.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:47.488+0000 7fd6797fa640 1 -- 172.21.15.152:0/1254190745 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd67c0fe4a0 con 0x7fd6680a2b80 2024-09-11T22:47:47.489 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 6 on host 'smithi152' 2024-09-11T22:47:47.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:47.491+0000 7fd682177640 1 -- 172.21.15.152:0/1254190745 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd6680a2b80 msgr2=0x7fd6680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:47:47.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:47.491+0000 7fd682177640 1 --2- 172.21.15.152:0/1254190745 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd6680a2b80 0x7fd6680a5040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd67c1047e0 tx=0x7fd66400a250 comp rx=0 tx=0).stop 2024-09-11T22:47:47.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:47.491+0000 7fd682177640 1 -- 172.21.15.152:0/1254190745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd67c100ad0 msgr2=0x7fd67c104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:47:47.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:47.491+0000 7fd682177640 1 --2- 172.21.15.152:0/1254190745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd67c100ad0 0x7fd67c104a60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00b450 tx=0x7fd66c00b910 comp rx=0 tx=0).stop 2024-09-11T22:47:47.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:47.491+0000 7fd682177640 1 -- 172.21.15.152:0/1254190745 shutdown_connections 2024-09-11T22:47:47.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:47.491+0000 7fd682177640 1 --2- 172.21.15.152:0/1254190745 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd6680a2b80 0x7fd6680a5040 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:47.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:47.491+0000 7fd682177640 1 --2- 172.21.15.152:0/1254190745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd67c102690 0x7fd67c1035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:47.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:47.491+0000 7fd682177640 1 --2- 172.21.15.152:0/1254190745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd67c101cd0 0x7fd67c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:47.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:47.491+0000 7fd682177640 1 --2- 172.21.15.152:0/1254190745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd67c100ad0 0x7fd67c104a60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:47.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:47.491+0000 7fd682177640 1 -- 172.21.15.152:0/1254190745 >> 172.21.15.152:0/1254190745 conn(0x7fd67c0fc280 msgr2=0x7fd67c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:47:47.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:47.491+0000 7fd682177640 1 -- 172.21.15.152:0/1254190745 shutdown_connections 2024-09-11T22:47:47.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:47:47.491+0000 7fd682177640 1 -- 172.21.15.152:0/1254190745 wait complete. 2024-09-11T22:47:47.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:47.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:47.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:47.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:47:47.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:47.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:47 smithi191 ceph-mon[35211]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:47.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:47.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:47.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:47.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:47:47.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:47.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:47 smithi204 ceph-mon[35454]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:47.845 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:47.845 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:47.845 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:47.845 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:47:47.845 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:47.845 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:47 smithi152 ceph-mon[31181]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:48.173 DEBUG:teuthology.orchestra.run.smithi152:osd.6> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T22:47:48.176 INFO:tasks.cephadm:Deploying osd.7 on smithi191 with /dev/vg_nvme/lv_2... 2024-09-11T22:47:48.176 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T22:47:48.430 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:47:48.730 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:48 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:47:48.731 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:48 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:48.731 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:48 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:48.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:48 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:47:48.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:48 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:48.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:48 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:48.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:48 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:47:48.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:48 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:48.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:48 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:49.350 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 22:47:49 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T22:47:49.011+0000 7fdf2e028640 -1 osd.6 0 log_to_monitors true 2024-09-11T22:47:49.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:49 smithi152 ceph-mon[31181]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:49.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:49 smithi152 ceph-mon[31181]: from='osd.6 [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T22:47:49.751 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:49 smithi191 ceph-mon[35211]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:49.752 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:49 smithi191 ceph-mon[35211]: from='osd.6 [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T22:47:49.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:49 smithi204 ceph-mon[35454]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:49.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:49 smithi204 ceph-mon[35454]: from='osd.6 [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T22:47:50.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:50 smithi191 ceph-mon[35211]: from='osd.6 [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T22:47:50.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:50 smithi191 ceph-mon[35211]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T22:47:50.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:50 smithi191 ceph-mon[35211]: from='osd.6 [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-09-11T22:47:50.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:50 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:50.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:50 smithi204 ceph-mon[35454]: from='osd.6 [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T22:47:50.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:50 smithi204 ceph-mon[35454]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T22:47:50.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:50 smithi204 ceph-mon[35454]: from='osd.6 [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-09-11T22:47:50.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:50 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:50.821 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 22:47:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T22:47:50.478+0000 7fdf29fab640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T22:47:50.821 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 22:47:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T22:47:50.489+0000 7fdf25dd3640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:47:50.821 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:50 smithi152 ceph-mon[31181]: from='osd.6 [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T22:47:50.821 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:50 smithi152 ceph-mon[31181]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T22:47:50.821 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:50 smithi152 ceph-mon[31181]: from='osd.6 [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-09-11T22:47:50.821 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:50 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:51.271 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:47:51.299 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi191:vg_nvme/lv_2 2024-09-11T22:47:51.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:51 smithi191 ceph-mon[35211]: from='osd.6 [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-09-11T22:47:51.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:51 smithi191 ceph-mon[35211]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T22:47:51.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:51.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:51.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:51 smithi191 ceph-mon[35211]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:51.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:51.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:51.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:51.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:51.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:51.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:51.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:51.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:47:51.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:51 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:51.549 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:47:51.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:51 smithi152 ceph-mon[31181]: from='osd.6 [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-09-11T22:47:51.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:51 smithi152 ceph-mon[31181]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T22:47:51.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:51.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:51.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:51 smithi152 ceph-mon[31181]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:51.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:51.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:51.625 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:51.626 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:51.626 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:51.626 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:51.626 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:51.626 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:47:51.626 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:51 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:51.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:51 smithi204 ceph-mon[35454]: from='osd.6 [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-09-11T22:47:51.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:51 smithi204 ceph-mon[35454]: osdmap e43: 7 total, 6 up, 7 in 2024-09-11T22:47:51.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:51.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:51.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:51 smithi204 ceph-mon[35454]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:47:51.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:51.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:51.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:51.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:51.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:47:51.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:51.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:51.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:47:51.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:51 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:47:52.013 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.995+0000 7f2ba56c9640 1 -- 172.21.15.191:0/1793078457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ba0101cd0 msgr2=0x7f2ba0102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:47:52.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.995+0000 7f2ba56c9640 1 --2- 172.21.15.191:0/1793078457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ba0101cd0 0x7f2ba0102150 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009900 tx=0x7f2b9402a410 comp rx=0 tx=0).stop 2024-09-11T22:47:52.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.995+0000 7f2ba56c9640 1 -- 172.21.15.191:0/1793078457 shutdown_connections 2024-09-11T22:47:52.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.995+0000 7f2ba56c9640 1 --2- 172.21.15.191:0/1793078457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ba0102690 0x7f2ba0108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:52.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.995+0000 7f2ba56c9640 1 --2- 172.21.15.191:0/1793078457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ba0101cd0 0x7f2ba0102150 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:52.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.995+0000 7f2ba56c9640 1 --2- 172.21.15.191:0/1793078457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba0100ad0 0x7f2ba0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:52.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.995+0000 7f2ba56c9640 1 -- 172.21.15.191:0/1793078457 >> 172.21.15.191:0/1793078457 conn(0x7f2ba00fc280 msgr2=0x7f2ba00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:47:52.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.995+0000 7f2ba56c9640 1 -- 172.21.15.191:0/1793078457 shutdown_connections 2024-09-11T22:47:52.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.995+0000 7f2ba56c9640 1 -- 172.21.15.191:0/1793078457 wait complete. 2024-09-11T22:47:52.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2ba56c9640 1 Processor -- start 2024-09-11T22:47:52.014 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2ba56c9640 1 -- start start 2024-09-11T22:47:52.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2ba56c9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba0100ad0 0x7f2ba0197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:47:52.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2ba56c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ba0101cd0 0x7f2ba0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:47:52.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2ba56c9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ba0102690 0x7f2ba019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:47:52.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2ba56c9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba010d880 con 0x7f2ba0100ad0 2024-09-11T22:47:52.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2ba56c9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba010d700 con 0x7f2ba0102690 2024-09-11T22:47:52.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2ba56c9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba010da00 con 0x7f2ba0101cd0 2024-09-11T22:47:52.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2b9f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ba0102690 0x7f2ba019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:47:52.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2b9f7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ba0102690 0x7f2ba019f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:33830/0 (socket says 172.21.15.191:33830) 2024-09-11T22:47:52.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2b9f7fe640 1 -- 172.21.15.191:0/3674875784 learned_addr learned my addr 172.21.15.191:0/3674875784 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-11T22:47:52.015 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2b9effd640 1 --2- 172.21.15.191:0/3674875784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba0100ad0 0x7f2ba0197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:47:52.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2b9f7fe640 1 -- 172.21.15.191:0/3674875784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ba0101cd0 msgr2=0x7f2ba0198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:47:52.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2b9e7fc640 1 --2- 172.21.15.191:0/3674875784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ba0101cd0 0x7f2ba0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:47:52.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2b9f7fe640 1 --2- 172.21.15.191:0/3674875784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ba0101cd0 0x7f2ba0198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:52.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2b9f7fe640 1 -- 172.21.15.191:0/3674875784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba0100ad0 msgr2=0x7f2ba0197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:47:52.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2b9f7fe640 1 --2- 172.21.15.191:0/3674875784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba0100ad0 0x7f2ba0197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:47:52.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.996+0000 7f2b9f7fe640 1 -- 172.21.15.191:0/3674875784 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba019fb60 con 0x7f2ba0102690 2024-09-11T22:47:52.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.997+0000 7f2b9f7fe640 1 --2- 172.21.15.191:0/3674875784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ba0102690 0x7f2ba019f4f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800cad0 tx=0x7f2b9800cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:47:52.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.997+0000 7f2b87fff640 1 -- 172.21.15.191:0/3674875784 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9800ae20 con 0x7f2ba0102690 2024-09-11T22:47:52.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.997+0000 7f2ba56c9640 1 -- 172.21.15.191:0/3674875784 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba019fe50 con 0x7f2ba0102690 2024-09-11T22:47:52.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.997+0000 7f2b87fff640 1 -- 172.21.15.191:0/3674875784 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b98005020 con 0x7f2ba0102690 2024-09-11T22:47:52.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.997+0000 7f2b87fff640 1 -- 172.21.15.191:0/3674875784 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9800f6f0 con 0x7f2ba0102690 2024-09-11T22:47:52.016 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.997+0000 7f2ba56c9640 1 -- 172.21.15.191:0/3674875784 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba01a02b0 con 0x7f2ba0102690 2024-09-11T22:47:52.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.998+0000 7f2ba56c9640 1 -- 172.21.15.191:0/3674875784 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba0107f20 con 0x7f2ba0102690 2024-09-11T22:47:52.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.999+0000 7f2b87fff640 1 -- 172.21.15.191:0/3674875784 <== mon.2 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2b980051d0 con 0x7f2ba0102690 2024-09-11T22:47:52.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.999+0000 7f2b87fff640 1 --2- 172.21.15.191:0/3674875784 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2b880a2bb0 0x7f2b880a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:47:52.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.999+0000 7f2b87fff640 1 -- 172.21.15.191:0/3674875784 <== mon.2 v2:172.21.15.191:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7f2b980ce370 con 0x7f2ba0102690 2024-09-11T22:47:52.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:51.999+0000 7f2b9effd640 1 --2- 172.21.15.191:0/3674875784 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2b880a2bb0 0x7f2b880a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:47:52.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:52.000+0000 7f2b9effd640 1 --2- 172.21.15.191:0/3674875784 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2b880a2bb0 0x7f2b880a5070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0101b30 tx=0x7f2b9000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:47:52.017 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:52.003+0000 7f2b87fff640 1 -- 172.21.15.191:0/3674875784 <== mon.2 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b98014030 con 0x7f2ba0102690 2024-09-11T22:47:52.180 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:47:52.178+0000 7f2ba56c9640 1 -- 172.21.15.191:0/3674875784 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2ba0062630 con 0x7f2b880a2bb0 2024-09-11T22:47:52.739 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:52 smithi191 ceph-mon[35211]: purged_snaps scrub starts 2024-09-11T22:47:52.739 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:52 smithi191 ceph-mon[35211]: purged_snaps scrub ok 2024-09-11T22:47:52.739 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:52 smithi191 ceph-mon[35211]: Detected new or changed devices on smithi152 2024-09-11T22:47:52.739 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:52 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi152 to 5696M 2024-09-11T22:47:52.739 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:52 smithi191 ceph-mon[35211]: osd.6 [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042] boot 2024-09-11T22:47:52.739 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:52 smithi191 ceph-mon[35211]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T22:47:52.739 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:52 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:52.739 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:52 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:47:52.740 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:52 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:47:52.740 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:52 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:52.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:52 smithi204 ceph-mon[35454]: purged_snaps scrub starts 2024-09-11T22:47:52.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:52 smithi204 ceph-mon[35454]: purged_snaps scrub ok 2024-09-11T22:47:52.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:52 smithi204 ceph-mon[35454]: Detected new or changed devices on smithi152 2024-09-11T22:47:52.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:52 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi152 to 5696M 2024-09-11T22:47:52.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:52 smithi204 ceph-mon[35454]: osd.6 [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042] boot 2024-09-11T22:47:52.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:52 smithi204 ceph-mon[35454]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T22:47:52.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:52 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:52.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:52 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:47:52.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:52 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:47:52.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:52 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:52.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:52 smithi152 ceph-mon[31181]: purged_snaps scrub starts 2024-09-11T22:47:52.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:52 smithi152 ceph-mon[31181]: purged_snaps scrub ok 2024-09-11T22:47:52.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:52 smithi152 ceph-mon[31181]: Detected new or changed devices on smithi152 2024-09-11T22:47:52.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:52 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi152 to 5696M 2024-09-11T22:47:52.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:52 smithi152 ceph-mon[31181]: osd.6 [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042] boot 2024-09-11T22:47:52.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:52 smithi152 ceph-mon[31181]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T22:47:52.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:52 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:47:52.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:52 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:47:52.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:52 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:47:52.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:52 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:47:53.647 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:53 smithi191 ceph-mon[35211]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:47:53.647 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:53 smithi191 ceph-mon[35211]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T22:47:53.647 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:53 smithi191 ceph-mon[35211]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:47:53.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:53 smithi204 ceph-mon[35454]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:47:53.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:53 smithi204 ceph-mon[35454]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T22:47:53.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:53 smithi204 ceph-mon[35454]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:47:53.857 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:53 smithi152 ceph-mon[31181]: from='client.24257 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:47:53.857 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:53 smithi152 ceph-mon[31181]: osdmap e45: 7 total, 7 up, 7 in 2024-09-11T22:47:53.857 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:53 smithi152 ceph-mon[31181]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:47:54.995 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:54 smithi191 ceph-mon[35211]: from='client.? 172.21.15.191:0/77179817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e674bfe9-e2db-44a3-b74b-a21f198747c9"}]: dispatch 2024-09-11T22:47:54.995 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:54 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e674bfe9-e2db-44a3-b74b-a21f198747c9"}]: dispatch 2024-09-11T22:47:54.995 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:54 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e674bfe9-e2db-44a3-b74b-a21f198747c9"}]': finished 2024-09-11T22:47:54.995 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:54 smithi191 ceph-mon[35211]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T22:47:54.995 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:54 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:47:55.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:54 smithi204 ceph-mon[35454]: from='client.? 172.21.15.191:0/77179817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e674bfe9-e2db-44a3-b74b-a21f198747c9"}]: dispatch 2024-09-11T22:47:55.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:54 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e674bfe9-e2db-44a3-b74b-a21f198747c9"}]: dispatch 2024-09-11T22:47:55.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:54 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e674bfe9-e2db-44a3-b74b-a21f198747c9"}]': finished 2024-09-11T22:47:55.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:54 smithi204 ceph-mon[35454]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T22:47:55.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:54 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:47:55.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:54 smithi152 ceph-mon[31181]: from='client.? 172.21.15.191:0/77179817' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e674bfe9-e2db-44a3-b74b-a21f198747c9"}]: dispatch 2024-09-11T22:47:55.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:54 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e674bfe9-e2db-44a3-b74b-a21f198747c9"}]: dispatch 2024-09-11T22:47:55.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:54 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e674bfe9-e2db-44a3-b74b-a21f198747c9"}]': finished 2024-09-11T22:47:55.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:54 smithi152 ceph-mon[31181]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T22:47:55.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:47:56.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:55 smithi191 ceph-mon[35211]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:47:56.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:55 smithi191 ceph-mon[35211]: from='client.? 172.21.15.191:0/626294605' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:47:56.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:55 smithi204 ceph-mon[35454]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:47:56.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:55 smithi204 ceph-mon[35454]: from='client.? 172.21.15.191:0/626294605' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:47:56.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:55 smithi152 ceph-mon[31181]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:47:56.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:55 smithi152 ceph-mon[31181]: from='client.? 172.21.15.191:0/626294605' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:47:58.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:57 smithi191 ceph-mon[35211]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:47:58.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:57 smithi204 ceph-mon[35454]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:47:58.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:57 smithi152 ceph-mon[31181]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:00.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:47:59 smithi191 ceph-mon[35211]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:00.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:47:59 smithi204 ceph-mon[35454]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:00.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:47:59 smithi152 ceph-mon[31181]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:01.822 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:01 smithi191 ceph-mon[35211]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:02.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:01 smithi204 ceph-mon[35454]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:02.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:01 smithi152 ceph-mon[31181]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:03.886 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:03 smithi191 ceph-mon[35211]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:03.887 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:03 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T22:48:03.887 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:03 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:04.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:03 smithi204 ceph-mon[35454]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:04.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:03 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T22:48:04.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:03 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:04.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:03 smithi152 ceph-mon[31181]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:04.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:03 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T22:48:04.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:03 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:04.999 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:04 smithi191 ceph-mon[35211]: Deploying daemon osd.7 on smithi191 2024-09-11T22:48:05.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:04 smithi204 ceph-mon[35454]: Deploying daemon osd.7 on smithi191 2024-09-11T22:48:05.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:04 smithi152 ceph-mon[31181]: Deploying daemon osd.7 on smithi191 2024-09-11T22:48:05.971 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:05 smithi191 ceph-mon[35211]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:06.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:05 smithi204 ceph-mon[35454]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:06.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:05 smithi152 ceph-mon[31181]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:08.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:07 smithi191 ceph-mon[35211]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:08.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:07 smithi204 ceph-mon[35454]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:08.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:07 smithi152 ceph-mon[31181]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:10.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:09 smithi191 ceph-mon[35211]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:10.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:48:10.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:10.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:09 smithi204 ceph-mon[35454]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:48:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:10.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:09 smithi152 ceph-mon[31181]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:10.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:48:10.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:10.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:12.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:11 smithi191 ceph-mon[35211]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:11 smithi204 ceph-mon[35454]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:12.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:11 smithi152 ceph-mon[31181]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:13.356 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:48:13.354+0000 7f2b87fff640 1 -- 172.21.15.191:0/3674875784 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2ba0062630 con 0x7f2b880a2bb0 2024-09-11T22:48:13.356 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 7 on host 'smithi191' 2024-09-11T22:48:13.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:48:13.357+0000 7f2ba56c9640 1 -- 172.21.15.191:0/3674875784 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2b880a2bb0 msgr2=0x7f2b880a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:48:13.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:48:13.357+0000 7f2ba56c9640 1 --2- 172.21.15.191:0/3674875784 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2b880a2bb0 0x7f2b880a5070 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0101b30 tx=0x7f2b9000b390 comp rx=0 tx=0).stop 2024-09-11T22:48:13.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:48:13.357+0000 7f2ba56c9640 1 -- 172.21.15.191:0/3674875784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ba0102690 msgr2=0x7f2ba019f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:48:13.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:48:13.357+0000 7f2ba56c9640 1 --2- 172.21.15.191:0/3674875784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ba0102690 0x7f2ba019f4f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800cad0 tx=0x7f2b9800cf90 comp rx=0 tx=0).stop 2024-09-11T22:48:13.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:48:13.357+0000 7f2ba56c9640 1 -- 172.21.15.191:0/3674875784 shutdown_connections 2024-09-11T22:48:13.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:48:13.357+0000 7f2ba56c9640 1 --2- 172.21.15.191:0/3674875784 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2b880a2bb0 0x7f2b880a5070 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:13.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:48:13.357+0000 7f2ba56c9640 1 --2- 172.21.15.191:0/3674875784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ba0102690 0x7f2ba019f4f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:13.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:48:13.357+0000 7f2ba56c9640 1 --2- 172.21.15.191:0/3674875784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ba0101cd0 0x7f2ba0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:13.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:48:13.357+0000 7f2ba56c9640 1 --2- 172.21.15.191:0/3674875784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ba0100ad0 0x7f2ba0197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:13.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:48:13.357+0000 7f2ba56c9640 1 -- 172.21.15.191:0/3674875784 >> 172.21.15.191:0/3674875784 conn(0x7f2ba00fc280 msgr2=0x7f2ba01050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:48:13.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:48:13.357+0000 7f2ba56c9640 1 -- 172.21.15.191:0/3674875784 shutdown_connections 2024-09-11T22:48:13.358 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:48:13.357+0000 7f2ba56c9640 1 -- 172.21.15.191:0/3674875784 wait complete. 2024-09-11T22:48:13.718 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:13 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:13.718 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:13 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:13.718 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:13 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:13.718 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:13 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:48:13.718 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:13 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:13.718 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:13 smithi191 ceph-mon[35211]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:13.718 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:13 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:48:13.718 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:13 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:13.718 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:13 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:13.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:13 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:13.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:13 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:13.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:13 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:13.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:13 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:48:13.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:13 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:13.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:13 smithi204 ceph-mon[35454]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:13.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:13 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:48:13.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:13 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:13.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:13 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:13.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:13.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:13.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:13.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:48:13.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:13.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:13 smithi152 ceph-mon[31181]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:13.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:48:13.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:13.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:13 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:14.053 DEBUG:teuthology.orchestra.run.smithi191:osd.7> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T22:48:14.056 INFO:tasks.cephadm:Deploying osd.8 on smithi204 with /dev/vg_nvme/lv_2... 2024-09-11T22:48:14.056 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T22:48:14.313 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:48:15.287 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 22:48:14 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T22:48:14.990+0000 7ff72d038640 -1 osd.7 0 log_to_monitors true 2024-09-11T22:48:16.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:15 smithi204 ceph-mon[35454]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:16.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:15 smithi204 ceph-mon[35454]: from='osd.7 [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:48:16.014 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:15 smithi204 ceph-mon[35454]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:48:16.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:15 smithi191 ceph-mon[35211]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:16.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:15 smithi191 ceph-mon[35211]: from='osd.7 [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:48:16.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:15 smithi191 ceph-mon[35211]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:48:16.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:15 smithi152 ceph-mon[31181]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:16.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:15 smithi152 ceph-mon[31181]: from='osd.7 [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:48:16.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:15 smithi152 ceph-mon[31181]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:48:16.994 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:16 smithi191 ceph-mon[35211]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T22:48:16.994 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:16 smithi191 ceph-mon[35211]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T22:48:16.994 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:16 smithi191 ceph-mon[35211]: from='osd.7 [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:48:16.994 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:16 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:48:16.994 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:16 smithi191 ceph-mon[35211]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:48:16.994 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:16 smithi191 ceph-mon[35211]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-11T22:48:16.995 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:16 smithi191 ceph-mon[35211]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T22:48:16.995 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:16 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:48:17.036 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:16 smithi204 ceph-mon[35454]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T22:48:17.036 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:16 smithi204 ceph-mon[35454]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T22:48:17.036 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:16 smithi204 ceph-mon[35454]: from='osd.7 [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:48:17.036 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:16 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:48:17.036 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:16 smithi204 ceph-mon[35454]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:48:17.036 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:16 smithi204 ceph-mon[35454]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-11T22:48:17.036 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:16 smithi204 ceph-mon[35454]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T22:48:17.036 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:16 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:48:17.039 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:48:17.067 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_2 2024-09-11T22:48:17.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:16 smithi152 ceph-mon[31181]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T22:48:17.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:16 smithi152 ceph-mon[31181]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T22:48:17.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:16 smithi152 ceph-mon[31181]: from='osd.7 [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:48:17.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:16 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:48:17.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:16 smithi152 ceph-mon[31181]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:48:17.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:16 smithi152 ceph-mon[31181]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-11T22:48:17.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:16 smithi152 ceph-mon[31181]: osdmap e48: 8 total, 7 up, 8 in 2024-09-11T22:48:17.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:16 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:48:17.287 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T22:48:17.209+0000 7ff728fbb640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T22:48:17.287 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T22:48:17.219+0000 7ff720ddb640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:48:17.318 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:48:17.805 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.796+0000 7f9bcf638640 1 -- 172.21.15.204:0/1788341989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bc8102690 msgr2=0x7f9bc8108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:48:17.805 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.796+0000 7f9bcf638640 1 --2- 172.21.15.204:0/1788341989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bc8102690 0x7f9bc8108f20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009ca0 tx=0x7f9bc002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:48:17.806 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.796+0000 7f9bcf638640 1 -- 172.21.15.204:0/1788341989 shutdown_connections 2024-09-11T22:48:17.806 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.796+0000 7f9bcf638640 1 --2- 172.21.15.204:0/1788341989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bc8102690 0x7f9bc8108f20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:17.806 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.796+0000 7f9bcf638640 1 --2- 172.21.15.204:0/1788341989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bc8101cd0 0x7f9bc8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:17.806 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.796+0000 7f9bcf638640 1 --2- 172.21.15.204:0/1788341989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc8100ad0 0x7f9bc8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:17.806 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.796+0000 7f9bcf638640 1 -- 172.21.15.204:0/1788341989 >> 172.21.15.204:0/1788341989 conn(0x7f9bc80fc280 msgr2=0x7f9bc80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:48:17.806 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.796+0000 7f9bcf638640 1 -- 172.21.15.204:0/1788341989 shutdown_connections 2024-09-11T22:48:17.806 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.796+0000 7f9bcf638640 1 -- 172.21.15.204:0/1788341989 wait complete. 2024-09-11T22:48:17.806 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.797+0000 7f9bcf638640 1 Processor -- start 2024-09-11T22:48:17.806 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.797+0000 7f9bcf638640 1 -- start start 2024-09-11T22:48:17.806 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.797+0000 7f9bcf638640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc8100ad0 0x7f9bc8197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:48:17.806 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.797+0000 7f9bcf638640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bc8101cd0 0x7f9bc81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:48:17.806 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.797+0000 7f9bcf638640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bc8102690 0x7f9bc819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:48:17.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.797+0000 7f9bcf638640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc810c780 con 0x7f9bc8102690 2024-09-11T22:48:17.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.797+0000 7f9bcf638640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc810c600 con 0x7f9bc8100ad0 2024-09-11T22:48:17.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.797+0000 7f9bcf638640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc810c900 con 0x7f9bc8101cd0 2024-09-11T22:48:17.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.797+0000 7f9bccbd3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bc8101cd0 0x7f9bc81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:48:17.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.797+0000 7f9bccbd3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bc8101cd0 0x7f9bc81983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:52948/0 (socket says 172.21.15.204:52948) 2024-09-11T22:48:17.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.797+0000 7f9bcd3d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc8100ad0 0x7f9bc8197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:48:17.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.797+0000 7f9bccbd3640 1 -- 172.21.15.204:0/3089512869 learned_addr learned my addr 172.21.15.204:0/3089512869 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-11T22:48:17.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.797+0000 7f9bcdbd5640 1 --2- 172.21.15.204:0/3089512869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bc8102690 0x7f9bc819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:48:17.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.798+0000 7f9bccbd3640 1 -- 172.21.15.204:0/3089512869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc8100ad0 msgr2=0x7f9bc8197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:48:17.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.798+0000 7f9bccbd3640 1 --2- 172.21.15.204:0/3089512869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc8100ad0 0x7f9bc8197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:17.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.798+0000 7f9bccbd3640 1 -- 172.21.15.204:0/3089512869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bc8102690 msgr2=0x7f9bc819f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:48:17.807 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.798+0000 7f9bccbd3640 1 --2- 172.21.15.204:0/3089512869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bc8102690 0x7f9bc819f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:17.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.798+0000 7f9bccbd3640 1 -- 172.21.15.204:0/3089512869 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc819fac0 con 0x7f9bc8101cd0 2024-09-11T22:48:17.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.798+0000 7f9bcdbd5640 1 --2- 172.21.15.204:0/3089512869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bc8102690 0x7f9bc819f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:48:17.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.798+0000 7f9bcd3d4640 1 --2- 172.21.15.204:0/3089512869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc8100ad0 0x7f9bc8197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:48:17.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.798+0000 7f9bccbd3640 1 --2- 172.21.15.204:0/3089512869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bc8101cd0 0x7f9bc81983d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9bb800d950 tx=0x7f9bb800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:48:17.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.799+0000 7f9bbe7fc640 1 -- 172.21.15.204:0/3089512869 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8017070 con 0x7f9bc8101cd0 2024-09-11T22:48:17.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.799+0000 7f9bbe7fc640 1 -- 172.21.15.204:0/3089512869 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bb8009c30 con 0x7f9bc8101cd0 2024-09-11T22:48:17.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.799+0000 7f9bcf638640 1 -- 172.21.15.204:0/3089512869 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc819fdb0 con 0x7f9bc8101cd0 2024-09-11T22:48:17.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.799+0000 7f9bbe7fc640 1 -- 172.21.15.204:0/3089512869 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb800f6b0 con 0x7f9bc8101cd0 2024-09-11T22:48:17.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.799+0000 7f9bcf638640 1 -- 172.21.15.204:0/3089512869 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc81a0260 con 0x7f9bc8101cd0 2024-09-11T22:48:17.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.800+0000 7f9bcf638640 1 -- 172.21.15.204:0/3089512869 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc81a0490 con 0x7f9bc8101cd0 2024-09-11T22:48:17.808 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.801+0000 7f9bbe7fc640 1 -- 172.21.15.204:0/3089512869 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9bb80109d0 con 0x7f9bc8101cd0 2024-09-11T22:48:17.809 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.802+0000 7f9bbe7fc640 1 --2- 172.21.15.204:0/3089512869 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f9bb40a2bb0 0x7f9bb40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:48:17.809 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.802+0000 7f9bbe7fc640 1 -- 172.21.15.204:0/3089512869 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5430+0+0 (secure 0 0 0) 0x7f9bb80ce050 con 0x7f9bc8101cd0 2024-09-11T22:48:17.809 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.802+0000 7f9bcd3d4640 1 --2- 172.21.15.204:0/3089512869 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f9bb40a2bb0 0x7f9bb40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:48:17.809 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.802+0000 7f9bcd3d4640 1 --2- 172.21.15.204:0/3089512869 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f9bb40a2bb0 0x7f9bb40a5070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8101b30 tx=0x7f9bc4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:48:17.809 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.806+0000 7f9bbe7fc640 1 -- 172.21.15.204:0/3089512869 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9bb8009de0 con 0x7f9bc8101cd0 2024-09-11T22:48:17.982 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:17.981+0000 7f9bcf638640 1 -- 172.21.15.204:0/3089512869 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9bc806a680 con 0x7f9bb40a2bb0 2024-09-11T22:48:17.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:17 smithi204 ceph-mon[35454]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:17.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:48:17.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:17.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:17.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:17.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:17.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:17.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:17.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:17.983 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:48:17.984 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:17.984 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:17 smithi204 ceph-mon[35454]: from='osd.7 ' entity='osd.7' 2024-09-11T22:48:17.984 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:17 smithi204 ceph-mon[35454]: osd.7 [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859] boot 2024-09-11T22:48:17.984 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:17 smithi204 ceph-mon[35454]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T22:48:17.984 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:48:18.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:17 smithi152 ceph-mon[31181]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:18.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:48:18.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:18.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:18.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:18.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:18.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:18.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:18.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:18.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:48:18.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:18.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:17 smithi152 ceph-mon[31181]: from='osd.7 ' entity='osd.7' 2024-09-11T22:48:18.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:17 smithi152 ceph-mon[31181]: osd.7 [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859] boot 2024-09-11T22:48:18.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:17 smithi152 ceph-mon[31181]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T22:48:18.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:48:18.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-mon[35211]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:48:18.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:48:18.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:18.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:18.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:18.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:18.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:18.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:18.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:18.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:48:18.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:18.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-mon[35211]: from='osd.7 ' entity='osd.7' 2024-09-11T22:48:18.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-mon[35211]: osd.7 [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859] boot 2024-09-11T22:48:18.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-mon[35211]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T22:48:18.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:48:19.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:18 smithi204 ceph-mon[35454]: purged_snaps scrub starts 2024-09-11T22:48:19.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:18 smithi204 ceph-mon[35454]: purged_snaps scrub ok 2024-09-11T22:48:19.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:18 smithi204 ceph-mon[35454]: Detected new or changed devices on smithi191 2024-09-11T22:48:19.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:18 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi191 to 5696M 2024-09-11T22:48:19.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:18 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:48:19.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:18 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:48:19.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:18 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:19.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:18 smithi204 ceph-mon[35454]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T22:48:19.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:18 smithi152 ceph-mon[31181]: purged_snaps scrub starts 2024-09-11T22:48:19.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:18 smithi152 ceph-mon[31181]: purged_snaps scrub ok 2024-09-11T22:48:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:18 smithi152 ceph-mon[31181]: Detected new or changed devices on smithi191 2024-09-11T22:48:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:18 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi191 to 5696M 2024-09-11T22:48:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:48:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:48:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:18 smithi152 ceph-mon[31181]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T22:48:19.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:18 smithi191 ceph-mon[35211]: purged_snaps scrub starts 2024-09-11T22:48:19.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:18 smithi191 ceph-mon[35211]: purged_snaps scrub ok 2024-09-11T22:48:19.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:18 smithi191 ceph-mon[35211]: Detected new or changed devices on smithi191 2024-09-11T22:48:19.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:18 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi191 to 5696M 2024-09-11T22:48:19.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:18 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:48:19.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:18 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:48:19.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:18 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:19.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:18 smithi191 ceph-mon[35211]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T22:48:20.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:19 smithi204 ceph-mon[35454]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:48:20.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:19 smithi204 ceph-mon[35454]: pgmap v171: 1 pgs: 1 unknown; 577 KiB data, 259 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:20.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:19 smithi204 ceph-mon[35454]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T22:48:20.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:19 smithi152 ceph-mon[31181]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:48:20.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:19 smithi152 ceph-mon[31181]: pgmap v171: 1 pgs: 1 unknown; 577 KiB data, 259 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:20.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:19 smithi152 ceph-mon[31181]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T22:48:20.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:19 smithi191 ceph-mon[35211]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:48:20.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:19 smithi191 ceph-mon[35211]: pgmap v171: 1 pgs: 1 unknown; 577 KiB data, 259 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:20.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:19 smithi191 ceph-mon[35211]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T22:48:21.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:21 smithi204 ceph-mon[35454]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T22:48:21.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:21 smithi204 ceph-mon[35454]: from='client.? 172.21.15.204:0/1501271696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0418bfc9-d6b6-488f-b4c7-0bc6291bb624"}]: dispatch 2024-09-11T22:48:21.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:21 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0418bfc9-d6b6-488f-b4c7-0bc6291bb624"}]: dispatch 2024-09-11T22:48:21.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:21 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0418bfc9-d6b6-488f-b4c7-0bc6291bb624"}]': finished 2024-09-11T22:48:21.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:21 smithi204 ceph-mon[35454]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T22:48:21.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:21 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:48:21.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:21 smithi204 ceph-mon[35454]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 259 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:21.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:21 smithi204 ceph-mon[35454]: from='client.? 172.21.15.204:0/3249097813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:48:21.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:21 smithi152 ceph-mon[31181]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T22:48:21.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:21 smithi152 ceph-mon[31181]: from='client.? 172.21.15.204:0/1501271696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0418bfc9-d6b6-488f-b4c7-0bc6291bb624"}]: dispatch 2024-09-11T22:48:21.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:21 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0418bfc9-d6b6-488f-b4c7-0bc6291bb624"}]: dispatch 2024-09-11T22:48:21.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:21 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0418bfc9-d6b6-488f-b4c7-0bc6291bb624"}]': finished 2024-09-11T22:48:21.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:21 smithi152 ceph-mon[31181]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T22:48:21.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:48:21.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:21 smithi152 ceph-mon[31181]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 259 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:21.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:21 smithi152 ceph-mon[31181]: from='client.? 172.21.15.204:0/3249097813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:48:21.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:21 smithi191 ceph-mon[35211]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T22:48:21.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:21 smithi191 ceph-mon[35211]: from='client.? 172.21.15.204:0/1501271696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0418bfc9-d6b6-488f-b4c7-0bc6291bb624"}]: dispatch 2024-09-11T22:48:21.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:21 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0418bfc9-d6b6-488f-b4c7-0bc6291bb624"}]: dispatch 2024-09-11T22:48:21.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:21 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0418bfc9-d6b6-488f-b4c7-0bc6291bb624"}]': finished 2024-09-11T22:48:21.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:21 smithi191 ceph-mon[35211]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T22:48:21.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:21 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:48:21.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:21 smithi191 ceph-mon[35211]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 259 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:21.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:21 smithi191 ceph-mon[35211]: from='client.? 172.21.15.204:0/3249097813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:48:24.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:23 smithi191 ceph-mon[35211]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 259 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:24.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:23 smithi204 ceph-mon[35454]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 259 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:24.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:23 smithi152 ceph-mon[31181]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 259 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:25.956 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:25 smithi204 ceph-mon[35454]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:26.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:25 smithi152 ceph-mon[31181]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:26.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:25 smithi191 ceph-mon[35211]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:27.922 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:27 smithi204 ceph-mon[35454]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:28.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:27 smithi152 ceph-mon[31181]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:28.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:27 smithi191 ceph-mon[35211]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:30.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:29 smithi204 ceph-mon[35454]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:30.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:29 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T22:48:30.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:29 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:30.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:29 smithi152 ceph-mon[31181]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:30.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:29 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T22:48:30.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:29 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:30.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:29 smithi191 ceph-mon[35211]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:30.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:29 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T22:48:30.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:29 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:30.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:30 smithi204 ceph-mon[35454]: Deploying daemon osd.8 on smithi204 2024-09-11T22:48:31.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:30 smithi152 ceph-mon[31181]: Deploying daemon osd.8 on smithi204 2024-09-11T22:48:31.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:30 smithi191 ceph-mon[35211]: Deploying daemon osd.8 on smithi204 2024-09-11T22:48:32.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:31 smithi204 ceph-mon[35454]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:32.119 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:31 smithi152 ceph-mon[31181]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:32.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:31 smithi191 ceph-mon[35211]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:34.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:33 smithi204 ceph-mon[35454]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:34.089 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:33 smithi191 ceph-mon[35211]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:34.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:33 smithi152 ceph-mon[31181]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:36.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:35 smithi204 ceph-mon[35454]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:36.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:35 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:48:36.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:35 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:36.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:35 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:36.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:35 smithi152 ceph-mon[31181]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:36.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:35 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:48:36.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:35 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:36.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:35 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:36.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:35 smithi191 ceph-mon[35211]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:36.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:35 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:48:36.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:35 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:36.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:35 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:38.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:37 smithi204 ceph-mon[35454]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:38.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:37 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:38.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:37 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:38.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:37 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:38.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:37 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:48:38.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:37 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:38.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:37 smithi152 ceph-mon[31181]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:38.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:37 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:38.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:37 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:38.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:37 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:38.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:37 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:48:38.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:37 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:38.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:37 smithi191 ceph-mon[35211]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:38.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:37 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:38.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:37 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:38.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:37 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:38.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:37 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:48:38.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:37 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:38.824 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:38.823+0000 7f9bbe7fc640 1 -- 172.21.15.204:0/3089512869 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9bc806a680 con 0x7f9bb40a2bb0 2024-09-11T22:48:38.824 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 8 on host 'smithi204' 2024-09-11T22:48:38.826 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:38.825+0000 7f9bcf638640 1 -- 172.21.15.204:0/3089512869 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f9bb40a2bb0 msgr2=0x7f9bb40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:48:38.826 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:38.825+0000 7f9bcf638640 1 --2- 172.21.15.204:0/3089512869 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f9bb40a2bb0 0x7f9bb40a5070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8101b30 tx=0x7f9bc4005f90 comp rx=0 tx=0).stop 2024-09-11T22:48:38.826 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:38.825+0000 7f9bcf638640 1 -- 172.21.15.204:0/3089512869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bc8101cd0 msgr2=0x7f9bc81983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:48:38.826 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:38.825+0000 7f9bcf638640 1 --2- 172.21.15.204:0/3089512869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bc8101cd0 0x7f9bc81983d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9bb800d950 tx=0x7f9bb800de10 comp rx=0 tx=0).stop 2024-09-11T22:48:38.826 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:38.825+0000 7f9bcf638640 1 -- 172.21.15.204:0/3089512869 shutdown_connections 2024-09-11T22:48:38.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:38.825+0000 7f9bcf638640 1 --2- 172.21.15.204:0/3089512869 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f9bb40a2bb0 0x7f9bb40a5070 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:38.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:38.825+0000 7f9bcf638640 1 --2- 172.21.15.204:0/3089512869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bc8102690 0x7f9bc819f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:38.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:38.825+0000 7f9bcf638640 1 --2- 172.21.15.204:0/3089512869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bc8101cd0 0x7f9bc81983d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:38.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:38.825+0000 7f9bcf638640 1 --2- 172.21.15.204:0/3089512869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bc8100ad0 0x7f9bc8197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:38.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:38.825+0000 7f9bcf638640 1 -- 172.21.15.204:0/3089512869 >> 172.21.15.204:0/3089512869 conn(0x7f9bc80fc280 msgr2=0x7f9bc8069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:48:38.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:38.826+0000 7f9bcf638640 1 -- 172.21.15.204:0/3089512869 shutdown_connections 2024-09-11T22:48:38.827 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:48:38.826+0000 7f9bcf638640 1 -- 172.21.15.204:0/3089512869 wait complete. 2024-09-11T22:48:39.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:38 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:48:39.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:38 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:39.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:38 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:39.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:38 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:48:39.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:38 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:39.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:38 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:39.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:38 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:48:39.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:38 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:39.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:38 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:39.550 DEBUG:teuthology.orchestra.run.smithi204:osd.8> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.8.service 2024-09-11T22:48:39.552 INFO:tasks.cephadm:Deploying osd.9 on smithi152 with /dev/vg_nvme/lv_1... 2024-09-11T22:48:39.553 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T22:48:39.812 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:48:40.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:39 smithi152 ceph-mon[31181]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:40.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:39 smithi191 ceph-mon[35211]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:40.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:39 smithi204 ceph-mon[35454]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:41.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 22:48:41 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T22:48:41.016+0000 7eff994ad640 -1 osd.8 0 log_to_monitors true 2024-09-11T22:48:42.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:41 smithi204 ceph-mon[35454]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:42.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:41 smithi204 ceph-mon[35454]: from='osd.8 [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:48:42.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:41 smithi204 ceph-mon[35454]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:48:42.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:41 smithi152 ceph-mon[31181]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:42.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:41 smithi152 ceph-mon[31181]: from='osd.8 [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:48:42.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:41 smithi152 ceph-mon[31181]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:48:42.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:41 smithi191 ceph-mon[35211]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:42.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:41 smithi191 ceph-mon[35211]: from='osd.8 [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:48:42.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:41 smithi191 ceph-mon[35211]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:48:42.736 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:48:42.762 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi152:vg_nvme/lv_1 2024-09-11T22:48:43.015 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:42 smithi152 ceph-mon[31181]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T22:48:43.015 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:42 smithi152 ceph-mon[31181]: osdmap e54: 9 total, 8 up, 9 in 2024-09-11T22:48:43.016 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:48:43.016 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:42 smithi152 ceph-mon[31181]: from='osd.8 [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:48:43.016 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:42 smithi152 ceph-mon[31181]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:48:43.016 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:43.016 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:43.016 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:43.016 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:43.016 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:43.016 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:43.016 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:43.016 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:48:43.016 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:42 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:43.018 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:48:43.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:42 smithi191 ceph-mon[35211]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T22:48:43.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:42 smithi191 ceph-mon[35211]: osdmap e54: 9 total, 8 up, 9 in 2024-09-11T22:48:43.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:48:43.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:42 smithi191 ceph-mon[35211]: from='osd.8 [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:48:43.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:42 smithi191 ceph-mon[35211]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:48:43.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:43.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:43.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:43.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:43.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:43.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:43.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:43.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:48:43.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:42 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:43.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 22:48:42 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T22:48:42.883+0000 7eff95430640 -1 osd.8 0 waiting for initial osdmap 2024-09-11T22:48:43.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 22:48:42 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T22:48:42.893+0000 7eff91258640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:48:43.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:42 smithi204 ceph-mon[35454]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T22:48:43.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:42 smithi204 ceph-mon[35454]: osdmap e54: 9 total, 8 up, 9 in 2024-09-11T22:48:43.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:48:43.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:42 smithi204 ceph-mon[35454]: from='osd.8 [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:48:43.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:42 smithi204 ceph-mon[35454]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:48:43.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:43.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:43.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:43.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:43.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:48:43.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:43.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:43.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:48:43.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:42 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:48:43.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.481+0000 7f8e12ff1640 1 -- 172.21.15.152:0/235659858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e0c1038b0 msgr2=0x7f8e0c10a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:48:43.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.481+0000 7f8e12ff1640 1 --2- 172.21.15.152:0/235659858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e0c1038b0 0x7f8e0c10a140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009cd0 tx=0x7f8e0402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:48:43.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.481+0000 7f8e12ff1640 1 -- 172.21.15.152:0/235659858 shutdown_connections 2024-09-11T22:48:43.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.481+0000 7f8e12ff1640 1 --2- 172.21.15.152:0/235659858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e0c1038b0 0x7f8e0c10a140 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:43.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.481+0000 7f8e12ff1640 1 --2- 172.21.15.152:0/235659858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e0c102ef0 0x7f8e0c103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:43.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.481+0000 7f8e12ff1640 1 --2- 172.21.15.152:0/235659858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e0c0fff10 0x7f8e0c100310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:43.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.481+0000 7f8e12ff1640 1 -- 172.21.15.152:0/235659858 >> 172.21.15.152:0/235659858 conn(0x7f8e0c0fba80 msgr2=0x7f8e0c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:48:43.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.481+0000 7f8e12ff1640 1 -- 172.21.15.152:0/235659858 shutdown_connections 2024-09-11T22:48:43.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.481+0000 7f8e12ff1640 1 -- 172.21.15.152:0/235659858 wait complete. 2024-09-11T22:48:43.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.481+0000 7f8e12ff1640 1 Processor -- start 2024-09-11T22:48:43.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.481+0000 7f8e12ff1640 1 -- start start 2024-09-11T22:48:43.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.481+0000 7f8e12ff1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e0c0fff10 0x7f8e0c102850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:48:43.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e12ff1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e0c102ef0 0x7f8e0c100ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:48:43.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e12ff1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e0c1038b0 0x7f8e0c1013e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:48:43.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e12ff1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c10d860 con 0x7f8e0c1038b0 2024-09-11T22:48:43.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e12ff1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c10d6e0 con 0x7f8e0c0fff10 2024-09-11T22:48:43.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e12ff1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c10d9e0 con 0x7f8e0c102ef0 2024-09-11T22:48:43.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e1158e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e0c1038b0 0x7f8e0c1013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:48:43.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e1158e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e0c1038b0 0x7f8e0c1013e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51440/0 (socket says 172.21.15.152:51440) 2024-09-11T22:48:43.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e1158e640 1 -- 172.21.15.152:0/808468666 learned_addr learned my addr 172.21.15.152:0/808468666 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:48:43.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e10d8d640 1 --2- 172.21.15.152:0/808468666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e0c0fff10 0x7f8e0c102850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:48:43.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e0bfff640 1 --2- 172.21.15.152:0/808468666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e0c102ef0 0x7f8e0c100ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:48:43.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e1158e640 1 -- 172.21.15.152:0/808468666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e0c102ef0 msgr2=0x7f8e0c100ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:48:43.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e1158e640 1 --2- 172.21.15.152:0/808468666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e0c102ef0 0x7f8e0c100ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:43.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e1158e640 1 -- 172.21.15.152:0/808468666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e0c0fff10 msgr2=0x7f8e0c102850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:48:43.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e1158e640 1 --2- 172.21.15.152:0/808468666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e0c0fff10 0x7f8e0c102850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:48:43.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e1158e640 1 -- 172.21.15.152:0/808468666 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c101ca0 con 0x7f8e0c1038b0 2024-09-11T22:48:43.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e1158e640 1 --2- 172.21.15.152:0/808468666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e0c1038b0 0x7f8e0c1013e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8e040317f0 tx=0x7f8e04003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:48:43.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e09ffb640 1 -- 172.21.15.152:0/808468666 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04004450 con 0x7f8e0c1038b0 2024-09-11T22:48:43.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e09ffb640 1 -- 172.21.15.152:0/808468666 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e040045f0 con 0x7f8e0c1038b0 2024-09-11T22:48:43.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e12ff1640 1 -- 172.21.15.152:0/808468666 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0c1a1f50 con 0x7f8e0c1038b0 2024-09-11T22:48:43.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.482+0000 7f8e09ffb640 1 -- 172.21.15.152:0/808468666 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e040467b0 con 0x7f8e0c1038b0 2024-09-11T22:48:43.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.483+0000 7f8e12ff1640 1 -- 172.21.15.152:0/808468666 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0c1a2400 con 0x7f8e0c1038b0 2024-09-11T22:48:43.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.484+0000 7f8e12ff1640 1 -- 172.21.15.152:0/808468666 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e0c109040 con 0x7f8e0c1038b0 2024-09-11T22:48:43.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.485+0000 7f8e09ffb640 1 -- 172.21.15.152:0/808468666 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8e04046950 con 0x7f8e0c1038b0 2024-09-11T22:48:43.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.485+0000 7f8e09ffb640 1 --2- 172.21.15.152:0/808468666 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8dfc0a2b80 0x7f8dfc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:48:43.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.486+0000 7f8e10d8d640 1 --2- 172.21.15.152:0/808468666 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8dfc0a2b80 0x7f8dfc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:48:43.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.486+0000 7f8e09ffb640 1 -- 172.21.15.152:0/808468666 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5549+0+0 (secure 0 0 0) 0x7f8e0402e030 con 0x7f8e0c1038b0 2024-09-11T22:48:43.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.486+0000 7f8e10d8d640 1 --2- 172.21.15.152:0/808468666 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8dfc0a2b80 0x7f8dfc0a5040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8e00005db0 tx=0x7f8e00005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:48:43.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.490+0000 7f8e09ffb640 1 -- 172.21.15.152:0/808468666 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e0404b060 con 0x7f8e0c1038b0 2024-09-11T22:48:43.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:48:43.673+0000 7f8e12ff1640 1 -- 172.21.15.152:0/808468666 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8e0c062630 con 0x7f8dfc0a2b80 2024-09-11T22:48:44.090 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:43 smithi152 ceph-mon[31181]: Detected new or changed devices on smithi204 2024-09-11T22:48:44.090 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:43 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi204 to 5696M 2024-09-11T22:48:44.090 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:43 smithi152 ceph-mon[31181]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:44.091 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:43 smithi152 ceph-mon[31181]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-11T22:48:44.091 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:43 smithi152 ceph-mon[31181]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T22:48:44.091 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:48:44.091 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:48:44.091 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:48:44.091 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:48:44.091 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:44.091 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:43 smithi152 ceph-mon[31181]: osd.8 [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090] boot 2024-09-11T22:48:44.091 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:43 smithi152 ceph-mon[31181]: osdmap e56: 9 total, 9 up, 9 in 2024-09-11T22:48:44.091 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:48:44.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:43 smithi191 ceph-mon[35211]: Detected new or changed devices on smithi204 2024-09-11T22:48:44.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:43 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi204 to 5696M 2024-09-11T22:48:44.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:43 smithi191 ceph-mon[35211]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:44.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:43 smithi191 ceph-mon[35211]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-11T22:48:44.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:43 smithi191 ceph-mon[35211]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T22:48:44.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:48:44.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:48:44.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:48:44.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:48:44.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:44.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:43 smithi191 ceph-mon[35211]: osd.8 [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090] boot 2024-09-11T22:48:44.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:43 smithi191 ceph-mon[35211]: osdmap e56: 9 total, 9 up, 9 in 2024-09-11T22:48:44.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:48:44.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:43 smithi204 ceph-mon[35454]: Detected new or changed devices on smithi204 2024-09-11T22:48:44.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:43 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi204 to 5696M 2024-09-11T22:48:44.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:43 smithi204 ceph-mon[35454]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:48:44.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:43 smithi204 ceph-mon[35454]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-11T22:48:44.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:43 smithi204 ceph-mon[35454]: osdmap e55: 9 total, 8 up, 9 in 2024-09-11T22:48:44.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:48:44.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:48:44.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:48:44.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:48:44.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:44.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:43 smithi204 ceph-mon[35454]: osd.8 [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090] boot 2024-09-11T22:48:44.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:43 smithi204 ceph-mon[35454]: osdmap e56: 9 total, 9 up, 9 in 2024-09-11T22:48:44.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:48:45.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:44 smithi152 ceph-mon[31181]: purged_snaps scrub starts 2024-09-11T22:48:45.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:44 smithi152 ceph-mon[31181]: purged_snaps scrub ok 2024-09-11T22:48:45.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:44 smithi152 ceph-mon[31181]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:48:45.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:44 smithi152 ceph-mon[31181]: osdmap e57: 9 total, 9 up, 9 in 2024-09-11T22:48:45.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:44 smithi191 ceph-mon[35211]: purged_snaps scrub starts 2024-09-11T22:48:45.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:44 smithi191 ceph-mon[35211]: purged_snaps scrub ok 2024-09-11T22:48:45.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:44 smithi191 ceph-mon[35211]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:48:45.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:44 smithi191 ceph-mon[35211]: osdmap e57: 9 total, 9 up, 9 in 2024-09-11T22:48:45.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:44 smithi204 ceph-mon[35454]: purged_snaps scrub starts 2024-09-11T22:48:45.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:44 smithi204 ceph-mon[35454]: purged_snaps scrub ok 2024-09-11T22:48:45.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:44 smithi204 ceph-mon[35454]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:48:45.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:44 smithi204 ceph-mon[35454]: osdmap e57: 9 total, 9 up, 9 in 2024-09-11T22:48:46.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:45 smithi204 ceph-mon[35454]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:48:46.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:45 smithi204 ceph-mon[35454]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T22:48:46.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:45 smithi191 ceph-mon[35211]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:48:46.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:45 smithi191 ceph-mon[35211]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T22:48:46.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:45 smithi152 ceph-mon[31181]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:48:46.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:45 smithi152 ceph-mon[31181]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T22:48:47.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:46 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4187490496' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "751a6964-798f-4240-a2ba-7fa9f0c72657"}]: dispatch 2024-09-11T22:48:47.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:46 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4187490496' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "751a6964-798f-4240-a2ba-7fa9f0c72657"}]': finished 2024-09-11T22:48:47.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:46 smithi152 ceph-mon[31181]: osdmap e59: 10 total, 9 up, 10 in 2024-09-11T22:48:47.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:46 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:48:47.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:46 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4289697446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:48:47.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:46 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/4187490496' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "751a6964-798f-4240-a2ba-7fa9f0c72657"}]: dispatch 2024-09-11T22:48:47.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:46 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/4187490496' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "751a6964-798f-4240-a2ba-7fa9f0c72657"}]': finished 2024-09-11T22:48:47.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:46 smithi191 ceph-mon[35211]: osdmap e59: 10 total, 9 up, 10 in 2024-09-11T22:48:47.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:46 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:48:47.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:46 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/4289697446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:48:47.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:46 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/4187490496' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "751a6964-798f-4240-a2ba-7fa9f0c72657"}]: dispatch 2024-09-11T22:48:47.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:46 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/4187490496' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "751a6964-798f-4240-a2ba-7fa9f0c72657"}]': finished 2024-09-11T22:48:47.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:46 smithi204 ceph-mon[35454]: osdmap e59: 10 total, 9 up, 10 in 2024-09-11T22:48:47.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:46 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:48:47.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:46 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/4289697446' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:48:48.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:47 smithi191 ceph-mon[35211]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:48:48.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:47 smithi191 ceph-mon[35211]: osdmap e60: 10 total, 9 up, 10 in 2024-09-11T22:48:48.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:48:48.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:47 smithi204 ceph-mon[35454]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:48:48.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:47 smithi204 ceph-mon[35454]: osdmap e60: 10 total, 9 up, 10 in 2024-09-11T22:48:48.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:48:48.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:47 smithi152 ceph-mon[31181]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:48:48.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:47 smithi152 ceph-mon[31181]: osdmap e60: 10 total, 9 up, 10 in 2024-09-11T22:48:48.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:48:50.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:49 smithi191 ceph-mon[35211]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:48:50.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:49 smithi204 ceph-mon[35454]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:48:50.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:49 smithi152 ceph-mon[31181]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:48:52.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:51 smithi191 ceph-mon[35211]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-11T22:48:52.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:51 smithi204 ceph-mon[35454]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-11T22:48:52.368 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:51 smithi152 ceph-mon[31181]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-11T22:48:54.066 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:53 smithi152 ceph-mon[31181]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-11T22:48:54.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:53 smithi191 ceph-mon[35211]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-11T22:48:54.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:53 smithi204 ceph-mon[35454]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-11T22:48:55.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T22:48:55.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:55.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:54 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T22:48:55.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:54 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:55.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:54 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T22:48:55.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:54 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:48:56.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:55 smithi191 ceph-mon[35211]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-11T22:48:56.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:55 smithi191 ceph-mon[35211]: Deploying daemon osd.9 on smithi152 2024-09-11T22:48:56.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:55 smithi204 ceph-mon[35454]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-11T22:48:56.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:55 smithi204 ceph-mon[35454]: Deploying daemon osd.9 on smithi152 2024-09-11T22:48:56.369 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:55 smithi152 ceph-mon[31181]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-11T22:48:56.369 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:55 smithi152 ceph-mon[31181]: Deploying daemon osd.9 on smithi152 2024-09-11T22:48:58.201 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:57 smithi152 ceph-mon[31181]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T22:48:58.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:57 smithi191 ceph-mon[35211]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T22:48:58.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:57 smithi204 ceph-mon[35454]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T22:49:00.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:48:59 smithi152 ceph-mon[31181]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T22:49:00.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:48:59 smithi191 ceph-mon[35211]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T22:49:00.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:48:59 smithi204 ceph-mon[35454]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T22:49:01.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:00 smithi191 ceph-mon[35211]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:01.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:00 smithi204 ceph-mon[35454]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:01.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:00 smithi152 ceph-mon[31181]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:02.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:01 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:49:02.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:01 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:02.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:01 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:02.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:01 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:49:02.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:01 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:02.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:01 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:02.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:01 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:49:02.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:01 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:02.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:01 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:03.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:02 smithi191 ceph-mon[35211]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:03.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:02 smithi204 ceph-mon[35454]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:03.334 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:02 smithi152 ceph-mon[31181]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:05.066 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 9 on host 'smithi152' 2024-09-11T22:49:05.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:05.065+0000 7f8e09ffb640 1 -- 172.21.15.152:0/808468666 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8e0c062630 con 0x7f8dfc0a2b80 2024-09-11T22:49:05.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:05.068+0000 7f8e12ff1640 1 -- 172.21.15.152:0/808468666 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8dfc0a2b80 msgr2=0x7f8dfc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:05.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:05.068+0000 7f8e12ff1640 1 --2- 172.21.15.152:0/808468666 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8dfc0a2b80 0x7f8dfc0a5040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8e00005db0 tx=0x7f8e00005d40 comp rx=0 tx=0).stop 2024-09-11T22:49:05.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:05.068+0000 7f8e12ff1640 1 -- 172.21.15.152:0/808468666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e0c1038b0 msgr2=0x7f8e0c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:05.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:05.068+0000 7f8e12ff1640 1 --2- 172.21.15.152:0/808468666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e0c1038b0 0x7f8e0c1013e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8e040317f0 tx=0x7f8e04003ce0 comp rx=0 tx=0).stop 2024-09-11T22:49:05.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:05.068+0000 7f8e12ff1640 1 -- 172.21.15.152:0/808468666 shutdown_connections 2024-09-11T22:49:05.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:05.068+0000 7f8e12ff1640 1 --2- 172.21.15.152:0/808468666 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8dfc0a2b80 0x7f8dfc0a5040 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:05.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:05.068+0000 7f8e12ff1640 1 --2- 172.21.15.152:0/808468666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e0c1038b0 0x7f8e0c1013e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:05.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:05.068+0000 7f8e12ff1640 1 --2- 172.21.15.152:0/808468666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e0c102ef0 0x7f8e0c100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:05.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:05.068+0000 7f8e12ff1640 1 --2- 172.21.15.152:0/808468666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e0c0fff10 0x7f8e0c102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:05.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:05.068+0000 7f8e12ff1640 1 -- 172.21.15.152:0/808468666 >> 172.21.15.152:0/808468666 conn(0x7f8e0c0fba80 msgr2=0x7f8e0c106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:49:05.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:05.069+0000 7f8e12ff1640 1 -- 172.21.15.152:0/808468666 shutdown_connections 2024-09-11T22:49:05.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:05.069+0000 7f8e12ff1640 1 -- 172.21.15.152:0/808468666 wait complete. 2024-09-11T22:49:05.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:05 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:05.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:05 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:05.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:05 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:05.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:05 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:49:05.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:05 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:05.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:05 smithi152 ceph-mon[31181]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:05.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:05 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:49:05.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:05 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:05.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:05 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:05.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:05 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:05.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:05 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:05 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:05 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:49:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:05 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:05 smithi191 ceph-mon[35211]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:05 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:49:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:05 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:05 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:05.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:05 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:05.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:05 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:05.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:05 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:05.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:05 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:49:05.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:05 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:05.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:05 smithi204 ceph-mon[35454]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:05.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:05 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:49:05.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:05 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:05.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:05 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:05.801 DEBUG:teuthology.orchestra.run.smithi152:osd.9> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.9.service 2024-09-11T22:49:05.804 INFO:tasks.cephadm:Deploying osd.10 on smithi191 with /dev/vg_nvme/lv_1... 2024-09-11T22:49:05.804 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T22:49:06.059 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:49:07.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 22:49:07 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T22:49:07.265+0000 7f4907941640 -1 osd.9 0 log_to_monitors true 2024-09-11T22:49:08.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:07 smithi191 ceph-mon[35211]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:08.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:07 smithi191 ceph-mon[35211]: from='osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T22:49:08.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:07 smithi204 ceph-mon[35454]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:08.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:07 smithi204 ceph-mon[35454]: from='osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T22:49:08.072 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:07 smithi152 ceph-mon[31181]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:08.072 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:07 smithi152 ceph-mon[31181]: from='osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T22:49:08.828 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-09-11T22:49:08.855 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi191:vg_nvme/lv_1 2024-09-11T22:49:09.120 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:49:09.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:08 smithi152 ceph-mon[31181]: from='osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T22:49:09.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:08 smithi152 ceph-mon[31181]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T22:49:09.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:08 smithi152 ceph-mon[31181]: from='osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-09-11T22:49:09.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:08 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:49:09.240 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:08 smithi191 ceph-mon[35211]: from='osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T22:49:09.240 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:08 smithi191 ceph-mon[35211]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T22:49:09.240 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:08 smithi191 ceph-mon[35211]: from='osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-09-11T22:49:09.240 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:08 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:49:09.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:08 smithi204 ceph-mon[35454]: from='osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T22:49:09.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:08 smithi204 ceph-mon[35454]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T22:49:09.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:08 smithi204 ceph-mon[35454]: from='osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-09-11T22:49:09.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:08 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:49:09.622 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T22:49:09.222+0000 7f49038c4640 -1 osd.9 0 waiting for initial osdmap 2024-09-11T22:49:09.623 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T22:49:09.235+0000 7f48fb6e4640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:49:09.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.611+0000 7f0db080f640 1 -- 172.21.15.191:0/3694125074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dac1026d0 msgr2=0x7f0dac108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:09.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.611+0000 7f0db080f640 1 --2- 172.21.15.191:0/3694125074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dac1026d0 0x7f0dac108f60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009ca0 tx=0x7f0da402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:49:09.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.611+0000 7f0db080f640 1 -- 172.21.15.191:0/3694125074 shutdown_connections 2024-09-11T22:49:09.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.611+0000 7f0db080f640 1 --2- 172.21.15.191:0/3694125074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dac1026d0 0x7f0dac108f60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:09.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.611+0000 7f0db080f640 1 --2- 172.21.15.191:0/3694125074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dac101d10 0x7f0dac102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:09.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.611+0000 7f0db080f640 1 --2- 172.21.15.191:0/3694125074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dac100b10 0x7f0dac100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:09.637 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.611+0000 7f0db080f640 1 -- 172.21.15.191:0/3694125074 >> 172.21.15.191:0/3694125074 conn(0x7f0dac0fc2a0 msgr2=0x7f0dac0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:49:09.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.611+0000 7f0db080f640 1 -- 172.21.15.191:0/3694125074 shutdown_connections 2024-09-11T22:49:09.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.611+0000 7f0db080f640 1 -- 172.21.15.191:0/3694125074 wait complete. 2024-09-11T22:49:09.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.612+0000 7f0db080f640 1 Processor -- start 2024-09-11T22:49:09.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.612+0000 7f0db080f640 1 -- start start 2024-09-11T22:49:09.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.612+0000 7f0db080f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dac100b10 0x7f0dac197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:49:09.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.612+0000 7f0db080f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dac101d10 0x7f0dac198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:49:09.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.612+0000 7f0db080f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dac1026d0 0x7f0dac19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:49:09.638 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.612+0000 7f0db080f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac10d8c0 con 0x7f0dac1026d0 2024-09-11T22:49:09.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.612+0000 7f0db080f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac10d740 con 0x7f0dac101d10 2024-09-11T22:49:09.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.612+0000 7f0db080f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac10da40 con 0x7f0dac100b10 2024-09-11T22:49:09.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.612+0000 7f0da9d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dac101d10 0x7f0dac198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:49:09.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.612+0000 7f0da9d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dac101d10 0x7f0dac198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:44160/0 (socket says 172.21.15.191:44160) 2024-09-11T22:49:09.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.612+0000 7f0da9d9b640 1 -- 172.21.15.191:0/2603754648 learned_addr learned my addr 172.21.15.191:0/2603754648 (peer_addr_for_me v2:172.21.15.191:0/0) 2024-09-11T22:49:09.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.612+0000 7f0daad9d640 1 --2- 172.21.15.191:0/2603754648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dac1026d0 0x7f0dac19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:49:09.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.612+0000 7f0daa59c640 1 --2- 172.21.15.191:0/2603754648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dac100b10 0x7f0dac197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:49:09.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.613+0000 7f0da9d9b640 1 -- 172.21.15.191:0/2603754648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dac100b10 msgr2=0x7f0dac197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:09.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.613+0000 7f0da9d9b640 1 --2- 172.21.15.191:0/2603754648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dac100b10 0x7f0dac197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:09.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.613+0000 7f0da9d9b640 1 -- 172.21.15.191:0/2603754648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dac1026d0 msgr2=0x7f0dac19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:09.639 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.613+0000 7f0da9d9b640 1 --2- 172.21.15.191:0/2603754648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dac1026d0 0x7f0dac19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:09.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.613+0000 7f0da9d9b640 1 -- 172.21.15.191:0/2603754648 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dac19faf0 con 0x7f0dac101d10 2024-09-11T22:49:09.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.613+0000 7f0da9d9b640 1 --2- 172.21.15.191:0/2603754648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dac101d10 0x7f0dac198400 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400d950 tx=0x7f0d9400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:49:09.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.613+0000 7f0da37fe640 1 -- 172.21.15.191:0/2603754648 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94017070 con 0x7f0dac101d10 2024-09-11T22:49:09.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.613+0000 7f0db080f640 1 -- 172.21.15.191:0/2603754648 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dac19fde0 con 0x7f0dac101d10 2024-09-11T22:49:09.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.613+0000 7f0da37fe640 1 -- 172.21.15.191:0/2603754648 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d94009c30 con 0x7f0dac101d10 2024-09-11T22:49:09.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.613+0000 7f0da37fe640 1 -- 172.21.15.191:0/2603754648 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9400f6b0 con 0x7f0dac101d10 2024-09-11T22:49:09.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.613+0000 7f0db080f640 1 -- 172.21.15.191:0/2603754648 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dac1a0290 con 0x7f0dac101d10 2024-09-11T22:49:09.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.614+0000 7f0db080f640 1 -- 172.21.15.191:0/2603754648 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dac107ee0 con 0x7f0dac101d10 2024-09-11T22:49:09.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.615+0000 7f0da37fe640 1 -- 172.21.15.191:0/2603754648 <== mon.2 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0d940109d0 con 0x7f0dac101d10 2024-09-11T22:49:09.640 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.616+0000 7f0da37fe640 1 --2- 172.21.15.191:0/2603754648 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0d980a2bb0 0x7f0d980a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:49:09.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.617+0000 7f0da37fe640 1 -- 172.21.15.191:0/2603754648 <== mon.2 v2:172.21.15.191:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5981+0+0 (secure 0 0 0) 0x7f0d940ce290 con 0x7f0dac101d10 2024-09-11T22:49:09.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.617+0000 7f0daa59c640 1 --2- 172.21.15.191:0/2603754648 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0d980a2bb0 0x7f0d980a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:49:09.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.617+0000 7f0daa59c640 1 --2- 172.21.15.191:0/2603754648 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0d980a2bb0 0x7f0d980a5070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0dac101b70 tx=0x7f0d9c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:49:09.641 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.621+0000 7f0da37fe640 1 -- 172.21.15.191:0/2603754648 <== mon.2 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d94009de0 con 0x7f0dac101d10 2024-09-11T22:49:09.797 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:09.796+0000 7f0db080f640 1 -- 172.21.15.191:0/2603754648 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f0dac062630 con 0x7f0d980a2bb0 2024-09-11T22:49:10.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:10.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-09-11T22:49:10.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T22:49:10.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:49:10.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:49:10.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: Detected new or changed devices on smithi152 2024-09-11T22:49:10.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:10.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:10.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:10.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:10.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:10.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:10.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi152 to 4272M 2024-09-11T22:49:10.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:10.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:10.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:49:10.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:10.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419]' entity='osd.9' 2024-09-11T22:49:10.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:49:10.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:49:10.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:10.203 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:10.203 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-09-11T22:49:10.203 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T22:49:10.203 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:49:10.204 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:49:10.204 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: Detected new or changed devices on smithi152 2024-09-11T22:49:10.204 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:10.204 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:10.204 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:10.204 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:10.204 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:10.204 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:10.204 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi152 to 4272M 2024-09-11T22:49:10.205 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:10.205 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:10.205 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:49:10.205 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:10.205 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419]' entity='osd.9' 2024-09-11T22:49:10.205 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:49:10.205 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:49:10.205 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:10.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:10.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-09-11T22:49:10.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: osdmap e62: 10 total, 9 up, 10 in 2024-09-11T22:49:10.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:49:10.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:49:10.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: Detected new or changed devices on smithi152 2024-09-11T22:49:10.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:10.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:10.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:10.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:10.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:10.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:10.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi152 to 4272M 2024-09-11T22:49:10.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:10.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:10.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:49:10.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:10.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419]' entity='osd.9' 2024-09-11T22:49:10.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:49:10.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:49:10.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:11.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:10 smithi152 ceph-mon[31181]: purged_snaps scrub starts 2024-09-11T22:49:11.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:10 smithi152 ceph-mon[31181]: purged_snaps scrub ok 2024-09-11T22:49:11.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:10 smithi152 ceph-mon[31181]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:49:11.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:10 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:49:11.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:10 smithi152 ceph-mon[31181]: osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419] boot 2024-09-11T22:49:11.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:10 smithi152 ceph-mon[31181]: osdmap e63: 10 total, 10 up, 10 in 2024-09-11T22:49:11.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:10 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:49:11.212 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:10 smithi191 ceph-mon[35211]: purged_snaps scrub starts 2024-09-11T22:49:11.212 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:10 smithi191 ceph-mon[35211]: purged_snaps scrub ok 2024-09-11T22:49:11.212 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:10 smithi191 ceph-mon[35211]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:49:11.212 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:10 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:49:11.212 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:10 smithi191 ceph-mon[35211]: osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419] boot 2024-09-11T22:49:11.212 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:10 smithi191 ceph-mon[35211]: osdmap e63: 10 total, 10 up, 10 in 2024-09-11T22:49:11.213 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:10 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:49:11.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:10 smithi204 ceph-mon[35454]: purged_snaps scrub starts 2024-09-11T22:49:11.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:10 smithi204 ceph-mon[35454]: purged_snaps scrub ok 2024-09-11T22:49:11.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:10 smithi204 ceph-mon[35454]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:49:11.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:10 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:49:11.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:10 smithi204 ceph-mon[35454]: osd.9 [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419] boot 2024-09-11T22:49:11.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:10 smithi204 ceph-mon[35454]: osdmap e63: 10 total, 10 up, 10 in 2024-09-11T22:49:11.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:10 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:49:12.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:11 smithi152 ceph-mon[31181]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:12.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:11 smithi152 ceph-mon[31181]: osdmap e64: 10 total, 10 up, 10 in 2024-09-11T22:49:12.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:11 smithi191 ceph-mon[35211]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:12.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:11 smithi191 ceph-mon[35211]: osdmap e64: 10 total, 10 up, 10 in 2024-09-11T22:49:12.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:11 smithi204 ceph-mon[35454]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:49:12.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:11 smithi204 ceph-mon[35454]: osdmap e64: 10 total, 10 up, 10 in 2024-09-11T22:49:13.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:12 smithi152 ceph-mon[31181]: from='client.? 172.21.15.191:0/2026981518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90e7c6ba-127c-4b33-b3c6-cedf72128451"}]: dispatch 2024-09-11T22:49:13.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:12 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90e7c6ba-127c-4b33-b3c6-cedf72128451"}]: dispatch 2024-09-11T22:49:13.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:12 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90e7c6ba-127c-4b33-b3c6-cedf72128451"}]': finished 2024-09-11T22:49:13.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:12 smithi152 ceph-mon[31181]: osdmap e65: 11 total, 10 up, 11 in 2024-09-11T22:49:13.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:12 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:49:13.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:12 smithi152 ceph-mon[31181]: from='client.? 172.21.15.191:0/1916909053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:49:13.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:12 smithi191 ceph-mon[35211]: from='client.? 172.21.15.191:0/2026981518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90e7c6ba-127c-4b33-b3c6-cedf72128451"}]: dispatch 2024-09-11T22:49:13.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:12 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90e7c6ba-127c-4b33-b3c6-cedf72128451"}]: dispatch 2024-09-11T22:49:13.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:12 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90e7c6ba-127c-4b33-b3c6-cedf72128451"}]': finished 2024-09-11T22:49:13.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:12 smithi191 ceph-mon[35211]: osdmap e65: 11 total, 10 up, 11 in 2024-09-11T22:49:13.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:12 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:49:13.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:12 smithi191 ceph-mon[35211]: from='client.? 172.21.15.191:0/1916909053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:49:13.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:12 smithi204 ceph-mon[35454]: from='client.? 172.21.15.191:0/2026981518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90e7c6ba-127c-4b33-b3c6-cedf72128451"}]: dispatch 2024-09-11T22:49:13.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:12 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90e7c6ba-127c-4b33-b3c6-cedf72128451"}]: dispatch 2024-09-11T22:49:13.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:12 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90e7c6ba-127c-4b33-b3c6-cedf72128451"}]': finished 2024-09-11T22:49:13.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:12 smithi204 ceph-mon[35454]: osdmap e65: 11 total, 10 up, 11 in 2024-09-11T22:49:13.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:12 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:49:13.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:12 smithi204 ceph-mon[35454]: from='client.? 172.21.15.191:0/1916909053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:49:14.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:13 smithi152 ceph-mon[31181]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:14.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:13 smithi191 ceph-mon[35211]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:14.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:13 smithi204 ceph-mon[35454]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:16.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:15 smithi152 ceph-mon[31181]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:16.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:15 smithi191 ceph-mon[35211]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:16.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:15 smithi204 ceph-mon[35454]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:18.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:17 smithi191 ceph-mon[35211]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:18.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:17 smithi152 ceph-mon[31181]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:18.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:17 smithi204 ceph-mon[35454]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:19.924 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:19 smithi191 ceph-mon[35211]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:20.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:19 smithi152 ceph-mon[31181]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:20.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:19 smithi204 ceph-mon[35454]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:22.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:21 smithi191 ceph-mon[35211]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:22.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:21 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T22:49:22.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:21 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:22.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:21 smithi152 ceph-mon[31181]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:22.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T22:49:22.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:22.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:21 smithi204 ceph-mon[35454]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:22.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:21 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T22:49:22.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:21 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:23.019 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:22 smithi191 ceph-mon[35211]: Deploying daemon osd.10 on smithi191 2024-09-11T22:49:23.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:22 smithi152 ceph-mon[31181]: Deploying daemon osd.10 on smithi191 2024-09-11T22:49:23.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:22 smithi204 ceph-mon[35454]: Deploying daemon osd.10 on smithi191 2024-09-11T22:49:24.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:23 smithi152 ceph-mon[31181]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:24.167 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:23 smithi191 ceph-mon[35211]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:24.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:23 smithi204 ceph-mon[35454]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:26.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:25 smithi191 ceph-mon[35211]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:26.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:25 smithi152 ceph-mon[31181]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:26.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:25 smithi204 ceph-mon[35454]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:28.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:27 smithi152 ceph-mon[31181]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:28.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:27 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:49:28.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:27 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:28.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:27 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:28.281 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:27 smithi191 ceph-mon[35211]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:28.281 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:27 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:49:28.281 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:27 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:28.281 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:27 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:28.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:27 smithi204 ceph-mon[35454]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:28.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:27 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:49:28.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:27 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:28.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:27 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:30.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:29 smithi191 ceph-mon[35211]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:30.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:29 smithi152 ceph-mon[31181]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:30.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:29 smithi204 ceph-mon[35454]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:30.830 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 10 on host 'smithi191' 2024-09-11T22:49:30.830 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:30.829+0000 7f0da37fe640 1 -- 172.21.15.191:0/2603754648 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f0dac062630 con 0x7f0d980a2bb0 2024-09-11T22:49:30.832 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:30.831+0000 7f0db080f640 1 -- 172.21.15.191:0/2603754648 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0d980a2bb0 msgr2=0x7f0d980a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:30.832 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:30.831+0000 7f0db080f640 1 --2- 172.21.15.191:0/2603754648 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0d980a2bb0 0x7f0d980a5070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0dac101b70 tx=0x7f0d9c005f90 comp rx=0 tx=0).stop 2024-09-11T22:49:30.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:30.831+0000 7f0db080f640 1 -- 172.21.15.191:0/2603754648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dac101d10 msgr2=0x7f0dac198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:30.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:30.831+0000 7f0db080f640 1 --2- 172.21.15.191:0/2603754648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dac101d10 0x7f0dac198400 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400d950 tx=0x7f0d9400de10 comp rx=0 tx=0).stop 2024-09-11T22:49:30.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:30.831+0000 7f0db080f640 1 -- 172.21.15.191:0/2603754648 shutdown_connections 2024-09-11T22:49:30.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:30.832+0000 7f0db080f640 1 --2- 172.21.15.191:0/2603754648 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0d980a2bb0 0x7f0d980a5070 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:30.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:30.832+0000 7f0db080f640 1 --2- 172.21.15.191:0/2603754648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0dac1026d0 0x7f0dac19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:30.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:30.832+0000 7f0db080f640 1 --2- 172.21.15.191:0/2603754648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0dac101d10 0x7f0dac198400 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:30.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:30.832+0000 7f0db080f640 1 --2- 172.21.15.191:0/2603754648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dac100b10 0x7f0dac197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:30.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:30.832+0000 7f0db080f640 1 -- 172.21.15.191:0/2603754648 >> 172.21.15.191:0/2603754648 conn(0x7f0dac0fc2a0 msgr2=0x7f0dac1050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:49:30.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:30.832+0000 7f0db080f640 1 -- 172.21.15.191:0/2603754648 shutdown_connections 2024-09-11T22:49:30.833 INFO:teuthology.orchestra.run.smithi191.stderr:2024-09-11T22:49:30.832+0000 7f0db080f640 1 -- 172.21.15.191:0/2603754648 wait complete. 2024-09-11T22:49:31.440 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:31 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:31.440 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:31 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:31.440 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:31 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:31.441 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:31 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:49:31.441 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:31 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:31.441 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:31 smithi191 ceph-mon[35211]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:31.441 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:31 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:49:31.441 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:31 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:31.441 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:31 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:31.467 DEBUG:teuthology.orchestra.run.smithi191:osd.10> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.10.service 2024-09-11T22:49:31.470 INFO:tasks.cephadm:Deploying osd.11 on smithi204 with /dev/vg_nvme/lv_1... 2024-09-11T22:49:31.470 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T22:49:31.498 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:31 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:31.498 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:31 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:31.498 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:31 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:31.499 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:31 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:49:31.499 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:31 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:31.499 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:31 smithi204 ceph-mon[35454]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:31.499 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:31 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:49:31.499 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:31 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:31.499 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:31 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:31.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:31 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:31.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:31 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:31.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:31 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:31.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:31 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:49:31.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:31 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:31.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:31 smithi152 ceph-mon[31181]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:31.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:31 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:49:31.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:31 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:31.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:31 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:31.728 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:49:33.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 22:49:32 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T22:49:32.877+0000 7fb2cefd9640 -1 osd.10 0 log_to_monitors true 2024-09-11T22:49:34.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:33 smithi191 ceph-mon[35211]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:34.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:33 smithi191 ceph-mon[35211]: from='osd.10 [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:49:34.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:33 smithi191 ceph-mon[35211]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:49:34.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:33 smithi204 ceph-mon[35454]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:34.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:33 smithi204 ceph-mon[35454]: from='osd.10 [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:49:34.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:33 smithi204 ceph-mon[35454]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:49:34.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:33 smithi152 ceph-mon[31181]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:34.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:33 smithi152 ceph-mon[31181]: from='osd.10 [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:49:34.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:33 smithi152 ceph-mon[31181]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:49:34.513 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-09-11T22:49:34.543 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi204:vg_nvme/lv_1 2024-09-11T22:49:34.845 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:49:35.112 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:34 smithi204 ceph-mon[35454]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T22:49:35.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:34 smithi204 ceph-mon[35454]: osdmap e66: 11 total, 10 up, 11 in 2024-09-11T22:49:35.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:49:35.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:34 smithi204 ceph-mon[35454]: from='osd.10 [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:49:35.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:34 smithi204 ceph-mon[35454]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:49:35.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:35.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:35.113 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:35.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:35.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:35.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:35.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:35.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:35.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:49:35.114 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:35.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:34 smithi152 ceph-mon[31181]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T22:49:35.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:34 smithi152 ceph-mon[31181]: osdmap e66: 11 total, 10 up, 11 in 2024-09-11T22:49:35.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:49:35.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:34 smithi152 ceph-mon[31181]: from='osd.10 [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:49:35.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:34 smithi152 ceph-mon[31181]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:49:35.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:35.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:35.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:35.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:35.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:35.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:35.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:35.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:35.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:49:35.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:35.287 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T22:49:34.821+0000 7fb2caf5c640 -1 osd.10 0 waiting for initial osdmap 2024-09-11T22:49:35.287 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T22:49:34.832+0000 7fb2c6d84640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:49:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-mon[35211]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T22:49:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-mon[35211]: osdmap e66: 11 total, 10 up, 11 in 2024-09-11T22:49:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:49:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-mon[35211]: from='osd.10 [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:49:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-mon[35211]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-09-11T22:49:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:49:35.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:35.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:35.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:49:35.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:35.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.331+0000 7f84d8911640 1 -- 172.21.15.204:0/2870766993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d4100ad0 msgr2=0x7f84d4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:35.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.331+0000 7f84d8911640 1 --2- 172.21.15.204:0/2870766993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d4100ad0 0x7f84d4100ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009ca0 tx=0x7f84cc02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:49:35.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.331+0000 7f84d8911640 1 -- 172.21.15.204:0/2870766993 shutdown_connections 2024-09-11T22:49:35.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.331+0000 7f84d8911640 1 --2- 172.21.15.204:0/2870766993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84d4102690 0x7f84d4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:35.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.331+0000 7f84d8911640 1 --2- 172.21.15.204:0/2870766993 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84d4101cd0 0x7f84d4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:35.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.331+0000 7f84d8911640 1 --2- 172.21.15.204:0/2870766993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d4100ad0 0x7f84d4100ed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:35.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.331+0000 7f84d8911640 1 -- 172.21.15.204:0/2870766993 >> 172.21.15.204:0/2870766993 conn(0x7f84d40fc280 msgr2=0x7f84d40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:49:35.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.331+0000 7f84d8911640 1 -- 172.21.15.204:0/2870766993 shutdown_connections 2024-09-11T22:49:35.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.331+0000 7f84d8911640 1 -- 172.21.15.204:0/2870766993 wait complete. 2024-09-11T22:49:35.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.331+0000 7f84d8911640 1 Processor -- start 2024-09-11T22:49:35.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d8911640 1 -- start start 2024-09-11T22:49:35.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d8911640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84d4100ad0 0x7f84d419a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:49:35.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d8911640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84d4101cd0 0x7f84d419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:49:35.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d8911640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d4102690 0x7f84d41a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:49:35.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d8911640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f84d410c750 con 0x7f84d4101cd0 2024-09-11T22:49:35.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d8911640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f84d410c5d0 con 0x7f84d4100ad0 2024-09-11T22:49:35.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d8911640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f84d410c8d0 con 0x7f84d4102690 2024-09-11T22:49:35.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d2d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d4102690 0x7f84d41a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:49:35.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d2d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d4102690 0x7f84d41a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33080/0 (socket says 172.21.15.204:33080) 2024-09-11T22:49:35.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d2d9d640 1 -- 172.21.15.204:0/1548829932 learned_addr learned my addr 172.21.15.204:0/1548829932 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-11T22:49:35.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d1d9b640 1 --2- 172.21.15.204:0/1548829932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84d4101cd0 0x7f84d419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:49:35.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d259c640 1 --2- 172.21.15.204:0/1548829932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84d4100ad0 0x7f84d419a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:49:35.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d2d9d640 1 -- 172.21.15.204:0/1548829932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84d4100ad0 msgr2=0x7f84d419a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:35.340 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d2d9d640 1 --2- 172.21.15.204:0/1548829932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84d4100ad0 0x7f84d419a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:35.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d2d9d640 1 -- 172.21.15.204:0/1548829932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84d4101cd0 msgr2=0x7f84d419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:35.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d2d9d640 1 --2- 172.21.15.204:0/1548829932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84d4101cd0 0x7f84d419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:35.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d2d9d640 1 -- 172.21.15.204:0/1548829932 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d41a1d60 con 0x7f84d4102690 2024-09-11T22:49:35.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d259c640 1 --2- 172.21.15.204:0/1548829932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84d4100ad0 0x7f84d419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:49:35.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d2d9d640 1 --2- 172.21.15.204:0/1548829932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d4102690 0x7f84d41a16f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f84c400e970 tx=0x7f84c400ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:49:35.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84cb7fe640 1 -- 172.21.15.204:0/1548829932 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c400ccc0 con 0x7f84d4102690 2024-09-11T22:49:35.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84cb7fe640 1 -- 172.21.15.204:0/1548829932 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c400ce60 con 0x7f84d4102690 2024-09-11T22:49:35.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.332+0000 7f84d8911640 1 -- 172.21.15.204:0/1548829932 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d41a1ff0 con 0x7f84d4102690 2024-09-11T22:49:35.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.333+0000 7f84cb7fe640 1 -- 172.21.15.204:0/1548829932 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4010490 con 0x7f84d4102690 2024-09-11T22:49:35.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.333+0000 7f84d8911640 1 -- 172.21.15.204:0/1548829932 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d41a24b0 con 0x7f84d4102690 2024-09-11T22:49:35.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.334+0000 7f84d8911640 1 -- 172.21.15.204:0/1548829932 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d4100ed0 con 0x7f84d4102690 2024-09-11T22:49:35.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.336+0000 7f84cb7fe640 1 -- 172.21.15.204:0/1548829932 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f84c4005000 con 0x7f84d4102690 2024-09-11T22:49:35.341 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.336+0000 7f84cb7fe640 1 --2- 172.21.15.204:0/1548829932 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f84c00a2bb0 0x7f84c00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:49:35.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.337+0000 7f84cb7fe640 1 -- 172.21.15.204:0/1548829932 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6727+0+0 (secure 0 0 0) 0x7f84c40cf0e0 con 0x7f84d4102690 2024-09-11T22:49:35.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.337+0000 7f84d259c640 1 --2- 172.21.15.204:0/1548829932 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f84c00a2bb0 0x7f84c00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:49:35.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.337+0000 7f84d259c640 1 --2- 172.21.15.204:0/1548829932 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f84c00a2bb0 0x7f84c00a5070 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f84cc006010 tx=0x7f84cc000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:49:35.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.340+0000 7f84cb7fe640 1 -- 172.21.15.204:0/1548829932 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84c40541b0 con 0x7f84d4102690 2024-09-11T22:49:35.519 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:35.518+0000 7f84d8911640 1 -- 172.21.15.204:0/1548829932 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f84d406a6f0 con 0x7f84c00a2bb0 2024-09-11T22:49:36.073 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:35 smithi204 ceph-mon[35454]: Detected new or changed devices on smithi191 2024-09-11T22:49:36.073 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:35 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi191 to 4272M 2024-09-11T22:49:36.073 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:35 smithi204 ceph-mon[35454]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:36.073 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:35 smithi204 ceph-mon[35454]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-11T22:49:36.073 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:35 smithi204 ceph-mon[35454]: osdmap e67: 11 total, 10 up, 11 in 2024-09-11T22:49:36.073 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:35 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:49:36.074 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:35 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:49:36.074 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:35 smithi204 ceph-mon[35454]: osd.10 [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503] boot 2024-09-11T22:49:36.074 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:35 smithi204 ceph-mon[35454]: osdmap e68: 11 total, 11 up, 11 in 2024-09-11T22:49:36.074 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:35 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:49:36.074 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:35 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:49:36.074 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:35 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:49:36.074 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:35 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:36.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:35 smithi152 ceph-mon[31181]: Detected new or changed devices on smithi191 2024-09-11T22:49:36.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:35 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi191 to 4272M 2024-09-11T22:49:36.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:35 smithi152 ceph-mon[31181]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:36.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:35 smithi152 ceph-mon[31181]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-11T22:49:36.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:35 smithi152 ceph-mon[31181]: osdmap e67: 11 total, 10 up, 11 in 2024-09-11T22:49:36.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:35 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:49:36.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:35 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:49:36.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:35 smithi152 ceph-mon[31181]: osd.10 [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503] boot 2024-09-11T22:49:36.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:35 smithi152 ceph-mon[31181]: osdmap e68: 11 total, 11 up, 11 in 2024-09-11T22:49:36.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:35 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:49:36.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:35 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:49:36.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:35 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:49:36.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:35 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:36.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:35 smithi191 ceph-mon[35211]: Detected new or changed devices on smithi191 2024-09-11T22:49:36.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:35 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi191 to 4272M 2024-09-11T22:49:36.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:35 smithi191 ceph-mon[35211]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:49:36.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:35 smithi191 ceph-mon[35211]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-09-11T22:49:36.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:35 smithi191 ceph-mon[35211]: osdmap e67: 11 total, 10 up, 11 in 2024-09-11T22:49:36.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:35 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:49:36.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:35 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:49:36.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:35 smithi191 ceph-mon[35211]: osd.10 [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503] boot 2024-09-11T22:49:36.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:35 smithi191 ceph-mon[35211]: osdmap e68: 11 total, 11 up, 11 in 2024-09-11T22:49:36.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:35 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:49:36.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:35 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:49:36.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:35 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:49:36.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:35 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:37.071 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:36 smithi204 ceph-mon[35454]: purged_snaps scrub starts 2024-09-11T22:49:37.071 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:36 smithi204 ceph-mon[35454]: purged_snaps scrub ok 2024-09-11T22:49:37.071 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:36 smithi204 ceph-mon[35454]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:49:37.071 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:36 smithi204 ceph-mon[35454]: osdmap e69: 11 total, 11 up, 11 in 2024-09-11T22:49:37.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:36 smithi152 ceph-mon[31181]: purged_snaps scrub starts 2024-09-11T22:49:37.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:36 smithi152 ceph-mon[31181]: purged_snaps scrub ok 2024-09-11T22:49:37.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:36 smithi152 ceph-mon[31181]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:49:37.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:36 smithi152 ceph-mon[31181]: osdmap e69: 11 total, 11 up, 11 in 2024-09-11T22:49:37.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:36 smithi191 ceph-mon[35211]: purged_snaps scrub starts 2024-09-11T22:49:37.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:36 smithi191 ceph-mon[35211]: purged_snaps scrub ok 2024-09-11T22:49:37.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:36 smithi191 ceph-mon[35211]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:49:37.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:36 smithi191 ceph-mon[35211]: osdmap e69: 11 total, 11 up, 11 in 2024-09-11T22:49:38.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:37 smithi204 ceph-mon[35454]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:38.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:37 smithi204 ceph-mon[35454]: osdmap e70: 11 total, 11 up, 11 in 2024-09-11T22:49:38.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:37 smithi152 ceph-mon[31181]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:38.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:37 smithi152 ceph-mon[31181]: osdmap e70: 11 total, 11 up, 11 in 2024-09-11T22:49:38.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:37 smithi191 ceph-mon[35211]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:38.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:37 smithi191 ceph-mon[35211]: osdmap e70: 11 total, 11 up, 11 in 2024-09-11T22:49:39.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:38 smithi204 ceph-mon[35454]: from='client.? 172.21.15.204:0/1424234574' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c76a93b1-a045-477b-98d0-58a2e9d44a7f"}]: dispatch 2024-09-11T22:49:39.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:38 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c76a93b1-a045-477b-98d0-58a2e9d44a7f"}]: dispatch 2024-09-11T22:49:39.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:38 smithi204 ceph-mon[35454]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c76a93b1-a045-477b-98d0-58a2e9d44a7f"}]': finished 2024-09-11T22:49:39.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:38 smithi204 ceph-mon[35454]: osdmap e71: 12 total, 11 up, 12 in 2024-09-11T22:49:39.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:38 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:49:39.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:38 smithi204 ceph-mon[35454]: from='client.? 172.21.15.204:0/537120715' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:49:39.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:38 smithi152 ceph-mon[31181]: from='client.? 172.21.15.204:0/1424234574' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c76a93b1-a045-477b-98d0-58a2e9d44a7f"}]: dispatch 2024-09-11T22:49:39.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:38 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c76a93b1-a045-477b-98d0-58a2e9d44a7f"}]: dispatch 2024-09-11T22:49:39.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:38 smithi152 ceph-mon[31181]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c76a93b1-a045-477b-98d0-58a2e9d44a7f"}]': finished 2024-09-11T22:49:39.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:38 smithi152 ceph-mon[31181]: osdmap e71: 12 total, 11 up, 12 in 2024-09-11T22:49:39.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:38 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:49:39.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:38 smithi152 ceph-mon[31181]: from='client.? 172.21.15.204:0/537120715' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:49:39.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:38 smithi191 ceph-mon[35211]: from='client.? 172.21.15.204:0/1424234574' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c76a93b1-a045-477b-98d0-58a2e9d44a7f"}]: dispatch 2024-09-11T22:49:39.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:38 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c76a93b1-a045-477b-98d0-58a2e9d44a7f"}]: dispatch 2024-09-11T22:49:39.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:38 smithi191 ceph-mon[35211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c76a93b1-a045-477b-98d0-58a2e9d44a7f"}]': finished 2024-09-11T22:49:39.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:38 smithi191 ceph-mon[35211]: osdmap e71: 12 total, 11 up, 12 in 2024-09-11T22:49:39.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:38 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:49:39.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:38 smithi191 ceph-mon[35211]: from='client.? 172.21.15.204:0/537120715' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:49:40.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:39 smithi152 ceph-mon[31181]: pgmap v233: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-09-11T22:49:40.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:39 smithi152 ceph-mon[31181]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T22:49:40.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:39 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:49:40.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:39 smithi191 ceph-mon[35211]: pgmap v233: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-09-11T22:49:40.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:39 smithi191 ceph-mon[35211]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T22:49:40.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:39 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:49:40.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:39 smithi204 ceph-mon[35454]: pgmap v233: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-09-11T22:49:40.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:39 smithi204 ceph-mon[35454]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T22:49:40.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:39 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:49:42.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:41 smithi152 ceph-mon[31181]: pgmap v235: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-09-11T22:49:42.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:41 smithi191 ceph-mon[35211]: pgmap v235: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-09-11T22:49:42.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:41 smithi204 ceph-mon[35454]: pgmap v235: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-09-11T22:49:44.105 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:43 smithi204 ceph-mon[35454]: pgmap v236: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T22:49:44.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:43 smithi152 ceph-mon[31181]: pgmap v236: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T22:49:44.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:43 smithi191 ceph-mon[35211]: pgmap v236: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T22:49:45.869 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:45 smithi204 ceph-mon[35454]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-11T22:49:46.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:45 smithi152 ceph-mon[31181]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-11T22:49:46.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:45 smithi191 ceph-mon[35211]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-11T22:49:47.068 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:46 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T22:49:47.069 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:46 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:47.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:46 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T22:49:47.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:46 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:47.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:46 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T22:49:47.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:46 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:49:48.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:47 smithi152 ceph-mon[31181]: Deploying daemon osd.11 on smithi204 2024-09-11T22:49:48.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:47 smithi152 ceph-mon[31181]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:48.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:47 smithi204 ceph-mon[35454]: Deploying daemon osd.11 on smithi204 2024-09-11T22:49:48.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:47 smithi204 ceph-mon[35454]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:48.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:47 smithi191 ceph-mon[35211]: Deploying daemon osd.11 on smithi204 2024-09-11T22:49:48.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:47 smithi191 ceph-mon[35211]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:50.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:49 smithi152 ceph-mon[31181]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:50.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:49 smithi204 ceph-mon[35454]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:50.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:49 smithi191 ceph-mon[35211]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:52.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:51 smithi152 ceph-mon[31181]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:52.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:51 smithi204 ceph-mon[35454]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:52.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:51 smithi191 ceph-mon[35211]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:53.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:52 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:49:53.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:52 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:53.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:52 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:53.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:52 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:49:53.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:52 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:53.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:52 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:53.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:52 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:49:53.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:52 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:53.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:52 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:54.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:53 smithi204 ceph-mon[35454]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:54.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:53 smithi152 ceph-mon[31181]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:54.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:53 smithi191 ceph-mon[35211]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:55.818 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:55.817+0000 7f84cb7fe640 1 -- 172.21.15.204:0/1548829932 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f84d406a6f0 con 0x7f84c00a2bb0 2024-09-11T22:49:55.818 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 11 on host 'smithi204' 2024-09-11T22:49:55.822 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:55.819+0000 7f84d8911640 1 -- 172.21.15.204:0/1548829932 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f84c00a2bb0 msgr2=0x7f84c00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:55.822 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:55.819+0000 7f84d8911640 1 --2- 172.21.15.204:0/1548829932 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f84c00a2bb0 0x7f84c00a5070 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f84cc006010 tx=0x7f84cc000f40 comp rx=0 tx=0).stop 2024-09-11T22:49:55.822 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:55.819+0000 7f84d8911640 1 -- 172.21.15.204:0/1548829932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d4102690 msgr2=0x7f84d41a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:55.822 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:55.819+0000 7f84d8911640 1 --2- 172.21.15.204:0/1548829932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d4102690 0x7f84d41a16f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f84c400e970 tx=0x7f84c400ed70 comp rx=0 tx=0).stop 2024-09-11T22:49:55.822 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:55.819+0000 7f84d8911640 1 -- 172.21.15.204:0/1548829932 shutdown_connections 2024-09-11T22:49:55.822 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:55.819+0000 7f84d8911640 1 --2- 172.21.15.204:0/1548829932 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f84c00a2bb0 0x7f84c00a5070 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:55.822 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:55.819+0000 7f84d8911640 1 --2- 172.21.15.204:0/1548829932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d4102690 0x7f84d41a16f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:55.822 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:55.820+0000 7f84d8911640 1 --2- 172.21.15.204:0/1548829932 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f84d4101cd0 0x7f84d419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:55.823 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:55.820+0000 7f84d8911640 1 --2- 172.21.15.204:0/1548829932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f84d4100ad0 0x7f84d419a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:55.823 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:55.820+0000 7f84d8911640 1 -- 172.21.15.204:0/1548829932 >> 172.21.15.204:0/1548829932 conn(0x7f84d40fc280 msgr2=0x7f84d4069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:49:55.823 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:55.820+0000 7f84d8911640 1 -- 172.21.15.204:0/1548829932 shutdown_connections 2024-09-11T22:49:55.823 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:49:55.820+0000 7f84d8911640 1 -- 172.21.15.204:0/1548829932 wait complete. 2024-09-11T22:49:56.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:55 smithi204 ceph-mon[35454]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:56.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:56.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:56.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:55 smithi152 ceph-mon[31181]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:56.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:56.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:56.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:55 smithi191 ceph-mon[35211]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:56.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:56.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:56.501 DEBUG:teuthology.orchestra.run.smithi204:osd.11> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.11.service 2024-09-11T22:49:56.503 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-11T22:49:56.503 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T22:49:56.762 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:49:57.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.255+0000 7f96081f9640 1 -- 172.21.15.152:0/3439847002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9600101c00 msgr2=0x7f9600102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:57.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.255+0000 7f96081f9640 1 --2- 172.21.15.152:0/3439847002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9600101c00 0x7f9600102000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f95f0009900 tx=0x7f95f002a410 comp rx=0 tx=0).stop 2024-09-11T22:49:57.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.255+0000 7f96081f9640 1 -- 172.21.15.152:0/3439847002 shutdown_connections 2024-09-11T22:49:57.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.255+0000 7f96081f9640 1 --2- 172.21.15.152:0/3439847002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96001037c0 0x7f960010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:57.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.255+0000 7f96081f9640 1 --2- 172.21.15.152:0/3439847002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9600102e00 0x7f9600103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:57.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.255+0000 7f96081f9640 1 --2- 172.21.15.152:0/3439847002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9600101c00 0x7f9600102000 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:57.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.255+0000 7f96081f9640 1 -- 172.21.15.152:0/3439847002 >> 172.21.15.152:0/3439847002 conn(0x7f96000fd390 msgr2=0x7f96000ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:49:57.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.255+0000 7f96081f9640 1 -- 172.21.15.152:0/3439847002 shutdown_connections 2024-09-11T22:49:57.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.255+0000 7f96081f9640 1 -- 172.21.15.152:0/3439847002 wait complete. 2024-09-11T22:49:57.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f96081f9640 1 Processor -- start 2024-09-11T22:49:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f96081f9640 1 -- start start 2024-09-11T22:49:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f96081f9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9600101c00 0x7f9600075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:49:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f96081f9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9600102e00 0x7f96000745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:49:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f96081f9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96001037c0 0x7f9600074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:49:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f96081f9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f960010d960 con 0x7f96001037c0 2024-09-11T22:49:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f96081f9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f960010d7e0 con 0x7f9600101c00 2024-09-11T22:49:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f96081f9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f960010dae0 con 0x7f9600102e00 2024-09-11T22:49:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f9606796640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96001037c0 0x7f9600074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:49:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f9606796640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96001037c0 0x7f9600074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49568/0 (socket says 172.21.15.152:49568) 2024-09-11T22:49:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f9606796640 1 -- 172.21.15.152:0/1533726755 learned_addr learned my addr 172.21.15.152:0/1533726755 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:49:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f9605f95640 1 --2- 172.21.15.152:0/1533726755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9600101c00 0x7f9600075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:49:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f9605794640 1 --2- 172.21.15.152:0/1533726755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9600102e00 0x7f96000745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:49:57.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f9606796640 1 -- 172.21.15.152:0/1533726755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9600102e00 msgr2=0x7f96000745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:57.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f9606796640 1 --2- 172.21.15.152:0/1533726755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9600102e00 0x7f96000745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:57.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f9606796640 1 -- 172.21.15.152:0/1533726755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9600101c00 msgr2=0x7f9600075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:57.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f9606796640 1 --2- 172.21.15.152:0/1533726755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9600101c00 0x7f9600075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:57.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f9606796640 1 -- 172.21.15.152:0/1533726755 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96000753e0 con 0x7f96001037c0 2024-09-11T22:49:57.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f9606796640 1 --2- 172.21.15.152:0/1533726755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96001037c0 0x7f9600074b20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f95f800b970 tx=0x7f95f800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:49:57.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.256+0000 7f95f6ffd640 1 -- 172.21.15.152:0/1533726755 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8019070 con 0x7f96001037c0 2024-09-11T22:49:57.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.257+0000 7f96081f9640 1 -- 172.21.15.152:0/1533726755 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96001a21e0 con 0x7f96001037c0 2024-09-11T22:49:57.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.257+0000 7f95f6ffd640 1 -- 172.21.15.152:0/1533726755 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95f800cb40 con 0x7f96001037c0 2024-09-11T22:49:57.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.257+0000 7f95f6ffd640 1 -- 172.21.15.152:0/1533726755 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f8007650 con 0x7f96001037c0 2024-09-11T22:49:57.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.257+0000 7f96081f9640 1 -- 172.21.15.152:0/1533726755 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96001a2720 con 0x7f96001037c0 2024-09-11T22:49:57.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.258+0000 7f96081f9640 1 -- 172.21.15.152:0/1533726755 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96001090a0 con 0x7f96001037c0 2024-09-11T22:49:57.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.259+0000 7f95f6ffd640 1 -- 172.21.15.152:0/1533726755 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f95f801c070 con 0x7f96001037c0 2024-09-11T22:49:57.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.259+0000 7f95f6ffd640 1 --2- 172.21.15.152:0/1533726755 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f95ec0a2b80 0x7f95ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:49:57.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.259+0000 7f9605f95640 1 --2- 172.21.15.152:0/1533726755 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f95ec0a2b80 0x7f95ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:49:57.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.259+0000 7f95f6ffd640 1 -- 172.21.15.152:0/1533726755 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6809+0+0 (secure 0 0 0) 0x7f95f80ce5d0 con 0x7f96001037c0 2024-09-11T22:49:57.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.260+0000 7f9605f95640 1 --2- 172.21.15.152:0/1533726755 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f95ec0a2b80 0x7f95ec0a5040 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f95f0006010 tx=0x7f95f0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:49:57.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.262+0000 7f95f6ffd640 1 -- 172.21.15.152:0/1533726755 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95f809ba70 con 0x7f96001037c0 2024-09-11T22:49:57.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.431+0000 7f96081f9640 1 -- 172.21.15.152:0/1533726755 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f9600062df0 con 0x7f96001037c0 2024-09-11T22:49:57.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.432+0000 7f95f6ffd640 1 -- 172.21.15.152:0/1533726755 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f95f809b370 con 0x7f96001037c0 2024-09-11T22:49:57.433 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:49:57.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.434+0000 7f96081f9640 1 -- 172.21.15.152:0/1533726755 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f95ec0a2b80 msgr2=0x7f95ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:57.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.434+0000 7f96081f9640 1 --2- 172.21.15.152:0/1533726755 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f95ec0a2b80 0x7f95ec0a5040 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f95f0006010 tx=0x7f95f0000bc0 comp rx=0 tx=0).stop 2024-09-11T22:49:57.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.434+0000 7f96081f9640 1 -- 172.21.15.152:0/1533726755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96001037c0 msgr2=0x7f9600074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:49:57.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.434+0000 7f96081f9640 1 --2- 172.21.15.152:0/1533726755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96001037c0 0x7f9600074b20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f95f800b970 tx=0x7f95f800be30 comp rx=0 tx=0).stop 2024-09-11T22:49:57.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.434+0000 7f96081f9640 1 -- 172.21.15.152:0/1533726755 shutdown_connections 2024-09-11T22:49:57.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.434+0000 7f96081f9640 1 --2- 172.21.15.152:0/1533726755 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f95ec0a2b80 0x7f95ec0a5040 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:57.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.434+0000 7f96081f9640 1 --2- 172.21.15.152:0/1533726755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96001037c0 0x7f9600074b20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:57.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.434+0000 7f96081f9640 1 --2- 172.21.15.152:0/1533726755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9600102e00 0x7f96000745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:57.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.434+0000 7f96081f9640 1 --2- 172.21.15.152:0/1533726755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9600101c00 0x7f9600075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:49:57.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.434+0000 7f96081f9640 1 -- 172.21.15.152:0/1533726755 >> 172.21.15.152:0/1533726755 conn(0x7f96000fd390 msgr2=0x7f9600106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:49:57.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.435+0000 7f96081f9640 1 -- 172.21.15.152:0/1533726755 shutdown_connections 2024-09-11T22:49:57.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:49:57.435+0000 7f96081f9640 1 -- 172.21.15.152:0/1533726755 wait complete. 2024-09-11T22:49:57.619 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:57 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:57.718 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:57 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:57.718 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:57 smithi152 ceph-mon[31181]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:57.718 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:57 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1533726755' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:49:58.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:57 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:58.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:57 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:58.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:57 smithi191 ceph-mon[35211]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:58.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:57 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1533726755' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:49:58.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:57 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:58.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:57 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:49:58.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:57 smithi204 ceph-mon[35454]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:58.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:57 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1533726755' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:49:58.389 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726094975,"num_in_osds":12,"osd_in_since":1726094978,"num_remapped_pgs":0} 2024-09-11T22:49:58.623 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 22:49:58 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T22:49:58.487+0000 7f95bc3fd640 -1 osd.11 0 log_to_monitors true 2024-09-11T22:49:58.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:58 smithi152 ceph-mon[31181]: from='osd.11 [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:49:58.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:58 smithi152 ceph-mon[31181]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:49:59.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:58 smithi191 ceph-mon[35211]: from='osd.11 [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:49:59.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:58 smithi191 ceph-mon[35211]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:49:59.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:58 smithi204 ceph-mon[35454]: from='osd.11 [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:49:59.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:58 smithi204 ceph-mon[35454]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:49:59.391 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T22:49:59.641 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:49:59.723 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:59 smithi204 ceph-mon[35454]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T22:49:59.723 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:59 smithi204 ceph-mon[35454]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T22:49:59.723 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:59 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:49:59.723 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:59 smithi204 ceph-mon[35454]: from='osd.11 [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:49:59.723 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:59 smithi204 ceph-mon[35454]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:49:59.723 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:49:59 smithi204 ceph-mon[35454]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:49:59.762 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:59 smithi152 ceph-mon[31181]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T22:49:59.762 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:59 smithi152 ceph-mon[31181]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T22:49:59.762 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:59 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:49:59.762 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:59 smithi152 ceph-mon[31181]: from='osd.11 [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:49:59.762 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:59 smithi152 ceph-mon[31181]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:49:59.763 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:49:59 smithi152 ceph-mon[31181]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:50:00.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:59 smithi191 ceph-mon[35211]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T22:50:00.040 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:59 smithi191 ceph-mon[35211]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T22:50:00.040 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:59 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:50:00.040 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:59 smithi191 ceph-mon[35211]: from='osd.11 [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:50:00.040 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:59 smithi191 ceph-mon[35211]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-09-11T22:50:00.041 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:49:59 smithi191 ceph-mon[35211]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:50:00.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.132+0000 7fc8dc11a640 1 -- 172.21.15.152:0/3851056998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8d4102e00 msgr2=0x7fc8d4103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:00.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.132+0000 7fc8dc11a640 1 --2- 172.21.15.152:0/3851056998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8d4102e00 0x7fc8d4103280 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009900 tx=0x7fc8c402a410 comp rx=0 tx=0).stop 2024-09-11T22:50:00.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.133+0000 7fc8dc11a640 1 -- 172.21.15.152:0/3851056998 shutdown_connections 2024-09-11T22:50:00.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.133+0000 7fc8dc11a640 1 --2- 172.21.15.152:0/3851056998 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8d41037c0 0x7fc8d410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:00.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.133+0000 7fc8dc11a640 1 --2- 172.21.15.152:0/3851056998 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8d4102e00 0x7fc8d4103280 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:00.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.133+0000 7fc8dc11a640 1 --2- 172.21.15.152:0/3851056998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d4101c00 0x7fc8d4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:00.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.133+0000 7fc8dc11a640 1 -- 172.21.15.152:0/3851056998 >> 172.21.15.152:0/3851056998 conn(0x7fc8d40fd390 msgr2=0x7fc8d40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:00.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.133+0000 7fc8dc11a640 1 -- 172.21.15.152:0/3851056998 shutdown_connections 2024-09-11T22:50:00.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.133+0000 7fc8dc11a640 1 -- 172.21.15.152:0/3851056998 wait complete. 2024-09-11T22:50:00.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.133+0000 7fc8dc11a640 1 Processor -- start 2024-09-11T22:50:00.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.133+0000 7fc8dc11a640 1 -- start start 2024-09-11T22:50:00.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.133+0000 7fc8dc11a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8d4101c00 0x7fc8d4197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:00.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.133+0000 7fc8dc11a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8d4102e00 0x7fc8d41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:00.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.133+0000 7fc8dc11a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d41037c0 0x7fc8d419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:00.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.133+0000 7fc8dc11a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d410d880 con 0x7fc8d4102e00 2024-09-11T22:50:00.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.133+0000 7fc8dc11a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d410d700 con 0x7fc8d4101c00 2024-09-11T22:50:00.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.133+0000 7fc8dc11a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d410da00 con 0x7fc8d41037c0 2024-09-11T22:50:00.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.133+0000 7fc8d96b5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8d4102e00 0x7fc8d41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:00.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.134+0000 7fc8d96b5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8d4102e00 0x7fc8d41983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40794/0 (socket says 172.21.15.152:40794) 2024-09-11T22:50:00.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.134+0000 7fc8d96b5640 1 -- 172.21.15.152:0/3169664780 learned_addr learned my addr 172.21.15.152:0/3169664780 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:00.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.134+0000 7fc8d9eb6640 1 --2- 172.21.15.152:0/3169664780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8d4101c00 0x7fc8d4197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:00.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.134+0000 7fc8da6b7640 1 --2- 172.21.15.152:0/3169664780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d41037c0 0x7fc8d419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:00.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.134+0000 7fc8d96b5640 1 -- 172.21.15.152:0/3169664780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d41037c0 msgr2=0x7fc8d419f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:00.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.134+0000 7fc8d96b5640 1 --2- 172.21.15.152:0/3169664780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d41037c0 0x7fc8d419f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:00.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.134+0000 7fc8d96b5640 1 -- 172.21.15.152:0/3169664780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8d4101c00 msgr2=0x7fc8d4197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:00.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.134+0000 7fc8d96b5640 1 --2- 172.21.15.152:0/3169664780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8d4101c00 0x7fc8d4197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:00.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.134+0000 7fc8d96b5640 1 -- 172.21.15.152:0/3169664780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d419fad0 con 0x7fc8d4102e00 2024-09-11T22:50:00.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.134+0000 7fc8d96b5640 1 --2- 172.21.15.152:0/3169664780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8d4102e00 0x7fc8d41983e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009230 tx=0x7fc8c4031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:00.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.134+0000 7fc8caffd640 1 -- 172.21.15.152:0/3169664780 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c403f030 con 0x7fc8d4102e00 2024-09-11T22:50:00.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.134+0000 7fc8caffd640 1 -- 172.21.15.152:0/3169664780 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8c4003ad0 con 0x7fc8d4102e00 2024-09-11T22:50:00.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.134+0000 7fc8dc11a640 1 -- 172.21.15.152:0/3169664780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d419fd60 con 0x7fc8d4102e00 2024-09-11T22:50:00.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.134+0000 7fc8caffd640 1 -- 172.21.15.152:0/3169664780 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4035070 con 0x7fc8d4102e00 2024-09-11T22:50:00.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.134+0000 7fc8dc11a640 1 -- 172.21.15.152:0/3169664780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d41a01e0 con 0x7fc8d4102e00 2024-09-11T22:50:00.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.135+0000 7fc8dc11a640 1 -- 172.21.15.152:0/3169664780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d4108fb0 con 0x7fc8d4102e00 2024-09-11T22:50:00.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.136+0000 7fc8caffd640 1 -- 172.21.15.152:0/3169664780 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc8c4042030 con 0x7fc8d4102e00 2024-09-11T22:50:00.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.137+0000 7fc8caffd640 1 --2- 172.21.15.152:0/3169664780 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc8c00a2bb0 0x7fc8c00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:00.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.137+0000 7fc8d9eb6640 1 --2- 172.21.15.152:0/3169664780 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc8c00a2bb0 0x7fc8c00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:00.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.137+0000 7fc8caffd640 1 -- 172.21.15.152:0/3169664780 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6847+0+0 (secure 0 0 0) 0x7fc8c40ef3b0 con 0x7fc8d4102e00 2024-09-11T22:50:00.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.137+0000 7fc8d9eb6640 1 --2- 172.21.15.152:0/3169664780 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc8c00a2bb0 0x7fc8c00a5070 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4102c60 tx=0x7fc8d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:00.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.140+0000 7fc8caffd640 1 -- 172.21.15.152:0/3169664780 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8c40bc8d0 con 0x7fc8d4102e00 2024-09-11T22:50:00.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T22:50:00.040+0000 7f95b8380640 -1 osd.11 0 waiting for initial osdmap 2024-09-11T22:50:00.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T22:50:00.053+0000 7f95b01a0640 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:50:00.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.307+0000 7fc8dc11a640 1 -- 172.21.15.152:0/3169664780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc8d402da10 con 0x7fc8d4102e00 2024-09-11T22:50:00.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.307+0000 7fc8caffd640 1 -- 172.21.15.152:0/3169664780 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7fc8c40bc1d0 con 0x7fc8d4102e00 2024-09-11T22:50:00.308 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:50:00.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.309+0000 7fc8dc11a640 1 -- 172.21.15.152:0/3169664780 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc8c00a2bb0 msgr2=0x7fc8c00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:00.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.309+0000 7fc8dc11a640 1 --2- 172.21.15.152:0/3169664780 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc8c00a2bb0 0x7fc8c00a5070 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4102c60 tx=0x7fc8d0005f90 comp rx=0 tx=0).stop 2024-09-11T22:50:00.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.309+0000 7fc8dc11a640 1 -- 172.21.15.152:0/3169664780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8d4102e00 msgr2=0x7fc8d41983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:00.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.309+0000 7fc8dc11a640 1 --2- 172.21.15.152:0/3169664780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8d4102e00 0x7fc8d41983e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009230 tx=0x7fc8c4031b60 comp rx=0 tx=0).stop 2024-09-11T22:50:00.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.309+0000 7fc8dc11a640 1 -- 172.21.15.152:0/3169664780 shutdown_connections 2024-09-11T22:50:00.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.309+0000 7fc8dc11a640 1 --2- 172.21.15.152:0/3169664780 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc8c00a2bb0 0x7fc8c00a5070 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:00.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.309+0000 7fc8dc11a640 1 --2- 172.21.15.152:0/3169664780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8d41037c0 0x7fc8d419f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:00.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.309+0000 7fc8dc11a640 1 --2- 172.21.15.152:0/3169664780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8d4102e00 0x7fc8d41983e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:00.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.309+0000 7fc8dc11a640 1 --2- 172.21.15.152:0/3169664780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8d4101c00 0x7fc8d4197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:00.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.309+0000 7fc8dc11a640 1 -- 172.21.15.152:0/3169664780 >> 172.21.15.152:0/3169664780 conn(0x7fc8d40fd390 msgr2=0x7fc8d4106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:00.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.309+0000 7fc8dc11a640 1 -- 172.21.15.152:0/3169664780 shutdown_connections 2024-09-11T22:50:00.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:00.309+0000 7fc8dc11a640 1 -- 172.21.15.152:0/3169664780 wait complete. 2024-09-11T22:50:01.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-11T22:50:01.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: osdmap e74: 12 total, 11 up, 12 in 2024-09-11T22:50:01.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:50:01.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:50:01.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: Detected new or changed devices on smithi204 2024-09-11T22:50:01.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:01.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:01.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:50:01.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:50:01.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:50:01.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:50:01.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi204 to 4272M 2024-09-11T22:50:01.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:01.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:01.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:50:01.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:01.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:50:01.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:01.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:50:01.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:01.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: overall HEALTH_OK 2024-09-11T22:50:01.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='osd.11 ' entity='osd.11' 2024-09-11T22:50:01.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:00 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3169664780' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:50:01.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-11T22:50:01.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: osdmap e74: 12 total, 11 up, 12 in 2024-09-11T22:50:01.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:50:01.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:50:01.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: Detected new or changed devices on smithi204 2024-09-11T22:50:01.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:01.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:01.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:50:01.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:50:01.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:50:01.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:50:01.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi204 to 4272M 2024-09-11T22:50:01.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:01.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:01.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:50:01.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:01.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:50:01.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:01.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:50:01.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:01.048 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: overall HEALTH_OK 2024-09-11T22:50:01.048 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='osd.11 ' entity='osd.11' 2024-09-11T22:50:01.048 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:00 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3169664780' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:50:01.098 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-09-11T22:50:01.098 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: osdmap e74: 12 total, 11 up, 12 in 2024-09-11T22:50:01.098 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:50:01.098 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:50:01.098 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: Detected new or changed devices on smithi204 2024-09-11T22:50:01.099 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:01.099 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:01.099 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:50:01.099 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:50:01.099 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:50:01.099 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:50:01.099 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi204 to 4272M 2024-09-11T22:50:01.099 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:01.099 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:01.099 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:50:01.099 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:01.099 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:50:01.099 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:01.100 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:50:01.100 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:01.100 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: overall HEALTH_OK 2024-09-11T22:50:01.100 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='osd.11 ' entity='osd.11' 2024-09-11T22:50:01.100 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:00 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3169664780' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:50:01.126 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1726094975,"num_in_osds":12,"osd_in_since":1726094978,"num_remapped_pgs":0} 2024-09-11T22:50:02.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:01 smithi191 ceph-mon[35211]: purged_snaps scrub starts 2024-09-11T22:50:02.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:01 smithi191 ceph-mon[35211]: purged_snaps scrub ok 2024-09-11T22:50:02.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:01 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:50:02.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:01 smithi191 ceph-mon[35211]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:50:02.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:01 smithi191 ceph-mon[35211]: osd.11 [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236] boot 2024-09-11T22:50:02.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:01 smithi191 ceph-mon[35211]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T22:50:02.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:01 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:50:02.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:01 smithi204 ceph-mon[35454]: purged_snaps scrub starts 2024-09-11T22:50:02.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:01 smithi204 ceph-mon[35454]: purged_snaps scrub ok 2024-09-11T22:50:02.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:01 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:50:02.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:01 smithi204 ceph-mon[35454]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:50:02.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:01 smithi204 ceph-mon[35454]: osd.11 [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236] boot 2024-09-11T22:50:02.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:01 smithi204 ceph-mon[35454]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T22:50:02.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:01 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:50:02.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:01 smithi152 ceph-mon[31181]: purged_snaps scrub starts 2024-09-11T22:50:02.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:01 smithi152 ceph-mon[31181]: purged_snaps scrub ok 2024-09-11T22:50:02.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:01 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:50:02.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:01 smithi152 ceph-mon[31181]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:50:02.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:01 smithi152 ceph-mon[31181]: osd.11 [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236] boot 2024-09-11T22:50:02.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:01 smithi152 ceph-mon[31181]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T22:50:02.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:01 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:50:02.127 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T22:50:02.375 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:02.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.848+0000 7f8de629f640 1 -- 172.21.15.152:0/713900255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de0100b10 msgr2=0x7f8de0100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:02.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.848+0000 7f8de629f640 1 --2- 172.21.15.152:0/713900255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de0100b10 0x7f8de0100f10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009900 tx=0x7f8dd002a410 comp rx=0 tx=0).stop 2024-09-11T22:50:02.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.848+0000 7f8de629f640 1 -- 172.21.15.152:0/713900255 shutdown_connections 2024-09-11T22:50:02.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.848+0000 7f8de629f640 1 --2- 172.21.15.152:0/713900255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8de01026d0 0x7f8de0108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:02.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.848+0000 7f8de629f640 1 --2- 172.21.15.152:0/713900255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8de0101d10 0x7f8de0102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:02.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.848+0000 7f8de629f640 1 --2- 172.21.15.152:0/713900255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de0100b10 0x7f8de0100f10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:02.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.848+0000 7f8de629f640 1 -- 172.21.15.152:0/713900255 >> 172.21.15.152:0/713900255 conn(0x7f8de00fc2a0 msgr2=0x7f8de00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:02.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.848+0000 7f8de629f640 1 -- 172.21.15.152:0/713900255 shutdown_connections 2024-09-11T22:50:02.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.848+0000 7f8de629f640 1 -- 172.21.15.152:0/713900255 wait complete. 2024-09-11T22:50:02.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.848+0000 7f8de629f640 1 Processor -- start 2024-09-11T22:50:02.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.848+0000 7f8de629f640 1 -- start start 2024-09-11T22:50:02.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8de629f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8de0100b10 0x7f8de0196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:02.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8de629f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de0101d10 0x7f8de0197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:02.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8de629f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8de01026d0 0x7f8de019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:02.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8de629f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8de010c7a0 con 0x7f8de0101d10 2024-09-11T22:50:02.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8de629f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8de010c620 con 0x7f8de01026d0 2024-09-11T22:50:02.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8de629f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8de010c920 con 0x7f8de0100b10 2024-09-11T22:50:02.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8ddf7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de0101d10 0x7f8de0197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:02.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8ddf7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de0101d10 0x7f8de0197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40820/0 (socket says 172.21.15.152:40820) 2024-09-11T22:50:02.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8ddffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8de0100b10 0x7f8de0196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:02.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8ddf7fe640 1 -- 172.21.15.152:0/239285328 learned_addr learned my addr 172.21.15.152:0/239285328 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:02.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8de483c640 1 --2- 172.21.15.152:0/239285328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8de01026d0 0x7f8de019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:02.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8ddf7fe640 1 -- 172.21.15.152:0/239285328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8de0100b10 msgr2=0x7f8de0196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:02.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8ddf7fe640 1 --2- 172.21.15.152:0/239285328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8de0100b10 0x7f8de0196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8ddf7fe640 1 -- 172.21.15.152:0/239285328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8de01026d0 msgr2=0x7f8de019e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8ddf7fe640 1 --2- 172.21.15.152:0/239285328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8de01026d0 0x7f8de019e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8ddf7fe640 1 -- 172.21.15.152:0/239285328 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de019ea20 con 0x7f8de0101d10 2024-09-11T22:50:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.849+0000 7f8ddf7fe640 1 --2- 172.21.15.152:0/239285328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de0101d10 0x7f8de0197330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400d950 tx=0x7f8dd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.850+0000 7f8ddd7fa640 1 -- 172.21.15.152:0/239285328 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4017070 con 0x7f8de0101d10 2024-09-11T22:50:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.850+0000 7f8de629f640 1 -- 172.21.15.152:0/239285328 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de019ed10 con 0x7f8de0101d10 2024-09-11T22:50:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.850+0000 7f8ddd7fa640 1 -- 172.21.15.152:0/239285328 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dd4009c30 con 0x7f8de0101d10 2024-09-11T22:50:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.850+0000 7f8ddd7fa640 1 -- 172.21.15.152:0/239285328 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd400f6b0 con 0x7f8de0101d10 2024-09-11T22:50:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.850+0000 7f8de629f640 1 -- 172.21.15.152:0/239285328 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de019f1f0 con 0x7f8de0101d10 2024-09-11T22:50:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.851+0000 7f8de629f640 1 -- 172.21.15.152:0/239285328 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de0107f60 con 0x7f8de0101d10 2024-09-11T22:50:02.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.852+0000 7f8ddd7fa640 1 -- 172.21.15.152:0/239285328 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8dd40109d0 con 0x7f8de0101d10 2024-09-11T22:50:02.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.852+0000 7f8ddd7fa640 1 --2- 172.21.15.152:0/239285328 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8dc80a2bb0 0x7f8dc80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:02.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.852+0000 7f8ddffff640 1 --2- 172.21.15.152:0/239285328 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8dc80a2bb0 0x7f8dc80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:02.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.852+0000 7f8ddd7fa640 1 -- 172.21.15.152:0/239285328 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f8dd40ce710 con 0x7f8de0101d10 2024-09-11T22:50:02.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.853+0000 7f8ddffff640 1 --2- 172.21.15.152:0/239285328 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8dc80a2bb0 0x7f8dc80a5070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0005ab0 tx=0x7f8dd0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:02.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:02.856+0000 7f8ddd7fa640 1 -- 172.21.15.152:0/239285328 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8dd4009de0 con 0x7f8de0101d10 2024-09-11T22:50:03.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:03.021+0000 7f8de629f640 1 -- 172.21.15.152:0/239285328 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8de002d9c0 con 0x7f8de0101d10 2024-09-11T22:50:03.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:03.021+0000 7f8ddd7fa640 1 -- 172.21.15.152:0/239285328 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f8dd409b4c0 con 0x7f8de0101d10 2024-09-11T22:50:03.022 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:50:03.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:03.023+0000 7f8de629f640 1 -- 172.21.15.152:0/239285328 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8dc80a2bb0 msgr2=0x7f8dc80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:03.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:03.023+0000 7f8de629f640 1 --2- 172.21.15.152:0/239285328 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8dc80a2bb0 0x7f8dc80a5070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0005ab0 tx=0x7f8dd0000bc0 comp rx=0 tx=0).stop 2024-09-11T22:50:03.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:03.023+0000 7f8de629f640 1 -- 172.21.15.152:0/239285328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de0101d10 msgr2=0x7f8de0197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:03.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:03.023+0000 7f8de629f640 1 --2- 172.21.15.152:0/239285328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de0101d10 0x7f8de0197330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400d950 tx=0x7f8dd400de10 comp rx=0 tx=0).stop 2024-09-11T22:50:03.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:03.024+0000 7f8de629f640 1 -- 172.21.15.152:0/239285328 shutdown_connections 2024-09-11T22:50:03.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:03.024+0000 7f8de629f640 1 --2- 172.21.15.152:0/239285328 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8dc80a2bb0 0x7f8dc80a5070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:03.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:03.024+0000 7f8de629f640 1 --2- 172.21.15.152:0/239285328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8de01026d0 0x7f8de019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:03.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:03.024+0000 7f8de629f640 1 --2- 172.21.15.152:0/239285328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8de0101d10 0x7f8de0197330 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:03.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:03.024+0000 7f8de629f640 1 --2- 172.21.15.152:0/239285328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8de0100b10 0x7f8de0196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:03.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:03.024+0000 7f8de629f640 1 -- 172.21.15.152:0/239285328 >> 172.21.15.152:0/239285328 conn(0x7f8de00fc2a0 msgr2=0x7f8de0105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:03.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:03.024+0000 7f8de629f640 1 -- 172.21.15.152:0/239285328 shutdown_connections 2024-09-11T22:50:03.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:03.024+0000 7f8de629f640 1 -- 172.21.15.152:0/239285328 wait complete. 2024-09-11T22:50:03.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:03 smithi152 ceph-mon[31181]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T22:50:03.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:03 smithi152 ceph-mon[31181]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:03.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:03 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/239285328' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:50:03.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:03 smithi191 ceph-mon[35211]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T22:50:03.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:03 smithi191 ceph-mon[35211]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:03.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:03 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/239285328' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:50:03.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:03 smithi204 ceph-mon[35454]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T22:50:03.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:03 smithi204 ceph-mon[35454]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:03.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:03 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/239285328' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:50:03.845 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1726095001,"num_in_osds":12,"osd_in_since":1726094978,"num_remapped_pgs":0} 2024-09-11T22:50:03.846 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:50:04.092 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:04.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.582+0000 7f285247e640 1 -- 172.21.15.152:0/2886508751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f284c102570 msgr2=0x7f284c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:04.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.582+0000 7f285247e640 1 --2- 172.21.15.152:0/2886508751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f284c102570 0x7f284c1029b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2834009900 tx=0x7f283402a480 comp rx=0 tx=0).stop 2024-09-11T22:50:04.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.582+0000 7f285247e640 1 -- 172.21.15.152:0/2886508751 shutdown_connections 2024-09-11T22:50:04.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.582+0000 7f285247e640 1 --2- 172.21.15.152:0/2886508751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f284c102ef0 0x7f284c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:04.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.582+0000 7f285247e640 1 --2- 172.21.15.152:0/2886508751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f284c102570 0x7f284c1029b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:04.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.582+0000 7f285247e640 1 --2- 172.21.15.152:0/2886508751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f284c100e20 0x7f284c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:04.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.582+0000 7f285247e640 1 -- 172.21.15.152:0/2886508751 >> 172.21.15.152:0/2886508751 conn(0x7f284c0fc7d0 msgr2=0x7f284c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.582+0000 7f285247e640 1 -- 172.21.15.152:0/2886508751 shutdown_connections 2024-09-11T22:50:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.582+0000 7f285247e640 1 -- 172.21.15.152:0/2886508751 wait complete. 2024-09-11T22:50:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f285247e640 1 Processor -- start 2024-09-11T22:50:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f285247e640 1 -- start start 2024-09-11T22:50:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f285247e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f284c100e20 0x7f284c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f285247e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f284c102570 0x7f284c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f285247e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f284c102ef0 0x7f284c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f285247e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f284c110fc0 con 0x7f284c102570 2024-09-11T22:50:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f285247e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f284c110e40 con 0x7f284c102ef0 2024-09-11T22:50:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f285247e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f284c111140 con 0x7f284c100e20 2024-09-11T22:50:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f2843fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f284c102570 0x7f284c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f2843fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f284c102570 0x7f284c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40840/0 (socket says 172.21.15.152:40840) 2024-09-11T22:50:04.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f2843fff640 1 -- 172.21.15.152:0/2758455649 learned_addr learned my addr 172.21.15.152:0/2758455649 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:04.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f284bfff640 1 --2- 172.21.15.152:0/2758455649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f284c100e20 0x7f284c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:04.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f2850a1b640 1 --2- 172.21.15.152:0/2758455649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f284c102ef0 0x7f284c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:04.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f2843fff640 1 -- 172.21.15.152:0/2758455649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f284c100e20 msgr2=0x7f284c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:04.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f2843fff640 1 --2- 172.21.15.152:0/2758455649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f284c100e20 0x7f284c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:04.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f2843fff640 1 -- 172.21.15.152:0/2758455649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f284c102ef0 msgr2=0x7f284c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:04.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f2843fff640 1 --2- 172.21.15.152:0/2758455649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f284c102ef0 0x7f284c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:04.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f2843fff640 1 -- 172.21.15.152:0/2758455649 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c1a6a40 con 0x7f284c102570 2024-09-11T22:50:04.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f284bfff640 1 --2- 172.21.15.152:0/2758455649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f284c100e20 0x7f284c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:50:04.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.583+0000 7f2850a1b640 1 --2- 172.21.15.152:0/2758455649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f284c102ef0 0x7f284c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:50:04.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.584+0000 7f2843fff640 1 --2- 172.21.15.152:0/2758455649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f284c102570 0x7f284c0fd1c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2834006010 tx=0x7f2834003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:04.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.584+0000 7f2849ffb640 1 -- 172.21.15.152:0/2758455649 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283403f070 con 0x7f284c102570 2024-09-11T22:50:04.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.584+0000 7f285247e640 1 -- 172.21.15.152:0/2758455649 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284c1a6cd0 con 0x7f284c102570 2024-09-11T22:50:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.584+0000 7f2849ffb640 1 -- 172.21.15.152:0/2758455649 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2834031e30 con 0x7f284c102570 2024-09-11T22:50:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.584+0000 7f2849ffb640 1 -- 172.21.15.152:0/2758455649 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834035030 con 0x7f284c102570 2024-09-11T22:50:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.584+0000 7f285247e640 1 -- 172.21.15.152:0/2758455649 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284c1a71b0 con 0x7f284c102570 2024-09-11T22:50:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.585+0000 7f285247e640 1 -- 172.21.15.152:0/2758455649 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f284c101200 con 0x7f284c102570 2024-09-11T22:50:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.586+0000 7f2849ffb640 1 -- 172.21.15.152:0/2758455649 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2834042070 con 0x7f284c102570 2024-09-11T22:50:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.587+0000 7f2849ffb640 1 --2- 172.21.15.152:0/2758455649 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f28380a2b80 0x7f28380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.587+0000 7f284bfff640 1 --2- 172.21.15.152:0/2758455649 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f28380a2b80 0x7f28380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.587+0000 7f2849ffb640 1 -- 172.21.15.152:0/2758455649 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f28340ef5a0 con 0x7f284c102570 2024-09-11T22:50:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.587+0000 7f284bfff640 1 --2- 172.21.15.152:0/2758455649 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f28380a2b80 0x7f28380a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f283c0058c0 tx=0x7f283c005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:04.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.591+0000 7f2849ffb640 1 -- 172.21.15.152:0/2758455649 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28340bc9a0 con 0x7f284c102570 2024-09-11T22:50:04.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.758+0000 7f285247e640 1 -- 172.21.15.152:0/2758455649 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f284c10a0a0 con 0x7f284c102570 2024-09-11T22:50:04.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.759+0000 7f2849ffb640 1 -- 172.21.15.152:0/2758455649 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18552 (secure 0 0 0) 0x7f28340bc2a0 con 0x7f284c102570 2024-09-11T22:50:04.760 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:50:04.760 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":76,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","created":"2024-09-11T22:43:00.379317+0000","modified":"2024-09-11T22:50:02.044592+0000","last_up_change":"2024-09-11T22:50:01.043181+0000","last_in_change":"2024-09-11T22:49:38.073870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:46:10.984707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3cd917b5-77f6-4a4a-8fd0-4ebe3034feac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6803","nonce":1270728007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6805","nonce":1270728007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6809","nonce":1270728007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6807","nonce":1270728007}]},"public_addr":"172.21.15.152:6803/1270728007","cluster_addr":"172.21.15.152:6805/1270728007","heartbeat_back_addr":"172.21.15.152:6809/1270728007","heartbeat_front_addr":"172.21.15.152:6807/1270728007","state":["exists","up"]},{"osd":1,"uuid":"ae589bc4-77b1-4522-9591-b0ce2a5432dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6801","nonce":354467155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6803","nonce":354467155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6807","nonce":354467155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6805","nonce":354467155}]},"public_addr":"172.21.15.191:6801/354467155","cluster_addr":"172.21.15.191:6803/354467155","heartbeat_back_addr":"172.21.15.191:6807/354467155","heartbeat_front_addr":"172.21.15.191:6805/354467155","state":["exists","up"]},{"osd":2,"uuid":"b35ffd0c-d084-43dd-a947-15132d78d59c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6801","nonce":3848396211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6803","nonce":3848396211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6807","nonce":3848396211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6805","nonce":3848396211}]},"public_addr":"172.21.15.204:6801/3848396211","cluster_addr":"172.21.15.204:6803/3848396211","heartbeat_back_addr":"172.21.15.204:6807/3848396211","heartbeat_front_addr":"172.21.15.204:6805/3848396211","state":["exists","up"]},{"osd":3,"uuid":"d66ba7e8-4ab7-449e-9460-2f44ffdf987f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6811","nonce":3351257181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6813","nonce":3351257181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6817","nonce":3351257181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6815","nonce":3351257181}]},"public_addr":"172.21.15.152:6811/3351257181","cluster_addr":"172.21.15.152:6813/3351257181","heartbeat_back_addr":"172.21.15.152:6817/3351257181","heartbeat_front_addr":"172.21.15.152:6815/3351257181","state":["exists","up"]},{"osd":4,"uuid":"e1b56468-6cc7-4d29-988e-9d281118ee36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6809","nonce":2967539563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6811","nonce":2967539563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6815","nonce":2967539563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6813","nonce":2967539563}]},"public_addr":"172.21.15.191:6809/2967539563","cluster_addr":"172.21.15.191:6811/2967539563","heartbeat_back_addr":"172.21.15.191:6815/2967539563","heartbeat_front_addr":"172.21.15.191:6813/2967539563","state":["exists","up"]},{"osd":5,"uuid":"89dabd3e-77e3-4294-9648-b112ef694e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6809","nonce":4283562158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6811","nonce":4283562158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6815","nonce":4283562158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6813","nonce":4283562158}]},"public_addr":"172.21.15.204:6809/4283562158","cluster_addr":"172.21.15.204:6811/4283562158","heartbeat_back_addr":"172.21.15.204:6815/4283562158","heartbeat_front_addr":"172.21.15.204:6813/4283562158","state":["exists","up"]},{"osd":6,"uuid":"46a0fded-b668-4aef-a68f-613cccc83483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6819","nonce":2484234042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6821","nonce":2484234042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6825","nonce":2484234042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6823","nonce":2484234042}]},"public_addr":"172.21.15.152:6819/2484234042","cluster_addr":"172.21.15.152:6821/2484234042","heartbeat_back_addr":"172.21.15.152:6825/2484234042","heartbeat_front_addr":"172.21.15.152:6823/2484234042","state":["exists","up"]},{"osd":7,"uuid":"e674bfe9-e2db-44a3-b74b-a21f198747c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6817","nonce":269608859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6819","nonce":269608859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6823","nonce":269608859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6821","nonce":269608859}]},"public_addr":"172.21.15.191:6817/269608859","cluster_addr":"172.21.15.191:6819/269608859","heartbeat_back_addr":"172.21.15.191:6823/269608859","heartbeat_front_addr":"172.21.15.191:6821/269608859","state":["exists","up"]},{"osd":8,"uuid":"0418bfc9-d6b6-488f-b4c7-0bc6291bb624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6817","nonce":1756190090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6819","nonce":1756190090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6823","nonce":1756190090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6821","nonce":1756190090}]},"public_addr":"172.21.15.204:6817/1756190090","cluster_addr":"172.21.15.204:6819/1756190090","heartbeat_back_addr":"172.21.15.204:6823/1756190090","heartbeat_front_addr":"172.21.15.204:6821/1756190090","state":["exists","up"]},{"osd":9,"uuid":"751a6964-798f-4240-a2ba-7fa9f0c72657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6827","nonce":1301191419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6829","nonce":1301191419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6833","nonce":1301191419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6831","nonce":1301191419}]},"public_addr":"172.21.15.152:6827/1301191419","cluster_addr":"172.21.15.152:6829/1301191419","heartbeat_back_addr":"172.21.15.152:6833/1301191419","heartbeat_front_addr":"172.21.15.152:6831/1301191419","state":["exists","up"]},{"osd":10,"uuid":"90e7c6ba-127c-4b33-b3c6-cedf72128451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6825","nonce":3695294503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6827","nonce":3695294503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6831","nonce":3695294503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6829","nonce":3695294503}]},"public_addr":"172.21.15.191:6825/3695294503","cluster_addr":"172.21.15.191:6827/3695294503","heartbeat_back_addr":"172.21.15.191:6831/3695294503","heartbeat_front_addr":"172.21.15.191:6829/3695294503","state":["exists","up"]},{"osd":11,"uuid":"c76a93b1-a045-477b-98d0-58a2e9d44a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6825","nonce":1709433236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6827","nonce":1709433236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6831","nonce":1709433236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6829","nonce":1709433236}]},"public_addr":"172.21.15.204:6825/1709433236","cluster_addr":"172.21.15.204:6827/1709433236","heartbeat_back_addr":"172.21.15.204:6831/1709433236","heartbeat_front_addr":"172.21.15.204:6829/1709433236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:17.259744+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:42.230389+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:07.172433+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:32.517060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:58.900615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:24.379803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:50.058928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:15.967299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:42.025873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:08.225448+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:33.910531+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:59.466423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:0/1542993850":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/3071755865":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/2452559014":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/4286398432":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6801/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6800/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6800/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/3960601922":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2270507277":"2024-09-12T22:43:31.970432+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:50:04.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.761+0000 7f285247e640 1 -- 172.21.15.152:0/2758455649 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f28380a2b80 msgr2=0x7f28380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:04.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.761+0000 7f285247e640 1 --2- 172.21.15.152:0/2758455649 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f28380a2b80 0x7f28380a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f283c0058c0 tx=0x7f283c005d30 comp rx=0 tx=0).stop 2024-09-11T22:50:04.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.761+0000 7f285247e640 1 -- 172.21.15.152:0/2758455649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f284c102570 msgr2=0x7f284c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:04.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.761+0000 7f285247e640 1 --2- 172.21.15.152:0/2758455649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f284c102570 0x7f284c0fd1c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2834006010 tx=0x7f2834003c80 comp rx=0 tx=0).stop 2024-09-11T22:50:04.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.762+0000 7f285247e640 1 -- 172.21.15.152:0/2758455649 shutdown_connections 2024-09-11T22:50:04.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.762+0000 7f285247e640 1 --2- 172.21.15.152:0/2758455649 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f28380a2b80 0x7f28380a5040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:04.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.762+0000 7f285247e640 1 --2- 172.21.15.152:0/2758455649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f284c102ef0 0x7f284c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:04.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.762+0000 7f285247e640 1 --2- 172.21.15.152:0/2758455649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f284c102570 0x7f284c0fd1c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:04.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.762+0000 7f285247e640 1 --2- 172.21.15.152:0/2758455649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f284c100e20 0x7f284c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:04.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.762+0000 7f285247e640 1 -- 172.21.15.152:0/2758455649 >> 172.21.15.152:0/2758455649 conn(0x7f284c0fc7d0 msgr2=0x7f284c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:04.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.762+0000 7f285247e640 1 -- 172.21.15.152:0/2758455649 shutdown_connections 2024-09-11T22:50:04.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:04.762+0000 7f285247e640 1 -- 172.21.15.152:0/2758455649 wait complete. 2024-09-11T22:50:05.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:04 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2758455649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:50:05.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:04 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2758455649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:50:05.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:04 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2758455649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:50:05.582 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T22:46:10.984707+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T22:50:05.582 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T22:50:05.831 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:06.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:05 smithi204 ceph-mon[35454]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:06.113 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:05 smithi152 ceph-mon[31181]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:06.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:05 smithi191 ceph-mon[35211]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:06.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.337+0000 7f5113dc7640 1 -- 172.21.15.152:0/2223083315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f510c100090 msgr2=0x7f510c100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:06.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.337+0000 7f5113dc7640 1 --2- 172.21.15.152:0/2223083315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f510c100090 0x7f510c100510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009900 tx=0x7f50fc02a410 comp rx=0 tx=0).stop 2024-09-11T22:50:06.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.337+0000 7f5113dc7640 1 -- 172.21.15.152:0/2223083315 shutdown_connections 2024-09-11T22:50:06.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.337+0000 7f5113dc7640 1 --2- 172.21.15.152:0/2223083315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f510c100a50 0x7f510c1057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:06.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.337+0000 7f5113dc7640 1 --2- 172.21.15.152:0/2223083315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f510c100090 0x7f510c100510 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:06.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.337+0000 7f5113dc7640 1 --2- 172.21.15.152:0/2223083315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f510c106b70 0x7f510c106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:06.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.337+0000 7f5113dc7640 1 -- 172.21.15.152:0/2223083315 >> 172.21.15.152:0/2223083315 conn(0x7f510c077210 msgr2=0x7f510c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:06.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.337+0000 7f5113dc7640 1 -- 172.21.15.152:0/2223083315 shutdown_connections 2024-09-11T22:50:06.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.337+0000 7f5113dc7640 1 -- 172.21.15.152:0/2223083315 wait complete. 2024-09-11T22:50:06.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.338+0000 7f5113dc7640 1 Processor -- start 2024-09-11T22:50:06.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.338+0000 7f5113dc7640 1 -- start start 2024-09-11T22:50:06.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.338+0000 7f5113dc7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f510c100090 0x7f510c1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:06.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.338+0000 7f5113dc7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f510c100a50 0x7f510c1a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:06.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.338+0000 7f5113dc7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f510c106b70 0x7f510c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:06.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.338+0000 7f5113dc7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f510c1185c0 con 0x7f510c100090 2024-09-11T22:50:06.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.338+0000 7f5113dc7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f510c118440 con 0x7f510c100a50 2024-09-11T22:50:06.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.338+0000 7f5113dc7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f510c118740 con 0x7f510c106b70 2024-09-11T22:50:06.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.338+0000 7f5111b63640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f510c100090 0x7f510c1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:06.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.338+0000 7f5111b63640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f510c100090 0x7f510c1a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40860/0 (socket says 172.21.15.152:40860) 2024-09-11T22:50:06.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.338+0000 7f5111b63640 1 -- 172.21.15.152:0/4168103252 learned_addr learned my addr 172.21.15.152:0/4168103252 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:06.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.338+0000 7f5112364640 1 --2- 172.21.15.152:0/4168103252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f510c106b70 0x7f510c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:06.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.339+0000 7f5111362640 1 --2- 172.21.15.152:0/4168103252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f510c100a50 0x7f510c1a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:06.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.339+0000 7f5111b63640 1 -- 172.21.15.152:0/4168103252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f510c106b70 msgr2=0x7f510c19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:06.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.339+0000 7f5111b63640 1 --2- 172.21.15.152:0/4168103252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f510c106b70 0x7f510c19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:06.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.339+0000 7f5111b63640 1 -- 172.21.15.152:0/4168103252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f510c100a50 msgr2=0x7f510c1a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:06.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.339+0000 7f5111b63640 1 --2- 172.21.15.152:0/4168103252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f510c100a50 0x7f510c1a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:06.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.339+0000 7f5111b63640 1 -- 172.21.15.152:0/4168103252 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c19efe0 con 0x7f510c100090 2024-09-11T22:50:06.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.339+0000 7f5111b63640 1 --2- 172.21.15.152:0/4168103252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f510c100090 0x7f510c1a46c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f510800d950 tx=0x7f510800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:06.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.339+0000 7f5102ffd640 1 -- 172.21.15.152:0/4168103252 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108017070 con 0x7f510c100090 2024-09-11T22:50:06.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.339+0000 7f5113dc7640 1 -- 172.21.15.152:0/4168103252 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510c19f2d0 con 0x7f510c100090 2024-09-11T22:50:06.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.339+0000 7f5102ffd640 1 -- 172.21.15.152:0/4168103252 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5108009c30 con 0x7f510c100090 2024-09-11T22:50:06.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.339+0000 7f5102ffd640 1 -- 172.21.15.152:0/4168103252 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510800f6b0 con 0x7f510c100090 2024-09-11T22:50:06.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.339+0000 7f5113dc7640 1 -- 172.21.15.152:0/4168103252 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510c1ab520 con 0x7f510c100090 2024-09-11T22:50:06.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.340+0000 7f5113dc7640 1 -- 172.21.15.152:0/4168103252 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510c1012b0 con 0x7f510c100090 2024-09-11T22:50:06.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.341+0000 7f5102ffd640 1 -- 172.21.15.152:0/4168103252 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f51080109d0 con 0x7f510c100090 2024-09-11T22:50:06.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.342+0000 7f5102ffd640 1 --2- 172.21.15.152:0/4168103252 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f50f80a2b80 0x7f50f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:06.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.342+0000 7f5102ffd640 1 -- 172.21.15.152:0/4168103252 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f51080ce7d0 con 0x7f510c100090 2024-09-11T22:50:06.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.342+0000 7f5111362640 1 --2- 172.21.15.152:0/4168103252 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f50f80a2b80 0x7f50f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:06.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.342+0000 7f5111362640 1 --2- 172.21.15.152:0/4168103252 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f50f80a2b80 0x7f50f80a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f510c19fb90 tx=0x7f50fc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:06.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.345+0000 7f5102ffd640 1 -- 172.21.15.152:0/4168103252 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5108009de0 con 0x7f510c100090 2024-09-11T22:50:06.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.513+0000 7f5113dc7640 1 -- 172.21.15.152:0/4168103252 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f510c1ab750 con 0x7f510c100090 2024-09-11T22:50:06.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.514+0000 7f5102ffd640 1 -- 172.21.15.152:0/4168103252 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7f510809bbd0 con 0x7f510c100090 2024-09-11T22:50:06.515 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-09-11T22:50:06.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.515+0000 7f5113dc7640 1 -- 172.21.15.152:0/4168103252 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f50f80a2b80 msgr2=0x7f50f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.515+0000 7f5113dc7640 1 --2- 172.21.15.152:0/4168103252 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f50f80a2b80 0x7f50f80a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f510c19fb90 tx=0x7f50fc005a20 comp rx=0 tx=0).stop 2024-09-11T22:50:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.515+0000 7f5113dc7640 1 -- 172.21.15.152:0/4168103252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f510c100090 msgr2=0x7f510c1a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.515+0000 7f5113dc7640 1 --2- 172.21.15.152:0/4168103252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f510c100090 0x7f510c1a46c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f510800d950 tx=0x7f510800de10 comp rx=0 tx=0).stop 2024-09-11T22:50:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.515+0000 7f5113dc7640 1 -- 172.21.15.152:0/4168103252 shutdown_connections 2024-09-11T22:50:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.515+0000 7f5113dc7640 1 --2- 172.21.15.152:0/4168103252 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f50f80a2b80 0x7f50f80a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.515+0000 7f5113dc7640 1 --2- 172.21.15.152:0/4168103252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f510c106b70 0x7f510c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.515+0000 7f5113dc7640 1 --2- 172.21.15.152:0/4168103252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f510c100a50 0x7f510c1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.515+0000 7f5113dc7640 1 --2- 172.21.15.152:0/4168103252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f510c100090 0x7f510c1a46c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.516+0000 7f5113dc7640 1 -- 172.21.15.152:0/4168103252 >> 172.21.15.152:0/4168103252 conn(0x7f510c077210 msgr2=0x7f510c1037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.516+0000 7f5113dc7640 1 -- 172.21.15.152:0/4168103252 shutdown_connections 2024-09-11T22:50:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:06.516+0000 7f5113dc7640 1 -- 172.21.15.152:0/4168103252 wait complete. 2024-09-11T22:50:07.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:06 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4168103252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:50:07.213 INFO:tasks.cephadm:Adding mds.a on smithi152 2024-09-11T22:50:07.213 INFO:tasks.cephadm:Adding mds.d on smithi152 2024-09-11T22:50:07.213 INFO:tasks.cephadm:Adding mds.g on smithi152 2024-09-11T22:50:07.214 INFO:tasks.cephadm:Adding mds.j on smithi152 2024-09-11T22:50:07.214 INFO:tasks.cephadm:Adding mds.b on smithi191 2024-09-11T22:50:07.214 INFO:tasks.cephadm:Adding mds.e on smithi191 2024-09-11T22:50:07.214 INFO:tasks.cephadm:Adding mds.h on smithi191 2024-09-11T22:50:07.214 INFO:tasks.cephadm:Adding mds.k on smithi191 2024-09-11T22:50:07.214 INFO:tasks.cephadm:Adding mds.c on smithi204 2024-09-11T22:50:07.214 INFO:tasks.cephadm:Adding mds.f on smithi204 2024-09-11T22:50:07.214 INFO:tasks.cephadm:Adding mds.i on smithi204 2024-09-11T22:50:07.214 INFO:tasks.cephadm:Adding mds.l on smithi204 2024-09-11T22:50:07.214 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi152=a;smithi152=d;smithi152=g;smithi152=j;smithi191=b;smithi191=e;smithi191=h;smithi191=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l' 2024-09-11T22:50:07.241 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:06 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/4168103252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:50:07.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:06 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/4168103252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:50:07.468 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:50:07.893 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:07 smithi204 ceph-mon[35454]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:07.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.977+0000 7f12f6462640 1 -- 172.21.15.204:0/2816964663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f010e1f0 msgr2=0x7f12f0114a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:07.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.977+0000 7f12f6462640 1 --2- 172.21.15.204:0/2816964663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f010e1f0 0x7f12f0114a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009ca0 tx=0x7f12e802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:50:07.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.977+0000 7f12f6462640 1 -- 172.21.15.204:0/2816964663 shutdown_connections 2024-09-11T22:50:07.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.977+0000 7f12f6462640 1 --2- 172.21.15.204:0/2816964663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f010e1f0 0x7f12f0114a80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:07.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.977+0000 7f12f6462640 1 --2- 172.21.15.204:0/2816964663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12f010d830 0x7f12f010dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:07.995 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.977+0000 7f12f6462640 1 --2- 172.21.15.204:0/2816964663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f12f010c640 0x7f12f010ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:07.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.977+0000 7f12f6462640 1 -- 172.21.15.204:0/2816964663 >> 172.21.15.204:0/2816964663 conn(0x7f12f006c860 msgr2=0x7f12f006cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:07.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.977+0000 7f12f6462640 1 -- 172.21.15.204:0/2816964663 shutdown_connections 2024-09-11T22:50:07.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.977+0000 7f12f6462640 1 -- 172.21.15.204:0/2816964663 wait complete. 2024-09-11T22:50:07.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.977+0000 7f12f6462640 1 Processor -- start 2024-09-11T22:50:07.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.977+0000 7f12f6462640 1 -- start start 2024-09-11T22:50:07.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12f6462640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12f010c640 0x7f12f01a2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:07.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12f6462640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f12f010d830 0x7f12f01a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:07.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12f6462640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f010e1f0 0x7f12f01a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:07.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12f6462640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f12f0118270 con 0x7f12f010d830 2024-09-11T22:50:07.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12f6462640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01180f0 con 0x7f12f010c640 2024-09-11T22:50:07.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12f6462640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01183f0 con 0x7f12f010e1f0 2024-09-11T22:50:07.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12f49ff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f010e1f0 0x7f12f01a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:07.996 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12f49ff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f010e1f0 0x7f12f01a9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:39688/0 (socket says 172.21.15.204:39688) 2024-09-11T22:50:07.997 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12f49ff640 1 -- 172.21.15.204:0/996557745 learned_addr learned my addr 172.21.15.204:0/996557745 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-09-11T22:50:07.997 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12effff640 1 --2- 172.21.15.204:0/996557745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12f010c640 0x7f12f01a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:07.997 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12ef7fe640 1 --2- 172.21.15.204:0/996557745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f12f010d830 0x7f12f01a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:07.997 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12f49ff640 1 -- 172.21.15.204:0/996557745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12f010c640 msgr2=0x7f12f01a2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:07.997 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12f49ff640 1 --2- 172.21.15.204:0/996557745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12f010c640 0x7f12f01a2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:07.997 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12f49ff640 1 -- 172.21.15.204:0/996557745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f12f010d830 msgr2=0x7f12f01a2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:07.997 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12f49ff640 1 --2- 172.21.15.204:0/996557745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f12f010d830 0x7f12f01a2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:07.997 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12f49ff640 1 -- 172.21.15.204:0/996557745 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f01aa4c0 con 0x7f12f010e1f0 2024-09-11T22:50:07.997 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12ef7fe640 1 --2- 172.21.15.204:0/996557745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f12f010d830 0x7f12f01a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:50:07.997 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12f49ff640 1 --2- 172.21.15.204:0/996557745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f010e1f0 0x7f12f01a9e50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f12e8031a40 tx=0x7f12e8031ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:07.997 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12ed7fa640 1 -- 172.21.15.204:0/996557745 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8003c30 con 0x7f12f010e1f0 2024-09-11T22:50:07.997 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12f6462640 1 -- 172.21.15.204:0/996557745 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f01aa750 con 0x7f12f010e1f0 2024-09-11T22:50:07.997 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12ed7fa640 1 -- 172.21.15.204:0/996557745 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e8003dd0 con 0x7f12f010e1f0 2024-09-11T22:50:07.998 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.978+0000 7f12ed7fa640 1 -- 172.21.15.204:0/996557745 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e80047a0 con 0x7f12f010e1f0 2024-09-11T22:50:07.998 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.979+0000 7f12f6462640 1 -- 172.21.15.204:0/996557745 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f01aac90 con 0x7f12f010e1f0 2024-09-11T22:50:07.998 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.980+0000 7f12f6462640 1 -- 172.21.15.204:0/996557745 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12f0113a20 con 0x7f12f010e1f0 2024-09-11T22:50:07.998 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.981+0000 7f12ed7fa640 1 -- 172.21.15.204:0/996557745 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f12e802f710 con 0x7f12f010e1f0 2024-09-11T22:50:07.998 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.982+0000 7f12ed7fa640 1 --2- 172.21.15.204:0/996557745 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f12d80a2bb0 0x7f12d80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:07.998 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.982+0000 7f12ed7fa640 1 -- 172.21.15.204:0/996557745 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f12e80eff90 con 0x7f12f010e1f0 2024-09-11T22:50:07.998 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.982+0000 7f12effff640 1 --2- 172.21.15.204:0/996557745 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f12d80a2bb0 0x7f12d80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:07.998 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.982+0000 7f12effff640 1 --2- 172.21.15.204:0/996557745 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f12d80a2bb0 0x7f12d80a5070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f12f010d340 tx=0x7f12e000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:07.998 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:07.986+0000 7f12ed7fa640 1 -- 172.21.15.204:0/996557745 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f12e80bd390 con 0x7f12f010e1f0 2024-09-11T22:50:08.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:07 smithi152 ceph-mon[31181]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:08.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:08.161+0000 7f12f6462640 1 -- 172.21.15.204:0/996557745 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi152=a;smithi152=d;smithi152=g;smithi152=j;smithi191=b;smithi191=e;smithi191=h;smithi191=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}) -- 0x7f12f0062730 con 0x7f12d80a2bb0 2024-09-11T22:50:08.172 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:08.171+0000 7f12ed7fa640 1 -- 172.21.15.204:0/996557745 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f12f0062730 con 0x7f12d80a2bb0 2024-09-11T22:50:08.172 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mds.all update... 2024-09-11T22:50:08.174 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:08.173+0000 7f12f6462640 1 -- 172.21.15.204:0/996557745 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f12d80a2bb0 msgr2=0x7f12d80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:08.174 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:08.173+0000 7f12f6462640 1 --2- 172.21.15.204:0/996557745 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f12d80a2bb0 0x7f12d80a5070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f12f010d340 tx=0x7f12e000a2d0 comp rx=0 tx=0).stop 2024-09-11T22:50:08.175 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:08.173+0000 7f12f6462640 1 -- 172.21.15.204:0/996557745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f010e1f0 msgr2=0x7f12f01a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:08.175 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:08.173+0000 7f12f6462640 1 --2- 172.21.15.204:0/996557745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f010e1f0 0x7f12f01a9e50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f12e8031a40 tx=0x7f12e8031ec0 comp rx=0 tx=0).stop 2024-09-11T22:50:08.175 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:08.174+0000 7f12f6462640 1 -- 172.21.15.204:0/996557745 shutdown_connections 2024-09-11T22:50:08.175 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:08.174+0000 7f12f6462640 1 --2- 172.21.15.204:0/996557745 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f12d80a2bb0 0x7f12d80a5070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:08.175 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:08.174+0000 7f12f6462640 1 --2- 172.21.15.204:0/996557745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12f010e1f0 0x7f12f01a9e50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:08.175 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:08.174+0000 7f12f6462640 1 --2- 172.21.15.204:0/996557745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f12f010d830 0x7f12f01a2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:08.175 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:08.174+0000 7f12f6462640 1 --2- 172.21.15.204:0/996557745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12f010c640 0x7f12f01a2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:08.175 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:08.174+0000 7f12f6462640 1 -- 172.21.15.204:0/996557745 >> 172.21.15.204:0/996557745 conn(0x7f12f006c860 msgr2=0x7f12f0110bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:08.175 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:08.174+0000 7f12f6462640 1 -- 172.21.15.204:0/996557745 shutdown_connections 2024-09-11T22:50:08.175 INFO:teuthology.orchestra.run.smithi204.stderr:2024-09-11T22:50:08.174+0000 7f12f6462640 1 -- 172.21.15.204:0/996557745 wait complete. 2024-09-11T22:50:08.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:07 smithi191 ceph-mon[35211]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:08.867 DEBUG:teuthology.orchestra.run.smithi152:mds.a> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.a.service 2024-09-11T22:50:08.870 DEBUG:teuthology.orchestra.run.smithi152:mds.d> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.d.service 2024-09-11T22:50:08.913 DEBUG:teuthology.orchestra.run.smithi152:mds.g> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.g.service 2024-09-11T22:50:08.956 DEBUG:teuthology.orchestra.run.smithi152:mds.j> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.j.service 2024-09-11T22:50:08.999 DEBUG:teuthology.orchestra.run.smithi191:mds.b> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.b.service 2024-09-11T22:50:09.002 DEBUG:teuthology.orchestra.run.smithi191:mds.e> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.e.service 2024-09-11T22:50:09.045 DEBUG:teuthology.orchestra.run.smithi191:mds.h> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.h.service 2024-09-11T22:50:09.087 DEBUG:teuthology.orchestra.run.smithi191:mds.k> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.k.service 2024-09-11T22:50:09.130 DEBUG:teuthology.orchestra.run.smithi204:mds.c> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.c.service 2024-09-11T22:50:09.133 DEBUG:teuthology.orchestra.run.smithi204:mds.f> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.f.service 2024-09-11T22:50:09.176 DEBUG:teuthology.orchestra.run.smithi204:mds.i> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.i.service 2024-09-11T22:50:09.219 DEBUG:teuthology.orchestra.run.smithi204:mds.l> sudo journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.l.service 2024-09-11T22:50:09.262 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-11T22:50:09.263 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:50:09.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:09 smithi191 ceph-mon[35211]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi152=a;smithi152=d;smithi152=g;smithi152=j;smithi191=b;smithi191=e;smithi191=h;smithi191=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:50:09.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:09 smithi191 ceph-mon[35211]: Saving service mds.all spec with placement smithi152=a;smithi152=d;smithi152=g;smithi152=j;smithi191=b;smithi191=e;smithi191=h;smithi191=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-09-11T22:50:09.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:09.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:50:09.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:09.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:50:09.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:09.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:09.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:09.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:09 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:09.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:09 smithi191 ceph-mon[35211]: Deploying daemon mds.l on smithi204 2024-09-11T22:50:09.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:09 smithi191 ceph-mon[35211]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:09.320 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:09 smithi204 ceph-mon[35454]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi152=a;smithi152=d;smithi152=g;smithi152=j;smithi191=b;smithi191=e;smithi191=h;smithi191=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:50:09.320 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:09 smithi204 ceph-mon[35454]: Saving service mds.all spec with placement smithi152=a;smithi152=d;smithi152=g;smithi152=j;smithi191=b;smithi191=e;smithi191=h;smithi191=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-09-11T22:50:09.320 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:09.320 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:50:09.320 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:09.320 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:50:09.320 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:09.320 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:09.320 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:09.320 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:09 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:09.320 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:09 smithi204 ceph-mon[35454]: Deploying daemon mds.l on smithi204 2024-09-11T22:50:09.321 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:09 smithi204 ceph-mon[35454]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:09.505 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:09.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:09 smithi152 ceph-mon[31181]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi152=a;smithi152=d;smithi152=g;smithi152=j;smithi191=b;smithi191=e;smithi191=h;smithi191=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:50:09.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:09 smithi152 ceph-mon[31181]: Saving service mds.all spec with placement smithi152=a;smithi152=d;smithi152=g;smithi152=j;smithi191=b;smithi191=e;smithi191=h;smithi191=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-09-11T22:50:09.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:09.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:50:09.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:09.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:50:09.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:09.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:09.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:09.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:09 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:09.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:09 smithi152 ceph-mon[31181]: Deploying daemon mds.l on smithi204 2024-09-11T22:50:09.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:09 smithi152 ceph-mon[31181]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.991+0000 7fc63ba09640 1 -- 172.21.15.152:0/3693218674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc634106f70 msgr2=0x7fc63410eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.991+0000 7fc63ba09640 1 --2- 172.21.15.152:0/3693218674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc634106f70 0x7fc63410eab0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009ca0 tx=0x7fc62c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:50:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.991+0000 7fc63ba09640 1 -- 172.21.15.152:0/3693218674 shutdown_connections 2024-09-11T22:50:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.992+0000 7fc63ba09640 1 --2- 172.21.15.152:0/3693218674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc634106f70 0x7fc63410eab0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.992+0000 7fc63ba09640 1 --2- 172.21.15.152:0/3693218674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc634100eb0 0x7fc634106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.992+0000 7fc63ba09640 1 --2- 172.21.15.152:0/3693218674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc634100500 0x7fc6341008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.992+0000 7fc63ba09640 1 -- 172.21.15.152:0/3693218674 >> 172.21.15.152:0/3693218674 conn(0x7fc6340fc5c0 msgr2=0x7fc6340fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:10.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.992+0000 7fc63ba09640 1 -- 172.21.15.152:0/3693218674 shutdown_connections 2024-09-11T22:50:10.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.992+0000 7fc63ba09640 1 -- 172.21.15.152:0/3693218674 wait complete. 2024-09-11T22:50:10.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.992+0000 7fc63ba09640 1 Processor -- start 2024-09-11T22:50:10.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.992+0000 7fc63ba09640 1 -- start start 2024-09-11T22:50:10.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.992+0000 7fc63ba09640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc634100500 0x7fc634198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:10.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc63ba09640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc634100eb0 0x7fc6341989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:10.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc63ba09640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc634106f70 0x7fc63419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:10.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc63ba09640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc634111ff0 con 0x7fc634100eb0 2024-09-11T22:50:10.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc63ba09640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc634111e70 con 0x7fc634100500 2024-09-11T22:50:10.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc63ba09640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc634112170 con 0x7fc634106f70 2024-09-11T22:50:10.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc638fa4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc634100eb0 0x7fc6341989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:10.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc638fa4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc634100eb0 0x7fc6341989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50390/0 (socket says 172.21.15.152:50390) 2024-09-11T22:50:10.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc638fa4640 1 -- 172.21.15.152:0/2179262613 learned_addr learned my addr 172.21.15.152:0/2179262613 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:10.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc639fa6640 1 --2- 172.21.15.152:0/2179262613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc634106f70 0x7fc63419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:10.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc6397a5640 1 --2- 172.21.15.152:0/2179262613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc634100500 0x7fc634198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:10.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc638fa4640 1 -- 172.21.15.152:0/2179262613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc634106f70 msgr2=0x7fc63419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:10.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc638fa4640 1 --2- 172.21.15.152:0/2179262613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc634106f70 0x7fc63419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:10.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc638fa4640 1 -- 172.21.15.152:0/2179262613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc634100500 msgr2=0x7fc634198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:10.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc638fa4640 1 --2- 172.21.15.152:0/2179262613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc634100500 0x7fc634198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:10.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc638fa4640 1 -- 172.21.15.152:0/2179262613 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63419d420 con 0x7fc634100eb0 2024-09-11T22:50:10.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc638fa4640 1 --2- 172.21.15.152:0/2179262613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc634100eb0 0x7fc6341989b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc624006fd0 tx=0x7fc62400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:10.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc62a7fc640 1 -- 172.21.15.152:0/2179262613 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624011830 con 0x7fc634100eb0 2024-09-11T22:50:10.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc62a7fc640 1 -- 172.21.15.152:0/2179262613 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc624016070 con 0x7fc634100eb0 2024-09-11T22:50:10.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc63ba09640 1 -- 172.21.15.152:0/2179262613 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63419d710 con 0x7fc634100eb0 2024-09-11T22:50:10.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc62a7fc640 1 -- 172.21.15.152:0/2179262613 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62400f2e0 con 0x7fc634100eb0 2024-09-11T22:50:10.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.993+0000 7fc63ba09640 1 -- 172.21.15.152:0/2179262613 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6341a4ff0 con 0x7fc634100eb0 2024-09-11T22:50:10.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.994+0000 7fc63ba09640 1 -- 172.21.15.152:0/2179262613 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6341008e0 con 0x7fc634100eb0 2024-09-11T22:50:10.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.996+0000 7fc62a7fc640 1 -- 172.21.15.152:0/2179262613 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc6240119d0 con 0x7fc634100eb0 2024-09-11T22:50:10.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.996+0000 7fc62a7fc640 1 --2- 172.21.15.152:0/2179262613 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc6200a2b80 0x7fc6200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:10.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.996+0000 7fc6397a5640 1 --2- 172.21.15.152:0/2179262613 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc6200a2b80 0x7fc6200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:10.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.996+0000 7fc62a7fc640 1 -- 172.21.15.152:0/2179262613 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fc6240ce890 con 0x7fc634100eb0 2024-09-11T22:50:10.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.997+0000 7fc6397a5640 1 --2- 172.21.15.152:0/2179262613 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc6200a2b80 0x7fc6200a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc630005fd0 tx=0x7fc630005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:10.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:09.999+0000 7fc62a7fc640 1 -- 172.21.15.152:0/2179262613 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc62409bc90 con 0x7fc634100eb0 2024-09-11T22:50:10.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:10.198+0000 7fc63ba09640 1 -- 172.21.15.152:0/2179262613 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc63402d7c0 con 0x7fc634100eb0 2024-09-11T22:50:10.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:10.199+0000 7fc62a7fc640 1 -- 172.21.15.152:0/2179262613 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fc62409b590 con 0x7fc634100eb0 2024-09-11T22:50:10.199 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:50:10.200 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"btime":"2024-09-11T22:43:00:379132+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-11T22:50:10.200 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 1 2024-09-11T22:50:10.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:10.201+0000 7fc63ba09640 1 -- 172.21.15.152:0/2179262613 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc6200a2b80 msgr2=0x7fc6200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:10.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:10.201+0000 7fc63ba09640 1 --2- 172.21.15.152:0/2179262613 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc6200a2b80 0x7fc6200a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc630005fd0 tx=0x7fc630005850 comp rx=0 tx=0).stop 2024-09-11T22:50:10.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:10.201+0000 7fc63ba09640 1 -- 172.21.15.152:0/2179262613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc634100eb0 msgr2=0x7fc6341989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:10.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:10.201+0000 7fc63ba09640 1 --2- 172.21.15.152:0/2179262613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc634100eb0 0x7fc6341989b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc624006fd0 tx=0x7fc62400b8e0 comp rx=0 tx=0).stop 2024-09-11T22:50:10.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:10.201+0000 7fc63ba09640 1 -- 172.21.15.152:0/2179262613 shutdown_connections 2024-09-11T22:50:10.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:10.201+0000 7fc63ba09640 1 --2- 172.21.15.152:0/2179262613 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc6200a2b80 0x7fc6200a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:10.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:10.201+0000 7fc63ba09640 1 --2- 172.21.15.152:0/2179262613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc634106f70 0x7fc63419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:10.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:10.201+0000 7fc63ba09640 1 --2- 172.21.15.152:0/2179262613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc634100eb0 0x7fc6341989b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:10.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:10.201+0000 7fc63ba09640 1 --2- 172.21.15.152:0/2179262613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc634100500 0x7fc634198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:10.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:10.201+0000 7fc63ba09640 1 -- 172.21.15.152:0/2179262613 >> 172.21.15.152:0/2179262613 conn(0x7fc6340fc5c0 msgr2=0x7fc634105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:10.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:10.201+0000 7fc63ba09640 1 -- 172.21.15.152:0/2179262613 shutdown_connections 2024-09-11T22:50:10.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:10.201+0000 7fc63ba09640 1 -- 172.21.15.152:0/2179262613 wait complete. 2024-09-11T22:50:10.737 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:10 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2179262613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:10.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:10 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2179262613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:11.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:10 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2179262613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:11.296 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 11 22:50:11 smithi204 systemd[1]: Started Ceph mds.l for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:50:11.853 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:11 smithi191 ceph-mon[35211]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:11.853 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:11 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.204:6832/2200342090,v1:172.21.15.204:6833/2200342090] up:boot 2024-09-11T22:50:11.853 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:11 smithi191 ceph-mon[35211]: fsmap 1 up:standby 2024-09-11T22:50:11.853 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:11 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:50:11.853 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:11 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:11.853 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:11 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:11.853 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:11 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:11.854 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:11 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:11.854 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:11 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:11.854 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:11 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:11.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:11 smithi152 ceph-mon[31181]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:11.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:11 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.204:6832/2200342090,v1:172.21.15.204:6833/2200342090] up:boot 2024-09-11T22:50:11.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:11 smithi152 ceph-mon[31181]: fsmap 1 up:standby 2024-09-11T22:50:11.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:11 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:50:11.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:11 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:11.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:11 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:11.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:11 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:11.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:11 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:11.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:11 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:11.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:11 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:11 smithi204 ceph-mon[35454]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:11 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.204:6832/2200342090,v1:172.21.15.204:6833/2200342090] up:boot 2024-09-11T22:50:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:11 smithi204 ceph-mon[35454]: fsmap 1 up:standby 2024-09-11T22:50:12.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:11 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:50:12.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:11 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:12.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:11 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:12.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:11 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:12.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:11 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:12.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:11 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:12.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:11 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:12.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:12 smithi152 ceph-mon[31181]: Deploying daemon mds.k on smithi191 2024-09-11T22:50:12.995 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:12 smithi191 ceph-mon[35211]: Deploying daemon mds.k on smithi191 2024-09-11T22:50:13.018 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:50:13.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:12 smithi204 ceph-mon[35454]: Deploying daemon mds.k on smithi191 2024-09-11T22:50:13.274 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:13.287 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 22:50:13 smithi191 systemd[1]: Starting Ceph mds.k for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T22:50:13.680 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:13 smithi152 ceph-mon[31181]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:13.727 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:13 smithi191 ceph-mon[35211]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:13.728 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 22:50:13 smithi191 podman[64378]: 2024-09-11 22:50:13.291249327 +0000 UTC m=+0.015468839 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:50:13.728 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 22:50:13 smithi191 podman[64378]: 2024-09-11 22:50:13.415818058 +0000 UTC m=+0.140037572 container create c100af8ff245fc7ca2e650cfef1586a8d9a65eeaf1aebbb9c5f758902de2587c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-11T22:50:13.728 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 22:50:13 smithi191 podman[64378]: 2024-09-11 22:50:13.72783465 +0000 UTC m=+0.452054161 container init c100af8ff245fc7ca2e650cfef1586a8d9a65eeaf1aebbb9c5f758902de2587c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:50:13.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.759+0000 7f37c8fa2640 1 -- 172.21.15.152:0/1814849859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37c410bfc0 msgr2=0x7f37c410e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:13.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.759+0000 7f37c8fa2640 1 --2- 172.21.15.152:0/1814849859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37c410bfc0 0x7f37c410e460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009ca0 tx=0x7f37bc02a950 comp rx=0 tx=0).stop 2024-09-11T22:50:13.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.760+0000 7f37c8fa2640 1 -- 172.21.15.152:0/1814849859 shutdown_connections 2024-09-11T22:50:13.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.760+0000 7f37c8fa2640 1 --2- 172.21.15.152:0/1814849859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37c410bfc0 0x7f37c410e460 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:13.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.760+0000 7f37c8fa2640 1 --2- 172.21.15.152:0/1814849859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37c41034b0 0x7f37c410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:13.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.760+0000 7f37c8fa2640 1 --2- 172.21.15.152:0/1814849859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c4102b90 0x7f37c4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:13.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.760+0000 7f37c8fa2640 1 -- 172.21.15.152:0/1814849859 >> 172.21.15.152:0/1814849859 conn(0x7f37c40fc630 msgr2=0x7f37c40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:13.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.760+0000 7f37c8fa2640 1 -- 172.21.15.152:0/1814849859 shutdown_connections 2024-09-11T22:50:13.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.760+0000 7f37c8fa2640 1 -- 172.21.15.152:0/1814849859 wait complete. 2024-09-11T22:50:13.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.760+0000 7f37c8fa2640 1 Processor -- start 2024-09-11T22:50:13.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.760+0000 7f37c8fa2640 1 -- start start 2024-09-11T22:50:13.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.760+0000 7f37c8fa2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37c4102b90 0x7f37c4198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:13.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.760+0000 7f37c8fa2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c41034b0 0x7f37c41987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:13.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.760+0000 7f37c8fa2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37c410bfc0 0x7f37c419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:13.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.760+0000 7f37c8fa2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4111d90 con 0x7f37c410bfc0 2024-09-11T22:50:13.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.760+0000 7f37c8fa2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4111c10 con 0x7f37c4102b90 2024-09-11T22:50:13.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.760+0000 7f37c8fa2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4111f10 con 0x7f37c41034b0 2024-09-11T22:50:13.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.760+0000 7f37c3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37c410bfc0 0x7f37c419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:13.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.761+0000 7f37c3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37c410bfc0 0x7f37c419cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50418/0 (socket says 172.21.15.152:50418) 2024-09-11T22:50:13.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.761+0000 7f37c3fff640 1 -- 172.21.15.152:0/287004921 learned_addr learned my addr 172.21.15.152:0/287004921 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:13.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.761+0000 7f37c37fe640 1 --2- 172.21.15.152:0/287004921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37c4102b90 0x7f37c4198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:13.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.761+0000 7f37bbfff640 1 --2- 172.21.15.152:0/287004921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c41034b0 0x7f37c41987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:13.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.761+0000 7f37c3fff640 1 -- 172.21.15.152:0/287004921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c41034b0 msgr2=0x7f37c41987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:13.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.761+0000 7f37c3fff640 1 --2- 172.21.15.152:0/287004921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c41034b0 0x7f37c41987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:13.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.761+0000 7f37c3fff640 1 -- 172.21.15.152:0/287004921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37c4102b90 msgr2=0x7f37c4198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:13.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.761+0000 7f37c3fff640 1 --2- 172.21.15.152:0/287004921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37c4102b90 0x7f37c4198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:13.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.761+0000 7f37c3fff640 1 -- 172.21.15.152:0/287004921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c419d220 con 0x7f37c410bfc0 2024-09-11T22:50:13.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.761+0000 7f37c3fff640 1 --2- 172.21.15.152:0/287004921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37c410bfc0 0x7f37c419cb40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f37bc004af0 tx=0x7f37bc02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:13.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.761+0000 7f37c17fa640 1 -- 172.21.15.152:0/287004921 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc034040 con 0x7f37c410bfc0 2024-09-11T22:50:13.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.761+0000 7f37c17fa640 1 -- 172.21.15.152:0/287004921 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37bc030920 con 0x7f37c410bfc0 2024-09-11T22:50:13.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.761+0000 7f37c8fa2640 1 -- 172.21.15.152:0/287004921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c419d4b0 con 0x7f37c410bfc0 2024-09-11T22:50:13.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.761+0000 7f37c17fa640 1 -- 172.21.15.152:0/287004921 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0047b0 con 0x7f37c410bfc0 2024-09-11T22:50:13.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.761+0000 7f37c8fa2640 1 -- 172.21.15.152:0/287004921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c41a4df0 con 0x7f37c410bfc0 2024-09-11T22:50:13.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.762+0000 7f37c8fa2640 1 -- 172.21.15.152:0/287004921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c406b070 con 0x7f37c410bfc0 2024-09-11T22:50:13.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.763+0000 7f37c17fa640 1 -- 172.21.15.152:0/287004921 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f37bc037020 con 0x7f37c410bfc0 2024-09-11T22:50:13.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.764+0000 7f37c17fa640 1 --2- 172.21.15.152:0/287004921 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f37b00a2b80 0x7f37b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:13.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.764+0000 7f37c37fe640 1 --2- 172.21.15.152:0/287004921 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f37b00a2b80 0x7f37b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:13.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.764+0000 7f37c17fa640 1 -- 172.21.15.152:0/287004921 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f37bc0efb20 con 0x7f37c410bfc0 2024-09-11T22:50:13.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.765+0000 7f37c37fe640 1 --2- 172.21.15.152:0/287004921 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f37b00a2b80 0x7f37b00a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f37b4005fd0 tx=0x7f37b400b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:13.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.768+0000 7f37c17fa640 1 -- 172.21.15.152:0/287004921 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37bc0bcf00 con 0x7f37c410bfc0 2024-09-11T22:50:13.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.974+0000 7f37c8fa2640 1 -- 172.21.15.152:0/287004921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37c4102f70 con 0x7f37c410bfc0 2024-09-11T22:50:13.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.975+0000 7f37c17fa640 1 -- 172.21.15.152:0/287004921 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f37bc0bc800 con 0x7f37c410bfc0 2024-09-11T22:50:13.975 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:50:13.975 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":2,"btime":"2024-09-11T22:50:11:044219+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T22:50:13.976 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 2 2024-09-11T22:50:13.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.977+0000 7f37c8fa2640 1 -- 172.21.15.152:0/287004921 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f37b00a2b80 msgr2=0x7f37b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:13.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.977+0000 7f37c8fa2640 1 --2- 172.21.15.152:0/287004921 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f37b00a2b80 0x7f37b00a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f37b4005fd0 tx=0x7f37b400b590 comp rx=0 tx=0).stop 2024-09-11T22:50:13.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.977+0000 7f37c8fa2640 1 -- 172.21.15.152:0/287004921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37c410bfc0 msgr2=0x7f37c419cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:13.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.977+0000 7f37c8fa2640 1 --2- 172.21.15.152:0/287004921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37c410bfc0 0x7f37c419cb40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f37bc004af0 tx=0x7f37bc02f5b0 comp rx=0 tx=0).stop 2024-09-11T22:50:13.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.977+0000 7f37c8fa2640 1 -- 172.21.15.152:0/287004921 shutdown_connections 2024-09-11T22:50:13.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.977+0000 7f37c8fa2640 1 --2- 172.21.15.152:0/287004921 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f37b00a2b80 0x7f37b00a5040 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:13.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.977+0000 7f37c8fa2640 1 --2- 172.21.15.152:0/287004921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37c410bfc0 0x7f37c419cb40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:13.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.977+0000 7f37c8fa2640 1 --2- 172.21.15.152:0/287004921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c41034b0 0x7f37c41987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:13.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.977+0000 7f37c8fa2640 1 --2- 172.21.15.152:0/287004921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37c4102b90 0x7f37c4198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:13.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.977+0000 7f37c8fa2640 1 -- 172.21.15.152:0/287004921 >> 172.21.15.152:0/287004921 conn(0x7f37c40fc630 msgr2=0x7f37c40fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:13.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.977+0000 7f37c8fa2640 1 -- 172.21.15.152:0/287004921 shutdown_connections 2024-09-11T22:50:13.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:13.977+0000 7f37c8fa2640 1 -- 172.21.15.152:0/287004921 wait complete. 2024-09-11T22:50:14.037 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 22:50:13 smithi191 podman[64378]: 2024-09-11 22:50:13.730873328 +0000 UTC m=+0.455092840 container start c100af8ff245fc7ca2e650cfef1586a8d9a65eeaf1aebbb9c5f758902de2587c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T22:50:14.037 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 22:50:13 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-k[64393]: starting mds.k at 2024-09-11T22:50:14.037 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 22:50:13 smithi191 bash[64378]: c100af8ff245fc7ca2e650cfef1586a8d9a65eeaf1aebbb9c5f758902de2587c 2024-09-11T22:50:14.037 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 22:50:13 smithi191 systemd[1]: Started Ceph mds.k for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:50:14.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:13 smithi204 ceph-mon[35454]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:15.082 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:14 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:15.082 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:14 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:15.082 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:14 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:15.082 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:14 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:15.082 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:14 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:15.082 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:14 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:15.082 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:14 smithi152 ceph-mon[31181]: Deploying daemon mds.j on smithi152 2024-09-11T22:50:15.082 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:14 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/287004921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:15.083 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:14 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.191:6832/2321157769,v1:172.21.15.191:6833/2321157769] up:boot 2024-09-11T22:50:15.083 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:14 smithi152 ceph-mon[31181]: fsmap 2 up:standby 2024-09-11T22:50:15.083 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:14 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:50:15.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:14 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:15.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:14 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:15.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:14 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:15.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:14 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:15.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:14 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:15.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:14 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:15.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:14 smithi191 ceph-mon[35211]: Deploying daemon mds.j on smithi152 2024-09-11T22:50:15.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:14 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/287004921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:15.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:14 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.191:6832/2321157769,v1:172.21.15.191:6833/2321157769] up:boot 2024-09-11T22:50:15.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:14 smithi191 ceph-mon[35211]: fsmap 2 up:standby 2024-09-11T22:50:15.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:14 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:50:15.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:14 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:15.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:14 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:15.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:14 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:15.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:14 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:15.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:14 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:15.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:14 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:15.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:14 smithi204 ceph-mon[35454]: Deploying daemon mds.j on smithi152 2024-09-11T22:50:15.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:14 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/287004921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:15.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:14 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.191:6832/2321157769,v1:172.21.15.191:6833/2321157769] up:boot 2024-09-11T22:50:15.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:14 smithi204 ceph-mon[35454]: fsmap 2 up:standby 2024-09-11T22:50:15.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:14 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:50:16.086 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:15 smithi152 ceph-mon[31181]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:16.158 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:15 smithi191 ceph-mon[35211]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:16.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:15 smithi204 ceph-mon[35454]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:16.762 INFO:journalctl@ceph.mds.j.smithi152.stdout:Sep 11 22:50:16 smithi152 systemd[1]: Starting Ceph mds.j for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T22:50:16.798 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:50:17.041 INFO:journalctl@ceph.mds.j.smithi152.stdout:Sep 11 22:50:16 smithi152 podman[68419]: 2024-09-11 22:50:16.664359854 +0000 UTC m=+0.016186512 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:50:17.041 INFO:journalctl@ceph.mds.j.smithi152.stdout:Sep 11 22:50:16 smithi152 podman[68419]: 2024-09-11 22:50:16.796845043 +0000 UTC m=+0.148671699 container create 829e5f06a64d2a51f3cc45318bcbfefc973f0c773f1ef16965192671a0786b8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T22:50:17.041 INFO:journalctl@ceph.mds.j.smithi152.stdout:Sep 11 22:50:17 smithi152 podman[68419]: 2024-09-11 22:50:17.041216665 +0000 UTC m=+0.393043322 container init 829e5f06a64d2a51f3cc45318bcbfefc973f0c773f1ef16965192671a0786b8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-j, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T22:50:17.190 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:17.315 INFO:journalctl@ceph.mds.j.smithi152.stdout:Sep 11 22:50:17 smithi152 podman[68419]: 2024-09-11 22:50:17.045700719 +0000 UTC m=+0.397527375 container start 829e5f06a64d2a51f3cc45318bcbfefc973f0c773f1ef16965192671a0786b8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T22:50:17.315 INFO:journalctl@ceph.mds.j.smithi152.stdout:Sep 11 22:50:17 smithi152 ceph-mds[68467]: -- 172.21.15.152:0/1186523109 <== mon.0 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557294cad860 con 0x557294c8b000 2024-09-11T22:50:17.315 INFO:journalctl@ceph.mds.j.smithi152.stdout:Sep 11 22:50:17 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-j[68452]: starting mds.j at 2024-09-11T22:50:17.316 INFO:journalctl@ceph.mds.j.smithi152.stdout:Sep 11 22:50:17 smithi152 bash[68419]: 829e5f06a64d2a51f3cc45318bcbfefc973f0c773f1ef16965192671a0786b8e 2024-09-11T22:50:17.316 INFO:journalctl@ceph.mds.j.smithi152.stdout:Sep 11 22:50:17 smithi152 systemd[1]: Started Ceph mds.j for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:50:17.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.723+0000 7f1af9e27640 1 -- 172.21.15.152:0/2468626366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af4106a80 msgr2=0x7f1af4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:17.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.723+0000 7f1af9e27640 1 --2- 172.21.15.152:0/2468626366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af4106a80 0x7f1af4106e60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009930 tx=0x7f1ae402a420 comp rx=0 tx=0).stop 2024-09-11T22:50:17.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.723+0000 7f1af9e27640 1 -- 172.21.15.152:0/2468626366 shutdown_connections 2024-09-11T22:50:17.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.723+0000 7f1af9e27640 1 --2- 172.21.15.152:0/2468626366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1af4101420 0x7f1af410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:17.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.723+0000 7f1af9e27640 1 --2- 172.21.15.152:0/2468626366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1af4100a80 0x7f1af4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:17.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.723+0000 7f1af9e27640 1 --2- 172.21.15.152:0/2468626366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af4106a80 0x7f1af4106e60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:17.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.723+0000 7f1af9e27640 1 -- 172.21.15.152:0/2468626366 >> 172.21.15.152:0/2468626366 conn(0x7f1af40fc7a0 msgr2=0x7f1af40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:17.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.723+0000 7f1af9e27640 1 -- 172.21.15.152:0/2468626366 shutdown_connections 2024-09-11T22:50:17.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.723+0000 7f1af9e27640 1 -- 172.21.15.152:0/2468626366 wait complete. 2024-09-11T22:50:17.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.724+0000 7f1af9e27640 1 Processor -- start 2024-09-11T22:50:17.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.724+0000 7f1af9e27640 1 -- start start 2024-09-11T22:50:17.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.724+0000 7f1af9e27640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af4100a80 0x7f1af4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:17.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.724+0000 7f1af9e27640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1af4101420 0x7f1af41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:17.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.724+0000 7f1af9e27640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1af4106a80 0x7f1af419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:17.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af9e27640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1af4110e80 con 0x7f1af4100a80 2024-09-11T22:50:17.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af9e27640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1af4110d00 con 0x7f1af4101420 2024-09-11T22:50:17.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af9e27640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1af4111000 con 0x7f1af4106a80 2024-09-11T22:50:17.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af4100a80 0x7f1af4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:17.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af4100a80 0x7f1af4198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50460/0 (socket says 172.21.15.152:50460) 2024-09-11T22:50:17.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af37fe640 1 -- 172.21.15.152:0/797461108 learned_addr learned my addr 172.21.15.152:0/797461108 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:17.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af2ffd640 1 --2- 172.21.15.152:0/797461108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1af4101420 0x7f1af41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:17.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af3fff640 1 --2- 172.21.15.152:0/797461108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1af4106a80 0x7f1af419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:17.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af37fe640 1 -- 172.21.15.152:0/797461108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1af4106a80 msgr2=0x7f1af419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:17.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af37fe640 1 --2- 172.21.15.152:0/797461108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1af4106a80 0x7f1af419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:17.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af37fe640 1 -- 172.21.15.152:0/797461108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1af4101420 msgr2=0x7f1af41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:17.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af37fe640 1 --2- 172.21.15.152:0/797461108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1af4101420 0x7f1af41989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:17.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af37fe640 1 -- 172.21.15.152:0/797461108 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af419d430 con 0x7f1af4100a80 2024-09-11T22:50:17.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af37fe640 1 --2- 172.21.15.152:0/797461108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af4100a80 0x7f1af4198480 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009230 tx=0x7f1ae4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:17.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af0ff9640 1 -- 172.21.15.152:0/797461108 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae403f030 con 0x7f1af4100a80 2024-09-11T22:50:17.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af9e27640 1 -- 172.21.15.152:0/797461108 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af419d6c0 con 0x7f1af4100a80 2024-09-11T22:50:17.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af0ff9640 1 -- 172.21.15.152:0/797461108 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ae4003ad0 con 0x7f1af4100a80 2024-09-11T22:50:17.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af0ff9640 1 -- 172.21.15.152:0/797461108 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4035070 con 0x7f1af4100a80 2024-09-11T22:50:17.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.725+0000 7f1af9e27640 1 -- 172.21.15.152:0/797461108 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af41a5000 con 0x7f1af4100a80 2024-09-11T22:50:17.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.726+0000 7f1af9e27640 1 -- 172.21.15.152:0/797461108 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af4100ee0 con 0x7f1af4100a80 2024-09-11T22:50:17.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.728+0000 7f1af0ff9640 1 -- 172.21.15.152:0/797461108 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1ae4042030 con 0x7f1af4100a80 2024-09-11T22:50:17.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.728+0000 7f1af0ff9640 1 --2- 172.21.15.152:0/797461108 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1adc0a2b80 0x7f1adc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:17.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.728+0000 7f1af2ffd640 1 --2- 172.21.15.152:0/797461108 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1adc0a2b80 0x7f1adc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:17.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.728+0000 7f1af0ff9640 1 -- 172.21.15.152:0/797461108 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f1ae40ef4d0 con 0x7f1af4100a80 2024-09-11T22:50:17.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.729+0000 7f1af2ffd640 1 --2- 172.21.15.152:0/797461108 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1adc0a2b80 0x7f1adc0a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1af406b870 tx=0x7f1ae800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:17.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.731+0000 7f1af0ff9640 1 -- 172.21.15.152:0/797461108 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ae40bc8d0 con 0x7f1af4100a80 2024-09-11T22:50:17.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.943+0000 7f1af9e27640 1 -- 172.21.15.152:0/797461108 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1af41a51d0 con 0x7f1af4100a80 2024-09-11T22:50:17.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.944+0000 7f1af0ff9640 1 -- 172.21.15.152:0/797461108 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3094 (secure 0 0 0) 0x7f1ae40bc1d0 con 0x7f1af4100a80 2024-09-11T22:50:17.944 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:50:17.945 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":4,"btime":"2024-09-11T22:50:17:893068+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T22:50:17.945 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 4 2024-09-11T22:50:17.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.946+0000 7f1af9e27640 1 -- 172.21.15.152:0/797461108 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1adc0a2b80 msgr2=0x7f1adc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:17.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.946+0000 7f1af9e27640 1 --2- 172.21.15.152:0/797461108 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1adc0a2b80 0x7f1adc0a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1af406b870 tx=0x7f1ae800a250 comp rx=0 tx=0).stop 2024-09-11T22:50:17.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.946+0000 7f1af9e27640 1 -- 172.21.15.152:0/797461108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af4100a80 msgr2=0x7f1af4198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:17.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.946+0000 7f1af9e27640 1 --2- 172.21.15.152:0/797461108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af4100a80 0x7f1af4198480 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009230 tx=0x7f1ae4004270 comp rx=0 tx=0).stop 2024-09-11T22:50:17.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.946+0000 7f1af9e27640 1 -- 172.21.15.152:0/797461108 shutdown_connections 2024-09-11T22:50:17.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.946+0000 7f1af9e27640 1 --2- 172.21.15.152:0/797461108 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1adc0a2b80 0x7f1adc0a5040 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:17.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.946+0000 7f1af9e27640 1 --2- 172.21.15.152:0/797461108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1af4106a80 0x7f1af419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:17.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.946+0000 7f1af9e27640 1 --2- 172.21.15.152:0/797461108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1af4101420 0x7f1af41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:17.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.946+0000 7f1af9e27640 1 --2- 172.21.15.152:0/797461108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1af4100a80 0x7f1af4198480 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:17.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.946+0000 7f1af9e27640 1 -- 172.21.15.152:0/797461108 >> 172.21.15.152:0/797461108 conn(0x7f1af40fc7a0 msgr2=0x7f1af41048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:17.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.946+0000 7f1af9e27640 1 -- 172.21.15.152:0/797461108 shutdown_connections 2024-09-11T22:50:17.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:17.946+0000 7f1af9e27640 1 -- 172.21.15.152:0/797461108 wait complete. 2024-09-11T22:50:18.159 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:17 smithi152 ceph-mon[31181]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:18.159 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:18.159 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:18.159 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:18.159 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:18.159 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:18.159 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:17 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:18.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:17 smithi191 ceph-mon[35211]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:18.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:18.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:18.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:18.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:18.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:18.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:17 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:18.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:17 smithi204 ceph-mon[35454]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:18.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:18.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:18.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:18.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:18.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:18.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:17 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:19.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:18 smithi152 ceph-mon[31181]: Deploying daemon mds.d on smithi152 2024-09-11T22:50:19.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:18 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.152:6834/1098962542,v1:172.21.15.152:6835/1098962542] up:boot 2024-09-11T22:50:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:18 smithi152 ceph-mon[31181]: fsmap 3 up:standby 2024-09-11T22:50:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:18 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T22:50:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:18 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/797461108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:19.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:18 smithi191 ceph-mon[35211]: Deploying daemon mds.d on smithi152 2024-09-11T22:50:19.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:18 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.152:6834/1098962542,v1:172.21.15.152:6835/1098962542] up:boot 2024-09-11T22:50:19.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:18 smithi191 ceph-mon[35211]: fsmap 3 up:standby 2024-09-11T22:50:19.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:18 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T22:50:19.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:18 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/797461108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:19.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:18 smithi204 ceph-mon[35454]: Deploying daemon mds.d on smithi152 2024-09-11T22:50:19.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:18 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.152:6834/1098962542,v1:172.21.15.152:6835/1098962542] up:boot 2024-09-11T22:50:19.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:18 smithi204 ceph-mon[35454]: fsmap 3 up:standby 2024-09-11T22:50:19.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:18 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T22:50:19.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:18 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/797461108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:19.873 INFO:journalctl@ceph.mds.d.smithi152.stdout:Sep 11 22:50:19 smithi152 systemd[1]: Starting Ceph mds.d for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T22:50:20.156 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:19 smithi152 ceph-mon[31181]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:20.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:19 smithi191 ceph-mon[35211]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:20.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:19 smithi204 ceph-mon[35454]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:20.456 INFO:journalctl@ceph.mds.d.smithi152.stdout:Sep 11 22:50:20 smithi152 podman[68827]: 2024-09-11 22:50:20.058927116 +0000 UTC m=+0.016317714 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:50:20.456 INFO:journalctl@ceph.mds.d.smithi152.stdout:Sep 11 22:50:20 smithi152 podman[68827]: 2024-09-11 22:50:20.186995835 +0000 UTC m=+0.144386416 container create f8beebbb7bf485156b398ebbe61ae863f3635fcf1b886435124becc4af18679d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-11T22:50:20.457 INFO:journalctl@ceph.mds.d.smithi152.stdout:Sep 11 22:50:20 smithi152 podman[68827]: 2024-09-11 22:50:20.456332047 +0000 UTC m=+0.413722631 container init f8beebbb7bf485156b398ebbe61ae863f3635fcf1b886435124becc4af18679d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-d, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-11T22:50:20.872 INFO:journalctl@ceph.mds.d.smithi152.stdout:Sep 11 22:50:20 smithi152 podman[68827]: 2024-09-11 22:50:20.459172842 +0000 UTC m=+0.416563434 container start f8beebbb7bf485156b398ebbe61ae863f3635fcf1b886435124becc4af18679d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-d, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:50:20.873 INFO:journalctl@ceph.mds.d.smithi152.stdout:Sep 11 22:50:20 smithi152 ceph-mds[68846]: -- 172.21.15.152:0/3447273072 <== mon.0 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5583f8a01860 con 0x5583f89df400 2024-09-11T22:50:20.873 INFO:journalctl@ceph.mds.d.smithi152.stdout:Sep 11 22:50:20 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-d[68842]: starting mds.d at 2024-09-11T22:50:20.873 INFO:journalctl@ceph.mds.d.smithi152.stdout:Sep 11 22:50:20 smithi152 bash[68827]: f8beebbb7bf485156b398ebbe61ae863f3635fcf1b886435124becc4af18679d 2024-09-11T22:50:20.873 INFO:journalctl@ceph.mds.d.smithi152.stdout:Sep 11 22:50:20 smithi152 systemd[1]: Started Ceph mds.d for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:50:21.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:21.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:21.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:21.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:21.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:21.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:21.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:21 smithi152 ceph-mon[31181]: Deploying daemon mds.e on smithi191 2024-09-11T22:50:21.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:21 smithi152 ceph-mon[31181]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:21.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:21 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.152:6836/2615590856,v1:172.21.15.152:6837/2615590856] up:boot 2024-09-11T22:50:21.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:21 smithi152 ceph-mon[31181]: fsmap 4 up:standby 2024-09-11T22:50:21.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:21 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T22:50:21.973 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:50:22.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:21 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:22.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:21 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:22.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:21 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:22.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:21 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:22.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:21 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:22.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:21 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:22.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:21 smithi191 ceph-mon[35211]: Deploying daemon mds.e on smithi191 2024-09-11T22:50:22.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:21 smithi191 ceph-mon[35211]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:22.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:21 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.152:6836/2615590856,v1:172.21.15.152:6837/2615590856] up:boot 2024-09-11T22:50:22.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:21 smithi191 ceph-mon[35211]: fsmap 4 up:standby 2024-09-11T22:50:22.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:21 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T22:50:22.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:21 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:22.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:21 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:22.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:21 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:22.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:21 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:22.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:21 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:22.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:21 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:22.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:21 smithi204 ceph-mon[35454]: Deploying daemon mds.e on smithi191 2024-09-11T22:50:22.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:21 smithi204 ceph-mon[35454]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:22.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:21 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.152:6836/2615590856,v1:172.21.15.152:6837/2615590856] up:boot 2024-09-11T22:50:22.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:21 smithi204 ceph-mon[35454]: fsmap 4 up:standby 2024-09-11T22:50:22.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:21 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T22:50:22.231 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:22.788 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 22:50:22 smithi191 systemd[1]: Starting Ceph mds.e for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T22:50:22.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.767+0000 7eff33b53640 1 -- 172.21.15.152:0/50549668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff2c102570 msgr2=0x7eff2c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:22.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.767+0000 7eff33b53640 1 --2- 172.21.15.152:0/50549668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff2c102570 0x7eff2c1029b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009900 tx=0x7eff1c02a480 comp rx=0 tx=0).stop 2024-09-11T22:50:22.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.767+0000 7eff33b53640 1 -- 172.21.15.152:0/50549668 shutdown_connections 2024-09-11T22:50:22.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.767+0000 7eff33b53640 1 --2- 172.21.15.152:0/50549668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff2c102ef0 0x7eff2c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:22.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.767+0000 7eff33b53640 1 --2- 172.21.15.152:0/50549668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff2c102570 0x7eff2c1029b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:22.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.767+0000 7eff33b53640 1 --2- 172.21.15.152:0/50549668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff2c100e20 0x7eff2c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:22.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.767+0000 7eff33b53640 1 -- 172.21.15.152:0/50549668 >> 172.21.15.152:0/50549668 conn(0x7eff2c0fc7d0 msgr2=0x7eff2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:22.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.767+0000 7eff33b53640 1 -- 172.21.15.152:0/50549668 shutdown_connections 2024-09-11T22:50:22.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.767+0000 7eff33b53640 1 -- 172.21.15.152:0/50549668 wait complete. 2024-09-11T22:50:22.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff33b53640 1 Processor -- start 2024-09-11T22:50:22.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff33b53640 1 -- start start 2024-09-11T22:50:22.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff33b53640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff2c100e20 0x7eff2c197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:22.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff33b53640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff2c102570 0x7eff2c197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:22.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff33b53640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff2c102ef0 0x7eff2c19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:22.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff33b53640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c110f50 con 0x7eff2c102570 2024-09-11T22:50:22.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff33b53640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c110dd0 con 0x7eff2c102ef0 2024-09-11T22:50:22.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff33b53640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7eff2c1110d0 con 0x7eff2c100e20 2024-09-11T22:50:22.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff310ee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff2c102570 0x7eff2c197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:22.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff310ee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff2c102570 0x7eff2c197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40802/0 (socket says 172.21.15.152:40802) 2024-09-11T22:50:22.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff310ee640 1 -- 172.21.15.152:0/1666144326 learned_addr learned my addr 172.21.15.152:0/1666144326 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:22.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff318ef640 1 --2- 172.21.15.152:0/1666144326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff2c100e20 0x7eff2c197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:22.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff320f0640 1 --2- 172.21.15.152:0/1666144326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff2c102ef0 0x7eff2c19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:22.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff310ee640 1 -- 172.21.15.152:0/1666144326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff2c100e20 msgr2=0x7eff2c197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:22.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff310ee640 1 --2- 172.21.15.152:0/1666144326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff2c100e20 0x7eff2c197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:22.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff310ee640 1 -- 172.21.15.152:0/1666144326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff2c102ef0 msgr2=0x7eff2c19bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:22.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff310ee640 1 --2- 172.21.15.152:0/1666144326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff2c102ef0 0x7eff2c19bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:22.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.768+0000 7eff310ee640 1 -- 172.21.15.152:0/1666144326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2c19c400 con 0x7eff2c102570 2024-09-11T22:50:22.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.769+0000 7eff310ee640 1 --2- 172.21.15.152:0/1666144326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff2c102570 0x7eff2c197990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7eff1c006010 tx=0x7eff1c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.769+0000 7eff22ffd640 1 -- 172.21.15.152:0/1666144326 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c03f070 con 0x7eff2c102570 2024-09-11T22:50:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.769+0000 7eff33b53640 1 -- 172.21.15.152:0/1666144326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff2c19c690 con 0x7eff2c102570 2024-09-11T22:50:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.769+0000 7eff22ffd640 1 -- 172.21.15.152:0/1666144326 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff1c031c30 con 0x7eff2c102570 2024-09-11T22:50:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.769+0000 7eff22ffd640 1 -- 172.21.15.152:0/1666144326 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1c035030 con 0x7eff2c102570 2024-09-11T22:50:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.769+0000 7eff33b53640 1 -- 172.21.15.152:0/1666144326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff2c1a3fd0 con 0x7eff2c102570 2024-09-11T22:50:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.770+0000 7eff33b53640 1 -- 172.21.15.152:0/1666144326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff2c101200 con 0x7eff2c102570 2024-09-11T22:50:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.771+0000 7eff22ffd640 1 -- 172.21.15.152:0/1666144326 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7eff1c042070 con 0x7eff2c102570 2024-09-11T22:50:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.772+0000 7eff22ffd640 1 --2- 172.21.15.152:0/1666144326 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7eff180a2b80 0x7eff180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.772+0000 7eff22ffd640 1 -- 172.21.15.152:0/1666144326 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7eff1c0ef360 con 0x7eff2c102570 2024-09-11T22:50:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.772+0000 7eff318ef640 1 --2- 172.21.15.152:0/1666144326 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7eff180a2b80 0x7eff180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.772+0000 7eff318ef640 1 --2- 172.21.15.152:0/1666144326 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7eff180a2b80 0x7eff180a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7eff280058c0 tx=0x7eff2800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:22.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.776+0000 7eff22ffd640 1 -- 172.21.15.152:0/1666144326 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7eff1c031de0 con 0x7eff2c102570 2024-09-11T22:50:22.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.986+0000 7eff33b53640 1 -- 172.21.15.152:0/1666144326 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff2c108b00 con 0x7eff2c102570 2024-09-11T22:50:22.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.986+0000 7eff22ffd640 1 -- 172.21.15.152:0/1666144326 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3922 (secure 0 0 0) 0x7eff1c0bc110 con 0x7eff2c102570 2024-09-11T22:50:22.987 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:50:22.987 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":5,"btime":"2024-09-11T22:50:20:903983+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T22:50:22.987 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 5 2024-09-11T22:50:22.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.988+0000 7eff33b53640 1 -- 172.21.15.152:0/1666144326 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7eff180a2b80 msgr2=0x7eff180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:22.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.988+0000 7eff33b53640 1 --2- 172.21.15.152:0/1666144326 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7eff180a2b80 0x7eff180a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7eff280058c0 tx=0x7eff2800a2d0 comp rx=0 tx=0).stop 2024-09-11T22:50:22.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.988+0000 7eff33b53640 1 -- 172.21.15.152:0/1666144326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff2c102570 msgr2=0x7eff2c197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:22.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.988+0000 7eff33b53640 1 --2- 172.21.15.152:0/1666144326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff2c102570 0x7eff2c197990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7eff1c006010 tx=0x7eff1c003c10 comp rx=0 tx=0).stop 2024-09-11T22:50:22.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.988+0000 7eff33b53640 1 -- 172.21.15.152:0/1666144326 shutdown_connections 2024-09-11T22:50:22.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.988+0000 7eff33b53640 1 --2- 172.21.15.152:0/1666144326 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7eff180a2b80 0x7eff180a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:22.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.988+0000 7eff33b53640 1 --2- 172.21.15.152:0/1666144326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff2c102ef0 0x7eff2c19bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:22.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.988+0000 7eff33b53640 1 --2- 172.21.15.152:0/1666144326 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff2c102570 0x7eff2c197990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:22.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.988+0000 7eff33b53640 1 --2- 172.21.15.152:0/1666144326 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff2c100e20 0x7eff2c197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:22.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.988+0000 7eff33b53640 1 -- 172.21.15.152:0/1666144326 >> 172.21.15.152:0/1666144326 conn(0x7eff2c0fc7d0 msgr2=0x7eff2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:22.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.989+0000 7eff33b53640 1 -- 172.21.15.152:0/1666144326 shutdown_connections 2024-09-11T22:50:22.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:22.989+0000 7eff33b53640 1 -- 172.21.15.152:0/1666144326 wait complete. 2024-09-11T22:50:23.133 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 22:50:22 smithi191 podman[64690]: 2024-09-11 22:50:22.768912286 +0000 UTC m=+0.015938039 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:50:23.133 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 22:50:22 smithi191 podman[64690]: 2024-09-11 22:50:22.895718759 +0000 UTC m=+0.142744540 container create 4f2b9716349a5467093a94829eaa05189479205786dcf3f6360f7b04df45a94c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T22:50:23.133 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 22:50:23 smithi191 podman[64690]: 2024-09-11 22:50:23.132802809 +0000 UTC m=+0.379828547 container init 4f2b9716349a5467093a94829eaa05189479205786dcf3f6360f7b04df45a94c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-09-11T22:50:23.537 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 22:50:23 smithi191 podman[64690]: 2024-09-11 22:50:23.135888095 +0000 UTC m=+0.382913842 container start 4f2b9716349a5467093a94829eaa05189479205786dcf3f6360f7b04df45a94c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T22:50:23.537 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 22:50:23 smithi191 ceph-mds[64709]: -- 172.21.15.191:0/3099802207 <== mon.2 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b05f95dc20 con 0x55b05f93a800 2024-09-11T22:50:23.537 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 22:50:23 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-e[64705]: starting mds.e at 2024-09-11T22:50:23.537 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 22:50:23 smithi191 bash[64690]: 4f2b9716349a5467093a94829eaa05189479205786dcf3f6360f7b04df45a94c 2024-09-11T22:50:23.537 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 22:50:23 smithi191 systemd[1]: Started Ceph mds.e for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:50:23.978 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:23 smithi204 ceph-mon[35454]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:23.979 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:23 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1666144326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:23.979 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:23 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:23.979 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:23 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:23.979 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:23 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:23.980 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:23 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:23.980 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:23 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:23.980 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:23 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:24.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:23 smithi152 ceph-mon[31181]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:24.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:23 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1666144326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:24.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:23 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:24.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:23 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:24.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:23 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:24.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:23 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:24.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:23 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:24.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:23 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:24.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:23 smithi191 ceph-mon[35211]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:24.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:23 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1666144326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:24.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:23 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:24.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:23 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:24.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:23 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:24.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:23 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:24.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:23 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:24.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:23 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:25.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:24 smithi204 ceph-mon[35454]: Deploying daemon mds.f on smithi204 2024-09-11T22:50:25.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:24 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.191:6834/2471607007,v1:172.21.15.191:6835/2471607007] up:boot 2024-09-11T22:50:25.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:24 smithi204 ceph-mon[35454]: fsmap 5 up:standby 2024-09-11T22:50:25.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:24 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:50:25.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:24 smithi152 ceph-mon[31181]: Deploying daemon mds.f on smithi204 2024-09-11T22:50:25.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:24 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.191:6834/2471607007,v1:172.21.15.191:6835/2471607007] up:boot 2024-09-11T22:50:25.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:24 smithi152 ceph-mon[31181]: fsmap 5 up:standby 2024-09-11T22:50:25.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:24 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:50:25.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:24 smithi191 ceph-mon[35211]: Deploying daemon mds.f on smithi204 2024-09-11T22:50:25.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:24 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.191:6834/2471607007,v1:172.21.15.191:6835/2471607007] up:boot 2024-09-11T22:50:25.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:24 smithi191 ceph-mon[35211]: fsmap 5 up:standby 2024-09-11T22:50:25.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:24 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:50:25.296 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 22:50:25 smithi204 systemd[1]: Starting Ceph mds.f for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T22:50:25.702 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 22:50:25 smithi204 podman[65485]: 2024-09-11 22:50:25.440759156 +0000 UTC m=+0.111535501 container create 57177cca0c8d4135367c3ab1fce8e7591499c9ea6103cf44b69233724656e236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-f, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:50:25.702 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 22:50:25 smithi204 podman[65485]: 2024-09-11 22:50:25.345138527 +0000 UTC m=+0.015914885 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:50:25.702 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 22:50:25 smithi204 podman[65485]: 2024-09-11 22:50:25.702261244 +0000 UTC m=+0.373037586 container init 57177cca0c8d4135367c3ab1fce8e7591499c9ea6103cf44b69233724656e236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T22:50:25.709 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:50:25.977 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:26.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:25 smithi204 ceph-mon[35454]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:26.046 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 22:50:25 smithi204 podman[65485]: 2024-09-11 22:50:25.705364584 +0000 UTC m=+0.376140927 container start 57177cca0c8d4135367c3ab1fce8e7591499c9ea6103cf44b69233724656e236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-11T22:50:26.046 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 22:50:25 smithi204 ceph-mds[65505]: -- 172.21.15.204:0/603150635 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a1bb1ff860 con 0x55a1bb1dc800 2024-09-11T22:50:26.047 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 22:50:25 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-f[65501]: starting mds.f at 2024-09-11T22:50:26.047 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 22:50:25 smithi204 bash[65485]: 57177cca0c8d4135367c3ab1fce8e7591499c9ea6103cf44b69233724656e236 2024-09-11T22:50:26.047 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 22:50:25 smithi204 systemd[1]: Started Ceph mds.f for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:50:26.097 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:25 smithi152 ceph-mon[31181]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:26.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:25 smithi191 ceph-mon[35211]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:26.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.446+0000 7f9fe54d7640 1 -- 172.21.15.152:0/1020212583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe0102780 msgr2=0x7f9fe01067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:26.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.446+0000 7f9fe54d7640 1 --2- 172.21.15.152:0/1020212583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe0102780 0x7f9fe01067d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009900 tx=0x7f9fc802a410 comp rx=0 tx=0).stop 2024-09-11T22:50:26.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.446+0000 7f9fe54d7640 1 -- 172.21.15.152:0/1020212583 shutdown_connections 2024-09-11T22:50:26.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.446+0000 7f9fe54d7640 1 --2- 172.21.15.152:0/1020212583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe0106f00 0x7f9fe010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:26.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.446+0000 7f9fe54d7640 1 --2- 172.21.15.152:0/1020212583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe0102780 0x7f9fe01067d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:26.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.446+0000 7f9fe54d7640 1 --2- 172.21.15.152:0/1020212583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe0101dd0 0x7f9fe01021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:26.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.446+0000 7f9fe54d7640 1 -- 172.21.15.152:0/1020212583 >> 172.21.15.152:0/1020212583 conn(0x7f9fe00fd8b0 msgr2=0x7f9fe00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:26.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.446+0000 7f9fe54d7640 1 -- 172.21.15.152:0/1020212583 shutdown_connections 2024-09-11T22:50:26.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.446+0000 7f9fe54d7640 1 -- 172.21.15.152:0/1020212583 wait complete. 2024-09-11T22:50:26.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fe54d7640 1 Processor -- start 2024-09-11T22:50:26.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fe54d7640 1 -- start start 2024-09-11T22:50:26.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fe54d7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe0101dd0 0x7f9fe0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:26.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fe54d7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe0102780 0x7f9fe0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:26.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fe54d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe0106f00 0x7f9fe019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:26.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fe54d7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe0111f60 con 0x7f9fe0101dd0 2024-09-11T22:50:26.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fe54d7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe0111de0 con 0x7f9fe0102780 2024-09-11T22:50:26.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fe54d7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe01120e0 con 0x7f9fe0106f00 2024-09-11T22:50:26.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fdeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe0101dd0 0x7f9fe0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:26.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fdeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe0101dd0 0x7f9fe0198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40822/0 (socket says 172.21.15.152:40822) 2024-09-11T22:50:26.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fdeffd640 1 -- 172.21.15.152:0/3617853202 learned_addr learned my addr 172.21.15.152:0/3617853202 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:26.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fdf7fe640 1 --2- 172.21.15.152:0/3617853202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe0106f00 0x7f9fe019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:26.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fdeffd640 1 -- 172.21.15.152:0/3617853202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe0106f00 msgr2=0x7f9fe019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:26.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fde7fc640 1 --2- 172.21.15.152:0/3617853202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe0102780 0x7f9fe0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:26.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fdeffd640 1 --2- 172.21.15.152:0/3617853202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe0106f00 0x7f9fe019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:26.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fdeffd640 1 -- 172.21.15.152:0/3617853202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe0102780 msgr2=0x7f9fe0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:26.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fdeffd640 1 --2- 172.21.15.152:0/3617853202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe0102780 0x7f9fe0198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:26.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.447+0000 7f9fdeffd640 1 -- 172.21.15.152:0/3617853202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe019d3e0 con 0x7f9fe0101dd0 2024-09-11T22:50:26.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.448+0000 7f9fdeffd640 1 --2- 172.21.15.152:0/3617853202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe0101dd0 0x7f9fe0198430 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800db80 tx=0x7f9fd800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:26.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.448+0000 7f9fd7fff640 1 -- 172.21.15.152:0/3617853202 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8017070 con 0x7f9fe0101dd0 2024-09-11T22:50:26.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.448+0000 7f9fd7fff640 1 -- 172.21.15.152:0/3617853202 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd8003b70 con 0x7f9fe0101dd0 2024-09-11T22:50:26.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.448+0000 7f9fe54d7640 1 -- 172.21.15.152:0/3617853202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe019d6d0 con 0x7f9fe0101dd0 2024-09-11T22:50:26.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.448+0000 7f9fd7fff640 1 -- 172.21.15.152:0/3617853202 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd80106c0 con 0x7f9fe0101dd0 2024-09-11T22:50:26.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.448+0000 7f9fe54d7640 1 -- 172.21.15.152:0/3617853202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe01a4fb0 con 0x7f9fe0101dd0 2024-09-11T22:50:26.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.449+0000 7f9fe54d7640 1 -- 172.21.15.152:0/3617853202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe01021b0 con 0x7f9fe0101dd0 2024-09-11T22:50:26.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.450+0000 7f9fd7fff640 1 -- 172.21.15.152:0/3617853202 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9fd8004020 con 0x7f9fe0101dd0 2024-09-11T22:50:26.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.451+0000 7f9fd7fff640 1 --2- 172.21.15.152:0/3617853202 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f9fcc0a2b80 0x7f9fcc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:26.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.451+0000 7f9fde7fc640 1 --2- 172.21.15.152:0/3617853202 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f9fcc0a2b80 0x7f9fcc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:26.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.451+0000 7f9fd7fff640 1 -- 172.21.15.152:0/3617853202 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f9fd80ceda0 con 0x7f9fe0101dd0 2024-09-11T22:50:26.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.451+0000 7f9fde7fc640 1 --2- 172.21.15.152:0/3617853202 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f9fcc0a2b80 0x7f9fcc0a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80096c0 tx=0x7f9fc8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:26.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.454+0000 7f9fd7fff640 1 -- 172.21.15.152:0/3617853202 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9fd8020070 con 0x7f9fe0101dd0 2024-09-11T22:50:26.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.656+0000 7f9fe54d7640 1 -- 172.21.15.152:0/3617853202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fe010a110 con 0x7f9fe0101dd0 2024-09-11T22:50:26.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.657+0000 7f9fd7fff640 1 -- 172.21.15.152:0/3617853202 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5575 (secure 0 0 0) 0x7f9fd809c100 con 0x7f9fe0101dd0 2024-09-11T22:50:26.658 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:50:26.658 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":7,"btime":"2024-09-11T22:50:25:819712+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-11T22:50:26.658 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 7 2024-09-11T22:50:26.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.659+0000 7f9fe54d7640 1 -- 172.21.15.152:0/3617853202 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f9fcc0a2b80 msgr2=0x7f9fcc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:26.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.659+0000 7f9fe54d7640 1 --2- 172.21.15.152:0/3617853202 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f9fcc0a2b80 0x7f9fcc0a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9fc80096c0 tx=0x7f9fc8000bc0 comp rx=0 tx=0).stop 2024-09-11T22:50:26.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.659+0000 7f9fe54d7640 1 -- 172.21.15.152:0/3617853202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe0101dd0 msgr2=0x7f9fe0198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:26.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.659+0000 7f9fe54d7640 1 --2- 172.21.15.152:0/3617853202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe0101dd0 0x7f9fe0198430 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800db80 tx=0x7f9fd800b630 comp rx=0 tx=0).stop 2024-09-11T22:50:26.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.659+0000 7f9fe54d7640 1 -- 172.21.15.152:0/3617853202 shutdown_connections 2024-09-11T22:50:26.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.659+0000 7f9fe54d7640 1 --2- 172.21.15.152:0/3617853202 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f9fcc0a2b80 0x7f9fcc0a5040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:26.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.659+0000 7f9fe54d7640 1 --2- 172.21.15.152:0/3617853202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe0106f00 0x7f9fe019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:26.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.659+0000 7f9fe54d7640 1 --2- 172.21.15.152:0/3617853202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fe0102780 0x7f9fe0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:26.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.659+0000 7f9fe54d7640 1 --2- 172.21.15.152:0/3617853202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fe0101dd0 0x7f9fe0198430 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:26.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.659+0000 7f9fe54d7640 1 -- 172.21.15.152:0/3617853202 >> 172.21.15.152:0/3617853202 conn(0x7f9fe00fd8b0 msgr2=0x7f9fe00fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:26.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.659+0000 7f9fe54d7640 1 -- 172.21.15.152:0/3617853202 shutdown_connections 2024-09-11T22:50:26.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:26.659+0000 7f9fe54d7640 1 -- 172.21.15.152:0/3617853202 wait complete. 2024-09-11T22:50:27.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:26 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.204:6834/763950562,v1:172.21.15.204:6835/763950562] up:boot 2024-09-11T22:50:27.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:26 smithi191 ceph-mon[35211]: fsmap 6 up:standby 2024-09-11T22:50:27.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:26 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:50:27.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:26 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:27.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:26 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:27.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:26 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:27.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:26 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:27.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:26 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:27.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:26 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:27.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:26 smithi191 ceph-mon[35211]: Deploying daemon mds.c on smithi204 2024-09-11T22:50:27.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:26 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3617853202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:27.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:26 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.204:6834/763950562,v1:172.21.15.204:6835/763950562] up:boot 2024-09-11T22:50:27.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:26 smithi152 ceph-mon[31181]: fsmap 6 up:standby 2024-09-11T22:50:27.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:26 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:50:27.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:26 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:27.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:26 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:27.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:26 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:27.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:26 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:27.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:26 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:27.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:26 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:27.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:26 smithi152 ceph-mon[31181]: Deploying daemon mds.c on smithi204 2024-09-11T22:50:27.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:26 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3617853202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:27.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:26 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.204:6834/763950562,v1:172.21.15.204:6835/763950562] up:boot 2024-09-11T22:50:27.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:26 smithi204 ceph-mon[35454]: fsmap 6 up:standby 2024-09-11T22:50:27.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:26 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:50:27.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:26 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:27.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:26 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:27.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:26 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:27.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:26 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:27.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:26 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:27.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:26 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:27.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:26 smithi204 ceph-mon[35454]: Deploying daemon mds.c on smithi204 2024-09-11T22:50:27.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:26 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3617853202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:27.868 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 22:50:27 smithi204 systemd[1]: Starting Ceph mds.c for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T22:50:28.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:27 smithi152 ceph-mon[31181]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:28.179 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:27 smithi204 ceph-mon[35454]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:28.179 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 22:50:28 smithi204 podman[65789]: 2024-09-11 22:50:27.919652192 +0000 UTC m=+0.015497405 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:50:28.179 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 22:50:28 smithi204 podman[65789]: 2024-09-11 22:50:28.040069455 +0000 UTC m=+0.135914656 container create 4e76b0e1763970001f661a6c7d07b59903d78880bd40ecd99c33f7f5225aef06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-11T22:50:28.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:27 smithi191 ceph-mon[35211]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:28.546 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 22:50:28 smithi204 podman[65789]: 2024-09-11 22:50:28.318061655 +0000 UTC m=+0.413906860 container init 4e76b0e1763970001f661a6c7d07b59903d78880bd40ecd99c33f7f5225aef06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True) 2024-09-11T22:50:28.547 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 22:50:28 smithi204 podman[65789]: 2024-09-11 22:50:28.321068872 +0000 UTC m=+0.416914075 container start 4e76b0e1763970001f661a6c7d07b59903d78880bd40ecd99c33f7f5225aef06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-11T22:50:28.547 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 22:50:28 smithi204 ceph-mds[65808]: -- 172.21.15.204:0/1637699496 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c27a519c20 con 0x55c27a4f7000 2024-09-11T22:50:28.547 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 22:50:28 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-c[65804]: starting mds.c at 2024-09-11T22:50:28.547 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 22:50:28 smithi204 bash[65789]: 4e76b0e1763970001f661a6c7d07b59903d78880bd40ecd99c33f7f5225aef06 2024-09-11T22:50:28.547 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 22:50:28 smithi204 systemd[1]: Started Ceph mds.c for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:50:29.477 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:50:29.737 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:29.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:29 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:29.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:29 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:29.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:29 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:29.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:29 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:29.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:29 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:29.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:29 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:29.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:29 smithi191 ceph-mon[35211]: Deploying daemon mds.h on smithi191 2024-09-11T22:50:29.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:29 smithi191 ceph-mon[35211]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:29.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:29 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] up:boot 2024-09-11T22:50:29.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:29 smithi191 ceph-mon[35211]: fsmap 7 up:standby 2024-09-11T22:50:29.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:29 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:50:29.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:29 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:29.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:29 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:29.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:29 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:29.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:29 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:29.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:29 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:29.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:29 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:29.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:29 smithi204 ceph-mon[35454]: Deploying daemon mds.h on smithi191 2024-09-11T22:50:29.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:29 smithi204 ceph-mon[35454]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:29.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:29 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] up:boot 2024-09-11T22:50:29.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:29 smithi204 ceph-mon[35454]: fsmap 7 up:standby 2024-09-11T22:50:29.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:29 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:50:29.858 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:29 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:29.858 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:29 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:29.858 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:29 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:29.858 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:29 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:29.858 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:29 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:29.858 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:29 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:29.858 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:29 smithi152 ceph-mon[31181]: Deploying daemon mds.h on smithi191 2024-09-11T22:50:29.858 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:29 smithi152 ceph-mon[31181]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:29.858 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:29 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] up:boot 2024-09-11T22:50:29.858 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:29 smithi152 ceph-mon[31181]: fsmap 7 up:standby 2024-09-11T22:50:29.859 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:29 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:50:30.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.290+0000 7ff9aa754640 1 -- 172.21.15.152:0/3513213104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9a4101420 msgr2=0x7ff9a410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:30.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.290+0000 7ff9aa754640 1 --2- 172.21.15.152:0/3513213104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9a4101420 0x7ff9a410d950 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff99c009cd0 tx=0x7ff99c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:50:30.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9aa754640 1 -- 172.21.15.152:0/3513213104 shutdown_connections 2024-09-11T22:50:30.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9aa754640 1 --2- 172.21.15.152:0/3513213104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9a4101420 0x7ff9a410d950 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:30.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9aa754640 1 --2- 172.21.15.152:0/3513213104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9a4100a80 0x7ff9a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:30.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9aa754640 1 --2- 172.21.15.152:0/3513213104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9a4106a80 0x7ff9a4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:30.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9aa754640 1 -- 172.21.15.152:0/3513213104 >> 172.21.15.152:0/3513213104 conn(0x7ff9a40fc7a0 msgr2=0x7ff9a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:30.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9aa754640 1 -- 172.21.15.152:0/3513213104 shutdown_connections 2024-09-11T22:50:30.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9aa754640 1 -- 172.21.15.152:0/3513213104 wait complete. 2024-09-11T22:50:30.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9aa754640 1 Processor -- start 2024-09-11T22:50:30.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9aa754640 1 -- start start 2024-09-11T22:50:30.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9aa754640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9a4100a80 0x7ff9a4197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:30.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9aa754640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9a4101420 0x7ff9a4197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:30.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9aa754640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9a4106a80 0x7ff9a419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:30.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9aa754640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a4110e80 con 0x7ff9a4106a80 2024-09-11T22:50:30.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9aa754640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a4110d00 con 0x7ff9a4101420 2024-09-11T22:50:30.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9aa754640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a4111000 con 0x7ff9a4100a80 2024-09-11T22:50:30.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9a8cf1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9a4106a80 0x7ff9a419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:30.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9a8cf1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9a4106a80 0x7ff9a419bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54926/0 (socket says 172.21.15.152:54926) 2024-09-11T22:50:30.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.291+0000 7ff9a8cf1640 1 -- 172.21.15.152:0/3981651254 learned_addr learned my addr 172.21.15.152:0/3981651254 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:30.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.292+0000 7ff99bfff640 1 --2- 172.21.15.152:0/3981651254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9a4101420 0x7ff9a4197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:30.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.292+0000 7ff9a3fff640 1 --2- 172.21.15.152:0/3981651254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9a4100a80 0x7ff9a4197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:30.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.292+0000 7ff9a8cf1640 1 -- 172.21.15.152:0/3981651254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9a4100a80 msgr2=0x7ff9a4197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:30.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.292+0000 7ff9a8cf1640 1 --2- 172.21.15.152:0/3981651254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9a4100a80 0x7ff9a4197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:30.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.292+0000 7ff9a8cf1640 1 -- 172.21.15.152:0/3981651254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9a4101420 msgr2=0x7ff9a4197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:30.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.292+0000 7ff9a8cf1640 1 --2- 172.21.15.152:0/3981651254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9a4101420 0x7ff9a4197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:30.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.292+0000 7ff9a8cf1640 1 -- 172.21.15.152:0/3981651254 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a419c2b0 con 0x7ff9a4106a80 2024-09-11T22:50:30.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.292+0000 7ff99bfff640 1 --2- 172.21.15.152:0/3981651254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9a4101420 0x7ff9a4197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:50:30.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.292+0000 7ff9a8cf1640 1 --2- 172.21.15.152:0/3981651254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9a4106a80 0x7ff9a419bbd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff99c005cc0 tx=0x7ff99c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:30.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.292+0000 7ff9a1ffb640 1 -- 172.21.15.152:0/3981651254 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c004040 con 0x7ff9a4106a80 2024-09-11T22:50:30.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.292+0000 7ff9a1ffb640 1 -- 172.21.15.152:0/3981651254 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff99c0046c0 con 0x7ff9a4106a80 2024-09-11T22:50:30.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.292+0000 7ff9aa754640 1 -- 172.21.15.152:0/3981651254 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a419c540 con 0x7ff9a4106a80 2024-09-11T22:50:30.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.292+0000 7ff9a1ffb640 1 -- 172.21.15.152:0/3981651254 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff99c030e00 con 0x7ff9a4106a80 2024-09-11T22:50:30.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.292+0000 7ff9aa754640 1 -- 172.21.15.152:0/3981651254 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a41a3e80 con 0x7ff9a4106a80 2024-09-11T22:50:30.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.293+0000 7ff9aa754640 1 -- 172.21.15.152:0/3981651254 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a4100ee0 con 0x7ff9a4106a80 2024-09-11T22:50:30.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.294+0000 7ff9a1ffb640 1 -- 172.21.15.152:0/3981651254 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff99c0305a0 con 0x7ff9a4106a80 2024-09-11T22:50:30.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.295+0000 7ff9a1ffb640 1 --2- 172.21.15.152:0/3981651254 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff9900a2b80 0x7ff9900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:30.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.295+0000 7ff9a3fff640 1 --2- 172.21.15.152:0/3981651254 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff9900a2b80 0x7ff9900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:30.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.296+0000 7ff9a1ffb640 1 -- 172.21.15.152:0/3981651254 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7ff99c02e030 con 0x7ff9a4106a80 2024-09-11T22:50:30.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.296+0000 7ff9a3fff640 1 --2- 172.21.15.152:0/3981651254 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff9900a2b80 0x7ff9900a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff994009ae0 tx=0x7ff994005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:30.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.300+0000 7ff9a1ffb640 1 -- 172.21.15.152:0/3981651254 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff99c04b060 con 0x7ff9a4106a80 2024-09-11T22:50:30.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.500+0000 7ff9aa754640 1 -- 172.21.15.152:0/3981651254 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9a4108ff0 con 0x7ff9a4106a80 2024-09-11T22:50:30.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.500+0000 7ff9a1ffb640 1 -- 172.21.15.152:0/3981651254 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6403 (secure 0 0 0) 0x7ff99c0bd7f0 con 0x7ff9a4106a80 2024-09-11T22:50:30.501 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:50:30.501 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":8,"btime":"2024-09-11T22:50:28:867085+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-11T22:50:30.502 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 8 2024-09-11T22:50:30.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.502+0000 7ff9aa754640 1 -- 172.21.15.152:0/3981651254 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff9900a2b80 msgr2=0x7ff9900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:30.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.502+0000 7ff9aa754640 1 --2- 172.21.15.152:0/3981651254 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff9900a2b80 0x7ff9900a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff994009ae0 tx=0x7ff994005f90 comp rx=0 tx=0).stop 2024-09-11T22:50:30.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.503+0000 7ff9aa754640 1 -- 172.21.15.152:0/3981651254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9a4106a80 msgr2=0x7ff9a419bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:30.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.503+0000 7ff9aa754640 1 --2- 172.21.15.152:0/3981651254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9a4106a80 0x7ff9a419bbd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff99c005cc0 tx=0x7ff99c032f40 comp rx=0 tx=0).stop 2024-09-11T22:50:30.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.503+0000 7ff9aa754640 1 -- 172.21.15.152:0/3981651254 shutdown_connections 2024-09-11T22:50:30.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.503+0000 7ff9aa754640 1 --2- 172.21.15.152:0/3981651254 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff9900a2b80 0x7ff9900a5040 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:30.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.503+0000 7ff9aa754640 1 --2- 172.21.15.152:0/3981651254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff9a4106a80 0x7ff9a419bbd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:30.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.503+0000 7ff9aa754640 1 --2- 172.21.15.152:0/3981651254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9a4101420 0x7ff9a4197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:30.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.503+0000 7ff9aa754640 1 --2- 172.21.15.152:0/3981651254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9a4100a80 0x7ff9a4197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:30.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.503+0000 7ff9aa754640 1 -- 172.21.15.152:0/3981651254 >> 172.21.15.152:0/3981651254 conn(0x7ff9a40fc7a0 msgr2=0x7ff9a40fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:30.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.503+0000 7ff9aa754640 1 -- 172.21.15.152:0/3981651254 shutdown_connections 2024-09-11T22:50:30.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:30.503+0000 7ff9aa754640 1 -- 172.21.15.152:0/3981651254 wait complete. 2024-09-11T22:50:30.741 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:30 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3981651254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:30.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:30 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3981651254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:31.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:30 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3981651254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:31.537 INFO:journalctl@ceph.mds.h.smithi191.stdout:Sep 11 22:50:31 smithi191 systemd[1]: Started Ceph mds.h for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:50:31.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:31 smithi152 ceph-mon[31181]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:31.937 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:31 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:31.937 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:31 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:31.937 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:31 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:31.938 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:31 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:31.938 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:31 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:31.938 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:31 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:31.938 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:31 smithi204 ceph-mon[35454]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:31.938 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:31 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:31.938 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:31 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:31.939 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:31 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:31.939 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:31 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:31.939 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:31 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:31.939 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:31 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:32.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:31 smithi191 ceph-mon[35211]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:32.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:31 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:32.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:31 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:32.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:31 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:32.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:31 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:32.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:31 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:32.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:31 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:32.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:32 smithi152 ceph-mon[31181]: Deploying daemon mds.i on smithi204 2024-09-11T22:50:32.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:32 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] up:boot 2024-09-11T22:50:32.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:32 smithi152 ceph-mon[31181]: fsmap 8 up:standby 2024-09-11T22:50:32.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:32 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:50:32.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:32 smithi204 ceph-mon[35454]: Deploying daemon mds.i on smithi204 2024-09-11T22:50:32.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:32 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] up:boot 2024-09-11T22:50:32.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:32 smithi204 ceph-mon[35454]: fsmap 8 up:standby 2024-09-11T22:50:32.969 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:32 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:50:33.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:32 smithi191 ceph-mon[35211]: Deploying daemon mds.i on smithi204 2024-09-11T22:50:33.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:32 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] up:boot 2024-09-11T22:50:33.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:32 smithi191 ceph-mon[35211]: fsmap 8 up:standby 2024-09-11T22:50:33.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:32 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:50:33.221 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:50:33.296 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 22:50:33 smithi204 systemd[1]: Starting Ceph mds.i for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T22:50:33.480 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:33.619 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:33 smithi204 ceph-mon[35454]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:33.619 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 22:50:33 smithi204 podman[66089]: 2024-09-11 22:50:33.270518865 +0000 UTC m=+0.015946686 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:50:33.619 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 22:50:33 smithi204 podman[66089]: 2024-09-11 22:50:33.387861178 +0000 UTC m=+0.133288982 container create fd6b87b2557e00e6a8f3d1833ab21091c9fe89ff0121bebe31465b67cecab0d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T22:50:33.619 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 22:50:33 smithi204 podman[66089]: 2024-09-11 22:50:33.616074506 +0000 UTC m=+0.361502310 container init fd6b87b2557e00e6a8f3d1833ab21091c9fe89ff0121bebe31465b67cecab0d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:50:33.620 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 22:50:33 smithi204 podman[66089]: 2024-09-11 22:50:33.61911818 +0000 UTC m=+0.364545986 container start fd6b87b2557e00e6a8f3d1833ab21091c9fe89ff0121bebe31465b67cecab0d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-i, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-11T22:50:33.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:33 smithi152 ceph-mon[31181]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:34.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.006+0000 7fd7b07bb640 1 -- 172.21.15.152:0/551596955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7a810e4f0 msgr2=0x7fd7a8110900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:34.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.006+0000 7fd7b07bb640 1 --2- 172.21.15.152:0/551596955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7a810e4f0 0x7fd7a8110900 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0009ca0 tx=0x7fd7a002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:50:34.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.006+0000 7fd7b07bb640 1 -- 172.21.15.152:0/551596955 shutdown_connections 2024-09-11T22:50:34.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.006+0000 7fd7b07bb640 1 --2- 172.21.15.152:0/551596955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7a810e4f0 0x7fd7a8110900 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:34.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.006+0000 7fd7b07bb640 1 --2- 172.21.15.152:0/551596955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7a8101380 0x7fd7a810dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:34.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.006+0000 7fd7b07bb640 1 --2- 172.21.15.152:0/551596955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a81009d0 0x7fd7a8100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:34.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.006+0000 7fd7b07bb640 1 -- 172.21.15.152:0/551596955 >> 172.21.15.152:0/551596955 conn(0x7fd7a80fc7d0 msgr2=0x7fd7a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:34.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.006+0000 7fd7b07bb640 1 -- 172.21.15.152:0/551596955 shutdown_connections 2024-09-11T22:50:34.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.006+0000 7fd7b07bb640 1 -- 172.21.15.152:0/551596955 wait complete. 2024-09-11T22:50:34.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.007+0000 7fd7b07bb640 1 Processor -- start 2024-09-11T22:50:34.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.007+0000 7fd7b07bb640 1 -- start start 2024-09-11T22:50:34.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.007+0000 7fd7b07bb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a81009d0 0x7fd7a819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:34.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.007+0000 7fd7b07bb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7a8101380 0x7fd7a819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:34.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.007+0000 7fd7b07bb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7a810e4f0 0x7fd7a819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:34.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.007+0000 7fd7b07bb640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8114230 con 0x7fd7a8101380 2024-09-11T22:50:34.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.007+0000 7fd7b07bb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81140b0 con 0x7fd7a810e4f0 2024-09-11T22:50:34.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.007+0000 7fd7b07bb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81143b0 con 0x7fd7a81009d0 2024-09-11T22:50:34.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.007+0000 7fd7add56640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7a8101380 0x7fd7a819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:34.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.007+0000 7fd7add56640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7a8101380 0x7fd7a819ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54950/0 (socket says 172.21.15.152:54950) 2024-09-11T22:50:34.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.007+0000 7fd7add56640 1 -- 172.21.15.152:0/844741907 learned_addr learned my addr 172.21.15.152:0/844741907 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:34.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.007+0000 7fd7ae557640 1 --2- 172.21.15.152:0/844741907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a81009d0 0x7fd7a819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:34.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.007+0000 7fd7aed58640 1 --2- 172.21.15.152:0/844741907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7a810e4f0 0x7fd7a819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:34.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.008+0000 7fd7add56640 1 -- 172.21.15.152:0/844741907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a81009d0 msgr2=0x7fd7a819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:34.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.008+0000 7fd7add56640 1 --2- 172.21.15.152:0/844741907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a81009d0 0x7fd7a819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:34.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.008+0000 7fd7add56640 1 -- 172.21.15.152:0/844741907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7a810e4f0 msgr2=0x7fd7a819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:34.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.008+0000 7fd7add56640 1 --2- 172.21.15.152:0/844741907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7a810e4f0 0x7fd7a819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:34.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.008+0000 7fd7add56640 1 -- 172.21.15.152:0/844741907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a819f6a0 con 0x7fd7a8101380 2024-09-11T22:50:34.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.008+0000 7fd7add56640 1 --2- 172.21.15.152:0/844741907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7a8101380 0x7fd7a819ac30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd79800c890 tx=0x7fd79800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:34.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.008+0000 7fd79f7fe640 1 -- 172.21.15.152:0/844741907 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798004220 con 0x7fd7a8101380 2024-09-11T22:50:34.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.008+0000 7fd79f7fe640 1 -- 172.21.15.152:0/844741907 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7980043c0 con 0x7fd7a8101380 2024-09-11T22:50:34.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.008+0000 7fd7b07bb640 1 -- 172.21.15.152:0/844741907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a819f990 con 0x7fd7a8101380 2024-09-11T22:50:34.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.008+0000 7fd79f7fe640 1 -- 172.21.15.152:0/844741907 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7980049e0 con 0x7fd7a8101380 2024-09-11T22:50:34.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.008+0000 7fd7b07bb640 1 -- 172.21.15.152:0/844741907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a81a7270 con 0x7fd7a8101380 2024-09-11T22:50:34.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.009+0000 7fd7b07bb640 1 -- 172.21.15.152:0/844741907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a8100db0 con 0x7fd7a8101380 2024-09-11T22:50:34.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.010+0000 7fd79f7fe640 1 -- 172.21.15.152:0/844741907 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd7980079b0 con 0x7fd7a8101380 2024-09-11T22:50:34.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.011+0000 7fd79f7fe640 1 --2- 172.21.15.152:0/844741907 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd7940a2b80 0x7fd7940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:34.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.011+0000 7fd7ae557640 1 --2- 172.21.15.152:0/844741907 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd7940a2b80 0x7fd7940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:34.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.011+0000 7fd79f7fe640 1 -- 172.21.15.152:0/844741907 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fd7980cebc0 con 0x7fd7a8101380 2024-09-11T22:50:34.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.011+0000 7fd7ae557640 1 --2- 172.21.15.152:0/844741907 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd7940a2b80 0x7fd7940a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009bd0 tx=0x7fd7a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:34.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.014+0000 7fd79f7fe640 1 -- 172.21.15.152:0/844741907 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd798014030 con 0x7fd7a8101380 2024-09-11T22:50:34.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:33 smithi191 ceph-mon[35211]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:34.046 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 22:50:33 smithi204 ceph-mds[66108]: -- 172.21.15.204:0/3811834385 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d1385fb860 con 0x55d1385d9400 2024-09-11T22:50:34.046 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 22:50:33 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-i[66104]: starting mds.i at 2024-09-11T22:50:34.046 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 22:50:33 smithi204 bash[66089]: fd6b87b2557e00e6a8f3d1833ab21091c9fe89ff0121bebe31465b67cecab0d8 2024-09-11T22:50:34.046 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 22:50:33 smithi204 systemd[1]: Started Ceph mds.i for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:50:34.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.228+0000 7fd7b07bb640 1 -- 172.21.15.152:0/844741907 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7a819ba50 con 0x7fd7a8101380 2024-09-11T22:50:34.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.229+0000 7fd79f7fe640 1 -- 172.21.15.152:0/844741907 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7228 (secure 0 0 0) 0x7fd7a819ba50 con 0x7fd7a8101380 2024-09-11T22:50:34.230 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:50:34.230 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":9,"btime":"2024-09-11T22:50:31:553307+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24434,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-11T22:50:34.230 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 9 2024-09-11T22:50:34.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.231+0000 7fd7b07bb640 1 -- 172.21.15.152:0/844741907 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd7940a2b80 msgr2=0x7fd7940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:34.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.231+0000 7fd7b07bb640 1 --2- 172.21.15.152:0/844741907 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd7940a2b80 0x7fd7940a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4009bd0 tx=0x7fd7a4005f90 comp rx=0 tx=0).stop 2024-09-11T22:50:34.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.231+0000 7fd7b07bb640 1 -- 172.21.15.152:0/844741907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7a8101380 msgr2=0x7fd7a819ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:34.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.231+0000 7fd7b07bb640 1 --2- 172.21.15.152:0/844741907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7a8101380 0x7fd7a819ac30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd79800c890 tx=0x7fd79800cd50 comp rx=0 tx=0).stop 2024-09-11T22:50:34.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.231+0000 7fd7b07bb640 1 -- 172.21.15.152:0/844741907 shutdown_connections 2024-09-11T22:50:34.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.231+0000 7fd7b07bb640 1 --2- 172.21.15.152:0/844741907 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd7940a2b80 0x7fd7940a5040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:34.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.231+0000 7fd7b07bb640 1 --2- 172.21.15.152:0/844741907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd7a810e4f0 0x7fd7a819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:34.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.231+0000 7fd7b07bb640 1 --2- 172.21.15.152:0/844741907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd7a8101380 0x7fd7a819ac30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:34.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.231+0000 7fd7b07bb640 1 --2- 172.21.15.152:0/844741907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7a81009d0 0x7fd7a819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:34.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.231+0000 7fd7b07bb640 1 -- 172.21.15.152:0/844741907 >> 172.21.15.152:0/844741907 conn(0x7fd7a80fc7d0 msgr2=0x7fd7a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:34.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.231+0000 7fd7b07bb640 1 -- 172.21.15.152:0/844741907 shutdown_connections 2024-09-11T22:50:34.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:34.231+0000 7fd7b07bb640 1 -- 172.21.15.152:0/844741907 wait complete. 2024-09-11T22:50:35.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:35.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:35.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:34 smithi191 ceph-mon[35211]: Deploying daemon mds.g on smithi152 2024-09-11T22:50:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:34 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/844741907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:34 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] up:boot 2024-09-11T22:50:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:34 smithi191 ceph-mon[35211]: fsmap 9 up:standby 2024-09-11T22:50:35.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:34 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:50:35.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:35.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:35.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:35.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:35.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:35.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:35.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:34 smithi204 ceph-mon[35454]: Deploying daemon mds.g on smithi152 2024-09-11T22:50:35.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:34 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/844741907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:35.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:34 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] up:boot 2024-09-11T22:50:35.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:34 smithi204 ceph-mon[35454]: fsmap 9 up:standby 2024-09-11T22:50:35.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:34 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:50:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:34 smithi152 ceph-mon[31181]: Deploying daemon mds.g on smithi152 2024-09-11T22:50:35.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:34 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/844741907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:34 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] up:boot 2024-09-11T22:50:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:34 smithi152 ceph-mon[31181]: fsmap 9 up:standby 2024-09-11T22:50:35.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:34 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:50:35.959 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:35 smithi152 ceph-mon[31181]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:36.189 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:35 smithi191 ceph-mon[35211]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:36.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:35 smithi204 ceph-mon[35454]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:36.623 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 22:50:36 smithi152 podman[69596]: 2024-09-11 22:50:36.509167098 +0000 UTC m=+0.149084744 container create 208d85f65604d98645c29e22d739bbf8b07363a197071857d0fa05a5c4e1bf63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-g, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-11T22:50:37.122 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 22:50:36 smithi152 podman[69596]: 2024-09-11 22:50:36.762045012 +0000 UTC m=+0.401962653 container init 208d85f65604d98645c29e22d739bbf8b07363a197071857d0fa05a5c4e1bf63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-g, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-11T22:50:37.123 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 22:50:36 smithi152 podman[69596]: 2024-09-11 22:50:36.765082842 +0000 UTC m=+0.405000493 container start 208d85f65604d98645c29e22d739bbf8b07363a197071857d0fa05a5c4e1bf63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-11T22:50:37.123 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 22:50:36 smithi152 ceph-mds[69616]: -- 172.21.15.152:0/793985038 <== mon.0 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e7f1c0b860 con 0x55e7f1be9400 2024-09-11T22:50:37.123 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 22:50:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-g[69612]: starting mds.g at 2024-09-11T22:50:37.123 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 22:50:36 smithi152 bash[69596]: 208d85f65604d98645c29e22d739bbf8b07363a197071857d0fa05a5c4e1bf63 2024-09-11T22:50:37.123 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 22:50:36 smithi152 systemd[1]: Started Ceph mds.g for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:50:38.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:37 smithi191 ceph-mon[35211]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:38.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:37 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.152:6838/97179606,v1:172.21.15.152:6839/97179606] up:boot 2024-09-11T22:50:38.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:37 smithi191 ceph-mon[35211]: fsmap 10 up:standby 2024-09-11T22:50:38.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:37 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:50:38.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:37 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:38.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:37 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:38.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:37 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:38.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:37 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:38.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:37 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:38.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:37 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:38.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:37 smithi191 ceph-mon[35211]: Deploying daemon mds.a on smithi152 2024-09-11T22:50:38.295 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:50:38.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:37 smithi204 ceph-mon[35454]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:38.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:37 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.152:6838/97179606,v1:172.21.15.152:6839/97179606] up:boot 2024-09-11T22:50:38.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:37 smithi204 ceph-mon[35454]: fsmap 10 up:standby 2024-09-11T22:50:38.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:37 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:50:38.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:37 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:38.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:37 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:38.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:37 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:38.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:37 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:38.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:37 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:38.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:37 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:38.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:37 smithi204 ceph-mon[35454]: Deploying daemon mds.a on smithi152 2024-09-11T22:50:38.318 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:37 smithi152 ceph-mon[31181]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:38.318 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:37 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.152:6838/97179606,v1:172.21.15.152:6839/97179606] up:boot 2024-09-11T22:50:38.318 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:37 smithi152 ceph-mon[31181]: fsmap 10 up:standby 2024-09-11T22:50:38.318 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:37 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:50:38.318 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:37 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:38.318 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:37 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:38.318 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:37 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:38.318 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:37 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:38.319 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:37 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:38.319 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:37 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:38.319 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:37 smithi152 ceph-mon[31181]: Deploying daemon mds.a on smithi152 2024-09-11T22:50:38.562 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:39.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.107+0000 7f27319ec640 1 -- 172.21.15.152:0/3653359249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f272c102570 msgr2=0x7f272c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:39.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.107+0000 7f27319ec640 1 --2- 172.21.15.152:0/3653359249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f272c102570 0x7f272c1029b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2714009900 tx=0x7f271402a480 comp rx=0 tx=0).stop 2024-09-11T22:50:39.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.108+0000 7f27319ec640 1 -- 172.21.15.152:0/3653359249 shutdown_connections 2024-09-11T22:50:39.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.108+0000 7f27319ec640 1 --2- 172.21.15.152:0/3653359249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f272c102ef0 0x7f272c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:39.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.108+0000 7f27319ec640 1 --2- 172.21.15.152:0/3653359249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f272c102570 0x7f272c1029b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:39.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.108+0000 7f27319ec640 1 --2- 172.21.15.152:0/3653359249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f272c100e20 0x7f272c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:39.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.108+0000 7f27319ec640 1 -- 172.21.15.152:0/3653359249 >> 172.21.15.152:0/3653359249 conn(0x7f272c0fc7d0 msgr2=0x7f272c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:39.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.108+0000 7f27319ec640 1 -- 172.21.15.152:0/3653359249 shutdown_connections 2024-09-11T22:50:39.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.108+0000 7f27319ec640 1 -- 172.21.15.152:0/3653359249 wait complete. 2024-09-11T22:50:39.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.108+0000 7f27319ec640 1 Processor -- start 2024-09-11T22:50:39.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.108+0000 7f27319ec640 1 -- start start 2024-09-11T22:50:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f27319ec640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f272c100e20 0x7f272c197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f27319ec640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f272c102570 0x7f272c197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f27319ec640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f272c102ef0 0x7f272c19bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f27319ec640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f272c110f50 con 0x7f272c102ef0 2024-09-11T22:50:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f27319ec640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f272c110dd0 con 0x7f272c102570 2024-09-11T22:50:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f27319ec640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f272c1110d0 con 0x7f272c100e20 2024-09-11T22:50:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f272b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f272c102ef0 0x7f272c19bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f272b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f272c102ef0 0x7f272c19bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60322/0 (socket says 172.21.15.152:60322) 2024-09-11T22:50:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f272b7fe640 1 -- 172.21.15.152:0/4082833900 learned_addr learned my addr 172.21.15.152:0/4082833900 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f272affd640 1 --2- 172.21.15.152:0/4082833900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f272c100e20 0x7f272c197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:39.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f272a7fc640 1 --2- 172.21.15.152:0/4082833900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f272c102570 0x7f272c197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f272b7fe640 1 -- 172.21.15.152:0/4082833900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f272c100e20 msgr2=0x7f272c197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f272b7fe640 1 --2- 172.21.15.152:0/4082833900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f272c100e20 0x7f272c197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f272b7fe640 1 -- 172.21.15.152:0/4082833900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f272c102570 msgr2=0x7f272c197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f272b7fe640 1 --2- 172.21.15.152:0/4082833900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f272c102570 0x7f272c197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f272b7fe640 1 -- 172.21.15.152:0/4082833900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272c19c3e0 con 0x7f272c102ef0 2024-09-11T22:50:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f272a7fc640 1 --2- 172.21.15.152:0/4082833900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f272c102570 0x7f272c197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:50:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f272affd640 1 --2- 172.21.15.152:0/4082833900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f272c100e20 0x7f272c197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:50:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f272b7fe640 1 --2- 172.21.15.152:0/4082833900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f272c102ef0 0x7f272c19bd00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f271c00eaa0 tx=0x7f271c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f27309ea640 1 -- 172.21.15.152:0/4082833900 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c00cdb0 con 0x7f272c102ef0 2024-09-11T22:50:39.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f27319ec640 1 -- 172.21.15.152:0/4082833900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f272c19c6d0 con 0x7f272c102ef0 2024-09-11T22:50:39.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f27309ea640 1 -- 172.21.15.152:0/4082833900 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f271c005020 con 0x7f272c102ef0 2024-09-11T22:50:39.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f27309ea640 1 -- 172.21.15.152:0/4082833900 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c0116b0 con 0x7f272c102ef0 2024-09-11T22:50:39.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.109+0000 7f27319ec640 1 -- 172.21.15.152:0/4082833900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f272c1a3fb0 con 0x7f272c102ef0 2024-09-11T22:50:39.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.110+0000 7f27319ec640 1 -- 172.21.15.152:0/4082833900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f272c101200 con 0x7f272c102ef0 2024-09-11T22:50:39.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.111+0000 7f27309ea640 1 -- 172.21.15.152:0/4082833900 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f271c005560 con 0x7f272c102ef0 2024-09-11T22:50:39.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.112+0000 7f27309ea640 1 --2- 172.21.15.152:0/4082833900 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f27180a2b80 0x7f27180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:39.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.112+0000 7f272affd640 1 --2- 172.21.15.152:0/4082833900 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f27180a2b80 0x7f27180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:39.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.112+0000 7f27309ea640 1 -- 172.21.15.152:0/4082833900 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f271c0cee60 con 0x7f272c102ef0 2024-09-11T22:50:39.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.113+0000 7f272affd640 1 --2- 172.21.15.152:0/4082833900 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f27180a2b80 0x7f27180a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2724005d00 tx=0x7f2724005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:39.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.116+0000 7f27309ea640 1 -- 172.21.15.152:0/4082833900 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f271c016030 con 0x7f272c102ef0 2024-09-11T22:50:39.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.329+0000 7f27319ec640 1 -- 172.21.15.152:0/4082833900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f272c109090 con 0x7f272c102ef0 2024-09-11T22:50:39.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.330+0000 7f27309ea640 1 -- 172.21.15.152:0/4082833900 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8881 (secure 0 0 0) 0x7f271c09c260 con 0x7f272c102ef0 2024-09-11T22:50:39.332 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:50:39.332 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":11,"btime":"2024-09-11T22:50:36:876758+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24434,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T22:50:39.332 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 11 2024-09-11T22:50:39.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.332+0000 7f27319ec640 1 -- 172.21.15.152:0/4082833900 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f27180a2b80 msgr2=0x7f27180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:39.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.332+0000 7f27319ec640 1 --2- 172.21.15.152:0/4082833900 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f27180a2b80 0x7f27180a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2724005d00 tx=0x7f2724005c70 comp rx=0 tx=0).stop 2024-09-11T22:50:39.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.332+0000 7f27319ec640 1 -- 172.21.15.152:0/4082833900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f272c102ef0 msgr2=0x7f272c19bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:39.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.332+0000 7f27319ec640 1 --2- 172.21.15.152:0/4082833900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f272c102ef0 0x7f272c19bd00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f271c00eaa0 tx=0x7f271c00ef60 comp rx=0 tx=0).stop 2024-09-11T22:50:39.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.332+0000 7f27319ec640 1 -- 172.21.15.152:0/4082833900 shutdown_connections 2024-09-11T22:50:39.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.332+0000 7f27319ec640 1 --2- 172.21.15.152:0/4082833900 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f27180a2b80 0x7f27180a5040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:39.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.332+0000 7f27319ec640 1 --2- 172.21.15.152:0/4082833900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f272c102ef0 0x7f272c19bd00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:39.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.332+0000 7f27319ec640 1 --2- 172.21.15.152:0/4082833900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f272c102570 0x7f272c197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:39.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.332+0000 7f27319ec640 1 --2- 172.21.15.152:0/4082833900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f272c100e20 0x7f272c197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:39.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.332+0000 7f27319ec640 1 -- 172.21.15.152:0/4082833900 >> 172.21.15.152:0/4082833900 conn(0x7f272c0fc7d0 msgr2=0x7f272c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:39.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.332+0000 7f27319ec640 1 -- 172.21.15.152:0/4082833900 shutdown_connections 2024-09-11T22:50:39.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:39.332+0000 7f27319ec640 1 -- 172.21.15.152:0/4082833900 wait complete. 2024-09-11T22:50:40.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:39 smithi152 ceph-mon[31181]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:40.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:39 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4082833900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:40.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:39 smithi191 ceph-mon[35211]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:40.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:39 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/4082833900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:40.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:39 smithi204 ceph-mon[35454]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:40.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:39 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/4082833900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:40.562 INFO:journalctl@ceph.mds.a.smithi152.stdout:Sep 11 22:50:40 smithi152 systemd[1]: Started Ceph mds.a for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:50:41.072 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:40 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] up:boot 2024-09-11T22:50:41.072 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:40 smithi191 ceph-mon[35211]: fsmap 11 up:standby 2024-09-11T22:50:41.072 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:40 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:50:41.072 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:40 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:41.072 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:40 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:41.072 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:40 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:41.072 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:40 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:41.072 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:40 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:41.072 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:40 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:41.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:40 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] up:boot 2024-09-11T22:50:41.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:40 smithi204 ceph-mon[35454]: fsmap 11 up:standby 2024-09-11T22:50:41.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:40 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:50:41.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:40 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:41.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:40 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:41.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:40 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:41.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:40 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:41.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:40 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:41.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:40 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:41.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:40 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] up:boot 2024-09-11T22:50:41.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:40 smithi152 ceph-mon[31181]: fsmap 11 up:standby 2024-09-11T22:50:41.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:40 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:50:41.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:40 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:41.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:40 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:41.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:40 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:41.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:40 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:50:41.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:40 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:50:41.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:40 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:42.152 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:41 smithi191 ceph-mon[35211]: Deploying daemon mds.b on smithi191 2024-09-11T22:50:42.152 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:41 smithi191 ceph-mon[35211]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:42.263 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:50:42.293 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:41 smithi152 ceph-mon[31181]: Deploying daemon mds.b on smithi191 2024-09-11T22:50:42.293 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:41 smithi152 ceph-mon[31181]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:42.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:41 smithi204 ceph-mon[35454]: Deploying daemon mds.b on smithi191 2024-09-11T22:50:42.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:41 smithi204 ceph-mon[35454]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:42.530 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:43.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.010+0000 7f3dc677f640 1 -- 172.21.15.152:0/83879122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc0100e20 msgr2=0x7f3dc0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:43.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.010+0000 7f3dc677f640 1 --2- 172.21.15.152:0/83879122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc0100e20 0x7f3dc0101200 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009900 tx=0x7f3db002a480 comp rx=0 tx=0).stop 2024-09-11T22:50:43.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.010+0000 7f3dc677f640 1 -- 172.21.15.152:0/83879122 shutdown_connections 2024-09-11T22:50:43.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.010+0000 7f3dc677f640 1 --2- 172.21.15.152:0/83879122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dc0102ef0 0x7f3dc010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:43.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.010+0000 7f3dc677f640 1 --2- 172.21.15.152:0/83879122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dc0102570 0x7f3dc01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:43.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.010+0000 7f3dc677f640 1 --2- 172.21.15.152:0/83879122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc0100e20 0x7f3dc0101200 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:43.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.010+0000 7f3dc677f640 1 -- 172.21.15.152:0/83879122 >> 172.21.15.152:0/83879122 conn(0x7f3dc00fc7d0 msgr2=0x7f3dc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:43.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.010+0000 7f3dc677f640 1 -- 172.21.15.152:0/83879122 shutdown_connections 2024-09-11T22:50:43.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.010+0000 7f3dc677f640 1 -- 172.21.15.152:0/83879122 wait complete. 2024-09-11T22:50:43.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.011+0000 7f3dc677f640 1 Processor -- start 2024-09-11T22:50:43.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.011+0000 7f3dc677f640 1 -- start start 2024-09-11T22:50:43.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.011+0000 7f3dc677f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dc0100e20 0x7f3dc01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:43.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.011+0000 7f3dc677f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dc0102570 0x7f3dc01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:43.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.011+0000 7f3dc677f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc0102ef0 0x7f3dc019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:43.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.011+0000 7f3dc677f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc0110f80 con 0x7f3dc0102ef0 2024-09-11T22:50:43.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.011+0000 7f3dc677f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc0110e00 con 0x7f3dc0100e20 2024-09-11T22:50:43.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.011+0000 7f3dc677f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc0111100 con 0x7f3dc0102570 2024-09-11T22:50:43.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.011+0000 7f3dc4d1c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc0102ef0 0x7f3dc019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:43.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.011+0000 7f3dc4d1c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc0102ef0 0x7f3dc019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60352/0 (socket says 172.21.15.152:60352) 2024-09-11T22:50:43.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.011+0000 7f3dc4d1c640 1 -- 172.21.15.152:0/83674205 learned_addr learned my addr 172.21.15.152:0/83674205 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:43.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.011+0000 7f3dbffff640 1 --2- 172.21.15.152:0/83674205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dc0100e20 0x7f3dc01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:43.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.011+0000 7f3dbf7fe640 1 --2- 172.21.15.152:0/83674205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dc0102570 0x7f3dc01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:43.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.012+0000 7f3dc4d1c640 1 -- 172.21.15.152:0/83674205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dc0102570 msgr2=0x7f3dc01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:43.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.012+0000 7f3dc4d1c640 1 --2- 172.21.15.152:0/83674205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dc0102570 0x7f3dc01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:43.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.012+0000 7f3dc4d1c640 1 -- 172.21.15.152:0/83674205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dc0100e20 msgr2=0x7f3dc01984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:43.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.012+0000 7f3dc4d1c640 1 --2- 172.21.15.152:0/83674205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dc0100e20 0x7f3dc01984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:43.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.012+0000 7f3dc4d1c640 1 -- 172.21.15.152:0/83674205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc019d460 con 0x7f3dc0102ef0 2024-09-11T22:50:43.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.012+0000 7f3dbf7fe640 1 --2- 172.21.15.152:0/83674205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dc0102570 0x7f3dc01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:50:43.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.012+0000 7f3dc4d1c640 1 --2- 172.21.15.152:0/83674205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc0102ef0 0x7f3dc019cd80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3db800eb10 tx=0x7f3db800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:43.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.012+0000 7f3dbd7fa640 1 -- 172.21.15.152:0/83674205 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8004b20 con 0x7f3dc0102ef0 2024-09-11T22:50:43.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.012+0000 7f3dc677f640 1 -- 172.21.15.152:0/83674205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc019d750 con 0x7f3dc0102ef0 2024-09-11T22:50:43.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.012+0000 7f3dbd7fa640 1 -- 172.21.15.152:0/83674205 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3db8004cc0 con 0x7f3dc0102ef0 2024-09-11T22:50:43.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.012+0000 7f3dbd7fa640 1 -- 172.21.15.152:0/83674205 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8010760 con 0x7f3dc0102ef0 2024-09-11T22:50:43.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.012+0000 7f3dc677f640 1 -- 172.21.15.152:0/83674205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc01a5030 con 0x7f3dc0102ef0 2024-09-11T22:50:43.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.013+0000 7f3dc677f640 1 -- 172.21.15.152:0/83674205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc0101200 con 0x7f3dc0102ef0 2024-09-11T22:50:43.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.014+0000 7f3dbd7fa640 1 -- 172.21.15.152:0/83674205 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3db8005020 con 0x7f3dc0102ef0 2024-09-11T22:50:43.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.014+0000 7f3dbd7fa640 1 --2- 172.21.15.152:0/83674205 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3da80a2b80 0x7f3da80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:43.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.015+0000 7f3dbffff640 1 --2- 172.21.15.152:0/83674205 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3da80a2b80 0x7f3da80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:43.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.015+0000 7f3dbd7fa640 1 -- 172.21.15.152:0/83674205 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f3db80cf200 con 0x7f3dc0102ef0 2024-09-11T22:50:43.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.015+0000 7f3dbffff640 1 --2- 172.21.15.152:0/83674205 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3da80a2b80 0x7f3da80a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009230 tx=0x7f3db00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:43.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.019+0000 7f3dbd7fa640 1 -- 172.21.15.152:0/83674205 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3db809c600 con 0x7f3dc0102ef0 2024-09-11T22:50:43.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.226+0000 7f3dc677f640 1 -- 172.21.15.152:0/83674205 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dc01a5200 con 0x7f3dc0102ef0 2024-09-11T22:50:43.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.227+0000 7f3dbd7fa640 1 -- 172.21.15.152:0/83674205 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9710 (secure 0 0 0) 0x7f3db809bf00 con 0x7f3dc0102ef0 2024-09-11T22:50:43.229 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:50:43.229 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":12,"btime":"2024-09-11T22:50:39:884267+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24434,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T22:50:43.230 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 12 2024-09-11T22:50:43.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.230+0000 7f3dc677f640 1 -- 172.21.15.152:0/83674205 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3da80a2b80 msgr2=0x7f3da80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:43.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.230+0000 7f3dc677f640 1 --2- 172.21.15.152:0/83674205 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3da80a2b80 0x7f3da80a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009230 tx=0x7f3db00058d0 comp rx=0 tx=0).stop 2024-09-11T22:50:43.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.230+0000 7f3dc677f640 1 -- 172.21.15.152:0/83674205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc0102ef0 msgr2=0x7f3dc019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:43.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.230+0000 7f3dc677f640 1 --2- 172.21.15.152:0/83674205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc0102ef0 0x7f3dc019cd80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3db800eb10 tx=0x7f3db800c2d0 comp rx=0 tx=0).stop 2024-09-11T22:50:43.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.230+0000 7f3dc677f640 1 -- 172.21.15.152:0/83674205 shutdown_connections 2024-09-11T22:50:43.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.230+0000 7f3dc677f640 1 --2- 172.21.15.152:0/83674205 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3da80a2b80 0x7f3da80a5040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.230+0000 7f3dc677f640 1 --2- 172.21.15.152:0/83674205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc0102ef0 0x7f3dc019cd80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.230+0000 7f3dc677f640 1 --2- 172.21.15.152:0/83674205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dc0102570 0x7f3dc01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.230+0000 7f3dc677f640 1 --2- 172.21.15.152:0/83674205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dc0100e20 0x7f3dc01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.230+0000 7f3dc677f640 1 -- 172.21.15.152:0/83674205 >> 172.21.15.152:0/83674205 conn(0x7f3dc00fc7d0 msgr2=0x7f3dc0069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.230+0000 7f3dc677f640 1 -- 172.21.15.152:0/83674205 shutdown_connections 2024-09-11T22:50:43.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:43.230+0000 7f3dc677f640 1 -- 172.21.15.152:0/83674205 wait complete. 2024-09-11T22:50:43.287 INFO:journalctl@ceph.mds.b.smithi191.stdout:Sep 11 22:50:43 smithi191 systemd[1]: Started Ceph mds.b for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T22:50:44.157 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:43 smithi191 ceph-mon[35211]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:44.157 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:44.157 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:44.157 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:44.160 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:44.161 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:44.161 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:43 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/83674205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:44.161 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:43 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:50:44.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:43 smithi204 ceph-mon[35454]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:44.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:44.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:44.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:44.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:44.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:44.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:43 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/83674205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:44.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:43 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:50:44.338 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:43 smithi152 ceph-mon[31181]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:44.338 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:44.338 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:44.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:44.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:44.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:44.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:43 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/83674205' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:44.339 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:43 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:50:45.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:44 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] up:boot 2024-09-11T22:50:45.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:44 smithi191 ceph-mon[35211]: fsmap 12 up:standby 2024-09-11T22:50:45.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:50:45.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:45.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:45.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:45.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:45.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:44 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:45.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:44 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] up:boot 2024-09-11T22:50:45.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:44 smithi204 ceph-mon[35454]: fsmap 12 up:standby 2024-09-11T22:50:45.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:50:45.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:45.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:45.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:45.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:45.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:44 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:45.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:44 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] up:boot 2024-09-11T22:50:45.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:44 smithi152 ceph-mon[31181]: fsmap 12 up:standby 2024-09-11T22:50:45.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:50:45.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:45.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:45.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:45.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:45.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:44 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:45.947 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:50:46.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:45 smithi204 ceph-mon[35454]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:46.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:46.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:46.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:46.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:50:46.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:46.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:45 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:50:46.189 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:45 smithi191 ceph-mon[35211]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:46.189 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:46.189 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:46.189 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:46.189 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:50:46.189 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:46.189 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:45 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:50:46.219 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:45 smithi152 ceph-mon[31181]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:46.219 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:46.219 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:46.220 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:46.220 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:50:46.220 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:46.220 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:45 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:50:46.537 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:47.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.052+0000 7f682031c640 1 -- 172.21.15.152:0/3602860886 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6818100a80 msgr2=0x7f6818100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:47.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.052+0000 7f682031c640 1 --2- 172.21.15.152:0/3602860886 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6818100a80 0x7f6818100ee0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6814009930 tx=0x7f681402a420 comp rx=0 tx=0).stop 2024-09-11T22:50:47.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.052+0000 7f682031c640 1 -- 172.21.15.152:0/3602860886 shutdown_connections 2024-09-11T22:50:47.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.052+0000 7f682031c640 1 --2- 172.21.15.152:0/3602860886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6818101420 0x7f681810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:47.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.052+0000 7f682031c640 1 --2- 172.21.15.152:0/3602860886 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6818100a80 0x7f6818100ee0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:47.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.052+0000 7f682031c640 1 --2- 172.21.15.152:0/3602860886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6818106a80 0x7f6818106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:47.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.052+0000 7f682031c640 1 -- 172.21.15.152:0/3602860886 >> 172.21.15.152:0/3602860886 conn(0x7f68180fc7a0 msgr2=0x7f68180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:47.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.052+0000 7f682031c640 1 -- 172.21.15.152:0/3602860886 shutdown_connections 2024-09-11T22:50:47.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.052+0000 7f682031c640 1 -- 172.21.15.152:0/3602860886 wait complete. 2024-09-11T22:50:47.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.052+0000 7f682031c640 1 Processor -- start 2024-09-11T22:50:47.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.052+0000 7f682031c640 1 -- start start 2024-09-11T22:50:47.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f682031c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6818100a80 0x7f681810e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:47.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f682031c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6818101420 0x7f681810e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:47.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f682031c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6818106a80 0x7f681810ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:47.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f682031c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6818111fa0 con 0x7f6818100a80 2024-09-11T22:50:47.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f682031c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6818111e20 con 0x7f6818106a80 2024-09-11T22:50:47.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f682031c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6818112120 con 0x7f6818101420 2024-09-11T22:50:47.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f681e0b8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6818100a80 0x7f681810e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:47.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f681e0b8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6818100a80 0x7f681810e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60372/0 (socket says 172.21.15.152:60372) 2024-09-11T22:50:47.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f681e0b8640 1 -- 172.21.15.152:0/2745030281 learned_addr learned my addr 172.21.15.152:0/2745030281 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:47.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f681d8b7640 1 --2- 172.21.15.152:0/2745030281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6818101420 0x7f681810e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:47.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f681e8b9640 1 --2- 172.21.15.152:0/2745030281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6818106a80 0x7f681810ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:47.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f681e0b8640 1 -- 172.21.15.152:0/2745030281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6818101420 msgr2=0x7f681810e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:47.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f681e0b8640 1 --2- 172.21.15.152:0/2745030281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6818101420 0x7f681810e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:47.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f681e0b8640 1 -- 172.21.15.152:0/2745030281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6818106a80 msgr2=0x7f681810ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:47.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f681e0b8640 1 --2- 172.21.15.152:0/2745030281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6818106a80 0x7f681810ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:47.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f681e0b8640 1 -- 172.21.15.152:0/2745030281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68181a6920 con 0x7f6818100a80 2024-09-11T22:50:47.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f681e0b8640 1 --2- 172.21.15.152:0/2745030281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6818100a80 0x7f681810e3e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f680c00c920 tx=0x7f680c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:47.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f680b7fe640 1 -- 172.21.15.152:0/2745030281 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c007c70 con 0x7f6818100a80 2024-09-11T22:50:47.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f680b7fe640 1 -- 172.21.15.152:0/2745030281 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f680c007e10 con 0x7f6818100a80 2024-09-11T22:50:47.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f682031c640 1 -- 172.21.15.152:0/2745030281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68181a6c10 con 0x7f6818100a80 2024-09-11T22:50:47.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f680b7fe640 1 -- 172.21.15.152:0/2745030281 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c018890 con 0x7f6818100a80 2024-09-11T22:50:47.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.053+0000 7f682031c640 1 -- 172.21.15.152:0/2745030281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68181a7120 con 0x7f6818100a80 2024-09-11T22:50:47.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.054+0000 7f682031c640 1 -- 172.21.15.152:0/2745030281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6818100ee0 con 0x7f6818100a80 2024-09-11T22:50:47.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.056+0000 7f680b7fe640 1 -- 172.21.15.152:0/2745030281 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f680c018a50 con 0x7f6818100a80 2024-09-11T22:50:47.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.057+0000 7f680b7fe640 1 --2- 172.21.15.152:0/2745030281 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f68040a2b80 0x7f68040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:47.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.057+0000 7f681d8b7640 1 --2- 172.21.15.152:0/2745030281 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f68040a2b80 0x7f68040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:47.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.057+0000 7f680b7fe640 1 -- 172.21.15.152:0/2745030281 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f680c0cef80 con 0x7f6818100a80 2024-09-11T22:50:47.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.057+0000 7f681d8b7640 1 --2- 172.21.15.152:0/2745030281 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f68040a2b80 0x7f68040a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6814009900 tx=0x7f68140058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:47.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.060+0000 7f680b7fe640 1 -- 172.21.15.152:0/2745030281 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f680c014030 con 0x7f6818100a80 2024-09-11T22:50:47.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.296+0000 7f682031c640 1 -- 172.21.15.152:0/2745030281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f681810f910 con 0x7f6818100a80 2024-09-11T22:50:47.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.296+0000 7f680b7fe640 1 -- 172.21.15.152:0/2745030281 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10539 (secure 0 0 0) 0x7f680c09c410 con 0x7f6818100a80 2024-09-11T22:50:47.298 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:50:47.299 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":13,"btime":"2024-09-11T22:50:43:894346+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24377,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24397,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24434,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24455,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-11T22:50:47.299 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 13 2024-09-11T22:50:47.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.299+0000 7f682031c640 1 -- 172.21.15.152:0/2745030281 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f68040a2b80 msgr2=0x7f68040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:47.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.299+0000 7f682031c640 1 --2- 172.21.15.152:0/2745030281 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f68040a2b80 0x7f68040a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6814009900 tx=0x7f68140058d0 comp rx=0 tx=0).stop 2024-09-11T22:50:47.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.299+0000 7f682031c640 1 -- 172.21.15.152:0/2745030281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6818100a80 msgr2=0x7f681810e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:47.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.299+0000 7f682031c640 1 --2- 172.21.15.152:0/2745030281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6818100a80 0x7f681810e3e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f680c00c920 tx=0x7f680c00cde0 comp rx=0 tx=0).stop 2024-09-11T22:50:47.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.299+0000 7f682031c640 1 -- 172.21.15.152:0/2745030281 shutdown_connections 2024-09-11T22:50:47.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.299+0000 7f682031c640 1 --2- 172.21.15.152:0/2745030281 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f68040a2b80 0x7f68040a5040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:47.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.299+0000 7f682031c640 1 --2- 172.21.15.152:0/2745030281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6818106a80 0x7f681810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:47.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.299+0000 7f682031c640 1 --2- 172.21.15.152:0/2745030281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6818101420 0x7f681810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:47.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.299+0000 7f682031c640 1 --2- 172.21.15.152:0/2745030281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6818100a80 0x7f681810e3e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:47.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.299+0000 7f682031c640 1 -- 172.21.15.152:0/2745030281 >> 172.21.15.152:0/2745030281 conn(0x7f68180fc7a0 msgr2=0x7f68180fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:47.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.299+0000 7f682031c640 1 -- 172.21.15.152:0/2745030281 shutdown_connections 2024-09-11T22:50:47.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:47.299+0000 7f682031c640 1 -- 172.21.15.152:0/2745030281 wait complete. 2024-09-11T22:50:47.978 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:47 smithi152 ceph-mon[31181]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:47.979 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:47.979 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:47.979 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:47.979 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:47.979 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:47 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2745030281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:47.979 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:47.979 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:47.979 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:47.979 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:50:47.979 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:48.007 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-11T22:50:48.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:47 smithi191 ceph-mon[35211]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:48.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:48.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:48.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:48.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:48.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:47 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2745030281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:48.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:48.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:48.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:48.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:50:48.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:48.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:47 smithi204 ceph-mon[35454]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:48.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:48.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:48.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:48.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:48.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:47 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2745030281' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:50:48.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:48.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:48.048 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:50:48.048 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:50:48.048 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:50:48.253 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:48.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.761+0000 7f74ec522640 1 -- 172.21.15.152:0/1993943865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74e4105a90 msgr2=0x7f74e4105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:48.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.761+0000 7f74ec522640 1 --2- 172.21.15.152:0/1993943865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74e4105a90 0x7f74e4105e70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f74d4009900 tx=0x7f74d402a410 comp rx=0 tx=0).stop 2024-09-11T22:50:48.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.761+0000 7f74ec522640 1 -- 172.21.15.152:0/1993943865 shutdown_connections 2024-09-11T22:50:48.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.761+0000 7f74ec522640 1 --2- 172.21.15.152:0/1993943865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74e4101bc0 0x7f74e410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:48.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.761+0000 7f74ec522640 1 --2- 172.21.15.152:0/1993943865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74e4101240 0x7f74e4101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:48.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.761+0000 7f74ec522640 1 --2- 172.21.15.152:0/1993943865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74e4105a90 0x7f74e4105e70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:48.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.761+0000 7f74ec522640 1 -- 172.21.15.152:0/1993943865 >> 172.21.15.152:0/1993943865 conn(0x7f74e4077210 msgr2=0x7f74e40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:48.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.761+0000 7f74ec522640 1 -- 172.21.15.152:0/1993943865 shutdown_connections 2024-09-11T22:50:48.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.761+0000 7f74ec522640 1 -- 172.21.15.152:0/1993943865 wait complete. 2024-09-11T22:50:48.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.762+0000 7f74ec522640 1 Processor -- start 2024-09-11T22:50:48.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.762+0000 7f74ec522640 1 -- start start 2024-09-11T22:50:48.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.762+0000 7f74ec522640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74e4101240 0x7f74e41973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:48.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.762+0000 7f74ec522640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74e4101bc0 0x7f74e4197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:48.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.762+0000 7f74ec522640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74e4105a90 0x7f74e419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:48.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.762+0000 7f74ec522640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f74e4110fc0 con 0x7f74e4101240 2024-09-11T22:50:48.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.762+0000 7f74ec522640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f74e4110e40 con 0x7f74e4101bc0 2024-09-11T22:50:48.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.762+0000 7f74ec522640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74e4111140 con 0x7f74e4105a90 2024-09-11T22:50:48.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.762+0000 7f74ea2be640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74e4101240 0x7f74e41973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:48.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.762+0000 7f74ea2be640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74e4101240 0x7f74e41973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39006/0 (socket says 172.21.15.152:39006) 2024-09-11T22:50:48.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.762+0000 7f74ea2be640 1 -- 172.21.15.152:0/516418908 learned_addr learned my addr 172.21.15.152:0/516418908 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:48.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.763+0000 7f74e9abd640 1 --2- 172.21.15.152:0/516418908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74e4101bc0 0x7f74e4197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:48.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.763+0000 7f74eaabf640 1 --2- 172.21.15.152:0/516418908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74e4105a90 0x7f74e419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:48.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.763+0000 7f74ea2be640 1 -- 172.21.15.152:0/516418908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74e4105a90 msgr2=0x7f74e419bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:48.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.763+0000 7f74ea2be640 1 --2- 172.21.15.152:0/516418908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74e4105a90 0x7f74e419bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:48.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.763+0000 7f74ea2be640 1 -- 172.21.15.152:0/516418908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74e4101bc0 msgr2=0x7f74e4197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:48.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.763+0000 7f74ea2be640 1 --2- 172.21.15.152:0/516418908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74e4101bc0 0x7f74e4197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:48.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.763+0000 7f74ea2be640 1 -- 172.21.15.152:0/516418908 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e419c3a0 con 0x7f74e4101240 2024-09-11T22:50:48.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.763+0000 7f74eaabf640 1 --2- 172.21.15.152:0/516418908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74e4105a90 0x7f74e419bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:50:48.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.763+0000 7f74ea2be640 1 --2- 172.21.15.152:0/516418908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74e4101240 0x7f74e41973f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f74d402aa00 tx=0x7f74d4003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:48.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.763+0000 7f74db7fe640 1 -- 172.21.15.152:0/516418908 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d403f070 con 0x7f74e4101240 2024-09-11T22:50:48.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.763+0000 7f74db7fe640 1 -- 172.21.15.152:0/516418908 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74d4031c20 con 0x7f74e4101240 2024-09-11T22:50:48.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.763+0000 7f74ec522640 1 -- 172.21.15.152:0/516418908 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e419c630 con 0x7f74e4101240 2024-09-11T22:50:48.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.763+0000 7f74db7fe640 1 -- 172.21.15.152:0/516418908 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4035030 con 0x7f74e4101240 2024-09-11T22:50:48.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.763+0000 7f74ec522640 1 -- 172.21.15.152:0/516418908 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e41a3f70 con 0x7f74e4101240 2024-09-11T22:50:48.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.764+0000 7f74ec522640 1 -- 172.21.15.152:0/516418908 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74e4102250 con 0x7f74e4101240 2024-09-11T22:50:48.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.765+0000 7f74db7fe640 1 -- 172.21.15.152:0/516418908 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f74d4042070 con 0x7f74e4101240 2024-09-11T22:50:48.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.766+0000 7f74db7fe640 1 --2- 172.21.15.152:0/516418908 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f74d00a2b80 0x7f74d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:48.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.766+0000 7f74e9abd640 1 --2- 172.21.15.152:0/516418908 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f74d00a2b80 0x7f74d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:48.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.766+0000 7f74db7fe640 1 -- 172.21.15.152:0/516418908 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f74d40ef390 con 0x7f74e4101240 2024-09-11T22:50:48.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.766+0000 7f74e9abd640 1 --2- 172.21.15.152:0/516418908 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f74d00a2b80 0x7f74d00a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f74e40ff730 tx=0x7f74e000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:48.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.772+0000 7f74db7fe640 1 -- 172.21.15.152:0/516418908 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74d40bc790 con 0x7f74e4101240 2024-09-11T22:50:48.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.941+0000 7f74ec522640 1 -- 172.21.15.152:0/516418908 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f74e41a4140 con 0x7f74e4101240 2024-09-11T22:50:48.944 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.942+0000 7f74db7fe640 1 -- 172.21.15.152:0/516418908 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7f74d40bc090 con 0x7f74e4101240 2024-09-11T22:50:48.944 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:50:48.944 INFO:teuthology.orchestra.run.smithi152.stdout:[ 2024-09-11T22:50:48.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T22:50:48.945 INFO:teuthology.orchestra.run.smithi152.stdout: "poolnum": 1, 2024-09-11T22:50:48.945 INFO:teuthology.orchestra.run.smithi152.stdout: "poolname": ".mgr" 2024-09-11T22:50:48.945 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:50:48.946 INFO:teuthology.orchestra.run.smithi152.stdout:] 2024-09-11T22:50:48.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.944+0000 7f74ec522640 1 -- 172.21.15.152:0/516418908 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f74d00a2b80 msgr2=0x7f74d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:48.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.944+0000 7f74ec522640 1 --2- 172.21.15.152:0/516418908 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f74d00a2b80 0x7f74d00a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f74e40ff730 tx=0x7f74e000b390 comp rx=0 tx=0).stop 2024-09-11T22:50:48.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.944+0000 7f74ec522640 1 -- 172.21.15.152:0/516418908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74e4101240 msgr2=0x7f74e41973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:48.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.944+0000 7f74ec522640 1 --2- 172.21.15.152:0/516418908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74e4101240 0x7f74e41973f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f74d402aa00 tx=0x7f74d4003b60 comp rx=0 tx=0).stop 2024-09-11T22:50:48.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.944+0000 7f74ec522640 1 -- 172.21.15.152:0/516418908 shutdown_connections 2024-09-11T22:50:48.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.944+0000 7f74ec522640 1 --2- 172.21.15.152:0/516418908 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f74d00a2b80 0x7f74d00a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:48.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.944+0000 7f74ec522640 1 --2- 172.21.15.152:0/516418908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74e4105a90 0x7f74e419bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:48.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.944+0000 7f74ec522640 1 --2- 172.21.15.152:0/516418908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74e4101bc0 0x7f74e4197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:48.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.944+0000 7f74ec522640 1 --2- 172.21.15.152:0/516418908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f74e4101240 0x7f74e41973f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:48.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.944+0000 7f74ec522640 1 -- 172.21.15.152:0/516418908 >> 172.21.15.152:0/516418908 conn(0x7f74e4077210 msgr2=0x7f74e4103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:48.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.944+0000 7f74ec522640 1 -- 172.21.15.152:0/516418908 shutdown_connections 2024-09-11T22:50:48.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:48.944+0000 7f74ec522640 1 -- 172.21.15.152:0/516418908 wait complete. 2024-09-11T22:50:49.674 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T22:50:49.675 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T22:50:49.925 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:50.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:49 smithi191 ceph-mon[35211]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:50.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:49 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/516418908' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T22:50:50.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:49 smithi152 ceph-mon[31181]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:50.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:49 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/516418908' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T22:50:50.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:49 smithi204 ceph-mon[35454]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:50.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:49 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/516418908' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T22:50:50.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.422+0000 7f1d78efc640 1 -- 172.21.15.152:0/1099297858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d74102570 msgr2=0x7f1d741029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:50.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.422+0000 7f1d78efc640 1 --2- 172.21.15.152:0/1099297858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d74102570 0x7f1d741029b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009900 tx=0x7f1d6802a480 comp rx=0 tx=0).stop 2024-09-11T22:50:50.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.422+0000 7f1d78efc640 1 -- 172.21.15.152:0/1099297858 shutdown_connections 2024-09-11T22:50:50.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.422+0000 7f1d78efc640 1 --2- 172.21.15.152:0/1099297858 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d74102ef0 0x7f1d7410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:50.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.422+0000 7f1d78efc640 1 --2- 172.21.15.152:0/1099297858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d74102570 0x7f1d741029b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:50.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.422+0000 7f1d78efc640 1 --2- 172.21.15.152:0/1099297858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d74100e20 0x7f1d74101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:50.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.422+0000 7f1d78efc640 1 -- 172.21.15.152:0/1099297858 >> 172.21.15.152:0/1099297858 conn(0x7f1d740fc7d0 msgr2=0x7f1d740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:50.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.423+0000 7f1d78efc640 1 -- 172.21.15.152:0/1099297858 shutdown_connections 2024-09-11T22:50:50.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.423+0000 7f1d78efc640 1 -- 172.21.15.152:0/1099297858 wait complete. 2024-09-11T22:50:50.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.423+0000 7f1d78efc640 1 Processor -- start 2024-09-11T22:50:50.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.423+0000 7f1d78efc640 1 -- start start 2024-09-11T22:50:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.423+0000 7f1d78efc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d74100e20 0x7f1d740fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.423+0000 7f1d78efc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d74102570 0x7f1d740fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d78efc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d74102ef0 0x7f1d740fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d78efc640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1d74110f50 con 0x7f1d74100e20 2024-09-11T22:50:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d78efc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1d74110dd0 con 0x7f1d74102ef0 2024-09-11T22:50:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d78efc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741110d0 con 0x7f1d74102570 2024-09-11T22:50:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d7259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d74100e20 0x7f1d740fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d7259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d74100e20 0x7f1d740fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39024/0 (socket says 172.21.15.152:39024) 2024-09-11T22:50:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d7259c640 1 -- 172.21.15.152:0/772086300 learned_addr learned my addr 172.21.15.152:0/772086300 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d71d9b640 1 --2- 172.21.15.152:0/772086300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d74102570 0x7f1d740fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d72d9d640 1 --2- 172.21.15.152:0/772086300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d74102ef0 0x7f1d740fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d7259c640 1 -- 172.21.15.152:0/772086300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d74102570 msgr2=0x7f1d740fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:50.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d7259c640 1 --2- 172.21.15.152:0/772086300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d74102570 0x7f1d740fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:50.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d7259c640 1 -- 172.21.15.152:0/772086300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d74102ef0 msgr2=0x7f1d740fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:50.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d7259c640 1 --2- 172.21.15.152:0/772086300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d74102ef0 0x7f1d740fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:50.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d7259c640 1 -- 172.21.15.152:0/772086300 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d741a6a40 con 0x7f1d74100e20 2024-09-11T22:50:50.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d7259c640 1 --2- 172.21.15.152:0/772086300 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d74100e20 0x7f1d740fcc80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400cce0 tx=0x7f1d6400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:50.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d637fe640 1 -- 172.21.15.152:0/772086300 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64004b20 con 0x7f1d74100e20 2024-09-11T22:50:50.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d637fe640 1 -- 172.21.15.152:0/772086300 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d64004cc0 con 0x7f1d74100e20 2024-09-11T22:50:50.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d78efc640 1 -- 172.21.15.152:0/772086300 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d741a6d30 con 0x7f1d74100e20 2024-09-11T22:50:50.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d637fe640 1 -- 172.21.15.152:0/772086300 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6400f7d0 con 0x7f1d74100e20 2024-09-11T22:50:50.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.424+0000 7f1d78efc640 1 -- 172.21.15.152:0/772086300 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d741a7210 con 0x7f1d74100e20 2024-09-11T22:50:50.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.425+0000 7f1d78efc640 1 -- 172.21.15.152:0/772086300 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d74101200 con 0x7f1d74100e20 2024-09-11T22:50:50.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.426+0000 7f1d637fe640 1 -- 172.21.15.152:0/772086300 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1d6400f970 con 0x7f1d74100e20 2024-09-11T22:50:50.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.427+0000 7f1d637fe640 1 --2- 172.21.15.152:0/772086300 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1d5c0a2b80 0x7f1d5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:50.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.427+0000 7f1d71d9b640 1 --2- 172.21.15.152:0/772086300 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1d5c0a2b80 0x7f1d5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:50.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.427+0000 7f1d637fe640 1 -- 172.21.15.152:0/772086300 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f1d640ced00 con 0x7f1d74100e20 2024-09-11T22:50:50.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.427+0000 7f1d71d9b640 1 --2- 172.21.15.152:0/772086300 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1d5c0a2b80 0x7f1d5c0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1d740fe710 tx=0x7f1d68005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:50.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.431+0000 7f1d637fe640 1 -- 172.21.15.152:0/772086300 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d64014030 con 0x7f1d74100e20 2024-09-11T22:50:50.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:50.598+0000 7f1d78efc640 1 -- 172.21.15.152:0/772086300 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1d7410a0a0 con 0x7f1d74100e20 2024-09-11T22:50:51.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:51.123+0000 7f1d637fe640 1 -- 172.21.15.152:0/772086300 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7f1d6409c100 con 0x7f1d74100e20 2024-09-11T22:50:51.125 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' created 2024-09-11T22:50:51.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:51.125+0000 7f1d78efc640 1 -- 172.21.15.152:0/772086300 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1d5c0a2b80 msgr2=0x7f1d5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:51.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:51.125+0000 7f1d78efc640 1 --2- 172.21.15.152:0/772086300 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1d5c0a2b80 0x7f1d5c0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1d740fe710 tx=0x7f1d68005a20 comp rx=0 tx=0).stop 2024-09-11T22:50:51.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:51.125+0000 7f1d78efc640 1 -- 172.21.15.152:0/772086300 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d74100e20 msgr2=0x7f1d740fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:51.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:51.125+0000 7f1d78efc640 1 --2- 172.21.15.152:0/772086300 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d74100e20 0x7f1d740fcc80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400cce0 tx=0x7f1d6400a620 comp rx=0 tx=0).stop 2024-09-11T22:50:51.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:51.126+0000 7f1d78efc640 1 -- 172.21.15.152:0/772086300 shutdown_connections 2024-09-11T22:50:51.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:51.126+0000 7f1d78efc640 1 --2- 172.21.15.152:0/772086300 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1d5c0a2b80 0x7f1d5c0a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:51.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:51.126+0000 7f1d78efc640 1 --2- 172.21.15.152:0/772086300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d74102ef0 0x7f1d740fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:51.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:51.126+0000 7f1d78efc640 1 --2- 172.21.15.152:0/772086300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d74102570 0x7f1d740fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:51.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:51.126+0000 7f1d78efc640 1 --2- 172.21.15.152:0/772086300 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d74100e20 0x7f1d740fcc80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:51.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:51.126+0000 7f1d78efc640 1 -- 172.21.15.152:0/772086300 >> 172.21.15.152:0/772086300 conn(0x7f1d740fc7d0 msgr2=0x7f1d7406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:51.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:51.126+0000 7f1d78efc640 1 -- 172.21.15.152:0/772086300 shutdown_connections 2024-09-11T22:50:51.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:51.126+0000 7f1d78efc640 1 -- 172.21.15.152:0/772086300 wait complete. 2024-09-11T22:50:51.503 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:51 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/772086300' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T22:50:51.503 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:51 smithi152 ceph-mon[31181]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:51.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:51 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/772086300' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T22:50:51.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:51 smithi191 ceph-mon[35211]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:51.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:51 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/772086300' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T22:50:51.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:51 smithi204 ceph-mon[35454]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:51.863 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T22:50:52.115 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:52.396 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:52 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/772086300' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T22:50:52.397 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:52 smithi152 ceph-mon[31181]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T22:50:52.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:52 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/772086300' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T22:50:52.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:52 smithi191 ceph-mon[35211]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T22:50:52.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:52 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/772086300' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T22:50:52.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:52 smithi204 ceph-mon[35454]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T22:50:52.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.635+0000 7fbc952e7640 1 -- 172.21.15.152:0/2723201761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc90102d40 msgr2=0x7fbc90103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:52.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.635+0000 7fbc952e7640 1 --2- 172.21.15.152:0/2723201761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc90102d40 0x7fbc90103120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009900 tx=0x7fbc8002a480 comp rx=0 tx=0).stop 2024-09-11T22:50:52.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.635+0000 7fbc952e7640 1 -- 172.21.15.152:0/2723201761 shutdown_connections 2024-09-11T22:50:52.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.635+0000 7fbc952e7640 1 --2- 172.21.15.152:0/2723201761 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc9010c170 0x7fbc9010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:52.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.635+0000 7fbc952e7640 1 --2- 172.21.15.152:0/2723201761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc90103660 0x7fbc9010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:52.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.635+0000 7fbc952e7640 1 --2- 172.21.15.152:0/2723201761 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc90102d40 0x7fbc90103120 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:52.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.635+0000 7fbc952e7640 1 -- 172.21.15.152:0/2723201761 >> 172.21.15.152:0/2723201761 conn(0x7fbc900fc7a0 msgr2=0x7fbc900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:52.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.635+0000 7fbc952e7640 1 -- 172.21.15.152:0/2723201761 shutdown_connections 2024-09-11T22:50:52.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.635+0000 7fbc952e7640 1 -- 172.21.15.152:0/2723201761 wait complete. 2024-09-11T22:50:52.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.636+0000 7fbc952e7640 1 Processor -- start 2024-09-11T22:50:52.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.636+0000 7fbc952e7640 1 -- start start 2024-09-11T22:50:52.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.636+0000 7fbc952e7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc90102d40 0x7fbc901984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:52.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.636+0000 7fbc952e7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc90103660 0x7fbc90198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:52.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.636+0000 7fbc952e7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc9010c170 0x7fbc9019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:52.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.636+0000 7fbc952e7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbc90111f70 con 0x7fbc90102d40 2024-09-11T22:50:52.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.636+0000 7fbc952e7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbc90111df0 con 0x7fbc90103660 2024-09-11T22:50:52.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.636+0000 7fbc952e7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901120f0 con 0x7fbc9010c170 2024-09-11T22:50:52.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.636+0000 7fbc8effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc90102d40 0x7fbc901984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:52.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.637+0000 7fbc8effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc90102d40 0x7fbc901984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39050/0 (socket says 172.21.15.152:39050) 2024-09-11T22:50:52.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.637+0000 7fbc8effd640 1 -- 172.21.15.152:0/1622790913 learned_addr learned my addr 172.21.15.152:0/1622790913 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:52.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.637+0000 7fbc8e7fc640 1 --2- 172.21.15.152:0/1622790913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc90103660 0x7fbc90198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:52.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.637+0000 7fbc8f7fe640 1 --2- 172.21.15.152:0/1622790913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc9010c170 0x7fbc9019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:52.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.637+0000 7fbc8effd640 1 -- 172.21.15.152:0/1622790913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc9010c170 msgr2=0x7fbc9019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:52.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.637+0000 7fbc8effd640 1 --2- 172.21.15.152:0/1622790913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc9010c170 0x7fbc9019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:52.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.637+0000 7fbc8effd640 1 -- 172.21.15.152:0/1622790913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc90103660 msgr2=0x7fbc90198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:52.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.637+0000 7fbc8effd640 1 --2- 172.21.15.152:0/1622790913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc90103660 0x7fbc90198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:52.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.637+0000 7fbc8effd640 1 -- 172.21.15.152:0/1622790913 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9019d490 con 0x7fbc90102d40 2024-09-11T22:50:52.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.637+0000 7fbc8effd640 1 --2- 172.21.15.152:0/1622790913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc90102d40 0x7fbc901984e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbc80005ab0 tx=0x7fbc800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:52.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.637+0000 7fbc77fff640 1 -- 172.21.15.152:0/1622790913 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8003f030 con 0x7fbc90102d40 2024-09-11T22:50:52.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.637+0000 7fbc952e7640 1 -- 172.21.15.152:0/1622790913 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9019d720 con 0x7fbc90102d40 2024-09-11T22:50:52.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.637+0000 7fbc77fff640 1 -- 172.21.15.152:0/1622790913 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc80003ce0 con 0x7fbc90102d40 2024-09-11T22:50:52.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.637+0000 7fbc77fff640 1 -- 172.21.15.152:0/1622790913 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc80035070 con 0x7fbc90102d40 2024-09-11T22:50:52.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.637+0000 7fbc952e7640 1 -- 172.21.15.152:0/1622790913 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc901a5060 con 0x7fbc90102d40 2024-09-11T22:50:52.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.638+0000 7fbc952e7640 1 -- 172.21.15.152:0/1622790913 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc90103120 con 0x7fbc90102d40 2024-09-11T22:50:52.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.640+0000 7fbc77fff640 1 -- 172.21.15.152:0/1622790913 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbc80042030 con 0x7fbc90102d40 2024-09-11T22:50:52.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.641+0000 7fbc77fff640 1 --2- 172.21.15.152:0/1622790913 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbc780a2b80 0x7fbc780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:52.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.641+0000 7fbc8e7fc640 1 --2- 172.21.15.152:0/1622790913 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbc780a2b80 0x7fbc780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:52.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.641+0000 7fbc77fff640 1 -- 172.21.15.152:0/1622790913 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7497+0+0 (secure 0 0 0) 0x7fbc800ef9f0 con 0x7fbc90102d40 2024-09-11T22:50:52.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.641+0000 7fbc8e7fc640 1 --2- 172.21.15.152:0/1622790913 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbc780a2b80 0x7fbc780a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbc900fee30 tx=0x7fbc84005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:52.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.644+0000 7fbc77fff640 1 -- 172.21.15.152:0/1622790913 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc800bcc80 con 0x7fbc90102d40 2024-09-11T22:50:52.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:52.812+0000 7fbc952e7640 1 -- 172.21.15.152:0/1622790913 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbc901071e0 con 0x7fbc90102d40 2024-09-11T22:50:53.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:53.137+0000 7fbc77fff640 1 -- 172.21.15.152:0/1622790913 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) ==== 170+0+0 (secure 0 0 0) 0x7fbc800bc580 con 0x7fbc90102d40 2024-09-11T22:50:53.139 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' created 2024-09-11T22:50:53.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:53.140+0000 7fbc952e7640 1 -- 172.21.15.152:0/1622790913 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbc780a2b80 msgr2=0x7fbc780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:53.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:53.140+0000 7fbc952e7640 1 --2- 172.21.15.152:0/1622790913 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbc780a2b80 0x7fbc780a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbc900fee30 tx=0x7fbc84005ec0 comp rx=0 tx=0).stop 2024-09-11T22:50:53.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:53.140+0000 7fbc952e7640 1 -- 172.21.15.152:0/1622790913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc90102d40 msgr2=0x7fbc901984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:53.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:53.140+0000 7fbc952e7640 1 --2- 172.21.15.152:0/1622790913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc90102d40 0x7fbc901984e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbc80005ab0 tx=0x7fbc800042f0 comp rx=0 tx=0).stop 2024-09-11T22:50:53.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:53.140+0000 7fbc952e7640 1 -- 172.21.15.152:0/1622790913 shutdown_connections 2024-09-11T22:50:53.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:53.140+0000 7fbc952e7640 1 --2- 172.21.15.152:0/1622790913 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbc780a2b80 0x7fbc780a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:53.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:53.140+0000 7fbc952e7640 1 --2- 172.21.15.152:0/1622790913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc9010c170 0x7fbc9019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:53.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:53.140+0000 7fbc952e7640 1 --2- 172.21.15.152:0/1622790913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc90103660 0x7fbc90198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:53.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:53.140+0000 7fbc952e7640 1 --2- 172.21.15.152:0/1622790913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc90102d40 0x7fbc901984e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:53.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:53.140+0000 7fbc952e7640 1 -- 172.21.15.152:0/1622790913 >> 172.21.15.152:0/1622790913 conn(0x7fbc900fc7a0 msgr2=0x7fbc900fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:53.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:53.140+0000 7fbc952e7640 1 -- 172.21.15.152:0/1622790913 shutdown_connections 2024-09-11T22:50:53.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:53.140+0000 7fbc952e7640 1 -- 172.21.15.152:0/1622790913 wait complete. 2024-09-11T22:50:53.424 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:53 smithi152 ceph-mon[31181]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T22:50:53.424 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:53 smithi152 ceph-mon[31181]: pgmap v277: 33 pgs: 3 creating+peering, 21 unknown, 9 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:53.425 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:53 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1622790913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T22:50:53.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:53 smithi191 ceph-mon[35211]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T22:50:53.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:53 smithi191 ceph-mon[35211]: pgmap v277: 33 pgs: 3 creating+peering, 21 unknown, 9 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:53.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:53 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1622790913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T22:50:53.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:53 smithi204 ceph-mon[35454]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T22:50:53.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:53 smithi204 ceph-mon[35454]: pgmap v277: 33 pgs: 3 creating+peering, 21 unknown, 9 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:53.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:53 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1622790913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T22:50:54.033 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T22:50:54.304 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:54.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:54 smithi152 ceph-mon[31181]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T22:50:54.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:54 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1622790913' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T22:50:54.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:54 smithi152 ceph-mon[31181]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T22:50:54.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:54 smithi191 ceph-mon[35211]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T22:50:54.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:54 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1622790913' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T22:50:54.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:54 smithi191 ceph-mon[35211]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T22:50:54.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:54 smithi204 ceph-mon[35454]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T22:50:54.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:54 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1622790913' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T22:50:54.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:54 smithi204 ceph-mon[35454]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T22:50:54.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.833+0000 7f920e560640 1 -- 172.21.15.152:0/3846038035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9208100e20 msgr2=0x7f9208101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:54.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.833+0000 7f920e560640 1 --2- 172.21.15.152:0/3846038035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9208100e20 0x7f9208101200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009900 tx=0x7f91f802a480 comp rx=0 tx=0).stop 2024-09-11T22:50:54.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.833+0000 7f920e560640 1 -- 172.21.15.152:0/3846038035 shutdown_connections 2024-09-11T22:50:54.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.833+0000 7f920e560640 1 --2- 172.21.15.152:0/3846038035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9208102ef0 0x7f920810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:54.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.833+0000 7f920e560640 1 --2- 172.21.15.152:0/3846038035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9208102570 0x7f92081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:54.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.833+0000 7f920e560640 1 --2- 172.21.15.152:0/3846038035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9208100e20 0x7f9208101200 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:54.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.833+0000 7f920e560640 1 -- 172.21.15.152:0/3846038035 >> 172.21.15.152:0/3846038035 conn(0x7f92080fc7d0 msgr2=0x7f92080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:54.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.833+0000 7f920e560640 1 -- 172.21.15.152:0/3846038035 shutdown_connections 2024-09-11T22:50:54.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.833+0000 7f920e560640 1 -- 172.21.15.152:0/3846038035 wait complete. 2024-09-11T22:50:54.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.833+0000 7f920e560640 1 Processor -- start 2024-09-11T22:50:54.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.833+0000 7f920e560640 1 -- start start 2024-09-11T22:50:54.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f920e560640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9208100e20 0x7f9208197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:54.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f920e560640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9208102570 0x7f9208197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:54.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f920e560640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9208102ef0 0x7f920819be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:54.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f920e560640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9208111020 con 0x7f9208100e20 2024-09-11T22:50:54.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f920e560640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9208110ea0 con 0x7f9208102570 2024-09-11T22:50:54.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f920e560640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f92081111a0 con 0x7f9208102ef0 2024-09-11T22:50:54.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f9207fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9208100e20 0x7f9208197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:54.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f9207fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9208100e20 0x7f9208197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39064/0 (socket says 172.21.15.152:39064) 2024-09-11T22:50:54.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f9207fff640 1 -- 172.21.15.152:0/116982588 learned_addr learned my addr 172.21.15.152:0/116982588 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:54.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f920cafd640 1 --2- 172.21.15.152:0/116982588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9208102ef0 0x7f920819be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:54.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f9207fff640 1 -- 172.21.15.152:0/116982588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9208102ef0 msgr2=0x7f920819be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:54.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f91fffff640 1 --2- 172.21.15.152:0/116982588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9208102570 0x7f9208197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:54.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f9207fff640 1 --2- 172.21.15.152:0/116982588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9208102ef0 0x7f920819be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:54.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f9207fff640 1 -- 172.21.15.152:0/116982588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9208102570 msgr2=0x7f9208197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:54.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f9207fff640 1 --2- 172.21.15.152:0/116982588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9208102570 0x7f9208197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:54.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f9207fff640 1 -- 172.21.15.152:0/116982588 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920819c530 con 0x7f9208100e20 2024-09-11T22:50:54.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f91fffff640 1 --2- 172.21.15.152:0/116982588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9208102570 0x7f9208197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:50:54.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f9207fff640 1 --2- 172.21.15.152:0/116982588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9208100e20 0x7f9208197580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f91f8006010 tx=0x7f91f8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:54.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f9205ffb640 1 -- 172.21.15.152:0/116982588 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f803f070 con 0x7f9208100e20 2024-09-11T22:50:54.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f920e560640 1 -- 172.21.15.152:0/116982588 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920819c7c0 con 0x7f9208100e20 2024-09-11T22:50:54.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f9205ffb640 1 -- 172.21.15.152:0/116982588 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f8031e30 con 0x7f9208100e20 2024-09-11T22:50:54.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f9205ffb640 1 -- 172.21.15.152:0/116982588 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8035030 con 0x7f9208100e20 2024-09-11T22:50:54.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.834+0000 7f920e560640 1 -- 172.21.15.152:0/116982588 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92081a4100 con 0x7f9208100e20 2024-09-11T22:50:54.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.835+0000 7f920e560640 1 -- 172.21.15.152:0/116982588 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9208101280 con 0x7f9208100e20 2024-09-11T22:50:54.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.837+0000 7f9205ffb640 1 -- 172.21.15.152:0/116982588 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f91f8042070 con 0x7f9208100e20 2024-09-11T22:50:54.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.837+0000 7f9205ffb640 1 --2- 172.21.15.152:0/116982588 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f91f40a2b80 0x7f91f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:54.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.837+0000 7f91fffff640 1 --2- 172.21.15.152:0/116982588 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f91f40a2b80 0x7f91f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:54.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.837+0000 7f9205ffb640 1 -- 172.21.15.152:0/116982588 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7879+0+0 (secure 0 0 0) 0x7f91f80efe80 con 0x7f9208100e20 2024-09-11T22:50:54.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.838+0000 7f91fffff640 1 --2- 172.21.15.152:0/116982588 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f91f40a2b80 0x7f91f40a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f91f0006fd0 tx=0x7f91f0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:54.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:54.841+0000 7f9205ffb640 1 -- 172.21.15.152:0/116982588 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f91f80bd020 con 0x7f9208100e20 2024-09-11T22:50:55.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:55.045+0000 7f920e560640 1 -- 172.21.15.152:0/116982588 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9208109090 con 0x7f9208100e20 2024-09-11T22:50:55.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:55.152+0000 7f9205ffb640 1 -- 172.21.15.152:0/116982588 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f91f80bc920 con 0x7f9208100e20 2024-09-11T22:50:55.154 INFO:teuthology.orchestra.run.smithi152.stderr:new fs with metadata pool 2 and data pool 3 2024-09-11T22:50:55.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:55.154+0000 7f920e560640 1 -- 172.21.15.152:0/116982588 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f91f40a2b80 msgr2=0x7f91f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:55.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:55.154+0000 7f920e560640 1 --2- 172.21.15.152:0/116982588 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f91f40a2b80 0x7f91f40a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f91f0006fd0 tx=0x7f91f0008040 comp rx=0 tx=0).stop 2024-09-11T22:50:55.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:55.154+0000 7f920e560640 1 -- 172.21.15.152:0/116982588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9208100e20 msgr2=0x7f9208197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:55.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:55.154+0000 7f920e560640 1 --2- 172.21.15.152:0/116982588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9208100e20 0x7f9208197580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f91f8006010 tx=0x7f91f8003c80 comp rx=0 tx=0).stop 2024-09-11T22:50:55.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:55.154+0000 7f920e560640 1 -- 172.21.15.152:0/116982588 shutdown_connections 2024-09-11T22:50:55.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:55.154+0000 7f920e560640 1 --2- 172.21.15.152:0/116982588 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f91f40a2b80 0x7f91f40a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:55.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:55.154+0000 7f920e560640 1 --2- 172.21.15.152:0/116982588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9208102ef0 0x7f920819be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:55.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:55.154+0000 7f920e560640 1 --2- 172.21.15.152:0/116982588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9208102570 0x7f9208197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:55.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:55.154+0000 7f920e560640 1 --2- 172.21.15.152:0/116982588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9208100e20 0x7f9208197580 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:55.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:55.154+0000 7f920e560640 1 -- 172.21.15.152:0/116982588 >> 172.21.15.152:0/116982588 conn(0x7f92080fc7d0 msgr2=0x7f92080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:55.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:55.155+0000 7f920e560640 1 -- 172.21.15.152:0/116982588 shutdown_connections 2024-09-11T22:50:55.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:55.155+0000 7f920e560640 1 -- 172.21.15.152:0/116982588 wait complete. 2024-09-11T22:50:55.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:55 smithi152 ceph-mon[31181]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T22:50:55.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:55 smithi152 ceph-mon[31181]: pgmap v280: 97 pgs: 7 creating+peering, 65 unknown, 25 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:55.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:55 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/116982588' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T22:50:55.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:55 smithi152 ceph-mon[31181]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T22:50:55.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:55 smithi191 ceph-mon[35211]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T22:50:55.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:55 smithi191 ceph-mon[35211]: pgmap v280: 97 pgs: 7 creating+peering, 65 unknown, 25 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:55.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:55 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/116982588' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T22:50:55.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:55 smithi191 ceph-mon[35211]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T22:50:55.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:55 smithi204 ceph-mon[35454]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T22:50:55.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:55 smithi204 ceph-mon[35454]: pgmap v280: 97 pgs: 7 creating+peering, 65 unknown, 25 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:50:55.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:55 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/116982588' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T22:50:55.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:55 smithi204 ceph-mon[35454]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T22:50:55.843 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:50:56.089 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:56.411 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:56 smithi152 ceph-mon[31181]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T22:50:56.412 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:56 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/116982588' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T22:50:56.412 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:56 smithi152 ceph-mon[31181]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T22:50:56.412 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:56 smithi152 ceph-mon[31181]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T22:50:56.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:56 smithi191 ceph-mon[35211]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T22:50:56.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:56 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/116982588' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T22:50:56.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:56 smithi191 ceph-mon[35211]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T22:50:56.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:56 smithi191 ceph-mon[35211]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T22:50:56.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:56 smithi204 ceph-mon[35454]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T22:50:56.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:56 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/116982588' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T22:50:56.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:56 smithi204 ceph-mon[35454]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T22:50:56.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:56 smithi204 ceph-mon[35454]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T22:50:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.594+0000 7f4b961d7640 1 -- 172.21.15.152:0/2041852209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b90103660 msgr2=0x7f4b9010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.594+0000 7f4b961d7640 1 --2- 172.21.15.152:0/2041852209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b90103660 0x7f4b9010bb50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009900 tx=0x7f4b7802a480 comp rx=0 tx=0).stop 2024-09-11T22:50:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.594+0000 7f4b961d7640 1 -- 172.21.15.152:0/2041852209 shutdown_connections 2024-09-11T22:50:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.594+0000 7f4b961d7640 1 --2- 172.21.15.152:0/2041852209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b9010c170 0x7f4b9010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.594+0000 7f4b961d7640 1 --2- 172.21.15.152:0/2041852209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b90103660 0x7f4b9010bb50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.594+0000 7f4b961d7640 1 --2- 172.21.15.152:0/2041852209 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b90102d40 0x7f4b90103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.594+0000 7f4b961d7640 1 -- 172.21.15.152:0/2041852209 >> 172.21.15.152:0/2041852209 conn(0x7f4b900fc7a0 msgr2=0x7f4b900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.594+0000 7f4b961d7640 1 -- 172.21.15.152:0/2041852209 shutdown_connections 2024-09-11T22:50:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.594+0000 7f4b961d7640 1 -- 172.21.15.152:0/2041852209 wait complete. 2024-09-11T22:50:56.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.595+0000 7f4b961d7640 1 Processor -- start 2024-09-11T22:50:56.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.595+0000 7f4b961d7640 1 -- start start 2024-09-11T22:50:56.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.595+0000 7f4b961d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b90102d40 0x7f4b901009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:56.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.595+0000 7f4b961d7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b90103660 0x7f4b90100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:56.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.595+0000 7f4b961d7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b9010c170 0x7f4b90101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:56.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.595+0000 7f4b961d7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4b90111fd0 con 0x7f4b90103660 2024-09-11T22:50:56.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.595+0000 7f4b961d7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4b90111e50 con 0x7f4b9010c170 2024-09-11T22:50:56.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.595+0000 7f4b961d7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4b90112150 con 0x7f4b90102d40 2024-09-11T22:50:56.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.595+0000 7f4b86ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b90103660 0x7f4b90100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:56.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.595+0000 7f4b86ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b90103660 0x7f4b90100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39072/0 (socket says 172.21.15.152:39072) 2024-09-11T22:50:56.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.595+0000 7f4b86ffd640 1 -- 172.21.15.152:0/4127475364 learned_addr learned my addr 172.21.15.152:0/4127475364 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:56.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.595+0000 7f4b8f7fe640 1 --2- 172.21.15.152:0/4127475364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b90102d40 0x7f4b901009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:56.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.596+0000 7f4b8ffff640 1 --2- 172.21.15.152:0/4127475364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b9010c170 0x7f4b90101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:56.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.596+0000 7f4b86ffd640 1 -- 172.21.15.152:0/4127475364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b90102d40 msgr2=0x7f4b901009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:56.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.596+0000 7f4b86ffd640 1 --2- 172.21.15.152:0/4127475364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b90102d40 0x7f4b901009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:56.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.596+0000 7f4b86ffd640 1 -- 172.21.15.152:0/4127475364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b9010c170 msgr2=0x7f4b90101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:56.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.596+0000 7f4b86ffd640 1 --2- 172.21.15.152:0/4127475364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b9010c170 0x7f4b90101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:56.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.596+0000 7f4b86ffd640 1 -- 172.21.15.152:0/4127475364 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b901a6990 con 0x7f4b90103660 2024-09-11T22:50:56.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.596+0000 7f4b86ffd640 1 --2- 172.21.15.152:0/4127475364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b90103660 0x7f4b90100ee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4b78005ab0 tx=0x7f4b780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:56.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.596+0000 7f4b8d7fa640 1 -- 172.21.15.152:0/4127475364 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7803f030 con 0x7f4b90103660 2024-09-11T22:50:56.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.596+0000 7f4b8d7fa640 1 -- 172.21.15.152:0/4127475364 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b78003ce0 con 0x7f4b90103660 2024-09-11T22:50:56.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.596+0000 7f4b961d7640 1 -- 172.21.15.152:0/4127475364 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b901a6c20 con 0x7f4b90103660 2024-09-11T22:50:56.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.596+0000 7f4b8d7fa640 1 -- 172.21.15.152:0/4127475364 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b78035070 con 0x7f4b90103660 2024-09-11T22:50:56.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.596+0000 7f4b961d7640 1 -- 172.21.15.152:0/4127475364 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b901a7100 con 0x7f4b90103660 2024-09-11T22:50:56.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.597+0000 7f4b961d7640 1 -- 172.21.15.152:0/4127475364 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b90103120 con 0x7f4b90103660 2024-09-11T22:50:56.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.598+0000 7f4b8d7fa640 1 -- 172.21.15.152:0/4127475364 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4b78042030 con 0x7f4b90103660 2024-09-11T22:50:56.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.599+0000 7f4b8d7fa640 1 --2- 172.21.15.152:0/4127475364 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4b7c0a2b80 0x7f4b7c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:56.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.599+0000 7f4b8d7fa640 1 -- 172.21.15.152:0/4127475364 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f4b780efa10 con 0x7f4b90103660 2024-09-11T22:50:56.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.599+0000 7f4b8f7fe640 1 --2- 172.21.15.152:0/4127475364 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4b7c0a2b80 0x7f4b7c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:56.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.599+0000 7f4b8f7fe640 1 --2- 172.21.15.152:0/4127475364 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4b7c0a2b80 0x7f4b7c0a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009890 tx=0x7f4b80008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:56.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.602+0000 7f4b8d7fa640 1 -- 172.21.15.152:0/4127475364 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b780bcac0 con 0x7f4b90103660 2024-09-11T22:50:56.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.771+0000 7f4b961d7640 1 -- 172.21.15.152:0/4127475364 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b9002d7c0 con 0x7f4b90103660 2024-09-11T22:50:56.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.772+0000 7f4b8d7fa640 1 -- 172.21.15.152:0/4127475364 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22363 (secure 0 0 0) 0x7f4b780bc3c0 con 0x7f4b90103660 2024-09-11T22:50:56.774 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:50:56.774 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":81,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","created":"2024-09-11T22:43:00.379317+0000","modified":"2024-09-11T22:50:55.047140+0000","last_up_change":"2024-09-11T22:50:01.043181+0000","last_in_change":"2024-09-11T22:49:38.073870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:46:10.984707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:50:50.833547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:50:53.047604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3cd917b5-77f6-4a4a-8fd0-4ebe3034feac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6803","nonce":1270728007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6805","nonce":1270728007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6809","nonce":1270728007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6807","nonce":1270728007}]},"public_addr":"172.21.15.152:6803/1270728007","cluster_addr":"172.21.15.152:6805/1270728007","heartbeat_back_addr":"172.21.15.152:6809/1270728007","heartbeat_front_addr":"172.21.15.152:6807/1270728007","state":["exists","up"]},{"osd":1,"uuid":"ae589bc4-77b1-4522-9591-b0ce2a5432dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6801","nonce":354467155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6803","nonce":354467155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6807","nonce":354467155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6805","nonce":354467155}]},"public_addr":"172.21.15.191:6801/354467155","cluster_addr":"172.21.15.191:6803/354467155","heartbeat_back_addr":"172.21.15.191:6807/354467155","heartbeat_front_addr":"172.21.15.191:6805/354467155","state":["exists","up"]},{"osd":2,"uuid":"b35ffd0c-d084-43dd-a947-15132d78d59c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6801","nonce":3848396211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6803","nonce":3848396211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6807","nonce":3848396211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6805","nonce":3848396211}]},"public_addr":"172.21.15.204:6801/3848396211","cluster_addr":"172.21.15.204:6803/3848396211","heartbeat_back_addr":"172.21.15.204:6807/3848396211","heartbeat_front_addr":"172.21.15.204:6805/3848396211","state":["exists","up"]},{"osd":3,"uuid":"d66ba7e8-4ab7-449e-9460-2f44ffdf987f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6811","nonce":3351257181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6813","nonce":3351257181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6817","nonce":3351257181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6815","nonce":3351257181}]},"public_addr":"172.21.15.152:6811/3351257181","cluster_addr":"172.21.15.152:6813/3351257181","heartbeat_back_addr":"172.21.15.152:6817/3351257181","heartbeat_front_addr":"172.21.15.152:6815/3351257181","state":["exists","up"]},{"osd":4,"uuid":"e1b56468-6cc7-4d29-988e-9d281118ee36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6809","nonce":2967539563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6811","nonce":2967539563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6815","nonce":2967539563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6813","nonce":2967539563}]},"public_addr":"172.21.15.191:6809/2967539563","cluster_addr":"172.21.15.191:6811/2967539563","heartbeat_back_addr":"172.21.15.191:6815/2967539563","heartbeat_front_addr":"172.21.15.191:6813/2967539563","state":["exists","up"]},{"osd":5,"uuid":"89dabd3e-77e3-4294-9648-b112ef694e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6809","nonce":4283562158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6811","nonce":4283562158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6815","nonce":4283562158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6813","nonce":4283562158}]},"public_addr":"172.21.15.204:6809/4283562158","cluster_addr":"172.21.15.204:6811/4283562158","heartbeat_back_addr":"172.21.15.204:6815/4283562158","heartbeat_front_addr":"172.21.15.204:6813/4283562158","state":["exists","up"]},{"osd":6,"uuid":"46a0fded-b668-4aef-a68f-613cccc83483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6819","nonce":2484234042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6821","nonce":2484234042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6825","nonce":2484234042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6823","nonce":2484234042}]},"public_addr":"172.21.15.152:6819/2484234042","cluster_addr":"172.21.15.152:6821/2484234042","heartbeat_back_addr":"172.21.15.152:6825/2484234042","heartbeat_front_addr":"172.21.15.152:6823/2484234042","state":["exists","up"]},{"osd":7,"uuid":"e674bfe9-e2db-44a3-b74b-a21f198747c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6817","nonce":269608859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6819","nonce":269608859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6823","nonce":269608859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6821","nonce":269608859}]},"public_addr":"172.21.15.191:6817/269608859","cluster_addr":"172.21.15.191:6819/269608859","heartbeat_back_addr":"172.21.15.191:6823/269608859","heartbeat_front_addr":"172.21.15.191:6821/269608859","state":["exists","up"]},{"osd":8,"uuid":"0418bfc9-d6b6-488f-b4c7-0bc6291bb624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6817","nonce":1756190090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6819","nonce":1756190090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6823","nonce":1756190090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6821","nonce":1756190090}]},"public_addr":"172.21.15.204:6817/1756190090","cluster_addr":"172.21.15.204:6819/1756190090","heartbeat_back_addr":"172.21.15.204:6823/1756190090","heartbeat_front_addr":"172.21.15.204:6821/1756190090","state":["exists","up"]},{"osd":9,"uuid":"751a6964-798f-4240-a2ba-7fa9f0c72657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6827","nonce":1301191419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6829","nonce":1301191419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6833","nonce":1301191419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6831","nonce":1301191419}]},"public_addr":"172.21.15.152:6827/1301191419","cluster_addr":"172.21.15.152:6829/1301191419","heartbeat_back_addr":"172.21.15.152:6833/1301191419","heartbeat_front_addr":"172.21.15.152:6831/1301191419","state":["exists","up"]},{"osd":10,"uuid":"90e7c6ba-127c-4b33-b3c6-cedf72128451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6825","nonce":3695294503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6827","nonce":3695294503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6831","nonce":3695294503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6829","nonce":3695294503}]},"public_addr":"172.21.15.191:6825/3695294503","cluster_addr":"172.21.15.191:6827/3695294503","heartbeat_back_addr":"172.21.15.191:6831/3695294503","heartbeat_front_addr":"172.21.15.191:6829/3695294503","state":["exists","up"]},{"osd":11,"uuid":"c76a93b1-a045-477b-98d0-58a2e9d44a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6825","nonce":1709433236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6827","nonce":1709433236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6831","nonce":1709433236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6829","nonce":1709433236}]},"public_addr":"172.21.15.204:6825/1709433236","cluster_addr":"172.21.15.204:6827/1709433236","heartbeat_back_addr":"172.21.15.204:6831/1709433236","heartbeat_front_addr":"172.21.15.204:6829/1709433236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:17.259744+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:42.230389+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:07.172433+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:32.517060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:58.900615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:24.379803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:50.058928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:15.967299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:42.025873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:08.225448+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:33.910531+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:59.466423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:0/1542993850":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/3071755865":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/2452559014":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/4286398432":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6801/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6800/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6800/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/3960601922":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2270507277":"2024-09-12T22:43:31.970432+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:50:56.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.774+0000 7f4b961d7640 1 -- 172.21.15.152:0/4127475364 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4b7c0a2b80 msgr2=0x7f4b7c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:56.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.774+0000 7f4b961d7640 1 --2- 172.21.15.152:0/4127475364 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4b7c0a2b80 0x7f4b7c0a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009890 tx=0x7f4b80008040 comp rx=0 tx=0).stop 2024-09-11T22:50:56.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.774+0000 7f4b961d7640 1 -- 172.21.15.152:0/4127475364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b90103660 msgr2=0x7f4b90100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:56.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.774+0000 7f4b961d7640 1 --2- 172.21.15.152:0/4127475364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b90103660 0x7f4b90100ee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4b78005ab0 tx=0x7f4b780042f0 comp rx=0 tx=0).stop 2024-09-11T22:50:56.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.774+0000 7f4b961d7640 1 -- 172.21.15.152:0/4127475364 shutdown_connections 2024-09-11T22:50:56.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.774+0000 7f4b961d7640 1 --2- 172.21.15.152:0/4127475364 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4b7c0a2b80 0x7f4b7c0a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:56.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.774+0000 7f4b961d7640 1 --2- 172.21.15.152:0/4127475364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b9010c170 0x7f4b90101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:56.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.774+0000 7f4b961d7640 1 --2- 172.21.15.152:0/4127475364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b90103660 0x7f4b90100ee0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:56.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.774+0000 7f4b961d7640 1 --2- 172.21.15.152:0/4127475364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b90102d40 0x7f4b901009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:56.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.774+0000 7f4b961d7640 1 -- 172.21.15.152:0/4127475364 >> 172.21.15.152:0/4127475364 conn(0x7f4b900fc7a0 msgr2=0x7f4b900fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:56.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.775+0000 7f4b961d7640 1 -- 172.21.15.152:0/4127475364 shutdown_connections 2024-09-11T22:50:56.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:56.775+0000 7f4b961d7640 1 -- 172.21.15.152:0/4127475364 wait complete. 2024-09-11T22:50:57.515 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:57 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] up:active 2024-09-11T22:50:57.515 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:57 smithi152 ceph-mon[31181]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:50:57.515 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:57 smithi152 ceph-mon[31181]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:50:57.515 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:57 smithi152 ceph-mon[31181]: pgmap v282: 97 pgs: 7 creating+peering, 21 unknown, 69 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 883 B/s wr, 2 op/s 2024-09-11T22:50:57.515 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:57 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4127475364' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:50:57.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:57 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] up:active 2024-09-11T22:50:57.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:57 smithi191 ceph-mon[35211]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:50:57.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:57 smithi191 ceph-mon[35211]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:50:57.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:57 smithi191 ceph-mon[35211]: pgmap v282: 97 pgs: 7 creating+peering, 21 unknown, 69 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 883 B/s wr, 2 op/s 2024-09-11T22:50:57.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:57 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/4127475364' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:50:57.543 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:50:57.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:57 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] up:active 2024-09-11T22:50:57.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:57 smithi204 ceph-mon[35454]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:50:57.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:57 smithi204 ceph-mon[35454]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:50:57.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:57 smithi204 ceph-mon[35454]: pgmap v282: 97 pgs: 7 creating+peering, 21 unknown, 69 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 883 B/s wr, 2 op/s 2024-09-11T22:50:57.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:57 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/4127475364' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:50:57.792 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:58.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.305+0000 7f94b9cfd640 1 -- 172.21.15.152:0/3976212349 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b4106a80 msgr2=0x7f94b4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:58.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.305+0000 7f94b9cfd640 1 --2- 172.21.15.152:0/3976212349 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b4106a80 0x7f94b4106e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009930 tx=0x7f94a402a420 comp rx=0 tx=0).stop 2024-09-11T22:50:58.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.305+0000 7f94b9cfd640 1 -- 172.21.15.152:0/3976212349 shutdown_connections 2024-09-11T22:50:58.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.305+0000 7f94b9cfd640 1 --2- 172.21.15.152:0/3976212349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b4101420 0x7f94b410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:58.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.305+0000 7f94b9cfd640 1 --2- 172.21.15.152:0/3976212349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94b4100a80 0x7f94b4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:58.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.305+0000 7f94b9cfd640 1 --2- 172.21.15.152:0/3976212349 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b4106a80 0x7f94b4106e60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:58.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.305+0000 7f94b9cfd640 1 -- 172.21.15.152:0/3976212349 >> 172.21.15.152:0/3976212349 conn(0x7f94b40fc7a0 msgr2=0x7f94b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:58.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.305+0000 7f94b9cfd640 1 -- 172.21.15.152:0/3976212349 shutdown_connections 2024-09-11T22:50:58.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.305+0000 7f94b9cfd640 1 -- 172.21.15.152:0/3976212349 wait complete. 2024-09-11T22:50:58.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.306+0000 7f94b9cfd640 1 Processor -- start 2024-09-11T22:50:58.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.306+0000 7f94b9cfd640 1 -- start start 2024-09-11T22:50:58.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.306+0000 7f94b9cfd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b4100a80 0x7f94b41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:58.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.306+0000 7f94b9cfd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94b4101420 0x7f94b4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:58.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.306+0000 7f94b9cfd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b4106a80 0x7f94b41a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:58.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.306+0000 7f94b9cfd640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f94b4110e80 con 0x7f94b4100a80 2024-09-11T22:50:58.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.306+0000 7f94b9cfd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f94b4110d00 con 0x7f94b4101420 2024-09-11T22:50:58.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.306+0000 7f94b9cfd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f94b4111000 con 0x7f94b4106a80 2024-09-11T22:50:58.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.306+0000 7f94b37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b4100a80 0x7f94b41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:58.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.306+0000 7f94b37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b4100a80 0x7f94b41992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39096/0 (socket says 172.21.15.152:39096) 2024-09-11T22:50:58.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.306+0000 7f94b37fe640 1 -- 172.21.15.152:0/79826978 learned_addr learned my addr 172.21.15.152:0/79826978 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:58.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.307+0000 7f94aaffd640 1 --2- 172.21.15.152:0/79826978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94b4101420 0x7f94b4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:58.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.307+0000 7f94b3fff640 1 --2- 172.21.15.152:0/79826978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b4106a80 0x7f94b41a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:58.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.307+0000 7f94b37fe640 1 -- 172.21.15.152:0/79826978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b4106a80 msgr2=0x7f94b41a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:58.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.307+0000 7f94b37fe640 1 --2- 172.21.15.152:0/79826978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b4106a80 0x7f94b41a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:58.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.307+0000 7f94b37fe640 1 -- 172.21.15.152:0/79826978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94b4101420 msgr2=0x7f94b4199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:58.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.307+0000 7f94b37fe640 1 --2- 172.21.15.152:0/79826978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94b4101420 0x7f94b4199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:58.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.307+0000 7f94b37fe640 1 -- 172.21.15.152:0/79826978 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b41a48b0 con 0x7f94b4100a80 2024-09-11T22:50:58.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.307+0000 7f94aaffd640 1 --2- 172.21.15.152:0/79826978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94b4101420 0x7f94b4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:50:58.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.307+0000 7f94b3fff640 1 --2- 172.21.15.152:0/79826978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b4106a80 0x7f94b41a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:50:58.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.307+0000 7f94b37fe640 1 --2- 172.21.15.152:0/79826978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b4100a80 0x7f94b41992f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009230 tx=0x7f94a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:58.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.307+0000 7f94b17fa640 1 -- 172.21.15.152:0/79826978 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a403f030 con 0x7f94b4100a80 2024-09-11T22:50:58.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.307+0000 7f94b9cfd640 1 -- 172.21.15.152:0/79826978 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b41a4a80 con 0x7f94b4100a80 2024-09-11T22:50:58.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.307+0000 7f94b17fa640 1 -- 172.21.15.152:0/79826978 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94a4003ad0 con 0x7f94b4100a80 2024-09-11T22:50:58.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.307+0000 7f94b17fa640 1 -- 172.21.15.152:0/79826978 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a4035070 con 0x7f94b4100a80 2024-09-11T22:50:58.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.307+0000 7f94b9cfd640 1 -- 172.21.15.152:0/79826978 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b41a4ed0 con 0x7f94b4100a80 2024-09-11T22:50:58.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.308+0000 7f94b9cfd640 1 -- 172.21.15.152:0/79826978 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b4100ee0 con 0x7f94b4100a80 2024-09-11T22:50:58.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.309+0000 7f94b17fa640 1 -- 172.21.15.152:0/79826978 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f94a4042030 con 0x7f94b4100a80 2024-09-11T22:50:58.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.310+0000 7f94b17fa640 1 --2- 172.21.15.152:0/79826978 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f94a00a2b80 0x7f94a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:58.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.310+0000 7f94aaffd640 1 --2- 172.21.15.152:0/79826978 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f94a00a2b80 0x7f94a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:58.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.310+0000 7f94b17fa640 1 -- 172.21.15.152:0/79826978 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f94a40ef850 con 0x7f94b4100a80 2024-09-11T22:50:58.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.310+0000 7f94aaffd640 1 --2- 172.21.15.152:0/79826978 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f94a00a2b80 0x7f94a00a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f94b406b2e0 tx=0x7f949c005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:58.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.313+0000 7f94b17fa640 1 -- 172.21.15.152:0/79826978 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f94a40bc900 con 0x7f94b4100a80 2024-09-11T22:50:58.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.480+0000 7f94b9cfd640 1 -- 172.21.15.152:0/79826978 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94b4108ff0 con 0x7f94b4100a80 2024-09-11T22:50:58.481 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:58 smithi152 ceph-mon[31181]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T22:50:58.482 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:58 smithi152 ceph-mon[31181]: Cluster is now healthy 2024-09-11T22:50:58.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.481+0000 7f94b17fa640 1 -- 172.21.15.152:0/79826978 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22363 (secure 0 0 0) 0x7f94a40bc200 con 0x7f94b4100a80 2024-09-11T22:50:58.483 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:50:58.483 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":81,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","created":"2024-09-11T22:43:00.379317+0000","modified":"2024-09-11T22:50:55.047140+0000","last_up_change":"2024-09-11T22:50:01.043181+0000","last_in_change":"2024-09-11T22:49:38.073870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:46:10.984707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:50:50.833547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:50:53.047604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3cd917b5-77f6-4a4a-8fd0-4ebe3034feac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6803","nonce":1270728007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6805","nonce":1270728007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6809","nonce":1270728007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6807","nonce":1270728007}]},"public_addr":"172.21.15.152:6803/1270728007","cluster_addr":"172.21.15.152:6805/1270728007","heartbeat_back_addr":"172.21.15.152:6809/1270728007","heartbeat_front_addr":"172.21.15.152:6807/1270728007","state":["exists","up"]},{"osd":1,"uuid":"ae589bc4-77b1-4522-9591-b0ce2a5432dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6801","nonce":354467155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6803","nonce":354467155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6807","nonce":354467155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6805","nonce":354467155}]},"public_addr":"172.21.15.191:6801/354467155","cluster_addr":"172.21.15.191:6803/354467155","heartbeat_back_addr":"172.21.15.191:6807/354467155","heartbeat_front_addr":"172.21.15.191:6805/354467155","state":["exists","up"]},{"osd":2,"uuid":"b35ffd0c-d084-43dd-a947-15132d78d59c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6801","nonce":3848396211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6803","nonce":3848396211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6807","nonce":3848396211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6805","nonce":3848396211}]},"public_addr":"172.21.15.204:6801/3848396211","cluster_addr":"172.21.15.204:6803/3848396211","heartbeat_back_addr":"172.21.15.204:6807/3848396211","heartbeat_front_addr":"172.21.15.204:6805/3848396211","state":["exists","up"]},{"osd":3,"uuid":"d66ba7e8-4ab7-449e-9460-2f44ffdf987f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6811","nonce":3351257181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6813","nonce":3351257181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6817","nonce":3351257181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6815","nonce":3351257181}]},"public_addr":"172.21.15.152:6811/3351257181","cluster_addr":"172.21.15.152:6813/3351257181","heartbeat_back_addr":"172.21.15.152:6817/3351257181","heartbeat_front_addr":"172.21.15.152:6815/3351257181","state":["exists","up"]},{"osd":4,"uuid":"e1b56468-6cc7-4d29-988e-9d281118ee36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6809","nonce":2967539563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6811","nonce":2967539563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6815","nonce":2967539563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6813","nonce":2967539563}]},"public_addr":"172.21.15.191:6809/2967539563","cluster_addr":"172.21.15.191:6811/2967539563","heartbeat_back_addr":"172.21.15.191:6815/2967539563","heartbeat_front_addr":"172.21.15.191:6813/2967539563","state":["exists","up"]},{"osd":5,"uuid":"89dabd3e-77e3-4294-9648-b112ef694e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6809","nonce":4283562158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6811","nonce":4283562158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6815","nonce":4283562158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6813","nonce":4283562158}]},"public_addr":"172.21.15.204:6809/4283562158","cluster_addr":"172.21.15.204:6811/4283562158","heartbeat_back_addr":"172.21.15.204:6815/4283562158","heartbeat_front_addr":"172.21.15.204:6813/4283562158","state":["exists","up"]},{"osd":6,"uuid":"46a0fded-b668-4aef-a68f-613cccc83483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6819","nonce":2484234042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6821","nonce":2484234042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6825","nonce":2484234042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6823","nonce":2484234042}]},"public_addr":"172.21.15.152:6819/2484234042","cluster_addr":"172.21.15.152:6821/2484234042","heartbeat_back_addr":"172.21.15.152:6825/2484234042","heartbeat_front_addr":"172.21.15.152:6823/2484234042","state":["exists","up"]},{"osd":7,"uuid":"e674bfe9-e2db-44a3-b74b-a21f198747c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6817","nonce":269608859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6819","nonce":269608859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6823","nonce":269608859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6821","nonce":269608859}]},"public_addr":"172.21.15.191:6817/269608859","cluster_addr":"172.21.15.191:6819/269608859","heartbeat_back_addr":"172.21.15.191:6823/269608859","heartbeat_front_addr":"172.21.15.191:6821/269608859","state":["exists","up"]},{"osd":8,"uuid":"0418bfc9-d6b6-488f-b4c7-0bc6291bb624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6817","nonce":1756190090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6819","nonce":1756190090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6823","nonce":1756190090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6821","nonce":1756190090}]},"public_addr":"172.21.15.204:6817/1756190090","cluster_addr":"172.21.15.204:6819/1756190090","heartbeat_back_addr":"172.21.15.204:6823/1756190090","heartbeat_front_addr":"172.21.15.204:6821/1756190090","state":["exists","up"]},{"osd":9,"uuid":"751a6964-798f-4240-a2ba-7fa9f0c72657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6827","nonce":1301191419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6829","nonce":1301191419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6833","nonce":1301191419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6831","nonce":1301191419}]},"public_addr":"172.21.15.152:6827/1301191419","cluster_addr":"172.21.15.152:6829/1301191419","heartbeat_back_addr":"172.21.15.152:6833/1301191419","heartbeat_front_addr":"172.21.15.152:6831/1301191419","state":["exists","up"]},{"osd":10,"uuid":"90e7c6ba-127c-4b33-b3c6-cedf72128451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6825","nonce":3695294503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6827","nonce":3695294503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6831","nonce":3695294503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6829","nonce":3695294503}]},"public_addr":"172.21.15.191:6825/3695294503","cluster_addr":"172.21.15.191:6827/3695294503","heartbeat_back_addr":"172.21.15.191:6831/3695294503","heartbeat_front_addr":"172.21.15.191:6829/3695294503","state":["exists","up"]},{"osd":11,"uuid":"c76a93b1-a045-477b-98d0-58a2e9d44a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6825","nonce":1709433236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6827","nonce":1709433236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6831","nonce":1709433236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6829","nonce":1709433236}]},"public_addr":"172.21.15.204:6825/1709433236","cluster_addr":"172.21.15.204:6827/1709433236","heartbeat_back_addr":"172.21.15.204:6831/1709433236","heartbeat_front_addr":"172.21.15.204:6829/1709433236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:17.259744+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:42.230389+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:07.172433+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:32.517060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:58.900615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:24.379803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:50.058928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:15.967299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:42.025873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:08.225448+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:33.910531+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:59.466423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:0/1542993850":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/3071755865":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/2452559014":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/4286398432":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6801/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6800/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6800/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/3960601922":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2270507277":"2024-09-12T22:43:31.970432+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:50:58.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.483+0000 7f94b9cfd640 1 -- 172.21.15.152:0/79826978 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f94a00a2b80 msgr2=0x7f94a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.483+0000 7f94b9cfd640 1 --2- 172.21.15.152:0/79826978 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f94a00a2b80 0x7f94a00a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f94b406b2e0 tx=0x7f949c005ec0 comp rx=0 tx=0).stop 2024-09-11T22:50:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.483+0000 7f94b9cfd640 1 -- 172.21.15.152:0/79826978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b4100a80 msgr2=0x7f94b41992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.483+0000 7f94b9cfd640 1 --2- 172.21.15.152:0/79826978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b4100a80 0x7f94b41992f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009230 tx=0x7f94a4004270 comp rx=0 tx=0).stop 2024-09-11T22:50:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.484+0000 7f94b9cfd640 1 -- 172.21.15.152:0/79826978 shutdown_connections 2024-09-11T22:50:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.484+0000 7f94b9cfd640 1 --2- 172.21.15.152:0/79826978 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f94a00a2b80 0x7f94a00a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.484+0000 7f94b9cfd640 1 --2- 172.21.15.152:0/79826978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94b4106a80 0x7f94b41a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.484+0000 7f94b9cfd640 1 --2- 172.21.15.152:0/79826978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94b4101420 0x7f94b4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.484+0000 7f94b9cfd640 1 --2- 172.21.15.152:0/79826978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f94b4100a80 0x7f94b41992f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.484+0000 7f94b9cfd640 1 -- 172.21.15.152:0/79826978 >> 172.21.15.152:0/79826978 conn(0x7f94b40fc7a0 msgr2=0x7f94b40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.484+0000 7f94b9cfd640 1 -- 172.21.15.152:0/79826978 shutdown_connections 2024-09-11T22:50:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:58.484+0000 7f94b9cfd640 1 -- 172.21.15.152:0/79826978 wait complete. 2024-09-11T22:50:58.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:58 smithi191 ceph-mon[35211]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T22:50:58.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:58 smithi191 ceph-mon[35211]: Cluster is now healthy 2024-09-11T22:50:58.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:58 smithi204 ceph-mon[35454]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T22:50:58.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:58 smithi204 ceph-mon[35454]: Cluster is now healthy 2024-09-11T22:50:59.177 INFO:journalctl@ceph.mds.b.smithi191.stdout:Sep 11 22:50:58 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-b[65317]: 2024-09-11T22:50:58.910+0000 7fbd59741640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T22:50:59.201 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:59 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/79826978' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:50:59.201 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:50:59 smithi152 ceph-mon[31181]: pgmap v283: 97 pgs: 4 creating+peering, 93 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-09-11T22:50:59.203 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-11T22:50:59.450 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:50:59.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:59 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/79826978' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:50:59.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:50:59 smithi191 ceph-mon[35211]: pgmap v283: 97 pgs: 4 creating+peering, 93 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-09-11T22:50:59.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:59 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/79826978' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:50:59.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:50:59 smithi204 ceph-mon[35454]: pgmap v283: 97 pgs: 4 creating+peering, 93 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-09-11T22:50:59.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.949+0000 7f9249424640 1 -- 172.21.15.152:0/531544275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92441036e0 msgr2=0x7f9244107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:59.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.949+0000 7f9249424640 1 --2- 172.21.15.152:0/531544275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92441036e0 0x7f9244107b90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9238009930 tx=0x7f923802a420 comp rx=0 tx=0).stop 2024-09-11T22:50:59.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.949+0000 7f9249424640 1 -- 172.21.15.152:0/531544275 shutdown_connections 2024-09-11T22:50:59.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.949+0000 7f9249424640 1 --2- 172.21.15.152:0/531544275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92441082a0 0x7f924410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:59.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.949+0000 7f9249424640 1 --2- 172.21.15.152:0/531544275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92441036e0 0x7f9244107b90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:59.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.949+0000 7f9249424640 1 --2- 172.21.15.152:0/531544275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9244102d30 0x7f9244103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:59.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.949+0000 7f9249424640 1 -- 172.21.15.152:0/531544275 >> 172.21.15.152:0/531544275 conn(0x7f92440fc7d0 msgr2=0x7f92440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:50:59.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.949+0000 7f9249424640 1 -- 172.21.15.152:0/531544275 shutdown_connections 2024-09-11T22:50:59.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.949+0000 7f9249424640 1 -- 172.21.15.152:0/531544275 wait complete. 2024-09-11T22:50:59.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.950+0000 7f9249424640 1 Processor -- start 2024-09-11T22:50:59.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.950+0000 7f9249424640 1 -- start start 2024-09-11T22:50:59.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.950+0000 7f9249424640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9244102d30 0x7f9244198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:59.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.950+0000 7f9249424640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92441036e0 0x7f9244198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:59.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.950+0000 7f9249424640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92441082a0 0x7f924419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:59.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.950+0000 7f9249424640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9244112140 con 0x7f9244102d30 2024-09-11T22:50:59.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.950+0000 7f9249424640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9244111fc0 con 0x7f92441036e0 2024-09-11T22:50:59.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.950+0000 7f9249424640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f92441122c0 con 0x7f92441082a0 2024-09-11T22:50:59.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.950+0000 7f9242ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9244102d30 0x7f9244198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:59.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.950+0000 7f9242ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9244102d30 0x7f9244198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49256/0 (socket says 172.21.15.152:49256) 2024-09-11T22:50:59.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.950+0000 7f9242ffd640 1 -- 172.21.15.152:0/846696407 learned_addr learned my addr 172.21.15.152:0/846696407 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:50:59.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.951+0000 7f92437fe640 1 --2- 172.21.15.152:0/846696407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92441082a0 0x7f924419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:59.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.951+0000 7f92427fc640 1 --2- 172.21.15.152:0/846696407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92441036e0 0x7f9244198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:59.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.951+0000 7f9242ffd640 1 -- 172.21.15.152:0/846696407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92441082a0 msgr2=0x7f924419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:59.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.951+0000 7f9242ffd640 1 --2- 172.21.15.152:0/846696407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92441082a0 0x7f924419d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:59.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.951+0000 7f9242ffd640 1 -- 172.21.15.152:0/846696407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92441036e0 msgr2=0x7f9244198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:50:59.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.951+0000 7f9242ffd640 1 --2- 172.21.15.152:0/846696407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92441036e0 0x7f9244198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:50:59.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.951+0000 7f9242ffd640 1 -- 172.21.15.152:0/846696407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924419d6f0 con 0x7f9244102d30 2024-09-11T22:50:59.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.951+0000 7f92437fe640 1 --2- 172.21.15.152:0/846696407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92441082a0 0x7f924419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:50:59.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.951+0000 7f92427fc640 1 --2- 172.21.15.152:0/846696407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92441036e0 0x7f9244198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:50:59.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.951+0000 7f9242ffd640 1 --2- 172.21.15.152:0/846696407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9244102d30 0x7f9244198740 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f923400ba50 tx=0x7f923400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:59.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.951+0000 7f922bfff640 1 -- 172.21.15.152:0/846696407 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9234019070 con 0x7f9244102d30 2024-09-11T22:50:59.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.951+0000 7f9249424640 1 -- 172.21.15.152:0/846696407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924419d9e0 con 0x7f9244102d30 2024-09-11T22:50:59.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.951+0000 7f922bfff640 1 -- 172.21.15.152:0/846696407 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f923400cbe0 con 0x7f9244102d30 2024-09-11T22:50:59.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.951+0000 7f922bfff640 1 -- 172.21.15.152:0/846696407 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92340076e0 con 0x7f9244102d30 2024-09-11T22:50:59.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.951+0000 7f9249424640 1 -- 172.21.15.152:0/846696407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92441a52c0 con 0x7f9244102d30 2024-09-11T22:50:59.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.952+0000 7f9249424640 1 -- 172.21.15.152:0/846696407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f924406a850 con 0x7f9244102d30 2024-09-11T22:50:59.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.953+0000 7f922bfff640 1 -- 172.21.15.152:0/846696407 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f923401c070 con 0x7f9244102d30 2024-09-11T22:50:59.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.954+0000 7f922bfff640 1 --2- 172.21.15.152:0/846696407 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f922c0a2b80 0x7f922c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:50:59.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.954+0000 7f92427fc640 1 --2- 172.21.15.152:0/846696407 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f922c0a2b80 0x7f922c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:50:59.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.955+0000 7f922bfff640 1 -- 172.21.15.152:0/846696407 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f92340ce550 con 0x7f9244102d30 2024-09-11T22:50:59.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.955+0000 7f92427fc640 1 --2- 172.21.15.152:0/846696407 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f922c0a2b80 0x7f922c0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f923802a940 tx=0x7f9238032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:50:59.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:50:59.959+0000 7f922bfff640 1 -- 172.21.15.152:0/846696407 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f923409b600 con 0x7f9244102d30 2024-09-11T22:51:00.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:00.162+0000 7f9249424640 1 -- 172.21.15.152:0/846696407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f924410a2b0 con 0x7f9244102d30 2024-09-11T22:51:00.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:00.181+0000 7f922bfff640 1 -- 172.21.15.152:0/846696407 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f923409af00 con 0x7f9244102d30 2024-09-11T22:51:00.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:00.184+0000 7f9249424640 1 -- 172.21.15.152:0/846696407 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f922c0a2b80 msgr2=0x7f922c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:00.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:00.184+0000 7f9249424640 1 --2- 172.21.15.152:0/846696407 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f922c0a2b80 0x7f922c0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f923802a940 tx=0x7f9238032d50 comp rx=0 tx=0).stop 2024-09-11T22:51:00.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:00.184+0000 7f9249424640 1 -- 172.21.15.152:0/846696407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9244102d30 msgr2=0x7f9244198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:00.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:00.184+0000 7f9249424640 1 --2- 172.21.15.152:0/846696407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9244102d30 0x7f9244198740 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f923400ba50 tx=0x7f923400bf10 comp rx=0 tx=0).stop 2024-09-11T22:51:00.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:00.184+0000 7f9249424640 1 -- 172.21.15.152:0/846696407 shutdown_connections 2024-09-11T22:51:00.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:00.184+0000 7f9249424640 1 --2- 172.21.15.152:0/846696407 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f922c0a2b80 0x7f922c0a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:00.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:00.184+0000 7f9249424640 1 --2- 172.21.15.152:0/846696407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92441082a0 0x7f924419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:00.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:00.184+0000 7f9249424640 1 --2- 172.21.15.152:0/846696407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92441036e0 0x7f9244198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:00.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:00.184+0000 7f9249424640 1 --2- 172.21.15.152:0/846696407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9244102d30 0x7f9244198740 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:00.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:00.184+0000 7f9249424640 1 -- 172.21.15.152:0/846696407 >> 172.21.15.152:0/846696407 conn(0x7f92440fc7d0 msgr2=0x7f9244105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:00.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:00.184+0000 7f9249424640 1 -- 172.21.15.152:0/846696407 shutdown_connections 2024-09-11T22:51:00.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:00.184+0000 7f9249424640 1 -- 172.21.15.152:0/846696407 wait complete. 2024-09-11T22:51:00.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:00 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/846696407' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T22:51:00.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:00 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/846696407' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T22:51:00.555 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:00 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/846696407' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T22:51:00.900 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-11T22:51:00.900 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-11T22:51:01.149 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:01.450 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:01 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/846696407' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T22:51:01.450 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:01 smithi152 ceph-mon[31181]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:51:01.450 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:01 smithi152 ceph-mon[31181]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-11T22:51:01.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:01 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/846696407' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T22:51:01.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:01 smithi191 ceph-mon[35211]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:51:01.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:01 smithi191 ceph-mon[35211]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-11T22:51:01.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:01 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/846696407' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T22:51:01.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:01 smithi204 ceph-mon[35454]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:51:01.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:01 smithi204 ceph-mon[35454]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-11T22:51:01.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.658+0000 7f1e54ed3640 1 -- 172.21.15.152:0/666889471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e50106f10 msgr2=0x7f1e5010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:01.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.658+0000 7f1e54ed3640 1 --2- 172.21.15.152:0/666889471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e50106f10 0x7f1e5010ea90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009ca0 tx=0x7f1e4802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:51:01.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.658+0000 7f1e54ed3640 1 -- 172.21.15.152:0/666889471 shutdown_connections 2024-09-11T22:51:01.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.658+0000 7f1e54ed3640 1 --2- 172.21.15.152:0/666889471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e50106f10 0x7f1e5010ea90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:01.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.658+0000 7f1e54ed3640 1 --2- 172.21.15.152:0/666889471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e50102790 0x7f1e501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:01.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.658+0000 7f1e54ed3640 1 --2- 172.21.15.152:0/666889471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e50101de0 0x7f1e501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:01.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.658+0000 7f1e54ed3640 1 -- 172.21.15.152:0/666889471 >> 172.21.15.152:0/666889471 conn(0x7f1e500fd8c0 msgr2=0x7f1e500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:01.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.659+0000 7f1e54ed3640 1 -- 172.21.15.152:0/666889471 shutdown_connections 2024-09-11T22:51:01.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.659+0000 7f1e54ed3640 1 -- 172.21.15.152:0/666889471 wait complete. 2024-09-11T22:51:01.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.659+0000 7f1e54ed3640 1 Processor -- start 2024-09-11T22:51:01.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.659+0000 7f1e54ed3640 1 -- start start 2024-09-11T22:51:01.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e54ed3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e50101de0 0x7f1e500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:01.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e54ed3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e50102790 0x7f1e50074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:01.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e54ed3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e50106f10 0x7f1e50074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:01.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e54ed3640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50111f70 con 0x7f1e50106f10 2024-09-11T22:51:01.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e54ed3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50111df0 con 0x7f1e50101de0 2024-09-11T22:51:01.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e54ed3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1e501120f0 con 0x7f1e50102790 2024-09-11T22:51:01.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e4ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e50106f10 0x7f1e50074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:01.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e4ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e50106f10 0x7f1e50074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49272/0 (socket says 172.21.15.152:49272) 2024-09-11T22:51:01.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e4ed9d640 1 -- 172.21.15.152:0/101741434 learned_addr learned my addr 172.21.15.152:0/101741434 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:01.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e4dd9b640 1 --2- 172.21.15.152:0/101741434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e50102790 0x7f1e50074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:01.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e4e59c640 1 --2- 172.21.15.152:0/101741434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e50101de0 0x7f1e500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:01.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e4ed9d640 1 -- 172.21.15.152:0/101741434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e50102790 msgr2=0x7f1e50074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:01.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e4ed9d640 1 --2- 172.21.15.152:0/101741434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e50102790 0x7f1e50074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:01.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e4ed9d640 1 -- 172.21.15.152:0/101741434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e50101de0 msgr2=0x7f1e500744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:01.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e4ed9d640 1 --2- 172.21.15.152:0/101741434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e50101de0 0x7f1e500744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:01.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e4ed9d640 1 -- 172.21.15.152:0/101741434 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e501a6890 con 0x7f1e50106f10 2024-09-11T22:51:01.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e4e59c640 1 --2- 172.21.15.152:0/101741434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e50101de0 0x7f1e500744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:01.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e4ed9d640 1 --2- 172.21.15.152:0/101741434 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e50106f10 0x7f1e50074f70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1e48031a20 tx=0x7f1e48031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:01.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e477fe640 1 -- 172.21.15.152:0/101741434 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e48003c30 con 0x7f1e50106f10 2024-09-11T22:51:01.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e54ed3640 1 -- 172.21.15.152:0/101741434 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e501a6b20 con 0x7f1e50106f10 2024-09-11T22:51:01.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e477fe640 1 -- 172.21.15.152:0/101741434 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e48003dd0 con 0x7f1e50106f10 2024-09-11T22:51:01.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e477fe640 1 -- 172.21.15.152:0/101741434 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4802fcf0 con 0x7f1e50106f10 2024-09-11T22:51:01.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.660+0000 7f1e54ed3640 1 -- 172.21.15.152:0/101741434 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e501a7060 con 0x7f1e50106f10 2024-09-11T22:51:01.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.661+0000 7f1e54ed3640 1 -- 172.21.15.152:0/101741434 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e501021c0 con 0x7f1e50106f10 2024-09-11T22:51:01.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.662+0000 7f1e477fe640 1 -- 172.21.15.152:0/101741434 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1e48004b40 con 0x7f1e50106f10 2024-09-11T22:51:01.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.663+0000 7f1e477fe640 1 --2- 172.21.15.152:0/101741434 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1e3c0a2b80 0x7f1e3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:01.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.663+0000 7f1e4e59c640 1 --2- 172.21.15.152:0/101741434 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1e3c0a2b80 0x7f1e3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:01.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.663+0000 7f1e477fe640 1 -- 172.21.15.152:0/101741434 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f1e480f0470 con 0x7f1e50106f10 2024-09-11T22:51:01.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.664+0000 7f1e4e59c640 1 --2- 172.21.15.152:0/101741434 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1e3c0a2b80 0x7f1e3c0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009b40 tx=0x7f1e40005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:01.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.666+0000 7f1e477fe640 1 -- 172.21.15.152:0/101741434 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1e480bd5b0 con 0x7f1e50106f10 2024-09-11T22:51:01.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:01.868+0000 7f1e54ed3640 1 -- 172.21.15.152:0/101741434 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f1e5010a150 con 0x7f1e50106f10 2024-09-11T22:51:02.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:02.192+0000 7f1e477fe640 1 -- 172.21.15.152:0/101741434 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f1e4802fe90 con 0x7f1e50106f10 2024-09-11T22:51:02.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:02.195+0000 7f1e54ed3640 1 -- 172.21.15.152:0/101741434 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1e3c0a2b80 msgr2=0x7f1e3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:02.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:02.196+0000 7f1e54ed3640 1 --2- 172.21.15.152:0/101741434 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1e3c0a2b80 0x7f1e3c0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009b40 tx=0x7f1e40005e50 comp rx=0 tx=0).stop 2024-09-11T22:51:02.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:02.196+0000 7f1e54ed3640 1 -- 172.21.15.152:0/101741434 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e50106f10 msgr2=0x7f1e50074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:02.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:02.196+0000 7f1e54ed3640 1 --2- 172.21.15.152:0/101741434 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e50106f10 0x7f1e50074f70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1e48031a20 tx=0x7f1e48031a50 comp rx=0 tx=0).stop 2024-09-11T22:51:02.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:02.196+0000 7f1e54ed3640 1 -- 172.21.15.152:0/101741434 shutdown_connections 2024-09-11T22:51:02.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:02.196+0000 7f1e54ed3640 1 --2- 172.21.15.152:0/101741434 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f1e3c0a2b80 0x7f1e3c0a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:02.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:02.196+0000 7f1e54ed3640 1 --2- 172.21.15.152:0/101741434 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1e50106f10 0x7f1e50074f70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:02.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:02.196+0000 7f1e54ed3640 1 --2- 172.21.15.152:0/101741434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e50102790 0x7f1e50074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:02.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:02.196+0000 7f1e54ed3640 1 --2- 172.21.15.152:0/101741434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e50101de0 0x7f1e500744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:02.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:02.196+0000 7f1e54ed3640 1 -- 172.21.15.152:0/101741434 >> 172.21.15.152:0/101741434 conn(0x7f1e500fd8c0 msgr2=0x7f1e500feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:02.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:02.196+0000 7f1e54ed3640 1 -- 172.21.15.152:0/101741434 shutdown_connections 2024-09-11T22:51:02.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:02.196+0000 7f1e54ed3640 1 -- 172.21.15.152:0/101741434 wait complete. 2024-09-11T22:51:02.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:02 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/101741434' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T22:51:02.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:02 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/101741434' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T22:51:02.564 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:02 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/101741434' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T22:51:02.914 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-11T22:51:03.166 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:03.448 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:03 smithi152 ceph-mon[31181]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T22:51:03.449 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:03 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/101741434' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T22:51:03.449 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:03 smithi152 ceph-mon[31181]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T22:51:03.449 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:03 smithi152 ceph-mon[31181]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:51:03.449 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:03 smithi152 ceph-mon[31181]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T22:51:03.449 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:03 smithi152 ceph-mon[31181]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T22:51:03.449 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:03 smithi152 ceph-mon[31181]: pgmap v285: 97 pgs: 97 active+clean; 580 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-11T22:51:03.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:03 smithi191 ceph-mon[35211]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T22:51:03.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:03 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/101741434' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T22:51:03.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:03 smithi191 ceph-mon[35211]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T22:51:03.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:03 smithi191 ceph-mon[35211]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:51:03.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:03 smithi191 ceph-mon[35211]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T22:51:03.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:03 smithi191 ceph-mon[35211]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T22:51:03.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:03 smithi191 ceph-mon[35211]: pgmap v285: 97 pgs: 97 active+clean; 580 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-11T22:51:03.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:03 smithi204 ceph-mon[35454]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T22:51:03.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:03 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/101741434' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T22:51:03.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:03 smithi204 ceph-mon[35454]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T22:51:03.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:03 smithi204 ceph-mon[35454]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:51:03.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:03 smithi204 ceph-mon[35454]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T22:51:03.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:03 smithi204 ceph-mon[35454]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T22:51:03.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:03 smithi204 ceph-mon[35454]: pgmap v285: 97 pgs: 97 active+clean; 580 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-11T22:51:03.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.707+0000 7f425b276640 1 -- 172.21.15.152:0/253776104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4254106fc0 msgr2=0x7f425410eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:03.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.707+0000 7f425b276640 1 --2- 172.21.15.152:0/253776104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4254106fc0 0x7f425410eaf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f424c009cd0 tx=0x7f424c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:51:03.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.707+0000 7f425b276640 1 -- 172.21.15.152:0/253776104 shutdown_connections 2024-09-11T22:51:03.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.707+0000 7f425b276640 1 --2- 172.21.15.152:0/253776104 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4254106fc0 0x7f425410eaf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:03.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.707+0000 7f425b276640 1 --2- 172.21.15.152:0/253776104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4254100d20 0x7f4254106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:03.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.707+0000 7f425b276640 1 --2- 172.21.15.152:0/253776104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4254100370 0x7f4254100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:03.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.707+0000 7f425b276640 1 -- 172.21.15.152:0/253776104 >> 172.21.15.152:0/253776104 conn(0x7f4254077210 msgr2=0x7f42540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:03.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.707+0000 7f425b276640 1 -- 172.21.15.152:0/253776104 shutdown_connections 2024-09-11T22:51:03.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.707+0000 7f425b276640 1 -- 172.21.15.152:0/253776104 wait complete. 2024-09-11T22:51:03.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.708+0000 7f425b276640 1 Processor -- start 2024-09-11T22:51:03.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.708+0000 7f425b276640 1 -- start start 2024-09-11T22:51:03.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.708+0000 7f425b276640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4254100370 0x7f4254198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:03.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.708+0000 7f425b276640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4254100d20 0x7f42541988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:03.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.708+0000 7f425b276640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4254106fc0 0x7f425419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:03.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.708+0000 7f425b276640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4254111fa0 con 0x7f4254106fc0 2024-09-11T22:51:03.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.708+0000 7f425b276640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4254111e20 con 0x7f4254100d20 2024-09-11T22:51:03.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.708+0000 7f425b276640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4254112120 con 0x7f4254100370 2024-09-11T22:51:03.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.708+0000 7f4259813640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4254106fc0 0x7f425419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:03.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.708+0000 7f4259813640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4254106fc0 0x7f425419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49290/0 (socket says 172.21.15.152:49290) 2024-09-11T22:51:03.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.708+0000 7f4259813640 1 -- 172.21.15.152:0/108004800 learned_addr learned my addr 172.21.15.152:0/108004800 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.708+0000 7f4258811640 1 --2- 172.21.15.152:0/108004800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4254100d20 0x7f42541988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.708+0000 7f4259012640 1 --2- 172.21.15.152:0/108004800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4254100370 0x7f4254198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.709+0000 7f4259813640 1 -- 172.21.15.152:0/108004800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4254100370 msgr2=0x7f4254198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.709+0000 7f4259813640 1 --2- 172.21.15.152:0/108004800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4254100370 0x7f4254198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.709+0000 7f4259813640 1 -- 172.21.15.152:0/108004800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4254100d20 msgr2=0x7f42541988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.709+0000 7f4259813640 1 --2- 172.21.15.152:0/108004800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4254100d20 0x7f42541988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.709+0000 7f4259813640 1 -- 172.21.15.152:0/108004800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425419d330 con 0x7f4254106fc0 2024-09-11T22:51:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.709+0000 7f4259813640 1 --2- 172.21.15.152:0/108004800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4254106fc0 0x7f425419cc50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f424c006010 tx=0x7f424c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.709+0000 7f42467fc640 1 -- 172.21.15.152:0/108004800 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c004450 con 0x7f4254106fc0 2024-09-11T22:51:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.709+0000 7f42467fc640 1 -- 172.21.15.152:0/108004800 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f424c003e10 con 0x7f4254106fc0 2024-09-11T22:51:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.709+0000 7f425b276640 1 -- 172.21.15.152:0/108004800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425419d5c0 con 0x7f4254106fc0 2024-09-11T22:51:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.709+0000 7f42467fc640 1 -- 172.21.15.152:0/108004800 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c046480 con 0x7f4254106fc0 2024-09-11T22:51:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.709+0000 7f425b276640 1 -- 172.21.15.152:0/108004800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42541a4f00 con 0x7f4254106fc0 2024-09-11T22:51:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.710+0000 7f425b276640 1 -- 172.21.15.152:0/108004800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4254100750 con 0x7f4254106fc0 2024-09-11T22:51:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.711+0000 7f42467fc640 1 -- 172.21.15.152:0/108004800 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f424c0045f0 con 0x7f4254106fc0 2024-09-11T22:51:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.712+0000 7f42467fc640 1 --2- 172.21.15.152:0/108004800 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f42400a2b80 0x7f42400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.712+0000 7f4259012640 1 --2- 172.21.15.152:0/108004800 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f42400a2b80 0x7f42400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.712+0000 7f42467fc640 1 -- 172.21.15.152:0/108004800 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f424c02e030 con 0x7f4254106fc0 2024-09-11T22:51:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.712+0000 7f4259012640 1 --2- 172.21.15.152:0/108004800 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f42400a2b80 0x7f42400a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f42500058c0 tx=0x7f425000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.716+0000 7f42467fc640 1 -- 172.21.15.152:0/108004800 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f424c04b060 con 0x7f4254106fc0 2024-09-11T22:51:03.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:03.920+0000 7f425b276640 1 -- 172.21.15.152:0/108004800 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f425410a120 con 0x7f4254106fc0 2024-09-11T22:51:04.208 INFO:journalctl@ceph.mds.b.smithi191.stdout:Sep 11 22:51:03 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-b[65317]: 2024-09-11T22:51:03.910+0000 7fbd59741640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-11T22:51:04.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:04.214+0000 7f42467fc640 1 -- 172.21.15.152:0/108004800 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f424c0bd860 con 0x7f4254106fc0 2024-09-11T22:51:04.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:04.216+0000 7f425b276640 1 -- 172.21.15.152:0/108004800 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f42400a2b80 msgr2=0x7f42400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:04.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:04.216+0000 7f425b276640 1 --2- 172.21.15.152:0/108004800 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f42400a2b80 0x7f42400a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f42500058c0 tx=0x7f425000a2d0 comp rx=0 tx=0).stop 2024-09-11T22:51:04.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:04.216+0000 7f425b276640 1 -- 172.21.15.152:0/108004800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4254106fc0 msgr2=0x7f425419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:04.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:04.216+0000 7f425b276640 1 --2- 172.21.15.152:0/108004800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4254106fc0 0x7f425419cc50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f424c006010 tx=0x7f424c003c20 comp rx=0 tx=0).stop 2024-09-11T22:51:04.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:04.216+0000 7f425b276640 1 -- 172.21.15.152:0/108004800 shutdown_connections 2024-09-11T22:51:04.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:04.216+0000 7f425b276640 1 --2- 172.21.15.152:0/108004800 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f42400a2b80 0x7f42400a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:04.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:04.216+0000 7f425b276640 1 --2- 172.21.15.152:0/108004800 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4254106fc0 0x7f425419cc50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:04.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:04.216+0000 7f425b276640 1 --2- 172.21.15.152:0/108004800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4254100d20 0x7f42541988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:04.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:04.216+0000 7f425b276640 1 --2- 172.21.15.152:0/108004800 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4254100370 0x7f4254198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:04.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:04.217+0000 7f425b276640 1 -- 172.21.15.152:0/108004800 >> 172.21.15.152:0/108004800 conn(0x7f4254077210 msgr2=0x7f4254105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:04.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:04.217+0000 7f425b276640 1 -- 172.21.15.152:0/108004800 shutdown_connections 2024-09-11T22:51:04.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:04.217+0000 7f425b276640 1 -- 172.21.15.152:0/108004800 wait complete. 2024-09-11T22:51:04.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:04 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] up:active 2024-09-11T22:51:04.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:04 smithi191 ceph-mon[35211]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T22:51:04.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:04 smithi191 ceph-mon[35211]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T22:51:04.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:04 smithi191 ceph-mon[35211]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-11T22:51:04.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:04 smithi191 ceph-mon[35211]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-11T22:51:04.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:04 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/108004800' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T22:51:04.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:04 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] up:active 2024-09-11T22:51:04.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:04 smithi204 ceph-mon[35454]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T22:51:04.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:04 smithi204 ceph-mon[35454]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T22:51:04.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:04 smithi204 ceph-mon[35454]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-11T22:51:04.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:04 smithi204 ceph-mon[35454]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-11T22:51:04.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:04 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/108004800' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T22:51:04.589 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:04 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] up:active 2024-09-11T22:51:04.589 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:04 smithi152 ceph-mon[31181]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T22:51:04.589 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:04 smithi152 ceph-mon[31181]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T22:51:04.589 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:04 smithi152 ceph-mon[31181]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-11T22:51:04.590 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:04 smithi152 ceph-mon[31181]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-11T22:51:04.590 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:04 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/108004800' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T22:51:04.953 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-11T22:51:05.206 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:05.477 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:05 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/108004800' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T22:51:05.477 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:05 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] up:active 2024-09-11T22:51:05.477 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:05 smithi152 ceph-mon[31181]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T22:51:05.477 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:05 smithi152 ceph-mon[31181]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-11T22:51:05.477 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:05 smithi152 ceph-mon[31181]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-11T22:51:05.477 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:05 smithi152 ceph-mon[31181]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-11T22:51:05.477 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:05 smithi152 ceph-mon[31181]: pgmap v286: 97 pgs: 97 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 8 op/s 2024-09-11T22:51:05.477 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:05 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] up:active 2024-09-11T22:51:05.477 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:05 smithi152 ceph-mon[31181]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T22:51:05.477 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:05 smithi152 ceph-mon[31181]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T22:51:05.478 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:05 smithi152 ceph-mon[31181]: Cluster is now healthy 2024-09-11T22:51:05.478 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:05 smithi152 ceph-mon[31181]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-09-11T22:51:05.478 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:05 smithi152 ceph-mon[31181]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-11T22:51:05.478 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:05 smithi152 ceph-mon[31181]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-11T22:51:05.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:05 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/108004800' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T22:51:05.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:05 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] up:active 2024-09-11T22:51:05.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:05 smithi191 ceph-mon[35211]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T22:51:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:05 smithi191 ceph-mon[35211]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-11T22:51:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:05 smithi191 ceph-mon[35211]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-11T22:51:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:05 smithi191 ceph-mon[35211]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-11T22:51:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:05 smithi191 ceph-mon[35211]: pgmap v286: 97 pgs: 97 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 8 op/s 2024-09-11T22:51:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:05 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] up:active 2024-09-11T22:51:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:05 smithi191 ceph-mon[35211]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T22:51:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:05 smithi191 ceph-mon[35211]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T22:51:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:05 smithi191 ceph-mon[35211]: Cluster is now healthy 2024-09-11T22:51:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:05 smithi191 ceph-mon[35211]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-09-11T22:51:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:05 smithi191 ceph-mon[35211]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-11T22:51:05.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:05 smithi191 ceph-mon[35211]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-11T22:51:05.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:05 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/108004800' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T22:51:05.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:05 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] up:active 2024-09-11T22:51:05.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:05 smithi204 ceph-mon[35454]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T22:51:05.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:05 smithi204 ceph-mon[35454]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-11T22:51:05.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:05 smithi204 ceph-mon[35454]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-11T22:51:05.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:05 smithi204 ceph-mon[35454]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-11T22:51:05.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:05 smithi204 ceph-mon[35454]: pgmap v286: 97 pgs: 97 active+clean; 581 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 8 op/s 2024-09-11T22:51:05.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:05 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] up:active 2024-09-11T22:51:05.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:05 smithi204 ceph-mon[35454]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T22:51:05.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:05 smithi204 ceph-mon[35454]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T22:51:05.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:05 smithi204 ceph-mon[35454]: Cluster is now healthy 2024-09-11T22:51:05.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:05 smithi204 ceph-mon[35454]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-09-11T22:51:05.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:05 smithi204 ceph-mon[35454]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-11T22:51:05.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:05 smithi204 ceph-mon[35454]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-09-11T22:51:05.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.668+0000 7f52e34f1640 1 -- 172.21.15.152:0/1051840580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52dc100a80 msgr2=0x7f52dc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:05.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.668+0000 7f52e34f1640 1 --2- 172.21.15.152:0/1051840580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52dc100a80 0x7f52dc100ee0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009930 tx=0x7f52d002a420 comp rx=0 tx=0).stop 2024-09-11T22:51:05.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.668+0000 7f52e34f1640 1 -- 172.21.15.152:0/1051840580 shutdown_connections 2024-09-11T22:51:05.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.668+0000 7f52e34f1640 1 --2- 172.21.15.152:0/1051840580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc101420 0x7f52dc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.668+0000 7f52e34f1640 1 --2- 172.21.15.152:0/1051840580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52dc100a80 0x7f52dc100ee0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.668+0000 7f52e34f1640 1 --2- 172.21.15.152:0/1051840580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52dc106a80 0x7f52dc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.668+0000 7f52e34f1640 1 -- 172.21.15.152:0/1051840580 >> 172.21.15.152:0/1051840580 conn(0x7f52dc0fc7a0 msgr2=0x7f52dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.668+0000 7f52e34f1640 1 -- 172.21.15.152:0/1051840580 shutdown_connections 2024-09-11T22:51:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.668+0000 7f52e34f1640 1 -- 172.21.15.152:0/1051840580 wait complete. 2024-09-11T22:51:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.669+0000 7f52e34f1640 1 Processor -- start 2024-09-11T22:51:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.669+0000 7f52e34f1640 1 -- start start 2024-09-11T22:51:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.669+0000 7f52e34f1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52dc100a80 0x7f52dc102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.669+0000 7f52e34f1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc101420 0x7f52dc102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.669+0000 7f52e34f1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52dc106a80 0x7f52dc1030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.669+0000 7f52e34f1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc110f20 con 0x7f52dc106a80 2024-09-11T22:51:05.711 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.669+0000 7f52e34f1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc110da0 con 0x7f52dc100a80 2024-09-11T22:51:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.669+0000 7f52e34f1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc1110a0 con 0x7f52dc101420 2024-09-11T22:51:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.669+0000 7f52e1a8e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52dc106a80 0x7f52dc1030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.669+0000 7f52e1a8e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52dc106a80 0x7f52dc1030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49308/0 (socket says 172.21.15.152:49308) 2024-09-11T22:51:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.669+0000 7f52e1a8e640 1 -- 172.21.15.152:0/3946001158 learned_addr learned my addr 172.21.15.152:0/3946001158 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.669+0000 7f52e0a8c640 1 --2- 172.21.15.152:0/3946001158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc101420 0x7f52dc102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.670+0000 7f52e128d640 1 --2- 172.21.15.152:0/3946001158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52dc100a80 0x7f52dc102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.670+0000 7f52e1a8e640 1 -- 172.21.15.152:0/3946001158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc101420 msgr2=0x7f52dc102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.670+0000 7f52e1a8e640 1 --2- 172.21.15.152:0/3946001158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc101420 0x7f52dc102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.670+0000 7f52e1a8e640 1 -- 172.21.15.152:0/3946001158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52dc100a80 msgr2=0x7f52dc102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.670+0000 7f52e1a8e640 1 --2- 172.21.15.152:0/3946001158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52dc100a80 0x7f52dc102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.670+0000 7f52e1a8e640 1 -- 172.21.15.152:0/3946001158 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc1a6a80 con 0x7f52dc106a80 2024-09-11T22:51:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.670+0000 7f52e128d640 1 --2- 172.21.15.152:0/3946001158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52dc100a80 0x7f52dc102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:05.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.670+0000 7f52e1a8e640 1 --2- 172.21.15.152:0/3946001158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52dc106a80 0x7f52dc1030d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f52d400b970 tx=0x7f52d400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.670+0000 7f52ce7fc640 1 -- 172.21.15.152:0/3946001158 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4019070 con 0x7f52dc106a80 2024-09-11T22:51:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.670+0000 7f52ce7fc640 1 -- 172.21.15.152:0/3946001158 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52d400cb40 con 0x7f52dc106a80 2024-09-11T22:51:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.670+0000 7f52e34f1640 1 -- 172.21.15.152:0/3946001158 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52dc1a6d70 con 0x7f52dc106a80 2024-09-11T22:51:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.670+0000 7f52ce7fc640 1 -- 172.21.15.152:0/3946001158 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4007650 con 0x7f52dc106a80 2024-09-11T22:51:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.670+0000 7f52e34f1640 1 -- 172.21.15.152:0/3946001158 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52dc1a72b0 con 0x7f52dc106a80 2024-09-11T22:51:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.671+0000 7f52e34f1640 1 -- 172.21.15.152:0/3946001158 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52dc100f60 con 0x7f52dc106a80 2024-09-11T22:51:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.672+0000 7f52ce7fc640 1 -- 172.21.15.152:0/3946001158 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f52d401c070 con 0x7f52dc106a80 2024-09-11T22:51:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.673+0000 7f52ce7fc640 1 --2- 172.21.15.152:0/3946001158 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f52c80a2b80 0x7f52c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.673+0000 7f52e128d640 1 --2- 172.21.15.152:0/3946001158 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f52c80a2b80 0x7f52c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.673+0000 7f52ce7fc640 1 -- 172.21.15.152:0/3946001158 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f52d40ce520 con 0x7f52dc106a80 2024-09-11T22:51:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.673+0000 7f52e128d640 1 --2- 172.21.15.152:0/3946001158 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f52c80a2b80 0x7f52c80a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f52d8006fd0 tx=0x7f52d8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:05.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.677+0000 7f52ce7fc640 1 -- 172.21.15.152:0/3946001158 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f52d409b5d0 con 0x7f52dc106a80 2024-09-11T22:51:05.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:05.880+0000 7f52e34f1640 1 -- 172.21.15.152:0/3946001158 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f52dc1a7540 con 0x7f52dc106a80 2024-09-11T22:51:06.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:06.127+0000 7f52ce7fc640 1 -- 172.21.15.152:0/3946001158 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7f52d409aed0 con 0x7f52dc106a80 2024-09-11T22:51:06.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:06.130+0000 7f52e34f1640 1 -- 172.21.15.152:0/3946001158 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f52c80a2b80 msgr2=0x7f52c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:06.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:06.130+0000 7f52e34f1640 1 --2- 172.21.15.152:0/3946001158 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f52c80a2b80 0x7f52c80a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f52d8006fd0 tx=0x7f52d8008040 comp rx=0 tx=0).stop 2024-09-11T22:51:06.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:06.130+0000 7f52e34f1640 1 -- 172.21.15.152:0/3946001158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52dc106a80 msgr2=0x7f52dc1030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:06.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:06.130+0000 7f52e34f1640 1 --2- 172.21.15.152:0/3946001158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52dc106a80 0x7f52dc1030d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f52d400b970 tx=0x7f52d400be30 comp rx=0 tx=0).stop 2024-09-11T22:51:06.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:06.130+0000 7f52e34f1640 1 -- 172.21.15.152:0/3946001158 shutdown_connections 2024-09-11T22:51:06.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:06.130+0000 7f52e34f1640 1 --2- 172.21.15.152:0/3946001158 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f52c80a2b80 0x7f52c80a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:06.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:06.130+0000 7f52e34f1640 1 --2- 172.21.15.152:0/3946001158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52dc106a80 0x7f52dc1030d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:06.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:06.130+0000 7f52e34f1640 1 --2- 172.21.15.152:0/3946001158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc101420 0x7f52dc102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:06.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:06.130+0000 7f52e34f1640 1 --2- 172.21.15.152:0/3946001158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52dc100a80 0x7f52dc102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:06.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:06.130+0000 7f52e34f1640 1 -- 172.21.15.152:0/3946001158 >> 172.21.15.152:0/3946001158 conn(0x7f52dc0fc7a0 msgr2=0x7f52dc104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:06.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:06.130+0000 7f52e34f1640 1 -- 172.21.15.152:0/3946001158 shutdown_connections 2024-09-11T22:51:06.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:06.130+0000 7f52e34f1640 1 -- 172.21.15.152:0/3946001158 wait complete. 2024-09-11T22:51:06.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:06 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3946001158' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T22:51:06.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:06 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3946001158' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T22:51:06.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:06 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] up:active 2024-09-11T22:51:06.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:06 smithi191 ceph-mon[35211]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:51:06.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:06 smithi191 ceph-mon[35211]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:51:06.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:06 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3946001158' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T22:51:06.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:06 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3946001158' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T22:51:06.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:06 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] up:active 2024-09-11T22:51:06.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:06 smithi204 ceph-mon[35454]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:51:06.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:06 smithi204 ceph-mon[35454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:51:06.606 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:06 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3946001158' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T22:51:06.606 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:06 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3946001158' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T22:51:06.607 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:06 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] up:active 2024-09-11T22:51:06.607 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:06 smithi152 ceph-mon[31181]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:51:06.607 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:06 smithi152 ceph-mon[31181]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:51:07.048 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-11T22:51:07.048 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-11T22:51:07.304 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:07.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:07 smithi152 ceph-mon[31181]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 528 B/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-11T22:51:07.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:07 smithi191 ceph-mon[35211]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 528 B/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-11T22:51:07.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:07 smithi204 ceph-mon[35454]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 528 B/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-11T22:51:07.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.800+0000 7fc757e6a640 1 -- 172.21.15.152:0/1251227069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc750100480 msgr2=0x7fc750100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:07.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.800+0000 7fc757e6a640 1 --2- 172.21.15.152:0/1251227069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc750100480 0x7fc750100860 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc740009ca0 tx=0x7fc74002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:51:07.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.800+0000 7fc757e6a640 1 -- 172.21.15.152:0/1251227069 shutdown_connections 2024-09-11T22:51:07.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.800+0000 7fc757e6a640 1 --2- 172.21.15.152:0/1251227069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7501055b0 0x7fc75010d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:07.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.800+0000 7fc757e6a640 1 --2- 172.21.15.152:0/1251227069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc750100e30 0x7fc750104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:07.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.800+0000 7fc757e6a640 1 --2- 172.21.15.152:0/1251227069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc750100480 0x7fc750100860 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:07.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.800+0000 7fc757e6a640 1 -- 172.21.15.152:0/1251227069 >> 172.21.15.152:0/1251227069 conn(0x7fc750077210 msgr2=0x7fc7500fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:07.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.800+0000 7fc757e6a640 1 -- 172.21.15.152:0/1251227069 shutdown_connections 2024-09-11T22:51:07.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.800+0000 7fc757e6a640 1 -- 172.21.15.152:0/1251227069 wait complete. 2024-09-11T22:51:07.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.801+0000 7fc757e6a640 1 Processor -- start 2024-09-11T22:51:07.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.801+0000 7fc757e6a640 1 -- start start 2024-09-11T22:51:07.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.801+0000 7fc757e6a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc750100480 0x7fc750199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:07.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.801+0000 7fc757e6a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc750100e30 0x7fc750199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:07.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.801+0000 7fc757e6a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7501055b0 0x7fc7501a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:07.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.801+0000 7fc757e6a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501106b0 con 0x7fc750100e30 2024-09-11T22:51:07.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.801+0000 7fc757e6a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc750110530 con 0x7fc750100480 2024-09-11T22:51:07.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.801+0000 7fc757e6a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc750110830 con 0x7fc7501055b0 2024-09-11T22:51:07.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.801+0000 7fc755405640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc750100e30 0x7fc750199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:07.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.801+0000 7fc755405640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc750100e30 0x7fc750199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49330/0 (socket says 172.21.15.152:49330) 2024-09-11T22:51:07.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.801+0000 7fc755405640 1 -- 172.21.15.152:0/2265249190 learned_addr learned my addr 172.21.15.152:0/2265249190 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:07.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.802+0000 7fc755c06640 1 --2- 172.21.15.152:0/2265249190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc750100480 0x7fc750199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:07.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.802+0000 7fc756407640 1 --2- 172.21.15.152:0/2265249190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7501055b0 0x7fc7501a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:07.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.802+0000 7fc755405640 1 -- 172.21.15.152:0/2265249190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7501055b0 msgr2=0x7fc7501a44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:07.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.802+0000 7fc755405640 1 --2- 172.21.15.152:0/2265249190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7501055b0 0x7fc7501a44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:07.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.802+0000 7fc755405640 1 -- 172.21.15.152:0/2265249190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc750100480 msgr2=0x7fc750199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:07.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.802+0000 7fc755405640 1 --2- 172.21.15.152:0/2265249190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc750100480 0x7fc750199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:07.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.802+0000 7fc755405640 1 -- 172.21.15.152:0/2265249190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7501a4a10 con 0x7fc750100e30 2024-09-11T22:51:07.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.802+0000 7fc756407640 1 --2- 172.21.15.152:0/2265249190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7501055b0 0x7fc7501a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:07.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.802+0000 7fc755405640 1 --2- 172.21.15.152:0/2265249190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc750100e30 0x7fc750199990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc74400b4a0 tx=0x7fc74400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:07.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.802+0000 7fc74effd640 1 -- 172.21.15.152:0/2265249190 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744014070 con 0x7fc750100e30 2024-09-11T22:51:07.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.802+0000 7fc74effd640 1 -- 172.21.15.152:0/2265249190 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc744010af0 con 0x7fc750100e30 2024-09-11T22:51:07.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.802+0000 7fc757e6a640 1 -- 172.21.15.152:0/2265249190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7501a4be0 con 0x7fc750100e30 2024-09-11T22:51:07.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.802+0000 7fc74effd640 1 -- 172.21.15.152:0/2265249190 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc744017d10 con 0x7fc750100e30 2024-09-11T22:51:07.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.802+0000 7fc757e6a640 1 -- 172.21.15.152:0/2265249190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7501a50f0 con 0x7fc750100e30 2024-09-11T22:51:07.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.803+0000 7fc757e6a640 1 -- 172.21.15.152:0/2265249190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7501008e0 con 0x7fc750100e30 2024-09-11T22:51:07.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.804+0000 7fc74effd640 1 -- 172.21.15.152:0/2265249190 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc744017070 con 0x7fc750100e30 2024-09-11T22:51:07.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.805+0000 7fc74effd640 1 --2- 172.21.15.152:0/2265249190 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc7380a2b80 0x7fc7380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:07.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.805+0000 7fc755c06640 1 --2- 172.21.15.152:0/2265249190 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc7380a2b80 0x7fc7380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:07.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.805+0000 7fc74effd640 1 -- 172.21.15.152:0/2265249190 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fc7440ce310 con 0x7fc750100e30 2024-09-11T22:51:07.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.805+0000 7fc755c06640 1 --2- 172.21.15.152:0/2265249190 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc7380a2b80 0x7fc7380a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc74002f540 tx=0x7fc74002f310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:07.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.809+0000 7fc74effd640 1 -- 172.21.15.152:0/2265249190 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc74409b3c0 con 0x7fc750100e30 2024-09-11T22:51:07.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:07.997+0000 7fc757e6a640 1 -- 172.21.15.152:0/2265249190 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc7500fece0 con 0x7fc7380a2b80 2024-09-11T22:51:08.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:08.021+0000 7fc74effd640 1 -- 172.21.15.152:0/2265249190 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc7500fece0 con 0x7fc7380a2b80 2024-09-11T22:51:08.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:08.023+0000 7fc757e6a640 1 -- 172.21.15.152:0/2265249190 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc7380a2b80 msgr2=0x7fc7380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:08.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:08.023+0000 7fc757e6a640 1 --2- 172.21.15.152:0/2265249190 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc7380a2b80 0x7fc7380a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc74002f540 tx=0x7fc74002f310 comp rx=0 tx=0).stop 2024-09-11T22:51:08.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:08.023+0000 7fc757e6a640 1 -- 172.21.15.152:0/2265249190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc750100e30 msgr2=0x7fc750199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:08.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:08.023+0000 7fc757e6a640 1 --2- 172.21.15.152:0/2265249190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc750100e30 0x7fc750199990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc74400b4a0 tx=0x7fc74400b960 comp rx=0 tx=0).stop 2024-09-11T22:51:08.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:08.023+0000 7fc757e6a640 1 -- 172.21.15.152:0/2265249190 shutdown_connections 2024-09-11T22:51:08.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:08.023+0000 7fc757e6a640 1 --2- 172.21.15.152:0/2265249190 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc7380a2b80 0x7fc7380a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:08.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:08.023+0000 7fc757e6a640 1 --2- 172.21.15.152:0/2265249190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7501055b0 0x7fc7501a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:08.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:08.023+0000 7fc757e6a640 1 --2- 172.21.15.152:0/2265249190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc750100e30 0x7fc750199990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:08.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:08.023+0000 7fc757e6a640 1 --2- 172.21.15.152:0/2265249190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc750100480 0x7fc750199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:08.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:08.023+0000 7fc757e6a640 1 -- 172.21.15.152:0/2265249190 >> 172.21.15.152:0/2265249190 conn(0x7fc750077210 msgr2=0x7fc7500fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:08.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:08.023+0000 7fc757e6a640 1 -- 172.21.15.152:0/2265249190 shutdown_connections 2024-09-11T22:51:08.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:08.023+0000 7fc757e6a640 1 -- 172.21.15.152:0/2265249190 wait complete. 2024-09-11T22:51:08.372 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:51:08 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:51:08.002+0000 7f57044e2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:51:08.373 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:51:08 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:51:08.002+0000 7f57044e2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:51:08.373 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:51:08 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:51:08.002+0000 7f57044e2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:51:08.373 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:51:08 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:51:08.002+0000 7f57044e2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:51:08.373 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:51:08 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:51:08.002+0000 7f57044e2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:51:08.912 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-11T22:51:09.160 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:09.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.638+0000 7f60e88dd640 1 -- 172.21.15.152:0/2307060069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60e41055b0 msgr2=0x7f60e410d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:09.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.638+0000 7f60e88dd640 1 --2- 172.21.15.152:0/2307060069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60e41055b0 0x7f60e410d130 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009ca0 tx=0x7f60dc02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:51:09.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.638+0000 7f60e88dd640 1 -- 172.21.15.152:0/2307060069 shutdown_connections 2024-09-11T22:51:09.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.638+0000 7f60e88dd640 1 --2- 172.21.15.152:0/2307060069 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60e41055b0 0x7f60e410d130 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:09.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.638+0000 7f60e88dd640 1 --2- 172.21.15.152:0/2307060069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e4100e30 0x7f60e4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:09.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.638+0000 7f60e88dd640 1 --2- 172.21.15.152:0/2307060069 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60e4100480 0x7f60e4100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:09.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.638+0000 7f60e88dd640 1 -- 172.21.15.152:0/2307060069 >> 172.21.15.152:0/2307060069 conn(0x7f60e4077210 msgr2=0x7f60e40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:09.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.638+0000 7f60e88dd640 1 -- 172.21.15.152:0/2307060069 shutdown_connections 2024-09-11T22:51:09.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.638+0000 7f60e88dd640 1 -- 172.21.15.152:0/2307060069 wait complete. 2024-09-11T22:51:09.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.639+0000 7f60e88dd640 1 Processor -- start 2024-09-11T22:51:09.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.639+0000 7f60e88dd640 1 -- start start 2024-09-11T22:51:09.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.639+0000 7f60e88dd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60e4100480 0x7f60e40fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:09.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.639+0000 7f60e88dd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60e4100e30 0x7f60e40fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:09.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.639+0000 7f60e88dd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e41055b0 0x7f60e40fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:09.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.639+0000 7f60e88dd640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f60e4110630 con 0x7f60e4100480 2024-09-11T22:51:09.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.639+0000 7f60e88dd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41104b0 con 0x7f60e4100e30 2024-09-11T22:51:09.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.639+0000 7f60e88dd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f60e41107b0 con 0x7f60e41055b0 2024-09-11T22:51:09.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.639+0000 7f60e259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60e4100480 0x7f60e40fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:09.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60e259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60e4100480 0x7f60e40fc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33910/0 (socket says 172.21.15.152:33910) 2024-09-11T22:51:09.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60e259c640 1 -- 172.21.15.152:0/380933814 learned_addr learned my addr 172.21.15.152:0/380933814 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:09.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60dbfff640 1 --2- 172.21.15.152:0/380933814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60e4100e30 0x7f60e40fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:09.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60e2d9d640 1 --2- 172.21.15.152:0/380933814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e41055b0 0x7f60e40fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:09.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60e259c640 1 -- 172.21.15.152:0/380933814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e41055b0 msgr2=0x7f60e40fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:09.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60e259c640 1 --2- 172.21.15.152:0/380933814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e41055b0 0x7f60e40fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:09.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60e259c640 1 -- 172.21.15.152:0/380933814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60e4100e30 msgr2=0x7f60e40fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:09.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60e259c640 1 --2- 172.21.15.152:0/380933814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60e4100e30 0x7f60e40fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:09.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60e259c640 1 -- 172.21.15.152:0/380933814 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e41a69b0 con 0x7f60e4100480 2024-09-11T22:51:09.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60dbfff640 1 --2- 172.21.15.152:0/380933814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60e4100e30 0x7f60e40fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:51:09.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60e2d9d640 1 --2- 172.21.15.152:0/380933814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e41055b0 0x7f60e40fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:09.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60e259c640 1 --2- 172.21.15.152:0/380933814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60e4100480 0x7f60e40fc690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f60d400d950 tx=0x7f60d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:09.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60db7fe640 1 -- 172.21.15.152:0/380933814 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4017070 con 0x7f60e4100480 2024-09-11T22:51:09.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60e88dd640 1 -- 172.21.15.152:0/380933814 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e41a6ca0 con 0x7f60e4100480 2024-09-11T22:51:09.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60db7fe640 1 -- 172.21.15.152:0/380933814 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60d4009c30 con 0x7f60e4100480 2024-09-11T22:51:09.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60db7fe640 1 -- 172.21.15.152:0/380933814 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d400f6b0 con 0x7f60e4100480 2024-09-11T22:51:09.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.640+0000 7f60e88dd640 1 -- 172.21.15.152:0/380933814 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e41a71d0 con 0x7f60e4100480 2024-09-11T22:51:09.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.641+0000 7f60e88dd640 1 -- 172.21.15.152:0/380933814 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e4100860 con 0x7f60e4100480 2024-09-11T22:51:09.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.642+0000 7f60db7fe640 1 -- 172.21.15.152:0/380933814 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f60d40109d0 con 0x7f60e4100480 2024-09-11T22:51:09.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.643+0000 7f60db7fe640 1 --2- 172.21.15.152:0/380933814 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f60d00a2b80 0x7f60d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:09.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.643+0000 7f60dbfff640 1 --2- 172.21.15.152:0/380933814 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f60d00a2b80 0x7f60d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:09.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.643+0000 7f60db7fe640 1 -- 172.21.15.152:0/380933814 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f60d40cebf0 con 0x7f60e4100480 2024-09-11T22:51:09.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.643+0000 7f60dbfff640 1 --2- 172.21.15.152:0/380933814 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f60d00a2b80 0x7f60d00a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f60e40fe120 tx=0x7f60cc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:09.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.647+0000 7f60db7fe640 1 -- 172.21.15.152:0/380933814 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60d4009de0 con 0x7f60e4100480 2024-09-11T22:51:09.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.832+0000 7f60db7fe640 1 -- 172.21.15.152:0/380933814 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f60d4097c10 con 0x7f60e4100480 2024-09-11T22:51:09.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.847+0000 7f60e88dd640 1 -- 172.21.15.152:0/380933814 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f60e4103cf0 con 0x7f60d00a2b80 2024-09-11T22:51:09.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.871+0000 7f60db7fe640 1 -- 172.21.15.152:0/380933814 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f60e4103cf0 con 0x7f60d00a2b80 2024-09-11T22:51:09.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.873+0000 7f60e88dd640 1 -- 172.21.15.152:0/380933814 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f60d00a2b80 msgr2=0x7f60d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:09.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.873+0000 7f60e88dd640 1 --2- 172.21.15.152:0/380933814 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f60d00a2b80 0x7f60d00a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f60e40fe120 tx=0x7f60cc008040 comp rx=0 tx=0).stop 2024-09-11T22:51:09.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.873+0000 7f60e88dd640 1 -- 172.21.15.152:0/380933814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60e4100480 msgr2=0x7f60e40fc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:09.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.873+0000 7f60e88dd640 1 --2- 172.21.15.152:0/380933814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60e4100480 0x7f60e40fc690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f60d400d950 tx=0x7f60d400de10 comp rx=0 tx=0).stop 2024-09-11T22:51:09.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.873+0000 7f60e88dd640 1 -- 172.21.15.152:0/380933814 shutdown_connections 2024-09-11T22:51:09.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.873+0000 7f60e88dd640 1 --2- 172.21.15.152:0/380933814 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f60d00a2b80 0x7f60d00a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:09.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.873+0000 7f60e88dd640 1 --2- 172.21.15.152:0/380933814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e41055b0 0x7f60e40fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:09.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.873+0000 7f60e88dd640 1 --2- 172.21.15.152:0/380933814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60e4100e30 0x7f60e40fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:09.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.873+0000 7f60e88dd640 1 --2- 172.21.15.152:0/380933814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f60e4100480 0x7f60e40fc690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:09.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.873+0000 7f60e88dd640 1 -- 172.21.15.152:0/380933814 >> 172.21.15.152:0/380933814 conn(0x7f60e4077210 msgr2=0x7f60e40fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:09.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.873+0000 7f60e88dd640 1 -- 172.21.15.152:0/380933814 shutdown_connections 2024-09-11T22:51:09.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:09.873+0000 7f60e88dd640 1 -- 172.21.15.152:0/380933814 wait complete. 2024-09-11T22:51:10.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:09 smithi152 ceph-mon[31181]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:51:10.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:09 smithi152 ceph-mon[31181]: pgmap v288: 97 pgs: 97 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2024-09-11T22:51:10.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:09 smithi191 ceph-mon[35211]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:51:10.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:09 smithi191 ceph-mon[35211]: pgmap v288: 97 pgs: 97 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2024-09-11T22:51:10.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:09 smithi204 ceph-mon[35454]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:51:10.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:09 smithi204 ceph-mon[35454]: pgmap v288: 97 pgs: 97 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4.7 KiB/s wr, 16 op/s 2024-09-11T22:51:10.707 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-11T22:51:10.958 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:11.079 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:10 smithi152 ceph-mon[31181]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T22:51:11.079 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:10 smithi152 ceph-mon[31181]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:51:11.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:10 smithi191 ceph-mon[35211]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T22:51:11.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:10 smithi191 ceph-mon[35211]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:51:11.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:10 smithi204 ceph-mon[35454]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T22:51:11.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:10 smithi204 ceph-mon[35454]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:51:11.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.449+0000 7f12621be640 1 -- 172.21.15.152:0/3242261844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f125c103660 msgr2=0x7f125c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:11.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.449+0000 7f12621be640 1 --2- 172.21.15.152:0/3242261844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f125c103660 0x7f125c10bb50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1250009900 tx=0x7f125002a480 comp rx=0 tx=0).stop 2024-09-11T22:51:11.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.449+0000 7f12621be640 1 -- 172.21.15.152:0/3242261844 shutdown_connections 2024-09-11T22:51:11.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.449+0000 7f12621be640 1 --2- 172.21.15.152:0/3242261844 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125c10c170 0x7f125c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:11.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.449+0000 7f12621be640 1 --2- 172.21.15.152:0/3242261844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f125c103660 0x7f125c10bb50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:11.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.449+0000 7f12621be640 1 --2- 172.21.15.152:0/3242261844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f125c102d40 0x7f125c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:11.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.449+0000 7f12621be640 1 -- 172.21.15.152:0/3242261844 >> 172.21.15.152:0/3242261844 conn(0x7f125c0fc7a0 msgr2=0x7f125c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:11.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.449+0000 7f12621be640 1 -- 172.21.15.152:0/3242261844 shutdown_connections 2024-09-11T22:51:11.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.449+0000 7f12621be640 1 -- 172.21.15.152:0/3242261844 wait complete. 2024-09-11T22:51:11.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.449+0000 7f12621be640 1 Processor -- start 2024-09-11T22:51:11.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f12621be640 1 -- start start 2024-09-11T22:51:11.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f12621be640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125c102d40 0x7f125c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:11.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f12621be640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f125c103660 0x7f125c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:11.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f12621be640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f125c10c170 0x7f125c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:11.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f12621be640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f125c111fb0 con 0x7f125c10c170 2024-09-11T22:51:11.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f12621be640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f125c111e30 con 0x7f125c102d40 2024-09-11T22:51:11.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f12621be640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f125c112130 con 0x7f125c103660 2024-09-11T22:51:11.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f125bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f125c10c170 0x7f125c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:11.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f125bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f125c10c170 0x7f125c101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33934/0 (socket says 172.21.15.152:33934) 2024-09-11T22:51:11.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f125bfff640 1 -- 172.21.15.152:0/2119663654 learned_addr learned my addr 172.21.15.152:0/2119663654 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:11.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f125b7fe640 1 --2- 172.21.15.152:0/2119663654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125c102d40 0x7f125c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:11.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f125affd640 1 --2- 172.21.15.152:0/2119663654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f125c103660 0x7f125c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:11.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f125bfff640 1 -- 172.21.15.152:0/2119663654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f125c103660 msgr2=0x7f125c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:11.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f125bfff640 1 --2- 172.21.15.152:0/2119663654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f125c103660 0x7f125c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:11.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f125bfff640 1 -- 172.21.15.152:0/2119663654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125c102d40 msgr2=0x7f125c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:11.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f125bfff640 1 --2- 172.21.15.152:0/2119663654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125c102d40 0x7f125c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:11.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f125bfff640 1 -- 172.21.15.152:0/2119663654 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125c1a6920 con 0x7f125c10c170 2024-09-11T22:51:11.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.450+0000 7f125bfff640 1 --2- 172.21.15.152:0/2119663654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f125c10c170 0x7f125c101420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f125400cce0 tx=0x7f125400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:11.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.451+0000 7f1258ff9640 1 -- 172.21.15.152:0/2119663654 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125400f450 con 0x7f125c10c170 2024-09-11T22:51:11.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.451+0000 7f12621be640 1 -- 172.21.15.152:0/2119663654 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125c1a6c10 con 0x7f125c10c170 2024-09-11T22:51:11.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.451+0000 7f1258ff9640 1 -- 172.21.15.152:0/2119663654 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f125400fad0 con 0x7f125c10c170 2024-09-11T22:51:11.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.451+0000 7f1258ff9640 1 -- 172.21.15.152:0/2119663654 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1254018b60 con 0x7f125c10c170 2024-09-11T22:51:11.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.451+0000 7f12621be640 1 -- 172.21.15.152:0/2119663654 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f125c1a70d0 con 0x7f125c10c170 2024-09-11T22:51:11.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.452+0000 7f12621be640 1 -- 172.21.15.152:0/2119663654 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125c103120 con 0x7f125c10c170 2024-09-11T22:51:11.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.453+0000 7f1258ff9640 1 -- 172.21.15.152:0/2119663654 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f125400fc80 con 0x7f125c10c170 2024-09-11T22:51:11.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.454+0000 7f1258ff9640 1 --2- 172.21.15.152:0/2119663654 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f12440a2c10 0x7f12440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:11.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.454+0000 7f125b7fe640 1 --2- 172.21.15.152:0/2119663654 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f12440a2c10 0x7f12440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:11.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.454+0000 7f1258ff9640 1 -- 172.21.15.152:0/2119663654 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f12540cf4b0 con 0x7f125c10c170 2024-09-11T22:51:11.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.454+0000 7f125b7fe640 1 --2- 172.21.15.152:0/2119663654 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f12440a2c10 0x7f12440a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f124c009990 tx=0x7f124c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:11.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.458+0000 7f1258ff9640 1 -- 172.21.15.152:0/2119663654 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1254014030 con 0x7f125c10c170 2024-09-11T22:51:11.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.650+0000 7f12621be640 1 -- 172.21.15.152:0/2119663654 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f125c107250 con 0x7f12440a2c10 2024-09-11T22:51:11.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.656+0000 7f1258ff9640 1 -- 172.21.15.152:0/2119663654 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f124c00e850 con 0x7f12440a2c10 2024-09-11T22:51:11.657 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/qa/sv_0/6a7889d5-d9b5-4028-b610-e2787456e45e 2024-09-11T22:51:11.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.658+0000 7f12621be640 1 -- 172.21.15.152:0/2119663654 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f12440a2c10 msgr2=0x7f12440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:11.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.658+0000 7f12621be640 1 --2- 172.21.15.152:0/2119663654 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f12440a2c10 0x7f12440a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f124c009990 tx=0x7f124c009110 comp rx=0 tx=0).stop 2024-09-11T22:51:11.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.658+0000 7f12621be640 1 -- 172.21.15.152:0/2119663654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f125c10c170 msgr2=0x7f125c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.658+0000 7f12621be640 1 --2- 172.21.15.152:0/2119663654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f125c10c170 0x7f125c101420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f125400cce0 tx=0x7f125400a620 comp rx=0 tx=0).stop 2024-09-11T22:51:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.658+0000 7f12621be640 1 -- 172.21.15.152:0/2119663654 shutdown_connections 2024-09-11T22:51:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.658+0000 7f12621be640 1 --2- 172.21.15.152:0/2119663654 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f12440a2c10 0x7f12440a50d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.658+0000 7f12621be640 1 --2- 172.21.15.152:0/2119663654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f125c10c170 0x7f125c101420 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.658+0000 7f12621be640 1 --2- 172.21.15.152:0/2119663654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f125c103660 0x7f125c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.658+0000 7f12621be640 1 --2- 172.21.15.152:0/2119663654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f125c102d40 0x7f125c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.658+0000 7f12621be640 1 -- 172.21.15.152:0/2119663654 >> 172.21.15.152:0/2119663654 conn(0x7f125c0fc7a0 msgr2=0x7f125c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.658+0000 7f12621be640 1 -- 172.21.15.152:0/2119663654 shutdown_connections 2024-09-11T22:51:11.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:11.658+0000 7f12621be640 1 -- 172.21.15.152:0/2119663654 wait complete. 2024-09-11T22:51:12.105 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:11 smithi152 ceph-mon[31181]: pgmap v289: 97 pgs: 97 active+clean; 585 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-09-11T22:51:12.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:11 smithi191 ceph-mon[35211]: pgmap v289: 97 pgs: 97 active+clean; 585 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-09-11T22:51:12.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:11 smithi204 ceph-mon[35454]: pgmap v289: 97 pgs: 97 active+clean; 585 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-09-11T22:51:12.370 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-11T22:51:12.614 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:13.031 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:12 smithi152 ceph-mon[31181]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:51:13.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.114+0000 7f833852a640 1 -- 172.21.15.152:0/2482911293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8330106f20 msgr2=0x7f833010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:13.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.114+0000 7f833852a640 1 --2- 172.21.15.152:0/2482911293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8330106f20 0x7f833010eaa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8328009ca0 tx=0x7f832802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:51:13.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.114+0000 7f833852a640 1 -- 172.21.15.152:0/2482911293 shutdown_connections 2024-09-11T22:51:13.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.114+0000 7f833852a640 1 --2- 172.21.15.152:0/2482911293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8330106f20 0x7f833010eaa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:13.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.114+0000 7f833852a640 1 --2- 172.21.15.152:0/2482911293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83301027a0 0x7f83301067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:13.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.114+0000 7f833852a640 1 --2- 172.21.15.152:0/2482911293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8330101df0 0x7f83301021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:13.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.114+0000 7f833852a640 1 -- 172.21.15.152:0/2482911293 >> 172.21.15.152:0/2482911293 conn(0x7f83300fd8b0 msgr2=0x7f83300ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:13.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.114+0000 7f833852a640 1 -- 172.21.15.152:0/2482911293 shutdown_connections 2024-09-11T22:51:13.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.114+0000 7f833852a640 1 -- 172.21.15.152:0/2482911293 wait complete. 2024-09-11T22:51:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f833852a640 1 Processor -- start 2024-09-11T22:51:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f833852a640 1 -- start start 2024-09-11T22:51:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f833852a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8330101df0 0x7f83301983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f833852a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83301027a0 0x7f83301988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f833852a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8330106f20 0x7f833019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f833852a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8330111f80 con 0x7f8330106f20 2024-09-11T22:51:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f833852a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8330111e00 con 0x7f83301027a0 2024-09-11T22:51:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f833852a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8330112100 con 0x7f8330101df0 2024-09-11T22:51:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f8336ac7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8330106f20 0x7f833019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f8336ac7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8330106f20 0x7f833019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33960/0 (socket says 172.21.15.152:33960) 2024-09-11T22:51:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f8336ac7640 1 -- 172.21.15.152:0/2089283256 learned_addr learned my addr 172.21.15.152:0/2089283256 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:13.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f83362c6640 1 --2- 172.21.15.152:0/2089283256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8330101df0 0x7f83301983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:13.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f8336ac7640 1 -- 172.21.15.152:0/2089283256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8330101df0 msgr2=0x7f83301983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:13.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f8336ac7640 1 --2- 172.21.15.152:0/2089283256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8330101df0 0x7f83301983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:13.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f8336ac7640 1 -- 172.21.15.152:0/2089283256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83301027a0 msgr2=0x7f83301988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:13.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f8335ac5640 1 --2- 172.21.15.152:0/2089283256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83301027a0 0x7f83301988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:13.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f8336ac7640 1 --2- 172.21.15.152:0/2089283256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83301027a0 0x7f83301988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:13.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.115+0000 7f8336ac7640 1 -- 172.21.15.152:0/2089283256 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833019d350 con 0x7f8330106f20 2024-09-11T22:51:13.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.116+0000 7f8336ac7640 1 --2- 172.21.15.152:0/2089283256 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8330106f20 0x7f833019cc70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8328031a20 tx=0x7f8328031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:13.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.116+0000 7f83277fe640 1 -- 172.21.15.152:0/2089283256 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328003c30 con 0x7f8330106f20 2024-09-11T22:51:13.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.116+0000 7f83277fe640 1 -- 172.21.15.152:0/2089283256 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8328003dd0 con 0x7f8330106f20 2024-09-11T22:51:13.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.116+0000 7f833852a640 1 -- 172.21.15.152:0/2089283256 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833019d5e0 con 0x7f8330106f20 2024-09-11T22:51:13.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.116+0000 7f83277fe640 1 -- 172.21.15.152:0/2089283256 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832802fcf0 con 0x7f8330106f20 2024-09-11T22:51:13.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.116+0000 7f833852a640 1 -- 172.21.15.152:0/2089283256 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83301a4f20 con 0x7f8330106f20 2024-09-11T22:51:13.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.117+0000 7f833852a640 1 -- 172.21.15.152:0/2089283256 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83301021d0 con 0x7f8330106f20 2024-09-11T22:51:13.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.118+0000 7f83277fe640 1 -- 172.21.15.152:0/2089283256 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8328004050 con 0x7f8330106f20 2024-09-11T22:51:13.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.118+0000 7f83277fe640 1 --2- 172.21.15.152:0/2089283256 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f831c0a2c90 0x7f831c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:13.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.119+0000 7f83362c6640 1 --2- 172.21.15.152:0/2089283256 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f831c0a2c90 0x7f831c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:13.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.119+0000 7f83277fe640 1 -- 172.21.15.152:0/2089283256 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f83280f0220 con 0x7f8330106f20 2024-09-11T22:51:13.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.119+0000 7f83362c6640 1 --2- 172.21.15.152:0/2089283256 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f831c0a2c90 0x7f831c0a5150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f832c00bbe0 tx=0x7f832c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:13.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.123+0000 7f83277fe640 1 -- 172.21.15.152:0/2089283256 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83280bd490 con 0x7f8330106f20 2024-09-11T22:51:13.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:12 smithi191 ceph-mon[35211]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:51:13.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:12 smithi204 ceph-mon[35454]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:51:13.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.314+0000 7f833852a640 1 -- 172.21.15.152:0/2089283256 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8330062630 con 0x7f831c0a2c90 2024-09-11T22:51:13.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.338+0000 7f83277fe640 1 -- 172.21.15.152:0/2089283256 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8330062630 con 0x7f831c0a2c90 2024-09-11T22:51:13.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.340+0000 7f833852a640 1 -- 172.21.15.152:0/2089283256 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f831c0a2c90 msgr2=0x7f831c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:13.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.340+0000 7f833852a640 1 --2- 172.21.15.152:0/2089283256 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f831c0a2c90 0x7f831c0a5150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f832c00bbe0 tx=0x7f832c00b390 comp rx=0 tx=0).stop 2024-09-11T22:51:13.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.340+0000 7f833852a640 1 -- 172.21.15.152:0/2089283256 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8330106f20 msgr2=0x7f833019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:13.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.340+0000 7f833852a640 1 --2- 172.21.15.152:0/2089283256 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8330106f20 0x7f833019cc70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8328031a20 tx=0x7f8328031a50 comp rx=0 tx=0).stop 2024-09-11T22:51:13.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.340+0000 7f833852a640 1 -- 172.21.15.152:0/2089283256 shutdown_connections 2024-09-11T22:51:13.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.340+0000 7f833852a640 1 --2- 172.21.15.152:0/2089283256 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f831c0a2c90 0x7f831c0a5150 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:13.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.340+0000 7f833852a640 1 --2- 172.21.15.152:0/2089283256 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8330106f20 0x7f833019cc70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:13.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.340+0000 7f833852a640 1 --2- 172.21.15.152:0/2089283256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83301027a0 0x7f83301988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:13.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.340+0000 7f833852a640 1 --2- 172.21.15.152:0/2089283256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8330101df0 0x7f83301983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:13.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.340+0000 7f833852a640 1 -- 172.21.15.152:0/2089283256 >> 172.21.15.152:0/2089283256 conn(0x7f83300fd8b0 msgr2=0x7f83300feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:13.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.340+0000 7f833852a640 1 -- 172.21.15.152:0/2089283256 shutdown_connections 2024-09-11T22:51:13.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:13.340+0000 7f833852a640 1 -- 172.21.15.152:0/2089283256 wait complete. 2024-09-11T22:51:14.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:13 smithi152 ceph-mon[31181]: pgmap v290: 97 pgs: 97 active+clean; 605 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 22 op/s 2024-09-11T22:51:14.156 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-11T22:51:14.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:13 smithi191 ceph-mon[35211]: pgmap v290: 97 pgs: 97 active+clean; 605 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 22 op/s 2024-09-11T22:51:14.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:13 smithi204 ceph-mon[35454]: pgmap v290: 97 pgs: 97 active+clean; 605 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.2 KiB/s wr, 22 op/s 2024-09-11T22:51:14.402 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:14.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.883+0000 7f370f720640 1 -- 172.21.15.152:0/324814835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3708106a80 msgr2=0x7f3708106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:14.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.883+0000 7f370f720640 1 --2- 172.21.15.152:0/324814835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3708106a80 0x7f3708106e60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3704009930 tx=0x7f370402a420 comp rx=0 tx=0).stop 2024-09-11T22:51:14.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.883+0000 7f370f720640 1 -- 172.21.15.152:0/324814835 shutdown_connections 2024-09-11T22:51:14.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.883+0000 7f370f720640 1 --2- 172.21.15.152:0/324814835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3708101420 0x7f370810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:14.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.883+0000 7f370f720640 1 --2- 172.21.15.152:0/324814835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3708100a80 0x7f3708100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:14.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.883+0000 7f370f720640 1 --2- 172.21.15.152:0/324814835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3708106a80 0x7f3708106e60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:14.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.883+0000 7f370f720640 1 -- 172.21.15.152:0/324814835 >> 172.21.15.152:0/324814835 conn(0x7f37080fc7a0 msgr2=0x7f37080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:14.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.883+0000 7f370f720640 1 -- 172.21.15.152:0/324814835 shutdown_connections 2024-09-11T22:51:14.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.883+0000 7f370f720640 1 -- 172.21.15.152:0/324814835 wait complete. 2024-09-11T22:51:14.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.884+0000 7f370f720640 1 Processor -- start 2024-09-11T22:51:14.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.884+0000 7f370f720640 1 -- start start 2024-09-11T22:51:14.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.884+0000 7f370f720640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3708100a80 0x7f37081972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:14.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.884+0000 7f370f720640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3708101420 0x7f3708197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:14.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.884+0000 7f370f720640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3708106a80 0x7f370819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:14.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.884+0000 7f370f720640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3708110e80 con 0x7f3708106a80 2024-09-11T22:51:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.884+0000 7f370f720640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3708110d00 con 0x7f3708100a80 2024-09-11T22:51:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.884+0000 7f370f720640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3708111000 con 0x7f3708101420 2024-09-11T22:51:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.884+0000 7f370dcbd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3708106a80 0x7f370819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.884+0000 7f370dcbd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3708106a80 0x7f370819bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33970/0 (socket says 172.21.15.152:33970) 2024-09-11T22:51:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.884+0000 7f370dcbd640 1 -- 172.21.15.152:0/3982934163 learned_addr learned my addr 172.21.15.152:0/3982934163 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.885+0000 7f370ccbb640 1 --2- 172.21.15.152:0/3982934163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3708101420 0x7f3708197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.885+0000 7f370d4bc640 1 --2- 172.21.15.152:0/3982934163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3708100a80 0x7f37081972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.885+0000 7f370dcbd640 1 -- 172.21.15.152:0/3982934163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3708101420 msgr2=0x7f3708197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.885+0000 7f370dcbd640 1 --2- 172.21.15.152:0/3982934163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3708101420 0x7f3708197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.885+0000 7f370dcbd640 1 -- 172.21.15.152:0/3982934163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3708100a80 msgr2=0x7f37081972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.885+0000 7f370dcbd640 1 --2- 172.21.15.152:0/3982934163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3708100a80 0x7f37081972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.885+0000 7f370dcbd640 1 -- 172.21.15.152:0/3982934163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370819c290 con 0x7f3708106a80 2024-09-11T22:51:14.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.885+0000 7f370ccbb640 1 --2- 172.21.15.152:0/3982934163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3708101420 0x7f3708197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.885+0000 7f370d4bc640 1 --2- 172.21.15.152:0/3982934163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3708100a80 0x7f37081972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.885+0000 7f370dcbd640 1 --2- 172.21.15.152:0/3982934163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3708106a80 0x7f370819bbb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f370000eb10 tx=0x7f370000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.885+0000 7f36fa7fc640 1 -- 172.21.15.152:0/3982934163 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700004b20 con 0x7f3708106a80 2024-09-11T22:51:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.885+0000 7f36fa7fc640 1 -- 172.21.15.152:0/3982934163 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3700004cc0 con 0x7f3708106a80 2024-09-11T22:51:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.885+0000 7f36fa7fc640 1 -- 172.21.15.152:0/3982934163 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3700010760 con 0x7f3708106a80 2024-09-11T22:51:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.885+0000 7f370f720640 1 -- 172.21.15.152:0/3982934163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370819c580 con 0x7f3708106a80 2024-09-11T22:51:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.885+0000 7f370f720640 1 -- 172.21.15.152:0/3982934163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37081a3e60 con 0x7f3708106a80 2024-09-11T22:51:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.886+0000 7f370f720640 1 -- 172.21.15.152:0/3982934163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3708100ee0 con 0x7f3708106a80 2024-09-11T22:51:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.887+0000 7f36fa7fc640 1 -- 172.21.15.152:0/3982934163 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3700005020 con 0x7f3708106a80 2024-09-11T22:51:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.888+0000 7f36fa7fc640 1 --2- 172.21.15.152:0/3982934163 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f36f40a2c10 0x7f36f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.888+0000 7f370d4bc640 1 --2- 172.21.15.152:0/3982934163 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f36f40a2c10 0x7f36f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:14.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.888+0000 7f36fa7fc640 1 -- 172.21.15.152:0/3982934163 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f37000cf7c0 con 0x7f3708106a80 2024-09-11T22:51:14.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.888+0000 7f370d4bc640 1 --2- 172.21.15.152:0/3982934163 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f36f40a2c10 0x7f36f40a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3704000c00 tx=0x7f3704032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:14.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:14.892+0000 7f36fa7fc640 1 -- 172.21.15.152:0/3982934163 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f370009c770 con 0x7f3708106a80 2024-09-11T22:51:15.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:15.087+0000 7f370f720640 1 -- 172.21.15.152:0/3982934163 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f37080fd5c0 con 0x7f36f40a2c10 2024-09-11T22:51:15.088 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:14 smithi152 ceph-mon[31181]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:51:15.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:15.091+0000 7f36fa7fc640 1 -- 172.21.15.152:0/3982934163 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f37080fd5c0 con 0x7f36f40a2c10 2024-09-11T22:51:15.092 INFO:teuthology.orchestra.run.smithi152.stdout:/volumes/qa/sv_1/97e40448-fd10-4885-b1e4-67d66e6e8574 2024-09-11T22:51:15.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:15.093+0000 7f370f720640 1 -- 172.21.15.152:0/3982934163 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f36f40a2c10 msgr2=0x7f36f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:15.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:15.093+0000 7f370f720640 1 --2- 172.21.15.152:0/3982934163 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f36f40a2c10 0x7f36f40a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3704000c00 tx=0x7f3704032c70 comp rx=0 tx=0).stop 2024-09-11T22:51:15.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:15.093+0000 7f370f720640 1 -- 172.21.15.152:0/3982934163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3708106a80 msgr2=0x7f370819bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:15.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:15.093+0000 7f370f720640 1 --2- 172.21.15.152:0/3982934163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3708106a80 0x7f370819bbb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f370000eb10 tx=0x7f370000c2d0 comp rx=0 tx=0).stop 2024-09-11T22:51:15.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:15.093+0000 7f370f720640 1 -- 172.21.15.152:0/3982934163 shutdown_connections 2024-09-11T22:51:15.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:15.093+0000 7f370f720640 1 --2- 172.21.15.152:0/3982934163 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f36f40a2c10 0x7f36f40a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:15.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:15.093+0000 7f370f720640 1 --2- 172.21.15.152:0/3982934163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3708106a80 0x7f370819bbb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:15.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:15.093+0000 7f370f720640 1 --2- 172.21.15.152:0/3982934163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3708101420 0x7f3708197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:15.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:15.093+0000 7f370f720640 1 --2- 172.21.15.152:0/3982934163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3708100a80 0x7f37081972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:15.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:15.093+0000 7f370f720640 1 -- 172.21.15.152:0/3982934163 >> 172.21.15.152:0/3982934163 conn(0x7f37080fc7a0 msgr2=0x7f37080fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:15.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:15.093+0000 7f370f720640 1 -- 172.21.15.152:0/3982934163 shutdown_connections 2024-09-11T22:51:15.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:15.093+0000 7f370f720640 1 -- 172.21.15.152:0/3982934163 wait complete. 2024-09-11T22:51:15.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:14 smithi191 ceph-mon[35211]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:51:15.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:14 smithi204 ceph-mon[35454]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:51:15.834 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:51:16.082 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:16.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:15 smithi152 ceph-mon[31181]: pgmap v291: 97 pgs: 97 active+clean; 605 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.0 KiB/s wr, 22 op/s 2024-09-11T22:51:16.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:15 smithi191 ceph-mon[35211]: pgmap v291: 97 pgs: 97 active+clean; 605 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.0 KiB/s wr, 22 op/s 2024-09-11T22:51:16.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:15 smithi204 ceph-mon[35454]: pgmap v291: 97 pgs: 97 active+clean; 605 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.0 KiB/s wr, 22 op/s 2024-09-11T22:51:16.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.563+0000 7f161776a640 1 -- 172.21.15.152:0/1221055037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1610102ef0 msgr2=0x7f161010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:16.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.563+0000 7f161776a640 1 --2- 172.21.15.152:0/1221055037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1610102ef0 0x7f161010da40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1608009ca0 tx=0x7f160802a950 comp rx=0 tx=0).stop 2024-09-11T22:51:16.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.563+0000 7f161776a640 1 -- 172.21.15.152:0/1221055037 shutdown_connections 2024-09-11T22:51:16.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.563+0000 7f161776a640 1 --2- 172.21.15.152:0/1221055037 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1610102ef0 0x7f161010da40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:16.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.563+0000 7f161776a640 1 --2- 172.21.15.152:0/1221055037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1610102570 0x7f16101029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:16.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.563+0000 7f161776a640 1 --2- 172.21.15.152:0/1221055037 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1610100e20 0x7f1610101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:16.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.563+0000 7f161776a640 1 -- 172.21.15.152:0/1221055037 >> 172.21.15.152:0/1221055037 conn(0x7f16100fc7d0 msgr2=0x7f16100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:16.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.563+0000 7f161776a640 1 -- 172.21.15.152:0/1221055037 shutdown_connections 2024-09-11T22:51:16.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.563+0000 7f161776a640 1 -- 172.21.15.152:0/1221055037 wait complete. 2024-09-11T22:51:16.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.563+0000 7f161776a640 1 Processor -- start 2024-09-11T22:51:16.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.563+0000 7f161776a640 1 -- start start 2024-09-11T22:51:16.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.563+0000 7f161776a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1610100e20 0x7f16101984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:16.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.563+0000 7f161776a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1610102570 0x7f1610198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:16.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.563+0000 7f161776a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1610102ef0 0x7f161019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:16.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f161776a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1610110fa0 con 0x7f1610102570 2024-09-11T22:51:16.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f161776a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1610110e20 con 0x7f1610102ef0 2024-09-11T22:51:16.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f161776a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1610111120 con 0x7f1610100e20 2024-09-11T22:51:16.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f1614d05640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1610102570 0x7f1610198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:16.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f1614d05640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1610102570 0x7f1610198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:34000/0 (socket says 172.21.15.152:34000) 2024-09-11T22:51:16.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f1614d05640 1 -- 172.21.15.152:0/1776793688 learned_addr learned my addr 172.21.15.152:0/1776793688 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:16.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f1615d07640 1 --2- 172.21.15.152:0/1776793688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1610102ef0 0x7f161019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:16.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f1615506640 1 --2- 172.21.15.152:0/1776793688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1610100e20 0x7f16101984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:16.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f1614d05640 1 -- 172.21.15.152:0/1776793688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1610100e20 msgr2=0x7f16101984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:16.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f1614d05640 1 --2- 172.21.15.152:0/1776793688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1610100e20 0x7f16101984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:16.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f1614d05640 1 -- 172.21.15.152:0/1776793688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1610102ef0 msgr2=0x7f161019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:16.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f1614d05640 1 --2- 172.21.15.152:0/1776793688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1610102ef0 0x7f161019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:16.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f1614d05640 1 -- 172.21.15.152:0/1776793688 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161019d480 con 0x7f1610102570 2024-09-11T22:51:16.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f1615d07640 1 --2- 172.21.15.152:0/1776793688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1610102ef0 0x7f161019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:16.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f1615506640 1 --2- 172.21.15.152:0/1776793688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1610100e20 0x7f16101984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:16.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f1614d05640 1 --2- 172.21.15.152:0/1776793688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1610102570 0x7f1610198a10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f160c00b4a0 tx=0x7f160c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:16.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f16027fc640 1 -- 172.21.15.152:0/1776793688 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c014070 con 0x7f1610102570 2024-09-11T22:51:16.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f161776a640 1 -- 172.21.15.152:0/1776793688 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161019d770 con 0x7f1610102570 2024-09-11T22:51:16.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f16027fc640 1 -- 172.21.15.152:0/1776793688 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f160c010af0 con 0x7f1610102570 2024-09-11T22:51:16.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f16027fc640 1 -- 172.21.15.152:0/1776793688 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160c017d10 con 0x7f1610102570 2024-09-11T22:51:16.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.564+0000 7f161776a640 1 -- 172.21.15.152:0/1776793688 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16101a5050 con 0x7f1610102570 2024-09-11T22:51:16.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.566+0000 7f161776a640 1 -- 172.21.15.152:0/1776793688 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1610101200 con 0x7f1610102570 2024-09-11T22:51:16.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.566+0000 7f16027fc640 1 -- 172.21.15.152:0/1776793688 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f160c017070 con 0x7f1610102570 2024-09-11T22:51:16.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.567+0000 7f16027fc640 1 --2- 172.21.15.152:0/1776793688 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f15fc0a2c10 0x7f15fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:16.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.567+0000 7f1615506640 1 --2- 172.21.15.152:0/1776793688 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f15fc0a2c10 0x7f15fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:16.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.567+0000 7f16027fc640 1 -- 172.21.15.152:0/1776793688 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f160c0ce400 con 0x7f1610102570 2024-09-11T22:51:16.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.568+0000 7f1615506640 1 --2- 172.21.15.152:0/1776793688 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f15fc0a2c10 0x7f15fc0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1604009b40 tx=0x7f1604005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:16.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.571+0000 7f16027fc640 1 -- 172.21.15.152:0/1776793688 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f160c09b4c0 con 0x7f1610102570 2024-09-11T22:51:16.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.770+0000 7f161776a640 1 -- 172.21.15.152:0/1776793688 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f161010a0a0 con 0x7f1610102570 2024-09-11T22:51:16.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.771+0000 7f16027fc640 1 -- 172.21.15.152:0/1776793688 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11988 (secure 0 0 0) 0x7f160c09adc0 con 0x7f1610102570 2024-09-11T22:51:16.772 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:51:16.772 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":27,"btime":"2024-09-11T22:51:06:131335+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:51:06.131313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[24455,24442,24434,24430,14628]},"id":1}]} 2024-09-11T22:51:16.772 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 27 2024-09-11T22:51:16.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.773+0000 7f161776a640 1 -- 172.21.15.152:0/1776793688 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f15fc0a2c10 msgr2=0x7f15fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:16.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.773+0000 7f161776a640 1 --2- 172.21.15.152:0/1776793688 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f15fc0a2c10 0x7f15fc0a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1604009b40 tx=0x7f1604005e50 comp rx=0 tx=0).stop 2024-09-11T22:51:16.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.773+0000 7f161776a640 1 -- 172.21.15.152:0/1776793688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1610102570 msgr2=0x7f1610198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:16.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.773+0000 7f161776a640 1 --2- 172.21.15.152:0/1776793688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1610102570 0x7f1610198a10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f160c00b4a0 tx=0x7f160c00b960 comp rx=0 tx=0).stop 2024-09-11T22:51:16.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.774+0000 7f161776a640 1 -- 172.21.15.152:0/1776793688 shutdown_connections 2024-09-11T22:51:16.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.774+0000 7f161776a640 1 --2- 172.21.15.152:0/1776793688 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f15fc0a2c10 0x7f15fc0a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:16.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.774+0000 7f161776a640 1 --2- 172.21.15.152:0/1776793688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1610102ef0 0x7f161019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:16.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.774+0000 7f161776a640 1 --2- 172.21.15.152:0/1776793688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1610102570 0x7f1610198a10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:16.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.774+0000 7f161776a640 1 --2- 172.21.15.152:0/1776793688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1610100e20 0x7f16101984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:16.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.774+0000 7f161776a640 1 -- 172.21.15.152:0/1776793688 >> 172.21.15.152:0/1776793688 conn(0x7f16100fc7d0 msgr2=0x7f161006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:16.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.774+0000 7f161776a640 1 -- 172.21.15.152:0/1776793688 shutdown_connections 2024-09-11T22:51:16.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:16.774+0000 7f161776a640 1 -- 172.21.15.152:0/1776793688 wait complete. 2024-09-11T22:51:17.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:16 smithi152 ceph-mon[31181]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:51:17.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:16 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1776793688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:51:17.182 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:16 smithi191 ceph-mon[35211]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:51:17.183 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:16 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1776793688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:51:17.199 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:16 smithi204 ceph-mon[35454]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:51:17.200 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:16 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1776793688' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:51:17.643 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:51:17.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:17 smithi152 ceph-mon[31181]: pgmap v292: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 KiB/s wr, 23 op/s 2024-09-11T22:51:17.892 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:18.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:17 smithi191 ceph-mon[35211]: pgmap v292: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 KiB/s wr, 23 op/s 2024-09-11T22:51:18.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:17 smithi204 ceph-mon[35454]: pgmap v292: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 KiB/s wr, 23 op/s 2024-09-11T22:51:18.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.407+0000 7fc8354ed640 1 -- 172.21.15.152:0/40172343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc830101420 msgr2=0x7fc83010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:18.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.407+0000 7fc8354ed640 1 --2- 172.21.15.152:0/40172343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc830101420 0x7fc83010d950 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc828009cd0 tx=0x7fc82802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:51:18.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.408+0000 7fc8354ed640 1 -- 172.21.15.152:0/40172343 shutdown_connections 2024-09-11T22:51:18.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.408+0000 7fc8354ed640 1 --2- 172.21.15.152:0/40172343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc830101420 0x7fc83010d950 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:18.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.408+0000 7fc8354ed640 1 --2- 172.21.15.152:0/40172343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc830100a80 0x7fc830100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:18.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.408+0000 7fc8354ed640 1 --2- 172.21.15.152:0/40172343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc830106a80 0x7fc830106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:18.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.408+0000 7fc8354ed640 1 -- 172.21.15.152:0/40172343 >> 172.21.15.152:0/40172343 conn(0x7fc8300fc7a0 msgr2=0x7fc8300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:18.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.408+0000 7fc8354ed640 1 -- 172.21.15.152:0/40172343 shutdown_connections 2024-09-11T22:51:18.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.408+0000 7fc8354ed640 1 -- 172.21.15.152:0/40172343 wait complete. 2024-09-11T22:51:18.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.408+0000 7fc8354ed640 1 Processor -- start 2024-09-11T22:51:18.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.408+0000 7fc8354ed640 1 -- start start 2024-09-11T22:51:18.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc8354ed640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc830100a80 0x7fc830198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:18.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc8354ed640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc830101420 0x7fc830198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:18.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc8354ed640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc830106a80 0x7fc83019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:18.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc8354ed640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc830110ed0 con 0x7fc830100a80 2024-09-11T22:51:18.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc8354ed640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc830110d50 con 0x7fc830101420 2024-09-11T22:51:18.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc8354ed640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc830111050 con 0x7fc830106a80 2024-09-11T22:51:18.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc82effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc830100a80 0x7fc830198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:18.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc82effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc830100a80 0x7fc830198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58542/0 (socket says 172.21.15.152:58542) 2024-09-11T22:51:18.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc82effd640 1 -- 172.21.15.152:0/1379140758 learned_addr learned my addr 172.21.15.152:0/1379140758 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:18.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc82e7fc640 1 --2- 172.21.15.152:0/1379140758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc830101420 0x7fc830198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:18.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc82f7fe640 1 --2- 172.21.15.152:0/1379140758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc830106a80 0x7fc83019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:18.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc82effd640 1 -- 172.21.15.152:0/1379140758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc830106a80 msgr2=0x7fc83019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:18.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc82effd640 1 --2- 172.21.15.152:0/1379140758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc830106a80 0x7fc83019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:18.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc82effd640 1 -- 172.21.15.152:0/1379140758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc830101420 msgr2=0x7fc830198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:18.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc82effd640 1 --2- 172.21.15.152:0/1379140758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc830101420 0x7fc830198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:18.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc82effd640 1 -- 172.21.15.152:0/1379140758 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83019d3d0 con 0x7fc830100a80 2024-09-11T22:51:18.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc82effd640 1 --2- 172.21.15.152:0/1379140758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc830100a80 0x7fc830198420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc82000c970 tx=0x7fc82000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:18.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc817fff640 1 -- 172.21.15.152:0/1379140758 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820007bb0 con 0x7fc830100a80 2024-09-11T22:51:18.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc8354ed640 1 -- 172.21.15.152:0/1379140758 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83019d6c0 con 0x7fc830100a80 2024-09-11T22:51:18.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc817fff640 1 -- 172.21.15.152:0/1379140758 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc820007d50 con 0x7fc830100a80 2024-09-11T22:51:18.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc817fff640 1 -- 172.21.15.152:0/1379140758 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82000f450 con 0x7fc830100a80 2024-09-11T22:51:18.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.409+0000 7fc8354ed640 1 -- 172.21.15.152:0/1379140758 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8301a4fa0 con 0x7fc830100a80 2024-09-11T22:51:18.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.410+0000 7fc8354ed640 1 -- 172.21.15.152:0/1379140758 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc830100ee0 con 0x7fc830100a80 2024-09-11T22:51:18.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.411+0000 7fc817fff640 1 -- 172.21.15.152:0/1379140758 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc820005020 con 0x7fc830100a80 2024-09-11T22:51:18.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.412+0000 7fc817fff640 1 --2- 172.21.15.152:0/1379140758 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc8180a2c10 0x7fc8180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:18.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.412+0000 7fc82e7fc640 1 --2- 172.21.15.152:0/1379140758 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc8180a2c10 0x7fc8180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:18.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.412+0000 7fc817fff640 1 -- 172.21.15.152:0/1379140758 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fc8200cedb0 con 0x7fc830100a80 2024-09-11T22:51:18.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.413+0000 7fc82e7fc640 1 --2- 172.21.15.152:0/1379140758 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc8180a2c10 0x7fc8180a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc830105100 tx=0x7fc82400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:18.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.417+0000 7fc817fff640 1 -- 172.21.15.152:0/1379140758 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc820014030 con 0x7fc830100a80 2024-09-11T22:51:18.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.583+0000 7fc8354ed640 1 -- 172.21.15.152:0/1379140758 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc83010a060 con 0x7fc830100a80 2024-09-11T22:51:18.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.585+0000 7fc817fff640 1 -- 172.21.15.152:0/1379140758 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22363 (secure 0 0 0) 0x7fc82009bd60 con 0x7fc830100a80 2024-09-11T22:51:18.586 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:51:18.586 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":81,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","created":"2024-09-11T22:43:00.379317+0000","modified":"2024-09-11T22:50:55.047140+0000","last_up_change":"2024-09-11T22:50:01.043181+0000","last_in_change":"2024-09-11T22:49:38.073870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:46:10.984707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:50:50.833547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:50:53.047604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3cd917b5-77f6-4a4a-8fd0-4ebe3034feac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6803","nonce":1270728007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6805","nonce":1270728007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6809","nonce":1270728007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6807","nonce":1270728007}]},"public_addr":"172.21.15.152:6803/1270728007","cluster_addr":"172.21.15.152:6805/1270728007","heartbeat_back_addr":"172.21.15.152:6809/1270728007","heartbeat_front_addr":"172.21.15.152:6807/1270728007","state":["exists","up"]},{"osd":1,"uuid":"ae589bc4-77b1-4522-9591-b0ce2a5432dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6801","nonce":354467155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6803","nonce":354467155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6807","nonce":354467155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6805","nonce":354467155}]},"public_addr":"172.21.15.191:6801/354467155","cluster_addr":"172.21.15.191:6803/354467155","heartbeat_back_addr":"172.21.15.191:6807/354467155","heartbeat_front_addr":"172.21.15.191:6805/354467155","state":["exists","up"]},{"osd":2,"uuid":"b35ffd0c-d084-43dd-a947-15132d78d59c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6801","nonce":3848396211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6803","nonce":3848396211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6807","nonce":3848396211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6805","nonce":3848396211}]},"public_addr":"172.21.15.204:6801/3848396211","cluster_addr":"172.21.15.204:6803/3848396211","heartbeat_back_addr":"172.21.15.204:6807/3848396211","heartbeat_front_addr":"172.21.15.204:6805/3848396211","state":["exists","up"]},{"osd":3,"uuid":"d66ba7e8-4ab7-449e-9460-2f44ffdf987f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6811","nonce":3351257181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6813","nonce":3351257181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6817","nonce":3351257181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6815","nonce":3351257181}]},"public_addr":"172.21.15.152:6811/3351257181","cluster_addr":"172.21.15.152:6813/3351257181","heartbeat_back_addr":"172.21.15.152:6817/3351257181","heartbeat_front_addr":"172.21.15.152:6815/3351257181","state":["exists","up"]},{"osd":4,"uuid":"e1b56468-6cc7-4d29-988e-9d281118ee36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6809","nonce":2967539563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6811","nonce":2967539563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6815","nonce":2967539563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6813","nonce":2967539563}]},"public_addr":"172.21.15.191:6809/2967539563","cluster_addr":"172.21.15.191:6811/2967539563","heartbeat_back_addr":"172.21.15.191:6815/2967539563","heartbeat_front_addr":"172.21.15.191:6813/2967539563","state":["exists","up"]},{"osd":5,"uuid":"89dabd3e-77e3-4294-9648-b112ef694e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6809","nonce":4283562158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6811","nonce":4283562158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6815","nonce":4283562158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6813","nonce":4283562158}]},"public_addr":"172.21.15.204:6809/4283562158","cluster_addr":"172.21.15.204:6811/4283562158","heartbeat_back_addr":"172.21.15.204:6815/4283562158","heartbeat_front_addr":"172.21.15.204:6813/4283562158","state":["exists","up"]},{"osd":6,"uuid":"46a0fded-b668-4aef-a68f-613cccc83483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6819","nonce":2484234042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6821","nonce":2484234042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6825","nonce":2484234042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6823","nonce":2484234042}]},"public_addr":"172.21.15.152:6819/2484234042","cluster_addr":"172.21.15.152:6821/2484234042","heartbeat_back_addr":"172.21.15.152:6825/2484234042","heartbeat_front_addr":"172.21.15.152:6823/2484234042","state":["exists","up"]},{"osd":7,"uuid":"e674bfe9-e2db-44a3-b74b-a21f198747c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6817","nonce":269608859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6819","nonce":269608859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6823","nonce":269608859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6821","nonce":269608859}]},"public_addr":"172.21.15.191:6817/269608859","cluster_addr":"172.21.15.191:6819/269608859","heartbeat_back_addr":"172.21.15.191:6823/269608859","heartbeat_front_addr":"172.21.15.191:6821/269608859","state":["exists","up"]},{"osd":8,"uuid":"0418bfc9-d6b6-488f-b4c7-0bc6291bb624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6817","nonce":1756190090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6819","nonce":1756190090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6823","nonce":1756190090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6821","nonce":1756190090}]},"public_addr":"172.21.15.204:6817/1756190090","cluster_addr":"172.21.15.204:6819/1756190090","heartbeat_back_addr":"172.21.15.204:6823/1756190090","heartbeat_front_addr":"172.21.15.204:6821/1756190090","state":["exists","up"]},{"osd":9,"uuid":"751a6964-798f-4240-a2ba-7fa9f0c72657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6827","nonce":1301191419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6829","nonce":1301191419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6833","nonce":1301191419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6831","nonce":1301191419}]},"public_addr":"172.21.15.152:6827/1301191419","cluster_addr":"172.21.15.152:6829/1301191419","heartbeat_back_addr":"172.21.15.152:6833/1301191419","heartbeat_front_addr":"172.21.15.152:6831/1301191419","state":["exists","up"]},{"osd":10,"uuid":"90e7c6ba-127c-4b33-b3c6-cedf72128451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6825","nonce":3695294503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6827","nonce":3695294503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6831","nonce":3695294503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6829","nonce":3695294503}]},"public_addr":"172.21.15.191:6825/3695294503","cluster_addr":"172.21.15.191:6827/3695294503","heartbeat_back_addr":"172.21.15.191:6831/3695294503","heartbeat_front_addr":"172.21.15.191:6829/3695294503","state":["exists","up"]},{"osd":11,"uuid":"c76a93b1-a045-477b-98d0-58a2e9d44a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6825","nonce":1709433236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6827","nonce":1709433236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6831","nonce":1709433236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6829","nonce":1709433236}]},"public_addr":"172.21.15.204:6825/1709433236","cluster_addr":"172.21.15.204:6827/1709433236","heartbeat_back_addr":"172.21.15.204:6831/1709433236","heartbeat_front_addr":"172.21.15.204:6829/1709433236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:17.259744+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:42.230389+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:07.172433+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:32.517060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:58.900615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:24.379803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:50.058928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:15.967299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:42.025873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:08.225448+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:33.910531+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:59.466423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:0/1542993850":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/3071755865":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/2452559014":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/4286398432":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6801/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6800/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6800/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/3960601922":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2270507277":"2024-09-12T22:43:31.970432+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:51:18.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.587+0000 7fc8354ed640 1 -- 172.21.15.152:0/1379140758 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc8180a2c10 msgr2=0x7fc8180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:18.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.587+0000 7fc8354ed640 1 --2- 172.21.15.152:0/1379140758 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc8180a2c10 0x7fc8180a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc830105100 tx=0x7fc82400a2d0 comp rx=0 tx=0).stop 2024-09-11T22:51:18.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.587+0000 7fc8354ed640 1 -- 172.21.15.152:0/1379140758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc830100a80 msgr2=0x7fc830198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:18.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.587+0000 7fc8354ed640 1 --2- 172.21.15.152:0/1379140758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc830100a80 0x7fc830198420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc82000c970 tx=0x7fc82000ce30 comp rx=0 tx=0).stop 2024-09-11T22:51:18.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.587+0000 7fc8354ed640 1 -- 172.21.15.152:0/1379140758 shutdown_connections 2024-09-11T22:51:18.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.587+0000 7fc8354ed640 1 --2- 172.21.15.152:0/1379140758 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fc8180a2c10 0x7fc8180a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:18.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.587+0000 7fc8354ed640 1 --2- 172.21.15.152:0/1379140758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc830106a80 0x7fc83019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:18.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.587+0000 7fc8354ed640 1 --2- 172.21.15.152:0/1379140758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc830101420 0x7fc830198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:18.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.587+0000 7fc8354ed640 1 --2- 172.21.15.152:0/1379140758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc830100a80 0x7fc830198420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:18.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.587+0000 7fc8354ed640 1 -- 172.21.15.152:0/1379140758 >> 172.21.15.152:0/1379140758 conn(0x7fc8300fc7a0 msgr2=0x7fc83006a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:18.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.588+0000 7fc8354ed640 1 -- 172.21.15.152:0/1379140758 shutdown_connections 2024-09-11T22:51:18.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:18.588+0000 7fc8354ed640 1 -- 172.21.15.152:0/1379140758 wait complete. 2024-09-11T22:51:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:18 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1379140758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:51:19.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:18 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1379140758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:51:19.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:18 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1379140758' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:51:19.416 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T22:51:19.416 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:51:19.665 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:20.097 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:19 smithi152 ceph-mon[31181]: pgmap v293: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 20 op/s 2024-09-11T22:51:20.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.177+0000 7fe9ed5bd640 1 -- 172.21.15.152:0/1725568324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9e8101420 msgr2=0x7fe9e810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:20.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.177+0000 7fe9ed5bd640 1 --2- 172.21.15.152:0/1725568324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9e8101420 0x7fe9e810d950 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009930 tx=0x7fe9e002a420 comp rx=0 tx=0).stop 2024-09-11T22:51:20.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.177+0000 7fe9ed5bd640 1 -- 172.21.15.152:0/1725568324 shutdown_connections 2024-09-11T22:51:20.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.177+0000 7fe9ed5bd640 1 --2- 172.21.15.152:0/1725568324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9e8101420 0x7fe9e810d950 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:20.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.177+0000 7fe9ed5bd640 1 --2- 172.21.15.152:0/1725568324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9e8100a80 0x7fe9e8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:20.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.177+0000 7fe9ed5bd640 1 --2- 172.21.15.152:0/1725568324 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e8106a80 0x7fe9e8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:20.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.177+0000 7fe9ed5bd640 1 -- 172.21.15.152:0/1725568324 >> 172.21.15.152:0/1725568324 conn(0x7fe9e80fc7a0 msgr2=0x7fe9e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:20.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.177+0000 7fe9ed5bd640 1 -- 172.21.15.152:0/1725568324 shutdown_connections 2024-09-11T22:51:20.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.177+0000 7fe9ed5bd640 1 -- 172.21.15.152:0/1725568324 wait complete. 2024-09-11T22:51:20.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.178+0000 7fe9ed5bd640 1 Processor -- start 2024-09-11T22:51:20.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.178+0000 7fe9ed5bd640 1 -- start start 2024-09-11T22:51:20.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.178+0000 7fe9ed5bd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9e8100a80 0x7fe9e81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:20.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.178+0000 7fe9ed5bd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e8101420 0x7fe9e81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:20.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.178+0000 7fe9ed5bd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9e8106a80 0x7fe9e819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:20.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.178+0000 7fe9ed5bd640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e8111fa0 con 0x7fe9e8100a80 2024-09-11T22:51:20.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.178+0000 7fe9ed5bd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e8111e20 con 0x7fe9e8106a80 2024-09-11T22:51:20.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.178+0000 7fe9ed5bd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e8112120 con 0x7fe9e8101420 2024-09-11T22:51:20.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.178+0000 7fe9e6ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9e8100a80 0x7fe9e81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:20.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.178+0000 7fe9e6ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9e8100a80 0x7fe9e81983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58564/0 (socket says 172.21.15.152:58564) 2024-09-11T22:51:20.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.178+0000 7fe9e6ffd640 1 -- 172.21.15.152:0/1963557779 learned_addr learned my addr 172.21.15.152:0/1963557779 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:20.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.179+0000 7fe9e77fe640 1 --2- 172.21.15.152:0/1963557779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9e8106a80 0x7fe9e819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:20.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.179+0000 7fe9e67fc640 1 --2- 172.21.15.152:0/1963557779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e8101420 0x7fe9e81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:20.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.179+0000 7fe9e6ffd640 1 -- 172.21.15.152:0/1963557779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e8101420 msgr2=0x7fe9e81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:20.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.179+0000 7fe9e6ffd640 1 --2- 172.21.15.152:0/1963557779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e8101420 0x7fe9e81988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:20.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.179+0000 7fe9e6ffd640 1 -- 172.21.15.152:0/1963557779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9e8106a80 msgr2=0x7fe9e819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:20.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.179+0000 7fe9e6ffd640 1 --2- 172.21.15.152:0/1963557779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9e8106a80 0x7fe9e819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:20.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.179+0000 7fe9e6ffd640 1 -- 172.21.15.152:0/1963557779 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e819d360 con 0x7fe9e8100a80 2024-09-11T22:51:20.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.179+0000 7fe9e77fe640 1 --2- 172.21.15.152:0/1963557779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9e8106a80 0x7fe9e819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:20.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.179+0000 7fe9e6ffd640 1 --2- 172.21.15.152:0/1963557779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9e8100a80 0x7fe9e81983b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800ea80 tx=0x7fe9d800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:20.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.179+0000 7fe9cffff640 1 -- 172.21.15.152:0/1963557779 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d800ce40 con 0x7fe9e8100a80 2024-09-11T22:51:20.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.179+0000 7fe9ed5bd640 1 -- 172.21.15.152:0/1963557779 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e819d650 con 0x7fe9e8100a80 2024-09-11T22:51:20.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.179+0000 7fe9cffff640 1 -- 172.21.15.152:0/1963557779 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9d8005020 con 0x7fe9e8100a80 2024-09-11T22:51:20.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.179+0000 7fe9cffff640 1 -- 172.21.15.152:0/1963557779 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8011760 con 0x7fe9e8100a80 2024-09-11T22:51:20.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.179+0000 7fe9ed5bd640 1 -- 172.21.15.152:0/1963557779 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e81a4f30 con 0x7fe9e8100a80 2024-09-11T22:51:20.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.180+0000 7fe9ed5bd640 1 -- 172.21.15.152:0/1963557779 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9e8100ee0 con 0x7fe9e8100a80 2024-09-11T22:51:20.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.181+0000 7fe9cffff640 1 -- 172.21.15.152:0/1963557779 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe9d8011900 con 0x7fe9e8100a80 2024-09-11T22:51:20.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.182+0000 7fe9cffff640 1 --2- 172.21.15.152:0/1963557779 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe9d00a2c10 0x7fe9d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:20.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.182+0000 7fe9e67fc640 1 --2- 172.21.15.152:0/1963557779 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe9d00a2c10 0x7fe9d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:20.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.182+0000 7fe9cffff640 1 -- 172.21.15.152:0/1963557779 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fe9d80cf6d0 con 0x7fe9e8100a80 2024-09-11T22:51:20.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.182+0000 7fe9e67fc640 1 --2- 172.21.15.152:0/1963557779 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe9d00a2c10 0x7fe9d00a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80fecb0 tx=0x7fe9dc005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:20.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.186+0000 7fe9cffff640 1 -- 172.21.15.152:0/1963557779 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9d8016030 con 0x7fe9e8100a80 2024-09-11T22:51:20.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:19 smithi191 ceph-mon[35211]: pgmap v293: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 20 op/s 2024-09-11T22:51:20.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:19 smithi204 ceph-mon[35454]: pgmap v293: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 20 op/s 2024-09-11T22:51:20.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.350+0000 7fe9ed5bd640 1 -- 172.21.15.152:0/1963557779 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe9e80fd580 con 0x7fe9d00a2c10 2024-09-11T22:51:20.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.357+0000 7fe9cffff640 1 -- 172.21.15.152:0/1963557779 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258914 (secure 0 0 0) 0x7fe9e80fd580 con 0x7fe9d00a2c10 2024-09-11T22:51:20.358 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:51:20.363 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-09-11T22:51:20.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.359+0000 7fe9ed5bd640 1 -- 172.21.15.152:0/1963557779 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe9d00a2c10 msgr2=0x7fe9d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:20.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.359+0000 7fe9ed5bd640 1 --2- 172.21.15.152:0/1963557779 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe9d00a2c10 0x7fe9d00a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80fecb0 tx=0x7fe9dc005cc0 comp rx=0 tx=0).stop 2024-09-11T22:51:20.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.359+0000 7fe9ed5bd640 1 -- 172.21.15.152:0/1963557779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9e8100a80 msgr2=0x7fe9e81983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:20.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.359+0000 7fe9ed5bd640 1 --2- 172.21.15.152:0/1963557779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9e8100a80 0x7fe9e81983b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe9d800ea80 tx=0x7fe9d800ef40 comp rx=0 tx=0).stop 2024-09-11T22:51:20.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.359+0000 7fe9ed5bd640 1 -- 172.21.15.152:0/1963557779 shutdown_connections 2024-09-11T22:51:20.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.359+0000 7fe9ed5bd640 1 --2- 172.21.15.152:0/1963557779 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe9d00a2c10 0x7fe9d00a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:20.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.359+0000 7fe9ed5bd640 1 --2- 172.21.15.152:0/1963557779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe9e8106a80 0x7fe9e819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:20.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.359+0000 7fe9ed5bd640 1 --2- 172.21.15.152:0/1963557779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e8101420 0x7fe9e81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:20.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.359+0000 7fe9ed5bd640 1 --2- 172.21.15.152:0/1963557779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9e8100a80 0x7fe9e81983b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:20.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.359+0000 7fe9ed5bd640 1 -- 172.21.15.152:0/1963557779 >> 172.21.15.152:0/1963557779 conn(0x7fe9e80fc7a0 msgr2=0x7fe9e80fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:20.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.360+0000 7fe9ed5bd640 1 -- 172.21.15.152:0/1963557779 shutdown_connections 2024-09-11T22:51:20.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:20.360+0000 7fe9ed5bd640 1 -- 172.21.15.152:0/1963557779 wait complete. 2024-09-11T22:51:21.218 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-11T22:51:18.768828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640013,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":288,"num_read_kb":250,"num_write":414,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":339,"ondisk_log_size":339,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368416,"kb_used_data":10848,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573408,"statfs":{"total":1151940427776,"available":1151563169792,"internally_reserved":0,"allocated":11108352,"data_stored":6951063,"data_compressed":107505,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":44333,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":145,"num_read_kb":125,"num_write":103,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003206"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151694+0000","last_change":"2024-09-11T22:50:54.171410+0000","last_active":"2024-09-11T22:50:55.151694+0000","last_peered":"2024-09-11T22:50:55.151694+0000","last_clean":"2024-09-11T22:50:55.151694+0000","last_became_active":"2024-09-11T22:50:54.171055+0000","last_became_peered":"2024-09-11T22:50:54.171055+0000","last_unstale":"2024-09-11T22:50:55.151694+0000","last_undegraded":"2024-09-11T22:50:55.151694+0000","last_fullsized":"2024-09-11T22:50:55.151694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:54.767454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.162876+0000","last_change":"2024-09-11T22:50:54.168534+0000","last_active":"2024-09-11T22:50:55.162876+0000","last_peered":"2024-09-11T22:50:55.162876+0000","last_clean":"2024-09-11T22:50:55.162876+0000","last_became_active":"2024-09-11T22:50:54.168228+0000","last_became_peered":"2024-09-11T22:50:54.168228+0000","last_unstale":"2024-09-11T22:50:55.162876+0000","last_undegraded":"2024-09-11T22:50:55.162876+0000","last_fullsized":"2024-09-11T22:50:55.162876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:11.380048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164082+0000","last_change":"2024-09-11T22:50:54.168975+0000","last_active":"2024-09-11T22:50:55.164082+0000","last_peered":"2024-09-11T22:50:55.164082+0000","last_clean":"2024-09-11T22:50:55.164082+0000","last_became_active":"2024-09-11T22:50:54.168676+0000","last_became_peered":"2024-09-11T22:50:54.168676+0000","last_unstale":"2024-09-11T22:50:55.164082+0000","last_undegraded":"2024-09-11T22:50:55.164082+0000","last_fullsized":"2024-09-11T22:50:55.164082+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:15.848273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.162347+0000","last_change":"2024-09-11T22:50:54.164164+0000","last_active":"2024-09-11T22:50:55.162347+0000","last_peered":"2024-09-11T22:50:55.162347+0000","last_clean":"2024-09-11T22:50:55.162347+0000","last_became_active":"2024-09-11T22:50:54.163805+0000","last_became_peered":"2024-09-11T22:50:54.163805+0000","last_unstale":"2024-09-11T22:50:55.162347+0000","last_undegraded":"2024-09-11T22:50:55.162347+0000","last_fullsized":"2024-09-11T22:50:55.162347+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:54.954046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.161794+0000","last_change":"2024-09-11T22:50:54.167289+0000","last_active":"2024-09-11T22:50:55.161794+0000","last_peered":"2024-09-11T22:50:55.161794+0000","last_clean":"2024-09-11T22:50:55.161794+0000","last_became_active":"2024-09-11T22:50:54.166958+0000","last_became_peered":"2024-09-11T22:50:54.166958+0000","last_unstale":"2024-09-11T22:50:55.161794+0000","last_undegraded":"2024-09-11T22:50:55.161794+0000","last_fullsized":"2024-09-11T22:50:55.161794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:25.261241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151127+0000","last_change":"2024-09-11T22:50:54.175605+0000","last_active":"2024-09-11T22:50:55.151127+0000","last_peered":"2024-09-11T22:50:55.151127+0000","last_clean":"2024-09-11T22:50:55.151127+0000","last_became_active":"2024-09-11T22:50:54.175207+0000","last_became_peered":"2024-09-11T22:50:54.175207+0000","last_unstale":"2024-09-11T22:50:55.151127+0000","last_undegraded":"2024-09-11T22:50:55.151127+0000","last_fullsized":"2024-09-11T22:50:55.151127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:47.008910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163674+0000","last_change":"2024-09-11T22:50:54.167930+0000","last_active":"2024-09-11T22:50:55.163674+0000","last_peered":"2024-09-11T22:50:55.163674+0000","last_clean":"2024-09-11T22:50:55.163674+0000","last_became_active":"2024-09-11T22:50:54.167082+0000","last_became_peered":"2024-09-11T22:50:54.167082+0000","last_unstale":"2024-09-11T22:50:55.163674+0000","last_undegraded":"2024-09-11T22:50:55.163674+0000","last_fullsized":"2024-09-11T22:50:55.163674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:50.931311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163835+0000","last_change":"2024-09-11T22:50:54.172528+0000","last_active":"2024-09-11T22:50:55.163835+0000","last_peered":"2024-09-11T22:50:55.163835+0000","last_clean":"2024-09-11T22:50:55.163835+0000","last_became_active":"2024-09-11T22:50:54.172257+0000","last_became_peered":"2024-09-11T22:50:54.172257+0000","last_unstale":"2024-09-11T22:50:55.163835+0000","last_undegraded":"2024-09-11T22:50:55.163835+0000","last_fullsized":"2024-09-11T22:50:55.163835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:13:43.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165459+0000","last_change":"2024-09-11T22:50:54.163357+0000","last_active":"2024-09-11T22:50:55.165459+0000","last_peered":"2024-09-11T22:50:55.165459+0000","last_clean":"2024-09-11T22:50:55.165459+0000","last_became_active":"2024-09-11T22:50:54.162943+0000","last_became_peered":"2024-09-11T22:50:54.162943+0000","last_unstale":"2024-09-11T22:50:55.165459+0000","last_undegraded":"2024-09-11T22:50:55.165459+0000","last_fullsized":"2024-09-11T22:50:55.165459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:57.055802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151603+0000","last_change":"2024-09-11T22:50:54.177773+0000","last_active":"2024-09-11T22:50:55.151603+0000","last_peered":"2024-09-11T22:50:55.151603+0000","last_clean":"2024-09-11T22:50:55.151603+0000","last_became_active":"2024-09-11T22:50:54.177388+0000","last_became_peered":"2024-09-11T22:50:54.177388+0000","last_unstale":"2024-09-11T22:50:55.151603+0000","last_undegraded":"2024-09-11T22:50:55.151603+0000","last_fullsized":"2024-09-11T22:50:55.151603+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:36:44.625089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163796+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:50:55.163796+0000","last_peered":"2024-09-11T22:50:55.163796+0000","last_clean":"2024-09-11T22:50:55.163796+0000","last_became_active":"2024-09-11T22:50:54.171491+0000","last_became_peered":"2024-09-11T22:50:54.171491+0000","last_unstale":"2024-09-11T22:50:55.163796+0000","last_undegraded":"2024-09-11T22:50:55.163796+0000","last_fullsized":"2024-09-11T22:50:55.163796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.996640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.162371+0000","last_change":"2024-09-11T22:50:54.171260+0000","last_active":"2024-09-11T22:50:55.162371+0000","last_peered":"2024-09-11T22:50:55.162371+0000","last_clean":"2024-09-11T22:50:55.162371+0000","last_became_active":"2024-09-11T22:50:54.170733+0000","last_became_peered":"2024-09-11T22:50:54.170733+0000","last_unstale":"2024-09-11T22:50:55.162371+0000","last_undegraded":"2024-09-11T22:50:55.162371+0000","last_fullsized":"2024-09-11T22:50:55.162371+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:57.877280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.161892+0000","last_change":"2024-09-11T22:50:54.170576+0000","last_active":"2024-09-11T22:50:55.161892+0000","last_peered":"2024-09-11T22:50:55.161892+0000","last_clean":"2024-09-11T22:50:55.161892+0000","last_became_active":"2024-09-11T22:50:54.170239+0000","last_became_peered":"2024-09-11T22:50:54.170239+0000","last_unstale":"2024-09-11T22:50:55.161892+0000","last_undegraded":"2024-09-11T22:50:55.161892+0000","last_fullsized":"2024-09-11T22:50:55.161892+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:09.683171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163616+0000","last_change":"2024-09-11T22:50:54.167176+0000","last_active":"2024-09-11T22:50:55.163616+0000","last_peered":"2024-09-11T22:50:55.163616+0000","last_clean":"2024-09-11T22:50:55.163616+0000","last_became_active":"2024-09-11T22:50:54.166894+0000","last_became_peered":"2024-09-11T22:50:54.166894+0000","last_unstale":"2024-09-11T22:50:55.163616+0000","last_undegraded":"2024-09-11T22:50:55.163616+0000","last_fullsized":"2024-09-11T22:50:55.163616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:43:08.158786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163050+0000","last_change":"2024-09-11T22:50:54.163100+0000","last_active":"2024-09-11T22:50:55.163050+0000","last_peered":"2024-09-11T22:50:55.163050+0000","last_clean":"2024-09-11T22:50:55.163050+0000","last_became_active":"2024-09-11T22:50:54.162776+0000","last_became_peered":"2024-09-11T22:50:54.162776+0000","last_unstale":"2024-09-11T22:50:55.163050+0000","last_undegraded":"2024-09-11T22:50:55.163050+0000","last_fullsized":"2024-09-11T22:50:55.163050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:15.215305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164805+0000","last_change":"2024-09-11T22:50:54.166013+0000","last_active":"2024-09-11T22:50:55.164805+0000","last_peered":"2024-09-11T22:50:55.164805+0000","last_clean":"2024-09-11T22:50:55.164805+0000","last_became_active":"2024-09-11T22:50:54.165680+0000","last_became_peered":"2024-09-11T22:50:54.165680+0000","last_unstale":"2024-09-11T22:50:55.164805+0000","last_undegraded":"2024-09-11T22:50:55.164805+0000","last_fullsized":"2024-09-11T22:50:55.164805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:12.192231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151814+0000","last_change":"2024-09-11T22:50:54.172105+0000","last_active":"2024-09-11T22:50:55.151814+0000","last_peered":"2024-09-11T22:50:55.151814+0000","last_clean":"2024-09-11T22:50:55.151814+0000","last_became_active":"2024-09-11T22:50:54.171782+0000","last_became_peered":"2024-09-11T22:50:54.171782+0000","last_unstale":"2024-09-11T22:50:55.151814+0000","last_undegraded":"2024-09-11T22:50:55.151814+0000","last_fullsized":"2024-09-11T22:50:55.151814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:17.152834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.161647+0000","last_change":"2024-09-11T22:50:54.167184+0000","last_active":"2024-09-11T22:50:55.161647+0000","last_peered":"2024-09-11T22:50:55.161647+0000","last_clean":"2024-09-11T22:50:55.161647+0000","last_became_active":"2024-09-11T22:50:54.166878+0000","last_became_peered":"2024-09-11T22:50:54.166878+0000","last_unstale":"2024-09-11T22:50:55.161647+0000","last_undegraded":"2024-09-11T22:50:55.161647+0000","last_fullsized":"2024-09-11T22:50:55.161647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:44.046552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.166652+0000","last_change":"2024-09-11T22:50:54.176260+0000","last_active":"2024-09-11T22:50:55.166652+0000","last_peered":"2024-09-11T22:50:55.166652+0000","last_clean":"2024-09-11T22:50:55.166652+0000","last_became_active":"2024-09-11T22:50:54.175873+0000","last_became_peered":"2024-09-11T22:50:54.175873+0000","last_unstale":"2024-09-11T22:50:55.166652+0000","last_undegraded":"2024-09-11T22:50:55.166652+0000","last_fullsized":"2024-09-11T22:50:55.166652+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:05.258677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151384+0000","last_change":"2024-09-11T22:50:54.172725+0000","last_active":"2024-09-11T22:50:55.151384+0000","last_peered":"2024-09-11T22:50:55.151384+0000","last_clean":"2024-09-11T22:50:55.151384+0000","last_became_active":"2024-09-11T22:50:54.172072+0000","last_became_peered":"2024-09-11T22:50:54.172072+0000","last_unstale":"2024-09-11T22:50:55.151384+0000","last_undegraded":"2024-09-11T22:50:55.151384+0000","last_fullsized":"2024-09-11T22:50:55.151384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:53.699670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.162028+0000","last_change":"2024-09-11T22:50:54.165118+0000","last_active":"2024-09-11T22:50:55.162028+0000","last_peered":"2024-09-11T22:50:55.162028+0000","last_clean":"2024-09-11T22:50:55.162028+0000","last_became_active":"2024-09-11T22:50:54.164793+0000","last_became_peered":"2024-09-11T22:50:54.164793+0000","last_unstale":"2024-09-11T22:50:55.162028+0000","last_undegraded":"2024-09-11T22:50:55.162028+0000","last_fullsized":"2024-09-11T22:50:55.162028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:42.883940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165155+0000","last_change":"2024-09-11T22:50:54.176900+0000","last_active":"2024-09-11T22:50:55.165155+0000","last_peered":"2024-09-11T22:50:55.165155+0000","last_clean":"2024-09-11T22:50:55.165155+0000","last_became_active":"2024-09-11T22:50:54.176595+0000","last_became_peered":"2024-09-11T22:50:54.176595+0000","last_unstale":"2024-09-11T22:50:55.165155+0000","last_undegraded":"2024-09-11T22:50:55.165155+0000","last_fullsized":"2024-09-11T22:50:55.165155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:36.925945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164083+0000","last_change":"2024-09-11T22:50:54.167656+0000","last_active":"2024-09-11T22:50:55.164083+0000","last_peered":"2024-09-11T22:50:55.164083+0000","last_clean":"2024-09-11T22:50:55.164083+0000","last_became_active":"2024-09-11T22:50:54.167349+0000","last_became_peered":"2024-09-11T22:50:54.167349+0000","last_unstale":"2024-09-11T22:50:55.164083+0000","last_undegraded":"2024-09-11T22:50:55.164083+0000","last_fullsized":"2024-09-11T22:50:55.164083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:38.100218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151867+0000","last_change":"2024-09-11T22:50:54.177059+0000","last_active":"2024-09-11T22:50:55.151867+0000","last_peered":"2024-09-11T22:50:55.151867+0000","last_clean":"2024-09-11T22:50:55.151867+0000","last_became_active":"2024-09-11T22:50:54.176726+0000","last_became_peered":"2024-09-11T22:50:54.176726+0000","last_unstale":"2024-09-11T22:50:55.151867+0000","last_undegraded":"2024-09-11T22:50:55.151867+0000","last_fullsized":"2024-09-11T22:50:55.151867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:27:58.058944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165322+0000","last_change":"2024-09-11T22:50:54.167566+0000","last_active":"2024-09-11T22:50:55.165322+0000","last_peered":"2024-09-11T22:50:55.165322+0000","last_clean":"2024-09-11T22:50:55.165322+0000","last_became_active":"2024-09-11T22:50:54.167049+0000","last_became_peered":"2024-09-11T22:50:54.167049+0000","last_unstale":"2024-09-11T22:50:55.165322+0000","last_undegraded":"2024-09-11T22:50:55.165322+0000","last_fullsized":"2024-09-11T22:50:55.165322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:06.833099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.167216+0000","last_change":"2024-09-11T22:50:54.169732+0000","last_active":"2024-09-11T22:50:55.167216+0000","last_peered":"2024-09-11T22:50:55.167216+0000","last_clean":"2024-09-11T22:50:55.167216+0000","last_became_active":"2024-09-11T22:50:54.169437+0000","last_became_peered":"2024-09-11T22:50:54.169437+0000","last_unstale":"2024-09-11T22:50:55.167216+0000","last_undegraded":"2024-09-11T22:50:55.167216+0000","last_fullsized":"2024-09-11T22:50:55.167216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:25.553387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151956+0000","last_change":"2024-09-11T22:50:54.166953+0000","last_active":"2024-09-11T22:50:55.151956+0000","last_peered":"2024-09-11T22:50:55.151956+0000","last_clean":"2024-09-11T22:50:55.151956+0000","last_became_active":"2024-09-11T22:50:54.166472+0000","last_became_peered":"2024-09-11T22:50:54.166472+0000","last_unstale":"2024-09-11T22:50:55.151956+0000","last_undegraded":"2024-09-11T22:50:55.151956+0000","last_fullsized":"2024-09-11T22:50:55.151956+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:01:20.234180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.161550+0000","last_change":"2024-09-11T22:50:54.171308+0000","last_active":"2024-09-11T22:50:55.161550+0000","last_peered":"2024-09-11T22:50:55.161550+0000","last_clean":"2024-09-11T22:50:55.161550+0000","last_became_active":"2024-09-11T22:50:54.168958+0000","last_became_peered":"2024-09-11T22:50:54.168958+0000","last_unstale":"2024-09-11T22:50:55.161550+0000","last_undegraded":"2024-09-11T22:50:55.161550+0000","last_fullsized":"2024-09-11T22:50:55.161550+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:20.224003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164891+0000","last_change":"2024-09-11T22:50:54.173846+0000","last_active":"2024-09-11T22:50:55.164891+0000","last_peered":"2024-09-11T22:50:55.164891+0000","last_clean":"2024-09-11T22:50:55.164891+0000","last_became_active":"2024-09-11T22:50:54.173542+0000","last_became_peered":"2024-09-11T22:50:54.173542+0000","last_unstale":"2024-09-11T22:50:55.164891+0000","last_undegraded":"2024-09-11T22:50:55.164891+0000","last_fullsized":"2024-09-11T22:50:55.164891+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:29:45.775902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165960+0000","last_change":"2024-09-11T22:50:54.176008+0000","last_active":"2024-09-11T22:50:55.165960+0000","last_peered":"2024-09-11T22:50:55.165960+0000","last_clean":"2024-09-11T22:50:55.165960+0000","last_became_active":"2024-09-11T22:50:54.175672+0000","last_became_peered":"2024-09-11T22:50:54.175672+0000","last_unstale":"2024-09-11T22:50:55.165960+0000","last_undegraded":"2024-09-11T22:50:55.165960+0000","last_fullsized":"2024-09-11T22:50:55.165960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:35.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.166402+0000","last_change":"2024-09-11T22:50:54.162504+0000","last_active":"2024-09-11T22:50:55.166402+0000","last_peered":"2024-09-11T22:50:55.166402+0000","last_clean":"2024-09-11T22:50:55.166402+0000","last_became_active":"2024-09-11T22:50:54.162190+0000","last_became_peered":"2024-09-11T22:50:54.162190+0000","last_unstale":"2024-09-11T22:50:55.166402+0000","last_undegraded":"2024-09-11T22:50:55.166402+0000","last_fullsized":"2024-09-11T22:50:55.166402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:52.138692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151544+0000","last_change":"2024-09-11T22:50:54.167097+0000","last_active":"2024-09-11T22:50:55.151544+0000","last_peered":"2024-09-11T22:50:55.151544+0000","last_clean":"2024-09-11T22:50:55.151544+0000","last_became_active":"2024-09-11T22:50:54.166608+0000","last_became_peered":"2024-09-11T22:50:54.166608+0000","last_unstale":"2024-09-11T22:50:55.151544+0000","last_undegraded":"2024-09-11T22:50:55.151544+0000","last_fullsized":"2024-09-11T22:50:55.151544+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:37.522803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.161603+0000","last_change":"2024-09-11T22:50:54.160696+0000","last_active":"2024-09-11T22:50:55.161603+0000","last_peered":"2024-09-11T22:50:55.161603+0000","last_clean":"2024-09-11T22:50:55.161603+0000","last_became_active":"2024-09-11T22:50:54.159249+0000","last_became_peered":"2024-09-11T22:50:54.159249+0000","last_unstale":"2024-09-11T22:50:55.161603+0000","last_undegraded":"2024-09-11T22:50:55.161603+0000","last_fullsized":"2024-09-11T22:50:55.161603+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:09.649208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:16.149689+0000","last_change":"2024-09-11T22:50:52.138986+0000","last_active":"2024-09-11T22:51:16.149689+0000","last_peered":"2024-09-11T22:51:16.149689+0000","last_clean":"2024-09-11T22:51:16.149689+0000","last_became_active":"2024-09-11T22:50:52.138595+0000","last_became_peered":"2024-09-11T22:50:52.138595+0000","last_unstale":"2024-09-11T22:51:16.149689+0000","last_undegraded":"2024-09-11T22:51:16.149689+0000","last_fullsized":"2024-09-11T22:51:16.149689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:06.232146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.150929+0000","last_change":"2024-09-11T22:50:54.176806+0000","last_active":"2024-09-11T22:50:55.150929+0000","last_peered":"2024-09-11T22:50:55.150929+0000","last_clean":"2024-09-11T22:50:55.150929+0000","last_became_active":"2024-09-11T22:50:54.176513+0000","last_became_peered":"2024-09-11T22:50:54.176513+0000","last_unstale":"2024-09-11T22:50:55.150929+0000","last_undegraded":"2024-09-11T22:50:55.150929+0000","last_fullsized":"2024-09-11T22:50:55.150929+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:43:12.086145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:04.231381+0000","last_change":"2024-09-11T22:50:52.142794+0000","last_active":"2024-09-11T22:51:04.231381+0000","last_peered":"2024-09-11T22:51:04.231381+0000","last_clean":"2024-09-11T22:51:04.231381+0000","last_became_active":"2024-09-11T22:50:52.142433+0000","last_became_peered":"2024-09-11T22:50:52.142433+0000","last_unstale":"2024-09-11T22:51:04.231381+0000","last_undegraded":"2024-09-11T22:51:04.231381+0000","last_fullsized":"2024-09-11T22:51:04.231381+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:47:12.090055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:03.221629+0000","last_change":"2024-09-11T22:50:52.139789+0000","last_active":"2024-09-11T22:51:03.221629+0000","last_peered":"2024-09-11T22:51:03.221629+0000","last_clean":"2024-09-11T22:51:03.221629+0000","last_became_active":"2024-09-11T22:50:52.139294+0000","last_became_peered":"2024-09-11T22:50:52.139294+0000","last_unstale":"2024-09-11T22:51:03.221629+0000","last_undegraded":"2024-09-11T22:51:03.221629+0000","last_fullsized":"2024-09-11T22:51:03.221629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:07.355155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163966+0000","last_change":"2024-09-11T22:50:54.166248+0000","last_active":"2024-09-11T22:50:55.163966+0000","last_peered":"2024-09-11T22:50:55.163966+0000","last_clean":"2024-09-11T22:50:55.163966+0000","last_became_active":"2024-09-11T22:50:54.165958+0000","last_became_peered":"2024-09-11T22:50:54.165958+0000","last_unstale":"2024-09-11T22:50:55.163966+0000","last_undegraded":"2024-09-11T22:50:55.163966+0000","last_fullsized":"2024-09-11T22:50:55.163966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:29.531741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.132013+0000","last_change":"2024-09-11T22:50:52.140340+0000","last_active":"2024-09-11T22:51:05.132013+0000","last_peered":"2024-09-11T22:51:05.132013+0000","last_clean":"2024-09-11T22:51:05.132013+0000","last_became_active":"2024-09-11T22:50:52.140011+0000","last_became_peered":"2024-09-11T22:50:52.140011+0000","last_unstale":"2024-09-11T22:51:05.132013+0000","last_undegraded":"2024-09-11T22:51:05.132013+0000","last_fullsized":"2024-09-11T22:51:05.132013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:58.862640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.160891+0000","last_change":"2024-09-11T22:50:54.166998+0000","last_active":"2024-09-11T22:50:55.160891+0000","last_peered":"2024-09-11T22:50:55.160891+0000","last_clean":"2024-09-11T22:50:55.160891+0000","last_became_active":"2024-09-11T22:50:54.166682+0000","last_became_peered":"2024-09-11T22:50:54.166682+0000","last_unstale":"2024-09-11T22:50:55.160891+0000","last_undegraded":"2024-09-11T22:50:55.160891+0000","last_fullsized":"2024-09-11T22:50:55.160891+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:57.815914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:15.147252+0000","last_change":"2024-09-11T22:50:52.141380+0000","last_active":"2024-09-11T22:51:15.147252+0000","last_peered":"2024-09-11T22:51:15.147252+0000","last_clean":"2024-09-11T22:51:15.147252+0000","last_became_active":"2024-09-11T22:50:52.139925+0000","last_became_peered":"2024-09-11T22:50:52.139925+0000","last_unstale":"2024-09-11T22:51:15.147252+0000","last_undegraded":"2024-09-11T22:51:15.147252+0000","last_fullsized":"2024-09-11T22:51:15.147252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:40:38.395055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.166509+0000","last_change":"2024-09-11T22:50:54.167909+0000","last_active":"2024-09-11T22:50:55.166509+0000","last_peered":"2024-09-11T22:50:55.166509+0000","last_clean":"2024-09-11T22:50:55.166509+0000","last_became_active":"2024-09-11T22:50:54.167571+0000","last_became_peered":"2024-09-11T22:50:54.167571+0000","last_unstale":"2024-09-11T22:50:55.166509+0000","last_undegraded":"2024-09-11T22:50:55.166509+0000","last_fullsized":"2024-09-11T22:50:55.166509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:34.179960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.136369+0000","last_change":"2024-09-11T22:50:52.144545+0000","last_active":"2024-09-11T22:51:05.136369+0000","last_peered":"2024-09-11T22:51:05.136369+0000","last_clean":"2024-09-11T22:51:05.136369+0000","last_became_active":"2024-09-11T22:50:52.144221+0000","last_became_peered":"2024-09-11T22:50:52.144221+0000","last_unstale":"2024-09-11T22:51:05.136369+0000","last_undegraded":"2024-09-11T22:51:05.136369+0000","last_fullsized":"2024-09-11T22:51:05.136369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:29.509165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164823+0000","last_change":"2024-09-11T22:50:54.164132+0000","last_active":"2024-09-11T22:50:55.164823+0000","last_peered":"2024-09-11T22:50:55.164823+0000","last_clean":"2024-09-11T22:50:55.164823+0000","last_became_active":"2024-09-11T22:50:54.163842+0000","last_became_peered":"2024-09-11T22:50:54.163842+0000","last_unstale":"2024-09-11T22:50:55.164823+0000","last_undegraded":"2024-09-11T22:50:55.164823+0000","last_fullsized":"2024-09-11T22:50:55.164823+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:28:55.490236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:04.229584+0000","last_change":"2024-09-11T22:50:52.264611+0000","last_active":"2024-09-11T22:51:04.229584+0000","last_peered":"2024-09-11T22:51:04.229584+0000","last_clean":"2024-09-11T22:51:04.229584+0000","last_became_active":"2024-09-11T22:50:52.264060+0000","last_became_peered":"2024-09-11T22:50:52.264060+0000","last_unstale":"2024-09-11T22:51:04.229584+0000","last_undegraded":"2024-09-11T22:51:04.229584+0000","last_fullsized":"2024-09-11T22:51:04.229584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:42.543997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165523+0000","last_change":"2024-09-11T22:50:54.165519+0000","last_active":"2024-09-11T22:50:55.165523+0000","last_peered":"2024-09-11T22:50:55.165523+0000","last_clean":"2024-09-11T22:50:55.165523+0000","last_became_active":"2024-09-11T22:50:54.165173+0000","last_became_peered":"2024-09-11T22:50:54.165173+0000","last_unstale":"2024-09-11T22:50:55.165523+0000","last_undegraded":"2024-09-11T22:50:55.165523+0000","last_fullsized":"2024-09-11T22:50:55.165523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:52.038578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'24","reported_seq":70,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:15.151208+0000","last_change":"2024-09-11T22:50:52.141782+0000","last_active":"2024-09-11T22:51:15.151208+0000","last_peered":"2024-09-11T22:51:15.151208+0000","last_clean":"2024-09-11T22:51:15.151208+0000","last_became_active":"2024-09-11T22:50:52.140662+0000","last_became_peered":"2024-09-11T22:50:52.140662+0000","last_unstale":"2024-09-11T22:51:15.151208+0000","last_undegraded":"2024-09-11T22:51:15.151208+0000","last_fullsized":"2024-09-11T22:51:15.151208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:21.158577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42203,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":28,"num_read_kb":54,"num_write":26,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.162371+0000","last_change":"2024-09-11T22:50:54.166678+0000","last_active":"2024-09-11T22:50:55.162371+0000","last_peered":"2024-09-11T22:50:55.162371+0000","last_clean":"2024-09-11T22:50:55.162371+0000","last_became_active":"2024-09-11T22:50:54.165777+0000","last_became_peered":"2024-09-11T22:50:54.165777+0000","last_unstale":"2024-09-11T22:50:55.162371+0000","last_undegraded":"2024-09-11T22:50:55.162371+0000","last_fullsized":"2024-09-11T22:50:55.162371+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:07.224563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.134896+0000","last_change":"2024-09-11T22:50:52.141292+0000","last_active":"2024-09-11T22:51:05.134896+0000","last_peered":"2024-09-11T22:51:05.134896+0000","last_clean":"2024-09-11T22:51:05.134896+0000","last_became_active":"2024-09-11T22:50:52.140971+0000","last_became_peered":"2024-09-11T22:50:52.140971+0000","last_unstale":"2024-09-11T22:51:05.134896+0000","last_undegraded":"2024-09-11T22:51:05.134896+0000","last_fullsized":"2024-09-11T22:51:05.134896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:53.039684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.161412+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:50:55.161412+0000","last_peered":"2024-09-11T22:50:55.161412+0000","last_clean":"2024-09-11T22:50:55.161412+0000","last_became_active":"2024-09-11T22:50:54.171520+0000","last_became_peered":"2024-09-11T22:50:54.171520+0000","last_unstale":"2024-09-11T22:50:55.161412+0000","last_undegraded":"2024-09-11T22:50:55.161412+0000","last_fullsized":"2024-09-11T22:50:55.161412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:56.594058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.138704+0000","last_change":"2024-09-11T22:50:52.138725+0000","last_active":"2024-09-11T22:51:05.138704+0000","last_peered":"2024-09-11T22:51:05.138704+0000","last_clean":"2024-09-11T22:51:05.138704+0000","last_became_active":"2024-09-11T22:50:52.138367+0000","last_became_peered":"2024-09-11T22:50:52.138367+0000","last_unstale":"2024-09-11T22:51:05.138704+0000","last_undegraded":"2024-09-11T22:51:05.138704+0000","last_fullsized":"2024-09-11T22:51:05.138704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:58.811444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163376+0000","last_change":"2024-09-11T22:50:54.170524+0000","last_active":"2024-09-11T22:50:55.163376+0000","last_peered":"2024-09-11T22:50:55.163376+0000","last_clean":"2024-09-11T22:50:55.163376+0000","last_became_active":"2024-09-11T22:50:54.169947+0000","last_became_peered":"2024-09-11T22:50:54.169947+0000","last_unstale":"2024-09-11T22:50:55.163376+0000","last_undegraded":"2024-09-11T22:50:55.163376+0000","last_fullsized":"2024-09-11T22:50:55.163376+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:51.768718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:15.151451+0000","last_change":"2024-09-11T22:50:52.139182+0000","last_active":"2024-09-11T22:51:15.151451+0000","last_peered":"2024-09-11T22:51:15.151451+0000","last_clean":"2024-09-11T22:51:15.151451+0000","last_became_active":"2024-09-11T22:50:52.138799+0000","last_became_peered":"2024-09-11T22:50:52.138799+0000","last_unstale":"2024-09-11T22:51:15.151451+0000","last_undegraded":"2024-09-11T22:51:15.151451+0000","last_fullsized":"2024-09-11T22:51:15.151451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:36.376162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163003+0000","last_change":"2024-09-11T22:50:54.168797+0000","last_active":"2024-09-11T22:50:55.163003+0000","last_peered":"2024-09-11T22:50:55.163003+0000","last_clean":"2024-09-11T22:50:55.163003+0000","last_became_active":"2024-09-11T22:50:54.168408+0000","last_became_peered":"2024-09-11T22:50:54.168408+0000","last_unstale":"2024-09-11T22:50:55.163003+0000","last_undegraded":"2024-09-11T22:50:55.163003+0000","last_fullsized":"2024-09-11T22:50:55.163003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:51.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.129163+0000","last_change":"2024-09-11T22:50:52.264693+0000","last_active":"2024-09-11T22:51:05.129163+0000","last_peered":"2024-09-11T22:51:05.129163+0000","last_clean":"2024-09-11T22:51:05.129163+0000","last_became_active":"2024-09-11T22:50:52.264058+0000","last_became_peered":"2024-09-11T22:50:52.264058+0000","last_unstale":"2024-09-11T22:51:05.129163+0000","last_undegraded":"2024-09-11T22:51:05.129163+0000","last_fullsized":"2024-09-11T22:51:05.129163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:46.985921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151346+0000","last_change":"2024-09-11T22:50:54.169215+0000","last_active":"2024-09-11T22:50:55.151346+0000","last_peered":"2024-09-11T22:50:55.151346+0000","last_clean":"2024-09-11T22:50:55.151346+0000","last_became_active":"2024-09-11T22:50:54.168890+0000","last_became_peered":"2024-09-11T22:50:54.168890+0000","last_unstale":"2024-09-11T22:50:55.151346+0000","last_undegraded":"2024-09-11T22:50:55.151346+0000","last_fullsized":"2024-09-11T22:50:55.151346+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:42.916489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.128581+0000","last_change":"2024-09-11T22:50:52.133049+0000","last_active":"2024-09-11T22:51:05.128581+0000","last_peered":"2024-09-11T22:51:05.128581+0000","last_clean":"2024-09-11T22:51:05.128581+0000","last_became_active":"2024-09-11T22:50:52.132647+0000","last_became_peered":"2024-09-11T22:50:52.132647+0000","last_unstale":"2024-09-11T22:51:05.128581+0000","last_undegraded":"2024-09-11T22:51:05.128581+0000","last_fullsized":"2024-09-11T22:51:05.128581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:13.213003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151651+0000","last_change":"2024-09-11T22:50:54.168004+0000","last_active":"2024-09-11T22:50:55.151651+0000","last_peered":"2024-09-11T22:50:55.151651+0000","last_clean":"2024-09-11T22:50:55.151651+0000","last_became_active":"2024-09-11T22:50:54.167503+0000","last_became_peered":"2024-09-11T22:50:54.167503+0000","last_unstale":"2024-09-11T22:50:55.151651+0000","last_undegraded":"2024-09-11T22:50:55.151651+0000","last_fullsized":"2024-09-11T22:50:55.151651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:12.282400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.133823+0000","last_change":"2024-09-11T22:50:52.139766+0000","last_active":"2024-09-11T22:51:05.133823+0000","last_peered":"2024-09-11T22:51:05.133823+0000","last_clean":"2024-09-11T22:51:05.133823+0000","last_became_active":"2024-09-11T22:50:52.139456+0000","last_became_peered":"2024-09-11T22:50:52.139456+0000","last_unstale":"2024-09-11T22:51:05.133823+0000","last_undegraded":"2024-09-11T22:51:05.133823+0000","last_fullsized":"2024-09-11T22:51:05.133823+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:08.358649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164115+0000","last_change":"2024-09-11T22:50:54.167717+0000","last_active":"2024-09-11T22:50:55.164115+0000","last_peered":"2024-09-11T22:50:55.164115+0000","last_clean":"2024-09-11T22:50:55.164115+0000","last_became_active":"2024-09-11T22:50:54.167331+0000","last_became_peered":"2024-09-11T22:50:54.167331+0000","last_unstale":"2024-09-11T22:50:55.164115+0000","last_undegraded":"2024-09-11T22:50:55.164115+0000","last_fullsized":"2024-09-11T22:50:55.164115+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:11.240351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.128730+0000","last_change":"2024-09-11T22:50:52.141682+0000","last_active":"2024-09-11T22:51:05.128730+0000","last_peered":"2024-09-11T22:51:05.128730+0000","last_clean":"2024-09-11T22:51:05.128730+0000","last_became_active":"2024-09-11T22:50:52.141251+0000","last_became_peered":"2024-09-11T22:50:52.141251+0000","last_unstale":"2024-09-11T22:51:05.128730+0000","last_undegraded":"2024-09-11T22:51:05.128730+0000","last_fullsized":"2024-09-11T22:51:05.128730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:03.068252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:57.812415+0000","last_change":"2024-09-11T22:50:54.174149+0000","last_active":"2024-09-11T22:50:57.812415+0000","last_peered":"2024-09-11T22:50:57.812415+0000","last_clean":"2024-09-11T22:50:57.812415+0000","last_became_active":"2024-09-11T22:50:54.173765+0000","last_became_peered":"2024-09-11T22:50:54.173765+0000","last_unstale":"2024-09-11T22:50:57.812415+0000","last_undegraded":"2024-09-11T22:50:57.812415+0000","last_fullsized":"2024-09-11T22:50:57.812415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.314334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:13.144981+0000","last_change":"2024-09-11T22:50:52.435019+0000","last_active":"2024-09-11T22:51:13.144981+0000","last_peered":"2024-09-11T22:51:13.144981+0000","last_clean":"2024-09-11T22:51:13.144981+0000","last_became_active":"2024-09-11T22:50:52.434642+0000","last_became_peered":"2024-09-11T22:50:52.434642+0000","last_unstale":"2024-09-11T22:51:13.144981+0000","last_undegraded":"2024-09-11T22:51:13.144981+0000","last_fullsized":"2024-09-11T22:51:13.144981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:18.441841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164006+0000","last_change":"2024-09-11T22:50:54.171291+0000","last_active":"2024-09-11T22:50:55.164006+0000","last_peered":"2024-09-11T22:50:55.164006+0000","last_clean":"2024-09-11T22:50:55.164006+0000","last_became_active":"2024-09-11T22:50:54.170961+0000","last_became_peered":"2024-09-11T22:50:54.170961+0000","last_unstale":"2024-09-11T22:50:55.164006+0000","last_undegraded":"2024-09-11T22:50:55.164006+0000","last_fullsized":"2024-09-11T22:50:55.164006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:07.211936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":341,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163284+0000","last_change":"2024-09-11T22:49:39.098881+0000","last_active":"2024-09-11T22:50:55.163284+0000","last_peered":"2024-09-11T22:50:55.163284+0000","last_clean":"2024-09-11T22:50:55.163284+0000","last_became_active":"2024-09-11T22:49:39.098458+0000","last_became_peered":"2024-09-11T22:49:39.098458+0000","last_unstale":"2024-09-11T22:50:55.163284+0000","last_undegraded":"2024-09-11T22:50:55.163284+0000","last_fullsized":"2024-09-11T22:50:55.163284+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_clean_scrub_stamp":"2024-09-11T22:46:11.418577+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:41.059384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.135942+0000","last_change":"2024-09-11T22:50:52.137691+0000","last_active":"2024-09-11T22:51:05.135942+0000","last_peered":"2024-09-11T22:51:05.135942+0000","last_clean":"2024-09-11T22:51:05.135942+0000","last_became_active":"2024-09-11T22:50:52.137282+0000","last_became_peered":"2024-09-11T22:50:52.137282+0000","last_unstale":"2024-09-11T22:51:05.135942+0000","last_undegraded":"2024-09-11T22:51:05.135942+0000","last_fullsized":"2024-09-11T22:51:05.135942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:00:27.651252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165455+0000","last_change":"2024-09-11T22:50:54.162417+0000","last_active":"2024-09-11T22:50:55.165455+0000","last_peered":"2024-09-11T22:50:55.165455+0000","last_clean":"2024-09-11T22:50:55.165455+0000","last_became_active":"2024-09-11T22:50:54.160457+0000","last_became_peered":"2024-09-11T22:50:54.160457+0000","last_unstale":"2024-09-11T22:50:55.165455+0000","last_undegraded":"2024-09-11T22:50:55.165455+0000","last_fullsized":"2024-09-11T22:50:55.165455+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:45.167775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:03.221448+0000","last_change":"2024-09-11T22:50:52.144324+0000","last_active":"2024-09-11T22:51:03.221448+0000","last_peered":"2024-09-11T22:51:03.221448+0000","last_clean":"2024-09-11T22:51:03.221448+0000","last_became_active":"2024-09-11T22:50:52.144011+0000","last_became_peered":"2024-09-11T22:50:52.144011+0000","last_unstale":"2024-09-11T22:51:03.221448+0000","last_undegraded":"2024-09-11T22:51:03.221448+0000","last_fullsized":"2024-09-11T22:51:03.221448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:29.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.162189+0000","last_change":"2024-09-11T22:50:54.173287+0000","last_active":"2024-09-11T22:50:55.162189+0000","last_peered":"2024-09-11T22:50:55.162189+0000","last_clean":"2024-09-11T22:50:55.162189+0000","last_became_active":"2024-09-11T22:50:54.172953+0000","last_became_peered":"2024-09-11T22:50:54.172953+0000","last_unstale":"2024-09-11T22:50:55.162189+0000","last_undegraded":"2024-09-11T22:50:55.162189+0000","last_fullsized":"2024-09-11T22:50:55.162189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:24.314604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:16.152042+0000","last_change":"2024-09-11T22:50:52.263777+0000","last_active":"2024-09-11T22:51:16.152042+0000","last_peered":"2024-09-11T22:51:16.152042+0000","last_clean":"2024-09-11T22:51:16.152042+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:51:16.152042+0000","last_undegraded":"2024-09-11T22:51:16.152042+0000","last_fullsized":"2024-09-11T22:51:16.152042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:37.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.150847+0000","last_change":"2024-09-11T22:50:54.164927+0000","last_active":"2024-09-11T22:50:55.150847+0000","last_peered":"2024-09-11T22:50:55.150847+0000","last_clean":"2024-09-11T22:50:55.150847+0000","last_became_active":"2024-09-11T22:50:54.164602+0000","last_became_peered":"2024-09-11T22:50:54.164602+0000","last_unstale":"2024-09-11T22:50:55.150847+0000","last_undegraded":"2024-09-11T22:50:55.150847+0000","last_fullsized":"2024-09-11T22:50:55.150847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:10.226232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:16.152325+0000","last_change":"2024-09-11T22:50:52.141199+0000","last_active":"2024-09-11T22:51:16.152325+0000","last_peered":"2024-09-11T22:51:16.152325+0000","last_clean":"2024-09-11T22:51:16.152325+0000","last_became_active":"2024-09-11T22:50:52.140830+0000","last_became_peered":"2024-09-11T22:50:52.140830+0000","last_unstale":"2024-09-11T22:51:16.152325+0000","last_undegraded":"2024-09-11T22:51:16.152325+0000","last_fullsized":"2024-09-11T22:51:16.152325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.767803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151265+0000","last_change":"2024-09-11T22:50:54.175054+0000","last_active":"2024-09-11T22:50:55.151265+0000","last_peered":"2024-09-11T22:50:55.151265+0000","last_clean":"2024-09-11T22:50:55.151265+0000","last_became_active":"2024-09-11T22:50:54.174715+0000","last_became_peered":"2024-09-11T22:50:54.174715+0000","last_unstale":"2024-09-11T22:50:55.151265+0000","last_undegraded":"2024-09-11T22:50:55.151265+0000","last_fullsized":"2024-09-11T22:50:55.151265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:06.694988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.820546+0000","last_change":"2024-09-11T22:50:52.264212+0000","last_active":"2024-09-11T22:50:55.820546+0000","last_peered":"2024-09-11T22:50:55.820546+0000","last_clean":"2024-09-11T22:50:55.820546+0000","last_became_active":"2024-09-11T22:50:52.263882+0000","last_became_peered":"2024-09-11T22:50:52.263882+0000","last_unstale":"2024-09-11T22:50:55.820546+0000","last_undegraded":"2024-09-11T22:50:55.820546+0000","last_fullsized":"2024-09-11T22:50:55.820546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:29.635043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.826728+0000","last_change":"2024-09-11T22:50:54.167361+0000","last_active":"2024-09-11T22:50:55.826728+0000","last_peered":"2024-09-11T22:50:55.826728+0000","last_clean":"2024-09-11T22:50:55.826728+0000","last_became_active":"2024-09-11T22:50:54.166963+0000","last_became_peered":"2024-09-11T22:50:54.166963+0000","last_unstale":"2024-09-11T22:50:55.826728+0000","last_undegraded":"2024-09-11T22:50:55.826728+0000","last_fullsized":"2024-09-11T22:50:55.826728+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:13.294544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.129353+0000","last_change":"2024-09-11T22:50:52.264484+0000","last_active":"2024-09-11T22:51:05.129353+0000","last_peered":"2024-09-11T22:51:05.129353+0000","last_clean":"2024-09-11T22:51:05.129353+0000","last_became_active":"2024-09-11T22:50:52.264038+0000","last_became_peered":"2024-09-11T22:50:52.264038+0000","last_unstale":"2024-09-11T22:51:05.129353+0000","last_undegraded":"2024-09-11T22:51:05.129353+0000","last_fullsized":"2024-09-11T22:51:05.129353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:21.325136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.161702+0000","last_change":"2024-09-11T22:50:54.162225+0000","last_active":"2024-09-11T22:50:55.161702+0000","last_peered":"2024-09-11T22:50:55.161702+0000","last_clean":"2024-09-11T22:50:55.161702+0000","last_became_active":"2024-09-11T22:50:54.161234+0000","last_became_peered":"2024-09-11T22:50:54.161234+0000","last_unstale":"2024-09-11T22:50:55.161702+0000","last_undegraded":"2024-09-11T22:50:55.161702+0000","last_fullsized":"2024-09-11T22:50:55.161702+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:00.346217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'12","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:14.161966+0000","last_change":"2024-09-11T22:50:52.142366+0000","last_active":"2024-09-11T22:51:14.161966+0000","last_peered":"2024-09-11T22:51:14.161966+0000","last_clean":"2024-09-11T22:51:14.161966+0000","last_became_active":"2024-09-11T22:50:52.141931+0000","last_became_peered":"2024-09-11T22:50:52.141931+0000","last_unstale":"2024-09-11T22:51:14.161966+0000","last_undegraded":"2024-09-11T22:51:14.161966+0000","last_fullsized":"2024-09-11T22:51:14.161966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:33.864079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165440+0000","last_change":"2024-09-11T22:50:54.167307+0000","last_active":"2024-09-11T22:50:55.165440+0000","last_peered":"2024-09-11T22:50:55.165440+0000","last_clean":"2024-09-11T22:50:55.165440+0000","last_became_active":"2024-09-11T22:50:54.166650+0000","last_became_peered":"2024-09-11T22:50:54.166650+0000","last_unstale":"2024-09-11T22:50:55.165440+0000","last_undegraded":"2024-09-11T22:50:55.165440+0000","last_fullsized":"2024-09-11T22:50:55.165440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:04.863258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.136774+0000","last_change":"2024-09-11T22:50:52.435107+0000","last_active":"2024-09-11T22:51:05.136774+0000","last_peered":"2024-09-11T22:51:05.136774+0000","last_clean":"2024-09-11T22:51:05.136774+0000","last_became_active":"2024-09-11T22:50:52.434791+0000","last_became_peered":"2024-09-11T22:50:52.434791+0000","last_unstale":"2024-09-11T22:51:05.136774+0000","last_undegraded":"2024-09-11T22:51:05.136774+0000","last_fullsized":"2024-09-11T22:51:05.136774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:11.128791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163017+0000","last_change":"2024-09-11T22:50:54.178041+0000","last_active":"2024-09-11T22:50:55.163017+0000","last_peered":"2024-09-11T22:50:55.163017+0000","last_clean":"2024-09-11T22:50:55.163017+0000","last_became_active":"2024-09-11T22:50:54.177393+0000","last_became_peered":"2024-09-11T22:50:54.177393+0000","last_unstale":"2024-09-11T22:50:55.163017+0000","last_undegraded":"2024-09-11T22:50:55.163017+0000","last_fullsized":"2024-09-11T22:50:55.163017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:21.608104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:02.210217+0000","last_change":"2024-09-11T22:50:52.264051+0000","last_active":"2024-09-11T22:51:02.210217+0000","last_peered":"2024-09-11T22:51:02.210217+0000","last_clean":"2024-09-11T22:51:02.210217+0000","last_became_active":"2024-09-11T22:50:52.263674+0000","last_became_peered":"2024-09-11T22:50:52.263674+0000","last_unstale":"2024-09-11T22:51:02.210217+0000","last_undegraded":"2024-09-11T22:51:02.210217+0000","last_fullsized":"2024-09-11T22:51:02.210217+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:44.655218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.162611+0000","last_change":"2024-09-11T22:50:54.166409+0000","last_active":"2024-09-11T22:50:55.162611+0000","last_peered":"2024-09-11T22:50:55.162611+0000","last_clean":"2024-09-11T22:50:55.162611+0000","last_became_active":"2024-09-11T22:50:54.166037+0000","last_became_peered":"2024-09-11T22:50:54.166037+0000","last_unstale":"2024-09-11T22:50:55.162611+0000","last_undegraded":"2024-09-11T22:50:55.162611+0000","last_fullsized":"2024-09-11T22:50:55.162611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:22.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.135220+0000","last_change":"2024-09-11T22:50:52.140218+0000","last_active":"2024-09-11T22:51:05.135220+0000","last_peered":"2024-09-11T22:51:05.135220+0000","last_clean":"2024-09-11T22:51:05.135220+0000","last_became_active":"2024-09-11T22:50:52.139744+0000","last_became_peered":"2024-09-11T22:50:52.139744+0000","last_unstale":"2024-09-11T22:51:05.135220+0000","last_undegraded":"2024-09-11T22:51:05.135220+0000","last_fullsized":"2024-09-11T22:51:05.135220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:50.178348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151900+0000","last_change":"2024-09-11T22:50:54.164059+0000","last_active":"2024-09-11T22:50:55.151900+0000","last_peered":"2024-09-11T22:50:55.151900+0000","last_clean":"2024-09-11T22:50:55.151900+0000","last_became_active":"2024-09-11T22:50:54.163724+0000","last_became_peered":"2024-09-11T22:50:54.163724+0000","last_unstale":"2024-09-11T22:50:55.151900+0000","last_undegraded":"2024-09-11T22:50:55.151900+0000","last_fullsized":"2024-09-11T22:50:55.151900+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:57.241660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:04.232193+0000","last_change":"2024-09-11T22:50:52.140169+0000","last_active":"2024-09-11T22:51:04.232193+0000","last_peered":"2024-09-11T22:51:04.232193+0000","last_clean":"2024-09-11T22:51:04.232193+0000","last_became_active":"2024-09-11T22:50:52.139579+0000","last_became_peered":"2024-09-11T22:50:52.139579+0000","last_unstale":"2024-09-11T22:51:04.232193+0000","last_undegraded":"2024-09-11T22:51:04.232193+0000","last_fullsized":"2024-09-11T22:51:04.232193+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:43.523961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163639+0000","last_change":"2024-09-11T22:50:54.171539+0000","last_active":"2024-09-11T22:50:55.163639+0000","last_peered":"2024-09-11T22:50:55.163639+0000","last_clean":"2024-09-11T22:50:55.163639+0000","last_became_active":"2024-09-11T22:50:54.171013+0000","last_became_peered":"2024-09-11T22:50:54.171013+0000","last_unstale":"2024-09-11T22:50:55.163639+0000","last_undegraded":"2024-09-11T22:50:55.163639+0000","last_fullsized":"2024-09-11T22:50:55.163639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:03.564389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:02.210120+0000","last_change":"2024-09-11T22:50:52.139544+0000","last_active":"2024-09-11T22:51:02.210120+0000","last_peered":"2024-09-11T22:51:02.210120+0000","last_clean":"2024-09-11T22:51:02.210120+0000","last_became_active":"2024-09-11T22:50:52.139201+0000","last_became_peered":"2024-09-11T22:50:52.139201+0000","last_unstale":"2024-09-11T22:51:02.210120+0000","last_undegraded":"2024-09-11T22:51:02.210120+0000","last_fullsized":"2024-09-11T22:51:02.210120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:37.343437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164279+0000","last_change":"2024-09-11T22:50:54.170052+0000","last_active":"2024-09-11T22:50:55.164279+0000","last_peered":"2024-09-11T22:50:55.164279+0000","last_clean":"2024-09-11T22:50:55.164279+0000","last_became_active":"2024-09-11T22:50:54.169735+0000","last_became_peered":"2024-09-11T22:50:54.169735+0000","last_unstale":"2024-09-11T22:50:55.164279+0000","last_undegraded":"2024-09-11T22:50:55.164279+0000","last_fullsized":"2024-09-11T22:50:55.164279+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:44.007747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151969+0000","last_change":"2024-09-11T22:50:54.169959+0000","last_active":"2024-09-11T22:50:55.151969+0000","last_peered":"2024-09-11T22:50:55.151969+0000","last_clean":"2024-09-11T22:50:55.151969+0000","last_became_active":"2024-09-11T22:50:54.169649+0000","last_became_peered":"2024-09-11T22:50:54.169649+0000","last_unstale":"2024-09-11T22:50:55.151969+0000","last_undegraded":"2024-09-11T22:50:55.151969+0000","last_fullsized":"2024-09-11T22:50:55.151969+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:51.974258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.137968+0000","last_change":"2024-09-11T22:50:52.140881+0000","last_active":"2024-09-11T22:51:05.137968+0000","last_peered":"2024-09-11T22:51:05.137968+0000","last_clean":"2024-09-11T22:51:05.137968+0000","last_became_active":"2024-09-11T22:50:52.140527+0000","last_became_peered":"2024-09-11T22:50:52.140527+0000","last_unstale":"2024-09-11T22:51:05.137968+0000","last_undegraded":"2024-09-11T22:51:05.137968+0000","last_fullsized":"2024-09-11T22:51:05.137968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:14.325816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165206+0000","last_change":"2024-09-11T22:50:54.166571+0000","last_active":"2024-09-11T22:50:55.165206+0000","last_peered":"2024-09-11T22:50:55.165206+0000","last_clean":"2024-09-11T22:50:55.165206+0000","last_became_active":"2024-09-11T22:50:54.166247+0000","last_became_peered":"2024-09-11T22:50:54.166247+0000","last_unstale":"2024-09-11T22:50:55.165206+0000","last_undegraded":"2024-09-11T22:50:55.165206+0000","last_fullsized":"2024-09-11T22:50:55.165206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:42.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.138249+0000","last_change":"2024-09-11T22:50:52.140298+0000","last_active":"2024-09-11T22:51:05.138249+0000","last_peered":"2024-09-11T22:51:05.138249+0000","last_clean":"2024-09-11T22:51:05.138249+0000","last_became_active":"2024-09-11T22:50:52.139917+0000","last_became_peered":"2024-09-11T22:50:52.139917+0000","last_unstale":"2024-09-11T22:51:05.138249+0000","last_undegraded":"2024-09-11T22:51:05.138249+0000","last_fullsized":"2024-09-11T22:51:05.138249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:23.599010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164777+0000","last_change":"2024-09-11T22:50:54.170849+0000","last_active":"2024-09-11T22:50:55.164777+0000","last_peered":"2024-09-11T22:50:55.164777+0000","last_clean":"2024-09-11T22:50:55.164777+0000","last_became_active":"2024-09-11T22:50:54.169547+0000","last_became_peered":"2024-09-11T22:50:54.169547+0000","last_unstale":"2024-09-11T22:50:55.164777+0000","last_undegraded":"2024-09-11T22:50:55.164777+0000","last_fullsized":"2024-09-11T22:50:55.164777+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:25.251158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.167153+0000","last_change":"2024-09-11T22:50:52.263778+0000","last_active":"2024-09-11T22:50:55.167153+0000","last_peered":"2024-09-11T22:50:55.167153+0000","last_clean":"2024-09-11T22:50:55.167153+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:50:55.167153+0000","last_undegraded":"2024-09-11T22:50:55.167153+0000","last_fullsized":"2024-09-11T22:50:55.167153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:03.439147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165281+0000","last_change":"2024-09-11T22:50:54.171200+0000","last_active":"2024-09-11T22:50:55.165281+0000","last_peered":"2024-09-11T22:50:55.165281+0000","last_clean":"2024-09-11T22:50:55.165281+0000","last_became_active":"2024-09-11T22:50:54.169320+0000","last_became_peered":"2024-09-11T22:50:54.169320+0000","last_unstale":"2024-09-11T22:50:55.165281+0000","last_undegraded":"2024-09-11T22:50:55.165281+0000","last_fullsized":"2024-09-11T22:50:55.165281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:14.864699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:14.149595+0000","last_change":"2024-09-11T22:50:52.139609+0000","last_active":"2024-09-11T22:51:14.149595+0000","last_peered":"2024-09-11T22:51:14.149595+0000","last_clean":"2024-09-11T22:51:14.149595+0000","last_became_active":"2024-09-11T22:50:52.139278+0000","last_became_peered":"2024-09-11T22:50:52.139278+0000","last_unstale":"2024-09-11T22:51:14.149595+0000","last_undegraded":"2024-09-11T22:51:14.149595+0000","last_fullsized":"2024-09-11T22:51:14.149595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:09.538734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":49411,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":152,"num_read_kb":131,"num_write":231,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":148233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281769,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":420837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1053676,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939676,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":420374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168608,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":420752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":1013272,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561068,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":420091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790001,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":1012950,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953526,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":419952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":419916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":954368,"data_stored":464213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":463207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":421823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":43955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":44961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":43939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:51:21.222 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:51:21.472 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:22.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.982+0000 7f92bddce640 1 -- 172.21.15.152:0/72129396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b810c2f0 msgr2=0x7f92b810e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.982+0000 7f92bddce640 1 --2- 172.21.15.152:0/72129396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b810c2f0 0x7f92b810e790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009ca0 tx=0x7f92b002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:51:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.982+0000 7f92bddce640 1 -- 172.21.15.152:0/72129396 shutdown_connections 2024-09-11T22:51:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.982+0000 7f92bddce640 1 --2- 172.21.15.152:0/72129396 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b810c2f0 0x7f92b810e790 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.982+0000 7f92bddce640 1 --2- 172.21.15.152:0/72129396 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92b81037a0 0x7f92b810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.982+0000 7f92bddce640 1 --2- 172.21.15.152:0/72129396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92b8102df0 0x7f92b81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.982+0000 7f92bddce640 1 -- 172.21.15.152:0/72129396 >> 172.21.15.152:0/72129396 conn(0x7f92b80fc7d0 msgr2=0x7f92b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.983+0000 7f92bddce640 1 -- 172.21.15.152:0/72129396 shutdown_connections 2024-09-11T22:51:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.983+0000 7f92bddce640 1 -- 172.21.15.152:0/72129396 wait complete. 2024-09-11T22:51:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.983+0000 7f92bddce640 1 Processor -- start 2024-09-11T22:51:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.983+0000 7f92bddce640 1 -- start start 2024-09-11T22:51:22.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.983+0000 7f92bddce640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92b8102df0 0x7f92b8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:22.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.983+0000 7f92bddce640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b81037a0 0x7f92b8198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:22.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92bddce640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92b810c2f0 0x7f92b819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:22.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92bddce640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f92b8112150 con 0x7f92b81037a0 2024-09-11T22:51:22.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92bddce640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f92b8111fd0 con 0x7f92b8102df0 2024-09-11T22:51:22.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92bddce640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f92b81122d0 con 0x7f92b810c2f0 2024-09-11T22:51:22.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92aeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b81037a0 0x7f92b8198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:22.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92aeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b81037a0 0x7f92b8198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58584/0 (socket says 172.21.15.152:58584) 2024-09-11T22:51:22.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92aeffd640 1 -- 172.21.15.152:0/2349300009 learned_addr learned my addr 172.21.15.152:0/2349300009 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:22.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92b7fff640 1 --2- 172.21.15.152:0/2349300009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92b810c2f0 0x7f92b819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:22.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92b77fe640 1 --2- 172.21.15.152:0/2349300009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92b8102df0 0x7f92b8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:22.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92aeffd640 1 -- 172.21.15.152:0/2349300009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92b810c2f0 msgr2=0x7f92b819cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:22.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92aeffd640 1 --2- 172.21.15.152:0/2349300009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92b810c2f0 0x7f92b819cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:22.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92aeffd640 1 -- 172.21.15.152:0/2349300009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92b8102df0 msgr2=0x7f92b8198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:22.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92aeffd640 1 --2- 172.21.15.152:0/2349300009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92b8102df0 0x7f92b8198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:22.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92aeffd640 1 -- 172.21.15.152:0/2349300009 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b819d5c0 con 0x7f92b81037a0 2024-09-11T22:51:22.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92b77fe640 1 --2- 172.21.15.152:0/2349300009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92b8102df0 0x7f92b8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:22.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92aeffd640 1 --2- 172.21.15.152:0/2349300009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b81037a0 0x7f92b8198b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f92a0006fd0 tx=0x7f92a000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:22.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92b57fa640 1 -- 172.21.15.152:0/2349300009 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0011830 con 0x7f92b81037a0 2024-09-11T22:51:22.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92bddce640 1 -- 172.21.15.152:0/2349300009 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b819d8b0 con 0x7f92b81037a0 2024-09-11T22:51:22.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92b57fa640 1 -- 172.21.15.152:0/2349300009 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92a0016070 con 0x7f92b81037a0 2024-09-11T22:51:22.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92b57fa640 1 -- 172.21.15.152:0/2349300009 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a000f2e0 con 0x7f92b81037a0 2024-09-11T22:51:22.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.984+0000 7f92bddce640 1 -- 172.21.15.152:0/2349300009 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b81a5190 con 0x7f92b81037a0 2024-09-11T22:51:22.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.985+0000 7f92bddce640 1 -- 172.21.15.152:0/2349300009 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b81031d0 con 0x7f92b81037a0 2024-09-11T22:51:22.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.986+0000 7f92b57fa640 1 -- 172.21.15.152:0/2349300009 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f92a001b460 con 0x7f92b81037a0 2024-09-11T22:51:22.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.987+0000 7f92b57fa640 1 --2- 172.21.15.152:0/2349300009 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f92a40a2c10 0x7f92a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:22.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.987+0000 7f92b77fe640 1 --2- 172.21.15.152:0/2349300009 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f92a40a2c10 0x7f92a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:22.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.987+0000 7f92b57fa640 1 -- 172.21.15.152:0/2349300009 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f92a00ce650 con 0x7f92b81037a0 2024-09-11T22:51:22.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.988+0000 7f92b77fe640 1 --2- 172.21.15.152:0/2349300009 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f92a40a2c10 0x7f92a40a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f92a80058c0 tx=0x7f92a8005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:22.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:21.991+0000 7f92b57fa640 1 -- 172.21.15.152:0/2349300009 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92a009b710 con 0x7f92b81037a0 2024-09-11T22:51:22.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:21 smithi152 ceph-mon[31181]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:51:22.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:21 smithi152 ceph-mon[31181]: pgmap v294: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.5 KiB/s wr, 14 op/s 2024-09-11T22:51:22.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:22.159+0000 7f92bddce640 1 -- 172.21.15.152:0/2349300009 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f92b8062630 con 0x7f92a40a2c10 2024-09-11T22:51:22.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:22.165+0000 7f92b57fa640 1 -- 172.21.15.152:0/2349300009 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258912 (secure 0 0 0) 0x7f92b8062630 con 0x7f92a40a2c10 2024-09-11T22:51:22.171 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:51:22.172 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-09-11T22:51:22.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:22.168+0000 7f92bddce640 1 -- 172.21.15.152:0/2349300009 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f92a40a2c10 msgr2=0x7f92a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:22.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:22.168+0000 7f92bddce640 1 --2- 172.21.15.152:0/2349300009 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f92a40a2c10 0x7f92a40a50d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f92a80058c0 tx=0x7f92a8005d30 comp rx=0 tx=0).stop 2024-09-11T22:51:22.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:22.168+0000 7f92bddce640 1 -- 172.21.15.152:0/2349300009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b81037a0 msgr2=0x7f92b8198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:22.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:22.168+0000 7f92bddce640 1 --2- 172.21.15.152:0/2349300009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b81037a0 0x7f92b8198b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f92a0006fd0 tx=0x7f92a000b8e0 comp rx=0 tx=0).stop 2024-09-11T22:51:22.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:22.168+0000 7f92bddce640 1 -- 172.21.15.152:0/2349300009 shutdown_connections 2024-09-11T22:51:22.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:22.168+0000 7f92bddce640 1 --2- 172.21.15.152:0/2349300009 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f92a40a2c10 0x7f92a40a50d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:22.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:22.168+0000 7f92bddce640 1 --2- 172.21.15.152:0/2349300009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92b810c2f0 0x7f92b819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:22.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:22.168+0000 7f92bddce640 1 --2- 172.21.15.152:0/2349300009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92b81037a0 0x7f92b8198b50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:22.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:22.168+0000 7f92bddce640 1 --2- 172.21.15.152:0/2349300009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92b8102df0 0x7f92b8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:22.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:22.168+0000 7f92bddce640 1 -- 172.21.15.152:0/2349300009 >> 172.21.15.152:0/2349300009 conn(0x7f92b80fc7d0 msgr2=0x7f92b80fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:22.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:22.168+0000 7f92bddce640 1 -- 172.21.15.152:0/2349300009 shutdown_connections 2024-09-11T22:51:22.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:22.168+0000 7f92bddce640 1 -- 172.21.15.152:0/2349300009 wait complete. 2024-09-11T22:51:22.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:21 smithi191 ceph-mon[35211]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:51:22.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:21 smithi191 ceph-mon[35211]: pgmap v294: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.5 KiB/s wr, 14 op/s 2024-09-11T22:51:22.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:21 smithi204 ceph-mon[35454]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:51:22.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:21 smithi204 ceph-mon[35454]: pgmap v294: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.5 KiB/s wr, 14 op/s 2024-09-11T22:51:23.076 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-11T22:51:20.769419+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640013,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":293,"num_read_kb":250,"num_write":414,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":339,"ondisk_log_size":339,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368440,"kb_used_data":10872,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573384,"statfs":{"total":1151940427776,"available":1151563145216,"internally_reserved":0,"allocated":11132928,"data_stored":6975547,"data_compressed":107505,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42283,"num_objects":15,"num_object_clones":0,"num_object_copies":45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":123,"num_read_kb":105,"num_write":55,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003306"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151694+0000","last_change":"2024-09-11T22:50:54.171410+0000","last_active":"2024-09-11T22:50:55.151694+0000","last_peered":"2024-09-11T22:50:55.151694+0000","last_clean":"2024-09-11T22:50:55.151694+0000","last_became_active":"2024-09-11T22:50:54.171055+0000","last_became_peered":"2024-09-11T22:50:54.171055+0000","last_unstale":"2024-09-11T22:50:55.151694+0000","last_undegraded":"2024-09-11T22:50:55.151694+0000","last_fullsized":"2024-09-11T22:50:55.151694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:54.767454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.162876+0000","last_change":"2024-09-11T22:50:54.168534+0000","last_active":"2024-09-11T22:50:55.162876+0000","last_peered":"2024-09-11T22:50:55.162876+0000","last_clean":"2024-09-11T22:50:55.162876+0000","last_became_active":"2024-09-11T22:50:54.168228+0000","last_became_peered":"2024-09-11T22:50:54.168228+0000","last_unstale":"2024-09-11T22:50:55.162876+0000","last_undegraded":"2024-09-11T22:50:55.162876+0000","last_fullsized":"2024-09-11T22:50:55.162876+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:11.380048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164082+0000","last_change":"2024-09-11T22:50:54.168975+0000","last_active":"2024-09-11T22:50:55.164082+0000","last_peered":"2024-09-11T22:50:55.164082+0000","last_clean":"2024-09-11T22:50:55.164082+0000","last_became_active":"2024-09-11T22:50:54.168676+0000","last_became_peered":"2024-09-11T22:50:54.168676+0000","last_unstale":"2024-09-11T22:50:55.164082+0000","last_undegraded":"2024-09-11T22:50:55.164082+0000","last_fullsized":"2024-09-11T22:50:55.164082+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:15.848273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.162347+0000","last_change":"2024-09-11T22:50:54.164164+0000","last_active":"2024-09-11T22:50:55.162347+0000","last_peered":"2024-09-11T22:50:55.162347+0000","last_clean":"2024-09-11T22:50:55.162347+0000","last_became_active":"2024-09-11T22:50:54.163805+0000","last_became_peered":"2024-09-11T22:50:54.163805+0000","last_unstale":"2024-09-11T22:50:55.162347+0000","last_undegraded":"2024-09-11T22:50:55.162347+0000","last_fullsized":"2024-09-11T22:50:55.162347+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:54.954046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.161794+0000","last_change":"2024-09-11T22:50:54.167289+0000","last_active":"2024-09-11T22:50:55.161794+0000","last_peered":"2024-09-11T22:50:55.161794+0000","last_clean":"2024-09-11T22:50:55.161794+0000","last_became_active":"2024-09-11T22:50:54.166958+0000","last_became_peered":"2024-09-11T22:50:54.166958+0000","last_unstale":"2024-09-11T22:50:55.161794+0000","last_undegraded":"2024-09-11T22:50:55.161794+0000","last_fullsized":"2024-09-11T22:50:55.161794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:25.261241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151127+0000","last_change":"2024-09-11T22:50:54.175605+0000","last_active":"2024-09-11T22:50:55.151127+0000","last_peered":"2024-09-11T22:50:55.151127+0000","last_clean":"2024-09-11T22:50:55.151127+0000","last_became_active":"2024-09-11T22:50:54.175207+0000","last_became_peered":"2024-09-11T22:50:54.175207+0000","last_unstale":"2024-09-11T22:50:55.151127+0000","last_undegraded":"2024-09-11T22:50:55.151127+0000","last_fullsized":"2024-09-11T22:50:55.151127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:47.008910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163674+0000","last_change":"2024-09-11T22:50:54.167930+0000","last_active":"2024-09-11T22:50:55.163674+0000","last_peered":"2024-09-11T22:50:55.163674+0000","last_clean":"2024-09-11T22:50:55.163674+0000","last_became_active":"2024-09-11T22:50:54.167082+0000","last_became_peered":"2024-09-11T22:50:54.167082+0000","last_unstale":"2024-09-11T22:50:55.163674+0000","last_undegraded":"2024-09-11T22:50:55.163674+0000","last_fullsized":"2024-09-11T22:50:55.163674+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:50.931311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163835+0000","last_change":"2024-09-11T22:50:54.172528+0000","last_active":"2024-09-11T22:50:55.163835+0000","last_peered":"2024-09-11T22:50:55.163835+0000","last_clean":"2024-09-11T22:50:55.163835+0000","last_became_active":"2024-09-11T22:50:54.172257+0000","last_became_peered":"2024-09-11T22:50:54.172257+0000","last_unstale":"2024-09-11T22:50:55.163835+0000","last_undegraded":"2024-09-11T22:50:55.163835+0000","last_fullsized":"2024-09-11T22:50:55.163835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:13:43.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165459+0000","last_change":"2024-09-11T22:50:54.163357+0000","last_active":"2024-09-11T22:50:55.165459+0000","last_peered":"2024-09-11T22:50:55.165459+0000","last_clean":"2024-09-11T22:50:55.165459+0000","last_became_active":"2024-09-11T22:50:54.162943+0000","last_became_peered":"2024-09-11T22:50:54.162943+0000","last_unstale":"2024-09-11T22:50:55.165459+0000","last_undegraded":"2024-09-11T22:50:55.165459+0000","last_fullsized":"2024-09-11T22:50:55.165459+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:57.055802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151603+0000","last_change":"2024-09-11T22:50:54.177773+0000","last_active":"2024-09-11T22:50:55.151603+0000","last_peered":"2024-09-11T22:50:55.151603+0000","last_clean":"2024-09-11T22:50:55.151603+0000","last_became_active":"2024-09-11T22:50:54.177388+0000","last_became_peered":"2024-09-11T22:50:54.177388+0000","last_unstale":"2024-09-11T22:50:55.151603+0000","last_undegraded":"2024-09-11T22:50:55.151603+0000","last_fullsized":"2024-09-11T22:50:55.151603+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:36:44.625089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163796+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:50:55.163796+0000","last_peered":"2024-09-11T22:50:55.163796+0000","last_clean":"2024-09-11T22:50:55.163796+0000","last_became_active":"2024-09-11T22:50:54.171491+0000","last_became_peered":"2024-09-11T22:50:54.171491+0000","last_unstale":"2024-09-11T22:50:55.163796+0000","last_undegraded":"2024-09-11T22:50:55.163796+0000","last_fullsized":"2024-09-11T22:50:55.163796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.996640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.162371+0000","last_change":"2024-09-11T22:50:54.171260+0000","last_active":"2024-09-11T22:50:55.162371+0000","last_peered":"2024-09-11T22:50:55.162371+0000","last_clean":"2024-09-11T22:50:55.162371+0000","last_became_active":"2024-09-11T22:50:54.170733+0000","last_became_peered":"2024-09-11T22:50:54.170733+0000","last_unstale":"2024-09-11T22:50:55.162371+0000","last_undegraded":"2024-09-11T22:50:55.162371+0000","last_fullsized":"2024-09-11T22:50:55.162371+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:57.877280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.161892+0000","last_change":"2024-09-11T22:50:54.170576+0000","last_active":"2024-09-11T22:50:55.161892+0000","last_peered":"2024-09-11T22:50:55.161892+0000","last_clean":"2024-09-11T22:50:55.161892+0000","last_became_active":"2024-09-11T22:50:54.170239+0000","last_became_peered":"2024-09-11T22:50:54.170239+0000","last_unstale":"2024-09-11T22:50:55.161892+0000","last_undegraded":"2024-09-11T22:50:55.161892+0000","last_fullsized":"2024-09-11T22:50:55.161892+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:09.683171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163616+0000","last_change":"2024-09-11T22:50:54.167176+0000","last_active":"2024-09-11T22:50:55.163616+0000","last_peered":"2024-09-11T22:50:55.163616+0000","last_clean":"2024-09-11T22:50:55.163616+0000","last_became_active":"2024-09-11T22:50:54.166894+0000","last_became_peered":"2024-09-11T22:50:54.166894+0000","last_unstale":"2024-09-11T22:50:55.163616+0000","last_undegraded":"2024-09-11T22:50:55.163616+0000","last_fullsized":"2024-09-11T22:50:55.163616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:43:08.158786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163050+0000","last_change":"2024-09-11T22:50:54.163100+0000","last_active":"2024-09-11T22:50:55.163050+0000","last_peered":"2024-09-11T22:50:55.163050+0000","last_clean":"2024-09-11T22:50:55.163050+0000","last_became_active":"2024-09-11T22:50:54.162776+0000","last_became_peered":"2024-09-11T22:50:54.162776+0000","last_unstale":"2024-09-11T22:50:55.163050+0000","last_undegraded":"2024-09-11T22:50:55.163050+0000","last_fullsized":"2024-09-11T22:50:55.163050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:15.215305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164805+0000","last_change":"2024-09-11T22:50:54.166013+0000","last_active":"2024-09-11T22:50:55.164805+0000","last_peered":"2024-09-11T22:50:55.164805+0000","last_clean":"2024-09-11T22:50:55.164805+0000","last_became_active":"2024-09-11T22:50:54.165680+0000","last_became_peered":"2024-09-11T22:50:54.165680+0000","last_unstale":"2024-09-11T22:50:55.164805+0000","last_undegraded":"2024-09-11T22:50:55.164805+0000","last_fullsized":"2024-09-11T22:50:55.164805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:12.192231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151814+0000","last_change":"2024-09-11T22:50:54.172105+0000","last_active":"2024-09-11T22:50:55.151814+0000","last_peered":"2024-09-11T22:50:55.151814+0000","last_clean":"2024-09-11T22:50:55.151814+0000","last_became_active":"2024-09-11T22:50:54.171782+0000","last_became_peered":"2024-09-11T22:50:54.171782+0000","last_unstale":"2024-09-11T22:50:55.151814+0000","last_undegraded":"2024-09-11T22:50:55.151814+0000","last_fullsized":"2024-09-11T22:50:55.151814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:17.152834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.161647+0000","last_change":"2024-09-11T22:50:54.167184+0000","last_active":"2024-09-11T22:50:55.161647+0000","last_peered":"2024-09-11T22:50:55.161647+0000","last_clean":"2024-09-11T22:50:55.161647+0000","last_became_active":"2024-09-11T22:50:54.166878+0000","last_became_peered":"2024-09-11T22:50:54.166878+0000","last_unstale":"2024-09-11T22:50:55.161647+0000","last_undegraded":"2024-09-11T22:50:55.161647+0000","last_fullsized":"2024-09-11T22:50:55.161647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:44.046552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.166652+0000","last_change":"2024-09-11T22:50:54.176260+0000","last_active":"2024-09-11T22:50:55.166652+0000","last_peered":"2024-09-11T22:50:55.166652+0000","last_clean":"2024-09-11T22:50:55.166652+0000","last_became_active":"2024-09-11T22:50:54.175873+0000","last_became_peered":"2024-09-11T22:50:54.175873+0000","last_unstale":"2024-09-11T22:50:55.166652+0000","last_undegraded":"2024-09-11T22:50:55.166652+0000","last_fullsized":"2024-09-11T22:50:55.166652+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:05.258677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151384+0000","last_change":"2024-09-11T22:50:54.172725+0000","last_active":"2024-09-11T22:50:55.151384+0000","last_peered":"2024-09-11T22:50:55.151384+0000","last_clean":"2024-09-11T22:50:55.151384+0000","last_became_active":"2024-09-11T22:50:54.172072+0000","last_became_peered":"2024-09-11T22:50:54.172072+0000","last_unstale":"2024-09-11T22:50:55.151384+0000","last_undegraded":"2024-09-11T22:50:55.151384+0000","last_fullsized":"2024-09-11T22:50:55.151384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:53.699670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.162028+0000","last_change":"2024-09-11T22:50:54.165118+0000","last_active":"2024-09-11T22:50:55.162028+0000","last_peered":"2024-09-11T22:50:55.162028+0000","last_clean":"2024-09-11T22:50:55.162028+0000","last_became_active":"2024-09-11T22:50:54.164793+0000","last_became_peered":"2024-09-11T22:50:54.164793+0000","last_unstale":"2024-09-11T22:50:55.162028+0000","last_undegraded":"2024-09-11T22:50:55.162028+0000","last_fullsized":"2024-09-11T22:50:55.162028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:42.883940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165155+0000","last_change":"2024-09-11T22:50:54.176900+0000","last_active":"2024-09-11T22:50:55.165155+0000","last_peered":"2024-09-11T22:50:55.165155+0000","last_clean":"2024-09-11T22:50:55.165155+0000","last_became_active":"2024-09-11T22:50:54.176595+0000","last_became_peered":"2024-09-11T22:50:54.176595+0000","last_unstale":"2024-09-11T22:50:55.165155+0000","last_undegraded":"2024-09-11T22:50:55.165155+0000","last_fullsized":"2024-09-11T22:50:55.165155+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:36.925945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164083+0000","last_change":"2024-09-11T22:50:54.167656+0000","last_active":"2024-09-11T22:50:55.164083+0000","last_peered":"2024-09-11T22:50:55.164083+0000","last_clean":"2024-09-11T22:50:55.164083+0000","last_became_active":"2024-09-11T22:50:54.167349+0000","last_became_peered":"2024-09-11T22:50:54.167349+0000","last_unstale":"2024-09-11T22:50:55.164083+0000","last_undegraded":"2024-09-11T22:50:55.164083+0000","last_fullsized":"2024-09-11T22:50:55.164083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:38.100218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151867+0000","last_change":"2024-09-11T22:50:54.177059+0000","last_active":"2024-09-11T22:50:55.151867+0000","last_peered":"2024-09-11T22:50:55.151867+0000","last_clean":"2024-09-11T22:50:55.151867+0000","last_became_active":"2024-09-11T22:50:54.176726+0000","last_became_peered":"2024-09-11T22:50:54.176726+0000","last_unstale":"2024-09-11T22:50:55.151867+0000","last_undegraded":"2024-09-11T22:50:55.151867+0000","last_fullsized":"2024-09-11T22:50:55.151867+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:27:58.058944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165322+0000","last_change":"2024-09-11T22:50:54.167566+0000","last_active":"2024-09-11T22:50:55.165322+0000","last_peered":"2024-09-11T22:50:55.165322+0000","last_clean":"2024-09-11T22:50:55.165322+0000","last_became_active":"2024-09-11T22:50:54.167049+0000","last_became_peered":"2024-09-11T22:50:54.167049+0000","last_unstale":"2024-09-11T22:50:55.165322+0000","last_undegraded":"2024-09-11T22:50:55.165322+0000","last_fullsized":"2024-09-11T22:50:55.165322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:06.833099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.167216+0000","last_change":"2024-09-11T22:50:54.169732+0000","last_active":"2024-09-11T22:50:55.167216+0000","last_peered":"2024-09-11T22:50:55.167216+0000","last_clean":"2024-09-11T22:50:55.167216+0000","last_became_active":"2024-09-11T22:50:54.169437+0000","last_became_peered":"2024-09-11T22:50:54.169437+0000","last_unstale":"2024-09-11T22:50:55.167216+0000","last_undegraded":"2024-09-11T22:50:55.167216+0000","last_fullsized":"2024-09-11T22:50:55.167216+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:25.553387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151956+0000","last_change":"2024-09-11T22:50:54.166953+0000","last_active":"2024-09-11T22:50:55.151956+0000","last_peered":"2024-09-11T22:50:55.151956+0000","last_clean":"2024-09-11T22:50:55.151956+0000","last_became_active":"2024-09-11T22:50:54.166472+0000","last_became_peered":"2024-09-11T22:50:54.166472+0000","last_unstale":"2024-09-11T22:50:55.151956+0000","last_undegraded":"2024-09-11T22:50:55.151956+0000","last_fullsized":"2024-09-11T22:50:55.151956+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:01:20.234180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.161550+0000","last_change":"2024-09-11T22:50:54.171308+0000","last_active":"2024-09-11T22:50:55.161550+0000","last_peered":"2024-09-11T22:50:55.161550+0000","last_clean":"2024-09-11T22:50:55.161550+0000","last_became_active":"2024-09-11T22:50:54.168958+0000","last_became_peered":"2024-09-11T22:50:54.168958+0000","last_unstale":"2024-09-11T22:50:55.161550+0000","last_undegraded":"2024-09-11T22:50:55.161550+0000","last_fullsized":"2024-09-11T22:50:55.161550+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:20.224003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164891+0000","last_change":"2024-09-11T22:50:54.173846+0000","last_active":"2024-09-11T22:50:55.164891+0000","last_peered":"2024-09-11T22:50:55.164891+0000","last_clean":"2024-09-11T22:50:55.164891+0000","last_became_active":"2024-09-11T22:50:54.173542+0000","last_became_peered":"2024-09-11T22:50:54.173542+0000","last_unstale":"2024-09-11T22:50:55.164891+0000","last_undegraded":"2024-09-11T22:50:55.164891+0000","last_fullsized":"2024-09-11T22:50:55.164891+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:29:45.775902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165960+0000","last_change":"2024-09-11T22:50:54.176008+0000","last_active":"2024-09-11T22:50:55.165960+0000","last_peered":"2024-09-11T22:50:55.165960+0000","last_clean":"2024-09-11T22:50:55.165960+0000","last_became_active":"2024-09-11T22:50:54.175672+0000","last_became_peered":"2024-09-11T22:50:54.175672+0000","last_unstale":"2024-09-11T22:50:55.165960+0000","last_undegraded":"2024-09-11T22:50:55.165960+0000","last_fullsized":"2024-09-11T22:50:55.165960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:35.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.166402+0000","last_change":"2024-09-11T22:50:54.162504+0000","last_active":"2024-09-11T22:50:55.166402+0000","last_peered":"2024-09-11T22:50:55.166402+0000","last_clean":"2024-09-11T22:50:55.166402+0000","last_became_active":"2024-09-11T22:50:54.162190+0000","last_became_peered":"2024-09-11T22:50:54.162190+0000","last_unstale":"2024-09-11T22:50:55.166402+0000","last_undegraded":"2024-09-11T22:50:55.166402+0000","last_fullsized":"2024-09-11T22:50:55.166402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:52.138692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151544+0000","last_change":"2024-09-11T22:50:54.167097+0000","last_active":"2024-09-11T22:50:55.151544+0000","last_peered":"2024-09-11T22:50:55.151544+0000","last_clean":"2024-09-11T22:50:55.151544+0000","last_became_active":"2024-09-11T22:50:54.166608+0000","last_became_peered":"2024-09-11T22:50:54.166608+0000","last_unstale":"2024-09-11T22:50:55.151544+0000","last_undegraded":"2024-09-11T22:50:55.151544+0000","last_fullsized":"2024-09-11T22:50:55.151544+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:49:37.522803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.161603+0000","last_change":"2024-09-11T22:50:54.160696+0000","last_active":"2024-09-11T22:50:55.161603+0000","last_peered":"2024-09-11T22:50:55.161603+0000","last_clean":"2024-09-11T22:50:55.161603+0000","last_became_active":"2024-09-11T22:50:54.159249+0000","last_became_peered":"2024-09-11T22:50:54.159249+0000","last_unstale":"2024-09-11T22:50:55.161603+0000","last_undegraded":"2024-09-11T22:50:55.161603+0000","last_fullsized":"2024-09-11T22:50:55.161603+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:09.649208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":40,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:16.149689+0000","last_change":"2024-09-11T22:50:52.138986+0000","last_active":"2024-09-11T22:51:16.149689+0000","last_peered":"2024-09-11T22:51:16.149689+0000","last_clean":"2024-09-11T22:51:16.149689+0000","last_became_active":"2024-09-11T22:50:52.138595+0000","last_became_peered":"2024-09-11T22:50:52.138595+0000","last_unstale":"2024-09-11T22:51:16.149689+0000","last_undegraded":"2024-09-11T22:51:16.149689+0000","last_fullsized":"2024-09-11T22:51:16.149689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:06.232146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.150929+0000","last_change":"2024-09-11T22:50:54.176806+0000","last_active":"2024-09-11T22:50:55.150929+0000","last_peered":"2024-09-11T22:50:55.150929+0000","last_clean":"2024-09-11T22:50:55.150929+0000","last_became_active":"2024-09-11T22:50:54.176513+0000","last_became_peered":"2024-09-11T22:50:54.176513+0000","last_unstale":"2024-09-11T22:50:55.150929+0000","last_undegraded":"2024-09-11T22:50:55.150929+0000","last_fullsized":"2024-09-11T22:50:55.150929+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:43:12.086145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:04.231381+0000","last_change":"2024-09-11T22:50:52.142794+0000","last_active":"2024-09-11T22:51:04.231381+0000","last_peered":"2024-09-11T22:51:04.231381+0000","last_clean":"2024-09-11T22:51:04.231381+0000","last_became_active":"2024-09-11T22:50:52.142433+0000","last_became_peered":"2024-09-11T22:50:52.142433+0000","last_unstale":"2024-09-11T22:51:04.231381+0000","last_undegraded":"2024-09-11T22:51:04.231381+0000","last_fullsized":"2024-09-11T22:51:04.231381+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:47:12.090055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:03.221629+0000","last_change":"2024-09-11T22:50:52.139789+0000","last_active":"2024-09-11T22:51:03.221629+0000","last_peered":"2024-09-11T22:51:03.221629+0000","last_clean":"2024-09-11T22:51:03.221629+0000","last_became_active":"2024-09-11T22:50:52.139294+0000","last_became_peered":"2024-09-11T22:50:52.139294+0000","last_unstale":"2024-09-11T22:51:03.221629+0000","last_undegraded":"2024-09-11T22:51:03.221629+0000","last_fullsized":"2024-09-11T22:51:03.221629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:07.355155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163966+0000","last_change":"2024-09-11T22:50:54.166248+0000","last_active":"2024-09-11T22:50:55.163966+0000","last_peered":"2024-09-11T22:50:55.163966+0000","last_clean":"2024-09-11T22:50:55.163966+0000","last_became_active":"2024-09-11T22:50:54.165958+0000","last_became_peered":"2024-09-11T22:50:54.165958+0000","last_unstale":"2024-09-11T22:50:55.163966+0000","last_undegraded":"2024-09-11T22:50:55.163966+0000","last_fullsized":"2024-09-11T22:50:55.163966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:29.531741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.132013+0000","last_change":"2024-09-11T22:50:52.140340+0000","last_active":"2024-09-11T22:51:05.132013+0000","last_peered":"2024-09-11T22:51:05.132013+0000","last_clean":"2024-09-11T22:51:05.132013+0000","last_became_active":"2024-09-11T22:50:52.140011+0000","last_became_peered":"2024-09-11T22:50:52.140011+0000","last_unstale":"2024-09-11T22:51:05.132013+0000","last_undegraded":"2024-09-11T22:51:05.132013+0000","last_fullsized":"2024-09-11T22:51:05.132013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:58.862640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.160891+0000","last_change":"2024-09-11T22:50:54.166998+0000","last_active":"2024-09-11T22:50:55.160891+0000","last_peered":"2024-09-11T22:50:55.160891+0000","last_clean":"2024-09-11T22:50:55.160891+0000","last_became_active":"2024-09-11T22:50:54.166682+0000","last_became_peered":"2024-09-11T22:50:54.166682+0000","last_unstale":"2024-09-11T22:50:55.160891+0000","last_undegraded":"2024-09-11T22:50:55.160891+0000","last_fullsized":"2024-09-11T22:50:55.160891+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:57.815914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":44,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:15.147252+0000","last_change":"2024-09-11T22:50:52.141380+0000","last_active":"2024-09-11T22:51:15.147252+0000","last_peered":"2024-09-11T22:51:15.147252+0000","last_clean":"2024-09-11T22:51:15.147252+0000","last_became_active":"2024-09-11T22:50:52.139925+0000","last_became_peered":"2024-09-11T22:50:52.139925+0000","last_unstale":"2024-09-11T22:51:15.147252+0000","last_undegraded":"2024-09-11T22:51:15.147252+0000","last_fullsized":"2024-09-11T22:51:15.147252+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:40:38.395055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.166509+0000","last_change":"2024-09-11T22:50:54.167909+0000","last_active":"2024-09-11T22:50:55.166509+0000","last_peered":"2024-09-11T22:50:55.166509+0000","last_clean":"2024-09-11T22:50:55.166509+0000","last_became_active":"2024-09-11T22:50:54.167571+0000","last_became_peered":"2024-09-11T22:50:54.167571+0000","last_unstale":"2024-09-11T22:50:55.166509+0000","last_undegraded":"2024-09-11T22:50:55.166509+0000","last_fullsized":"2024-09-11T22:50:55.166509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:34.179960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.136369+0000","last_change":"2024-09-11T22:50:52.144545+0000","last_active":"2024-09-11T22:51:05.136369+0000","last_peered":"2024-09-11T22:51:05.136369+0000","last_clean":"2024-09-11T22:51:05.136369+0000","last_became_active":"2024-09-11T22:50:52.144221+0000","last_became_peered":"2024-09-11T22:50:52.144221+0000","last_unstale":"2024-09-11T22:51:05.136369+0000","last_undegraded":"2024-09-11T22:51:05.136369+0000","last_fullsized":"2024-09-11T22:51:05.136369+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:29.509165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164823+0000","last_change":"2024-09-11T22:50:54.164132+0000","last_active":"2024-09-11T22:50:55.164823+0000","last_peered":"2024-09-11T22:50:55.164823+0000","last_clean":"2024-09-11T22:50:55.164823+0000","last_became_active":"2024-09-11T22:50:54.163842+0000","last_became_peered":"2024-09-11T22:50:54.163842+0000","last_unstale":"2024-09-11T22:50:55.164823+0000","last_undegraded":"2024-09-11T22:50:55.164823+0000","last_fullsized":"2024-09-11T22:50:55.164823+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:28:55.490236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:04.229584+0000","last_change":"2024-09-11T22:50:52.264611+0000","last_active":"2024-09-11T22:51:04.229584+0000","last_peered":"2024-09-11T22:51:04.229584+0000","last_clean":"2024-09-11T22:51:04.229584+0000","last_became_active":"2024-09-11T22:50:52.264060+0000","last_became_peered":"2024-09-11T22:50:52.264060+0000","last_unstale":"2024-09-11T22:51:04.229584+0000","last_undegraded":"2024-09-11T22:51:04.229584+0000","last_fullsized":"2024-09-11T22:51:04.229584+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:42.543997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165523+0000","last_change":"2024-09-11T22:50:54.165519+0000","last_active":"2024-09-11T22:50:55.165523+0000","last_peered":"2024-09-11T22:50:55.165523+0000","last_clean":"2024-09-11T22:50:55.165523+0000","last_became_active":"2024-09-11T22:50:54.165173+0000","last_became_peered":"2024-09-11T22:50:54.165173+0000","last_unstale":"2024-09-11T22:50:55.165523+0000","last_undegraded":"2024-09-11T22:50:55.165523+0000","last_fullsized":"2024-09-11T22:50:55.165523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:52.038578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'24","reported_seq":70,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:15.151208+0000","last_change":"2024-09-11T22:50:52.141782+0000","last_active":"2024-09-11T22:51:15.151208+0000","last_peered":"2024-09-11T22:51:15.151208+0000","last_clean":"2024-09-11T22:51:15.151208+0000","last_became_active":"2024-09-11T22:50:52.140662+0000","last_became_peered":"2024-09-11T22:50:52.140662+0000","last_unstale":"2024-09-11T22:51:15.151208+0000","last_undegraded":"2024-09-11T22:51:15.151208+0000","last_fullsized":"2024-09-11T22:51:15.151208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:21.158577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42203,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":28,"num_read_kb":54,"num_write":26,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.162371+0000","last_change":"2024-09-11T22:50:54.166678+0000","last_active":"2024-09-11T22:50:55.162371+0000","last_peered":"2024-09-11T22:50:55.162371+0000","last_clean":"2024-09-11T22:50:55.162371+0000","last_became_active":"2024-09-11T22:50:54.165777+0000","last_became_peered":"2024-09-11T22:50:54.165777+0000","last_unstale":"2024-09-11T22:50:55.162371+0000","last_undegraded":"2024-09-11T22:50:55.162371+0000","last_fullsized":"2024-09-11T22:50:55.162371+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:07.224563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.134896+0000","last_change":"2024-09-11T22:50:52.141292+0000","last_active":"2024-09-11T22:51:05.134896+0000","last_peered":"2024-09-11T22:51:05.134896+0000","last_clean":"2024-09-11T22:51:05.134896+0000","last_became_active":"2024-09-11T22:50:52.140971+0000","last_became_peered":"2024-09-11T22:50:52.140971+0000","last_unstale":"2024-09-11T22:51:05.134896+0000","last_undegraded":"2024-09-11T22:51:05.134896+0000","last_fullsized":"2024-09-11T22:51:05.134896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:53.039684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.161412+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:50:55.161412+0000","last_peered":"2024-09-11T22:50:55.161412+0000","last_clean":"2024-09-11T22:50:55.161412+0000","last_became_active":"2024-09-11T22:50:54.171520+0000","last_became_peered":"2024-09-11T22:50:54.171520+0000","last_unstale":"2024-09-11T22:50:55.161412+0000","last_undegraded":"2024-09-11T22:50:55.161412+0000","last_fullsized":"2024-09-11T22:50:55.161412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:56.594058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.138704+0000","last_change":"2024-09-11T22:50:52.138725+0000","last_active":"2024-09-11T22:51:05.138704+0000","last_peered":"2024-09-11T22:51:05.138704+0000","last_clean":"2024-09-11T22:51:05.138704+0000","last_became_active":"2024-09-11T22:50:52.138367+0000","last_became_peered":"2024-09-11T22:50:52.138367+0000","last_unstale":"2024-09-11T22:51:05.138704+0000","last_undegraded":"2024-09-11T22:51:05.138704+0000","last_fullsized":"2024-09-11T22:51:05.138704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:58.811444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163376+0000","last_change":"2024-09-11T22:50:54.170524+0000","last_active":"2024-09-11T22:50:55.163376+0000","last_peered":"2024-09-11T22:50:55.163376+0000","last_clean":"2024-09-11T22:50:55.163376+0000","last_became_active":"2024-09-11T22:50:54.169947+0000","last_became_peered":"2024-09-11T22:50:54.169947+0000","last_unstale":"2024-09-11T22:50:55.163376+0000","last_undegraded":"2024-09-11T22:50:55.163376+0000","last_fullsized":"2024-09-11T22:50:55.163376+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:51.768718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:15.151451+0000","last_change":"2024-09-11T22:50:52.139182+0000","last_active":"2024-09-11T22:51:15.151451+0000","last_peered":"2024-09-11T22:51:15.151451+0000","last_clean":"2024-09-11T22:51:15.151451+0000","last_became_active":"2024-09-11T22:50:52.138799+0000","last_became_peered":"2024-09-11T22:50:52.138799+0000","last_unstale":"2024-09-11T22:51:15.151451+0000","last_undegraded":"2024-09-11T22:51:15.151451+0000","last_fullsized":"2024-09-11T22:51:15.151451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:36.376162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163003+0000","last_change":"2024-09-11T22:50:54.168797+0000","last_active":"2024-09-11T22:50:55.163003+0000","last_peered":"2024-09-11T22:50:55.163003+0000","last_clean":"2024-09-11T22:50:55.163003+0000","last_became_active":"2024-09-11T22:50:54.168408+0000","last_became_peered":"2024-09-11T22:50:54.168408+0000","last_unstale":"2024-09-11T22:50:55.163003+0000","last_undegraded":"2024-09-11T22:50:55.163003+0000","last_fullsized":"2024-09-11T22:50:55.163003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:51.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.129163+0000","last_change":"2024-09-11T22:50:52.264693+0000","last_active":"2024-09-11T22:51:05.129163+0000","last_peered":"2024-09-11T22:51:05.129163+0000","last_clean":"2024-09-11T22:51:05.129163+0000","last_became_active":"2024-09-11T22:50:52.264058+0000","last_became_peered":"2024-09-11T22:50:52.264058+0000","last_unstale":"2024-09-11T22:51:05.129163+0000","last_undegraded":"2024-09-11T22:51:05.129163+0000","last_fullsized":"2024-09-11T22:51:05.129163+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:46.985921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151346+0000","last_change":"2024-09-11T22:50:54.169215+0000","last_active":"2024-09-11T22:50:55.151346+0000","last_peered":"2024-09-11T22:50:55.151346+0000","last_clean":"2024-09-11T22:50:55.151346+0000","last_became_active":"2024-09-11T22:50:54.168890+0000","last_became_peered":"2024-09-11T22:50:54.168890+0000","last_unstale":"2024-09-11T22:50:55.151346+0000","last_undegraded":"2024-09-11T22:50:55.151346+0000","last_fullsized":"2024-09-11T22:50:55.151346+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:46:42.916489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.128581+0000","last_change":"2024-09-11T22:50:52.133049+0000","last_active":"2024-09-11T22:51:05.128581+0000","last_peered":"2024-09-11T22:51:05.128581+0000","last_clean":"2024-09-11T22:51:05.128581+0000","last_became_active":"2024-09-11T22:50:52.132647+0000","last_became_peered":"2024-09-11T22:50:52.132647+0000","last_unstale":"2024-09-11T22:51:05.128581+0000","last_undegraded":"2024-09-11T22:51:05.128581+0000","last_fullsized":"2024-09-11T22:51:05.128581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:13.213003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151651+0000","last_change":"2024-09-11T22:50:54.168004+0000","last_active":"2024-09-11T22:50:55.151651+0000","last_peered":"2024-09-11T22:50:55.151651+0000","last_clean":"2024-09-11T22:50:55.151651+0000","last_became_active":"2024-09-11T22:50:54.167503+0000","last_became_peered":"2024-09-11T22:50:54.167503+0000","last_unstale":"2024-09-11T22:50:55.151651+0000","last_undegraded":"2024-09-11T22:50:55.151651+0000","last_fullsized":"2024-09-11T22:50:55.151651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:12.282400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.133823+0000","last_change":"2024-09-11T22:50:52.139766+0000","last_active":"2024-09-11T22:51:05.133823+0000","last_peered":"2024-09-11T22:51:05.133823+0000","last_clean":"2024-09-11T22:51:05.133823+0000","last_became_active":"2024-09-11T22:50:52.139456+0000","last_became_peered":"2024-09-11T22:50:52.139456+0000","last_unstale":"2024-09-11T22:51:05.133823+0000","last_undegraded":"2024-09-11T22:51:05.133823+0000","last_fullsized":"2024-09-11T22:51:05.133823+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:08.358649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164115+0000","last_change":"2024-09-11T22:50:54.167717+0000","last_active":"2024-09-11T22:50:55.164115+0000","last_peered":"2024-09-11T22:50:55.164115+0000","last_clean":"2024-09-11T22:50:55.164115+0000","last_became_active":"2024-09-11T22:50:54.167331+0000","last_became_peered":"2024-09-11T22:50:54.167331+0000","last_unstale":"2024-09-11T22:50:55.164115+0000","last_undegraded":"2024-09-11T22:50:55.164115+0000","last_fullsized":"2024-09-11T22:50:55.164115+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:11.240351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.128730+0000","last_change":"2024-09-11T22:50:52.141682+0000","last_active":"2024-09-11T22:51:05.128730+0000","last_peered":"2024-09-11T22:51:05.128730+0000","last_clean":"2024-09-11T22:51:05.128730+0000","last_became_active":"2024-09-11T22:50:52.141251+0000","last_became_peered":"2024-09-11T22:50:52.141251+0000","last_unstale":"2024-09-11T22:51:05.128730+0000","last_undegraded":"2024-09-11T22:51:05.128730+0000","last_fullsized":"2024-09-11T22:51:05.128730+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:03.068252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:57.812415+0000","last_change":"2024-09-11T22:50:54.174149+0000","last_active":"2024-09-11T22:50:57.812415+0000","last_peered":"2024-09-11T22:50:57.812415+0000","last_clean":"2024-09-11T22:50:57.812415+0000","last_became_active":"2024-09-11T22:50:54.173765+0000","last_became_peered":"2024-09-11T22:50:54.173765+0000","last_unstale":"2024-09-11T22:50:57.812415+0000","last_undegraded":"2024-09-11T22:50:57.812415+0000","last_fullsized":"2024-09-11T22:50:57.812415+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.314334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:18.152328+0000","last_change":"2024-09-11T22:50:52.435019+0000","last_active":"2024-09-11T22:51:18.152328+0000","last_peered":"2024-09-11T22:51:18.152328+0000","last_clean":"2024-09-11T22:51:18.152328+0000","last_became_active":"2024-09-11T22:50:52.434642+0000","last_became_peered":"2024-09-11T22:50:52.434642+0000","last_unstale":"2024-09-11T22:51:18.152328+0000","last_undegraded":"2024-09-11T22:51:18.152328+0000","last_fullsized":"2024-09-11T22:51:18.152328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:18.441841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164006+0000","last_change":"2024-09-11T22:50:54.171291+0000","last_active":"2024-09-11T22:50:55.164006+0000","last_peered":"2024-09-11T22:50:55.164006+0000","last_clean":"2024-09-11T22:50:55.164006+0000","last_became_active":"2024-09-11T22:50:54.170961+0000","last_became_peered":"2024-09-11T22:50:54.170961+0000","last_unstale":"2024-09-11T22:50:55.164006+0000","last_undegraded":"2024-09-11T22:50:55.164006+0000","last_fullsized":"2024-09-11T22:50:55.164006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:07.211936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":341,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163284+0000","last_change":"2024-09-11T22:49:39.098881+0000","last_active":"2024-09-11T22:50:55.163284+0000","last_peered":"2024-09-11T22:50:55.163284+0000","last_clean":"2024-09-11T22:50:55.163284+0000","last_became_active":"2024-09-11T22:49:39.098458+0000","last_became_peered":"2024-09-11T22:49:39.098458+0000","last_unstale":"2024-09-11T22:50:55.163284+0000","last_undegraded":"2024-09-11T22:50:55.163284+0000","last_fullsized":"2024-09-11T22:50:55.163284+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_clean_scrub_stamp":"2024-09-11T22:46:11.418577+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:41.059384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.135942+0000","last_change":"2024-09-11T22:50:52.137691+0000","last_active":"2024-09-11T22:51:05.135942+0000","last_peered":"2024-09-11T22:51:05.135942+0000","last_clean":"2024-09-11T22:51:05.135942+0000","last_became_active":"2024-09-11T22:50:52.137282+0000","last_became_peered":"2024-09-11T22:50:52.137282+0000","last_unstale":"2024-09-11T22:51:05.135942+0000","last_undegraded":"2024-09-11T22:51:05.135942+0000","last_fullsized":"2024-09-11T22:51:05.135942+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:00:27.651252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165455+0000","last_change":"2024-09-11T22:50:54.162417+0000","last_active":"2024-09-11T22:50:55.165455+0000","last_peered":"2024-09-11T22:50:55.165455+0000","last_clean":"2024-09-11T22:50:55.165455+0000","last_became_active":"2024-09-11T22:50:54.160457+0000","last_became_peered":"2024-09-11T22:50:54.160457+0000","last_unstale":"2024-09-11T22:50:55.165455+0000","last_undegraded":"2024-09-11T22:50:55.165455+0000","last_fullsized":"2024-09-11T22:50:55.165455+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:20:45.167775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:03.221448+0000","last_change":"2024-09-11T22:50:52.144324+0000","last_active":"2024-09-11T22:51:03.221448+0000","last_peered":"2024-09-11T22:51:03.221448+0000","last_clean":"2024-09-11T22:51:03.221448+0000","last_became_active":"2024-09-11T22:50:52.144011+0000","last_became_peered":"2024-09-11T22:50:52.144011+0000","last_unstale":"2024-09-11T22:51:03.221448+0000","last_undegraded":"2024-09-11T22:51:03.221448+0000","last_fullsized":"2024-09-11T22:51:03.221448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:29.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.162189+0000","last_change":"2024-09-11T22:50:54.173287+0000","last_active":"2024-09-11T22:50:55.162189+0000","last_peered":"2024-09-11T22:50:55.162189+0000","last_clean":"2024-09-11T22:50:55.162189+0000","last_became_active":"2024-09-11T22:50:54.172953+0000","last_became_peered":"2024-09-11T22:50:54.172953+0000","last_unstale":"2024-09-11T22:50:55.162189+0000","last_undegraded":"2024-09-11T22:50:55.162189+0000","last_fullsized":"2024-09-11T22:50:55.162189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:24.314604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:16.152042+0000","last_change":"2024-09-11T22:50:52.263777+0000","last_active":"2024-09-11T22:51:16.152042+0000","last_peered":"2024-09-11T22:51:16.152042+0000","last_clean":"2024-09-11T22:51:16.152042+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:51:16.152042+0000","last_undegraded":"2024-09-11T22:51:16.152042+0000","last_fullsized":"2024-09-11T22:51:16.152042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:37.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.150847+0000","last_change":"2024-09-11T22:50:54.164927+0000","last_active":"2024-09-11T22:50:55.150847+0000","last_peered":"2024-09-11T22:50:55.150847+0000","last_clean":"2024-09-11T22:50:55.150847+0000","last_became_active":"2024-09-11T22:50:54.164602+0000","last_became_peered":"2024-09-11T22:50:54.164602+0000","last_unstale":"2024-09-11T22:50:55.150847+0000","last_undegraded":"2024-09-11T22:50:55.150847+0000","last_fullsized":"2024-09-11T22:50:55.150847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:10.226232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:16.152325+0000","last_change":"2024-09-11T22:50:52.141199+0000","last_active":"2024-09-11T22:51:16.152325+0000","last_peered":"2024-09-11T22:51:16.152325+0000","last_clean":"2024-09-11T22:51:16.152325+0000","last_became_active":"2024-09-11T22:50:52.140830+0000","last_became_peered":"2024-09-11T22:50:52.140830+0000","last_unstale":"2024-09-11T22:51:16.152325+0000","last_undegraded":"2024-09-11T22:51:16.152325+0000","last_fullsized":"2024-09-11T22:51:16.152325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.767803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151265+0000","last_change":"2024-09-11T22:50:54.175054+0000","last_active":"2024-09-11T22:50:55.151265+0000","last_peered":"2024-09-11T22:50:55.151265+0000","last_clean":"2024-09-11T22:50:55.151265+0000","last_became_active":"2024-09-11T22:50:54.174715+0000","last_became_peered":"2024-09-11T22:50:54.174715+0000","last_unstale":"2024-09-11T22:50:55.151265+0000","last_undegraded":"2024-09-11T22:50:55.151265+0000","last_fullsized":"2024-09-11T22:50:55.151265+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:06.694988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.820546+0000","last_change":"2024-09-11T22:50:52.264212+0000","last_active":"2024-09-11T22:50:55.820546+0000","last_peered":"2024-09-11T22:50:55.820546+0000","last_clean":"2024-09-11T22:50:55.820546+0000","last_became_active":"2024-09-11T22:50:52.263882+0000","last_became_peered":"2024-09-11T22:50:52.263882+0000","last_unstale":"2024-09-11T22:50:55.820546+0000","last_undegraded":"2024-09-11T22:50:55.820546+0000","last_fullsized":"2024-09-11T22:50:55.820546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:29.635043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.826728+0000","last_change":"2024-09-11T22:50:54.167361+0000","last_active":"2024-09-11T22:50:55.826728+0000","last_peered":"2024-09-11T22:50:55.826728+0000","last_clean":"2024-09-11T22:50:55.826728+0000","last_became_active":"2024-09-11T22:50:54.166963+0000","last_became_peered":"2024-09-11T22:50:54.166963+0000","last_unstale":"2024-09-11T22:50:55.826728+0000","last_undegraded":"2024-09-11T22:50:55.826728+0000","last_fullsized":"2024-09-11T22:50:55.826728+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:13.294544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.129353+0000","last_change":"2024-09-11T22:50:52.264484+0000","last_active":"2024-09-11T22:51:05.129353+0000","last_peered":"2024-09-11T22:51:05.129353+0000","last_clean":"2024-09-11T22:51:05.129353+0000","last_became_active":"2024-09-11T22:50:52.264038+0000","last_became_peered":"2024-09-11T22:50:52.264038+0000","last_unstale":"2024-09-11T22:51:05.129353+0000","last_undegraded":"2024-09-11T22:51:05.129353+0000","last_fullsized":"2024-09-11T22:51:05.129353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:21.325136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.161702+0000","last_change":"2024-09-11T22:50:54.162225+0000","last_active":"2024-09-11T22:50:55.161702+0000","last_peered":"2024-09-11T22:50:55.161702+0000","last_clean":"2024-09-11T22:50:55.161702+0000","last_became_active":"2024-09-11T22:50:54.161234+0000","last_became_peered":"2024-09-11T22:50:54.161234+0000","last_unstale":"2024-09-11T22:50:55.161702+0000","last_undegraded":"2024-09-11T22:50:55.161702+0000","last_fullsized":"2024-09-11T22:50:55.161702+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:00.346217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'12","reported_seq":30,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:14.161966+0000","last_change":"2024-09-11T22:50:52.142366+0000","last_active":"2024-09-11T22:51:14.161966+0000","last_peered":"2024-09-11T22:51:14.161966+0000","last_clean":"2024-09-11T22:51:14.161966+0000","last_became_active":"2024-09-11T22:50:52.141931+0000","last_became_peered":"2024-09-11T22:50:52.141931+0000","last_unstale":"2024-09-11T22:51:14.161966+0000","last_undegraded":"2024-09-11T22:51:14.161966+0000","last_fullsized":"2024-09-11T22:51:14.161966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:02:33.864079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165440+0000","last_change":"2024-09-11T22:50:54.167307+0000","last_active":"2024-09-11T22:50:55.165440+0000","last_peered":"2024-09-11T22:50:55.165440+0000","last_clean":"2024-09-11T22:50:55.165440+0000","last_became_active":"2024-09-11T22:50:54.166650+0000","last_became_peered":"2024-09-11T22:50:54.166650+0000","last_unstale":"2024-09-11T22:50:55.165440+0000","last_undegraded":"2024-09-11T22:50:55.165440+0000","last_fullsized":"2024-09-11T22:50:55.165440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:04.863258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.136774+0000","last_change":"2024-09-11T22:50:52.435107+0000","last_active":"2024-09-11T22:51:05.136774+0000","last_peered":"2024-09-11T22:51:05.136774+0000","last_clean":"2024-09-11T22:51:05.136774+0000","last_became_active":"2024-09-11T22:50:52.434791+0000","last_became_peered":"2024-09-11T22:50:52.434791+0000","last_unstale":"2024-09-11T22:51:05.136774+0000","last_undegraded":"2024-09-11T22:51:05.136774+0000","last_fullsized":"2024-09-11T22:51:05.136774+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:11.128791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163017+0000","last_change":"2024-09-11T22:50:54.178041+0000","last_active":"2024-09-11T22:50:55.163017+0000","last_peered":"2024-09-11T22:50:55.163017+0000","last_clean":"2024-09-11T22:50:55.163017+0000","last_became_active":"2024-09-11T22:50:54.177393+0000","last_became_peered":"2024-09-11T22:50:54.177393+0000","last_unstale":"2024-09-11T22:50:55.163017+0000","last_undegraded":"2024-09-11T22:50:55.163017+0000","last_fullsized":"2024-09-11T22:50:55.163017+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:21.608104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:02.210217+0000","last_change":"2024-09-11T22:50:52.264051+0000","last_active":"2024-09-11T22:51:02.210217+0000","last_peered":"2024-09-11T22:51:02.210217+0000","last_clean":"2024-09-11T22:51:02.210217+0000","last_became_active":"2024-09-11T22:50:52.263674+0000","last_became_peered":"2024-09-11T22:50:52.263674+0000","last_unstale":"2024-09-11T22:51:02.210217+0000","last_undegraded":"2024-09-11T22:51:02.210217+0000","last_fullsized":"2024-09-11T22:51:02.210217+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:44.655218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.162611+0000","last_change":"2024-09-11T22:50:54.166409+0000","last_active":"2024-09-11T22:50:55.162611+0000","last_peered":"2024-09-11T22:50:55.162611+0000","last_clean":"2024-09-11T22:50:55.162611+0000","last_became_active":"2024-09-11T22:50:54.166037+0000","last_became_peered":"2024-09-11T22:50:54.166037+0000","last_unstale":"2024-09-11T22:50:55.162611+0000","last_undegraded":"2024-09-11T22:50:55.162611+0000","last_fullsized":"2024-09-11T22:50:55.162611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:22.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.135220+0000","last_change":"2024-09-11T22:50:52.140218+0000","last_active":"2024-09-11T22:51:05.135220+0000","last_peered":"2024-09-11T22:51:05.135220+0000","last_clean":"2024-09-11T22:51:05.135220+0000","last_became_active":"2024-09-11T22:50:52.139744+0000","last_became_peered":"2024-09-11T22:50:52.139744+0000","last_unstale":"2024-09-11T22:51:05.135220+0000","last_undegraded":"2024-09-11T22:51:05.135220+0000","last_fullsized":"2024-09-11T22:51:05.135220+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:50.178348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151900+0000","last_change":"2024-09-11T22:50:54.164059+0000","last_active":"2024-09-11T22:50:55.151900+0000","last_peered":"2024-09-11T22:50:55.151900+0000","last_clean":"2024-09-11T22:50:55.151900+0000","last_became_active":"2024-09-11T22:50:54.163724+0000","last_became_peered":"2024-09-11T22:50:54.163724+0000","last_unstale":"2024-09-11T22:50:55.151900+0000","last_undegraded":"2024-09-11T22:50:55.151900+0000","last_fullsized":"2024-09-11T22:50:55.151900+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:57.241660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:04.232193+0000","last_change":"2024-09-11T22:50:52.140169+0000","last_active":"2024-09-11T22:51:04.232193+0000","last_peered":"2024-09-11T22:51:04.232193+0000","last_clean":"2024-09-11T22:51:04.232193+0000","last_became_active":"2024-09-11T22:50:52.139579+0000","last_became_peered":"2024-09-11T22:50:52.139579+0000","last_unstale":"2024-09-11T22:51:04.232193+0000","last_undegraded":"2024-09-11T22:51:04.232193+0000","last_fullsized":"2024-09-11T22:51:04.232193+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:43.523961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.163639+0000","last_change":"2024-09-11T22:50:54.171539+0000","last_active":"2024-09-11T22:50:55.163639+0000","last_peered":"2024-09-11T22:50:55.163639+0000","last_clean":"2024-09-11T22:50:55.163639+0000","last_became_active":"2024-09-11T22:50:54.171013+0000","last_became_peered":"2024-09-11T22:50:54.171013+0000","last_unstale":"2024-09-11T22:50:55.163639+0000","last_undegraded":"2024-09-11T22:50:55.163639+0000","last_fullsized":"2024-09-11T22:50:55.163639+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:03.564389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:02.210120+0000","last_change":"2024-09-11T22:50:52.139544+0000","last_active":"2024-09-11T22:51:02.210120+0000","last_peered":"2024-09-11T22:51:02.210120+0000","last_clean":"2024-09-11T22:51:02.210120+0000","last_became_active":"2024-09-11T22:50:52.139201+0000","last_became_peered":"2024-09-11T22:50:52.139201+0000","last_unstale":"2024-09-11T22:51:02.210120+0000","last_undegraded":"2024-09-11T22:51:02.210120+0000","last_fullsized":"2024-09-11T22:51:02.210120+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:37.343437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164279+0000","last_change":"2024-09-11T22:50:54.170052+0000","last_active":"2024-09-11T22:50:55.164279+0000","last_peered":"2024-09-11T22:50:55.164279+0000","last_clean":"2024-09-11T22:50:55.164279+0000","last_became_active":"2024-09-11T22:50:54.169735+0000","last_became_peered":"2024-09-11T22:50:54.169735+0000","last_unstale":"2024-09-11T22:50:55.164279+0000","last_undegraded":"2024-09-11T22:50:55.164279+0000","last_fullsized":"2024-09-11T22:50:55.164279+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:44.007747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.151969+0000","last_change":"2024-09-11T22:50:54.169959+0000","last_active":"2024-09-11T22:50:55.151969+0000","last_peered":"2024-09-11T22:50:55.151969+0000","last_clean":"2024-09-11T22:50:55.151969+0000","last_became_active":"2024-09-11T22:50:54.169649+0000","last_became_peered":"2024-09-11T22:50:54.169649+0000","last_unstale":"2024-09-11T22:50:55.151969+0000","last_undegraded":"2024-09-11T22:50:55.151969+0000","last_fullsized":"2024-09-11T22:50:55.151969+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:51.974258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.137968+0000","last_change":"2024-09-11T22:50:52.140881+0000","last_active":"2024-09-11T22:51:05.137968+0000","last_peered":"2024-09-11T22:51:05.137968+0000","last_clean":"2024-09-11T22:51:05.137968+0000","last_became_active":"2024-09-11T22:50:52.140527+0000","last_became_peered":"2024-09-11T22:50:52.140527+0000","last_unstale":"2024-09-11T22:51:05.137968+0000","last_undegraded":"2024-09-11T22:51:05.137968+0000","last_fullsized":"2024-09-11T22:51:05.137968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:14.325816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165206+0000","last_change":"2024-09-11T22:50:54.166571+0000","last_active":"2024-09-11T22:50:55.165206+0000","last_peered":"2024-09-11T22:50:55.165206+0000","last_clean":"2024-09-11T22:50:55.165206+0000","last_became_active":"2024-09-11T22:50:54.166247+0000","last_became_peered":"2024-09-11T22:50:54.166247+0000","last_unstale":"2024-09-11T22:50:55.165206+0000","last_undegraded":"2024-09-11T22:50:55.165206+0000","last_fullsized":"2024-09-11T22:50:55.165206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:42.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:05.138249+0000","last_change":"2024-09-11T22:50:52.140298+0000","last_active":"2024-09-11T22:51:05.138249+0000","last_peered":"2024-09-11T22:51:05.138249+0000","last_clean":"2024-09-11T22:51:05.138249+0000","last_became_active":"2024-09-11T22:50:52.139917+0000","last_became_peered":"2024-09-11T22:50:52.139917+0000","last_unstale":"2024-09-11T22:51:05.138249+0000","last_undegraded":"2024-09-11T22:51:05.138249+0000","last_fullsized":"2024-09-11T22:51:05.138249+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:23.599010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.164777+0000","last_change":"2024-09-11T22:50:54.170849+0000","last_active":"2024-09-11T22:50:55.164777+0000","last_peered":"2024-09-11T22:50:55.164777+0000","last_clean":"2024-09-11T22:50:55.164777+0000","last_became_active":"2024-09-11T22:50:54.169547+0000","last_became_peered":"2024-09-11T22:50:54.169547+0000","last_unstale":"2024-09-11T22:50:55.164777+0000","last_undegraded":"2024-09-11T22:50:55.164777+0000","last_fullsized":"2024-09-11T22:50:55.164777+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:25.251158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.167153+0000","last_change":"2024-09-11T22:50:52.263778+0000","last_active":"2024-09-11T22:50:55.167153+0000","last_peered":"2024-09-11T22:50:55.167153+0000","last_clean":"2024-09-11T22:50:55.167153+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:50:55.167153+0000","last_undegraded":"2024-09-11T22:50:55.167153+0000","last_fullsized":"2024-09-11T22:50:55.167153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:03.439147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:50:55.165281+0000","last_change":"2024-09-11T22:50:54.171200+0000","last_active":"2024-09-11T22:50:55.165281+0000","last_peered":"2024-09-11T22:50:55.165281+0000","last_clean":"2024-09-11T22:50:55.165281+0000","last_became_active":"2024-09-11T22:50:54.169320+0000","last_became_peered":"2024-09-11T22:50:54.169320+0000","last_unstale":"2024-09-11T22:50:55.165281+0000","last_undegraded":"2024-09-11T22:50:55.165281+0000","last_fullsized":"2024-09-11T22:50:55.165281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:14.864699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":42,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-11T22:51:14.149595+0000","last_change":"2024-09-11T22:50:52.139609+0000","last_active":"2024-09-11T22:51:14.149595+0000","last_peered":"2024-09-11T22:51:14.149595+0000","last_clean":"2024-09-11T22:51:14.149595+0000","last_became_active":"2024-09-11T22:50:52.139278+0000","last_became_peered":"2024-09-11T22:50:52.139278+0000","last_unstale":"2024-09-11T22:51:14.149595+0000","last_undegraded":"2024-09-11T22:51:14.149595+0000","last_fullsized":"2024-09-11T22:51:14.149595+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:09.538734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":49411,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":157,"num_read_kb":131,"num_write":231,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":172717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281769,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":880640,"data_stored":420837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":1053676,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939676,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":420374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168609,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":420752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397542,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":1013272,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561068,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":420091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790001,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":1012950,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953526,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":419952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182460,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":419916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":476455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":475449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":421823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":56197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":43939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:51:23.083 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T22:51:23.084 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T22:51:23.084 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T22:51:23.339 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:23.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.851+0000 7f37deddf640 1 -- 172.21.15.152:0/3951108339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37d8101cd0 msgr2=0x7f37d8102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:23.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.851+0000 7f37deddf640 1 --2- 172.21.15.152:0/3951108339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37d8101cd0 0x7f37d8102150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009900 tx=0x7f37c402a410 comp rx=0 tx=0).stop 2024-09-11T22:51:23.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.851+0000 7f37deddf640 1 -- 172.21.15.152:0/3951108339 shutdown_connections 2024-09-11T22:51:23.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.851+0000 7f37deddf640 1 --2- 172.21.15.152:0/3951108339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d8102690 0x7f37d8108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:23.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.851+0000 7f37deddf640 1 --2- 172.21.15.152:0/3951108339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37d8101cd0 0x7f37d8102150 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:23.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.851+0000 7f37deddf640 1 --2- 172.21.15.152:0/3951108339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37d8100ad0 0x7f37d8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:23.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.851+0000 7f37deddf640 1 -- 172.21.15.152:0/3951108339 >> 172.21.15.152:0/3951108339 conn(0x7f37d80fc280 msgr2=0x7f37d80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:23.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.851+0000 7f37deddf640 1 -- 172.21.15.152:0/3951108339 shutdown_connections 2024-09-11T22:51:23.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.851+0000 7f37deddf640 1 -- 172.21.15.152:0/3951108339 wait complete. 2024-09-11T22:51:23.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.851+0000 7f37deddf640 1 Processor -- start 2024-09-11T22:51:23.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.851+0000 7f37deddf640 1 -- start start 2024-09-11T22:51:23.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37deddf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37d8100ad0 0x7f37d8197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:23.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37deddf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d8101cd0 0x7f37d81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:23.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37deddf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37d8102690 0x7f37d819f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:23.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37deddf640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f37d810d880 con 0x7f37d8102690 2024-09-11T22:51:23.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37deddf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f37d810d700 con 0x7f37d8100ad0 2024-09-11T22:51:23.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37deddf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37d810da00 con 0x7f37d8101cd0 2024-09-11T22:51:23.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37dd37c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37d8102690 0x7f37d819f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:23.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37dd37c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37d8102690 0x7f37d819f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58602/0 (socket says 172.21.15.152:58602) 2024-09-11T22:51:23.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37dd37c640 1 -- 172.21.15.152:0/2345068764 learned_addr learned my addr 172.21.15.152:0/2345068764 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:23.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37dcb7b640 1 --2- 172.21.15.152:0/2345068764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37d8100ad0 0x7f37d8197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:23.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37d7fff640 1 --2- 172.21.15.152:0/2345068764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d8101cd0 0x7f37d81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:23.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37dd37c640 1 -- 172.21.15.152:0/2345068764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d8101cd0 msgr2=0x7f37d81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:23.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37dd37c640 1 --2- 172.21.15.152:0/2345068764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d8101cd0 0x7f37d81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:23.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37dd37c640 1 -- 172.21.15.152:0/2345068764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37d8100ad0 msgr2=0x7f37d8197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:23.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37dd37c640 1 --2- 172.21.15.152:0/2345068764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37d8100ad0 0x7f37d8197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:23.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37dd37c640 1 -- 172.21.15.152:0/2345068764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d819fae0 con 0x7f37d8102690 2024-09-11T22:51:23.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37dd37c640 1 --2- 172.21.15.152:0/2345068764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37d8102690 0x7f37d819f470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f37d000cad0 tx=0x7f37d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:23.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.852+0000 7f37d5ffb640 1 -- 172.21.15.152:0/2345068764 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d000ae20 con 0x7f37d8102690 2024-09-11T22:51:23.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.853+0000 7f37d5ffb640 1 -- 172.21.15.152:0/2345068764 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d0005020 con 0x7f37d8102690 2024-09-11T22:51:23.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.853+0000 7f37deddf640 1 -- 172.21.15.152:0/2345068764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d819fdd0 con 0x7f37d8102690 2024-09-11T22:51:23.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.853+0000 7f37d5ffb640 1 -- 172.21.15.152:0/2345068764 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d000f6f0 con 0x7f37d8102690 2024-09-11T22:51:23.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.853+0000 7f37deddf640 1 -- 172.21.15.152:0/2345068764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d81a02d0 con 0x7f37d8102690 2024-09-11T22:51:23.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.854+0000 7f37deddf640 1 -- 172.21.15.152:0/2345068764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d8107ed0 con 0x7f37d8102690 2024-09-11T22:51:23.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.855+0000 7f37d5ffb640 1 -- 172.21.15.152:0/2345068764 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f37d00051d0 con 0x7f37d8102690 2024-09-11T22:51:23.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.855+0000 7f37d5ffb640 1 --2- 172.21.15.152:0/2345068764 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f37c80a2cc0 0x7f37c80a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:23.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.856+0000 7f37dcb7b640 1 --2- 172.21.15.152:0/2345068764 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f37c80a2cc0 0x7f37c80a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:23.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.856+0000 7f37d5ffb640 1 -- 172.21.15.152:0/2345068764 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f37d00cef90 con 0x7f37d8102690 2024-09-11T22:51:23.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.856+0000 7f37dcb7b640 1 --2- 172.21.15.152:0/2345068764 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f37c80a2cc0 0x7f37c80a5180 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f37d8101b30 tx=0x7f37cc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:23.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:23.859+0000 7f37d5ffb640 1 -- 172.21.15.152:0/2345068764 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f37d0014030 con 0x7f37d8102690 2024-09-11T22:51:24.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:24.094+0000 7f37deddf640 1 -- 172.21.15.152:0/2345068764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f37d802da00 con 0x7f37d8102690 2024-09-11T22:51:24.094 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:23 smithi152 ceph-mon[31181]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:51:24.094 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:23 smithi152 ceph-mon[31181]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 15 op/s 2024-09-11T22:51:24.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:24.096+0000 7f37d5ffb640 1 -- 172.21.15.152:0/2345068764 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f37d009bf40 con 0x7f37d8102690 2024-09-11T22:51:24.096 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2024-09-11T22:51:24.096 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQDsHuJmYsWrBRAAo9FFyiq7IgyaUtAORMngQg== 2024-09-11T22:51:24.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:24.098+0000 7f37deddf640 1 -- 172.21.15.152:0/2345068764 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f37c80a2cc0 msgr2=0x7f37c80a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:24.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:24.098+0000 7f37deddf640 1 --2- 172.21.15.152:0/2345068764 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f37c80a2cc0 0x7f37c80a5180 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f37d8101b30 tx=0x7f37cc005f90 comp rx=0 tx=0).stop 2024-09-11T22:51:24.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:24.098+0000 7f37deddf640 1 -- 172.21.15.152:0/2345068764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37d8102690 msgr2=0x7f37d819f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:24.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:24.098+0000 7f37deddf640 1 --2- 172.21.15.152:0/2345068764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37d8102690 0x7f37d819f470 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f37d000cad0 tx=0x7f37d000cf90 comp rx=0 tx=0).stop 2024-09-11T22:51:24.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:24.098+0000 7f37deddf640 1 -- 172.21.15.152:0/2345068764 shutdown_connections 2024-09-11T22:51:24.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:24.098+0000 7f37deddf640 1 --2- 172.21.15.152:0/2345068764 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f37c80a2cc0 0x7f37c80a5180 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:24.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:24.098+0000 7f37deddf640 1 --2- 172.21.15.152:0/2345068764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37d8102690 0x7f37d819f470 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:24.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:24.098+0000 7f37deddf640 1 --2- 172.21.15.152:0/2345068764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d8101cd0 0x7f37d81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:24.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:24.098+0000 7f37deddf640 1 --2- 172.21.15.152:0/2345068764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37d8100ad0 0x7f37d8197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:24.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:24.098+0000 7f37deddf640 1 -- 172.21.15.152:0/2345068764 >> 172.21.15.152:0/2345068764 conn(0x7f37d80fc280 msgr2=0x7f37d81050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:24.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:24.098+0000 7f37deddf640 1 -- 172.21.15.152:0/2345068764 shutdown_connections 2024-09-11T22:51:24.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:24.098+0000 7f37deddf640 1 -- 172.21.15.152:0/2345068764 wait complete. 2024-09-11T22:51:24.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:23 smithi191 ceph-mon[35211]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:51:24.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:23 smithi191 ceph-mon[35211]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 15 op/s 2024-09-11T22:51:24.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:23 smithi204 ceph-mon[35454]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:51:24.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:23 smithi204 ceph-mon[35454]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 15 op/s 2024-09-11T22:51:24.816 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:51:24.816 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T22:51:24.816 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T22:51:24.856 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T22:51:24.856 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T22:51:24.856 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T22:51:25.125 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:24 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2345068764' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T22:51:25.125 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:24 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2345068764' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T22:51:25.145 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:25.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:24 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2345068764' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T22:51:25.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:24 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2345068764' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T22:51:25.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:24 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2345068764' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T22:51:25.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:24 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2345068764' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T22:51:25.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.657+0000 7fd54d824640 1 -- 172.21.15.152:0/525253514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548100a80 msgr2=0x7fd548100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:25.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.657+0000 7fd54d824640 1 --2- 172.21.15.152:0/525253514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548100a80 0x7fd548100ee0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd530009930 tx=0x7fd53002a420 comp rx=0 tx=0).stop 2024-09-11T22:51:25.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.657+0000 7fd54d824640 1 -- 172.21.15.152:0/525253514 shutdown_connections 2024-09-11T22:51:25.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.657+0000 7fd54d824640 1 --2- 172.21.15.152:0/525253514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd548101420 0x7fd54810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:25.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.657+0000 7fd54d824640 1 --2- 172.21.15.152:0/525253514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548100a80 0x7fd548100ee0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:25.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.657+0000 7fd54d824640 1 --2- 172.21.15.152:0/525253514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd548106a80 0x7fd548106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:25.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.657+0000 7fd54d824640 1 -- 172.21.15.152:0/525253514 >> 172.21.15.152:0/525253514 conn(0x7fd5480fc7a0 msgr2=0x7fd5480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.657+0000 7fd54d824640 1 -- 172.21.15.152:0/525253514 shutdown_connections 2024-09-11T22:51:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.657+0000 7fd54d824640 1 -- 172.21.15.152:0/525253514 wait complete. 2024-09-11T22:51:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.658+0000 7fd54d824640 1 Processor -- start 2024-09-11T22:51:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.658+0000 7fd54d824640 1 -- start start 2024-09-11T22:51:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.658+0000 7fd54d824640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd548100a80 0x7fd548198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.658+0000 7fd54d824640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548101420 0x7fd548198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.658+0000 7fd54d824640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd548106a80 0x7fd54819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.658+0000 7fd54d824640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd548110e80 con 0x7fd548101420 2024-09-11T22:51:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.658+0000 7fd54d824640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd548110d00 con 0x7fd548106a80 2024-09-11T22:51:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.658+0000 7fd54d824640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd548111000 con 0x7fd548100a80 2024-09-11T22:51:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.658+0000 7fd5467fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548101420 0x7fd548198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.658+0000 7fd5467fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548101420 0x7fd548198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58630/0 (socket says 172.21.15.152:58630) 2024-09-11T22:51:25.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.658+0000 7fd5467fc640 1 -- 172.21.15.152:0/4287477821 learned_addr learned my addr 172.21.15.152:0/4287477821 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:25.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.659+0000 7fd546ffd640 1 --2- 172.21.15.152:0/4287477821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd548100a80 0x7fd548198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:25.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.659+0000 7fd5477fe640 1 --2- 172.21.15.152:0/4287477821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd548106a80 0x7fd54819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:25.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.659+0000 7fd5467fc640 1 -- 172.21.15.152:0/4287477821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd548100a80 msgr2=0x7fd548198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:25.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.659+0000 7fd5467fc640 1 --2- 172.21.15.152:0/4287477821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd548100a80 0x7fd548198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:25.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.659+0000 7fd5467fc640 1 -- 172.21.15.152:0/4287477821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd548106a80 msgr2=0x7fd54819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:25.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.659+0000 7fd5467fc640 1 --2- 172.21.15.152:0/4287477821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd548106a80 0x7fd54819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:25.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.659+0000 7fd5467fc640 1 -- 172.21.15.152:0/4287477821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54819d3d0 con 0x7fd548101420 2024-09-11T22:51:25.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.659+0000 7fd5477fe640 1 --2- 172.21.15.152:0/4287477821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd548106a80 0x7fd54819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:25.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.659+0000 7fd546ffd640 1 --2- 172.21.15.152:0/4287477821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd548100a80 0x7fd548198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:25.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.659+0000 7fd5467fc640 1 --2- 172.21.15.152:0/4287477821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548101420 0x7fd548198960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd530009230 tx=0x7fd530004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:25.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.659+0000 7fd54c822640 1 -- 172.21.15.152:0/4287477821 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53003f030 con 0x7fd548101420 2024-09-11T22:51:25.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.659+0000 7fd54c822640 1 -- 172.21.15.152:0/4287477821 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd530003ad0 con 0x7fd548101420 2024-09-11T22:51:25.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.659+0000 7fd54d824640 1 -- 172.21.15.152:0/4287477821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54819d660 con 0x7fd548101420 2024-09-11T22:51:25.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.659+0000 7fd54c822640 1 -- 172.21.15.152:0/4287477821 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530035070 con 0x7fd548101420 2024-09-11T22:51:25.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.659+0000 7fd54d824640 1 -- 172.21.15.152:0/4287477821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5481a4fa0 con 0x7fd548101420 2024-09-11T22:51:25.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.660+0000 7fd54d824640 1 -- 172.21.15.152:0/4287477821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5481021c0 con 0x7fd548101420 2024-09-11T22:51:25.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.661+0000 7fd54c822640 1 -- 172.21.15.152:0/4287477821 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd530042030 con 0x7fd548101420 2024-09-11T22:51:25.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.662+0000 7fd54c822640 1 --2- 172.21.15.152:0/4287477821 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd5340a2c10 0x7fd5340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:25.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.662+0000 7fd546ffd640 1 --2- 172.21.15.152:0/4287477821 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd5340a2c10 0x7fd5340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:25.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.662+0000 7fd54c822640 1 -- 172.21.15.152:0/4287477821 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fd5300ef5c0 con 0x7fd548101420 2024-09-11T22:51:25.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.662+0000 7fd546ffd640 1 --2- 172.21.15.152:0/4287477821 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd5340a2c10 0x7fd5340a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd53800baa0 tx=0x7fd538005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:25.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.666+0000 7fd54c822640 1 -- 172.21.15.152:0/4287477821 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5300bc570 con 0x7fd548101420 2024-09-11T22:51:25.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.882+0000 7fd54d824640 1 -- 172.21.15.152:0/4287477821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd54810a020 con 0x7fd548101420 2024-09-11T22:51:25.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.886+0000 7fd54c822640 1 -- 172.21.15.152:0/4287477821 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263210 (secure 0 0 0) 0x7fd5300bbe70 con 0x7fd548101420 2024-09-11T22:51:25.891 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:51:25.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.889+0000 7fd54d824640 1 -- 172.21.15.152:0/4287477821 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd5340a2c10 msgr2=0x7fd5340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:25.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.889+0000 7fd54d824640 1 --2- 172.21.15.152:0/4287477821 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd5340a2c10 0x7fd5340a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd53800baa0 tx=0x7fd538005d20 comp rx=0 tx=0).stop 2024-09-11T22:51:25.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.889+0000 7fd54d824640 1 -- 172.21.15.152:0/4287477821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548101420 msgr2=0x7fd548198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:25.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.889+0000 7fd54d824640 1 --2- 172.21.15.152:0/4287477821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548101420 0x7fd548198960 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd530009230 tx=0x7fd530004270 comp rx=0 tx=0).stop 2024-09-11T22:51:25.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.889+0000 7fd54d824640 1 -- 172.21.15.152:0/4287477821 shutdown_connections 2024-09-11T22:51:25.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.889+0000 7fd54d824640 1 --2- 172.21.15.152:0/4287477821 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd5340a2c10 0x7fd5340a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:25.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.889+0000 7fd54d824640 1 --2- 172.21.15.152:0/4287477821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd548106a80 0x7fd54819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:25.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.889+0000 7fd54d824640 1 --2- 172.21.15.152:0/4287477821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd548101420 0x7fd548198960 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:25.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.889+0000 7fd54d824640 1 --2- 172.21.15.152:0/4287477821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd548100a80 0x7fd548198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:25.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.889+0000 7fd54d824640 1 -- 172.21.15.152:0/4287477821 >> 172.21.15.152:0/4287477821 conn(0x7fd5480fc7a0 msgr2=0x7fd54806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:25.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.889+0000 7fd54d824640 1 -- 172.21.15.152:0/4287477821 shutdown_connections 2024-09-11T22:51:25.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:25.889+0000 7fd54d824640 1 -- 172.21.15.152:0/4287477821 wait complete. 2024-09-11T22:51:26.283 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:25 smithi152 ceph-mon[31181]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-11T22:51:26.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:25 smithi191 ceph-mon[35211]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-11T22:51:26.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:25 smithi204 ceph-mon[35454]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-09-11T22:51:26.783 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":3438861948},{"type":"v1","addr":"172.21.15.152:6801","nonce":3438861948}]},"active_addr":"172.21.15.152:6801/3438861948","active_change":"2024-09-11T22:43:54.679480+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.152:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.152:0","nonce":2557435316}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.152:0","nonce":3900732888}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.152:0","nonce":1721817189}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.152:0","nonce":2259758812}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.152:0","nonce":3854035250}]}]} 2024-09-11T22:51:26.790 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T22:51:26.790 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T22:51:26.790 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:51:27.041 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:27.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:26 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4287477821' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T22:51:27.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:26 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/4287477821' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T22:51:27.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:26 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/4287477821' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T22:51:27.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.527+0000 7fe61b447640 1 -- 172.21.15.152:0/2640353222 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe614106a80 msgr2=0x7fe614106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:27.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.527+0000 7fe61b447640 1 --2- 172.21.15.152:0/2640353222 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe614106a80 0x7fe614106e60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe610009930 tx=0x7fe61002a420 comp rx=0 tx=0).stop 2024-09-11T22:51:27.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.527+0000 7fe61b447640 1 -- 172.21.15.152:0/2640353222 shutdown_connections 2024-09-11T22:51:27.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.527+0000 7fe61b447640 1 --2- 172.21.15.152:0/2640353222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe614101420 0x7fe61410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:27.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.527+0000 7fe61b447640 1 --2- 172.21.15.152:0/2640353222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe614100a80 0x7fe614100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:27.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.527+0000 7fe61b447640 1 --2- 172.21.15.152:0/2640353222 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe614106a80 0x7fe614106e60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:27.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.527+0000 7fe61b447640 1 -- 172.21.15.152:0/2640353222 >> 172.21.15.152:0/2640353222 conn(0x7fe6140fc7a0 msgr2=0x7fe6140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:27.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.527+0000 7fe61b447640 1 -- 172.21.15.152:0/2640353222 shutdown_connections 2024-09-11T22:51:27.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.527+0000 7fe61b447640 1 -- 172.21.15.152:0/2640353222 wait complete. 2024-09-11T22:51:27.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.528+0000 7fe61b447640 1 Processor -- start 2024-09-11T22:51:27.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.528+0000 7fe61b447640 1 -- start start 2024-09-11T22:51:27.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.528+0000 7fe61b447640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe614100a80 0x7fe61419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:27.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.528+0000 7fe61b447640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe614101420 0x7fe61419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:27.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.528+0000 7fe61b447640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe614106a80 0x7fe61419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:27.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.528+0000 7fe61b447640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe614111f40 con 0x7fe614100a80 2024-09-11T22:51:27.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.528+0000 7fe61b447640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe614111dc0 con 0x7fe614101420 2024-09-11T22:51:27.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.528+0000 7fe61b447640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe6141120c0 con 0x7fe614106a80 2024-09-11T22:51:27.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.528+0000 7fe6191e3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe614100a80 0x7fe61419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:27.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.528+0000 7fe6191e3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe614100a80 0x7fe61419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58638/0 (socket says 172.21.15.152:58638) 2024-09-11T22:51:27.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.528+0000 7fe6191e3640 1 -- 172.21.15.152:0/3139515593 learned_addr learned my addr 172.21.15.152:0/3139515593 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:27.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.529+0000 7fe6199e4640 1 --2- 172.21.15.152:0/3139515593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe614106a80 0x7fe61419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:27.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.529+0000 7fe6189e2640 1 --2- 172.21.15.152:0/3139515593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe614101420 0x7fe61419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:27.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.529+0000 7fe6191e3640 1 -- 172.21.15.152:0/3139515593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe614106a80 msgr2=0x7fe61419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:27.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.529+0000 7fe6191e3640 1 --2- 172.21.15.152:0/3139515593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe614106a80 0x7fe61419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:27.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.529+0000 7fe6191e3640 1 -- 172.21.15.152:0/3139515593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe614101420 msgr2=0x7fe61419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:27.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.529+0000 7fe6191e3640 1 --2- 172.21.15.152:0/3139515593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe614101420 0x7fe61419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:27.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.529+0000 7fe6191e3640 1 -- 172.21.15.152:0/3139515593 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61419f5d0 con 0x7fe614100a80 2024-09-11T22:51:27.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.529+0000 7fe6191e3640 1 --2- 172.21.15.152:0/3139515593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe614100a80 0x7fe61419a620 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe610009a50 tx=0x7fe610004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:27.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.529+0000 7fe60a7fc640 1 -- 172.21.15.152:0/3139515593 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61003f030 con 0x7fe614100a80 2024-09-11T22:51:27.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.529+0000 7fe61b447640 1 -- 172.21.15.152:0/3139515593 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61419f860 con 0x7fe614100a80 2024-09-11T22:51:27.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.529+0000 7fe60a7fc640 1 -- 172.21.15.152:0/3139515593 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe610003ce0 con 0x7fe614100a80 2024-09-11T22:51:27.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.529+0000 7fe60a7fc640 1 -- 172.21.15.152:0/3139515593 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610035070 con 0x7fe614100a80 2024-09-11T22:51:27.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.529+0000 7fe61b447640 1 -- 172.21.15.152:0/3139515593 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6141a71a0 con 0x7fe614100a80 2024-09-11T22:51:27.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.530+0000 7fe61b447640 1 -- 172.21.15.152:0/3139515593 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6141021c0 con 0x7fe614100a80 2024-09-11T22:51:27.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.531+0000 7fe60a7fc640 1 -- 172.21.15.152:0/3139515593 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe610042030 con 0x7fe614100a80 2024-09-11T22:51:27.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.532+0000 7fe60a7fc640 1 --2- 172.21.15.152:0/3139515593 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6000a2c10 0x7fe6000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:27.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.532+0000 7fe6189e2640 1 --2- 172.21.15.152:0/3139515593 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6000a2c10 0x7fe6000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:27.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.532+0000 7fe60a7fc640 1 -- 172.21.15.152:0/3139515593 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fe6100ef670 con 0x7fe614100a80 2024-09-11T22:51:27.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.532+0000 7fe6189e2640 1 --2- 172.21.15.152:0/3139515593 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6000a2c10 0x7fe6000a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe61410db30 tx=0x7fe604005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:27.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.536+0000 7fe60a7fc640 1 -- 172.21.15.152:0/3139515593 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6100bc620 con 0x7fe614100a80 2024-09-11T22:51:27.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.703+0000 7fe61b447640 1 -- 172.21.15.152:0/3139515593 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6141a7370 con 0x7fe614100a80 2024-09-11T22:51:27.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.705+0000 7fe60a7fc640 1 -- 172.21.15.152:0/3139515593 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22363 (secure 0 0 0) 0x7fe6100bbf20 con 0x7fe614100a80 2024-09-11T22:51:27.706 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:51:27.706 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":81,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","created":"2024-09-11T22:43:00.379317+0000","modified":"2024-09-11T22:50:55.047140+0000","last_up_change":"2024-09-11T22:50:01.043181+0000","last_in_change":"2024-09-11T22:49:38.073870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:46:10.984707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:50:50.833547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:50:53.047604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3cd917b5-77f6-4a4a-8fd0-4ebe3034feac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6803","nonce":1270728007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6805","nonce":1270728007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6809","nonce":1270728007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6807","nonce":1270728007}]},"public_addr":"172.21.15.152:6803/1270728007","cluster_addr":"172.21.15.152:6805/1270728007","heartbeat_back_addr":"172.21.15.152:6809/1270728007","heartbeat_front_addr":"172.21.15.152:6807/1270728007","state":["exists","up"]},{"osd":1,"uuid":"ae589bc4-77b1-4522-9591-b0ce2a5432dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6801","nonce":354467155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6803","nonce":354467155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6807","nonce":354467155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6805","nonce":354467155}]},"public_addr":"172.21.15.191:6801/354467155","cluster_addr":"172.21.15.191:6803/354467155","heartbeat_back_addr":"172.21.15.191:6807/354467155","heartbeat_front_addr":"172.21.15.191:6805/354467155","state":["exists","up"]},{"osd":2,"uuid":"b35ffd0c-d084-43dd-a947-15132d78d59c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6801","nonce":3848396211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6803","nonce":3848396211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6807","nonce":3848396211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6805","nonce":3848396211}]},"public_addr":"172.21.15.204:6801/3848396211","cluster_addr":"172.21.15.204:6803/3848396211","heartbeat_back_addr":"172.21.15.204:6807/3848396211","heartbeat_front_addr":"172.21.15.204:6805/3848396211","state":["exists","up"]},{"osd":3,"uuid":"d66ba7e8-4ab7-449e-9460-2f44ffdf987f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6811","nonce":3351257181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6813","nonce":3351257181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6817","nonce":3351257181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6815","nonce":3351257181}]},"public_addr":"172.21.15.152:6811/3351257181","cluster_addr":"172.21.15.152:6813/3351257181","heartbeat_back_addr":"172.21.15.152:6817/3351257181","heartbeat_front_addr":"172.21.15.152:6815/3351257181","state":["exists","up"]},{"osd":4,"uuid":"e1b56468-6cc7-4d29-988e-9d281118ee36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6809","nonce":2967539563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6811","nonce":2967539563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6815","nonce":2967539563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6813","nonce":2967539563}]},"public_addr":"172.21.15.191:6809/2967539563","cluster_addr":"172.21.15.191:6811/2967539563","heartbeat_back_addr":"172.21.15.191:6815/2967539563","heartbeat_front_addr":"172.21.15.191:6813/2967539563","state":["exists","up"]},{"osd":5,"uuid":"89dabd3e-77e3-4294-9648-b112ef694e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6809","nonce":4283562158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6811","nonce":4283562158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6815","nonce":4283562158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6813","nonce":4283562158}]},"public_addr":"172.21.15.204:6809/4283562158","cluster_addr":"172.21.15.204:6811/4283562158","heartbeat_back_addr":"172.21.15.204:6815/4283562158","heartbeat_front_addr":"172.21.15.204:6813/4283562158","state":["exists","up"]},{"osd":6,"uuid":"46a0fded-b668-4aef-a68f-613cccc83483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6819","nonce":2484234042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6821","nonce":2484234042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6825","nonce":2484234042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6823","nonce":2484234042}]},"public_addr":"172.21.15.152:6819/2484234042","cluster_addr":"172.21.15.152:6821/2484234042","heartbeat_back_addr":"172.21.15.152:6825/2484234042","heartbeat_front_addr":"172.21.15.152:6823/2484234042","state":["exists","up"]},{"osd":7,"uuid":"e674bfe9-e2db-44a3-b74b-a21f198747c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6817","nonce":269608859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6819","nonce":269608859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6823","nonce":269608859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6821","nonce":269608859}]},"public_addr":"172.21.15.191:6817/269608859","cluster_addr":"172.21.15.191:6819/269608859","heartbeat_back_addr":"172.21.15.191:6823/269608859","heartbeat_front_addr":"172.21.15.191:6821/269608859","state":["exists","up"]},{"osd":8,"uuid":"0418bfc9-d6b6-488f-b4c7-0bc6291bb624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6817","nonce":1756190090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6819","nonce":1756190090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6823","nonce":1756190090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6821","nonce":1756190090}]},"public_addr":"172.21.15.204:6817/1756190090","cluster_addr":"172.21.15.204:6819/1756190090","heartbeat_back_addr":"172.21.15.204:6823/1756190090","heartbeat_front_addr":"172.21.15.204:6821/1756190090","state":["exists","up"]},{"osd":9,"uuid":"751a6964-798f-4240-a2ba-7fa9f0c72657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6827","nonce":1301191419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6829","nonce":1301191419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6833","nonce":1301191419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6831","nonce":1301191419}]},"public_addr":"172.21.15.152:6827/1301191419","cluster_addr":"172.21.15.152:6829/1301191419","heartbeat_back_addr":"172.21.15.152:6833/1301191419","heartbeat_front_addr":"172.21.15.152:6831/1301191419","state":["exists","up"]},{"osd":10,"uuid":"90e7c6ba-127c-4b33-b3c6-cedf72128451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6825","nonce":3695294503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6827","nonce":3695294503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6831","nonce":3695294503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6829","nonce":3695294503}]},"public_addr":"172.21.15.191:6825/3695294503","cluster_addr":"172.21.15.191:6827/3695294503","heartbeat_back_addr":"172.21.15.191:6831/3695294503","heartbeat_front_addr":"172.21.15.191:6829/3695294503","state":["exists","up"]},{"osd":11,"uuid":"c76a93b1-a045-477b-98d0-58a2e9d44a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6825","nonce":1709433236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6827","nonce":1709433236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6831","nonce":1709433236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6829","nonce":1709433236}]},"public_addr":"172.21.15.204:6825/1709433236","cluster_addr":"172.21.15.204:6827/1709433236","heartbeat_back_addr":"172.21.15.204:6831/1709433236","heartbeat_front_addr":"172.21.15.204:6829/1709433236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:17.259744+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:42.230389+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:07.172433+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:32.517060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:58.900615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:24.379803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:50.058928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:15.967299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:42.025873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:08.225448+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:33.910531+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:59.466423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:0/1542993850":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/3071755865":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/2452559014":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/4286398432":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6801/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6800/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6800/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/3960601922":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2270507277":"2024-09-12T22:43:31.970432+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:51:27.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.707+0000 7fe61b447640 1 -- 172.21.15.152:0/3139515593 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6000a2c10 msgr2=0x7fe6000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:27.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.707+0000 7fe61b447640 1 --2- 172.21.15.152:0/3139515593 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6000a2c10 0x7fe6000a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe61410db30 tx=0x7fe604005cc0 comp rx=0 tx=0).stop 2024-09-11T22:51:27.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.707+0000 7fe61b447640 1 -- 172.21.15.152:0/3139515593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe614100a80 msgr2=0x7fe61419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:27.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.707+0000 7fe61b447640 1 --2- 172.21.15.152:0/3139515593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe614100a80 0x7fe61419a620 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe610009a50 tx=0x7fe610004270 comp rx=0 tx=0).stop 2024-09-11T22:51:27.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.707+0000 7fe61b447640 1 -- 172.21.15.152:0/3139515593 shutdown_connections 2024-09-11T22:51:27.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.707+0000 7fe61b447640 1 --2- 172.21.15.152:0/3139515593 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6000a2c10 0x7fe6000a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:27.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.707+0000 7fe61b447640 1 --2- 172.21.15.152:0/3139515593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe614106a80 0x7fe61419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:27.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.707+0000 7fe61b447640 1 --2- 172.21.15.152:0/3139515593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe614101420 0x7fe61419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:27.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.707+0000 7fe61b447640 1 --2- 172.21.15.152:0/3139515593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe614100a80 0x7fe61419a620 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:27.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.707+0000 7fe61b447640 1 -- 172.21.15.152:0/3139515593 >> 172.21.15.152:0/3139515593 conn(0x7fe6140fc7a0 msgr2=0x7fe61410c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:27.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.707+0000 7fe61b447640 1 -- 172.21.15.152:0/3139515593 shutdown_connections 2024-09-11T22:51:27.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:27.707+0000 7fe61b447640 1 -- 172.21.15.152:0/3139515593 wait complete. 2024-09-11T22:51:28.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:27 smithi152 ceph-mon[31181]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-09-11T22:51:28.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:27 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3139515593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:51:28.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:27 smithi191 ceph-mon[35211]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-09-11T22:51:28.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:27 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3139515593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:51:28.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:27 smithi204 ceph-mon[35454]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.8 KiB/s wr, 12 op/s 2024-09-11T22:51:28.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:27 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3139515593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:51:28.529 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T22:51:28.529 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:51:28.778 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:29.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.250+0000 7fce2b974640 1 -- 172.21.15.152:0/3376940417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce241082a0 msgr2=0x7fce2410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:29.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.250+0000 7fce2b974640 1 --2- 172.21.15.152:0/3376940417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce241082a0 0x7fce2410ec40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fce1c009930 tx=0x7fce1c02a420 comp rx=0 tx=0).stop 2024-09-11T22:51:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.250+0000 7fce2b974640 1 -- 172.21.15.152:0/3376940417 shutdown_connections 2024-09-11T22:51:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.250+0000 7fce2b974640 1 --2- 172.21.15.152:0/3376940417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce241082a0 0x7fce2410ec40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.250+0000 7fce2b974640 1 --2- 172.21.15.152:0/3376940417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce241036e0 0x7fce24107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.250+0000 7fce2b974640 1 --2- 172.21.15.152:0/3376940417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce24102d30 0x7fce24103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.250+0000 7fce2b974640 1 -- 172.21.15.152:0/3376940417 >> 172.21.15.152:0/3376940417 conn(0x7fce240fc7d0 msgr2=0x7fce240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.250+0000 7fce2b974640 1 -- 172.21.15.152:0/3376940417 shutdown_connections 2024-09-11T22:51:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.250+0000 7fce2b974640 1 -- 172.21.15.152:0/3376940417 wait complete. 2024-09-11T22:51:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.250+0000 7fce2b974640 1 Processor -- start 2024-09-11T22:51:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.250+0000 7fce2b974640 1 -- start start 2024-09-11T22:51:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce2b974640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce24102d30 0x7fce24100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce2b974640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce241036e0 0x7fce24100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:29.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce2b974640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce241082a0 0x7fce24101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:29.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce2b974640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fce241120a0 con 0x7fce241082a0 2024-09-11T22:51:29.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce2b974640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fce24111f20 con 0x7fce24102d30 2024-09-11T22:51:29.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce2b974640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fce24112220 con 0x7fce241036e0 2024-09-11T22:51:29.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce29f11640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce241082a0 0x7fce24101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:29.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce29f11640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce241082a0 0x7fce24101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33838/0 (socket says 172.21.15.152:33838) 2024-09-11T22:51:29.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce29f11640 1 -- 172.21.15.152:0/520207194 learned_addr learned my addr 172.21.15.152:0/520207194 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:29.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce28f0f640 1 --2- 172.21.15.152:0/520207194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce241036e0 0x7fce24100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:29.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce29710640 1 --2- 172.21.15.152:0/520207194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce24102d30 0x7fce24100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:29.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce29f11640 1 -- 172.21.15.152:0/520207194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce241036e0 msgr2=0x7fce24100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:29.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce29f11640 1 --2- 172.21.15.152:0/520207194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce241036e0 0x7fce24100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:29.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce29f11640 1 -- 172.21.15.152:0/520207194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce24102d30 msgr2=0x7fce24100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce29f11640 1 --2- 172.21.15.152:0/520207194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce24102d30 0x7fce24100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce29f11640 1 -- 172.21.15.152:0/520207194 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce241a6a10 con 0x7fce241082a0 2024-09-11T22:51:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce29f11640 1 --2- 172.21.15.152:0/520207194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce241082a0 0x7fce24101410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fce1c005ab0 tx=0x7fce1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce227fc640 1 -- 172.21.15.152:0/520207194 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c03f030 con 0x7fce241082a0 2024-09-11T22:51:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce2b974640 1 -- 172.21.15.152:0/520207194 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce241a6ca0 con 0x7fce241082a0 2024-09-11T22:51:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce227fc640 1 -- 172.21.15.152:0/520207194 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce1c003ce0 con 0x7fce241082a0 2024-09-11T22:51:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce227fc640 1 -- 172.21.15.152:0/520207194 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c035070 con 0x7fce241082a0 2024-09-11T22:51:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.251+0000 7fce2b974640 1 -- 172.21.15.152:0/520207194 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce241a7180 con 0x7fce241082a0 2024-09-11T22:51:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.252+0000 7fce2b974640 1 -- 172.21.15.152:0/520207194 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce24103110 con 0x7fce241082a0 2024-09-11T22:51:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.254+0000 7fce227fc640 1 -- 172.21.15.152:0/520207194 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fce1c042030 con 0x7fce241082a0 2024-09-11T22:51:29.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.254+0000 7fce227fc640 1 --2- 172.21.15.152:0/520207194 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fce0c0a2c10 0x7fce0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:29.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.254+0000 7fce29710640 1 --2- 172.21.15.152:0/520207194 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fce0c0a2c10 0x7fce0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:29.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.254+0000 7fce227fc640 1 -- 172.21.15.152:0/520207194 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fce1c0ef900 con 0x7fce241082a0 2024-09-11T22:51:29.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.255+0000 7fce29710640 1 --2- 172.21.15.152:0/520207194 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fce0c0a2c10 0x7fce0c0a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce14005fd0 tx=0x7fce14009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:29.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.257+0000 7fce227fc640 1 -- 172.21.15.152:0/520207194 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce1c0bc8b0 con 0x7fce241082a0 2024-09-11T22:51:29.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.428+0000 7fce2b974640 1 -- 172.21.15.152:0/520207194 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce2410a2b0 con 0x7fce241082a0 2024-09-11T22:51:29.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.429+0000 7fce227fc640 1 -- 172.21.15.152:0/520207194 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22363 (secure 0 0 0) 0x7fce1c0bc1b0 con 0x7fce241082a0 2024-09-11T22:51:29.430 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:51:29.430 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":81,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","created":"2024-09-11T22:43:00.379317+0000","modified":"2024-09-11T22:50:55.047140+0000","last_up_change":"2024-09-11T22:50:01.043181+0000","last_in_change":"2024-09-11T22:49:38.073870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:46:10.984707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:50:50.833547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:50:53.047604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3cd917b5-77f6-4a4a-8fd0-4ebe3034feac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6803","nonce":1270728007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6805","nonce":1270728007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6809","nonce":1270728007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6807","nonce":1270728007}]},"public_addr":"172.21.15.152:6803/1270728007","cluster_addr":"172.21.15.152:6805/1270728007","heartbeat_back_addr":"172.21.15.152:6809/1270728007","heartbeat_front_addr":"172.21.15.152:6807/1270728007","state":["exists","up"]},{"osd":1,"uuid":"ae589bc4-77b1-4522-9591-b0ce2a5432dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6801","nonce":354467155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6803","nonce":354467155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6807","nonce":354467155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6805","nonce":354467155}]},"public_addr":"172.21.15.191:6801/354467155","cluster_addr":"172.21.15.191:6803/354467155","heartbeat_back_addr":"172.21.15.191:6807/354467155","heartbeat_front_addr":"172.21.15.191:6805/354467155","state":["exists","up"]},{"osd":2,"uuid":"b35ffd0c-d084-43dd-a947-15132d78d59c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6801","nonce":3848396211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6803","nonce":3848396211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6807","nonce":3848396211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6805","nonce":3848396211}]},"public_addr":"172.21.15.204:6801/3848396211","cluster_addr":"172.21.15.204:6803/3848396211","heartbeat_back_addr":"172.21.15.204:6807/3848396211","heartbeat_front_addr":"172.21.15.204:6805/3848396211","state":["exists","up"]},{"osd":3,"uuid":"d66ba7e8-4ab7-449e-9460-2f44ffdf987f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6811","nonce":3351257181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6813","nonce":3351257181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6817","nonce":3351257181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6815","nonce":3351257181}]},"public_addr":"172.21.15.152:6811/3351257181","cluster_addr":"172.21.15.152:6813/3351257181","heartbeat_back_addr":"172.21.15.152:6817/3351257181","heartbeat_front_addr":"172.21.15.152:6815/3351257181","state":["exists","up"]},{"osd":4,"uuid":"e1b56468-6cc7-4d29-988e-9d281118ee36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6809","nonce":2967539563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6811","nonce":2967539563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6815","nonce":2967539563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6813","nonce":2967539563}]},"public_addr":"172.21.15.191:6809/2967539563","cluster_addr":"172.21.15.191:6811/2967539563","heartbeat_back_addr":"172.21.15.191:6815/2967539563","heartbeat_front_addr":"172.21.15.191:6813/2967539563","state":["exists","up"]},{"osd":5,"uuid":"89dabd3e-77e3-4294-9648-b112ef694e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6809","nonce":4283562158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6811","nonce":4283562158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6815","nonce":4283562158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6813","nonce":4283562158}]},"public_addr":"172.21.15.204:6809/4283562158","cluster_addr":"172.21.15.204:6811/4283562158","heartbeat_back_addr":"172.21.15.204:6815/4283562158","heartbeat_front_addr":"172.21.15.204:6813/4283562158","state":["exists","up"]},{"osd":6,"uuid":"46a0fded-b668-4aef-a68f-613cccc83483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6819","nonce":2484234042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6821","nonce":2484234042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6825","nonce":2484234042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6823","nonce":2484234042}]},"public_addr":"172.21.15.152:6819/2484234042","cluster_addr":"172.21.15.152:6821/2484234042","heartbeat_back_addr":"172.21.15.152:6825/2484234042","heartbeat_front_addr":"172.21.15.152:6823/2484234042","state":["exists","up"]},{"osd":7,"uuid":"e674bfe9-e2db-44a3-b74b-a21f198747c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6817","nonce":269608859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6819","nonce":269608859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6823","nonce":269608859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6821","nonce":269608859}]},"public_addr":"172.21.15.191:6817/269608859","cluster_addr":"172.21.15.191:6819/269608859","heartbeat_back_addr":"172.21.15.191:6823/269608859","heartbeat_front_addr":"172.21.15.191:6821/269608859","state":["exists","up"]},{"osd":8,"uuid":"0418bfc9-d6b6-488f-b4c7-0bc6291bb624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6817","nonce":1756190090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6819","nonce":1756190090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6823","nonce":1756190090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6821","nonce":1756190090}]},"public_addr":"172.21.15.204:6817/1756190090","cluster_addr":"172.21.15.204:6819/1756190090","heartbeat_back_addr":"172.21.15.204:6823/1756190090","heartbeat_front_addr":"172.21.15.204:6821/1756190090","state":["exists","up"]},{"osd":9,"uuid":"751a6964-798f-4240-a2ba-7fa9f0c72657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6827","nonce":1301191419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6829","nonce":1301191419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6833","nonce":1301191419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6831","nonce":1301191419}]},"public_addr":"172.21.15.152:6827/1301191419","cluster_addr":"172.21.15.152:6829/1301191419","heartbeat_back_addr":"172.21.15.152:6833/1301191419","heartbeat_front_addr":"172.21.15.152:6831/1301191419","state":["exists","up"]},{"osd":10,"uuid":"90e7c6ba-127c-4b33-b3c6-cedf72128451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6825","nonce":3695294503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6827","nonce":3695294503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6831","nonce":3695294503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6829","nonce":3695294503}]},"public_addr":"172.21.15.191:6825/3695294503","cluster_addr":"172.21.15.191:6827/3695294503","heartbeat_back_addr":"172.21.15.191:6831/3695294503","heartbeat_front_addr":"172.21.15.191:6829/3695294503","state":["exists","up"]},{"osd":11,"uuid":"c76a93b1-a045-477b-98d0-58a2e9d44a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6825","nonce":1709433236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6827","nonce":1709433236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6831","nonce":1709433236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6829","nonce":1709433236}]},"public_addr":"172.21.15.204:6825/1709433236","cluster_addr":"172.21.15.204:6827/1709433236","heartbeat_back_addr":"172.21.15.204:6831/1709433236","heartbeat_front_addr":"172.21.15.204:6829/1709433236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:17.259744+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:42.230389+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:07.172433+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:32.517060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:58.900615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:24.379803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:50.058928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:15.967299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:42.025873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:08.225448+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:33.910531+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:59.466423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:0/1542993850":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/3071755865":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/2452559014":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/4286398432":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6801/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6800/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6800/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/3960601922":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2270507277":"2024-09-12T22:43:31.970432+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:51:29.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.431+0000 7fce2b974640 1 -- 172.21.15.152:0/520207194 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fce0c0a2c10 msgr2=0x7fce0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:29.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.431+0000 7fce2b974640 1 --2- 172.21.15.152:0/520207194 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fce0c0a2c10 0x7fce0c0a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce14005fd0 tx=0x7fce14009870 comp rx=0 tx=0).stop 2024-09-11T22:51:29.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.431+0000 7fce2b974640 1 -- 172.21.15.152:0/520207194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce241082a0 msgr2=0x7fce24101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:29.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.431+0000 7fce2b974640 1 --2- 172.21.15.152:0/520207194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce241082a0 0x7fce24101410 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fce1c005ab0 tx=0x7fce1c0042f0 comp rx=0 tx=0).stop 2024-09-11T22:51:29.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.431+0000 7fce2b974640 1 -- 172.21.15.152:0/520207194 shutdown_connections 2024-09-11T22:51:29.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.431+0000 7fce2b974640 1 --2- 172.21.15.152:0/520207194 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fce0c0a2c10 0x7fce0c0a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:29.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.431+0000 7fce2b974640 1 --2- 172.21.15.152:0/520207194 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce241082a0 0x7fce24101410 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:29.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.431+0000 7fce2b974640 1 --2- 172.21.15.152:0/520207194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fce241036e0 0x7fce24100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:29.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.431+0000 7fce2b974640 1 --2- 172.21.15.152:0/520207194 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce24102d30 0x7fce24100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:29.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.431+0000 7fce2b974640 1 -- 172.21.15.152:0/520207194 >> 172.21.15.152:0/520207194 conn(0x7fce240fc7d0 msgr2=0x7fce24105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:29.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.432+0000 7fce2b974640 1 -- 172.21.15.152:0/520207194 shutdown_connections 2024-09-11T22:51:29.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:29.432+0000 7fce2b974640 1 -- 172.21.15.152:0/520207194 wait complete. 2024-09-11T22:51:30.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:29 smithi152 ceph-mon[31181]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-11T22:51:30.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:29 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/520207194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:51:30.282 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T22:51:30.283 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T22:51:30.283 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T22:51:30.284 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T22:51:30.284 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T22:51:30.284 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T22:51:30.285 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T22:51:30.285 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T22:51:30.286 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-11T22:51:30.286 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-11T22:51:30.286 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-11T22:51:30.287 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-11T22:51:30.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:29 smithi191 ceph-mon[35211]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-11T22:51:30.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:29 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/520207194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:51:30.299 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:29 smithi204 ceph-mon[35454]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-11T22:51:30.299 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:29 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/520207194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:51:30.929 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:30.994 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:31.000 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:31.002 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:31.002 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:31.008 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:31.014 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:31.014 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.133+0000 7fdc6d557640 1 -- 172.21.15.152:0/979794366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc68101400 msgr2=0x7fdc6810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.133+0000 7fdc6d557640 1 --2- 172.21.15.152:0/979794366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc68101400 0x7fdc6810d930 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009cd0 tx=0x7fdc6002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:51:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.133+0000 7fdc6d557640 1 -- 172.21.15.152:0/979794366 shutdown_connections 2024-09-11T22:51:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.133+0000 7fdc6d557640 1 --2- 172.21.15.152:0/979794366 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc68101400 0x7fdc6810d930 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:32.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.133+0000 7fdc6d557640 1 --2- 172.21.15.152:0/979794366 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc68100a60 0x7fdc68100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:32.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.133+0000 7fdc6d557640 1 --2- 172.21.15.152:0/979794366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc68106a60 0x7fdc68106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:32.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.133+0000 7fdc6d557640 1 -- 172.21.15.152:0/979794366 >> 172.21.15.152:0/979794366 conn(0x7fdc680fc780 msgr2=0x7fdc680feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:32.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.133+0000 7fdc6d557640 1 -- 172.21.15.152:0/979794366 shutdown_connections 2024-09-11T22:51:32.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.133+0000 7fdc6d557640 1 -- 172.21.15.152:0/979794366 wait complete. 2024-09-11T22:51:32.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.134+0000 7fdc6d557640 1 Processor -- start 2024-09-11T22:51:32.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.134+0000 7fdc6d557640 1 -- start start 2024-09-11T22:51:32.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.134+0000 7fdc6d557640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc68100a60 0x7fdc681972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:32.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.134+0000 7fdc6d557640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc68101400 0x7fdc681977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:32.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.134+0000 7fdc6d557640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc68106a60 0x7fdc6819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:32.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.134+0000 7fdc6d557640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdc68110e60 con 0x7fdc68101400 2024-09-11T22:51:32.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.134+0000 7fdc6d557640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdc68110ce0 con 0x7fdc68106a60 2024-09-11T22:51:32.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.134+0000 7fdc6d557640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdc68110fe0 con 0x7fdc68100a60 2024-09-11T22:51:32.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.134+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc68101400 0x7fdc681977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.134+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc68101400 0x7fdc681977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33842/0 (socket says 172.21.15.152:33842) 2024-09-11T22:51:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.134+0000 7fdc5ffff640 1 -- 172.21.15.152:0/3974354404 learned_addr learned my addr 172.21.15.152:0/3974354404 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.134+0000 7fdc66ffd640 1 --2- 172.21.15.152:0/3974354404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc68100a60 0x7fdc681972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.134+0000 7fdc677fe640 1 --2- 172.21.15.152:0/3974354404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc68106a60 0x7fdc6819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.135+0000 7fdc5ffff640 1 -- 172.21.15.152:0/3974354404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc68100a60 msgr2=0x7fdc681972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.135+0000 7fdc5ffff640 1 --2- 172.21.15.152:0/3974354404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc68100a60 0x7fdc681972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.135+0000 7fdc5ffff640 1 -- 172.21.15.152:0/3974354404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc68106a60 msgr2=0x7fdc6819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.135+0000 7fdc5ffff640 1 --2- 172.21.15.152:0/3974354404 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdc68106a60 0x7fdc6819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.135+0000 7fdc5ffff640 1 -- 172.21.15.152:0/3974354404 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6819c250 con 0x7fdc68101400 2024-09-11T22:51:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.135+0000 7fdc5ffff640 1 --2- 172.21.15.152:0/3974354404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc68101400 0x7fdc681977e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c920 tx=0x7fdc5000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.135+0000 7fdc64ff9640 1 -- 172.21.15.152:0/3974354404 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50007c70 con 0x7fdc68101400 2024-09-11T22:51:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.135+0000 7fdc64ff9640 1 -- 172.21.15.152:0/3974354404 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc50007e10 con 0x7fdc68101400 2024-09-11T22:51:32.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.135+0000 7fdc6d557640 1 -- 172.21.15.152:0/3974354404 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6819c540 con 0x7fdc68101400 2024-09-11T22:51:32.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.135+0000 7fdc64ff9640 1 -- 172.21.15.152:0/3974354404 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50018890 con 0x7fdc68101400 2024-09-11T22:51:32.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.135+0000 7fdc6d557640 1 -- 172.21.15.152:0/3974354404 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc681a3e20 con 0x7fdc68101400 2024-09-11T22:51:32.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.136+0000 7fdc6d557640 1 -- 172.21.15.152:0/3974354404 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdc680fd960 con 0x7fdc68101400 2024-09-11T22:51:32.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.137+0000 7fdc64ff9640 1 -- 172.21.15.152:0/3974354404 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fdc50018a30 con 0x7fdc68101400 2024-09-11T22:51:32.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.138+0000 7fdc64ff9640 1 --2- 172.21.15.152:0/3974354404 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fdc540a2c90 0x7fdc540a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:32.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.138+0000 7fdc66ffd640 1 --2- 172.21.15.152:0/3974354404 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fdc540a2c90 0x7fdc540a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:32.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.138+0000 7fdc64ff9640 1 -- 172.21.15.152:0/3974354404 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fdc500cf520 con 0x7fdc68101400 2024-09-11T22:51:32.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.138+0000 7fdc64ff9640 1 --2- 172.21.15.152:0/3974354404 >> [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155] conn(0x7fdc540ac4e0 0x7fdc540ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:32.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.138+0000 7fdc64ff9640 1 -- 172.21.15.152:0/3974354404 --> [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc540aefb0 con 0x7fdc540ac4e0 2024-09-11T22:51:32.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.138+0000 7fdc64ff9640 1 -- 172.21.15.152:0/3974354404 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fdc500cfa00 con 0x7fdc68101400 2024-09-11T22:51:32.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.138+0000 7fdc66ffd640 1 --2- 172.21.15.152:0/3974354404 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fdc540a2c90 0x7fdc540a5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdc580058c0 tx=0x7fdc5800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:32.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.138+0000 7fdc677fe640 1 --2- 172.21.15.152:0/3974354404 >> [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155] conn(0x7fdc540ac4e0 0x7fdc540ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:32.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.139+0000 7fdc677fe640 1 --2- 172.21.15.152:0/3974354404 >> [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155] conn(0x7fdc540ac4e0 0x7fdc540ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:32.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.140+0000 7fdc64ff9640 1 -- 172.21.15.152:0/3974354404 <== osd.1 v2:172.21.15.191:6800/354467155 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc540aefb0 con 0x7fdc540ac4e0 2024-09-11T22:51:32.157 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:31 smithi152 ceph-mon[31181]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-11T22:51:32.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.154+0000 7fdc6d557640 1 -- 172.21.15.152:0/3974354404 --> [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdc681021a0 con 0x7fdc540ac4e0 2024-09-11T22:51:32.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.155+0000 7fdc64ff9640 1 -- 172.21.15.152:0/3974354404 <== osd.1 v2:172.21.15.191:6800/354467155 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdc681021a0 con 0x7fdc540ac4e0 2024-09-11T22:51:32.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.155+0000 7fdc6d557640 1 -- 172.21.15.152:0/3974354404 >> [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155] conn(0x7fdc540ac4e0 msgr2=0x7fdc540ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:32.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.155+0000 7fdc6d557640 1 --2- 172.21.15.152:0/3974354404 >> [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155] conn(0x7fdc540ac4e0 0x7fdc540ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:32.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.155+0000 7fdc6d557640 1 -- 172.21.15.152:0/3974354404 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fdc540a2c90 msgr2=0x7fdc540a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:32.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.155+0000 7fdc6d557640 1 --2- 172.21.15.152:0/3974354404 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fdc540a2c90 0x7fdc540a5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdc580058c0 tx=0x7fdc5800a2d0 comp rx=0 tx=0).stop 2024-09-11T22:51:32.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.155+0000 7fdc6d557640 1 -- 172.21.15.152:0/3974354404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc68101400 msgr2=0x7fdc681977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:32.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.155+0000 7fdc6d557640 1 --2- 172.21.15.152:0/3974354404 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc68101400 0x7fdc681977e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c920 tx=0x7fdc5000cde0 comp rx=0 tx=0).stop 2024-09-11T22:51:32.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.155+0000 7fdc677fe640 1 -- 172.21.15.152:0/3974354404 reap_dead start 2024-09-11T22:51:32.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.156+0000 7fdc6d557640 1 -- 172.21.15.152:0/3974354404 shutdown_connections 2024-09-11T22:51:32.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.156+0000 7fdc6d557640 1 -- 172.21.15.152:0/3974354404 >> 172.21.15.152:0/3974354404 conn(0x7fdc680fc780 msgr2=0x7fdc680fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:32.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.156+0000 7fdc6d557640 1 -- 172.21.15.152:0/3974354404 shutdown_connections 2024-09-11T22:51:32.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.156+0000 7fdc6d557640 1 -- 172.21.15.152:0/3974354404 wait complete. 2024-09-11T22:51:32.168 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:32.168 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:32.168 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:32.169 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:32.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:31 smithi191 ceph-mon[35211]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-11T22:51:32.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:31 smithi204 ceph-mon[35454]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-11T22:51:32.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.671+0000 7f0b94692640 1 -- 172.21.15.152:0/1821447401 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b8c106f10 msgr2=0x7f0b8c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:32.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.671+0000 7f0b94692640 1 --2- 172.21.15.152:0/1821447401 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b8c106f10 0x7f0b8c10ea90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0b84009ca0 tx=0x7f0b8402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:51:32.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.671+0000 7f0b94692640 1 -- 172.21.15.152:0/1821447401 shutdown_connections 2024-09-11T22:51:32.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.671+0000 7f0b94692640 1 --2- 172.21.15.152:0/1821447401 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b8c106f10 0x7f0b8c10ea90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:32.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.671+0000 7f0b94692640 1 --2- 172.21.15.152:0/1821447401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b8c102790 0x7f0b8c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:32.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.671+0000 7f0b94692640 1 --2- 172.21.15.152:0/1821447401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b8c101de0 0x7f0b8c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:32.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.671+0000 7f0b94692640 1 -- 172.21.15.152:0/1821447401 >> 172.21.15.152:0/1821447401 conn(0x7f0b8c0fd8c0 msgr2=0x7f0b8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:32.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.671+0000 7f0b94692640 1 -- 172.21.15.152:0/1821447401 shutdown_connections 2024-09-11T22:51:32.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.671+0000 7f0b94692640 1 -- 172.21.15.152:0/1821447401 wait complete. 2024-09-11T22:51:32.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.671+0000 7f0b94692640 1 Processor -- start 2024-09-11T22:51:32.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.671+0000 7f0b94692640 1 -- start start 2024-09-11T22:51:32.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b94692640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b8c101de0 0x7f0b8c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:32.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b94692640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b8c102790 0x7f0b8c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:32.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b94692640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b8c106f10 0x7f0b8c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:32.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b94692640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c112010 con 0x7f0b8c101de0 2024-09-11T22:51:32.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b94692640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c111e90 con 0x7f0b8c102790 2024-09-11T22:51:32.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b94692640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c112190 con 0x7f0b8c106f10 2024-09-11T22:51:32.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b9242e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b8c101de0 0x7f0b8c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:32.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b9242e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b8c101de0 0x7f0b8c198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33856/0 (socket says 172.21.15.152:33856) 2024-09-11T22:51:32.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b9242e640 1 -- 172.21.15.152:0/2157520197 learned_addr learned my addr 172.21.15.152:0/2157520197 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:32.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b92c2f640 1 --2- 172.21.15.152:0/2157520197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b8c106f10 0x7f0b8c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:32.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b91c2d640 1 --2- 172.21.15.152:0/2157520197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b8c102790 0x7f0b8c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:32.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b9242e640 1 -- 172.21.15.152:0/2157520197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b8c106f10 msgr2=0x7f0b8c19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:32.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b9242e640 1 --2- 172.21.15.152:0/2157520197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b8c106f10 0x7f0b8c19ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:32.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b9242e640 1 -- 172.21.15.152:0/2157520197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b8c102790 msgr2=0x7f0b8c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:32.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b9242e640 1 --2- 172.21.15.152:0/2157520197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b8c102790 0x7f0b8c198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:32.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b9242e640 1 -- 172.21.15.152:0/2157520197 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c19d540 con 0x7f0b8c101de0 2024-09-11T22:51:32.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b91c2d640 1 --2- 172.21.15.152:0/2157520197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b8c102790 0x7f0b8c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:51:32.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.672+0000 7f0b9242e640 1 --2- 172.21.15.152:0/2157520197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b8c101de0 0x7f0b8c198590 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800b4a0 tx=0x7f0b8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:32.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.673+0000 7f0b837fe640 1 -- 172.21.15.152:0/2157520197 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88014070 con 0x7f0b8c101de0 2024-09-11T22:51:32.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.673+0000 7f0b94692640 1 -- 172.21.15.152:0/2157520197 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8c19d830 con 0x7f0b8c101de0 2024-09-11T22:51:32.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.673+0000 7f0b837fe640 1 -- 172.21.15.152:0/2157520197 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b88010af0 con 0x7f0b8c101de0 2024-09-11T22:51:32.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.673+0000 7f0b837fe640 1 -- 172.21.15.152:0/2157520197 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88017d10 con 0x7f0b8c101de0 2024-09-11T22:51:32.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.673+0000 7f0b94692640 1 -- 172.21.15.152:0/2157520197 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b8c1a5110 con 0x7f0b8c101de0 2024-09-11T22:51:32.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.674+0000 7f0b94692640 1 -- 172.21.15.152:0/2157520197 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b8c062980 con 0x7f0b8c101de0 2024-09-11T22:51:32.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.675+0000 7f0b837fe640 1 -- 172.21.15.152:0/2157520197 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0b88017070 con 0x7f0b8c101de0 2024-09-11T22:51:32.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.675+0000 7f0b837fe640 1 --2- 172.21.15.152:0/2157520197 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0b780a2c90 0x7f0b780a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:32.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.676+0000 7f0b91c2d640 1 --2- 172.21.15.152:0/2157520197 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0b780a2c90 0x7f0b780a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:32.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.676+0000 7f0b837fe640 1 -- 172.21.15.152:0/2157520197 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f0b880ce780 con 0x7f0b8c101de0 2024-09-11T22:51:32.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.676+0000 7f0b837fe640 1 --2- 172.21.15.152:0/2157520197 >> [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236] conn(0x7f0b780ac4e0 0x7f0b780ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:32.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.676+0000 7f0b837fe640 1 -- 172.21.15.152:0/2157520197 --> [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b8801ae50 con 0x7f0b780ac4e0 2024-09-11T22:51:32.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.676+0000 7f0b837fe640 1 -- 172.21.15.152:0/2157520197 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f0b880ceb70 con 0x7f0b8c101de0 2024-09-11T22:51:32.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.676+0000 7f0b91c2d640 1 --2- 172.21.15.152:0/2157520197 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0b780a2c90 0x7f0b780a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0fdd50 tx=0x7f0b7c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:32.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.676+0000 7f0b92c2f640 1 --2- 172.21.15.152:0/2157520197 >> [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236] conn(0x7f0b780ac4e0 0x7f0b780ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:32.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.676+0000 7f0b92c2f640 1 --2- 172.21.15.152:0/2157520197 >> [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236] conn(0x7f0b780ac4e0 0x7f0b780ae900 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:32.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.677+0000 7f0b837fe640 1 -- 172.21.15.152:0/2157520197 <== osd.11 v2:172.21.15.204:6824/1709433236 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b8801ae50 con 0x7f0b780ac4e0 2024-09-11T22:51:32.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.691+0000 7f0b94692640 1 -- 172.21.15.152:0/2157520197 --> [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b8c062e00 con 0x7f0b780ac4e0 2024-09-11T22:51:32.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.691+0000 7f0b837fe640 1 -- 172.21.15.152:0/2157520197 <== osd.11 v2:172.21.15.204:6824/1709433236 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0b8c062e00 con 0x7f0b780ac4e0 2024-09-11T22:51:32.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.692+0000 7f0b94692640 1 -- 172.21.15.152:0/2157520197 >> [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236] conn(0x7f0b780ac4e0 msgr2=0x7f0b780ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:32.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.692+0000 7f0b94692640 1 --2- 172.21.15.152:0/2157520197 >> [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236] conn(0x7f0b780ac4e0 0x7f0b780ae900 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:32.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.692+0000 7f0b94692640 1 -- 172.21.15.152:0/2157520197 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0b780a2c90 msgr2=0x7f0b780a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:32.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.692+0000 7f0b94692640 1 --2- 172.21.15.152:0/2157520197 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f0b780a2c90 0x7f0b780a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0fdd50 tx=0x7f0b7c008040 comp rx=0 tx=0).stop 2024-09-11T22:51:32.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.692+0000 7f0b94692640 1 -- 172.21.15.152:0/2157520197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b8c101de0 msgr2=0x7f0b8c198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:32.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.692+0000 7f0b94692640 1 --2- 172.21.15.152:0/2157520197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0b8c101de0 0x7f0b8c198590 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0b8800b4a0 tx=0x7f0b8800b960 comp rx=0 tx=0).stop 2024-09-11T22:51:32.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.692+0000 7f0b92c2f640 1 -- 172.21.15.152:0/2157520197 reap_dead start 2024-09-11T22:51:32.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.692+0000 7f0b94692640 1 -- 172.21.15.152:0/2157520197 shutdown_connections 2024-09-11T22:51:32.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.692+0000 7f0b94692640 1 -- 172.21.15.152:0/2157520197 >> 172.21.15.152:0/2157520197 conn(0x7f0b8c0fd8c0 msgr2=0x7f0b8c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:32.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.692+0000 7f0b94692640 1 -- 172.21.15.152:0/2157520197 shutdown_connections 2024-09-11T22:51:32.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:32.692+0000 7f0b94692640 1 -- 172.21.15.152:0/2157520197 wait complete. 2024-09-11T22:51:34.106 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:33 smithi152 ceph-mon[31181]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-11T22:51:34.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.105+0000 7f4a55869640 1 -- 172.21.15.152:0/3678688465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a50100e20 msgr2=0x7f4a50101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:34.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.105+0000 7f4a55869640 1 --2- 172.21.15.152:0/3678688465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a50100e20 0x7f4a50101200 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009ca0 tx=0x7f4a4802a950 comp rx=0 tx=0).stop 2024-09-11T22:51:34.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.105+0000 7f4a55869640 1 -- 172.21.15.152:0/3678688465 shutdown_connections 2024-09-11T22:51:34.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.105+0000 7f4a55869640 1 --2- 172.21.15.152:0/3678688465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a50102ef0 0x7f4a5010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:34.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.105+0000 7f4a55869640 1 --2- 172.21.15.152:0/3678688465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a50102570 0x7f4a501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:34.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.105+0000 7f4a55869640 1 --2- 172.21.15.152:0/3678688465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a50100e20 0x7f4a50101200 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:34.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.105+0000 7f4a55869640 1 -- 172.21.15.152:0/3678688465 >> 172.21.15.152:0/3678688465 conn(0x7f4a500fc7d0 msgr2=0x7f4a500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:34.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.105+0000 7f4a55869640 1 -- 172.21.15.152:0/3678688465 shutdown_connections 2024-09-11T22:51:34.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.105+0000 7f4a55869640 1 -- 172.21.15.152:0/3678688465 wait complete. 2024-09-11T22:51:34.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.105+0000 7f4a55869640 1 Processor -- start 2024-09-11T22:51:34.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a55869640 1 -- start start 2024-09-11T22:51:34.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a55869640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a50100e20 0x7f4a5010e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:34.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a55869640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a50102570 0x7f4a5010ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:34.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a55869640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a50102ef0 0x7f4a5010ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:34.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a55869640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4a501120a0 con 0x7f4a50102ef0 2024-09-11T22:51:34.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a55869640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4a50111f20 con 0x7f4a50100e20 2024-09-11T22:51:34.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a55869640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4a50112220 con 0x7f4a50102570 2024-09-11T22:51:34.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a4f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a50102ef0 0x7f4a5010ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:34.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a4f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a50102ef0 0x7f4a5010ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33868/0 (socket says 172.21.15.152:33868) 2024-09-11T22:51:34.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a4f7fe640 1 -- 172.21.15.152:0/1111102419 learned_addr learned my addr 172.21.15.152:0/1111102419 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:34.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a4e7fc640 1 --2- 172.21.15.152:0/1111102419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a50102570 0x7f4a5010ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:34.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a4effd640 1 --2- 172.21.15.152:0/1111102419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a50100e20 0x7f4a5010e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:34.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a4f7fe640 1 -- 172.21.15.152:0/1111102419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a50102570 msgr2=0x7f4a5010ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:34.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a4f7fe640 1 --2- 172.21.15.152:0/1111102419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a50102570 0x7f4a5010ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:34.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a4f7fe640 1 -- 172.21.15.152:0/1111102419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a50100e20 msgr2=0x7f4a5010e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:34.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a4f7fe640 1 --2- 172.21.15.152:0/1111102419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a50100e20 0x7f4a5010e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:34.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a4f7fe640 1 -- 172.21.15.152:0/1111102419 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a501a69c0 con 0x7f4a50102ef0 2024-09-11T22:51:34.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.106+0000 7f4a4f7fe640 1 --2- 172.21.15.152:0/1111102419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a50102ef0 0x7f4a5010ef50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000c970 tx=0x7f4a4000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:34.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.107+0000 7f4a54867640 1 -- 172.21.15.152:0/1111102419 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40007bb0 con 0x7f4a50102ef0 2024-09-11T22:51:34.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.107+0000 7f4a54867640 1 -- 172.21.15.152:0/1111102419 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a40007d50 con 0x7f4a50102ef0 2024-09-11T22:51:34.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.107+0000 7f4a55869640 1 -- 172.21.15.152:0/1111102419 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a501a6cb0 con 0x7f4a50102ef0 2024-09-11T22:51:34.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.107+0000 7f4a54867640 1 -- 172.21.15.152:0/1111102419 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40018800 con 0x7f4a50102ef0 2024-09-11T22:51:34.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.107+0000 7f4a55869640 1 -- 172.21.15.152:0/1111102419 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a501a71a0 con 0x7f4a50102ef0 2024-09-11T22:51:34.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.108+0000 7f4a55869640 1 -- 172.21.15.152:0/1111102419 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a50107280 con 0x7f4a50102ef0 2024-09-11T22:51:34.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.109+0000 7f4a54867640 1 -- 172.21.15.152:0/1111102419 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4a4000fa50 con 0x7f4a50102ef0 2024-09-11T22:51:34.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.110+0000 7f4a54867640 1 --2- 172.21.15.152:0/1111102419 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4a3c0a2c10 0x7f4a3c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:34.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.110+0000 7f4a4effd640 1 --2- 172.21.15.152:0/1111102419 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4a3c0a2c10 0x7f4a3c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:34.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.110+0000 7f4a54867640 1 -- 172.21.15.152:0/1111102419 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f4a400cf480 con 0x7f4a50102ef0 2024-09-11T22:51:34.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.110+0000 7f4a54867640 1 --2- 172.21.15.152:0/1111102419 >> [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503] conn(0x7f4a3c0ac460 0x7f4a3c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:34.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.110+0000 7f4a54867640 1 -- 172.21.15.152:0/1111102419 --> [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a4000eec0 con 0x7f4a3c0ac460 2024-09-11T22:51:34.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.110+0000 7f4a54867640 1 -- 172.21.15.152:0/1111102419 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f4a400cf960 con 0x7f4a50102ef0 2024-09-11T22:51:34.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.110+0000 7f4a4effd640 1 --2- 172.21.15.152:0/1111102419 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4a3c0a2c10 0x7f4a3c0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4a480093d0 tx=0x7f4a48009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:34.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.110+0000 7f4a4e7fc640 1 --2- 172.21.15.152:0/1111102419 >> [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503] conn(0x7f4a3c0ac460 0x7f4a3c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:34.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.111+0000 7f4a4e7fc640 1 --2- 172.21.15.152:0/1111102419 >> [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503] conn(0x7f4a3c0ac460 0x7f4a3c0ae880 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:34.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.112+0000 7f4a54867640 1 -- 172.21.15.152:0/1111102419 <== osd.10 v2:172.21.15.191:6824/3695294503 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a4000eec0 con 0x7f4a3c0ac460 2024-09-11T22:51:34.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.125+0000 7f4a55869640 1 -- 172.21.15.152:0/1111102419 --> [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a501021b0 con 0x7f4a3c0ac460 2024-09-11T22:51:34.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.126+0000 7f4a54867640 1 -- 172.21.15.152:0/1111102419 <== osd.10 v2:172.21.15.191:6824/3695294503 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4a501021b0 con 0x7f4a3c0ac460 2024-09-11T22:51:34.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.126+0000 7f4a55869640 1 -- 172.21.15.152:0/1111102419 >> [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503] conn(0x7f4a3c0ac460 msgr2=0x7f4a3c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:34.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.126+0000 7f4a55869640 1 --2- 172.21.15.152:0/1111102419 >> [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503] conn(0x7f4a3c0ac460 0x7f4a3c0ae880 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:34.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.126+0000 7f4a55869640 1 -- 172.21.15.152:0/1111102419 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4a3c0a2c10 msgr2=0x7f4a3c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:34.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.126+0000 7f4a55869640 1 --2- 172.21.15.152:0/1111102419 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4a3c0a2c10 0x7f4a3c0a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4a480093d0 tx=0x7f4a48009360 comp rx=0 tx=0).stop 2024-09-11T22:51:34.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.126+0000 7f4a55869640 1 -- 172.21.15.152:0/1111102419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a50102ef0 msgr2=0x7f4a5010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:34.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.126+0000 7f4a55869640 1 --2- 172.21.15.152:0/1111102419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a50102ef0 0x7f4a5010ef50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000c970 tx=0x7f4a4000ce30 comp rx=0 tx=0).stop 2024-09-11T22:51:34.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.126+0000 7f4a4f7fe640 1 -- 172.21.15.152:0/1111102419 reap_dead start 2024-09-11T22:51:34.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.126+0000 7f4a55869640 1 -- 172.21.15.152:0/1111102419 shutdown_connections 2024-09-11T22:51:34.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.126+0000 7f4a55869640 1 -- 172.21.15.152:0/1111102419 >> 172.21.15.152:0/1111102419 conn(0x7f4a500fc7d0 msgr2=0x7f4a500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:34.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.126+0000 7f4a55869640 1 -- 172.21.15.152:0/1111102419 shutdown_connections 2024-09-11T22:51:34.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.126+0000 7f4a55869640 1 -- 172.21.15.152:0/1111102419 wait complete. 2024-09-11T22:51:34.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:33 smithi191 ceph-mon[35211]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-11T22:51:34.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:33 smithi204 ceph-mon[35454]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.0 KiB/s wr, 13 op/s 2024-09-11T22:51:34.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.667+0000 7f33333ab640 1 -- 172.21.15.152:0/1762846995 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f332c106a80 msgr2=0x7f332c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:34.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.667+0000 7f33333ab640 1 --2- 172.21.15.152:0/1762846995 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f332c106a80 0x7f332c106e60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3328009930 tx=0x7f332802a420 comp rx=0 tx=0).stop 2024-09-11T22:51:34.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.667+0000 7f33333ab640 1 -- 172.21.15.152:0/1762846995 shutdown_connections 2024-09-11T22:51:34.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.667+0000 7f33333ab640 1 --2- 172.21.15.152:0/1762846995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f332c101420 0x7f332c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:34.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.667+0000 7f33333ab640 1 --2- 172.21.15.152:0/1762846995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f332c100a80 0x7f332c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:34.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.667+0000 7f33333ab640 1 --2- 172.21.15.152:0/1762846995 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f332c106a80 0x7f332c106e60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:34.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.667+0000 7f33333ab640 1 -- 172.21.15.152:0/1762846995 >> 172.21.15.152:0/1762846995 conn(0x7f332c0fc7a0 msgr2=0x7f332c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:34.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.667+0000 7f33333ab640 1 -- 172.21.15.152:0/1762846995 shutdown_connections 2024-09-11T22:51:34.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.667+0000 7f33333ab640 1 -- 172.21.15.152:0/1762846995 wait complete. 2024-09-11T22:51:34.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f33333ab640 1 Processor -- start 2024-09-11T22:51:34.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f33333ab640 1 -- start start 2024-09-11T22:51:34.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f33333ab640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f332c100a80 0x7f332c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:34.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f33333ab640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f332c101420 0x7f332c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:34.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f33333ab640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f332c106a80 0x7f332c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:34.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f33333ab640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f332c110e80 con 0x7f332c106a80 2024-09-11T22:51:34.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f33333ab640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f332c110d00 con 0x7f332c100a80 2024-09-11T22:51:34.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f33333ab640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f332c111000 con 0x7f332c101420 2024-09-11T22:51:34.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f3331948640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f332c106a80 0x7f332c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:34.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f3331948640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f332c106a80 0x7f332c19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33884/0 (socket says 172.21.15.152:33884) 2024-09-11T22:51:34.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f3331948640 1 -- 172.21.15.152:0/4041383430 learned_addr learned my addr 172.21.15.152:0/4041383430 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:34.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f3330946640 1 --2- 172.21.15.152:0/4041383430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f332c101420 0x7f332c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:34.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f3331147640 1 --2- 172.21.15.152:0/4041383430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f332c100a80 0x7f332c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:34.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f3331948640 1 -- 172.21.15.152:0/4041383430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f332c101420 msgr2=0x7f332c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:34.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f3331948640 1 --2- 172.21.15.152:0/4041383430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f332c101420 0x7f332c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:34.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f3331948640 1 -- 172.21.15.152:0/4041383430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f332c100a80 msgr2=0x7f332c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:34.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f3331948640 1 --2- 172.21.15.152:0/4041383430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f332c100a80 0x7f332c1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:34.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.668+0000 7f3331948640 1 -- 172.21.15.152:0/4041383430 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332c19d360 con 0x7f332c106a80 2024-09-11T22:51:34.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.669+0000 7f3331948640 1 --2- 172.21.15.152:0/4041383430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f332c106a80 0x7f332c19cc80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f332400eaa0 tx=0x7f332400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:34.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.669+0000 7f331e7fc640 1 -- 172.21.15.152:0/4041383430 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332400cdb0 con 0x7f332c106a80 2024-09-11T22:51:34.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.669+0000 7f331e7fc640 1 -- 172.21.15.152:0/4041383430 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3324005020 con 0x7f332c106a80 2024-09-11T22:51:34.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.669+0000 7f331e7fc640 1 -- 172.21.15.152:0/4041383430 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33240116b0 con 0x7f332c106a80 2024-09-11T22:51:34.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.669+0000 7f33333ab640 1 -- 172.21.15.152:0/4041383430 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332c19d650 con 0x7f332c106a80 2024-09-11T22:51:34.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.669+0000 7f33333ab640 1 -- 172.21.15.152:0/4041383430 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f332c1a4f30 con 0x7f332c106a80 2024-09-11T22:51:34.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.670+0000 7f33333ab640 1 -- 172.21.15.152:0/4041383430 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f332c0fd070 con 0x7f332c106a80 2024-09-11T22:51:34.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.671+0000 7f331e7fc640 1 -- 172.21.15.152:0/4041383430 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3324005560 con 0x7f332c106a80 2024-09-11T22:51:34.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.671+0000 7f331e7fc640 1 --2- 172.21.15.152:0/4041383430 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f33180a2c10 0x7f33180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:34.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.671+0000 7f3331147640 1 --2- 172.21.15.152:0/4041383430 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f33180a2c10 0x7f33180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:34.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.671+0000 7f331e7fc640 1 -- 172.21.15.152:0/4041383430 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f33240cf620 con 0x7f332c106a80 2024-09-11T22:51:34.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.672+0000 7f331e7fc640 1 --2- 172.21.15.152:0/4041383430 >> [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158] conn(0x7f33180ac450 0x7f33180ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:34.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.672+0000 7f331e7fc640 1 -- 172.21.15.152:0/4041383430 --> [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3324014ec0 con 0x7f33180ac450 2024-09-11T22:51:34.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.672+0000 7f331e7fc640 1 -- 172.21.15.152:0/4041383430 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f33240cfb00 con 0x7f332c106a80 2024-09-11T22:51:34.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.672+0000 7f3331147640 1 --2- 172.21.15.152:0/4041383430 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f33180a2c10 0x7f33180a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f33280096c0 tx=0x7f3328000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:34.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.672+0000 7f3330946640 1 --2- 172.21.15.152:0/4041383430 >> [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158] conn(0x7f33180ac450 0x7f33180ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:34.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.672+0000 7f3330946640 1 --2- 172.21.15.152:0/4041383430 >> [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158] conn(0x7f33180ac450 0x7f33180ae870 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:34.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.673+0000 7f331e7fc640 1 -- 172.21.15.152:0/4041383430 <== osd.5 v2:172.21.15.204:6808/4283562158 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3324014ec0 con 0x7f33180ac450 2024-09-11T22:51:34.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.686+0000 7f33333ab640 1 -- 172.21.15.152:0/4041383430 --> [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f332c1021c0 con 0x7f33180ac450 2024-09-11T22:51:34.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.687+0000 7f331e7fc640 1 -- 172.21.15.152:0/4041383430 <== osd.5 v2:172.21.15.204:6808/4283562158 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f332c1021c0 con 0x7f33180ac450 2024-09-11T22:51:34.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.687+0000 7f33333ab640 1 -- 172.21.15.152:0/4041383430 >> [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158] conn(0x7f33180ac450 msgr2=0x7f33180ae870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:34.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.687+0000 7f33333ab640 1 --2- 172.21.15.152:0/4041383430 >> [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158] conn(0x7f33180ac450 0x7f33180ae870 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:34.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.687+0000 7f33333ab640 1 -- 172.21.15.152:0/4041383430 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f33180a2c10 msgr2=0x7f33180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:34.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.687+0000 7f33333ab640 1 --2- 172.21.15.152:0/4041383430 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f33180a2c10 0x7f33180a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f33280096c0 tx=0x7f3328000bc0 comp rx=0 tx=0).stop 2024-09-11T22:51:34.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.687+0000 7f33333ab640 1 -- 172.21.15.152:0/4041383430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f332c106a80 msgr2=0x7f332c19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:34.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.687+0000 7f33333ab640 1 --2- 172.21.15.152:0/4041383430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f332c106a80 0x7f332c19cc80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f332400eaa0 tx=0x7f332400ef60 comp rx=0 tx=0).stop 2024-09-11T22:51:34.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.687+0000 7f3331948640 1 -- 172.21.15.152:0/4041383430 reap_dead start 2024-09-11T22:51:34.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.687+0000 7f33333ab640 1 -- 172.21.15.152:0/4041383430 shutdown_connections 2024-09-11T22:51:34.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.687+0000 7f33333ab640 1 -- 172.21.15.152:0/4041383430 >> 172.21.15.152:0/4041383430 conn(0x7f332c0fc7a0 msgr2=0x7f332c1047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:34.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.687+0000 7f33333ab640 1 -- 172.21.15.152:0/4041383430 shutdown_connections 2024-09-11T22:51:34.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:34.687+0000 7f33333ab640 1 -- 172.21.15.152:0/4041383430 wait complete. 2024-09-11T22:51:35.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.347+0000 7fa9eafb2640 1 -- 172.21.15.152:0/603474763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e4101de0 msgr2=0x7fa9e41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:35.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.347+0000 7fa9eafb2640 1 --2- 172.21.15.152:0/603474763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e4101de0 0x7fa9e41021c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009900 tx=0x7fa9e002a410 comp rx=0 tx=0).stop 2024-09-11T22:51:35.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.347+0000 7fa9eafb2640 1 -- 172.21.15.152:0/603474763 shutdown_connections 2024-09-11T22:51:35.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.347+0000 7fa9eafb2640 1 --2- 172.21.15.152:0/603474763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9e4106f10 0x7fa9e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:35.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.347+0000 7fa9eafb2640 1 --2- 172.21.15.152:0/603474763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9e4102790 0x7fa9e41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:35.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.347+0000 7fa9eafb2640 1 --2- 172.21.15.152:0/603474763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e4101de0 0x7fa9e41021c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:35.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.347+0000 7fa9eafb2640 1 -- 172.21.15.152:0/603474763 >> 172.21.15.152:0/603474763 conn(0x7fa9e40fd8c0 msgr2=0x7fa9e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:35.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.347+0000 7fa9eafb2640 1 -- 172.21.15.152:0/603474763 shutdown_connections 2024-09-11T22:51:35.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.348+0000 7fa9eafb2640 1 -- 172.21.15.152:0/603474763 wait complete. 2024-09-11T22:51:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.348+0000 7fa9eafb2640 1 Processor -- start 2024-09-11T22:51:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.348+0000 7fa9eafb2640 1 -- start start 2024-09-11T22:51:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.348+0000 7fa9eafb2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9e4101de0 0x7fa9e41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.348+0000 7fa9eafb2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9e4102790 0x7fa9e4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.348+0000 7fa9eafb2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e4106f10 0x7fa9e419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.348+0000 7fa9eafb2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4111f70 con 0x7fa9e4106f10 2024-09-11T22:51:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.348+0000 7fa9eafb2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4111df0 con 0x7fa9e4102790 2024-09-11T22:51:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.348+0000 7fa9eafb2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e41120f0 con 0x7fa9e4101de0 2024-09-11T22:51:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.348+0000 7fa9e954f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e4106f10 0x7fa9e419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.349+0000 7fa9e8d4e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9e4101de0 0x7fa9e41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.349+0000 7fa9e954f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e4106f10 0x7fa9e419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33912/0 (socket says 172.21.15.152:33912) 2024-09-11T22:51:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.349+0000 7fa9e954f640 1 -- 172.21.15.152:0/380390425 learned_addr learned my addr 172.21.15.152:0/380390425 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:35.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.349+0000 7fa9dbfff640 1 --2- 172.21.15.152:0/380390425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9e4102790 0x7fa9e4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:35.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.349+0000 7fa9e954f640 1 -- 172.21.15.152:0/380390425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9e4101de0 msgr2=0x7fa9e41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:35.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.349+0000 7fa9e954f640 1 --2- 172.21.15.152:0/380390425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9e4101de0 0x7fa9e41983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:35.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.349+0000 7fa9e954f640 1 -- 172.21.15.152:0/380390425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9e4102790 msgr2=0x7fa9e4198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:35.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.349+0000 7fa9e954f640 1 --2- 172.21.15.152:0/380390425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9e4102790 0x7fa9e4198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:35.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.349+0000 7fa9e954f640 1 -- 172.21.15.152:0/380390425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e419d380 con 0x7fa9e4106f10 2024-09-11T22:51:35.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.349+0000 7fa9dbfff640 1 --2- 172.21.15.152:0/380390425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9e4102790 0x7fa9e4198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:51:35.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.349+0000 7fa9e954f640 1 --2- 172.21.15.152:0/380390425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e4106f10 0x7fa9e419cca0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00cce0 tx=0x7fa9dc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:35.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.349+0000 7fa9d9ffb640 1 -- 172.21.15.152:0/380390425 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc00f450 con 0x7fa9e4106f10 2024-09-11T22:51:35.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.349+0000 7fa9eafb2640 1 -- 172.21.15.152:0/380390425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e419d670 con 0x7fa9e4106f10 2024-09-11T22:51:35.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.349+0000 7fa9d9ffb640 1 -- 172.21.15.152:0/380390425 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9dc00fad0 con 0x7fa9e4106f10 2024-09-11T22:51:35.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.349+0000 7fa9d9ffb640 1 -- 172.21.15.152:0/380390425 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc018c10 con 0x7fa9e4106f10 2024-09-11T22:51:35.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.349+0000 7fa9eafb2640 1 -- 172.21.15.152:0/380390425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e41a4f50 con 0x7fa9e4106f10 2024-09-11T22:51:35.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.350+0000 7fa9eafb2640 1 -- 172.21.15.152:0/380390425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa9e4062980 con 0x7fa9e4106f10 2024-09-11T22:51:35.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.351+0000 7fa9d9ffb640 1 -- 172.21.15.152:0/380390425 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa9dc00f5f0 con 0x7fa9e4106f10 2024-09-11T22:51:35.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.352+0000 7fa9d9ffb640 1 --2- 172.21.15.152:0/380390425 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa9cc0a2c10 0x7fa9cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:35.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.352+0000 7fa9e8d4e640 1 --2- 172.21.15.152:0/380390425 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa9cc0a2c10 0x7fa9cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:35.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.352+0000 7fa9d9ffb640 1 -- 172.21.15.152:0/380390425 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fa9dc0cf650 con 0x7fa9e4106f10 2024-09-11T22:51:35.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.352+0000 7fa9d9ffb640 1 --2- 172.21.15.152:0/380390425 >> [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211] conn(0x7fa9cc0ac460 0x7fa9cc0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:35.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.352+0000 7fa9d9ffb640 1 -- 172.21.15.152:0/380390425 --> [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa9dc015e70 con 0x7fa9cc0ac460 2024-09-11T22:51:35.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.352+0000 7fa9d9ffb640 1 -- 172.21.15.152:0/380390425 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fa9dc0d6050 con 0x7fa9e4106f10 2024-09-11T22:51:35.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.352+0000 7fa9e8d4e640 1 --2- 172.21.15.152:0/380390425 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa9cc0a2c10 0x7fa9cc0a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009230 tx=0x7fa9e00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:35.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.353+0000 7fa9dbfff640 1 --2- 172.21.15.152:0/380390425 >> [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211] conn(0x7fa9cc0ac460 0x7fa9cc0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:35.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.353+0000 7fa9dbfff640 1 --2- 172.21.15.152:0/380390425 >> [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211] conn(0x7fa9cc0ac460 0x7fa9cc0ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:35.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.354+0000 7fa9d9ffb640 1 -- 172.21.15.152:0/380390425 <== osd.2 v2:172.21.15.204:6800/3848396211 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa9dc015e70 con 0x7fa9cc0ac460 2024-09-11T22:51:35.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.367+0000 7fa9eafb2640 1 -- 172.21.15.152:0/380390425 --> [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa9e4062e00 con 0x7fa9cc0ac460 2024-09-11T22:51:35.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.368+0000 7fa9d9ffb640 1 -- 172.21.15.152:0/380390425 <== osd.2 v2:172.21.15.204:6800/3848396211 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa9e4062e00 con 0x7fa9cc0ac460 2024-09-11T22:51:35.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.368+0000 7fa9eafb2640 1 -- 172.21.15.152:0/380390425 >> [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211] conn(0x7fa9cc0ac460 msgr2=0x7fa9cc0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:35.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.368+0000 7fa9eafb2640 1 --2- 172.21.15.152:0/380390425 >> [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211] conn(0x7fa9cc0ac460 0x7fa9cc0ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:35.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.368+0000 7fa9eafb2640 1 -- 172.21.15.152:0/380390425 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa9cc0a2c10 msgr2=0x7fa9cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:35.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.368+0000 7fa9eafb2640 1 --2- 172.21.15.152:0/380390425 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa9cc0a2c10 0x7fa9cc0a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0009230 tx=0x7fa9e00058d0 comp rx=0 tx=0).stop 2024-09-11T22:51:35.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.368+0000 7fa9eafb2640 1 -- 172.21.15.152:0/380390425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e4106f10 msgr2=0x7fa9e419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:35.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.368+0000 7fa9eafb2640 1 --2- 172.21.15.152:0/380390425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9e4106f10 0x7fa9e419cca0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00cce0 tx=0x7fa9dc00a620 comp rx=0 tx=0).stop 2024-09-11T22:51:35.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.368+0000 7fa9e954f640 1 -- 172.21.15.152:0/380390425 reap_dead start 2024-09-11T22:51:35.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.368+0000 7fa9eafb2640 1 -- 172.21.15.152:0/380390425 shutdown_connections 2024-09-11T22:51:35.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.368+0000 7fa9eafb2640 1 -- 172.21.15.152:0/380390425 >> 172.21.15.152:0/380390425 conn(0x7fa9e40fd8c0 msgr2=0x7fa9e40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:35.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.369+0000 7fa9eafb2640 1 -- 172.21.15.152:0/380390425 shutdown_connections 2024-09-11T22:51:35.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:35.369+0000 7fa9eafb2640 1 -- 172.21.15.152:0/380390425 wait complete. 2024-09-11T22:51:35.979 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:35 smithi152 ceph-mon[31181]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-11T22:51:36.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:35 smithi191 ceph-mon[35211]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-11T22:51:36.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:35 smithi204 ceph-mon[35454]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-11T22:51:36.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.241+0000 7fbb9138b640 1 -- 172.21.15.152:0/2356148927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb8c101420 msgr2=0x7fbb8c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.241+0000 7fbb9138b640 1 --2- 172.21.15.152:0/2356148927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb8c101420 0x7fbb8c10d950 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009cd0 tx=0x7fbb8402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:51:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.241+0000 7fbb9138b640 1 -- 172.21.15.152:0/2356148927 shutdown_connections 2024-09-11T22:51:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.241+0000 7fbb9138b640 1 --2- 172.21.15.152:0/2356148927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb8c101420 0x7fbb8c10d950 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.241+0000 7fbb9138b640 1 --2- 172.21.15.152:0/2356148927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb8c100a80 0x7fbb8c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.241+0000 7fbb9138b640 1 --2- 172.21.15.152:0/2356148927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb8c106a80 0x7fbb8c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.241+0000 7fbb9138b640 1 -- 172.21.15.152:0/2356148927 >> 172.21.15.152:0/2356148927 conn(0x7fbb8c0fc7a0 msgr2=0x7fbb8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.241+0000 7fbb9138b640 1 -- 172.21.15.152:0/2356148927 shutdown_connections 2024-09-11T22:51:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.241+0000 7fbb9138b640 1 -- 172.21.15.152:0/2356148927 wait complete. 2024-09-11T22:51:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.242+0000 7fbb9138b640 1 Processor -- start 2024-09-11T22:51:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.242+0000 7fbb9138b640 1 -- start start 2024-09-11T22:51:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.242+0000 7fbb9138b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb8c100a80 0x7fbb8c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:36.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.242+0000 7fbb9138b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb8c101420 0x7fbb8c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.242+0000 7fbb9138b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb8c106a80 0x7fbb8c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.242+0000 7fbb9138b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8c110e80 con 0x7fbb8c100a80 2024-09-11T22:51:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.242+0000 7fbb9138b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8c110d00 con 0x7fbb8c106a80 2024-09-11T22:51:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.242+0000 7fbb9138b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8c111000 con 0x7fbb8c101420 2024-09-11T22:51:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.242+0000 7fbb8affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb8c100a80 0x7fbb8c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.242+0000 7fbb8affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb8c100a80 0x7fbb8c19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33922/0 (socket says 172.21.15.152:33922) 2024-09-11T22:51:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.242+0000 7fbb8affd640 1 -- 172.21.15.152:0/3369967929 learned_addr learned my addr 172.21.15.152:0/3369967929 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.242+0000 7fbb8b7fe640 1 --2- 172.21.15.152:0/3369967929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb8c106a80 0x7fbb8c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.242+0000 7fbb83fff640 1 --2- 172.21.15.152:0/3369967929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb8c101420 0x7fbb8c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.243+0000 7fbb8affd640 1 -- 172.21.15.152:0/3369967929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb8c101420 msgr2=0x7fbb8c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.243+0000 7fbb8affd640 1 --2- 172.21.15.152:0/3369967929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb8c101420 0x7fbb8c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.243+0000 7fbb8affd640 1 -- 172.21.15.152:0/3369967929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb8c106a80 msgr2=0x7fbb8c19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:36.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.243+0000 7fbb8affd640 1 --2- 172.21.15.152:0/3369967929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb8c106a80 0x7fbb8c19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:36.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.243+0000 7fbb8affd640 1 -- 172.21.15.152:0/3369967929 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8c19f550 con 0x7fbb8c100a80 2024-09-11T22:51:36.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.243+0000 7fbb8b7fe640 1 --2- 172.21.15.152:0/3369967929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb8c106a80 0x7fbb8c19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:36.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.243+0000 7fbb83fff640 1 --2- 172.21.15.152:0/3369967929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb8c101420 0x7fbb8c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:36.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.243+0000 7fbb8affd640 1 --2- 172.21.15.152:0/3369967929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb8c100a80 0x7fbb8c19a5a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00c920 tx=0x7fbb7c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:36.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.243+0000 7fbb88ff9640 1 -- 172.21.15.152:0/3369967929 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c007c70 con 0x7fbb8c100a80 2024-09-11T22:51:36.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.243+0000 7fbb88ff9640 1 -- 172.21.15.152:0/3369967929 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb7c007e10 con 0x7fbb8c100a80 2024-09-11T22:51:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.243+0000 7fbb9138b640 1 -- 172.21.15.152:0/3369967929 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb8c19f840 con 0x7fbb8c100a80 2024-09-11T22:51:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.243+0000 7fbb88ff9640 1 -- 172.21.15.152:0/3369967929 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c018890 con 0x7fbb8c100a80 2024-09-11T22:51:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.243+0000 7fbb9138b640 1 -- 172.21.15.152:0/3369967929 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb8c1a7120 con 0x7fbb8c100a80 2024-09-11T22:51:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.244+0000 7fbb9138b640 1 -- 172.21.15.152:0/3369967929 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb8c06b260 con 0x7fbb8c100a80 2024-09-11T22:51:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.245+0000 7fbb88ff9640 1 -- 172.21.15.152:0/3369967929 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fbb7c018a30 con 0x7fbb8c100a80 2024-09-11T22:51:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.246+0000 7fbb88ff9640 1 --2- 172.21.15.152:0/3369967929 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbb780a2c90 0x7fbb780a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.246+0000 7fbb83fff640 1 --2- 172.21.15.152:0/3369967929 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbb780a2c90 0x7fbb780a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.246+0000 7fbb88ff9640 1 -- 172.21.15.152:0/3369967929 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fbb7c0cf570 con 0x7fbb8c100a80 2024-09-11T22:51:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.246+0000 7fbb88ff9640 1 --2- 172.21.15.152:0/3369967929 >> [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563] conn(0x7fbb780ac4e0 0x7fbb780ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.246+0000 7fbb88ff9640 1 -- 172.21.15.152:0/3369967929 --> [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb780aefb0 con 0x7fbb780ac4e0 2024-09-11T22:51:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.246+0000 7fbb88ff9640 1 -- 172.21.15.152:0/3369967929 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fbb7c0cfa50 con 0x7fbb8c100a80 2024-09-11T22:51:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.246+0000 7fbb83fff640 1 --2- 172.21.15.152:0/3369967929 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbb780a2c90 0x7fbb780a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0febe0 tx=0x7fbb74005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:36.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.246+0000 7fbb8b7fe640 1 --2- 172.21.15.152:0/3369967929 >> [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563] conn(0x7fbb780ac4e0 0x7fbb780ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:36.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.247+0000 7fbb8b7fe640 1 --2- 172.21.15.152:0/3369967929 >> [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563] conn(0x7fbb780ac4e0 0x7fbb780ae900 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:36.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.248+0000 7fbb88ff9640 1 -- 172.21.15.152:0/3369967929 <== osd.4 v2:172.21.15.191:6808/2967539563 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb780aefb0 con 0x7fbb780ac4e0 2024-09-11T22:51:36.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.261+0000 7fbb9138b640 1 -- 172.21.15.152:0/3369967929 --> [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb8c1021c0 con 0x7fbb780ac4e0 2024-09-11T22:51:36.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.261+0000 7fbb88ff9640 1 -- 172.21.15.152:0/3369967929 <== osd.4 v2:172.21.15.191:6808/2967539563 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbb8c1021c0 con 0x7fbb780ac4e0 2024-09-11T22:51:36.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.261+0000 7fbb9138b640 1 -- 172.21.15.152:0/3369967929 >> [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563] conn(0x7fbb780ac4e0 msgr2=0x7fbb780ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:36.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.261+0000 7fbb9138b640 1 --2- 172.21.15.152:0/3369967929 >> [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563] conn(0x7fbb780ac4e0 0x7fbb780ae900 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:36.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.261+0000 7fbb9138b640 1 -- 172.21.15.152:0/3369967929 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbb780a2c90 msgr2=0x7fbb780a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:36.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.261+0000 7fbb9138b640 1 --2- 172.21.15.152:0/3369967929 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fbb780a2c90 0x7fbb780a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0febe0 tx=0x7fbb74005d30 comp rx=0 tx=0).stop 2024-09-11T22:51:36.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.261+0000 7fbb9138b640 1 -- 172.21.15.152:0/3369967929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb8c100a80 msgr2=0x7fbb8c19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:36.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.261+0000 7fbb9138b640 1 --2- 172.21.15.152:0/3369967929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb8c100a80 0x7fbb8c19a5a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00c920 tx=0x7fbb7c00cde0 comp rx=0 tx=0).stop 2024-09-11T22:51:36.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.261+0000 7fbb8b7fe640 1 -- 172.21.15.152:0/3369967929 reap_dead start 2024-09-11T22:51:36.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.261+0000 7fbb9138b640 1 -- 172.21.15.152:0/3369967929 shutdown_connections 2024-09-11T22:51:36.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.261+0000 7fbb9138b640 1 -- 172.21.15.152:0/3369967929 >> 172.21.15.152:0/3369967929 conn(0x7fbb8c0fc7a0 msgr2=0x7fbb8c06a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:36.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.261+0000 7fbb9138b640 1 -- 172.21.15.152:0/3369967929 shutdown_connections 2024-09-11T22:51:36.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:36.262+0000 7fbb9138b640 1 -- 172.21.15.152:0/3369967929 wait complete. 2024-09-11T22:51:37.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.421+0000 7f06853b9640 1 -- 172.21.15.152:0/2552095504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0680106f00 msgr2=0x7f068010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:37.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.421+0000 7f06853b9640 1 --2- 172.21.15.152:0/2552095504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0680106f00 0x7f068010ea80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0670009900 tx=0x7f067002a410 comp rx=0 tx=0).stop 2024-09-11T22:51:37.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.421+0000 7f06853b9640 1 -- 172.21.15.152:0/2552095504 shutdown_connections 2024-09-11T22:51:37.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.421+0000 7f06853b9640 1 --2- 172.21.15.152:0/2552095504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0680106f00 0x7f068010ea80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:37.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.421+0000 7f06853b9640 1 --2- 172.21.15.152:0/2552095504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0680102780 0x7f06801067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:37.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.421+0000 7f06853b9640 1 --2- 172.21.15.152:0/2552095504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0680101dd0 0x7f06801021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:37.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.421+0000 7f06853b9640 1 -- 172.21.15.152:0/2552095504 >> 172.21.15.152:0/2552095504 conn(0x7f06800fd8b0 msgr2=0x7f06800ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:37.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.421+0000 7f06853b9640 1 -- 172.21.15.152:0/2552095504 shutdown_connections 2024-09-11T22:51:37.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.421+0000 7f06853b9640 1 -- 172.21.15.152:0/2552095504 wait complete. 2024-09-11T22:51:37.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.421+0000 7f06853b9640 1 Processor -- start 2024-09-11T22:51:37.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f06853b9640 1 -- start start 2024-09-11T22:51:37.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f06853b9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0680101dd0 0x7f06801983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:37.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f06853b9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0680102780 0x7f0680198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:37.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f06853b9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0680106f00 0x7f068019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:37.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f06853b9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0680111f60 con 0x7f0680101dd0 2024-09-11T22:51:37.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f06853b9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0680111de0 con 0x7f0680102780 2024-09-11T22:51:37.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f06853b9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f06801120e0 con 0x7f0680106f00 2024-09-11T22:51:37.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f067effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0680101dd0 0x7f06801983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:37.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f067effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0680101dd0 0x7f06801983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33974/0 (socket says 172.21.15.152:33974) 2024-09-11T22:51:37.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f067effd640 1 -- 172.21.15.152:0/131436892 learned_addr learned my addr 172.21.15.152:0/131436892 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:37.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f067f7fe640 1 --2- 172.21.15.152:0/131436892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0680106f00 0x7f068019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:37.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f067e7fc640 1 --2- 172.21.15.152:0/131436892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0680102780 0x7f0680198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:37.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f067effd640 1 -- 172.21.15.152:0/131436892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0680106f00 msgr2=0x7f068019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:37.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f067effd640 1 --2- 172.21.15.152:0/131436892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0680106f00 0x7f068019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:37.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f067effd640 1 -- 172.21.15.152:0/131436892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0680102780 msgr2=0x7f0680198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:37.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f067effd640 1 --2- 172.21.15.152:0/131436892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0680102780 0x7f0680198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:37.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f067effd640 1 -- 172.21.15.152:0/131436892 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068019d390 con 0x7f0680101dd0 2024-09-11T22:51:37.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.422+0000 7f067effd640 1 --2- 172.21.15.152:0/131436892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0680101dd0 0x7f06801983e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f067800ebc0 tx=0x7f067800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:37.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.423+0000 7f0677fff640 1 -- 172.21.15.152:0/131436892 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678010630 con 0x7f0680101dd0 2024-09-11T22:51:37.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.423+0000 7f06853b9640 1 -- 172.21.15.152:0/131436892 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068019d680 con 0x7f0680101dd0 2024-09-11T22:51:37.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.423+0000 7f0677fff640 1 -- 172.21.15.152:0/131436892 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0678010cb0 con 0x7f0680101dd0 2024-09-11T22:51:37.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.423+0000 7f0677fff640 1 -- 172.21.15.152:0/131436892 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678018bd0 con 0x7f0680101dd0 2024-09-11T22:51:37.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.423+0000 7f06853b9640 1 -- 172.21.15.152:0/131436892 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06801a4f60 con 0x7f0680101dd0 2024-09-11T22:51:37.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.424+0000 7f06853b9640 1 -- 172.21.15.152:0/131436892 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0680062940 con 0x7f0680101dd0 2024-09-11T22:51:37.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.425+0000 7f0677fff640 1 -- 172.21.15.152:0/131436892 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f06780107d0 con 0x7f0680101dd0 2024-09-11T22:51:37.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.425+0000 7f0677fff640 1 --2- 172.21.15.152:0/131436892 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f066c0a2c10 0x7f066c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:37.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.425+0000 7f067e7fc640 1 --2- 172.21.15.152:0/131436892 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f066c0a2c10 0x7f066c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:37.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.426+0000 7f0677fff640 1 -- 172.21.15.152:0/131436892 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f06780cf770 con 0x7f0680101dd0 2024-09-11T22:51:37.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.426+0000 7f0677fff640 1 --2- 172.21.15.152:0/131436892 >> [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007] conn(0x7f066c0ac460 0x7f066c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:37.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.426+0000 7f0677fff640 1 -- 172.21.15.152:0/131436892 --> [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f066c0aef30 con 0x7f066c0ac460 2024-09-11T22:51:37.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.426+0000 7f0677fff640 1 -- 172.21.15.152:0/131436892 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f06780d6050 con 0x7f0680101dd0 2024-09-11T22:51:37.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.426+0000 7f067f7fe640 1 --2- 172.21.15.152:0/131436892 >> [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007] conn(0x7f066c0ac460 0x7f066c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:37.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.426+0000 7f067e7fc640 1 --2- 172.21.15.152:0/131436892 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f066c0a2c10 0x7f066c0a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0668005fd0 tx=0x7f0668005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:37.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.426+0000 7f067f7fe640 1 --2- 172.21.15.152:0/131436892 >> [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007] conn(0x7f066c0ac460 0x7f066c0ae880 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:37.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.427+0000 7f0677fff640 1 -- 172.21.15.152:0/131436892 <== osd.0 v2:172.21.15.152:6802/1270728007 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f066c0aef30 con 0x7f066c0ac460 2024-09-11T22:51:37.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.440+0000 7f06853b9640 1 -- 172.21.15.152:0/131436892 --> [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0680062dc0 con 0x7f066c0ac460 2024-09-11T22:51:37.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.440+0000 7f0677fff640 1 -- 172.21.15.152:0/131436892 <== osd.0 v2:172.21.15.152:6802/1270728007 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0680062dc0 con 0x7f066c0ac460 2024-09-11T22:51:37.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.441+0000 7f06853b9640 1 -- 172.21.15.152:0/131436892 >> [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007] conn(0x7f066c0ac460 msgr2=0x7f066c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:37.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.441+0000 7f06853b9640 1 --2- 172.21.15.152:0/131436892 >> [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007] conn(0x7f066c0ac460 0x7f066c0ae880 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:37.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.441+0000 7f06853b9640 1 -- 172.21.15.152:0/131436892 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f066c0a2c10 msgr2=0x7f066c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:37.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.441+0000 7f06853b9640 1 --2- 172.21.15.152:0/131436892 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f066c0a2c10 0x7f066c0a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0668005fd0 tx=0x7f0668005850 comp rx=0 tx=0).stop 2024-09-11T22:51:37.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.441+0000 7f06853b9640 1 -- 172.21.15.152:0/131436892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0680101dd0 msgr2=0x7f06801983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:37.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.441+0000 7f06853b9640 1 --2- 172.21.15.152:0/131436892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0680101dd0 0x7f06801983e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f067800ebc0 tx=0x7f067800c6a0 comp rx=0 tx=0).stop 2024-09-11T22:51:37.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.441+0000 7f067f7fe640 1 -- 172.21.15.152:0/131436892 reap_dead start 2024-09-11T22:51:37.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.441+0000 7f06853b9640 1 -- 172.21.15.152:0/131436892 shutdown_connections 2024-09-11T22:51:37.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.441+0000 7f06853b9640 1 -- 172.21.15.152:0/131436892 >> 172.21.15.152:0/131436892 conn(0x7f06800fd8b0 msgr2=0x7f06800feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:37.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.441+0000 7f06853b9640 1 -- 172.21.15.152:0/131436892 shutdown_connections 2024-09-11T22:51:37.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.441+0000 7f06853b9640 1 -- 172.21.15.152:0/131436892 wait complete. 2024-09-11T22:51:37.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.310+0000 7f01ae76e640 1 -- 172.21.15.152:0/1243089555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01a8102570 msgr2=0x7f01a81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:37.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.310+0000 7f01ae76e640 1 --2- 172.21.15.152:0/1243089555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01a8102570 0x7f01a81029b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0190009900 tx=0x7f019002a480 comp rx=0 tx=0).stop 2024-09-11T22:51:37.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.310+0000 7f01ae76e640 1 -- 172.21.15.152:0/1243089555 shutdown_connections 2024-09-11T22:51:37.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.310+0000 7f01ae76e640 1 --2- 172.21.15.152:0/1243089555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01a8102ef0 0x7f01a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:37.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.310+0000 7f01ae76e640 1 --2- 172.21.15.152:0/1243089555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01a8102570 0x7f01a81029b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:37.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.310+0000 7f01ae76e640 1 --2- 172.21.15.152:0/1243089555 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01a8100e20 0x7f01a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:37.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.310+0000 7f01ae76e640 1 -- 172.21.15.152:0/1243089555 >> 172.21.15.152:0/1243089555 conn(0x7f01a80fc7d0 msgr2=0x7f01a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:37.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.310+0000 7f01ae76e640 1 -- 172.21.15.152:0/1243089555 shutdown_connections 2024-09-11T22:51:37.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.310+0000 7f01ae76e640 1 -- 172.21.15.152:0/1243089555 wait complete. 2024-09-11T22:51:37.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f01ae76e640 1 Processor -- start 2024-09-11T22:51:37.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f01ae76e640 1 -- start start 2024-09-11T22:51:37.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f01ae76e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01a8100e20 0x7f01a8199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:37.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f01ae76e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01a8102570 0x7f01a81998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:37.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f01ae76e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01a8102ef0 0x7f01a81a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:37.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f01ae76e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f01a8110f80 con 0x7f01a8102570 2024-09-11T22:51:37.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f01ae76e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f01a8110e00 con 0x7f01a8102ef0 2024-09-11T22:51:37.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f01ae76e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01a8111100 con 0x7f01a8100e20 2024-09-11T22:51:37.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f019ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01a8102570 0x7f01a81998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:37.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f019ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01a8102570 0x7f01a81998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33952/0 (socket says 172.21.15.152:33952) 2024-09-11T22:51:37.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f019ffff640 1 -- 172.21.15.152:0/3167476272 learned_addr learned my addr 172.21.15.152:0/3167476272 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:37.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f01a7fff640 1 --2- 172.21.15.152:0/3167476272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01a8100e20 0x7f01a8199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:37.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f01acd0b640 1 --2- 172.21.15.152:0/3167476272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01a8102ef0 0x7f01a81a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:37.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f019ffff640 1 -- 172.21.15.152:0/3167476272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01a8100e20 msgr2=0x7f01a8199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:37.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f019ffff640 1 --2- 172.21.15.152:0/3167476272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01a8100e20 0x7f01a8199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:37.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f019ffff640 1 -- 172.21.15.152:0/3167476272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01a8102ef0 msgr2=0x7f01a81a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:37.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f019ffff640 1 --2- 172.21.15.152:0/3167476272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01a8102ef0 0x7f01a81a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:37.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.311+0000 7f019ffff640 1 -- 172.21.15.152:0/3167476272 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a81a4930 con 0x7f01a8102570 2024-09-11T22:51:37.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.312+0000 7f019ffff640 1 --2- 172.21.15.152:0/3167476272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01a8102570 0x7f01a81998b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0190006010 tx=0x7f0190003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:37.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.312+0000 7f01a5ffb640 1 -- 172.21.15.152:0/3167476272 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019003f070 con 0x7f01a8102570 2024-09-11T22:51:37.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.312+0000 7f01a5ffb640 1 -- 172.21.15.152:0/3167476272 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0190031c30 con 0x7f01a8102570 2024-09-11T22:51:37.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.312+0000 7f01ae76e640 1 -- 172.21.15.152:0/3167476272 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a81a4b00 con 0x7f01a8102570 2024-09-11T22:51:37.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.312+0000 7f01a5ffb640 1 -- 172.21.15.152:0/3167476272 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190035030 con 0x7f01a8102570 2024-09-11T22:51:37.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.312+0000 7f01ae76e640 1 -- 172.21.15.152:0/3167476272 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a81a4f50 con 0x7f01a8102570 2024-09-11T22:51:37.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.313+0000 7f01ae76e640 1 -- 172.21.15.152:0/3167476272 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f01a8062980 con 0x7f01a8102570 2024-09-11T22:51:37.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.314+0000 7f01a5ffb640 1 -- 172.21.15.152:0/3167476272 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0190042070 con 0x7f01a8102570 2024-09-11T22:51:37.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.314+0000 7f01a5ffb640 1 --2- 172.21.15.152:0/3167476272 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01940a2c10 0x7f01940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:37.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.315+0000 7f01a7fff640 1 --2- 172.21.15.152:0/3167476272 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01940a2c10 0x7f01940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:37.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.315+0000 7f01a5ffb640 1 -- 172.21.15.152:0/3167476272 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f01900f0020 con 0x7f01a8102570 2024-09-11T22:51:37.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.315+0000 7f01a5ffb640 1 --2- 172.21.15.152:0/3167476272 >> [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181] conn(0x7f01940ac460 0x7f01940ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:37.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.315+0000 7f01a5ffb640 1 -- 172.21.15.152:0/3167476272 --> [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01940aef30 con 0x7f01940ac460 2024-09-11T22:51:37.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.315+0000 7f01a5ffb640 1 -- 172.21.15.152:0/3167476272 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f01900f0500 con 0x7f01a8102570 2024-09-11T22:51:37.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.315+0000 7f01acd0b640 1 --2- 172.21.15.152:0/3167476272 >> [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181] conn(0x7f01940ac460 0x7f01940ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:37.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.315+0000 7f01a7fff640 1 --2- 172.21.15.152:0/3167476272 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01940a2c10 0x7f01940a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0198005fd0 tx=0x7f019800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:37.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.315+0000 7f01acd0b640 1 --2- 172.21.15.152:0/3167476272 >> [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181] conn(0x7f01940ac460 0x7f01940ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:37.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.316+0000 7f01a5ffb640 1 -- 172.21.15.152:0/3167476272 <== osd.3 v2:172.21.15.152:6810/3351257181 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f01940aef30 con 0x7f01940ac460 2024-09-11T22:51:37.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.329+0000 7f01ae76e640 1 -- 172.21.15.152:0/3167476272 --> [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f01a8062e00 con 0x7f01940ac460 2024-09-11T22:51:37.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.330+0000 7f01a5ffb640 1 -- 172.21.15.152:0/3167476272 <== osd.3 v2:172.21.15.152:6810/3351257181 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f01a8062e00 con 0x7f01940ac460 2024-09-11T22:51:37.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.330+0000 7f01ae76e640 1 -- 172.21.15.152:0/3167476272 >> [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181] conn(0x7f01940ac460 msgr2=0x7f01940ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:37.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.330+0000 7f01ae76e640 1 --2- 172.21.15.152:0/3167476272 >> [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181] conn(0x7f01940ac460 0x7f01940ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:37.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.330+0000 7f01ae76e640 1 -- 172.21.15.152:0/3167476272 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01940a2c10 msgr2=0x7f01940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:37.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.330+0000 7f01ae76e640 1 --2- 172.21.15.152:0/3167476272 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01940a2c10 0x7f01940a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0198005fd0 tx=0x7f019800b390 comp rx=0 tx=0).stop 2024-09-11T22:51:37.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.330+0000 7f01ae76e640 1 -- 172.21.15.152:0/3167476272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01a8102570 msgr2=0x7f01a81998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:37.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.330+0000 7f01ae76e640 1 --2- 172.21.15.152:0/3167476272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01a8102570 0x7f01a81998b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0190006010 tx=0x7f0190003c10 comp rx=0 tx=0).stop 2024-09-11T22:51:37.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.330+0000 7f01acd0b640 1 -- 172.21.15.152:0/3167476272 reap_dead start 2024-09-11T22:51:37.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.330+0000 7f01ae76e640 1 -- 172.21.15.152:0/3167476272 shutdown_connections 2024-09-11T22:51:37.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.330+0000 7f01ae76e640 1 -- 172.21.15.152:0/3167476272 >> 172.21.15.152:0/3167476272 conn(0x7f01a80fc7d0 msgr2=0x7f01a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:37.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.330+0000 7f01ae76e640 1 -- 172.21.15.152:0/3167476272 shutdown_connections 2024-09-11T22:51:37.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.330+0000 7f01ae76e640 1 -- 172.21.15.152:0/3167476272 wait complete. 2024-09-11T22:51:38.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:37 smithi152 ceph-mon[31181]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-11T22:51:38.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:37 smithi191 ceph-mon[35211]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-11T22:51:38.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:37 smithi204 ceph-mon[35454]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-11T22:51:38.651 INFO:teuthology.orchestra.run.smithi152.stdout:34359738447 2024-09-11T22:51:38.652 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T22:51:38.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.852+0000 7ffa25f39640 1 -- 172.21.15.152:0/4069649496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa201008c0 msgr2=0x7ffa20100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:38.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.852+0000 7ffa25f39640 1 --2- 172.21.15.152:0/4069649496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa201008c0 0x7ffa20100ca0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009900 tx=0x7ffa1002a410 comp rx=0 tx=0).stop 2024-09-11T22:51:38.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.852+0000 7ffa25f39640 1 -- 172.21.15.152:0/4069649496 shutdown_connections 2024-09-11T22:51:38.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.852+0000 7ffa25f39640 1 --2- 172.21.15.152:0/4069649496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa20105ec0 0x7ffa20106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:38.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.852+0000 7ffa25f39640 1 --2- 172.21.15.152:0/4069649496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa201011e0 0x7ffa20105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:38.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.852+0000 7ffa25f39640 1 --2- 172.21.15.152:0/4069649496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa201008c0 0x7ffa20100ca0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:38.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.852+0000 7ffa25f39640 1 -- 172.21.15.152:0/4069649496 >> 172.21.15.152:0/4069649496 conn(0x7ffa200fc7d0 msgr2=0x7ffa200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:38.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.852+0000 7ffa25f39640 1 -- 172.21.15.152:0/4069649496 shutdown_connections 2024-09-11T22:51:38.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.852+0000 7ffa25f39640 1 -- 172.21.15.152:0/4069649496 wait complete. 2024-09-11T22:51:38.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.853+0000 7ffa25f39640 1 Processor -- start 2024-09-11T22:51:38.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.853+0000 7ffa25f39640 1 -- start start 2024-09-11T22:51:38.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.853+0000 7ffa25f39640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa201008c0 0x7ffa201a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:38.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.853+0000 7ffa25f39640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa201011e0 0x7ffa201a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:38.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.853+0000 7ffa25f39640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa20105ec0 0x7ffa2019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:38.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.853+0000 7ffa25f39640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20118690 con 0x7ffa201011e0 2024-09-11T22:51:38.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.853+0000 7ffa25f39640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20118510 con 0x7ffa20105ec0 2024-09-11T22:51:38.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.853+0000 7ffa25f39640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20118810 con 0x7ffa201008c0 2024-09-11T22:51:38.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.853+0000 7ffa1effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa201011e0 0x7ffa201a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:38.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.853+0000 7ffa1effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa201011e0 0x7ffa201a4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33660/0 (socket says 172.21.15.152:33660) 2024-09-11T22:51:38.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.853+0000 7ffa1effd640 1 -- 172.21.15.152:0/2961634699 learned_addr learned my addr 172.21.15.152:0/2961634699 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:38.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.854+0000 7ffa1ffff640 1 --2- 172.21.15.152:0/2961634699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa20105ec0 0x7ffa2019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:38.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.854+0000 7ffa1f7fe640 1 --2- 172.21.15.152:0/2961634699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa201008c0 0x7ffa201a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:38.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.854+0000 7ffa1effd640 1 -- 172.21.15.152:0/2961634699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa201008c0 msgr2=0x7ffa201a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:38.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.854+0000 7ffa1effd640 1 --2- 172.21.15.152:0/2961634699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa201008c0 0x7ffa201a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:38.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.854+0000 7ffa1effd640 1 -- 172.21.15.152:0/2961634699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa20105ec0 msgr2=0x7ffa2019ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:38.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.854+0000 7ffa1effd640 1 --2- 172.21.15.152:0/2961634699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa20105ec0 0x7ffa2019ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:38.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.854+0000 7ffa1effd640 1 -- 172.21.15.152:0/2961634699 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2019f170 con 0x7ffa201011e0 2024-09-11T22:51:38.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.854+0000 7ffa1ffff640 1 --2- 172.21.15.152:0/2961634699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa20105ec0 0x7ffa2019ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:38.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.854+0000 7ffa1effd640 1 --2- 172.21.15.152:0/2961634699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa201011e0 0x7ffa201a4dc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400b4a0 tx=0x7ffa1400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:38.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.854+0000 7ffa1cff9640 1 -- 172.21.15.152:0/2961634699 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14014070 con 0x7ffa201011e0 2024-09-11T22:51:38.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.854+0000 7ffa25f39640 1 -- 172.21.15.152:0/2961634699 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2019f460 con 0x7ffa201011e0 2024-09-11T22:51:38.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.854+0000 7ffa1cff9640 1 -- 172.21.15.152:0/2961634699 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa14010af0 con 0x7ffa201011e0 2024-09-11T22:51:38.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.854+0000 7ffa1cff9640 1 -- 172.21.15.152:0/2961634699 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14017d10 con 0x7ffa201011e0 2024-09-11T22:51:38.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.854+0000 7ffa25f39640 1 -- 172.21.15.152:0/2961634699 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa201ab750 con 0x7ffa201011e0 2024-09-11T22:51:38.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.855+0000 7ffa25f39640 1 -- 172.21.15.152:0/2961634699 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa201100d0 con 0x7ffa201011e0 2024-09-11T22:51:38.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.856+0000 7ffa1cff9640 1 -- 172.21.15.152:0/2961634699 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ffa14017070 con 0x7ffa201011e0 2024-09-11T22:51:38.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.857+0000 7ffa1cff9640 1 --2- 172.21.15.152:0/2961634699 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ffa080a2c10 0x7ffa080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:38.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.857+0000 7ffa1f7fe640 1 --2- 172.21.15.152:0/2961634699 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ffa080a2c10 0x7ffa080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:38.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.857+0000 7ffa1cff9640 1 -- 172.21.15.152:0/2961634699 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7ffa140ce730 con 0x7ffa201011e0 2024-09-11T22:51:38.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.857+0000 7ffa1cff9640 1 --2- 172.21.15.152:0/2961634699 >> [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419] conn(0x7ffa080ac460 0x7ffa080ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:38.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.857+0000 7ffa1cff9640 1 -- 172.21.15.152:0/2961634699 --> [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa1401ae50 con 0x7ffa080ac460 2024-09-11T22:51:38.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.857+0000 7ffa1cff9640 1 -- 172.21.15.152:0/2961634699 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7ffa140cec10 con 0x7ffa201011e0 2024-09-11T22:51:38.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.857+0000 7ffa1ffff640 1 --2- 172.21.15.152:0/2961634699 >> [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419] conn(0x7ffa080ac460 0x7ffa080ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:38.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.857+0000 7ffa1f7fe640 1 --2- 172.21.15.152:0/2961634699 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ffa080a2c10 0x7ffa080a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffa100096c0 tx=0x7ffa10000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:38.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.857+0000 7ffa1ffff640 1 --2- 172.21.15.152:0/2961634699 >> [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419] conn(0x7ffa080ac460 0x7ffa080ae880 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:38.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.858+0000 7ffa1cff9640 1 -- 172.21.15.152:0/2961634699 <== osd.9 v2:172.21.15.152:6826/1301191419 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa1401ae50 con 0x7ffa080ac460 2024-09-11T22:51:38.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.872+0000 7ffa25f39640 1 -- 172.21.15.152:0/2961634699 --> [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa2006a850 con 0x7ffa080ac460 2024-09-11T22:51:38.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.872+0000 7ffa1cff9640 1 -- 172.21.15.152:0/2961634699 <== osd.9 v2:172.21.15.152:6826/1301191419 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffa2006a850 con 0x7ffa080ac460 2024-09-11T22:51:38.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.872+0000 7ffa25f39640 1 -- 172.21.15.152:0/2961634699 >> [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419] conn(0x7ffa080ac460 msgr2=0x7ffa080ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:38.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.872+0000 7ffa25f39640 1 --2- 172.21.15.152:0/2961634699 >> [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419] conn(0x7ffa080ac460 0x7ffa080ae880 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:38.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.872+0000 7ffa25f39640 1 -- 172.21.15.152:0/2961634699 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ffa080a2c10 msgr2=0x7ffa080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:38.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.872+0000 7ffa25f39640 1 --2- 172.21.15.152:0/2961634699 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ffa080a2c10 0x7ffa080a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffa100096c0 tx=0x7ffa10000bc0 comp rx=0 tx=0).stop 2024-09-11T22:51:38.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.872+0000 7ffa25f39640 1 -- 172.21.15.152:0/2961634699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa201011e0 msgr2=0x7ffa201a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:38.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.872+0000 7ffa25f39640 1 --2- 172.21.15.152:0/2961634699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa201011e0 0x7ffa201a4dc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400b4a0 tx=0x7ffa1400b960 comp rx=0 tx=0).stop 2024-09-11T22:51:38.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.872+0000 7ffa1ffff640 1 -- 172.21.15.152:0/2961634699 reap_dead start 2024-09-11T22:51:38.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.872+0000 7ffa25f39640 1 -- 172.21.15.152:0/2961634699 shutdown_connections 2024-09-11T22:51:38.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.872+0000 7ffa25f39640 1 -- 172.21.15.152:0/2961634699 >> 172.21.15.152:0/2961634699 conn(0x7ffa200fc7d0 msgr2=0x7ffa2010f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:38.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.872+0000 7ffa25f39640 1 -- 172.21.15.152:0/2961634699 shutdown_connections 2024-09-11T22:51:38.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:38.872+0000 7ffa25f39640 1 -- 172.21.15.152:0/2961634699 wait complete. 2024-09-11T22:51:39.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.944+0000 7fa5d579d640 1 -- 172.21.15.152:0/175557157 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5d0102d30 msgr2=0x7fa5d0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:39.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.944+0000 7fa5d579d640 1 --2- 172.21.15.152:0/175557157 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5d0102d30 0x7fa5d0103110 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009cd0 tx=0x7fa5c002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:51:39.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.944+0000 7fa5d579d640 1 -- 172.21.15.152:0/175557157 shutdown_connections 2024-09-11T22:51:39.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.944+0000 7fa5d579d640 1 --2- 172.21.15.152:0/175557157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5d01082a0 0x7fa5d010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:39.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.944+0000 7fa5d579d640 1 --2- 172.21.15.152:0/175557157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5d01036e0 0x7fa5d0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:39.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.944+0000 7fa5d579d640 1 --2- 172.21.15.152:0/175557157 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5d0102d30 0x7fa5d0103110 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:39.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.944+0000 7fa5d579d640 1 -- 172.21.15.152:0/175557157 >> 172.21.15.152:0/175557157 conn(0x7fa5d00fc7d0 msgr2=0x7fa5d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:39.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.944+0000 7fa5d579d640 1 -- 172.21.15.152:0/175557157 shutdown_connections 2024-09-11T22:51:39.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.944+0000 7fa5d579d640 1 -- 172.21.15.152:0/175557157 wait complete. 2024-09-11T22:51:39.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.944+0000 7fa5d579d640 1 Processor -- start 2024-09-11T22:51:39.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5d579d640 1 -- start start 2024-09-11T22:51:39.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5d579d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5d0102d30 0x7fa5d0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:39.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5d579d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5d01036e0 0x7fa5d0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:39.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5d579d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5d01082a0 0x7fa5d0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:39.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5d579d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01120a0 con 0x7fa5d01036e0 2024-09-11T22:51:39.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5d579d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d0111f20 con 0x7fa5d0102d30 2024-09-11T22:51:39.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5d579d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d0112220 con 0x7fa5d01082a0 2024-09-11T22:51:39.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5ce7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5d01036e0 0x7fa5d0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:39.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5ce7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5d01036e0 0x7fa5d0100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33990/0 (socket says 172.21.15.152:33990) 2024-09-11T22:51:39.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5ce7fc640 1 -- 172.21.15.152:0/180567174 learned_addr learned my addr 172.21.15.152:0/180567174 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:39.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5ceffd640 1 --2- 172.21.15.152:0/180567174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5d0102d30 0x7fa5d0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:39.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5cf7fe640 1 --2- 172.21.15.152:0/180567174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5d01082a0 0x7fa5d0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:39.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5ce7fc640 1 -- 172.21.15.152:0/180567174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5d01082a0 msgr2=0x7fa5d0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:39.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5ce7fc640 1 --2- 172.21.15.152:0/180567174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa5d01082a0 0x7fa5d0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:39.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5ce7fc640 1 -- 172.21.15.152:0/180567174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5d0102d30 msgr2=0x7fa5d0100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:39.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5ce7fc640 1 --2- 172.21.15.152:0/180567174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5d0102d30 0x7fa5d0100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:39.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5ce7fc640 1 -- 172.21.15.152:0/180567174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d01a6aa0 con 0x7fa5d01036e0 2024-09-11T22:51:39.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.945+0000 7fa5ce7fc640 1 --2- 172.21.15.152:0/180567174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5d01036e0 0x7fa5d0100ed0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800d750 tx=0x7fa5b800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:39.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.946+0000 7fa5c7fff640 1 -- 172.21.15.152:0/180567174 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8017070 con 0x7fa5d01036e0 2024-09-11T22:51:39.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.946+0000 7fa5c7fff640 1 -- 172.21.15.152:0/180567174 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b8005500 con 0x7fa5d01036e0 2024-09-11T22:51:39.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.946+0000 7fa5d579d640 1 -- 172.21.15.152:0/180567174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d01a6d90 con 0x7fa5d01036e0 2024-09-11T22:51:39.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.946+0000 7fa5c7fff640 1 -- 172.21.15.152:0/180567174 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8004e00 con 0x7fa5d01036e0 2024-09-11T22:51:39.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.946+0000 7fa5d579d640 1 -- 172.21.15.152:0/180567174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d01a7220 con 0x7fa5d01036e0 2024-09-11T22:51:39.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.946+0000 7fa5d579d640 1 -- 172.21.15.152:0/180567174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa5d00628c0 con 0x7fa5d01036e0 2024-09-11T22:51:39.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.948+0000 7fa5c7fff640 1 -- 172.21.15.152:0/180567174 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa5b8004fa0 con 0x7fa5d01036e0 2024-09-11T22:51:39.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.948+0000 7fa5c7fff640 1 --2- 172.21.15.152:0/180567174 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa5bc0a2c10 0x7fa5bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:39.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.948+0000 7fa5ceffd640 1 --2- 172.21.15.152:0/180567174 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa5bc0a2c10 0x7fa5bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:39.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.948+0000 7fa5c7fff640 1 -- 172.21.15.152:0/180567174 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fa5b80cecf0 con 0x7fa5d01036e0 2024-09-11T22:51:39.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.949+0000 7fa5c7fff640 1 --2- 172.21.15.152:0/180567174 >> [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090] conn(0x7fa5bc0ac460 0x7fa5bc0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:39.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.949+0000 7fa5c7fff640 1 -- 172.21.15.152:0/180567174 --> [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5bc0aef30 con 0x7fa5bc0ac460 2024-09-11T22:51:39.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.949+0000 7fa5c7fff640 1 -- 172.21.15.152:0/180567174 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fa5b80cf1d0 con 0x7fa5d01036e0 2024-09-11T22:51:39.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.949+0000 7fa5ceffd640 1 --2- 172.21.15.152:0/180567174 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa5bc0a2c10 0x7fa5bc0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0006010 tx=0x7fa5c0009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:39.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.949+0000 7fa5cf7fe640 1 --2- 172.21.15.152:0/180567174 >> [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090] conn(0x7fa5bc0ac460 0x7fa5bc0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:39.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.949+0000 7fa5cf7fe640 1 --2- 172.21.15.152:0/180567174 >> [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090] conn(0x7fa5bc0ac460 0x7fa5bc0ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:39.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.950+0000 7fa5c7fff640 1 -- 172.21.15.152:0/180567174 <== osd.8 v2:172.21.15.204:6816/1756190090 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa5bc0aef30 con 0x7fa5bc0ac460 2024-09-11T22:51:39.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.963+0000 7fa5d579d640 1 -- 172.21.15.152:0/180567174 --> [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa5d0062d40 con 0x7fa5bc0ac460 2024-09-11T22:51:39.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.964+0000 7fa5c7fff640 1 -- 172.21.15.152:0/180567174 <== osd.8 v2:172.21.15.204:6816/1756190090 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa5d0062d40 con 0x7fa5bc0ac460 2024-09-11T22:51:39.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.964+0000 7fa5d579d640 1 -- 172.21.15.152:0/180567174 >> [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090] conn(0x7fa5bc0ac460 msgr2=0x7fa5bc0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:39.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.964+0000 7fa5d579d640 1 --2- 172.21.15.152:0/180567174 >> [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090] conn(0x7fa5bc0ac460 0x7fa5bc0ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:39.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.964+0000 7fa5d579d640 1 -- 172.21.15.152:0/180567174 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa5bc0a2c10 msgr2=0x7fa5bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:39.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.964+0000 7fa5d579d640 1 --2- 172.21.15.152:0/180567174 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa5bc0a2c10 0x7fa5bc0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0006010 tx=0x7fa5c0009360 comp rx=0 tx=0).stop 2024-09-11T22:51:39.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.964+0000 7fa5d579d640 1 -- 172.21.15.152:0/180567174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5d01036e0 msgr2=0x7fa5d0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:39.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.964+0000 7fa5d579d640 1 --2- 172.21.15.152:0/180567174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa5d01036e0 0x7fa5d0100ed0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800d750 tx=0x7fa5b800dc10 comp rx=0 tx=0).stop 2024-09-11T22:51:39.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.964+0000 7fa5cf7fe640 1 -- 172.21.15.152:0/180567174 reap_dead start 2024-09-11T22:51:39.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.964+0000 7fa5d579d640 1 -- 172.21.15.152:0/180567174 shutdown_connections 2024-09-11T22:51:39.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.964+0000 7fa5d579d640 1 -- 172.21.15.152:0/180567174 >> 172.21.15.152:0/180567174 conn(0x7fa5d00fc7d0 msgr2=0x7fa5d0105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:39.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.965+0000 7fa5d579d640 1 -- 172.21.15.152:0/180567174 shutdown_connections 2024-09-11T22:51:39.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:37.965+0000 7fa5d579d640 1 -- 172.21.15.152:0/180567174 wait complete. 2024-09-11T22:51:39.998 INFO:teuthology.orchestra.run.smithi152.stdout:240518168613 2024-09-11T22:51:39.998 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-11T22:51:40.166 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:39 smithi152 ceph-mon[31181]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T22:51:40.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.165+0000 7ff27376f640 1 -- 172.21.15.152:0/1011097075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c101420 msgr2=0x7ff26c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:40.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.165+0000 7ff27376f640 1 --2- 172.21.15.152:0/1011097075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c101420 0x7ff26c10d950 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009cd0 tx=0x7ff25c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:51:40.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.165+0000 7ff27376f640 1 -- 172.21.15.152:0/1011097075 shutdown_connections 2024-09-11T22:51:40.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.165+0000 7ff27376f640 1 --2- 172.21.15.152:0/1011097075 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c101420 0x7ff26c10d950 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:40.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.165+0000 7ff27376f640 1 --2- 172.21.15.152:0/1011097075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff26c100a80 0x7ff26c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:40.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.165+0000 7ff27376f640 1 --2- 172.21.15.152:0/1011097075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff26c106a80 0x7ff26c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:40.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.165+0000 7ff27376f640 1 -- 172.21.15.152:0/1011097075 >> 172.21.15.152:0/1011097075 conn(0x7ff26c0fc7a0 msgr2=0x7ff26c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:40.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.165+0000 7ff27376f640 1 -- 172.21.15.152:0/1011097075 shutdown_connections 2024-09-11T22:51:40.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.165+0000 7ff27376f640 1 -- 172.21.15.152:0/1011097075 wait complete. 2024-09-11T22:51:40.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff27376f640 1 Processor -- start 2024-09-11T22:51:40.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff27376f640 1 -- start start 2024-09-11T22:51:40.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff27376f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c100a80 0x7ff26c0fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:40.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff27376f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff26c101420 0x7ff26c0fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:40.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff27376f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff26c106a80 0x7ff26c0fd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:40.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff27376f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c110e80 con 0x7ff26c100a80 2024-09-11T22:51:40.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff27376f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c110d00 con 0x7ff26c101420 2024-09-11T22:51:40.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff27376f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c111000 con 0x7ff26c106a80 2024-09-11T22:51:40.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff27150b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c100a80 0x7ff26c0fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:40.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff27150b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c100a80 0x7ff26c0fcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33684/0 (socket says 172.21.15.152:33684) 2024-09-11T22:51:40.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff27150b640 1 -- 172.21.15.152:0/3847755638 learned_addr learned my addr 172.21.15.152:0/3847755638 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:40.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff271d0c640 1 --2- 172.21.15.152:0/3847755638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff26c106a80 0x7ff26c0fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:40.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff270d0a640 1 --2- 172.21.15.152:0/3847755638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff26c101420 0x7ff26c0fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:40.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff27150b640 1 -- 172.21.15.152:0/3847755638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff26c106a80 msgr2=0x7ff26c0fd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:40.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff27150b640 1 --2- 172.21.15.152:0/3847755638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff26c106a80 0x7ff26c0fd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:40.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff27150b640 1 -- 172.21.15.152:0/3847755638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff26c101420 msgr2=0x7ff26c0fd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:40.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff27150b640 1 --2- 172.21.15.152:0/3847755638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff26c101420 0x7ff26c0fd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:40.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff27150b640 1 -- 172.21.15.152:0/3847755638 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26c1a6930 con 0x7ff26c100a80 2024-09-11T22:51:40.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff271d0c640 1 --2- 172.21.15.152:0/3847755638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff26c106a80 0x7ff26c0fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:40.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.166+0000 7ff270d0a640 1 --2- 172.21.15.152:0/3847755638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff26c101420 0x7ff26c0fd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:40.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.167+0000 7ff27150b640 1 --2- 172.21.15.152:0/3847755638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c100a80 0x7ff26c0fcd90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff26400d770 tx=0x7ff26400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:40.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.167+0000 7ff26a7fc640 1 -- 172.21.15.152:0/3847755638 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264017070 con 0x7ff26c100a80 2024-09-11T22:51:40.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.167+0000 7ff26a7fc640 1 -- 172.21.15.152:0/3847755638 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff264005500 con 0x7ff26c100a80 2024-09-11T22:51:40.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.167+0000 7ff27376f640 1 -- 172.21.15.152:0/3847755638 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26c1a6c20 con 0x7ff26c100a80 2024-09-11T22:51:40.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.167+0000 7ff26a7fc640 1 -- 172.21.15.152:0/3847755638 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264004e00 con 0x7ff26c100a80 2024-09-11T22:51:40.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.167+0000 7ff27376f640 1 -- 172.21.15.152:0/3847755638 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26c1a7110 con 0x7ff26c100a80 2024-09-11T22:51:40.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.168+0000 7ff27376f640 1 -- 172.21.15.152:0/3847755638 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff26c062880 con 0x7ff26c100a80 2024-09-11T22:51:40.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.169+0000 7ff26a7fc640 1 -- 172.21.15.152:0/3847755638 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff264004fa0 con 0x7ff26c100a80 2024-09-11T22:51:40.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.170+0000 7ff26a7fc640 1 --2- 172.21.15.152:0/3847755638 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff2540a2c10 0x7ff2540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:40.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.170+0000 7ff26a7fc640 1 -- 172.21.15.152:0/3847755638 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7ff2640ceeb0 con 0x7ff26c100a80 2024-09-11T22:51:40.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.170+0000 7ff270d0a640 1 --2- 172.21.15.152:0/3847755638 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff2540a2c10 0x7ff2540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:40.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.170+0000 7ff26a7fc640 1 --2- 172.21.15.152:0/3847755638 >> [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859] conn(0x7ff2540ac460 0x7ff2540ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:40.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.170+0000 7ff26a7fc640 1 -- 172.21.15.152:0/3847755638 --> [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff26400fec0 con 0x7ff2540ac460 2024-09-11T22:51:40.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.170+0000 7ff26a7fc640 1 -- 172.21.15.152:0/3847755638 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7ff2640cf390 con 0x7ff26c100a80 2024-09-11T22:51:40.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.170+0000 7ff271d0c640 1 --2- 172.21.15.152:0/3847755638 >> [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859] conn(0x7ff2540ac460 0x7ff2540ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:40.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.170+0000 7ff270d0a640 1 --2- 172.21.15.152:0/3847755638 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff2540a2c10 0x7ff2540a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0fe820 tx=0x7ff260005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:40.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.171+0000 7ff271d0c640 1 --2- 172.21.15.152:0/3847755638 >> [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859] conn(0x7ff2540ac460 0x7ff2540ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:40.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.172+0000 7ff26a7fc640 1 -- 172.21.15.152:0/3847755638 <== osd.7 v2:172.21.15.191:6816/269608859 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff26400fec0 con 0x7ff2540ac460 2024-09-11T22:51:40.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.185+0000 7ff27376f640 1 -- 172.21.15.152:0/3847755638 --> [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff26c062d00 con 0x7ff2540ac460 2024-09-11T22:51:40.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.186+0000 7ff26a7fc640 1 -- 172.21.15.152:0/3847755638 <== osd.7 v2:172.21.15.191:6816/269608859 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff26c062d00 con 0x7ff2540ac460 2024-09-11T22:51:40.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.186+0000 7ff27376f640 1 -- 172.21.15.152:0/3847755638 >> [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859] conn(0x7ff2540ac460 msgr2=0x7ff2540ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:40.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.186+0000 7ff27376f640 1 --2- 172.21.15.152:0/3847755638 >> [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859] conn(0x7ff2540ac460 0x7ff2540ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:40.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.186+0000 7ff27376f640 1 -- 172.21.15.152:0/3847755638 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff2540a2c10 msgr2=0x7ff2540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:40.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.186+0000 7ff27376f640 1 --2- 172.21.15.152:0/3847755638 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ff2540a2c10 0x7ff2540a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0fe820 tx=0x7ff260005d30 comp rx=0 tx=0).stop 2024-09-11T22:51:40.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.186+0000 7ff27376f640 1 -- 172.21.15.152:0/3847755638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c100a80 msgr2=0x7ff26c0fcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:40.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.186+0000 7ff27376f640 1 --2- 172.21.15.152:0/3847755638 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff26c100a80 0x7ff26c0fcd90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff26400d770 tx=0x7ff26400dc30 comp rx=0 tx=0).stop 2024-09-11T22:51:40.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.186+0000 7ff271d0c640 1 -- 172.21.15.152:0/3847755638 reap_dead start 2024-09-11T22:51:40.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.186+0000 7ff27376f640 1 -- 172.21.15.152:0/3847755638 shutdown_connections 2024-09-11T22:51:40.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.186+0000 7ff27376f640 1 -- 172.21.15.152:0/3847755638 >> 172.21.15.152:0/3847755638 conn(0x7ff26c0fc7a0 msgr2=0x7ff26c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:40.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.187+0000 7ff27376f640 1 -- 172.21.15.152:0/3847755638 shutdown_connections 2024-09-11T22:51:40.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:40.187+0000 7ff27376f640 1 -- 172.21.15.152:0/3847755638 wait complete. 2024-09-11T22:51:40.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:39 smithi191 ceph-mon[35211]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T22:51:40.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:39 smithi204 ceph-mon[35454]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T22:51:41.195 INFO:teuthology.orchestra.run.smithi152.stdout:292057776154 2024-09-11T22:51:41.196 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-11T22:51:41.673 INFO:teuthology.orchestra.run.smithi152.stdout:107374182464 2024-09-11T22:51:41.674 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T22:51:42.099 INFO:teuthology.orchestra.run.smithi152.stdout:270582939681 2024-09-11T22:51:42.100 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-11T22:51:42.115 INFO:teuthology.orchestra.run.smithi152.stdout:55834574921 2024-09-11T22:51:42.115 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T22:51:42.181 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:41 smithi152 ceph-mon[31181]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-11T22:51:42.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:41 smithi191 ceph-mon[35211]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-11T22:51:42.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:41 smithi204 ceph-mon[35454]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-11T22:51:42.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.373+0000 7fb22bc77640 1 -- 172.21.15.152:0/472231300 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb224108290 msgr2=0x7fb22410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:42.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.373+0000 7fb22bc77640 1 --2- 172.21.15.152:0/472231300 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb224108290 0x7fb22410ec30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb214009930 tx=0x7fb21402a420 comp rx=0 tx=0).stop 2024-09-11T22:51:42.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.373+0000 7fb22bc77640 1 -- 172.21.15.152:0/472231300 shutdown_connections 2024-09-11T22:51:42.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.373+0000 7fb22bc77640 1 --2- 172.21.15.152:0/472231300 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb224108290 0x7fb22410ec30 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:42.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.373+0000 7fb22bc77640 1 --2- 172.21.15.152:0/472231300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb224103740 0x7fb224107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:42.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.373+0000 7fb22bc77640 1 --2- 172.21.15.152:0/472231300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb224102d90 0x7fb224103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:42.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.373+0000 7fb22bc77640 1 -- 172.21.15.152:0/472231300 >> 172.21.15.152:0/472231300 conn(0x7fb2240fc7d0 msgr2=0x7fb2240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:42.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.373+0000 7fb22bc77640 1 -- 172.21.15.152:0/472231300 shutdown_connections 2024-09-11T22:51:42.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.373+0000 7fb22bc77640 1 -- 172.21.15.152:0/472231300 wait complete. 2024-09-11T22:51:42.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb22bc77640 1 Processor -- start 2024-09-11T22:51:42.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb22bc77640 1 -- start start 2024-09-11T22:51:42.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb22bc77640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb224102d90 0x7fb224198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:42.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb22bc77640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb224103740 0x7fb224198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:42.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb22bc77640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb224108290 0x7fb22419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:42.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb22bc77640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb224112090 con 0x7fb224103740 2024-09-11T22:51:42.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb22bc77640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb224111f10 con 0x7fb224102d90 2024-09-11T22:51:42.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb22bc77640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb224112210 con 0x7fb224108290 2024-09-11T22:51:42.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb229212640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb224103740 0x7fb224198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:42.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb229212640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb224103740 0x7fb224198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33712/0 (socket says 172.21.15.152:33712) 2024-09-11T22:51:42.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb229212640 1 -- 172.21.15.152:0/2214111855 learned_addr learned my addr 172.21.15.152:0/2214111855 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:42.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb22a214640 1 --2- 172.21.15.152:0/2214111855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb224108290 0x7fb22419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:42.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb229a13640 1 --2- 172.21.15.152:0/2214111855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb224102d90 0x7fb224198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:42.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb229212640 1 -- 172.21.15.152:0/2214111855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb224108290 msgr2=0x7fb22419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:42.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb229212640 1 --2- 172.21.15.152:0/2214111855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb224108290 0x7fb22419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:42.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb229212640 1 -- 172.21.15.152:0/2214111855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb224102d90 msgr2=0x7fb224198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:42.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb229212640 1 --2- 172.21.15.152:0/2214111855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb224102d90 0x7fb224198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:42.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.374+0000 7fb229212640 1 -- 172.21.15.152:0/2214111855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22419d500 con 0x7fb224103740 2024-09-11T22:51:42.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.375+0000 7fb229212640 1 --2- 172.21.15.152:0/2214111855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb224103740 0x7fb224198a90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb21800d740 tx=0x7fb21800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:42.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.375+0000 7fb222ffd640 1 -- 172.21.15.152:0/2214111855 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218017070 con 0x7fb224103740 2024-09-11T22:51:42.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.375+0000 7fb222ffd640 1 -- 172.21.15.152:0/2214111855 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb21800de50 con 0x7fb224103740 2024-09-11T22:51:42.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.375+0000 7fb222ffd640 1 -- 172.21.15.152:0/2214111855 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218004e00 con 0x7fb224103740 2024-09-11T22:51:42.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.375+0000 7fb22bc77640 1 -- 172.21.15.152:0/2214111855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22419d7f0 con 0x7fb224103740 2024-09-11T22:51:42.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.375+0000 7fb22bc77640 1 -- 172.21.15.152:0/2214111855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2241a50d0 con 0x7fb224103740 2024-09-11T22:51:42.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.376+0000 7fb22bc77640 1 -- 172.21.15.152:0/2214111855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb224062980 con 0x7fb224103740 2024-09-11T22:51:42.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.377+0000 7fb222ffd640 1 -- 172.21.15.152:0/2214111855 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb218004fa0 con 0x7fb224103740 2024-09-11T22:51:42.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.377+0000 7fb222ffd640 1 --2- 172.21.15.152:0/2214111855 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fb20c0a2c90 0x7fb20c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:42.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.378+0000 7fb229a13640 1 --2- 172.21.15.152:0/2214111855 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fb20c0a2c90 0x7fb20c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:42.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.378+0000 7fb222ffd640 1 -- 172.21.15.152:0/2214111855 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fb2180cedc0 con 0x7fb224103740 2024-09-11T22:51:42.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.378+0000 7fb222ffd640 1 --2- 172.21.15.152:0/2214111855 >> [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042] conn(0x7fb20c0ac4e0 0x7fb20c0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:42.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.378+0000 7fb222ffd640 1 -- 172.21.15.152:0/2214111855 --> [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb218001d30 con 0x7fb20c0ac4e0 2024-09-11T22:51:42.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.378+0000 7fb222ffd640 1 -- 172.21.15.152:0/2214111855 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fb2180cf2a0 con 0x7fb224103740 2024-09-11T22:51:42.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.378+0000 7fb22a214640 1 --2- 172.21.15.152:0/2214111855 >> [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042] conn(0x7fb20c0ac4e0 0x7fb20c0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:42.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.378+0000 7fb229a13640 1 --2- 172.21.15.152:0/2214111855 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fb20c0a2c90 0x7fb20c0a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb21c005fd0 tx=0x7fb21c009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:42.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.378+0000 7fb22a214640 1 --2- 172.21.15.152:0/2214111855 >> [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042] conn(0x7fb20c0ac4e0 0x7fb20c0ae900 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:42.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.379+0000 7fb222ffd640 1 -- 172.21.15.152:0/2214111855 <== osd.6 v2:172.21.15.152:6818/2484234042 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb218001d30 con 0x7fb20c0ac4e0 2024-09-11T22:51:42.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.392+0000 7fb22bc77640 1 -- 172.21.15.152:0/2214111855 --> [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb224062e00 con 0x7fb20c0ac4e0 2024-09-11T22:51:42.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.392+0000 7fb222ffd640 1 -- 172.21.15.152:0/2214111855 <== osd.6 v2:172.21.15.152:6818/2484234042 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb224062e00 con 0x7fb20c0ac4e0 2024-09-11T22:51:42.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.393+0000 7fb22bc77640 1 -- 172.21.15.152:0/2214111855 >> [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042] conn(0x7fb20c0ac4e0 msgr2=0x7fb20c0ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:42.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.393+0000 7fb22bc77640 1 --2- 172.21.15.152:0/2214111855 >> [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042] conn(0x7fb20c0ac4e0 0x7fb20c0ae900 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:42.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.393+0000 7fb22bc77640 1 -- 172.21.15.152:0/2214111855 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fb20c0a2c90 msgr2=0x7fb20c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:42.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.393+0000 7fb22bc77640 1 --2- 172.21.15.152:0/2214111855 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fb20c0a2c90 0x7fb20c0a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb21c005fd0 tx=0x7fb21c009870 comp rx=0 tx=0).stop 2024-09-11T22:51:42.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.393+0000 7fb22bc77640 1 -- 172.21.15.152:0/2214111855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb224103740 msgr2=0x7fb224198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:42.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.393+0000 7fb22bc77640 1 --2- 172.21.15.152:0/2214111855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb224103740 0x7fb224198a90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb21800d740 tx=0x7fb21800dc00 comp rx=0 tx=0).stop 2024-09-11T22:51:42.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.393+0000 7fb22a214640 1 -- 172.21.15.152:0/2214111855 reap_dead start 2024-09-11T22:51:42.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.393+0000 7fb22bc77640 1 -- 172.21.15.152:0/2214111855 shutdown_connections 2024-09-11T22:51:42.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.393+0000 7fb22bc77640 1 -- 172.21.15.152:0/2214111855 >> 172.21.15.152:0/2214111855 conn(0x7fb2240fc7d0 msgr2=0x7fb224106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:42.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.393+0000 7fb22bc77640 1 -- 172.21.15.152:0/2214111855 shutdown_connections 2024-09-11T22:51:42.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:42.393+0000 7fb22bc77640 1 -- 172.21.15.152:0/2214111855 wait complete. 2024-09-11T22:51:42.902 INFO:teuthology.orchestra.run.smithi152.stdout:137438953530 2024-09-11T22:51:42.902 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T22:51:43.372 INFO:teuthology.orchestra.run.smithi152.stdout:158913790005 2024-09-11T22:51:43.372 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T22:51:43.797 INFO:teuthology.orchestra.run.smithi152.stdout:322122547222 2024-09-11T22:51:43.797 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-11T22:51:44.044 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:43 smithi152 ceph-mon[31181]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T22:51:44.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:43 smithi191 ceph-mon[35211]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T22:51:44.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:43 smithi204 ceph-mon[35454]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T22:51:44.379 INFO:teuthology.orchestra.run.smithi152.stdout:77309411397 2024-09-11T22:51:44.380 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T22:51:45.202 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:45.202 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:45.203 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:45.237 INFO:teuthology.orchestra.run.smithi152.stdout:210453397547 2024-09-11T22:51:45.238 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T22:51:45.788 INFO:teuthology.orchestra.run.smithi152.stdout:188978561073 2024-09-11T22:51:45.789 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T22:51:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:45.843 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:45.844 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:45.849 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:45.874 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:45.900 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:46.016 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:45 smithi152 ceph-mon[31181]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:51:46.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:45 smithi191 ceph-mon[35211]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:51:46.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:45 smithi204 ceph-mon[35454]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:51:47.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.150+0000 7f6366ef9640 1 -- 172.21.15.152:0/1639887768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6360102ef0 msgr2=0x7f636010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:47.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.150+0000 7f6366ef9640 1 --2- 172.21.15.152:0/1639887768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6360102ef0 0x7f636010da40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6358009ca0 tx=0x7f635802a950 comp rx=0 tx=0).stop 2024-09-11T22:51:47.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.150+0000 7f6366ef9640 1 -- 172.21.15.152:0/1639887768 shutdown_connections 2024-09-11T22:51:47.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.150+0000 7f6366ef9640 1 --2- 172.21.15.152:0/1639887768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6360102ef0 0x7f636010da40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:47.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.150+0000 7f6366ef9640 1 --2- 172.21.15.152:0/1639887768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6360102570 0x7f63601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:47.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.150+0000 7f6366ef9640 1 --2- 172.21.15.152:0/1639887768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6360100e20 0x7f6360101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:47.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.150+0000 7f6366ef9640 1 -- 172.21.15.152:0/1639887768 >> 172.21.15.152:0/1639887768 conn(0x7f63600fc7d0 msgr2=0x7f63600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:47.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.150+0000 7f6366ef9640 1 -- 172.21.15.152:0/1639887768 shutdown_connections 2024-09-11T22:51:47.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.150+0000 7f6366ef9640 1 -- 172.21.15.152:0/1639887768 wait complete. 2024-09-11T22:51:47.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.151+0000 7f6366ef9640 1 Processor -- start 2024-09-11T22:51:47.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.151+0000 7f6366ef9640 1 -- start start 2024-09-11T22:51:47.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.151+0000 7f6366ef9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6360100e20 0x7f63601972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:47.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.151+0000 7f6366ef9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6360102570 0x7f63601977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:47.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.151+0000 7f6366ef9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6360102ef0 0x7f636019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:47.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.151+0000 7f6366ef9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6360110f80 con 0x7f6360102570 2024-09-11T22:51:47.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.151+0000 7f6366ef9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6360110e00 con 0x7f6360102ef0 2024-09-11T22:51:47.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.151+0000 7f6366ef9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6360111100 con 0x7f6360100e20 2024-09-11T22:51:47.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.152+0000 7f635ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6360102570 0x7f63601977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:47.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.152+0000 7f635ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6360102570 0x7f63601977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33732/0 (socket says 172.21.15.152:33732) 2024-09-11T22:51:47.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.152+0000 7f635ffff640 1 -- 172.21.15.152:0/3800406036 learned_addr learned my addr 172.21.15.152:0/3800406036 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:47.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.152+0000 7f6364c95640 1 --2- 172.21.15.152:0/3800406036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6360100e20 0x7f63601972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:47.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.152+0000 7f6365496640 1 --2- 172.21.15.152:0/3800406036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6360102ef0 0x7f636019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:47.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.152+0000 7f635ffff640 1 -- 172.21.15.152:0/3800406036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6360100e20 msgr2=0x7f63601972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:47.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.152+0000 7f635ffff640 1 --2- 172.21.15.152:0/3800406036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6360100e20 0x7f63601972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:47.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.152+0000 7f635ffff640 1 -- 172.21.15.152:0/3800406036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6360102ef0 msgr2=0x7f636019bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:47.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.152+0000 7f635ffff640 1 --2- 172.21.15.152:0/3800406036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6360102ef0 0x7f636019bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:47.685 INFO:teuthology.orchestra.run.smithi152.stdout:322122547225 2024-09-11T22:51:47.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.152+0000 7f635ffff640 1 -- 172.21.15.152:0/3800406036 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636019c250 con 0x7f6360102570 2024-09-11T22:51:47.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.152+0000 7f635ffff640 1 --2- 172.21.15.152:0/3800406036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6360102570 0x7f63601977e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f634c00c920 tx=0x7f634c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:47.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.152+0000 7f635dffb640 1 -- 172.21.15.152:0/3800406036 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c007c70 con 0x7f6360102570 2024-09-11T22:51:47.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.152+0000 7f6366ef9640 1 -- 172.21.15.152:0/3800406036 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636019c540 con 0x7f6360102570 2024-09-11T22:51:47.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.152+0000 7f635dffb640 1 -- 172.21.15.152:0/3800406036 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f634c007e10 con 0x7f6360102570 2024-09-11T22:51:47.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.152+0000 7f635dffb640 1 -- 172.21.15.152:0/3800406036 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c018890 con 0x7f6360102570 2024-09-11T22:51:47.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.153+0000 7f6366ef9640 1 -- 172.21.15.152:0/3800406036 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63601a3e20 con 0x7f6360102570 2024-09-11T22:51:47.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.154+0000 7f6366ef9640 1 -- 172.21.15.152:0/3800406036 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6360101200 con 0x7f6360102570 2024-09-11T22:51:47.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.155+0000 7f635dffb640 1 -- 172.21.15.152:0/3800406036 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f634c018a50 con 0x7f6360102570 2024-09-11T22:51:47.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.155+0000 7f635dffb640 1 --2- 172.21.15.152:0/3800406036 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f63500a2c10 0x7f63500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:47.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.156+0000 7f635dffb640 1 -- 172.21.15.152:0/3800406036 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f634c0cf350 con 0x7f6360102570 2024-09-11T22:51:47.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.156+0000 7f6364c95640 1 --2- 172.21.15.152:0/3800406036 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f63500a2c10 0x7f63500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:47.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.156+0000 7f6364c95640 1 --2- 172.21.15.152:0/3800406036 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f63500a2c10 0x7f63500a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f635400bbe0 tx=0x7f635400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:47.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.159+0000 7f635dffb640 1 -- 172.21.15.152:0/3800406036 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f634c014030 con 0x7f6360102570 2024-09-11T22:51:47.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.333+0000 7f6366ef9640 1 -- 172.21.15.152:0/3800406036 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f6360198600 con 0x7f6360102570 2024-09-11T22:51:47.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.334+0000 7f635dffb640 1 -- 172.21.15.152:0/3800406036 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6360198600 con 0x7f6360102570 2024-09-11T22:51:47.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.336+0000 7f6366ef9640 1 -- 172.21.15.152:0/3800406036 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f63500a2c10 msgr2=0x7f63500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:47.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.336+0000 7f6366ef9640 1 --2- 172.21.15.152:0/3800406036 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f63500a2c10 0x7f63500a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f635400bbe0 tx=0x7f635400b390 comp rx=0 tx=0).stop 2024-09-11T22:51:47.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.336+0000 7f6366ef9640 1 -- 172.21.15.152:0/3800406036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6360102570 msgr2=0x7f63601977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:47.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.336+0000 7f6366ef9640 1 --2- 172.21.15.152:0/3800406036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6360102570 0x7f63601977e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f634c00c920 tx=0x7f634c00cde0 comp rx=0 tx=0).stop 2024-09-11T22:51:47.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.336+0000 7f6366ef9640 1 -- 172.21.15.152:0/3800406036 shutdown_connections 2024-09-11T22:51:47.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.336+0000 7f6366ef9640 1 --2- 172.21.15.152:0/3800406036 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f63500a2c10 0x7f63500a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:47.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.336+0000 7f6366ef9640 1 --2- 172.21.15.152:0/3800406036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6360102ef0 0x7f636019bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:47.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.336+0000 7f6366ef9640 1 --2- 172.21.15.152:0/3800406036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6360102570 0x7f63601977e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:47.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.336+0000 7f6366ef9640 1 --2- 172.21.15.152:0/3800406036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6360100e20 0x7f63601972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:47.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.336+0000 7f6366ef9640 1 -- 172.21.15.152:0/3800406036 >> 172.21.15.152:0/3800406036 conn(0x7f63600fc7d0 msgr2=0x7f63601067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:47.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.336+0000 7f6366ef9640 1 -- 172.21.15.152:0/3800406036 shutdown_connections 2024-09-11T22:51:47.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:47.336+0000 7f6366ef9640 1 -- 172.21.15.152:0/3800406036 wait complete. 2024-09-11T22:51:48.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:47 smithi204 ceph-mon[35454]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-11T22:51:48.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:47 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3800406036' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T22:51:48.178 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:47 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:51:48.214 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:47 smithi191 ceph-mon[35211]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-11T22:51:48.214 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:47 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3800406036' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T22:51:48.214 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:47 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:51:48.313 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:47 smithi152 ceph-mon[31181]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-11T22:51:48.313 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:47 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3800406036' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T22:51:48.313 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:47 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:51:48.340 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547222 got 322122547225 for osd.11 2024-09-11T22:51:48.341 DEBUG:teuthology.parallel:result is None 2024-09-11T22:51:48.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.505+0000 7fcd672b5640 1 -- 172.21.15.152:0/3761948329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd60100a80 msgr2=0x7fcd60100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:48.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.505+0000 7fcd672b5640 1 --2- 172.21.15.152:0/3761948329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd60100a80 0x7fcd60100ee0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009930 tx=0x7fcd5402a420 comp rx=0 tx=0).stop 2024-09-11T22:51:48.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.505+0000 7fcd672b5640 1 -- 172.21.15.152:0/3761948329 shutdown_connections 2024-09-11T22:51:48.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.505+0000 7fcd672b5640 1 --2- 172.21.15.152:0/3761948329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd60101420 0x7fcd6010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.505+0000 7fcd672b5640 1 --2- 172.21.15.152:0/3761948329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd60100a80 0x7fcd60100ee0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.505+0000 7fcd672b5640 1 --2- 172.21.15.152:0/3761948329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd60106a80 0x7fcd60106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.505+0000 7fcd672b5640 1 -- 172.21.15.152:0/3761948329 >> 172.21.15.152:0/3761948329 conn(0x7fcd600fc7a0 msgr2=0x7fcd600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:48.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.505+0000 7fcd672b5640 1 -- 172.21.15.152:0/3761948329 shutdown_connections 2024-09-11T22:51:48.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.505+0000 7fcd672b5640 1 -- 172.21.15.152:0/3761948329 wait complete. 2024-09-11T22:51:48.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.505+0000 7fcd672b5640 1 Processor -- start 2024-09-11T22:51:48.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd672b5640 1 -- start start 2024-09-11T22:51:48.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd672b5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd60100a80 0x7fcd60198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:48.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd672b5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd60101420 0x7fcd60198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:48.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd672b5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd60106a80 0x7fcd6019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:48.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd672b5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60110e80 con 0x7fcd60101420 2024-09-11T22:51:48.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd672b5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60110d00 con 0x7fcd60106a80 2024-09-11T22:51:48.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd672b5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60111000 con 0x7fcd60100a80 2024-09-11T22:51:48.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd64850640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd60101420 0x7fcd60198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:48.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd64850640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd60101420 0x7fcd60198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51906/0 (socket says 172.21.15.152:51906) 2024-09-11T22:51:48.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd64850640 1 -- 172.21.15.152:0/3514651952 learned_addr learned my addr 172.21.15.152:0/3514651952 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:48.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd65051640 1 --2- 172.21.15.152:0/3514651952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd60100a80 0x7fcd60198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:48.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd65852640 1 --2- 172.21.15.152:0/3514651952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd60106a80 0x7fcd6019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:48.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd64850640 1 -- 172.21.15.152:0/3514651952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd60100a80 msgr2=0x7fcd60198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:48.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd64850640 1 --2- 172.21.15.152:0/3514651952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd60100a80 0x7fcd60198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd64850640 1 -- 172.21.15.152:0/3514651952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd60106a80 msgr2=0x7fcd6019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:48.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd64850640 1 --2- 172.21.15.152:0/3514651952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd60106a80 0x7fcd6019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd64850640 1 -- 172.21.15.152:0/3514651952 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6019d3f0 con 0x7fcd60101420 2024-09-11T22:51:48.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd65852640 1 --2- 172.21.15.152:0/3514651952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd60106a80 0x7fcd6019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:51:48.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.506+0000 7fcd64850640 1 --2- 172.21.15.152:0/3514651952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd60101420 0x7fcd60198980 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009230 tx=0x7fcd54004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:48.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.507+0000 7fcd5e7fc640 1 -- 172.21.15.152:0/3514651952 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5403f030 con 0x7fcd60101420 2024-09-11T22:51:48.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.507+0000 7fcd5e7fc640 1 -- 172.21.15.152:0/3514651952 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd54003ad0 con 0x7fcd60101420 2024-09-11T22:51:48.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.507+0000 7fcd672b5640 1 -- 172.21.15.152:0/3514651952 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6019d680 con 0x7fcd60101420 2024-09-11T22:51:48.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.507+0000 7fcd5e7fc640 1 -- 172.21.15.152:0/3514651952 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54035070 con 0x7fcd60101420 2024-09-11T22:51:48.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.507+0000 7fcd672b5640 1 -- 172.21.15.152:0/3514651952 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd601a4fc0 con 0x7fcd60101420 2024-09-11T22:51:48.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.508+0000 7fcd672b5640 1 -- 172.21.15.152:0/3514651952 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd60100ee0 con 0x7fcd60101420 2024-09-11T22:51:48.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.509+0000 7fcd5e7fc640 1 -- 172.21.15.152:0/3514651952 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fcd54042030 con 0x7fcd60101420 2024-09-11T22:51:48.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.509+0000 7fcd5e7fc640 1 --2- 172.21.15.152:0/3514651952 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcd480a2c10 0x7fcd480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:48.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.510+0000 7fcd65051640 1 --2- 172.21.15.152:0/3514651952 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcd480a2c10 0x7fcd480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:48.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.510+0000 7fcd5e7fc640 1 -- 172.21.15.152:0/3514651952 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fcd540efae0 con 0x7fcd60101420 2024-09-11T22:51:48.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.510+0000 7fcd65051640 1 --2- 172.21.15.152:0/3514651952 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcd480a2c10 0x7fcd480a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcd58005c40 tx=0x7fcd5800a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:48.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.512+0000 7fcd5e7fc640 1 -- 172.21.15.152:0/3514651952 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd540bca90 con 0x7fcd60101420 2024-09-11T22:51:48.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.686+0000 7fcd672b5640 1 -- 172.21.15.152:0/3514651952 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fcd6010a000 con 0x7fcd60101420 2024-09-11T22:51:48.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.686+0000 7fcd5e7fc640 1 -- 172.21.15.152:0/3514651952 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcd540bc390 con 0x7fcd60101420 2024-09-11T22:51:48.687 INFO:teuthology.orchestra.run.smithi152.stdout:270582939683 2024-09-11T22:51:48.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.688+0000 7fcd672b5640 1 -- 172.21.15.152:0/3514651952 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcd480a2c10 msgr2=0x7fcd480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:48.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.688+0000 7fcd672b5640 1 --2- 172.21.15.152:0/3514651952 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcd480a2c10 0x7fcd480a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcd58005c40 tx=0x7fcd5800a590 comp rx=0 tx=0).stop 2024-09-11T22:51:48.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.688+0000 7fcd672b5640 1 -- 172.21.15.152:0/3514651952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd60101420 msgr2=0x7fcd60198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:48.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.688+0000 7fcd672b5640 1 --2- 172.21.15.152:0/3514651952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd60101420 0x7fcd60198980 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009230 tx=0x7fcd54004270 comp rx=0 tx=0).stop 2024-09-11T22:51:48.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.688+0000 7fcd672b5640 1 -- 172.21.15.152:0/3514651952 shutdown_connections 2024-09-11T22:51:48.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.688+0000 7fcd672b5640 1 --2- 172.21.15.152:0/3514651952 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcd480a2c10 0x7fcd480a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.688+0000 7fcd672b5640 1 --2- 172.21.15.152:0/3514651952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd60106a80 0x7fcd6019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.688+0000 7fcd672b5640 1 --2- 172.21.15.152:0/3514651952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd60101420 0x7fcd60198980 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.688+0000 7fcd672b5640 1 --2- 172.21.15.152:0/3514651952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd60100a80 0x7fcd60198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.688+0000 7fcd672b5640 1 -- 172.21.15.152:0/3514651952 >> 172.21.15.152:0/3514651952 conn(0x7fcd600fc7a0 msgr2=0x7fcd6006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:48.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.688+0000 7fcd672b5640 1 -- 172.21.15.152:0/3514651952 shutdown_connections 2024-09-11T22:51:48.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.688+0000 7fcd672b5640 1 -- 172.21.15.152:0/3514651952 wait complete. 2024-09-11T22:51:48.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.732+0000 7f3ef1bd0640 1 -- 172.21.15.152:0/683292879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec106f10 msgr2=0x7f3eec10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:48.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.732+0000 7f3ef1bd0640 1 --2- 172.21.15.152:0/683292879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec106f10 0x7f3eec10ea90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009ca0 tx=0x7f3ee402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.732+0000 7f3ef1bd0640 1 -- 172.21.15.152:0/683292879 shutdown_connections 2024-09-11T22:51:48.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.732+0000 7f3ef1bd0640 1 --2- 172.21.15.152:0/683292879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec106f10 0x7f3eec10ea90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.732+0000 7f3ef1bd0640 1 --2- 172.21.15.152:0/683292879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3eec102790 0x7f3eec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.732+0000 7f3ef1bd0640 1 --2- 172.21.15.152:0/683292879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eec101de0 0x7f3eec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.732+0000 7f3ef1bd0640 1 -- 172.21.15.152:0/683292879 >> 172.21.15.152:0/683292879 conn(0x7f3eec0fd8c0 msgr2=0x7f3eec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:48.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.732+0000 7f3ef1bd0640 1 -- 172.21.15.152:0/683292879 shutdown_connections 2024-09-11T22:51:48.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.732+0000 7f3ef1bd0640 1 -- 172.21.15.152:0/683292879 wait complete. 2024-09-11T22:51:48.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.733+0000 7f3ef1bd0640 1 Processor -- start 2024-09-11T22:51:48.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.733+0000 7f3ef1bd0640 1 -- start start 2024-09-11T22:51:48.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.733+0000 7f3ef1bd0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eec101de0 0x7f3eec1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:48.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.733+0000 7f3ef1bd0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3eec102790 0x7f3eec198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:48.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.733+0000 7f3ef1bd0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec106f10 0x7f3eec19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:48.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.733+0000 7f3ef1bd0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec111f70 con 0x7f3eec106f10 2024-09-11T22:51:48.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.733+0000 7f3ef1bd0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec111df0 con 0x7f3eec102790 2024-09-11T22:51:48.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.733+0000 7f3ef1bd0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3eec1120f0 con 0x7f3eec101de0 2024-09-11T22:51:48.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.733+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec106f10 0x7f3eec19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:48.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.733+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec106f10 0x7f3eec19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51926/0 (socket says 172.21.15.152:51926) 2024-09-11T22:51:48.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.733+0000 7f3eebfff640 1 -- 172.21.15.152:0/2651630198 learned_addr learned my addr 172.21.15.152:0/2651630198 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:48.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.733+0000 7f3eeaffd640 1 --2- 172.21.15.152:0/2651630198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3eec102790 0x7f3eec198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:48.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.733+0000 7f3eeb7fe640 1 --2- 172.21.15.152:0/2651630198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eec101de0 0x7f3eec1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:48.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.734+0000 7f3eebfff640 1 -- 172.21.15.152:0/2651630198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eec101de0 msgr2=0x7f3eec1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:48.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.734+0000 7f3eebfff640 1 --2- 172.21.15.152:0/2651630198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eec101de0 0x7f3eec1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.734+0000 7f3eebfff640 1 -- 172.21.15.152:0/2651630198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3eec102790 msgr2=0x7f3eec198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:48.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.734+0000 7f3eebfff640 1 --2- 172.21.15.152:0/2651630198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3eec102790 0x7f3eec198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.734+0000 7f3eebfff640 1 -- 172.21.15.152:0/2651630198 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eec19d380 con 0x7f3eec106f10 2024-09-11T22:51:48.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.734+0000 7f3eebfff640 1 --2- 172.21.15.152:0/2651630198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec106f10 0x7f3eec19cca0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4031a20 tx=0x7f3ee4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:48.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.734+0000 7f3ee8ff9640 1 -- 172.21.15.152:0/2651630198 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4003c30 con 0x7f3eec106f10 2024-09-11T22:51:48.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.734+0000 7f3ee8ff9640 1 -- 172.21.15.152:0/2651630198 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ee4003dd0 con 0x7f3eec106f10 2024-09-11T22:51:48.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.734+0000 7f3ef1bd0640 1 -- 172.21.15.152:0/2651630198 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eec19d610 con 0x7f3eec106f10 2024-09-11T22:51:48.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.734+0000 7f3ee8ff9640 1 -- 172.21.15.152:0/2651630198 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee402fc40 con 0x7f3eec106f10 2024-09-11T22:51:48.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.734+0000 7f3ef1bd0640 1 -- 172.21.15.152:0/2651630198 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eec1a4f50 con 0x7f3eec106f10 2024-09-11T22:51:48.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.735+0000 7f3ef1bd0640 1 -- 172.21.15.152:0/2651630198 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eec1021c0 con 0x7f3eec106f10 2024-09-11T22:51:48.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.736+0000 7f3ee8ff9640 1 -- 172.21.15.152:0/2651630198 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f3ee4004050 con 0x7f3eec106f10 2024-09-11T22:51:48.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.737+0000 7f3ee8ff9640 1 --2- 172.21.15.152:0/2651630198 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3ed40a2c90 0x7f3ed40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:48.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.737+0000 7f3eeb7fe640 1 --2- 172.21.15.152:0/2651630198 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3ed40a2c90 0x7f3ed40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:48.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.737+0000 7f3ee8ff9640 1 -- 172.21.15.152:0/2651630198 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f3ee40f02d0 con 0x7f3eec106f10 2024-09-11T22:51:48.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.738+0000 7f3eeb7fe640 1 --2- 172.21.15.152:0/2651630198 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3ed40a2c90 0x7f3ed40a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3edc005fd0 tx=0x7f3edc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:48.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.740+0000 7f3ee8ff9640 1 -- 172.21.15.152:0/2651630198 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ee40bd320 con 0x7f3eec106f10 2024-09-11T22:51:48.766 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.926+0000 7f3ef1bd0640 1 -- 172.21.15.152:0/2651630198 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3eec199730 con 0x7f3eec106f10 2024-09-11T22:51:48.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.926+0000 7f3ee8ff9640 1 -- 172.21.15.152:0/2651630198 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3eec199730 con 0x7f3eec106f10 2024-09-11T22:51:48.928 INFO:teuthology.orchestra.run.smithi152.stdout:34359738449 2024-09-11T22:51:48.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.928+0000 7f3ef1bd0640 1 -- 172.21.15.152:0/2651630198 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3ed40a2c90 msgr2=0x7f3ed40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:48.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.928+0000 7f3ef1bd0640 1 --2- 172.21.15.152:0/2651630198 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3ed40a2c90 0x7f3ed40a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3edc005fd0 tx=0x7f3edc00b390 comp rx=0 tx=0).stop 2024-09-11T22:51:48.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.928+0000 7f3ef1bd0640 1 -- 172.21.15.152:0/2651630198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec106f10 msgr2=0x7f3eec19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:48.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.928+0000 7f3ef1bd0640 1 --2- 172.21.15.152:0/2651630198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec106f10 0x7f3eec19cca0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4031a20 tx=0x7f3ee4031a50 comp rx=0 tx=0).stop 2024-09-11T22:51:48.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.928+0000 7f3ef1bd0640 1 -- 172.21.15.152:0/2651630198 shutdown_connections 2024-09-11T22:51:48.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.928+0000 7f3ef1bd0640 1 --2- 172.21.15.152:0/2651630198 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f3ed40a2c90 0x7f3ed40a5150 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.928+0000 7f3ef1bd0640 1 --2- 172.21.15.152:0/2651630198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eec106f10 0x7f3eec19cca0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.928+0000 7f3ef1bd0640 1 --2- 172.21.15.152:0/2651630198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3eec102790 0x7f3eec198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.928+0000 7f3ef1bd0640 1 --2- 172.21.15.152:0/2651630198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3eec101de0 0x7f3eec1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:48.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.928+0000 7f3ef1bd0640 1 -- 172.21.15.152:0/2651630198 >> 172.21.15.152:0/2651630198 conn(0x7f3eec0fd8c0 msgr2=0x7f3eec0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:48.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.929+0000 7f3ef1bd0640 1 -- 172.21.15.152:0/2651630198 shutdown_connections 2024-09-11T22:51:48.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.929+0000 7f3ef1bd0640 1 -- 172.21.15.152:0/2651630198 wait complete. 2024-09-11T22:51:49.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.992+0000 7febda145640 1 -- 172.21.15.152:0/3355008488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7febd4102ef0 msgr2=0x7febd410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:49.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.992+0000 7febda145640 1 --2- 172.21.15.152:0/3355008488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7febd4102ef0 0x7febd410da40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7febcc009ca0 tx=0x7febcc02a950 comp rx=0 tx=0).stop 2024-09-11T22:51:49.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.992+0000 7febda145640 1 -- 172.21.15.152:0/3355008488 shutdown_connections 2024-09-11T22:51:49.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.992+0000 7febda145640 1 --2- 172.21.15.152:0/3355008488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7febd4102ef0 0x7febd410da40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:49.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.992+0000 7febda145640 1 --2- 172.21.15.152:0/3355008488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febd4102570 0x7febd41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:49.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.992+0000 7febda145640 1 --2- 172.21.15.152:0/3355008488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febd4100e20 0x7febd4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:49.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.992+0000 7febda145640 1 -- 172.21.15.152:0/3355008488 >> 172.21.15.152:0/3355008488 conn(0x7febd40fc7d0 msgr2=0x7febd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:49.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.992+0000 7febda145640 1 -- 172.21.15.152:0/3355008488 shutdown_connections 2024-09-11T22:51:49.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.992+0000 7febda145640 1 -- 172.21.15.152:0/3355008488 wait complete. 2024-09-11T22:51:49.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.992+0000 7febda145640 1 Processor -- start 2024-09-11T22:51:49.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.992+0000 7febda145640 1 -- start start 2024-09-11T22:51:49.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.992+0000 7febda145640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7febd4100e20 0x7febd419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:49.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.992+0000 7febda145640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febd4102570 0x7febd419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:49.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.992+0000 7febda145640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febd4102ef0 0x7febd419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:49.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.992+0000 7febda145640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7febd4110f80 con 0x7febd4100e20 2024-09-11T22:51:49.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febda145640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7febd4110e00 con 0x7febd4102ef0 2024-09-11T22:51:49.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febda145640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7febd4111100 con 0x7febd4102570 2024-09-11T22:51:49.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febd37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7febd4100e20 0x7febd419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:49.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febd37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7febd4100e20 0x7febd419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51940/0 (socket says 172.21.15.152:51940) 2024-09-11T22:51:49.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febd37fe640 1 -- 172.21.15.152:0/3167640190 learned_addr learned my addr 172.21.15.152:0/3167640190 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:49.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febd3fff640 1 --2- 172.21.15.152:0/3167640190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febd4102ef0 0x7febd419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:49.012 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febcbfff640 1 --2- 172.21.15.152:0/3167640190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febd4102570 0x7febd419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:49.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febd37fe640 1 -- 172.21.15.152:0/3167640190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febd4102570 msgr2=0x7febd419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:49.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febd37fe640 1 --2- 172.21.15.152:0/3167640190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febd4102570 0x7febd419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:49.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febd37fe640 1 -- 172.21.15.152:0/3167640190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febd4102ef0 msgr2=0x7febd419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:49.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febd37fe640 1 --2- 172.21.15.152:0/3167640190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febd4102ef0 0x7febd419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:49.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febd37fe640 1 -- 172.21.15.152:0/3167640190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd419f6a0 con 0x7febd4100e20 2024-09-11T22:51:49.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febd37fe640 1 --2- 172.21.15.152:0/3167640190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7febd4100e20 0x7febd419a6f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7febc400d950 tx=0x7febc400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:49.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febd17fa640 1 -- 172.21.15.152:0/3167640190 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4017070 con 0x7febd4100e20 2024-09-11T22:51:49.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febd17fa640 1 -- 172.21.15.152:0/3167640190 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febc4009c30 con 0x7febd4100e20 2024-09-11T22:51:49.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febd17fa640 1 -- 172.21.15.152:0/3167640190 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc400f6b0 con 0x7febd4100e20 2024-09-11T22:51:49.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febda145640 1 -- 172.21.15.152:0/3167640190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd419f990 con 0x7febd4100e20 2024-09-11T22:51:49.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.993+0000 7febda145640 1 -- 172.21.15.152:0/3167640190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd41a7270 con 0x7febd4100e20 2024-09-11T22:51:49.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.994+0000 7febda145640 1 -- 172.21.15.152:0/3167640190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd4101200 con 0x7febd4100e20 2024-09-11T22:51:49.013 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.995+0000 7febd17fa640 1 -- 172.21.15.152:0/3167640190 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7febc40109d0 con 0x7febd4100e20 2024-09-11T22:51:49.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.996+0000 7febd17fa640 1 --2- 172.21.15.152:0/3167640190 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7febc00a2c10 0x7febc00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:49.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.996+0000 7febcbfff640 1 --2- 172.21.15.152:0/3167640190 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7febc00a2c10 0x7febc00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:49.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.997+0000 7febd17fa640 1 -- 172.21.15.152:0/3167640190 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7febc40cec70 con 0x7febd4100e20 2024-09-11T22:51:49.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:48.997+0000 7febcbfff640 1 --2- 172.21.15.152:0/3167640190 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7febc00a2c10 0x7febc00a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7febd40ff0c0 tx=0x7febbc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:49.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:49.000+0000 7febd17fa640 1 -- 172.21.15.152:0/3167640190 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7febc4009de0 con 0x7febd4100e20 2024-09-11T22:51:49.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:49.175+0000 7febda145640 1 -- 172.21.15.152:0/3167640190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7febd41a7440 con 0x7febd4100e20 2024-09-11T22:51:49.176 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:48 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3514651952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T22:51:49.176 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:48 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:51:49.177 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:48 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:51:49.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:49.175+0000 7febd17fa640 1 -- 172.21.15.152:0/3167640190 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7febc409b510 con 0x7febd4100e20 2024-09-11T22:51:49.177 INFO:teuthology.orchestra.run.smithi152.stdout:158913790008 2024-09-11T22:51:49.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:49.178+0000 7febda145640 1 -- 172.21.15.152:0/3167640190 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7febc00a2c10 msgr2=0x7febc00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:49.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:49.178+0000 7febda145640 1 --2- 172.21.15.152:0/3167640190 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7febc00a2c10 0x7febc00a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7febd40ff0c0 tx=0x7febbc005f90 comp rx=0 tx=0).stop 2024-09-11T22:51:49.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:49.178+0000 7febda145640 1 -- 172.21.15.152:0/3167640190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7febd4100e20 msgr2=0x7febd419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:49.178+0000 7febda145640 1 --2- 172.21.15.152:0/3167640190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7febd4100e20 0x7febd419a6f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7febc400d950 tx=0x7febc400de10 comp rx=0 tx=0).stop 2024-09-11T22:51:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:49.178+0000 7febda145640 1 -- 172.21.15.152:0/3167640190 shutdown_connections 2024-09-11T22:51:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:49.178+0000 7febda145640 1 --2- 172.21.15.152:0/3167640190 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7febc00a2c10 0x7febc00a50d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:49.178+0000 7febda145640 1 --2- 172.21.15.152:0/3167640190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7febd4102ef0 0x7febd419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:49.178+0000 7febda145640 1 --2- 172.21.15.152:0/3167640190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febd4102570 0x7febd419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:49.178+0000 7febda145640 1 --2- 172.21.15.152:0/3167640190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7febd4100e20 0x7febd419a6f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:49.178+0000 7febda145640 1 -- 172.21.15.152:0/3167640190 >> 172.21.15.152:0/3167640190 conn(0x7febd40fc7d0 msgr2=0x7febd406a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:49.178+0000 7febda145640 1 -- 172.21.15.152:0/3167640190 shutdown_connections 2024-09-11T22:51:49.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:49.178+0000 7febda145640 1 -- 172.21.15.152:0/3167640190 wait complete. 2024-09-11T22:51:49.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:48 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3514651952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T22:51:49.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:48 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:51:49.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:48 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:51:49.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:48 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3514651952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T22:51:49.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:48 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:51:49.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:48 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:51:50.217 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:49 smithi152 ceph-mon[31181]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T22:51:50.217 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:49 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2651630198' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T22:51:50.217 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:49 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:51:50.217 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:49 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:51:50.217 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:49 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3167640190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T22:51:50.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:49 smithi191 ceph-mon[35211]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T22:51:50.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:49 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2651630198' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T22:51:50.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:49 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:51:50.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:49 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:51:50.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:49 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3167640190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T22:51:50.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:49 smithi204 ceph-mon[35454]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T22:51:50.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:49 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2651630198' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T22:51:50.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:49 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:51:50.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:49 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:51:50.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:49 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3167640190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T22:51:50.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.299+0000 7f84746ff640 1 -- 172.21.15.152:0/3795123794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f846c101de0 msgr2=0x7f846c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:50.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.299+0000 7f84746ff640 1 --2- 172.21.15.152:0/3795123794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f846c101de0 0x7f846c1021c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8468009900 tx=0x7f846802a410 comp rx=0 tx=0).stop 2024-09-11T22:51:50.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.299+0000 7f84746ff640 1 -- 172.21.15.152:0/3795123794 shutdown_connections 2024-09-11T22:51:50.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.299+0000 7f84746ff640 1 --2- 172.21.15.152:0/3795123794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f846c106f10 0x7f846c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:50.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.299+0000 7f84746ff640 1 --2- 172.21.15.152:0/3795123794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f846c102790 0x7f846c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:50.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.299+0000 7f84746ff640 1 --2- 172.21.15.152:0/3795123794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f846c101de0 0x7f846c1021c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:50.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.299+0000 7f84746ff640 1 -- 172.21.15.152:0/3795123794 >> 172.21.15.152:0/3795123794 conn(0x7f846c0fd8c0 msgr2=0x7f846c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:50.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.299+0000 7f84746ff640 1 -- 172.21.15.152:0/3795123794 shutdown_connections 2024-09-11T22:51:50.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.299+0000 7f84746ff640 1 -- 172.21.15.152:0/3795123794 wait complete. 2024-09-11T22:51:50.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.300+0000 7f84746ff640 1 Processor -- start 2024-09-11T22:51:50.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.300+0000 7f84746ff640 1 -- start start 2024-09-11T22:51:50.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.300+0000 7f84746ff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f846c101de0 0x7f846c1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:50.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.300+0000 7f84746ff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f846c102790 0x7f846c1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:50.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.300+0000 7f84746ff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f846c106f10 0x7f846c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:50.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.300+0000 7f84746ff640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f846c112010 con 0x7f846c102790 2024-09-11T22:51:50.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.300+0000 7f84746ff640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f846c111e90 con 0x7f846c106f10 2024-09-11T22:51:50.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.300+0000 7f84746ff640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f846c112190 con 0x7f846c101de0 2024-09-11T22:51:50.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.300+0000 7f8471c9a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f846c102790 0x7f846c1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:50.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.300+0000 7f8471c9a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f846c102790 0x7f846c1a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51960/0 (socket says 172.21.15.152:51960) 2024-09-11T22:51:50.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.300+0000 7f8471c9a640 1 -- 172.21.15.152:0/2341969079 learned_addr learned my addr 172.21.15.152:0/2341969079 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:50.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.300+0000 7f847249b640 1 --2- 172.21.15.152:0/2341969079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f846c101de0 0x7f846c1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:50.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.301+0000 7f8471c9a640 1 -- 172.21.15.152:0/2341969079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f846c101de0 msgr2=0x7f846c1a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:50.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.301+0000 7f8471c9a640 1 --2- 172.21.15.152:0/2341969079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f846c101de0 0x7f846c1a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:50.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.301+0000 7f8471c9a640 1 -- 172.21.15.152:0/2341969079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f846c106f10 msgr2=0x7f846c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:50.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.301+0000 7f8472c9c640 1 --2- 172.21.15.152:0/2341969079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f846c106f10 0x7f846c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:50.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.301+0000 7f8471c9a640 1 --2- 172.21.15.152:0/2341969079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f846c106f10 0x7f846c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:50.957 INFO:teuthology.orchestra.run.smithi152.stdout:240518168615 2024-09-11T22:51:50.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.301+0000 7f8471c9a640 1 -- 172.21.15.152:0/2341969079 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846c19ad30 con 0x7f846c102790 2024-09-11T22:51:50.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.301+0000 7f847249b640 1 --2- 172.21.15.152:0/2341969079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f846c101de0 0x7f846c1a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:50.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.301+0000 7f8472c9c640 1 --2- 172.21.15.152:0/2341969079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f846c106f10 0x7f846c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:50.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.301+0000 7f8471c9a640 1 --2- 172.21.15.152:0/2341969079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f846c102790 0x7f846c1a0980 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f845c00b4a0 tx=0x7f845c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:50.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.301+0000 7f84637fe640 1 -- 172.21.15.152:0/2341969079 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c014070 con 0x7f846c102790 2024-09-11T22:51:50.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.301+0000 7f84637fe640 1 -- 172.21.15.152:0/2341969079 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f845c010af0 con 0x7f846c102790 2024-09-11T22:51:50.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.301+0000 7f84746ff640 1 -- 172.21.15.152:0/2341969079 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846c19b020 con 0x7f846c102790 2024-09-11T22:51:50.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.301+0000 7f84637fe640 1 -- 172.21.15.152:0/2341969079 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c017d10 con 0x7f846c102790 2024-09-11T22:51:50.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.301+0000 7f84746ff640 1 -- 172.21.15.152:0/2341969079 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846c1a7310 con 0x7f846c102790 2024-09-11T22:51:50.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.302+0000 7f84746ff640 1 -- 172.21.15.152:0/2341969079 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f846c102240 con 0x7f846c102790 2024-09-11T22:51:50.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.303+0000 7f84637fe640 1 -- 172.21.15.152:0/2341969079 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f845c017070 con 0x7f846c102790 2024-09-11T22:51:50.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.304+0000 7f84637fe640 1 --2- 172.21.15.152:0/2341969079 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f84580a2c90 0x7f84580a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:50.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.304+0000 7f847249b640 1 --2- 172.21.15.152:0/2341969079 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f84580a2c90 0x7f84580a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:50.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.304+0000 7f84637fe640 1 -- 172.21.15.152:0/2341969079 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f845c0ce7a0 con 0x7f846c102790 2024-09-11T22:51:50.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.305+0000 7f847249b640 1 --2- 172.21.15.152:0/2341969079 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f84580a2c90 0x7f84580a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8468000c00 tx=0x7f8468032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:50.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.308+0000 7f84637fe640 1 -- 172.21.15.152:0/2341969079 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f845c09b760 con 0x7f846c102790 2024-09-11T22:51:50.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.478+0000 7f84746ff640 1 -- 172.21.15.152:0/2341969079 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f846c19bb30 con 0x7f846c102790 2024-09-11T22:51:50.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.478+0000 7f84637fe640 1 -- 172.21.15.152:0/2341969079 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f846c19bb30 con 0x7f846c102790 2024-09-11T22:51:50.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.481+0000 7f84746ff640 1 -- 172.21.15.152:0/2341969079 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f84580a2c90 msgr2=0x7f84580a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:50.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.481+0000 7f84746ff640 1 --2- 172.21.15.152:0/2341969079 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f84580a2c90 0x7f84580a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8468000c00 tx=0x7f8468032d50 comp rx=0 tx=0).stop 2024-09-11T22:51:50.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.481+0000 7f84746ff640 1 -- 172.21.15.152:0/2341969079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f846c102790 msgr2=0x7f846c1a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:50.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.481+0000 7f84746ff640 1 --2- 172.21.15.152:0/2341969079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f846c102790 0x7f846c1a0980 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f845c00b4a0 tx=0x7f845c00b960 comp rx=0 tx=0).stop 2024-09-11T22:51:50.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.481+0000 7f84746ff640 1 -- 172.21.15.152:0/2341969079 shutdown_connections 2024-09-11T22:51:50.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.481+0000 7f84746ff640 1 --2- 172.21.15.152:0/2341969079 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f84580a2c90 0x7f84580a5150 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:50.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.481+0000 7f84746ff640 1 --2- 172.21.15.152:0/2341969079 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f846c106f10 0x7f846c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:50.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.481+0000 7f84746ff640 1 --2- 172.21.15.152:0/2341969079 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f846c102790 0x7f846c1a0980 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:50.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.481+0000 7f84746ff640 1 --2- 172.21.15.152:0/2341969079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f846c101de0 0x7f846c1a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:50.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.481+0000 7f84746ff640 1 -- 172.21.15.152:0/2341969079 >> 172.21.15.152:0/2341969079 conn(0x7f846c0fd8c0 msgr2=0x7f846c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:50.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.481+0000 7f84746ff640 1 -- 172.21.15.152:0/2341969079 shutdown_connections 2024-09-11T22:51:50.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:50.481+0000 7f84746ff640 1 -- 172.21.15.152:0/2341969079 wait complete. 2024-09-11T22:51:51.276 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:50 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2341969079' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T22:51:51.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:50 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2341969079' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T22:51:51.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:50 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2341969079' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T22:51:51.636 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168613 got 240518168615 for osd.8 2024-09-11T22:51:51.636 DEBUG:teuthology.parallel:result is None 2024-09-11T22:51:52.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.086+0000 7f69048a7640 1 -- 172.21.15.152:0/853766553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6900102570 msgr2=0x7f69001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.086+0000 7f69048a7640 1 --2- 172.21.15.152:0/853766553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6900102570 0x7f69001029b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009900 tx=0x7f68e802a480 comp rx=0 tx=0).stop 2024-09-11T22:51:52.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.086+0000 7f69048a7640 1 -- 172.21.15.152:0/853766553 shutdown_connections 2024-09-11T22:51:52.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.086+0000 7f69048a7640 1 --2- 172.21.15.152:0/853766553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6900102ef0 0x7f690010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.086+0000 7f69048a7640 1 --2- 172.21.15.152:0/853766553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6900102570 0x7f69001029b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.086+0000 7f69048a7640 1 --2- 172.21.15.152:0/853766553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6900100e20 0x7f6900101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.086+0000 7f69048a7640 1 -- 172.21.15.152:0/853766553 >> 172.21.15.152:0/853766553 conn(0x7f69000fc7d0 msgr2=0x7f69000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:52.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.086+0000 7f69048a7640 1 -- 172.21.15.152:0/853766553 shutdown_connections 2024-09-11T22:51:52.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.086+0000 7f69048a7640 1 -- 172.21.15.152:0/853766553 wait complete. 2024-09-11T22:51:52.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f69048a7640 1 Processor -- start 2024-09-11T22:51:52.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f69048a7640 1 -- start start 2024-09-11T22:51:52.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f69048a7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6900100e20 0x7f69001972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:52.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f69048a7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6900102570 0x7f69001977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:52.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f69048a7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6900102ef0 0x7f6900197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:52.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f69048a7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6900111020 con 0x7f6900100e20 2024-09-11T22:51:52.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f69048a7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6900110ea0 con 0x7f6900102570 2024-09-11T22:51:52.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f69048a7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f69001111a0 con 0x7f6900102ef0 2024-09-11T22:51:52.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f68fe59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6900100e20 0x7f69001972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:52.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f68fe59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6900100e20 0x7f69001972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51974/0 (socket says 172.21.15.152:51974) 2024-09-11T22:51:52.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f68fe59c640 1 -- 172.21.15.152:0/3813082958 learned_addr learned my addr 172.21.15.152:0/3813082958 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:52.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f68fdd9b640 1 --2- 172.21.15.152:0/3813082958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6900102570 0x7f69001977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:52.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f68fe59c640 1 -- 172.21.15.152:0/3813082958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6900102ef0 msgr2=0x7f6900197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f68fe59c640 1 --2- 172.21.15.152:0/3813082958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6900102ef0 0x7f6900197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f68fe59c640 1 -- 172.21.15.152:0/3813082958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6900102570 msgr2=0x7f69001977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f68fe59c640 1 --2- 172.21.15.152:0/3813082958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6900102570 0x7f69001977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.087+0000 7f68fe59c640 1 -- 172.21.15.152:0/3813082958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69001a6b10 con 0x7f6900100e20 2024-09-11T22:51:52.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.088+0000 7f68fe59c640 1 --2- 172.21.15.152:0/3813082958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6900100e20 0x7f69001972a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f68f000b4a0 tx=0x7f68f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:52.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.088+0000 7f68f77fe640 1 -- 172.21.15.152:0/3813082958 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0014070 con 0x7f6900100e20 2024-09-11T22:51:52.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.088+0000 7f69048a7640 1 -- 172.21.15.152:0/3813082958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69001a6e00 con 0x7f6900100e20 2024-09-11T22:51:52.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.088+0000 7f68f77fe640 1 -- 172.21.15.152:0/3813082958 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68f0010af0 con 0x7f6900100e20 2024-09-11T22:51:52.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.088+0000 7f68f77fe640 1 -- 172.21.15.152:0/3813082958 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0017d10 con 0x7f6900100e20 2024-09-11T22:51:52.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.088+0000 7f69048a7640 1 -- 172.21.15.152:0/3813082958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69001a72e0 con 0x7f6900100e20 2024-09-11T22:51:52.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.089+0000 7f69048a7640 1 -- 172.21.15.152:0/3813082958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6900101280 con 0x7f6900100e20 2024-09-11T22:51:52.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.090+0000 7f68f77fe640 1 -- 172.21.15.152:0/3813082958 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f68f0017070 con 0x7f6900100e20 2024-09-11T22:51:52.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.091+0000 7f68f77fe640 1 --2- 172.21.15.152:0/3813082958 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f68ec0a2c10 0x7f68ec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:52.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.091+0000 7f68fdd9b640 1 --2- 172.21.15.152:0/3813082958 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f68ec0a2c10 0x7f68ec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:52.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.092+0000 7f68f77fe640 1 -- 172.21.15.152:0/3813082958 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f68f00ce390 con 0x7f6900100e20 2024-09-11T22:51:52.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.092+0000 7f68fdd9b640 1 --2- 172.21.15.152:0/3813082958 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f68ec0a2c10 0x7f68ec0a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6900199400 tx=0x7f68e8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:52.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.095+0000 7f68f77fe640 1 -- 172.21.15.152:0/3813082958 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68f009b450 con 0x7f6900100e20 2024-09-11T22:51:52.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:51 smithi152 ceph-mon[31181]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:51:52.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.270+0000 7f69048a7640 1 -- 172.21.15.152:0/3813082958 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f690002d7c0 con 0x7f6900100e20 2024-09-11T22:51:52.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.270+0000 7f68f77fe640 1 -- 172.21.15.152:0/3813082958 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f68f009ad50 con 0x7f6900100e20 2024-09-11T22:51:52.271 INFO:teuthology.orchestra.run.smithi152.stdout:107374182467 2024-09-11T22:51:52.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.272+0000 7f69048a7640 1 -- 172.21.15.152:0/3813082958 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f68ec0a2c10 msgr2=0x7f68ec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.272+0000 7f69048a7640 1 --2- 172.21.15.152:0/3813082958 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f68ec0a2c10 0x7f68ec0a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6900199400 tx=0x7f68e8005a20 comp rx=0 tx=0).stop 2024-09-11T22:51:52.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.272+0000 7f69048a7640 1 -- 172.21.15.152:0/3813082958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6900100e20 msgr2=0x7f69001972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.272+0000 7f69048a7640 1 --2- 172.21.15.152:0/3813082958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6900100e20 0x7f69001972a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f68f000b4a0 tx=0x7f68f000b960 comp rx=0 tx=0).stop 2024-09-11T22:51:52.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.272+0000 7f69048a7640 1 -- 172.21.15.152:0/3813082958 shutdown_connections 2024-09-11T22:51:52.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.272+0000 7f69048a7640 1 --2- 172.21.15.152:0/3813082958 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f68ec0a2c10 0x7f68ec0a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.272+0000 7f69048a7640 1 --2- 172.21.15.152:0/3813082958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6900102ef0 0x7f6900197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.272+0000 7f69048a7640 1 --2- 172.21.15.152:0/3813082958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6900102570 0x7f69001977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.272+0000 7f69048a7640 1 --2- 172.21.15.152:0/3813082958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6900100e20 0x7f69001972a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.272+0000 7f69048a7640 1 -- 172.21.15.152:0/3813082958 >> 172.21.15.152:0/3813082958 conn(0x7f69000fc7d0 msgr2=0x7f69000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:52.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.272+0000 7f69048a7640 1 -- 172.21.15.152:0/3813082958 shutdown_connections 2024-09-11T22:51:52.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.272+0000 7f69048a7640 1 -- 172.21.15.152:0/3813082958 wait complete. 2024-09-11T22:51:52.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:51 smithi191 ceph-mon[35211]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:51:52.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:51 smithi204 ceph-mon[35454]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:51:52.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.449+0000 7f4ca4fea640 1 -- 172.21.15.152:0/3335587584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ca01082a0 msgr2=0x7f4ca010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.449+0000 7f4ca4fea640 1 --2- 172.21.15.152:0/3335587584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ca01082a0 0x7f4ca010ec40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009cd0 tx=0x7f4c9802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.449+0000 7f4ca4fea640 1 -- 172.21.15.152:0/3335587584 shutdown_connections 2024-09-11T22:51:52.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.449+0000 7f4ca4fea640 1 --2- 172.21.15.152:0/3335587584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ca01082a0 0x7f4ca010ec40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.449+0000 7f4ca4fea640 1 --2- 172.21.15.152:0/3335587584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ca01036e0 0x7f4ca0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.449+0000 7f4ca4fea640 1 --2- 172.21.15.152:0/3335587584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca0102d30 0x7f4ca0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.449+0000 7f4ca4fea640 1 -- 172.21.15.152:0/3335587584 >> 172.21.15.152:0/3335587584 conn(0x7f4ca00fc7d0 msgr2=0x7f4ca00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:52.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.450+0000 7f4ca4fea640 1 -- 172.21.15.152:0/3335587584 shutdown_connections 2024-09-11T22:51:52.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.450+0000 7f4ca4fea640 1 -- 172.21.15.152:0/3335587584 wait complete. 2024-09-11T22:51:52.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.450+0000 7f4ca4fea640 1 Processor -- start 2024-09-11T22:51:52.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.450+0000 7f4ca4fea640 1 -- start start 2024-09-11T22:51:52.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4ca4fea640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ca0102d30 0x7f4ca019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:52.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4ca4fea640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca01036e0 0x7f4ca019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:52.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4ca4fea640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ca01082a0 0x7f4ca019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:52.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4ca4fea640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca01120a0 con 0x7f4ca0102d30 2024-09-11T22:51:52.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4ca4fea640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca0111f20 con 0x7f4ca01082a0 2024-09-11T22:51:52.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4ca4fea640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca0112220 con 0x7f4ca01036e0 2024-09-11T22:51:52.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4c9e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ca0102d30 0x7f4ca019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:52.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4c9e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ca0102d30 0x7f4ca019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52012/0 (socket says 172.21.15.152:52012) 2024-09-11T22:51:52.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4c9e59c640 1 -- 172.21.15.152:0/888139833 learned_addr learned my addr 172.21.15.152:0/888139833 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:52.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4c9dd9b640 1 --2- 172.21.15.152:0/888139833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca01036e0 0x7f4ca019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:52.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4c9ed9d640 1 --2- 172.21.15.152:0/888139833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ca01082a0 0x7f4ca019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:52.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4c9e59c640 1 -- 172.21.15.152:0/888139833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca01036e0 msgr2=0x7f4ca019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4c9e59c640 1 --2- 172.21.15.152:0/888139833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca01036e0 0x7f4ca019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4c9e59c640 1 -- 172.21.15.152:0/888139833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ca01082a0 msgr2=0x7f4ca019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4c9e59c640 1 --2- 172.21.15.152:0/888139833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ca01082a0 0x7f4ca019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4c9e59c640 1 -- 172.21.15.152:0/888139833 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca019f6f0 con 0x7f4ca0102d30 2024-09-11T22:51:52.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4c9ed9d640 1 --2- 172.21.15.152:0/888139833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ca01082a0 0x7f4ca019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:52.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.451+0000 7f4c9e59c640 1 --2- 172.21.15.152:0/888139833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ca0102d30 0x7f4ca019a740 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000e9f0 tx=0x7f4c9000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:52.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.452+0000 7f4c8f7fe640 1 -- 172.21.15.152:0/888139833 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9000cd80 con 0x7f4ca0102d30 2024-09-11T22:51:52.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.452+0000 7f4c8f7fe640 1 -- 172.21.15.152:0/888139833 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c90005500 con 0x7f4ca0102d30 2024-09-11T22:51:52.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.452+0000 7f4ca4fea640 1 -- 172.21.15.152:0/888139833 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca019f9e0 con 0x7f4ca0102d30 2024-09-11T22:51:52.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.452+0000 7f4c8f7fe640 1 -- 172.21.15.152:0/888139833 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90010690 con 0x7f4ca0102d30 2024-09-11T22:51:52.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.452+0000 7f4ca4fea640 1 -- 172.21.15.152:0/888139833 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca01a72c0 con 0x7f4ca0102d30 2024-09-11T22:51:52.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.453+0000 7f4ca4fea640 1 -- 172.21.15.152:0/888139833 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca0103110 con 0x7f4ca0102d30 2024-09-11T22:51:52.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.455+0000 7f4c8f7fe640 1 -- 172.21.15.152:0/888139833 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4c900046c0 con 0x7f4ca0102d30 2024-09-11T22:51:52.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.456+0000 7f4c8f7fe640 1 --2- 172.21.15.152:0/888139833 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4c880a2c90 0x7f4c880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:52.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.456+0000 7f4c9dd9b640 1 --2- 172.21.15.152:0/888139833 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4c880a2c90 0x7f4c880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:52.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.456+0000 7f4c8f7fe640 1 -- 172.21.15.152:0/888139833 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f4c900cf6f0 con 0x7f4ca0102d30 2024-09-11T22:51:52.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.456+0000 7f4c9dd9b640 1 --2- 172.21.15.152:0/888139833 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4c880a2c90 0x7f4c880a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4c94005e00 tx=0x7f4c94005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:52.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.462+0000 7f4c8f7fe640 1 -- 172.21.15.152:0/888139833 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c900541b0 con 0x7f4ca0102d30 2024-09-11T22:51:52.654 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790005 got 158913790008 for osd.5 2024-09-11T22:51:52.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.653+0000 7f4ca4fea640 1 -- 172.21.15.152:0/888139833 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f4ca010c2c0 con 0x7f4ca0102d30 2024-09-11T22:51:52.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.653+0000 7f4c8f7fe640 1 -- 172.21.15.152:0/888139833 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f4c9001f340 con 0x7f4ca0102d30 2024-09-11T22:51:52.655 INFO:teuthology.orchestra.run.smithi152.stdout:292057776157 2024-09-11T22:51:52.655 DEBUG:teuthology.parallel:result is None 2024-09-11T22:51:52.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.655+0000 7f4ca4fea640 1 -- 172.21.15.152:0/888139833 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4c880a2c90 msgr2=0x7f4c880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.655+0000 7f4ca4fea640 1 --2- 172.21.15.152:0/888139833 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4c880a2c90 0x7f4c880a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4c94005e00 tx=0x7f4c94005c50 comp rx=0 tx=0).stop 2024-09-11T22:51:52.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.655+0000 7f4ca4fea640 1 -- 172.21.15.152:0/888139833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ca0102d30 msgr2=0x7f4ca019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.655+0000 7f4ca4fea640 1 --2- 172.21.15.152:0/888139833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ca0102d30 0x7f4ca019a740 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000e9f0 tx=0x7f4c9000edf0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.655+0000 7f4ca4fea640 1 -- 172.21.15.152:0/888139833 shutdown_connections 2024-09-11T22:51:52.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.655+0000 7f4ca4fea640 1 --2- 172.21.15.152:0/888139833 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4c880a2c90 0x7f4c880a5150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.655+0000 7f4ca4fea640 1 --2- 172.21.15.152:0/888139833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ca01082a0 0x7f4ca019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.655+0000 7f4ca4fea640 1 --2- 172.21.15.152:0/888139833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ca01036e0 0x7f4ca019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.655+0000 7f4ca4fea640 1 --2- 172.21.15.152:0/888139833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ca0102d30 0x7f4ca019a740 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.655+0000 7f4ca4fea640 1 -- 172.21.15.152:0/888139833 >> 172.21.15.152:0/888139833 conn(0x7f4ca00fc7d0 msgr2=0x7f4ca01013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:52.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.656+0000 7f4ca4fea640 1 -- 172.21.15.152:0/888139833 shutdown_connections 2024-09-11T22:51:52.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.656+0000 7f4ca4fea640 1 -- 172.21.15.152:0/888139833 wait complete. 2024-09-11T22:51:52.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.820+0000 7f2d753b1640 1 -- 172.21.15.152:0/84593487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d70102570 msgr2=0x7f2d701029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.820+0000 7f2d753b1640 1 --2- 172.21.15.152:0/84593487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d70102570 0x7f2d701029b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009900 tx=0x7f2d6402a480 comp rx=0 tx=0).stop 2024-09-11T22:51:52.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.820+0000 7f2d753b1640 1 -- 172.21.15.152:0/84593487 shutdown_connections 2024-09-11T22:51:52.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.820+0000 7f2d753b1640 1 --2- 172.21.15.152:0/84593487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d70102ef0 0x7f2d7010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.820+0000 7f2d753b1640 1 --2- 172.21.15.152:0/84593487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d70102570 0x7f2d701029b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.820+0000 7f2d753b1640 1 --2- 172.21.15.152:0/84593487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d70100e20 0x7f2d70101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.820+0000 7f2d753b1640 1 -- 172.21.15.152:0/84593487 >> 172.21.15.152:0/84593487 conn(0x7f2d700fc7d0 msgr2=0x7f2d700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:52.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.820+0000 7f2d753b1640 1 -- 172.21.15.152:0/84593487 shutdown_connections 2024-09-11T22:51:52.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.820+0000 7f2d753b1640 1 -- 172.21.15.152:0/84593487 wait complete. 2024-09-11T22:51:52.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.821+0000 7f2d753b1640 1 Processor -- start 2024-09-11T22:51:52.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.821+0000 7f2d753b1640 1 -- start start 2024-09-11T22:51:52.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.821+0000 7f2d753b1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d70100e20 0x7f2d701993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:52.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.821+0000 7f2d753b1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d70102570 0x7f2d701998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:52.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.821+0000 7f2d753b1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d70102ef0 0x7f2d701a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:52.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.821+0000 7f2d753b1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70110fc0 con 0x7f2d70100e20 2024-09-11T22:51:52.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.821+0000 7f2d753b1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70110e40 con 0x7f2d70102570 2024-09-11T22:51:52.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.821+0000 7f2d753b1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70111140 con 0x7f2d70102ef0 2024-09-11T22:51:52.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.821+0000 7f2d6effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d70100e20 0x7f2d701993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:52.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.821+0000 7f2d6effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d70100e20 0x7f2d701993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52042/0 (socket says 172.21.15.152:52042) 2024-09-11T22:51:52.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.821+0000 7f2d6effd640 1 -- 172.21.15.152:0/1113642322 learned_addr learned my addr 172.21.15.152:0/1113642322 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:52.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.822+0000 7f2d6e7fc640 1 --2- 172.21.15.152:0/1113642322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d70102570 0x7f2d701998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:52.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.822+0000 7f2d6f7fe640 1 --2- 172.21.15.152:0/1113642322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d70102ef0 0x7f2d701a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:52.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.822+0000 7f2d6effd640 1 -- 172.21.15.152:0/1113642322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d70102ef0 msgr2=0x7f2d701a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.822+0000 7f2d6effd640 1 --2- 172.21.15.152:0/1113642322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d70102ef0 0x7f2d701a4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.822+0000 7f2d6effd640 1 -- 172.21.15.152:0/1113642322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d70102570 msgr2=0x7f2d701998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.822+0000 7f2d6effd640 1 --2- 172.21.15.152:0/1113642322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d70102570 0x7f2d701998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.822+0000 7f2d6effd640 1 -- 172.21.15.152:0/1113642322 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d701a4970 con 0x7f2d70100e20 2024-09-11T22:51:52.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.822+0000 7f2d6e7fc640 1 --2- 172.21.15.152:0/1113642322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d70102570 0x7f2d701998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:52.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.822+0000 7f2d6effd640 1 --2- 172.21.15.152:0/1113642322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d70100e20 0x7f2d701993b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000c970 tx=0x7f2d6000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:52.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.822+0000 7f2d57fff640 1 -- 172.21.15.152:0/1113642322 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60007bb0 con 0x7f2d70100e20 2024-09-11T22:51:52.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.822+0000 7f2d57fff640 1 -- 172.21.15.152:0/1113642322 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d60007d50 con 0x7f2d70100e20 2024-09-11T22:51:52.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.822+0000 7f2d753b1640 1 -- 172.21.15.152:0/1113642322 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d701a4b40 con 0x7f2d70100e20 2024-09-11T22:51:52.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.822+0000 7f2d57fff640 1 -- 172.21.15.152:0/1113642322 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000f450 con 0x7f2d70100e20 2024-09-11T22:51:52.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.822+0000 7f2d753b1640 1 -- 172.21.15.152:0/1113642322 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d701a5000 con 0x7f2d70100e20 2024-09-11T22:51:52.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.823+0000 7f2d753b1640 1 -- 172.21.15.152:0/1113642322 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d701021b0 con 0x7f2d70100e20 2024-09-11T22:51:52.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.824+0000 7f2d57fff640 1 -- 172.21.15.152:0/1113642322 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2d60005020 con 0x7f2d70100e20 2024-09-11T22:51:52.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.825+0000 7f2d57fff640 1 --2- 172.21.15.152:0/1113642322 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2d580a2c10 0x7f2d580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:52.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.825+0000 7f2d6e7fc640 1 --2- 172.21.15.152:0/1113642322 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2d580a2c10 0x7f2d580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:52.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.825+0000 7f2d57fff640 1 -- 172.21.15.152:0/1113642322 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f2d600cefc0 con 0x7f2d70100e20 2024-09-11T22:51:52.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.826+0000 7f2d6e7fc640 1 --2- 172.21.15.152:0/1113642322 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2d580a2c10 0x7f2d580a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2d64000c00 tx=0x7f2d640041e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:52.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.829+0000 7f2d57fff640 1 -- 172.21.15.152:0/1113642322 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d60014030 con 0x7f2d70100e20 2024-09-11T22:51:52.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.336+0000 7f6df43c6640 1 -- 172.21.15.152:0/306291317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dec102570 msgr2=0x7f6dec1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.336+0000 7f6df43c6640 1 --2- 172.21.15.152:0/306291317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dec102570 0x7f6dec1029b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc009900 tx=0x7f6ddc02a480 comp rx=0 tx=0).stop 2024-09-11T22:51:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.336+0000 7f6df43c6640 1 -- 172.21.15.152:0/306291317 shutdown_connections 2024-09-11T22:51:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.336+0000 7f6df43c6640 1 --2- 172.21.15.152:0/306291317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dec102ef0 0x7f6dec10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.336+0000 7f6df43c6640 1 --2- 172.21.15.152:0/306291317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dec102570 0x7f6dec1029b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.336+0000 7f6df43c6640 1 --2- 172.21.15.152:0/306291317 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dec100e20 0x7f6dec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.336+0000 7f6df43c6640 1 -- 172.21.15.152:0/306291317 >> 172.21.15.152:0/306291317 conn(0x7f6dec0fc7d0 msgr2=0x7f6dec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:52.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.336+0000 7f6df43c6640 1 -- 172.21.15.152:0/306291317 shutdown_connections 2024-09-11T22:51:52.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.336+0000 7f6df43c6640 1 -- 172.21.15.152:0/306291317 wait complete. 2024-09-11T22:51:52.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.336+0000 7f6df43c6640 1 Processor -- start 2024-09-11T22:51:52.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df43c6640 1 -- start start 2024-09-11T22:51:52.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df43c6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dec100e20 0x7f6dec1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:52.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df43c6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dec102570 0x7f6dec198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:52.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df43c6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dec102ef0 0x7f6dec19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:52.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df43c6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec110f80 con 0x7f6dec102ef0 2024-09-11T22:51:52.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df43c6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec110e00 con 0x7f6dec100e20 2024-09-11T22:51:52.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df43c6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec111100 con 0x7f6dec102570 2024-09-11T22:51:52.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df2963640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dec102ef0 0x7f6dec19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:52.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df2963640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dec102ef0 0x7f6dec19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51990/0 (socket says 172.21.15.152:51990) 2024-09-11T22:51:52.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df2963640 1 -- 172.21.15.152:0/3446328063 learned_addr learned my addr 172.21.15.152:0/3446328063 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:52.994 INFO:teuthology.orchestra.run.smithi152.stdout:77309411400 2024-09-11T22:51:52.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df2162640 1 --2- 172.21.15.152:0/3446328063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dec100e20 0x7f6dec1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:52.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df1961640 1 --2- 172.21.15.152:0/3446328063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dec102570 0x7f6dec198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:52.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df2963640 1 -- 172.21.15.152:0/3446328063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dec102570 msgr2=0x7f6dec198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df2963640 1 --2- 172.21.15.152:0/3446328063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dec102570 0x7f6dec198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df2963640 1 -- 172.21.15.152:0/3446328063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dec100e20 msgr2=0x7f6dec1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df2963640 1 --2- 172.21.15.152:0/3446328063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dec100e20 0x7f6dec1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df2963640 1 -- 172.21.15.152:0/3446328063 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dec19d490 con 0x7f6dec102ef0 2024-09-11T22:51:52.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df1961640 1 --2- 172.21.15.152:0/3446328063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dec102570 0x7f6dec198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:51:52.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.337+0000 7f6df2963640 1 --2- 172.21.15.152:0/3446328063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dec102ef0 0x7f6dec19cdb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6de800eb10 tx=0x7f6de800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:52.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.338+0000 7f6de37fe640 1 -- 172.21.15.152:0/3446328063 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de8004b20 con 0x7f6dec102ef0 2024-09-11T22:51:52.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.338+0000 7f6df43c6640 1 -- 172.21.15.152:0/3446328063 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dec19d780 con 0x7f6dec102ef0 2024-09-11T22:51:52.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.338+0000 7f6de37fe640 1 -- 172.21.15.152:0/3446328063 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de8004cc0 con 0x7f6dec102ef0 2024-09-11T22:51:52.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.338+0000 7f6de37fe640 1 -- 172.21.15.152:0/3446328063 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de8010760 con 0x7f6dec102ef0 2024-09-11T22:51:52.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.338+0000 7f6df43c6640 1 -- 172.21.15.152:0/3446328063 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dec1a5060 con 0x7f6dec102ef0 2024-09-11T22:51:52.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.339+0000 7f6df43c6640 1 -- 172.21.15.152:0/3446328063 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dec101200 con 0x7f6dec102ef0 2024-09-11T22:51:52.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.340+0000 7f6de37fe640 1 -- 172.21.15.152:0/3446328063 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6de8005020 con 0x7f6dec102ef0 2024-09-11T22:51:52.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.340+0000 7f6de37fe640 1 --2- 172.21.15.152:0/3446328063 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f6dd80a2c10 0x7f6dd80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:52.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.341+0000 7f6df2162640 1 --2- 172.21.15.152:0/3446328063 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f6dd80a2c10 0x7f6dd80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:52.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.341+0000 7f6de37fe640 1 -- 172.21.15.152:0/3446328063 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f6de80cf770 con 0x7f6dec102ef0 2024-09-11T22:51:52.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.341+0000 7f6df2162640 1 --2- 172.21.15.152:0/3446328063 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f6dd80a2c10 0x7f6dd80a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009bd0 tx=0x7f6de4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:52.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.345+0000 7f6de37fe640 1 -- 172.21.15.152:0/3446328063 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6de809c720 con 0x7f6dec102ef0 2024-09-11T22:51:52.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.532+0000 7f6df43c6640 1 -- 172.21.15.152:0/3446328063 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f6dec1a5290 con 0x7f6dec102ef0 2024-09-11T22:51:52.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.533+0000 7f6de37fe640 1 -- 172.21.15.152:0/3446328063 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6de809c020 con 0x7f6dec102ef0 2024-09-11T22:51:52.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.536+0000 7f6df43c6640 1 -- 172.21.15.152:0/3446328063 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f6dd80a2c10 msgr2=0x7f6dd80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.536+0000 7f6df43c6640 1 --2- 172.21.15.152:0/3446328063 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f6dd80a2c10 0x7f6dd80a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009bd0 tx=0x7f6de4005f90 comp rx=0 tx=0).stop 2024-09-11T22:51:52.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.536+0000 7f6df43c6640 1 -- 172.21.15.152:0/3446328063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dec102ef0 msgr2=0x7f6dec19cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:52.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.536+0000 7f6df43c6640 1 --2- 172.21.15.152:0/3446328063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dec102ef0 0x7f6dec19cdb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6de800eb10 tx=0x7f6de800c2d0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.536+0000 7f6df43c6640 1 -- 172.21.15.152:0/3446328063 shutdown_connections 2024-09-11T22:51:52.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.536+0000 7f6df43c6640 1 --2- 172.21.15.152:0/3446328063 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f6dd80a2c10 0x7f6dd80a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.536+0000 7f6df43c6640 1 --2- 172.21.15.152:0/3446328063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6dec102ef0 0x7f6dec19cdb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:52.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.536+0000 7f6df43c6640 1 --2- 172.21.15.152:0/3446328063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dec102570 0x7f6dec198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:53.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.536+0000 7f6df43c6640 1 --2- 172.21.15.152:0/3446328063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dec100e20 0x7f6dec1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:53.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.536+0000 7f6df43c6640 1 -- 172.21.15.152:0/3446328063 >> 172.21.15.152:0/3446328063 conn(0x7f6dec0fc7d0 msgr2=0x7f6dec069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:53.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.536+0000 7f6df43c6640 1 -- 172.21.15.152:0/3446328063 shutdown_connections 2024-09-11T22:51:53.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:52.536+0000 7f6df43c6640 1 -- 172.21.15.152:0/3446328063 wait complete. 2024-09-11T22:51:53.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:53.006+0000 7f2d753b1640 1 -- 172.21.15.152:0/1113642322 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2d70109090 con 0x7f2d70100e20 2024-09-11T22:51:53.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:53.006+0000 7f2d57fff640 1 -- 172.21.15.152:0/1113642322 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2d6009be70 con 0x7f2d70100e20 2024-09-11T22:51:53.008 INFO:teuthology.orchestra.run.smithi152.stdout:137438953534 2024-09-11T22:51:53.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:53.008+0000 7f2d753b1640 1 -- 172.21.15.152:0/1113642322 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2d580a2c10 msgr2=0x7f2d580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:53.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:53.008+0000 7f2d753b1640 1 --2- 172.21.15.152:0/1113642322 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2d580a2c10 0x7f2d580a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2d64000c00 tx=0x7f2d640041e0 comp rx=0 tx=0).stop 2024-09-11T22:51:53.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:53.008+0000 7f2d753b1640 1 -- 172.21.15.152:0/1113642322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d70100e20 msgr2=0x7f2d701993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:53.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:53.009+0000 7f2d753b1640 1 --2- 172.21.15.152:0/1113642322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d70100e20 0x7f2d701993b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000c970 tx=0x7f2d6000ce30 comp rx=0 tx=0).stop 2024-09-11T22:51:53.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:53.009+0000 7f2d753b1640 1 -- 172.21.15.152:0/1113642322 shutdown_connections 2024-09-11T22:51:53.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:53.009+0000 7f2d753b1640 1 --2- 172.21.15.152:0/1113642322 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2d580a2c10 0x7f2d580a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:53.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:53.009+0000 7f2d753b1640 1 --2- 172.21.15.152:0/1113642322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d70102ef0 0x7f2d701a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:53.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:53.009+0000 7f2d753b1640 1 --2- 172.21.15.152:0/1113642322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d70102570 0x7f2d701998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:53.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:53.009+0000 7f2d753b1640 1 --2- 172.21.15.152:0/1113642322 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d70100e20 0x7f2d701993b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:53.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:53.009+0000 7f2d753b1640 1 -- 172.21.15.152:0/1113642322 >> 172.21.15.152:0/1113642322 conn(0x7f2d700fc7d0 msgr2=0x7f2d700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:53.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:53.009+0000 7f2d753b1640 1 -- 172.21.15.152:0/1113642322 shutdown_connections 2024-09-11T22:51:53.011 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:53.009+0000 7f2d753b1640 1 -- 172.21.15.152:0/1113642322 wait complete. 2024-09-11T22:51:53.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:52 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3813082958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T22:51:53.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:52 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3446328063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T22:51:53.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:52 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/888139833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T22:51:53.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:52 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3813082958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T22:51:53.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:52 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3446328063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T22:51:53.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:52 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/888139833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T22:51:53.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:52 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3813082958' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T22:51:53.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:52 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3446328063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T22:51:53.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:52 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/888139833' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T22:51:53.690 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411400 for osd.2 2024-09-11T22:51:53.691 DEBUG:teuthology.parallel:result is None 2024-09-11T22:51:54.227 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:53 smithi152 ceph-mon[31181]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-11T22:51:54.227 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:53 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1113642322' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T22:51:54.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:53 smithi191 ceph-mon[35211]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-11T22:51:54.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:53 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1113642322' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T22:51:54.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:53 smithi204 ceph-mon[35454]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-11T22:51:54.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:53 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1113642322' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T22:51:54.723 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953530 got 137438953534 for osd.4 2024-09-11T22:51:54.724 DEBUG:teuthology.parallel:result is None 2024-09-11T22:51:55.164 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738447 got 34359738449 for osd.0 2024-09-11T22:51:55.165 DEBUG:teuthology.parallel:result is None 2024-09-11T22:51:55.256 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-09-11T22:51:55.256 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-11T22:51:55.257 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-09-11T22:51:55.257 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]: dispatch 2024-09-11T22:51:55.257 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-11T22:51:55.257 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-09-11T22:51:55.257 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:54 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-09-11T22:51:55.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:54 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-09-11T22:51:55.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:54 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-11T22:51:55.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:54 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-09-11T22:51:55.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:54 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]: dispatch 2024-09-11T22:51:55.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:54 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-11T22:51:55.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:54 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-09-11T22:51:55.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:54 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-09-11T22:51:55.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:54 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-09-11T22:51:55.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:54 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-11T22:51:55.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:54 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-09-11T22:51:55.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:54 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]: dispatch 2024-09-11T22:51:55.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:54 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-11T22:51:55.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:54 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-09-11T22:51:55.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:54 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-09-11T22:51:55.791 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776154 got 292057776157 for osd.10 2024-09-11T22:51:55.791 DEBUG:teuthology.parallel:result is None 2024-09-11T22:51:55.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.968+0000 7f55ead33640 1 -- 172.21.15.152:0/969321758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55e4100e20 msgr2=0x7f55e4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:55.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.968+0000 7f55ead33640 1 --2- 172.21.15.152:0/969321758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55e4100e20 0x7f55e4101200 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009900 tx=0x7f55d802a480 comp rx=0 tx=0).stop 2024-09-11T22:51:55.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.968+0000 7f55ead33640 1 -- 172.21.15.152:0/969321758 shutdown_connections 2024-09-11T22:51:55.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.968+0000 7f55ead33640 1 --2- 172.21.15.152:0/969321758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55e4102ef0 0x7f55e410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:55.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.968+0000 7f55ead33640 1 --2- 172.21.15.152:0/969321758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55e4102570 0x7f55e41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:55.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.968+0000 7f55ead33640 1 --2- 172.21.15.152:0/969321758 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55e4100e20 0x7f55e4101200 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:55.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.968+0000 7f55ead33640 1 -- 172.21.15.152:0/969321758 >> 172.21.15.152:0/969321758 conn(0x7f55e40fc7d0 msgr2=0x7f55e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:55.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.968+0000 7f55ead33640 1 -- 172.21.15.152:0/969321758 shutdown_connections 2024-09-11T22:51:55.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.968+0000 7f55ead33640 1 -- 172.21.15.152:0/969321758 wait complete. 2024-09-11T22:51:55.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55ead33640 1 Processor -- start 2024-09-11T22:51:55.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55ead33640 1 -- start start 2024-09-11T22:51:55.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55ead33640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55e4100e20 0x7f55e4199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:55.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55ead33640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55e4102570 0x7f55e4199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:55.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55ead33640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55e4102ef0 0x7f55e41a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:55.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55ead33640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f55e4110f80 con 0x7f55e4102ef0 2024-09-11T22:51:55.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55ead33640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f55e4110e00 con 0x7f55e4102570 2024-09-11T22:51:55.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55ead33640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f55e4111100 con 0x7f55e4100e20 2024-09-11T22:51:55.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55e92d0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55e4102ef0 0x7f55e41a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:55.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55e92d0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55e4102ef0 0x7f55e41a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52048/0 (socket says 172.21.15.152:52048) 2024-09-11T22:51:55.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55e92d0640 1 -- 172.21.15.152:0/2141964754 learned_addr learned my addr 172.21.15.152:0/2141964754 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:55.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55e8acf640 1 --2- 172.21.15.152:0/2141964754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55e4100e20 0x7f55e4199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:55.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55e3fff640 1 --2- 172.21.15.152:0/2141964754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55e4102570 0x7f55e4199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:55.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55e92d0640 1 -- 172.21.15.152:0/2141964754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55e4100e20 msgr2=0x7f55e4199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:55.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55e92d0640 1 --2- 172.21.15.152:0/2141964754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55e4100e20 0x7f55e4199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:55.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55e92d0640 1 -- 172.21.15.152:0/2141964754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55e4102570 msgr2=0x7f55e4199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:55.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55e92d0640 1 --2- 172.21.15.152:0/2141964754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55e4102570 0x7f55e4199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:55.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55e92d0640 1 -- 172.21.15.152:0/2141964754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e41a48d0 con 0x7f55e4102ef0 2024-09-11T22:51:55.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55e3fff640 1 --2- 172.21.15.152:0/2141964754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55e4102570 0x7f55e4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:51:55.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55e92d0640 1 --2- 172.21.15.152:0/2141964754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55e4102ef0 0x7f55e41a4390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00cce0 tx=0x7f55dc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:55.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.969+0000 7f55e1ffb640 1 -- 172.21.15.152:0/2141964754 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc00f450 con 0x7f55e4102ef0 2024-09-11T22:51:55.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.970+0000 7f55ead33640 1 -- 172.21.15.152:0/2141964754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e41a4aa0 con 0x7f55e4102ef0 2024-09-11T22:51:55.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.970+0000 7f55e1ffb640 1 -- 172.21.15.152:0/2141964754 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55dc00fad0 con 0x7f55e4102ef0 2024-09-11T22:51:55.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.970+0000 7f55e1ffb640 1 -- 172.21.15.152:0/2141964754 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc018c10 con 0x7f55e4102ef0 2024-09-11T22:51:55.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.970+0000 7f55ead33640 1 -- 172.21.15.152:0/2141964754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e41a4fb0 con 0x7f55e4102ef0 2024-09-11T22:51:55.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.971+0000 7f55ead33640 1 -- 172.21.15.152:0/2141964754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e41021b0 con 0x7f55e4102ef0 2024-09-11T22:51:55.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.972+0000 7f55e1ffb640 1 -- 172.21.15.152:0/2141964754 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f55dc00f5f0 con 0x7f55e4102ef0 2024-09-11T22:51:55.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.972+0000 7f55e1ffb640 1 --2- 172.21.15.152:0/2141964754 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f55d40a2c10 0x7f55d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:55.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.972+0000 7f55e8acf640 1 --2- 172.21.15.152:0/2141964754 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f55d40a2c10 0x7f55d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:55.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.972+0000 7f55e1ffb640 1 -- 172.21.15.152:0/2141964754 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f55dc0cf7d0 con 0x7f55e4102ef0 2024-09-11T22:51:55.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.973+0000 7f55e8acf640 1 --2- 172.21.15.152:0/2141964754 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f55d40a2c10 0x7f55d40a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009230 tx=0x7f55d80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:55.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:55.977+0000 7f55e1ffb640 1 -- 172.21.15.152:0/2141964754 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55dc014030 con 0x7f55e4102ef0 2024-09-11T22:51:56.011 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:56.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:55 smithi152 ceph-mon[31181]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:51:56.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-11T22:51:56.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-11T22:51:56.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-11T22:51:56.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]': finished 2024-09-11T22:51:56.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-11T22:51:56.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-09-11T22:51:56.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:55 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-09-11T22:51:56.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:55 smithi152 ceph-mon[31181]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T22:51:56.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:56.156+0000 7f55ead33640 1 -- 172.21.15.152:0/2141964754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f55e4109090 con 0x7f55e4102ef0 2024-09-11T22:51:56.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:56.156+0000 7f55e1ffb640 1 -- 172.21.15.152:0/2141964754 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f55dc09c600 con 0x7f55e4102ef0 2024-09-11T22:51:56.157 INFO:teuthology.orchestra.run.smithi152.stdout:55834574925 2024-09-11T22:51:56.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:56.158+0000 7f55ead33640 1 -- 172.21.15.152:0/2141964754 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f55d40a2c10 msgr2=0x7f55d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:56.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:56.158+0000 7f55ead33640 1 --2- 172.21.15.152:0/2141964754 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f55d40a2c10 0x7f55d40a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f55d8009230 tx=0x7f55d80058d0 comp rx=0 tx=0).stop 2024-09-11T22:51:56.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:56.158+0000 7f55ead33640 1 -- 172.21.15.152:0/2141964754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55e4102ef0 msgr2=0x7f55e41a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:56.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:56.158+0000 7f55ead33640 1 --2- 172.21.15.152:0/2141964754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55e4102ef0 0x7f55e41a4390 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f55dc00cce0 tx=0x7f55dc00a620 comp rx=0 tx=0).stop 2024-09-11T22:51:56.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:56.158+0000 7f55ead33640 1 -- 172.21.15.152:0/2141964754 shutdown_connections 2024-09-11T22:51:56.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:56.158+0000 7f55ead33640 1 --2- 172.21.15.152:0/2141964754 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f55d40a2c10 0x7f55d40a50d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:56.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:56.158+0000 7f55ead33640 1 --2- 172.21.15.152:0/2141964754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f55e4102ef0 0x7f55e41a4390 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:56.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:56.158+0000 7f55ead33640 1 --2- 172.21.15.152:0/2141964754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55e4102570 0x7f55e4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:56.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:56.158+0000 7f55ead33640 1 --2- 172.21.15.152:0/2141964754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55e4100e20 0x7f55e4199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:56.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:56.158+0000 7f55ead33640 1 -- 172.21.15.152:0/2141964754 >> 172.21.15.152:0/2141964754 conn(0x7f55e40fc7d0 msgr2=0x7f55e40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:56.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:56.159+0000 7f55ead33640 1 -- 172.21.15.152:0/2141964754 shutdown_connections 2024-09-11T22:51:56.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:56.159+0000 7f55ead33640 1 -- 172.21.15.152:0/2141964754 wait complete. 2024-09-11T22:51:56.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:55 smithi191 ceph-mon[35211]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:51:56.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-11T22:51:56.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-11T22:51:56.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-11T22:51:56.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]': finished 2024-09-11T22:51:56.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-11T22:51:56.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-09-11T22:51:56.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:55 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-09-11T22:51:56.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:55 smithi191 ceph-mon[35211]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T22:51:56.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:55 smithi204 ceph-mon[35454]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:51:56.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-11T22:51:56.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-11T22:51:56.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-09-11T22:51:56.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6, 5, 9]}]': finished 2024-09-11T22:51:56.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-11T22:51:56.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-09-11T22:51:56.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:55 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-09-11T22:51:56.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:55 smithi204 ceph-mon[35454]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T22:51:56.489 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939681 got 270582939683 for osd.9 2024-09-11T22:51:56.489 DEBUG:teuthology.parallel:result is None 2024-09-11T22:51:57.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:56 smithi152 ceph-mon[31181]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T22:51:57.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:56 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2141964754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T22:51:57.045 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:56 smithi152 ceph-mon[31181]: pgmap v314: 97 pgs: 4 peering, 93 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-09-11T22:51:57.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:56 smithi191 ceph-mon[35211]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T22:51:57.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:56 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2141964754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T22:51:57.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:56 smithi191 ceph-mon[35211]: pgmap v314: 97 pgs: 4 peering, 93 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-09-11T22:51:57.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:56 smithi204 ceph-mon[35454]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T22:51:57.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:56 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2141964754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T22:51:57.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:56 smithi204 ceph-mon[35454]: pgmap v314: 97 pgs: 4 peering, 93 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-09-11T22:51:57.449 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182464 got 107374182467 for osd.3 2024-09-11T22:51:57.449 DEBUG:teuthology.parallel:result is None 2024-09-11T22:51:57.694 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:51:58.225 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:57 smithi152 ceph-mon[31181]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-11T22:51:58.226 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574925 for osd.1 2024-09-11T22:51:58.226 DEBUG:teuthology.parallel:result is None 2024-09-11T22:51:58.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:57 smithi191 ceph-mon[35211]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-11T22:51:58.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:57 smithi204 ceph-mon[35454]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-11T22:51:58.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.463+0000 7f2030f90640 1 -- 172.21.15.152:0/3620984429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f202c100e20 msgr2=0x7f202c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:58.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.463+0000 7f2030f90640 1 --2- 172.21.15.152:0/3620984429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f202c100e20 0x7f202c101200 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f201c009900 tx=0x7f201c02a480 comp rx=0 tx=0).stop 2024-09-11T22:51:58.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.463+0000 7f2030f90640 1 -- 172.21.15.152:0/3620984429 shutdown_connections 2024-09-11T22:51:58.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.463+0000 7f2030f90640 1 --2- 172.21.15.152:0/3620984429 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c102ef0 0x7f202c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:58.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.463+0000 7f2030f90640 1 --2- 172.21.15.152:0/3620984429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f202c102570 0x7f202c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:58.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.463+0000 7f2030f90640 1 --2- 172.21.15.152:0/3620984429 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f202c100e20 0x7f202c101200 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:58.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.463+0000 7f2030f90640 1 -- 172.21.15.152:0/3620984429 >> 172.21.15.152:0/3620984429 conn(0x7f202c0fc7d0 msgr2=0x7f202c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:58.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.463+0000 7f2030f90640 1 -- 172.21.15.152:0/3620984429 shutdown_connections 2024-09-11T22:51:58.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.463+0000 7f2030f90640 1 -- 172.21.15.152:0/3620984429 wait complete. 2024-09-11T22:51:58.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.463+0000 7f2030f90640 1 Processor -- start 2024-09-11T22:51:58.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.463+0000 7f2030f90640 1 -- start start 2024-09-11T22:51:58.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f2030f90640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c100e20 0x7f202c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:58.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f2030f90640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f202c102570 0x7f202c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:58.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f2030f90640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f202c102ef0 0x7f202c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:58.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f2030f90640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f202c1120a0 con 0x7f202c102570 2024-09-11T22:51:58.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f2030f90640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f202c111f20 con 0x7f202c100e20 2024-09-11T22:51:58.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f2030f90640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f202c112220 con 0x7f202c102ef0 2024-09-11T22:51:58.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f2023fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f202c102570 0x7f202c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:58.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f2023fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f202c102570 0x7f202c198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38608/0 (socket says 172.21.15.152:38608) 2024-09-11T22:51:58.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f2023fff640 1 -- 172.21.15.152:0/4017786115 learned_addr learned my addr 172.21.15.152:0/4017786115 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:58.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f202a59c640 1 --2- 172.21.15.152:0/4017786115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c100e20 0x7f202c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:58.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f2023fff640 1 -- 172.21.15.152:0/4017786115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f202c102ef0 msgr2=0x7f202c19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:58.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f202ad9d640 1 --2- 172.21.15.152:0/4017786115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f202c102ef0 0x7f202c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f2023fff640 1 --2- 172.21.15.152:0/4017786115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f202c102ef0 0x7f202c19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f2023fff640 1 -- 172.21.15.152:0/4017786115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c100e20 msgr2=0x7f202c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f2023fff640 1 --2- 172.21.15.152:0/4017786115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c100e20 0x7f202c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f2023fff640 1 -- 172.21.15.152:0/4017786115 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202c19d570 con 0x7f202c102570 2024-09-11T22:51:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f202a59c640 1 --2- 172.21.15.152:0/4017786115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c100e20 0x7f202c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:51:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f202ad9d640 1 --2- 172.21.15.152:0/4017786115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f202c102ef0 0x7f202c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:51:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f2023fff640 1 --2- 172.21.15.152:0/4017786115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f202c102570 0x7f202c198b00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f201400b4a0 tx=0x7f201400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f20237fe640 1 -- 172.21.15.152:0/4017786115 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014014070 con 0x7f202c102570 2024-09-11T22:51:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f20237fe640 1 -- 172.21.15.152:0/4017786115 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2014010af0 con 0x7f202c102570 2024-09-11T22:51:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f20237fe640 1 -- 172.21.15.152:0/4017786115 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2014017d10 con 0x7f202c102570 2024-09-11T22:51:58.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.464+0000 7f2030f90640 1 -- 172.21.15.152:0/4017786115 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202c19d860 con 0x7f202c102570 2024-09-11T22:51:58.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.465+0000 7f2030f90640 1 -- 172.21.15.152:0/4017786115 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f202c1a5140 con 0x7f202c102570 2024-09-11T22:51:58.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.465+0000 7f2030f90640 1 -- 172.21.15.152:0/4017786115 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f202c101200 con 0x7f202c102570 2024-09-11T22:51:58.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.466+0000 7f20237fe640 1 -- 172.21.15.152:0/4017786115 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2014017070 con 0x7f202c102570 2024-09-11T22:51:58.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.467+0000 7f20237fe640 1 --2- 172.21.15.152:0/4017786115 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f20180a2c10 0x7f20180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:58.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.467+0000 7f202a59c640 1 --2- 172.21.15.152:0/4017786115 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f20180a2c10 0x7f20180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:58.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.468+0000 7f20237fe640 1 -- 172.21.15.152:0/4017786115 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f20140ce580 con 0x7f202c102570 2024-09-11T22:51:58.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.468+0000 7f202a59c640 1 --2- 172.21.15.152:0/4017786115 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f20180a2c10 0x7f20180a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f201c005ab0 tx=0x7f201c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:58.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.472+0000 7f20237fe640 1 -- 172.21.15.152:0/4017786115 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f201409b4c0 con 0x7f202c102570 2024-09-11T22:51:58.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.649+0000 7f2030f90640 1 -- 172.21.15.152:0/4017786115 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f202c1090c0 con 0x7f202c102570 2024-09-11T22:51:58.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.650+0000 7f20237fe640 1 -- 172.21.15.152:0/4017786115 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f201409adc0 con 0x7f202c102570 2024-09-11T22:51:58.652 INFO:teuthology.orchestra.run.smithi152.stdout:210453397550 2024-09-11T22:51:58.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.652+0000 7f2030f90640 1 -- 172.21.15.152:0/4017786115 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f20180a2c10 msgr2=0x7f20180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:58.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.652+0000 7f2030f90640 1 --2- 172.21.15.152:0/4017786115 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f20180a2c10 0x7f20180a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f201c005ab0 tx=0x7f201c005a20 comp rx=0 tx=0).stop 2024-09-11T22:51:58.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.652+0000 7f2030f90640 1 -- 172.21.15.152:0/4017786115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f202c102570 msgr2=0x7f202c198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:58.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.652+0000 7f2030f90640 1 --2- 172.21.15.152:0/4017786115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f202c102570 0x7f202c198b00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f201400b4a0 tx=0x7f201400b960 comp rx=0 tx=0).stop 2024-09-11T22:51:58.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.652+0000 7f2030f90640 1 -- 172.21.15.152:0/4017786115 shutdown_connections 2024-09-11T22:51:58.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.652+0000 7f2030f90640 1 --2- 172.21.15.152:0/4017786115 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f20180a2c10 0x7f20180a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:58.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.652+0000 7f2030f90640 1 --2- 172.21.15.152:0/4017786115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f202c102ef0 0x7f202c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:58.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.652+0000 7f2030f90640 1 --2- 172.21.15.152:0/4017786115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f202c102570 0x7f202c198b00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:58.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.652+0000 7f2030f90640 1 --2- 172.21.15.152:0/4017786115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f202c100e20 0x7f202c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:58.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.652+0000 7f2030f90640 1 -- 172.21.15.152:0/4017786115 >> 172.21.15.152:0/4017786115 conn(0x7f202c0fc7d0 msgr2=0x7f202c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:58.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.652+0000 7f2030f90640 1 -- 172.21.15.152:0/4017786115 shutdown_connections 2024-09-11T22:51:58.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:58.652+0000 7f2030f90640 1 -- 172.21.15.152:0/4017786115 wait complete. 2024-09-11T22:51:59.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:58 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4017786115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T22:51:59.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:58 smithi152 ceph-mon[31181]: pgmap v315: 97 pgs: 4 peering, 93 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T22:51:59.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:58 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/4017786115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T22:51:59.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:58 smithi191 ceph-mon[35211]: pgmap v315: 97 pgs: 4 peering, 93 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T22:51:59.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:58 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/4017786115' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T22:51:59.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:58 smithi204 ceph-mon[35454]: pgmap v315: 97 pgs: 4 peering, 93 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T22:51:59.422 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397547 got 210453397550 for osd.7 2024-09-11T22:51:59.423 DEBUG:teuthology.parallel:result is None 2024-09-11T22:51:59.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.607+0000 7f4e76663640 1 -- 172.21.15.152:0/1718634319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e70102dd0 msgr2=0x7f4e701031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:59.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.607+0000 7f4e76663640 1 --2- 172.21.15.152:0/1718634319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e70102dd0 0x7f4e701031b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009cd0 tx=0x7f4e6002a8f0 comp rx=0 tx=0).stop 2024-09-11T22:51:59.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.607+0000 7f4e76663640 1 -- 172.21.15.152:0/1718634319 shutdown_connections 2024-09-11T22:51:59.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.607+0000 7f4e76663640 1 --2- 172.21.15.152:0/1718634319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e70108300 0x7f4e7010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:59.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.607+0000 7f4e76663640 1 --2- 172.21.15.152:0/1718634319 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e70103780 0x7f4e70107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:59.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.607+0000 7f4e76663640 1 --2- 172.21.15.152:0/1718634319 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e70102dd0 0x7f4e701031b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:59.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.607+0000 7f4e76663640 1 -- 172.21.15.152:0/1718634319 >> 172.21.15.152:0/1718634319 conn(0x7f4e700fc7d0 msgr2=0x7f4e700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:59.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.607+0000 7f4e76663640 1 -- 172.21.15.152:0/1718634319 shutdown_connections 2024-09-11T22:51:59.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.607+0000 7f4e76663640 1 -- 172.21.15.152:0/1718634319 wait complete. 2024-09-11T22:51:59.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.607+0000 7f4e76663640 1 Processor -- start 2024-09-11T22:51:59.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.607+0000 7f4e76663640 1 -- start start 2024-09-11T22:51:59.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e76663640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e70102dd0 0x7f4e70198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:59.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e76663640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e70103780 0x7f4e70198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:59.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e76663640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e70108300 0x7f4e7019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:59.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e76663640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4e701120f0 con 0x7f4e70102dd0 2024-09-11T22:51:59.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e76663640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70111f70 con 0x7f4e70103780 2024-09-11T22:51:59.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e76663640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70112270 con 0x7f4e70108300 2024-09-11T22:51:59.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e6ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e70102dd0 0x7f4e70198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:59.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e6ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e70102dd0 0x7f4e70198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38630/0 (socket says 172.21.15.152:38630) 2024-09-11T22:51:59.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e6ffff640 1 -- 172.21.15.152:0/3838376398 learned_addr learned my addr 172.21.15.152:0/3838376398 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:51:59.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e6f7fe640 1 --2- 172.21.15.152:0/3838376398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e70103780 0x7f4e70198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:59.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e74c00640 1 --2- 172.21.15.152:0/3838376398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e70108300 0x7f4e7019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:59.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e6ffff640 1 -- 172.21.15.152:0/3838376398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e70108300 msgr2=0x7f4e7019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:59.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e6ffff640 1 --2- 172.21.15.152:0/3838376398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e70108300 0x7f4e7019ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:59.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e6ffff640 1 -- 172.21.15.152:0/3838376398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e70103780 msgr2=0x7f4e70198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:59.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e6ffff640 1 --2- 172.21.15.152:0/3838376398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e70103780 0x7f4e70198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:59.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e6ffff640 1 -- 172.21.15.152:0/3838376398 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7019d540 con 0x7f4e70102dd0 2024-09-11T22:51:59.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.608+0000 7f4e6ffff640 1 --2- 172.21.15.152:0/3838376398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e70102dd0 0x7f4e70198590 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009df0 tx=0x7f4e60003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:59.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.609+0000 7f4e6d7fa640 1 -- 172.21.15.152:0/3838376398 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60004240 con 0x7f4e70102dd0 2024-09-11T22:51:59.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.609+0000 7f4e6d7fa640 1 -- 172.21.15.152:0/3838376398 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e60040070 con 0x7f4e70102dd0 2024-09-11T22:51:59.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.609+0000 7f4e76663640 1 -- 172.21.15.152:0/3838376398 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7019d7d0 con 0x7f4e70102dd0 2024-09-11T22:51:59.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.609+0000 7f4e6d7fa640 1 -- 172.21.15.152:0/3838376398 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e60045cd0 con 0x7f4e70102dd0 2024-09-11T22:51:59.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.609+0000 7f4e76663640 1 -- 172.21.15.152:0/3838376398 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e701a5110 con 0x7f4e70102dd0 2024-09-11T22:51:59.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.610+0000 7f4e76663640 1 -- 172.21.15.152:0/3838376398 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e701031b0 con 0x7f4e70102dd0 2024-09-11T22:51:59.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.611+0000 7f4e6d7fa640 1 -- 172.21.15.152:0/3838376398 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4e60045470 con 0x7f4e70102dd0 2024-09-11T22:51:59.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.611+0000 7f4e6d7fa640 1 --2- 172.21.15.152:0/3838376398 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4e5c0a2c10 0x7f4e5c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:51:59.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.612+0000 7f4e6f7fe640 1 --2- 172.21.15.152:0/3838376398 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4e5c0a2c10 0x7f4e5c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:51:59.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.612+0000 7f4e6d7fa640 1 -- 172.21.15.152:0/3838376398 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f4e600f05c0 con 0x7f4e70102dd0 2024-09-11T22:51:59.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.612+0000 7f4e6f7fe640 1 --2- 172.21.15.152:0/3838376398 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4e5c0a2c10 0x7f4e5c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4e700fedf0 tx=0x7f4e58009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:51:59.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.614+0000 7f4e6d7fa640 1 -- 172.21.15.152:0/3838376398 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e600bd620 con 0x7f4e70102dd0 2024-09-11T22:51:59.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.793+0000 7f4e76663640 1 -- 172.21.15.152:0/3838376398 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f4e7010a290 con 0x7f4e70102dd0 2024-09-11T22:51:59.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.794+0000 7f4e6d7fa640 1 -- 172.21.15.152:0/3838376398 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4e600bcf20 con 0x7f4e70102dd0 2024-09-11T22:51:59.795 INFO:teuthology.orchestra.run.smithi152.stdout:188978561077 2024-09-11T22:51:59.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.796+0000 7f4e76663640 1 -- 172.21.15.152:0/3838376398 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4e5c0a2c10 msgr2=0x7f4e5c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:59.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.796+0000 7f4e76663640 1 --2- 172.21.15.152:0/3838376398 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4e5c0a2c10 0x7f4e5c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4e700fedf0 tx=0x7f4e58009110 comp rx=0 tx=0).stop 2024-09-11T22:51:59.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.796+0000 7f4e76663640 1 -- 172.21.15.152:0/3838376398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e70102dd0 msgr2=0x7f4e70198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:51:59.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.796+0000 7f4e76663640 1 --2- 172.21.15.152:0/3838376398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e70102dd0 0x7f4e70198590 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009df0 tx=0x7f4e60003ba0 comp rx=0 tx=0).stop 2024-09-11T22:51:59.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.796+0000 7f4e76663640 1 -- 172.21.15.152:0/3838376398 shutdown_connections 2024-09-11T22:51:59.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.796+0000 7f4e76663640 1 --2- 172.21.15.152:0/3838376398 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4e5c0a2c10 0x7f4e5c0a50d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:59.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.796+0000 7f4e76663640 1 --2- 172.21.15.152:0/3838376398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e70108300 0x7f4e7019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:59.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.796+0000 7f4e76663640 1 --2- 172.21.15.152:0/3838376398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e70103780 0x7f4e70198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:59.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.796+0000 7f4e76663640 1 --2- 172.21.15.152:0/3838376398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e70102dd0 0x7f4e70198590 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:51:59.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.796+0000 7f4e76663640 1 -- 172.21.15.152:0/3838376398 >> 172.21.15.152:0/3838376398 conn(0x7f4e700fc7d0 msgr2=0x7f4e70105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:51:59.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.796+0000 7f4e76663640 1 -- 172.21.15.152:0/3838376398 shutdown_connections 2024-09-11T22:51:59.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:51:59.796+0000 7f4e76663640 1 -- 172.21.15.152:0/3838376398 wait complete. 2024-09-11T22:52:00.111 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:51:59 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3838376398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T22:52:00.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:51:59 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3838376398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T22:52:00.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:51:59 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3838376398' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T22:52:01.068 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:00 smithi152 ceph-mon[31181]: pgmap v316: 97 pgs: 3 peering, 94 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-09-11T22:52:01.104 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561073 got 188978561077 for osd.6 2024-09-11T22:52:01.105 DEBUG:teuthology.parallel:result is None 2024-09-11T22:52:01.105 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T22:52:01.105 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:52:01.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:00 smithi191 ceph-mon[35211]: pgmap v316: 97 pgs: 3 peering, 94 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-09-11T22:52:01.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:00 smithi204 ceph-mon[35454]: pgmap v316: 97 pgs: 3 peering, 94 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s 2024-09-11T22:52:01.402 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:01.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.914+0000 7f85a7000640 1 -- 172.21.15.152:0/3077796167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85a0102570 msgr2=0x7f85a01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:01.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.914+0000 7f85a7000640 1 --2- 172.21.15.152:0/3077796167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85a0102570 0x7f85a01029b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f858c009900 tx=0x7f858c02a480 comp rx=0 tx=0).stop 2024-09-11T22:52:01.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.914+0000 7f85a7000640 1 -- 172.21.15.152:0/3077796167 shutdown_connections 2024-09-11T22:52:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.914+0000 7f85a7000640 1 --2- 172.21.15.152:0/3077796167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85a0102ef0 0x7f85a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.914+0000 7f85a7000640 1 --2- 172.21.15.152:0/3077796167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85a0102570 0x7f85a01029b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.914+0000 7f85a7000640 1 --2- 172.21.15.152:0/3077796167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a0100e20 0x7f85a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.914+0000 7f85a7000640 1 -- 172.21.15.152:0/3077796167 >> 172.21.15.152:0/3077796167 conn(0x7f85a00fc7d0 msgr2=0x7f85a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.914+0000 7f85a7000640 1 -- 172.21.15.152:0/3077796167 shutdown_connections 2024-09-11T22:52:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.914+0000 7f85a7000640 1 -- 172.21.15.152:0/3077796167 wait complete. 2024-09-11T22:52:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.915+0000 7f85a7000640 1 Processor -- start 2024-09-11T22:52:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.915+0000 7f85a7000640 1 -- start start 2024-09-11T22:52:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.915+0000 7f85a7000640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85a0100e20 0x7f85a01973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.915+0000 7f85a7000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a0102570 0x7f85a0197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.915+0000 7f85a7000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85a0102ef0 0x7f85a019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:01.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.915+0000 7f85a7000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f85a0110f80 con 0x7f85a0102ef0 2024-09-11T22:52:01.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.915+0000 7f85a7000640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f85a0110e00 con 0x7f85a0100e20 2024-09-11T22:52:01.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.915+0000 7f85a7000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f85a0111100 con 0x7f85a0102570 2024-09-11T22:52:01.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.915+0000 7f85a559d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85a0102ef0 0x7f85a019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:01.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.915+0000 7f85a559d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85a0102ef0 0x7f85a019bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38650/0 (socket says 172.21.15.152:38650) 2024-09-11T22:52:01.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.915+0000 7f85a559d640 1 -- 172.21.15.152:0/2421150464 learned_addr learned my addr 172.21.15.152:0/2421150464 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:01.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.916+0000 7f859ffff640 1 --2- 172.21.15.152:0/2421150464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a0102570 0x7f85a0197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:01.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.916+0000 7f85a4d9c640 1 --2- 172.21.15.152:0/2421150464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85a0100e20 0x7f85a01973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:01.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.916+0000 7f85a559d640 1 -- 172.21.15.152:0/2421150464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a0102570 msgr2=0x7f85a0197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:01.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.916+0000 7f85a559d640 1 --2- 172.21.15.152:0/2421150464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a0102570 0x7f85a0197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:01.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.916+0000 7f85a559d640 1 -- 172.21.15.152:0/2421150464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85a0100e20 msgr2=0x7f85a01973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:01.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.916+0000 7f85a559d640 1 --2- 172.21.15.152:0/2421150464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85a0100e20 0x7f85a01973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:01.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.916+0000 7f85a559d640 1 -- 172.21.15.152:0/2421150464 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a019c390 con 0x7f85a0102ef0 2024-09-11T22:52:01.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.916+0000 7f85a559d640 1 --2- 172.21.15.152:0/2421150464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85a0102ef0 0x7f85a019bcb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8598009fd0 tx=0x7f859800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:01.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.916+0000 7f859dffb640 1 -- 172.21.15.152:0/2421150464 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598017070 con 0x7f85a0102ef0 2024-09-11T22:52:01.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.916+0000 7f85a7000640 1 -- 172.21.15.152:0/2421150464 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a019c680 con 0x7f85a0102ef0 2024-09-11T22:52:01.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.916+0000 7f859dffb640 1 -- 172.21.15.152:0/2421150464 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8598003e50 con 0x7f85a0102ef0 2024-09-11T22:52:01.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.916+0000 7f859dffb640 1 -- 172.21.15.152:0/2421150464 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85980106c0 con 0x7f85a0102ef0 2024-09-11T22:52:01.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.916+0000 7f85a7000640 1 -- 172.21.15.152:0/2421150464 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a01a3f60 con 0x7f85a0102ef0 2024-09-11T22:52:01.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.918+0000 7f85a7000640 1 -- 172.21.15.152:0/2421150464 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a01021b0 con 0x7f85a0102ef0 2024-09-11T22:52:01.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.918+0000 7f859dffb640 1 -- 172.21.15.152:0/2421150464 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8598003970 con 0x7f85a0102ef0 2024-09-11T22:52:01.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.919+0000 7f859dffb640 1 --2- 172.21.15.152:0/2421150464 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f85900a2c10 0x7f85900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:01.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.919+0000 7f85a4d9c640 1 --2- 172.21.15.152:0/2421150464 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f85900a2c10 0x7f85900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:01.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.919+0000 7f859dffb640 1 -- 172.21.15.152:0/2421150464 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f85980cee90 con 0x7f85a0102ef0 2024-09-11T22:52:01.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.920+0000 7f85a4d9c640 1 --2- 172.21.15.152:0/2421150464 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f85900a2c10 0x7f85900a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8594009bd0 tx=0x7f8594005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:01.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:01.923+0000 7f859dffb640 1 -- 172.21.15.152:0/2421150464 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f859809b670 con 0x7f85a0102ef0 2024-09-11T22:52:02.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:02.143+0000 7f85a7000640 1 -- 172.21.15.152:0/2421150464 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85a0062630 con 0x7f85900a2c10 2024-09-11T22:52:02.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:02.150+0000 7f859dffb640 1 -- 172.21.15.152:0/2421150464 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258773 (secure 0 0 0) 0x7f85a0062630 con 0x7f85900a2c10 2024-09-11T22:52:02.154 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:02.155 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-09-11T22:52:02.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:02.153+0000 7f85a7000640 1 -- 172.21.15.152:0/2421150464 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f85900a2c10 msgr2=0x7f85900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:02.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:02.153+0000 7f85a7000640 1 --2- 172.21.15.152:0/2421150464 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f85900a2c10 0x7f85900a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8594009bd0 tx=0x7f8594005f90 comp rx=0 tx=0).stop 2024-09-11T22:52:02.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:02.153+0000 7f85a7000640 1 -- 172.21.15.152:0/2421150464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85a0102ef0 msgr2=0x7f85a019bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:02.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:02.153+0000 7f85a7000640 1 --2- 172.21.15.152:0/2421150464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85a0102ef0 0x7f85a019bcb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8598009fd0 tx=0x7f859800b260 comp rx=0 tx=0).stop 2024-09-11T22:52:02.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:02.153+0000 7f85a7000640 1 -- 172.21.15.152:0/2421150464 shutdown_connections 2024-09-11T22:52:02.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:02.153+0000 7f85a7000640 1 --2- 172.21.15.152:0/2421150464 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f85900a2c10 0x7f85900a50d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:02.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:02.153+0000 7f85a7000640 1 --2- 172.21.15.152:0/2421150464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85a0102ef0 0x7f85a019bcb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:02.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:02.153+0000 7f85a7000640 1 --2- 172.21.15.152:0/2421150464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a0102570 0x7f85a0197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:02.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:02.153+0000 7f85a7000640 1 --2- 172.21.15.152:0/2421150464 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85a0100e20 0x7f85a01973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:02.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:02.153+0000 7f85a7000640 1 -- 172.21.15.152:0/2421150464 >> 172.21.15.152:0/2421150464 conn(0x7f85a00fc7d0 msgr2=0x7f85a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:02.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:02.154+0000 7f85a7000640 1 -- 172.21.15.152:0/2421150464 shutdown_connections 2024-09-11T22:52:02.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:02.154+0000 7f85a7000640 1 -- 172.21.15.152:0/2421150464 wait complete. 2024-09-11T22:52:02.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:02 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:52:02.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:02 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:52:02.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:02 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:52:02.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:02 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:52:02.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:02 smithi152 ceph-mon[31181]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:52:02.940 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-11T22:52:00.779890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659638,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":696,"num_read_kb":470,"num_write":429,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368804,"kb_used_data":11160,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124573020,"statfs":{"total":1151940427776,"available":1151562772480,"internally_reserved":0,"allocated":11427840,"data_stored":7197370,"data_compressed":107505,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":25,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001822"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958996+0000","last_change":"2024-09-11T22:50:54.171410+0000","last_active":"2024-09-11T22:51:55.958996+0000","last_peered":"2024-09-11T22:51:55.958996+0000","last_clean":"2024-09-11T22:51:55.958996+0000","last_became_active":"2024-09-11T22:50:54.171055+0000","last_became_peered":"2024-09-11T22:50:54.171055+0000","last_unstale":"2024-09-11T22:51:55.958996+0000","last_undegraded":"2024-09-11T22:51:55.958996+0000","last_fullsized":"2024-09-11T22:51:55.958996+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:54.767454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956559+0000","last_change":"2024-09-11T22:50:54.168534+0000","last_active":"2024-09-11T22:51:55.956559+0000","last_peered":"2024-09-11T22:51:55.956559+0000","last_clean":"2024-09-11T22:51:55.956559+0000","last_became_active":"2024-09-11T22:50:54.168228+0000","last_became_peered":"2024-09-11T22:50:54.168228+0000","last_unstale":"2024-09-11T22:51:55.956559+0000","last_undegraded":"2024-09-11T22:51:55.956559+0000","last_fullsized":"2024-09-11T22:51:55.956559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:11.380048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956320+0000","last_change":"2024-09-11T22:50:54.168975+0000","last_active":"2024-09-11T22:51:55.956320+0000","last_peered":"2024-09-11T22:51:55.956320+0000","last_clean":"2024-09-11T22:51:55.956320+0000","last_became_active":"2024-09-11T22:50:54.168676+0000","last_became_peered":"2024-09-11T22:50:54.168676+0000","last_unstale":"2024-09-11T22:51:55.956320+0000","last_undegraded":"2024-09-11T22:51:55.956320+0000","last_fullsized":"2024-09-11T22:51:55.956320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:15.848273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.954812+0000","last_change":"2024-09-11T22:50:54.164164+0000","last_active":"2024-09-11T22:51:55.954812+0000","last_peered":"2024-09-11T22:51:55.954812+0000","last_clean":"2024-09-11T22:51:55.954812+0000","last_became_active":"2024-09-11T22:50:54.163805+0000","last_became_peered":"2024-09-11T22:50:54.163805+0000","last_unstale":"2024-09-11T22:51:55.954812+0000","last_undegraded":"2024-09-11T22:51:55.954812+0000","last_fullsized":"2024-09-11T22:51:55.954812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:54.954046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957528+0000","last_change":"2024-09-11T22:50:54.167289+0000","last_active":"2024-09-11T22:51:55.957528+0000","last_peered":"2024-09-11T22:51:55.957528+0000","last_clean":"2024-09-11T22:51:55.957528+0000","last_became_active":"2024-09-11T22:50:54.166958+0000","last_became_peered":"2024-09-11T22:50:54.166958+0000","last_unstale":"2024-09-11T22:51:55.957528+0000","last_undegraded":"2024-09-11T22:51:55.957528+0000","last_fullsized":"2024-09-11T22:51:55.957528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:25.261241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:51:54.949135+0000","last_change":"2024-09-11T22:50:54.175605+0000","last_active":"2024-09-11T22:51:54.949135+0000","last_peered":"2024-09-11T22:51:54.949135+0000","last_clean":"2024-09-11T22:51:54.949135+0000","last_became_active":"2024-09-11T22:50:54.175207+0000","last_became_peered":"2024-09-11T22:50:54.175207+0000","last_unstale":"2024-09-11T22:51:54.949135+0000","last_undegraded":"2024-09-11T22:51:54.949135+0000","last_fullsized":"2024-09-11T22:51:54.949135+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:47.008910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.204888+0000","last_change":"2024-09-11T22:50:54.167930+0000","last_active":"2024-09-11T22:51:56.204888+0000","last_peered":"2024-09-11T22:51:56.204888+0000","last_clean":"2024-09-11T22:51:56.204888+0000","last_became_active":"2024-09-11T22:50:54.167082+0000","last_became_peered":"2024-09-11T22:50:54.167082+0000","last_unstale":"2024-09-11T22:51:56.204888+0000","last_undegraded":"2024-09-11T22:51:56.204888+0000","last_fullsized":"2024-09-11T22:51:56.204888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:50.931311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956868+0000","last_change":"2024-09-11T22:50:54.172528+0000","last_active":"2024-09-11T22:51:55.956868+0000","last_peered":"2024-09-11T22:51:55.956868+0000","last_clean":"2024-09-11T22:51:55.956868+0000","last_became_active":"2024-09-11T22:50:54.172257+0000","last_became_peered":"2024-09-11T22:50:54.172257+0000","last_unstale":"2024-09-11T22:51:55.956868+0000","last_undegraded":"2024-09-11T22:51:55.956868+0000","last_fullsized":"2024-09-11T22:51:55.956868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:13:43.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956803+0000","last_change":"2024-09-11T22:50:54.163357+0000","last_active":"2024-09-11T22:51:55.956803+0000","last_peered":"2024-09-11T22:51:55.956803+0000","last_clean":"2024-09-11T22:51:55.956803+0000","last_became_active":"2024-09-11T22:50:54.162943+0000","last_became_peered":"2024-09-11T22:50:54.162943+0000","last_unstale":"2024-09-11T22:51:55.956803+0000","last_undegraded":"2024-09-11T22:51:55.956803+0000","last_fullsized":"2024-09-11T22:51:55.956803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:57.055802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:51:54.949441+0000","last_change":"2024-09-11T22:50:54.177773+0000","last_active":"2024-09-11T22:51:54.949441+0000","last_peered":"2024-09-11T22:51:54.949441+0000","last_clean":"2024-09-11T22:51:54.949441+0000","last_became_active":"2024-09-11T22:50:54.177388+0000","last_became_peered":"2024-09-11T22:50:54.177388+0000","last_unstale":"2024-09-11T22:51:54.949441+0000","last_undegraded":"2024-09-11T22:51:54.949441+0000","last_fullsized":"2024-09-11T22:51:54.949441+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:36:44.625089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.205239+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:51:56.205239+0000","last_peered":"2024-09-11T22:51:56.205239+0000","last_clean":"2024-09-11T22:51:56.205239+0000","last_became_active":"2024-09-11T22:50:54.171491+0000","last_became_peered":"2024-09-11T22:50:54.171491+0000","last_unstale":"2024-09-11T22:51:56.205239+0000","last_undegraded":"2024-09-11T22:51:56.205239+0000","last_fullsized":"2024-09-11T22:51:56.205239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.996640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.954936+0000","last_change":"2024-09-11T22:50:54.171260+0000","last_active":"2024-09-11T22:51:55.954936+0000","last_peered":"2024-09-11T22:51:55.954936+0000","last_clean":"2024-09-11T22:51:55.954936+0000","last_became_active":"2024-09-11T22:50:54.170733+0000","last_became_peered":"2024-09-11T22:50:54.170733+0000","last_unstale":"2024-09-11T22:51:55.954936+0000","last_undegraded":"2024-09-11T22:51:55.954936+0000","last_fullsized":"2024-09-11T22:51:55.954936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:57.877280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236568+0000","last_change":"2024-09-11T22:50:54.170576+0000","last_active":"2024-09-11T22:51:56.236568+0000","last_peered":"2024-09-11T22:51:56.236568+0000","last_clean":"2024-09-11T22:51:56.236568+0000","last_became_active":"2024-09-11T22:50:54.170239+0000","last_became_peered":"2024-09-11T22:50:54.170239+0000","last_unstale":"2024-09-11T22:51:56.236568+0000","last_undegraded":"2024-09-11T22:51:56.236568+0000","last_fullsized":"2024-09-11T22:51:56.236568+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:09.683171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956983+0000","last_change":"2024-09-11T22:50:54.167176+0000","last_active":"2024-09-11T22:51:55.956983+0000","last_peered":"2024-09-11T22:51:55.956983+0000","last_clean":"2024-09-11T22:51:55.956983+0000","last_became_active":"2024-09-11T22:50:54.166894+0000","last_became_peered":"2024-09-11T22:50:54.166894+0000","last_unstale":"2024-09-11T22:51:55.956983+0000","last_undegraded":"2024-09-11T22:51:55.956983+0000","last_fullsized":"2024-09-11T22:51:55.956983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:43:08.158786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956483+0000","last_change":"2024-09-11T22:50:54.163100+0000","last_active":"2024-09-11T22:51:55.956483+0000","last_peered":"2024-09-11T22:51:55.956483+0000","last_clean":"2024-09-11T22:51:55.956483+0000","last_became_active":"2024-09-11T22:50:54.162776+0000","last_became_peered":"2024-09-11T22:50:54.162776+0000","last_unstale":"2024-09-11T22:51:55.956483+0000","last_undegraded":"2024-09-11T22:51:55.956483+0000","last_fullsized":"2024-09-11T22:51:55.956483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:15.215305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956239+0000","last_change":"2024-09-11T22:50:54.166013+0000","last_active":"2024-09-11T22:51:55.956239+0000","last_peered":"2024-09-11T22:51:55.956239+0000","last_clean":"2024-09-11T22:51:55.956239+0000","last_became_active":"2024-09-11T22:50:54.165680+0000","last_became_peered":"2024-09-11T22:50:54.165680+0000","last_unstale":"2024-09-11T22:51:55.956239+0000","last_undegraded":"2024-09-11T22:51:55.956239+0000","last_fullsized":"2024-09-11T22:51:55.956239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:12.192231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.959096+0000","last_change":"2024-09-11T22:50:54.172105+0000","last_active":"2024-09-11T22:51:55.959096+0000","last_peered":"2024-09-11T22:51:55.959096+0000","last_clean":"2024-09-11T22:51:55.959096+0000","last_became_active":"2024-09-11T22:50:54.171782+0000","last_became_peered":"2024-09-11T22:50:54.171782+0000","last_unstale":"2024-09-11T22:51:55.959096+0000","last_undegraded":"2024-09-11T22:51:55.959096+0000","last_fullsized":"2024-09-11T22:51:55.959096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:17.152834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:51:54.953013+0000","last_change":"2024-09-11T22:50:54.167184+0000","last_active":"2024-09-11T22:51:54.953013+0000","last_peered":"2024-09-11T22:51:54.953013+0000","last_clean":"2024-09-11T22:51:54.953013+0000","last_became_active":"2024-09-11T22:50:54.166878+0000","last_became_peered":"2024-09-11T22:50:54.166878+0000","last_unstale":"2024-09-11T22:51:54.953013+0000","last_undegraded":"2024-09-11T22:51:54.953013+0000","last_fullsized":"2024-09-11T22:51:54.953013+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:44.046552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956241+0000","last_change":"2024-09-11T22:50:54.176260+0000","last_active":"2024-09-11T22:51:55.956241+0000","last_peered":"2024-09-11T22:51:55.956241+0000","last_clean":"2024-09-11T22:51:55.956241+0000","last_became_active":"2024-09-11T22:50:54.175873+0000","last_became_peered":"2024-09-11T22:50:54.175873+0000","last_unstale":"2024-09-11T22:51:55.956241+0000","last_undegraded":"2024-09-11T22:51:55.956241+0000","last_fullsized":"2024-09-11T22:51:55.956241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:05.258677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958831+0000","last_change":"2024-09-11T22:50:54.172725+0000","last_active":"2024-09-11T22:51:55.958831+0000","last_peered":"2024-09-11T22:51:55.958831+0000","last_clean":"2024-09-11T22:51:55.958831+0000","last_became_active":"2024-09-11T22:50:54.172072+0000","last_became_peered":"2024-09-11T22:50:54.172072+0000","last_unstale":"2024-09-11T22:51:55.958831+0000","last_undegraded":"2024-09-11T22:51:55.958831+0000","last_fullsized":"2024-09-11T22:51:55.958831+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:53.699670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236659+0000","last_change":"2024-09-11T22:50:54.165118+0000","last_active":"2024-09-11T22:51:56.236659+0000","last_peered":"2024-09-11T22:51:56.236659+0000","last_clean":"2024-09-11T22:51:56.236659+0000","last_became_active":"2024-09-11T22:50:54.164793+0000","last_became_peered":"2024-09-11T22:50:54.164793+0000","last_unstale":"2024-09-11T22:51:56.236659+0000","last_undegraded":"2024-09-11T22:51:56.236659+0000","last_fullsized":"2024-09-11T22:51:56.236659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:42.883940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956745+0000","last_change":"2024-09-11T22:50:54.176900+0000","last_active":"2024-09-11T22:51:55.956745+0000","last_peered":"2024-09-11T22:51:55.956745+0000","last_clean":"2024-09-11T22:51:55.956745+0000","last_became_active":"2024-09-11T22:50:54.176595+0000","last_became_peered":"2024-09-11T22:50:54.176595+0000","last_unstale":"2024-09-11T22:51:55.956745+0000","last_undegraded":"2024-09-11T22:51:55.956745+0000","last_fullsized":"2024-09-11T22:51:55.956745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:36.925945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956267+0000","last_change":"2024-09-11T22:50:54.167656+0000","last_active":"2024-09-11T22:51:55.956267+0000","last_peered":"2024-09-11T22:51:55.956267+0000","last_clean":"2024-09-11T22:51:55.956267+0000","last_became_active":"2024-09-11T22:50:54.167349+0000","last_became_peered":"2024-09-11T22:50:54.167349+0000","last_unstale":"2024-09-11T22:51:55.956267+0000","last_undegraded":"2024-09-11T22:51:55.956267+0000","last_fullsized":"2024-09-11T22:51:55.956267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:38.100218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957960+0000","last_change":"2024-09-11T22:50:54.177059+0000","last_active":"2024-09-11T22:51:55.957960+0000","last_peered":"2024-09-11T22:51:55.957960+0000","last_clean":"2024-09-11T22:51:55.957960+0000","last_became_active":"2024-09-11T22:50:54.176726+0000","last_became_peered":"2024-09-11T22:50:54.176726+0000","last_unstale":"2024-09-11T22:51:55.957960+0000","last_undegraded":"2024-09-11T22:51:55.957960+0000","last_fullsized":"2024-09-11T22:51:55.957960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:27:58.058944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957129+0000","last_change":"2024-09-11T22:50:54.167566+0000","last_active":"2024-09-11T22:51:55.957129+0000","last_peered":"2024-09-11T22:51:55.957129+0000","last_clean":"2024-09-11T22:51:55.957129+0000","last_became_active":"2024-09-11T22:50:54.167049+0000","last_became_peered":"2024-09-11T22:50:54.167049+0000","last_unstale":"2024-09-11T22:51:55.957129+0000","last_undegraded":"2024-09-11T22:51:55.957129+0000","last_fullsized":"2024-09-11T22:51:55.957129+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:06.833099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956518+0000","last_change":"2024-09-11T22:50:54.169732+0000","last_active":"2024-09-11T22:51:55.956518+0000","last_peered":"2024-09-11T22:51:55.956518+0000","last_clean":"2024-09-11T22:51:55.956518+0000","last_became_active":"2024-09-11T22:50:54.169437+0000","last_became_peered":"2024-09-11T22:50:54.169437+0000","last_unstale":"2024-09-11T22:51:55.956518+0000","last_undegraded":"2024-09-11T22:51:55.956518+0000","last_fullsized":"2024-09-11T22:51:55.956518+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:25.553387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957033+0000","last_change":"2024-09-11T22:50:54.166953+0000","last_active":"2024-09-11T22:51:55.957033+0000","last_peered":"2024-09-11T22:51:55.957033+0000","last_clean":"2024-09-11T22:51:55.957033+0000","last_became_active":"2024-09-11T22:50:54.166472+0000","last_became_peered":"2024-09-11T22:50:54.166472+0000","last_unstale":"2024-09-11T22:51:55.957033+0000","last_undegraded":"2024-09-11T22:51:55.957033+0000","last_fullsized":"2024-09-11T22:51:55.957033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:01:20.234180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236582+0000","last_change":"2024-09-11T22:50:54.171308+0000","last_active":"2024-09-11T22:51:56.236582+0000","last_peered":"2024-09-11T22:51:56.236582+0000","last_clean":"2024-09-11T22:51:56.236582+0000","last_became_active":"2024-09-11T22:50:54.168958+0000","last_became_peered":"2024-09-11T22:50:54.168958+0000","last_unstale":"2024-09-11T22:51:56.236582+0000","last_undegraded":"2024-09-11T22:51:56.236582+0000","last_fullsized":"2024-09-11T22:51:56.236582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:20.224003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956736+0000","last_change":"2024-09-11T22:50:54.173846+0000","last_active":"2024-09-11T22:51:55.956736+0000","last_peered":"2024-09-11T22:51:55.956736+0000","last_clean":"2024-09-11T22:51:55.956736+0000","last_became_active":"2024-09-11T22:50:54.173542+0000","last_became_peered":"2024-09-11T22:50:54.173542+0000","last_unstale":"2024-09-11T22:51:55.956736+0000","last_undegraded":"2024-09-11T22:51:55.956736+0000","last_fullsized":"2024-09-11T22:51:55.956736+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:29:45.775902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956423+0000","last_change":"2024-09-11T22:50:54.176008+0000","last_active":"2024-09-11T22:51:55.956423+0000","last_peered":"2024-09-11T22:51:55.956423+0000","last_clean":"2024-09-11T22:51:55.956423+0000","last_became_active":"2024-09-11T22:50:54.175672+0000","last_became_peered":"2024-09-11T22:50:54.175672+0000","last_unstale":"2024-09-11T22:51:55.956423+0000","last_undegraded":"2024-09-11T22:51:55.956423+0000","last_fullsized":"2024-09-11T22:51:55.956423+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:35.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.955997+0000","last_change":"2024-09-11T22:50:54.162504+0000","last_active":"2024-09-11T22:51:55.955997+0000","last_peered":"2024-09-11T22:51:55.955997+0000","last_clean":"2024-09-11T22:51:55.955997+0000","last_became_active":"2024-09-11T22:50:54.162190+0000","last_became_peered":"2024-09-11T22:50:54.162190+0000","last_unstale":"2024-09-11T22:51:55.955997+0000","last_undegraded":"2024-09-11T22:51:55.955997+0000","last_fullsized":"2024-09-11T22:51:55.955997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:52.138692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.960882+0000","last_change":"2024-09-11T22:51:55.960882+0000","last_active":"2024-09-11T22:51:55.960882+0000","last_peered":"2024-09-11T22:51:55.960882+0000","last_clean":"2024-09-11T22:51:55.960882+0000","last_became_active":"2024-09-11T22:51:55.960246+0000","last_became_peered":"2024-09-11T22:51:55.960246+0000","last_unstale":"2024-09-11T22:51:55.960882+0000","last_undegraded":"2024-09-11T22:51:55.960882+0000","last_fullsized":"2024-09-11T22:51:55.960882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:21.247840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957287+0000","last_change":"2024-09-11T22:50:54.160696+0000","last_active":"2024-09-11T22:51:55.957287+0000","last_peered":"2024-09-11T22:51:55.957287+0000","last_clean":"2024-09-11T22:51:55.957287+0000","last_became_active":"2024-09-11T22:50:54.159249+0000","last_became_peered":"2024-09-11T22:50:54.159249+0000","last_unstale":"2024-09-11T22:51:55.957287+0000","last_undegraded":"2024-09-11T22:51:55.957287+0000","last_fullsized":"2024-09-11T22:51:55.957287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:09.649208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":84,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.203771+0000","last_change":"2024-09-11T22:50:52.138986+0000","last_active":"2024-09-11T22:51:56.203771+0000","last_peered":"2024-09-11T22:51:56.203771+0000","last_clean":"2024-09-11T22:51:56.203771+0000","last_became_active":"2024-09-11T22:50:52.138595+0000","last_became_peered":"2024-09-11T22:50:52.138595+0000","last_unstale":"2024-09-11T22:51:56.203771+0000","last_undegraded":"2024-09-11T22:51:56.203771+0000","last_fullsized":"2024-09-11T22:51:56.203771+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:06.232146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957880+0000","last_change":"2024-09-11T22:50:54.176806+0000","last_active":"2024-09-11T22:51:55.957880+0000","last_peered":"2024-09-11T22:51:55.957880+0000","last_clean":"2024-09-11T22:51:55.957880+0000","last_became_active":"2024-09-11T22:50:54.176513+0000","last_became_peered":"2024-09-11T22:50:54.176513+0000","last_unstale":"2024-09-11T22:51:55.957880+0000","last_undegraded":"2024-09-11T22:51:55.957880+0000","last_fullsized":"2024-09-11T22:51:55.957880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:43:12.086145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:51:54.949193+0000","last_change":"2024-09-11T22:50:52.142794+0000","last_active":"2024-09-11T22:51:54.949193+0000","last_peered":"2024-09-11T22:51:54.949193+0000","last_clean":"2024-09-11T22:51:54.949193+0000","last_became_active":"2024-09-11T22:50:52.142433+0000","last_became_peered":"2024-09-11T22:50:52.142433+0000","last_unstale":"2024-09-11T22:51:54.949193+0000","last_undegraded":"2024-09-11T22:51:54.949193+0000","last_fullsized":"2024-09-11T22:51:54.949193+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:47:12.090055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236342+0000","last_change":"2024-09-11T22:50:52.139789+0000","last_active":"2024-09-11T22:51:56.236342+0000","last_peered":"2024-09-11T22:51:56.236342+0000","last_clean":"2024-09-11T22:51:56.236342+0000","last_became_active":"2024-09-11T22:50:52.139294+0000","last_became_peered":"2024-09-11T22:50:52.139294+0000","last_unstale":"2024-09-11T22:51:56.236342+0000","last_undegraded":"2024-09-11T22:51:56.236342+0000","last_fullsized":"2024-09-11T22:51:56.236342+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:07.355155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956967+0000","last_change":"2024-09-11T22:50:54.166248+0000","last_active":"2024-09-11T22:51:55.956967+0000","last_peered":"2024-09-11T22:51:55.956967+0000","last_clean":"2024-09-11T22:51:55.956967+0000","last_became_active":"2024-09-11T22:50:54.165958+0000","last_became_peered":"2024-09-11T22:50:54.165958+0000","last_unstale":"2024-09-11T22:51:55.956967+0000","last_undegraded":"2024-09-11T22:51:55.956967+0000","last_fullsized":"2024-09-11T22:51:55.956967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:29.531741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958597+0000","last_change":"2024-09-11T22:50:52.140340+0000","last_active":"2024-09-11T22:51:55.958597+0000","last_peered":"2024-09-11T22:51:55.958597+0000","last_clean":"2024-09-11T22:51:55.958597+0000","last_became_active":"2024-09-11T22:50:52.140011+0000","last_became_peered":"2024-09-11T22:50:52.140011+0000","last_unstale":"2024-09-11T22:51:55.958597+0000","last_undegraded":"2024-09-11T22:51:55.958597+0000","last_fullsized":"2024-09-11T22:51:55.958597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:58.862640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:51:54.952305+0000","last_change":"2024-09-11T22:50:54.166998+0000","last_active":"2024-09-11T22:51:54.952305+0000","last_peered":"2024-09-11T22:51:54.952305+0000","last_clean":"2024-09-11T22:51:54.952305+0000","last_became_active":"2024-09-11T22:50:54.166682+0000","last_became_peered":"2024-09-11T22:50:54.166682+0000","last_unstale":"2024-09-11T22:51:54.952305+0000","last_undegraded":"2024-09-11T22:51:54.952305+0000","last_fullsized":"2024-09-11T22:51:54.952305+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:57.815914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.201882+0000","last_change":"2024-09-11T22:50:52.141380+0000","last_active":"2024-09-11T22:51:55.201882+0000","last_peered":"2024-09-11T22:51:55.201882+0000","last_clean":"2024-09-11T22:51:55.201882+0000","last_became_active":"2024-09-11T22:50:52.139925+0000","last_became_peered":"2024-09-11T22:50:52.139925+0000","last_unstale":"2024-09-11T22:51:55.201882+0000","last_undegraded":"2024-09-11T22:51:55.201882+0000","last_fullsized":"2024-09-11T22:51:55.201882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:40:38.395055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956655+0000","last_change":"2024-09-11T22:50:54.167909+0000","last_active":"2024-09-11T22:51:55.956655+0000","last_peered":"2024-09-11T22:51:55.956655+0000","last_clean":"2024-09-11T22:51:55.956655+0000","last_became_active":"2024-09-11T22:50:54.167571+0000","last_became_peered":"2024-09-11T22:50:54.167571+0000","last_unstale":"2024-09-11T22:51:55.956655+0000","last_undegraded":"2024-09-11T22:51:55.956655+0000","last_fullsized":"2024-09-11T22:51:55.956655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:34.179960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.204211+0000","last_change":"2024-09-11T22:50:52.144545+0000","last_active":"2024-09-11T22:51:56.204211+0000","last_peered":"2024-09-11T22:51:56.204211+0000","last_clean":"2024-09-11T22:51:56.204211+0000","last_became_active":"2024-09-11T22:50:52.144221+0000","last_became_peered":"2024-09-11T22:50:52.144221+0000","last_unstale":"2024-09-11T22:51:56.204211+0000","last_undegraded":"2024-09-11T22:51:56.204211+0000","last_fullsized":"2024-09-11T22:51:56.204211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:29.509165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957020+0000","last_change":"2024-09-11T22:50:54.164132+0000","last_active":"2024-09-11T22:51:55.957020+0000","last_peered":"2024-09-11T22:51:55.957020+0000","last_clean":"2024-09-11T22:51:55.957020+0000","last_became_active":"2024-09-11T22:50:54.163842+0000","last_became_peered":"2024-09-11T22:50:54.163842+0000","last_unstale":"2024-09-11T22:51:55.957020+0000","last_undegraded":"2024-09-11T22:51:55.957020+0000","last_fullsized":"2024-09-11T22:51:55.957020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:28:55.490236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958373+0000","last_change":"2024-09-11T22:50:52.264611+0000","last_active":"2024-09-11T22:51:55.958373+0000","last_peered":"2024-09-11T22:51:55.958373+0000","last_clean":"2024-09-11T22:51:55.958373+0000","last_became_active":"2024-09-11T22:50:52.264060+0000","last_became_peered":"2024-09-11T22:50:52.264060+0000","last_unstale":"2024-09-11T22:51:55.958373+0000","last_undegraded":"2024-09-11T22:51:55.958373+0000","last_fullsized":"2024-09-11T22:51:55.958373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:42.543997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.206155+0000","last_change":"2024-09-11T22:51:56.206155+0000","last_active":"2024-09-11T22:51:56.206155+0000","last_peered":"2024-09-11T22:51:56.206155+0000","last_clean":"2024-09-11T22:51:56.206155+0000","last_became_active":"2024-09-11T22:51:56.205797+0000","last_became_peered":"2024-09-11T22:51:56.205797+0000","last_unstale":"2024-09-11T22:51:56.206155+0000","last_undegraded":"2024-09-11T22:51:56.206155+0000","last_fullsized":"2024-09-11T22:51:56.206155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:28.063825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'31","reported_seq":162,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.202250+0000","last_change":"2024-09-11T22:50:52.141782+0000","last_active":"2024-09-11T22:51:55.202250+0000","last_peered":"2024-09-11T22:51:55.202250+0000","last_clean":"2024-09-11T22:51:55.202250+0000","last_became_active":"2024-09-11T22:50:52.140662+0000","last_became_peered":"2024-09-11T22:50:52.140662+0000","last_unstale":"2024-09-11T22:51:55.202250+0000","last_undegraded":"2024-09-11T22:51:55.202250+0000","last_fullsized":"2024-09-11T22:51:55.202250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:21.158577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61828,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":111,"num_read_kb":114,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957900+0000","last_change":"2024-09-11T22:50:54.166678+0000","last_active":"2024-09-11T22:51:55.957900+0000","last_peered":"2024-09-11T22:51:55.957900+0000","last_clean":"2024-09-11T22:51:55.957900+0000","last_became_active":"2024-09-11T22:50:54.165777+0000","last_became_peered":"2024-09-11T22:50:54.165777+0000","last_unstale":"2024-09-11T22:51:55.957900+0000","last_undegraded":"2024-09-11T22:51:55.957900+0000","last_fullsized":"2024-09-11T22:51:55.957900+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:07.224563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956250+0000","last_change":"2024-09-11T22:50:52.141292+0000","last_active":"2024-09-11T22:51:55.956250+0000","last_peered":"2024-09-11T22:51:55.956250+0000","last_clean":"2024-09-11T22:51:55.956250+0000","last_became_active":"2024-09-11T22:50:52.140971+0000","last_became_peered":"2024-09-11T22:50:52.140971+0000","last_unstale":"2024-09-11T22:51:55.956250+0000","last_undegraded":"2024-09-11T22:51:55.956250+0000","last_fullsized":"2024-09-11T22:51:55.956250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:53.039684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236436+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:51:56.236436+0000","last_peered":"2024-09-11T22:51:56.236436+0000","last_clean":"2024-09-11T22:51:56.236436+0000","last_became_active":"2024-09-11T22:50:54.171520+0000","last_became_peered":"2024-09-11T22:50:54.171520+0000","last_unstale":"2024-09-11T22:51:56.236436+0000","last_undegraded":"2024-09-11T22:51:56.236436+0000","last_fullsized":"2024-09-11T22:51:56.236436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:56.594058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957905+0000","last_change":"2024-09-11T22:50:52.138725+0000","last_active":"2024-09-11T22:51:55.957905+0000","last_peered":"2024-09-11T22:51:55.957905+0000","last_clean":"2024-09-11T22:51:55.957905+0000","last_became_active":"2024-09-11T22:50:52.138367+0000","last_became_peered":"2024-09-11T22:50:52.138367+0000","last_unstale":"2024-09-11T22:51:55.957905+0000","last_undegraded":"2024-09-11T22:51:55.957905+0000","last_fullsized":"2024-09-11T22:51:55.957905+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:58.811444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.205138+0000","last_change":"2024-09-11T22:50:54.170524+0000","last_active":"2024-09-11T22:51:56.205138+0000","last_peered":"2024-09-11T22:51:56.205138+0000","last_clean":"2024-09-11T22:51:56.205138+0000","last_became_active":"2024-09-11T22:50:54.169947+0000","last_became_peered":"2024-09-11T22:50:54.169947+0000","last_unstale":"2024-09-11T22:51:56.205138+0000","last_undegraded":"2024-09-11T22:51:56.205138+0000","last_fullsized":"2024-09-11T22:51:56.205138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:51.768718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.954981+0000","last_change":"2024-09-11T22:50:52.139182+0000","last_active":"2024-09-11T22:51:55.954981+0000","last_peered":"2024-09-11T22:51:55.954981+0000","last_clean":"2024-09-11T22:51:55.954981+0000","last_became_active":"2024-09-11T22:50:52.138799+0000","last_became_peered":"2024-09-11T22:50:52.138799+0000","last_unstale":"2024-09-11T22:51:55.954981+0000","last_undegraded":"2024-09-11T22:51:55.954981+0000","last_fullsized":"2024-09-11T22:51:55.954981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:36.376162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.204819+0000","last_change":"2024-09-11T22:50:54.168797+0000","last_active":"2024-09-11T22:51:56.204819+0000","last_peered":"2024-09-11T22:51:56.204819+0000","last_clean":"2024-09-11T22:51:56.204819+0000","last_became_active":"2024-09-11T22:50:54.168408+0000","last_became_peered":"2024-09-11T22:50:54.168408+0000","last_unstale":"2024-09-11T22:51:56.204819+0000","last_undegraded":"2024-09-11T22:51:56.204819+0000","last_fullsized":"2024-09-11T22:51:56.204819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:51.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956437+0000","last_change":"2024-09-11T22:50:52.264693+0000","last_active":"2024-09-11T22:51:55.956437+0000","last_peered":"2024-09-11T22:51:55.956437+0000","last_clean":"2024-09-11T22:51:55.956437+0000","last_became_active":"2024-09-11T22:50:52.264058+0000","last_became_peered":"2024-09-11T22:50:52.264058+0000","last_unstale":"2024-09-11T22:51:55.956437+0000","last_undegraded":"2024-09-11T22:51:55.956437+0000","last_fullsized":"2024-09-11T22:51:55.956437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:46.985921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.961118+0000","last_change":"2024-09-11T22:51:55.961118+0000","last_active":"2024-09-11T22:51:55.961118+0000","last_peered":"2024-09-11T22:51:55.961118+0000","last_clean":"2024-09-11T22:51:55.961118+0000","last_became_active":"2024-09-11T22:51:55.960614+0000","last_became_peered":"2024-09-11T22:51:55.960614+0000","last_unstale":"2024-09-11T22:51:55.961118+0000","last_undegraded":"2024-09-11T22:51:55.961118+0000","last_fullsized":"2024-09-11T22:51:55.961118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:47.184327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958401+0000","last_change":"2024-09-11T22:50:52.133049+0000","last_active":"2024-09-11T22:51:55.958401+0000","last_peered":"2024-09-11T22:51:55.958401+0000","last_clean":"2024-09-11T22:51:55.958401+0000","last_became_active":"2024-09-11T22:50:52.132647+0000","last_became_peered":"2024-09-11T22:50:52.132647+0000","last_unstale":"2024-09-11T22:51:55.958401+0000","last_undegraded":"2024-09-11T22:51:55.958401+0000","last_fullsized":"2024-09-11T22:51:55.958401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:13.213003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.238023+0000","last_change":"2024-09-11T22:51:56.238023+0000","last_active":"2024-09-11T22:51:56.238023+0000","last_peered":"2024-09-11T22:51:56.238023+0000","last_clean":"2024-09-11T22:51:56.238023+0000","last_became_active":"2024-09-11T22:51:56.237676+0000","last_became_peered":"2024-09-11T22:51:56.237676+0000","last_unstale":"2024-09-11T22:51:56.238023+0000","last_undegraded":"2024-09-11T22:51:56.238023+0000","last_fullsized":"2024-09-11T22:51:56.238023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:16.215961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236361+0000","last_change":"2024-09-11T22:50:52.139766+0000","last_active":"2024-09-11T22:51:56.236361+0000","last_peered":"2024-09-11T22:51:56.236361+0000","last_clean":"2024-09-11T22:51:56.236361+0000","last_became_active":"2024-09-11T22:50:52.139456+0000","last_became_peered":"2024-09-11T22:50:52.139456+0000","last_unstale":"2024-09-11T22:51:56.236361+0000","last_undegraded":"2024-09-11T22:51:56.236361+0000","last_fullsized":"2024-09-11T22:51:56.236361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:08.358649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.205373+0000","last_change":"2024-09-11T22:50:54.167717+0000","last_active":"2024-09-11T22:51:56.205373+0000","last_peered":"2024-09-11T22:51:56.205373+0000","last_clean":"2024-09-11T22:51:56.205373+0000","last_became_active":"2024-09-11T22:50:54.167331+0000","last_became_peered":"2024-09-11T22:50:54.167331+0000","last_unstale":"2024-09-11T22:51:56.205373+0000","last_undegraded":"2024-09-11T22:51:56.205373+0000","last_fullsized":"2024-09-11T22:51:56.205373+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:11.240351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958495+0000","last_change":"2024-09-11T22:50:52.141682+0000","last_active":"2024-09-11T22:51:55.958495+0000","last_peered":"2024-09-11T22:51:55.958495+0000","last_clean":"2024-09-11T22:51:55.958495+0000","last_became_active":"2024-09-11T22:50:52.141251+0000","last_became_peered":"2024-09-11T22:50:52.141251+0000","last_unstale":"2024-09-11T22:51:55.958495+0000","last_undegraded":"2024-09-11T22:51:55.958495+0000","last_fullsized":"2024-09-11T22:51:55.958495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:03.068252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.205207+0000","last_change":"2024-09-11T22:50:54.174149+0000","last_active":"2024-09-11T22:51:56.205207+0000","last_peered":"2024-09-11T22:51:56.205207+0000","last_clean":"2024-09-11T22:51:56.205207+0000","last_became_active":"2024-09-11T22:50:54.173765+0000","last_became_peered":"2024-09-11T22:50:54.173765+0000","last_unstale":"2024-09-11T22:51:56.205207+0000","last_undegraded":"2024-09-11T22:51:56.205207+0000","last_fullsized":"2024-09-11T22:51:56.205207+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.314334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":86,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:58.206840+0000","last_change":"2024-09-11T22:50:52.435019+0000","last_active":"2024-09-11T22:51:58.206840+0000","last_peered":"2024-09-11T22:51:58.206840+0000","last_clean":"2024-09-11T22:51:58.206840+0000","last_became_active":"2024-09-11T22:50:52.434642+0000","last_became_peered":"2024-09-11T22:50:52.434642+0000","last_unstale":"2024-09-11T22:51:58.206840+0000","last_undegraded":"2024-09-11T22:51:58.206840+0000","last_fullsized":"2024-09-11T22:51:58.206840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:18.441841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.205327+0000","last_change":"2024-09-11T22:50:54.171291+0000","last_active":"2024-09-11T22:51:56.205327+0000","last_peered":"2024-09-11T22:51:56.205327+0000","last_clean":"2024-09-11T22:51:56.205327+0000","last_became_active":"2024-09-11T22:50:54.170961+0000","last_became_peered":"2024-09-11T22:50:54.170961+0000","last_unstale":"2024-09-11T22:51:56.205327+0000","last_undegraded":"2024-09-11T22:51:56.205327+0000","last_fullsized":"2024-09-11T22:51:56.205327+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:07.211936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":345,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.204956+0000","last_change":"2024-09-11T22:49:39.098881+0000","last_active":"2024-09-11T22:51:56.204956+0000","last_peered":"2024-09-11T22:51:56.204956+0000","last_clean":"2024-09-11T22:51:56.204956+0000","last_became_active":"2024-09-11T22:49:39.098458+0000","last_became_peered":"2024-09-11T22:49:39.098458+0000","last_unstale":"2024-09-11T22:51:56.204956+0000","last_undegraded":"2024-09-11T22:51:56.204956+0000","last_fullsized":"2024-09-11T22:51:56.204956+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_clean_scrub_stamp":"2024-09-11T22:46:11.418577+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:41.059384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.954679+0000","last_change":"2024-09-11T22:50:52.137691+0000","last_active":"2024-09-11T22:51:55.954679+0000","last_peered":"2024-09-11T22:51:55.954679+0000","last_clean":"2024-09-11T22:51:55.954679+0000","last_became_active":"2024-09-11T22:50:52.137282+0000","last_became_peered":"2024-09-11T22:50:52.137282+0000","last_unstale":"2024-09-11T22:51:55.954679+0000","last_undegraded":"2024-09-11T22:51:55.954679+0000","last_fullsized":"2024-09-11T22:51:55.954679+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:00:27.651252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":2,"reported_epoch":82,"state":"peering","last_fresh":"2024-09-11T22:50:54.149424+0000","last_change":"2024-09-11T22:51:55.825779+0000","last_active":"2024-09-11T22:50:53.134241+0000","last_peered":"2024-09-11T22:50:53.134241+0000","last_clean":"2024-09-11T22:50:53.134241+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T22:50:54.149424+0000","last_undegraded":"2024-09-11T22:50:54.149424+0000","last_fullsized":"2024-09-11T22:50:54.149424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.204129+0000","last_change":"2024-09-11T22:50:52.144324+0000","last_active":"2024-09-11T22:51:56.204129+0000","last_peered":"2024-09-11T22:51:56.204129+0000","last_clean":"2024-09-11T22:51:56.204129+0000","last_became_active":"2024-09-11T22:50:52.144011+0000","last_became_peered":"2024-09-11T22:50:52.144011+0000","last_unstale":"2024-09-11T22:51:56.204129+0000","last_undegraded":"2024-09-11T22:51:56.204129+0000","last_fullsized":"2024-09-11T22:51:56.204129+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:29.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236482+0000","last_change":"2024-09-11T22:50:54.173287+0000","last_active":"2024-09-11T22:51:56.236482+0000","last_peered":"2024-09-11T22:51:56.236482+0000","last_clean":"2024-09-11T22:51:56.236482+0000","last_became_active":"2024-09-11T22:50:54.172953+0000","last_became_peered":"2024-09-11T22:50:54.172953+0000","last_unstale":"2024-09-11T22:51:56.236482+0000","last_undegraded":"2024-09-11T22:51:56.236482+0000","last_fullsized":"2024-09-11T22:51:56.236482+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:24.314604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236837+0000","last_change":"2024-09-11T22:50:52.263777+0000","last_active":"2024-09-11T22:51:56.236837+0000","last_peered":"2024-09-11T22:51:56.236837+0000","last_clean":"2024-09-11T22:51:56.236837+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:51:56.236837+0000","last_undegraded":"2024-09-11T22:51:56.236837+0000","last_fullsized":"2024-09-11T22:51:56.236837+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:37.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958445+0000","last_change":"2024-09-11T22:50:54.164927+0000","last_active":"2024-09-11T22:51:55.958445+0000","last_peered":"2024-09-11T22:51:55.958445+0000","last_clean":"2024-09-11T22:51:55.958445+0000","last_became_active":"2024-09-11T22:50:54.164602+0000","last_became_peered":"2024-09-11T22:50:54.164602+0000","last_unstale":"2024-09-11T22:51:55.958445+0000","last_undegraded":"2024-09-11T22:51:55.958445+0000","last_fullsized":"2024-09-11T22:51:55.958445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:10.226232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":80,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.204561+0000","last_change":"2024-09-11T22:50:52.141199+0000","last_active":"2024-09-11T22:51:56.204561+0000","last_peered":"2024-09-11T22:51:56.204561+0000","last_clean":"2024-09-11T22:51:56.204561+0000","last_became_active":"2024-09-11T22:50:52.140830+0000","last_became_peered":"2024-09-11T22:50:52.140830+0000","last_unstale":"2024-09-11T22:51:56.204561+0000","last_undegraded":"2024-09-11T22:51:56.204561+0000","last_fullsized":"2024-09-11T22:51:56.204561+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.767803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:51:54.949266+0000","last_change":"2024-09-11T22:50:54.175054+0000","last_active":"2024-09-11T22:51:54.949266+0000","last_peered":"2024-09-11T22:51:54.949266+0000","last_clean":"2024-09-11T22:51:54.949266+0000","last_became_active":"2024-09-11T22:50:54.174715+0000","last_became_peered":"2024-09-11T22:50:54.174715+0000","last_unstale":"2024-09-11T22:51:54.949266+0000","last_undegraded":"2024-09-11T22:51:54.949266+0000","last_fullsized":"2024-09-11T22:51:54.949266+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:06.694988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"peering","last_fresh":"2024-09-11T22:51:04.230690+0000","last_change":"2024-09-11T22:51:54.953395+0000","last_active":"2024-09-11T22:51:54.953395+0000","last_peered":"2024-09-11T22:51:04.230690+0000","last_clean":"2024-09-11T22:51:04.230690+0000","last_became_active":"2024-09-11T22:50:52.263882+0000","last_became_peered":"2024-09-11T22:50:52.263882+0000","last_unstale":"2024-09-11T22:51:04.230690+0000","last_undegraded":"2024-09-11T22:51:04.230690+0000","last_fullsized":"2024-09-11T22:51:04.230690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,2,11],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:51:54.949513+0000","last_change":"2024-09-11T22:50:54.167361+0000","last_active":"2024-09-11T22:51:54.949513+0000","last_peered":"2024-09-11T22:51:54.949513+0000","last_clean":"2024-09-11T22:51:54.949513+0000","last_became_active":"2024-09-11T22:50:54.166963+0000","last_became_peered":"2024-09-11T22:50:54.166963+0000","last_unstale":"2024-09-11T22:51:54.949513+0000","last_undegraded":"2024-09-11T22:51:54.949513+0000","last_fullsized":"2024-09-11T22:51:54.949513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:13.294544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958100+0000","last_change":"2024-09-11T22:50:52.264484+0000","last_active":"2024-09-11T22:51:55.958100+0000","last_peered":"2024-09-11T22:51:55.958100+0000","last_clean":"2024-09-11T22:51:55.958100+0000","last_became_active":"2024-09-11T22:50:52.264038+0000","last_became_peered":"2024-09-11T22:50:52.264038+0000","last_unstale":"2024-09-11T22:51:55.958100+0000","last_undegraded":"2024-09-11T22:51:55.958100+0000","last_fullsized":"2024-09-11T22:51:55.958100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:21.325136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236614+0000","last_change":"2024-09-11T22:50:54.162225+0000","last_active":"2024-09-11T22:51:56.236614+0000","last_peered":"2024-09-11T22:51:56.236614+0000","last_clean":"2024-09-11T22:51:56.236614+0000","last_became_active":"2024-09-11T22:50:54.161234+0000","last_became_peered":"2024-09-11T22:50:54.161234+0000","last_unstale":"2024-09-11T22:51:56.236614+0000","last_undegraded":"2024-09-11T22:51:56.236614+0000","last_fullsized":"2024-09-11T22:51:56.236614+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:00.346217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'16","reported_seq":35,"reported_epoch":81,"state":"peering","last_fresh":"2024-09-11T22:51:21.163176+0000","last_change":"2024-09-11T22:51:54.953659+0000","last_active":"2024-09-11T22:51:54.953659+0000","last_peered":"2024-09-11T22:51:21.163176+0000","last_clean":"2024-09-11T22:51:21.163176+0000","last_became_active":"2024-09-11T22:50:52.141931+0000","last_became_peered":"2024-09-11T22:50:52.141931+0000","last_unstale":"2024-09-11T22:51:21.163176+0000","last_undegraded":"2024-09-11T22:51:21.163176+0000","last_fullsized":"2024-09-11T22:51:21.163176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,8,11],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957222+0000","last_change":"2024-09-11T22:50:54.167307+0000","last_active":"2024-09-11T22:51:55.957222+0000","last_peered":"2024-09-11T22:51:55.957222+0000","last_clean":"2024-09-11T22:51:55.957222+0000","last_became_active":"2024-09-11T22:50:54.166650+0000","last_became_peered":"2024-09-11T22:50:54.166650+0000","last_unstale":"2024-09-11T22:51:55.957222+0000","last_undegraded":"2024-09-11T22:51:55.957222+0000","last_fullsized":"2024-09-11T22:51:55.957222+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:04.863258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.955051+0000","last_change":"2024-09-11T22:50:52.435107+0000","last_active":"2024-09-11T22:51:55.955051+0000","last_peered":"2024-09-11T22:51:55.955051+0000","last_clean":"2024-09-11T22:51:55.955051+0000","last_became_active":"2024-09-11T22:50:52.434791+0000","last_became_peered":"2024-09-11T22:50:52.434791+0000","last_unstale":"2024-09-11T22:51:55.955051+0000","last_undegraded":"2024-09-11T22:51:55.955051+0000","last_fullsized":"2024-09-11T22:51:55.955051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:11.128791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956652+0000","last_change":"2024-09-11T22:50:54.178041+0000","last_active":"2024-09-11T22:51:55.956652+0000","last_peered":"2024-09-11T22:51:55.956652+0000","last_clean":"2024-09-11T22:51:55.956652+0000","last_became_active":"2024-09-11T22:50:54.177393+0000","last_became_peered":"2024-09-11T22:50:54.177393+0000","last_unstale":"2024-09-11T22:51:55.956652+0000","last_undegraded":"2024-09-11T22:51:55.956652+0000","last_fullsized":"2024-09-11T22:51:55.956652+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:21.608104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956338+0000","last_change":"2024-09-11T22:50:52.264051+0000","last_active":"2024-09-11T22:51:55.956338+0000","last_peered":"2024-09-11T22:51:55.956338+0000","last_clean":"2024-09-11T22:51:55.956338+0000","last_became_active":"2024-09-11T22:50:52.263674+0000","last_became_peered":"2024-09-11T22:50:52.263674+0000","last_unstale":"2024-09-11T22:51:55.956338+0000","last_undegraded":"2024-09-11T22:51:55.956338+0000","last_fullsized":"2024-09-11T22:51:55.956338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:44.655218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957107+0000","last_change":"2024-09-11T22:50:54.166409+0000","last_active":"2024-09-11T22:51:55.957107+0000","last_peered":"2024-09-11T22:51:55.957107+0000","last_clean":"2024-09-11T22:51:55.957107+0000","last_became_active":"2024-09-11T22:50:54.166037+0000","last_became_peered":"2024-09-11T22:50:54.166037+0000","last_unstale":"2024-09-11T22:51:55.957107+0000","last_undegraded":"2024-09-11T22:51:55.957107+0000","last_fullsized":"2024-09-11T22:51:55.957107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:22.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956458+0000","last_change":"2024-09-11T22:50:52.140218+0000","last_active":"2024-09-11T22:51:55.956458+0000","last_peered":"2024-09-11T22:51:55.956458+0000","last_clean":"2024-09-11T22:51:55.956458+0000","last_became_active":"2024-09-11T22:50:52.139744+0000","last_became_peered":"2024-09-11T22:50:52.139744+0000","last_unstale":"2024-09-11T22:51:55.956458+0000","last_undegraded":"2024-09-11T22:51:55.956458+0000","last_fullsized":"2024-09-11T22:51:55.956458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:50.178348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957560+0000","last_change":"2024-09-11T22:50:54.164059+0000","last_active":"2024-09-11T22:51:55.957560+0000","last_peered":"2024-09-11T22:51:55.957560+0000","last_clean":"2024-09-11T22:51:55.957560+0000","last_became_active":"2024-09-11T22:50:54.163724+0000","last_became_peered":"2024-09-11T22:50:54.163724+0000","last_unstale":"2024-09-11T22:51:55.957560+0000","last_undegraded":"2024-09-11T22:51:55.957560+0000","last_fullsized":"2024-09-11T22:51:55.957560+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:57.241660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956526+0000","last_change":"2024-09-11T22:50:52.140169+0000","last_active":"2024-09-11T22:51:55.956526+0000","last_peered":"2024-09-11T22:51:55.956526+0000","last_clean":"2024-09-11T22:51:55.956526+0000","last_became_active":"2024-09-11T22:50:52.139579+0000","last_became_peered":"2024-09-11T22:50:52.139579+0000","last_unstale":"2024-09-11T22:51:55.956526+0000","last_undegraded":"2024-09-11T22:51:55.956526+0000","last_fullsized":"2024-09-11T22:51:55.956526+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:43.523961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956621+0000","last_change":"2024-09-11T22:50:54.171539+0000","last_active":"2024-09-11T22:51:55.956621+0000","last_peered":"2024-09-11T22:51:55.956621+0000","last_clean":"2024-09-11T22:51:55.956621+0000","last_became_active":"2024-09-11T22:50:54.171013+0000","last_became_peered":"2024-09-11T22:50:54.171013+0000","last_unstale":"2024-09-11T22:51:55.956621+0000","last_undegraded":"2024-09-11T22:51:55.956621+0000","last_fullsized":"2024-09-11T22:51:55.956621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:03.564389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957633+0000","last_change":"2024-09-11T22:50:52.139544+0000","last_active":"2024-09-11T22:51:55.957633+0000","last_peered":"2024-09-11T22:51:55.957633+0000","last_clean":"2024-09-11T22:51:55.957633+0000","last_became_active":"2024-09-11T22:50:52.139201+0000","last_became_peered":"2024-09-11T22:50:52.139201+0000","last_unstale":"2024-09-11T22:51:55.957633+0000","last_undegraded":"2024-09-11T22:51:55.957633+0000","last_fullsized":"2024-09-11T22:51:55.957633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:37.343437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.205084+0000","last_change":"2024-09-11T22:50:54.170052+0000","last_active":"2024-09-11T22:51:56.205084+0000","last_peered":"2024-09-11T22:51:56.205084+0000","last_clean":"2024-09-11T22:51:56.205084+0000","last_became_active":"2024-09-11T22:50:54.169735+0000","last_became_peered":"2024-09-11T22:50:54.169735+0000","last_unstale":"2024-09-11T22:51:56.205084+0000","last_undegraded":"2024-09-11T22:51:56.205084+0000","last_fullsized":"2024-09-11T22:51:56.205084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:44.007747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958279+0000","last_change":"2024-09-11T22:50:54.169959+0000","last_active":"2024-09-11T22:51:55.958279+0000","last_peered":"2024-09-11T22:51:55.958279+0000","last_clean":"2024-09-11T22:51:55.958279+0000","last_became_active":"2024-09-11T22:50:54.169649+0000","last_became_peered":"2024-09-11T22:50:54.169649+0000","last_unstale":"2024-09-11T22:51:55.958279+0000","last_undegraded":"2024-09-11T22:51:55.958279+0000","last_fullsized":"2024-09-11T22:51:55.958279+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:51.974258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957383+0000","last_change":"2024-09-11T22:50:52.140881+0000","last_active":"2024-09-11T22:51:55.957383+0000","last_peered":"2024-09-11T22:51:55.957383+0000","last_clean":"2024-09-11T22:51:55.957383+0000","last_became_active":"2024-09-11T22:50:52.140527+0000","last_became_peered":"2024-09-11T22:50:52.140527+0000","last_unstale":"2024-09-11T22:51:55.957383+0000","last_undegraded":"2024-09-11T22:51:55.957383+0000","last_fullsized":"2024-09-11T22:51:55.957383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:14.325816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956384+0000","last_change":"2024-09-11T22:50:54.166571+0000","last_active":"2024-09-11T22:51:55.956384+0000","last_peered":"2024-09-11T22:51:55.956384+0000","last_clean":"2024-09-11T22:51:55.956384+0000","last_became_active":"2024-09-11T22:50:54.166247+0000","last_became_peered":"2024-09-11T22:50:54.166247+0000","last_unstale":"2024-09-11T22:51:55.956384+0000","last_undegraded":"2024-09-11T22:51:55.956384+0000","last_fullsized":"2024-09-11T22:51:55.956384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:42.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957998+0000","last_change":"2024-09-11T22:50:52.140298+0000","last_active":"2024-09-11T22:51:55.957998+0000","last_peered":"2024-09-11T22:51:55.957998+0000","last_clean":"2024-09-11T22:51:55.957998+0000","last_became_active":"2024-09-11T22:50:52.139917+0000","last_became_peered":"2024-09-11T22:50:52.139917+0000","last_unstale":"2024-09-11T22:51:55.957998+0000","last_undegraded":"2024-09-11T22:51:55.957998+0000","last_fullsized":"2024-09-11T22:51:55.957998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:23.599010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956938+0000","last_change":"2024-09-11T22:50:54.170849+0000","last_active":"2024-09-11T22:51:55.956938+0000","last_peered":"2024-09-11T22:51:55.956938+0000","last_clean":"2024-09-11T22:51:55.956938+0000","last_became_active":"2024-09-11T22:50:54.169547+0000","last_became_peered":"2024-09-11T22:50:54.169547+0000","last_unstale":"2024-09-11T22:51:55.956938+0000","last_undegraded":"2024-09-11T22:51:55.956938+0000","last_fullsized":"2024-09-11T22:51:55.956938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:25.251158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956616+0000","last_change":"2024-09-11T22:50:52.263778+0000","last_active":"2024-09-11T22:51:55.956616+0000","last_peered":"2024-09-11T22:51:55.956616+0000","last_clean":"2024-09-11T22:51:55.956616+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:51:55.956616+0000","last_undegraded":"2024-09-11T22:51:55.956616+0000","last_fullsized":"2024-09-11T22:51:55.956616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:03.439147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956438+0000","last_change":"2024-09-11T22:50:54.171200+0000","last_active":"2024-09-11T22:51:55.956438+0000","last_peered":"2024-09-11T22:51:55.956438+0000","last_clean":"2024-09-11T22:51:55.956438+0000","last_became_active":"2024-09-11T22:50:54.169320+0000","last_became_peered":"2024-09-11T22:50:54.169320+0000","last_unstale":"2024-09-11T22:51:55.956438+0000","last_undegraded":"2024-09-11T22:51:55.956438+0000","last_fullsized":"2024-09-11T22:51:55.956438+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:14.864699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":84,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:51:54.953331+0000","last_change":"2024-09-11T22:50:52.139609+0000","last_active":"2024-09-11T22:51:54.953331+0000","last_peered":"2024-09-11T22:51:54.953331+0000","last_clean":"2024-09-11T22:51:54.953331+0000","last_became_active":"2024-09-11T22:50:52.139278+0000","last_became_peered":"2024-09-11T22:50:52.139278+0000","last_unstale":"2024-09-11T22:51:54.953331+0000","last_undegraded":"2024-09-11T22:51:54.953331+0000","last_fullsized":"2024-09-11T22:51:54.953331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:09.538734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69036,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":560,"num_read_kb":351,"num_write":246,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":207108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281769,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":905216,"data_stored":437860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776159,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1048576,"data_stored":1081902,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":917504,"data_stored":428975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168618,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":925696,"data_stored":437775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397551,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":978944,"data_stored":1030295,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561077,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":437114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790010,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1029973,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":913408,"data_stored":436975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":436939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":995328,"data_stored":500861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":974848,"data_stored":499855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":933888,"data_stored":438846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:52:02.945 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:52:03.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:02 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:52:03.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:02 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:52:03.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:02 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:52:03.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:02 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:52:03.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:02 smithi191 ceph-mon[35211]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:52:03.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:02 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:52:03.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:02 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:52:03.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:02 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:52:03.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:02 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:52:03.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:02 smithi204 ceph-mon[35454]: from='mgr.14150 172.21.15.152:0/1902049444' entity='mgr.x' 2024-09-11T22:52:03.194 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.715+0000 7f670ab25640 1 -- 172.21.15.152:0/308389804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6704102570 msgr2=0x7f67041029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.715+0000 7f670ab25640 1 --2- 172.21.15.152:0/308389804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6704102570 0x7f67041029b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009900 tx=0x7f66f002a480 comp rx=0 tx=0).stop 2024-09-11T22:52:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.715+0000 7f670ab25640 1 -- 172.21.15.152:0/308389804 shutdown_connections 2024-09-11T22:52:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.715+0000 7f670ab25640 1 --2- 172.21.15.152:0/308389804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6704102ef0 0x7f670410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.715+0000 7f670ab25640 1 --2- 172.21.15.152:0/308389804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6704102570 0x7f67041029b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.715+0000 7f670ab25640 1 --2- 172.21.15.152:0/308389804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6704100e20 0x7f6704101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:03.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.715+0000 7f670ab25640 1 -- 172.21.15.152:0/308389804 >> 172.21.15.152:0/308389804 conn(0x7f67040fc7d0 msgr2=0x7f67040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.715+0000 7f670ab25640 1 -- 172.21.15.152:0/308389804 shutdown_connections 2024-09-11T22:52:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.715+0000 7f670ab25640 1 -- 172.21.15.152:0/308389804 wait complete. 2024-09-11T22:52:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.716+0000 7f670ab25640 1 Processor -- start 2024-09-11T22:52:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.716+0000 7f670ab25640 1 -- start start 2024-09-11T22:52:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.716+0000 7f670ab25640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6704100e20 0x7f6704198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.716+0000 7f670ab25640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6704102570 0x7f6704198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.716+0000 7f670ab25640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6704102ef0 0x7f670419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.716+0000 7f670ab25640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6704110fd0 con 0x7f6704102570 2024-09-11T22:52:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.716+0000 7f670ab25640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6704110e50 con 0x7f6704100e20 2024-09-11T22:52:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.716+0000 7f670ab25640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6704111150 con 0x7f6704102ef0 2024-09-11T22:52:03.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.716+0000 7f6703fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6704102570 0x7f6704198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:03.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.716+0000 7f6703fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6704102570 0x7f6704198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38660/0 (socket says 172.21.15.152:38660) 2024-09-11T22:52:03.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.716+0000 7f6703fff640 1 -- 172.21.15.152:0/982426150 learned_addr learned my addr 172.21.15.152:0/982426150 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:03.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.716+0000 7f67088c1640 1 --2- 172.21.15.152:0/982426150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6704100e20 0x7f6704198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:03.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.716+0000 7f67090c2640 1 --2- 172.21.15.152:0/982426150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6704102ef0 0x7f670419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:03.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.717+0000 7f6703fff640 1 -- 172.21.15.152:0/982426150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6704102ef0 msgr2=0x7f670419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:03.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.717+0000 7f6703fff640 1 --2- 172.21.15.152:0/982426150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6704102ef0 0x7f670419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:03.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.717+0000 7f6703fff640 1 -- 172.21.15.152:0/982426150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6704100e20 msgr2=0x7f6704198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:03.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.717+0000 7f6703fff640 1 --2- 172.21.15.152:0/982426150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6704100e20 0x7f6704198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:03.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.717+0000 7f6703fff640 1 -- 172.21.15.152:0/982426150 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670419d3f0 con 0x7f6704102570 2024-09-11T22:52:03.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.717+0000 7f67090c2640 1 --2- 172.21.15.152:0/982426150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6704102ef0 0x7f670419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:03.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.717+0000 7f6703fff640 1 --2- 172.21.15.152:0/982426150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6704102570 0x7f6704198980 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f66f0006010 tx=0x7f66f0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:03.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.717+0000 7f6701ffb640 1 -- 172.21.15.152:0/982426150 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f003f070 con 0x7f6704102570 2024-09-11T22:52:03.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.717+0000 7f670ab25640 1 -- 172.21.15.152:0/982426150 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670419d680 con 0x7f6704102570 2024-09-11T22:52:03.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.717+0000 7f6701ffb640 1 -- 172.21.15.152:0/982426150 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66f0003e30 con 0x7f6704102570 2024-09-11T22:52:03.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.717+0000 7f6701ffb640 1 -- 172.21.15.152:0/982426150 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0035030 con 0x7f6704102570 2024-09-11T22:52:03.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.717+0000 7f670ab25640 1 -- 172.21.15.152:0/982426150 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67041a4fc0 con 0x7f6704102570 2024-09-11T22:52:03.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.718+0000 7f670ab25640 1 -- 172.21.15.152:0/982426150 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67041021b0 con 0x7f6704102570 2024-09-11T22:52:03.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.719+0000 7f6701ffb640 1 -- 172.21.15.152:0/982426150 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f66f0042070 con 0x7f6704102570 2024-09-11T22:52:03.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.720+0000 7f6701ffb640 1 --2- 172.21.15.152:0/982426150 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f66f40a2c10 0x7f66f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:03.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.720+0000 7f67088c1640 1 --2- 172.21.15.152:0/982426150 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f66f40a2c10 0x7f66f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:03.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.720+0000 7f6701ffb640 1 -- 172.21.15.152:0/982426150 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f66f00efb30 con 0x7f6704102570 2024-09-11T22:52:03.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.720+0000 7f67088c1640 1 --2- 172.21.15.152:0/982426150 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f66f40a2c10 0x7f66f40a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f66f800abe0 tx=0x7f66f800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:03.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.724+0000 7f6701ffb640 1 -- 172.21.15.152:0/982426150 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f66f00bc960 con 0x7f6704102570 2024-09-11T22:52:03.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.887+0000 7f670ab25640 1 -- 172.21.15.152:0/982426150 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6704062630 con 0x7f66f40a2c10 2024-09-11T22:52:03.889 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:03 smithi152 ceph-mon[31181]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:52:03.889 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:03 smithi152 ceph-mon[31181]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:03.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.893+0000 7f6701ffb640 1 -- 172.21.15.152:0/982426150 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258911 (secure 0 0 0) 0x7f6704062630 con 0x7f66f40a2c10 2024-09-11T22:52:03.898 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:03.899 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-09-11T22:52:03.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.896+0000 7f670ab25640 1 -- 172.21.15.152:0/982426150 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f66f40a2c10 msgr2=0x7f66f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:03.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.896+0000 7f670ab25640 1 --2- 172.21.15.152:0/982426150 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f66f40a2c10 0x7f66f40a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f66f800abe0 tx=0x7f66f800a2d0 comp rx=0 tx=0).stop 2024-09-11T22:52:03.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.896+0000 7f670ab25640 1 -- 172.21.15.152:0/982426150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6704102570 msgr2=0x7f6704198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:03.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.896+0000 7f670ab25640 1 --2- 172.21.15.152:0/982426150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6704102570 0x7f6704198980 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f66f0006010 tx=0x7f66f0003c10 comp rx=0 tx=0).stop 2024-09-11T22:52:03.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.896+0000 7f670ab25640 1 -- 172.21.15.152:0/982426150 shutdown_connections 2024-09-11T22:52:03.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.896+0000 7f670ab25640 1 --2- 172.21.15.152:0/982426150 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f66f40a2c10 0x7f66f40a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:03.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.896+0000 7f670ab25640 1 --2- 172.21.15.152:0/982426150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6704102ef0 0x7f670419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:03.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.896+0000 7f670ab25640 1 --2- 172.21.15.152:0/982426150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6704102570 0x7f6704198980 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:03.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.896+0000 7f670ab25640 1 --2- 172.21.15.152:0/982426150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6704100e20 0x7f6704198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:03.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.896+0000 7f670ab25640 1 -- 172.21.15.152:0/982426150 >> 172.21.15.152:0/982426150 conn(0x7f67040fc7d0 msgr2=0x7f670406a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:03.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.896+0000 7f670ab25640 1 -- 172.21.15.152:0/982426150 shutdown_connections 2024-09-11T22:52:03.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:03.896+0000 7f670ab25640 1 -- 172.21.15.152:0/982426150 wait complete. 2024-09-11T22:52:04.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:03 smithi191 ceph-mon[35211]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:52:04.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:03 smithi191 ceph-mon[35211]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:04.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:03 smithi204 ceph-mon[35454]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:52:04.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:03 smithi204 ceph-mon[35454]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:04.674 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-11T22:52:02.780476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659638,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":736,"num_read_kb":495,"num_write":429,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368836,"kb_used_data":11184,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572988,"statfs":{"total":1151940427776,"available":1151562739712,"internally_reserved":0,"allocated":11452416,"data_stored":7214214,"data_compressed":107505,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001935"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958996+0000","last_change":"2024-09-11T22:50:54.171410+0000","last_active":"2024-09-11T22:51:55.958996+0000","last_peered":"2024-09-11T22:51:55.958996+0000","last_clean":"2024-09-11T22:51:55.958996+0000","last_became_active":"2024-09-11T22:50:54.171055+0000","last_became_peered":"2024-09-11T22:50:54.171055+0000","last_unstale":"2024-09-11T22:51:55.958996+0000","last_undegraded":"2024-09-11T22:51:55.958996+0000","last_fullsized":"2024-09-11T22:51:55.958996+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:54.767454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956559+0000","last_change":"2024-09-11T22:50:54.168534+0000","last_active":"2024-09-11T22:51:55.956559+0000","last_peered":"2024-09-11T22:51:55.956559+0000","last_clean":"2024-09-11T22:51:55.956559+0000","last_became_active":"2024-09-11T22:50:54.168228+0000","last_became_peered":"2024-09-11T22:50:54.168228+0000","last_unstale":"2024-09-11T22:51:55.956559+0000","last_undegraded":"2024-09-11T22:51:55.956559+0000","last_fullsized":"2024-09-11T22:51:55.956559+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:11.380048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956320+0000","last_change":"2024-09-11T22:50:54.168975+0000","last_active":"2024-09-11T22:51:55.956320+0000","last_peered":"2024-09-11T22:51:55.956320+0000","last_clean":"2024-09-11T22:51:55.956320+0000","last_became_active":"2024-09-11T22:50:54.168676+0000","last_became_peered":"2024-09-11T22:50:54.168676+0000","last_unstale":"2024-09-11T22:51:55.956320+0000","last_undegraded":"2024-09-11T22:51:55.956320+0000","last_fullsized":"2024-09-11T22:51:55.956320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:15.848273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.954812+0000","last_change":"2024-09-11T22:50:54.164164+0000","last_active":"2024-09-11T22:51:55.954812+0000","last_peered":"2024-09-11T22:51:55.954812+0000","last_clean":"2024-09-11T22:51:55.954812+0000","last_became_active":"2024-09-11T22:50:54.163805+0000","last_became_peered":"2024-09-11T22:50:54.163805+0000","last_unstale":"2024-09-11T22:51:55.954812+0000","last_undegraded":"2024-09-11T22:51:55.954812+0000","last_fullsized":"2024-09-11T22:51:55.954812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:54.954046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957528+0000","last_change":"2024-09-11T22:50:54.167289+0000","last_active":"2024-09-11T22:51:55.957528+0000","last_peered":"2024-09-11T22:51:55.957528+0000","last_clean":"2024-09-11T22:51:55.957528+0000","last_became_active":"2024-09-11T22:50:54.166958+0000","last_became_peered":"2024-09-11T22:50:54.166958+0000","last_unstale":"2024-09-11T22:51:55.957528+0000","last_undegraded":"2024-09-11T22:51:55.957528+0000","last_fullsized":"2024-09-11T22:51:55.957528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:25.261241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.147801+0000","last_change":"2024-09-11T22:50:54.175605+0000","last_active":"2024-09-11T22:51:56.147801+0000","last_peered":"2024-09-11T22:51:56.147801+0000","last_clean":"2024-09-11T22:51:56.147801+0000","last_became_active":"2024-09-11T22:50:54.175207+0000","last_became_peered":"2024-09-11T22:50:54.175207+0000","last_unstale":"2024-09-11T22:51:56.147801+0000","last_undegraded":"2024-09-11T22:51:56.147801+0000","last_fullsized":"2024-09-11T22:51:56.147801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:47.008910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.204888+0000","last_change":"2024-09-11T22:50:54.167930+0000","last_active":"2024-09-11T22:51:56.204888+0000","last_peered":"2024-09-11T22:51:56.204888+0000","last_clean":"2024-09-11T22:51:56.204888+0000","last_became_active":"2024-09-11T22:50:54.167082+0000","last_became_peered":"2024-09-11T22:50:54.167082+0000","last_unstale":"2024-09-11T22:51:56.204888+0000","last_undegraded":"2024-09-11T22:51:56.204888+0000","last_fullsized":"2024-09-11T22:51:56.204888+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:50.931311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956868+0000","last_change":"2024-09-11T22:50:54.172528+0000","last_active":"2024-09-11T22:51:55.956868+0000","last_peered":"2024-09-11T22:51:55.956868+0000","last_clean":"2024-09-11T22:51:55.956868+0000","last_became_active":"2024-09-11T22:50:54.172257+0000","last_became_peered":"2024-09-11T22:50:54.172257+0000","last_unstale":"2024-09-11T22:51:55.956868+0000","last_undegraded":"2024-09-11T22:51:55.956868+0000","last_fullsized":"2024-09-11T22:51:55.956868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:13:43.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956803+0000","last_change":"2024-09-11T22:50:54.163357+0000","last_active":"2024-09-11T22:51:55.956803+0000","last_peered":"2024-09-11T22:51:55.956803+0000","last_clean":"2024-09-11T22:51:55.956803+0000","last_became_active":"2024-09-11T22:50:54.162943+0000","last_became_peered":"2024-09-11T22:50:54.162943+0000","last_unstale":"2024-09-11T22:51:55.956803+0000","last_undegraded":"2024-09-11T22:51:55.956803+0000","last_fullsized":"2024-09-11T22:51:55.956803+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:57.055802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.148197+0000","last_change":"2024-09-11T22:50:54.177773+0000","last_active":"2024-09-11T22:51:56.148197+0000","last_peered":"2024-09-11T22:51:56.148197+0000","last_clean":"2024-09-11T22:51:56.148197+0000","last_became_active":"2024-09-11T22:50:54.177388+0000","last_became_peered":"2024-09-11T22:50:54.177388+0000","last_unstale":"2024-09-11T22:51:56.148197+0000","last_undegraded":"2024-09-11T22:51:56.148197+0000","last_fullsized":"2024-09-11T22:51:56.148197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:36:44.625089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.205239+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:51:56.205239+0000","last_peered":"2024-09-11T22:51:56.205239+0000","last_clean":"2024-09-11T22:51:56.205239+0000","last_became_active":"2024-09-11T22:50:54.171491+0000","last_became_peered":"2024-09-11T22:50:54.171491+0000","last_unstale":"2024-09-11T22:51:56.205239+0000","last_undegraded":"2024-09-11T22:51:56.205239+0000","last_fullsized":"2024-09-11T22:51:56.205239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.996640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.954936+0000","last_change":"2024-09-11T22:50:54.171260+0000","last_active":"2024-09-11T22:51:55.954936+0000","last_peered":"2024-09-11T22:51:55.954936+0000","last_clean":"2024-09-11T22:51:55.954936+0000","last_became_active":"2024-09-11T22:50:54.170733+0000","last_became_peered":"2024-09-11T22:50:54.170733+0000","last_unstale":"2024-09-11T22:51:55.954936+0000","last_undegraded":"2024-09-11T22:51:55.954936+0000","last_fullsized":"2024-09-11T22:51:55.954936+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:57.877280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236568+0000","last_change":"2024-09-11T22:50:54.170576+0000","last_active":"2024-09-11T22:51:56.236568+0000","last_peered":"2024-09-11T22:51:56.236568+0000","last_clean":"2024-09-11T22:51:56.236568+0000","last_became_active":"2024-09-11T22:50:54.170239+0000","last_became_peered":"2024-09-11T22:50:54.170239+0000","last_unstale":"2024-09-11T22:51:56.236568+0000","last_undegraded":"2024-09-11T22:51:56.236568+0000","last_fullsized":"2024-09-11T22:51:56.236568+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:09.683171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956983+0000","last_change":"2024-09-11T22:50:54.167176+0000","last_active":"2024-09-11T22:51:55.956983+0000","last_peered":"2024-09-11T22:51:55.956983+0000","last_clean":"2024-09-11T22:51:55.956983+0000","last_became_active":"2024-09-11T22:50:54.166894+0000","last_became_peered":"2024-09-11T22:50:54.166894+0000","last_unstale":"2024-09-11T22:51:55.956983+0000","last_undegraded":"2024-09-11T22:51:55.956983+0000","last_fullsized":"2024-09-11T22:51:55.956983+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:43:08.158786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956483+0000","last_change":"2024-09-11T22:50:54.163100+0000","last_active":"2024-09-11T22:51:55.956483+0000","last_peered":"2024-09-11T22:51:55.956483+0000","last_clean":"2024-09-11T22:51:55.956483+0000","last_became_active":"2024-09-11T22:50:54.162776+0000","last_became_peered":"2024-09-11T22:50:54.162776+0000","last_unstale":"2024-09-11T22:51:55.956483+0000","last_undegraded":"2024-09-11T22:51:55.956483+0000","last_fullsized":"2024-09-11T22:51:55.956483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:15.215305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956239+0000","last_change":"2024-09-11T22:50:54.166013+0000","last_active":"2024-09-11T22:51:55.956239+0000","last_peered":"2024-09-11T22:51:55.956239+0000","last_clean":"2024-09-11T22:51:55.956239+0000","last_became_active":"2024-09-11T22:50:54.165680+0000","last_became_peered":"2024-09-11T22:50:54.165680+0000","last_unstale":"2024-09-11T22:51:55.956239+0000","last_undegraded":"2024-09-11T22:51:55.956239+0000","last_fullsized":"2024-09-11T22:51:55.956239+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:12.192231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.959096+0000","last_change":"2024-09-11T22:50:54.172105+0000","last_active":"2024-09-11T22:51:55.959096+0000","last_peered":"2024-09-11T22:51:55.959096+0000","last_clean":"2024-09-11T22:51:55.959096+0000","last_became_active":"2024-09-11T22:50:54.171782+0000","last_became_peered":"2024-09-11T22:50:54.171782+0000","last_unstale":"2024-09-11T22:51:55.959096+0000","last_undegraded":"2024-09-11T22:51:55.959096+0000","last_fullsized":"2024-09-11T22:51:55.959096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:17.152834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.954193+0000","last_change":"2024-09-11T22:50:54.167184+0000","last_active":"2024-09-11T22:51:55.954193+0000","last_peered":"2024-09-11T22:51:55.954193+0000","last_clean":"2024-09-11T22:51:55.954193+0000","last_became_active":"2024-09-11T22:50:54.166878+0000","last_became_peered":"2024-09-11T22:50:54.166878+0000","last_unstale":"2024-09-11T22:51:55.954193+0000","last_undegraded":"2024-09-11T22:51:55.954193+0000","last_fullsized":"2024-09-11T22:51:55.954193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:44.046552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956241+0000","last_change":"2024-09-11T22:50:54.176260+0000","last_active":"2024-09-11T22:51:55.956241+0000","last_peered":"2024-09-11T22:51:55.956241+0000","last_clean":"2024-09-11T22:51:55.956241+0000","last_became_active":"2024-09-11T22:50:54.175873+0000","last_became_peered":"2024-09-11T22:50:54.175873+0000","last_unstale":"2024-09-11T22:51:55.956241+0000","last_undegraded":"2024-09-11T22:51:55.956241+0000","last_fullsized":"2024-09-11T22:51:55.956241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:05.258677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958831+0000","last_change":"2024-09-11T22:50:54.172725+0000","last_active":"2024-09-11T22:51:55.958831+0000","last_peered":"2024-09-11T22:51:55.958831+0000","last_clean":"2024-09-11T22:51:55.958831+0000","last_became_active":"2024-09-11T22:50:54.172072+0000","last_became_peered":"2024-09-11T22:50:54.172072+0000","last_unstale":"2024-09-11T22:51:55.958831+0000","last_undegraded":"2024-09-11T22:51:55.958831+0000","last_fullsized":"2024-09-11T22:51:55.958831+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:53.699670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236659+0000","last_change":"2024-09-11T22:50:54.165118+0000","last_active":"2024-09-11T22:51:56.236659+0000","last_peered":"2024-09-11T22:51:56.236659+0000","last_clean":"2024-09-11T22:51:56.236659+0000","last_became_active":"2024-09-11T22:50:54.164793+0000","last_became_peered":"2024-09-11T22:50:54.164793+0000","last_unstale":"2024-09-11T22:51:56.236659+0000","last_undegraded":"2024-09-11T22:51:56.236659+0000","last_fullsized":"2024-09-11T22:51:56.236659+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:42.883940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956745+0000","last_change":"2024-09-11T22:50:54.176900+0000","last_active":"2024-09-11T22:51:55.956745+0000","last_peered":"2024-09-11T22:51:55.956745+0000","last_clean":"2024-09-11T22:51:55.956745+0000","last_became_active":"2024-09-11T22:50:54.176595+0000","last_became_peered":"2024-09-11T22:50:54.176595+0000","last_unstale":"2024-09-11T22:51:55.956745+0000","last_undegraded":"2024-09-11T22:51:55.956745+0000","last_fullsized":"2024-09-11T22:51:55.956745+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:36.925945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956267+0000","last_change":"2024-09-11T22:50:54.167656+0000","last_active":"2024-09-11T22:51:55.956267+0000","last_peered":"2024-09-11T22:51:55.956267+0000","last_clean":"2024-09-11T22:51:55.956267+0000","last_became_active":"2024-09-11T22:50:54.167349+0000","last_became_peered":"2024-09-11T22:50:54.167349+0000","last_unstale":"2024-09-11T22:51:55.956267+0000","last_undegraded":"2024-09-11T22:51:55.956267+0000","last_fullsized":"2024-09-11T22:51:55.956267+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:38.100218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957960+0000","last_change":"2024-09-11T22:50:54.177059+0000","last_active":"2024-09-11T22:51:55.957960+0000","last_peered":"2024-09-11T22:51:55.957960+0000","last_clean":"2024-09-11T22:51:55.957960+0000","last_became_active":"2024-09-11T22:50:54.176726+0000","last_became_peered":"2024-09-11T22:50:54.176726+0000","last_unstale":"2024-09-11T22:51:55.957960+0000","last_undegraded":"2024-09-11T22:51:55.957960+0000","last_fullsized":"2024-09-11T22:51:55.957960+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:27:58.058944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957129+0000","last_change":"2024-09-11T22:50:54.167566+0000","last_active":"2024-09-11T22:51:55.957129+0000","last_peered":"2024-09-11T22:51:55.957129+0000","last_clean":"2024-09-11T22:51:55.957129+0000","last_became_active":"2024-09-11T22:50:54.167049+0000","last_became_peered":"2024-09-11T22:50:54.167049+0000","last_unstale":"2024-09-11T22:51:55.957129+0000","last_undegraded":"2024-09-11T22:51:55.957129+0000","last_fullsized":"2024-09-11T22:51:55.957129+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:06.833099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956518+0000","last_change":"2024-09-11T22:50:54.169732+0000","last_active":"2024-09-11T22:51:55.956518+0000","last_peered":"2024-09-11T22:51:55.956518+0000","last_clean":"2024-09-11T22:51:55.956518+0000","last_became_active":"2024-09-11T22:50:54.169437+0000","last_became_peered":"2024-09-11T22:50:54.169437+0000","last_unstale":"2024-09-11T22:51:55.956518+0000","last_undegraded":"2024-09-11T22:51:55.956518+0000","last_fullsized":"2024-09-11T22:51:55.956518+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:25.553387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957033+0000","last_change":"2024-09-11T22:50:54.166953+0000","last_active":"2024-09-11T22:51:55.957033+0000","last_peered":"2024-09-11T22:51:55.957033+0000","last_clean":"2024-09-11T22:51:55.957033+0000","last_became_active":"2024-09-11T22:50:54.166472+0000","last_became_peered":"2024-09-11T22:50:54.166472+0000","last_unstale":"2024-09-11T22:51:55.957033+0000","last_undegraded":"2024-09-11T22:51:55.957033+0000","last_fullsized":"2024-09-11T22:51:55.957033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:01:20.234180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236582+0000","last_change":"2024-09-11T22:50:54.171308+0000","last_active":"2024-09-11T22:51:56.236582+0000","last_peered":"2024-09-11T22:51:56.236582+0000","last_clean":"2024-09-11T22:51:56.236582+0000","last_became_active":"2024-09-11T22:50:54.168958+0000","last_became_peered":"2024-09-11T22:50:54.168958+0000","last_unstale":"2024-09-11T22:51:56.236582+0000","last_undegraded":"2024-09-11T22:51:56.236582+0000","last_fullsized":"2024-09-11T22:51:56.236582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:20.224003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956736+0000","last_change":"2024-09-11T22:50:54.173846+0000","last_active":"2024-09-11T22:51:55.956736+0000","last_peered":"2024-09-11T22:51:55.956736+0000","last_clean":"2024-09-11T22:51:55.956736+0000","last_became_active":"2024-09-11T22:50:54.173542+0000","last_became_peered":"2024-09-11T22:50:54.173542+0000","last_unstale":"2024-09-11T22:51:55.956736+0000","last_undegraded":"2024-09-11T22:51:55.956736+0000","last_fullsized":"2024-09-11T22:51:55.956736+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:29:45.775902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956423+0000","last_change":"2024-09-11T22:50:54.176008+0000","last_active":"2024-09-11T22:51:55.956423+0000","last_peered":"2024-09-11T22:51:55.956423+0000","last_clean":"2024-09-11T22:51:55.956423+0000","last_became_active":"2024-09-11T22:50:54.175672+0000","last_became_peered":"2024-09-11T22:50:54.175672+0000","last_unstale":"2024-09-11T22:51:55.956423+0000","last_undegraded":"2024-09-11T22:51:55.956423+0000","last_fullsized":"2024-09-11T22:51:55.956423+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:35.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.955997+0000","last_change":"2024-09-11T22:50:54.162504+0000","last_active":"2024-09-11T22:51:55.955997+0000","last_peered":"2024-09-11T22:51:55.955997+0000","last_clean":"2024-09-11T22:51:55.955997+0000","last_became_active":"2024-09-11T22:50:54.162190+0000","last_became_peered":"2024-09-11T22:50:54.162190+0000","last_unstale":"2024-09-11T22:51:55.955997+0000","last_undegraded":"2024-09-11T22:51:55.955997+0000","last_fullsized":"2024-09-11T22:51:55.955997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:52.138692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.960882+0000","last_change":"2024-09-11T22:51:55.960882+0000","last_active":"2024-09-11T22:51:55.960882+0000","last_peered":"2024-09-11T22:51:55.960882+0000","last_clean":"2024-09-11T22:51:55.960882+0000","last_became_active":"2024-09-11T22:51:55.960246+0000","last_became_peered":"2024-09-11T22:51:55.960246+0000","last_unstale":"2024-09-11T22:51:55.960882+0000","last_undegraded":"2024-09-11T22:51:55.960882+0000","last_fullsized":"2024-09-11T22:51:55.960882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:21.247840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957287+0000","last_change":"2024-09-11T22:50:54.160696+0000","last_active":"2024-09-11T22:51:55.957287+0000","last_peered":"2024-09-11T22:51:55.957287+0000","last_clean":"2024-09-11T22:51:55.957287+0000","last_became_active":"2024-09-11T22:50:54.159249+0000","last_became_peered":"2024-09-11T22:50:54.159249+0000","last_unstale":"2024-09-11T22:51:55.957287+0000","last_undegraded":"2024-09-11T22:51:55.957287+0000","last_fullsized":"2024-09-11T22:51:55.957287+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:09.649208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'6","reported_seq":89,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:52:01.210624+0000","last_change":"2024-09-11T22:50:52.138986+0000","last_active":"2024-09-11T22:52:01.210624+0000","last_peered":"2024-09-11T22:52:01.210624+0000","last_clean":"2024-09-11T22:52:01.210624+0000","last_became_active":"2024-09-11T22:50:52.138595+0000","last_became_peered":"2024-09-11T22:50:52.138595+0000","last_unstale":"2024-09-11T22:52:01.210624+0000","last_undegraded":"2024-09-11T22:52:01.210624+0000","last_fullsized":"2024-09-11T22:52:01.210624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:06.232146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957880+0000","last_change":"2024-09-11T22:50:54.176806+0000","last_active":"2024-09-11T22:51:55.957880+0000","last_peered":"2024-09-11T22:51:55.957880+0000","last_clean":"2024-09-11T22:51:55.957880+0000","last_became_active":"2024-09-11T22:50:54.176513+0000","last_became_peered":"2024-09-11T22:50:54.176513+0000","last_unstale":"2024-09-11T22:51:55.957880+0000","last_undegraded":"2024-09-11T22:51:55.957880+0000","last_fullsized":"2024-09-11T22:51:55.957880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:43:12.086145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.147520+0000","last_change":"2024-09-11T22:50:52.142794+0000","last_active":"2024-09-11T22:51:56.147520+0000","last_peered":"2024-09-11T22:51:56.147520+0000","last_clean":"2024-09-11T22:51:56.147520+0000","last_became_active":"2024-09-11T22:50:52.142433+0000","last_became_peered":"2024-09-11T22:50:52.142433+0000","last_unstale":"2024-09-11T22:51:56.147520+0000","last_undegraded":"2024-09-11T22:51:56.147520+0000","last_fullsized":"2024-09-11T22:51:56.147520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:47:12.090055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236342+0000","last_change":"2024-09-11T22:50:52.139789+0000","last_active":"2024-09-11T22:51:56.236342+0000","last_peered":"2024-09-11T22:51:56.236342+0000","last_clean":"2024-09-11T22:51:56.236342+0000","last_became_active":"2024-09-11T22:50:52.139294+0000","last_became_peered":"2024-09-11T22:50:52.139294+0000","last_unstale":"2024-09-11T22:51:56.236342+0000","last_undegraded":"2024-09-11T22:51:56.236342+0000","last_fullsized":"2024-09-11T22:51:56.236342+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:07.355155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956967+0000","last_change":"2024-09-11T22:50:54.166248+0000","last_active":"2024-09-11T22:51:55.956967+0000","last_peered":"2024-09-11T22:51:55.956967+0000","last_clean":"2024-09-11T22:51:55.956967+0000","last_became_active":"2024-09-11T22:50:54.165958+0000","last_became_peered":"2024-09-11T22:50:54.165958+0000","last_unstale":"2024-09-11T22:51:55.956967+0000","last_undegraded":"2024-09-11T22:51:55.956967+0000","last_fullsized":"2024-09-11T22:51:55.956967+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:29.531741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958597+0000","last_change":"2024-09-11T22:50:52.140340+0000","last_active":"2024-09-11T22:51:55.958597+0000","last_peered":"2024-09-11T22:51:55.958597+0000","last_clean":"2024-09-11T22:51:55.958597+0000","last_became_active":"2024-09-11T22:50:52.140011+0000","last_became_peered":"2024-09-11T22:50:52.140011+0000","last_unstale":"2024-09-11T22:51:55.958597+0000","last_undegraded":"2024-09-11T22:51:55.958597+0000","last_fullsized":"2024-09-11T22:51:55.958597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:58.862640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.953910+0000","last_change":"2024-09-11T22:50:54.166998+0000","last_active":"2024-09-11T22:51:55.953910+0000","last_peered":"2024-09-11T22:51:55.953910+0000","last_clean":"2024-09-11T22:51:55.953910+0000","last_became_active":"2024-09-11T22:50:54.166682+0000","last_became_peered":"2024-09-11T22:50:54.166682+0000","last_unstale":"2024-09-11T22:51:55.953910+0000","last_undegraded":"2024-09-11T22:51:55.953910+0000","last_fullsized":"2024-09-11T22:51:55.953910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:57.815914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:52:00.208673+0000","last_change":"2024-09-11T22:50:52.141380+0000","last_active":"2024-09-11T22:52:00.208673+0000","last_peered":"2024-09-11T22:52:00.208673+0000","last_clean":"2024-09-11T22:52:00.208673+0000","last_became_active":"2024-09-11T22:50:52.139925+0000","last_became_peered":"2024-09-11T22:50:52.139925+0000","last_unstale":"2024-09-11T22:52:00.208673+0000","last_undegraded":"2024-09-11T22:52:00.208673+0000","last_fullsized":"2024-09-11T22:52:00.208673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:40:38.395055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956655+0000","last_change":"2024-09-11T22:50:54.167909+0000","last_active":"2024-09-11T22:51:55.956655+0000","last_peered":"2024-09-11T22:51:55.956655+0000","last_clean":"2024-09-11T22:51:55.956655+0000","last_became_active":"2024-09-11T22:50:54.167571+0000","last_became_peered":"2024-09-11T22:50:54.167571+0000","last_unstale":"2024-09-11T22:51:55.956655+0000","last_undegraded":"2024-09-11T22:51:55.956655+0000","last_fullsized":"2024-09-11T22:51:55.956655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:34.179960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.204211+0000","last_change":"2024-09-11T22:50:52.144545+0000","last_active":"2024-09-11T22:51:56.204211+0000","last_peered":"2024-09-11T22:51:56.204211+0000","last_clean":"2024-09-11T22:51:56.204211+0000","last_became_active":"2024-09-11T22:50:52.144221+0000","last_became_peered":"2024-09-11T22:50:52.144221+0000","last_unstale":"2024-09-11T22:51:56.204211+0000","last_undegraded":"2024-09-11T22:51:56.204211+0000","last_fullsized":"2024-09-11T22:51:56.204211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:29.509165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957020+0000","last_change":"2024-09-11T22:50:54.164132+0000","last_active":"2024-09-11T22:51:55.957020+0000","last_peered":"2024-09-11T22:51:55.957020+0000","last_clean":"2024-09-11T22:51:55.957020+0000","last_became_active":"2024-09-11T22:50:54.163842+0000","last_became_peered":"2024-09-11T22:50:54.163842+0000","last_unstale":"2024-09-11T22:51:55.957020+0000","last_undegraded":"2024-09-11T22:51:55.957020+0000","last_fullsized":"2024-09-11T22:51:55.957020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:28:55.490236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958373+0000","last_change":"2024-09-11T22:50:52.264611+0000","last_active":"2024-09-11T22:51:55.958373+0000","last_peered":"2024-09-11T22:51:55.958373+0000","last_clean":"2024-09-11T22:51:55.958373+0000","last_became_active":"2024-09-11T22:50:52.264060+0000","last_became_peered":"2024-09-11T22:50:52.264060+0000","last_unstale":"2024-09-11T22:51:55.958373+0000","last_undegraded":"2024-09-11T22:51:55.958373+0000","last_fullsized":"2024-09-11T22:51:55.958373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:42.543997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.206155+0000","last_change":"2024-09-11T22:51:56.206155+0000","last_active":"2024-09-11T22:51:56.206155+0000","last_peered":"2024-09-11T22:51:56.206155+0000","last_clean":"2024-09-11T22:51:56.206155+0000","last_became_active":"2024-09-11T22:51:56.205797+0000","last_became_peered":"2024-09-11T22:51:56.205797+0000","last_unstale":"2024-09-11T22:51:56.206155+0000","last_undegraded":"2024-09-11T22:51:56.206155+0000","last_fullsized":"2024-09-11T22:51:56.206155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:28.063825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'31","reported_seq":174,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:52:00.208673+0000","last_change":"2024-09-11T22:50:52.141782+0000","last_active":"2024-09-11T22:52:00.208673+0000","last_peered":"2024-09-11T22:52:00.208673+0000","last_clean":"2024-09-11T22:52:00.208673+0000","last_became_active":"2024-09-11T22:50:52.140662+0000","last_became_peered":"2024-09-11T22:50:52.140662+0000","last_unstale":"2024-09-11T22:52:00.208673+0000","last_undegraded":"2024-09-11T22:52:00.208673+0000","last_fullsized":"2024-09-11T22:52:00.208673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:21.158577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61828,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":119,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957900+0000","last_change":"2024-09-11T22:50:54.166678+0000","last_active":"2024-09-11T22:51:55.957900+0000","last_peered":"2024-09-11T22:51:55.957900+0000","last_clean":"2024-09-11T22:51:55.957900+0000","last_became_active":"2024-09-11T22:50:54.165777+0000","last_became_peered":"2024-09-11T22:50:54.165777+0000","last_unstale":"2024-09-11T22:51:55.957900+0000","last_undegraded":"2024-09-11T22:51:55.957900+0000","last_fullsized":"2024-09-11T22:51:55.957900+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:07.224563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956250+0000","last_change":"2024-09-11T22:50:52.141292+0000","last_active":"2024-09-11T22:51:55.956250+0000","last_peered":"2024-09-11T22:51:55.956250+0000","last_clean":"2024-09-11T22:51:55.956250+0000","last_became_active":"2024-09-11T22:50:52.140971+0000","last_became_peered":"2024-09-11T22:50:52.140971+0000","last_unstale":"2024-09-11T22:51:55.956250+0000","last_undegraded":"2024-09-11T22:51:55.956250+0000","last_fullsized":"2024-09-11T22:51:55.956250+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:53.039684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236436+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:51:56.236436+0000","last_peered":"2024-09-11T22:51:56.236436+0000","last_clean":"2024-09-11T22:51:56.236436+0000","last_became_active":"2024-09-11T22:50:54.171520+0000","last_became_peered":"2024-09-11T22:50:54.171520+0000","last_unstale":"2024-09-11T22:51:56.236436+0000","last_undegraded":"2024-09-11T22:51:56.236436+0000","last_fullsized":"2024-09-11T22:51:56.236436+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:56.594058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957905+0000","last_change":"2024-09-11T22:50:52.138725+0000","last_active":"2024-09-11T22:51:55.957905+0000","last_peered":"2024-09-11T22:51:55.957905+0000","last_clean":"2024-09-11T22:51:55.957905+0000","last_became_active":"2024-09-11T22:50:52.138367+0000","last_became_peered":"2024-09-11T22:50:52.138367+0000","last_unstale":"2024-09-11T22:51:55.957905+0000","last_undegraded":"2024-09-11T22:51:55.957905+0000","last_fullsized":"2024-09-11T22:51:55.957905+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:58.811444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.205138+0000","last_change":"2024-09-11T22:50:54.170524+0000","last_active":"2024-09-11T22:51:56.205138+0000","last_peered":"2024-09-11T22:51:56.205138+0000","last_clean":"2024-09-11T22:51:56.205138+0000","last_became_active":"2024-09-11T22:50:54.169947+0000","last_became_peered":"2024-09-11T22:50:54.169947+0000","last_unstale":"2024-09-11T22:51:56.205138+0000","last_undegraded":"2024-09-11T22:51:56.205138+0000","last_fullsized":"2024-09-11T22:51:56.205138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:51.768718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'5","reported_seq":86,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:52:00.244057+0000","last_change":"2024-09-11T22:50:52.139182+0000","last_active":"2024-09-11T22:52:00.244057+0000","last_peered":"2024-09-11T22:52:00.244057+0000","last_clean":"2024-09-11T22:52:00.244057+0000","last_became_active":"2024-09-11T22:50:52.138799+0000","last_became_peered":"2024-09-11T22:50:52.138799+0000","last_unstale":"2024-09-11T22:52:00.244057+0000","last_undegraded":"2024-09-11T22:52:00.244057+0000","last_fullsized":"2024-09-11T22:52:00.244057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:36.376162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":60,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.204819+0000","last_change":"2024-09-11T22:50:54.168797+0000","last_active":"2024-09-11T22:51:56.204819+0000","last_peered":"2024-09-11T22:51:56.204819+0000","last_clean":"2024-09-11T22:51:56.204819+0000","last_became_active":"2024-09-11T22:50:54.168408+0000","last_became_peered":"2024-09-11T22:50:54.168408+0000","last_unstale":"2024-09-11T22:51:56.204819+0000","last_undegraded":"2024-09-11T22:51:56.204819+0000","last_fullsized":"2024-09-11T22:51:56.204819+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:51.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956437+0000","last_change":"2024-09-11T22:50:52.264693+0000","last_active":"2024-09-11T22:51:55.956437+0000","last_peered":"2024-09-11T22:51:55.956437+0000","last_clean":"2024-09-11T22:51:55.956437+0000","last_became_active":"2024-09-11T22:50:52.264058+0000","last_became_peered":"2024-09-11T22:50:52.264058+0000","last_unstale":"2024-09-11T22:51:55.956437+0000","last_undegraded":"2024-09-11T22:51:55.956437+0000","last_fullsized":"2024-09-11T22:51:55.956437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:46.985921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.961118+0000","last_change":"2024-09-11T22:51:55.961118+0000","last_active":"2024-09-11T22:51:55.961118+0000","last_peered":"2024-09-11T22:51:55.961118+0000","last_clean":"2024-09-11T22:51:55.961118+0000","last_became_active":"2024-09-11T22:51:55.960614+0000","last_became_peered":"2024-09-11T22:51:55.960614+0000","last_unstale":"2024-09-11T22:51:55.961118+0000","last_undegraded":"2024-09-11T22:51:55.961118+0000","last_fullsized":"2024-09-11T22:51:55.961118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:47.184327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958401+0000","last_change":"2024-09-11T22:50:52.133049+0000","last_active":"2024-09-11T22:51:55.958401+0000","last_peered":"2024-09-11T22:51:55.958401+0000","last_clean":"2024-09-11T22:51:55.958401+0000","last_became_active":"2024-09-11T22:50:52.132647+0000","last_became_peered":"2024-09-11T22:50:52.132647+0000","last_unstale":"2024-09-11T22:51:55.958401+0000","last_undegraded":"2024-09-11T22:51:55.958401+0000","last_fullsized":"2024-09-11T22:51:55.958401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:13.213003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.238023+0000","last_change":"2024-09-11T22:51:56.238023+0000","last_active":"2024-09-11T22:51:56.238023+0000","last_peered":"2024-09-11T22:51:56.238023+0000","last_clean":"2024-09-11T22:51:56.238023+0000","last_became_active":"2024-09-11T22:51:56.237676+0000","last_became_peered":"2024-09-11T22:51:56.237676+0000","last_unstale":"2024-09-11T22:51:56.238023+0000","last_undegraded":"2024-09-11T22:51:56.238023+0000","last_fullsized":"2024-09-11T22:51:56.238023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:16.215961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236361+0000","last_change":"2024-09-11T22:50:52.139766+0000","last_active":"2024-09-11T22:51:56.236361+0000","last_peered":"2024-09-11T22:51:56.236361+0000","last_clean":"2024-09-11T22:51:56.236361+0000","last_became_active":"2024-09-11T22:50:52.139456+0000","last_became_peered":"2024-09-11T22:50:52.139456+0000","last_unstale":"2024-09-11T22:51:56.236361+0000","last_undegraded":"2024-09-11T22:51:56.236361+0000","last_fullsized":"2024-09-11T22:51:56.236361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:08.358649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.205373+0000","last_change":"2024-09-11T22:50:54.167717+0000","last_active":"2024-09-11T22:51:56.205373+0000","last_peered":"2024-09-11T22:51:56.205373+0000","last_clean":"2024-09-11T22:51:56.205373+0000","last_became_active":"2024-09-11T22:50:54.167331+0000","last_became_peered":"2024-09-11T22:50:54.167331+0000","last_unstale":"2024-09-11T22:51:56.205373+0000","last_undegraded":"2024-09-11T22:51:56.205373+0000","last_fullsized":"2024-09-11T22:51:56.205373+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:11.240351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958495+0000","last_change":"2024-09-11T22:50:52.141682+0000","last_active":"2024-09-11T22:51:55.958495+0000","last_peered":"2024-09-11T22:51:55.958495+0000","last_clean":"2024-09-11T22:51:55.958495+0000","last_became_active":"2024-09-11T22:50:52.141251+0000","last_became_peered":"2024-09-11T22:50:52.141251+0000","last_unstale":"2024-09-11T22:51:55.958495+0000","last_undegraded":"2024-09-11T22:51:55.958495+0000","last_fullsized":"2024-09-11T22:51:55.958495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:03.068252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.205207+0000","last_change":"2024-09-11T22:50:54.174149+0000","last_active":"2024-09-11T22:51:56.205207+0000","last_peered":"2024-09-11T22:51:56.205207+0000","last_clean":"2024-09-11T22:51:56.205207+0000","last_became_active":"2024-09-11T22:50:54.173765+0000","last_became_peered":"2024-09-11T22:50:54.173765+0000","last_unstale":"2024-09-11T22:51:56.205207+0000","last_undegraded":"2024-09-11T22:51:56.205207+0000","last_fullsized":"2024-09-11T22:51:56.205207+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.314334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":86,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:58.206840+0000","last_change":"2024-09-11T22:50:52.435019+0000","last_active":"2024-09-11T22:51:58.206840+0000","last_peered":"2024-09-11T22:51:58.206840+0000","last_clean":"2024-09-11T22:51:58.206840+0000","last_became_active":"2024-09-11T22:50:52.434642+0000","last_became_peered":"2024-09-11T22:50:52.434642+0000","last_unstale":"2024-09-11T22:51:58.206840+0000","last_undegraded":"2024-09-11T22:51:58.206840+0000","last_fullsized":"2024-09-11T22:51:58.206840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:18.441841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.205327+0000","last_change":"2024-09-11T22:50:54.171291+0000","last_active":"2024-09-11T22:51:56.205327+0000","last_peered":"2024-09-11T22:51:56.205327+0000","last_clean":"2024-09-11T22:51:56.205327+0000","last_became_active":"2024-09-11T22:50:54.170961+0000","last_became_peered":"2024-09-11T22:50:54.170961+0000","last_unstale":"2024-09-11T22:51:56.205327+0000","last_undegraded":"2024-09-11T22:51:56.205327+0000","last_fullsized":"2024-09-11T22:51:56.205327+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:07.211936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":345,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.204956+0000","last_change":"2024-09-11T22:49:39.098881+0000","last_active":"2024-09-11T22:51:56.204956+0000","last_peered":"2024-09-11T22:51:56.204956+0000","last_clean":"2024-09-11T22:51:56.204956+0000","last_became_active":"2024-09-11T22:49:39.098458+0000","last_became_peered":"2024-09-11T22:49:39.098458+0000","last_unstale":"2024-09-11T22:51:56.204956+0000","last_undegraded":"2024-09-11T22:51:56.204956+0000","last_fullsized":"2024-09-11T22:51:56.204956+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_clean_scrub_stamp":"2024-09-11T22:46:11.418577+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:41.059384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.954679+0000","last_change":"2024-09-11T22:50:52.137691+0000","last_active":"2024-09-11T22:51:55.954679+0000","last_peered":"2024-09-11T22:51:55.954679+0000","last_clean":"2024-09-11T22:51:55.954679+0000","last_became_active":"2024-09-11T22:50:52.137282+0000","last_became_peered":"2024-09-11T22:50:52.137282+0000","last_unstale":"2024-09-11T22:51:55.954679+0000","last_undegraded":"2024-09-11T22:51:55.954679+0000","last_fullsized":"2024-09-11T22:51:55.954679+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:00:27.651252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.959937+0000","last_change":"2024-09-11T22:51:55.959937+0000","last_active":"2024-09-11T22:51:55.959937+0000","last_peered":"2024-09-11T22:51:55.959937+0000","last_clean":"2024-09-11T22:51:55.959937+0000","last_became_active":"2024-09-11T22:51:55.959575+0000","last_became_peered":"2024-09-11T22:51:55.959575+0000","last_unstale":"2024-09-11T22:51:55.959937+0000","last_undegraded":"2024-09-11T22:51:55.959937+0000","last_fullsized":"2024-09-11T22:51:55.959937+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:54.097552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.204129+0000","last_change":"2024-09-11T22:50:52.144324+0000","last_active":"2024-09-11T22:51:56.204129+0000","last_peered":"2024-09-11T22:51:56.204129+0000","last_clean":"2024-09-11T22:51:56.204129+0000","last_became_active":"2024-09-11T22:50:52.144011+0000","last_became_peered":"2024-09-11T22:50:52.144011+0000","last_unstale":"2024-09-11T22:51:56.204129+0000","last_undegraded":"2024-09-11T22:51:56.204129+0000","last_fullsized":"2024-09-11T22:51:56.204129+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:29.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236482+0000","last_change":"2024-09-11T22:50:54.173287+0000","last_active":"2024-09-11T22:51:56.236482+0000","last_peered":"2024-09-11T22:51:56.236482+0000","last_clean":"2024-09-11T22:51:56.236482+0000","last_became_active":"2024-09-11T22:50:54.172953+0000","last_became_peered":"2024-09-11T22:50:54.172953+0000","last_unstale":"2024-09-11T22:51:56.236482+0000","last_undegraded":"2024-09-11T22:51:56.236482+0000","last_fullsized":"2024-09-11T22:51:56.236482+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:24.314604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":86,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:52:01.244949+0000","last_change":"2024-09-11T22:50:52.263777+0000","last_active":"2024-09-11T22:52:01.244949+0000","last_peered":"2024-09-11T22:52:01.244949+0000","last_clean":"2024-09-11T22:52:01.244949+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:52:01.244949+0000","last_undegraded":"2024-09-11T22:52:01.244949+0000","last_fullsized":"2024-09-11T22:52:01.244949+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:37.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":59,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958445+0000","last_change":"2024-09-11T22:50:54.164927+0000","last_active":"2024-09-11T22:51:55.958445+0000","last_peered":"2024-09-11T22:51:55.958445+0000","last_clean":"2024-09-11T22:51:55.958445+0000","last_became_active":"2024-09-11T22:50:54.164602+0000","last_became_peered":"2024-09-11T22:50:54.164602+0000","last_unstale":"2024-09-11T22:51:55.958445+0000","last_undegraded":"2024-09-11T22:51:55.958445+0000","last_fullsized":"2024-09-11T22:51:55.958445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:10.226232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":80,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.204561+0000","last_change":"2024-09-11T22:50:52.141199+0000","last_active":"2024-09-11T22:51:56.204561+0000","last_peered":"2024-09-11T22:51:56.204561+0000","last_clean":"2024-09-11T22:51:56.204561+0000","last_became_active":"2024-09-11T22:50:52.140830+0000","last_became_peered":"2024-09-11T22:50:52.140830+0000","last_unstale":"2024-09-11T22:51:56.204561+0000","last_undegraded":"2024-09-11T22:51:56.204561+0000","last_fullsized":"2024-09-11T22:51:56.204561+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.767803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.148122+0000","last_change":"2024-09-11T22:50:54.175054+0000","last_active":"2024-09-11T22:51:56.148122+0000","last_peered":"2024-09-11T22:51:56.148122+0000","last_clean":"2024-09-11T22:51:56.148122+0000","last_became_active":"2024-09-11T22:50:54.174715+0000","last_became_peered":"2024-09-11T22:50:54.174715+0000","last_unstale":"2024-09-11T22:51:56.148122+0000","last_undegraded":"2024-09-11T22:51:56.148122+0000","last_fullsized":"2024-09-11T22:51:56.148122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:06.694988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.961660+0000","last_change":"2024-09-11T22:51:55.961660+0000","last_active":"2024-09-11T22:51:55.961660+0000","last_peered":"2024-09-11T22:51:55.961660+0000","last_clean":"2024-09-11T22:51:55.961660+0000","last_became_active":"2024-09-11T22:51:55.961278+0000","last_became_peered":"2024-09-11T22:51:55.961278+0000","last_unstale":"2024-09-11T22:51:55.961660+0000","last_undegraded":"2024-09-11T22:51:55.961660+0000","last_fullsized":"2024-09-11T22:51:55.961660+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:52.352490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.148085+0000","last_change":"2024-09-11T22:50:54.167361+0000","last_active":"2024-09-11T22:51:56.148085+0000","last_peered":"2024-09-11T22:51:56.148085+0000","last_clean":"2024-09-11T22:51:56.148085+0000","last_became_active":"2024-09-11T22:50:54.166963+0000","last_became_peered":"2024-09-11T22:50:54.166963+0000","last_unstale":"2024-09-11T22:51:56.148085+0000","last_undegraded":"2024-09-11T22:51:56.148085+0000","last_fullsized":"2024-09-11T22:51:56.148085+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:13.294544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958100+0000","last_change":"2024-09-11T22:50:52.264484+0000","last_active":"2024-09-11T22:51:55.958100+0000","last_peered":"2024-09-11T22:51:55.958100+0000","last_clean":"2024-09-11T22:51:55.958100+0000","last_became_active":"2024-09-11T22:50:52.264038+0000","last_became_peered":"2024-09-11T22:50:52.264038+0000","last_unstale":"2024-09-11T22:51:55.958100+0000","last_undegraded":"2024-09-11T22:51:55.958100+0000","last_fullsized":"2024-09-11T22:51:55.958100+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:21.325136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.236614+0000","last_change":"2024-09-11T22:50:54.162225+0000","last_active":"2024-09-11T22:51:56.236614+0000","last_peered":"2024-09-11T22:51:56.236614+0000","last_clean":"2024-09-11T22:51:56.236614+0000","last_became_active":"2024-09-11T22:50:54.161234+0000","last_became_peered":"2024-09-11T22:50:54.161234+0000","last_unstale":"2024-09-11T22:51:56.236614+0000","last_undegraded":"2024-09-11T22:51:56.236614+0000","last_fullsized":"2024-09-11T22:51:56.236614+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:00.346217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'16","reported_seq":54,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.967804+0000","last_change":"2024-09-11T22:51:55.967804+0000","last_active":"2024-09-11T22:51:55.967804+0000","last_peered":"2024-09-11T22:51:55.967804+0000","last_clean":"2024-09-11T22:51:55.967804+0000","last_became_active":"2024-09-11T22:51:55.961136+0000","last_became_peered":"2024-09-11T22:51:55.961136+0000","last_unstale":"2024-09-11T22:51:55.967804+0000","last_undegraded":"2024-09-11T22:51:55.967804+0000","last_fullsized":"2024-09-11T22:51:55.967804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:55.315179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957222+0000","last_change":"2024-09-11T22:50:54.167307+0000","last_active":"2024-09-11T22:51:55.957222+0000","last_peered":"2024-09-11T22:51:55.957222+0000","last_clean":"2024-09-11T22:51:55.957222+0000","last_became_active":"2024-09-11T22:50:54.166650+0000","last_became_peered":"2024-09-11T22:50:54.166650+0000","last_unstale":"2024-09-11T22:51:55.957222+0000","last_undegraded":"2024-09-11T22:51:55.957222+0000","last_fullsized":"2024-09-11T22:51:55.957222+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:04.863258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.955051+0000","last_change":"2024-09-11T22:50:52.435107+0000","last_active":"2024-09-11T22:51:55.955051+0000","last_peered":"2024-09-11T22:51:55.955051+0000","last_clean":"2024-09-11T22:51:55.955051+0000","last_became_active":"2024-09-11T22:50:52.434791+0000","last_became_peered":"2024-09-11T22:50:52.434791+0000","last_unstale":"2024-09-11T22:51:55.955051+0000","last_undegraded":"2024-09-11T22:51:55.955051+0000","last_fullsized":"2024-09-11T22:51:55.955051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:11.128791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956652+0000","last_change":"2024-09-11T22:50:54.178041+0000","last_active":"2024-09-11T22:51:55.956652+0000","last_peered":"2024-09-11T22:51:55.956652+0000","last_clean":"2024-09-11T22:51:55.956652+0000","last_became_active":"2024-09-11T22:50:54.177393+0000","last_became_peered":"2024-09-11T22:50:54.177393+0000","last_unstale":"2024-09-11T22:51:55.956652+0000","last_undegraded":"2024-09-11T22:51:55.956652+0000","last_fullsized":"2024-09-11T22:51:55.956652+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:21.608104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956338+0000","last_change":"2024-09-11T22:50:52.264051+0000","last_active":"2024-09-11T22:51:55.956338+0000","last_peered":"2024-09-11T22:51:55.956338+0000","last_clean":"2024-09-11T22:51:55.956338+0000","last_became_active":"2024-09-11T22:50:52.263674+0000","last_became_peered":"2024-09-11T22:50:52.263674+0000","last_unstale":"2024-09-11T22:51:55.956338+0000","last_undegraded":"2024-09-11T22:51:55.956338+0000","last_fullsized":"2024-09-11T22:51:55.956338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:44.655218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957107+0000","last_change":"2024-09-11T22:50:54.166409+0000","last_active":"2024-09-11T22:51:55.957107+0000","last_peered":"2024-09-11T22:51:55.957107+0000","last_clean":"2024-09-11T22:51:55.957107+0000","last_became_active":"2024-09-11T22:50:54.166037+0000","last_became_peered":"2024-09-11T22:50:54.166037+0000","last_unstale":"2024-09-11T22:51:55.957107+0000","last_undegraded":"2024-09-11T22:51:55.957107+0000","last_fullsized":"2024-09-11T22:51:55.957107+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:22.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956458+0000","last_change":"2024-09-11T22:50:52.140218+0000","last_active":"2024-09-11T22:51:55.956458+0000","last_peered":"2024-09-11T22:51:55.956458+0000","last_clean":"2024-09-11T22:51:55.956458+0000","last_became_active":"2024-09-11T22:50:52.139744+0000","last_became_peered":"2024-09-11T22:50:52.139744+0000","last_unstale":"2024-09-11T22:51:55.956458+0000","last_undegraded":"2024-09-11T22:51:55.956458+0000","last_fullsized":"2024-09-11T22:51:55.956458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:50.178348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957560+0000","last_change":"2024-09-11T22:50:54.164059+0000","last_active":"2024-09-11T22:51:55.957560+0000","last_peered":"2024-09-11T22:51:55.957560+0000","last_clean":"2024-09-11T22:51:55.957560+0000","last_became_active":"2024-09-11T22:50:54.163724+0000","last_became_peered":"2024-09-11T22:50:54.163724+0000","last_unstale":"2024-09-11T22:51:55.957560+0000","last_undegraded":"2024-09-11T22:51:55.957560+0000","last_fullsized":"2024-09-11T22:51:55.957560+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:57.241660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956526+0000","last_change":"2024-09-11T22:50:52.140169+0000","last_active":"2024-09-11T22:51:55.956526+0000","last_peered":"2024-09-11T22:51:55.956526+0000","last_clean":"2024-09-11T22:51:55.956526+0000","last_became_active":"2024-09-11T22:50:52.139579+0000","last_became_peered":"2024-09-11T22:50:52.139579+0000","last_unstale":"2024-09-11T22:51:55.956526+0000","last_undegraded":"2024-09-11T22:51:55.956526+0000","last_fullsized":"2024-09-11T22:51:55.956526+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:43.523961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956621+0000","last_change":"2024-09-11T22:50:54.171539+0000","last_active":"2024-09-11T22:51:55.956621+0000","last_peered":"2024-09-11T22:51:55.956621+0000","last_clean":"2024-09-11T22:51:55.956621+0000","last_became_active":"2024-09-11T22:50:54.171013+0000","last_became_peered":"2024-09-11T22:50:54.171013+0000","last_unstale":"2024-09-11T22:51:55.956621+0000","last_undegraded":"2024-09-11T22:51:55.956621+0000","last_fullsized":"2024-09-11T22:51:55.956621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:03.564389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957633+0000","last_change":"2024-09-11T22:50:52.139544+0000","last_active":"2024-09-11T22:51:55.957633+0000","last_peered":"2024-09-11T22:51:55.957633+0000","last_clean":"2024-09-11T22:51:55.957633+0000","last_became_active":"2024-09-11T22:50:52.139201+0000","last_became_peered":"2024-09-11T22:50:52.139201+0000","last_unstale":"2024-09-11T22:51:55.957633+0000","last_undegraded":"2024-09-11T22:51:55.957633+0000","last_fullsized":"2024-09-11T22:51:55.957633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:37.343437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:56.205084+0000","last_change":"2024-09-11T22:50:54.170052+0000","last_active":"2024-09-11T22:51:56.205084+0000","last_peered":"2024-09-11T22:51:56.205084+0000","last_clean":"2024-09-11T22:51:56.205084+0000","last_became_active":"2024-09-11T22:50:54.169735+0000","last_became_peered":"2024-09-11T22:50:54.169735+0000","last_unstale":"2024-09-11T22:51:56.205084+0000","last_undegraded":"2024-09-11T22:51:56.205084+0000","last_fullsized":"2024-09-11T22:51:56.205084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:44.007747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.958279+0000","last_change":"2024-09-11T22:50:54.169959+0000","last_active":"2024-09-11T22:51:55.958279+0000","last_peered":"2024-09-11T22:51:55.958279+0000","last_clean":"2024-09-11T22:51:55.958279+0000","last_became_active":"2024-09-11T22:50:54.169649+0000","last_became_peered":"2024-09-11T22:50:54.169649+0000","last_unstale":"2024-09-11T22:51:55.958279+0000","last_undegraded":"2024-09-11T22:51:55.958279+0000","last_fullsized":"2024-09-11T22:51:55.958279+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:51.974258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957383+0000","last_change":"2024-09-11T22:50:52.140881+0000","last_active":"2024-09-11T22:51:55.957383+0000","last_peered":"2024-09-11T22:51:55.957383+0000","last_clean":"2024-09-11T22:51:55.957383+0000","last_became_active":"2024-09-11T22:50:52.140527+0000","last_became_peered":"2024-09-11T22:50:52.140527+0000","last_unstale":"2024-09-11T22:51:55.957383+0000","last_undegraded":"2024-09-11T22:51:55.957383+0000","last_fullsized":"2024-09-11T22:51:55.957383+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:14.325816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956384+0000","last_change":"2024-09-11T22:50:54.166571+0000","last_active":"2024-09-11T22:51:55.956384+0000","last_peered":"2024-09-11T22:51:55.956384+0000","last_clean":"2024-09-11T22:51:55.956384+0000","last_became_active":"2024-09-11T22:50:54.166247+0000","last_became_peered":"2024-09-11T22:50:54.166247+0000","last_unstale":"2024-09-11T22:51:55.956384+0000","last_undegraded":"2024-09-11T22:51:55.956384+0000","last_fullsized":"2024-09-11T22:51:55.956384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:42.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.957998+0000","last_change":"2024-09-11T22:50:52.140298+0000","last_active":"2024-09-11T22:51:55.957998+0000","last_peered":"2024-09-11T22:51:55.957998+0000","last_clean":"2024-09-11T22:51:55.957998+0000","last_became_active":"2024-09-11T22:50:52.139917+0000","last_became_peered":"2024-09-11T22:50:52.139917+0000","last_unstale":"2024-09-11T22:51:55.957998+0000","last_undegraded":"2024-09-11T22:51:55.957998+0000","last_fullsized":"2024-09-11T22:51:55.957998+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:23.599010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956938+0000","last_change":"2024-09-11T22:50:54.170849+0000","last_active":"2024-09-11T22:51:55.956938+0000","last_peered":"2024-09-11T22:51:55.956938+0000","last_clean":"2024-09-11T22:51:55.956938+0000","last_became_active":"2024-09-11T22:50:54.169547+0000","last_became_peered":"2024-09-11T22:50:54.169547+0000","last_unstale":"2024-09-11T22:51:55.956938+0000","last_undegraded":"2024-09-11T22:51:55.956938+0000","last_fullsized":"2024-09-11T22:51:55.956938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:25.251158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956616+0000","last_change":"2024-09-11T22:50:52.263778+0000","last_active":"2024-09-11T22:51:55.956616+0000","last_peered":"2024-09-11T22:51:55.956616+0000","last_clean":"2024-09-11T22:51:55.956616+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:51:55.956616+0000","last_undegraded":"2024-09-11T22:51:55.956616+0000","last_fullsized":"2024-09-11T22:51:55.956616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:03.439147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:55.956438+0000","last_change":"2024-09-11T22:50:54.171200+0000","last_active":"2024-09-11T22:51:55.956438+0000","last_peered":"2024-09-11T22:51:55.956438+0000","last_clean":"2024-09-11T22:51:55.956438+0000","last_became_active":"2024-09-11T22:50:54.169320+0000","last_became_peered":"2024-09-11T22:50:54.169320+0000","last_unstale":"2024-09-11T22:51:55.956438+0000","last_undegraded":"2024-09-11T22:51:55.956438+0000","last_fullsized":"2024-09-11T22:51:55.956438+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:14.864699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":91,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:51:59.207830+0000","last_change":"2024-09-11T22:50:52.139609+0000","last_active":"2024-09-11T22:51:59.207830+0000","last_peered":"2024-09-11T22:51:59.207830+0000","last_clean":"2024-09-11T22:51:59.207830+0000","last_became_active":"2024-09-11T22:50:52.139278+0000","last_became_peered":"2024-09-11T22:50:52.139278+0000","last_unstale":"2024-09-11T22:51:59.207830+0000","last_undegraded":"2024-09-11T22:51:59.207830+0000","last_fullsized":"2024-09-11T22:51:59.207830+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:09.538734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69036,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":600,"num_read_kb":376,"num_write":246,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":207108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281769,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547228,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":905216,"data_stored":437860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776160,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1060864,"data_stored":1090324,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939686,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":929792,"data_stored":437397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168618,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":925696,"data_stored":437775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397551,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":978944,"data_stored":1030295,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561078,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":437114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1029973,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":913408,"data_stored":436975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":929792,"data_stored":436939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":995328,"data_stored":500861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":974848,"data_stored":499855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":933888,"data_stored":438846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34198,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:52:04.678 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T22:52:04.678 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T22:52:04.678 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T22:52:04.679 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T22:52:04.929 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:05.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:04 smithi204 ceph-mon[35454]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-11T22:52:05.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:04 smithi204 ceph-mon[35454]: Cluster is now healthy 2024-09-11T22:52:05.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:04 smithi204 ceph-mon[35454]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:52:05.137 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:04 smithi191 ceph-mon[35211]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-11T22:52:05.137 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:04 smithi191 ceph-mon[35211]: Cluster is now healthy 2024-09-11T22:52:05.137 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:04 smithi191 ceph-mon[35211]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:52:05.137 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:04 smithi152 ceph-mon[31181]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-11T22:52:05.137 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:04 smithi152 ceph-mon[31181]: Cluster is now healthy 2024-09-11T22:52:05.137 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:04 smithi152 ceph-mon[31181]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:52:05.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.481+0000 7fa1a51c4640 1 -- 172.21.15.152:0/522097573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1a0101420 msgr2=0x7fa1a010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:05.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.481+0000 7fa1a51c4640 1 --2- 172.21.15.152:0/522097573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1a0101420 0x7fa1a010d950 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa198009cd0 tx=0x7fa19802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:52:05.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.481+0000 7fa1a51c4640 1 -- 172.21.15.152:0/522097573 shutdown_connections 2024-09-11T22:52:05.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.481+0000 7fa1a51c4640 1 --2- 172.21.15.152:0/522097573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1a0101420 0x7fa1a010d950 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:05.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.481+0000 7fa1a51c4640 1 --2- 172.21.15.152:0/522097573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a0100a80 0x7fa1a0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:05.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.481+0000 7fa1a51c4640 1 --2- 172.21.15.152:0/522097573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1a0106a80 0x7fa1a0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:05.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.481+0000 7fa1a51c4640 1 -- 172.21.15.152:0/522097573 >> 172.21.15.152:0/522097573 conn(0x7fa1a00fc7a0 msgr2=0x7fa1a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:05.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.481+0000 7fa1a51c4640 1 -- 172.21.15.152:0/522097573 shutdown_connections 2024-09-11T22:52:05.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.482+0000 7fa1a51c4640 1 -- 172.21.15.152:0/522097573 wait complete. 2024-09-11T22:52:05.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.482+0000 7fa1a51c4640 1 Processor -- start 2024-09-11T22:52:05.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.482+0000 7fa1a51c4640 1 -- start start 2024-09-11T22:52:05.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa1a51c4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1a0100a80 0x7fa1a019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:05.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa1a51c4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a0101420 0x7fa1a019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:05.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa1a51c4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1a0106a80 0x7fa1a019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:05.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa1a51c4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a0111fe0 con 0x7fa1a0106a80 2024-09-11T22:52:05.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa1a51c4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a0111e60 con 0x7fa1a0100a80 2024-09-11T22:52:05.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa1a51c4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a0112160 con 0x7fa1a0101420 2024-09-11T22:52:05.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa19f59e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1a0106a80 0x7fa1a019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:05.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa19f59e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1a0106a80 0x7fa1a019eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38686/0 (socket says 172.21.15.152:38686) 2024-09-11T22:52:05.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa19f59e640 1 -- 172.21.15.152:0/2235617877 learned_addr learned my addr 172.21.15.152:0/2235617877 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:05.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa19e59c640 1 --2- 172.21.15.152:0/2235617877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a0101420 0x7fa1a019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:05.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa19ed9d640 1 --2- 172.21.15.152:0/2235617877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1a0100a80 0x7fa1a019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:05.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa19f59e640 1 -- 172.21.15.152:0/2235617877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a0101420 msgr2=0x7fa1a019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:05.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa19f59e640 1 --2- 172.21.15.152:0/2235617877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a0101420 0x7fa1a019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:05.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa19f59e640 1 -- 172.21.15.152:0/2235617877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1a0100a80 msgr2=0x7fa1a019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:05.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa19f59e640 1 --2- 172.21.15.152:0/2235617877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1a0100a80 0x7fa1a019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:05.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa19f59e640 1 -- 172.21.15.152:0/2235617877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a019f590 con 0x7fa1a0106a80 2024-09-11T22:52:05.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa19f59e640 1 --2- 172.21.15.152:0/2235617877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1a0106a80 0x7fa1a019eeb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa198033040 tx=0x7fa198003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:05.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa197fff640 1 -- 172.21.15.152:0/2235617877 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198004040 con 0x7fa1a0106a80 2024-09-11T22:52:05.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa1a51c4640 1 -- 172.21.15.152:0/2235617877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a019f820 con 0x7fa1a0106a80 2024-09-11T22:52:05.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.483+0000 7fa197fff640 1 -- 172.21.15.152:0/2235617877 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1980046c0 con 0x7fa1a0106a80 2024-09-11T22:52:05.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.484+0000 7fa197fff640 1 -- 172.21.15.152:0/2235617877 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198030eb0 con 0x7fa1a0106a80 2024-09-11T22:52:05.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.484+0000 7fa1a51c4640 1 -- 172.21.15.152:0/2235617877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a01a7160 con 0x7fa1a0106a80 2024-09-11T22:52:05.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.484+0000 7fa1a51c4640 1 -- 172.21.15.152:0/2235617877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a0100ee0 con 0x7fa1a0106a80 2024-09-11T22:52:05.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.486+0000 7fa197fff640 1 -- 172.21.15.152:0/2235617877 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa1980041e0 con 0x7fa1a0106a80 2024-09-11T22:52:05.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.487+0000 7fa197fff640 1 --2- 172.21.15.152:0/2235617877 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa18c0a2c10 0x7fa18c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:05.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.487+0000 7fa19ed9d640 1 --2- 172.21.15.152:0/2235617877 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa18c0a2c10 0x7fa18c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:05.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.487+0000 7fa197fff640 1 -- 172.21.15.152:0/2235617877 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fa19802e030 con 0x7fa1a0106a80 2024-09-11T22:52:05.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.487+0000 7fa19ed9d640 1 --2- 172.21.15.152:0/2235617877 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa18c0a2c10 0x7fa18c0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa18800ab60 tx=0x7fa18800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:05.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.490+0000 7fa197fff640 1 -- 172.21.15.152:0/2235617877 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1980bd600 con 0x7fa1a0106a80 2024-09-11T22:52:05.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.693+0000 7fa1a51c4640 1 -- 172.21.15.152:0/2235617877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa1a01a73f0 con 0x7fa1a0106a80 2024-09-11T22:52:05.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.694+0000 7fa197fff640 1 -- 172.21.15.152:0/2235617877 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa1980bcf00 con 0x7fa1a0106a80 2024-09-11T22:52:05.698 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:05.698 INFO:teuthology.orchestra.run.smithi152.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T22:52:05.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.696+0000 7fa1a51c4640 1 -- 172.21.15.152:0/2235617877 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa18c0a2c10 msgr2=0x7fa18c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:05.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.696+0000 7fa1a51c4640 1 --2- 172.21.15.152:0/2235617877 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa18c0a2c10 0x7fa18c0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa18800ab60 tx=0x7fa18800a2d0 comp rx=0 tx=0).stop 2024-09-11T22:52:05.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.696+0000 7fa1a51c4640 1 -- 172.21.15.152:0/2235617877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1a0106a80 msgr2=0x7fa1a019eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:05.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.696+0000 7fa1a51c4640 1 --2- 172.21.15.152:0/2235617877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1a0106a80 0x7fa1a019eeb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa198033040 tx=0x7fa198003970 comp rx=0 tx=0).stop 2024-09-11T22:52:05.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.696+0000 7fa1a51c4640 1 -- 172.21.15.152:0/2235617877 shutdown_connections 2024-09-11T22:52:05.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.696+0000 7fa1a51c4640 1 --2- 172.21.15.152:0/2235617877 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa18c0a2c10 0x7fa18c0a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:05.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.696+0000 7fa1a51c4640 1 --2- 172.21.15.152:0/2235617877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1a0106a80 0x7fa1a019eeb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:05.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.696+0000 7fa1a51c4640 1 --2- 172.21.15.152:0/2235617877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a0101420 0x7fa1a019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:05.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.696+0000 7fa1a51c4640 1 --2- 172.21.15.152:0/2235617877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1a0100a80 0x7fa1a019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:05.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.696+0000 7fa1a51c4640 1 -- 172.21.15.152:0/2235617877 >> 172.21.15.152:0/2235617877 conn(0x7fa1a00fc7a0 msgr2=0x7fa1a010c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:05.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.696+0000 7fa1a51c4640 1 -- 172.21.15.152:0/2235617877 shutdown_connections 2024-09-11T22:52:05.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:05.696+0000 7fa1a51c4640 1 -- 172.21.15.152:0/2235617877 wait complete. 2024-09-11T22:52:06.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:05 smithi191 ceph-mon[35211]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:06.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:05 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2235617877' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T22:52:06.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:05 smithi204 ceph-mon[35454]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:06.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:05 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2235617877' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T22:52:06.087 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:05 smithi152 ceph-mon[31181]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:06.087 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:05 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2235617877' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T22:52:06.447 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T22:52:06.448 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:52:06.698 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:07.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.218+0000 7f5c09c92640 1 -- 172.21.15.152:0/2964282562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c04103740 msgr2=0x7f5c04107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:07.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.218+0000 7f5c09c92640 1 --2- 172.21.15.152:0/2964282562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c04103740 0x7f5c04107b80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009930 tx=0x7f5bec02a420 comp rx=0 tx=0).stop 2024-09-11T22:52:07.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.218+0000 7f5c09c92640 1 -- 172.21.15.152:0/2964282562 shutdown_connections 2024-09-11T22:52:07.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.218+0000 7f5c09c92640 1 --2- 172.21.15.152:0/2964282562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c04108290 0x7f5c0410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:07.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.218+0000 7f5c09c92640 1 --2- 172.21.15.152:0/2964282562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c04103740 0x7f5c04107b80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:07.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.218+0000 7f5c09c92640 1 --2- 172.21.15.152:0/2964282562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c04102d90 0x7f5c04103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:07.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.218+0000 7f5c09c92640 1 -- 172.21.15.152:0/2964282562 >> 172.21.15.152:0/2964282562 conn(0x7f5c040fc7d0 msgr2=0x7f5c040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:07.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.218+0000 7f5c09c92640 1 -- 172.21.15.152:0/2964282562 shutdown_connections 2024-09-11T22:52:07.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.218+0000 7f5c09c92640 1 -- 172.21.15.152:0/2964282562 wait complete. 2024-09-11T22:52:07.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5c09c92640 1 Processor -- start 2024-09-11T22:52:07.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5c09c92640 1 -- start start 2024-09-11T22:52:07.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5c09c92640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c04102d90 0x7f5c041985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:07.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5c09c92640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c04103740 0x7f5c04198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:07.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5c09c92640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c04108290 0x7f5c0419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:07.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5c09c92640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5c04112090 con 0x7f5c04103740 2024-09-11T22:52:07.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5c09c92640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5c04111f10 con 0x7f5c04102d90 2024-09-11T22:52:07.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5c09c92640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5c04112210 con 0x7f5c04108290 2024-09-11T22:52:07.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5bfbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c04103740 0x7f5c04198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:07.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5bfbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c04103740 0x7f5c04198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38694/0 (socket says 172.21.15.152:38694) 2024-09-11T22:52:07.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5bfbfff640 1 -- 172.21.15.152:0/3872979513 learned_addr learned my addr 172.21.15.152:0/3872979513 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:07.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5c037fe640 1 --2- 172.21.15.152:0/3872979513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c04102d90 0x7f5c041985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:07.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5c03fff640 1 --2- 172.21.15.152:0/3872979513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c04108290 0x7f5c0419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:07.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5bfbfff640 1 -- 172.21.15.152:0/3872979513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c04108290 msgr2=0x7f5c0419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:07.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5bfbfff640 1 --2- 172.21.15.152:0/3872979513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c04108290 0x7f5c0419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:07.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5bfbfff640 1 -- 172.21.15.152:0/3872979513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c04102d90 msgr2=0x7f5c041985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:07.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5bfbfff640 1 --2- 172.21.15.152:0/3872979513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c04102d90 0x7f5c041985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:07.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5bfbfff640 1 -- 172.21.15.152:0/3872979513 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0419d560 con 0x7f5c04103740 2024-09-11T22:52:07.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.219+0000 7f5bfbfff640 1 --2- 172.21.15.152:0/3872979513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c04103740 0x7f5c04198af0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5bec005940 tx=0x7f5bec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:07.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.220+0000 7f5c017fa640 1 -- 172.21.15.152:0/3872979513 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec03f030 con 0x7f5c04103740 2024-09-11T22:52:07.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.220+0000 7f5c017fa640 1 -- 172.21.15.152:0/3872979513 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bec003ad0 con 0x7f5c04103740 2024-09-11T22:52:07.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.220+0000 7f5c09c92640 1 -- 172.21.15.152:0/3872979513 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c0419d7f0 con 0x7f5c04103740 2024-09-11T22:52:07.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.220+0000 7f5c017fa640 1 -- 172.21.15.152:0/3872979513 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec035070 con 0x7f5c04103740 2024-09-11T22:52:07.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.220+0000 7f5c09c92640 1 -- 172.21.15.152:0/3872979513 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c041a5130 con 0x7f5c04103740 2024-09-11T22:52:07.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.221+0000 7f5c09c92640 1 -- 172.21.15.152:0/3872979513 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c04103170 con 0x7f5c04103740 2024-09-11T22:52:07.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.222+0000 7f5c017fa640 1 -- 172.21.15.152:0/3872979513 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5bec042030 con 0x7f5c04103740 2024-09-11T22:52:07.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.222+0000 7f5c017fa640 1 --2- 172.21.15.152:0/3872979513 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f5bf00a2c90 0x7f5bf00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:07.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.222+0000 7f5c037fe640 1 --2- 172.21.15.152:0/3872979513 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f5bf00a2c90 0x7f5bf00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:07.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.222+0000 7f5c017fa640 1 -- 172.21.15.152:0/3872979513 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f5bec0ef700 con 0x7f5c04103740 2024-09-11T22:52:07.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.223+0000 7f5c037fe640 1 --2- 172.21.15.152:0/3872979513 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f5bf00a2c90 0x7f5bf00a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4005d00 tx=0x7f5bf400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:07.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.226+0000 7f5c017fa640 1 -- 172.21.15.152:0/3872979513 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5bec0bbee0 con 0x7f5c04103740 2024-09-11T22:52:07.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.428+0000 7f5c09c92640 1 -- 172.21.15.152:0/3872979513 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c0410a2a0 con 0x7f5c04103740 2024-09-11T22:52:07.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.428+0000 7f5c017fa640 1 -- 172.21.15.152:0/3872979513 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11988 (secure 0 0 0) 0x7f5bec0c01e0 con 0x7f5c04103740 2024-09-11T22:52:07.431 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:07.431 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 27 2024-09-11T22:52:07.431 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":27,"btime":"2024-09-11T22:51:06:131335+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:51:06.131313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[24455,24442,24434,24430,14628]},"id":1}]} 2024-09-11T22:52:07.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.431+0000 7f5c09c92640 1 -- 172.21.15.152:0/3872979513 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f5bf00a2c90 msgr2=0x7f5bf00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:07.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.431+0000 7f5c09c92640 1 --2- 172.21.15.152:0/3872979513 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f5bf00a2c90 0x7f5bf00a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4005d00 tx=0x7f5bf400b410 comp rx=0 tx=0).stop 2024-09-11T22:52:07.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.431+0000 7f5c09c92640 1 -- 172.21.15.152:0/3872979513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c04103740 msgr2=0x7f5c04198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:07.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.431+0000 7f5c09c92640 1 --2- 172.21.15.152:0/3872979513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c04103740 0x7f5c04198af0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5bec005940 tx=0x7f5bec004270 comp rx=0 tx=0).stop 2024-09-11T22:52:07.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.431+0000 7f5c09c92640 1 -- 172.21.15.152:0/3872979513 shutdown_connections 2024-09-11T22:52:07.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.431+0000 7f5c09c92640 1 --2- 172.21.15.152:0/3872979513 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f5bf00a2c90 0x7f5bf00a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:07.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.431+0000 7f5c09c92640 1 --2- 172.21.15.152:0/3872979513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c04108290 0x7f5c0419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:07.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.431+0000 7f5c09c92640 1 --2- 172.21.15.152:0/3872979513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c04103740 0x7f5c04198af0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:07.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.431+0000 7f5c09c92640 1 --2- 172.21.15.152:0/3872979513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c04102d90 0x7f5c041985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:07.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.431+0000 7f5c09c92640 1 -- 172.21.15.152:0/3872979513 >> 172.21.15.152:0/3872979513 conn(0x7f5c040fc7d0 msgr2=0x7f5c04106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:07.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.431+0000 7f5c09c92640 1 -- 172.21.15.152:0/3872979513 shutdown_connections 2024-09-11T22:52:07.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:07.431+0000 7f5c09c92640 1 -- 172.21.15.152:0/3872979513 wait complete. 2024-09-11T22:52:08.118 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:07 smithi191 ceph-mon[35211]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:08.118 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:07 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3872979513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:08.119 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:07 smithi204 ceph-mon[35454]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:08.120 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:07 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3872979513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:08.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:07 smithi152 ceph-mon[31181]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:08.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:07 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3872979513' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:08.294 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:52:08.543 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:09.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.071+0000 7f722a32f640 1 -- 172.21.15.152:0/1229212694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7224106f50 msgr2=0x7f722410eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:09.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.071+0000 7f722a32f640 1 --2- 172.21.15.152:0/1229212694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7224106f50 0x7f722410eaf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f721c009cd0 tx=0x7f721c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:52:09.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.071+0000 7f722a32f640 1 -- 172.21.15.152:0/1229212694 shutdown_connections 2024-09-11T22:52:09.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.071+0000 7f722a32f640 1 --2- 172.21.15.152:0/1229212694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7224106f50 0x7f722410eaf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:09.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.071+0000 7f722a32f640 1 --2- 172.21.15.152:0/1229212694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7224100ca0 0x7f7224106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:09.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.071+0000 7f722a32f640 1 --2- 172.21.15.152:0/1229212694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72241002f0 0x7f72241006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:09.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.071+0000 7f722a32f640 1 -- 172.21.15.152:0/1229212694 >> 172.21.15.152:0/1229212694 conn(0x7f7224077210 msgr2=0x7f72240fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:09.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.071+0000 7f722a32f640 1 -- 172.21.15.152:0/1229212694 shutdown_connections 2024-09-11T22:52:09.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.071+0000 7f722a32f640 1 -- 172.21.15.152:0/1229212694 wait complete. 2024-09-11T22:52:09.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.071+0000 7f722a32f640 1 Processor -- start 2024-09-11T22:52:09.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f722a32f640 1 -- start start 2024-09-11T22:52:09.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f722a32f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72241002f0 0x7f7224101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:09.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f722a32f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7224100ca0 0x7f72241017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:09.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f722a32f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7224106f50 0x7f7224101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:09.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f722a32f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7224111fa0 con 0x7f7224100ca0 2024-09-11T22:52:09.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f722a32f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7224111e20 con 0x7f7224106f50 2024-09-11T22:52:09.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f722a32f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7224112120 con 0x7f72241002f0 2024-09-11T22:52:09.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f72237fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7224100ca0 0x7f72241017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:09.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f72237fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7224100ca0 0x7f72241017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39774/0 (socket says 172.21.15.152:39774) 2024-09-11T22:52:09.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f72237fe640 1 -- 172.21.15.152:0/3684241841 learned_addr learned my addr 172.21.15.152:0/3684241841 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:09.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f72288cc640 1 --2- 172.21.15.152:0/3684241841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7224106f50 0x7f7224101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:09.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f7223fff640 1 --2- 172.21.15.152:0/3684241841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72241002f0 0x7f7224101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:09.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f72237fe640 1 -- 172.21.15.152:0/3684241841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72241002f0 msgr2=0x7f7224101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:09.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f72237fe640 1 --2- 172.21.15.152:0/3684241841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72241002f0 0x7f7224101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:09.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f72237fe640 1 -- 172.21.15.152:0/3684241841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7224106f50 msgr2=0x7f7224101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:09.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f72237fe640 1 --2- 172.21.15.152:0/3684241841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7224106f50 0x7f7224101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:09.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.072+0000 7f72237fe640 1 -- 172.21.15.152:0/3684241841 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72241a6920 con 0x7f7224100ca0 2024-09-11T22:52:09.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.073+0000 7f72237fe640 1 --2- 172.21.15.152:0/3684241841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7224100ca0 0x7f72241017b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f721800e9f0 tx=0x7f721800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:09.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.073+0000 7f72217fa640 1 -- 172.21.15.152:0/3684241841 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721800ccc0 con 0x7f7224100ca0 2024-09-11T22:52:09.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.073+0000 7f72217fa640 1 -- 172.21.15.152:0/3684241841 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7218005500 con 0x7f7224100ca0 2024-09-11T22:52:09.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.073+0000 7f72217fa640 1 -- 172.21.15.152:0/3684241841 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7218010690 con 0x7f7224100ca0 2024-09-11T22:52:09.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.073+0000 7f722a32f640 1 -- 172.21.15.152:0/3684241841 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72241a6c10 con 0x7f7224100ca0 2024-09-11T22:52:09.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.073+0000 7f722a32f640 1 -- 172.21.15.152:0/3684241841 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72241a7100 con 0x7f7224100ca0 2024-09-11T22:52:09.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.074+0000 7f722a32f640 1 -- 172.21.15.152:0/3684241841 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72241006d0 con 0x7f7224100ca0 2024-09-11T22:52:09.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.075+0000 7f72217fa640 1 -- 172.21.15.152:0/3684241841 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f72180046c0 con 0x7f7224100ca0 2024-09-11T22:52:09.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.075+0000 7f72217fa640 1 --2- 172.21.15.152:0/3684241841 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f720c0a2c10 0x7f720c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:09.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.076+0000 7f72217fa640 1 -- 172.21.15.152:0/3684241841 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f72180cf490 con 0x7f7224100ca0 2024-09-11T22:52:09.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.076+0000 7f7223fff640 1 --2- 172.21.15.152:0/3684241841 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f720c0a2c10 0x7f720c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:09.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.076+0000 7f7223fff640 1 --2- 172.21.15.152:0/3684241841 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f720c0a2c10 0x7f720c0a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f721400bbe0 tx=0x7f721400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:09.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.079+0000 7f72217fa640 1 -- 172.21.15.152:0/3684241841 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f721809c2c0 con 0x7f7224100ca0 2024-09-11T22:52:09.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.281+0000 7f722a32f640 1 -- 172.21.15.152:0/3684241841 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7224109b90 con 0x7f7224100ca0 2024-09-11T22:52:09.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.281+0000 7f72217fa640 1 -- 172.21.15.152:0/3684241841 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11988 (secure 0 0 0) 0x7f721809bbc0 con 0x7f7224100ca0 2024-09-11T22:52:09.283 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:09.283 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":27,"btime":"2024-09-11T22:51:06:131335+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:51:06.131313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[24455,24442,24434,24430,14628]},"id":1}]} 2024-09-11T22:52:09.284 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 27 2024-09-11T22:52:09.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.284+0000 7f722a32f640 1 -- 172.21.15.152:0/3684241841 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f720c0a2c10 msgr2=0x7f720c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:09.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.284+0000 7f722a32f640 1 --2- 172.21.15.152:0/3684241841 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f720c0a2c10 0x7f720c0a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f721400bbe0 tx=0x7f721400b390 comp rx=0 tx=0).stop 2024-09-11T22:52:09.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.284+0000 7f722a32f640 1 -- 172.21.15.152:0/3684241841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7224100ca0 msgr2=0x7f72241017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:09.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.284+0000 7f722a32f640 1 --2- 172.21.15.152:0/3684241841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7224100ca0 0x7f72241017b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f721800e9f0 tx=0x7f721800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:52:09.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.284+0000 7f722a32f640 1 -- 172.21.15.152:0/3684241841 shutdown_connections 2024-09-11T22:52:09.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.284+0000 7f722a32f640 1 --2- 172.21.15.152:0/3684241841 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f720c0a2c10 0x7f720c0a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:09.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.284+0000 7f722a32f640 1 --2- 172.21.15.152:0/3684241841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7224106f50 0x7f7224101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:09.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.284+0000 7f722a32f640 1 --2- 172.21.15.152:0/3684241841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7224100ca0 0x7f72241017b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:09.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.284+0000 7f722a32f640 1 --2- 172.21.15.152:0/3684241841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72241002f0 0x7f7224101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:09.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.284+0000 7f722a32f640 1 -- 172.21.15.152:0/3684241841 >> 172.21.15.152:0/3684241841 conn(0x7f7224077210 msgr2=0x7f7224105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:09.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.284+0000 7f722a32f640 1 -- 172.21.15.152:0/3684241841 shutdown_connections 2024-09-11T22:52:09.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:09.284+0000 7f722a32f640 1 -- 172.21.15.152:0/3684241841 wait complete. 2024-09-11T22:52:09.991 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:09 smithi152 ceph-mon[31181]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:09.991 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:09 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3684241841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:10.021 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:52:10.274 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:10.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:09 smithi191 ceph-mon[35211]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:10.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:09 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3684241841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:10.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:09 smithi204 ceph-mon[35454]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:10.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:09 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3684241841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:10.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.755+0000 7f2d34a08640 1 -- 172.21.15.152:0/3157387916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d301082e0 msgr2=0x7f2d3010ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:10.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.755+0000 7f2d34a08640 1 --2- 172.21.15.152:0/3157387916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d301082e0 0x7f2d3010ec50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009930 tx=0x7f2d2802a420 comp rx=0 tx=0).stop 2024-09-11T22:52:10.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.756+0000 7f2d34a08640 1 -- 172.21.15.152:0/3157387916 shutdown_connections 2024-09-11T22:52:10.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.756+0000 7f2d34a08640 1 --2- 172.21.15.152:0/3157387916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d301082e0 0x7f2d3010ec50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:10.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.756+0000 7f2d34a08640 1 --2- 172.21.15.152:0/3157387916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d30103760 0x7f2d30107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:10.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.756+0000 7f2d34a08640 1 --2- 172.21.15.152:0/3157387916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d30102db0 0x7f2d30103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:10.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.756+0000 7f2d34a08640 1 -- 172.21.15.152:0/3157387916 >> 172.21.15.152:0/3157387916 conn(0x7f2d300fc7d0 msgr2=0x7f2d300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:10.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.756+0000 7f2d34a08640 1 -- 172.21.15.152:0/3157387916 shutdown_connections 2024-09-11T22:52:10.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.756+0000 7f2d34a08640 1 -- 172.21.15.152:0/3157387916 wait complete. 2024-09-11T22:52:10.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.756+0000 7f2d34a08640 1 Processor -- start 2024-09-11T22:52:10.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.756+0000 7f2d34a08640 1 -- start start 2024-09-11T22:52:10.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d34a08640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d30102db0 0x7f2d30100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:10.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d34a08640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d30103760 0x7f2d30100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:10.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d34a08640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d301082e0 0x7f2d30101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:10.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d34a08640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2d301120b0 con 0x7f2d301082e0 2024-09-11T22:52:10.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d34a08640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2d30111f30 con 0x7f2d30103760 2024-09-11T22:52:10.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d34a08640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2d30112230 con 0x7f2d30102db0 2024-09-11T22:52:10.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d2ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d301082e0 0x7f2d30101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:10.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d2e59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d30102db0 0x7f2d30100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:10.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d2ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d301082e0 0x7f2d30101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39792/0 (socket says 172.21.15.152:39792) 2024-09-11T22:52:10.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d2ed9d640 1 -- 172.21.15.152:0/1162783415 learned_addr learned my addr 172.21.15.152:0/1162783415 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:10.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d2dd9b640 1 --2- 172.21.15.152:0/1162783415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d30103760 0x7f2d30100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:10.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d2ed9d640 1 -- 172.21.15.152:0/1162783415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d30102db0 msgr2=0x7f2d30100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:10.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d2ed9d640 1 --2- 172.21.15.152:0/1162783415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d30102db0 0x7f2d30100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:10.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d2ed9d640 1 -- 172.21.15.152:0/1162783415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d30103760 msgr2=0x7f2d30100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:10.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d2ed9d640 1 --2- 172.21.15.152:0/1162783415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d30103760 0x7f2d30100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:10.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d2ed9d640 1 -- 172.21.15.152:0/1162783415 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d301a6a20 con 0x7f2d301082e0 2024-09-11T22:52:10.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d2ed9d640 1 --2- 172.21.15.152:0/1162783415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d301082e0 0x7f2d30101490 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2d28005ab0 tx=0x7f2d28003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:10.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d277fe640 1 -- 172.21.15.152:0/1162783415 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2803f070 con 0x7f2d301082e0 2024-09-11T22:52:10.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d277fe640 1 -- 172.21.15.152:0/1162783415 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d28031c30 con 0x7f2d301082e0 2024-09-11T22:52:10.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.757+0000 7f2d277fe640 1 -- 172.21.15.152:0/1162783415 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28035030 con 0x7f2d301082e0 2024-09-11T22:52:10.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.758+0000 7f2d34a08640 1 -- 172.21.15.152:0/1162783415 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d301a6cb0 con 0x7f2d301082e0 2024-09-11T22:52:10.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.758+0000 7f2d34a08640 1 -- 172.21.15.152:0/1162783415 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d301a7140 con 0x7f2d301082e0 2024-09-11T22:52:10.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.758+0000 7f2d34a08640 1 -- 172.21.15.152:0/1162783415 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d30103190 con 0x7f2d301082e0 2024-09-11T22:52:10.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.760+0000 7f2d277fe640 1 -- 172.21.15.152:0/1162783415 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2d28042070 con 0x7f2d301082e0 2024-09-11T22:52:10.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.760+0000 7f2d277fe640 1 --2- 172.21.15.152:0/1162783415 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2d1c0a2c10 0x7f2d1c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:10.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.761+0000 7f2d2e59c640 1 --2- 172.21.15.152:0/1162783415 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2d1c0a2c10 0x7f2d1c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:10.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.761+0000 7f2d277fe640 1 -- 172.21.15.152:0/1162783415 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f2d280efc70 con 0x7f2d301082e0 2024-09-11T22:52:10.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.761+0000 7f2d2e59c640 1 --2- 172.21.15.152:0/1162783415 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2d1c0a2c10 0x7f2d1c0a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2d20005c40 tx=0x7f2d2000a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:10.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.764+0000 7f2d277fe640 1 -- 172.21.15.152:0/1162783415 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d280bc450 con 0x7f2d301082e0 2024-09-11T22:52:10.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.935+0000 7f2d34a08640 1 -- 172.21.15.152:0/1162783415 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d3010a2c0 con 0x7f2d301082e0 2024-09-11T22:52:10.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.936+0000 7f2d277fe640 1 -- 172.21.15.152:0/1162783415 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22710 (secure 0 0 0) 0x7f2d280c0750 con 0x7f2d301082e0 2024-09-11T22:52:10.938 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:10.938 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":83,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","created":"2024-09-11T22:43:00.379317+0000","modified":"2024-09-11T22:51:55.946729+0000","last_up_change":"2024-09-11T22:50:01.043181+0000","last_in_change":"2024-09-11T22:49:38.073870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:46:10.984707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:50:50.833547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:50:53.047604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3cd917b5-77f6-4a4a-8fd0-4ebe3034feac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6803","nonce":1270728007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6805","nonce":1270728007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6809","nonce":1270728007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6807","nonce":1270728007}]},"public_addr":"172.21.15.152:6803/1270728007","cluster_addr":"172.21.15.152:6805/1270728007","heartbeat_back_addr":"172.21.15.152:6809/1270728007","heartbeat_front_addr":"172.21.15.152:6807/1270728007","state":["exists","up"]},{"osd":1,"uuid":"ae589bc4-77b1-4522-9591-b0ce2a5432dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6801","nonce":354467155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6803","nonce":354467155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6807","nonce":354467155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6805","nonce":354467155}]},"public_addr":"172.21.15.191:6801/354467155","cluster_addr":"172.21.15.191:6803/354467155","heartbeat_back_addr":"172.21.15.191:6807/354467155","heartbeat_front_addr":"172.21.15.191:6805/354467155","state":["exists","up"]},{"osd":2,"uuid":"b35ffd0c-d084-43dd-a947-15132d78d59c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6801","nonce":3848396211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6803","nonce":3848396211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6807","nonce":3848396211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6805","nonce":3848396211}]},"public_addr":"172.21.15.204:6801/3848396211","cluster_addr":"172.21.15.204:6803/3848396211","heartbeat_back_addr":"172.21.15.204:6807/3848396211","heartbeat_front_addr":"172.21.15.204:6805/3848396211","state":["exists","up"]},{"osd":3,"uuid":"d66ba7e8-4ab7-449e-9460-2f44ffdf987f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6811","nonce":3351257181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6813","nonce":3351257181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6817","nonce":3351257181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6815","nonce":3351257181}]},"public_addr":"172.21.15.152:6811/3351257181","cluster_addr":"172.21.15.152:6813/3351257181","heartbeat_back_addr":"172.21.15.152:6817/3351257181","heartbeat_front_addr":"172.21.15.152:6815/3351257181","state":["exists","up"]},{"osd":4,"uuid":"e1b56468-6cc7-4d29-988e-9d281118ee36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6809","nonce":2967539563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6811","nonce":2967539563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6815","nonce":2967539563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6813","nonce":2967539563}]},"public_addr":"172.21.15.191:6809/2967539563","cluster_addr":"172.21.15.191:6811/2967539563","heartbeat_back_addr":"172.21.15.191:6815/2967539563","heartbeat_front_addr":"172.21.15.191:6813/2967539563","state":["exists","up"]},{"osd":5,"uuid":"89dabd3e-77e3-4294-9648-b112ef694e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6809","nonce":4283562158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6811","nonce":4283562158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6815","nonce":4283562158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6813","nonce":4283562158}]},"public_addr":"172.21.15.204:6809/4283562158","cluster_addr":"172.21.15.204:6811/4283562158","heartbeat_back_addr":"172.21.15.204:6815/4283562158","heartbeat_front_addr":"172.21.15.204:6813/4283562158","state":["exists","up"]},{"osd":6,"uuid":"46a0fded-b668-4aef-a68f-613cccc83483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6819","nonce":2484234042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6821","nonce":2484234042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6825","nonce":2484234042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6823","nonce":2484234042}]},"public_addr":"172.21.15.152:6819/2484234042","cluster_addr":"172.21.15.152:6821/2484234042","heartbeat_back_addr":"172.21.15.152:6825/2484234042","heartbeat_front_addr":"172.21.15.152:6823/2484234042","state":["exists","up"]},{"osd":7,"uuid":"e674bfe9-e2db-44a3-b74b-a21f198747c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6817","nonce":269608859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6819","nonce":269608859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6823","nonce":269608859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6821","nonce":269608859}]},"public_addr":"172.21.15.191:6817/269608859","cluster_addr":"172.21.15.191:6819/269608859","heartbeat_back_addr":"172.21.15.191:6823/269608859","heartbeat_front_addr":"172.21.15.191:6821/269608859","state":["exists","up"]},{"osd":8,"uuid":"0418bfc9-d6b6-488f-b4c7-0bc6291bb624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6817","nonce":1756190090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6819","nonce":1756190090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6823","nonce":1756190090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6821","nonce":1756190090}]},"public_addr":"172.21.15.204:6817/1756190090","cluster_addr":"172.21.15.204:6819/1756190090","heartbeat_back_addr":"172.21.15.204:6823/1756190090","heartbeat_front_addr":"172.21.15.204:6821/1756190090","state":["exists","up"]},{"osd":9,"uuid":"751a6964-798f-4240-a2ba-7fa9f0c72657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6827","nonce":1301191419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6829","nonce":1301191419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6833","nonce":1301191419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6831","nonce":1301191419}]},"public_addr":"172.21.15.152:6827/1301191419","cluster_addr":"172.21.15.152:6829/1301191419","heartbeat_back_addr":"172.21.15.152:6833/1301191419","heartbeat_front_addr":"172.21.15.152:6831/1301191419","state":["exists","up"]},{"osd":10,"uuid":"90e7c6ba-127c-4b33-b3c6-cedf72128451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6825","nonce":3695294503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6827","nonce":3695294503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6831","nonce":3695294503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6829","nonce":3695294503}]},"public_addr":"172.21.15.191:6825/3695294503","cluster_addr":"172.21.15.191:6827/3695294503","heartbeat_back_addr":"172.21.15.191:6831/3695294503","heartbeat_front_addr":"172.21.15.191:6829/3695294503","state":["exists","up"]},{"osd":11,"uuid":"c76a93b1-a045-477b-98d0-58a2e9d44a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6825","nonce":1709433236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6827","nonce":1709433236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6831","nonce":1709433236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6829","nonce":1709433236}]},"public_addr":"172.21.15.204:6825/1709433236","cluster_addr":"172.21.15.204:6827/1709433236","heartbeat_back_addr":"172.21.15.204:6831/1709433236","heartbeat_front_addr":"172.21.15.204:6829/1709433236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:17.259744+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:42.230389+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:07.172433+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:32.517060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:58.900615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:24.379803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:50.058928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:15.967299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:42.025873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:08.225448+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:33.910531+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:59.466423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:0/1542993850":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/3071755865":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/2452559014":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/4286398432":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6801/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6800/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6800/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/3960601922":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2270507277":"2024-09-12T22:43:31.970432+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:52:10.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.938+0000 7f2d34a08640 1 -- 172.21.15.152:0/1162783415 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2d1c0a2c10 msgr2=0x7f2d1c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:10.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.938+0000 7f2d34a08640 1 --2- 172.21.15.152:0/1162783415 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2d1c0a2c10 0x7f2d1c0a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2d20005c40 tx=0x7f2d2000a590 comp rx=0 tx=0).stop 2024-09-11T22:52:10.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.938+0000 7f2d34a08640 1 -- 172.21.15.152:0/1162783415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d301082e0 msgr2=0x7f2d30101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:10.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.938+0000 7f2d34a08640 1 --2- 172.21.15.152:0/1162783415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d301082e0 0x7f2d30101490 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2d28005ab0 tx=0x7f2d28003c10 comp rx=0 tx=0).stop 2024-09-11T22:52:10.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.938+0000 7f2d34a08640 1 -- 172.21.15.152:0/1162783415 shutdown_connections 2024-09-11T22:52:10.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.938+0000 7f2d34a08640 1 --2- 172.21.15.152:0/1162783415 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f2d1c0a2c10 0x7f2d1c0a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:10.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.938+0000 7f2d34a08640 1 --2- 172.21.15.152:0/1162783415 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d301082e0 0x7f2d30101490 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:10.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.938+0000 7f2d34a08640 1 --2- 172.21.15.152:0/1162783415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d30103760 0x7f2d30100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:10.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.939+0000 7f2d34a08640 1 --2- 172.21.15.152:0/1162783415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d30102db0 0x7f2d30100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:10.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.939+0000 7f2d34a08640 1 -- 172.21.15.152:0/1162783415 >> 172.21.15.152:0/1162783415 conn(0x7f2d300fc7d0 msgr2=0x7f2d30105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:10.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.939+0000 7f2d34a08640 1 -- 172.21.15.152:0/1162783415 shutdown_connections 2024-09-11T22:52:10.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:10.939+0000 7f2d34a08640 1 -- 172.21.15.152:0/1162783415 wait complete. 2024-09-11T22:52:11.680 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:52:11.945 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:12.066 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:11 smithi152 ceph-mon[31181]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:12.066 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:11 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1162783415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:52:12.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:11 smithi191 ceph-mon[35211]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:12.290 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:11 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1162783415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:52:12.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:11 smithi204 ceph-mon[35454]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:12.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:11 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1162783415' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:52:12.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.469+0000 7fe6d808b640 1 -- 172.21.15.152:0/1920384021 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6d0102ef0 msgr2=0x7fe6d010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:12.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.469+0000 7fe6d808b640 1 --2- 172.21.15.152:0/1920384021 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6d0102ef0 0x7fe6d010da40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009ca0 tx=0x7fe6c802a950 comp rx=0 tx=0).stop 2024-09-11T22:52:12.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.469+0000 7fe6d808b640 1 -- 172.21.15.152:0/1920384021 shutdown_connections 2024-09-11T22:52:12.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.469+0000 7fe6d808b640 1 --2- 172.21.15.152:0/1920384021 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6d0102ef0 0x7fe6d010da40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:12.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.469+0000 7fe6d808b640 1 --2- 172.21.15.152:0/1920384021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6d0102570 0x7fe6d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:12.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.469+0000 7fe6d808b640 1 --2- 172.21.15.152:0/1920384021 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6d0100e20 0x7fe6d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:12.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.469+0000 7fe6d808b640 1 -- 172.21.15.152:0/1920384021 >> 172.21.15.152:0/1920384021 conn(0x7fe6d00fc7d0 msgr2=0x7fe6d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:12.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.469+0000 7fe6d808b640 1 -- 172.21.15.152:0/1920384021 shutdown_connections 2024-09-11T22:52:12.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.469+0000 7fe6d808b640 1 -- 172.21.15.152:0/1920384021 wait complete. 2024-09-11T22:52:12.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d808b640 1 Processor -- start 2024-09-11T22:52:12.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d808b640 1 -- start start 2024-09-11T22:52:12.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d808b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6d0100e20 0x7fe6d0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:12.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d808b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6d0102570 0x7fe6d0199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:12.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d808b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6d0102ef0 0x7fe6d01a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:12.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d808b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d0110fa0 con 0x7fe6d0102ef0 2024-09-11T22:52:12.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d808b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d0110e20 con 0x7fe6d0102570 2024-09-11T22:52:12.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d808b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d0111120 con 0x7fe6d0100e20 2024-09-11T22:52:12.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d6628640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6d0102ef0 0x7fe6d01a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:12.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d6628640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6d0102ef0 0x7fe6d01a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39802/0 (socket says 172.21.15.152:39802) 2024-09-11T22:52:12.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d6628640 1 -- 172.21.15.152:0/2989869602 learned_addr learned my addr 172.21.15.152:0/2989869602 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:12.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d5626640 1 --2- 172.21.15.152:0/2989869602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6d0102570 0x7fe6d0199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:12.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d6628640 1 -- 172.21.15.152:0/2989869602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6d0100e20 msgr2=0x7fe6d0199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:12.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d6628640 1 --2- 172.21.15.152:0/2989869602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6d0100e20 0x7fe6d0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:12.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d6628640 1 -- 172.21.15.152:0/2989869602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6d0102570 msgr2=0x7fe6d0199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:12.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d6628640 1 --2- 172.21.15.152:0/2989869602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6d0102570 0x7fe6d0199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:12.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d6628640 1 -- 172.21.15.152:0/2989869602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d01a4910 con 0x7fe6d0102ef0 2024-09-11T22:52:12.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.470+0000 7fe6d5626640 1 --2- 172.21.15.152:0/2989869602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6d0102570 0x7fe6d0199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:52:12.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.471+0000 7fe6d6628640 1 --2- 172.21.15.152:0/2989869602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6d0102ef0 0x7fe6d01a43d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8006010 tx=0x7fe6c802f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:12.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.471+0000 7fe6c6ffd640 1 -- 172.21.15.152:0/2989869602 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8034040 con 0x7fe6d0102ef0 2024-09-11T22:52:12.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.471+0000 7fe6c6ffd640 1 -- 172.21.15.152:0/2989869602 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c8003e00 con 0x7fe6d0102ef0 2024-09-11T22:52:12.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.471+0000 7fe6c6ffd640 1 -- 172.21.15.152:0/2989869602 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8045480 con 0x7fe6d0102ef0 2024-09-11T22:52:12.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.471+0000 7fe6d808b640 1 -- 172.21.15.152:0/2989869602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d01a4ae0 con 0x7fe6d0102ef0 2024-09-11T22:52:12.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.471+0000 7fe6d808b640 1 -- 172.21.15.152:0/2989869602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d01a4ee0 con 0x7fe6d0102ef0 2024-09-11T22:52:12.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.472+0000 7fe6d808b640 1 -- 172.21.15.152:0/2989869602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d0101200 con 0x7fe6d0102ef0 2024-09-11T22:52:12.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.473+0000 7fe6c6ffd640 1 -- 172.21.15.152:0/2989869602 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe6c8037020 con 0x7fe6d0102ef0 2024-09-11T22:52:12.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.474+0000 7fe6c6ffd640 1 --2- 172.21.15.152:0/2989869602 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6bc0a2c10 0x7fe6bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:12.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.474+0000 7fe6d5e27640 1 --2- 172.21.15.152:0/2989869602 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6bc0a2c10 0x7fe6bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:12.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.474+0000 7fe6c6ffd640 1 -- 172.21.15.152:0/2989869602 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fe6c80f0260 con 0x7fe6d0102ef0 2024-09-11T22:52:12.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.474+0000 7fe6d5e27640 1 --2- 172.21.15.152:0/2989869602 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6bc0a2c10 0x7fe6bc0a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00a8f0 tx=0x7fe6cc005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:12.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.477+0000 7fe6c6ffd640 1 -- 172.21.15.152:0/2989869602 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6c8043380 con 0x7fe6d0102ef0 2024-09-11T22:52:12.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.676+0000 7fe6d808b640 1 -- 172.21.15.152:0/2989869602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6d01a51a0 con 0x7fe6d0102ef0 2024-09-11T22:52:12.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.677+0000 7fe6c6ffd640 1 -- 172.21.15.152:0/2989869602 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11988 (secure 0 0 0) 0x7fe6c80bd230 con 0x7fe6d0102ef0 2024-09-11T22:52:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:12.679 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":27,"btime":"2024-09-11T22:51:06:131335+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:51:06.131313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[24455,24442,24434,24430,14628]},"id":1}]} 2024-09-11T22:52:12.679 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 27 2024-09-11T22:52:12.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.679+0000 7fe6d808b640 1 -- 172.21.15.152:0/2989869602 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6bc0a2c10 msgr2=0x7fe6bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:12.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.679+0000 7fe6d808b640 1 --2- 172.21.15.152:0/2989869602 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6bc0a2c10 0x7fe6bc0a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00a8f0 tx=0x7fe6cc005bf0 comp rx=0 tx=0).stop 2024-09-11T22:52:12.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.679+0000 7fe6d808b640 1 -- 172.21.15.152:0/2989869602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6d0102ef0 msgr2=0x7fe6d01a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:12.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.679+0000 7fe6d808b640 1 --2- 172.21.15.152:0/2989869602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6d0102ef0 0x7fe6d01a43d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8006010 tx=0x7fe6c802f6f0 comp rx=0 tx=0).stop 2024-09-11T22:52:12.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.679+0000 7fe6d808b640 1 -- 172.21.15.152:0/2989869602 shutdown_connections 2024-09-11T22:52:12.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.679+0000 7fe6d808b640 1 --2- 172.21.15.152:0/2989869602 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6bc0a2c10 0x7fe6bc0a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:12.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.679+0000 7fe6d808b640 1 --2- 172.21.15.152:0/2989869602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6d0102ef0 0x7fe6d01a43d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:12.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.679+0000 7fe6d808b640 1 --2- 172.21.15.152:0/2989869602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe6d0102570 0x7fe6d0199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:12.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.679+0000 7fe6d808b640 1 --2- 172.21.15.152:0/2989869602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6d0100e20 0x7fe6d0199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:12.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.679+0000 7fe6d808b640 1 -- 172.21.15.152:0/2989869602 >> 172.21.15.152:0/2989869602 conn(0x7fe6d00fc7d0 msgr2=0x7fe6d0106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:12.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.679+0000 7fe6d808b640 1 -- 172.21.15.152:0/2989869602 shutdown_connections 2024-09-11T22:52:12.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:12.679+0000 7fe6d808b640 1 -- 172.21.15.152:0/2989869602 wait complete. 2024-09-11T22:52:13.053 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:12 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2989869602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:13.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:12 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2989869602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:13.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:12 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2989869602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:13.440 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T22:50:55.047332+0000', 'modified': '2024-09-11T22:51:06.131313+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24455, 'mds_1': 24442, 'mds_2': 24434, 'mds_3': 24430, 'mds_4': 14628}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6839/97179606', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6838', 'nonce': 97179606}, {'type': 'v1', 'addr': '172.21.15.152:6839', 'nonce': 97179606}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.152:6841/2492281944', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6840', 'nonce': 2492281944}, {'type': 'v1', 'addr': '172.21.15.152:6841', 'nonce': 2492281944}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24377': {'gid': 24377, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6833/2321157769', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6832', 'nonce': 2321157769}, {'type': 'v1', 'addr': '172.21.15.191:6833', 'nonce': 2321157769}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24397': {'gid': 24397, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6833/2200342090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6832', 'nonce': 2200342090}, {'type': 'v1', 'addr': '172.21.15.204:6833', 'nonce': 2200342090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6835/2471607007', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6834', 'nonce': 2471607007}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 2471607007}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/763950562', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 763950562}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 763950562}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.204:6837/1863048373', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 1863048373}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 1863048373}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.191:6837/314110375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6836', 'nonce': 314110375}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 314110375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.204:6839/3360312546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 3360312546}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 3360312546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.191:6839/2588730805', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6838', 'nonce': 2588730805}, {'type': 'v1', 'addr': '172.21.15.191:6839', 'nonce': 2588730805}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24455, 'qdb_cluster': [24455, 24442, 24434, 24430, 14628]} 2024-09-11T22:52:13.440 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T22:52:13.440 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.a status 2024-09-11T22:52:13.686 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:14.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:13 smithi152 ceph-mon[31181]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:14.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.222+0000 7f10cc1d1640 1 -- 172.21.15.152:0/559684041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10c4100a50 msgr2=0x7f10c41057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:14.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.222+0000 7f10cc1d1640 1 --2- 172.21.15.152:0/559684041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10c4100a50 0x7f10c41057e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f10bc009ca0 tx=0x7f10bc02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.222+0000 7f10cc1d1640 1 -- 172.21.15.152:0/559684041 shutdown_connections 2024-09-11T22:52:14.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.222+0000 7f10cc1d1640 1 --2- 172.21.15.152:0/559684041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10c4100a50 0x7f10c41057e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.222+0000 7f10cc1d1640 1 --2- 172.21.15.152:0/559684041 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10c4100090 0x7f10c4100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.222+0000 7f10cc1d1640 1 --2- 172.21.15.152:0/559684041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10c4106b70 0x7f10c4106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.222+0000 7f10cc1d1640 1 -- 172.21.15.152:0/559684041 >> 172.21.15.152:0/559684041 conn(0x7f10c4077210 msgr2=0x7f10c40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:14.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.222+0000 7f10cc1d1640 1 -- 172.21.15.152:0/559684041 shutdown_connections 2024-09-11T22:52:14.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.222+0000 7f10cc1d1640 1 -- 172.21.15.152:0/559684041 wait complete. 2024-09-11T22:52:14.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.223+0000 7f10cc1d1640 1 Processor -- start 2024-09-11T22:52:14.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.223+0000 7f10cc1d1640 1 -- start start 2024-09-11T22:52:14.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.223+0000 7f10cc1d1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10c4100090 0x7f10c41a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.223+0000 7f10cc1d1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10c4100a50 0x7f10c41a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.223+0000 7f10cc1d1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10c4106b70 0x7f10c419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.223+0000 7f10cc1d1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f10c41185c0 con 0x7f10c4100a50 2024-09-11T22:52:14.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.223+0000 7f10cc1d1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f10c4118440 con 0x7f10c4100090 2024-09-11T22:52:14.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.223+0000 7f10cc1d1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f10c4118740 con 0x7f10c4106b70 2024-09-11T22:52:14.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.223+0000 7f10c976c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10c4100a50 0x7f10c41a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.223+0000 7f10c976c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10c4100a50 0x7f10c41a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39820/0 (socket says 172.21.15.152:39820) 2024-09-11T22:52:14.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.223+0000 7f10c976c640 1 -- 172.21.15.152:0/1322876864 learned_addr learned my addr 172.21.15.152:0/1322876864 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:14.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.224+0000 7f10c9f6d640 1 --2- 172.21.15.152:0/1322876864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10c4100090 0x7f10c41a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.224+0000 7f10ca76e640 1 --2- 172.21.15.152:0/1322876864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10c4106b70 0x7f10c419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.224+0000 7f10c976c640 1 -- 172.21.15.152:0/1322876864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10c4106b70 msgr2=0x7f10c419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:14.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.224+0000 7f10c976c640 1 --2- 172.21.15.152:0/1322876864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10c4106b70 0x7f10c419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.224+0000 7f10c976c640 1 -- 172.21.15.152:0/1322876864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10c4100090 msgr2=0x7f10c41a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:14.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.224+0000 7f10c976c640 1 --2- 172.21.15.152:0/1322876864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10c4100090 0x7f10c41a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.224+0000 7f10c976c640 1 -- 172.21.15.152:0/1322876864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c419f000 con 0x7f10c4100a50 2024-09-11T22:52:14.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.224+0000 7f10ca76e640 1 --2- 172.21.15.152:0/1322876864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10c4106b70 0x7f10c419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:52:14.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.224+0000 7f10c9f6d640 1 --2- 172.21.15.152:0/1322876864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10c4100090 0x7f10c41a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:52:14.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.224+0000 7f10c976c640 1 --2- 172.21.15.152:0/1322876864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10c4100a50 0x7f10c41a4c50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f10c000c890 tx=0x7f10c000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:14.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.224+0000 7f10baffd640 1 -- 172.21.15.152:0/1322876864 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0004220 con 0x7f10c4100a50 2024-09-11T22:52:14.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.224+0000 7f10cc1d1640 1 -- 172.21.15.152:0/1322876864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c419f2f0 con 0x7f10c4100a50 2024-09-11T22:52:14.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.224+0000 7f10baffd640 1 -- 172.21.15.152:0/1322876864 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10c00043c0 con 0x7f10c4100a50 2024-09-11T22:52:14.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.224+0000 7f10baffd640 1 -- 172.21.15.152:0/1322876864 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0004a00 con 0x7f10c4100a50 2024-09-11T22:52:14.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.224+0000 7f10cc1d1640 1 -- 172.21.15.152:0/1322876864 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c41ab5e0 con 0x7f10c4100a50 2024-09-11T22:52:14.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.226+0000 7f10baffd640 1 -- 172.21.15.152:0/1322876864 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f10c00079b0 con 0x7f10c4100a50 2024-09-11T22:52:14.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.226+0000 7f10ce5aa740 1 Processor -- start 2024-09-11T22:52:14.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10ce5aa740 1 -- start start 2024-09-11T22:52:14.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10ce5aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399d9d0 0x561de399ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10ce5aa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399e2f0 0x561de39a8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10ce5aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de39a8bb0 0x561de39aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10ce5aa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x561de3904090 con 0x561de399e2f0 2024-09-11T22:52:14.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10ce5aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561de38aba70 con 0x561de399d9d0 2024-09-11T22:52:14.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10ce5aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561de3900050 con 0x561de39a8bb0 2024-09-11T22:52:14.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10c9f6d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399e2f0 0x561de39a8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10c9f6d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399e2f0 0x561de39a8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39822/0 (socket says 172.21.15.152:39822) 2024-09-11T22:52:14.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10c9f6d640 1 -- 172.21.15.152:0/1459990091 learned_addr learned my addr 172.21.15.152:0/1459990091 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:14.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10c976c640 1 --2- 172.21.15.152:0/1459990091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de39a8bb0 0x561de39aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10ca76e640 1 --2- 172.21.15.152:0/1459990091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399d9d0 0x561de399ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10baffd640 1 --2- 172.21.15.152:0/1322876864 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f10b00a2c10 0x7f10b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10ca76e640 1 --2- 172.21.15.152:0/1322876864 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f10b00a2c10 0x7f10b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10c9f6d640 1 -- 172.21.15.152:0/1459990091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de39a8bb0 msgr2=0x561de39aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:14.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10c9f6d640 1 --2- 172.21.15.152:0/1459990091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de39a8bb0 0x561de39aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10c9f6d640 1 -- 172.21.15.152:0/1459990091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399d9d0 msgr2=0x561de399ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:14.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10c9f6d640 1 --2- 172.21.15.152:0/1459990091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399d9d0 0x561de399ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10c9f6d640 1 -- 172.21.15.152:0/1459990091 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561de38b8130 con 0x561de399e2f0 2024-09-11T22:52:14.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10baffd640 1 -- 172.21.15.152:0/1322876864 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f10c00cef50 con 0x7f10c4100a50 2024-09-11T22:52:14.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.227+0000 7f10ca76e640 1 --2- 172.21.15.152:0/1459990091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399d9d0 0x561de399ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:52:14.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.228+0000 7f10c9f6d640 1 --2- 172.21.15.152:0/1459990091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399e2f0 0x561de39a8670 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f10b400a9d0 tx=0x7f10b400ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64fa583ecb872d63 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:14.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.228+0000 7f10ca76e640 1 --2- 172.21.15.152:0/1322876864 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f10b00a2c10 0x7f10b00a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f10bc009bf0 tx=0x7f10bc02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:14.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.228+0000 7f10affff640 1 -- 172.21.15.152:0/1459990091 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4015070 con 0x561de399e2f0 2024-09-11T22:52:14.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.228+0000 7f10affff640 1 -- 172.21.15.152:0/1459990091 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10b400cc60 con 0x561de399e2f0 2024-09-11T22:52:14.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.228+0000 7f10affff640 1 -- 172.21.15.152:0/1459990091 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b40105b0 con 0x561de399e2f0 2024-09-11T22:52:14.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.229+0000 7f10ce5aa740 1 -- 172.21.15.152:0/1459990091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399e2f0 msgr2=0x561de39a8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:14.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.229+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/1459990091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399e2f0 0x561de39a8670 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f10b400a9d0 tx=0x7f10b400ae90 comp rx=0 tx=0).stop 2024-09-11T22:52:14.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.229+0000 7f10ce5aa740 1 -- 172.21.15.152:0/1459990091 shutdown_connections 2024-09-11T22:52:14.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.229+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/1459990091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de39a8bb0 0x561de39aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.229+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/1459990091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399e2f0 0x561de39a8670 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.229+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/1459990091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399d9d0 0x561de399ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.229+0000 7f10ce5aa740 1 -- 172.21.15.152:0/1459990091 >> 172.21.15.152:0/1459990091 conn(0x561de3999f60 msgr2=0x561de399a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:14.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.229+0000 7f10ce5aa740 1 -- 172.21.15.152:0/1459990091 shutdown_connections 2024-09-11T22:52:14.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.229+0000 7f10ce5aa740 1 -- 172.21.15.152:0/1459990091 wait complete. 2024-09-11T22:52:14.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.229+0000 7f10ce5aa740 1 -- start start 2024-09-11T22:52:14.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.229+0000 7f10ce5aa740 1 Processor -- start 2024-09-11T22:52:14.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10ce5aa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399d9d0 0x561de39a8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10ce5aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399e2f0 0x561de39a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10ce5aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de39a8bb0 0x561de39a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10ce5aa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x561de39ab780 con 0x561de399d9d0 2024-09-11T22:52:14.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10ce5aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561de38bef20 con 0x561de399e2f0 2024-09-11T22:52:14.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10ce5aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561de39ab900 con 0x561de39a8bb0 2024-09-11T22:52:14.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10ce5aa740 10 client.0 _ll_register_callbacks cb 0x561de3888030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:14.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10c9f6d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399d9d0 0x561de39a8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10c9f6d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399d9d0 0x561de39a8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39824/0 (socket says 172.21.15.152:39824) 2024-09-11T22:52:14.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10c9f6d640 1 -- 172.21.15.152:0/2911999770 learned_addr learned my addr 172.21.15.152:0/2911999770 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:14.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10ca76e640 1 --2- 172.21.15.152:0/2911999770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de39a8bb0 0x561de39a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10c976c640 1 --2- 172.21.15.152:0/2911999770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399e2f0 0x561de39a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10c9f6d640 1 -- 172.21.15.152:0/2911999770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de39a8bb0 msgr2=0x561de39a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:14.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10c9f6d640 1 --2- 172.21.15.152:0/2911999770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de39a8bb0 0x561de39a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10c9f6d640 1 -- 172.21.15.152:0/2911999770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399e2f0 msgr2=0x561de39a56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:14.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10c9f6d640 1 --2- 172.21.15.152:0/2911999770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399e2f0 0x561de39a56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10c9f6d640 1 -- 172.21.15.152:0/2911999770 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561de36752b0 con 0x561de399d9d0 2024-09-11T22:52:14.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10c976c640 1 --2- 172.21.15.152:0/2911999770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399e2f0 0x561de39a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:52:14.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10c9f6d640 1 --2- 172.21.15.152:0/2911999770 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399d9d0 0x561de39a8380 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f10b4004810 tx=0x7f10b40119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965e7865ceed6545 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:14.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:14.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10ad7fa640 1 -- 172.21.15.152:0/2911999770 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4020070 con 0x561de399d9d0 2024-09-11T22:52:14.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10ce5aa740 1 -- 172.21.15.152:0/2911999770 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10b400cc60 con 0x561de399d9d0 2024-09-11T22:52:14.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10ad7fa640 1 -- 172.21.15.152:0/2911999770 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f10b400a4e0 con 0x561de399d9d0 2024-09-11T22:52:14.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10ad7fa640 1 -- 172.21.15.152:0/2911999770 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f10b4015040 con 0x561de399d9d0 2024-09-11T22:52:14.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10ce5aa740 10 client.14988 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:14.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.230+0000 7f10ce5aa740 1 -- 172.21.15.152:0/2911999770 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561de39b6fe0 con 0x561de399d9d0 2024-09-11T22:52:14.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.231+0000 7f10ad7fa640 1 -- 172.21.15.152:0/2911999770 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10b400cc60 con 0x561de399d9d0 2024-09-11T22:52:14.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.231+0000 7f10ad7fa640 1 -- 172.21.15.152:0/2911999770 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f10b402f550 con 0x561de399d9d0 2024-09-11T22:52:14.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.231+0000 7f10ce5aa740 10 client.14988 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:14.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.231+0000 7f10ce5aa740 10 client.14988 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T22:52:14.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.231+0000 7f10ce5aa740 20 client.14988 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:14.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.231+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/2911999770 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x561de39b92b0 0x561de39ba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.231+0000 7f10ce5aa740 4 client.14988 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:52:14.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.231+0000 7f10ce5aa740 1 -- 172.21.15.152:0/2911999770 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561de3911600 con 0x561de39b92b0 2024-09-11T22:52:14.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.231+0000 7f10c976c640 1 --2- 172.21.15.152:0/2911999770 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x561de39b92b0 0x561de39ba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.231+0000 7f10ad7fa640 10 client.14988 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:52:14.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.231+0000 7f10c976c640 1 --2- 172.21.15.152:0/2911999770 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x561de39b92b0 0x561de39ba720 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ac0c3be97e348baa server_cookie=4c9c557da4f143f5 in_seq=0 out_seq=0 2024-09-11T22:52:14.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.232+0000 7f10ad7fa640 1 -- 172.21.15.152:0/2911999770 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561de3911600 con 0x561de39b92b0 2024-09-11T22:52:14.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.232+0000 7f10ad7fa640 10 client.14988 handle_command_reply: tid=0 2024-09-11T22:52:14.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.232+0000 7f10ce5aa740 1 client.14988 shutdown 2024-09-11T22:52:14.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.232+0000 7f10ce5aa740 1 -- 172.21.15.152:0/2911999770 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399d9d0 msgr2=0x561de39a8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:14.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.232+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/2911999770 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399d9d0 0x561de39a8380 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f10b4004810 tx=0x7f10b40119b0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.232+0000 7f10ce5aa740 1 -- 172.21.15.152:0/2911999770 shutdown_connections 2024-09-11T22:52:14.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.232+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/2911999770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de39a8bb0 0x561de39a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.232+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/2911999770 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x561de39b92b0 0x561de39ba720 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.232+0000 7f10ad7fa640 0 client.14988 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:52:14.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.232+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/2911999770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399e2f0 0x561de39a56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.232+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/2911999770 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399d9d0 0x561de39a8380 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.232+0000 7f10ce5aa740 1 -- 172.21.15.152:0/2911999770 >> 172.21.15.152:0/2911999770 conn(0x561de3999f60 msgr2=0x561de399fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:14.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.232+0000 7f10ce5aa740 1 -- 172.21.15.152:0/2911999770 shutdown_connections 2024-09-11T22:52:14.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.232+0000 7f10ce5aa740 1 -- 172.21.15.152:0/2911999770 wait complete. 2024-09-11T22:52:14.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.232+0000 7f10ce5aa740 20 client.14988 trim_cache size 0 max 16384 2024-09-11T22:52:14.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.250+0000 7f10ce5aa740 1 Processor -- start 2024-09-11T22:52:14.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.250+0000 7f10ce5aa740 1 -- start start 2024-09-11T22:52:14.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10ce5aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399d9d0 0x561de39ba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10ce5aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de399e2f0 0x561de39a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10ce5aa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de39a8bb0 0x561de39a7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10ce5aa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x561de39b8a60 con 0x561de39a8bb0 2024-09-11T22:52:14.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10ce5aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561de39b9130 con 0x561de399d9d0 2024-09-11T22:52:14.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10ce5aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561de39b9800 con 0x561de399e2f0 2024-09-11T22:52:14.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10ca76e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de39a8bb0 0x561de39a7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10ca76e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de39a8bb0 0x561de39a7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39830/0 (socket says 172.21.15.152:39830) 2024-09-11T22:52:14.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10ca76e640 1 -- 172.21.15.152:0/2477069564 learned_addr learned my addr 172.21.15.152:0/2477069564 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:14.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10c976c640 1 --2- 172.21.15.152:0/2477069564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de399e2f0 0x561de39a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10c9f6d640 1 --2- 172.21.15.152:0/2477069564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399d9d0 0x561de39ba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10ca76e640 1 -- 172.21.15.152:0/2477069564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de399e2f0 msgr2=0x561de39a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:14.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10ca76e640 1 --2- 172.21.15.152:0/2477069564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de399e2f0 0x561de39a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10ca76e640 1 -- 172.21.15.152:0/2477069564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399d9d0 msgr2=0x561de39ba720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:14.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10ca76e640 1 --2- 172.21.15.152:0/2477069564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399d9d0 0x561de39ba720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10ca76e640 1 -- 172.21.15.152:0/2477069564 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561de3999d90 con 0x561de39a8bb0 2024-09-11T22:52:14.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10ca76e640 1 --2- 172.21.15.152:0/2477069564 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de39a8bb0 0x561de39a7640 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f10bc02b010 tx=0x7f10bc0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8985cc72b84e0934 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:14.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10af7fe640 1 -- 172.21.15.152:0/2477069564 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc051070 con 0x561de39a8bb0 2024-09-11T22:52:14.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10af7fe640 1 -- 172.21.15.152:0/2477069564 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10bc036040 con 0x561de39a8bb0 2024-09-11T22:52:14.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.251+0000 7f10af7fe640 1 -- 172.21.15.152:0/2477069564 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc044030 con 0x561de39a8bb0 2024-09-11T22:52:14.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.252+0000 7f10ce5aa740 1 -- 172.21.15.152:0/2477069564 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de39a8bb0 msgr2=0x561de39a7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:14.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.252+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/2477069564 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de39a8bb0 0x561de39a7640 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f10bc02b010 tx=0x7f10bc0093d0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.252+0000 7f10ce5aa740 1 -- 172.21.15.152:0/2477069564 shutdown_connections 2024-09-11T22:52:14.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.252+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/2477069564 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de39a8bb0 0x561de39a7640 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.252+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/2477069564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de399e2f0 0x561de39a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.252+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/2477069564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399d9d0 0x561de39ba720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.252+0000 7f10ce5aa740 1 -- 172.21.15.152:0/2477069564 >> 172.21.15.152:0/2477069564 conn(0x561de3999f60 msgr2=0x561de399a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:14.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.252+0000 7f10ce5aa740 1 -- 172.21.15.152:0/2477069564 shutdown_connections 2024-09-11T22:52:14.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.252+0000 7f10ce5aa740 1 -- 172.21.15.152:0/2477069564 wait complete. 2024-09-11T22:52:14.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10ce5aa740 1 -- start start 2024-09-11T22:52:14.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10ce5aa740 1 Processor -- start 2024-09-11T22:52:14.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10ce5aa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399d9d0 0x561de39c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10ce5aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399e2f0 0x561de39bc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10ce5aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de39a8bb0 0x561de39bc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10ce5aa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x561de39ba5a0 con 0x561de399d9d0 2024-09-11T22:52:14.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10ce5aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561de39b9ed0 con 0x561de399e2f0 2024-09-11T22:52:14.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10ce5aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561de39bac70 con 0x561de39a8bb0 2024-09-11T22:52:14.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10ce5aa740 10 client.0 _ll_register_callbacks cb 0x7f10c4107710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:14.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10c9f6d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399d9d0 0x561de39c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10c9f6d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399d9d0 0x561de39c2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39836/0 (socket says 172.21.15.152:39836) 2024-09-11T22:52:14.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10c9f6d640 1 -- 172.21.15.152:0/3499554153 learned_addr learned my addr 172.21.15.152:0/3499554153 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:14.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10c976c640 1 --2- 172.21.15.152:0/3499554153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399e2f0 0x561de39bc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10ca76e640 1 --2- 172.21.15.152:0/3499554153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de39a8bb0 0x561de39bc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10c9f6d640 1 -- 172.21.15.152:0/3499554153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de39a8bb0 msgr2=0x561de39bc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:14.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10c9f6d640 1 --2- 172.21.15.152:0/3499554153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de39a8bb0 0x561de39bc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10c9f6d640 1 -- 172.21.15.152:0/3499554153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399e2f0 msgr2=0x561de39bc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:14.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10c9f6d640 1 --2- 172.21.15.152:0/3499554153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399e2f0 0x561de39bc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.253+0000 7f10c9f6d640 1 -- 172.21.15.152:0/3499554153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561de399d1c0 con 0x561de399d9d0 2024-09-11T22:52:14.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.254+0000 7f10c9f6d640 1 --2- 172.21.15.152:0/3499554153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399d9d0 0x561de39c2eb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f10b400a9d0 tx=0x7f10b4006cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bda1339a4ea180 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:14.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.254+0000 7f10ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:14.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.254+0000 7f10ad7fa640 1 -- 172.21.15.152:0/3499554153 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4012e60 con 0x561de399d9d0 2024-09-11T22:52:14.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.254+0000 7f10ad7fa640 1 -- 172.21.15.152:0/3499554153 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f10b402fde0 con 0x561de399d9d0 2024-09-11T22:52:14.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.254+0000 7f10ce5aa740 1 -- 172.21.15.152:0/3499554153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10bc036040 con 0x561de399d9d0 2024-09-11T22:52:14.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.254+0000 7f10ad7fa640 1 -- 172.21.15.152:0/3499554153 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f10b403e070 con 0x561de399d9d0 2024-09-11T22:52:14.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.254+0000 7f10ce5aa740 10 client.14994 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:14.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.254+0000 7f10ce5aa740 1 -- 172.21.15.152:0/3499554153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561de39c1590 con 0x561de399d9d0 2024-09-11T22:52:14.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.254+0000 7f10ad7fa640 1 -- 172.21.15.152:0/3499554153 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10bc036040 con 0x561de399d9d0 2024-09-11T22:52:14.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.254+0000 7f10ad7fa640 1 -- 172.21.15.152:0/3499554153 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f10b4036030 con 0x561de399d9d0 2024-09-11T22:52:14.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.254+0000 7f10ce5aa740 10 client.14994 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:14.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.254+0000 7f10ce5aa740 10 client.14994 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T22:52:14.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.254+0000 7f10ce5aa740 20 client.14994 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:14.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/3499554153 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x561de39b92b0 0x561de3a26e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:14.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ce5aa740 4 client.14994 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:52:14.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ce5aa740 1 -- 172.21.15.152:0/3499554153 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "status"}) -- 0x561de390bb10 con 0x561de39b92b0 2024-09-11T22:52:14.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10c976c640 1 --2- 172.21.15.152:0/3499554153 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x561de39b92b0 0x561de3a26e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:14.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ad7fa640 10 client.14994 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:52:14.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10c976c640 1 --2- 172.21.15.152:0/3499554153 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x561de39b92b0 0x561de3a26e80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=88bca72610131ac9 server_cookie=21fdd228b5ea1f5f in_seq=0 out_seq=0 2024-09-11T22:52:14.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ad7fa640 1 -- 172.21.15.152:0/3499554153 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x561de390bb10 con 0x561de39b92b0 2024-09-11T22:52:14.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ad7fa640 10 client.14994 handle_command_reply: tid=0 2024-09-11T22:52:14.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ce5aa740 1 client.14994 shutdown 2024-09-11T22:52:14.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ce5aa740 1 -- 172.21.15.152:0/3499554153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399d9d0 msgr2=0x561de39c2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:14.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/3499554153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399d9d0 0x561de39c2eb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f10b400a9d0 tx=0x7f10b4006cd0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ce5aa740 1 -- 172.21.15.152:0/3499554153 shutdown_connections 2024-09-11T22:52:14.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/3499554153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561de39a8bb0 0x561de39bc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/3499554153 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x561de39b92b0 0x561de3a26e80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/3499554153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561de399e2f0 0x561de39bc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ce5aa740 1 --2- 172.21.15.152:0/3499554153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561de399d9d0 0x561de39c2eb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ad7fa640 0 client.14994 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:52:14.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ce5aa740 1 -- 172.21.15.152:0/3499554153 >> 172.21.15.152:0/3499554153 conn(0x561de3999f60 msgr2=0x561de3998de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:14.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.255+0000 7f10ce5aa740 1 -- 172.21.15.152:0/3499554153 shutdown_connections 2024-09-11T22:52:14.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.256+0000 7f10ce5aa740 1 -- 172.21.15.152:0/3499554153 wait complete. 2024-09-11T22:52:14.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.256+0000 7f10ce5aa740 20 client.14994 trim_cache size 0 max 16384 2024-09-11T22:52:14.267 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:52:14.267 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:52:14.268 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 4, 2024-09-11T22:52:14.268 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 14628, 2024-09-11T22:52:14.268 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-09-11T22:52:14.268 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-09-11T22:52:14.268 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-09-11T22:52:14.268 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 69.129500018000002, 2024-09-11T22:52:14.268 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 27, 2024-09-11T22:52:14.268 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 83, 2024-09-11T22:52:14.268 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T22:52:14.268 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 94.419257814999995 2024-09-11T22:52:14.268 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:52:14.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.256+0000 7f10cc1d1640 1 -- 172.21.15.152:0/1322876864 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f10b00a2c10 msgr2=0x7f10b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:14.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.256+0000 7f10cc1d1640 1 --2- 172.21.15.152:0/1322876864 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f10b00a2c10 0x7f10b00a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f10bc009bf0 tx=0x7f10bc02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.256+0000 7f10cc1d1640 1 -- 172.21.15.152:0/1322876864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10c4100a50 msgr2=0x7f10c41a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:14.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.256+0000 7f10cc1d1640 1 --2- 172.21.15.152:0/1322876864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10c4100a50 0x7f10c41a4c50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f10c000c890 tx=0x7f10c000cd50 comp rx=0 tx=0).stop 2024-09-11T22:52:14.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.256+0000 7f10cc1d1640 1 -- 172.21.15.152:0/1322876864 shutdown_connections 2024-09-11T22:52:14.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.256+0000 7f10cc1d1640 1 --2- 172.21.15.152:0/1322876864 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f10b00a2c10 0x7f10b00a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.256+0000 7f10cc1d1640 1 --2- 172.21.15.152:0/1322876864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10c4106b70 0x7f10c419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.256+0000 7f10cc1d1640 1 --2- 172.21.15.152:0/1322876864 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f10c4100a50 0x7f10c41a4c50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.256+0000 7f10cc1d1640 1 --2- 172.21.15.152:0/1322876864 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f10c4100090 0x7f10c41a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:14.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.256+0000 7f10cc1d1640 1 -- 172.21.15.152:0/1322876864 >> 172.21.15.152:0/1322876864 conn(0x7f10c4077210 msgr2=0x7f10c41037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:14.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.256+0000 7f10cc1d1640 1 -- 172.21.15.152:0/1322876864 shutdown_connections 2024-09-11T22:52:14.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:14.256+0000 7f10cc1d1640 1 -- 172.21.15.152:0/1322876864 wait complete. 2024-09-11T22:52:14.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:13 smithi191 ceph-mon[35211]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:14.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:13 smithi204 ceph-mon[35454]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-09-11T22:52:15.044 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T22:52:15.292 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:15.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.840+0000 7fa525db5640 1 -- 172.21.15.152:0/244773463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa520102790 msgr2=0x7fa5201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:15.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.840+0000 7fa525db5640 1 --2- 172.21.15.152:0/244773463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa520102790 0x7fa5201067e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa514009900 tx=0x7fa51402a410 comp rx=0 tx=0).stop 2024-09-11T22:52:15.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.840+0000 7fa525db5640 1 -- 172.21.15.152:0/244773463 shutdown_connections 2024-09-11T22:52:15.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.840+0000 7fa525db5640 1 --2- 172.21.15.152:0/244773463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa520106f10 0x7fa52010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.840+0000 7fa525db5640 1 --2- 172.21.15.152:0/244773463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa520102790 0x7fa5201067e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.840+0000 7fa525db5640 1 --2- 172.21.15.152:0/244773463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa520101de0 0x7fa5201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.840+0000 7fa525db5640 1 -- 172.21.15.152:0/244773463 >> 172.21.15.152:0/244773463 conn(0x7fa5200fd8c0 msgr2=0x7fa5200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:15.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.840+0000 7fa525db5640 1 -- 172.21.15.152:0/244773463 shutdown_connections 2024-09-11T22:52:15.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.840+0000 7fa525db5640 1 -- 172.21.15.152:0/244773463 wait complete. 2024-09-11T22:52:15.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.840+0000 7fa525db5640 1 Processor -- start 2024-09-11T22:52:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa525db5640 1 -- start start 2024-09-11T22:52:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa525db5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa520101de0 0x7fa5200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa525db5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa520102790 0x7fa520074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa525db5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa520106f10 0x7fa520074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa525db5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa520111f70 con 0x7fa520106f10 2024-09-11T22:52:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa525db5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa520111df0 con 0x7fa520101de0 2024-09-11T22:52:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa525db5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201120f0 con 0x7fa520102790 2024-09-11T22:52:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa51ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa520106f10 0x7fa520074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa51ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa520106f10 0x7fa520074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39862/0 (socket says 172.21.15.152:39862) 2024-09-11T22:52:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa51ffff640 1 -- 172.21.15.152:0/2610434953 learned_addr learned my addr 172.21.15.152:0/2610434953 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa51f7fe640 1 --2- 172.21.15.152:0/2610434953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa520101de0 0x7fa5200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa51effd640 1 --2- 172.21.15.152:0/2610434953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa520102790 0x7fa520074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa51ffff640 1 -- 172.21.15.152:0/2610434953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa520102790 msgr2=0x7fa520074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:15.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa51ffff640 1 --2- 172.21.15.152:0/2610434953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa520102790 0x7fa520074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa51ffff640 1 -- 172.21.15.152:0/2610434953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa520101de0 msgr2=0x7fa5200744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:15.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa51ffff640 1 --2- 172.21.15.152:0/2610434953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa520101de0 0x7fa5200744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa51ffff640 1 -- 172.21.15.152:0/2610434953 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5201a68d0 con 0x7fa520106f10 2024-09-11T22:52:15.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.841+0000 7fa51ffff640 1 --2- 172.21.15.152:0/2610434953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa520106f10 0x7fa520074f70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa51800cce0 tx=0x7fa51800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:15.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.842+0000 7fa51cff9640 1 -- 172.21.15.152:0/2610434953 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51800f450 con 0x7fa520106f10 2024-09-11T22:52:15.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.842+0000 7fa525db5640 1 -- 172.21.15.152:0/2610434953 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5201a6bc0 con 0x7fa520106f10 2024-09-11T22:52:15.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.842+0000 7fa51cff9640 1 -- 172.21.15.152:0/2610434953 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa51800fad0 con 0x7fa520106f10 2024-09-11T22:52:15.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.842+0000 7fa51cff9640 1 -- 172.21.15.152:0/2610434953 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518018c10 con 0x7fa520106f10 2024-09-11T22:52:15.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.842+0000 7fa525db5640 1 -- 172.21.15.152:0/2610434953 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5201a7100 con 0x7fa520106f10 2024-09-11T22:52:15.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.844+0000 7fa51cff9640 1 -- 172.21.15.152:0/2610434953 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa51800f5f0 con 0x7fa520106f10 2024-09-11T22:52:15.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.844+0000 7fa52818e740 1 Processor -- start 2024-09-11T22:52:15.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.844+0000 7fa52818e740 1 -- start start 2024-09-11T22:52:15.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.844+0000 7fa51cff9640 1 --2- 172.21.15.152:0/2610434953 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa5080a2c10 0x7fa5080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.844+0000 7fa52818e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f339d0 0x5593a7f33db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.844+0000 7fa52818e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f342f0 0x5593a7f3e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.844+0000 7fa52818e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f3ebb0 0x5593a7f40fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.844+0000 7fa52818e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5593a7e9a090 con 0x5593a7f342f0 2024-09-11T22:52:15.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.844+0000 7fa52818e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5593a7e41a70 con 0x5593a7f3ebb0 2024-09-11T22:52:15.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.844+0000 7fa52818e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5593a7e96050 con 0x5593a7f339d0 2024-09-11T22:52:15.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa51effd640 1 --2- 172.21.15.152:0/2610434953 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa5080a2c10 0x7fa5080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa51effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f342f0 0x5593a7f3e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa51cff9640 1 -- 172.21.15.152:0/2610434953 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fa5180cf820 con 0x7fa520106f10 2024-09-11T22:52:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa51effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f342f0 0x5593a7f3e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39866/0 (socket says 172.21.15.152:39866) 2024-09-11T22:52:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa51effd640 1 -- 172.21.15.152:0/1107469459 learned_addr learned my addr 172.21.15.152:0/1107469459 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa51ffff640 1 --2- 172.21.15.152:0/1107469459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f3ebb0 0x5593a7f40fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa51f7fe640 1 --2- 172.21.15.152:0/1107469459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f339d0 0x5593a7f33db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa51effd640 1 -- 172.21.15.152:0/1107469459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f339d0 msgr2=0x5593a7f33db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa51effd640 1 --2- 172.21.15.152:0/1107469459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f339d0 0x5593a7f33db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa51effd640 1 -- 172.21.15.152:0/1107469459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f3ebb0 msgr2=0x5593a7f40fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa51effd640 1 --2- 172.21.15.152:0/1107469459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f3ebb0 0x5593a7f40fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa51effd640 1 -- 172.21.15.152:0/1107469459 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593a7e4e130 con 0x5593a7f342f0 2024-09-11T22:52:15.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa51ffff640 1 --2- 172.21.15.152:0/1107469459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f3ebb0 0x5593a7f40fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:15.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa51f7fe640 1 --2- 172.21.15.152:0/1107469459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f339d0 0x5593a7f33db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:15.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa51effd640 1 --2- 172.21.15.152:0/1107469459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f342f0 0x5593a7f3e670 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa51402a930 tx=0x7fa514004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea119bd5abd90e72 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:15.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa51effd640 1 --2- 172.21.15.152:0/2610434953 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa5080a2c10 0x7fa5080a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa514005ab0 tx=0x7fa514040000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:15.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa505ffb640 1 -- 172.21.15.152:0/1107469459 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514035040 con 0x5593a7f342f0 2024-09-11T22:52:15.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa505ffb640 1 -- 172.21.15.152:0/1107469459 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa51404d070 con 0x5593a7f342f0 2024-09-11T22:52:15.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.845+0000 7fa505ffb640 1 -- 172.21.15.152:0/1107469459 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514004b70 con 0x5593a7f342f0 2024-09-11T22:52:15.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.846+0000 7fa52818e740 1 -- 172.21.15.152:0/1107469459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f342f0 msgr2=0x5593a7f3e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:15.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.846+0000 7fa52818e740 1 --2- 172.21.15.152:0/1107469459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f342f0 0x5593a7f3e670 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa51402a930 tx=0x7fa514004040 comp rx=0 tx=0).stop 2024-09-11T22:52:15.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.846+0000 7fa52818e740 1 -- 172.21.15.152:0/1107469459 shutdown_connections 2024-09-11T22:52:15.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.846+0000 7fa52818e740 1 --2- 172.21.15.152:0/1107469459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f3ebb0 0x5593a7f40fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.846+0000 7fa52818e740 1 --2- 172.21.15.152:0/1107469459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f342f0 0x5593a7f3e670 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.846+0000 7fa52818e740 1 --2- 172.21.15.152:0/1107469459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f339d0 0x5593a7f33db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.846+0000 7fa52818e740 1 -- 172.21.15.152:0/1107469459 >> 172.21.15.152:0/1107469459 conn(0x5593a7f2ff60 msgr2=0x5593a7f30340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.846+0000 7fa52818e740 1 -- 172.21.15.152:0/1107469459 shutdown_connections 2024-09-11T22:52:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.846+0000 7fa52818e740 1 -- 172.21.15.152:0/1107469459 wait complete. 2024-09-11T22:52:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.846+0000 7fa52818e740 1 -- start start 2024-09-11T22:52:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa52818e740 1 Processor -- start 2024-09-11T22:52:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa52818e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f339d0 0x5593a7f3e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa52818e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f342f0 0x5593a7f3b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa52818e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f3ebb0 0x5593a7f3bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa52818e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5593a7f41780 con 0x5593a7f342f0 2024-09-11T22:52:15.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa52818e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5593a7e54f20 con 0x5593a7f339d0 2024-09-11T22:52:15.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa52818e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5593a7f41900 con 0x5593a7f3ebb0 2024-09-11T22:52:15.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa52818e740 10 client.0 _ll_register_callbacks cb 0x5593a7e1e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:15.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa51effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f342f0 0x5593a7f3b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa51effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f342f0 0x5593a7f3b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39876/0 (socket says 172.21.15.152:39876) 2024-09-11T22:52:15.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa51effd640 1 -- 172.21.15.152:0/3887760658 learned_addr learned my addr 172.21.15.152:0/3887760658 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:15.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa51f7fe640 1 --2- 172.21.15.152:0/3887760658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f339d0 0x5593a7f3e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa51ffff640 1 --2- 172.21.15.152:0/3887760658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f3ebb0 0x5593a7f3bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa51effd640 1 -- 172.21.15.152:0/3887760658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f3ebb0 msgr2=0x5593a7f3bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:15.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa51effd640 1 --2- 172.21.15.152:0/3887760658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f3ebb0 0x5593a7f3bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa51effd640 1 -- 172.21.15.152:0/3887760658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f339d0 msgr2=0x5593a7f3e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa51effd640 1 --2- 172.21.15.152:0/3887760658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f339d0 0x5593a7f3e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.847+0000 7fa51effd640 1 -- 172.21.15.152:0/3887760658 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593a7c0b2b0 con 0x5593a7f342f0 2024-09-11T22:52:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.848+0000 7fa51f7fe640 1 --2- 172.21.15.152:0/3887760658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f339d0 0x5593a7f3e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:52:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.848+0000 7fa51effd640 1 --2- 172.21.15.152:0/3887760658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f342f0 0x5593a7f3b830 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa514004770 tx=0x7fa5140057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadca4104bbf5fab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.848+0000 7fa4feffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.850+0000 7fa4feffd640 1 -- 172.21.15.152:0/3887760658 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514056070 con 0x5593a7f342f0 2024-09-11T22:52:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.850+0000 7fa4feffd640 1 -- 172.21.15.152:0/3887760658 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fa514003a00 con 0x5593a7f342f0 2024-09-11T22:52:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.850+0000 7fa52818e740 1 -- 172.21.15.152:0/3887760658 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa51404d070 con 0x5593a7f342f0 2024-09-11T22:52:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.850+0000 7fa4feffd640 1 -- 172.21.15.152:0/3887760658 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa514062050 con 0x5593a7f342f0 2024-09-11T22:52:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.850+0000 7fa52818e740 10 client.15006 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:15.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.850+0000 7fa52818e740 1 -- 172.21.15.152:0/3887760658 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5593a7f4d130 con 0x5593a7f342f0 2024-09-11T22:52:15.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.850+0000 7fa4feffd640 1 -- 172.21.15.152:0/3887760658 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa51404d070 con 0x5593a7f342f0 2024-09-11T22:52:15.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.851+0000 7fa4feffd640 1 -- 172.21.15.152:0/3887760658 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa51403bc40 con 0x5593a7f342f0 2024-09-11T22:52:15.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.851+0000 7fa52818e740 10 client.15006 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:15.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.851+0000 7fa52818e740 10 client.15006 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T22:52:15.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.851+0000 7fa52818e740 20 client.15006 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:15.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.851+0000 7fa52818e740 1 --2- 172.21.15.152:0/3887760658 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5593a7f4f320 0x5593a7f50790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.851+0000 7fa52818e740 4 client.15006 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:52:15.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.851+0000 7fa52818e740 1 -- 172.21.15.152:0/3887760658 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593a7ea7600 con 0x5593a7f4f320 2024-09-11T22:52:15.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.851+0000 7fa51f7fe640 1 --2- 172.21.15.152:0/3887760658 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5593a7f4f320 0x5593a7f50790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.851+0000 7fa51f7fe640 1 --2- 172.21.15.152:0/3887760658 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5593a7f4f320 0x5593a7f50790 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f339a50b4e226373 server_cookie=113071614205fdf8 in_seq=0 out_seq=0 2024-09-11T22:52:15.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.851+0000 7fa4feffd640 10 client.15006 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:52:15.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.852+0000 7fa4feffd640 1 -- 172.21.15.152:0/3887760658 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5593a7ea7600 con 0x5593a7f4f320 2024-09-11T22:52:15.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.852+0000 7fa4feffd640 10 client.15006 handle_command_reply: tid=0 2024-09-11T22:52:15.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.852+0000 7fa52818e740 1 client.15006 shutdown 2024-09-11T22:52:15.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.852+0000 7fa52818e740 1 -- 172.21.15.152:0/3887760658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f342f0 msgr2=0x5593a7f3b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:15.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.852+0000 7fa52818e740 1 --2- 172.21.15.152:0/3887760658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f342f0 0x5593a7f3b830 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa514004770 tx=0x7fa5140057b0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.852+0000 7fa52818e740 1 -- 172.21.15.152:0/3887760658 shutdown_connections 2024-09-11T22:52:15.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.852+0000 7fa52818e740 1 --2- 172.21.15.152:0/3887760658 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5593a7f4f320 0x5593a7f50790 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.852+0000 7fa52818e740 1 --2- 172.21.15.152:0/3887760658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f3ebb0 0x5593a7f3bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.852+0000 7fa52818e740 1 --2- 172.21.15.152:0/3887760658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f342f0 0x5593a7f3b830 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.852+0000 7fa4feffd640 0 client.15006 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:52:15.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.852+0000 7fa52818e740 1 --2- 172.21.15.152:0/3887760658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f339d0 0x5593a7f3e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.852+0000 7fa52818e740 1 -- 172.21.15.152:0/3887760658 >> 172.21.15.152:0/3887760658 conn(0x5593a7f2ff60 msgr2=0x5593a7f35c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:15.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.852+0000 7fa52818e740 1 -- 172.21.15.152:0/3887760658 shutdown_connections 2024-09-11T22:52:15.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.852+0000 7fa52818e740 1 -- 172.21.15.152:0/3887760658 wait complete. 2024-09-11T22:52:15.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.853+0000 7fa52818e740 20 client.15006 trim_cache size 0 max 16384 2024-09-11T22:52:15.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.870+0000 7fa52818e740 1 Processor -- start 2024-09-11T22:52:15.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.870+0000 7fa52818e740 1 -- start start 2024-09-11T22:52:15.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.870+0000 7fa52818e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f339d0 0x5593a7f4ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.870+0000 7fa52818e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f342f0 0x5593a7f35b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.870+0000 7fa52818e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f3ebb0 0x5593a7f38ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.870+0000 7fa52818e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5593a7f4ead0 con 0x5593a7f3ebb0 2024-09-11T22:52:15.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.870+0000 7fa52818e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5593a7f4f1a0 con 0x5593a7f339d0 2024-09-11T22:52:15.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.870+0000 7fa52818e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5593a7f4f870 con 0x5593a7f342f0 2024-09-11T22:52:15.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.870+0000 7fa51ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f3ebb0 0x5593a7f38ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.870+0000 7fa51ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f3ebb0 0x5593a7f38ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39890/0 (socket says 172.21.15.152:39890) 2024-09-11T22:52:15.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.870+0000 7fa51ffff640 1 -- 172.21.15.152:0/1203496669 learned_addr learned my addr 172.21.15.152:0/1203496669 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:15.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.870+0000 7fa51effd640 1 --2- 172.21.15.152:0/1203496669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f342f0 0x5593a7f35b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.870+0000 7fa51f7fe640 1 --2- 172.21.15.152:0/1203496669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f339d0 0x5593a7f4ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.871+0000 7fa51ffff640 1 -- 172.21.15.152:0/1203496669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f342f0 msgr2=0x5593a7f35b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:15.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.871+0000 7fa51ffff640 1 --2- 172.21.15.152:0/1203496669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f342f0 0x5593a7f35b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.871+0000 7fa51ffff640 1 -- 172.21.15.152:0/1203496669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f339d0 msgr2=0x5593a7f4ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:15.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.871+0000 7fa51ffff640 1 --2- 172.21.15.152:0/1203496669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f339d0 0x5593a7f4ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.871+0000 7fa51ffff640 1 -- 172.21.15.152:0/1203496669 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593a7f34120 con 0x5593a7f3ebb0 2024-09-11T22:52:15.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.871+0000 7fa51effd640 1 --2- 172.21.15.152:0/1203496669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f342f0 0x5593a7f35b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:15.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.871+0000 7fa51ffff640 1 --2- 172.21.15.152:0/1203496669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f3ebb0 0x5593a7f38ab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa51809bee0 tx=0x7fa518018510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412d2bf1748c501d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:15.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.871+0000 7fa5057fa640 1 -- 172.21.15.152:0/1203496669 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51809c650 con 0x5593a7f3ebb0 2024-09-11T22:52:15.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.871+0000 7fa5057fa640 1 -- 172.21.15.152:0/1203496669 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa51800d340 con 0x5593a7f3ebb0 2024-09-11T22:52:15.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.871+0000 7fa5057fa640 1 -- 172.21.15.152:0/1203496669 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51800f8e0 con 0x5593a7f3ebb0 2024-09-11T22:52:15.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 -- 172.21.15.152:0/1203496669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f3ebb0 msgr2=0x5593a7f38ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:15.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 --2- 172.21.15.152:0/1203496669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f3ebb0 0x5593a7f38ab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa51809bee0 tx=0x7fa518018510 comp rx=0 tx=0).stop 2024-09-11T22:52:15.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 -- 172.21.15.152:0/1203496669 shutdown_connections 2024-09-11T22:52:15.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 --2- 172.21.15.152:0/1203496669 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f3ebb0 0x5593a7f38ab0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 --2- 172.21.15.152:0/1203496669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f342f0 0x5593a7f35b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 --2- 172.21.15.152:0/1203496669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f339d0 0x5593a7f4ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 -- 172.21.15.152:0/1203496669 >> 172.21.15.152:0/1203496669 conn(0x5593a7f2ff60 msgr2=0x5593a7f35420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:15.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 -- 172.21.15.152:0/1203496669 shutdown_connections 2024-09-11T22:52:15.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 -- 172.21.15.152:0/1203496669 wait complete. 2024-09-11T22:52:15.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 -- start start 2024-09-11T22:52:15.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 Processor -- start 2024-09-11T22:52:15.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f339d0 0x5593a7f59220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f342f0 0x5593a7f38650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f3ebb0 0x5593a7f38c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5593a7f50610 con 0x5593a7f339d0 2024-09-11T22:52:15.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5593a7f4ff40 con 0x5593a7f3ebb0 2024-09-11T22:52:15.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.872+0000 7fa52818e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5593a7f50ce0 con 0x5593a7f342f0 2024-09-11T22:52:15.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa52818e740 10 client.0 _ll_register_callbacks cb 0x7fa5140319d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:15.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f339d0 0x5593a7f59220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa51f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f339d0 0x5593a7f59220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39902/0 (socket says 172.21.15.152:39902) 2024-09-11T22:52:15.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa51f7fe640 1 -- 172.21.15.152:0/4110373155 learned_addr learned my addr 172.21.15.152:0/4110373155 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:15.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa51ffff640 1 --2- 172.21.15.152:0/4110373155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f3ebb0 0x5593a7f38c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa51effd640 1 --2- 172.21.15.152:0/4110373155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f342f0 0x5593a7f38650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa51f7fe640 1 -- 172.21.15.152:0/4110373155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f342f0 msgr2=0x5593a7f38650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:15.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa51f7fe640 1 --2- 172.21.15.152:0/4110373155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f342f0 0x5593a7f38650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa51f7fe640 1 -- 172.21.15.152:0/4110373155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f3ebb0 msgr2=0x5593a7f38c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:15.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa51f7fe640 1 --2- 172.21.15.152:0/4110373155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f3ebb0 0x5593a7f38c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa51f7fe640 1 -- 172.21.15.152:0/4110373155 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593a7f39460 con 0x5593a7f339d0 2024-09-11T22:52:15.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa51ffff640 1 --2- 172.21.15.152:0/4110373155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f3ebb0 0x5593a7f38c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:15.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa51f7fe640 1 --2- 172.21.15.152:0/4110373155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f339d0 0x5593a7f59220 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa510004700 tx=0x7fa51000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5252df9a3701ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:15.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa4feffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:15.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa4feffd640 1 -- 172.21.15.152:0/4110373155 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510005030 con 0x5593a7f339d0 2024-09-11T22:52:15.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa4feffd640 1 -- 172.21.15.152:0/4110373155 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fa5100051d0 con 0x5593a7f339d0 2024-09-11T22:52:15.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa52818e740 1 -- 172.21.15.152:0/4110373155 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa51800d340 con 0x5593a7f339d0 2024-09-11T22:52:15.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa4feffd640 1 -- 172.21.15.152:0/4110373155 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5100053d0 con 0x5593a7f339d0 2024-09-11T22:52:15.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa52818e740 10 client.15012 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:15.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.873+0000 7fa52818e740 1 -- 172.21.15.152:0/4110373155 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5593a7fb9b80 con 0x5593a7f339d0 2024-09-11T22:52:15.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.874+0000 7fa4feffd640 1 -- 172.21.15.152:0/4110373155 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa51800d340 con 0x5593a7f339d0 2024-09-11T22:52:15.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.874+0000 7fa4feffd640 1 -- 172.21.15.152:0/4110373155 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa51001d710 con 0x5593a7f339d0 2024-09-11T22:52:15.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.874+0000 7fa52818e740 10 client.15012 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:15.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.874+0000 7fa52818e740 10 client.15012 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T22:52:15.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.874+0000 7fa52818e740 20 client.15012 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:15.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.874+0000 7fa52818e740 1 --2- 172.21.15.152:0/4110373155 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5593a7f4f320 0x5593a7fbd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:15.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.874+0000 7fa52818e740 4 client.15012 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:52:15.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.874+0000 7fa52818e740 1 -- 172.21.15.152:0/4110373155 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "status"}) -- 0x5593a7ea1b10 con 0x5593a7f4f320 2024-09-11T22:52:15.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.874+0000 7fa51effd640 1 --2- 172.21.15.152:0/4110373155 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5593a7f4f320 0x5593a7fbd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:15.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.874+0000 7fa51effd640 1 --2- 172.21.15.152:0/4110373155 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5593a7f4f320 0x5593a7fbd4c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d894a1bd29be27ed server_cookie=7848f39099bb1ce5 in_seq=0 out_seq=0 2024-09-11T22:52:15.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.874+0000 7fa4feffd640 10 client.15012 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:52:15.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.875+0000 7fa4feffd640 1 -- 172.21.15.152:0/4110373155 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5593a7ea1b10 con 0x5593a7f4f320 2024-09-11T22:52:15.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.875+0000 7fa4feffd640 10 client.15012 handle_command_reply: tid=0 2024-09-11T22:52:15.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.875+0000 7fa52818e740 1 client.15012 shutdown 2024-09-11T22:52:15.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.875+0000 7fa52818e740 1 -- 172.21.15.152:0/4110373155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f339d0 msgr2=0x5593a7f59220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:15.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.875+0000 7fa52818e740 1 --2- 172.21.15.152:0/4110373155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f339d0 0x5593a7f59220 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa510004700 tx=0x7fa51000bf10 comp rx=0 tx=0).stop 2024-09-11T22:52:15.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.875+0000 7fa52818e740 1 -- 172.21.15.152:0/4110373155 shutdown_connections 2024-09-11T22:52:15.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.875+0000 7fa52818e740 1 --2- 172.21.15.152:0/4110373155 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5593a7f4f320 0x5593a7fbd4c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.875+0000 7fa52818e740 1 --2- 172.21.15.152:0/4110373155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5593a7f3ebb0 0x5593a7f38c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.875+0000 7fa52818e740 1 --2- 172.21.15.152:0/4110373155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5593a7f342f0 0x5593a7f38650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.875+0000 7fa52818e740 1 --2- 172.21.15.152:0/4110373155 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5593a7f339d0 0x5593a7f59220 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.875+0000 7fa4feffd640 0 client.15012 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:52:15.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.875+0000 7fa52818e740 1 -- 172.21.15.152:0/4110373155 >> 172.21.15.152:0/4110373155 conn(0x5593a7f2ff60 msgr2=0x5593a7f35c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:15.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.875+0000 7fa52818e740 1 -- 172.21.15.152:0/4110373155 shutdown_connections 2024-09-11T22:52:15.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.875+0000 7fa52818e740 1 -- 172.21.15.152:0/4110373155 wait complete. 2024-09-11T22:52:15.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.875+0000 7fa52818e740 20 client.15012 trim_cache size 0 max 16384 2024-09-11T22:52:15.887 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:52:15.887 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:52:15.887 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 3, 2024-09-11T22:52:15.887 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 24430, 2024-09-11T22:52:15.887 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-09-11T22:52:15.887 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-09-11T22:52:15.888 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-09-11T22:52:15.888 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 71.654072710999998, 2024-09-11T22:52:15.888 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 27, 2024-09-11T22:52:15.888 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 83, 2024-09-11T22:52:15.888 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T22:52:15.888 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 107.529918278 2024-09-11T22:52:15.888 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:52:15.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.876+0000 7fa525db5640 1 -- 172.21.15.152:0/2610434953 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa5080a2c10 msgr2=0x7fa5080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:15.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.876+0000 7fa525db5640 1 --2- 172.21.15.152:0/2610434953 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa5080a2c10 0x7fa5080a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa514005ab0 tx=0x7fa514040000 comp rx=0 tx=0).stop 2024-09-11T22:52:15.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.876+0000 7fa525db5640 1 -- 172.21.15.152:0/2610434953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa520106f10 msgr2=0x7fa520074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:15.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.876+0000 7fa525db5640 1 --2- 172.21.15.152:0/2610434953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa520106f10 0x7fa520074f70 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa51800cce0 tx=0x7fa51800a620 comp rx=0 tx=0).stop 2024-09-11T22:52:15.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.876+0000 7fa525db5640 1 -- 172.21.15.152:0/2610434953 shutdown_connections 2024-09-11T22:52:15.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.876+0000 7fa525db5640 1 --2- 172.21.15.152:0/2610434953 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa5080a2c10 0x7fa5080a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.876+0000 7fa525db5640 1 --2- 172.21.15.152:0/2610434953 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa520106f10 0x7fa520074f70 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.876+0000 7fa525db5640 1 --2- 172.21.15.152:0/2610434953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa520102790 0x7fa520074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.876+0000 7fa525db5640 1 --2- 172.21.15.152:0/2610434953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa520101de0 0x7fa5200744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:15.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.876+0000 7fa525db5640 1 -- 172.21.15.152:0/2610434953 >> 172.21.15.152:0/2610434953 conn(0x7fa5200fd8c0 msgr2=0x7fa5200febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:15.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.876+0000 7fa525db5640 1 -- 172.21.15.152:0/2610434953 shutdown_connections 2024-09-11T22:52:15.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:15.876+0000 7fa525db5640 1 -- 172.21.15.152:0/2610434953 wait complete. 2024-09-11T22:52:16.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:15 smithi152 ceph-mon[31181]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:52:16.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:15 smithi191 ceph-mon[35211]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:52:16.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:15 smithi204 ceph-mon[35454]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:52:16.627 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T22:52:16.876 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:17.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.434+0000 7f045d289640 1 -- 172.21.15.152:0/11104028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0458100a60 msgr2=0x7f0458100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:17.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.434+0000 7f045d289640 1 --2- 172.21.15.152:0/11104028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0458100a60 0x7f0458100ec0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0440009930 tx=0x7f044002a420 comp rx=0 tx=0).stop 2024-09-11T22:52:17.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.434+0000 7f045d289640 1 -- 172.21.15.152:0/11104028 shutdown_connections 2024-09-11T22:52:17.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.434+0000 7f045d289640 1 --2- 172.21.15.152:0/11104028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0458101400 0x7f045810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.434+0000 7f045d289640 1 --2- 172.21.15.152:0/11104028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0458100a60 0x7f0458100ec0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.434+0000 7f045d289640 1 --2- 172.21.15.152:0/11104028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0458106a60 0x7f0458106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.434+0000 7f045d289640 1 -- 172.21.15.152:0/11104028 >> 172.21.15.152:0/11104028 conn(0x7f04580fc780 msgr2=0x7f04580feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:17.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.434+0000 7f045d289640 1 -- 172.21.15.152:0/11104028 shutdown_connections 2024-09-11T22:52:17.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.434+0000 7f045d289640 1 -- 172.21.15.152:0/11104028 wait complete. 2024-09-11T22:52:17.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f045d289640 1 Processor -- start 2024-09-11T22:52:17.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f045d289640 1 -- start start 2024-09-11T22:52:17.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f045d289640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0458100a60 0x7f0458102630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f045d289640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0458101400 0x7f0458102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f045d289640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0458106a60 0x7f04581030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f045d289640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0458110e80 con 0x7f0458106a60 2024-09-11T22:52:17.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f045d289640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0458110d00 con 0x7f0458100a60 2024-09-11T22:52:17.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f045d289640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0458111000 con 0x7f0458101400 2024-09-11T22:52:17.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f04577fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0458106a60 0x7f04581030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f04577fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0458106a60 0x7f04581030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39924/0 (socket says 172.21.15.152:39924) 2024-09-11T22:52:17.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f04577fe640 1 -- 172.21.15.152:0/2455415685 learned_addr learned my addr 172.21.15.152:0/2455415685 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:17.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f0456ffd640 1 --2- 172.21.15.152:0/2455415685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0458100a60 0x7f0458102630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f044ffff640 1 --2- 172.21.15.152:0/2455415685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0458101400 0x7f0458102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f04577fe640 1 -- 172.21.15.152:0/2455415685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0458101400 msgr2=0x7f0458102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:17.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f04577fe640 1 --2- 172.21.15.152:0/2455415685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0458101400 0x7f0458102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f04577fe640 1 -- 172.21.15.152:0/2455415685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0458100a60 msgr2=0x7f0458102630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:17.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f04577fe640 1 --2- 172.21.15.152:0/2455415685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0458100a60 0x7f0458102630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.435+0000 7f04577fe640 1 -- 172.21.15.152:0/2455415685 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04581a6990 con 0x7f0458106a60 2024-09-11T22:52:17.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.436+0000 7f044ffff640 1 --2- 172.21.15.152:0/2455415685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0458101400 0x7f0458102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:52:17.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.436+0000 7f04577fe640 1 --2- 172.21.15.152:0/2455415685 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0458106a60 0x7f04581030b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f045000b970 tx=0x7f045000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:17.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.436+0000 7f0454ff9640 1 -- 172.21.15.152:0/2455415685 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450019070 con 0x7f0458106a60 2024-09-11T22:52:17.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.436+0000 7f0454ff9640 1 -- 172.21.15.152:0/2455415685 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f045000cb40 con 0x7f0458106a60 2024-09-11T22:52:17.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.436+0000 7f045d289640 1 -- 172.21.15.152:0/2455415685 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04581a6c80 con 0x7f0458106a60 2024-09-11T22:52:17.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.436+0000 7f0454ff9640 1 -- 172.21.15.152:0/2455415685 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450007650 con 0x7f0458106a60 2024-09-11T22:52:17.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.436+0000 7f045d289640 1 -- 172.21.15.152:0/2455415685 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04581a71c0 con 0x7f0458106a60 2024-09-11T22:52:17.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.438+0000 7f0454ff9640 1 -- 172.21.15.152:0/2455415685 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f045001c070 con 0x7f0458106a60 2024-09-11T22:52:17.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.438+0000 7f045f662740 1 Processor -- start 2024-09-11T22:52:17.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f045f662740 1 -- start start 2024-09-11T22:52:17.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f045f662740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bed9d0 0x55aa41beddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f045f662740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bee2f0 0x55aa41bf8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f045f662740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bf8bb0 0x55aa41bfafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f045f662740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55aa41b54090 con 0x55aa41bf8bb0 2024-09-11T22:52:17.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f045f662740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55aa41afba70 con 0x55aa41bee2f0 2024-09-11T22:52:17.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f045f662740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aa41b50050 con 0x55aa41bed9d0 2024-09-11T22:52:17.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f044ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bf8bb0 0x55aa41bfafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f044ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bf8bb0 0x55aa41bfafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39930/0 (socket says 172.21.15.152:39930) 2024-09-11T22:52:17.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f044ffff640 1 -- 172.21.15.152:0/3543610381 learned_addr learned my addr 172.21.15.152:0/3543610381 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:17.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f044ffff640 1 --2- 172.21.15.152:0/3543610381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bed9d0 0x55aa41beddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f0456ffd640 1 --2- 172.21.15.152:0/3543610381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bee2f0 0x55aa41bf8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f0454ff9640 1 --2- 172.21.15.152:0/2455415685 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f04440a2c90 0x7f04440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f044ffff640 1 -- 172.21.15.152:0/3543610381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bed9d0 msgr2=0x55aa41beddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:17.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f044ffff640 1 --2- 172.21.15.152:0/3543610381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bed9d0 0x55aa41beddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f044ffff640 1 -- 172.21.15.152:0/3543610381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bee2f0 msgr2=0x55aa41bf8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:17.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f044ffff640 1 --2- 172.21.15.152:0/3543610381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bee2f0 0x55aa41bf8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f044ffff640 1 -- 172.21.15.152:0/3543610381 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa41b08130 con 0x55aa41bf8bb0 2024-09-11T22:52:17.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.439+0000 7f04577fe640 1 --2- 172.21.15.152:0/2455415685 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f04440a2c90 0x7f04440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.440+0000 7f0454ff9640 1 -- 172.21.15.152:0/2455415685 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f04500ce690 con 0x7f0458106a60 2024-09-11T22:52:17.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.440+0000 7f044ffff640 1 --2- 172.21.15.152:0/3543610381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bf8bb0 0x55aa41bfafa0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f04400057c0 tx=0x7f0440000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69cb8b993d4759fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:17.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.440+0000 7f044d7fa640 1 -- 172.21.15.152:0/3543610381 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0440035040 con 0x55aa41bf8bb0 2024-09-11T22:52:17.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.440+0000 7f044d7fa640 1 -- 172.21.15.152:0/3543610381 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04400035f0 con 0x55aa41bf8bb0 2024-09-11T22:52:17.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.440+0000 7f044d7fa640 1 -- 172.21.15.152:0/3543610381 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044003c650 con 0x55aa41bf8bb0 2024-09-11T22:52:17.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 -- 172.21.15.152:0/3543610381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bf8bb0 msgr2=0x55aa41bfafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:17.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f04577fe640 1 --2- 172.21.15.152:0/2455415685 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f04440a2c90 0x7f04440a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f045009ae60 tx=0x7f045006df90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:17.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 --2- 172.21.15.152:0/3543610381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bf8bb0 0x55aa41bfafa0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f04400057c0 tx=0x7f0440000bc0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 -- 172.21.15.152:0/3543610381 shutdown_connections 2024-09-11T22:52:17.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 --2- 172.21.15.152:0/3543610381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bf8bb0 0x55aa41bfafa0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 --2- 172.21.15.152:0/3543610381 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bee2f0 0x55aa41bf8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 --2- 172.21.15.152:0/3543610381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bed9d0 0x55aa41beddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 -- 172.21.15.152:0/3543610381 >> 172.21.15.152:0/3543610381 conn(0x55aa41be9f60 msgr2=0x55aa41bea340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:17.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 -- 172.21.15.152:0/3543610381 shutdown_connections 2024-09-11T22:52:17.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 -- 172.21.15.152:0/3543610381 wait complete. 2024-09-11T22:52:17.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 -- start start 2024-09-11T22:52:17.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 Processor -- start 2024-09-11T22:52:17.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bed9d0 0x55aa41bf83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bee2f0 0x55aa41bf5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bf8bb0 0x55aa41bf5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55aa41bfb780 con 0x55aa41bee2f0 2024-09-11T22:52:17.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55aa41b0ef20 con 0x55aa41bf8bb0 2024-09-11T22:52:17.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.441+0000 7f045f662740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aa41bfb900 con 0x55aa41bed9d0 2024-09-11T22:52:17.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f0456ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bee2f0 0x55aa41bf5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f045f662740 10 client.0 _ll_register_callbacks cb 0x55aa41ad8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:17.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f0456ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bee2f0 0x55aa41bf5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39940/0 (socket says 172.21.15.152:39940) 2024-09-11T22:52:17.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f0456ffd640 1 -- 172.21.15.152:0/2915232484 learned_addr learned my addr 172.21.15.152:0/2915232484 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:17.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f044ffff640 1 --2- 172.21.15.152:0/2915232484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bed9d0 0x55aa41bf83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f044ffff640 1 --2- 172.21.15.152:0/2915232484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bf8bb0 0x55aa41bf5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f0456ffd640 1 -- 172.21.15.152:0/2915232484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bed9d0 msgr2=0x55aa41bf83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:17.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f0456ffd640 1 --2- 172.21.15.152:0/2915232484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bed9d0 0x55aa41bf83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f0456ffd640 1 -- 172.21.15.152:0/2915232484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bf8bb0 msgr2=0x55aa41bf5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:17.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f0456ffd640 1 --2- 172.21.15.152:0/2915232484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bf8bb0 0x55aa41bf5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f0456ffd640 1 -- 172.21.15.152:0/2915232484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa418c52b0 con 0x55aa41bee2f0 2024-09-11T22:52:17.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f044ffff640 1 --2- 172.21.15.152:0/2915232484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bed9d0 0x55aa41bf83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:17.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f0456ffd640 1 --2- 172.21.15.152:0/2915232484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bee2f0 0x55aa41bf5740 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0448009e00 tx=0x7f04480117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9331ad34fba7cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:17.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f043e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:17.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f043e7fc640 1 -- 172.21.15.152:0/2915232484 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448012860 con 0x55aa41bee2f0 2024-09-11T22:52:17.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f043e7fc640 1 -- 172.21.15.152:0/2915232484 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f0448012a00 con 0x55aa41bee2f0 2024-09-11T22:52:17.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f045f662740 1 -- 172.21.15.152:0/2915232484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04400035f0 con 0x55aa41bee2f0 2024-09-11T22:52:17.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f043e7fc640 1 -- 172.21.15.152:0/2915232484 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0448012c00 con 0x55aa41bee2f0 2024-09-11T22:52:17.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f045f662740 10 client.15024 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:17.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.442+0000 7f045f662740 1 -- 172.21.15.152:0/2915232484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa41c070d0 con 0x55aa41bee2f0 2024-09-11T22:52:17.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.443+0000 7f043e7fc640 1 -- 172.21.15.152:0/2915232484 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04400035f0 con 0x55aa41bee2f0 2024-09-11T22:52:17.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.443+0000 7f043e7fc640 1 -- 172.21.15.152:0/2915232484 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f044800f040 con 0x55aa41bee2f0 2024-09-11T22:52:17.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.443+0000 7f045f662740 10 client.15024 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:17.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.443+0000 7f045f662740 10 client.15024 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T22:52:17.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.443+0000 7f045f662740 20 client.15024 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:17.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.443+0000 7f045f662740 1 --2- 172.21.15.152:0/2915232484 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55aa41c09750 0x55aa41c0abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.443+0000 7f045f662740 4 client.15024 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:52:17.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.443+0000 7f045f662740 1 -- 172.21.15.152:0/2915232484 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa41b61600 con 0x55aa41c09750 2024-09-11T22:52:17.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.443+0000 7f044ffff640 1 --2- 172.21.15.152:0/2915232484 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55aa41c09750 0x55aa41c0abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.444+0000 7f044ffff640 1 --2- 172.21.15.152:0/2915232484 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55aa41c09750 0x55aa41c0abc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db6ec95c1f13fbd0 server_cookie=c4f6e55446bba0ce in_seq=0 out_seq=0 2024-09-11T22:52:17.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.444+0000 7f043e7fc640 10 client.15024 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:52:17.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.444+0000 7f043e7fc640 1 -- 172.21.15.152:0/2915232484 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa41b61600 con 0x55aa41c09750 2024-09-11T22:52:17.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.444+0000 7f043e7fc640 10 client.15024 handle_command_reply: tid=0 2024-09-11T22:52:17.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.444+0000 7f045f662740 1 client.15024 shutdown 2024-09-11T22:52:17.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.444+0000 7f045f662740 1 -- 172.21.15.152:0/2915232484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bee2f0 msgr2=0x55aa41bf5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:17.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.444+0000 7f045f662740 1 --2- 172.21.15.152:0/2915232484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bee2f0 0x55aa41bf5740 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0448009e00 tx=0x7f04480117b0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.444+0000 7f045f662740 1 -- 172.21.15.152:0/2915232484 shutdown_connections 2024-09-11T22:52:17.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.444+0000 7f045f662740 1 --2- 172.21.15.152:0/2915232484 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55aa41c09750 0x55aa41c0abc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.444+0000 7f045f662740 1 --2- 172.21.15.152:0/2915232484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bf8bb0 0x55aa41bf5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.444+0000 7f045f662740 1 --2- 172.21.15.152:0/2915232484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bee2f0 0x55aa41bf5740 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.444+0000 7f045f662740 1 --2- 172.21.15.152:0/2915232484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bed9d0 0x55aa41bf83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.444+0000 7f043e7fc640 0 client.15024 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:52:17.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.444+0000 7f045f662740 1 -- 172.21.15.152:0/2915232484 >> 172.21.15.152:0/2915232484 conn(0x55aa41be9f60 msgr2=0x55aa41befc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:17.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.445+0000 7f045f662740 1 -- 172.21.15.152:0/2915232484 shutdown_connections 2024-09-11T22:52:17.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.445+0000 7f045f662740 1 -- 172.21.15.152:0/2915232484 wait complete. 2024-09-11T22:52:17.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.445+0000 7f045f662740 20 client.15024 trim_cache size 0 max 16384 2024-09-11T22:52:17.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.462+0000 7f045f662740 1 Processor -- start 2024-09-11T22:52:17.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.462+0000 7f045f662740 1 -- start start 2024-09-11T22:52:17.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.462+0000 7f045f662740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bed9d0 0x55aa41c09750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.462+0000 7f045f662740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bee2f0 0x55aa41c0a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.462+0000 7f045f662740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bf8bb0 0x55aa41c09080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.462+0000 7f045f662740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55aa41c08f00 con 0x55aa41bed9d0 2024-09-11T22:52:17.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.462+0000 7f045f662740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55aa41c095d0 con 0x55aa41bf8bb0 2024-09-11T22:52:17.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.462+0000 7f045f662740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aa41c09ca0 con 0x55aa41bee2f0 2024-09-11T22:52:17.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.463+0000 7f044ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bed9d0 0x55aa41c09750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.463+0000 7f044ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bed9d0 0x55aa41c09750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39946/0 (socket says 172.21.15.152:39946) 2024-09-11T22:52:17.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.463+0000 7f044ffff640 1 -- 172.21.15.152:0/2746595473 learned_addr learned my addr 172.21.15.152:0/2746595473 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:17.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.463+0000 7f0456ffd640 1 --2- 172.21.15.152:0/2746595473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bee2f0 0x55aa41c0a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.463+0000 7f044ffff640 1 -- 172.21.15.152:0/2746595473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bee2f0 msgr2=0x55aa41c0a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:17.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.463+0000 7f044ffff640 1 --2- 172.21.15.152:0/2746595473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bee2f0 0x55aa41c0a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.463+0000 7f044ffff640 1 -- 172.21.15.152:0/2746595473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bf8bb0 msgr2=0x55aa41c09080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:17.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.463+0000 7f044ffff640 1 --2- 172.21.15.152:0/2746595473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bf8bb0 0x55aa41c09080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.463+0000 7f044ffff640 1 -- 172.21.15.152:0/2746595473 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa41bf8760 con 0x55aa41bed9d0 2024-09-11T22:52:17.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.463+0000 7f0456ffd640 1 --2- 172.21.15.152:0/2746595473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bee2f0 0x55aa41c0a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:52:17.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.463+0000 7f044ffff640 1 --2- 172.21.15.152:0/2746595473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bed9d0 0x55aa41c09750 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0440038000 tx=0x7f044002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972ed0dc286b9bdc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:17.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.463+0000 7f044cff9640 1 -- 172.21.15.152:0/2746595473 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044005a070 con 0x55aa41bed9d0 2024-09-11T22:52:17.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.463+0000 7f044cff9640 1 -- 172.21.15.152:0/2746595473 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0440003ad0 con 0x55aa41bed9d0 2024-09-11T22:52:17.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.463+0000 7f044cff9640 1 -- 172.21.15.152:0/2746595473 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0440035070 con 0x55aa41bed9d0 2024-09-11T22:52:17.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.464+0000 7f045f662740 1 -- 172.21.15.152:0/2746595473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bed9d0 msgr2=0x55aa41c09750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:17.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.464+0000 7f045f662740 1 --2- 172.21.15.152:0/2746595473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bed9d0 0x55aa41c09750 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0440038000 tx=0x7f044002a420 comp rx=0 tx=0).stop 2024-09-11T22:52:17.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.464+0000 7f045f662740 1 -- 172.21.15.152:0/2746595473 shutdown_connections 2024-09-11T22:52:17.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.464+0000 7f045f662740 1 --2- 172.21.15.152:0/2746595473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bf8bb0 0x55aa41c09080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.464+0000 7f045f662740 1 --2- 172.21.15.152:0/2746595473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bee2f0 0x55aa41c0a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.464+0000 7f045f662740 1 --2- 172.21.15.152:0/2746595473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bed9d0 0x55aa41c09750 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.464+0000 7f045f662740 1 -- 172.21.15.152:0/2746595473 >> 172.21.15.152:0/2746595473 conn(0x55aa41be9f60 msgr2=0x55aa41bea340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:17.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.464+0000 7f045f662740 1 -- 172.21.15.152:0/2746595473 shutdown_connections 2024-09-11T22:52:17.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.464+0000 7f045f662740 1 -- 172.21.15.152:0/2746595473 wait complete. 2024-09-11T22:52:17.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.464+0000 7f045f662740 1 -- start start 2024-09-11T22:52:17.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.464+0000 7f045f662740 1 Processor -- start 2024-09-11T22:52:17.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f045f662740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bed9d0 0x55aa41c0db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f045f662740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bee2f0 0x55aa41c0e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f045f662740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bf8bb0 0x55aa41c0e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f045f662740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55aa41c0aa40 con 0x55aa41bf8bb0 2024-09-11T22:52:17.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f045f662740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55aa41c0a370 con 0x55aa41bed9d0 2024-09-11T22:52:17.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f045f662740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55aa41c0b110 con 0x55aa41bee2f0 2024-09-11T22:52:17.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f045f662740 10 client.0 _ll_register_callbacks cb 0x7f0448009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:17.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f044ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bf8bb0 0x55aa41c0e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f044ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bf8bb0 0x55aa41c0e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39954/0 (socket says 172.21.15.152:39954) 2024-09-11T22:52:17.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f044ffff640 1 -- 172.21.15.152:0/56580679 learned_addr learned my addr 172.21.15.152:0/56580679 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:17.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f0456ffd640 1 --2- 172.21.15.152:0/56580679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bee2f0 0x55aa41c0e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f044ffff640 1 --2- 172.21.15.152:0/56580679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bed9d0 0x55aa41c0db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f044ffff640 1 -- 172.21.15.152:0/56580679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bee2f0 msgr2=0x55aa41c0e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:17.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f044ffff640 1 --2- 172.21.15.152:0/56580679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bee2f0 0x55aa41c0e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f044ffff640 1 -- 172.21.15.152:0/56580679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bed9d0 msgr2=0x55aa41c0db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:17.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f044ffff640 1 --2- 172.21.15.152:0/56580679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bed9d0 0x55aa41c0db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f044ffff640 1 -- 172.21.15.152:0/56580679 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa41c71e50 con 0x55aa41bf8bb0 2024-09-11T22:52:17.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f044ffff640 1 --2- 172.21.15.152:0/56580679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bf8bb0 0x55aa41c0e5a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0440058040 tx=0x7f0440004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc454ac2816981 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:17.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f043e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:17.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.465+0000 7f043e7fc640 1 -- 172.21.15.152:0/56580679 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0440035070 con 0x55aa41bf8bb0 2024-09-11T22:52:17.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.466+0000 7f043e7fc640 1 -- 172.21.15.152:0/56580679 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f04400035f0 con 0x55aa41bf8bb0 2024-09-11T22:52:17.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.466+0000 7f045f662740 1 -- 172.21.15.152:0/56580679 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0440003ad0 con 0x55aa41bf8bb0 2024-09-11T22:52:17.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.466+0000 7f043e7fc640 1 -- 172.21.15.152:0/56580679 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0440068020 con 0x55aa41bf8bb0 2024-09-11T22:52:17.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.466+0000 7f045f662740 10 client.15030 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:17.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.466+0000 7f045f662740 1 -- 172.21.15.152:0/56580679 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aa41c73060 con 0x55aa41bf8bb0 2024-09-11T22:52:17.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.466+0000 7f043e7fc640 1 -- 172.21.15.152:0/56580679 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0440003ad0 con 0x55aa41bf8bb0 2024-09-11T22:52:17.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.466+0000 7f043e7fc640 1 -- 172.21.15.152:0/56580679 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f044003a040 con 0x55aa41bf8bb0 2024-09-11T22:52:17.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.466+0000 7f045f662740 10 client.15030 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:17.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.466+0000 7f045f662740 10 client.15030 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T22:52:17.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.466+0000 7f045f662740 20 client.15030 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:17.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.466+0000 7f045f662740 1 --2- 172.21.15.152:0/56580679 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55aa41c774d0 0x55aa41c761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:17.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.466+0000 7f045f662740 4 client.15030 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:52:17.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.466+0000 7f045f662740 1 -- 172.21.15.152:0/56580679 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "status"}) -- 0x55aa41b5bb10 con 0x55aa41c774d0 2024-09-11T22:52:17.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.466+0000 7f0456ffd640 1 --2- 172.21.15.152:0/56580679 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55aa41c774d0 0x55aa41c761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:17.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f0456ffd640 1 --2- 172.21.15.152:0/56580679 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55aa41c774d0 0x55aa41c761e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4cb295ab8e710f server_cookie=764bc0eb75a00829 in_seq=0 out_seq=0 2024-09-11T22:52:17.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f043e7fc640 10 client.15030 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:52:17.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f043e7fc640 1 -- 172.21.15.152:0/56580679 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55aa41b5bb10 con 0x55aa41c774d0 2024-09-11T22:52:17.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f043e7fc640 10 client.15030 handle_command_reply: tid=0 2024-09-11T22:52:17.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f045f662740 1 client.15030 shutdown 2024-09-11T22:52:17.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f045f662740 1 -- 172.21.15.152:0/56580679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bf8bb0 msgr2=0x55aa41c0e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:17.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f045f662740 1 --2- 172.21.15.152:0/56580679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bf8bb0 0x55aa41c0e5a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0440058040 tx=0x7f0440004260 comp rx=0 tx=0).stop 2024-09-11T22:52:17.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f045f662740 1 -- 172.21.15.152:0/56580679 shutdown_connections 2024-09-11T22:52:17.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f045f662740 1 --2- 172.21.15.152:0/56580679 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55aa41c774d0 0x55aa41c761e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f045f662740 1 --2- 172.21.15.152:0/56580679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55aa41bf8bb0 0x55aa41c0e5a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f043e7fc640 0 client.15030 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:52:17.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f045f662740 1 --2- 172.21.15.152:0/56580679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55aa41bee2f0 0x55aa41c0e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f045f662740 1 --2- 172.21.15.152:0/56580679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55aa41bed9d0 0x55aa41c0db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f045f662740 1 -- 172.21.15.152:0/56580679 >> 172.21.15.152:0/56580679 conn(0x55aa41be9f60 msgr2=0x55aa41c08df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:17.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f045f662740 1 -- 172.21.15.152:0/56580679 shutdown_connections 2024-09-11T22:52:17.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f045f662740 1 -- 172.21.15.152:0/56580679 wait complete. 2024-09-11T22:52:17.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.467+0000 7f045f662740 20 client.15030 trim_cache size 0 max 16384 2024-09-11T22:52:17.489 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:52:17.489 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:52:17.490 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 2, 2024-09-11T22:52:17.490 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 24434, 2024-09-11T22:52:17.490 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-09-11T22:52:17.490 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-09-11T22:52:17.490 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-09-11T22:52:17.490 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 74.256011611000005, 2024-09-11T22:52:17.490 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 27, 2024-09-11T22:52:17.490 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 83, 2024-09-11T22:52:17.490 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T22:52:17.490 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 106.386884537 2024-09-11T22:52:17.490 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:52:17.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.468+0000 7f045d289640 1 -- 172.21.15.152:0/2455415685 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f04440a2c90 msgr2=0x7f04440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:17.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.468+0000 7f045d289640 1 --2- 172.21.15.152:0/2455415685 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f04440a2c90 0x7f04440a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f045009ae60 tx=0x7f045006df90 comp rx=0 tx=0).stop 2024-09-11T22:52:17.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.468+0000 7f045d289640 1 -- 172.21.15.152:0/2455415685 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0458106a60 msgr2=0x7f04581030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:17.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.468+0000 7f045d289640 1 --2- 172.21.15.152:0/2455415685 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0458106a60 0x7f04581030b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f045000b970 tx=0x7f045000be30 comp rx=0 tx=0).stop 2024-09-11T22:52:17.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.468+0000 7f045d289640 1 -- 172.21.15.152:0/2455415685 shutdown_connections 2024-09-11T22:52:17.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.468+0000 7f045d289640 1 --2- 172.21.15.152:0/2455415685 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f04440a2c90 0x7f04440a5150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.468+0000 7f045d289640 1 --2- 172.21.15.152:0/2455415685 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0458106a60 0x7f04581030b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.468+0000 7f045d289640 1 --2- 172.21.15.152:0/2455415685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0458101400 0x7f0458102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.468+0000 7f045d289640 1 --2- 172.21.15.152:0/2455415685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0458100a60 0x7f0458102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:17.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.468+0000 7f045d289640 1 -- 172.21.15.152:0/2455415685 >> 172.21.15.152:0/2455415685 conn(0x7f04580fc780 msgr2=0x7f04581048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:17.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.468+0000 7f045d289640 1 -- 172.21.15.152:0/2455415685 shutdown_connections 2024-09-11T22:52:17.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:17.468+0000 7f045d289640 1 -- 172.21.15.152:0/2455415685 wait complete. 2024-09-11T22:52:18.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:17 smithi152 ceph-mon[31181]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T22:52:18.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:17 smithi191 ceph-mon[35211]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T22:52:18.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:17 smithi204 ceph-mon[35454]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T22:52:18.305 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T22:52:18.553 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:19.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.087+0000 7fe65684d640 1 -- 172.21.15.152:0/2105299453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe650100a80 msgr2=0x7fe650100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:19.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.087+0000 7fe65684d640 1 --2- 172.21.15.152:0/2105299453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe650100a80 0x7fe650100ee0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe644009930 tx=0x7fe64402a420 comp rx=0 tx=0).stop 2024-09-11T22:52:19.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.087+0000 7fe65684d640 1 -- 172.21.15.152:0/2105299453 shutdown_connections 2024-09-11T22:52:19.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.087+0000 7fe65684d640 1 --2- 172.21.15.152:0/2105299453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe650101420 0x7fe65010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.087+0000 7fe65684d640 1 --2- 172.21.15.152:0/2105299453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe650100a80 0x7fe650100ee0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.087+0000 7fe65684d640 1 --2- 172.21.15.152:0/2105299453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe650106a80 0x7fe650106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.087+0000 7fe65684d640 1 -- 172.21.15.152:0/2105299453 >> 172.21.15.152:0/2105299453 conn(0x7fe6500fc7a0 msgr2=0x7fe6500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:19.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.087+0000 7fe65684d640 1 -- 172.21.15.152:0/2105299453 shutdown_connections 2024-09-11T22:52:19.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.087+0000 7fe65684d640 1 -- 172.21.15.152:0/2105299453 wait complete. 2024-09-11T22:52:19.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.087+0000 7fe65684d640 1 Processor -- start 2024-09-11T22:52:19.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.087+0000 7fe65684d640 1 -- start start 2024-09-11T22:52:19.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe65684d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe650100a80 0x7fe65010e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe65684d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe650101420 0x7fe65010e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe65684d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe650106a80 0x7fe65010ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe65684d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe650111f60 con 0x7fe650101420 2024-09-11T22:52:19.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe65684d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe650111de0 con 0x7fe650100a80 2024-09-11T22:52:19.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe65684d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe6501120e0 con 0x7fe650106a80 2024-09-11T22:52:19.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe64f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe650101420 0x7fe65010e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe64f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe650101420 0x7fe65010e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57674/0 (socket says 172.21.15.152:57674) 2024-09-11T22:52:19.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe64f7fe640 1 -- 172.21.15.152:0/406805714 learned_addr learned my addr 172.21.15.152:0/406805714 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:19.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe64ffff640 1 --2- 172.21.15.152:0/406805714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe650100a80 0x7fe65010e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe654dea640 1 --2- 172.21.15.152:0/406805714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe650106a80 0x7fe65010ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe64f7fe640 1 -- 172.21.15.152:0/406805714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe650106a80 msgr2=0x7fe65010ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:19.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe64f7fe640 1 --2- 172.21.15.152:0/406805714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe650106a80 0x7fe65010ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe64f7fe640 1 -- 172.21.15.152:0/406805714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe650100a80 msgr2=0x7fe65010e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:19.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe64f7fe640 1 --2- 172.21.15.152:0/406805714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe650100a80 0x7fe65010e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe64f7fe640 1 -- 172.21.15.152:0/406805714 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6501a6880 con 0x7fe650101420 2024-09-11T22:52:19.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe654dea640 1 --2- 172.21.15.152:0/406805714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe650106a80 0x7fe65010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:19.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe64f7fe640 1 --2- 172.21.15.152:0/406805714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe650101420 0x7fe65010e920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe644009230 tx=0x7fe644004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:19.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.088+0000 7fe64d7fa640 1 -- 172.21.15.152:0/406805714 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64403f030 con 0x7fe650101420 2024-09-11T22:52:19.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.089+0000 7fe65684d640 1 -- 172.21.15.152:0/406805714 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6501a6b10 con 0x7fe650101420 2024-09-11T22:52:19.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.089+0000 7fe64d7fa640 1 -- 172.21.15.152:0/406805714 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe644003ad0 con 0x7fe650101420 2024-09-11T22:52:19.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.089+0000 7fe64d7fa640 1 -- 172.21.15.152:0/406805714 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644035070 con 0x7fe650101420 2024-09-11T22:52:19.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.089+0000 7fe65684d640 1 -- 172.21.15.152:0/406805714 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6501a6ff0 con 0x7fe650101420 2024-09-11T22:52:19.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.091+0000 7fe64d7fa640 1 -- 172.21.15.152:0/406805714 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe644042030 con 0x7fe650101420 2024-09-11T22:52:19.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.091+0000 7fe658c26740 1 Processor -- start 2024-09-11T22:52:19.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.091+0000 7fe658c26740 1 -- start start 2024-09-11T22:52:19.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.091+0000 7fe658c26740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ac9d0 0x56171c4acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.091+0000 7fe658c26740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4b7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.091+0000 7fe658c26740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4b7bb0 0x56171c4b9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.091+0000 7fe64d7fa640 1 --2- 172.21.15.152:0/406805714 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6380a2c10 0x7fe6380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.091+0000 7fe658c26740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56171c413090 con 0x56171c4ad2f0 2024-09-11T22:52:19.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.091+0000 7fe658c26740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56171c3baa70 con 0x56171c4b7bb0 2024-09-11T22:52:19.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.091+0000 7fe658c26740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56171c40f050 con 0x56171c4ac9d0 2024-09-11T22:52:19.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.091+0000 7fe64ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4b7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.091+0000 7fe64ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4b7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57684/0 (socket says 172.21.15.152:57684) 2024-09-11T22:52:19.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.091+0000 7fe64ffff640 1 -- 172.21.15.152:0/4087460739 learned_addr learned my addr 172.21.15.152:0/4087460739 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:19.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.092+0000 7fe64f7fe640 1 --2- 172.21.15.152:0/406805714 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6380a2c10 0x7fe6380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.092+0000 7fe654dea640 1 --2- 172.21.15.152:0/4087460739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4b7bb0 0x56171c4b9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.092+0000 7fe654dea640 1 --2- 172.21.15.152:0/4087460739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ac9d0 0x56171c4acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.092+0000 7fe64d7fa640 1 -- 172.21.15.152:0/406805714 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fe6440ef680 con 0x7fe650101420 2024-09-11T22:52:19.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.092+0000 7fe64ffff640 1 -- 172.21.15.152:0/4087460739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ac9d0 msgr2=0x56171c4acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:19.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.092+0000 7fe64ffff640 1 --2- 172.21.15.152:0/4087460739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ac9d0 0x56171c4acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.092+0000 7fe64ffff640 1 -- 172.21.15.152:0/4087460739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4b7bb0 msgr2=0x56171c4b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:19.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.092+0000 7fe64ffff640 1 --2- 172.21.15.152:0/4087460739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4b7bb0 0x56171c4b9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.092+0000 7fe64ffff640 1 -- 172.21.15.152:0/4087460739 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56171c3c7130 con 0x56171c4ad2f0 2024-09-11T22:52:19.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.092+0000 7fe654dea640 1 --2- 172.21.15.152:0/4087460739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ac9d0 0x56171c4acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:19.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.092+0000 7fe64ffff640 1 --2- 172.21.15.152:0/4087460739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4b7670 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe64000ac90 tx=0x7fe64000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7abe54c8e0d51a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:19.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.092+0000 7fe64f7fe640 1 --2- 172.21.15.152:0/406805714 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6380a2c10 0x7fe6380a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe6440bbd40 tx=0x7fe64408ddc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:19.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.092+0000 7fe6367fc640 1 -- 172.21.15.152:0/4087460739 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64001d070 con 0x56171c4ad2f0 2024-09-11T22:52:19.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.092+0000 7fe6367fc640 1 -- 172.21.15.152:0/4087460739 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe640018b20 con 0x56171c4ad2f0 2024-09-11T22:52:19.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.092+0000 7fe6367fc640 1 -- 172.21.15.152:0/4087460739 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe640010ba0 con 0x56171c4ad2f0 2024-09-11T22:52:19.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.093+0000 7fe658c26740 1 -- 172.21.15.152:0/4087460739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 msgr2=0x56171c4b7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:19.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.093+0000 7fe658c26740 1 --2- 172.21.15.152:0/4087460739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4b7670 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe64000ac90 tx=0x7fe64000f790 comp rx=0 tx=0).stop 2024-09-11T22:52:19.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.093+0000 7fe658c26740 1 -- 172.21.15.152:0/4087460739 shutdown_connections 2024-09-11T22:52:19.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.093+0000 7fe658c26740 1 --2- 172.21.15.152:0/4087460739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4b7bb0 0x56171c4b9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.093+0000 7fe658c26740 1 --2- 172.21.15.152:0/4087460739 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4b7670 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.093+0000 7fe658c26740 1 --2- 172.21.15.152:0/4087460739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ac9d0 0x56171c4acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.093+0000 7fe658c26740 1 -- 172.21.15.152:0/4087460739 >> 172.21.15.152:0/4087460739 conn(0x56171c4a8f60 msgr2=0x56171c4a9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:19.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.093+0000 7fe658c26740 1 -- 172.21.15.152:0/4087460739 shutdown_connections 2024-09-11T22:52:19.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.093+0000 7fe658c26740 1 -- 172.21.15.152:0/4087460739 wait complete. 2024-09-11T22:52:19.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.093+0000 7fe658c26740 1 -- start start 2024-09-11T22:52:19.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.093+0000 7fe658c26740 1 Processor -- start 2024-09-11T22:52:19.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe658c26740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ac9d0 0x56171c4b7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe658c26740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe658c26740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4b7bb0 0x56171c4b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe658c26740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56171c4ba780 con 0x56171c4ad2f0 2024-09-11T22:52:19.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe658c26740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56171c3cdf20 con 0x56171c4b7bb0 2024-09-11T22:52:19.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe658c26740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56171c4ba900 con 0x56171c4ac9d0 2024-09-11T22:52:19.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe658c26740 10 client.0 _ll_register_callbacks cb 0x56171c397030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:19.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe64ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe64ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57690/0 (socket says 172.21.15.152:57690) 2024-09-11T22:52:19.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe64ffff640 1 -- 172.21.15.152:0/3450578963 learned_addr learned my addr 172.21.15.152:0/3450578963 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:19.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe654dea640 1 --2- 172.21.15.152:0/3450578963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ac9d0 0x56171c4b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe654dea640 1 --2- 172.21.15.152:0/3450578963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4b7bb0 0x56171c4b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe64ffff640 1 -- 172.21.15.152:0/3450578963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ac9d0 msgr2=0x56171c4b7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:19.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe64ffff640 1 --2- 172.21.15.152:0/3450578963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ac9d0 0x56171c4b7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe64ffff640 1 -- 172.21.15.152:0/3450578963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4b7bb0 msgr2=0x56171c4b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:19.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe64ffff640 1 --2- 172.21.15.152:0/3450578963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4b7bb0 0x56171c4b4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe64ffff640 1 -- 172.21.15.152:0/3450578963 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56171c1842b0 con 0x56171c4ad2f0 2024-09-11T22:52:19.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe64ffff640 1 --2- 172.21.15.152:0/3450578963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4b47c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe64001b040 tx=0x7fe640018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315e2fa3779b25b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:19.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe62f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:19.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe62f7fe640 1 -- 172.21.15.152:0/3450578963 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64001d040 con 0x56171c4ad2f0 2024-09-11T22:52:19.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe658c26740 1 -- 172.21.15.152:0/3450578963 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe640018b20 con 0x56171c4ad2f0 2024-09-11T22:52:19.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe62f7fe640 1 -- 172.21.15.152:0/3450578963 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fe6400108e0 con 0x56171c4ad2f0 2024-09-11T22:52:19.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe62f7fe640 1 -- 172.21.15.152:0/3450578963 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe640010ae0 con 0x56171c4ad2f0 2024-09-11T22:52:19.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe658c26740 10 client.15042 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:19.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.094+0000 7fe658c26740 1 -- 172.21.15.152:0/3450578963 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56171c4c6100 con 0x56171c4ad2f0 2024-09-11T22:52:19.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.095+0000 7fe62f7fe640 1 -- 172.21.15.152:0/3450578963 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe640018b20 con 0x56171c4ad2f0 2024-09-11T22:52:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.095+0000 7fe62f7fe640 1 -- 172.21.15.152:0/3450578963 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe640020030 con 0x56171c4ad2f0 2024-09-11T22:52:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.095+0000 7fe658c26740 10 client.15042 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.095+0000 7fe658c26740 10 client.15042 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T22:52:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.095+0000 7fe658c26740 20 client.15042 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.095+0000 7fe658c26740 1 --2- 172.21.15.152:0/3450578963 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x56171c4c83f0 0x56171c4c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.095+0000 7fe658c26740 4 client.15042 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:52:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.095+0000 7fe658c26740 1 -- 172.21.15.152:0/3450578963 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56171c420600 con 0x56171c4c83f0 2024-09-11T22:52:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.095+0000 7fe654dea640 1 --2- 172.21.15.152:0/3450578963 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x56171c4c83f0 0x56171c4c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.096+0000 7fe62f7fe640 10 client.15042 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:52:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.096+0000 7fe654dea640 1 --2- 172.21.15.152:0/3450578963 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x56171c4c83f0 0x56171c4c9860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93aceb95ac8bfcbc server_cookie=9c77b3957b81d5c1 in_seq=0 out_seq=0 2024-09-11T22:52:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.096+0000 7fe62f7fe640 1 -- 172.21.15.152:0/3450578963 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56171c420600 con 0x56171c4c83f0 2024-09-11T22:52:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.096+0000 7fe62f7fe640 10 client.15042 handle_command_reply: tid=0 2024-09-11T22:52:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.096+0000 7fe658c26740 1 client.15042 shutdown 2024-09-11T22:52:19.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.097+0000 7fe658c26740 1 -- 172.21.15.152:0/3450578963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 msgr2=0x56171c4b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:19.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.097+0000 7fe658c26740 1 --2- 172.21.15.152:0/3450578963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4b47c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe64001b040 tx=0x7fe640018670 comp rx=0 tx=0).stop 2024-09-11T22:52:19.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.097+0000 7fe658c26740 1 -- 172.21.15.152:0/3450578963 shutdown_connections 2024-09-11T22:52:19.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.097+0000 7fe658c26740 1 --2- 172.21.15.152:0/3450578963 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x56171c4c83f0 0x56171c4c9860 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.097+0000 7fe658c26740 1 --2- 172.21.15.152:0/3450578963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4b7bb0 0x56171c4b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.097+0000 7fe658c26740 1 --2- 172.21.15.152:0/3450578963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4b47c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.097+0000 7fe658c26740 1 --2- 172.21.15.152:0/3450578963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ac9d0 0x56171c4b7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.097+0000 7fe62f7fe640 0 client.15042 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:52:19.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.097+0000 7fe658c26740 1 -- 172.21.15.152:0/3450578963 >> 172.21.15.152:0/3450578963 conn(0x56171c4a8f60 msgr2=0x56171c4aec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:19.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.097+0000 7fe658c26740 1 -- 172.21.15.152:0/3450578963 shutdown_connections 2024-09-11T22:52:19.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.097+0000 7fe658c26740 1 -- 172.21.15.152:0/3450578963 wait complete. 2024-09-11T22:52:19.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.097+0000 7fe658c26740 20 client.15042 trim_cache size 0 max 16384 2024-09-11T22:52:19.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.114+0000 7fe658c26740 1 Processor -- start 2024-09-11T22:52:19.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.114+0000 7fe658c26740 1 -- start start 2024-09-11T22:52:19.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe658c26740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4ac9d0 0x56171c4c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe658c26740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe658c26740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4b7bb0 0x56171c4c7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe658c26740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56171c4c7ba0 con 0x56171c4ad2f0 2024-09-11T22:52:19.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe658c26740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56171c4c8270 con 0x56171c4ac9d0 2024-09-11T22:52:19.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe658c26740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56171c4c8940 con 0x56171c4b7bb0 2024-09-11T22:52:19.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe64ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe64ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57700/0 (socket says 172.21.15.152:57700) 2024-09-11T22:52:19.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe64ffff640 1 -- 172.21.15.152:0/3882180956 learned_addr learned my addr 172.21.15.152:0/3882180956 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:19.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe64ffff640 1 -- 172.21.15.152:0/3882180956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4b7bb0 msgr2=0x56171c4c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:19.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe654dea640 1 --2- 172.21.15.152:0/3882180956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4ac9d0 0x56171c4c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe64ffff640 1 --2- 172.21.15.152:0/3882180956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4b7bb0 0x56171c4c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe64ffff640 1 -- 172.21.15.152:0/3882180956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4ac9d0 msgr2=0x56171c4c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:19.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe64ffff640 1 --2- 172.21.15.152:0/3882180956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4ac9d0 0x56171c4c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe64ffff640 1 -- 172.21.15.152:0/3882180956 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56171c4b7090 con 0x56171c4ad2f0 2024-09-11T22:52:19.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe64ffff640 1 --2- 172.21.15.152:0/3882180956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4c8ac0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe64001b010 tx=0x7fe64000a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd18cb8af96ceb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:19.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe635ffb640 1 -- 172.21.15.152:0/3882180956 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64001d070 con 0x56171c4ad2f0 2024-09-11T22:52:19.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe635ffb640 1 -- 172.21.15.152:0/3882180956 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe640028070 con 0x56171c4ad2f0 2024-09-11T22:52:19.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.115+0000 7fe635ffb640 1 -- 172.21.15.152:0/3882180956 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe640004d60 con 0x56171c4ad2f0 2024-09-11T22:52:19.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.116+0000 7fe658c26740 1 -- 172.21.15.152:0/3882180956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 msgr2=0x56171c4c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:19.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.116+0000 7fe658c26740 1 --2- 172.21.15.152:0/3882180956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4c8ac0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe64001b010 tx=0x7fe64000a480 comp rx=0 tx=0).stop 2024-09-11T22:52:19.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.116+0000 7fe658c26740 1 -- 172.21.15.152:0/3882180956 shutdown_connections 2024-09-11T22:52:19.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.116+0000 7fe658c26740 1 --2- 172.21.15.152:0/3882180956 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4b7bb0 0x56171c4c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.116+0000 7fe658c26740 1 --2- 172.21.15.152:0/3882180956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4ad2f0 0x56171c4c8ac0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.116+0000 7fe658c26740 1 --2- 172.21.15.152:0/3882180956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4ac9d0 0x56171c4c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.116+0000 7fe658c26740 1 -- 172.21.15.152:0/3882180956 >> 172.21.15.152:0/3882180956 conn(0x56171c4a8f60 msgr2=0x56171c4a9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:19.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.116+0000 7fe658c26740 1 -- 172.21.15.152:0/3882180956 shutdown_connections 2024-09-11T22:52:19.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.116+0000 7fe658c26740 1 -- 172.21.15.152:0/3882180956 wait complete. 2024-09-11T22:52:19.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.116+0000 7fe658c26740 1 -- start start 2024-09-11T22:52:19.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe658c26740 1 Processor -- start 2024-09-11T22:52:19.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe658c26740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4ac9d0 0x56171c4cd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe658c26740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ad2f0 0x56171c4cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe658c26740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4b7bb0 0x56171c530af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe658c26740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56171c4c96e0 con 0x56171c4b7bb0 2024-09-11T22:52:19.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe658c26740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56171c4c9010 con 0x56171c4ac9d0 2024-09-11T22:52:19.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe658c26740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56171c4c9db0 con 0x56171c4ad2f0 2024-09-11T22:52:19.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe658c26740 10 client.0 _ll_register_callbacks cb 0x7fe64001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:19.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe654dea640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4b7bb0 0x56171c530af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe654dea640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4b7bb0 0x56171c530af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57712/0 (socket says 172.21.15.152:57712) 2024-09-11T22:52:19.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe654dea640 1 -- 172.21.15.152:0/1216745918 learned_addr learned my addr 172.21.15.152:0/1216745918 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:19.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe654dea640 1 --2- 172.21.15.152:0/1216745918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4ac9d0 0x56171c4cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe64ffff640 1 --2- 172.21.15.152:0/1216745918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ad2f0 0x56171c4cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.127 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe654dea640 1 -- 172.21.15.152:0/1216745918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ad2f0 msgr2=0x56171c4cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:19.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe654dea640 1 --2- 172.21.15.152:0/1216745918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ad2f0 0x56171c4cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe654dea640 1 -- 172.21.15.152:0/1216745918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4ac9d0 msgr2=0x56171c4cd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:19.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe654dea640 1 --2- 172.21.15.152:0/1216745918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4ac9d0 0x56171c4cd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe654dea640 1 -- 172.21.15.152:0/1216745918 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56171c531030 con 0x56171c4b7bb0 2024-09-11T22:52:19.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe64ffff640 1 --2- 172.21.15.152:0/1216745918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ad2f0 0x56171c4cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:19.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe654dea640 1 --2- 172.21.15.152:0/1216745918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4b7bb0 0x56171c530af0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe648005b20 tx=0x7fe648009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920e7f202340809c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:19.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe62f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:19.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.117+0000 7fe62f7fe640 1 -- 172.21.15.152:0/1216745918 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64801b040 con 0x56171c4b7bb0 2024-09-11T22:52:19.128 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.118+0000 7fe62f7fe640 1 -- 172.21.15.152:0/1216745918 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fe648012cb0 con 0x56171c4b7bb0 2024-09-11T22:52:19.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.118+0000 7fe658c26740 1 -- 172.21.15.152:0/1216745918 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe640028070 con 0x56171c4b7bb0 2024-09-11T22:52:19.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.118+0000 7fe62f7fe640 1 -- 172.21.15.152:0/1216745918 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe6480223d0 con 0x56171c4b7bb0 2024-09-11T22:52:19.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.118+0000 7fe658c26740 10 client.15048 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:19.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.118+0000 7fe658c26740 1 -- 172.21.15.152:0/1216745918 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56171c5322d0 con 0x56171c4b7bb0 2024-09-11T22:52:19.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.118+0000 7fe62f7fe640 1 -- 172.21.15.152:0/1216745918 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe640028070 con 0x56171c4b7bb0 2024-09-11T22:52:19.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.118+0000 7fe62f7fe640 1 -- 172.21.15.152:0/1216745918 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe648021040 con 0x56171c4b7bb0 2024-09-11T22:52:19.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.118+0000 7fe658c26740 10 client.15048 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:19.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.118+0000 7fe658c26740 10 client.15048 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T22:52:19.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.118+0000 7fe658c26740 20 client.15048 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:19.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.118+0000 7fe658c26740 1 --2- 172.21.15.152:0/1216745918 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x56171c536dd0 0x56171c535ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:19.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.118+0000 7fe658c26740 4 client.15048 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:52:19.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.118+0000 7fe658c26740 1 -- 172.21.15.152:0/1216745918 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "status"}) -- 0x56171c41ab10 con 0x56171c536dd0 2024-09-11T22:52:19.129 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.118+0000 7fe64ffff640 1 --2- 172.21.15.152:0/1216745918 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x56171c536dd0 0x56171c535ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:19.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe62f7fe640 10 client.15048 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:52:19.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe64ffff640 1 --2- 172.21.15.152:0/1216745918 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x56171c536dd0 0x56171c535ae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=54143e96a9e277d8 server_cookie=c6bb55cb4f110a04 in_seq=0 out_seq=0 2024-09-11T22:52:19.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe62f7fe640 1 -- 172.21.15.152:0/1216745918 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56171c41ab10 con 0x56171c536dd0 2024-09-11T22:52:19.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe62f7fe640 10 client.15048 handle_command_reply: tid=0 2024-09-11T22:52:19.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe658c26740 1 client.15048 shutdown 2024-09-11T22:52:19.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe658c26740 1 -- 172.21.15.152:0/1216745918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4b7bb0 msgr2=0x56171c530af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:19.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe658c26740 1 --2- 172.21.15.152:0/1216745918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4b7bb0 0x56171c530af0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe648005b20 tx=0x7fe648009770 comp rx=0 tx=0).stop 2024-09-11T22:52:19.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe658c26740 1 -- 172.21.15.152:0/1216745918 shutdown_connections 2024-09-11T22:52:19.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe658c26740 1 --2- 172.21.15.152:0/1216745918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56171c4b7bb0 0x56171c530af0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe658c26740 1 --2- 172.21.15.152:0/1216745918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56171c4ad2f0 0x56171c4cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe658c26740 1 --2- 172.21.15.152:0/1216745918 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x56171c536dd0 0x56171c535ae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe658c26740 1 --2- 172.21.15.152:0/1216745918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56171c4ac9d0 0x56171c4cd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.130 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe658c26740 1 -- 172.21.15.152:0/1216745918 >> 172.21.15.152:0/1216745918 conn(0x56171c4a8f60 msgr2=0x56171c4b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:19.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe62f7fe640 0 client.15048 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:52:19.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe658c26740 1 -- 172.21.15.152:0/1216745918 shutdown_connections 2024-09-11T22:52:19.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe658c26740 1 -- 172.21.15.152:0/1216745918 wait complete. 2024-09-11T22:52:19.131 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.119+0000 7fe658c26740 20 client.15048 trim_cache size 0 max 16384 2024-09-11T22:52:19.131 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:52:19.131 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:52:19.131 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 1, 2024-09-11T22:52:19.131 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 24442, 2024-09-11T22:52:19.131 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-09-11T22:52:19.132 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-09-11T22:52:19.132 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-09-11T22:52:19.132 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 76.919960919999994, 2024-09-11T22:52:19.132 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 27, 2024-09-11T22:52:19.132 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 83, 2024-09-11T22:52:19.132 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T22:52:19.132 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 105.475722164 2024-09-11T22:52:19.132 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:52:19.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.120+0000 7fe65684d640 1 -- 172.21.15.152:0/406805714 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6380a2c10 msgr2=0x7fe6380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:19.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.120+0000 7fe65684d640 1 --2- 172.21.15.152:0/406805714 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6380a2c10 0x7fe6380a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe6440bbd40 tx=0x7fe64408ddc0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.120+0000 7fe65684d640 1 -- 172.21.15.152:0/406805714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe650101420 msgr2=0x7fe65010e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:19.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.120+0000 7fe65684d640 1 --2- 172.21.15.152:0/406805714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe650101420 0x7fe65010e920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe644009230 tx=0x7fe644004270 comp rx=0 tx=0).stop 2024-09-11T22:52:19.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.120+0000 7fe65684d640 1 -- 172.21.15.152:0/406805714 shutdown_connections 2024-09-11T22:52:19.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.120+0000 7fe65684d640 1 --2- 172.21.15.152:0/406805714 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe6380a2c10 0x7fe6380a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.120+0000 7fe65684d640 1 --2- 172.21.15.152:0/406805714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe650106a80 0x7fe65010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.120+0000 7fe65684d640 1 --2- 172.21.15.152:0/406805714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe650101420 0x7fe65010e920 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.120+0000 7fe65684d640 1 --2- 172.21.15.152:0/406805714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe650100a80 0x7fe65010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:19.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.120+0000 7fe65684d640 1 -- 172.21.15.152:0/406805714 >> 172.21.15.152:0/406805714 conn(0x7fe6500fc7a0 msgr2=0x7fe6500fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:19.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.120+0000 7fe65684d640 1 -- 172.21.15.152:0/406805714 shutdown_connections 2024-09-11T22:52:19.133 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:19.120+0000 7fe65684d640 1 -- 172.21.15.152:0/406805714 wait complete. 2024-09-11T22:52:19.877 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T22:52:20.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:19 smithi152 ceph-mon[31181]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T22:52:20.123 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:20.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:19 smithi191 ceph-mon[35211]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T22:52:20.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:19 smithi204 ceph-mon[35454]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T22:52:20.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.653+0000 7f112b125640 1 -- 172.21.15.152:0/1632758472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1124102da0 msgr2=0x7f1124103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:20.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.653+0000 7f112b125640 1 --2- 172.21.15.152:0/1632758472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1124102da0 0x7f1124103180 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1118009900 tx=0x7f111802a480 comp rx=0 tx=0).stop 2024-09-11T22:52:20.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.653+0000 7f112b125640 1 -- 172.21.15.152:0/1632758472 shutdown_connections 2024-09-11T22:52:20.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.653+0000 7f112b125640 1 --2- 172.21.15.152:0/1632758472 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f112410c180 0x7f112410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.653+0000 7f112b125640 1 --2- 172.21.15.152:0/1632758472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11241036c0 0x7f112410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.653+0000 7f112b125640 1 --2- 172.21.15.152:0/1632758472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1124102da0 0x7f1124103180 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.653+0000 7f112b125640 1 -- 172.21.15.152:0/1632758472 >> 172.21.15.152:0/1632758472 conn(0x7f11240fc7a0 msgr2=0x7f11240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:20.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.653+0000 7f112b125640 1 -- 172.21.15.152:0/1632758472 shutdown_connections 2024-09-11T22:52:20.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.653+0000 7f112b125640 1 -- 172.21.15.152:0/1632758472 wait complete. 2024-09-11T22:52:20.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.654+0000 7f112b125640 1 Processor -- start 2024-09-11T22:52:20.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.654+0000 7f112b125640 1 -- start start 2024-09-11T22:52:20.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.654+0000 7f112b125640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1124102da0 0x7f1124198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.654+0000 7f112b125640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11241036c0 0x7f1124198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.654+0000 7f112b125640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f112410c180 0x7f112419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.654+0000 7f112b125640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1124111fa0 con 0x7f1124102da0 2024-09-11T22:52:20.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.654+0000 7f112b125640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1124111e20 con 0x7f112410c180 2024-09-11T22:52:20.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.654+0000 7f112b125640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1124112120 con 0x7f11241036c0 2024-09-11T22:52:20.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.654+0000 7f1128ec1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1124102da0 0x7f1124198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.654+0000 7f1128ec1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1124102da0 0x7f1124198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57730/0 (socket says 172.21.15.152:57730) 2024-09-11T22:52:20.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.654+0000 7f1128ec1640 1 -- 172.21.15.152:0/3846185089 learned_addr learned my addr 172.21.15.152:0/3846185089 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:20.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.655+0000 7f1123fff640 1 --2- 172.21.15.152:0/3846185089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11241036c0 0x7f1124198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.655+0000 7f1128ec1640 1 -- 172.21.15.152:0/3846185089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11241036c0 msgr2=0x7f1124198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:20.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.655+0000 7f1128ec1640 1 --2- 172.21.15.152:0/3846185089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11241036c0 0x7f1124198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.655+0000 7f11296c2640 1 --2- 172.21.15.152:0/3846185089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f112410c180 0x7f112419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.655+0000 7f1128ec1640 1 -- 172.21.15.152:0/3846185089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f112410c180 msgr2=0x7f112419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:20.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.655+0000 7f1128ec1640 1 --2- 172.21.15.152:0/3846185089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f112410c180 0x7f112419ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.655+0000 7f1128ec1640 1 -- 172.21.15.152:0/3846185089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112419d3d0 con 0x7f1124102da0 2024-09-11T22:52:20.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.655+0000 7f1123fff640 1 --2- 172.21.15.152:0/3846185089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11241036c0 0x7f1124198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:52:20.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.655+0000 7f1128ec1640 1 --2- 172.21.15.152:0/3846185089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1124102da0 0x7f1124198420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1118005ab0 tx=0x7f11180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:20.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.655+0000 7f1121ffb640 1 -- 172.21.15.152:0/3846185089 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111803f030 con 0x7f1124102da0 2024-09-11T22:52:20.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.655+0000 7f1121ffb640 1 -- 172.21.15.152:0/3846185089 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1118003ce0 con 0x7f1124102da0 2024-09-11T22:52:20.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.655+0000 7f1121ffb640 1 -- 172.21.15.152:0/3846185089 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118035070 con 0x7f1124102da0 2024-09-11T22:52:20.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.655+0000 7f112b125640 1 -- 172.21.15.152:0/3846185089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112419d660 con 0x7f1124102da0 2024-09-11T22:52:20.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.655+0000 7f112b125640 1 -- 172.21.15.152:0/3846185089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11241a4fa0 con 0x7f1124102da0 2024-09-11T22:52:20.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.657+0000 7f1121ffb640 1 -- 172.21.15.152:0/3846185089 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1118042030 con 0x7f1124102da0 2024-09-11T22:52:20.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f112d4fe740 1 Processor -- start 2024-09-11T22:52:20.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f112d4fe740 1 -- start start 2024-09-11T22:52:20.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f112d4fe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349e9d0 0x55ece349edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f112d4fe740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece349f2f0 0x55ece34a9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f112d4fe740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece34a9bb0 0x55ece34abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f112d4fe740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ece3405090 con 0x55ece34a9bb0 2024-09-11T22:52:20.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f112d4fe740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ece33aca70 con 0x55ece349f2f0 2024-09-11T22:52:20.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f112d4fe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ece3401050 con 0x55ece349e9d0 2024-09-11T22:52:20.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f1123fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece34a9bb0 0x55ece34abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f1123fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece34a9bb0 0x55ece34abfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57740/0 (socket says 172.21.15.152:57740) 2024-09-11T22:52:20.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f1123fff640 1 -- 172.21.15.152:0/661305223 learned_addr learned my addr 172.21.15.152:0/661305223 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:20.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f11296c2640 1 --2- 172.21.15.152:0/661305223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349e9d0 0x55ece349edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f1121ffb640 1 --2- 172.21.15.152:0/3846185089 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f11140a2c90 0x7f11140a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f1128ec1640 1 --2- 172.21.15.152:0/661305223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece349f2f0 0x55ece34a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f11296c2640 1 --2- 172.21.15.152:0/3846185089 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f11140a2c90 0x7f11140a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f1123fff640 1 -- 172.21.15.152:0/661305223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349e9d0 msgr2=0x55ece349edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:20.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f1123fff640 1 --2- 172.21.15.152:0/661305223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349e9d0 0x55ece349edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f1123fff640 1 -- 172.21.15.152:0/661305223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece349f2f0 msgr2=0x55ece34a9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:20.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f1123fff640 1 --2- 172.21.15.152:0/661305223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece349f2f0 0x55ece34a9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.658+0000 7f1123fff640 1 -- 172.21.15.152:0/661305223 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ece33b9130 con 0x55ece34a9bb0 2024-09-11T22:52:20.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.659+0000 7f1121ffb640 1 -- 172.21.15.152:0/3846185089 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f11180efae0 con 0x7f1124102da0 2024-09-11T22:52:20.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.659+0000 7f1123fff640 1 --2- 172.21.15.152:0/661305223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece34a9bb0 0x55ece34abfa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f111000a9d0 tx=0x7f111000ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff25729bea91928 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:20.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.659+0000 7f110effd640 1 -- 172.21.15.152:0/661305223 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111001e070 con 0x55ece34a9bb0 2024-09-11T22:52:20.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.659+0000 7f110effd640 1 -- 172.21.15.152:0/661305223 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111000cc60 con 0x55ece34a9bb0 2024-09-11T22:52:20.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.659+0000 7f110effd640 1 -- 172.21.15.152:0/661305223 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110019860 con 0x55ece34a9bb0 2024-09-11T22:52:20.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.659+0000 7f11296c2640 1 --2- 172.21.15.152:0/3846185089 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f11140a2c90 0x7f11140a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f111c014750 tx=0x7f111c004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:20.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 -- 172.21.15.152:0/661305223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece34a9bb0 msgr2=0x55ece34abfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:20.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 --2- 172.21.15.152:0/661305223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece34a9bb0 0x55ece34abfa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f111000a9d0 tx=0x7f111000ae90 comp rx=0 tx=0).stop 2024-09-11T22:52:20.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 -- 172.21.15.152:0/661305223 shutdown_connections 2024-09-11T22:52:20.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 --2- 172.21.15.152:0/661305223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece34a9bb0 0x55ece34abfa0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 --2- 172.21.15.152:0/661305223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece349f2f0 0x55ece34a9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 --2- 172.21.15.152:0/661305223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349e9d0 0x55ece349edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 -- 172.21.15.152:0/661305223 >> 172.21.15.152:0/661305223 conn(0x55ece349af60 msgr2=0x55ece349b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:20.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 -- 172.21.15.152:0/661305223 shutdown_connections 2024-09-11T22:52:20.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 -- 172.21.15.152:0/661305223 wait complete. 2024-09-11T22:52:20.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 -- start start 2024-09-11T22:52:20.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 Processor -- start 2024-09-11T22:52:20.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349e9d0 0x55ece34a9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece349f2f0 0x55ece34a6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece34a9bb0 0x55ece34a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ece34ac780 con 0x55ece349f2f0 2024-09-11T22:52:20.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ece33bff20 con 0x55ece34a9bb0 2024-09-11T22:52:20.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ece34ac900 con 0x55ece349e9d0 2024-09-11T22:52:20.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.660+0000 7f112d4fe740 10 client.0 _ll_register_callbacks cb 0x55ece3389030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:20.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f1123fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece349f2f0 0x55ece34a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f1123fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece349f2f0 0x55ece34a6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57750/0 (socket says 172.21.15.152:57750) 2024-09-11T22:52:20.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f1123fff640 1 -- 172.21.15.152:0/1107454715 learned_addr learned my addr 172.21.15.152:0/1107454715 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:20.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f1128ec1640 1 --2- 172.21.15.152:0/1107454715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349e9d0 0x55ece34a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f1123fff640 1 -- 172.21.15.152:0/1107454715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349e9d0 msgr2=0x55ece34a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:20.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f1123fff640 1 --2- 172.21.15.152:0/1107454715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349e9d0 0x55ece34a9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f1123fff640 1 -- 172.21.15.152:0/1107454715 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece34a9bb0 msgr2=0x55ece34a6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:20.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f11296c2640 1 --2- 172.21.15.152:0/1107454715 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece34a9bb0 0x55ece34a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f1123fff640 1 --2- 172.21.15.152:0/1107454715 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece34a9bb0 0x55ece34a6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f1123fff640 1 -- 172.21.15.152:0/1107454715 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ece31762b0 con 0x55ece349f2f0 2024-09-11T22:52:20.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f1128ec1640 1 --2- 172.21.15.152:0/1107454715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349e9d0 0x55ece34a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:52:20.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f1123fff640 1 --2- 172.21.15.152:0/1107454715 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece349f2f0 0x55ece34a6790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1110019480 tx=0x7f1110011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399b8cc34001450 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:20.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f1103fff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:20.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f1103fff640 1 -- 172.21.15.152:0/1107454715 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110029060 con 0x55ece349f2f0 2024-09-11T22:52:20.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f1103fff640 1 -- 172.21.15.152:0/1107454715 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f11100195e0 con 0x55ece349f2f0 2024-09-11T22:52:20.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f112d4fe740 1 -- 172.21.15.152:0/1107454715 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f111000cc60 con 0x55ece349f2f0 2024-09-11T22:52:20.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f1103fff640 1 -- 172.21.15.152:0/1107454715 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f111001e070 con 0x55ece349f2f0 2024-09-11T22:52:20.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f112d4fe740 10 client.15060 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:20.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f112d4fe740 1 -- 172.21.15.152:0/1107454715 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ece34b8090 con 0x55ece349f2f0 2024-09-11T22:52:20.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.661+0000 7f1103fff640 1 -- 172.21.15.152:0/1107454715 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111000cc60 con 0x55ece349f2f0 2024-09-11T22:52:20.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.662+0000 7f1103fff640 1 -- 172.21.15.152:0/1107454715 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1110024920 con 0x55ece349f2f0 2024-09-11T22:52:20.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.662+0000 7f112d4fe740 10 client.15060 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:20.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.662+0000 7f112d4fe740 10 client.15060 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:52:20.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.662+0000 7f112d4fe740 20 client.15060 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:20.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.662+0000 7f112d4fe740 1 --2- 172.21.15.152:0/1107454715 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ece34ba2e0 0x55ece34bb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.662+0000 7f112d4fe740 4 client.15060 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:52:20.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.662+0000 7f112d4fe740 1 -- 172.21.15.152:0/1107454715 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ece3412600 con 0x55ece34ba2e0 2024-09-11T22:52:20.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.662+0000 7f1128ec1640 1 --2- 172.21.15.152:0/1107454715 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ece34ba2e0 0x55ece34bb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.662+0000 7f1103fff640 10 client.15060 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:52:20.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.662+0000 7f1128ec1640 1 --2- 172.21.15.152:0/1107454715 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ece34ba2e0 0x55ece34bb750 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48511c1e6908080 server_cookie=31179ef9c72e97ec in_seq=0 out_seq=0 2024-09-11T22:52:20.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.663+0000 7f1103fff640 1 -- 172.21.15.152:0/1107454715 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ece3412600 con 0x55ece34ba2e0 2024-09-11T22:52:20.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.663+0000 7f1103fff640 10 client.15060 handle_command_reply: tid=0 2024-09-11T22:52:20.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.663+0000 7f112d4fe740 1 client.15060 shutdown 2024-09-11T22:52:20.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.663+0000 7f112d4fe740 1 -- 172.21.15.152:0/1107454715 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece349f2f0 msgr2=0x55ece34a6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:20.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.663+0000 7f112d4fe740 1 --2- 172.21.15.152:0/1107454715 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece349f2f0 0x55ece34a6790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1110019480 tx=0x7f1110011ce0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.663+0000 7f112d4fe740 1 -- 172.21.15.152:0/1107454715 shutdown_connections 2024-09-11T22:52:20.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.663+0000 7f112d4fe740 1 --2- 172.21.15.152:0/1107454715 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ece34ba2e0 0x55ece34bb750 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.663+0000 7f112d4fe740 1 --2- 172.21.15.152:0/1107454715 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece34a9bb0 0x55ece34a6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.663+0000 7f1103fff640 0 client.15060 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:52:20.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.663+0000 7f112d4fe740 1 --2- 172.21.15.152:0/1107454715 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece349f2f0 0x55ece34a6790 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.663+0000 7f112d4fe740 1 --2- 172.21.15.152:0/1107454715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349e9d0 0x55ece34a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.663+0000 7f112d4fe740 1 -- 172.21.15.152:0/1107454715 >> 172.21.15.152:0/1107454715 conn(0x55ece349af60 msgr2=0x55ece34a0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:20.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.664+0000 7f112d4fe740 1 -- 172.21.15.152:0/1107454715 shutdown_connections 2024-09-11T22:52:20.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.664+0000 7f112d4fe740 1 -- 172.21.15.152:0/1107454715 wait complete. 2024-09-11T22:52:20.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.664+0000 7f112d4fe740 20 client.15060 trim_cache size 0 max 16384 2024-09-11T22:52:20.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.681+0000 7f112d4fe740 1 Processor -- start 2024-09-11T22:52:20.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.681+0000 7f112d4fe740 1 -- start start 2024-09-11T22:52:20.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.681+0000 7f112d4fe740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece349e9d0 0x55ece34bb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.681+0000 7f112d4fe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349f2f0 0x55ece34a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.681+0000 7f112d4fe740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece34a9bb0 0x55ece34a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.681+0000 7f112d4fe740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ece34b9a90 con 0x55ece34a9bb0 2024-09-11T22:52:20.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.681+0000 7f112d4fe740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ece34ba160 con 0x55ece349e9d0 2024-09-11T22:52:20.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.681+0000 7f112d4fe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ece34ba830 con 0x55ece349f2f0 2024-09-11T22:52:20.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.681+0000 7f11296c2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece34a9bb0 0x55ece34a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.681+0000 7f11296c2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece34a9bb0 0x55ece34a3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57764/0 (socket says 172.21.15.152:57764) 2024-09-11T22:52:20.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.681+0000 7f11296c2640 1 -- 172.21.15.152:0/3172193551 learned_addr learned my addr 172.21.15.152:0/3172193551 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:20.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.682+0000 7f11296c2640 1 -- 172.21.15.152:0/3172193551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349f2f0 msgr2=0x55ece34a3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:20.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.682+0000 7f11296c2640 1 --2- 172.21.15.152:0/3172193551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349f2f0 0x55ece34a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.682+0000 7f11296c2640 1 -- 172.21.15.152:0/3172193551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece349e9d0 msgr2=0x55ece34bb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:20.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.682+0000 7f1128ec1640 1 --2- 172.21.15.152:0/3172193551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece349e9d0 0x55ece34bb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.682+0000 7f11296c2640 1 --2- 172.21.15.152:0/3172193551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece349e9d0 0x55ece34bb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.682+0000 7f11296c2640 1 -- 172.21.15.152:0/3172193551 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ece34ac300 con 0x55ece34a9bb0 2024-09-11T22:52:20.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.682+0000 7f11296c2640 1 --2- 172.21.15.152:0/3172193551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece34a9bb0 0x55ece34a3f50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f111c014300 tx=0x7f111c0086d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aae265e53fbdd06 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:20.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.682+0000 7f110e7fc640 1 -- 172.21.15.152:0/3172193551 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c013660 con 0x55ece34a9bb0 2024-09-11T22:52:20.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.682+0000 7f110e7fc640 1 -- 172.21.15.152:0/3172193551 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111c025070 con 0x55ece34a9bb0 2024-09-11T22:52:20.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.682+0000 7f110e7fc640 1 -- 172.21.15.152:0/3172193551 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c020b40 con 0x55ece34a9bb0 2024-09-11T22:52:20.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 -- 172.21.15.152:0/3172193551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece34a9bb0 msgr2=0x55ece34a3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:20.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 --2- 172.21.15.152:0/3172193551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece34a9bb0 0x55ece34a3f50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f111c014300 tx=0x7f111c0086d0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 -- 172.21.15.152:0/3172193551 shutdown_connections 2024-09-11T22:52:20.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 --2- 172.21.15.152:0/3172193551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece34a9bb0 0x55ece34a3f50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 --2- 172.21.15.152:0/3172193551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349f2f0 0x55ece34a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 --2- 172.21.15.152:0/3172193551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece349e9d0 0x55ece34bb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 -- 172.21.15.152:0/3172193551 >> 172.21.15.152:0/3172193551 conn(0x55ece349af60 msgr2=0x55ece34a5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:20.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 -- 172.21.15.152:0/3172193551 shutdown_connections 2024-09-11T22:52:20.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 -- 172.21.15.152:0/3172193551 wait complete. 2024-09-11T22:52:20.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 -- start start 2024-09-11T22:52:20.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 Processor -- start 2024-09-11T22:52:20.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349e9d0 0x55ece34c4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece349f2f0 0x55ece34bf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece34a9bb0 0x55ece34bf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ece34bb5d0 con 0x55ece349f2f0 2024-09-11T22:52:20.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ece34baf00 con 0x55ece34a9bb0 2024-09-11T22:52:20.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.683+0000 7f112d4fe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ece34bbca0 con 0x55ece349e9d0 2024-09-11T22:52:20.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f112d4fe740 10 client.0 _ll_register_callbacks cb 0x7f111001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:20.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f1123fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece349f2f0 0x55ece34bf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f1123fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece349f2f0 0x55ece34bf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57776/0 (socket says 172.21.15.152:57776) 2024-09-11T22:52:20.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f1123fff640 1 -- 172.21.15.152:0/3510555173 learned_addr learned my addr 172.21.15.152:0/3510555173 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:20.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f11296c2640 1 --2- 172.21.15.152:0/3510555173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece34a9bb0 0x55ece34bf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f1128ec1640 1 --2- 172.21.15.152:0/3510555173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349e9d0 0x55ece34c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f1123fff640 1 -- 172.21.15.152:0/3510555173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349e9d0 msgr2=0x55ece34c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:20.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f1123fff640 1 --2- 172.21.15.152:0/3510555173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349e9d0 0x55ece34c4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f1123fff640 1 -- 172.21.15.152:0/3510555173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece34a9bb0 msgr2=0x55ece34bf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:20.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f1123fff640 1 --2- 172.21.15.152:0/3510555173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece34a9bb0 0x55ece34bf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f1123fff640 1 -- 172.21.15.152:0/3510555173 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ece34bfff0 con 0x55ece349f2f0 2024-09-11T22:52:20.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f1123fff640 1 --2- 172.21.15.152:0/3510555173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece349f2f0 0x55ece34bf3c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1110024ee0 tx=0x7f111000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ff4e6318ea0833 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:20.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f1103fff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:20.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f1103fff640 1 -- 172.21.15.152:0/3510555173 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110034070 con 0x55ece349f2f0 2024-09-11T22:52:20.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f1103fff640 1 -- 172.21.15.152:0/3510555173 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f1110024460 con 0x55ece349f2f0 2024-09-11T22:52:20.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f112d4fe740 1 -- 172.21.15.152:0/3510555173 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f111c025070 con 0x55ece349f2f0 2024-09-11T22:52:20.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f1103fff640 1 -- 172.21.15.152:0/3510555173 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f111003f070 con 0x55ece349f2f0 2024-09-11T22:52:20.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f112d4fe740 10 client.15066 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:20.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.684+0000 7f112d4fe740 1 -- 172.21.15.152:0/3510555173 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ece35248d0 con 0x55ece349f2f0 2024-09-11T22:52:20.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.685+0000 7f1103fff640 1 -- 172.21.15.152:0/3510555173 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111c025070 con 0x55ece349f2f0 2024-09-11T22:52:20.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.685+0000 7f1103fff640 1 -- 172.21.15.152:0/3510555173 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1110019c10 con 0x55ece349f2f0 2024-09-11T22:52:20.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.685+0000 7f112d4fe740 10 client.15066 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:20.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.685+0000 7f112d4fe740 10 client.15066 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:52:20.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.685+0000 7f112d4fe740 20 client.15066 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:20.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.685+0000 7f112d4fe740 1 --2- 172.21.15.152:0/3510555173 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ece3529360 0x55ece3528070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:20.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.685+0000 7f112d4fe740 4 client.15066 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:52:20.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.685+0000 7f112d4fe740 1 -- 172.21.15.152:0/3510555173 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "status"}) -- 0x55ece340cb10 con 0x55ece3529360 2024-09-11T22:52:20.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.685+0000 7f1128ec1640 1 --2- 172.21.15.152:0/3510555173 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ece3529360 0x55ece3528070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:20.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.685+0000 7f1103fff640 10 client.15066 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:52:20.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.685+0000 7f1128ec1640 1 --2- 172.21.15.152:0/3510555173 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ece3529360 0x55ece3528070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4a12ac84f7e0ffe server_cookie=e65b9008712fba62 in_seq=0 out_seq=0 2024-09-11T22:52:20.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.686+0000 7f1103fff640 1 -- 172.21.15.152:0/3510555173 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ece340cb10 con 0x55ece3529360 2024-09-11T22:52:20.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.686+0000 7f1103fff640 10 client.15066 handle_command_reply: tid=0 2024-09-11T22:52:20.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.686+0000 7f112d4fe740 1 client.15066 shutdown 2024-09-11T22:52:20.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.686+0000 7f112d4fe740 1 -- 172.21.15.152:0/3510555173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece349f2f0 msgr2=0x55ece34bf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:20.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.686+0000 7f112d4fe740 1 --2- 172.21.15.152:0/3510555173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece349f2f0 0x55ece34bf3c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1110024ee0 tx=0x7f111000ca60 comp rx=0 tx=0).stop 2024-09-11T22:52:20.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.686+0000 7f112d4fe740 1 -- 172.21.15.152:0/3510555173 shutdown_connections 2024-09-11T22:52:20.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.686+0000 7f112d4fe740 1 --2- 172.21.15.152:0/3510555173 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ece3529360 0x55ece3528070 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.686+0000 7f112d4fe740 1 --2- 172.21.15.152:0/3510555173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ece34a9bb0 0x55ece34bf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.686+0000 7f112d4fe740 1 --2- 172.21.15.152:0/3510555173 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ece349f2f0 0x55ece34bf3c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.686+0000 7f1103fff640 0 client.15066 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:52:20.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.686+0000 7f112d4fe740 1 --2- 172.21.15.152:0/3510555173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ece349e9d0 0x55ece34c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.686+0000 7f112d4fe740 1 -- 172.21.15.152:0/3510555173 >> 172.21.15.152:0/3510555173 conn(0x55ece349af60 msgr2=0x55ece34a5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:20.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.686+0000 7f112d4fe740 1 -- 172.21.15.152:0/3510555173 shutdown_connections 2024-09-11T22:52:20.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.686+0000 7f112d4fe740 1 -- 172.21.15.152:0/3510555173 wait complete. 2024-09-11T22:52:20.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.686+0000 7f112d4fe740 20 client.15066 trim_cache size 0 max 16384 2024-09-11T22:52:20.698 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:52:20.698 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:52:20.698 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 0, 2024-09-11T22:52:20.698 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 24455, 2024-09-11T22:52:20.698 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-09-11T22:52:20.698 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-09-11T22:52:20.698 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-09-11T22:52:20.698 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 85.532546733999993, 2024-09-11T22:52:20.699 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 27, 2024-09-11T22:52:20.699 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 83, 2024-09-11T22:52:20.699 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T22:52:20.699 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 97.641090446000007 2024-09-11T22:52:20.699 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:52:20.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.687+0000 7f112b125640 1 -- 172.21.15.152:0/3846185089 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f11140a2c90 msgr2=0x7f11140a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:20.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.687+0000 7f112b125640 1 --2- 172.21.15.152:0/3846185089 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f11140a2c90 0x7f11140a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f111c014750 tx=0x7f111c004ab0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.687+0000 7f112b125640 1 -- 172.21.15.152:0/3846185089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1124102da0 msgr2=0x7f1124198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:20.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.687+0000 7f112b125640 1 --2- 172.21.15.152:0/3846185089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1124102da0 0x7f1124198420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1118005ab0 tx=0x7f11180042f0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.687+0000 7f112b125640 1 -- 172.21.15.152:0/3846185089 shutdown_connections 2024-09-11T22:52:20.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.687+0000 7f112b125640 1 --2- 172.21.15.152:0/3846185089 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f11140a2c90 0x7f11140a5150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.687+0000 7f112b125640 1 --2- 172.21.15.152:0/3846185089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f112410c180 0x7f112419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.687+0000 7f112b125640 1 --2- 172.21.15.152:0/3846185089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11241036c0 0x7f1124198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.687+0000 7f112b125640 1 --2- 172.21.15.152:0/3846185089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1124102da0 0x7f1124198420 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:20.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.687+0000 7f112b125640 1 -- 172.21.15.152:0/3846185089 >> 172.21.15.152:0/3846185089 conn(0x7f11240fc7a0 msgr2=0x7f11240fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:20.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.687+0000 7f112b125640 1 -- 172.21.15.152:0/3846185089 shutdown_connections 2024-09-11T22:52:20.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:20.687+0000 7f112b125640 1 -- 172.21.15.152:0/3846185089 wait complete. 2024-09-11T22:52:21.471 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T22:52:21.471 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T22:52:21.481 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi152.front.sepia.ceph.com 2024-09-11T22:52:21.481 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-11T22:52:21.725 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:22.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:21 smithi152 ceph-mon[31181]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:52:22.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:21 smithi191 ceph-mon[35211]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:52:22.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:21 smithi204 ceph-mon[35454]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:52:22.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.283+0000 7fd78cf8b640 1 -- 172.21.15.152:0/2480260385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd788102690 msgr2=0x7fd788108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:22.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.283+0000 7fd78cf8b640 1 --2- 172.21.15.152:0/2480260385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd788102690 0x7fd788108f20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd780009ca0 tx=0x7fd78002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:52:22.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.284+0000 7fd78cf8b640 1 -- 172.21.15.152:0/2480260385 shutdown_connections 2024-09-11T22:52:22.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.284+0000 7fd78cf8b640 1 --2- 172.21.15.152:0/2480260385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd788102690 0x7fd788108f20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:22.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.284+0000 7fd78cf8b640 1 --2- 172.21.15.152:0/2480260385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd788101cd0 0x7fd788102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:22.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.284+0000 7fd78cf8b640 1 --2- 172.21.15.152:0/2480260385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd788100ad0 0x7fd788100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:22.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.284+0000 7fd78cf8b640 1 -- 172.21.15.152:0/2480260385 >> 172.21.15.152:0/2480260385 conn(0x7fd7880fc280 msgr2=0x7fd7880fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:22.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.284+0000 7fd78cf8b640 1 -- 172.21.15.152:0/2480260385 shutdown_connections 2024-09-11T22:52:22.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.284+0000 7fd78cf8b640 1 -- 172.21.15.152:0/2480260385 wait complete. 2024-09-11T22:52:22.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.284+0000 7fd78cf8b640 1 Processor -- start 2024-09-11T22:52:22.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.284+0000 7fd78cf8b640 1 -- start start 2024-09-11T22:52:22.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd78cf8b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd788100ad0 0x7fd78806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:22.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd78cf8b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd788101cd0 0x7fd78819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:22.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd78cf8b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd788102690 0x7fd78819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:22.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd78cf8b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd78810c750 con 0x7fd788101cd0 2024-09-11T22:52:22.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd78cf8b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd78810c5d0 con 0x7fd788100ad0 2024-09-11T22:52:22.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd78cf8b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd78810c8d0 con 0x7fd788102690 2024-09-11T22:52:22.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd785d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd788101cd0 0x7fd78819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:22.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd785d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd788101cd0 0x7fd78819cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57790/0 (socket says 172.21.15.152:57790) 2024-09-11T22:52:22.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd785d9b640 1 -- 172.21.15.152:0/2258338603 learned_addr learned my addr 172.21.15.152:0/2258338603 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:22.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd78659c640 1 --2- 172.21.15.152:0/2258338603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd788100ad0 0x7fd78806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:22.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd786d9d640 1 --2- 172.21.15.152:0/2258338603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd788102690 0x7fd78819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:22.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd785d9b640 1 -- 172.21.15.152:0/2258338603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd788102690 msgr2=0x7fd78819f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:22.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd785d9b640 1 --2- 172.21.15.152:0/2258338603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd788102690 0x7fd78819f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:22.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd785d9b640 1 -- 172.21.15.152:0/2258338603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd788100ad0 msgr2=0x7fd78806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:22.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd785d9b640 1 --2- 172.21.15.152:0/2258338603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd788100ad0 0x7fd78806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:22.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd785d9b640 1 -- 172.21.15.152:0/2258338603 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78819fad0 con 0x7fd788101cd0 2024-09-11T22:52:22.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd785d9b640 1 --2- 172.21.15.152:0/2258338603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd788101cd0 0x7fd78819cba0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00d770 tx=0x7fd77c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:22.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd7777fe640 1 -- 172.21.15.152:0/2258338603 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c017070 con 0x7fd788101cd0 2024-09-11T22:52:22.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.285+0000 7fd78cf8b640 1 -- 172.21.15.152:0/2258338603 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78819fd60 con 0x7fd788101cd0 2024-09-11T22:52:22.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.286+0000 7fd7777fe640 1 -- 172.21.15.152:0/2258338603 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd77c005500 con 0x7fd788101cd0 2024-09-11T22:52:22.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.286+0000 7fd7777fe640 1 -- 172.21.15.152:0/2258338603 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c004e00 con 0x7fd788101cd0 2024-09-11T22:52:22.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.286+0000 7fd78cf8b640 1 -- 172.21.15.152:0/2258338603 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7881a0240 con 0x7fd788101cd0 2024-09-11T22:52:22.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.287+0000 7fd78cf8b640 1 -- 172.21.15.152:0/2258338603 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd788107e50 con 0x7fd788101cd0 2024-09-11T22:52:22.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.288+0000 7fd7777fe640 1 -- 172.21.15.152:0/2258338603 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd77c004fa0 con 0x7fd788101cd0 2024-09-11T22:52:22.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.288+0000 7fd7777fe640 1 --2- 172.21.15.152:0/2258338603 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd7700a2c10 0x7fd7700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:22.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.288+0000 7fd78659c640 1 --2- 172.21.15.152:0/2258338603 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd7700a2c10 0x7fd7700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:22.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.289+0000 7fd7777fe640 1 -- 172.21.15.152:0/2258338603 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fd77c0cef40 con 0x7fd788101cd0 2024-09-11T22:52:22.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.289+0000 7fd78659c640 1 --2- 172.21.15.152:0/2258338603 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd7700a2c10 0x7fd7700a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd788101b30 tx=0x7fd77800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:22.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.292+0000 7fd7777fe640 1 -- 172.21.15.152:0/2258338603 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd77c09b720 con 0x7fd788101cd0 2024-09-11T22:52:22.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.482+0000 7fd78cf8b640 1 -- 172.21.15.152:0/2258338603 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fd788062630 con 0x7fd7700a2c10 2024-09-11T22:52:22.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.483+0000 7fd7777fe640 1 -- 172.21.15.152:0/2258338603 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fd788062630 con 0x7fd7700a2c10 2024-09-11T22:52:22.484 INFO:teuthology.orchestra.run.smithi152.stdout:Backend: cephadm 2024-09-11T22:52:22.484 INFO:teuthology.orchestra.run.smithi152.stdout:Available: Yes 2024-09-11T22:52:22.484 INFO:teuthology.orchestra.run.smithi152.stdout:Paused: No 2024-09-11T22:52:22.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.485+0000 7fd78cf8b640 1 -- 172.21.15.152:0/2258338603 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd7700a2c10 msgr2=0x7fd7700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.485+0000 7fd78cf8b640 1 --2- 172.21.15.152:0/2258338603 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd7700a2c10 0x7fd7700a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd788101b30 tx=0x7fd77800b390 comp rx=0 tx=0).stop 2024-09-11T22:52:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.485+0000 7fd78cf8b640 1 -- 172.21.15.152:0/2258338603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd788101cd0 msgr2=0x7fd78819cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.485+0000 7fd78cf8b640 1 --2- 172.21.15.152:0/2258338603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd788101cd0 0x7fd78819cba0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00d770 tx=0x7fd77c00dc30 comp rx=0 tx=0).stop 2024-09-11T22:52:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.485+0000 7fd78cf8b640 1 -- 172.21.15.152:0/2258338603 shutdown_connections 2024-09-11T22:52:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.485+0000 7fd78cf8b640 1 --2- 172.21.15.152:0/2258338603 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd7700a2c10 0x7fd7700a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.485+0000 7fd78cf8b640 1 --2- 172.21.15.152:0/2258338603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd788102690 0x7fd78819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.485+0000 7fd78cf8b640 1 --2- 172.21.15.152:0/2258338603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd788101cd0 0x7fd78819cba0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.485+0000 7fd78cf8b640 1 --2- 172.21.15.152:0/2258338603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd788100ad0 0x7fd78806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.485+0000 7fd78cf8b640 1 -- 172.21.15.152:0/2258338603 >> 172.21.15.152:0/2258338603 conn(0x7fd7880fc280 msgr2=0x7fd788105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.485+0000 7fd78cf8b640 1 -- 172.21.15.152:0/2258338603 shutdown_connections 2024-09-11T22:52:22.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:22.485+0000 7fd78cf8b640 1 -- 172.21.15.152:0/2258338603 wait complete. 2024-09-11T22:52:23.205 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-11T22:52:23.458 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:24.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.003+0000 7fb56ede1640 1 -- 172.21.15.152:0/1038414617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5681037c0 msgr2=0x7fb56810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:24.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.003+0000 7fb56ede1640 1 --2- 172.21.15.152:0/1038414617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5681037c0 0x7fb56810a050 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb560009ca0 tx=0x7fb56002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:52:24.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.003+0000 7fb56ede1640 1 -- 172.21.15.152:0/1038414617 shutdown_connections 2024-09-11T22:52:24.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.003+0000 7fb56ede1640 1 --2- 172.21.15.152:0/1038414617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5681037c0 0x7fb56810a050 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:24.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.003+0000 7fb56ede1640 1 --2- 172.21.15.152:0/1038414617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb568102e00 0x7fb568103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:24.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.003+0000 7fb56ede1640 1 --2- 172.21.15.152:0/1038414617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb568101c00 0x7fb568102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:24.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.003+0000 7fb56ede1640 1 -- 172.21.15.152:0/1038414617 >> 172.21.15.152:0/1038414617 conn(0x7fb5680fd390 msgr2=0x7fb5680ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:24.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.003+0000 7fb56ede1640 1 -- 172.21.15.152:0/1038414617 shutdown_connections 2024-09-11T22:52:24.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.003+0000 7fb56ede1640 1 -- 172.21.15.152:0/1038414617 wait complete. 2024-09-11T22:52:24.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.004+0000 7fb56ede1640 1 Processor -- start 2024-09-11T22:52:24.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.004+0000 7fb56ede1640 1 -- start start 2024-09-11T22:52:24.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.004+0000 7fb56ede1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb568101c00 0x7fb568197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:24.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.004+0000 7fb56ede1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb568102e00 0x7fb5681983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:24.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.004+0000 7fb56ede1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5681037c0 0x7fb56819f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:24.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.004+0000 7fb56ede1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb56810d860 con 0x7fb568101c00 2024-09-11T22:52:24.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.004+0000 7fb56ede1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb56810d6e0 con 0x7fb568102e00 2024-09-11T22:52:24.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.004+0000 7fb56ede1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb56810d9e0 con 0x7fb5681037c0 2024-09-11T22:52:24.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.004+0000 7fb56cb7d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb568101c00 0x7fb568197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:24.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.004+0000 7fb56cb7d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb568101c00 0x7fb568197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57810/0 (socket says 172.21.15.152:57810) 2024-09-11T22:52:24.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.004+0000 7fb56cb7d640 1 -- 172.21.15.152:0/4163237399 learned_addr learned my addr 172.21.15.152:0/4163237399 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:24.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.004+0000 7fb567fff640 1 --2- 172.21.15.152:0/4163237399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb568102e00 0x7fb5681983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:24.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.005+0000 7fb56d37e640 1 --2- 172.21.15.152:0/4163237399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5681037c0 0x7fb56819f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:24.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.005+0000 7fb56cb7d640 1 -- 172.21.15.152:0/4163237399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5681037c0 msgr2=0x7fb56819f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:24.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.005+0000 7fb56cb7d640 1 --2- 172.21.15.152:0/4163237399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5681037c0 0x7fb56819f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:24.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.005+0000 7fb56cb7d640 1 -- 172.21.15.152:0/4163237399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb568102e00 msgr2=0x7fb5681983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:24.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.005+0000 7fb56cb7d640 1 --2- 172.21.15.152:0/4163237399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb568102e00 0x7fb5681983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:24.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.005+0000 7fb56cb7d640 1 -- 172.21.15.152:0/4163237399 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56819fab0 con 0x7fb568101c00 2024-09-11T22:52:24.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.005+0000 7fb56d37e640 1 --2- 172.21.15.152:0/4163237399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5681037c0 0x7fb56819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:52:24.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.005+0000 7fb56cb7d640 1 --2- 172.21.15.152:0/4163237399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb568101c00 0x7fb568197e80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00d950 tx=0x7fb55c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:24.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.005+0000 7fb565ffb640 1 -- 172.21.15.152:0/4163237399 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c017020 con 0x7fb568101c00 2024-09-11T22:52:24.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.005+0000 7fb565ffb640 1 -- 172.21.15.152:0/4163237399 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb55c009c30 con 0x7fb568101c00 2024-09-11T22:52:24.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.005+0000 7fb56ede1640 1 -- 172.21.15.152:0/4163237399 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb56819fda0 con 0x7fb568101c00 2024-09-11T22:52:24.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.005+0000 7fb565ffb640 1 -- 172.21.15.152:0/4163237399 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c00f740 con 0x7fb568101c00 2024-09-11T22:52:24.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.005+0000 7fb56ede1640 1 -- 172.21.15.152:0/4163237399 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5681a0200 con 0x7fb568101c00 2024-09-11T22:52:24.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.006+0000 7fb56ede1640 1 -- 172.21.15.152:0/4163237399 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb568109000 con 0x7fb568101c00 2024-09-11T22:52:24.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.007+0000 7fb565ffb640 1 -- 172.21.15.152:0/4163237399 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb55c01a020 con 0x7fb568101c00 2024-09-11T22:52:24.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.008+0000 7fb565ffb640 1 --2- 172.21.15.152:0/4163237399 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fb5580a2c40 0x7fb5580a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:24.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.008+0000 7fb567fff640 1 --2- 172.21.15.152:0/4163237399 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fb5580a2c40 0x7fb5580a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:24.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.008+0000 7fb565ffb640 1 -- 172.21.15.152:0/4163237399 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fb55c0ceb80 con 0x7fb568101c00 2024-09-11T22:52:24.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.008+0000 7fb567fff640 1 --2- 172.21.15.152:0/4163237399 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fb5580a2c40 0x7fb5580a5100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb5681993a0 tx=0x7fb55400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:24.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.012+0000 7fb565ffb640 1 -- 172.21.15.152:0/4163237399 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb55c021070 con 0x7fb568101c00 2024-09-11T22:52:24.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:23 smithi152 ceph-mon[31181]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:52:24.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:23 smithi152 ceph-mon[31181]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T22:52:24.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.189+0000 7fb56ede1640 1 -- 172.21.15.152:0/4163237399 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fb568063020 con 0x7fb5580a2c40 2024-09-11T22:52:24.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.201+0000 7fb565ffb640 1 -- 172.21.15.152:0/4163237399 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fb568063020 con 0x7fb5580a2c40 2024-09-11T22:52:24.202 INFO:teuthology.orchestra.run.smithi152.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-11T22:52:24.203 INFO:teuthology.orchestra.run.smithi152.stdout:mds.a smithi152 running (104s) 22s ago 103s 17.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 6c6fb2d53bc1 2024-09-11T22:52:24.203 INFO:teuthology.orchestra.run.smithi152.stdout:mds.b smithi191 running (101s) 35s ago 101s 25.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 d511cfc17a7e 2024-09-11T22:52:24.203 INFO:teuthology.orchestra.run.smithi152.stdout:mds.c smithi204 running (116s) 35s ago 115s 16.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 4e76b0e17639 2024-09-11T22:52:24.203 INFO:teuthology.orchestra.run.smithi152.stdout:mds.d smithi152 running (2m) 22s ago 2m 16.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 f8beebbb7bf4 2024-09-11T22:52:24.203 INFO:teuthology.orchestra.run.smithi152.stdout:mds.e smithi191 running (2m) 35s ago 2m 16.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 4f2b9716349a 2024-09-11T22:52:24.203 INFO:teuthology.orchestra.run.smithi152.stdout:mds.f smithi204 running (118s) 35s ago 118s 20.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 57177cca0c8d 2024-09-11T22:52:24.203 INFO:teuthology.orchestra.run.smithi152.stdout:mds.g smithi152 running (107s) 22s ago 107s 19.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 208d85f65604 2024-09-11T22:52:24.203 INFO:teuthology.orchestra.run.smithi152.stdout:mds.h smithi191 running (113s) 35s ago 113s 17.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 348b5c3bc707 2024-09-11T22:52:24.203 INFO:teuthology.orchestra.run.smithi152.stdout:mds.i smithi204 running (110s) 35s ago 110s 20.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 fd6b87b2557e 2024-09-11T22:52:24.203 INFO:teuthology.orchestra.run.smithi152.stdout:mds.j smithi152 running (2m) 22s ago 2m 16.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 829e5f06a64d 2024-09-11T22:52:24.203 INFO:teuthology.orchestra.run.smithi152.stdout:mds.k smithi191 running (2m) 35s ago 2m 18.3M - 19.1.1-298-gb53b6d3e 54467abc55f8 c100af8ff245 2024-09-11T22:52:24.203 INFO:teuthology.orchestra.run.smithi152.stdout:mds.l smithi204 running (2m) 35s ago 2m 16.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 122efa3fbe8b 2024-09-11T22:52:24.204 INFO:teuthology.orchestra.run.smithi152.stdout:mgr.x smithi152 *:9283,8765 running (9m) 22s ago 9m 508M - 19.1.1-298-gb53b6d3e 54467abc55f8 4c789e59e2f4 2024-09-11T22:52:24.204 INFO:teuthology.orchestra.run.smithi152.stdout:mgr.y smithi191 *:8443,8765 running (7m) 35s ago 7m 444M - 19.1.1-298-gb53b6d3e 54467abc55f8 d4400e87ae3b 2024-09-11T22:52:24.204 INFO:teuthology.orchestra.run.smithi152.stdout:mgr.z smithi204 *:8443,8765 running (7m) 35s ago 7m 441M - 19.1.1-298-gb53b6d3e 54467abc55f8 88afd8e36c63 2024-09-11T22:52:24.204 INFO:teuthology.orchestra.run.smithi152.stdout:mon.a smithi152 running (9m) 22s ago 9m 50.7M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 316177fd5605 2024-09-11T22:52:24.204 INFO:teuthology.orchestra.run.smithi152.stdout:mon.b smithi191 running (7m) 35s ago 7m 39.2M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 ee3efee9ccad 2024-09-11T22:52:24.204 INFO:teuthology.orchestra.run.smithi152.stdout:mon.c smithi204 running (7m) 35s ago 7m 39.8M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 eebdd83eb921 2024-09-11T22:52:24.204 INFO:teuthology.orchestra.run.smithi152.stdout:osd.0 smithi152 running (7m) 22s ago 7m 58.4M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 adc0ccdef486 2024-09-11T22:52:24.204 INFO:teuthology.orchestra.run.smithi152.stdout:osd.1 smithi191 running (6m) 35s ago 6m 45.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 1aaee7eebc3d 2024-09-11T22:52:24.204 INFO:teuthology.orchestra.run.smithi152.stdout:osd.2 smithi204 running (6m) 35s ago 6m 57.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 24e3b0ff1398 2024-09-11T22:52:24.204 INFO:teuthology.orchestra.run.smithi152.stdout:osd.3 smithi152 running (5m) 22s ago 5m 43.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 b4ea50a87771 2024-09-11T22:52:24.204 INFO:teuthology.orchestra.run.smithi152.stdout:osd.4 smithi191 running (5m) 35s ago 5m 53.5M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 fb4ee4c5c8c2 2024-09-11T22:52:24.204 INFO:teuthology.orchestra.run.smithi152.stdout:osd.5 smithi204 running (5m) 35s ago 5m 55.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 0d2c7069792c 2024-09-11T22:52:24.204 INFO:teuthology.orchestra.run.smithi152.stdout:osd.6 smithi152 running (4m) 22s ago 4m 42.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 1dfe07878656 2024-09-11T22:52:24.205 INFO:teuthology.orchestra.run.smithi152.stdout:osd.7 smithi191 running (4m) 35s ago 4m 54.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 438878d3e765 2024-09-11T22:52:24.205 INFO:teuthology.orchestra.run.smithi152.stdout:osd.8 smithi204 running (3m) 35s ago 3m 42.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 8be7291c9836 2024-09-11T22:52:24.205 INFO:teuthology.orchestra.run.smithi152.stdout:osd.9 smithi152 running (3m) 22s ago 3m 52.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 64e13f18f496 2024-09-11T22:52:24.205 INFO:teuthology.orchestra.run.smithi152.stdout:osd.10 smithi191 running (2m) 35s ago 2m 42.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 d9909006d101 2024-09-11T22:52:24.205 INFO:teuthology.orchestra.run.smithi152.stdout:osd.11 smithi204 running (2m) 35s ago 2m 51.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 e9a4d88e1eb1 2024-09-11T22:52:24.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.204+0000 7fb56ede1640 1 -- 172.21.15.152:0/4163237399 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fb5580a2c40 msgr2=0x7fb5580a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.204+0000 7fb56ede1640 1 --2- 172.21.15.152:0/4163237399 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fb5580a2c40 0x7fb5580a5100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb5681993a0 tx=0x7fb55400a2d0 comp rx=0 tx=0).stop 2024-09-11T22:52:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.204+0000 7fb56ede1640 1 -- 172.21.15.152:0/4163237399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb568101c00 msgr2=0x7fb568197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.204+0000 7fb56ede1640 1 --2- 172.21.15.152:0/4163237399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb568101c00 0x7fb568197e80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00d950 tx=0x7fb55c00de10 comp rx=0 tx=0).stop 2024-09-11T22:52:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.204+0000 7fb56ede1640 1 -- 172.21.15.152:0/4163237399 shutdown_connections 2024-09-11T22:52:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.204+0000 7fb56ede1640 1 --2- 172.21.15.152:0/4163237399 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fb5580a2c40 0x7fb5580a5100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.204+0000 7fb56ede1640 1 --2- 172.21.15.152:0/4163237399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5681037c0 0x7fb56819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.204+0000 7fb56ede1640 1 --2- 172.21.15.152:0/4163237399 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb568102e00 0x7fb5681983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.204+0000 7fb56ede1640 1 --2- 172.21.15.152:0/4163237399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb568101c00 0x7fb568197e80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.204+0000 7fb56ede1640 1 -- 172.21.15.152:0/4163237399 >> 172.21.15.152:0/4163237399 conn(0x7fb5680fd390 msgr2=0x7fb5681061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.204+0000 7fb56ede1640 1 -- 172.21.15.152:0/4163237399 shutdown_connections 2024-09-11T22:52:24.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:24.204+0000 7fb56ede1640 1 -- 172.21.15.152:0/4163237399 wait complete. 2024-09-11T22:52:24.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:23 smithi191 ceph-mon[35211]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:52:24.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:23 smithi191 ceph-mon[35211]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T22:52:24.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:23 smithi204 ceph-mon[35454]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:52:24.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:23 smithi204 ceph-mon[35454]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T22:52:25.091 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-11T22:52:25.342 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:25.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.911+0000 7f76e77bb640 1 -- 172.21.15.152:0/389826529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76e01026d0 msgr2=0x7f76e0108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:25.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.911+0000 7f76e77bb640 1 --2- 172.21.15.152:0/389826529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76e01026d0 0x7f76e0108f60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009ca0 tx=0x7f76d802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:52:25.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.911+0000 7f76e77bb640 1 -- 172.21.15.152:0/389826529 shutdown_connections 2024-09-11T22:52:25.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.911+0000 7f76e77bb640 1 --2- 172.21.15.152:0/389826529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76e01026d0 0x7f76e0108f60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:25.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.911+0000 7f76e77bb640 1 --2- 172.21.15.152:0/389826529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e0101d10 0x7f76e0102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:25.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.911+0000 7f76e77bb640 1 --2- 172.21.15.152:0/389826529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76e0100b10 0x7f76e0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:25.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.911+0000 7f76e77bb640 1 -- 172.21.15.152:0/389826529 >> 172.21.15.152:0/389826529 conn(0x7f76e00fc2a0 msgr2=0x7f76e00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:25.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.911+0000 7f76e77bb640 1 -- 172.21.15.152:0/389826529 shutdown_connections 2024-09-11T22:52:25.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.911+0000 7f76e77bb640 1 -- 172.21.15.152:0/389826529 wait complete. 2024-09-11T22:52:25.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e77bb640 1 Processor -- start 2024-09-11T22:52:25.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e77bb640 1 -- start start 2024-09-11T22:52:25.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e77bb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76e0100b10 0x7f76e0104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:25.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e77bb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e0101d10 0x7f76e01030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:25.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e77bb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76e01026d0 0x7f76e0103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:25.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e77bb640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f76e010c770 con 0x7f76e0100b10 2024-09-11T22:52:25.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e77bb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f76e010c5f0 con 0x7f76e01026d0 2024-09-11T22:52:25.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e77bb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f76e010c8f0 con 0x7f76e0101d10 2024-09-11T22:52:25.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e5557640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76e0100b10 0x7f76e0104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:25.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e5557640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76e0100b10 0x7f76e0104aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57834/0 (socket says 172.21.15.152:57834) 2024-09-11T22:52:25.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e5557640 1 -- 172.21.15.152:0/1002320714 learned_addr learned my addr 172.21.15.152:0/1002320714 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:25.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e4d56640 1 --2- 172.21.15.152:0/1002320714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e0101d10 0x7f76e01030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:25.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e5d58640 1 --2- 172.21.15.152:0/1002320714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76e01026d0 0x7f76e0103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:25.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e5557640 1 -- 172.21.15.152:0/1002320714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e0101d10 msgr2=0x7f76e01030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:25.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e5557640 1 --2- 172.21.15.152:0/1002320714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e0101d10 0x7f76e01030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:25.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e5557640 1 -- 172.21.15.152:0/1002320714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76e01026d0 msgr2=0x7f76e0103630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:25.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e5557640 1 --2- 172.21.15.152:0/1002320714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76e01026d0 0x7f76e0103630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:25.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e5557640 1 -- 172.21.15.152:0/1002320714 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e0103ef0 con 0x7f76e0100b10 2024-09-11T22:52:25.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e5d58640 1 --2- 172.21.15.152:0/1002320714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76e01026d0 0x7f76e0103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:52:25.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.912+0000 7f76e5557640 1 --2- 172.21.15.152:0/1002320714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76e0100b10 0x7f76e0104aa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00c970 tx=0x7f76dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:25.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.913+0000 7f76d67fc640 1 -- 172.21.15.152:0/1002320714 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc004220 con 0x7f76e0100b10 2024-09-11T22:52:25.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.913+0000 7f76d67fc640 1 -- 172.21.15.152:0/1002320714 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76dc0043c0 con 0x7f76e0100b10 2024-09-11T22:52:25.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.913+0000 7f76e77bb640 1 -- 172.21.15.152:0/1002320714 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e01a1ff0 con 0x7f76e0100b10 2024-09-11T22:52:25.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.913+0000 7f76d67fc640 1 -- 172.21.15.152:0/1002320714 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc00f450 con 0x7f76e0100b10 2024-09-11T22:52:25.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.913+0000 7f76e77bb640 1 -- 172.21.15.152:0/1002320714 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e01a24a0 con 0x7f76e0100b10 2024-09-11T22:52:25.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.914+0000 7f76e77bb640 1 -- 172.21.15.152:0/1002320714 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e01a2730 con 0x7f76e0100b10 2024-09-11T22:52:25.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.915+0000 7f76d67fc640 1 -- 172.21.15.152:0/1002320714 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f76dc0079b0 con 0x7f76e0100b10 2024-09-11T22:52:25.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.915+0000 7f76d67fc640 1 --2- 172.21.15.152:0/1002320714 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f76cc0a2c00 0x7f76cc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:25.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.915+0000 7f76e4d56640 1 --2- 172.21.15.152:0/1002320714 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f76cc0a2c00 0x7f76cc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:25.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.915+0000 7f76d67fc640 1 -- 172.21.15.152:0/1002320714 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f76dc0cede0 con 0x7f76e0100b10 2024-09-11T22:52:25.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.916+0000 7f76e4d56640 1 --2- 172.21.15.152:0/1002320714 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f76cc0a2c00 0x7f76cc0a50c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f76e0104820 tx=0x7f76d000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:25.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:25.919+0000 7f76d67fc640 1 -- 172.21.15.152:0/1002320714 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76dc014030 con 0x7f76e0100b10 2024-09-11T22:52:26.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:26.095+0000 7f76e77bb640 1 -- 172.21.15.152:0/1002320714 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f76e01a2900 con 0x7f76cc0a2c00 2024-09-11T22:52:26.097 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:25 smithi152 ceph-mon[31181]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:52:26.097 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:25 smithi152 ceph-mon[31181]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:52:26.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:26.101+0000 7f76d67fc640 1 -- 172.21.15.152:0/1002320714 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f76e01a2900 con 0x7f76cc0a2c00 2024-09-11T22:52:26.102 INFO:teuthology.orchestra.run.smithi152.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-11T22:52:26.102 INFO:teuthology.orchestra.run.smithi152.stdout:mds.all 12/12 37s ago 2m smithi152=a;smithi152=d;smithi152=g;smithi152=j;smithi191=b;smithi191=e;smithi191=h;smithi191=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-09-11T22:52:26.102 INFO:teuthology.orchestra.run.smithi152.stdout:mgr 3/3 37s ago 7m smithi152=x;smithi191=y;smithi204=z;count:3 2024-09-11T22:52:26.102 INFO:teuthology.orchestra.run.smithi152.stdout:mon 3/3 37s ago 8m smithi152:172.21.15.152=a;smithi191:172.21.15.191=b;smithi204:172.21.15.204=c;count:3 2024-09-11T22:52:26.102 INFO:teuthology.orchestra.run.smithi152.stdout:osd 12 37s ago - 2024-09-11T22:52:26.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:26.103+0000 7f76e77bb640 1 -- 172.21.15.152:0/1002320714 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f76cc0a2c00 msgr2=0x7f76cc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:26.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:26.103+0000 7f76e77bb640 1 --2- 172.21.15.152:0/1002320714 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f76cc0a2c00 0x7f76cc0a50c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f76e0104820 tx=0x7f76d000a4c0 comp rx=0 tx=0).stop 2024-09-11T22:52:26.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:26.103+0000 7f76e77bb640 1 -- 172.21.15.152:0/1002320714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76e0100b10 msgr2=0x7f76e0104aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:26.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:26.103+0000 7f76e77bb640 1 --2- 172.21.15.152:0/1002320714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76e0100b10 0x7f76e0104aa0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f76dc00c970 tx=0x7f76dc00ce30 comp rx=0 tx=0).stop 2024-09-11T22:52:26.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:26.103+0000 7f76e77bb640 1 -- 172.21.15.152:0/1002320714 shutdown_connections 2024-09-11T22:52:26.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:26.103+0000 7f76e77bb640 1 --2- 172.21.15.152:0/1002320714 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f76cc0a2c00 0x7f76cc0a50c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:26.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:26.103+0000 7f76e77bb640 1 --2- 172.21.15.152:0/1002320714 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76e01026d0 0x7f76e0103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:26.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:26.103+0000 7f76e77bb640 1 --2- 172.21.15.152:0/1002320714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e0101d10 0x7f76e01030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:26.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:26.103+0000 7f76e77bb640 1 --2- 172.21.15.152:0/1002320714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f76e0100b10 0x7f76e0104aa0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:26.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:26.103+0000 7f76e77bb640 1 -- 172.21.15.152:0/1002320714 >> 172.21.15.152:0/1002320714 conn(0x7f76e00fc2a0 msgr2=0x7f76e00fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:26.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:26.103+0000 7f76e77bb640 1 -- 172.21.15.152:0/1002320714 shutdown_connections 2024-09-11T22:52:26.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:26.103+0000 7f76e77bb640 1 -- 172.21.15.152:0/1002320714 wait complete. 2024-09-11T22:52:26.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:25 smithi191 ceph-mon[35211]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:52:26.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:25 smithi191 ceph-mon[35211]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:52:26.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:25 smithi204 ceph-mon[35454]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:52:26.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:25 smithi204 ceph-mon[35454]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:52:26.805 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-11T22:52:27.057 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:27.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:26 smithi152 ceph-mon[31181]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T22:52:27.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:26 smithi191 ceph-mon[35211]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T22:52:27.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:26 smithi204 ceph-mon[35454]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T22:52:27.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.608+0000 7f19f3070640 1 -- 172.21.15.152:0/651021236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19ec102e00 msgr2=0x7f19ec103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:27.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.608+0000 7f19f3070640 1 --2- 172.21.15.152:0/651021236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19ec102e00 0x7f19ec103280 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009900 tx=0x7f19d802a410 comp rx=0 tx=0).stop 2024-09-11T22:52:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.608+0000 7f19f3070640 1 -- 172.21.15.152:0/651021236 shutdown_connections 2024-09-11T22:52:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.608+0000 7f19f3070640 1 --2- 172.21.15.152:0/651021236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19ec1037c0 0x7f19ec10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.608+0000 7f19f3070640 1 --2- 172.21.15.152:0/651021236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19ec102e00 0x7f19ec103280 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.608+0000 7f19f3070640 1 --2- 172.21.15.152:0/651021236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19ec101c00 0x7f19ec102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.608+0000 7f19f3070640 1 -- 172.21.15.152:0/651021236 >> 172.21.15.152:0/651021236 conn(0x7f19ec0fd390 msgr2=0x7f19ec0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.608+0000 7f19f3070640 1 -- 172.21.15.152:0/651021236 shutdown_connections 2024-09-11T22:52:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.608+0000 7f19f3070640 1 -- 172.21.15.152:0/651021236 wait complete. 2024-09-11T22:52:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f3070640 1 Processor -- start 2024-09-11T22:52:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f3070640 1 -- start start 2024-09-11T22:52:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f3070640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19ec101c00 0x7f19ec197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f3070640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19ec102e00 0x7f19ec198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:27.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f3070640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19ec1037c0 0x7f19ec19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f3070640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec10d880 con 0x7f19ec101c00 2024-09-11T22:52:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f3070640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec10d700 con 0x7f19ec102e00 2024-09-11T22:52:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f3070640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f19ec10da00 con 0x7f19ec1037c0 2024-09-11T22:52:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f0e0c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19ec101c00 0x7f19ec197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f0e0c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19ec101c00 0x7f19ec197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57848/0 (socket says 172.21.15.152:57848) 2024-09-11T22:52:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f0e0c640 1 -- 172.21.15.152:0/584927573 learned_addr learned my addr 172.21.15.152:0/584927573 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19ebfff640 1 --2- 172.21.15.152:0/584927573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19ec102e00 0x7f19ec198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f160d640 1 --2- 172.21.15.152:0/584927573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19ec1037c0 0x7f19ec19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f0e0c640 1 -- 172.21.15.152:0/584927573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19ec1037c0 msgr2=0x7f19ec19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f0e0c640 1 --2- 172.21.15.152:0/584927573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19ec1037c0 0x7f19ec19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f0e0c640 1 -- 172.21.15.152:0/584927573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19ec102e00 msgr2=0x7f19ec198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f0e0c640 1 --2- 172.21.15.152:0/584927573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19ec102e00 0x7f19ec198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:27.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19f0e0c640 1 -- 172.21.15.152:0/584927573 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19ec19fb40 con 0x7f19ec101c00 2024-09-11T22:52:27.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.609+0000 7f19ebfff640 1 --2- 172.21.15.152:0/584927573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19ec102e00 0x7f19ec198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:27.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.610+0000 7f19f0e0c640 1 --2- 172.21.15.152:0/584927573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19ec101c00 0x7f19ec197f10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f19e000c970 tx=0x7f19e000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:27.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.610+0000 7f19e9ffb640 1 -- 172.21.15.152:0/584927573 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0004220 con 0x7f19ec101c00 2024-09-11T22:52:27.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.610+0000 7f19e9ffb640 1 -- 172.21.15.152:0/584927573 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e00043c0 con 0x7f19ec101c00 2024-09-11T22:52:27.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.610+0000 7f19f3070640 1 -- 172.21.15.152:0/584927573 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19ec19fe30 con 0x7f19ec101c00 2024-09-11T22:52:27.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.610+0000 7f19e9ffb640 1 -- 172.21.15.152:0/584927573 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e000f450 con 0x7f19ec101c00 2024-09-11T22:52:27.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.610+0000 7f19f3070640 1 -- 172.21.15.152:0/584927573 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19ec1a0290 con 0x7f19ec101c00 2024-09-11T22:52:27.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.611+0000 7f19f3070640 1 -- 172.21.15.152:0/584927573 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19ec108fb0 con 0x7f19ec101c00 2024-09-11T22:52:27.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.612+0000 7f19e9ffb640 1 -- 172.21.15.152:0/584927573 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f19e00079b0 con 0x7f19ec101c00 2024-09-11T22:52:27.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.612+0000 7f19e9ffb640 1 --2- 172.21.15.152:0/584927573 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f19dc0a2c30 0x7f19dc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:27.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.613+0000 7f19ebfff640 1 --2- 172.21.15.152:0/584927573 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f19dc0a2c30 0x7f19dc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:27.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.613+0000 7f19e9ffb640 1 -- 172.21.15.152:0/584927573 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f19e00cecd0 con 0x7f19ec101c00 2024-09-11T22:52:27.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.613+0000 7f19ebfff640 1 --2- 172.21.15.152:0/584927573 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f19dc0a2c30 0x7f19dc0a50f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f19d8000c00 tx=0x7f19d80095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:27.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.616+0000 7f19e9ffb640 1 -- 172.21.15.152:0/584927573 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19e0014030 con 0x7f19ec101c00 2024-09-11T22:52:27.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.788+0000 7f19f3070640 1 -- 172.21.15.152:0/584927573 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f19ec062630 con 0x7f19dc0a2c30 2024-09-11T22:52:27.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.790+0000 7f19e9ffb640 1 -- 172.21.15.152:0/584927573 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f19ec062630 con 0x7f19dc0a2c30 2024-09-11T22:52:27.791 INFO:teuthology.orchestra.run.smithi152.stdout:HOST ADDR LABELS STATUS 2024-09-11T22:52:27.791 INFO:teuthology.orchestra.run.smithi152.stdout:smithi152 172.21.15.152 2024-09-11T22:52:27.791 INFO:teuthology.orchestra.run.smithi152.stdout:smithi191 172.21.15.191 2024-09-11T22:52:27.791 INFO:teuthology.orchestra.run.smithi152.stdout:smithi204 172.21.15.204 2024-09-11T22:52:27.791 INFO:teuthology.orchestra.run.smithi152.stdout:3 hosts in cluster 2024-09-11T22:52:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.792+0000 7f19f3070640 1 -- 172.21.15.152:0/584927573 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f19dc0a2c30 msgr2=0x7f19dc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.792+0000 7f19f3070640 1 --2- 172.21.15.152:0/584927573 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f19dc0a2c30 0x7f19dc0a50f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f19d8000c00 tx=0x7f19d80095f0 comp rx=0 tx=0).stop 2024-09-11T22:52:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.792+0000 7f19f3070640 1 -- 172.21.15.152:0/584927573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19ec101c00 msgr2=0x7f19ec197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.792+0000 7f19f3070640 1 --2- 172.21.15.152:0/584927573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19ec101c00 0x7f19ec197f10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f19e000c970 tx=0x7f19e000ce30 comp rx=0 tx=0).stop 2024-09-11T22:52:27.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.792+0000 7f19f3070640 1 -- 172.21.15.152:0/584927573 shutdown_connections 2024-09-11T22:52:27.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.792+0000 7f19f3070640 1 --2- 172.21.15.152:0/584927573 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f19dc0a2c30 0x7f19dc0a50f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:27.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.792+0000 7f19f3070640 1 --2- 172.21.15.152:0/584927573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f19ec1037c0 0x7f19ec19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:27.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.792+0000 7f19f3070640 1 --2- 172.21.15.152:0/584927573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f19ec102e00 0x7f19ec198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:27.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.792+0000 7f19f3070640 1 --2- 172.21.15.152:0/584927573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f19ec101c00 0x7f19ec197f10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:27.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.792+0000 7f19f3070640 1 -- 172.21.15.152:0/584927573 >> 172.21.15.152:0/584927573 conn(0x7f19ec0fd390 msgr2=0x7f19ec106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:27.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.792+0000 7f19f3070640 1 -- 172.21.15.152:0/584927573 shutdown_connections 2024-09-11T22:52:27.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:27.792+0000 7f19f3070640 1 -- 172.21.15.152:0/584927573 wait complete. 2024-09-11T22:52:28.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:27 smithi152 ceph-mon[31181]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:52:28.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:27 smithi152 ceph-mon[31181]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T22:52:28.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:27 smithi191 ceph-mon[35211]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:52:28.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:27 smithi191 ceph-mon[35211]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T22:52:28.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:27 smithi204 ceph-mon[35454]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:52:28.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:27 smithi204 ceph-mon[35454]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-11T22:52:28.605 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-11T22:52:28.853 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:29.141 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:28 smithi152 ceph-mon[31181]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:52:29.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:28 smithi191 ceph-mon[35211]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:52:29.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:28 smithi204 ceph-mon[35454]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:52:29.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.369+0000 7f924e783640 1 -- 172.21.15.152:0/1968259486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9248101cd0 msgr2=0x7f9248102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:29.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.369+0000 7f924e783640 1 --2- 172.21.15.152:0/1968259486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9248101cd0 0x7f9248102150 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9230009900 tx=0x7f923002a410 comp rx=0 tx=0).stop 2024-09-11T22:52:29.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.369+0000 7f924e783640 1 -- 172.21.15.152:0/1968259486 shutdown_connections 2024-09-11T22:52:29.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.369+0000 7f924e783640 1 --2- 172.21.15.152:0/1968259486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9248102690 0x7f9248108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:29.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.369+0000 7f924e783640 1 --2- 172.21.15.152:0/1968259486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9248101cd0 0x7f9248102150 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:29.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.369+0000 7f924e783640 1 --2- 172.21.15.152:0/1968259486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9248100ad0 0x7f9248100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:29.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.369+0000 7f924e783640 1 -- 172.21.15.152:0/1968259486 >> 172.21.15.152:0/1968259486 conn(0x7f92480fc280 msgr2=0x7f92480fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:29.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.369+0000 7f924e783640 1 -- 172.21.15.152:0/1968259486 shutdown_connections 2024-09-11T22:52:29.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.369+0000 7f924e783640 1 -- 172.21.15.152:0/1968259486 wait complete. 2024-09-11T22:52:29.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.369+0000 7f924e783640 1 Processor -- start 2024-09-11T22:52:29.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.369+0000 7f924e783640 1 -- start start 2024-09-11T22:52:29.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f924e783640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9248100ad0 0x7f9248197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:29.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f924e783640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9248101cd0 0x7f9248198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:29.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f924e783640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9248102690 0x7f924819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:29.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f924e783640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f924810d8c0 con 0x7f9248100ad0 2024-09-11T22:52:29.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f924e783640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f924810d740 con 0x7f9248101cd0 2024-09-11T22:52:29.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f924e783640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f924810da40 con 0x7f9248102690 2024-09-11T22:52:29.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f9247fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9248100ad0 0x7f9248197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:29.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f9247fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9248100ad0 0x7f9248197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40004/0 (socket says 172.21.15.152:40004) 2024-09-11T22:52:29.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f9247fff640 1 -- 172.21.15.152:0/1607135231 learned_addr learned my addr 172.21.15.152:0/1607135231 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:29.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f924cd20640 1 --2- 172.21.15.152:0/1607135231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9248102690 0x7f924819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:29.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f923ffff640 1 --2- 172.21.15.152:0/1607135231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9248101cd0 0x7f9248198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:29.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f9247fff640 1 -- 172.21.15.152:0/1607135231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9248102690 msgr2=0x7f924819f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:29.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f9247fff640 1 --2- 172.21.15.152:0/1607135231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9248102690 0x7f924819f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:29.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f9247fff640 1 -- 172.21.15.152:0/1607135231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9248101cd0 msgr2=0x7f9248198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:29.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f9247fff640 1 --2- 172.21.15.152:0/1607135231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9248101cd0 0x7f9248198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:29.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f9247fff640 1 -- 172.21.15.152:0/1607135231 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924819fb10 con 0x7f9248100ad0 2024-09-11T22:52:29.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f924cd20640 1 --2- 172.21.15.152:0/1607135231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9248102690 0x7f924819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:52:29.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f9247fff640 1 --2- 172.21.15.152:0/1607135231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9248100ad0 0x7f9248197ee0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f923800c970 tx=0x7f923800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:29.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f9245ffb640 1 -- 172.21.15.152:0/1607135231 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238007c10 con 0x7f9248100ad0 2024-09-11T22:52:29.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f9245ffb640 1 -- 172.21.15.152:0/1607135231 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9238007db0 con 0x7f9248100ad0 2024-09-11T22:52:29.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f924e783640 1 -- 172.21.15.152:0/1607135231 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924819fe00 con 0x7f9248100ad0 2024-09-11T22:52:29.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f9245ffb640 1 -- 172.21.15.152:0/1607135231 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923800f940 con 0x7f9248100ad0 2024-09-11T22:52:29.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.370+0000 7f924e783640 1 -- 172.21.15.152:0/1607135231 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92481a0330 con 0x7f9248100ad0 2024-09-11T22:52:29.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.371+0000 7f924e783640 1 -- 172.21.15.152:0/1607135231 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9248107ea0 con 0x7f9248100ad0 2024-09-11T22:52:29.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.372+0000 7f9245ffb640 1 -- 172.21.15.152:0/1607135231 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f923800fae0 con 0x7f9248100ad0 2024-09-11T22:52:29.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.373+0000 7f9245ffb640 1 --2- 172.21.15.152:0/1607135231 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f92340a2c30 0x7f92340a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:29.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.373+0000 7f923ffff640 1 --2- 172.21.15.152:0/1607135231 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f92340a2c30 0x7f92340a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:29.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.374+0000 7f9245ffb640 1 -- 172.21.15.152:0/1607135231 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f92380cf3c0 con 0x7f9248100ad0 2024-09-11T22:52:29.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.374+0000 7f923ffff640 1 --2- 172.21.15.152:0/1607135231 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f92340a2c30 0x7f92340a50f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9230031610 tx=0x7f9230000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:29.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.378+0000 7f9245ffb640 1 -- 172.21.15.152:0/1607135231 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f923809c3e0 con 0x7f9248100ad0 2024-09-11T22:52:29.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.550+0000 7f924e783640 1 -- 172.21.15.152:0/1607135231 --> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f9248062630 con 0x7f92340a2c30 2024-09-11T22:52:29.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.552+0000 7f9245ffb640 1 -- 172.21.15.152:0/1607135231 <== mgr.14150 v2:172.21.15.152:6800/3438861948 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f9248062630 con 0x7f92340a2c30 2024-09-11T22:52:29.553 INFO:teuthology.orchestra.run.smithi152.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-11T22:52:29.553 INFO:teuthology.orchestra.run.smithi152.stdout:smithi152 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JT400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-11T22:52:29.553 INFO:teuthology.orchestra.run.smithi152.stdout:smithi191 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303008M400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-11T22:52:29.553 INFO:teuthology.orchestra.run.smithi152.stdout:smithi204 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GD400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-11T22:52:29.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.555+0000 7f924e783640 1 -- 172.21.15.152:0/1607135231 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f92340a2c30 msgr2=0x7f92340a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:29.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.555+0000 7f924e783640 1 --2- 172.21.15.152:0/1607135231 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f92340a2c30 0x7f92340a50f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9230031610 tx=0x7f9230000bc0 comp rx=0 tx=0).stop 2024-09-11T22:52:29.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.555+0000 7f924e783640 1 -- 172.21.15.152:0/1607135231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9248100ad0 msgr2=0x7f9248197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.555+0000 7f924e783640 1 --2- 172.21.15.152:0/1607135231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9248100ad0 0x7f9248197ee0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f923800c970 tx=0x7f923800ce30 comp rx=0 tx=0).stop 2024-09-11T22:52:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.555+0000 7f924e783640 1 -- 172.21.15.152:0/1607135231 shutdown_connections 2024-09-11T22:52:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.555+0000 7f924e783640 1 --2- 172.21.15.152:0/1607135231 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f92340a2c30 0x7f92340a50f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.555+0000 7f924e783640 1 --2- 172.21.15.152:0/1607135231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9248102690 0x7f924819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.555+0000 7f924e783640 1 --2- 172.21.15.152:0/1607135231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9248101cd0 0x7f9248198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.555+0000 7f924e783640 1 --2- 172.21.15.152:0/1607135231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9248100ad0 0x7f9248197ee0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.555+0000 7f924e783640 1 -- 172.21.15.152:0/1607135231 >> 172.21.15.152:0/1607135231 conn(0x7f92480fc280 msgr2=0x7f9248105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.555+0000 7f924e783640 1 -- 172.21.15.152:0/1607135231 shutdown_connections 2024-09-11T22:52:29.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:29.555+0000 7f924e783640 1 -- 172.21.15.152:0/1607135231 wait complete. 2024-09-11T22:52:30.245 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:29 smithi152 ceph-mon[31181]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T22:52:30.272 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T22:52:30.282 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi152.front.sepia.ceph.com 2024-09-11T22:52:30.283 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-11T22:52:30.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:29 smithi191 ceph-mon[35211]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T22:52:30.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:29 smithi204 ceph-mon[35454]: pgmap v330: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T22:52:30.529 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:30.983 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:30 smithi152 ceph-mon[31181]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:52:31.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.063+0000 7f0302017640 1 -- 172.21.15.152:0/754709280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02fc104f90 msgr2=0x7f02fc107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:31.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.063+0000 7f0302017640 1 --2- 172.21.15.152:0/754709280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02fc104f90 0x7f02fc107380 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f02e4009900 tx=0x7f02e402a410 comp rx=0 tx=0).stop 2024-09-11T22:52:31.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.063+0000 7f0302017640 1 -- 172.21.15.152:0/754709280 shutdown_connections 2024-09-11T22:52:31.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.063+0000 7f0302017640 1 --2- 172.21.15.152:0/754709280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02fc1078c0 0x7f02fc109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:31.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.063+0000 7f0302017640 1 --2- 172.21.15.152:0/754709280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02fc104f90 0x7f02fc107380 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:31.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.063+0000 7f0302017640 1 --2- 172.21.15.152:0/754709280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc102660 0x7f02fc104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:31.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.063+0000 7f0302017640 1 -- 172.21.15.152:0/754709280 >> 172.21.15.152:0/754709280 conn(0x7f02fc0fc280 msgr2=0x7f02fc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:31.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.063+0000 7f0302017640 1 -- 172.21.15.152:0/754709280 shutdown_connections 2024-09-11T22:52:31.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.063+0000 7f0302017640 1 -- 172.21.15.152:0/754709280 wait complete. 2024-09-11T22:52:31.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.064+0000 7f0302017640 1 Processor -- start 2024-09-11T22:52:31.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.064+0000 7f0302017640 1 -- start start 2024-09-11T22:52:31.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f0302017640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc102660 0x7f02fc198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:31.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f0302017640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02fc104f90 0x7f02fc1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:31.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f0302017640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02fc1078c0 0x7f02fc19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:31.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f0302017640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc10d930 con 0x7f02fc104f90 2024-09-11T22:52:31.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f0302017640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc10d7b0 con 0x7f02fc102660 2024-09-11T22:52:31.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f0302017640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc10dab0 con 0x7f02fc1078c0 2024-09-11T22:52:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f02faffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02fc104f90 0x7f02fc1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f02faffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02fc104f90 0x7f02fc1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40022/0 (socket says 172.21.15.152:40022) 2024-09-11T22:52:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f02faffd640 1 -- 172.21.15.152:0/358137796 learned_addr learned my addr 172.21.15.152:0/358137796 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f02fb7fe640 1 --2- 172.21.15.152:0/358137796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc102660 0x7f02fc198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f02fbfff640 1 --2- 172.21.15.152:0/358137796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02fc1078c0 0x7f02fc19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f02faffd640 1 -- 172.21.15.152:0/358137796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02fc1078c0 msgr2=0x7f02fc19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f02faffd640 1 --2- 172.21.15.152:0/358137796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02fc1078c0 0x7f02fc19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f02faffd640 1 -- 172.21.15.152:0/358137796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc102660 msgr2=0x7f02fc198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f02faffd640 1 --2- 172.21.15.152:0/358137796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc102660 0x7f02fc198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f02faffd640 1 -- 172.21.15.152:0/358137796 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02fc19fcb0 con 0x7f02fc104f90 2024-09-11T22:52:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f02fb7fe640 1 --2- 172.21.15.152:0/358137796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc102660 0x7f02fc198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f02faffd640 1 --2- 172.21.15.152:0/358137796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02fc104f90 0x7f02fc1985c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f02e4031610 tx=0x7f02e40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:31.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f02f8ff9640 1 -- 172.21.15.152:0/358137796 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e403f030 con 0x7f02fc104f90 2024-09-11T22:52:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f02f8ff9640 1 -- 172.21.15.152:0/358137796 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e4003ce0 con 0x7f02fc104f90 2024-09-11T22:52:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f0302017640 1 -- 172.21.15.152:0/358137796 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02fc19ff40 con 0x7f02fc104f90 2024-09-11T22:52:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f02f8ff9640 1 -- 172.21.15.152:0/358137796 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e4035070 con 0x7f02fc104f90 2024-09-11T22:52:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.065+0000 7f0302017640 1 -- 172.21.15.152:0/358137796 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02fc1a0410 con 0x7f02fc104f90 2024-09-11T22:52:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.066+0000 7f0302017640 1 -- 172.21.15.152:0/358137796 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02fc0fdd70 con 0x7f02fc104f90 2024-09-11T22:52:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.068+0000 7f02f8ff9640 1 -- 172.21.15.152:0/358137796 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f02e4042030 con 0x7f02fc104f90 2024-09-11T22:52:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.068+0000 7f02f8ff9640 1 --2- 172.21.15.152:0/358137796 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f02e80a2bb0 0x7f02e80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.068+0000 7f02fb7fe640 1 --2- 172.21.15.152:0/358137796 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f02e80a2bb0 0x7f02e80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.069+0000 7f02f8ff9640 1 -- 172.21.15.152:0/358137796 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f02e40ef960 con 0x7f02fc104f90 2024-09-11T22:52:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.069+0000 7f02fb7fe640 1 --2- 172.21.15.152:0/358137796 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f02e80a2bb0 0x7f02e80a5070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009fa0 tx=0x7f02ec009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:31.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.071+0000 7f02f8ff9640 1 -- 172.21.15.152:0/358137796 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02e40bc8f0 con 0x7f02fc104f90 2024-09-11T22:52:31.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.274+0000 7f0302017640 1 -- 172.21.15.152:0/358137796 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f02fc02d7c0 con 0x7f02fc104f90 2024-09-11T22:52:31.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.274+0000 7f02f8ff9640 1 -- 172.21.15.152:0/358137796 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3138 (secure 0 0 0) 0x7f02e40bc1f0 con 0x7f02fc104f90 2024-09-11T22:52:31.275 INFO:teuthology.orchestra.run.smithi152.stdout:e27 2024-09-11T22:52:31.275 INFO:teuthology.orchestra.run.smithi152.stdout:btime 2024-09-11T22:51:06:131335+0000 2024-09-11T22:52:31.275 INFO:teuthology.orchestra.run.smithi152.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T22:52:31.275 INFO:teuthology.orchestra.run.smithi152.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T22:52:31.275 INFO:teuthology.orchestra.run.smithi152.stdout:legacy client fscid: 1 2024-09-11T22:52:31.275 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:31.275 INFO:teuthology.orchestra.run.smithi152.stdout:Filesystem 'cephfs' (1) 2024-09-11T22:52:31.275 INFO:teuthology.orchestra.run.smithi152.stdout:fs_name cephfs 2024-09-11T22:52:31.275 INFO:teuthology.orchestra.run.smithi152.stdout:epoch 27 2024-09-11T22:52:31.276 INFO:teuthology.orchestra.run.smithi152.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-11T22:52:31.276 INFO:teuthology.orchestra.run.smithi152.stdout:created 2024-09-11T22:50:55.047332+0000 2024-09-11T22:52:31.276 INFO:teuthology.orchestra.run.smithi152.stdout:modified 2024-09-11T22:51:06.131313+0000 2024-09-11T22:52:31.276 INFO:teuthology.orchestra.run.smithi152.stdout:tableserver 0 2024-09-11T22:52:31.276 INFO:teuthology.orchestra.run.smithi152.stdout:root 0 2024-09-11T22:52:31.276 INFO:teuthology.orchestra.run.smithi152.stdout:session_timeout 300 2024-09-11T22:52:31.276 INFO:teuthology.orchestra.run.smithi152.stdout:session_autoclose 300 2024-09-11T22:52:31.276 INFO:teuthology.orchestra.run.smithi152.stdout:max_file_size 1099511627776 2024-09-11T22:52:31.276 INFO:teuthology.orchestra.run.smithi152.stdout:max_xattr_size 65536 2024-09-11T22:52:31.276 INFO:teuthology.orchestra.run.smithi152.stdout:required_client_features {} 2024-09-11T22:52:31.276 INFO:teuthology.orchestra.run.smithi152.stdout:last_failure 0 2024-09-11T22:52:31.276 INFO:teuthology.orchestra.run.smithi152.stdout:last_failure_osd_epoch 0 2024-09-11T22:52:31.277 INFO:teuthology.orchestra.run.smithi152.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T22:52:31.277 INFO:teuthology.orchestra.run.smithi152.stdout:max_mds 5 2024-09-11T22:52:31.277 INFO:teuthology.orchestra.run.smithi152.stdout:in 0,1,2,3,4 2024-09-11T22:52:31.277 INFO:teuthology.orchestra.run.smithi152.stdout:up {0=24455,1=24442,2=24434,3=24430,4=14628} 2024-09-11T22:52:31.277 INFO:teuthology.orchestra.run.smithi152.stdout:failed 2024-09-11T22:52:31.277 INFO:teuthology.orchestra.run.smithi152.stdout:damaged 2024-09-11T22:52:31.277 INFO:teuthology.orchestra.run.smithi152.stdout:stopped 2024-09-11T22:52:31.277 INFO:teuthology.orchestra.run.smithi152.stdout:data_pools [3] 2024-09-11T22:52:31.277 INFO:teuthology.orchestra.run.smithi152.stdout:metadata_pool 2 2024-09-11T22:52:31.277 INFO:teuthology.orchestra.run.smithi152.stdout:inline_data disabled 2024-09-11T22:52:31.277 INFO:teuthology.orchestra.run.smithi152.stdout:balancer 2024-09-11T22:52:31.277 INFO:teuthology.orchestra.run.smithi152.stdout:bal_rank_mask -1 2024-09-11T22:52:31.277 INFO:teuthology.orchestra.run.smithi152.stdout:standby_count_wanted 0 2024-09-11T22:52:31.278 INFO:teuthology.orchestra.run.smithi152.stdout:qdb_cluster leader: 24455 members: 24455,24442,24434,24430,14628 2024-09-11T22:52:31.278 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.b{0:24455} state up:active seq 5 addr [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:52:31.278 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.f{0:24418} state up:standby-replay seq 1 addr [v2:172.21.15.204:6834/763950562,v1:172.21.15.204:6835/763950562] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:52:31.278 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.i{1:24442} state up:active seq 9 addr [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:52:31.278 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.e{1:24407} state up:standby-replay seq 1 addr [v2:172.21.15.191:6834/2471607007,v1:172.21.15.191:6835/2471607007] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:52:31.278 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.h{2:24434} state up:active seq 10 addr [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:52:31.278 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.l{2:24397} state up:standby-replay seq 1 addr [v2:172.21.15.204:6832/2200342090,v1:172.21.15.204:6833/2200342090] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:52:31.278 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.c{3:24430} state up:active seq 10 addr [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:52:31.278 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.k{3:24377} state up:standby-replay seq 1 addr [v2:172.21.15.191:6832/2321157769,v1:172.21.15.191:6833/2321157769] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:52:31.278 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.a{4:14628} state up:active seq 8 addr [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:52:31.278 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.g{4:14616} state up:standby-replay seq 1 addr [v2:172.21.15.152:6838/97179606,v1:172.21.15.152:6839/97179606] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:52:31.278 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:31.278 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:31.279 INFO:teuthology.orchestra.run.smithi152.stdout:Standby daemons: 2024-09-11T22:52:31.279 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:31.279 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.j{-1:14553} state up:standby seq 1 addr [v2:172.21.15.152:6834/1098962542,v1:172.21.15.152:6835/1098962542] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:52:31.279 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.d{-1:14565} state up:standby seq 1 addr [v2:172.21.15.152:6836/2615590856,v1:172.21.15.152:6837/2615590856] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:52:31.279 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 27 2024-09-11T22:52:31.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.276+0000 7f0302017640 1 -- 172.21.15.152:0/358137796 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f02e80a2bb0 msgr2=0x7f02e80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:31.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.276+0000 7f0302017640 1 --2- 172.21.15.152:0/358137796 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f02e80a2bb0 0x7f02e80a5070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009fa0 tx=0x7f02ec009570 comp rx=0 tx=0).stop 2024-09-11T22:52:31.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.276+0000 7f0302017640 1 -- 172.21.15.152:0/358137796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02fc104f90 msgr2=0x7f02fc1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:31.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.276+0000 7f0302017640 1 --2- 172.21.15.152:0/358137796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02fc104f90 0x7f02fc1985c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f02e4031610 tx=0x7f02e40043b0 comp rx=0 tx=0).stop 2024-09-11T22:52:31.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.276+0000 7f0302017640 1 -- 172.21.15.152:0/358137796 shutdown_connections 2024-09-11T22:52:31.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.276+0000 7f0302017640 1 --2- 172.21.15.152:0/358137796 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f02e80a2bb0 0x7f02e80a5070 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:31.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.276+0000 7f0302017640 1 --2- 172.21.15.152:0/358137796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02fc1078c0 0x7f02fc19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:31.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.276+0000 7f0302017640 1 --2- 172.21.15.152:0/358137796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02fc104f90 0x7f02fc1985c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:31.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.276+0000 7f0302017640 1 --2- 172.21.15.152:0/358137796 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc102660 0x7f02fc198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:31.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.276+0000 7f0302017640 1 -- 172.21.15.152:0/358137796 >> 172.21.15.152:0/358137796 conn(0x7f02fc0fc280 msgr2=0x7f02fc1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:31.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.277+0000 7f0302017640 1 -- 172.21.15.152:0/358137796 shutdown_connections 2024-09-11T22:52:31.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:31.277+0000 7f0302017640 1 -- 172.21.15.152:0/358137796 wait complete. 2024-09-11T22:52:31.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:30 smithi191 ceph-mon[35211]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:52:31.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:30 smithi204 ceph-mon[35454]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:52:32.018 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-11T22:52:32.265 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:32.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:31 smithi191 ceph-mon[35211]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-11T22:52:32.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:31 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/358137796' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T22:52:32.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:31 smithi204 ceph-mon[35454]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-11T22:52:32.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:31 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/358137796' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T22:52:32.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:31 smithi152 ceph-mon[31181]: pgmap v331: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-11T22:52:32.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:31 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/358137796' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T22:52:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.773+0000 7f017ef3e640 1 -- 172.21.15.152:0/2264318268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01780ff9e0 msgr2=0x7f01780ffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.773+0000 7f017ef3e640 1 --2- 172.21.15.152:0/2264318268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01780ff9e0 0x7f01780ffde0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f016c009900 tx=0x7f016c02a410 comp rx=0 tx=0).stop 2024-09-11T22:52:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.773+0000 7f017ef3e640 1 -- 172.21.15.152:0/2264318268 shutdown_connections 2024-09-11T22:52:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.773+0000 7f017ef3e640 1 --2- 172.21.15.152:0/2264318268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01781078e0 0x7f0178109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.773+0000 7f017ef3e640 1 --2- 172.21.15.152:0/2264318268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0178100320 0x7f0178107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.773+0000 7f017ef3e640 1 --2- 172.21.15.152:0/2264318268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01780ff9e0 0x7f01780ffde0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.773+0000 7f017ef3e640 1 -- 172.21.15.152:0/2264318268 >> 172.21.15.152:0/2264318268 conn(0x7f01780fb610 msgr2=0x7f01780fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.773+0000 7f017ef3e640 1 -- 172.21.15.152:0/2264318268 shutdown_connections 2024-09-11T22:52:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.773+0000 7f017ef3e640 1 -- 172.21.15.152:0/2264318268 wait complete. 2024-09-11T22:52:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.774+0000 7f017ef3e640 1 Processor -- start 2024-09-11T22:52:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.774+0000 7f017ef3e640 1 -- start start 2024-09-11T22:52:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.774+0000 7f017ef3e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01780ff9e0 0x7f0178197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.774+0000 7f017ef3e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0178100320 0x7f01781984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.774+0000 7f017ef3e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01781078e0 0x7f017819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.774+0000 7f017ef3e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f017810d8f0 con 0x7f0178100320 2024-09-11T22:52:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.774+0000 7f017ef3e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f017810d770 con 0x7f01780ff9e0 2024-09-11T22:52:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.774+0000 7f017ef3e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f017810da70 con 0x7f01781078e0 2024-09-11T22:52:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.774+0000 7f0177fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0178100320 0x7f01781984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.775+0000 7f0177fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0178100320 0x7f01781984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40040/0 (socket says 172.21.15.152:40040) 2024-09-11T22:52:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.775+0000 7f0177fff640 1 -- 172.21.15.152:0/2677665830 learned_addr learned my addr 172.21.15.152:0/2677665830 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.775+0000 7f017ccda640 1 --2- 172.21.15.152:0/2677665830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01780ff9e0 0x7f0178197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.775+0000 7f017d4db640 1 --2- 172.21.15.152:0/2677665830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01781078e0 0x7f017819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.775+0000 7f0177fff640 1 -- 172.21.15.152:0/2677665830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01781078e0 msgr2=0x7f017819f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.775+0000 7f0177fff640 1 --2- 172.21.15.152:0/2677665830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01781078e0 0x7f017819f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.775+0000 7f0177fff640 1 -- 172.21.15.152:0/2677665830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01780ff9e0 msgr2=0x7f0178197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.775+0000 7f0177fff640 1 --2- 172.21.15.152:0/2677665830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01780ff9e0 0x7f0178197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.775+0000 7f0177fff640 1 -- 172.21.15.152:0/2677665830 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017819fbd0 con 0x7f0178100320 2024-09-11T22:52:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.775+0000 7f0177fff640 1 --2- 172.21.15.152:0/2677665830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0178100320 0x7f01781984e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0164006fd0 tx=0x7f016400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.775+0000 7f0175ffb640 1 -- 172.21.15.152:0/2677665830 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164011830 con 0x7f0178100320 2024-09-11T22:52:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.775+0000 7f0175ffb640 1 -- 172.21.15.152:0/2677665830 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0164016070 con 0x7f0178100320 2024-09-11T22:52:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.775+0000 7f0175ffb640 1 -- 172.21.15.152:0/2677665830 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016400f2e0 con 0x7f0178100320 2024-09-11T22:52:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.775+0000 7f017ef3e640 1 -- 172.21.15.152:0/2677665830 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017819fec0 con 0x7f0178100320 2024-09-11T22:52:32.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.776+0000 7f017ef3e640 1 -- 172.21.15.152:0/2677665830 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01781a03f0 con 0x7f0178100320 2024-09-11T22:52:32.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.777+0000 7f017ef3e640 1 -- 172.21.15.152:0/2677665830 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0178106160 con 0x7f0178100320 2024-09-11T22:52:32.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.778+0000 7f0175ffb640 1 -- 172.21.15.152:0/2677665830 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f016401b460 con 0x7f0178100320 2024-09-11T22:52:32.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.778+0000 7f0175ffb640 1 --2- 172.21.15.152:0/2677665830 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01680a2bb0 0x7f01680a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:32.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.778+0000 7f017ccda640 1 --2- 172.21.15.152:0/2677665830 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01680a2bb0 0x7f01680a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:32.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.778+0000 7f0175ffb640 1 -- 172.21.15.152:0/2677665830 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f01640ce610 con 0x7f0178100320 2024-09-11T22:52:32.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.779+0000 7f017ccda640 1 --2- 172.21.15.152:0/2677665830 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01680a2bb0 0x7f01680a5070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f016c0314e0 tx=0x7f016c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:32.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.783+0000 7f0175ffb640 1 -- 172.21.15.152:0/2677665830 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f016409b5a0 con 0x7f0178100320 2024-09-11T22:52:32.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.958+0000 7f017ef3e640 1 -- 172.21.15.152:0/2677665830 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f0178062df0 con 0x7f0178100320 2024-09-11T22:52:32.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.959+0000 7f0175ffb640 1 -- 172.21.15.152:0/2677665830 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) ==== 56+0+5292 (secure 0 0 0) 0x7f016409aea0 con 0x7f0178100320 2024-09-11T22:52:32.960 INFO:teuthology.orchestra.run.smithi152.stdout:epoch 83 2024-09-11T22:52:32.960 INFO:teuthology.orchestra.run.smithi152.stdout:fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T22:52:32.960 INFO:teuthology.orchestra.run.smithi152.stdout:created 2024-09-11T22:43:00.379317+0000 2024-09-11T22:52:32.961 INFO:teuthology.orchestra.run.smithi152.stdout:modified 2024-09-11T22:51:55.946729+0000 2024-09-11T22:52:32.961 INFO:teuthology.orchestra.run.smithi152.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T22:52:32.961 INFO:teuthology.orchestra.run.smithi152.stdout:crush_version 26 2024-09-11T22:52:32.961 INFO:teuthology.orchestra.run.smithi152.stdout:full_ratio 0.95 2024-09-11T22:52:32.961 INFO:teuthology.orchestra.run.smithi152.stdout:backfillfull_ratio 0.9 2024-09-11T22:52:32.961 INFO:teuthology.orchestra.run.smithi152.stdout:nearfull_ratio 0.85 2024-09-11T22:52:32.961 INFO:teuthology.orchestra.run.smithi152.stdout:require_min_compat_client luminous 2024-09-11T22:52:32.961 INFO:teuthology.orchestra.run.smithi152.stdout:min_compat_client luminous 2024-09-11T22:52:32.961 INFO:teuthology.orchestra.run.smithi152.stdout:require_osd_release squid 2024-09-11T22:52:32.961 INFO:teuthology.orchestra.run.smithi152.stdout:stretch_mode_enabled false 2024-09-11T22:52:32.961 INFO:teuthology.orchestra.run.smithi152.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-11T22:52:32.962 INFO:teuthology.orchestra.run.smithi152.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-11T22:52:32.962 INFO:teuthology.orchestra.run.smithi152.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-11T22:52:32.962 INFO:teuthology.orchestra.run.smithi152.stdout:max_osd 12 2024-09-11T22:52:32.962 INFO:teuthology.orchestra.run.smithi152.stdout:osd.0 up in weight 1 up_from 8 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6802/1270728007,v1:172.21.15.152:6803/1270728007] [v2:172.21.15.152:6804/1270728007,v1:172.21.15.152:6805/1270728007] exists,up 3cd917b5-77f6-4a4a-8fd0-4ebe3034feac 2024-09-11T22:52:32.962 INFO:teuthology.orchestra.run.smithi152.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6800/354467155,v1:172.21.15.191:6801/354467155] [v2:172.21.15.191:6802/354467155,v1:172.21.15.191:6803/354467155] exists,up ae589bc4-77b1-4522-9591-b0ce2a5432dc 2024-09-11T22:52:32.962 INFO:teuthology.orchestra.run.smithi152.stdout:osd.2 up in weight 1 up_from 18 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6800/3848396211,v1:172.21.15.204:6801/3848396211] [v2:172.21.15.204:6802/3848396211,v1:172.21.15.204:6803/3848396211] exists,up b35ffd0c-d084-43dd-a947-15132d78d59c 2024-09-11T22:52:32.962 INFO:teuthology.orchestra.run.smithi152.stdout:osd.3 up in weight 1 up_from 25 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6810/3351257181,v1:172.21.15.152:6811/3351257181] [v2:172.21.15.152:6812/3351257181,v1:172.21.15.152:6813/3351257181] exists,up d66ba7e8-4ab7-449e-9460-2f44ffdf987f 2024-09-11T22:52:32.962 INFO:teuthology.orchestra.run.smithi152.stdout:osd.4 up in weight 1 up_from 32 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6808/2967539563,v1:172.21.15.191:6809/2967539563] [v2:172.21.15.191:6810/2967539563,v1:172.21.15.191:6811/2967539563] exists,up e1b56468-6cc7-4d29-988e-9d281118ee36 2024-09-11T22:52:32.962 INFO:teuthology.orchestra.run.smithi152.stdout:osd.5 up in weight 1 up_from 37 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6808/4283562158,v1:172.21.15.204:6809/4283562158] [v2:172.21.15.204:6810/4283562158,v1:172.21.15.204:6811/4283562158] exists,up 89dabd3e-77e3-4294-9648-b112ef694e0a 2024-09-11T22:52:32.962 INFO:teuthology.orchestra.run.smithi152.stdout:osd.6 up in weight 1 up_from 44 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6818/2484234042,v1:172.21.15.152:6819/2484234042] [v2:172.21.15.152:6820/2484234042,v1:172.21.15.152:6821/2484234042] exists,up 46a0fded-b668-4aef-a68f-613cccc83483 2024-09-11T22:52:32.962 INFO:teuthology.orchestra.run.smithi152.stdout:osd.7 up in weight 1 up_from 49 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6816/269608859,v1:172.21.15.191:6817/269608859] [v2:172.21.15.191:6818/269608859,v1:172.21.15.191:6819/269608859] exists,up e674bfe9-e2db-44a3-b74b-a21f198747c9 2024-09-11T22:52:32.962 INFO:teuthology.orchestra.run.smithi152.stdout:osd.8 up in weight 1 up_from 56 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6816/1756190090,v1:172.21.15.204:6817/1756190090] [v2:172.21.15.204:6818/1756190090,v1:172.21.15.204:6819/1756190090] exists,up 0418bfc9-d6b6-488f-b4c7-0bc6291bb624 2024-09-11T22:52:32.962 INFO:teuthology.orchestra.run.smithi152.stdout:osd.9 up in weight 1 up_from 63 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6826/1301191419,v1:172.21.15.152:6827/1301191419] [v2:172.21.15.152:6828/1301191419,v1:172.21.15.152:6829/1301191419] exists,up 751a6964-798f-4240-a2ba-7fa9f0c72657 2024-09-11T22:52:32.963 INFO:teuthology.orchestra.run.smithi152.stdout:osd.10 up in weight 1 up_from 68 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.191:6824/3695294503,v1:172.21.15.191:6825/3695294503] [v2:172.21.15.191:6826/3695294503,v1:172.21.15.191:6827/3695294503] exists,up 90e7c6ba-127c-4b33-b3c6-cedf72128451 2024-09-11T22:52:32.963 INFO:teuthology.orchestra.run.smithi152.stdout:osd.11 up in weight 1 up_from 75 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6824/1709433236,v1:172.21.15.204:6825/1709433236] [v2:172.21.15.204:6826/1709433236,v1:172.21.15.204:6827/1709433236] exists,up c76a93b1-a045-477b-98d0-58a2e9d44a7f 2024-09-11T22:52:32.963 INFO:teuthology.orchestra.run.smithi152.stdout:pg_upmap_items 2.4 [2,11] 2024-09-11T22:52:32.963 INFO:teuthology.orchestra.run.smithi152.stdout:pg_upmap_items 2.6 [8,11] 2024-09-11T22:52:32.963 INFO:teuthology.orchestra.run.smithi152.stdout:pg_upmap_items 3.2 [3,9] 2024-09-11T22:52:32.963 INFO:teuthology.orchestra.run.smithi152.stdout:pg_upmap_items 3.e [4,6,5,9] 2024-09-11T22:52:32.963 INFO:teuthology.orchestra.run.smithi152.stdout:pg_upmap_items 3.11 [4,8] 2024-09-11T22:52:32.963 INFO:teuthology.orchestra.run.smithi152.stdout:pg_upmap_items 3.14 [4,7] 2024-09-11T22:52:32.963 INFO:teuthology.orchestra.run.smithi152.stdout:pg_upmap_items 3.20 [5,9] 2024-09-11T22:52:32.963 INFO:teuthology.orchestra.run.smithi152.stdout:blocklist 172.21.15.152:0/1542993850 expires 2024-09-12T22:43:54.679375+0000 2024-09-11T22:52:32.963 INFO:teuthology.orchestra.run.smithi152.stdout:blocklist 172.21.15.152:6801/1868213162 expires 2024-09-12T22:43:54.679375+0000 2024-09-11T22:52:32.963 INFO:teuthology.orchestra.run.smithi152.stdout:blocklist 172.21.15.152:0/3071755865 expires 2024-09-12T22:43:54.679375+0000 2024-09-11T22:52:32.963 INFO:teuthology.orchestra.run.smithi152.stdout:blocklist 172.21.15.152:0/2452559014 expires 2024-09-12T22:43:54.679375+0000 2024-09-11T22:52:32.964 INFO:teuthology.orchestra.run.smithi152.stdout:blocklist 172.21.15.152:0/4286398432 expires 2024-09-12T22:43:31.970432+0000 2024-09-11T22:52:32.964 INFO:teuthology.orchestra.run.smithi152.stdout:blocklist 172.21.15.152:6801/2200357575 expires 2024-09-12T22:43:31.970432+0000 2024-09-11T22:52:32.964 INFO:teuthology.orchestra.run.smithi152.stdout:blocklist 172.21.15.152:6800/1868213162 expires 2024-09-12T22:43:54.679375+0000 2024-09-11T22:52:32.964 INFO:teuthology.orchestra.run.smithi152.stdout:blocklist 172.21.15.152:6800/2200357575 expires 2024-09-12T22:43:31.970432+0000 2024-09-11T22:52:32.964 INFO:teuthology.orchestra.run.smithi152.stdout:blocklist 172.21.15.152:0/3960601922 expires 2024-09-12T22:43:31.970432+0000 2024-09-11T22:52:32.964 INFO:teuthology.orchestra.run.smithi152.stdout:blocklist 172.21.15.152:0/2270507277 expires 2024-09-12T22:43:31.970432+0000 2024-09-11T22:52:32.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.962+0000 7f017ef3e640 1 -- 172.21.15.152:0/2677665830 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01680a2bb0 msgr2=0x7f01680a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:32.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.962+0000 7f017ef3e640 1 --2- 172.21.15.152:0/2677665830 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01680a2bb0 0x7f01680a5070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f016c0314e0 tx=0x7f016c000bc0 comp rx=0 tx=0).stop 2024-09-11T22:52:32.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.962+0000 7f017ef3e640 1 -- 172.21.15.152:0/2677665830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0178100320 msgr2=0x7f01781984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:32.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.962+0000 7f017ef3e640 1 --2- 172.21.15.152:0/2677665830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0178100320 0x7f01781984e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0164006fd0 tx=0x7f016400b8e0 comp rx=0 tx=0).stop 2024-09-11T22:52:32.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.962+0000 7f017ef3e640 1 -- 172.21.15.152:0/2677665830 shutdown_connections 2024-09-11T22:52:32.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.962+0000 7f017ef3e640 1 --2- 172.21.15.152:0/2677665830 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01680a2bb0 0x7f01680a5070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:32.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.962+0000 7f017ef3e640 1 --2- 172.21.15.152:0/2677665830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01781078e0 0x7f017819f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:32.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.962+0000 7f017ef3e640 1 --2- 172.21.15.152:0/2677665830 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0178100320 0x7f01781984e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:32.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.962+0000 7f017ef3e640 1 --2- 172.21.15.152:0/2677665830 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01780ff9e0 0x7f0178197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:32.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.962+0000 7f017ef3e640 1 -- 172.21.15.152:0/2677665830 >> 172.21.15.152:0/2677665830 conn(0x7f01780fb610 msgr2=0x7f0178103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:32.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.962+0000 7f017ef3e640 1 -- 172.21.15.152:0/2677665830 shutdown_connections 2024-09-11T22:52:32.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:32.962+0000 7f017ef3e640 1 -- 172.21.15.152:0/2677665830 wait complete. 2024-09-11T22:52:33.791 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-11T22:52:33.803 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:52:34.062 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:34.183 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:33 smithi152 ceph-mon[31181]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-11T22:52:34.184 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:33 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2677665830' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T22:52:34.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:33 smithi191 ceph-mon[35211]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-11T22:52:34.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:33 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2677665830' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T22:52:34.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:33 smithi204 ceph-mon[35454]: pgmap v332: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-11T22:52:34.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:33 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2677665830' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T22:52:34.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.596+0000 7fe750c31640 1 -- 172.21.15.152:0/2503711772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe74c1082a0 msgr2=0x7fe74c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:34.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.596+0000 7fe750c31640 1 --2- 172.21.15.152:0/2503711772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe74c1082a0 0x7fe74c10ec40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe744009cd0 tx=0x7fe74402a910 comp rx=0 tx=0).stop 2024-09-11T22:52:34.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.596+0000 7fe750c31640 1 -- 172.21.15.152:0/2503711772 shutdown_connections 2024-09-11T22:52:34.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.596+0000 7fe750c31640 1 --2- 172.21.15.152:0/2503711772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe74c1082a0 0x7fe74c10ec40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:34.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.596+0000 7fe750c31640 1 --2- 172.21.15.152:0/2503711772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe74c1036e0 0x7fe74c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:34.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.596+0000 7fe750c31640 1 --2- 172.21.15.152:0/2503711772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe74c102d30 0x7fe74c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:34.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.596+0000 7fe750c31640 1 -- 172.21.15.152:0/2503711772 >> 172.21.15.152:0/2503711772 conn(0x7fe74c0fc7d0 msgr2=0x7fe74c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:34.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.596+0000 7fe750c31640 1 -- 172.21.15.152:0/2503711772 shutdown_connections 2024-09-11T22:52:34.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.596+0000 7fe750c31640 1 -- 172.21.15.152:0/2503711772 wait complete. 2024-09-11T22:52:34.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.597+0000 7fe750c31640 1 Processor -- start 2024-09-11T22:52:34.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.597+0000 7fe750c31640 1 -- start start 2024-09-11T22:52:34.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.597+0000 7fe750c31640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe74c102d30 0x7fe74c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:34.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.597+0000 7fe750c31640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe74c1036e0 0x7fe74c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:34.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.597+0000 7fe750c31640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe74c1082a0 0x7fe74c19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:34.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.597+0000 7fe750c31640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c1120e0 con 0x7fe74c1036e0 2024-09-11T22:52:34.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.597+0000 7fe750c31640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c111f60 con 0x7fe74c102d30 2024-09-11T22:52:34.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.597+0000 7fe750c31640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe74c112260 con 0x7fe74c1082a0 2024-09-11T22:52:34.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.597+0000 7fe749d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe74c1036e0 0x7fe74c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:34.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.597+0000 7fe749d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe74c1036e0 0x7fe74c198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40062/0 (socket says 172.21.15.152:40062) 2024-09-11T22:52:34.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.597+0000 7fe749d9b640 1 -- 172.21.15.152:0/1930032908 learned_addr learned my addr 172.21.15.152:0/1930032908 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:34.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.598+0000 7fe74a59c640 1 --2- 172.21.15.152:0/1930032908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe74c102d30 0x7fe74c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:34.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.598+0000 7fe749d9b640 1 -- 172.21.15.152:0/1930032908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe74c1082a0 msgr2=0x7fe74c19ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:34.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.598+0000 7fe74ad9d640 1 --2- 172.21.15.152:0/1930032908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe74c1082a0 0x7fe74c19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:34.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.598+0000 7fe749d9b640 1 --2- 172.21.15.152:0/1930032908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe74c1082a0 0x7fe74c19ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:34.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.598+0000 7fe749d9b640 1 -- 172.21.15.152:0/1930032908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe74c102d30 msgr2=0x7fe74c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:34.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.598+0000 7fe749d9b640 1 --2- 172.21.15.152:0/1930032908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe74c102d30 0x7fe74c198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:34.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.598+0000 7fe749d9b640 1 -- 172.21.15.152:0/1930032908 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74c19d530 con 0x7fe74c1036e0 2024-09-11T22:52:34.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.598+0000 7fe74a59c640 1 --2- 172.21.15.152:0/1930032908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe74c102d30 0x7fe74c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:52:34.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.598+0000 7fe749d9b640 1 --2- 172.21.15.152:0/1930032908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe74c1036e0 0x7fe74c198ac0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe740006fd0 tx=0x7fe74000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:34.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.598+0000 7fe73b7fe640 1 -- 172.21.15.152:0/1930032908 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe740011830 con 0x7fe74c1036e0 2024-09-11T22:52:34.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.598+0000 7fe73b7fe640 1 -- 172.21.15.152:0/1930032908 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe740016070 con 0x7fe74c1036e0 2024-09-11T22:52:34.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.598+0000 7fe750c31640 1 -- 172.21.15.152:0/1930032908 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74c19d820 con 0x7fe74c1036e0 2024-09-11T22:52:34.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.598+0000 7fe73b7fe640 1 -- 172.21.15.152:0/1930032908 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74000f2e0 con 0x7fe74c1036e0 2024-09-11T22:52:34.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.598+0000 7fe750c31640 1 -- 172.21.15.152:0/1930032908 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe74c1a5100 con 0x7fe74c1036e0 2024-09-11T22:52:34.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.599+0000 7fe750c31640 1 -- 172.21.15.152:0/1930032908 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe74c103110 con 0x7fe74c1036e0 2024-09-11T22:52:34.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.600+0000 7fe73b7fe640 1 -- 172.21.15.152:0/1930032908 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe74001b460 con 0x7fe74c1036e0 2024-09-11T22:52:34.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.601+0000 7fe73b7fe640 1 --2- 172.21.15.152:0/1930032908 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe7340a2b80 0x7fe7340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:34.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.601+0000 7fe74a59c640 1 --2- 172.21.15.152:0/1930032908 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe7340a2b80 0x7fe7340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:34.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.601+0000 7fe73b7fe640 1 -- 172.21.15.152:0/1930032908 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fe7400ce750 con 0x7fe74c1036e0 2024-09-11T22:52:34.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.601+0000 7fe74a59c640 1 --2- 172.21.15.152:0/1930032908 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe7340a2b80 0x7fe7340a5040 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009b40 tx=0x7fe73c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:34.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.605+0000 7fe73b7fe640 1 -- 172.21.15.152:0/1930032908 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe74009b6e0 con 0x7fe74c1036e0 2024-09-11T22:52:34.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.821+0000 7fe750c31640 1 -- 172.21.15.152:0/1930032908 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe74c1a5300 con 0x7fe74c1036e0 2024-09-11T22:52:34.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.822+0000 7fe73b7fe640 1 -- 172.21.15.152:0/1930032908 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11988 (secure 0 0 0) 0x7fe74009afe0 con 0x7fe74c1036e0 2024-09-11T22:52:34.823 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:34.823 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":27,"btime":"2024-09-11T22:51:06:131335+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:51:06.131313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[24455,24442,24434,24430,14628]},"id":1}]} 2024-09-11T22:52:34.823 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 27 2024-09-11T22:52:34.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.825+0000 7fe750c31640 1 -- 172.21.15.152:0/1930032908 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe7340a2b80 msgr2=0x7fe7340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:34.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.825+0000 7fe750c31640 1 --2- 172.21.15.152:0/1930032908 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe7340a2b80 0x7fe7340a5040 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009b40 tx=0x7fe73c005e50 comp rx=0 tx=0).stop 2024-09-11T22:52:34.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.825+0000 7fe750c31640 1 -- 172.21.15.152:0/1930032908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe74c1036e0 msgr2=0x7fe74c198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:34.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.825+0000 7fe750c31640 1 --2- 172.21.15.152:0/1930032908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe74c1036e0 0x7fe74c198ac0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe740006fd0 tx=0x7fe74000b8e0 comp rx=0 tx=0).stop 2024-09-11T22:52:34.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.825+0000 7fe750c31640 1 -- 172.21.15.152:0/1930032908 shutdown_connections 2024-09-11T22:52:34.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.825+0000 7fe750c31640 1 --2- 172.21.15.152:0/1930032908 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fe7340a2b80 0x7fe7340a5040 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:34.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.825+0000 7fe750c31640 1 --2- 172.21.15.152:0/1930032908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe74c1082a0 0x7fe74c19ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:34.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.825+0000 7fe750c31640 1 --2- 172.21.15.152:0/1930032908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe74c1036e0 0x7fe74c198ac0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:34.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.825+0000 7fe750c31640 1 --2- 172.21.15.152:0/1930032908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe74c102d30 0x7fe74c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:34.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.825+0000 7fe750c31640 1 -- 172.21.15.152:0/1930032908 >> 172.21.15.152:0/1930032908 conn(0x7fe74c0fc7d0 msgr2=0x7fe74c105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:34.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.825+0000 7fe750c31640 1 -- 172.21.15.152:0/1930032908 shutdown_connections 2024-09-11T22:52:34.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:34.825+0000 7fe750c31640 1 -- 172.21.15.152:0/1930032908 wait complete. 2024-09-11T22:52:35.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:34 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1930032908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:35.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:34 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1930032908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:35.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:34 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1930032908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:35.679 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:52:35.934 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:36.200 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:35 smithi152 ceph-mon[31181]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-11T22:52:36.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:35 smithi191 ceph-mon[35211]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-11T22:52:36.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:35 smithi204 ceph-mon[35454]: pgmap v333: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-11T22:52:36.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.429+0000 7fd251b57640 1 -- 172.21.15.152:0/823282704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd24c106f10 msgr2=0x7fd24c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:36.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.429+0000 7fd251b57640 1 --2- 172.21.15.152:0/823282704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd24c106f10 0x7fd24c10ea90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd244009ca0 tx=0x7fd24402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:52:36.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.429+0000 7fd251b57640 1 -- 172.21.15.152:0/823282704 shutdown_connections 2024-09-11T22:52:36.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.429+0000 7fd251b57640 1 --2- 172.21.15.152:0/823282704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd24c106f10 0x7fd24c10ea90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:36.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.429+0000 7fd251b57640 1 --2- 172.21.15.152:0/823282704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd24c102790 0x7fd24c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:36.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.429+0000 7fd251b57640 1 --2- 172.21.15.152:0/823282704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd24c101de0 0x7fd24c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:36.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.429+0000 7fd251b57640 1 -- 172.21.15.152:0/823282704 >> 172.21.15.152:0/823282704 conn(0x7fd24c0fd8c0 msgr2=0x7fd24c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:36.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.429+0000 7fd251b57640 1 -- 172.21.15.152:0/823282704 shutdown_connections 2024-09-11T22:52:36.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.429+0000 7fd251b57640 1 -- 172.21.15.152:0/823282704 wait complete. 2024-09-11T22:52:36.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.430+0000 7fd251b57640 1 Processor -- start 2024-09-11T22:52:36.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.430+0000 7fd251b57640 1 -- start start 2024-09-11T22:52:36.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.430+0000 7fd251b57640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd24c101de0 0x7fd24c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:36.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.430+0000 7fd251b57640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd24c102790 0x7fd24c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:36.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.430+0000 7fd251b57640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd24c106f10 0x7fd24c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:36.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.430+0000 7fd251b57640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c111f70 con 0x7fd24c101de0 2024-09-11T22:52:36.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.430+0000 7fd251b57640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c111df0 con 0x7fd24c102790 2024-09-11T22:52:36.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.430+0000 7fd251b57640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c1120f0 con 0x7fd24c106f10 2024-09-11T22:52:36.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.430+0000 7fd24b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd24c101de0 0x7fd24c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:36.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.430+0000 7fd24b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd24c101de0 0x7fd24c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40088/0 (socket says 172.21.15.152:40088) 2024-09-11T22:52:36.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.431+0000 7fd24b7fe640 1 -- 172.21.15.152:0/649200482 learned_addr learned my addr 172.21.15.152:0/649200482 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:36.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.431+0000 7fd24affd640 1 --2- 172.21.15.152:0/649200482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd24c102790 0x7fd24c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:36.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.431+0000 7fd24bfff640 1 --2- 172.21.15.152:0/649200482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd24c106f10 0x7fd24c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:36.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.431+0000 7fd24b7fe640 1 -- 172.21.15.152:0/649200482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd24c106f10 msgr2=0x7fd24c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:36.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.431+0000 7fd24b7fe640 1 --2- 172.21.15.152:0/649200482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd24c106f10 0x7fd24c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:36.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.431+0000 7fd24b7fe640 1 -- 172.21.15.152:0/649200482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd24c102790 msgr2=0x7fd24c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:36.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.431+0000 7fd24b7fe640 1 --2- 172.21.15.152:0/649200482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd24c102790 0x7fd24c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:36.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.431+0000 7fd24b7fe640 1 -- 172.21.15.152:0/649200482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24c19d3a0 con 0x7fd24c101de0 2024-09-11T22:52:36.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.431+0000 7fd24b7fe640 1 --2- 172.21.15.152:0/649200482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd24c101de0 0x7fd24c1983f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00d770 tx=0x7fd23c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:36.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.431+0000 7fd248ff9640 1 -- 172.21.15.152:0/649200482 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c017070 con 0x7fd24c101de0 2024-09-11T22:52:36.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.431+0000 7fd248ff9640 1 -- 172.21.15.152:0/649200482 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd23c005500 con 0x7fd24c101de0 2024-09-11T22:52:36.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.431+0000 7fd251b57640 1 -- 172.21.15.152:0/649200482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24c19d690 con 0x7fd24c101de0 2024-09-11T22:52:36.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.431+0000 7fd248ff9640 1 -- 172.21.15.152:0/649200482 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c004e00 con 0x7fd24c101de0 2024-09-11T22:52:36.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.431+0000 7fd251b57640 1 -- 172.21.15.152:0/649200482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd24c1a4f70 con 0x7fd24c101de0 2024-09-11T22:52:36.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.432+0000 7fd251b57640 1 -- 172.21.15.152:0/649200482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd24c1021c0 con 0x7fd24c101de0 2024-09-11T22:52:36.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.433+0000 7fd248ff9640 1 -- 172.21.15.152:0/649200482 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd23c004fa0 con 0x7fd24c101de0 2024-09-11T22:52:36.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.434+0000 7fd248ff9640 1 --2- 172.21.15.152:0/649200482 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd2340a2b80 0x7fd2340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:36.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.434+0000 7fd24affd640 1 --2- 172.21.15.152:0/649200482 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd2340a2b80 0x7fd2340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:36.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.434+0000 7fd248ff9640 1 -- 172.21.15.152:0/649200482 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fd23c0ce990 con 0x7fd24c101de0 2024-09-11T22:52:36.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.435+0000 7fd24affd640 1 --2- 172.21.15.152:0/649200482 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd2340a2b80 0x7fd2340a5040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0fe0c0 tx=0x7fd24000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:36.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.438+0000 7fd248ff9640 1 -- 172.21.15.152:0/649200482 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd23c09b2d0 con 0x7fd24c101de0 2024-09-11T22:52:36.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.641+0000 7fd251b57640 1 -- 172.21.15.152:0/649200482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd24c199750 con 0x7fd24c101de0 2024-09-11T22:52:36.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.642+0000 7fd248ff9640 1 -- 172.21.15.152:0/649200482 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11988 (secure 0 0 0) 0x7fd23c09f5d0 con 0x7fd24c101de0 2024-09-11T22:52:36.643 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:36.643 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":27,"btime":"2024-09-11T22:51:06:131335+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:51:06.131313+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[24455,24442,24434,24430,14628]},"id":1}]} 2024-09-11T22:52:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 27 2024-09-11T22:52:36.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.644+0000 7fd251b57640 1 -- 172.21.15.152:0/649200482 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd2340a2b80 msgr2=0x7fd2340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:36.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.644+0000 7fd251b57640 1 --2- 172.21.15.152:0/649200482 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd2340a2b80 0x7fd2340a5040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0fe0c0 tx=0x7fd24000b390 comp rx=0 tx=0).stop 2024-09-11T22:52:36.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.644+0000 7fd251b57640 1 -- 172.21.15.152:0/649200482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd24c101de0 msgr2=0x7fd24c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:36.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.644+0000 7fd251b57640 1 --2- 172.21.15.152:0/649200482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd24c101de0 0x7fd24c1983f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00d770 tx=0x7fd23c00dc30 comp rx=0 tx=0).stop 2024-09-11T22:52:36.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.644+0000 7fd251b57640 1 -- 172.21.15.152:0/649200482 shutdown_connections 2024-09-11T22:52:36.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.644+0000 7fd251b57640 1 --2- 172.21.15.152:0/649200482 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fd2340a2b80 0x7fd2340a5040 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:36.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.644+0000 7fd251b57640 1 --2- 172.21.15.152:0/649200482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd24c106f10 0x7fd24c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:36.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.644+0000 7fd251b57640 1 --2- 172.21.15.152:0/649200482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd24c102790 0x7fd24c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:36.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.644+0000 7fd251b57640 1 --2- 172.21.15.152:0/649200482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd24c101de0 0x7fd24c1983f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:36.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.644+0000 7fd251b57640 1 -- 172.21.15.152:0/649200482 >> 172.21.15.152:0/649200482 conn(0x7fd24c0fd8c0 msgr2=0x7fd24c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:36.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.645+0000 7fd251b57640 1 -- 172.21.15.152:0/649200482 shutdown_connections 2024-09-11T22:52:36.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:36.645+0000 7fd251b57640 1 -- 172.21.15.152:0/649200482 wait complete. 2024-09-11T22:52:37.188 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:36 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/649200482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:37.197 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:36 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/649200482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:37.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:36 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/649200482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:37.433 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:52:37.683 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:38.109 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:37 smithi152 ceph-mon[31181]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-11T22:52:38.404 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:37 smithi204 ceph-mon[35454]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-11T22:52:38.404 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:37 smithi191 ceph-mon[35211]: pgmap v334: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-11T22:52:38.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.192+0000 7f88b2f96640 1 -- 172.21.15.152:0/2439048849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88ac100a80 msgr2=0x7f88ac100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:38.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.192+0000 7f88b2f96640 1 --2- 172.21.15.152:0/2439048849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88ac100a80 0x7f88ac100ee0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8898009930 tx=0x7f889802a420 comp rx=0 tx=0).stop 2024-09-11T22:52:38.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.192+0000 7f88b2f96640 1 -- 172.21.15.152:0/2439048849 shutdown_connections 2024-09-11T22:52:38.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.192+0000 7f88b2f96640 1 --2- 172.21.15.152:0/2439048849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88ac101420 0x7f88ac10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:38.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.192+0000 7f88b2f96640 1 --2- 172.21.15.152:0/2439048849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88ac100a80 0x7f88ac100ee0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:38.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.192+0000 7f88b2f96640 1 --2- 172.21.15.152:0/2439048849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88ac106a80 0x7f88ac106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:38.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.192+0000 7f88b2f96640 1 -- 172.21.15.152:0/2439048849 >> 172.21.15.152:0/2439048849 conn(0x7f88ac0fc7a0 msgr2=0x7f88ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:38.405 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.192+0000 7f88b2f96640 1 -- 172.21.15.152:0/2439048849 shutdown_connections 2024-09-11T22:52:38.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.192+0000 7f88b2f96640 1 -- 172.21.15.152:0/2439048849 wait complete. 2024-09-11T22:52:38.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88b2f96640 1 Processor -- start 2024-09-11T22:52:38.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88b2f96640 1 -- start start 2024-09-11T22:52:38.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88b2f96640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88ac100a80 0x7f88ac1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:38.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88b2f96640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88ac101420 0x7f88ac198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:38.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88b2f96640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88ac106a80 0x7f88ac19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:38.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88b2f96640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac112020 con 0x7f88ac101420 2024-09-11T22:52:38.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88b2f96640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac111ea0 con 0x7f88ac106a80 2024-09-11T22:52:38.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88b2f96640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac1121a0 con 0x7f88ac100a80 2024-09-11T22:52:38.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88abfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88ac101420 0x7f88ac198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:38.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88abfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88ac101420 0x7f88ac198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40118/0 (socket says 172.21.15.152:40118) 2024-09-11T22:52:38.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88abfff640 1 -- 172.21.15.152:0/3955700801 learned_addr learned my addr 172.21.15.152:0/3955700801 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:38.406 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88b0d32640 1 --2- 172.21.15.152:0/3955700801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88ac100a80 0x7f88ac1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:38.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88b1533640 1 --2- 172.21.15.152:0/3955700801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88ac106a80 0x7f88ac19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:38.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88abfff640 1 -- 172.21.15.152:0/3955700801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88ac100a80 msgr2=0x7f88ac1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:38.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88abfff640 1 --2- 172.21.15.152:0/3955700801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88ac100a80 0x7f88ac1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:38.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88abfff640 1 -- 172.21.15.152:0/3955700801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88ac106a80 msgr2=0x7f88ac19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:38.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.193+0000 7f88abfff640 1 --2- 172.21.15.152:0/3955700801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88ac106a80 0x7f88ac19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:38.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.194+0000 7f88abfff640 1 -- 172.21.15.152:0/3955700801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ac19d590 con 0x7f88ac101420 2024-09-11T22:52:38.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.194+0000 7f88abfff640 1 --2- 172.21.15.152:0/3955700801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88ac101420 0x7f88ac198b20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8898009a50 tx=0x7f8898004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:38.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.194+0000 7f88a9ffb640 1 -- 172.21.15.152:0/3955700801 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889803f030 con 0x7f88ac101420 2024-09-11T22:52:38.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.194+0000 7f88a9ffb640 1 -- 172.21.15.152:0/3955700801 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8898003ce0 con 0x7f88ac101420 2024-09-11T22:52:38.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.194+0000 7f88b2f96640 1 -- 172.21.15.152:0/3955700801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ac19d820 con 0x7f88ac101420 2024-09-11T22:52:38.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.194+0000 7f88a9ffb640 1 -- 172.21.15.152:0/3955700801 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898035070 con 0x7f88ac101420 2024-09-11T22:52:38.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.194+0000 7f88b2f96640 1 -- 172.21.15.152:0/3955700801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ac1a5160 con 0x7f88ac101420 2024-09-11T22:52:38.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.195+0000 7f88b2f96640 1 -- 172.21.15.152:0/3955700801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88ac1021c0 con 0x7f88ac101420 2024-09-11T22:52:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.196+0000 7f88a9ffb640 1 -- 172.21.15.152:0/3955700801 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8898042030 con 0x7f88ac101420 2024-09-11T22:52:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.197+0000 7f88a9ffb640 1 --2- 172.21.15.152:0/3955700801 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f889c0a2b80 0x7f889c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.197+0000 7f88b0d32640 1 --2- 172.21.15.152:0/3955700801 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f889c0a2b80 0x7f889c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.197+0000 7f88a9ffb640 1 -- 172.21.15.152:0/3955700801 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f88980efab0 con 0x7f88ac101420 2024-09-11T22:52:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.197+0000 7f88b0d32640 1 --2- 172.21.15.152:0/3955700801 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f889c0a2b80 0x7f889c0a5040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009890 tx=0x7f88a0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.200+0000 7f88a9ffb640 1 -- 172.21.15.152:0/3955700801 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88980bca40 con 0x7f88ac101420 2024-09-11T22:52:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.368+0000 7f88b2f96640 1 -- 172.21.15.152:0/3955700801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88ac109010 con 0x7f88ac101420 2024-09-11T22:52:38.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.370+0000 7f88a9ffb640 1 -- 172.21.15.152:0/3955700801 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22710 (secure 0 0 0) 0x7f88980a18d0 con 0x7f88ac101420 2024-09-11T22:52:38.409 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:38.409 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":83,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","created":"2024-09-11T22:43:00.379317+0000","modified":"2024-09-11T22:51:55.946729+0000","last_up_change":"2024-09-11T22:50:01.043181+0000","last_in_change":"2024-09-11T22:49:38.073870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:46:10.984707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:50:50.833547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:50:53.047604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3cd917b5-77f6-4a4a-8fd0-4ebe3034feac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6803","nonce":1270728007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6805","nonce":1270728007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6809","nonce":1270728007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6807","nonce":1270728007}]},"public_addr":"172.21.15.152:6803/1270728007","cluster_addr":"172.21.15.152:6805/1270728007","heartbeat_back_addr":"172.21.15.152:6809/1270728007","heartbeat_front_addr":"172.21.15.152:6807/1270728007","state":["exists","up"]},{"osd":1,"uuid":"ae589bc4-77b1-4522-9591-b0ce2a5432dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6801","nonce":354467155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6803","nonce":354467155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6807","nonce":354467155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6805","nonce":354467155}]},"public_addr":"172.21.15.191:6801/354467155","cluster_addr":"172.21.15.191:6803/354467155","heartbeat_back_addr":"172.21.15.191:6807/354467155","heartbeat_front_addr":"172.21.15.191:6805/354467155","state":["exists","up"]},{"osd":2,"uuid":"b35ffd0c-d084-43dd-a947-15132d78d59c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6801","nonce":3848396211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6803","nonce":3848396211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6807","nonce":3848396211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6805","nonce":3848396211}]},"public_addr":"172.21.15.204:6801/3848396211","cluster_addr":"172.21.15.204:6803/3848396211","heartbeat_back_addr":"172.21.15.204:6807/3848396211","heartbeat_front_addr":"172.21.15.204:6805/3848396211","state":["exists","up"]},{"osd":3,"uuid":"d66ba7e8-4ab7-449e-9460-2f44ffdf987f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6811","nonce":3351257181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6813","nonce":3351257181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6817","nonce":3351257181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6815","nonce":3351257181}]},"public_addr":"172.21.15.152:6811/3351257181","cluster_addr":"172.21.15.152:6813/3351257181","heartbeat_back_addr":"172.21.15.152:6817/3351257181","heartbeat_front_addr":"172.21.15.152:6815/3351257181","state":["exists","up"]},{"osd":4,"uuid":"e1b56468-6cc7-4d29-988e-9d281118ee36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6809","nonce":2967539563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6811","nonce":2967539563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6815","nonce":2967539563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6813","nonce":2967539563}]},"public_addr":"172.21.15.191:6809/2967539563","cluster_addr":"172.21.15.191:6811/2967539563","heartbeat_back_addr":"172.21.15.191:6815/2967539563","heartbeat_front_addr":"172.21.15.191:6813/2967539563","state":["exists","up"]},{"osd":5,"uuid":"89dabd3e-77e3-4294-9648-b112ef694e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6809","nonce":4283562158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6811","nonce":4283562158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6815","nonce":4283562158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6813","nonce":4283562158}]},"public_addr":"172.21.15.204:6809/4283562158","cluster_addr":"172.21.15.204:6811/4283562158","heartbeat_back_addr":"172.21.15.204:6815/4283562158","heartbeat_front_addr":"172.21.15.204:6813/4283562158","state":["exists","up"]},{"osd":6,"uuid":"46a0fded-b668-4aef-a68f-613cccc83483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6819","nonce":2484234042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6821","nonce":2484234042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6825","nonce":2484234042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6823","nonce":2484234042}]},"public_addr":"172.21.15.152:6819/2484234042","cluster_addr":"172.21.15.152:6821/2484234042","heartbeat_back_addr":"172.21.15.152:6825/2484234042","heartbeat_front_addr":"172.21.15.152:6823/2484234042","state":["exists","up"]},{"osd":7,"uuid":"e674bfe9-e2db-44a3-b74b-a21f198747c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6817","nonce":269608859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6819","nonce":269608859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6823","nonce":269608859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6821","nonce":269608859}]},"public_addr":"172.21.15.191:6817/269608859","cluster_addr":"172.21.15.191:6819/269608859","heartbeat_back_addr":"172.21.15.191:6823/269608859","heartbeat_front_addr":"172.21.15.191:6821/269608859","state":["exists","up"]},{"osd":8,"uuid":"0418bfc9-d6b6-488f-b4c7-0bc6291bb624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6817","nonce":1756190090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6819","nonce":1756190090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6823","nonce":1756190090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6821","nonce":1756190090}]},"public_addr":"172.21.15.204:6817/1756190090","cluster_addr":"172.21.15.204:6819/1756190090","heartbeat_back_addr":"172.21.15.204:6823/1756190090","heartbeat_front_addr":"172.21.15.204:6821/1756190090","state":["exists","up"]},{"osd":9,"uuid":"751a6964-798f-4240-a2ba-7fa9f0c72657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6827","nonce":1301191419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6829","nonce":1301191419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6833","nonce":1301191419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6831","nonce":1301191419}]},"public_addr":"172.21.15.152:6827/1301191419","cluster_addr":"172.21.15.152:6829/1301191419","heartbeat_back_addr":"172.21.15.152:6833/1301191419","heartbeat_front_addr":"172.21.15.152:6831/1301191419","state":["exists","up"]},{"osd":10,"uuid":"90e7c6ba-127c-4b33-b3c6-cedf72128451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6825","nonce":3695294503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6827","nonce":3695294503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6831","nonce":3695294503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6829","nonce":3695294503}]},"public_addr":"172.21.15.191:6825/3695294503","cluster_addr":"172.21.15.191:6827/3695294503","heartbeat_back_addr":"172.21.15.191:6831/3695294503","heartbeat_front_addr":"172.21.15.191:6829/3695294503","state":["exists","up"]},{"osd":11,"uuid":"c76a93b1-a045-477b-98d0-58a2e9d44a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6825","nonce":1709433236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6827","nonce":1709433236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6831","nonce":1709433236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6829","nonce":1709433236}]},"public_addr":"172.21.15.204:6825/1709433236","cluster_addr":"172.21.15.204:6827/1709433236","heartbeat_back_addr":"172.21.15.204:6831/1709433236","heartbeat_front_addr":"172.21.15.204:6829/1709433236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:17.259744+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:42.230389+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:07.172433+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:32.517060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:58.900615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:24.379803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:50.058928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:15.967299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:42.025873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:08.225448+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:33.910531+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:59.466423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:0/1542993850":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/3071755865":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/2452559014":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/4286398432":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6801/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6800/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6800/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/3960601922":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2270507277":"2024-09-12T22:43:31.970432+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:52:38.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.372+0000 7f88b2f96640 1 -- 172.21.15.152:0/3955700801 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f889c0a2b80 msgr2=0x7f889c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:38.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.372+0000 7f88b2f96640 1 --2- 172.21.15.152:0/3955700801 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f889c0a2b80 0x7f889c0a5040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009890 tx=0x7f88a0008040 comp rx=0 tx=0).stop 2024-09-11T22:52:38.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.372+0000 7f88b2f96640 1 -- 172.21.15.152:0/3955700801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88ac101420 msgr2=0x7f88ac198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:38.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.372+0000 7f88b2f96640 1 --2- 172.21.15.152:0/3955700801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88ac101420 0x7f88ac198b20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8898009a50 tx=0x7f8898004270 comp rx=0 tx=0).stop 2024-09-11T22:52:38.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.372+0000 7f88b2f96640 1 -- 172.21.15.152:0/3955700801 shutdown_connections 2024-09-11T22:52:38.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.372+0000 7f88b2f96640 1 --2- 172.21.15.152:0/3955700801 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f889c0a2b80 0x7f889c0a5040 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:38.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.372+0000 7f88b2f96640 1 --2- 172.21.15.152:0/3955700801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88ac106a80 0x7f88ac19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:38.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.372+0000 7f88b2f96640 1 --2- 172.21.15.152:0/3955700801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88ac101420 0x7f88ac198b20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:38.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.372+0000 7f88b2f96640 1 --2- 172.21.15.152:0/3955700801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88ac100a80 0x7f88ac1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:38.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.372+0000 7f88b2f96640 1 -- 172.21.15.152:0/3955700801 >> 172.21.15.152:0/3955700801 conn(0x7f88ac0fc7a0 msgr2=0x7f88ac0fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:38.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.372+0000 7f88b2f96640 1 -- 172.21.15.152:0/3955700801 shutdown_connections 2024-09-11T22:52:38.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:38.372+0000 7f88b2f96640 1 -- 172.21.15.152:0/3955700801 wait complete. 2024-09-11T22:52:39.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:38 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3955700801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:52:39.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:38 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3955700801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:52:39.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:38 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3955700801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:52:39.353 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T22:50:55.047332+0000', 'modified': '2024-09-11T22:51:06.131313+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24455, 'mds_1': 24442, 'mds_2': 24434, 'mds_3': 24430, 'mds_4': 14628}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6839/97179606', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6838', 'nonce': 97179606}, {'type': 'v1', 'addr': '172.21.15.152:6839', 'nonce': 97179606}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.152:6841/2492281944', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6840', 'nonce': 2492281944}, {'type': 'v1', 'addr': '172.21.15.152:6841', 'nonce': 2492281944}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24377': {'gid': 24377, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6833/2321157769', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6832', 'nonce': 2321157769}, {'type': 'v1', 'addr': '172.21.15.191:6833', 'nonce': 2321157769}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24397': {'gid': 24397, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6833/2200342090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6832', 'nonce': 2200342090}, {'type': 'v1', 'addr': '172.21.15.204:6833', 'nonce': 2200342090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6835/2471607007', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6834', 'nonce': 2471607007}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 2471607007}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/763950562', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 763950562}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 763950562}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.204:6837/1863048373', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 1863048373}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 1863048373}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.191:6837/314110375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6836', 'nonce': 314110375}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 314110375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.204:6839/3360312546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 3360312546}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 3360312546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.191:6839/2588730805', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6838', 'nonce': 2588730805}, {'type': 'v1', 'addr': '172.21.15.191:6839', 'nonce': 2588730805}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24455, 'qdb_cluster': [24455, 24442, 24434, 24430, 14628]} 2024-09-11T22:52:39.353 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T22:52:39.353 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.a status 2024-09-11T22:52:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:39.996 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:39 smithi152 ceph-mon[31181]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-11T22:52:40.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.076+0000 7fda6e887640 1 -- 172.21.15.152:0/1077153188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda68102570 msgr2=0x7fda681029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:40.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.076+0000 7fda6e887640 1 --2- 172.21.15.152:0/1077153188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda68102570 0x7fda681029b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fda50009900 tx=0x7fda5002a480 comp rx=0 tx=0).stop 2024-09-11T22:52:40.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.076+0000 7fda6e887640 1 -- 172.21.15.152:0/1077153188 shutdown_connections 2024-09-11T22:52:40.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.076+0000 7fda6e887640 1 --2- 172.21.15.152:0/1077153188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda68102ef0 0x7fda6810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.076+0000 7fda6e887640 1 --2- 172.21.15.152:0/1077153188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda68102570 0x7fda681029b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.076+0000 7fda6e887640 1 --2- 172.21.15.152:0/1077153188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda68100e20 0x7fda68101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.076+0000 7fda6e887640 1 -- 172.21.15.152:0/1077153188 >> 172.21.15.152:0/1077153188 conn(0x7fda680fc7d0 msgr2=0x7fda680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:40.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.076+0000 7fda6e887640 1 -- 172.21.15.152:0/1077153188 shutdown_connections 2024-09-11T22:52:40.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.076+0000 7fda6e887640 1 -- 172.21.15.152:0/1077153188 wait complete. 2024-09-11T22:52:40.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda6e887640 1 Processor -- start 2024-09-11T22:52:40.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda6e887640 1 -- start start 2024-09-11T22:52:40.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda6e887640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda68100e20 0x7fda680fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda6e887640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda68102570 0x7fda680fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda6e887640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda68102ef0 0x7fda680fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda6e887640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fda68110f50 con 0x7fda68100e20 2024-09-11T22:52:40.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda6e887640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fda68110dd0 con 0x7fda68102ef0 2024-09-11T22:52:40.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda6e887640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fda681110d0 con 0x7fda68102570 2024-09-11T22:52:40.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda67fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda68100e20 0x7fda680fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda67fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda68100e20 0x7fda680fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58650/0 (socket says 172.21.15.152:58650) 2024-09-11T22:52:40.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda67fff640 1 -- 172.21.15.152:0/1968258706 learned_addr learned my addr 172.21.15.152:0/1968258706 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:40.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda5ffff640 1 --2- 172.21.15.152:0/1968258706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda68102570 0x7fda680fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda67fff640 1 -- 172.21.15.152:0/1968258706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda68102570 msgr2=0x7fda680fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:40.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda67fff640 1 --2- 172.21.15.152:0/1968258706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda68102570 0x7fda680fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.098 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda6ce24640 1 --2- 172.21.15.152:0/1968258706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda68102ef0 0x7fda680fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda67fff640 1 -- 172.21.15.152:0/1968258706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda68102ef0 msgr2=0x7fda680fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:40.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda67fff640 1 --2- 172.21.15.152:0/1968258706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda68102ef0 0x7fda680fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.077+0000 7fda67fff640 1 -- 172.21.15.152:0/1968258706 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda681a69a0 con 0x7fda68100e20 2024-09-11T22:52:40.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.078+0000 7fda5ffff640 1 --2- 172.21.15.152:0/1968258706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda68102570 0x7fda680fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:40.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.078+0000 7fda67fff640 1 --2- 172.21.15.152:0/1968258706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda68100e20 0x7fda680fcc80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fda5800d950 tx=0x7fda5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:40.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.078+0000 7fda65ffb640 1 -- 172.21.15.152:0/1968258706 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda58017030 con 0x7fda68100e20 2024-09-11T22:52:40.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.078+0000 7fda6e887640 1 -- 172.21.15.152:0/1968258706 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda681a6c90 con 0x7fda68100e20 2024-09-11T22:52:40.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.078+0000 7fda65ffb640 1 -- 172.21.15.152:0/1968258706 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda58004020 con 0x7fda68100e20 2024-09-11T22:52:40.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.078+0000 7fda65ffb640 1 -- 172.21.15.152:0/1968258706 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda580047a0 con 0x7fda68100e20 2024-09-11T22:52:40.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.078+0000 7fda6e887640 1 -- 172.21.15.152:0/1968258706 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda681a71c0 con 0x7fda68100e20 2024-09-11T22:52:40.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.080+0000 7fda65ffb640 1 -- 172.21.15.152:0/1968258706 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fda580041d0 con 0x7fda68100e20 2024-09-11T22:52:40.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda70c60740 1 Processor -- start 2024-09-11T22:52:40.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda70c60740 1 -- start start 2024-09-11T22:52:40.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda70c60740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420b79d0 0x5564420b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda70c60740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 0x5564420c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda70c60740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420c2bb0 0x5564420c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda70c60740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55644201e090 con 0x5564420b79d0 2024-09-11T22:52:40.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda70c60740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556441fc5a70 con 0x5564420b82f0 2024-09-11T22:52:40.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda70c60740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55644201a050 con 0x5564420c2bb0 2024-09-11T22:52:40.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda6ce24640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420b79d0 0x5564420b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda6ce24640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420b79d0 0x5564420b7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58662/0 (socket says 172.21.15.152:58662) 2024-09-11T22:52:40.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda6ce24640 1 -- 172.21.15.152:0/4225909403 learned_addr learned my addr 172.21.15.152:0/4225909403 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:40.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda67fff640 1 --2- 172.21.15.152:0/4225909403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 0x5564420c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda5ffff640 1 --2- 172.21.15.152:0/4225909403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420c2bb0 0x5564420c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda65ffb640 1 --2- 172.21.15.152:0/1968258706 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fda540a2b80 0x7fda540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda6ce24640 1 -- 172.21.15.152:0/4225909403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420c2bb0 msgr2=0x5564420c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:40.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda6ce24640 1 --2- 172.21.15.152:0/4225909403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420c2bb0 0x5564420c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda6ce24640 1 -- 172.21.15.152:0/4225909403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 msgr2=0x5564420c2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:40.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda6ce24640 1 --2- 172.21.15.152:0/4225909403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 0x5564420c2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda6ce24640 1 -- 172.21.15.152:0/4225909403 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556441fd2130 con 0x5564420b79d0 2024-09-11T22:52:40.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.081+0000 7fda6ce24640 1 --2- 172.21.15.152:0/1968258706 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fda540a2b80 0x7fda540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.082+0000 7fda65ffb640 1 -- 172.21.15.152:0/1968258706 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fda580ceaa0 con 0x7fda68100e20 2024-09-11T22:52:40.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.082+0000 7fda6ce24640 1 --2- 172.21.15.152:0/4225909403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420b79d0 0x5564420b7db0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fda60005fd0 tx=0x7fda6000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c3bf408377f91 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:40.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.082+0000 7fda5e7fc640 1 -- 172.21.15.152:0/4225909403 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60011e00 con 0x5564420b79d0 2024-09-11T22:52:40.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.082+0000 7fda5e7fc640 1 -- 172.21.15.152:0/4225909403 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda6001adc0 con 0x5564420b79d0 2024-09-11T22:52:40.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.082+0000 7fda5e7fc640 1 -- 172.21.15.152:0/4225909403 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60019cb0 con 0x5564420b79d0 2024-09-11T22:52:40.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.082+0000 7fda6ce24640 1 --2- 172.21.15.152:0/1968258706 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fda540a2b80 0x7fda540a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fda6001e000 tx=0x7fda6001f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:40.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.082+0000 7fda70c60740 1 -- 172.21.15.152:0/4225909403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420b79d0 msgr2=0x5564420b7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:40.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.083+0000 7fda70c60740 1 --2- 172.21.15.152:0/4225909403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420b79d0 0x5564420b7db0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fda60005fd0 tx=0x7fda6000d710 comp rx=0 tx=0).stop 2024-09-11T22:52:40.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.083+0000 7fda70c60740 1 -- 172.21.15.152:0/4225909403 shutdown_connections 2024-09-11T22:52:40.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.083+0000 7fda70c60740 1 --2- 172.21.15.152:0/4225909403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420c2bb0 0x5564420c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.083+0000 7fda70c60740 1 --2- 172.21.15.152:0/4225909403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 0x5564420c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.083+0000 7fda70c60740 1 --2- 172.21.15.152:0/4225909403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420b79d0 0x5564420b7db0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.083+0000 7fda70c60740 1 -- 172.21.15.152:0/4225909403 >> 172.21.15.152:0/4225909403 conn(0x5564420b3f60 msgr2=0x5564420b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:40.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.083+0000 7fda70c60740 1 -- 172.21.15.152:0/4225909403 shutdown_connections 2024-09-11T22:52:40.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.083+0000 7fda70c60740 1 -- 172.21.15.152:0/4225909403 wait complete. 2024-09-11T22:52:40.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.083+0000 7fda70c60740 1 -- start start 2024-09-11T22:52:40.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.083+0000 7fda70c60740 1 Processor -- start 2024-09-11T22:52:40.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.083+0000 7fda70c60740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420b79d0 0x5564420c2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.083+0000 7fda70c60740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 0x5564420bf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.083+0000 7fda70c60740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420c2bb0 0x5564420bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.083+0000 7fda70c60740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5564420c5780 con 0x5564420c2bb0 2024-09-11T22:52:40.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.083+0000 7fda70c60740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556441fd8f20 con 0x5564420b82f0 2024-09-11T22:52:40.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.083+0000 7fda70c60740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5564420c5900 con 0x5564420b79d0 2024-09-11T22:52:40.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda70c60740 10 client.0 _ll_register_callbacks cb 0x556441fa2030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:40.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda6ce24640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420c2bb0 0x5564420bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda5ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 0x5564420bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda6ce24640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420c2bb0 0x5564420bfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58676/0 (socket says 172.21.15.152:58676) 2024-09-11T22:52:40.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda5ffff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 0x5564420bf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:47788/0 (socket says 172.21.15.152:47788) 2024-09-11T22:52:40.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda6ce24640 1 -- 172.21.15.152:0/1522945949 learned_addr learned my addr 172.21.15.152:0/1522945949 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:40.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda67fff640 1 --2- 172.21.15.152:0/1522945949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420b79d0 0x5564420c2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda6ce24640 1 -- 172.21.15.152:0/1522945949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420b79d0 msgr2=0x5564420c2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:40.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda6ce24640 1 --2- 172.21.15.152:0/1522945949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420b79d0 0x5564420c2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda6ce24640 1 -- 172.21.15.152:0/1522945949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 msgr2=0x5564420bf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:40.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda6ce24640 1 --2- 172.21.15.152:0/1522945949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 0x5564420bf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda6ce24640 1 -- 172.21.15.152:0/1522945949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556441d8f2b0 con 0x5564420c2bb0 2024-09-11T22:52:40.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda5ffff640 1 --2- 172.21.15.152:0/1522945949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 0x5564420bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:52:40.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda6ce24640 1 --2- 172.21.15.152:0/1522945949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420c2bb0 0x5564420bfca0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fda60019e60 tx=0x7fda600085b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd12640450b9493 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:40.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda477fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:40.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda477fe640 1 -- 172.21.15.152:0/1522945949 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60032060 con 0x5564420c2bb0 2024-09-11T22:52:40.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda477fe640 1 -- 172.21.15.152:0/1522945949 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fda60020070 con 0x5564420c2bb0 2024-09-11T22:52:40.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda70c60740 1 -- 172.21.15.152:0/1522945949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda6001adc0 con 0x5564420c2bb0 2024-09-11T22:52:40.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda477fe640 1 -- 172.21.15.152:0/1522945949 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fda6000de80 con 0x5564420c2bb0 2024-09-11T22:52:40.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda70c60740 10 client.15138 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:40.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.084+0000 7fda70c60740 1 -- 172.21.15.152:0/1522945949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5564420d1070 con 0x5564420c2bb0 2024-09-11T22:52:40.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.085+0000 7fda477fe640 1 -- 172.21.15.152:0/1522945949 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda6001adc0 con 0x5564420c2bb0 2024-09-11T22:52:40.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.085+0000 7fda477fe640 1 -- 172.21.15.152:0/1522945949 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fda60024c30 con 0x5564420c2bb0 2024-09-11T22:52:40.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.085+0000 7fda70c60740 10 client.15138 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:40.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.085+0000 7fda70c60740 10 client.15138 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T22:52:40.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.085+0000 7fda70c60740 20 client.15138 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:40.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.085+0000 7fda70c60740 1 --2- 172.21.15.152:0/1522945949 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5564420d3280 0x5564420d46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.085+0000 7fda70c60740 4 client.15138 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:52:40.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.085+0000 7fda70c60740 1 -- 172.21.15.152:0/1522945949 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55644202b600 con 0x5564420d3280 2024-09-11T22:52:40.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.085+0000 7fda67fff640 1 --2- 172.21.15.152:0/1522945949 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5564420d3280 0x5564420d46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.085+0000 7fda67fff640 1 --2- 172.21.15.152:0/1522945949 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5564420d3280 0x5564420d46f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=61069e0c6ecb7f3c server_cookie=3eb4ab9414075a0c in_seq=0 out_seq=0 2024-09-11T22:52:40.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.085+0000 7fda477fe640 10 client.15138 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:52:40.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.086+0000 7fda477fe640 1 -- 172.21.15.152:0/1522945949 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55644202b600 con 0x5564420d3280 2024-09-11T22:52:40.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.086+0000 7fda477fe640 10 client.15138 handle_command_reply: tid=0 2024-09-11T22:52:40.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.086+0000 7fda70c60740 1 client.15138 shutdown 2024-09-11T22:52:40.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.086+0000 7fda70c60740 1 -- 172.21.15.152:0/1522945949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420c2bb0 msgr2=0x5564420bfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:40.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.086+0000 7fda70c60740 1 --2- 172.21.15.152:0/1522945949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420c2bb0 0x5564420bfca0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fda60019e60 tx=0x7fda600085b0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.086+0000 7fda70c60740 1 -- 172.21.15.152:0/1522945949 shutdown_connections 2024-09-11T22:52:40.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.086+0000 7fda70c60740 1 --2- 172.21.15.152:0/1522945949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420c2bb0 0x5564420bfca0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.086+0000 7fda70c60740 1 --2- 172.21.15.152:0/1522945949 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5564420d3280 0x5564420d46f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.086+0000 7fda70c60740 1 --2- 172.21.15.152:0/1522945949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 0x5564420bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.086+0000 7fda477fe640 0 client.15138 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:52:40.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.086+0000 7fda70c60740 1 --2- 172.21.15.152:0/1522945949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420b79d0 0x5564420c2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.086+0000 7fda70c60740 1 -- 172.21.15.152:0/1522945949 >> 172.21.15.152:0/1522945949 conn(0x5564420b3f60 msgr2=0x5564420b9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:40.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.086+0000 7fda70c60740 1 -- 172.21.15.152:0/1522945949 shutdown_connections 2024-09-11T22:52:40.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.086+0000 7fda70c60740 1 -- 172.21.15.152:0/1522945949 wait complete. 2024-09-11T22:52:40.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.086+0000 7fda70c60740 20 client.15138 trim_cache size 0 max 16384 2024-09-11T22:52:40.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda70c60740 1 Processor -- start 2024-09-11T22:52:40.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda70c60740 1 -- start start 2024-09-11T22:52:40.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda70c60740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420b79d0 0x5564420d46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda70c60740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 0x5564420b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda70c60740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420c2bb0 0x5564420bc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda70c60740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5564420d2a30 con 0x5564420c2bb0 2024-09-11T22:52:40.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda70c60740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5564420d3100 con 0x5564420b82f0 2024-09-11T22:52:40.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda70c60740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5564420d37d0 con 0x5564420b79d0 2024-09-11T22:52:40.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda6ce24640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420c2bb0 0x5564420bc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda6ce24640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420c2bb0 0x5564420bc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58678/0 (socket says 172.21.15.152:58678) 2024-09-11T22:52:40.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda6ce24640 1 -- 172.21.15.152:0/4227439668 learned_addr learned my addr 172.21.15.152:0/4227439668 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:40.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda5ffff640 1 --2- 172.21.15.152:0/4227439668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 0x5564420b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda67fff640 1 --2- 172.21.15.152:0/4227439668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420b79d0 0x5564420d46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda6ce24640 1 -- 172.21.15.152:0/4227439668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420b79d0 msgr2=0x5564420d46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:40.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda6ce24640 1 --2- 172.21.15.152:0/4227439668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420b79d0 0x5564420d46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda6ce24640 1 -- 172.21.15.152:0/4227439668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 msgr2=0x5564420b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:40.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda6ce24640 1 --2- 172.21.15.152:0/4227439668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 0x5564420b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda6ce24640 1 -- 172.21.15.152:0/4227439668 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564420c0ae0 con 0x5564420c2bb0 2024-09-11T22:52:40.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda6ce24640 1 --2- 172.21.15.152:0/4227439668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420c2bb0 0x5564420bc9c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fda60008620 tx=0x7fda60011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4679fae9caa952e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:40.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.104+0000 7fda5dffb640 1 -- 172.21.15.152:0/4227439668 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6003d070 con 0x5564420c2bb0 2024-09-11T22:52:40.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.105+0000 7fda5dffb640 1 -- 172.21.15.152:0/4227439668 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda60019620 con 0x5564420c2bb0 2024-09-11T22:52:40.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.105+0000 7fda5dffb640 1 -- 172.21.15.152:0/4227439668 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60032070 con 0x5564420c2bb0 2024-09-11T22:52:40.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.105+0000 7fda70c60740 1 -- 172.21.15.152:0/4227439668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420c2bb0 msgr2=0x5564420bc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:40.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.105+0000 7fda70c60740 1 --2- 172.21.15.152:0/4227439668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420c2bb0 0x5564420bc9c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fda60008620 tx=0x7fda60011440 comp rx=0 tx=0).stop 2024-09-11T22:52:40.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.105+0000 7fda70c60740 1 -- 172.21.15.152:0/4227439668 shutdown_connections 2024-09-11T22:52:40.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.105+0000 7fda70c60740 1 --2- 172.21.15.152:0/4227439668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420c2bb0 0x5564420bc9c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.105+0000 7fda70c60740 1 --2- 172.21.15.152:0/4227439668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b82f0 0x5564420b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.105+0000 7fda70c60740 1 --2- 172.21.15.152:0/4227439668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420b79d0 0x5564420d46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.105+0000 7fda70c60740 1 -- 172.21.15.152:0/4227439668 >> 172.21.15.152:0/4227439668 conn(0x5564420b3f60 msgr2=0x5564420b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:40.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.105+0000 7fda70c60740 1 -- 172.21.15.152:0/4227439668 shutdown_connections 2024-09-11T22:52:40.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.105+0000 7fda70c60740 1 -- 172.21.15.152:0/4227439668 wait complete. 2024-09-11T22:52:40.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.106+0000 7fda70c60740 1 -- start start 2024-09-11T22:52:40.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.106+0000 7fda70c60740 1 Processor -- start 2024-09-11T22:52:40.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.106+0000 7fda70c60740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b79d0 0x5564420dcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.106+0000 7fda70c60740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420b82f0 0x5564420d82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.106+0000 7fda70c60740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420c2bb0 0x5564420d87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.106+0000 7fda70c60740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5564420d4570 con 0x5564420b82f0 2024-09-11T22:52:40.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.106+0000 7fda70c60740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5564420d3ea0 con 0x5564420b79d0 2024-09-11T22:52:40.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.106+0000 7fda70c60740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5564420d4c40 con 0x5564420c2bb0 2024-09-11T22:52:40.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.106+0000 7fda70c60740 10 client.0 _ll_register_callbacks cb 0x7fda6001d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:40.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.106+0000 7fda5ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420b82f0 0x5564420d82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.106+0000 7fda67fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b79d0 0x5564420dcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.106+0000 7fda5ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420b82f0 0x5564420d82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58694/0 (socket says 172.21.15.152:58694) 2024-09-11T22:52:40.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.106+0000 7fda5ffff640 1 -- 172.21.15.152:0/2956619000 learned_addr learned my addr 172.21.15.152:0/2956619000 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:40.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda5ffff640 1 -- 172.21.15.152:0/2956619000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420c2bb0 msgr2=0x5564420d87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:40.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda6ce24640 1 --2- 172.21.15.152:0/2956619000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420c2bb0 0x5564420d87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda5ffff640 1 --2- 172.21.15.152:0/2956619000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420c2bb0 0x5564420d87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda5ffff640 1 -- 172.21.15.152:0/2956619000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b79d0 msgr2=0x5564420dcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:40.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda5ffff640 1 --2- 172.21.15.152:0/2956619000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b79d0 0x5564420dcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda5ffff640 1 -- 172.21.15.152:0/2956619000 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564420d8e50 con 0x5564420b82f0 2024-09-11T22:52:40.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda67fff640 1 --2- 172.21.15.152:0/2956619000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b79d0 0x5564420dcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:52:40.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda5ffff640 1 --2- 172.21.15.152:0/2956619000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420b82f0 0x5564420d82a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fda50005c40 tx=0x7fda50005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58960b40e322e541 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:40.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda477fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:40.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda477fe640 1 -- 172.21.15.152:0/2956619000 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50035040 con 0x5564420b82f0 2024-09-11T22:52:40.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda477fe640 1 -- 172.21.15.152:0/2956619000 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fda500048d0 con 0x5564420b82f0 2024-09-11T22:52:40.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda70c60740 1 -- 172.21.15.152:0/2956619000 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda60019620 con 0x5564420b82f0 2024-09-11T22:52:40.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda477fe640 1 -- 172.21.15.152:0/2956619000 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fda50004ad0 con 0x5564420b82f0 2024-09-11T22:52:40.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda70c60740 10 client.15144 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:40.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda70c60740 1 -- 172.21.15.152:0/2956619000 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55644213d830 con 0x5564420b82f0 2024-09-11T22:52:40.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda477fe640 1 -- 172.21.15.152:0/2956619000 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda60019620 con 0x5564420b82f0 2024-09-11T22:52:40.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.107+0000 7fda477fe640 1 -- 172.21.15.152:0/2956619000 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fda50031640 con 0x5564420b82f0 2024-09-11T22:52:40.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda70c60740 10 client.15144 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:40.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda70c60740 10 client.15144 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T22:52:40.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda70c60740 20 client.15144 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:40.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda70c60740 1 --2- 172.21.15.152:0/2956619000 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5564421424b0 0x5564421411c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:40.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda70c60740 4 client.15144 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:52:40.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda70c60740 1 -- 172.21.15.152:0/2956619000 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "status"}) -- 0x556442025b10 con 0x5564421424b0 2024-09-11T22:52:40.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda67fff640 1 --2- 172.21.15.152:0/2956619000 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5564421424b0 0x5564421411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:40.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda477fe640 10 client.15144 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:52:40.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda67fff640 1 --2- 172.21.15.152:0/2956619000 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5564421424b0 0x5564421411c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4f59790b38cef62b server_cookie=31addf62d592f1e0 in_seq=0 out_seq=0 2024-09-11T22:52:40.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda477fe640 1 -- 172.21.15.152:0/2956619000 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556442025b10 con 0x5564421424b0 2024-09-11T22:52:40.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda477fe640 10 client.15144 handle_command_reply: tid=0 2024-09-11T22:52:40.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda70c60740 1 client.15144 shutdown 2024-09-11T22:52:40.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda70c60740 1 -- 172.21.15.152:0/2956619000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420b82f0 msgr2=0x5564420d82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:40.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda70c60740 1 --2- 172.21.15.152:0/2956619000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420b82f0 0x5564420d82a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fda50005c40 tx=0x7fda50005660 comp rx=0 tx=0).stop 2024-09-11T22:52:40.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda70c60740 1 -- 172.21.15.152:0/2956619000 shutdown_connections 2024-09-11T22:52:40.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda70c60740 1 --2- 172.21.15.152:0/2956619000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564420c2bb0 0x5564420d87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.108+0000 7fda70c60740 1 --2- 172.21.15.152:0/2956619000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564420b82f0 0x5564420d82a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda70c60740 1 --2- 172.21.15.152:0/2956619000 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5564421424b0 0x5564421411c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda70c60740 1 --2- 172.21.15.152:0/2956619000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564420b79d0 0x5564420dcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda70c60740 1 -- 172.21.15.152:0/2956619000 >> 172.21.15.152:0/2956619000 conn(0x5564420b3f60 msgr2=0x5564420b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:40.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda477fe640 0 client.15144 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:52:40.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda70c60740 1 -- 172.21.15.152:0/2956619000 shutdown_connections 2024-09-11T22:52:40.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda70c60740 1 -- 172.21.15.152:0/2956619000 wait complete. 2024-09-11T22:52:40.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda70c60740 20 client.15144 trim_cache size 0 max 16384 2024-09-11T22:52:40.121 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:52:40.121 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:52:40.121 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 4, 2024-09-11T22:52:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 14628, 2024-09-11T22:52:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-09-11T22:52:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-09-11T22:52:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-09-11T22:52:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 94.981609239999997, 2024-09-11T22:52:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 27, 2024-09-11T22:52:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 83, 2024-09-11T22:52:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T22:52:40.122 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 120.271366379 2024-09-11T22:52:40.122 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:52:40.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda6e887640 1 -- 172.21.15.152:0/1968258706 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fda540a2b80 msgr2=0x7fda540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:40.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda6e887640 1 --2- 172.21.15.152:0/1968258706 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fda540a2b80 0x7fda540a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fda6001e000 tx=0x7fda6001f000 comp rx=0 tx=0).stop 2024-09-11T22:52:40.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda6e887640 1 -- 172.21.15.152:0/1968258706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda68100e20 msgr2=0x7fda680fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:40.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda6e887640 1 --2- 172.21.15.152:0/1968258706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda68100e20 0x7fda680fcc80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fda5800d950 tx=0x7fda5800de10 comp rx=0 tx=0).stop 2024-09-11T22:52:40.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda6e887640 1 -- 172.21.15.152:0/1968258706 shutdown_connections 2024-09-11T22:52:40.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda6e887640 1 --2- 172.21.15.152:0/1968258706 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fda540a2b80 0x7fda540a5040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda6e887640 1 --2- 172.21.15.152:0/1968258706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fda68102ef0 0x7fda680fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda6e887640 1 --2- 172.21.15.152:0/1968258706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fda68102570 0x7fda680fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda6e887640 1 --2- 172.21.15.152:0/1968258706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda68100e20 0x7fda680fcc80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:40.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda6e887640 1 -- 172.21.15.152:0/1968258706 >> 172.21.15.152:0/1968258706 conn(0x7fda680fc7d0 msgr2=0x7fda6806a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:40.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda6e887640 1 -- 172.21.15.152:0/1968258706 shutdown_connections 2024-09-11T22:52:40.124 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:40.109+0000 7fda6e887640 1 -- 172.21.15.152:0/1968258706 wait complete. 2024-09-11T22:52:40.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:39 smithi191 ceph-mon[35211]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-11T22:52:40.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:39 smithi204 ceph-mon[35454]: pgmap v335: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-11T22:52:40.865 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T22:52:41.111 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:41.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.629+0000 7ffa53c5c640 1 -- 172.21.15.152:0/2574349978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa4c102790 msgr2=0x7ffa4c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:41.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.629+0000 7ffa53c5c640 1 --2- 172.21.15.152:0/2574349978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa4c102790 0x7ffa4c1067e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009900 tx=0x7ffa4002a410 comp rx=0 tx=0).stop 2024-09-11T22:52:41.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.629+0000 7ffa53c5c640 1 -- 172.21.15.152:0/2574349978 shutdown_connections 2024-09-11T22:52:41.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.629+0000 7ffa53c5c640 1 --2- 172.21.15.152:0/2574349978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa4c106f10 0x7ffa4c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.629+0000 7ffa53c5c640 1 --2- 172.21.15.152:0/2574349978 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa4c102790 0x7ffa4c1067e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.629+0000 7ffa53c5c640 1 --2- 172.21.15.152:0/2574349978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa4c101de0 0x7ffa4c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.629+0000 7ffa53c5c640 1 -- 172.21.15.152:0/2574349978 >> 172.21.15.152:0/2574349978 conn(0x7ffa4c0fd880 msgr2=0x7ffa4c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:41.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.629+0000 7ffa53c5c640 1 -- 172.21.15.152:0/2574349978 shutdown_connections 2024-09-11T22:52:41.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.629+0000 7ffa53c5c640 1 -- 172.21.15.152:0/2574349978 wait complete. 2024-09-11T22:52:41.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.629+0000 7ffa53c5c640 1 Processor -- start 2024-09-11T22:52:41.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.629+0000 7ffa53c5c640 1 -- start start 2024-09-11T22:52:41.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa53c5c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa4c101de0 0x7ffa4c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa53c5c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa4c102790 0x7ffa4c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa53c5c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa4c106f10 0x7ffa4c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa53c5c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c111f70 con 0x7ffa4c106f10 2024-09-11T22:52:41.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa53c5c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c111df0 con 0x7ffa4c102790 2024-09-11T22:52:41.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa53c5c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c1120f0 con 0x7ffa4c101de0 2024-09-11T22:52:41.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa521f9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa4c106f10 0x7ffa4c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:41.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa521f9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa4c106f10 0x7ffa4c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58708/0 (socket says 172.21.15.152:58708) 2024-09-11T22:52:41.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa521f9640 1 -- 172.21.15.152:0/2128139536 learned_addr learned my addr 172.21.15.152:0/2128139536 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:41.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa519f8640 1 --2- 172.21.15.152:0/2128139536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa4c101de0 0x7ffa4c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:41.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa511f7640 1 --2- 172.21.15.152:0/2128139536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa4c102790 0x7ffa4c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:41.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa521f9640 1 -- 172.21.15.152:0/2128139536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa4c101de0 msgr2=0x7ffa4c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:41.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa521f9640 1 --2- 172.21.15.152:0/2128139536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa4c101de0 0x7ffa4c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa521f9640 1 -- 172.21.15.152:0/2128139536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa4c102790 msgr2=0x7ffa4c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:41.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa521f9640 1 --2- 172.21.15.152:0/2128139536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa4c102790 0x7ffa4c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa521f9640 1 -- 172.21.15.152:0/2128139536 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4c1a6990 con 0x7ffa4c106f10 2024-09-11T22:52:41.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa521f9640 1 --2- 172.21.15.152:0/2128139536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa4c106f10 0x7ffa4c074f70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009fd0 tx=0x7ffa4400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:41.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa3effd640 1 -- 172.21.15.152:0/2128139536 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44017070 con 0x7ffa4c106f10 2024-09-11T22:52:41.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa3effd640 1 -- 172.21.15.152:0/2128139536 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa44003e50 con 0x7ffa4c106f10 2024-09-11T22:52:41.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa3effd640 1 -- 172.21.15.152:0/2128139536 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa440106c0 con 0x7ffa4c106f10 2024-09-11T22:52:41.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.630+0000 7ffa53c5c640 1 -- 172.21.15.152:0/2128139536 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4c1a6c80 con 0x7ffa4c106f10 2024-09-11T22:52:41.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.631+0000 7ffa53c5c640 1 -- 172.21.15.152:0/2128139536 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4c1a7190 con 0x7ffa4c106f10 2024-09-11T22:52:41.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.633+0000 7ffa3effd640 1 -- 172.21.15.152:0/2128139536 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7ffa44003970 con 0x7ffa4c106f10 2024-09-11T22:52:41.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.633+0000 7ffa56035740 1 Processor -- start 2024-09-11T22:52:41.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.633+0000 7ffa56035740 1 -- start start 2024-09-11T22:52:41.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa56035740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac539d0 0x55d42ac53db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa56035740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac542f0 0x55d42ac5e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa56035740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac5ebb0 0x55d42ac60fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa56035740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d42abba090 con 0x55d42ac539d0 2024-09-11T22:52:41.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa56035740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d42ab61a70 con 0x55d42ac5ebb0 2024-09-11T22:52:41.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa56035740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d42abb6050 con 0x55d42ac542f0 2024-09-11T22:52:41.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa3effd640 1 --2- 172.21.15.152:0/2128139536 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ffa380a2c00 0x7ffa380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa511f7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac539d0 0x55d42ac53db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:41.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa511f7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac539d0 0x55d42ac53db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58716/0 (socket says 172.21.15.152:58716) 2024-09-11T22:52:41.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa511f7640 1 -- 172.21.15.152:0/1411485367 learned_addr learned my addr 172.21.15.152:0/1411485367 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:41.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa519f8640 1 --2- 172.21.15.152:0/1411485367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac542f0 0x55d42ac5e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:41.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa521f9640 1 --2- 172.21.15.152:0/2128139536 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ffa380a2c00 0x7ffa380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:41.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa511f7640 1 --2- 172.21.15.152:0/1411485367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac5ebb0 0x55d42ac60fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:41.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa511f7640 1 -- 172.21.15.152:0/1411485367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac542f0 msgr2=0x55d42ac5e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:41.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa511f7640 1 --2- 172.21.15.152:0/1411485367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac542f0 0x55d42ac5e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa511f7640 1 -- 172.21.15.152:0/1411485367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac5ebb0 msgr2=0x55d42ac60fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:41.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa3effd640 1 -- 172.21.15.152:0/2128139536 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7ffa440cec00 con 0x7ffa4c106f10 2024-09-11T22:52:41.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa511f7640 1 --2- 172.21.15.152:0/1411485367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac5ebb0 0x55d42ac60fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa511f7640 1 -- 172.21.15.152:0/1411485367 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d42ab6e130 con 0x55d42ac539d0 2024-09-11T22:52:41.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa511f7640 1 --2- 172.21.15.152:0/1411485367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac539d0 0x55d42ac53db0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ffa4002a930 tx=0x7ffa40000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd73bf327e4ff236 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:41.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa521f9640 1 --2- 172.21.15.152:0/2128139536 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ffa380a2c00 0x7ffa380a50c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffa4409b240 tx=0x7ffa44090a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:41.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa37fff640 1 -- 172.21.15.152:0/1411485367 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40044030 con 0x55d42ac539d0 2024-09-11T22:52:41.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa37fff640 1 -- 172.21.15.152:0/1411485367 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa40005020 con 0x55d42ac539d0 2024-09-11T22:52:41.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.634+0000 7ffa37fff640 1 -- 172.21.15.152:0/1411485367 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40035040 con 0x55d42ac539d0 2024-09-11T22:52:41.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.635+0000 7ffa56035740 1 -- 172.21.15.152:0/1411485367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac539d0 msgr2=0x55d42ac53db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:41.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.635+0000 7ffa56035740 1 --2- 172.21.15.152:0/1411485367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac539d0 0x55d42ac53db0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ffa4002a930 tx=0x7ffa40000bc0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.635+0000 7ffa56035740 1 -- 172.21.15.152:0/1411485367 shutdown_connections 2024-09-11T22:52:41.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.635+0000 7ffa56035740 1 --2- 172.21.15.152:0/1411485367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac5ebb0 0x55d42ac60fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.635+0000 7ffa56035740 1 --2- 172.21.15.152:0/1411485367 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac542f0 0x55d42ac5e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.635+0000 7ffa56035740 1 --2- 172.21.15.152:0/1411485367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac539d0 0x55d42ac53db0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.635+0000 7ffa56035740 1 -- 172.21.15.152:0/1411485367 >> 172.21.15.152:0/1411485367 conn(0x55d42ac4ff60 msgr2=0x55d42ac50340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:41.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.635+0000 7ffa56035740 1 -- 172.21.15.152:0/1411485367 shutdown_connections 2024-09-11T22:52:41.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.635+0000 7ffa56035740 1 -- 172.21.15.152:0/1411485367 wait complete. 2024-09-11T22:52:41.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.636+0000 7ffa56035740 1 -- start start 2024-09-11T22:52:41.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.636+0000 7ffa56035740 1 Processor -- start 2024-09-11T22:52:41.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.636+0000 7ffa56035740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac539d0 0x55d42ac5e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.636+0000 7ffa56035740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac5b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.636+0000 7ffa56035740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac5ebb0 0x55d42ac5bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.636+0000 7ffa56035740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d42ac61780 con 0x55d42ac542f0 2024-09-11T22:52:41.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.636+0000 7ffa56035740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d42ab74f20 con 0x55d42ac5ebb0 2024-09-11T22:52:41.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.636+0000 7ffa56035740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d42ac61900 con 0x55d42ac539d0 2024-09-11T22:52:41.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.636+0000 7ffa56035740 10 client.0 _ll_register_callbacks cb 0x55d42ab3e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:41.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.636+0000 7ffa519f8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac5b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:41.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.636+0000 7ffa519f8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac5b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58730/0 (socket says 172.21.15.152:58730) 2024-09-11T22:52:41.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.636+0000 7ffa519f8640 1 -- 172.21.15.152:0/4234790062 learned_addr learned my addr 172.21.15.152:0/4234790062 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:41.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.636+0000 7ffa511f7640 1 --2- 172.21.15.152:0/4234790062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac539d0 0x55d42ac5e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:41.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa519f8640 1 -- 172.21.15.152:0/4234790062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac539d0 msgr2=0x55d42ac5e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:41.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa519f8640 1 --2- 172.21.15.152:0/4234790062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac539d0 0x55d42ac5e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa519f8640 1 -- 172.21.15.152:0/4234790062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac5ebb0 msgr2=0x55d42ac5bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:41.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa519f8640 1 --2- 172.21.15.152:0/4234790062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac5ebb0 0x55d42ac5bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa519f8640 1 -- 172.21.15.152:0/4234790062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d42a92b2b0 con 0x55d42ac542f0 2024-09-11T22:52:41.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa511f7640 1 --2- 172.21.15.152:0/4234790062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac539d0 0x55d42ac5e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:41.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa519f8640 1 --2- 172.21.15.152:0/4234790062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac5b760 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800ba10 tx=0x7ffa4800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04c5261843b7d4e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:41.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa357fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:41.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa357fa640 1 -- 172.21.15.152:0/4234790062 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48003e80 con 0x55d42ac542f0 2024-09-11T22:52:41.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa357fa640 1 -- 172.21.15.152:0/4234790062 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7ffa48011440 con 0x55d42ac542f0 2024-09-11T22:52:41.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa56035740 1 -- 172.21.15.152:0/4234790062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa40005020 con 0x55d42ac542f0 2024-09-11T22:52:41.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa357fa640 1 -- 172.21.15.152:0/4234790062 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa48011640 con 0x55d42ac542f0 2024-09-11T22:52:41.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa56035740 10 client.15156 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:41.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa56035740 1 -- 172.21.15.152:0/4234790062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d42ac6d150 con 0x55d42ac542f0 2024-09-11T22:52:41.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa357fa640 1 -- 172.21.15.152:0/4234790062 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa40005020 con 0x55d42ac542f0 2024-09-11T22:52:41.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa357fa640 1 -- 172.21.15.152:0/4234790062 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ffa48026ad0 con 0x55d42ac542f0 2024-09-11T22:52:41.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa56035740 10 client.15156 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:41.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa56035740 10 client.15156 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T22:52:41.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa56035740 20 client.15156 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:41.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.637+0000 7ffa56035740 1 --2- 172.21.15.152:0/4234790062 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55d42ac6f810 0x55d42ac70c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.638+0000 7ffa56035740 4 client.15156 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:52:41.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.638+0000 7ffa56035740 1 -- 172.21.15.152:0/4234790062 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d42abc7600 con 0x55d42ac6f810 2024-09-11T22:52:41.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.638+0000 7ffa511f7640 1 --2- 172.21.15.152:0/4234790062 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55d42ac6f810 0x55d42ac70c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:41.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.638+0000 7ffa511f7640 1 --2- 172.21.15.152:0/4234790062 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55d42ac6f810 0x55d42ac70c80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4c21379d811100b7 server_cookie=2a2daca99092264 in_seq=0 out_seq=0 2024-09-11T22:52:41.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.638+0000 7ffa357fa640 10 client.15156 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:52:41.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.639+0000 7ffa357fa640 1 -- 172.21.15.152:0/4234790062 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d42abc7600 con 0x55d42ac6f810 2024-09-11T22:52:41.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.639+0000 7ffa357fa640 10 client.15156 handle_command_reply: tid=0 2024-09-11T22:52:41.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.639+0000 7ffa56035740 1 client.15156 shutdown 2024-09-11T22:52:41.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.639+0000 7ffa56035740 1 -- 172.21.15.152:0/4234790062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 msgr2=0x55d42ac5b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:41.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.639+0000 7ffa56035740 1 --2- 172.21.15.152:0/4234790062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac5b760 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800ba10 tx=0x7ffa4800bed0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.639+0000 7ffa56035740 1 -- 172.21.15.152:0/4234790062 shutdown_connections 2024-09-11T22:52:41.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.639+0000 7ffa56035740 1 --2- 172.21.15.152:0/4234790062 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55d42ac6f810 0x55d42ac70c80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.639+0000 7ffa56035740 1 --2- 172.21.15.152:0/4234790062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac5ebb0 0x55d42ac5bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.639+0000 7ffa56035740 1 --2- 172.21.15.152:0/4234790062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac5b760 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.639+0000 7ffa56035740 1 --2- 172.21.15.152:0/4234790062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac539d0 0x55d42ac5e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.639+0000 7ffa357fa640 0 client.15156 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:52:41.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.639+0000 7ffa56035740 1 -- 172.21.15.152:0/4234790062 >> 172.21.15.152:0/4234790062 conn(0x55d42ac4ff60 msgr2=0x55d42ac55c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:41.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.639+0000 7ffa56035740 1 -- 172.21.15.152:0/4234790062 shutdown_connections 2024-09-11T22:52:41.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.640+0000 7ffa56035740 1 -- 172.21.15.152:0/4234790062 wait complete. 2024-09-11T22:52:41.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.640+0000 7ffa56035740 20 client.15156 trim_cache size 0 max 16384 2024-09-11T22:52:41.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa56035740 1 Processor -- start 2024-09-11T22:52:41.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa56035740 1 -- start start 2024-09-11T22:52:41.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa56035740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac539d0 0x55d42ac6f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa56035740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac6f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa56035740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac5ebb0 0x55d42ac58a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa56035740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d42ac6efc0 con 0x55d42ac542f0 2024-09-11T22:52:41.678 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa56035740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d42ac6f690 con 0x55d42ac539d0 2024-09-11T22:52:41.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa56035740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d42ac6fd60 con 0x55d42ac5ebb0 2024-09-11T22:52:41.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa519f8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac6f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:41.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa519f8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac6f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58746/0 (socket says 172.21.15.152:58746) 2024-09-11T22:52:41.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa519f8640 1 -- 172.21.15.152:0/2488012005 learned_addr learned my addr 172.21.15.152:0/2488012005 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:41.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa511f7640 1 --2- 172.21.15.152:0/2488012005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac539d0 0x55d42ac6f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:41.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa519f8640 1 -- 172.21.15.152:0/2488012005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac5ebb0 msgr2=0x55d42ac58a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:41.679 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa519f8640 1 --2- 172.21.15.152:0/2488012005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac5ebb0 0x55d42ac58a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa519f8640 1 -- 172.21.15.152:0/2488012005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac539d0 msgr2=0x55d42ac6f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:41.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa519f8640 1 --2- 172.21.15.152:0/2488012005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac539d0 0x55d42ac6f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.657+0000 7ffa519f8640 1 -- 172.21.15.152:0/2488012005 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d42ac54120 con 0x55d42ac542f0 2024-09-11T22:52:41.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.658+0000 7ffa519f8640 1 --2- 172.21.15.152:0/2488012005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac6f810 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ffa48026e40 tx=0x7ffa48003a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde3a3d780a80679 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:41.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.658+0000 7ffa377fe640 1 -- 172.21.15.152:0/2488012005 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48011440 con 0x55d42ac542f0 2024-09-11T22:52:41.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.658+0000 7ffa377fe640 1 -- 172.21.15.152:0/2488012005 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa4801bdf0 con 0x55d42ac542f0 2024-09-11T22:52:41.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.658+0000 7ffa377fe640 1 -- 172.21.15.152:0/2488012005 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa480117e0 con 0x55d42ac542f0 2024-09-11T22:52:41.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 -- 172.21.15.152:0/2488012005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 msgr2=0x55d42ac6f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:41.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 --2- 172.21.15.152:0/2488012005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac6f810 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ffa48026e40 tx=0x7ffa48003a40 comp rx=0 tx=0).stop 2024-09-11T22:52:41.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 -- 172.21.15.152:0/2488012005 shutdown_connections 2024-09-11T22:52:41.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 --2- 172.21.15.152:0/2488012005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac5ebb0 0x55d42ac58a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 --2- 172.21.15.152:0/2488012005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac6f810 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.680 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 --2- 172.21.15.152:0/2488012005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac539d0 0x55d42ac6f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 -- 172.21.15.152:0/2488012005 >> 172.21.15.152:0/2488012005 conn(0x55d42ac4ff60 msgr2=0x55d42ac5a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:41.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 -- 172.21.15.152:0/2488012005 shutdown_connections 2024-09-11T22:52:41.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 -- 172.21.15.152:0/2488012005 wait complete. 2024-09-11T22:52:41.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 -- start start 2024-09-11T22:52:41.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 Processor -- start 2024-09-11T22:52:41.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac539d0 0x55d42ac71f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac72480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac5ebb0 0x55d42acd7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d42ac70b00 con 0x55d42ac542f0 2024-09-11T22:52:41.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d42ac70430 con 0x55d42ac5ebb0 2024-09-11T22:52:41.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d42ac711d0 con 0x55d42ac539d0 2024-09-11T22:52:41.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.659+0000 7ffa56035740 10 client.0 _ll_register_callbacks cb 0x7ffa480042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:41.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa519f8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac72480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:41.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa519f8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac72480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58760/0 (socket says 172.21.15.152:58760) 2024-09-11T22:52:41.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa519f8640 1 -- 172.21.15.152:0/3475257848 learned_addr learned my addr 172.21.15.152:0/3475257848 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:41.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa511f7640 1 --2- 172.21.15.152:0/3475257848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac539d0 0x55d42ac71f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:41.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa511f7640 1 --2- 172.21.15.152:0/3475257848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac5ebb0 0x55d42acd7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:41.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa519f8640 1 -- 172.21.15.152:0/3475257848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac539d0 msgr2=0x55d42ac71f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:41.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa519f8640 1 --2- 172.21.15.152:0/3475257848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac539d0 0x55d42ac71f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa519f8640 1 -- 172.21.15.152:0/3475257848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac5ebb0 msgr2=0x55d42acd7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:41.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa519f8640 1 --2- 172.21.15.152:0/3475257848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac5ebb0 0x55d42acd7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa519f8640 1 -- 172.21.15.152:0/3475257848 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d42ac72d10 con 0x55d42ac542f0 2024-09-11T22:52:41.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa519f8640 1 --2- 172.21.15.152:0/3475257848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac72480 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ffa48005d00 tx=0x7ffa480052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a19bacf362ec09 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:41.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa357fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:41.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa357fa640 1 -- 172.21.15.152:0/3475257848 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4800b450 con 0x55d42ac542f0 2024-09-11T22:52:41.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa357fa640 1 -- 172.21.15.152:0/3475257848 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7ffa480055f0 con 0x55d42ac542f0 2024-09-11T22:52:41.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa56035740 1 -- 172.21.15.152:0/3475257848 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa4801bdf0 con 0x55d42ac542f0 2024-09-11T22:52:41.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa357fa640 1 -- 172.21.15.152:0/3475257848 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa480117e0 con 0x55d42ac542f0 2024-09-11T22:52:41.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa56035740 10 client.15162 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:41.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa56035740 1 -- 172.21.15.152:0/3475257848 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d42acd9520 con 0x55d42ac542f0 2024-09-11T22:52:41.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.660+0000 7ffa357fa640 1 -- 172.21.15.152:0/3475257848 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa4801bdf0 con 0x55d42ac542f0 2024-09-11T22:52:41.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.661+0000 7ffa357fa640 1 -- 172.21.15.152:0/3475257848 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ffa48026420 con 0x55d42ac542f0 2024-09-11T22:52:41.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.661+0000 7ffa56035740 10 client.15162 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:41.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.661+0000 7ffa56035740 10 client.15162 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T22:52:41.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.661+0000 7ffa56035740 20 client.15162 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:41.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.661+0000 7ffa56035740 1 --2- 172.21.15.152:0/3475257848 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55d42acdde30 0x55d42acdcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:41.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.661+0000 7ffa56035740 4 client.15162 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:52:41.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.661+0000 7ffa56035740 1 -- 172.21.15.152:0/3475257848 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "status"}) -- 0x55d42abc1b10 con 0x55d42acdde30 2024-09-11T22:52:41.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.661+0000 7ffa511f7640 1 --2- 172.21.15.152:0/3475257848 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55d42acdde30 0x55d42acdcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:41.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.661+0000 7ffa357fa640 10 client.15162 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:52:41.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.661+0000 7ffa511f7640 1 --2- 172.21.15.152:0/3475257848 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55d42acdde30 0x55d42acdcb40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b8b2a7c8cf8325bf server_cookie=f5cc2706850176e in_seq=0 out_seq=0 2024-09-11T22:52:41.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.661+0000 7ffa357fa640 1 -- 172.21.15.152:0/3475257848 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d42abc1b10 con 0x55d42acdde30 2024-09-11T22:52:41.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.661+0000 7ffa357fa640 10 client.15162 handle_command_reply: tid=0 2024-09-11T22:52:41.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa56035740 1 client.15162 shutdown 2024-09-11T22:52:41.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa56035740 1 -- 172.21.15.152:0/3475257848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 msgr2=0x55d42ac72480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:41.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa56035740 1 --2- 172.21.15.152:0/3475257848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac72480 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ffa48005d00 tx=0x7ffa480052d0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa56035740 1 -- 172.21.15.152:0/3475257848 shutdown_connections 2024-09-11T22:52:41.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa56035740 1 --2- 172.21.15.152:0/3475257848 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55d42acdde30 0x55d42acdcb40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa56035740 1 --2- 172.21.15.152:0/3475257848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d42ac5ebb0 0x55d42acd7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa56035740 1 --2- 172.21.15.152:0/3475257848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d42ac542f0 0x55d42ac72480 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa56035740 1 --2- 172.21.15.152:0/3475257848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d42ac539d0 0x55d42ac71f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa357fa640 0 client.15162 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:52:41.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa56035740 1 -- 172.21.15.152:0/3475257848 >> 172.21.15.152:0/3475257848 conn(0x55d42ac4ff60 msgr2=0x55d42ac4e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:41.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa56035740 1 -- 172.21.15.152:0/3475257848 shutdown_connections 2024-09-11T22:52:41.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa56035740 1 -- 172.21.15.152:0/3475257848 wait complete. 2024-09-11T22:52:41.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa56035740 20 client.15162 trim_cache size 0 max 16384 2024-09-11T22:52:41.685 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:52:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:52:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 3, 2024-09-11T22:52:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 24430, 2024-09-11T22:52:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-09-11T22:52:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-09-11T22:52:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-09-11T22:52:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 97.439905573000004, 2024-09-11T22:52:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 27, 2024-09-11T22:52:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 83, 2024-09-11T22:52:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T22:52:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 133.315753392 2024-09-11T22:52:41.686 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:52:41.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa53c5c640 1 -- 172.21.15.152:0/2128139536 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ffa380a2c00 msgr2=0x7ffa380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:41.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa53c5c640 1 --2- 172.21.15.152:0/2128139536 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ffa380a2c00 0x7ffa380a50c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffa4409b240 tx=0x7ffa44090a20 comp rx=0 tx=0).stop 2024-09-11T22:52:41.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa53c5c640 1 -- 172.21.15.152:0/2128139536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa4c106f10 msgr2=0x7ffa4c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:41.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa53c5c640 1 --2- 172.21.15.152:0/2128139536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa4c106f10 0x7ffa4c074f70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009fd0 tx=0x7ffa4400b260 comp rx=0 tx=0).stop 2024-09-11T22:52:41.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa53c5c640 1 -- 172.21.15.152:0/2128139536 shutdown_connections 2024-09-11T22:52:41.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.662+0000 7ffa53c5c640 1 --2- 172.21.15.152:0/2128139536 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7ffa380a2c00 0x7ffa380a50c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.663+0000 7ffa53c5c640 1 --2- 172.21.15.152:0/2128139536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffa4c106f10 0x7ffa4c074f70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.663+0000 7ffa53c5c640 1 --2- 172.21.15.152:0/2128139536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa4c102790 0x7ffa4c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.663+0000 7ffa53c5c640 1 --2- 172.21.15.152:0/2128139536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa4c101de0 0x7ffa4c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:41.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.663+0000 7ffa53c5c640 1 -- 172.21.15.152:0/2128139536 >> 172.21.15.152:0/2128139536 conn(0x7ffa4c0fd880 msgr2=0x7ffa4c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:41.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.663+0000 7ffa53c5c640 1 -- 172.21.15.152:0/2128139536 shutdown_connections 2024-09-11T22:52:41.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:41.663+0000 7ffa53c5c640 1 -- 172.21.15.152:0/2128139536 wait complete. 2024-09-11T22:52:42.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:41 smithi191 ceph-mon[35211]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T22:52:42.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:41 smithi204 ceph-mon[35454]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T22:52:42.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:41 smithi152 ceph-mon[31181]: pgmap v336: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T22:52:42.476 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T22:52:42.721 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:43.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.299+0000 7fcfce799640 1 -- 172.21.15.152:0/4218227160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfc8102570 msgr2=0x7fcfc81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:43.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.299+0000 7fcfce799640 1 --2- 172.21.15.152:0/4218227160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfc8102570 0x7fcfc81029b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009900 tx=0x7fcfb002a480 comp rx=0 tx=0).stop 2024-09-11T22:52:43.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.299+0000 7fcfce799640 1 -- 172.21.15.152:0/4218227160 shutdown_connections 2024-09-11T22:52:43.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.299+0000 7fcfce799640 1 --2- 172.21.15.152:0/4218227160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfc8102ef0 0x7fcfc810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.299+0000 7fcfce799640 1 --2- 172.21.15.152:0/4218227160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfc8102570 0x7fcfc81029b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.299+0000 7fcfce799640 1 --2- 172.21.15.152:0/4218227160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfc8100e20 0x7fcfc8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.299+0000 7fcfce799640 1 -- 172.21.15.152:0/4218227160 >> 172.21.15.152:0/4218227160 conn(0x7fcfc80fc7d0 msgr2=0x7fcfc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:43.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.299+0000 7fcfce799640 1 -- 172.21.15.152:0/4218227160 shutdown_connections 2024-09-11T22:52:43.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.299+0000 7fcfce799640 1 -- 172.21.15.152:0/4218227160 wait complete. 2024-09-11T22:52:43.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.299+0000 7fcfce799640 1 Processor -- start 2024-09-11T22:52:43.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.299+0000 7fcfce799640 1 -- start start 2024-09-11T22:52:43.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfce799640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfc8100e20 0x7fcfc80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfce799640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfc8102570 0x7fcfc80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfce799640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfc8102ef0 0x7fcfc80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfce799640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc8110f80 con 0x7fcfc8102570 2024-09-11T22:52:43.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfce799640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc8110e00 con 0x7fcfc8102ef0 2024-09-11T22:52:43.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfce799640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc8111100 con 0x7fcfc8100e20 2024-09-11T22:52:43.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfbffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfc8102570 0x7fcfc80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:43.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfbffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfc8102570 0x7fcfc80fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58766/0 (socket says 172.21.15.152:58766) 2024-09-11T22:52:43.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfbffff640 1 -- 172.21.15.152:0/2649897814 learned_addr learned my addr 172.21.15.152:0/2649897814 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:43.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfc7fff640 1 --2- 172.21.15.152:0/2649897814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfc8100e20 0x7fcfc80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:43.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfbffff640 1 -- 172.21.15.152:0/2649897814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfc8100e20 msgr2=0x7fcfc80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:43.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfbffff640 1 --2- 172.21.15.152:0/2649897814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfc8100e20 0x7fcfc80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfccd36640 1 --2- 172.21.15.152:0/2649897814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfc8102ef0 0x7fcfc80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:43.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfbffff640 1 -- 172.21.15.152:0/2649897814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfc8102ef0 msgr2=0x7fcfc80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:43.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfbffff640 1 --2- 172.21.15.152:0/2649897814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfc8102ef0 0x7fcfc80fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfbffff640 1 -- 172.21.15.152:0/2649897814 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc81a6a30 con 0x7fcfc8102570 2024-09-11T22:52:43.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfccd36640 1 --2- 172.21.15.152:0/2649897814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfc8102ef0 0x7fcfc80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:52:43.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfbffff640 1 --2- 172.21.15.152:0/2649897814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfc8102570 0x7fcfc80fd1c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0006010 tx=0x7fcfb0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:43.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfc5ffb640 1 -- 172.21.15.152:0/2649897814 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb003f070 con 0x7fcfc8102570 2024-09-11T22:52:43.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfc5ffb640 1 -- 172.21.15.152:0/2649897814 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb0031c30 con 0x7fcfc8102570 2024-09-11T22:52:43.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfce799640 1 -- 172.21.15.152:0/2649897814 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc81a6cc0 con 0x7fcfc8102570 2024-09-11T22:52:43.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfc5ffb640 1 -- 172.21.15.152:0/2649897814 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0035030 con 0x7fcfc8102570 2024-09-11T22:52:43.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.300+0000 7fcfce799640 1 -- 172.21.15.152:0/2649897814 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc81a71a0 con 0x7fcfc8102570 2024-09-11T22:52:43.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.303+0000 7fcfc5ffb640 1 -- 172.21.15.152:0/2649897814 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fcfb0042070 con 0x7fcfc8102570 2024-09-11T22:52:43.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.303+0000 7fcfd0b72740 1 Processor -- start 2024-09-11T22:52:43.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.303+0000 7fcfd0b72740 1 -- start start 2024-09-11T22:52:43.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.303+0000 7fcfd0b72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a49d0 0x55be118a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.303+0000 7fcfd0b72740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a52f0 0x55be118af670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.303+0000 7fcfd0b72740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 0x55be118b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.303+0000 7fcfd0b72740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55be1180b090 con 0x55be118a52f0 2024-09-11T22:52:43.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.303+0000 7fcfd0b72740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55be117b2a70 con 0x55be118afbb0 2024-09-11T22:52:43.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.303+0000 7fcfd0b72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55be11807050 con 0x55be118a49d0 2024-09-11T22:52:43.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.303+0000 7fcfc7fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a52f0 0x55be118af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:43.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.303+0000 7fcfc7fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a52f0 0x55be118af670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58778/0 (socket says 172.21.15.152:58778) 2024-09-11T22:52:43.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.303+0000 7fcfc7fff640 1 -- 172.21.15.152:0/744262774 learned_addr learned my addr 172.21.15.152:0/744262774 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:43.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.303+0000 7fcfccd36640 1 --2- 172.21.15.152:0/744262774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a49d0 0x55be118a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:43.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.304+0000 7fcfc5ffb640 1 --2- 172.21.15.152:0/2649897814 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcfb40a2c00 0x7fcfb40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.304+0000 7fcfbffff640 1 --2- 172.21.15.152:0/744262774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 0x55be118b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:43.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.304+0000 7fcfc7fff640 1 -- 172.21.15.152:0/744262774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a49d0 msgr2=0x55be118a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:43.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.304+0000 7fcfc7fff640 1 --2- 172.21.15.152:0/744262774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a49d0 0x55be118a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.304+0000 7fcfc7fff640 1 -- 172.21.15.152:0/744262774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 msgr2=0x55be118b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:43.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.304+0000 7fcfc7fff640 1 --2- 172.21.15.152:0/744262774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 0x55be118b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.304+0000 7fcfc7fff640 1 -- 172.21.15.152:0/744262774 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be117bf130 con 0x55be118a52f0 2024-09-11T22:52:43.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.304+0000 7fcfccd36640 1 --2- 172.21.15.152:0/2649897814 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcfb40a2c00 0x7fcfb40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:43.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.304+0000 7fcfbffff640 1 --2- 172.21.15.152:0/744262774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 0x55be118b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:52:43.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.304+0000 7fcfc5ffb640 1 -- 172.21.15.152:0/2649897814 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fcfb00ef8f0 con 0x7fcfc8102570 2024-09-11T22:52:43.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.304+0000 7fcfc7fff640 1 --2- 172.21.15.152:0/744262774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a52f0 0x55be118af670 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800cbe0 tx=0x7fcfb800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a167e632d2ee6ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:43.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.304+0000 7fcfbe7fc640 1 -- 172.21.15.152:0/744262774 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8010630 con 0x55be118a52f0 2024-09-11T22:52:43.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.304+0000 7fcfbe7fc640 1 -- 172.21.15.152:0/744262774 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb8010cb0 con 0x55be118a52f0 2024-09-11T22:52:43.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.304+0000 7fcfbe7fc640 1 -- 172.21.15.152:0/744262774 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb801ac20 con 0x55be118a52f0 2024-09-11T22:52:43.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.305+0000 7fcfd0b72740 1 -- 172.21.15.152:0/744262774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a52f0 msgr2=0x55be118af670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:43.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.305+0000 7fcfccd36640 1 --2- 172.21.15.152:0/2649897814 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcfb40a2c00 0x7fcfb40a50c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0005fd0 tx=0x7fcfc0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:43.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.305+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/744262774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a52f0 0x55be118af670 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800cbe0 tx=0x7fcfb800e720 comp rx=0 tx=0).stop 2024-09-11T22:52:43.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.305+0000 7fcfd0b72740 1 -- 172.21.15.152:0/744262774 shutdown_connections 2024-09-11T22:52:43.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.305+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/744262774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 0x55be118b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.305+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/744262774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a52f0 0x55be118af670 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.305+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/744262774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a49d0 0x55be118a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.305+0000 7fcfd0b72740 1 -- 172.21.15.152:0/744262774 >> 172.21.15.152:0/744262774 conn(0x55be118a0f60 msgr2=0x55be118a1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:43.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.305+0000 7fcfd0b72740 1 -- 172.21.15.152:0/744262774 shutdown_connections 2024-09-11T22:52:43.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.305+0000 7fcfd0b72740 1 -- 172.21.15.152:0/744262774 wait complete. 2024-09-11T22:52:43.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.305+0000 7fcfd0b72740 1 -- start start 2024-09-11T22:52:43.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.305+0000 7fcfd0b72740 1 Processor -- start 2024-09-11T22:52:43.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfd0b72740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a49d0 0x55be118af430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfd0b72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a52f0 0x55be118ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfd0b72740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 0x55be118accc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfd0b72740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55be118b2780 con 0x55be118a49d0 2024-09-11T22:52:43.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfd0b72740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55be117c5f20 con 0x55be118afbb0 2024-09-11T22:52:43.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfd0b72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55be118b2900 con 0x55be118a52f0 2024-09-11T22:52:43.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfd0b72740 10 client.0 _ll_register_callbacks cb 0x55be1178f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:43.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfc7fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a49d0 0x55be118af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:43.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfc7fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a49d0 0x55be118af430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58792/0 (socket says 172.21.15.152:58792) 2024-09-11T22:52:43.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfc7fff640 1 -- 172.21.15.152:0/2985934027 learned_addr learned my addr 172.21.15.152:0/2985934027 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:43.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfbffff640 1 --2- 172.21.15.152:0/2985934027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a52f0 0x55be118ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:43.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfccd36640 1 --2- 172.21.15.152:0/2985934027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 0x55be118accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:43.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfc7fff640 1 -- 172.21.15.152:0/2985934027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a52f0 msgr2=0x55be118ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:43.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfc7fff640 1 --2- 172.21.15.152:0/2985934027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a52f0 0x55be118ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfc7fff640 1 -- 172.21.15.152:0/2985934027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 msgr2=0x55be118accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:43.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfc7fff640 1 --2- 172.21.15.152:0/2985934027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 0x55be118accc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfc7fff640 1 -- 172.21.15.152:0/2985934027 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be1157c2b0 con 0x55be118a49d0 2024-09-11T22:52:43.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfc7fff640 1 --2- 172.21.15.152:0/2985934027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a49d0 0x55be118af430 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800ec20 tx=0x7fcfb800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d0c850f2dd1661 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:43.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfa77fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:43.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfa77fe640 1 -- 172.21.15.152:0/2985934027 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8010e60 con 0x55be118a49d0 2024-09-11T22:52:43.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfa77fe640 1 -- 172.21.15.152:0/2985934027 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fcfb801abf0 con 0x55be118a49d0 2024-09-11T22:52:43.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.306+0000 7fcfd0b72740 1 -- 172.21.15.152:0/2985934027 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb8010cb0 con 0x55be118a49d0 2024-09-11T22:52:43.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.307+0000 7fcfa77fe640 1 -- 172.21.15.152:0/2985934027 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcfb801adf0 con 0x55be118a49d0 2024-09-11T22:52:43.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.307+0000 7fcfd0b72740 10 client.15174 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:43.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.307+0000 7fcfd0b72740 1 -- 172.21.15.152:0/2985934027 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55be118be030 con 0x55be118a49d0 2024-09-11T22:52:43.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.307+0000 7fcfa77fe640 1 -- 172.21.15.152:0/2985934027 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb8010cb0 con 0x55be118a49d0 2024-09-11T22:52:43.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.307+0000 7fcfa77fe640 1 -- 172.21.15.152:0/2985934027 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcfb80035f0 con 0x55be118a49d0 2024-09-11T22:52:43.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.307+0000 7fcfd0b72740 10 client.15174 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:43.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.307+0000 7fcfd0b72740 10 client.15174 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T22:52:43.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.307+0000 7fcfd0b72740 20 client.15174 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:43.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.307+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/2985934027 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55be118c0270 0x55be118c16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.307+0000 7fcfd0b72740 4 client.15174 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:52:43.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.307+0000 7fcfd0b72740 1 -- 172.21.15.152:0/2985934027 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be11818600 con 0x55be118c0270 2024-09-11T22:52:43.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.307+0000 7fcfbffff640 1 --2- 172.21.15.152:0/2985934027 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55be118c0270 0x55be118c16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:43.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.308+0000 7fcfbffff640 1 --2- 172.21.15.152:0/2985934027 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55be118c0270 0x55be118c16e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=906af742fb75220c server_cookie=1df64bf2320f6bfc in_seq=0 out_seq=0 2024-09-11T22:52:43.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.308+0000 7fcfa77fe640 10 client.15174 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:52:43.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.308+0000 7fcfa77fe640 1 -- 172.21.15.152:0/2985934027 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be11818600 con 0x55be118c0270 2024-09-11T22:52:43.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.308+0000 7fcfa77fe640 10 client.15174 handle_command_reply: tid=0 2024-09-11T22:52:43.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.308+0000 7fcfd0b72740 1 client.15174 shutdown 2024-09-11T22:52:43.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.308+0000 7fcfd0b72740 1 -- 172.21.15.152:0/2985934027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a49d0 msgr2=0x55be118af430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:43.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.308+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/2985934027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a49d0 0x55be118af430 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800ec20 tx=0x7fcfb800c920 comp rx=0 tx=0).stop 2024-09-11T22:52:43.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.309+0000 7fcfd0b72740 1 -- 172.21.15.152:0/2985934027 shutdown_connections 2024-09-11T22:52:43.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.309+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/2985934027 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55be118c0270 0x55be118c16e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.309+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/2985934027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 0x55be118accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.309+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/2985934027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a52f0 0x55be118ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.309+0000 7fcfa77fe640 0 client.15174 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:52:43.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.309+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/2985934027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a49d0 0x55be118af430 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.309+0000 7fcfd0b72740 1 -- 172.21.15.152:0/2985934027 >> 172.21.15.152:0/2985934027 conn(0x55be118a0f60 msgr2=0x55be118a6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:43.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.309+0000 7fcfd0b72740 1 -- 172.21.15.152:0/2985934027 shutdown_connections 2024-09-11T22:52:43.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.309+0000 7fcfd0b72740 1 -- 172.21.15.152:0/2985934027 wait complete. 2024-09-11T22:52:43.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.309+0000 7fcfd0b72740 20 client.15174 trim_cache size 0 max 16384 2024-09-11T22:52:43.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.326+0000 7fcfd0b72740 1 Processor -- start 2024-09-11T22:52:43.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.326+0000 7fcfd0b72740 1 -- start start 2024-09-11T22:52:43.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfd0b72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a49d0 0x55be118c16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfd0b72740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a52f0 0x55be1189fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfd0b72740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 0x55be118a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfd0b72740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55be118bfa20 con 0x55be118a52f0 2024-09-11T22:52:43.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfd0b72740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55be118c00f0 con 0x55be118afbb0 2024-09-11T22:52:43.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfd0b72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55be118c07c0 con 0x55be118a49d0 2024-09-11T22:52:43.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfbffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a52f0 0x55be1189fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:43.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfbffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a52f0 0x55be1189fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58796/0 (socket says 172.21.15.152:58796) 2024-09-11T22:52:43.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfbffff640 1 -- 172.21.15.152:0/2556296269 learned_addr learned my addr 172.21.15.152:0/2556296269 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:43.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfbffff640 1 -- 172.21.15.152:0/2556296269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a49d0 msgr2=0x55be118c16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:43.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfc7fff640 1 --2- 172.21.15.152:0/2556296269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a49d0 0x55be118c16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:43.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfbffff640 1 --2- 172.21.15.152:0/2556296269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a49d0 0x55be118c16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfbffff640 1 -- 172.21.15.152:0/2556296269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 msgr2=0x55be118a99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:43.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfbffff640 1 --2- 172.21.15.152:0/2556296269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 0x55be118a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfbffff640 1 -- 172.21.15.152:0/2556296269 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be118a5120 con 0x55be118a52f0 2024-09-11T22:52:43.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfbffff640 1 --2- 172.21.15.152:0/2556296269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a52f0 0x55be1189fef0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00096c0 tx=0x7fcfb00ed2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb695a00b585a6c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:43.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfbdffb640 1 -- 172.21.15.152:0/2556296269 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0049040 con 0x55be118a52f0 2024-09-11T22:52:43.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfbdffb640 1 -- 172.21.15.152:0/2556296269 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb00c0530 con 0x55be118a52f0 2024-09-11T22:52:43.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.327+0000 7fcfbdffb640 1 -- 172.21.15.152:0/2556296269 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00c0170 con 0x55be118a52f0 2024-09-11T22:52:43.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.328+0000 7fcfd0b72740 1 -- 172.21.15.152:0/2556296269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a52f0 msgr2=0x55be1189fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:43.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.328+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/2556296269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a52f0 0x55be1189fef0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00096c0 tx=0x7fcfb00ed2f0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.328+0000 7fcfd0b72740 1 -- 172.21.15.152:0/2556296269 shutdown_connections 2024-09-11T22:52:43.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.328+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/2556296269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 0x55be118a99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.328+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/2556296269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a52f0 0x55be1189fef0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.328+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/2556296269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a49d0 0x55be118c16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.328+0000 7fcfd0b72740 1 -- 172.21.15.152:0/2556296269 >> 172.21.15.152:0/2556296269 conn(0x55be118a0f60 msgr2=0x55be1189f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:43.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.328+0000 7fcfd0b72740 1 -- 172.21.15.152:0/2556296269 shutdown_connections 2024-09-11T22:52:43.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.328+0000 7fcfd0b72740 1 -- 172.21.15.152:0/2556296269 wait complete. 2024-09-11T22:52:43.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.328+0000 7fcfd0b72740 1 -- start start 2024-09-11T22:52:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfd0b72740 1 Processor -- start 2024-09-11T22:52:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfd0b72740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a49d0 0x55be118c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfd0b72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a52f0 0x55be118c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfd0b72740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 0x55be118c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfd0b72740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55be118c1560 con 0x55be118a49d0 2024-09-11T22:52:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfd0b72740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55be118c0e90 con 0x55be118afbb0 2024-09-11T22:52:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfd0b72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55be118c1c30 con 0x55be118a52f0 2024-09-11T22:52:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfd0b72740 10 client.0 _ll_register_callbacks cb 0x7fcfb8014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfc7fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a49d0 0x55be118c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfbffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a52f0 0x55be118c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfc7fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a49d0 0x55be118c50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58812/0 (socket says 172.21.15.152:58812) 2024-09-11T22:52:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfc7fff640 1 -- 172.21.15.152:0/1641738247 learned_addr learned my addr 172.21.15.152:0/1641738247 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:43.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfc7fff640 1 -- 172.21.15.152:0/1641738247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a52f0 msgr2=0x55be118c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfc7fff640 1 --2- 172.21.15.152:0/1641738247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a52f0 0x55be118c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfc7fff640 1 -- 172.21.15.152:0/1641738247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 msgr2=0x55be118c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfc7fff640 1 --2- 172.21.15.152:0/1641738247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 0x55be118c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.329+0000 7fcfc7fff640 1 -- 172.21.15.152:0/1641738247 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be118c9e30 con 0x55be118a49d0 2024-09-11T22:52:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.330+0000 7fcfc7fff640 1 --2- 172.21.15.152:0/1641738247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a49d0 0x55be118c50c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80058c0 tx=0x7fcfb8003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92795741eebc60de server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.330+0000 7fcfa77fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.330+0000 7fcfa77fe640 1 -- 172.21.15.152:0/1641738247 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800ca30 con 0x55be118a49d0 2024-09-11T22:52:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.330+0000 7fcfa77fe640 1 -- 172.21.15.152:0/1641738247 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fcfb8020040 con 0x55be118a49d0 2024-09-11T22:52:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.330+0000 7fcfd0b72740 1 -- 172.21.15.152:0/1641738247 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb00c0530 con 0x55be118a49d0 2024-09-11T22:52:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.330+0000 7fcfa77fe640 1 -- 172.21.15.152:0/1641738247 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcfb801abf0 con 0x55be118a49d0 2024-09-11T22:52:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.330+0000 7fcfd0b72740 10 client.15180 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.330+0000 7fcfd0b72740 1 -- 172.21.15.152:0/1641738247 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55be1192a4b0 con 0x55be118a49d0 2024-09-11T22:52:43.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.330+0000 7fcfa77fe640 1 -- 172.21.15.152:0/1641738247 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb00c0530 con 0x55be118a49d0 2024-09-11T22:52:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.330+0000 7fcfa77fe640 1 -- 172.21.15.152:0/1641738247 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fcfb8030c30 con 0x55be118a49d0 2024-09-11T22:52:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.330+0000 7fcfd0b72740 10 client.15180 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.330+0000 7fcfd0b72740 10 client.15180 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T22:52:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.330+0000 7fcfd0b72740 20 client.15180 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.330+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/1641738247 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55be1192efc0 0x55be1192dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.330+0000 7fcfd0b72740 4 client.15180 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:52:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.330+0000 7fcfd0b72740 1 -- 172.21.15.152:0/1641738247 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "status"}) -- 0x55be11812b10 con 0x55be1192efc0 2024-09-11T22:52:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.331+0000 7fcfbffff640 1 --2- 172.21.15.152:0/1641738247 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55be1192efc0 0x55be1192dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.331+0000 7fcfa77fe640 10 client.15180 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:52:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.331+0000 7fcfbffff640 1 --2- 172.21.15.152:0/1641738247 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55be1192efc0 0x55be1192dcd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc3ec124a6fce40b server_cookie=e2265c80807038c1 in_seq=0 out_seq=0 2024-09-11T22:52:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.331+0000 7fcfa77fe640 1 -- 172.21.15.152:0/1641738247 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55be11812b10 con 0x55be1192efc0 2024-09-11T22:52:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.331+0000 7fcfa77fe640 10 client.15180 handle_command_reply: tid=0 2024-09-11T22:52:43.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.331+0000 7fcfd0b72740 1 client.15180 shutdown 2024-09-11T22:52:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.331+0000 7fcfd0b72740 1 -- 172.21.15.152:0/1641738247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a49d0 msgr2=0x55be118c50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.331+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/1641738247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a49d0 0x55be118c50c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80058c0 tx=0x7fcfb8003ee0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.331+0000 7fcfd0b72740 1 -- 172.21.15.152:0/1641738247 shutdown_connections 2024-09-11T22:52:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.331+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/1641738247 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55be1192efc0 0x55be1192dcd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.331+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/1641738247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55be118afbb0 0x55be118c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.331+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/1641738247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55be118a52f0 0x55be118c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.331+0000 7fcfd0b72740 1 --2- 172.21.15.152:0/1641738247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55be118a49d0 0x55be118c50c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.331+0000 7fcfa77fe640 0 client.15180 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:52:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.331+0000 7fcfd0b72740 1 -- 172.21.15.152:0/1641738247 >> 172.21.15.152:0/1641738247 conn(0x55be118a0f60 msgr2=0x55be118a0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.331+0000 7fcfd0b72740 1 -- 172.21.15.152:0/1641738247 shutdown_connections 2024-09-11T22:52:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.332+0000 7fcfd0b72740 1 -- 172.21.15.152:0/1641738247 wait complete. 2024-09-11T22:52:43.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.332+0000 7fcfd0b72740 20 client.15180 trim_cache size 0 max 16384 2024-09-11T22:52:43.341 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:52:43.341 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:52:43.341 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 2, 2024-09-11T22:52:43.341 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 24434, 2024-09-11T22:52:43.341 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-09-11T22:52:43.341 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-09-11T22:52:43.341 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-09-11T22:52:43.341 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 100.119304129, 2024-09-11T22:52:43.341 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 27, 2024-09-11T22:52:43.341 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 83, 2024-09-11T22:52:43.342 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T22:52:43.342 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 132.25017698900001 2024-09-11T22:52:43.342 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:52:43.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.332+0000 7fcfce799640 1 -- 172.21.15.152:0/2649897814 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcfb40a2c00 msgr2=0x7fcfb40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:43.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.332+0000 7fcfce799640 1 --2- 172.21.15.152:0/2649897814 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcfb40a2c00 0x7fcfb40a50c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0005fd0 tx=0x7fcfc0004a40 comp rx=0 tx=0).stop 2024-09-11T22:52:43.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.332+0000 7fcfce799640 1 -- 172.21.15.152:0/2649897814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfc8102570 msgr2=0x7fcfc80fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:43.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.332+0000 7fcfce799640 1 --2- 172.21.15.152:0/2649897814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfc8102570 0x7fcfc80fd1c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0006010 tx=0x7fcfb0003c10 comp rx=0 tx=0).stop 2024-09-11T22:52:43.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.332+0000 7fcfce799640 1 -- 172.21.15.152:0/2649897814 shutdown_connections 2024-09-11T22:52:43.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.332+0000 7fcfce799640 1 --2- 172.21.15.152:0/2649897814 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fcfb40a2c00 0x7fcfb40a50c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.332+0000 7fcfce799640 1 --2- 172.21.15.152:0/2649897814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfc8102ef0 0x7fcfc80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.332+0000 7fcfce799640 1 --2- 172.21.15.152:0/2649897814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcfc8102570 0x7fcfc80fd1c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.332+0000 7fcfce799640 1 --2- 172.21.15.152:0/2649897814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfc8100e20 0x7fcfc80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:43.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.332+0000 7fcfce799640 1 -- 172.21.15.152:0/2649897814 >> 172.21.15.152:0/2649897814 conn(0x7fcfc80fc7d0 msgr2=0x7fcfc806a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:43.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.332+0000 7fcfce799640 1 -- 172.21.15.152:0/2649897814 shutdown_connections 2024-09-11T22:52:43.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:43.332+0000 7fcfce799640 1 -- 172.21.15.152:0/2649897814 wait complete. 2024-09-11T22:52:44.104 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T22:52:44.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:43 smithi191 ceph-mon[35211]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T22:52:44.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:43 smithi204 ceph-mon[35454]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T22:52:44.354 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:44.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:43 smithi152 ceph-mon[31181]: pgmap v337: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-11T22:52:44.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.845+0000 7fa203925640 1 -- 172.21.15.152:0/2705488193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1fc102570 msgr2=0x7fa1fc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:44.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.845+0000 7fa203925640 1 --2- 172.21.15.152:0/2705488193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1fc102570 0x7fa1fc1029b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009900 tx=0x7fa1ec02a480 comp rx=0 tx=0).stop 2024-09-11T22:52:44.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.845+0000 7fa203925640 1 -- 172.21.15.152:0/2705488193 shutdown_connections 2024-09-11T22:52:44.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.845+0000 7fa203925640 1 --2- 172.21.15.152:0/2705488193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1fc102ef0 0x7fa1fc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.845+0000 7fa203925640 1 --2- 172.21.15.152:0/2705488193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1fc102570 0x7fa1fc1029b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.845+0000 7fa203925640 1 --2- 172.21.15.152:0/2705488193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1fc100e20 0x7fa1fc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.845+0000 7fa203925640 1 -- 172.21.15.152:0/2705488193 >> 172.21.15.152:0/2705488193 conn(0x7fa1fc0fc7d0 msgr2=0x7fa1fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:44.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.845+0000 7fa203925640 1 -- 172.21.15.152:0/2705488193 shutdown_connections 2024-09-11T22:52:44.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.845+0000 7fa203925640 1 -- 172.21.15.152:0/2705488193 wait complete. 2024-09-11T22:52:44.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.845+0000 7fa203925640 1 Processor -- start 2024-09-11T22:52:44.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa203925640 1 -- start start 2024-09-11T22:52:44.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa203925640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1fc100e20 0x7fa1fc196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa203925640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1fc102570 0x7fa1fc1968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa203925640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1fc102ef0 0x7fa1fc19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa203925640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc10fe80 con 0x7fa1fc102570 2024-09-11T22:52:44.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa203925640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc10fd00 con 0x7fa1fc102ef0 2024-09-11T22:52:44.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa203925640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc110000 con 0x7fa1fc100e20 2024-09-11T22:52:44.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa200ec0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1fc102570 0x7fa1fc1968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa200ec0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1fc102570 0x7fa1fc1968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58828/0 (socket says 172.21.15.152:58828) 2024-09-11T22:52:44.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa200ec0640 1 -- 172.21.15.152:0/3664640141 learned_addr learned my addr 172.21.15.152:0/3664640141 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:44.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa201ec2640 1 --2- 172.21.15.152:0/3664640141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1fc102ef0 0x7fa1fc19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa2016c1640 1 --2- 172.21.15.152:0/3664640141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1fc100e20 0x7fa1fc196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa200ec0640 1 -- 172.21.15.152:0/3664640141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1fc100e20 msgr2=0x7fa1fc196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:44.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa200ec0640 1 --2- 172.21.15.152:0/3664640141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1fc100e20 0x7fa1fc196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa200ec0640 1 -- 172.21.15.152:0/3664640141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1fc102ef0 msgr2=0x7fa1fc19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:44.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa200ec0640 1 --2- 172.21.15.152:0/3664640141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1fc102ef0 0x7fa1fc19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa200ec0640 1 -- 172.21.15.152:0/3664640141 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1fc19b310 con 0x7fa1fc102570 2024-09-11T22:52:44.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa2016c1640 1 --2- 172.21.15.152:0/3664640141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1fc100e20 0x7fa1fc196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:44.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa200ec0640 1 --2- 172.21.15.152:0/3664640141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1fc102570 0x7fa1fc1968a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec006010 tx=0x7fa1ec003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:44.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.846+0000 7fa1f27fc640 1 -- 172.21.15.152:0/3664640141 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec03f070 con 0x7fa1fc102570 2024-09-11T22:52:44.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.847+0000 7fa203925640 1 -- 172.21.15.152:0/3664640141 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1fc19b5a0 con 0x7fa1fc102570 2024-09-11T22:52:44.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.847+0000 7fa1f27fc640 1 -- 172.21.15.152:0/3664640141 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ec031c30 con 0x7fa1fc102570 2024-09-11T22:52:44.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.847+0000 7fa1f27fc640 1 -- 172.21.15.152:0/3664640141 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec035030 con 0x7fa1fc102570 2024-09-11T22:52:44.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.847+0000 7fa203925640 1 -- 172.21.15.152:0/3664640141 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1fc1a2ee0 con 0x7fa1fc102570 2024-09-11T22:52:44.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.849+0000 7fa1f27fc640 1 -- 172.21.15.152:0/3664640141 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa1ec042070 con 0x7fa1fc102570 2024-09-11T22:52:44.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.849+0000 7fa205cfe740 1 Processor -- start 2024-09-11T22:52:44.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.849+0000 7fa205cfe740 1 -- start start 2024-09-11T22:52:44.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.849+0000 7fa205cfe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbada79d0 0x564bbada7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.849+0000 7fa205cfe740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada82f0 0x564bbadb2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.849+0000 7fa205cfe740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbadb2bb0 0x564bbadb4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.849+0000 7fa205cfe740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x564bbad0e090 con 0x564bbadb2bb0 2024-09-11T22:52:44.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.849+0000 7fa205cfe740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564bbacb5a70 con 0x564bbada82f0 2024-09-11T22:52:44.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.849+0000 7fa205cfe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564bbad0a050 con 0x564bbada79d0 2024-09-11T22:52:44.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa200ec0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbadb2bb0 0x564bbadb4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa200ec0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbadb2bb0 0x564bbadb4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58844/0 (socket says 172.21.15.152:58844) 2024-09-11T22:52:44.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa200ec0640 1 -- 172.21.15.152:0/1431392959 learned_addr learned my addr 172.21.15.152:0/1431392959 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:44.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa2016c1640 1 --2- 172.21.15.152:0/1431392959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada82f0 0x564bbadb2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa201ec2640 1 --2- 172.21.15.152:0/1431392959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbada79d0 0x564bbada7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa1f27fc640 1 --2- 172.21.15.152:0/3664640141 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa1e80a4d90 0x7fa1e80a7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.903 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:52:44.903 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:52:44.903 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 1, 2024-09-11T22:52:44.903 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 24442, 2024-09-11T22:52:44.903 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-09-11T22:52:44.903 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-09-11T22:52:44.904 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-09-11T22:52:44.904 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 102.677348487, 2024-09-11T22:52:44.904 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 27, 2024-09-11T22:52:44.904 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 83, 2024-09-11T22:52:44.904 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T22:52:44.904 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 131.233107136 2024-09-11T22:52:44.904 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:52:44.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa200ec0640 1 -- 172.21.15.152:0/1431392959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbada79d0 msgr2=0x564bbada7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:44.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa200ec0640 1 --2- 172.21.15.152:0/1431392959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbada79d0 0x564bbada7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa200ec0640 1 -- 172.21.15.152:0/1431392959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada82f0 msgr2=0x564bbadb2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:44.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa200ec0640 1 --2- 172.21.15.152:0/1431392959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada82f0 0x564bbadb2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa200ec0640 1 -- 172.21.15.152:0/1431392959 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bbacc2130 con 0x564bbadb2bb0 2024-09-11T22:52:44.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa2016c1640 1 --2- 172.21.15.152:0/1431392959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada82f0 0x564bbadb2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:52:44.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa201ec2640 1 --2- 172.21.15.152:0/3664640141 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa1e80a4d90 0x7fa1e80a7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa1f27fc640 1 -- 172.21.15.152:0/3664640141 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fa1ec0ef720 con 0x7fa1fc102570 2024-09-11T22:52:44.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa200ec0640 1 --2- 172.21.15.152:0/1431392959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbadb2bb0 0x564bbadb4fa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec04a000 tx=0x7fa1ec0b1490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59807114992c705 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:44.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa201ec2640 1 --2- 172.21.15.152:0/3664640141 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa1e80a4d90 0x7fa1e80a7250 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4005fd0 tx=0x7fa1f4014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa1e77fe640 1 -- 172.21.15.152:0/1431392959 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0bb910 con 0x564bbadb2bb0 2024-09-11T22:52:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa1e77fe640 1 -- 172.21.15.152:0/1431392959 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ec0dabf0 con 0x564bbadb2bb0 2024-09-11T22:52:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.850+0000 7fa1e77fe640 1 -- 172.21.15.152:0/1431392959 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0dc370 con 0x564bbadb2bb0 2024-09-11T22:52:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.851+0000 7fa205cfe740 1 -- 172.21.15.152:0/1431392959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbadb2bb0 msgr2=0x564bbadb4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.851+0000 7fa205cfe740 1 --2- 172.21.15.152:0/1431392959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbadb2bb0 0x564bbadb4fa0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec04a000 tx=0x7fa1ec0b1490 comp rx=0 tx=0).stop 2024-09-11T22:52:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.851+0000 7fa205cfe740 1 -- 172.21.15.152:0/1431392959 shutdown_connections 2024-09-11T22:52:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.851+0000 7fa205cfe740 1 --2- 172.21.15.152:0/1431392959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbadb2bb0 0x564bbadb4fa0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.851+0000 7fa205cfe740 1 --2- 172.21.15.152:0/1431392959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada82f0 0x564bbadb2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.851+0000 7fa205cfe740 1 --2- 172.21.15.152:0/1431392959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbada79d0 0x564bbada7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.851+0000 7fa205cfe740 1 -- 172.21.15.152:0/1431392959 >> 172.21.15.152:0/1431392959 conn(0x564bbada3f60 msgr2=0x564bbada4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:44.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.851+0000 7fa205cfe740 1 -- 172.21.15.152:0/1431392959 shutdown_connections 2024-09-11T22:52:44.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.851+0000 7fa205cfe740 1 -- 172.21.15.152:0/1431392959 wait complete. 2024-09-11T22:52:44.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.851+0000 7fa205cfe740 1 -- start start 2024-09-11T22:52:44.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa205cfe740 1 Processor -- start 2024-09-11T22:52:44.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa205cfe740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada79d0 0x564bbadb23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa205cfe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbada82f0 0x564bbadaf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa205cfe740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbadb2bb0 0x564bbadafc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa205cfe740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x564bbadb5540 con 0x564bbadb2bb0 2024-09-11T22:52:44.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa205cfe740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564bbacc8f20 con 0x564bbada79d0 2024-09-11T22:52:44.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa205cfe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564bbadb56c0 con 0x564bbada82f0 2024-09-11T22:52:44.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa205cfe740 10 client.0 _ll_register_callbacks cb 0x564bbac92030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:44.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa201ec2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbadb2bb0 0x564bbadafc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa201ec2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbadb2bb0 0x564bbadafc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58856/0 (socket says 172.21.15.152:58856) 2024-09-11T22:52:44.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa201ec2640 1 -- 172.21.15.152:0/370331598 learned_addr learned my addr 172.21.15.152:0/370331598 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:44.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa2016c1640 1 --2- 172.21.15.152:0/370331598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada79d0 0x564bbadb23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa200ec0640 1 --2- 172.21.15.152:0/370331598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbada82f0 0x564bbadaf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa201ec2640 1 -- 172.21.15.152:0/370331598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbada82f0 msgr2=0x564bbadaf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:44.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa201ec2640 1 --2- 172.21.15.152:0/370331598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbada82f0 0x564bbadaf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa201ec2640 1 -- 172.21.15.152:0/370331598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada79d0 msgr2=0x564bbadb23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:44.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa201ec2640 1 --2- 172.21.15.152:0/370331598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada79d0 0x564bbadb23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa201ec2640 1 -- 172.21.15.152:0/370331598 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bbaa7f2b0 con 0x564bbadb2bb0 2024-09-11T22:52:44.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.852+0000 7fa2016c1640 1 --2- 172.21.15.152:0/370331598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada79d0 0x564bbadb23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:44.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.853+0000 7fa201ec2640 1 --2- 172.21.15.152:0/370331598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbadb2bb0 0x564bbadafc60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40108c0 tx=0x7fa1f4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff867659ab721c42 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:44.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.853+0000 7fa1e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:44.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.853+0000 7fa1e4ff9640 1 -- 172.21.15.152:0/370331598 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f401c890 con 0x564bbadb2bb0 2024-09-11T22:52:44.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.853+0000 7fa1e4ff9640 1 -- 172.21.15.152:0/370331598 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fa1f401ca30 con 0x564bbadb2bb0 2024-09-11T22:52:44.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.853+0000 7fa205cfe740 1 -- 172.21.15.152:0/370331598 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1ec0dabf0 con 0x564bbadb2bb0 2024-09-11T22:52:44.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.853+0000 7fa1e4ff9640 1 -- 172.21.15.152:0/370331598 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa1f402e070 con 0x564bbadb2bb0 2024-09-11T22:52:44.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.853+0000 7fa205cfe740 10 client.15192 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:44.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.853+0000 7fa205cfe740 1 -- 172.21.15.152:0/370331598 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564bbadc0e10 con 0x564bbadb2bb0 2024-09-11T22:52:44.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.853+0000 7fa1e4ff9640 1 -- 172.21.15.152:0/370331598 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ec0dabf0 con 0x564bbadb2bb0 2024-09-11T22:52:44.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.853+0000 7fa1e4ff9640 1 -- 172.21.15.152:0/370331598 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa1f4025020 con 0x564bbadb2bb0 2024-09-11T22:52:44.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.853+0000 7fa205cfe740 10 client.15192 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:44.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.853+0000 7fa205cfe740 10 client.15192 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T22:52:44.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.853+0000 7fa205cfe740 20 client.15192 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:44.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.853+0000 7fa205cfe740 1 --2- 172.21.15.152:0/370331598 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x564bbadc3450 0x564bbadc48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.853+0000 7fa205cfe740 4 client.15192 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:52:44.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.853+0000 7fa205cfe740 1 -- 172.21.15.152:0/370331598 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bbad1b600 con 0x564bbadc3450 2024-09-11T22:52:44.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.854+0000 7fa2016c1640 1 --2- 172.21.15.152:0/370331598 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x564bbadc3450 0x564bbadc48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.854+0000 7fa1e4ff9640 10 client.15192 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:52:44.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.854+0000 7fa2016c1640 1 --2- 172.21.15.152:0/370331598 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x564bbadc3450 0x564bbadc48c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=68d94586fd63e1d1 server_cookie=43c1d3e445763801 in_seq=0 out_seq=0 2024-09-11T22:52:44.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.855+0000 7fa1e4ff9640 1 -- 172.21.15.152:0/370331598 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564bbad1b600 con 0x564bbadc3450 2024-09-11T22:52:44.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.855+0000 7fa1e4ff9640 10 client.15192 handle_command_reply: tid=0 2024-09-11T22:52:44.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.855+0000 7fa205cfe740 1 client.15192 shutdown 2024-09-11T22:52:44.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.855+0000 7fa205cfe740 1 -- 172.21.15.152:0/370331598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbadb2bb0 msgr2=0x564bbadafc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:44.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.855+0000 7fa205cfe740 1 --2- 172.21.15.152:0/370331598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbadb2bb0 0x564bbadafc60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40108c0 tx=0x7fa1f4009500 comp rx=0 tx=0).stop 2024-09-11T22:52:44.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.855+0000 7fa205cfe740 1 -- 172.21.15.152:0/370331598 shutdown_connections 2024-09-11T22:52:44.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.855+0000 7fa205cfe740 1 --2- 172.21.15.152:0/370331598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbadb2bb0 0x564bbadafc60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.855+0000 7fa205cfe740 1 --2- 172.21.15.152:0/370331598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbada82f0 0x564bbadaf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.855+0000 7fa205cfe740 1 --2- 172.21.15.152:0/370331598 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x564bbadc3450 0x564bbadc48c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.855+0000 7fa205cfe740 1 --2- 172.21.15.152:0/370331598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada79d0 0x564bbadb23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.855+0000 7fa1e4ff9640 0 client.15192 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:52:44.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.855+0000 7fa205cfe740 1 -- 172.21.15.152:0/370331598 >> 172.21.15.152:0/370331598 conn(0x564bbada3f60 msgr2=0x564bbada9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:44.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.855+0000 7fa205cfe740 1 -- 172.21.15.152:0/370331598 shutdown_connections 2024-09-11T22:52:44.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.855+0000 7fa205cfe740 1 -- 172.21.15.152:0/370331598 wait complete. 2024-09-11T22:52:44.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.855+0000 7fa205cfe740 20 client.15192 trim_cache size 0 max 16384 2024-09-11T22:52:44.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa205cfe740 1 Processor -- start 2024-09-11T22:52:44.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa205cfe740 1 -- start start 2024-09-11T22:52:44.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa205cfe740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbada79d0 0x564bbadbe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa205cfe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbada82f0 0x564bbadccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa205cfe740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbadb2bb0 0x564bbadadff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa205cfe740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x564bbadc2c00 con 0x564bbada79d0 2024-09-11T22:52:44.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa205cfe740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564bbadc32d0 con 0x564bbadb2bb0 2024-09-11T22:52:44.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa205cfe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564bbadc39a0 con 0x564bbada82f0 2024-09-11T22:52:44.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa2016c1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbada79d0 0x564bbadbe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa2016c1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbada79d0 0x564bbadbe590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58862/0 (socket says 172.21.15.152:58862) 2024-09-11T22:52:44.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa2016c1640 1 -- 172.21.15.152:0/3713123412 learned_addr learned my addr 172.21.15.152:0/3713123412 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:44.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa2016c1640 1 -- 172.21.15.152:0/3713123412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbada82f0 msgr2=0x564bbadccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:44.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa2016c1640 1 --2- 172.21.15.152:0/3713123412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbada82f0 0x564bbadccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa201ec2640 1 --2- 172.21.15.152:0/3713123412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbadb2bb0 0x564bbadadff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa2016c1640 1 -- 172.21.15.152:0/3713123412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbadb2bb0 msgr2=0x564bbadadff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:44.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa2016c1640 1 --2- 172.21.15.152:0/3713123412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbadb2bb0 0x564bbadadff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa2016c1640 1 -- 172.21.15.152:0/3713123412 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bbada50f0 con 0x564bbada79d0 2024-09-11T22:52:44.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.873+0000 7fa2016c1640 1 --2- 172.21.15.152:0/3713123412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbada79d0 0x564bbadbe590 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x564bbadc53d0 tx=0x7fa1f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5731eb88bff690a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:44.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.874+0000 7fa1e6ffd640 1 -- 172.21.15.152:0/3713123412 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8011aa0 con 0x564bbada79d0 2024-09-11T22:52:44.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.874+0000 7fa1e6ffd640 1 -- 172.21.15.152:0/3713123412 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f8005500 con 0x564bbada79d0 2024-09-11T22:52:44.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.874+0000 7fa1e6ffd640 1 -- 172.21.15.152:0/3713123412 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f801e420 con 0x564bbada79d0 2024-09-11T22:52:44.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.874+0000 7fa205cfe740 1 -- 172.21.15.152:0/3713123412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbada79d0 msgr2=0x564bbadbe590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:44.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.874+0000 7fa205cfe740 1 --2- 172.21.15.152:0/3713123412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbada79d0 0x564bbadbe590 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x564bbadc53d0 tx=0x7fa1f80046c0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.874+0000 7fa205cfe740 1 -- 172.21.15.152:0/3713123412 shutdown_connections 2024-09-11T22:52:44.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.874+0000 7fa205cfe740 1 --2- 172.21.15.152:0/3713123412 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbadb2bb0 0x564bbadadff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.874+0000 7fa205cfe740 1 --2- 172.21.15.152:0/3713123412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbada82f0 0x564bbadccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.874+0000 7fa205cfe740 1 --2- 172.21.15.152:0/3713123412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbada79d0 0x564bbadbe590 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.874+0000 7fa205cfe740 1 -- 172.21.15.152:0/3713123412 >> 172.21.15.152:0/3713123412 conn(0x564bbada3f60 msgr2=0x564bbadaec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:44.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.874+0000 7fa205cfe740 1 -- 172.21.15.152:0/3713123412 shutdown_connections 2024-09-11T22:52:44.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.874+0000 7fa205cfe740 1 -- 172.21.15.152:0/3713123412 wait complete. 2024-09-11T22:52:44.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.875+0000 7fa205cfe740 1 -- start start 2024-09-11T22:52:44.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.875+0000 7fa205cfe740 1 Processor -- start 2024-09-11T22:52:44.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.875+0000 7fa205cfe740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbada79d0 0x564bbadca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.875+0000 7fa205cfe740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada82f0 0x564bbadcade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.875+0000 7fa205cfe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbadb2bb0 0x564bbadcb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.875+0000 7fa205cfe740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x564bbadc4740 con 0x564bbada79d0 2024-09-11T22:52:44.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.875+0000 7fa205cfe740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564bbadc4070 con 0x564bbada82f0 2024-09-11T22:52:44.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.875+0000 7fa205cfe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564bbadc4e10 con 0x564bbadb2bb0 2024-09-11T22:52:44.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.875+0000 7fa205cfe740 10 client.0 _ll_register_callbacks cb 0x7fa1f4009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:44.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.875+0000 7fa2016c1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbada79d0 0x564bbadca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.875+0000 7fa2016c1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbada79d0 0x564bbadca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58866/0 (socket says 172.21.15.152:58866) 2024-09-11T22:52:44.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.875+0000 7fa2016c1640 1 -- 172.21.15.152:0/1069889931 learned_addr learned my addr 172.21.15.152:0/1069889931 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:44.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.875+0000 7fa200ec0640 1 --2- 172.21.15.152:0/1069889931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada82f0 0x564bbadcade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.876+0000 7fa2016c1640 1 -- 172.21.15.152:0/1069889931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbadb2bb0 msgr2=0x564bbadcb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:44.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.876+0000 7fa201ec2640 1 --2- 172.21.15.152:0/1069889931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbadb2bb0 0x564bbadcb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.876+0000 7fa2016c1640 1 --2- 172.21.15.152:0/1069889931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbadb2bb0 0x564bbadcb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.876+0000 7fa2016c1640 1 -- 172.21.15.152:0/1069889931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada82f0 msgr2=0x564bbadcade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:44.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.876+0000 7fa2016c1640 1 --2- 172.21.15.152:0/1069889931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada82f0 0x564bbadcade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.876+0000 7fa2016c1640 1 -- 172.21.15.152:0/1069889931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bbae30b90 con 0x564bbada79d0 2024-09-11T22:52:44.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.876+0000 7fa2016c1640 1 --2- 172.21.15.152:0/1069889931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbada79d0 0x564bbadca8a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80124d0 tx=0x7fa1f8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ededf9778d76093c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:44.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.876+0000 7fa1e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:44.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.876+0000 7fa1e4ff9640 1 -- 172.21.15.152:0/1069889931 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8011aa0 con 0x564bbada79d0 2024-09-11T22:52:44.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.876+0000 7fa205cfe740 1 -- 172.21.15.152:0/1069889931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1f8005500 con 0x564bbada79d0 2024-09-11T22:52:44.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.876+0000 7fa1e4ff9640 1 -- 172.21.15.152:0/1069889931 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7fa1f8011c40 con 0x564bbada79d0 2024-09-11T22:52:44.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.876+0000 7fa1e4ff9640 1 -- 172.21.15.152:0/1069889931 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa1f8011e40 con 0x564bbada79d0 2024-09-11T22:52:44.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.876+0000 7fa205cfe740 10 client.15198 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:44.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.876+0000 7fa205cfe740 1 -- 172.21.15.152:0/1069889931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564bbae31e10 con 0x564bbada79d0 2024-09-11T22:52:44.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.876+0000 7fa1e4ff9640 1 -- 172.21.15.152:0/1069889931 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f8005500 con 0x564bbada79d0 2024-09-11T22:52:44.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.877+0000 7fa1e4ff9640 1 -- 172.21.15.152:0/1069889931 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa1f8025030 con 0x564bbada79d0 2024-09-11T22:52:44.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.877+0000 7fa205cfe740 10 client.15198 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:44.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.877+0000 7fa205cfe740 10 client.15198 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T22:52:44.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.877+0000 7fa205cfe740 20 client.15198 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:44.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.877+0000 7fa205cfe740 1 --2- 172.21.15.152:0/1069889931 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x564bbae366a0 0x564bbae353b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:44.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.877+0000 7fa205cfe740 4 client.15198 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:52:44.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.877+0000 7fa205cfe740 1 -- 172.21.15.152:0/1069889931 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "status"}) -- 0x564bbad15b10 con 0x564bbae366a0 2024-09-11T22:52:44.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.877+0000 7fa200ec0640 1 --2- 172.21.15.152:0/1069889931 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x564bbae366a0 0x564bbae353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:44.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.877+0000 7fa1e4ff9640 10 client.15198 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:52:44.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.877+0000 7fa200ec0640 1 --2- 172.21.15.152:0/1069889931 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x564bbae366a0 0x564bbae353b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4111d2d9ee0d88b6 server_cookie=2064eb55019a30c in_seq=0 out_seq=0 2024-09-11T22:52:44.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.877+0000 7fa1e4ff9640 1 -- 172.21.15.152:0/1069889931 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x564bbad15b10 con 0x564bbae366a0 2024-09-11T22:52:44.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.877+0000 7fa1e4ff9640 10 client.15198 handle_command_reply: tid=0 2024-09-11T22:52:44.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.877+0000 7fa205cfe740 1 client.15198 shutdown 2024-09-11T22:52:44.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa205cfe740 1 -- 172.21.15.152:0/1069889931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbada79d0 msgr2=0x564bbadca8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:44.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa205cfe740 1 --2- 172.21.15.152:0/1069889931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbada79d0 0x564bbadca8a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80124d0 tx=0x7fa1f8005fd0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa205cfe740 1 -- 172.21.15.152:0/1069889931 shutdown_connections 2024-09-11T22:52:44.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa205cfe740 1 --2- 172.21.15.152:0/1069889931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564bbadb2bb0 0x564bbadcb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa205cfe740 1 --2- 172.21.15.152:0/1069889931 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x564bbae366a0 0x564bbae353b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa205cfe740 1 --2- 172.21.15.152:0/1069889931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564bbada82f0 0x564bbadcade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa205cfe740 1 --2- 172.21.15.152:0/1069889931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564bbada79d0 0x564bbadca8a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa205cfe740 1 -- 172.21.15.152:0/1069889931 >> 172.21.15.152:0/1069889931 conn(0x564bbada3f60 msgr2=0x564bbadaa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:44.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa1e4ff9640 0 client.15198 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:52:44.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa205cfe740 1 -- 172.21.15.152:0/1069889931 shutdown_connections 2024-09-11T22:52:44.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa205cfe740 1 -- 172.21.15.152:0/1069889931 wait complete. 2024-09-11T22:52:44.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa205cfe740 20 client.15198 trim_cache size 0 max 16384 2024-09-11T22:52:44.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa203925640 1 -- 172.21.15.152:0/3664640141 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa1e80a4d90 msgr2=0x7fa1e80a7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:44.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa203925640 1 --2- 172.21.15.152:0/3664640141 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa1e80a4d90 0x7fa1e80a7250 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4005fd0 tx=0x7fa1f4014040 comp rx=0 tx=0).stop 2024-09-11T22:52:44.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa203925640 1 -- 172.21.15.152:0/3664640141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1fc102570 msgr2=0x7fa1fc1968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:44.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa203925640 1 --2- 172.21.15.152:0/3664640141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1fc102570 0x7fa1fc1968a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec006010 tx=0x7fa1ec003c10 comp rx=0 tx=0).stop 2024-09-11T22:52:44.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa203925640 1 -- 172.21.15.152:0/3664640141 shutdown_connections 2024-09-11T22:52:44.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa203925640 1 --2- 172.21.15.152:0/3664640141 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7fa1e80a4d90 0x7fa1e80a7250 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa203925640 1 --2- 172.21.15.152:0/3664640141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1fc102ef0 0x7fa1fc19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa203925640 1 --2- 172.21.15.152:0/3664640141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa1fc102570 0x7fa1fc1968a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa203925640 1 --2- 172.21.15.152:0/3664640141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1fc100e20 0x7fa1fc196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:44.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa203925640 1 -- 172.21.15.152:0/3664640141 >> 172.21.15.152:0/3664640141 conn(0x7fa1fc0fc7d0 msgr2=0x7fa1fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:44.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa203925640 1 -- 172.21.15.152:0/3664640141 shutdown_connections 2024-09-11T22:52:44.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:44.878+0000 7fa203925640 1 -- 172.21.15.152:0/3664640141 wait complete. 2024-09-11T22:52:45.824 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T22:52:46.069 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:46.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:45 smithi152 ceph-mon[31181]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:52:46.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:45 smithi191 ceph-mon[35211]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:52:46.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:45 smithi204 ceph-mon[35454]: pgmap v338: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:52:46.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.564+0000 7f35f552e640 1 -- 172.21.15.152:0/1828371253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35f0101de0 msgr2=0x7f35f01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:46.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.564+0000 7f35f552e640 1 --2- 172.21.15.152:0/1828371253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35f0101de0 0x7f35f01021c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009ca0 tx=0x7f35e002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.564+0000 7f35f552e640 1 -- 172.21.15.152:0/1828371253 shutdown_connections 2024-09-11T22:52:46.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.564+0000 7f35f552e640 1 --2- 172.21.15.152:0/1828371253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35f0106f10 0x7f35f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.564+0000 7f35f552e640 1 --2- 172.21.15.152:0/1828371253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35f0102790 0x7f35f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.564+0000 7f35f552e640 1 --2- 172.21.15.152:0/1828371253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35f0101de0 0x7f35f01021c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.564+0000 7f35f552e640 1 -- 172.21.15.152:0/1828371253 >> 172.21.15.152:0/1828371253 conn(0x7f35f00fd8c0 msgr2=0x7f35f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:46.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.564+0000 7f35f552e640 1 -- 172.21.15.152:0/1828371253 shutdown_connections 2024-09-11T22:52:46.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.564+0000 7f35f552e640 1 -- 172.21.15.152:0/1828371253 wait complete. 2024-09-11T22:52:46.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.565+0000 7f35f552e640 1 Processor -- start 2024-09-11T22:52:46.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.565+0000 7f35f552e640 1 -- start start 2024-09-11T22:52:46.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.565+0000 7f35f552e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35f0101de0 0x7f35f01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.565+0000 7f35f552e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35f0102790 0x7f35f0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.565+0000 7f35f552e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35f0106f10 0x7f35f019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.565+0000 7f35f552e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0112010 con 0x7f35f0101de0 2024-09-11T22:52:46.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.565+0000 7f35f552e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0111e90 con 0x7f35f0102790 2024-09-11T22:52:46.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.565+0000 7f35f552e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0112190 con 0x7f35f0106f10 2024-09-11T22:52:46.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.565+0000 7f35eeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35f0101de0 0x7f35f01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:46.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.565+0000 7f35eeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35f0101de0 0x7f35f01985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58888/0 (socket says 172.21.15.152:58888) 2024-09-11T22:52:46.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.565+0000 7f35eeffd640 1 -- 172.21.15.152:0/3110384490 learned_addr learned my addr 172.21.15.152:0/3110384490 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:46.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.566+0000 7f35ef7fe640 1 --2- 172.21.15.152:0/3110384490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35f0106f10 0x7f35f019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:46.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.566+0000 7f35ee7fc640 1 --2- 172.21.15.152:0/3110384490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35f0102790 0x7f35f0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:46.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.566+0000 7f35eeffd640 1 -- 172.21.15.152:0/3110384490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35f0106f10 msgr2=0x7f35f019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:46.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.566+0000 7f35eeffd640 1 --2- 172.21.15.152:0/3110384490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35f0106f10 0x7f35f019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.566+0000 7f35eeffd640 1 -- 172.21.15.152:0/3110384490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35f0102790 msgr2=0x7f35f0198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:46.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.566+0000 7f35eeffd640 1 --2- 172.21.15.152:0/3110384490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35f0102790 0x7f35f0198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.566+0000 7f35eeffd640 1 -- 172.21.15.152:0/3110384490 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f019d560 con 0x7f35f0101de0 2024-09-11T22:52:46.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.566+0000 7f35ee7fc640 1 --2- 172.21.15.152:0/3110384490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35f0102790 0x7f35f0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:46.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.566+0000 7f35eeffd640 1 --2- 172.21.15.152:0/3110384490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35f0101de0 0x7f35f01985b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f35e00097b0 tx=0x7f35e0031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:46.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.566+0000 7f35d7fff640 1 -- 172.21.15.152:0/3110384490 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e002fba0 con 0x7f35f0101de0 2024-09-11T22:52:46.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.566+0000 7f35d7fff640 1 -- 172.21.15.152:0/3110384490 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e002fd40 con 0x7f35f0101de0 2024-09-11T22:52:46.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.566+0000 7f35d7fff640 1 -- 172.21.15.152:0/3110384490 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e003b930 con 0x7f35f0101de0 2024-09-11T22:52:46.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.566+0000 7f35f552e640 1 -- 172.21.15.152:0/3110384490 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f019d7f0 con 0x7f35f0101de0 2024-09-11T22:52:46.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.566+0000 7f35f552e640 1 -- 172.21.15.152:0/3110384490 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f01a5130 con 0x7f35f0101de0 2024-09-11T22:52:46.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.568+0000 7f35d7fff640 1 -- 172.21.15.152:0/3110384490 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f35e0030cc0 con 0x7f35f0101de0 2024-09-11T22:52:46.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.569+0000 7f35f7907740 1 Processor -- start 2024-09-11T22:52:46.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.569+0000 7f35f7907740 1 -- start start 2024-09-11T22:52:46.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.569+0000 7f35f7907740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b49d0 0x55bcb53b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.569+0000 7f35f7907740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53b52f0 0x55bcb53bf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.569+0000 7f35f7907740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53bfbb0 0x55bcb53c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.569+0000 7f35f7907740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55bcb531b090 con 0x55bcb53b52f0 2024-09-11T22:52:46.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.569+0000 7f35f7907740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bcb52c2a70 con 0x55bcb53b49d0 2024-09-11T22:52:46.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.569+0000 7f35f7907740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bcb5317050 con 0x55bcb53bfbb0 2024-09-11T22:52:46.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.569+0000 7f35eeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53b52f0 0x55bcb53bf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:46.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.569+0000 7f35eeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53b52f0 0x55bcb53bf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58900/0 (socket says 172.21.15.152:58900) 2024-09-11T22:52:46.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.569+0000 7f35eeffd640 1 -- 172.21.15.152:0/1753540091 learned_addr learned my addr 172.21.15.152:0/1753540091 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:46.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.569+0000 7f35d7fff640 1 --2- 172.21.15.152:0/3110384490 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f35d80a2b80 0x7f35d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.569+0000 7f35ef7fe640 1 --2- 172.21.15.152:0/1753540091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b49d0 0x55bcb53b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:46.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.569+0000 7f35ee7fc640 1 --2- 172.21.15.152:0/1753540091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53bfbb0 0x55bcb53c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:46.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.570+0000 7f35ef7fe640 1 --2- 172.21.15.152:0/3110384490 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f35d80a2b80 0x7f35d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:46.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.570+0000 7f35d7fff640 1 -- 172.21.15.152:0/3110384490 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f35e00f0610 con 0x7f35f0101de0 2024-09-11T22:52:46.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.570+0000 7f35eeffd640 1 -- 172.21.15.152:0/1753540091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53bfbb0 msgr2=0x55bcb53c1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:46.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.570+0000 7f35eeffd640 1 --2- 172.21.15.152:0/1753540091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53bfbb0 0x55bcb53c1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.570+0000 7f35eeffd640 1 -- 172.21.15.152:0/1753540091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b49d0 msgr2=0x55bcb53b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:46.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.570+0000 7f35eeffd640 1 --2- 172.21.15.152:0/1753540091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b49d0 0x55bcb53b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.570+0000 7f35eeffd640 1 -- 172.21.15.152:0/1753540091 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcb52cf130 con 0x55bcb53b52f0 2024-09-11T22:52:46.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.570+0000 7f35eeffd640 1 --2- 172.21.15.152:0/1753540091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53b52f0 0x55bcb53bf670 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f35e008b710 tx=0x7f35e002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b207d9f17ae6317 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:46.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.570+0000 7f35ef7fe640 1 --2- 172.21.15.152:0/3110384490 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f35d80a2b80 0x7f35d80a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f35e8006aa0 tx=0x7f35e8006da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:46.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.570+0000 7f35d57fa640 1 -- 172.21.15.152:0/1753540091 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0043330 con 0x55bcb53b52f0 2024-09-11T22:52:46.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.570+0000 7f35d57fa640 1 -- 172.21.15.152:0/1753540091 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e0036020 con 0x55bcb53b52f0 2024-09-11T22:52:46.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.570+0000 7f35d57fa640 1 -- 172.21.15.152:0/1753540091 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e00ee2c0 con 0x55bcb53b52f0 2024-09-11T22:52:46.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.571+0000 7f35f7907740 1 -- 172.21.15.152:0/1753540091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53b52f0 msgr2=0x55bcb53bf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:46.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.571+0000 7f35f7907740 1 --2- 172.21.15.152:0/1753540091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53b52f0 0x55bcb53bf670 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f35e008b710 tx=0x7f35e002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.571+0000 7f35f7907740 1 -- 172.21.15.152:0/1753540091 shutdown_connections 2024-09-11T22:52:46.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.571+0000 7f35f7907740 1 --2- 172.21.15.152:0/1753540091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53bfbb0 0x55bcb53c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.571+0000 7f35f7907740 1 --2- 172.21.15.152:0/1753540091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53b52f0 0x55bcb53bf670 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.571+0000 7f35f7907740 1 --2- 172.21.15.152:0/1753540091 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b49d0 0x55bcb53b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.571+0000 7f35f7907740 1 -- 172.21.15.152:0/1753540091 >> 172.21.15.152:0/1753540091 conn(0x55bcb53b0f60 msgr2=0x55bcb53b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:46.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.571+0000 7f35f7907740 1 -- 172.21.15.152:0/1753540091 shutdown_connections 2024-09-11T22:52:46.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.571+0000 7f35f7907740 1 -- 172.21.15.152:0/1753540091 wait complete. 2024-09-11T22:52:46.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.571+0000 7f35f7907740 1 -- start start 2024-09-11T22:52:46.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.571+0000 7f35f7907740 1 Processor -- start 2024-09-11T22:52:46.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35f7907740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53b49d0 0x55bcb53bf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35f7907740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b52f0 0x55bcb53bc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35f7907740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53bfbb0 0x55bcb53bcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35f7907740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55bcb531b090 con 0x55bcb53bfbb0 2024-09-11T22:52:46.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35f7907740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bcb52d5f20 con 0x55bcb53b52f0 2024-09-11T22:52:46.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35f7907740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bcb53c2540 con 0x55bcb53b49d0 2024-09-11T22:52:46.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35f7907740 10 client.0 _ll_register_callbacks cb 0x55bcb529f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:46.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35ef7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53bfbb0 0x55bcb53bcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:46.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35ef7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53bfbb0 0x55bcb53bcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58906/0 (socket says 172.21.15.152:58906) 2024-09-11T22:52:46.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35ef7fe640 1 -- 172.21.15.152:0/2953331425 learned_addr learned my addr 172.21.15.152:0/2953331425 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:46.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35eeffd640 1 --2- 172.21.15.152:0/2953331425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53b49d0 0x55bcb53bf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:46.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35ee7fc640 1 --2- 172.21.15.152:0/2953331425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b52f0 0x55bcb53bc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:46.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35ef7fe640 1 -- 172.21.15.152:0/2953331425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53b49d0 msgr2=0x55bcb53bf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:46.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35ef7fe640 1 --2- 172.21.15.152:0/2953331425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53b49d0 0x55bcb53bf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35ef7fe640 1 -- 172.21.15.152:0/2953331425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b52f0 msgr2=0x55bcb53bc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:46.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35ef7fe640 1 --2- 172.21.15.152:0/2953331425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b52f0 0x55bcb53bc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35ef7fe640 1 -- 172.21.15.152:0/2953331425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcb508c2b0 con 0x55bcb53bfbb0 2024-09-11T22:52:46.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35ef7fe640 1 --2- 172.21.15.152:0/2953331425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53bfbb0 0x55bcb53bcc10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f35e8017010 tx=0x7f35e80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7172ba6280c4f29 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:46.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:46.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35d2ff5640 1 -- 172.21.15.152:0/2953331425 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8014b20 con 0x55bcb53bfbb0 2024-09-11T22:52:46.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35d2ff5640 1 -- 172.21.15.152:0/2953331425 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f35e8014cc0 con 0x55bcb53bfbb0 2024-09-11T22:52:46.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.572+0000 7f35f7907740 1 -- 172.21.15.152:0/2953331425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e0036020 con 0x55bcb53bfbb0 2024-09-11T22:52:46.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.573+0000 7f35d2ff5640 1 -- 172.21.15.152:0/2953331425 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f35e802e460 con 0x55bcb53bfbb0 2024-09-11T22:52:46.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.573+0000 7f35f7907740 10 client.15210 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:46.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.573+0000 7f35f7907740 1 -- 172.21.15.152:0/2953331425 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bcb53cdb60 con 0x55bcb53bfbb0 2024-09-11T22:52:46.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.573+0000 7f35d2ff5640 1 -- 172.21.15.152:0/2953331425 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e0036020 con 0x55bcb53bfbb0 2024-09-11T22:52:46.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.573+0000 7f35d2ff5640 1 -- 172.21.15.152:0/2953331425 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f35e8015040 con 0x55bcb53bfbb0 2024-09-11T22:52:46.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.573+0000 7f35f7907740 10 client.15210 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:46.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.573+0000 7f35f7907740 10 client.15210 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:52:46.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.573+0000 7f35f7907740 20 client.15210 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:46.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.573+0000 7f35f7907740 1 --2- 172.21.15.152:0/2953331425 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55bcb53d0280 0x55bcb53d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.573+0000 7f35f7907740 4 client.15210 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:52:46.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.573+0000 7f35f7907740 1 -- 172.21.15.152:0/2953331425 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcb5328600 con 0x55bcb53d0280 2024-09-11T22:52:46.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.573+0000 7f35eeffd640 1 --2- 172.21.15.152:0/2953331425 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55bcb53d0280 0x55bcb53d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:46.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.574+0000 7f35d2ff5640 10 client.15210 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:52:46.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.574+0000 7f35eeffd640 1 --2- 172.21.15.152:0/2953331425 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55bcb53d0280 0x55bcb53d16f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9b854798286ed4 server_cookie=16efd223d9db8396 in_seq=0 out_seq=0 2024-09-11T22:52:46.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.574+0000 7f35d2ff5640 1 -- 172.21.15.152:0/2953331425 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bcb5328600 con 0x55bcb53d0280 2024-09-11T22:52:46.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.574+0000 7f35d2ff5640 10 client.15210 handle_command_reply: tid=0 2024-09-11T22:52:46.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.574+0000 7f35f7907740 1 client.15210 shutdown 2024-09-11T22:52:46.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.575+0000 7f35f7907740 1 -- 172.21.15.152:0/2953331425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53bfbb0 msgr2=0x55bcb53bcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:46.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.575+0000 7f35f7907740 1 --2- 172.21.15.152:0/2953331425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53bfbb0 0x55bcb53bcc10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f35e8017010 tx=0x7f35e80047d0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.575+0000 7f35f7907740 1 -- 172.21.15.152:0/2953331425 shutdown_connections 2024-09-11T22:52:46.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.575+0000 7f35f7907740 1 --2- 172.21.15.152:0/2953331425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53bfbb0 0x55bcb53bcc10 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.575+0000 7f35f7907740 1 --2- 172.21.15.152:0/2953331425 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55bcb53d0280 0x55bcb53d16f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.575+0000 7f35d2ff5640 0 client.15210 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:52:46.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.575+0000 7f35f7907740 1 --2- 172.21.15.152:0/2953331425 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b52f0 0x55bcb53bc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.575+0000 7f35f7907740 1 --2- 172.21.15.152:0/2953331425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53b49d0 0x55bcb53bf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.575+0000 7f35f7907740 1 -- 172.21.15.152:0/2953331425 >> 172.21.15.152:0/2953331425 conn(0x55bcb53b0f60 msgr2=0x55bcb53b6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:46.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.575+0000 7f35f7907740 1 -- 172.21.15.152:0/2953331425 shutdown_connections 2024-09-11T22:52:46.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.575+0000 7f35f7907740 1 -- 172.21.15.152:0/2953331425 wait complete. 2024-09-11T22:52:46.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.575+0000 7f35f7907740 20 client.15210 trim_cache size 0 max 16384 2024-09-11T22:52:46.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.592+0000 7f35f7907740 1 Processor -- start 2024-09-11T22:52:46.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.592+0000 7f35f7907740 1 -- start start 2024-09-11T22:52:46.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35f7907740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53b49d0 0x55bcb53d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35f7907740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b52f0 0x55bcb53bc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35f7907740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53bfbb0 0x55bcb53bcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35f7907740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55bcb53cfa30 con 0x55bcb53bfbb0 2024-09-11T22:52:46.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35f7907740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bcb53d0100 con 0x55bcb53b52f0 2024-09-11T22:52:46.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35f7907740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bcb53d07d0 con 0x55bcb53b49d0 2024-09-11T22:52:46.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35ef7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53bfbb0 0x55bcb53bcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:46.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35ef7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53bfbb0 0x55bcb53bcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58920/0 (socket says 172.21.15.152:58920) 2024-09-11T22:52:46.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35ef7fe640 1 -- 172.21.15.152:0/767440001 learned_addr learned my addr 172.21.15.152:0/767440001 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:46.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35ef7fe640 1 -- 172.21.15.152:0/767440001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53b49d0 msgr2=0x55bcb53d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:46.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35ef7fe640 1 --2- 172.21.15.152:0/767440001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53b49d0 0x55bcb53d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35ef7fe640 1 -- 172.21.15.152:0/767440001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b52f0 msgr2=0x55bcb53bc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:46.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35ef7fe640 1 --2- 172.21.15.152:0/767440001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b52f0 0x55bcb53bc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35ef7fe640 1 -- 172.21.15.152:0/767440001 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcb53b0d90 con 0x55bcb53bfbb0 2024-09-11T22:52:46.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35ef7fe640 1 --2- 172.21.15.152:0/767440001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53bfbb0 0x55bcb53bcbd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f35e800dcb0 tx=0x7f35e800d470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe89a99005ac6ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:46.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35d4ff9640 1 -- 172.21.15.152:0/767440001 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8016430 con 0x55bcb53bfbb0 2024-09-11T22:52:46.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35d4ff9640 1 -- 172.21.15.152:0/767440001 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e8027070 con 0x55bcb53bfbb0 2024-09-11T22:52:46.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.593+0000 7f35d4ff9640 1 -- 172.21.15.152:0/767440001 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8014e80 con 0x55bcb53bfbb0 2024-09-11T22:52:46.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.594+0000 7f35f7907740 1 -- 172.21.15.152:0/767440001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53bfbb0 msgr2=0x55bcb53bcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:46.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.594+0000 7f35f7907740 1 --2- 172.21.15.152:0/767440001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53bfbb0 0x55bcb53bcbd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f35e800dcb0 tx=0x7f35e800d470 comp rx=0 tx=0).stop 2024-09-11T22:52:46.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.594+0000 7f35f7907740 1 -- 172.21.15.152:0/767440001 shutdown_connections 2024-09-11T22:52:46.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.594+0000 7f35f7907740 1 --2- 172.21.15.152:0/767440001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53bfbb0 0x55bcb53bcbd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.594+0000 7f35f7907740 1 --2- 172.21.15.152:0/767440001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b52f0 0x55bcb53bc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.594+0000 7f35f7907740 1 --2- 172.21.15.152:0/767440001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53b49d0 0x55bcb53d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.594+0000 7f35f7907740 1 -- 172.21.15.152:0/767440001 >> 172.21.15.152:0/767440001 conn(0x55bcb53b0f60 msgr2=0x55bcb53abce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:46.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.594+0000 7f35f7907740 1 -- 172.21.15.152:0/767440001 shutdown_connections 2024-09-11T22:52:46.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.594+0000 7f35f7907740 1 -- 172.21.15.152:0/767440001 wait complete. 2024-09-11T22:52:46.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.594+0000 7f35f7907740 1 -- start start 2024-09-11T22:52:46.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.594+0000 7f35f7907740 1 Processor -- start 2024-09-11T22:52:46.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35f7907740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b49d0 0x55bcb53b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35f7907740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53b52f0 0x55bcb53b9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35f7907740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53bfbb0 0x55bcb53d8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35f7907740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55bcb53d1570 con 0x55bcb53b52f0 2024-09-11T22:52:46.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35f7907740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bcb53d0ea0 con 0x55bcb53b49d0 2024-09-11T22:52:46.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35f7907740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bcb53d1c40 con 0x55bcb53bfbb0 2024-09-11T22:52:46.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35f7907740 10 client.0 _ll_register_callbacks cb 0x7f35e800dbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:52:46.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35ee7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53b52f0 0x55bcb53b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:46.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35ee7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53b52f0 0x55bcb53b9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58934/0 (socket says 172.21.15.152:58934) 2024-09-11T22:52:46.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35ee7fc640 1 -- 172.21.15.152:0/1524178632 learned_addr learned my addr 172.21.15.152:0/1524178632 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:46.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35ef7fe640 1 --2- 172.21.15.152:0/1524178632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53bfbb0 0x55bcb53d8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:46.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35eeffd640 1 --2- 172.21.15.152:0/1524178632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b49d0 0x55bcb53b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:46.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35ee7fc640 1 -- 172.21.15.152:0/1524178632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53bfbb0 msgr2=0x55bcb53d8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:46.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35ee7fc640 1 --2- 172.21.15.152:0/1524178632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53bfbb0 0x55bcb53d8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35ee7fc640 1 -- 172.21.15.152:0/1524178632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b49d0 msgr2=0x55bcb53b8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:52:46.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35ee7fc640 1 --2- 172.21.15.152:0/1524178632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b49d0 0x55bcb53b8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35ee7fc640 1 -- 172.21.15.152:0/1524178632 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcb53d8e80 con 0x55bcb53b52f0 2024-09-11T22:52:46.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35eeffd640 1 --2- 172.21.15.152:0/1524178632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b49d0 0x55bcb53b8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:46.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35ee7fc640 1 --2- 172.21.15.152:0/1524178632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53b52f0 0x55bcb53b9200 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f35e4004810 tx=0x7f35e40126a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1efb5c799536c78 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:46.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:52:46.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35d2ff5640 1 -- 172.21.15.152:0/1524178632 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4019030 con 0x55bcb53b52f0 2024-09-11T22:52:46.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35d2ff5640 1 -- 172.21.15.152:0/1524178632 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f35e400f040 con 0x55bcb53b52f0 2024-09-11T22:52:46.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.595+0000 7f35f7907740 1 -- 172.21.15.152:0/1524178632 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e8027070 con 0x55bcb53b52f0 2024-09-11T22:52:46.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.596+0000 7f35d2ff5640 1 -- 172.21.15.152:0/1524178632 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f35e4024020 con 0x55bcb53b52f0 2024-09-11T22:52:46.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.596+0000 7f35f7907740 10 client.15216 fetch_fsmap learned FSMap version 27 2024-09-11T22:52:46.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.596+0000 7f35f7907740 1 -- 172.21.15.152:0/1524178632 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bcb53da170 con 0x55bcb53b52f0 2024-09-11T22:52:46.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.596+0000 7f35d2ff5640 1 -- 172.21.15.152:0/1524178632 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e8027070 con 0x55bcb53b52f0 2024-09-11T22:52:46.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.596+0000 7f35d2ff5640 1 -- 172.21.15.152:0/1524178632 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f35e4015450 con 0x55bcb53b52f0 2024-09-11T22:52:46.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.596+0000 7f35f7907740 10 client.15216 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:52:46.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.596+0000 7f35f7907740 10 client.15216 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:52:46.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.596+0000 7f35f7907740 20 client.15216 populate_metadata read hostname 'smithi152' 2024-09-11T22:52:46.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.596+0000 7f35f7907740 1 --2- 172.21.15.152:0/1524178632 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55bcb543ef00 0x55bcb543dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:46.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.596+0000 7f35f7907740 4 client.15216 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:52:46.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.596+0000 7f35f7907740 1 -- 172.21.15.152:0/1524178632 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "status"}) -- 0x55bcb5322b10 con 0x55bcb543ef00 2024-09-11T22:52:46.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.596+0000 7f35eeffd640 1 --2- 172.21.15.152:0/1524178632 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55bcb543ef00 0x55bcb543dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:46.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35eeffd640 1 --2- 172.21.15.152:0/1524178632 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55bcb543ef00 0x55bcb543dc10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa36365af1a0597 server_cookie=8be5ee5f1973df81 in_seq=0 out_seq=0 2024-09-11T22:52:46.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35d2ff5640 10 client.15216 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:52:46.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35d2ff5640 1 -- 172.21.15.152:0/1524178632 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bcb5322b10 con 0x55bcb543ef00 2024-09-11T22:52:46.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35d2ff5640 10 client.15216 handle_command_reply: tid=0 2024-09-11T22:52:46.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35f7907740 1 client.15216 shutdown 2024-09-11T22:52:46.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35f7907740 1 -- 172.21.15.152:0/1524178632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53b52f0 msgr2=0x55bcb53b9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:46.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35f7907740 1 --2- 172.21.15.152:0/1524178632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53b52f0 0x55bcb53b9200 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f35e4004810 tx=0x7f35e40126a0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35f7907740 1 -- 172.21.15.152:0/1524178632 shutdown_connections 2024-09-11T22:52:46.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35f7907740 1 --2- 172.21.15.152:0/1524178632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bcb53bfbb0 0x55bcb53d8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35f7907740 1 --2- 172.21.15.152:0/1524178632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bcb53b52f0 0x55bcb53b9200 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35f7907740 1 --2- 172.21.15.152:0/1524178632 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55bcb543ef00 0x55bcb543dc10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35d2ff5640 0 client.15216 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:52:46.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35f7907740 1 --2- 172.21.15.152:0/1524178632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bcb53b49d0 0x55bcb53b8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35f7907740 1 -- 172.21.15.152:0/1524178632 >> 172.21.15.152:0/1524178632 conn(0x55bcb53b0f60 msgr2=0x55bcb53bc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:46.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35f7907740 1 -- 172.21.15.152:0/1524178632 shutdown_connections 2024-09-11T22:52:46.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35f7907740 1 -- 172.21.15.152:0/1524178632 wait complete. 2024-09-11T22:52:46.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.597+0000 7f35f7907740 20 client.15216 trim_cache size 0 max 16384 2024-09-11T22:52:46.604 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:52:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:52:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 0, 2024-09-11T22:52:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 24455, 2024-09-11T22:52:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-09-11T22:52:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-09-11T22:52:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-09-11T22:52:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 111.442785946, 2024-09-11T22:52:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 27, 2024-09-11T22:52:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 83, 2024-09-11T22:52:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 81, 2024-09-11T22:52:46.605 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 123.55132991799999 2024-09-11T22:52:46.605 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:52:46.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.598+0000 7f35f552e640 1 -- 172.21.15.152:0/3110384490 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f35d80a2b80 msgr2=0x7f35d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:46.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.598+0000 7f35f552e640 1 --2- 172.21.15.152:0/3110384490 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f35d80a2b80 0x7f35d80a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f35e8006aa0 tx=0x7f35e8006da0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.598+0000 7f35f552e640 1 -- 172.21.15.152:0/3110384490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35f0101de0 msgr2=0x7f35f01985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:46.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.598+0000 7f35f552e640 1 --2- 172.21.15.152:0/3110384490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35f0101de0 0x7f35f01985b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f35e00097b0 tx=0x7f35e0031b10 comp rx=0 tx=0).stop 2024-09-11T22:52:46.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.598+0000 7f35f552e640 1 -- 172.21.15.152:0/3110384490 shutdown_connections 2024-09-11T22:52:46.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.598+0000 7f35f552e640 1 --2- 172.21.15.152:0/3110384490 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f35d80a2b80 0x7f35d80a5040 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.598+0000 7f35f552e640 1 --2- 172.21.15.152:0/3110384490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35f0106f10 0x7f35f019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.598+0000 7f35f552e640 1 --2- 172.21.15.152:0/3110384490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35f0102790 0x7f35f0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.598+0000 7f35f552e640 1 --2- 172.21.15.152:0/3110384490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35f0101de0 0x7f35f01985b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:46.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.598+0000 7f35f552e640 1 -- 172.21.15.152:0/3110384490 >> 172.21.15.152:0/3110384490 conn(0x7f35f00fd8c0 msgr2=0x7f35f00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:46.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.598+0000 7f35f552e640 1 -- 172.21.15.152:0/3110384490 shutdown_connections 2024-09-11T22:52:46.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:46.598+0000 7f35f552e640 1 -- 172.21.15.152:0/3110384490 wait complete. 2024-09-11T22:52:47.493 INFO:teuthology.run_tasks:Running task sequential... 2024-09-11T22:52:47.504 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-11T22:52:47.504 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-11T22:52:47.505 INFO:teuthology.task.print:Enabling mgr modules 2024-09-11T22:52:47.505 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-11T22:52:47.507 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T22:52:47.507 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi152.front.sepia.ceph.com 2024-09-11T22:52:47.507 DEBUG:teuthology.orchestra.run.smithi152:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-11T22:52:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.608+0000 7f8a7aa2e640 1 -- 172.21.15.152:0/497874211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a74104f10 msgr2=0x7f8a74107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.608+0000 7f8a7aa2e640 1 --2- 172.21.15.152:0/497874211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a74104f10 0x7f8a74107300 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009900 tx=0x7f8a6802a410 comp rx=0 tx=0).stop 2024-09-11T22:52:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.608+0000 7f8a7aa2e640 1 -- 172.21.15.152:0/497874211 shutdown_connections 2024-09-11T22:52:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.608+0000 7f8a7aa2e640 1 --2- 172.21.15.152:0/497874211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74107840 0x7f8a74109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.608+0000 7f8a7aa2e640 1 --2- 172.21.15.152:0/497874211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a74104f10 0x7f8a74107300 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.608+0000 7f8a7aa2e640 1 --2- 172.21.15.152:0/497874211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a741025e0 0x7f8a741049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.608+0000 7f8a7aa2e640 1 -- 172.21.15.152:0/497874211 >> 172.21.15.152:0/497874211 conn(0x7f8a740fc1e0 msgr2=0x7f8a740fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.608+0000 7f8a7aa2e640 1 -- 172.21.15.152:0/497874211 shutdown_connections 2024-09-11T22:52:47.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.608+0000 7f8a7aa2e640 1 -- 172.21.15.152:0/497874211 wait complete. 2024-09-11T22:52:47.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.608+0000 7f8a7aa2e640 1 Processor -- start 2024-09-11T22:52:47.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a7aa2e640 1 -- start start 2024-09-11T22:52:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a7aa2e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a741025e0 0x7f8a74197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a7aa2e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74104f10 0x7f8a74198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a7aa2e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a74107840 0x7f8a7419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a7aa2e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7410d810 con 0x7f8a741025e0 2024-09-11T22:52:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a7aa2e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7410d690 con 0x7f8a74104f10 2024-09-11T22:52:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a7aa2e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7410d990 con 0x7f8a74107840 2024-09-11T22:52:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a73fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a741025e0 0x7f8a74197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a73fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a741025e0 0x7f8a74197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58956/0 (socket says 172.21.15.152:58956) 2024-09-11T22:52:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a73fff640 1 -- 172.21.15.152:0/3849114834 learned_addr learned my addr 172.21.15.152:0/3849114834 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a737fe640 1 --2- 172.21.15.152:0/3849114834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74104f10 0x7f8a74198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a78fcb640 1 --2- 172.21.15.152:0/3849114834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a74107840 0x7f8a7419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a73fff640 1 -- 172.21.15.152:0/3849114834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a74107840 msgr2=0x7f8a7419f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a73fff640 1 --2- 172.21.15.152:0/3849114834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a74107840 0x7f8a7419f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a73fff640 1 -- 172.21.15.152:0/3849114834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74104f10 msgr2=0x7f8a74198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a73fff640 1 --2- 172.21.15.152:0/3849114834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74104f10 0x7f8a74198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a73fff640 1 -- 172.21.15.152:0/3849114834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7419fa50 con 0x7f8a741025e0 2024-09-11T22:52:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a78fcb640 1 --2- 172.21.15.152:0/3849114834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a74107840 0x7f8a7419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:52:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a737fe640 1 --2- 172.21.15.152:0/3849114834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74104f10 0x7f8a74198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:52:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.609+0000 7f8a73fff640 1 --2- 172.21.15.152:0/3849114834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a741025e0 0x7f8a74197e30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400e9f0 tx=0x7f8a6400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.610+0000 7f8a717fa640 1 -- 172.21.15.152:0/3849114834 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6400cd80 con 0x7f8a741025e0 2024-09-11T22:52:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.610+0000 7f8a717fa640 1 -- 172.21.15.152:0/3849114834 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a64005500 con 0x7f8a741025e0 2024-09-11T22:52:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.610+0000 7f8a7aa2e640 1 -- 172.21.15.152:0/3849114834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7419fd40 con 0x7f8a741025e0 2024-09-11T22:52:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.610+0000 7f8a717fa640 1 -- 172.21.15.152:0/3849114834 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64010690 con 0x7f8a741025e0 2024-09-11T22:52:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.610+0000 7f8a7aa2e640 1 -- 172.21.15.152:0/3849114834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a741a01d0 con 0x7f8a741025e0 2024-09-11T22:52:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.611+0000 7f8a7aa2e640 1 -- 172.21.15.152:0/3849114834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a740fdcd0 con 0x7f8a741025e0 2024-09-11T22:52:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.612+0000 7f8a717fa640 1 -- 172.21.15.152:0/3849114834 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8a640046c0 con 0x7f8a741025e0 2024-09-11T22:52:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.612+0000 7f8a717fa640 1 --2- 172.21.15.152:0/3849114834 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8a5c0a2cb0 0x7f8a5c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.613+0000 7f8a737fe640 1 --2- 172.21.15.152:0/3849114834 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8a5c0a2cb0 0x7f8a5c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.613+0000 7f8a717fa640 1 -- 172.21.15.152:0/3849114834 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f8a640cf500 con 0x7f8a741025e0 2024-09-11T22:52:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.613+0000 7f8a737fe640 1 --2- 172.21.15.152:0/3849114834 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8a5c0a2cb0 0x7f8a5c0a5170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8a68000c00 tx=0x7f8a68031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.617+0000 7f8a717fa640 1 -- 172.21.15.152:0/3849114834 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a6409c490 con 0x7f8a741025e0 2024-09-11T22:52:47.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.827+0000 7f8a7aa2e640 1 -- 172.21.15.152:0/3849114834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f8a74062ef0 con 0x7f8a741025e0 2024-09-11T22:52:47.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.937+0000 7f8a717fa640 1 -- 172.21.15.152:0/3849114834 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f8a6409bd90 con 0x7f8a741025e0 2024-09-11T22:52:47.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.939+0000 7f8a7aa2e640 1 -- 172.21.15.152:0/3849114834 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8a5c0a2cb0 msgr2=0x7f8a5c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:47.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.939+0000 7f8a7aa2e640 1 --2- 172.21.15.152:0/3849114834 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8a5c0a2cb0 0x7f8a5c0a5170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8a68000c00 tx=0x7f8a68031400 comp rx=0 tx=0).stop 2024-09-11T22:52:47.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.939+0000 7f8a7aa2e640 1 -- 172.21.15.152:0/3849114834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a741025e0 msgr2=0x7f8a74197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:47.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.939+0000 7f8a7aa2e640 1 --2- 172.21.15.152:0/3849114834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a741025e0 0x7f8a74197e30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400e9f0 tx=0x7f8a6400edf0 comp rx=0 tx=0).stop 2024-09-11T22:52:47.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.939+0000 7f8a7aa2e640 1 -- 172.21.15.152:0/3849114834 shutdown_connections 2024-09-11T22:52:47.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.939+0000 7f8a7aa2e640 1 --2- 172.21.15.152:0/3849114834 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f8a5c0a2cb0 0x7f8a5c0a5170 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:47.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.939+0000 7f8a7aa2e640 1 --2- 172.21.15.152:0/3849114834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a74107840 0x7f8a7419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:47.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.940+0000 7f8a7aa2e640 1 --2- 172.21.15.152:0/3849114834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a74104f10 0x7f8a74198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:47.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.940+0000 7f8a7aa2e640 1 --2- 172.21.15.152:0/3849114834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8a741025e0 0x7f8a74197e30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:47.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.940+0000 7f8a7aa2e640 1 -- 172.21.15.152:0/3849114834 >> 172.21.15.152:0/3849114834 conn(0x7f8a740fc1e0 msgr2=0x7f8a74102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:47.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.940+0000 7f8a7aa2e640 1 -- 172.21.15.152:0/3849114834 shutdown_connections 2024-09-11T22:52:47.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:47.940+0000 7f8a7aa2e640 1 -- 172.21.15.152:0/3849114834 wait complete. 2024-09-11T22:52:47.958 INFO:teuthology.run_tasks:Running task kclient... 2024-09-11T22:52:47.985 INFO:tasks.kclient:Mounting kernel clients... 2024-09-11T22:52:47.985 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-11T22:52:47.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:52:47.986 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-09-11T22:52:48.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:52:48.004 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link delete ceph-brx 2024-09-11T22:52:48.072 INFO:teuthology.orchestra.run.smithi152.stderr:Cannot find device "ceph-brx" 2024-09-11T22:52:48.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:52:48.074 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-11T22:52:48.074 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T22:52:48.074 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T22:52:48.074 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-11T22:52:48.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:47 smithi191 ceph-mon[35211]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T22:52:48.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:47 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3849114834' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T22:52:48.287 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:47 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: ignoring --setuser ceph since I am not root 2024-09-11T22:52:48.287 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:47 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: ignoring --setgroup ceph since I am not root 2024-09-11T22:52:48.288 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:47 smithi191 ceph-mgr[36576]: -- 172.21.15.191:0/2614728502 <== mon.2 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a46c16d0e0 con 0x55a46c14b000 2024-09-11T22:52:48.288 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:48 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:48.088+0000 7fafe005e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:52:48.288 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:48 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:48.145+0000 7fafe005e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:52:48.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:47 smithi204 ceph-mon[35454]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T22:52:48.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:47 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3849114834' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T22:52:48.296 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:47 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: ignoring --setuser ceph since I am not root 2024-09-11T22:52:48.296 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:47 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: ignoring --setgroup ceph since I am not root 2024-09-11T22:52:48.296 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:47 smithi204 ceph-mgr[37262]: -- 172.21.15.204:0/936039502 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c1223dd0e0 con 0x55c1223bb000 2024-09-11T22:52:48.296 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:48 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:48.090+0000 7ff20f132040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:52:48.297 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:48 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:48.147+0000 7ff20f132040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:52:48.313 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:47 smithi152 ceph-mon[31181]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-11T22:52:48.313 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:47 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3849114834' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T22:52:48.313 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:47 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: ignoring --setuser ceph since I am not root 2024-09-11T22:52:48.313 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:47 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: ignoring --setgroup ceph since I am not root 2024-09-11T22:52:48.313 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:48 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:48.094+0000 7f5f741a0040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:52:48.313 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:48 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:48.154+0000 7f5f741a0040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:52:48.313 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:48 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:48.312+0000 7f5f741a0040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:52:48.375 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:48.622 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:48 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:48.390+0000 7f5f741a0040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:52:48.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:48 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:48.541+0000 7f5f741a0040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:52:48.651 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:48 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:48.295+0000 7fafe005e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:52:48.651 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:48 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:48.362+0000 7fafe005e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:52:48.652 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:48 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:48.508+0000 7fafe005e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:52:48.652 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:48 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:48.297+0000 7ff20f132040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:52:48.652 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:48 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:48.364+0000 7ff20f132040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:52:48.652 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:48 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:48.511+0000 7ff20f132040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:52:48.939 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:48 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:48.689+0000 7f5f741a0040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:52:48.939 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:48 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:48.650+0000 7fafe005e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:52:48.939 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:48 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:48.651+0000 7ff20f132040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:52:48.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.893+0000 7f23c9bd5640 1 -- 172.21.15.152:0/3854706481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c4100e20 msgr2=0x7f23c4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:48.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.893+0000 7f23c9bd5640 1 --2- 172.21.15.152:0/3854706481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c4100e20 0x7f23c4101200 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009900 tx=0x7f23b402a480 comp rx=0 tx=0).stop 2024-09-11T22:52:48.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.893+0000 7f23c9bd5640 1 -- 172.21.15.152:0/3854706481 shutdown_connections 2024-09-11T22:52:48.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.893+0000 7f23c9bd5640 1 --2- 172.21.15.152:0/3854706481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c4102ef0 0x7f23c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:48.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.893+0000 7f23c9bd5640 1 --2- 172.21.15.152:0/3854706481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23c4102570 0x7f23c41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:48.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.893+0000 7f23c9bd5640 1 --2- 172.21.15.152:0/3854706481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c4100e20 0x7f23c4101200 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:48.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.893+0000 7f23c9bd5640 1 -- 172.21.15.152:0/3854706481 >> 172.21.15.152:0/3854706481 conn(0x7f23c40fc7d0 msgr2=0x7f23c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:48.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.893+0000 7f23c9bd5640 1 -- 172.21.15.152:0/3854706481 shutdown_connections 2024-09-11T22:52:48.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.893+0000 7f23c9bd5640 1 -- 172.21.15.152:0/3854706481 wait complete. 2024-09-11T22:52:48.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.894+0000 7f23c9bd5640 1 Processor -- start 2024-09-11T22:52:48.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.894+0000 7f23c9bd5640 1 -- start start 2024-09-11T22:52:48.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.894+0000 7f23c9bd5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c4100e20 0x7f23c41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:48.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.894+0000 7f23c9bd5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c4102570 0x7f23c41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:48.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.894+0000 7f23c9bd5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23c4102ef0 0x7f23c419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:48.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.894+0000 7f23c9bd5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f23c4110f50 con 0x7f23c4102570 2024-09-11T22:52:48.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.894+0000 7f23c9bd5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f23c4110dd0 con 0x7f23c4102ef0 2024-09-11T22:52:48.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.894+0000 7f23c9bd5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f23c41110d0 con 0x7f23c4100e20 2024-09-11T22:52:48.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.894+0000 7f23c2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c4102570 0x7f23c41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:48.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.894+0000 7f23c2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c4102570 0x7f23c41989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45268/0 (socket says 172.21.15.152:45268) 2024-09-11T22:52:48.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.894+0000 7f23c2ffd640 1 -- 172.21.15.152:0/2596754189 learned_addr learned my addr 172.21.15.152:0/2596754189 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:48.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.894+0000 7f23c37fe640 1 --2- 172.21.15.152:0/2596754189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c4100e20 0x7f23c41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:48.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.894+0000 7f23c3fff640 1 --2- 172.21.15.152:0/2596754189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23c4102ef0 0x7f23c419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:48.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.895+0000 7f23c2ffd640 1 -- 172.21.15.152:0/2596754189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c4100e20 msgr2=0x7f23c41984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:48.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.895+0000 7f23c2ffd640 1 --2- 172.21.15.152:0/2596754189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c4100e20 0x7f23c41984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:48.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.895+0000 7f23c2ffd640 1 -- 172.21.15.152:0/2596754189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23c4102ef0 msgr2=0x7f23c419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:48.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.895+0000 7f23c2ffd640 1 --2- 172.21.15.152:0/2596754189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23c4102ef0 0x7f23c419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:48.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.895+0000 7f23c2ffd640 1 -- 172.21.15.152:0/2596754189 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c419d450 con 0x7f23c4102570 2024-09-11T22:52:48.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.895+0000 7f23c2ffd640 1 --2- 172.21.15.152:0/2596754189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c4102570 0x7f23c41989e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f23b800d950 tx=0x7f23b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:48.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.895+0000 7f23c0ff9640 1 -- 172.21.15.152:0/2596754189 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8017030 con 0x7f23c4102570 2024-09-11T22:52:48.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.895+0000 7f23c0ff9640 1 -- 172.21.15.152:0/2596754189 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b8004020 con 0x7f23c4102570 2024-09-11T22:52:48.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.895+0000 7f23c9bd5640 1 -- 172.21.15.152:0/2596754189 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c419d740 con 0x7f23c4102570 2024-09-11T22:52:48.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.895+0000 7f23c0ff9640 1 -- 172.21.15.152:0/2596754189 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b80047a0 con 0x7f23c4102570 2024-09-11T22:52:48.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.895+0000 7f23c9bd5640 1 -- 172.21.15.152:0/2596754189 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c41a5020 con 0x7f23c4102570 2024-09-11T22:52:48.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.896+0000 7f23c9bd5640 1 -- 172.21.15.152:0/2596754189 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c4101200 con 0x7f23c4102570 2024-09-11T22:52:48.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.897+0000 7f23c0ff9640 1 -- 172.21.15.152:0/2596754189 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7f23b80041d0 con 0x7f23c4102570 2024-09-11T22:52:48.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.898+0000 7f23c0ff9640 1 --2- 172.21.15.152:0/2596754189 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f23ac0a2c50 0x7f23ac0a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:48.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.898+0000 7f23c37fe640 1 -- 172.21.15.152:0/2596754189 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f23ac0a2c50 msgr2=0x7f23ac0a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/3438861948 2024-09-11T22:52:48.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.898+0000 7f23c0ff9640 1 -- 172.21.15.152:0/2596754189 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f23b80cefa0 con 0x7f23c4102570 2024-09-11T22:52:48.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.898+0000 7f23c37fe640 1 --2- 172.21.15.152:0/2596754189 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f23ac0a2c50 0x7f23ac0a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:52:48.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:48.905+0000 7f23c0ff9640 1 -- 172.21.15.152:0/2596754189 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f23b809c050 con 0x7f23c4102570 2024-09-11T22:52:49.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:49.099+0000 7f23c37fe640 1 -- 172.21.15.152:0/2596754189 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f23ac0a2c50 msgr2=0x7f23ac0a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/3438861948 2024-09-11T22:52:49.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:49.099+0000 7f23c37fe640 1 --2- 172.21.15.152:0/2596754189 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f23ac0a2c50 0x7f23ac0a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:52:49.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:49.119+0000 7f23c9bd5640 1 -- 172.21.15.152:0/2596754189 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f23c4199800 con 0x7f23c4102570 2024-09-11T22:52:49.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:49.119+0000 7f23c0ff9640 1 -- 172.21.15.152:0/2596754189 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f23c4199800 con 0x7f23c4102570 2024-09-11T22:52:49.120 INFO:teuthology.orchestra.run.smithi152.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T22:52:49.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:49.121+0000 7f23c9bd5640 1 -- 172.21.15.152:0/2596754189 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f23ac0a2c50 msgr2=0x7f23ac0a5110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:52:49.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:49.121+0000 7f23c9bd5640 1 --2- 172.21.15.152:0/2596754189 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f23ac0a2c50 0x7f23ac0a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:49.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:49.121+0000 7f23c9bd5640 1 -- 172.21.15.152:0/2596754189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c4102570 msgr2=0x7f23c41989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:49.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:49.121+0000 7f23c9bd5640 1 --2- 172.21.15.152:0/2596754189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c4102570 0x7f23c41989e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f23b800d950 tx=0x7f23b800de10 comp rx=0 tx=0).stop 2024-09-11T22:52:49.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:49.121+0000 7f23c9bd5640 1 -- 172.21.15.152:0/2596754189 shutdown_connections 2024-09-11T22:52:49.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:49.121+0000 7f23c9bd5640 1 --2- 172.21.15.152:0/2596754189 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f23ac0a2c50 0x7f23ac0a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:49.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:49.121+0000 7f23c9bd5640 1 --2- 172.21.15.152:0/2596754189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23c4102ef0 0x7f23c419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:49.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:49.121+0000 7f23c9bd5640 1 --2- 172.21.15.152:0/2596754189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c4102570 0x7f23c41989e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:49.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:49.121+0000 7f23c9bd5640 1 --2- 172.21.15.152:0/2596754189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c4100e20 0x7f23c41984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:49.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:49.121+0000 7f23c9bd5640 1 -- 172.21.15.152:0/2596754189 >> 172.21.15.152:0/2596754189 conn(0x7f23c40fc7d0 msgr2=0x7f23c4069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:49.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:49.121+0000 7f23c9bd5640 1 -- 172.21.15.152:0/2596754189 shutdown_connections 2024-09-11T22:52:49.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:49.121+0000 7f23c9bd5640 1 -- 172.21.15.152:0/2596754189 wait complete. 2024-09-11T22:52:49.212 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:48 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3849114834' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T22:52:49.212 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:48 smithi191 ceph-mon[35211]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T22:52:49.213 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:48 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:48.963+0000 7fafe005e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:52:49.213 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:49 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:49.024+0000 7fafe005e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:52:49.213 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:48 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3849114834' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T22:52:49.213 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:48 smithi204 ceph-mon[35454]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T22:52:49.213 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:48 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:48.966+0000 7ff20f132040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:52:49.213 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:49 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:49.027+0000 7ff20f132040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:52:49.214 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:49 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:49.212+0000 7ff20f132040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:52:49.276 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:48 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3849114834' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T22:52:49.276 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:48 smithi152 ceph-mon[31181]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T22:52:49.276 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:49 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:49.020+0000 7f5f741a0040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:52:49.276 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:49 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:49.083+0000 7f5f741a0040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:52:49.276 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:49 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:49.275+0000 7f5f741a0040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:52:49.537 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:49 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:49.211+0000 7fafe005e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:52:49.947 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-11T22:52:49.948 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T22:52:49.948 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi152.front.sepia.ceph.com 2024-09-11T22:52:49.948 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T22:52:49.948 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T22:52:49.948 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-11T22:52:49.948 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T22:52:49.948 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T22:52:49.948 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T22:52:49.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:52:49.948 DEBUG:teuthology.orchestra.run.smithi152:> ip addr 2024-09-11T22:52:49.965 INFO:teuthology.orchestra.run.smithi152.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T22:52:49.966 INFO:teuthology.orchestra.run.smithi152.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T22:52:49.966 INFO:teuthology.orchestra.run.smithi152.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T22:52:49.966 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-09-11T22:52:49.966 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 ::1/128 scope host 2024-09-11T22:52:49.966 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-09-11T22:52:49.966 INFO:teuthology.orchestra.run.smithi152.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T22:52:49.966 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:44 brd ff:ff:ff:ff:ff:ff 2024-09-11T22:52:49.966 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f0 2024-09-11T22:52:49.966 INFO:teuthology.orchestra.run.smithi152.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T22:52:49.967 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:da:6c:45 brd ff:ff:ff:ff:ff:ff 2024-09-11T22:52:49.967 INFO:teuthology.orchestra.run.smithi152.stdout: altname enp7s0f1 2024-09-11T22:52:49.967 INFO:teuthology.orchestra.run.smithi152.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T22:52:49.967 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e2 brd ff:ff:ff:ff:ff:ff 2024-09-11T22:52:49.967 INFO:teuthology.orchestra.run.smithi152.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T22:52:49.967 INFO:teuthology.orchestra.run.smithi152.stdout: link/ether 0c:c4:7a:88:6d:e3 brd ff:ff:ff:ff:ff:ff 2024-09-11T22:52:49.967 INFO:teuthology.orchestra.run.smithi152.stdout: inet 172.21.15.152/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-11T22:52:49.967 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft 42154sec preferred_lft 42154sec 2024-09-11T22:52:49.967 INFO:teuthology.orchestra.run.smithi152.stdout: inet6 fe80::ec4:7aff:fe88:6de3/64 scope link noprefixroute 2024-09-11T22:52:49.967 INFO:teuthology.orchestra.run.smithi152.stdout: valid_lft forever preferred_lft forever 2024-09-11T22:52:49.968 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-11T22:52:49.968 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:52:49.968 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-09-11T22:52:49.968 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link add name ceph-brx type bridge 2024-09-11T22:52:49.968 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip addr flush dev ceph-brx 2024-09-11T22:52:49.968 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set ceph-brx up 2024-09-11T22:52:49.968 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-11T22:52:49.969 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-09-11T22:52:50.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:52:50.166 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:49 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2596754189' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T22:52:50.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:52:50.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:52:50.188 DEBUG:teuthology.orchestra.run.smithi152:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T22:52:50.213 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-09-11T22:52:50.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:52:50.215 DEBUG:teuthology.orchestra.run.smithi152:> route 2024-09-11T22:52:50.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:49 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2596754189' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T22:52:50.287 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:50 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:50.099+0000 7fafe005e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:52:50.287 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:50 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:50.168+0000 7fafe005e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:52:50.288 INFO:teuthology.orchestra.run.smithi152.stdout:Kernel IP routing table 2024-09-11T22:52:50.288 INFO:teuthology.orchestra.run.smithi152.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T22:52:50.288 INFO:teuthology.orchestra.run.smithi152.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T22:52:50.288 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T22:52:50.289 INFO:teuthology.orchestra.run.smithi152.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T22:52:50.289 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:52:50.289 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-09-11T22:52:50.289 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T22:52:50.289 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T22:52:50.289 DEBUG:teuthology.orchestra.run.smithi152:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T22:52:50.289 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-09-11T22:52:50.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:49 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2596754189' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T22:52:50.296 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:50 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:50.101+0000 7ff20f132040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:52:50.296 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:50 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:50.170+0000 7ff20f132040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:52:50.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:52:50.490 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:50.187+0000 7f5f741a0040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:52:50.490 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:50.260+0000 7f5f741a0040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:52:50.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:52:50.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:52:50.598 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-09-11T22:52:50.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:52:50.615 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list-id 2024-09-11T22:52:50.671 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:52:50.672 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-09-11T22:52:50.672 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T22:52:50.672 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-11T22:52:50.672 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-09-11T22:52:50.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:52:50.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:52:50.791 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-11T22:52:50.791 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:52:50.792 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-09-11T22:52:50.792 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-11T22:52:50.792 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-11T22:52:50.792 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-11T22:52:50.792 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-11T22:52:50.792 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-11T22:52:50.792 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-09-11T22:52:50.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:52:50.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:52:50.930 DEBUG:teuthology.orchestra.run.smithi152:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:52:50.930 DEBUG:teuthology.orchestra.run.smithi152:> set -e 2024-09-11T22:52:50.930 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set brx.0 up 2024-09-11T22:52:50.930 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link set dev brx.0 master ceph-brx 2024-09-11T22:52:50.930 DEBUG:teuthology.orchestra.run.smithi152:> ') 2024-09-11T22:52:50.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:52:51.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:52:51.012 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T22:52:51.012 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T22:52:51.028 INFO:teuthology.orchestra.run.smithi152.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T22:52:51.028 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T22:52:51.029 DEBUG:teuthology.orchestra.run.smithi152:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T22:52:51.084 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-11T22:52:51.084 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-11T22:52:51.084 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/97e40448-fd10-4885-b1e4-67d66e6e8574 2024-09-11T22:52:51.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:52:51.085 DEBUG:teuthology.orchestra.run.smithi152:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/97e40448-fd10-4885-b1e4-67d66e6e8574 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-11T22:52:51.137 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:50 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:50.875+0000 7fafe005e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:52:51.137 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:50 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:50.992+0000 7fafe005e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:52:51.146 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:50 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:50.883+0000 7ff20f132040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:52:51.146 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:50.999+0000 7ff20f132040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:52:51.178 INFO:teuthology.orchestra.run.smithi152.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-11T22:52:51.178 INFO:teuthology.orchestra.run.smithi152.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-11T22:52:51.178 INFO:teuthology.orchestra.run.smithi152.stdout:invalid new device string format 2024-09-11T22:52:51.309 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:51 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:51.022+0000 7f5f741a0040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:52:51.309 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:51 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:51.148+0000 7f5f741a0040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:52:51.325 INFO:teuthology.orchestra.run.smithi152.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-11T22:52:51.325 INFO:teuthology.orchestra.run.smithi152.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-11T22:52:51.325 INFO:teuthology.orchestra.run.smithi152.stdout:invalid new device string format 2024-09-11T22:52:51.325 INFO:teuthology.orchestra.run.smithi152.stdout:mount.ceph: resolved to: "172.21.15.152:3300,172.21.15.191:3300,172.21.15.204:3300" 2024-09-11T22:52:51.325 INFO:teuthology.orchestra.run.smithi152.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.152:3300,172.21.15.191:3300,172.21.15.204:3300:/volumes/qa/sv_1/97e40448-fd10-4885-b1e4-67d66e6e8574 2024-09-11T22:52:51.326 INFO:teuthology.orchestra.run.smithi152.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=0e4e82bc-708f-11ef-bcea-c7b262605968" will pass to kernel 2024-09-11T22:52:51.327 INFO:teuthology.orchestra.run.smithi152.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-11T22:52:51.327 INFO:teuthology.orchestra.run.smithi152.stdout: You just mounted a file system that supports labels which does not 2024-09-11T22:52:51.327 INFO:teuthology.orchestra.run.smithi152.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-11T22:52:51.327 INFO:teuthology.orchestra.run.smithi152.stdout: applications will generate AVC messages and not be allowed access to 2024-09-11T22:52:51.327 INFO:teuthology.orchestra.run.smithi152.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-11T22:52:51.328 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T22:52:51.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:52:51.328 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T22:52:51.358 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T22:52:51.421 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:51.136+0000 7fafe005e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:52:51.421 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:51.199+0000 7fafe005e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:52:51.421 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:51.352+0000 7fafe005e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:52:51.425 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:51.145+0000 7ff20f132040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:52:51.425 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:51.207+0000 7ff20f132040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:52:51.426 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:51.357+0000 7ff20f132040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:52:51.426 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:51.425+0000 7ff20f132040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:52:51.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T22:52:51.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:52:51.458 INFO:teuthology.orchestra.run.smithi152.stdout:client24829 2024-09-11T22:52:51.458 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:52:51.458 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T22:52:51.476 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T22:52:51.563 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:51 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:51.330+0000 7f5f741a0040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:52:51.563 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:51 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:51.396+0000 7f5f741a0040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:52:51.563 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:51 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:51.562+0000 7f5f741a0040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:52:51.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T22:52:51.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:52:51.575 INFO:teuthology.orchestra.run.smithi152.stdout:client24829 2024-09-11T22:52:51.576 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T22:52:51.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T22:52:51.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:52:51.633 INFO:teuthology.orchestra.run.smithi152.stdout:client24829 2024-09-11T22:52:51.633 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968.client24829/status') 2024-09-11T22:52:51.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:52:51.695 INFO:teuthology.orchestra.run.smithi152.stdout:instance: client.24829 (3)172.21.15.152:0/3066145003 2024-09-11T22:52:51.695 INFO:teuthology.orchestra.run.smithi152.stdout:blocklisted: false 2024-09-11T22:52:51.695 INFO:teuthology.orchestra.run.smithi152.stderr:0+1 records in 2024-09-11T22:52:51.695 INFO:teuthology.orchestra.run.smithi152.stderr:0+1 records out 2024-09-11T22:52:51.695 INFO:teuthology.orchestra.run.smithi152.stderr:72 bytes copied, 2.3191e-05 s, 3.1 MB/s 2024-09-11T22:52:51.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:52:51.703 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T22:52:51.713 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T22:52:51.713 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi152.front.sepia.ceph.com 2024-09-11T22:52:51.713 DEBUG:teuthology.orchestra.run.smithi152:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-11T22:52:51.714 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:51.484+0000 7ff20f132040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:52:51.714 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:51.642+0000 7ff20f132040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:52:51.714 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:51.708+0000 7ff20f132040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:52:51.714 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:51.420+0000 7fafe005e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:52:51.714 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:51.480+0000 7fafe005e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:52:51.714 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:51.639+0000 7fafe005e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:52:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.820+0000 7f511eff6640 1 -- 172.21.15.152:0/1322612734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51181078c0 msgr2=0x7f5118109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.820+0000 7f511eff6640 1 --2- 172.21.15.152:0/1322612734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51181078c0 0x7f5118109cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5110009be0 tx=0x7f511002a6c0 comp rx=0 tx=0).stop 2024-09-11T22:52:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.820+0000 7f511eff6640 1 -- 172.21.15.152:0/1322612734 shutdown_connections 2024-09-11T22:52:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.820+0000 7f511eff6640 1 --2- 172.21.15.152:0/1322612734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51181078c0 0x7f5118109cb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.820+0000 7f511eff6640 1 --2- 172.21.15.152:0/1322612734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5118104f90 0x7f5118107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.820+0000 7f511eff6640 1 --2- 172.21.15.152:0/1322612734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5118102660 0x7f5118104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.820+0000 7f511eff6640 1 -- 172.21.15.152:0/1322612734 >> 172.21.15.152:0/1322612734 conn(0x7f51180fc1e0 msgr2=0x7f51180fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.820+0000 7f511eff6640 1 -- 172.21.15.152:0/1322612734 shutdown_connections 2024-09-11T22:52:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.820+0000 7f511eff6640 1 -- 172.21.15.152:0/1322612734 wait complete. 2024-09-11T22:52:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f511eff6640 1 Processor -- start 2024-09-11T22:52:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f511eff6640 1 -- start start 2024-09-11T22:52:51.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f511eff6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5118102660 0x7f5118197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f511eff6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5118104f90 0x7f5118198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f511eff6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51181078c0 0x7f511819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f511eff6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f511810d7b0 con 0x7f5118104f90 2024-09-11T22:52:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f511eff6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f511810d630 con 0x7f51181078c0 2024-09-11T22:52:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f511eff6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f511810d930 con 0x7f5118102660 2024-09-11T22:52:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f5117fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5118104f90 0x7f5118198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f511cd92640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5118102660 0x7f5118197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f5117fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5118104f90 0x7f5118198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45278/0 (socket says 172.21.15.152:45278) 2024-09-11T22:52:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f5117fff640 1 -- 172.21.15.152:0/933831703 learned_addr learned my addr 172.21.15.152:0/933831703 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f511d593640 1 --2- 172.21.15.152:0/933831703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51181078c0 0x7f511819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f511cd92640 1 -- 172.21.15.152:0/933831703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51181078c0 msgr2=0x7f511819f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f511cd92640 1 --2- 172.21.15.152:0/933831703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51181078c0 0x7f511819f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f511cd92640 1 -- 172.21.15.152:0/933831703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5118104f90 msgr2=0x7f5118198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f511cd92640 1 --2- 172.21.15.152:0/933831703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5118104f90 0x7f5118198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.821+0000 7f511cd92640 1 -- 172.21.15.152:0/933831703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511819fa00 con 0x7f5118102660 2024-09-11T22:52:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.822+0000 7f511cd92640 1 --2- 172.21.15.152:0/933831703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5118102660 0x7f5118197de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f510c005fd0 tx=0x7f510c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.822+0000 7f5115ffb640 1 -- 172.21.15.152:0/933831703 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c00cd00 con 0x7f5118102660 2024-09-11T22:52:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.822+0000 7f5115ffb640 1 -- 172.21.15.152:0/933831703 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f510c0035f0 con 0x7f5118102660 2024-09-11T22:52:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.822+0000 7f511eff6640 1 -- 172.21.15.152:0/933831703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511819fcf0 con 0x7f5118102660 2024-09-11T22:52:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.822+0000 7f5115ffb640 1 -- 172.21.15.152:0/933831703 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c010690 con 0x7f5118102660 2024-09-11T22:52:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.822+0000 7f511eff6640 1 -- 172.21.15.152:0/933831703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51181a0190 con 0x7f5118102660 2024-09-11T22:52:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.823+0000 7f511eff6640 1 -- 172.21.15.152:0/933831703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51180fdcd0 con 0x7f5118102660 2024-09-11T22:52:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.824+0000 7f5115ffb640 1 -- 172.21.15.152:0/933831703 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7f510c0037a0 con 0x7f5118102660 2024-09-11T22:52:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.825+0000 7f5115ffb640 1 --2- 172.21.15.152:0/933831703 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f51080a2c80 0x7f51080a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.825+0000 7f5117fff640 1 -- 172.21.15.152:0/933831703 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f51080a2c80 msgr2=0x7f51080a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/3438861948 2024-09-11T22:52:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.825+0000 7f5115ffb640 1 -- 172.21.15.152:0/933831703 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f510c0cf2f0 con 0x7f5118102660 2024-09-11T22:52:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.825+0000 7f5117fff640 1 --2- 172.21.15.152:0/933831703 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f51080a2c80 0x7f51080a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:52:51.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:51.829+0000 7f5115ffb640 1 -- 172.21.15.152:0/933831703 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f510c009510 con 0x7f5118102660 2024-09-11T22:52:51.872 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:51 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:51.637+0000 7f5f741a0040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:52:51.872 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:51 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:51.702+0000 7f5f741a0040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:52:51.974 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:51.704+0000 7fafe005e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:52:51.975 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:51.772+0000 7fafe005e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:52:51.975 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:51.855+0000 7fafe005e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:52:51.975 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:51.914+0000 7fafe005e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:52:51.975 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:51.973+0000 7fafe005e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:52:51.977 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:51.774+0000 7ff20f132040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:52:51.977 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:51.858+0000 7ff20f132040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:52:51.977 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:51.917+0000 7ff20f132040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:52:51.977 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:51.976+0000 7ff20f132040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:52:52.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.026+0000 7f5117fff640 1 -- 172.21.15.152:0/933831703 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f51080a2c80 msgr2=0x7f51080a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/3438861948 2024-09-11T22:52:52.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.026+0000 7f5117fff640 1 --2- 172.21.15.152:0/933831703 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f51080a2c80 0x7f51080a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:52:52.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.043+0000 7f511eff6640 1 -- 172.21.15.152:0/933831703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f5118062e90 con 0x7f5118102660 2024-09-11T22:52:52.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.098+0000 7f5115ffb640 1 -- 172.21.15.152:0/933831703 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7f510c09c280 con 0x7f5118102660 2024-09-11T22:52:52.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.101+0000 7f511eff6640 1 -- 172.21.15.152:0/933831703 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f51080a2c80 msgr2=0x7f51080a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:52:52.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.101+0000 7f511eff6640 1 --2- 172.21.15.152:0/933831703 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f51080a2c80 0x7f51080a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.101+0000 7f511eff6640 1 -- 172.21.15.152:0/933831703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5118102660 msgr2=0x7f5118197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:52.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.101+0000 7f511eff6640 1 --2- 172.21.15.152:0/933831703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5118102660 0x7f5118197de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f510c005fd0 tx=0x7f510c00ee50 comp rx=0 tx=0).stop 2024-09-11T22:52:52.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.101+0000 7f511eff6640 1 -- 172.21.15.152:0/933831703 shutdown_connections 2024-09-11T22:52:52.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.101+0000 7f511eff6640 1 --2- 172.21.15.152:0/933831703 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f51080a2c80 0x7f51080a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.101+0000 7f511eff6640 1 --2- 172.21.15.152:0/933831703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51181078c0 0x7f511819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.101+0000 7f511eff6640 1 --2- 172.21.15.152:0/933831703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5118104f90 0x7f5118198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.101+0000 7f511eff6640 1 --2- 172.21.15.152:0/933831703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5118102660 0x7f5118197de0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.101+0000 7f511eff6640 1 -- 172.21.15.152:0/933831703 >> 172.21.15.152:0/933831703 conn(0x7f51180fc1e0 msgr2=0x7f5118102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:52.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.101+0000 7f511eff6640 1 -- 172.21.15.152:0/933831703 shutdown_connections 2024-09-11T22:52:52.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.101+0000 7f511eff6640 1 -- 172.21.15.152:0/933831703 wait complete. 2024-09-11T22:52:52.113 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-11T22:52:52.126 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-11T22:52:52.137 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T22:52:52.160 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:52 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/933831703' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T22:52:52.160 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:52 smithi152 ceph-mon[31181]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T22:52:52.160 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:51 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:51.873+0000 7f5f741a0040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:52:52.161 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:51 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:51.941+0000 7f5f741a0040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:52:52.161 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:52 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:52.011+0000 7f5f741a0040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:52:52.161 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:52 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:52.099+0000 7f5f741a0040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:52:52.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.242+0000 7f4f83c2e640 1 -- 172.21.15.152:0/2549718376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f7c104f60 msgr2=0x7f4f7c107350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:52.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.242+0000 7f4f83c2e640 1 --2- 172.21.15.152:0/2549718376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f7c104f60 0x7f4f7c107350 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c009900 tx=0x7f4f6c02a410 comp rx=0 tx=0).stop 2024-09-11T22:52:52.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.242+0000 7f4f83c2e640 1 -- 172.21.15.152:0/2549718376 shutdown_connections 2024-09-11T22:52:52.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.242+0000 7f4f83c2e640 1 --2- 172.21.15.152:0/2549718376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f7c107890 0x7f4f7c109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.242+0000 7f4f83c2e640 1 --2- 172.21.15.152:0/2549718376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f7c104f60 0x7f4f7c107350 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.242+0000 7f4f83c2e640 1 --2- 172.21.15.152:0/2549718376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f7c102630 0x7f4f7c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.242+0000 7f4f83c2e640 1 -- 172.21.15.152:0/2549718376 >> 172.21.15.152:0/2549718376 conn(0x7f4f7c0fc170 msgr2=0x7f4f7c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:52.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.242+0000 7f4f83c2e640 1 -- 172.21.15.152:0/2549718376 shutdown_connections 2024-09-11T22:52:52.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.242+0000 7f4f83c2e640 1 -- 172.21.15.152:0/2549718376 wait complete. 2024-09-11T22:52:52.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.242+0000 7f4f83c2e640 1 Processor -- start 2024-09-11T22:52:52.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.242+0000 7f4f83c2e640 1 -- start start 2024-09-11T22:52:52.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f83c2e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f7c102630 0x7f4f7c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:52.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f83c2e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f7c104f60 0x7f4f7c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:52.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f83c2e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f7c107890 0x7f4f7c19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:52.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f83c2e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c10d7a0 con 0x7f4f7c102630 2024-09-11T22:52:52.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f83c2e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c10d620 con 0x7f4f7c107890 2024-09-11T22:52:52.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f83c2e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c10d920 con 0x7f4f7c104f60 2024-09-11T22:52:52.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f819ca640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f7c102630 0x7f4f7c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:52.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f819ca640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f7c102630 0x7f4f7c197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45296/0 (socket says 172.21.15.152:45296) 2024-09-11T22:52:52.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f819ca640 1 -- 172.21.15.152:0/3061457767 learned_addr learned my addr 172.21.15.152:0/3061457767 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:52.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f811c9640 1 --2- 172.21.15.152:0/3061457767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f7c104f60 0x7f4f7c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:52.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f821cb640 1 --2- 172.21.15.152:0/3061457767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f7c107890 0x7f4f7c19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:52.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f819ca640 1 -- 172.21.15.152:0/3061457767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f7c104f60 msgr2=0x7f4f7c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:52.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f819ca640 1 --2- 172.21.15.152:0/3061457767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f7c104f60 0x7f4f7c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f819ca640 1 -- 172.21.15.152:0/3061457767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f7c107890 msgr2=0x7f4f7c19f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:52.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f819ca640 1 --2- 172.21.15.152:0/3061457767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f7c107890 0x7f4f7c19f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f819ca640 1 -- 172.21.15.152:0/3061457767 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7c19fad0 con 0x7f4f7c102630 2024-09-11T22:52:52.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f811c9640 1 --2- 172.21.15.152:0/3061457767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f7c104f60 0x7f4f7c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:52:52.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f821cb640 1 --2- 172.21.15.152:0/3061457767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f7c107890 0x7f4f7c19f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:52:52.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f819ca640 1 --2- 172.21.15.152:0/3061457767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f7c102630 0x7f4f7c197eb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800e9f0 tx=0x7f4f7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:52.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.243+0000 7f4f72ffd640 1 -- 172.21.15.152:0/3061457767 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7800cd80 con 0x7f4f7c102630 2024-09-11T22:52:52.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.244+0000 7f4f72ffd640 1 -- 172.21.15.152:0/3061457767 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f78010910 con 0x7f4f7c102630 2024-09-11T22:52:52.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.244+0000 7f4f83c2e640 1 -- 172.21.15.152:0/3061457767 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7c19fdc0 con 0x7f4f7c102630 2024-09-11T22:52:52.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.244+0000 7f4f72ffd640 1 -- 172.21.15.152:0/3061457767 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78018990 con 0x7f4f7c102630 2024-09-11T22:52:52.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.244+0000 7f4f83c2e640 1 -- 172.21.15.152:0/3061457767 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f7c1a0220 con 0x7f4f7c102630 2024-09-11T22:52:52.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.245+0000 7f4f83c2e640 1 -- 172.21.15.152:0/3061457767 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f7c0fdc60 con 0x7f4f7c102630 2024-09-11T22:52:52.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.246+0000 7f4f72ffd640 1 -- 172.21.15.152:0/3061457767 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7f4f78010ac0 con 0x7f4f7c102630 2024-09-11T22:52:52.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.247+0000 7f4f72ffd640 1 --2- 172.21.15.152:0/3061457767 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4f680a2be0 0x7f4f680a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:52.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.247+0000 7f4f811c9640 1 -- 172.21.15.152:0/3061457767 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4f680a2be0 msgr2=0x7f4f680a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/3438861948 2024-09-11T22:52:52.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.247+0000 7f4f72ffd640 1 -- 172.21.15.152:0/3061457767 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f4f780cf360 con 0x7f4f7c102630 2024-09-11T22:52:52.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.247+0000 7f4f811c9640 1 --2- 172.21.15.152:0/3061457767 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4f680a2be0 0x7f4f680a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:52:52.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.252+0000 7f4f72ffd640 1 -- 172.21.15.152:0/3061457767 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f7809bca0 con 0x7f4f7c102630 2024-09-11T22:52:52.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:52 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/933831703' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T22:52:52.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:52 smithi191 ceph-mon[35211]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T22:52:52.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:52 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/933831703' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T22:52:52.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:52 smithi204 ceph-mon[35454]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-11T22:52:52.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.428+0000 7f4f83c2e640 1 -- 172.21.15.152:0/3061457767 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f7c062ef0 con 0x7f4f7c102630 2024-09-11T22:52:52.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.429+0000 7f4f72ffd640 1 -- 172.21.15.152:0/3061457767 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22710 (secure 0 0 0) 0x7f4f7809ffa0 con 0x7f4f7c102630 2024-09-11T22:52:52.430 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:52.430 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":83,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","created":"2024-09-11T22:43:00.379317+0000","modified":"2024-09-11T22:51:55.946729+0000","last_up_change":"2024-09-11T22:50:01.043181+0000","last_in_change":"2024-09-11T22:49:38.073870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:46:10.984707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:50:50.833547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:50:53.047604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3cd917b5-77f6-4a4a-8fd0-4ebe3034feac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6803","nonce":1270728007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6805","nonce":1270728007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6809","nonce":1270728007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6807","nonce":1270728007}]},"public_addr":"172.21.15.152:6803/1270728007","cluster_addr":"172.21.15.152:6805/1270728007","heartbeat_back_addr":"172.21.15.152:6809/1270728007","heartbeat_front_addr":"172.21.15.152:6807/1270728007","state":["exists","up"]},{"osd":1,"uuid":"ae589bc4-77b1-4522-9591-b0ce2a5432dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6801","nonce":354467155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6803","nonce":354467155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6807","nonce":354467155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6805","nonce":354467155}]},"public_addr":"172.21.15.191:6801/354467155","cluster_addr":"172.21.15.191:6803/354467155","heartbeat_back_addr":"172.21.15.191:6807/354467155","heartbeat_front_addr":"172.21.15.191:6805/354467155","state":["exists","up"]},{"osd":2,"uuid":"b35ffd0c-d084-43dd-a947-15132d78d59c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6801","nonce":3848396211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6803","nonce":3848396211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6807","nonce":3848396211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6805","nonce":3848396211}]},"public_addr":"172.21.15.204:6801/3848396211","cluster_addr":"172.21.15.204:6803/3848396211","heartbeat_back_addr":"172.21.15.204:6807/3848396211","heartbeat_front_addr":"172.21.15.204:6805/3848396211","state":["exists","up"]},{"osd":3,"uuid":"d66ba7e8-4ab7-449e-9460-2f44ffdf987f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6811","nonce":3351257181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6813","nonce":3351257181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6817","nonce":3351257181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6815","nonce":3351257181}]},"public_addr":"172.21.15.152:6811/3351257181","cluster_addr":"172.21.15.152:6813/3351257181","heartbeat_back_addr":"172.21.15.152:6817/3351257181","heartbeat_front_addr":"172.21.15.152:6815/3351257181","state":["exists","up"]},{"osd":4,"uuid":"e1b56468-6cc7-4d29-988e-9d281118ee36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6809","nonce":2967539563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6811","nonce":2967539563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6815","nonce":2967539563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6813","nonce":2967539563}]},"public_addr":"172.21.15.191:6809/2967539563","cluster_addr":"172.21.15.191:6811/2967539563","heartbeat_back_addr":"172.21.15.191:6815/2967539563","heartbeat_front_addr":"172.21.15.191:6813/2967539563","state":["exists","up"]},{"osd":5,"uuid":"89dabd3e-77e3-4294-9648-b112ef694e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6809","nonce":4283562158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6811","nonce":4283562158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6815","nonce":4283562158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6813","nonce":4283562158}]},"public_addr":"172.21.15.204:6809/4283562158","cluster_addr":"172.21.15.204:6811/4283562158","heartbeat_back_addr":"172.21.15.204:6815/4283562158","heartbeat_front_addr":"172.21.15.204:6813/4283562158","state":["exists","up"]},{"osd":6,"uuid":"46a0fded-b668-4aef-a68f-613cccc83483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6819","nonce":2484234042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6821","nonce":2484234042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6825","nonce":2484234042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6823","nonce":2484234042}]},"public_addr":"172.21.15.152:6819/2484234042","cluster_addr":"172.21.15.152:6821/2484234042","heartbeat_back_addr":"172.21.15.152:6825/2484234042","heartbeat_front_addr":"172.21.15.152:6823/2484234042","state":["exists","up"]},{"osd":7,"uuid":"e674bfe9-e2db-44a3-b74b-a21f198747c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6817","nonce":269608859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6819","nonce":269608859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6823","nonce":269608859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6821","nonce":269608859}]},"public_addr":"172.21.15.191:6817/269608859","cluster_addr":"172.21.15.191:6819/269608859","heartbeat_back_addr":"172.21.15.191:6823/269608859","heartbeat_front_addr":"172.21.15.191:6821/269608859","state":["exists","up"]},{"osd":8,"uuid":"0418bfc9-d6b6-488f-b4c7-0bc6291bb624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6817","nonce":1756190090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6819","nonce":1756190090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6823","nonce":1756190090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6821","nonce":1756190090}]},"public_addr":"172.21.15.204:6817/1756190090","cluster_addr":"172.21.15.204:6819/1756190090","heartbeat_back_addr":"172.21.15.204:6823/1756190090","heartbeat_front_addr":"172.21.15.204:6821/1756190090","state":["exists","up"]},{"osd":9,"uuid":"751a6964-798f-4240-a2ba-7fa9f0c72657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6827","nonce":1301191419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6829","nonce":1301191419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6833","nonce":1301191419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6831","nonce":1301191419}]},"public_addr":"172.21.15.152:6827/1301191419","cluster_addr":"172.21.15.152:6829/1301191419","heartbeat_back_addr":"172.21.15.152:6833/1301191419","heartbeat_front_addr":"172.21.15.152:6831/1301191419","state":["exists","up"]},{"osd":10,"uuid":"90e7c6ba-127c-4b33-b3c6-cedf72128451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6825","nonce":3695294503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6827","nonce":3695294503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6831","nonce":3695294503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6829","nonce":3695294503}]},"public_addr":"172.21.15.191:6825/3695294503","cluster_addr":"172.21.15.191:6827/3695294503","heartbeat_back_addr":"172.21.15.191:6831/3695294503","heartbeat_front_addr":"172.21.15.191:6829/3695294503","state":["exists","up"]},{"osd":11,"uuid":"c76a93b1-a045-477b-98d0-58a2e9d44a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6825","nonce":1709433236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6827","nonce":1709433236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6831","nonce":1709433236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6829","nonce":1709433236}]},"public_addr":"172.21.15.204:6825/1709433236","cluster_addr":"172.21.15.204:6827/1709433236","heartbeat_back_addr":"172.21.15.204:6831/1709433236","heartbeat_front_addr":"172.21.15.204:6829/1709433236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:17.259744+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:42.230389+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:07.172433+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:32.517060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:58.900615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:24.379803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:50.058928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:15.967299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:42.025873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:08.225448+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:33.910531+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:59.466423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:0/1542993850":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/3071755865":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/2452559014":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/4286398432":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6801/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6800/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6800/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/3960601922":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2270507277":"2024-09-12T22:43:31.970432+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:52:52.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.433+0000 7f4f83c2e640 1 -- 172.21.15.152:0/3061457767 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4f680a2be0 msgr2=0x7f4f680a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:52:52.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.433+0000 7f4f83c2e640 1 --2- 172.21.15.152:0/3061457767 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4f680a2be0 0x7f4f680a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.433+0000 7f4f83c2e640 1 -- 172.21.15.152:0/3061457767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f7c102630 msgr2=0x7f4f7c197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:52.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.433+0000 7f4f83c2e640 1 --2- 172.21.15.152:0/3061457767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f7c102630 0x7f4f7c197eb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4f7800e9f0 tx=0x7f4f7800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.433+0000 7f4f83c2e640 1 -- 172.21.15.152:0/3061457767 shutdown_connections 2024-09-11T22:52:52.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.433+0000 7f4f83c2e640 1 --2- 172.21.15.152:0/3061457767 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f4f680a2be0 0x7f4f680a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.433+0000 7f4f83c2e640 1 --2- 172.21.15.152:0/3061457767 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f7c107890 0x7f4f7c19f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.433+0000 7f4f83c2e640 1 --2- 172.21.15.152:0/3061457767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f7c104f60 0x7f4f7c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.433+0000 7f4f83c2e640 1 --2- 172.21.15.152:0/3061457767 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f7c102630 0x7f4f7c197eb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.433+0000 7f4f83c2e640 1 -- 172.21.15.152:0/3061457767 >> 172.21.15.152:0/3061457767 conn(0x7f4f7c0fc170 msgr2=0x7f4f7c1030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:52.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.433+0000 7f4f83c2e640 1 -- 172.21.15.152:0/3061457767 shutdown_connections 2024-09-11T22:52:52.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.433+0000 7f4f83c2e640 1 -- 172.21.15.152:0/3061457767 wait complete. 2024-09-11T22:52:52.448 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:52 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:52.167+0000 7f5f741a0040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:52:52.449 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:52 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:52.231+0000 7f5f741a0040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:52:52.451 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T22:46:10.984707+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T22:50:50.833547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T22:50:53.047604+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T22:52:52.452 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T22:52:52.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.596+0000 7f01ef653640 1 -- 172.21.15.152:0/3015851515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e8102620 msgr2=0x7f01e8104a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:52.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.596+0000 7f01ef653640 1 --2- 172.21.15.152:0/3015851515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e8102620 0x7f01e8104a10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009900 tx=0x7f01e402a410 comp rx=0 tx=0).stop 2024-09-11T22:52:52.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.596+0000 7f01ef653640 1 -- 172.21.15.152:0/3015851515 shutdown_connections 2024-09-11T22:52:52.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.596+0000 7f01ef653640 1 --2- 172.21.15.152:0/3015851515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01e8107880 0x7f01e8109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.596+0000 7f01ef653640 1 --2- 172.21.15.152:0/3015851515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01e8104f50 0x7f01e8107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.596+0000 7f01ef653640 1 --2- 172.21.15.152:0/3015851515 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e8102620 0x7f01e8104a10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.596+0000 7f01ef653640 1 -- 172.21.15.152:0/3015851515 >> 172.21.15.152:0/3015851515 conn(0x7f01e80fc140 msgr2=0x7f01e80fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:52.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.596+0000 7f01ef653640 1 -- 172.21.15.152:0/3015851515 shutdown_connections 2024-09-11T22:52:52.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.596+0000 7f01ef653640 1 -- 172.21.15.152:0/3015851515 wait complete. 2024-09-11T22:52:52.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.597+0000 7f01ef653640 1 Processor -- start 2024-09-11T22:52:52.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.597+0000 7f01ef653640 1 -- start start 2024-09-11T22:52:52.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.597+0000 7f01ef653640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01e8102620 0x7f01e8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:52.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.597+0000 7f01ef653640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e8104f50 0x7f01e8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:52.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.597+0000 7f01ef653640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01e8107880 0x7f01e819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:52.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.597+0000 7f01ef653640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f01e810d810 con 0x7f01e8104f50 2024-09-11T22:52:52.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.597+0000 7f01ef653640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f01e810d690 con 0x7f01e8102620 2024-09-11T22:52:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.597+0000 7f01ef653640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f01e810d990 con 0x7f01e8107880 2024-09-11T22:52:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.597+0000 7f01ecbee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e8104f50 0x7f01e8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.597+0000 7f01ecbee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e8104f50 0x7f01e8198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45310/0 (socket says 172.21.15.152:45310) 2024-09-11T22:52:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.598+0000 7f01ecbee640 1 -- 172.21.15.152:0/2644444904 learned_addr learned my addr 172.21.15.152:0/2644444904 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.598+0000 7f01ed3ef640 1 --2- 172.21.15.152:0/2644444904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01e8102620 0x7f01e8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.598+0000 7f01edbf0640 1 --2- 172.21.15.152:0/2644444904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01e8107880 0x7f01e819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.598+0000 7f01ecbee640 1 -- 172.21.15.152:0/2644444904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01e8107880 msgr2=0x7f01e819f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.598+0000 7f01ecbee640 1 --2- 172.21.15.152:0/2644444904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01e8107880 0x7f01e819f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.598+0000 7f01ecbee640 1 -- 172.21.15.152:0/2644444904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01e8102620 msgr2=0x7f01e8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.598+0000 7f01ecbee640 1 --2- 172.21.15.152:0/2644444904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01e8102620 0x7f01e8197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.598+0000 7f01ecbee640 1 -- 172.21.15.152:0/2644444904 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e819fb10 con 0x7f01e8104f50 2024-09-11T22:52:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.598+0000 7f01ed3ef640 1 --2- 172.21.15.152:0/2644444904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01e8102620 0x7f01e8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:52:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.598+0000 7f01ecbee640 1 --2- 172.21.15.152:0/2644444904 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e8104f50 0x7f01e8198430 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f01d800e9c0 tx=0x7f01d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:52.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.598+0000 7f01de7fc640 1 -- 172.21.15.152:0/2644444904 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d800cd80 con 0x7f01e8104f50 2024-09-11T22:52:52.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.598+0000 7f01de7fc640 1 -- 172.21.15.152:0/2644444904 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01d8005500 con 0x7f01e8104f50 2024-09-11T22:52:52.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.598+0000 7f01de7fc640 1 -- 172.21.15.152:0/2644444904 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d8010690 con 0x7f01e8104f50 2024-09-11T22:52:52.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.598+0000 7f01ef653640 1 -- 172.21.15.152:0/2644444904 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e819fe00 con 0x7f01e8104f50 2024-09-11T22:52:52.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.598+0000 7f01ef653640 1 -- 172.21.15.152:0/2644444904 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e81a02c0 con 0x7f01e8104f50 2024-09-11T22:52:52.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.599+0000 7f01ef653640 1 -- 172.21.15.152:0/2644444904 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e80fdc30 con 0x7f01e8104f50 2024-09-11T22:52:52.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.600+0000 7f01de7fc640 1 -- 172.21.15.152:0/2644444904 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7f01d80046c0 con 0x7f01e8104f50 2024-09-11T22:52:52.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.601+0000 7f01de7fc640 1 --2- 172.21.15.152:0/2644444904 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01d40a2be0 0x7f01d40a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:52.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.601+0000 7f01ed3ef640 1 -- 172.21.15.152:0/2644444904 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01d40a2be0 msgr2=0x7f01d40a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/3438861948 2024-09-11T22:52:52.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.601+0000 7f01de7fc640 1 -- 172.21.15.152:0/2644444904 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f01d80cf3e0 con 0x7f01e8104f50 2024-09-11T22:52:52.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.601+0000 7f01ed3ef640 1 --2- 172.21.15.152:0/2644444904 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01d40a2be0 0x7f01d40a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:52:52.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.605+0000 7f01de7fc640 1 -- 172.21.15.152:0/2644444904 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01d809c370 con 0x7f01e8104f50 2024-09-11T22:52:52.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.774+0000 7f01ef653640 1 -- 172.21.15.152:0/2644444904 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f01e8062e90 con 0x7f01e8104f50 2024-09-11T22:52:52.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.775+0000 7f01de7fc640 1 -- 172.21.15.152:0/2644444904 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f01d8086070 con 0x7f01e8104f50 2024-09-11T22:52:52.776 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-09-11T22:52:52.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.777+0000 7f01ef653640 1 -- 172.21.15.152:0/2644444904 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01d40a2be0 msgr2=0x7f01d40a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:52:52.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.777+0000 7f01ef653640 1 --2- 172.21.15.152:0/2644444904 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01d40a2be0 0x7f01d40a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.777+0000 7f01ef653640 1 -- 172.21.15.152:0/2644444904 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e8104f50 msgr2=0x7f01e8198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:52.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.777+0000 7f01ef653640 1 --2- 172.21.15.152:0/2644444904 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e8104f50 0x7f01e8198430 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f01d800e9c0 tx=0x7f01d800ee80 comp rx=0 tx=0).stop 2024-09-11T22:52:52.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.777+0000 7f01ef653640 1 -- 172.21.15.152:0/2644444904 shutdown_connections 2024-09-11T22:52:52.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.777+0000 7f01ef653640 1 --2- 172.21.15.152:0/2644444904 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f01d40a2be0 0x7f01d40a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.777+0000 7f01ef653640 1 --2- 172.21.15.152:0/2644444904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01e8107880 0x7f01e819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.777+0000 7f01ef653640 1 --2- 172.21.15.152:0/2644444904 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e8104f50 0x7f01e8198430 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.777+0000 7f01ef653640 1 --2- 172.21.15.152:0/2644444904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01e8102620 0x7f01e8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.777+0000 7f01ef653640 1 -- 172.21.15.152:0/2644444904 >> 172.21.15.152:0/2644444904 conn(0x7f01e80fc140 msgr2=0x7f01e8103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:52.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.777+0000 7f01ef653640 1 -- 172.21.15.152:0/2644444904 shutdown_connections 2024-09-11T22:52:52.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.777+0000 7f01ef653640 1 -- 172.21.15.152:0/2644444904 wait complete. 2024-09-11T22:52:52.789 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T22:52:52.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.893+0000 7f661d6d0640 1 -- 172.21.15.152:0/169573981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6618100a00 msgr2=0x7f6618100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:52.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.893+0000 7f661d6d0640 1 --2- 172.21.15.152:0/169573981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6618100a00 0x7f6618100e00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6608009900 tx=0x7f660802a410 comp rx=0 tx=0).stop 2024-09-11T22:52:52.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.893+0000 7f661d6d0640 1 -- 172.21.15.152:0/169573981 shutdown_connections 2024-09-11T22:52:52.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.893+0000 7f661d6d0640 1 --2- 172.21.15.152:0/169573981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66181025b0 0x7f6618108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.893+0000 7f661d6d0640 1 --2- 172.21.15.152:0/169573981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6618101bf0 0x7f6618102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.893+0000 7f661d6d0640 1 --2- 172.21.15.152:0/169573981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6618100a00 0x7f6618100e00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.893+0000 7f661d6d0640 1 -- 172.21.15.152:0/169573981 >> 172.21.15.152:0/169573981 conn(0x7f66180fc140 msgr2=0x7f66180fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:52.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.893+0000 7f661d6d0640 1 -- 172.21.15.152:0/169573981 shutdown_connections 2024-09-11T22:52:52.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.893+0000 7f661d6d0640 1 -- 172.21.15.152:0/169573981 wait complete. 2024-09-11T22:52:52.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.894+0000 7f661d6d0640 1 Processor -- start 2024-09-11T22:52:52.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.894+0000 7f661d6d0640 1 -- start start 2024-09-11T22:52:52.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.894+0000 7f661d6d0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6618100a00 0x7f6618196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:52.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.894+0000 7f661d6d0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6618101bf0 0x7f66181972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:52.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.894+0000 7f661d6d0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66181025b0 0x7f661819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:52.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.894+0000 7f661d6d0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f661810c680 con 0x7f6618101bf0 2024-09-11T22:52:52.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.894+0000 7f661d6d0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f661810c500 con 0x7f6618100a00 2024-09-11T22:52:52.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.894+0000 7f661d6d0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f661810c800 con 0x7f66181025b0 2024-09-11T22:52:52.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.894+0000 7f66167fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6618101bf0 0x7f66181972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:52.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.894+0000 7f66177fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66181025b0 0x7f661819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:52.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.895+0000 7f66167fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6618101bf0 0x7f66181972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45326/0 (socket says 172.21.15.152:45326) 2024-09-11T22:52:52.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.895+0000 7f66167fc640 1 -- 172.21.15.152:0/4232529076 learned_addr learned my addr 172.21.15.152:0/4232529076 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:52.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.895+0000 7f6616ffd640 1 --2- 172.21.15.152:0/4232529076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6618100a00 0x7f6618196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:52.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.895+0000 7f66167fc640 1 -- 172.21.15.152:0/4232529076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66181025b0 msgr2=0x7f661819e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:52.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.895+0000 7f66167fc640 1 --2- 172.21.15.152:0/4232529076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66181025b0 0x7f661819e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.895+0000 7f66167fc640 1 -- 172.21.15.152:0/4232529076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6618100a00 msgr2=0x7f6618196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:52.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.895+0000 7f66167fc640 1 --2- 172.21.15.152:0/4232529076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6618100a00 0x7f6618196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:52.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.895+0000 7f66167fc640 1 -- 172.21.15.152:0/4232529076 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661819e9c0 con 0x7f6618101bf0 2024-09-11T22:52:52.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.895+0000 7f66167fc640 1 --2- 172.21.15.152:0/4232529076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6618101bf0 0x7f66181972f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6600009970 tx=0x7f6600009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:52.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.895+0000 7f660ffff640 1 -- 172.21.15.152:0/4232529076 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660000d020 con 0x7f6618101bf0 2024-09-11T22:52:52.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.895+0000 7f661d6d0640 1 -- 172.21.15.152:0/4232529076 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66180693e0 con 0x7f6618101bf0 2024-09-11T22:52:52.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.895+0000 7f660ffff640 1 -- 172.21.15.152:0/4232529076 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6600007d40 con 0x7f6618101bf0 2024-09-11T22:52:52.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.895+0000 7f660ffff640 1 -- 172.21.15.152:0/4232529076 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660000ccc0 con 0x7f6618101bf0 2024-09-11T22:52:52.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.895+0000 7f661d6d0640 1 -- 172.21.15.152:0/4232529076 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6618069900 con 0x7f6618101bf0 2024-09-11T22:52:52.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.896+0000 7f661d6d0640 1 -- 172.21.15.152:0/4232529076 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6618100e00 con 0x7f6618101bf0 2024-09-11T22:52:52.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.898+0000 7f660ffff640 1 -- 172.21.15.152:0/4232529076 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7f660000c460 con 0x7f6618101bf0 2024-09-11T22:52:52.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.899+0000 7f660ffff640 1 --2- 172.21.15.152:0/4232529076 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f66040a2be0 0x7f66040a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:52.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.899+0000 7f6616ffd640 1 -- 172.21.15.152:0/4232529076 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f66040a2be0 msgr2=0x7f66040a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.152:6800/3438861948 2024-09-11T22:52:52.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.899+0000 7f660ffff640 1 -- 172.21.15.152:0/4232529076 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8190+0+0 (secure 0 0 0) 0x7f66000d0940 con 0x7f6618101bf0 2024-09-11T22:52:52.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.899+0000 7f6616ffd640 1 --2- 172.21.15.152:0/4232529076 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f66040a2be0 0x7f66040a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:52:52.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:52.901+0000 7f660ffff640 1 -- 172.21.15.152:0/4232529076 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f660009d280 con 0x7f6618101bf0 2024-09-11T22:52:53.037 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 22:52:52 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y[36572]: 2024-09-11T22:52:52.772+0000 7fafe005e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:52:53.046 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 22:52:52 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z[37258]: 2024-09-11T22:52:52.775+0000 7ff20f132040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:52:53.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.073+0000 7f661d6d0640 1 -- 172.21.15.152:0/4232529076 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6618107ab0 con 0x7f6618101bf0 2024-09-11T22:52:53.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.078+0000 7f660ffff640 1 -- 172.21.15.152:0/4232529076 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7f6600099860 con 0x7f6618101bf0 2024-09-11T22:52:53.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.078+0000 7f660ffff640 1 -- 172.21.15.152:0/4232529076 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f66040a2be0 msgr2=0x7f66040a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:52:53.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.078+0000 7f660ffff640 1 --2- 172.21.15.152:0/4232529076 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f66040a2be0 0x7f66040a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:53.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.080+0000 7f660ffff640 1 -- 172.21.15.152:0/4232529076 <== mon.0 v2:172.21.15.152:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7f66000ca1a0 con 0x7f6618101bf0 2024-09-11T22:52:53.080 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-09-11T22:52:53.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.082+0000 7f661d6d0640 1 -- 172.21.15.152:0/4232529076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6618101bf0 msgr2=0x7f66181972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:53.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.082+0000 7f661d6d0640 1 --2- 172.21.15.152:0/4232529076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6618101bf0 0x7f66181972f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6600009970 tx=0x7f6600009e30 comp rx=0 tx=0).stop 2024-09-11T22:52:53.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.082+0000 7f661d6d0640 1 -- 172.21.15.152:0/4232529076 shutdown_connections 2024-09-11T22:52:53.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.082+0000 7f661d6d0640 1 --2- 172.21.15.152:0/4232529076 >> [v2:172.21.15.152:6800/3438861948,v1:172.21.15.152:6801/3438861948] conn(0x7f66040a2be0 0x7f66040a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:53.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.082+0000 7f661d6d0640 1 --2- 172.21.15.152:0/4232529076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66181025b0 0x7f661819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:53.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.082+0000 7f661d6d0640 1 --2- 172.21.15.152:0/4232529076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6618101bf0 0x7f66181972f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:53.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.082+0000 7f661d6d0640 1 --2- 172.21.15.152:0/4232529076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6618100a00 0x7f6618196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:53.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.082+0000 7f661d6d0640 1 -- 172.21.15.152:0/4232529076 >> 172.21.15.152:0/4232529076 conn(0x7f66180fc140 msgr2=0x7f6618105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:53.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.082+0000 7f661d6d0640 1 -- 172.21.15.152:0/4232529076 shutdown_connections 2024-09-11T22:52:53.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.082+0000 7f661d6d0640 1 -- 172.21.15.152:0/4232529076 wait complete. 2024-09-11T22:52:53.098 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T22:52:53.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.237+0000 7feb41263640 1 -- 172.21.15.152:0/455861507 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0319b0 con 0x7feb3c109330 2024-09-11T22:52:53.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.238+0000 7feb444c9640 1 -- 172.21.15.152:0/455861507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb3c109330 msgr2=0x7feb3c1097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:53.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.238+0000 7feb444c9640 1 --2- 172.21.15.152:0/455861507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb3c109330 0x7feb3c1097b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009900 tx=0x7feb2c02a420 comp rx=0 tx=0).stop 2024-09-11T22:52:53.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.240+0000 7feb444c9640 1 -- 172.21.15.152:0/455861507 shutdown_connections 2024-09-11T22:52:53.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.240+0000 7feb444c9640 1 --2- 172.21.15.152:0/455861507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb3c109cf0 0x7feb3c119130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:53.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.240+0000 7feb444c9640 1 --2- 172.21.15.152:0/455861507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb3c109330 0x7feb3c1097b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:53.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.240+0000 7feb444c9640 1 --2- 172.21.15.152:0/455861507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb3c073440 0x7feb3c073840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:53.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.240+0000 7feb444c9640 1 -- 172.21.15.152:0/455861507 >> 172.21.15.152:0/455861507 conn(0x7feb3c06eb50 msgr2=0x7feb3c070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:53.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.240+0000 7feb444c9640 1 -- 172.21.15.152:0/455861507 shutdown_connections 2024-09-11T22:52:53.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.240+0000 7feb444c9640 1 -- 172.21.15.152:0/455861507 wait complete. 2024-09-11T22:52:53.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.241+0000 7feb444c9640 1 Processor -- start 2024-09-11T22:52:53.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.241+0000 7feb444c9640 1 -- start start 2024-09-11T22:52:53.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.241+0000 7feb444c9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb3c073440 0x7feb3c1a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:53.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.241+0000 7feb444c9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb3c109330 0x7feb3c1a7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:53.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.241+0000 7feb444c9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb3c109cf0 0x7feb3c1ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:53.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.241+0000 7feb444c9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c11c610 con 0x7feb3c109cf0 2024-09-11T22:52:53.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.241+0000 7feb444c9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c11c490 con 0x7feb3c073440 2024-09-11T22:52:53.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.241+0000 7feb444c9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c11c790 con 0x7feb3c109330 2024-09-11T22:52:53.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.241+0000 7feb41a64640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb3c109330 0x7feb3c1a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:53.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.241+0000 7feb41a64640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb3c109330 0x7feb3c1a7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:51706/0 (socket says 172.21.15.152:51706) 2024-09-11T22:52:53.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.241+0000 7feb41a64640 1 -- 172.21.15.152:0/252207087 learned_addr learned my addr 172.21.15.152:0/252207087 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:53.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.241+0000 7feb42a66640 1 --2- 172.21.15.152:0/252207087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb3c109cf0 0x7feb3c1ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:53.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.242+0000 7feb42a66640 1 -- 172.21.15.152:0/252207087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb3c109330 msgr2=0x7feb3c1a7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:53.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.242+0000 7feb42a66640 1 --2- 172.21.15.152:0/252207087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb3c109330 0x7feb3c1a7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:53.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.242+0000 7feb42a66640 1 -- 172.21.15.152:0/252207087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb3c073440 msgr2=0x7feb3c1a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:53.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.242+0000 7feb42a66640 1 --2- 172.21.15.152:0/252207087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb3c073440 0x7feb3c1a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:53.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.242+0000 7feb42a66640 1 -- 172.21.15.152:0/252207087 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3c110a50 con 0x7feb3c109cf0 2024-09-11T22:52:53.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.242+0000 7feb42a66640 1 --2- 172.21.15.152:0/252207087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb3c109cf0 0x7feb3c1ae290 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7feb3c06b9e0 tx=0x7feb3400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:53.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.242+0000 7feb337fe640 1 -- 172.21.15.152:0/252207087 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34015020 con 0x7feb3c109cf0 2024-09-11T22:52:53.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.242+0000 7feb337fe640 1 -- 172.21.15.152:0/252207087 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb34011d10 con 0x7feb3c109cf0 2024-09-11T22:52:53.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.242+0000 7feb337fe640 1 -- 172.21.15.152:0/252207087 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3400f2e0 con 0x7feb3c109cf0 2024-09-11T22:52:53.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.242+0000 7feb444c9640 1 -- 172.21.15.152:0/252207087 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3c110d40 con 0x7feb3c109cf0 2024-09-11T22:52:53.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.243+0000 7feb444c9640 1 -- 172.21.15.152:0/252207087 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb3c1112c0 con 0x7feb3c109cf0 2024-09-11T22:52:53.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.244+0000 7feb444c9640 1 -- 172.21.15.152:0/252207087 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb3c10ee20 con 0x7feb3c109cf0 2024-09-11T22:52:53.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.247+0000 7feb337fe640 1 -- 172.21.15.152:0/252207087 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7feb34018020 con 0x7feb3c109cf0 2024-09-11T22:52:53.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.248+0000 7feb337fe640 1 -- 172.21.15.152:0/252207087 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7feb340ce380 con 0x7feb3c109cf0 2024-09-11T22:52:53.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.256+0000 7feb337fe640 1 -- 172.21.15.152:0/252207087 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feb3401f020 con 0x7feb3c109cf0 2024-09-11T22:52:53.324 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:53.065+0000 7f5f741a0040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:52:53.326 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T22:52:53.326 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:52:53.326 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3061457767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:52:53.326 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2644444904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:52:53.326 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: Standby manager daemon y restarted 2024-09-11T22:52:53.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: Standby manager daemon y started 2024-09-11T22:52:53.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.? 172.21.15.191:0/1872963845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T22:52:53.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: Standby manager daemon z restarted 2024-09-11T22:52:53.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: Standby manager daemon z started 2024-09-11T22:52:53.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.? 172.21.15.191:0/1872963845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:52:53.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.? 172.21.15.191:0/1872963845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T22:52:53.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.? 172.21.15.191:0/1872963845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:52:53.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.? 172.21.15.204:0/4165979187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T22:52:53.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.? 172.21.15.204:0/4165979187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:52:53.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.? 172.21.15.204:0/4165979187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T22:52:53.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.? 172.21.15.204:0/4165979187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:52:53.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: Active manager daemon x restarted 2024-09-11T22:52:53.327 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: Activating manager daemon x 2024-09-11T22:52:53.328 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T22:52:53.328 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: mgrmap e20: x(active, starting, since 0.0126714s), standbys: z, y 2024-09-11T22:52:53.328 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4232529076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T22:52:53.328 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:52:53.328 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:52:53.328 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:52:53.328 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T22:52:53.328 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T22:52:53.328 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:52:53.328 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:52:53.328 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:52:53.328 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:52:53.330 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:52:53.330 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:52:53.330 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:52:53.330 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:52:53.330 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:52:53.331 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:52:53.331 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T22:52:53.331 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T22:52:53.331 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T22:52:53.331 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:52:53.331 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:52:53.331 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:52:53.331 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:52:53.331 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:52:53.331 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:52:53.331 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:52:53.331 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:52:53.331 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:52:53.331 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:52:53.332 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:52:53.332 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:52:53.332 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T22:52:53.332 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T22:52:53.332 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T22:52:53.332 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-mon[31181]: Manager daemon x is now available 2024-09-11T22:52:53.441 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T22:52:53.441 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:52:53.441 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3061457767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:52:53.442 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2644444904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:52:53.442 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: Standby manager daemon y restarted 2024-09-11T22:52:53.442 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: Standby manager daemon y started 2024-09-11T22:52:53.442 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.? 172.21.15.191:0/1872963845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T22:52:53.442 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: Standby manager daemon z restarted 2024-09-11T22:52:53.442 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: Standby manager daemon z started 2024-09-11T22:52:53.442 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.? 172.21.15.191:0/1872963845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:52:53.442 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.? 172.21.15.191:0/1872963845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T22:52:53.442 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.? 172.21.15.191:0/1872963845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:52:53.442 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.? 172.21.15.204:0/4165979187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T22:52:53.442 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.? 172.21.15.204:0/4165979187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:52:53.442 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.? 172.21.15.204:0/4165979187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T22:52:53.442 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.? 172.21.15.204:0/4165979187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:52:53.443 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: Active manager daemon x restarted 2024-09-11T22:52:53.443 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: Activating manager daemon x 2024-09-11T22:52:53.443 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T22:52:53.443 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: mgrmap e20: x(active, starting, since 0.0126714s), standbys: z, y 2024-09-11T22:52:53.443 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/4232529076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T22:52:53.443 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:52:53.443 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:52:53.443 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:52:53.443 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T22:52:53.443 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T22:52:53.443 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:52:53.443 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:52:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:52:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:52:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:52:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:52:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:52:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:52:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:52:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:52:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T22:52:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T22:52:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T22:52:53.444 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:52:53.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:52:53.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:52:53.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:52:53.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:52:53.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:52:53.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:52:53.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:52:53.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:52:53.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:52:53.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:52:53.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:52:53.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T22:52:53.445 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T22:52:53.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T22:52:53.446 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:53 smithi204 ceph-mon[35454]: Manager daemon x is now available 2024-09-11T22:52:53.459 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T22:52:53.459 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:52:53.459 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3061457767' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:52:53.459 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2644444904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:52:53.459 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: Standby manager daemon y restarted 2024-09-11T22:52:53.459 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: Standby manager daemon y started 2024-09-11T22:52:53.459 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.? 172.21.15.191:0/1872963845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T22:52:53.460 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: Standby manager daemon z restarted 2024-09-11T22:52:53.460 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: Standby manager daemon z started 2024-09-11T22:52:53.460 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.? 172.21.15.191:0/1872963845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:52:53.460 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.? 172.21.15.191:0/1872963845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T22:52:53.460 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.? 172.21.15.191:0/1872963845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:52:53.460 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.? 172.21.15.204:0/4165979187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T22:52:53.460 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.? 172.21.15.204:0/4165979187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:52:53.460 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.? 172.21.15.204:0/4165979187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T22:52:53.460 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.? 172.21.15.204:0/4165979187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:52:53.460 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: Active manager daemon x restarted 2024-09-11T22:52:53.460 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: Activating manager daemon x 2024-09-11T22:52:53.460 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T22:52:53.461 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: mgrmap e20: x(active, starting, since 0.0126714s), standbys: z, y 2024-09-11T22:52:53.461 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/4232529076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T22:52:53.461 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:52:53.461 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:52:53.461 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:52:53.461 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T22:52:53.461 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T22:52:53.461 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:52:53.461 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:52:53.462 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:52:53.462 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:52:53.462 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:52:53.462 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:52:53.462 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:52:53.462 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:52:53.462 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:52:53.462 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:52:53.462 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T22:52:53.462 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T22:52:53.462 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T22:52:53.462 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:52:53.462 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:52:53.463 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:52:53.463 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:52:53.463 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:52:53.463 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:52:53.463 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:52:53.463 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:52:53.463 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:52:53.463 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:52:53.463 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:52:53.463 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:52:53.463 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T22:52:53.463 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T22:52:53.463 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T22:52:53.464 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:53 smithi191 ceph-mon[35211]: Manager daemon x is now available 2024-09-11T22:52:53.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.473+0000 7feb444c9640 1 -- 172.21.15.152:0/252207087 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feb3c062df0 con 0x7feb3c109cf0 2024-09-11T22:52:53.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.474+0000 7feb337fe640 1 -- 172.21.15.152:0/252207087 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7feb3409b5d0 con 0x7feb3c109cf0 2024-09-11T22:52:53.474 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-09-11T22:52:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.478+0000 7feb444c9640 1 -- 172.21.15.152:0/252207087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb3c109cf0 msgr2=0x7feb3c1ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.478+0000 7feb444c9640 1 --2- 172.21.15.152:0/252207087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb3c109cf0 0x7feb3c1ae290 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7feb3c06b9e0 tx=0x7feb3400b870 comp rx=0 tx=0).stop 2024-09-11T22:52:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.478+0000 7feb444c9640 1 -- 172.21.15.152:0/252207087 shutdown_connections 2024-09-11T22:52:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.478+0000 7feb444c9640 1 --2- 172.21.15.152:0/252207087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb3c109cf0 0x7feb3c1ae290 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.478+0000 7feb444c9640 1 --2- 172.21.15.152:0/252207087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb3c109330 0x7feb3c1a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.478+0000 7feb444c9640 1 --2- 172.21.15.152:0/252207087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb3c073440 0x7feb3c1a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.478+0000 7feb444c9640 1 -- 172.21.15.152:0/252207087 >> 172.21.15.152:0/252207087 conn(0x7feb3c06eb50 msgr2=0x7feb3c071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.478+0000 7feb444c9640 1 -- 172.21.15.152:0/252207087 shutdown_connections 2024-09-11T22:52:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:53.478+0000 7feb444c9640 1 -- 172.21.15.152:0/252207087 wait complete. 2024-09-11T22:52:53.495 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-11T22:52:53.495 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:52:53.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:53.323+0000 7f5edb2f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:52:53.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:53.323+0000 7f5edb2f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:52:53.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:53.323+0000 7f5edb2f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:52:53.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:53.323+0000 7f5edb2f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:52:53.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:53.323+0000 7f5edb2f1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:52:53.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:53.329+0000 7f5ede337640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:52:53.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:53.329+0000 7f5ede337640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:52:53.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:53.329+0000 7f5ede337640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:52:53.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:53.329+0000 7f5ede337640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:52:53.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:53.329+0000 7f5ede337640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:52:53.779 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:54.203 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:52:54.203 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T22:52:54.203 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T22:52:54.203 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/252207087' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T22:52:54.203 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-mon[31181]: mgrmap e21: x(active, since 1.02718s), standbys: z, y 2024-09-11T22:52:54.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.286+0000 7feb31b79640 1 -- 172.21.15.152:0/961944009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb2c106a90 msgr2=0x7feb2c106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:54.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.286+0000 7feb31b79640 1 --2- 172.21.15.152:0/961944009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb2c106a90 0x7feb2c106e70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009930 tx=0x7feb1c02a420 comp rx=0 tx=0).stop 2024-09-11T22:52:54.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.286+0000 7feb31b79640 1 -- 172.21.15.152:0/961944009 shutdown_connections 2024-09-11T22:52:54.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.286+0000 7feb31b79640 1 --2- 172.21.15.152:0/961944009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb2c101430 0x7feb2c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:54.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.286+0000 7feb31b79640 1 --2- 172.21.15.152:0/961944009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb2c100a90 0x7feb2c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:54.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.286+0000 7feb31b79640 1 --2- 172.21.15.152:0/961944009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb2c106a90 0x7feb2c106e70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:54.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.286+0000 7feb31b79640 1 -- 172.21.15.152:0/961944009 >> 172.21.15.152:0/961944009 conn(0x7feb2c0fc7d0 msgr2=0x7feb2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:54.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.286+0000 7feb31b79640 1 -- 172.21.15.152:0/961944009 shutdown_connections 2024-09-11T22:52:54.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.286+0000 7feb31b79640 1 -- 172.21.15.152:0/961944009 wait complete. 2024-09-11T22:52:54.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb31b79640 1 Processor -- start 2024-09-11T22:52:54.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb31b79640 1 -- start start 2024-09-11T22:52:54.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb31b79640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb2c100a90 0x7feb2c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:54.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb31b79640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb2c101430 0x7feb2c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:54.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb31b79640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb2c106a90 0x7feb2c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:54.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb31b79640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c110e90 con 0x7feb2c101430 2024-09-11T22:52:54.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb31b79640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c110d10 con 0x7feb2c100a90 2024-09-11T22:52:54.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb31b79640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c111010 con 0x7feb2c106a90 2024-09-11T22:52:54.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb23fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb2c101430 0x7feb2c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:54.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb23fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb2c101430 0x7feb2c19abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45470/0 (socket says 172.21.15.152:45470) 2024-09-11T22:52:54.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb23fff640 1 -- 172.21.15.152:0/1212876622 learned_addr learned my addr 172.21.15.152:0/1212876622 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:54.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb2b7fe640 1 --2- 172.21.15.152:0/1212876622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb2c100a90 0x7feb2c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:54.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb2bfff640 1 --2- 172.21.15.152:0/1212876622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb2c106a90 0x7feb2c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:54.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb23fff640 1 -- 172.21.15.152:0/1212876622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb2c106a90 msgr2=0x7feb2c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:54.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb23fff640 1 --2- 172.21.15.152:0/1212876622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb2c106a90 0x7feb2c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:54.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb23fff640 1 -- 172.21.15.152:0/1212876622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb2c100a90 msgr2=0x7feb2c19a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:54.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb23fff640 1 --2- 172.21.15.152:0/1212876622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb2c100a90 0x7feb2c19a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:54.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb23fff640 1 -- 172.21.15.152:0/1212876622 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c19f630 con 0x7feb2c101430 2024-09-11T22:52:54.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb23fff640 1 --2- 172.21.15.152:0/1212876622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb2c101430 0x7feb2c19abc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7feb1400e9f0 tx=0x7feb1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:54.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.287+0000 7feb297fa640 1 -- 172.21.15.152:0/1212876622 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1400cd80 con 0x7feb2c101430 2024-09-11T22:52:54.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.288+0000 7feb297fa640 1 -- 172.21.15.152:0/1212876622 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb14005500 con 0x7feb2c101430 2024-09-11T22:52:54.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.288+0000 7feb297fa640 1 -- 172.21.15.152:0/1212876622 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14010690 con 0x7feb2c101430 2024-09-11T22:52:54.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.288+0000 7feb31b79640 1 -- 172.21.15.152:0/1212876622 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2c19f920 con 0x7feb2c101430 2024-09-11T22:52:54.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.288+0000 7feb31b79640 1 -- 172.21.15.152:0/1212876622 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2c1a7200 con 0x7feb2c101430 2024-09-11T22:52:54.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.289+0000 7feb31b79640 1 -- 172.21.15.152:0/1212876622 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb2c100ef0 con 0x7feb2c101430 2024-09-11T22:52:54.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.290+0000 7feb297fa640 1 -- 172.21.15.152:0/1212876622 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7feb140046c0 con 0x7feb2c101430 2024-09-11T22:52:54.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.291+0000 7feb297fa640 1 --2- 172.21.15.152:0/1212876622 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7feb180a29f0 0x7feb180a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:54.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.291+0000 7feb2b7fe640 1 --2- 172.21.15.152:0/1212876622 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7feb180a29f0 0x7feb180a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:54.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.291+0000 7feb297fa640 1 -- 172.21.15.152:0/1212876622 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7feb140cf270 con 0x7feb2c101430 2024-09-11T22:52:54.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.291+0000 7feb2b7fe640 1 --2- 172.21.15.152:0/1212876622 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7feb180a29f0 0x7feb180a4eb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009900 tx=0x7feb1c0058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:54.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.294+0000 7feb297fa640 1 -- 172.21.15.152:0/1212876622 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feb140541b0 con 0x7feb2c101430 2024-09-11T22:52:54.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:54 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:52:54.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:54 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T22:52:54.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:54 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T22:52:54.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:54 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/252207087' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T22:52:54.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:54 smithi204 ceph-mon[35454]: mgrmap e21: x(active, since 1.02718s), standbys: z, y 2024-09-11T22:52:54.411 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:54 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:52:54.411 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:54 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T22:52:54.411 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:54 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T22:52:54.411 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:54 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/252207087' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T22:52:54.411 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:54 smithi191 ceph-mon[35211]: mgrmap e21: x(active, since 1.02718s), standbys: z, y 2024-09-11T22:52:54.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.520+0000 7feb31b79640 1 -- 172.21.15.152:0/1212876622 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb2c19b9e0 con 0x7feb2c101430 2024-09-11T22:52:54.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.521+0000 7feb297fa640 1 -- 172.21.15.152:0/1212876622 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7feb140cf090 con 0x7feb2c101430 2024-09-11T22:52:54.522 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:54.522 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:52:54.522 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:52:54.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.523+0000 7feb31b79640 1 -- 172.21.15.152:0/1212876622 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7feb180a29f0 msgr2=0x7feb180a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:54.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.523+0000 7feb31b79640 1 --2- 172.21.15.152:0/1212876622 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7feb180a29f0 0x7feb180a4eb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009900 tx=0x7feb1c0058d0 comp rx=0 tx=0).stop 2024-09-11T22:52:54.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.523+0000 7feb31b79640 1 -- 172.21.15.152:0/1212876622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb2c101430 msgr2=0x7feb2c19abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:54.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.523+0000 7feb31b79640 1 --2- 172.21.15.152:0/1212876622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb2c101430 0x7feb2c19abc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7feb1400e9f0 tx=0x7feb1400edf0 comp rx=0 tx=0).stop 2024-09-11T22:52:54.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.523+0000 7feb31b79640 1 -- 172.21.15.152:0/1212876622 shutdown_connections 2024-09-11T22:52:54.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.523+0000 7feb31b79640 1 --2- 172.21.15.152:0/1212876622 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7feb180a29f0 0x7feb180a4eb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:54.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.523+0000 7feb31b79640 1 --2- 172.21.15.152:0/1212876622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb2c106a90 0x7feb2c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:54.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.523+0000 7feb31b79640 1 --2- 172.21.15.152:0/1212876622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb2c101430 0x7feb2c19abc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:54.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.523+0000 7feb31b79640 1 --2- 172.21.15.152:0/1212876622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb2c100a90 0x7feb2c19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:54.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.523+0000 7feb31b79640 1 -- 172.21.15.152:0/1212876622 >> 172.21.15.152:0/1212876622 conn(0x7feb2c0fc7d0 msgr2=0x7feb2c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:54.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.524+0000 7feb31b79640 1 -- 172.21.15.152:0/1212876622 shutdown_connections 2024-09-11T22:52:54.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:54.524+0000 7feb31b79640 1 -- 172.21.15.152:0/1212876622 wait complete. 2024-09-11T22:52:54.840 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:54.749+0000 7f5ec7a0b640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:22:52:54] ENGINE Error in HTTPServer.serve 2024-09-11T22:52:54.840 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: Traceback (most recent call last): 2024-09-11T22:52:54.840 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:52:54.841 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: self._connections.run(self.expiration_interval) 2024-09-11T22:52:54.841 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:52:54.841 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: self._run(expiration_interval) 2024-09-11T22:52:54.841 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:52:54.841 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:52:54.841 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:52:54.841 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:52:54.841 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:52:54.841 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: s = self.context.wrap_socket( 2024-09-11T22:52:54.841 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:52:54.841 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: return self.sslsocket_class._create( 2024-09-11T22:52:54.841 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:52:54.841 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: self.do_handshake() 2024-09-11T22:52:54.842 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:52:54.842 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: self._sslobj.do_handshake() 2024-09-11T22:52:54.842 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:52:54.842 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:52:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:52:55.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:55 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1212876622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:55.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:55 smithi152 ceph-mon[31181]: [11/Sep/2024:22:52:54] ENGINE Bus STARTING 2024-09-11T22:52:55.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:55 smithi152 ceph-mon[31181]: [11/Sep/2024:22:52:54] ENGINE Serving on https://172.21.15.152:7150 2024-09-11T22:52:55.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:55 smithi152 ceph-mon[31181]: [11/Sep/2024:22:52:54] ENGINE Error in HTTPServer.serve 2024-09-11T22:52:55.373 INFO:journalctl@ceph.mon.a.smithi152.stdout: Traceback (most recent call last): 2024-09-11T22:52:55.374 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:52:55.374 INFO:journalctl@ceph.mon.a.smithi152.stdout: self._connections.run(self.expiration_interval) 2024-09-11T22:52:55.374 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:52:55.374 INFO:journalctl@ceph.mon.a.smithi152.stdout: self._run(expiration_interval) 2024-09-11T22:52:55.374 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:52:55.374 INFO:journalctl@ceph.mon.a.smithi152.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:52:55.374 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:52:55.374 INFO:journalctl@ceph.mon.a.smithi152.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:52:55.374 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:52:55.374 INFO:journalctl@ceph.mon.a.smithi152.stdout: s = self.context.wrap_socket( 2024-09-11T22:52:55.374 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:52:55.374 INFO:journalctl@ceph.mon.a.smithi152.stdout: return self.sslsocket_class._create( 2024-09-11T22:52:55.374 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:52:55.375 INFO:journalctl@ceph.mon.a.smithi152.stdout: self.do_handshake() 2024-09-11T22:52:55.375 INFO:journalctl@ceph.mon.a.smithi152.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:52:55.375 INFO:journalctl@ceph.mon.a.smithi152.stdout: self._sslobj.do_handshake() 2024-09-11T22:52:55.375 INFO:journalctl@ceph.mon.a.smithi152.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:52:55.375 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:55 smithi152 ceph-mon[31181]: [11/Sep/2024:22:52:54] ENGINE Serving on http://172.21.15.152:8765 2024-09-11T22:52:55.375 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:55 smithi152 ceph-mon[31181]: [11/Sep/2024:22:52:54] ENGINE Bus STARTED 2024-09-11T22:52:55.375 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:55 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:55.375 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:55 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:55.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:55 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1212876622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:55.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:55 smithi191 ceph-mon[35211]: [11/Sep/2024:22:52:54] ENGINE Bus STARTING 2024-09-11T22:52:55.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:55 smithi191 ceph-mon[35211]: [11/Sep/2024:22:52:54] ENGINE Serving on https://172.21.15.152:7150 2024-09-11T22:52:55.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:55 smithi191 ceph-mon[35211]: [11/Sep/2024:22:52:54] ENGINE Error in HTTPServer.serve 2024-09-11T22:52:55.537 INFO:journalctl@ceph.mon.b.smithi191.stdout: Traceback (most recent call last): 2024-09-11T22:52:55.537 INFO:journalctl@ceph.mon.b.smithi191.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:52:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout: self._connections.run(self.expiration_interval) 2024-09-11T22:52:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:52:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout: self._run(expiration_interval) 2024-09-11T22:52:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:52:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:52:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:52:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:52:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:52:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout: s = self.context.wrap_socket( 2024-09-11T22:52:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:52:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout: return self.sslsocket_class._create( 2024-09-11T22:52:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:52:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout: self.do_handshake() 2024-09-11T22:52:55.538 INFO:journalctl@ceph.mon.b.smithi191.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:52:55.539 INFO:journalctl@ceph.mon.b.smithi191.stdout: self._sslobj.do_handshake() 2024-09-11T22:52:55.539 INFO:journalctl@ceph.mon.b.smithi191.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:52:55.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:55 smithi191 ceph-mon[35211]: [11/Sep/2024:22:52:54] ENGINE Serving on http://172.21.15.152:8765 2024-09-11T22:52:55.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:55 smithi191 ceph-mon[35211]: [11/Sep/2024:22:52:54] ENGINE Bus STARTED 2024-09-11T22:52:55.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:55 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:55.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:55 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:55.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:55 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1212876622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:55.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:55 smithi204 ceph-mon[35454]: [11/Sep/2024:22:52:54] ENGINE Bus STARTING 2024-09-11T22:52:55.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:55 smithi204 ceph-mon[35454]: [11/Sep/2024:22:52:54] ENGINE Serving on https://172.21.15.152:7150 2024-09-11T22:52:55.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:55 smithi204 ceph-mon[35454]: [11/Sep/2024:22:52:54] ENGINE Error in HTTPServer.serve 2024-09-11T22:52:55.546 INFO:journalctl@ceph.mon.c.smithi204.stdout: Traceback (most recent call last): 2024-09-11T22:52:55.546 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:52:55.546 INFO:journalctl@ceph.mon.c.smithi204.stdout: self._connections.run(self.expiration_interval) 2024-09-11T22:52:55.546 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:52:55.546 INFO:journalctl@ceph.mon.c.smithi204.stdout: self._run(expiration_interval) 2024-09-11T22:52:55.547 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:52:55.547 INFO:journalctl@ceph.mon.c.smithi204.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:52:55.547 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:52:55.547 INFO:journalctl@ceph.mon.c.smithi204.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:52:55.547 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:52:55.547 INFO:journalctl@ceph.mon.c.smithi204.stdout: s = self.context.wrap_socket( 2024-09-11T22:52:55.547 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:52:55.547 INFO:journalctl@ceph.mon.c.smithi204.stdout: return self.sslsocket_class._create( 2024-09-11T22:52:55.547 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:52:55.547 INFO:journalctl@ceph.mon.c.smithi204.stdout: self.do_handshake() 2024-09-11T22:52:55.547 INFO:journalctl@ceph.mon.c.smithi204.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:52:55.547 INFO:journalctl@ceph.mon.c.smithi204.stdout: self._sslobj.do_handshake() 2024-09-11T22:52:55.547 INFO:journalctl@ceph.mon.c.smithi204.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:52:55.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:55 smithi204 ceph-mon[35454]: [11/Sep/2024:22:52:54] ENGINE Serving on http://172.21.15.152:8765 2024-09-11T22:52:55.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:55 smithi204 ceph-mon[35454]: [11/Sep/2024:22:52:54] ENGINE Bus STARTED 2024-09-11T22:52:55.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:55 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:55.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:55 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.036 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-11T22:52:56.036 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-11T22:52:56.036 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T22:52:56.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.148+0000 7f4bbfd9f640 1 -- 172.21.15.152:0/203747019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bb8101c20 msgr2=0x7f4bb81020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:56.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.148+0000 7f4bbfd9f640 1 --2- 172.21.15.152:0/203747019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bb8101c20 0x7f4bb81020a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8009860 tx=0x7f4ba802a340 comp rx=0 tx=0).stop 2024-09-11T22:52:56.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.148+0000 7f4bbfd9f640 1 -- 172.21.15.152:0/203747019 shutdown_connections 2024-09-11T22:52:56.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.148+0000 7f4bbfd9f640 1 --2- 172.21.15.152:0/203747019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bb81025e0 0x7f4bb8108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.148+0000 7f4bbfd9f640 1 --2- 172.21.15.152:0/203747019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bb8101c20 0x7f4bb81020a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.148+0000 7f4bbfd9f640 1 --2- 172.21.15.152:0/203747019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bb8100a30 0x7f4bb8100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.148+0000 7f4bbfd9f640 1 -- 172.21.15.152:0/203747019 >> 172.21.15.152:0/203747019 conn(0x7f4bb80fc1b0 msgr2=0x7f4bb80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:56.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.149+0000 7f4bbfd9f640 1 -- 172.21.15.152:0/203747019 shutdown_connections 2024-09-11T22:52:56.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.149+0000 7f4bbfd9f640 1 -- 172.21.15.152:0/203747019 wait complete. 2024-09-11T22:52:56.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.149+0000 7f4bbfd9f640 1 Processor -- start 2024-09-11T22:52:56.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.149+0000 7f4bbfd9f640 1 -- start start 2024-09-11T22:52:56.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbfd9f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bb8100a30 0x7f4bb819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:56.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbfd9f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bb8101c20 0x7f4bb819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:56.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbfd9f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bb81025e0 0x7f4bb81a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:56.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbfd9f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb810c6b0 con 0x7f4bb8101c20 2024-09-11T22:52:56.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbfd9f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb810c530 con 0x7f4bb81025e0 2024-09-11T22:52:56.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbfd9f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb810c830 con 0x7f4bb8100a30 2024-09-11T22:52:56.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbd33a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bb8101c20 0x7f4bb819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:56.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbdb3b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bb8100a30 0x7f4bb819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:56.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbd33a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bb8101c20 0x7f4bb819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45498/0 (socket says 172.21.15.152:45498) 2024-09-11T22:52:56.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbdb3b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bb8100a30 0x7f4bb819a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:51806/0 (socket says 172.21.15.152:51806) 2024-09-11T22:52:56.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbd33a640 1 -- 172.21.15.152:0/2394236766 learned_addr learned my addr 172.21.15.152:0/2394236766 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:56.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbe33c640 1 --2- 172.21.15.152:0/2394236766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bb81025e0 0x7f4bb81a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:56.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbdb3b640 1 -- 172.21.15.152:0/2394236766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bb81025e0 msgr2=0x7f4bb81a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:56.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbdb3b640 1 --2- 172.21.15.152:0/2394236766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bb81025e0 0x7f4bb81a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbdb3b640 1 -- 172.21.15.152:0/2394236766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bb8101c20 msgr2=0x7f4bb819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:56.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbdb3b640 1 --2- 172.21.15.152:0/2394236766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bb8101c20 0x7f4bb819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbdb3b640 1 -- 172.21.15.152:0/2394236766 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb81a1cd0 con 0x7f4bb8100a30 2024-09-11T22:52:56.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4bbdb3b640 1 --2- 172.21.15.152:0/2394236766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bb8100a30 0x7f4bb819a0b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400e9c0 tx=0x7f4bb400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:56.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4baeffd640 1 -- 172.21.15.152:0/2394236766 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb400ccc0 con 0x7f4bb8100a30 2024-09-11T22:52:56.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.150+0000 7f4baeffd640 1 -- 172.21.15.152:0/2394236766 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bb40035f0 con 0x7f4bb8100a30 2024-09-11T22:52:56.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.151+0000 7f4bbfd9f640 1 -- 172.21.15.152:0/2394236766 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb81a1fc0 con 0x7f4bb8100a30 2024-09-11T22:52:56.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.151+0000 7f4baeffd640 1 -- 172.21.15.152:0/2394236766 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb400ce60 con 0x7f4bb8100a30 2024-09-11T22:52:56.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.151+0000 7f4bbfd9f640 1 -- 172.21.15.152:0/2394236766 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb81a2420 con 0x7f4bb8100a30 2024-09-11T22:52:56.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.152+0000 7f4bbfd9f640 1 -- 172.21.15.152:0/2394236766 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb81a26b0 con 0x7f4bb8100a30 2024-09-11T22:52:56.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.152+0000 7f4baeffd640 1 -- 172.21.15.152:0/2394236766 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 22) ==== 137954+0+0 (secure 0 0 0) 0x7f4bb4004030 con 0x7f4bb8100a30 2024-09-11T22:52:56.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.153+0000 7f4baeffd640 1 --2- 172.21.15.152:0/2394236766 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4ba40a2d80 0x7f4ba40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:56.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.153+0000 7f4baeffd640 1 -- 172.21.15.152:0/2394236766 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4bb40cf8a0 con 0x7f4bb8100a30 2024-09-11T22:52:56.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.153+0000 7f4bbd33a640 1 --2- 172.21.15.152:0/2394236766 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4ba40a2d80 0x7f4ba40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:56.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.154+0000 7f4bbd33a640 1 --2- 172.21.15.152:0/2394236766 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4ba40a2d80 0x7f4ba40a5240 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ba802a930 tx=0x7f4ba80058b0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:56.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.157+0000 7f4baeffd640 1 -- 172.21.15.152:0/2394236766 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4bb409c510 con 0x7f4bb8100a30 2024-09-11T22:52:56.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.350+0000 7f4bbfd9f640 1 -- 172.21.15.152:0/2394236766 --> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4bb81a2880 con 0x7f4ba40a2d80 2024-09-11T22:52:56.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.357+0000 7f4baeffd640 1 -- 172.21.15.152:0/2394236766 <== mgr.15240 v2:172.21.15.152:6800/4130474708 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258049 (secure 0 0 0) 0x7f4bb81a2880 con 0x7f4ba40a2d80 2024-09-11T22:52:56.360 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:56.362 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-09-11T22:52:56.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.360+0000 7f4bbfd9f640 1 -- 172.21.15.152:0/2394236766 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4ba40a2d80 msgr2=0x7f4ba40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:56.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.360+0000 7f4bbfd9f640 1 --2- 172.21.15.152:0/2394236766 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4ba40a2d80 0x7f4ba40a5240 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ba802a930 tx=0x7f4ba80058b0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.360+0000 7f4bbfd9f640 1 -- 172.21.15.152:0/2394236766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bb8100a30 msgr2=0x7f4bb819a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:56.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.360+0000 7f4bbfd9f640 1 --2- 172.21.15.152:0/2394236766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bb8100a30 0x7f4bb819a0b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400e9c0 tx=0x7f4bb400ee80 comp rx=0 tx=0).stop 2024-09-11T22:52:56.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.360+0000 7f4bbfd9f640 1 -- 172.21.15.152:0/2394236766 shutdown_connections 2024-09-11T22:52:56.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.360+0000 7f4bbfd9f640 1 --2- 172.21.15.152:0/2394236766 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4ba40a2d80 0x7f4ba40a5240 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.360+0000 7f4bbfd9f640 1 --2- 172.21.15.152:0/2394236766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4bb81025e0 0x7f4bb81a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.360+0000 7f4bbfd9f640 1 --2- 172.21.15.152:0/2394236766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4bb8101c20 0x7f4bb819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.360+0000 7f4bbfd9f640 1 --2- 172.21.15.152:0/2394236766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bb8100a30 0x7f4bb819a0b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.360+0000 7f4bbfd9f640 1 -- 172.21.15.152:0/2394236766 >> 172.21.15.152:0/2394236766 conn(0x7f4bb80fc1b0 msgr2=0x7f4bb806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:56.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.360+0000 7f4bbfd9f640 1 -- 172.21.15.152:0/2394236766 shutdown_connections 2024-09-11T22:52:56.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.360+0000 7f4bbfd9f640 1 -- 172.21.15.152:0/2394236766 wait complete. 2024-09-11T22:52:56.371 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:52:56.371 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-11T22:52:56.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi191 to 176.0M 2024-09-11T22:52:56.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: Unable to set osd_memory_target on smithi191 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-11T22:52:56.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-11T22:52:56.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:56 smithi152 ceph-mon[31181]: Unable to set osd_memory_target on smithi204 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-11T22:52:56.374 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-11T22:52:55.086644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193417,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1363,"num_read_kb":848,"num_write":561,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":944,"ondisk_log_size":944,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369080,"kb_used_data":11472,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572744,"statfs":{"total":1151940427776,"available":1151562489856,"internally_reserved":0,"allocated":11747328,"data_stored":8912731,"data_compressed":215322,"data_compressed_allocated":278528,"data_compressed_original":3407872,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.972893"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169130+0000","last_change":"2024-09-11T22:50:54.171410+0000","last_active":"2024-09-11T22:52:53.169130+0000","last_peered":"2024-09-11T22:52:53.169130+0000","last_clean":"2024-09-11T22:52:53.169130+0000","last_became_active":"2024-09-11T22:50:54.171055+0000","last_became_peered":"2024-09-11T22:50:54.171055+0000","last_unstale":"2024-09-11T22:52:53.169130+0000","last_undegraded":"2024-09-11T22:52:53.169130+0000","last_fullsized":"2024-09-11T22:52:53.169130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:54.767454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131329+0000","last_change":"2024-09-11T22:50:54.168534+0000","last_active":"2024-09-11T22:52:53.131329+0000","last_peered":"2024-09-11T22:52:53.131329+0000","last_clean":"2024-09-11T22:52:53.131329+0000","last_became_active":"2024-09-11T22:50:54.168228+0000","last_became_peered":"2024-09-11T22:50:54.168228+0000","last_unstale":"2024-09-11T22:52:53.131329+0000","last_undegraded":"2024-09-11T22:52:53.131329+0000","last_fullsized":"2024-09-11T22:52:53.131329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:11.380048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131130+0000","last_change":"2024-09-11T22:50:54.168975+0000","last_active":"2024-09-11T22:52:53.131130+0000","last_peered":"2024-09-11T22:52:53.131130+0000","last_clean":"2024-09-11T22:52:53.131130+0000","last_became_active":"2024-09-11T22:50:54.168676+0000","last_became_peered":"2024-09-11T22:50:54.168676+0000","last_unstale":"2024-09-11T22:52:53.131130+0000","last_undegraded":"2024-09-11T22:52:53.131130+0000","last_fullsized":"2024-09-11T22:52:53.131130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:15.848273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130051+0000","last_change":"2024-09-11T22:50:54.164164+0000","last_active":"2024-09-11T22:52:53.130051+0000","last_peered":"2024-09-11T22:52:53.130051+0000","last_clean":"2024-09-11T22:52:53.130051+0000","last_became_active":"2024-09-11T22:50:54.163805+0000","last_became_peered":"2024-09-11T22:50:54.163805+0000","last_unstale":"2024-09-11T22:52:53.130051+0000","last_undegraded":"2024-09-11T22:52:53.130051+0000","last_fullsized":"2024-09-11T22:52:53.130051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:54.954046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074671+0000","last_change":"2024-09-11T22:50:54.167289+0000","last_active":"2024-09-11T22:52:53.074671+0000","last_peered":"2024-09-11T22:52:53.074671+0000","last_clean":"2024-09-11T22:52:53.074671+0000","last_became_active":"2024-09-11T22:50:54.166958+0000","last_became_peered":"2024-09-11T22:50:54.166958+0000","last_unstale":"2024-09-11T22:52:53.074671+0000","last_undegraded":"2024-09-11T22:52:53.074671+0000","last_fullsized":"2024-09-11T22:52:53.074671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:25.261241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129676+0000","last_change":"2024-09-11T22:50:54.175605+0000","last_active":"2024-09-11T22:52:53.129676+0000","last_peered":"2024-09-11T22:52:53.129676+0000","last_clean":"2024-09-11T22:52:53.129676+0000","last_became_active":"2024-09-11T22:50:54.175207+0000","last_became_peered":"2024-09-11T22:50:54.175207+0000","last_unstale":"2024-09-11T22:52:53.129676+0000","last_undegraded":"2024-09-11T22:52:53.129676+0000","last_fullsized":"2024-09-11T22:52:53.129676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:47.008910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129084+0000","last_change":"2024-09-11T22:50:54.167930+0000","last_active":"2024-09-11T22:52:53.129084+0000","last_peered":"2024-09-11T22:52:53.129084+0000","last_clean":"2024-09-11T22:52:53.129084+0000","last_became_active":"2024-09-11T22:50:54.167082+0000","last_became_peered":"2024-09-11T22:50:54.167082+0000","last_unstale":"2024-09-11T22:52:53.129084+0000","last_undegraded":"2024-09-11T22:52:53.129084+0000","last_fullsized":"2024-09-11T22:52:53.129084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:50.931311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129336+0000","last_change":"2024-09-11T22:50:54.172528+0000","last_active":"2024-09-11T22:52:53.129336+0000","last_peered":"2024-09-11T22:52:53.129336+0000","last_clean":"2024-09-11T22:52:53.129336+0000","last_became_active":"2024-09-11T22:50:54.172257+0000","last_became_peered":"2024-09-11T22:50:54.172257+0000","last_unstale":"2024-09-11T22:52:53.129336+0000","last_undegraded":"2024-09-11T22:52:53.129336+0000","last_fullsized":"2024-09-11T22:52:53.129336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:13:43.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074665+0000","last_change":"2024-09-11T22:50:54.163357+0000","last_active":"2024-09-11T22:52:53.074665+0000","last_peered":"2024-09-11T22:52:53.074665+0000","last_clean":"2024-09-11T22:52:53.074665+0000","last_became_active":"2024-09-11T22:50:54.162943+0000","last_became_peered":"2024-09-11T22:50:54.162943+0000","last_unstale":"2024-09-11T22:52:53.074665+0000","last_undegraded":"2024-09-11T22:52:53.074665+0000","last_fullsized":"2024-09-11T22:52:53.074665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:57.055802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129448+0000","last_change":"2024-09-11T22:50:54.177773+0000","last_active":"2024-09-11T22:52:53.129448+0000","last_peered":"2024-09-11T22:52:53.129448+0000","last_clean":"2024-09-11T22:52:53.129448+0000","last_became_active":"2024-09-11T22:50:54.177388+0000","last_became_peered":"2024-09-11T22:50:54.177388+0000","last_unstale":"2024-09-11T22:52:53.129448+0000","last_undegraded":"2024-09-11T22:52:53.129448+0000","last_fullsized":"2024-09-11T22:52:53.129448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:36:44.625089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129792+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:52:53.129792+0000","last_peered":"2024-09-11T22:52:53.129792+0000","last_clean":"2024-09-11T22:52:53.129792+0000","last_became_active":"2024-09-11T22:50:54.171491+0000","last_became_peered":"2024-09-11T22:50:54.171491+0000","last_unstale":"2024-09-11T22:52:53.129792+0000","last_undegraded":"2024-09-11T22:52:53.129792+0000","last_fullsized":"2024-09-11T22:52:53.129792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.996640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130199+0000","last_change":"2024-09-11T22:50:54.171260+0000","last_active":"2024-09-11T22:52:53.130199+0000","last_peered":"2024-09-11T22:52:53.130199+0000","last_clean":"2024-09-11T22:52:53.130199+0000","last_became_active":"2024-09-11T22:50:54.170733+0000","last_became_peered":"2024-09-11T22:50:54.170733+0000","last_unstale":"2024-09-11T22:52:53.130199+0000","last_undegraded":"2024-09-11T22:52:53.130199+0000","last_fullsized":"2024-09-11T22:52:53.130199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:57.877280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129997+0000","last_change":"2024-09-11T22:50:54.170576+0000","last_active":"2024-09-11T22:52:53.129997+0000","last_peered":"2024-09-11T22:52:53.129997+0000","last_clean":"2024-09-11T22:52:53.129997+0000","last_became_active":"2024-09-11T22:50:54.170239+0000","last_became_peered":"2024-09-11T22:50:54.170239+0000","last_unstale":"2024-09-11T22:52:53.129997+0000","last_undegraded":"2024-09-11T22:52:53.129997+0000","last_fullsized":"2024-09-11T22:52:53.129997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:09.683171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130651+0000","last_change":"2024-09-11T22:50:54.167176+0000","last_active":"2024-09-11T22:52:53.130651+0000","last_peered":"2024-09-11T22:52:53.130651+0000","last_clean":"2024-09-11T22:52:53.130651+0000","last_became_active":"2024-09-11T22:50:54.166894+0000","last_became_peered":"2024-09-11T22:50:54.166894+0000","last_unstale":"2024-09-11T22:52:53.130651+0000","last_undegraded":"2024-09-11T22:52:53.130651+0000","last_fullsized":"2024-09-11T22:52:53.130651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:43:08.158786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130901+0000","last_change":"2024-09-11T22:50:54.163100+0000","last_active":"2024-09-11T22:52:53.130901+0000","last_peered":"2024-09-11T22:52:53.130901+0000","last_clean":"2024-09-11T22:52:53.130901+0000","last_became_active":"2024-09-11T22:50:54.162776+0000","last_became_peered":"2024-09-11T22:50:54.162776+0000","last_unstale":"2024-09-11T22:52:53.130901+0000","last_undegraded":"2024-09-11T22:52:53.130901+0000","last_fullsized":"2024-09-11T22:52:53.130901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:15.215305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074034+0000","last_change":"2024-09-11T22:50:54.166013+0000","last_active":"2024-09-11T22:52:53.074034+0000","last_peered":"2024-09-11T22:52:53.074034+0000","last_clean":"2024-09-11T22:52:53.074034+0000","last_became_active":"2024-09-11T22:50:54.165680+0000","last_became_peered":"2024-09-11T22:50:54.165680+0000","last_unstale":"2024-09-11T22:52:53.074034+0000","last_undegraded":"2024-09-11T22:52:53.074034+0000","last_fullsized":"2024-09-11T22:52:53.074034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:12.192231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169175+0000","last_change":"2024-09-11T22:50:54.172105+0000","last_active":"2024-09-11T22:52:53.169175+0000","last_peered":"2024-09-11T22:52:53.169175+0000","last_clean":"2024-09-11T22:52:53.169175+0000","last_became_active":"2024-09-11T22:50:54.171782+0000","last_became_peered":"2024-09-11T22:50:54.171782+0000","last_unstale":"2024-09-11T22:52:53.169175+0000","last_undegraded":"2024-09-11T22:52:53.169175+0000","last_fullsized":"2024-09-11T22:52:53.169175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:17.152834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130880+0000","last_change":"2024-09-11T22:50:54.167184+0000","last_active":"2024-09-11T22:52:53.130880+0000","last_peered":"2024-09-11T22:52:53.130880+0000","last_clean":"2024-09-11T22:52:53.130880+0000","last_became_active":"2024-09-11T22:50:54.166878+0000","last_became_peered":"2024-09-11T22:50:54.166878+0000","last_unstale":"2024-09-11T22:52:53.130880+0000","last_undegraded":"2024-09-11T22:52:53.130880+0000","last_fullsized":"2024-09-11T22:52:53.130880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:44.046552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129108+0000","last_change":"2024-09-11T22:50:54.176260+0000","last_active":"2024-09-11T22:52:53.129108+0000","last_peered":"2024-09-11T22:52:53.129108+0000","last_clean":"2024-09-11T22:52:53.129108+0000","last_became_active":"2024-09-11T22:50:54.175873+0000","last_became_peered":"2024-09-11T22:50:54.175873+0000","last_unstale":"2024-09-11T22:52:53.129108+0000","last_undegraded":"2024-09-11T22:52:53.129108+0000","last_fullsized":"2024-09-11T22:52:53.129108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:05.258677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168910+0000","last_change":"2024-09-11T22:50:54.172725+0000","last_active":"2024-09-11T22:52:53.168910+0000","last_peered":"2024-09-11T22:52:53.168910+0000","last_clean":"2024-09-11T22:52:53.168910+0000","last_became_active":"2024-09-11T22:50:54.172072+0000","last_became_peered":"2024-09-11T22:50:54.172072+0000","last_unstale":"2024-09-11T22:52:53.168910+0000","last_undegraded":"2024-09-11T22:52:53.168910+0000","last_fullsized":"2024-09-11T22:52:53.168910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:53.699670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130148+0000","last_change":"2024-09-11T22:50:54.165118+0000","last_active":"2024-09-11T22:52:53.130148+0000","last_peered":"2024-09-11T22:52:53.130148+0000","last_clean":"2024-09-11T22:52:53.130148+0000","last_became_active":"2024-09-11T22:50:54.164793+0000","last_became_peered":"2024-09-11T22:50:54.164793+0000","last_unstale":"2024-09-11T22:52:53.130148+0000","last_undegraded":"2024-09-11T22:52:53.130148+0000","last_fullsized":"2024-09-11T22:52:53.130148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:42.883940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074224+0000","last_change":"2024-09-11T22:50:54.176900+0000","last_active":"2024-09-11T22:52:53.074224+0000","last_peered":"2024-09-11T22:52:53.074224+0000","last_clean":"2024-09-11T22:52:53.074224+0000","last_became_active":"2024-09-11T22:50:54.176595+0000","last_became_peered":"2024-09-11T22:50:54.176595+0000","last_unstale":"2024-09-11T22:52:53.074224+0000","last_undegraded":"2024-09-11T22:52:53.074224+0000","last_fullsized":"2024-09-11T22:52:53.074224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:36.925945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129485+0000","last_change":"2024-09-11T22:50:54.167656+0000","last_active":"2024-09-11T22:52:53.129485+0000","last_peered":"2024-09-11T22:52:53.129485+0000","last_clean":"2024-09-11T22:52:53.129485+0000","last_became_active":"2024-09-11T22:50:54.167349+0000","last_became_peered":"2024-09-11T22:50:54.167349+0000","last_unstale":"2024-09-11T22:52:53.129485+0000","last_undegraded":"2024-09-11T22:52:53.129485+0000","last_fullsized":"2024-09-11T22:52:53.129485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:38.100218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073470+0000","last_change":"2024-09-11T22:50:54.177059+0000","last_active":"2024-09-11T22:52:53.073470+0000","last_peered":"2024-09-11T22:52:53.073470+0000","last_clean":"2024-09-11T22:52:53.073470+0000","last_became_active":"2024-09-11T22:50:54.176726+0000","last_became_peered":"2024-09-11T22:50:54.176726+0000","last_unstale":"2024-09-11T22:52:53.073470+0000","last_undegraded":"2024-09-11T22:52:53.073470+0000","last_fullsized":"2024-09-11T22:52:53.073470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:27:58.058944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130570+0000","last_change":"2024-09-11T22:50:54.167566+0000","last_active":"2024-09-11T22:52:53.130570+0000","last_peered":"2024-09-11T22:52:53.130570+0000","last_clean":"2024-09-11T22:52:53.130570+0000","last_became_active":"2024-09-11T22:50:54.167049+0000","last_became_peered":"2024-09-11T22:50:54.167049+0000","last_unstale":"2024-09-11T22:52:53.130570+0000","last_undegraded":"2024-09-11T22:52:53.130570+0000","last_fullsized":"2024-09-11T22:52:53.130570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:06.833099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129193+0000","last_change":"2024-09-11T22:50:54.169732+0000","last_active":"2024-09-11T22:52:53.129193+0000","last_peered":"2024-09-11T22:52:53.129193+0000","last_clean":"2024-09-11T22:52:53.129193+0000","last_became_active":"2024-09-11T22:50:54.169437+0000","last_became_peered":"2024-09-11T22:50:54.169437+0000","last_unstale":"2024-09-11T22:52:53.129193+0000","last_undegraded":"2024-09-11T22:52:53.129193+0000","last_fullsized":"2024-09-11T22:52:53.129193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:25.553387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169106+0000","last_change":"2024-09-11T22:50:54.166953+0000","last_active":"2024-09-11T22:52:53.169106+0000","last_peered":"2024-09-11T22:52:53.169106+0000","last_clean":"2024-09-11T22:52:53.169106+0000","last_became_active":"2024-09-11T22:50:54.166472+0000","last_became_peered":"2024-09-11T22:50:54.166472+0000","last_unstale":"2024-09-11T22:52:53.169106+0000","last_undegraded":"2024-09-11T22:52:53.169106+0000","last_fullsized":"2024-09-11T22:52:53.169106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:01:20.234180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129423+0000","last_change":"2024-09-11T22:50:54.171308+0000","last_active":"2024-09-11T22:52:53.129423+0000","last_peered":"2024-09-11T22:52:53.129423+0000","last_clean":"2024-09-11T22:52:53.129423+0000","last_became_active":"2024-09-11T22:50:54.168958+0000","last_became_peered":"2024-09-11T22:50:54.168958+0000","last_unstale":"2024-09-11T22:52:53.129423+0000","last_undegraded":"2024-09-11T22:52:53.129423+0000","last_fullsized":"2024-09-11T22:52:53.129423+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:20.224003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131183+0000","last_change":"2024-09-11T22:50:54.173846+0000","last_active":"2024-09-11T22:52:53.131183+0000","last_peered":"2024-09-11T22:52:53.131183+0000","last_clean":"2024-09-11T22:52:53.131183+0000","last_became_active":"2024-09-11T22:50:54.173542+0000","last_became_peered":"2024-09-11T22:50:54.173542+0000","last_unstale":"2024-09-11T22:52:53.131183+0000","last_undegraded":"2024-09-11T22:52:53.131183+0000","last_fullsized":"2024-09-11T22:52:53.131183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:29:45.775902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.128929+0000","last_change":"2024-09-11T22:50:54.176008+0000","last_active":"2024-09-11T22:52:53.128929+0000","last_peered":"2024-09-11T22:52:53.128929+0000","last_clean":"2024-09-11T22:52:53.128929+0000","last_became_active":"2024-09-11T22:50:54.175672+0000","last_became_peered":"2024-09-11T22:50:54.175672+0000","last_unstale":"2024-09-11T22:52:53.128929+0000","last_undegraded":"2024-09-11T22:52:53.128929+0000","last_fullsized":"2024-09-11T22:52:53.128929+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:35.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.128953+0000","last_change":"2024-09-11T22:50:54.162504+0000","last_active":"2024-09-11T22:52:53.128953+0000","last_peered":"2024-09-11T22:52:53.128953+0000","last_clean":"2024-09-11T22:52:53.128953+0000","last_became_active":"2024-09-11T22:50:54.162190+0000","last_became_peered":"2024-09-11T22:50:54.162190+0000","last_unstale":"2024-09-11T22:52:53.128953+0000","last_undegraded":"2024-09-11T22:52:53.128953+0000","last_fullsized":"2024-09-11T22:52:53.128953+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:52.138692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168828+0000","last_change":"2024-09-11T22:51:55.960882+0000","last_active":"2024-09-11T22:52:53.168828+0000","last_peered":"2024-09-11T22:52:53.168828+0000","last_clean":"2024-09-11T22:52:53.168828+0000","last_became_active":"2024-09-11T22:51:55.960246+0000","last_became_peered":"2024-09-11T22:51:55.960246+0000","last_unstale":"2024-09-11T22:52:53.168828+0000","last_undegraded":"2024-09-11T22:52:53.168828+0000","last_fullsized":"2024-09-11T22:52:53.168828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:21.247840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074402+0000","last_change":"2024-09-11T22:50:54.160696+0000","last_active":"2024-09-11T22:52:53.074402+0000","last_peered":"2024-09-11T22:52:53.074402+0000","last_clean":"2024-09-11T22:52:53.074402+0000","last_became_active":"2024-09-11T22:50:54.159249+0000","last_became_peered":"2024-09-11T22:50:54.159249+0000","last_unstale":"2024-09-11T22:52:53.074402+0000","last_undegraded":"2024-09-11T22:52:53.074402+0000","last_fullsized":"2024-09-11T22:52:53.074402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:09.649208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'9","reported_seq":146,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.331651+0000","last_change":"2024-09-11T22:50:52.138986+0000","last_active":"2024-09-11T22:52:53.331651+0000","last_peered":"2024-09-11T22:52:53.331651+0000","last_clean":"2024-09-11T22:52:53.331651+0000","last_became_active":"2024-09-11T22:50:52.138595+0000","last_became_peered":"2024-09-11T22:50:52.138595+0000","last_unstale":"2024-09-11T22:52:53.331651+0000","last_undegraded":"2024-09-11T22:52:53.331651+0000","last_fullsized":"2024-09-11T22:52:53.331651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:06.232146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1942,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073853+0000","last_change":"2024-09-11T22:50:54.176806+0000","last_active":"2024-09-11T22:52:53.073853+0000","last_peered":"2024-09-11T22:52:53.073853+0000","last_clean":"2024-09-11T22:52:53.073853+0000","last_became_active":"2024-09-11T22:50:54.176513+0000","last_became_peered":"2024-09-11T22:50:54.176513+0000","last_unstale":"2024-09-11T22:52:53.073853+0000","last_undegraded":"2024-09-11T22:52:53.073853+0000","last_fullsized":"2024-09-11T22:52:53.073853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:43:12.086145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129699+0000","last_change":"2024-09-11T22:50:52.142794+0000","last_active":"2024-09-11T22:52:53.129699+0000","last_peered":"2024-09-11T22:52:53.129699+0000","last_clean":"2024-09-11T22:52:53.129699+0000","last_became_active":"2024-09-11T22:50:52.142433+0000","last_became_peered":"2024-09-11T22:50:52.142433+0000","last_unstale":"2024-09-11T22:52:53.129699+0000","last_undegraded":"2024-09-11T22:52:53.129699+0000","last_fullsized":"2024-09-11T22:52:53.129699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:47:12.090055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130326+0000","last_change":"2024-09-11T22:50:52.139789+0000","last_active":"2024-09-11T22:52:53.130326+0000","last_peered":"2024-09-11T22:52:53.130326+0000","last_clean":"2024-09-11T22:52:53.130326+0000","last_became_active":"2024-09-11T22:50:52.139294+0000","last_became_peered":"2024-09-11T22:50:52.139294+0000","last_unstale":"2024-09-11T22:52:53.130326+0000","last_undegraded":"2024-09-11T22:52:53.130326+0000","last_fullsized":"2024-09-11T22:52:53.130326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:07.355155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129486+0000","last_change":"2024-09-11T22:50:54.166248+0000","last_active":"2024-09-11T22:52:53.129486+0000","last_peered":"2024-09-11T22:52:53.129486+0000","last_clean":"2024-09-11T22:52:53.129486+0000","last_became_active":"2024-09-11T22:50:54.165958+0000","last_became_peered":"2024-09-11T22:50:54.165958+0000","last_unstale":"2024-09-11T22:52:53.129486+0000","last_undegraded":"2024-09-11T22:52:53.129486+0000","last_fullsized":"2024-09-11T22:52:53.129486+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:29.531741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073597+0000","last_change":"2024-09-11T22:50:52.140340+0000","last_active":"2024-09-11T22:52:53.073597+0000","last_peered":"2024-09-11T22:52:53.073597+0000","last_clean":"2024-09-11T22:52:53.073597+0000","last_became_active":"2024-09-11T22:50:52.140011+0000","last_became_peered":"2024-09-11T22:50:52.140011+0000","last_unstale":"2024-09-11T22:52:53.073597+0000","last_undegraded":"2024-09-11T22:52:53.073597+0000","last_fullsized":"2024-09-11T22:52:53.073597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:58.862640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129625+0000","last_change":"2024-09-11T22:50:54.166998+0000","last_active":"2024-09-11T22:52:53.129625+0000","last_peered":"2024-09-11T22:52:53.129625+0000","last_clean":"2024-09-11T22:52:53.129625+0000","last_became_active":"2024-09-11T22:50:54.166682+0000","last_became_peered":"2024-09-11T22:50:54.166682+0000","last_unstale":"2024-09-11T22:52:53.129625+0000","last_undegraded":"2024-09-11T22:52:53.129625+0000","last_fullsized":"2024-09-11T22:52:53.129625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:57.815914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'8","reported_seq":250,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.332242+0000","last_change":"2024-09-11T22:50:52.141380+0000","last_active":"2024-09-11T22:52:53.332242+0000","last_peered":"2024-09-11T22:52:53.332242+0000","last_clean":"2024-09-11T22:52:53.332242+0000","last_became_active":"2024-09-11T22:50:52.139925+0000","last_became_peered":"2024-09-11T22:50:52.139925+0000","last_unstale":"2024-09-11T22:52:53.332242+0000","last_undegraded":"2024-09-11T22:52:53.332242+0000","last_fullsized":"2024-09-11T22:52:53.332242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:40:38.395055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":218,"num_read_kb":218,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074552+0000","last_change":"2024-09-11T22:50:54.167909+0000","last_active":"2024-09-11T22:52:53.074552+0000","last_peered":"2024-09-11T22:52:53.074552+0000","last_clean":"2024-09-11T22:52:53.074552+0000","last_became_active":"2024-09-11T22:50:54.167571+0000","last_became_peered":"2024-09-11T22:50:54.167571+0000","last_unstale":"2024-09-11T22:52:53.074552+0000","last_undegraded":"2024-09-11T22:52:53.074552+0000","last_fullsized":"2024-09-11T22:52:53.074552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:34.179960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129913+0000","last_change":"2024-09-11T22:50:52.144545+0000","last_active":"2024-09-11T22:52:53.129913+0000","last_peered":"2024-09-11T22:52:53.129913+0000","last_clean":"2024-09-11T22:52:53.129913+0000","last_became_active":"2024-09-11T22:50:52.144221+0000","last_became_peered":"2024-09-11T22:50:52.144221+0000","last_unstale":"2024-09-11T22:52:53.129913+0000","last_undegraded":"2024-09-11T22:52:53.129913+0000","last_fullsized":"2024-09-11T22:52:53.129913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:29.509165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131618+0000","last_change":"2024-09-11T22:50:54.164132+0000","last_active":"2024-09-11T22:52:53.131618+0000","last_peered":"2024-09-11T22:52:53.131618+0000","last_clean":"2024-09-11T22:52:53.131618+0000","last_became_active":"2024-09-11T22:50:54.163842+0000","last_became_peered":"2024-09-11T22:50:54.163842+0000","last_unstale":"2024-09-11T22:52:53.131618+0000","last_undegraded":"2024-09-11T22:52:53.131618+0000","last_fullsized":"2024-09-11T22:52:53.131618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:28:55.490236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074349+0000","last_change":"2024-09-11T22:50:52.264611+0000","last_active":"2024-09-11T22:52:53.074349+0000","last_peered":"2024-09-11T22:52:53.074349+0000","last_clean":"2024-09-11T22:52:53.074349+0000","last_became_active":"2024-09-11T22:50:52.264060+0000","last_became_peered":"2024-09-11T22:50:52.264060+0000","last_unstale":"2024-09-11T22:52:53.074349+0000","last_undegraded":"2024-09-11T22:52:53.074349+0000","last_fullsized":"2024-09-11T22:52:53.074349+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:42.543997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074271+0000","last_change":"2024-09-11T22:51:56.206155+0000","last_active":"2024-09-11T22:52:53.074271+0000","last_peered":"2024-09-11T22:52:53.074271+0000","last_clean":"2024-09-11T22:52:53.074271+0000","last_became_active":"2024-09-11T22:51:56.205797+0000","last_became_peered":"2024-09-11T22:51:56.205797+0000","last_unstale":"2024-09-11T22:52:53.074271+0000","last_undegraded":"2024-09-11T22:52:53.074271+0000","last_fullsized":"2024-09-11T22:52:53.074271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:28.063825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'39","reported_seq":293,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.333681+0000","last_change":"2024-09-11T22:50:52.141782+0000","last_active":"2024-09-11T22:52:53.333681+0000","last_peered":"2024-09-11T22:52:53.333681+0000","last_clean":"2024-09-11T22:52:53.333681+0000","last_became_active":"2024-09-11T22:50:52.140662+0000","last_became_peered":"2024-09-11T22:50:52.140662+0000","last_unstale":"2024-09-11T22:52:53.333681+0000","last_undegraded":"2024-09-11T22:52:53.333681+0000","last_fullsized":"2024-09-11T22:52:53.333681+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:21.158577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68544,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":230,"num_read_kb":180,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074760+0000","last_change":"2024-09-11T22:50:54.166678+0000","last_active":"2024-09-11T22:52:53.074760+0000","last_peered":"2024-09-11T22:52:53.074760+0000","last_clean":"2024-09-11T22:52:53.074760+0000","last_became_active":"2024-09-11T22:50:54.165777+0000","last_became_peered":"2024-09-11T22:50:54.165777+0000","last_unstale":"2024-09-11T22:52:53.074760+0000","last_undegraded":"2024-09-11T22:52:53.074760+0000","last_fullsized":"2024-09-11T22:52:53.074760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:07.224563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129051+0000","last_change":"2024-09-11T22:50:52.141292+0000","last_active":"2024-09-11T22:52:53.129051+0000","last_peered":"2024-09-11T22:52:53.129051+0000","last_clean":"2024-09-11T22:52:53.129051+0000","last_became_active":"2024-09-11T22:50:52.140971+0000","last_became_peered":"2024-09-11T22:50:52.140971+0000","last_unstale":"2024-09-11T22:52:53.129051+0000","last_undegraded":"2024-09-11T22:52:53.129051+0000","last_fullsized":"2024-09-11T22:52:53.129051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:53.039684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129489+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:52:53.129489+0000","last_peered":"2024-09-11T22:52:53.129489+0000","last_clean":"2024-09-11T22:52:53.129489+0000","last_became_active":"2024-09-11T22:50:54.171520+0000","last_became_peered":"2024-09-11T22:50:54.171520+0000","last_unstale":"2024-09-11T22:52:53.129489+0000","last_undegraded":"2024-09-11T22:52:53.129489+0000","last_fullsized":"2024-09-11T22:52:53.129489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:56.594058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073605+0000","last_change":"2024-09-11T22:50:52.138725+0000","last_active":"2024-09-11T22:52:53.073605+0000","last_peered":"2024-09-11T22:52:53.073605+0000","last_clean":"2024-09-11T22:52:53.073605+0000","last_became_active":"2024-09-11T22:50:52.138367+0000","last_became_peered":"2024-09-11T22:50:52.138367+0000","last_unstale":"2024-09-11T22:52:53.073605+0000","last_undegraded":"2024-09-11T22:52:53.073605+0000","last_fullsized":"2024-09-11T22:52:53.073605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:58.811444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129217+0000","last_change":"2024-09-11T22:50:54.170524+0000","last_active":"2024-09-11T22:52:53.129217+0000","last_peered":"2024-09-11T22:52:53.129217+0000","last_clean":"2024-09-11T22:52:53.129217+0000","last_became_active":"2024-09-11T22:50:54.169947+0000","last_became_peered":"2024-09-11T22:50:54.169947+0000","last_unstale":"2024-09-11T22:52:53.129217+0000","last_undegraded":"2024-09-11T22:52:53.129217+0000","last_fullsized":"2024-09-11T22:52:53.129217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:51.768718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":143,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.337266+0000","last_change":"2024-09-11T22:50:52.139182+0000","last_active":"2024-09-11T22:52:53.337266+0000","last_peered":"2024-09-11T22:52:53.337266+0000","last_clean":"2024-09-11T22:52:53.337266+0000","last_became_active":"2024-09-11T22:50:52.138799+0000","last_became_peered":"2024-09-11T22:50:52.138799+0000","last_unstale":"2024-09-11T22:52:53.337266+0000","last_undegraded":"2024-09-11T22:52:53.337266+0000","last_fullsized":"2024-09-11T22:52:53.337266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:36.376162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129122+0000","last_change":"2024-09-11T22:50:54.168797+0000","last_active":"2024-09-11T22:52:53.129122+0000","last_peered":"2024-09-11T22:52:53.129122+0000","last_clean":"2024-09-11T22:52:53.129122+0000","last_became_active":"2024-09-11T22:50:54.168408+0000","last_became_peered":"2024-09-11T22:50:54.168408+0000","last_unstale":"2024-09-11T22:52:53.129122+0000","last_undegraded":"2024-09-11T22:52:53.129122+0000","last_fullsized":"2024-09-11T22:52:53.129122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:51.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074207+0000","last_change":"2024-09-11T22:50:52.264693+0000","last_active":"2024-09-11T22:52:53.074207+0000","last_peered":"2024-09-11T22:52:53.074207+0000","last_clean":"2024-09-11T22:52:53.074207+0000","last_became_active":"2024-09-11T22:50:52.264058+0000","last_became_peered":"2024-09-11T22:50:52.264058+0000","last_unstale":"2024-09-11T22:52:53.074207+0000","last_undegraded":"2024-09-11T22:52:53.074207+0000","last_fullsized":"2024-09-11T22:52:53.074207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:46.985921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073438+0000","last_change":"2024-09-11T22:51:55.961118+0000","last_active":"2024-09-11T22:52:53.073438+0000","last_peered":"2024-09-11T22:52:53.073438+0000","last_clean":"2024-09-11T22:52:53.073438+0000","last_became_active":"2024-09-11T22:51:55.960614+0000","last_became_peered":"2024-09-11T22:51:55.960614+0000","last_unstale":"2024-09-11T22:52:53.073438+0000","last_undegraded":"2024-09-11T22:52:53.073438+0000","last_fullsized":"2024-09-11T22:52:53.073438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:47.184327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.075042+0000","last_change":"2024-09-11T22:50:52.133049+0000","last_active":"2024-09-11T22:52:53.075042+0000","last_peered":"2024-09-11T22:52:53.075042+0000","last_clean":"2024-09-11T22:52:53.075042+0000","last_became_active":"2024-09-11T22:50:52.132647+0000","last_became_peered":"2024-09-11T22:50:52.132647+0000","last_unstale":"2024-09-11T22:52:53.075042+0000","last_undegraded":"2024-09-11T22:52:53.075042+0000","last_fullsized":"2024-09-11T22:52:53.075042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:13.213003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169115+0000","last_change":"2024-09-11T22:51:56.238023+0000","last_active":"2024-09-11T22:52:53.169115+0000","last_peered":"2024-09-11T22:52:53.169115+0000","last_clean":"2024-09-11T22:52:53.169115+0000","last_became_active":"2024-09-11T22:51:56.237676+0000","last_became_peered":"2024-09-11T22:51:56.237676+0000","last_unstale":"2024-09-11T22:52:53.169115+0000","last_undegraded":"2024-09-11T22:52:53.169115+0000","last_fullsized":"2024-09-11T22:52:53.169115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:16.215961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130047+0000","last_change":"2024-09-11T22:50:52.139766+0000","last_active":"2024-09-11T22:52:53.130047+0000","last_peered":"2024-09-11T22:52:53.130047+0000","last_clean":"2024-09-11T22:52:53.130047+0000","last_became_active":"2024-09-11T22:50:52.139456+0000","last_became_peered":"2024-09-11T22:50:52.139456+0000","last_unstale":"2024-09-11T22:52:53.130047+0000","last_undegraded":"2024-09-11T22:52:53.130047+0000","last_fullsized":"2024-09-11T22:52:53.130047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:08.358649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130080+0000","last_change":"2024-09-11T22:50:54.167717+0000","last_active":"2024-09-11T22:52:53.130080+0000","last_peered":"2024-09-11T22:52:53.130080+0000","last_clean":"2024-09-11T22:52:53.130080+0000","last_became_active":"2024-09-11T22:50:54.167331+0000","last_became_peered":"2024-09-11T22:50:54.167331+0000","last_unstale":"2024-09-11T22:52:53.130080+0000","last_undegraded":"2024-09-11T22:52:53.130080+0000","last_fullsized":"2024-09-11T22:52:53.130080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:11.240351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'41","reported_seq":73,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.283886+0000","last_change":"2024-09-11T22:50:52.141682+0000","last_active":"2024-09-11T22:52:53.283886+0000","last_peered":"2024-09-11T22:52:53.283886+0000","last_clean":"2024-09-11T22:52:53.283886+0000","last_became_active":"2024-09-11T22:50:52.141251+0000","last_became_peered":"2024-09-11T22:50:52.141251+0000","last_unstale":"2024-09-11T22:52:53.283886+0000","last_undegraded":"2024-09-11T22:52:53.283886+0000","last_fullsized":"2024-09-11T22:52:53.283886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:03.068252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129793+0000","last_change":"2024-09-11T22:50:54.174149+0000","last_active":"2024-09-11T22:52:53.129793+0000","last_peered":"2024-09-11T22:52:53.129793+0000","last_clean":"2024-09-11T22:52:53.129793+0000","last_became_active":"2024-09-11T22:50:54.173765+0000","last_became_peered":"2024-09-11T22:50:54.173765+0000","last_unstale":"2024-09-11T22:52:53.129793+0000","last_undegraded":"2024-09-11T22:52:53.129793+0000","last_fullsized":"2024-09-11T22:52:53.129793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.314334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'8","reported_seq":145,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.279965+0000","last_change":"2024-09-11T22:50:52.435019+0000","last_active":"2024-09-11T22:52:53.279965+0000","last_peered":"2024-09-11T22:52:53.279965+0000","last_clean":"2024-09-11T22:52:53.279965+0000","last_became_active":"2024-09-11T22:50:52.434642+0000","last_became_peered":"2024-09-11T22:50:52.434642+0000","last_unstale":"2024-09-11T22:52:53.279965+0000","last_undegraded":"2024-09-11T22:52:53.279965+0000","last_fullsized":"2024-09-11T22:52:53.279965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:18.441841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130004+0000","last_change":"2024-09-11T22:50:54.171291+0000","last_active":"2024-09-11T22:52:53.130004+0000","last_peered":"2024-09-11T22:52:53.130004+0000","last_clean":"2024-09-11T22:52:53.130004+0000","last_became_active":"2024-09-11T22:50:54.170961+0000","last_became_peered":"2024-09-11T22:50:54.170961+0000","last_unstale":"2024-09-11T22:52:53.130004+0000","last_undegraded":"2024-09-11T22:52:53.130004+0000","last_fullsized":"2024-09-11T22:52:53.130004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:07.211936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129647+0000","last_change":"2024-09-11T22:49:39.098881+0000","last_active":"2024-09-11T22:52:53.129647+0000","last_peered":"2024-09-11T22:52:53.129647+0000","last_clean":"2024-09-11T22:52:53.129647+0000","last_became_active":"2024-09-11T22:49:39.098458+0000","last_became_peered":"2024-09-11T22:49:39.098458+0000","last_unstale":"2024-09-11T22:52:53.129647+0000","last_undegraded":"2024-09-11T22:52:53.129647+0000","last_fullsized":"2024-09-11T22:52:53.129647+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_clean_scrub_stamp":"2024-09-11T22:46:11.418577+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:41.059384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130035+0000","last_change":"2024-09-11T22:50:52.137691+0000","last_active":"2024-09-11T22:52:53.130035+0000","last_peered":"2024-09-11T22:52:53.130035+0000","last_clean":"2024-09-11T22:52:53.130035+0000","last_became_active":"2024-09-11T22:50:52.137282+0000","last_became_peered":"2024-09-11T22:50:52.137282+0000","last_unstale":"2024-09-11T22:52:53.130035+0000","last_undegraded":"2024-09-11T22:52:53.130035+0000","last_fullsized":"2024-09-11T22:52:53.130035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:00:27.651252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129789+0000","last_change":"2024-09-11T22:51:55.959937+0000","last_active":"2024-09-11T22:52:53.129789+0000","last_peered":"2024-09-11T22:52:53.129789+0000","last_clean":"2024-09-11T22:52:53.129789+0000","last_became_active":"2024-09-11T22:51:55.959575+0000","last_became_peered":"2024-09-11T22:51:55.959575+0000","last_unstale":"2024-09-11T22:52:53.129789+0000","last_undegraded":"2024-09-11T22:52:53.129789+0000","last_fullsized":"2024-09-11T22:52:53.129789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:54.097552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129633+0000","last_change":"2024-09-11T22:50:52.144324+0000","last_active":"2024-09-11T22:52:53.129633+0000","last_peered":"2024-09-11T22:52:53.129633+0000","last_clean":"2024-09-11T22:52:53.129633+0000","last_became_active":"2024-09-11T22:50:52.144011+0000","last_became_peered":"2024-09-11T22:50:52.144011+0000","last_unstale":"2024-09-11T22:52:53.129633+0000","last_undegraded":"2024-09-11T22:52:53.129633+0000","last_fullsized":"2024-09-11T22:52:53.129633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:29.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130192+0000","last_change":"2024-09-11T22:50:54.173287+0000","last_active":"2024-09-11T22:52:53.130192+0000","last_peered":"2024-09-11T22:52:53.130192+0000","last_clean":"2024-09-11T22:52:53.130192+0000","last_became_active":"2024-09-11T22:50:54.172953+0000","last_became_peered":"2024-09-11T22:50:54.172953+0000","last_unstale":"2024-09-11T22:52:53.130192+0000","last_undegraded":"2024-09-11T22:52:53.130192+0000","last_fullsized":"2024-09-11T22:52:53.130192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:24.314604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":141,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.336909+0000","last_change":"2024-09-11T22:50:52.263777+0000","last_active":"2024-09-11T22:52:53.336909+0000","last_peered":"2024-09-11T22:52:53.336909+0000","last_clean":"2024-09-11T22:52:53.336909+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:52:53.336909+0000","last_undegraded":"2024-09-11T22:52:53.336909+0000","last_fullsized":"2024-09-11T22:52:53.336909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:37.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073403+0000","last_change":"2024-09-11T22:50:54.164927+0000","last_active":"2024-09-11T22:52:53.073403+0000","last_peered":"2024-09-11T22:52:53.073403+0000","last_clean":"2024-09-11T22:52:53.073403+0000","last_became_active":"2024-09-11T22:50:54.164602+0000","last_became_peered":"2024-09-11T22:50:54.164602+0000","last_unstale":"2024-09-11T22:52:53.073403+0000","last_undegraded":"2024-09-11T22:52:53.073403+0000","last_fullsized":"2024-09-11T22:52:53.073403+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:10.226232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":138,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129575+0000","last_change":"2024-09-11T22:50:52.141199+0000","last_active":"2024-09-11T22:52:53.129575+0000","last_peered":"2024-09-11T22:52:53.129575+0000","last_clean":"2024-09-11T22:52:53.129575+0000","last_became_active":"2024-09-11T22:50:52.140830+0000","last_became_peered":"2024-09-11T22:50:52.140830+0000","last_unstale":"2024-09-11T22:52:53.129575+0000","last_undegraded":"2024-09-11T22:52:53.129575+0000","last_fullsized":"2024-09-11T22:52:53.129575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.767803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129565+0000","last_change":"2024-09-11T22:50:54.175054+0000","last_active":"2024-09-11T22:52:53.129565+0000","last_peered":"2024-09-11T22:52:53.129565+0000","last_clean":"2024-09-11T22:52:53.129565+0000","last_became_active":"2024-09-11T22:50:54.174715+0000","last_became_peered":"2024-09-11T22:50:54.174715+0000","last_unstale":"2024-09-11T22:52:53.129565+0000","last_undegraded":"2024-09-11T22:52:53.129565+0000","last_fullsized":"2024-09-11T22:52:53.129565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:06.694988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130210+0000","last_change":"2024-09-11T22:51:55.961660+0000","last_active":"2024-09-11T22:52:53.130210+0000","last_peered":"2024-09-11T22:52:53.130210+0000","last_clean":"2024-09-11T22:52:53.130210+0000","last_became_active":"2024-09-11T22:51:55.961278+0000","last_became_peered":"2024-09-11T22:51:55.961278+0000","last_unstale":"2024-09-11T22:52:53.130210+0000","last_undegraded":"2024-09-11T22:52:53.130210+0000","last_fullsized":"2024-09-11T22:52:53.130210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:52.352490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129357+0000","last_change":"2024-09-11T22:50:54.167361+0000","last_active":"2024-09-11T22:52:53.129357+0000","last_peered":"2024-09-11T22:52:53.129357+0000","last_clean":"2024-09-11T22:52:53.129357+0000","last_became_active":"2024-09-11T22:50:54.166963+0000","last_became_peered":"2024-09-11T22:50:54.166963+0000","last_unstale":"2024-09-11T22:52:53.129357+0000","last_undegraded":"2024-09-11T22:52:53.129357+0000","last_fullsized":"2024-09-11T22:52:53.129357+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:13.294544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.075030+0000","last_change":"2024-09-11T22:50:52.264484+0000","last_active":"2024-09-11T22:52:53.075030+0000","last_peered":"2024-09-11T22:52:53.075030+0000","last_clean":"2024-09-11T22:52:53.075030+0000","last_became_active":"2024-09-11T22:50:52.264038+0000","last_became_peered":"2024-09-11T22:50:52.264038+0000","last_unstale":"2024-09-11T22:52:53.075030+0000","last_undegraded":"2024-09-11T22:52:53.075030+0000","last_fullsized":"2024-09-11T22:52:53.075030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:21.325136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129556+0000","last_change":"2024-09-11T22:50:54.162225+0000","last_active":"2024-09-11T22:52:53.129556+0000","last_peered":"2024-09-11T22:52:53.129556+0000","last_clean":"2024-09-11T22:52:53.129556+0000","last_became_active":"2024-09-11T22:50:54.161234+0000","last_became_peered":"2024-09-11T22:50:54.161234+0000","last_unstale":"2024-09-11T22:52:53.129556+0000","last_undegraded":"2024-09-11T22:52:53.129556+0000","last_fullsized":"2024-09-11T22:52:53.129556+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:00.346217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'52","reported_seq":115,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.286808+0000","last_change":"2024-09-11T22:51:55.967804+0000","last_active":"2024-09-11T22:52:53.286808+0000","last_peered":"2024-09-11T22:52:53.286808+0000","last_clean":"2024-09-11T22:52:53.286808+0000","last_became_active":"2024-09-11T22:51:55.961136+0000","last_became_peered":"2024-09-11T22:51:55.961136+0000","last_unstale":"2024-09-11T22:52:53.286808+0000","last_undegraded":"2024-09-11T22:52:53.286808+0000","last_fullsized":"2024-09-11T22:52:53.286808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:55.315179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130600+0000","last_change":"2024-09-11T22:50:54.167307+0000","last_active":"2024-09-11T22:52:53.130600+0000","last_peered":"2024-09-11T22:52:53.130600+0000","last_clean":"2024-09-11T22:52:53.130600+0000","last_became_active":"2024-09-11T22:50:54.166650+0000","last_became_peered":"2024-09-11T22:50:54.166650+0000","last_unstale":"2024-09-11T22:52:53.130600+0000","last_undegraded":"2024-09-11T22:52:53.130600+0000","last_fullsized":"2024-09-11T22:52:53.130600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:04.863258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130309+0000","last_change":"2024-09-11T22:50:52.435107+0000","last_active":"2024-09-11T22:52:53.130309+0000","last_peered":"2024-09-11T22:52:53.130309+0000","last_clean":"2024-09-11T22:52:53.130309+0000","last_became_active":"2024-09-11T22:50:52.434791+0000","last_became_peered":"2024-09-11T22:50:52.434791+0000","last_unstale":"2024-09-11T22:52:53.130309+0000","last_undegraded":"2024-09-11T22:52:53.130309+0000","last_fullsized":"2024-09-11T22:52:53.130309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:11.128791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131410+0000","last_change":"2024-09-11T22:50:54.178041+0000","last_active":"2024-09-11T22:52:53.131410+0000","last_peered":"2024-09-11T22:52:53.131410+0000","last_clean":"2024-09-11T22:52:53.131410+0000","last_became_active":"2024-09-11T22:50:54.177393+0000","last_became_peered":"2024-09-11T22:50:54.177393+0000","last_unstale":"2024-09-11T22:52:53.131410+0000","last_undegraded":"2024-09-11T22:52:53.131410+0000","last_fullsized":"2024-09-11T22:52:53.131410+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:21.608104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074102+0000","last_change":"2024-09-11T22:50:52.264051+0000","last_active":"2024-09-11T22:52:53.074102+0000","last_peered":"2024-09-11T22:52:53.074102+0000","last_clean":"2024-09-11T22:52:53.074102+0000","last_became_active":"2024-09-11T22:50:52.263674+0000","last_became_peered":"2024-09-11T22:50:52.263674+0000","last_unstale":"2024-09-11T22:52:53.074102+0000","last_undegraded":"2024-09-11T22:52:53.074102+0000","last_fullsized":"2024-09-11T22:52:53.074102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:44.655218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074213+0000","last_change":"2024-09-11T22:50:54.166409+0000","last_active":"2024-09-11T22:52:53.074213+0000","last_peered":"2024-09-11T22:52:53.074213+0000","last_clean":"2024-09-11T22:52:53.074213+0000","last_became_active":"2024-09-11T22:50:54.166037+0000","last_became_peered":"2024-09-11T22:50:54.166037+0000","last_unstale":"2024-09-11T22:52:53.074213+0000","last_undegraded":"2024-09-11T22:52:53.074213+0000","last_fullsized":"2024-09-11T22:52:53.074213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:22.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129590+0000","last_change":"2024-09-11T22:50:52.140218+0000","last_active":"2024-09-11T22:52:53.129590+0000","last_peered":"2024-09-11T22:52:53.129590+0000","last_clean":"2024-09-11T22:52:53.129590+0000","last_became_active":"2024-09-11T22:50:52.139744+0000","last_became_peered":"2024-09-11T22:50:52.139744+0000","last_unstale":"2024-09-11T22:52:53.129590+0000","last_undegraded":"2024-09-11T22:52:53.129590+0000","last_fullsized":"2024-09-11T22:52:53.129590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:50.178348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168909+0000","last_change":"2024-09-11T22:50:54.164059+0000","last_active":"2024-09-11T22:52:53.168909+0000","last_peered":"2024-09-11T22:52:53.168909+0000","last_clean":"2024-09-11T22:52:53.168909+0000","last_became_active":"2024-09-11T22:50:54.163724+0000","last_became_peered":"2024-09-11T22:50:54.163724+0000","last_unstale":"2024-09-11T22:52:53.168909+0000","last_undegraded":"2024-09-11T22:52:53.168909+0000","last_fullsized":"2024-09-11T22:52:53.168909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:57.241660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.128932+0000","last_change":"2024-09-11T22:50:52.140169+0000","last_active":"2024-09-11T22:52:53.128932+0000","last_peered":"2024-09-11T22:52:53.128932+0000","last_clean":"2024-09-11T22:52:53.128932+0000","last_became_active":"2024-09-11T22:50:52.139579+0000","last_became_peered":"2024-09-11T22:50:52.139579+0000","last_unstale":"2024-09-11T22:52:53.128932+0000","last_undegraded":"2024-09-11T22:52:53.128932+0000","last_fullsized":"2024-09-11T22:52:53.128932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:43.523961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130075+0000","last_change":"2024-09-11T22:50:54.171539+0000","last_active":"2024-09-11T22:52:53.130075+0000","last_peered":"2024-09-11T22:52:53.130075+0000","last_clean":"2024-09-11T22:52:53.130075+0000","last_became_active":"2024-09-11T22:50:54.171013+0000","last_became_peered":"2024-09-11T22:50:54.171013+0000","last_unstale":"2024-09-11T22:52:53.130075+0000","last_undegraded":"2024-09-11T22:52:53.130075+0000","last_fullsized":"2024-09-11T22:52:53.130075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:03.564389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074787+0000","last_change":"2024-09-11T22:50:52.139544+0000","last_active":"2024-09-11T22:52:53.074787+0000","last_peered":"2024-09-11T22:52:53.074787+0000","last_clean":"2024-09-11T22:52:53.074787+0000","last_became_active":"2024-09-11T22:50:52.139201+0000","last_became_peered":"2024-09-11T22:50:52.139201+0000","last_unstale":"2024-09-11T22:52:53.074787+0000","last_undegraded":"2024-09-11T22:52:53.074787+0000","last_fullsized":"2024-09-11T22:52:53.074787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:37.343437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129337+0000","last_change":"2024-09-11T22:50:54.170052+0000","last_active":"2024-09-11T22:52:53.129337+0000","last_peered":"2024-09-11T22:52:53.129337+0000","last_clean":"2024-09-11T22:52:53.129337+0000","last_became_active":"2024-09-11T22:50:54.169735+0000","last_became_peered":"2024-09-11T22:50:54.169735+0000","last_unstale":"2024-09-11T22:52:53.129337+0000","last_undegraded":"2024-09-11T22:52:53.129337+0000","last_fullsized":"2024-09-11T22:52:53.129337+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:44.007747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074181+0000","last_change":"2024-09-11T22:50:54.169959+0000","last_active":"2024-09-11T22:52:53.074181+0000","last_peered":"2024-09-11T22:52:53.074181+0000","last_clean":"2024-09-11T22:52:53.074181+0000","last_became_active":"2024-09-11T22:50:54.169649+0000","last_became_peered":"2024-09-11T22:50:54.169649+0000","last_unstale":"2024-09-11T22:52:53.074181+0000","last_undegraded":"2024-09-11T22:52:53.074181+0000","last_fullsized":"2024-09-11T22:52:53.074181+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:51.974258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074520+0000","last_change":"2024-09-11T22:50:52.140881+0000","last_active":"2024-09-11T22:52:53.074520+0000","last_peered":"2024-09-11T22:52:53.074520+0000","last_clean":"2024-09-11T22:52:53.074520+0000","last_became_active":"2024-09-11T22:50:52.140527+0000","last_became_peered":"2024-09-11T22:50:52.140527+0000","last_unstale":"2024-09-11T22:52:53.074520+0000","last_undegraded":"2024-09-11T22:52:53.074520+0000","last_fullsized":"2024-09-11T22:52:53.074520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:14.325816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074263+0000","last_change":"2024-09-11T22:50:54.166571+0000","last_active":"2024-09-11T22:52:53.074263+0000","last_peered":"2024-09-11T22:52:53.074263+0000","last_clean":"2024-09-11T22:52:53.074263+0000","last_became_active":"2024-09-11T22:50:54.166247+0000","last_became_peered":"2024-09-11T22:50:54.166247+0000","last_unstale":"2024-09-11T22:52:53.074263+0000","last_undegraded":"2024-09-11T22:52:53.074263+0000","last_fullsized":"2024-09-11T22:52:53.074263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:42.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074895+0000","last_change":"2024-09-11T22:50:52.140298+0000","last_active":"2024-09-11T22:52:53.074895+0000","last_peered":"2024-09-11T22:52:53.074895+0000","last_clean":"2024-09-11T22:52:53.074895+0000","last_became_active":"2024-09-11T22:50:52.139917+0000","last_became_peered":"2024-09-11T22:50:52.139917+0000","last_unstale":"2024-09-11T22:52:53.074895+0000","last_undegraded":"2024-09-11T22:52:53.074895+0000","last_fullsized":"2024-09-11T22:52:53.074895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:23.599010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131581+0000","last_change":"2024-09-11T22:50:54.170849+0000","last_active":"2024-09-11T22:52:53.131581+0000","last_peered":"2024-09-11T22:52:53.131581+0000","last_clean":"2024-09-11T22:52:53.131581+0000","last_became_active":"2024-09-11T22:50:54.169547+0000","last_became_peered":"2024-09-11T22:50:54.169547+0000","last_unstale":"2024-09-11T22:52:53.131581+0000","last_undegraded":"2024-09-11T22:52:53.131581+0000","last_fullsized":"2024-09-11T22:52:53.131581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:25.251158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130553+0000","last_change":"2024-09-11T22:50:52.263778+0000","last_active":"2024-09-11T22:52:53.130553+0000","last_peered":"2024-09-11T22:52:53.130553+0000","last_clean":"2024-09-11T22:52:53.130553+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:52:53.130553+0000","last_undegraded":"2024-09-11T22:52:53.130553+0000","last_fullsized":"2024-09-11T22:52:53.130553+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:03.439147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130427+0000","last_change":"2024-09-11T22:50:54.171200+0000","last_active":"2024-09-11T22:52:53.130427+0000","last_peered":"2024-09-11T22:52:53.130427+0000","last_clean":"2024-09-11T22:52:53.130427+0000","last_became_active":"2024-09-11T22:50:54.169320+0000","last_became_peered":"2024-09-11T22:50:54.169320+0000","last_unstale":"2024-09-11T22:52:53.130427+0000","last_undegraded":"2024-09-11T22:52:53.130427+0000","last_fullsized":"2024-09-11T22:52:53.130427+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:14.864699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":146,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130612+0000","last_change":"2024-09-11T22:50:52.139609+0000","last_active":"2024-09-11T22:52:53.130612+0000","last_peered":"2024-09-11T22:52:53.130612+0000","last_clean":"2024-09-11T22:52:53.130612+0000","last_became_active":"2024-09-11T22:50:52.139278+0000","last_became_peered":"2024-09-11T22:50:52.139278+0000","last_unstale":"2024-09-11T22:52:53.130612+0000","last_undegraded":"2024-09-11T22:52:53.130612+0000","last_fullsized":"2024-09-11T22:52:53.130612+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:09.538734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602815,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1227,"num_read_kb":729,"num_write":378,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1808445,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107505,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547238,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":942080,"data_stored":840516,"data_compressed":22047,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1238332,"data_compressed":44892,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939696,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":839519,"data_compressed":22047,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168628,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":437775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":991232,"data_stored":1040914,"data_compressed":38736,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561088,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":446020,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1171406,"data_compressed":44892,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":925696,"data_stored":445881,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":987136,"data_stored":970661,"data_compressed":28203,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":517944,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":515477,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":946176,"data_stored":448286,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135740,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202666,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":395335,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:52:56.386 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T22:52:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.519+0000 7f2d29cd0640 1 -- 172.21.15.152:0/3247169993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d241025c0 msgr2=0x7f2d24108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.519+0000 7f2d29cd0640 1 --2- 172.21.15.152:0/3247169993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d241025c0 0x7f2d24108e50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009ca0 tx=0x7f2d1c02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.519+0000 7f2d29cd0640 1 -- 172.21.15.152:0/3247169993 shutdown_connections 2024-09-11T22:52:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.519+0000 7f2d29cd0640 1 --2- 172.21.15.152:0/3247169993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d241025c0 0x7f2d24108e50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.519+0000 7f2d29cd0640 1 --2- 172.21.15.152:0/3247169993 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d24101c00 0x7f2d24102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.519+0000 7f2d29cd0640 1 --2- 172.21.15.152:0/3247169993 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d24100a10 0x7f2d24100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.519+0000 7f2d29cd0640 1 -- 172.21.15.152:0/3247169993 >> 172.21.15.152:0/3247169993 conn(0x7f2d240fc170 msgr2=0x7f2d240fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:56.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.519+0000 7f2d29cd0640 1 -- 172.21.15.152:0/3247169993 shutdown_connections 2024-09-11T22:52:56.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.519+0000 7f2d29cd0640 1 -- 172.21.15.152:0/3247169993 wait complete. 2024-09-11T22:52:56.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.519+0000 7f2d29cd0640 1 Processor -- start 2024-09-11T22:52:56.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.519+0000 7f2d29cd0640 1 -- start start 2024-09-11T22:52:56.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d29cd0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d24100a10 0x7f2d24069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:56.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d29cd0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d24101c00 0x7f2d2419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:56.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d29cd0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d241025c0 0x7f2d2419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:56.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d29cd0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2410c670 con 0x7f2d24101c00 2024-09-11T22:52:56.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d29cd0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2410c4f0 con 0x7f2d24100a10 2024-09-11T22:52:56.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d29cd0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2410c7f0 con 0x7f2d241025c0 2024-09-11T22:52:56.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d22ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d24101c00 0x7f2d2419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:56.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d22ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d24101c00 0x7f2d2419ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45512/0 (socket says 172.21.15.152:45512) 2024-09-11T22:52:56.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d22ffd640 1 -- 172.21.15.152:0/963638100 learned_addr learned my addr 172.21.15.152:0/963638100 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:56.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d237fe640 1 --2- 172.21.15.152:0/963638100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d24100a10 0x7f2d24069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:56.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d23fff640 1 --2- 172.21.15.152:0/963638100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d241025c0 0x7f2d2419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:56.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d22ffd640 1 -- 172.21.15.152:0/963638100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d241025c0 msgr2=0x7f2d2419f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:56.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d22ffd640 1 --2- 172.21.15.152:0/963638100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d241025c0 0x7f2d2419f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d22ffd640 1 -- 172.21.15.152:0/963638100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d24100a10 msgr2=0x7f2d24069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:56.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d22ffd640 1 --2- 172.21.15.152:0/963638100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d24100a10 0x7f2d24069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d22ffd640 1 -- 172.21.15.152:0/963638100 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2419f9c0 con 0x7f2d24101c00 2024-09-11T22:52:56.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d22ffd640 1 --2- 172.21.15.152:0/963638100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d24101c00 0x7f2d2419ca90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c920 tx=0x7f2d1400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:56.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d20ff9640 1 -- 172.21.15.152:0/963638100 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14007c70 con 0x7f2d24101c00 2024-09-11T22:52:56.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d20ff9640 1 -- 172.21.15.152:0/963638100 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d14007e10 con 0x7f2d24101c00 2024-09-11T22:52:56.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d29cd0640 1 -- 172.21.15.152:0/963638100 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2419fc50 con 0x7f2d24101c00 2024-09-11T22:52:56.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.520+0000 7f2d20ff9640 1 -- 172.21.15.152:0/963638100 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14018890 con 0x7f2d24101c00 2024-09-11T22:52:56.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.521+0000 7f2d29cd0640 1 -- 172.21.15.152:0/963638100 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d241a01a0 con 0x7f2d24101c00 2024-09-11T22:52:56.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.522+0000 7f2d29cd0640 1 -- 172.21.15.152:0/963638100 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d24107d30 con 0x7f2d24101c00 2024-09-11T22:52:56.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.523+0000 7f2d20ff9640 1 -- 172.21.15.152:0/963638100 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 137954+0+0 (secure 0 0 0) 0x7f2d14018a70 con 0x7f2d24101c00 2024-09-11T22:52:56.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.523+0000 7f2d20ff9640 1 --2- 172.21.15.152:0/963638100 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f2d100a2d00 0x7f2d100a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:56.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.524+0000 7f2d237fe640 1 --2- 172.21.15.152:0/963638100 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f2d100a2d00 0x7f2d100a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:56.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.524+0000 7f2d20ff9640 1 -- 172.21.15.152:0/963638100 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f2d140cf4a0 con 0x7f2d24101c00 2024-09-11T22:52:56.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.524+0000 7f2d237fe640 1 --2- 172.21.15.152:0/963638100 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f2d100a2d00 0x7f2d100a51c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d24101a60 tx=0x7f2d0c00b390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:56.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.528+0000 7f2d20ff9640 1 -- 172.21.15.152:0/963638100 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d14014030 con 0x7f2d24101c00 2024-09-11T22:52:56.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:52:56.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-11T22:52:56.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi191 to 176.0M 2024-09-11T22:52:56.539 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.540 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: Unable to set osd_memory_target on smithi191 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-11T22:52:56.540 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.540 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-11T22:52:56.540 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:56 smithi191 ceph-mon[35211]: Unable to set osd_memory_target on smithi204 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-11T22:52:56.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:52:56.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-09-11T22:52:56.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:56.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.549 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.549 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi191 to 176.0M 2024-09-11T22:52:56.549 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.549 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: Unable to set osd_memory_target on smithi191 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-11T22:52:56.550 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:56.550 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-11T22:52:56.550 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:56 smithi204 ceph-mon[35454]: Unable to set osd_memory_target on smithi204 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-11T22:52:56.744 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.744+0000 7f2d29cd0640 1 -- 172.21.15.152:0/963638100 --> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d24062630 con 0x7f2d100a2d00 2024-09-11T22:52:56.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.750+0000 7f2d20ff9640 1 -- 172.21.15.152:0/963638100 <== mgr.15240 v2:172.21.15.152:6800/4130474708 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258049 (secure 0 0 0) 0x7f2d24062630 con 0x7f2d100a2d00 2024-09-11T22:52:56.753 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:56.755 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-09-11T22:52:56.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.753+0000 7f2d29cd0640 1 -- 172.21.15.152:0/963638100 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f2d100a2d00 msgr2=0x7f2d100a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:56.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.753+0000 7f2d29cd0640 1 --2- 172.21.15.152:0/963638100 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f2d100a2d00 0x7f2d100a51c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d24101a60 tx=0x7f2d0c00b390 comp rx=0 tx=0).stop 2024-09-11T22:52:56.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.753+0000 7f2d29cd0640 1 -- 172.21.15.152:0/963638100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d24101c00 msgr2=0x7f2d2419ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:56.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.753+0000 7f2d29cd0640 1 --2- 172.21.15.152:0/963638100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d24101c00 0x7f2d2419ca90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c920 tx=0x7f2d1400cde0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.754+0000 7f2d29cd0640 1 -- 172.21.15.152:0/963638100 shutdown_connections 2024-09-11T22:52:56.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.754+0000 7f2d29cd0640 1 --2- 172.21.15.152:0/963638100 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f2d100a2d00 0x7f2d100a51c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.754+0000 7f2d29cd0640 1 --2- 172.21.15.152:0/963638100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d241025c0 0x7f2d2419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.754+0000 7f2d29cd0640 1 --2- 172.21.15.152:0/963638100 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2d24101c00 0x7f2d2419ca90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.754+0000 7f2d29cd0640 1 --2- 172.21.15.152:0/963638100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d24100a10 0x7f2d24069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.754+0000 7f2d29cd0640 1 -- 172.21.15.152:0/963638100 >> 172.21.15.152:0/963638100 conn(0x7f2d240fc170 msgr2=0x7f2d24104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:56.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.754+0000 7f2d29cd0640 1 -- 172.21.15.152:0/963638100 shutdown_connections 2024-09-11T22:52:56.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.754+0000 7f2d29cd0640 1 -- 172.21.15.152:0/963638100 wait complete. 2024-09-11T22:52:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-09-11T22:52:55.086644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193417,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1363,"num_read_kb":848,"num_write":561,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":944,"ondisk_log_size":944,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369080,"kb_used_data":11472,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572744,"statfs":{"total":1151940427776,"available":1151562489856,"internally_reserved":0,"allocated":11747328,"data_stored":8912731,"data_compressed":215322,"data_compressed_allocated":278528,"data_compressed_original":3407872,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.972893"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169130+0000","last_change":"2024-09-11T22:50:54.171410+0000","last_active":"2024-09-11T22:52:53.169130+0000","last_peered":"2024-09-11T22:52:53.169130+0000","last_clean":"2024-09-11T22:52:53.169130+0000","last_became_active":"2024-09-11T22:50:54.171055+0000","last_became_peered":"2024-09-11T22:50:54.171055+0000","last_unstale":"2024-09-11T22:52:53.169130+0000","last_undegraded":"2024-09-11T22:52:53.169130+0000","last_fullsized":"2024-09-11T22:52:53.169130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:54.767454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131329+0000","last_change":"2024-09-11T22:50:54.168534+0000","last_active":"2024-09-11T22:52:53.131329+0000","last_peered":"2024-09-11T22:52:53.131329+0000","last_clean":"2024-09-11T22:52:53.131329+0000","last_became_active":"2024-09-11T22:50:54.168228+0000","last_became_peered":"2024-09-11T22:50:54.168228+0000","last_unstale":"2024-09-11T22:52:53.131329+0000","last_undegraded":"2024-09-11T22:52:53.131329+0000","last_fullsized":"2024-09-11T22:52:53.131329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:11.380048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131130+0000","last_change":"2024-09-11T22:50:54.168975+0000","last_active":"2024-09-11T22:52:53.131130+0000","last_peered":"2024-09-11T22:52:53.131130+0000","last_clean":"2024-09-11T22:52:53.131130+0000","last_became_active":"2024-09-11T22:50:54.168676+0000","last_became_peered":"2024-09-11T22:50:54.168676+0000","last_unstale":"2024-09-11T22:52:53.131130+0000","last_undegraded":"2024-09-11T22:52:53.131130+0000","last_fullsized":"2024-09-11T22:52:53.131130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:15.848273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130051+0000","last_change":"2024-09-11T22:50:54.164164+0000","last_active":"2024-09-11T22:52:53.130051+0000","last_peered":"2024-09-11T22:52:53.130051+0000","last_clean":"2024-09-11T22:52:53.130051+0000","last_became_active":"2024-09-11T22:50:54.163805+0000","last_became_peered":"2024-09-11T22:50:54.163805+0000","last_unstale":"2024-09-11T22:52:53.130051+0000","last_undegraded":"2024-09-11T22:52:53.130051+0000","last_fullsized":"2024-09-11T22:52:53.130051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:54.954046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074671+0000","last_change":"2024-09-11T22:50:54.167289+0000","last_active":"2024-09-11T22:52:53.074671+0000","last_peered":"2024-09-11T22:52:53.074671+0000","last_clean":"2024-09-11T22:52:53.074671+0000","last_became_active":"2024-09-11T22:50:54.166958+0000","last_became_peered":"2024-09-11T22:50:54.166958+0000","last_unstale":"2024-09-11T22:52:53.074671+0000","last_undegraded":"2024-09-11T22:52:53.074671+0000","last_fullsized":"2024-09-11T22:52:53.074671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:25.261241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129676+0000","last_change":"2024-09-11T22:50:54.175605+0000","last_active":"2024-09-11T22:52:53.129676+0000","last_peered":"2024-09-11T22:52:53.129676+0000","last_clean":"2024-09-11T22:52:53.129676+0000","last_became_active":"2024-09-11T22:50:54.175207+0000","last_became_peered":"2024-09-11T22:50:54.175207+0000","last_unstale":"2024-09-11T22:52:53.129676+0000","last_undegraded":"2024-09-11T22:52:53.129676+0000","last_fullsized":"2024-09-11T22:52:53.129676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:47.008910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129084+0000","last_change":"2024-09-11T22:50:54.167930+0000","last_active":"2024-09-11T22:52:53.129084+0000","last_peered":"2024-09-11T22:52:53.129084+0000","last_clean":"2024-09-11T22:52:53.129084+0000","last_became_active":"2024-09-11T22:50:54.167082+0000","last_became_peered":"2024-09-11T22:50:54.167082+0000","last_unstale":"2024-09-11T22:52:53.129084+0000","last_undegraded":"2024-09-11T22:52:53.129084+0000","last_fullsized":"2024-09-11T22:52:53.129084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:50.931311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129336+0000","last_change":"2024-09-11T22:50:54.172528+0000","last_active":"2024-09-11T22:52:53.129336+0000","last_peered":"2024-09-11T22:52:53.129336+0000","last_clean":"2024-09-11T22:52:53.129336+0000","last_became_active":"2024-09-11T22:50:54.172257+0000","last_became_peered":"2024-09-11T22:50:54.172257+0000","last_unstale":"2024-09-11T22:52:53.129336+0000","last_undegraded":"2024-09-11T22:52:53.129336+0000","last_fullsized":"2024-09-11T22:52:53.129336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:13:43.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074665+0000","last_change":"2024-09-11T22:50:54.163357+0000","last_active":"2024-09-11T22:52:53.074665+0000","last_peered":"2024-09-11T22:52:53.074665+0000","last_clean":"2024-09-11T22:52:53.074665+0000","last_became_active":"2024-09-11T22:50:54.162943+0000","last_became_peered":"2024-09-11T22:50:54.162943+0000","last_unstale":"2024-09-11T22:52:53.074665+0000","last_undegraded":"2024-09-11T22:52:53.074665+0000","last_fullsized":"2024-09-11T22:52:53.074665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:57.055802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129448+0000","last_change":"2024-09-11T22:50:54.177773+0000","last_active":"2024-09-11T22:52:53.129448+0000","last_peered":"2024-09-11T22:52:53.129448+0000","last_clean":"2024-09-11T22:52:53.129448+0000","last_became_active":"2024-09-11T22:50:54.177388+0000","last_became_peered":"2024-09-11T22:50:54.177388+0000","last_unstale":"2024-09-11T22:52:53.129448+0000","last_undegraded":"2024-09-11T22:52:53.129448+0000","last_fullsized":"2024-09-11T22:52:53.129448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:36:44.625089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129792+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:52:53.129792+0000","last_peered":"2024-09-11T22:52:53.129792+0000","last_clean":"2024-09-11T22:52:53.129792+0000","last_became_active":"2024-09-11T22:50:54.171491+0000","last_became_peered":"2024-09-11T22:50:54.171491+0000","last_unstale":"2024-09-11T22:52:53.129792+0000","last_undegraded":"2024-09-11T22:52:53.129792+0000","last_fullsized":"2024-09-11T22:52:53.129792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.996640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130199+0000","last_change":"2024-09-11T22:50:54.171260+0000","last_active":"2024-09-11T22:52:53.130199+0000","last_peered":"2024-09-11T22:52:53.130199+0000","last_clean":"2024-09-11T22:52:53.130199+0000","last_became_active":"2024-09-11T22:50:54.170733+0000","last_became_peered":"2024-09-11T22:50:54.170733+0000","last_unstale":"2024-09-11T22:52:53.130199+0000","last_undegraded":"2024-09-11T22:52:53.130199+0000","last_fullsized":"2024-09-11T22:52:53.130199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:57.877280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129997+0000","last_change":"2024-09-11T22:50:54.170576+0000","last_active":"2024-09-11T22:52:53.129997+0000","last_peered":"2024-09-11T22:52:53.129997+0000","last_clean":"2024-09-11T22:52:53.129997+0000","last_became_active":"2024-09-11T22:50:54.170239+0000","last_became_peered":"2024-09-11T22:50:54.170239+0000","last_unstale":"2024-09-11T22:52:53.129997+0000","last_undegraded":"2024-09-11T22:52:53.129997+0000","last_fullsized":"2024-09-11T22:52:53.129997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:09.683171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130651+0000","last_change":"2024-09-11T22:50:54.167176+0000","last_active":"2024-09-11T22:52:53.130651+0000","last_peered":"2024-09-11T22:52:53.130651+0000","last_clean":"2024-09-11T22:52:53.130651+0000","last_became_active":"2024-09-11T22:50:54.166894+0000","last_became_peered":"2024-09-11T22:50:54.166894+0000","last_unstale":"2024-09-11T22:52:53.130651+0000","last_undegraded":"2024-09-11T22:52:53.130651+0000","last_fullsized":"2024-09-11T22:52:53.130651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:43:08.158786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130901+0000","last_change":"2024-09-11T22:50:54.163100+0000","last_active":"2024-09-11T22:52:53.130901+0000","last_peered":"2024-09-11T22:52:53.130901+0000","last_clean":"2024-09-11T22:52:53.130901+0000","last_became_active":"2024-09-11T22:50:54.162776+0000","last_became_peered":"2024-09-11T22:50:54.162776+0000","last_unstale":"2024-09-11T22:52:53.130901+0000","last_undegraded":"2024-09-11T22:52:53.130901+0000","last_fullsized":"2024-09-11T22:52:53.130901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:15.215305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074034+0000","last_change":"2024-09-11T22:50:54.166013+0000","last_active":"2024-09-11T22:52:53.074034+0000","last_peered":"2024-09-11T22:52:53.074034+0000","last_clean":"2024-09-11T22:52:53.074034+0000","last_became_active":"2024-09-11T22:50:54.165680+0000","last_became_peered":"2024-09-11T22:50:54.165680+0000","last_unstale":"2024-09-11T22:52:53.074034+0000","last_undegraded":"2024-09-11T22:52:53.074034+0000","last_fullsized":"2024-09-11T22:52:53.074034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:12.192231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169175+0000","last_change":"2024-09-11T22:50:54.172105+0000","last_active":"2024-09-11T22:52:53.169175+0000","last_peered":"2024-09-11T22:52:53.169175+0000","last_clean":"2024-09-11T22:52:53.169175+0000","last_became_active":"2024-09-11T22:50:54.171782+0000","last_became_peered":"2024-09-11T22:50:54.171782+0000","last_unstale":"2024-09-11T22:52:53.169175+0000","last_undegraded":"2024-09-11T22:52:53.169175+0000","last_fullsized":"2024-09-11T22:52:53.169175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:17.152834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130880+0000","last_change":"2024-09-11T22:50:54.167184+0000","last_active":"2024-09-11T22:52:53.130880+0000","last_peered":"2024-09-11T22:52:53.130880+0000","last_clean":"2024-09-11T22:52:53.130880+0000","last_became_active":"2024-09-11T22:50:54.166878+0000","last_became_peered":"2024-09-11T22:50:54.166878+0000","last_unstale":"2024-09-11T22:52:53.130880+0000","last_undegraded":"2024-09-11T22:52:53.130880+0000","last_fullsized":"2024-09-11T22:52:53.130880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:44.046552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129108+0000","last_change":"2024-09-11T22:50:54.176260+0000","last_active":"2024-09-11T22:52:53.129108+0000","last_peered":"2024-09-11T22:52:53.129108+0000","last_clean":"2024-09-11T22:52:53.129108+0000","last_became_active":"2024-09-11T22:50:54.175873+0000","last_became_peered":"2024-09-11T22:50:54.175873+0000","last_unstale":"2024-09-11T22:52:53.129108+0000","last_undegraded":"2024-09-11T22:52:53.129108+0000","last_fullsized":"2024-09-11T22:52:53.129108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:05.258677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168910+0000","last_change":"2024-09-11T22:50:54.172725+0000","last_active":"2024-09-11T22:52:53.168910+0000","last_peered":"2024-09-11T22:52:53.168910+0000","last_clean":"2024-09-11T22:52:53.168910+0000","last_became_active":"2024-09-11T22:50:54.172072+0000","last_became_peered":"2024-09-11T22:50:54.172072+0000","last_unstale":"2024-09-11T22:52:53.168910+0000","last_undegraded":"2024-09-11T22:52:53.168910+0000","last_fullsized":"2024-09-11T22:52:53.168910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:53.699670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130148+0000","last_change":"2024-09-11T22:50:54.165118+0000","last_active":"2024-09-11T22:52:53.130148+0000","last_peered":"2024-09-11T22:52:53.130148+0000","last_clean":"2024-09-11T22:52:53.130148+0000","last_became_active":"2024-09-11T22:50:54.164793+0000","last_became_peered":"2024-09-11T22:50:54.164793+0000","last_unstale":"2024-09-11T22:52:53.130148+0000","last_undegraded":"2024-09-11T22:52:53.130148+0000","last_fullsized":"2024-09-11T22:52:53.130148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:42.883940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074224+0000","last_change":"2024-09-11T22:50:54.176900+0000","last_active":"2024-09-11T22:52:53.074224+0000","last_peered":"2024-09-11T22:52:53.074224+0000","last_clean":"2024-09-11T22:52:53.074224+0000","last_became_active":"2024-09-11T22:50:54.176595+0000","last_became_peered":"2024-09-11T22:50:54.176595+0000","last_unstale":"2024-09-11T22:52:53.074224+0000","last_undegraded":"2024-09-11T22:52:53.074224+0000","last_fullsized":"2024-09-11T22:52:53.074224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:36.925945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129485+0000","last_change":"2024-09-11T22:50:54.167656+0000","last_active":"2024-09-11T22:52:53.129485+0000","last_peered":"2024-09-11T22:52:53.129485+0000","last_clean":"2024-09-11T22:52:53.129485+0000","last_became_active":"2024-09-11T22:50:54.167349+0000","last_became_peered":"2024-09-11T22:50:54.167349+0000","last_unstale":"2024-09-11T22:52:53.129485+0000","last_undegraded":"2024-09-11T22:52:53.129485+0000","last_fullsized":"2024-09-11T22:52:53.129485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:38.100218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073470+0000","last_change":"2024-09-11T22:50:54.177059+0000","last_active":"2024-09-11T22:52:53.073470+0000","last_peered":"2024-09-11T22:52:53.073470+0000","last_clean":"2024-09-11T22:52:53.073470+0000","last_became_active":"2024-09-11T22:50:54.176726+0000","last_became_peered":"2024-09-11T22:50:54.176726+0000","last_unstale":"2024-09-11T22:52:53.073470+0000","last_undegraded":"2024-09-11T22:52:53.073470+0000","last_fullsized":"2024-09-11T22:52:53.073470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:27:58.058944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130570+0000","last_change":"2024-09-11T22:50:54.167566+0000","last_active":"2024-09-11T22:52:53.130570+0000","last_peered":"2024-09-11T22:52:53.130570+0000","last_clean":"2024-09-11T22:52:53.130570+0000","last_became_active":"2024-09-11T22:50:54.167049+0000","last_became_peered":"2024-09-11T22:50:54.167049+0000","last_unstale":"2024-09-11T22:52:53.130570+0000","last_undegraded":"2024-09-11T22:52:53.130570+0000","last_fullsized":"2024-09-11T22:52:53.130570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:06.833099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129193+0000","last_change":"2024-09-11T22:50:54.169732+0000","last_active":"2024-09-11T22:52:53.129193+0000","last_peered":"2024-09-11T22:52:53.129193+0000","last_clean":"2024-09-11T22:52:53.129193+0000","last_became_active":"2024-09-11T22:50:54.169437+0000","last_became_peered":"2024-09-11T22:50:54.169437+0000","last_unstale":"2024-09-11T22:52:53.129193+0000","last_undegraded":"2024-09-11T22:52:53.129193+0000","last_fullsized":"2024-09-11T22:52:53.129193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:25.553387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169106+0000","last_change":"2024-09-11T22:50:54.166953+0000","last_active":"2024-09-11T22:52:53.169106+0000","last_peered":"2024-09-11T22:52:53.169106+0000","last_clean":"2024-09-11T22:52:53.169106+0000","last_became_active":"2024-09-11T22:50:54.166472+0000","last_became_peered":"2024-09-11T22:50:54.166472+0000","last_unstale":"2024-09-11T22:52:53.169106+0000","last_undegraded":"2024-09-11T22:52:53.169106+0000","last_fullsized":"2024-09-11T22:52:53.169106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:01:20.234180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129423+0000","last_change":"2024-09-11T22:50:54.171308+0000","last_active":"2024-09-11T22:52:53.129423+0000","last_peered":"2024-09-11T22:52:53.129423+0000","last_clean":"2024-09-11T22:52:53.129423+0000","last_became_active":"2024-09-11T22:50:54.168958+0000","last_became_peered":"2024-09-11T22:50:54.168958+0000","last_unstale":"2024-09-11T22:52:53.129423+0000","last_undegraded":"2024-09-11T22:52:53.129423+0000","last_fullsized":"2024-09-11T22:52:53.129423+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:20.224003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131183+0000","last_change":"2024-09-11T22:50:54.173846+0000","last_active":"2024-09-11T22:52:53.131183+0000","last_peered":"2024-09-11T22:52:53.131183+0000","last_clean":"2024-09-11T22:52:53.131183+0000","last_became_active":"2024-09-11T22:50:54.173542+0000","last_became_peered":"2024-09-11T22:50:54.173542+0000","last_unstale":"2024-09-11T22:52:53.131183+0000","last_undegraded":"2024-09-11T22:52:53.131183+0000","last_fullsized":"2024-09-11T22:52:53.131183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:29:45.775902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.128929+0000","last_change":"2024-09-11T22:50:54.176008+0000","last_active":"2024-09-11T22:52:53.128929+0000","last_peered":"2024-09-11T22:52:53.128929+0000","last_clean":"2024-09-11T22:52:53.128929+0000","last_became_active":"2024-09-11T22:50:54.175672+0000","last_became_peered":"2024-09-11T22:50:54.175672+0000","last_unstale":"2024-09-11T22:52:53.128929+0000","last_undegraded":"2024-09-11T22:52:53.128929+0000","last_fullsized":"2024-09-11T22:52:53.128929+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:35.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.128953+0000","last_change":"2024-09-11T22:50:54.162504+0000","last_active":"2024-09-11T22:52:53.128953+0000","last_peered":"2024-09-11T22:52:53.128953+0000","last_clean":"2024-09-11T22:52:53.128953+0000","last_became_active":"2024-09-11T22:50:54.162190+0000","last_became_peered":"2024-09-11T22:50:54.162190+0000","last_unstale":"2024-09-11T22:52:53.128953+0000","last_undegraded":"2024-09-11T22:52:53.128953+0000","last_fullsized":"2024-09-11T22:52:53.128953+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:52.138692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168828+0000","last_change":"2024-09-11T22:51:55.960882+0000","last_active":"2024-09-11T22:52:53.168828+0000","last_peered":"2024-09-11T22:52:53.168828+0000","last_clean":"2024-09-11T22:52:53.168828+0000","last_became_active":"2024-09-11T22:51:55.960246+0000","last_became_peered":"2024-09-11T22:51:55.960246+0000","last_unstale":"2024-09-11T22:52:53.168828+0000","last_undegraded":"2024-09-11T22:52:53.168828+0000","last_fullsized":"2024-09-11T22:52:53.168828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:21.247840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074402+0000","last_change":"2024-09-11T22:50:54.160696+0000","last_active":"2024-09-11T22:52:53.074402+0000","last_peered":"2024-09-11T22:52:53.074402+0000","last_clean":"2024-09-11T22:52:53.074402+0000","last_became_active":"2024-09-11T22:50:54.159249+0000","last_became_peered":"2024-09-11T22:50:54.159249+0000","last_unstale":"2024-09-11T22:52:53.074402+0000","last_undegraded":"2024-09-11T22:52:53.074402+0000","last_fullsized":"2024-09-11T22:52:53.074402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:09.649208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'9","reported_seq":146,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.331651+0000","last_change":"2024-09-11T22:50:52.138986+0000","last_active":"2024-09-11T22:52:53.331651+0000","last_peered":"2024-09-11T22:52:53.331651+0000","last_clean":"2024-09-11T22:52:53.331651+0000","last_became_active":"2024-09-11T22:50:52.138595+0000","last_became_peered":"2024-09-11T22:50:52.138595+0000","last_unstale":"2024-09-11T22:52:53.331651+0000","last_undegraded":"2024-09-11T22:52:53.331651+0000","last_fullsized":"2024-09-11T22:52:53.331651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:06.232146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1942,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073853+0000","last_change":"2024-09-11T22:50:54.176806+0000","last_active":"2024-09-11T22:52:53.073853+0000","last_peered":"2024-09-11T22:52:53.073853+0000","last_clean":"2024-09-11T22:52:53.073853+0000","last_became_active":"2024-09-11T22:50:54.176513+0000","last_became_peered":"2024-09-11T22:50:54.176513+0000","last_unstale":"2024-09-11T22:52:53.073853+0000","last_undegraded":"2024-09-11T22:52:53.073853+0000","last_fullsized":"2024-09-11T22:52:53.073853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:43:12.086145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129699+0000","last_change":"2024-09-11T22:50:52.142794+0000","last_active":"2024-09-11T22:52:53.129699+0000","last_peered":"2024-09-11T22:52:53.129699+0000","last_clean":"2024-09-11T22:52:53.129699+0000","last_became_active":"2024-09-11T22:50:52.142433+0000","last_became_peered":"2024-09-11T22:50:52.142433+0000","last_unstale":"2024-09-11T22:52:53.129699+0000","last_undegraded":"2024-09-11T22:52:53.129699+0000","last_fullsized":"2024-09-11T22:52:53.129699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:47:12.090055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130326+0000","last_change":"2024-09-11T22:50:52.139789+0000","last_active":"2024-09-11T22:52:53.130326+0000","last_peered":"2024-09-11T22:52:53.130326+0000","last_clean":"2024-09-11T22:52:53.130326+0000","last_became_active":"2024-09-11T22:50:52.139294+0000","last_became_peered":"2024-09-11T22:50:52.139294+0000","last_unstale":"2024-09-11T22:52:53.130326+0000","last_undegraded":"2024-09-11T22:52:53.130326+0000","last_fullsized":"2024-09-11T22:52:53.130326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:07.355155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129486+0000","last_change":"2024-09-11T22:50:54.166248+0000","last_active":"2024-09-11T22:52:53.129486+0000","last_peered":"2024-09-11T22:52:53.129486+0000","last_clean":"2024-09-11T22:52:53.129486+0000","last_became_active":"2024-09-11T22:50:54.165958+0000","last_became_peered":"2024-09-11T22:50:54.165958+0000","last_unstale":"2024-09-11T22:52:53.129486+0000","last_undegraded":"2024-09-11T22:52:53.129486+0000","last_fullsized":"2024-09-11T22:52:53.129486+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:29.531741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073597+0000","last_change":"2024-09-11T22:50:52.140340+0000","last_active":"2024-09-11T22:52:53.073597+0000","last_peered":"2024-09-11T22:52:53.073597+0000","last_clean":"2024-09-11T22:52:53.073597+0000","last_became_active":"2024-09-11T22:50:52.140011+0000","last_became_peered":"2024-09-11T22:50:52.140011+0000","last_unstale":"2024-09-11T22:52:53.073597+0000","last_undegraded":"2024-09-11T22:52:53.073597+0000","last_fullsized":"2024-09-11T22:52:53.073597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:58.862640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129625+0000","last_change":"2024-09-11T22:50:54.166998+0000","last_active":"2024-09-11T22:52:53.129625+0000","last_peered":"2024-09-11T22:52:53.129625+0000","last_clean":"2024-09-11T22:52:53.129625+0000","last_became_active":"2024-09-11T22:50:54.166682+0000","last_became_peered":"2024-09-11T22:50:54.166682+0000","last_unstale":"2024-09-11T22:52:53.129625+0000","last_undegraded":"2024-09-11T22:52:53.129625+0000","last_fullsized":"2024-09-11T22:52:53.129625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:57.815914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'8","reported_seq":250,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.332242+0000","last_change":"2024-09-11T22:50:52.141380+0000","last_active":"2024-09-11T22:52:53.332242+0000","last_peered":"2024-09-11T22:52:53.332242+0000","last_clean":"2024-09-11T22:52:53.332242+0000","last_became_active":"2024-09-11T22:50:52.139925+0000","last_became_peered":"2024-09-11T22:50:52.139925+0000","last_unstale":"2024-09-11T22:52:53.332242+0000","last_undegraded":"2024-09-11T22:52:53.332242+0000","last_fullsized":"2024-09-11T22:52:53.332242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:40:38.395055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":218,"num_read_kb":218,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074552+0000","last_change":"2024-09-11T22:50:54.167909+0000","last_active":"2024-09-11T22:52:53.074552+0000","last_peered":"2024-09-11T22:52:53.074552+0000","last_clean":"2024-09-11T22:52:53.074552+0000","last_became_active":"2024-09-11T22:50:54.167571+0000","last_became_peered":"2024-09-11T22:50:54.167571+0000","last_unstale":"2024-09-11T22:52:53.074552+0000","last_undegraded":"2024-09-11T22:52:53.074552+0000","last_fullsized":"2024-09-11T22:52:53.074552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:34.179960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129913+0000","last_change":"2024-09-11T22:50:52.144545+0000","last_active":"2024-09-11T22:52:53.129913+0000","last_peered":"2024-09-11T22:52:53.129913+0000","last_clean":"2024-09-11T22:52:53.129913+0000","last_became_active":"2024-09-11T22:50:52.144221+0000","last_became_peered":"2024-09-11T22:50:52.144221+0000","last_unstale":"2024-09-11T22:52:53.129913+0000","last_undegraded":"2024-09-11T22:52:53.129913+0000","last_fullsized":"2024-09-11T22:52:53.129913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:29.509165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131618+0000","last_change":"2024-09-11T22:50:54.164132+0000","last_active":"2024-09-11T22:52:53.131618+0000","last_peered":"2024-09-11T22:52:53.131618+0000","last_clean":"2024-09-11T22:52:53.131618+0000","last_became_active":"2024-09-11T22:50:54.163842+0000","last_became_peered":"2024-09-11T22:50:54.163842+0000","last_unstale":"2024-09-11T22:52:53.131618+0000","last_undegraded":"2024-09-11T22:52:53.131618+0000","last_fullsized":"2024-09-11T22:52:53.131618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:28:55.490236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074349+0000","last_change":"2024-09-11T22:50:52.264611+0000","last_active":"2024-09-11T22:52:53.074349+0000","last_peered":"2024-09-11T22:52:53.074349+0000","last_clean":"2024-09-11T22:52:53.074349+0000","last_became_active":"2024-09-11T22:50:52.264060+0000","last_became_peered":"2024-09-11T22:50:52.264060+0000","last_unstale":"2024-09-11T22:52:53.074349+0000","last_undegraded":"2024-09-11T22:52:53.074349+0000","last_fullsized":"2024-09-11T22:52:53.074349+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:42.543997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074271+0000","last_change":"2024-09-11T22:51:56.206155+0000","last_active":"2024-09-11T22:52:53.074271+0000","last_peered":"2024-09-11T22:52:53.074271+0000","last_clean":"2024-09-11T22:52:53.074271+0000","last_became_active":"2024-09-11T22:51:56.205797+0000","last_became_peered":"2024-09-11T22:51:56.205797+0000","last_unstale":"2024-09-11T22:52:53.074271+0000","last_undegraded":"2024-09-11T22:52:53.074271+0000","last_fullsized":"2024-09-11T22:52:53.074271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:28.063825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'39","reported_seq":293,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.333681+0000","last_change":"2024-09-11T22:50:52.141782+0000","last_active":"2024-09-11T22:52:53.333681+0000","last_peered":"2024-09-11T22:52:53.333681+0000","last_clean":"2024-09-11T22:52:53.333681+0000","last_became_active":"2024-09-11T22:50:52.140662+0000","last_became_peered":"2024-09-11T22:50:52.140662+0000","last_unstale":"2024-09-11T22:52:53.333681+0000","last_undegraded":"2024-09-11T22:52:53.333681+0000","last_fullsized":"2024-09-11T22:52:53.333681+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:21.158577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68544,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":230,"num_read_kb":180,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074760+0000","last_change":"2024-09-11T22:50:54.166678+0000","last_active":"2024-09-11T22:52:53.074760+0000","last_peered":"2024-09-11T22:52:53.074760+0000","last_clean":"2024-09-11T22:52:53.074760+0000","last_became_active":"2024-09-11T22:50:54.165777+0000","last_became_peered":"2024-09-11T22:50:54.165777+0000","last_unstale":"2024-09-11T22:52:53.074760+0000","last_undegraded":"2024-09-11T22:52:53.074760+0000","last_fullsized":"2024-09-11T22:52:53.074760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:07.224563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129051+0000","last_change":"2024-09-11T22:50:52.141292+0000","last_active":"2024-09-11T22:52:53.129051+0000","last_peered":"2024-09-11T22:52:53.129051+0000","last_clean":"2024-09-11T22:52:53.129051+0000","last_became_active":"2024-09-11T22:50:52.140971+0000","last_became_peered":"2024-09-11T22:50:52.140971+0000","last_unstale":"2024-09-11T22:52:53.129051+0000","last_undegraded":"2024-09-11T22:52:53.129051+0000","last_fullsized":"2024-09-11T22:52:53.129051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:53.039684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129489+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:52:53.129489+0000","last_peered":"2024-09-11T22:52:53.129489+0000","last_clean":"2024-09-11T22:52:53.129489+0000","last_became_active":"2024-09-11T22:50:54.171520+0000","last_became_peered":"2024-09-11T22:50:54.171520+0000","last_unstale":"2024-09-11T22:52:53.129489+0000","last_undegraded":"2024-09-11T22:52:53.129489+0000","last_fullsized":"2024-09-11T22:52:53.129489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:56.594058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073605+0000","last_change":"2024-09-11T22:50:52.138725+0000","last_active":"2024-09-11T22:52:53.073605+0000","last_peered":"2024-09-11T22:52:53.073605+0000","last_clean":"2024-09-11T22:52:53.073605+0000","last_became_active":"2024-09-11T22:50:52.138367+0000","last_became_peered":"2024-09-11T22:50:52.138367+0000","last_unstale":"2024-09-11T22:52:53.073605+0000","last_undegraded":"2024-09-11T22:52:53.073605+0000","last_fullsized":"2024-09-11T22:52:53.073605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:58.811444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129217+0000","last_change":"2024-09-11T22:50:54.170524+0000","last_active":"2024-09-11T22:52:53.129217+0000","last_peered":"2024-09-11T22:52:53.129217+0000","last_clean":"2024-09-11T22:52:53.129217+0000","last_became_active":"2024-09-11T22:50:54.169947+0000","last_became_peered":"2024-09-11T22:50:54.169947+0000","last_unstale":"2024-09-11T22:52:53.129217+0000","last_undegraded":"2024-09-11T22:52:53.129217+0000","last_fullsized":"2024-09-11T22:52:53.129217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:51.768718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":143,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.337266+0000","last_change":"2024-09-11T22:50:52.139182+0000","last_active":"2024-09-11T22:52:53.337266+0000","last_peered":"2024-09-11T22:52:53.337266+0000","last_clean":"2024-09-11T22:52:53.337266+0000","last_became_active":"2024-09-11T22:50:52.138799+0000","last_became_peered":"2024-09-11T22:50:52.138799+0000","last_unstale":"2024-09-11T22:52:53.337266+0000","last_undegraded":"2024-09-11T22:52:53.337266+0000","last_fullsized":"2024-09-11T22:52:53.337266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:36.376162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129122+0000","last_change":"2024-09-11T22:50:54.168797+0000","last_active":"2024-09-11T22:52:53.129122+0000","last_peered":"2024-09-11T22:52:53.129122+0000","last_clean":"2024-09-11T22:52:53.129122+0000","last_became_active":"2024-09-11T22:50:54.168408+0000","last_became_peered":"2024-09-11T22:50:54.168408+0000","last_unstale":"2024-09-11T22:52:53.129122+0000","last_undegraded":"2024-09-11T22:52:53.129122+0000","last_fullsized":"2024-09-11T22:52:53.129122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:51.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074207+0000","last_change":"2024-09-11T22:50:52.264693+0000","last_active":"2024-09-11T22:52:53.074207+0000","last_peered":"2024-09-11T22:52:53.074207+0000","last_clean":"2024-09-11T22:52:53.074207+0000","last_became_active":"2024-09-11T22:50:52.264058+0000","last_became_peered":"2024-09-11T22:50:52.264058+0000","last_unstale":"2024-09-11T22:52:53.074207+0000","last_undegraded":"2024-09-11T22:52:53.074207+0000","last_fullsized":"2024-09-11T22:52:53.074207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:46.985921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073438+0000","last_change":"2024-09-11T22:51:55.961118+0000","last_active":"2024-09-11T22:52:53.073438+0000","last_peered":"2024-09-11T22:52:53.073438+0000","last_clean":"2024-09-11T22:52:53.073438+0000","last_became_active":"2024-09-11T22:51:55.960614+0000","last_became_peered":"2024-09-11T22:51:55.960614+0000","last_unstale":"2024-09-11T22:52:53.073438+0000","last_undegraded":"2024-09-11T22:52:53.073438+0000","last_fullsized":"2024-09-11T22:52:53.073438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:47.184327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.075042+0000","last_change":"2024-09-11T22:50:52.133049+0000","last_active":"2024-09-11T22:52:53.075042+0000","last_peered":"2024-09-11T22:52:53.075042+0000","last_clean":"2024-09-11T22:52:53.075042+0000","last_became_active":"2024-09-11T22:50:52.132647+0000","last_became_peered":"2024-09-11T22:50:52.132647+0000","last_unstale":"2024-09-11T22:52:53.075042+0000","last_undegraded":"2024-09-11T22:52:53.075042+0000","last_fullsized":"2024-09-11T22:52:53.075042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:13.213003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169115+0000","last_change":"2024-09-11T22:51:56.238023+0000","last_active":"2024-09-11T22:52:53.169115+0000","last_peered":"2024-09-11T22:52:53.169115+0000","last_clean":"2024-09-11T22:52:53.169115+0000","last_became_active":"2024-09-11T22:51:56.237676+0000","last_became_peered":"2024-09-11T22:51:56.237676+0000","last_unstale":"2024-09-11T22:52:53.169115+0000","last_undegraded":"2024-09-11T22:52:53.169115+0000","last_fullsized":"2024-09-11T22:52:53.169115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:16.215961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130047+0000","last_change":"2024-09-11T22:50:52.139766+0000","last_active":"2024-09-11T22:52:53.130047+0000","last_peered":"2024-09-11T22:52:53.130047+0000","last_clean":"2024-09-11T22:52:53.130047+0000","last_became_active":"2024-09-11T22:50:52.139456+0000","last_became_peered":"2024-09-11T22:50:52.139456+0000","last_unstale":"2024-09-11T22:52:53.130047+0000","last_undegraded":"2024-09-11T22:52:53.130047+0000","last_fullsized":"2024-09-11T22:52:53.130047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:08.358649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130080+0000","last_change":"2024-09-11T22:50:54.167717+0000","last_active":"2024-09-11T22:52:53.130080+0000","last_peered":"2024-09-11T22:52:53.130080+0000","last_clean":"2024-09-11T22:52:53.130080+0000","last_became_active":"2024-09-11T22:50:54.167331+0000","last_became_peered":"2024-09-11T22:50:54.167331+0000","last_unstale":"2024-09-11T22:52:53.130080+0000","last_undegraded":"2024-09-11T22:52:53.130080+0000","last_fullsized":"2024-09-11T22:52:53.130080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:11.240351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'41","reported_seq":73,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.283886+0000","last_change":"2024-09-11T22:50:52.141682+0000","last_active":"2024-09-11T22:52:53.283886+0000","last_peered":"2024-09-11T22:52:53.283886+0000","last_clean":"2024-09-11T22:52:53.283886+0000","last_became_active":"2024-09-11T22:50:52.141251+0000","last_became_peered":"2024-09-11T22:50:52.141251+0000","last_unstale":"2024-09-11T22:52:53.283886+0000","last_undegraded":"2024-09-11T22:52:53.283886+0000","last_fullsized":"2024-09-11T22:52:53.283886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:03.068252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129793+0000","last_change":"2024-09-11T22:50:54.174149+0000","last_active":"2024-09-11T22:52:53.129793+0000","last_peered":"2024-09-11T22:52:53.129793+0000","last_clean":"2024-09-11T22:52:53.129793+0000","last_became_active":"2024-09-11T22:50:54.173765+0000","last_became_peered":"2024-09-11T22:50:54.173765+0000","last_unstale":"2024-09-11T22:52:53.129793+0000","last_undegraded":"2024-09-11T22:52:53.129793+0000","last_fullsized":"2024-09-11T22:52:53.129793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.314334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'8","reported_seq":145,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.279965+0000","last_change":"2024-09-11T22:50:52.435019+0000","last_active":"2024-09-11T22:52:53.279965+0000","last_peered":"2024-09-11T22:52:53.279965+0000","last_clean":"2024-09-11T22:52:53.279965+0000","last_became_active":"2024-09-11T22:50:52.434642+0000","last_became_peered":"2024-09-11T22:50:52.434642+0000","last_unstale":"2024-09-11T22:52:53.279965+0000","last_undegraded":"2024-09-11T22:52:53.279965+0000","last_fullsized":"2024-09-11T22:52:53.279965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:18.441841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130004+0000","last_change":"2024-09-11T22:50:54.171291+0000","last_active":"2024-09-11T22:52:53.130004+0000","last_peered":"2024-09-11T22:52:53.130004+0000","last_clean":"2024-09-11T22:52:53.130004+0000","last_became_active":"2024-09-11T22:50:54.170961+0000","last_became_peered":"2024-09-11T22:50:54.170961+0000","last_unstale":"2024-09-11T22:52:53.130004+0000","last_undegraded":"2024-09-11T22:52:53.130004+0000","last_fullsized":"2024-09-11T22:52:53.130004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:07.211936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129647+0000","last_change":"2024-09-11T22:49:39.098881+0000","last_active":"2024-09-11T22:52:53.129647+0000","last_peered":"2024-09-11T22:52:53.129647+0000","last_clean":"2024-09-11T22:52:53.129647+0000","last_became_active":"2024-09-11T22:49:39.098458+0000","last_became_peered":"2024-09-11T22:49:39.098458+0000","last_unstale":"2024-09-11T22:52:53.129647+0000","last_undegraded":"2024-09-11T22:52:53.129647+0000","last_fullsized":"2024-09-11T22:52:53.129647+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_clean_scrub_stamp":"2024-09-11T22:46:11.418577+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:41.059384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130035+0000","last_change":"2024-09-11T22:50:52.137691+0000","last_active":"2024-09-11T22:52:53.130035+0000","last_peered":"2024-09-11T22:52:53.130035+0000","last_clean":"2024-09-11T22:52:53.130035+0000","last_became_active":"2024-09-11T22:50:52.137282+0000","last_became_peered":"2024-09-11T22:50:52.137282+0000","last_unstale":"2024-09-11T22:52:53.130035+0000","last_undegraded":"2024-09-11T22:52:53.130035+0000","last_fullsized":"2024-09-11T22:52:53.130035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:00:27.651252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129789+0000","last_change":"2024-09-11T22:51:55.959937+0000","last_active":"2024-09-11T22:52:53.129789+0000","last_peered":"2024-09-11T22:52:53.129789+0000","last_clean":"2024-09-11T22:52:53.129789+0000","last_became_active":"2024-09-11T22:51:55.959575+0000","last_became_peered":"2024-09-11T22:51:55.959575+0000","last_unstale":"2024-09-11T22:52:53.129789+0000","last_undegraded":"2024-09-11T22:52:53.129789+0000","last_fullsized":"2024-09-11T22:52:53.129789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:54.097552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129633+0000","last_change":"2024-09-11T22:50:52.144324+0000","last_active":"2024-09-11T22:52:53.129633+0000","last_peered":"2024-09-11T22:52:53.129633+0000","last_clean":"2024-09-11T22:52:53.129633+0000","last_became_active":"2024-09-11T22:50:52.144011+0000","last_became_peered":"2024-09-11T22:50:52.144011+0000","last_unstale":"2024-09-11T22:52:53.129633+0000","last_undegraded":"2024-09-11T22:52:53.129633+0000","last_fullsized":"2024-09-11T22:52:53.129633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:29.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130192+0000","last_change":"2024-09-11T22:50:54.173287+0000","last_active":"2024-09-11T22:52:53.130192+0000","last_peered":"2024-09-11T22:52:53.130192+0000","last_clean":"2024-09-11T22:52:53.130192+0000","last_became_active":"2024-09-11T22:50:54.172953+0000","last_became_peered":"2024-09-11T22:50:54.172953+0000","last_unstale":"2024-09-11T22:52:53.130192+0000","last_undegraded":"2024-09-11T22:52:53.130192+0000","last_fullsized":"2024-09-11T22:52:53.130192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:24.314604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":141,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.336909+0000","last_change":"2024-09-11T22:50:52.263777+0000","last_active":"2024-09-11T22:52:53.336909+0000","last_peered":"2024-09-11T22:52:53.336909+0000","last_clean":"2024-09-11T22:52:53.336909+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:52:53.336909+0000","last_undegraded":"2024-09-11T22:52:53.336909+0000","last_fullsized":"2024-09-11T22:52:53.336909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:37.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073403+0000","last_change":"2024-09-11T22:50:54.164927+0000","last_active":"2024-09-11T22:52:53.073403+0000","last_peered":"2024-09-11T22:52:53.073403+0000","last_clean":"2024-09-11T22:52:53.073403+0000","last_became_active":"2024-09-11T22:50:54.164602+0000","last_became_peered":"2024-09-11T22:50:54.164602+0000","last_unstale":"2024-09-11T22:52:53.073403+0000","last_undegraded":"2024-09-11T22:52:53.073403+0000","last_fullsized":"2024-09-11T22:52:53.073403+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:10.226232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":138,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129575+0000","last_change":"2024-09-11T22:50:52.141199+0000","last_active":"2024-09-11T22:52:53.129575+0000","last_peered":"2024-09-11T22:52:53.129575+0000","last_clean":"2024-09-11T22:52:53.129575+0000","last_became_active":"2024-09-11T22:50:52.140830+0000","last_became_peered":"2024-09-11T22:50:52.140830+0000","last_unstale":"2024-09-11T22:52:53.129575+0000","last_undegraded":"2024-09-11T22:52:53.129575+0000","last_fullsized":"2024-09-11T22:52:53.129575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.767803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129565+0000","last_change":"2024-09-11T22:50:54.175054+0000","last_active":"2024-09-11T22:52:53.129565+0000","last_peered":"2024-09-11T22:52:53.129565+0000","last_clean":"2024-09-11T22:52:53.129565+0000","last_became_active":"2024-09-11T22:50:54.174715+0000","last_became_peered":"2024-09-11T22:50:54.174715+0000","last_unstale":"2024-09-11T22:52:53.129565+0000","last_undegraded":"2024-09-11T22:52:53.129565+0000","last_fullsized":"2024-09-11T22:52:53.129565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:06.694988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130210+0000","last_change":"2024-09-11T22:51:55.961660+0000","last_active":"2024-09-11T22:52:53.130210+0000","last_peered":"2024-09-11T22:52:53.130210+0000","last_clean":"2024-09-11T22:52:53.130210+0000","last_became_active":"2024-09-11T22:51:55.961278+0000","last_became_peered":"2024-09-11T22:51:55.961278+0000","last_unstale":"2024-09-11T22:52:53.130210+0000","last_undegraded":"2024-09-11T22:52:53.130210+0000","last_fullsized":"2024-09-11T22:52:53.130210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:52.352490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129357+0000","last_change":"2024-09-11T22:50:54.167361+0000","last_active":"2024-09-11T22:52:53.129357+0000","last_peered":"2024-09-11T22:52:53.129357+0000","last_clean":"2024-09-11T22:52:53.129357+0000","last_became_active":"2024-09-11T22:50:54.166963+0000","last_became_peered":"2024-09-11T22:50:54.166963+0000","last_unstale":"2024-09-11T22:52:53.129357+0000","last_undegraded":"2024-09-11T22:52:53.129357+0000","last_fullsized":"2024-09-11T22:52:53.129357+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:13.294544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.075030+0000","last_change":"2024-09-11T22:50:52.264484+0000","last_active":"2024-09-11T22:52:53.075030+0000","last_peered":"2024-09-11T22:52:53.075030+0000","last_clean":"2024-09-11T22:52:53.075030+0000","last_became_active":"2024-09-11T22:50:52.264038+0000","last_became_peered":"2024-09-11T22:50:52.264038+0000","last_unstale":"2024-09-11T22:52:53.075030+0000","last_undegraded":"2024-09-11T22:52:53.075030+0000","last_fullsized":"2024-09-11T22:52:53.075030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:21.325136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129556+0000","last_change":"2024-09-11T22:50:54.162225+0000","last_active":"2024-09-11T22:52:53.129556+0000","last_peered":"2024-09-11T22:52:53.129556+0000","last_clean":"2024-09-11T22:52:53.129556+0000","last_became_active":"2024-09-11T22:50:54.161234+0000","last_became_peered":"2024-09-11T22:50:54.161234+0000","last_unstale":"2024-09-11T22:52:53.129556+0000","last_undegraded":"2024-09-11T22:52:53.129556+0000","last_fullsized":"2024-09-11T22:52:53.129556+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:00.346217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'52","reported_seq":115,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.286808+0000","last_change":"2024-09-11T22:51:55.967804+0000","last_active":"2024-09-11T22:52:53.286808+0000","last_peered":"2024-09-11T22:52:53.286808+0000","last_clean":"2024-09-11T22:52:53.286808+0000","last_became_active":"2024-09-11T22:51:55.961136+0000","last_became_peered":"2024-09-11T22:51:55.961136+0000","last_unstale":"2024-09-11T22:52:53.286808+0000","last_undegraded":"2024-09-11T22:52:53.286808+0000","last_fullsized":"2024-09-11T22:52:53.286808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:55.315179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130600+0000","last_change":"2024-09-11T22:50:54.167307+0000","last_active":"2024-09-11T22:52:53.130600+0000","last_peered":"2024-09-11T22:52:53.130600+0000","last_clean":"2024-09-11T22:52:53.130600+0000","last_became_active":"2024-09-11T22:50:54.166650+0000","last_became_peered":"2024-09-11T22:50:54.166650+0000","last_unstale":"2024-09-11T22:52:53.130600+0000","last_undegraded":"2024-09-11T22:52:53.130600+0000","last_fullsized":"2024-09-11T22:52:53.130600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:04.863258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130309+0000","last_change":"2024-09-11T22:50:52.435107+0000","last_active":"2024-09-11T22:52:53.130309+0000","last_peered":"2024-09-11T22:52:53.130309+0000","last_clean":"2024-09-11T22:52:53.130309+0000","last_became_active":"2024-09-11T22:50:52.434791+0000","last_became_peered":"2024-09-11T22:50:52.434791+0000","last_unstale":"2024-09-11T22:52:53.130309+0000","last_undegraded":"2024-09-11T22:52:53.130309+0000","last_fullsized":"2024-09-11T22:52:53.130309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:11.128791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131410+0000","last_change":"2024-09-11T22:50:54.178041+0000","last_active":"2024-09-11T22:52:53.131410+0000","last_peered":"2024-09-11T22:52:53.131410+0000","last_clean":"2024-09-11T22:52:53.131410+0000","last_became_active":"2024-09-11T22:50:54.177393+0000","last_became_peered":"2024-09-11T22:50:54.177393+0000","last_unstale":"2024-09-11T22:52:53.131410+0000","last_undegraded":"2024-09-11T22:52:53.131410+0000","last_fullsized":"2024-09-11T22:52:53.131410+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:21.608104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074102+0000","last_change":"2024-09-11T22:50:52.264051+0000","last_active":"2024-09-11T22:52:53.074102+0000","last_peered":"2024-09-11T22:52:53.074102+0000","last_clean":"2024-09-11T22:52:53.074102+0000","last_became_active":"2024-09-11T22:50:52.263674+0000","last_became_peered":"2024-09-11T22:50:52.263674+0000","last_unstale":"2024-09-11T22:52:53.074102+0000","last_undegraded":"2024-09-11T22:52:53.074102+0000","last_fullsized":"2024-09-11T22:52:53.074102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:44.655218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074213+0000","last_change":"2024-09-11T22:50:54.166409+0000","last_active":"2024-09-11T22:52:53.074213+0000","last_peered":"2024-09-11T22:52:53.074213+0000","last_clean":"2024-09-11T22:52:53.074213+0000","last_became_active":"2024-09-11T22:50:54.166037+0000","last_became_peered":"2024-09-11T22:50:54.166037+0000","last_unstale":"2024-09-11T22:52:53.074213+0000","last_undegraded":"2024-09-11T22:52:53.074213+0000","last_fullsized":"2024-09-11T22:52:53.074213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:22.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129590+0000","last_change":"2024-09-11T22:50:52.140218+0000","last_active":"2024-09-11T22:52:53.129590+0000","last_peered":"2024-09-11T22:52:53.129590+0000","last_clean":"2024-09-11T22:52:53.129590+0000","last_became_active":"2024-09-11T22:50:52.139744+0000","last_became_peered":"2024-09-11T22:50:52.139744+0000","last_unstale":"2024-09-11T22:52:53.129590+0000","last_undegraded":"2024-09-11T22:52:53.129590+0000","last_fullsized":"2024-09-11T22:52:53.129590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:50.178348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168909+0000","last_change":"2024-09-11T22:50:54.164059+0000","last_active":"2024-09-11T22:52:53.168909+0000","last_peered":"2024-09-11T22:52:53.168909+0000","last_clean":"2024-09-11T22:52:53.168909+0000","last_became_active":"2024-09-11T22:50:54.163724+0000","last_became_peered":"2024-09-11T22:50:54.163724+0000","last_unstale":"2024-09-11T22:52:53.168909+0000","last_undegraded":"2024-09-11T22:52:53.168909+0000","last_fullsized":"2024-09-11T22:52:53.168909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:57.241660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.128932+0000","last_change":"2024-09-11T22:50:52.140169+0000","last_active":"2024-09-11T22:52:53.128932+0000","last_peered":"2024-09-11T22:52:53.128932+0000","last_clean":"2024-09-11T22:52:53.128932+0000","last_became_active":"2024-09-11T22:50:52.139579+0000","last_became_peered":"2024-09-11T22:50:52.139579+0000","last_unstale":"2024-09-11T22:52:53.128932+0000","last_undegraded":"2024-09-11T22:52:53.128932+0000","last_fullsized":"2024-09-11T22:52:53.128932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:43.523961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130075+0000","last_change":"2024-09-11T22:50:54.171539+0000","last_active":"2024-09-11T22:52:53.130075+0000","last_peered":"2024-09-11T22:52:53.130075+0000","last_clean":"2024-09-11T22:52:53.130075+0000","last_became_active":"2024-09-11T22:50:54.171013+0000","last_became_peered":"2024-09-11T22:50:54.171013+0000","last_unstale":"2024-09-11T22:52:53.130075+0000","last_undegraded":"2024-09-11T22:52:53.130075+0000","last_fullsized":"2024-09-11T22:52:53.130075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:03.564389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074787+0000","last_change":"2024-09-11T22:50:52.139544+0000","last_active":"2024-09-11T22:52:53.074787+0000","last_peered":"2024-09-11T22:52:53.074787+0000","last_clean":"2024-09-11T22:52:53.074787+0000","last_became_active":"2024-09-11T22:50:52.139201+0000","last_became_peered":"2024-09-11T22:50:52.139201+0000","last_unstale":"2024-09-11T22:52:53.074787+0000","last_undegraded":"2024-09-11T22:52:53.074787+0000","last_fullsized":"2024-09-11T22:52:53.074787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:37.343437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129337+0000","last_change":"2024-09-11T22:50:54.170052+0000","last_active":"2024-09-11T22:52:53.129337+0000","last_peered":"2024-09-11T22:52:53.129337+0000","last_clean":"2024-09-11T22:52:53.129337+0000","last_became_active":"2024-09-11T22:50:54.169735+0000","last_became_peered":"2024-09-11T22:50:54.169735+0000","last_unstale":"2024-09-11T22:52:53.129337+0000","last_undegraded":"2024-09-11T22:52:53.129337+0000","last_fullsized":"2024-09-11T22:52:53.129337+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:44.007747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074181+0000","last_change":"2024-09-11T22:50:54.169959+0000","last_active":"2024-09-11T22:52:53.074181+0000","last_peered":"2024-09-11T22:52:53.074181+0000","last_clean":"2024-09-11T22:52:53.074181+0000","last_became_active":"2024-09-11T22:50:54.169649+0000","last_became_peered":"2024-09-11T22:50:54.169649+0000","last_unstale":"2024-09-11T22:52:53.074181+0000","last_undegraded":"2024-09-11T22:52:53.074181+0000","last_fullsized":"2024-09-11T22:52:53.074181+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:51.974258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074520+0000","last_change":"2024-09-11T22:50:52.140881+0000","last_active":"2024-09-11T22:52:53.074520+0000","last_peered":"2024-09-11T22:52:53.074520+0000","last_clean":"2024-09-11T22:52:53.074520+0000","last_became_active":"2024-09-11T22:50:52.140527+0000","last_became_peered":"2024-09-11T22:50:52.140527+0000","last_unstale":"2024-09-11T22:52:53.074520+0000","last_undegraded":"2024-09-11T22:52:53.074520+0000","last_fullsized":"2024-09-11T22:52:53.074520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:14.325816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074263+0000","last_change":"2024-09-11T22:50:54.166571+0000","last_active":"2024-09-11T22:52:53.074263+0000","last_peered":"2024-09-11T22:52:53.074263+0000","last_clean":"2024-09-11T22:52:53.074263+0000","last_became_active":"2024-09-11T22:50:54.166247+0000","last_became_peered":"2024-09-11T22:50:54.166247+0000","last_unstale":"2024-09-11T22:52:53.074263+0000","last_undegraded":"2024-09-11T22:52:53.074263+0000","last_fullsized":"2024-09-11T22:52:53.074263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:42.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074895+0000","last_change":"2024-09-11T22:50:52.140298+0000","last_active":"2024-09-11T22:52:53.074895+0000","last_peered":"2024-09-11T22:52:53.074895+0000","last_clean":"2024-09-11T22:52:53.074895+0000","last_became_active":"2024-09-11T22:50:52.139917+0000","last_became_peered":"2024-09-11T22:50:52.139917+0000","last_unstale":"2024-09-11T22:52:53.074895+0000","last_undegraded":"2024-09-11T22:52:53.074895+0000","last_fullsized":"2024-09-11T22:52:53.074895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:23.599010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131581+0000","last_change":"2024-09-11T22:50:54.170849+0000","last_active":"2024-09-11T22:52:53.131581+0000","last_peered":"2024-09-11T22:52:53.131581+0000","last_clean":"2024-09-11T22:52:53.131581+0000","last_became_active":"2024-09-11T22:50:54.169547+0000","last_became_peered":"2024-09-11T22:50:54.169547+0000","last_unstale":"2024-09-11T22:52:53.131581+0000","last_undegraded":"2024-09-11T22:52:53.131581+0000","last_fullsized":"2024-09-11T22:52:53.131581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:25.251158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130553+0000","last_change":"2024-09-11T22:50:52.263778+0000","last_active":"2024-09-11T22:52:53.130553+0000","last_peered":"2024-09-11T22:52:53.130553+0000","last_clean":"2024-09-11T22:52:53.130553+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:52:53.130553+0000","last_undegraded":"2024-09-11T22:52:53.130553+0000","last_fullsized":"2024-09-11T22:52:53.130553+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:03.439147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130427+0000","last_change":"2024-09-11T22:50:54.171200+0000","last_active":"2024-09-11T22:52:53.130427+0000","last_peered":"2024-09-11T22:52:53.130427+0000","last_clean":"2024-09-11T22:52:53.130427+0000","last_became_active":"2024-09-11T22:50:54.169320+0000","last_became_peered":"2024-09-11T22:50:54.169320+0000","last_unstale":"2024-09-11T22:52:53.130427+0000","last_undegraded":"2024-09-11T22:52:53.130427+0000","last_fullsized":"2024-09-11T22:52:53.130427+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:14.864699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":146,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130612+0000","last_change":"2024-09-11T22:50:52.139609+0000","last_active":"2024-09-11T22:52:53.130612+0000","last_peered":"2024-09-11T22:52:53.130612+0000","last_clean":"2024-09-11T22:52:53.130612+0000","last_became_active":"2024-09-11T22:50:52.139278+0000","last_became_peered":"2024-09-11T22:50:52.139278+0000","last_unstale":"2024-09-11T22:52:53.130612+0000","last_undegraded":"2024-09-11T22:52:53.130612+0000","last_fullsized":"2024-09-11T22:52:53.130612+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:09.538734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602815,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1227,"num_read_kb":729,"num_write":378,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1808445,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107505,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547238,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":942080,"data_stored":840516,"data_compressed":22047,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1238332,"data_compressed":44892,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939696,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":839519,"data_compressed":22047,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168628,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":437775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":991232,"data_stored":1040914,"data_compressed":38736,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561088,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":446020,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1171406,"data_compressed":44892,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":925696,"data_stored":445881,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":987136,"data_stored":970661,"data_compressed":28203,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":517944,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":515477,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":946176,"data_stored":448286,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135740,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202666,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":395335,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:52:56.769 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-11T22:52:56.770 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T22:52:56.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.906+0000 7fa95e4dd640 1 -- 172.21.15.152:0/112963787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9581025c0 msgr2=0x7fa958108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:56.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.906+0000 7fa95e4dd640 1 --2- 172.21.15.152:0/112963787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9581025c0 0x7fa958108e50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa950009ca0 tx=0x7fa95002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.906+0000 7fa95e4dd640 1 -- 172.21.15.152:0/112963787 shutdown_connections 2024-09-11T22:52:56.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.906+0000 7fa95e4dd640 1 --2- 172.21.15.152:0/112963787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa9581025c0 0x7fa958108e50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.906+0000 7fa95e4dd640 1 --2- 172.21.15.152:0/112963787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa958101c00 0x7fa958102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.906+0000 7fa95e4dd640 1 --2- 172.21.15.152:0/112963787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa958100a10 0x7fa958100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.906+0000 7fa95e4dd640 1 -- 172.21.15.152:0/112963787 >> 172.21.15.152:0/112963787 conn(0x7fa9580fc170 msgr2=0x7fa9580fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:56.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.906+0000 7fa95e4dd640 1 -- 172.21.15.152:0/112963787 shutdown_connections 2024-09-11T22:52:56.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.906+0000 7fa95e4dd640 1 -- 172.21.15.152:0/112963787 wait complete. 2024-09-11T22:52:56.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.906+0000 7fa95e4dd640 1 Processor -- start 2024-09-11T22:52:56.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.906+0000 7fa95e4dd640 1 -- start start 2024-09-11T22:52:56.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa95e4dd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa958100a10 0x7fa958197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:56.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa95e4dd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa958101c00 0x7fa958198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa95e4dd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9581025c0 0x7fa95819f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa95e4dd640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa95810d7e0 con 0x7fa958100a10 2024-09-11T22:52:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa95e4dd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa95810d660 con 0x7fa9581025c0 2024-09-11T22:52:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa95e4dd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa95810d960 con 0x7fa958101c00 2024-09-11T22:52:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa957fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa958100a10 0x7fa958197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa957fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa958100a10 0x7fa958197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45536/0 (socket says 172.21.15.152:45536) 2024-09-11T22:52:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa957fff640 1 -- 172.21.15.152:0/871212942 learned_addr learned my addr 172.21.15.152:0/871212942 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa95ca7a640 1 --2- 172.21.15.152:0/871212942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9581025c0 0x7fa95819f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa94ffff640 1 --2- 172.21.15.152:0/871212942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa958101c00 0x7fa958198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa957fff640 1 -- 172.21.15.152:0/871212942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa958101c00 msgr2=0x7fa958198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa957fff640 1 --2- 172.21.15.152:0/871212942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa958101c00 0x7fa958198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa957fff640 1 -- 172.21.15.152:0/871212942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9581025c0 msgr2=0x7fa95819f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:56.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa957fff640 1 --2- 172.21.15.152:0/871212942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9581025c0 0x7fa95819f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:56.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa957fff640 1 -- 172.21.15.152:0/871212942 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95819fa50 con 0x7fa958100a10 2024-09-11T22:52:56.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa957fff640 1 --2- 172.21.15.152:0/871212942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa958100a10 0x7fa958197e30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa94800d770 tx=0x7fa94800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:56.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa955ffb640 1 -- 172.21.15.152:0/871212942 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948017070 con 0x7fa958100a10 2024-09-11T22:52:56.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa955ffb640 1 -- 172.21.15.152:0/871212942 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa948009c30 con 0x7fa958100a10 2024-09-11T22:52:56.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa955ffb640 1 -- 172.21.15.152:0/871212942 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948003dd0 con 0x7fa958100a10 2024-09-11T22:52:56.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.907+0000 7fa95e4dd640 1 -- 172.21.15.152:0/871212942 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95819fd40 con 0x7fa958100a10 2024-09-11T22:52:56.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.908+0000 7fa95e4dd640 1 -- 172.21.15.152:0/871212942 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9581a01d0 con 0x7fa958100a10 2024-09-11T22:52:56.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.909+0000 7fa95e4dd640 1 -- 172.21.15.152:0/871212942 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa958107da0 con 0x7fa958100a10 2024-09-11T22:52:56.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.910+0000 7fa955ffb640 1 -- 172.21.15.152:0/871212942 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 137954+0+0 (secure 0 0 0) 0x7fa948003fb0 con 0x7fa958100a10 2024-09-11T22:52:56.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.910+0000 7fa955ffb640 1 --2- 172.21.15.152:0/871212942 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa9440a2d80 0x7fa9440a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:56.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.911+0000 7fa94ffff640 1 --2- 172.21.15.152:0/871212942 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa9440a2d80 0x7fa9440a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:56.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.911+0000 7fa955ffb640 1 -- 172.21.15.152:0/871212942 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa9480cedb0 con 0x7fa958100a10 2024-09-11T22:52:56.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.911+0000 7fa94ffff640 1 --2- 172.21.15.152:0/871212942 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa9440a2d80 0x7fa9440a5240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa940005fd0 tx=0x7fa940005850 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:56.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:56.914+0000 7fa955ffb640 1 -- 172.21.15.152:0/871212942 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa9480d3070 con 0x7fa958100a10 2024-09-11T22:52:57.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:57.089+0000 7fa95e4dd640 1 -- 172.21.15.152:0/871212942 --> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa958062630 con 0x7fa9440a2d80 2024-09-11T22:52:57.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:57.095+0000 7fa955ffb640 1 -- 172.21.15.152:0/871212942 <== mgr.15240 v2:172.21.15.152:6800/4130474708 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258049 (secure 0 0 0) 0x7fa958062630 con 0x7fa9440a2d80 2024-09-11T22:52:57.099 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:57.101 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-09-11T22:52:57.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:57.100+0000 7fa95e4dd640 1 -- 172.21.15.152:0/871212942 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa9440a2d80 msgr2=0x7fa9440a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:57.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:57.100+0000 7fa95e4dd640 1 --2- 172.21.15.152:0/871212942 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa9440a2d80 0x7fa9440a5240 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa940005fd0 tx=0x7fa940005850 comp rx=0 tx=0).stop 2024-09-11T22:52:57.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:57.100+0000 7fa95e4dd640 1 -- 172.21.15.152:0/871212942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa958100a10 msgr2=0x7fa958197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:57.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:57.100+0000 7fa95e4dd640 1 --2- 172.21.15.152:0/871212942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa958100a10 0x7fa958197e30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa94800d770 tx=0x7fa94800dc30 comp rx=0 tx=0).stop 2024-09-11T22:52:57.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:57.100+0000 7fa95e4dd640 1 -- 172.21.15.152:0/871212942 shutdown_connections 2024-09-11T22:52:57.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:57.100+0000 7fa95e4dd640 1 --2- 172.21.15.152:0/871212942 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa9440a2d80 0x7fa9440a5240 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:57.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:57.100+0000 7fa95e4dd640 1 --2- 172.21.15.152:0/871212942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9581025c0 0x7fa95819f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:57.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:57.100+0000 7fa95e4dd640 1 --2- 172.21.15.152:0/871212942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa958101c00 0x7fa958198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:57.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:57.100+0000 7fa95e4dd640 1 --2- 172.21.15.152:0/871212942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa958100a10 0x7fa958197e30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:57.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:57.100+0000 7fa95e4dd640 1 -- 172.21.15.152:0/871212942 >> 172.21.15.152:0/871212942 conn(0x7fa9580fc170 msgr2=0x7fa958104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:57.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:57.100+0000 7fa95e4dd640 1 -- 172.21.15.152:0/871212942 shutdown_connections 2024-09-11T22:52:57.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:57.100+0000 7fa95e4dd640 1 -- 172.21.15.152:0/871212942 wait complete. 2024-09-11T22:52:57.115 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-11T22:52:57.086949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193417,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1363,"num_read_kb":848,"num_write":561,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":944,"ondisk_log_size":944,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369080,"kb_used_data":11472,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572744,"statfs":{"total":1151940427776,"available":1151562489856,"internally_reserved":0,"allocated":11747328,"data_stored":8912731,"data_compressed":215322,"data_compressed_allocated":278528,"data_compressed_original":3407872,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.973199"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169130+0000","last_change":"2024-09-11T22:50:54.171410+0000","last_active":"2024-09-11T22:52:53.169130+0000","last_peered":"2024-09-11T22:52:53.169130+0000","last_clean":"2024-09-11T22:52:53.169130+0000","last_became_active":"2024-09-11T22:50:54.171055+0000","last_became_peered":"2024-09-11T22:50:54.171055+0000","last_unstale":"2024-09-11T22:52:53.169130+0000","last_undegraded":"2024-09-11T22:52:53.169130+0000","last_fullsized":"2024-09-11T22:52:53.169130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:54.767454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131329+0000","last_change":"2024-09-11T22:50:54.168534+0000","last_active":"2024-09-11T22:52:53.131329+0000","last_peered":"2024-09-11T22:52:53.131329+0000","last_clean":"2024-09-11T22:52:53.131329+0000","last_became_active":"2024-09-11T22:50:54.168228+0000","last_became_peered":"2024-09-11T22:50:54.168228+0000","last_unstale":"2024-09-11T22:52:53.131329+0000","last_undegraded":"2024-09-11T22:52:53.131329+0000","last_fullsized":"2024-09-11T22:52:53.131329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:11.380048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131130+0000","last_change":"2024-09-11T22:50:54.168975+0000","last_active":"2024-09-11T22:52:53.131130+0000","last_peered":"2024-09-11T22:52:53.131130+0000","last_clean":"2024-09-11T22:52:53.131130+0000","last_became_active":"2024-09-11T22:50:54.168676+0000","last_became_peered":"2024-09-11T22:50:54.168676+0000","last_unstale":"2024-09-11T22:52:53.131130+0000","last_undegraded":"2024-09-11T22:52:53.131130+0000","last_fullsized":"2024-09-11T22:52:53.131130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:15.848273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130051+0000","last_change":"2024-09-11T22:50:54.164164+0000","last_active":"2024-09-11T22:52:53.130051+0000","last_peered":"2024-09-11T22:52:53.130051+0000","last_clean":"2024-09-11T22:52:53.130051+0000","last_became_active":"2024-09-11T22:50:54.163805+0000","last_became_peered":"2024-09-11T22:50:54.163805+0000","last_unstale":"2024-09-11T22:52:53.130051+0000","last_undegraded":"2024-09-11T22:52:53.130051+0000","last_fullsized":"2024-09-11T22:52:53.130051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:54.954046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074671+0000","last_change":"2024-09-11T22:50:54.167289+0000","last_active":"2024-09-11T22:52:53.074671+0000","last_peered":"2024-09-11T22:52:53.074671+0000","last_clean":"2024-09-11T22:52:53.074671+0000","last_became_active":"2024-09-11T22:50:54.166958+0000","last_became_peered":"2024-09-11T22:50:54.166958+0000","last_unstale":"2024-09-11T22:52:53.074671+0000","last_undegraded":"2024-09-11T22:52:53.074671+0000","last_fullsized":"2024-09-11T22:52:53.074671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:25.261241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129676+0000","last_change":"2024-09-11T22:50:54.175605+0000","last_active":"2024-09-11T22:52:53.129676+0000","last_peered":"2024-09-11T22:52:53.129676+0000","last_clean":"2024-09-11T22:52:53.129676+0000","last_became_active":"2024-09-11T22:50:54.175207+0000","last_became_peered":"2024-09-11T22:50:54.175207+0000","last_unstale":"2024-09-11T22:52:53.129676+0000","last_undegraded":"2024-09-11T22:52:53.129676+0000","last_fullsized":"2024-09-11T22:52:53.129676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:47.008910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129084+0000","last_change":"2024-09-11T22:50:54.167930+0000","last_active":"2024-09-11T22:52:53.129084+0000","last_peered":"2024-09-11T22:52:53.129084+0000","last_clean":"2024-09-11T22:52:53.129084+0000","last_became_active":"2024-09-11T22:50:54.167082+0000","last_became_peered":"2024-09-11T22:50:54.167082+0000","last_unstale":"2024-09-11T22:52:53.129084+0000","last_undegraded":"2024-09-11T22:52:53.129084+0000","last_fullsized":"2024-09-11T22:52:53.129084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:50.931311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129336+0000","last_change":"2024-09-11T22:50:54.172528+0000","last_active":"2024-09-11T22:52:53.129336+0000","last_peered":"2024-09-11T22:52:53.129336+0000","last_clean":"2024-09-11T22:52:53.129336+0000","last_became_active":"2024-09-11T22:50:54.172257+0000","last_became_peered":"2024-09-11T22:50:54.172257+0000","last_unstale":"2024-09-11T22:52:53.129336+0000","last_undegraded":"2024-09-11T22:52:53.129336+0000","last_fullsized":"2024-09-11T22:52:53.129336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:13:43.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074665+0000","last_change":"2024-09-11T22:50:54.163357+0000","last_active":"2024-09-11T22:52:53.074665+0000","last_peered":"2024-09-11T22:52:53.074665+0000","last_clean":"2024-09-11T22:52:53.074665+0000","last_became_active":"2024-09-11T22:50:54.162943+0000","last_became_peered":"2024-09-11T22:50:54.162943+0000","last_unstale":"2024-09-11T22:52:53.074665+0000","last_undegraded":"2024-09-11T22:52:53.074665+0000","last_fullsized":"2024-09-11T22:52:53.074665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:57.055802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129448+0000","last_change":"2024-09-11T22:50:54.177773+0000","last_active":"2024-09-11T22:52:53.129448+0000","last_peered":"2024-09-11T22:52:53.129448+0000","last_clean":"2024-09-11T22:52:53.129448+0000","last_became_active":"2024-09-11T22:50:54.177388+0000","last_became_peered":"2024-09-11T22:50:54.177388+0000","last_unstale":"2024-09-11T22:52:53.129448+0000","last_undegraded":"2024-09-11T22:52:53.129448+0000","last_fullsized":"2024-09-11T22:52:53.129448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:36:44.625089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129792+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:52:53.129792+0000","last_peered":"2024-09-11T22:52:53.129792+0000","last_clean":"2024-09-11T22:52:53.129792+0000","last_became_active":"2024-09-11T22:50:54.171491+0000","last_became_peered":"2024-09-11T22:50:54.171491+0000","last_unstale":"2024-09-11T22:52:53.129792+0000","last_undegraded":"2024-09-11T22:52:53.129792+0000","last_fullsized":"2024-09-11T22:52:53.129792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.996640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130199+0000","last_change":"2024-09-11T22:50:54.171260+0000","last_active":"2024-09-11T22:52:53.130199+0000","last_peered":"2024-09-11T22:52:53.130199+0000","last_clean":"2024-09-11T22:52:53.130199+0000","last_became_active":"2024-09-11T22:50:54.170733+0000","last_became_peered":"2024-09-11T22:50:54.170733+0000","last_unstale":"2024-09-11T22:52:53.130199+0000","last_undegraded":"2024-09-11T22:52:53.130199+0000","last_fullsized":"2024-09-11T22:52:53.130199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:57.877280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129997+0000","last_change":"2024-09-11T22:50:54.170576+0000","last_active":"2024-09-11T22:52:53.129997+0000","last_peered":"2024-09-11T22:52:53.129997+0000","last_clean":"2024-09-11T22:52:53.129997+0000","last_became_active":"2024-09-11T22:50:54.170239+0000","last_became_peered":"2024-09-11T22:50:54.170239+0000","last_unstale":"2024-09-11T22:52:53.129997+0000","last_undegraded":"2024-09-11T22:52:53.129997+0000","last_fullsized":"2024-09-11T22:52:53.129997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:09.683171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130651+0000","last_change":"2024-09-11T22:50:54.167176+0000","last_active":"2024-09-11T22:52:53.130651+0000","last_peered":"2024-09-11T22:52:53.130651+0000","last_clean":"2024-09-11T22:52:53.130651+0000","last_became_active":"2024-09-11T22:50:54.166894+0000","last_became_peered":"2024-09-11T22:50:54.166894+0000","last_unstale":"2024-09-11T22:52:53.130651+0000","last_undegraded":"2024-09-11T22:52:53.130651+0000","last_fullsized":"2024-09-11T22:52:53.130651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:43:08.158786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130901+0000","last_change":"2024-09-11T22:50:54.163100+0000","last_active":"2024-09-11T22:52:53.130901+0000","last_peered":"2024-09-11T22:52:53.130901+0000","last_clean":"2024-09-11T22:52:53.130901+0000","last_became_active":"2024-09-11T22:50:54.162776+0000","last_became_peered":"2024-09-11T22:50:54.162776+0000","last_unstale":"2024-09-11T22:52:53.130901+0000","last_undegraded":"2024-09-11T22:52:53.130901+0000","last_fullsized":"2024-09-11T22:52:53.130901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:15.215305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074034+0000","last_change":"2024-09-11T22:50:54.166013+0000","last_active":"2024-09-11T22:52:53.074034+0000","last_peered":"2024-09-11T22:52:53.074034+0000","last_clean":"2024-09-11T22:52:53.074034+0000","last_became_active":"2024-09-11T22:50:54.165680+0000","last_became_peered":"2024-09-11T22:50:54.165680+0000","last_unstale":"2024-09-11T22:52:53.074034+0000","last_undegraded":"2024-09-11T22:52:53.074034+0000","last_fullsized":"2024-09-11T22:52:53.074034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:12.192231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169175+0000","last_change":"2024-09-11T22:50:54.172105+0000","last_active":"2024-09-11T22:52:53.169175+0000","last_peered":"2024-09-11T22:52:53.169175+0000","last_clean":"2024-09-11T22:52:53.169175+0000","last_became_active":"2024-09-11T22:50:54.171782+0000","last_became_peered":"2024-09-11T22:50:54.171782+0000","last_unstale":"2024-09-11T22:52:53.169175+0000","last_undegraded":"2024-09-11T22:52:53.169175+0000","last_fullsized":"2024-09-11T22:52:53.169175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:17.152834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130880+0000","last_change":"2024-09-11T22:50:54.167184+0000","last_active":"2024-09-11T22:52:53.130880+0000","last_peered":"2024-09-11T22:52:53.130880+0000","last_clean":"2024-09-11T22:52:53.130880+0000","last_became_active":"2024-09-11T22:50:54.166878+0000","last_became_peered":"2024-09-11T22:50:54.166878+0000","last_unstale":"2024-09-11T22:52:53.130880+0000","last_undegraded":"2024-09-11T22:52:53.130880+0000","last_fullsized":"2024-09-11T22:52:53.130880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:44.046552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129108+0000","last_change":"2024-09-11T22:50:54.176260+0000","last_active":"2024-09-11T22:52:53.129108+0000","last_peered":"2024-09-11T22:52:53.129108+0000","last_clean":"2024-09-11T22:52:53.129108+0000","last_became_active":"2024-09-11T22:50:54.175873+0000","last_became_peered":"2024-09-11T22:50:54.175873+0000","last_unstale":"2024-09-11T22:52:53.129108+0000","last_undegraded":"2024-09-11T22:52:53.129108+0000","last_fullsized":"2024-09-11T22:52:53.129108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:05.258677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168910+0000","last_change":"2024-09-11T22:50:54.172725+0000","last_active":"2024-09-11T22:52:53.168910+0000","last_peered":"2024-09-11T22:52:53.168910+0000","last_clean":"2024-09-11T22:52:53.168910+0000","last_became_active":"2024-09-11T22:50:54.172072+0000","last_became_peered":"2024-09-11T22:50:54.172072+0000","last_unstale":"2024-09-11T22:52:53.168910+0000","last_undegraded":"2024-09-11T22:52:53.168910+0000","last_fullsized":"2024-09-11T22:52:53.168910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:53.699670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130148+0000","last_change":"2024-09-11T22:50:54.165118+0000","last_active":"2024-09-11T22:52:53.130148+0000","last_peered":"2024-09-11T22:52:53.130148+0000","last_clean":"2024-09-11T22:52:53.130148+0000","last_became_active":"2024-09-11T22:50:54.164793+0000","last_became_peered":"2024-09-11T22:50:54.164793+0000","last_unstale":"2024-09-11T22:52:53.130148+0000","last_undegraded":"2024-09-11T22:52:53.130148+0000","last_fullsized":"2024-09-11T22:52:53.130148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:42.883940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074224+0000","last_change":"2024-09-11T22:50:54.176900+0000","last_active":"2024-09-11T22:52:53.074224+0000","last_peered":"2024-09-11T22:52:53.074224+0000","last_clean":"2024-09-11T22:52:53.074224+0000","last_became_active":"2024-09-11T22:50:54.176595+0000","last_became_peered":"2024-09-11T22:50:54.176595+0000","last_unstale":"2024-09-11T22:52:53.074224+0000","last_undegraded":"2024-09-11T22:52:53.074224+0000","last_fullsized":"2024-09-11T22:52:53.074224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:36.925945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129485+0000","last_change":"2024-09-11T22:50:54.167656+0000","last_active":"2024-09-11T22:52:53.129485+0000","last_peered":"2024-09-11T22:52:53.129485+0000","last_clean":"2024-09-11T22:52:53.129485+0000","last_became_active":"2024-09-11T22:50:54.167349+0000","last_became_peered":"2024-09-11T22:50:54.167349+0000","last_unstale":"2024-09-11T22:52:53.129485+0000","last_undegraded":"2024-09-11T22:52:53.129485+0000","last_fullsized":"2024-09-11T22:52:53.129485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:38.100218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073470+0000","last_change":"2024-09-11T22:50:54.177059+0000","last_active":"2024-09-11T22:52:53.073470+0000","last_peered":"2024-09-11T22:52:53.073470+0000","last_clean":"2024-09-11T22:52:53.073470+0000","last_became_active":"2024-09-11T22:50:54.176726+0000","last_became_peered":"2024-09-11T22:50:54.176726+0000","last_unstale":"2024-09-11T22:52:53.073470+0000","last_undegraded":"2024-09-11T22:52:53.073470+0000","last_fullsized":"2024-09-11T22:52:53.073470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:27:58.058944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130570+0000","last_change":"2024-09-11T22:50:54.167566+0000","last_active":"2024-09-11T22:52:53.130570+0000","last_peered":"2024-09-11T22:52:53.130570+0000","last_clean":"2024-09-11T22:52:53.130570+0000","last_became_active":"2024-09-11T22:50:54.167049+0000","last_became_peered":"2024-09-11T22:50:54.167049+0000","last_unstale":"2024-09-11T22:52:53.130570+0000","last_undegraded":"2024-09-11T22:52:53.130570+0000","last_fullsized":"2024-09-11T22:52:53.130570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:06.833099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129193+0000","last_change":"2024-09-11T22:50:54.169732+0000","last_active":"2024-09-11T22:52:53.129193+0000","last_peered":"2024-09-11T22:52:53.129193+0000","last_clean":"2024-09-11T22:52:53.129193+0000","last_became_active":"2024-09-11T22:50:54.169437+0000","last_became_peered":"2024-09-11T22:50:54.169437+0000","last_unstale":"2024-09-11T22:52:53.129193+0000","last_undegraded":"2024-09-11T22:52:53.129193+0000","last_fullsized":"2024-09-11T22:52:53.129193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:25.553387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169106+0000","last_change":"2024-09-11T22:50:54.166953+0000","last_active":"2024-09-11T22:52:53.169106+0000","last_peered":"2024-09-11T22:52:53.169106+0000","last_clean":"2024-09-11T22:52:53.169106+0000","last_became_active":"2024-09-11T22:50:54.166472+0000","last_became_peered":"2024-09-11T22:50:54.166472+0000","last_unstale":"2024-09-11T22:52:53.169106+0000","last_undegraded":"2024-09-11T22:52:53.169106+0000","last_fullsized":"2024-09-11T22:52:53.169106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:01:20.234180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129423+0000","last_change":"2024-09-11T22:50:54.171308+0000","last_active":"2024-09-11T22:52:53.129423+0000","last_peered":"2024-09-11T22:52:53.129423+0000","last_clean":"2024-09-11T22:52:53.129423+0000","last_became_active":"2024-09-11T22:50:54.168958+0000","last_became_peered":"2024-09-11T22:50:54.168958+0000","last_unstale":"2024-09-11T22:52:53.129423+0000","last_undegraded":"2024-09-11T22:52:53.129423+0000","last_fullsized":"2024-09-11T22:52:53.129423+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:20.224003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131183+0000","last_change":"2024-09-11T22:50:54.173846+0000","last_active":"2024-09-11T22:52:53.131183+0000","last_peered":"2024-09-11T22:52:53.131183+0000","last_clean":"2024-09-11T22:52:53.131183+0000","last_became_active":"2024-09-11T22:50:54.173542+0000","last_became_peered":"2024-09-11T22:50:54.173542+0000","last_unstale":"2024-09-11T22:52:53.131183+0000","last_undegraded":"2024-09-11T22:52:53.131183+0000","last_fullsized":"2024-09-11T22:52:53.131183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:29:45.775902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.128929+0000","last_change":"2024-09-11T22:50:54.176008+0000","last_active":"2024-09-11T22:52:53.128929+0000","last_peered":"2024-09-11T22:52:53.128929+0000","last_clean":"2024-09-11T22:52:53.128929+0000","last_became_active":"2024-09-11T22:50:54.175672+0000","last_became_peered":"2024-09-11T22:50:54.175672+0000","last_unstale":"2024-09-11T22:52:53.128929+0000","last_undegraded":"2024-09-11T22:52:53.128929+0000","last_fullsized":"2024-09-11T22:52:53.128929+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:35.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.128953+0000","last_change":"2024-09-11T22:50:54.162504+0000","last_active":"2024-09-11T22:52:53.128953+0000","last_peered":"2024-09-11T22:52:53.128953+0000","last_clean":"2024-09-11T22:52:53.128953+0000","last_became_active":"2024-09-11T22:50:54.162190+0000","last_became_peered":"2024-09-11T22:50:54.162190+0000","last_unstale":"2024-09-11T22:52:53.128953+0000","last_undegraded":"2024-09-11T22:52:53.128953+0000","last_fullsized":"2024-09-11T22:52:53.128953+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:52.138692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168828+0000","last_change":"2024-09-11T22:51:55.960882+0000","last_active":"2024-09-11T22:52:53.168828+0000","last_peered":"2024-09-11T22:52:53.168828+0000","last_clean":"2024-09-11T22:52:53.168828+0000","last_became_active":"2024-09-11T22:51:55.960246+0000","last_became_peered":"2024-09-11T22:51:55.960246+0000","last_unstale":"2024-09-11T22:52:53.168828+0000","last_undegraded":"2024-09-11T22:52:53.168828+0000","last_fullsized":"2024-09-11T22:52:53.168828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:21.247840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074402+0000","last_change":"2024-09-11T22:50:54.160696+0000","last_active":"2024-09-11T22:52:53.074402+0000","last_peered":"2024-09-11T22:52:53.074402+0000","last_clean":"2024-09-11T22:52:53.074402+0000","last_became_active":"2024-09-11T22:50:54.159249+0000","last_became_peered":"2024-09-11T22:50:54.159249+0000","last_unstale":"2024-09-11T22:52:53.074402+0000","last_undegraded":"2024-09-11T22:52:53.074402+0000","last_fullsized":"2024-09-11T22:52:53.074402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:09.649208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'9","reported_seq":146,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.331651+0000","last_change":"2024-09-11T22:50:52.138986+0000","last_active":"2024-09-11T22:52:53.331651+0000","last_peered":"2024-09-11T22:52:53.331651+0000","last_clean":"2024-09-11T22:52:53.331651+0000","last_became_active":"2024-09-11T22:50:52.138595+0000","last_became_peered":"2024-09-11T22:50:52.138595+0000","last_unstale":"2024-09-11T22:52:53.331651+0000","last_undegraded":"2024-09-11T22:52:53.331651+0000","last_fullsized":"2024-09-11T22:52:53.331651+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:06.232146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1942,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073853+0000","last_change":"2024-09-11T22:50:54.176806+0000","last_active":"2024-09-11T22:52:53.073853+0000","last_peered":"2024-09-11T22:52:53.073853+0000","last_clean":"2024-09-11T22:52:53.073853+0000","last_became_active":"2024-09-11T22:50:54.176513+0000","last_became_peered":"2024-09-11T22:50:54.176513+0000","last_unstale":"2024-09-11T22:52:53.073853+0000","last_undegraded":"2024-09-11T22:52:53.073853+0000","last_fullsized":"2024-09-11T22:52:53.073853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:43:12.086145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129699+0000","last_change":"2024-09-11T22:50:52.142794+0000","last_active":"2024-09-11T22:52:53.129699+0000","last_peered":"2024-09-11T22:52:53.129699+0000","last_clean":"2024-09-11T22:52:53.129699+0000","last_became_active":"2024-09-11T22:50:52.142433+0000","last_became_peered":"2024-09-11T22:50:52.142433+0000","last_unstale":"2024-09-11T22:52:53.129699+0000","last_undegraded":"2024-09-11T22:52:53.129699+0000","last_fullsized":"2024-09-11T22:52:53.129699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:47:12.090055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130326+0000","last_change":"2024-09-11T22:50:52.139789+0000","last_active":"2024-09-11T22:52:53.130326+0000","last_peered":"2024-09-11T22:52:53.130326+0000","last_clean":"2024-09-11T22:52:53.130326+0000","last_became_active":"2024-09-11T22:50:52.139294+0000","last_became_peered":"2024-09-11T22:50:52.139294+0000","last_unstale":"2024-09-11T22:52:53.130326+0000","last_undegraded":"2024-09-11T22:52:53.130326+0000","last_fullsized":"2024-09-11T22:52:53.130326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:07.355155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129486+0000","last_change":"2024-09-11T22:50:54.166248+0000","last_active":"2024-09-11T22:52:53.129486+0000","last_peered":"2024-09-11T22:52:53.129486+0000","last_clean":"2024-09-11T22:52:53.129486+0000","last_became_active":"2024-09-11T22:50:54.165958+0000","last_became_peered":"2024-09-11T22:50:54.165958+0000","last_unstale":"2024-09-11T22:52:53.129486+0000","last_undegraded":"2024-09-11T22:52:53.129486+0000","last_fullsized":"2024-09-11T22:52:53.129486+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:29.531741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073597+0000","last_change":"2024-09-11T22:50:52.140340+0000","last_active":"2024-09-11T22:52:53.073597+0000","last_peered":"2024-09-11T22:52:53.073597+0000","last_clean":"2024-09-11T22:52:53.073597+0000","last_became_active":"2024-09-11T22:50:52.140011+0000","last_became_peered":"2024-09-11T22:50:52.140011+0000","last_unstale":"2024-09-11T22:52:53.073597+0000","last_undegraded":"2024-09-11T22:52:53.073597+0000","last_fullsized":"2024-09-11T22:52:53.073597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:58.862640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129625+0000","last_change":"2024-09-11T22:50:54.166998+0000","last_active":"2024-09-11T22:52:53.129625+0000","last_peered":"2024-09-11T22:52:53.129625+0000","last_clean":"2024-09-11T22:52:53.129625+0000","last_became_active":"2024-09-11T22:50:54.166682+0000","last_became_peered":"2024-09-11T22:50:54.166682+0000","last_unstale":"2024-09-11T22:52:53.129625+0000","last_undegraded":"2024-09-11T22:52:53.129625+0000","last_fullsized":"2024-09-11T22:52:53.129625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:57.815914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'8","reported_seq":250,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.332242+0000","last_change":"2024-09-11T22:50:52.141380+0000","last_active":"2024-09-11T22:52:53.332242+0000","last_peered":"2024-09-11T22:52:53.332242+0000","last_clean":"2024-09-11T22:52:53.332242+0000","last_became_active":"2024-09-11T22:50:52.139925+0000","last_became_peered":"2024-09-11T22:50:52.139925+0000","last_unstale":"2024-09-11T22:52:53.332242+0000","last_undegraded":"2024-09-11T22:52:53.332242+0000","last_fullsized":"2024-09-11T22:52:53.332242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:40:38.395055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":218,"num_read_kb":218,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074552+0000","last_change":"2024-09-11T22:50:54.167909+0000","last_active":"2024-09-11T22:52:53.074552+0000","last_peered":"2024-09-11T22:52:53.074552+0000","last_clean":"2024-09-11T22:52:53.074552+0000","last_became_active":"2024-09-11T22:50:54.167571+0000","last_became_peered":"2024-09-11T22:50:54.167571+0000","last_unstale":"2024-09-11T22:52:53.074552+0000","last_undegraded":"2024-09-11T22:52:53.074552+0000","last_fullsized":"2024-09-11T22:52:53.074552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:34.179960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129913+0000","last_change":"2024-09-11T22:50:52.144545+0000","last_active":"2024-09-11T22:52:53.129913+0000","last_peered":"2024-09-11T22:52:53.129913+0000","last_clean":"2024-09-11T22:52:53.129913+0000","last_became_active":"2024-09-11T22:50:52.144221+0000","last_became_peered":"2024-09-11T22:50:52.144221+0000","last_unstale":"2024-09-11T22:52:53.129913+0000","last_undegraded":"2024-09-11T22:52:53.129913+0000","last_fullsized":"2024-09-11T22:52:53.129913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:29.509165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131618+0000","last_change":"2024-09-11T22:50:54.164132+0000","last_active":"2024-09-11T22:52:53.131618+0000","last_peered":"2024-09-11T22:52:53.131618+0000","last_clean":"2024-09-11T22:52:53.131618+0000","last_became_active":"2024-09-11T22:50:54.163842+0000","last_became_peered":"2024-09-11T22:50:54.163842+0000","last_unstale":"2024-09-11T22:52:53.131618+0000","last_undegraded":"2024-09-11T22:52:53.131618+0000","last_fullsized":"2024-09-11T22:52:53.131618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:28:55.490236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074349+0000","last_change":"2024-09-11T22:50:52.264611+0000","last_active":"2024-09-11T22:52:53.074349+0000","last_peered":"2024-09-11T22:52:53.074349+0000","last_clean":"2024-09-11T22:52:53.074349+0000","last_became_active":"2024-09-11T22:50:52.264060+0000","last_became_peered":"2024-09-11T22:50:52.264060+0000","last_unstale":"2024-09-11T22:52:53.074349+0000","last_undegraded":"2024-09-11T22:52:53.074349+0000","last_fullsized":"2024-09-11T22:52:53.074349+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:42.543997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074271+0000","last_change":"2024-09-11T22:51:56.206155+0000","last_active":"2024-09-11T22:52:53.074271+0000","last_peered":"2024-09-11T22:52:53.074271+0000","last_clean":"2024-09-11T22:52:53.074271+0000","last_became_active":"2024-09-11T22:51:56.205797+0000","last_became_peered":"2024-09-11T22:51:56.205797+0000","last_unstale":"2024-09-11T22:52:53.074271+0000","last_undegraded":"2024-09-11T22:52:53.074271+0000","last_fullsized":"2024-09-11T22:52:53.074271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:28.063825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'39","reported_seq":293,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.333681+0000","last_change":"2024-09-11T22:50:52.141782+0000","last_active":"2024-09-11T22:52:53.333681+0000","last_peered":"2024-09-11T22:52:53.333681+0000","last_clean":"2024-09-11T22:52:53.333681+0000","last_became_active":"2024-09-11T22:50:52.140662+0000","last_became_peered":"2024-09-11T22:50:52.140662+0000","last_unstale":"2024-09-11T22:52:53.333681+0000","last_undegraded":"2024-09-11T22:52:53.333681+0000","last_fullsized":"2024-09-11T22:52:53.333681+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:21.158577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68544,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":230,"num_read_kb":180,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074760+0000","last_change":"2024-09-11T22:50:54.166678+0000","last_active":"2024-09-11T22:52:53.074760+0000","last_peered":"2024-09-11T22:52:53.074760+0000","last_clean":"2024-09-11T22:52:53.074760+0000","last_became_active":"2024-09-11T22:50:54.165777+0000","last_became_peered":"2024-09-11T22:50:54.165777+0000","last_unstale":"2024-09-11T22:52:53.074760+0000","last_undegraded":"2024-09-11T22:52:53.074760+0000","last_fullsized":"2024-09-11T22:52:53.074760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:07.224563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129051+0000","last_change":"2024-09-11T22:50:52.141292+0000","last_active":"2024-09-11T22:52:53.129051+0000","last_peered":"2024-09-11T22:52:53.129051+0000","last_clean":"2024-09-11T22:52:53.129051+0000","last_became_active":"2024-09-11T22:50:52.140971+0000","last_became_peered":"2024-09-11T22:50:52.140971+0000","last_unstale":"2024-09-11T22:52:53.129051+0000","last_undegraded":"2024-09-11T22:52:53.129051+0000","last_fullsized":"2024-09-11T22:52:53.129051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:53.039684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129489+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:52:53.129489+0000","last_peered":"2024-09-11T22:52:53.129489+0000","last_clean":"2024-09-11T22:52:53.129489+0000","last_became_active":"2024-09-11T22:50:54.171520+0000","last_became_peered":"2024-09-11T22:50:54.171520+0000","last_unstale":"2024-09-11T22:52:53.129489+0000","last_undegraded":"2024-09-11T22:52:53.129489+0000","last_fullsized":"2024-09-11T22:52:53.129489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:56.594058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073605+0000","last_change":"2024-09-11T22:50:52.138725+0000","last_active":"2024-09-11T22:52:53.073605+0000","last_peered":"2024-09-11T22:52:53.073605+0000","last_clean":"2024-09-11T22:52:53.073605+0000","last_became_active":"2024-09-11T22:50:52.138367+0000","last_became_peered":"2024-09-11T22:50:52.138367+0000","last_unstale":"2024-09-11T22:52:53.073605+0000","last_undegraded":"2024-09-11T22:52:53.073605+0000","last_fullsized":"2024-09-11T22:52:53.073605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:58.811444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129217+0000","last_change":"2024-09-11T22:50:54.170524+0000","last_active":"2024-09-11T22:52:53.129217+0000","last_peered":"2024-09-11T22:52:53.129217+0000","last_clean":"2024-09-11T22:52:53.129217+0000","last_became_active":"2024-09-11T22:50:54.169947+0000","last_became_peered":"2024-09-11T22:50:54.169947+0000","last_unstale":"2024-09-11T22:52:53.129217+0000","last_undegraded":"2024-09-11T22:52:53.129217+0000","last_fullsized":"2024-09-11T22:52:53.129217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:51.768718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":143,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.337266+0000","last_change":"2024-09-11T22:50:52.139182+0000","last_active":"2024-09-11T22:52:53.337266+0000","last_peered":"2024-09-11T22:52:53.337266+0000","last_clean":"2024-09-11T22:52:53.337266+0000","last_became_active":"2024-09-11T22:50:52.138799+0000","last_became_peered":"2024-09-11T22:50:52.138799+0000","last_unstale":"2024-09-11T22:52:53.337266+0000","last_undegraded":"2024-09-11T22:52:53.337266+0000","last_fullsized":"2024-09-11T22:52:53.337266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:36.376162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129122+0000","last_change":"2024-09-11T22:50:54.168797+0000","last_active":"2024-09-11T22:52:53.129122+0000","last_peered":"2024-09-11T22:52:53.129122+0000","last_clean":"2024-09-11T22:52:53.129122+0000","last_became_active":"2024-09-11T22:50:54.168408+0000","last_became_peered":"2024-09-11T22:50:54.168408+0000","last_unstale":"2024-09-11T22:52:53.129122+0000","last_undegraded":"2024-09-11T22:52:53.129122+0000","last_fullsized":"2024-09-11T22:52:53.129122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:51.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074207+0000","last_change":"2024-09-11T22:50:52.264693+0000","last_active":"2024-09-11T22:52:53.074207+0000","last_peered":"2024-09-11T22:52:53.074207+0000","last_clean":"2024-09-11T22:52:53.074207+0000","last_became_active":"2024-09-11T22:50:52.264058+0000","last_became_peered":"2024-09-11T22:50:52.264058+0000","last_unstale":"2024-09-11T22:52:53.074207+0000","last_undegraded":"2024-09-11T22:52:53.074207+0000","last_fullsized":"2024-09-11T22:52:53.074207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:46.985921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073438+0000","last_change":"2024-09-11T22:51:55.961118+0000","last_active":"2024-09-11T22:52:53.073438+0000","last_peered":"2024-09-11T22:52:53.073438+0000","last_clean":"2024-09-11T22:52:53.073438+0000","last_became_active":"2024-09-11T22:51:55.960614+0000","last_became_peered":"2024-09-11T22:51:55.960614+0000","last_unstale":"2024-09-11T22:52:53.073438+0000","last_undegraded":"2024-09-11T22:52:53.073438+0000","last_fullsized":"2024-09-11T22:52:53.073438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:47.184327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.075042+0000","last_change":"2024-09-11T22:50:52.133049+0000","last_active":"2024-09-11T22:52:53.075042+0000","last_peered":"2024-09-11T22:52:53.075042+0000","last_clean":"2024-09-11T22:52:53.075042+0000","last_became_active":"2024-09-11T22:50:52.132647+0000","last_became_peered":"2024-09-11T22:50:52.132647+0000","last_unstale":"2024-09-11T22:52:53.075042+0000","last_undegraded":"2024-09-11T22:52:53.075042+0000","last_fullsized":"2024-09-11T22:52:53.075042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:13.213003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169115+0000","last_change":"2024-09-11T22:51:56.238023+0000","last_active":"2024-09-11T22:52:53.169115+0000","last_peered":"2024-09-11T22:52:53.169115+0000","last_clean":"2024-09-11T22:52:53.169115+0000","last_became_active":"2024-09-11T22:51:56.237676+0000","last_became_peered":"2024-09-11T22:51:56.237676+0000","last_unstale":"2024-09-11T22:52:53.169115+0000","last_undegraded":"2024-09-11T22:52:53.169115+0000","last_fullsized":"2024-09-11T22:52:53.169115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:16.215961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130047+0000","last_change":"2024-09-11T22:50:52.139766+0000","last_active":"2024-09-11T22:52:53.130047+0000","last_peered":"2024-09-11T22:52:53.130047+0000","last_clean":"2024-09-11T22:52:53.130047+0000","last_became_active":"2024-09-11T22:50:52.139456+0000","last_became_peered":"2024-09-11T22:50:52.139456+0000","last_unstale":"2024-09-11T22:52:53.130047+0000","last_undegraded":"2024-09-11T22:52:53.130047+0000","last_fullsized":"2024-09-11T22:52:53.130047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:08.358649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130080+0000","last_change":"2024-09-11T22:50:54.167717+0000","last_active":"2024-09-11T22:52:53.130080+0000","last_peered":"2024-09-11T22:52:53.130080+0000","last_clean":"2024-09-11T22:52:53.130080+0000","last_became_active":"2024-09-11T22:50:54.167331+0000","last_became_peered":"2024-09-11T22:50:54.167331+0000","last_unstale":"2024-09-11T22:52:53.130080+0000","last_undegraded":"2024-09-11T22:52:53.130080+0000","last_fullsized":"2024-09-11T22:52:53.130080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:11.240351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'41","reported_seq":73,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.283886+0000","last_change":"2024-09-11T22:50:52.141682+0000","last_active":"2024-09-11T22:52:53.283886+0000","last_peered":"2024-09-11T22:52:53.283886+0000","last_clean":"2024-09-11T22:52:53.283886+0000","last_became_active":"2024-09-11T22:50:52.141251+0000","last_became_peered":"2024-09-11T22:50:52.141251+0000","last_unstale":"2024-09-11T22:52:53.283886+0000","last_undegraded":"2024-09-11T22:52:53.283886+0000","last_fullsized":"2024-09-11T22:52:53.283886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:03.068252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129793+0000","last_change":"2024-09-11T22:50:54.174149+0000","last_active":"2024-09-11T22:52:53.129793+0000","last_peered":"2024-09-11T22:52:53.129793+0000","last_clean":"2024-09-11T22:52:53.129793+0000","last_became_active":"2024-09-11T22:50:54.173765+0000","last_became_peered":"2024-09-11T22:50:54.173765+0000","last_unstale":"2024-09-11T22:52:53.129793+0000","last_undegraded":"2024-09-11T22:52:53.129793+0000","last_fullsized":"2024-09-11T22:52:53.129793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.314334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'8","reported_seq":145,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.279965+0000","last_change":"2024-09-11T22:50:52.435019+0000","last_active":"2024-09-11T22:52:53.279965+0000","last_peered":"2024-09-11T22:52:53.279965+0000","last_clean":"2024-09-11T22:52:53.279965+0000","last_became_active":"2024-09-11T22:50:52.434642+0000","last_became_peered":"2024-09-11T22:50:52.434642+0000","last_unstale":"2024-09-11T22:52:53.279965+0000","last_undegraded":"2024-09-11T22:52:53.279965+0000","last_fullsized":"2024-09-11T22:52:53.279965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:18.441841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":113,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130004+0000","last_change":"2024-09-11T22:50:54.171291+0000","last_active":"2024-09-11T22:52:53.130004+0000","last_peered":"2024-09-11T22:52:53.130004+0000","last_clean":"2024-09-11T22:52:53.130004+0000","last_became_active":"2024-09-11T22:50:54.170961+0000","last_became_peered":"2024-09-11T22:50:54.170961+0000","last_unstale":"2024-09-11T22:52:53.130004+0000","last_undegraded":"2024-09-11T22:52:53.130004+0000","last_fullsized":"2024-09-11T22:52:53.130004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:07.211936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129647+0000","last_change":"2024-09-11T22:49:39.098881+0000","last_active":"2024-09-11T22:52:53.129647+0000","last_peered":"2024-09-11T22:52:53.129647+0000","last_clean":"2024-09-11T22:52:53.129647+0000","last_became_active":"2024-09-11T22:49:39.098458+0000","last_became_peered":"2024-09-11T22:49:39.098458+0000","last_unstale":"2024-09-11T22:52:53.129647+0000","last_undegraded":"2024-09-11T22:52:53.129647+0000","last_fullsized":"2024-09-11T22:52:53.129647+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_clean_scrub_stamp":"2024-09-11T22:46:11.418577+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:41.059384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130035+0000","last_change":"2024-09-11T22:50:52.137691+0000","last_active":"2024-09-11T22:52:53.130035+0000","last_peered":"2024-09-11T22:52:53.130035+0000","last_clean":"2024-09-11T22:52:53.130035+0000","last_became_active":"2024-09-11T22:50:52.137282+0000","last_became_peered":"2024-09-11T22:50:52.137282+0000","last_unstale":"2024-09-11T22:52:53.130035+0000","last_undegraded":"2024-09-11T22:52:53.130035+0000","last_fullsized":"2024-09-11T22:52:53.130035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:00:27.651252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129789+0000","last_change":"2024-09-11T22:51:55.959937+0000","last_active":"2024-09-11T22:52:53.129789+0000","last_peered":"2024-09-11T22:52:53.129789+0000","last_clean":"2024-09-11T22:52:53.129789+0000","last_became_active":"2024-09-11T22:51:55.959575+0000","last_became_peered":"2024-09-11T22:51:55.959575+0000","last_unstale":"2024-09-11T22:52:53.129789+0000","last_undegraded":"2024-09-11T22:52:53.129789+0000","last_fullsized":"2024-09-11T22:52:53.129789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:54.097552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129633+0000","last_change":"2024-09-11T22:50:52.144324+0000","last_active":"2024-09-11T22:52:53.129633+0000","last_peered":"2024-09-11T22:52:53.129633+0000","last_clean":"2024-09-11T22:52:53.129633+0000","last_became_active":"2024-09-11T22:50:52.144011+0000","last_became_peered":"2024-09-11T22:50:52.144011+0000","last_unstale":"2024-09-11T22:52:53.129633+0000","last_undegraded":"2024-09-11T22:52:53.129633+0000","last_fullsized":"2024-09-11T22:52:53.129633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:29.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130192+0000","last_change":"2024-09-11T22:50:54.173287+0000","last_active":"2024-09-11T22:52:53.130192+0000","last_peered":"2024-09-11T22:52:53.130192+0000","last_clean":"2024-09-11T22:52:53.130192+0000","last_became_active":"2024-09-11T22:50:54.172953+0000","last_became_peered":"2024-09-11T22:50:54.172953+0000","last_unstale":"2024-09-11T22:52:53.130192+0000","last_undegraded":"2024-09-11T22:52:53.130192+0000","last_fullsized":"2024-09-11T22:52:53.130192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:24.314604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":141,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.336909+0000","last_change":"2024-09-11T22:50:52.263777+0000","last_active":"2024-09-11T22:52:53.336909+0000","last_peered":"2024-09-11T22:52:53.336909+0000","last_clean":"2024-09-11T22:52:53.336909+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:52:53.336909+0000","last_undegraded":"2024-09-11T22:52:53.336909+0000","last_fullsized":"2024-09-11T22:52:53.336909+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:37.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073403+0000","last_change":"2024-09-11T22:50:54.164927+0000","last_active":"2024-09-11T22:52:53.073403+0000","last_peered":"2024-09-11T22:52:53.073403+0000","last_clean":"2024-09-11T22:52:53.073403+0000","last_became_active":"2024-09-11T22:50:54.164602+0000","last_became_peered":"2024-09-11T22:50:54.164602+0000","last_unstale":"2024-09-11T22:52:53.073403+0000","last_undegraded":"2024-09-11T22:52:53.073403+0000","last_fullsized":"2024-09-11T22:52:53.073403+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:10.226232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":138,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129575+0000","last_change":"2024-09-11T22:50:52.141199+0000","last_active":"2024-09-11T22:52:53.129575+0000","last_peered":"2024-09-11T22:52:53.129575+0000","last_clean":"2024-09-11T22:52:53.129575+0000","last_became_active":"2024-09-11T22:50:52.140830+0000","last_became_peered":"2024-09-11T22:50:52.140830+0000","last_unstale":"2024-09-11T22:52:53.129575+0000","last_undegraded":"2024-09-11T22:52:53.129575+0000","last_fullsized":"2024-09-11T22:52:53.129575+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.767803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129565+0000","last_change":"2024-09-11T22:50:54.175054+0000","last_active":"2024-09-11T22:52:53.129565+0000","last_peered":"2024-09-11T22:52:53.129565+0000","last_clean":"2024-09-11T22:52:53.129565+0000","last_became_active":"2024-09-11T22:50:54.174715+0000","last_became_peered":"2024-09-11T22:50:54.174715+0000","last_unstale":"2024-09-11T22:52:53.129565+0000","last_undegraded":"2024-09-11T22:52:53.129565+0000","last_fullsized":"2024-09-11T22:52:53.129565+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:06.694988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130210+0000","last_change":"2024-09-11T22:51:55.961660+0000","last_active":"2024-09-11T22:52:53.130210+0000","last_peered":"2024-09-11T22:52:53.130210+0000","last_clean":"2024-09-11T22:52:53.130210+0000","last_became_active":"2024-09-11T22:51:55.961278+0000","last_became_peered":"2024-09-11T22:51:55.961278+0000","last_unstale":"2024-09-11T22:52:53.130210+0000","last_undegraded":"2024-09-11T22:52:53.130210+0000","last_fullsized":"2024-09-11T22:52:53.130210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:52.352490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129357+0000","last_change":"2024-09-11T22:50:54.167361+0000","last_active":"2024-09-11T22:52:53.129357+0000","last_peered":"2024-09-11T22:52:53.129357+0000","last_clean":"2024-09-11T22:52:53.129357+0000","last_became_active":"2024-09-11T22:50:54.166963+0000","last_became_peered":"2024-09-11T22:50:54.166963+0000","last_unstale":"2024-09-11T22:52:53.129357+0000","last_undegraded":"2024-09-11T22:52:53.129357+0000","last_fullsized":"2024-09-11T22:52:53.129357+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:13.294544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.075030+0000","last_change":"2024-09-11T22:50:52.264484+0000","last_active":"2024-09-11T22:52:53.075030+0000","last_peered":"2024-09-11T22:52:53.075030+0000","last_clean":"2024-09-11T22:52:53.075030+0000","last_became_active":"2024-09-11T22:50:52.264038+0000","last_became_peered":"2024-09-11T22:50:52.264038+0000","last_unstale":"2024-09-11T22:52:53.075030+0000","last_undegraded":"2024-09-11T22:52:53.075030+0000","last_fullsized":"2024-09-11T22:52:53.075030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:21.325136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129556+0000","last_change":"2024-09-11T22:50:54.162225+0000","last_active":"2024-09-11T22:52:53.129556+0000","last_peered":"2024-09-11T22:52:53.129556+0000","last_clean":"2024-09-11T22:52:53.129556+0000","last_became_active":"2024-09-11T22:50:54.161234+0000","last_became_peered":"2024-09-11T22:50:54.161234+0000","last_unstale":"2024-09-11T22:52:53.129556+0000","last_undegraded":"2024-09-11T22:52:53.129556+0000","last_fullsized":"2024-09-11T22:52:53.129556+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:00.346217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'52","reported_seq":115,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.286808+0000","last_change":"2024-09-11T22:51:55.967804+0000","last_active":"2024-09-11T22:52:53.286808+0000","last_peered":"2024-09-11T22:52:53.286808+0000","last_clean":"2024-09-11T22:52:53.286808+0000","last_became_active":"2024-09-11T22:51:55.961136+0000","last_became_peered":"2024-09-11T22:51:55.961136+0000","last_unstale":"2024-09-11T22:52:53.286808+0000","last_undegraded":"2024-09-11T22:52:53.286808+0000","last_fullsized":"2024-09-11T22:52:53.286808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:55.315179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130600+0000","last_change":"2024-09-11T22:50:54.167307+0000","last_active":"2024-09-11T22:52:53.130600+0000","last_peered":"2024-09-11T22:52:53.130600+0000","last_clean":"2024-09-11T22:52:53.130600+0000","last_became_active":"2024-09-11T22:50:54.166650+0000","last_became_peered":"2024-09-11T22:50:54.166650+0000","last_unstale":"2024-09-11T22:52:53.130600+0000","last_undegraded":"2024-09-11T22:52:53.130600+0000","last_fullsized":"2024-09-11T22:52:53.130600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:04.863258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130309+0000","last_change":"2024-09-11T22:50:52.435107+0000","last_active":"2024-09-11T22:52:53.130309+0000","last_peered":"2024-09-11T22:52:53.130309+0000","last_clean":"2024-09-11T22:52:53.130309+0000","last_became_active":"2024-09-11T22:50:52.434791+0000","last_became_peered":"2024-09-11T22:50:52.434791+0000","last_unstale":"2024-09-11T22:52:53.130309+0000","last_undegraded":"2024-09-11T22:52:53.130309+0000","last_fullsized":"2024-09-11T22:52:53.130309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:11.128791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131410+0000","last_change":"2024-09-11T22:50:54.178041+0000","last_active":"2024-09-11T22:52:53.131410+0000","last_peered":"2024-09-11T22:52:53.131410+0000","last_clean":"2024-09-11T22:52:53.131410+0000","last_became_active":"2024-09-11T22:50:54.177393+0000","last_became_peered":"2024-09-11T22:50:54.177393+0000","last_unstale":"2024-09-11T22:52:53.131410+0000","last_undegraded":"2024-09-11T22:52:53.131410+0000","last_fullsized":"2024-09-11T22:52:53.131410+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:21.608104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074102+0000","last_change":"2024-09-11T22:50:52.264051+0000","last_active":"2024-09-11T22:52:53.074102+0000","last_peered":"2024-09-11T22:52:53.074102+0000","last_clean":"2024-09-11T22:52:53.074102+0000","last_became_active":"2024-09-11T22:50:52.263674+0000","last_became_peered":"2024-09-11T22:50:52.263674+0000","last_unstale":"2024-09-11T22:52:53.074102+0000","last_undegraded":"2024-09-11T22:52:53.074102+0000","last_fullsized":"2024-09-11T22:52:53.074102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:44.655218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074213+0000","last_change":"2024-09-11T22:50:54.166409+0000","last_active":"2024-09-11T22:52:53.074213+0000","last_peered":"2024-09-11T22:52:53.074213+0000","last_clean":"2024-09-11T22:52:53.074213+0000","last_became_active":"2024-09-11T22:50:54.166037+0000","last_became_peered":"2024-09-11T22:50:54.166037+0000","last_unstale":"2024-09-11T22:52:53.074213+0000","last_undegraded":"2024-09-11T22:52:53.074213+0000","last_fullsized":"2024-09-11T22:52:53.074213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:22.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129590+0000","last_change":"2024-09-11T22:50:52.140218+0000","last_active":"2024-09-11T22:52:53.129590+0000","last_peered":"2024-09-11T22:52:53.129590+0000","last_clean":"2024-09-11T22:52:53.129590+0000","last_became_active":"2024-09-11T22:50:52.139744+0000","last_became_peered":"2024-09-11T22:50:52.139744+0000","last_unstale":"2024-09-11T22:52:53.129590+0000","last_undegraded":"2024-09-11T22:52:53.129590+0000","last_fullsized":"2024-09-11T22:52:53.129590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:50.178348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168909+0000","last_change":"2024-09-11T22:50:54.164059+0000","last_active":"2024-09-11T22:52:53.168909+0000","last_peered":"2024-09-11T22:52:53.168909+0000","last_clean":"2024-09-11T22:52:53.168909+0000","last_became_active":"2024-09-11T22:50:54.163724+0000","last_became_peered":"2024-09-11T22:50:54.163724+0000","last_unstale":"2024-09-11T22:52:53.168909+0000","last_undegraded":"2024-09-11T22:52:53.168909+0000","last_fullsized":"2024-09-11T22:52:53.168909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:57.241660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.128932+0000","last_change":"2024-09-11T22:50:52.140169+0000","last_active":"2024-09-11T22:52:53.128932+0000","last_peered":"2024-09-11T22:52:53.128932+0000","last_clean":"2024-09-11T22:52:53.128932+0000","last_became_active":"2024-09-11T22:50:52.139579+0000","last_became_peered":"2024-09-11T22:50:52.139579+0000","last_unstale":"2024-09-11T22:52:53.128932+0000","last_undegraded":"2024-09-11T22:52:53.128932+0000","last_fullsized":"2024-09-11T22:52:53.128932+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:43.523961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130075+0000","last_change":"2024-09-11T22:50:54.171539+0000","last_active":"2024-09-11T22:52:53.130075+0000","last_peered":"2024-09-11T22:52:53.130075+0000","last_clean":"2024-09-11T22:52:53.130075+0000","last_became_active":"2024-09-11T22:50:54.171013+0000","last_became_peered":"2024-09-11T22:50:54.171013+0000","last_unstale":"2024-09-11T22:52:53.130075+0000","last_undegraded":"2024-09-11T22:52:53.130075+0000","last_fullsized":"2024-09-11T22:52:53.130075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:03.564389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074787+0000","last_change":"2024-09-11T22:50:52.139544+0000","last_active":"2024-09-11T22:52:53.074787+0000","last_peered":"2024-09-11T22:52:53.074787+0000","last_clean":"2024-09-11T22:52:53.074787+0000","last_became_active":"2024-09-11T22:50:52.139201+0000","last_became_peered":"2024-09-11T22:50:52.139201+0000","last_unstale":"2024-09-11T22:52:53.074787+0000","last_undegraded":"2024-09-11T22:52:53.074787+0000","last_fullsized":"2024-09-11T22:52:53.074787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:37.343437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129337+0000","last_change":"2024-09-11T22:50:54.170052+0000","last_active":"2024-09-11T22:52:53.129337+0000","last_peered":"2024-09-11T22:52:53.129337+0000","last_clean":"2024-09-11T22:52:53.129337+0000","last_became_active":"2024-09-11T22:50:54.169735+0000","last_became_peered":"2024-09-11T22:50:54.169735+0000","last_unstale":"2024-09-11T22:52:53.129337+0000","last_undegraded":"2024-09-11T22:52:53.129337+0000","last_fullsized":"2024-09-11T22:52:53.129337+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:44.007747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074181+0000","last_change":"2024-09-11T22:50:54.169959+0000","last_active":"2024-09-11T22:52:53.074181+0000","last_peered":"2024-09-11T22:52:53.074181+0000","last_clean":"2024-09-11T22:52:53.074181+0000","last_became_active":"2024-09-11T22:50:54.169649+0000","last_became_peered":"2024-09-11T22:50:54.169649+0000","last_unstale":"2024-09-11T22:52:53.074181+0000","last_undegraded":"2024-09-11T22:52:53.074181+0000","last_fullsized":"2024-09-11T22:52:53.074181+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:51.974258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074520+0000","last_change":"2024-09-11T22:50:52.140881+0000","last_active":"2024-09-11T22:52:53.074520+0000","last_peered":"2024-09-11T22:52:53.074520+0000","last_clean":"2024-09-11T22:52:53.074520+0000","last_became_active":"2024-09-11T22:50:52.140527+0000","last_became_peered":"2024-09-11T22:50:52.140527+0000","last_unstale":"2024-09-11T22:52:53.074520+0000","last_undegraded":"2024-09-11T22:52:53.074520+0000","last_fullsized":"2024-09-11T22:52:53.074520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:14.325816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074263+0000","last_change":"2024-09-11T22:50:54.166571+0000","last_active":"2024-09-11T22:52:53.074263+0000","last_peered":"2024-09-11T22:52:53.074263+0000","last_clean":"2024-09-11T22:52:53.074263+0000","last_became_active":"2024-09-11T22:50:54.166247+0000","last_became_peered":"2024-09-11T22:50:54.166247+0000","last_unstale":"2024-09-11T22:52:53.074263+0000","last_undegraded":"2024-09-11T22:52:53.074263+0000","last_fullsized":"2024-09-11T22:52:53.074263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:42.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074895+0000","last_change":"2024-09-11T22:50:52.140298+0000","last_active":"2024-09-11T22:52:53.074895+0000","last_peered":"2024-09-11T22:52:53.074895+0000","last_clean":"2024-09-11T22:52:53.074895+0000","last_became_active":"2024-09-11T22:50:52.139917+0000","last_became_peered":"2024-09-11T22:50:52.139917+0000","last_unstale":"2024-09-11T22:52:53.074895+0000","last_undegraded":"2024-09-11T22:52:53.074895+0000","last_fullsized":"2024-09-11T22:52:53.074895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:23.599010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131581+0000","last_change":"2024-09-11T22:50:54.170849+0000","last_active":"2024-09-11T22:52:53.131581+0000","last_peered":"2024-09-11T22:52:53.131581+0000","last_clean":"2024-09-11T22:52:53.131581+0000","last_became_active":"2024-09-11T22:50:54.169547+0000","last_became_peered":"2024-09-11T22:50:54.169547+0000","last_unstale":"2024-09-11T22:52:53.131581+0000","last_undegraded":"2024-09-11T22:52:53.131581+0000","last_fullsized":"2024-09-11T22:52:53.131581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:25.251158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130553+0000","last_change":"2024-09-11T22:50:52.263778+0000","last_active":"2024-09-11T22:52:53.130553+0000","last_peered":"2024-09-11T22:52:53.130553+0000","last_clean":"2024-09-11T22:52:53.130553+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:52:53.130553+0000","last_undegraded":"2024-09-11T22:52:53.130553+0000","last_fullsized":"2024-09-11T22:52:53.130553+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:03.439147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130427+0000","last_change":"2024-09-11T22:50:54.171200+0000","last_active":"2024-09-11T22:52:53.130427+0000","last_peered":"2024-09-11T22:52:53.130427+0000","last_clean":"2024-09-11T22:52:53.130427+0000","last_became_active":"2024-09-11T22:50:54.169320+0000","last_became_peered":"2024-09-11T22:50:54.169320+0000","last_unstale":"2024-09-11T22:52:53.130427+0000","last_undegraded":"2024-09-11T22:52:53.130427+0000","last_fullsized":"2024-09-11T22:52:53.130427+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:14.864699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":146,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130612+0000","last_change":"2024-09-11T22:50:52.139609+0000","last_active":"2024-09-11T22:52:53.130612+0000","last_peered":"2024-09-11T22:52:53.130612+0000","last_clean":"2024-09-11T22:52:53.130612+0000","last_became_active":"2024-09-11T22:50:52.139278+0000","last_became_peered":"2024-09-11T22:50:52.139278+0000","last_unstale":"2024-09-11T22:52:53.130612+0000","last_undegraded":"2024-09-11T22:52:53.130612+0000","last_fullsized":"2024-09-11T22:52:53.130612+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:09.538734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602815,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1227,"num_read_kb":729,"num_write":378,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1808445,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107505,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547238,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":942080,"data_stored":840516,"data_compressed":22047,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776170,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1238332,"data_compressed":44892,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939696,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":839519,"data_compressed":22047,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168628,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":437775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":991232,"data_stored":1040914,"data_compressed":38736,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561088,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":446020,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1171406,"data_compressed":44892,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":925696,"data_stored":445881,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":987136,"data_stored":970661,"data_compressed":28203,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":517944,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":515477,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":946176,"data_stored":448286,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135740,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202666,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":395335,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:52:57.120 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:52:57.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:57 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:57.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:57 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:57.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:57 smithi152 ceph-mon[31181]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:52:57.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:57 smithi152 ceph-mon[31181]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:52:57.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:57 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:57.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:57 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:57.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:57 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:57.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:57 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:57.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:57 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:57.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:57 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:57.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:57 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:52:57.374 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:57 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:52:57.447 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:57 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:57 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:57 smithi191 ceph-mon[35211]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:52:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:57 smithi191 ceph-mon[35211]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:52:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:57 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:57 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:57 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:57.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:57 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:57.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:57 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:57.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:57 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:57.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:57 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:52:57.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:57 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:52:57.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:57 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:57.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:57 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:57.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:57 smithi204 ceph-mon[35454]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:52:57.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:57 smithi204 ceph-mon[35454]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:52:57.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:57 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:57.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:57 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:57.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:57 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:57.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:57 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:57.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:57 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:57.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:57 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:52:57.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:57 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:52:57.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:57 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:52:58.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.018+0000 7f443263f640 1 -- 172.21.15.152:0/1085835656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f442c102570 msgr2=0x7f442c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:58.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.018+0000 7f443263f640 1 --2- 172.21.15.152:0/1085835656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f442c102570 0x7f442c1029b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4420009900 tx=0x7f442002a480 comp rx=0 tx=0).stop 2024-09-11T22:52:58.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.018+0000 7f443263f640 1 -- 172.21.15.152:0/1085835656 shutdown_connections 2024-09-11T22:52:58.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.018+0000 7f443263f640 1 --2- 172.21.15.152:0/1085835656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f442c102ef0 0x7f442c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:58.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.018+0000 7f443263f640 1 --2- 172.21.15.152:0/1085835656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f442c102570 0x7f442c1029b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:58.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.018+0000 7f443263f640 1 --2- 172.21.15.152:0/1085835656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f442c100e20 0x7f442c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:58.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.018+0000 7f443263f640 1 -- 172.21.15.152:0/1085835656 >> 172.21.15.152:0/1085835656 conn(0x7f442c0fc7d0 msgr2=0x7f442c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:58.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.018+0000 7f443263f640 1 -- 172.21.15.152:0/1085835656 shutdown_connections 2024-09-11T22:52:58.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.018+0000 7f443263f640 1 -- 172.21.15.152:0/1085835656 wait complete. 2024-09-11T22:52:58.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.019+0000 7f443263f640 1 Processor -- start 2024-09-11T22:52:58.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.019+0000 7f443263f640 1 -- start start 2024-09-11T22:52:58.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f443263f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f442c100e20 0x7f442c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:58.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f443263f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f442c102570 0x7f442c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:58.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f443263f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f442c102ef0 0x7f442c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:58.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f443263f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f442c110f50 con 0x7f442c102570 2024-09-11T22:52:58.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f443263f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f442c110dd0 con 0x7f442c102ef0 2024-09-11T22:52:58.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f443263f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f442c1110d0 con 0x7f442c100e20 2024-09-11T22:52:58.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f442b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f442c102570 0x7f442c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:58.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f442b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f442c102570 0x7f442c19ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45542/0 (socket says 172.21.15.152:45542) 2024-09-11T22:52:58.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f442b7fe640 1 -- 172.21.15.152:0/2033946072 learned_addr learned my addr 172.21.15.152:0/2033946072 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:58.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f442bfff640 1 --2- 172.21.15.152:0/2033946072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f442c100e20 0x7f442c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:58.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f4430bdc640 1 --2- 172.21.15.152:0/2033946072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f442c102ef0 0x7f442c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:58.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f442b7fe640 1 -- 172.21.15.152:0/2033946072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f442c100e20 msgr2=0x7f442c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:58.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f442b7fe640 1 --2- 172.21.15.152:0/2033946072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f442c100e20 0x7f442c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:58.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f442b7fe640 1 -- 172.21.15.152:0/2033946072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f442c102ef0 msgr2=0x7f442c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:58.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f442b7fe640 1 --2- 172.21.15.152:0/2033946072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f442c102ef0 0x7f442c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:58.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f442b7fe640 1 -- 172.21.15.152:0/2033946072 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442c19f690 con 0x7f442c102570 2024-09-11T22:52:58.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f4430bdc640 1 --2- 172.21.15.152:0/2033946072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f442c102ef0 0x7f442c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:52:58.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f442bfff640 1 --2- 172.21.15.152:0/2033946072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f442c100e20 0x7f442c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:52:58.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f442b7fe640 1 --2- 172.21.15.152:0/2033946072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f442c102570 0x7f442c19ac20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4420006010 tx=0x7f4420003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:58.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.020+0000 7f44297fa640 1 -- 172.21.15.152:0/2033946072 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442003f070 con 0x7f442c102570 2024-09-11T22:52:58.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.021+0000 7f443263f640 1 -- 172.21.15.152:0/2033946072 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442c19f920 con 0x7f442c102570 2024-09-11T22:52:58.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.021+0000 7f44297fa640 1 -- 172.21.15.152:0/2033946072 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4420031c30 con 0x7f442c102570 2024-09-11T22:52:58.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.021+0000 7f44297fa640 1 -- 172.21.15.152:0/2033946072 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420035030 con 0x7f442c102570 2024-09-11T22:52:58.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.021+0000 7f443263f640 1 -- 172.21.15.152:0/2033946072 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f442c1a7260 con 0x7f442c102570 2024-09-11T22:52:58.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.022+0000 7f443263f640 1 -- 172.21.15.152:0/2033946072 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f442c101200 con 0x7f442c102570 2024-09-11T22:52:58.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.023+0000 7f44297fa640 1 -- 172.21.15.152:0/2033946072 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 22) ==== 137954+0+0 (secure 0 0 0) 0x7f4420042070 con 0x7f442c102570 2024-09-11T22:52:58.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.023+0000 7f44297fa640 1 --2- 172.21.15.152:0/2033946072 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f44140a2cf0 0x7f44140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:58.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.024+0000 7f442bfff640 1 --2- 172.21.15.152:0/2033946072 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f44140a2cf0 0x7f44140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:58.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.024+0000 7f44297fa640 1 -- 172.21.15.152:0/2033946072 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f44200efa60 con 0x7f442c102570 2024-09-11T22:52:58.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.024+0000 7f442bfff640 1 --2- 172.21.15.152:0/2033946072 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f44140a2cf0 0x7f44140a51b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f441c00bfd0 tx=0x7f441c00b390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:58.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.028+0000 7f44297fa640 1 -- 172.21.15.152:0/2033946072 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44200bc080 con 0x7f442c102570 2024-09-11T22:52:58.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.049+0000 7f44297fa640 1 -- 172.21.15.152:0/2033946072 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f442003b460 con 0x7f442c102570 2024-09-11T22:52:58.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.241+0000 7f443263f640 1 -- 172.21.15.152:0/2033946072 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f442c19ba40 con 0x7f442c102570 2024-09-11T22:52:58.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.242+0000 7f44297fa640 1 -- 172.21.15.152:0/2033946072 <== mon.0 v2:172.21.15.152:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f442c19ba40 con 0x7f442c102570 2024-09-11T22:52:58.242 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:52:58.243 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:52:58.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.244+0000 7f443263f640 1 -- 172.21.15.152:0/2033946072 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f44140a2cf0 msgr2=0x7f44140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:58.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.244+0000 7f443263f640 1 --2- 172.21.15.152:0/2033946072 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f44140a2cf0 0x7f44140a51b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f441c00bfd0 tx=0x7f441c00b390 comp rx=0 tx=0).stop 2024-09-11T22:52:58.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.244+0000 7f443263f640 1 -- 172.21.15.152:0/2033946072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f442c102570 msgr2=0x7f442c19ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:58.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.244+0000 7f443263f640 1 --2- 172.21.15.152:0/2033946072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f442c102570 0x7f442c19ac20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4420006010 tx=0x7f4420003c10 comp rx=0 tx=0).stop 2024-09-11T22:52:58.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.244+0000 7f443263f640 1 -- 172.21.15.152:0/2033946072 shutdown_connections 2024-09-11T22:52:58.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.244+0000 7f443263f640 1 --2- 172.21.15.152:0/2033946072 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f44140a2cf0 0x7f44140a51b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:58.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.244+0000 7f443263f640 1 --2- 172.21.15.152:0/2033946072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f442c102ef0 0x7f442c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:58.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.244+0000 7f443263f640 1 --2- 172.21.15.152:0/2033946072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f442c102570 0x7f442c19ac20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:58.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.244+0000 7f443263f640 1 --2- 172.21.15.152:0/2033946072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f442c100e20 0x7f442c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:58.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.244+0000 7f443263f640 1 -- 172.21.15.152:0/2033946072 >> 172.21.15.152:0/2033946072 conn(0x7f442c0fc7d0 msgr2=0x7f442c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:58.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.244+0000 7f443263f640 1 -- 172.21.15.152:0/2033946072 shutdown_connections 2024-09-11T22:52:58.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:58.244+0000 7f443263f640 1 -- 172.21.15.152:0/2033946072 wait complete. 2024-09-11T22:52:58.945 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:52:59.200 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:52:59.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi152 to 176.0M 2024-09-11T22:52:59.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: Unable to set osd_memory_target on smithi152 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-11T22:52:59.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: Updating smithi152:/etc/ceph/ceph.conf 2024-09-11T22:52:59.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: Updating smithi191:/etc/ceph/ceph.conf 2024-09-11T22:52:59.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-11T22:52:59.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:52:59.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:52:59.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: Updating smithi152:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:52:59.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: Updating smithi204:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:52:59.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: Updating smithi191:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:52:59.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:52:59.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:52:59.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:52:59.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: Updating smithi152:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:52:59.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: Updating smithi204:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:52:59.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: Updating smithi191:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:52:59.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-11T22:52:59.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:52:59.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:52:59.290 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:52:59.290 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.290 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:58 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2033946072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:59.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi152 to 176.0M 2024-09-11T22:52:59.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: Unable to set osd_memory_target on smithi152 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-11T22:52:59.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: Updating smithi152:/etc/ceph/ceph.conf 2024-09-11T22:52:59.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: Updating smithi191:/etc/ceph/ceph.conf 2024-09-11T22:52:59.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-11T22:52:59.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:52:59.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:52:59.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: Updating smithi152:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:52:59.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: Updating smithi204:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:52:59.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: Updating smithi191:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:52:59.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:52:59.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:52:59.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:52:59.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: Updating smithi152:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:52:59.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: Updating smithi204:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:52:59.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: Updating smithi191:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:52:59.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-11T22:52:59.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:52:59.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:52:59.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:52:59.299 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.299 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:58 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2033946072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:59.322 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi152 to 176.0M 2024-09-11T22:52:59.323 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: Unable to set osd_memory_target on smithi152 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-11T22:52:59.323 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: Updating smithi152:/etc/ceph/ceph.conf 2024-09-11T22:52:59.323 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: Updating smithi191:/etc/ceph/ceph.conf 2024-09-11T22:52:59.323 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: Updating smithi204:/etc/ceph/ceph.conf 2024-09-11T22:52:59.323 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:52:59.323 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:52:59.323 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: Updating smithi152:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:52:59.323 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: Updating smithi204:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:52:59.323 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: Updating smithi191:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:52:59.323 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:52:59.323 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:52:59.323 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:52:59.323 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: Updating smithi152:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:52:59.324 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: Updating smithi204:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:52:59.324 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: Updating smithi191:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:52:59.324 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.324 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.324 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.324 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.324 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.324 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.324 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.324 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-09-11T22:52:59.324 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:52:59.324 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:52:59.325 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:52:59.325 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:52:59.325 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:58 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2033946072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:52:59.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.731+0000 7f58df64c640 1 -- 172.21.15.152:0/2619926529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58d8106a80 msgr2=0x7f58d8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:59.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.731+0000 7f58df64c640 1 --2- 172.21.15.152:0/2619926529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58d8106a80 0x7f58d8106e60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009930 tx=0x7f58cc02a420 comp rx=0 tx=0).stop 2024-09-11T22:52:59.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.731+0000 7f58df64c640 1 -- 172.21.15.152:0/2619926529 shutdown_connections 2024-09-11T22:52:59.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.731+0000 7f58df64c640 1 --2- 172.21.15.152:0/2619926529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d8101420 0x7f58d810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:59.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.731+0000 7f58df64c640 1 --2- 172.21.15.152:0/2619926529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58d8100a80 0x7f58d8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:59.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.731+0000 7f58df64c640 1 --2- 172.21.15.152:0/2619926529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58d8106a80 0x7f58d8106e60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:59.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.731+0000 7f58df64c640 1 -- 172.21.15.152:0/2619926529 >> 172.21.15.152:0/2619926529 conn(0x7f58d80fc7a0 msgr2=0x7f58d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:59.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.731+0000 7f58df64c640 1 -- 172.21.15.152:0/2619926529 shutdown_connections 2024-09-11T22:52:59.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.731+0000 7f58df64c640 1 -- 172.21.15.152:0/2619926529 wait complete. 2024-09-11T22:52:59.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58df64c640 1 Processor -- start 2024-09-11T22:52:59.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58df64c640 1 -- start start 2024-09-11T22:52:59.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58df64c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58d8100a80 0x7f58d8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:59.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58df64c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58d8101420 0x7f58d81997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:59.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58df64c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d8106a80 0x7f58d81a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:59.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58df64c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f58d8110ea0 con 0x7f58d8101420 2024-09-11T22:52:59.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58df64c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f58d8110d20 con 0x7f58d8100a80 2024-09-11T22:52:59.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58df64c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f58d8111020 con 0x7f58d8106a80 2024-09-11T22:52:59.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58dcbe7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58d8101420 0x7f58d81997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:59.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58dcbe7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58d8101420 0x7f58d81997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43542/0 (socket says 172.21.15.152:43542) 2024-09-11T22:52:59.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58dcbe7640 1 -- 172.21.15.152:0/1585444734 learned_addr learned my addr 172.21.15.152:0/1585444734 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:52:59.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58dd3e8640 1 --2- 172.21.15.152:0/1585444734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58d8100a80 0x7f58d8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:59.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58ddbe9640 1 --2- 172.21.15.152:0/1585444734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d8106a80 0x7f58d81a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:59.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58dcbe7640 1 -- 172.21.15.152:0/1585444734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d8106a80 msgr2=0x7f58d81a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:59.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58dcbe7640 1 --2- 172.21.15.152:0/1585444734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d8106a80 0x7f58d81a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:59.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58dcbe7640 1 -- 172.21.15.152:0/1585444734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58d8100a80 msgr2=0x7f58d8199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:59.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58dcbe7640 1 --2- 172.21.15.152:0/1585444734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58d8100a80 0x7f58d8199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:59.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58dcbe7640 1 -- 172.21.15.152:0/1585444734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d81a4830 con 0x7f58d8101420 2024-09-11T22:52:59.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.732+0000 7f58ddbe9640 1 --2- 172.21.15.152:0/1585444734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d8106a80 0x7f58d81a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:52:59.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.733+0000 7f58dcbe7640 1 --2- 172.21.15.152:0/1585444734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58d8101420 0x7f58d81997b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f58d400c920 tx=0x7f58d400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:59.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.733+0000 7f58ca7fc640 1 -- 172.21.15.152:0/1585444734 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4007c70 con 0x7f58d8101420 2024-09-11T22:52:59.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.733+0000 7f58ca7fc640 1 -- 172.21.15.152:0/1585444734 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58d4007e10 con 0x7f58d8101420 2024-09-11T22:52:59.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.733+0000 7f58df64c640 1 -- 172.21.15.152:0/1585444734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d81a4a00 con 0x7f58d8101420 2024-09-11T22:52:59.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.733+0000 7f58ca7fc640 1 -- 172.21.15.152:0/1585444734 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4018890 con 0x7f58d8101420 2024-09-11T22:52:59.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.733+0000 7f58df64c640 1 -- 172.21.15.152:0/1585444734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d81a4f40 con 0x7f58d8101420 2024-09-11T22:52:59.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.734+0000 7f58df64c640 1 -- 172.21.15.152:0/1585444734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d8100ee0 con 0x7f58d8101420 2024-09-11T22:52:59.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.735+0000 7f58ca7fc640 1 -- 172.21.15.152:0/1585444734 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f58d4018a70 con 0x7f58d8101420 2024-09-11T22:52:59.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.736+0000 7f58ca7fc640 1 --2- 172.21.15.152:0/1585444734 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58c40a2e00 0x7f58c40a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:52:59.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.736+0000 7f58dd3e8640 1 --2- 172.21.15.152:0/1585444734 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58c40a2e00 0x7f58c40a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:52:59.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.736+0000 7f58ca7fc640 1 -- 172.21.15.152:0/1585444734 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f58d40cf690 con 0x7f58d8101420 2024-09-11T22:52:59.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.736+0000 7f58dd3e8640 1 --2- 172.21.15.152:0/1585444734 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58c40a2e00 0x7f58c40a52c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009900 tx=0x7f58cc0058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:52:59.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.740+0000 7f58ca7fc640 1 -- 172.21.15.152:0/1585444734 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58d4014030 con 0x7f58d8101420 2024-09-11T22:52:59.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.907+0000 7f58df64c640 1 -- 172.21.15.152:0/1585444734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58d81a51d0 con 0x7f58d8101420 2024-09-11T22:52:59.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.909+0000 7f58ca7fc640 1 -- 172.21.15.152:0/1585444734 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+23094 (secure 0 0 0) 0x7f58d409c190 con 0x7f58d8101420 2024-09-11T22:52:59.910 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:52:59.910 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":84,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","created":"2024-09-11T22:43:00.379317+0000","modified":"2024-09-11T22:52:53.067599+0000","last_up_change":"2024-09-11T22:50:01.043181+0000","last_in_change":"2024-09-11T22:49:38.073870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:46:10.984707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:50:50.833547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:50:53.047604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3cd917b5-77f6-4a4a-8fd0-4ebe3034feac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6803","nonce":1270728007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6805","nonce":1270728007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6809","nonce":1270728007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6807","nonce":1270728007}]},"public_addr":"172.21.15.152:6803/1270728007","cluster_addr":"172.21.15.152:6805/1270728007","heartbeat_back_addr":"172.21.15.152:6809/1270728007","heartbeat_front_addr":"172.21.15.152:6807/1270728007","state":["exists","up"]},{"osd":1,"uuid":"ae589bc4-77b1-4522-9591-b0ce2a5432dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6801","nonce":354467155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6803","nonce":354467155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6807","nonce":354467155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6805","nonce":354467155}]},"public_addr":"172.21.15.191:6801/354467155","cluster_addr":"172.21.15.191:6803/354467155","heartbeat_back_addr":"172.21.15.191:6807/354467155","heartbeat_front_addr":"172.21.15.191:6805/354467155","state":["exists","up"]},{"osd":2,"uuid":"b35ffd0c-d084-43dd-a947-15132d78d59c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6801","nonce":3848396211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6803","nonce":3848396211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6807","nonce":3848396211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6805","nonce":3848396211}]},"public_addr":"172.21.15.204:6801/3848396211","cluster_addr":"172.21.15.204:6803/3848396211","heartbeat_back_addr":"172.21.15.204:6807/3848396211","heartbeat_front_addr":"172.21.15.204:6805/3848396211","state":["exists","up"]},{"osd":3,"uuid":"d66ba7e8-4ab7-449e-9460-2f44ffdf987f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6811","nonce":3351257181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6813","nonce":3351257181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6817","nonce":3351257181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6815","nonce":3351257181}]},"public_addr":"172.21.15.152:6811/3351257181","cluster_addr":"172.21.15.152:6813/3351257181","heartbeat_back_addr":"172.21.15.152:6817/3351257181","heartbeat_front_addr":"172.21.15.152:6815/3351257181","state":["exists","up"]},{"osd":4,"uuid":"e1b56468-6cc7-4d29-988e-9d281118ee36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6809","nonce":2967539563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6811","nonce":2967539563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6815","nonce":2967539563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6813","nonce":2967539563}]},"public_addr":"172.21.15.191:6809/2967539563","cluster_addr":"172.21.15.191:6811/2967539563","heartbeat_back_addr":"172.21.15.191:6815/2967539563","heartbeat_front_addr":"172.21.15.191:6813/2967539563","state":["exists","up"]},{"osd":5,"uuid":"89dabd3e-77e3-4294-9648-b112ef694e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6809","nonce":4283562158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6811","nonce":4283562158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6815","nonce":4283562158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6813","nonce":4283562158}]},"public_addr":"172.21.15.204:6809/4283562158","cluster_addr":"172.21.15.204:6811/4283562158","heartbeat_back_addr":"172.21.15.204:6815/4283562158","heartbeat_front_addr":"172.21.15.204:6813/4283562158","state":["exists","up"]},{"osd":6,"uuid":"46a0fded-b668-4aef-a68f-613cccc83483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6819","nonce":2484234042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6821","nonce":2484234042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6825","nonce":2484234042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6823","nonce":2484234042}]},"public_addr":"172.21.15.152:6819/2484234042","cluster_addr":"172.21.15.152:6821/2484234042","heartbeat_back_addr":"172.21.15.152:6825/2484234042","heartbeat_front_addr":"172.21.15.152:6823/2484234042","state":["exists","up"]},{"osd":7,"uuid":"e674bfe9-e2db-44a3-b74b-a21f198747c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6817","nonce":269608859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6819","nonce":269608859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6823","nonce":269608859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6821","nonce":269608859}]},"public_addr":"172.21.15.191:6817/269608859","cluster_addr":"172.21.15.191:6819/269608859","heartbeat_back_addr":"172.21.15.191:6823/269608859","heartbeat_front_addr":"172.21.15.191:6821/269608859","state":["exists","up"]},{"osd":8,"uuid":"0418bfc9-d6b6-488f-b4c7-0bc6291bb624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6817","nonce":1756190090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6819","nonce":1756190090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6823","nonce":1756190090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6821","nonce":1756190090}]},"public_addr":"172.21.15.204:6817/1756190090","cluster_addr":"172.21.15.204:6819/1756190090","heartbeat_back_addr":"172.21.15.204:6823/1756190090","heartbeat_front_addr":"172.21.15.204:6821/1756190090","state":["exists","up"]},{"osd":9,"uuid":"751a6964-798f-4240-a2ba-7fa9f0c72657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6827","nonce":1301191419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6829","nonce":1301191419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6833","nonce":1301191419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6831","nonce":1301191419}]},"public_addr":"172.21.15.152:6827/1301191419","cluster_addr":"172.21.15.152:6829/1301191419","heartbeat_back_addr":"172.21.15.152:6833/1301191419","heartbeat_front_addr":"172.21.15.152:6831/1301191419","state":["exists","up"]},{"osd":10,"uuid":"90e7c6ba-127c-4b33-b3c6-cedf72128451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6825","nonce":3695294503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6827","nonce":3695294503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6831","nonce":3695294503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6829","nonce":3695294503}]},"public_addr":"172.21.15.191:6825/3695294503","cluster_addr":"172.21.15.191:6827/3695294503","heartbeat_back_addr":"172.21.15.191:6831/3695294503","heartbeat_front_addr":"172.21.15.191:6829/3695294503","state":["exists","up"]},{"osd":11,"uuid":"c76a93b1-a045-477b-98d0-58a2e9d44a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6825","nonce":1709433236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6827","nonce":1709433236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6831","nonce":1709433236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6829","nonce":1709433236}]},"public_addr":"172.21.15.204:6825/1709433236","cluster_addr":"172.21.15.204:6827/1709433236","heartbeat_back_addr":"172.21.15.204:6831/1709433236","heartbeat_front_addr":"172.21.15.204:6829/1709433236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:17.259744+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:42.230389+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:07.172433+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:32.517060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:58.900615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:24.379803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:50.058928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:15.967299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:42.025873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:08.225448+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:33.910531+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:59.466423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:0/2259758812":"2024-09-12T22:52:53.067551+0000","172.21.15.152:0/3900732888":"2024-09-12T22:52:53.067551+0000","172.21.15.152:0/2557435316":"2024-09-12T22:52:53.067551+0000","172.21.15.152:0/3960601922":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6800/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6801/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/4286398432":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2270507277":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2452559014":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/3071755865":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6800/3438861948":"2024-09-12T22:52:53.067551+0000","172.21.15.152:6800/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/1721817189":"2024-09-12T22:52:53.067551+0000","172.21.15.152:0/1542993850":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/3438861948":"2024-09-12T22:52:53.067551+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:52:59.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.911+0000 7f58df64c640 1 -- 172.21.15.152:0/1585444734 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58c40a2e00 msgr2=0x7f58c40a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:59.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.911+0000 7f58df64c640 1 --2- 172.21.15.152:0/1585444734 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58c40a2e00 0x7f58c40a52c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009900 tx=0x7f58cc0058d0 comp rx=0 tx=0).stop 2024-09-11T22:52:59.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.911+0000 7f58df64c640 1 -- 172.21.15.152:0/1585444734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58d8101420 msgr2=0x7f58d81997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:52:59.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.911+0000 7f58df64c640 1 --2- 172.21.15.152:0/1585444734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58d8101420 0x7f58d81997b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f58d400c920 tx=0x7f58d400cde0 comp rx=0 tx=0).stop 2024-09-11T22:52:59.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.911+0000 7f58df64c640 1 -- 172.21.15.152:0/1585444734 shutdown_connections 2024-09-11T22:52:59.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.911+0000 7f58df64c640 1 --2- 172.21.15.152:0/1585444734 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58c40a2e00 0x7f58c40a52c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:59.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.911+0000 7f58df64c640 1 --2- 172.21.15.152:0/1585444734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58d8106a80 0x7f58d81a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:59.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.911+0000 7f58df64c640 1 --2- 172.21.15.152:0/1585444734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58d8101420 0x7f58d81997b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:59.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.911+0000 7f58df64c640 1 --2- 172.21.15.152:0/1585444734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58d8100a80 0x7f58d8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:52:59.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.911+0000 7f58df64c640 1 -- 172.21.15.152:0/1585444734 >> 172.21.15.152:0/1585444734 conn(0x7f58d80fc7a0 msgr2=0x7f58d80fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:52:59.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.911+0000 7f58df64c640 1 -- 172.21.15.152:0/1585444734 shutdown_connections 2024-09-11T22:52:59.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:52:59.911+0000 7f58df64c640 1 -- 172.21.15.152:0/1585444734 wait complete. 2024-09-11T22:53:00.217 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:52:59 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1585444734' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:53:00.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:52:59 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1585444734' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:53:00.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:52:59 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1585444734' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:53:00.633 DEBUG:tasks.fwd_scrub:Yielding 2024-09-11T22:53:00.633 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T22:53:00.644 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T22:53:00.644 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi152.front.sepia.ceph.com 2024-09-11T22:53:00.645 DEBUG:teuthology.orchestra.run.smithi152:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-11T22:53:00.645 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-11T22:53:00.645 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:53:00.645 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:53:00.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.752+0000 7ff66dd66640 1 -- 172.21.15.152:0/974644285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff668101350 msgr2=0x7ff66810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:00.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.752+0000 7ff66dd66640 1 --2- 172.21.15.152:0/974644285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff668101350 0x7ff66810d880 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff660009cd0 tx=0x7ff66002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:53:00.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.752+0000 7ff66dd66640 1 -- 172.21.15.152:0/974644285 shutdown_connections 2024-09-11T22:53:00.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.752+0000 7ff66dd66640 1 --2- 172.21.15.152:0/974644285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff668101350 0x7ff66810d880 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:00.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.752+0000 7ff66dd66640 1 --2- 172.21.15.152:0/974644285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6681009b0 0x7ff668100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:00.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.752+0000 7ff66dd66640 1 --2- 172.21.15.152:0/974644285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6681069c0 0x7ff668106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:00.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.752+0000 7ff66dd66640 1 -- 172.21.15.152:0/974644285 >> 172.21.15.152:0/974644285 conn(0x7ff668077990 msgr2=0x7ff6680fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:00.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.753+0000 7ff66dd66640 1 -- 172.21.15.152:0/974644285 shutdown_connections 2024-09-11T22:53:00.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.753+0000 7ff66dd66640 1 -- 172.21.15.152:0/974644285 wait complete. 2024-09-11T22:53:00.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.753+0000 7ff66dd66640 1 Processor -- start 2024-09-11T22:53:00.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.753+0000 7ff66dd66640 1 -- start start 2024-09-11T22:53:00.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.753+0000 7ff66dd66640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6681009b0 0x7ff668198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:00.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.753+0000 7ff66dd66640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff668101350 0x7ff6681989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:00.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff66dd66640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6681069c0 0x7ff66819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:00.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff66dd66640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff668110e60 con 0x7ff6681009b0 2024-09-11T22:53:00.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff66dd66640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff668110ce0 con 0x7ff668101350 2024-09-11T22:53:00.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff66dd66640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff668110fe0 con 0x7ff6681069c0 2024-09-11T22:53:00.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff6677fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6681009b0 0x7ff668198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:00.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff6677fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6681009b0 0x7ff668198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43560/0 (socket says 172.21.15.152:43560) 2024-09-11T22:53:00.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff6677fe640 1 -- 172.21.15.152:0/3706893558 learned_addr learned my addr 172.21.15.152:0/3706893558 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:00.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff667fff640 1 --2- 172.21.15.152:0/3706893558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6681069c0 0x7ff66819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:00.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff666ffd640 1 --2- 172.21.15.152:0/3706893558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff668101350 0x7ff6681989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:00.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff6677fe640 1 -- 172.21.15.152:0/3706893558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6681069c0 msgr2=0x7ff66819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:00.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff6677fe640 1 --2- 172.21.15.152:0/3706893558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6681069c0 0x7ff66819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:00.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff6677fe640 1 -- 172.21.15.152:0/3706893558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff668101350 msgr2=0x7ff6681989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:00.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff6677fe640 1 --2- 172.21.15.152:0/3706893558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff668101350 0x7ff6681989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:00.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff6677fe640 1 -- 172.21.15.152:0/3706893558 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66819d460 con 0x7ff6681009b0 2024-09-11T22:53:00.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff6677fe640 1 --2- 172.21.15.152:0/3706893558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6681009b0 0x7ff668198470 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff65800b4a0 tx=0x7ff65800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:00.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff664ff9640 1 -- 172.21.15.152:0/3706893558 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658014070 con 0x7ff6681009b0 2024-09-11T22:53:00.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff664ff9640 1 -- 172.21.15.152:0/3706893558 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff658010af0 con 0x7ff6681009b0 2024-09-11T22:53:00.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff664ff9640 1 -- 172.21.15.152:0/3706893558 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658017d10 con 0x7ff6681009b0 2024-09-11T22:53:00.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff66dd66640 1 -- 172.21.15.152:0/3706893558 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66819d750 con 0x7ff6681009b0 2024-09-11T22:53:00.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.754+0000 7ff66dd66640 1 -- 172.21.15.152:0/3706893558 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6681a4fe0 con 0x7ff6681009b0 2024-09-11T22:53:00.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.755+0000 7ff66dd66640 1 -- 172.21.15.152:0/3706893558 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff668100e90 con 0x7ff6681009b0 2024-09-11T22:53:00.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.757+0000 7ff664ff9640 1 -- 172.21.15.152:0/3706893558 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7ff658017070 con 0x7ff6681009b0 2024-09-11T22:53:00.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.757+0000 7ff664ff9640 1 --2- 172.21.15.152:0/3706893558 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff6500a2ec0 0x7ff6500a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:00.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.757+0000 7ff664ff9640 1 -- 172.21.15.152:0/3706893558 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7ff6580cec30 con 0x7ff6681009b0 2024-09-11T22:53:00.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.757+0000 7ff666ffd640 1 --2- 172.21.15.152:0/3706893558 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff6500a2ec0 0x7ff6500a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:00.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.758+0000 7ff666ffd640 1 --2- 172.21.15.152:0/3706893558 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff6500a2ec0 0x7ff6500a5380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff6680ffe80 tx=0x7ff65c008040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:00.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.761+0000 7ff664ff9640 1 -- 172.21.15.152:0/3706893558 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff65809b840 con 0x7ff6681009b0 2024-09-11T22:53:00.910 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:00.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.955+0000 7ff66dd66640 1 -- 172.21.15.152:0/3706893558 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7ff6681a53b0 con 0x7ff6681009b0 2024-09-11T22:53:00.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.959+0000 7ff664ff9640 1 -- 172.21.15.152:0/3706893558 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v30)=0 v30) ==== 129+0+0 (secure 0 0 0) 0x7ff65809b140 con 0x7ff6681009b0 2024-09-11T22:53:00.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.961+0000 7ff66dd66640 1 -- 172.21.15.152:0/3706893558 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff6500a2ec0 msgr2=0x7ff6500a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:00.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.961+0000 7ff66dd66640 1 --2- 172.21.15.152:0/3706893558 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff6500a2ec0 0x7ff6500a5380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff6680ffe80 tx=0x7ff65c008040 comp rx=0 tx=0).stop 2024-09-11T22:53:00.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.961+0000 7ff66dd66640 1 -- 172.21.15.152:0/3706893558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6681009b0 msgr2=0x7ff668198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:00.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.961+0000 7ff66dd66640 1 --2- 172.21.15.152:0/3706893558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6681009b0 0x7ff668198470 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff65800b4a0 tx=0x7ff65800b960 comp rx=0 tx=0).stop 2024-09-11T22:53:00.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.961+0000 7ff66dd66640 1 -- 172.21.15.152:0/3706893558 shutdown_connections 2024-09-11T22:53:00.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.961+0000 7ff66dd66640 1 --2- 172.21.15.152:0/3706893558 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff6500a2ec0 0x7ff6500a5380 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:00.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.961+0000 7ff66dd66640 1 --2- 172.21.15.152:0/3706893558 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6681069c0 0x7ff66819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:00.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.961+0000 7ff66dd66640 1 --2- 172.21.15.152:0/3706893558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff668101350 0x7ff6681989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:00.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.961+0000 7ff66dd66640 1 --2- 172.21.15.152:0/3706893558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff6681009b0 0x7ff668198470 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:00.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.961+0000 7ff66dd66640 1 -- 172.21.15.152:0/3706893558 >> 172.21.15.152:0/3706893558 conn(0x7ff668077990 msgr2=0x7ff6680699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:00.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.962+0000 7ff66dd66640 1 -- 172.21.15.152:0/3706893558 shutdown_connections 2024-09-11T22:53:00.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:00.962+0000 7ff66dd66640 1 -- 172.21.15.152:0/3706893558 wait complete. 2024-09-11T22:53:00.974 DEBUG:teuthology.orchestra.run.smithi152:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-11T22:53:01.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.077+0000 7f721a850640 1 -- 172.21.15.152:0/1840142363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7214101350 msgr2=0x7f721410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.077+0000 7f721a850640 1 --2- 172.21.15.152:0/1840142363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7214101350 0x7f721410d880 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f720c009cd0 tx=0x7f720c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.077+0000 7f721a850640 1 -- 172.21.15.152:0/1840142363 shutdown_connections 2024-09-11T22:53:01.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.077+0000 7f721a850640 1 --2- 172.21.15.152:0/1840142363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7214101350 0x7f721410d880 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.077+0000 7f721a850640 1 --2- 172.21.15.152:0/1840142363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72141009b0 0x7f7214100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.077+0000 7f721a850640 1 --2- 172.21.15.152:0/1840142363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72141069c0 0x7f7214106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.077+0000 7f721a850640 1 -- 172.21.15.152:0/1840142363 >> 172.21.15.152:0/1840142363 conn(0x7f7214077990 msgr2=0x7f72140fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:01.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.077+0000 7f721a850640 1 -- 172.21.15.152:0/1840142363 shutdown_connections 2024-09-11T22:53:01.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.077+0000 7f721a850640 1 -- 172.21.15.152:0/1840142363 wait complete. 2024-09-11T22:53:01.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f721a850640 1 Processor -- start 2024-09-11T22:53:01.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f721a850640 1 -- start start 2024-09-11T22:53:01.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f721a850640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72141009b0 0x7f721419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f721a850640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7214101350 0x7f721419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f721a850640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72141069c0 0x7f721419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f721a850640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7214110e00 con 0x7f72141009b0 2024-09-11T22:53:01.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f721a850640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7214110c80 con 0x7f7214101350 2024-09-11T22:53:01.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f721a850640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7214110f80 con 0x7f72141069c0 2024-09-11T22:53:01.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f7213fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72141009b0 0x7f721419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f7213fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72141009b0 0x7f721419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43582/0 (socket says 172.21.15.152:43582) 2024-09-11T22:53:01.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f7213fff640 1 -- 172.21.15.152:0/2698260817 learned_addr learned my addr 172.21.15.152:0/2698260817 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:01.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f7218ded640 1 --2- 172.21.15.152:0/2698260817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72141069c0 0x7f721419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f72137fe640 1 --2- 172.21.15.152:0/2698260817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7214101350 0x7f721419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f7213fff640 1 -- 172.21.15.152:0/2698260817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72141069c0 msgr2=0x7f721419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f7213fff640 1 --2- 172.21.15.152:0/2698260817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72141069c0 0x7f721419ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f7213fff640 1 -- 172.21.15.152:0/2698260817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7214101350 msgr2=0x7f721419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f7213fff640 1 --2- 172.21.15.152:0/2698260817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7214101350 0x7f721419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f7213fff640 1 -- 172.21.15.152:0/2698260817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721419f4c0 con 0x7f72141009b0 2024-09-11T22:53:01.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f7218ded640 1 --2- 172.21.15.152:0/2698260817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72141069c0 0x7f721419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:01.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.078+0000 7f7213fff640 1 --2- 172.21.15.152:0/2698260817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72141009b0 0x7f721419a4d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f720400e9f0 tx=0x7f720400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:01.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.079+0000 7f72117fa640 1 -- 172.21.15.152:0/2698260817 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720400cd80 con 0x7f72141009b0 2024-09-11T22:53:01.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.079+0000 7f72117fa640 1 -- 172.21.15.152:0/2698260817 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7204010910 con 0x7f72141009b0 2024-09-11T22:53:01.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.079+0000 7f721a850640 1 -- 172.21.15.152:0/2698260817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721419f7b0 con 0x7f72141009b0 2024-09-11T22:53:01.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.079+0000 7f72117fa640 1 -- 172.21.15.152:0/2698260817 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72040188f0 con 0x7f72141009b0 2024-09-11T22:53:01.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.079+0000 7f721a850640 1 -- 172.21.15.152:0/2698260817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72141a7040 con 0x7f72141009b0 2024-09-11T22:53:01.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.080+0000 7f721a850640 1 -- 172.21.15.152:0/2698260817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7214100e10 con 0x7f72141009b0 2024-09-11T22:53:01.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.081+0000 7f72117fa640 1 -- 172.21.15.152:0/2698260817 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f7204010ac0 con 0x7f72141009b0 2024-09-11T22:53:01.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.081+0000 7f72117fa640 1 --2- 172.21.15.152:0/2698260817 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f72000a2e40 0x7f72000a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.082+0000 7f72137fe640 1 --2- 172.21.15.152:0/2698260817 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f72000a2e40 0x7f72000a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.082+0000 7f72117fa640 1 -- 172.21.15.152:0/2698260817 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f72040cfa40 con 0x7f72141009b0 2024-09-11T22:53:01.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.082+0000 7f72137fe640 1 --2- 172.21.15.152:0/2698260817 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f72000a2e40 0x7f72000a5300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7214105380 tx=0x7f71fc00a4c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:01.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.086+0000 7f72117fa640 1 -- 172.21.15.152:0/2698260817 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f72040d1e20 con 0x7f72141009b0 2024-09-11T22:53:01.235 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:00 smithi152 ceph-mon[31181]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 205 B/s wr, 1 op/s 2024-09-11T22:53:01.235 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:00 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3706893558' entity='client.admin' 2024-09-11T22:53:01.235 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:00 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:53:01.235 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:00 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:53:01.235 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:00 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:53:01.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.261+0000 7f721a850640 1 -- 172.21.15.152:0/2698260817 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f72140fe420 con 0x7f72141009b0 2024-09-11T22:53:01.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.266+0000 7f72117fa640 1 -- 172.21.15.152:0/2698260817 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7f720409c540 con 0x7f72141009b0 2024-09-11T22:53:01.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.268+0000 7f721a850640 1 -- 172.21.15.152:0/2698260817 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f72000a2e40 msgr2=0x7f72000a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.268+0000 7f721a850640 1 --2- 172.21.15.152:0/2698260817 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f72000a2e40 0x7f72000a5300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7214105380 tx=0x7f71fc00a4c0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.268+0000 7f721a850640 1 -- 172.21.15.152:0/2698260817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72141009b0 msgr2=0x7f721419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.268+0000 7f721a850640 1 --2- 172.21.15.152:0/2698260817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72141009b0 0x7f721419a4d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f720400e9f0 tx=0x7f720400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.269+0000 7f721a850640 1 -- 172.21.15.152:0/2698260817 shutdown_connections 2024-09-11T22:53:01.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.269+0000 7f721a850640 1 --2- 172.21.15.152:0/2698260817 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f72000a2e40 0x7f72000a5300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.269+0000 7f721a850640 1 --2- 172.21.15.152:0/2698260817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f72141069c0 0x7f721419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.269+0000 7f721a850640 1 --2- 172.21.15.152:0/2698260817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7214101350 0x7f721419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.269+0000 7f721a850640 1 --2- 172.21.15.152:0/2698260817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72141009b0 0x7f721419a4d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.269+0000 7f721a850640 1 -- 172.21.15.152:0/2698260817 >> 172.21.15.152:0/2698260817 conn(0x7f7214077990 msgr2=0x7f72141047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:01.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.269+0000 7f721a850640 1 -- 172.21.15.152:0/2698260817 shutdown_connections 2024-09-11T22:53:01.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.269+0000 7f721a850640 1 -- 172.21.15.152:0/2698260817 wait complete. 2024-09-11T22:53:01.282 DEBUG:teuthology.orchestra.run.smithi152:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-11T22:53:01.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:00 smithi191 ceph-mon[35211]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 205 B/s wr, 1 op/s 2024-09-11T22:53:01.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:00 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3706893558' entity='client.admin' 2024-09-11T22:53:01.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:00 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:53:01.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:00 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:53:01.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:00 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:53:01.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:00 smithi204 ceph-mon[35454]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 205 B/s wr, 1 op/s 2024-09-11T22:53:01.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:00 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3706893558' entity='client.admin' 2024-09-11T22:53:01.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:00 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:53:01.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:00 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:53:01.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:00 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:53:01.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.385+0000 7f31823e8640 1 -- 172.21.15.152:0/1987360369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c101350 msgr2=0x7f317c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.385+0000 7f31823e8640 1 --2- 172.21.15.152:0/1987360369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c101350 0x7f317c10d880 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3174009cd0 tx=0x7f317402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.385+0000 7f31823e8640 1 -- 172.21.15.152:0/1987360369 shutdown_connections 2024-09-11T22:53:01.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.385+0000 7f31823e8640 1 --2- 172.21.15.152:0/1987360369 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c101350 0x7f317c10d880 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.385+0000 7f31823e8640 1 --2- 172.21.15.152:0/1987360369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f317c1009b0 0x7f317c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.385+0000 7f31823e8640 1 --2- 172.21.15.152:0/1987360369 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f317c1069c0 0x7f317c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.385+0000 7f31823e8640 1 -- 172.21.15.152:0/1987360369 >> 172.21.15.152:0/1987360369 conn(0x7f317c077990 msgr2=0x7f317c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:01.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.385+0000 7f31823e8640 1 -- 172.21.15.152:0/1987360369 shutdown_connections 2024-09-11T22:53:01.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.386+0000 7f31823e8640 1 -- 172.21.15.152:0/1987360369 wait complete. 2024-09-11T22:53:01.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.386+0000 7f31823e8640 1 Processor -- start 2024-09-11T22:53:01.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.386+0000 7f31823e8640 1 -- start start 2024-09-11T22:53:01.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.386+0000 7f31823e8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c1009b0 0x7f317c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.386+0000 7f31823e8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f317c101350 0x7f317c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.386+0000 7f31823e8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f317c1069c0 0x7f317c1a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.386+0000 7f31823e8640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f317c110e00 con 0x7f317c1009b0 2024-09-11T22:53:01.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.386+0000 7f31823e8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f317c110c80 con 0x7f317c1069c0 2024-09-11T22:53:01.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.386+0000 7f31823e8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f317c110f80 con 0x7f317c101350 2024-09-11T22:53:01.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f317bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c1009b0 0x7f317c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f317bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c1009b0 0x7f317c199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43598/0 (socket says 172.21.15.152:43598) 2024-09-11T22:53:01.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f317bfff640 1 -- 172.21.15.152:0/3536149386 learned_addr learned my addr 172.21.15.152:0/3536149386 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:01.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f3180985640 1 --2- 172.21.15.152:0/3536149386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f317c1069c0 0x7f317c1a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f317b7fe640 1 --2- 172.21.15.152:0/3536149386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f317c101350 0x7f317c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f317bfff640 1 -- 172.21.15.152:0/3536149386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f317c101350 msgr2=0x7f317c1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f317bfff640 1 --2- 172.21.15.152:0/3536149386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f317c101350 0x7f317c1996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f317bfff640 1 -- 172.21.15.152:0/3536149386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f317c1069c0 msgr2=0x7f317c1a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f317bfff640 1 --2- 172.21.15.152:0/3536149386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f317c1069c0 0x7f317c1a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f317bfff640 1 -- 172.21.15.152:0/3536149386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c1a4730 con 0x7f317c1009b0 2024-09-11T22:53:01.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f3180985640 1 --2- 172.21.15.152:0/3536149386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f317c1069c0 0x7f317c1a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:01.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f317b7fe640 1 --2- 172.21.15.152:0/3536149386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f317c101350 0x7f317c1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:01.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f317bfff640 1 --2- 172.21.15.152:0/3536149386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c1009b0 0x7f317c199170 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f316c00e9f0 tx=0x7f316c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:01.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f31797fa640 1 -- 172.21.15.152:0/3536149386 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c00cd80 con 0x7f317c1009b0 2024-09-11T22:53:01.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f31823e8640 1 -- 172.21.15.152:0/3536149386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317c1a4900 con 0x7f317c1009b0 2024-09-11T22:53:01.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f31797fa640 1 -- 172.21.15.152:0/3536149386 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f316c010910 con 0x7f317c1009b0 2024-09-11T22:53:01.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f31797fa640 1 -- 172.21.15.152:0/3536149386 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c0188f0 con 0x7f317c1009b0 2024-09-11T22:53:01.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.387+0000 7f31823e8640 1 -- 172.21.15.152:0/3536149386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317c1a4e20 con 0x7f317c1009b0 2024-09-11T22:53:01.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.388+0000 7f31823e8640 1 -- 172.21.15.152:0/3536149386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f317c100e10 con 0x7f317c1009b0 2024-09-11T22:53:01.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.389+0000 7f31797fa640 1 -- 172.21.15.152:0/3536149386 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f316c010ac0 con 0x7f317c1009b0 2024-09-11T22:53:01.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.390+0000 7f31797fa640 1 --2- 172.21.15.152:0/3536149386 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f31680a2e40 0x7f31680a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.390+0000 7f317b7fe640 1 --2- 172.21.15.152:0/3536149386 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f31680a2e40 0x7f31680a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.390+0000 7f31797fa640 1 -- 172.21.15.152:0/3536149386 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f316c0cfa40 con 0x7f317c1009b0 2024-09-11T22:53:01.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.390+0000 7f317b7fe640 1 --2- 172.21.15.152:0/3536149386 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f31680a2e40 0x7f31680a5300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f317c06a3f0 tx=0x7f3164005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:01.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.394+0000 7f31797fa640 1 -- 172.21.15.152:0/3536149386 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f316c0d1e20 con 0x7f317c1009b0 2024-09-11T22:53:01.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.476+0000 7fc3a2b0c640 1 -- 172.21.15.152:0/3559261874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39c102570 msgr2=0x7fc39c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.476+0000 7fc3a2b0c640 1 --2- 172.21.15.152:0/3559261874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39c102570 0x7fc39c1029b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc388009900 tx=0x7fc38802a480 comp rx=0 tx=0).stop 2024-09-11T22:53:01.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.476+0000 7fc3a2b0c640 1 -- 172.21.15.152:0/3559261874 shutdown_connections 2024-09-11T22:53:01.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.476+0000 7fc3a2b0c640 1 --2- 172.21.15.152:0/3559261874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c102ef0 0x7fc39c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.476+0000 7fc3a2b0c640 1 --2- 172.21.15.152:0/3559261874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39c102570 0x7fc39c1029b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.476+0000 7fc3a2b0c640 1 --2- 172.21.15.152:0/3559261874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc39c100e20 0x7fc39c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.476+0000 7fc3a2b0c640 1 -- 172.21.15.152:0/3559261874 >> 172.21.15.152:0/3559261874 conn(0x7fc39c0fc7d0 msgr2=0x7fc39c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:01.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.476+0000 7fc3a2b0c640 1 -- 172.21.15.152:0/3559261874 shutdown_connections 2024-09-11T22:53:01.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.476+0000 7fc3a2b0c640 1 -- 172.21.15.152:0/3559261874 wait complete. 2024-09-11T22:53:01.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a2b0c640 1 Processor -- start 2024-09-11T22:53:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a2b0c640 1 -- start start 2024-09-11T22:53:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a2b0c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39c100e20 0x7fc39c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a2b0c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c102570 0x7fc39c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a2b0c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc39c102ef0 0x7fc39c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a2b0c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c110f50 con 0x7fc39c100e20 2024-09-11T22:53:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a2b0c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c110dd0 con 0x7fc39c102ef0 2024-09-11T22:53:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a2b0c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c1110d0 con 0x7fc39c102570 2024-09-11T22:53:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a08a8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39c100e20 0x7fc39c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a08a8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39c100e20 0x7fc39c19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43612/0 (socket says 172.21.15.152:43612) 2024-09-11T22:53:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a08a8640 1 -- 172.21.15.152:0/27925646 learned_addr learned my addr 172.21.15.152:0/27925646 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a10a9640 1 --2- 172.21.15.152:0/27925646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc39c102ef0 0x7fc39c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc39bfff640 1 --2- 172.21.15.152:0/27925646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c102570 0x7fc39c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a08a8640 1 -- 172.21.15.152:0/27925646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c102570 msgr2=0x7fc39c19ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a08a8640 1 --2- 172.21.15.152:0/27925646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c102570 0x7fc39c19ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a08a8640 1 -- 172.21.15.152:0/27925646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc39c102ef0 msgr2=0x7fc39c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a08a8640 1 --2- 172.21.15.152:0/27925646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc39c102ef0 0x7fc39c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.477+0000 7fc3a08a8640 1 -- 172.21.15.152:0/27925646 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39c19f690 con 0x7fc39c100e20 2024-09-11T22:53:01.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.478+0000 7fc3a08a8640 1 --2- 172.21.15.152:0/27925646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39c100e20 0x7fc39c19a6e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc39000c890 tx=0x7fc39000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:01.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.478+0000 7fc399ffb640 1 -- 172.21.15.152:0/27925646 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390004220 con 0x7fc39c100e20 2024-09-11T22:53:01.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.478+0000 7fc399ffb640 1 -- 172.21.15.152:0/27925646 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3900043c0 con 0x7fc39c100e20 2024-09-11T22:53:01.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.478+0000 7fc399ffb640 1 -- 172.21.15.152:0/27925646 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3900049e0 con 0x7fc39c100e20 2024-09-11T22:53:01.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.478+0000 7fc3a2b0c640 1 -- 172.21.15.152:0/27925646 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39c19f980 con 0x7fc39c100e20 2024-09-11T22:53:01.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.478+0000 7fc3a2b0c640 1 -- 172.21.15.152:0/27925646 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc39c1a7260 con 0x7fc39c100e20 2024-09-11T22:53:01.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc3a4ee5740 1 Processor -- start 2024-09-11T22:53:01.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc3a4ee5740 1 -- start start 2024-09-11T22:53:01.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc3a4ee5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559865551b60 0x559865551f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc3a4ee5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559865552480 0x55986555c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc3a4ee5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55986555cd40 0x55986555f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc3a4ee5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5598654b7280 con 0x55986555cd40 2024-09-11T22:53:01.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc3a4ee5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55986545eb10 con 0x559865551b60 2024-09-11T22:53:01.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc3a4ee5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5598654b3240 con 0x559865552480 2024-09-11T22:53:01.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc399ffb640 1 -- 172.21.15.152:0/27925646 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7fc3900079b0 con 0x7fc39c100e20 2024-09-11T22:53:01.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc39bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55986555cd40 0x55986555f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc39bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55986555cd40 0x55986555f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43618/0 (socket says 172.21.15.152:43618) 2024-09-11T22:53:01.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc39bfff640 1 -- 172.21.15.152:0/4079138679 learned_addr learned my addr 172.21.15.152:0/4079138679 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:01.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc3a10a9640 1 --2- 172.21.15.152:0/4079138679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559865551b60 0x559865551f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc39bfff640 1 -- 172.21.15.152:0/4079138679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559865552480 msgr2=0x55986555c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:01.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc39bfff640 1 --2- 172.21.15.152:0/4079138679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559865552480 0x55986555c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc39bfff640 1 -- 172.21.15.152:0/4079138679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559865551b60 msgr2=0x559865551f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:01.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc39bfff640 1 --2- 172.21.15.152:0/4079138679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559865551b60 0x559865551f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc39bfff640 1 -- 172.21.15.152:0/4079138679 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55986546b1f0 con 0x55986555cd40 2024-09-11T22:53:01.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.481+0000 7fc3a10a9640 1 --2- 172.21.15.152:0/4079138679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559865551b60 0x559865551f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:01.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.482+0000 7fc39bfff640 1 --2- 172.21.15.152:0/4079138679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55986555cd40 0x55986555f130 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc3880314e0 tx=0x7fc388004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1ab2a16920f7b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:01.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.482+0000 7fc399ffb640 1 --2- 172.21.15.152:0/27925646 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc38c0a2e00 0x7fc38c0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.482+0000 7fc386ffd640 1 -- 172.21.15.152:0/4079138679 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388035030 con 0x55986555cd40 2024-09-11T22:53:01.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.482+0000 7fc386ffd640 1 -- 172.21.15.152:0/4079138679 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc388005520 con 0x55986555cd40 2024-09-11T22:53:01.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.482+0000 7fc386ffd640 1 -- 172.21.15.152:0/4079138679 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3880037f0 con 0x55986555cd40 2024-09-11T22:53:01.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.482+0000 7fc399ffb640 1 -- 172.21.15.152:0/27925646 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc3900cf5c0 con 0x7fc39c100e20 2024-09-11T22:53:01.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.482+0000 7fc3a10a9640 1 --2- 172.21.15.152:0/27925646 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc38c0a2e00 0x7fc38c0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.483+0000 7fc3a10a9640 1 --2- 172.21.15.152:0/27925646 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc38c0a2e00 0x7fc38c0a52c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc394005fd0 tx=0x7fc394009600 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:01.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.483+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/4079138679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55986555cd40 msgr2=0x55986555f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.483+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/4079138679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55986555cd40 0x55986555f130 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc3880314e0 tx=0x7fc388004170 comp rx=0 tx=0).stop 2024-09-11T22:53:01.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.483+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/4079138679 shutdown_connections 2024-09-11T22:53:01.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.483+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/4079138679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55986555cd40 0x55986555f130 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.483+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/4079138679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559865552480 0x55986555c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.483+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/4079138679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559865551b60 0x559865551f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.483+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/4079138679 >> 172.21.15.152:0/4079138679 conn(0x55986554e0f0 msgr2=0x55986554e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:01.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.483+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/4079138679 shutdown_connections 2024-09-11T22:53:01.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.483+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/4079138679 wait complete. 2024-09-11T22:53:01.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.483+0000 7fc3a4ee5740 1 -- start start 2024-09-11T22:53:01.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.483+0000 7fc3a4ee5740 1 Processor -- start 2024-09-11T22:53:01.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a4ee5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865551b60 0x559865559c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a4ee5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559865552480 0x55986555a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a4ee5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55986555cd40 0x55986555a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a4ee5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55986555f910 con 0x559865551b60 2024-09-11T22:53:01.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a4ee5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559865471f10 con 0x55986555cd40 2024-09-11T22:53:01.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a4ee5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55986555fa90 con 0x559865552480 2024-09-11T22:53:01.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a4ee5740 10 client.0 _ll_register_callbacks cb 0x55986543aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:01.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a08a8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865551b60 0x559865559c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a08a8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865551b60 0x559865559c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43634/0 (socket says 172.21.15.152:43634) 2024-09-11T22:53:01.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a08a8640 1 -- 172.21.15.152:0/1653170356 learned_addr learned my addr 172.21.15.152:0/1653170356 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:01.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc39bfff640 1 --2- 172.21.15.152:0/1653170356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559865552480 0x55986555a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a10a9640 1 --2- 172.21.15.152:0/1653170356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55986555cd40 0x55986555a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a08a8640 1 -- 172.21.15.152:0/1653170356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559865552480 msgr2=0x55986555a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:01.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a08a8640 1 --2- 172.21.15.152:0/1653170356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559865552480 0x55986555a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a08a8640 1 -- 172.21.15.152:0/1653170356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55986555cd40 msgr2=0x55986555a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:01.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a08a8640 1 --2- 172.21.15.152:0/1653170356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55986555cd40 0x55986555a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a08a8640 1 -- 172.21.15.152:0/1653170356 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598652286a0 con 0x559865551b60 2024-09-11T22:53:01.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a10a9640 1 --2- 172.21.15.152:0/1653170356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55986555cd40 0x55986555a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:01.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc3a08a8640 1 --2- 172.21.15.152:0/1653170356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865551b60 0x559865559c60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc3900ccd30 tx=0x7fc39009b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914eeb23e325c500 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:01.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.484+0000 7fc37bfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:01.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.485+0000 7fc37bfff640 1 -- 172.21.15.152:0/1653170356 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39001d040 con 0x559865551b60 2024-09-11T22:53:01.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.485+0000 7fc37bfff640 1 -- 172.21.15.152:0/1653170356 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc3900a43a0 con 0x559865551b60 2024-09-11T22:53:01.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.485+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/1653170356 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc388005520 con 0x559865551b60 2024-09-11T22:53:01.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.485+0000 7fc37bfff640 1 -- 172.21.15.152:0/1653170356 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc39009b9c0 con 0x559865551b60 2024-09-11T22:53:01.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.485+0000 7fc3a4ee5740 10 client.15384 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:01.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.485+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/1653170356 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55986556b550 con 0x559865551b60 2024-09-11T22:53:01.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.485+0000 7fc37bfff640 1 -- 172.21.15.152:0/1653170356 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc388005520 con 0x559865551b60 2024-09-11T22:53:01.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.485+0000 7fc37bfff640 1 -- 172.21.15.152:0/1653170356 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc3900ccfe0 con 0x559865551b60 2024-09-11T22:53:01.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.485+0000 7fc3a4ee5740 10 client.15384 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:01.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.485+0000 7fc3a4ee5740 10 client.15384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:01.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.485+0000 7fc3a4ee5740 20 client.15384 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:01.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.485+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/1653170356 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55986556ca10 0x55986556ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.485+0000 7fc3a4ee5740 4 client.15384 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:01.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.485+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/1653170356 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598654c47f0 con 0x55986556ca10 2024-09-11T22:53:01.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.486+0000 7fc39bfff640 1 --2- 172.21.15.152:0/1653170356 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55986556ca10 0x55986556ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.486+0000 7fc39bfff640 1 --2- 172.21.15.152:0/1653170356 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55986556ca10 0x55986556ee00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f07123a2e5f47ee3 server_cookie=73d122767f1858cc in_seq=0 out_seq=0 2024-09-11T22:53:01.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.486+0000 7fc37bfff640 10 client.15384 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:01.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.487+0000 7fc37bfff640 1 -- 172.21.15.152:0/1653170356 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5598654c47f0 con 0x55986556ca10 2024-09-11T22:53:01.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.487+0000 7fc37bfff640 10 client.15384 handle_command_reply: tid=0 2024-09-11T22:53:01.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.487+0000 7fc3a4ee5740 1 client.15384 shutdown 2024-09-11T22:53:01.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.487+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/1653170356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865551b60 msgr2=0x559865559c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.487+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/1653170356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865551b60 0x559865559c60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc3900ccd30 tx=0x7fc39009b500 comp rx=0 tx=0).stop 2024-09-11T22:53:01.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.487+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/1653170356 shutdown_connections 2024-09-11T22:53:01.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.487+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/1653170356 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55986556ca10 0x55986556ee00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.487+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/1653170356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55986555cd40 0x55986555a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.487+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/1653170356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559865552480 0x55986555a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.487+0000 7fc37bfff640 0 client.15384 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:01.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.487+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/1653170356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865551b60 0x559865559c60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.487+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/1653170356 >> 172.21.15.152:0/1653170356 conn(0x55986554e0f0 msgr2=0x559865553dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:01.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.487+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/1653170356 shutdown_connections 2024-09-11T22:53:01.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.487+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/1653170356 wait complete. 2024-09-11T22:53:01.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.487+0000 7fc3a4ee5740 20 client.15384 trim_cache size 0 max 16384 2024-09-11T22:53:01.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.504+0000 7fc3a4ee5740 1 Processor -- start 2024-09-11T22:53:01.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.504+0000 7fc3a4ee5740 1 -- start start 2024-09-11T22:53:01.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.504+0000 7fc3a4ee5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865551b60 0x55986555bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.504+0000 7fc3a4ee5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559865552480 0x559865574610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.504+0000 7fc3a4ee5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55986555cd40 0x559865574b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.504+0000 7fc3a4ee5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c1af870 con 0x559865551b60 2024-09-11T22:53:01.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.504+0000 7fc3a4ee5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c1b13b0 con 0x559865552480 2024-09-11T22:53:01.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.504+0000 7fc3a4ee5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c1aff40 con 0x55986555cd40 2024-09-11T22:53:01.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.505+0000 7fc3a08a8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865551b60 0x55986555bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.505+0000 7fc3a08a8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865551b60 0x55986555bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43638/0 (socket says 172.21.15.152:43638) 2024-09-11T22:53:01.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.505+0000 7fc3a08a8640 1 -- 172.21.15.152:0/3549613746 learned_addr learned my addr 172.21.15.152:0/3549613746 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:01.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.505+0000 7fc3a10a9640 1 --2- 172.21.15.152:0/3549613746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55986555cd40 0x559865574b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.505+0000 7fc39bfff640 1 --2- 172.21.15.152:0/3549613746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559865552480 0x559865574610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.505+0000 7fc3a08a8640 1 -- 172.21.15.152:0/3549613746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55986555cd40 msgr2=0x559865574b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:01.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.505+0000 7fc3a08a8640 1 --2- 172.21.15.152:0/3549613746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55986555cd40 0x559865574b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.505+0000 7fc3a08a8640 1 -- 172.21.15.152:0/3549613746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559865552480 msgr2=0x559865574610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:01.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.505+0000 7fc3a08a8640 1 --2- 172.21.15.152:0/3549613746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559865552480 0x559865574610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.505+0000 7fc3a08a8640 1 -- 172.21.15.152:0/3549613746 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598655581b0 con 0x559865551b60 2024-09-11T22:53:01.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.505+0000 7fc3a08a8640 1 --2- 172.21.15.152:0/3549613746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865551b60 0x55986555bac0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc39009b6c0 tx=0x7fc39009b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea5655c1eae13bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:01.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.505+0000 7fc3867fc640 1 -- 172.21.15.152:0/3549613746 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39001d050 con 0x559865551b60 2024-09-11T22:53:01.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.505+0000 7fc3867fc640 1 -- 172.21.15.152:0/3549613746 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3900a4630 con 0x559865551b60 2024-09-11T22:53:01.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.505+0000 7fc3867fc640 1 -- 172.21.15.152:0/3549613746 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3900a41a0 con 0x559865551b60 2024-09-11T22:53:01.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.506+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/3549613746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865551b60 msgr2=0x55986555bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.506+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/3549613746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865551b60 0x55986555bac0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc39009b6c0 tx=0x7fc39009b630 comp rx=0 tx=0).stop 2024-09-11T22:53:01.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.506+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/3549613746 shutdown_connections 2024-09-11T22:53:01.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.506+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/3549613746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55986555cd40 0x559865574b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.506+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/3549613746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559865552480 0x559865574610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.506+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/3549613746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865551b60 0x55986555bac0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.506+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/3549613746 >> 172.21.15.152:0/3549613746 conn(0x55986554e0f0 msgr2=0x55986555b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:01.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.506+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/3549613746 shutdown_connections 2024-09-11T22:53:01.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.506+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/3549613746 wait complete. 2024-09-11T22:53:01.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.506+0000 7fc3a4ee5740 1 -- start start 2024-09-11T22:53:01.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc3a4ee5740 1 Processor -- start 2024-09-11T22:53:01.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc3a4ee5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559865551b60 0x559865556e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc3a4ee5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865552480 0x559865557350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc3a4ee5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55986555cd40 0x559865557890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc3a4ee5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c1b0ce0 con 0x559865552480 2024-09-11T22:53:01.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc3a4ee5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c1b0610 con 0x55986555cd40 2024-09-11T22:53:01.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc3a4ee5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c1ad830 con 0x559865551b60 2024-09-11T22:53:01.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc39bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865552480 0x559865557350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc3a4ee5740 10 client.0 _ll_register_callbacks cb 0x7fc3900ccb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:01.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc39bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865552480 0x559865557350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43648/0 (socket says 172.21.15.152:43648) 2024-09-11T22:53:01.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc39bfff640 1 -- 172.21.15.152:0/3451673398 learned_addr learned my addr 172.21.15.152:0/3451673398 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:01.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc3a08a8640 1 --2- 172.21.15.152:0/3451673398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559865551b60 0x559865556e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc3a10a9640 1 --2- 172.21.15.152:0/3451673398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55986555cd40 0x559865557890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc39bfff640 1 -- 172.21.15.152:0/3451673398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559865551b60 msgr2=0x559865556e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:01.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc39bfff640 1 --2- 172.21.15.152:0/3451673398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559865551b60 0x559865556e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc39bfff640 1 -- 172.21.15.152:0/3451673398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55986555cd40 msgr2=0x559865557890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:01.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.507+0000 7fc39bfff640 1 --2- 172.21.15.152:0/3451673398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55986555cd40 0x559865557890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.508+0000 7fc39bfff640 1 -- 172.21.15.152:0/3451673398 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598655d6750 con 0x559865552480 2024-09-11T22:53:01.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.508+0000 7fc39bfff640 1 --2- 172.21.15.152:0/3451673398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865552480 0x559865557350 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc388005c40 tx=0x7fc3880058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6959f8b3fe47b457 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:01.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.508+0000 7fc37bfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:01.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.508+0000 7fc37bfff640 1 -- 172.21.15.152:0/3451673398 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388035040 con 0x559865552480 2024-09-11T22:53:01.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.508+0000 7fc37bfff640 1 -- 172.21.15.152:0/3451673398 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc388004d70 con 0x559865552480 2024-09-11T22:53:01.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.508+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/3451673398 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3900a4630 con 0x559865552480 2024-09-11T22:53:01.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.508+0000 7fc37bfff640 1 -- 172.21.15.152:0/3451673398 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc38805b020 con 0x559865552480 2024-09-11T22:53:01.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.508+0000 7fc3a4ee5740 10 client.15390 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:01.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.508+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/3451673398 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598655d7a80 con 0x559865552480 2024-09-11T22:53:01.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.508+0000 7fc37bfff640 1 -- 172.21.15.152:0/3451673398 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3900a4630 con 0x559865552480 2024-09-11T22:53:01.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.509+0000 7fc37bfff640 1 -- 172.21.15.152:0/3451673398 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc38804c070 con 0x559865552480 2024-09-11T22:53:01.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.509+0000 7fc3a4ee5740 10 client.15390 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:01.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.509+0000 7fc3a4ee5740 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:01.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.509+0000 7fc3a4ee5740 20 client.15390 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:01.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.509+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/3451673398 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55986556ca10 0x5598655d9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.509+0000 7fc3a4ee5740 4 client.15390 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:53:01.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.509+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/3451673398 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5598654bed00 con 0x55986556ca10 2024-09-11T22:53:01.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.509+0000 7fc3a08a8640 1 --2- 172.21.15.152:0/3451673398 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55986556ca10 0x5598655d9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.509+0000 7fc3a08a8640 1 --2- 172.21.15.152:0/3451673398 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55986556ca10 0x5598655d9e20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3827aa9a87ffd69 server_cookie=8b718fcc6b6b9131 in_seq=0 out_seq=0 2024-09-11T22:53:01.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.509+0000 7fc37bfff640 10 client.15390 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:01.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.510+0000 7fc37bfff640 1 -- 172.21.15.152:0/3451673398 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5598654bed00 con 0x55986556ca10 2024-09-11T22:53:01.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.510+0000 7fc37bfff640 10 client.15390 handle_command_reply: tid=0 2024-09-11T22:53:01.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.510+0000 7fc3a4ee5740 1 client.15390 shutdown 2024-09-11T22:53:01.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.510+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/3451673398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865552480 msgr2=0x559865557350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.510+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/3451673398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865552480 0x559865557350 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc388005c40 tx=0x7fc3880058b0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.511+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/3451673398 shutdown_connections 2024-09-11T22:53:01.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.511+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/3451673398 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55986556ca10 0x5598655d9e20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.511+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/3451673398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55986555cd40 0x559865557890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.511+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/3451673398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559865552480 0x559865557350 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.511+0000 7fc37bfff640 0 client.15390 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:01.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.511+0000 7fc3a4ee5740 1 --2- 172.21.15.152:0/3451673398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559865551b60 0x559865556e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.511+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/3451673398 >> 172.21.15.152:0/3451673398 conn(0x55986554e0f0 msgr2=0x55986554d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:01.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.511+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/3451673398 shutdown_connections 2024-09-11T22:53:01.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.511+0000 7fc3a4ee5740 1 -- 172.21.15.152:0/3451673398 wait complete. 2024-09-11T22:53:01.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.511+0000 7fc3a4ee5740 20 client.15390 trim_cache size 0 max 16384 2024-09-11T22:53:01.519 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:53:01.519 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T22:53:01.520 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "b595479b-b04b-4bf7-afa4-3414b25b7cdb", 2024-09-11T22:53:01.520 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T22:53:01.520 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:53:01.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.512+0000 7fc3a2b0c640 1 -- 172.21.15.152:0/27925646 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc38c0a2e00 msgr2=0x7fc38c0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.512+0000 7fc3a2b0c640 1 --2- 172.21.15.152:0/27925646 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc38c0a2e00 0x7fc38c0a52c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc394005fd0 tx=0x7fc394009600 comp rx=0 tx=0).stop 2024-09-11T22:53:01.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.512+0000 7fc3a2b0c640 1 -- 172.21.15.152:0/27925646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39c100e20 msgr2=0x7fc39c19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.512+0000 7fc3a2b0c640 1 --2- 172.21.15.152:0/27925646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39c100e20 0x7fc39c19a6e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc39000c890 tx=0x7fc39000cd50 comp rx=0 tx=0).stop 2024-09-11T22:53:01.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.512+0000 7fc3a2b0c640 1 -- 172.21.15.152:0/27925646 shutdown_connections 2024-09-11T22:53:01.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.512+0000 7fc3a2b0c640 1 --2- 172.21.15.152:0/27925646 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc38c0a2e00 0x7fc38c0a52c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.512+0000 7fc3a2b0c640 1 --2- 172.21.15.152:0/27925646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc39c102ef0 0x7fc39c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.512+0000 7fc3a2b0c640 1 --2- 172.21.15.152:0/27925646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc39c102570 0x7fc39c19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.512+0000 7fc3a2b0c640 1 --2- 172.21.15.152:0/27925646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc39c100e20 0x7fc39c19a6e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.512+0000 7fc3a2b0c640 1 -- 172.21.15.152:0/27925646 >> 172.21.15.152:0/27925646 conn(0x7fc39c0fc7d0 msgr2=0x7fc39c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:01.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.512+0000 7fc3a2b0c640 1 -- 172.21.15.152:0/27925646 shutdown_connections 2024-09-11T22:53:01.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.512+0000 7fc3a2b0c640 1 -- 172.21.15.152:0/27925646 wait complete. 2024-09-11T22:53:01.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.584+0000 7f31823e8640 1 -- 172.21.15.152:0/3536149386 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f317c1a50c0 con 0x7f317c1009b0 2024-09-11T22:53:01.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.588+0000 7f31797fa640 1 -- 172.21.15.152:0/3536149386 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) ==== 133+0+0 (secure 0 0 0) 0x7f316c09c540 con 0x7f317c1009b0 2024-09-11T22:53:01.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.591+0000 7f31823e8640 1 -- 172.21.15.152:0/3536149386 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f31680a2e40 msgr2=0x7f31680a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.591+0000 7f31823e8640 1 --2- 172.21.15.152:0/3536149386 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f31680a2e40 0x7f31680a5300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f317c06a3f0 tx=0x7f3164005f90 comp rx=0 tx=0).stop 2024-09-11T22:53:01.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.591+0000 7f31823e8640 1 -- 172.21.15.152:0/3536149386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c1009b0 msgr2=0x7f317c199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.591+0000 7f31823e8640 1 --2- 172.21.15.152:0/3536149386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c1009b0 0x7f317c199170 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f316c00e9f0 tx=0x7f316c00eeb0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.591+0000 7f31823e8640 1 -- 172.21.15.152:0/3536149386 shutdown_connections 2024-09-11T22:53:01.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.591+0000 7f31823e8640 1 --2- 172.21.15.152:0/3536149386 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f31680a2e40 0x7f31680a5300 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.591+0000 7f31823e8640 1 --2- 172.21.15.152:0/3536149386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f317c1069c0 0x7f317c1a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.591+0000 7f31823e8640 1 --2- 172.21.15.152:0/3536149386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f317c101350 0x7f317c1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.591+0000 7f31823e8640 1 --2- 172.21.15.152:0/3536149386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f317c1009b0 0x7f317c199170 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.591+0000 7f31823e8640 1 -- 172.21.15.152:0/3536149386 >> 172.21.15.152:0/3536149386 conn(0x7f317c077990 msgr2=0x7f317c103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:01.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.591+0000 7f31823e8640 1 -- 172.21.15.152:0/3536149386 shutdown_connections 2024-09-11T22:53:01.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.591+0000 7f31823e8640 1 -- 172.21.15.152:0/3536149386 wait complete. 2024-09-11T22:53:01.604 DEBUG:teuthology.orchestra.run.smithi152:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-11T22:53:01.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.709+0000 7f0235696640 1 -- 172.21.15.152:0/114364217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0230105af0 msgr2=0x7f0230105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.709+0000 7f0235696640 1 --2- 172.21.15.152:0/114364217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0230105af0 0x7f0230105ed0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0220009900 tx=0x7f022002a480 comp rx=0 tx=0).stop 2024-09-11T22:53:01.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.709+0000 7f0235696640 1 -- 172.21.15.152:0/114364217 shutdown_connections 2024-09-11T22:53:01.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.709+0000 7f0235696640 1 --2- 172.21.15.152:0/114364217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0230102050 0x7f023010d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.709+0000 7f0235696640 1 --2- 172.21.15.152:0/114364217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02301016d0 0x7f0230101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.709+0000 7f0235696640 1 --2- 172.21.15.152:0/114364217 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0230105af0 0x7f0230105ed0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.709+0000 7f0235696640 1 -- 172.21.15.152:0/114364217 >> 172.21.15.152:0/114364217 conn(0x7f0230075f20 msgr2=0x7f0230076330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:01.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.709+0000 7f0235696640 1 -- 172.21.15.152:0/114364217 shutdown_connections 2024-09-11T22:53:01.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.709+0000 7f0235696640 1 -- 172.21.15.152:0/114364217 wait complete. 2024-09-11T22:53:01.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.710+0000 7f0235696640 1 Processor -- start 2024-09-11T22:53:01.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.710+0000 7f0235696640 1 -- start start 2024-09-11T22:53:01.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.710+0000 7f0235696640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02301016d0 0x7f0230198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.710+0000 7f0235696640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0230102050 0x7f02301987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.710+0000 7f0235696640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0230105af0 0x7f023019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.710+0000 7f0235696640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0230111ec0 con 0x7f02301016d0 2024-09-11T22:53:01.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.710+0000 7f0235696640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0230111d40 con 0x7f0230105af0 2024-09-11T22:53:01.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.710+0000 7f0235696640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0230112040 con 0x7f0230102050 2024-09-11T22:53:01.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.710+0000 7f022effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02301016d0 0x7f0230198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.710+0000 7f022effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02301016d0 0x7f0230198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43670/0 (socket says 172.21.15.152:43670) 2024-09-11T22:53:01.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.710+0000 7f022effd640 1 -- 172.21.15.152:0/977949433 learned_addr learned my addr 172.21.15.152:0/977949433 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:01.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.710+0000 7f02267fc640 1 --2- 172.21.15.152:0/977949433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0230102050 0x7f02301987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.711+0000 7f022f7fe640 1 --2- 172.21.15.152:0/977949433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0230105af0 0x7f023019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.711+0000 7f022effd640 1 -- 172.21.15.152:0/977949433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0230102050 msgr2=0x7f02301987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.711+0000 7f022effd640 1 --2- 172.21.15.152:0/977949433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0230102050 0x7f02301987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.711+0000 7f022effd640 1 -- 172.21.15.152:0/977949433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0230105af0 msgr2=0x7f023019cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.711+0000 7f022effd640 1 --2- 172.21.15.152:0/977949433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0230105af0 0x7f023019cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.711+0000 7f022effd640 1 -- 172.21.15.152:0/977949433 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023019d260 con 0x7f02301016d0 2024-09-11T22:53:01.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.711+0000 7f022f7fe640 1 --2- 172.21.15.152:0/977949433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0230105af0 0x7f023019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:01.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.711+0000 7f022effd640 1 --2- 172.21.15.152:0/977949433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02301016d0 0x7f0230198270 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0220006010 tx=0x7f02200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:01.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.711+0000 7f022cff9640 1 -- 172.21.15.152:0/977949433 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022003f030 con 0x7f02301016d0 2024-09-11T22:53:01.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.711+0000 7f022cff9640 1 -- 172.21.15.152:0/977949433 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0220003ce0 con 0x7f02301016d0 2024-09-11T22:53:01.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.711+0000 7f022cff9640 1 -- 172.21.15.152:0/977949433 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220035070 con 0x7f02301016d0 2024-09-11T22:53:01.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.711+0000 7f0235696640 1 -- 172.21.15.152:0/977949433 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f023019d4f0 con 0x7f02301016d0 2024-09-11T22:53:01.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.711+0000 7f0235696640 1 -- 172.21.15.152:0/977949433 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02301a4de0 con 0x7f02301016d0 2024-09-11T22:53:01.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.712+0000 7f0235696640 1 -- 172.21.15.152:0/977949433 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0230101b10 con 0x7f02301016d0 2024-09-11T22:53:01.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.713+0000 7f022cff9640 1 -- 172.21.15.152:0/977949433 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f0220042030 con 0x7f02301016d0 2024-09-11T22:53:01.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.714+0000 7f022cff9640 1 --2- 172.21.15.152:0/977949433 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f021c0a2ec0 0x7f021c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:01.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.714+0000 7f02267fc640 1 --2- 172.21.15.152:0/977949433 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f021c0a2ec0 0x7f021c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:01.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.714+0000 7f022cff9640 1 -- 172.21.15.152:0/977949433 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f02200ef990 con 0x7f02301016d0 2024-09-11T22:53:01.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.714+0000 7f02267fc640 1 --2- 172.21.15.152:0/977949433 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f021c0a2ec0 0x7f021c0a5380 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0230076420 tx=0x7f021800a2d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:01.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.719+0000 7f022cff9640 1 -- 172.21.15.152:0/977949433 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02200bc450 con 0x7f02301016d0 2024-09-11T22:53:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.914+0000 7f0235696640 1 -- 172.21.15.152:0/977949433 --> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f02300ff880 con 0x7f021c0a2ec0 2024-09-11T22:53:01.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.979+0000 7f022cff9640 1 -- 172.21.15.152:0/977949433 <== mgr.15240 v2:172.21.15.152:6800/4130474708 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f02300ff880 con 0x7f021c0a2ec0 2024-09-11T22:53:01.980 INFO:teuthology.orchestra.run.smithi152.stdout:Schedule set for path / 2024-09-11T22:53:01.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.982+0000 7f0235696640 1 -- 172.21.15.152:0/977949433 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f021c0a2ec0 msgr2=0x7f021c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.982+0000 7f0235696640 1 --2- 172.21.15.152:0/977949433 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f021c0a2ec0 0x7f021c0a5380 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0230076420 tx=0x7f021800a2d0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.982+0000 7f0235696640 1 -- 172.21.15.152:0/977949433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02301016d0 msgr2=0x7f0230198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:01.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.982+0000 7f0235696640 1 --2- 172.21.15.152:0/977949433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02301016d0 0x7f0230198270 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0220006010 tx=0x7f02200042f0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.982+0000 7f0235696640 1 -- 172.21.15.152:0/977949433 shutdown_connections 2024-09-11T22:53:01.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.982+0000 7f0235696640 1 --2- 172.21.15.152:0/977949433 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f021c0a2ec0 0x7f021c0a5380 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.982+0000 7f0235696640 1 --2- 172.21.15.152:0/977949433 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0230105af0 0x7f023019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.982+0000 7f0235696640 1 --2- 172.21.15.152:0/977949433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0230102050 0x7f02301987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.982+0000 7f0235696640 1 --2- 172.21.15.152:0/977949433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f02301016d0 0x7f0230198270 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:01.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.982+0000 7f0235696640 1 -- 172.21.15.152:0/977949433 >> 172.21.15.152:0/977949433 conn(0x7f0230075f20 msgr2=0x7f02300ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:01.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.982+0000 7f0235696640 1 -- 172.21.15.152:0/977949433 shutdown_connections 2024-09-11T22:53:01.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:01.982+0000 7f0235696640 1 -- 172.21.15.152:0/977949433 wait complete. 2024-09-11T22:53:01.994 DEBUG:teuthology.orchestra.run.smithi152:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-11T22:53:02.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.098+0000 7f62d14d7640 1 -- 172.21.15.152:0/3100508619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62cc101ad0 msgr2=0x7f62cc101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:02.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.098+0000 7f62d14d7640 1 --2- 172.21.15.152:0/3100508619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62cc101ad0 0x7f62cc101f30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009930 tx=0x7f62c002a420 comp rx=0 tx=0).stop 2024-09-11T22:53:02.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.098+0000 7f62d14d7640 1 -- 172.21.15.152:0/3100508619 shutdown_connections 2024-09-11T22:53:02.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.098+0000 7f62d14d7640 1 --2- 172.21.15.152:0/3100508619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62cc102470 0x7f62cc10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.098+0000 7f62d14d7640 1 --2- 172.21.15.152:0/3100508619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62cc101ad0 0x7f62cc101f30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.098+0000 7f62d14d7640 1 --2- 172.21.15.152:0/3100508619 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62cc107b70 0x7f62cc107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.098+0000 7f62d14d7640 1 -- 172.21.15.152:0/3100508619 >> 172.21.15.152:0/3100508619 conn(0x7f62cc0fd7d0 msgr2=0x7f62cc0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:02.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.098+0000 7f62d14d7640 1 -- 172.21.15.152:0/3100508619 shutdown_connections 2024-09-11T22:53:02.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.098+0000 7f62d14d7640 1 -- 172.21.15.152:0/3100508619 wait complete. 2024-09-11T22:53:02.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.099+0000 7f62d14d7640 1 Processor -- start 2024-09-11T22:53:02.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.099+0000 7f62d14d7640 1 -- start start 2024-09-11T22:53:02.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.099+0000 7f62d14d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62cc101ad0 0x7f62cc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:02.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.099+0000 7f62d14d7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62cc102470 0x7f62cc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:02.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.099+0000 7f62d14d7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62cc107b70 0x7f62cc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:02.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.099+0000 7f62d14d7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc111f20 con 0x7f62cc102470 2024-09-11T22:53:02.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.099+0000 7f62d14d7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc111da0 con 0x7f62cc107b70 2024-09-11T22:53:02.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.099+0000 7f62d14d7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc1120a0 con 0x7f62cc101ad0 2024-09-11T22:53:02.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.099+0000 7f62ca7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62cc102470 0x7f62cc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:02.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.099+0000 7f62ca7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62cc102470 0x7f62cc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43686/0 (socket says 172.21.15.152:43686) 2024-09-11T22:53:02.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.099+0000 7f62ca7fc640 1 -- 172.21.15.152:0/3839441486 learned_addr learned my addr 172.21.15.152:0/3839441486 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:02.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.099+0000 7f62caffd640 1 --2- 172.21.15.152:0/3839441486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62cc101ad0 0x7f62cc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:02.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.100+0000 7f62cb7fe640 1 --2- 172.21.15.152:0/3839441486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62cc107b70 0x7f62cc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:02.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.100+0000 7f62ca7fc640 1 -- 172.21.15.152:0/3839441486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62cc101ad0 msgr2=0x7f62cc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:02.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.100+0000 7f62ca7fc640 1 --2- 172.21.15.152:0/3839441486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62cc101ad0 0x7f62cc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.100+0000 7f62ca7fc640 1 -- 172.21.15.152:0/3839441486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62cc107b70 msgr2=0x7f62cc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:02.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.100+0000 7f62ca7fc640 1 --2- 172.21.15.152:0/3839441486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62cc107b70 0x7f62cc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.100+0000 7f62ca7fc640 1 -- 172.21.15.152:0/3839441486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62cc1a6880 con 0x7f62cc102470 2024-09-11T22:53:02.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.100+0000 7f62cb7fe640 1 --2- 172.21.15.152:0/3839441486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62cc107b70 0x7f62cc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:02.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.100+0000 7f62ca7fc640 1 --2- 172.21.15.152:0/3839441486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62cc102470 0x7f62cc074a30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009230 tx=0x7f62c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:02.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.100+0000 7f62b3fff640 1 -- 172.21.15.152:0/3839441486 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c003f030 con 0x7f62cc102470 2024-09-11T22:53:02.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.100+0000 7f62d14d7640 1 -- 172.21.15.152:0/3839441486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62cc1a6b10 con 0x7f62cc102470 2024-09-11T22:53:02.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.100+0000 7f62b3fff640 1 -- 172.21.15.152:0/3839441486 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c0003ad0 con 0x7f62cc102470 2024-09-11T22:53:02.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.100+0000 7f62b3fff640 1 -- 172.21.15.152:0/3839441486 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0035070 con 0x7f62cc102470 2024-09-11T22:53:02.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.100+0000 7f62d14d7640 1 -- 172.21.15.152:0/3839441486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62cc1a7000 con 0x7f62cc102470 2024-09-11T22:53:02.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.101+0000 7f62d14d7640 1 -- 172.21.15.152:0/3839441486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62cc101f30 con 0x7f62cc102470 2024-09-11T22:53:02.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.103+0000 7f62b3fff640 1 -- 172.21.15.152:0/3839441486 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f62c0042030 con 0x7f62cc102470 2024-09-11T22:53:02.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.103+0000 7f62b3fff640 1 --2- 172.21.15.152:0/3839441486 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f62b40a2e40 0x7f62b40a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:02.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.103+0000 7f62caffd640 1 --2- 172.21.15.152:0/3839441486 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f62b40a2e40 0x7f62b40a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:02.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.104+0000 7f62b3fff640 1 -- 172.21.15.152:0/3839441486 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f62c00f0000 con 0x7f62cc102470 2024-09-11T22:53:02.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.104+0000 7f62caffd640 1 --2- 172.21.15.152:0/3839441486 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f62b40a2e40 0x7f62b40a5300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f62bc005d00 tx=0x7f62bc00b410 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:02.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.107+0000 7f62b3fff640 1 -- 172.21.15.152:0/3839441486 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62c00bcb00 con 0x7f62cc102470 2024-09-11T22:53:02.255 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:01 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:53:02.256 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:01 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2698260817' entity='client.admin' 2024-09-11T22:53:02.256 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:01 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:53:02.256 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:01 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:53:02.256 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:01 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:53:02.256 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:01 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:53:02.256 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:01 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3536149386' entity='client.admin' 2024-09-11T22:53:02.256 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:01 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:53:02.256 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:01 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:53:02.256 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:01 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:53:02.256 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:01 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:53:02.283 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:53:02.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:01 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:53:02.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:01 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2698260817' entity='client.admin' 2024-09-11T22:53:02.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:01 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:53:02.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:01 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:53:02.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:01 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:53:02.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:01 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:53:02.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:01 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3536149386' entity='client.admin' 2024-09-11T22:53:02.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:01 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:53:02.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:01 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:53:02.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:01 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:53:02.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:01 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:53:02.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:01 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:53:02.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:01 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2698260817' entity='client.admin' 2024-09-11T22:53:02.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:01 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:53:02.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:01 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:53:02.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:01 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:53:02.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:01 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:53:02.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:01 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3536149386' entity='client.admin' 2024-09-11T22:53:02.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:01 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:53:02.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:01 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:53:02.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:01 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:53:02.297 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:01 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:53:02.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.303+0000 7f62d14d7640 1 -- 172.21.15.152:0/3839441486 --> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f62cc0fe690 con 0x7f62b40a2e40 2024-09-11T22:53:02.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.368+0000 7f62b3fff640 1 -- 172.21.15.152:0/3839441486 <== mgr.15240 v2:172.21.15.152:6800/4130474708 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f62cc0fe690 con 0x7f62b40a2e40 2024-09-11T22:53:02.369 INFO:teuthology.orchestra.run.smithi152.stdout:Retention added to path / 2024-09-11T22:53:02.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.372+0000 7f62d14d7640 1 -- 172.21.15.152:0/3839441486 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f62b40a2e40 msgr2=0x7f62b40a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:02.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.372+0000 7f62d14d7640 1 --2- 172.21.15.152:0/3839441486 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f62b40a2e40 0x7f62b40a5300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f62bc005d00 tx=0x7f62bc00b410 comp rx=0 tx=0).stop 2024-09-11T22:53:02.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.372+0000 7f62d14d7640 1 -- 172.21.15.152:0/3839441486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62cc102470 msgr2=0x7f62cc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:02.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.372+0000 7f62d14d7640 1 --2- 172.21.15.152:0/3839441486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62cc102470 0x7f62cc074a30 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009230 tx=0x7f62c0004270 comp rx=0 tx=0).stop 2024-09-11T22:53:02.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.372+0000 7f62d14d7640 1 -- 172.21.15.152:0/3839441486 shutdown_connections 2024-09-11T22:53:02.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.372+0000 7f62d14d7640 1 --2- 172.21.15.152:0/3839441486 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f62b40a2e40 0x7f62b40a5300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.372+0000 7f62d14d7640 1 --2- 172.21.15.152:0/3839441486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62cc107b70 0x7f62cc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.372+0000 7f62d14d7640 1 --2- 172.21.15.152:0/3839441486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f62cc102470 0x7f62cc074a30 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.372+0000 7f62d14d7640 1 --2- 172.21.15.152:0/3839441486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62cc101ad0 0x7f62cc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.372+0000 7f62d14d7640 1 -- 172.21.15.152:0/3839441486 >> 172.21.15.152:0/3839441486 conn(0x7f62cc0fd7d0 msgr2=0x7f62cc0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:02.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.372+0000 7f62d14d7640 1 -- 172.21.15.152:0/3839441486 shutdown_connections 2024-09-11T22:53:02.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.372+0000 7f62d14d7640 1 -- 172.21.15.152:0/3839441486 wait complete. 2024-09-11T22:53:02.391 DEBUG:teuthology.orchestra.run.smithi152:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-11T22:53:02.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.537+0000 7fc90c311640 1 -- 172.21.15.152:0/702366007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9041069e0 msgr2=0x7fc904106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:02.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.537+0000 7fc90c311640 1 --2- 172.21.15.152:0/702366007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9041069e0 0x7fc904106dc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc900009930 tx=0x7fc90002a420 comp rx=0 tx=0).stop 2024-09-11T22:53:02.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.537+0000 7fc90c311640 1 -- 172.21.15.152:0/702366007 shutdown_connections 2024-09-11T22:53:02.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.537+0000 7fc90c311640 1 --2- 172.21.15.152:0/702366007 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc904101370 0x7fc90410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.537+0000 7fc90c311640 1 --2- 172.21.15.152:0/702366007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9041009d0 0x7fc904100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.537+0000 7fc90c311640 1 --2- 172.21.15.152:0/702366007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9041069e0 0x7fc904106dc0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.537+0000 7fc90c311640 1 -- 172.21.15.152:0/702366007 >> 172.21.15.152:0/702366007 conn(0x7fc9040fc6f0 msgr2=0x7fc9040feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:02.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.537+0000 7fc90c311640 1 -- 172.21.15.152:0/702366007 shutdown_connections 2024-09-11T22:53:02.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.537+0000 7fc90c311640 1 -- 172.21.15.152:0/702366007 wait complete. 2024-09-11T22:53:02.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.538+0000 7fc90c311640 1 Processor -- start 2024-09-11T22:53:02.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.538+0000 7fc90c311640 1 -- start start 2024-09-11T22:53:02.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.538+0000 7fc90c311640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9041009d0 0x7fc904198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:02.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.538+0000 7fc90c311640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc904101370 0x7fc904198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:02.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.538+0000 7fc90c311640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9041069e0 0x7fc90419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:02.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.538+0000 7fc90c311640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc904110e20 con 0x7fc904101370 2024-09-11T22:53:02.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.538+0000 7fc90c311640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc904110ca0 con 0x7fc9041069e0 2024-09-11T22:53:02.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.538+0000 7fc90c311640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc904110fa0 con 0x7fc9041009d0 2024-09-11T22:53:02.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.538+0000 7fc9098ac640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc904101370 0x7fc904198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:02.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.538+0000 7fc9098ac640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc904101370 0x7fc904198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43716/0 (socket says 172.21.15.152:43716) 2024-09-11T22:53:02.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.538+0000 7fc9098ac640 1 -- 172.21.15.152:0/3712648569 learned_addr learned my addr 172.21.15.152:0/3712648569 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:02.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.539+0000 7fc90a0ad640 1 --2- 172.21.15.152:0/3712648569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9041009d0 0x7fc904198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:02.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.539+0000 7fc90a8ae640 1 --2- 172.21.15.152:0/3712648569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9041069e0 0x7fc90419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:02.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.539+0000 7fc9098ac640 1 -- 172.21.15.152:0/3712648569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9041009d0 msgr2=0x7fc904198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:02.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.539+0000 7fc9098ac640 1 --2- 172.21.15.152:0/3712648569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9041009d0 0x7fc904198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.539+0000 7fc9098ac640 1 -- 172.21.15.152:0/3712648569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9041069e0 msgr2=0x7fc90419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:02.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.539+0000 7fc9098ac640 1 --2- 172.21.15.152:0/3712648569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9041069e0 0x7fc90419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.539+0000 7fc9098ac640 1 -- 172.21.15.152:0/3712648569 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90419d320 con 0x7fc904101370 2024-09-11T22:53:02.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.539+0000 7fc9098ac640 1 --2- 172.21.15.152:0/3712648569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc904101370 0x7fc904198870 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400c970 tx=0x7fc8f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:02.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.539+0000 7fc8fb7fe640 1 -- 172.21.15.152:0/3712648569 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4007bb0 con 0x7fc904101370 2024-09-11T22:53:02.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.539+0000 7fc8fb7fe640 1 -- 172.21.15.152:0/3712648569 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f4007d50 con 0x7fc904101370 2024-09-11T22:53:02.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.539+0000 7fc90c311640 1 -- 172.21.15.152:0/3712648569 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90419d610 con 0x7fc904101370 2024-09-11T22:53:02.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.539+0000 7fc8fb7fe640 1 -- 172.21.15.152:0/3712648569 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f400f450 con 0x7fc904101370 2024-09-11T22:53:02.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.539+0000 7fc90c311640 1 -- 172.21.15.152:0/3712648569 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9041a4ea0 con 0x7fc904101370 2024-09-11T22:53:02.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.541+0000 7fc90c311640 1 -- 172.21.15.152:0/3712648569 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc904100e30 con 0x7fc904101370 2024-09-11T22:53:02.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.541+0000 7fc8fb7fe640 1 -- 172.21.15.152:0/3712648569 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7fc8f4005020 con 0x7fc904101370 2024-09-11T22:53:02.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.542+0000 7fc8fb7fe640 1 --2- 172.21.15.152:0/3712648569 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8f00a2e40 0x7fc8f00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:02.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.542+0000 7fc8fb7fe640 1 -- 172.21.15.152:0/3712648569 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc8f40cf2b0 con 0x7fc904101370 2024-09-11T22:53:02.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.542+0000 7fc90a0ad640 1 --2- 172.21.15.152:0/3712648569 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8f00a2e40 0x7fc8f00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:02.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.543+0000 7fc90a0ad640 1 --2- 172.21.15.152:0/3712648569 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8f00a2e40 0x7fc8f00a5300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc900009900 tx=0x7fc9000058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:02.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.549+0000 7fc8fb7fe640 1 -- 172.21.15.152:0/3712648569 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8f4014030 con 0x7fc904101370 2024-09-11T22:53:02.559 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:02.743 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.743+0000 7fc90c311640 1 -- 172.21.15.152:0/3712648569 --> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fc904109fb0 con 0x7fc8f00a2e40 2024-09-11T22:53:02.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.750+0000 7fc8fb7fe640 1 -- 172.21.15.152:0/3712648569 <== mgr.15240 v2:172.21.15.152:6800/4130474708 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fc904109fb0 con 0x7fc8f00a2e40 2024-09-11T22:53:02.751 INFO:teuthology.orchestra.run.smithi152.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-11T00:00:00", "created": "2024-09-11T22:53:01", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-11T22:53:02.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.752+0000 7fc90c311640 1 -- 172.21.15.152:0/3712648569 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8f00a2e40 msgr2=0x7fc8f00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:02.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.752+0000 7fc90c311640 1 --2- 172.21.15.152:0/3712648569 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8f00a2e40 0x7fc8f00a5300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc900009900 tx=0x7fc9000058d0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.752+0000 7fc90c311640 1 -- 172.21.15.152:0/3712648569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc904101370 msgr2=0x7fc904198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:02.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.752+0000 7fc90c311640 1 --2- 172.21.15.152:0/3712648569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc904101370 0x7fc904198870 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400c970 tx=0x7fc8f400ce30 comp rx=0 tx=0).stop 2024-09-11T22:53:02.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.752+0000 7fc90c311640 1 -- 172.21.15.152:0/3712648569 shutdown_connections 2024-09-11T22:53:02.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.752+0000 7fc90c311640 1 --2- 172.21.15.152:0/3712648569 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8f00a2e40 0x7fc8f00a5300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.752+0000 7fc90c311640 1 --2- 172.21.15.152:0/3712648569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9041069e0 0x7fc90419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.752+0000 7fc90c311640 1 --2- 172.21.15.152:0/3712648569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc904101370 0x7fc904198870 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.752+0000 7fc90c311640 1 --2- 172.21.15.152:0/3712648569 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc9041009d0 0x7fc904198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.752+0000 7fc90c311640 1 -- 172.21.15.152:0/3712648569 >> 172.21.15.152:0/3712648569 conn(0x7fc9040fc6f0 msgr2=0x7fc90406a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:02.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.752+0000 7fc90c311640 1 -- 172.21.15.152:0/3712648569 shutdown_connections 2024-09-11T22:53:02.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.752+0000 7fc90c311640 1 -- 172.21.15.152:0/3712648569 wait complete. 2024-09-11T22:53:02.764 DEBUG:teuthology.orchestra.run.smithi152:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-11T22:53:02.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.869+0000 7f4c46b78640 1 -- 172.21.15.152:0/4109012266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c401009b0 msgr2=0x7f4c40100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:02.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.869+0000 7f4c46b78640 1 --2- 172.21.15.152:0/4109012266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c401009b0 0x7f4c40100e10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009930 tx=0x7f4c2c02a420 comp rx=0 tx=0).stop 2024-09-11T22:53:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.869+0000 7f4c46b78640 1 -- 172.21.15.152:0/4109012266 shutdown_connections 2024-09-11T22:53:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.869+0000 7f4c46b78640 1 --2- 172.21.15.152:0/4109012266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c40101350 0x7f4c4010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.869+0000 7f4c46b78640 1 --2- 172.21.15.152:0/4109012266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c401009b0 0x7f4c40100e10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.869+0000 7f4c46b78640 1 --2- 172.21.15.152:0/4109012266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c401069c0 0x7f4c40106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.869+0000 7f4c46b78640 1 -- 172.21.15.152:0/4109012266 >> 172.21.15.152:0/4109012266 conn(0x7f4c40077990 msgr2=0x7f4c400fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.869+0000 7f4c46b78640 1 -- 172.21.15.152:0/4109012266 shutdown_connections 2024-09-11T22:53:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.869+0000 7f4c46b78640 1 -- 172.21.15.152:0/4109012266 wait complete. 2024-09-11T22:53:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.870+0000 7f4c46b78640 1 Processor -- start 2024-09-11T22:53:02.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.870+0000 7f4c46b78640 1 -- start start 2024-09-11T22:53:02.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.870+0000 7f4c46b78640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c401009b0 0x7f4c40199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:02.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.870+0000 7f4c46b78640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c40101350 0x7f4c40199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:02.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.870+0000 7f4c46b78640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c401069c0 0x7f4c401a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:02.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c46b78640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4c40110e00 con 0x7f4c401009b0 2024-09-11T22:53:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c46b78640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4c40110c80 con 0x7f4c40101350 2024-09-11T22:53:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c46b78640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4c40110f80 con 0x7f4c401069c0 2024-09-11T22:53:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c44914640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c401009b0 0x7f4c40199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c44914640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c401009b0 0x7f4c40199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43748/0 (socket says 172.21.15.152:43748) 2024-09-11T22:53:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c44914640 1 -- 172.21.15.152:0/1236519088 learned_addr learned my addr 172.21.15.152:0/1236519088 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c45115640 1 --2- 172.21.15.152:0/1236519088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c401069c0 0x7f4c401a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c3ffff640 1 --2- 172.21.15.152:0/1236519088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c40101350 0x7f4c40199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c44914640 1 -- 172.21.15.152:0/1236519088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c401069c0 msgr2=0x7f4c401a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c44914640 1 --2- 172.21.15.152:0/1236519088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c401069c0 0x7f4c401a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c44914640 1 -- 172.21.15.152:0/1236519088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c40101350 msgr2=0x7f4c40199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c44914640 1 --2- 172.21.15.152:0/1236519088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c40101350 0x7f4c40199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c44914640 1 -- 172.21.15.152:0/1236519088 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c401a47c0 con 0x7f4c401009b0 2024-09-11T22:53:02.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c44914640 1 --2- 172.21.15.152:0/1236519088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c401009b0 0x7f4c40199200 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400d950 tx=0x7f4c3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:02.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c3dffb640 1 -- 172.21.15.152:0/1236519088 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34017070 con 0x7f4c401009b0 2024-09-11T22:53:02.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c3dffb640 1 -- 172.21.15.152:0/1236519088 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c34009c30 con 0x7f4c401009b0 2024-09-11T22:53:02.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c46b78640 1 -- 172.21.15.152:0/1236519088 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c401a4990 con 0x7f4c401009b0 2024-09-11T22:53:02.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.871+0000 7f4c3dffb640 1 -- 172.21.15.152:0/1236519088 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3400f6b0 con 0x7f4c401009b0 2024-09-11T22:53:02.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.872+0000 7f4c46b78640 1 -- 172.21.15.152:0/1236519088 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c401a4eb0 con 0x7f4c401009b0 2024-09-11T22:53:02.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.873+0000 7f4c46b78640 1 -- 172.21.15.152:0/1236519088 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c40100e10 con 0x7f4c401009b0 2024-09-11T22:53:02.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.874+0000 7f4c3dffb640 1 -- 172.21.15.152:0/1236519088 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f4c340109d0 con 0x7f4c401009b0 2024-09-11T22:53:02.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.875+0000 7f4c3dffb640 1 --2- 172.21.15.152:0/1236519088 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4c300a2e40 0x7f4c300a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:02.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.875+0000 7f4c3ffff640 1 --2- 172.21.15.152:0/1236519088 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4c300a2e40 0x7f4c300a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:02.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.875+0000 7f4c3dffb640 1 -- 172.21.15.152:0/1236519088 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4c340cf0b0 con 0x7f4c401009b0 2024-09-11T22:53:02.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.875+0000 7f4c3ffff640 1 --2- 172.21.15.152:0/1236519088 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4c300a2e40 0x7f4c300a5300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c000c00 tx=0x7f4c2c005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:02.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:02.879+0000 7f4c3dffb640 1 -- 172.21.15.152:0/1236519088 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c34009de0 con 0x7f4c401009b0 2024-09-11T22:53:03.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.060+0000 7fb2c0388640 1 -- 172.21.15.152:0/1043927905 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2b8100e20 msgr2=0x7fb2b8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:03.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.060+0000 7fb2c0388640 1 --2- 172.21.15.152:0/1043927905 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2b8100e20 0x7fb2b8101200 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009900 tx=0x7fb2a802a480 comp rx=0 tx=0).stop 2024-09-11T22:53:03.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.061+0000 7fb2c0388640 1 -- 172.21.15.152:0/1043927905 shutdown_connections 2024-09-11T22:53:03.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.061+0000 7fb2c0388640 1 --2- 172.21.15.152:0/1043927905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2b8102ef0 0x7fb2b810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.061+0000 7fb2c0388640 1 --2- 172.21.15.152:0/1043927905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2b8102570 0x7fb2b81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.061+0000 7fb2c0388640 1 --2- 172.21.15.152:0/1043927905 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2b8100e20 0x7fb2b8101200 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.061+0000 7fb2c0388640 1 -- 172.21.15.152:0/1043927905 >> 172.21.15.152:0/1043927905 conn(0x7fb2b80fc7d0 msgr2=0x7fb2b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:03.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.061+0000 7fb2c0388640 1 -- 172.21.15.152:0/1043927905 shutdown_connections 2024-09-11T22:53:03.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.061+0000 7fb2c0388640 1 -- 172.21.15.152:0/1043927905 wait complete. 2024-09-11T22:53:03.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.061+0000 7fb2c0388640 1 Processor -- start 2024-09-11T22:53:03.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.061+0000 7fb2c0388640 1 -- start start 2024-09-11T22:53:03.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.061+0000 7fb2c0388640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2b8100e20 0x7fb2b8103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.061+0000 7fb2c0388640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2b8102570 0x7fb2b81041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2c0388640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2b8102ef0 0x7fb2b8104710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2c0388640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b8110f80 con 0x7fb2b8102ef0 2024-09-11T22:53:03.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2c0388640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b8110e00 con 0x7fb2b8100e20 2024-09-11T22:53:03.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2c0388640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b8111100 con 0x7fb2b8102570 2024-09-11T22:53:03.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2be925640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2b8102ef0 0x7fb2b8104710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2be925640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2b8102ef0 0x7fb2b8104710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43762/0 (socket says 172.21.15.152:43762) 2024-09-11T22:53:03.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2be925640 1 -- 172.21.15.152:0/441228732 learned_addr learned my addr 172.21.15.152:0/441228732 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:03.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2be124640 1 --2- 172.21.15.152:0/441228732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2b8100e20 0x7fb2b8103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2bd923640 1 --2- 172.21.15.152:0/441228732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2b8102570 0x7fb2b81041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2be925640 1 -- 172.21.15.152:0/441228732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2b8102570 msgr2=0x7fb2b81041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:03.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2be925640 1 --2- 172.21.15.152:0/441228732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2b8102570 0x7fb2b81041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2be925640 1 -- 172.21.15.152:0/441228732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2b8100e20 msgr2=0x7fb2b8103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2be925640 1 --2- 172.21.15.152:0/441228732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2b8100e20 0x7fb2b8103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2be925640 1 -- 172.21.15.152:0/441228732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b81a69b0 con 0x7fb2b8102ef0 2024-09-11T22:53:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2be124640 1 --2- 172.21.15.152:0/441228732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2b8100e20 0x7fb2b8103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2bd923640 1 --2- 172.21.15.152:0/441228732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2b8102570 0x7fb2b81041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2be925640 1 --2- 172.21.15.152:0/441228732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2b8102ef0 0x7fb2b8104710 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009fd0 tx=0x7fb2b000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2af7fe640 1 -- 172.21.15.152:0/441228732 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0017070 con 0x7fb2b8102ef0 2024-09-11T22:53:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2c0388640 1 -- 172.21.15.152:0/441228732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b81a6ca0 con 0x7fb2b8102ef0 2024-09-11T22:53:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2af7fe640 1 -- 172.21.15.152:0/441228732 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b0003e50 con 0x7fb2b8102ef0 2024-09-11T22:53:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.062+0000 7fb2af7fe640 1 -- 172.21.15.152:0/441228732 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b00106c0 con 0x7fb2b8102ef0 2024-09-11T22:53:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.063+0000 7fb2c0388640 1 -- 172.21.15.152:0/441228732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b81a71e0 con 0x7fb2b8102ef0 2024-09-11T22:53:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.065+0000 7fb2af7fe640 1 -- 172.21.15.152:0/441228732 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7fb2b0003970 con 0x7fb2b8102ef0 2024-09-11T22:53:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.065+0000 7fb2af7fe640 1 --2- 172.21.15.152:0/441228732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb2a40a2d80 0x7fb2a40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.065+0000 7fb2be124640 1 --2- 172.21.15.152:0/441228732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb2a40a2d80 0x7fb2a40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.065+0000 7fb2af7fe640 1 -- 172.21.15.152:0/441228732 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb2b00cf390 con 0x7fb2b8102ef0 2024-09-11T22:53:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.066+0000 7fb2be124640 1 --2- 172.21.15.152:0/441228732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb2a40a2d80 0x7fb2a40a5240 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8005ab0 tx=0x7fb2a8005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.066+0000 7fb2c2761740 1 Processor -- start 2024-09-11T22:53:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.066+0000 7fb2c2761740 1 -- start start 2024-09-11T22:53:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2c2761740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537c24a0 0x561b537c2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2c2761740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c2dc0 0x561b537cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2c2761740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537cd680 0x561b537cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2c2761740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x561b537282e0 con 0x561b537c2dc0 2024-09-11T22:53:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2c2761740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561b536cfce0 con 0x561b537cd680 2024-09-11T22:53:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2c2761740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561b537242a0 con 0x561b537c24a0 2024-09-11T22:53:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2bd923640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c2dc0 0x561b537cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2bd923640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c2dc0 0x561b537cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43766/0 (socket says 172.21.15.152:43766) 2024-09-11T22:53:03.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2bd923640 1 -- 172.21.15.152:0/3732025538 learned_addr learned my addr 172.21.15.152:0/3732025538 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2be124640 1 --2- 172.21.15.152:0/3732025538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537c24a0 0x561b537c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2be925640 1 --2- 172.21.15.152:0/3732025538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537cd680 0x561b537cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2bd923640 1 -- 172.21.15.152:0/3732025538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537c24a0 msgr2=0x561b537c2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2bd923640 1 --2- 172.21.15.152:0/3732025538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537c24a0 0x561b537c2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2bd923640 1 -- 172.21.15.152:0/3732025538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537cd680 msgr2=0x561b537cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2bd923640 1 --2- 172.21.15.152:0/3732025538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537cd680 0x561b537cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2bd923640 1 -- 172.21.15.152:0/3732025538 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b536b4440 con 0x561b537c2dc0 2024-09-11T22:53:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2bd923640 1 --2- 172.21.15.152:0/3732025538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c2dc0 0x561b537cd140 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400bcb0 tx=0x7fb2b400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393f9b5598e0653c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.067+0000 7fb2acdea640 1 -- 172.21.15.152:0/3732025538 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4011440 con 0x561b537c2dc0 2024-09-11T22:53:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.068+0000 7fb2acdea640 1 -- 172.21.15.152:0/3732025538 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b4011ac0 con 0x561b537c2dc0 2024-09-11T22:53:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.068+0000 7fb2acdea640 1 -- 172.21.15.152:0/3732025538 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b401baf0 con 0x561b537c2dc0 2024-09-11T22:53:03.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.068+0000 7fb2c2761740 1 -- 172.21.15.152:0/3732025538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c2dc0 msgr2=0x561b537cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.068+0000 7fb2c2761740 1 --2- 172.21.15.152:0/3732025538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c2dc0 0x561b537cd140 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400bcb0 tx=0x7fb2b400f730 comp rx=0 tx=0).stop 2024-09-11T22:53:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.069+0000 7fb2c2761740 1 -- 172.21.15.152:0/3732025538 shutdown_connections 2024-09-11T22:53:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.069+0000 7fb2c2761740 1 --2- 172.21.15.152:0/3732025538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537cd680 0x561b537cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.069+0000 7fb2c2761740 1 --2- 172.21.15.152:0/3732025538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c2dc0 0x561b537cd140 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.069+0000 7fb2c2761740 1 --2- 172.21.15.152:0/3732025538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537c24a0 0x561b537c2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.069+0000 7fb2c2761740 1 -- 172.21.15.152:0/3732025538 >> 172.21.15.152:0/3732025538 conn(0x561b537bea30 msgr2=0x561b537bee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.069+0000 7fb2c2761740 1 -- 172.21.15.152:0/3732025538 shutdown_connections 2024-09-11T22:53:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.069+0000 7fb2c2761740 1 -- 172.21.15.152:0/3732025538 wait complete. 2024-09-11T22:53:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.069+0000 7fb2c2761740 1 -- start start 2024-09-11T22:53:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.069+0000 7fb2c2761740 1 Processor -- start 2024-09-11T22:53:03.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2c2761740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c24a0 0x561b537ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2c2761740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537c2dc0 0x561b537ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2c2761740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 0x561b537ca750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2c2761740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x561b537d0250 con 0x561b537c24a0 2024-09-11T22:53:03.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2c2761740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561b536e3050 con 0x561b537c2dc0 2024-09-11T22:53:03.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2c2761740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561b537d03d0 con 0x561b537cd680 2024-09-11T22:53:03.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2c2761740 10 client.0 _ll_register_callbacks cb 0x561b53723160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:03.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2be124640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c24a0 0x561b537ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2be124640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c24a0 0x561b537ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43778/0 (socket says 172.21.15.152:43778) 2024-09-11T22:53:03.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2be124640 1 -- 172.21.15.152:0/2176084605 learned_addr learned my addr 172.21.15.152:0/2176084605 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:03.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2bd923640 1 --2- 172.21.15.152:0/2176084605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537c2dc0 0x561b537ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2be925640 1 --2- 172.21.15.152:0/2176084605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 0x561b537ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2be124640 1 -- 172.21.15.152:0/2176084605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 msgr2=0x561b537ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:03.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2be124640 1 --2- 172.21.15.152:0/2176084605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 0x561b537ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2be124640 1 -- 172.21.15.152:0/2176084605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537c2dc0 msgr2=0x561b537ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2be124640 1 --2- 172.21.15.152:0/2176084605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537c2dc0 0x561b537ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2be124640 1 -- 172.21.15.152:0/2176084605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b534993c0 con 0x561b537c24a0 2024-09-11T22:53:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2be925640 1 --2- 172.21.15.152:0/2176084605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 0x561b537ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2be124640 1 --2- 172.21.15.152:0/2176084605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c24a0 0x561b537ccf60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8033000 tx=0x7fb2a8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a886af75a2ce518 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2a1ffb640 1 -- 172.21.15.152:0/2176084605 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8035050 con 0x561b537c24a0 2024-09-11T22:53:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2a1ffb640 1 -- 172.21.15.152:0/2176084605 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb2a8032750 con 0x561b537c24a0 2024-09-11T22:53:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.070+0000 7fb2c2761740 1 -- 172.21.15.152:0/2176084605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2b4011ac0 con 0x561b537c24a0 2024-09-11T22:53:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.071+0000 7fb2a1ffb640 1 -- 172.21.15.152:0/2176084605 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2a8004e50 con 0x561b537c24a0 2024-09-11T22:53:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.071+0000 7fb2c2761740 10 client.15426 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.071+0000 7fb2c2761740 1 -- 172.21.15.152:0/2176084605 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b537dbbc0 con 0x561b537c24a0 2024-09-11T22:53:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.071+0000 7fb2a1ffb640 1 -- 172.21.15.152:0/2176084605 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b4011ac0 con 0x561b537c24a0 2024-09-11T22:53:03.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.071+0000 7fb2a1ffb640 1 -- 172.21.15.152:0/2176084605 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb2a8003b40 con 0x561b537c24a0 2024-09-11T22:53:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.071+0000 7fb2c2761740 10 client.15426 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.071+0000 7fb2c2761740 10 client.15426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.071+0000 7fb2c2761740 20 client.15426 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.071+0000 7fb2c2761740 1 --2- 172.21.15.152:0/2176084605 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x561b537dcfd0 0x561b537df400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.071+0000 7fb2c2761740 4 client.15426 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.071+0000 7fb2c2761740 1 -- 172.21.15.152:0/2176084605 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b53735850 con 0x561b537dcfd0 2024-09-11T22:53:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.071+0000 7fb2bd923640 1 --2- 172.21.15.152:0/2176084605 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x561b537dcfd0 0x561b537df400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.072+0000 7fb2a1ffb640 10 client.15426 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:03.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.072+0000 7fb2bd923640 1 --2- 172.21.15.152:0/2176084605 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x561b537dcfd0 0x561b537df400 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a88ff5fa64a09c0 server_cookie=36efebd3a70862ef in_seq=0 out_seq=0 2024-09-11T22:53:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.072+0000 7fb2a1ffb640 1 -- 172.21.15.152:0/2176084605 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b53735850 con 0x561b537dcfd0 2024-09-11T22:53:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.072+0000 7fb2a1ffb640 10 client.15426 handle_command_reply: tid=0 2024-09-11T22:53:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.072+0000 7fb2c2761740 1 client.15426 shutdown 2024-09-11T22:53:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.072+0000 7fb2c2761740 1 -- 172.21.15.152:0/2176084605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c24a0 msgr2=0x561b537ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.072+0000 7fb2c2761740 1 --2- 172.21.15.152:0/2176084605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c24a0 0x561b537ccf60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8033000 tx=0x7fb2a8009260 comp rx=0 tx=0).stop 2024-09-11T22:53:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.073+0000 7fb2c2761740 1 -- 172.21.15.152:0/2176084605 shutdown_connections 2024-09-11T22:53:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.073+0000 7fb2c2761740 1 --2- 172.21.15.152:0/2176084605 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 0x561b537ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.073+0000 7fb2c2761740 1 --2- 172.21.15.152:0/2176084605 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x561b537dcfd0 0x561b537df400 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.073+0000 7fb2c2761740 1 --2- 172.21.15.152:0/2176084605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537c2dc0 0x561b537ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.073+0000 7fb2c2761740 1 --2- 172.21.15.152:0/2176084605 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c24a0 0x561b537ccf60 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.073+0000 7fb2c2761740 1 -- 172.21.15.152:0/2176084605 >> 172.21.15.152:0/2176084605 conn(0x561b537bea30 msgr2=0x561b537c4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.073+0000 7fb2a1ffb640 0 client.15426 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.073+0000 7fb2c2761740 1 -- 172.21.15.152:0/2176084605 shutdown_connections 2024-09-11T22:53:03.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.073+0000 7fb2c2761740 1 -- 172.21.15.152:0/2176084605 wait complete. 2024-09-11T22:53:03.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.073+0000 7fb2c2761740 20 client.15426 trim_cache size 0 max 16384 2024-09-11T22:53:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.090+0000 7fb2c2761740 1 Processor -- start 2024-09-11T22:53:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.090+0000 7fb2c2761740 1 -- start start 2024-09-11T22:53:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.090+0000 7fb2c2761740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c24a0 0x561b537d94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.090+0000 7fb2c2761740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537c2dc0 0x561b537e4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.090+0000 7fb2c2761740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 0x561b537e51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.090+0000 7fb2c2761740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81afa70 con 0x561b537c24a0 2024-09-11T22:53:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2c2761740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81b15b0 con 0x561b537c2dc0 2024-09-11T22:53:03.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2c2761740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81b0140 con 0x561b537cd680 2024-09-11T22:53:03.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2be124640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c24a0 0x561b537d94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2be124640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c24a0 0x561b537d94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43794/0 (socket says 172.21.15.152:43794) 2024-09-11T22:53:03.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2be124640 1 -- 172.21.15.152:0/1135210098 learned_addr learned my addr 172.21.15.152:0/1135210098 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:03.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2bd923640 1 --2- 172.21.15.152:0/1135210098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537c2dc0 0x561b537e4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2be925640 1 --2- 172.21.15.152:0/1135210098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 0x561b537e51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2be124640 1 -- 172.21.15.152:0/1135210098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 msgr2=0x561b537e51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:03.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2be124640 1 --2- 172.21.15.152:0/1135210098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 0x561b537e51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2be124640 1 -- 172.21.15.152:0/1135210098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537c2dc0 msgr2=0x561b537e4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:03.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2be124640 1 --2- 172.21.15.152:0/1135210098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537c2dc0 0x561b537e4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2be124640 1 -- 172.21.15.152:0/1135210098 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b537c91c0 con 0x561b537c24a0 2024-09-11T22:53:03.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2bd923640 1 --2- 172.21.15.152:0/1135210098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537c2dc0 0x561b537e4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:03.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2be925640 1 --2- 172.21.15.152:0/1135210098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 0x561b537e51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:03.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2be124640 1 --2- 172.21.15.152:0/1135210098 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c24a0 0x561b537d94b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8000c00 tx=0x7fb2a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6df1bfc4815a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:03.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2a3fff640 1 -- 172.21.15.152:0/1135210098 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8068070 con 0x561b537c24a0 2024-09-11T22:53:03.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2a3fff640 1 -- 172.21.15.152:0/1135210098 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2a80044b0 con 0x561b537c24a0 2024-09-11T22:53:03.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.091+0000 7fb2a3fff640 1 -- 172.21.15.152:0/1135210098 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8032c00 con 0x561b537c24a0 2024-09-11T22:53:03.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.092+0000 7fb2c2761740 1 -- 172.21.15.152:0/1135210098 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c24a0 msgr2=0x561b537d94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:03.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.092+0000 7fb2c2761740 1 --2- 172.21.15.152:0/1135210098 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c24a0 0x561b537d94b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8000c00 tx=0x7fb2a8004270 comp rx=0 tx=0).stop 2024-09-11T22:53:03.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.092+0000 7fb2c2761740 1 -- 172.21.15.152:0/1135210098 shutdown_connections 2024-09-11T22:53:03.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.092+0000 7fb2c2761740 1 --2- 172.21.15.152:0/1135210098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 0x561b537e51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.092+0000 7fb2c2761740 1 --2- 172.21.15.152:0/1135210098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537c2dc0 0x561b537e4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.092+0000 7fb2c2761740 1 --2- 172.21.15.152:0/1135210098 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c24a0 0x561b537d94b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.092+0000 7fb2c2761740 1 -- 172.21.15.152:0/1135210098 >> 172.21.15.152:0/1135210098 conn(0x561b537bea30 msgr2=0x561b537c50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:03.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.092+0000 7fb2c2761740 1 -- 172.21.15.152:0/1135210098 shutdown_connections 2024-09-11T22:53:03.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.092+0000 7fb2c2761740 1 -- 172.21.15.152:0/1135210098 wait complete. 2024-09-11T22:53:03.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2c2761740 1 -- start start 2024-09-11T22:53:03.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2c2761740 1 Processor -- start 2024-09-11T22:53:03.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2c2761740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537c24a0 0x561b537d5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2c2761740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c2dc0 0x561b537d1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2c2761740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 0x561b537d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2c2761740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81b0ee0 con 0x561b537c2dc0 2024-09-11T22:53:03.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2c2761740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81b0810 con 0x561b537c24a0 2024-09-11T22:53:03.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2c2761740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b81ada30 con 0x561b537cd680 2024-09-11T22:53:03.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2c2761740 10 client.0 _ll_register_callbacks cb 0x7fb2a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:03.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2bd923640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c2dc0 0x561b537d1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2bd923640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c2dc0 0x561b537d1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43802/0 (socket says 172.21.15.152:43802) 2024-09-11T22:53:03.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2bd923640 1 -- 172.21.15.152:0/2796650835 learned_addr learned my addr 172.21.15.152:0/2796650835 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:03.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2be124640 1 --2- 172.21.15.152:0/2796650835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537c24a0 0x561b537d5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2be925640 1 --2- 172.21.15.152:0/2796650835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 0x561b537d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2bd923640 1 -- 172.21.15.152:0/2796650835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 msgr2=0x561b537d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:03.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2bd923640 1 --2- 172.21.15.152:0/2796650835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 0x561b537d1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2bd923640 1 -- 172.21.15.152:0/2796650835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537c24a0 msgr2=0x561b537d5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:03.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2bd923640 1 --2- 172.21.15.152:0/2796650835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537c24a0 0x561b537d5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.093+0000 7fb2bd923640 1 -- 172.21.15.152:0/2796650835 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b537d1f80 con 0x561b537c2dc0 2024-09-11T22:53:03.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.094+0000 7fb2bd923640 1 --2- 172.21.15.152:0/2796650835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c2dc0 0x561b537d1190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400bcb0 tx=0x7fb2b401b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60de4c9341730719 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:03.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.094+0000 7fb2a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:03.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.094+0000 7fb2a1ffb640 1 -- 172.21.15.152:0/2796650835 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4034060 con 0x561b537c2dc0 2024-09-11T22:53:03.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.094+0000 7fb2a1ffb640 1 -- 172.21.15.152:0/2796650835 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb2b40050b0 con 0x561b537c2dc0 2024-09-11T22:53:03.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.094+0000 7fb2c2761740 1 -- 172.21.15.152:0/2796650835 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2a80044b0 con 0x561b537c2dc0 2024-09-11T22:53:03.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.094+0000 7fb2a1ffb640 1 -- 172.21.15.152:0/2796650835 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb2b40052b0 con 0x561b537c2dc0 2024-09-11T22:53:03.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.094+0000 7fb2c2761740 10 client.15432 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:03.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.094+0000 7fb2c2761740 1 -- 172.21.15.152:0/2796650835 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b53847d60 con 0x561b537c2dc0 2024-09-11T22:53:03.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.094+0000 7fb2a1ffb640 1 -- 172.21.15.152:0/2796650835 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2a80044b0 con 0x561b537c2dc0 2024-09-11T22:53:03.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.094+0000 7fb2a1ffb640 1 -- 172.21.15.152:0/2796650835 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb2b4003680 con 0x561b537c2dc0 2024-09-11T22:53:03.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.094+0000 7fb2c2761740 10 client.15432 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:03.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.094+0000 7fb2c2761740 10 client.15432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:03.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.094+0000 7fb2c2761740 20 client.15432 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:03.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.094+0000 7fb2c2761740 1 --2- 172.21.15.152:0/2796650835 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x561b53848250 0x561b5384a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:03.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.094+0000 7fb2c2761740 4 client.15432 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:53:03.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.094+0000 7fb2c2761740 1 -- 172.21.15.152:0/2796650835 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561b5372fd60 con 0x561b53848250 2024-09-11T22:53:03.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.095+0000 7fb2be124640 1 --2- 172.21.15.152:0/2796650835 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x561b53848250 0x561b5384a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:03.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.095+0000 7fb2be124640 1 --2- 172.21.15.152:0/2796650835 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x561b53848250 0x561b5384a640 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=500a0961c5b53507 server_cookie=6297c99cac0d0a1d in_seq=0 out_seq=0 2024-09-11T22:53:03.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.095+0000 7fb2a1ffb640 10 client.15432 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:03.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.095+0000 7fb2a1ffb640 1 -- 172.21.15.152:0/2796650835 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561b5372fd60 con 0x561b53848250 2024-09-11T22:53:03.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.095+0000 7fb2a1ffb640 10 client.15432 handle_command_reply: tid=0 2024-09-11T22:53:03.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.095+0000 7fb2c2761740 1 client.15432 shutdown 2024-09-11T22:53:03.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.095+0000 7fb2c2761740 1 -- 172.21.15.152:0/2796650835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c2dc0 msgr2=0x561b537d1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:03.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.095+0000 7fb2c2761740 1 --2- 172.21.15.152:0/2796650835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c2dc0 0x561b537d1190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400bcb0 tx=0x7fb2b401b630 comp rx=0 tx=0).stop 2024-09-11T22:53:03.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c2761740 1 -- 172.21.15.152:0/2796650835 shutdown_connections 2024-09-11T22:53:03.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c2761740 1 --2- 172.21.15.152:0/2796650835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b537cd680 0x561b537d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c2761740 1 --2- 172.21.15.152:0/2796650835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b537c2dc0 0x561b537d1190 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c2761740 1 --2- 172.21.15.152:0/2796650835 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x561b53848250 0x561b5384a640 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c2761740 1 --2- 172.21.15.152:0/2796650835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b537c24a0 0x561b537d5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c2761740 1 -- 172.21.15.152:0/2796650835 >> 172.21.15.152:0/2796650835 conn(0x561b537bea30 msgr2=0x561b537bd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:03.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2a1ffb640 0 client.15432 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:03.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c2761740 1 -- 172.21.15.152:0/2796650835 shutdown_connections 2024-09-11T22:53:03.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c2761740 1 -- 172.21.15.152:0/2796650835 wait complete. 2024-09-11T22:53:03.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c2761740 20 client.15432 trim_cache size 0 max 16384 2024-09-11T22:53:03.109 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:53:03.109 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:53:03.109 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": { 2024-09-11T22:53:03.110 INFO:teuthology.orchestra.run.smithi152.stdout: "b595479b-b04b-4bf7-afa4-3414b25b7cdb": { 2024-09-11T22:53:03.110 INFO:teuthology.orchestra.run.smithi152.stdout: "path": "/", 2024-09-11T22:53:03.110 INFO:teuthology.orchestra.run.smithi152.stdout: "tag": "b595479b-b04b-4bf7-afa4-3414b25b7cdb", 2024-09-11T22:53:03.110 INFO:teuthology.orchestra.run.smithi152.stdout: "options": "recursive,force" 2024-09-11T22:53:03.110 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:53:03.110 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:53:03.110 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:53:03.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c0388640 1 -- 172.21.15.152:0/441228732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb2a40a2d80 msgr2=0x7fb2a40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:03.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c0388640 1 --2- 172.21.15.152:0/441228732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb2a40a2d80 0x7fb2a40a5240 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8005ab0 tx=0x7fb2a8005a20 comp rx=0 tx=0).stop 2024-09-11T22:53:03.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c0388640 1 -- 172.21.15.152:0/441228732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2b8102ef0 msgr2=0x7fb2b8104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:03.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c0388640 1 --2- 172.21.15.152:0/441228732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2b8102ef0 0x7fb2b8104710 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009fd0 tx=0x7fb2b000b260 comp rx=0 tx=0).stop 2024-09-11T22:53:03.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c0388640 1 -- 172.21.15.152:0/441228732 shutdown_connections 2024-09-11T22:53:03.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c0388640 1 --2- 172.21.15.152:0/441228732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb2a40a2d80 0x7fb2a40a5240 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c0388640 1 --2- 172.21.15.152:0/441228732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb2b8102ef0 0x7fb2b8104710 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c0388640 1 --2- 172.21.15.152:0/441228732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2b8102570 0x7fb2b81041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c0388640 1 --2- 172.21.15.152:0/441228732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb2b8100e20 0x7fb2b8103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.096+0000 7fb2c0388640 1 -- 172.21.15.152:0/441228732 >> 172.21.15.152:0/441228732 conn(0x7fb2b80fc7d0 msgr2=0x7fb2b8105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:03.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.097+0000 7fb2c0388640 1 -- 172.21.15.152:0/441228732 shutdown_connections 2024-09-11T22:53:03.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.097+0000 7fb2c0388640 1 -- 172.21.15.152:0/441228732 wait complete. 2024-09-11T22:53:03.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.099+0000 7f4c46b78640 1 -- 172.21.15.152:0/1236519088 --> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f4c40108f90 con 0x7f4c300a2e40 2024-09-11T22:53:03.112 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.106+0000 7f4c3dffb640 1 -- 172.21.15.152:0/1236519088 <== mgr.15240 v2:172.21.15.152:6800/4130474708 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f4c2c032db0 con 0x7f4c300a2e40 2024-09-11T22:53:03.112 INFO:teuthology.orchestra.run.smithi152.stdout:/ 1m 6m3h 2024-09-11T22:53:03.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.108+0000 7f4c46b78640 1 -- 172.21.15.152:0/1236519088 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4c300a2e40 msgr2=0x7f4c300a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:03.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.108+0000 7f4c46b78640 1 --2- 172.21.15.152:0/1236519088 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4c300a2e40 0x7f4c300a5300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c000c00 tx=0x7f4c2c005a20 comp rx=0 tx=0).stop 2024-09-11T22:53:03.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.108+0000 7f4c46b78640 1 -- 172.21.15.152:0/1236519088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c401009b0 msgr2=0x7f4c40199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:03.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.108+0000 7f4c46b78640 1 --2- 172.21.15.152:0/1236519088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c401009b0 0x7f4c40199200 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400d950 tx=0x7f4c3400de10 comp rx=0 tx=0).stop 2024-09-11T22:53:03.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.108+0000 7f4c46b78640 1 -- 172.21.15.152:0/1236519088 shutdown_connections 2024-09-11T22:53:03.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.108+0000 7f4c46b78640 1 --2- 172.21.15.152:0/1236519088 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4c300a2e40 0x7f4c300a5300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.108+0000 7f4c46b78640 1 --2- 172.21.15.152:0/1236519088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c401069c0 0x7f4c401a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.108+0000 7f4c46b78640 1 --2- 172.21.15.152:0/1236519088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c40101350 0x7f4c40199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.108+0000 7f4c46b78640 1 --2- 172.21.15.152:0/1236519088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c401009b0 0x7f4c40199200 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:03.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.109+0000 7f4c46b78640 1 -- 172.21.15.152:0/1236519088 >> 172.21.15.152:0/1236519088 conn(0x7f4c40077990 msgr2=0x7f4c40103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:03.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.109+0000 7f4c46b78640 1 -- 172.21.15.152:0/1236519088 shutdown_connections 2024-09-11T22:53:03.114 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:03.109+0000 7f4c46b78640 1 -- 172.21.15.152:0/1236519088 wait complete. 2024-09-11T22:53:03.123 DEBUG:teuthology.orchestra.run.smithi152:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-11T22:53:03.152 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-11T22:53:03.166 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-11T22:53:03.178 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi152.front.sepia.ceph.com 2024-09-11T22:53:03.178 DEBUG:teuthology.orchestra.run.smithi152:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-11T22:53:03.195 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-11T22:53:03.210 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T22:53:03.210 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:53:03.232 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:02 smithi152 ceph-mon[31181]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 293 B/s wr, 25 op/s 2024-09-11T22:53:03.232 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:02 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T22:53:03.232 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:02 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:53:03.233 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:02 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T22:53:03.233 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:02 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:53:03.233 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:02 smithi152 ceph-mon[31181]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:53:03.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:02 smithi191 ceph-mon[35211]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 293 B/s wr, 25 op/s 2024-09-11T22:53:03.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:02 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T22:53:03.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:02 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:53:03.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:02 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T22:53:03.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:02 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:53:03.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:02 smithi191 ceph-mon[35211]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:53:03.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:02 smithi204 ceph-mon[35454]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 293 B/s wr, 25 op/s 2024-09-11T22:53:03.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:02 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T22:53:03.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:02 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:53:03.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:02 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T22:53:03.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:02 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:53:03.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:02 smithi204 ceph-mon[35454]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:53:03.468 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:03.911 INFO:tasks.cephfs.filesystem:scrub status for tag:b595479b-b04b-4bf7-afa4-3414b25b7cdb - {'path': '/', 'tag': 'b595479b-b04b-4bf7-afa4-3414b25b7cdb', 'options': 'recursive,force'} 2024-09-11T22:53:04.127 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:03 smithi152 ceph-mon[31181]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:53:04.127 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:03 smithi152 ceph-mon[31181]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:53:04.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:03 smithi191 ceph-mon[35211]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:53:04.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:03 smithi191 ceph-mon[35211]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:53:04.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:03 smithi204 ceph-mon[35454]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:53:04.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:03 smithi204 ceph-mon[35454]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:53:04.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.347+0000 7f499799c640 1 -- 172.21.15.152:0/1277518450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49901082a0 msgr2=0x7f499010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:04.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.347+0000 7f499799c640 1 --2- 172.21.15.152:0/1277518450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49901082a0 0x7f499010ec40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4988009cd0 tx=0x7f498802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:53:04.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.348+0000 7f499799c640 1 -- 172.21.15.152:0/1277518450 shutdown_connections 2024-09-11T22:53:04.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.348+0000 7f499799c640 1 --2- 172.21.15.152:0/1277518450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49901082a0 0x7f499010ec40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:04.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.348+0000 7f499799c640 1 --2- 172.21.15.152:0/1277518450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49901036e0 0x7f4990107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:04.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.348+0000 7f499799c640 1 --2- 172.21.15.152:0/1277518450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4990102d30 0x7f4990103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:04.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.348+0000 7f499799c640 1 -- 172.21.15.152:0/1277518450 >> 172.21.15.152:0/1277518450 conn(0x7f49900fc7d0 msgr2=0x7f49900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:04.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.348+0000 7f499799c640 1 -- 172.21.15.152:0/1277518450 shutdown_connections 2024-09-11T22:53:04.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.348+0000 7f499799c640 1 -- 172.21.15.152:0/1277518450 wait complete. 2024-09-11T22:53:04.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.348+0000 7f499799c640 1 Processor -- start 2024-09-11T22:53:04.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.348+0000 7f499799c640 1 -- start start 2024-09-11T22:53:04.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f499799c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4990102d30 0x7f4990198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:04.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f499799c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49901036e0 0x7f4990198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:04.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f499799c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49901082a0 0x7f499019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:04.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f499799c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f49901120a0 con 0x7f4990102d30 2024-09-11T22:53:04.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f499799c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4990111f20 con 0x7f49901036e0 2024-09-11T22:53:04.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f499799c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4990112220 con 0x7f49901082a0 2024-09-11T22:53:04.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f4995738640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4990102d30 0x7f4990198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:04.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f4995738640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4990102d30 0x7f4990198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43820/0 (socket says 172.21.15.152:43820) 2024-09-11T22:53:04.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f4995738640 1 -- 172.21.15.152:0/1034026143 learned_addr learned my addr 172.21.15.152:0/1034026143 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:04.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f4994f37640 1 --2- 172.21.15.152:0/1034026143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49901036e0 0x7f4990198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:04.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f4995f39640 1 --2- 172.21.15.152:0/1034026143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49901082a0 0x7f499019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:04.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f4995738640 1 -- 172.21.15.152:0/1034026143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49901082a0 msgr2=0x7f499019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:04.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f4995738640 1 --2- 172.21.15.152:0/1034026143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49901082a0 0x7f499019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:04.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f4995738640 1 -- 172.21.15.152:0/1034026143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49901036e0 msgr2=0x7f4990198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:04.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f4995738640 1 --2- 172.21.15.152:0/1034026143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49901036e0 0x7f4990198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:04.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f4995738640 1 -- 172.21.15.152:0/1034026143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499019d520 con 0x7f4990102d30 2024-09-11T22:53:04.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f4995738640 1 --2- 172.21.15.152:0/1034026143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4990102d30 0x7f4990198570 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f498c00c970 tx=0x7f498c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:04.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f49867fc640 1 -- 172.21.15.152:0/1034026143 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c007bb0 con 0x7f4990102d30 2024-09-11T22:53:04.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.349+0000 7f49867fc640 1 -- 172.21.15.152:0/1034026143 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f498c007d50 con 0x7f4990102d30 2024-09-11T22:53:04.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.350+0000 7f49867fc640 1 -- 172.21.15.152:0/1034026143 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c00f450 con 0x7f4990102d30 2024-09-11T22:53:04.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.350+0000 7f499799c640 1 -- 172.21.15.152:0/1034026143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499019d810 con 0x7f4990102d30 2024-09-11T22:53:04.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.350+0000 7f499799c640 1 -- 172.21.15.152:0/1034026143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49901a50f0 con 0x7f4990102d30 2024-09-11T22:53:04.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.351+0000 7f499799c640 1 -- 172.21.15.152:0/1034026143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4990103110 con 0x7f4990102d30 2024-09-11T22:53:04.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.352+0000 7f49867fc640 1 -- 172.21.15.152:0/1034026143 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f498c005020 con 0x7f4990102d30 2024-09-11T22:53:04.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.352+0000 7f49867fc640 1 --2- 172.21.15.152:0/1034026143 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f497c0a2e00 0x7f497c0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:04.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.352+0000 7f4994f37640 1 --2- 172.21.15.152:0/1034026143 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f497c0a2e00 0x7f497c0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:04.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.352+0000 7f49867fc640 1 -- 172.21.15.152:0/1034026143 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f498c0cf300 con 0x7f4990102d30 2024-09-11T22:53:04.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.353+0000 7f4994f37640 1 --2- 172.21.15.152:0/1034026143 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f497c0a2e00 0x7f497c0a52c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f49900fede0 tx=0x7f498000a2d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:04.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.357+0000 7f49867fc640 1 -- 172.21.15.152:0/1034026143 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f498c014030 con 0x7f4990102d30 2024-09-11T22:53:04.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.525+0000 7f499799c640 1 -- 172.21.15.152:0/1034026143 --> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4990062630 con 0x7f497c0a2e00 2024-09-11T22:53:04.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.532+0000 7f49867fc640 1 -- 172.21.15.152:0/1034026143 <== mgr.15240 v2:172.21.15.152:6800/4130474708 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258040 (secure 0 0 0) 0x7f4990062630 con 0x7f497c0a2e00 2024-09-11T22:53:04.535 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:53:04.536 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-09-11T22:53:04.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.536+0000 7f499799c640 1 -- 172.21.15.152:0/1034026143 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f497c0a2e00 msgr2=0x7f497c0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:04.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.536+0000 7f499799c640 1 --2- 172.21.15.152:0/1034026143 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f497c0a2e00 0x7f497c0a52c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f49900fede0 tx=0x7f498000a2d0 comp rx=0 tx=0).stop 2024-09-11T22:53:04.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.536+0000 7f499799c640 1 -- 172.21.15.152:0/1034026143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4990102d30 msgr2=0x7f4990198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:04.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.536+0000 7f499799c640 1 --2- 172.21.15.152:0/1034026143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4990102d30 0x7f4990198570 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f498c00c970 tx=0x7f498c00ce30 comp rx=0 tx=0).stop 2024-09-11T22:53:04.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.536+0000 7f499799c640 1 -- 172.21.15.152:0/1034026143 shutdown_connections 2024-09-11T22:53:04.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.536+0000 7f499799c640 1 --2- 172.21.15.152:0/1034026143 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f497c0a2e00 0x7f497c0a52c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:04.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.536+0000 7f499799c640 1 --2- 172.21.15.152:0/1034026143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49901082a0 0x7f499019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:04.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.536+0000 7f499799c640 1 --2- 172.21.15.152:0/1034026143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49901036e0 0x7f4990198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:04.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.536+0000 7f499799c640 1 --2- 172.21.15.152:0/1034026143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4990102d30 0x7f4990198570 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:04.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.536+0000 7f499799c640 1 -- 172.21.15.152:0/1034026143 >> 172.21.15.152:0/1034026143 conn(0x7f49900fc7d0 msgr2=0x7f4990105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:04.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.536+0000 7f499799c640 1 -- 172.21.15.152:0/1034026143 shutdown_connections 2024-09-11T22:53:04.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:04.536+0000 7f499799c640 1 -- 172.21.15.152:0/1034026143 wait complete. 2024-09-11T22:53:05.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:04 smithi191 ceph-mon[35211]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 228 B/s wr, 19 op/s 2024-09-11T22:53:05.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:04 smithi191 ceph-mon[35211]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:53:05.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:04 smithi204 ceph-mon[35454]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 228 B/s wr, 19 op/s 2024-09-11T22:53:05.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:04 smithi204 ceph-mon[35454]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:53:05.311 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:04 smithi152 ceph-mon[31181]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 228 B/s wr, 19 op/s 2024-09-11T22:53:05.311 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:04 smithi152 ceph-mon[31181]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:53:05.341 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-09-11T22:53:03.088118+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193417,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1517,"num_read_kb":1087,"num_write":585,"num_write_kb":4410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":957,"ondisk_log_size":957,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369088,"kb_used_data":11484,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572736,"statfs":{"total":1151940427776,"available":1151562481664,"internally_reserved":0,"allocated":11759616,"data_stored":8922423,"data_compressed":218223,"data_compressed_allocated":282624,"data_compressed_original":3416064,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":154,"num_read_kb":239,"num_write":24,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.974368"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169130+0000","last_change":"2024-09-11T22:50:54.171410+0000","last_active":"2024-09-11T22:52:53.169130+0000","last_peered":"2024-09-11T22:52:53.169130+0000","last_clean":"2024-09-11T22:52:53.169130+0000","last_became_active":"2024-09-11T22:50:54.171055+0000","last_became_peered":"2024-09-11T22:50:54.171055+0000","last_unstale":"2024-09-11T22:52:53.169130+0000","last_undegraded":"2024-09-11T22:52:53.169130+0000","last_fullsized":"2024-09-11T22:52:53.169130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:54.767454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131374+0000","last_change":"2024-09-11T22:50:54.168534+0000","last_active":"2024-09-11T22:52:53.131374+0000","last_peered":"2024-09-11T22:52:53.131374+0000","last_clean":"2024-09-11T22:52:53.131374+0000","last_became_active":"2024-09-11T22:50:54.168228+0000","last_became_peered":"2024-09-11T22:50:54.168228+0000","last_unstale":"2024-09-11T22:52:53.131374+0000","last_undegraded":"2024-09-11T22:52:53.131374+0000","last_fullsized":"2024-09-11T22:52:53.131374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:11.380048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131174+0000","last_change":"2024-09-11T22:50:54.168975+0000","last_active":"2024-09-11T22:52:53.131174+0000","last_peered":"2024-09-11T22:52:53.131174+0000","last_clean":"2024-09-11T22:52:53.131174+0000","last_became_active":"2024-09-11T22:50:54.168676+0000","last_became_peered":"2024-09-11T22:50:54.168676+0000","last_unstale":"2024-09-11T22:52:53.131174+0000","last_undegraded":"2024-09-11T22:52:53.131174+0000","last_fullsized":"2024-09-11T22:52:53.131174+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:15.848273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130100+0000","last_change":"2024-09-11T22:50:54.164164+0000","last_active":"2024-09-11T22:52:53.130100+0000","last_peered":"2024-09-11T22:52:53.130100+0000","last_clean":"2024-09-11T22:52:53.130100+0000","last_became_active":"2024-09-11T22:50:54.163805+0000","last_became_peered":"2024-09-11T22:50:54.163805+0000","last_unstale":"2024-09-11T22:52:53.130100+0000","last_undegraded":"2024-09-11T22:52:53.130100+0000","last_fullsized":"2024-09-11T22:52:53.130100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:54.954046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074671+0000","last_change":"2024-09-11T22:50:54.167289+0000","last_active":"2024-09-11T22:52:53.074671+0000","last_peered":"2024-09-11T22:52:53.074671+0000","last_clean":"2024-09-11T22:52:53.074671+0000","last_became_active":"2024-09-11T22:50:54.166958+0000","last_became_peered":"2024-09-11T22:50:54.166958+0000","last_unstale":"2024-09-11T22:52:53.074671+0000","last_undegraded":"2024-09-11T22:52:53.074671+0000","last_fullsized":"2024-09-11T22:52:53.074671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:25.261241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129676+0000","last_change":"2024-09-11T22:50:54.175605+0000","last_active":"2024-09-11T22:52:53.129676+0000","last_peered":"2024-09-11T22:52:53.129676+0000","last_clean":"2024-09-11T22:52:53.129676+0000","last_became_active":"2024-09-11T22:50:54.175207+0000","last_became_peered":"2024-09-11T22:50:54.175207+0000","last_unstale":"2024-09-11T22:52:53.129676+0000","last_undegraded":"2024-09-11T22:52:53.129676+0000","last_fullsized":"2024-09-11T22:52:53.129676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:47.008910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129084+0000","last_change":"2024-09-11T22:50:54.167930+0000","last_active":"2024-09-11T22:52:53.129084+0000","last_peered":"2024-09-11T22:52:53.129084+0000","last_clean":"2024-09-11T22:52:53.129084+0000","last_became_active":"2024-09-11T22:50:54.167082+0000","last_became_peered":"2024-09-11T22:50:54.167082+0000","last_unstale":"2024-09-11T22:52:53.129084+0000","last_undegraded":"2024-09-11T22:52:53.129084+0000","last_fullsized":"2024-09-11T22:52:53.129084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:50.931311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129336+0000","last_change":"2024-09-11T22:50:54.172528+0000","last_active":"2024-09-11T22:52:53.129336+0000","last_peered":"2024-09-11T22:52:53.129336+0000","last_clean":"2024-09-11T22:52:53.129336+0000","last_became_active":"2024-09-11T22:50:54.172257+0000","last_became_peered":"2024-09-11T22:50:54.172257+0000","last_unstale":"2024-09-11T22:52:53.129336+0000","last_undegraded":"2024-09-11T22:52:53.129336+0000","last_fullsized":"2024-09-11T22:52:53.129336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:13:43.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074665+0000","last_change":"2024-09-11T22:50:54.163357+0000","last_active":"2024-09-11T22:52:53.074665+0000","last_peered":"2024-09-11T22:52:53.074665+0000","last_clean":"2024-09-11T22:52:53.074665+0000","last_became_active":"2024-09-11T22:50:54.162943+0000","last_became_peered":"2024-09-11T22:50:54.162943+0000","last_unstale":"2024-09-11T22:52:53.074665+0000","last_undegraded":"2024-09-11T22:52:53.074665+0000","last_fullsized":"2024-09-11T22:52:53.074665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:57.055802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129448+0000","last_change":"2024-09-11T22:50:54.177773+0000","last_active":"2024-09-11T22:52:53.129448+0000","last_peered":"2024-09-11T22:52:53.129448+0000","last_clean":"2024-09-11T22:52:53.129448+0000","last_became_active":"2024-09-11T22:50:54.177388+0000","last_became_peered":"2024-09-11T22:50:54.177388+0000","last_unstale":"2024-09-11T22:52:53.129448+0000","last_undegraded":"2024-09-11T22:52:53.129448+0000","last_fullsized":"2024-09-11T22:52:53.129448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:36:44.625089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129792+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:52:53.129792+0000","last_peered":"2024-09-11T22:52:53.129792+0000","last_clean":"2024-09-11T22:52:53.129792+0000","last_became_active":"2024-09-11T22:50:54.171491+0000","last_became_peered":"2024-09-11T22:50:54.171491+0000","last_unstale":"2024-09-11T22:52:53.129792+0000","last_undegraded":"2024-09-11T22:52:53.129792+0000","last_fullsized":"2024-09-11T22:52:53.129792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.996640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130199+0000","last_change":"2024-09-11T22:50:54.171260+0000","last_active":"2024-09-11T22:52:53.130199+0000","last_peered":"2024-09-11T22:52:53.130199+0000","last_clean":"2024-09-11T22:52:53.130199+0000","last_became_active":"2024-09-11T22:50:54.170733+0000","last_became_peered":"2024-09-11T22:50:54.170733+0000","last_unstale":"2024-09-11T22:52:53.130199+0000","last_undegraded":"2024-09-11T22:52:53.130199+0000","last_fullsized":"2024-09-11T22:52:53.130199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:57.877280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129997+0000","last_change":"2024-09-11T22:50:54.170576+0000","last_active":"2024-09-11T22:52:53.129997+0000","last_peered":"2024-09-11T22:52:53.129997+0000","last_clean":"2024-09-11T22:52:53.129997+0000","last_became_active":"2024-09-11T22:50:54.170239+0000","last_became_peered":"2024-09-11T22:50:54.170239+0000","last_unstale":"2024-09-11T22:52:53.129997+0000","last_undegraded":"2024-09-11T22:52:53.129997+0000","last_fullsized":"2024-09-11T22:52:53.129997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:09.683171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130651+0000","last_change":"2024-09-11T22:50:54.167176+0000","last_active":"2024-09-11T22:52:53.130651+0000","last_peered":"2024-09-11T22:52:53.130651+0000","last_clean":"2024-09-11T22:52:53.130651+0000","last_became_active":"2024-09-11T22:50:54.166894+0000","last_became_peered":"2024-09-11T22:50:54.166894+0000","last_unstale":"2024-09-11T22:52:53.130651+0000","last_undegraded":"2024-09-11T22:52:53.130651+0000","last_fullsized":"2024-09-11T22:52:53.130651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:43:08.158786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130901+0000","last_change":"2024-09-11T22:50:54.163100+0000","last_active":"2024-09-11T22:52:53.130901+0000","last_peered":"2024-09-11T22:52:53.130901+0000","last_clean":"2024-09-11T22:52:53.130901+0000","last_became_active":"2024-09-11T22:50:54.162776+0000","last_became_peered":"2024-09-11T22:50:54.162776+0000","last_unstale":"2024-09-11T22:52:53.130901+0000","last_undegraded":"2024-09-11T22:52:53.130901+0000","last_fullsized":"2024-09-11T22:52:53.130901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:15.215305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074034+0000","last_change":"2024-09-11T22:50:54.166013+0000","last_active":"2024-09-11T22:52:53.074034+0000","last_peered":"2024-09-11T22:52:53.074034+0000","last_clean":"2024-09-11T22:52:53.074034+0000","last_became_active":"2024-09-11T22:50:54.165680+0000","last_became_peered":"2024-09-11T22:50:54.165680+0000","last_unstale":"2024-09-11T22:52:53.074034+0000","last_undegraded":"2024-09-11T22:52:53.074034+0000","last_fullsized":"2024-09-11T22:52:53.074034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:12.192231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169175+0000","last_change":"2024-09-11T22:50:54.172105+0000","last_active":"2024-09-11T22:52:53.169175+0000","last_peered":"2024-09-11T22:52:53.169175+0000","last_clean":"2024-09-11T22:52:53.169175+0000","last_became_active":"2024-09-11T22:50:54.171782+0000","last_became_peered":"2024-09-11T22:50:54.171782+0000","last_unstale":"2024-09-11T22:52:53.169175+0000","last_undegraded":"2024-09-11T22:52:53.169175+0000","last_fullsized":"2024-09-11T22:52:53.169175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:17.152834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130880+0000","last_change":"2024-09-11T22:50:54.167184+0000","last_active":"2024-09-11T22:52:53.130880+0000","last_peered":"2024-09-11T22:52:53.130880+0000","last_clean":"2024-09-11T22:52:53.130880+0000","last_became_active":"2024-09-11T22:50:54.166878+0000","last_became_peered":"2024-09-11T22:50:54.166878+0000","last_unstale":"2024-09-11T22:52:53.130880+0000","last_undegraded":"2024-09-11T22:52:53.130880+0000","last_fullsized":"2024-09-11T22:52:53.130880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:44.046552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129108+0000","last_change":"2024-09-11T22:50:54.176260+0000","last_active":"2024-09-11T22:52:53.129108+0000","last_peered":"2024-09-11T22:52:53.129108+0000","last_clean":"2024-09-11T22:52:53.129108+0000","last_became_active":"2024-09-11T22:50:54.175873+0000","last_became_peered":"2024-09-11T22:50:54.175873+0000","last_unstale":"2024-09-11T22:52:53.129108+0000","last_undegraded":"2024-09-11T22:52:53.129108+0000","last_fullsized":"2024-09-11T22:52:53.129108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:05.258677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168910+0000","last_change":"2024-09-11T22:50:54.172725+0000","last_active":"2024-09-11T22:52:53.168910+0000","last_peered":"2024-09-11T22:52:53.168910+0000","last_clean":"2024-09-11T22:52:53.168910+0000","last_became_active":"2024-09-11T22:50:54.172072+0000","last_became_peered":"2024-09-11T22:50:54.172072+0000","last_unstale":"2024-09-11T22:52:53.168910+0000","last_undegraded":"2024-09-11T22:52:53.168910+0000","last_fullsized":"2024-09-11T22:52:53.168910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:53.699670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130148+0000","last_change":"2024-09-11T22:50:54.165118+0000","last_active":"2024-09-11T22:52:53.130148+0000","last_peered":"2024-09-11T22:52:53.130148+0000","last_clean":"2024-09-11T22:52:53.130148+0000","last_became_active":"2024-09-11T22:50:54.164793+0000","last_became_peered":"2024-09-11T22:50:54.164793+0000","last_unstale":"2024-09-11T22:52:53.130148+0000","last_undegraded":"2024-09-11T22:52:53.130148+0000","last_fullsized":"2024-09-11T22:52:53.130148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:42.883940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074224+0000","last_change":"2024-09-11T22:50:54.176900+0000","last_active":"2024-09-11T22:52:53.074224+0000","last_peered":"2024-09-11T22:52:53.074224+0000","last_clean":"2024-09-11T22:52:53.074224+0000","last_became_active":"2024-09-11T22:50:54.176595+0000","last_became_peered":"2024-09-11T22:50:54.176595+0000","last_unstale":"2024-09-11T22:52:53.074224+0000","last_undegraded":"2024-09-11T22:52:53.074224+0000","last_fullsized":"2024-09-11T22:52:53.074224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:36.925945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129485+0000","last_change":"2024-09-11T22:50:54.167656+0000","last_active":"2024-09-11T22:52:53.129485+0000","last_peered":"2024-09-11T22:52:53.129485+0000","last_clean":"2024-09-11T22:52:53.129485+0000","last_became_active":"2024-09-11T22:50:54.167349+0000","last_became_peered":"2024-09-11T22:50:54.167349+0000","last_unstale":"2024-09-11T22:52:53.129485+0000","last_undegraded":"2024-09-11T22:52:53.129485+0000","last_fullsized":"2024-09-11T22:52:53.129485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:38.100218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073470+0000","last_change":"2024-09-11T22:50:54.177059+0000","last_active":"2024-09-11T22:52:53.073470+0000","last_peered":"2024-09-11T22:52:53.073470+0000","last_clean":"2024-09-11T22:52:53.073470+0000","last_became_active":"2024-09-11T22:50:54.176726+0000","last_became_peered":"2024-09-11T22:50:54.176726+0000","last_unstale":"2024-09-11T22:52:53.073470+0000","last_undegraded":"2024-09-11T22:52:53.073470+0000","last_fullsized":"2024-09-11T22:52:53.073470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:27:58.058944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130570+0000","last_change":"2024-09-11T22:50:54.167566+0000","last_active":"2024-09-11T22:52:53.130570+0000","last_peered":"2024-09-11T22:52:53.130570+0000","last_clean":"2024-09-11T22:52:53.130570+0000","last_became_active":"2024-09-11T22:50:54.167049+0000","last_became_peered":"2024-09-11T22:50:54.167049+0000","last_unstale":"2024-09-11T22:52:53.130570+0000","last_undegraded":"2024-09-11T22:52:53.130570+0000","last_fullsized":"2024-09-11T22:52:53.130570+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:06.833099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129193+0000","last_change":"2024-09-11T22:50:54.169732+0000","last_active":"2024-09-11T22:52:53.129193+0000","last_peered":"2024-09-11T22:52:53.129193+0000","last_clean":"2024-09-11T22:52:53.129193+0000","last_became_active":"2024-09-11T22:50:54.169437+0000","last_became_peered":"2024-09-11T22:50:54.169437+0000","last_unstale":"2024-09-11T22:52:53.129193+0000","last_undegraded":"2024-09-11T22:52:53.129193+0000","last_fullsized":"2024-09-11T22:52:53.129193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:25.553387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169106+0000","last_change":"2024-09-11T22:50:54.166953+0000","last_active":"2024-09-11T22:52:53.169106+0000","last_peered":"2024-09-11T22:52:53.169106+0000","last_clean":"2024-09-11T22:52:53.169106+0000","last_became_active":"2024-09-11T22:50:54.166472+0000","last_became_peered":"2024-09-11T22:50:54.166472+0000","last_unstale":"2024-09-11T22:52:53.169106+0000","last_undegraded":"2024-09-11T22:52:53.169106+0000","last_fullsized":"2024-09-11T22:52:53.169106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:01:20.234180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129423+0000","last_change":"2024-09-11T22:50:54.171308+0000","last_active":"2024-09-11T22:52:53.129423+0000","last_peered":"2024-09-11T22:52:53.129423+0000","last_clean":"2024-09-11T22:52:53.129423+0000","last_became_active":"2024-09-11T22:50:54.168958+0000","last_became_peered":"2024-09-11T22:50:54.168958+0000","last_unstale":"2024-09-11T22:52:53.129423+0000","last_undegraded":"2024-09-11T22:52:53.129423+0000","last_fullsized":"2024-09-11T22:52:53.129423+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:20.224003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131183+0000","last_change":"2024-09-11T22:50:54.173846+0000","last_active":"2024-09-11T22:52:53.131183+0000","last_peered":"2024-09-11T22:52:53.131183+0000","last_clean":"2024-09-11T22:52:53.131183+0000","last_became_active":"2024-09-11T22:50:54.173542+0000","last_became_peered":"2024-09-11T22:50:54.173542+0000","last_unstale":"2024-09-11T22:52:53.131183+0000","last_undegraded":"2024-09-11T22:52:53.131183+0000","last_fullsized":"2024-09-11T22:52:53.131183+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:29:45.775902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129000+0000","last_change":"2024-09-11T22:50:54.176008+0000","last_active":"2024-09-11T22:52:53.129000+0000","last_peered":"2024-09-11T22:52:53.129000+0000","last_clean":"2024-09-11T22:52:53.129000+0000","last_became_active":"2024-09-11T22:50:54.175672+0000","last_became_peered":"2024-09-11T22:50:54.175672+0000","last_unstale":"2024-09-11T22:52:53.129000+0000","last_undegraded":"2024-09-11T22:52:53.129000+0000","last_fullsized":"2024-09-11T22:52:53.129000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:35.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129006+0000","last_change":"2024-09-11T22:50:54.162504+0000","last_active":"2024-09-11T22:52:53.129006+0000","last_peered":"2024-09-11T22:52:53.129006+0000","last_clean":"2024-09-11T22:52:53.129006+0000","last_became_active":"2024-09-11T22:50:54.162190+0000","last_became_peered":"2024-09-11T22:50:54.162190+0000","last_unstale":"2024-09-11T22:52:53.129006+0000","last_undegraded":"2024-09-11T22:52:53.129006+0000","last_fullsized":"2024-09-11T22:52:53.129006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:52.138692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168887+0000","last_change":"2024-09-11T22:51:55.960882+0000","last_active":"2024-09-11T22:52:53.168887+0000","last_peered":"2024-09-11T22:52:53.168887+0000","last_clean":"2024-09-11T22:52:53.168887+0000","last_became_active":"2024-09-11T22:51:55.960246+0000","last_became_peered":"2024-09-11T22:51:55.960246+0000","last_unstale":"2024-09-11T22:52:53.168887+0000","last_undegraded":"2024-09-11T22:52:53.168887+0000","last_fullsized":"2024-09-11T22:52:53.168887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:21.247840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074402+0000","last_change":"2024-09-11T22:50:54.160696+0000","last_active":"2024-09-11T22:52:53.074402+0000","last_peered":"2024-09-11T22:52:53.074402+0000","last_clean":"2024-09-11T22:52:53.074402+0000","last_became_active":"2024-09-11T22:50:54.159249+0000","last_became_peered":"2024-09-11T22:50:54.159249+0000","last_unstale":"2024-09-11T22:52:53.074402+0000","last_undegraded":"2024-09-11T22:52:53.074402+0000","last_fullsized":"2024-09-11T22:52:53.074402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:09.649208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'9","reported_seq":152,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:57.285394+0000","last_change":"2024-09-11T22:50:52.138986+0000","last_active":"2024-09-11T22:52:57.285394+0000","last_peered":"2024-09-11T22:52:57.285394+0000","last_clean":"2024-09-11T22:52:57.285394+0000","last_became_active":"2024-09-11T22:50:52.138595+0000","last_became_peered":"2024-09-11T22:50:52.138595+0000","last_unstale":"2024-09-11T22:52:57.285394+0000","last_undegraded":"2024-09-11T22:52:57.285394+0000","last_fullsized":"2024-09-11T22:52:57.285394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:06.232146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1942,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073853+0000","last_change":"2024-09-11T22:50:54.176806+0000","last_active":"2024-09-11T22:52:53.073853+0000","last_peered":"2024-09-11T22:52:53.073853+0000","last_clean":"2024-09-11T22:52:53.073853+0000","last_became_active":"2024-09-11T22:50:54.176513+0000","last_became_peered":"2024-09-11T22:50:54.176513+0000","last_unstale":"2024-09-11T22:52:53.073853+0000","last_undegraded":"2024-09-11T22:52:53.073853+0000","last_fullsized":"2024-09-11T22:52:53.073853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:43:12.086145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129699+0000","last_change":"2024-09-11T22:50:52.142794+0000","last_active":"2024-09-11T22:52:53.129699+0000","last_peered":"2024-09-11T22:52:53.129699+0000","last_clean":"2024-09-11T22:52:53.129699+0000","last_became_active":"2024-09-11T22:50:52.142433+0000","last_became_peered":"2024-09-11T22:50:52.142433+0000","last_unstale":"2024-09-11T22:52:53.129699+0000","last_undegraded":"2024-09-11T22:52:53.129699+0000","last_fullsized":"2024-09-11T22:52:53.129699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:47:12.090055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130326+0000","last_change":"2024-09-11T22:50:52.139789+0000","last_active":"2024-09-11T22:52:53.130326+0000","last_peered":"2024-09-11T22:52:53.130326+0000","last_clean":"2024-09-11T22:52:53.130326+0000","last_became_active":"2024-09-11T22:50:52.139294+0000","last_became_peered":"2024-09-11T22:50:52.139294+0000","last_unstale":"2024-09-11T22:52:53.130326+0000","last_undegraded":"2024-09-11T22:52:53.130326+0000","last_fullsized":"2024-09-11T22:52:53.130326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:07.355155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129486+0000","last_change":"2024-09-11T22:50:54.166248+0000","last_active":"2024-09-11T22:52:53.129486+0000","last_peered":"2024-09-11T22:52:53.129486+0000","last_clean":"2024-09-11T22:52:53.129486+0000","last_became_active":"2024-09-11T22:50:54.165958+0000","last_became_peered":"2024-09-11T22:50:54.165958+0000","last_unstale":"2024-09-11T22:52:53.129486+0000","last_undegraded":"2024-09-11T22:52:53.129486+0000","last_fullsized":"2024-09-11T22:52:53.129486+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:29.531741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073597+0000","last_change":"2024-09-11T22:50:52.140340+0000","last_active":"2024-09-11T22:52:53.073597+0000","last_peered":"2024-09-11T22:52:53.073597+0000","last_clean":"2024-09-11T22:52:53.073597+0000","last_became_active":"2024-09-11T22:50:52.140011+0000","last_became_peered":"2024-09-11T22:50:52.140011+0000","last_unstale":"2024-09-11T22:52:53.073597+0000","last_undegraded":"2024-09-11T22:52:53.073597+0000","last_fullsized":"2024-09-11T22:52:53.073597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:58.862640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129625+0000","last_change":"2024-09-11T22:50:54.166998+0000","last_active":"2024-09-11T22:52:53.129625+0000","last_peered":"2024-09-11T22:52:53.129625+0000","last_clean":"2024-09-11T22:52:53.129625+0000","last_became_active":"2024-09-11T22:50:54.166682+0000","last_became_peered":"2024-09-11T22:50:54.166682+0000","last_unstale":"2024-09-11T22:52:53.129625+0000","last_undegraded":"2024-09-11T22:52:53.129625+0000","last_fullsized":"2024-09-11T22:52:53.129625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:57.815914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'8","reported_seq":260,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:58.286061+0000","last_change":"2024-09-11T22:50:52.141380+0000","last_active":"2024-09-11T22:52:58.286061+0000","last_peered":"2024-09-11T22:52:58.286061+0000","last_clean":"2024-09-11T22:52:58.286061+0000","last_became_active":"2024-09-11T22:50:52.139925+0000","last_became_peered":"2024-09-11T22:50:52.139925+0000","last_unstale":"2024-09-11T22:52:58.286061+0000","last_undegraded":"2024-09-11T22:52:58.286061+0000","last_fullsized":"2024-09-11T22:52:58.286061+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:40:38.395055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":228,"num_read_kb":228,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074552+0000","last_change":"2024-09-11T22:50:54.167909+0000","last_active":"2024-09-11T22:52:53.074552+0000","last_peered":"2024-09-11T22:52:53.074552+0000","last_clean":"2024-09-11T22:52:53.074552+0000","last_became_active":"2024-09-11T22:50:54.167571+0000","last_became_peered":"2024-09-11T22:50:54.167571+0000","last_unstale":"2024-09-11T22:52:53.074552+0000","last_undegraded":"2024-09-11T22:52:53.074552+0000","last_fullsized":"2024-09-11T22:52:53.074552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:34.179960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129913+0000","last_change":"2024-09-11T22:50:52.144545+0000","last_active":"2024-09-11T22:52:53.129913+0000","last_peered":"2024-09-11T22:52:53.129913+0000","last_clean":"2024-09-11T22:52:53.129913+0000","last_became_active":"2024-09-11T22:50:52.144221+0000","last_became_peered":"2024-09-11T22:50:52.144221+0000","last_unstale":"2024-09-11T22:52:53.129913+0000","last_undegraded":"2024-09-11T22:52:53.129913+0000","last_fullsized":"2024-09-11T22:52:53.129913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:29.509165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131618+0000","last_change":"2024-09-11T22:50:54.164132+0000","last_active":"2024-09-11T22:52:53.131618+0000","last_peered":"2024-09-11T22:52:53.131618+0000","last_clean":"2024-09-11T22:52:53.131618+0000","last_became_active":"2024-09-11T22:50:54.163842+0000","last_became_peered":"2024-09-11T22:50:54.163842+0000","last_unstale":"2024-09-11T22:52:53.131618+0000","last_undegraded":"2024-09-11T22:52:53.131618+0000","last_fullsized":"2024-09-11T22:52:53.131618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:28:55.490236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074349+0000","last_change":"2024-09-11T22:50:52.264611+0000","last_active":"2024-09-11T22:52:53.074349+0000","last_peered":"2024-09-11T22:52:53.074349+0000","last_clean":"2024-09-11T22:52:53.074349+0000","last_became_active":"2024-09-11T22:50:52.264060+0000","last_became_peered":"2024-09-11T22:50:52.264060+0000","last_unstale":"2024-09-11T22:52:53.074349+0000","last_undegraded":"2024-09-11T22:52:53.074349+0000","last_fullsized":"2024-09-11T22:52:53.074349+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:42.543997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074271+0000","last_change":"2024-09-11T22:51:56.206155+0000","last_active":"2024-09-11T22:52:53.074271+0000","last_peered":"2024-09-11T22:52:53.074271+0000","last_clean":"2024-09-11T22:52:53.074271+0000","last_became_active":"2024-09-11T22:51:56.205797+0000","last_became_peered":"2024-09-11T22:51:56.205797+0000","last_unstale":"2024-09-11T22:52:53.074271+0000","last_undegraded":"2024-09-11T22:52:53.074271+0000","last_fullsized":"2024-09-11T22:52:53.074271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:28.063825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'39","reported_seq":304,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:58.286191+0000","last_change":"2024-09-11T22:50:52.141782+0000","last_active":"2024-09-11T22:52:58.286191+0000","last_peered":"2024-09-11T22:52:58.286191+0000","last_clean":"2024-09-11T22:52:58.286191+0000","last_became_active":"2024-09-11T22:50:52.140662+0000","last_became_peered":"2024-09-11T22:50:52.140662+0000","last_unstale":"2024-09-11T22:52:58.286191+0000","last_undegraded":"2024-09-11T22:52:58.286191+0000","last_fullsized":"2024-09-11T22:52:58.286191+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:21.158577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68544,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":241,"num_read_kb":186,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074760+0000","last_change":"2024-09-11T22:50:54.166678+0000","last_active":"2024-09-11T22:52:53.074760+0000","last_peered":"2024-09-11T22:52:53.074760+0000","last_clean":"2024-09-11T22:52:53.074760+0000","last_became_active":"2024-09-11T22:50:54.165777+0000","last_became_peered":"2024-09-11T22:50:54.165777+0000","last_unstale":"2024-09-11T22:52:53.074760+0000","last_undegraded":"2024-09-11T22:52:53.074760+0000","last_fullsized":"2024-09-11T22:52:53.074760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:07.224563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129051+0000","last_change":"2024-09-11T22:50:52.141292+0000","last_active":"2024-09-11T22:52:53.129051+0000","last_peered":"2024-09-11T22:52:53.129051+0000","last_clean":"2024-09-11T22:52:53.129051+0000","last_became_active":"2024-09-11T22:50:52.140971+0000","last_became_peered":"2024-09-11T22:50:52.140971+0000","last_unstale":"2024-09-11T22:52:53.129051+0000","last_undegraded":"2024-09-11T22:52:53.129051+0000","last_fullsized":"2024-09-11T22:52:53.129051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:53.039684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129489+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:52:53.129489+0000","last_peered":"2024-09-11T22:52:53.129489+0000","last_clean":"2024-09-11T22:52:53.129489+0000","last_became_active":"2024-09-11T22:50:54.171520+0000","last_became_peered":"2024-09-11T22:50:54.171520+0000","last_unstale":"2024-09-11T22:52:53.129489+0000","last_undegraded":"2024-09-11T22:52:53.129489+0000","last_fullsized":"2024-09-11T22:52:53.129489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:56.594058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073605+0000","last_change":"2024-09-11T22:50:52.138725+0000","last_active":"2024-09-11T22:52:53.073605+0000","last_peered":"2024-09-11T22:52:53.073605+0000","last_clean":"2024-09-11T22:52:53.073605+0000","last_became_active":"2024-09-11T22:50:52.138367+0000","last_became_peered":"2024-09-11T22:50:52.138367+0000","last_unstale":"2024-09-11T22:52:53.073605+0000","last_undegraded":"2024-09-11T22:52:53.073605+0000","last_fullsized":"2024-09-11T22:52:53.073605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:58.811444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129217+0000","last_change":"2024-09-11T22:50:54.170524+0000","last_active":"2024-09-11T22:52:53.129217+0000","last_peered":"2024-09-11T22:52:53.129217+0000","last_clean":"2024-09-11T22:52:53.129217+0000","last_became_active":"2024-09-11T22:50:54.169947+0000","last_became_peered":"2024-09-11T22:50:54.169947+0000","last_unstale":"2024-09-11T22:52:53.129217+0000","last_undegraded":"2024-09-11T22:52:53.129217+0000","last_fullsized":"2024-09-11T22:52:53.129217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:51.768718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":148,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:57.343451+0000","last_change":"2024-09-11T22:50:52.139182+0000","last_active":"2024-09-11T22:52:57.343451+0000","last_peered":"2024-09-11T22:52:57.343451+0000","last_clean":"2024-09-11T22:52:57.343451+0000","last_became_active":"2024-09-11T22:50:52.138799+0000","last_became_peered":"2024-09-11T22:50:52.138799+0000","last_unstale":"2024-09-11T22:52:57.343451+0000","last_undegraded":"2024-09-11T22:52:57.343451+0000","last_fullsized":"2024-09-11T22:52:57.343451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:36.376162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129122+0000","last_change":"2024-09-11T22:50:54.168797+0000","last_active":"2024-09-11T22:52:53.129122+0000","last_peered":"2024-09-11T22:52:53.129122+0000","last_clean":"2024-09-11T22:52:53.129122+0000","last_became_active":"2024-09-11T22:50:54.168408+0000","last_became_peered":"2024-09-11T22:50:54.168408+0000","last_unstale":"2024-09-11T22:52:53.129122+0000","last_undegraded":"2024-09-11T22:52:53.129122+0000","last_fullsized":"2024-09-11T22:52:53.129122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:51.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074207+0000","last_change":"2024-09-11T22:50:52.264693+0000","last_active":"2024-09-11T22:52:53.074207+0000","last_peered":"2024-09-11T22:52:53.074207+0000","last_clean":"2024-09-11T22:52:53.074207+0000","last_became_active":"2024-09-11T22:50:52.264058+0000","last_became_peered":"2024-09-11T22:50:52.264058+0000","last_unstale":"2024-09-11T22:52:53.074207+0000","last_undegraded":"2024-09-11T22:52:53.074207+0000","last_fullsized":"2024-09-11T22:52:53.074207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:46.985921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073438+0000","last_change":"2024-09-11T22:51:55.961118+0000","last_active":"2024-09-11T22:52:53.073438+0000","last_peered":"2024-09-11T22:52:53.073438+0000","last_clean":"2024-09-11T22:52:53.073438+0000","last_became_active":"2024-09-11T22:51:55.960614+0000","last_became_peered":"2024-09-11T22:51:55.960614+0000","last_unstale":"2024-09-11T22:52:53.073438+0000","last_undegraded":"2024-09-11T22:52:53.073438+0000","last_fullsized":"2024-09-11T22:52:53.073438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:47.184327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.075083+0000","last_change":"2024-09-11T22:50:52.133049+0000","last_active":"2024-09-11T22:52:53.075083+0000","last_peered":"2024-09-11T22:52:53.075083+0000","last_clean":"2024-09-11T22:52:53.075083+0000","last_became_active":"2024-09-11T22:50:52.132647+0000","last_became_peered":"2024-09-11T22:50:52.132647+0000","last_unstale":"2024-09-11T22:52:53.075083+0000","last_undegraded":"2024-09-11T22:52:53.075083+0000","last_fullsized":"2024-09-11T22:52:53.075083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:13.213003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169115+0000","last_change":"2024-09-11T22:51:56.238023+0000","last_active":"2024-09-11T22:52:53.169115+0000","last_peered":"2024-09-11T22:52:53.169115+0000","last_clean":"2024-09-11T22:52:53.169115+0000","last_became_active":"2024-09-11T22:51:56.237676+0000","last_became_peered":"2024-09-11T22:51:56.237676+0000","last_unstale":"2024-09-11T22:52:53.169115+0000","last_undegraded":"2024-09-11T22:52:53.169115+0000","last_fullsized":"2024-09-11T22:52:53.169115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:16.215961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130047+0000","last_change":"2024-09-11T22:50:52.139766+0000","last_active":"2024-09-11T22:52:53.130047+0000","last_peered":"2024-09-11T22:52:53.130047+0000","last_clean":"2024-09-11T22:52:53.130047+0000","last_became_active":"2024-09-11T22:50:52.139456+0000","last_became_peered":"2024-09-11T22:50:52.139456+0000","last_unstale":"2024-09-11T22:52:53.130047+0000","last_undegraded":"2024-09-11T22:52:53.130047+0000","last_fullsized":"2024-09-11T22:52:53.130047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:08.358649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130080+0000","last_change":"2024-09-11T22:50:54.167717+0000","last_active":"2024-09-11T22:52:53.130080+0000","last_peered":"2024-09-11T22:52:53.130080+0000","last_clean":"2024-09-11T22:52:53.130080+0000","last_became_active":"2024-09-11T22:50:54.167331+0000","last_became_peered":"2024-09-11T22:50:54.167331+0000","last_unstale":"2024-09-11T22:52:53.130080+0000","last_undegraded":"2024-09-11T22:52:53.130080+0000","last_fullsized":"2024-09-11T22:52:53.130080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:11.240351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'41","reported_seq":73,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.283886+0000","last_change":"2024-09-11T22:50:52.141682+0000","last_active":"2024-09-11T22:52:53.283886+0000","last_peered":"2024-09-11T22:52:53.283886+0000","last_clean":"2024-09-11T22:52:53.283886+0000","last_became_active":"2024-09-11T22:50:52.141251+0000","last_became_peered":"2024-09-11T22:50:52.141251+0000","last_unstale":"2024-09-11T22:52:53.283886+0000","last_undegraded":"2024-09-11T22:52:53.283886+0000","last_fullsized":"2024-09-11T22:52:53.283886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:03.068252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129793+0000","last_change":"2024-09-11T22:50:54.174149+0000","last_active":"2024-09-11T22:52:53.129793+0000","last_peered":"2024-09-11T22:52:53.129793+0000","last_clean":"2024-09-11T22:52:53.129793+0000","last_became_active":"2024-09-11T22:50:54.173765+0000","last_became_peered":"2024-09-11T22:50:54.173765+0000","last_unstale":"2024-09-11T22:52:53.129793+0000","last_undegraded":"2024-09-11T22:52:53.129793+0000","last_fullsized":"2024-09-11T22:52:53.129793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.314334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'8","reported_seq":151,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:57.285457+0000","last_change":"2024-09-11T22:50:52.435019+0000","last_active":"2024-09-11T22:52:57.285457+0000","last_peered":"2024-09-11T22:52:57.285457+0000","last_clean":"2024-09-11T22:52:57.285457+0000","last_became_active":"2024-09-11T22:50:52.434642+0000","last_became_peered":"2024-09-11T22:50:52.434642+0000","last_unstale":"2024-09-11T22:52:57.285457+0000","last_undegraded":"2024-09-11T22:52:57.285457+0000","last_fullsized":"2024-09-11T22:52:57.285457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:18.441841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130004+0000","last_change":"2024-09-11T22:50:54.171291+0000","last_active":"2024-09-11T22:52:53.130004+0000","last_peered":"2024-09-11T22:52:53.130004+0000","last_clean":"2024-09-11T22:52:53.130004+0000","last_became_active":"2024-09-11T22:50:54.170961+0000","last_became_peered":"2024-09-11T22:50:54.170961+0000","last_unstale":"2024-09-11T22:52:53.130004+0000","last_undegraded":"2024-09-11T22:52:53.130004+0000","last_fullsized":"2024-09-11T22:52:53.130004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:07.211936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"84'148","reported_seq":392,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:55.179683+0000","last_change":"2024-09-11T22:49:39.098881+0000","last_active":"2024-09-11T22:52:55.179683+0000","last_peered":"2024-09-11T22:52:55.179683+0000","last_clean":"2024-09-11T22:52:55.179683+0000","last_became_active":"2024-09-11T22:49:39.098458+0000","last_became_peered":"2024-09-11T22:49:39.098458+0000","last_unstale":"2024-09-11T22:52:55.179683+0000","last_undegraded":"2024-09-11T22:52:55.179683+0000","last_fullsized":"2024-09-11T22:52:55.179683+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_clean_scrub_stamp":"2024-09-11T22:46:11.418577+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:41.059384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130085+0000","last_change":"2024-09-11T22:50:52.137691+0000","last_active":"2024-09-11T22:52:53.130085+0000","last_peered":"2024-09-11T22:52:53.130085+0000","last_clean":"2024-09-11T22:52:53.130085+0000","last_became_active":"2024-09-11T22:50:52.137282+0000","last_became_peered":"2024-09-11T22:50:52.137282+0000","last_unstale":"2024-09-11T22:52:53.130085+0000","last_undegraded":"2024-09-11T22:52:53.130085+0000","last_fullsized":"2024-09-11T22:52:53.130085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:00:27.651252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129789+0000","last_change":"2024-09-11T22:51:55.959937+0000","last_active":"2024-09-11T22:52:53.129789+0000","last_peered":"2024-09-11T22:52:53.129789+0000","last_clean":"2024-09-11T22:52:53.129789+0000","last_became_active":"2024-09-11T22:51:55.959575+0000","last_became_peered":"2024-09-11T22:51:55.959575+0000","last_unstale":"2024-09-11T22:52:53.129789+0000","last_undegraded":"2024-09-11T22:52:53.129789+0000","last_fullsized":"2024-09-11T22:52:53.129789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:54.097552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129633+0000","last_change":"2024-09-11T22:50:52.144324+0000","last_active":"2024-09-11T22:52:53.129633+0000","last_peered":"2024-09-11T22:52:53.129633+0000","last_clean":"2024-09-11T22:52:53.129633+0000","last_became_active":"2024-09-11T22:50:52.144011+0000","last_became_peered":"2024-09-11T22:50:52.144011+0000","last_unstale":"2024-09-11T22:52:53.129633+0000","last_undegraded":"2024-09-11T22:52:53.129633+0000","last_fullsized":"2024-09-11T22:52:53.129633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:29.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130247+0000","last_change":"2024-09-11T22:50:54.173287+0000","last_active":"2024-09-11T22:52:53.130247+0000","last_peered":"2024-09-11T22:52:53.130247+0000","last_clean":"2024-09-11T22:52:53.130247+0000","last_became_active":"2024-09-11T22:50:54.172953+0000","last_became_peered":"2024-09-11T22:50:54.172953+0000","last_unstale":"2024-09-11T22:52:53.130247+0000","last_undegraded":"2024-09-11T22:52:53.130247+0000","last_fullsized":"2024-09-11T22:52:53.130247+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:24.314604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":146,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:57.342996+0000","last_change":"2024-09-11T22:50:52.263777+0000","last_active":"2024-09-11T22:52:57.342996+0000","last_peered":"2024-09-11T22:52:57.342996+0000","last_clean":"2024-09-11T22:52:57.342996+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:52:57.342996+0000","last_undegraded":"2024-09-11T22:52:57.342996+0000","last_fullsized":"2024-09-11T22:52:57.342996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:37.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073458+0000","last_change":"2024-09-11T22:50:54.164927+0000","last_active":"2024-09-11T22:52:53.073458+0000","last_peered":"2024-09-11T22:52:53.073458+0000","last_clean":"2024-09-11T22:52:53.073458+0000","last_became_active":"2024-09-11T22:50:54.164602+0000","last_became_peered":"2024-09-11T22:50:54.164602+0000","last_unstale":"2024-09-11T22:52:53.073458+0000","last_undegraded":"2024-09-11T22:52:53.073458+0000","last_fullsized":"2024-09-11T22:52:53.073458+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:10.226232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":144,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:58.295009+0000","last_change":"2024-09-11T22:50:52.141199+0000","last_active":"2024-09-11T22:52:58.295009+0000","last_peered":"2024-09-11T22:52:58.295009+0000","last_clean":"2024-09-11T22:52:58.295009+0000","last_became_active":"2024-09-11T22:50:52.140830+0000","last_became_peered":"2024-09-11T22:50:52.140830+0000","last_unstale":"2024-09-11T22:52:58.295009+0000","last_undegraded":"2024-09-11T22:52:58.295009+0000","last_fullsized":"2024-09-11T22:52:58.295009+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.767803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129616+0000","last_change":"2024-09-11T22:50:54.175054+0000","last_active":"2024-09-11T22:52:53.129616+0000","last_peered":"2024-09-11T22:52:53.129616+0000","last_clean":"2024-09-11T22:52:53.129616+0000","last_became_active":"2024-09-11T22:50:54.174715+0000","last_became_peered":"2024-09-11T22:50:54.174715+0000","last_unstale":"2024-09-11T22:52:53.129616+0000","last_undegraded":"2024-09-11T22:52:53.129616+0000","last_fullsized":"2024-09-11T22:52:53.129616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:06.694988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130210+0000","last_change":"2024-09-11T22:51:55.961660+0000","last_active":"2024-09-11T22:52:53.130210+0000","last_peered":"2024-09-11T22:52:53.130210+0000","last_clean":"2024-09-11T22:52:53.130210+0000","last_became_active":"2024-09-11T22:51:55.961278+0000","last_became_peered":"2024-09-11T22:51:55.961278+0000","last_unstale":"2024-09-11T22:52:53.130210+0000","last_undegraded":"2024-09-11T22:52:53.130210+0000","last_fullsized":"2024-09-11T22:52:53.130210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:52.352490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129357+0000","last_change":"2024-09-11T22:50:54.167361+0000","last_active":"2024-09-11T22:52:53.129357+0000","last_peered":"2024-09-11T22:52:53.129357+0000","last_clean":"2024-09-11T22:52:53.129357+0000","last_became_active":"2024-09-11T22:50:54.166963+0000","last_became_peered":"2024-09-11T22:50:54.166963+0000","last_unstale":"2024-09-11T22:52:53.129357+0000","last_undegraded":"2024-09-11T22:52:53.129357+0000","last_fullsized":"2024-09-11T22:52:53.129357+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:13.294544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.075075+0000","last_change":"2024-09-11T22:50:52.264484+0000","last_active":"2024-09-11T22:52:53.075075+0000","last_peered":"2024-09-11T22:52:53.075075+0000","last_clean":"2024-09-11T22:52:53.075075+0000","last_became_active":"2024-09-11T22:50:52.264038+0000","last_became_peered":"2024-09-11T22:50:52.264038+0000","last_unstale":"2024-09-11T22:52:53.075075+0000","last_undegraded":"2024-09-11T22:52:53.075075+0000","last_fullsized":"2024-09-11T22:52:53.075075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:21.325136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129556+0000","last_change":"2024-09-11T22:50:54.162225+0000","last_active":"2024-09-11T22:52:53.129556+0000","last_peered":"2024-09-11T22:52:53.129556+0000","last_clean":"2024-09-11T22:52:53.129556+0000","last_became_active":"2024-09-11T22:50:54.161234+0000","last_became_peered":"2024-09-11T22:50:54.161234+0000","last_unstale":"2024-09-11T22:52:53.129556+0000","last_undegraded":"2024-09-11T22:52:53.129556+0000","last_fullsized":"2024-09-11T22:52:53.129556+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:00.346217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'52","reported_seq":115,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.286808+0000","last_change":"2024-09-11T22:51:55.967804+0000","last_active":"2024-09-11T22:52:53.286808+0000","last_peered":"2024-09-11T22:52:53.286808+0000","last_clean":"2024-09-11T22:52:53.286808+0000","last_became_active":"2024-09-11T22:51:55.961136+0000","last_became_peered":"2024-09-11T22:51:55.961136+0000","last_unstale":"2024-09-11T22:52:53.286808+0000","last_undegraded":"2024-09-11T22:52:53.286808+0000","last_fullsized":"2024-09-11T22:52:53.286808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:55.315179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130600+0000","last_change":"2024-09-11T22:50:54.167307+0000","last_active":"2024-09-11T22:52:53.130600+0000","last_peered":"2024-09-11T22:52:53.130600+0000","last_clean":"2024-09-11T22:52:53.130600+0000","last_became_active":"2024-09-11T22:50:54.166650+0000","last_became_peered":"2024-09-11T22:50:54.166650+0000","last_unstale":"2024-09-11T22:52:53.130600+0000","last_undegraded":"2024-09-11T22:52:53.130600+0000","last_fullsized":"2024-09-11T22:52:53.130600+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:04.863258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130339+0000","last_change":"2024-09-11T22:50:52.435107+0000","last_active":"2024-09-11T22:52:53.130339+0000","last_peered":"2024-09-11T22:52:53.130339+0000","last_clean":"2024-09-11T22:52:53.130339+0000","last_became_active":"2024-09-11T22:50:52.434791+0000","last_became_peered":"2024-09-11T22:50:52.434791+0000","last_unstale":"2024-09-11T22:52:53.130339+0000","last_undegraded":"2024-09-11T22:52:53.130339+0000","last_fullsized":"2024-09-11T22:52:53.130339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:11.128791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131410+0000","last_change":"2024-09-11T22:50:54.178041+0000","last_active":"2024-09-11T22:52:53.131410+0000","last_peered":"2024-09-11T22:52:53.131410+0000","last_clean":"2024-09-11T22:52:53.131410+0000","last_became_active":"2024-09-11T22:50:54.177393+0000","last_became_peered":"2024-09-11T22:50:54.177393+0000","last_unstale":"2024-09-11T22:52:53.131410+0000","last_undegraded":"2024-09-11T22:52:53.131410+0000","last_fullsized":"2024-09-11T22:52:53.131410+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:21.608104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074102+0000","last_change":"2024-09-11T22:50:52.264051+0000","last_active":"2024-09-11T22:52:53.074102+0000","last_peered":"2024-09-11T22:52:53.074102+0000","last_clean":"2024-09-11T22:52:53.074102+0000","last_became_active":"2024-09-11T22:50:52.263674+0000","last_became_peered":"2024-09-11T22:50:52.263674+0000","last_unstale":"2024-09-11T22:52:53.074102+0000","last_undegraded":"2024-09-11T22:52:53.074102+0000","last_fullsized":"2024-09-11T22:52:53.074102+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:44.655218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074213+0000","last_change":"2024-09-11T22:50:54.166409+0000","last_active":"2024-09-11T22:52:53.074213+0000","last_peered":"2024-09-11T22:52:53.074213+0000","last_clean":"2024-09-11T22:52:53.074213+0000","last_became_active":"2024-09-11T22:50:54.166037+0000","last_became_peered":"2024-09-11T22:50:54.166037+0000","last_unstale":"2024-09-11T22:52:53.074213+0000","last_undegraded":"2024-09-11T22:52:53.074213+0000","last_fullsized":"2024-09-11T22:52:53.074213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:22.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129590+0000","last_change":"2024-09-11T22:50:52.140218+0000","last_active":"2024-09-11T22:52:53.129590+0000","last_peered":"2024-09-11T22:52:53.129590+0000","last_clean":"2024-09-11T22:52:53.129590+0000","last_became_active":"2024-09-11T22:50:52.139744+0000","last_became_peered":"2024-09-11T22:50:52.139744+0000","last_unstale":"2024-09-11T22:52:53.129590+0000","last_undegraded":"2024-09-11T22:52:53.129590+0000","last_fullsized":"2024-09-11T22:52:53.129590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:50.178348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168909+0000","last_change":"2024-09-11T22:50:54.164059+0000","last_active":"2024-09-11T22:52:53.168909+0000","last_peered":"2024-09-11T22:52:53.168909+0000","last_clean":"2024-09-11T22:52:53.168909+0000","last_became_active":"2024-09-11T22:50:54.163724+0000","last_became_peered":"2024-09-11T22:50:54.163724+0000","last_unstale":"2024-09-11T22:52:53.168909+0000","last_undegraded":"2024-09-11T22:52:53.168909+0000","last_fullsized":"2024-09-11T22:52:53.168909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:57.241660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129028+0000","last_change":"2024-09-11T22:50:52.140169+0000","last_active":"2024-09-11T22:52:53.129028+0000","last_peered":"2024-09-11T22:52:53.129028+0000","last_clean":"2024-09-11T22:52:53.129028+0000","last_became_active":"2024-09-11T22:50:52.139579+0000","last_became_peered":"2024-09-11T22:50:52.139579+0000","last_unstale":"2024-09-11T22:52:53.129028+0000","last_undegraded":"2024-09-11T22:52:53.129028+0000","last_fullsized":"2024-09-11T22:52:53.129028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:43.523961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130075+0000","last_change":"2024-09-11T22:50:54.171539+0000","last_active":"2024-09-11T22:52:53.130075+0000","last_peered":"2024-09-11T22:52:53.130075+0000","last_clean":"2024-09-11T22:52:53.130075+0000","last_became_active":"2024-09-11T22:50:54.171013+0000","last_became_peered":"2024-09-11T22:50:54.171013+0000","last_unstale":"2024-09-11T22:52:53.130075+0000","last_undegraded":"2024-09-11T22:52:53.130075+0000","last_fullsized":"2024-09-11T22:52:53.130075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:03.564389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074787+0000","last_change":"2024-09-11T22:50:52.139544+0000","last_active":"2024-09-11T22:52:53.074787+0000","last_peered":"2024-09-11T22:52:53.074787+0000","last_clean":"2024-09-11T22:52:53.074787+0000","last_became_active":"2024-09-11T22:50:52.139201+0000","last_became_peered":"2024-09-11T22:50:52.139201+0000","last_unstale":"2024-09-11T22:52:53.074787+0000","last_undegraded":"2024-09-11T22:52:53.074787+0000","last_fullsized":"2024-09-11T22:52:53.074787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:37.343437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129337+0000","last_change":"2024-09-11T22:50:54.170052+0000","last_active":"2024-09-11T22:52:53.129337+0000","last_peered":"2024-09-11T22:52:53.129337+0000","last_clean":"2024-09-11T22:52:53.129337+0000","last_became_active":"2024-09-11T22:50:54.169735+0000","last_became_peered":"2024-09-11T22:50:54.169735+0000","last_unstale":"2024-09-11T22:52:53.129337+0000","last_undegraded":"2024-09-11T22:52:53.129337+0000","last_fullsized":"2024-09-11T22:52:53.129337+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:44.007747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074181+0000","last_change":"2024-09-11T22:50:54.169959+0000","last_active":"2024-09-11T22:52:53.074181+0000","last_peered":"2024-09-11T22:52:53.074181+0000","last_clean":"2024-09-11T22:52:53.074181+0000","last_became_active":"2024-09-11T22:50:54.169649+0000","last_became_peered":"2024-09-11T22:50:54.169649+0000","last_unstale":"2024-09-11T22:52:53.074181+0000","last_undegraded":"2024-09-11T22:52:53.074181+0000","last_fullsized":"2024-09-11T22:52:53.074181+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:51.974258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074520+0000","last_change":"2024-09-11T22:50:52.140881+0000","last_active":"2024-09-11T22:52:53.074520+0000","last_peered":"2024-09-11T22:52:53.074520+0000","last_clean":"2024-09-11T22:52:53.074520+0000","last_became_active":"2024-09-11T22:50:52.140527+0000","last_became_peered":"2024-09-11T22:50:52.140527+0000","last_unstale":"2024-09-11T22:52:53.074520+0000","last_undegraded":"2024-09-11T22:52:53.074520+0000","last_fullsized":"2024-09-11T22:52:53.074520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:14.325816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074263+0000","last_change":"2024-09-11T22:50:54.166571+0000","last_active":"2024-09-11T22:52:53.074263+0000","last_peered":"2024-09-11T22:52:53.074263+0000","last_clean":"2024-09-11T22:52:53.074263+0000","last_became_active":"2024-09-11T22:50:54.166247+0000","last_became_peered":"2024-09-11T22:50:54.166247+0000","last_unstale":"2024-09-11T22:52:53.074263+0000","last_undegraded":"2024-09-11T22:52:53.074263+0000","last_fullsized":"2024-09-11T22:52:53.074263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:42.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074895+0000","last_change":"2024-09-11T22:50:52.140298+0000","last_active":"2024-09-11T22:52:53.074895+0000","last_peered":"2024-09-11T22:52:53.074895+0000","last_clean":"2024-09-11T22:52:53.074895+0000","last_became_active":"2024-09-11T22:50:52.139917+0000","last_became_peered":"2024-09-11T22:50:52.139917+0000","last_unstale":"2024-09-11T22:52:53.074895+0000","last_undegraded":"2024-09-11T22:52:53.074895+0000","last_fullsized":"2024-09-11T22:52:53.074895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:23.599010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131626+0000","last_change":"2024-09-11T22:50:54.170849+0000","last_active":"2024-09-11T22:52:53.131626+0000","last_peered":"2024-09-11T22:52:53.131626+0000","last_clean":"2024-09-11T22:52:53.131626+0000","last_became_active":"2024-09-11T22:50:54.169547+0000","last_became_peered":"2024-09-11T22:50:54.169547+0000","last_unstale":"2024-09-11T22:52:53.131626+0000","last_undegraded":"2024-09-11T22:52:53.131626+0000","last_fullsized":"2024-09-11T22:52:53.131626+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:25.251158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130596+0000","last_change":"2024-09-11T22:50:52.263778+0000","last_active":"2024-09-11T22:52:53.130596+0000","last_peered":"2024-09-11T22:52:53.130596+0000","last_clean":"2024-09-11T22:52:53.130596+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:52:53.130596+0000","last_undegraded":"2024-09-11T22:52:53.130596+0000","last_fullsized":"2024-09-11T22:52:53.130596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:03.439147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130475+0000","last_change":"2024-09-11T22:50:54.171200+0000","last_active":"2024-09-11T22:52:53.130475+0000","last_peered":"2024-09-11T22:52:53.130475+0000","last_clean":"2024-09-11T22:52:53.130475+0000","last_became_active":"2024-09-11T22:50:54.169320+0000","last_became_peered":"2024-09-11T22:50:54.169320+0000","last_unstale":"2024-09-11T22:52:53.130475+0000","last_undegraded":"2024-09-11T22:52:53.130475+0000","last_fullsized":"2024-09-11T22:52:53.130475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:14.864699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":152,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:58.294477+0000","last_change":"2024-09-11T22:50:52.139609+0000","last_active":"2024-09-11T22:52:58.294477+0000","last_peered":"2024-09-11T22:52:58.294477+0000","last_clean":"2024-09-11T22:52:58.294477+0000","last_became_active":"2024-09-11T22:50:52.139278+0000","last_became_peered":"2024-09-11T22:50:52.139278+0000","last_unstale":"2024-09-11T22:52:58.294477+0000","last_undegraded":"2024-09-11T22:52:58.294477+0000","last_fullsized":"2024-09-11T22:52:58.294477+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:09.538734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602815,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1280,"num_read_kb":757,"num_write":382,"num_write_kb":1470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1808445,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107505,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547239,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":942080,"data_stored":840516,"data_compressed":22047,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":1238332,"data_compressed":44892,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939697,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":966656,"data_stored":839519,"data_compressed":22047,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168629,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":446681,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397563,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1040914,"data_compressed":38736,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561089,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":446020,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1171406,"data_compressed":44892,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953547,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":925696,"data_stored":446667,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":987136,"data_stored":970661,"data_compressed":28203,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":517944,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":515477,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":946176,"data_stored":448286,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135740,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":202666,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":395335,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:53:05.346 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:53:05.599 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:06.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.097+0000 7f602f7d4640 1 -- 172.21.15.152:0/3666119543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6028100a60 msgr2=0x7f6028100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:06.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.097+0000 7f602f7d4640 1 --2- 172.21.15.152:0/3666119543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6028100a60 0x7f6028100ec0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6024009930 tx=0x7f602402a420 comp rx=0 tx=0).stop 2024-09-11T22:53:06.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.097+0000 7f602f7d4640 1 -- 172.21.15.152:0/3666119543 shutdown_connections 2024-09-11T22:53:06.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.097+0000 7f602f7d4640 1 --2- 172.21.15.152:0/3666119543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6028101400 0x7f602810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:06.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.097+0000 7f602f7d4640 1 --2- 172.21.15.152:0/3666119543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6028100a60 0x7f6028100ec0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:06.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.097+0000 7f602f7d4640 1 --2- 172.21.15.152:0/3666119543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6028106a60 0x7f6028106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:06.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.097+0000 7f602f7d4640 1 -- 172.21.15.152:0/3666119543 >> 172.21.15.152:0/3666119543 conn(0x7f60280fc780 msgr2=0x7f60280feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:06.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.097+0000 7f602f7d4640 1 -- 172.21.15.152:0/3666119543 shutdown_connections 2024-09-11T22:53:06.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.097+0000 7f602f7d4640 1 -- 172.21.15.152:0/3666119543 wait complete. 2024-09-11T22:53:06.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.097+0000 7f602f7d4640 1 Processor -- start 2024-09-11T22:53:06.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.097+0000 7f602f7d4640 1 -- start start 2024-09-11T22:53:06.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602f7d4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6028100a60 0x7f60281970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:06.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602f7d4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6028101400 0x7f6028197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:06.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602f7d4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6028106a60 0x7f6028197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:06.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602f7d4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6028110e80 con 0x7f6028101400 2024-09-11T22:53:06.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602f7d4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6028110d00 con 0x7f6028106a60 2024-09-11T22:53:06.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602f7d4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6028111000 con 0x7f6028100a60 2024-09-11T22:53:06.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602cd6f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6028101400 0x7f6028197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:06.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602cd6f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6028101400 0x7f6028197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43842/0 (socket says 172.21.15.152:43842) 2024-09-11T22:53:06.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602cd6f640 1 -- 172.21.15.152:0/748554002 learned_addr learned my addr 172.21.15.152:0/748554002 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:06.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602dd71640 1 --2- 172.21.15.152:0/748554002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6028106a60 0x7f6028197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:06.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602d570640 1 --2- 172.21.15.152:0/748554002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6028100a60 0x7f60281970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:06.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602cd6f640 1 -- 172.21.15.152:0/748554002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6028100a60 msgr2=0x7f60281970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:06.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602cd6f640 1 --2- 172.21.15.152:0/748554002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6028100a60 0x7f60281970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:06.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602cd6f640 1 -- 172.21.15.152:0/748554002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6028106a60 msgr2=0x7f6028197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:06.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602cd6f640 1 --2- 172.21.15.152:0/748554002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6028106a60 0x7f6028197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:06.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602cd6f640 1 -- 172.21.15.152:0/748554002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60281a6930 con 0x7f6028101400 2024-09-11T22:53:06.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602d570640 1 --2- 172.21.15.152:0/748554002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6028100a60 0x7f60281970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:06.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602cd6f640 1 --2- 172.21.15.152:0/748554002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6028101400 0x7f6028197600 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6024009a50 tx=0x7f6024004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:06.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f601e7fc640 1 -- 172.21.15.152:0/748554002 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602403f030 con 0x7f6028101400 2024-09-11T22:53:06.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f602f7d4640 1 -- 172.21.15.152:0/748554002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60281a6bc0 con 0x7f6028101400 2024-09-11T22:53:06.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f601e7fc640 1 -- 172.21.15.152:0/748554002 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6024003ce0 con 0x7f6028101400 2024-09-11T22:53:06.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.098+0000 7f601e7fc640 1 -- 172.21.15.152:0/748554002 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024035070 con 0x7f6028101400 2024-09-11T22:53:06.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.099+0000 7f602f7d4640 1 -- 172.21.15.152:0/748554002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60281a70f0 con 0x7f6028101400 2024-09-11T22:53:06.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.099+0000 7f602f7d4640 1 -- 172.21.15.152:0/748554002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6028100ec0 con 0x7f6028101400 2024-09-11T22:53:06.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.101+0000 7f601e7fc640 1 -- 172.21.15.152:0/748554002 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f6024042030 con 0x7f6028101400 2024-09-11T22:53:06.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.102+0000 7f601e7fc640 1 --2- 172.21.15.152:0/748554002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f60140a2d80 0x7f60140a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:06.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.102+0000 7f602d570640 1 --2- 172.21.15.152:0/748554002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f60140a2d80 0x7f60140a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:06.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.102+0000 7f601e7fc640 1 -- 172.21.15.152:0/748554002 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f60240efbd0 con 0x7f6028101400 2024-09-11T22:53:06.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.102+0000 7f602d570640 1 --2- 172.21.15.152:0/748554002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f60140a2d80 0x7f60140a5240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f601800baa0 tx=0x7f6018005d90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:06.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.106+0000 7f601e7fc640 1 -- 172.21.15.152:0/748554002 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60240bc6d0 con 0x7f6028101400 2024-09-11T22:53:06.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.270+0000 7f602f7d4640 1 -- 172.21.15.152:0/748554002 --> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60280fd5f0 con 0x7f60140a2d80 2024-09-11T22:53:06.271 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:06 smithi152 ceph-mon[31181]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:53:06.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.276+0000 7f601e7fc640 1 -- 172.21.15.152:0/748554002 <== mgr.15240 v2:172.21.15.152:6800/4130474708 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258068 (secure 0 0 0) 0x7f60280fd5f0 con 0x7f60140a2d80 2024-09-11T22:53:06.279 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:53:06.280 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-09-11T22:53:06.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.279+0000 7f602f7d4640 1 -- 172.21.15.152:0/748554002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f60140a2d80 msgr2=0x7f60140a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:06.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.279+0000 7f602f7d4640 1 --2- 172.21.15.152:0/748554002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f60140a2d80 0x7f60140a5240 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f601800baa0 tx=0x7f6018005d90 comp rx=0 tx=0).stop 2024-09-11T22:53:06.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.279+0000 7f602f7d4640 1 -- 172.21.15.152:0/748554002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6028101400 msgr2=0x7f6028197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:06.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.279+0000 7f602f7d4640 1 --2- 172.21.15.152:0/748554002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6028101400 0x7f6028197600 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6024009a50 tx=0x7f6024004270 comp rx=0 tx=0).stop 2024-09-11T22:53:06.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.280+0000 7f602f7d4640 1 -- 172.21.15.152:0/748554002 shutdown_connections 2024-09-11T22:53:06.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.280+0000 7f602f7d4640 1 --2- 172.21.15.152:0/748554002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f60140a2d80 0x7f60140a5240 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:06.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.280+0000 7f602f7d4640 1 --2- 172.21.15.152:0/748554002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6028106a60 0x7f6028197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:06.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.280+0000 7f602f7d4640 1 --2- 172.21.15.152:0/748554002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6028101400 0x7f6028197600 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:06.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.280+0000 7f602f7d4640 1 --2- 172.21.15.152:0/748554002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6028100a60 0x7f60281970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:06.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.280+0000 7f602f7d4640 1 -- 172.21.15.152:0/748554002 >> 172.21.15.152:0/748554002 conn(0x7f60280fc780 msgr2=0x7f60280fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:06.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.280+0000 7f602f7d4640 1 -- 172.21.15.152:0/748554002 shutdown_connections 2024-09-11T22:53:06.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:06.280+0000 7f602f7d4640 1 -- 172.21.15.152:0/748554002 wait complete. 2024-09-11T22:53:06.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:06 smithi191 ceph-mon[35211]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:53:06.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:06 smithi204 ceph-mon[35454]: from='client.15438 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:53:07.096 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-11T22:53:05.088538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1390049,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1822,"num_read_kb":1344,"num_write":711,"num_write_kb":5324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1036,"ondisk_log_size":1036,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369164,"kb_used_data":11556,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572660,"statfs":{"total":1151940427776,"available":1151562403840,"internally_reserved":0,"allocated":11833344,"data_stored":9512319,"data_compressed":245790,"data_compressed_allocated":319488,"data_compressed_original":3969024,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":459,"num_read_kb":496,"num_write":150,"num_write_kb":916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.974787"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169180+0000","last_change":"2024-09-11T22:50:54.171410+0000","last_active":"2024-09-11T22:52:53.169180+0000","last_peered":"2024-09-11T22:52:53.169180+0000","last_clean":"2024-09-11T22:52:53.169180+0000","last_became_active":"2024-09-11T22:50:54.171055+0000","last_became_peered":"2024-09-11T22:50:54.171055+0000","last_unstale":"2024-09-11T22:52:53.169180+0000","last_undegraded":"2024-09-11T22:52:53.169180+0000","last_fullsized":"2024-09-11T22:52:53.169180+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:54.767454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131374+0000","last_change":"2024-09-11T22:50:54.168534+0000","last_active":"2024-09-11T22:52:53.131374+0000","last_peered":"2024-09-11T22:52:53.131374+0000","last_clean":"2024-09-11T22:52:53.131374+0000","last_became_active":"2024-09-11T22:50:54.168228+0000","last_became_peered":"2024-09-11T22:50:54.168228+0000","last_unstale":"2024-09-11T22:52:53.131374+0000","last_undegraded":"2024-09-11T22:52:53.131374+0000","last_fullsized":"2024-09-11T22:52:53.131374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:11.380048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131174+0000","last_change":"2024-09-11T22:50:54.168975+0000","last_active":"2024-09-11T22:52:53.131174+0000","last_peered":"2024-09-11T22:52:53.131174+0000","last_clean":"2024-09-11T22:52:53.131174+0000","last_became_active":"2024-09-11T22:50:54.168676+0000","last_became_peered":"2024-09-11T22:50:54.168676+0000","last_unstale":"2024-09-11T22:52:53.131174+0000","last_undegraded":"2024-09-11T22:52:53.131174+0000","last_fullsized":"2024-09-11T22:52:53.131174+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:15.848273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130100+0000","last_change":"2024-09-11T22:50:54.164164+0000","last_active":"2024-09-11T22:52:53.130100+0000","last_peered":"2024-09-11T22:52:53.130100+0000","last_clean":"2024-09-11T22:52:53.130100+0000","last_became_active":"2024-09-11T22:50:54.163805+0000","last_became_peered":"2024-09-11T22:50:54.163805+0000","last_unstale":"2024-09-11T22:52:53.130100+0000","last_undegraded":"2024-09-11T22:52:53.130100+0000","last_fullsized":"2024-09-11T22:52:53.130100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:54.954046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074671+0000","last_change":"2024-09-11T22:50:54.167289+0000","last_active":"2024-09-11T22:52:53.074671+0000","last_peered":"2024-09-11T22:52:53.074671+0000","last_clean":"2024-09-11T22:52:53.074671+0000","last_became_active":"2024-09-11T22:50:54.166958+0000","last_became_peered":"2024-09-11T22:50:54.166958+0000","last_unstale":"2024-09-11T22:52:53.074671+0000","last_undegraded":"2024-09-11T22:52:53.074671+0000","last_fullsized":"2024-09-11T22:52:53.074671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:25.261241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129676+0000","last_change":"2024-09-11T22:50:54.175605+0000","last_active":"2024-09-11T22:52:53.129676+0000","last_peered":"2024-09-11T22:52:53.129676+0000","last_clean":"2024-09-11T22:52:53.129676+0000","last_became_active":"2024-09-11T22:50:54.175207+0000","last_became_peered":"2024-09-11T22:50:54.175207+0000","last_unstale":"2024-09-11T22:52:53.129676+0000","last_undegraded":"2024-09-11T22:52:53.129676+0000","last_fullsized":"2024-09-11T22:52:53.129676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:47.008910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129084+0000","last_change":"2024-09-11T22:50:54.167930+0000","last_active":"2024-09-11T22:52:53.129084+0000","last_peered":"2024-09-11T22:52:53.129084+0000","last_clean":"2024-09-11T22:52:53.129084+0000","last_became_active":"2024-09-11T22:50:54.167082+0000","last_became_peered":"2024-09-11T22:50:54.167082+0000","last_unstale":"2024-09-11T22:52:53.129084+0000","last_undegraded":"2024-09-11T22:52:53.129084+0000","last_fullsized":"2024-09-11T22:52:53.129084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:50.931311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129336+0000","last_change":"2024-09-11T22:50:54.172528+0000","last_active":"2024-09-11T22:52:53.129336+0000","last_peered":"2024-09-11T22:52:53.129336+0000","last_clean":"2024-09-11T22:52:53.129336+0000","last_became_active":"2024-09-11T22:50:54.172257+0000","last_became_peered":"2024-09-11T22:50:54.172257+0000","last_unstale":"2024-09-11T22:52:53.129336+0000","last_undegraded":"2024-09-11T22:52:53.129336+0000","last_fullsized":"2024-09-11T22:52:53.129336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:13:43.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074665+0000","last_change":"2024-09-11T22:50:54.163357+0000","last_active":"2024-09-11T22:52:53.074665+0000","last_peered":"2024-09-11T22:52:53.074665+0000","last_clean":"2024-09-11T22:52:53.074665+0000","last_became_active":"2024-09-11T22:50:54.162943+0000","last_became_peered":"2024-09-11T22:50:54.162943+0000","last_unstale":"2024-09-11T22:52:53.074665+0000","last_undegraded":"2024-09-11T22:52:53.074665+0000","last_fullsized":"2024-09-11T22:52:53.074665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:57.055802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129448+0000","last_change":"2024-09-11T22:50:54.177773+0000","last_active":"2024-09-11T22:52:53.129448+0000","last_peered":"2024-09-11T22:52:53.129448+0000","last_clean":"2024-09-11T22:52:53.129448+0000","last_became_active":"2024-09-11T22:50:54.177388+0000","last_became_peered":"2024-09-11T22:50:54.177388+0000","last_unstale":"2024-09-11T22:52:53.129448+0000","last_undegraded":"2024-09-11T22:52:53.129448+0000","last_fullsized":"2024-09-11T22:52:53.129448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:36:44.625089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129792+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:52:53.129792+0000","last_peered":"2024-09-11T22:52:53.129792+0000","last_clean":"2024-09-11T22:52:53.129792+0000","last_became_active":"2024-09-11T22:50:54.171491+0000","last_became_peered":"2024-09-11T22:50:54.171491+0000","last_unstale":"2024-09-11T22:52:53.129792+0000","last_undegraded":"2024-09-11T22:52:53.129792+0000","last_fullsized":"2024-09-11T22:52:53.129792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.996640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130199+0000","last_change":"2024-09-11T22:50:54.171260+0000","last_active":"2024-09-11T22:52:53.130199+0000","last_peered":"2024-09-11T22:52:53.130199+0000","last_clean":"2024-09-11T22:52:53.130199+0000","last_became_active":"2024-09-11T22:50:54.170733+0000","last_became_peered":"2024-09-11T22:50:54.170733+0000","last_unstale":"2024-09-11T22:52:53.130199+0000","last_undegraded":"2024-09-11T22:52:53.130199+0000","last_fullsized":"2024-09-11T22:52:53.130199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:57.877280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129997+0000","last_change":"2024-09-11T22:50:54.170576+0000","last_active":"2024-09-11T22:52:53.129997+0000","last_peered":"2024-09-11T22:52:53.129997+0000","last_clean":"2024-09-11T22:52:53.129997+0000","last_became_active":"2024-09-11T22:50:54.170239+0000","last_became_peered":"2024-09-11T22:50:54.170239+0000","last_unstale":"2024-09-11T22:52:53.129997+0000","last_undegraded":"2024-09-11T22:52:53.129997+0000","last_fullsized":"2024-09-11T22:52:53.129997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:09.683171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130651+0000","last_change":"2024-09-11T22:50:54.167176+0000","last_active":"2024-09-11T22:52:53.130651+0000","last_peered":"2024-09-11T22:52:53.130651+0000","last_clean":"2024-09-11T22:52:53.130651+0000","last_became_active":"2024-09-11T22:50:54.166894+0000","last_became_peered":"2024-09-11T22:50:54.166894+0000","last_unstale":"2024-09-11T22:52:53.130651+0000","last_undegraded":"2024-09-11T22:52:53.130651+0000","last_fullsized":"2024-09-11T22:52:53.130651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:43:08.158786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130901+0000","last_change":"2024-09-11T22:50:54.163100+0000","last_active":"2024-09-11T22:52:53.130901+0000","last_peered":"2024-09-11T22:52:53.130901+0000","last_clean":"2024-09-11T22:52:53.130901+0000","last_became_active":"2024-09-11T22:50:54.162776+0000","last_became_peered":"2024-09-11T22:50:54.162776+0000","last_unstale":"2024-09-11T22:52:53.130901+0000","last_undegraded":"2024-09-11T22:52:53.130901+0000","last_fullsized":"2024-09-11T22:52:53.130901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:15.215305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074034+0000","last_change":"2024-09-11T22:50:54.166013+0000","last_active":"2024-09-11T22:52:53.074034+0000","last_peered":"2024-09-11T22:52:53.074034+0000","last_clean":"2024-09-11T22:52:53.074034+0000","last_became_active":"2024-09-11T22:50:54.165680+0000","last_became_peered":"2024-09-11T22:50:54.165680+0000","last_unstale":"2024-09-11T22:52:53.074034+0000","last_undegraded":"2024-09-11T22:52:53.074034+0000","last_fullsized":"2024-09-11T22:52:53.074034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:12.192231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169175+0000","last_change":"2024-09-11T22:50:54.172105+0000","last_active":"2024-09-11T22:52:53.169175+0000","last_peered":"2024-09-11T22:52:53.169175+0000","last_clean":"2024-09-11T22:52:53.169175+0000","last_became_active":"2024-09-11T22:50:54.171782+0000","last_became_peered":"2024-09-11T22:50:54.171782+0000","last_unstale":"2024-09-11T22:52:53.169175+0000","last_undegraded":"2024-09-11T22:52:53.169175+0000","last_fullsized":"2024-09-11T22:52:53.169175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:17.152834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130880+0000","last_change":"2024-09-11T22:50:54.167184+0000","last_active":"2024-09-11T22:52:53.130880+0000","last_peered":"2024-09-11T22:52:53.130880+0000","last_clean":"2024-09-11T22:52:53.130880+0000","last_became_active":"2024-09-11T22:50:54.166878+0000","last_became_peered":"2024-09-11T22:50:54.166878+0000","last_unstale":"2024-09-11T22:52:53.130880+0000","last_undegraded":"2024-09-11T22:52:53.130880+0000","last_fullsized":"2024-09-11T22:52:53.130880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:44.046552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129108+0000","last_change":"2024-09-11T22:50:54.176260+0000","last_active":"2024-09-11T22:52:53.129108+0000","last_peered":"2024-09-11T22:52:53.129108+0000","last_clean":"2024-09-11T22:52:53.129108+0000","last_became_active":"2024-09-11T22:50:54.175873+0000","last_became_peered":"2024-09-11T22:50:54.175873+0000","last_unstale":"2024-09-11T22:52:53.129108+0000","last_undegraded":"2024-09-11T22:52:53.129108+0000","last_fullsized":"2024-09-11T22:52:53.129108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:05.258677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168910+0000","last_change":"2024-09-11T22:50:54.172725+0000","last_active":"2024-09-11T22:52:53.168910+0000","last_peered":"2024-09-11T22:52:53.168910+0000","last_clean":"2024-09-11T22:52:53.168910+0000","last_became_active":"2024-09-11T22:50:54.172072+0000","last_became_peered":"2024-09-11T22:50:54.172072+0000","last_unstale":"2024-09-11T22:52:53.168910+0000","last_undegraded":"2024-09-11T22:52:53.168910+0000","last_fullsized":"2024-09-11T22:52:53.168910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:53.699670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130148+0000","last_change":"2024-09-11T22:50:54.165118+0000","last_active":"2024-09-11T22:52:53.130148+0000","last_peered":"2024-09-11T22:52:53.130148+0000","last_clean":"2024-09-11T22:52:53.130148+0000","last_became_active":"2024-09-11T22:50:54.164793+0000","last_became_peered":"2024-09-11T22:50:54.164793+0000","last_unstale":"2024-09-11T22:52:53.130148+0000","last_undegraded":"2024-09-11T22:52:53.130148+0000","last_fullsized":"2024-09-11T22:52:53.130148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:42.883940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074224+0000","last_change":"2024-09-11T22:50:54.176900+0000","last_active":"2024-09-11T22:52:53.074224+0000","last_peered":"2024-09-11T22:52:53.074224+0000","last_clean":"2024-09-11T22:52:53.074224+0000","last_became_active":"2024-09-11T22:50:54.176595+0000","last_became_peered":"2024-09-11T22:50:54.176595+0000","last_unstale":"2024-09-11T22:52:53.074224+0000","last_undegraded":"2024-09-11T22:52:53.074224+0000","last_fullsized":"2024-09-11T22:52:53.074224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:36.925945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129485+0000","last_change":"2024-09-11T22:50:54.167656+0000","last_active":"2024-09-11T22:52:53.129485+0000","last_peered":"2024-09-11T22:52:53.129485+0000","last_clean":"2024-09-11T22:52:53.129485+0000","last_became_active":"2024-09-11T22:50:54.167349+0000","last_became_peered":"2024-09-11T22:50:54.167349+0000","last_unstale":"2024-09-11T22:52:53.129485+0000","last_undegraded":"2024-09-11T22:52:53.129485+0000","last_fullsized":"2024-09-11T22:52:53.129485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:38.100218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073470+0000","last_change":"2024-09-11T22:50:54.177059+0000","last_active":"2024-09-11T22:52:53.073470+0000","last_peered":"2024-09-11T22:52:53.073470+0000","last_clean":"2024-09-11T22:52:53.073470+0000","last_became_active":"2024-09-11T22:50:54.176726+0000","last_became_peered":"2024-09-11T22:50:54.176726+0000","last_unstale":"2024-09-11T22:52:53.073470+0000","last_undegraded":"2024-09-11T22:52:53.073470+0000","last_fullsized":"2024-09-11T22:52:53.073470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:27:58.058944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130622+0000","last_change":"2024-09-11T22:50:54.167566+0000","last_active":"2024-09-11T22:52:53.130622+0000","last_peered":"2024-09-11T22:52:53.130622+0000","last_clean":"2024-09-11T22:52:53.130622+0000","last_became_active":"2024-09-11T22:50:54.167049+0000","last_became_peered":"2024-09-11T22:50:54.167049+0000","last_unstale":"2024-09-11T22:52:53.130622+0000","last_undegraded":"2024-09-11T22:52:53.130622+0000","last_fullsized":"2024-09-11T22:52:53.130622+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:06.833099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129228+0000","last_change":"2024-09-11T22:50:54.169732+0000","last_active":"2024-09-11T22:52:53.129228+0000","last_peered":"2024-09-11T22:52:53.129228+0000","last_clean":"2024-09-11T22:52:53.129228+0000","last_became_active":"2024-09-11T22:50:54.169437+0000","last_became_peered":"2024-09-11T22:50:54.169437+0000","last_unstale":"2024-09-11T22:52:53.129228+0000","last_undegraded":"2024-09-11T22:52:53.129228+0000","last_fullsized":"2024-09-11T22:52:53.129228+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:25.553387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169167+0000","last_change":"2024-09-11T22:50:54.166953+0000","last_active":"2024-09-11T22:52:53.169167+0000","last_peered":"2024-09-11T22:52:53.169167+0000","last_clean":"2024-09-11T22:52:53.169167+0000","last_became_active":"2024-09-11T22:50:54.166472+0000","last_became_peered":"2024-09-11T22:50:54.166472+0000","last_unstale":"2024-09-11T22:52:53.169167+0000","last_undegraded":"2024-09-11T22:52:53.169167+0000","last_fullsized":"2024-09-11T22:52:53.169167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:01:20.234180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129475+0000","last_change":"2024-09-11T22:50:54.171308+0000","last_active":"2024-09-11T22:52:53.129475+0000","last_peered":"2024-09-11T22:52:53.129475+0000","last_clean":"2024-09-11T22:52:53.129475+0000","last_became_active":"2024-09-11T22:50:54.168958+0000","last_became_peered":"2024-09-11T22:50:54.168958+0000","last_unstale":"2024-09-11T22:52:53.129475+0000","last_undegraded":"2024-09-11T22:52:53.129475+0000","last_fullsized":"2024-09-11T22:52:53.129475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:20.224003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131226+0000","last_change":"2024-09-11T22:50:54.173846+0000","last_active":"2024-09-11T22:52:53.131226+0000","last_peered":"2024-09-11T22:52:53.131226+0000","last_clean":"2024-09-11T22:52:53.131226+0000","last_became_active":"2024-09-11T22:50:54.173542+0000","last_became_peered":"2024-09-11T22:50:54.173542+0000","last_unstale":"2024-09-11T22:52:53.131226+0000","last_undegraded":"2024-09-11T22:52:53.131226+0000","last_fullsized":"2024-09-11T22:52:53.131226+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:29:45.775902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129000+0000","last_change":"2024-09-11T22:50:54.176008+0000","last_active":"2024-09-11T22:52:53.129000+0000","last_peered":"2024-09-11T22:52:53.129000+0000","last_clean":"2024-09-11T22:52:53.129000+0000","last_became_active":"2024-09-11T22:50:54.175672+0000","last_became_peered":"2024-09-11T22:50:54.175672+0000","last_unstale":"2024-09-11T22:52:53.129000+0000","last_undegraded":"2024-09-11T22:52:53.129000+0000","last_fullsized":"2024-09-11T22:52:53.129000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:35.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129006+0000","last_change":"2024-09-11T22:50:54.162504+0000","last_active":"2024-09-11T22:52:53.129006+0000","last_peered":"2024-09-11T22:52:53.129006+0000","last_clean":"2024-09-11T22:52:53.129006+0000","last_became_active":"2024-09-11T22:50:54.162190+0000","last_became_peered":"2024-09-11T22:50:54.162190+0000","last_unstale":"2024-09-11T22:52:53.129006+0000","last_undegraded":"2024-09-11T22:52:53.129006+0000","last_fullsized":"2024-09-11T22:52:53.129006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:52.138692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168887+0000","last_change":"2024-09-11T22:51:55.960882+0000","last_active":"2024-09-11T22:52:53.168887+0000","last_peered":"2024-09-11T22:52:53.168887+0000","last_clean":"2024-09-11T22:52:53.168887+0000","last_became_active":"2024-09-11T22:51:55.960246+0000","last_became_peered":"2024-09-11T22:51:55.960246+0000","last_unstale":"2024-09-11T22:52:53.168887+0000","last_undegraded":"2024-09-11T22:52:53.168887+0000","last_fullsized":"2024-09-11T22:52:53.168887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:21.247840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074402+0000","last_change":"2024-09-11T22:50:54.160696+0000","last_active":"2024-09-11T22:52:53.074402+0000","last_peered":"2024-09-11T22:52:53.074402+0000","last_clean":"2024-09-11T22:52:53.074402+0000","last_became_active":"2024-09-11T22:50:54.159249+0000","last_became_peered":"2024-09-11T22:50:54.159249+0000","last_unstale":"2024-09-11T22:52:53.074402+0000","last_undegraded":"2024-09-11T22:52:53.074402+0000","last_fullsized":"2024-09-11T22:52:53.074402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:09.649208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'9","reported_seq":157,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:02.292263+0000","last_change":"2024-09-11T22:50:52.138986+0000","last_active":"2024-09-11T22:53:02.292263+0000","last_peered":"2024-09-11T22:53:02.292263+0000","last_clean":"2024-09-11T22:53:02.292263+0000","last_became_active":"2024-09-11T22:50:52.138595+0000","last_became_peered":"2024-09-11T22:50:52.138595+0000","last_unstale":"2024-09-11T22:53:02.292263+0000","last_undegraded":"2024-09-11T22:53:02.292263+0000","last_fullsized":"2024-09-11T22:53:02.292263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:06.232146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1942,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073853+0000","last_change":"2024-09-11T22:50:54.176806+0000","last_active":"2024-09-11T22:52:53.073853+0000","last_peered":"2024-09-11T22:52:53.073853+0000","last_clean":"2024-09-11T22:52:53.073853+0000","last_became_active":"2024-09-11T22:50:54.176513+0000","last_became_peered":"2024-09-11T22:50:54.176513+0000","last_unstale":"2024-09-11T22:52:53.073853+0000","last_undegraded":"2024-09-11T22:52:53.073853+0000","last_fullsized":"2024-09-11T22:52:53.073853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:43:12.086145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129699+0000","last_change":"2024-09-11T22:50:52.142794+0000","last_active":"2024-09-11T22:52:53.129699+0000","last_peered":"2024-09-11T22:52:53.129699+0000","last_clean":"2024-09-11T22:52:53.129699+0000","last_became_active":"2024-09-11T22:50:52.142433+0000","last_became_peered":"2024-09-11T22:50:52.142433+0000","last_unstale":"2024-09-11T22:52:53.129699+0000","last_undegraded":"2024-09-11T22:52:53.129699+0000","last_fullsized":"2024-09-11T22:52:53.129699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:47:12.090055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130326+0000","last_change":"2024-09-11T22:50:52.139789+0000","last_active":"2024-09-11T22:52:53.130326+0000","last_peered":"2024-09-11T22:52:53.130326+0000","last_clean":"2024-09-11T22:52:53.130326+0000","last_became_active":"2024-09-11T22:50:52.139294+0000","last_became_peered":"2024-09-11T22:50:52.139294+0000","last_unstale":"2024-09-11T22:52:53.130326+0000","last_undegraded":"2024-09-11T22:52:53.130326+0000","last_fullsized":"2024-09-11T22:52:53.130326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:07.355155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129486+0000","last_change":"2024-09-11T22:50:54.166248+0000","last_active":"2024-09-11T22:52:53.129486+0000","last_peered":"2024-09-11T22:52:53.129486+0000","last_clean":"2024-09-11T22:52:53.129486+0000","last_became_active":"2024-09-11T22:50:54.165958+0000","last_became_peered":"2024-09-11T22:50:54.165958+0000","last_unstale":"2024-09-11T22:52:53.129486+0000","last_undegraded":"2024-09-11T22:52:53.129486+0000","last_fullsized":"2024-09-11T22:52:53.129486+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:29.531741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073597+0000","last_change":"2024-09-11T22:50:52.140340+0000","last_active":"2024-09-11T22:52:53.073597+0000","last_peered":"2024-09-11T22:52:53.073597+0000","last_clean":"2024-09-11T22:52:53.073597+0000","last_became_active":"2024-09-11T22:50:52.140011+0000","last_became_peered":"2024-09-11T22:50:52.140011+0000","last_unstale":"2024-09-11T22:52:53.073597+0000","last_undegraded":"2024-09-11T22:52:53.073597+0000","last_fullsized":"2024-09-11T22:52:53.073597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:58.862640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129625+0000","last_change":"2024-09-11T22:50:54.166998+0000","last_active":"2024-09-11T22:52:53.129625+0000","last_peered":"2024-09-11T22:52:53.129625+0000","last_clean":"2024-09-11T22:52:53.129625+0000","last_became_active":"2024-09-11T22:50:54.166682+0000","last_became_peered":"2024-09-11T22:50:54.166682+0000","last_unstale":"2024-09-11T22:52:53.129625+0000","last_undegraded":"2024-09-11T22:52:53.129625+0000","last_fullsized":"2024-09-11T22:52:53.129625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:57.815914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'8","reported_seq":270,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:03.292945+0000","last_change":"2024-09-11T22:50:52.141380+0000","last_active":"2024-09-11T22:53:03.292945+0000","last_peered":"2024-09-11T22:53:03.292945+0000","last_clean":"2024-09-11T22:53:03.292945+0000","last_became_active":"2024-09-11T22:50:52.139925+0000","last_became_peered":"2024-09-11T22:50:52.139925+0000","last_unstale":"2024-09-11T22:53:03.292945+0000","last_undegraded":"2024-09-11T22:53:03.292945+0000","last_fullsized":"2024-09-11T22:53:03.292945+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:40:38.395055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":238,"num_read_kb":238,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074552+0000","last_change":"2024-09-11T22:50:54.167909+0000","last_active":"2024-09-11T22:52:53.074552+0000","last_peered":"2024-09-11T22:52:53.074552+0000","last_clean":"2024-09-11T22:52:53.074552+0000","last_became_active":"2024-09-11T22:50:54.167571+0000","last_became_peered":"2024-09-11T22:50:54.167571+0000","last_unstale":"2024-09-11T22:52:53.074552+0000","last_undegraded":"2024-09-11T22:52:53.074552+0000","last_fullsized":"2024-09-11T22:52:53.074552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:34.179960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129913+0000","last_change":"2024-09-11T22:50:52.144545+0000","last_active":"2024-09-11T22:52:53.129913+0000","last_peered":"2024-09-11T22:52:53.129913+0000","last_clean":"2024-09-11T22:52:53.129913+0000","last_became_active":"2024-09-11T22:50:52.144221+0000","last_became_peered":"2024-09-11T22:50:52.144221+0000","last_unstale":"2024-09-11T22:52:53.129913+0000","last_undegraded":"2024-09-11T22:52:53.129913+0000","last_fullsized":"2024-09-11T22:52:53.129913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:29.509165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131618+0000","last_change":"2024-09-11T22:50:54.164132+0000","last_active":"2024-09-11T22:52:53.131618+0000","last_peered":"2024-09-11T22:52:53.131618+0000","last_clean":"2024-09-11T22:52:53.131618+0000","last_became_active":"2024-09-11T22:50:54.163842+0000","last_became_peered":"2024-09-11T22:50:54.163842+0000","last_unstale":"2024-09-11T22:52:53.131618+0000","last_undegraded":"2024-09-11T22:52:53.131618+0000","last_fullsized":"2024-09-11T22:52:53.131618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:28:55.490236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074349+0000","last_change":"2024-09-11T22:50:52.264611+0000","last_active":"2024-09-11T22:52:53.074349+0000","last_peered":"2024-09-11T22:52:53.074349+0000","last_clean":"2024-09-11T22:52:53.074349+0000","last_became_active":"2024-09-11T22:50:52.264060+0000","last_became_peered":"2024-09-11T22:50:52.264060+0000","last_unstale":"2024-09-11T22:52:53.074349+0000","last_undegraded":"2024-09-11T22:52:53.074349+0000","last_fullsized":"2024-09-11T22:52:53.074349+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:42.543997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074271+0000","last_change":"2024-09-11T22:51:56.206155+0000","last_active":"2024-09-11T22:52:53.074271+0000","last_peered":"2024-09-11T22:52:53.074271+0000","last_clean":"2024-09-11T22:52:53.074271+0000","last_became_active":"2024-09-11T22:51:56.205797+0000","last_became_peered":"2024-09-11T22:51:56.205797+0000","last_unstale":"2024-09-11T22:52:53.074271+0000","last_undegraded":"2024-09-11T22:52:53.074271+0000","last_fullsized":"2024-09-11T22:52:53.074271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:28.063825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'39","reported_seq":315,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:03.293288+0000","last_change":"2024-09-11T22:50:52.141782+0000","last_active":"2024-09-11T22:53:03.293288+0000","last_peered":"2024-09-11T22:53:03.293288+0000","last_clean":"2024-09-11T22:53:03.293288+0000","last_became_active":"2024-09-11T22:50:52.140662+0000","last_became_peered":"2024-09-11T22:50:52.140662+0000","last_unstale":"2024-09-11T22:53:03.293288+0000","last_undegraded":"2024-09-11T22:53:03.293288+0000","last_fullsized":"2024-09-11T22:53:03.293288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:21.158577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68544,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":252,"num_read_kb":192,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074760+0000","last_change":"2024-09-11T22:50:54.166678+0000","last_active":"2024-09-11T22:52:53.074760+0000","last_peered":"2024-09-11T22:52:53.074760+0000","last_clean":"2024-09-11T22:52:53.074760+0000","last_became_active":"2024-09-11T22:50:54.165777+0000","last_became_peered":"2024-09-11T22:50:54.165777+0000","last_unstale":"2024-09-11T22:52:53.074760+0000","last_undegraded":"2024-09-11T22:52:53.074760+0000","last_fullsized":"2024-09-11T22:52:53.074760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:07.224563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129051+0000","last_change":"2024-09-11T22:50:52.141292+0000","last_active":"2024-09-11T22:52:53.129051+0000","last_peered":"2024-09-11T22:52:53.129051+0000","last_clean":"2024-09-11T22:52:53.129051+0000","last_became_active":"2024-09-11T22:50:52.140971+0000","last_became_peered":"2024-09-11T22:50:52.140971+0000","last_unstale":"2024-09-11T22:52:53.129051+0000","last_undegraded":"2024-09-11T22:52:53.129051+0000","last_fullsized":"2024-09-11T22:52:53.129051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:53.039684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129489+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:52:53.129489+0000","last_peered":"2024-09-11T22:52:53.129489+0000","last_clean":"2024-09-11T22:52:53.129489+0000","last_became_active":"2024-09-11T22:50:54.171520+0000","last_became_peered":"2024-09-11T22:50:54.171520+0000","last_unstale":"2024-09-11T22:52:53.129489+0000","last_undegraded":"2024-09-11T22:52:53.129489+0000","last_fullsized":"2024-09-11T22:52:53.129489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:56.594058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073605+0000","last_change":"2024-09-11T22:50:52.138725+0000","last_active":"2024-09-11T22:52:53.073605+0000","last_peered":"2024-09-11T22:52:53.073605+0000","last_clean":"2024-09-11T22:52:53.073605+0000","last_became_active":"2024-09-11T22:50:52.138367+0000","last_became_peered":"2024-09-11T22:50:52.138367+0000","last_unstale":"2024-09-11T22:52:53.073605+0000","last_undegraded":"2024-09-11T22:52:53.073605+0000","last_fullsized":"2024-09-11T22:52:53.073605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:58.811444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129217+0000","last_change":"2024-09-11T22:50:54.170524+0000","last_active":"2024-09-11T22:52:53.129217+0000","last_peered":"2024-09-11T22:52:53.129217+0000","last_clean":"2024-09-11T22:52:53.129217+0000","last_became_active":"2024-09-11T22:50:54.169947+0000","last_became_peered":"2024-09-11T22:50:54.169947+0000","last_unstale":"2024-09-11T22:52:53.129217+0000","last_undegraded":"2024-09-11T22:52:53.129217+0000","last_fullsized":"2024-09-11T22:52:53.129217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:51.768718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":153,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:02.351154+0000","last_change":"2024-09-11T22:50:52.139182+0000","last_active":"2024-09-11T22:53:02.351154+0000","last_peered":"2024-09-11T22:53:02.351154+0000","last_clean":"2024-09-11T22:53:02.351154+0000","last_became_active":"2024-09-11T22:50:52.138799+0000","last_became_peered":"2024-09-11T22:50:52.138799+0000","last_unstale":"2024-09-11T22:53:02.351154+0000","last_undegraded":"2024-09-11T22:53:02.351154+0000","last_fullsized":"2024-09-11T22:53:02.351154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:36.376162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129122+0000","last_change":"2024-09-11T22:50:54.168797+0000","last_active":"2024-09-11T22:52:53.129122+0000","last_peered":"2024-09-11T22:52:53.129122+0000","last_clean":"2024-09-11T22:52:53.129122+0000","last_became_active":"2024-09-11T22:50:54.168408+0000","last_became_peered":"2024-09-11T22:50:54.168408+0000","last_unstale":"2024-09-11T22:52:53.129122+0000","last_undegraded":"2024-09-11T22:52:53.129122+0000","last_fullsized":"2024-09-11T22:52:53.129122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:51.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074207+0000","last_change":"2024-09-11T22:50:52.264693+0000","last_active":"2024-09-11T22:52:53.074207+0000","last_peered":"2024-09-11T22:52:53.074207+0000","last_clean":"2024-09-11T22:52:53.074207+0000","last_became_active":"2024-09-11T22:50:52.264058+0000","last_became_peered":"2024-09-11T22:50:52.264058+0000","last_unstale":"2024-09-11T22:52:53.074207+0000","last_undegraded":"2024-09-11T22:52:53.074207+0000","last_fullsized":"2024-09-11T22:52:53.074207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:46.985921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073438+0000","last_change":"2024-09-11T22:51:55.961118+0000","last_active":"2024-09-11T22:52:53.073438+0000","last_peered":"2024-09-11T22:52:53.073438+0000","last_clean":"2024-09-11T22:52:53.073438+0000","last_became_active":"2024-09-11T22:51:55.960614+0000","last_became_peered":"2024-09-11T22:51:55.960614+0000","last_unstale":"2024-09-11T22:52:53.073438+0000","last_undegraded":"2024-09-11T22:52:53.073438+0000","last_fullsized":"2024-09-11T22:52:53.073438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:47.184327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.075083+0000","last_change":"2024-09-11T22:50:52.133049+0000","last_active":"2024-09-11T22:52:53.075083+0000","last_peered":"2024-09-11T22:52:53.075083+0000","last_clean":"2024-09-11T22:52:53.075083+0000","last_became_active":"2024-09-11T22:50:52.132647+0000","last_became_peered":"2024-09-11T22:50:52.132647+0000","last_unstale":"2024-09-11T22:52:53.075083+0000","last_undegraded":"2024-09-11T22:52:53.075083+0000","last_fullsized":"2024-09-11T22:52:53.075083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:13.213003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169115+0000","last_change":"2024-09-11T22:51:56.238023+0000","last_active":"2024-09-11T22:52:53.169115+0000","last_peered":"2024-09-11T22:52:53.169115+0000","last_clean":"2024-09-11T22:52:53.169115+0000","last_became_active":"2024-09-11T22:51:56.237676+0000","last_became_peered":"2024-09-11T22:51:56.237676+0000","last_unstale":"2024-09-11T22:52:53.169115+0000","last_undegraded":"2024-09-11T22:52:53.169115+0000","last_fullsized":"2024-09-11T22:52:53.169115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:16.215961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130047+0000","last_change":"2024-09-11T22:50:52.139766+0000","last_active":"2024-09-11T22:52:53.130047+0000","last_peered":"2024-09-11T22:52:53.130047+0000","last_clean":"2024-09-11T22:52:53.130047+0000","last_became_active":"2024-09-11T22:50:52.139456+0000","last_became_peered":"2024-09-11T22:50:52.139456+0000","last_unstale":"2024-09-11T22:52:53.130047+0000","last_undegraded":"2024-09-11T22:52:53.130047+0000","last_fullsized":"2024-09-11T22:52:53.130047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:08.358649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130080+0000","last_change":"2024-09-11T22:50:54.167717+0000","last_active":"2024-09-11T22:52:53.130080+0000","last_peered":"2024-09-11T22:52:53.130080+0000","last_clean":"2024-09-11T22:52:53.130080+0000","last_became_active":"2024-09-11T22:50:54.167331+0000","last_became_peered":"2024-09-11T22:50:54.167331+0000","last_unstale":"2024-09-11T22:52:53.130080+0000","last_undegraded":"2024-09-11T22:52:53.130080+0000","last_fullsized":"2024-09-11T22:52:53.130080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:11.240351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'67","reported_seq":122,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:02.748201+0000","last_change":"2024-09-11T22:50:52.141682+0000","last_active":"2024-09-11T22:53:02.748201+0000","last_peered":"2024-09-11T22:53:02.748201+0000","last_clean":"2024-09-11T22:53:02.748201+0000","last_became_active":"2024-09-11T22:50:52.141251+0000","last_became_peered":"2024-09-11T22:50:52.141251+0000","last_unstale":"2024-09-11T22:53:02.748201+0000","last_undegraded":"2024-09-11T22:53:02.748201+0000","last_fullsized":"2024-09-11T22:53:02.748201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:03.068252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129793+0000","last_change":"2024-09-11T22:50:54.174149+0000","last_active":"2024-09-11T22:52:53.129793+0000","last_peered":"2024-09-11T22:52:53.129793+0000","last_clean":"2024-09-11T22:52:53.129793+0000","last_became_active":"2024-09-11T22:50:54.173765+0000","last_became_peered":"2024-09-11T22:50:54.173765+0000","last_unstale":"2024-09-11T22:52:53.129793+0000","last_undegraded":"2024-09-11T22:52:53.129793+0000","last_fullsized":"2024-09-11T22:52:53.129793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.314334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'8","reported_seq":156,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:02.292200+0000","last_change":"2024-09-11T22:50:52.435019+0000","last_active":"2024-09-11T22:53:02.292200+0000","last_peered":"2024-09-11T22:53:02.292200+0000","last_clean":"2024-09-11T22:53:02.292200+0000","last_became_active":"2024-09-11T22:50:52.434642+0000","last_became_peered":"2024-09-11T22:50:52.434642+0000","last_unstale":"2024-09-11T22:53:02.292200+0000","last_undegraded":"2024-09-11T22:53:02.292200+0000","last_fullsized":"2024-09-11T22:53:02.292200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:18.441841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130004+0000","last_change":"2024-09-11T22:50:54.171291+0000","last_active":"2024-09-11T22:52:53.130004+0000","last_peered":"2024-09-11T22:52:53.130004+0000","last_clean":"2024-09-11T22:52:53.130004+0000","last_became_active":"2024-09-11T22:50:54.170961+0000","last_became_peered":"2024-09-11T22:50:54.170961+0000","last_unstale":"2024-09-11T22:52:53.130004+0000","last_undegraded":"2024-09-11T22:52:53.130004+0000","last_fullsized":"2024-09-11T22:52:53.130004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:07.211936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"84'148","reported_seq":392,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:55.179683+0000","last_change":"2024-09-11T22:49:39.098881+0000","last_active":"2024-09-11T22:52:55.179683+0000","last_peered":"2024-09-11T22:52:55.179683+0000","last_clean":"2024-09-11T22:52:55.179683+0000","last_became_active":"2024-09-11T22:49:39.098458+0000","last_became_peered":"2024-09-11T22:49:39.098458+0000","last_unstale":"2024-09-11T22:52:55.179683+0000","last_undegraded":"2024-09-11T22:52:55.179683+0000","last_fullsized":"2024-09-11T22:52:55.179683+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_clean_scrub_stamp":"2024-09-11T22:46:11.418577+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:41.059384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130085+0000","last_change":"2024-09-11T22:50:52.137691+0000","last_active":"2024-09-11T22:52:53.130085+0000","last_peered":"2024-09-11T22:52:53.130085+0000","last_clean":"2024-09-11T22:52:53.130085+0000","last_became_active":"2024-09-11T22:50:52.137282+0000","last_became_peered":"2024-09-11T22:50:52.137282+0000","last_unstale":"2024-09-11T22:52:53.130085+0000","last_undegraded":"2024-09-11T22:52:53.130085+0000","last_fullsized":"2024-09-11T22:52:53.130085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:00:27.651252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129789+0000","last_change":"2024-09-11T22:51:55.959937+0000","last_active":"2024-09-11T22:52:53.129789+0000","last_peered":"2024-09-11T22:52:53.129789+0000","last_clean":"2024-09-11T22:52:53.129789+0000","last_became_active":"2024-09-11T22:51:55.959575+0000","last_became_peered":"2024-09-11T22:51:55.959575+0000","last_unstale":"2024-09-11T22:52:53.129789+0000","last_undegraded":"2024-09-11T22:52:53.129789+0000","last_fullsized":"2024-09-11T22:52:53.129789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:54.097552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129633+0000","last_change":"2024-09-11T22:50:52.144324+0000","last_active":"2024-09-11T22:52:53.129633+0000","last_peered":"2024-09-11T22:52:53.129633+0000","last_clean":"2024-09-11T22:52:53.129633+0000","last_became_active":"2024-09-11T22:50:52.144011+0000","last_became_peered":"2024-09-11T22:50:52.144011+0000","last_unstale":"2024-09-11T22:52:53.129633+0000","last_undegraded":"2024-09-11T22:52:53.129633+0000","last_fullsized":"2024-09-11T22:52:53.129633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:29.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130247+0000","last_change":"2024-09-11T22:50:54.173287+0000","last_active":"2024-09-11T22:52:53.130247+0000","last_peered":"2024-09-11T22:52:53.130247+0000","last_clean":"2024-09-11T22:52:53.130247+0000","last_became_active":"2024-09-11T22:50:54.172953+0000","last_became_peered":"2024-09-11T22:50:54.172953+0000","last_unstale":"2024-09-11T22:52:53.130247+0000","last_undegraded":"2024-09-11T22:52:53.130247+0000","last_fullsized":"2024-09-11T22:52:53.130247+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:24.314604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":151,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:02.350556+0000","last_change":"2024-09-11T22:50:52.263777+0000","last_active":"2024-09-11T22:53:02.350556+0000","last_peered":"2024-09-11T22:53:02.350556+0000","last_clean":"2024-09-11T22:53:02.350556+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:53:02.350556+0000","last_undegraded":"2024-09-11T22:53:02.350556+0000","last_fullsized":"2024-09-11T22:53:02.350556+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:37.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073458+0000","last_change":"2024-09-11T22:50:54.164927+0000","last_active":"2024-09-11T22:52:53.073458+0000","last_peered":"2024-09-11T22:52:53.073458+0000","last_clean":"2024-09-11T22:52:53.073458+0000","last_became_active":"2024-09-11T22:50:54.164602+0000","last_became_peered":"2024-09-11T22:50:54.164602+0000","last_unstale":"2024-09-11T22:52:53.073458+0000","last_undegraded":"2024-09-11T22:52:53.073458+0000","last_fullsized":"2024-09-11T22:52:53.073458+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:10.226232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":149,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:02.300304+0000","last_change":"2024-09-11T22:50:52.141199+0000","last_active":"2024-09-11T22:53:02.300304+0000","last_peered":"2024-09-11T22:53:02.300304+0000","last_clean":"2024-09-11T22:53:02.300304+0000","last_became_active":"2024-09-11T22:50:52.140830+0000","last_became_peered":"2024-09-11T22:50:52.140830+0000","last_unstale":"2024-09-11T22:53:02.300304+0000","last_undegraded":"2024-09-11T22:53:02.300304+0000","last_fullsized":"2024-09-11T22:53:02.300304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.767803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129616+0000","last_change":"2024-09-11T22:50:54.175054+0000","last_active":"2024-09-11T22:52:53.129616+0000","last_peered":"2024-09-11T22:52:53.129616+0000","last_clean":"2024-09-11T22:52:53.129616+0000","last_became_active":"2024-09-11T22:50:54.174715+0000","last_became_peered":"2024-09-11T22:50:54.174715+0000","last_unstale":"2024-09-11T22:52:53.129616+0000","last_undegraded":"2024-09-11T22:52:53.129616+0000","last_fullsized":"2024-09-11T22:52:53.129616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:06.694988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130259+0000","last_change":"2024-09-11T22:51:55.961660+0000","last_active":"2024-09-11T22:52:53.130259+0000","last_peered":"2024-09-11T22:52:53.130259+0000","last_clean":"2024-09-11T22:52:53.130259+0000","last_became_active":"2024-09-11T22:51:55.961278+0000","last_became_peered":"2024-09-11T22:51:55.961278+0000","last_unstale":"2024-09-11T22:52:53.130259+0000","last_undegraded":"2024-09-11T22:52:53.130259+0000","last_fullsized":"2024-09-11T22:52:53.130259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:52.352490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129410+0000","last_change":"2024-09-11T22:50:54.167361+0000","last_active":"2024-09-11T22:52:53.129410+0000","last_peered":"2024-09-11T22:52:53.129410+0000","last_clean":"2024-09-11T22:52:53.129410+0000","last_became_active":"2024-09-11T22:50:54.166963+0000","last_became_peered":"2024-09-11T22:50:54.166963+0000","last_unstale":"2024-09-11T22:52:53.129410+0000","last_undegraded":"2024-09-11T22:52:53.129410+0000","last_fullsized":"2024-09-11T22:52:53.129410+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:13.294544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.075075+0000","last_change":"2024-09-11T22:50:52.264484+0000","last_active":"2024-09-11T22:52:53.075075+0000","last_peered":"2024-09-11T22:52:53.075075+0000","last_clean":"2024-09-11T22:52:53.075075+0000","last_became_active":"2024-09-11T22:50:52.264038+0000","last_became_peered":"2024-09-11T22:50:52.264038+0000","last_unstale":"2024-09-11T22:52:53.075075+0000","last_undegraded":"2024-09-11T22:52:53.075075+0000","last_fullsized":"2024-09-11T22:52:53.075075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:21.325136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129602+0000","last_change":"2024-09-11T22:50:54.162225+0000","last_active":"2024-09-11T22:52:53.129602+0000","last_peered":"2024-09-11T22:52:53.129602+0000","last_clean":"2024-09-11T22:52:53.129602+0000","last_became_active":"2024-09-11T22:50:54.161234+0000","last_became_peered":"2024-09-11T22:50:54.161234+0000","last_unstale":"2024-09-11T22:52:53.129602+0000","last_undegraded":"2024-09-11T22:52:53.129602+0000","last_fullsized":"2024-09-11T22:52:53.129602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:00.346217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'105","reported_seq":238,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:03.106484+0000","last_change":"2024-09-11T22:51:55.967804+0000","last_active":"2024-09-11T22:53:03.106484+0000","last_peered":"2024-09-11T22:53:03.106484+0000","last_clean":"2024-09-11T22:53:03.106484+0000","last_became_active":"2024-09-11T22:51:55.961136+0000","last_became_peered":"2024-09-11T22:51:55.961136+0000","last_unstale":"2024-09-11T22:53:03.106484+0000","last_undegraded":"2024-09-11T22:53:03.106484+0000","last_fullsized":"2024-09-11T22:53:03.106484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:55.315179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":191,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130644+0000","last_change":"2024-09-11T22:50:54.167307+0000","last_active":"2024-09-11T22:52:53.130644+0000","last_peered":"2024-09-11T22:52:53.130644+0000","last_clean":"2024-09-11T22:52:53.130644+0000","last_became_active":"2024-09-11T22:50:54.166650+0000","last_became_peered":"2024-09-11T22:50:54.166650+0000","last_unstale":"2024-09-11T22:52:53.130644+0000","last_undegraded":"2024-09-11T22:52:53.130644+0000","last_fullsized":"2024-09-11T22:52:53.130644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:04.863258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130339+0000","last_change":"2024-09-11T22:50:52.435107+0000","last_active":"2024-09-11T22:52:53.130339+0000","last_peered":"2024-09-11T22:52:53.130339+0000","last_clean":"2024-09-11T22:52:53.130339+0000","last_became_active":"2024-09-11T22:50:52.434791+0000","last_became_peered":"2024-09-11T22:50:52.434791+0000","last_unstale":"2024-09-11T22:52:53.130339+0000","last_undegraded":"2024-09-11T22:52:53.130339+0000","last_fullsized":"2024-09-11T22:52:53.130339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:11.128791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131456+0000","last_change":"2024-09-11T22:50:54.178041+0000","last_active":"2024-09-11T22:52:53.131456+0000","last_peered":"2024-09-11T22:52:53.131456+0000","last_clean":"2024-09-11T22:52:53.131456+0000","last_became_active":"2024-09-11T22:50:54.177393+0000","last_became_peered":"2024-09-11T22:50:54.177393+0000","last_unstale":"2024-09-11T22:52:53.131456+0000","last_undegraded":"2024-09-11T22:52:53.131456+0000","last_fullsized":"2024-09-11T22:52:53.131456+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:21.608104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074146+0000","last_change":"2024-09-11T22:50:52.264051+0000","last_active":"2024-09-11T22:52:53.074146+0000","last_peered":"2024-09-11T22:52:53.074146+0000","last_clean":"2024-09-11T22:52:53.074146+0000","last_became_active":"2024-09-11T22:50:52.263674+0000","last_became_peered":"2024-09-11T22:50:52.263674+0000","last_unstale":"2024-09-11T22:52:53.074146+0000","last_undegraded":"2024-09-11T22:52:53.074146+0000","last_fullsized":"2024-09-11T22:52:53.074146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:44.655218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074263+0000","last_change":"2024-09-11T22:50:54.166409+0000","last_active":"2024-09-11T22:52:53.074263+0000","last_peered":"2024-09-11T22:52:53.074263+0000","last_clean":"2024-09-11T22:52:53.074263+0000","last_became_active":"2024-09-11T22:50:54.166037+0000","last_became_peered":"2024-09-11T22:50:54.166037+0000","last_unstale":"2024-09-11T22:52:53.074263+0000","last_undegraded":"2024-09-11T22:52:53.074263+0000","last_fullsized":"2024-09-11T22:52:53.074263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:22.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129632+0000","last_change":"2024-09-11T22:50:52.140218+0000","last_active":"2024-09-11T22:52:53.129632+0000","last_peered":"2024-09-11T22:52:53.129632+0000","last_clean":"2024-09-11T22:52:53.129632+0000","last_became_active":"2024-09-11T22:50:52.139744+0000","last_became_peered":"2024-09-11T22:50:52.139744+0000","last_unstale":"2024-09-11T22:52:53.129632+0000","last_undegraded":"2024-09-11T22:52:53.129632+0000","last_fullsized":"2024-09-11T22:52:53.129632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:50.178348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168909+0000","last_change":"2024-09-11T22:50:54.164059+0000","last_active":"2024-09-11T22:52:53.168909+0000","last_peered":"2024-09-11T22:52:53.168909+0000","last_clean":"2024-09-11T22:52:53.168909+0000","last_became_active":"2024-09-11T22:50:54.163724+0000","last_became_peered":"2024-09-11T22:50:54.163724+0000","last_unstale":"2024-09-11T22:52:53.168909+0000","last_undegraded":"2024-09-11T22:52:53.168909+0000","last_fullsized":"2024-09-11T22:52:53.168909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:57.241660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129028+0000","last_change":"2024-09-11T22:50:52.140169+0000","last_active":"2024-09-11T22:52:53.129028+0000","last_peered":"2024-09-11T22:52:53.129028+0000","last_clean":"2024-09-11T22:52:53.129028+0000","last_became_active":"2024-09-11T22:50:52.139579+0000","last_became_peered":"2024-09-11T22:50:52.139579+0000","last_unstale":"2024-09-11T22:52:53.129028+0000","last_undegraded":"2024-09-11T22:52:53.129028+0000","last_fullsized":"2024-09-11T22:52:53.129028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:43.523961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130075+0000","last_change":"2024-09-11T22:50:54.171539+0000","last_active":"2024-09-11T22:52:53.130075+0000","last_peered":"2024-09-11T22:52:53.130075+0000","last_clean":"2024-09-11T22:52:53.130075+0000","last_became_active":"2024-09-11T22:50:54.171013+0000","last_became_peered":"2024-09-11T22:50:54.171013+0000","last_unstale":"2024-09-11T22:52:53.130075+0000","last_undegraded":"2024-09-11T22:52:53.130075+0000","last_fullsized":"2024-09-11T22:52:53.130075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:03.564389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074787+0000","last_change":"2024-09-11T22:50:52.139544+0000","last_active":"2024-09-11T22:52:53.074787+0000","last_peered":"2024-09-11T22:52:53.074787+0000","last_clean":"2024-09-11T22:52:53.074787+0000","last_became_active":"2024-09-11T22:50:52.139201+0000","last_became_peered":"2024-09-11T22:50:52.139201+0000","last_unstale":"2024-09-11T22:52:53.074787+0000","last_undegraded":"2024-09-11T22:52:53.074787+0000","last_fullsized":"2024-09-11T22:52:53.074787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:37.343437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129337+0000","last_change":"2024-09-11T22:50:54.170052+0000","last_active":"2024-09-11T22:52:53.129337+0000","last_peered":"2024-09-11T22:52:53.129337+0000","last_clean":"2024-09-11T22:52:53.129337+0000","last_became_active":"2024-09-11T22:50:54.169735+0000","last_became_peered":"2024-09-11T22:50:54.169735+0000","last_unstale":"2024-09-11T22:52:53.129337+0000","last_undegraded":"2024-09-11T22:52:53.129337+0000","last_fullsized":"2024-09-11T22:52:53.129337+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:44.007747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074181+0000","last_change":"2024-09-11T22:50:54.169959+0000","last_active":"2024-09-11T22:52:53.074181+0000","last_peered":"2024-09-11T22:52:53.074181+0000","last_clean":"2024-09-11T22:52:53.074181+0000","last_became_active":"2024-09-11T22:50:54.169649+0000","last_became_peered":"2024-09-11T22:50:54.169649+0000","last_unstale":"2024-09-11T22:52:53.074181+0000","last_undegraded":"2024-09-11T22:52:53.074181+0000","last_fullsized":"2024-09-11T22:52:53.074181+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:51.974258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074520+0000","last_change":"2024-09-11T22:50:52.140881+0000","last_active":"2024-09-11T22:52:53.074520+0000","last_peered":"2024-09-11T22:52:53.074520+0000","last_clean":"2024-09-11T22:52:53.074520+0000","last_became_active":"2024-09-11T22:50:52.140527+0000","last_became_peered":"2024-09-11T22:50:52.140527+0000","last_unstale":"2024-09-11T22:52:53.074520+0000","last_undegraded":"2024-09-11T22:52:53.074520+0000","last_fullsized":"2024-09-11T22:52:53.074520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:14.325816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074263+0000","last_change":"2024-09-11T22:50:54.166571+0000","last_active":"2024-09-11T22:52:53.074263+0000","last_peered":"2024-09-11T22:52:53.074263+0000","last_clean":"2024-09-11T22:52:53.074263+0000","last_became_active":"2024-09-11T22:50:54.166247+0000","last_became_peered":"2024-09-11T22:50:54.166247+0000","last_unstale":"2024-09-11T22:52:53.074263+0000","last_undegraded":"2024-09-11T22:52:53.074263+0000","last_fullsized":"2024-09-11T22:52:53.074263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:42.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074895+0000","last_change":"2024-09-11T22:50:52.140298+0000","last_active":"2024-09-11T22:52:53.074895+0000","last_peered":"2024-09-11T22:52:53.074895+0000","last_clean":"2024-09-11T22:52:53.074895+0000","last_became_active":"2024-09-11T22:50:52.139917+0000","last_became_peered":"2024-09-11T22:50:52.139917+0000","last_unstale":"2024-09-11T22:52:53.074895+0000","last_undegraded":"2024-09-11T22:52:53.074895+0000","last_fullsized":"2024-09-11T22:52:53.074895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:23.599010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131626+0000","last_change":"2024-09-11T22:50:54.170849+0000","last_active":"2024-09-11T22:52:53.131626+0000","last_peered":"2024-09-11T22:52:53.131626+0000","last_clean":"2024-09-11T22:52:53.131626+0000","last_became_active":"2024-09-11T22:50:54.169547+0000","last_became_peered":"2024-09-11T22:50:54.169547+0000","last_unstale":"2024-09-11T22:52:53.131626+0000","last_undegraded":"2024-09-11T22:52:53.131626+0000","last_fullsized":"2024-09-11T22:52:53.131626+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:25.251158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130596+0000","last_change":"2024-09-11T22:50:52.263778+0000","last_active":"2024-09-11T22:52:53.130596+0000","last_peered":"2024-09-11T22:52:53.130596+0000","last_clean":"2024-09-11T22:52:53.130596+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:52:53.130596+0000","last_undegraded":"2024-09-11T22:52:53.130596+0000","last_fullsized":"2024-09-11T22:52:53.130596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:03.439147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130475+0000","last_change":"2024-09-11T22:50:54.171200+0000","last_active":"2024-09-11T22:52:53.130475+0000","last_peered":"2024-09-11T22:52:53.130475+0000","last_clean":"2024-09-11T22:52:53.130475+0000","last_became_active":"2024-09-11T22:50:54.169320+0000","last_became_peered":"2024-09-11T22:50:54.169320+0000","last_unstale":"2024-09-11T22:52:53.130475+0000","last_undegraded":"2024-09-11T22:52:53.130475+0000","last_fullsized":"2024-09-11T22:52:53.130475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:14.864699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":158,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:02.299866+0000","last_change":"2024-09-11T22:50:52.139609+0000","last_active":"2024-09-11T22:53:02.299866+0000","last_peered":"2024-09-11T22:53:02.299866+0000","last_clean":"2024-09-11T22:53:02.299866+0000","last_became_active":"2024-09-11T22:50:52.139278+0000","last_became_peered":"2024-09-11T22:50:52.139278+0000","last_unstale":"2024-09-11T22:53:02.299866+0000","last_undegraded":"2024-09-11T22:53:02.299866+0000","last_fullsized":"2024-09-11T22:53:02.299866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:09.538734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799447,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1585,"num_read_kb":1014,"num_write":508,"num_write_kb":2384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":2398341,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":386,"ondisk_log_size":386,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107505,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547240,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":840516,"data_compressed":22190,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776172,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1126400,"data_stored":1434964,"data_compressed":53938,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939698,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":839519,"data_compressed":22190,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168630,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":446681,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397564,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1040914,"data_compressed":38736,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561090,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":446020,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790023,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1048576,"data_stored":1368038,"data_compressed":53938,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953548,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":925696,"data_stored":446667,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1011712,"data_stored":1167293,"data_compressed":37392,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":517944,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":515477,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":946176,"data_stored":448286,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":332372,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":399298,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":395335,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:53:07.100 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T22:53:07.100 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:53:07.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:07 smithi191 ceph-mon[35211]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 55 op/s 2024-09-11T22:53:07.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:07 smithi204 ceph-mon[35454]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 55 op/s 2024-09-11T22:53:07.350 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:07.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:07 smithi152 ceph-mon[31181]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 55 op/s 2024-09-11T22:53:07.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.889+0000 7f181d437640 1 -- 172.21.15.152:0/3409947161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1818106a80 msgr2=0x7f1818106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:07.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.889+0000 7f181d437640 1 --2- 172.21.15.152:0/3409947161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1818106a80 0x7f1818106e60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1800009930 tx=0x7f180002a420 comp rx=0 tx=0).stop 2024-09-11T22:53:07.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.889+0000 7f181d437640 1 -- 172.21.15.152:0/3409947161 shutdown_connections 2024-09-11T22:53:07.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.889+0000 7f181d437640 1 --2- 172.21.15.152:0/3409947161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1818101420 0x7f181810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:07.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.889+0000 7f181d437640 1 --2- 172.21.15.152:0/3409947161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818100a80 0x7f1818100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.889+0000 7f181d437640 1 --2- 172.21.15.152:0/3409947161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1818106a80 0x7f1818106e60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.889+0000 7f181d437640 1 -- 172.21.15.152:0/3409947161 >> 172.21.15.152:0/3409947161 conn(0x7f18180fc7a0 msgr2=0x7f18180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.889+0000 7f181d437640 1 -- 172.21.15.152:0/3409947161 shutdown_connections 2024-09-11T22:53:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.889+0000 7f181d437640 1 -- 172.21.15.152:0/3409947161 wait complete. 2024-09-11T22:53:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f181d437640 1 Processor -- start 2024-09-11T22:53:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f181d437640 1 -- start start 2024-09-11T22:53:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f181d437640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1818100a80 0x7f1818198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f181d437640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818101420 0x7f1818198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f181d437640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1818106a80 0x7f181819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f181d437640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1818110ef0 con 0x7f1818106a80 2024-09-11T22:53:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f181d437640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1818110d70 con 0x7f1818100a80 2024-09-11T22:53:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f181d437640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1818111070 con 0x7f1818101420 2024-09-11T22:53:07.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f18177fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1818106a80 0x7f181819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f18177fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1818106a80 0x7f181819cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43870/0 (socket says 172.21.15.152:43870) 2024-09-11T22:53:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f18177fe640 1 -- 172.21.15.152:0/500412127 learned_addr learned my addr 172.21.15.152:0/500412127 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f1816ffd640 1 --2- 172.21.15.152:0/500412127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1818100a80 0x7f1818198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f18167fc640 1 --2- 172.21.15.152:0/500412127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818101420 0x7f1818198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f18177fe640 1 -- 172.21.15.152:0/500412127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818101420 msgr2=0x7f1818198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f18177fe640 1 --2- 172.21.15.152:0/500412127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818101420 0x7f1818198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f18177fe640 1 -- 172.21.15.152:0/500412127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1818100a80 msgr2=0x7f1818198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f18177fe640 1 --2- 172.21.15.152:0/500412127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1818100a80 0x7f1818198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.890+0000 7f18177fe640 1 -- 172.21.15.152:0/500412127 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181819d4d0 con 0x7f1818106a80 2024-09-11T22:53:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.891+0000 7f18177fe640 1 --2- 172.21.15.152:0/500412127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1818106a80 0x7f181819cdf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1810009fd0 tx=0x7f181000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.891+0000 7f180ffff640 1 -- 172.21.15.152:0/500412127 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810017070 con 0x7f1818106a80 2024-09-11T22:53:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.891+0000 7f180ffff640 1 -- 172.21.15.152:0/500412127 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1810003e50 con 0x7f1818106a80 2024-09-11T22:53:07.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.891+0000 7f181d437640 1 -- 172.21.15.152:0/500412127 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181819d7c0 con 0x7f1818106a80 2024-09-11T22:53:07.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.891+0000 7f180ffff640 1 -- 172.21.15.152:0/500412127 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18100106c0 con 0x7f1818106a80 2024-09-11T22:53:07.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.891+0000 7f181d437640 1 -- 172.21.15.152:0/500412127 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18181a50a0 con 0x7f1818106a80 2024-09-11T22:53:07.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.892+0000 7f181d437640 1 -- 172.21.15.152:0/500412127 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1818100ee0 con 0x7f1818106a80 2024-09-11T22:53:07.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.893+0000 7f180ffff640 1 -- 172.21.15.152:0/500412127 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f1810003970 con 0x7f1818106a80 2024-09-11T22:53:07.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.894+0000 7f180ffff640 1 --2- 172.21.15.152:0/500412127 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f18040a2d80 0x7f18040a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:07.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.894+0000 7f1816ffd640 1 --2- 172.21.15.152:0/500412127 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f18040a2d80 0x7f18040a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:07.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.894+0000 7f180ffff640 1 -- 172.21.15.152:0/500412127 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f18100cf330 con 0x7f1818106a80 2024-09-11T22:53:07.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.894+0000 7f1816ffd640 1 --2- 172.21.15.152:0/500412127 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f18040a2d80 0x7f18040a5240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1800006010 tx=0x7f1800000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:07.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:07.897+0000 7f180ffff640 1 -- 172.21.15.152:0/500412127 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f181009be30 con 0x7f1818106a80 2024-09-11T22:53:08.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:08.064+0000 7f181d437640 1 -- 172.21.15.152:0/500412127 --> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18180fce80 con 0x7f18040a2d80 2024-09-11T22:53:08.065 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:08 smithi152 ceph-mon[31181]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:53:08.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:08.070+0000 7f180ffff640 1 -- 172.21.15.152:0/500412127 <== mgr.15240 v2:172.21.15.152:6800/4130474708 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258069 (secure 0 0 0) 0x7f18180fce80 con 0x7f18040a2d80 2024-09-11T22:53:08.073 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:53:08.074 INFO:teuthology.orchestra.run.smithi152.stderr:dumped all 2024-09-11T22:53:08.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:08.073+0000 7f181d437640 1 -- 172.21.15.152:0/500412127 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f18040a2d80 msgr2=0x7f18040a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:08.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:08.073+0000 7f181d437640 1 --2- 172.21.15.152:0/500412127 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f18040a2d80 0x7f18040a5240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1800006010 tx=0x7f1800000bc0 comp rx=0 tx=0).stop 2024-09-11T22:53:08.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:08.073+0000 7f181d437640 1 -- 172.21.15.152:0/500412127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1818106a80 msgr2=0x7f181819cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:08.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:08.073+0000 7f181d437640 1 --2- 172.21.15.152:0/500412127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1818106a80 0x7f181819cdf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1810009fd0 tx=0x7f181000b260 comp rx=0 tx=0).stop 2024-09-11T22:53:08.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:08.073+0000 7f181d437640 1 -- 172.21.15.152:0/500412127 shutdown_connections 2024-09-11T22:53:08.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:08.073+0000 7f181d437640 1 --2- 172.21.15.152:0/500412127 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f18040a2d80 0x7f18040a5240 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:08.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:08.073+0000 7f181d437640 1 --2- 172.21.15.152:0/500412127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1818106a80 0x7f181819cdf0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:08.073+0000 7f181d437640 1 --2- 172.21.15.152:0/500412127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1818101420 0x7f1818198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:08.073+0000 7f181d437640 1 --2- 172.21.15.152:0/500412127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1818100a80 0x7f1818198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:08.073+0000 7f181d437640 1 -- 172.21.15.152:0/500412127 >> 172.21.15.152:0/500412127 conn(0x7f18180fc7a0 msgr2=0x7f18181048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:08.073+0000 7f181d437640 1 -- 172.21.15.152:0/500412127 shutdown_connections 2024-09-11T22:53:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:08.073+0000 7f181d437640 1 -- 172.21.15.152:0/500412127 wait complete. 2024-09-11T22:53:08.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:08 smithi191 ceph-mon[35211]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:53:08.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:08 smithi204 ceph-mon[35454]: from='client.15444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:53:08.835 INFO:teuthology.orchestra.run.smithi152.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-11T22:53:07.089242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1390049,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1822,"num_read_kb":1344,"num_write":711,"num_write_kb":5324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1036,"ondisk_log_size":1036,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369164,"kb_used_data":11556,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572660,"statfs":{"total":1151940427776,"available":1151562403840,"internally_reserved":0,"allocated":11833344,"data_stored":9512319,"data_compressed":245790,"data_compressed_allocated":319488,"data_compressed_original":3969024,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":459,"num_read_kb":496,"num_write":150,"num_write_kb":916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002597"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169180+0000","last_change":"2024-09-11T22:50:54.171410+0000","last_active":"2024-09-11T22:52:53.169180+0000","last_peered":"2024-09-11T22:52:53.169180+0000","last_clean":"2024-09-11T22:52:53.169180+0000","last_became_active":"2024-09-11T22:50:54.171055+0000","last_became_peered":"2024-09-11T22:50:54.171055+0000","last_unstale":"2024-09-11T22:52:53.169180+0000","last_undegraded":"2024-09-11T22:52:53.169180+0000","last_fullsized":"2024-09-11T22:52:53.169180+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:48:54.767454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131374+0000","last_change":"2024-09-11T22:50:54.168534+0000","last_active":"2024-09-11T22:52:53.131374+0000","last_peered":"2024-09-11T22:52:53.131374+0000","last_clean":"2024-09-11T22:52:53.131374+0000","last_became_active":"2024-09-11T22:50:54.168228+0000","last_became_peered":"2024-09-11T22:50:54.168228+0000","last_unstale":"2024-09-11T22:52:53.131374+0000","last_undegraded":"2024-09-11T22:52:53.131374+0000","last_fullsized":"2024-09-11T22:52:53.131374+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:11.380048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131174+0000","last_change":"2024-09-11T22:50:54.168975+0000","last_active":"2024-09-11T22:52:53.131174+0000","last_peered":"2024-09-11T22:52:53.131174+0000","last_clean":"2024-09-11T22:52:53.131174+0000","last_became_active":"2024-09-11T22:50:54.168676+0000","last_became_peered":"2024-09-11T22:50:54.168676+0000","last_unstale":"2024-09-11T22:52:53.131174+0000","last_undegraded":"2024-09-11T22:52:53.131174+0000","last_fullsized":"2024-09-11T22:52:53.131174+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:15.848273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130100+0000","last_change":"2024-09-11T22:50:54.164164+0000","last_active":"2024-09-11T22:52:53.130100+0000","last_peered":"2024-09-11T22:52:53.130100+0000","last_clean":"2024-09-11T22:52:53.130100+0000","last_became_active":"2024-09-11T22:50:54.163805+0000","last_became_peered":"2024-09-11T22:50:54.163805+0000","last_unstale":"2024-09-11T22:52:53.130100+0000","last_undegraded":"2024-09-11T22:52:53.130100+0000","last_fullsized":"2024-09-11T22:52:53.130100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:10:54.954046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074671+0000","last_change":"2024-09-11T22:50:54.167289+0000","last_active":"2024-09-11T22:52:53.074671+0000","last_peered":"2024-09-11T22:52:53.074671+0000","last_clean":"2024-09-11T22:52:53.074671+0000","last_became_active":"2024-09-11T22:50:54.166958+0000","last_became_peered":"2024-09-11T22:50:54.166958+0000","last_unstale":"2024-09-11T22:52:53.074671+0000","last_undegraded":"2024-09-11T22:52:53.074671+0000","last_fullsized":"2024-09-11T22:52:53.074671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:30:25.261241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129676+0000","last_change":"2024-09-11T22:50:54.175605+0000","last_active":"2024-09-11T22:52:53.129676+0000","last_peered":"2024-09-11T22:52:53.129676+0000","last_clean":"2024-09-11T22:52:53.129676+0000","last_became_active":"2024-09-11T22:50:54.175207+0000","last_became_peered":"2024-09-11T22:50:54.175207+0000","last_unstale":"2024-09-11T22:52:53.129676+0000","last_undegraded":"2024-09-11T22:52:53.129676+0000","last_fullsized":"2024-09-11T22:52:53.129676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:31:47.008910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129084+0000","last_change":"2024-09-11T22:50:54.167930+0000","last_active":"2024-09-11T22:52:53.129084+0000","last_peered":"2024-09-11T22:52:53.129084+0000","last_clean":"2024-09-11T22:52:53.129084+0000","last_became_active":"2024-09-11T22:50:54.167082+0000","last_became_peered":"2024-09-11T22:50:54.167082+0000","last_unstale":"2024-09-11T22:52:53.129084+0000","last_undegraded":"2024-09-11T22:52:53.129084+0000","last_fullsized":"2024-09-11T22:52:53.129084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:52:50.931311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129336+0000","last_change":"2024-09-11T22:50:54.172528+0000","last_active":"2024-09-11T22:52:53.129336+0000","last_peered":"2024-09-11T22:52:53.129336+0000","last_clean":"2024-09-11T22:52:53.129336+0000","last_became_active":"2024-09-11T22:50:54.172257+0000","last_became_peered":"2024-09-11T22:50:54.172257+0000","last_unstale":"2024-09-11T22:52:53.129336+0000","last_undegraded":"2024-09-11T22:52:53.129336+0000","last_fullsized":"2024-09-11T22:52:53.129336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:13:43.184795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074665+0000","last_change":"2024-09-11T22:50:54.163357+0000","last_active":"2024-09-11T22:52:53.074665+0000","last_peered":"2024-09-11T22:52:53.074665+0000","last_clean":"2024-09-11T22:52:53.074665+0000","last_became_active":"2024-09-11T22:50:54.162943+0000","last_became_peered":"2024-09-11T22:50:54.162943+0000","last_unstale":"2024-09-11T22:52:53.074665+0000","last_undegraded":"2024-09-11T22:52:53.074665+0000","last_fullsized":"2024-09-11T22:52:53.074665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:23:57.055802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129448+0000","last_change":"2024-09-11T22:50:54.177773+0000","last_active":"2024-09-11T22:52:53.129448+0000","last_peered":"2024-09-11T22:52:53.129448+0000","last_clean":"2024-09-11T22:52:53.129448+0000","last_became_active":"2024-09-11T22:50:54.177388+0000","last_became_peered":"2024-09-11T22:50:54.177388+0000","last_unstale":"2024-09-11T22:52:53.129448+0000","last_undegraded":"2024-09-11T22:52:53.129448+0000","last_fullsized":"2024-09-11T22:52:53.129448+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:36:44.625089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129792+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:52:53.129792+0000","last_peered":"2024-09-11T22:52:53.129792+0000","last_clean":"2024-09-11T22:52:53.129792+0000","last_became_active":"2024-09-11T22:50:54.171491+0000","last_became_peered":"2024-09-11T22:50:54.171491+0000","last_unstale":"2024-09-11T22:52:53.129792+0000","last_undegraded":"2024-09-11T22:52:53.129792+0000","last_fullsized":"2024-09-11T22:52:53.129792+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.996640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130199+0000","last_change":"2024-09-11T22:50:54.171260+0000","last_active":"2024-09-11T22:52:53.130199+0000","last_peered":"2024-09-11T22:52:53.130199+0000","last_clean":"2024-09-11T22:52:53.130199+0000","last_became_active":"2024-09-11T22:50:54.170733+0000","last_became_peered":"2024-09-11T22:50:54.170733+0000","last_unstale":"2024-09-11T22:52:53.130199+0000","last_undegraded":"2024-09-11T22:52:53.130199+0000","last_fullsized":"2024-09-11T22:52:53.130199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:57.877280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129997+0000","last_change":"2024-09-11T22:50:54.170576+0000","last_active":"2024-09-11T22:52:53.129997+0000","last_peered":"2024-09-11T22:52:53.129997+0000","last_clean":"2024-09-11T22:52:53.129997+0000","last_became_active":"2024-09-11T22:50:54.170239+0000","last_became_peered":"2024-09-11T22:50:54.170239+0000","last_unstale":"2024-09-11T22:52:53.129997+0000","last_undegraded":"2024-09-11T22:52:53.129997+0000","last_fullsized":"2024-09-11T22:52:53.129997+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:09.683171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130651+0000","last_change":"2024-09-11T22:50:54.167176+0000","last_active":"2024-09-11T22:52:53.130651+0000","last_peered":"2024-09-11T22:52:53.130651+0000","last_clean":"2024-09-11T22:52:53.130651+0000","last_became_active":"2024-09-11T22:50:54.166894+0000","last_became_peered":"2024-09-11T22:50:54.166894+0000","last_unstale":"2024-09-11T22:52:53.130651+0000","last_undegraded":"2024-09-11T22:52:53.130651+0000","last_fullsized":"2024-09-11T22:52:53.130651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:43:08.158786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130901+0000","last_change":"2024-09-11T22:50:54.163100+0000","last_active":"2024-09-11T22:52:53.130901+0000","last_peered":"2024-09-11T22:52:53.130901+0000","last_clean":"2024-09-11T22:52:53.130901+0000","last_became_active":"2024-09-11T22:50:54.162776+0000","last_became_peered":"2024-09-11T22:50:54.162776+0000","last_unstale":"2024-09-11T22:52:53.130901+0000","last_undegraded":"2024-09-11T22:52:53.130901+0000","last_fullsized":"2024-09-11T22:52:53.130901+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:02:15.215305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074034+0000","last_change":"2024-09-11T22:50:54.166013+0000","last_active":"2024-09-11T22:52:53.074034+0000","last_peered":"2024-09-11T22:52:53.074034+0000","last_clean":"2024-09-11T22:52:53.074034+0000","last_became_active":"2024-09-11T22:50:54.165680+0000","last_became_peered":"2024-09-11T22:50:54.165680+0000","last_unstale":"2024-09-11T22:52:53.074034+0000","last_undegraded":"2024-09-11T22:52:53.074034+0000","last_fullsized":"2024-09-11T22:52:53.074034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:05:12.192231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169175+0000","last_change":"2024-09-11T22:50:54.172105+0000","last_active":"2024-09-11T22:52:53.169175+0000","last_peered":"2024-09-11T22:52:53.169175+0000","last_clean":"2024-09-11T22:52:53.169175+0000","last_became_active":"2024-09-11T22:50:54.171782+0000","last_became_peered":"2024-09-11T22:50:54.171782+0000","last_unstale":"2024-09-11T22:52:53.169175+0000","last_undegraded":"2024-09-11T22:52:53.169175+0000","last_fullsized":"2024-09-11T22:52:53.169175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:53:17.152834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130880+0000","last_change":"2024-09-11T22:50:54.167184+0000","last_active":"2024-09-11T22:52:53.130880+0000","last_peered":"2024-09-11T22:52:53.130880+0000","last_clean":"2024-09-11T22:52:53.130880+0000","last_became_active":"2024-09-11T22:50:54.166878+0000","last_became_peered":"2024-09-11T22:50:54.166878+0000","last_unstale":"2024-09-11T22:52:53.130880+0000","last_undegraded":"2024-09-11T22:52:53.130880+0000","last_fullsized":"2024-09-11T22:52:53.130880+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:44.046552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129108+0000","last_change":"2024-09-11T22:50:54.176260+0000","last_active":"2024-09-11T22:52:53.129108+0000","last_peered":"2024-09-11T22:52:53.129108+0000","last_clean":"2024-09-11T22:52:53.129108+0000","last_became_active":"2024-09-11T22:50:54.175873+0000","last_became_peered":"2024-09-11T22:50:54.175873+0000","last_unstale":"2024-09-11T22:52:53.129108+0000","last_undegraded":"2024-09-11T22:52:53.129108+0000","last_fullsized":"2024-09-11T22:52:53.129108+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:44:05.258677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168910+0000","last_change":"2024-09-11T22:50:54.172725+0000","last_active":"2024-09-11T22:52:53.168910+0000","last_peered":"2024-09-11T22:52:53.168910+0000","last_clean":"2024-09-11T22:52:53.168910+0000","last_became_active":"2024-09-11T22:50:54.172072+0000","last_became_peered":"2024-09-11T22:50:54.172072+0000","last_unstale":"2024-09-11T22:52:53.168910+0000","last_undegraded":"2024-09-11T22:52:53.168910+0000","last_fullsized":"2024-09-11T22:52:53.168910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:53.699670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130148+0000","last_change":"2024-09-11T22:50:54.165118+0000","last_active":"2024-09-11T22:52:53.130148+0000","last_peered":"2024-09-11T22:52:53.130148+0000","last_clean":"2024-09-11T22:52:53.130148+0000","last_became_active":"2024-09-11T22:50:54.164793+0000","last_became_peered":"2024-09-11T22:50:54.164793+0000","last_unstale":"2024-09-11T22:52:53.130148+0000","last_undegraded":"2024-09-11T22:52:53.130148+0000","last_fullsized":"2024-09-11T22:52:53.130148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:08:42.883940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074224+0000","last_change":"2024-09-11T22:50:54.176900+0000","last_active":"2024-09-11T22:52:53.074224+0000","last_peered":"2024-09-11T22:52:53.074224+0000","last_clean":"2024-09-11T22:52:53.074224+0000","last_became_active":"2024-09-11T22:50:54.176595+0000","last_became_peered":"2024-09-11T22:50:54.176595+0000","last_unstale":"2024-09-11T22:52:53.074224+0000","last_undegraded":"2024-09-11T22:52:53.074224+0000","last_fullsized":"2024-09-11T22:52:53.074224+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:36.925945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129485+0000","last_change":"2024-09-11T22:50:54.167656+0000","last_active":"2024-09-11T22:52:53.129485+0000","last_peered":"2024-09-11T22:52:53.129485+0000","last_clean":"2024-09-11T22:52:53.129485+0000","last_became_active":"2024-09-11T22:50:54.167349+0000","last_became_peered":"2024-09-11T22:50:54.167349+0000","last_unstale":"2024-09-11T22:52:53.129485+0000","last_undegraded":"2024-09-11T22:52:53.129485+0000","last_fullsized":"2024-09-11T22:52:53.129485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:53:38.100218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073470+0000","last_change":"2024-09-11T22:50:54.177059+0000","last_active":"2024-09-11T22:52:53.073470+0000","last_peered":"2024-09-11T22:52:53.073470+0000","last_clean":"2024-09-11T22:52:53.073470+0000","last_became_active":"2024-09-11T22:50:54.176726+0000","last_became_peered":"2024-09-11T22:50:54.176726+0000","last_unstale":"2024-09-11T22:52:53.073470+0000","last_undegraded":"2024-09-11T22:52:53.073470+0000","last_fullsized":"2024-09-11T22:52:53.073470+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:27:58.058944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130622+0000","last_change":"2024-09-11T22:50:54.167566+0000","last_active":"2024-09-11T22:52:53.130622+0000","last_peered":"2024-09-11T22:52:53.130622+0000","last_clean":"2024-09-11T22:52:53.130622+0000","last_became_active":"2024-09-11T22:50:54.167049+0000","last_became_peered":"2024-09-11T22:50:54.167049+0000","last_unstale":"2024-09-11T22:52:53.130622+0000","last_undegraded":"2024-09-11T22:52:53.130622+0000","last_fullsized":"2024-09-11T22:52:53.130622+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:06.833099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129228+0000","last_change":"2024-09-11T22:50:54.169732+0000","last_active":"2024-09-11T22:52:53.129228+0000","last_peered":"2024-09-11T22:52:53.129228+0000","last_clean":"2024-09-11T22:52:53.129228+0000","last_became_active":"2024-09-11T22:50:54.169437+0000","last_became_peered":"2024-09-11T22:50:54.169437+0000","last_unstale":"2024-09-11T22:52:53.129228+0000","last_undegraded":"2024-09-11T22:52:53.129228+0000","last_fullsized":"2024-09-11T22:52:53.129228+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:14:25.553387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169167+0000","last_change":"2024-09-11T22:50:54.166953+0000","last_active":"2024-09-11T22:52:53.169167+0000","last_peered":"2024-09-11T22:52:53.169167+0000","last_clean":"2024-09-11T22:52:53.169167+0000","last_became_active":"2024-09-11T22:50:54.166472+0000","last_became_peered":"2024-09-11T22:50:54.166472+0000","last_unstale":"2024-09-11T22:52:53.169167+0000","last_undegraded":"2024-09-11T22:52:53.169167+0000","last_fullsized":"2024-09-11T22:52:53.169167+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:01:20.234180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129475+0000","last_change":"2024-09-11T22:50:54.171308+0000","last_active":"2024-09-11T22:52:53.129475+0000","last_peered":"2024-09-11T22:52:53.129475+0000","last_clean":"2024-09-11T22:52:53.129475+0000","last_became_active":"2024-09-11T22:50:54.168958+0000","last_became_peered":"2024-09-11T22:50:54.168958+0000","last_unstale":"2024-09-11T22:52:53.129475+0000","last_undegraded":"2024-09-11T22:52:53.129475+0000","last_fullsized":"2024-09-11T22:52:53.129475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:20.224003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131226+0000","last_change":"2024-09-11T22:50:54.173846+0000","last_active":"2024-09-11T22:52:53.131226+0000","last_peered":"2024-09-11T22:52:53.131226+0000","last_clean":"2024-09-11T22:52:53.131226+0000","last_became_active":"2024-09-11T22:50:54.173542+0000","last_became_peered":"2024-09-11T22:50:54.173542+0000","last_unstale":"2024-09-11T22:52:53.131226+0000","last_undegraded":"2024-09-11T22:52:53.131226+0000","last_fullsized":"2024-09-11T22:52:53.131226+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:29:45.775902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129000+0000","last_change":"2024-09-11T22:50:54.176008+0000","last_active":"2024-09-11T22:52:53.129000+0000","last_peered":"2024-09-11T22:52:53.129000+0000","last_clean":"2024-09-11T22:52:53.129000+0000","last_became_active":"2024-09-11T22:50:54.175672+0000","last_became_peered":"2024-09-11T22:50:54.175672+0000","last_unstale":"2024-09-11T22:52:53.129000+0000","last_undegraded":"2024-09-11T22:52:53.129000+0000","last_fullsized":"2024-09-11T22:52:53.129000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:35:35.276514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129006+0000","last_change":"2024-09-11T22:50:54.162504+0000","last_active":"2024-09-11T22:52:53.129006+0000","last_peered":"2024-09-11T22:52:53.129006+0000","last_clean":"2024-09-11T22:52:53.129006+0000","last_became_active":"2024-09-11T22:50:54.162190+0000","last_became_peered":"2024-09-11T22:50:54.162190+0000","last_unstale":"2024-09-11T22:52:53.129006+0000","last_undegraded":"2024-09-11T22:52:53.129006+0000","last_fullsized":"2024-09-11T22:52:53.129006+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:01:52.138692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168887+0000","last_change":"2024-09-11T22:51:55.960882+0000","last_active":"2024-09-11T22:52:53.168887+0000","last_peered":"2024-09-11T22:52:53.168887+0000","last_clean":"2024-09-11T22:52:53.168887+0000","last_became_active":"2024-09-11T22:51:55.960246+0000","last_became_peered":"2024-09-11T22:51:55.960246+0000","last_unstale":"2024-09-11T22:52:53.168887+0000","last_undegraded":"2024-09-11T22:52:53.168887+0000","last_fullsized":"2024-09-11T22:52:53.168887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:52:21.247840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074402+0000","last_change":"2024-09-11T22:50:54.160696+0000","last_active":"2024-09-11T22:52:53.074402+0000","last_peered":"2024-09-11T22:52:53.074402+0000","last_clean":"2024-09-11T22:52:53.074402+0000","last_became_active":"2024-09-11T22:50:54.159249+0000","last_became_peered":"2024-09-11T22:50:54.159249+0000","last_unstale":"2024-09-11T22:52:53.074402+0000","last_undegraded":"2024-09-11T22:52:53.074402+0000","last_fullsized":"2024-09-11T22:52:53.074402+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:57:09.649208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'9","reported_seq":157,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:02.292263+0000","last_change":"2024-09-11T22:50:52.138986+0000","last_active":"2024-09-11T22:53:02.292263+0000","last_peered":"2024-09-11T22:53:02.292263+0000","last_clean":"2024-09-11T22:53:02.292263+0000","last_became_active":"2024-09-11T22:50:52.138595+0000","last_became_peered":"2024-09-11T22:50:52.138595+0000","last_unstale":"2024-09-11T22:53:02.292263+0000","last_undegraded":"2024-09-11T22:53:02.292263+0000","last_fullsized":"2024-09-11T22:53:02.292263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:06.232146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1942,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073853+0000","last_change":"2024-09-11T22:50:54.176806+0000","last_active":"2024-09-11T22:52:53.073853+0000","last_peered":"2024-09-11T22:52:53.073853+0000","last_clean":"2024-09-11T22:52:53.073853+0000","last_became_active":"2024-09-11T22:50:54.176513+0000","last_became_peered":"2024-09-11T22:50:54.176513+0000","last_unstale":"2024-09-11T22:52:53.073853+0000","last_undegraded":"2024-09-11T22:52:53.073853+0000","last_fullsized":"2024-09-11T22:52:53.073853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:43:12.086145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129699+0000","last_change":"2024-09-11T22:50:52.142794+0000","last_active":"2024-09-11T22:52:53.129699+0000","last_peered":"2024-09-11T22:52:53.129699+0000","last_clean":"2024-09-11T22:52:53.129699+0000","last_became_active":"2024-09-11T22:50:52.142433+0000","last_became_peered":"2024-09-11T22:50:52.142433+0000","last_unstale":"2024-09-11T22:52:53.129699+0000","last_undegraded":"2024-09-11T22:52:53.129699+0000","last_fullsized":"2024-09-11T22:52:53.129699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:47:12.090055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130326+0000","last_change":"2024-09-11T22:50:52.139789+0000","last_active":"2024-09-11T22:52:53.130326+0000","last_peered":"2024-09-11T22:52:53.130326+0000","last_clean":"2024-09-11T22:52:53.130326+0000","last_became_active":"2024-09-11T22:50:52.139294+0000","last_became_peered":"2024-09-11T22:50:52.139294+0000","last_unstale":"2024-09-11T22:52:53.130326+0000","last_undegraded":"2024-09-11T22:52:53.130326+0000","last_fullsized":"2024-09-11T22:52:53.130326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:03:07.355155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129486+0000","last_change":"2024-09-11T22:50:54.166248+0000","last_active":"2024-09-11T22:52:53.129486+0000","last_peered":"2024-09-11T22:52:53.129486+0000","last_clean":"2024-09-11T22:52:53.129486+0000","last_became_active":"2024-09-11T22:50:54.165958+0000","last_became_peered":"2024-09-11T22:50:54.165958+0000","last_unstale":"2024-09-11T22:52:53.129486+0000","last_undegraded":"2024-09-11T22:52:53.129486+0000","last_fullsized":"2024-09-11T22:52:53.129486+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:29.531741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073597+0000","last_change":"2024-09-11T22:50:52.140340+0000","last_active":"2024-09-11T22:52:53.073597+0000","last_peered":"2024-09-11T22:52:53.073597+0000","last_clean":"2024-09-11T22:52:53.073597+0000","last_became_active":"2024-09-11T22:50:52.140011+0000","last_became_peered":"2024-09-11T22:50:52.140011+0000","last_unstale":"2024-09-11T22:52:53.073597+0000","last_undegraded":"2024-09-11T22:52:53.073597+0000","last_fullsized":"2024-09-11T22:52:53.073597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:29:58.862640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129625+0000","last_change":"2024-09-11T22:50:54.166998+0000","last_active":"2024-09-11T22:52:53.129625+0000","last_peered":"2024-09-11T22:52:53.129625+0000","last_clean":"2024-09-11T22:52:53.129625+0000","last_became_active":"2024-09-11T22:50:54.166682+0000","last_became_peered":"2024-09-11T22:50:54.166682+0000","last_unstale":"2024-09-11T22:52:53.129625+0000","last_undegraded":"2024-09-11T22:52:53.129625+0000","last_fullsized":"2024-09-11T22:52:53.129625+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:44:57.815914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'8","reported_seq":270,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:03.292945+0000","last_change":"2024-09-11T22:50:52.141380+0000","last_active":"2024-09-11T22:53:03.292945+0000","last_peered":"2024-09-11T22:53:03.292945+0000","last_clean":"2024-09-11T22:53:03.292945+0000","last_became_active":"2024-09-11T22:50:52.139925+0000","last_became_peered":"2024-09-11T22:50:52.139925+0000","last_unstale":"2024-09-11T22:53:03.292945+0000","last_undegraded":"2024-09-11T22:53:03.292945+0000","last_fullsized":"2024-09-11T22:53:03.292945+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:40:38.395055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":238,"num_read_kb":238,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074552+0000","last_change":"2024-09-11T22:50:54.167909+0000","last_active":"2024-09-11T22:52:53.074552+0000","last_peered":"2024-09-11T22:52:53.074552+0000","last_clean":"2024-09-11T22:52:53.074552+0000","last_became_active":"2024-09-11T22:50:54.167571+0000","last_became_peered":"2024-09-11T22:50:54.167571+0000","last_unstale":"2024-09-11T22:52:53.074552+0000","last_undegraded":"2024-09-11T22:52:53.074552+0000","last_fullsized":"2024-09-11T22:52:53.074552+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:33:34.179960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129913+0000","last_change":"2024-09-11T22:50:52.144545+0000","last_active":"2024-09-11T22:52:53.129913+0000","last_peered":"2024-09-11T22:52:53.129913+0000","last_clean":"2024-09-11T22:52:53.129913+0000","last_became_active":"2024-09-11T22:50:52.144221+0000","last_became_peered":"2024-09-11T22:50:52.144221+0000","last_unstale":"2024-09-11T22:52:53.129913+0000","last_undegraded":"2024-09-11T22:52:53.129913+0000","last_fullsized":"2024-09-11T22:52:53.129913+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:29.509165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131618+0000","last_change":"2024-09-11T22:50:54.164132+0000","last_active":"2024-09-11T22:52:53.131618+0000","last_peered":"2024-09-11T22:52:53.131618+0000","last_clean":"2024-09-11T22:52:53.131618+0000","last_became_active":"2024-09-11T22:50:54.163842+0000","last_became_peered":"2024-09-11T22:50:54.163842+0000","last_unstale":"2024-09-11T22:52:53.131618+0000","last_undegraded":"2024-09-11T22:52:53.131618+0000","last_fullsized":"2024-09-11T22:52:53.131618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:28:55.490236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074349+0000","last_change":"2024-09-11T22:50:52.264611+0000","last_active":"2024-09-11T22:52:53.074349+0000","last_peered":"2024-09-11T22:52:53.074349+0000","last_clean":"2024-09-11T22:52:53.074349+0000","last_became_active":"2024-09-11T22:50:52.264060+0000","last_became_peered":"2024-09-11T22:50:52.264060+0000","last_unstale":"2024-09-11T22:52:53.074349+0000","last_undegraded":"2024-09-11T22:52:53.074349+0000","last_fullsized":"2024-09-11T22:52:53.074349+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:30:42.543997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074271+0000","last_change":"2024-09-11T22:51:56.206155+0000","last_active":"2024-09-11T22:52:53.074271+0000","last_peered":"2024-09-11T22:52:53.074271+0000","last_clean":"2024-09-11T22:52:53.074271+0000","last_became_active":"2024-09-11T22:51:56.205797+0000","last_became_peered":"2024-09-11T22:51:56.205797+0000","last_unstale":"2024-09-11T22:52:53.074271+0000","last_undegraded":"2024-09-11T22:52:53.074271+0000","last_fullsized":"2024-09-11T22:52:53.074271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:39:28.063825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'39","reported_seq":315,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:03.293288+0000","last_change":"2024-09-11T22:50:52.141782+0000","last_active":"2024-09-11T22:53:03.293288+0000","last_peered":"2024-09-11T22:53:03.293288+0000","last_clean":"2024-09-11T22:53:03.293288+0000","last_became_active":"2024-09-11T22:50:52.140662+0000","last_became_peered":"2024-09-11T22:50:52.140662+0000","last_unstale":"2024-09-11T22:53:03.293288+0000","last_undegraded":"2024-09-11T22:53:03.293288+0000","last_fullsized":"2024-09-11T22:53:03.293288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:21.158577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68544,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":252,"num_read_kb":192,"num_write":41,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074760+0000","last_change":"2024-09-11T22:50:54.166678+0000","last_active":"2024-09-11T22:52:53.074760+0000","last_peered":"2024-09-11T22:52:53.074760+0000","last_clean":"2024-09-11T22:52:53.074760+0000","last_became_active":"2024-09-11T22:50:54.165777+0000","last_became_peered":"2024-09-11T22:50:54.165777+0000","last_unstale":"2024-09-11T22:52:53.074760+0000","last_undegraded":"2024-09-11T22:52:53.074760+0000","last_fullsized":"2024-09-11T22:52:53.074760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:14:07.224563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129051+0000","last_change":"2024-09-11T22:50:52.141292+0000","last_active":"2024-09-11T22:52:53.129051+0000","last_peered":"2024-09-11T22:52:53.129051+0000","last_clean":"2024-09-11T22:52:53.129051+0000","last_became_active":"2024-09-11T22:50:52.140971+0000","last_became_peered":"2024-09-11T22:50:52.140971+0000","last_unstale":"2024-09-11T22:52:53.129051+0000","last_undegraded":"2024-09-11T22:52:53.129051+0000","last_fullsized":"2024-09-11T22:52:53.129051+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:53.039684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129489+0000","last_change":"2024-09-11T22:50:54.171821+0000","last_active":"2024-09-11T22:52:53.129489+0000","last_peered":"2024-09-11T22:52:53.129489+0000","last_clean":"2024-09-11T22:52:53.129489+0000","last_became_active":"2024-09-11T22:50:54.171520+0000","last_became_peered":"2024-09-11T22:50:54.171520+0000","last_unstale":"2024-09-11T22:52:53.129489+0000","last_undegraded":"2024-09-11T22:52:53.129489+0000","last_fullsized":"2024-09-11T22:52:53.129489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:15:56.594058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073605+0000","last_change":"2024-09-11T22:50:52.138725+0000","last_active":"2024-09-11T22:52:53.073605+0000","last_peered":"2024-09-11T22:52:53.073605+0000","last_clean":"2024-09-11T22:52:53.073605+0000","last_became_active":"2024-09-11T22:50:52.138367+0000","last_became_peered":"2024-09-11T22:50:52.138367+0000","last_unstale":"2024-09-11T22:52:53.073605+0000","last_undegraded":"2024-09-11T22:52:53.073605+0000","last_fullsized":"2024-09-11T22:52:53.073605+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:58.811444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129217+0000","last_change":"2024-09-11T22:50:54.170524+0000","last_active":"2024-09-11T22:52:53.129217+0000","last_peered":"2024-09-11T22:52:53.129217+0000","last_clean":"2024-09-11T22:52:53.129217+0000","last_became_active":"2024-09-11T22:50:54.169947+0000","last_became_peered":"2024-09-11T22:50:54.169947+0000","last_unstale":"2024-09-11T22:52:53.129217+0000","last_undegraded":"2024-09-11T22:52:53.129217+0000","last_fullsized":"2024-09-11T22:52:53.129217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:01:51.768718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":153,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:02.351154+0000","last_change":"2024-09-11T22:50:52.139182+0000","last_active":"2024-09-11T22:53:02.351154+0000","last_peered":"2024-09-11T22:53:02.351154+0000","last_clean":"2024-09-11T22:53:02.351154+0000","last_became_active":"2024-09-11T22:50:52.138799+0000","last_became_peered":"2024-09-11T22:50:52.138799+0000","last_unstale":"2024-09-11T22:53:02.351154+0000","last_undegraded":"2024-09-11T22:53:02.351154+0000","last_fullsized":"2024-09-11T22:53:02.351154+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:53:36.376162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129122+0000","last_change":"2024-09-11T22:50:54.168797+0000","last_active":"2024-09-11T22:52:53.129122+0000","last_peered":"2024-09-11T22:52:53.129122+0000","last_clean":"2024-09-11T22:52:53.129122+0000","last_became_active":"2024-09-11T22:50:54.168408+0000","last_became_peered":"2024-09-11T22:50:54.168408+0000","last_unstale":"2024-09-11T22:52:53.129122+0000","last_undegraded":"2024-09-11T22:52:53.129122+0000","last_fullsized":"2024-09-11T22:52:53.129122+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:15:51.847666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":42,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074207+0000","last_change":"2024-09-11T22:50:52.264693+0000","last_active":"2024-09-11T22:52:53.074207+0000","last_peered":"2024-09-11T22:52:53.074207+0000","last_clean":"2024-09-11T22:52:53.074207+0000","last_became_active":"2024-09-11T22:50:52.264058+0000","last_became_peered":"2024-09-11T22:50:52.264058+0000","last_unstale":"2024-09-11T22:52:53.074207+0000","last_undegraded":"2024-09-11T22:52:53.074207+0000","last_fullsized":"2024-09-11T22:52:53.074207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:46.985921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073438+0000","last_change":"2024-09-11T22:51:55.961118+0000","last_active":"2024-09-11T22:52:53.073438+0000","last_peered":"2024-09-11T22:52:53.073438+0000","last_clean":"2024-09-11T22:52:53.073438+0000","last_became_active":"2024-09-11T22:51:55.960614+0000","last_became_peered":"2024-09-11T22:51:55.960614+0000","last_unstale":"2024-09-11T22:52:53.073438+0000","last_undegraded":"2024-09-11T22:52:53.073438+0000","last_fullsized":"2024-09-11T22:52:53.073438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:47.184327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.075083+0000","last_change":"2024-09-11T22:50:52.133049+0000","last_active":"2024-09-11T22:52:53.075083+0000","last_peered":"2024-09-11T22:52:53.075083+0000","last_clean":"2024-09-11T22:52:53.075083+0000","last_became_active":"2024-09-11T22:50:52.132647+0000","last_became_peered":"2024-09-11T22:50:52.132647+0000","last_unstale":"2024-09-11T22:52:53.075083+0000","last_undegraded":"2024-09-11T22:52:53.075083+0000","last_fullsized":"2024-09-11T22:52:53.075083+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:32:13.213003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.169115+0000","last_change":"2024-09-11T22:51:56.238023+0000","last_active":"2024-09-11T22:52:53.169115+0000","last_peered":"2024-09-11T22:52:53.169115+0000","last_clean":"2024-09-11T22:52:53.169115+0000","last_became_active":"2024-09-11T22:51:56.237676+0000","last_became_peered":"2024-09-11T22:51:56.237676+0000","last_unstale":"2024-09-11T22:52:53.169115+0000","last_undegraded":"2024-09-11T22:52:53.169115+0000","last_fullsized":"2024-09-11T22:52:53.169115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:09:16.215961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,6],"acting":[11,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130047+0000","last_change":"2024-09-11T22:50:52.139766+0000","last_active":"2024-09-11T22:52:53.130047+0000","last_peered":"2024-09-11T22:52:53.130047+0000","last_clean":"2024-09-11T22:52:53.130047+0000","last_became_active":"2024-09-11T22:50:52.139456+0000","last_became_peered":"2024-09-11T22:50:52.139456+0000","last_unstale":"2024-09-11T22:52:53.130047+0000","last_undegraded":"2024-09-11T22:52:53.130047+0000","last_fullsized":"2024-09-11T22:52:53.130047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:31:08.358649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130080+0000","last_change":"2024-09-11T22:50:54.167717+0000","last_active":"2024-09-11T22:52:53.130080+0000","last_peered":"2024-09-11T22:52:53.130080+0000","last_clean":"2024-09-11T22:52:53.130080+0000","last_became_active":"2024-09-11T22:50:54.167331+0000","last_became_peered":"2024-09-11T22:50:54.167331+0000","last_unstale":"2024-09-11T22:52:53.130080+0000","last_undegraded":"2024-09-11T22:52:53.130080+0000","last_fullsized":"2024-09-11T22:52:53.130080+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:11.240351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'67","reported_seq":122,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:02.748201+0000","last_change":"2024-09-11T22:50:52.141682+0000","last_active":"2024-09-11T22:53:02.748201+0000","last_peered":"2024-09-11T22:53:02.748201+0000","last_clean":"2024-09-11T22:53:02.748201+0000","last_became_active":"2024-09-11T22:50:52.141251+0000","last_became_peered":"2024-09-11T22:50:52.141251+0000","last_unstale":"2024-09-11T22:53:02.748201+0000","last_undegraded":"2024-09-11T22:53:02.748201+0000","last_fullsized":"2024-09-11T22:53:02.748201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:03.068252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129793+0000","last_change":"2024-09-11T22:50:54.174149+0000","last_active":"2024-09-11T22:52:53.129793+0000","last_peered":"2024-09-11T22:52:53.129793+0000","last_clean":"2024-09-11T22:52:53.129793+0000","last_became_active":"2024-09-11T22:50:54.173765+0000","last_became_peered":"2024-09-11T22:50:54.173765+0000","last_unstale":"2024-09-11T22:52:53.129793+0000","last_undegraded":"2024-09-11T22:52:53.129793+0000","last_fullsized":"2024-09-11T22:52:53.129793+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:11:03.314334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"84'8","reported_seq":156,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:02.292200+0000","last_change":"2024-09-11T22:50:52.435019+0000","last_active":"2024-09-11T22:53:02.292200+0000","last_peered":"2024-09-11T22:53:02.292200+0000","last_clean":"2024-09-11T22:53:02.292200+0000","last_became_active":"2024-09-11T22:50:52.434642+0000","last_became_peered":"2024-09-11T22:50:52.434642+0000","last_unstale":"2024-09-11T22:53:02.292200+0000","last_undegraded":"2024-09-11T22:53:02.292200+0000","last_fullsized":"2024-09-11T22:53:02.292200+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:53:18.441841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130004+0000","last_change":"2024-09-11T22:50:54.171291+0000","last_active":"2024-09-11T22:52:53.130004+0000","last_peered":"2024-09-11T22:52:53.130004+0000","last_clean":"2024-09-11T22:52:53.130004+0000","last_became_active":"2024-09-11T22:50:54.170961+0000","last_became_peered":"2024-09-11T22:50:54.170961+0000","last_unstale":"2024-09-11T22:52:53.130004+0000","last_undegraded":"2024-09-11T22:52:53.130004+0000","last_fullsized":"2024-09-11T22:52:53.130004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:19:07.211936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"84'148","reported_seq":392,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:55.179683+0000","last_change":"2024-09-11T22:49:39.098881+0000","last_active":"2024-09-11T22:52:55.179683+0000","last_peered":"2024-09-11T22:52:55.179683+0000","last_clean":"2024-09-11T22:52:55.179683+0000","last_became_active":"2024-09-11T22:49:39.098458+0000","last_became_peered":"2024-09-11T22:49:39.098458+0000","last_unstale":"2024-09-11T22:52:55.179683+0000","last_undegraded":"2024-09-11T22:52:55.179683+0000","last_fullsized":"2024-09-11T22:52:55.179683+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:46:11.418577+0000","last_clean_scrub_stamp":"2024-09-11T22:46:11.418577+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:41.059384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130085+0000","last_change":"2024-09-11T22:50:52.137691+0000","last_active":"2024-09-11T22:52:53.130085+0000","last_peered":"2024-09-11T22:52:53.130085+0000","last_clean":"2024-09-11T22:52:53.130085+0000","last_became_active":"2024-09-11T22:50:52.137282+0000","last_became_peered":"2024-09-11T22:50:52.137282+0000","last_unstale":"2024-09-11T22:52:53.130085+0000","last_undegraded":"2024-09-11T22:52:53.130085+0000","last_fullsized":"2024-09-11T22:52:53.130085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:00:27.651252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129789+0000","last_change":"2024-09-11T22:51:55.959937+0000","last_active":"2024-09-11T22:52:53.129789+0000","last_peered":"2024-09-11T22:52:53.129789+0000","last_clean":"2024-09-11T22:52:53.129789+0000","last_became_active":"2024-09-11T22:51:55.959575+0000","last_became_peered":"2024-09-11T22:51:55.959575+0000","last_unstale":"2024-09-11T22:52:53.129789+0000","last_undegraded":"2024-09-11T22:52:53.129789+0000","last_fullsized":"2024-09-11T22:52:53.129789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:57:54.097552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129633+0000","last_change":"2024-09-11T22:50:52.144324+0000","last_active":"2024-09-11T22:52:53.129633+0000","last_peered":"2024-09-11T22:52:53.129633+0000","last_clean":"2024-09-11T22:52:53.129633+0000","last_became_active":"2024-09-11T22:50:52.144011+0000","last_became_peered":"2024-09-11T22:50:52.144011+0000","last_unstale":"2024-09-11T22:52:53.129633+0000","last_undegraded":"2024-09-11T22:52:53.129633+0000","last_fullsized":"2024-09-11T22:52:53.129633+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:29.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130247+0000","last_change":"2024-09-11T22:50:54.173287+0000","last_active":"2024-09-11T22:52:53.130247+0000","last_peered":"2024-09-11T22:52:53.130247+0000","last_clean":"2024-09-11T22:52:53.130247+0000","last_became_active":"2024-09-11T22:50:54.172953+0000","last_became_peered":"2024-09-11T22:50:54.172953+0000","last_unstale":"2024-09-11T22:52:53.130247+0000","last_undegraded":"2024-09-11T22:52:53.130247+0000","last_fullsized":"2024-09-11T22:52:53.130247+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:24.314604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":151,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:02.350556+0000","last_change":"2024-09-11T22:50:52.263777+0000","last_active":"2024-09-11T22:53:02.350556+0000","last_peered":"2024-09-11T22:53:02.350556+0000","last_clean":"2024-09-11T22:53:02.350556+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:53:02.350556+0000","last_undegraded":"2024-09-11T22:53:02.350556+0000","last_fullsized":"2024-09-11T22:53:02.350556+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:37.171410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.073458+0000","last_change":"2024-09-11T22:50:54.164927+0000","last_active":"2024-09-11T22:52:53.073458+0000","last_peered":"2024-09-11T22:52:53.073458+0000","last_clean":"2024-09-11T22:52:53.073458+0000","last_became_active":"2024-09-11T22:50:54.164602+0000","last_became_peered":"2024-09-11T22:50:54.164602+0000","last_unstale":"2024-09-11T22:52:53.073458+0000","last_undegraded":"2024-09-11T22:52:53.073458+0000","last_fullsized":"2024-09-11T22:52:53.073458+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:37:10.226232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":149,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:02.300304+0000","last_change":"2024-09-11T22:50:52.141199+0000","last_active":"2024-09-11T22:53:02.300304+0000","last_peered":"2024-09-11T22:53:02.300304+0000","last_clean":"2024-09-11T22:53:02.300304+0000","last_became_active":"2024-09-11T22:50:52.140830+0000","last_became_peered":"2024-09-11T22:50:52.140830+0000","last_unstale":"2024-09-11T22:53:02.300304+0000","last_undegraded":"2024-09-11T22:53:02.300304+0000","last_fullsized":"2024-09-11T22:53:02.300304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:11.767803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129616+0000","last_change":"2024-09-11T22:50:54.175054+0000","last_active":"2024-09-11T22:52:53.129616+0000","last_peered":"2024-09-11T22:52:53.129616+0000","last_clean":"2024-09-11T22:52:53.129616+0000","last_became_active":"2024-09-11T22:50:54.174715+0000","last_became_peered":"2024-09-11T22:50:54.174715+0000","last_unstale":"2024-09-11T22:52:53.129616+0000","last_undegraded":"2024-09-11T22:52:53.129616+0000","last_fullsized":"2024-09-11T22:52:53.129616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:03:06.694988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130259+0000","last_change":"2024-09-11T22:51:55.961660+0000","last_active":"2024-09-11T22:52:53.130259+0000","last_peered":"2024-09-11T22:52:53.130259+0000","last_clean":"2024-09-11T22:52:53.130259+0000","last_became_active":"2024-09-11T22:51:55.961278+0000","last_became_peered":"2024-09-11T22:51:55.961278+0000","last_unstale":"2024-09-11T22:52:53.130259+0000","last_undegraded":"2024-09-11T22:52:53.130259+0000","last_fullsized":"2024-09-11T22:52:53.130259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:29:52.352490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129410+0000","last_change":"2024-09-11T22:50:54.167361+0000","last_active":"2024-09-11T22:52:53.129410+0000","last_peered":"2024-09-11T22:52:53.129410+0000","last_clean":"2024-09-11T22:52:53.129410+0000","last_became_active":"2024-09-11T22:50:54.166963+0000","last_became_peered":"2024-09-11T22:50:54.166963+0000","last_unstale":"2024-09-11T22:52:53.129410+0000","last_undegraded":"2024-09-11T22:52:53.129410+0000","last_fullsized":"2024-09-11T22:52:53.129410+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:13.294544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.075075+0000","last_change":"2024-09-11T22:50:52.264484+0000","last_active":"2024-09-11T22:52:53.075075+0000","last_peered":"2024-09-11T22:52:53.075075+0000","last_clean":"2024-09-11T22:52:53.075075+0000","last_became_active":"2024-09-11T22:50:52.264038+0000","last_became_peered":"2024-09-11T22:50:52.264038+0000","last_unstale":"2024-09-11T22:52:53.075075+0000","last_undegraded":"2024-09-11T22:52:53.075075+0000","last_fullsized":"2024-09-11T22:52:53.075075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:21.325136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129602+0000","last_change":"2024-09-11T22:50:54.162225+0000","last_active":"2024-09-11T22:52:53.129602+0000","last_peered":"2024-09-11T22:52:53.129602+0000","last_clean":"2024-09-11T22:52:53.129602+0000","last_became_active":"2024-09-11T22:50:54.161234+0000","last_became_peered":"2024-09-11T22:50:54.161234+0000","last_unstale":"2024-09-11T22:52:53.129602+0000","last_undegraded":"2024-09-11T22:52:53.129602+0000","last_fullsized":"2024-09-11T22:52:53.129602+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:50:00.346217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'105","reported_seq":238,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:03.106484+0000","last_change":"2024-09-11T22:51:55.967804+0000","last_active":"2024-09-11T22:53:03.106484+0000","last_peered":"2024-09-11T22:53:03.106484+0000","last_clean":"2024-09-11T22:53:03.106484+0000","last_became_active":"2024-09-11T22:51:55.961136+0000","last_became_peered":"2024-09-11T22:51:55.961136+0000","last_unstale":"2024-09-11T22:53:03.106484+0000","last_undegraded":"2024-09-11T22:53:03.106484+0000","last_fullsized":"2024-09-11T22:53:03.106484+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:41:55.315179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":191,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130644+0000","last_change":"2024-09-11T22:50:54.167307+0000","last_active":"2024-09-11T22:52:53.130644+0000","last_peered":"2024-09-11T22:52:53.130644+0000","last_clean":"2024-09-11T22:52:53.130644+0000","last_became_active":"2024-09-11T22:50:54.166650+0000","last_became_peered":"2024-09-11T22:50:54.166650+0000","last_unstale":"2024-09-11T22:52:53.130644+0000","last_undegraded":"2024-09-11T22:52:53.130644+0000","last_fullsized":"2024-09-11T22:52:53.130644+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:04.863258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130339+0000","last_change":"2024-09-11T22:50:52.435107+0000","last_active":"2024-09-11T22:52:53.130339+0000","last_peered":"2024-09-11T22:52:53.130339+0000","last_clean":"2024-09-11T22:52:53.130339+0000","last_became_active":"2024-09-11T22:50:52.434791+0000","last_became_peered":"2024-09-11T22:50:52.434791+0000","last_unstale":"2024-09-11T22:52:53.130339+0000","last_undegraded":"2024-09-11T22:52:53.130339+0000","last_fullsized":"2024-09-11T22:52:53.130339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:22:11.128791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131456+0000","last_change":"2024-09-11T22:50:54.178041+0000","last_active":"2024-09-11T22:52:53.131456+0000","last_peered":"2024-09-11T22:52:53.131456+0000","last_clean":"2024-09-11T22:52:53.131456+0000","last_became_active":"2024-09-11T22:50:54.177393+0000","last_became_peered":"2024-09-11T22:50:54.177393+0000","last_unstale":"2024-09-11T22:52:53.131456+0000","last_undegraded":"2024-09-11T22:52:53.131456+0000","last_fullsized":"2024-09-11T22:52:53.131456+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:04:21.608104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074146+0000","last_change":"2024-09-11T22:50:52.264051+0000","last_active":"2024-09-11T22:52:53.074146+0000","last_peered":"2024-09-11T22:52:53.074146+0000","last_clean":"2024-09-11T22:52:53.074146+0000","last_became_active":"2024-09-11T22:50:52.263674+0000","last_became_peered":"2024-09-11T22:50:52.263674+0000","last_unstale":"2024-09-11T22:52:53.074146+0000","last_undegraded":"2024-09-11T22:52:53.074146+0000","last_fullsized":"2024-09-11T22:52:53.074146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:44.655218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074263+0000","last_change":"2024-09-11T22:50:54.166409+0000","last_active":"2024-09-11T22:52:53.074263+0000","last_peered":"2024-09-11T22:52:53.074263+0000","last_clean":"2024-09-11T22:52:53.074263+0000","last_became_active":"2024-09-11T22:50:54.166037+0000","last_became_peered":"2024-09-11T22:50:54.166037+0000","last_unstale":"2024-09-11T22:52:53.074263+0000","last_undegraded":"2024-09-11T22:52:53.074263+0000","last_fullsized":"2024-09-11T22:52:53.074263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:22.702543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129632+0000","last_change":"2024-09-11T22:50:52.140218+0000","last_active":"2024-09-11T22:52:53.129632+0000","last_peered":"2024-09-11T22:52:53.129632+0000","last_clean":"2024-09-11T22:52:53.129632+0000","last_became_active":"2024-09-11T22:50:52.139744+0000","last_became_peered":"2024-09-11T22:50:52.139744+0000","last_unstale":"2024-09-11T22:52:53.129632+0000","last_undegraded":"2024-09-11T22:52:53.129632+0000","last_fullsized":"2024-09-11T22:52:53.129632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:21:50.178348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.168909+0000","last_change":"2024-09-11T22:50:54.164059+0000","last_active":"2024-09-11T22:52:53.168909+0000","last_peered":"2024-09-11T22:52:53.168909+0000","last_clean":"2024-09-11T22:52:53.168909+0000","last_became_active":"2024-09-11T22:50:54.163724+0000","last_became_peered":"2024-09-11T22:50:54.163724+0000","last_unstale":"2024-09-11T22:52:53.168909+0000","last_undegraded":"2024-09-11T22:52:53.168909+0000","last_fullsized":"2024-09-11T22:52:53.168909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:57.241660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129028+0000","last_change":"2024-09-11T22:50:52.140169+0000","last_active":"2024-09-11T22:52:53.129028+0000","last_peered":"2024-09-11T22:52:53.129028+0000","last_clean":"2024-09-11T22:52:53.129028+0000","last_became_active":"2024-09-11T22:50:52.139579+0000","last_became_peered":"2024-09-11T22:50:52.139579+0000","last_unstale":"2024-09-11T22:52:53.129028+0000","last_undegraded":"2024-09-11T22:52:53.129028+0000","last_fullsized":"2024-09-11T22:52:53.129028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:50:43.523961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130075+0000","last_change":"2024-09-11T22:50:54.171539+0000","last_active":"2024-09-11T22:52:53.130075+0000","last_peered":"2024-09-11T22:52:53.130075+0000","last_clean":"2024-09-11T22:52:53.130075+0000","last_became_active":"2024-09-11T22:50:54.171013+0000","last_became_peered":"2024-09-11T22:50:54.171013+0000","last_unstale":"2024-09-11T22:52:53.130075+0000","last_undegraded":"2024-09-11T22:52:53.130075+0000","last_fullsized":"2024-09-11T22:52:53.130075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:03.564389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074787+0000","last_change":"2024-09-11T22:50:52.139544+0000","last_active":"2024-09-11T22:52:53.074787+0000","last_peered":"2024-09-11T22:52:53.074787+0000","last_clean":"2024-09-11T22:52:53.074787+0000","last_became_active":"2024-09-11T22:50:52.139201+0000","last_became_peered":"2024-09-11T22:50:52.139201+0000","last_unstale":"2024-09-11T22:52:53.074787+0000","last_undegraded":"2024-09-11T22:52:53.074787+0000","last_fullsized":"2024-09-11T22:52:53.074787+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:38:37.343437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.129337+0000","last_change":"2024-09-11T22:50:54.170052+0000","last_active":"2024-09-11T22:52:53.129337+0000","last_peered":"2024-09-11T22:52:53.129337+0000","last_clean":"2024-09-11T22:52:53.129337+0000","last_became_active":"2024-09-11T22:50:54.169735+0000","last_became_peered":"2024-09-11T22:50:54.169735+0000","last_unstale":"2024-09-11T22:52:53.129337+0000","last_undegraded":"2024-09-11T22:52:53.129337+0000","last_fullsized":"2024-09-11T22:52:53.129337+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:56:44.007747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074181+0000","last_change":"2024-09-11T22:50:54.169959+0000","last_active":"2024-09-11T22:52:53.074181+0000","last_peered":"2024-09-11T22:52:53.074181+0000","last_clean":"2024-09-11T22:52:53.074181+0000","last_became_active":"2024-09-11T22:50:54.169649+0000","last_became_peered":"2024-09-11T22:50:54.169649+0000","last_unstale":"2024-09-11T22:52:53.074181+0000","last_undegraded":"2024-09-11T22:52:53.074181+0000","last_fullsized":"2024-09-11T22:52:53.074181+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:49:51.974258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074520+0000","last_change":"2024-09-11T22:50:52.140881+0000","last_active":"2024-09-11T22:52:53.074520+0000","last_peered":"2024-09-11T22:52:53.074520+0000","last_clean":"2024-09-11T22:52:53.074520+0000","last_became_active":"2024-09-11T22:50:52.140527+0000","last_became_peered":"2024-09-11T22:50:52.140527+0000","last_unstale":"2024-09-11T22:52:53.074520+0000","last_undegraded":"2024-09-11T22:52:53.074520+0000","last_fullsized":"2024-09-11T22:52:53.074520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:14.325816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074263+0000","last_change":"2024-09-11T22:50:54.166571+0000","last_active":"2024-09-11T22:52:53.074263+0000","last_peered":"2024-09-11T22:52:53.074263+0000","last_clean":"2024-09-11T22:52:53.074263+0000","last_became_active":"2024-09-11T22:50:54.166247+0000","last_became_peered":"2024-09-11T22:50:54.166247+0000","last_unstale":"2024-09-11T22:52:53.074263+0000","last_undegraded":"2024-09-11T22:52:53.074263+0000","last_fullsized":"2024-09-11T22:52:53.074263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:02:42.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.074895+0000","last_change":"2024-09-11T22:50:52.140298+0000","last_active":"2024-09-11T22:52:53.074895+0000","last_peered":"2024-09-11T22:52:53.074895+0000","last_clean":"2024-09-11T22:52:53.074895+0000","last_became_active":"2024-09-11T22:50:52.139917+0000","last_became_peered":"2024-09-11T22:50:52.139917+0000","last_unstale":"2024-09-11T22:52:53.074895+0000","last_undegraded":"2024-09-11T22:52:53.074895+0000","last_fullsized":"2024-09-11T22:52:53.074895+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:44:23.599010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.131626+0000","last_change":"2024-09-11T22:50:54.170849+0000","last_active":"2024-09-11T22:52:53.131626+0000","last_peered":"2024-09-11T22:52:53.131626+0000","last_clean":"2024-09-11T22:52:53.131626+0000","last_became_active":"2024-09-11T22:50:54.169547+0000","last_became_peered":"2024-09-11T22:50:54.169547+0000","last_unstale":"2024-09-11T22:52:53.131626+0000","last_undegraded":"2024-09-11T22:52:53.131626+0000","last_fullsized":"2024-09-11T22:52:53.131626+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:25.251158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130596+0000","last_change":"2024-09-11T22:50:52.263778+0000","last_active":"2024-09-11T22:52:53.130596+0000","last_peered":"2024-09-11T22:52:53.130596+0000","last_clean":"2024-09-11T22:52:53.130596+0000","last_became_active":"2024-09-11T22:50:52.263454+0000","last_became_peered":"2024-09-11T22:50:52.263454+0000","last_unstale":"2024-09-11T22:52:53.130596+0000","last_undegraded":"2024-09-11T22:52:53.130596+0000","last_fullsized":"2024-09-11T22:52:53.130596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:03.439147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:52:53.130475+0000","last_change":"2024-09-11T22:50:54.171200+0000","last_active":"2024-09-11T22:52:53.130475+0000","last_peered":"2024-09-11T22:52:53.130475+0000","last_clean":"2024-09-11T22:52:53.130475+0000","last_became_active":"2024-09-11T22:50:54.169320+0000","last_became_peered":"2024-09-11T22:50:54.169320+0000","last_unstale":"2024-09-11T22:52:53.130475+0000","last_undegraded":"2024-09-11T22:52:53.130475+0000","last_fullsized":"2024-09-11T22:52:53.130475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:53.134241+0000","last_clean_scrub_stamp":"2024-09-11T22:50:53.134241+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:14.864699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":158,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-11T22:53:02.299866+0000","last_change":"2024-09-11T22:50:52.139609+0000","last_active":"2024-09-11T22:53:02.299866+0000","last_peered":"2024-09-11T22:53:02.299866+0000","last_clean":"2024-09-11T22:53:02.299866+0000","last_became_active":"2024-09-11T22:50:52.139278+0000","last_became_peered":"2024-09-11T22:50:52.139278+0000","last_unstale":"2024-09-11T22:53:02.299866+0000","last_undegraded":"2024-09-11T22:53:02.299866+0000","last_fullsized":"2024-09-11T22:53:02.299866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:50:51.118132+0000","last_clean_scrub_stamp":"2024-09-11T22:50:51.118132+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:46:09.538734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799447,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1585,"num_read_kb":1014,"num_write":508,"num_write_kb":2384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":2398341,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":386,"ondisk_log_size":386,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107505,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547240,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":840516,"data_compressed":22190,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776172,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1126400,"data_stored":1434964,"data_compressed":53938,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939698,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":839519,"data_compressed":22190,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168630,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":446681,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397564,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":1040914,"data_compressed":38736,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561090,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":446020,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790023,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1048576,"data_stored":1368038,"data_compressed":53938,"data_compressed_allocated":69632,"data_compressed_original":905216,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953548,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":925696,"data_stored":446667,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1011712,"data_stored":1167293,"data_compressed":37392,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":517944,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":995328,"data_stored":515477,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":946176,"data_stored":448286,"data_compressed":2901,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35835,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":332372,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":399298,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":395335,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:53:08.839 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:53:09.093 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:09.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:09 smithi152 ceph-mon[31181]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-11T22:53:09.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:09 smithi152 ceph-mon[31181]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:53:09.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:09 smithi191 ceph-mon[35211]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-11T22:53:09.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:09 smithi191 ceph-mon[35211]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:53:09.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:09 smithi204 ceph-mon[35454]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-09-11T22:53:09.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:09 smithi204 ceph-mon[35454]: from='client.15450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:53:09.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.568+0000 7fd100d63640 1 -- 172.21.15.152:0/3987855154 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0fc100e10 msgr2=0x7fd0fc104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:09.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.568+0000 7fd100d63640 1 --2- 172.21.15.152:0/3987855154 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0fc100e10 0x7fd0fc104e60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009900 tx=0x7fd0f002a410 comp rx=0 tx=0).stop 2024-09-11T22:53:09.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.569+0000 7fd100d63640 1 -- 172.21.15.152:0/3987855154 shutdown_connections 2024-09-11T22:53:09.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.569+0000 7fd100d63640 1 --2- 172.21.15.152:0/3987855154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0fc105590 0x7fd0fc10d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:09.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.569+0000 7fd100d63640 1 --2- 172.21.15.152:0/3987855154 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0fc100e10 0x7fd0fc104e60 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:09.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.569+0000 7fd100d63640 1 --2- 172.21.15.152:0/3987855154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0fc100460 0x7fd0fc100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:09.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.569+0000 7fd100d63640 1 -- 172.21.15.152:0/3987855154 >> 172.21.15.152:0/3987855154 conn(0x7fd0fc0fbee0 msgr2=0x7fd0fc0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:09.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.569+0000 7fd100d63640 1 -- 172.21.15.152:0/3987855154 shutdown_connections 2024-09-11T22:53:09.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.569+0000 7fd100d63640 1 -- 172.21.15.152:0/3987855154 wait complete. 2024-09-11T22:53:09.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.569+0000 7fd100d63640 1 Processor -- start 2024-09-11T22:53:09.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.569+0000 7fd100d63640 1 -- start start 2024-09-11T22:53:09.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.569+0000 7fd100d63640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0fc100460 0x7fd0fc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:09.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.569+0000 7fd100d63640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0fc100e10 0x7fd0fc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:09.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd100d63640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0fc105590 0x7fd0fc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:09.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd100d63640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc10fd70 con 0x7fd0fc105590 2024-09-11T22:53:09.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd100d63640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc10fbf0 con 0x7fd0fc100460 2024-09-11T22:53:09.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd100d63640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc10fef0 con 0x7fd0fc100e10 2024-09-11T22:53:09.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd0fad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0fc105590 0x7fd0fc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:09.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd0fad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0fc105590 0x7fd0fc19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47484/0 (socket says 172.21.15.152:47484) 2024-09-11T22:53:09.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd0fad9d640 1 -- 172.21.15.152:0/3039996534 learned_addr learned my addr 172.21.15.152:0/3039996534 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:09.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd0f9d9b640 1 --2- 172.21.15.152:0/3039996534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0fc100e10 0x7fd0fc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:09.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd0fa59c640 1 --2- 172.21.15.152:0/3039996534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0fc100460 0x7fd0fc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:09.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd0fad9d640 1 -- 172.21.15.152:0/3039996534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0fc100e10 msgr2=0x7fd0fc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:09.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd0fad9d640 1 --2- 172.21.15.152:0/3039996534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0fc100e10 0x7fd0fc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:09.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd0fad9d640 1 -- 172.21.15.152:0/3039996534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0fc100460 msgr2=0x7fd0fc1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:09.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd0fad9d640 1 --2- 172.21.15.152:0/3039996534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0fc100460 0x7fd0fc1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:09.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd0fad9d640 1 -- 172.21.15.152:0/3039996534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0fc19d380 con 0x7fd0fc105590 2024-09-11T22:53:09.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd0fad9d640 1 --2- 172.21.15.152:0/3039996534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0fc105590 0x7fd0fc19cca0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400c970 tx=0x7fd0f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:09.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd0eb7fe640 1 -- 172.21.15.152:0/3039996534 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4007bb0 con 0x7fd0fc105590 2024-09-11T22:53:09.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd100d63640 1 -- 172.21.15.152:0/3039996534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0fc19d670 con 0x7fd0fc105590 2024-09-11T22:53:09.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd0eb7fe640 1 -- 172.21.15.152:0/3039996534 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f4007d50 con 0x7fd0fc105590 2024-09-11T22:53:09.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd0eb7fe640 1 -- 172.21.15.152:0/3039996534 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4018800 con 0x7fd0fc105590 2024-09-11T22:53:09.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.570+0000 7fd100d63640 1 -- 172.21.15.152:0/3039996534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0fc1a4f50 con 0x7fd0fc105590 2024-09-11T22:53:09.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.571+0000 7fd100d63640 1 -- 172.21.15.152:0/3039996534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0fc100840 con 0x7fd0fc105590 2024-09-11T22:53:09.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.573+0000 7fd0eb7fe640 1 -- 172.21.15.152:0/3039996534 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7fd0f400fa50 con 0x7fd0fc105590 2024-09-11T22:53:09.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.573+0000 7fd0eb7fe640 1 --2- 172.21.15.152:0/3039996534 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd0e40a2d80 0x7fd0e40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:09.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.573+0000 7fd0eb7fe640 1 -- 172.21.15.152:0/3039996534 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fd0f40cf5c0 con 0x7fd0fc105590 2024-09-11T22:53:09.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.573+0000 7fd0fa59c640 1 --2- 172.21.15.152:0/3039996534 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd0e40a2d80 0x7fd0e40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:09.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.574+0000 7fd0fa59c640 1 --2- 172.21.15.152:0/3039996534 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd0e40a2d80 0x7fd0e40a5240 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec005c40 tx=0x7fd0ec005bd0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:09.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.577+0000 7fd0eb7fe640 1 -- 172.21.15.152:0/3039996534 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd0f4014030 con 0x7fd0fc105590 2024-09-11T22:53:09.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.779+0000 7fd100d63640 1 -- 172.21.15.152:0/3039996534 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0fc02d7c0 con 0x7fd0fc105590 2024-09-11T22:53:09.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.780+0000 7fd0eb7fe640 1 -- 172.21.15.152:0/3039996534 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7fd0f409c080 con 0x7fd0fc105590 2024-09-11T22:53:09.781 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:53:09.781 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:53:09.781 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:53:09.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.782+0000 7fd100d63640 1 -- 172.21.15.152:0/3039996534 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd0e40a2d80 msgr2=0x7fd0e40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:09.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.782+0000 7fd100d63640 1 --2- 172.21.15.152:0/3039996534 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd0e40a2d80 0x7fd0e40a5240 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec005c40 tx=0x7fd0ec005bd0 comp rx=0 tx=0).stop 2024-09-11T22:53:09.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.782+0000 7fd100d63640 1 -- 172.21.15.152:0/3039996534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0fc105590 msgr2=0x7fd0fc19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:09.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.782+0000 7fd100d63640 1 --2- 172.21.15.152:0/3039996534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0fc105590 0x7fd0fc19cca0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400c970 tx=0x7fd0f400ce30 comp rx=0 tx=0).stop 2024-09-11T22:53:09.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.782+0000 7fd100d63640 1 -- 172.21.15.152:0/3039996534 shutdown_connections 2024-09-11T22:53:09.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.782+0000 7fd100d63640 1 --2- 172.21.15.152:0/3039996534 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd0e40a2d80 0x7fd0e40a5240 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:09.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.782+0000 7fd100d63640 1 --2- 172.21.15.152:0/3039996534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd0fc105590 0x7fd0fc19cca0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:09.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.782+0000 7fd100d63640 1 --2- 172.21.15.152:0/3039996534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0fc100e10 0x7fd0fc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:09.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.782+0000 7fd100d63640 1 --2- 172.21.15.152:0/3039996534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd0fc100460 0x7fd0fc1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:09.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.782+0000 7fd100d63640 1 -- 172.21.15.152:0/3039996534 >> 172.21.15.152:0/3039996534 conn(0x7fd0fc0fbee0 msgr2=0x7fd0fc0fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:09.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.782+0000 7fd100d63640 1 -- 172.21.15.152:0/3039996534 shutdown_connections 2024-09-11T22:53:09.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:09.782+0000 7fd100d63640 1 -- 172.21.15.152:0/3039996534 wait complete. 2024-09-11T22:53:10.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:10 smithi152 ceph-mon[31181]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 51 op/s 2024-09-11T22:53:10.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:10 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3039996534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:53:10.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:10 smithi191 ceph-mon[35211]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 51 op/s 2024-09-11T22:53:10.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:10 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3039996534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:53:10.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:10 smithi204 ceph-mon[35454]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 76 KiB/s wr, 51 op/s 2024-09-11T22:53:10.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:10 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3039996534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:53:10.498 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:53:10.744 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:11.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.229+0000 7f8c8a94f640 1 -- 172.21.15.152:0/3086280237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c84100a80 msgr2=0x7f8c84100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:11.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.229+0000 7f8c8a94f640 1 --2- 172.21.15.152:0/3086280237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c84100a80 0x7f8c84100ee0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009930 tx=0x7f8c6c02a420 comp rx=0 tx=0).stop 2024-09-11T22:53:11.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.229+0000 7f8c8a94f640 1 -- 172.21.15.152:0/3086280237 shutdown_connections 2024-09-11T22:53:11.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.229+0000 7f8c8a94f640 1 --2- 172.21.15.152:0/3086280237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c84101420 0x7f8c8410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:11.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.229+0000 7f8c8a94f640 1 --2- 172.21.15.152:0/3086280237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c84100a80 0x7f8c84100ee0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:11.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.229+0000 7f8c8a94f640 1 --2- 172.21.15.152:0/3086280237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c84106a80 0x7f8c84106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:11.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.229+0000 7f8c8a94f640 1 -- 172.21.15.152:0/3086280237 >> 172.21.15.152:0/3086280237 conn(0x7f8c840fc7a0 msgr2=0x7f8c840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:11.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.229+0000 7f8c8a94f640 1 -- 172.21.15.152:0/3086280237 shutdown_connections 2024-09-11T22:53:11.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.229+0000 7f8c8a94f640 1 -- 172.21.15.152:0/3086280237 wait complete. 2024-09-11T22:53:11.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.230+0000 7f8c8a94f640 1 Processor -- start 2024-09-11T22:53:11.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.230+0000 7f8c8a94f640 1 -- start start 2024-09-11T22:53:11.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.230+0000 7f8c8a94f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c84100a80 0x7f8c84199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:11.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.230+0000 7f8c8a94f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c84101420 0x7f8c841997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:11.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.230+0000 7f8c8a94f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c84106a80 0x7f8c841a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:11.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.230+0000 7f8c8a94f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84110ea0 con 0x7f8c84106a80 2024-09-11T22:53:11.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.230+0000 7f8c8a94f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84110d20 con 0x7f8c84100a80 2024-09-11T22:53:11.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.230+0000 7f8c8a94f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84111020 con 0x7f8c84101420 2024-09-11T22:53:11.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.231+0000 7f8c88eec640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c84106a80 0x7f8c841a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:11.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.231+0000 7f8c88eec640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c84106a80 0x7f8c841a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47508/0 (socket says 172.21.15.152:47508) 2024-09-11T22:53:11.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.231+0000 7f8c88eec640 1 -- 172.21.15.152:0/2901975976 learned_addr learned my addr 172.21.15.152:0/2901975976 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:11.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.231+0000 7f8c83fff640 1 --2- 172.21.15.152:0/2901975976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c84100a80 0x7f8c84199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:11.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.231+0000 7f8c7bfff640 1 --2- 172.21.15.152:0/2901975976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c84101420 0x7f8c841997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:11.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.231+0000 7f8c88eec640 1 -- 172.21.15.152:0/2901975976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c84101420 msgr2=0x7f8c841997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:11.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.231+0000 7f8c88eec640 1 --2- 172.21.15.152:0/2901975976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c84101420 0x7f8c841997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:11.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.231+0000 7f8c88eec640 1 -- 172.21.15.152:0/2901975976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c84100a80 msgr2=0x7f8c84199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:11.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.231+0000 7f8c88eec640 1 --2- 172.21.15.152:0/2901975976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c84100a80 0x7f8c84199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:11.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.231+0000 7f8c88eec640 1 -- 172.21.15.152:0/2901975976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c841a4850 con 0x7f8c84106a80 2024-09-11T22:53:11.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.231+0000 7f8c88eec640 1 --2- 172.21.15.152:0/2901975976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c84106a80 0x7f8c841a4310 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00b970 tx=0x7f8c7c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:11.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.231+0000 7f8c81ffb640 1 -- 172.21.15.152:0/2901975976 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c019070 con 0x7f8c84106a80 2024-09-11T22:53:11.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.231+0000 7f8c81ffb640 1 -- 172.21.15.152:0/2901975976 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c014440 con 0x7f8c84106a80 2024-09-11T22:53:11.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.231+0000 7f8c8a94f640 1 -- 172.21.15.152:0/2901975976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c841a4a20 con 0x7f8c84106a80 2024-09-11T22:53:11.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.231+0000 7f8c81ffb640 1 -- 172.21.15.152:0/2901975976 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0075d0 con 0x7f8c84106a80 2024-09-11T22:53:11.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.231+0000 7f8c8a94f640 1 -- 172.21.15.152:0/2901975976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c841a4ee0 con 0x7f8c84106a80 2024-09-11T22:53:11.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.232+0000 7f8c8a94f640 1 -- 172.21.15.152:0/2901975976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c84100ee0 con 0x7f8c84106a80 2024-09-11T22:53:11.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.234+0000 7f8c81ffb640 1 -- 172.21.15.152:0/2901975976 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f8c7c01c070 con 0x7f8c84106a80 2024-09-11T22:53:11.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.234+0000 7f8c81ffb640 1 --2- 172.21.15.152:0/2901975976 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8c700a2d80 0x7f8c700a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:11.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.234+0000 7f8c83fff640 1 --2- 172.21.15.152:0/2901975976 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8c700a2d80 0x7f8c700a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:11.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.234+0000 7f8c81ffb640 1 -- 172.21.15.152:0/2901975976 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f8c7c0cea10 con 0x7f8c84106a80 2024-09-11T22:53:11.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.235+0000 7f8c83fff640 1 --2- 172.21.15.152:0/2901975976 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8c700a2d80 0x7f8c700a5240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400abe0 tx=0x7f8c7400a2d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:11.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.239+0000 7f8c81ffb640 1 -- 172.21.15.152:0/2901975976 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c7c09afd0 con 0x7f8c84106a80 2024-09-11T22:53:11.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.441+0000 7f8c8a94f640 1 -- 172.21.15.152:0/2901975976 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c841a5170 con 0x7f8c84106a80 2024-09-11T22:53:11.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.442+0000 7f8c81ffb640 1 -- 172.21.15.152:0/2901975976 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f8c7c09f2d0 con 0x7f8c84106a80 2024-09-11T22:53:11.443 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:53:11.443 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:53:11.443 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:53:11.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.444+0000 7f8c8a94f640 1 -- 172.21.15.152:0/2901975976 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8c700a2d80 msgr2=0x7f8c700a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:11.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.444+0000 7f8c8a94f640 1 --2- 172.21.15.152:0/2901975976 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8c700a2d80 0x7f8c700a5240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400abe0 tx=0x7f8c7400a2d0 comp rx=0 tx=0).stop 2024-09-11T22:53:11.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.444+0000 7f8c8a94f640 1 -- 172.21.15.152:0/2901975976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c84106a80 msgr2=0x7f8c841a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:11.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.444+0000 7f8c8a94f640 1 --2- 172.21.15.152:0/2901975976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c84106a80 0x7f8c841a4310 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00b970 tx=0x7f8c7c00bd70 comp rx=0 tx=0).stop 2024-09-11T22:53:11.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.444+0000 7f8c8a94f640 1 -- 172.21.15.152:0/2901975976 shutdown_connections 2024-09-11T22:53:11.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.444+0000 7f8c8a94f640 1 --2- 172.21.15.152:0/2901975976 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8c700a2d80 0x7f8c700a5240 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:11.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.444+0000 7f8c8a94f640 1 --2- 172.21.15.152:0/2901975976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c84106a80 0x7f8c841a4310 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:11.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.444+0000 7f8c8a94f640 1 --2- 172.21.15.152:0/2901975976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c84101420 0x7f8c841997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:11.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.444+0000 7f8c8a94f640 1 --2- 172.21.15.152:0/2901975976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c84100a80 0x7f8c84199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:11.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.444+0000 7f8c8a94f640 1 -- 172.21.15.152:0/2901975976 >> 172.21.15.152:0/2901975976 conn(0x7f8c840fc7a0 msgr2=0x7f8c840fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:11.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.444+0000 7f8c8a94f640 1 -- 172.21.15.152:0/2901975976 shutdown_connections 2024-09-11T22:53:11.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:11.445+0000 7f8c8a94f640 1 -- 172.21.15.152:0/2901975976 wait complete. 2024-09-11T22:53:12.137 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:53:12.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:12 smithi152 ceph-mon[31181]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 KiB/s wr, 54 op/s 2024-09-11T22:53:12.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:12 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2901975976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:53:12.388 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:12.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:12 smithi191 ceph-mon[35211]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 KiB/s wr, 54 op/s 2024-09-11T22:53:12.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:12 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2901975976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:53:12.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:12 smithi204 ceph-mon[35454]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 76 KiB/s wr, 54 op/s 2024-09-11T22:53:12.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:12 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2901975976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:53:12.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.921+0000 7f7e07514640 1 -- 172.21.15.152:0/3287152248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e00100ca0 msgr2=0x7f7e001067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:12.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.921+0000 7f7e07514640 1 --2- 172.21.15.152:0/3287152248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e00100ca0 0x7f7e001067e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009900 tx=0x7f7dfc02a410 comp rx=0 tx=0).stop 2024-09-11T22:53:12.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.921+0000 7f7e07514640 1 -- 172.21.15.152:0/3287152248 shutdown_connections 2024-09-11T22:53:12.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.921+0000 7f7e07514640 1 --2- 172.21.15.152:0/3287152248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e00106f10 0x7f7e0010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:12.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.921+0000 7f7e07514640 1 --2- 172.21.15.152:0/3287152248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e00100ca0 0x7f7e001067e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:12.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.921+0000 7f7e07514640 1 --2- 172.21.15.152:0/3287152248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e001002f0 0x7f7e001006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:12.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.921+0000 7f7e07514640 1 -- 172.21.15.152:0/3287152248 >> 172.21.15.152:0/3287152248 conn(0x7f7e000fbee0 msgr2=0x7f7e000fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:12.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.921+0000 7f7e07514640 1 -- 172.21.15.152:0/3287152248 shutdown_connections 2024-09-11T22:53:12.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.921+0000 7f7e07514640 1 -- 172.21.15.152:0/3287152248 wait complete. 2024-09-11T22:53:12.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e07514640 1 Processor -- start 2024-09-11T22:53:12.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e07514640 1 -- start start 2024-09-11T22:53:12.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e07514640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e001002f0 0x7f7e00101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:12.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e07514640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e00100ca0 0x7f7e001017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:12.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e07514640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e00106f10 0x7f7e00101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:12.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e07514640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7e00111f90 con 0x7f7e00106f10 2024-09-11T22:53:12.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e07514640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7e00111e10 con 0x7f7e001002f0 2024-09-11T22:53:12.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e07514640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7e00112110 con 0x7f7e00100ca0 2024-09-11T22:53:12.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e05ab1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e00106f10 0x7f7e00101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:12.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e05ab1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e00106f10 0x7f7e00101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47520/0 (socket says 172.21.15.152:47520) 2024-09-11T22:53:12.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e05ab1640 1 -- 172.21.15.152:0/4076169950 learned_addr learned my addr 172.21.15.152:0/4076169950 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:12.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e052b0640 1 --2- 172.21.15.152:0/4076169950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e001002f0 0x7f7e00101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:12.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e04aaf640 1 --2- 172.21.15.152:0/4076169950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e00100ca0 0x7f7e001017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:12.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e05ab1640 1 -- 172.21.15.152:0/4076169950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e00100ca0 msgr2=0x7f7e001017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:12.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e05ab1640 1 --2- 172.21.15.152:0/4076169950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e00100ca0 0x7f7e001017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:12.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e05ab1640 1 -- 172.21.15.152:0/4076169950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e001002f0 msgr2=0x7f7e00101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:12.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e05ab1640 1 --2- 172.21.15.152:0/4076169950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e001002f0 0x7f7e00101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:12.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e05ab1640 1 -- 172.21.15.152:0/4076169950 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e001a6970 con 0x7f7e00106f10 2024-09-11T22:53:12.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.922+0000 7f7e04aaf640 1 --2- 172.21.15.152:0/4076169950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e00100ca0 0x7f7e001017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:12.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.923+0000 7f7e05ab1640 1 --2- 172.21.15.152:0/4076169950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e00106f10 0x7f7e00101cf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7df800eaa0 tx=0x7f7df800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:12.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.923+0000 7f7df67fc640 1 -- 172.21.15.152:0/4076169950 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df8004d20 con 0x7f7e00106f10 2024-09-11T22:53:12.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.923+0000 7f7df67fc640 1 -- 172.21.15.152:0/4076169950 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df8005020 con 0x7f7e00106f10 2024-09-11T22:53:12.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.923+0000 7f7df67fc640 1 -- 172.21.15.152:0/4076169950 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df80116d0 con 0x7f7e00106f10 2024-09-11T22:53:12.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.923+0000 7f7e07514640 1 -- 172.21.15.152:0/4076169950 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e001a6c60 con 0x7f7e00106f10 2024-09-11T22:53:12.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.923+0000 7f7e07514640 1 -- 172.21.15.152:0/4076169950 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e001a71a0 con 0x7f7e00106f10 2024-09-11T22:53:12.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.924+0000 7f7e07514640 1 -- 172.21.15.152:0/4076169950 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e001006d0 con 0x7f7e00106f10 2024-09-11T22:53:12.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.925+0000 7f7df67fc640 1 -- 172.21.15.152:0/4076169950 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f7df80051d0 con 0x7f7e00106f10 2024-09-11T22:53:12.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.926+0000 7f7df67fc640 1 --2- 172.21.15.152:0/4076169950 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7dec0a2e00 0x7f7dec0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:12.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.926+0000 7f7e052b0640 1 --2- 172.21.15.152:0/4076169950 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7dec0a2e00 0x7f7dec0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:12.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.926+0000 7f7df67fc640 1 -- 172.21.15.152:0/4076169950 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7df80cfb90 con 0x7f7e00106f10 2024-09-11T22:53:12.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.926+0000 7f7e052b0640 1 --2- 172.21.15.152:0/4076169950 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7dec0a2e00 0x7f7dec0a52c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009990 tx=0x7f7df0009110 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:12.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:12.929+0000 7f7df67fc640 1 -- 172.21.15.152:0/4076169950 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7df8016030 con 0x7f7e00106f10 2024-09-11T22:53:13.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:13.097+0000 7f7e07514640 1 -- 172.21.15.152:0/4076169950 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e0002d7c0 con 0x7f7e00106f10 2024-09-11T22:53:13.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:13.098+0000 7f7df67fc640 1 -- 172.21.15.152:0/4076169950 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+23094 (secure 0 0 0) 0x7f7df809c6d0 con 0x7f7e00106f10 2024-09-11T22:53:13.101 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:53:13.103 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":84,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","created":"2024-09-11T22:43:00.379317+0000","modified":"2024-09-11T22:52:53.067599+0000","last_up_change":"2024-09-11T22:50:01.043181+0000","last_in_change":"2024-09-11T22:49:38.073870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:46:10.984707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:50:50.833547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:50:53.047604+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3cd917b5-77f6-4a4a-8fd0-4ebe3034feac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6803","nonce":1270728007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6805","nonce":1270728007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6809","nonce":1270728007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6807","nonce":1270728007}]},"public_addr":"172.21.15.152:6803/1270728007","cluster_addr":"172.21.15.152:6805/1270728007","heartbeat_back_addr":"172.21.15.152:6809/1270728007","heartbeat_front_addr":"172.21.15.152:6807/1270728007","state":["exists","up"]},{"osd":1,"uuid":"ae589bc4-77b1-4522-9591-b0ce2a5432dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6801","nonce":354467155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6803","nonce":354467155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6807","nonce":354467155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6805","nonce":354467155}]},"public_addr":"172.21.15.191:6801/354467155","cluster_addr":"172.21.15.191:6803/354467155","heartbeat_back_addr":"172.21.15.191:6807/354467155","heartbeat_front_addr":"172.21.15.191:6805/354467155","state":["exists","up"]},{"osd":2,"uuid":"b35ffd0c-d084-43dd-a947-15132d78d59c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6801","nonce":3848396211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6803","nonce":3848396211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6807","nonce":3848396211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6805","nonce":3848396211}]},"public_addr":"172.21.15.204:6801/3848396211","cluster_addr":"172.21.15.204:6803/3848396211","heartbeat_back_addr":"172.21.15.204:6807/3848396211","heartbeat_front_addr":"172.21.15.204:6805/3848396211","state":["exists","up"]},{"osd":3,"uuid":"d66ba7e8-4ab7-449e-9460-2f44ffdf987f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6811","nonce":3351257181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6813","nonce":3351257181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6817","nonce":3351257181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6815","nonce":3351257181}]},"public_addr":"172.21.15.152:6811/3351257181","cluster_addr":"172.21.15.152:6813/3351257181","heartbeat_back_addr":"172.21.15.152:6817/3351257181","heartbeat_front_addr":"172.21.15.152:6815/3351257181","state":["exists","up"]},{"osd":4,"uuid":"e1b56468-6cc7-4d29-988e-9d281118ee36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6809","nonce":2967539563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6811","nonce":2967539563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6815","nonce":2967539563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6813","nonce":2967539563}]},"public_addr":"172.21.15.191:6809/2967539563","cluster_addr":"172.21.15.191:6811/2967539563","heartbeat_back_addr":"172.21.15.191:6815/2967539563","heartbeat_front_addr":"172.21.15.191:6813/2967539563","state":["exists","up"]},{"osd":5,"uuid":"89dabd3e-77e3-4294-9648-b112ef694e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6809","nonce":4283562158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6811","nonce":4283562158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6815","nonce":4283562158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6813","nonce":4283562158}]},"public_addr":"172.21.15.204:6809/4283562158","cluster_addr":"172.21.15.204:6811/4283562158","heartbeat_back_addr":"172.21.15.204:6815/4283562158","heartbeat_front_addr":"172.21.15.204:6813/4283562158","state":["exists","up"]},{"osd":6,"uuid":"46a0fded-b668-4aef-a68f-613cccc83483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6819","nonce":2484234042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6821","nonce":2484234042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6825","nonce":2484234042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6823","nonce":2484234042}]},"public_addr":"172.21.15.152:6819/2484234042","cluster_addr":"172.21.15.152:6821/2484234042","heartbeat_back_addr":"172.21.15.152:6825/2484234042","heartbeat_front_addr":"172.21.15.152:6823/2484234042","state":["exists","up"]},{"osd":7,"uuid":"e674bfe9-e2db-44a3-b74b-a21f198747c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6817","nonce":269608859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6819","nonce":269608859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6823","nonce":269608859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6821","nonce":269608859}]},"public_addr":"172.21.15.191:6817/269608859","cluster_addr":"172.21.15.191:6819/269608859","heartbeat_back_addr":"172.21.15.191:6823/269608859","heartbeat_front_addr":"172.21.15.191:6821/269608859","state":["exists","up"]},{"osd":8,"uuid":"0418bfc9-d6b6-488f-b4c7-0bc6291bb624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6817","nonce":1756190090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6819","nonce":1756190090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6823","nonce":1756190090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6821","nonce":1756190090}]},"public_addr":"172.21.15.204:6817/1756190090","cluster_addr":"172.21.15.204:6819/1756190090","heartbeat_back_addr":"172.21.15.204:6823/1756190090","heartbeat_front_addr":"172.21.15.204:6821/1756190090","state":["exists","up"]},{"osd":9,"uuid":"751a6964-798f-4240-a2ba-7fa9f0c72657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6827","nonce":1301191419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6829","nonce":1301191419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6833","nonce":1301191419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6831","nonce":1301191419}]},"public_addr":"172.21.15.152:6827/1301191419","cluster_addr":"172.21.15.152:6829/1301191419","heartbeat_back_addr":"172.21.15.152:6833/1301191419","heartbeat_front_addr":"172.21.15.152:6831/1301191419","state":["exists","up"]},{"osd":10,"uuid":"90e7c6ba-127c-4b33-b3c6-cedf72128451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6825","nonce":3695294503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6827","nonce":3695294503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6831","nonce":3695294503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6829","nonce":3695294503}]},"public_addr":"172.21.15.191:6825/3695294503","cluster_addr":"172.21.15.191:6827/3695294503","heartbeat_back_addr":"172.21.15.191:6831/3695294503","heartbeat_front_addr":"172.21.15.191:6829/3695294503","state":["exists","up"]},{"osd":11,"uuid":"c76a93b1-a045-477b-98d0-58a2e9d44a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6825","nonce":1709433236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6827","nonce":1709433236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6831","nonce":1709433236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6829","nonce":1709433236}]},"public_addr":"172.21.15.204:6825/1709433236","cluster_addr":"172.21.15.204:6827/1709433236","heartbeat_back_addr":"172.21.15.204:6831/1709433236","heartbeat_front_addr":"172.21.15.204:6829/1709433236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:17.259744+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:42.230389+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:07.172433+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:32.517060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:58.900615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:24.379803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:50.058928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:15.967299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:42.025873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:08.225448+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:33.910531+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:59.466423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:0/2259758812":"2024-09-12T22:52:53.067551+0000","172.21.15.152:0/3900732888":"2024-09-12T22:52:53.067551+0000","172.21.15.152:0/2557435316":"2024-09-12T22:52:53.067551+0000","172.21.15.152:0/3960601922":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6800/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6801/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/4286398432":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2270507277":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2452559014":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/3071755865":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6800/3438861948":"2024-09-12T22:52:53.067551+0000","172.21.15.152:6800/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/1721817189":"2024-09-12T22:52:53.067551+0000","172.21.15.152:0/1542993850":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/3438861948":"2024-09-12T22:52:53.067551+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:53:13.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:13.101+0000 7f7e07514640 1 -- 172.21.15.152:0/4076169950 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7dec0a2e00 msgr2=0x7f7dec0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:13.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:13.101+0000 7f7e07514640 1 --2- 172.21.15.152:0/4076169950 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7dec0a2e00 0x7f7dec0a52c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009990 tx=0x7f7df0009110 comp rx=0 tx=0).stop 2024-09-11T22:53:13.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:13.101+0000 7f7e07514640 1 -- 172.21.15.152:0/4076169950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e00106f10 msgr2=0x7f7e00101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:13.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:13.101+0000 7f7e07514640 1 --2- 172.21.15.152:0/4076169950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e00106f10 0x7f7e00101cf0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7df800eaa0 tx=0x7f7df800eea0 comp rx=0 tx=0).stop 2024-09-11T22:53:13.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:13.101+0000 7f7e07514640 1 -- 172.21.15.152:0/4076169950 shutdown_connections 2024-09-11T22:53:13.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:13.101+0000 7f7e07514640 1 --2- 172.21.15.152:0/4076169950 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7dec0a2e00 0x7f7dec0a52c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:13.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:13.101+0000 7f7e07514640 1 --2- 172.21.15.152:0/4076169950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e00106f10 0x7f7e00101cf0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:13.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:13.101+0000 7f7e07514640 1 --2- 172.21.15.152:0/4076169950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e00100ca0 0x7f7e001017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:13.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:13.101+0000 7f7e07514640 1 --2- 172.21.15.152:0/4076169950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7e001002f0 0x7f7e00101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:13.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:13.101+0000 7f7e07514640 1 -- 172.21.15.152:0/4076169950 >> 172.21.15.152:0/4076169950 conn(0x7f7e000fbee0 msgr2=0x7f7e000fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:13.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:13.101+0000 7f7e07514640 1 -- 172.21.15.152:0/4076169950 shutdown_connections 2024-09-11T22:53:13.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:13.101+0000 7f7e07514640 1 -- 172.21.15.152:0/4076169950 wait complete. 2024-09-11T22:53:13.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:13 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T22:53:13.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:13 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4076169950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:53:13.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:13 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T22:53:13.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:13 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/4076169950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:53:13.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:13 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T22:53:13.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:13 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/4076169950' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:53:13.801 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-11T22:53:14.053 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:14.469 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:14 smithi152 ceph-mon[31181]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-11T22:53:14.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:14 smithi191 ceph-mon[35211]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-11T22:53:14.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:14 smithi204 ceph-mon[35454]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-11T22:53:15.278 INFO:teuthology.orchestra.run.smithi152.stdout:{"run_dir":"/var/run/ceph"} 2024-09-11T22:53:15.278 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-11T22:53:15.278 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 481 2024-09-11T22:53:15.278 DEBUG:tasks.quiescer:Yielding 2024-09-11T22:53:15.279 INFO:teuthology.run_tasks:Running task workunit... 2024-09-11T22:53:15.294 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-11T22:53:15.294 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-11T22:53:15.294 INFO:tasks.workunit:timeout=3h 2024-09-11T22:53:15.294 INFO:tasks.workunit:cleanup=True 2024-09-11T22:53:15.295 DEBUG:teuthology.orchestra.run.smithi152:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-11T22:53:15.295 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:53:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-11T22:53:15.312 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-11T22:53:15.312 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 6ch/108d Inode: 1099511628283 Links: 2 2024-09-11T22:53:15.312 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:53:15.312 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-11T22:53:15.312 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-09-11 22:51:13.319473607 +0000 2024-09-11T22:53:15.312 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-09-11 22:51:13.319473607 +0000 2024-09-11T22:53:15.313 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-09-11 22:52:51.355342702 +0000 2024-09-11T22:53:15.313 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-09-11 22:51:13.319473607 +0000 2024-09-11T22:53:15.313 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-11T22:53:15.313 DEBUG:teuthology.orchestra.run.smithi152:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-11T22:53:15.385 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-11T22:53:15.442 INFO:tasks.workunit.client.0.smithi152.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-11T22:53:15.554 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:16.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.024+0000 7fdcca0a8640 1 -- 172.21.15.152:0/3980575207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcc4100e20 msgr2=0x7fdcc4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:16.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.024+0000 7fdcca0a8640 1 --2- 172.21.15.152:0/3980575207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcc4100e20 0x7fdcc4101200 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009860 tx=0x7fdcb402a3d0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.024+0000 7fdcca0a8640 1 -- 172.21.15.152:0/3980575207 shutdown_connections 2024-09-11T22:53:16.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.024+0000 7fdcca0a8640 1 --2- 172.21.15.152:0/3980575207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcc4102ef0 0x7fdcc410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.024+0000 7fdcca0a8640 1 --2- 172.21.15.152:0/3980575207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcc4102570 0x7fdcc41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.024+0000 7fdcca0a8640 1 --2- 172.21.15.152:0/3980575207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcc4100e20 0x7fdcc4101200 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.024+0000 7fdcca0a8640 1 -- 172.21.15.152:0/3980575207 >> 172.21.15.152:0/3980575207 conn(0x7fdcc40fc7d0 msgr2=0x7fdcc40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:16.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.024+0000 7fdcca0a8640 1 -- 172.21.15.152:0/3980575207 shutdown_connections 2024-09-11T22:53:16.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.024+0000 7fdcca0a8640 1 -- 172.21.15.152:0/3980575207 wait complete. 2024-09-11T22:53:16.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.025+0000 7fdcca0a8640 1 Processor -- start 2024-09-11T22:53:16.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.025+0000 7fdcca0a8640 1 -- start start 2024-09-11T22:53:16.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.025+0000 7fdcca0a8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcc4100e20 0x7fdcc4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.025+0000 7fdcca0a8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcc4102570 0x7fdcc4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.025+0000 7fdcca0a8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcc4102ef0 0x7fdcc419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.025+0000 7fdcca0a8640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4110f80 con 0x7fdcc4102ef0 2024-09-11T22:53:16.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.025+0000 7fdcca0a8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4110e00 con 0x7fdcc4102570 2024-09-11T22:53:16.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.025+0000 7fdcca0a8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4111100 con 0x7fdcc4100e20 2024-09-11T22:53:16.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.026+0000 7fdcc3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcc4102ef0 0x7fdcc419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.026+0000 7fdcc3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcc4102ef0 0x7fdcc419cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47534/0 (socket says 172.21.15.152:47534) 2024-09-11T22:53:16.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.026+0000 7fdcc3fff640 1 -- 172.21.15.152:0/2520071661 learned_addr learned my addr 172.21.15.152:0/2520071661 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:16.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.026+0000 7fdcc37fe640 1 --2- 172.21.15.152:0/2520071661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcc4100e20 0x7fdcc4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.026+0000 7fdcc2ffd640 1 --2- 172.21.15.152:0/2520071661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcc4102570 0x7fdcc4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.026+0000 7fdcc3fff640 1 -- 172.21.15.152:0/2520071661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcc4100e20 msgr2=0x7fdcc4198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:16.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.026+0000 7fdcc3fff640 1 --2- 172.21.15.152:0/2520071661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcc4100e20 0x7fdcc4198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.026+0000 7fdcc3fff640 1 -- 172.21.15.152:0/2520071661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcc4102570 msgr2=0x7fdcc4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:16.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.026+0000 7fdcc3fff640 1 --2- 172.21.15.152:0/2520071661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcc4102570 0x7fdcc4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.026+0000 7fdcc3fff640 1 -- 172.21.15.152:0/2520071661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc419d4c0 con 0x7fdcc4102ef0 2024-09-11T22:53:16.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.026+0000 7fdcc3fff640 1 --2- 172.21.15.152:0/2520071661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcc4102ef0 0x7fdcc419cde0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009fd0 tx=0x7fdcbc00def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:16.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.026+0000 7fdcc0ff9640 1 -- 172.21.15.152:0/2520071661 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc017070 con 0x7fdcc4102ef0 2024-09-11T22:53:16.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.026+0000 7fdcca0a8640 1 -- 172.21.15.152:0/2520071661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc419d7b0 con 0x7fdcc4102ef0 2024-09-11T22:53:16.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.026+0000 7fdcc0ff9640 1 -- 172.21.15.152:0/2520071661 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcbc003e50 con 0x7fdcc4102ef0 2024-09-11T22:53:16.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.026+0000 7fdcc0ff9640 1 -- 172.21.15.152:0/2520071661 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc010460 con 0x7fdcc4102ef0 2024-09-11T22:53:16.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.026+0000 7fdcca0a8640 1 -- 172.21.15.152:0/2520071661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc4103680 con 0x7fdcc4102ef0 2024-09-11T22:53:16.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.028+0000 7fdcca0a8640 1 -- 172.21.15.152:0/2520071661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc41021b0 con 0x7fdcc4102ef0 2024-09-11T22:53:16.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.029+0000 7fdcc0ff9640 1 -- 172.21.15.152:0/2520071661 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7fdcbc003970 con 0x7fdcc4102ef0 2024-09-11T22:53:16.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.030+0000 7fdcc0ff9640 1 --2- 172.21.15.152:0/2520071661 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fdcac0a2d80 0x7fdcac0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.030+0000 7fdcc37fe640 1 --2- 172.21.15.152:0/2520071661 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fdcac0a2d80 0x7fdcac0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.030+0000 7fdcc0ff9640 1 -- 172.21.15.152:0/2520071661 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fdcbc0cf0f0 con 0x7fdcc4102ef0 2024-09-11T22:53:16.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.030+0000 7fdcc37fe640 1 --2- 172.21.15.152:0/2520071661 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fdcac0a2d80 0x7fdcac0a5240 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009830 tx=0x7fdcb40094d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:16.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.033+0000 7fdcc0ff9640 1 -- 172.21.15.152:0/2520071661 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdcbc09bbf0 con 0x7fdcc4102ef0 2024-09-11T22:53:16.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.241+0000 7fdcca0a8640 1 -- 172.21.15.152:0/2520071661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcc41038e0 con 0x7fdcc4102ef0 2024-09-11T22:53:16.242 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:16 smithi152 ceph-mon[31181]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-11T22:53:16.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.242+0000 7fdcc0ff9640 1 -- 172.21.15.152:0/2520071661 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7fdcbc09b4f0 con 0x7fdcc4102ef0 2024-09-11T22:53:16.243 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:53:16.243 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:53:16.243 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:53:16.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.244+0000 7fdcca0a8640 1 -- 172.21.15.152:0/2520071661 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fdcac0a2d80 msgr2=0x7fdcac0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:16.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.244+0000 7fdcca0a8640 1 --2- 172.21.15.152:0/2520071661 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fdcac0a2d80 0x7fdcac0a5240 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009830 tx=0x7fdcb40094d0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.244+0000 7fdcca0a8640 1 -- 172.21.15.152:0/2520071661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcc4102ef0 msgr2=0x7fdcc419cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:16.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.244+0000 7fdcca0a8640 1 --2- 172.21.15.152:0/2520071661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcc4102ef0 0x7fdcc419cde0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009fd0 tx=0x7fdcbc00def0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.244+0000 7fdcca0a8640 1 -- 172.21.15.152:0/2520071661 shutdown_connections 2024-09-11T22:53:16.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.245+0000 7fdcca0a8640 1 --2- 172.21.15.152:0/2520071661 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fdcac0a2d80 0x7fdcac0a5240 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.245+0000 7fdcca0a8640 1 --2- 172.21.15.152:0/2520071661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdcc4102ef0 0x7fdcc419cde0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.245+0000 7fdcca0a8640 1 --2- 172.21.15.152:0/2520071661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcc4102570 0x7fdcc4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.245+0000 7fdcca0a8640 1 --2- 172.21.15.152:0/2520071661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcc4100e20 0x7fdcc4198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.245+0000 7fdcca0a8640 1 -- 172.21.15.152:0/2520071661 >> 172.21.15.152:0/2520071661 conn(0x7fdcc40fc7d0 msgr2=0x7fdcc406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:16.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.245+0000 7fdcca0a8640 1 -- 172.21.15.152:0/2520071661 shutdown_connections 2024-09-11T22:53:16.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:16.245+0000 7fdcca0a8640 1 -- 172.21.15.152:0/2520071661 wait complete. 2024-09-11T22:53:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.327+0000 7ff097ef8640 1 -- 172.21.15.152:0/3637505487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff090102050 msgr2=0x7ff09010d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.327+0000 7ff097ef8640 1 --2- 172.21.15.152:0/3637505487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff090102050 0x7ff09010d7d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff088009ca0 tx=0x7ff08802a950 comp rx=0 tx=0).stop 2024-09-11T22:53:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.328+0000 7ff097ef8640 1 -- 172.21.15.152:0/3637505487 shutdown_connections 2024-09-11T22:53:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.328+0000 7ff097ef8640 1 --2- 172.21.15.152:0/3637505487 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff090102050 0x7ff09010d7d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.328+0000 7ff097ef8640 1 --2- 172.21.15.152:0/3637505487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff0901016d0 0x7ff090101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.328+0000 7ff097ef8640 1 --2- 172.21.15.152:0/3637505487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff090105af0 0x7ff090105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.328+0000 7ff097ef8640 1 -- 172.21.15.152:0/3637505487 >> 172.21.15.152:0/3637505487 conn(0x7ff090075f20 msgr2=0x7ff090076330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.328+0000 7ff097ef8640 1 -- 172.21.15.152:0/3637505487 shutdown_connections 2024-09-11T22:53:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.328+0000 7ff097ef8640 1 -- 172.21.15.152:0/3637505487 wait complete. 2024-09-11T22:53:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.328+0000 7ff097ef8640 1 Processor -- start 2024-09-11T22:53:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.328+0000 7ff097ef8640 1 -- start start 2024-09-11T22:53:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.328+0000 7ff097ef8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0901016d0 0x7ff0901991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.328+0000 7ff097ef8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff090102050 0x7ff0901996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.328+0000 7ff097ef8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff090105af0 0x7ff0901a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.328+0000 7ff097ef8640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff090110e00 con 0x7ff090105af0 2024-09-11T22:53:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.328+0000 7ff097ef8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff090110c80 con 0x7ff090102050 2024-09-11T22:53:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.328+0000 7ff097ef8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff090110f80 con 0x7ff0901016d0 2024-09-11T22:53:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff096495640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff090105af0 0x7ff0901a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff096495640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff090105af0 0x7ff0901a4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47556/0 (socket says 172.21.15.152:47556) 2024-09-11T22:53:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff096495640 1 -- 172.21.15.152:0/2007431264 learned_addr learned my addr 172.21.15.152:0/2007431264 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff095493640 1 --2- 172.21.15.152:0/2007431264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff090102050 0x7ff0901996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff096495640 1 -- 172.21.15.152:0/2007431264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0901016d0 msgr2=0x7ff0901991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff095c94640 1 --2- 172.21.15.152:0/2007431264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0901016d0 0x7ff0901991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff096495640 1 --2- 172.21.15.152:0/2007431264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0901016d0 0x7ff0901991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff096495640 1 -- 172.21.15.152:0/2007431264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff090102050 msgr2=0x7ff0901996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff096495640 1 --2- 172.21.15.152:0/2007431264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff090102050 0x7ff0901996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff096495640 1 -- 172.21.15.152:0/2007431264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0901a4770 con 0x7ff090105af0 2024-09-11T22:53:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff095493640 1 --2- 172.21.15.152:0/2007431264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff090102050 0x7ff0901996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff096495640 1 --2- 172.21.15.152:0/2007431264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff090105af0 0x7ff0901a4230 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff088006010 tx=0x7ff08802f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff082ffd640 1 -- 172.21.15.152:0/2007431264 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff088034040 con 0x7ff090105af0 2024-09-11T22:53:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff082ffd640 1 -- 172.21.15.152:0/2007431264 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff088003e00 con 0x7ff090105af0 2024-09-11T22:53:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff082ffd640 1 -- 172.21.15.152:0/2007431264 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0880456e0 con 0x7ff090105af0 2024-09-11T22:53:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff097ef8640 1 -- 172.21.15.152:0/2007431264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0901a4940 con 0x7ff090105af0 2024-09-11T22:53:16.333 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.329+0000 7ff097ef8640 1 -- 172.21.15.152:0/2007431264 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0901a4da0 con 0x7ff090105af0 2024-09-11T22:53:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.332+0000 7ff082ffd640 1 -- 172.21.15.152:0/2007431264 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7ff088037020 con 0x7ff090105af0 2024-09-11T22:53:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.332+0000 7ff09ac97740 1 Processor -- start 2024-09-11T22:53:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.332+0000 7ff09ac97740 1 -- start start 2024-09-11T22:53:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff09ac97740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac76af0 0x560d2ac76ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff09ac97740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac77410 0x560d2ac81770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.339 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff09ac97740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac81cb0 0x560d2ac840a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff09ac97740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560d2abdb2f0 con 0x560d2ac77410 2024-09-11T22:53:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff09ac97740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560d2ab83270 con 0x560d2ac81cb0 2024-09-11T22:53:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff09ac97740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560d2abd72b0 con 0x560d2ac76af0 2024-09-11T22:53:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff095c94640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac77410 0x560d2ac81770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff095c94640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac77410 0x560d2ac81770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47564/0 (socket says 172.21.15.152:47564) 2024-09-11T22:53:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff095c94640 1 -- 172.21.15.152:0/2087528658 learned_addr learned my addr 172.21.15.152:0/2087528658 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff095493640 1 --2- 172.21.15.152:0/2087528658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac76af0 0x560d2ac76ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff082ffd640 1 --2- 172.21.15.152:0/2007431264 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff07c0a2e40 0x7ff07c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff095493640 1 --2- 172.21.15.152:0/2087528658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac81cb0 0x560d2ac840a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff095c94640 1 -- 172.21.15.152:0/2087528658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac76af0 msgr2=0x560d2ac76ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff095c94640 1 --2- 172.21.15.152:0/2087528658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac76af0 0x560d2ac76ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff095c94640 1 -- 172.21.15.152:0/2087528658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac81cb0 msgr2=0x560d2ac840a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff095c94640 1 --2- 172.21.15.152:0/2087528658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac81cb0 0x560d2ac840a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff095c94640 1 -- 172.21.15.152:0/2087528658 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d2ab82340 con 0x560d2ac77410 2024-09-11T22:53:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff096495640 1 --2- 172.21.15.152:0/2007431264 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff07c0a2e40 0x7ff07c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff082ffd640 1 -- 172.21.15.152:0/2007431264 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7ff0880f06d0 con 0x7ff090105af0 2024-09-11T22:53:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff095c94640 1 --2- 172.21.15.152:0/2087528658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac77410 0x560d2ac81770 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00baa0 tx=0x7ff08c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5293237929b317d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.333+0000 7ff07bfff640 1 -- 172.21.15.152:0/2087528658 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c017420 con 0x560d2ac77410 2024-09-11T22:53:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.334+0000 7ff096495640 1 --2- 172.21.15.152:0/2007431264 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff07c0a2e40 0x7ff07c0a5300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff0880edf20 tx=0x7ff08808fc50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.334+0000 7ff07bfff640 1 -- 172.21.15.152:0/2087528658 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c017aa0 con 0x560d2ac77410 2024-09-11T22:53:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.334+0000 7ff07bfff640 1 -- 172.21.15.152:0/2087528658 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c00fb70 con 0x560d2ac77410 2024-09-11T22:53:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.334+0000 7ff09ac97740 1 -- 172.21.15.152:0/2087528658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac77410 msgr2=0x560d2ac81770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.334+0000 7ff09ac97740 1 --2- 172.21.15.152:0/2087528658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac77410 0x560d2ac81770 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00baa0 tx=0x7ff08c00bf60 comp rx=0 tx=0).stop 2024-09-11T22:53:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.334+0000 7ff09ac97740 1 -- 172.21.15.152:0/2087528658 shutdown_connections 2024-09-11T22:53:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.334+0000 7ff09ac97740 1 --2- 172.21.15.152:0/2087528658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac81cb0 0x560d2ac840a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.334+0000 7ff09ac97740 1 --2- 172.21.15.152:0/2087528658 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac77410 0x560d2ac81770 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.334+0000 7ff09ac97740 1 --2- 172.21.15.152:0/2087528658 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac76af0 0x560d2ac76ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.334+0000 7ff09ac97740 1 -- 172.21.15.152:0/2087528658 >> 172.21.15.152:0/2087528658 conn(0x560d2ac73060 msgr2=0x560d2ac73440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.334+0000 7ff09ac97740 1 -- 172.21.15.152:0/2087528658 shutdown_connections 2024-09-11T22:53:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.334+0000 7ff09ac97740 1 -- 172.21.15.152:0/2087528658 wait complete. 2024-09-11T22:53:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.335+0000 7ff09ac97740 1 -- start start 2024-09-11T22:53:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.335+0000 7ff09ac97740 1 Processor -- start 2024-09-11T22:53:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.335+0000 7ff09ac97740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac76af0 0x560d2ac7ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.335+0000 7ff09ac97740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac77410 0x560d2ac7f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.335+0000 7ff09ac97740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac81cb0 0x560d2ac7f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.335+0000 7ff09ac97740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560d2ac84880 con 0x560d2ac76af0 2024-09-11T22:53:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.335+0000 7ff09ac97740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560d2ab64290 con 0x560d2ac81cb0 2024-09-11T22:53:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.335+0000 7ff09ac97740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560d2ac84a00 con 0x560d2ac77410 2024-09-11T22:53:16.342 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.335+0000 7ff09ac97740 10 client.0 _ll_register_callbacks cb 0x560d2ab95140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.335+0000 7ff095493640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac76af0 0x560d2ac7ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.335+0000 7ff095493640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac76af0 0x560d2ac7ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47576/0 (socket says 172.21.15.152:47576) 2024-09-11T22:53:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.335+0000 7ff095493640 1 -- 172.21.15.152:0/1472794536 learned_addr learned my addr 172.21.15.152:0/1472794536 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.335+0000 7ff095493640 1 -- 172.21.15.152:0/1472794536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac77410 msgr2=0x560d2ac7f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.335+0000 7ff095c94640 1 --2- 172.21.15.152:0/1472794536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac77410 0x560d2ac7f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff095493640 1 --2- 172.21.15.152:0/1472794536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac77410 0x560d2ac7f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff095493640 1 -- 172.21.15.152:0/1472794536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac81cb0 msgr2=0x560d2ac7f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff095493640 1 --2- 172.21.15.152:0/1472794536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac81cb0 0x560d2ac7f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff095493640 1 -- 172.21.15.152:0/1472794536 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d2a8ee210 con 0x560d2ac76af0 2024-09-11T22:53:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff095c94640 1 --2- 172.21.15.152:0/1472794536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac77410 0x560d2ac7f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff095493640 1 --2- 172.21.15.152:0/1472794536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac76af0 0x560d2ac7ebe0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff0840136a0 tx=0x7ff084013b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b211c68e91c11489 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff0797fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff0797fa640 1 -- 172.21.15.152:0/1472794536 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08401a070 con 0x560d2ac76af0 2024-09-11T22:53:16.343 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff0797fa640 1 -- 172.21.15.152:0/1472794536 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7ff084011960 con 0x560d2ac76af0 2024-09-11T22:53:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff09ac97740 1 -- 172.21.15.152:0/1472794536 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff08c017aa0 con 0x560d2ac76af0 2024-09-11T22:53:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff0797fa640 1 -- 172.21.15.152:0/1472794536 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff084025050 con 0x560d2ac76af0 2024-09-11T22:53:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff09ac97740 10 client.15486 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff09ac97740 1 -- 172.21.15.152:0/1472794536 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d2a9ba620 con 0x560d2ac76af0 2024-09-11T22:53:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff0797fa640 1 -- 172.21.15.152:0/1472794536 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c017aa0 con 0x560d2ac76af0 2024-09-11T22:53:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff0797fa640 1 -- 172.21.15.152:0/1472794536 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff084015070 con 0x560d2ac76af0 2024-09-11T22:53:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff09ac97740 10 client.15486 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff09ac97740 10 client.15486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff09ac97740 20 client.15486 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff09ac97740 1 --2- 172.21.15.152:0/1472794536 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560d2ac91c50 0x560d2ac940a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff09ac97740 4 client.15486 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.336+0000 7ff09ac97740 1 -- 172.21.15.152:0/1472794536 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d2abe8a20 con 0x560d2ac91c50 2024-09-11T22:53:16.344 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.337+0000 7ff095c94640 1 --2- 172.21.15.152:0/1472794536 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560d2ac91c50 0x560d2ac940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.337+0000 7ff0797fa640 10 client.15486 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.337+0000 7ff095c94640 1 --2- 172.21.15.152:0/1472794536 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560d2ac91c50 0x560d2ac940a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7977ef897c7ad6f server_cookie=1a2afa4e8e82e2af in_seq=0 out_seq=0 2024-09-11T22:53:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.338+0000 7ff0797fa640 1 -- 172.21.15.152:0/1472794536 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d2abe8a20 con 0x560d2ac91c50 2024-09-11T22:53:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.338+0000 7ff0797fa640 10 client.15486 handle_command_reply: tid=0 2024-09-11T22:53:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.338+0000 7ff09ac97740 1 client.15486 shutdown 2024-09-11T22:53:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.338+0000 7ff09ac97740 1 -- 172.21.15.152:0/1472794536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac76af0 msgr2=0x560d2ac7ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.338+0000 7ff09ac97740 1 --2- 172.21.15.152:0/1472794536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac76af0 0x560d2ac7ebe0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff0840136a0 tx=0x7ff084013b60 comp rx=0 tx=0).stop 2024-09-11T22:53:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.338+0000 7ff09ac97740 1 -- 172.21.15.152:0/1472794536 shutdown_connections 2024-09-11T22:53:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.338+0000 7ff09ac97740 1 --2- 172.21.15.152:0/1472794536 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560d2ac91c50 0x560d2ac940a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.338+0000 7ff09ac97740 1 --2- 172.21.15.152:0/1472794536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac81cb0 0x560d2ac7f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.338+0000 7ff09ac97740 1 --2- 172.21.15.152:0/1472794536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac77410 0x560d2ac7f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.338+0000 7ff09ac97740 1 --2- 172.21.15.152:0/1472794536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac76af0 0x560d2ac7ebe0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.338+0000 7ff0797fa640 0 client.15486 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.338+0000 7ff09ac97740 1 -- 172.21.15.152:0/1472794536 >> 172.21.15.152:0/1472794536 conn(0x560d2ac73060 msgr2=0x560d2ac78d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.338+0000 7ff09ac97740 1 -- 172.21.15.152:0/1472794536 shutdown_connections 2024-09-11T22:53:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.338+0000 7ff09ac97740 1 -- 172.21.15.152:0/1472794536 wait complete. 2024-09-11T22:53:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.338+0000 7ff09ac97740 20 client.15486 trim_cache size 0 max 16384 2024-09-11T22:53:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.352+0000 7ff09ac97740 1 Processor -- start 2024-09-11T22:53:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.352+0000 7ff09ac97740 1 -- start start 2024-09-11T22:53:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.352+0000 7ff09ac97740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac76af0 0x560d2ac79560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.352+0000 7ff09ac97740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac77410 0x560d2ac7f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.352+0000 7ff09ac97740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac81cb0 0x560d2ac8a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.352+0000 7ff09ac97740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901ad670 con 0x560d2ac77410 2024-09-11T22:53:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.352+0000 7ff09ac97740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901af1b0 con 0x560d2ac81cb0 2024-09-11T22:53:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.352+0000 7ff09ac97740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901add40 con 0x560d2ac76af0 2024-09-11T22:53:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.353+0000 7ff095c94640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac77410 0x560d2ac7f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.353+0000 7ff095c94640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac77410 0x560d2ac7f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47586/0 (socket says 172.21.15.152:47586) 2024-09-11T22:53:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.353+0000 7ff095c94640 1 -- 172.21.15.152:0/168567320 learned_addr learned my addr 172.21.15.152:0/168567320 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.353+0000 7ff095c94640 1 -- 172.21.15.152:0/168567320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac76af0 msgr2=0x560d2ac79560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.353+0000 7ff095c94640 1 --2- 172.21.15.152:0/168567320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac76af0 0x560d2ac79560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.353+0000 7ff095c94640 1 -- 172.21.15.152:0/168567320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac81cb0 msgr2=0x560d2ac8a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.353+0000 7ff095c94640 1 --2- 172.21.15.152:0/168567320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac81cb0 0x560d2ac8a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.353+0000 7ff095c94640 1 -- 172.21.15.152:0/168567320 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d2ac76920 con 0x560d2ac77410 2024-09-11T22:53:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.353+0000 7ff095c94640 1 --2- 172.21.15.152:0/168567320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac77410 0x560d2ac7f490 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff08c000c00 tx=0x7ff08c005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a9e87fb31826f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.353+0000 7ff07b7fe640 1 -- 172.21.15.152:0/168567320 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c00dce0 con 0x560d2ac77410 2024-09-11T22:53:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.353+0000 7ff07b7fe640 1 -- 172.21.15.152:0/168567320 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c00de80 con 0x560d2ac77410 2024-09-11T22:53:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.353+0000 7ff07b7fe640 1 -- 172.21.15.152:0/168567320 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c00fdd0 con 0x560d2ac77410 2024-09-11T22:53:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.354+0000 7ff09ac97740 1 -- 172.21.15.152:0/168567320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac77410 msgr2=0x560d2ac7f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.354+0000 7ff09ac97740 1 --2- 172.21.15.152:0/168567320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac77410 0x560d2ac7f490 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff08c000c00 tx=0x7ff08c005e90 comp rx=0 tx=0).stop 2024-09-11T22:53:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.354+0000 7ff09ac97740 1 -- 172.21.15.152:0/168567320 shutdown_connections 2024-09-11T22:53:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.354+0000 7ff09ac97740 1 --2- 172.21.15.152:0/168567320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac81cb0 0x560d2ac8a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.354+0000 7ff09ac97740 1 --2- 172.21.15.152:0/168567320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac77410 0x560d2ac7f490 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.354+0000 7ff09ac97740 1 --2- 172.21.15.152:0/168567320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac76af0 0x560d2ac79560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.354+0000 7ff09ac97740 1 -- 172.21.15.152:0/168567320 >> 172.21.15.152:0/168567320 conn(0x560d2ac73060 msgr2=0x560d2ac80c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.354+0000 7ff09ac97740 1 -- 172.21.15.152:0/168567320 shutdown_connections 2024-09-11T22:53:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.354+0000 7ff09ac97740 1 -- 172.21.15.152:0/168567320 wait complete. 2024-09-11T22:53:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.354+0000 7ff09ac97740 1 -- start start 2024-09-11T22:53:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.354+0000 7ff09ac97740 1 Processor -- start 2024-09-11T22:53:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff09ac97740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac76af0 0x560d2ac7b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff09ac97740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac77410 0x560d2ac7b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff09ac97740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac81cb0 0x560d2ac7bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff09ac97740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901aeae0 con 0x560d2ac81cb0 2024-09-11T22:53:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff09ac97740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901ae410 con 0x560d2ac76af0 2024-09-11T22:53:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff09ac97740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff0901ab630 con 0x560d2ac77410 2024-09-11T22:53:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff09ac97740 10 client.0 _ll_register_callbacks cb 0x7ff084006cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff095493640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac81cb0 0x560d2ac7bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff095493640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac81cb0 0x560d2ac7bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47596/0 (socket says 172.21.15.152:47596) 2024-09-11T22:53:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff095493640 1 -- 172.21.15.152:0/2973294089 learned_addr learned my addr 172.21.15.152:0/2973294089 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff095c94640 1 --2- 172.21.15.152:0/2973294089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac77410 0x560d2ac7b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff095493640 1 --2- 172.21.15.152:0/2973294089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac76af0 0x560d2ac7b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff095493640 1 -- 172.21.15.152:0/2973294089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac77410 msgr2=0x560d2ac7b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff095493640 1 --2- 172.21.15.152:0/2973294089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac77410 0x560d2ac7b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff095493640 1 -- 172.21.15.152:0/2973294089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac76af0 msgr2=0x560d2ac7b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff095493640 1 --2- 172.21.15.152:0/2973294089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac76af0 0x560d2ac7b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff095493640 1 -- 172.21.15.152:0/2973294089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d2ac7c430 con 0x560d2ac81cb0 2024-09-11T22:53:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff095493640 1 --2- 172.21.15.152:0/2973294089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac81cb0 0x560d2ac7bc30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff08400c1e0 tx=0x7ff084004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d568500f91227e5f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff0797fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff0797fa640 1 -- 172.21.15.152:0/2973294089 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08402e060 con 0x560d2ac81cb0 2024-09-11T22:53:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff0797fa640 1 -- 172.21.15.152:0/2973294089 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7ff084015a00 con 0x560d2ac81cb0 2024-09-11T22:53:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff09ac97740 1 -- 172.21.15.152:0/2973294089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560d2abe2d70 con 0x560d2ac81cb0 2024-09-11T22:53:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff0797fa640 1 -- 172.21.15.152:0/2973294089 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff08401a040 con 0x560d2ac81cb0 2024-09-11T22:53:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff09ac97740 10 client.15492 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.355+0000 7ff09ac97740 1 -- 172.21.15.152:0/2973294089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560d2ad01330 con 0x560d2ac81cb0 2024-09-11T22:53:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.356+0000 7ff0797fa640 1 -- 172.21.15.152:0/2973294089 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560d2abe2d70 con 0x560d2ac81cb0 2024-09-11T22:53:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.356+0000 7ff0797fa640 1 -- 172.21.15.152:0/2973294089 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff084015040 con 0x560d2ac81cb0 2024-09-11T22:53:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.356+0000 7ff09ac97740 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.356+0000 7ff09ac97740 10 client.15492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.356+0000 7ff09ac97740 20 client.15492 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.356+0000 7ff09ac97740 1 --2- 172.21.15.152:0/2973294089 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560d2ad01ab0 0x560d2ad03ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.356+0000 7ff09ac97740 4 client.15492 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:53:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.356+0000 7ff09ac97740 1 -- 172.21.15.152:0/2973294089 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d2ab5f900 con 0x560d2ad01ab0 2024-09-11T22:53:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.356+0000 7ff095c94640 1 --2- 172.21.15.152:0/2973294089 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560d2ad01ab0 0x560d2ad03ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.357+0000 7ff0797fa640 10 client.15492 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.357+0000 7ff095c94640 1 --2- 172.21.15.152:0/2973294089 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560d2ad01ab0 0x560d2ad03ed0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64410cd1445e567e server_cookie=70da097731e05d66 in_seq=0 out_seq=0 2024-09-11T22:53:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.367+0000 7ff0797fa640 1 -- 172.21.15.152:0/2973294089 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d2ab5f900 con 0x560d2ad01ab0 2024-09-11T22:53:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.367+0000 7ff0797fa640 10 client.15492 handle_command_reply: tid=0 2024-09-11T22:53:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.367+0000 7ff09ac97740 1 client.15492 shutdown 2024-09-11T22:53:16.368 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.367+0000 7ff09ac97740 1 -- 172.21.15.152:0/2973294089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac81cb0 msgr2=0x560d2ac7bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff09ac97740 1 --2- 172.21.15.152:0/2973294089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac81cb0 0x560d2ac7bc30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff08400c1e0 tx=0x7ff084004660 comp rx=0 tx=0).stop 2024-09-11T22:53:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff09ac97740 1 -- 172.21.15.152:0/2973294089 shutdown_connections 2024-09-11T22:53:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff09ac97740 1 --2- 172.21.15.152:0/2973294089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560d2ac81cb0 0x560d2ac7bc30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff09ac97740 1 --2- 172.21.15.152:0/2973294089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560d2ac77410 0x560d2ac7b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff09ac97740 1 --2- 172.21.15.152:0/2973294089 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560d2ad01ab0 0x560d2ad03ed0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff09ac97740 1 --2- 172.21.15.152:0/2973294089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560d2ac76af0 0x560d2ac7b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff0797fa640 0 client.15492 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff09ac97740 1 -- 172.21.15.152:0/2973294089 >> 172.21.15.152:0/2973294089 conn(0x560d2ac73060 msgr2=0x560d2ac719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff09ac97740 1 -- 172.21.15.152:0/2973294089 shutdown_connections 2024-09-11T22:53:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff09ac97740 1 -- 172.21.15.152:0/2973294089 wait complete. 2024-09-11T22:53:16.369 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff09ac97740 20 client.15492 trim_cache size 0 max 16384 2024-09-11T22:53:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:{ 2024-09-11T22:53:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "message": "", 2024-09-11T22:53:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "return_code": 0 2024-09-11T22:53:16.370 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:} 2024-09-11T22:53:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff097ef8640 1 -- 172.21.15.152:0/2007431264 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff07c0a2e40 msgr2=0x7ff07c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff097ef8640 1 --2- 172.21.15.152:0/2007431264 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff07c0a2e40 0x7ff07c0a5300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff0880edf20 tx=0x7ff08808fc50 comp rx=0 tx=0).stop 2024-09-11T22:53:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff097ef8640 1 -- 172.21.15.152:0/2007431264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff090105af0 msgr2=0x7ff0901a4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff097ef8640 1 --2- 172.21.15.152:0/2007431264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff090105af0 0x7ff0901a4230 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff088006010 tx=0x7ff08802f710 comp rx=0 tx=0).stop 2024-09-11T22:53:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff097ef8640 1 -- 172.21.15.152:0/2007431264 shutdown_connections 2024-09-11T22:53:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff097ef8640 1 --2- 172.21.15.152:0/2007431264 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff07c0a2e40 0x7ff07c0a5300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff097ef8640 1 --2- 172.21.15.152:0/2007431264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff090105af0 0x7ff0901a4230 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff097ef8640 1 --2- 172.21.15.152:0/2007431264 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff090102050 0x7ff0901996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff097ef8640 1 --2- 172.21.15.152:0/2007431264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0901016d0 0x7ff0901991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.368+0000 7ff097ef8640 1 -- 172.21.15.152:0/2007431264 >> 172.21.15.152:0/2007431264 conn(0x7ff090075f20 msgr2=0x7ff0900ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.369+0000 7ff097ef8640 1 -- 172.21.15.152:0/2007431264 shutdown_connections 2024-09-11T22:53:16.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:16.369+0000 7ff097ef8640 1 -- 172.21.15.152:0/2007431264 wait complete. 2024-09-11T22:53:16.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:16 smithi191 ceph-mon[35211]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-11T22:53:16.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:16 smithi204 ceph-mon[35454]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-11T22:53:16.932 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T22:50:55.047332+0000', 'modified': '2024-09-11T22:52:52.044945+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24455, 'mds_1': 24442, 'mds_2': 24434, 'mds_3': 24430, 'mds_4': 14628}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6839/97179606', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6838', 'nonce': 97179606}, {'type': 'v1', 'addr': '172.21.15.152:6839', 'nonce': 97179606}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.152:6841/2492281944', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6840', 'nonce': 2492281944}, {'type': 'v1', 'addr': '172.21.15.152:6841', 'nonce': 2492281944}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24377': {'gid': 24377, 'name': 'k', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6833/2321157769', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6832', 'nonce': 2321157769}, {'type': 'v1', 'addr': '172.21.15.191:6833', 'nonce': 2321157769}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24397': {'gid': 24397, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6833/2200342090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6832', 'nonce': 2200342090}, {'type': 'v1', 'addr': '172.21.15.204:6833', 'nonce': 2200342090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.191:6835/2471607007', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6834', 'nonce': 2471607007}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 2471607007}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/763950562', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 763950562}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 763950562}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.204:6837/1863048373', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 1863048373}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 1863048373}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.191:6837/314110375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6836', 'nonce': 314110375}, {'type': 'v1', 'addr': '172.21.15.191:6837', 'nonce': 314110375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.204:6839/3360312546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 3360312546}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 3360312546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.191:6839/2588730805', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6838', 'nonce': 2588730805}, {'type': 'v1', 'addr': '172.21.15.191:6839', 'nonce': 2588730805}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24455, 'qdb_cluster': [14628, 24430, 24434, 24442, 24455]} 2024-09-11T22:53:16.932 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T22:53:16.932 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.a status 2024-09-11T22:53:17.158 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:17 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2520071661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:53:17.223 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:17.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:17 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2520071661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:53:17.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:17 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2520071661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:53:17.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.719+0000 7f6082b63640 1 -- 172.21.15.152:0/1233487820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f607c102790 msgr2=0x7f607c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:17.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.719+0000 7f6082b63640 1 --2- 172.21.15.152:0/1233487820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f607c102790 0x7f607c1067e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6068009900 tx=0x7f606802a410 comp rx=0 tx=0).stop 2024-09-11T22:53:17.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.719+0000 7f6082b63640 1 -- 172.21.15.152:0/1233487820 shutdown_connections 2024-09-11T22:53:17.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.719+0000 7f6082b63640 1 --2- 172.21.15.152:0/1233487820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f607c106f10 0x7f607c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.719+0000 7f6082b63640 1 --2- 172.21.15.152:0/1233487820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f607c102790 0x7f607c1067e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.719+0000 7f6082b63640 1 --2- 172.21.15.152:0/1233487820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f607c101de0 0x7f607c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.719+0000 7f6082b63640 1 -- 172.21.15.152:0/1233487820 >> 172.21.15.152:0/1233487820 conn(0x7f607c0fd8c0 msgr2=0x7f607c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:17.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.719+0000 7f6082b63640 1 -- 172.21.15.152:0/1233487820 shutdown_connections 2024-09-11T22:53:17.723 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.719+0000 7f6082b63640 1 -- 172.21.15.152:0/1233487820 wait complete. 2024-09-11T22:53:17.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.720+0000 7f6082b63640 1 Processor -- start 2024-09-11T22:53:17.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.720+0000 7f6082b63640 1 -- start start 2024-09-11T22:53:17.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.720+0000 7f6082b63640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f607c101de0 0x7f607c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.720+0000 7f6082b63640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f607c102790 0x7f607c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.720+0000 7f6082b63640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f607c106f10 0x7f607c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.720+0000 7f6082b63640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f607c111f70 con 0x7f607c102790 2024-09-11T22:53:17.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.720+0000 7f6082b63640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f607c111df0 con 0x7f607c101de0 2024-09-11T22:53:17.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.720+0000 7f6082b63640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f607c1120f0 con 0x7f607c106f10 2024-09-11T22:53:17.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.720+0000 7f607bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f607c102790 0x7f607c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.720+0000 7f607bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f607c102790 0x7f607c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47604/0 (socket says 172.21.15.152:47604) 2024-09-11T22:53:17.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.720+0000 7f607bfff640 1 -- 172.21.15.152:0/2884265116 learned_addr learned my addr 172.21.15.152:0/2884265116 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:17.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.721+0000 7f60808ff640 1 --2- 172.21.15.152:0/2884265116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f607c101de0 0x7f607c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.721+0000 7f607bfff640 1 -- 172.21.15.152:0/2884265116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f607c106f10 msgr2=0x7f607c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:17.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.721+0000 7f6081100640 1 --2- 172.21.15.152:0/2884265116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f607c106f10 0x7f607c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.721+0000 7f607bfff640 1 --2- 172.21.15.152:0/2884265116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f607c106f10 0x7f607c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.721+0000 7f607bfff640 1 -- 172.21.15.152:0/2884265116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f607c101de0 msgr2=0x7f607c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:17.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.721+0000 7f607bfff640 1 --2- 172.21.15.152:0/2884265116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f607c101de0 0x7f607c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.721+0000 7f607bfff640 1 -- 172.21.15.152:0/2884265116 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607c19d380 con 0x7f607c102790 2024-09-11T22:53:17.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.721+0000 7f60808ff640 1 --2- 172.21.15.152:0/2884265116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f607c101de0 0x7f607c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:17.725 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.721+0000 7f6081100640 1 --2- 172.21.15.152:0/2884265116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f607c106f10 0x7f607c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:17.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.721+0000 7f607bfff640 1 --2- 172.21.15.152:0/2884265116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f607c102790 0x7f607c198910 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6068005ab0 tx=0x7f60680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:17.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.721+0000 7f6079ffb640 1 -- 172.21.15.152:0/2884265116 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606803f030 con 0x7f607c102790 2024-09-11T22:53:17.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.721+0000 7f6082b63640 1 -- 172.21.15.152:0/2884265116 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f607c19d610 con 0x7f607c102790 2024-09-11T22:53:17.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.721+0000 7f6079ffb640 1 -- 172.21.15.152:0/2884265116 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6068003ce0 con 0x7f607c102790 2024-09-11T22:53:17.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.721+0000 7f6079ffb640 1 -- 172.21.15.152:0/2884265116 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6068035070 con 0x7f607c102790 2024-09-11T22:53:17.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.721+0000 7f6082b63640 1 -- 172.21.15.152:0/2884265116 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f607c1a4f50 con 0x7f607c102790 2024-09-11T22:53:17.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.724+0000 7f6079ffb640 1 -- 172.21.15.152:0/2884265116 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f6068042030 con 0x7f607c102790 2024-09-11T22:53:17.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.724+0000 7f6084f3c740 1 Processor -- start 2024-09-11T22:53:17.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.724+0000 7f6084f3c740 1 -- start start 2024-09-11T22:53:17.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f6084f3c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bd8fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f6084f3c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd902f0 0x55eb9bd9a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f6084f3c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd9abb0 0x55eb9bd9cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f6084f3c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55eb9bcf6090 con 0x55eb9bd902f0 2024-09-11T22:53:17.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f6084f3c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55eb9bc9da70 con 0x55eb9bd9abb0 2024-09-11T22:53:17.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f6084f3c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55eb9bcf2050 con 0x55eb9bd8f9d0 2024-09-11T22:53:17.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f60808ff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd902f0 0x55eb9bd9a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.732 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f60808ff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd902f0 0x55eb9bd9a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47612/0 (socket says 172.21.15.152:47612) 2024-09-11T22:53:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f60808ff640 1 -- 172.21.15.152:0/14383771 learned_addr learned my addr 172.21.15.152:0/14383771 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f607bfff640 1 --2- 172.21.15.152:0/14383771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd9abb0 0x55eb9bd9cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f6081100640 1 --2- 172.21.15.152:0/14383771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bd8fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f60808ff640 1 -- 172.21.15.152:0/14383771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd8f9d0 msgr2=0x55eb9bd8fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f60808ff640 1 --2- 172.21.15.152:0/14383771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bd8fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f60808ff640 1 -- 172.21.15.152:0/14383771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd9abb0 msgr2=0x55eb9bd9cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f60808ff640 1 --2- 172.21.15.152:0/14383771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd9abb0 0x55eb9bd9cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f60808ff640 1 -- 172.21.15.152:0/14383771 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb9bcaa130 con 0x55eb9bd902f0 2024-09-11T22:53:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f6079ffb640 1 --2- 172.21.15.152:0/2884265116 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f606c0a2d80 0x7f606c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f6081100640 1 --2- 172.21.15.152:0/2884265116 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f606c0a2d80 0x7f606c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f60808ff640 1 --2- 172.21.15.152:0/14383771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd902f0 0x55eb9bd9a670 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f607000bb70 tx=0x7f607000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf085eb720f9e81 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.725+0000 7f6079ffb640 1 -- 172.21.15.152:0/2884265116 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f60680eff40 con 0x7f607c102790 2024-09-11T22:53:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.726+0000 7f6066ffd640 1 -- 172.21.15.152:0/14383771 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070017620 con 0x55eb9bd902f0 2024-09-11T22:53:17.733 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.726+0000 7f6066ffd640 1 -- 172.21.15.152:0/14383771 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6070017ca0 con 0x55eb9bd902f0 2024-09-11T22:53:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.726+0000 7f6066ffd640 1 -- 172.21.15.152:0/14383771 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607000fcf0 con 0x55eb9bd902f0 2024-09-11T22:53:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.726+0000 7f6084f3c740 1 -- 172.21.15.152:0/14383771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd902f0 msgr2=0x55eb9bd9a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.726+0000 7f6081100640 1 --2- 172.21.15.152:0/2884265116 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f606c0a2d80 0x7f606c0a5240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6074005cc0 tx=0x7f60740083f0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.726+0000 7f6084f3c740 1 --2- 172.21.15.152:0/14383771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd902f0 0x55eb9bd9a670 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f607000bb70 tx=0x7f607000d710 comp rx=0 tx=0).stop 2024-09-11T22:53:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.726+0000 7f6084f3c740 1 -- 172.21.15.152:0/14383771 shutdown_connections 2024-09-11T22:53:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.726+0000 7f6084f3c740 1 --2- 172.21.15.152:0/14383771 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd9abb0 0x55eb9bd9cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.726+0000 7f6084f3c740 1 --2- 172.21.15.152:0/14383771 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd902f0 0x55eb9bd9a670 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.726+0000 7f6084f3c740 1 --2- 172.21.15.152:0/14383771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bd8fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.726+0000 7f6084f3c740 1 -- 172.21.15.152:0/14383771 >> 172.21.15.152:0/14383771 conn(0x55eb9bd8bf60 msgr2=0x55eb9bd8c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.727+0000 7f6084f3c740 1 -- 172.21.15.152:0/14383771 shutdown_connections 2024-09-11T22:53:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.727+0000 7f6084f3c740 1 -- 172.21.15.152:0/14383771 wait complete. 2024-09-11T22:53:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.727+0000 7f6084f3c740 1 -- start start 2024-09-11T22:53:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.727+0000 7f6084f3c740 1 Processor -- start 2024-09-11T22:53:17.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.727+0000 7f6084f3c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bd9a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.727+0000 7f6084f3c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd902f0 0x55eb9bd976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.727+0000 7f6084f3c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd9abb0 0x55eb9bd97c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.727+0000 7f6084f3c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55eb9bd9d780 con 0x55eb9bd9abb0 2024-09-11T22:53:17.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.727+0000 7f6084f3c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55eb9bcb0f20 con 0x55eb9bd8f9d0 2024-09-11T22:53:17.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.727+0000 7f6084f3c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55eb9bd9d900 con 0x55eb9bd902f0 2024-09-11T22:53:17.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.727+0000 7f6084f3c740 10 client.0 _ll_register_callbacks cb 0x55eb9bc7a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:17.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f6081100640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd9abb0 0x55eb9bd97c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f6081100640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd9abb0 0x55eb9bd97c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47614/0 (socket says 172.21.15.152:47614) 2024-09-11T22:53:17.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f60808ff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bd9a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f6081100640 1 -- 172.21.15.152:0/2502310769 learned_addr learned my addr 172.21.15.152:0/2502310769 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:17.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f607bfff640 1 --2- 172.21.15.152:0/2502310769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd902f0 0x55eb9bd976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f6081100640 1 -- 172.21.15.152:0/2502310769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd902f0 msgr2=0x55eb9bd976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f6081100640 1 --2- 172.21.15.152:0/2502310769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd902f0 0x55eb9bd976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f6081100640 1 -- 172.21.15.152:0/2502310769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd8f9d0 msgr2=0x55eb9bd9a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f6081100640 1 --2- 172.21.15.152:0/2502310769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bd9a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f6081100640 1 -- 172.21.15.152:0/2502310769 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb9ba672b0 con 0x55eb9bd9abb0 2024-09-11T22:53:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f6081100640 1 --2- 172.21.15.152:0/2502310769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd9abb0 0x55eb9bd97c10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6074010d90 tx=0x7f6074004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1db10044288568d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f605bfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f605bfff640 1 -- 172.21.15.152:0/2502310769 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607401d620 con 0x55eb9bd9abb0 2024-09-11T22:53:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f605bfff640 1 -- 172.21.15.152:0/2502310769 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f607401d7c0 con 0x55eb9bd9abb0 2024-09-11T22:53:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f6084f3c740 1 -- 172.21.15.152:0/2502310769 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6070017ca0 con 0x55eb9bd9abb0 2024-09-11T22:53:17.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f605bfff640 1 -- 172.21.15.152:0/2502310769 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f607401d9c0 con 0x55eb9bd9abb0 2024-09-11T22:53:17.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f6084f3c740 10 client.15504 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:17.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.728+0000 7f6084f3c740 1 -- 172.21.15.152:0/2502310769 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb9bda9010 con 0x55eb9bd9abb0 2024-09-11T22:53:17.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.729+0000 7f605bfff640 1 -- 172.21.15.152:0/2502310769 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6070017ca0 con 0x55eb9bd9abb0 2024-09-11T22:53:17.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.729+0000 7f605bfff640 1 -- 172.21.15.152:0/2502310769 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f607402fc10 con 0x55eb9bd9abb0 2024-09-11T22:53:17.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.729+0000 7f6084f3c740 10 client.15504 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:17.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.729+0000 7f6084f3c740 10 client.15504 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T22:53:17.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.729+0000 7f6084f3c740 20 client.15504 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:17.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.729+0000 7f6084f3c740 1 --2- 172.21.15.152:0/2502310769 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55eb9bdab630 0x55eb9bdacaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.729+0000 7f6084f3c740 4 client.15504 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:17.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.729+0000 7f6084f3c740 1 -- 172.21.15.152:0/2502310769 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb9bd03600 con 0x55eb9bdab630 2024-09-11T22:53:17.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.729+0000 7f60808ff640 1 --2- 172.21.15.152:0/2502310769 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55eb9bdab630 0x55eb9bdacaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.729+0000 7f60808ff640 1 --2- 172.21.15.152:0/2502310769 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55eb9bdab630 0x55eb9bdacaa0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1fdf77f810fd39ae server_cookie=ea46450e2da9e27d in_seq=0 out_seq=0 2024-09-11T22:53:17.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.729+0000 7f605bfff640 10 client.15504 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:53:17.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.730+0000 7f605bfff640 1 -- 172.21.15.152:0/2502310769 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb9bd03600 con 0x55eb9bdab630 2024-09-11T22:53:17.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.730+0000 7f605bfff640 10 client.15504 handle_command_reply: tid=0 2024-09-11T22:53:17.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.730+0000 7f6084f3c740 1 client.15504 shutdown 2024-09-11T22:53:17.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.730+0000 7f6084f3c740 1 -- 172.21.15.152:0/2502310769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd9abb0 msgr2=0x55eb9bd97c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:17.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.730+0000 7f6084f3c740 1 --2- 172.21.15.152:0/2502310769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd9abb0 0x55eb9bd97c10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6074010d90 tx=0x7f6074004a40 comp rx=0 tx=0).stop 2024-09-11T22:53:17.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.730+0000 7f6084f3c740 1 -- 172.21.15.152:0/2502310769 shutdown_connections 2024-09-11T22:53:17.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.730+0000 7f6084f3c740 1 --2- 172.21.15.152:0/2502310769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd9abb0 0x55eb9bd97c10 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.730+0000 7f6084f3c740 1 --2- 172.21.15.152:0/2502310769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd902f0 0x55eb9bd976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.730+0000 7f6084f3c740 1 --2- 172.21.15.152:0/2502310769 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55eb9bdab630 0x55eb9bdacaa0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.730+0000 7f6084f3c740 1 --2- 172.21.15.152:0/2502310769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bd9a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.730+0000 7f6084f3c740 1 -- 172.21.15.152:0/2502310769 >> 172.21.15.152:0/2502310769 conn(0x55eb9bd8bf60 msgr2=0x55eb9bd91c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:17.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.730+0000 7f605bfff640 0 client.15504 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:53:17.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.730+0000 7f6084f3c740 1 -- 172.21.15.152:0/2502310769 shutdown_connections 2024-09-11T22:53:17.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.730+0000 7f6084f3c740 1 -- 172.21.15.152:0/2502310769 wait complete. 2024-09-11T22:53:17.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.730+0000 7f6084f3c740 20 client.15504 trim_cache size 0 max 16384 2024-09-11T22:53:17.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.748+0000 7f6084f3c740 1 Processor -- start 2024-09-11T22:53:17.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.748+0000 7f6084f3c740 1 -- start start 2024-09-11T22:53:17.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.748+0000 7f6084f3c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bdaaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.748+0000 7f6084f3c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd902f0 0x55eb9bdac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.748+0000 7f6084f3c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd9abb0 0x55eb9bd983a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.748+0000 7f6084f3c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55eb9bdaade0 con 0x55eb9bd8f9d0 2024-09-11T22:53:17.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.748+0000 7f6084f3c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55eb9bdab4b0 con 0x55eb9bd9abb0 2024-09-11T22:53:17.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.748+0000 7f6084f3c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55eb9bdabb80 con 0x55eb9bd902f0 2024-09-11T22:53:17.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.748+0000 7f60808ff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bdaaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.748+0000 7f60808ff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bdaaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47630/0 (socket says 172.21.15.152:47630) 2024-09-11T22:53:17.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.748+0000 7f60808ff640 1 -- 172.21.15.152:0/1235942124 learned_addr learned my addr 172.21.15.152:0/1235942124 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:17.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.748+0000 7f607bfff640 1 --2- 172.21.15.152:0/1235942124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd902f0 0x55eb9bdac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.748+0000 7f6081100640 1 --2- 172.21.15.152:0/1235942124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd9abb0 0x55eb9bd983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.749+0000 7f60808ff640 1 -- 172.21.15.152:0/1235942124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd902f0 msgr2=0x55eb9bdac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:17.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.749+0000 7f60808ff640 1 --2- 172.21.15.152:0/1235942124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd902f0 0x55eb9bdac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.749+0000 7f60808ff640 1 -- 172.21.15.152:0/1235942124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd9abb0 msgr2=0x55eb9bd983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:17.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.749+0000 7f60808ff640 1 --2- 172.21.15.152:0/1235942124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd9abb0 0x55eb9bd983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.749+0000 7f60808ff640 1 -- 172.21.15.152:0/1235942124 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb9bd8bd90 con 0x55eb9bd8f9d0 2024-09-11T22:53:17.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.749+0000 7f607bfff640 1 --2- 172.21.15.152:0/1235942124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd902f0 0x55eb9bdac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:17.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.749+0000 7f6081100640 1 --2- 172.21.15.152:0/1235942124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd9abb0 0x55eb9bd983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:17.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.749+0000 7f60808ff640 1 --2- 172.21.15.152:0/1235942124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bdaaf60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f607000ba90 tx=0x7f6070022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85905bb5faaaf928 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:17.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.749+0000 7f60667fc640 1 -- 172.21.15.152:0/1235942124 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607000f620 con 0x55eb9bd8f9d0 2024-09-11T22:53:17.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.749+0000 7f60667fc640 1 -- 172.21.15.152:0/1235942124 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f607002f070 con 0x55eb9bd8f9d0 2024-09-11T22:53:17.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.749+0000 7f60667fc640 1 -- 172.21.15.152:0/1235942124 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607002aa10 con 0x55eb9bd8f9d0 2024-09-11T22:53:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.750+0000 7f6084f3c740 1 -- 172.21.15.152:0/1235942124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd8f9d0 msgr2=0x55eb9bdaaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.750+0000 7f6084f3c740 1 --2- 172.21.15.152:0/1235942124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bdaaf60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f607000ba90 tx=0x7f6070022040 comp rx=0 tx=0).stop 2024-09-11T22:53:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.750+0000 7f6084f3c740 1 -- 172.21.15.152:0/1235942124 shutdown_connections 2024-09-11T22:53:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.750+0000 7f6084f3c740 1 --2- 172.21.15.152:0/1235942124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd9abb0 0x55eb9bd983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.750+0000 7f6084f3c740 1 --2- 172.21.15.152:0/1235942124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd902f0 0x55eb9bdac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.750+0000 7f6084f3c740 1 --2- 172.21.15.152:0/1235942124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bdaaf60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.750+0000 7f6084f3c740 1 -- 172.21.15.152:0/1235942124 >> 172.21.15.152:0/1235942124 conn(0x55eb9bd8bf60 msgr2=0x55eb9bd86ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.750+0000 7f6084f3c740 1 -- 172.21.15.152:0/1235942124 shutdown_connections 2024-09-11T22:53:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.750+0000 7f6084f3c740 1 -- 172.21.15.152:0/1235942124 wait complete. 2024-09-11T22:53:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.750+0000 7f6084f3c740 1 -- start start 2024-09-11T22:53:17.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.750+0000 7f6084f3c740 1 Processor -- start 2024-09-11T22:53:17.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f6084f3c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bdb2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f6084f3c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd902f0 0x55eb9bdb2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f6084f3c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd9abb0 0x55eb9be17d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f6084f3c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55eb9bdac920 con 0x55eb9bd9abb0 2024-09-11T22:53:17.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f6084f3c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55eb9bdac250 con 0x55eb9bd8f9d0 2024-09-11T22:53:17.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f6084f3c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55eb9bdacff0 con 0x55eb9bd902f0 2024-09-11T22:53:17.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f6084f3c740 10 client.0 _ll_register_callbacks cb 0x7f6074008a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:17.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f6081100640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd9abb0 0x55eb9be17d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f6081100640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd9abb0 0x55eb9be17d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47644/0 (socket says 172.21.15.152:47644) 2024-09-11T22:53:17.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f6081100640 1 -- 172.21.15.152:0/565605295 learned_addr learned my addr 172.21.15.152:0/565605295 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:17.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f60808ff640 1 --2- 172.21.15.152:0/565605295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bdb2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f607bfff640 1 --2- 172.21.15.152:0/565605295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd902f0 0x55eb9bdb2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f6081100640 1 -- 172.21.15.152:0/565605295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd902f0 msgr2=0x55eb9bdb2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:17.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f6081100640 1 --2- 172.21.15.152:0/565605295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd902f0 0x55eb9bdb2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f6081100640 1 -- 172.21.15.152:0/565605295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd8f9d0 msgr2=0x55eb9bdb2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:17.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f6081100640 1 --2- 172.21.15.152:0/565605295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bdb2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.751+0000 7f6081100640 1 -- 172.21.15.152:0/565605295 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb9be182a0 con 0x55eb9bd9abb0 2024-09-11T22:53:17.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.752+0000 7f60808ff640 1 --2- 172.21.15.152:0/565605295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bdb2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:17.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.752+0000 7f6081100640 1 --2- 172.21.15.152:0/565605295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd9abb0 0x55eb9be17d60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f60740107e0 tx=0x7f6074008b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f8c6dd71b04186 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:17.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.752+0000 7f605bfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:17.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.752+0000 7f605bfff640 1 -- 172.21.15.152:0/565605295 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607403e060 con 0x55eb9bd9abb0 2024-09-11T22:53:17.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.752+0000 7f605bfff640 1 -- 172.21.15.152:0/565605295 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f607401d620 con 0x55eb9bd9abb0 2024-09-11T22:53:17.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.752+0000 7f6084f3c740 1 -- 172.21.15.152:0/565605295 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f607002f070 con 0x55eb9bd9abb0 2024-09-11T22:53:17.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.752+0000 7f605bfff640 1 -- 172.21.15.152:0/565605295 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6074035070 con 0x55eb9bd9abb0 2024-09-11T22:53:17.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.752+0000 7f6084f3c740 10 client.15510 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:17.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.752+0000 7f6084f3c740 1 -- 172.21.15.152:0/565605295 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb9be194e0 con 0x55eb9bd9abb0 2024-09-11T22:53:17.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.752+0000 7f605bfff640 1 -- 172.21.15.152:0/565605295 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f607002f070 con 0x55eb9bd9abb0 2024-09-11T22:53:17.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.753+0000 7f605bfff640 1 -- 172.21.15.152:0/565605295 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f607402f720 con 0x55eb9bd9abb0 2024-09-11T22:53:17.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.753+0000 7f6084f3c740 10 client.15510 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:17.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.753+0000 7f6084f3c740 10 client.15510 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T22:53:17.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.753+0000 7f6084f3c740 20 client.15510 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:17.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.753+0000 7f6084f3c740 1 --2- 172.21.15.152:0/565605295 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55eb9be1e0f0 0x55eb9be1ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:17.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.753+0000 7f6084f3c740 4 client.15510 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:53:17.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.753+0000 7f6084f3c740 1 -- 172.21.15.152:0/565605295 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "status"}) -- 0x55eb9bcfdb10 con 0x55eb9be1e0f0 2024-09-11T22:53:17.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.753+0000 7f60808ff640 1 --2- 172.21.15.152:0/565605295 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55eb9be1e0f0 0x55eb9be1ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:17.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.753+0000 7f60808ff640 1 --2- 172.21.15.152:0/565605295 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55eb9be1e0f0 0x55eb9be1ce00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=947b5b070b3f48df server_cookie=6def08269845c7a3 in_seq=0 out_seq=0 2024-09-11T22:53:17.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.753+0000 7f605bfff640 10 client.15510 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:53:17.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.753+0000 7f605bfff640 1 -- 172.21.15.152:0/565605295 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55eb9bcfdb10 con 0x55eb9be1e0f0 2024-09-11T22:53:17.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.753+0000 7f605bfff640 10 client.15510 handle_command_reply: tid=0 2024-09-11T22:53:17.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.753+0000 7f6084f3c740 1 client.15510 shutdown 2024-09-11T22:53:17.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.754+0000 7f6084f3c740 1 -- 172.21.15.152:0/565605295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd9abb0 msgr2=0x55eb9be17d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:17.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.754+0000 7f6084f3c740 1 --2- 172.21.15.152:0/565605295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd9abb0 0x55eb9be17d60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f60740107e0 tx=0x7f6074008b70 comp rx=0 tx=0).stop 2024-09-11T22:53:17.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.754+0000 7f6084f3c740 1 -- 172.21.15.152:0/565605295 shutdown_connections 2024-09-11T22:53:17.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.754+0000 7f6084f3c740 1 --2- 172.21.15.152:0/565605295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb9bd9abb0 0x55eb9be17d60 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.754+0000 7f6084f3c740 1 --2- 172.21.15.152:0/565605295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb9bd902f0 0x55eb9bdb2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.754+0000 7f6084f3c740 1 --2- 172.21.15.152:0/565605295 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55eb9be1e0f0 0x55eb9be1ce00 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.754+0000 7f6084f3c740 1 --2- 172.21.15.152:0/565605295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb9bd8f9d0 0x55eb9bdb2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.754+0000 7f605bfff640 0 client.15510 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:53:17.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.754+0000 7f6084f3c740 1 -- 172.21.15.152:0/565605295 >> 172.21.15.152:0/565605295 conn(0x55eb9bd8bf60 msgr2=0x55eb9bd8c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:17.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.754+0000 7f6084f3c740 1 -- 172.21.15.152:0/565605295 shutdown_connections 2024-09-11T22:53:17.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.754+0000 7f6084f3c740 1 -- 172.21.15.152:0/565605295 wait complete. 2024-09-11T22:53:17.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.754+0000 7f6084f3c740 20 client.15510 trim_cache size 0 max 16384 2024-09-11T22:53:17.761 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:53:17.761 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:53:17.761 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 4, 2024-09-11T22:53:17.762 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 14628, 2024-09-11T22:53:17.762 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-09-11T22:53:17.762 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-09-11T22:53:17.762 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-09-11T22:53:17.762 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 132.62672065999999, 2024-09-11T22:53:17.762 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 28, 2024-09-11T22:53:17.762 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 84, 2024-09-11T22:53:17.762 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 83, 2024-09-11T22:53:17.762 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 157.91647867699999 2024-09-11T22:53:17.762 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:53:17.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.755+0000 7f6082b63640 1 -- 172.21.15.152:0/2884265116 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f606c0a2d80 msgr2=0x7f606c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:17.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.755+0000 7f6082b63640 1 --2- 172.21.15.152:0/2884265116 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f606c0a2d80 0x7f606c0a5240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6074005cc0 tx=0x7f60740083f0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.755+0000 7f6082b63640 1 -- 172.21.15.152:0/2884265116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f607c102790 msgr2=0x7f607c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:17.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.755+0000 7f6082b63640 1 --2- 172.21.15.152:0/2884265116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f607c102790 0x7f607c198910 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6068005ab0 tx=0x7f60680042f0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.755+0000 7f6082b63640 1 -- 172.21.15.152:0/2884265116 shutdown_connections 2024-09-11T22:53:17.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.755+0000 7f6082b63640 1 --2- 172.21.15.152:0/2884265116 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f606c0a2d80 0x7f606c0a5240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.755+0000 7f6082b63640 1 --2- 172.21.15.152:0/2884265116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f607c106f10 0x7f607c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.755+0000 7f6082b63640 1 --2- 172.21.15.152:0/2884265116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f607c102790 0x7f607c198910 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.755+0000 7f6082b63640 1 --2- 172.21.15.152:0/2884265116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f607c101de0 0x7f607c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:17.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.755+0000 7f6082b63640 1 -- 172.21.15.152:0/2884265116 >> 172.21.15.152:0/2884265116 conn(0x7f607c0fd8c0 msgr2=0x7f607c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:17.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.755+0000 7f6082b63640 1 -- 172.21.15.152:0/2884265116 shutdown_connections 2024-09-11T22:53:17.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:17.755+0000 7f6082b63640 1 -- 172.21.15.152:0/2884265116 wait complete. 2024-09-11T22:53:18.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:18 smithi191 ceph-mon[35211]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:53:18.470 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:18 smithi152 ceph-mon[31181]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:53:18.498 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T22:53:18.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:18 smithi204 ceph-mon[35454]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:53:18.758 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:19.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.261+0000 7fb63a960640 1 -- 172.21.15.152:0/1202971223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb634106a80 msgr2=0x7fb634106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:19.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.261+0000 7fb63a960640 1 --2- 172.21.15.152:0/1202971223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb634106a80 0x7fb634106e60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb624009930 tx=0x7fb62402a420 comp rx=0 tx=0).stop 2024-09-11T22:53:19.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.261+0000 7fb63a960640 1 -- 172.21.15.152:0/1202971223 shutdown_connections 2024-09-11T22:53:19.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.261+0000 7fb63a960640 1 --2- 172.21.15.152:0/1202971223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb634101420 0x7fb63410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.261+0000 7fb63a960640 1 --2- 172.21.15.152:0/1202971223 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb634100a80 0x7fb634100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.261+0000 7fb63a960640 1 --2- 172.21.15.152:0/1202971223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb634106a80 0x7fb634106e60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.261+0000 7fb63a960640 1 -- 172.21.15.152:0/1202971223 >> 172.21.15.152:0/1202971223 conn(0x7fb6340fc7a0 msgr2=0x7fb6340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:19.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.261+0000 7fb63a960640 1 -- 172.21.15.152:0/1202971223 shutdown_connections 2024-09-11T22:53:19.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.261+0000 7fb63a960640 1 -- 172.21.15.152:0/1202971223 wait complete. 2024-09-11T22:53:19.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.262+0000 7fb63a960640 1 Processor -- start 2024-09-11T22:53:19.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.262+0000 7fb63a960640 1 -- start start 2024-09-11T22:53:19.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.262+0000 7fb63a960640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb634100a80 0x7fb634198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.262+0000 7fb63a960640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb634101420 0x7fb634198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.262+0000 7fb63a960640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb634106a80 0x7fb63419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.262+0000 7fb63a960640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb634110ea0 con 0x7fb634101420 2024-09-11T22:53:19.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.262+0000 7fb63a960640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb634110d20 con 0x7fb634100a80 2024-09-11T22:53:19.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.262+0000 7fb63a960640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb634111020 con 0x7fb634106a80 2024-09-11T22:53:19.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.263+0000 7fb6337fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb634101420 0x7fb634198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:19.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.263+0000 7fb6337fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb634101420 0x7fb634198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36938/0 (socket says 172.21.15.152:36938) 2024-09-11T22:53:19.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.263+0000 7fb6337fe640 1 -- 172.21.15.152:0/3845419190 learned_addr learned my addr 172.21.15.152:0/3845419190 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:19.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.263+0000 7fb633fff640 1 --2- 172.21.15.152:0/3845419190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb634100a80 0x7fb634198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:19.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.263+0000 7fb638efd640 1 --2- 172.21.15.152:0/3845419190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb634106a80 0x7fb63419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:19.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.263+0000 7fb6337fe640 1 -- 172.21.15.152:0/3845419190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb634106a80 msgr2=0x7fb63419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:19.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.263+0000 7fb6337fe640 1 --2- 172.21.15.152:0/3845419190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb634106a80 0x7fb63419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.263+0000 7fb6337fe640 1 -- 172.21.15.152:0/3845419190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb634100a80 msgr2=0x7fb634198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:19.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.263+0000 7fb6337fe640 1 --2- 172.21.15.152:0/3845419190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb634100a80 0x7fb634198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.263+0000 7fb6337fe640 1 -- 172.21.15.152:0/3845419190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63419d4c0 con 0x7fb634101420 2024-09-11T22:53:19.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.263+0000 7fb6337fe640 1 --2- 172.21.15.152:0/3845419190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb634101420 0x7fb634198a50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb62800d950 tx=0x7fb62800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:19.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.263+0000 7fb6317fa640 1 -- 172.21.15.152:0/3845419190 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb628017070 con 0x7fb634101420 2024-09-11T22:53:19.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.263+0000 7fb6317fa640 1 -- 172.21.15.152:0/3845419190 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb628009c30 con 0x7fb634101420 2024-09-11T22:53:19.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.263+0000 7fb63a960640 1 -- 172.21.15.152:0/3845419190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb63419d7b0 con 0x7fb634101420 2024-09-11T22:53:19.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.263+0000 7fb6317fa640 1 -- 172.21.15.152:0/3845419190 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62800f6b0 con 0x7fb634101420 2024-09-11T22:53:19.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.263+0000 7fb63a960640 1 -- 172.21.15.152:0/3845419190 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6341a5090 con 0x7fb634101420 2024-09-11T22:53:19.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.265+0000 7fb6317fa640 1 -- 172.21.15.152:0/3845419190 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7fb6280109d0 con 0x7fb634101420 2024-09-11T22:53:19.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.266+0000 7fb63cd39740 1 Processor -- start 2024-09-11T22:53:19.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.266+0000 7fb63cd39740 1 -- start start 2024-09-11T22:53:19.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.266+0000 7fb6317fa640 1 --2- 172.21.15.152:0/3845419190 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb61c0a2d80 0x7fb61c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.266+0000 7fb638efd640 1 --2- 172.21.15.152:0/3845419190 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb61c0a2d80 0x7fb61c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:19.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.266+0000 7fb6317fa640 1 -- 172.21.15.152:0/3845419190 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb6280cf160 con 0x7fb634101420 2024-09-11T22:53:19.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.266+0000 7fb63cd39740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b207890 0x55ca9b207c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.266+0000 7fb63cd39740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b2081b0 0x55ca9b216a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.266+0000 7fb63cd39740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b216fc0 0x55ca9b2193b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.266+0000 7fb63cd39740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ca9b16e090 con 0x55ca9b207890 2024-09-11T22:53:19.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.266+0000 7fb63cd39740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ca9b115a70 con 0x55ca9b2081b0 2024-09-11T22:53:19.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.266+0000 7fb63cd39740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ca9b16a050 con 0x55ca9b216fc0 2024-09-11T22:53:19.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.266+0000 7fb633fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b207890 0x55ca9b207c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:19.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.266+0000 7fb633fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b207890 0x55ca9b207c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36954/0 (socket says 172.21.15.152:36954) 2024-09-11T22:53:19.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.266+0000 7fb633fff640 1 -- 172.21.15.152:0/530830414 learned_addr learned my addr 172.21.15.152:0/530830414 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:19.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.267+0000 7fb6337fe640 1 --2- 172.21.15.152:0/530830414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b2081b0 0x55ca9b216a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:19.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.267+0000 7fb638efd640 1 --2- 172.21.15.152:0/530830414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b216fc0 0x55ca9b2193b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:19.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.267+0000 7fb633fff640 1 -- 172.21.15.152:0/530830414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b216fc0 msgr2=0x55ca9b2193b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:19.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.267+0000 7fb633fff640 1 --2- 172.21.15.152:0/530830414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b216fc0 0x55ca9b2193b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.267+0000 7fb633fff640 1 -- 172.21.15.152:0/530830414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b2081b0 msgr2=0x55ca9b216a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:19.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.267+0000 7fb633fff640 1 --2- 172.21.15.152:0/530830414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b2081b0 0x55ca9b216a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.267+0000 7fb633fff640 1 -- 172.21.15.152:0/530830414 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca9b122130 con 0x55ca9b207890 2024-09-11T22:53:19.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.267+0000 7fb638efd640 1 --2- 172.21.15.152:0/3845419190 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb61c0a2d80 0x7fb61c0a5240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009f70 tx=0x7fb62c009650 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:19.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.267+0000 7fb633fff640 1 --2- 172.21.15.152:0/530830414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b207890 0x55ca9b207c70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb624009930 tx=0x7fb624031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c289586286709a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:19.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.267+0000 7fb61a7fc640 1 -- 172.21.15.152:0/530830414 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624040070 con 0x55ca9b207890 2024-09-11T22:53:19.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.267+0000 7fb61a7fc640 1 -- 172.21.15.152:0/530830414 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb624005020 con 0x55ca9b207890 2024-09-11T22:53:19.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.267+0000 7fb61a7fc640 1 -- 172.21.15.152:0/530830414 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624035040 con 0x55ca9b207890 2024-09-11T22:53:19.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.268+0000 7fb63cd39740 1 -- 172.21.15.152:0/530830414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b207890 msgr2=0x55ca9b207c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:19.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.268+0000 7fb63cd39740 1 --2- 172.21.15.152:0/530830414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b207890 0x55ca9b207c70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb624009930 tx=0x7fb624031630 comp rx=0 tx=0).stop 2024-09-11T22:53:19.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.268+0000 7fb63cd39740 1 -- 172.21.15.152:0/530830414 shutdown_connections 2024-09-11T22:53:19.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.268+0000 7fb63cd39740 1 --2- 172.21.15.152:0/530830414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b216fc0 0x55ca9b2193b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.268+0000 7fb63cd39740 1 --2- 172.21.15.152:0/530830414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b2081b0 0x55ca9b216a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.268+0000 7fb63cd39740 1 --2- 172.21.15.152:0/530830414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b207890 0x55ca9b207c70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.268+0000 7fb63cd39740 1 -- 172.21.15.152:0/530830414 >> 172.21.15.152:0/530830414 conn(0x55ca9b203f60 msgr2=0x55ca9b204340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:19.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.268+0000 7fb63cd39740 1 -- 172.21.15.152:0/530830414 shutdown_connections 2024-09-11T22:53:19.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.268+0000 7fb63cd39740 1 -- 172.21.15.152:0/530830414 wait complete. 2024-09-11T22:53:19.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.268+0000 7fb63cd39740 1 -- start start 2024-09-11T22:53:19.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.268+0000 7fb63cd39740 1 Processor -- start 2024-09-11T22:53:19.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb63cd39740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b207890 0x55ca9b216780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb63cd39740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b2081b0 0x55ca9b213b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb63cd39740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b216fc0 0x55ca9b214060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb63cd39740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ca9b219b30 con 0x55ca9b216fc0 2024-09-11T22:53:19.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb63cd39740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ca9b128f20 con 0x55ca9b2081b0 2024-09-11T22:53:19.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb63cd39740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ca9b219cb0 con 0x55ca9b207890 2024-09-11T22:53:19.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb63cd39740 10 client.0 _ll_register_callbacks cb 0x55ca9b0f2030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:19.307 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:53:19.307 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:53:19.308 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 3, 2024-09-11T22:53:19.308 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 24430, 2024-09-11T22:53:19.308 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-09-11T22:53:19.308 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-09-11T22:53:19.308 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-09-11T22:53:19.308 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 135.07370660000001, 2024-09-11T22:53:19.308 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 28, 2024-09-11T22:53:19.308 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 84, 2024-09-11T22:53:19.308 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 83, 2024-09-11T22:53:19.308 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 170.949555694 2024-09-11T22:53:19.308 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:53:19.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb638efd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b216fc0 0x55ca9b214060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:19.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb6337fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b2081b0 0x55ca9b213b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:19.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb638efd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b216fc0 0x55ca9b214060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36964/0 (socket says 172.21.15.152:36964) 2024-09-11T22:53:19.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb638efd640 1 -- 172.21.15.152:0/527824768 learned_addr learned my addr 172.21.15.152:0/527824768 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:19.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb633fff640 1 --2- 172.21.15.152:0/527824768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b207890 0x55ca9b216780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:19.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb638efd640 1 -- 172.21.15.152:0/527824768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b207890 msgr2=0x55ca9b216780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:19.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb638efd640 1 --2- 172.21.15.152:0/527824768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b207890 0x55ca9b216780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb638efd640 1 -- 172.21.15.152:0/527824768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b2081b0 msgr2=0x55ca9b213b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:19.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb638efd640 1 --2- 172.21.15.152:0/527824768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b2081b0 0x55ca9b213b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb638efd640 1 -- 172.21.15.152:0/527824768 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca9aedf2b0 con 0x55ca9b216fc0 2024-09-11T22:53:19.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb638efd640 1 --2- 172.21.15.152:0/527824768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b216fc0 0x55ca9b214060 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb62c018f20 tx=0x7fb62c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f61cb313d751ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:19.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb6137fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:19.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.269+0000 7fb6137fe640 1 -- 172.21.15.152:0/527824768 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c017e60 con 0x55ca9b216fc0 2024-09-11T22:53:19.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.270+0000 7fb63cd39740 1 -- 172.21.15.152:0/527824768 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb624005020 con 0x55ca9b216fc0 2024-09-11T22:53:19.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.270+0000 7fb6137fe640 1 -- 172.21.15.152:0/527824768 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb62c0207b0 con 0x55ca9b216fc0 2024-09-11T22:53:19.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.270+0000 7fb6137fe640 1 -- 172.21.15.152:0/527824768 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb62c02f050 con 0x55ca9b216fc0 2024-09-11T22:53:19.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.270+0000 7fb63cd39740 10 client.15522 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:19.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.270+0000 7fb63cd39740 1 -- 172.21.15.152:0/527824768 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca9b225440 con 0x55ca9b216fc0 2024-09-11T22:53:19.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.270+0000 7fb6137fe640 1 -- 172.21.15.152:0/527824768 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb624005020 con 0x55ca9b216fc0 2024-09-11T22:53:19.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.270+0000 7fb6137fe640 1 -- 172.21.15.152:0/527824768 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb62c017070 con 0x55ca9b216fc0 2024-09-11T22:53:19.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.270+0000 7fb63cd39740 10 client.15522 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:19.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.270+0000 7fb63cd39740 10 client.15522 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T22:53:19.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.270+0000 7fb63cd39740 20 client.15522 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:19.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.270+0000 7fb63cd39740 1 --2- 172.21.15.152:0/527824768 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55ca9b227ae0 0x55ca9b228f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.270+0000 7fb63cd39740 4 client.15522 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:19.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.270+0000 7fb63cd39740 1 -- 172.21.15.152:0/527824768 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca9b17b600 con 0x55ca9b227ae0 2024-09-11T22:53:19.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.270+0000 7fb633fff640 1 --2- 172.21.15.152:0/527824768 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55ca9b227ae0 0x55ca9b228f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:19.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.271+0000 7fb6137fe640 10 client.15522 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:53:19.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.271+0000 7fb633fff640 1 --2- 172.21.15.152:0/527824768 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55ca9b227ae0 0x55ca9b228f50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b0b123403840b2ee server_cookie=bbb31070632f1fbe in_seq=0 out_seq=0 2024-09-11T22:53:19.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.272+0000 7fb6137fe640 1 -- 172.21.15.152:0/527824768 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca9b17b600 con 0x55ca9b227ae0 2024-09-11T22:53:19.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.272+0000 7fb6137fe640 10 client.15522 handle_command_reply: tid=0 2024-09-11T22:53:19.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.272+0000 7fb63cd39740 1 client.15522 shutdown 2024-09-11T22:53:19.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.272+0000 7fb63cd39740 1 -- 172.21.15.152:0/527824768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b216fc0 msgr2=0x55ca9b214060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:19.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.272+0000 7fb63cd39740 1 --2- 172.21.15.152:0/527824768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b216fc0 0x55ca9b214060 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb62c018f20 tx=0x7fb62c013000 comp rx=0 tx=0).stop 2024-09-11T22:53:19.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.272+0000 7fb63cd39740 1 -- 172.21.15.152:0/527824768 shutdown_connections 2024-09-11T22:53:19.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.272+0000 7fb63cd39740 1 --2- 172.21.15.152:0/527824768 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55ca9b227ae0 0x55ca9b228f50 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.272+0000 7fb63cd39740 1 --2- 172.21.15.152:0/527824768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b216fc0 0x55ca9b214060 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.272+0000 7fb63cd39740 1 --2- 172.21.15.152:0/527824768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b2081b0 0x55ca9b213b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.272+0000 7fb6137fe640 0 client.15522 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:53:19.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.272+0000 7fb63cd39740 1 --2- 172.21.15.152:0/527824768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b207890 0x55ca9b216780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.272+0000 7fb63cd39740 1 -- 172.21.15.152:0/527824768 >> 172.21.15.152:0/527824768 conn(0x55ca9b203f60 msgr2=0x55ca9b209cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:19.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.272+0000 7fb63cd39740 1 -- 172.21.15.152:0/527824768 shutdown_connections 2024-09-11T22:53:19.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.272+0000 7fb63cd39740 1 -- 172.21.15.152:0/527824768 wait complete. 2024-09-11T22:53:19.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.272+0000 7fb63cd39740 20 client.15522 trim_cache size 0 max 16384 2024-09-11T22:53:19.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb63cd39740 1 Processor -- start 2024-09-11T22:53:19.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb63cd39740 1 -- start start 2024-09-11T22:53:19.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb63cd39740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b207890 0x55ca9b228f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb63cd39740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b2081b0 0x55ca9b1fece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb63cd39740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b216fc0 0x55ca9b231230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb63cd39740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ca9b227290 con 0x55ca9b216fc0 2024-09-11T22:53:19.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb63cd39740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ca9b227960 con 0x55ca9b2081b0 2024-09-11T22:53:19.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb63cd39740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ca9b228030 con 0x55ca9b207890 2024-09-11T22:53:19.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb638efd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b216fc0 0x55ca9b231230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:19.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb638efd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b216fc0 0x55ca9b231230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36972/0 (socket says 172.21.15.152:36972) 2024-09-11T22:53:19.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb638efd640 1 -- 172.21.15.152:0/2810377822 learned_addr learned my addr 172.21.15.152:0/2810377822 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:19.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb633fff640 1 --2- 172.21.15.152:0/2810377822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b207890 0x55ca9b228f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:19.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb638efd640 1 -- 172.21.15.152:0/2810377822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b207890 msgr2=0x55ca9b228f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:19.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb638efd640 1 --2- 172.21.15.152:0/2810377822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b207890 0x55ca9b228f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb638efd640 1 -- 172.21.15.152:0/2810377822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b2081b0 msgr2=0x55ca9b1fece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:19.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb6337fe640 1 --2- 172.21.15.152:0/2810377822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b2081b0 0x55ca9b1fece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:19.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb638efd640 1 --2- 172.21.15.152:0/2810377822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b2081b0 0x55ca9b1fece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb638efd640 1 -- 172.21.15.152:0/2810377822 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca9b219ca0 con 0x55ca9b216fc0 2024-09-11T22:53:19.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.290+0000 7fb6337fe640 1 --2- 172.21.15.152:0/2810377822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b2081b0 0x55ca9b1fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:19.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.291+0000 7fb638efd640 1 --2- 172.21.15.152:0/2810377822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b216fc0 0x55ca9b231230 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00d080 tx=0x7fb62c009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b8f6f1a5d64052 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:19.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.291+0000 7fb619ffb640 1 -- 172.21.15.152:0/2810377822 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c037020 con 0x55ca9b216fc0 2024-09-11T22:53:19.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.291+0000 7fb619ffb640 1 -- 172.21.15.152:0/2810377822 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb62c026070 con 0x55ca9b216fc0 2024-09-11T22:53:19.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.291+0000 7fb619ffb640 1 -- 172.21.15.152:0/2810377822 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c020cf0 con 0x55ca9b216fc0 2024-09-11T22:53:19.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.291+0000 7fb63cd39740 1 -- 172.21.15.152:0/2810377822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b216fc0 msgr2=0x55ca9b231230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:19.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.291+0000 7fb63cd39740 1 --2- 172.21.15.152:0/2810377822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b216fc0 0x55ca9b231230 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00d080 tx=0x7fb62c009970 comp rx=0 tx=0).stop 2024-09-11T22:53:19.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.291+0000 7fb63cd39740 1 -- 172.21.15.152:0/2810377822 shutdown_connections 2024-09-11T22:53:19.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.291+0000 7fb63cd39740 1 --2- 172.21.15.152:0/2810377822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b216fc0 0x55ca9b231230 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.291+0000 7fb63cd39740 1 --2- 172.21.15.152:0/2810377822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b2081b0 0x55ca9b1fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.291+0000 7fb63cd39740 1 --2- 172.21.15.152:0/2810377822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b207890 0x55ca9b228f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.291+0000 7fb63cd39740 1 -- 172.21.15.152:0/2810377822 >> 172.21.15.152:0/2810377822 conn(0x55ca9b203f60 msgr2=0x55ca9b204340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:19.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.291+0000 7fb63cd39740 1 -- 172.21.15.152:0/2810377822 shutdown_connections 2024-09-11T22:53:19.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.292+0000 7fb63cd39740 1 -- 172.21.15.152:0/2810377822 wait complete. 2024-09-11T22:53:19.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.292+0000 7fb63cd39740 1 -- start start 2024-09-11T22:53:19.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.292+0000 7fb63cd39740 1 Processor -- start 2024-09-11T22:53:19.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.292+0000 7fb63cd39740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b207890 0x55ca9b22a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.292+0000 7fb63cd39740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b2081b0 0x55ca9b22ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.292+0000 7fb63cd39740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b216fc0 0x55ca9b2901e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.292+0000 7fb63cd39740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ca9b228dd0 con 0x55ca9b207890 2024-09-11T22:53:19.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.292+0000 7fb63cd39740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ca9b228700 con 0x55ca9b216fc0 2024-09-11T22:53:19.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.292+0000 7fb63cd39740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ca9b2294a0 con 0x55ca9b2081b0 2024-09-11T22:53:19.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.292+0000 7fb633fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b207890 0x55ca9b22a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:19.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.292+0000 7fb63cd39740 10 client.0 _ll_register_callbacks cb 0x7fb62c018c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:19.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.292+0000 7fb633fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b207890 0x55ca9b22a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36988/0 (socket says 172.21.15.152:36988) 2024-09-11T22:53:19.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.292+0000 7fb633fff640 1 -- 172.21.15.152:0/3564102281 learned_addr learned my addr 172.21.15.152:0/3564102281 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:19.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb633fff640 1 -- 172.21.15.152:0/3564102281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b2081b0 msgr2=0x55ca9b22ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:19.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb633fff640 1 --2- 172.21.15.152:0/3564102281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b2081b0 0x55ca9b22ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb633fff640 1 -- 172.21.15.152:0/3564102281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b216fc0 msgr2=0x55ca9b2901e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:19.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb633fff640 1 --2- 172.21.15.152:0/3564102281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b216fc0 0x55ca9b2901e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb633fff640 1 -- 172.21.15.152:0/3564102281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca9b290720 con 0x55ca9b207890 2024-09-11T22:53:19.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb633fff640 1 --2- 172.21.15.152:0/3564102281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b207890 0x55ca9b22a6f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb6240096c0 tx=0x7fb6240095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3ab261c8439aa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:19.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb6137fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:19.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb6137fe640 1 -- 172.21.15.152:0/3564102281 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624040040 con 0x55ca9b207890 2024-09-11T22:53:19.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb6137fe640 1 -- 172.21.15.152:0/3564102281 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb62404f630 con 0x55ca9b207890 2024-09-11T22:53:19.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb63cd39740 1 -- 172.21.15.152:0/3564102281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb62c026070 con 0x55ca9b207890 2024-09-11T22:53:19.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb6137fe640 1 -- 172.21.15.152:0/3564102281 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb62405d050 con 0x55ca9b207890 2024-09-11T22:53:19.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb63cd39740 10 client.15528 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:19.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb63cd39740 1 -- 172.21.15.152:0/3564102281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca9b2918d0 con 0x55ca9b207890 2024-09-11T22:53:19.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb6137fe640 1 -- 172.21.15.152:0/3564102281 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb62c026070 con 0x55ca9b207890 2024-09-11T22:53:19.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb6137fe640 1 -- 172.21.15.152:0/3564102281 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb62403cb60 con 0x55ca9b207890 2024-09-11T22:53:19.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb63cd39740 10 client.15528 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:19.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb63cd39740 10 client.15528 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T22:53:19.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb63cd39740 20 client.15528 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:19.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb63cd39740 1 --2- 172.21.15.152:0/3564102281 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55ca9b2961a0 0x55ca9b294eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:19.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb63cd39740 4 client.15528 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:53:19.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.293+0000 7fb63cd39740 1 -- 172.21.15.152:0/3564102281 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "status"}) -- 0x55ca9b175b10 con 0x55ca9b2961a0 2024-09-11T22:53:19.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.294+0000 7fb6337fe640 1 --2- 172.21.15.152:0/3564102281 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55ca9b2961a0 0x55ca9b294eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:19.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.294+0000 7fb6337fe640 1 --2- 172.21.15.152:0/3564102281 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55ca9b2961a0 0x55ca9b294eb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2c48747160886e08 server_cookie=eaf25c89ef6d2106 in_seq=0 out_seq=0 2024-09-11T22:53:19.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.294+0000 7fb6137fe640 10 client.15528 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:53:19.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.294+0000 7fb6137fe640 1 -- 172.21.15.152:0/3564102281 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ca9b175b10 con 0x55ca9b2961a0 2024-09-11T22:53:19.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.294+0000 7fb6137fe640 10 client.15528 handle_command_reply: tid=0 2024-09-11T22:53:19.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.294+0000 7fb63cd39740 1 client.15528 shutdown 2024-09-11T22:53:19.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.294+0000 7fb63cd39740 1 -- 172.21.15.152:0/3564102281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b207890 msgr2=0x55ca9b22a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:19.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.294+0000 7fb63cd39740 1 --2- 172.21.15.152:0/3564102281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b207890 0x55ca9b22a6f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb6240096c0 tx=0x7fb6240095b0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.294+0000 7fb63cd39740 1 -- 172.21.15.152:0/3564102281 shutdown_connections 2024-09-11T22:53:19.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.294+0000 7fb63cd39740 1 --2- 172.21.15.152:0/3564102281 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55ca9b2961a0 0x55ca9b294eb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63cd39740 1 --2- 172.21.15.152:0/3564102281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca9b216fc0 0x55ca9b2901e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb6137fe640 0 client.15528 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:53:19.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63cd39740 1 --2- 172.21.15.152:0/3564102281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9b2081b0 0x55ca9b22ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63cd39740 1 --2- 172.21.15.152:0/3564102281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca9b207890 0x55ca9b22a6f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63cd39740 1 -- 172.21.15.152:0/3564102281 >> 172.21.15.152:0/3564102281 conn(0x55ca9b203f60 msgr2=0x55ca9b1fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:19.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63cd39740 1 -- 172.21.15.152:0/3564102281 shutdown_connections 2024-09-11T22:53:19.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63cd39740 1 -- 172.21.15.152:0/3564102281 wait complete. 2024-09-11T22:53:19.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63cd39740 20 client.15528 trim_cache size 0 max 16384 2024-09-11T22:53:19.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63a960640 1 -- 172.21.15.152:0/3845419190 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb61c0a2d80 msgr2=0x7fb61c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:19.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63a960640 1 --2- 172.21.15.152:0/3845419190 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb61c0a2d80 0x7fb61c0a5240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009f70 tx=0x7fb62c009650 comp rx=0 tx=0).stop 2024-09-11T22:53:19.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63a960640 1 -- 172.21.15.152:0/3845419190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb634101420 msgr2=0x7fb634198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:19.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63a960640 1 --2- 172.21.15.152:0/3845419190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb634101420 0x7fb634198a50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb62800d950 tx=0x7fb62800de10 comp rx=0 tx=0).stop 2024-09-11T22:53:19.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63a960640 1 -- 172.21.15.152:0/3845419190 shutdown_connections 2024-09-11T22:53:19.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63a960640 1 --2- 172.21.15.152:0/3845419190 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb61c0a2d80 0x7fb61c0a5240 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63a960640 1 --2- 172.21.15.152:0/3845419190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb634106a80 0x7fb63419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63a960640 1 --2- 172.21.15.152:0/3845419190 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb634101420 0x7fb634198a50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63a960640 1 --2- 172.21.15.152:0/3845419190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb634100a80 0x7fb634198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:19.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63a960640 1 -- 172.21.15.152:0/3845419190 >> 172.21.15.152:0/3845419190 conn(0x7fb6340fc7a0 msgr2=0x7fb63406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:19.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63a960640 1 -- 172.21.15.152:0/3845419190 shutdown_connections 2024-09-11T22:53:19.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:19.295+0000 7fb63a960640 1 -- 172.21.15.152:0/3845419190 wait complete. 2024-09-11T22:53:20.128 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T22:53:20.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:20 smithi152 ceph-mon[31181]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T22:53:20.379 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:20.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:20 smithi191 ceph-mon[35211]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T22:53:20.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:20 smithi204 ceph-mon[35454]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-11T22:53:20.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.881+0000 7f7a6f4c3640 1 -- 172.21.15.152:0/736387018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a68101420 msgr2=0x7f7a6810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:20.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.881+0000 7f7a6f4c3640 1 --2- 172.21.15.152:0/736387018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a68101420 0x7f7a6810d950 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009cd0 tx=0x7f7a6002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.881+0000 7f7a6f4c3640 1 -- 172.21.15.152:0/736387018 shutdown_connections 2024-09-11T22:53:20.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.881+0000 7f7a6f4c3640 1 --2- 172.21.15.152:0/736387018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a68101420 0x7f7a6810d950 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.881+0000 7f7a6f4c3640 1 --2- 172.21.15.152:0/736387018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a68100a80 0x7f7a68100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.881+0000 7f7a6f4c3640 1 --2- 172.21.15.152:0/736387018 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a68106a80 0x7f7a68106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.881+0000 7f7a6f4c3640 1 -- 172.21.15.152:0/736387018 >> 172.21.15.152:0/736387018 conn(0x7f7a680fc7a0 msgr2=0x7f7a680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:20.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.881+0000 7f7a6f4c3640 1 -- 172.21.15.152:0/736387018 shutdown_connections 2024-09-11T22:53:20.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.881+0000 7f7a6f4c3640 1 -- 172.21.15.152:0/736387018 wait complete. 2024-09-11T22:53:20.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.882+0000 7f7a6f4c3640 1 Processor -- start 2024-09-11T22:53:20.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.882+0000 7f7a6f4c3640 1 -- start start 2024-09-11T22:53:20.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.882+0000 7f7a6f4c3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a68100a80 0x7f7a68199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.882+0000 7f7a6f4c3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a68101420 0x7f7a681997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.882+0000 7f7a6f4c3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a68106a80 0x7f7a681a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.882+0000 7f7a6f4c3640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7a68110e80 con 0x7f7a68101420 2024-09-11T22:53:20.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.882+0000 7f7a6f4c3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7a68110d00 con 0x7f7a68106a80 2024-09-11T22:53:20.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.882+0000 7f7a6f4c3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7a68111000 con 0x7f7a68100a80 2024-09-11T22:53:20.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.882+0000 7f7a6ca5e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a68101420 0x7f7a681997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.882+0000 7f7a6ca5e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a68101420 0x7f7a681997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36996/0 (socket says 172.21.15.152:36996) 2024-09-11T22:53:20.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.882+0000 7f7a6ca5e640 1 -- 172.21.15.152:0/3092540772 learned_addr learned my addr 172.21.15.152:0/3092540772 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.882+0000 7f7a6da60640 1 --2- 172.21.15.152:0/3092540772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a68106a80 0x7f7a681a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.882+0000 7f7a6d25f640 1 --2- 172.21.15.152:0/3092540772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a68100a80 0x7f7a68199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.883+0000 7f7a6ca5e640 1 -- 172.21.15.152:0/3092540772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a68100a80 msgr2=0x7f7a68199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.883+0000 7f7a6ca5e640 1 --2- 172.21.15.152:0/3092540772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a68100a80 0x7f7a68199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.883+0000 7f7a6ca5e640 1 -- 172.21.15.152:0/3092540772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a68106a80 msgr2=0x7f7a681a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.883+0000 7f7a6ca5e640 1 --2- 172.21.15.152:0/3092540772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a68106a80 0x7f7a681a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.883+0000 7f7a6ca5e640 1 -- 172.21.15.152:0/3092540772 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a681a4850 con 0x7f7a68101420 2024-09-11T22:53:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.883+0000 7f7a6da60640 1 --2- 172.21.15.152:0/3092540772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a68106a80 0x7f7a681a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.883+0000 7f7a6ca5e640 1 --2- 172.21.15.152:0/3092540772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a68101420 0x7f7a681997d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800c970 tx=0x7f7a5800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.883+0000 7f7a5e7fc640 1 -- 172.21.15.152:0/3092540772 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58007bb0 con 0x7f7a68101420 2024-09-11T22:53:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.883+0000 7f7a5e7fc640 1 -- 172.21.15.152:0/3092540772 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a58007d50 con 0x7f7a68101420 2024-09-11T22:53:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.883+0000 7f7a6f4c3640 1 -- 172.21.15.152:0/3092540772 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a681a4a20 con 0x7f7a68101420 2024-09-11T22:53:20.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.883+0000 7f7a5e7fc640 1 -- 172.21.15.152:0/3092540772 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5800f450 con 0x7f7a68101420 2024-09-11T22:53:20.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.883+0000 7f7a6f4c3640 1 -- 172.21.15.152:0/3092540772 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a681a4ee0 con 0x7f7a68101420 2024-09-11T22:53:20.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.885+0000 7f7a5e7fc640 1 -- 172.21.15.152:0/3092540772 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f7a58005020 con 0x7f7a68101420 2024-09-11T22:53:20.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.886+0000 7f7a7189c740 1 Processor -- start 2024-09-11T22:53:20.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.886+0000 7f7a7189c740 1 -- start start 2024-09-11T22:53:20.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.886+0000 7f7a7189c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c19d0 0x55934f4c1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.886+0000 7f7a7189c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c22f0 0x55934f4cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.886+0000 7f7a7189c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 0x55934f4cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.886+0000 7f7a7189c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55934f428090 con 0x55934f4c19d0 2024-09-11T22:53:20.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.886+0000 7f7a5e7fc640 1 --2- 172.21.15.152:0/3092540772 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7a540a2d80 0x7f7a540a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.886+0000 7f7a7189c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55934f3cfa70 con 0x55934f4ccbb0 2024-09-11T22:53:20.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.886+0000 7f7a7189c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55934f424050 con 0x55934f4c22f0 2024-09-11T22:53:20.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.886+0000 7f7a6da60640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c19d0 0x55934f4c1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.886+0000 7f7a6da60640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c19d0 0x55934f4c1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37002/0 (socket says 172.21.15.152:37002) 2024-09-11T22:53:20.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.886+0000 7f7a6da60640 1 -- 172.21.15.152:0/1679739388 learned_addr learned my addr 172.21.15.152:0/1679739388 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.886+0000 7f7a6da60640 1 --2- 172.21.15.152:0/3092540772 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7a540a2d80 0x7f7a540a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.886+0000 7f7a6ca5e640 1 --2- 172.21.15.152:0/1679739388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 0x55934f4cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.886+0000 7f7a5e7fc640 1 -- 172.21.15.152:0/3092540772 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7a580cf430 con 0x7f7a68101420 2024-09-11T22:53:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.886+0000 7f7a6d25f640 1 --2- 172.21.15.152:0/1679739388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c22f0 0x55934f4cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.887+0000 7f7a6da60640 1 -- 172.21.15.152:0/1679739388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c22f0 msgr2=0x55934f4cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.887+0000 7f7a6da60640 1 --2- 172.21.15.152:0/1679739388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c22f0 0x55934f4cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.887+0000 7f7a6da60640 1 -- 172.21.15.152:0/1679739388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 msgr2=0x55934f4cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.887+0000 7f7a6da60640 1 --2- 172.21.15.152:0/1679739388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 0x55934f4cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.887+0000 7f7a6da60640 1 -- 172.21.15.152:0/1679739388 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55934f3dc130 con 0x55934f4c19d0 2024-09-11T22:53:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.887+0000 7f7a6ca5e640 1 --2- 172.21.15.152:0/1679739388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 0x55934f4cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.887+0000 7f7a6da60640 1 --2- 172.21.15.152:0/1679739388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c19d0 0x55934f4c1db0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7a60031d00 tx=0x7f7a60031d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca4a0f565f57cae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.887+0000 7f7a6da60640 1 --2- 172.21.15.152:0/3092540772 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7a540a2d80 0x7f7a540a5240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7a60031da0 tx=0x7f7a60048000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:20.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.887+0000 7f7a537fe640 1 -- 172.21.15.152:0/1679739388 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6004e070 con 0x55934f4c19d0 2024-09-11T22:53:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.887+0000 7f7a537fe640 1 -- 172.21.15.152:0/1679739388 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a60036040 con 0x55934f4c19d0 2024-09-11T22:53:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.887+0000 7f7a537fe640 1 -- 172.21.15.152:0/1679739388 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a600307a0 con 0x55934f4c19d0 2024-09-11T22:53:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.888+0000 7f7a7189c740 1 -- 172.21.15.152:0/1679739388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c19d0 msgr2=0x55934f4c1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.888+0000 7f7a7189c740 1 --2- 172.21.15.152:0/1679739388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c19d0 0x55934f4c1db0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7a60031d00 tx=0x7f7a60031d30 comp rx=0 tx=0).stop 2024-09-11T22:53:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.888+0000 7f7a7189c740 1 -- 172.21.15.152:0/1679739388 shutdown_connections 2024-09-11T22:53:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.888+0000 7f7a7189c740 1 --2- 172.21.15.152:0/1679739388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 0x55934f4cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.888+0000 7f7a7189c740 1 --2- 172.21.15.152:0/1679739388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c22f0 0x55934f4cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.888+0000 7f7a7189c740 1 --2- 172.21.15.152:0/1679739388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c19d0 0x55934f4c1db0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.888+0000 7f7a7189c740 1 -- 172.21.15.152:0/1679739388 >> 172.21.15.152:0/1679739388 conn(0x55934f4bdf60 msgr2=0x55934f4be340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.888+0000 7f7a7189c740 1 -- 172.21.15.152:0/1679739388 shutdown_connections 2024-09-11T22:53:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.888+0000 7f7a7189c740 1 -- 172.21.15.152:0/1679739388 wait complete. 2024-09-11T22:53:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.888+0000 7f7a7189c740 1 -- start start 2024-09-11T22:53:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.888+0000 7f7a7189c740 1 Processor -- start 2024-09-11T22:53:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a7189c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c19d0 0x55934f4cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a7189c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c22f0 0x55934f4c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a7189c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 0x55934f4c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a7189c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55934f4cf780 con 0x55934f4c19d0 2024-09-11T22:53:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a7189c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55934f3e2f20 con 0x55934f4ccbb0 2024-09-11T22:53:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a7189c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55934f4cf900 con 0x55934f4c22f0 2024-09-11T22:53:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a7189c740 10 client.0 _ll_register_callbacks cb 0x55934f3ac030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a6d25f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c19d0 0x55934f4cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a6d25f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c19d0 0x55934f4cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37014/0 (socket says 172.21.15.152:37014) 2024-09-11T22:53:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a6d25f640 1 -- 172.21.15.152:0/2446731413 learned_addr learned my addr 172.21.15.152:0/2446731413 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a6da60640 1 --2- 172.21.15.152:0/2446731413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 0x55934f4c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a6ca5e640 1 --2- 172.21.15.152:0/2446731413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c22f0 0x55934f4c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a6d25f640 1 -- 172.21.15.152:0/2446731413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c22f0 msgr2=0x55934f4c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a6d25f640 1 --2- 172.21.15.152:0/2446731413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c22f0 0x55934f4c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a6d25f640 1 -- 172.21.15.152:0/2446731413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 msgr2=0x55934f4c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:20.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a6d25f640 1 --2- 172.21.15.152:0/2446731413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 0x55934f4c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a6d25f640 1 -- 172.21.15.152:0/2446731413 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55934f1992b0 con 0x55934f4c19d0 2024-09-11T22:53:20.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a6d25f640 1 --2- 172.21.15.152:0/2446731413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c19d0 0x55934f4cc480 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7a64011760 tx=0x7f7a64011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf9accd799a915 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:20.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.889+0000 7f7a50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:20.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.890+0000 7f7a50ff9640 1 -- 172.21.15.152:0/2446731413 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6401e060 con 0x55934f4c19d0 2024-09-11T22:53:20.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.890+0000 7f7a50ff9640 1 -- 172.21.15.152:0/2446731413 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7a6400fd50 con 0x55934f4c19d0 2024-09-11T22:53:20.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.890+0000 7f7a7189c740 1 -- 172.21.15.152:0/2446731413 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a60036040 con 0x55934f4c19d0 2024-09-11T22:53:20.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.890+0000 7f7a50ff9640 1 -- 172.21.15.152:0/2446731413 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a64004020 con 0x55934f4c19d0 2024-09-11T22:53:20.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.890+0000 7f7a7189c740 10 client.15540 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:20.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.890+0000 7f7a7189c740 1 -- 172.21.15.152:0/2446731413 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55934f4db130 con 0x55934f4c19d0 2024-09-11T22:53:20.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.890+0000 7f7a50ff9640 1 -- 172.21.15.152:0/2446731413 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a60036040 con 0x55934f4c19d0 2024-09-11T22:53:20.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.890+0000 7f7a50ff9640 1 -- 172.21.15.152:0/2446731413 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7a64018030 con 0x55934f4c19d0 2024-09-11T22:53:20.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.890+0000 7f7a7189c740 10 client.15540 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.890+0000 7f7a7189c740 10 client.15540 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T22:53:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.890+0000 7f7a7189c740 20 client.15540 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.890+0000 7f7a7189c740 1 --2- 172.21.15.152:0/2446731413 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55934f4dd740 0x55934f4debb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.890+0000 7f7a7189c740 4 client.15540 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.890+0000 7f7a7189c740 1 -- 172.21.15.152:0/2446731413 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55934f435600 con 0x55934f4dd740 2024-09-11T22:53:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.891+0000 7f7a6ca5e640 1 --2- 172.21.15.152:0/2446731413 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55934f4dd740 0x55934f4debb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.891+0000 7f7a6ca5e640 1 --2- 172.21.15.152:0/2446731413 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55934f4dd740 0x55934f4debb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d320dcab586ef521 server_cookie=9d7dea0275275950 in_seq=0 out_seq=0 2024-09-11T22:53:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.891+0000 7f7a50ff9640 10 client.15540 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:53:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.892+0000 7f7a50ff9640 1 -- 172.21.15.152:0/2446731413 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55934f435600 con 0x55934f4dd740 2024-09-11T22:53:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.892+0000 7f7a50ff9640 10 client.15540 handle_command_reply: tid=0 2024-09-11T22:53:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.892+0000 7f7a7189c740 1 client.15540 shutdown 2024-09-11T22:53:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.892+0000 7f7a7189c740 1 -- 172.21.15.152:0/2446731413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c19d0 msgr2=0x55934f4cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.892+0000 7f7a7189c740 1 --2- 172.21.15.152:0/2446731413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c19d0 0x55934f4cc480 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7a64011760 tx=0x7f7a64011c20 comp rx=0 tx=0).stop 2024-09-11T22:53:20.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.892+0000 7f7a7189c740 1 -- 172.21.15.152:0/2446731413 shutdown_connections 2024-09-11T22:53:20.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.892+0000 7f7a7189c740 1 --2- 172.21.15.152:0/2446731413 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55934f4dd740 0x55934f4debb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.892+0000 7f7a7189c740 1 --2- 172.21.15.152:0/2446731413 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 0x55934f4c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.892+0000 7f7a7189c740 1 --2- 172.21.15.152:0/2446731413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c22f0 0x55934f4c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.892+0000 7f7a50ff9640 0 client.15540 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:53:20.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.892+0000 7f7a7189c740 1 --2- 172.21.15.152:0/2446731413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c19d0 0x55934f4cc480 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.892+0000 7f7a7189c740 1 -- 172.21.15.152:0/2446731413 >> 172.21.15.152:0/2446731413 conn(0x55934f4bdf60 msgr2=0x55934f4c3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:20.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.892+0000 7f7a7189c740 1 -- 172.21.15.152:0/2446731413 shutdown_connections 2024-09-11T22:53:20.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.892+0000 7f7a7189c740 1 -- 172.21.15.152:0/2446731413 wait complete. 2024-09-11T22:53:20.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.892+0000 7f7a7189c740 20 client.15540 trim_cache size 0 max 16384 2024-09-11T22:53:20.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.909+0000 7f7a7189c740 1 Processor -- start 2024-09-11T22:53:20.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.909+0000 7f7a7189c740 1 -- start start 2024-09-11T22:53:20.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a7189c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c19d0 0x55934f4c3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a7189c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c22f0 0x55934f4c3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a7189c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 0x55934f4c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a7189c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55934f4dcef0 con 0x55934f4c22f0 2024-09-11T22:53:20.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a7189c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55934f4dd5c0 con 0x55934f4ccbb0 2024-09-11T22:53:20.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a7189c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55934f4ddc90 con 0x55934f4c19d0 2024-09-11T22:53:20.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a6ca5e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c22f0 0x55934f4c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a6ca5e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c22f0 0x55934f4c3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37026/0 (socket says 172.21.15.152:37026) 2024-09-11T22:53:20.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a6ca5e640 1 -- 172.21.15.152:0/2910563307 learned_addr learned my addr 172.21.15.152:0/2910563307 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:20.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a6d25f640 1 --2- 172.21.15.152:0/2910563307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c19d0 0x55934f4c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a6da60640 1 --2- 172.21.15.152:0/2910563307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 0x55934f4c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a6ca5e640 1 -- 172.21.15.152:0/2910563307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c19d0 msgr2=0x55934f4c3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:20.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a6ca5e640 1 --2- 172.21.15.152:0/2910563307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c19d0 0x55934f4c3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a6ca5e640 1 -- 172.21.15.152:0/2910563307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 msgr2=0x55934f4c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:20.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a6ca5e640 1 --2- 172.21.15.152:0/2910563307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 0x55934f4c3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a6ca5e640 1 -- 172.21.15.152:0/2910563307 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55934f4bec30 con 0x55934f4c22f0 2024-09-11T22:53:20.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a6da60640 1 --2- 172.21.15.152:0/2910563307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 0x55934f4c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:20.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a6ca5e640 1 --2- 172.21.15.152:0/2910563307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c22f0 0x55934f4c3960 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7a5809b7c0 tx=0x7f7a5801baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f835925d29cc09 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:20.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.910+0000 7f7a52ffd640 1 -- 172.21.15.152:0/2910563307 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5809b830 con 0x55934f4c22f0 2024-09-11T22:53:20.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.911+0000 7f7a52ffd640 1 -- 172.21.15.152:0/2910563307 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a5809fbe0 con 0x55934f4c22f0 2024-09-11T22:53:20.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.911+0000 7f7a52ffd640 1 -- 172.21.15.152:0/2910563307 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58097ea0 con 0x55934f4c22f0 2024-09-11T22:53:20.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.911+0000 7f7a7189c740 1 -- 172.21.15.152:0/2910563307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c22f0 msgr2=0x55934f4c3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:20.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.911+0000 7f7a7189c740 1 --2- 172.21.15.152:0/2910563307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c22f0 0x55934f4c3960 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7a5809b7c0 tx=0x7f7a5801baf0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.911+0000 7f7a7189c740 1 -- 172.21.15.152:0/2910563307 shutdown_connections 2024-09-11T22:53:20.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.911+0000 7f7a7189c740 1 --2- 172.21.15.152:0/2910563307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4ccbb0 0x55934f4c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.911+0000 7f7a7189c740 1 --2- 172.21.15.152:0/2910563307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4c22f0 0x55934f4c3960 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.911+0000 7f7a7189c740 1 --2- 172.21.15.152:0/2910563307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c19d0 0x55934f4c3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.911+0000 7f7a7189c740 1 -- 172.21.15.152:0/2910563307 >> 172.21.15.152:0/2910563307 conn(0x55934f4bdf60 msgr2=0x55934f4c80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:20.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.911+0000 7f7a7189c740 1 -- 172.21.15.152:0/2910563307 shutdown_connections 2024-09-11T22:53:20.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.911+0000 7f7a7189c740 1 -- 172.21.15.152:0/2910563307 wait complete. 2024-09-11T22:53:20.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.912+0000 7f7a7189c740 1 -- start start 2024-09-11T22:53:20.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.912+0000 7f7a7189c740 1 Processor -- start 2024-09-11T22:53:20.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.912+0000 7f7a7189c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c19d0 0x55934f4e7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.912+0000 7f7a7189c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4c22f0 0x55934f4c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.912+0000 7f7a7189c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4ccbb0 0x55934f4c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.912+0000 7f7a7189c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55934f4dea30 con 0x55934f4ccbb0 2024-09-11T22:53:20.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.912+0000 7f7a7189c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55934f4de360 con 0x55934f4c22f0 2024-09-11T22:53:20.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.912+0000 7f7a7189c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55934f4df100 con 0x55934f4c19d0 2024-09-11T22:53:20.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.912+0000 7f7a7189c740 10 client.0 _ll_register_callbacks cb 0x7f7a6400ced0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:20.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.912+0000 7f7a6da60640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4ccbb0 0x55934f4c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.912+0000 7f7a6d25f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c19d0 0x55934f4e7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.912+0000 7f7a6da60640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4ccbb0 0x55934f4c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37034/0 (socket says 172.21.15.152:37034) 2024-09-11T22:53:20.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.912+0000 7f7a6da60640 1 -- 172.21.15.152:0/1763394339 learned_addr learned my addr 172.21.15.152:0/1763394339 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:20.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.912+0000 7f7a6ca5e640 1 --2- 172.21.15.152:0/1763394339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4c22f0 0x55934f4c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.913+0000 7f7a6da60640 1 -- 172.21.15.152:0/1763394339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c19d0 msgr2=0x55934f4e7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:20.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.913+0000 7f7a6da60640 1 --2- 172.21.15.152:0/1763394339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c19d0 0x55934f4e7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.913+0000 7f7a6da60640 1 -- 172.21.15.152:0/1763394339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4c22f0 msgr2=0x55934f4c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:20.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.913+0000 7f7a6da60640 1 --2- 172.21.15.152:0/1763394339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4c22f0 0x55934f4c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.913+0000 7f7a6da60640 1 -- 172.21.15.152:0/1763394339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55934f4c6f80 con 0x55934f4ccbb0 2024-09-11T22:53:20.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.913+0000 7f7a6da60640 1 --2- 172.21.15.152:0/1763394339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4ccbb0 0x55934f4c6910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7a6002ae00 tx=0x7f7a60009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca24a3bf2e9ee35 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:20.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.913+0000 7f7a50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:20.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.913+0000 7f7a50ff9640 1 -- 172.21.15.152:0/1763394339 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60059070 con 0x55934f4ccbb0 2024-09-11T22:53:20.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.913+0000 7f7a50ff9640 1 -- 172.21.15.152:0/1763394339 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7a600305a0 con 0x55934f4ccbb0 2024-09-11T22:53:20.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.913+0000 7f7a7189c740 1 -- 172.21.15.152:0/1763394339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a5809fbe0 con 0x55934f4ccbb0 2024-09-11T22:53:20.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.913+0000 7f7a50ff9640 1 -- 172.21.15.152:0/1763394339 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a6004e040 con 0x55934f4ccbb0 2024-09-11T22:53:20.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.913+0000 7f7a7189c740 10 client.15546 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:20.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.913+0000 7f7a7189c740 1 -- 172.21.15.152:0/1763394339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55934f4e3d60 con 0x55934f4ccbb0 2024-09-11T22:53:20.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.913+0000 7f7a50ff9640 1 -- 172.21.15.152:0/1763394339 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a5809fbe0 con 0x55934f4ccbb0 2024-09-11T22:53:20.928 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.913+0000 7f7a50ff9640 1 -- 172.21.15.152:0/1763394339 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7a60030070 con 0x55934f4ccbb0 2024-09-11T22:53:20.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.914+0000 7f7a7189c740 10 client.15546 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:20.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.914+0000 7f7a7189c740 10 client.15546 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T22:53:20.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.914+0000 7f7a7189c740 20 client.15546 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:20.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.914+0000 7f7a7189c740 1 --2- 172.21.15.152:0/1763394339 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55934f550740 0x55934f54f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:20.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.914+0000 7f7a7189c740 4 client.15546 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:53:20.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.914+0000 7f7a7189c740 1 -- 172.21.15.152:0/1763394339 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "status"}) -- 0x55934f42fb10 con 0x55934f550740 2024-09-11T22:53:20.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.914+0000 7f7a6d25f640 1 --2- 172.21.15.152:0/1763394339 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55934f550740 0x55934f54f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:20.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.914+0000 7f7a6d25f640 1 --2- 172.21.15.152:0/1763394339 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55934f550740 0x55934f54f450 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfcb73632cde6940 server_cookie=9a41d547472da6f1 in_seq=0 out_seq=0 2024-09-11T22:53:20.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.914+0000 7f7a50ff9640 10 client.15546 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:53:20.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.914+0000 7f7a50ff9640 1 -- 172.21.15.152:0/1763394339 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55934f42fb10 con 0x55934f550740 2024-09-11T22:53:20.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.914+0000 7f7a50ff9640 10 client.15546 handle_command_reply: tid=0 2024-09-11T22:53:20.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.914+0000 7f7a7189c740 1 client.15546 shutdown 2024-09-11T22:53:20.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.915+0000 7f7a7189c740 1 -- 172.21.15.152:0/1763394339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4ccbb0 msgr2=0x55934f4c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:20.929 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.915+0000 7f7a7189c740 1 --2- 172.21.15.152:0/1763394339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4ccbb0 0x55934f4c6910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7a6002ae00 tx=0x7f7a60009c90 comp rx=0 tx=0).stop 2024-09-11T22:53:20.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.915+0000 7f7a7189c740 1 -- 172.21.15.152:0/1763394339 shutdown_connections 2024-09-11T22:53:20.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.915+0000 7f7a7189c740 1 --2- 172.21.15.152:0/1763394339 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55934f550740 0x55934f54f450 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.915+0000 7f7a7189c740 1 --2- 172.21.15.152:0/1763394339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55934f4ccbb0 0x55934f4c6910 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.915+0000 7f7a50ff9640 0 client.15546 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:53:20.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.915+0000 7f7a7189c740 1 --2- 172.21.15.152:0/1763394339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55934f4c22f0 0x55934f4c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.915+0000 7f7a7189c740 1 --2- 172.21.15.152:0/1763394339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55934f4c19d0 0x55934f4e7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.915+0000 7f7a7189c740 1 -- 172.21.15.152:0/1763394339 >> 172.21.15.152:0/1763394339 conn(0x55934f4bdf60 msgr2=0x55934f4c3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:20.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.915+0000 7f7a7189c740 1 -- 172.21.15.152:0/1763394339 shutdown_connections 2024-09-11T22:53:20.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.915+0000 7f7a7189c740 1 -- 172.21.15.152:0/1763394339 wait complete. 2024-09-11T22:53:20.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.915+0000 7f7a7189c740 20 client.15546 trim_cache size 0 max 16384 2024-09-11T22:53:20.930 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:53:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:53:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 2, 2024-09-11T22:53:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 24434, 2024-09-11T22:53:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-09-11T22:53:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-09-11T22:53:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-09-11T22:53:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 137.70356861299999, 2024-09-11T22:53:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 28, 2024-09-11T22:53:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 84, 2024-09-11T22:53:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 83, 2024-09-11T22:53:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 169.83444168700001 2024-09-11T22:53:20.931 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:53:20.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.915+0000 7f7a6f4c3640 1 -- 172.21.15.152:0/3092540772 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7a540a2d80 msgr2=0x7f7a540a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:20.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.915+0000 7f7a6f4c3640 1 --2- 172.21.15.152:0/3092540772 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7a540a2d80 0x7f7a540a5240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7a60031da0 tx=0x7f7a60048000 comp rx=0 tx=0).stop 2024-09-11T22:53:20.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.915+0000 7f7a6f4c3640 1 -- 172.21.15.152:0/3092540772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a68101420 msgr2=0x7f7a681997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:20.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.915+0000 7f7a6f4c3640 1 --2- 172.21.15.152:0/3092540772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a68101420 0x7f7a681997d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800c970 tx=0x7f7a5800ce30 comp rx=0 tx=0).stop 2024-09-11T22:53:20.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.916+0000 7f7a6f4c3640 1 -- 172.21.15.152:0/3092540772 shutdown_connections 2024-09-11T22:53:20.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.916+0000 7f7a6f4c3640 1 --2- 172.21.15.152:0/3092540772 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7a540a2d80 0x7f7a540a5240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.916+0000 7f7a6f4c3640 1 --2- 172.21.15.152:0/3092540772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a68106a80 0x7f7a681a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.916+0000 7f7a6f4c3640 1 --2- 172.21.15.152:0/3092540772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a68101420 0x7f7a681997d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.916+0000 7f7a6f4c3640 1 --2- 172.21.15.152:0/3092540772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a68100a80 0x7f7a68199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:20.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.916+0000 7f7a6f4c3640 1 -- 172.21.15.152:0/3092540772 >> 172.21.15.152:0/3092540772 conn(0x7f7a680fc7a0 msgr2=0x7f7a680fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:20.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.916+0000 7f7a6f4c3640 1 -- 172.21.15.152:0/3092540772 shutdown_connections 2024-09-11T22:53:20.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:20.916+0000 7f7a6f4c3640 1 -- 172.21.15.152:0/3092540772 wait complete. 2024-09-11T22:53:21.702 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T22:53:21.948 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:22.370 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:22 smithi152 ceph-mon[31181]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.3 KiB/s wr, 16 op/s 2024-09-11T22:53:22.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.453+0000 7f8300a80640 1 -- 172.21.15.152:0/3861125976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82fc1082a0 msgr2=0x7f82fc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:22.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.453+0000 7f8300a80640 1 --2- 172.21.15.152:0/3861125976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82fc1082a0 0x7f82fc10ec40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009cd0 tx=0x7f82f402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.453+0000 7f8300a80640 1 -- 172.21.15.152:0/3861125976 shutdown_connections 2024-09-11T22:53:22.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.453+0000 7f8300a80640 1 --2- 172.21.15.152:0/3861125976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82fc1082a0 0x7f82fc10ec40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.453+0000 7f8300a80640 1 --2- 172.21.15.152:0/3861125976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82fc1036e0 0x7f82fc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.453+0000 7f8300a80640 1 --2- 172.21.15.152:0/3861125976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82fc102d30 0x7f82fc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.453+0000 7f8300a80640 1 -- 172.21.15.152:0/3861125976 >> 172.21.15.152:0/3861125976 conn(0x7f82fc0fc7d0 msgr2=0x7f82fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:22.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.453+0000 7f8300a80640 1 -- 172.21.15.152:0/3861125976 shutdown_connections 2024-09-11T22:53:22.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.453+0000 7f8300a80640 1 -- 172.21.15.152:0/3861125976 wait complete. 2024-09-11T22:53:22.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.454+0000 7f8300a80640 1 Processor -- start 2024-09-11T22:53:22.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.454+0000 7f8300a80640 1 -- start start 2024-09-11T22:53:22.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.454+0000 7f8300a80640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82fc102d30 0x7f82fc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.454+0000 7f8300a80640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82fc1036e0 0x7f82fc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.454+0000 7f8300a80640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82fc1082a0 0x7f82fc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.454+0000 7f8300a80640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc1120a0 con 0x7f82fc1082a0 2024-09-11T22:53:22.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.454+0000 7f8300a80640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc111f20 con 0x7f82fc102d30 2024-09-11T22:53:22.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.454+0000 7f8300a80640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc112220 con 0x7f82fc1036e0 2024-09-11T22:53:22.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.454+0000 7f82fad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82fc1082a0 0x7f82fc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.454+0000 7f82fad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82fc1082a0 0x7f82fc101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37050/0 (socket says 172.21.15.152:37050) 2024-09-11T22:53:22.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.454+0000 7f82fad9d640 1 -- 172.21.15.152:0/1283036012 learned_addr learned my addr 172.21.15.152:0/1283036012 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:22.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.455+0000 7f82fa59c640 1 --2- 172.21.15.152:0/1283036012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82fc102d30 0x7f82fc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.455+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/1283036012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82fc1036e0 0x7f82fc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.455+0000 7f82fad9d640 1 -- 172.21.15.152:0/1283036012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82fc1036e0 msgr2=0x7f82fc100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:22.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.455+0000 7f82fad9d640 1 --2- 172.21.15.152:0/1283036012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82fc1036e0 0x7f82fc100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.455+0000 7f82fad9d640 1 -- 172.21.15.152:0/1283036012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82fc102d30 msgr2=0x7f82fc100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:22.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.455+0000 7f82fad9d640 1 --2- 172.21.15.152:0/1283036012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82fc102d30 0x7f82fc100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.455+0000 7f82fad9d640 1 -- 172.21.15.152:0/1283036012 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82fc1a6a20 con 0x7f82fc1082a0 2024-09-11T22:53:22.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.455+0000 7f82fad9d640 1 --2- 172.21.15.152:0/1283036012 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82fc1082a0 0x7f82fc101410 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f82f4005cc0 tx=0x7f82f40093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:22.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.455+0000 7f82f37fe640 1 -- 172.21.15.152:0/1283036012 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4003e00 con 0x7f82fc1082a0 2024-09-11T22:53:22.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.455+0000 7f8300a80640 1 -- 172.21.15.152:0/1283036012 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82fc1a6cb0 con 0x7f82fc1082a0 2024-09-11T22:53:22.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.455+0000 7f82f37fe640 1 -- 172.21.15.152:0/1283036012 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82f4004520 con 0x7f82fc1082a0 2024-09-11T22:53:22.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.455+0000 7f82f37fe640 1 -- 172.21.15.152:0/1283036012 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4030db0 con 0x7f82fc1082a0 2024-09-11T22:53:22.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.455+0000 7f8300a80640 1 -- 172.21.15.152:0/1283036012 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82fc1a71f0 con 0x7f82fc1082a0 2024-09-11T22:53:22.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.458+0000 7f8302e59740 1 Processor -- start 2024-09-11T22:53:22.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.458+0000 7f82f37fe640 1 -- 172.21.15.152:0/1283036012 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f82f4004040 con 0x7f82fc1082a0 2024-09-11T22:53:22.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f8302e59740 1 -- start start 2024-09-11T22:53:22.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f8302e59740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360c9d0 0x55dc8360cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f8302e59740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc8360d2f0 0x55dc83617670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f8302e59740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc83617bb0 0x55dc83619fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f8302e59740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55dc83573090 con 0x55dc8360c9d0 2024-09-11T22:53:22.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f8302e59740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55dc8351aa70 con 0x55dc8360d2f0 2024-09-11T22:53:22.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f8302e59740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dc8356f050 con 0x55dc83617bb0 2024-09-11T22:53:22.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f82f9d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360c9d0 0x55dc8360cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f82f9d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360c9d0 0x55dc8360cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37052/0 (socket says 172.21.15.152:37052) 2024-09-11T22:53:22.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f82f9d9b640 1 -- 172.21.15.152:0/3267008641 learned_addr learned my addr 172.21.15.152:0/3267008641 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:22.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/3267008641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc83617bb0 0x55dc83619fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f82fa59c640 1 --2- 172.21.15.152:0/3267008641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc8360d2f0 0x55dc83617670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f82f9d9b640 1 -- 172.21.15.152:0/3267008641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc83617bb0 msgr2=0x55dc83619fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:22.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/3267008641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc83617bb0 0x55dc83619fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f82f9d9b640 1 -- 172.21.15.152:0/3267008641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc8360d2f0 msgr2=0x55dc83617670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:22.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/3267008641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc8360d2f0 0x55dc83617670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f82f37fe640 1 --2- 172.21.15.152:0/1283036012 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f82e80a2d80 0x7f82e80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.459+0000 7f82f9d9b640 1 -- 172.21.15.152:0/3267008641 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc83527130 con 0x55dc8360c9d0 2024-09-11T22:53:22.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.460+0000 7f82fad9d640 1 --2- 172.21.15.152:0/1283036012 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f82e80a2d80 0x7f82e80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.460+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/3267008641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360c9d0 0x55dc8360cdb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009fb0 tx=0x7f82e40096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde4ffa31305a17 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:22.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.460+0000 7f82f37fe640 1 -- 172.21.15.152:0/1283036012 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f82f402e030 con 0x7f82fc1082a0 2024-09-11T22:53:22.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.460+0000 7f82f0ff9640 1 -- 172.21.15.152:0/3267008641 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4005020 con 0x55dc8360c9d0 2024-09-11T22:53:22.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.460+0000 7f82f0ff9640 1 -- 172.21.15.152:0/3267008641 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82e40047a0 con 0x55dc8360c9d0 2024-09-11T22:53:22.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.460+0000 7f82f0ff9640 1 -- 172.21.15.152:0/3267008641 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e400f6e0 con 0x55dc8360c9d0 2024-09-11T22:53:22.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.461+0000 7f8302e59740 1 -- 172.21.15.152:0/3267008641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360c9d0 msgr2=0x55dc8360cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:22.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.461+0000 7f8302e59740 1 --2- 172.21.15.152:0/3267008641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360c9d0 0x55dc8360cdb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009fb0 tx=0x7f82e40096a0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.461+0000 7f82fad9d640 1 --2- 172.21.15.152:0/1283036012 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f82e80a2d80 0x7f82e80a5240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f82f40ee450 tx=0x7f82f408f0d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:22.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.461+0000 7f8302e59740 1 -- 172.21.15.152:0/3267008641 shutdown_connections 2024-09-11T22:53:22.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.461+0000 7f8302e59740 1 --2- 172.21.15.152:0/3267008641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc83617bb0 0x55dc83619fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.461+0000 7f8302e59740 1 --2- 172.21.15.152:0/3267008641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc8360d2f0 0x55dc83617670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.461+0000 7f8302e59740 1 --2- 172.21.15.152:0/3267008641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360c9d0 0x55dc8360cdb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.461+0000 7f8302e59740 1 -- 172.21.15.152:0/3267008641 >> 172.21.15.152:0/3267008641 conn(0x55dc83608f60 msgr2=0x55dc83609340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:22.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.461+0000 7f8302e59740 1 -- 172.21.15.152:0/3267008641 shutdown_connections 2024-09-11T22:53:22.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.461+0000 7f8302e59740 1 -- 172.21.15.152:0/3267008641 wait complete. 2024-09-11T22:53:22.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.461+0000 7f8302e59740 1 -- start start 2024-09-11T22:53:22.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.461+0000 7f8302e59740 1 Processor -- start 2024-09-11T22:53:22.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.461+0000 7f8302e59740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc8360c9d0 0x55dc83617310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.461+0000 7f8302e59740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360d2f0 0x55dc83614660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.461+0000 7f8302e59740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc83617bb0 0x55dc83614ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.461+0000 7f8302e59740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55dc8361a780 con 0x55dc8360d2f0 2024-09-11T22:53:22.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f8302e59740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55dc8352df20 con 0x55dc8360c9d0 2024-09-11T22:53:22.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f8302e59740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dc8361a900 con 0x55dc83617bb0 2024-09-11T22:53:22.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f8302e59740 10 client.0 _ll_register_callbacks cb 0x55dc834f7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:22.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f82fa59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360d2f0 0x55dc83614660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f82fa59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360d2f0 0x55dc83614660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37064/0 (socket says 172.21.15.152:37064) 2024-09-11T22:53:22.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f82fa59c640 1 -- 172.21.15.152:0/3285535478 learned_addr learned my addr 172.21.15.152:0/3285535478 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:22.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/3285535478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc8360c9d0 0x55dc83617310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/3285535478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc83617bb0 0x55dc83614ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f82fa59c640 1 -- 172.21.15.152:0/3285535478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc83617bb0 msgr2=0x55dc83614ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:22.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f82fa59c640 1 --2- 172.21.15.152:0/3285535478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc83617bb0 0x55dc83614ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f82fa59c640 1 -- 172.21.15.152:0/3285535478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc8360c9d0 msgr2=0x55dc83617310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:22.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f82fa59c640 1 --2- 172.21.15.152:0/3285535478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc8360c9d0 0x55dc83617310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f82fa59c640 1 -- 172.21.15.152:0/3285535478 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc832e42b0 con 0x55dc8360d2f0 2024-09-11T22:53:22.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/3285535478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc8360c9d0 0x55dc83617310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:22.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f82fa59c640 1 --2- 172.21.15.152:0/3285535478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360d2f0 0x55dc83614660 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f82ec013710 tx=0x7f82ec013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ee3f0af5249a38 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:22.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f82e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:22.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f82e1ffb640 1 -- 172.21.15.152:0/3285535478 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec020060 con 0x55dc8360d2f0 2024-09-11T22:53:22.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f82e1ffb640 1 -- 172.21.15.152:0/3285535478 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f82ec011dd0 con 0x55dc8360d2f0 2024-09-11T22:53:22.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f8302e59740 1 -- 172.21.15.152:0/3285535478 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82e40047a0 con 0x55dc8360d2f0 2024-09-11T22:53:22.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.462+0000 7f82e1ffb640 1 -- 172.21.15.152:0/3285535478 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f82ec025460 con 0x55dc8360d2f0 2024-09-11T22:53:22.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.463+0000 7f8302e59740 10 client.15558 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:22.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.463+0000 7f8302e59740 1 -- 172.21.15.152:0/3285535478 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc83626000 con 0x55dc8360d2f0 2024-09-11T22:53:22.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.463+0000 7f82e1ffb640 1 -- 172.21.15.152:0/3285535478 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82e40047a0 con 0x55dc8360d2f0 2024-09-11T22:53:22.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.463+0000 7f82e1ffb640 1 -- 172.21.15.152:0/3285535478 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f82ec01a030 con 0x55dc8360d2f0 2024-09-11T22:53:22.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.463+0000 7f8302e59740 10 client.15558 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:22.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.463+0000 7f8302e59740 10 client.15558 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T22:53:22.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.463+0000 7f8302e59740 20 client.15558 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:22.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.463+0000 7f8302e59740 1 --2- 172.21.15.152:0/3285535478 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55dc83628650 0x55dc83629ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.463+0000 7f8302e59740 4 client.15558 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:22.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.463+0000 7f8302e59740 1 -- 172.21.15.152:0/3285535478 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc83580600 con 0x55dc83628650 2024-09-11T22:53:22.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.463+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/3285535478 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55dc83628650 0x55dc83629ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.464+0000 7f82e1ffb640 10 client.15558 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:53:22.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.464+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/3285535478 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55dc83628650 0x55dc83629ac0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52375d2d18b050f0 server_cookie=5b818a83bd54fa9e in_seq=0 out_seq=0 2024-09-11T22:53:22.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.464+0000 7f82e1ffb640 1 -- 172.21.15.152:0/3285535478 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc83580600 con 0x55dc83628650 2024-09-11T22:53:22.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.464+0000 7f82e1ffb640 10 client.15558 handle_command_reply: tid=0 2024-09-11T22:53:22.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.465+0000 7f8302e59740 1 client.15558 shutdown 2024-09-11T22:53:22.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.465+0000 7f8302e59740 1 -- 172.21.15.152:0/3285535478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360d2f0 msgr2=0x55dc83614660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:22.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.465+0000 7f8302e59740 1 --2- 172.21.15.152:0/3285535478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360d2f0 0x55dc83614660 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f82ec013710 tx=0x7f82ec013bd0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.465+0000 7f8302e59740 1 -- 172.21.15.152:0/3285535478 shutdown_connections 2024-09-11T22:53:22.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.465+0000 7f8302e59740 1 --2- 172.21.15.152:0/3285535478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc83617bb0 0x55dc83614ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.465+0000 7f8302e59740 1 --2- 172.21.15.152:0/3285535478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360d2f0 0x55dc83614660 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.465+0000 7f8302e59740 1 --2- 172.21.15.152:0/3285535478 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55dc83628650 0x55dc83629ac0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.465+0000 7f8302e59740 1 --2- 172.21.15.152:0/3285535478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc8360c9d0 0x55dc83617310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.465+0000 7f8302e59740 1 -- 172.21.15.152:0/3285535478 >> 172.21.15.152:0/3285535478 conn(0x55dc83608f60 msgr2=0x55dc8360ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:22.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.465+0000 7f82e1ffb640 0 client.15558 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:53:22.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.465+0000 7f8302e59740 1 -- 172.21.15.152:0/3285535478 shutdown_connections 2024-09-11T22:53:22.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.465+0000 7f8302e59740 1 -- 172.21.15.152:0/3285535478 wait complete. 2024-09-11T22:53:22.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.465+0000 7f8302e59740 20 client.15558 trim_cache size 0 max 16384 2024-09-11T22:53:22.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.483+0000 7f8302e59740 1 Processor -- start 2024-09-11T22:53:22.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.483+0000 7f8302e59740 1 -- start start 2024-09-11T22:53:22.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.483+0000 7f8302e59740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8360c9d0 0x55dc83629ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.483+0000 7f8302e59740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360d2f0 0x55dc836118c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.483+0000 7f8302e59740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc83617bb0 0x55dc83611e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.483+0000 7f8302e59740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55dc83627e00 con 0x55dc8360d2f0 2024-09-11T22:53:22.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.483+0000 7f8302e59740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55dc836284d0 con 0x55dc83617bb0 2024-09-11T22:53:22.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.483+0000 7f8302e59740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dc83628ba0 con 0x55dc8360c9d0 2024-09-11T22:53:22.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.483+0000 7f82fa59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360d2f0 0x55dc836118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.483+0000 7f82fa59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360d2f0 0x55dc836118c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37068/0 (socket says 172.21.15.152:37068) 2024-09-11T22:53:22.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.483+0000 7f82fa59c640 1 -- 172.21.15.152:0/221883747 learned_addr learned my addr 172.21.15.152:0/221883747 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.483+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/221883747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8360c9d0 0x55dc83629ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.483+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/221883747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc83617bb0 0x55dc83611e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.484+0000 7f82fa59c640 1 -- 172.21.15.152:0/221883747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8360c9d0 msgr2=0x55dc83629ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.484+0000 7f82fa59c640 1 --2- 172.21.15.152:0/221883747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8360c9d0 0x55dc83629ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.484+0000 7f82fa59c640 1 -- 172.21.15.152:0/221883747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc83617bb0 msgr2=0x55dc83611e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.484+0000 7f82fa59c640 1 --2- 172.21.15.152:0/221883747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc83617bb0 0x55dc83611e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.484+0000 7f82fa59c640 1 -- 172.21.15.152:0/221883747 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc8360d120 con 0x55dc8360d2f0 2024-09-11T22:53:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.484+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/221883747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8360c9d0 0x55dc83629ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.484+0000 7f82fa59c640 1 --2- 172.21.15.152:0/221883747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360d2f0 0x55dc836118c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009d00 tx=0x7f82ec009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00db464674c6ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.484+0000 7f82e3fff640 1 -- 172.21.15.152:0/221883747 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec00f450 con 0x55dc8360d2f0 2024-09-11T22:53:22.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.484+0000 7f82e3fff640 1 -- 172.21.15.152:0/221883747 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82ec0055f0 con 0x55dc8360d2f0 2024-09-11T22:53:22.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.484+0000 7f82e3fff640 1 -- 172.21.15.152:0/221883747 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec025820 con 0x55dc8360d2f0 2024-09-11T22:53:22.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.485+0000 7f8302e59740 1 -- 172.21.15.152:0/221883747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360d2f0 msgr2=0x55dc836118c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:22.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.485+0000 7f8302e59740 1 --2- 172.21.15.152:0/221883747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360d2f0 0x55dc836118c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009d00 tx=0x7f82ec009e20 comp rx=0 tx=0).stop 2024-09-11T22:53:22.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.485+0000 7f8302e59740 1 -- 172.21.15.152:0/221883747 shutdown_connections 2024-09-11T22:53:22.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.485+0000 7f8302e59740 1 --2- 172.21.15.152:0/221883747 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc83617bb0 0x55dc83611e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.485+0000 7f8302e59740 1 --2- 172.21.15.152:0/221883747 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360d2f0 0x55dc836118c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.485+0000 7f8302e59740 1 --2- 172.21.15.152:0/221883747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8360c9d0 0x55dc83629ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.485+0000 7f8302e59740 1 -- 172.21.15.152:0/221883747 >> 172.21.15.152:0/221883747 conn(0x55dc83608f60 msgr2=0x55dc83609340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:22.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.485+0000 7f8302e59740 1 -- 172.21.15.152:0/221883747 shutdown_connections 2024-09-11T22:53:22.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.485+0000 7f8302e59740 1 -- 172.21.15.152:0/221883747 wait complete. 2024-09-11T22:53:22.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.485+0000 7f8302e59740 1 -- start start 2024-09-11T22:53:22.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.485+0000 7f8302e59740 1 Processor -- start 2024-09-11T22:53:22.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f8302e59740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360c9d0 0x55dc83632190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f8302e59740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8360d2f0 0x55dc8362b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f8302e59740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc83617bb0 0x55dc8362ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f8302e59740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55dc83629940 con 0x55dc8360c9d0 2024-09-11T22:53:22.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f8302e59740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55dc83629270 con 0x55dc83617bb0 2024-09-11T22:53:22.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f8302e59740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55dc8362a010 con 0x55dc8360d2f0 2024-09-11T22:53:22.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f8302e59740 10 client.0 _ll_register_callbacks cb 0x7f82ec0096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:22.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f82f9d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360c9d0 0x55dc83632190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f82f9d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360c9d0 0x55dc83632190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37082/0 (socket says 172.21.15.152:37082) 2024-09-11T22:53:22.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f82f9d9b640 1 -- 172.21.15.152:0/1354129305 learned_addr learned my addr 172.21.15.152:0/1354129305 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:22.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f82fa59c640 1 --2- 172.21.15.152:0/1354129305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8360d2f0 0x55dc8362b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/1354129305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc83617bb0 0x55dc8362ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f82f9d9b640 1 -- 172.21.15.152:0/1354129305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8360d2f0 msgr2=0x55dc8362b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:22.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/1354129305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8360d2f0 0x55dc8362b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f82f9d9b640 1 -- 172.21.15.152:0/1354129305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc83617bb0 msgr2=0x55dc8362ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:22.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/1354129305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc83617bb0 0x55dc8362ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f82f9d9b640 1 -- 172.21.15.152:0/1354129305 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8362bf40 con 0x55dc8360c9d0 2024-09-11T22:53:22.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f82f9d9b640 1 --2- 172.21.15.152:0/1354129305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360c9d0 0x55dc83632190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009750 tx=0x7f82e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b55636f18a5f00 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:22.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f82e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:22.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f82e1ffb640 1 -- 172.21.15.152:0/1354129305 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4004fd0 con 0x55dc8360c9d0 2024-09-11T22:53:22.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f82e1ffb640 1 -- 172.21.15.152:0/1354129305 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f82e4005170 con 0x55dc8360c9d0 2024-09-11T22:53:22.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.486+0000 7f8302e59740 1 -- 172.21.15.152:0/1354129305 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82ec0055f0 con 0x55dc8360c9d0 2024-09-11T22:53:22.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.487+0000 7f82e1ffb640 1 -- 172.21.15.152:0/1354129305 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f82e4005370 con 0x55dc8360c9d0 2024-09-11T22:53:22.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.487+0000 7f8302e59740 10 client.15564 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:22.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.487+0000 7f8302e59740 1 -- 172.21.15.152:0/1354129305 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc83630930 con 0x55dc8360c9d0 2024-09-11T22:53:22.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.487+0000 7f82e1ffb640 1 -- 172.21.15.152:0/1354129305 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82ec0055f0 con 0x55dc8360c9d0 2024-09-11T22:53:22.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.487+0000 7f82e1ffb640 1 -- 172.21.15.152:0/1354129305 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f82e40035f0 con 0x55dc8360c9d0 2024-09-11T22:53:22.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.487+0000 7f8302e59740 10 client.15564 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:22.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.487+0000 7f8302e59740 10 client.15564 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T22:53:22.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.487+0000 7f8302e59740 20 client.15564 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:22.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.487+0000 7f8302e59740 1 --2- 172.21.15.152:0/1354129305 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55dc836973d0 0x55dc836960e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:22.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.487+0000 7f8302e59740 4 client.15564 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:53:22.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.487+0000 7f8302e59740 1 -- 172.21.15.152:0/1354129305 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "status"}) -- 0x55dc8357ab10 con 0x55dc836973d0 2024-09-11T22:53:22.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.487+0000 7f82fa59c640 1 --2- 172.21.15.152:0/1354129305 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55dc836973d0 0x55dc836960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:22.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f82e1ffb640 10 client.15564 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:53:22.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f82fa59c640 1 --2- 172.21.15.152:0/1354129305 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55dc836973d0 0x55dc836960e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e58c31bb1812178 server_cookie=1e347e6997dd2947 in_seq=0 out_seq=0 2024-09-11T22:53:22.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f82e1ffb640 1 -- 172.21.15.152:0/1354129305 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55dc8357ab10 con 0x55dc836973d0 2024-09-11T22:53:22.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f82e1ffb640 10 client.15564 handle_command_reply: tid=0 2024-09-11T22:53:22.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f8302e59740 1 client.15564 shutdown 2024-09-11T22:53:22.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f8302e59740 1 -- 172.21.15.152:0/1354129305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360c9d0 msgr2=0x55dc83632190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:22.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f8302e59740 1 --2- 172.21.15.152:0/1354129305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360c9d0 0x55dc83632190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009750 tx=0x7f82e4009780 comp rx=0 tx=0).stop 2024-09-11T22:53:22.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f8302e59740 1 -- 172.21.15.152:0/1354129305 shutdown_connections 2024-09-11T22:53:22.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f8302e59740 1 --2- 172.21.15.152:0/1354129305 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55dc836973d0 0x55dc836960e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f8302e59740 1 --2- 172.21.15.152:0/1354129305 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55dc83617bb0 0x55dc8362ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f8302e59740 1 --2- 172.21.15.152:0/1354129305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc8360d2f0 0x55dc8362b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f8302e59740 1 --2- 172.21.15.152:0/1354129305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc8360c9d0 0x55dc83632190 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f8302e59740 1 -- 172.21.15.152:0/1354129305 >> 172.21.15.152:0/1354129305 conn(0x55dc83608f60 msgr2=0x55dc83615b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:22.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f82e1ffb640 0 client.15564 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:53:22.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f8302e59740 1 -- 172.21.15.152:0/1354129305 shutdown_connections 2024-09-11T22:53:22.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f8302e59740 1 -- 172.21.15.152:0/1354129305 wait complete. 2024-09-11T22:53:22.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.488+0000 7f8302e59740 20 client.15564 trim_cache size 0 max 16384 2024-09-11T22:53:22.498 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:53:22.498 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:53:22.498 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 1, 2024-09-11T22:53:22.498 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 24442, 2024-09-11T22:53:22.498 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-09-11T22:53:22.499 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-09-11T22:53:22.499 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-09-11T22:53:22.499 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 140.28887416699999, 2024-09-11T22:53:22.499 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 28, 2024-09-11T22:53:22.499 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 84, 2024-09-11T22:53:22.499 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 83, 2024-09-11T22:53:22.499 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 168.844633141 2024-09-11T22:53:22.499 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:53:22.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.489+0000 7f8300a80640 1 -- 172.21.15.152:0/1283036012 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f82e80a2d80 msgr2=0x7f82e80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:22.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.489+0000 7f8300a80640 1 --2- 172.21.15.152:0/1283036012 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f82e80a2d80 0x7f82e80a5240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f82f40ee450 tx=0x7f82f408f0d0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.489+0000 7f8300a80640 1 -- 172.21.15.152:0/1283036012 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82fc1082a0 msgr2=0x7f82fc101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:22.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.489+0000 7f8300a80640 1 --2- 172.21.15.152:0/1283036012 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82fc1082a0 0x7f82fc101410 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f82f4005cc0 tx=0x7f82f40093d0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.489+0000 7f8300a80640 1 -- 172.21.15.152:0/1283036012 shutdown_connections 2024-09-11T22:53:22.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.489+0000 7f8300a80640 1 --2- 172.21.15.152:0/1283036012 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f82e80a2d80 0x7f82e80a5240 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.489+0000 7f8300a80640 1 --2- 172.21.15.152:0/1283036012 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82fc1082a0 0x7f82fc101410 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.489+0000 7f8300a80640 1 --2- 172.21.15.152:0/1283036012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82fc1036e0 0x7f82fc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.489+0000 7f8300a80640 1 --2- 172.21.15.152:0/1283036012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82fc102d30 0x7f82fc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:22.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.489+0000 7f8300a80640 1 -- 172.21.15.152:0/1283036012 >> 172.21.15.152:0/1283036012 conn(0x7f82fc0fc7d0 msgr2=0x7f82fc105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:22.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.489+0000 7f8300a80640 1 -- 172.21.15.152:0/1283036012 shutdown_connections 2024-09-11T22:53:22.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:22.489+0000 7f8300a80640 1 -- 172.21.15.152:0/1283036012 wait complete. 2024-09-11T22:53:22.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:22 smithi191 ceph-mon[35211]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.3 KiB/s wr, 16 op/s 2024-09-11T22:53:22.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:22 smithi204 ceph-mon[35454]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.3 KiB/s wr, 16 op/s 2024-09-11T22:53:23.282 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T22:53:23.539 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:24.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.036+0000 7f90b5738640 1 -- 172.21.15.152:0/2268954568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90b0102570 msgr2=0x7f90b01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:24.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.036+0000 7f90b5738640 1 --2- 172.21.15.152:0/2268954568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90b0102570 0x7f90b01029b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009900 tx=0x7f90a402a480 comp rx=0 tx=0).stop 2024-09-11T22:53:24.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.036+0000 7f90b5738640 1 -- 172.21.15.152:0/2268954568 shutdown_connections 2024-09-11T22:53:24.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.036+0000 7f90b5738640 1 --2- 172.21.15.152:0/2268954568 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90b0102ef0 0x7f90b010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.036+0000 7f90b5738640 1 --2- 172.21.15.152:0/2268954568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90b0102570 0x7f90b01029b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.036+0000 7f90b5738640 1 --2- 172.21.15.152:0/2268954568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90b0100e20 0x7f90b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.036+0000 7f90b5738640 1 -- 172.21.15.152:0/2268954568 >> 172.21.15.152:0/2268954568 conn(0x7f90b00fc7d0 msgr2=0x7f90b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:24.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.036+0000 7f90b5738640 1 -- 172.21.15.152:0/2268954568 shutdown_connections 2024-09-11T22:53:24.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.036+0000 7f90b5738640 1 -- 172.21.15.152:0/2268954568 wait complete. 2024-09-11T22:53:24.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.036+0000 7f90b5738640 1 Processor -- start 2024-09-11T22:53:24.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90b5738640 1 -- start start 2024-09-11T22:53:24.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90b5738640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90b0100e20 0x7f90b01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90b5738640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90b0102570 0x7f90b0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90b5738640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90b0102ef0 0x7f90b019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90b5738640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f90b0110f80 con 0x7f90b0102ef0 2024-09-11T22:53:24.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90b5738640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f90b0110e00 con 0x7f90b0102570 2024-09-11T22:53:24.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90b5738640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f90b0111100 con 0x7f90b0100e20 2024-09-11T22:53:24.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90af7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90b0102ef0 0x7f90b019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90af7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90b0102ef0 0x7f90b019cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37104/0 (socket says 172.21.15.152:37104) 2024-09-11T22:53:24.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90af7fe640 1 -- 172.21.15.152:0/2384340736 learned_addr learned my addr 172.21.15.152:0/2384340736 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:24.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90aeffd640 1 --2- 172.21.15.152:0/2384340736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90b0100e20 0x7f90b01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90ae7fc640 1 --2- 172.21.15.152:0/2384340736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90b0102570 0x7f90b0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90af7fe640 1 -- 172.21.15.152:0/2384340736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90b0100e20 msgr2=0x7f90b01984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:24.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90af7fe640 1 --2- 172.21.15.152:0/2384340736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90b0100e20 0x7f90b01984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90af7fe640 1 -- 172.21.15.152:0/2384340736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90b0102570 msgr2=0x7f90b0198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:24.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90af7fe640 1 --2- 172.21.15.152:0/2384340736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90b0102570 0x7f90b0198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90af7fe640 1 -- 172.21.15.152:0/2384340736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b019d4a0 con 0x7f90b0102ef0 2024-09-11T22:53:24.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.037+0000 7f90ae7fc640 1 --2- 172.21.15.152:0/2384340736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90b0102570 0x7f90b0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:24.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.038+0000 7f90af7fe640 1 --2- 172.21.15.152:0/2384340736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90b0102ef0 0x7f90b019cdc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f90a800cce0 tx=0x7f90a800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:24.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.038+0000 7f9097fff640 1 -- 172.21.15.152:0/2384340736 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a800f450 con 0x7f90b0102ef0 2024-09-11T22:53:24.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.038+0000 7f90b5738640 1 -- 172.21.15.152:0/2384340736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b019d790 con 0x7f90b0102ef0 2024-09-11T22:53:24.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.038+0000 7f9097fff640 1 -- 172.21.15.152:0/2384340736 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a800fad0 con 0x7f90b0102ef0 2024-09-11T22:53:24.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.038+0000 7f9097fff640 1 -- 172.21.15.152:0/2384340736 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8018c10 con 0x7f90b0102ef0 2024-09-11T22:53:24.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.038+0000 7f90b5738640 1 -- 172.21.15.152:0/2384340736 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b01a5070 con 0x7f90b0102ef0 2024-09-11T22:53:24.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.040+0000 7f9097fff640 1 -- 172.21.15.152:0/2384340736 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f90a800f5f0 con 0x7f90b0102ef0 2024-09-11T22:53:24.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.040+0000 7f90b7b11740 1 Processor -- start 2024-09-11T22:53:24.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.040+0000 7f90b7b11740 1 -- start start 2024-09-11T22:53:24.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.040+0000 7f90b7b11740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad819d0 0x55a72ad81db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.040+0000 7f90b7b11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad822f0 0x55a72ad8c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.040+0000 7f90b7b11740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad8cbb0 0x55a72ad8efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.040+0000 7f90b7b11740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a72ace8090 con 0x55a72ad8cbb0 2024-09-11T22:53:24.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.040+0000 7f90b7b11740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a72ac8fa70 con 0x55a72ad819d0 2024-09-11T22:53:24.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.040+0000 7f90b7b11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a72ace4050 con 0x55a72ad822f0 2024-09-11T22:53:24.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f90ae7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad8cbb0 0x55a72ad8efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f90ae7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad8cbb0 0x55a72ad8efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37118/0 (socket says 172.21.15.152:37118) 2024-09-11T22:53:24.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f90ae7fc640 1 -- 172.21.15.152:0/1706657551 learned_addr learned my addr 172.21.15.152:0/1706657551 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:24.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f90aeffd640 1 --2- 172.21.15.152:0/1706657551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad822f0 0x55a72ad8c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f90ae7fc640 1 --2- 172.21.15.152:0/1706657551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad819d0 0x55a72ad81db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f90aeffd640 1 -- 172.21.15.152:0/1706657551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad819d0 msgr2=0x55a72ad81db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:24.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f90aeffd640 1 --2- 172.21.15.152:0/1706657551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad819d0 0x55a72ad81db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f90aeffd640 1 -- 172.21.15.152:0/1706657551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad8cbb0 msgr2=0x55a72ad8efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:24.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f90aeffd640 1 --2- 172.21.15.152:0/1706657551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad8cbb0 0x55a72ad8efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f90aeffd640 1 -- 172.21.15.152:0/1706657551 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a72ac9c130 con 0x55a72ad822f0 2024-09-11T22:53:24.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f9097fff640 1 --2- 172.21.15.152:0/2384340736 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f90980a2d80 0x7f90980a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f90ae7fc640 1 --2- 172.21.15.152:0/1706657551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad8cbb0 0x55a72ad8efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:24.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f90aeffd640 1 --2- 172.21.15.152:0/1706657551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad822f0 0x55a72ad8c670 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f90a0004630 tx=0x7f90a000bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9afb3adf5c1c96 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:24.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f90ae7fc640 1 --2- 172.21.15.152:0/2384340736 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f90980a2d80 0x7f90980a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f9097fff640 1 -- 172.21.15.152:0/2384340736 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f90a80cfbd0 con 0x7f90b0102ef0 2024-09-11T22:53:24.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f90957fa640 1 -- 172.21.15.152:0/1706657551 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0003bd0 con 0x55a72ad822f0 2024-09-11T22:53:24.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f90957fa640 1 -- 172.21.15.152:0/1706657551 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a0015070 con 0x55a72ad822f0 2024-09-11T22:53:24.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.041+0000 7f90957fa640 1 -- 172.21.15.152:0/1706657551 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0004d30 con 0x55a72ad822f0 2024-09-11T22:53:24.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.042+0000 7f90b7b11740 1 -- 172.21.15.152:0/1706657551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad822f0 msgr2=0x55a72ad8c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:24.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.042+0000 7f90b7b11740 1 --2- 172.21.15.152:0/1706657551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad822f0 0x55a72ad8c670 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f90a0004630 tx=0x7f90a000bb20 comp rx=0 tx=0).stop 2024-09-11T22:53:24.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.042+0000 7f90ae7fc640 1 --2- 172.21.15.152:0/2384340736 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f90980a2d80 0x7f90980a5240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55a72ad7f2e0 tx=0x7f90a402a480 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:24.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.042+0000 7f90b7b11740 1 -- 172.21.15.152:0/1706657551 shutdown_connections 2024-09-11T22:53:24.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.042+0000 7f90b7b11740 1 --2- 172.21.15.152:0/1706657551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad8cbb0 0x55a72ad8efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.042+0000 7f90b7b11740 1 --2- 172.21.15.152:0/1706657551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad822f0 0x55a72ad8c670 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.042+0000 7f90b7b11740 1 --2- 172.21.15.152:0/1706657551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad819d0 0x55a72ad81db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.042+0000 7f90b7b11740 1 -- 172.21.15.152:0/1706657551 >> 172.21.15.152:0/1706657551 conn(0x55a72ad7df60 msgr2=0x55a72ad7e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:24.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.042+0000 7f90b7b11740 1 -- 172.21.15.152:0/1706657551 shutdown_connections 2024-09-11T22:53:24.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.042+0000 7f90b7b11740 1 -- 172.21.15.152:0/1706657551 wait complete. 2024-09-11T22:53:24.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90b7b11740 1 -- start start 2024-09-11T22:53:24.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90b7b11740 1 Processor -- start 2024-09-11T22:53:24.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90b7b11740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad819d0 0x55a72ad8c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90b7b11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad822f0 0x55a72ad89710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90b7b11740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad8cbb0 0x55a72ad89c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90b7b11740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a72ad8f780 con 0x55a72ad8cbb0 2024-09-11T22:53:24.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90b7b11740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a72aca2f20 con 0x55a72ad819d0 2024-09-11T22:53:24.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90b7b11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a72ad8f900 con 0x55a72ad822f0 2024-09-11T22:53:24.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90b7b11740 10 client.0 _ll_register_callbacks cb 0x55a72ac6c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:24.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90af7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad8cbb0 0x55a72ad89c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90af7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad8cbb0 0x55a72ad89c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37126/0 (socket says 172.21.15.152:37126) 2024-09-11T22:53:24.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90af7fe640 1 -- 172.21.15.152:0/2408331716 learned_addr learned my addr 172.21.15.152:0/2408331716 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:24.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90aeffd640 1 --2- 172.21.15.152:0/2408331716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad819d0 0x55a72ad8c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90ae7fc640 1 --2- 172.21.15.152:0/2408331716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad822f0 0x55a72ad89710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90af7fe640 1 -- 172.21.15.152:0/2408331716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad822f0 msgr2=0x55a72ad89710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:24.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90af7fe640 1 --2- 172.21.15.152:0/2408331716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad822f0 0x55a72ad89710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90af7fe640 1 -- 172.21.15.152:0/2408331716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad819d0 msgr2=0x55a72ad8c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:24.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90af7fe640 1 --2- 172.21.15.152:0/2408331716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad819d0 0x55a72ad8c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.043+0000 7f90af7fe640 1 -- 172.21.15.152:0/2408331716 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a72aa592b0 con 0x55a72ad8cbb0 2024-09-11T22:53:24.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f90af7fe640 1 --2- 172.21.15.152:0/2408331716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad8cbb0 0x55a72ad89c50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f90a809bf60 tx=0x7f90a809bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbf02104b4662ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:24.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f9092ff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:24.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f9092ff5640 1 -- 172.21.15.152:0/2408331716 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a809bfd0 con 0x55a72ad8cbb0 2024-09-11T22:53:24.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f9092ff5640 1 -- 172.21.15.152:0/2408331716 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f90a80cd560 con 0x55a72ad8cbb0 2024-09-11T22:53:24.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f90b7b11740 1 -- 172.21.15.152:0/2408331716 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a0015070 con 0x55a72ad8cbb0 2024-09-11T22:53:24.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f9092ff5640 1 -- 172.21.15.152:0/2408331716 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f90a80cd760 con 0x55a72ad8cbb0 2024-09-11T22:53:24.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f90b7b11740 10 client.15576 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:24.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f90b7b11740 1 -- 172.21.15.152:0/2408331716 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a72ad9b010 con 0x55a72ad8cbb0 2024-09-11T22:53:24.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f9092ff5640 1 -- 172.21.15.152:0/2408331716 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a0015070 con 0x55a72ad8cbb0 2024-09-11T22:53:24.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f9092ff5640 1 -- 172.21.15.152:0/2408331716 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f90a809b420 con 0x55a72ad8cbb0 2024-09-11T22:53:24.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f90b7b11740 10 client.15576 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:24.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f90b7b11740 10 client.15576 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:53:24.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f90b7b11740 20 client.15576 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:24.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f90b7b11740 1 --2- 172.21.15.152:0/2408331716 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a72ad9d5f0 0x55a72ad9ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f90b7b11740 4 client.15576 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:24.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f90b7b11740 1 -- 172.21.15.152:0/2408331716 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a72acf5600 con 0x55a72ad9d5f0 2024-09-11T22:53:24.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.044+0000 7f90aeffd640 1 --2- 172.21.15.152:0/2408331716 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a72ad9d5f0 0x55a72ad9ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.045+0000 7f90aeffd640 1 --2- 172.21.15.152:0/2408331716 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a72ad9d5f0 0x55a72ad9ea60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6abc69d2449ec26e server_cookie=fddfd4766c6fe44d in_seq=0 out_seq=0 2024-09-11T22:53:24.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.045+0000 7f9092ff5640 10 client.15576 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:24.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.046+0000 7f9092ff5640 1 -- 172.21.15.152:0/2408331716 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a72acf5600 con 0x55a72ad9d5f0 2024-09-11T22:53:24.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.046+0000 7f9092ff5640 10 client.15576 handle_command_reply: tid=0 2024-09-11T22:53:24.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.046+0000 7f90b7b11740 1 client.15576 shutdown 2024-09-11T22:53:24.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.046+0000 7f90b7b11740 1 -- 172.21.15.152:0/2408331716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad8cbb0 msgr2=0x55a72ad89c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:24.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.046+0000 7f90b7b11740 1 --2- 172.21.15.152:0/2408331716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad8cbb0 0x55a72ad89c50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f90a809bf60 tx=0x7f90a809bb90 comp rx=0 tx=0).stop 2024-09-11T22:53:24.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.046+0000 7f90b7b11740 1 -- 172.21.15.152:0/2408331716 shutdown_connections 2024-09-11T22:53:24.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.046+0000 7f90b7b11740 1 --2- 172.21.15.152:0/2408331716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad8cbb0 0x55a72ad89c50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.046+0000 7f90b7b11740 1 --2- 172.21.15.152:0/2408331716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad822f0 0x55a72ad89710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.046+0000 7f90b7b11740 1 --2- 172.21.15.152:0/2408331716 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a72ad9d5f0 0x55a72ad9ea60 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.046+0000 7f90b7b11740 1 --2- 172.21.15.152:0/2408331716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad819d0 0x55a72ad8c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.046+0000 7f90b7b11740 1 -- 172.21.15.152:0/2408331716 >> 172.21.15.152:0/2408331716 conn(0x55a72ad7df60 msgr2=0x55a72ad83c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:24.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.046+0000 7f9092ff5640 0 client.15576 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:24.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.046+0000 7f90b7b11740 1 -- 172.21.15.152:0/2408331716 shutdown_connections 2024-09-11T22:53:24.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.046+0000 7f90b7b11740 1 -- 172.21.15.152:0/2408331716 wait complete. 2024-09-11T22:53:24.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.046+0000 7f90b7b11740 20 client.15576 trim_cache size 0 max 16384 2024-09-11T22:53:24.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.063+0000 7f90b7b11740 1 Processor -- start 2024-09-11T22:53:24.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.063+0000 7f90b7b11740 1 -- start start 2024-09-11T22:53:24.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90b7b11740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad819d0 0x55a72ad9ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90b7b11740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad822f0 0x55a72ad7e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90b7b11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad8cbb0 0x55a72ada7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90b7b11740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a72ad9cda0 con 0x55a72ad822f0 2024-09-11T22:53:24.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90b7b11740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a72ad9d470 con 0x55a72ad819d0 2024-09-11T22:53:24.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90b7b11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a72ad9db40 con 0x55a72ad8cbb0 2024-09-11T22:53:24.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90ae7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad822f0 0x55a72ad7e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90ae7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad822f0 0x55a72ad7e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37132/0 (socket says 172.21.15.152:37132) 2024-09-11T22:53:24.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90ae7fc640 1 -- 172.21.15.152:0/3526646512 learned_addr learned my addr 172.21.15.152:0/3526646512 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:24.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90aeffd640 1 --2- 172.21.15.152:0/3526646512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad819d0 0x55a72ad9ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90af7fe640 1 --2- 172.21.15.152:0/3526646512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad8cbb0 0x55a72ada7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90ae7fc640 1 -- 172.21.15.152:0/3526646512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad8cbb0 msgr2=0x55a72ada7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:24.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90ae7fc640 1 --2- 172.21.15.152:0/3526646512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad8cbb0 0x55a72ada7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90ae7fc640 1 -- 172.21.15.152:0/3526646512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad819d0 msgr2=0x55a72ad9ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:24.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90ae7fc640 1 --2- 172.21.15.152:0/3526646512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad819d0 0x55a72ad9ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90ae7fc640 1 -- 172.21.15.152:0/3526646512 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a72ad7dd90 con 0x55a72ad822f0 2024-09-11T22:53:24.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90aeffd640 1 --2- 172.21.15.152:0/3526646512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad819d0 0x55a72ad9ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:24.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f90ae7fc640 1 --2- 172.21.15.152:0/3526646512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad822f0 0x55a72ad7e340 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009230 tx=0x7f90a4040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e799c240a0dce7d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:24.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f9094ff9640 1 -- 172.21.15.152:0/3526646512 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4035040 con 0x55a72ad822f0 2024-09-11T22:53:24.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.064+0000 7f9094ff9640 1 -- 172.21.15.152:0/3526646512 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a4045070 con 0x55a72ad822f0 2024-09-11T22:53:24.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.065+0000 7f9094ff9640 1 -- 172.21.15.152:0/3526646512 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a403db00 con 0x55a72ad822f0 2024-09-11T22:53:24.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.065+0000 7f90b7b11740 1 -- 172.21.15.152:0/3526646512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad822f0 msgr2=0x55a72ad7e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:24.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.065+0000 7f90b7b11740 1 --2- 172.21.15.152:0/3526646512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad822f0 0x55a72ad7e340 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009230 tx=0x7f90a4040000 comp rx=0 tx=0).stop 2024-09-11T22:53:24.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.065+0000 7f90b7b11740 1 -- 172.21.15.152:0/3526646512 shutdown_connections 2024-09-11T22:53:24.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.065+0000 7f90b7b11740 1 --2- 172.21.15.152:0/3526646512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad8cbb0 0x55a72ada7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.065+0000 7f90b7b11740 1 --2- 172.21.15.152:0/3526646512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad822f0 0x55a72ad7e340 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.065+0000 7f90b7b11740 1 --2- 172.21.15.152:0/3526646512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad819d0 0x55a72ad9ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.065+0000 7f90b7b11740 1 -- 172.21.15.152:0/3526646512 >> 172.21.15.152:0/3526646512 conn(0x55a72ad7df60 msgr2=0x55a72ad896d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:24.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.065+0000 7f90b7b11740 1 -- 172.21.15.152:0/3526646512 shutdown_connections 2024-09-11T22:53:24.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.065+0000 7f90b7b11740 1 -- 172.21.15.152:0/3526646512 wait complete. 2024-09-11T22:53:24.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.065+0000 7f90b7b11740 1 -- start start 2024-09-11T22:53:24.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90b7b11740 1 Processor -- start 2024-09-11T22:53:24.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90b7b11740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad819d0 0x55a72ada16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90b7b11740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad822f0 0x55a72ada68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90b7b11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad8cbb0 0x55a72ada6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90b7b11740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a72ad9e8e0 con 0x55a72ad819d0 2024-09-11T22:53:24.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90b7b11740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a72ad9e210 con 0x55a72ad822f0 2024-09-11T22:53:24.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90b7b11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a72ad9efb0 con 0x55a72ad8cbb0 2024-09-11T22:53:24.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90b7b11740 10 client.0 _ll_register_callbacks cb 0x7f90a80cd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:24.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90aeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad819d0 0x55a72ada16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90aeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad819d0 0x55a72ada16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37136/0 (socket says 172.21.15.152:37136) 2024-09-11T22:53:24.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90aeffd640 1 -- 172.21.15.152:0/1885630147 learned_addr learned my addr 172.21.15.152:0/1885630147 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:24.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90ae7fc640 1 --2- 172.21.15.152:0/1885630147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad822f0 0x55a72ada68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90af7fe640 1 --2- 172.21.15.152:0/1885630147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad8cbb0 0x55a72ada6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90aeffd640 1 -- 172.21.15.152:0/1885630147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad8cbb0 msgr2=0x55a72ada6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:24.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90aeffd640 1 --2- 172.21.15.152:0/1885630147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad8cbb0 0x55a72ada6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90aeffd640 1 -- 172.21.15.152:0/1885630147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad822f0 msgr2=0x55a72ada68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:24.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90aeffd640 1 --2- 172.21.15.152:0/1885630147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad822f0 0x55a72ada68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90aeffd640 1 -- 172.21.15.152:0/1885630147 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a72ada75f0 con 0x55a72ad819d0 2024-09-11T22:53:24.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90ae7fc640 1 --2- 172.21.15.152:0/1885630147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad822f0 0x55a72ada68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:24.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f90aeffd640 1 --2- 172.21.15.152:0/1885630147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad819d0 0x55a72ada16a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f90a0004630 tx=0x7f90a000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d358dd7d912f53d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:24.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.066+0000 7f9092ff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:24.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.067+0000 7f9092ff5640 1 -- 172.21.15.152:0/1885630147 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0011440 con 0x55a72ad819d0 2024-09-11T22:53:24.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.067+0000 7f9092ff5640 1 -- 172.21.15.152:0/1885630147 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f90a00115e0 con 0x55a72ad819d0 2024-09-11T22:53:24.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.067+0000 7f90b7b11740 1 -- 172.21.15.152:0/1885630147 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90a4045070 con 0x55a72ad819d0 2024-09-11T22:53:24.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.067+0000 7f9092ff5640 1 -- 172.21.15.152:0/1885630147 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f90a00117e0 con 0x55a72ad819d0 2024-09-11T22:53:24.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.067+0000 7f90b7b11740 10 client.15582 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:24.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.067+0000 7f90b7b11740 1 -- 172.21.15.152:0/1885630147 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a72ae0bcd0 con 0x55a72ad819d0 2024-09-11T22:53:24.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.067+0000 7f9092ff5640 1 -- 172.21.15.152:0/1885630147 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a4045070 con 0x55a72ad819d0 2024-09-11T22:53:24.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.067+0000 7f9092ff5640 1 -- 172.21.15.152:0/1885630147 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f90a0019840 con 0x55a72ad819d0 2024-09-11T22:53:24.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.067+0000 7f90b7b11740 10 client.15582 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:24.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.067+0000 7f90b7b11740 10 client.15582 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:53:24.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.067+0000 7f90b7b11740 20 client.15582 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:24.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.067+0000 7f90b7b11740 1 --2- 172.21.15.152:0/1885630147 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a72ae10790 0x55a72ae0f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:24.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.067+0000 7f90b7b11740 4 client.15582 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:53:24.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.067+0000 7f90b7b11740 1 -- 172.21.15.152:0/1885630147 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "status"}) -- 0x55a72acefb10 con 0x55a72ae10790 2024-09-11T22:53:24.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.067+0000 7f90ae7fc640 1 --2- 172.21.15.152:0/1885630147 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a72ae10790 0x55a72ae0f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:24.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f90ae7fc640 1 --2- 172.21.15.152:0/1885630147 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a72ae10790 0x55a72ae0f4a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ee65d9d707f5b6f server_cookie=cf8c0f7baa6c1194 in_seq=0 out_seq=0 2024-09-11T22:53:24.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f9092ff5640 10 client.15582 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:24.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f9092ff5640 1 -- 172.21.15.152:0/1885630147 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a72acefb10 con 0x55a72ae10790 2024-09-11T22:53:24.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f9092ff5640 10 client.15582 handle_command_reply: tid=0 2024-09-11T22:53:24.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f90b7b11740 1 client.15582 shutdown 2024-09-11T22:53:24.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f90b7b11740 1 -- 172.21.15.152:0/1885630147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad819d0 msgr2=0x55a72ada16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:24.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f90b7b11740 1 --2- 172.21.15.152:0/1885630147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad819d0 0x55a72ada16a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f90a0004630 tx=0x7f90a000bd70 comp rx=0 tx=0).stop 2024-09-11T22:53:24.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f90b7b11740 1 -- 172.21.15.152:0/1885630147 shutdown_connections 2024-09-11T22:53:24.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f90b7b11740 1 --2- 172.21.15.152:0/1885630147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a72ad8cbb0 0x55a72ada6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f90b7b11740 1 --2- 172.21.15.152:0/1885630147 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a72ae10790 0x55a72ae0f4a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f90b7b11740 1 --2- 172.21.15.152:0/1885630147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a72ad822f0 0x55a72ada68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f90b7b11740 1 --2- 172.21.15.152:0/1885630147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a72ad819d0 0x55a72ada16a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f90b7b11740 1 -- 172.21.15.152:0/1885630147 >> 172.21.15.152:0/1885630147 conn(0x55a72ad7df60 msgr2=0x55a72ad844a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:24.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f9092ff5640 0 client.15582 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:24.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f90b7b11740 1 -- 172.21.15.152:0/1885630147 shutdown_connections 2024-09-11T22:53:24.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f90b7b11740 1 -- 172.21.15.152:0/1885630147 wait complete. 2024-09-11T22:53:24.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.068+0000 7f90b7b11740 20 client.15582 trim_cache size 0 max 16384 2024-09-11T22:53:24.084 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:53:24.084 INFO:teuthology.orchestra.run.smithi152.stdout: "cluster_fsid": "0e4e82bc-708f-11ef-bcea-c7b262605968", 2024-09-11T22:53:24.084 INFO:teuthology.orchestra.run.smithi152.stdout: "whoami": 0, 2024-09-11T22:53:24.084 INFO:teuthology.orchestra.run.smithi152.stdout: "id": 24455, 2024-09-11T22:53:24.084 INFO:teuthology.orchestra.run.smithi152.stdout: "want_state": "up:active", 2024-09-11T22:53:24.084 INFO:teuthology.orchestra.run.smithi152.stdout: "state": "up:active", 2024-09-11T22:53:24.084 INFO:teuthology.orchestra.run.smithi152.stdout: "fs_name": "cephfs", 2024-09-11T22:53:24.084 INFO:teuthology.orchestra.run.smithi152.stdout: "rank_uptime": 148.91491573600001, 2024-09-11T22:53:24.084 INFO:teuthology.orchestra.run.smithi152.stdout: "mdsmap_epoch": 28, 2024-09-11T22:53:24.084 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch": 84, 2024-09-11T22:53:24.085 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_epoch_barrier": 83, 2024-09-11T22:53:24.085 INFO:teuthology.orchestra.run.smithi152.stdout: "uptime": 161.02346174100001 2024-09-11T22:53:24.085 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:53:24.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.069+0000 7f90b5738640 1 -- 172.21.15.152:0/2384340736 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f90980a2d80 msgr2=0x7f90980a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:24.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.069+0000 7f90b5738640 1 --2- 172.21.15.152:0/2384340736 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f90980a2d80 0x7f90980a5240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55a72ad7f2e0 tx=0x7f90a402a480 comp rx=0 tx=0).stop 2024-09-11T22:53:24.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.069+0000 7f90b5738640 1 -- 172.21.15.152:0/2384340736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90b0102ef0 msgr2=0x7f90b019cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:24.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.069+0000 7f90b5738640 1 --2- 172.21.15.152:0/2384340736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90b0102ef0 0x7f90b019cdc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f90a800cce0 tx=0x7f90a800a620 comp rx=0 tx=0).stop 2024-09-11T22:53:24.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.069+0000 7f90b5738640 1 -- 172.21.15.152:0/2384340736 shutdown_connections 2024-09-11T22:53:24.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.069+0000 7f90b5738640 1 --2- 172.21.15.152:0/2384340736 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f90980a2d80 0x7f90980a5240 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.069+0000 7f90b5738640 1 --2- 172.21.15.152:0/2384340736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90b0102ef0 0x7f90b019cdc0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.069+0000 7f90b5738640 1 --2- 172.21.15.152:0/2384340736 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90b0102570 0x7f90b0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.069+0000 7f90b5738640 1 --2- 172.21.15.152:0/2384340736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90b0100e20 0x7f90b01984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:24.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.069+0000 7f90b5738640 1 -- 172.21.15.152:0/2384340736 >> 172.21.15.152:0/2384340736 conn(0x7f90b00fc7d0 msgr2=0x7f90b006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:24.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.069+0000 7f90b5738640 1 -- 172.21.15.152:0/2384340736 shutdown_connections 2024-09-11T22:53:24.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:24.069+0000 7f90b5738640 1 -- 172.21.15.152:0/2384340736 wait complete. 2024-09-11T22:53:24.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:24 smithi152 ceph-mon[31181]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-09-11T22:53:24.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:24 smithi191 ceph-mon[35211]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-09-11T22:53:24.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:24 smithi204 ceph-mon[35454]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-09-11T22:53:24.983 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-11T22:53:26.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:26 smithi191 ceph-mon[35211]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.3 KiB/s wr, 17 op/s 2024-09-11T22:53:26.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:26 smithi204 ceph-mon[35454]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.3 KiB/s wr, 17 op/s 2024-09-11T22:53:26.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:26 smithi152 ceph-mon[31181]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.3 KiB/s wr, 17 op/s 2024-09-11T22:53:28.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:28 smithi191 ceph-mon[35211]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-09-11T22:53:28.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:28 smithi204 ceph-mon[35454]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-09-11T22:53:28.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:28 smithi152 ceph-mon[31181]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-09-11T22:53:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.465+0000 7f4d64d4b640 1 -- 172.21.15.152:0/168453887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d6010c0d0 msgr2=0x7f4d6010e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.465+0000 7f4d64d4b640 1 --2- 172.21.15.152:0/168453887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d6010c0d0 0x7f4d6010e550 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009ca0 tx=0x7f4d5802a950 comp rx=0 tx=0).stop 2024-09-11T22:53:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.465+0000 7f4d64d4b640 1 -- 172.21.15.152:0/168453887 shutdown_connections 2024-09-11T22:53:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.465+0000 7f4d64d4b640 1 --2- 172.21.15.152:0/168453887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d6010c0d0 0x7f4d6010e550 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.465+0000 7f4d64d4b640 1 --2- 172.21.15.152:0/168453887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d601035a0 0x7f4d6010ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.465+0000 7f4d64d4b640 1 --2- 172.21.15.152:0/168453887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d60102c80 0x7f4d60103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.465+0000 7f4d64d4b640 1 -- 172.21.15.152:0/168453887 >> 172.21.15.152:0/168453887 conn(0x7f4d60077990 msgr2=0x7f4d600fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.465+0000 7f4d64d4b640 1 -- 172.21.15.152:0/168453887 shutdown_connections 2024-09-11T22:53:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.465+0000 7f4d64d4b640 1 -- 172.21.15.152:0/168453887 wait complete. 2024-09-11T22:53:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.466+0000 7f4d64d4b640 1 Processor -- start 2024-09-11T22:53:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.466+0000 7f4d64d4b640 1 -- start start 2024-09-11T22:53:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d64d4b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d60102c80 0x7f4d601982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d64d4b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d601035a0 0x7f4d60198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d64d4b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d6010c0d0 0x7f4d6019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.470 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d64d4b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4d60111ec0 con 0x7f4d60102c80 2024-09-11T22:53:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d64d4b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4d60111d40 con 0x7f4d6010c0d0 2024-09-11T22:53:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d64d4b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d60112040 con 0x7f4d601035a0 2024-09-11T22:53:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d5e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d60102c80 0x7f4d601982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d5e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d60102c80 0x7f4d601982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59840/0 (socket says 172.21.15.152:59840) 2024-09-11T22:53:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d5e59c640 1 -- 172.21.15.152:0/108297766 learned_addr learned my addr 172.21.15.152:0/108297766 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d55d9b640 1 --2- 172.21.15.152:0/108297766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d601035a0 0x7f4d60198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d5ed9d640 1 --2- 172.21.15.152:0/108297766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d6010c0d0 0x7f4d6019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d5e59c640 1 -- 172.21.15.152:0/108297766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d601035a0 msgr2=0x7f4d60198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d5e59c640 1 --2- 172.21.15.152:0/108297766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d601035a0 0x7f4d60198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d5e59c640 1 -- 172.21.15.152:0/108297766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d6010c0d0 msgr2=0x7f4d6019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d5e59c640 1 --2- 172.21.15.152:0/108297766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d6010c0d0 0x7f4d6019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d5e59c640 1 -- 172.21.15.152:0/108297766 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6019d2c0 con 0x7f4d60102c80 2024-09-11T22:53:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d5e59c640 1 --2- 172.21.15.152:0/108297766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d60102c80 0x7f4d601982d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000d950 tx=0x7f4d5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d57fff640 1 -- 172.21.15.152:0/108297766 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d50017030 con 0x7f4d60102c80 2024-09-11T22:53:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d57fff640 1 -- 172.21.15.152:0/108297766 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d50004020 con 0x7f4d60102c80 2024-09-11T22:53:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d64d4b640 1 -- 172.21.15.152:0/108297766 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6019d5b0 con 0x7f4d60102c80 2024-09-11T22:53:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.467+0000 7f4d57fff640 1 -- 172.21.15.152:0/108297766 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d500047a0 con 0x7f4d60102c80 2024-09-11T22:53:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.468+0000 7f4d64d4b640 1 -- 172.21.15.152:0/108297766 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d601a4e40 con 0x7f4d60102c80 2024-09-11T22:53:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.470+0000 7f4d57fff640 1 -- 172.21.15.152:0/108297766 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f4d500041d0 con 0x7f4d60102c80 2024-09-11T22:53:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.470+0000 7f4d67bc0740 1 Processor -- start 2024-09-11T22:53:29.483 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.470+0000 7f4d67bc0740 1 -- start start 2024-09-11T22:53:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d67bc0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d67bc0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0d02f0 0x55ab9d0deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d67bc0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0e14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d67bc0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ab9d0342f0 con 0x55ab9d0d02f0 2024-09-11T22:53:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d67bc0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ab9cfdc270 con 0x55ab9d0cf9d0 2024-09-11T22:53:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d67bc0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ab9d0302b0 con 0x55ab9d0df0c0 2024-09-11T22:53:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d5e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0d02f0 0x55ab9d0deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d5e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0d02f0 0x55ab9d0deb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59844/0 (socket says 172.21.15.152:59844) 2024-09-11T22:53:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d5e59c640 1 -- 172.21.15.152:0/2165239802 learned_addr learned my addr 172.21.15.152:0/2165239802 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:29.484 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d5ed9d640 1 --2- 172.21.15.152:0/2165239802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d55d9b640 1 --2- 172.21.15.152:0/2165239802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0e14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d57fff640 1 --2- 172.21.15.152:0/108297766 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4d4c0a2e40 0x7f4d4c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d5e59c640 1 -- 172.21.15.152:0/2165239802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0df0c0 msgr2=0x55ab9d0e14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d5e59c640 1 --2- 172.21.15.152:0/2165239802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0e14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d5e59c640 1 -- 172.21.15.152:0/2165239802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0cf9d0 msgr2=0x55ab9d0cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d5e59c640 1 --2- 172.21.15.152:0/2165239802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d5e59c640 1 -- 172.21.15.152:0/2165239802 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab9cfdb340 con 0x55ab9d0d02f0 2024-09-11T22:53:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d5ed9d640 1 --2- 172.21.15.152:0/108297766 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4d4c0a2e40 0x7f4d4c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d57fff640 1 -- 172.21.15.152:0/108297766 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4d500cef70 con 0x7f4d60102c80 2024-09-11T22:53:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.471+0000 7f4d5e59c640 1 --2- 172.21.15.152:0/2165239802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0d02f0 0x55ab9d0deb80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4d500cc590 tx=0x7f4d50007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f7cca2fd5a3fee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.472+0000 7f4d54d99640 1 -- 172.21.15.152:0/2165239802 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5009b140 con 0x55ab9d0d02f0 2024-09-11T22:53:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.472+0000 7f4d54d99640 1 -- 172.21.15.152:0/2165239802 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d50018270 con 0x55ab9d0d02f0 2024-09-11T22:53:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.472+0000 7f4d5ed9d640 1 --2- 172.21.15.152:0/108297766 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4d4c0a2e40 0x7f4d4c0a5300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4d58004af0 tx=0x7f4d5803e040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.472+0000 7f4d54d99640 1 -- 172.21.15.152:0/2165239802 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d50066690 con 0x55ab9d0d02f0 2024-09-11T22:53:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.472+0000 7f4d67bc0740 1 -- 172.21.15.152:0/2165239802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0d02f0 msgr2=0x55ab9d0deb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.472+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/2165239802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0d02f0 0x55ab9d0deb80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4d500cc590 tx=0x7f4d50007f90 comp rx=0 tx=0).stop 2024-09-11T22:53:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.472+0000 7f4d67bc0740 1 -- 172.21.15.152:0/2165239802 shutdown_connections 2024-09-11T22:53:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/2165239802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0e14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/2165239802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0d02f0 0x55ab9d0deb80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/2165239802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d67bc0740 1 -- 172.21.15.152:0/2165239802 >> 172.21.15.152:0/2165239802 conn(0x55ab9d0cc060 msgr2=0x55ab9d0cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d67bc0740 1 -- 172.21.15.152:0/2165239802 shutdown_connections 2024-09-11T22:53:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d67bc0740 1 -- 172.21.15.152:0/2165239802 wait complete. 2024-09-11T22:53:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d67bc0740 1 -- start start 2024-09-11T22:53:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d67bc0740 1 Processor -- start 2024-09-11T22:53:29.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d67bc0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0dc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d67bc0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0d02f0 0x55ab9d0dc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d67bc0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0dca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d67bc0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ab9d0e1c90 con 0x55ab9d0cf9d0 2024-09-11T22:53:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d67bc0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ab9cfbd290 con 0x55ab9d0d02f0 2024-09-11T22:53:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d67bc0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ab9d0e1e10 con 0x55ab9d0df0c0 2024-09-11T22:53:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d67bc0740 10 client.0 _ll_register_callbacks cb 0x55ab9cfee140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d5e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0dc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d5e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0dc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59856/0 (socket says 172.21.15.152:59856) 2024-09-11T22:53:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.473+0000 7f4d5e59c640 1 -- 172.21.15.152:0/4145680022 learned_addr learned my addr 172.21.15.152:0/4145680022 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d5e59c640 1 -- 172.21.15.152:0/4145680022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0df0c0 msgr2=0x55ab9d0dca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d5e59c640 1 --2- 172.21.15.152:0/4145680022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0dca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d5e59c640 1 -- 172.21.15.152:0/4145680022 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0d02f0 msgr2=0x55ab9d0dc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d5e59c640 1 --2- 172.21.15.152:0/4145680022 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0d02f0 0x55ab9d0dc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d5e59c640 1 -- 172.21.15.152:0/4145680022 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab9cd47210 con 0x55ab9d0cf9d0 2024-09-11T22:53:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d5e59c640 1 --2- 172.21.15.152:0/4145680022 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0dc000 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4d500a3ab0 tx=0x7f4d5009ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d96d408bc23c2a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d45ffb640 1 -- 172.21.15.152:0/4145680022 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d500cce60 con 0x55ab9d0cf9d0 2024-09-11T22:53:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d45ffb640 1 -- 172.21.15.152:0/4145680022 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4d50099670 con 0x55ab9d0cf9d0 2024-09-11T22:53:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d67bc0740 1 -- 172.21.15.152:0/4145680022 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ab9d041a20 con 0x55ab9d0cf9d0 2024-09-11T22:53:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d45ffb640 1 -- 172.21.15.152:0/4145680022 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d5009acd0 con 0x55ab9d0cf9d0 2024-09-11T22:53:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d67bc0740 10 client.15594 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d67bc0740 1 -- 172.21.15.152:0/4145680022 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab9ce13620 con 0x55ab9d0cf9d0 2024-09-11T22:53:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d45ffb640 1 -- 172.21.15.152:0/4145680022 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ab9d041a20 con 0x55ab9d0cf9d0 2024-09-11T22:53:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d45ffb640 1 -- 172.21.15.152:0/4145680022 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4d500979e0 con 0x55ab9d0cf9d0 2024-09-11T22:53:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d67bc0740 10 client.15594 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d67bc0740 10 client.15594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d67bc0740 20 client.15594 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/4145680022 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ab9d0eeef0 0x55ab9d0f12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d67bc0740 4 client.15594 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.474+0000 7f4d67bc0740 1 -- 172.21.15.152:0/4145680022 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab9d03bd70 con 0x55ab9d0eeef0 2024-09-11T22:53:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.475+0000 7f4d55d9b640 1 --2- 172.21.15.152:0/4145680022 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ab9d0eeef0 0x55ab9d0f12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.475+0000 7f4d45ffb640 10 client.15594 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.475+0000 7f4d55d9b640 1 --2- 172.21.15.152:0/4145680022 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ab9d0eeef0 0x55ab9d0f12e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7428821e4e87f2e server_cookie=4c60402654a79392 in_seq=0 out_seq=0 2024-09-11T22:53:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.476+0000 7f4d45ffb640 1 -- 172.21.15.152:0/4145680022 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab9d03bd70 con 0x55ab9d0eeef0 2024-09-11T22:53:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.476+0000 7f4d45ffb640 10 client.15594 handle_command_reply: tid=0 2024-09-11T22:53:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.476+0000 7f4d67bc0740 1 client.15594 shutdown 2024-09-11T22:53:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.476+0000 7f4d67bc0740 1 -- 172.21.15.152:0/4145680022 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0cf9d0 msgr2=0x55ab9d0dc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.476+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/4145680022 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0dc000 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4d500a3ab0 tx=0x7f4d5009ba70 comp rx=0 tx=0).stop 2024-09-11T22:53:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.476+0000 7f4d67bc0740 1 -- 172.21.15.152:0/4145680022 shutdown_connections 2024-09-11T22:53:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.476+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/4145680022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0dca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.476+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/4145680022 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ab9d0eeef0 0x55ab9d0f12e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.476+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/4145680022 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0d02f0 0x55ab9d0dc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.476+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/4145680022 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0dc000 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.476+0000 7f4d45ffb640 0 client.15594 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.476+0000 7f4d67bc0740 1 -- 172.21.15.152:0/4145680022 >> 172.21.15.152:0/4145680022 conn(0x55ab9d0cc060 msgr2=0x55ab9d0d1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.476+0000 7f4d67bc0740 1 -- 172.21.15.152:0/4145680022 shutdown_connections 2024-09-11T22:53:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.476+0000 7f4d67bc0740 1 -- 172.21.15.152:0/4145680022 wait complete. 2024-09-11T22:53:29.490 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.476+0000 7f4d67bc0740 20 client.15594 trim_cache size 0 max 16384 2024-09-11T22:53:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.490+0000 7f4d67bc0740 1 Processor -- start 2024-09-11T22:53:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.490+0000 7f4d67bc0740 1 -- start start 2024-09-11T22:53:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d67bc0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d67bc0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0d02f0 0x55ab9d0f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d67bc0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0d9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d67bc0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4d601ab4c0 con 0x55ab9d0cf9d0 2024-09-11T22:53:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d67bc0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4d601ad000 con 0x55ab9d0df0c0 2024-09-11T22:53:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d67bc0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d601abb90 con 0x55ab9d0d02f0 2024-09-11T22:53:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d5e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d5e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0f69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59868/0 (socket says 172.21.15.152:59868) 2024-09-11T22:53:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d5e59c640 1 -- 172.21.15.152:0/1542754725 learned_addr learned my addr 172.21.15.152:0/1542754725 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d5ed9d640 1 --2- 172.21.15.152:0/1542754725 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0d9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d5e59c640 1 -- 172.21.15.152:0/1542754725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0d02f0 msgr2=0x55ab9d0f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d5e59c640 1 --2- 172.21.15.152:0/1542754725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0d02f0 0x55ab9d0f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d5e59c640 1 -- 172.21.15.152:0/1542754725 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0df0c0 msgr2=0x55ab9d0d9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d5e59c640 1 --2- 172.21.15.152:0/1542754725 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0d9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d5e59c640 1 -- 172.21.15.152:0/1542754725 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab9d0e17f0 con 0x55ab9d0cf9d0 2024-09-11T22:53:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d5e59c640 1 --2- 172.21.15.152:0/1542754725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0f69d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4d50005fd0 tx=0x7f4d50005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e23db846fb3988d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d47fff640 1 -- 172.21.15.152:0/1542754725 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d500cce60 con 0x55ab9d0cf9d0 2024-09-11T22:53:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d47fff640 1 -- 172.21.15.152:0/1542754725 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5009b370 con 0x55ab9d0cf9d0 2024-09-11T22:53:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.491+0000 7f4d47fff640 1 -- 172.21.15.152:0/1542754725 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5009aeb0 con 0x55ab9d0cf9d0 2024-09-11T22:53:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.492+0000 7f4d67bc0740 1 -- 172.21.15.152:0/1542754725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0cf9d0 msgr2=0x55ab9d0f69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.492+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/1542754725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0f69d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4d50005fd0 tx=0x7f4d50005bf0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.492+0000 7f4d67bc0740 1 -- 172.21.15.152:0/1542754725 shutdown_connections 2024-09-11T22:53:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.492+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/1542754725 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0d9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.492+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/1542754725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0d02f0 0x55ab9d0f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.492+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/1542754725 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0f69d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.492+0000 7f4d67bc0740 1 -- 172.21.15.152:0/1542754725 >> 172.21.15.152:0/1542754725 conn(0x55ab9d0cc060 msgr2=0x55ab9d0cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.492+0000 7f4d67bc0740 1 -- 172.21.15.152:0/1542754725 shutdown_connections 2024-09-11T22:53:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.492+0000 7f4d67bc0740 1 -- 172.21.15.152:0/1542754725 wait complete. 2024-09-11T22:53:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.492+0000 7f4d67bc0740 1 -- start start 2024-09-11T22:53:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.492+0000 7f4d67bc0740 1 Processor -- start 2024-09-11T22:53:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.492+0000 7f4d67bc0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.492+0000 7f4d67bc0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0d02f0 0x55ab9d0e34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.492+0000 7f4d67bc0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0e3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d67bc0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4d601ac930 con 0x55ab9d0df0c0 2024-09-11T22:53:29.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d67bc0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4d601ac260 con 0x55ab9d0d02f0 2024-09-11T22:53:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d67bc0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4d601a94a0 con 0x55ab9d0cf9d0 2024-09-11T22:53:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d67bc0740 10 client.0 _ll_register_callbacks cb 0x7f4d500976a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d5ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0e3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d5ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0e3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59878/0 (socket says 172.21.15.152:59878) 2024-09-11T22:53:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d5ed9d640 1 -- 172.21.15.152:0/1112188117 learned_addr learned my addr 172.21.15.152:0/1112188117 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d55d9b640 1 --2- 172.21.15.152:0/1112188117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0d02f0 0x55ab9d0e34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d5e59c640 1 --2- 172.21.15.152:0/1112188117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d5ed9d640 1 -- 172.21.15.152:0/1112188117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0cf9d0 msgr2=0x55ab9d0e2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d5ed9d640 1 --2- 172.21.15.152:0/1112188117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0e2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d5ed9d640 1 -- 172.21.15.152:0/1112188117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0d02f0 msgr2=0x55ab9d0e34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d5ed9d640 1 --2- 172.21.15.152:0/1112188117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0d02f0 0x55ab9d0e34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d5ed9d640 1 -- 172.21.15.152:0/1112188117 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab9d158b40 con 0x55ab9d0df0c0 2024-09-11T22:53:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d5ed9d640 1 --2- 172.21.15.152:0/1112188117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0e3a00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4d58005fa0 tx=0x7f4d580095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ee9fc27abe1558 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d45ffb640 1 -- 172.21.15.152:0/1112188117 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58034040 con 0x55ab9d0df0c0 2024-09-11T22:53:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d45ffb640 1 -- 172.21.15.152:0/1112188117 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4d58030ad0 con 0x55ab9d0df0c0 2024-09-11T22:53:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.493+0000 7f4d67bc0740 1 -- 172.21.15.152:0/1112188117 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d5009b370 con 0x55ab9d0df0c0 2024-09-11T22:53:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.494+0000 7f4d45ffb640 1 -- 172.21.15.152:0/1112188117 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4d5802f730 con 0x55ab9d0df0c0 2024-09-11T22:53:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.494+0000 7f4d67bc0740 10 client.15600 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.494+0000 7f4d67bc0740 1 -- 172.21.15.152:0/1112188117 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab9d159e30 con 0x55ab9d0df0c0 2024-09-11T22:53:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.494+0000 7f4d45ffb640 1 -- 172.21.15.152:0/1112188117 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d5009b370 con 0x55ab9d0df0c0 2024-09-11T22:53:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.494+0000 7f4d45ffb640 1 -- 172.21.15.152:0/1112188117 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4d5803a430 con 0x55ab9d0df0c0 2024-09-11T22:53:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.494+0000 7f4d67bc0740 10 client.15600 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.494+0000 7f4d67bc0740 10 client.15600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:29.501 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.494+0000 7f4d67bc0740 20 client.15600 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.494+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/1112188117 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ab9d15a5b0 0x55ab9d15c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.494+0000 7f4d67bc0740 4 client.15600 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:53:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.494+0000 7f4d67bc0740 1 -- 172.21.15.152:0/1112188117 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab9d0cb930 con 0x55ab9d15a5b0 2024-09-11T22:53:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.494+0000 7f4d5e59c640 1 --2- 172.21.15.152:0/1112188117 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ab9d15a5b0 0x55ab9d15c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.495+0000 7f4d5e59c640 1 --2- 172.21.15.152:0/1112188117 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ab9d15a5b0 0x55ab9d15c9d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da7f3b164409578 server_cookie=ef5ae12ab1819d70 in_seq=0 out_seq=0 2024-09-11T22:53:29.502 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.495+0000 7f4d45ffb640 10 client.15600 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.498+0000 7f4d45ffb640 1 -- 172.21.15.152:0/1112188117 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ab9d0cb930 con 0x55ab9d15a5b0 2024-09-11T22:53:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.498+0000 7f4d45ffb640 10 client.15600 handle_command_reply: tid=0 2024-09-11T22:53:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.498+0000 7f4d67bc0740 1 client.15600 shutdown 2024-09-11T22:53:29.503 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.498+0000 7f4d67bc0740 1 -- 172.21.15.152:0/1112188117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0df0c0 msgr2=0x55ab9d0e3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.498+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/1112188117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0e3a00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4d58005fa0 tx=0x7f4d580095c0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.498+0000 7f4d67bc0740 1 -- 172.21.15.152:0/1112188117 shutdown_connections 2024-09-11T22:53:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.498+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/1112188117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab9d0df0c0 0x55ab9d0e3a00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.498+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/1112188117 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ab9d15a5b0 0x55ab9d15c9d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.498+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/1112188117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab9d0d02f0 0x55ab9d0e34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.498+0000 7f4d45ffb640 0 client.15600 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.498+0000 7f4d67bc0740 1 --2- 172.21.15.152:0/1112188117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab9d0cf9d0 0x55ab9d0e2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.498+0000 7f4d67bc0740 1 -- 172.21.15.152:0/1112188117 >> 172.21.15.152:0/1112188117 conn(0x55ab9d0cc060 msgr2=0x55ab9d0d14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.498+0000 7f4d67bc0740 1 -- 172.21.15.152:0/1112188117 shutdown_connections 2024-09-11T22:53:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.498+0000 7f4d67bc0740 1 -- 172.21.15.152:0/1112188117 wait complete. 2024-09-11T22:53:29.504 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.499+0000 7f4d67bc0740 20 client.15600 trim_cache size 0 max 16384 2024-09-11T22:53:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:{ 2024-09-11T22:53:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "message": "", 2024-09-11T22:53:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "return_code": 0 2024-09-11T22:53:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:} 2024-09-11T22:53:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.499+0000 7f4d64d4b640 1 -- 172.21.15.152:0/108297766 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4d4c0a2e40 msgr2=0x7f4d4c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.499+0000 7f4d64d4b640 1 --2- 172.21.15.152:0/108297766 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4d4c0a2e40 0x7f4d4c0a5300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4d58004af0 tx=0x7f4d5803e040 comp rx=0 tx=0).stop 2024-09-11T22:53:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.499+0000 7f4d64d4b640 1 -- 172.21.15.152:0/108297766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d60102c80 msgr2=0x7f4d601982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.499+0000 7f4d64d4b640 1 --2- 172.21.15.152:0/108297766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d60102c80 0x7f4d601982d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000d950 tx=0x7f4d5000de10 comp rx=0 tx=0).stop 2024-09-11T22:53:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.499+0000 7f4d64d4b640 1 -- 172.21.15.152:0/108297766 shutdown_connections 2024-09-11T22:53:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.499+0000 7f4d64d4b640 1 --2- 172.21.15.152:0/108297766 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4d4c0a2e40 0x7f4d4c0a5300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.499+0000 7f4d64d4b640 1 --2- 172.21.15.152:0/108297766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d6010c0d0 0x7f4d6019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.499+0000 7f4d64d4b640 1 --2- 172.21.15.152:0/108297766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d601035a0 0x7f4d60198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.499+0000 7f4d64d4b640 1 --2- 172.21.15.152:0/108297766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d60102c80 0x7f4d601982d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.499+0000 7f4d64d4b640 1 -- 172.21.15.152:0/108297766 >> 172.21.15.152:0/108297766 conn(0x7f4d60077990 msgr2=0x7f4d600fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.499+0000 7f4d64d4b640 1 -- 172.21.15.152:0/108297766 shutdown_connections 2024-09-11T22:53:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:29.499+0000 7f4d64d4b640 1 -- 172.21.15.152:0/108297766 wait complete. 2024-09-11T22:53:30.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:30 smithi191 ceph-mon[35211]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-09-11T22:53:30.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:30 smithi204 ceph-mon[35454]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-09-11T22:53:30.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:30 smithi152 ceph-mon[31181]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-09-11T22:53:32.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:32 smithi191 ceph-mon[35211]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-11T22:53:32.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:32 smithi204 ceph-mon[35454]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-11T22:53:32.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:32 smithi152 ceph-mon[31181]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-09-11T22:53:33.912 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:53:34.191 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:34.313 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:34 smithi152 ceph-mon[31181]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:53:34.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:34 smithi191 ceph-mon[35211]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:53:34.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:34 smithi204 ceph-mon[35454]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:53:34.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.745+0000 7f949820b640 1 -- 172.21.15.152:0/19211294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9490102780 msgr2=0x7f94901067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:34.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.745+0000 7f949820b640 1 --2- 172.21.15.152:0/19211294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9490102780 0x7f94901067d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9480009900 tx=0x7f948002a410 comp rx=0 tx=0).stop 2024-09-11T22:53:34.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.745+0000 7f949820b640 1 -- 172.21.15.152:0/19211294 shutdown_connections 2024-09-11T22:53:34.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.745+0000 7f949820b640 1 --2- 172.21.15.152:0/19211294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9490106f00 0x7f949010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.745+0000 7f949820b640 1 --2- 172.21.15.152:0/19211294 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9490102780 0x7f94901067d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.745+0000 7f949820b640 1 --2- 172.21.15.152:0/19211294 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9490101dd0 0x7f94901021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.745+0000 7f949820b640 1 -- 172.21.15.152:0/19211294 >> 172.21.15.152:0/19211294 conn(0x7f94900fd890 msgr2=0x7f94900ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:34.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.745+0000 7f949820b640 1 -- 172.21.15.152:0/19211294 shutdown_connections 2024-09-11T22:53:34.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.745+0000 7f949820b640 1 -- 172.21.15.152:0/19211294 wait complete. 2024-09-11T22:53:34.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.746+0000 7f949820b640 1 Processor -- start 2024-09-11T22:53:34.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.746+0000 7f949820b640 1 -- start start 2024-09-11T22:53:34.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.746+0000 7f949820b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9490101dd0 0x7f94901985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.746+0000 7f949820b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9490102780 0x7f9490198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.746+0000 7f949820b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9490106f00 0x7f949019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.746+0000 7f949820b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9490111f80 con 0x7f9490106f00 2024-09-11T22:53:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.746+0000 7f949820b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9490111e00 con 0x7f9490101dd0 2024-09-11T22:53:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.746+0000 7f949820b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9490112100 con 0x7f9490102780 2024-09-11T22:53:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.747+0000 7f94967a8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9490106f00 0x7f949019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.747+0000 7f9495fa7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9490101dd0 0x7f94901985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.747+0000 7f94967a8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9490106f00 0x7f949019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59888/0 (socket says 172.21.15.152:59888) 2024-09-11T22:53:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.747+0000 7f94967a8640 1 -- 172.21.15.152:0/3085075625 learned_addr learned my addr 172.21.15.152:0/3085075625 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.747+0000 7f94957a6640 1 --2- 172.21.15.152:0/3085075625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9490102780 0x7f9490198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.747+0000 7f94967a8640 1 -- 172.21.15.152:0/3085075625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9490102780 msgr2=0x7f9490198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.747+0000 7f94967a8640 1 --2- 172.21.15.152:0/3085075625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9490102780 0x7f9490198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.747+0000 7f94967a8640 1 -- 172.21.15.152:0/3085075625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9490101dd0 msgr2=0x7f94901985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:34.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.747+0000 7f94967a8640 1 --2- 172.21.15.152:0/3085075625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9490101dd0 0x7f94901985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.747+0000 7f94967a8640 1 -- 172.21.15.152:0/3085075625 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949019d570 con 0x7f9490106f00 2024-09-11T22:53:34.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.747+0000 7f94967a8640 1 --2- 172.21.15.152:0/3085075625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9490106f00 0x7f949019ce90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f948800b970 tx=0x7f948800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:34.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.747+0000 7f9486ffd640 1 -- 172.21.15.152:0/3085075625 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488019070 con 0x7f9490106f00 2024-09-11T22:53:34.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.747+0000 7f9486ffd640 1 -- 172.21.15.152:0/3085075625 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f948800cb40 con 0x7f9490106f00 2024-09-11T22:53:34.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.747+0000 7f949820b640 1 -- 172.21.15.152:0/3085075625 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f949019d860 con 0x7f9490106f00 2024-09-11T22:53:34.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.747+0000 7f9486ffd640 1 -- 172.21.15.152:0/3085075625 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488007650 con 0x7f9490106f00 2024-09-11T22:53:34.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.748+0000 7f949820b640 1 -- 172.21.15.152:0/3085075625 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94901a5140 con 0x7f9490106f00 2024-09-11T22:53:34.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.750+0000 7f949a5e4740 1 Processor -- start 2024-09-11T22:53:34.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.751+0000 7f9486ffd640 1 -- 172.21.15.152:0/3085075625 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f948801c070 con 0x7f9490106f00 2024-09-11T22:53:34.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.751+0000 7f949a5e4740 1 -- start start 2024-09-11T22:53:34.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.751+0000 7f949a5e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7ab4a0 0x5580ea7ab880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.751+0000 7f949a5e4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7abdc0 0x5580ea7b6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.751+0000 7f949a5e4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7b6680 0x5580ea7b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.751+0000 7f949a5e4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5580ea7112e0 con 0x5580ea7abdc0 2024-09-11T22:53:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.751+0000 7f949a5e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5580ea6b8ce0 con 0x5580ea7ab4a0 2024-09-11T22:53:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.751+0000 7f949a5e4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5580ea70d2a0 con 0x5580ea7b6680 2024-09-11T22:53:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.751+0000 7f9495fa7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7abdc0 0x5580ea7b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.752+0000 7f9495fa7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7abdc0 0x5580ea7b6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59900/0 (socket says 172.21.15.152:59900) 2024-09-11T22:53:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.752+0000 7f9495fa7640 1 -- 172.21.15.152:0/680409447 learned_addr learned my addr 172.21.15.152:0/680409447 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.752+0000 7f94957a6640 1 --2- 172.21.15.152:0/680409447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7ab4a0 0x5580ea7ab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.752+0000 7f9486ffd640 1 --2- 172.21.15.152:0/3085075625 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f947c0a2e00 0x7f947c0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.752+0000 7f94957a6640 1 --2- 172.21.15.152:0/680409447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7b6680 0x5580ea7b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.752+0000 7f94967a8640 1 --2- 172.21.15.152:0/3085075625 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f947c0a2e00 0x7f947c0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.752+0000 7f9495fa7640 1 -- 172.21.15.152:0/680409447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7b6680 msgr2=0x5580ea7b8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:34.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.752+0000 7f9495fa7640 1 --2- 172.21.15.152:0/680409447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7b6680 0x5580ea7b8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.752+0000 7f9495fa7640 1 -- 172.21.15.152:0/680409447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7ab4a0 msgr2=0x5580ea7ab880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.752+0000 7f9495fa7640 1 --2- 172.21.15.152:0/680409447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7ab4a0 0x5580ea7ab880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.752+0000 7f9495fa7640 1 -- 172.21.15.152:0/680409447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580ea69d440 con 0x5580ea7abdc0 2024-09-11T22:53:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.752+0000 7f9486ffd640 1 -- 172.21.15.152:0/3085075625 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f94880cea10 con 0x7f9490106f00 2024-09-11T22:53:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.752+0000 7f9495fa7640 1 --2- 172.21.15.152:0/680409447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7abdc0 0x5580ea7b6140 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f948c00bd10 tx=0x7f948c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e11fd1465c21a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.752+0000 7f94967a8640 1 --2- 172.21.15.152:0/3085075625 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f947c0a2e00 0x7f947c0a52c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f94880cc260 tx=0x7f948800e040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.753+0000 7f947bfff640 1 -- 172.21.15.152:0/680409447 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c004030 con 0x5580ea7abdc0 2024-09-11T22:53:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.753+0000 7f947bfff640 1 -- 172.21.15.152:0/680409447 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f948c0041d0 con 0x5580ea7abdc0 2024-09-11T22:53:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.753+0000 7f947bfff640 1 -- 172.21.15.152:0/680409447 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c0237e0 con 0x5580ea7abdc0 2024-09-11T22:53:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.753+0000 7f949a5e4740 1 -- 172.21.15.152:0/680409447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7abdc0 msgr2=0x5580ea7b6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.753+0000 7f949a5e4740 1 --2- 172.21.15.152:0/680409447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7abdc0 0x5580ea7b6140 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f948c00bd10 tx=0x7f948c0107a0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.753+0000 7f949a5e4740 1 -- 172.21.15.152:0/680409447 shutdown_connections 2024-09-11T22:53:34.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.753+0000 7f949a5e4740 1 --2- 172.21.15.152:0/680409447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7b6680 0x5580ea7b8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.753+0000 7f949a5e4740 1 --2- 172.21.15.152:0/680409447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7abdc0 0x5580ea7b6140 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.753+0000 7f949a5e4740 1 --2- 172.21.15.152:0/680409447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7ab4a0 0x5580ea7ab880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.753+0000 7f949a5e4740 1 -- 172.21.15.152:0/680409447 >> 172.21.15.152:0/680409447 conn(0x5580ea7a7a30 msgr2=0x5580ea7a7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:34.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.754+0000 7f949a5e4740 1 -- 172.21.15.152:0/680409447 shutdown_connections 2024-09-11T22:53:34.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.754+0000 7f949a5e4740 1 -- 172.21.15.152:0/680409447 wait complete. 2024-09-11T22:53:34.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.754+0000 7f949a5e4740 1 -- start start 2024-09-11T22:53:34.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.754+0000 7f949a5e4740 1 Processor -- start 2024-09-11T22:53:34.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.755+0000 7f949a5e4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7ab4a0 0x5580ea7af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.755+0000 7f949a5e4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7abdc0 0x5580ea7af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.755+0000 7f949a5e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7b6680 0x5580ea7aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.755+0000 7f949a5e4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5580ea7b91f0 con 0x5580ea7ab4a0 2024-09-11T22:53:34.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.755+0000 7f94957a6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7ab4a0 0x5580ea7af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.755+0000 7f94957a6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7ab4a0 0x5580ea7af480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59914/0 (socket says 172.21.15.152:59914) 2024-09-11T22:53:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.755+0000 7f94957a6640 1 -- 172.21.15.152:0/1866004727 learned_addr learned my addr 172.21.15.152:0/1866004727 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.755+0000 7f949a5e4740 1 -- 172.21.15.152:0/1866004727 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5580ea6cc050 con 0x5580ea7b6680 2024-09-11T22:53:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.755+0000 7f9495fa7640 1 --2- 172.21.15.152:0/1866004727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7abdc0 0x5580ea7af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.755+0000 7f949a5e4740 1 -- 172.21.15.152:0/1866004727 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5580ea7b9370 con 0x5580ea7abdc0 2024-09-11T22:53:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.756+0000 7f949a5e4740 10 client.0 _ll_register_callbacks cb 0x5580ea70c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.756+0000 7f94957a6640 1 -- 172.21.15.152:0/1866004727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7abdc0 msgr2=0x5580ea7af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.756+0000 7f94957a6640 1 --2- 172.21.15.152:0/1866004727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7abdc0 0x5580ea7af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.756+0000 7f94957a6640 1 -- 172.21.15.152:0/1866004727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7b6680 msgr2=0x5580ea7aff00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:53:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.756+0000 7f94957a6640 1 --2- 172.21.15.152:0/1866004727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7b6680 0x5580ea7aff00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.756+0000 7f94957a6640 1 -- 172.21.15.152:0/1866004727 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580ea4823c0 con 0x5580ea7ab4a0 2024-09-11T22:53:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.756+0000 7f9495fa7640 1 --2- 172.21.15.152:0/1866004727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7abdc0 0x5580ea7af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.756+0000 7f94957a6640 1 --2- 172.21.15.152:0/1866004727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7ab4a0 0x5580ea7af480 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f948002a930 tx=0x7f9480004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597f961c28b35c24 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.756+0000 7f94797fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.756+0000 7f94797fa640 1 -- 172.21.15.152:0/1866004727 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480035040 con 0x5580ea7ab4a0 2024-09-11T22:53:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.756+0000 7f94797fa640 1 -- 172.21.15.152:0/1866004727 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f94800043f0 con 0x5580ea7ab4a0 2024-09-11T22:53:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.756+0000 7f949a5e4740 1 -- 172.21.15.152:0/1866004727 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f948c0041d0 con 0x5580ea7ab4a0 2024-09-11T22:53:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.757+0000 7f94797fa640 1 -- 172.21.15.152:0/1866004727 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f94800035f0 con 0x5580ea7ab4a0 2024-09-11T22:53:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.757+0000 7f949a5e4740 10 client.15612 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.757+0000 7f949a5e4740 1 -- 172.21.15.152:0/1866004727 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5580ea7c4c10 con 0x5580ea7ab4a0 2024-09-11T22:53:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.757+0000 7f94797fa640 1 -- 172.21.15.152:0/1866004727 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f948c0041d0 con 0x5580ea7ab4a0 2024-09-11T22:53:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.757+0000 7f94797fa640 1 -- 172.21.15.152:0/1866004727 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f948003a040 con 0x5580ea7ab4a0 2024-09-11T22:53:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.757+0000 7f949a5e4740 10 client.15612 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.757+0000 7f949a5e4740 10 client.15612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.757+0000 7f949a5e4740 20 client.15612 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.757+0000 7f949a5e4740 1 --2- 172.21.15.152:0/1866004727 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5580ea7c6020 0x5580ea7c8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.757+0000 7f949a5e4740 4 client.15612 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.757+0000 7f949a5e4740 1 -- 172.21.15.152:0/1866004727 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580ea71e850 con 0x5580ea7c6020 2024-09-11T22:53:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.758+0000 7f9495fa7640 1 --2- 172.21.15.152:0/1866004727 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5580ea7c6020 0x5580ea7c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.758+0000 7f94797fa640 10 client.15612 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.758+0000 7f9495fa7640 1 --2- 172.21.15.152:0/1866004727 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5580ea7c6020 0x5580ea7c8410 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddb53f0855a905dd server_cookie=2180c1ba125ad36e in_seq=0 out_seq=0 2024-09-11T22:53:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.759+0000 7f94797fa640 1 -- 172.21.15.152:0/1866004727 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5580ea71e850 con 0x5580ea7c6020 2024-09-11T22:53:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.759+0000 7f94797fa640 10 client.15612 handle_command_reply: tid=0 2024-09-11T22:53:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.759+0000 7f949a5e4740 1 client.15612 shutdown 2024-09-11T22:53:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.759+0000 7f949a5e4740 1 -- 172.21.15.152:0/1866004727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7ab4a0 msgr2=0x5580ea7af480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.759+0000 7f949a5e4740 1 --2- 172.21.15.152:0/1866004727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7ab4a0 0x5580ea7af480 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f948002a930 tx=0x7f9480004040 comp rx=0 tx=0).stop 2024-09-11T22:53:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.759+0000 7f949a5e4740 1 -- 172.21.15.152:0/1866004727 shutdown_connections 2024-09-11T22:53:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.759+0000 7f949a5e4740 1 --2- 172.21.15.152:0/1866004727 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5580ea7c6020 0x5580ea7c8410 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.759+0000 7f949a5e4740 1 --2- 172.21.15.152:0/1866004727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7b6680 0x5580ea7aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.759+0000 7f949a5e4740 1 --2- 172.21.15.152:0/1866004727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7abdc0 0x5580ea7af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.759+0000 7f949a5e4740 1 --2- 172.21.15.152:0/1866004727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7ab4a0 0x5580ea7af480 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.759+0000 7f949a5e4740 1 -- 172.21.15.152:0/1866004727 >> 172.21.15.152:0/1866004727 conn(0x5580ea7a7a30 msgr2=0x5580ea7a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.759+0000 7f94797fa640 0 client.15612 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.759+0000 7f949a5e4740 1 -- 172.21.15.152:0/1866004727 shutdown_connections 2024-09-11T22:53:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.759+0000 7f949a5e4740 1 -- 172.21.15.152:0/1866004727 wait complete. 2024-09-11T22:53:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.759+0000 7f949a5e4740 20 client.15612 trim_cache size 0 max 16384 2024-09-11T22:53:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.786+0000 7f949a5e4740 1 Processor -- start 2024-09-11T22:53:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.786+0000 7f949a5e4740 1 -- start start 2024-09-11T22:53:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.786+0000 7f949a5e4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7ab4a0 0x5580ea7afd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.786+0000 7f949a5e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7abdc0 0x5580ea7cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.786+0000 7f949a5e4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7b6680 0x5580ea7ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.786+0000 7f949a5e4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f94901ab620 con 0x5580ea7ab4a0 2024-09-11T22:53:34.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.786+0000 7f949a5e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f94901ad160 con 0x5580ea7abdc0 2024-09-11T22:53:34.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.786+0000 7f949a5e4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f94901abcf0 con 0x5580ea7b6680 2024-09-11T22:53:34.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.786+0000 7f94957a6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7ab4a0 0x5580ea7afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.786+0000 7f94957a6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7ab4a0 0x5580ea7afd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59926/0 (socket says 172.21.15.152:59926) 2024-09-11T22:53:34.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.786+0000 7f94957a6640 1 -- 172.21.15.152:0/3873066867 learned_addr learned my addr 172.21.15.152:0/3873066867 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:34.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.786+0000 7f9495fa7640 1 --2- 172.21.15.152:0/3873066867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7abdc0 0x5580ea7cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.787+0000 7f94957a6640 1 --2- 172.21.15.152:0/3873066867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7b6680 0x5580ea7ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.787+0000 7f94957a6640 1 -- 172.21.15.152:0/3873066867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7b6680 msgr2=0x5580ea7ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:34.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.787+0000 7f94957a6640 1 --2- 172.21.15.152:0/3873066867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7b6680 0x5580ea7ce200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.787+0000 7f94957a6640 1 -- 172.21.15.152:0/3873066867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7abdc0 msgr2=0x5580ea7cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:34.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.787+0000 7f94957a6640 1 --2- 172.21.15.152:0/3873066867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7abdc0 0x5580ea7cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.787+0000 7f94957a6640 1 -- 172.21.15.152:0/3873066867 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580ea7ac330 con 0x5580ea7ab4a0 2024-09-11T22:53:34.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.787+0000 7f9495fa7640 1 --2- 172.21.15.152:0/3873066867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7abdc0 0x5580ea7cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:34.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.787+0000 7f94957a6640 1 --2- 172.21.15.152:0/3873066867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7ab4a0 0x5580ea7afd50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9480006010 tx=0x7f9480005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd2f497662812d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:34.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.787+0000 7f947b7fe640 1 -- 172.21.15.152:0/3873066867 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480035040 con 0x5580ea7ab4a0 2024-09-11T22:53:34.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.788+0000 7f947b7fe640 1 -- 172.21.15.152:0/3873066867 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94800043f0 con 0x5580ea7ab4a0 2024-09-11T22:53:34.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.788+0000 7f947b7fe640 1 -- 172.21.15.152:0/3873066867 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948003d720 con 0x5580ea7ab4a0 2024-09-11T22:53:34.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.788+0000 7f949a5e4740 1 -- 172.21.15.152:0/3873066867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7ab4a0 msgr2=0x5580ea7afd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:34.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.788+0000 7f949a5e4740 1 --2- 172.21.15.152:0/3873066867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7ab4a0 0x5580ea7afd50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9480006010 tx=0x7f9480005990 comp rx=0 tx=0).stop 2024-09-11T22:53:34.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.788+0000 7f949a5e4740 1 -- 172.21.15.152:0/3873066867 shutdown_connections 2024-09-11T22:53:34.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.788+0000 7f949a5e4740 1 --2- 172.21.15.152:0/3873066867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7b6680 0x5580ea7ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.788+0000 7f949a5e4740 1 --2- 172.21.15.152:0/3873066867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7abdc0 0x5580ea7cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.788+0000 7f949a5e4740 1 --2- 172.21.15.152:0/3873066867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7ab4a0 0x5580ea7afd50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.788+0000 7f949a5e4740 1 -- 172.21.15.152:0/3873066867 >> 172.21.15.152:0/3873066867 conn(0x5580ea7a7a30 msgr2=0x5580ea7b9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:34.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.788+0000 7f949a5e4740 1 -- 172.21.15.152:0/3873066867 shutdown_connections 2024-09-11T22:53:34.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.788+0000 7f949a5e4740 1 -- 172.21.15.152:0/3873066867 wait complete. 2024-09-11T22:53:34.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.789+0000 7f949a5e4740 1 -- start start 2024-09-11T22:53:34.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.789+0000 7f949a5e4740 1 Processor -- start 2024-09-11T22:53:34.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.789+0000 7f949a5e4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7abdc0 0x5580ea7bdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.789+0000 7f949a5e4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7b6680 0x5580ea7be140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.789+0000 7f949a5e4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7c22b0 0x5580ea7c2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.789+0000 7f949a5e4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f94901aca90 con 0x5580ea7b6680 2024-09-11T22:53:34.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.789+0000 7f949a5e4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f94901ac3c0 con 0x5580ea7abdc0 2024-09-11T22:53:34.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.789+0000 7f949a5e4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f94901a9600 con 0x5580ea7c22b0 2024-09-11T22:53:34.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.789+0000 7f949a5e4740 10 client.0 _ll_register_callbacks cb 0x7f94800318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:34.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.789+0000 7f9495fa7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7b6680 0x5580ea7be140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.789+0000 7f9495fa7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7b6680 0x5580ea7be140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59930/0 (socket says 172.21.15.152:59930) 2024-09-11T22:53:34.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.789+0000 7f9495fa7640 1 -- 172.21.15.152:0/3507282435 learned_addr learned my addr 172.21.15.152:0/3507282435 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:34.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.790+0000 7f94957a6640 1 --2- 172.21.15.152:0/3507282435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7abdc0 0x5580ea7bdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.790+0000 7f94957a6640 1 --2- 172.21.15.152:0/3507282435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7c22b0 0x5580ea7c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.790+0000 7f9495fa7640 1 -- 172.21.15.152:0/3507282435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7c22b0 msgr2=0x5580ea7c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:34.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.790+0000 7f9495fa7640 1 --2- 172.21.15.152:0/3507282435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7c22b0 0x5580ea7c2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.790+0000 7f9495fa7640 1 -- 172.21.15.152:0/3507282435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7abdc0 msgr2=0x5580ea7bdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:34.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.790+0000 7f9495fa7640 1 --2- 172.21.15.152:0/3507282435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7abdc0 0x5580ea7bdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.864 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:53:34.864 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T22:53:34.864 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T22:53:34.864 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:53:34.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.790+0000 7f9495fa7640 1 -- 172.21.15.152:0/3507282435 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580ea7ab2d0 con 0x5580ea7b6680 2024-09-11T22:53:34.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.790+0000 7f94957a6640 1 --2- 172.21.15.152:0/3507282435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7c22b0 0x5580ea7c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:34.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.790+0000 7f9495fa7640 1 --2- 172.21.15.152:0/3507282435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7b6680 0x5580ea7be140 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f948c003f70 tx=0x7f948c0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866e7954e407a55c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:34.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.790+0000 7f94797fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:34.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.790+0000 7f94797fa640 1 -- 172.21.15.152:0/3507282435 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c03f060 con 0x5580ea7b6680 2024-09-11T22:53:34.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.790+0000 7f94797fa640 1 -- 172.21.15.152:0/3507282435 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f948c005450 con 0x5580ea7b6680 2024-09-11T22:53:34.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.790+0000 7f949a5e4740 1 -- 172.21.15.152:0/3507282435 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94800043f0 con 0x5580ea7b6680 2024-09-11T22:53:34.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.790+0000 7f94797fa640 1 -- 172.21.15.152:0/3507282435 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f948c005650 con 0x5580ea7b6680 2024-09-11T22:53:34.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.790+0000 7f949a5e4740 10 client.15618 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:34.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.790+0000 7f949a5e4740 1 -- 172.21.15.152:0/3507282435 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5580ea830910 con 0x5580ea7b6680 2024-09-11T22:53:34.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.791+0000 7f94797fa640 1 -- 172.21.15.152:0/3507282435 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94800043f0 con 0x5580ea7b6680 2024-09-11T22:53:34.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.791+0000 7f94797fa640 1 -- 172.21.15.152:0/3507282435 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f948c03a770 con 0x5580ea7b6680 2024-09-11T22:53:34.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.791+0000 7f949a5e4740 10 client.15618 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:34.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.791+0000 7f949a5e4740 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:34.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.791+0000 7f949a5e4740 20 client.15618 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:34.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.791+0000 7f949a5e4740 1 --2- 172.21.15.152:0/3507282435 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5580ea830ec0 0x5580ea8332e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:34.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.791+0000 7f949a5e4740 4 client.15618 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:53:34.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.791+0000 7f949a5e4740 1 -- 172.21.15.152:0/3507282435 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5580ea718d60 con 0x5580ea830ec0 2024-09-11T22:53:34.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.791+0000 7f94957a6640 1 --2- 172.21.15.152:0/3507282435 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5580ea830ec0 0x5580ea8332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:34.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f94797fa640 10 client.15618 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:34.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f94957a6640 1 --2- 172.21.15.152:0/3507282435 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5580ea830ec0 0x5580ea8332e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53b325985e074409 server_cookie=9e4b801d64648ef1 in_seq=0 out_seq=0 2024-09-11T22:53:34.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f94797fa640 1 -- 172.21.15.152:0/3507282435 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5580ea718d60 con 0x5580ea830ec0 2024-09-11T22:53:34.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f94797fa640 10 client.15618 handle_command_reply: tid=0 2024-09-11T22:53:34.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f949a5e4740 1 client.15618 shutdown 2024-09-11T22:53:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f949a5e4740 1 -- 172.21.15.152:0/3507282435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7b6680 msgr2=0x5580ea7be140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f949a5e4740 1 --2- 172.21.15.152:0/3507282435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7b6680 0x5580ea7be140 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f948c003f70 tx=0x7f948c0058e0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f949a5e4740 1 -- 172.21.15.152:0/3507282435 shutdown_connections 2024-09-11T22:53:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f949a5e4740 1 --2- 172.21.15.152:0/3507282435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5580ea7c22b0 0x5580ea7c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f949a5e4740 1 --2- 172.21.15.152:0/3507282435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5580ea7b6680 0x5580ea7be140 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f949a5e4740 1 --2- 172.21.15.152:0/3507282435 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5580ea830ec0 0x5580ea8332e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f949a5e4740 1 --2- 172.21.15.152:0/3507282435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5580ea7abdc0 0x5580ea7bdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f94797fa640 0 client.15618 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f949a5e4740 1 -- 172.21.15.152:0/3507282435 >> 172.21.15.152:0/3507282435 conn(0x5580ea7a7a30 msgr2=0x5580ea7acf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f949a5e4740 1 -- 172.21.15.152:0/3507282435 shutdown_connections 2024-09-11T22:53:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f949a5e4740 1 -- 172.21.15.152:0/3507282435 wait complete. 2024-09-11T22:53:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.792+0000 7f949a5e4740 20 client.15618 trim_cache size 0 max 16384 2024-09-11T22:53:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.793+0000 7f949820b640 1 -- 172.21.15.152:0/3085075625 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f947c0a2e00 msgr2=0x7f947c0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.793+0000 7f949820b640 1 --2- 172.21.15.152:0/3085075625 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f947c0a2e00 0x7f947c0a52c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f94880cc260 tx=0x7f948800e040 comp rx=0 tx=0).stop 2024-09-11T22:53:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.793+0000 7f949820b640 1 -- 172.21.15.152:0/3085075625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9490106f00 msgr2=0x7f949019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.793+0000 7f949820b640 1 --2- 172.21.15.152:0/3085075625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9490106f00 0x7f949019ce90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f948800b970 tx=0x7f948800be30 comp rx=0 tx=0).stop 2024-09-11T22:53:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.793+0000 7f949820b640 1 -- 172.21.15.152:0/3085075625 shutdown_connections 2024-09-11T22:53:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.793+0000 7f949820b640 1 --2- 172.21.15.152:0/3085075625 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f947c0a2e00 0x7f947c0a52c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.793+0000 7f949820b640 1 --2- 172.21.15.152:0/3085075625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9490106f00 0x7f949019ce90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.793+0000 7f949820b640 1 --2- 172.21.15.152:0/3085075625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9490102780 0x7f9490198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.793+0000 7f949820b640 1 --2- 172.21.15.152:0/3085075625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9490101dd0 0x7f94901985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:34.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.793+0000 7f949820b640 1 -- 172.21.15.152:0/3085075625 >> 172.21.15.152:0/3085075625 conn(0x7f94900fd890 msgr2=0x7f94900fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:34.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.793+0000 7f949820b640 1 -- 172.21.15.152:0/3085075625 shutdown_connections 2024-09-11T22:53:34.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:34.793+0000 7f949820b640 1 -- 172.21.15.152:0/3085075625 wait complete. 2024-09-11T22:53:35.613 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:53:35.614 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:53:35.885 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:36.310 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:36 smithi152 ceph-mon[31181]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T22:53:36.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.393+0000 7f243f466640 1 -- 172.21.15.152:0/3518786435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2438102570 msgr2=0x7f24381029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:36.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.393+0000 7f243f466640 1 --2- 172.21.15.152:0/3518786435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2438102570 0x7f24381029b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2428009900 tx=0x7f242802a480 comp rx=0 tx=0).stop 2024-09-11T22:53:36.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.393+0000 7f243f466640 1 -- 172.21.15.152:0/3518786435 shutdown_connections 2024-09-11T22:53:36.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.393+0000 7f243f466640 1 --2- 172.21.15.152:0/3518786435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2438102ef0 0x7f243810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:36.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.393+0000 7f243f466640 1 --2- 172.21.15.152:0/3518786435 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2438102570 0x7f24381029b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:36.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.393+0000 7f243f466640 1 --2- 172.21.15.152:0/3518786435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2438100e20 0x7f2438101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:36.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.393+0000 7f243f466640 1 -- 172.21.15.152:0/3518786435 >> 172.21.15.152:0/3518786435 conn(0x7f24380fc7d0 msgr2=0x7f24380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:36.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.393+0000 7f243f466640 1 -- 172.21.15.152:0/3518786435 shutdown_connections 2024-09-11T22:53:36.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.393+0000 7f243f466640 1 -- 172.21.15.152:0/3518786435 wait complete. 2024-09-11T22:53:36.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.393+0000 7f243f466640 1 Processor -- start 2024-09-11T22:53:36.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243f466640 1 -- start start 2024-09-11T22:53:36.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243f466640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2438100e20 0x7f24381984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:36.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243f466640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2438102570 0x7f2438198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:36.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243f466640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2438102ef0 0x7f243819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:36.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243f466640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2438110f80 con 0x7f2438100e20 2024-09-11T22:53:36.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243f466640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2438110e00 con 0x7f2438102570 2024-09-11T22:53:36.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243f466640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2438111100 con 0x7f2438102ef0 2024-09-11T22:53:36.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243d202640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2438100e20 0x7f24381984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:36.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243d202640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2438100e20 0x7f24381984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59944/0 (socket says 172.21.15.152:59944) 2024-09-11T22:53:36.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243d202640 1 -- 172.21.15.152:0/3490001757 learned_addr learned my addr 172.21.15.152:0/3490001757 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:36.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243ca01640 1 --2- 172.21.15.152:0/3490001757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2438102570 0x7f2438198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:36.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243da03640 1 --2- 172.21.15.152:0/3490001757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2438102ef0 0x7f243819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:36.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243d202640 1 -- 172.21.15.152:0/3490001757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2438102ef0 msgr2=0x7f243819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:36.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243d202640 1 --2- 172.21.15.152:0/3490001757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2438102ef0 0x7f243819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:36.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243d202640 1 -- 172.21.15.152:0/3490001757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2438102570 msgr2=0x7f2438198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:36.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243d202640 1 --2- 172.21.15.152:0/3490001757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2438102570 0x7f2438198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:36.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243d202640 1 -- 172.21.15.152:0/3490001757 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243819d490 con 0x7f2438100e20 2024-09-11T22:53:36.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.394+0000 7f243d202640 1 --2- 172.21.15.152:0/3490001757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2438100e20 0x7f24381984e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f243400d950 tx=0x7f243400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:36.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.395+0000 7f242e7fc640 1 -- 172.21.15.152:0/3490001757 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2434017070 con 0x7f2438100e20 2024-09-11T22:53:36.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.395+0000 7f243f466640 1 -- 172.21.15.152:0/3490001757 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243819d780 con 0x7f2438100e20 2024-09-11T22:53:36.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.395+0000 7f242e7fc640 1 -- 172.21.15.152:0/3490001757 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2434009c30 con 0x7f2438100e20 2024-09-11T22:53:36.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.395+0000 7f242e7fc640 1 -- 172.21.15.152:0/3490001757 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243400f6b0 con 0x7f2438100e20 2024-09-11T22:53:36.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.395+0000 7f243f466640 1 -- 172.21.15.152:0/3490001757 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24381a5060 con 0x7f2438100e20 2024-09-11T22:53:36.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.396+0000 7f243f466640 1 -- 172.21.15.152:0/3490001757 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2438101200 con 0x7f2438100e20 2024-09-11T22:53:36.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.397+0000 7f242e7fc640 1 -- 172.21.15.152:0/3490001757 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f24340109d0 con 0x7f2438100e20 2024-09-11T22:53:36.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.398+0000 7f242e7fc640 1 --2- 172.21.15.152:0/3490001757 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f24240a2d80 0x7f24240a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:36.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.398+0000 7f242e7fc640 1 -- 172.21.15.152:0/3490001757 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f24340cf1d0 con 0x7f2438100e20 2024-09-11T22:53:36.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.398+0000 7f243ca01640 1 --2- 172.21.15.152:0/3490001757 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f24240a2d80 0x7f24240a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:36.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.398+0000 7f243ca01640 1 --2- 172.21.15.152:0/3490001757 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f24240a2d80 0x7f24240a5240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2428009230 tx=0x7f24280058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:36.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.401+0000 7f242e7fc640 1 -- 172.21.15.152:0/3490001757 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2434009de0 con 0x7f2438100e20 2024-09-11T22:53:36.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:36 smithi191 ceph-mon[35211]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T22:53:36.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:36 smithi204 ceph-mon[35454]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T22:53:36.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.612+0000 7f243f466640 1 -- 172.21.15.152:0/3490001757 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2438199840 con 0x7f2438100e20 2024-09-11T22:53:36.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.613+0000 7f242e7fc640 1 -- 172.21.15.152:0/3490001757 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f243409bcd0 con 0x7f2438100e20 2024-09-11T22:53:36.614 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:53:36.615 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:53:36.615 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:53:36.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.616+0000 7f243f466640 1 -- 172.21.15.152:0/3490001757 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f24240a2d80 msgr2=0x7f24240a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:36.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.616+0000 7f243f466640 1 --2- 172.21.15.152:0/3490001757 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f24240a2d80 0x7f24240a5240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2428009230 tx=0x7f24280058d0 comp rx=0 tx=0).stop 2024-09-11T22:53:36.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.616+0000 7f243f466640 1 -- 172.21.15.152:0/3490001757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2438100e20 msgr2=0x7f24381984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:36.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.616+0000 7f243f466640 1 --2- 172.21.15.152:0/3490001757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2438100e20 0x7f24381984e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f243400d950 tx=0x7f243400de10 comp rx=0 tx=0).stop 2024-09-11T22:53:36.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.616+0000 7f243f466640 1 -- 172.21.15.152:0/3490001757 shutdown_connections 2024-09-11T22:53:36.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.616+0000 7f243f466640 1 --2- 172.21.15.152:0/3490001757 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f24240a2d80 0x7f24240a5240 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:36.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.616+0000 7f243f466640 1 --2- 172.21.15.152:0/3490001757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2438102ef0 0x7f243819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:36.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.616+0000 7f243f466640 1 --2- 172.21.15.152:0/3490001757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2438102570 0x7f2438198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:36.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.616+0000 7f243f466640 1 --2- 172.21.15.152:0/3490001757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2438100e20 0x7f24381984e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:36.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.616+0000 7f243f466640 1 -- 172.21.15.152:0/3490001757 >> 172.21.15.152:0/3490001757 conn(0x7f24380fc7d0 msgr2=0x7f243806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:36.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.616+0000 7f243f466640 1 -- 172.21.15.152:0/3490001757 shutdown_connections 2024-09-11T22:53:36.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:36.616+0000 7f243f466640 1 -- 172.21.15.152:0/3490001757 wait complete. 2024-09-11T22:53:37.300 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:37 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3490001757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:53:37.327 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:53:37.439 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:37 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3490001757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:53:37.447 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:37 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3490001757' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:53:37.592 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:38.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.119+0000 7fa59be89640 1 -- 172.21.15.152:0/2691242209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa59410c180 msgr2=0x7fa59410e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:38.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.119+0000 7fa59be89640 1 --2- 172.21.15.152:0/2691242209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa59410c180 0x7fa59410e620 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009ca0 tx=0x7fa58c02a950 comp rx=0 tx=0).stop 2024-09-11T22:53:38.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.119+0000 7fa59be89640 1 -- 172.21.15.152:0/2691242209 shutdown_connections 2024-09-11T22:53:38.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.119+0000 7fa59be89640 1 --2- 172.21.15.152:0/2691242209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa59410c180 0x7fa59410e620 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.119+0000 7fa59be89640 1 --2- 172.21.15.152:0/2691242209 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa594103670 0x7fa59410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.119+0000 7fa59be89640 1 --2- 172.21.15.152:0/2691242209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa594102d50 0x7fa594103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.119+0000 7fa59be89640 1 -- 172.21.15.152:0/2691242209 >> 172.21.15.152:0/2691242209 conn(0x7fa5940fc7d0 msgr2=0x7fa5940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:38.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.119+0000 7fa59be89640 1 -- 172.21.15.152:0/2691242209 shutdown_connections 2024-09-11T22:53:38.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.119+0000 7fa59be89640 1 -- 172.21.15.152:0/2691242209 wait complete. 2024-09-11T22:53:38.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.120+0000 7fa59be89640 1 Processor -- start 2024-09-11T22:53:38.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.120+0000 7fa59be89640 1 -- start start 2024-09-11T22:53:38.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.120+0000 7fa59be89640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa594102d50 0x7fa5941009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.120+0000 7fa59be89640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa594103670 0x7fa594100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.120+0000 7fa59be89640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa59410c180 0x7fa594101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.120+0000 7fa59be89640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa594111f80 con 0x7fa594102d50 2024-09-11T22:53:38.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.120+0000 7fa59be89640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa594111e00 con 0x7fa594103670 2024-09-11T22:53:38.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.120+0000 7fa59be89640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa594112100 con 0x7fa59410c180 2024-09-11T22:53:38.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.120+0000 7fa599c25640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa594102d50 0x7fa5941009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:38.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.120+0000 7fa599c25640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa594102d50 0x7fa5941009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59960/0 (socket says 172.21.15.152:59960) 2024-09-11T22:53:38.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.120+0000 7fa599c25640 1 -- 172.21.15.152:0/364002754 learned_addr learned my addr 172.21.15.152:0/364002754 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:38.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.121+0000 7fa59a426640 1 --2- 172.21.15.152:0/364002754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa59410c180 0x7fa594101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:38.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.121+0000 7fa599424640 1 --2- 172.21.15.152:0/364002754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa594103670 0x7fa594100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:38.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.121+0000 7fa599c25640 1 -- 172.21.15.152:0/364002754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa59410c180 msgr2=0x7fa594101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:38.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.121+0000 7fa599c25640 1 --2- 172.21.15.152:0/364002754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa59410c180 0x7fa594101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.121+0000 7fa599c25640 1 -- 172.21.15.152:0/364002754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa594103670 msgr2=0x7fa594100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:38.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.121+0000 7fa599c25640 1 --2- 172.21.15.152:0/364002754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa594103670 0x7fa594100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.121+0000 7fa599c25640 1 -- 172.21.15.152:0/364002754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5941a6870 con 0x7fa594102d50 2024-09-11T22:53:38.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.121+0000 7fa599424640 1 --2- 172.21.15.152:0/364002754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa594103670 0x7fa594100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:38.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.121+0000 7fa599c25640 1 --2- 172.21.15.152:0/364002754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa594102d50 0x7fa5941009b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa59000c890 tx=0x7fa59000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:38.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.121+0000 7fa58affd640 1 -- 172.21.15.152:0/364002754 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590004220 con 0x7fa594102d50 2024-09-11T22:53:38.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.121+0000 7fa59be89640 1 -- 172.21.15.152:0/364002754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5941a6b60 con 0x7fa594102d50 2024-09-11T22:53:38.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.121+0000 7fa58affd640 1 -- 172.21.15.152:0/364002754 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5900043c0 con 0x7fa594102d50 2024-09-11T22:53:38.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.121+0000 7fa58affd640 1 -- 172.21.15.152:0/364002754 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5900049e0 con 0x7fa594102d50 2024-09-11T22:53:38.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.121+0000 7fa59be89640 1 -- 172.21.15.152:0/364002754 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5941a70a0 con 0x7fa594102d50 2024-09-11T22:53:38.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.124+0000 7fa59e262740 1 Processor -- start 2024-09-11T22:53:38.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.124+0000 7fa59e262740 1 -- start start 2024-09-11T22:53:38.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.124+0000 7fa58affd640 1 -- 172.21.15.152:0/364002754 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7fa5900079b0 con 0x7fa594102d50 2024-09-11T22:53:38.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.124+0000 7fa59e262740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c151b40 0x55cc9c150090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.124+0000 7fa59e262740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c150a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.124+0000 7fa59e262740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c15b370 0x55cc9c15d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.124+0000 7fa59e262740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55cc9c0b62f0 con 0x55cc9c1505d0 2024-09-11T22:53:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.124+0000 7fa59e262740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55cc9c05dcf0 con 0x55cc9c151b40 2024-09-11T22:53:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.124+0000 7fa59e262740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cc9c0b22b0 con 0x55cc9c15b370 2024-09-11T22:53:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.124+0000 7fa599c25640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.124+0000 7fa599c25640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59968/0 (socket says 172.21.15.152:59968) 2024-09-11T22:53:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.124+0000 7fa599c25640 1 -- 172.21.15.152:0/3231488459 learned_addr learned my addr 172.21.15.152:0/3231488459 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.124+0000 7fa59a426640 1 --2- 172.21.15.152:0/3231488459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c151b40 0x55cc9c150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.125+0000 7fa599424640 1 --2- 172.21.15.152:0/3231488459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c15b370 0x55cc9c15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.125+0000 7fa599c25640 1 -- 172.21.15.152:0/3231488459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c15b370 msgr2=0x55cc9c15d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.125+0000 7fa599c25640 1 --2- 172.21.15.152:0/3231488459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c15b370 0x55cc9c15d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.125+0000 7fa599c25640 1 -- 172.21.15.152:0/3231488459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c151b40 msgr2=0x55cc9c150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.125+0000 7fa599c25640 1 --2- 172.21.15.152:0/3231488459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c151b40 0x55cc9c150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.125+0000 7fa599c25640 1 -- 172.21.15.152:0/3231488459 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc9c0423a0 con 0x55cc9c1505d0 2024-09-11T22:53:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.125+0000 7fa599424640 1 --2- 172.21.15.152:0/3231488459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c15b370 0x55cc9c15d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.125+0000 7fa59a426640 1 --2- 172.21.15.152:0/3231488459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c151b40 0x55cc9c150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.125+0000 7fa58affd640 1 --2- 172.21.15.152:0/364002754 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa5800a2d80 0x7fa5800a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.125+0000 7fa59a426640 1 --2- 172.21.15.152:0/364002754 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa5800a2d80 0x7fa5800a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.125+0000 7fa599c25640 1 --2- 172.21.15.152:0/3231488459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c150a10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa5900ccd00 tx=0x7fa5900d5c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebdf122b550227f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.125+0000 7fa58affd640 1 -- 172.21.15.152:0/364002754 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa5900cf6b0 con 0x7fa594102d50 2024-09-11T22:53:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.125+0000 7fa57ffff640 1 -- 172.21.15.152:0/3231488459 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59001d020 con 0x55cc9c1505d0 2024-09-11T22:53:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.125+0000 7fa57ffff640 1 -- 172.21.15.152:0/3231488459 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5900d5e50 con 0x55cc9c1505d0 2024-09-11T22:53:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.126+0000 7fa57ffff640 1 -- 172.21.15.152:0/3231488459 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59009b5f0 con 0x55cc9c1505d0 2024-09-11T22:53:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.126+0000 7fa59e262740 1 -- 172.21.15.152:0/3231488459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 msgr2=0x55cc9c150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.126+0000 7fa59e262740 1 --2- 172.21.15.152:0/3231488459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c150a10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa5900ccd00 tx=0x7fa5900d5c00 comp rx=0 tx=0).stop 2024-09-11T22:53:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.126+0000 7fa59a426640 1 --2- 172.21.15.152:0/364002754 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa5800a2d80 0x7fa5800a5240 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa58c030040 tx=0x7fa58c005ef0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:38.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.126+0000 7fa59e262740 1 -- 172.21.15.152:0/3231488459 shutdown_connections 2024-09-11T22:53:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.126+0000 7fa59e262740 1 --2- 172.21.15.152:0/3231488459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c15b370 0x55cc9c15d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.126+0000 7fa59e262740 1 --2- 172.21.15.152:0/3231488459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c150a10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.126+0000 7fa59e262740 1 --2- 172.21.15.152:0/3231488459 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c151b40 0x55cc9c150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.126+0000 7fa59e262740 1 -- 172.21.15.152:0/3231488459 >> 172.21.15.152:0/3231488459 conn(0x55cc9c119310 msgr2=0x55cc9c14cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.126+0000 7fa59e262740 1 -- 172.21.15.152:0/3231488459 shutdown_connections 2024-09-11T22:53:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.126+0000 7fa59e262740 1 -- 172.21.15.152:0/3231488459 wait complete. 2024-09-11T22:53:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.127+0000 7fa59e262740 1 -- start start 2024-09-11T22:53:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.127+0000 7fa59e262740 1 Processor -- start 2024-09-11T22:53:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.127+0000 7fa59e262740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.127+0000 7fa59e262740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c151b40 0x55cc9c158040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.127+0000 7fa59e262740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c15b370 0x55cc9c158650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.127+0000 7fa59e262740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55cc9c15e070 con 0x55cc9c1505d0 2024-09-11T22:53:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.127+0000 7fa59e262740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55cc9c071060 con 0x55cc9c151b40 2024-09-11T22:53:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.127+0000 7fa59e262740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cc9c15e1f0 con 0x55cc9c15b370 2024-09-11T22:53:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.127+0000 7fa59e262740 10 client.0 _ll_register_callbacks cb 0x55cc9bdc48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.127+0000 7fa599c25640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.127+0000 7fa599c25640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c15ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59980/0 (socket says 172.21.15.152:59980) 2024-09-11T22:53:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.127+0000 7fa599c25640 1 -- 172.21.15.152:0/3551405199 learned_addr learned my addr 172.21.15.152:0/3551405199 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.128+0000 7fa599c25640 1 -- 172.21.15.152:0/3551405199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c15b370 msgr2=0x55cc9c158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.128+0000 7fa599c25640 1 --2- 172.21.15.152:0/3551405199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c15b370 0x55cc9c158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.128+0000 7fa599c25640 1 -- 172.21.15.152:0/3551405199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c151b40 msgr2=0x55cc9c158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.128+0000 7fa599c25640 1 --2- 172.21.15.152:0/3551405199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c151b40 0x55cc9c158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.128+0000 7fa599c25640 1 -- 172.21.15.152:0/3551405199 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc9be273c0 con 0x55cc9c1505d0 2024-09-11T22:53:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.128+0000 7fa599c25640 1 --2- 172.21.15.152:0/3551405199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c15ad90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa5900a41f0 tx=0x7fa590005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef6ccf77e3aa9ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:38.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.128+0000 7fa57d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:38.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.128+0000 7fa57d7fa640 1 -- 172.21.15.152:0/3551405199 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59001d040 con 0x55cc9c1505d0 2024-09-11T22:53:38.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.128+0000 7fa57d7fa640 1 -- 172.21.15.152:0/3551405199 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa59009bd10 con 0x55cc9c1505d0 2024-09-11T22:53:38.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.128+0000 7fa59e262740 1 -- 172.21.15.152:0/3551405199 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5900d5e50 con 0x55cc9c1505d0 2024-09-11T22:53:38.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.128+0000 7fa57d7fa640 1 -- 172.21.15.152:0/3551405199 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa59009b410 con 0x55cc9c1505d0 2024-09-11T22:53:38.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.128+0000 7fa59e262740 10 client.15636 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:38.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.128+0000 7fa59e262740 1 -- 172.21.15.152:0/3551405199 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc9c169b70 con 0x55cc9c1505d0 2024-09-11T22:53:38.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.128+0000 7fa57d7fa640 1 -- 172.21.15.152:0/3551405199 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5900d5e50 con 0x55cc9c1505d0 2024-09-11T22:53:38.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.128+0000 7fa57d7fa640 1 -- 172.21.15.152:0/3551405199 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa5900a4290 con 0x55cc9c1505d0 2024-09-11T22:53:38.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.129+0000 7fa59e262740 10 client.15636 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:38.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.129+0000 7fa59e262740 10 client.15636 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:53:38.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.129+0000 7fa59e262740 20 client.15636 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:38.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.129+0000 7fa59e262740 1 --2- 172.21.15.152:0/3551405199 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55cc9c16aec0 0x55cc9c16d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.129+0000 7fa59e262740 4 client.15636 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:38.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.129+0000 7fa59e262740 1 -- 172.21.15.152:0/3551405199 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc9c0c3860 con 0x55cc9c16aec0 2024-09-11T22:53:38.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.129+0000 7fa599424640 1 --2- 172.21.15.152:0/3551405199 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55cc9c16aec0 0x55cc9c16d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:38.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.129+0000 7fa57d7fa640 10 client.15636 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:53:38.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.129+0000 7fa599424640 1 --2- 172.21.15.152:0/3551405199 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55cc9c16aec0 0x55cc9c16d2b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f4c43b8e2b36ae40 server_cookie=51797d0f381e2d81 in_seq=0 out_seq=0 2024-09-11T22:53:38.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.130+0000 7fa57d7fa640 1 -- 172.21.15.152:0/3551405199 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cc9c0c3860 con 0x55cc9c16aec0 2024-09-11T22:53:38.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.130+0000 7fa57d7fa640 10 client.15636 handle_command_reply: tid=0 2024-09-11T22:53:38.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.130+0000 7fa59e262740 1 client.15636 shutdown 2024-09-11T22:53:38.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.130+0000 7fa59e262740 1 -- 172.21.15.152:0/3551405199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 msgr2=0x55cc9c15ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:38.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.130+0000 7fa59e262740 1 --2- 172.21.15.152:0/3551405199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c15ad90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa5900a41f0 tx=0x7fa590005410 comp rx=0 tx=0).stop 2024-09-11T22:53:38.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.130+0000 7fa59e262740 1 -- 172.21.15.152:0/3551405199 shutdown_connections 2024-09-11T22:53:38.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.130+0000 7fa59e262740 1 --2- 172.21.15.152:0/3551405199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c15b370 0x55cc9c158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.130+0000 7fa59e262740 1 --2- 172.21.15.152:0/3551405199 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55cc9c16aec0 0x55cc9c16d2b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.130+0000 7fa59e262740 1 --2- 172.21.15.152:0/3551405199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c151b40 0x55cc9c158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.130+0000 7fa59e262740 1 --2- 172.21.15.152:0/3551405199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c15ad90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.130+0000 7fa57d7fa640 0 client.15636 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:53:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.130+0000 7fa59e262740 1 -- 172.21.15.152:0/3551405199 >> 172.21.15.152:0/3551405199 conn(0x55cc9c119310 msgr2=0x55cc9c152930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.130+0000 7fa59e262740 1 -- 172.21.15.152:0/3551405199 shutdown_connections 2024-09-11T22:53:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.130+0000 7fa59e262740 1 -- 172.21.15.152:0/3551405199 wait complete. 2024-09-11T22:53:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.130+0000 7fa59e262740 20 client.15636 trim_cache size 0 max 16384 2024-09-11T22:53:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.144+0000 7fa59e262740 1 Processor -- start 2024-09-11T22:53:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.144+0000 7fa59e262740 1 -- start start 2024-09-11T22:53:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa59e262740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c1505d0 0x55cc9c1674c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa59e262740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c151b40 0x55cc9c157900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa59e262740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c15b370 0x55cc9c14bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa59e262740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941af9e0 con 0x55cc9c15b370 2024-09-11T22:53:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa59e262740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941b1520 con 0x55cc9c151b40 2024-09-11T22:53:38.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa59e262740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941b00b0 con 0x55cc9c1505d0 2024-09-11T22:53:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa59a426640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c15b370 0x55cc9c14bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa59a426640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c15b370 0x55cc9c14bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59982/0 (socket says 172.21.15.152:59982) 2024-09-11T22:53:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa59a426640 1 -- 172.21.15.152:0/3319635933 learned_addr learned my addr 172.21.15.152:0/3319635933 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa599424640 1 --2- 172.21.15.152:0/3319635933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c151b40 0x55cc9c157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa599c25640 1 --2- 172.21.15.152:0/3319635933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c1505d0 0x55cc9c1674c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa59a426640 1 -- 172.21.15.152:0/3319635933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c1505d0 msgr2=0x55cc9c1674c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa59a426640 1 --2- 172.21.15.152:0/3319635933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c1505d0 0x55cc9c1674c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa59a426640 1 -- 172.21.15.152:0/3319635933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c151b40 msgr2=0x55cc9c157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa59a426640 1 --2- 172.21.15.152:0/3319635933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c151b40 0x55cc9c157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa59a426640 1 -- 172.21.15.152:0/3319635933 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc9c156ff0 con 0x55cc9c15b370 2024-09-11T22:53:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa599424640 1 --2- 172.21.15.152:0/3319635933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c151b40 0x55cc9c157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.145+0000 7fa59a426640 1 --2- 172.21.15.152:0/3319635933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c15b370 0x55cc9c14bbf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa58c035000 tx=0x7fa58c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fab38d585bc740 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.146+0000 7fa57f7fe640 1 -- 172.21.15.152:0/3319635933 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c034040 con 0x55cc9c15b370 2024-09-11T22:53:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.146+0000 7fa57f7fe640 1 -- 172.21.15.152:0/3319635933 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa58c04d030 con 0x55cc9c15b370 2024-09-11T22:53:38.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.146+0000 7fa57f7fe640 1 -- 172.21.15.152:0/3319635933 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0306a0 con 0x55cc9c15b370 2024-09-11T22:53:38.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.146+0000 7fa59e262740 1 -- 172.21.15.152:0/3319635933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c15b370 msgr2=0x55cc9c14bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:38.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.146+0000 7fa59e262740 1 --2- 172.21.15.152:0/3319635933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c15b370 0x55cc9c14bbf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa58c035000 tx=0x7fa58c0095c0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.146+0000 7fa59e262740 1 -- 172.21.15.152:0/3319635933 shutdown_connections 2024-09-11T22:53:38.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.146+0000 7fa59e262740 1 --2- 172.21.15.152:0/3319635933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c15b370 0x55cc9c14bbf0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.146+0000 7fa59e262740 1 --2- 172.21.15.152:0/3319635933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c151b40 0x55cc9c157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.146+0000 7fa59e262740 1 --2- 172.21.15.152:0/3319635933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c1505d0 0x55cc9c1674c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.146+0000 7fa59e262740 1 -- 172.21.15.152:0/3319635933 >> 172.21.15.152:0/3319635933 conn(0x55cc9c119310 msgr2=0x55cc9c14b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:38.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.147+0000 7fa59e262740 1 -- 172.21.15.152:0/3319635933 shutdown_connections 2024-09-11T22:53:38.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.147+0000 7fa59e262740 1 -- 172.21.15.152:0/3319635933 wait complete. 2024-09-11T22:53:38.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.147+0000 7fa59e262740 1 -- start start 2024-09-11T22:53:38.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.147+0000 7fa59e262740 1 Processor -- start 2024-09-11T22:53:38.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.147+0000 7fa59e262740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c170b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.147+0000 7fa59e262740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c151b40 0x55cc9c16def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.147+0000 7fa59e262740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c15b370 0x55cc9c16e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.147+0000 7fa59e262740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941b0e50 con 0x55cc9c1505d0 2024-09-11T22:53:38.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.147+0000 7fa59e262740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941b0780 con 0x55cc9c15b370 2024-09-11T22:53:38.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.147+0000 7fa59e262740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941ad9a0 con 0x55cc9c151b40 2024-09-11T22:53:38.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa59e262740 10 client.0 _ll_register_callbacks cb 0x7fa58c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:38.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa599c25640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c170b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:38.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa599c25640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c170b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59992/0 (socket says 172.21.15.152:59992) 2024-09-11T22:53:38.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa599c25640 1 -- 172.21.15.152:0/2557744955 learned_addr learned my addr 172.21.15.152:0/2557744955 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:38.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa59a426640 1 --2- 172.21.15.152:0/2557744955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c15b370 0x55cc9c16e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:38.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa599424640 1 --2- 172.21.15.152:0/2557744955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c151b40 0x55cc9c16def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:38.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa599c25640 1 -- 172.21.15.152:0/2557744955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c151b40 msgr2=0x55cc9c16def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:38.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa599c25640 1 --2- 172.21.15.152:0/2557744955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c151b40 0x55cc9c16def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa599c25640 1 -- 172.21.15.152:0/2557744955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c15b370 msgr2=0x55cc9c16e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:38.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa599c25640 1 --2- 172.21.15.152:0/2557744955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c15b370 0x55cc9c16e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa599c25640 1 -- 172.21.15.152:0/2557744955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc9c16ed10 con 0x55cc9c1505d0 2024-09-11T22:53:38.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa599c25640 1 --2- 172.21.15.152:0/2557744955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c170b50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa590097f10 tx=0x7fa5900d5c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe03603b2767b2a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:38.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa57d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:38.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa57d7fa640 1 -- 172.21.15.152:0/2557744955 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59001d040 con 0x55cc9c1505d0 2024-09-11T22:53:38.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa57d7fa640 1 -- 172.21.15.152:0/2557744955 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa59009bf20 con 0x55cc9c1505d0 2024-09-11T22:53:38.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa59e262740 1 -- 172.21.15.152:0/2557744955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa58c04d030 con 0x55cc9c1505d0 2024-09-11T22:53:38.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.148+0000 7fa57d7fa640 1 -- 172.21.15.152:0/2557744955 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa590097d50 con 0x55cc9c1505d0 2024-09-11T22:53:38.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.149+0000 7fa59e262740 10 client.15642 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:38.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.149+0000 7fa59e262740 1 -- 172.21.15.152:0/2557744955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cc9c16ed10 con 0x55cc9c1505d0 2024-09-11T22:53:38.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.149+0000 7fa57d7fa640 1 -- 172.21.15.152:0/2557744955 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5900b4020 con 0x55cc9c1505d0 2024-09-11T22:53:38.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.149+0000 7fa57d7fa640 1 -- 172.21.15.152:0/2557744955 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa5900cd040 con 0x55cc9c1505d0 2024-09-11T22:53:38.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.149+0000 7fa59e262740 10 client.15642 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:38.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.149+0000 7fa59e262740 10 client.15642 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:53:38.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.149+0000 7fa59e262740 20 client.15642 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:38.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.149+0000 7fa59e262740 1 --2- 172.21.15.152:0/2557744955 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55cc9c1d5c20 0x55cc9c1d8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:38.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.149+0000 7fa59e262740 4 client.15642 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:53:38.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.149+0000 7fa59e262740 1 -- 172.21.15.152:0/2557744955 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa58c04d030 con 0x55cc9c1d5c20 2024-09-11T22:53:38.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.149+0000 7fa599424640 1 --2- 172.21.15.152:0/2557744955 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55cc9c1d5c20 0x55cc9c1d8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:38.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.149+0000 7fa57d7fa640 10 client.15642 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:53:38.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.150+0000 7fa599424640 1 --2- 172.21.15.152:0/2557744955 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55cc9c1d5c20 0x55cc9c1d8010 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4fee47e689ad14b7 server_cookie=b4b976e725775a57 in_seq=0 out_seq=0 2024-09-11T22:53:38.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.150+0000 7fa57d7fa640 1 -- 172.21.15.152:0/2557744955 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa58c04d030 con 0x55cc9c1d5c20 2024-09-11T22:53:38.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.150+0000 7fa57d7fa640 10 client.15642 handle_command_reply: tid=0 2024-09-11T22:53:38.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.150+0000 7fa59e262740 1 client.15642 shutdown 2024-09-11T22:53:38.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.150+0000 7fa59e262740 1 -- 172.21.15.152:0/2557744955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 msgr2=0x55cc9c170b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:38.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.150+0000 7fa59e262740 1 --2- 172.21.15.152:0/2557744955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c170b50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa590097f10 tx=0x7fa5900d5c00 comp rx=0 tx=0).stop 2024-09-11T22:53:38.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.150+0000 7fa59e262740 1 -- 172.21.15.152:0/2557744955 shutdown_connections 2024-09-11T22:53:38.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.150+0000 7fa59e262740 1 --2- 172.21.15.152:0/2557744955 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55cc9c1d5c20 0x55cc9c1d8010 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.150+0000 7fa59e262740 1 --2- 172.21.15.152:0/2557744955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cc9c15b370 0x55cc9c16e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.150+0000 7fa59e262740 1 --2- 172.21.15.152:0/2557744955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cc9c151b40 0x55cc9c16def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.150+0000 7fa59e262740 1 --2- 172.21.15.152:0/2557744955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc9c1505d0 0x55cc9c170b50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.150+0000 7fa57d7fa640 0 client.15642 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:53:38.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.150+0000 7fa59e262740 1 -- 172.21.15.152:0/2557744955 >> 172.21.15.152:0/2557744955 conn(0x55cc9c119310 msgr2=0x55cc9c14bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:38.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.150+0000 7fa59e262740 1 -- 172.21.15.152:0/2557744955 shutdown_connections 2024-09-11T22:53:38.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.150+0000 7fa59e262740 1 -- 172.21.15.152:0/2557744955 wait complete. 2024-09-11T22:53:38.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.150+0000 7fa59e262740 20 client.15642 trim_cache size 0 max 16384 2024-09-11T22:53:38.168 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:53:38.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.151+0000 7fa59be89640 1 -- 172.21.15.152:0/364002754 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa5800a2d80 msgr2=0x7fa5800a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:38.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.151+0000 7fa59be89640 1 --2- 172.21.15.152:0/364002754 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa5800a2d80 0x7fa5800a5240 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa58c030040 tx=0x7fa58c005ef0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.151+0000 7fa59be89640 1 -- 172.21.15.152:0/364002754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa594102d50 msgr2=0x7fa5941009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:38.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.151+0000 7fa59be89640 1 --2- 172.21.15.152:0/364002754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa594102d50 0x7fa5941009b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa59000c890 tx=0x7fa59000cd50 comp rx=0 tx=0).stop 2024-09-11T22:53:38.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.151+0000 7fa59be89640 1 -- 172.21.15.152:0/364002754 shutdown_connections 2024-09-11T22:53:38.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.151+0000 7fa59be89640 1 --2- 172.21.15.152:0/364002754 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa5800a2d80 0x7fa5800a5240 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.151+0000 7fa59be89640 1 --2- 172.21.15.152:0/364002754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa59410c180 0x7fa594101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.151+0000 7fa59be89640 1 --2- 172.21.15.152:0/364002754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa594103670 0x7fa594100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.151+0000 7fa59be89640 1 --2- 172.21.15.152:0/364002754 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa594102d50 0x7fa5941009b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:38.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.151+0000 7fa59be89640 1 -- 172.21.15.152:0/364002754 >> 172.21.15.152:0/364002754 conn(0x7fa5940fc7d0 msgr2=0x7fa5940fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:38.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.151+0000 7fa59be89640 1 -- 172.21.15.152:0/364002754 shutdown_connections 2024-09-11T22:53:38.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:38.151+0000 7fa59be89640 1 -- 172.21.15.152:0/364002754 wait complete. 2024-09-11T22:53:38.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:38 smithi152 ceph-mon[31181]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T22:53:38.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:38 smithi191 ceph-mon[35211]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T22:53:38.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:38 smithi204 ceph-mon[35454]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T22:53:38.813 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:53:39.066 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:39.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.577+0000 7f77294ac640 1 -- 172.21.15.152:0/2283417057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77241082a0 msgr2=0x7f772410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:39.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.577+0000 7f77294ac640 1 --2- 172.21.15.152:0/2283417057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77241082a0 0x7f772410ec40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f771c009cd0 tx=0x7f771c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.577+0000 7f77294ac640 1 -- 172.21.15.152:0/2283417057 shutdown_connections 2024-09-11T22:53:39.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.577+0000 7f77294ac640 1 --2- 172.21.15.152:0/2283417057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77241082a0 0x7f772410ec40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.577+0000 7f77294ac640 1 --2- 172.21.15.152:0/2283417057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77241036e0 0x7f7724107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.577+0000 7f77294ac640 1 --2- 172.21.15.152:0/2283417057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7724102d30 0x7f7724103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.577+0000 7f77294ac640 1 -- 172.21.15.152:0/2283417057 >> 172.21.15.152:0/2283417057 conn(0x7f77240fc7d0 msgr2=0x7f77240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:39.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.577+0000 7f77294ac640 1 -- 172.21.15.152:0/2283417057 shutdown_connections 2024-09-11T22:53:39.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.577+0000 7f77294ac640 1 -- 172.21.15.152:0/2283417057 wait complete. 2024-09-11T22:53:39.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.578+0000 7f77294ac640 1 Processor -- start 2024-09-11T22:53:39.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.578+0000 7f77294ac640 1 -- start start 2024-09-11T22:53:39.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.578+0000 7f77294ac640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7724102d30 0x7f7724100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.578+0000 7f77294ac640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77241036e0 0x7f7724100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.578+0000 7f77294ac640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77241082a0 0x7f7724101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.578+0000 7f77294ac640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f77241120a0 con 0x7f77241082a0 2024-09-11T22:53:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.578+0000 7f77294ac640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7724111f20 con 0x7f77241036e0 2024-09-11T22:53:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.578+0000 7f77294ac640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7724112220 con 0x7f7724102d30 2024-09-11T22:53:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.578+0000 7f77237fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77241082a0 0x7f7724101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.578+0000 7f77237fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77241082a0 0x7f7724101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58646/0 (socket says 172.21.15.152:58646) 2024-09-11T22:53:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.578+0000 7f77237fe640 1 -- 172.21.15.152:0/3326256561 learned_addr learned my addr 172.21.15.152:0/3326256561 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.578+0000 7f77227fc640 1 --2- 172.21.15.152:0/3326256561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77241036e0 0x7f7724100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.578+0000 7f7722ffd640 1 --2- 172.21.15.152:0/3326256561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7724102d30 0x7f7724100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.579+0000 7f77237fe640 1 -- 172.21.15.152:0/3326256561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7724102d30 msgr2=0x7f7724100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:39.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.579+0000 7f77237fe640 1 --2- 172.21.15.152:0/3326256561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7724102d30 0x7f7724100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.579+0000 7f77237fe640 1 -- 172.21.15.152:0/3326256561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77241036e0 msgr2=0x7f7724100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:39.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.579+0000 7f77237fe640 1 --2- 172.21.15.152:0/3326256561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77241036e0 0x7f7724100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.579+0000 7f77237fe640 1 -- 172.21.15.152:0/3326256561 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77241a6a10 con 0x7f77241082a0 2024-09-11T22:53:39.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.579+0000 7f77227fc640 1 --2- 172.21.15.152:0/3326256561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77241036e0 0x7f7724100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:39.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.579+0000 7f77237fe640 1 --2- 172.21.15.152:0/3326256561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77241082a0 0x7f7724101410 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f771c005cc0 tx=0x7f771c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:39.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.579+0000 7f771bfff640 1 -- 172.21.15.152:0/3326256561 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c004040 con 0x7f77241082a0 2024-09-11T22:53:39.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.579+0000 7f77294ac640 1 -- 172.21.15.152:0/3326256561 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77241a6ca0 con 0x7f77241082a0 2024-09-11T22:53:39.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.579+0000 7f771bfff640 1 -- 172.21.15.152:0/3326256561 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771c0046c0 con 0x7f77241082a0 2024-09-11T22:53:39.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.579+0000 7f771bfff640 1 -- 172.21.15.152:0/3326256561 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c046480 con 0x7f77241082a0 2024-09-11T22:53:39.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.579+0000 7f77294ac640 1 -- 172.21.15.152:0/3326256561 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77241a7230 con 0x7f77241082a0 2024-09-11T22:53:39.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.582+0000 7f771bfff640 1 -- 172.21.15.152:0/3326256561 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f771c0305a0 con 0x7f77241082a0 2024-09-11T22:53:39.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.582+0000 7f772b885740 1 Processor -- start 2024-09-11T22:53:39.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.582+0000 7f772b885740 1 -- start start 2024-09-11T22:53:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.582+0000 7f772b885740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273da6b40 0x563273da5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.582+0000 7f772b885740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273da55d0 0x563273da5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.582+0000 7f772b885740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273db0370 0x563273db2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.582+0000 7f772b885740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x563273d0b2f0 con 0x563273db0370 2024-09-11T22:53:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.582+0000 7f772b885740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563273cb2cf0 con 0x563273da55d0 2024-09-11T22:53:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.582+0000 7f772b885740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563273d072b0 con 0x563273da6b40 2024-09-11T22:53:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.582+0000 7f77227fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273db0370 0x563273db2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.582+0000 7f77227fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273db0370 0x563273db2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58652/0 (socket says 172.21.15.152:58652) 2024-09-11T22:53:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.582+0000 7f77227fc640 1 -- 172.21.15.152:0/3718295480 learned_addr learned my addr 172.21.15.152:0/3718295480 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.583+0000 7f77227fc640 1 --2- 172.21.15.152:0/3718295480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273da6b40 0x563273da5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.583+0000 7f7722ffd640 1 --2- 172.21.15.152:0/3718295480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273da55d0 0x563273da5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.583+0000 7f771bfff640 1 --2- 172.21.15.152:0/3326256561 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f77100a2d80 0x7f77100a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.583+0000 7f77227fc640 1 -- 172.21.15.152:0/3718295480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273da6b40 msgr2=0x563273da5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:39.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.583+0000 7f77227fc640 1 --2- 172.21.15.152:0/3718295480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273da6b40 0x563273da5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.583+0000 7f77227fc640 1 -- 172.21.15.152:0/3718295480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273da55d0 msgr2=0x563273da5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.583+0000 7f77227fc640 1 --2- 172.21.15.152:0/3718295480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273da55d0 0x563273da5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.583+0000 7f77227fc640 1 -- 172.21.15.152:0/3718295480 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563273c973a0 con 0x563273db0370 2024-09-11T22:53:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.583+0000 7f77237fe640 1 --2- 172.21.15.152:0/3326256561 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f77100a2d80 0x7f77100a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.583+0000 7f77227fc640 1 --2- 172.21.15.152:0/3718295480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273db0370 0x563273db2760 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7714014900 tx=0x7f771400b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca2fe3d8f3a58ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.583+0000 7f771bfff640 1 -- 172.21.15.152:0/3326256561 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f771c02e030 con 0x7f77241082a0 2024-09-11T22:53:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.583+0000 7f77197fa640 1 -- 172.21.15.152:0/3718295480 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7714004030 con 0x563273db0370 2024-09-11T22:53:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.583+0000 7f77197fa640 1 -- 172.21.15.152:0/3718295480 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7714005500 con 0x563273db0370 2024-09-11T22:53:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.583+0000 7f77197fa640 1 -- 172.21.15.152:0/3718295480 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7714004a90 con 0x563273db0370 2024-09-11T22:53:39.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.584+0000 7f772b885740 1 -- 172.21.15.152:0/3718295480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273db0370 msgr2=0x563273db2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:39.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.584+0000 7f77237fe640 1 --2- 172.21.15.152:0/3326256561 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f77100a2d80 0x7f77100a5240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f771c0ee5f0 tx=0x7f771c0b2740 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:39.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.584+0000 7f772b885740 1 --2- 172.21.15.152:0/3718295480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273db0370 0x563273db2760 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7714014900 tx=0x7f771400b420 comp rx=0 tx=0).stop 2024-09-11T22:53:39.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.584+0000 7f772b885740 1 -- 172.21.15.152:0/3718295480 shutdown_connections 2024-09-11T22:53:39.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.584+0000 7f772b885740 1 --2- 172.21.15.152:0/3718295480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273db0370 0x563273db2760 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.584+0000 7f772b885740 1 --2- 172.21.15.152:0/3718295480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273da55d0 0x563273da5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.584+0000 7f772b885740 1 --2- 172.21.15.152:0/3718295480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273da6b40 0x563273da5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.584+0000 7f772b885740 1 -- 172.21.15.152:0/3718295480 >> 172.21.15.152:0/3718295480 conn(0x563273d6e310 msgr2=0x563273da1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.584+0000 7f772b885740 1 -- 172.21.15.152:0/3718295480 shutdown_connections 2024-09-11T22:53:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.584+0000 7f772b885740 1 -- 172.21.15.152:0/3718295480 wait complete. 2024-09-11T22:53:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.585+0000 7f772b885740 1 -- start start 2024-09-11T22:53:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.585+0000 7f772b885740 1 Processor -- start 2024-09-11T22:53:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.585+0000 7f772b885740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273da55d0 0x563273dafd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.585+0000 7f772b885740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273da6b40 0x563273dacfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.585+0000 7f772b885740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273db0370 0x563273dad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.585+0000 7f772b885740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x563273db3070 con 0x563273da6b40 2024-09-11T22:53:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.585+0000 7f772b885740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563273cc6060 con 0x563273da55d0 2024-09-11T22:53:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.585+0000 7f772b885740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563273db31f0 con 0x563273db0370 2024-09-11T22:53:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.585+0000 7f772b885740 10 client.0 _ll_register_callbacks cb 0x563273a198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:39.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.585+0000 7f7722ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273da6b40 0x563273dacfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.585+0000 7f7722ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273da6b40 0x563273dacfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58658/0 (socket says 172.21.15.152:58658) 2024-09-11T22:53:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.585+0000 7f7722ffd640 1 -- 172.21.15.152:0/342125153 learned_addr learned my addr 172.21.15.152:0/342125153 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.585+0000 7f77227fc640 1 --2- 172.21.15.152:0/342125153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273da55d0 0x563273dafd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.586+0000 7f7722ffd640 1 -- 172.21.15.152:0/342125153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273db0370 msgr2=0x563273dad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.586+0000 7f77227fc640 1 --2- 172.21.15.152:0/342125153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273db0370 0x563273dad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.586+0000 7f7722ffd640 1 --2- 172.21.15.152:0/342125153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273db0370 0x563273dad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.586+0000 7f7722ffd640 1 -- 172.21.15.152:0/342125153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273da55d0 msgr2=0x563273dafd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.586+0000 7f7722ffd640 1 --2- 172.21.15.152:0/342125153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273da55d0 0x563273dafd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.586+0000 7f7722ffd640 1 -- 172.21.15.152:0/342125153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563273a7c3c0 con 0x563273da6b40 2024-09-11T22:53:39.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.586+0000 7f77227fc640 1 --2- 172.21.15.152:0/342125153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273db0370 0x563273dad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:39.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.586+0000 7f7722ffd640 1 --2- 172.21.15.152:0/342125153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273da6b40 0x563273dacfb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f770c00df20 tx=0x7f770c011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d7d7d739da717c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:39.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.586+0000 7f770a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:39.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.586+0000 7f770a7fc640 1 -- 172.21.15.152:0/342125153 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c004230 con 0x563273da6b40 2024-09-11T22:53:39.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.586+0000 7f770a7fc640 1 -- 172.21.15.152:0/342125153 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f770c0043d0 con 0x563273da6b40 2024-09-11T22:53:39.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.586+0000 7f772b885740 1 -- 172.21.15.152:0/342125153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7714005500 con 0x563273da6b40 2024-09-11T22:53:39.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.586+0000 7f770a7fc640 1 -- 172.21.15.152:0/342125153 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f770c0035f0 con 0x563273da6b40 2024-09-11T22:53:39.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.586+0000 7f772b885740 10 client.15654 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:39.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.586+0000 7f772b885740 1 -- 172.21.15.152:0/342125153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563273dbeb60 con 0x563273da6b40 2024-09-11T22:53:39.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.586+0000 7f770a7fc640 1 -- 172.21.15.152:0/342125153 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7714005500 con 0x563273da6b40 2024-09-11T22:53:39.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.587+0000 7f770a7fc640 1 -- 172.21.15.152:0/342125153 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f770c012a00 con 0x563273da6b40 2024-09-11T22:53:39.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.587+0000 7f772b885740 10 client.15654 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:39.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.587+0000 7f772b885740 10 client.15654 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:53:39.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.587+0000 7f772b885740 20 client.15654 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:39.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.587+0000 7f772b885740 1 --2- 172.21.15.152:0/342125153 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x563273dbffc0 0x563273dc23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.587+0000 7f772b885740 4 client.15654 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.587+0000 7f772b885740 1 -- 172.21.15.152:0/342125153 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563273d18860 con 0x563273dbffc0 2024-09-11T22:53:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.587+0000 7f77227fc640 1 --2- 172.21.15.152:0/342125153 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x563273dbffc0 0x563273dc23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.588+0000 7f77227fc640 1 --2- 172.21.15.152:0/342125153 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x563273dbffc0 0x563273dc23e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8527986ac072e269 server_cookie=69bc48d169b1f5c3 in_seq=0 out_seq=0 2024-09-11T22:53:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.588+0000 7f770a7fc640 10 client.15654 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:53:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.588+0000 7f770a7fc640 1 -- 172.21.15.152:0/342125153 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563273d18860 con 0x563273dbffc0 2024-09-11T22:53:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.588+0000 7f770a7fc640 10 client.15654 handle_command_reply: tid=0 2024-09-11T22:53:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.588+0000 7f772b885740 1 client.15654 shutdown 2024-09-11T22:53:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.588+0000 7f772b885740 1 -- 172.21.15.152:0/342125153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273da6b40 msgr2=0x563273dacfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.588+0000 7f772b885740 1 --2- 172.21.15.152:0/342125153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273da6b40 0x563273dacfb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f770c00df20 tx=0x7f770c011870 comp rx=0 tx=0).stop 2024-09-11T22:53:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.589+0000 7f772b885740 1 -- 172.21.15.152:0/342125153 shutdown_connections 2024-09-11T22:53:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.589+0000 7f772b885740 1 --2- 172.21.15.152:0/342125153 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x563273dbffc0 0x563273dc23e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.589+0000 7f772b885740 1 --2- 172.21.15.152:0/342125153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273db0370 0x563273dad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.589+0000 7f772b885740 1 --2- 172.21.15.152:0/342125153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273da6b40 0x563273dacfb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.589+0000 7f770a7fc640 0 client.15654 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:53:39.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.589+0000 7f772b885740 1 --2- 172.21.15.152:0/342125153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273da55d0 0x563273dafd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.589+0000 7f772b885740 1 -- 172.21.15.152:0/342125153 >> 172.21.15.152:0/342125153 conn(0x563273d6e310 msgr2=0x563273da7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:39.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.589+0000 7f772b885740 1 -- 172.21.15.152:0/342125153 shutdown_connections 2024-09-11T22:53:39.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.589+0000 7f772b885740 1 -- 172.21.15.152:0/342125153 wait complete. 2024-09-11T22:53:39.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.589+0000 7f772b885740 20 client.15654 trim_cache size 0 max 16384 2024-09-11T22:53:39.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.602+0000 7f772b885740 1 Processor -- start 2024-09-11T22:53:39.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.602+0000 7f772b885740 1 -- start start 2024-09-11T22:53:39.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.602+0000 7f772b885740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273da55d0 0x563273dbc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.602+0000 7f772b885740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273da6b40 0x563273dc7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.602+0000 7f772b885740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273db0370 0x563273dc81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.602+0000 7f772b885740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f77241af990 con 0x563273db0370 2024-09-11T22:53:39.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.602+0000 7f772b885740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f77241b14d0 con 0x563273da55d0 2024-09-11T22:53:39.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.602+0000 7f772b885740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f77241b0060 con 0x563273da6b40 2024-09-11T22:53:39.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.603+0000 7f77227fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273db0370 0x563273dc81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.603+0000 7f77227fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273db0370 0x563273dc81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58660/0 (socket says 172.21.15.152:58660) 2024-09-11T22:53:39.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.603+0000 7f77227fc640 1 -- 172.21.15.152:0/4192702826 learned_addr learned my addr 172.21.15.152:0/4192702826 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:39.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.603+0000 7f77227fc640 1 --2- 172.21.15.152:0/4192702826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273da55d0 0x563273dbc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.603+0000 7f7722ffd640 1 --2- 172.21.15.152:0/4192702826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273da6b40 0x563273dc7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.603+0000 7f77227fc640 1 -- 172.21.15.152:0/4192702826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273da6b40 msgr2=0x563273dc7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:39.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.603+0000 7f77227fc640 1 --2- 172.21.15.152:0/4192702826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273da6b40 0x563273dc7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.603+0000 7f77227fc640 1 -- 172.21.15.152:0/4192702826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273da55d0 msgr2=0x563273dbc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:39.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.603+0000 7f77227fc640 1 --2- 172.21.15.152:0/4192702826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273da55d0 0x563273dbc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.603+0000 7f77227fc640 1 -- 172.21.15.152:0/4192702826 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563273da2960 con 0x563273db0370 2024-09-11T22:53:39.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.603+0000 7f77227fc640 1 --2- 172.21.15.152:0/4192702826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273db0370 0x563273dc81f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f771400ba40 tx=0x7f7714021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd36ab836b9551e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:39.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.603+0000 7f7718ff9640 1 -- 172.21.15.152:0/4192702826 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771400f7b0 con 0x563273db0370 2024-09-11T22:53:39.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.603+0000 7f7718ff9640 1 -- 172.21.15.152:0/4192702826 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771400fe30 con 0x563273db0370 2024-09-11T22:53:39.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.603+0000 7f7718ff9640 1 -- 172.21.15.152:0/4192702826 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771402bcb0 con 0x563273db0370 2024-09-11T22:53:39.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.604+0000 7f772b885740 1 -- 172.21.15.152:0/4192702826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273db0370 msgr2=0x563273dc81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:39.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.604+0000 7f772b885740 1 --2- 172.21.15.152:0/4192702826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273db0370 0x563273dc81f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f771400ba40 tx=0x7f7714021040 comp rx=0 tx=0).stop 2024-09-11T22:53:39.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.604+0000 7f772b885740 1 -- 172.21.15.152:0/4192702826 shutdown_connections 2024-09-11T22:53:39.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.604+0000 7f772b885740 1 --2- 172.21.15.152:0/4192702826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273db0370 0x563273dc81f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.604+0000 7f772b885740 1 --2- 172.21.15.152:0/4192702826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273da6b40 0x563273dc7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.604+0000 7f772b885740 1 --2- 172.21.15.152:0/4192702826 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273da55d0 0x563273dbc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.604+0000 7f772b885740 1 -- 172.21.15.152:0/4192702826 >> 172.21.15.152:0/4192702826 conn(0x563273d6e310 msgr2=0x563273daf030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:39.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.604+0000 7f772b885740 1 -- 172.21.15.152:0/4192702826 shutdown_connections 2024-09-11T22:53:39.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.604+0000 7f772b885740 1 -- 172.21.15.152:0/4192702826 wait complete. 2024-09-11T22:53:39.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f772b885740 1 -- start start 2024-09-11T22:53:39.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f772b885740 1 Processor -- start 2024-09-11T22:53:39.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f772b885740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273da55d0 0x563273db8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f772b885740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273da6b40 0x563273db4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f772b885740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273db0370 0x563273db4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f772b885740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f77241b0e00 con 0x563273da6b40 2024-09-11T22:53:39.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f772b885740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f77241b0730 con 0x563273db0370 2024-09-11T22:53:39.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f772b885740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f77241ad950 con 0x563273da55d0 2024-09-11T22:53:39.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f772b885740 10 client.0 _ll_register_callbacks cb 0x7f770c00d440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:39.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f7722ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273da6b40 0x563273db4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f7722ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273da6b40 0x563273db4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58664/0 (socket says 172.21.15.152:58664) 2024-09-11T22:53:39.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f7722ffd640 1 -- 172.21.15.152:0/3794916865 learned_addr learned my addr 172.21.15.152:0/3794916865 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:39.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f77227fc640 1 --2- 172.21.15.152:0/3794916865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273da55d0 0x563273db8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f77227fc640 1 --2- 172.21.15.152:0/3794916865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273db0370 0x563273db4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f7722ffd640 1 -- 172.21.15.152:0/3794916865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273da55d0 msgr2=0x563273db8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:39.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f7722ffd640 1 --2- 172.21.15.152:0/3794916865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273da55d0 0x563273db8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f7722ffd640 1 -- 172.21.15.152:0/3794916865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273db0370 msgr2=0x563273db4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:39.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f7722ffd640 1 --2- 172.21.15.152:0/3794916865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273db0370 0x563273db4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.605+0000 7f7722ffd640 1 -- 172.21.15.152:0/3794916865 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563273db4e60 con 0x563273da6b40 2024-09-11T22:53:39.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.606+0000 7f77227fc640 1 --2- 172.21.15.152:0/3794916865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273db0370 0x563273db4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:39.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.606+0000 7f7722ffd640 1 --2- 172.21.15.152:0/3794916865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273da6b40 0x563273db4040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f770c012dd0 tx=0x7f770c025b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5251e482339bee9d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:39.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.606+0000 7f770a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:39.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.606+0000 7f770a7fc640 1 -- 172.21.15.152:0/3794916865 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c02b040 con 0x563273da6b40 2024-09-11T22:53:39.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.606+0000 7f770a7fc640 1 -- 172.21.15.152:0/3794916865 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f770c005030 con 0x563273da6b40 2024-09-11T22:53:39.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.606+0000 7f772b885740 1 -- 172.21.15.152:0/3794916865 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f771400fe30 con 0x563273da6b40 2024-09-11T22:53:39.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.606+0000 7f770a7fc640 1 -- 172.21.15.152:0/3794916865 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f770c011d70 con 0x563273da6b40 2024-09-11T22:53:39.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.606+0000 7f772b885740 10 client.15660 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:39.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.606+0000 7f772b885740 1 -- 172.21.15.152:0/3794916865 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563273e2ad60 con 0x563273da6b40 2024-09-11T22:53:39.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.606+0000 7f770a7fc640 1 -- 172.21.15.152:0/3794916865 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f771400fe30 con 0x563273da6b40 2024-09-11T22:53:39.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.606+0000 7f770a7fc640 1 -- 172.21.15.152:0/3794916865 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f770c02e070 con 0x563273da6b40 2024-09-11T22:53:39.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.607+0000 7f772b885740 10 client.15660 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:39.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.607+0000 7f772b885740 10 client.15660 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:53:39.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.607+0000 7f772b885740 20 client.15660 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:39.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.607+0000 7f772b885740 1 --2- 172.21.15.152:0/3794916865 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x563273dbffc0 0x563273e2d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:39.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.607+0000 7f772b885740 4 client.15660 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:53:39.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.607+0000 7f772b885740 1 -- 172.21.15.152:0/3794916865 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563273d12d70 con 0x563273dbffc0 2024-09-11T22:53:39.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.607+0000 7f77227fc640 1 --2- 172.21.15.152:0/3794916865 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x563273dbffc0 0x563273e2d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:39.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.607+0000 7f770a7fc640 10 client.15660 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:53:39.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.607+0000 7f77227fc640 1 --2- 172.21.15.152:0/3794916865 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x563273dbffc0 0x563273e2d5a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f4510c3698b137ff server_cookie=80f91c47c17d1301 in_seq=0 out_seq=0 2024-09-11T22:53:39.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.607+0000 7f770a7fc640 1 -- 172.21.15.152:0/3794916865 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563273d12d70 con 0x563273dbffc0 2024-09-11T22:53:39.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.607+0000 7f770a7fc640 10 client.15660 handle_command_reply: tid=0 2024-09-11T22:53:39.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f772b885740 1 client.15660 shutdown 2024-09-11T22:53:39.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f772b885740 1 -- 172.21.15.152:0/3794916865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273da6b40 msgr2=0x563273db4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:39.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f772b885740 1 --2- 172.21.15.152:0/3794916865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273da6b40 0x563273db4040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f770c012dd0 tx=0x7f770c025b00 comp rx=0 tx=0).stop 2024-09-11T22:53:39.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f772b885740 1 -- 172.21.15.152:0/3794916865 shutdown_connections 2024-09-11T22:53:39.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f772b885740 1 --2- 172.21.15.152:0/3794916865 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x563273dbffc0 0x563273e2d5a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f772b885740 1 --2- 172.21.15.152:0/3794916865 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563273db0370 0x563273db4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f772b885740 1 --2- 172.21.15.152:0/3794916865 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563273da6b40 0x563273db4040 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f770a7fc640 0 client.15660 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:53:39.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f772b885740 1 --2- 172.21.15.152:0/3794916865 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563273da55d0 0x563273db8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f772b885740 1 -- 172.21.15.152:0/3794916865 >> 172.21.15.152:0/3794916865 conn(0x563273d6e310 msgr2=0x563273dac870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:39.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f772b885740 1 -- 172.21.15.152:0/3794916865 shutdown_connections 2024-09-11T22:53:39.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f772b885740 1 -- 172.21.15.152:0/3794916865 wait complete. 2024-09-11T22:53:39.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f772b885740 20 client.15660 trim_cache size 0 max 16384 2024-09-11T22:53:39.615 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:53:39.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f77294ac640 1 -- 172.21.15.152:0/3326256561 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f77100a2d80 msgr2=0x7f77100a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:39.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f77294ac640 1 --2- 172.21.15.152:0/3326256561 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f77100a2d80 0x7f77100a5240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f771c0ee5f0 tx=0x7f771c0b2740 comp rx=0 tx=0).stop 2024-09-11T22:53:39.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f77294ac640 1 -- 172.21.15.152:0/3326256561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77241082a0 msgr2=0x7f7724101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:39.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.608+0000 7f77294ac640 1 --2- 172.21.15.152:0/3326256561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77241082a0 0x7f7724101410 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f771c005cc0 tx=0x7f771c003970 comp rx=0 tx=0).stop 2024-09-11T22:53:39.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.609+0000 7f77294ac640 1 -- 172.21.15.152:0/3326256561 shutdown_connections 2024-09-11T22:53:39.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.609+0000 7f77294ac640 1 --2- 172.21.15.152:0/3326256561 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f77100a2d80 0x7f77100a5240 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.609+0000 7f77294ac640 1 --2- 172.21.15.152:0/3326256561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77241082a0 0x7f7724101410 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.609+0000 7f77294ac640 1 --2- 172.21.15.152:0/3326256561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f77241036e0 0x7f7724100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.609+0000 7f77294ac640 1 --2- 172.21.15.152:0/3326256561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7724102d30 0x7f7724100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:39.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.609+0000 7f77294ac640 1 -- 172.21.15.152:0/3326256561 >> 172.21.15.152:0/3326256561 conn(0x7f77240fc7d0 msgr2=0x7f7724105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:39.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.609+0000 7f77294ac640 1 -- 172.21.15.152:0/3326256561 shutdown_connections 2024-09-11T22:53:39.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:39.609+0000 7f77294ac640 1 -- 172.21.15.152:0/3326256561 wait complete. 2024-09-11T22:53:39.983 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-11T22:53:39.984 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:53:40.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:40 smithi152 ceph-mon[31181]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T22:53:40.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:40 smithi191 ceph-mon[35211]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T22:53:40.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:40 smithi204 ceph-mon[35454]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T22:53:42.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:42 smithi191 ceph-mon[35211]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T22:53:42.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:42 smithi204 ceph-mon[35454]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T22:53:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.591+0000 7f92e8f66640 1 -- 172.21.15.152:0/991833186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92e41009b0 msgr2=0x7f92e4100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.591+0000 7f92e8f66640 1 --2- 172.21.15.152:0/991833186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92e41009b0 0x7f92e4100e10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009930 tx=0x7f92d802a420 comp rx=0 tx=0).stop 2024-09-11T22:53:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.591+0000 7f92e8f66640 1 -- 172.21.15.152:0/991833186 shutdown_connections 2024-09-11T22:53:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.591+0000 7f92e8f66640 1 --2- 172.21.15.152:0/991833186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92e4101350 0x7f92e410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.591+0000 7f92e8f66640 1 --2- 172.21.15.152:0/991833186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92e41009b0 0x7f92e4100e10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.591+0000 7f92e8f66640 1 --2- 172.21.15.152:0/991833186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92e41069c0 0x7f92e4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.591+0000 7f92e8f66640 1 -- 172.21.15.152:0/991833186 >> 172.21.15.152:0/991833186 conn(0x7f92e4077990 msgr2=0x7f92e40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.591+0000 7f92e8f66640 1 -- 172.21.15.152:0/991833186 shutdown_connections 2024-09-11T22:53:42.593 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.591+0000 7f92e8f66640 1 -- 172.21.15.152:0/991833186 wait complete. 2024-09-11T22:53:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.591+0000 7f92e8f66640 1 Processor -- start 2024-09-11T22:53:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e8f66640 1 -- start start 2024-09-11T22:53:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e8f66640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92e41009b0 0x7f92e41991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e8f66640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92e4101350 0x7f92e41996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e8f66640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92e41069c0 0x7f92e41a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e8f66640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f92e4110e10 con 0x7f92e41009b0 2024-09-11T22:53:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e8f66640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f92e4110c90 con 0x7f92e4101350 2024-09-11T22:53:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e8f66640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f92e4110f90 con 0x7f92e41069c0 2024-09-11T22:53:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92e41009b0 0x7f92e41991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92e41009b0 0x7f92e41991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58668/0 (socket says 172.21.15.152:58668) 2024-09-11T22:53:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e259c640 1 -- 172.21.15.152:0/3205724538 learned_addr learned my addr 172.21.15.152:0/3205724538 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e2d9d640 1 --2- 172.21.15.152:0/3205724538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92e41069c0 0x7f92e41a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e1d9b640 1 --2- 172.21.15.152:0/3205724538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92e4101350 0x7f92e41996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e259c640 1 -- 172.21.15.152:0/3205724538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92e41069c0 msgr2=0x7f92e41a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e259c640 1 --2- 172.21.15.152:0/3205724538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92e41069c0 0x7f92e41a4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e259c640 1 -- 172.21.15.152:0/3205724538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92e4101350 msgr2=0x7f92e41996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e259c640 1 --2- 172.21.15.152:0/3205724538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92e4101350 0x7f92e41996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e259c640 1 -- 172.21.15.152:0/3205724538 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e41a4770 con 0x7f92e41009b0 2024-09-11T22:53:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e1d9b640 1 --2- 172.21.15.152:0/3205724538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92e4101350 0x7f92e41996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e259c640 1 --2- 172.21.15.152:0/3205724538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92e41009b0 0x7f92e41991b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f92d400d950 tx=0x7f92d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92d37fe640 1 -- 172.21.15.152:0/3205724538 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4017070 con 0x7f92e41009b0 2024-09-11T22:53:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92d37fe640 1 -- 172.21.15.152:0/3205724538 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d4009c30 con 0x7f92e41009b0 2024-09-11T22:53:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e8f66640 1 -- 172.21.15.152:0/3205724538 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e41a4940 con 0x7f92e41009b0 2024-09-11T22:53:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92d37fe640 1 -- 172.21.15.152:0/3205724538 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d400f6b0 con 0x7f92e41009b0 2024-09-11T22:53:42.597 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.592+0000 7f92e8f66640 1 -- 172.21.15.152:0/3205724538 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e41a4e60 con 0x7f92e41009b0 2024-09-11T22:53:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.595+0000 7f92d37fe640 1 -- 172.21.15.152:0/3205724538 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f92d40109d0 con 0x7f92e41009b0 2024-09-11T22:53:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.595+0000 7f92ebddd740 1 Processor -- start 2024-09-11T22:53:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.595+0000 7f92ebddd740 1 -- start start 2024-09-11T22:53:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.595+0000 7f92ebddd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 0x55fa446f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.595+0000 7f92ebddd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa446f6410 0x55fa44700770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.595+0000 7f92ebddd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa44700cb0 0x55fa447030a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.595+0000 7f92ebddd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55fa4465a2f0 con 0x55fa446f6410 2024-09-11T22:53:42.606 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.595+0000 7f92ebddd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55fa44602270 con 0x55fa44700cb0 2024-09-11T22:53:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.595+0000 7f92ebddd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fa446562b0 con 0x55fa446f5af0 2024-09-11T22:53:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.596+0000 7f92e259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa446f6410 0x55fa44700770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.596+0000 7f92e2d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 0x55fa446f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.596+0000 7f92e2d9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 0x55fa446f5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:44088/0 (socket says 172.21.15.152:44088) 2024-09-11T22:53:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.596+0000 7f92e259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa446f6410 0x55fa44700770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58676/0 (socket says 172.21.15.152:58676) 2024-09-11T22:53:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.596+0000 7f92e2d9d640 1 -- 172.21.15.152:0/1461176552 learned_addr learned my addr 172.21.15.152:0/1461176552 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.596+0000 7f92e1d9b640 1 --2- 172.21.15.152:0/1461176552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa44700cb0 0x55fa447030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.596+0000 7f92d37fe640 1 --2- 172.21.15.152:0/3205724538 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f92cc0a2e40 0x7f92cc0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.596+0000 7f92e2d9d640 1 -- 172.21.15.152:0/1461176552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa44700cb0 msgr2=0x55fa447030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.596+0000 7f92e2d9d640 1 --2- 172.21.15.152:0/1461176552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa44700cb0 0x55fa447030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.596+0000 7f92e2d9d640 1 -- 172.21.15.152:0/1461176552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa446f6410 msgr2=0x55fa44700770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.596+0000 7f92e2d9d640 1 --2- 172.21.15.152:0/1461176552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa446f6410 0x55fa44700770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.596+0000 7f92e2d9d640 1 -- 172.21.15.152:0/1461176552 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa44601340 con 0x55fa446f5af0 2024-09-11T22:53:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.596+0000 7f92e259c640 1 --2- 172.21.15.152:0/1461176552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa446f6410 0x55fa44700770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.596+0000 7f92d37fe640 1 -- 172.21.15.152:0/3205724538 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f92d40cf1f0 con 0x7f92e41009b0 2024-09-11T22:53:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.596+0000 7f92e2d9d640 1 --2- 172.21.15.152:0/3205724538 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f92cc0a2e40 0x7f92cc0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.596+0000 7f92e2d9d640 1 --2- 172.21.15.152:0/1461176552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 0x55fa446f5ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009ee0 tx=0x7f92dc010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd32acbb3f0d0206 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.597+0000 7f92d0ff9640 1 -- 172.21.15.152:0/1461176552 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc021070 con 0x55fa446f5af0 2024-09-11T22:53:42.608 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.597+0000 7f92d0ff9640 1 -- 172.21.15.152:0/1461176552 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92dc0113e0 con 0x55fa446f5af0 2024-09-11T22:53:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.597+0000 7f92d0ff9640 1 -- 172.21.15.152:0/1461176552 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc016070 con 0x55fa446f5af0 2024-09-11T22:53:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.597+0000 7f92ebddd740 1 -- 172.21.15.152:0/1461176552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 msgr2=0x55fa446f5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.597+0000 7f92e2d9d640 1 --2- 172.21.15.152:0/3205724538 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f92cc0a2e40 0x7f92cc0a5300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f92dc01e000 tx=0x7f92dc021000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.597+0000 7f92ebddd740 1 --2- 172.21.15.152:0/1461176552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 0x55fa446f5ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009ee0 tx=0x7f92dc010740 comp rx=0 tx=0).stop 2024-09-11T22:53:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92ebddd740 1 -- 172.21.15.152:0/1461176552 shutdown_connections 2024-09-11T22:53:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92ebddd740 1 --2- 172.21.15.152:0/1461176552 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa44700cb0 0x55fa447030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92ebddd740 1 --2- 172.21.15.152:0/1461176552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa446f6410 0x55fa44700770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92ebddd740 1 --2- 172.21.15.152:0/1461176552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 0x55fa446f5ed0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92ebddd740 1 -- 172.21.15.152:0/1461176552 >> 172.21.15.152:0/1461176552 conn(0x55fa446f2060 msgr2=0x55fa446f2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92ebddd740 1 -- 172.21.15.152:0/1461176552 shutdown_connections 2024-09-11T22:53:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92ebddd740 1 -- 172.21.15.152:0/1461176552 wait complete. 2024-09-11T22:53:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92ebddd740 1 -- start start 2024-09-11T22:53:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92ebddd740 1 Processor -- start 2024-09-11T22:53:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92ebddd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 0x55fa446fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92ebddd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa446f6410 0x55fa446fe100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92ebddd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa44700cb0 0x55fa446fe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92ebddd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55fa44707c90 con 0x55fa446f6410 2024-09-11T22:53:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92ebddd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55fa445e3290 con 0x55fa44700cb0 2024-09-11T22:53:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92ebddd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fa44707e10 con 0x55fa446f5af0 2024-09-11T22:53:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92ebddd740 10 client.0 _ll_register_callbacks cb 0x55fa44614140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.598+0000 7f92e1d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa446f6410 0x55fa446fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92e1d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa446f6410 0x55fa446fe100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58684/0 (socket says 172.21.15.152:58684) 2024-09-11T22:53:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92e1d9b640 1 -- 172.21.15.152:0/2342844020 learned_addr learned my addr 172.21.15.152:0/2342844020 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92e259c640 1 --2- 172.21.15.152:0/2342844020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 0x55fa446fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92e2d9d640 1 --2- 172.21.15.152:0/2342844020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa44700cb0 0x55fa446fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.610 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92e1d9b640 1 -- 172.21.15.152:0/2342844020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 msgr2=0x55fa446fdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92e1d9b640 1 --2- 172.21.15.152:0/2342844020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 0x55fa446fdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92e1d9b640 1 -- 172.21.15.152:0/2342844020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa44700cb0 msgr2=0x55fa446fe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92e1d9b640 1 --2- 172.21.15.152:0/2342844020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa44700cb0 0x55fa446fe640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92e1d9b640 1 -- 172.21.15.152:0/2342844020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa4436d210 con 0x55fa446f6410 2024-09-11T22:53:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92e2d9d640 1 --2- 172.21.15.152:0/2342844020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa44700cb0 0x55fa446fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92e1d9b640 1 --2- 172.21.15.152:0/2342844020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa446f6410 0x55fa446fe100 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f92d8000c00 tx=0x7f92d8004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736813cc9e391d3c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92c9ffb640 1 -- 172.21.15.152:0/2342844020 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d8035040 con 0x55fa446f6410 2024-09-11T22:53:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92c9ffb640 1 -- 172.21.15.152:0/2342844020 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f92d8005450 con 0x55fa446f6410 2024-09-11T22:53:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92ebddd740 1 -- 172.21.15.152:0/2342844020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92dc0113e0 con 0x55fa446f6410 2024-09-11T22:53:42.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92c9ffb640 1 -- 172.21.15.152:0/2342844020 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f92d8005650 con 0x55fa446f6410 2024-09-11T22:53:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92ebddd740 10 client.15672 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.599+0000 7f92ebddd740 1 -- 172.21.15.152:0/2342844020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa44439620 con 0x55fa446f6410 2024-09-11T22:53:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.600+0000 7f92c9ffb640 1 -- 172.21.15.152:0/2342844020 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92dc0113e0 con 0x55fa446f6410 2024-09-11T22:53:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.600+0000 7f92c9ffb640 1 -- 172.21.15.152:0/2342844020 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f92d8003b10 con 0x55fa446f6410 2024-09-11T22:53:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.600+0000 7f92ebddd740 10 client.15672 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.600+0000 7f92ebddd740 10 client.15672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.600+0000 7f92ebddd740 20 client.15672 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.600+0000 7f92ebddd740 1 --2- 172.21.15.152:0/2342844020 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55fa44714fe0 0x55fa447173d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.600+0000 7f92ebddd740 4 client.15672 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.600+0000 7f92ebddd740 1 -- 172.21.15.152:0/2342844020 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa44667a20 con 0x55fa44714fe0 2024-09-11T22:53:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.600+0000 7f92e259c640 1 --2- 172.21.15.152:0/2342844020 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55fa44714fe0 0x55fa447173d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.601+0000 7f92e259c640 1 --2- 172.21.15.152:0/2342844020 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55fa44714fe0 0x55fa447173d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c239122d58cf9fb8 server_cookie=97488dc0db852383 in_seq=0 out_seq=0 2024-09-11T22:53:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.601+0000 7f92c9ffb640 10 client.15672 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.601+0000 7f92c9ffb640 1 -- 172.21.15.152:0/2342844020 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa44667a20 con 0x55fa44714fe0 2024-09-11T22:53:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.601+0000 7f92c9ffb640 10 client.15672 handle_command_reply: tid=0 2024-09-11T22:53:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.601+0000 7f92ebddd740 1 client.15672 shutdown 2024-09-11T22:53:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.602+0000 7f92ebddd740 1 -- 172.21.15.152:0/2342844020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa446f6410 msgr2=0x55fa446fe100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.602+0000 7f92ebddd740 1 --2- 172.21.15.152:0/2342844020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa446f6410 0x55fa446fe100 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f92d8000c00 tx=0x7f92d8004440 comp rx=0 tx=0).stop 2024-09-11T22:53:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.602+0000 7f92ebddd740 1 -- 172.21.15.152:0/2342844020 shutdown_connections 2024-09-11T22:53:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.602+0000 7f92ebddd740 1 --2- 172.21.15.152:0/2342844020 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55fa44714fe0 0x55fa447173d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.602+0000 7f92ebddd740 1 --2- 172.21.15.152:0/2342844020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa44700cb0 0x55fa446fe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.602+0000 7f92ebddd740 1 --2- 172.21.15.152:0/2342844020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa446f6410 0x55fa446fe100 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.602+0000 7f92ebddd740 1 --2- 172.21.15.152:0/2342844020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 0x55fa446fdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.602+0000 7f92ebddd740 1 -- 172.21.15.152:0/2342844020 >> 172.21.15.152:0/2342844020 conn(0x55fa446f2060 msgr2=0x55fa446f7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.602+0000 7f92c9ffb640 0 client.15672 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.602+0000 7f92ebddd740 1 -- 172.21.15.152:0/2342844020 shutdown_connections 2024-09-11T22:53:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.602+0000 7f92ebddd740 1 -- 172.21.15.152:0/2342844020 wait complete. 2024-09-11T22:53:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.602+0000 7f92ebddd740 20 client.15672 trim_cache size 0 max 16384 2024-09-11T22:53:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92ebddd740 1 Processor -- start 2024-09-11T22:53:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92ebddd740 1 -- start start 2024-09-11T22:53:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92ebddd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 0x55fa446f8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92ebddd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa446f6410 0x55fa446ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92ebddd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa44700cb0 0x55fa4471cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92ebddd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f92e41ab5d0 con 0x55fa44700cb0 2024-09-11T22:53:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92ebddd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f92e41ad110 con 0x55fa446f6410 2024-09-11T22:53:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92ebddd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f92e41abca0 con 0x55fa446f5af0 2024-09-11T22:53:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92e2d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa44700cb0 0x55fa4471cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92e2d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa44700cb0 0x55fa4471cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58686/0 (socket says 172.21.15.152:58686) 2024-09-11T22:53:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92e2d9d640 1 -- 172.21.15.152:0/538432226 learned_addr learned my addr 172.21.15.152:0/538432226 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92e259c640 1 --2- 172.21.15.152:0/538432226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 0x55fa446f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92e1d9b640 1 --2- 172.21.15.152:0/538432226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa446f6410 0x55fa446ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92e2d9d640 1 -- 172.21.15.152:0/538432226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 msgr2=0x55fa446f8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92e2d9d640 1 --2- 172.21.15.152:0/538432226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 0x55fa446f8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92e2d9d640 1 -- 172.21.15.152:0/538432226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa446f6410 msgr2=0x55fa446ffb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92e2d9d640 1 --2- 172.21.15.152:0/538432226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa446f6410 0x55fa446ffb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.616+0000 7f92e2d9d640 1 -- 172.21.15.152:0/538432226 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa446ed2d0 con 0x55fa44700cb0 2024-09-11T22:53:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.617+0000 7f92e2d9d640 1 --2- 172.21.15.152:0/538432226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa44700cb0 0x55fa4471cce0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f92dc006010 tx=0x7f92dc024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02088de0f74f29e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.617+0000 7f92cbfff640 1 -- 172.21.15.152:0/538432226 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc016040 con 0x55fa44700cb0 2024-09-11T22:53:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.617+0000 7f92cbfff640 1 -- 172.21.15.152:0/538432226 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92dc012910 con 0x55fa44700cb0 2024-09-11T22:53:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.617+0000 7f92cbfff640 1 -- 172.21.15.152:0/538432226 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc005370 con 0x55fa44700cb0 2024-09-11T22:53:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 -- 172.21.15.152:0/538432226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa44700cb0 msgr2=0x55fa4471cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 --2- 172.21.15.152:0/538432226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa44700cb0 0x55fa4471cce0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f92dc006010 tx=0x7f92dc024000 comp rx=0 tx=0).stop 2024-09-11T22:53:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 -- 172.21.15.152:0/538432226 shutdown_connections 2024-09-11T22:53:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 --2- 172.21.15.152:0/538432226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa44700cb0 0x55fa4471cce0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 --2- 172.21.15.152:0/538432226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa446f6410 0x55fa446ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 --2- 172.21.15.152:0/538432226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f5af0 0x55fa446f8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 -- 172.21.15.152:0/538432226 >> 172.21.15.152:0/538432226 conn(0x55fa446f2060 msgr2=0x55fa446ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 -- 172.21.15.152:0/538432226 shutdown_connections 2024-09-11T22:53:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 -- 172.21.15.152:0/538432226 wait complete. 2024-09-11T22:53:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 -- start start 2024-09-11T22:53:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 Processor -- start 2024-09-11T22:53:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa446f5af0 0x55fa44708810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f6410 0x55fa44708d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa44700cb0 0x55fa44777ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f92e41aca40 con 0x55fa44700cb0 2024-09-11T22:53:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f92e41ac370 con 0x55fa446f5af0 2024-09-11T22:53:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f92e41a95b0 con 0x55fa446f6410 2024-09-11T22:53:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92ebddd740 10 client.0 _ll_register_callbacks cb 0x7f92d8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92e2d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa44700cb0 0x55fa44777ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.618+0000 7f92e2d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa44700cb0 0x55fa44777ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58696/0 (socket says 172.21.15.152:58696) 2024-09-11T22:53:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92e2d9d640 1 -- 172.21.15.152:0/61083966 learned_addr learned my addr 172.21.15.152:0/61083966 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92e1d9b640 1 --2- 172.21.15.152:0/61083966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f6410 0x55fa44708d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92e259c640 1 --2- 172.21.15.152:0/61083966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa446f5af0 0x55fa44708810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92e2d9d640 1 -- 172.21.15.152:0/61083966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f6410 msgr2=0x55fa44708d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92e2d9d640 1 --2- 172.21.15.152:0/61083966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f6410 0x55fa44708d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92e2d9d640 1 -- 172.21.15.152:0/61083966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa446f5af0 msgr2=0x55fa44708810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92e2d9d640 1 --2- 172.21.15.152:0/61083966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa446f5af0 0x55fa44708810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92e2d9d640 1 -- 172.21.15.152:0/61083966 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa44778380 con 0x55fa44700cb0 2024-09-11T22:53:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92e259c640 1 --2- 172.21.15.152:0/61083966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa446f5af0 0x55fa44708810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92e2d9d640 1 --2- 172.21.15.152:0/61083966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa44700cb0 0x55fa44777ce0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f92dc01f040 tx=0x7f92dc01be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9cab70e505b8e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92c9ffb640 1 -- 172.21.15.152:0/61083966 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc016040 con 0x55fa44700cb0 2024-09-11T22:53:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92c9ffb640 1 -- 172.21.15.152:0/61083966 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f92dc004bf0 con 0x55fa44700cb0 2024-09-11T22:53:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92ebddd740 1 -- 172.21.15.152:0/61083966 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92dc012910 con 0x55fa44700cb0 2024-09-11T22:53:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92c9ffb640 1 -- 172.21.15.152:0/61083966 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f92dc004df0 con 0x55fa44700cb0 2024-09-11T22:53:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92ebddd740 10 client.15678 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.619+0000 7f92ebddd740 1 -- 172.21.15.152:0/61083966 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa4477f740 con 0x55fa44700cb0 2024-09-11T22:53:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.620+0000 7f92c9ffb640 1 -- 172.21.15.152:0/61083966 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92dc012910 con 0x55fa44700cb0 2024-09-11T22:53:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.620+0000 7f92c9ffb640 1 -- 172.21.15.152:0/61083966 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f92dc03a070 con 0x55fa44700cb0 2024-09-11T22:53:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.620+0000 7f92ebddd740 10 client.15678 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.620+0000 7f92ebddd740 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.620+0000 7f92ebddd740 20 client.15678 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.620+0000 7f92ebddd740 1 --2- 172.21.15.152:0/61083966 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55fa4477fcb0 0x55fa447820a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.620+0000 7f92ebddd740 4 client.15678 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:53:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.620+0000 7f92ebddd740 1 -- 172.21.15.152:0/61083966 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa44661d70 con 0x55fa4477fcb0 2024-09-11T22:53:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.620+0000 7f92e259c640 1 --2- 172.21.15.152:0/61083966 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55fa4477fcb0 0x55fa447820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:42.628 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:42 smithi152 ceph-mon[31181]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T22:53:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.621+0000 7f92c9ffb640 10 client.15678 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.621+0000 7f92e259c640 1 --2- 172.21.15.152:0/61083966 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55fa4477fcb0 0x55fa447820a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5808bd586b6c65fb server_cookie=cacf151417a8b44d in_seq=0 out_seq=0 2024-09-11T22:53:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.624+0000 7f92c9ffb640 1 -- 172.21.15.152:0/61083966 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa44661d70 con 0x55fa4477fcb0 2024-09-11T22:53:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.624+0000 7f92c9ffb640 10 client.15678 handle_command_reply: tid=0 2024-09-11T22:53:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.624+0000 7f92ebddd740 1 client.15678 shutdown 2024-09-11T22:53:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.624+0000 7f92ebddd740 1 -- 172.21.15.152:0/61083966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa44700cb0 msgr2=0x55fa44777ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.624+0000 7f92ebddd740 1 --2- 172.21.15.152:0/61083966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa44700cb0 0x55fa44777ce0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f92dc01f040 tx=0x7f92dc01be20 comp rx=0 tx=0).stop 2024-09-11T22:53:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92ebddd740 1 -- 172.21.15.152:0/61083966 shutdown_connections 2024-09-11T22:53:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92ebddd740 1 --2- 172.21.15.152:0/61083966 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa44700cb0 0x55fa44777ce0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92ebddd740 1 --2- 172.21.15.152:0/61083966 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa446f6410 0x55fa44708d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92ebddd740 1 --2- 172.21.15.152:0/61083966 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55fa4477fcb0 0x55fa447820a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92ebddd740 1 --2- 172.21.15.152:0/61083966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa446f5af0 0x55fa44708810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92ebddd740 1 -- 172.21.15.152:0/61083966 >> 172.21.15.152:0/61083966 conn(0x55fa446f2060 msgr2=0x55fa446f11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92c9ffb640 0 client.15678 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92ebddd740 1 -- 172.21.15.152:0/61083966 shutdown_connections 2024-09-11T22:53:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92ebddd740 1 -- 172.21.15.152:0/61083966 wait complete. 2024-09-11T22:53:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92ebddd740 20 client.15678 trim_cache size 0 max 16384 2024-09-11T22:53:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:{ 2024-09-11T22:53:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "message": "", 2024-09-11T22:53:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "return_code": 0 2024-09-11T22:53:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:} 2024-09-11T22:53:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92e8f66640 1 -- 172.21.15.152:0/3205724538 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f92cc0a2e40 msgr2=0x7f92cc0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92e8f66640 1 --2- 172.21.15.152:0/3205724538 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f92cc0a2e40 0x7f92cc0a5300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f92dc01e000 tx=0x7f92dc021000 comp rx=0 tx=0).stop 2024-09-11T22:53:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92e8f66640 1 -- 172.21.15.152:0/3205724538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92e41009b0 msgr2=0x7f92e41991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92e8f66640 1 --2- 172.21.15.152:0/3205724538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92e41009b0 0x7f92e41991b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f92d400d950 tx=0x7f92d400de10 comp rx=0 tx=0).stop 2024-09-11T22:53:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92e8f66640 1 -- 172.21.15.152:0/3205724538 shutdown_connections 2024-09-11T22:53:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92e8f66640 1 --2- 172.21.15.152:0/3205724538 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f92cc0a2e40 0x7f92cc0a5300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92e8f66640 1 --2- 172.21.15.152:0/3205724538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92e41069c0 0x7f92e41a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92e8f66640 1 --2- 172.21.15.152:0/3205724538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92e4101350 0x7f92e41996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92e8f66640 1 --2- 172.21.15.152:0/3205724538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f92e41009b0 0x7f92e41991b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92e8f66640 1 -- 172.21.15.152:0/3205724538 >> 172.21.15.152:0/3205724538 conn(0x7f92e4077990 msgr2=0x7f92e4103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92e8f66640 1 -- 172.21.15.152:0/3205724538 shutdown_connections 2024-09-11T22:53:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:42.625+0000 7f92e8f66640 1 -- 172.21.15.152:0/3205724538 wait complete. 2024-09-11T22:53:44.545 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:44 smithi191 ceph-mon[35211]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T22:53:44.581 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:44 smithi204 ceph-mon[35454]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T22:53:44.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:44 smithi152 ceph-mon[31181]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T22:53:45.003 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:53:45.046 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:45.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.603+0000 7f8f29d4c640 1 -- 172.21.15.152:0/3218741833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f241036e0 msgr2=0x7f8f24107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:45.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.603+0000 7f8f29d4c640 1 --2- 172.21.15.152:0/3218741833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f241036e0 0x7f8f24107b90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009930 tx=0x7f8f1802a420 comp rx=0 tx=0).stop 2024-09-11T22:53:45.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.603+0000 7f8f29d4c640 1 -- 172.21.15.152:0/3218741833 shutdown_connections 2024-09-11T22:53:45.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.603+0000 7f8f29d4c640 1 --2- 172.21.15.152:0/3218741833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f241082a0 0x7f8f2410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:45.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.603+0000 7f8f29d4c640 1 --2- 172.21.15.152:0/3218741833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f241036e0 0x7f8f24107b90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:45.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.603+0000 7f8f29d4c640 1 --2- 172.21.15.152:0/3218741833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f24102d30 0x7f8f24103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:45.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.603+0000 7f8f29d4c640 1 -- 172.21.15.152:0/3218741833 >> 172.21.15.152:0/3218741833 conn(0x7f8f240fc7d0 msgr2=0x7f8f240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:45.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.603+0000 7f8f29d4c640 1 -- 172.21.15.152:0/3218741833 shutdown_connections 2024-09-11T22:53:45.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.603+0000 7f8f29d4c640 1 -- 172.21.15.152:0/3218741833 wait complete. 2024-09-11T22:53:45.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.604+0000 7f8f29d4c640 1 Processor -- start 2024-09-11T22:53:45.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.604+0000 7f8f29d4c640 1 -- start start 2024-09-11T22:53:45.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.604+0000 7f8f29d4c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f24102d30 0x7f8f24198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:45.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.604+0000 7f8f29d4c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f241036e0 0x7f8f24198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:45.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.604+0000 7f8f29d4c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f241082a0 0x7f8f2419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:45.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.604+0000 7f8f29d4c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8f241120a0 con 0x7f8f24102d30 2024-09-11T22:53:45.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.604+0000 7f8f29d4c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24111f20 con 0x7f8f241082a0 2024-09-11T22:53:45.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.604+0000 7f8f29d4c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24112220 con 0x7f8f241036e0 2024-09-11T22:53:45.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.604+0000 7f8f237fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f24102d30 0x7f8f24198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:45.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.604+0000 7f8f237fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f24102d30 0x7f8f24198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58712/0 (socket says 172.21.15.152:58712) 2024-09-11T22:53:45.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.604+0000 7f8f237fe640 1 -- 172.21.15.152:0/3644090087 learned_addr learned my addr 172.21.15.152:0/3644090087 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:45.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.605+0000 7f8f22ffd640 1 --2- 172.21.15.152:0/3644090087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f241036e0 0x7f8f24198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:45.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.605+0000 7f8f23fff640 1 --2- 172.21.15.152:0/3644090087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f241082a0 0x7f8f2419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:45.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.605+0000 7f8f237fe640 1 -- 172.21.15.152:0/3644090087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f241036e0 msgr2=0x7f8f24198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:45.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.605+0000 7f8f237fe640 1 --2- 172.21.15.152:0/3644090087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f241036e0 0x7f8f24198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:45.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.605+0000 7f8f237fe640 1 -- 172.21.15.152:0/3644090087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f241082a0 msgr2=0x7f8f2419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:45.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.605+0000 7f8f237fe640 1 --2- 172.21.15.152:0/3644090087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f241082a0 0x7f8f2419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:45.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.605+0000 7f8f237fe640 1 -- 172.21.15.152:0/3644090087 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2419d530 con 0x7f8f24102d30 2024-09-11T22:53:45.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.605+0000 7f8f237fe640 1 --2- 172.21.15.152:0/3644090087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f24102d30 0x7f8f24198580 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400c970 tx=0x7f8f1400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:45.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.605+0000 7f8f20ff9640 1 -- 172.21.15.152:0/3644090087 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14007bb0 con 0x7f8f24102d30 2024-09-11T22:53:45.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.605+0000 7f8f29d4c640 1 -- 172.21.15.152:0/3644090087 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2419d820 con 0x7f8f24102d30 2024-09-11T22:53:45.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.605+0000 7f8f20ff9640 1 -- 172.21.15.152:0/3644090087 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f14007d50 con 0x7f8f24102d30 2024-09-11T22:53:45.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.605+0000 7f8f20ff9640 1 -- 172.21.15.152:0/3644090087 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1400f450 con 0x7f8f24102d30 2024-09-11T22:53:45.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.606+0000 7f8f29d4c640 1 -- 172.21.15.152:0/3644090087 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f241a5100 con 0x7f8f24102d30 2024-09-11T22:53:45.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.607+0000 7f8f29d4c640 1 -- 172.21.15.152:0/3644090087 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f24103110 con 0x7f8f24102d30 2024-09-11T22:53:45.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.608+0000 7f8f20ff9640 1 -- 172.21.15.152:0/3644090087 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f8f14005020 con 0x7f8f24102d30 2024-09-11T22:53:45.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.609+0000 7f8f20ff9640 1 --2- 172.21.15.152:0/3644090087 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8f0c0a2d80 0x7f8f0c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:45.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.609+0000 7f8f22ffd640 1 --2- 172.21.15.152:0/3644090087 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8f0c0a2d80 0x7f8f0c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:45.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.609+0000 7f8f20ff9640 1 -- 172.21.15.152:0/3644090087 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f8f140cf2d0 con 0x7f8f24102d30 2024-09-11T22:53:45.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.610+0000 7f8f22ffd640 1 --2- 172.21.15.152:0/3644090087 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8f0c0a2d80 0x7f8f0c0a5240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f18005940 tx=0x7f8f180058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:45.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.613+0000 7f8f20ff9640 1 -- 172.21.15.152:0/3644090087 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f14014030 con 0x7f8f24102d30 2024-09-11T22:53:45.690 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:45.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.935+0000 7f8f29d4c640 1 -- 172.21.15.152:0/3644090087 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f2410a2b0 con 0x7f8f24102d30 2024-09-11T22:53:45.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.936+0000 7f8f20ff9640 1 -- 172.21.15.152:0/3644090087 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f8f1409b780 con 0x7f8f24102d30 2024-09-11T22:53:45.938 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:53:45.939 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:53:45.939 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:53:45.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.938+0000 7f8f29d4c640 1 -- 172.21.15.152:0/3644090087 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8f0c0a2d80 msgr2=0x7f8f0c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:45.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.938+0000 7f8f29d4c640 1 --2- 172.21.15.152:0/3644090087 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8f0c0a2d80 0x7f8f0c0a5240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8f18005940 tx=0x7f8f180058d0 comp rx=0 tx=0).stop 2024-09-11T22:53:45.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.938+0000 7f8f29d4c640 1 -- 172.21.15.152:0/3644090087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f24102d30 msgr2=0x7f8f24198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:45.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.938+0000 7f8f29d4c640 1 --2- 172.21.15.152:0/3644090087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f24102d30 0x7f8f24198580 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400c970 tx=0x7f8f1400ce30 comp rx=0 tx=0).stop 2024-09-11T22:53:45.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.938+0000 7f8f29d4c640 1 -- 172.21.15.152:0/3644090087 shutdown_connections 2024-09-11T22:53:45.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.938+0000 7f8f29d4c640 1 --2- 172.21.15.152:0/3644090087 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8f0c0a2d80 0x7f8f0c0a5240 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:45.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.938+0000 7f8f29d4c640 1 --2- 172.21.15.152:0/3644090087 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f241082a0 0x7f8f2419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:45.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.938+0000 7f8f29d4c640 1 --2- 172.21.15.152:0/3644090087 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f241036e0 0x7f8f24198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:45.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.938+0000 7f8f29d4c640 1 --2- 172.21.15.152:0/3644090087 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f24102d30 0x7f8f24198580 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:45.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.939+0000 7f8f29d4c640 1 -- 172.21.15.152:0/3644090087 >> 172.21.15.152:0/3644090087 conn(0x7f8f240fc7d0 msgr2=0x7f8f24105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:45.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.939+0000 7f8f29d4c640 1 -- 172.21.15.152:0/3644090087 shutdown_connections 2024-09-11T22:53:45.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:45.939+0000 7f8f29d4c640 1 -- 172.21.15.152:0/3644090087 wait complete. 2024-09-11T22:53:46.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:46 smithi191 ceph-mon[35211]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-09-11T22:53:46.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:46 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3644090087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:53:46.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:46 smithi204 ceph-mon[35454]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-09-11T22:53:46.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:46 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3644090087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:53:46.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:46 smithi152 ceph-mon[31181]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-09-11T22:53:46.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:46 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3644090087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:53:46.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.857+0000 7faac2b43640 1 -- 172.21.15.152:0/2522376417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faabc102db0 msgr2=0x7faabc103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:46.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.857+0000 7faac2b43640 1 --2- 172.21.15.152:0/2522376417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faabc102db0 0x7faabc103190 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7faab0009930 tx=0x7faab002a420 comp rx=0 tx=0).stop 2024-09-11T22:53:46.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.857+0000 7faac2b43640 1 -- 172.21.15.152:0/2522376417 shutdown_connections 2024-09-11T22:53:46.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.857+0000 7faac2b43640 1 --2- 172.21.15.152:0/2522376417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faabc1082e0 0x7faabc10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.857+0000 7faac2b43640 1 --2- 172.21.15.152:0/2522376417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faabc103760 0x7faabc107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.857+0000 7faac2b43640 1 --2- 172.21.15.152:0/2522376417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faabc102db0 0x7faabc103190 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.857+0000 7faac2b43640 1 -- 172.21.15.152:0/2522376417 >> 172.21.15.152:0/2522376417 conn(0x7faabc0fc7d0 msgr2=0x7faabc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:46.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.857+0000 7faac2b43640 1 -- 172.21.15.152:0/2522376417 shutdown_connections 2024-09-11T22:53:46.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.857+0000 7faac2b43640 1 -- 172.21.15.152:0/2522376417 wait complete. 2024-09-11T22:53:46.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.858+0000 7faac2b43640 1 Processor -- start 2024-09-11T22:53:46.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.858+0000 7faac2b43640 1 -- start start 2024-09-11T22:53:46.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.858+0000 7faac2b43640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faabc102db0 0x7faabc100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.858+0000 7faac2b43640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faabc103760 0x7faabc100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.859+0000 7faac2b43640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faabc1082e0 0x7faabc101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.859+0000 7faac2b43640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faabc112110 con 0x7faabc102db0 2024-09-11T22:53:46.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.859+0000 7faac08df640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faabc102db0 0x7faabc100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.859+0000 7faac2b43640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faabc111f90 con 0x7faabc103760 2024-09-11T22:53:46.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.859+0000 7faac08df640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faabc102db0 0x7faabc100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58736/0 (socket says 172.21.15.152:58736) 2024-09-11T22:53:46.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.859+0000 7faac08df640 1 -- 172.21.15.152:0/1617544484 learned_addr learned my addr 172.21.15.152:0/1617544484 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:46.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.859+0000 7faac2b43640 1 -- 172.21.15.152:0/1617544484 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faabc112290 con 0x7faabc1082e0 2024-09-11T22:53:46.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.859+0000 7faabbfff640 1 --2- 172.21.15.152:0/1617544484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faabc103760 0x7faabc100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.859+0000 7faac10e0640 1 --2- 172.21.15.152:0/1617544484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faabc1082e0 0x7faabc101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.859+0000 7faac08df640 1 -- 172.21.15.152:0/1617544484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faabc1082e0 msgr2=0x7faabc101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:46.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.859+0000 7faac08df640 1 --2- 172.21.15.152:0/1617544484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faabc1082e0 0x7faabc101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.859+0000 7faac08df640 1 -- 172.21.15.152:0/1617544484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faabc103760 msgr2=0x7faabc100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:46.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.859+0000 7faac08df640 1 --2- 172.21.15.152:0/1617544484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faabc103760 0x7faabc100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.859+0000 7faac08df640 1 -- 172.21.15.152:0/1617544484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc1a6ac0 con 0x7faabc102db0 2024-09-11T22:53:46.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.859+0000 7faabbfff640 1 --2- 172.21.15.152:0/1617544484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faabc103760 0x7faabc100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:46.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.859+0000 7faac08df640 1 --2- 172.21.15.152:0/1617544484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faabc102db0 0x7faabc100a10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7faab0004770 tx=0x7faab0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:46.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.860+0000 7faab9ffb640 1 -- 172.21.15.152:0/1617544484 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab003f070 con 0x7faabc102db0 2024-09-11T22:53:46.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.860+0000 7faab9ffb640 1 -- 172.21.15.152:0/1617544484 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab0031c20 con 0x7faabc102db0 2024-09-11T22:53:46.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.860+0000 7faab9ffb640 1 -- 172.21.15.152:0/1617544484 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0035030 con 0x7faabc102db0 2024-09-11T22:53:46.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.860+0000 7faac2b43640 1 -- 172.21.15.152:0/1617544484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faabc1a6d50 con 0x7faabc102db0 2024-09-11T22:53:46.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.860+0000 7faac2b43640 1 -- 172.21.15.152:0/1617544484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faabc1a71b0 con 0x7faabc102db0 2024-09-11T22:53:46.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.864+0000 7faac4f1c740 1 Processor -- start 2024-09-11T22:53:46.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.864+0000 7faac4f1c740 1 -- start start 2024-09-11T22:53:46.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.864+0000 7faab9ffb640 1 -- 172.21.15.152:0/1617544484 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7faab0042070 con 0x7faabc102db0 2024-09-11T22:53:46.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.864+0000 7faac4f1c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a186745090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.864+0000 7faac4f1c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a1867455d0 0x55a186745a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.864+0000 7faac10e0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a186745090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.864+0000 7faac10e0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a186745090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58748/0 (socket says 172.21.15.152:58748) 2024-09-11T22:53:46.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.864+0000 7faac4f1c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a186750370 0x55a186752760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.864+0000 7faac4f1c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a1866ab2f0 con 0x55a186746b40 2024-09-11T22:53:46.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.864+0000 7faac4f1c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a186652cf0 con 0x55a1867455d0 2024-09-11T22:53:46.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.864+0000 7faac4f1c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a1866a72b0 con 0x55a186750370 2024-09-11T22:53:46.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.864+0000 7faac10e0640 1 -- 172.21.15.152:0/3849361168 learned_addr learned my addr 172.21.15.152:0/3849361168 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:46.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.864+0000 7faac08df640 1 --2- 172.21.15.152:0/3849361168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a1867455d0 0x55a186745a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.865+0000 7faabbfff640 1 --2- 172.21.15.152:0/3849361168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a186750370 0x55a186752760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.865+0000 7faac10e0640 1 -- 172.21.15.152:0/3849361168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a186750370 msgr2=0x55a186752760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:46.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.865+0000 7faac10e0640 1 --2- 172.21.15.152:0/3849361168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a186750370 0x55a186752760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.865+0000 7faac10e0640 1 -- 172.21.15.152:0/3849361168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a1867455d0 msgr2=0x55a186745a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:46.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.865+0000 7faac10e0640 1 --2- 172.21.15.152:0/3849361168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a1867455d0 0x55a186745a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.865+0000 7faac10e0640 1 -- 172.21.15.152:0/3849361168 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1866373a0 con 0x55a186746b40 2024-09-11T22:53:46.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.865+0000 7faabbfff640 1 --2- 172.21.15.152:0/3849361168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a186750370 0x55a186752760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:46.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.865+0000 7faac08df640 1 --2- 172.21.15.152:0/3849361168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a1867455d0 0x55a186745a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:46.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.865+0000 7faab9ffb640 1 --2- 172.21.15.152:0/1617544484 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7faaac0a2e00 0x7faaac0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.865+0000 7faac10e0640 1 --2- 172.21.15.152:0/3849361168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a186745090 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7faab400a430 tx=0x7faab4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee3581a63d0ce56 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:46.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.865+0000 7faac08df640 1 --2- 172.21.15.152:0/1617544484 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7faaac0a2e00 0x7faaac0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.865+0000 7faab9ffb640 1 -- 172.21.15.152:0/1617544484 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7faab00f01e0 con 0x7faabc102db0 2024-09-11T22:53:46.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.866+0000 7faaa6ffd640 1 -- 172.21.15.152:0/3849361168 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab4013790 con 0x55a186746b40 2024-09-11T22:53:46.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.866+0000 7faaa6ffd640 1 -- 172.21.15.152:0/3849361168 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab4019070 con 0x55a186746b40 2024-09-11T22:53:46.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.866+0000 7faaa6ffd640 1 -- 172.21.15.152:0/3849361168 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab4014de0 con 0x55a186746b40 2024-09-11T22:53:46.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.867+0000 7faac4f1c740 1 -- 172.21.15.152:0/3849361168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 msgr2=0x55a186745090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:46.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.867+0000 7faac08df640 1 --2- 172.21.15.152:0/1617544484 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7faaac0a2e00 0x7faaac0a52c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faab00ed920 tx=0x7faab00f3c20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:46.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.867+0000 7faac4f1c740 1 --2- 172.21.15.152:0/3849361168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a186745090 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7faab400a430 tx=0x7faab4009650 comp rx=0 tx=0).stop 2024-09-11T22:53:46.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.867+0000 7faac4f1c740 1 -- 172.21.15.152:0/3849361168 shutdown_connections 2024-09-11T22:53:46.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.867+0000 7faac4f1c740 1 --2- 172.21.15.152:0/3849361168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a186750370 0x55a186752760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.867+0000 7faac4f1c740 1 --2- 172.21.15.152:0/3849361168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a1867455d0 0x55a186745a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.867+0000 7faac4f1c740 1 --2- 172.21.15.152:0/3849361168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a186745090 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.867+0000 7faac4f1c740 1 -- 172.21.15.152:0/3849361168 >> 172.21.15.152:0/3849361168 conn(0x55a18670e310 msgr2=0x55a186741c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:46.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.867+0000 7faac4f1c740 1 -- 172.21.15.152:0/3849361168 shutdown_connections 2024-09-11T22:53:46.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.867+0000 7faac4f1c740 1 -- 172.21.15.152:0/3849361168 wait complete. 2024-09-11T22:53:46.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.867+0000 7faac4f1c740 1 -- start start 2024-09-11T22:53:46.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.867+0000 7faac4f1c740 1 Processor -- start 2024-09-11T22:53:46.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faac4f1c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1867455d0 0x55a18674fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faac4f1c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a18674cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faac4f1c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 0x55a18674d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faac4f1c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a186753070 con 0x55a186746b40 2024-09-11T22:53:46.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faac4f1c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a186666060 con 0x55a186750370 2024-09-11T22:53:46.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faac4f1c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a1867531f0 con 0x55a1867455d0 2024-09-11T22:53:46.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faac4f1c740 10 client.0 _ll_register_callbacks cb 0x55a1863b98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:46.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faabbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a18674cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faabbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a18674cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58760/0 (socket says 172.21.15.152:58760) 2024-09-11T22:53:46.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faabbfff640 1 -- 172.21.15.152:0/553212392 learned_addr learned my addr 172.21.15.152:0/553212392 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:46.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faac10e0640 1 --2- 172.21.15.152:0/553212392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 0x55a18674d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faac10e0640 1 --2- 172.21.15.152:0/553212392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1867455d0 0x55a18674fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faac10e0640 1 -- 172.21.15.152:0/553212392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1867455d0 msgr2=0x55a18674fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:46.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faac10e0640 1 --2- 172.21.15.152:0/553212392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1867455d0 0x55a18674fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faac10e0640 1 -- 172.21.15.152:0/553212392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 msgr2=0x55a18674cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:46.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faac10e0640 1 --2- 172.21.15.152:0/553212392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a18674cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faac10e0640 1 -- 172.21.15.152:0/553212392 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18641c3c0 con 0x55a186750370 2024-09-11T22:53:46.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faabbfff640 1 --2- 172.21.15.152:0/553212392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a18674cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:46.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faac10e0640 1 --2- 172.21.15.152:0/553212392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 0x55a18674d570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faab4014a40 tx=0x7faab4014940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1ac869484bf9774 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:46.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.868+0000 7faa9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-11T22:53:46.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.869+0000 7faa9bfff640 1 -- 172.21.15.152:0/553212392 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab404a040 con 0x55a186750370 2024-09-11T22:53:46.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.869+0000 7faa9bfff640 1 -- 172.21.15.152:0/553212392 <== mon.2 v2:172.21.15.191:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7faab40139a0 con 0x55a186750370 2024-09-11T22:53:46.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.869+0000 7faac4f1c740 1 -- 172.21.15.152:0/553212392 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faab4019070 con 0x55a186750370 2024-09-11T22:53:46.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.869+0000 7faa9bfff640 1 -- 172.21.15.152:0/553212392 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faab4013ba0 con 0x55a186750370 2024-09-11T22:53:46.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.869+0000 7faac4f1c740 10 client.25112 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:46.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.869+0000 7faac4f1c740 1 -- 172.21.15.152:0/553212392 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a18675c520 con 0x55a186750370 2024-09-11T22:53:46.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.869+0000 7faa9bfff640 1 -- 172.21.15.152:0/553212392 <== mon.2 v2:172.21.15.191:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab4019070 con 0x55a186750370 2024-09-11T22:53:46.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.869+0000 7faa9bfff640 1 -- 172.21.15.152:0/553212392 <== mon.2 v2:172.21.15.191:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faab401f410 con 0x55a186750370 2024-09-11T22:53:46.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.869+0000 7faac4f1c740 10 client.25112 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:46.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.869+0000 7faac4f1c740 10 client.25112 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:53:46.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.869+0000 7faac4f1c740 20 client.25112 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:46.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.869+0000 7faac4f1c740 1 --2- 172.21.15.152:0/553212392 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55a18675d870 0x55a18675fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.869+0000 7faac4f1c740 4 client.25112 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:46.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.869+0000 7faac4f1c740 1 -- 172.21.15.152:0/553212392 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1866b8860 con 0x55a18675d870 2024-09-11T22:53:46.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.870+0000 7faabbfff640 1 --2- 172.21.15.152:0/553212392 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55a18675d870 0x55a18675fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.870+0000 7faa9bfff640 10 client.25112 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:53:46.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.870+0000 7faabbfff640 1 --2- 172.21.15.152:0/553212392 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55a18675d870 0x55a18675fc90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c8bf7bac03c671e server_cookie=df9459f80aefa129 in_seq=0 out_seq=0 2024-09-11T22:53:46.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.871+0000 7faa9bfff640 1 -- 172.21.15.152:0/553212392 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1866b8860 con 0x55a18675d870 2024-09-11T22:53:46.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.871+0000 7faa9bfff640 10 client.25112 handle_command_reply: tid=0 2024-09-11T22:53:46.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.871+0000 7faac4f1c740 1 client.25112 shutdown 2024-09-11T22:53:46.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.871+0000 7faac4f1c740 1 -- 172.21.15.152:0/553212392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 msgr2=0x55a18674d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:46.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.871+0000 7faac4f1c740 1 --2- 172.21.15.152:0/553212392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 0x55a18674d570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faab4014a40 tx=0x7faab4014940 comp rx=0 tx=0).stop 2024-09-11T22:53:46.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.871+0000 7faac4f1c740 1 -- 172.21.15.152:0/553212392 shutdown_connections 2024-09-11T22:53:46.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.871+0000 7faac4f1c740 1 --2- 172.21.15.152:0/553212392 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55a18675d870 0x55a18675fc90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.871+0000 7faa9bfff640 0 client.25112 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:53:46.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.871+0000 7faac4f1c740 1 --2- 172.21.15.152:0/553212392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 0x55a18674d570 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.871+0000 7faac4f1c740 1 --2- 172.21.15.152:0/553212392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a18674cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.871+0000 7faac4f1c740 1 --2- 172.21.15.152:0/553212392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1867455d0 0x55a18674fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.871+0000 7faac4f1c740 1 -- 172.21.15.152:0/553212392 >> 172.21.15.152:0/553212392 conn(0x55a18670e310 msgr2=0x55a186747930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:46.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.871+0000 7faac4f1c740 1 -- 172.21.15.152:0/553212392 shutdown_connections 2024-09-11T22:53:46.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.871+0000 7faac4f1c740 1 -- 172.21.15.152:0/553212392 wait complete. 2024-09-11T22:53:46.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.871+0000 7faac4f1c740 20 client.25112 trim_cache size 0 max 16384 2024-09-11T22:53:46.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.884+0000 7faac4f1c740 1 Processor -- start 2024-09-11T22:53:46.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.884+0000 7faac4f1c740 1 -- start start 2024-09-11T22:53:46.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faac4f1c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1867455d0 0x55a18676cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faac4f1c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a186765290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faac4f1c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 0x55a1867657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faac4f1c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faabc06e130 con 0x55a186746b40 2024-09-11T22:53:46.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faac4f1c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faabc06fc70 con 0x55a186750370 2024-09-11T22:53:46.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faac4f1c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faabc06e800 con 0x55a1867455d0 2024-09-11T22:53:46.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faabbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a186765290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faabbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a186765290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:58772/0 (socket says 172.21.15.152:58772) 2024-09-11T22:53:46.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faabbfff640 1 -- 172.21.15.152:0/2026880328 learned_addr learned my addr 172.21.15.152:0/2026880328 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:46.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faac10e0640 1 --2- 172.21.15.152:0/2026880328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 0x55a1867657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faac10e0640 1 --2- 172.21.15.152:0/2026880328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1867455d0 0x55a18676cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faabbfff640 1 -- 172.21.15.152:0/2026880328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1867455d0 msgr2=0x55a18676cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:46.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faabbfff640 1 --2- 172.21.15.152:0/2026880328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1867455d0 0x55a18676cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faabbfff640 1 -- 172.21.15.152:0/2026880328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 msgr2=0x55a1867657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:46.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faabbfff640 1 --2- 172.21.15.152:0/2026880328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 0x55a1867657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faabbfff640 1 -- 172.21.15.152:0/2026880328 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a186748120 con 0x55a186746b40 2024-09-11T22:53:46.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faac10e0640 1 --2- 172.21.15.152:0/2026880328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1867455d0 0x55a18676cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:46.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.885+0000 7faabbfff640 1 --2- 172.21.15.152:0/2026880328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a186765290 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7faaa8012950 tx=0x7faaa8012e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38b7d65c78036d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:46.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.886+0000 7faaa67fc640 1 -- 172.21.15.152:0/2026880328 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8004020 con 0x55a186746b40 2024-09-11T22:53:46.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.886+0000 7faaa67fc640 1 -- 172.21.15.152:0/2026880328 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa8011e80 con 0x55a186746b40 2024-09-11T22:53:46.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.886+0000 7faaa67fc640 1 -- 172.21.15.152:0/2026880328 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa80047a0 con 0x55a186746b40 2024-09-11T22:53:46.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.886+0000 7faac4f1c740 1 -- 172.21.15.152:0/2026880328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 msgr2=0x55a186765290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:46.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.886+0000 7faac4f1c740 1 --2- 172.21.15.152:0/2026880328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a186765290 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7faaa8012950 tx=0x7faaa8012e10 comp rx=0 tx=0).stop 2024-09-11T22:53:46.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.887+0000 7faac4f1c740 1 -- 172.21.15.152:0/2026880328 shutdown_connections 2024-09-11T22:53:46.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.887+0000 7faac4f1c740 1 --2- 172.21.15.152:0/2026880328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 0x55a1867657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.887+0000 7faac4f1c740 1 --2- 172.21.15.152:0/2026880328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a186746b40 0x55a186765290 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.887+0000 7faac4f1c740 1 --2- 172.21.15.152:0/2026880328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1867455d0 0x55a18676cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.887+0000 7faac4f1c740 1 -- 172.21.15.152:0/2026880328 >> 172.21.15.152:0/2026880328 conn(0x55a18670e310 msgr2=0x55a18674bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:46.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.887+0000 7faac4f1c740 1 -- 172.21.15.152:0/2026880328 shutdown_connections 2024-09-11T22:53:46.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.887+0000 7faac4f1c740 1 -- 172.21.15.152:0/2026880328 wait complete. 2024-09-11T22:53:46.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.887+0000 7faac4f1c740 1 -- start start 2024-09-11T22:53:46.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.887+0000 7faac4f1c740 1 Processor -- start 2024-09-11T22:53:46.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faac4f1c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1867455d0 0x55a18674a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faac4f1c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 0x55a18674a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faac4f1c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a18675d870 0x55a18674ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faac4f1c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faabc06f5a0 con 0x55a1867455d0 2024-09-11T22:53:46.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faac4f1c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faabc06eed0 con 0x55a186750370 2024-09-11T22:53:46.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faac4f1c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faabc06c0f0 con 0x55a18675d870 2024-09-11T22:53:46.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faac4f1c740 10 client.0 _ll_register_callbacks cb 0x7faab00ed750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:46.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faabbfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 0x55a18674a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faabbfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 0x55a18674a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:43212/0 (socket says 172.21.15.152:43212) 2024-09-11T22:53:46.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faabbfff640 1 -- 172.21.15.152:0/2543180618 learned_addr learned my addr 172.21.15.152:0/2543180618 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:46.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faac10e0640 1 --2- 172.21.15.152:0/2543180618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1867455d0 0x55a18674a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faac10e0640 1 --2- 172.21.15.152:0/2543180618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a18675d870 0x55a18674ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faabbfff640 1 -- 172.21.15.152:0/2543180618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a18675d870 msgr2=0x55a18674ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:46.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faabbfff640 1 --2- 172.21.15.152:0/2543180618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a18675d870 0x55a18674ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faabbfff640 1 -- 172.21.15.152:0/2543180618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1867455d0 msgr2=0x55a18674a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:46.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faabbfff640 1 --2- 172.21.15.152:0/2543180618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1867455d0 0x55a18674a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.888+0000 7faabbfff640 1 -- 172.21.15.152:0/2543180618 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a186756d30 con 0x55a186750370 2024-09-11T22:53:46.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.889+0000 7faabbfff640 1 --2- 172.21.15.152:0/2543180618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 0x55a18674a5e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55a1867362d0 tx=0x7faaa800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd925fa3b34467a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:46.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.889+0000 7faa9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-11T22:53:46.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.889+0000 7faa9bfff640 1 -- 172.21.15.152:0/2543180618 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8036060 con 0x55a186750370 2024-09-11T22:53:46.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.889+0000 7faac4f1c740 1 -- 172.21.15.152:0/2543180618 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a1866b2d70 con 0x55a186750370 2024-09-11T22:53:46.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.889+0000 7faa9bfff640 1 -- 172.21.15.152:0/2543180618 <== mon.2 v2:172.21.15.191:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7faaa8022040 con 0x55a186750370 2024-09-11T22:53:46.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.889+0000 7faa9bfff640 1 -- 172.21.15.152:0/2543180618 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faaa80119a0 con 0x55a186750370 2024-09-11T22:53:46.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.889+0000 7faac4f1c740 10 client.25118 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:46.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.889+0000 7faac4f1c740 1 -- 172.21.15.152:0/2543180618 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a1867db120 con 0x55a186750370 2024-09-11T22:53:46.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.890+0000 7faa9bfff640 1 -- 172.21.15.152:0/2543180618 <== mon.2 v2:172.21.15.191:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a1866b2d70 con 0x55a186750370 2024-09-11T22:53:46.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.890+0000 7faa9bfff640 1 -- 172.21.15.152:0/2543180618 <== mon.2 v2:172.21.15.191:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faaa8026470 con 0x55a186750370 2024-09-11T22:53:46.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.890+0000 7faac4f1c740 10 client.25118 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:46.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.890+0000 7faac4f1c740 10 client.25118 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:53:46.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.890+0000 7faac4f1c740 20 client.25118 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:46.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.890+0000 7faac4f1c740 1 --2- 172.21.15.152:0/2543180618 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55a1867db850 0x55a1867ddc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:46.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.890+0000 7faac4f1c740 4 client.25118 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:53:46.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.890+0000 7faac4f1c740 1 -- 172.21.15.152:0/2543180618 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a18662f170 con 0x55a1867db850 2024-09-11T22:53:46.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.890+0000 7faac10e0640 1 --2- 172.21.15.152:0/2543180618 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55a1867db850 0x55a1867ddc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:46.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.891+0000 7faac10e0640 1 --2- 172.21.15.152:0/2543180618 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55a1867db850 0x55a1867ddc70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4ac4f58e66d47d6 server_cookie=3e5d31623736713e in_seq=0 out_seq=0 2024-09-11T22:53:46.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.891+0000 7faa9bfff640 10 client.25118 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:53:46.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.891+0000 7faa9bfff640 1 -- 172.21.15.152:0/2543180618 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a18662f170 con 0x55a1867db850 2024-09-11T22:53:46.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.891+0000 7faa9bfff640 10 client.25118 handle_command_reply: tid=0 2024-09-11T22:53:46.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.891+0000 7faac4f1c740 1 client.25118 shutdown 2024-09-11T22:53:46.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.891+0000 7faac4f1c740 1 -- 172.21.15.152:0/2543180618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 msgr2=0x55a18674a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:46.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.891+0000 7faac4f1c740 1 --2- 172.21.15.152:0/2543180618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 0x55a18674a5e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55a1867362d0 tx=0x7faaa800bf90 comp rx=0 tx=0).stop 2024-09-11T22:53:46.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.891+0000 7faac4f1c740 1 -- 172.21.15.152:0/2543180618 shutdown_connections 2024-09-11T22:53:46.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.891+0000 7faac4f1c740 1 --2- 172.21.15.152:0/2543180618 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55a1867db850 0x55a1867ddc70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.891+0000 7faac4f1c740 1 --2- 172.21.15.152:0/2543180618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a18675d870 0x55a18674ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.891+0000 7faac4f1c740 1 --2- 172.21.15.152:0/2543180618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a186750370 0x55a18674a5e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.891+0000 7faac4f1c740 1 --2- 172.21.15.152:0/2543180618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1867455d0 0x55a18674a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.891+0000 7faa9bfff640 0 client.25118 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:53:46.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.891+0000 7faac4f1c740 1 -- 172.21.15.152:0/2543180618 >> 172.21.15.152:0/2543180618 conn(0x55a18670e310 msgr2=0x55a18675b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:46.907 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:53:46.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.892+0000 7faac4f1c740 1 -- 172.21.15.152:0/2543180618 shutdown_connections 2024-09-11T22:53:46.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.892+0000 7faac4f1c740 1 -- 172.21.15.152:0/2543180618 wait complete. 2024-09-11T22:53:46.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.892+0000 7faac4f1c740 20 client.25118 trim_cache size 0 max 16384 2024-09-11T22:53:46.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.892+0000 7faac2b43640 1 -- 172.21.15.152:0/1617544484 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7faaac0a2e00 msgr2=0x7faaac0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:46.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.892+0000 7faac2b43640 1 --2- 172.21.15.152:0/1617544484 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7faaac0a2e00 0x7faaac0a52c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faab00ed920 tx=0x7faab00f3c20 comp rx=0 tx=0).stop 2024-09-11T22:53:46.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.892+0000 7faac2b43640 1 -- 172.21.15.152:0/1617544484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faabc102db0 msgr2=0x7faabc100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:46.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.892+0000 7faac2b43640 1 --2- 172.21.15.152:0/1617544484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faabc102db0 0x7faabc100a10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7faab0004770 tx=0x7faab0003b90 comp rx=0 tx=0).stop 2024-09-11T22:53:46.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.892+0000 7faac2b43640 1 -- 172.21.15.152:0/1617544484 shutdown_connections 2024-09-11T22:53:46.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.892+0000 7faac2b43640 1 --2- 172.21.15.152:0/1617544484 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7faaac0a2e00 0x7faaac0a52c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.892+0000 7faac2b43640 1 --2- 172.21.15.152:0/1617544484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faabc1082e0 0x7faabc101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.892+0000 7faac2b43640 1 --2- 172.21.15.152:0/1617544484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faabc103760 0x7faabc100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.892+0000 7faac2b43640 1 --2- 172.21.15.152:0/1617544484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faabc102db0 0x7faabc100a10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:46.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.892+0000 7faac2b43640 1 -- 172.21.15.152:0/1617544484 >> 172.21.15.152:0/1617544484 conn(0x7faabc0fc7d0 msgr2=0x7faabc105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:46.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.893+0000 7faac2b43640 1 -- 172.21.15.152:0/1617544484 shutdown_connections 2024-09-11T22:53:46.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:46.893+0000 7faac2b43640 1 -- 172.21.15.152:0/1617544484 wait complete. 2024-09-11T22:53:47.011 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24455 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-11T22:53:47.012 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24455 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-11T22:53:47.802 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:53:47.820 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:48.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:48 smithi152 ceph-mon[31181]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T22:53:48.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.403+0000 7efe59537640 1 -- 172.21.15.152:0/1071942280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe54101420 msgr2=0x7efe5410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:48.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.403+0000 7efe59537640 1 --2- 172.21.15.152:0/1071942280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe54101420 0x7efe5410d950 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009cd0 tx=0x7efe4c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.403+0000 7efe59537640 1 -- 172.21.15.152:0/1071942280 shutdown_connections 2024-09-11T22:53:48.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.403+0000 7efe59537640 1 --2- 172.21.15.152:0/1071942280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe54101420 0x7efe5410d950 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.403+0000 7efe59537640 1 --2- 172.21.15.152:0/1071942280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe54100a80 0x7efe54100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.403+0000 7efe59537640 1 --2- 172.21.15.152:0/1071942280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe54106a80 0x7efe54106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.403+0000 7efe59537640 1 -- 172.21.15.152:0/1071942280 >> 172.21.15.152:0/1071942280 conn(0x7efe540fc7a0 msgr2=0x7efe540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:48.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.403+0000 7efe59537640 1 -- 172.21.15.152:0/1071942280 shutdown_connections 2024-09-11T22:53:48.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.403+0000 7efe59537640 1 -- 172.21.15.152:0/1071942280 wait complete. 2024-09-11T22:53:48.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.404+0000 7efe59537640 1 Processor -- start 2024-09-11T22:53:48.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.404+0000 7efe59537640 1 -- start start 2024-09-11T22:53:48.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.404+0000 7efe59537640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe54100a80 0x7efe54197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.404+0000 7efe59537640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe54101420 0x7efe541977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.404+0000 7efe59537640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe54106a80 0x7efe5419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.404+0000 7efe59537640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efe54110ed0 con 0x7efe54106a80 2024-09-11T22:53:48.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.404+0000 7efe59537640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efe54110d50 con 0x7efe54101420 2024-09-11T22:53:48.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.404+0000 7efe59537640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efe54111050 con 0x7efe54100a80 2024-09-11T22:53:48.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.404+0000 7efe537fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe54106a80 0x7efe5419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.404+0000 7efe537fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe54106a80 0x7efe5419bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49250/0 (socket says 172.21.15.152:49250) 2024-09-11T22:53:48.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.404+0000 7efe537fe640 1 -- 172.21.15.152:0/687724323 learned_addr learned my addr 172.21.15.152:0/687724323 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:48.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.404+0000 7efe52ffd640 1 --2- 172.21.15.152:0/687724323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe54100a80 0x7efe54197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.404+0000 7efe527fc640 1 --2- 172.21.15.152:0/687724323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe54101420 0x7efe541977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.405+0000 7efe537fe640 1 -- 172.21.15.152:0/687724323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe54100a80 msgr2=0x7efe54197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:48.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.405+0000 7efe537fe640 1 --2- 172.21.15.152:0/687724323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe54100a80 0x7efe54197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.405+0000 7efe537fe640 1 -- 172.21.15.152:0/687724323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe54101420 msgr2=0x7efe541977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:48.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.405+0000 7efe537fe640 1 --2- 172.21.15.152:0/687724323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe54101420 0x7efe541977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.405+0000 7efe537fe640 1 -- 172.21.15.152:0/687724323 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5419c240 con 0x7efe54106a80 2024-09-11T22:53:48.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.405+0000 7efe527fc640 1 --2- 172.21.15.152:0/687724323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe54101420 0x7efe541977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:48.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.405+0000 7efe537fe640 1 --2- 172.21.15.152:0/687724323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe54106a80 0x7efe5419bb60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efe4c033040 tx=0x7efe4c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:48.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.405+0000 7efe3bfff640 1 -- 172.21.15.152:0/687724323 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c04a070 con 0x7efe54106a80 2024-09-11T22:53:48.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.405+0000 7efe3bfff640 1 -- 172.21.15.152:0/687724323 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe4c004040 con 0x7efe54106a80 2024-09-11T22:53:48.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.405+0000 7efe59537640 1 -- 172.21.15.152:0/687724323 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5419c4d0 con 0x7efe54106a80 2024-09-11T22:53:48.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.405+0000 7efe3bfff640 1 -- 172.21.15.152:0/687724323 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c0308b0 con 0x7efe54106a80 2024-09-11T22:53:48.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.405+0000 7efe59537640 1 -- 172.21.15.152:0/687724323 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5406a790 con 0x7efe54106a80 2024-09-11T22:53:48.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.408+0000 7efe5b910740 1 Processor -- start 2024-09-11T22:53:48.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.409+0000 7efe3bfff640 1 -- 172.21.15.152:0/687724323 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7efe4c0041f0 con 0x7efe54106a80 2024-09-11T22:53:48.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.409+0000 7efe5b910740 1 -- start start 2024-09-11T22:53:48.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.409+0000 7efe5b910740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e6c620 0x556c04e6ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.409+0000 7efe5b910740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e6cf40 0x556c04e7b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.409+0000 7efe5b910740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e7bd50 0x556c04e7e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.409+0000 7efe5b910740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x556c04dd22a0 con 0x556c04e6c620 2024-09-11T22:53:48.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.409+0000 7efe5b910740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556c04d79b10 con 0x556c04e6cf40 2024-09-11T22:53:48.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.409+0000 7efe5b910740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556c04dce260 con 0x556c04e7bd50 2024-09-11T22:53:48.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.409+0000 7efe527fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e6c620 0x556c04e6ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.409+0000 7efe527fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e6c620 0x556c04e6ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49252/0 (socket says 172.21.15.152:49252) 2024-09-11T22:53:48.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.409+0000 7efe527fc640 1 -- 172.21.15.152:0/2030546884 learned_addr learned my addr 172.21.15.152:0/2030546884 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:48.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.409+0000 7efe52ffd640 1 --2- 172.21.15.152:0/2030546884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e6cf40 0x556c04e7b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.410+0000 7efe527fc640 1 --2- 172.21.15.152:0/2030546884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e7bd50 0x556c04e7e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.410+0000 7efe527fc640 1 -- 172.21.15.152:0/2030546884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e7bd50 msgr2=0x556c04e7e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:48.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.410+0000 7efe3bfff640 1 --2- 172.21.15.152:0/687724323 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efe3c0a2e00 0x7efe3c0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.410+0000 7efe527fc640 1 --2- 172.21.15.152:0/2030546884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e7bd50 0x556c04e7e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.410+0000 7efe527fc640 1 -- 172.21.15.152:0/2030546884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e6cf40 msgr2=0x556c04e7b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:48.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.410+0000 7efe527fc640 1 --2- 172.21.15.152:0/2030546884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e6cf40 0x556c04e7b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.410+0000 7efe527fc640 1 -- 172.21.15.152:0/2030546884 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c04b43b90 con 0x556c04e6c620 2024-09-11T22:53:48.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.410+0000 7efe3bfff640 1 -- 172.21.15.152:0/687724323 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7efe4c02e020 con 0x7efe54106a80 2024-09-11T22:53:48.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.410+0000 7efe527fc640 1 --2- 172.21.15.152:0/2030546884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e6c620 0x556c04e6ca00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efe48015660 tx=0x7efe48009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847987acfb911270 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:48.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.410+0000 7efe537fe640 1 --2- 172.21.15.152:0/687724323 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efe3c0a2e00 0x7efe3c0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.411+0000 7efe397fa640 1 -- 172.21.15.152:0/2030546884 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48010550 con 0x556c04e6c620 2024-09-11T22:53:48.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.411+0000 7efe397fa640 1 -- 172.21.15.152:0/2030546884 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe4801b070 con 0x556c04e6c620 2024-09-11T22:53:48.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.411+0000 7efe397fa640 1 -- 172.21.15.152:0/2030546884 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe480138c0 con 0x556c04e6c620 2024-09-11T22:53:48.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.411+0000 7efe537fe640 1 --2- 172.21.15.152:0/687724323 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efe3c0a2e00 0x7efe3c0a52c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0ef080 tx=0x7efe4c04ff90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:48.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.412+0000 7efe5b910740 1 -- 172.21.15.152:0/2030546884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e6c620 msgr2=0x556c04e6ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:48.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.412+0000 7efe5b910740 1 --2- 172.21.15.152:0/2030546884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e6c620 0x556c04e6ca00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efe48015660 tx=0x7efe48009710 comp rx=0 tx=0).stop 2024-09-11T22:53:48.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.412+0000 7efe5b910740 1 -- 172.21.15.152:0/2030546884 shutdown_connections 2024-09-11T22:53:48.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.412+0000 7efe5b910740 1 --2- 172.21.15.152:0/2030546884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e7bd50 0x556c04e7e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.412+0000 7efe5b910740 1 --2- 172.21.15.152:0/2030546884 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e6cf40 0x556c04e7b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.412+0000 7efe5b910740 1 --2- 172.21.15.152:0/2030546884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e6c620 0x556c04e6ca00 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.412+0000 7efe5b910740 1 -- 172.21.15.152:0/2030546884 >> 172.21.15.152:0/2030546884 conn(0x556c04e68cf0 msgr2=0x556c04e690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:48.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.413+0000 7efe5b910740 1 -- 172.21.15.152:0/2030546884 shutdown_connections 2024-09-11T22:53:48.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.413+0000 7efe5b910740 1 -- 172.21.15.152:0/2030546884 wait complete. 2024-09-11T22:53:48.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.413+0000 7efe5b910740 1 -- start start 2024-09-11T22:53:48.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.413+0000 7efe5b910740 1 Processor -- start 2024-09-11T22:53:48.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.413+0000 7efe5b910740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e6cf40 0x556c04e78eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.413+0000 7efe5b910740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e7bd50 0x556c04e793f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.414+0000 7efe5b910740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e79930 0x556c04e87e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.414+0000 7efe5b910740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x556c04e7e920 con 0x556c04e79930 2024-09-11T22:53:48.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.414+0000 7efe5b910740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556c04d5a6e0 con 0x556c04e6cf40 2024-09-11T22:53:48.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.414+0000 7efe5b910740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556c04e7eaa0 con 0x556c04e7bd50 2024-09-11T22:53:48.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.414+0000 7efe5b910740 10 client.0 _ll_register_callbacks cb 0x556c04d7ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:48.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.414+0000 7efe52ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e7bd50 0x556c04e793f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.414+0000 7efe52ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e7bd50 0x556c04e793f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:55556/0 (socket says 172.21.15.152:55556) 2024-09-11T22:53:48.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.414+0000 7efe52ffd640 1 -- 172.21.15.152:0/2063490285 learned_addr learned my addr 172.21.15.152:0/2063490285 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:48.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.414+0000 7efe52ffd640 1 -- 172.21.15.152:0/2063490285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e6cf40 msgr2=0x556c04e78eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:53:48.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.414+0000 7efe52ffd640 1 --2- 172.21.15.152:0/2063490285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e6cf40 0x556c04e78eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.414+0000 7efe52ffd640 1 -- 172.21.15.152:0/2063490285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e79930 msgr2=0x556c04e87e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:48.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.414+0000 7efe527fc640 1 --2- 172.21.15.152:0/2063490285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e79930 0x556c04e87e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.414+0000 7efe52ffd640 1 --2- 172.21.15.152:0/2063490285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e79930 0x556c04e87e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.414+0000 7efe52ffd640 1 -- 172.21.15.152:0/2063490285 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c04e884e0 con 0x556c04e7bd50 2024-09-11T22:53:48.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.414+0000 7efe527fc640 1 --2- 172.21.15.152:0/2063490285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e79930 0x556c04e87e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:48.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.415+0000 7efe52ffd640 1 --2- 172.21.15.152:0/2063490285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e7bd50 0x556c04e793f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efe4400fa10 tx=0x7efe4400fed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93ac502bdcd8557 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:48.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.415+0000 7efe36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-11T22:53:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.415+0000 7efe36ff5640 1 -- 172.21.15.152:0/2063490285 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44026060 con 0x556c04e7bd50 2024-09-11T22:53:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.415+0000 7efe5b910740 1 -- 172.21.15.152:0/2063490285 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe4801b070 con 0x556c04e7bd50 2024-09-11T22:53:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.415+0000 7efe36ff5640 1 -- 172.21.15.152:0/2063490285 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7efe4400eb20 con 0x556c04e7bd50 2024-09-11T22:53:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.415+0000 7efe36ff5640 1 -- 172.21.15.152:0/2063490285 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efe4400ed20 con 0x556c04e7bd50 2024-09-11T22:53:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.415+0000 7efe5b910740 10 client.25183 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.415+0000 7efe5b910740 1 -- 172.21.15.152:0/2063490285 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c04e6f450 con 0x556c04e7bd50 2024-09-11T22:53:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.416+0000 7efe36ff5640 1 -- 172.21.15.152:0/2063490285 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe4801b070 con 0x556c04e7bd50 2024-09-11T22:53:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.416+0000 7efe36ff5640 1 -- 172.21.15.152:0/2063490285 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efe44021530 con 0x556c04e7bd50 2024-09-11T22:53:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.416+0000 7efe5b910740 10 client.25183 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.416+0000 7efe5b910740 10 client.25183 resolve_mds: validated gid 24455 aka daemon mds.b 2024-09-11T22:53:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.416+0000 7efe5b910740 20 client.25183 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.416+0000 7efe5b910740 1 --2- 172.21.15.152:0/2063490285 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556c04e709b0 0x556c04e72e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.416+0000 7efe5b910740 4 client.25183 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:48.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.416+0000 7efe5b910740 1 -- 172.21.15.152:0/2063490285 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c04ddf9d0 con 0x556c04e709b0 2024-09-11T22:53:48.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.417+0000 7efe527fc640 1 --2- 172.21.15.152:0/2063490285 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556c04e709b0 0x556c04e72e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.417+0000 7efe36ff5640 10 client.25183 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:48.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.417+0000 7efe527fc640 1 --2- 172.21.15.152:0/2063490285 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556c04e709b0 0x556c04e72e10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=573590519e591b0 server_cookie=99d303ee720497c5 in_seq=0 out_seq=0 2024-09-11T22:53:48.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.418+0000 7efe36ff5640 1 -- 172.21.15.152:0/2063490285 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c04ddf9d0 con 0x556c04e709b0 2024-09-11T22:53:48.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.418+0000 7efe36ff5640 10 client.25183 handle_command_reply: tid=0 2024-09-11T22:53:48.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.418+0000 7efe5b910740 1 client.25183 shutdown 2024-09-11T22:53:48.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.418+0000 7efe5b910740 1 -- 172.21.15.152:0/2063490285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e7bd50 msgr2=0x556c04e793f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:48.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.418+0000 7efe5b910740 1 --2- 172.21.15.152:0/2063490285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e7bd50 0x556c04e793f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efe4400fa10 tx=0x7efe4400fed0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.418+0000 7efe5b910740 1 -- 172.21.15.152:0/2063490285 shutdown_connections 2024-09-11T22:53:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.418+0000 7efe5b910740 1 --2- 172.21.15.152:0/2063490285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e79930 0x556c04e87e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.418+0000 7efe5b910740 1 --2- 172.21.15.152:0/2063490285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e7bd50 0x556c04e793f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.418+0000 7efe5b910740 1 --2- 172.21.15.152:0/2063490285 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556c04e709b0 0x556c04e72e10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.418+0000 7efe36ff5640 0 client.25183 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.418+0000 7efe5b910740 1 --2- 172.21.15.152:0/2063490285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e6cf40 0x556c04e78eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.418+0000 7efe5b910740 1 -- 172.21.15.152:0/2063490285 >> 172.21.15.152:0/2063490285 conn(0x556c04e68cf0 msgr2=0x556c04e6ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.418+0000 7efe5b910740 1 -- 172.21.15.152:0/2063490285 shutdown_connections 2024-09-11T22:53:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.418+0000 7efe5b910740 1 -- 172.21.15.152:0/2063490285 wait complete. 2024-09-11T22:53:48.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.419+0000 7efe5b910740 20 client.25183 trim_cache size 0 max 16384 2024-09-11T22:53:48.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.438+0000 7efe5b910740 1 Processor -- start 2024-09-11T22:53:48.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.438+0000 7efe5b910740 1 -- start start 2024-09-11T22:53:48.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.438+0000 7efe5b910740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e6cf40 0x556c04e67cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.438+0000 7efe5b910740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e709b0 0x556c04e87d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.438+0000 7efe5b910740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e79930 0x556c04e76110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.438+0000 7efe5b910740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efe541ab410 con 0x556c04e709b0 2024-09-11T22:53:48.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.438+0000 7efe5b910740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efe541acf50 con 0x556c04e79930 2024-09-11T22:53:48.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.438+0000 7efe5b910740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efe541abae0 con 0x556c04e6cf40 2024-09-11T22:53:48.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.438+0000 7efe52ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e709b0 0x556c04e87d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.438+0000 7efe52ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e709b0 0x556c04e87d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49260/0 (socket says 172.21.15.152:49260) 2024-09-11T22:53:48.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.438+0000 7efe52ffd640 1 -- 172.21.15.152:0/3663306711 learned_addr learned my addr 172.21.15.152:0/3663306711 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:48.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.438+0000 7efe527fc640 1 --2- 172.21.15.152:0/3663306711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e6cf40 0x556c04e67cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.439+0000 7efe527fc640 1 --2- 172.21.15.152:0/3663306711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e79930 0x556c04e76110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.439+0000 7efe52ffd640 1 -- 172.21.15.152:0/3663306711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e6cf40 msgr2=0x556c04e67cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:48.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.439+0000 7efe52ffd640 1 --2- 172.21.15.152:0/3663306711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e6cf40 0x556c04e67cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.439+0000 7efe52ffd640 1 -- 172.21.15.152:0/3663306711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e79930 msgr2=0x556c04e76110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:48.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.439+0000 7efe52ffd640 1 --2- 172.21.15.152:0/3663306711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e79930 0x556c04e76110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.439+0000 7efe52ffd640 1 -- 172.21.15.152:0/3663306711 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c04e882d0 con 0x556c04e709b0 2024-09-11T22:53:48.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.439+0000 7efe52ffd640 1 --2- 172.21.15.152:0/3663306711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e709b0 0x556c04e87d90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efe44021980 tx=0x7efe44000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2f94318897f376 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:48.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.439+0000 7efe38ff9640 1 -- 172.21.15.152:0/3663306711 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44021b70 con 0x556c04e709b0 2024-09-11T22:53:48.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.439+0000 7efe38ff9640 1 -- 172.21.15.152:0/3663306711 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe44021d10 con 0x556c04e709b0 2024-09-11T22:53:48.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.439+0000 7efe38ff9640 1 -- 172.21.15.152:0/3663306711 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44004a80 con 0x556c04e709b0 2024-09-11T22:53:48.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.440+0000 7efe5b910740 1 -- 172.21.15.152:0/3663306711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e709b0 msgr2=0x556c04e87d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:48.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.440+0000 7efe5b910740 1 --2- 172.21.15.152:0/3663306711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e709b0 0x556c04e87d90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efe44021980 tx=0x7efe44000bc0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.440+0000 7efe5b910740 1 -- 172.21.15.152:0/3663306711 shutdown_connections 2024-09-11T22:53:48.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.440+0000 7efe5b910740 1 --2- 172.21.15.152:0/3663306711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e79930 0x556c04e76110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.440+0000 7efe5b910740 1 --2- 172.21.15.152:0/3663306711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e709b0 0x556c04e87d90 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.440+0000 7efe5b910740 1 --2- 172.21.15.152:0/3663306711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e6cf40 0x556c04e67cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.440+0000 7efe5b910740 1 -- 172.21.15.152:0/3663306711 >> 172.21.15.152:0/3663306711 conn(0x556c04e68cf0 msgr2=0x556c04e78f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:48.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.441+0000 7efe5b910740 1 -- 172.21.15.152:0/3663306711 shutdown_connections 2024-09-11T22:53:48.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.441+0000 7efe5b910740 1 -- 172.21.15.152:0/3663306711 wait complete. 2024-09-11T22:53:48.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.441+0000 7efe5b910740 1 -- start start 2024-09-11T22:53:48.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.441+0000 7efe5b910740 1 Processor -- start 2024-09-11T22:53:48.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.442+0000 7efe5b910740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e6cf40 0x556c04e90220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.442+0000 7efe5b910740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e79930 0x556c04e90760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.442+0000 7efe5b910740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e7bd50 0x556c04e8e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.442+0000 7efe5b910740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efe541ac880 con 0x556c04e7bd50 2024-09-11T22:53:48.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.442+0000 7efe5b910740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efe541ac1b0 con 0x556c04e6cf40 2024-09-11T22:53:48.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.442+0000 7efe5b910740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efe541a93f0 con 0x556c04e79930 2024-09-11T22:53:48.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.442+0000 7efe5b910740 10 client.0 _ll_register_callbacks cb 0x7efe4400f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:48.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.442+0000 7efe527fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e7bd50 0x556c04e8e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.442+0000 7efe527fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e7bd50 0x556c04e8e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49268/0 (socket says 172.21.15.152:49268) 2024-09-11T22:53:48.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.442+0000 7efe527fc640 1 -- 172.21.15.152:0/3477935454 learned_addr learned my addr 172.21.15.152:0/3477935454 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:48.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.442+0000 7efe527fc640 1 --2- 172.21.15.152:0/3477935454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e6cf40 0x556c04e90220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.442+0000 7efe52ffd640 1 --2- 172.21.15.152:0/3477935454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e79930 0x556c04e90760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.443+0000 7efe527fc640 1 -- 172.21.15.152:0/3477935454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e79930 msgr2=0x556c04e90760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:48.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.443+0000 7efe527fc640 1 --2- 172.21.15.152:0/3477935454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e79930 0x556c04e90760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.443+0000 7efe527fc640 1 -- 172.21.15.152:0/3477935454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e6cf40 msgr2=0x556c04e90220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:48.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.443+0000 7efe527fc640 1 --2- 172.21.15.152:0/3477935454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e6cf40 0x556c04e90220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.443+0000 7efe527fc640 1 -- 172.21.15.152:0/3477935454 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c04e8efd0 con 0x556c04e7bd50 2024-09-11T22:53:48.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.443+0000 7efe52ffd640 1 --2- 172.21.15.152:0/3477935454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e79930 0x556c04e90760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:48.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.443+0000 7efe527fc640 1 --2- 172.21.15.152:0/3477935454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e7bd50 0x556c04e8e7d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7efe48013470 tx=0x7efe4400ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b80cb369b6e3bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:48.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.443+0000 7efe36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:48.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.443+0000 7efe36ff5640 1 -- 172.21.15.152:0/3477935454 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48005020 con 0x556c04e7bd50 2024-09-11T22:53:48.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.443+0000 7efe36ff5640 1 -- 172.21.15.152:0/3477935454 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7efe480051c0 con 0x556c04e7bd50 2024-09-11T22:53:48.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.444+0000 7efe5b910740 1 -- 172.21.15.152:0/3477935454 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe44021d10 con 0x556c04e7bd50 2024-09-11T22:53:48.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.444+0000 7efe36ff5640 1 -- 172.21.15.152:0/3477935454 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efe480053c0 con 0x556c04e7bd50 2024-09-11T22:53:48.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.444+0000 7efe5b910740 10 client.15717 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:48.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.444+0000 7efe5b910740 1 -- 172.21.15.152:0/3477935454 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556c04ef3b70 con 0x556c04e7bd50 2024-09-11T22:53:48.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.445+0000 7efe36ff5640 1 -- 172.21.15.152:0/3477935454 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe4801b040 con 0x556c04e7bd50 2024-09-11T22:53:48.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.445+0000 7efe36ff5640 1 -- 172.21.15.152:0/3477935454 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efe48033020 con 0x556c04e7bd50 2024-09-11T22:53:48.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.445+0000 7efe5b910740 10 client.15717 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:48.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.445+0000 7efe5b910740 10 client.15717 resolve_mds: validated gid 24455 aka daemon mds.b 2024-09-11T22:53:48.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.445+0000 7efe5b910740 20 client.15717 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:48.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.445+0000 7efe5b910740 1 --2- 172.21.15.152:0/3477935454 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556c04ef3fd0 0x556c04ef63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:48.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.445+0000 7efe5b910740 4 client.15717 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-11T22:53:48.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.445+0000 7efe5b910740 1 -- 172.21.15.152:0/3477935454 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x7efe44021d10 con 0x556c04ef3fd0 2024-09-11T22:53:48.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.446+0000 7efe52ffd640 1 --2- 172.21.15.152:0/3477935454 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556c04ef3fd0 0x556c04ef63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:48.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.446+0000 7efe52ffd640 1 --2- 172.21.15.152:0/3477935454 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556c04ef3fd0 0x556c04ef63c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16012d3e767e67ca server_cookie=b0bdee7ac7f2a30a in_seq=0 out_seq=0 2024-09-11T22:53:48.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.446+0000 7efe36ff5640 10 client.15717 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:48.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.453+0000 7efe36ff5640 1 -- 172.21.15.152:0/3477935454 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7efe44021d10 con 0x556c04ef3fd0 2024-09-11T22:53:48.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.453+0000 7efe36ff5640 10 client.15717 handle_command_reply: tid=0 2024-09-11T22:53:48.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.453+0000 7efe5b910740 1 client.15717 shutdown 2024-09-11T22:53:48.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.453+0000 7efe5b910740 1 -- 172.21.15.152:0/3477935454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e7bd50 msgr2=0x556c04e8e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:48.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.453+0000 7efe5b910740 1 --2- 172.21.15.152:0/3477935454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e7bd50 0x556c04e8e7d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7efe48013470 tx=0x7efe4400ae40 comp rx=0 tx=0).stop 2024-09-11T22:53:48.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.453+0000 7efe5b910740 1 -- 172.21.15.152:0/3477935454 shutdown_connections 2024-09-11T22:53:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.453+0000 7efe5b910740 1 --2- 172.21.15.152:0/3477935454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556c04e7bd50 0x556c04e8e7d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.453+0000 7efe5b910740 1 --2- 172.21.15.152:0/3477935454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556c04e79930 0x556c04e90760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.453+0000 7efe5b910740 1 --2- 172.21.15.152:0/3477935454 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556c04ef3fd0 0x556c04ef63c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.453+0000 7efe5b910740 1 --2- 172.21.15.152:0/3477935454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556c04e6cf40 0x556c04e90220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.453+0000 7efe5b910740 1 -- 172.21.15.152:0/3477935454 >> 172.21.15.152:0/3477935454 conn(0x556c04e68cf0 msgr2=0x556c04e6e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.453+0000 7efe36ff5640 0 client.15717 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.454+0000 7efe5b910740 1 -- 172.21.15.152:0/3477935454 shutdown_connections 2024-09-11T22:53:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.454+0000 7efe5b910740 1 -- 172.21.15.152:0/3477935454 wait complete. 2024-09-11T22:53:48.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.454+0000 7efe5b910740 20 client.15717 trim_cache size 0 max 16384 2024-09-11T22:53:48.456 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:53:48.456 INFO:teuthology.orchestra.run.smithi152.stdout: "epoch": 28, 2024-09-11T22:53:48.456 INFO:teuthology.orchestra.run.smithi152.stdout: "leader": 24455, 2024-09-11T22:53:48.456 INFO:teuthology.orchestra.run.smithi152.stdout: "set_version": 2, 2024-09-11T22:53:48.456 INFO:teuthology.orchestra.run.smithi152.stdout: "sets": { 2024-09-11T22:53:48.456 INFO:teuthology.orchestra.run.smithi152.stdout: "7c8da22d": { 2024-09-11T22:53:48.456 INFO:teuthology.orchestra.run.smithi152.stdout: "version": 2, 2024-09-11T22:53:48.456 INFO:teuthology.orchestra.run.smithi152.stdout: "age_ref": 0.0, 2024-09-11T22:53:48.456 INFO:teuthology.orchestra.run.smithi152.stdout: "state": { 2024-09-11T22:53:48.456 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "QUIESCED", 2024-09-11T22:53:48.456 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.0 2024-09-11T22:53:48.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T22:53:48.457 INFO:teuthology.orchestra.run.smithi152.stdout: "timeout": 90.0, 2024-09-11T22:53:48.457 INFO:teuthology.orchestra.run.smithi152.stdout: "expiration": 144.5, 2024-09-11T22:53:48.457 INFO:teuthology.orchestra.run.smithi152.stdout: "members": { 2024-09-11T22:53:48.457 INFO:teuthology.orchestra.run.smithi152.stdout: "file:/": { 2024-09-11T22:53:48.457 INFO:teuthology.orchestra.run.smithi152.stdout: "excluded": false, 2024-09-11T22:53:48.457 INFO:teuthology.orchestra.run.smithi152.stdout: "state": { 2024-09-11T22:53:48.457 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "QUIESCED", 2024-09-11T22:53:48.457 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.0 2024-09-11T22:53:48.457 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:53:48.457 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:53:48.457 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:53:48.458 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:53:48.458 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:53:48.458 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:53:48.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.455+0000 7efe59537640 1 -- 172.21.15.152:0/687724323 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efe3c0a2e00 msgr2=0x7efe3c0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:48.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.455+0000 7efe59537640 1 --2- 172.21.15.152:0/687724323 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efe3c0a2e00 0x7efe3c0a52c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0ef080 tx=0x7efe4c04ff90 comp rx=0 tx=0).stop 2024-09-11T22:53:48.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.455+0000 7efe59537640 1 -- 172.21.15.152:0/687724323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe54106a80 msgr2=0x7efe5419bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:48.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.455+0000 7efe59537640 1 --2- 172.21.15.152:0/687724323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe54106a80 0x7efe5419bb60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efe4c033040 tx=0x7efe4c003970 comp rx=0 tx=0).stop 2024-09-11T22:53:48.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.455+0000 7efe59537640 1 -- 172.21.15.152:0/687724323 shutdown_connections 2024-09-11T22:53:48.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.455+0000 7efe59537640 1 --2- 172.21.15.152:0/687724323 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efe3c0a2e00 0x7efe3c0a52c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.455+0000 7efe59537640 1 --2- 172.21.15.152:0/687724323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe54106a80 0x7efe5419bb60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.455+0000 7efe59537640 1 --2- 172.21.15.152:0/687724323 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe54101420 0x7efe541977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.455+0000 7efe59537640 1 --2- 172.21.15.152:0/687724323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe54100a80 0x7efe54197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:48.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.455+0000 7efe59537640 1 -- 172.21.15.152:0/687724323 >> 172.21.15.152:0/687724323 conn(0x7efe540fc7a0 msgr2=0x7efe540fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:48.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.455+0000 7efe59537640 1 -- 172.21.15.152:0/687724323 shutdown_connections 2024-09-11T22:53:48.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:48.455+0000 7efe59537640 1 -- 172.21.15.152:0/687724323 wait complete. 2024-09-11T22:53:48.461 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:48.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:48 smithi191 ceph-mon[35211]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T22:53:48.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:48 smithi204 ceph-mon[35454]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T22:53:49.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.309+0000 7f88e4103640 1 -- 172.21.15.152:0/826506554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88dc102570 msgr2=0x7f88dc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:49.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.309+0000 7f88e4103640 1 --2- 172.21.15.152:0/826506554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88dc102570 0x7f88dc1029b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009900 tx=0x7f88cc02a480 comp rx=0 tx=0).stop 2024-09-11T22:53:49.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.310+0000 7f88e4103640 1 -- 172.21.15.152:0/826506554 shutdown_connections 2024-09-11T22:53:49.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.310+0000 7f88e4103640 1 --2- 172.21.15.152:0/826506554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88dc102ef0 0x7f88dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.310+0000 7f88e4103640 1 --2- 172.21.15.152:0/826506554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88dc102570 0x7f88dc1029b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.310+0000 7f88e4103640 1 --2- 172.21.15.152:0/826506554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88dc100e20 0x7f88dc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.310+0000 7f88e4103640 1 -- 172.21.15.152:0/826506554 >> 172.21.15.152:0/826506554 conn(0x7f88dc0fc7d0 msgr2=0x7f88dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:49.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.310+0000 7f88e4103640 1 -- 172.21.15.152:0/826506554 shutdown_connections 2024-09-11T22:53:49.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.310+0000 7f88e4103640 1 -- 172.21.15.152:0/826506554 wait complete. 2024-09-11T22:53:49.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.310+0000 7f88e4103640 1 Processor -- start 2024-09-11T22:53:49.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.311+0000 7f88e4103640 1 -- start start 2024-09-11T22:53:49.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.311+0000 7f88e4103640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88dc100e20 0x7f88dc198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.311+0000 7f88e4103640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88dc102570 0x7f88dc198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.311+0000 7f88e4103640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88dc102ef0 0x7f88dc19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.311+0000 7f88e4103640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc110f80 con 0x7f88dc100e20 2024-09-11T22:53:49.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.311+0000 7f88e4103640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc110e00 con 0x7f88dc102570 2024-09-11T22:53:49.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.311+0000 7f88e4103640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc111100 con 0x7f88dc102ef0 2024-09-11T22:53:49.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.311+0000 7f88e1e9f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88dc100e20 0x7f88dc198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.311+0000 7f88e1e9f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88dc100e20 0x7f88dc198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49274/0 (socket says 172.21.15.152:49274) 2024-09-11T22:53:49.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.311+0000 7f88e1e9f640 1 -- 172.21.15.152:0/1888347124 learned_addr learned my addr 172.21.15.152:0/1888347124 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:49.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.311+0000 7f88e169e640 1 --2- 172.21.15.152:0/1888347124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88dc102570 0x7f88dc198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.311+0000 7f88e26a0640 1 --2- 172.21.15.152:0/1888347124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88dc102ef0 0x7f88dc19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.312+0000 7f88e1e9f640 1 -- 172.21.15.152:0/1888347124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88dc102ef0 msgr2=0x7f88dc19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:49.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.312+0000 7f88e1e9f640 1 --2- 172.21.15.152:0/1888347124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88dc102ef0 0x7f88dc19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.312+0000 7f88e1e9f640 1 -- 172.21.15.152:0/1888347124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88dc102570 msgr2=0x7f88dc198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:49.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.312+0000 7f88e1e9f640 1 --2- 172.21.15.152:0/1888347124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88dc102570 0x7f88dc198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.312+0000 7f88e1e9f640 1 -- 172.21.15.152:0/1888347124 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88dc19d520 con 0x7f88dc100e20 2024-09-11T22:53:49.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.312+0000 7f88e1e9f640 1 --2- 172.21.15.152:0/1888347124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88dc100e20 0x7f88dc198570 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f88d800d770 tx=0x7f88d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:49.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.312+0000 7f88d2ffd640 1 -- 172.21.15.152:0/1888347124 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8017070 con 0x7f88dc100e20 2024-09-11T22:53:49.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.312+0000 7f88d2ffd640 1 -- 172.21.15.152:0/1888347124 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88d8005500 con 0x7f88dc100e20 2024-09-11T22:53:49.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.312+0000 7f88e4103640 1 -- 172.21.15.152:0/1888347124 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88dc19d810 con 0x7f88dc100e20 2024-09-11T22:53:49.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.313+0000 7f88d2ffd640 1 -- 172.21.15.152:0/1888347124 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8004e00 con 0x7f88dc100e20 2024-09-11T22:53:49.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.313+0000 7f88e4103640 1 -- 172.21.15.152:0/1888347124 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88dc1a50f0 con 0x7f88dc100e20 2024-09-11T22:53:49.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.315+0000 7f88d2ffd640 1 -- 172.21.15.152:0/1888347124 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f88d8009c30 con 0x7f88dc100e20 2024-09-11T22:53:49.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.316+0000 7f88d2ffd640 1 --2- 172.21.15.152:0/1888347124 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f88c80a2c60 0x7f88c80a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.316+0000 7f88d2ffd640 1 -- 172.21.15.152:0/1888347124 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f88d80cebb0 con 0x7f88dc100e20 2024-09-11T22:53:49.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.316+0000 7f88e169e640 1 --2- 172.21.15.152:0/1888347124 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f88c80a2c60 0x7f88c80a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.316+0000 7f88e64dc740 1 Processor -- start 2024-09-11T22:53:49.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.317+0000 7f88e64dc740 1 -- start start 2024-09-11T22:53:49.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.317+0000 7f88e64dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccb49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.317+0000 7f88e64dc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccaf280 0x55709ccaf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.317+0000 7f88e64dc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccafc00 0x55709ccbb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.317+0000 7f88e64dc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55709cc0b2f0 con 0x55709ccaf280 2024-09-11T22:53:49.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.317+0000 7f88e64dc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55709cbb2cf0 con 0x55709ccafc00 2024-09-11T22:53:49.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.317+0000 7f88e64dc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55709cc072b0 con 0x55709ccb45d0 2024-09-11T22:53:49.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.317+0000 7f88e169e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccaf280 0x55709ccaf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.317+0000 7f88e169e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccaf280 0x55709ccaf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49286/0 (socket says 172.21.15.152:49286) 2024-09-11T22:53:49.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.317+0000 7f88e169e640 1 -- 172.21.15.152:0/4024421593 learned_addr learned my addr 172.21.15.152:0/4024421593 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:49.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.317+0000 7f88e26a0640 1 --2- 172.21.15.152:0/4024421593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccafc00 0x55709ccbb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.317+0000 7f88e1e9f640 1 --2- 172.21.15.152:0/4024421593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccb49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.318+0000 7f88e26a0640 1 -- 172.21.15.152:0/4024421593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 msgr2=0x55709ccb49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:49.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.318+0000 7f88e26a0640 1 --2- 172.21.15.152:0/4024421593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccb49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.318+0000 7f88e26a0640 1 -- 172.21.15.152:0/4024421593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccaf280 msgr2=0x55709ccaf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:49.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.318+0000 7f88e26a0640 1 --2- 172.21.15.152:0/4024421593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccaf280 0x55709ccaf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.318+0000 7f88e26a0640 1 -- 172.21.15.152:0/4024421593 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55709cb973a0 con 0x55709ccafc00 2024-09-11T22:53:49.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.318+0000 7f88e169e640 1 --2- 172.21.15.152:0/4024421593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccaf280 0x55709ccaf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:49.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.318+0000 7f88e169e640 1 --2- 172.21.15.152:0/1888347124 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f88c80a2c60 0x7f88c80a5120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009230 tx=0x7f88cc0058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:49.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.318+0000 7f88e26a0640 1 --2- 172.21.15.152:0/4024421593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccafc00 0x55709ccbb330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f88d400ad50 tx=0x7f88d400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f47c5764ccadbf1d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:49.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.318+0000 7f88c7fff640 1 -- 172.21.15.152:0/4024421593 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4003b70 con 0x55709ccafc00 2024-09-11T22:53:49.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.318+0000 7f88c7fff640 1 -- 172.21.15.152:0/4024421593 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88d400ae40 con 0x55709ccafc00 2024-09-11T22:53:49.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.318+0000 7f88c7fff640 1 -- 172.21.15.152:0/4024421593 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4010a60 con 0x55709ccafc00 2024-09-11T22:53:49.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.319+0000 7f88e64dc740 1 -- 172.21.15.152:0/4024421593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccafc00 msgr2=0x55709ccbb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:49.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.319+0000 7f88e64dc740 1 --2- 172.21.15.152:0/4024421593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccafc00 0x55709ccbb330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f88d400ad50 tx=0x7f88d400f790 comp rx=0 tx=0).stop 2024-09-11T22:53:49.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.319+0000 7f88e64dc740 1 -- 172.21.15.152:0/4024421593 shutdown_connections 2024-09-11T22:53:49.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.319+0000 7f88e64dc740 1 --2- 172.21.15.152:0/4024421593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccafc00 0x55709ccbb330 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.319+0000 7f88e64dc740 1 --2- 172.21.15.152:0/4024421593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccaf280 0x55709ccaf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.319+0000 7f88e64dc740 1 --2- 172.21.15.152:0/4024421593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccb49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.319+0000 7f88e64dc740 1 -- 172.21.15.152:0/4024421593 >> 172.21.15.152:0/4024421593 conn(0x55709cc6e310 msgr2=0x55709cca16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:49.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.320+0000 7f88e64dc740 1 -- 172.21.15.152:0/4024421593 shutdown_connections 2024-09-11T22:53:49.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.320+0000 7f88e64dc740 1 -- 172.21.15.152:0/4024421593 wait complete. 2024-09-11T22:53:49.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.320+0000 7f88e64dc740 1 -- start start 2024-09-11T22:53:49.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.320+0000 7f88e64dc740 1 Processor -- start 2024-09-11T22:53:49.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e64dc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccaf280 0x55709ccbb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e64dc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccafc00 0x55709ccb84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e64dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccb8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e64dc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55709ccbb8d0 con 0x55709ccaf280 2024-09-11T22:53:49.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e64dc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55709cbc6060 con 0x55709ccafc00 2024-09-11T22:53:49.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e64dc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55709ccbba50 con 0x55709ccb45d0 2024-09-11T22:53:49.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e64dc740 10 client.0 _ll_register_callbacks cb 0x55709c9198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:49.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e1e9f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccaf280 0x55709ccbb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e1e9f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccaf280 0x55709ccbb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49302/0 (socket says 172.21.15.152:49302) 2024-09-11T22:53:49.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e1e9f640 1 -- 172.21.15.152:0/725266913 learned_addr learned my addr 172.21.15.152:0/725266913 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:49.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e169e640 1 --2- 172.21.15.152:0/725266913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccafc00 0x55709ccb84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e26a0640 1 --2- 172.21.15.152:0/725266913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccb8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e1e9f640 1 -- 172.21.15.152:0/725266913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 msgr2=0x55709ccb8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:49.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e1e9f640 1 --2- 172.21.15.152:0/725266913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccb8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e1e9f640 1 -- 172.21.15.152:0/725266913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccafc00 msgr2=0x55709ccb84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:49.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e1e9f640 1 --2- 172.21.15.152:0/725266913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccafc00 0x55709ccb84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e1e9f640 1 -- 172.21.15.152:0/725266913 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55709c97c3c0 con 0x55709ccaf280 2024-09-11T22:53:49.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e26a0640 1 --2- 172.21.15.152:0/725266913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccb8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:49.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e169e640 1 --2- 172.21.15.152:0/725266913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccafc00 0x55709ccb84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:49.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88e1e9f640 1 --2- 172.21.15.152:0/725266913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccaf280 0x55709ccbb230 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f88d80cc320 tx=0x7f88d80056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171ce887f95928ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:49.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.321+0000 7f88c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:49.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.322+0000 7f88c57fa640 1 -- 172.21.15.152:0/725266913 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d80b3070 con 0x55709ccaf280 2024-09-11T22:53:49.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.322+0000 7f88c57fa640 1 -- 172.21.15.152:0/725266913 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f88d80a3790 con 0x55709ccaf280 2024-09-11T22:53:49.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.322+0000 7f88e64dc740 1 -- 172.21.15.152:0/725266913 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55709cc18860 con 0x55709ccaf280 2024-09-11T22:53:49.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.322+0000 7f88c57fa640 1 -- 172.21.15.152:0/725266913 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f88d8020040 con 0x55709ccaf280 2024-09-11T22:53:49.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.322+0000 7f88e64dc740 10 client.15729 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:49.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.322+0000 7f88e64dc740 1 -- 172.21.15.152:0/725266913 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55709ccc73a0 con 0x55709ccaf280 2024-09-11T22:53:49.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.322+0000 7f88c57fa640 1 -- 172.21.15.152:0/725266913 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55709cc18860 con 0x55709ccaf280 2024-09-11T22:53:49.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.322+0000 7f88c57fa640 1 -- 172.21.15.152:0/725266913 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f88d80b8070 con 0x55709ccaf280 2024-09-11T22:53:49.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.322+0000 7f88e64dc740 10 client.15729 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:49.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.322+0000 7f88e64dc740 10 client.15729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:53:49.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.322+0000 7f88e64dc740 20 client.15729 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:49.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.322+0000 7f88e64dc740 1 --2- 172.21.15.152:0/725266913 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55709ccc87b0 0x55709cccabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.323+0000 7f88e64dc740 4 client.15729 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:49.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.323+0000 7f88e64dc740 1 -- 172.21.15.152:0/725266913 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55709cc12d70 con 0x55709ccc87b0 2024-09-11T22:53:49.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.323+0000 7f88e169e640 1 --2- 172.21.15.152:0/725266913 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55709ccc87b0 0x55709cccabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.323+0000 7f88e169e640 1 --2- 172.21.15.152:0/725266913 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55709ccc87b0 0x55709cccabc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9441abba75de7707 server_cookie=c462ece6e3d5c80f in_seq=0 out_seq=0 2024-09-11T22:53:49.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.323+0000 7f88c57fa640 10 client.15729 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:53:49.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.324+0000 7f88c57fa640 1 -- 172.21.15.152:0/725266913 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55709cc12d70 con 0x55709ccc87b0 2024-09-11T22:53:49.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.324+0000 7f88c57fa640 10 client.15729 handle_command_reply: tid=0 2024-09-11T22:53:49.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.324+0000 7f88e64dc740 1 client.15729 shutdown 2024-09-11T22:53:49.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.324+0000 7f88e64dc740 1 -- 172.21.15.152:0/725266913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccaf280 msgr2=0x55709ccbb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:49.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.324+0000 7f88e64dc740 1 --2- 172.21.15.152:0/725266913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccaf280 0x55709ccbb230 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f88d80cc320 tx=0x7f88d80056b0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.324+0000 7f88e64dc740 1 -- 172.21.15.152:0/725266913 shutdown_connections 2024-09-11T22:53:49.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.324+0000 7f88e64dc740 1 --2- 172.21.15.152:0/725266913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccb8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.324+0000 7f88e64dc740 1 --2- 172.21.15.152:0/725266913 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55709ccc87b0 0x55709cccabc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.324+0000 7f88c57fa640 0 client.15729 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:53:49.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.324+0000 7f88e64dc740 1 --2- 172.21.15.152:0/725266913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccafc00 0x55709ccb84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.324+0000 7f88e64dc740 1 --2- 172.21.15.152:0/725266913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccaf280 0x55709ccbb230 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.324+0000 7f88e64dc740 1 -- 172.21.15.152:0/725266913 >> 172.21.15.152:0/725266913 conn(0x55709cc6e310 msgr2=0x55709ccaea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:49.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.325+0000 7f88e64dc740 1 -- 172.21.15.152:0/725266913 shutdown_connections 2024-09-11T22:53:49.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.325+0000 7f88e64dc740 1 -- 172.21.15.152:0/725266913 wait complete. 2024-09-11T22:53:49.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.325+0000 7f88e64dc740 20 client.15729 trim_cache size 0 max 16384 2024-09-11T22:53:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.343+0000 7f88e64dc740 1 Processor -- start 2024-09-11T22:53:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.343+0000 7f88e64dc740 1 -- start start 2024-09-11T22:53:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.343+0000 7f88e64dc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccaf280 0x55709ccc4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.343+0000 7f88e64dc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccafc00 0x55709ccae980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.343+0000 7f88e64dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccd0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.344+0000 7f88e64dc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1ab240 con 0x55709ccafc00 2024-09-11T22:53:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.344+0000 7f88e64dc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1acd80 con 0x55709ccaf280 2024-09-11T22:53:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.344+0000 7f88e64dc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1ab910 con 0x55709ccb45d0 2024-09-11T22:53:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.344+0000 7f88e169e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccafc00 0x55709ccae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.344+0000 7f88e169e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccafc00 0x55709ccae980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49306/0 (socket says 172.21.15.152:49306) 2024-09-11T22:53:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.344+0000 7f88e169e640 1 -- 172.21.15.152:0/2021061719 learned_addr learned my addr 172.21.15.152:0/2021061719 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.344+0000 7f88e1e9f640 1 --2- 172.21.15.152:0/2021061719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccaf280 0x55709ccc4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.344+0000 7f88e26a0640 1 --2- 172.21.15.152:0/2021061719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccd0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.344+0000 7f88e169e640 1 -- 172.21.15.152:0/2021061719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 msgr2=0x55709ccd0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.344+0000 7f88e169e640 1 --2- 172.21.15.152:0/2021061719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccd0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.344+0000 7f88e169e640 1 -- 172.21.15.152:0/2021061719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccaf280 msgr2=0x55709ccc4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.344+0000 7f88e169e640 1 --2- 172.21.15.152:0/2021061719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccaf280 0x55709ccc4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.344+0000 7f88e169e640 1 -- 172.21.15.152:0/2021061719 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55709ccb6a50 con 0x55709ccafc00 2024-09-11T22:53:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.344+0000 7f88e26a0640 1 --2- 172.21.15.152:0/2021061719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccd0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.344+0000 7f88e1e9f640 1 --2- 172.21.15.152:0/2021061719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccaf280 0x55709ccc4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:49.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.344+0000 7f88e169e640 1 --2- 172.21.15.152:0/2021061719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccafc00 0x55709ccae980 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f88cc005940 tx=0x7f88cc043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cd7cf42c594b38 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:49.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.345+0000 7f88c77fe640 1 -- 172.21.15.152:0/2021061719 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc05b070 con 0x55709ccafc00 2024-09-11T22:53:49.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.345+0000 7f88c77fe640 1 -- 172.21.15.152:0/2021061719 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88cc053bd0 con 0x55709ccafc00 2024-09-11T22:53:49.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.345+0000 7f88c77fe640 1 -- 172.21.15.152:0/2021061719 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc035050 con 0x55709ccafc00 2024-09-11T22:53:49.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.345+0000 7f88e64dc740 1 -- 172.21.15.152:0/2021061719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccafc00 msgr2=0x55709ccae980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.345+0000 7f88e64dc740 1 --2- 172.21.15.152:0/2021061719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccafc00 0x55709ccae980 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f88cc005940 tx=0x7f88cc043000 comp rx=0 tx=0).stop 2024-09-11T22:53:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.346+0000 7f88e64dc740 1 -- 172.21.15.152:0/2021061719 shutdown_connections 2024-09-11T22:53:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.346+0000 7f88e64dc740 1 --2- 172.21.15.152:0/2021061719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccd0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.346+0000 7f88e64dc740 1 --2- 172.21.15.152:0/2021061719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccafc00 0x55709ccae980 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.346+0000 7f88e64dc740 1 --2- 172.21.15.152:0/2021061719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccaf280 0x55709ccc4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.346+0000 7f88e64dc740 1 -- 172.21.15.152:0/2021061719 >> 172.21.15.152:0/2021061719 conn(0x55709cc6e310 msgr2=0x55709ccae270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.346+0000 7f88e64dc740 1 -- 172.21.15.152:0/2021061719 shutdown_connections 2024-09-11T22:53:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.346+0000 7f88e64dc740 1 -- 172.21.15.152:0/2021061719 wait complete. 2024-09-11T22:53:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.346+0000 7f88e64dc740 1 -- start start 2024-09-11T22:53:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.346+0000 7f88e64dc740 1 Processor -- start 2024-09-11T22:53:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e64dc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccaf280 0x55709ccc0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e64dc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccafc00 0x55709ccc11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e169e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccafc00 0x55709ccc11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e169e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccafc00 0x55709ccc11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49320/0 (socket says 172.21.15.152:49320) 2024-09-11T22:53:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e169e640 1 -- 172.21.15.152:0/3166499888 learned_addr learned my addr 172.21.15.152:0/3166499888 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e64dc740 1 --2- 172.21.15.152:0/3166499888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccbdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e64dc740 1 -- 172.21.15.152:0/3166499888 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1ac6b0 con 0x55709ccafc00 2024-09-11T22:53:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e64dc740 1 -- 172.21.15.152:0/3166499888 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1abfe0 con 0x55709ccaf280 2024-09-11T22:53:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e64dc740 1 -- 172.21.15.152:0/3166499888 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1a9220 con 0x55709ccb45d0 2024-09-11T22:53:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e64dc740 10 client.0 _ll_register_callbacks cb 0x7f88d809ae80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e1e9f640 1 --2- 172.21.15.152:0/3166499888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccaf280 0x55709ccc0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e169e640 1 -- 172.21.15.152:0/3166499888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 msgr2=0x55709ccbdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e26a0640 1 --2- 172.21.15.152:0/3166499888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccbdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e169e640 1 --2- 172.21.15.152:0/3166499888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccbdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e169e640 1 -- 172.21.15.152:0/3166499888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccaf280 msgr2=0x55709ccc0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e169e640 1 --2- 172.21.15.152:0/3166499888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccaf280 0x55709ccc0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.347+0000 7f88e169e640 1 -- 172.21.15.152:0/3166499888 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55709ccbe1b0 con 0x55709ccafc00 2024-09-11T22:53:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.348+0000 7f88e169e640 1 --2- 172.21.15.152:0/3166499888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccafc00 0x55709ccc11e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f88cc02a9a0 tx=0x7f88cc02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca84caf67e8f60 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.348+0000 7f88c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.348+0000 7f88c57fa640 1 -- 172.21.15.152:0/3166499888 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc05b070 con 0x55709ccafc00 2024-09-11T22:53:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.348+0000 7f88c57fa640 1 -- 172.21.15.152:0/3166499888 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f88cc031740 con 0x55709ccafc00 2024-09-11T22:53:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.348+0000 7f88e64dc740 1 -- 172.21.15.152:0/3166499888 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88cc053bd0 con 0x55709ccafc00 2024-09-11T22:53:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.348+0000 7f88c57fa640 1 -- 172.21.15.152:0/3166499888 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f88cc035040 con 0x55709ccafc00 2024-09-11T22:53:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.348+0000 7f88e64dc740 10 client.15735 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.348+0000 7f88e64dc740 1 -- 172.21.15.152:0/3166499888 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55709cd33ae0 con 0x55709ccafc00 2024-09-11T22:53:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.348+0000 7f88c57fa640 1 -- 172.21.15.152:0/3166499888 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88cc0539d0 con 0x55709ccafc00 2024-09-11T22:53:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.349+0000 7f88c57fa640 1 -- 172.21.15.152:0/3166499888 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f88cc004750 con 0x55709ccafc00 2024-09-11T22:53:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.349+0000 7f88e64dc740 10 client.15735 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.349+0000 7f88e64dc740 10 client.15735 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:53:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.349+0000 7f88e64dc740 20 client.15735 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.349+0000 7f88e64dc740 1 --2- 172.21.15.152:0/3166499888 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55709cd34200 0x55709cd36620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.349+0000 7f88e64dc740 4 client.15735 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:53:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.349+0000 7f88e64dc740 1 -- 172.21.15.152:0/3166499888 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f88cc053bd0 con 0x55709cd34200 2024-09-11T22:53:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.349+0000 7f88e1e9f640 1 --2- 172.21.15.152:0/3166499888 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55709cd34200 0x55709cd36620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.350+0000 7f88e1e9f640 1 --2- 172.21.15.152:0/3166499888 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55709cd34200 0x55709cd36620 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37587e8bc856f48a server_cookie=efc58dea66bdcd5e in_seq=0 out_seq=0 2024-09-11T22:53:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.350+0000 7f88c57fa640 10 client.15735 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:53:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.350+0000 7f88c57fa640 1 -- 172.21.15.152:0/3166499888 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f88d8015020 con 0x55709cd34200 2024-09-11T22:53:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.350+0000 7f88c57fa640 10 client.15735 handle_command_reply: tid=0 2024-09-11T22:53:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.350+0000 7f88e64dc740 1 client.15735 shutdown 2024-09-11T22:53:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.350+0000 7f88e64dc740 1 -- 172.21.15.152:0/3166499888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccafc00 msgr2=0x55709ccc11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.350+0000 7f88e64dc740 1 --2- 172.21.15.152:0/3166499888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccafc00 0x55709ccc11e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f88cc02a9a0 tx=0x7f88cc02a480 comp rx=0 tx=0).stop 2024-09-11T22:53:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.350+0000 7f88e64dc740 1 -- 172.21.15.152:0/3166499888 shutdown_connections 2024-09-11T22:53:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.350+0000 7f88e64dc740 1 --2- 172.21.15.152:0/3166499888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55709ccb45d0 0x55709ccbdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.350+0000 7f88e64dc740 1 --2- 172.21.15.152:0/3166499888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55709ccafc00 0x55709ccc11e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.350+0000 7f88e64dc740 1 --2- 172.21.15.152:0/3166499888 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55709cd34200 0x55709cd36620 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.350+0000 7f88e64dc740 1 --2- 172.21.15.152:0/3166499888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55709ccaf280 0x55709ccc0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.350+0000 7f88c57fa640 0 client.15735 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:53:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.350+0000 7f88e64dc740 1 -- 172.21.15.152:0/3166499888 >> 172.21.15.152:0/3166499888 conn(0x55709cc6e310 msgr2=0x55709ccaea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.350+0000 7f88e64dc740 1 -- 172.21.15.152:0/3166499888 shutdown_connections 2024-09-11T22:53:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.351+0000 7f88e64dc740 1 -- 172.21.15.152:0/3166499888 wait complete. 2024-09-11T22:53:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.351+0000 7f88e64dc740 20 client.15735 trim_cache size 0 max 16384 2024-09-11T22:53:49.355 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:53:49.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.351+0000 7f88e4103640 1 -- 172.21.15.152:0/1888347124 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f88c80a2c60 msgr2=0x7f88c80a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:49.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.351+0000 7f88e4103640 1 --2- 172.21.15.152:0/1888347124 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f88c80a2c60 0x7f88c80a5120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009230 tx=0x7f88cc0058d0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.351+0000 7f88e4103640 1 -- 172.21.15.152:0/1888347124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88dc100e20 msgr2=0x7f88dc198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:49.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.351+0000 7f88e4103640 1 --2- 172.21.15.152:0/1888347124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88dc100e20 0x7f88dc198570 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f88d800d770 tx=0x7f88d800dc30 comp rx=0 tx=0).stop 2024-09-11T22:53:49.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.351+0000 7f88e4103640 1 -- 172.21.15.152:0/1888347124 shutdown_connections 2024-09-11T22:53:49.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.351+0000 7f88e4103640 1 --2- 172.21.15.152:0/1888347124 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f88c80a2c60 0x7f88c80a5120 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.351+0000 7f88e4103640 1 --2- 172.21.15.152:0/1888347124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88dc102ef0 0x7f88dc19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.351+0000 7f88e4103640 1 --2- 172.21.15.152:0/1888347124 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88dc102570 0x7f88dc198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.351+0000 7f88e4103640 1 --2- 172.21.15.152:0/1888347124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88dc100e20 0x7f88dc198570 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:49.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.351+0000 7f88e4103640 1 -- 172.21.15.152:0/1888347124 >> 172.21.15.152:0/1888347124 conn(0x7f88dc0fc7d0 msgr2=0x7f88dc06a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:49.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.352+0000 7f88e4103640 1 -- 172.21.15.152:0/1888347124 shutdown_connections 2024-09-11T22:53:49.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:49.352+0000 7f88e4103640 1 -- 172.21.15.152:0/1888347124 wait complete. 2024-09-11T22:53:49.767 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7c8da22d', quiesce took 9.8 seconds. Will release after: 14.7 2024-09-11T22:53:50.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:50 smithi152 ceph-mon[31181]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-11T22:53:50.455 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:53:50.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:50 smithi191 ceph-mon[35211]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-11T22:53:50.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:50 smithi204 ceph-mon[35454]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-11T22:53:50.734 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:51.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.299+0000 7f9d292e6640 1 -- 172.21.15.152:0/869160764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d24100ca0 msgr2=0x7f9d24106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:51.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.299+0000 7f9d292e6640 1 --2- 172.21.15.152:0/869160764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d24100ca0 0x7f9d24106840 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009930 tx=0x7f9d1802a420 comp rx=0 tx=0).stop 2024-09-11T22:53:51.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.299+0000 7f9d292e6640 1 -- 172.21.15.152:0/869160764 shutdown_connections 2024-09-11T22:53:51.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.299+0000 7f9d292e6640 1 --2- 172.21.15.152:0/869160764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d24106f50 0x7f9d2410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.299+0000 7f9d292e6640 1 --2- 172.21.15.152:0/869160764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d24100ca0 0x7f9d24106840 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.299+0000 7f9d292e6640 1 --2- 172.21.15.152:0/869160764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d241002f0 0x7f9d241006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.300+0000 7f9d292e6640 1 -- 172.21.15.152:0/869160764 >> 172.21.15.152:0/869160764 conn(0x7f9d24077220 msgr2=0x7f9d240fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:51.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.300+0000 7f9d292e6640 1 -- 172.21.15.152:0/869160764 shutdown_connections 2024-09-11T22:53:51.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.300+0000 7f9d292e6640 1 -- 172.21.15.152:0/869160764 wait complete. 2024-09-11T22:53:51.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.300+0000 7f9d292e6640 1 Processor -- start 2024-09-11T22:53:51.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.301+0000 7f9d292e6640 1 -- start start 2024-09-11T22:53:51.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.301+0000 7f9d292e6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d241002f0 0x7f9d2419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.301+0000 7f9d292e6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d24100ca0 0x7f9d2419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.301+0000 7f9d292e6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d24106f50 0x7f9d2419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.301+0000 7f9d292e6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9d24111fa0 con 0x7f9d241002f0 2024-09-11T22:53:51.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.301+0000 7f9d237fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d24106f50 0x7f9d2419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.301+0000 7f9d22ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d241002f0 0x7f9d2419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.301+0000 7f9d237fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d24106f50 0x7f9d2419efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:45084/0 (socket says 172.21.15.152:45084) 2024-09-11T22:53:51.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.301+0000 7f9d237fe640 1 -- 172.21.15.152:0/2933239143 learned_addr learned my addr 172.21.15.152:0/2933239143 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:51.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.301+0000 7f9d292e6640 1 -- 172.21.15.152:0/2933239143 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9d24111e20 con 0x7f9d24106f50 2024-09-11T22:53:51.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.301+0000 7f9d227fc640 1 --2- 172.21.15.152:0/2933239143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d24100ca0 0x7f9d2419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.301+0000 7f9d292e6640 1 -- 172.21.15.152:0/2933239143 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d24112120 con 0x7f9d24100ca0 2024-09-11T22:53:51.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.302+0000 7f9d237fe640 1 -- 172.21.15.152:0/2933239143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d24100ca0 msgr2=0x7f9d2419ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:51.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.302+0000 7f9d237fe640 1 --2- 172.21.15.152:0/2933239143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d24100ca0 0x7f9d2419ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.302+0000 7f9d237fe640 1 -- 172.21.15.152:0/2933239143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d241002f0 msgr2=0x7f9d2419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:51.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.302+0000 7f9d237fe640 1 --2- 172.21.15.152:0/2933239143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d241002f0 0x7f9d2419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.302+0000 7f9d237fe640 1 -- 172.21.15.152:0/2933239143 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2419f6b0 con 0x7f9d24106f50 2024-09-11T22:53:51.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.302+0000 7f9d22ffd640 1 --2- 172.21.15.152:0/2933239143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d241002f0 0x7f9d2419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:51.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.302+0000 7f9d237fe640 1 --2- 172.21.15.152:0/2933239143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d24106f50 0x7f9d2419efd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009fd0 tx=0x7f9d1c00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:51.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.302+0000 7f9d0bfff640 1 -- 172.21.15.152:0/2933239143 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c017070 con 0x7f9d24106f50 2024-09-11T22:53:51.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.303+0000 7f9d292e6640 1 -- 172.21.15.152:0/2933239143 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d2419f9a0 con 0x7f9d24106f50 2024-09-11T22:53:51.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.303+0000 7f9d292e6640 1 -- 172.21.15.152:0/2933239143 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d241a7280 con 0x7f9d24106f50 2024-09-11T22:53:51.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.303+0000 7f9d0bfff640 1 -- 172.21.15.152:0/2933239143 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d1c003e50 con 0x7f9d24106f50 2024-09-11T22:53:51.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.304+0000 7f9d0bfff640 1 -- 172.21.15.152:0/2933239143 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c0106c0 con 0x7f9d24106f50 2024-09-11T22:53:51.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.305+0000 7f9d2b6bf740 1 Processor -- start 2024-09-11T22:53:51.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.306+0000 7f9d0bfff640 1 -- 172.21.15.152:0/2933239143 <== mon.2 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f9d1c010920 con 0x7f9d24106f50 2024-09-11T22:53:51.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.307+0000 7f9d2b6bf740 1 -- start start 2024-09-11T22:53:51.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.307+0000 7f9d0bfff640 1 --2- 172.21.15.152:0/2933239143 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9d0c0a2e00 0x7f9d0c0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.307+0000 7f9d0bfff640 1 -- 172.21.15.152:0/2933239143 <== mon.2 v2:172.21.15.191:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f9d1c0cf0b0 con 0x7f9d24106f50 2024-09-11T22:53:51.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.307+0000 7f9d227fc640 1 --2- 172.21.15.152:0/2933239143 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9d0c0a2e00 0x7f9d0c0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.307+0000 7f9d2b6bf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55946240ab40 0x559462409090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.307+0000 7f9d2b6bf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594624095d0 0x559462409a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.307+0000 7f9d2b6bf740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559462414370 0x559462416760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.307+0000 7f9d2b6bf740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55946236f2f0 con 0x559462414370 2024-09-11T22:53:51.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.307+0000 7f9d2b6bf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559462316cf0 con 0x5594624095d0 2024-09-11T22:53:51.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.307+0000 7f9d2b6bf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55946236b2b0 con 0x55946240ab40 2024-09-11T22:53:51.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.307+0000 7f9d227fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594624095d0 0x559462409a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.307+0000 7f9d227fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594624095d0 0x559462409a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:45090/0 (socket says 172.21.15.152:45090) 2024-09-11T22:53:51.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.307+0000 7f9d227fc640 1 -- 172.21.15.152:0/3152404728 learned_addr learned my addr 172.21.15.152:0/3152404728 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:51.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.307+0000 7f9d22ffd640 1 --2- 172.21.15.152:0/3152404728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55946240ab40 0x559462409090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.308+0000 7f9d237fe640 1 --2- 172.21.15.152:0/3152404728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559462414370 0x559462416760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.308+0000 7f9d227fc640 1 --2- 172.21.15.152:0/2933239143 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9d0c0a2e00 0x7f9d0c0a52c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9d18000c00 tx=0x7f9d18032f20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:51.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.308+0000 7f9d227fc640 1 -- 172.21.15.152:0/3152404728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55946240ab40 msgr2=0x559462409090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:51.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.308+0000 7f9d227fc640 1 --2- 172.21.15.152:0/3152404728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55946240ab40 0x559462409090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.308+0000 7f9d227fc640 1 -- 172.21.15.152:0/3152404728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559462414370 msgr2=0x559462416760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:51.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.308+0000 7f9d227fc640 1 --2- 172.21.15.152:0/3152404728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559462414370 0x559462416760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.308+0000 7f9d227fc640 1 -- 172.21.15.152:0/3152404728 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594622fb3a0 con 0x5594624095d0 2024-09-11T22:53:51.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.308+0000 7f9d237fe640 1 --2- 172.21.15.152:0/3152404728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559462414370 0x559462416760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:51.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.308+0000 7f9d227fc640 1 --2- 172.21.15.152:0/3152404728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594624095d0 0x559462409a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009930 tx=0x7f9d18047000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7327e08eb171930e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:51.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.308+0000 7f9d097fa640 1 -- 172.21.15.152:0/3152404728 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18035050 con 0x5594624095d0 2024-09-11T22:53:51.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.308+0000 7f9d097fa640 1 -- 172.21.15.152:0/3152404728 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d1804e070 con 0x5594624095d0 2024-09-11T22:53:51.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.308+0000 7f9d097fa640 1 -- 172.21.15.152:0/3152404728 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d180035f0 con 0x5594624095d0 2024-09-11T22:53:51.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.309+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/3152404728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594624095d0 msgr2=0x559462409a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:51.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.309+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/3152404728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594624095d0 0x559462409a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009930 tx=0x7f9d18047000 comp rx=0 tx=0).stop 2024-09-11T22:53:51.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.309+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/3152404728 shutdown_connections 2024-09-11T22:53:51.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.309+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/3152404728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559462414370 0x559462416760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.309+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/3152404728 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594624095d0 0x559462409a10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.309+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/3152404728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55946240ab40 0x559462409090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.309+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/3152404728 >> 172.21.15.152:0/3152404728 conn(0x5594623d2310 msgr2=0x559462405c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:51.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.310+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/3152404728 shutdown_connections 2024-09-11T22:53:51.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.310+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/3152404728 wait complete. 2024-09-11T22:53:51.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.310+0000 7f9d2b6bf740 1 -- start start 2024-09-11T22:53:51.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.310+0000 7f9d2b6bf740 1 Processor -- start 2024-09-11T22:53:51.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.310+0000 7f9d2b6bf740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5594624095d0 0x559462413ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.310+0000 7f9d2b6bf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55946240ab40 0x559462410f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.310+0000 7f9d2b6bf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559462414370 0x559462411490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.310+0000 7f9d2b6bf740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x559462417010 con 0x5594624095d0 2024-09-11T22:53:51.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.310+0000 7f9d2b6bf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55946232a060 con 0x559462414370 2024-09-11T22:53:51.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.310+0000 7f9d2b6bf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559462417190 con 0x55946240ab40 2024-09-11T22:53:51.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.310+0000 7f9d2b6bf740 10 client.0 _ll_register_callbacks cb 0x55946207d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:51.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.311+0000 7f9d22ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5594624095d0 0x559462413ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.311+0000 7f9d22ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5594624095d0 0x559462413ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49356/0 (socket says 172.21.15.152:49356) 2024-09-11T22:53:51.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.311+0000 7f9d22ffd640 1 -- 172.21.15.152:0/3984669341 learned_addr learned my addr 172.21.15.152:0/3984669341 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:51.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.311+0000 7f9d237fe640 1 --2- 172.21.15.152:0/3984669341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559462414370 0x559462411490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.311+0000 7f9d227fc640 1 --2- 172.21.15.152:0/3984669341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55946240ab40 0x559462410f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.311+0000 7f9d22ffd640 1 -- 172.21.15.152:0/3984669341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55946240ab40 msgr2=0x559462410f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:51.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.311+0000 7f9d22ffd640 1 --2- 172.21.15.152:0/3984669341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55946240ab40 0x559462410f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.311+0000 7f9d22ffd640 1 -- 172.21.15.152:0/3984669341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559462414370 msgr2=0x559462411490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:51.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.311+0000 7f9d22ffd640 1 --2- 172.21.15.152:0/3984669341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559462414370 0x559462411490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.311+0000 7f9d22ffd640 1 -- 172.21.15.152:0/3984669341 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594620e03c0 con 0x5594624095d0 2024-09-11T22:53:51.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.311+0000 7f9d227fc640 1 --2- 172.21.15.152:0/3984669341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55946240ab40 0x559462410f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:51.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.311+0000 7f9d22ffd640 1 --2- 172.21.15.152:0/3984669341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5594624095d0 0x559462413ca0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9d14011b20 tx=0x7f9d14010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a082c29b4f7a517 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:51.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.311+0000 7f9d06ff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:53:51.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.312+0000 7f9d06ff5640 1 -- 172.21.15.152:0/3984669341 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1401f060 con 0x5594624095d0 2024-09-11T22:53:51.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.312+0000 7f9d06ff5640 1 -- 172.21.15.152:0/3984669341 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f9d14010be0 con 0x5594624095d0 2024-09-11T22:53:51.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.312+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/3984669341 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d1804e070 con 0x5594624095d0 2024-09-11T22:53:51.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.312+0000 7f9d06ff5640 1 -- 172.21.15.152:0/3984669341 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9d14010de0 con 0x5594624095d0 2024-09-11T22:53:51.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.312+0000 7f9d2b6bf740 10 client.15747 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:51.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.312+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/3984669341 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55946242b2d0 con 0x5594624095d0 2024-09-11T22:53:51.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.312+0000 7f9d06ff5640 1 -- 172.21.15.152:0/3984669341 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d1804e070 con 0x5594624095d0 2024-09-11T22:53:51.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.313+0000 7f9d06ff5640 1 -- 172.21.15.152:0/3984669341 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9d1400f590 con 0x5594624095d0 2024-09-11T22:53:51.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.313+0000 7f9d2b6bf740 10 client.15747 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:51.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.313+0000 7f9d2b6bf740 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:51.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.313+0000 7f9d2b6bf740 20 client.15747 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:51.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.314+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/3984669341 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55946242c790 0x55946242eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.314+0000 7f9d2b6bf740 4 client.15747 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:51.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.314+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/3984669341 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55946237c860 con 0x55946242c790 2024-09-11T22:53:51.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.314+0000 7f9d227fc640 1 --2- 172.21.15.152:0/3984669341 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55946242c790 0x55946242eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.314+0000 7f9d227fc640 1 --2- 172.21.15.152:0/3984669341 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55946242c790 0x55946242eba0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2658edd9176b765c server_cookie=4f918eba7c1e09a0 in_seq=0 out_seq=0 2024-09-11T22:53:51.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.314+0000 7f9d06ff5640 10 client.15747 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:51.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.315+0000 7f9d06ff5640 1 -- 172.21.15.152:0/3984669341 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55946237c860 con 0x55946242c790 2024-09-11T22:53:51.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.315+0000 7f9d06ff5640 10 client.15747 handle_command_reply: tid=0 2024-09-11T22:53:51.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.315+0000 7f9d2b6bf740 1 client.15747 shutdown 2024-09-11T22:53:51.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.315+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/3984669341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5594624095d0 msgr2=0x559462413ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:51.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.315+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/3984669341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5594624095d0 0x559462413ca0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9d14011b20 tx=0x7f9d14010430 comp rx=0 tx=0).stop 2024-09-11T22:53:51.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.315+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/3984669341 shutdown_connections 2024-09-11T22:53:51.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.315+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/3984669341 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55946242c790 0x55946242eba0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.315+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/3984669341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559462414370 0x559462411490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.315+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/3984669341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55946240ab40 0x559462410f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.315+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/3984669341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5594624095d0 0x559462413ca0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.315+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/3984669341 >> 172.21.15.152:0/3984669341 conn(0x5594623d2310 msgr2=0x55946240b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:51.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.315+0000 7f9d06ff5640 0 client.15747 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:51.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.316+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/3984669341 shutdown_connections 2024-09-11T22:53:51.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.316+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/3984669341 wait complete. 2024-09-11T22:53:51.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.316+0000 7f9d2b6bf740 20 client.15747 trim_cache size 0 max 16384 2024-09-11T22:53:51.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.329+0000 7f9d2b6bf740 1 Processor -- start 2024-09-11T22:53:51.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.329+0000 7f9d2b6bf740 1 -- start start 2024-09-11T22:53:51.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.329+0000 7f9d2b6bf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55946240ab40 0x559462405e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.329+0000 7f9d2b6bf740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559462414370 0x559462420310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.330+0000 7f9d2b6bf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559462410040 0x559462434440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.330+0000 7f9d2b6bf740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1000e670 con 0x559462414370 2024-09-11T22:53:51.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.330+0000 7f9d2b6bf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9d100101b0 con 0x55946240ab40 2024-09-11T22:53:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.330+0000 7f9d2b6bf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1000ed40 con 0x559462410040 2024-09-11T22:53:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.330+0000 7f9d227fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559462414370 0x559462420310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.330+0000 7f9d227fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559462414370 0x559462420310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49368/0 (socket says 172.21.15.152:49368) 2024-09-11T22:53:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.330+0000 7f9d227fc640 1 -- 172.21.15.152:0/4156062001 learned_addr learned my addr 172.21.15.152:0/4156062001 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.330+0000 7f9d227fc640 1 -- 172.21.15.152:0/4156062001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559462410040 msgr2=0x559462434440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.330+0000 7f9d227fc640 1 --2- 172.21.15.152:0/4156062001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559462410040 0x559462434440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.330+0000 7f9d227fc640 1 -- 172.21.15.152:0/4156062001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55946240ab40 msgr2=0x559462405e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.330+0000 7f9d22ffd640 1 --2- 172.21.15.152:0/4156062001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55946240ab40 0x559462405e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.330+0000 7f9d227fc640 1 --2- 172.21.15.152:0/4156062001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55946240ab40 0x559462405e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.330+0000 7f9d227fc640 1 -- 172.21.15.152:0/4156062001 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559462416f50 con 0x559462414370 2024-09-11T22:53:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.330+0000 7f9d22ffd640 1 --2- 172.21.15.152:0/4156062001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55946240ab40 0x559462405e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.330+0000 7f9d227fc640 1 --2- 172.21.15.152:0/4156062001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559462414370 0x559462420310 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009930 tx=0x7f9d1802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a3062c574ba8f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:51.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.331+0000 7f9d08ff9640 1 -- 172.21.15.152:0/4156062001 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18035070 con 0x559462414370 2024-09-11T22:53:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.331+0000 7f9d08ff9640 1 -- 172.21.15.152:0/4156062001 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d1804e040 con 0x559462414370 2024-09-11T22:53:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.331+0000 7f9d08ff9640 1 -- 172.21.15.152:0/4156062001 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1803d6c0 con 0x559462414370 2024-09-11T22:53:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.331+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/4156062001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559462414370 msgr2=0x559462420310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.331+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/4156062001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559462414370 0x559462420310 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009930 tx=0x7f9d1802a420 comp rx=0 tx=0).stop 2024-09-11T22:53:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.331+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/4156062001 shutdown_connections 2024-09-11T22:53:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.331+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/4156062001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559462410040 0x559462434440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.331+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/4156062001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559462414370 0x559462420310 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.331+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/4156062001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55946240ab40 0x559462405e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.331+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/4156062001 >> 172.21.15.152:0/4156062001 conn(0x5594623d2310 msgr2=0x559462405700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.332+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/4156062001 shutdown_connections 2024-09-11T22:53:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.332+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/4156062001 wait complete. 2024-09-11T22:53:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.332+0000 7f9d2b6bf740 1 -- start start 2024-09-11T22:53:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.332+0000 7f9d2b6bf740 1 Processor -- start 2024-09-11T22:53:51.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.332+0000 7f9d2b6bf740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55946240ab40 0x55946241cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.332+0000 7f9d2b6bf740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559462410040 0x55946240dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.332+0000 7f9d2b6bf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559462414370 0x55946240e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.332+0000 7f9d2b6bf740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1000fae0 con 0x55946240ab40 2024-09-11T22:53:51.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.332+0000 7f9d2b6bf740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1000f410 con 0x559462410040 2024-09-11T22:53:51.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.332+0000 7f9d2b6bf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1000c630 con 0x559462414370 2024-09-11T22:53:51.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.332+0000 7f9d2b6bf740 10 client.0 _ll_register_callbacks cb 0x7f9d1400bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:51.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.332+0000 7f9d22ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55946240ab40 0x55946241cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.333+0000 7f9d22ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55946240ab40 0x55946241cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49370/0 (socket says 172.21.15.152:49370) 2024-09-11T22:53:51.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.333+0000 7f9d227fc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559462410040 0x55946240dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.333+0000 7f9d22ffd640 1 -- 172.21.15.152:0/2755859176 learned_addr learned my addr 172.21.15.152:0/2755859176 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:51.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.333+0000 7f9d237fe640 1 --2- 172.21.15.152:0/2755859176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559462414370 0x55946240e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.333+0000 7f9d227fc640 1 -- 172.21.15.152:0/2755859176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559462414370 msgr2=0x55946240e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:51.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.333+0000 7f9d227fc640 1 --2- 172.21.15.152:0/2755859176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559462414370 0x55946240e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.333+0000 7f9d227fc640 1 -- 172.21.15.152:0/2755859176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55946240ab40 msgr2=0x55946241cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:51.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.333+0000 7f9d227fc640 1 --2- 172.21.15.152:0/2755859176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55946240ab40 0x55946241cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.333+0000 7f9d227fc640 1 -- 172.21.15.152:0/2755859176 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55946240e8f0 con 0x559462410040 2024-09-11T22:53:51.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.333+0000 7f9d237fe640 1 --2- 172.21.15.152:0/2755859176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559462414370 0x55946240e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:51.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.333+0000 7f9d22ffd640 1 --2- 172.21.15.152:0/2755859176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55946240ab40 0x55946241cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:51.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.333+0000 7f9d227fc640 1 --2- 172.21.15.152:0/2755859176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559462410040 0x55946240dc30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55946232a140 tx=0x7f9d18009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f46dcc9ed77e42f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:51.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.333+0000 7f9d06ff5640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-11T22:53:51.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.333+0000 7f9d06ff5640 1 -- 172.21.15.152:0/2755859176 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1808b060 con 0x559462410040 2024-09-11T22:53:51.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.333+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/2755859176 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d1804e040 con 0x559462410040 2024-09-11T22:53:51.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.333+0000 7f9d06ff5640 1 -- 172.21.15.152:0/2755859176 <== mon.2 v2:172.21.15.191:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f9d180035f0 con 0x559462410040 2024-09-11T22:53:51.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.334+0000 7f9d06ff5640 1 -- 172.21.15.152:0/2755859176 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9d18082040 con 0x559462410040 2024-09-11T22:53:51.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.334+0000 7f9d2b6bf740 10 client.25157 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:51.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.334+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/2755859176 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5594624220f0 con 0x559462410040 2024-09-11T22:53:51.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.334+0000 7f9d06ff5640 1 -- 172.21.15.152:0/2755859176 <== mon.2 v2:172.21.15.191:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d1804e040 con 0x559462410040 2024-09-11T22:53:51.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.334+0000 7f9d06ff5640 1 -- 172.21.15.152:0/2755859176 <== mon.2 v2:172.21.15.191:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9d1803d820 con 0x559462410040 2024-09-11T22:53:51.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.334+0000 7f9d2b6bf740 10 client.25157 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:51.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.334+0000 7f9d2b6bf740 10 client.25157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:51.346 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:53:51.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.334+0000 7f9d2b6bf740 20 client.25157 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:51.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.334+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/2755859176 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559462422380 0x5594624247a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:51.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.334+0000 7f9d2b6bf740 4 client.25157 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:53:51.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.334+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/2755859176 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559462376d70 con 0x559462422380 2024-09-11T22:53:51.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.335+0000 7f9d22ffd640 1 --2- 172.21.15.152:0/2755859176 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559462422380 0x5594624247a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:51.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.335+0000 7f9d22ffd640 1 --2- 172.21.15.152:0/2755859176 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559462422380 0x5594624247a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8be9648ee0196cb server_cookie=33277c617e109074 in_seq=0 out_seq=0 2024-09-11T22:53:51.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.335+0000 7f9d06ff5640 10 client.25157 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:51.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.335+0000 7f9d06ff5640 1 -- 172.21.15.152:0/2755859176 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559462376d70 con 0x559462422380 2024-09-11T22:53:51.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.335+0000 7f9d06ff5640 10 client.25157 handle_command_reply: tid=0 2024-09-11T22:53:51.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.335+0000 7f9d2b6bf740 1 client.25157 shutdown 2024-09-11T22:53:51.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.335+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/2755859176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559462410040 msgr2=0x55946240dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:51.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.335+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/2755859176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559462410040 0x55946240dc30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55946232a140 tx=0x7f9d18009780 comp rx=0 tx=0).stop 2024-09-11T22:53:51.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.335+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/2755859176 shutdown_connections 2024-09-11T22:53:51.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.335+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/2755859176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559462414370 0x55946240e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.335+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/2755859176 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559462422380 0x5594624247a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.335+0000 7f9d06ff5640 0 client.25157 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:51.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.335+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/2755859176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559462410040 0x55946240dc30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.335+0000 7f9d2b6bf740 1 --2- 172.21.15.152:0/2755859176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55946240ab40 0x55946241cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.336+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/2755859176 >> 172.21.15.152:0/2755859176 conn(0x5594623d2310 msgr2=0x5594624044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:51.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.336+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/2755859176 shutdown_connections 2024-09-11T22:53:51.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.336+0000 7f9d2b6bf740 1 -- 172.21.15.152:0/2755859176 wait complete. 2024-09-11T22:53:51.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.336+0000 7f9d2b6bf740 20 client.25157 trim_cache size 0 max 16384 2024-09-11T22:53:51.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.336+0000 7f9d292e6640 1 -- 172.21.15.152:0/2933239143 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9d0c0a2e00 msgr2=0x7f9d0c0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:51.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.336+0000 7f9d292e6640 1 --2- 172.21.15.152:0/2933239143 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9d0c0a2e00 0x7f9d0c0a52c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9d18000c00 tx=0x7f9d18032f20 comp rx=0 tx=0).stop 2024-09-11T22:53:51.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.336+0000 7f9d292e6640 1 -- 172.21.15.152:0/2933239143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d24106f50 msgr2=0x7f9d2419efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:51.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.336+0000 7f9d292e6640 1 --2- 172.21.15.152:0/2933239143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d24106f50 0x7f9d2419efd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009fd0 tx=0x7f9d1c00b260 comp rx=0 tx=0).stop 2024-09-11T22:53:51.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.337+0000 7f9d292e6640 1 -- 172.21.15.152:0/2933239143 shutdown_connections 2024-09-11T22:53:51.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.337+0000 7f9d292e6640 1 --2- 172.21.15.152:0/2933239143 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9d0c0a2e00 0x7f9d0c0a52c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.337+0000 7f9d292e6640 1 --2- 172.21.15.152:0/2933239143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d24106f50 0x7f9d2419efd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.337+0000 7f9d292e6640 1 --2- 172.21.15.152:0/2933239143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d24100ca0 0x7f9d2419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.337+0000 7f9d292e6640 1 --2- 172.21.15.152:0/2933239143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d241002f0 0x7f9d2419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:51.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.337+0000 7f9d292e6640 1 -- 172.21.15.152:0/2933239143 >> 172.21.15.152:0/2933239143 conn(0x7f9d24077220 msgr2=0x7f9d24105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:51.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.337+0000 7f9d292e6640 1 -- 172.21.15.152:0/2933239143 shutdown_connections 2024-09-11T22:53:51.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:51.337+0000 7f9d292e6640 1 -- 172.21.15.152:0/2933239143 wait complete. 2024-09-11T22:53:52.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:52 smithi191 ceph-mon[35211]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T22:53:52.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:52 smithi204 ceph-mon[35454]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T22:53:52.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:52 smithi152 ceph-mon[31181]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-11T22:53:53.190 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:53:53.190 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:53:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:54.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.007+0000 7f3e4b473640 1 -- 172.21.15.152:0/1330905833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e44102d90 msgr2=0x7f3e44103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:54.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.007+0000 7f3e4b473640 1 --2- 172.21.15.152:0/1330905833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e44102d90 0x7f3e44103170 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009cd0 tx=0x7f3e4002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.007+0000 7f3e4b473640 1 -- 172.21.15.152:0/1330905833 shutdown_connections 2024-09-11T22:53:54.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.007+0000 7f3e4b473640 1 --2- 172.21.15.152:0/1330905833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e44108290 0x7f3e4410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.007+0000 7f3e4b473640 1 --2- 172.21.15.152:0/1330905833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e44103740 0x7f3e44107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.007+0000 7f3e4b473640 1 --2- 172.21.15.152:0/1330905833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e44102d90 0x7f3e44103170 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.007+0000 7f3e4b473640 1 -- 172.21.15.152:0/1330905833 >> 172.21.15.152:0/1330905833 conn(0x7f3e440fc7d0 msgr2=0x7f3e440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:54.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.007+0000 7f3e4b473640 1 -- 172.21.15.152:0/1330905833 shutdown_connections 2024-09-11T22:53:54.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.007+0000 7f3e4b473640 1 -- 172.21.15.152:0/1330905833 wait complete. 2024-09-11T22:53:54.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.007+0000 7f3e4b473640 1 Processor -- start 2024-09-11T22:53:54.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.008+0000 7f3e4b473640 1 -- start start 2024-09-11T22:53:54.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.008+0000 7f3e4b473640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e44102d90 0x7f3e441009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.008+0000 7f3e4b473640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e44103740 0x7f3e44100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.008+0000 7f3e4b473640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e44108290 0x7f3e44101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.008+0000 7f3e4b473640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44112090 con 0x7f3e44102d90 2024-09-11T22:53:54.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.008+0000 7f3e4b473640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44111f10 con 0x7f3e44103740 2024-09-11T22:53:54.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.008+0000 7f3e4b473640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44112210 con 0x7f3e44108290 2024-09-11T22:53:54.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.008+0000 7f3e4920f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e44102d90 0x7f3e441009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.008+0000 7f3e4920f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e44102d90 0x7f3e441009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49384/0 (socket says 172.21.15.152:49384) 2024-09-11T22:53:54.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.008+0000 7f3e4920f640 1 -- 172.21.15.152:0/2575178355 learned_addr learned my addr 172.21.15.152:0/2575178355 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:54.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.008+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/2575178355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e44103740 0x7f3e44100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.008+0000 7f3e49a10640 1 --2- 172.21.15.152:0/2575178355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e44108290 0x7f3e44101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.008+0000 7f3e48a0e640 1 -- 172.21.15.152:0/2575178355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e44108290 msgr2=0x7f3e44101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:54.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.008+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/2575178355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e44108290 0x7f3e44101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.009+0000 7f3e48a0e640 1 -- 172.21.15.152:0/2575178355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e44102d90 msgr2=0x7f3e441009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:54.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.009+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/2575178355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e44102d90 0x7f3e441009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.009+0000 7f3e48a0e640 1 -- 172.21.15.152:0/2575178355 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e441a6a90 con 0x7f3e44103740 2024-09-11T22:53:54.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.009+0000 7f3e4920f640 1 --2- 172.21.15.152:0/2575178355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e44102d90 0x7f3e441009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:54.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.009+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/2575178355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e44103740 0x7f3e44100f30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3e440697d0 tx=0x7f3e3400dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:54.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.009+0000 7f3e3a7fc640 1 -- 172.21.15.152:0/2575178355 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34017070 con 0x7f3e44103740 2024-09-11T22:53:54.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.009+0000 7f3e3a7fc640 1 -- 172.21.15.152:0/2575178355 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e340099d0 con 0x7f3e44103740 2024-09-11T22:53:54.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.009+0000 7f3e4b473640 1 -- 172.21.15.152:0/2575178355 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e441a6d80 con 0x7f3e44103740 2024-09-11T22:53:54.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.009+0000 7f3e3a7fc640 1 -- 172.21.15.152:0/2575178355 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e340047a0 con 0x7f3e44103740 2024-09-11T22:53:54.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.009+0000 7f3e4b473640 1 -- 172.21.15.152:0/2575178355 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e441a7210 con 0x7f3e44103740 2024-09-11T22:53:54.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.012+0000 7f3e3a7fc640 1 -- 172.21.15.152:0/2575178355 <== mon.2 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f3e34009b80 con 0x7f3e44103740 2024-09-11T22:53:54.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.012+0000 7f3e4d84c740 1 Processor -- start 2024-09-11T22:53:54.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.012+0000 7f3e4d84c740 1 -- start start 2024-09-11T22:53:54.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.012+0000 7f3e4d84c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.012+0000 7f3e4d84c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 0x56299e3d8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.012+0000 7f3e3a7fc640 1 --2- 172.21.15.152:0/2575178355 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3e300a2e00 0x7f3e300a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.012+0000 7f3e4d84c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3d8550 0x56299e3da940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.012+0000 7f3e4d84c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56299e32a280 con 0x56299e3c4a80 2024-09-11T22:53:54.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.012+0000 7f3e4d84c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56299e2d1b10 con 0x56299e3c53a0 2024-09-11T22:53:54.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.012+0000 7f3e4d84c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56299e326240 con 0x56299e3d8550 2024-09-11T22:53:54.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.013+0000 7f3e49a10640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.013+0000 7f3e49a10640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3c4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49394/0 (socket says 172.21.15.152:49394) 2024-09-11T22:53:54.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.013+0000 7f3e49a10640 1 -- 172.21.15.152:0/1209881766 learned_addr learned my addr 172.21.15.152:0/1209881766 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:54.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.013+0000 7f3e4920f640 1 --2- 172.21.15.152:0/2575178355 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3e300a2e00 0x7f3e300a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.013+0000 7f3e3a7fc640 1 -- 172.21.15.152:0/2575178355 <== mon.2 v2:172.21.15.191:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f3e340cf1e0 con 0x7f3e44103740 2024-09-11T22:53:54.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.013+0000 7f3e49a10640 1 -- 172.21.15.152:0/1209881766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3d8550 msgr2=0x56299e3da940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:53:54.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.013+0000 7f3e49a10640 1 --2- 172.21.15.152:0/1209881766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3d8550 0x56299e3da940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.013+0000 7f3e49a10640 1 -- 172.21.15.152:0/1209881766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 msgr2=0x56299e3d8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:54.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.013+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/1209881766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 0x56299e3d8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.013+0000 7f3e49a10640 1 --2- 172.21.15.152:0/1209881766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 0x56299e3d8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.013+0000 7f3e49a10640 1 -- 172.21.15.152:0/1209881766 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56299e2de1f0 con 0x56299e3c4a80 2024-09-11T22:53:54.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.014+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/1209881766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 0x56299e3d8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:54.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.014+0000 7f3e4920f640 1 --2- 172.21.15.152:0/2575178355 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3e300a2e00 0x7f3e300a52c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3e40000f80 tx=0x7f3e400317e0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:54.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.014+0000 7f3e49a10640 1 --2- 172.21.15.152:0/1209881766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3c4e60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00b630 tx=0x7f3e3c00baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec4afbfcaccde5f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:54.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.014+0000 7f3e2f7fe640 1 -- 172.21.15.152:0/1209881766 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c003b20 con 0x56299e3c4a80 2024-09-11T22:53:54.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.014+0000 7f3e2f7fe640 1 -- 172.21.15.152:0/1209881766 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3c015070 con 0x56299e3c4a80 2024-09-11T22:53:54.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.014+0000 7f3e2f7fe640 1 -- 172.21.15.152:0/1209881766 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0106a0 con 0x56299e3c4a80 2024-09-11T22:53:54.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.015+0000 7f3e4d84c740 1 -- 172.21.15.152:0/1209881766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 msgr2=0x56299e3c4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:54.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.015+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/1209881766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3c4e60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00b630 tx=0x7f3e3c00baf0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.015+0000 7f3e4d84c740 1 -- 172.21.15.152:0/1209881766 shutdown_connections 2024-09-11T22:53:54.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.015+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/1209881766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3d8550 0x56299e3da940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.015+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/1209881766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 0x56299e3d8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.015+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/1209881766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3c4e60 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.015+0000 7f3e4d84c740 1 -- 172.21.15.152:0/1209881766 >> 172.21.15.152:0/1209881766 conn(0x56299e3c10f0 msgr2=0x56299e3c14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:54.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.015+0000 7f3e4d84c740 1 -- 172.21.15.152:0/1209881766 shutdown_connections 2024-09-11T22:53:54.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.015+0000 7f3e4d84c740 1 -- 172.21.15.152:0/1209881766 wait complete. 2024-09-11T22:53:54.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.015+0000 7f3e4d84c740 1 -- start start 2024-09-11T22:53:54.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.015+0000 7f3e4d84c740 1 Processor -- start 2024-09-11T22:53:54.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.016+0000 7f3e4d84c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.016+0000 7f3e4d84c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 0x56299e3d5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.016+0000 7f3e4920f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.016+0000 7f3e4920f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3d55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49404/0 (socket says 172.21.15.152:49404) 2024-09-11T22:53:54.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.016+0000 7f3e4920f640 1 -- 172.21.15.152:0/225733086 learned_addr learned my addr 172.21.15.152:0/225733086 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:54.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.016+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/225733086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3d8550 0x56299e3d6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.016+0000 7f3e4d84c740 1 -- 172.21.15.152:0/225733086 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56299e3db0c0 con 0x56299e3c4a80 2024-09-11T22:53:54.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.016+0000 7f3e4d84c740 1 -- 172.21.15.152:0/225733086 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56299e2e4f10 con 0x56299e3c53a0 2024-09-11T22:53:54.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.016+0000 7f3e4d84c740 1 -- 172.21.15.152:0/225733086 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56299e3db240 con 0x56299e3d8550 2024-09-11T22:53:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.016+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/225733086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 0x56299e3d5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.017+0000 7f3e49a10640 1 --2- 172.21.15.152:0/225733086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3d8550 0x56299e3d6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.017+0000 7f3e48a0e640 1 -- 172.21.15.152:0/225733086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3d8550 msgr2=0x56299e3d6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.017+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/225733086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3d8550 0x56299e3d6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.017+0000 7f3e48a0e640 1 -- 172.21.15.152:0/225733086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 msgr2=0x56299e3d55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.017+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/225733086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3d55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.016+0000 7f3e4d84c740 10 client.0 _ll_register_callbacks cb 0x56299e2ada40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.017+0000 7f3e48a0e640 1 -- 172.21.15.152:0/225733086 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56299e09b6a0 con 0x56299e3c53a0 2024-09-11T22:53:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.017+0000 7f3e4920f640 1 --2- 172.21.15.152:0/225733086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3d55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.017+0000 7f3e4d84c740 1 -- 172.21.15.152:0/225733086 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e3c015070 con 0x56299e3c53a0 2024-09-11T22:53:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.017+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/225733086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 0x56299e3d5b00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3e340cc880 tx=0x7f3e3409af00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc04b701430578fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.017+0000 7f3e2cff9640 10 client.25166 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-11T22:53:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.017+0000 7f3e2cff9640 1 -- 172.21.15.152:0/225733086 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e340b4060 con 0x56299e3c53a0 2024-09-11T22:53:54.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.017+0000 7f3e2cff9640 1 -- 172.21.15.152:0/225733086 <== mon.2 v2:172.21.15.191:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f3e340ccb30 con 0x56299e3c53a0 2024-09-11T22:53:54.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.017+0000 7f3e2cff9640 1 -- 172.21.15.152:0/225733086 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e340ccd30 con 0x56299e3c53a0 2024-09-11T22:53:54.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.017+0000 7f3e4d84c740 10 client.25166 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:54.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.017+0000 7f3e4d84c740 1 -- 172.21.15.152:0/225733086 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({fsmap=28}) -- 0x56299e3e6d30 con 0x56299e3c53a0 2024-09-11T22:53:54.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.018+0000 7f3e2cff9640 1 -- 172.21.15.152:0/225733086 <== mon.2 v2:172.21.15.191:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3e3409a9f0 con 0x56299e3c53a0 2024-09-11T22:53:54.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.018+0000 7f3e4d84c740 10 client.25166 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:54.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.018+0000 7f3e4d84c740 10 client.25166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:54.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.018+0000 7f3e4d84c740 20 client.25166 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:54.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.018+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/225733086 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56299e3e80e0 0x56299e3ea4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.018+0000 7f3e4d84c740 4 client.25166 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:54.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.018+0000 7f3e4d84c740 1 -- 172.21.15.152:0/225733086 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56299e3377f0 con 0x56299e3e80e0 2024-09-11T22:53:54.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.018+0000 7f3e4920f640 1 --2- 172.21.15.152:0/225733086 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56299e3e80e0 0x56299e3ea4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.018+0000 7f3e2cff9640 10 client.25166 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:54.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.018+0000 7f3e4920f640 1 --2- 172.21.15.152:0/225733086 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56299e3e80e0 0x56299e3ea4d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cc132006c72ccc server_cookie=251b224f1c670132 in_seq=0 out_seq=0 2024-09-11T22:53:54.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.019+0000 7f3e2cff9640 1 -- 172.21.15.152:0/225733086 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56299e3377f0 con 0x56299e3e80e0 2024-09-11T22:53:54.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.019+0000 7f3e2cff9640 10 client.25166 handle_command_reply: tid=0 2024-09-11T22:53:54.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.019+0000 7f3e4d84c740 1 client.25166 shutdown 2024-09-11T22:53:54.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.019+0000 7f3e4d84c740 1 -- 172.21.15.152:0/225733086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 msgr2=0x56299e3d5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:54.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.019+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/225733086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 0x56299e3d5b00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3e340cc880 tx=0x7f3e3409af00 comp rx=0 tx=0).stop 2024-09-11T22:53:54.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.020+0000 7f3e4d84c740 1 -- 172.21.15.152:0/225733086 shutdown_connections 2024-09-11T22:53:54.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.020+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/225733086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3d8550 0x56299e3d6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.020+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/225733086 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56299e3e80e0 0x56299e3ea4d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.020+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/225733086 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 0x56299e3d5b00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.020+0000 7f3e2cff9640 0 client.25166 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:54.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.020+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/225733086 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3d55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.020+0000 7f3e4d84c740 1 -- 172.21.15.152:0/225733086 >> 172.21.15.152:0/225733086 conn(0x56299e3c10f0 msgr2=0x56299e3c6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:54.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.020+0000 7f3e4d84c740 1 -- 172.21.15.152:0/225733086 shutdown_connections 2024-09-11T22:53:54.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.020+0000 7f3e4d84c740 1 -- 172.21.15.152:0/225733086 wait complete. 2024-09-11T22:53:54.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.020+0000 7f3e4d84c740 20 client.25166 trim_cache size 0 max 16384 2024-09-11T22:53:54.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.037+0000 7f3e4d84c740 1 Processor -- start 2024-09-11T22:53:54.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.037+0000 7f3e4d84c740 1 -- start start 2024-09-11T22:53:54.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e4d84c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3c0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e4d84c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3c53a0 0x56299e3efda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e4d84c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3d8550 0x56299e3f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e4d84c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441ab540 con 0x56299e3c4a80 2024-09-11T22:53:54.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e4d84c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441ad080 con 0x56299e3d8550 2024-09-11T22:53:54.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e4d84c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441abc10 con 0x56299e3c53a0 2024-09-11T22:53:54.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e4920f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3c0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e4920f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3c0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49420/0 (socket says 172.21.15.152:49420) 2024-09-11T22:53:54.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e4920f640 1 -- 172.21.15.152:0/1587327496 learned_addr learned my addr 172.21.15.152:0/1587327496 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:54.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/1587327496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3c53a0 0x56299e3efda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e49a10640 1 --2- 172.21.15.152:0/1587327496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3d8550 0x56299e3f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e48a0e640 1 -- 172.21.15.152:0/1587327496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3d8550 msgr2=0x56299e3f02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:54.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/1587327496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3d8550 0x56299e3f02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e48a0e640 1 -- 172.21.15.152:0/1587327496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 msgr2=0x56299e3c0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:54.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/1587327496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3c0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e48a0e640 1 -- 172.21.15.152:0/1587327496 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56299e3d4570 con 0x56299e3c53a0 2024-09-11T22:53:54.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.038+0000 7f3e4920f640 1 --2- 172.21.15.152:0/1587327496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3c0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:54.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.039+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/1587327496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3c53a0 0x56299e3efda0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3e3409b2a0 tx=0x7f3e340cf000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e0a712706ee17ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:54.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.039+0000 7f3e2effd640 1 -- 172.21.15.152:0/1587327496 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34014020 con 0x56299e3c53a0 2024-09-11T22:53:54.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.039+0000 7f3e2effd640 1 -- 172.21.15.152:0/1587327496 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3c015070 con 0x56299e3c53a0 2024-09-11T22:53:54.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.039+0000 7f3e2effd640 1 -- 172.21.15.152:0/1587327496 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3409a9f0 con 0x56299e3c53a0 2024-09-11T22:53:54.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.040+0000 7f3e4d84c740 1 -- 172.21.15.152:0/1587327496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3c53a0 msgr2=0x56299e3efda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:54.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.040+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/1587327496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3c53a0 0x56299e3efda0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3e3409b2a0 tx=0x7f3e340cf000 comp rx=0 tx=0).stop 2024-09-11T22:53:54.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.040+0000 7f3e4d84c740 1 -- 172.21.15.152:0/1587327496 shutdown_connections 2024-09-11T22:53:54.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.040+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/1587327496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3d8550 0x56299e3f02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.040+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/1587327496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3c53a0 0x56299e3efda0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.040+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/1587327496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3c0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.040+0000 7f3e4d84c740 1 -- 172.21.15.152:0/1587327496 >> 172.21.15.152:0/1587327496 conn(0x56299e3c10f0 msgr2=0x56299e3bfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:54.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.040+0000 7f3e4d84c740 1 -- 172.21.15.152:0/1587327496 shutdown_connections 2024-09-11T22:53:54.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.040+0000 7f3e4d84c740 1 -- 172.21.15.152:0/1587327496 wait complete. 2024-09-11T22:53:54.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.040+0000 7f3e4d84c740 1 -- start start 2024-09-11T22:53:54.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.040+0000 7f3e4d84c740 1 Processor -- start 2024-09-11T22:53:54.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e4d84c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3e08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e4d84c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 0x56299e3e0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e4d84c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3d8550 0x56299e3dd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e4d84c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441ac9b0 con 0x56299e3c4a80 2024-09-11T22:53:54.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e4d84c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441ac2e0 con 0x56299e3c53a0 2024-09-11T22:53:54.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e4d84c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441a9520 con 0x56299e3d8550 2024-09-11T22:53:54.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e4d84c740 10 client.0 _ll_register_callbacks cb 0x7f3e340cc710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:54.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e48a0e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 0x56299e3e0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e48a0e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 0x56299e3e0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:45160/0 (socket says 172.21.15.152:45160) 2024-09-11T22:53:54.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e48a0e640 1 -- 172.21.15.152:0/3816721293 learned_addr learned my addr 172.21.15.152:0/3816721293 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:54.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e49a10640 1 --2- 172.21.15.152:0/3816721293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3d8550 0x56299e3dd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e4920f640 1 --2- 172.21.15.152:0/3816721293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3e08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e48a0e640 1 -- 172.21.15.152:0/3816721293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3d8550 msgr2=0x56299e3dd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:54.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/3816721293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3d8550 0x56299e3dd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e48a0e640 1 -- 172.21.15.152:0/3816721293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 msgr2=0x56299e3e08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:54.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/3816721293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3e08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e48a0e640 1 -- 172.21.15.152:0/3816721293 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56299e3dacb0 con 0x56299e3c53a0 2024-09-11T22:53:54.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.041+0000 7f3e48a0e640 1 --2- 172.21.15.152:0/3816721293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 0x56299e3e0e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3e3409b180 tx=0x7f3e34097ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99a515b463d60cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:54.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.042+0000 7f3e2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-11T22:53:54.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.042+0000 7f3e2cff9640 1 -- 172.21.15.152:0/3816721293 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34014030 con 0x56299e3c53a0 2024-09-11T22:53:54.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.042+0000 7f3e2cff9640 1 -- 172.21.15.152:0/3816721293 <== mon.2 v2:172.21.15.191:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f3e340ccb30 con 0x56299e3c53a0 2024-09-11T22:53:54.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.042+0000 7f3e4d84c740 1 -- 172.21.15.152:0/3816721293 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e3c015070 con 0x56299e3c53a0 2024-09-11T22:53:54.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.042+0000 7f3e2cff9640 1 -- 172.21.15.152:0/3816721293 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e3409b5a0 con 0x56299e3c53a0 2024-09-11T22:53:54.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.042+0000 7f3e4d84c740 10 client.25172 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.042+0000 7f3e4d84c740 1 -- 172.21.15.152:0/3816721293 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56299e3dacb0 con 0x56299e3c53a0 2024-09-11T22:53:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.042+0000 7f3e2cff9640 1 -- 172.21.15.152:0/3816721293 <== mon.2 v2:172.21.15.191:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3409f950 con 0x56299e3c53a0 2024-09-11T22:53:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.042+0000 7f3e2cff9640 1 -- 172.21.15.152:0/3816721293 <== mon.2 v2:172.21.15.191:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3e3409a460 con 0x56299e3c53a0 2024-09-11T22:53:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.042+0000 7f3e4d84c740 10 client.25172 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.042+0000 7f3e4d84c740 10 client.25172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.042+0000 7f3e4d84c740 20 client.25172 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.042+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/3816721293 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56299e3e80e0 0x56299e455ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.043+0000 7f3e4d84c740 4 client.25172 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:53:54.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.043+0000 7f3e4d84c740 1 -- 172.21.15.152:0/3816721293 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f3e3c015070 con 0x56299e3e80e0 2024-09-11T22:53:54.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.043+0000 7f3e4920f640 1 --2- 172.21.15.152:0/3816721293 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56299e3e80e0 0x56299e455ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:54.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.044+0000 7f3e4920f640 1 --2- 172.21.15.152:0/3816721293 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56299e3e80e0 0x56299e455ba0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b07c1b5085e5d2b server_cookie=12476e71d62a975b in_seq=0 out_seq=0 2024-09-11T22:53:54.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.044+0000 7f3e2cff9640 10 client.25172 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:54.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.044+0000 7f3e2cff9640 1 -- 172.21.15.152:0/3816721293 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f3e3c015070 con 0x56299e3e80e0 2024-09-11T22:53:54.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.045+0000 7f3e2cff9640 10 client.25172 handle_command_reply: tid=0 2024-09-11T22:53:54.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.045+0000 7f3e4d84c740 1 client.25172 shutdown 2024-09-11T22:53:54.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.045+0000 7f3e4d84c740 1 -- 172.21.15.152:0/3816721293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 msgr2=0x56299e3e0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:54.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.045+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/3816721293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 0x56299e3e0e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3e3409b180 tx=0x7f3e34097ee0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.045+0000 7f3e4d84c740 1 -- 172.21.15.152:0/3816721293 shutdown_connections 2024-09-11T22:53:54.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.045+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/3816721293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56299e3d8550 0x56299e3dd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.045+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/3816721293 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56299e3e80e0 0x56299e455ba0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.045+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/3816721293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56299e3c53a0 0x56299e3e0e20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.045+0000 7f3e2cff9640 0 client.25172 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:54.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.045+0000 7f3e4d84c740 1 --2- 172.21.15.152:0/3816721293 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56299e3c4a80 0x56299e3e08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.045+0000 7f3e4d84c740 1 -- 172.21.15.152:0/3816721293 >> 172.21.15.152:0/3816721293 conn(0x56299e3c10f0 msgr2=0x56299e3c0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:54.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.045+0000 7f3e4d84c740 1 -- 172.21.15.152:0/3816721293 shutdown_connections 2024-09-11T22:53:54.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.046+0000 7f3e4d84c740 1 -- 172.21.15.152:0/3816721293 wait complete. 2024-09-11T22:53:54.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.046+0000 7f3e4d84c740 20 client.25172 trim_cache size 0 max 16384 2024-09-11T22:53:54.050 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:53:54.050 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T22:53:54.050 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "cfa50f24-529f-410f-ac75-dd50313f4112", 2024-09-11T22:53:54.050 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T22:53:54.051 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:53:54.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.046+0000 7f3e4b473640 1 -- 172.21.15.152:0/2575178355 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3e300a2e00 msgr2=0x7f3e300a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:54.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.047+0000 7f3e4b473640 1 --2- 172.21.15.152:0/2575178355 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3e300a2e00 0x7f3e300a52c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3e40000f80 tx=0x7f3e400317e0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.047+0000 7f3e4b473640 1 -- 172.21.15.152:0/2575178355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e44103740 msgr2=0x7f3e44100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:54.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.047+0000 7f3e4b473640 1 --2- 172.21.15.152:0/2575178355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e44103740 0x7f3e44100f30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3e440697d0 tx=0x7f3e3400dbb0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.047+0000 7f3e4b473640 1 -- 172.21.15.152:0/2575178355 shutdown_connections 2024-09-11T22:53:54.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.047+0000 7f3e4b473640 1 --2- 172.21.15.152:0/2575178355 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3e300a2e00 0x7f3e300a52c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.047+0000 7f3e4b473640 1 --2- 172.21.15.152:0/2575178355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e44108290 0x7f3e44101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.047+0000 7f3e4b473640 1 --2- 172.21.15.152:0/2575178355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e44103740 0x7f3e44100f30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.047+0000 7f3e4b473640 1 --2- 172.21.15.152:0/2575178355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3e44102d90 0x7f3e441009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:54.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.047+0000 7f3e4b473640 1 -- 172.21.15.152:0/2575178355 >> 172.21.15.152:0/2575178355 conn(0x7f3e440fc7d0 msgr2=0x7f3e44105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:54.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.047+0000 7f3e4b473640 1 -- 172.21.15.152:0/2575178355 shutdown_connections 2024-09-11T22:53:54.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:54.047+0000 7f3e4b473640 1 -- 172.21.15.152:0/2575178355 wait complete. 2024-09-11T22:53:54.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:54 smithi191 ceph-mon[35211]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-11T22:53:54.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:54 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T22:53:54.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:54 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:53:54.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:54 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T22:53:54.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:54 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:53:54.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:54 smithi204 ceph-mon[35454]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-11T22:53:54.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:54 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T22:53:54.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:54 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:53:54.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:54 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T22:53:54.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:54 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:53:54.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:54 smithi152 ceph-mon[31181]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-11T22:53:54.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:54 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T22:53:54.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:54 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:53:54.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:54 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T22:53:54.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:54 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:53:54.882 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:53:55.167 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:53:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.719+0000 7f0e3a4c0640 1 -- 172.21.15.152:0/1307624350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e341080f0 msgr2=0x7f0e3410ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.719+0000 7f0e3a4c0640 1 --2- 172.21.15.152:0/1307624350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e341080f0 0x7f0e3410ea40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009cd0 tx=0x7f0e2c02a970 comp rx=0 tx=0).stop 2024-09-11T22:53:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.719+0000 7f0e3a4c0640 1 -- 172.21.15.152:0/1307624350 shutdown_connections 2024-09-11T22:53:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.719+0000 7f0e3a4c0640 1 --2- 172.21.15.152:0/1307624350 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e341080f0 0x7f0e3410ea40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.719+0000 7f0e3a4c0640 1 --2- 172.21.15.152:0/1307624350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e34101150 0x7f0e34107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.719+0000 7f0e3a4c0640 1 --2- 172.21.15.152:0/1307624350 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e34100830 0x7f0e34100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.719+0000 7f0e3a4c0640 1 -- 172.21.15.152:0/1307624350 >> 172.21.15.152:0/1307624350 conn(0x7f0e34075f20 msgr2=0x7f0e34076330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.719+0000 7f0e3a4c0640 1 -- 172.21.15.152:0/1307624350 shutdown_connections 2024-09-11T22:53:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.720+0000 7f0e3a4c0640 1 -- 172.21.15.152:0/1307624350 wait complete. 2024-09-11T22:53:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.721+0000 7f0e3a4c0640 1 Processor -- start 2024-09-11T22:53:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.721+0000 7f0e3a4c0640 1 -- start start 2024-09-11T22:53:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.721+0000 7f0e3a4c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e34100830 0x7f0e341017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.721+0000 7f0e3a4c0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e34101150 0x7f0e34101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.721+0000 7f0e3a4c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e341080f0 0x7f0e34102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.721+0000 7f0e3a4c0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0e34111f20 con 0x7f0e34101150 2024-09-11T22:53:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.721+0000 7f0e3a4c0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0e34111da0 con 0x7f0e34100830 2024-09-11T22:53:55.722 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.721+0000 7f0e3a4c0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0e341120a0 con 0x7f0e341080f0 2024-09-11T22:53:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.721+0000 7f0e337fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e34101150 0x7f0e34101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.721+0000 7f0e337fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e34101150 0x7f0e34101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49460/0 (socket says 172.21.15.152:49460) 2024-09-11T22:53:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.721+0000 7f0e337fe640 1 -- 172.21.15.152:0/1007066131 learned_addr learned my addr 172.21.15.152:0/1007066131 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.722+0000 7f0e38a5d640 1 --2- 172.21.15.152:0/1007066131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e341080f0 0x7f0e34102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.722+0000 7f0e33fff640 1 --2- 172.21.15.152:0/1007066131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e34100830 0x7f0e341017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.722+0000 7f0e337fe640 1 -- 172.21.15.152:0/1007066131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e341080f0 msgr2=0x7f0e34102220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.722+0000 7f0e337fe640 1 --2- 172.21.15.152:0/1007066131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e341080f0 0x7f0e34102220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.723 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.722+0000 7f0e337fe640 1 -- 172.21.15.152:0/1007066131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e34100830 msgr2=0x7f0e341017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.722+0000 7f0e337fe640 1 --2- 172.21.15.152:0/1007066131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e34100830 0x7f0e341017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.722+0000 7f0e337fe640 1 -- 172.21.15.152:0/1007066131 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e341a6830 con 0x7f0e34101150 2024-09-11T22:53:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.722+0000 7f0e337fe640 1 --2- 172.21.15.152:0/1007066131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e34101150 0x7f0e34101ce0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800d770 tx=0x7f0e2800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.723+0000 7f0e317fa640 1 -- 172.21.15.152:0/1007066131 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28017070 con 0x7f0e34101150 2024-09-11T22:53:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.723+0000 7f0e317fa640 1 -- 172.21.15.152:0/1007066131 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e28009c30 con 0x7f0e34101150 2024-09-11T22:53:55.724 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.723+0000 7f0e3a4c0640 1 -- 172.21.15.152:0/1007066131 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e341a6b20 con 0x7f0e34101150 2024-09-11T22:53:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.724+0000 7f0e317fa640 1 -- 172.21.15.152:0/1007066131 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2800f620 con 0x7f0e34101150 2024-09-11T22:53:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.725+0000 7f0e3a4c0640 1 -- 172.21.15.152:0/1007066131 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e341a70c0 con 0x7f0e34101150 2024-09-11T22:53:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.726+0000 7f0e317fa640 1 -- 172.21.15.152:0/1007066131 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7f0e2801a070 con 0x7f0e34101150 2024-09-11T22:53:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.727+0000 7f0e317fa640 1 --2- 172.21.15.152:0/1007066131 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0e1c0a2d10 0x7f0e1c0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.727+0000 7f0e33fff640 1 --2- 172.21.15.152:0/1007066131 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0e1c0a2d10 0x7f0e1c0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.727 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.727+0000 7f0e3d254740 1 Processor -- start 2024-09-11T22:53:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.727+0000 7f0e317fa640 1 -- 172.21.15.152:0/1007066131 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f0e280d14b0 con 0x7f0e34101150 2024-09-11T22:53:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e3d254740 1 -- start start 2024-09-11T22:53:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e3d254740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887aeaf0 0x55eb887aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e3d254740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887af410 0x55eb887b9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e3d254740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887b9cb0 0x55eb887bc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e3d254740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55eb887132f0 con 0x55eb887aeaf0 2024-09-11T22:53:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e3d254740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55eb886bb270 con 0x55eb887af410 2024-09-11T22:53:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e3d254740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55eb8870f2b0 con 0x55eb887b9cb0 2024-09-11T22:53:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e33fff640 1 --2- 172.21.15.152:0/1007066131 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0e1c0a2d10 0x7f0e1c0a51d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009bd0 tx=0x7f0e24005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e38a5d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887b9cb0 0x55eb887bc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e38a5d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887b9cb0 0x55eb887bc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:55746/0 (socket says 172.21.15.152:55746) 2024-09-11T22:53:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e33fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887aeaf0 0x55eb887aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e38a5d640 1 -- 172.21.15.152:0/2771364015 learned_addr learned my addr 172.21.15.152:0/2771364015 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e337fe640 1 --2- 172.21.15.152:0/2771364015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887af410 0x55eb887b9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e38a5d640 1 -- 172.21.15.152:0/2771364015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887af410 msgr2=0x55eb887b9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e38a5d640 1 --2- 172.21.15.152:0/2771364015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887af410 0x55eb887b9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e38a5d640 1 -- 172.21.15.152:0/2771364015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887aeaf0 msgr2=0x55eb887aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e38a5d640 1 --2- 172.21.15.152:0/2771364015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887aeaf0 0x55eb887aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.728+0000 7f0e38a5d640 1 -- 172.21.15.152:0/2771364015 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb886ba340 con 0x55eb887b9cb0 2024-09-11T22:53:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.729+0000 7f0e38a5d640 1 --2- 172.21.15.152:0/2771364015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887b9cb0 0x55eb887bc0a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009cd0 tx=0x7f0e2c005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e5850d0f6e4bd61 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.729+0000 7f0e1a7fc640 1 -- 172.21.15.152:0/2771364015 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c048070 con 0x55eb887b9cb0 2024-09-11T22:53:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.729+0000 7f0e1a7fc640 1 -- 172.21.15.152:0/2771364015 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e2c003e50 con 0x55eb887b9cb0 2024-09-11T22:53:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.729+0000 7f0e1a7fc640 1 -- 172.21.15.152:0/2771364015 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c034070 con 0x55eb887b9cb0 2024-09-11T22:53:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.730+0000 7f0e3d254740 1 -- 172.21.15.152:0/2771364015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887b9cb0 msgr2=0x55eb887bc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.730+0000 7f0e3d254740 1 --2- 172.21.15.152:0/2771364015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887b9cb0 0x55eb887bc0a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009cd0 tx=0x7f0e2c005c50 comp rx=0 tx=0).stop 2024-09-11T22:53:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.730+0000 7f0e3d254740 1 -- 172.21.15.152:0/2771364015 shutdown_connections 2024-09-11T22:53:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.730+0000 7f0e3d254740 1 --2- 172.21.15.152:0/2771364015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887b9cb0 0x55eb887bc0a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.730+0000 7f0e3d254740 1 --2- 172.21.15.152:0/2771364015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887af410 0x55eb887b9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.730+0000 7f0e3d254740 1 --2- 172.21.15.152:0/2771364015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887aeaf0 0x55eb887aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.730+0000 7f0e3d254740 1 -- 172.21.15.152:0/2771364015 >> 172.21.15.152:0/2771364015 conn(0x55eb887ab060 msgr2=0x55eb887ab440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.730+0000 7f0e3d254740 1 -- 172.21.15.152:0/2771364015 shutdown_connections 2024-09-11T22:53:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.730+0000 7f0e3d254740 1 -- 172.21.15.152:0/2771364015 wait complete. 2024-09-11T22:53:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.730+0000 7f0e3d254740 1 -- start start 2024-09-11T22:53:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.730+0000 7f0e3d254740 1 Processor -- start 2024-09-11T22:53:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e3d254740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 0x55eb887b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e3d254740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887af410 0x55eb887b7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e3d254740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887b9cb0 0x55eb887b7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e3d254740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55eb887bc880 con 0x55eb887af410 2024-09-11T22:53:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e3d254740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55eb8869c290 con 0x55eb887aeaf0 2024-09-11T22:53:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e3d254740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55eb887bca00 con 0x55eb887b9cb0 2024-09-11T22:53:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e3d254740 10 client.0 _ll_register_callbacks cb 0x55eb886cd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e33fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 0x55eb887b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e337fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887af410 0x55eb887b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e33fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 0x55eb887b6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:45192/0 (socket says 172.21.15.152:45192) 2024-09-11T22:53:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e33fff640 1 -- 172.21.15.152:0/2447521625 learned_addr learned my addr 172.21.15.152:0/2447521625 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e33fff640 1 -- 172.21.15.152:0/2447521625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887b9cb0 msgr2=0x55eb887b7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e33fff640 1 --2- 172.21.15.152:0/2447521625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887b9cb0 0x55eb887b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e33fff640 1 -- 172.21.15.152:0/2447521625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887af410 msgr2=0x55eb887b7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e33fff640 1 --2- 172.21.15.152:0/2447521625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887af410 0x55eb887b7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e33fff640 1 -- 172.21.15.152:0/2447521625 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb88426210 con 0x55eb887aeaf0 2024-09-11T22:53:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.731+0000 7f0e337fe640 1 --2- 172.21.15.152:0/2447521625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887af410 0x55eb887b7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.732+0000 7f0e33fff640 1 --2- 172.21.15.152:0/2447521625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 0x55eb887b6bd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0e24003a10 tx=0x7f0e24004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ee4fa1eba33fe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.732+0000 7f0e137fe640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-11T22:53:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.732+0000 7f0e137fe640 1 -- 172.21.15.152:0/2447521625 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2401fe00 con 0x55eb887aeaf0 2024-09-11T22:53:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.732+0000 7f0e3d254740 1 -- 172.21.15.152:0/2447521625 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e2c003e50 con 0x55eb887aeaf0 2024-09-11T22:53:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.732+0000 7f0e137fe640 1 -- 172.21.15.152:0/2447521625 <== mon.2 v2:172.21.15.191:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f0e240273e0 con 0x55eb887aeaf0 2024-09-11T22:53:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.732+0000 7f0e137fe640 1 -- 172.21.15.152:0/2447521625 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0e240275e0 con 0x55eb887aeaf0 2024-09-11T22:53:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.732+0000 7f0e3d254740 10 client.25184 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.732+0000 7f0e3d254740 1 -- 172.21.15.152:0/2447521625 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb884f2620 con 0x55eb887aeaf0 2024-09-11T22:53:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.733+0000 7f0e137fe640 1 -- 172.21.15.152:0/2447521625 <== mon.2 v2:172.21.15.191:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e2c003e50 con 0x55eb887aeaf0 2024-09-11T22:53:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.733+0000 7f0e137fe640 1 -- 172.21.15.152:0/2447521625 <== mon.2 v2:172.21.15.191:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0e24026070 con 0x55eb887aeaf0 2024-09-11T22:53:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.733+0000 7f0e3d254740 10 client.25184 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.733+0000 7f0e3d254740 10 client.25184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.733+0000 7f0e3d254740 20 client.25184 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.733+0000 7f0e3d254740 1 --2- 172.21.15.152:0/2447521625 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55eb887d2500 0x55eb887d4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.733+0000 7f0e3d254740 4 client.25184 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.733+0000 7f0e3d254740 1 -- 172.21.15.152:0/2447521625 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb88720a20 con 0x55eb887d2500 2024-09-11T22:53:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.733+0000 7f0e337fe640 1 --2- 172.21.15.152:0/2447521625 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55eb887d2500 0x55eb887d4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.734+0000 7f0e137fe640 10 client.25184 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:55.739 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.734+0000 7f0e337fe640 1 --2- 172.21.15.152:0/2447521625 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55eb887d2500 0x55eb887d4990 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d21eb2d27285e51 server_cookie=ae6adaae18cd0065 in_seq=0 out_seq=0 2024-09-11T22:53:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.734+0000 7f0e137fe640 1 -- 172.21.15.152:0/2447521625 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb88720a20 con 0x55eb887d2500 2024-09-11T22:53:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.734+0000 7f0e137fe640 10 client.25184 handle_command_reply: tid=0 2024-09-11T22:53:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.734+0000 7f0e3d254740 1 client.25184 shutdown 2024-09-11T22:53:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.735+0000 7f0e3d254740 1 -- 172.21.15.152:0/2447521625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 msgr2=0x55eb887b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.735+0000 7f0e3d254740 1 --2- 172.21.15.152:0/2447521625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 0x55eb887b6bd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0e24003a10 tx=0x7f0e24004730 comp rx=0 tx=0).stop 2024-09-11T22:53:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.735+0000 7f0e3d254740 1 -- 172.21.15.152:0/2447521625 shutdown_connections 2024-09-11T22:53:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.735+0000 7f0e3d254740 1 --2- 172.21.15.152:0/2447521625 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887b9cb0 0x55eb887b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.735+0000 7f0e3d254740 1 --2- 172.21.15.152:0/2447521625 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887af410 0x55eb887b7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.735+0000 7f0e3d254740 1 --2- 172.21.15.152:0/2447521625 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55eb887d2500 0x55eb887d4990 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.735+0000 7f0e3d254740 1 --2- 172.21.15.152:0/2447521625 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 0x55eb887b6bd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.735+0000 7f0e137fe640 0 client.25184 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.735+0000 7f0e3d254740 1 -- 172.21.15.152:0/2447521625 >> 172.21.15.152:0/2447521625 conn(0x55eb887ab060 msgr2=0x55eb887b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.735+0000 7f0e3d254740 1 -- 172.21.15.152:0/2447521625 shutdown_connections 2024-09-11T22:53:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.735+0000 7f0e3d254740 1 -- 172.21.15.152:0/2447521625 wait complete. 2024-09-11T22:53:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.735+0000 7f0e3d254740 20 client.25184 trim_cache size 0 max 16384 2024-09-11T22:53:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.749+0000 7f0e3d254740 1 Processor -- start 2024-09-11T22:53:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.749+0000 7f0e3d254740 1 -- start start 2024-09-11T22:53:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e3d254740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 0x55eb887b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e3d254740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887af410 0x55eb887da2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e3d254740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887b9cb0 0x55eb887da7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e3d254740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0e341ab320 con 0x55eb887af410 2024-09-11T22:53:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e3d254740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0e341ace60 con 0x55eb887aeaf0 2024-09-11T22:53:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e3d254740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0e341ab9f0 con 0x55eb887b9cb0 2024-09-11T22:53:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e337fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887af410 0x55eb887da2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e337fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887af410 0x55eb887da2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49490/0 (socket says 172.21.15.152:49490) 2024-09-11T22:53:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e337fe640 1 -- 172.21.15.152:0/1860055441 learned_addr learned my addr 172.21.15.152:0/1860055441 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e33fff640 1 --2- 172.21.15.152:0/1860055441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 0x55eb887b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e38a5d640 1 --2- 172.21.15.152:0/1860055441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887b9cb0 0x55eb887da7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e33fff640 1 -- 172.21.15.152:0/1860055441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887b9cb0 msgr2=0x55eb887da7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e33fff640 1 --2- 172.21.15.152:0/1860055441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887b9cb0 0x55eb887da7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e33fff640 1 -- 172.21.15.152:0/1860055441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887af410 msgr2=0x55eb887da2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e33fff640 1 --2- 172.21.15.152:0/1860055441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887af410 0x55eb887da2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e33fff640 1 -- 172.21.15.152:0/1860055441 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb887d10a0 con 0x55eb887aeaf0 2024-09-11T22:53:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.750+0000 7f0e337fe640 1 --2- 172.21.15.152:0/1860055441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887af410 0x55eb887da2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.751+0000 7f0e33fff640 1 --2- 172.21.15.152:0/1860055441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 0x55eb887b8b40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0e24016040 tx=0x7f0e2401f8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1f2bcdf86d4b102 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.751+0000 7f0e19ffb640 1 -- 172.21.15.152:0/1860055441 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2403e070 con 0x55eb887aeaf0 2024-09-11T22:53:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.751+0000 7f0e19ffb640 1 -- 172.21.15.152:0/1860055441 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e24003f90 con 0x55eb887aeaf0 2024-09-11T22:53:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.751+0000 7f0e19ffb640 1 -- 172.21.15.152:0/1860055441 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24035030 con 0x55eb887aeaf0 2024-09-11T22:53:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.752+0000 7f0e3d254740 1 -- 172.21.15.152:0/1860055441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 msgr2=0x55eb887b8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.752+0000 7f0e3d254740 1 --2- 172.21.15.152:0/1860055441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 0x55eb887b8b40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0e24016040 tx=0x7f0e2401f8c0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.752+0000 7f0e3d254740 1 -- 172.21.15.152:0/1860055441 shutdown_connections 2024-09-11T22:53:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.752+0000 7f0e3d254740 1 --2- 172.21.15.152:0/1860055441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887b9cb0 0x55eb887da7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.752+0000 7f0e3d254740 1 --2- 172.21.15.152:0/1860055441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887af410 0x55eb887da2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.752+0000 7f0e3d254740 1 --2- 172.21.15.152:0/1860055441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 0x55eb887b8b40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.752+0000 7f0e3d254740 1 -- 172.21.15.152:0/1860055441 >> 172.21.15.152:0/1860055441 conn(0x55eb887ab060 msgr2=0x55eb887b8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.752+0000 7f0e3d254740 1 -- 172.21.15.152:0/1860055441 shutdown_connections 2024-09-11T22:53:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.752+0000 7f0e3d254740 1 -- 172.21.15.152:0/1860055441 wait complete. 2024-09-11T22:53:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.752+0000 7f0e3d254740 1 -- start start 2024-09-11T22:53:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.753+0000 7f0e3d254740 1 Processor -- start 2024-09-11T22:53:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.753+0000 7f0e3d254740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 0x55eb887cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.753+0000 7f0e3d254740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887af410 0x55eb887c6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.753+0000 7f0e3d254740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887b9cb0 0x55eb887c6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.753+0000 7f0e3d254740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0e341ac790 con 0x55eb887b9cb0 2024-09-11T22:53:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.753+0000 7f0e3d254740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0e341ac0c0 con 0x55eb887aeaf0 2024-09-11T22:53:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.753+0000 7f0e3d254740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0e341a9360 con 0x55eb887af410 2024-09-11T22:53:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.753+0000 7f0e3d254740 10 client.0 _ll_register_callbacks cb 0x7f0e24004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.754+0000 7f0e33fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 0x55eb887cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.754+0000 7f0e337fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887af410 0x55eb887c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.754+0000 7f0e38a5d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887b9cb0 0x55eb887c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.754+0000 7f0e337fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887af410 0x55eb887c6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:55766/0 (socket says 172.21.15.152:55766) 2024-09-11T22:53:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.754+0000 7f0e337fe640 1 -- 172.21.15.152:0/995036090 learned_addr learned my addr 172.21.15.152:0/995036090 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.754+0000 7f0e337fe640 1 -- 172.21.15.152:0/995036090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 msgr2=0x55eb887cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.754+0000 7f0e337fe640 1 --2- 172.21.15.152:0/995036090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 0x55eb887cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.754+0000 7f0e337fe640 1 -- 172.21.15.152:0/995036090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887b9cb0 msgr2=0x55eb887c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.754+0000 7f0e337fe640 1 --2- 172.21.15.152:0/995036090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887b9cb0 0x55eb887c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.754+0000 7f0e337fe640 1 -- 172.21.15.152:0/995036090 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb887c6db0 con 0x55eb887af410 2024-09-11T22:53:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.754+0000 7f0e38a5d640 1 --2- 172.21.15.152:0/995036090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887b9cb0 0x55eb887c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.754+0000 7f0e337fe640 1 --2- 172.21.15.152:0/995036090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887af410 0x55eb887c6020 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e280d3030 tx=0x7f0e28014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13f6e1b71ade3a07 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.754+0000 7f0e137fe640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-11T22:53:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.755+0000 7f0e137fe640 1 -- 172.21.15.152:0/995036090 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e280b7070 con 0x55eb887af410 2024-09-11T22:53:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.755+0000 7f0e137fe640 1 -- 172.21.15.152:0/995036090 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f0e28096f20 con 0x55eb887af410 2024-09-11T22:53:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.755+0000 7f0e3d254740 1 -- 172.21.15.152:0/995036090 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e24003f90 con 0x55eb887af410 2024-09-11T22:53:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.755+0000 7f0e137fe640 1 -- 172.21.15.152:0/995036090 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0e280a3470 con 0x55eb887af410 2024-09-11T22:53:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.755+0000 7f0e3d254740 10 client.25243 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.755+0000 7f0e3d254740 1 -- 172.21.15.152:0/995036090 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eb8883d370 con 0x55eb887af410 2024-09-11T22:53:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.755+0000 7f0e137fe640 1 -- 172.21.15.152:0/995036090 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e24003f90 con 0x55eb887af410 2024-09-11T22:53:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.755+0000 7f0e137fe640 1 -- 172.21.15.152:0/995036090 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0e2809bb30 con 0x55eb887af410 2024-09-11T22:53:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.756+0000 7f0e3d254740 10 client.25243 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.756+0000 7f0e3d254740 10 client.25243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.756+0000 7f0e3d254740 20 client.25243 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.756+0000 7f0e3d254740 1 --2- 172.21.15.152:0/995036090 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55eb8883daa0 0x55eb8883fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.756+0000 7f0e3d254740 4 client.25243 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:53:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.756+0000 7f0e3d254740 1 -- 172.21.15.152:0/995036090 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb8871ad70 con 0x55eb8883daa0 2024-09-11T22:53:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.756+0000 7f0e33fff640 1 --2- 172.21.15.152:0/995036090 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55eb8883daa0 0x55eb8883fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.756+0000 7f0e137fe640 10 client.25243 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.756+0000 7f0e33fff640 1 --2- 172.21.15.152:0/995036090 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55eb8883daa0 0x55eb8883fec0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=301941efd40b9403 server_cookie=fc8b652cd5151b31 in_seq=0 out_seq=0 2024-09-11T22:53:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.764+0000 7f0e137fe640 1 -- 172.21.15.152:0/995036090 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb8871ad70 con 0x55eb8883daa0 2024-09-11T22:53:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.764+0000 7f0e137fe640 10 client.25243 handle_command_reply: tid=0 2024-09-11T22:53:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.764+0000 7f0e3d254740 1 client.25243 shutdown 2024-09-11T22:53:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.764+0000 7f0e3d254740 1 -- 172.21.15.152:0/995036090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887af410 msgr2=0x55eb887c6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.764+0000 7f0e3d254740 1 --2- 172.21.15.152:0/995036090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887af410 0x55eb887c6020 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e280d3030 tx=0x7f0e28014040 comp rx=0 tx=0).stop 2024-09-11T22:53:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.764+0000 7f0e3d254740 1 -- 172.21.15.152:0/995036090 shutdown_connections 2024-09-11T22:53:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.764+0000 7f0e3d254740 1 --2- 172.21.15.152:0/995036090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55eb887b9cb0 0x55eb887c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.764+0000 7f0e3d254740 1 --2- 172.21.15.152:0/995036090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55eb887af410 0x55eb887c6020 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.764+0000 7f0e3d254740 1 --2- 172.21.15.152:0/995036090 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55eb8883daa0 0x55eb8883fec0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.764+0000 7f0e3d254740 1 --2- 172.21.15.152:0/995036090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55eb887aeaf0 0x55eb887cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.764+0000 7f0e137fe640 0 client.25243 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.764+0000 7f0e3d254740 1 -- 172.21.15.152:0/995036090 >> 172.21.15.152:0/995036090 conn(0x55eb887ab060 msgr2=0x55eb887b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.764+0000 7f0e3d254740 1 -- 172.21.15.152:0/995036090 shutdown_connections 2024-09-11T22:53:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.764+0000 7f0e3d254740 1 -- 172.21.15.152:0/995036090 wait complete. 2024-09-11T22:53:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.764+0000 7f0e3d254740 20 client.25243 trim_cache size 0 max 16384 2024-09-11T22:53:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:{ 2024-09-11T22:53:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "message": "", 2024-09-11T22:53:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "return_code": 0 2024-09-11T22:53:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:} 2024-09-11T22:53:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.765+0000 7f0e3a4c0640 1 -- 172.21.15.152:0/1007066131 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0e1c0a2d10 msgr2=0x7f0e1c0a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.765+0000 7f0e3a4c0640 1 --2- 172.21.15.152:0/1007066131 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0e1c0a2d10 0x7f0e1c0a51d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009bd0 tx=0x7f0e24005f90 comp rx=0 tx=0).stop 2024-09-11T22:53:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.765+0000 7f0e3a4c0640 1 -- 172.21.15.152:0/1007066131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e34101150 msgr2=0x7f0e34101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.765+0000 7f0e3a4c0640 1 --2- 172.21.15.152:0/1007066131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e34101150 0x7f0e34101ce0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800d770 tx=0x7f0e2800dc30 comp rx=0 tx=0).stop 2024-09-11T22:53:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.765+0000 7f0e3a4c0640 1 -- 172.21.15.152:0/1007066131 shutdown_connections 2024-09-11T22:53:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.765+0000 7f0e3a4c0640 1 --2- 172.21.15.152:0/1007066131 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0e1c0a2d10 0x7f0e1c0a51d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.765+0000 7f0e3a4c0640 1 --2- 172.21.15.152:0/1007066131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e341080f0 0x7f0e34102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.765+0000 7f0e3a4c0640 1 --2- 172.21.15.152:0/1007066131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e34101150 0x7f0e34101ce0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.765+0000 7f0e3a4c0640 1 --2- 172.21.15.152:0/1007066131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e34100830 0x7f0e341017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.765+0000 7f0e3a4c0640 1 -- 172.21.15.152:0/1007066131 >> 172.21.15.152:0/1007066131 conn(0x7f0e34075f20 msgr2=0x7f0e340ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.765+0000 7f0e3a4c0640 1 -- 172.21.15.152:0/1007066131 shutdown_connections 2024-09-11T22:53:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:53:55.765+0000 7f0e3a4c0640 1 -- 172.21.15.152:0/1007066131 wait complete. 2024-09-11T22:53:55.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.769+0000 7ff8e8dc7640 1 -- 172.21.15.152:0/3251419918 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc031de0 con 0x7ff8e4106f10 2024-09-11T22:53:55.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.770+0000 7ff8ec02d640 1 -- 172.21.15.152:0/3251419918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e4106f10 msgr2=0x7ff8e410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.770+0000 7ff8ec02d640 1 --2- 172.21.15.152:0/3251419918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e4106f10 0x7ff8e410ea90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009ca0 tx=0x7ff8dc02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.772+0000 7ff8ec02d640 1 -- 172.21.15.152:0/3251419918 shutdown_connections 2024-09-11T22:53:55.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.772+0000 7ff8ec02d640 1 --2- 172.21.15.152:0/3251419918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e4106f10 0x7ff8e410ea90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.772+0000 7ff8ec02d640 1 --2- 172.21.15.152:0/3251419918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e4102790 0x7ff8e41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.772+0000 7ff8ec02d640 1 --2- 172.21.15.152:0/3251419918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8e4101de0 0x7ff8e41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.772+0000 7ff8ec02d640 1 -- 172.21.15.152:0/3251419918 >> 172.21.15.152:0/3251419918 conn(0x7ff8e40fd8e0 msgr2=0x7ff8e40ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:55.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.773+0000 7ff8ec02d640 1 -- 172.21.15.152:0/3251419918 shutdown_connections 2024-09-11T22:53:55.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.773+0000 7ff8ec02d640 1 -- 172.21.15.152:0/3251419918 wait complete. 2024-09-11T22:53:55.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.774+0000 7ff8ec02d640 1 Processor -- start 2024-09-11T22:53:55.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.774+0000 7ff8ec02d640 1 -- start start 2024-09-11T22:53:55.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.774+0000 7ff8ec02d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8e4101de0 0x7ff8e41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.774+0000 7ff8ec02d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e4102790 0x7ff8e4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.774+0000 7ff8ec02d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e4106f10 0x7ff8e419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.774+0000 7ff8ec02d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e4111fc0 con 0x7ff8e4106f10 2024-09-11T22:53:55.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.774+0000 7ff8ec02d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e4111e40 con 0x7ff8e4101de0 2024-09-11T22:53:55.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.774+0000 7ff8ec02d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e4112140 con 0x7ff8e4102790 2024-09-11T22:53:55.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.774+0000 7ff8e9dc9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8e4101de0 0x7ff8e41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.775+0000 7ff8e9dc9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8e4101de0 0x7ff8e41983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:45242/0 (socket says 172.21.15.152:45242) 2024-09-11T22:53:55.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.775+0000 7ff8e9dc9640 1 -- 172.21.15.152:0/998413572 learned_addr learned my addr 172.21.15.152:0/998413572 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:55.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.775+0000 7ff8e95c8640 1 --2- 172.21.15.152:0/998413572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e4102790 0x7ff8e4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.775+0000 7ff8ea5ca640 1 --2- 172.21.15.152:0/998413572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e4106f10 0x7ff8e419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.775+0000 7ff8e9dc9640 1 -- 172.21.15.152:0/998413572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e4102790 msgr2=0x7ff8e4198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.775+0000 7ff8e9dc9640 1 --2- 172.21.15.152:0/998413572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e4102790 0x7ff8e4198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.775+0000 7ff8e9dc9640 1 -- 172.21.15.152:0/998413572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e4106f10 msgr2=0x7ff8e419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.775+0000 7ff8e9dc9640 1 --2- 172.21.15.152:0/998413572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e4106f10 0x7ff8e419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.775+0000 7ff8e9dc9640 1 -- 172.21.15.152:0/998413572 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e419d3b0 con 0x7ff8e4101de0 2024-09-11T22:53:55.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.775+0000 7ff8ea5ca640 1 --2- 172.21.15.152:0/998413572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e4106f10 0x7ff8e419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:53:55.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.775+0000 7ff8e9dc9640 1 --2- 172.21.15.152:0/998413572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8e4101de0 0x7ff8e41983e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800d950 tx=0x7ff8d800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:55.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.776+0000 7ff8d6ffd640 1 -- 172.21.15.152:0/998413572 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8017070 con 0x7ff8e4101de0 2024-09-11T22:53:55.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.776+0000 7ff8d6ffd640 1 -- 172.21.15.152:0/998413572 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8d8009c30 con 0x7ff8e4101de0 2024-09-11T22:53:55.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.776+0000 7ff8d6ffd640 1 -- 172.21.15.152:0/998413572 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d800f6b0 con 0x7ff8e4101de0 2024-09-11T22:53:55.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.776+0000 7ff8ec02d640 1 -- 172.21.15.152:0/998413572 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e419d6a0 con 0x7ff8e4101de0 2024-09-11T22:53:55.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.776+0000 7ff8ec02d640 1 -- 172.21.15.152:0/998413572 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e4074420 con 0x7ff8e4101de0 2024-09-11T22:53:55.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.779+0000 7ff8d6ffd640 1 -- 172.21.15.152:0/998413572 <== mon.2 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 23) ==== 138010+0+0 (secure 0 0 0) 0x7ff8d800f850 con 0x7ff8e4101de0 2024-09-11T22:53:55.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.779+0000 7ff8ee406740 1 Processor -- start 2024-09-11T22:53:55.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.779+0000 7ff8ee406740 1 -- start start 2024-09-11T22:53:55.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.780+0000 7ff8ee406740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de533cb0 0x55f9de534090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.780+0000 7ff8ee406740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de5345d0 0x55f9de53e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.780+0000 7ff8ee406740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de53ee90 0x55f9de541280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.780+0000 7ff8ee406740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f9de4912e0 con 0x55f9de5345d0 2024-09-11T22:53:55.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.780+0000 7ff8ee406740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f9de438ce0 con 0x55f9de533cb0 2024-09-11T22:53:55.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.780+0000 7ff8ee406740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f9de48d2a0 con 0x55f9de53ee90 2024-09-11T22:53:55.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.780+0000 7ff8e9dc9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de5345d0 0x55f9de53e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.780+0000 7ff8d6ffd640 1 --2- 172.21.15.152:0/998413572 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff8d00a2e80 0x7ff8d00a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.780+0000 7ff8e9dc9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de5345d0 0x55f9de53e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49520/0 (socket says 172.21.15.152:49520) 2024-09-11T22:53:55.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.780+0000 7ff8e9dc9640 1 -- 172.21.15.152:0/1528522661 learned_addr learned my addr 172.21.15.152:0/1528522661 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:55.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.780+0000 7ff8e95c8640 1 --2- 172.21.15.152:0/1528522661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de53ee90 0x55f9de541280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.780+0000 7ff8e95c8640 1 -- 172.21.15.152:0/1528522661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de533cb0 msgr2=0x55f9de534090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:55.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.780+0000 7ff8ea5ca640 1 --2- 172.21.15.152:0/1528522661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de533cb0 0x55f9de534090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.780+0000 7ff8d6ffd640 1 -- 172.21.15.152:0/998413572 <== mon.2 v2:172.21.15.191:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7ff8d80cf1e0 con 0x7ff8e4101de0 2024-09-11T22:53:55.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.781+0000 7ff8e95c8640 1 --2- 172.21.15.152:0/1528522661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de533cb0 0x55f9de534090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.781+0000 7ff8e95c8640 1 -- 172.21.15.152:0/1528522661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de5345d0 msgr2=0x55f9de53e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:55.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.781+0000 7ff8e95c8640 1 --2- 172.21.15.152:0/1528522661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de5345d0 0x55f9de53e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.781+0000 7ff8e95c8640 1 -- 172.21.15.152:0/1528522661 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9de41d440 con 0x55f9de53ee90 2024-09-11T22:53:55.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.781+0000 7ff8e9dc9640 1 --2- 172.21.15.152:0/1528522661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de5345d0 0x55f9de53e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:55.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.781+0000 7ff8ea5ca640 1 --2- 172.21.15.152:0/1528522661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de533cb0 0x55f9de534090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:53:55.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.781+0000 7ff8e95c8640 1 --2- 172.21.15.152:0/1528522661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de53ee90 0x55f9de541280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009bd0 tx=0x7ff8e0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=409a0ce0011fa19d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:55.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.781+0000 7ff8ea5ca640 1 --2- 172.21.15.152:0/998413572 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff8d00a2e80 0x7ff8d00a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.781+0000 7ff8cffff640 1 -- 172.21.15.152:0/1528522661 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0009d80 con 0x55f9de53ee90 2024-09-11T22:53:55.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.781+0000 7ff8cffff640 1 -- 172.21.15.152:0/1528522661 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e0005500 con 0x55f9de53ee90 2024-09-11T22:53:55.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.781+0000 7ff8cffff640 1 -- 172.21.15.152:0/1528522661 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0019610 con 0x55f9de53ee90 2024-09-11T22:53:55.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.782+0000 7ff8ee406740 1 -- 172.21.15.152:0/1528522661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de53ee90 msgr2=0x55f9de541280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.782+0000 7ff8ea5ca640 1 --2- 172.21.15.152:0/998413572 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff8d00a2e80 0x7ff8d00a5340 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc005fe0 tx=0x7ff8dc005ed0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:55.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.782+0000 7ff8ee406740 1 --2- 172.21.15.152:0/1528522661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de53ee90 0x55f9de541280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0009bd0 tx=0x7ff8e0011810 comp rx=0 tx=0).stop 2024-09-11T22:53:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.782+0000 7ff8ee406740 1 -- 172.21.15.152:0/1528522661 shutdown_connections 2024-09-11T22:53:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.782+0000 7ff8ee406740 1 --2- 172.21.15.152:0/1528522661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de53ee90 0x55f9de541280 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.782+0000 7ff8ee406740 1 --2- 172.21.15.152:0/1528522661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de5345d0 0x55f9de53e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.782+0000 7ff8ee406740 1 --2- 172.21.15.152:0/1528522661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de533cb0 0x55f9de534090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.782+0000 7ff8ee406740 1 -- 172.21.15.152:0/1528522661 >> 172.21.15.152:0/1528522661 conn(0x55f9de530240 msgr2=0x55f9de530620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.782+0000 7ff8ee406740 1 -- 172.21.15.152:0/1528522661 shutdown_connections 2024-09-11T22:53:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.782+0000 7ff8ee406740 1 -- 172.21.15.152:0/1528522661 wait complete. 2024-09-11T22:53:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.783+0000 7ff8ee406740 1 -- start start 2024-09-11T22:53:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.783+0000 7ff8ee406740 1 Processor -- start 2024-09-11T22:53:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.784+0000 7ff8ee406740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de533cb0 0x55f9de53e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.784+0000 7ff8ee406740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de5345d0 0x55f9de53ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.784+0000 7ff8ee406740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de53ee90 0x55f9de53bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.784+0000 7ff8ee406740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f9de541a60 con 0x55f9de53ee90 2024-09-11T22:53:55.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.784+0000 7ff8ee406740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f9de44c050 con 0x55f9de5345d0 2024-09-11T22:53:55.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.784+0000 7ff8e9dc9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de533cb0 0x55f9de53e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.784+0000 7ff8e9dc9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de533cb0 0x55f9de53e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:55808/0 (socket says 172.21.15.152:55808) 2024-09-11T22:53:55.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.784+0000 7ff8e9dc9640 1 -- 172.21.15.152:0/2033380660 learned_addr learned my addr 172.21.15.152:0/2033380660 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:55.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.784+0000 7ff8ee406740 1 -- 172.21.15.152:0/2033380660 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f9de541be0 con 0x55f9de533cb0 2024-09-11T22:53:55.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.784+0000 7ff8ea5ca640 1 --2- 172.21.15.152:0/2033380660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de53ee90 0x55f9de53bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.784+0000 7ff8ee406740 10 client.0 _ll_register_callbacks cb 0x55f9de48c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:55.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.784+0000 7ff8e9dc9640 1 -- 172.21.15.152:0/2033380660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de5345d0 msgr2=0x55f9de53ba00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:53:55.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.784+0000 7ff8e9dc9640 1 --2- 172.21.15.152:0/2033380660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de5345d0 0x55f9de53ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.784+0000 7ff8e9dc9640 1 -- 172.21.15.152:0/2033380660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de53ee90 msgr2=0x55f9de53bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:55.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.785+0000 7ff8e9dc9640 1 --2- 172.21.15.152:0/2033380660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de53ee90 0x55f9de53bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.785+0000 7ff8e9dc9640 1 -- 172.21.15.152:0/2033380660 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9de2023c0 con 0x55f9de533cb0 2024-09-11T22:53:55.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.785+0000 7ff8ea5ca640 1 --2- 172.21.15.152:0/2033380660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de53ee90 0x55f9de53bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:55.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.785+0000 7ff8e9dc9640 1 --2- 172.21.15.152:0/2033380660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de533cb0 0x55f9de53e750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8097a00 tx=0x7ff8d80cc8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baf759aa9e653967 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:55.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.785+0000 7ff8cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-11T22:53:55.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.785+0000 7ff8cd7fa640 1 -- 172.21.15.152:0/2033380660 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8020030 con 0x55f9de533cb0 2024-09-11T22:53:55.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.785+0000 7ff8cd7fa640 1 -- 172.21.15.152:0/2033380660 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7ff8d8097c10 con 0x55f9de533cb0 2024-09-11T22:53:55.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.785+0000 7ff8ee406740 1 -- 172.21.15.152:0/2033380660 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e0005500 con 0x55f9de533cb0 2024-09-11T22:53:55.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.785+0000 7ff8cd7fa640 1 -- 172.21.15.152:0/2033380660 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8d809b5a0 con 0x55f9de533cb0 2024-09-11T22:53:55.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.785+0000 7ff8ee406740 10 client.25255 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:55.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.785+0000 7ff8ee406740 1 -- 172.21.15.152:0/2033380660 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f9de54d380 con 0x55f9de533cb0 2024-09-11T22:53:55.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.786+0000 7ff8cd7fa640 1 -- 172.21.15.152:0/2033380660 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8d809f950 con 0x55f9de533cb0 2024-09-11T22:53:55.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.786+0000 7ff8cd7fa640 1 -- 172.21.15.152:0/2033380660 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff8d80d1110 con 0x55f9de533cb0 2024-09-11T22:53:55.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.786+0000 7ff8ee406740 10 client.25255 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:55.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.786+0000 7ff8ee406740 10 client.25255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:55.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.786+0000 7ff8ee406740 20 client.25255 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:55.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.786+0000 7ff8ee406740 1 --2- 172.21.15.152:0/2033380660 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f9de54ed10 0x55f9de5511d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.786+0000 7ff8ee406740 4 client.25255 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:53:55.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.786+0000 7ff8ee406740 1 -- 172.21.15.152:0/2033380660 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff8e0005500 con 0x55f9de54ed10 2024-09-11T22:53:55.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.787+0000 7ff8e95c8640 1 --2- 172.21.15.152:0/2033380660 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f9de54ed10 0x55f9de5511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.787+0000 7ff8cd7fa640 10 client.25255 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:55.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.787+0000 7ff8e95c8640 1 --2- 172.21.15.152:0/2033380660 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f9de54ed10 0x55f9de5511d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a99b00b1d82bc2f8 server_cookie=f357224dedc1a04a in_seq=0 out_seq=0 2024-09-11T22:53:55.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.788+0000 7ff8cd7fa640 1 -- 172.21.15.152:0/2033380660 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff8e0005500 con 0x55f9de54ed10 2024-09-11T22:53:55.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.788+0000 7ff8cd7fa640 10 client.25255 handle_command_reply: tid=0 2024-09-11T22:53:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.788+0000 7ff8ee406740 1 client.25255 shutdown 2024-09-11T22:53:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.789+0000 7ff8ee406740 1 -- 172.21.15.152:0/2033380660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de533cb0 msgr2=0x55f9de53e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.789+0000 7ff8ee406740 1 --2- 172.21.15.152:0/2033380660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de533cb0 0x55f9de53e750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8097a00 tx=0x7ff8d80cc8d0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.789+0000 7ff8ee406740 1 -- 172.21.15.152:0/2033380660 shutdown_connections 2024-09-11T22:53:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.789+0000 7ff8ee406740 1 --2- 172.21.15.152:0/2033380660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de53ee90 0x55f9de53bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.790+0000 7ff8ee406740 1 --2- 172.21.15.152:0/2033380660 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f9de54ed10 0x55f9de5511d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.790+0000 7ff8ee406740 1 --2- 172.21.15.152:0/2033380660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de5345d0 0x55f9de53ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.790+0000 7ff8cd7fa640 0 client.25255 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.790+0000 7ff8ee406740 1 --2- 172.21.15.152:0/2033380660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de533cb0 0x55f9de53e750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.790+0000 7ff8ee406740 1 -- 172.21.15.152:0/2033380660 >> 172.21.15.152:0/2033380660 conn(0x55f9de530240 msgr2=0x55f9de526b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.790+0000 7ff8ee406740 1 -- 172.21.15.152:0/2033380660 shutdown_connections 2024-09-11T22:53:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.790+0000 7ff8ee406740 1 -- 172.21.15.152:0/2033380660 wait complete. 2024-09-11T22:53:55.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.790+0000 7ff8ee406740 20 client.25255 trim_cache size 0 max 16384 2024-09-11T22:53:55.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.823+0000 7ff8ee406740 1 Processor -- start 2024-09-11T22:53:55.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.823+0000 7ff8ee406740 1 -- start start 2024-09-11T22:53:55.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.823+0000 7ff8ee406740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de533cb0 0x55f9de54acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.823+0000 7ff8ee406740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de5345d0 0x55f9de530620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.823+0000 7ff8ee406740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de53ee90 0x55f9de536f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.823+0000 7ff8ee406740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e41ab530 con 0x55f9de533cb0 2024-09-11T22:53:55.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.823+0000 7ff8ee406740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e41ad070 con 0x55f9de53ee90 2024-09-11T22:53:55.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.823+0000 7ff8ee406740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e41abc00 con 0x55f9de5345d0 2024-09-11T22:53:55.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.823+0000 7ff8e9dc9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de533cb0 0x55f9de54acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.823+0000 7ff8ea5ca640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de53ee90 0x55f9de536f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.823+0000 7ff8e9dc9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de533cb0 0x55f9de54acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49542/0 (socket says 172.21.15.152:49542) 2024-09-11T22:53:55.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.823+0000 7ff8e9dc9640 1 -- 172.21.15.152:0/356428174 learned_addr learned my addr 172.21.15.152:0/356428174 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:55.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.823+0000 7ff8e95c8640 1 --2- 172.21.15.152:0/356428174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de5345d0 0x55f9de530620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.824+0000 7ff8e9dc9640 1 -- 172.21.15.152:0/356428174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de5345d0 msgr2=0x55f9de530620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:55.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.824+0000 7ff8e9dc9640 1 --2- 172.21.15.152:0/356428174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de5345d0 0x55f9de530620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.824+0000 7ff8e9dc9640 1 -- 172.21.15.152:0/356428174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de53ee90 msgr2=0x55f9de536f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:55.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.824+0000 7ff8e9dc9640 1 --2- 172.21.15.152:0/356428174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de53ee90 0x55f9de536f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.824+0000 7ff8e9dc9640 1 -- 172.21.15.152:0/356428174 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9de53eac0 con 0x55f9de533cb0 2024-09-11T22:53:55.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.824+0000 7ff8e9dc9640 1 --2- 172.21.15.152:0/356428174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de533cb0 0x55f9de54acc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8000c00 tx=0x7ff8d800df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa7357d44b6125d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:55.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.824+0000 7ff8cf7fe640 1 -- 172.21.15.152:0/356428174 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d80b9070 con 0x55f9de533cb0 2024-09-11T22:53:55.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.824+0000 7ff8cf7fe640 1 -- 172.21.15.152:0/356428174 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8d80ccdc0 con 0x55f9de533cb0 2024-09-11T22:53:55.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.824+0000 7ff8cf7fe640 1 -- 172.21.15.152:0/356428174 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8020070 con 0x55f9de533cb0 2024-09-11T22:53:55.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.825+0000 7ff8ee406740 1 -- 172.21.15.152:0/356428174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de533cb0 msgr2=0x55f9de54acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.825+0000 7ff8ee406740 1 --2- 172.21.15.152:0/356428174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de533cb0 0x55f9de54acc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8000c00 tx=0x7ff8d800df40 comp rx=0 tx=0).stop 2024-09-11T22:53:55.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.825+0000 7ff8ee406740 1 -- 172.21.15.152:0/356428174 shutdown_connections 2024-09-11T22:53:55.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.825+0000 7ff8ee406740 1 --2- 172.21.15.152:0/356428174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de53ee90 0x55f9de536f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.825+0000 7ff8ee406740 1 --2- 172.21.15.152:0/356428174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de5345d0 0x55f9de530620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.825+0000 7ff8ee406740 1 --2- 172.21.15.152:0/356428174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de533cb0 0x55f9de54acc0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.825+0000 7ff8ee406740 1 -- 172.21.15.152:0/356428174 >> 172.21.15.152:0/356428174 conn(0x55f9de530240 msgr2=0x55f9de536840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:55.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.826+0000 7ff8ee406740 1 -- 172.21.15.152:0/356428174 shutdown_connections 2024-09-11T22:53:55.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.826+0000 7ff8ee406740 1 -- 172.21.15.152:0/356428174 wait complete. 2024-09-11T22:53:55.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.826+0000 7ff8ee406740 1 -- start start 2024-09-11T22:53:55.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.826+0000 7ff8ee406740 1 Processor -- start 2024-09-11T22:53:55.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8ee406740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de5345d0 0x55f9de547240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8ee406740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de53ee90 0x55f9de5424d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8ee406740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de54ed10 0x55f9de542a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8ee406740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e41ac9a0 con 0x55f9de5345d0 2024-09-11T22:53:55.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8ee406740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e41ac2d0 con 0x55f9de54ed10 2024-09-11T22:53:55.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8ee406740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e41a94f0 con 0x55f9de53ee90 2024-09-11T22:53:55.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8ee406740 10 client.0 _ll_register_callbacks cb 0x7ff8d809b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:53:55.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8e9dc9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de5345d0 0x55f9de547240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8e9dc9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de5345d0 0x55f9de547240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49546/0 (socket says 172.21.15.152:49546) 2024-09-11T22:53:55.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8e9dc9640 1 -- 172.21.15.152:0/1596539685 learned_addr learned my addr 172.21.15.152:0/1596539685 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:53:55.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8ea5ca640 1 --2- 172.21.15.152:0/1596539685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de54ed10 0x55f9de542a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8e95c8640 1 --2- 172.21.15.152:0/1596539685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de53ee90 0x55f9de5424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8ea5ca640 1 -- 172.21.15.152:0/1596539685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de53ee90 msgr2=0x55f9de5424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8ea5ca640 1 --2- 172.21.15.152:0/1596539685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de53ee90 0x55f9de5424d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8ea5ca640 1 -- 172.21.15.152:0/1596539685 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de5345d0 msgr2=0x55f9de547240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:53:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8ea5ca640 1 --2- 172.21.15.152:0/1596539685 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de5345d0 0x55f9de547240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8ea5ca640 1 -- 172.21.15.152:0/1596539685 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9de542f50 con 0x55f9de54ed10 2024-09-11T22:53:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8e9dc9640 1 --2- 172.21.15.152:0/1596539685 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de5345d0 0x55f9de547240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:53:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.827+0000 7ff8ea5ca640 1 --2- 172.21.15.152:0/1596539685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de54ed10 0x55f9de542a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc006010 tx=0x7ff8dc02d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d34c36d7143f2f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:53:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.828+0000 7ff8cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-11T22:53:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.828+0000 7ff8cd7fa640 1 -- 172.21.15.152:0/1596539685 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc055060 con 0x55f9de54ed10 2024-09-11T22:53:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.828+0000 7ff8cd7fa640 1 -- 172.21.15.152:0/1596539685 <== mon.2 v2:172.21.15.191:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7ff8dc004250 con 0x55f9de54ed10 2024-09-11T22:53:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.828+0000 7ff8ee406740 1 -- 172.21.15.152:0/1596539685 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8d80ccdc0 con 0x55f9de54ed10 2024-09-11T22:53:55.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.828+0000 7ff8cd7fa640 1 -- 172.21.15.152:0/1596539685 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff8dc004450 con 0x55f9de54ed10 2024-09-11T22:53:55.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.828+0000 7ff8ee406740 10 client.25202 fetch_fsmap learned FSMap version 28 2024-09-11T22:53:55.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.828+0000 7ff8ee406740 1 -- 172.21.15.152:0/1596539685 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f9de5b9790 con 0x55f9de54ed10 2024-09-11T22:53:55.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.828+0000 7ff8cd7fa640 1 -- 172.21.15.152:0/1596539685 <== mon.2 v2:172.21.15.191:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8dc036030 con 0x55f9de54ed10 2024-09-11T22:53:55.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.829+0000 7ff8cd7fa640 1 -- 172.21.15.152:0/1596539685 <== mon.2 v2:172.21.15.191:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff8dc050c90 con 0x55f9de54ed10 2024-09-11T22:53:55.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.829+0000 7ff8ee406740 10 client.25202 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:53:55.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.829+0000 7ff8ee406740 10 client.25202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:53:55.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.829+0000 7ff8ee406740 20 client.25202 populate_metadata read hostname 'smithi152' 2024-09-11T22:53:55.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.829+0000 7ff8ee406740 1 --2- 172.21.15.152:0/1596539685 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f9de5b9f00 0x55f9de5bc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:53:55.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.829+0000 7ff8ee406740 4 client.25202 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:53:55.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.829+0000 7ff8ee406740 1 -- 172.21.15.152:0/1596539685 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ff8d80ccdc0 con 0x55f9de5b9f00 2024-09-11T22:53:55.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.829+0000 7ff8e9dc9640 1 --2- 172.21.15.152:0/1596539685 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f9de5b9f00 0x55f9de5bc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:53:55.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.829+0000 7ff8cd7fa640 10 client.25202 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:55.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.829+0000 7ff8e9dc9640 1 --2- 172.21.15.152:0/1596539685 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f9de5b9f00 0x55f9de5bc320 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=609e2c0cde7ec2 server_cookie=a17b3baf3d2c806a in_seq=0 out_seq=0 2024-09-11T22:53:55.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.830+0000 7ff8cd7fa640 1 -- 172.21.15.152:0/1596539685 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7ff8d80ccdc0 con 0x55f9de5b9f00 2024-09-11T22:53:55.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.830+0000 7ff8cd7fa640 10 client.25202 handle_command_reply: tid=0 2024-09-11T22:53:55.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.830+0000 7ff8ee406740 1 client.25202 shutdown 2024-09-11T22:53:55.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.830+0000 7ff8ee406740 1 -- 172.21.15.152:0/1596539685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de54ed10 msgr2=0x55f9de542a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.830+0000 7ff8ee406740 1 --2- 172.21.15.152:0/1596539685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de54ed10 0x55f9de542a10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc006010 tx=0x7ff8dc02d000 comp rx=0 tx=0).stop 2024-09-11T22:53:55.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.830+0000 7ff8ee406740 1 -- 172.21.15.152:0/1596539685 shutdown_connections 2024-09-11T22:53:55.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.830+0000 7ff8ee406740 1 --2- 172.21.15.152:0/1596539685 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f9de5b9f00 0x55f9de5bc320 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.830+0000 7ff8ee406740 1 --2- 172.21.15.152:0/1596539685 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f9de54ed10 0x55f9de542a10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.830+0000 7ff8ee406740 1 --2- 172.21.15.152:0/1596539685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9de53ee90 0x55f9de5424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.830+0000 7ff8ee406740 1 --2- 172.21.15.152:0/1596539685 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f9de5345d0 0x55f9de547240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.830+0000 7ff8ee406740 1 -- 172.21.15.152:0/1596539685 >> 172.21.15.152:0/1596539685 conn(0x55f9de530240 msgr2=0x55f9de537040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:55.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.830+0000 7ff8cd7fa640 0 client.25202 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:53:55.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.830+0000 7ff8ee406740 1 -- 172.21.15.152:0/1596539685 shutdown_connections 2024-09-11T22:53:55.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.830+0000 7ff8ee406740 1 -- 172.21.15.152:0/1596539685 wait complete. 2024-09-11T22:53:55.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.831+0000 7ff8ee406740 20 client.25202 trim_cache size 0 max 16384 2024-09-11T22:53:55.843 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:53:55.844 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:53:55.844 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": { 2024-09-11T22:53:55.844 INFO:teuthology.orchestra.run.smithi152.stdout: "cfa50f24-529f-410f-ac75-dd50313f4112": { 2024-09-11T22:53:55.844 INFO:teuthology.orchestra.run.smithi152.stdout: "path": "/", 2024-09-11T22:53:55.844 INFO:teuthology.orchestra.run.smithi152.stdout: "tag": "cfa50f24-529f-410f-ac75-dd50313f4112", 2024-09-11T22:53:55.844 INFO:teuthology.orchestra.run.smithi152.stdout: "options": "recursive,force" 2024-09-11T22:53:55.844 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:53:55.844 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:53:55.844 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:53:55.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.831+0000 7ff8ec02d640 1 -- 172.21.15.152:0/998413572 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff8d00a2e80 msgr2=0x7ff8d00a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.831+0000 7ff8ec02d640 1 --2- 172.21.15.152:0/998413572 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff8d00a2e80 0x7ff8d00a5340 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc005fe0 tx=0x7ff8dc005ed0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.831+0000 7ff8ec02d640 1 -- 172.21.15.152:0/998413572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8e4101de0 msgr2=0x7ff8e41983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:53:55.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.831+0000 7ff8ec02d640 1 --2- 172.21.15.152:0/998413572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8e4101de0 0x7ff8e41983e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800d950 tx=0x7ff8d800de10 comp rx=0 tx=0).stop 2024-09-11T22:53:55.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.831+0000 7ff8ec02d640 1 -- 172.21.15.152:0/998413572 shutdown_connections 2024-09-11T22:53:55.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.831+0000 7ff8ec02d640 1 --2- 172.21.15.152:0/998413572 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff8d00a2e80 0x7ff8d00a5340 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.831+0000 7ff8ec02d640 1 --2- 172.21.15.152:0/998413572 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff8e4106f10 0x7ff8e419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.831+0000 7ff8ec02d640 1 --2- 172.21.15.152:0/998413572 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e4102790 0x7ff8e4198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.831+0000 7ff8ec02d640 1 --2- 172.21.15.152:0/998413572 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8e4101de0 0x7ff8e41983e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:53:55.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.831+0000 7ff8ec02d640 1 -- 172.21.15.152:0/998413572 >> 172.21.15.152:0/998413572 conn(0x7ff8e40fd8e0 msgr2=0x7ff8e40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:53:55.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.832+0000 7ff8ec02d640 1 -- 172.21.15.152:0/998413572 shutdown_connections 2024-09-11T22:53:55.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:53:55.832+0000 7ff8ec02d640 1 -- 172.21.15.152:0/998413572 wait complete. 2024-09-11T22:53:56.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:56 smithi191 ceph-mon[35211]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-11T22:53:56.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:56 smithi204 ceph-mon[35454]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-11T22:53:56.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:56 smithi152 ceph-mon[31181]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-11T22:53:56.784 INFO:tasks.cephfs.filesystem:scrub status for tag:cfa50f24-529f-410f-ac75-dd50313f4112 - {'path': '/', 'tag': 'cfa50f24-529f-410f-ac75-dd50313f4112', 'options': 'recursive,force'} 2024-09-11T22:53:58.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:58 smithi191 ceph-mon[35211]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 937 B/s wr, 9 op/s 2024-09-11T22:53:58.796 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:53:58 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T22:53:58.797 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:58 smithi152 ceph-mon[31181]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 937 B/s wr, 9 op/s 2024-09-11T22:53:58.797 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:53:58 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T22:53:58.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:58 smithi204 ceph-mon[35454]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 937 B/s wr, 9 op/s 2024-09-11T22:53:58.798 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:53:58 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T22:54:00.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:00 smithi191 ceph-mon[35211]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-09-11T22:54:00.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:00 smithi204 ceph-mon[35454]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-09-11T22:54:00.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:00 smithi152 ceph-mon[31181]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.7 KiB/s wr, 15 op/s 2024-09-11T22:54:00.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:54:00 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:54:00.379+0000 7f5eb806c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:54:00.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:54:00 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:54:00.379+0000 7f5eb806c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:54:00.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:54:00 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:54:00.380+0000 7f5eb806c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:54:00.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:54:00 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:54:00.380+0000 7f5eb806c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:54:00.623 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 22:54:00 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T22:54:00.380+0000 7f5eb806c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:54:02.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:02 smithi191 ceph-mon[35211]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-11T22:54:02.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:02 smithi191 ceph-mon[35211]: mgrmap e24: x(active, since 68s), standbys: z, y 2024-09-11T22:54:02.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:02 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:54:02.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:02 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:54:02.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:02 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:54:02.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:02 smithi204 ceph-mon[35454]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-11T22:54:02.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:02 smithi204 ceph-mon[35454]: mgrmap e24: x(active, since 68s), standbys: z, y 2024-09-11T22:54:02.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:02 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:54:02.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:02 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:54:02.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:02 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:54:02.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:02 smithi152 ceph-mon[31181]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-11T22:54:02.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:02 smithi152 ceph-mon[31181]: mgrmap e24: x(active, since 68s), standbys: z, y 2024-09-11T22:54:02.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:02 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:54:02.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:02 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:54:02.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:02 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:54:03.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:03 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:54:03.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:03 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:54:03.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:03 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:54:03.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:03 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:54:03.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:03 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:54:03.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:03 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:54:03.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:03 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:54:03.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:03 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:54:03.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:03 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:54:04.468 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:54:04.825 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:54:04.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:04 smithi152 ceph-mon[31181]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-11T22:54:05.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:04 smithi191 ceph-mon[35211]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-11T22:54:05.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:04 smithi204 ceph-mon[35454]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-11T22:54:05.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.425+0000 7fc25a54c640 1 -- 172.21.15.152:0/1717015584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc254100e20 msgr2=0x7fc254101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:05.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.425+0000 7fc25a54c640 1 --2- 172.21.15.152:0/1717015584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc254100e20 0x7fc254101200 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc244009900 tx=0x7fc24402a480 comp rx=0 tx=0).stop 2024-09-11T22:54:05.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.425+0000 7fc25a54c640 1 -- 172.21.15.152:0/1717015584 shutdown_connections 2024-09-11T22:54:05.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.425+0000 7fc25a54c640 1 --2- 172.21.15.152:0/1717015584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc254102ef0 0x7fc25410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:05.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.425+0000 7fc25a54c640 1 --2- 172.21.15.152:0/1717015584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc254102570 0x7fc2541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:05.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.425+0000 7fc25a54c640 1 --2- 172.21.15.152:0/1717015584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc254100e20 0x7fc254101200 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:05.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.425+0000 7fc25a54c640 1 -- 172.21.15.152:0/1717015584 >> 172.21.15.152:0/1717015584 conn(0x7fc2540fc7d0 msgr2=0x7fc2540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:05.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.426+0000 7fc25a54c640 1 -- 172.21.15.152:0/1717015584 shutdown_connections 2024-09-11T22:54:05.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.426+0000 7fc25a54c640 1 -- 172.21.15.152:0/1717015584 wait complete. 2024-09-11T22:54:05.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.427+0000 7fc25a54c640 1 Processor -- start 2024-09-11T22:54:05.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.427+0000 7fc25a54c640 1 -- start start 2024-09-11T22:54:05.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.428+0000 7fc25a54c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc254100e20 0x7fc254197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:05.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.428+0000 7fc25a54c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc254102570 0x7fc254197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:05.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.428+0000 7fc25a54c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc254102ef0 0x7fc25419bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:05.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.428+0000 7fc25a54c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc254110fd0 con 0x7fc254102ef0 2024-09-11T22:54:05.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.428+0000 7fc25a54c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc254110e50 con 0x7fc254102570 2024-09-11T22:54:05.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.428+0000 7fc25a54c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc254111150 con 0x7fc254100e20 2024-09-11T22:54:05.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.428+0000 7fc2537fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc254102570 0x7fc254197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:05.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.428+0000 7fc2537fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc254102570 0x7fc254197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:50124/0 (socket says 172.21.15.152:50124) 2024-09-11T22:54:05.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.428+0000 7fc2537fe640 1 -- 172.21.15.152:0/489912544 learned_addr learned my addr 172.21.15.152:0/489912544 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:05.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.428+0000 7fc258ae9640 1 --2- 172.21.15.152:0/489912544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc254102ef0 0x7fc25419bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:05.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.429+0000 7fc2537fe640 1 -- 172.21.15.152:0/489912544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc254100e20 msgr2=0x7fc254197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:05.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.429+0000 7fc2537fe640 1 --2- 172.21.15.152:0/489912544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc254100e20 0x7fc254197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:05.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.429+0000 7fc2537fe640 1 -- 172.21.15.152:0/489912544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc254102ef0 msgr2=0x7fc25419bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:05.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.429+0000 7fc2537fe640 1 --2- 172.21.15.152:0/489912544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc254102ef0 0x7fc25419bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:05.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.429+0000 7fc2537fe640 1 -- 172.21.15.152:0/489912544 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25419c3e0 con 0x7fc254102570 2024-09-11T22:54:05.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.429+0000 7fc2537fe640 1 --2- 172.21.15.152:0/489912544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc254102570 0x7fc254197970 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc24800e9f0 tx=0x7fc24800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:05.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.429+0000 7fc2517fa640 1 -- 172.21.15.152:0/489912544 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24800cd80 con 0x7fc254102570 2024-09-11T22:54:05.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.429+0000 7fc2517fa640 1 -- 172.21.15.152:0/489912544 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc248005500 con 0x7fc254102570 2024-09-11T22:54:05.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.429+0000 7fc25a54c640 1 -- 172.21.15.152:0/489912544 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc25419c6d0 con 0x7fc254102570 2024-09-11T22:54:05.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.429+0000 7fc25a54c640 1 -- 172.21.15.152:0/489912544 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc254069f70 con 0x7fc254102570 2024-09-11T22:54:05.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.429+0000 7fc2517fa640 1 -- 172.21.15.152:0/489912544 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248010690 con 0x7fc254102570 2024-09-11T22:54:05.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.431+0000 7fc25a54c640 1 -- 172.21.15.152:0/489912544 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc254101200 con 0x7fc254102570 2024-09-11T22:54:05.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.433+0000 7fc2517fa640 1 -- 172.21.15.152:0/489912544 <== mon.2 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 24) ==== 138067+0+0 (secure 0 0 0) 0x7fc2480046c0 con 0x7fc254102570 2024-09-11T22:54:05.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.433+0000 7fc2517fa640 1 --2- 172.21.15.152:0/489912544 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc23c0a2e90 0x7fc23c0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:05.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.434+0000 7fc253fff640 1 --2- 172.21.15.152:0/489912544 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc23c0a2e90 0x7fc23c0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:05.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.434+0000 7fc2517fa640 1 -- 172.21.15.152:0/489912544 <== mon.2 v2:172.21.15.191:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc2480cfaf0 con 0x7fc254102570 2024-09-11T22:54:05.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.434+0000 7fc253fff640 1 --2- 172.21.15.152:0/489912544 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc23c0a2e90 0x7fc23c0a5350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc2440096c0 tx=0x7fc244000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:05.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.438+0000 7fc2517fa640 1 -- 172.21.15.152:0/489912544 <== mon.2 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc24809c510 con 0x7fc254102570 2024-09-11T22:54:05.716 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.715+0000 7fc25a54c640 1 -- 172.21.15.152:0/489912544 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc25406a280 con 0x7fc254102570 2024-09-11T22:54:05.717 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.716+0000 7fc2517fa640 1 -- 172.21.15.152:0/489912544 <== mon.2 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7fc24809be10 con 0x7fc254102570 2024-09-11T22:54:05.717 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:54:05.718 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:54:05.718 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:54:05.719 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.719+0000 7fc25a54c640 1 -- 172.21.15.152:0/489912544 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc23c0a2e90 msgr2=0x7fc23c0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:05.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.719+0000 7fc25a54c640 1 --2- 172.21.15.152:0/489912544 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc23c0a2e90 0x7fc23c0a5350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc2440096c0 tx=0x7fc244000bc0 comp rx=0 tx=0).stop 2024-09-11T22:54:05.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.719+0000 7fc25a54c640 1 -- 172.21.15.152:0/489912544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc254102570 msgr2=0x7fc254197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:05.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.719+0000 7fc25a54c640 1 --2- 172.21.15.152:0/489912544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc254102570 0x7fc254197970 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc24800e9f0 tx=0x7fc24800edf0 comp rx=0 tx=0).stop 2024-09-11T22:54:05.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.719+0000 7fc25a54c640 1 -- 172.21.15.152:0/489912544 shutdown_connections 2024-09-11T22:54:05.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.719+0000 7fc25a54c640 1 --2- 172.21.15.152:0/489912544 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc23c0a2e90 0x7fc23c0a5350 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:05.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.719+0000 7fc25a54c640 1 --2- 172.21.15.152:0/489912544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc254102ef0 0x7fc25419bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:05.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.719+0000 7fc25a54c640 1 --2- 172.21.15.152:0/489912544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc254102570 0x7fc254197970 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc24800e9f0 tx=0x7fc24800edf0 comp rx=0 tx=0).stop 2024-09-11T22:54:05.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.719+0000 7fc25a54c640 1 --2- 172.21.15.152:0/489912544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc254100e20 0x7fc254197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:05.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.719+0000 7fc25a54c640 1 -- 172.21.15.152:0/489912544 >> 172.21.15.152:0/489912544 conn(0x7fc2540fc7d0 msgr2=0x7fc2540fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:05.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.719+0000 7fc25a54c640 1 -- 172.21.15.152:0/489912544 shutdown_connections 2024-09-11T22:54:05.720 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:05.719+0000 7fc25a54c640 1 -- 172.21.15.152:0/489912544 wait complete. 2024-09-11T22:54:06.577 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:06 smithi152 ceph-mon[31181]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 31 op/s 2024-09-11T22:54:06.578 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:06 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/489912544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:54:06.582 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24455 quiesce db --set-id 7c8da22d --release --await' 2024-09-11T22:54:06.582 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24455 quiesce db --set-id 7c8da22d --release --await 2024-09-11T22:54:06.976 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:54:07.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:06 smithi191 ceph-mon[35211]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 31 op/s 2024-09-11T22:54:07.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:06 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/489912544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:54:07.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:06 smithi204 ceph-mon[35454]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 31 op/s 2024-09-11T22:54:07.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:06 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/489912544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:54:07.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.581+0000 7fd1f8bd0640 1 -- 172.21.15.152:0/4019022566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1f4100e20 msgr2=0x7fd1f4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:07.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.581+0000 7fd1f8bd0640 1 --2- 172.21.15.152:0/4019022566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1f4100e20 0x7fd1f4101200 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009900 tx=0x7fd1e402a480 comp rx=0 tx=0).stop 2024-09-11T22:54:07.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.582+0000 7fd1f8bd0640 1 -- 172.21.15.152:0/4019022566 shutdown_connections 2024-09-11T22:54:07.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.582+0000 7fd1f8bd0640 1 --2- 172.21.15.152:0/4019022566 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f4102ef0 0x7fd1f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.582+0000 7fd1f8bd0640 1 --2- 172.21.15.152:0/4019022566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f4102570 0x7fd1f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.582+0000 7fd1f8bd0640 1 --2- 172.21.15.152:0/4019022566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1f4100e20 0x7fd1f4101200 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.582+0000 7fd1f8bd0640 1 -- 172.21.15.152:0/4019022566 >> 172.21.15.152:0/4019022566 conn(0x7fd1f40fc7d0 msgr2=0x7fd1f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:07.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.582+0000 7fd1f8bd0640 1 -- 172.21.15.152:0/4019022566 shutdown_connections 2024-09-11T22:54:07.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.582+0000 7fd1f8bd0640 1 -- 172.21.15.152:0/4019022566 wait complete. 2024-09-11T22:54:07.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.582+0000 7fd1f8bd0640 1 Processor -- start 2024-09-11T22:54:07.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.583+0000 7fd1f8bd0640 1 -- start start 2024-09-11T22:54:07.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.583+0000 7fd1f8bd0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1f4100e20 0x7fd1f419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.583+0000 7fd1f8bd0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f4102570 0x7fd1f419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.583+0000 7fd1f8bd0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f4102ef0 0x7fd1f41a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.583+0000 7fd1f8bd0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4110f80 con 0x7fd1f4100e20 2024-09-11T22:54:07.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.583+0000 7fd1f8bd0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4110e00 con 0x7fd1f4102ef0 2024-09-11T22:54:07.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.583+0000 7fd1f8bd0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4111100 con 0x7fd1f4102570 2024-09-11T22:54:07.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.583+0000 7fd1f1d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f4102570 0x7fd1f419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.583+0000 7fd1f259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1f4100e20 0x7fd1f419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.583+0000 7fd1f259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1f4100e20 0x7fd1f419edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48106/0 (socket says 172.21.15.152:48106) 2024-09-11T22:54:07.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.583+0000 7fd1f259c640 1 -- 172.21.15.152:0/906485876 learned_addr learned my addr 172.21.15.152:0/906485876 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:07.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.583+0000 7fd1f2d9d640 1 --2- 172.21.15.152:0/906485876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f4102ef0 0x7fd1f41a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.584+0000 7fd1f259c640 1 -- 172.21.15.152:0/906485876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f4102570 msgr2=0x7fd1f419f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:07.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.584+0000 7fd1f259c640 1 --2- 172.21.15.152:0/906485876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f4102570 0x7fd1f419f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.584+0000 7fd1f259c640 1 -- 172.21.15.152:0/906485876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f4102ef0 msgr2=0x7fd1f41a3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:07.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.584+0000 7fd1f259c640 1 --2- 172.21.15.152:0/906485876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f4102ef0 0x7fd1f41a3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.584+0000 7fd1f259c640 1 -- 172.21.15.152:0/906485876 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f41a3b30 con 0x7fd1f4100e20 2024-09-11T22:54:07.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.584+0000 7fd1f259c640 1 --2- 172.21.15.152:0/906485876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1f4100e20 0x7fd1f419edd0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4006010 tx=0x7fd1e4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:07.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.584+0000 7fd1e37fe640 1 -- 172.21.15.152:0/906485876 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e403f070 con 0x7fd1f4100e20 2024-09-11T22:54:07.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.584+0000 7fd1f8bd0640 1 -- 172.21.15.152:0/906485876 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f41a3dc0 con 0x7fd1f4100e20 2024-09-11T22:54:07.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.584+0000 7fd1e37fe640 1 -- 172.21.15.152:0/906485876 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e4031c30 con 0x7fd1f4100e20 2024-09-11T22:54:07.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.584+0000 7fd1e37fe640 1 -- 172.21.15.152:0/906485876 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4035030 con 0x7fd1f4100e20 2024-09-11T22:54:07.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.584+0000 7fd1f8bd0640 1 -- 172.21.15.152:0/906485876 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f41ab6b0 con 0x7fd1f4100e20 2024-09-11T22:54:07.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.588+0000 7fd1e37fe640 1 -- 172.21.15.152:0/906485876 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 138067+0+0 (secure 0 0 0) 0x7fd1e4042070 con 0x7fd1f4100e20 2024-09-11T22:54:07.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.588+0000 7fd1fafa9740 1 Processor -- start 2024-09-11T22:54:07.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.588+0000 7fd1fafa9740 1 -- start start 2024-09-11T22:54:07.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.588+0000 7fd1fafa9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a3795060 0x5578a3795440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.589+0000 7fd1fafa9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795980 0x5578a379fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.589+0000 7fd1fafa9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a37a0240 0x5578a37a2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.589+0000 7fd1fafa9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5578a36fa620 con 0x5578a3795980 2024-09-11T22:54:07.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.589+0000 7fd1fafa9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5578a36a1e50 con 0x5578a3795060 2024-09-11T22:54:07.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.589+0000 7fd1fafa9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5578a36f65e0 con 0x5578a37a0240 2024-09-11T22:54:07.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.589+0000 7fd1f259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795980 0x5578a379fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.589+0000 7fd1f2d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a3795060 0x5578a3795440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.589+0000 7fd1f1d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a37a0240 0x5578a37a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.589+0000 7fd1f2d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a3795060 0x5578a3795440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:50144/0 (socket says 172.21.15.152:50144) 2024-09-11T22:54:07.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.589+0000 7fd1f2d9d640 1 -- 172.21.15.152:0/170050026 learned_addr learned my addr 172.21.15.152:0/170050026 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:07.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.589+0000 7fd1f1d9b640 1 -- 172.21.15.152:0/170050026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a3795060 msgr2=0x5578a3795440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:07.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.589+0000 7fd1f1d9b640 1 --2- 172.21.15.152:0/170050026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a3795060 0x5578a3795440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.589+0000 7fd1f1d9b640 1 -- 172.21.15.152:0/170050026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795980 msgr2=0x5578a379fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:07.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.589+0000 7fd1f1d9b640 1 --2- 172.21.15.152:0/170050026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795980 0x5578a379fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.590+0000 7fd1f1d9b640 1 -- 172.21.15.152:0/170050026 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a3692d40 con 0x5578a37a0240 2024-09-11T22:54:07.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.590+0000 7fd1f259c640 1 --2- 172.21.15.152:0/170050026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795980 0x5578a379fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:54:07.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.590+0000 7fd1e37fe640 1 --2- 172.21.15.152:0/906485876 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd1dc0a2e10 0x7fd1dc0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.590+0000 7fd1e37fe640 1 -- 172.21.15.152:0/906485876 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fd1e40efd50 con 0x7fd1f4100e20 2024-09-11T22:54:07.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.590+0000 7fd1f1d9b640 1 --2- 172.21.15.152:0/170050026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a37a0240 0x5578a37a2630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd1f40ffe70 tx=0x7fd1e80113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1457d04ad132a02 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:07.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.590+0000 7fd1f2d9d640 1 --2- 172.21.15.152:0/906485876 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd1dc0a2e10 0x7fd1dc0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.590+0000 7fd1e0ff9640 1 -- 172.21.15.152:0/170050026 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8011de0 con 0x5578a37a0240 2024-09-11T22:54:07.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.591+0000 7fd1e0ff9640 1 -- 172.21.15.152:0/170050026 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e8012930 con 0x5578a37a0240 2024-09-11T22:54:07.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.591+0000 7fd1e0ff9640 1 -- 172.21.15.152:0/170050026 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e800faa0 con 0x5578a37a0240 2024-09-11T22:54:07.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.591+0000 7fd1f2d9d640 1 --2- 172.21.15.152:0/906485876 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd1dc0a2e10 0x7fd1dc0a52d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec011700 tx=0x7fd1ec009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:07.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.592+0000 7fd1fafa9740 1 -- 172.21.15.152:0/170050026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a37a0240 msgr2=0x5578a37a2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:07.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.592+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/170050026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a37a0240 0x5578a37a2630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd1f40ffe70 tx=0x7fd1e80113e0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.592+0000 7fd1fafa9740 1 -- 172.21.15.152:0/170050026 shutdown_connections 2024-09-11T22:54:07.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.592+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/170050026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a37a0240 0x5578a37a2630 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.592+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/170050026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795980 0x5578a379fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.592+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/170050026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a3795060 0x5578a3795440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.592+0000 7fd1fafa9740 1 -- 172.21.15.152:0/170050026 >> 172.21.15.152:0/170050026 conn(0x5578a37915f0 msgr2=0x5578a37919d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:07.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.592+0000 7fd1fafa9740 1 -- 172.21.15.152:0/170050026 shutdown_connections 2024-09-11T22:54:07.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.592+0000 7fd1fafa9740 1 -- 172.21.15.152:0/170050026 wait complete. 2024-09-11T22:54:07.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.593+0000 7fd1fafa9740 1 -- start start 2024-09-11T22:54:07.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.593+0000 7fd1fafa9740 1 Processor -- start 2024-09-11T22:54:07.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.593+0000 7fd1fafa9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a3795060 0x5578a379d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.593+0000 7fd1fafa9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 0x5578a379d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1fafa9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a37a0240 0x5578a379db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1fafa9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5578a37a2e10 con 0x5578a37a0240 2024-09-11T22:54:07.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1fafa9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5578a3682f10 con 0x5578a3795060 2024-09-11T22:54:07.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1fafa9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5578a37a2f90 con 0x5578a3795980 2024-09-11T22:54:07.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1f259c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a3795060 0x5578a379d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1f259c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a3795060 0x5578a379d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:50154/0 (socket says 172.21.15.152:50154) 2024-09-11T22:54:07.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1f1d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 0x5578a379d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1f259c640 1 -- 172.21.15.152:0/1185000295 learned_addr learned my addr 172.21.15.152:0/1185000295 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:07.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1f2d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a37a0240 0x5578a379db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1fafa9740 10 client.0 _ll_register_callbacks cb 0x5578a36a6db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:07.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1f1d9b640 1 -- 172.21.15.152:0/1185000295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a3795060 msgr2=0x5578a379d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:07.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1f1d9b640 1 --2- 172.21.15.152:0/1185000295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a3795060 0x5578a379d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1f1d9b640 1 -- 172.21.15.152:0/1185000295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a37a0240 msgr2=0x5578a379db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:07.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1f1d9b640 1 --2- 172.21.15.152:0/1185000295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a37a0240 0x5578a379db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1f1d9b640 1 -- 172.21.15.152:0/1185000295 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a3409580 con 0x5578a3795980 2024-09-11T22:54:07.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1f2d9d640 1 --2- 172.21.15.152:0/1185000295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a37a0240 0x5578a379db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:54:07.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.594+0000 7fd1fafa9740 1 -- 172.21.15.152:0/1185000295 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1e8012930 con 0x5578a3795980 2024-09-11T22:54:07.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.595+0000 7fd1f1d9b640 1 --2- 172.21.15.152:0/1185000295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 0x5578a379d5f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd1e801c040 tx=0x7fd1e8012680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2537771a2424f01 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:07.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.595+0000 7fd1d9ffb640 10 client.25282 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-11T22:54:07.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.595+0000 7fd1d9ffb640 1 -- 172.21.15.152:0/1185000295 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8032050 con 0x5578a3795980 2024-09-11T22:54:07.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.595+0000 7fd1d9ffb640 1 -- 172.21.15.152:0/1185000295 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fd1e8005040 con 0x5578a3795980 2024-09-11T22:54:07.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.595+0000 7fd1d9ffb640 1 -- 172.21.15.152:0/1185000295 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1e8028070 con 0x5578a3795980 2024-09-11T22:54:07.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.595+0000 7fd1fafa9740 10 client.25282 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:07.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.595+0000 7fd1fafa9740 1 -- 172.21.15.152:0/1185000295 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5578a34d3960 con 0x5578a3795980 2024-09-11T22:54:07.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.596+0000 7fd1d9ffb640 1 -- 172.21.15.152:0/1185000295 <== mon.1 v2:172.21.15.204:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd1e8005280 con 0x5578a3795980 2024-09-11T22:54:07.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.596+0000 7fd1fafa9740 10 client.25282 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:07.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.596+0000 7fd1fafa9740 10 client.25282 resolve_mds: validated gid 24455 aka daemon mds.b 2024-09-11T22:54:07.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.596+0000 7fd1fafa9740 20 client.25282 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:07.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.596+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/1185000295 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5578a37afb10 0x5578a37b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.596+0000 7fd1fafa9740 4 client.25282 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:54:07.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.596+0000 7fd1fafa9740 1 -- 172.21.15.152:0/1185000295 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578a3707b90 con 0x5578a37afb10 2024-09-11T22:54:07.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.597+0000 7fd1f259c640 1 --2- 172.21.15.152:0/1185000295 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5578a37afb10 0x5578a37b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.597+0000 7fd1d9ffb640 10 client.25282 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:07.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.597+0000 7fd1f259c640 1 --2- 172.21.15.152:0/1185000295 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5578a37afb10 0x5578a37b1f00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5236ce63cf762fd6 server_cookie=31bb909d5b391c57 in_seq=0 out_seq=0 2024-09-11T22:54:07.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.598+0000 7fd1d9ffb640 1 -- 172.21.15.152:0/1185000295 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578a3707b90 con 0x5578a37afb10 2024-09-11T22:54:07.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.598+0000 7fd1d9ffb640 10 client.25282 handle_command_reply: tid=0 2024-09-11T22:54:07.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.598+0000 7fd1fafa9740 1 client.25282 shutdown 2024-09-11T22:54:07.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.598+0000 7fd1fafa9740 1 -- 172.21.15.152:0/1185000295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 msgr2=0x5578a379d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:07.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.598+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/1185000295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 0x5578a379d5f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd1e801c040 tx=0x7fd1e8012680 comp rx=0 tx=0).stop 2024-09-11T22:54:07.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.598+0000 7fd1fafa9740 1 -- 172.21.15.152:0/1185000295 shutdown_connections 2024-09-11T22:54:07.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.598+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/1185000295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a37a0240 0x5578a379db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.598+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/1185000295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 0x5578a379d5f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.598+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/1185000295 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5578a37afb10 0x5578a37b1f00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.598+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/1185000295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a3795060 0x5578a379d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.598+0000 7fd1fafa9740 1 -- 172.21.15.152:0/1185000295 >> 172.21.15.152:0/1185000295 conn(0x5578a37915f0 msgr2=0x5578a3797320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:07.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.598+0000 7fd1d9ffb640 0 client.25282 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:07.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.598+0000 7fd1fafa9740 1 -- 172.21.15.152:0/1185000295 shutdown_connections 2024-09-11T22:54:07.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.598+0000 7fd1fafa9740 1 -- 172.21.15.152:0/1185000295 wait complete. 2024-09-11T22:54:07.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.598+0000 7fd1fafa9740 20 client.25282 trim_cache size 0 max 16384 2024-09-11T22:54:07.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.631+0000 7fd1fafa9740 1 Processor -- start 2024-09-11T22:54:07.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.631+0000 7fd1fafa9740 1 -- start start 2024-09-11T22:54:07.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.632+0000 7fd1fafa9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795060 0x5578a379f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.632+0000 7fd1fafa9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 0x5578a37b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.632+0000 7fd1f259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795060 0x5578a379f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.632+0000 7fd1f259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795060 0x5578a379f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48140/0 (socket says 172.21.15.152:48140) 2024-09-11T22:54:07.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.632+0000 7fd1fafa9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a37a0240 0x5578a37b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.632+0000 7fd1f1d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 0x5578a37b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.632+0000 7fd1f259c640 1 -- 172.21.15.152:0/2514816071 learned_addr learned my addr 172.21.15.152:0/2514816071 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:07.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.632+0000 7fd1fafa9740 1 -- 172.21.15.152:0/2514816071 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41b3d20 con 0x5578a3795060 2024-09-11T22:54:07.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.632+0000 7fd1fafa9740 1 -- 172.21.15.152:0/2514816071 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41b5860 con 0x5578a37a0240 2024-09-11T22:54:07.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.632+0000 7fd1fafa9740 1 -- 172.21.15.152:0/2514816071 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41b43f0 con 0x5578a3795980 2024-09-11T22:54:07.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.632+0000 7fd1f2d9d640 1 --2- 172.21.15.152:0/2514816071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a37a0240 0x5578a37b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.632+0000 7fd1f1d9b640 1 -- 172.21.15.152:0/2514816071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a37a0240 msgr2=0x5578a37b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:07.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.632+0000 7fd1f1d9b640 1 --2- 172.21.15.152:0/2514816071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a37a0240 0x5578a37b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.632+0000 7fd1f1d9b640 1 -- 172.21.15.152:0/2514816071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795060 msgr2=0x5578a379f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:07.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.632+0000 7fd1f1d9b640 1 --2- 172.21.15.152:0/2514816071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795060 0x5578a379f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.632+0000 7fd1f1d9b640 1 -- 172.21.15.152:0/2514816071 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a379c060 con 0x5578a3795980 2024-09-11T22:54:07.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.633+0000 7fd1f1d9b640 1 --2- 172.21.15.152:0/2514816071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 0x5578a37b74d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8000c00 tx=0x7fd1e8023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f3b564a044f8323 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:07.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.633+0000 7fd1dbfff640 1 -- 172.21.15.152:0/2514816071 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8032070 con 0x5578a3795980 2024-09-11T22:54:07.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.633+0000 7fd1dbfff640 1 -- 172.21.15.152:0/2514816071 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e8012930 con 0x5578a3795980 2024-09-11T22:54:07.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.633+0000 7fd1dbfff640 1 -- 172.21.15.152:0/2514816071 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e800fda0 con 0x5578a3795980 2024-09-11T22:54:07.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.634+0000 7fd1fafa9740 1 -- 172.21.15.152:0/2514816071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 msgr2=0x5578a37b74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:07.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.634+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/2514816071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 0x5578a37b74d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8000c00 tx=0x7fd1e8023000 comp rx=0 tx=0).stop 2024-09-11T22:54:07.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.634+0000 7fd1fafa9740 1 -- 172.21.15.152:0/2514816071 shutdown_connections 2024-09-11T22:54:07.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.634+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/2514816071 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a37a0240 0x5578a37b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.634+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/2514816071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 0x5578a37b74d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.634+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/2514816071 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795060 0x5578a379f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.634+0000 7fd1fafa9740 1 -- 172.21.15.152:0/2514816071 >> 172.21.15.152:0/2514816071 conn(0x5578a37915f0 msgr2=0x5578a379e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:07.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.634+0000 7fd1fafa9740 1 -- 172.21.15.152:0/2514816071 shutdown_connections 2024-09-11T22:54:07.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.635+0000 7fd1fafa9740 1 -- 172.21.15.152:0/2514816071 wait complete. 2024-09-11T22:54:07.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.635+0000 7fd1fafa9740 1 -- start start 2024-09-11T22:54:07.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.635+0000 7fd1fafa9740 1 Processor -- start 2024-09-11T22:54:07.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.635+0000 7fd1fafa9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795060 0x5578a37a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.635+0000 7fd1fafa9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 0x5578a37a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.635+0000 7fd1fafa9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a37a0240 0x5578a38194a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.635+0000 7fd1fafa9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41b5190 con 0x5578a3795060 2024-09-11T22:54:07.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.635+0000 7fd1fafa9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41b4ac0 con 0x5578a37a0240 2024-09-11T22:54:07.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.635+0000 7fd1fafa9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41b1ce0 con 0x5578a3795980 2024-09-11T22:54:07.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.635+0000 7fd1fafa9740 10 client.0 _ll_register_callbacks cb 0x7fd1e80099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:07.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.635+0000 7fd1f259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795060 0x5578a37a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.635+0000 7fd1f259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795060 0x5578a37a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48144/0 (socket says 172.21.15.152:48144) 2024-09-11T22:54:07.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.635+0000 7fd1f259c640 1 -- 172.21.15.152:0/167486364 learned_addr learned my addr 172.21.15.152:0/167486364 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:07.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.635+0000 7fd1f2d9d640 1 --2- 172.21.15.152:0/167486364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a37a0240 0x5578a38194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.636+0000 7fd1f1d9b640 1 --2- 172.21.15.152:0/167486364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 0x5578a37a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.636+0000 7fd1f2d9d640 1 -- 172.21.15.152:0/167486364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 msgr2=0x5578a37a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:07.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.636+0000 7fd1f2d9d640 1 --2- 172.21.15.152:0/167486364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 0x5578a37a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.636+0000 7fd1f2d9d640 1 -- 172.21.15.152:0/167486364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795060 msgr2=0x5578a37a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:07.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.636+0000 7fd1f2d9d640 1 --2- 172.21.15.152:0/167486364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795060 0x5578a37a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.636+0000 7fd1f2d9d640 1 -- 172.21.15.152:0/167486364 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a3819ae0 con 0x5578a37a0240 2024-09-11T22:54:07.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.636+0000 7fd1f259c640 1 --2- 172.21.15.152:0/167486364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795060 0x5578a37a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:54:07.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.636+0000 7fd1f2d9d640 1 --2- 172.21.15.152:0/167486364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a37a0240 0x5578a38194a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00fb30 tx=0x7fd1ec004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76d654e507677b56 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:07.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.636+0000 7fd1d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-11T22:54:07.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.636+0000 7fd1d9ffb640 1 -- 172.21.15.152:0/167486364 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec02e060 con 0x5578a37a0240 2024-09-11T22:54:07.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.636+0000 7fd1fafa9740 1 -- 172.21.15.152:0/167486364 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1e8012930 con 0x5578a37a0240 2024-09-11T22:54:07.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.636+0000 7fd1d9ffb640 1 -- 172.21.15.152:0/167486364 <== mon.2 v2:172.21.15.191:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fd1ec0107c0 con 0x5578a37a0240 2024-09-11T22:54:07.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.637+0000 7fd1d9ffb640 1 -- 172.21.15.152:0/167486364 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1ec0109c0 con 0x5578a37a0240 2024-09-11T22:54:07.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.637+0000 7fd1fafa9740 10 client.25229 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:07.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.637+0000 7fd1fafa9740 1 -- 172.21.15.152:0/167486364 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578a381ae90 con 0x5578a37a0240 2024-09-11T22:54:07.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.637+0000 7fd1d9ffb640 1 -- 172.21.15.152:0/167486364 <== mon.2 v2:172.21.15.191:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1e8012930 con 0x5578a37a0240 2024-09-11T22:54:07.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.637+0000 7fd1d9ffb640 1 -- 172.21.15.152:0/167486364 <== mon.2 v2:172.21.15.191:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd1ec01ecb0 con 0x5578a37a0240 2024-09-11T22:54:07.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.637+0000 7fd1fafa9740 10 client.25229 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:07.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.637+0000 7fd1fafa9740 10 client.25229 resolve_mds: validated gid 24455 aka daemon mds.b 2024-09-11T22:54:07.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.637+0000 7fd1fafa9740 20 client.25229 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:07.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.637+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/167486364 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5578a381b560 0x5578a381d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:07.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.637+0000 7fd1fafa9740 4 client.25229 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7c8da22d", "release": true, "await": true}] 2024-09-11T22:54:07.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.637+0000 7fd1fafa9740 1 -- 172.21.15.152:0/167486364 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7c8da22d", "release": true, "await": true}) -- 0x5578a37020a0 con 0x5578a381b560 2024-09-11T22:54:07.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.637+0000 7fd1f259c640 1 --2- 172.21.15.152:0/167486364 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5578a381b560 0x5578a381d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:07.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.638+0000 7fd1f259c640 1 --2- 172.21.15.152:0/167486364 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5578a381b560 0x5578a381d950 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bae7c3c0202ce40e server_cookie=a4099e1dd6b3827a in_seq=0 out_seq=0 2024-09-11T22:54:07.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.638+0000 7fd1d9ffb640 10 client.25229 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:07.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.639+0000 7fd1d9ffb640 1 -- 172.21.15.152:0/167486364 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5578a37020a0 con 0x5578a381b560 2024-09-11T22:54:07.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.639+0000 7fd1d9ffb640 10 client.25229 handle_command_reply: tid=0 2024-09-11T22:54:07.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.639+0000 7fd1fafa9740 1 client.25229 shutdown 2024-09-11T22:54:07.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.639+0000 7fd1fafa9740 1 -- 172.21.15.152:0/167486364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a37a0240 msgr2=0x5578a38194a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:07.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.639+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/167486364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a37a0240 0x5578a38194a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00fb30 tx=0x7fd1ec004a40 comp rx=0 tx=0).stop 2024-09-11T22:54:07.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.639+0000 7fd1fafa9740 1 -- 172.21.15.152:0/167486364 shutdown_connections 2024-09-11T22:54:07.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.639+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/167486364 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5578a381b560 0x5578a381d950 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.640+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/167486364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578a37a0240 0x5578a38194a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.640+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/167486364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578a3795980 0x5578a37a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.640+0000 7fd1fafa9740 1 --2- 172.21.15.152:0/167486364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578a3795060 0x5578a37a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.640+0000 7fd1fafa9740 1 -- 172.21.15.152:0/167486364 >> 172.21.15.152:0/167486364 conn(0x5578a37915f0 msgr2=0x5578a379f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:07.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.640+0000 7fd1d9ffb640 0 client.25229 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:07.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.640+0000 7fd1fafa9740 1 -- 172.21.15.152:0/167486364 shutdown_connections 2024-09-11T22:54:07.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.640+0000 7fd1fafa9740 1 -- 172.21.15.152:0/167486364 wait complete. 2024-09-11T22:54:07.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.640+0000 7fd1fafa9740 20 client.25229 trim_cache size 0 max 16384 2024-09-11T22:54:07.646 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:54:07.646 INFO:teuthology.orchestra.run.smithi152.stdout: "epoch": 28, 2024-09-11T22:54:07.646 INFO:teuthology.orchestra.run.smithi152.stdout: "leader": 24455, 2024-09-11T22:54:07.646 INFO:teuthology.orchestra.run.smithi152.stdout: "set_version": 4, 2024-09-11T22:54:07.646 INFO:teuthology.orchestra.run.smithi152.stdout: "sets": { 2024-09-11T22:54:07.646 INFO:teuthology.orchestra.run.smithi152.stdout: "7c8da22d": { 2024-09-11T22:54:07.646 INFO:teuthology.orchestra.run.smithi152.stdout: "version": 4, 2024-09-11T22:54:07.646 INFO:teuthology.orchestra.run.smithi152.stdout: "age_ref": 0.0, 2024-09-11T22:54:07.646 INFO:teuthology.orchestra.run.smithi152.stdout: "state": { 2024-09-11T22:54:07.647 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RELEASED", 2024-09-11T22:54:07.647 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.0 2024-09-11T22:54:07.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T22:54:07.647 INFO:teuthology.orchestra.run.smithi152.stdout: "timeout": 90.0, 2024-09-11T22:54:07.647 INFO:teuthology.orchestra.run.smithi152.stdout: "expiration": 144.5, 2024-09-11T22:54:07.647 INFO:teuthology.orchestra.run.smithi152.stdout: "members": { 2024-09-11T22:54:07.647 INFO:teuthology.orchestra.run.smithi152.stdout: "file:/": { 2024-09-11T22:54:07.647 INFO:teuthology.orchestra.run.smithi152.stdout: "excluded": false, 2024-09-11T22:54:07.647 INFO:teuthology.orchestra.run.smithi152.stdout: "state": { 2024-09-11T22:54:07.647 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RELEASED", 2024-09-11T22:54:07.647 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.0 2024-09-11T22:54:07.647 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:54:07.648 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:54:07.648 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:54:07.648 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:54:07.648 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:54:07.648 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:54:07.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.641+0000 7fd1f8bd0640 1 -- 172.21.15.152:0/906485876 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd1dc0a2e10 msgr2=0x7fd1dc0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:07.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.641+0000 7fd1f8bd0640 1 --2- 172.21.15.152:0/906485876 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd1dc0a2e10 0x7fd1dc0a52d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec011700 tx=0x7fd1ec009500 comp rx=0 tx=0).stop 2024-09-11T22:54:07.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.641+0000 7fd1f8bd0640 1 -- 172.21.15.152:0/906485876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1f4100e20 msgr2=0x7fd1f419edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:07.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.641+0000 7fd1f8bd0640 1 --2- 172.21.15.152:0/906485876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1f4100e20 0x7fd1f419edd0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4006010 tx=0x7fd1e4003c10 comp rx=0 tx=0).stop 2024-09-11T22:54:07.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.641+0000 7fd1f8bd0640 1 -- 172.21.15.152:0/906485876 shutdown_connections 2024-09-11T22:54:07.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.641+0000 7fd1f8bd0640 1 --2- 172.21.15.152:0/906485876 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd1dc0a2e10 0x7fd1dc0a52d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.641+0000 7fd1f8bd0640 1 --2- 172.21.15.152:0/906485876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1f4102ef0 0x7fd1f41a3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.641+0000 7fd1f8bd0640 1 --2- 172.21.15.152:0/906485876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f4102570 0x7fd1f419f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.641+0000 7fd1f8bd0640 1 --2- 172.21.15.152:0/906485876 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1f4100e20 0x7fd1f419edd0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:07.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.641+0000 7fd1f8bd0640 1 -- 172.21.15.152:0/906485876 >> 172.21.15.152:0/906485876 conn(0x7fd1f40fc7d0 msgr2=0x7fd1f406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:07.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.641+0000 7fd1f8bd0640 1 -- 172.21.15.152:0/906485876 shutdown_connections 2024-09-11T22:54:07.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:07.641+0000 7fd1f8bd0640 1 -- 172.21.15.152:0/906485876 wait complete. 2024-09-11T22:54:08.490 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7c8da22d', total seconds elapsed: 28.5 2024-09-11T22:54:08.839 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:08 smithi191 ceph-mon[35211]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-11T22:54:08.848 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:08 smithi204 ceph-mon[35454]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-11T22:54:08.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:08 smithi152 ceph-mon[31181]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-11T22:54:08.892 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.891+0000 7f7d2b59e640 1 -- 172.21.15.152:0/3451114332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d2c103400 msgr2=0x7f7d2c10b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.891+0000 7f7d2b59e640 1 --2- 172.21.15.152:0/3451114332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d2c103400 0x7f7d2c10b8e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7d20009900 tx=0x7f7d2002a480 comp rx=0 tx=0).stop 2024-09-11T22:54:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.892+0000 7f7d2b59e640 1 -- 172.21.15.152:0/3451114332 shutdown_connections 2024-09-11T22:54:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.892+0000 7f7d2b59e640 1 --2- 172.21.15.152:0/3451114332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d2c10bf00 0x7f7d2c10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.892+0000 7f7d2b59e640 1 --2- 172.21.15.152:0/3451114332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d2c103400 0x7f7d2c10b8e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.892+0000 7f7d2b59e640 1 --2- 172.21.15.152:0/3451114332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c102ae0 0x7f7d2c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.892+0000 7f7d2b59e640 1 -- 172.21.15.152:0/3451114332 >> 172.21.15.152:0/3451114332 conn(0x7f7d2c0777c0 msgr2=0x7f7d2c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.892+0000 7f7d2b59e640 1 -- 172.21.15.152:0/3451114332 shutdown_connections 2024-09-11T22:54:08.893 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.892+0000 7f7d2b59e640 1 -- 172.21.15.152:0/3451114332 wait complete. 2024-09-11T22:54:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.892+0000 7f7d2b59e640 1 Processor -- start 2024-09-11T22:54:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.892+0000 7f7d2b59e640 1 -- start start 2024-09-11T22:54:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d2b59e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d2c102ae0 0x7f7d2c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d2b59e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c103400 0x7f7d2c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d2b59e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d2c10bf00 0x7f7d2c19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d2b59e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c111cd0 con 0x7f7d2c102ae0 2024-09-11T22:54:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d2b59e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c111b50 con 0x7f7d2c10bf00 2024-09-11T22:54:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d2b59e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c111e50 con 0x7f7d2c103400 2024-09-11T22:54:08.896 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d29d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c103400 0x7f7d2c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d29d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c103400 0x7f7d2c1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:35418/0 (socket says 172.21.15.152:35418) 2024-09-11T22:54:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d29d9b640 1 -- 172.21.15.152:0/3068060580 learned_addr learned my addr 172.21.15.152:0/3068060580 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d2ad9d640 1 --2- 172.21.15.152:0/3068060580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d2c10bf00 0x7f7d2c19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d2a59c640 1 --2- 172.21.15.152:0/3068060580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d2c102ae0 0x7f7d2c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d2ad9d640 1 -- 172.21.15.152:0/3068060580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c103400 msgr2=0x7f7d2c1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d2ad9d640 1 --2- 172.21.15.152:0/3068060580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c103400 0x7f7d2c1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d2ad9d640 1 -- 172.21.15.152:0/3068060580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d2c102ae0 msgr2=0x7f7d2c198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d2ad9d640 1 --2- 172.21.15.152:0/3068060580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d2c102ae0 0x7f7d2c198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d2ad9d640 1 -- 172.21.15.152:0/3068060580 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d2c19d250 con 0x7f7d2c10bf00 2024-09-11T22:54:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d29d9b640 1 --2- 172.21.15.152:0/3068060580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c103400 0x7f7d2c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:54:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.893+0000 7f7d2a59c640 1 --2- 172.21.15.152:0/3068060580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d2c102ae0 0x7f7d2c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:54:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.894+0000 7f7d2ad9d640 1 --2- 172.21.15.152:0/3068060580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d2c10bf00 0x7f7d2c19cb20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009fd0 tx=0x7f7d2400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.894+0000 7f7d1b7fe640 1 -- 172.21.15.152:0/3068060580 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2400ce20 con 0x7f7d2c10bf00 2024-09-11T22:54:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.894+0000 7f7d2b59e640 1 -- 172.21.15.152:0/3068060580 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d2c19d540 con 0x7f7d2c10bf00 2024-09-11T22:54:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.894+0000 7f7d1b7fe640 1 -- 172.21.15.152:0/3068060580 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d24005020 con 0x7f7d2c10bf00 2024-09-11T22:54:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.894+0000 7f7d1b7fe640 1 -- 172.21.15.152:0/3068060580 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24010720 con 0x7f7d2c10bf00 2024-09-11T22:54:08.898 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.894+0000 7f7d2b59e640 1 -- 172.21.15.152:0/3068060580 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d2c100660 con 0x7f7d2c10bf00 2024-09-11T22:54:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.897+0000 7f7d1b7fe640 1 -- 172.21.15.152:0/3068060580 <== mon.2 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 24) ==== 138067+0+0 (secure 0 0 0) 0x7f7d24005540 con 0x7f7d2c10bf00 2024-09-11T22:54:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.897+0000 7f7d33a71740 1 Processor -- start 2024-09-11T22:54:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.898+0000 7f7d1b7fe640 1 --2- 172.21.15.152:0/3068060580 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7d140a2e90 0x7f7d140a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.898+0000 7f7d1b7fe640 1 -- 172.21.15.152:0/3068060580 <== mon.2 v2:172.21.15.191:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7d240cfcc0 con 0x7f7d2c10bf00 2024-09-11T22:54:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.898+0000 7f7d29d9b640 1 --2- 172.21.15.152:0/3068060580 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7d140a2e90 0x7f7d140a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.898+0000 7f7d33a71740 1 -- start start 2024-09-11T22:54:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.898+0000 7f7d33a71740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 0x55ee0f5490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.898+0000 7f7d33a71740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f553940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.898+0000 7f7d33a71740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f556270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.898+0000 7f7d33a71740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f4ad2f0 con 0x55ee0f553e80 2024-09-11T22:54:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.898+0000 7f7d33a71740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f455270 con 0x55ee0f5495e0 2024-09-11T22:54:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.898+0000 7f7d33a71740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f4a92b0 con 0x55ee0f548cc0 2024-09-11T22:54:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.898+0000 7f7d2ad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f556270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.898+0000 7f7d2ad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f556270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40678/0 (socket says 172.21.15.152:40678) 2024-09-11T22:54:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.898+0000 7f7d2ad9d640 1 -- 172.21.15.152:0/2774871312 learned_addr learned my addr 172.21.15.152:0/2774871312 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.899+0000 7f7d29d9b640 1 --2- 172.21.15.152:0/3068060580 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7d140a2e90 0x7f7d140a5350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7d20031550 tx=0x7f7d20005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.899+0000 7f7d2a59c640 1 --2- 172.21.15.152:0/2774871312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 0x55ee0f5490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.899+0000 7f7d29d9b640 1 --2- 172.21.15.152:0/2774871312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f553940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.899+0000 7f7d2ad9d640 1 -- 172.21.15.152:0/2774871312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 msgr2=0x55ee0f5490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.899+0000 7f7d2ad9d640 1 --2- 172.21.15.152:0/2774871312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 0x55ee0f5490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.899+0000 7f7d2ad9d640 1 -- 172.21.15.152:0/2774871312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 msgr2=0x55ee0f553940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.899+0000 7f7d2ad9d640 1 --2- 172.21.15.152:0/2774871312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f553940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.899+0000 7f7d2ad9d640 1 -- 172.21.15.152:0/2774871312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0f454340 con 0x55ee0f553e80 2024-09-11T22:54:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.899+0000 7f7d2ad9d640 1 --2- 172.21.15.152:0/2774871312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f556270 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7d24098970 tx=0x7f7d2409bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f1b6f59c7eddad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.902+0000 7f7d18ff9640 1 -- 172.21.15.152:0/2774871312 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d240051d0 con 0x55ee0f553e80 2024-09-11T22:54:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.902+0000 7f7d18ff9640 1 -- 172.21.15.152:0/2774871312 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d24098610 con 0x55ee0f553e80 2024-09-11T22:54:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.902+0000 7f7d18ff9640 1 -- 172.21.15.152:0/2774871312 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2409c460 con 0x55ee0f553e80 2024-09-11T22:54:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.903+0000 7f7d33a71740 1 -- 172.21.15.152:0/2774871312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 msgr2=0x55ee0f556270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.903+0000 7f7d33a71740 1 --2- 172.21.15.152:0/2774871312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f556270 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7d24098970 tx=0x7f7d2409bfa0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.903+0000 7f7d33a71740 1 -- 172.21.15.152:0/2774871312 shutdown_connections 2024-09-11T22:54:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.903+0000 7f7d33a71740 1 --2- 172.21.15.152:0/2774871312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f556270 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.903+0000 7f7d33a71740 1 --2- 172.21.15.152:0/2774871312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f553940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.903+0000 7f7d33a71740 1 --2- 172.21.15.152:0/2774871312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 0x55ee0f5490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.903+0000 7f7d33a71740 1 -- 172.21.15.152:0/2774871312 >> 172.21.15.152:0/2774871312 conn(0x55ee0f545230 msgr2=0x55ee0f545610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.903+0000 7f7d33a71740 1 -- 172.21.15.152:0/2774871312 shutdown_connections 2024-09-11T22:54:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.903+0000 7f7d33a71740 1 -- 172.21.15.152:0/2774871312 wait complete. 2024-09-11T22:54:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.903+0000 7f7d33a71740 1 -- start start 2024-09-11T22:54:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.903+0000 7f7d33a71740 1 Processor -- start 2024-09-11T22:54:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.904+0000 7f7d33a71740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 0x55ee0f550e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.904+0000 7f7d33a71740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f551340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.904+0000 7f7d33a71740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f551880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.904+0000 7f7d2ad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f551880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.904+0000 7f7d2ad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f551880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40692/0 (socket says 172.21.15.152:40692) 2024-09-11T22:54:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.904+0000 7f7d29d9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f551340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.904+0000 7f7d2ad9d640 1 -- 172.21.15.152:0/1521861523 learned_addr learned my addr 172.21.15.152:0/1521861523 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.904+0000 7f7d33a71740 1 -- 172.21.15.152:0/1521861523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f556a50 con 0x55ee0f553e80 2024-09-11T22:54:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.904+0000 7f7d33a71740 1 -- 172.21.15.152:0/1521861523 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f436290 con 0x55ee0f5495e0 2024-09-11T22:54:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.904+0000 7f7d2a59c640 1 --2- 172.21.15.152:0/1521861523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 0x55ee0f550e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.904+0000 7f7d33a71740 1 -- 172.21.15.152:0/1521861523 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ee0f556bd0 con 0x55ee0f548cc0 2024-09-11T22:54:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.904+0000 7f7d33a71740 10 client.0 _ll_register_callbacks cb 0x55ee0f467140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.904+0000 7f7d2ad9d640 1 -- 172.21.15.152:0/1521861523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 msgr2=0x55ee0f550e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.904+0000 7f7d2ad9d640 1 --2- 172.21.15.152:0/1521861523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 0x55ee0f550e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.904+0000 7f7d2ad9d640 1 -- 172.21.15.152:0/1521861523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 msgr2=0x55ee0f551340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.904+0000 7f7d2ad9d640 1 --2- 172.21.15.152:0/1521861523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f551340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.905+0000 7f7d2ad9d640 1 -- 172.21.15.152:0/1521861523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0f1c0210 con 0x55ee0f553e80 2024-09-11T22:54:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.905+0000 7f7d29d9b640 1 --2- 172.21.15.152:0/1521861523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f551340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:54:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.905+0000 7f7d2a59c640 1 --2- 172.21.15.152:0/1521861523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 0x55ee0f550e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:54:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.905+0000 7f7d2ad9d640 1 --2- 172.21.15.152:0/1521861523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f551880 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7d2401b000 tx=0x7f7d240a49b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a69473382012c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.905+0000 7f7d11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.905+0000 7f7d11ffb640 1 -- 172.21.15.152:0/1521861523 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d240babe0 con 0x55ee0f553e80 2024-09-11T22:54:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.905+0000 7f7d11ffb640 1 -- 172.21.15.152:0/1521861523 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7d240cd790 con 0x55ee0f553e80 2024-09-11T22:54:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.905+0000 7f7d33a71740 1 -- 172.21.15.152:0/1521861523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d24098610 con 0x55ee0f553e80 2024-09-11T22:54:08.910 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.905+0000 7f7d11ffb640 1 -- 172.21.15.152:0/1521861523 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7d2407f050 con 0x55ee0f553e80 2024-09-11T22:54:08.910 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.905+0000 7f7d33a71740 10 client.15849 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:08.910 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.905+0000 7f7d33a71740 1 -- 172.21.15.152:0/1521861523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55ee0f28c620 con 0x55ee0f553e80 2024-09-11T22:54:08.910 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.906+0000 7f7d11ffb640 1 -- 172.21.15.152:0/1521861523 <== mon.0 v2:172.21.15.152:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7d240cdb70 con 0x55ee0f553e80 2024-09-11T22:54:08.910 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.906+0000 7f7d33a71740 10 client.15849 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:08.910 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.906+0000 7f7d33a71740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:54:08.910 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.906+0000 7f7d33a71740 20 client.15849 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:08.910 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.906+0000 7f7d33a71740 1 --2- 172.21.15.152:0/1521861523 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ee0f563b70 0x55ee0f565f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.906+0000 7f7d33a71740 4 client.15849 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:54:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.906+0000 7f7d33a71740 1 -- 172.21.15.152:0/1521861523 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7d24098610 con 0x55ee0f563b70 2024-09-11T22:54:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.906+0000 7f7d2a59c640 1 --2- 172.21.15.152:0/1521861523 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ee0f563b70 0x55ee0f565f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.906+0000 7f7d2a59c640 1 --2- 172.21.15.152:0/1521861523 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ee0f563b70 0x55ee0f565f60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=665a17f2304d732f server_cookie=6eba925cfad59fb5 in_seq=0 out_seq=0 2024-09-11T22:54:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.906+0000 7f7d11ffb640 10 client.15849 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.907+0000 7f7d11ffb640 1 -- 172.21.15.152:0/1521861523 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f7d24098610 con 0x55ee0f563b70 2024-09-11T22:54:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.907+0000 7f7d11ffb640 10 client.15849 handle_command_reply: tid=0 2024-09-11T22:54:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.907+0000 7f7d33a71740 1 client.15849 shutdown 2024-09-11T22:54:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.907+0000 7f7d33a71740 1 -- 172.21.15.152:0/1521861523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 msgr2=0x55ee0f551880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.907+0000 7f7d33a71740 1 --2- 172.21.15.152:0/1521861523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f551880 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7d2401b000 tx=0x7f7d240a49b0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.908+0000 7f7d33a71740 1 -- 172.21.15.152:0/1521861523 shutdown_connections 2024-09-11T22:54:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.908+0000 7f7d33a71740 1 --2- 172.21.15.152:0/1521861523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f551880 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.908+0000 7f7d33a71740 1 --2- 172.21.15.152:0/1521861523 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ee0f563b70 0x55ee0f565f60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.908+0000 7f7d33a71740 1 --2- 172.21.15.152:0/1521861523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f551340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.908+0000 7f7d11ffb640 0 client.15849 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.908+0000 7f7d33a71740 1 --2- 172.21.15.152:0/1521861523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 0x55ee0f550e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.908+0000 7f7d33a71740 1 -- 172.21.15.152:0/1521861523 >> 172.21.15.152:0/1521861523 conn(0x55ee0f545230 msgr2=0x55ee0f54af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.908+0000 7f7d33a71740 1 -- 172.21.15.152:0/1521861523 shutdown_connections 2024-09-11T22:54:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.908+0000 7f7d33a71740 1 -- 172.21.15.152:0/1521861523 wait complete. 2024-09-11T22:54:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.908+0000 7f7d33a71740 20 client.15849 trim_cache size 0 max 16384 2024-09-11T22:54:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.935+0000 7f7d33a71740 1 Processor -- start 2024-09-11T22:54:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.935+0000 7f7d33a71740 1 -- start start 2024-09-11T22:54:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d33a71740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 0x55ee0f54b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d33a71740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f55f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d33a71740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f551330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d33a71740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1af730 con 0x55ee0f553e80 2024-09-11T22:54:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d33a71740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1b1270 con 0x55ee0f5495e0 2024-09-11T22:54:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d33a71740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1afe00 con 0x55ee0f548cc0 2024-09-11T22:54:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d2a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 0x55ee0f54b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d2ad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f551330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d2a59c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 0x55ee0f54b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:35454/0 (socket says 172.21.15.152:35454) 2024-09-11T22:54:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d2a59c640 1 -- 172.21.15.152:0/2439504928 learned_addr learned my addr 172.21.15.152:0/2439504928 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d29d9b640 1 --2- 172.21.15.152:0/2439504928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f55f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d2a59c640 1 -- 172.21.15.152:0/2439504928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 msgr2=0x55ee0f55f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d2a59c640 1 --2- 172.21.15.152:0/2439504928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f55f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d2a59c640 1 -- 172.21.15.152:0/2439504928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 msgr2=0x55ee0f551330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d2a59c640 1 --2- 172.21.15.152:0/2439504928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f551330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d2a59c640 1 -- 172.21.15.152:0/2439504928 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0f549410 con 0x55ee0f548cc0 2024-09-11T22:54:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d29d9b640 1 --2- 172.21.15.152:0/2439504928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f55f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:54:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.936+0000 7f7d2ad9d640 1 --2- 172.21.15.152:0/2439504928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f551330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:54:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.937+0000 7f7d2a59c640 1 --2- 172.21.15.152:0/2439504928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 0x55ee0f54b730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00bd90 tx=0x7f7d1c0039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92ed1c674e158eac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.937+0000 7f7d13fff640 1 -- 172.21.15.152:0/2439504928 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c01d630 con 0x55ee0f548cc0 2024-09-11T22:54:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.937+0000 7f7d13fff640 1 -- 172.21.15.152:0/2439504928 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d1c01dcb0 con 0x55ee0f548cc0 2024-09-11T22:54:08.942 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.937+0000 7f7d13fff640 1 -- 172.21.15.152:0/2439504928 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c010430 con 0x55ee0f548cc0 2024-09-11T22:54:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.938+0000 7f7d33a71740 1 -- 172.21.15.152:0/2439504928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 msgr2=0x55ee0f54b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.938+0000 7f7d33a71740 1 --2- 172.21.15.152:0/2439504928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 0x55ee0f54b730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00bd90 tx=0x7f7d1c0039c0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.938+0000 7f7d33a71740 1 -- 172.21.15.152:0/2439504928 shutdown_connections 2024-09-11T22:54:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.938+0000 7f7d33a71740 1 --2- 172.21.15.152:0/2439504928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f553e80 0x55ee0f551330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.938+0000 7f7d33a71740 1 --2- 172.21.15.152:0/2439504928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f55f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.938+0000 7f7d33a71740 1 --2- 172.21.15.152:0/2439504928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f548cc0 0x55ee0f54b730 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.938+0000 7f7d33a71740 1 -- 172.21.15.152:0/2439504928 >> 172.21.15.152:0/2439504928 conn(0x55ee0f545230 msgr2=0x55ee0f55ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.938+0000 7f7d33a71740 1 -- 172.21.15.152:0/2439504928 shutdown_connections 2024-09-11T22:54:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.938+0000 7f7d33a71740 1 -- 172.21.15.152:0/2439504928 wait complete. 2024-09-11T22:54:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.939+0000 7f7d33a71740 1 -- start start 2024-09-11T22:54:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.939+0000 7f7d33a71740 1 Processor -- start 2024-09-11T22:54:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.939+0000 7f7d33a71740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f548cc0 0x55ee0f5665d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.939+0000 7f7d33a71740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f566b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.939+0000 7f7d33a71740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f553e80 0x55ee0f5c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.939+0000 7f7d33a71740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1b0ba0 con 0x55ee0f548cc0 2024-09-11T22:54:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.939+0000 7f7d33a71740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1b04d0 con 0x55ee0f5495e0 2024-09-11T22:54:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.939+0000 7f7d33a71740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2c1ad6f0 con 0x55ee0f553e80 2024-09-11T22:54:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.939+0000 7f7d2a59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f548cc0 0x55ee0f5665d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.939+0000 7f7d33a71740 10 client.0 _ll_register_callbacks cb 0x7f7d2409b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.939+0000 7f7d2a59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f548cc0 0x55ee0f5665d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40718/0 (socket says 172.21.15.152:40718) 2024-09-11T22:54:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.939+0000 7f7d2a59c640 1 -- 172.21.15.152:0/3974982287 learned_addr learned my addr 172.21.15.152:0/3974982287 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.939+0000 7f7d29d9b640 1 --2- 172.21.15.152:0/3974982287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f566b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.940+0000 7f7d2a59c640 1 -- 172.21.15.152:0/3974982287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f553e80 msgr2=0x55ee0f5c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.940+0000 7f7d2a59c640 1 --2- 172.21.15.152:0/3974982287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f553e80 0x55ee0f5c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.940+0000 7f7d2a59c640 1 -- 172.21.15.152:0/3974982287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 msgr2=0x55ee0f566b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.940+0000 7f7d2a59c640 1 --2- 172.21.15.152:0/3974982287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f566b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.940+0000 7f7d2a59c640 1 -- 172.21.15.152:0/3974982287 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0f5c6b50 con 0x55ee0f548cc0 2024-09-11T22:54:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.940+0000 7f7d2a59c640 1 --2- 172.21.15.152:0/3974982287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f548cc0 0x55ee0f5665d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c004620 tx=0x7f7d1c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321f0e0ce12b6d78 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.940+0000 7f7d11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.940+0000 7f7d11ffb640 1 -- 172.21.15.152:0/3974982287 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c033060 con 0x55ee0f548cc0 2024-09-11T22:54:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.940+0000 7f7d11ffb640 1 -- 172.21.15.152:0/3974982287 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7d1c011890 con 0x55ee0f548cc0 2024-09-11T22:54:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.940+0000 7f7d33a71740 1 -- 172.21.15.152:0/3974982287 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d1c01dcb0 con 0x55ee0f548cc0 2024-09-11T22:54:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.940+0000 7f7d11ffb640 1 -- 172.21.15.152:0/3974982287 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7d1c01d430 con 0x55ee0f548cc0 2024-09-11T22:54:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.940+0000 7f7d33a71740 10 client.15855 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.940+0000 7f7d33a71740 1 -- 172.21.15.152:0/3974982287 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ee0f5cdee0 con 0x55ee0f548cc0 2024-09-11T22:54:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.941+0000 7f7d11ffb640 1 -- 172.21.15.152:0/3974982287 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d1c01dcb0 con 0x55ee0f548cc0 2024-09-11T22:54:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.941+0000 7f7d11ffb640 1 -- 172.21.15.152:0/3974982287 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7d1c02ea50 con 0x55ee0f548cc0 2024-09-11T22:54:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.941+0000 7f7d33a71740 10 client.15855 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.941+0000 7f7d33a71740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:54:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.941+0000 7f7d33a71740 20 client.15855 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.941+0000 7f7d33a71740 1 --2- 172.21.15.152:0/3974982287 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ee0f563b70 0x55ee0f5d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.941+0000 7f7d33a71740 4 client.15855 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:54:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.941+0000 7f7d33a71740 1 -- 172.21.15.152:0/3974982287 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee0f4baa20 con 0x55ee0f563b70 2024-09-11T22:54:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.941+0000 7f7d29d9b640 1 --2- 172.21.15.152:0/3974982287 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ee0f563b70 0x55ee0f5d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.942+0000 7f7d29d9b640 1 --2- 172.21.15.152:0/3974982287 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ee0f563b70 0x55ee0f5d0510 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9659a113539a1a server_cookie=bc6d8efa07d810bc in_seq=0 out_seq=0 2024-09-11T22:54:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.942+0000 7f7d11ffb640 10 client.15855 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.948+0000 7f7d11ffb640 1 -- 172.21.15.152:0/3974982287 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee0f4baa20 con 0x55ee0f563b70 2024-09-11T22:54:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.948+0000 7f7d11ffb640 10 client.15855 handle_command_reply: tid=0 2024-09-11T22:54:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.948+0000 7f7d33a71740 1 client.15855 shutdown 2024-09-11T22:54:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.948+0000 7f7d33a71740 1 -- 172.21.15.152:0/3974982287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f548cc0 msgr2=0x55ee0f5665d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.948+0000 7f7d33a71740 1 --2- 172.21.15.152:0/3974982287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f548cc0 0x55ee0f5665d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c004620 tx=0x7f7d1c005800 comp rx=0 tx=0).stop 2024-09-11T22:54:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.948+0000 7f7d33a71740 1 -- 172.21.15.152:0/3974982287 shutdown_connections 2024-09-11T22:54:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.948+0000 7f7d33a71740 1 --2- 172.21.15.152:0/3974982287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ee0f553e80 0x55ee0f5c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.948+0000 7f7d33a71740 1 --2- 172.21.15.152:0/3974982287 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ee0f563b70 0x55ee0f5d0510 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.948+0000 7f7d33a71740 1 --2- 172.21.15.152:0/3974982287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee0f5495e0 0x55ee0f566b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.948+0000 7f7d33a71740 1 --2- 172.21.15.152:0/3974982287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ee0f548cc0 0x55ee0f5665d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.948+0000 7f7d11ffb640 0 client.15855 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.948+0000 7f7d33a71740 1 -- 172.21.15.152:0/3974982287 >> 172.21.15.152:0/3974982287 conn(0x55ee0f545230 msgr2=0x55ee0f54f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.948+0000 7f7d33a71740 1 -- 172.21.15.152:0/3974982287 shutdown_connections 2024-09-11T22:54:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.948+0000 7f7d33a71740 1 -- 172.21.15.152:0/3974982287 wait complete. 2024-09-11T22:54:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.948+0000 7f7d33a71740 20 client.15855 trim_cache size 0 max 16384 2024-09-11T22:54:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:{ 2024-09-11T22:54:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "message": "", 2024-09-11T22:54:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "return_code": 0 2024-09-11T22:54:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:} 2024-09-11T22:54:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.949+0000 7f7d2b59e640 1 -- 172.21.15.152:0/3068060580 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7d140a2e90 msgr2=0x7f7d140a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.949+0000 7f7d2b59e640 1 --2- 172.21.15.152:0/3068060580 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7d140a2e90 0x7f7d140a5350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7d20031550 tx=0x7f7d20005a20 comp rx=0 tx=0).stop 2024-09-11T22:54:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.949+0000 7f7d2b59e640 1 -- 172.21.15.152:0/3068060580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d2c10bf00 msgr2=0x7f7d2c19cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.949+0000 7f7d2b59e640 1 --2- 172.21.15.152:0/3068060580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d2c10bf00 0x7f7d2c19cb20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7d24009fd0 tx=0x7f7d2400ef10 comp rx=0 tx=0).stop 2024-09-11T22:54:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.949+0000 7f7d2b59e640 1 -- 172.21.15.152:0/3068060580 shutdown_connections 2024-09-11T22:54:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.949+0000 7f7d2b59e640 1 --2- 172.21.15.152:0/3068060580 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7d140a2e90 0x7f7d140a5350 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.949+0000 7f7d2b59e640 1 --2- 172.21.15.152:0/3068060580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d2c10bf00 0x7f7d2c19cb20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.949+0000 7f7d2b59e640 1 --2- 172.21.15.152:0/3068060580 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d2c103400 0x7f7d2c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.949+0000 7f7d2b59e640 1 --2- 172.21.15.152:0/3068060580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d2c102ae0 0x7f7d2c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.949+0000 7f7d2b59e640 1 -- 172.21.15.152:0/3068060580 >> 172.21.15.152:0/3068060580 conn(0x7f7d2c0777c0 msgr2=0x7f7d2c0fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.949+0000 7f7d2b59e640 1 -- 172.21.15.152:0/3068060580 shutdown_connections 2024-09-11T22:54:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:08.949+0000 7f7d2b59e640 1 -- 172.21.15.152:0/3068060580 wait complete. 2024-09-11T22:54:10.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:10 smithi152 ceph-mon[31181]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-09-11T22:54:11.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:10 smithi191 ceph-mon[35211]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-09-11T22:54:11.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:10 smithi204 ceph-mon[35454]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-09-11T22:54:12.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:12 smithi152 ceph-mon[31181]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T22:54:13.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:12 smithi191 ceph-mon[35211]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T22:54:13.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:12 smithi204 ceph-mon[35454]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T22:54:14.240 INFO:tasks.workunit.client.0.smithi152.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-11T22:54:14.241 INFO:tasks.workunit.client.0.smithi152.stderr: 2024-09-11T22:54:14.241 INFO:tasks.workunit.client.0.smithi152.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-11T22:54:14.241 INFO:tasks.workunit.client.0.smithi152.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-11T22:54:14.241 INFO:tasks.workunit.client.0.smithi152.stderr:state without impacting any branches by switching back to a branch. 2024-09-11T22:54:14.241 INFO:tasks.workunit.client.0.smithi152.stderr: 2024-09-11T22:54:14.241 INFO:tasks.workunit.client.0.smithi152.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-11T22:54:14.241 INFO:tasks.workunit.client.0.smithi152.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-11T22:54:14.241 INFO:tasks.workunit.client.0.smithi152.stderr: 2024-09-11T22:54:14.241 INFO:tasks.workunit.client.0.smithi152.stderr: git switch -c 2024-09-11T22:54:14.241 INFO:tasks.workunit.client.0.smithi152.stderr: 2024-09-11T22:54:14.241 INFO:tasks.workunit.client.0.smithi152.stderr:Or undo this operation with: 2024-09-11T22:54:14.241 INFO:tasks.workunit.client.0.smithi152.stderr: 2024-09-11T22:54:14.242 INFO:tasks.workunit.client.0.smithi152.stderr: git switch - 2024-09-11T22:54:14.242 INFO:tasks.workunit.client.0.smithi152.stderr: 2024-09-11T22:54:14.242 INFO:tasks.workunit.client.0.smithi152.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-11T22:54:14.242 INFO:tasks.workunit.client.0.smithi152.stderr: 2024-09-11T22:54:14.242 INFO:tasks.workunit.client.0.smithi152.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-11T22:54:14.246 DEBUG:teuthology.orchestra.run.smithi152:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-11T22:54:14.309 INFO:tasks.workunit.client.0.smithi152.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-11T22:54:14.311 INFO:tasks.workunit.client.0.smithi152.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T22:54:14.311 INFO:tasks.workunit.client.0.smithi152.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-11T22:54:14.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:14 smithi152 ceph-mon[31181]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T22:54:14.995 INFO:tasks.workunit.client.0.smithi152.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-11T22:54:15.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:14 smithi191 ceph-mon[35211]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T22:54:15.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:14 smithi204 ceph-mon[35454]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T22:54:15.064 INFO:tasks.workunit.client.0.smithi152.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-11T22:54:15.104 INFO:tasks.workunit.client.0.smithi152.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T22:54:15.106 INFO:tasks.workunit.client.0.smithi152.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T22:54:15.106 INFO:tasks.workunit.client.0.smithi152.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-11T22:54:15.145 INFO:tasks.workunit.client.0.smithi152.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T22:54:15.148 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T22:54:15.148 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-11T22:54:15.205 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-09-11T22:54:15.206 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-09-11T22:54:15.207 DEBUG:teuthology.orchestra.run.smithi152:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-09-11T22:54:15.267 INFO:tasks.workunit.client.0.smithi152.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-11T22:54:15.341 INFO:tasks.workunit.client.0.smithi152.stdout: Iozone: Performance Test of File I/O 2024-09-11T22:54:15.342 INFO:tasks.workunit.client.0.smithi152.stdout: Version $Revision: 3.494 $ 2024-09-11T22:54:15.342 INFO:tasks.workunit.client.0.smithi152.stdout: Compiled for 64 bit mode. 2024-09-11T22:54:15.342 INFO:tasks.workunit.client.0.smithi152.stdout: Build: linux-AMD64 2024-09-11T22:54:15.342 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:15.342 INFO:tasks.workunit.client.0.smithi152.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-11T22:54:15.342 INFO:tasks.workunit.client.0.smithi152.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-11T22:54:15.342 INFO:tasks.workunit.client.0.smithi152.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-11T22:54:15.342 INFO:tasks.workunit.client.0.smithi152.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-11T22:54:15.343 INFO:tasks.workunit.client.0.smithi152.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-11T22:54:15.343 INFO:tasks.workunit.client.0.smithi152.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-11T22:54:15.343 INFO:tasks.workunit.client.0.smithi152.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-11T22:54:15.343 INFO:tasks.workunit.client.0.smithi152.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-11T22:54:15.343 INFO:tasks.workunit.client.0.smithi152.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-11T22:54:15.343 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:15.343 INFO:tasks.workunit.client.0.smithi152.stdout: Run began: Wed Sep 11 22:54:15 2024 2024-09-11T22:54:15.343 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:15.343 INFO:tasks.workunit.client.0.smithi152.stdout: Include close in write timing 2024-09-11T22:54:15.343 INFO:tasks.workunit.client.0.smithi152.stdout: Include fsync in write timing 2024-09-11T22:54:15.343 INFO:tasks.workunit.client.0.smithi152.stdout: File size set to 1048576 kB 2024-09-11T22:54:15.343 INFO:tasks.workunit.client.0.smithi152.stdout: Record Size 16 kB 2024-09-11T22:54:15.343 INFO:tasks.workunit.client.0.smithi152.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-09-11T22:54:15.343 INFO:tasks.workunit.client.0.smithi152.stdout: Output is in kBytes/sec 2024-09-11T22:54:15.344 INFO:tasks.workunit.client.0.smithi152.stdout: Time Resolution = 0.000001 seconds. 2024-09-11T22:54:15.344 INFO:tasks.workunit.client.0.smithi152.stdout: Processor cache size set to 1024 kBytes. 2024-09-11T22:54:15.344 INFO:tasks.workunit.client.0.smithi152.stdout: Processor cache line size set to 32 bytes. 2024-09-11T22:54:15.344 INFO:tasks.workunit.client.0.smithi152.stdout: File stride size set to 17 * record size. 2024-09-11T22:54:15.344 INFO:tasks.workunit.client.0.smithi152.stdout: Throughput test with 1 process 2024-09-11T22:54:15.344 INFO:tasks.workunit.client.0.smithi152.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-09-11T22:54:16.820 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:16.820 INFO:tasks.workunit.client.0.smithi152.stdout: Children see throughput for 1 initial writers = 709969.00 kB/sec 2024-09-11T22:54:16.821 INFO:tasks.workunit.client.0.smithi152.stdout: Parent sees throughput for 1 initial writers = 709888.25 kB/sec 2024-09-11T22:54:16.821 INFO:tasks.workunit.client.0.smithi152.stdout: Min throughput per process = 709969.00 kB/sec 2024-09-11T22:54:16.821 INFO:tasks.workunit.client.0.smithi152.stdout: Max throughput per process = 709969.00 kB/sec 2024-09-11T22:54:16.821 INFO:tasks.workunit.client.0.smithi152.stdout: Avg throughput per process = 709969.00 kB/sec 2024-09-11T22:54:16.821 INFO:tasks.workunit.client.0.smithi152.stdout: Min xfer = 1048576.00 kB 2024-09-11T22:54:16.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:16 smithi152 ceph-mon[31181]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-11T22:54:17.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:16 smithi191 ceph-mon[35211]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-11T22:54:17.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:16 smithi204 ceph-mon[35454]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-11T22:54:18.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:18 smithi152 ceph-mon[31181]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-11T22:54:19.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:18 smithi191 ceph-mon[35211]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-11T22:54:19.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:18 smithi204 ceph-mon[35454]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-11T22:54:20.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:20 smithi152 ceph-mon[31181]: pgmap v46: 97 pgs: 97 active+clean; 1.0 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 MiB/s wr, 36 op/s 2024-09-11T22:54:21.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:20 smithi191 ceph-mon[35211]: pgmap v46: 97 pgs: 97 active+clean; 1.0 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 MiB/s wr, 36 op/s 2024-09-11T22:54:21.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:20 smithi204 ceph-mon[35454]: pgmap v46: 97 pgs: 97 active+clean; 1.0 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 MiB/s wr, 36 op/s 2024-09-11T22:54:22.150 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.149+0000 7f821a85e640 1 -- 172.21.15.152:0/1331430497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8214108020 msgr2=0x7f8214108460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.149+0000 7f821a85e640 1 --2- 172.21.15.152:0/1331430497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8214108020 0x7f8214108460 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8208009900 tx=0x7f820802a410 comp rx=0 tx=0).stop 2024-09-11T22:54:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.150+0000 7f821a85e640 1 -- 172.21.15.152:0/1331430497 shutdown_connections 2024-09-11T22:54:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.150+0000 7f821a85e640 1 --2- 172.21.15.152:0/1331430497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82141089a0 0x7f8214114dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.150+0000 7f821a85e640 1 --2- 172.21.15.152:0/1331430497 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8214108020 0x7f8214108460 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.150+0000 7f821a85e640 1 --2- 172.21.15.152:0/1331430497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f821410cf50 0x7f821410d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.150+0000 7f821a85e640 1 -- 172.21.15.152:0/1331430497 >> 172.21.15.152:0/1331430497 conn(0x7f8214077940 msgr2=0x7f8214079d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.150+0000 7f821a85e640 1 -- 172.21.15.152:0/1331430497 shutdown_connections 2024-09-11T22:54:22.151 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.150+0000 7f821a85e640 1 -- 172.21.15.152:0/1331430497 wait complete. 2024-09-11T22:54:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.151+0000 7f821a85e640 1 Processor -- start 2024-09-11T22:54:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.151+0000 7f821a85e640 1 -- start start 2024-09-11T22:54:22.152 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.152+0000 7f821a85e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8214108020 0x7f82141a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.152+0000 7f821a85e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82141089a0 0x7f82141a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.152+0000 7f821a85e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f821410cf50 0x7f821419e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.152+0000 7f821a85e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f82141184f0 con 0x7f8214108020 2024-09-11T22:54:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.152+0000 7f821a85e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8214118370 con 0x7f82141089a0 2024-09-11T22:54:22.153 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.152+0000 7f821a85e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8214118670 con 0x7f821410cf50 2024-09-11T22:54:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.152+0000 7f8213fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8214108020 0x7f82141a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.152+0000 7f8213fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8214108020 0x7f82141a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40500/0 (socket says 172.21.15.152:40500) 2024-09-11T22:54:22.154 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.153+0000 7f8213fff640 1 -- 172.21.15.152:0/4102494567 learned_addr learned my addr 172.21.15.152:0/4102494567 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.153+0000 7f8213fff640 1 -- 172.21.15.152:0/4102494567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f821410cf50 msgr2=0x7f821419e740 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:54:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.153+0000 7f8213fff640 1 --2- 172.21.15.152:0/4102494567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f821410cf50 0x7f821419e740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.153+0000 7f8213fff640 1 -- 172.21.15.152:0/4102494567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82141089a0 msgr2=0x7f82141a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.153+0000 7f8213fff640 1 --2- 172.21.15.152:0/4102494567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82141089a0 0x7f82141a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.155 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.153+0000 7f8213fff640 1 -- 172.21.15.152:0/4102494567 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821419f000 con 0x7f8214108020 2024-09-11T22:54:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.153+0000 7f8213fff640 1 --2- 172.21.15.152:0/4102494567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8214108020 0x7f82141a4680 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f820400b4a0 tx=0x7f820400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.154+0000 7f82117fa640 1 -- 172.21.15.152:0/4102494567 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204014070 con 0x7f8214108020 2024-09-11T22:54:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.154+0000 7f82117fa640 1 -- 172.21.15.152:0/4102494567 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8204010af0 con 0x7f8214108020 2024-09-11T22:54:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.154+0000 7f82117fa640 1 -- 172.21.15.152:0/4102494567 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204017d10 con 0x7f8214108020 2024-09-11T22:54:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.154+0000 7f821a85e640 1 -- 172.21.15.152:0/4102494567 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821419f2f0 con 0x7f8214108020 2024-09-11T22:54:22.156 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.154+0000 7f821a85e640 1 -- 172.21.15.152:0/4102494567 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82141ab420 con 0x7f8214108020 2024-09-11T22:54:22.160 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.159+0000 7f82117fa640 1 -- 172.21.15.152:0/4102494567 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 24) ==== 138067+0+0 (secure 0 0 0) 0x7f8204017070 con 0x7f8214108020 2024-09-11T22:54:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.159+0000 7f821d5fd740 1 Processor -- start 2024-09-11T22:54:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.159+0000 7f821d5fd740 1 -- start start 2024-09-11T22:54:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f821d5fd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e3340a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f821d5fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e3345e0 0x55b72e33e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f821d5fd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b72e33ee80 0x55b72e341270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f821d5fd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55b72e2982f0 con 0x55b72e333cc0 2024-09-11T22:54:22.163 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f821d5fd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b72e240270 con 0x55b72e33ee80 2024-09-11T22:54:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f821d5fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b72e2942b0 con 0x55b72e3345e0 2024-09-11T22:54:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f8218dfb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e3340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f8218dfb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e3340a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40514/0 (socket says 172.21.15.152:40514) 2024-09-11T22:54:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f8218dfb640 1 -- 172.21.15.152:0/2394681463 learned_addr learned my addr 172.21.15.152:0/2394681463 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f82117fa640 1 --2- 172.21.15.152:0/4102494567 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f81fc0a2f60 0x7f81fc0a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f8218dfb640 1 --2- 172.21.15.152:0/4102494567 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f81fc0a2f60 0x7f81fc0a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f82117fa640 1 -- 172.21.15.152:0/4102494567 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f82040ce960 con 0x7f8214108020 2024-09-11T22:54:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f8218dfb640 1 -- 172.21.15.152:0/2394681463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e3345e0 msgr2=0x55b72e33e940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:54:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f8218dfb640 1 --2- 172.21.15.152:0/2394681463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e3345e0 0x55b72e33e940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f8218dfb640 1 -- 172.21.15.152:0/2394681463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b72e33ee80 msgr2=0x55b72e341270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:22.164 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f8218dfb640 1 --2- 172.21.15.152:0/2394681463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b72e33ee80 0x55b72e341270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.160+0000 7f8218dfb640 1 -- 172.21.15.152:0/2394681463 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b72e23f340 con 0x55b72e333cc0 2024-09-11T22:54:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.161+0000 7f8218dfb640 1 --2- 172.21.15.152:0/2394681463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e3340a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f820c0136b0 tx=0x7f820c013b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125ff1e3e883ec68 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.161+0000 7f8218dfb640 1 --2- 172.21.15.152:0/4102494567 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f81fc0a2f60 0x7f81fc0a5420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f820c01d000 tx=0x7f820c018000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.161+0000 7f81fa7fc640 1 -- 172.21.15.152:0/2394681463 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c00d070 con 0x55b72e333cc0 2024-09-11T22:54:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.161+0000 7f81fa7fc640 1 -- 172.21.15.152:0/2394681463 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f820c010030 con 0x55b72e333cc0 2024-09-11T22:54:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.161+0000 7f81fa7fc640 1 -- 172.21.15.152:0/2394681463 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c01b040 con 0x55b72e333cc0 2024-09-11T22:54:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.162+0000 7f821d5fd740 1 -- 172.21.15.152:0/2394681463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 msgr2=0x55b72e3340a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.162+0000 7f821d5fd740 1 --2- 172.21.15.152:0/2394681463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e3340a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f820c0136b0 tx=0x7f820c013b70 comp rx=0 tx=0).stop 2024-09-11T22:54:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.162+0000 7f821d5fd740 1 -- 172.21.15.152:0/2394681463 shutdown_connections 2024-09-11T22:54:22.165 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.162+0000 7f821d5fd740 1 --2- 172.21.15.152:0/2394681463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b72e33ee80 0x55b72e341270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.166 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.162+0000 7f821d5fd740 1 --2- 172.21.15.152:0/2394681463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e3345e0 0x55b72e33e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.162+0000 7f821d5fd740 1 --2- 172.21.15.152:0/2394681463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e3340a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.162+0000 7f821d5fd740 1 -- 172.21.15.152:0/2394681463 >> 172.21.15.152:0/2394681463 conn(0x55b72e330230 msgr2=0x55b72e330610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.162+0000 7f821d5fd740 1 -- 172.21.15.152:0/2394681463 shutdown_connections 2024-09-11T22:54:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.162+0000 7f821d5fd740 1 -- 172.21.15.152:0/2394681463 wait complete. 2024-09-11T22:54:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.163+0000 7f821d5fd740 1 -- start start 2024-09-11T22:54:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.163+0000 7f821d5fd740 1 Processor -- start 2024-09-11T22:54:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.163+0000 7f821d5fd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e33bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.163+0000 7f821d5fd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b72e3345e0 0x55b72e33c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.163+0000 7f821d5fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e33ee80 0x55b72e33c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.163+0000 7f821d5fd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55b72e341a50 con 0x55b72e333cc0 2024-09-11T22:54:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.163+0000 7f821d5fd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b72e221290 con 0x55b72e3345e0 2024-09-11T22:54:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.163+0000 7f821d5fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b72e341bd0 con 0x55b72e33ee80 2024-09-11T22:54:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.164+0000 7f8213fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e33bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.164+0000 7f8213fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e33bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40530/0 (socket says 172.21.15.152:40530) 2024-09-11T22:54:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.164+0000 7f821d5fd740 10 client.0 _ll_register_callbacks cb 0x55b72e252140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.164+0000 7f8213fff640 1 -- 172.21.15.152:0/2443348010 learned_addr learned my addr 172.21.15.152:0/2443348010 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.164+0000 7f8218dfb640 1 --2- 172.21.15.152:0/2443348010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e33ee80 0x55b72e33c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.164+0000 7f8213fff640 1 -- 172.21.15.152:0/2443348010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e33ee80 msgr2=0x55b72e33c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.164+0000 7f8213fff640 1 --2- 172.21.15.152:0/2443348010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e33ee80 0x55b72e33c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.164+0000 7f8213fff640 1 -- 172.21.15.152:0/2443348010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b72e3345e0 msgr2=0x55b72e33c320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:54:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.164+0000 7f8213fff640 1 --2- 172.21.15.152:0/2443348010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b72e3345e0 0x55b72e33c320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.164+0000 7f8213fff640 1 -- 172.21.15.152:0/2443348010 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b72dfab210 con 0x55b72e333cc0 2024-09-11T22:54:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.164+0000 7f8213fff640 1 --2- 172.21.15.152:0/2443348010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e33bde0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f820400be60 tx=0x7f820409b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9c899913a9c456 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.164+0000 7f81f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.165+0000 7f81f37fe640 1 -- 172.21.15.152:0/2443348010 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820401e040 con 0x55b72e333cc0 2024-09-11T22:54:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.165+0000 7f81f37fe640 1 -- 172.21.15.152:0/2443348010 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f8204010ca0 con 0x55b72e333cc0 2024-09-11T22:54:22.171 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.165+0000 7f821d5fd740 1 -- 172.21.15.152:0/2443348010 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f820c010030 con 0x55b72e333cc0 2024-09-11T22:54:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.165+0000 7f81f37fe640 1 -- 172.21.15.152:0/2443348010 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f820409f100 con 0x55b72e333cc0 2024-09-11T22:54:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.165+0000 7f821d5fd740 10 client.15867 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.165+0000 7f821d5fd740 1 -- 172.21.15.152:0/2443348010 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b72e077620 con 0x55b72e333cc0 2024-09-11T22:54:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.165+0000 7f81f37fe640 1 -- 172.21.15.152:0/2443348010 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f820c010030 con 0x55b72e333cc0 2024-09-11T22:54:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.165+0000 7f81f37fe640 1 -- 172.21.15.152:0/2443348010 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f820409a1a0 con 0x55b72e333cc0 2024-09-11T22:54:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.165+0000 7f821d5fd740 10 client.15867 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.165+0000 7f821d5fd740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:54:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.165+0000 7f821d5fd740 20 client.15867 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.165+0000 7f821d5fd740 1 --2- 172.21.15.152:0/2443348010 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b72e34edf0 0x55b72e3511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.165+0000 7f821d5fd740 4 client.15867 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:54:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.165+0000 7f821d5fd740 1 -- 172.21.15.152:0/2443348010 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b72e2a5a20 con 0x55b72e34edf0 2024-09-11T22:54:22.172 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.168+0000 7f82137fe640 1 --2- 172.21.15.152:0/2443348010 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b72e34edf0 0x55b72e3511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.174+0000 7f81f37fe640 10 client.15867 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:22.176 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.174+0000 7f82137fe640 1 --2- 172.21.15.152:0/2443348010 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b72e34edf0 0x55b72e3511e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=371c8575068a2ea6 server_cookie=3bdb9306a9b5d5b9 in_seq=0 out_seq=0 2024-09-11T22:54:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.177+0000 7f81f37fe640 1 -- 172.21.15.152:0/2443348010 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b72e2a5a20 con 0x55b72e34edf0 2024-09-11T22:54:22.177 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.177+0000 7f81f37fe640 10 client.15867 handle_command_reply: tid=0 2024-09-11T22:54:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.177+0000 7f821d5fd740 1 client.15867 shutdown 2024-09-11T22:54:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.177+0000 7f821d5fd740 1 -- 172.21.15.152:0/2443348010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 msgr2=0x55b72e33bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.177+0000 7f821d5fd740 1 --2- 172.21.15.152:0/2443348010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e33bde0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f820400be60 tx=0x7f820409b450 comp rx=0 tx=0).stop 2024-09-11T22:54:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.177+0000 7f821d5fd740 1 -- 172.21.15.152:0/2443348010 shutdown_connections 2024-09-11T22:54:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.177+0000 7f821d5fd740 1 --2- 172.21.15.152:0/2443348010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e33ee80 0x55b72e33c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.177+0000 7f821d5fd740 1 --2- 172.21.15.152:0/2443348010 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b72e34edf0 0x55b72e3511e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.177+0000 7f821d5fd740 1 --2- 172.21.15.152:0/2443348010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b72e3345e0 0x55b72e33c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.177+0000 7f821d5fd740 1 --2- 172.21.15.152:0/2443348010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e33bde0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.177+0000 7f81f37fe640 0 client.15867 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.177+0000 7f821d5fd740 1 -- 172.21.15.152:0/2443348010 >> 172.21.15.152:0/2443348010 conn(0x55b72e330230 msgr2=0x55b72e335f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.177+0000 7f821d5fd740 1 -- 172.21.15.152:0/2443348010 shutdown_connections 2024-09-11T22:54:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.177+0000 7f821d5fd740 1 -- 172.21.15.152:0/2443348010 wait complete. 2024-09-11T22:54:22.180 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.177+0000 7f821d5fd740 20 client.15867 trim_cache size 0 max 16384 2024-09-11T22:54:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.194+0000 7f821d5fd740 1 Processor -- start 2024-09-11T22:54:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.194+0000 7f821d5fd740 1 -- start start 2024-09-11T22:54:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.195+0000 7f821d5fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e333cc0 0x55b72e33c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.195+0000 7f821d5fd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e3345e0 0x55b72e356ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.195+0000 7f821d5fd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b72e33ee80 0x55b72e356ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.195+0000 7f821d5fd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b3bb0 con 0x55b72e3345e0 2024-09-11T22:54:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.195+0000 7f821d5fd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b56f0 con 0x55b72e33ee80 2024-09-11T22:54:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.195+0000 7f821d5fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b4280 con 0x55b72e333cc0 2024-09-11T22:54:22.197 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.195+0000 7f82137fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e3345e0 0x55b72e356ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.195+0000 7f82137fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e3345e0 0x55b72e356ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40544/0 (socket says 172.21.15.152:40544) 2024-09-11T22:54:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.195+0000 7f82137fe640 1 -- 172.21.15.152:0/714494989 learned_addr learned my addr 172.21.15.152:0/714494989 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.195+0000 7f82137fe640 1 -- 172.21.15.152:0/714494989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e333cc0 msgr2=0x55b72e33c690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:54:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.195+0000 7f82137fe640 1 --2- 172.21.15.152:0/714494989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e333cc0 0x55b72e33c690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.195+0000 7f82137fe640 1 -- 172.21.15.152:0/714494989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b72e33ee80 msgr2=0x55b72e356ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:54:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.195+0000 7f82137fe640 1 --2- 172.21.15.152:0/714494989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b72e33ee80 0x55b72e356ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.195+0000 7f82137fe640 1 -- 172.21.15.152:0/714494989 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b72e334410 con 0x55b72e3345e0 2024-09-11T22:54:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.195+0000 7f82137fe640 1 --2- 172.21.15.152:0/714494989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e3345e0 0x55b72e356ab0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f820802a930 tx=0x7f82080053d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc3f0e5b4ebd13e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.196+0000 7f81f9ffb640 1 -- 172.21.15.152:0/714494989 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820804a070 con 0x55b72e3345e0 2024-09-11T22:54:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.196+0000 7f81f9ffb640 1 -- 172.21.15.152:0/714494989 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8208005440 con 0x55b72e3345e0 2024-09-11T22:54:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.196+0000 7f81f9ffb640 1 -- 172.21.15.152:0/714494989 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208035020 con 0x55b72e3345e0 2024-09-11T22:54:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.196+0000 7f821d5fd740 1 -- 172.21.15.152:0/714494989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e3345e0 msgr2=0x55b72e356ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.196+0000 7f821d5fd740 1 --2- 172.21.15.152:0/714494989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e3345e0 0x55b72e356ab0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f820802a930 tx=0x7f82080053d0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.196+0000 7f821d5fd740 1 -- 172.21.15.152:0/714494989 shutdown_connections 2024-09-11T22:54:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.196+0000 7f821d5fd740 1 --2- 172.21.15.152:0/714494989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b72e33ee80 0x55b72e356ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.196+0000 7f821d5fd740 1 --2- 172.21.15.152:0/714494989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e3345e0 0x55b72e356ab0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.196+0000 7f821d5fd740 1 --2- 172.21.15.152:0/714494989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e333cc0 0x55b72e33c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.196+0000 7f821d5fd740 1 -- 172.21.15.152:0/714494989 >> 172.21.15.152:0/714494989 conn(0x55b72e330230 msgr2=0x55b72e33de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.197+0000 7f821d5fd740 1 -- 172.21.15.152:0/714494989 shutdown_connections 2024-09-11T22:54:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.197+0000 7f821d5fd740 1 -- 172.21.15.152:0/714494989 wait complete. 2024-09-11T22:54:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.197+0000 7f821d5fd740 1 -- start start 2024-09-11T22:54:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.197+0000 7f821d5fd740 1 Processor -- start 2024-09-11T22:54:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.197+0000 7f821d5fd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e346e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.197+0000 7f821d5fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e3345e0 0x55b72e3473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.197+0000 7f821d5fd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b72e33ee80 0x55b72e343e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.197+0000 7f821d5fd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b5020 con 0x55b72e333cc0 2024-09-11T22:54:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.197+0000 7f821d5fd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b4950 con 0x55b72e33ee80 2024-09-11T22:54:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.197+0000 7f821d5fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82141b1b70 con 0x55b72e3345e0 2024-09-11T22:54:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f821d5fd740 10 client.0 _ll_register_callbacks cb 0x7f82040092b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f8213fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e346e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f8213fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e346e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40552/0 (socket says 172.21.15.152:40552) 2024-09-11T22:54:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f8213fff640 1 -- 172.21.15.152:0/1796262256 learned_addr learned my addr 172.21.15.152:0/1796262256 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f8213fff640 1 -- 172.21.15.152:0/1796262256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e3345e0 msgr2=0x55b72e3473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f8213fff640 1 --2- 172.21.15.152:0/1796262256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e3345e0 0x55b72e3473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f8213fff640 1 -- 172.21.15.152:0/1796262256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b72e33ee80 msgr2=0x55b72e343e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:54:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f8213fff640 1 --2- 172.21.15.152:0/1796262256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b72e33ee80 0x55b72e343e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f8213fff640 1 -- 172.21.15.152:0/1796262256 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b72e344370 con 0x55b72e333cc0 2024-09-11T22:54:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f8213fff640 1 --2- 172.21.15.152:0/1796262256 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e346e60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f82040cc160 tx=0x7f82040975f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d349616520103f69 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f81f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f81f37fe640 1 -- 172.21.15.152:0/1796262256 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820401e070 con 0x55b72e333cc0 2024-09-11T22:54:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f81f37fe640 1 -- 172.21.15.152:0/1796262256 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f82040cc220 con 0x55b72e333cc0 2024-09-11T22:54:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f821d5fd740 1 -- 172.21.15.152:0/1796262256 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8208005440 con 0x55b72e333cc0 2024-09-11T22:54:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f81f37fe640 1 -- 172.21.15.152:0/1796262256 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f820409a890 con 0x55b72e333cc0 2024-09-11T22:54:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f821d5fd740 10 client.15873 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.198+0000 7f821d5fd740 1 -- 172.21.15.152:0/1796262256 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b72e3ba1b0 con 0x55b72e333cc0 2024-09-11T22:54:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.199+0000 7f81f37fe640 1 -- 172.21.15.152:0/1796262256 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8208005440 con 0x55b72e333cc0 2024-09-11T22:54:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.199+0000 7f81f37fe640 1 -- 172.21.15.152:0/1796262256 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8204099c10 con 0x55b72e333cc0 2024-09-11T22:54:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.199+0000 7f821d5fd740 10 client.15873 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.199+0000 7f821d5fd740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:54:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.199+0000 7f821d5fd740 20 client.15873 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.199+0000 7f821d5fd740 1 --2- 172.21.15.152:0/1796262256 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b72e3ba740 0x55b72e3bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.199+0000 7f821d5fd740 4 client.15873 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:54:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.199+0000 7f821d5fd740 1 -- 172.21.15.152:0/1796262256 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b72e29fd70 con 0x55b72e3ba740 2024-09-11T22:54:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.200+0000 7f82137fe640 1 --2- 172.21.15.152:0/1796262256 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b72e3ba740 0x55b72e3bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.202+0000 7f82137fe640 1 --2- 172.21.15.152:0/1796262256 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b72e3ba740 0x55b72e3bcb70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16976071fe465c8 server_cookie=bdb94ceca9efd01 in_seq=0 out_seq=0 2024-09-11T22:54:22.206 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.202+0000 7f81f37fe640 10 client.15873 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:22.302 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.302+0000 7f81f37fe640 1 -- 172.21.15.152:0/1796262256 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b72e29fd70 con 0x55b72e3ba740 2024-09-11T22:54:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.302+0000 7f81f37fe640 10 client.15873 handle_command_reply: tid=0 2024-09-11T22:54:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.302+0000 7f821d5fd740 1 client.15873 shutdown 2024-09-11T22:54:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.302+0000 7f821d5fd740 1 -- 172.21.15.152:0/1796262256 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 msgr2=0x55b72e346e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.302+0000 7f821d5fd740 1 --2- 172.21.15.152:0/1796262256 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e346e60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f82040cc160 tx=0x7f82040975f0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.302+0000 7f821d5fd740 1 -- 172.21.15.152:0/1796262256 shutdown_connections 2024-09-11T22:54:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.302+0000 7f821d5fd740 1 --2- 172.21.15.152:0/1796262256 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b72e3ba740 0x55b72e3bcb70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.302+0000 7f821d5fd740 1 --2- 172.21.15.152:0/1796262256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b72e33ee80 0x55b72e343e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.302+0000 7f81f37fe640 0 client.15873 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.302+0000 7f821d5fd740 1 --2- 172.21.15.152:0/1796262256 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b72e3345e0 0x55b72e3473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.304 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.302+0000 7f821d5fd740 1 --2- 172.21.15.152:0/1796262256 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b72e333cc0 0x55b72e346e60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.302+0000 7f821d5fd740 1 -- 172.21.15.152:0/1796262256 >> 172.21.15.152:0/1796262256 conn(0x55b72e330230 msgr2=0x55b72e335720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.302+0000 7f821d5fd740 1 -- 172.21.15.152:0/1796262256 shutdown_connections 2024-09-11T22:54:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.302+0000 7f821d5fd740 1 -- 172.21.15.152:0/1796262256 wait complete. 2024-09-11T22:54:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.303+0000 7f821d5fd740 20 client.15873 trim_cache size 0 max 16384 2024-09-11T22:54:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:{ 2024-09-11T22:54:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "message": "", 2024-09-11T22:54:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "return_code": 0 2024-09-11T22:54:22.305 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:} 2024-09-11T22:54:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.303+0000 7f821a85e640 1 -- 172.21.15.152:0/4102494567 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f81fc0a2f60 msgr2=0x7f81fc0a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.303+0000 7f821a85e640 1 --2- 172.21.15.152:0/4102494567 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f81fc0a2f60 0x7f81fc0a5420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f820c01d000 tx=0x7f820c018000 comp rx=0 tx=0).stop 2024-09-11T22:54:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.303+0000 7f821a85e640 1 -- 172.21.15.152:0/4102494567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8214108020 msgr2=0x7f82141a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:22.306 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.303+0000 7f821a85e640 1 --2- 172.21.15.152:0/4102494567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8214108020 0x7f82141a4680 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f820400b4a0 tx=0x7f820400b960 comp rx=0 tx=0).stop 2024-09-11T22:54:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.304+0000 7f821a85e640 1 -- 172.21.15.152:0/4102494567 shutdown_connections 2024-09-11T22:54:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.304+0000 7f821a85e640 1 --2- 172.21.15.152:0/4102494567 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f81fc0a2f60 0x7f81fc0a5420 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.304+0000 7f821a85e640 1 --2- 172.21.15.152:0/4102494567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f821410cf50 0x7f821419e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.304+0000 7f821a85e640 1 --2- 172.21.15.152:0/4102494567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82141089a0 0x7f82141a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.304+0000 7f821a85e640 1 --2- 172.21.15.152:0/4102494567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8214108020 0x7f82141a4680 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.304+0000 7f821a85e640 1 -- 172.21.15.152:0/4102494567 >> 172.21.15.152:0/4102494567 conn(0x7f8214077940 msgr2=0x7f8214078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.304+0000 7f821a85e640 1 -- 172.21.15.152:0/4102494567 shutdown_connections 2024-09-11T22:54:22.307 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:22.304+0000 7f821a85e640 1 -- 172.21.15.152:0/4102494567 wait complete. 2024-09-11T22:54:22.722 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:22.722 INFO:tasks.workunit.client.0.smithi152.stdout: Children see throughput for 1 rewriters = 743216.06 kB/sec 2024-09-11T22:54:22.722 INFO:tasks.workunit.client.0.smithi152.stdout: Parent sees throughput for 1 rewriters = 743102.83 kB/sec 2024-09-11T22:54:22.722 INFO:tasks.workunit.client.0.smithi152.stdout: Min throughput per process = 743216.06 kB/sec 2024-09-11T22:54:22.722 INFO:tasks.workunit.client.0.smithi152.stdout: Max throughput per process = 743216.06 kB/sec 2024-09-11T22:54:22.722 INFO:tasks.workunit.client.0.smithi152.stdout: Avg throughput per process = 743216.06 kB/sec 2024-09-11T22:54:22.722 INFO:tasks.workunit.client.0.smithi152.stdout: Min xfer = 1048576.00 kB 2024-09-11T22:54:22.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:22 smithi152 ceph-mon[31181]: pgmap v47: 97 pgs: 97 active+clean; 1.0 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 85 MiB/s wr, 31 op/s 2024-09-11T22:54:23.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:22 smithi191 ceph-mon[35211]: pgmap v47: 97 pgs: 97 active+clean; 1.0 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 85 MiB/s wr, 31 op/s 2024-09-11T22:54:23.084 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:22 smithi204 ceph-mon[35454]: pgmap v47: 97 pgs: 97 active+clean; 1.0 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 85 MiB/s wr, 31 op/s 2024-09-11T22:54:24.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:24 smithi152 ceph-mon[31181]: pgmap v48: 97 pgs: 97 active+clean; 1.0 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 85 MiB/s wr, 31 op/s 2024-09-11T22:54:25.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:24 smithi191 ceph-mon[35211]: pgmap v48: 97 pgs: 97 active+clean; 1.0 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 85 MiB/s wr, 31 op/s 2024-09-11T22:54:25.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:24 smithi204 ceph-mon[35454]: pgmap v48: 97 pgs: 97 active+clean; 1.0 GiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 85 MiB/s wr, 31 op/s 2024-09-11T22:54:25.118 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:25.118 INFO:tasks.workunit.client.0.smithi152.stdout: Children see throughput for 1 readers = 6775546.50 kB/sec 2024-09-11T22:54:25.118 INFO:tasks.workunit.client.0.smithi152.stdout: Parent sees throughput for 1 readers = 6767414.33 kB/sec 2024-09-11T22:54:25.119 INFO:tasks.workunit.client.0.smithi152.stdout: Min throughput per process = 6775546.50 kB/sec 2024-09-11T22:54:25.119 INFO:tasks.workunit.client.0.smithi152.stdout: Max throughput per process = 6775546.50 kB/sec 2024-09-11T22:54:25.119 INFO:tasks.workunit.client.0.smithi152.stdout: Avg throughput per process = 6775546.50 kB/sec 2024-09-11T22:54:25.119 INFO:tasks.workunit.client.0.smithi152.stdout: Min xfer = 1048576.00 kB 2024-09-11T22:54:26.785 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:54:26.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:26 smithi152 ceph-mon[31181]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 171 MiB/s wr, 61 op/s 2024-09-11T22:54:26.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:26 smithi152 ceph-mon[31181]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-11T22:54:27.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:26 smithi191 ceph-mon[35211]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 171 MiB/s wr, 61 op/s 2024-09-11T22:54:27.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:26 smithi191 ceph-mon[35211]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-11T22:54:27.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:26 smithi204 ceph-mon[35454]: pgmap v49: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 171 MiB/s wr, 61 op/s 2024-09-11T22:54:27.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:26 smithi204 ceph-mon[35454]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-11T22:54:27.051 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:54:27.363 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:27.364 INFO:tasks.workunit.client.0.smithi152.stdout: Children see throughput for 1 re-readers = 7032702.00 kB/sec 2024-09-11T22:54:27.364 INFO:tasks.workunit.client.0.smithi152.stdout: Parent sees throughput for 1 re-readers = 7022954.30 kB/sec 2024-09-11T22:54:27.364 INFO:tasks.workunit.client.0.smithi152.stdout: Min throughput per process = 7032702.00 kB/sec 2024-09-11T22:54:27.364 INFO:tasks.workunit.client.0.smithi152.stdout: Max throughput per process = 7032702.00 kB/sec 2024-09-11T22:54:27.364 INFO:tasks.workunit.client.0.smithi152.stdout: Avg throughput per process = 7032702.00 kB/sec 2024-09-11T22:54:27.364 INFO:tasks.workunit.client.0.smithi152.stdout: Min xfer = 1048576.00 kB 2024-09-11T22:54:27.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.790+0000 7faeafad5640 1 -- 172.21.15.152:0/695915810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faea8102ef0 msgr2=0x7faea810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:27.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.790+0000 7faeafad5640 1 --2- 172.21.15.152:0/695915810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faea8102ef0 0x7faea810da40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7faea0009ca0 tx=0x7faea002a950 comp rx=0 tx=0).stop 2024-09-11T22:54:27.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.790+0000 7faeafad5640 1 -- 172.21.15.152:0/695915810 shutdown_connections 2024-09-11T22:54:27.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.790+0000 7faeafad5640 1 --2- 172.21.15.152:0/695915810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faea8102ef0 0x7faea810da40 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.790+0000 7faeafad5640 1 --2- 172.21.15.152:0/695915810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faea8102570 0x7faea81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.790+0000 7faeafad5640 1 --2- 172.21.15.152:0/695915810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faea8100e20 0x7faea8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.790+0000 7faeafad5640 1 -- 172.21.15.152:0/695915810 >> 172.21.15.152:0/695915810 conn(0x7faea80fc7d0 msgr2=0x7faea80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:27.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.790+0000 7faeafad5640 1 -- 172.21.15.152:0/695915810 shutdown_connections 2024-09-11T22:54:27.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.790+0000 7faeafad5640 1 -- 172.21.15.152:0/695915810 wait complete. 2024-09-11T22:54:27.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.790+0000 7faeafad5640 1 Processor -- start 2024-09-11T22:54:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faeafad5640 1 -- start start 2024-09-11T22:54:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faeafad5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faea8100e20 0x7faea80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faeafad5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faea8102570 0x7faea80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faeafad5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faea8102ef0 0x7faea80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faeafad5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faea8110f80 con 0x7faea8102570 2024-09-11T22:54:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faeafad5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faea8110e00 con 0x7faea8102ef0 2024-09-11T22:54:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faeafad5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faea8111100 con 0x7faea8100e20 2024-09-11T22:54:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faead070640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faea8102570 0x7faea80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faead070640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faea8102570 0x7faea80fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40560/0 (socket says 172.21.15.152:40560) 2024-09-11T22:54:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faead070640 1 -- 172.21.15.152:0/2319415928 learned_addr learned my addr 172.21.15.152:0/2319415928 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faead871640 1 --2- 172.21.15.152:0/2319415928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faea8100e20 0x7faea80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faeae072640 1 --2- 172.21.15.152:0/2319415928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faea8102ef0 0x7faea80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:27.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faead070640 1 -- 172.21.15.152:0/2319415928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faea8100e20 msgr2=0x7faea80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:27.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faead070640 1 --2- 172.21.15.152:0/2319415928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faea8100e20 0x7faea80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faead070640 1 -- 172.21.15.152:0/2319415928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faea8102ef0 msgr2=0x7faea80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:27.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faead070640 1 --2- 172.21.15.152:0/2319415928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faea8102ef0 0x7faea80fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faead070640 1 -- 172.21.15.152:0/2319415928 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea81a6930 con 0x7faea8102570 2024-09-11T22:54:27.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.791+0000 7faead070640 1 --2- 172.21.15.152:0/2319415928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faea8102570 0x7faea80fd1c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c920 tx=0x7fae9800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:27.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.792+0000 7fae9effd640 1 -- 172.21.15.152:0/2319415928 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98007c70 con 0x7faea8102570 2024-09-11T22:54:27.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.792+0000 7fae9effd640 1 -- 172.21.15.152:0/2319415928 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae98007e10 con 0x7faea8102570 2024-09-11T22:54:27.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.792+0000 7faeafad5640 1 -- 172.21.15.152:0/2319415928 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea81a6c20 con 0x7faea8102570 2024-09-11T22:54:27.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.792+0000 7fae9effd640 1 -- 172.21.15.152:0/2319415928 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98018890 con 0x7faea8102570 2024-09-11T22:54:27.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.792+0000 7faeafad5640 1 -- 172.21.15.152:0/2319415928 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea81a7160 con 0x7faea8102570 2024-09-11T22:54:27.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.794+0000 7fae9effd640 1 -- 172.21.15.152:0/2319415928 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fae98018b30 con 0x7faea8102570 2024-09-11T22:54:27.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.794+0000 7faeb1eae740 1 Processor -- start 2024-09-11T22:54:27.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faeb1eae740 1 -- start start 2024-09-11T22:54:27.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7fae9effd640 1 --2- 172.21.15.152:0/2319415928 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fae940a2cf0 0x7fae940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faeb1eae740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf67880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faeb1eae740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf67dc0 0x5555bdf72140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faeb1eae740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf72680 0x5555bdf74a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faeb1eae740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5555bdecd2e0 con 0x5555bdf674a0 2024-09-11T22:54:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faeb1eae740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5555bde74ce0 con 0x5555bdf72680 2024-09-11T22:54:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faeb1eae740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5555bdec92a0 con 0x5555bdf67dc0 2024-09-11T22:54:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7fae9effd640 1 -- 172.21.15.152:0/2319415928 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fae980cf420 con 0x7faea8102570 2024-09-11T22:54:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faeae072640 1 --2- 172.21.15.152:0/2319415928 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fae940a2cf0 0x7fae940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faead871640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf67880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faead871640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf67880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40568/0 (socket says 172.21.15.152:40568) 2024-09-11T22:54:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faead871640 1 -- 172.21.15.152:0/2886702646 learned_addr learned my addr 172.21.15.152:0/2886702646 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faead070640 1 --2- 172.21.15.152:0/2886702646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf67dc0 0x5555bdf72140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faead871640 1 -- 172.21.15.152:0/2886702646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf67dc0 msgr2=0x5555bdf72140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faead871640 1 --2- 172.21.15.152:0/2886702646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf67dc0 0x5555bdf72140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faead871640 1 -- 172.21.15.152:0/2886702646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf72680 msgr2=0x5555bdf74a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:27.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faead871640 1 --2- 172.21.15.152:0/2886702646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf72680 0x5555bdf74a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faead871640 1 -- 172.21.15.152:0/2886702646 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555bde59440 con 0x5555bdf674a0 2024-09-11T22:54:27.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faeae072640 1 --2- 172.21.15.152:0/2319415928 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fae940a2cf0 0x7fae940a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faea0005e30 tx=0x7faea0009820 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:27.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.795+0000 7faead871640 1 --2- 172.21.15.152:0/2886702646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf67880 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7faea400b760 tx=0x7faea400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11a9a6aaeb8db52 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:27.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.796+0000 7fae93fff640 1 -- 172.21.15.152:0/2886702646 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4003dd0 con 0x5555bdf674a0 2024-09-11T22:54:27.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.796+0000 7fae93fff640 1 -- 172.21.15.152:0/2886702646 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea4015070 con 0x5555bdf674a0 2024-09-11T22:54:27.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.796+0000 7fae93fff640 1 -- 172.21.15.152:0/2886702646 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4004de0 con 0x5555bdf674a0 2024-09-11T22:54:27.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.796+0000 7faeb1eae740 1 -- 172.21.15.152:0/2886702646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 msgr2=0x5555bdf67880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:27.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.796+0000 7faeb1eae740 1 --2- 172.21.15.152:0/2886702646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf67880 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7faea400b760 tx=0x7faea400bc20 comp rx=0 tx=0).stop 2024-09-11T22:54:27.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.796+0000 7faeb1eae740 1 -- 172.21.15.152:0/2886702646 shutdown_connections 2024-09-11T22:54:27.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.796+0000 7faeb1eae740 1 --2- 172.21.15.152:0/2886702646 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf72680 0x5555bdf74a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.796+0000 7faeb1eae740 1 --2- 172.21.15.152:0/2886702646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf67dc0 0x5555bdf72140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.796+0000 7faeb1eae740 1 --2- 172.21.15.152:0/2886702646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf67880 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.796+0000 7faeb1eae740 1 -- 172.21.15.152:0/2886702646 >> 172.21.15.152:0/2886702646 conn(0x5555bdf63a30 msgr2=0x5555bdf63e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:27.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.797+0000 7faeb1eae740 1 -- 172.21.15.152:0/2886702646 shutdown_connections 2024-09-11T22:54:27.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.797+0000 7faeb1eae740 1 -- 172.21.15.152:0/2886702646 wait complete. 2024-09-11T22:54:27.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.797+0000 7faeb1eae740 1 -- start start 2024-09-11T22:54:27.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.797+0000 7faeb1eae740 1 Processor -- start 2024-09-11T22:54:27.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.797+0000 7faeb1eae740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf71f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.797+0000 7faeb1eae740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf67dc0 0x5555bdf6f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.797+0000 7faeb1eae740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf72680 0x5555bdf6f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.797+0000 7faeb1eae740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5555bdf75250 con 0x5555bdf674a0 2024-09-11T22:54:27.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.797+0000 7faeb1eae740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5555bde88050 con 0x5555bdf67dc0 2024-09-11T22:54:27.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.797+0000 7faeb1eae740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5555bdf753d0 con 0x5555bdf72680 2024-09-11T22:54:27.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.797+0000 7faeb1eae740 10 client.0 _ll_register_callbacks cb 0x5555bdec8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:27.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.797+0000 7faead871640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf71f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7faead871640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf71f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40578/0 (socket says 172.21.15.152:40578) 2024-09-11T22:54:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7faead871640 1 -- 172.21.15.152:0/3436889528 learned_addr learned my addr 172.21.15.152:0/3436889528 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7faead070640 1 --2- 172.21.15.152:0/3436889528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf67dc0 0x5555bdf6f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7faead871640 1 -- 172.21.15.152:0/3436889528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf72680 msgr2=0x5555bdf6f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7faeae072640 1 --2- 172.21.15.152:0/3436889528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf72680 0x5555bdf6f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7faead871640 1 --2- 172.21.15.152:0/3436889528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf72680 0x5555bdf6f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7faead871640 1 -- 172.21.15.152:0/3436889528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf67dc0 msgr2=0x5555bdf6f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7faead871640 1 --2- 172.21.15.152:0/3436889528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf67dc0 0x5555bdf6f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7faead871640 1 -- 172.21.15.152:0/3436889528 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555bdc3e3c0 con 0x5555bdf674a0 2024-09-11T22:54:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7faead070640 1 --2- 172.21.15.152:0/3436889528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf67dc0 0x5555bdf6f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:54:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7faead871640 1 --2- 172.21.15.152:0/3436889528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf71f90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7faea4006010 tx=0x7faea4003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9435e7446cef4adf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7fae917fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:27.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7fae917fa640 1 -- 172.21.15.152:0/3436889528 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4028060 con 0x5555bdf674a0 2024-09-11T22:54:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7fae917fa640 1 -- 172.21.15.152:0/3436889528 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7faea4011850 con 0x5555bdf674a0 2024-09-11T22:54:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7faeb1eae740 1 -- 172.21.15.152:0/3436889528 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faea4015070 con 0x5555bdf674a0 2024-09-11T22:54:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7fae917fa640 1 -- 172.21.15.152:0/3436889528 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faea4011a50 con 0x5555bdf674a0 2024-09-11T22:54:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7faeb1eae740 10 client.15885 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.798+0000 7faeb1eae740 1 -- 172.21.15.152:0/3436889528 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555bdf80c00 con 0x5555bdf674a0 2024-09-11T22:54:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.799+0000 7fae917fa640 1 -- 172.21.15.152:0/3436889528 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faea4015070 con 0x5555bdf674a0 2024-09-11T22:54:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.799+0000 7fae917fa640 1 -- 172.21.15.152:0/3436889528 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faea4022070 con 0x5555bdf674a0 2024-09-11T22:54:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.799+0000 7faeb1eae740 10 client.15885 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.799+0000 7faeb1eae740 10 client.15885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:54:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.799+0000 7faeb1eae740 20 client.15885 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.799+0000 7faeb1eae740 1 --2- 172.21.15.152:0/3436889528 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5555bdf820c0 0x5555bdf844b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.799+0000 7faeb1eae740 4 client.15885 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:54:27.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.799+0000 7faeb1eae740 1 -- 172.21.15.152:0/3436889528 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555bdeda850 con 0x5555bdf820c0 2024-09-11T22:54:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.799+0000 7faead070640 1 --2- 172.21.15.152:0/3436889528 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5555bdf820c0 0x5555bdf844b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.800+0000 7faead070640 1 --2- 172.21.15.152:0/3436889528 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5555bdf820c0 0x5555bdf844b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=610bec29274ceaa4 server_cookie=1e0aeb03456b19fb in_seq=0 out_seq=0 2024-09-11T22:54:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.800+0000 7fae917fa640 10 client.15885 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.800+0000 7fae917fa640 1 -- 172.21.15.152:0/3436889528 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555bdeda850 con 0x5555bdf820c0 2024-09-11T22:54:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.800+0000 7fae917fa640 10 client.15885 handle_command_reply: tid=0 2024-09-11T22:54:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.801+0000 7faeb1eae740 1 client.15885 shutdown 2024-09-11T22:54:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.801+0000 7faeb1eae740 1 -- 172.21.15.152:0/3436889528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 msgr2=0x5555bdf71f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.801+0000 7faeb1eae740 1 --2- 172.21.15.152:0/3436889528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf71f90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7faea4006010 tx=0x7faea4003e00 comp rx=0 tx=0).stop 2024-09-11T22:54:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.801+0000 7faeb1eae740 1 -- 172.21.15.152:0/3436889528 shutdown_connections 2024-09-11T22:54:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.801+0000 7faeb1eae740 1 --2- 172.21.15.152:0/3436889528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf72680 0x5555bdf6f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.801+0000 7faeb1eae740 1 --2- 172.21.15.152:0/3436889528 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5555bdf820c0 0x5555bdf844b0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.801+0000 7faeb1eae740 1 --2- 172.21.15.152:0/3436889528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf67dc0 0x5555bdf6f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.801+0000 7faeb1eae740 1 --2- 172.21.15.152:0/3436889528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf71f90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.801+0000 7fae917fa640 0 client.15885 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:27.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.801+0000 7faeb1eae740 1 -- 172.21.15.152:0/3436889528 >> 172.21.15.152:0/3436889528 conn(0x5555bdf63a30 msgr2=0x5555bdf69710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:27.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.801+0000 7faeb1eae740 1 -- 172.21.15.152:0/3436889528 shutdown_connections 2024-09-11T22:54:27.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.801+0000 7faeb1eae740 1 -- 172.21.15.152:0/3436889528 wait complete. 2024-09-11T22:54:27.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.801+0000 7faeb1eae740 20 client.15885 trim_cache size 0 max 16384 2024-09-11T22:54:27.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faeb1eae740 1 Processor -- start 2024-09-11T22:54:27.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faeb1eae740 1 -- start start 2024-09-11T22:54:27.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faeb1eae740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf674a0 0x5555bdf7e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faeb1eae740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf67dc0 0x5555bdf89ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faeb1eae740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf72680 0x5555bdf89ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faeb1eae740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faea81af910 con 0x5555bdf67dc0 2024-09-11T22:54:27.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faeb1eae740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faea81b1450 con 0x5555bdf674a0 2024-09-11T22:54:27.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faeb1eae740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faea81affe0 con 0x5555bdf72680 2024-09-11T22:54:27.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faead070640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf67dc0 0x5555bdf89ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:27.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faead070640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf67dc0 0x5555bdf89ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40580/0 (socket says 172.21.15.152:40580) 2024-09-11T22:54:27.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faead070640 1 -- 172.21.15.152:0/2130959331 learned_addr learned my addr 172.21.15.152:0/2130959331 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:27.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faead871640 1 --2- 172.21.15.152:0/2130959331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf674a0 0x5555bdf7e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:27.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faead070640 1 -- 172.21.15.152:0/2130959331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf72680 msgr2=0x5555bdf89ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:27.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faead070640 1 --2- 172.21.15.152:0/2130959331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf72680 0x5555bdf89ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faead070640 1 -- 172.21.15.152:0/2130959331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf674a0 msgr2=0x5555bdf7e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:27.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faead070640 1 --2- 172.21.15.152:0/2130959331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf674a0 0x5555bdf7e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faead070640 1 -- 172.21.15.152:0/2130959331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555bdf634d0 con 0x5555bdf67dc0 2024-09-11T22:54:27.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.818+0000 7faead070640 1 --2- 172.21.15.152:0/2130959331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf67dc0 0x5555bdf89ab0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fae98007590 tx=0x7fae98086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b25b13e829eac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:27.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.819+0000 7fae937fe640 1 -- 172.21.15.152:0/2130959331 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae980b8020 con 0x5555bdf67dc0 2024-09-11T22:54:27.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.819+0000 7fae937fe640 1 -- 172.21.15.152:0/2130959331 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae980a4600 con 0x5555bdf67dc0 2024-09-11T22:54:27.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.819+0000 7fae937fe640 1 -- 172.21.15.152:0/2130959331 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae980a4170 con 0x5555bdf67dc0 2024-09-11T22:54:27.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.819+0000 7faeb1eae740 1 -- 172.21.15.152:0/2130959331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf67dc0 msgr2=0x5555bdf89ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:27.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.819+0000 7faeb1eae740 1 --2- 172.21.15.152:0/2130959331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf67dc0 0x5555bdf89ab0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fae98007590 tx=0x7fae98086040 comp rx=0 tx=0).stop 2024-09-11T22:54:27.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.819+0000 7faeb1eae740 1 -- 172.21.15.152:0/2130959331 shutdown_connections 2024-09-11T22:54:27.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.819+0000 7faeb1eae740 1 --2- 172.21.15.152:0/2130959331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf72680 0x5555bdf89ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.819+0000 7faeb1eae740 1 --2- 172.21.15.152:0/2130959331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf67dc0 0x5555bdf89ab0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.819+0000 7faeb1eae740 1 --2- 172.21.15.152:0/2130959331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf674a0 0x5555bdf7e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.819+0000 7faeb1eae740 1 -- 172.21.15.152:0/2130959331 >> 172.21.15.152:0/2130959331 conn(0x5555bdf63a30 msgr2=0x5555bdf6e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:27.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.819+0000 7faeb1eae740 1 -- 172.21.15.152:0/2130959331 shutdown_connections 2024-09-11T22:54:27.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.819+0000 7faeb1eae740 1 -- 172.21.15.152:0/2130959331 wait complete. 2024-09-11T22:54:27.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.820+0000 7faeb1eae740 1 -- start start 2024-09-11T22:54:27.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.820+0000 7faeb1eae740 1 Processor -- start 2024-09-11T22:54:27.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.820+0000 7faeb1eae740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf763f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.820+0000 7faeb1eae740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf67dc0 0x5555bdf76930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.820+0000 7faeb1eae740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf72680 0x5555bdf76e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.820+0000 7faeb1eae740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faea81b0d80 con 0x5555bdf674a0 2024-09-11T22:54:27.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.820+0000 7faeb1eae740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faea81b06b0 con 0x5555bdf67dc0 2024-09-11T22:54:27.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.820+0000 7faeb1eae740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faea81ad8d0 con 0x5555bdf72680 2024-09-11T22:54:27.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.820+0000 7faeb1eae740 10 client.0 _ll_register_callbacks cb 0x7faea4004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:27.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.820+0000 7faead871640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:27.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.820+0000 7faead871640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf763f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40596/0 (socket says 172.21.15.152:40596) 2024-09-11T22:54:27.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.820+0000 7faead871640 1 -- 172.21.15.152:0/3474524379 learned_addr learned my addr 172.21.15.152:0/3474524379 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:27.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.820+0000 7faead070640 1 --2- 172.21.15.152:0/3474524379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf67dc0 0x5555bdf76930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:27.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.820+0000 7faead871640 1 -- 172.21.15.152:0/3474524379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf72680 msgr2=0x5555bdf76e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:27.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.820+0000 7faead871640 1 --2- 172.21.15.152:0/3474524379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf72680 0x5555bdf76e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7faead871640 1 -- 172.21.15.152:0/3474524379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf67dc0 msgr2=0x5555bdf76930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:27.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7faead871640 1 --2- 172.21.15.152:0/3474524379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf67dc0 0x5555bdf76930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7faead871640 1 -- 172.21.15.152:0/3474524379 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555bdfebad0 con 0x5555bdf674a0 2024-09-11T22:54:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7faead871640 1 --2- 172.21.15.152:0/3474524379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf763f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7faea40046f0 tx=0x7faea4010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa26ccf70a5654b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7fae917fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7fae917fa640 1 -- 172.21.15.152:0/3474524379 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4010b80 con 0x5555bdf674a0 2024-09-11T22:54:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7fae917fa640 1 -- 172.21.15.152:0/3474524379 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7faea4010d20 con 0x5555bdf674a0 2024-09-11T22:54:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7faeb1eae740 1 -- 172.21.15.152:0/3474524379 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae980a4600 con 0x5555bdf674a0 2024-09-11T22:54:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7fae917fa640 1 -- 172.21.15.152:0/3474524379 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faea403d050 con 0x5555bdf674a0 2024-09-11T22:54:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7faeb1eae740 10 client.15891 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7faeb1eae740 1 -- 172.21.15.152:0/3474524379 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555bdfecdc0 con 0x5555bdf674a0 2024-09-11T22:54:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7fae917fa640 1 -- 172.21.15.152:0/3474524379 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae980a4600 con 0x5555bdf674a0 2024-09-11T22:54:27.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7fae917fa640 1 -- 172.21.15.152:0/3474524379 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faea4035020 con 0x5555bdf674a0 2024-09-11T22:54:27.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7faeb1eae740 10 client.15891 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:27.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7faeb1eae740 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:54:27.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7faeb1eae740 20 client.15891 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:27.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7faeb1eae740 1 --2- 172.21.15.152:0/3474524379 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5555bdfed500 0x5555bdfef940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:27.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7faeb1eae740 4 client.15891 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:54:27.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.821+0000 7faeb1eae740 1 -- 172.21.15.152:0/3474524379 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5555bdf6fb10 con 0x5555bdfed500 2024-09-11T22:54:27.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.822+0000 7faead070640 1 --2- 172.21.15.152:0/3474524379 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5555bdfed500 0x5555bdfef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:27.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.822+0000 7fae917fa640 10 client.15891 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:27.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.822+0000 7faead070640 1 --2- 172.21.15.152:0/3474524379 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5555bdfed500 0x5555bdfef940 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33449d0bee33415b server_cookie=6b50ac1f4d675448 in_seq=0 out_seq=0 2024-09-11T22:54:27.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.822+0000 7fae917fa640 1 -- 172.21.15.152:0/3474524379 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5555bdf6fb10 con 0x5555bdfed500 2024-09-11T22:54:27.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.822+0000 7fae917fa640 10 client.15891 handle_command_reply: tid=0 2024-09-11T22:54:27.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.822+0000 7faeb1eae740 1 client.15891 shutdown 2024-09-11T22:54:27.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.822+0000 7faeb1eae740 1 -- 172.21.15.152:0/3474524379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 msgr2=0x5555bdf763f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:27.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.822+0000 7faeb1eae740 1 --2- 172.21.15.152:0/3474524379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf763f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7faea40046f0 tx=0x7faea4010640 comp rx=0 tx=0).stop 2024-09-11T22:54:27.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeb1eae740 1 -- 172.21.15.152:0/3474524379 shutdown_connections 2024-09-11T22:54:27.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeb1eae740 1 --2- 172.21.15.152:0/3474524379 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555bdf72680 0x5555bdf76e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeb1eae740 1 --2- 172.21.15.152:0/3474524379 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5555bdfed500 0x5555bdfef940 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeb1eae740 1 --2- 172.21.15.152:0/3474524379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555bdf67dc0 0x5555bdf76930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7fae917fa640 0 client.15891 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:27.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeb1eae740 1 --2- 172.21.15.152:0/3474524379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555bdf674a0 0x5555bdf763f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeb1eae740 1 -- 172.21.15.152:0/3474524379 >> 172.21.15.152:0/3474524379 conn(0x5555bdf63a30 msgr2=0x5555bdf62350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:27.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeb1eae740 1 -- 172.21.15.152:0/3474524379 shutdown_connections 2024-09-11T22:54:27.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeb1eae740 1 -- 172.21.15.152:0/3474524379 wait complete. 2024-09-11T22:54:27.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeb1eae740 20 client.15891 trim_cache size 0 max 16384 2024-09-11T22:54:27.831 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:54:27.831 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T22:54:27.831 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T22:54:27.831 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:54:27.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeafad5640 1 -- 172.21.15.152:0/2319415928 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fae940a2cf0 msgr2=0x7fae940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:27.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeafad5640 1 --2- 172.21.15.152:0/2319415928 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fae940a2cf0 0x7fae940a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faea0005e30 tx=0x7faea0009820 comp rx=0 tx=0).stop 2024-09-11T22:54:27.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeafad5640 1 -- 172.21.15.152:0/2319415928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faea8102570 msgr2=0x7faea80fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:27.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeafad5640 1 --2- 172.21.15.152:0/2319415928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faea8102570 0x7faea80fd1c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c920 tx=0x7fae9800cde0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeafad5640 1 -- 172.21.15.152:0/2319415928 shutdown_connections 2024-09-11T22:54:27.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeafad5640 1 --2- 172.21.15.152:0/2319415928 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fae940a2cf0 0x7fae940a51b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeafad5640 1 --2- 172.21.15.152:0/2319415928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faea8102ef0 0x7faea80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeafad5640 1 --2- 172.21.15.152:0/2319415928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faea8102570 0x7faea80fd1c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeafad5640 1 --2- 172.21.15.152:0/2319415928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faea8100e20 0x7faea80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:27.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.823+0000 7faeafad5640 1 -- 172.21.15.152:0/2319415928 >> 172.21.15.152:0/2319415928 conn(0x7faea80fc7d0 msgr2=0x7faea806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:27.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.824+0000 7faeafad5640 1 -- 172.21.15.152:0/2319415928 shutdown_connections 2024-09-11T22:54:27.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:27.824+0000 7faeafad5640 1 -- 172.21.15.152:0/2319415928 wait complete. 2024-09-11T22:54:28.648 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:54:28.649 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:54:28.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:28 smithi152 ceph-mon[31181]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 171 MiB/s wr, 56 op/s 2024-09-11T22:54:28.894 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:54:29.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:28 smithi191 ceph-mon[35211]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 171 MiB/s wr, 56 op/s 2024-09-11T22:54:29.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:28 smithi204 ceph-mon[35454]: pgmap v50: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 171 MiB/s wr, 56 op/s 2024-09-11T22:54:29.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.395+0000 7f075f47c640 1 -- 172.21.15.152:0/3527074475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0758101420 msgr2=0x7f075810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:29.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.395+0000 7f075f47c640 1 --2- 172.21.15.152:0/3527074475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0758101420 0x7f075810d950 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0750009cd0 tx=0x7f075002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:54:29.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.395+0000 7f075f47c640 1 -- 172.21.15.152:0/3527074475 shutdown_connections 2024-09-11T22:54:29.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.395+0000 7f075f47c640 1 --2- 172.21.15.152:0/3527074475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0758101420 0x7f075810d950 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:29.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.395+0000 7f075f47c640 1 --2- 172.21.15.152:0/3527074475 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0758100a80 0x7f0758100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:29.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.395+0000 7f075f47c640 1 --2- 172.21.15.152:0/3527074475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0758106a80 0x7f0758106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:29.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.395+0000 7f075f47c640 1 -- 172.21.15.152:0/3527074475 >> 172.21.15.152:0/3527074475 conn(0x7f07580fc7a0 msgr2=0x7f07580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:29.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.395+0000 7f075f47c640 1 -- 172.21.15.152:0/3527074475 shutdown_connections 2024-09-11T22:54:29.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.395+0000 7f075f47c640 1 -- 172.21.15.152:0/3527074475 wait complete. 2024-09-11T22:54:29.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.396+0000 7f075f47c640 1 Processor -- start 2024-09-11T22:54:29.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.396+0000 7f075f47c640 1 -- start start 2024-09-11T22:54:29.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.396+0000 7f075f47c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0758100a80 0x7f0758199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:29.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.396+0000 7f075f47c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0758101420 0x7f07581997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:29.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.396+0000 7f075f47c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0758106a80 0x7f07581a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:29.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.396+0000 7f075f47c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0758110ea0 con 0x7f0758106a80 2024-09-11T22:54:29.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.396+0000 7f075f47c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0758110d20 con 0x7f0758100a80 2024-09-11T22:54:29.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.396+0000 7f075f47c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0758111020 con 0x7f0758101420 2024-09-11T22:54:29.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.396+0000 7f075da19640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0758106a80 0x7f07581a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:29.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.396+0000 7f075da19640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0758106a80 0x7f07581a42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49056/0 (socket says 172.21.15.152:49056) 2024-09-11T22:54:29.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.396+0000 7f075da19640 1 -- 172.21.15.152:0/1667723183 learned_addr learned my addr 172.21.15.152:0/1667723183 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:29.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.397+0000 7f075d218640 1 --2- 172.21.15.152:0/1667723183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0758100a80 0x7f0758199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:29.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.397+0000 7f075ca17640 1 --2- 172.21.15.152:0/1667723183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0758101420 0x7f07581997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:29.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.397+0000 7f075da19640 1 -- 172.21.15.152:0/1667723183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0758101420 msgr2=0x7f07581997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:29.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.397+0000 7f075da19640 1 --2- 172.21.15.152:0/1667723183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0758101420 0x7f07581997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:29.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.397+0000 7f075da19640 1 -- 172.21.15.152:0/1667723183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0758100a80 msgr2=0x7f0758199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:29.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.397+0000 7f075da19640 1 --2- 172.21.15.152:0/1667723183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0758100a80 0x7f0758199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:29.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.397+0000 7f075da19640 1 -- 172.21.15.152:0/1667723183 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07581a4830 con 0x7f0758106a80 2024-09-11T22:54:29.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.397+0000 7f075da19640 1 --2- 172.21.15.152:0/1667723183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0758106a80 0x7f07581a42f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0750006010 tx=0x7f0750003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:29.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.397+0000 7f074e7fc640 1 -- 172.21.15.152:0/1667723183 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750004450 con 0x7f0758106a80 2024-09-11T22:54:29.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.397+0000 7f074e7fc640 1 -- 172.21.15.152:0/1667723183 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0750003e10 con 0x7f0758106a80 2024-09-11T22:54:29.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.397+0000 7f075f47c640 1 -- 172.21.15.152:0/1667723183 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07581a4a00 con 0x7f0758106a80 2024-09-11T22:54:29.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.397+0000 7f074e7fc640 1 -- 172.21.15.152:0/1667723183 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07500466e0 con 0x7f0758106a80 2024-09-11T22:54:29.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.397+0000 7f075f47c640 1 -- 172.21.15.152:0/1667723183 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07581a4eb0 con 0x7f0758106a80 2024-09-11T22:54:29.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.398+0000 7f075f47c640 1 -- 172.21.15.152:0/1667723183 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0758100ee0 con 0x7f0758106a80 2024-09-11T22:54:29.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.399+0000 7f074e7fc640 1 -- 172.21.15.152:0/1667723183 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f07500305a0 con 0x7f0758106a80 2024-09-11T22:54:29.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.400+0000 7f074e7fc640 1 --2- 172.21.15.152:0/1667723183 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f07440a2cf0 0x7f07440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:29.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.400+0000 7f075d218640 1 --2- 172.21.15.152:0/1667723183 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f07440a2cf0 0x7f07440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:29.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.400+0000 7f074e7fc640 1 -- 172.21.15.152:0/1667723183 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f075002e030 con 0x7f0758106a80 2024-09-11T22:54:29.403 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.401+0000 7f075d218640 1 --2- 172.21.15.152:0/1667723183 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f07440a2cf0 0x7f07440a51b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0754005e10 tx=0x7f0754005da0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:29.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.404+0000 7f074e7fc640 1 -- 172.21.15.152:0/1667723183 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f075004b060 con 0x7f0758106a80 2024-09-11T22:54:29.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.604+0000 7f075f47c640 1 -- 172.21.15.152:0/1667723183 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07581a5140 con 0x7f0758106a80 2024-09-11T22:54:29.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.605+0000 7f074e7fc640 1 -- 172.21.15.152:0/1667723183 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f07500bdac0 con 0x7f0758106a80 2024-09-11T22:54:29.605 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:54:29.605 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:54:29.606 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:54:29.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.607+0000 7f075f47c640 1 -- 172.21.15.152:0/1667723183 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f07440a2cf0 msgr2=0x7f07440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:29.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.607+0000 7f075f47c640 1 --2- 172.21.15.152:0/1667723183 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f07440a2cf0 0x7f07440a51b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0754005e10 tx=0x7f0754005da0 comp rx=0 tx=0).stop 2024-09-11T22:54:29.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.607+0000 7f075f47c640 1 -- 172.21.15.152:0/1667723183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0758106a80 msgr2=0x7f07581a42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:29.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.607+0000 7f075f47c640 1 --2- 172.21.15.152:0/1667723183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0758106a80 0x7f07581a42f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0750006010 tx=0x7f0750003c20 comp rx=0 tx=0).stop 2024-09-11T22:54:29.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.607+0000 7f075f47c640 1 -- 172.21.15.152:0/1667723183 shutdown_connections 2024-09-11T22:54:29.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.607+0000 7f075f47c640 1 --2- 172.21.15.152:0/1667723183 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f07440a2cf0 0x7f07440a51b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:29.607 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.607+0000 7f075f47c640 1 --2- 172.21.15.152:0/1667723183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0758106a80 0x7f07581a42f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:29.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.607+0000 7f075f47c640 1 --2- 172.21.15.152:0/1667723183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0758101420 0x7f07581997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:29.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.607+0000 7f075f47c640 1 --2- 172.21.15.152:0/1667723183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0758100a80 0x7f0758199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:29.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.607+0000 7f075f47c640 1 -- 172.21.15.152:0/1667723183 >> 172.21.15.152:0/1667723183 conn(0x7f07580fc7a0 msgr2=0x7f07580fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:29.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.607+0000 7f075f47c640 1 -- 172.21.15.152:0/1667723183 shutdown_connections 2024-09-11T22:54:29.608 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:29.607+0000 7f075f47c640 1 -- 172.21.15.152:0/1667723183 wait complete. 2024-09-11T22:54:29.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:29 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1667723183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:54:30.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:29 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1667723183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:54:30.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:29 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1667723183' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:54:30.333 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:54:30.584 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:54:30.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:30 smithi152 ceph-mon[31181]: pgmap v51: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 171 MiB/s wr, 61 op/s 2024-09-11T22:54:31.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:30 smithi191 ceph-mon[35211]: pgmap v51: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 171 MiB/s wr, 61 op/s 2024-09-11T22:54:31.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:30 smithi204 ceph-mon[35454]: pgmap v51: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 171 MiB/s wr, 61 op/s 2024-09-11T22:54:31.099 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.080+0000 7f763b397640 1 -- 172.21.15.152:0/1344638574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7634106a80 msgr2=0x7f7634106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:31.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.080+0000 7f763b397640 1 --2- 172.21.15.152:0/1344638574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7634106a80 0x7f7634106e60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7630009930 tx=0x7f763002a420 comp rx=0 tx=0).stop 2024-09-11T22:54:31.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.081+0000 7f763b397640 1 -- 172.21.15.152:0/1344638574 shutdown_connections 2024-09-11T22:54:31.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.081+0000 7f763b397640 1 --2- 172.21.15.152:0/1344638574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7634101420 0x7f763410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.081+0000 7f763b397640 1 --2- 172.21.15.152:0/1344638574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7634100a80 0x7f7634100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.081+0000 7f763b397640 1 --2- 172.21.15.152:0/1344638574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7634106a80 0x7f7634106e60 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.081+0000 7f763b397640 1 -- 172.21.15.152:0/1344638574 >> 172.21.15.152:0/1344638574 conn(0x7f76340fc7a0 msgr2=0x7f76340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:31.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.081+0000 7f763b397640 1 -- 172.21.15.152:0/1344638574 shutdown_connections 2024-09-11T22:54:31.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.081+0000 7f763b397640 1 -- 172.21.15.152:0/1344638574 wait complete. 2024-09-11T22:54:31.100 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.081+0000 7f763b397640 1 Processor -- start 2024-09-11T22:54:31.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.081+0000 7f763b397640 1 -- start start 2024-09-11T22:54:31.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.081+0000 7f763b397640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7634100a80 0x7f76340fcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.081+0000 7f763b397640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7634101420 0x7f76340fd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.081+0000 7f763b397640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7634106a80 0x7f76340fd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.081+0000 7f763b397640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7634110ef0 con 0x7f7634106a80 2024-09-11T22:54:31.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.081+0000 7f763b397640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7634110d70 con 0x7f7634101420 2024-09-11T22:54:31.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.081+0000 7f763b397640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7634111070 con 0x7f7634100a80 2024-09-11T22:54:31.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.082+0000 7f7639934640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7634106a80 0x7f76340fd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.082+0000 7f7639934640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7634106a80 0x7f76340fd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49074/0 (socket says 172.21.15.152:49074) 2024-09-11T22:54:31.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.082+0000 7f7639934640 1 -- 172.21.15.152:0/927561292 learned_addr learned my addr 172.21.15.152:0/927561292 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:31.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.082+0000 7f7638932640 1 --2- 172.21.15.152:0/927561292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7634101420 0x7f76340fd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.101 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.082+0000 7f7639133640 1 --2- 172.21.15.152:0/927561292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7634100a80 0x7f76340fcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.082+0000 7f7639934640 1 -- 172.21.15.152:0/927561292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7634100a80 msgr2=0x7f76340fcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:31.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.082+0000 7f7639934640 1 --2- 172.21.15.152:0/927561292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7634100a80 0x7f76340fcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.082+0000 7f7639934640 1 -- 172.21.15.152:0/927561292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7634101420 msgr2=0x7f76340fd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:31.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.082+0000 7f7639934640 1 --2- 172.21.15.152:0/927561292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7634101420 0x7f76340fd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.082+0000 7f7639934640 1 -- 172.21.15.152:0/927561292 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76341a69d0 con 0x7f7634106a80 2024-09-11T22:54:31.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.082+0000 7f7639934640 1 --2- 172.21.15.152:0/927561292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7634106a80 0x7f76340fd830 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f762c00cad0 tx=0x7f762c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:31.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.082+0000 7f762a7fc640 1 -- 172.21.15.152:0/927561292 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c00ae20 con 0x7f7634106a80 2024-09-11T22:54:31.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.082+0000 7f763b397640 1 -- 172.21.15.152:0/927561292 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76341a6cc0 con 0x7f7634106a80 2024-09-11T22:54:31.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.082+0000 7f762a7fc640 1 -- 172.21.15.152:0/927561292 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f762c005020 con 0x7f7634106a80 2024-09-11T22:54:31.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.082+0000 7f762a7fc640 1 -- 172.21.15.152:0/927561292 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c00f6f0 con 0x7f7634106a80 2024-09-11T22:54:31.102 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.082+0000 7f763b397640 1 -- 172.21.15.152:0/927561292 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76341a7200 con 0x7f7634106a80 2024-09-11T22:54:31.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.085+0000 7f762a7fc640 1 -- 172.21.15.152:0/927561292 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f762c0051d0 con 0x7f7634106a80 2024-09-11T22:54:31.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.085+0000 7f763d770740 1 Processor -- start 2024-09-11T22:54:31.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.085+0000 7f763d770740 1 -- start start 2024-09-11T22:54:31.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f763d770740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf431e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f763d770740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf431e5d0 0x557cf431ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f763d770740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf4329370 0x557cf432b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f763d770740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557cf42842f0 con 0x557cf431fb40 2024-09-11T22:54:31.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f763d770740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557cf422bcf0 con 0x557cf4329370 2024-09-11T22:54:31.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f763d770740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557cf42802b0 con 0x557cf431e5d0 2024-09-11T22:54:31.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f7638932640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf431e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f7638932640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf431e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49080/0 (socket says 172.21.15.152:49080) 2024-09-11T22:54:31.103 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f7638932640 1 -- 172.21.15.152:0/6193486 learned_addr learned my addr 172.21.15.152:0/6193486 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:31.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f7639133640 1 --2- 172.21.15.152:0/6193486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf431e5d0 0x557cf431ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f7638932640 1 --2- 172.21.15.152:0/6193486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf4329370 0x557cf432b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f762a7fc640 1 --2- 172.21.15.152:0/927561292 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f76200a2cf0 0x7f76200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f7638932640 1 -- 172.21.15.152:0/6193486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf431e5d0 msgr2=0x557cf431ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:31.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f7638932640 1 --2- 172.21.15.152:0/6193486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf431e5d0 0x557cf431ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f7638932640 1 -- 172.21.15.152:0/6193486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf4329370 msgr2=0x557cf432b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:31.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f7638932640 1 --2- 172.21.15.152:0/6193486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf4329370 0x557cf432b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f7638932640 1 -- 172.21.15.152:0/6193486 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cf42103a0 con 0x557cf431fb40 2024-09-11T22:54:31.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f7639934640 1 --2- 172.21.15.152:0/927561292 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f76200a2cf0 0x7f76200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f7638932640 1 --2- 172.21.15.152:0/6193486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf431e090 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7624006e10 tx=0x7f7624009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722f16b99a3f3b7d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:31.104 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f762a7fc640 1 -- 172.21.15.152:0/927561292 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f762c0cf570 con 0x7f7634106a80 2024-09-11T22:54:31.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.086+0000 7f761f7fe640 1 -- 172.21.15.152:0/6193486 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762401c070 con 0x557cf431fb40 2024-09-11T22:54:31.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.087+0000 7f761f7fe640 1 -- 172.21.15.152:0/6193486 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7624010040 con 0x557cf431fb40 2024-09-11T22:54:31.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.087+0000 7f761f7fe640 1 -- 172.21.15.152:0/6193486 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76240146f0 con 0x557cf431fb40 2024-09-11T22:54:31.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.087+0000 7f763d770740 1 -- 172.21.15.152:0/6193486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 msgr2=0x557cf431e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:31.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.087+0000 7f7639934640 1 --2- 172.21.15.152:0/927561292 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f76200a2cf0 0x7f76200a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f762c09b7e0 tx=0x7f762c06b940 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:31.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.087+0000 7f763d770740 1 --2- 172.21.15.152:0/6193486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf431e090 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7624006e10 tx=0x7f7624009420 comp rx=0 tx=0).stop 2024-09-11T22:54:31.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.087+0000 7f763d770740 1 -- 172.21.15.152:0/6193486 shutdown_connections 2024-09-11T22:54:31.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.087+0000 7f763d770740 1 --2- 172.21.15.152:0/6193486 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf4329370 0x557cf432b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.087+0000 7f763d770740 1 --2- 172.21.15.152:0/6193486 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf431e5d0 0x557cf431ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.087+0000 7f763d770740 1 --2- 172.21.15.152:0/6193486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf431e090 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.105 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.087+0000 7f763d770740 1 -- 172.21.15.152:0/6193486 >> 172.21.15.152:0/6193486 conn(0x557cf42e7310 msgr2=0x557cf431ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:31.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.087+0000 7f763d770740 1 -- 172.21.15.152:0/6193486 shutdown_connections 2024-09-11T22:54:31.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.087+0000 7f763d770740 1 -- 172.21.15.152:0/6193486 wait complete. 2024-09-11T22:54:31.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.088+0000 7f763d770740 1 -- start start 2024-09-11T22:54:31.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.088+0000 7f763d770740 1 Processor -- start 2024-09-11T22:54:31.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.088+0000 7f763d770740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf431e5d0 0x557cf4328db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.088+0000 7f763d770740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf4326060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.088+0000 7f763d770740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf4329370 0x557cf43267f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.088+0000 7f763d770740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557cf432c070 con 0x557cf431fb40 2024-09-11T22:54:31.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.088+0000 7f763d770740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557cf423f060 con 0x557cf431e5d0 2024-09-11T22:54:31.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.088+0000 7f763d770740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557cf432c1f0 con 0x557cf4329370 2024-09-11T22:54:31.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.088+0000 7f763d770740 10 client.0 _ll_register_callbacks cb 0x557cf3f928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:31.106 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.088+0000 7f7639133640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf4326060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.088+0000 7f7639133640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf4326060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49084/0 (socket says 172.21.15.152:49084) 2024-09-11T22:54:31.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.088+0000 7f7639133640 1 -- 172.21.15.152:0/347186344 learned_addr learned my addr 172.21.15.152:0/347186344 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:31.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.088+0000 7f7638932640 1 --2- 172.21.15.152:0/347186344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf431e5d0 0x557cf4328db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.088+0000 7f7638932640 1 --2- 172.21.15.152:0/347186344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf4329370 0x557cf43267f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f7639133640 1 -- 172.21.15.152:0/347186344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf4329370 msgr2=0x557cf43267f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:31.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f7639133640 1 --2- 172.21.15.152:0/347186344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf4329370 0x557cf43267f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f7639133640 1 -- 172.21.15.152:0/347186344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf431e5d0 msgr2=0x557cf4328db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:31.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f7639133640 1 --2- 172.21.15.152:0/347186344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf431e5d0 0x557cf4328db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f7639133640 1 -- 172.21.15.152:0/347186344 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cf3ff53c0 con 0x557cf431fb40 2024-09-11T22:54:31.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f7638932640 1 --2- 172.21.15.152:0/347186344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf4329370 0x557cf43267f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:54:31.107 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f7639133640 1 --2- 172.21.15.152:0/347186344 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf4326060 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7630031550 tx=0x7f7630003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162014a84fd88a8e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:31.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f761cff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:31.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f761cff9640 1 -- 172.21.15.152:0/347186344 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630035070 con 0x557cf431fb40 2024-09-11T22:54:31.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f761cff9640 1 -- 172.21.15.152:0/347186344 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7630003c10 con 0x557cf431fb40 2024-09-11T22:54:31.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f763d770740 1 -- 172.21.15.152:0/347186344 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7624010040 con 0x557cf431fb40 2024-09-11T22:54:31.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f761cff9640 1 -- 172.21.15.152:0/347186344 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7630005d50 con 0x557cf431fb40 2024-09-11T22:54:31.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f763d770740 10 client.15909 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:31.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f763d770740 1 -- 172.21.15.152:0/347186344 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557cf4337b60 con 0x557cf431fb40 2024-09-11T22:54:31.108 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f761cff9640 1 -- 172.21.15.152:0/347186344 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7624010040 con 0x557cf431fb40 2024-09-11T22:54:31.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f761cff9640 1 -- 172.21.15.152:0/347186344 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7630004c10 con 0x557cf431fb40 2024-09-11T22:54:31.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f763d770740 10 client.15909 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:31.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.089+0000 7f763d770740 10 client.15909 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:54:31.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.090+0000 7f763d770740 20 client.15909 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:31.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.090+0000 7f763d770740 1 --2- 172.21.15.152:0/347186344 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x557cf4339020 0x557cf433b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.090+0000 7f763d770740 4 client.15909 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:54:31.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.090+0000 7f763d770740 1 -- 172.21.15.152:0/347186344 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cf4291860 con 0x557cf4339020 2024-09-11T22:54:31.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.090+0000 7f7638932640 1 --2- 172.21.15.152:0/347186344 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x557cf4339020 0x557cf433b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.090+0000 7f761cff9640 10 client.15909 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:54:31.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.090+0000 7f7638932640 1 --2- 172.21.15.152:0/347186344 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x557cf4339020 0x557cf433b410 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7e351ef3aa33983c server_cookie=77a71f949b79e167 in_seq=0 out_seq=0 2024-09-11T22:54:31.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.091+0000 7f761cff9640 1 -- 172.21.15.152:0/347186344 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557cf4291860 con 0x557cf4339020 2024-09-11T22:54:31.109 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.091+0000 7f761cff9640 10 client.15909 handle_command_reply: tid=0 2024-09-11T22:54:31.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.091+0000 7f763d770740 1 client.15909 shutdown 2024-09-11T22:54:31.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.091+0000 7f763d770740 1 -- 172.21.15.152:0/347186344 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 msgr2=0x557cf4326060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:31.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.091+0000 7f763d770740 1 --2- 172.21.15.152:0/347186344 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf4326060 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7630031550 tx=0x7f7630003820 comp rx=0 tx=0).stop 2024-09-11T22:54:31.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.091+0000 7f763d770740 1 -- 172.21.15.152:0/347186344 shutdown_connections 2024-09-11T22:54:31.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.091+0000 7f763d770740 1 --2- 172.21.15.152:0/347186344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf4329370 0x557cf43267f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.091+0000 7f763d770740 1 --2- 172.21.15.152:0/347186344 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf4326060 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.091+0000 7f763d770740 1 --2- 172.21.15.152:0/347186344 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x557cf4339020 0x557cf433b410 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.091+0000 7f763d770740 1 --2- 172.21.15.152:0/347186344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf431e5d0 0x557cf4328db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.091+0000 7f761cff9640 0 client.15909 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:54:31.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.091+0000 7f763d770740 1 -- 172.21.15.152:0/347186344 >> 172.21.15.152:0/347186344 conn(0x557cf42e7310 msgr2=0x557cf4320960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:31.110 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.091+0000 7f763d770740 1 -- 172.21.15.152:0/347186344 shutdown_connections 2024-09-11T22:54:31.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.091+0000 7f763d770740 1 -- 172.21.15.152:0/347186344 wait complete. 2024-09-11T22:54:31.111 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.091+0000 7f763d770740 20 client.15909 trim_cache size 0 max 16384 2024-09-11T22:54:31.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f763d770740 1 Processor -- start 2024-09-11T22:54:31.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f763d770740 1 -- start start 2024-09-11T22:54:31.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f763d770740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf431e5d0 0x557cf4327fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f763d770740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf4321120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f763d770740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf4329370 0x557cf4321660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.116 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f763d770740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f76341afa10 con 0x557cf431fb40 2024-09-11T22:54:31.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f763d770740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f76341b1550 con 0x557cf431e5d0 2024-09-11T22:54:31.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f763d770740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f76341b00e0 con 0x557cf4329370 2024-09-11T22:54:31.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f7639133640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf4321120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f7639133640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf4321120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49086/0 (socket says 172.21.15.152:49086) 2024-09-11T22:54:31.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f7639133640 1 -- 172.21.15.152:0/1736508967 learned_addr learned my addr 172.21.15.152:0/1736508967 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:31.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f7638932640 1 --2- 172.21.15.152:0/1736508967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf431e5d0 0x557cf4327fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f7638932640 1 --2- 172.21.15.152:0/1736508967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf4329370 0x557cf4321660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f7639133640 1 -- 172.21.15.152:0/1736508967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf4329370 msgr2=0x557cf4321660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:31.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f7639133640 1 --2- 172.21.15.152:0/1736508967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf4329370 0x557cf4321660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f7639133640 1 -- 172.21.15.152:0/1736508967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf431e5d0 msgr2=0x557cf4327fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:31.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.104+0000 7f7639133640 1 --2- 172.21.15.152:0/1736508967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf431e5d0 0x557cf4327fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.117 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.105+0000 7f7639133640 1 -- 172.21.15.152:0/1736508967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cf4326b60 con 0x557cf431fb40 2024-09-11T22:54:31.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.105+0000 7f7639133640 1 --2- 172.21.15.152:0/1736508967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf4321120 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7630006010 tx=0x7f76300319d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577641dc4be43f8d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:31.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.105+0000 7f761effd640 1 -- 172.21.15.152:0/1736508967 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630035070 con 0x557cf431fb40 2024-09-11T22:54:31.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.105+0000 7f761effd640 1 -- 172.21.15.152:0/1736508967 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76300047a0 con 0x557cf431fb40 2024-09-11T22:54:31.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.105+0000 7f761effd640 1 -- 172.21.15.152:0/1736508967 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630005240 con 0x557cf431fb40 2024-09-11T22:54:31.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 -- 172.21.15.152:0/1736508967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 msgr2=0x557cf4321120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:31.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 --2- 172.21.15.152:0/1736508967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf4321120 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7630006010 tx=0x7f76300319d0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 -- 172.21.15.152:0/1736508967 shutdown_connections 2024-09-11T22:54:31.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 --2- 172.21.15.152:0/1736508967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf4329370 0x557cf4321660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 --2- 172.21.15.152:0/1736508967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431fb40 0x557cf4321120 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 --2- 172.21.15.152:0/1736508967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf431e5d0 0x557cf4327fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.118 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 -- 172.21.15.152:0/1736508967 >> 172.21.15.152:0/1736508967 conn(0x557cf42e7310 msgr2=0x557cf43278c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:31.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 -- 172.21.15.152:0/1736508967 shutdown_connections 2024-09-11T22:54:31.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 -- 172.21.15.152:0/1736508967 wait complete. 2024-09-11T22:54:31.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 -- start start 2024-09-11T22:54:31.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 Processor -- start 2024-09-11T22:54:31.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431e5d0 0x557cf432d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf431fb40 0x557cf432d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf4329370 0x557cf432deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f76341b0e80 con 0x557cf431e5d0 2024-09-11T22:54:31.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f76341b07b0 con 0x557cf4329370 2024-09-11T22:54:31.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.106+0000 7f763d770740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f76341ad9d0 con 0x557cf431fb40 2024-09-11T22:54:31.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.107+0000 7f763d770740 10 client.0 _ll_register_callbacks cb 0x7f7630009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:31.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.107+0000 7f7638932640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431e5d0 0x557cf432d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.107+0000 7f7638932640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431e5d0 0x557cf432d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49096/0 (socket says 172.21.15.152:49096) 2024-09-11T22:54:31.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.107+0000 7f7638932640 1 -- 172.21.15.152:0/4184470889 learned_addr learned my addr 172.21.15.152:0/4184470889 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:31.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.107+0000 7f7639133640 1 --2- 172.21.15.152:0/4184470889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf431fb40 0x557cf432d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.107+0000 7f7638932640 1 --2- 172.21.15.152:0/4184470889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf4329370 0x557cf432deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.107+0000 7f7638932640 1 -- 172.21.15.152:0/4184470889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf431fb40 msgr2=0x557cf432d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:31.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.107+0000 7f7638932640 1 --2- 172.21.15.152:0/4184470889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf431fb40 0x557cf432d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.107+0000 7f7638932640 1 -- 172.21.15.152:0/4184470889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf4329370 msgr2=0x557cf432deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:31.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.107+0000 7f7638932640 1 --2- 172.21.15.152:0/4184470889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf4329370 0x557cf432deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.107+0000 7f7638932640 1 -- 172.21.15.152:0/4184470889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cf43a2eb0 con 0x557cf431e5d0 2024-09-11T22:54:31.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.107+0000 7f7638932640 1 --2- 172.21.15.152:0/4184470889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431e5d0 0x557cf432d430 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7624017da0 tx=0x7f762402f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2025bc6c9a8725f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:31.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.107+0000 7f761cff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:31.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.107+0000 7f761cff9640 1 -- 172.21.15.152:0/4184470889 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762401c040 con 0x557cf431e5d0 2024-09-11T22:54:31.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.107+0000 7f761cff9640 1 -- 172.21.15.152:0/4184470889 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f76240176c0 con 0x557cf431e5d0 2024-09-11T22:54:31.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.107+0000 7f763d770740 1 -- 172.21.15.152:0/4184470889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76300047a0 con 0x557cf431e5d0 2024-09-11T22:54:31.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.108+0000 7f761cff9640 1 -- 172.21.15.152:0/4184470889 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f76240178c0 con 0x557cf431e5d0 2024-09-11T22:54:31.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.108+0000 7f763d770740 10 client.15915 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:31.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.108+0000 7f763d770740 1 -- 172.21.15.152:0/4184470889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557cf43a4130 con 0x557cf431e5d0 2024-09-11T22:54:31.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.108+0000 7f761cff9640 1 -- 172.21.15.152:0/4184470889 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76300047a0 con 0x557cf431e5d0 2024-09-11T22:54:31.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.108+0000 7f761cff9640 1 -- 172.21.15.152:0/4184470889 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7624022040 con 0x557cf431e5d0 2024-09-11T22:54:31.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.108+0000 7f763d770740 10 client.15915 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:31.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.108+0000 7f763d770740 10 client.15915 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:54:31.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.108+0000 7f763d770740 20 client.15915 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:31.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.108+0000 7f763d770740 1 --2- 172.21.15.152:0/4184470889 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x557cf43a4600 0x557cf43a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:31.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.108+0000 7f763d770740 4 client.15915 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:54:31.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.108+0000 7f763d770740 1 -- 172.21.15.152:0/4184470889 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557cf428bd70 con 0x557cf43a4600 2024-09-11T22:54:31.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.108+0000 7f7639133640 1 --2- 172.21.15.152:0/4184470889 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x557cf43a4600 0x557cf43a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:31.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.108+0000 7f761cff9640 10 client.15915 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:54:31.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.109+0000 7f7639133640 1 --2- 172.21.15.152:0/4184470889 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x557cf43a4600 0x557cf43a69f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7cd87c425e2bbb07 server_cookie=c53d452d5e1f60d0 in_seq=0 out_seq=0 2024-09-11T22:54:31.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.109+0000 7f761cff9640 1 -- 172.21.15.152:0/4184470889 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557cf428bd70 con 0x557cf43a4600 2024-09-11T22:54:31.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.109+0000 7f761cff9640 10 client.15915 handle_command_reply: tid=0 2024-09-11T22:54:31.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.109+0000 7f763d770740 1 client.15915 shutdown 2024-09-11T22:54:31.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.109+0000 7f763d770740 1 -- 172.21.15.152:0/4184470889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431e5d0 msgr2=0x557cf432d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:31.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.109+0000 7f763d770740 1 --2- 172.21.15.152:0/4184470889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431e5d0 0x557cf432d430 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7624017da0 tx=0x7f762402f040 comp rx=0 tx=0).stop 2024-09-11T22:54:31.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.109+0000 7f763d770740 1 -- 172.21.15.152:0/4184470889 shutdown_connections 2024-09-11T22:54:31.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.109+0000 7f763d770740 1 --2- 172.21.15.152:0/4184470889 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x557cf43a4600 0x557cf43a69f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.109+0000 7f763d770740 1 --2- 172.21.15.152:0/4184470889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557cf4329370 0x557cf432deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.109+0000 7f763d770740 1 --2- 172.21.15.152:0/4184470889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557cf431fb40 0x557cf432d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.109+0000 7f763d770740 1 --2- 172.21.15.152:0/4184470889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557cf431e5d0 0x557cf432d430 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.109+0000 7f763d770740 1 -- 172.21.15.152:0/4184470889 >> 172.21.15.152:0/4184470889 conn(0x557cf42e7310 msgr2=0x557cf43280e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:31.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.109+0000 7f761cff9640 0 client.15915 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:54:31.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.109+0000 7f763d770740 1 -- 172.21.15.152:0/4184470889 shutdown_connections 2024-09-11T22:54:31.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.109+0000 7f763d770740 1 -- 172.21.15.152:0/4184470889 wait complete. 2024-09-11T22:54:31.123 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.109+0000 7f763d770740 20 client.15915 trim_cache size 0 max 16384 2024-09-11T22:54:31.124 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:54:31.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.110+0000 7f763b397640 1 -- 172.21.15.152:0/927561292 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f76200a2cf0 msgr2=0x7f76200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:31.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.110+0000 7f763b397640 1 --2- 172.21.15.152:0/927561292 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f76200a2cf0 0x7f76200a51b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f762c09b7e0 tx=0x7f762c06b940 comp rx=0 tx=0).stop 2024-09-11T22:54:31.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.110+0000 7f763b397640 1 -- 172.21.15.152:0/927561292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7634106a80 msgr2=0x7f76340fd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:31.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.110+0000 7f763b397640 1 --2- 172.21.15.152:0/927561292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7634106a80 0x7f76340fd830 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f762c00cad0 tx=0x7f762c00cf90 comp rx=0 tx=0).stop 2024-09-11T22:54:31.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.110+0000 7f763b397640 1 -- 172.21.15.152:0/927561292 shutdown_connections 2024-09-11T22:54:31.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.110+0000 7f763b397640 1 --2- 172.21.15.152:0/927561292 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f76200a2cf0 0x7f76200a51b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.110+0000 7f763b397640 1 --2- 172.21.15.152:0/927561292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7634106a80 0x7f76340fd830 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.110+0000 7f763b397640 1 --2- 172.21.15.152:0/927561292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7634101420 0x7f76340fd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.110+0000 7f763b397640 1 --2- 172.21.15.152:0/927561292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7634100a80 0x7f76340fcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:31.125 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.110+0000 7f763b397640 1 -- 172.21.15.152:0/927561292 >> 172.21.15.152:0/927561292 conn(0x7f76340fc7a0 msgr2=0x7f763406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:31.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.110+0000 7f763b397640 1 -- 172.21.15.152:0/927561292 shutdown_connections 2024-09-11T22:54:31.126 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:31.110+0000 7f763b397640 1 -- 172.21.15.152:0/927561292 wait complete. 2024-09-11T22:54:32.205 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:54:32.455 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:54:32.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:32 smithi152 ceph-mon[31181]: pgmap v52: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-09-11T22:54:32.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.956+0000 7f404d5fb640 1 -- 172.21.15.152:0/3868930633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40481008e0 msgr2=0x7f4048100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:32.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.956+0000 7f404d5fb640 1 --2- 172.21.15.152:0/3868930633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40481008e0 0x7f4048100cc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4038009900 tx=0x7f403802a410 comp rx=0 tx=0).stop 2024-09-11T22:54:32.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.956+0000 7f404d5fb640 1 -- 172.21.15.152:0/3868930633 shutdown_connections 2024-09-11T22:54:32.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.956+0000 7f404d5fb640 1 --2- 172.21.15.152:0/3868930633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4048108110 0x7f404810ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.956+0000 7f404d5fb640 1 --2- 172.21.15.152:0/3868930633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4048101290 0x7f40481079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.956+0000 7f404d5fb640 1 --2- 172.21.15.152:0/3868930633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40481008e0 0x7f4048100cc0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.956+0000 7f404d5fb640 1 -- 172.21.15.152:0/3868930633 >> 172.21.15.152:0/3868930633 conn(0x7f4048077210 msgr2=0x7f4048077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:32.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.956+0000 7f404d5fb640 1 -- 172.21.15.152:0/3868930633 shutdown_connections 2024-09-11T22:54:32.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.956+0000 7f404d5fb640 1 -- 172.21.15.152:0/3868930633 wait complete. 2024-09-11T22:54:32.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.957+0000 7f404d5fb640 1 Processor -- start 2024-09-11T22:54:32.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.957+0000 7f404d5fb640 1 -- start start 2024-09-11T22:54:32.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.957+0000 7f404d5fb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40481008e0 0x7f4048198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.957+0000 7f404d5fb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4048101290 0x7f4048198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.957+0000 7f404d5fb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4048108110 0x7f404819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.957+0000 7f404d5fb640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4048111fb0 con 0x7f40481008e0 2024-09-11T22:54:32.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.957+0000 7f404d5fb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4048111e30 con 0x7f4048108110 2024-09-11T22:54:32.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.957+0000 7f404d5fb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4048112130 con 0x7f4048101290 2024-09-11T22:54:32.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.957+0000 7f4046ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40481008e0 0x7f4048198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.957+0000 7f4046ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40481008e0 0x7f4048198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49108/0 (socket says 172.21.15.152:49108) 2024-09-11T22:54:32.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.957+0000 7f4046ffd640 1 -- 172.21.15.152:0/1021724544 learned_addr learned my addr 172.21.15.152:0/1021724544 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:32.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.957+0000 7f403ffff640 1 --2- 172.21.15.152:0/1021724544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4048101290 0x7f4048198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.957+0000 7f40477fe640 1 --2- 172.21.15.152:0/1021724544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4048108110 0x7f404819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.958+0000 7f4046ffd640 1 -- 172.21.15.152:0/1021724544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4048101290 msgr2=0x7f4048198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:32.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.958+0000 7f4046ffd640 1 --2- 172.21.15.152:0/1021724544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4048101290 0x7f4048198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.958+0000 7f4046ffd640 1 -- 172.21.15.152:0/1021724544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4048108110 msgr2=0x7f404819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:32.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.958+0000 7f4046ffd640 1 --2- 172.21.15.152:0/1021724544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4048108110 0x7f404819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.958+0000 7f4046ffd640 1 -- 172.21.15.152:0/1021724544 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404819d3f0 con 0x7f40481008e0 2024-09-11T22:54:32.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.958+0000 7f4046ffd640 1 --2- 172.21.15.152:0/1021724544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40481008e0 0x7f4048198440 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4038006010 tx=0x7f40380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:32.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.958+0000 7f4044ff9640 1 -- 172.21.15.152:0/1021724544 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403803f030 con 0x7f40481008e0 2024-09-11T22:54:32.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.958+0000 7f4044ff9640 1 -- 172.21.15.152:0/1021724544 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4038003ce0 con 0x7f40481008e0 2024-09-11T22:54:32.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.958+0000 7f404d5fb640 1 -- 172.21.15.152:0/1021724544 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404819d680 con 0x7f40481008e0 2024-09-11T22:54:32.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.958+0000 7f4044ff9640 1 -- 172.21.15.152:0/1021724544 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038035070 con 0x7f40481008e0 2024-09-11T22:54:32.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.958+0000 7f404d5fb640 1 -- 172.21.15.152:0/1021724544 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40481a4fc0 con 0x7f40481008e0 2024-09-11T22:54:32.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.960+0000 7f4044ff9640 1 -- 172.21.15.152:0/1021724544 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4038042030 con 0x7f40481008e0 2024-09-11T22:54:32.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.961+0000 7f404f9d4740 1 Processor -- start 2024-09-11T22:54:32.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.961+0000 7f404f9d4740 1 -- start start 2024-09-11T22:54:32.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.961+0000 7f404f9d4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e6b40 0x5652930e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.961+0000 7f404f9d4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930e55d0 0x5652930e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.961+0000 7f404f9d4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930f0370 0x5652930f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.961+0000 7f404f9d4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56529304b2f0 con 0x5652930e55d0 2024-09-11T22:54:32.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.961+0000 7f404f9d4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x565292ff2cf0 con 0x5652930e6b40 2024-09-11T22:54:32.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.961+0000 7f404f9d4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5652930472b0 con 0x5652930f0370 2024-09-11T22:54:32.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.961+0000 7f4044ff9640 1 --2- 172.21.15.152:0/1021724544 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f40340a2cf0 0x7f40340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.961+0000 7f4046ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930e55d0 0x5652930e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.961+0000 7f4046ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930e55d0 0x5652930e5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49124/0 (socket says 172.21.15.152:49124) 2024-09-11T22:54:32.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.961+0000 7f4046ffd640 1 -- 172.21.15.152:0/2405559649 learned_addr learned my addr 172.21.15.152:0/2405559649 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:32.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.961+0000 7f40477fe640 1 --2- 172.21.15.152:0/1021724544 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f40340a2cf0 0x7f40340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.961+0000 7f40477fe640 1 --2- 172.21.15.152:0/2405559649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e6b40 0x5652930e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.961+0000 7f403ffff640 1 --2- 172.21.15.152:0/2405559649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930f0370 0x5652930f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.961+0000 7f4044ff9640 1 -- 172.21.15.152:0/1021724544 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f40380efa10 con 0x7f40481008e0 2024-09-11T22:54:32.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.962+0000 7f4046ffd640 1 -- 172.21.15.152:0/2405559649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930f0370 msgr2=0x5652930f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:32.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.962+0000 7f4046ffd640 1 --2- 172.21.15.152:0/2405559649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930f0370 0x5652930f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.962+0000 7f4046ffd640 1 -- 172.21.15.152:0/2405559649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e6b40 msgr2=0x5652930e5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:32.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.962+0000 7f4046ffd640 1 --2- 172.21.15.152:0/2405559649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e6b40 0x5652930e5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.962+0000 7f4046ffd640 1 -- 172.21.15.152:0/2405559649 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565292fd73a0 con 0x5652930e55d0 2024-09-11T22:54:32.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.962+0000 7f403ffff640 1 --2- 172.21.15.152:0/2405559649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930f0370 0x5652930f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:54:32.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.962+0000 7f4046ffd640 1 --2- 172.21.15.152:0/2405559649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930e55d0 0x5652930e5a10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4038000c00 tx=0x7f403802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0842c9f4915cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:32.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.962+0000 7f40477fe640 1 --2- 172.21.15.152:0/1021724544 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f40340a2cf0 0x7f40340a51b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4040005fd0 tx=0x7f4040004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:32.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.962+0000 7f403d7fa640 1 -- 172.21.15.152:0/2405559649 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038049030 con 0x5652930e55d0 2024-09-11T22:54:32.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.962+0000 7f403d7fa640 1 -- 172.21.15.152:0/2405559649 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f403803b580 con 0x5652930e55d0 2024-09-11T22:54:32.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.962+0000 7f403d7fa640 1 -- 172.21.15.152:0/2405559649 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40380c03f0 con 0x5652930e55d0 2024-09-11T22:54:32.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.963+0000 7f404f9d4740 1 -- 172.21.15.152:0/2405559649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930e55d0 msgr2=0x5652930e5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:32.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.963+0000 7f404f9d4740 1 --2- 172.21.15.152:0/2405559649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930e55d0 0x5652930e5a10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4038000c00 tx=0x7f403802a410 comp rx=0 tx=0).stop 2024-09-11T22:54:32.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.963+0000 7f404f9d4740 1 -- 172.21.15.152:0/2405559649 shutdown_connections 2024-09-11T22:54:32.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.963+0000 7f404f9d4740 1 --2- 172.21.15.152:0/2405559649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930f0370 0x5652930f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.963+0000 7f404f9d4740 1 --2- 172.21.15.152:0/2405559649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930e55d0 0x5652930e5a10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.963+0000 7f404f9d4740 1 --2- 172.21.15.152:0/2405559649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e6b40 0x5652930e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.963+0000 7f404f9d4740 1 -- 172.21.15.152:0/2405559649 >> 172.21.15.152:0/2405559649 conn(0x5652930ae310 msgr2=0x5652930e1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:32.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.963+0000 7f404f9d4740 1 -- 172.21.15.152:0/2405559649 shutdown_connections 2024-09-11T22:54:32.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.963+0000 7f404f9d4740 1 -- 172.21.15.152:0/2405559649 wait complete. 2024-09-11T22:54:32.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.963+0000 7f404f9d4740 1 -- start start 2024-09-11T22:54:32.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.963+0000 7f404f9d4740 1 Processor -- start 2024-09-11T22:54:32.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f404f9d4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e55d0 0x5652930efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f404f9d4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930e6b40 0x5652930ed0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f404f9d4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930f0370 0x5652930ed7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f404f9d4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5652930f2e30 con 0x5652930e6b40 2024-09-11T22:54:32.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f404f9d4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x565293006060 con 0x5652930e55d0 2024-09-11T22:54:32.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f404f9d4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5652930f2fb0 con 0x5652930f0370 2024-09-11T22:54:32.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f404f9d4740 10 client.0 _ll_register_callbacks cb 0x565292d598c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:32.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f403ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930e6b40 0x5652930ed0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f403ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930e6b40 0x5652930ed0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49132/0 (socket says 172.21.15.152:49132) 2024-09-11T22:54:32.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f403ffff640 1 -- 172.21.15.152:0/2510786632 learned_addr learned my addr 172.21.15.152:0/2510786632 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:32.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f4046ffd640 1 --2- 172.21.15.152:0/2510786632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e55d0 0x5652930efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f40477fe640 1 --2- 172.21.15.152:0/2510786632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930f0370 0x5652930ed7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f403ffff640 1 -- 172.21.15.152:0/2510786632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930f0370 msgr2=0x5652930ed7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:32.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f403ffff640 1 --2- 172.21.15.152:0/2510786632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930f0370 0x5652930ed7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f403ffff640 1 -- 172.21.15.152:0/2510786632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e55d0 msgr2=0x5652930efdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:32.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f403ffff640 1 --2- 172.21.15.152:0/2510786632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e55d0 0x5652930efdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f403ffff640 1 -- 172.21.15.152:0/2510786632 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565292dbc3c0 con 0x5652930e6b40 2024-09-11T22:54:32.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f40477fe640 1 --2- 172.21.15.152:0/2510786632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930f0370 0x5652930ed7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:54:32.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f4046ffd640 1 --2- 172.21.15.152:0/2510786632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e55d0 0x5652930efdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:54:32.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f403ffff640 1 --2- 172.21.15.152:0/2510786632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930e6b40 0x5652930ed0a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f403000b9a0 tx=0x7f403000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500d3345650ebb4d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:32.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f402e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:32.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f402e7fc640 1 -- 172.21.15.152:0/2510786632 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403001b440 con 0x5652930e6b40 2024-09-11T22:54:32.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f404f9d4740 1 -- 172.21.15.152:0/2510786632 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565293058860 con 0x5652930e6b40 2024-09-11T22:54:32.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f402e7fc640 1 -- 172.21.15.152:0/2510786632 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f403001b5e0 con 0x5652930e6b40 2024-09-11T22:54:32.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f402e7fc640 1 -- 172.21.15.152:0/2510786632 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f403001b7e0 con 0x5652930e6b40 2024-09-11T22:54:32.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f404f9d4740 10 client.15927 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:32.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.964+0000 7f404f9d4740 1 -- 172.21.15.152:0/2510786632 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5652930fe8b0 con 0x5652930e6b40 2024-09-11T22:54:32.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.965+0000 7f402e7fc640 1 -- 172.21.15.152:0/2510786632 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x565293058860 con 0x5652930e6b40 2024-09-11T22:54:32.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.965+0000 7f402e7fc640 1 -- 172.21.15.152:0/2510786632 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4030019020 con 0x5652930e6b40 2024-09-11T22:54:32.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.965+0000 7f404f9d4740 10 client.15927 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:32.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.965+0000 7f404f9d4740 10 client.15927 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:54:32.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.965+0000 7f404f9d4740 20 client.15927 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:32.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.965+0000 7f404f9d4740 1 --2- 172.21.15.152:0/2510786632 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5652930ffcc0 0x5652931020f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.965+0000 7f404f9d4740 4 client.15927 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:54:32.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.965+0000 7f404f9d4740 1 -- 172.21.15.152:0/2510786632 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565293052d70 con 0x5652930ffcc0 2024-09-11T22:54:32.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.965+0000 7f4046ffd640 1 --2- 172.21.15.152:0/2510786632 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5652930ffcc0 0x5652931020f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.966+0000 7f4046ffd640 1 --2- 172.21.15.152:0/2510786632 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5652930ffcc0 0x5652931020f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a043636bcf5d7127 server_cookie=e8d7a33626dd9095 in_seq=0 out_seq=0 2024-09-11T22:54:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.966+0000 7f402e7fc640 10 client.15927 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:54:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.966+0000 7f402e7fc640 1 -- 172.21.15.152:0/2510786632 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565293052d70 con 0x5652930ffcc0 2024-09-11T22:54:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.966+0000 7f402e7fc640 10 client.15927 handle_command_reply: tid=0 2024-09-11T22:54:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.966+0000 7f404f9d4740 1 client.15927 shutdown 2024-09-11T22:54:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.967+0000 7f404f9d4740 1 -- 172.21.15.152:0/2510786632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930e6b40 msgr2=0x5652930ed0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.967+0000 7f404f9d4740 1 --2- 172.21.15.152:0/2510786632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930e6b40 0x5652930ed0a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f403000b9a0 tx=0x7f403000be60 comp rx=0 tx=0).stop 2024-09-11T22:54:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.967+0000 7f404f9d4740 1 -- 172.21.15.152:0/2510786632 shutdown_connections 2024-09-11T22:54:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.967+0000 7f404f9d4740 1 --2- 172.21.15.152:0/2510786632 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5652930ffcc0 0x5652931020f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.967+0000 7f404f9d4740 1 --2- 172.21.15.152:0/2510786632 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930f0370 0x5652930ed7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.967+0000 7f404f9d4740 1 --2- 172.21.15.152:0/2510786632 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930e6b40 0x5652930ed0a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.967+0000 7f404f9d4740 1 --2- 172.21.15.152:0/2510786632 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e55d0 0x5652930efdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.967+0000 7f402e7fc640 0 client.15927 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:54:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.967+0000 7f404f9d4740 1 -- 172.21.15.152:0/2510786632 >> 172.21.15.152:0/2510786632 conn(0x5652930ae310 msgr2=0x5652930e7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:32.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.967+0000 7f404f9d4740 1 -- 172.21.15.152:0/2510786632 shutdown_connections 2024-09-11T22:54:32.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.967+0000 7f404f9d4740 1 -- 172.21.15.152:0/2510786632 wait complete. 2024-09-11T22:54:32.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.967+0000 7f404f9d4740 20 client.15927 trim_cache size 0 max 16384 2024-09-11T22:54:32.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.979+0000 7f404f9d4740 1 Processor -- start 2024-09-11T22:54:32.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.979+0000 7f404f9d4740 1 -- start start 2024-09-11T22:54:32.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.980+0000 7f404f9d4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e55d0 0x5652930e1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.980+0000 7f404f9d4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930e6b40 0x5652930fbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.980+0000 7f404f9d4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930f0370 0x5652930f1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.980+0000 7f404f9d4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f40481ad780 con 0x5652930f0370 2024-09-11T22:54:32.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.980+0000 7f404f9d4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f40481af2c0 con 0x5652930e55d0 2024-09-11T22:54:32.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.980+0000 7f404f9d4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f40481ade50 con 0x5652930e6b40 2024-09-11T22:54:32.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.980+0000 7f40477fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930f0370 0x5652930f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.980+0000 7f40477fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930f0370 0x5652930f1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49136/0 (socket says 172.21.15.152:49136) 2024-09-11T22:54:32.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.980+0000 7f40477fe640 1 -- 172.21.15.152:0/3273047859 learned_addr learned my addr 172.21.15.152:0/3273047859 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:32.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.980+0000 7f403ffff640 1 --2- 172.21.15.152:0/3273047859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930e6b40 0x5652930fbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.980+0000 7f4046ffd640 1 --2- 172.21.15.152:0/3273047859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e55d0 0x5652930e1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.980+0000 7f40477fe640 1 -- 172.21.15.152:0/3273047859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930e6b40 msgr2=0x5652930fbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.980+0000 7f40477fe640 1 --2- 172.21.15.152:0/3273047859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930e6b40 0x5652930fbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.980+0000 7f40477fe640 1 -- 172.21.15.152:0/3273047859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e55d0 msgr2=0x5652930e1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.980+0000 7f40477fe640 1 --2- 172.21.15.152:0/3273047859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e55d0 0x5652930e1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.980+0000 7f40477fe640 1 -- 172.21.15.152:0/3273047859 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652930fc4f0 con 0x5652930f0370 2024-09-11T22:54:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.981+0000 7f40477fe640 1 --2- 172.21.15.152:0/3273047859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930f0370 0x5652930f1760 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4040012940 tx=0x7f4040017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208a8575ca565924 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.981+0000 7f403cff9640 1 -- 172.21.15.152:0/3273047859 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404000fd20 con 0x5652930f0370 2024-09-11T22:54:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.981+0000 7f403cff9640 1 -- 172.21.15.152:0/3273047859 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f404001f900 con 0x5652930f0370 2024-09-11T22:54:32.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.981+0000 7f403cff9640 1 -- 172.21.15.152:0/3273047859 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040005210 con 0x5652930f0370 2024-09-11T22:54:32.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.981+0000 7f404f9d4740 1 -- 172.21.15.152:0/3273047859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930f0370 msgr2=0x5652930f1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:32.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.981+0000 7f404f9d4740 1 --2- 172.21.15.152:0/3273047859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930f0370 0x5652930f1760 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4040012940 tx=0x7f4040017000 comp rx=0 tx=0).stop 2024-09-11T22:54:32.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.981+0000 7f404f9d4740 1 -- 172.21.15.152:0/3273047859 shutdown_connections 2024-09-11T22:54:32.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.981+0000 7f404f9d4740 1 --2- 172.21.15.152:0/3273047859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930f0370 0x5652930f1760 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.981+0000 7f404f9d4740 1 --2- 172.21.15.152:0/3273047859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930e6b40 0x5652930fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.981+0000 7f404f9d4740 1 --2- 172.21.15.152:0/3273047859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e55d0 0x5652930e1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.981+0000 7f404f9d4740 1 -- 172.21.15.152:0/3273047859 >> 172.21.15.152:0/3273047859 conn(0x5652930ae310 msgr2=0x5652930e1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:32.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.981+0000 7f404f9d4740 1 -- 172.21.15.152:0/3273047859 shutdown_connections 2024-09-11T22:54:32.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.982+0000 7f404f9d4740 1 -- 172.21.15.152:0/3273047859 wait complete. 2024-09-11T22:54:32.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.982+0000 7f404f9d4740 1 -- start start 2024-09-11T22:54:32.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.982+0000 7f404f9d4740 1 Processor -- start 2024-09-11T22:54:32.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.982+0000 7f404f9d4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e55d0 0x5652930f8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.982+0000 7f404f9d4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930e6b40 0x5652930f8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.982+0000 7f404f9d4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930f0370 0x5652930ea320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.982+0000 7f404f9d4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f40481aebf0 con 0x5652930f0370 2024-09-11T22:54:32.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.982+0000 7f404f9d4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f40481ae520 con 0x5652930e55d0 2024-09-11T22:54:32.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.982+0000 7f404f9d4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f40481ab760 con 0x5652930e6b40 2024-09-11T22:54:32.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.982+0000 7f404f9d4740 10 client.0 _ll_register_callbacks cb 0x7f4030004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:32.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.982+0000 7f40477fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930f0370 0x5652930ea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.982+0000 7f40477fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930f0370 0x5652930ea320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49142/0 (socket says 172.21.15.152:49142) 2024-09-11T22:54:32.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.982+0000 7f40477fe640 1 -- 172.21.15.152:0/3324837362 learned_addr learned my addr 172.21.15.152:0/3324837362 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:32.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.982+0000 7f403ffff640 1 --2- 172.21.15.152:0/3324837362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930e6b40 0x5652930f8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.982+0000 7f4046ffd640 1 --2- 172.21.15.152:0/3324837362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e55d0 0x5652930f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.983+0000 7f40477fe640 1 -- 172.21.15.152:0/3324837362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930e6b40 msgr2=0x5652930f8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:32.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.983+0000 7f40477fe640 1 --2- 172.21.15.152:0/3324837362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930e6b40 0x5652930f8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.983+0000 7f40477fe640 1 -- 172.21.15.152:0/3324837362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e55d0 msgr2=0x5652930f8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:32.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.983+0000 7f40477fe640 1 --2- 172.21.15.152:0/3324837362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e55d0 0x5652930f8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.983+0000 7f40477fe640 1 -- 172.21.15.152:0/3324837362 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652930eab50 con 0x5652930f0370 2024-09-11T22:54:32.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.983+0000 7f40477fe640 1 --2- 172.21.15.152:0/3324837362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930f0370 0x5652930ea320 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f404001f7d0 tx=0x7f404000fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc638dd5e689a6e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:32.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.983+0000 7f402e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:32.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.983+0000 7f402e7fc640 1 -- 172.21.15.152:0/3324837362 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040004ab0 con 0x5652930f0370 2024-09-11T22:54:32.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.983+0000 7f402e7fc640 1 -- 172.21.15.152:0/3324837362 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4040025040 con 0x5652930f0370 2024-09-11T22:54:32.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.983+0000 7f404f9d4740 1 -- 172.21.15.152:0/3324837362 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f404001f900 con 0x5652930f0370 2024-09-11T22:54:32.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.983+0000 7f402e7fc640 1 -- 172.21.15.152:0/3324837362 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4040004c50 con 0x5652930f0370 2024-09-11T22:54:32.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.983+0000 7f404f9d4740 10 client.15933 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:32.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.983+0000 7f404f9d4740 1 -- 172.21.15.152:0/3324837362 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5652930f1830 con 0x5652930f0370 2024-09-11T22:54:32.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.983+0000 7f402e7fc640 1 -- 172.21.15.152:0/3324837362 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f404001f900 con 0x5652930f0370 2024-09-11T22:54:32.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.984+0000 7f402e7fc640 1 -- 172.21.15.152:0/3324837362 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f404000f040 con 0x5652930f0370 2024-09-11T22:54:32.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.984+0000 7f404f9d4740 10 client.15933 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:32.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.984+0000 7f404f9d4740 10 client.15933 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:54:32.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.984+0000 7f404f9d4740 20 client.15933 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:32.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.984+0000 7f404f9d4740 1 --2- 172.21.15.152:0/3324837362 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x56529316ac80 0x56529316d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:32.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.984+0000 7f404f9d4740 4 client.15933 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:54:32.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.984+0000 7f404f9d4740 1 -- 172.21.15.152:0/3324837362 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565292fcf170 con 0x56529316ac80 2024-09-11T22:54:32.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.984+0000 7f4046ffd640 1 --2- 172.21.15.152:0/3324837362 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x56529316ac80 0x56529316d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:32.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.984+0000 7f4046ffd640 1 --2- 172.21.15.152:0/3324837362 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x56529316ac80 0x56529316d0a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7dbb36d6a52fa85d server_cookie=6fedb091fc117b42 in_seq=0 out_seq=0 2024-09-11T22:54:32.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.984+0000 7f402e7fc640 10 client.15933 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:54:32.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.984+0000 7f402e7fc640 1 -- 172.21.15.152:0/3324837362 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565292fcf170 con 0x56529316ac80 2024-09-11T22:54:32.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.984+0000 7f402e7fc640 10 client.15933 handle_command_reply: tid=0 2024-09-11T22:54:32.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.984+0000 7f404f9d4740 1 client.15933 shutdown 2024-09-11T22:54:32.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.984+0000 7f404f9d4740 1 -- 172.21.15.152:0/3324837362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930f0370 msgr2=0x5652930ea320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:32.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.984+0000 7f404f9d4740 1 --2- 172.21.15.152:0/3324837362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930f0370 0x5652930ea320 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f404001f7d0 tx=0x7f404000fef0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404f9d4740 1 -- 172.21.15.152:0/3324837362 shutdown_connections 2024-09-11T22:54:32.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404f9d4740 1 --2- 172.21.15.152:0/3324837362 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x56529316ac80 0x56529316d0a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404f9d4740 1 --2- 172.21.15.152:0/3324837362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5652930f0370 0x5652930ea320 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404f9d4740 1 --2- 172.21.15.152:0/3324837362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5652930e6b40 0x5652930f8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404f9d4740 1 --2- 172.21.15.152:0/3324837362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5652930e55d0 0x5652930f8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f402e7fc640 0 client.15933 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:54:32.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404f9d4740 1 -- 172.21.15.152:0/3324837362 >> 172.21.15.152:0/3324837362 conn(0x5652930ae310 msgr2=0x5652930ec190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:32.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404f9d4740 1 -- 172.21.15.152:0/3324837362 shutdown_connections 2024-09-11T22:54:32.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404f9d4740 1 -- 172.21.15.152:0/3324837362 wait complete. 2024-09-11T22:54:32.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404f9d4740 20 client.15933 trim_cache size 0 max 16384 2024-09-11T22:54:32.993 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:54:32.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404d5fb640 1 -- 172.21.15.152:0/1021724544 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f40340a2cf0 msgr2=0x7f40340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:32.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404d5fb640 1 --2- 172.21.15.152:0/1021724544 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f40340a2cf0 0x7f40340a51b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4040005fd0 tx=0x7f4040004a40 comp rx=0 tx=0).stop 2024-09-11T22:54:32.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404d5fb640 1 -- 172.21.15.152:0/1021724544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40481008e0 msgr2=0x7f4048198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:32.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404d5fb640 1 --2- 172.21.15.152:0/1021724544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40481008e0 0x7f4048198440 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4038006010 tx=0x7f40380042f0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404d5fb640 1 -- 172.21.15.152:0/1021724544 shutdown_connections 2024-09-11T22:54:32.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404d5fb640 1 --2- 172.21.15.152:0/1021724544 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f40340a2cf0 0x7f40340a51b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404d5fb640 1 --2- 172.21.15.152:0/1021724544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4048108110 0x7f404819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404d5fb640 1 --2- 172.21.15.152:0/1021724544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4048101290 0x7f4048198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404d5fb640 1 --2- 172.21.15.152:0/1021724544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40481008e0 0x7f4048198440 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:32.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404d5fb640 1 -- 172.21.15.152:0/1021724544 >> 172.21.15.152:0/1021724544 conn(0x7f4048077210 msgr2=0x7f4048105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:32.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404d5fb640 1 -- 172.21.15.152:0/1021724544 shutdown_connections 2024-09-11T22:54:32.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:32.985+0000 7f404d5fb640 1 -- 172.21.15.152:0/1021724544 wait complete. 2024-09-11T22:54:33.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:32 smithi191 ceph-mon[35211]: pgmap v52: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-09-11T22:54:33.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:32 smithi204 ceph-mon[35454]: pgmap v52: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-09-11T22:54:33.777 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:54:34.022 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:34.023 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:34.023 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:34.023 INFO:tasks.workunit.client.0.smithi152.stdout:iozone test complete. 2024-09-11T22:54:34.023 INFO:tasks.workunit.client.0.smithi152.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-11T22:54:34.029 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:54:34.089 INFO:tasks.workunit.client.0.smithi152.stdout: Iozone: Performance Test of File I/O 2024-09-11T22:54:34.090 INFO:tasks.workunit.client.0.smithi152.stdout: Version $Revision: 3.494 $ 2024-09-11T22:54:34.090 INFO:tasks.workunit.client.0.smithi152.stdout: Compiled for 64 bit mode. 2024-09-11T22:54:34.090 INFO:tasks.workunit.client.0.smithi152.stdout: Build: linux-AMD64 2024-09-11T22:54:34.090 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:34.090 INFO:tasks.workunit.client.0.smithi152.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-11T22:54:34.090 INFO:tasks.workunit.client.0.smithi152.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-11T22:54:34.090 INFO:tasks.workunit.client.0.smithi152.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-11T22:54:34.090 INFO:tasks.workunit.client.0.smithi152.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-11T22:54:34.090 INFO:tasks.workunit.client.0.smithi152.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-11T22:54:34.090 INFO:tasks.workunit.client.0.smithi152.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-11T22:54:34.090 INFO:tasks.workunit.client.0.smithi152.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-11T22:54:34.091 INFO:tasks.workunit.client.0.smithi152.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-11T22:54:34.091 INFO:tasks.workunit.client.0.smithi152.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-11T22:54:34.091 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:34.091 INFO:tasks.workunit.client.0.smithi152.stdout: Run began: Wed Sep 11 22:54:34 2024 2024-09-11T22:54:34.091 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:34.091 INFO:tasks.workunit.client.0.smithi152.stdout: Include close in write timing 2024-09-11T22:54:34.091 INFO:tasks.workunit.client.0.smithi152.stdout: Include fsync in write timing 2024-09-11T22:54:34.091 INFO:tasks.workunit.client.0.smithi152.stdout: File size set to 1048576 kB 2024-09-11T22:54:34.091 INFO:tasks.workunit.client.0.smithi152.stdout: Record Size 1024 kB 2024-09-11T22:54:34.091 INFO:tasks.workunit.client.0.smithi152.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-09-11T22:54:34.092 INFO:tasks.workunit.client.0.smithi152.stdout: Output is in kBytes/sec 2024-09-11T22:54:34.092 INFO:tasks.workunit.client.0.smithi152.stdout: Time Resolution = 0.000001 seconds. 2024-09-11T22:54:34.092 INFO:tasks.workunit.client.0.smithi152.stdout: Processor cache size set to 1024 kBytes. 2024-09-11T22:54:34.092 INFO:tasks.workunit.client.0.smithi152.stdout: Processor cache line size set to 32 bytes. 2024-09-11T22:54:34.092 INFO:tasks.workunit.client.0.smithi152.stdout: File stride size set to 17 * record size. 2024-09-11T22:54:34.092 INFO:tasks.workunit.client.0.smithi152.stdout: Throughput test with 1 process 2024-09-11T22:54:34.092 INFO:tasks.workunit.client.0.smithi152.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-09-11T22:54:34.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.771+0000 7f0913e16640 1 -- 172.21.15.152:0/2473962283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f090c1091f0 msgr2=0x7f090c1095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:34.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.772+0000 7f0913e16640 1 --2- 172.21.15.152:0/2473962283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f090c1091f0 0x7f090c1095d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0908009860 tx=0x7f090802a3f0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.776 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:34 smithi152 ceph-mon[31181]: pgmap v53: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-09-11T22:54:34.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.778+0000 7f0913e16640 1 -- 172.21.15.152:0/2473962283 shutdown_connections 2024-09-11T22:54:34.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.778+0000 7f0913e16640 1 --2- 172.21.15.152:0/2473962283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f090c114ab0 0x7f090c116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.778+0000 7f0913e16640 1 --2- 172.21.15.152:0/2473962283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f090c109b10 0x7f090c114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.778+0000 7f0913e16640 1 --2- 172.21.15.152:0/2473962283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f090c1091f0 0x7f090c1095d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.778+0000 7f0913e16640 1 -- 172.21.15.152:0/2473962283 >> 172.21.15.152:0/2473962283 conn(0x7f090c06e140 msgr2=0x7f090c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:34.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.780+0000 7f0913e16640 1 -- 172.21.15.152:0/2473962283 shutdown_connections 2024-09-11T22:54:34.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.781+0000 7f0913e16640 1 -- 172.21.15.152:0/2473962283 wait complete. 2024-09-11T22:54:34.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.783+0000 7f0913e16640 1 Processor -- start 2024-09-11T22:54:34.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.783+0000 7f0913e16640 1 -- start start 2024-09-11T22:54:34.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.784+0000 7f0913e16640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f090c1091f0 0x7f090c1a6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.784+0000 7f0913e16640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f090c109b10 0x7f090c1a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.784+0000 7f0913e16640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f090c114ab0 0x7f090c1a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.784+0000 7f0913e16640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f090c11a850 con 0x7f090c1091f0 2024-09-11T22:54:34.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.784+0000 7f0913e16640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f090c11a6d0 con 0x7f090c109b10 2024-09-11T22:54:34.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.784+0000 7f0913e16640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f090c11a9d0 con 0x7f090c114ab0 2024-09-11T22:54:34.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.786+0000 7f09113b1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f090c109b10 0x7f090c1a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:34.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.786+0000 7f09113b1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f090c109b10 0x7f090c1a6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:43120/0 (socket says 172.21.15.152:43120) 2024-09-11T22:54:34.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.786+0000 7f09113b1640 1 -- 172.21.15.152:0/4127793609 learned_addr learned my addr 172.21.15.152:0/4127793609 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:34.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.787+0000 7f0911bb2640 1 --2- 172.21.15.152:0/4127793609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f090c1091f0 0x7f090c1a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:34.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.787+0000 7f09123b3640 1 --2- 172.21.15.152:0/4127793609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f090c114ab0 0x7f090c1a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:34.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.788+0000 7f0911bb2640 1 -- 172.21.15.152:0/4127793609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f090c114ab0 msgr2=0x7f090c1a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:34.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.788+0000 7f0911bb2640 1 --2- 172.21.15.152:0/4127793609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f090c114ab0 0x7f090c1a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.788+0000 7f0911bb2640 1 -- 172.21.15.152:0/4127793609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f090c109b10 msgr2=0x7f090c1a6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:34.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.788+0000 7f0911bb2640 1 --2- 172.21.15.152:0/4127793609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f090c109b10 0x7f090c1a6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.788+0000 7f0911bb2640 1 -- 172.21.15.152:0/4127793609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090c1a10f0 con 0x7f090c1091f0 2024-09-11T22:54:34.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.789+0000 7f0911bb2640 1 --2- 172.21.15.152:0/4127793609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f090c1091f0 0x7f090c1a6800 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0908009830 tx=0x7f0908005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:34.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.789+0000 7f0902ffd640 1 -- 172.21.15.152:0/4127793609 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908040070 con 0x7f090c1091f0 2024-09-11T22:54:34.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.790+0000 7f0913e16640 1 -- 172.21.15.152:0/4127793609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090c1a1380 con 0x7f090c1091f0 2024-09-11T22:54:34.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.790+0000 7f0913e16640 1 -- 172.21.15.152:0/4127793609 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f090c1ad6d0 con 0x7f090c1091f0 2024-09-11T22:54:34.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.791+0000 7f0902ffd640 1 -- 172.21.15.152:0/4127793609 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09080318e0 con 0x7f090c1091f0 2024-09-11T22:54:34.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.791+0000 7f0902ffd640 1 -- 172.21.15.152:0/4127793609 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908004920 con 0x7f090c1091f0 2024-09-11T22:54:34.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.797+0000 7f09161ef740 1 Processor -- start 2024-09-11T22:54:34.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.797+0000 7f09161ef740 1 -- start start 2024-09-11T22:54:34.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.798+0000 7f09161ef740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f15b40 0x55cac9f14090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.798+0000 7f09161ef740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f145d0 0x55cac9f14a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.798+0000 7f09161ef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 0x55cac9f21760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.798+0000 7f09161ef740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55cac9e7a2f0 con 0x55cac9f15b40 2024-09-11T22:54:34.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.798+0000 7f09161ef740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55cac9e21cf0 con 0x55cac9f145d0 2024-09-11T22:54:34.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.798+0000 7f09161ef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cac9e762b0 con 0x55cac9f1f370 2024-09-11T22:54:34.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.800+0000 7f09113b1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 0x55cac9f21760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:34.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.802+0000 7f09113b1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 0x55cac9f21760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:50626/0 (socket says 172.21.15.152:50626) 2024-09-11T22:54:34.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.802+0000 7f09113b1640 1 -- 172.21.15.152:0/171873679 learned_addr learned my addr 172.21.15.152:0/171873679 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:34.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.802+0000 7f09123b3640 1 --2- 172.21.15.152:0/171873679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f15b40 0x55cac9f14090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:34.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.804+0000 7f09113b1640 1 -- 172.21.15.152:0/171873679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f145d0 msgr2=0x55cac9f14a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:54:34.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.804+0000 7f09113b1640 1 --2- 172.21.15.152:0/171873679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f145d0 0x55cac9f14a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.804+0000 7f09113b1640 1 -- 172.21.15.152:0/171873679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f15b40 msgr2=0x55cac9f14090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:34.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.804+0000 7f09113b1640 1 --2- 172.21.15.152:0/171873679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f15b40 0x55cac9f14090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.804+0000 7f09113b1640 1 -- 172.21.15.152:0/171873679 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cac9e063a0 con 0x55cac9f1f370 2024-09-11T22:54:34.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.804+0000 7f0902ffd640 1 -- 172.21.15.152:0/4127793609 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0908043070 con 0x7f090c1091f0 2024-09-11T22:54:34.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.805+0000 7f09113b1640 1 --2- 172.21.15.152:0/171873679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 0x55cac9f21760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0045f0 tx=0x7f08fc00bc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d66217102a33ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:34.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.806+0000 7f08f7fff640 1 -- 172.21.15.152:0/171873679 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc003dd0 con 0x55cac9f1f370 2024-09-11T22:54:34.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.806+0000 7f08f7fff640 1 -- 172.21.15.152:0/171873679 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08fc015070 con 0x55cac9f1f370 2024-09-11T22:54:34.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.806+0000 7f08f7fff640 1 -- 172.21.15.152:0/171873679 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc0117c0 con 0x55cac9f1f370 2024-09-11T22:54:34.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.808+0000 7f09161ef740 1 -- 172.21.15.152:0/171873679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 msgr2=0x55cac9f21760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:34.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.808+0000 7f09161ef740 1 --2- 172.21.15.152:0/171873679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 0x55cac9f21760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0045f0 tx=0x7f08fc00bc70 comp rx=0 tx=0).stop 2024-09-11T22:54:34.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.808+0000 7f0902ffd640 1 --2- 172.21.15.152:0/4127793609 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08f80a2e40 0x7f08f80a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.809+0000 7f0902ffd640 1 -- 172.21.15.152:0/4127793609 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f09080f0320 con 0x7f090c1091f0 2024-09-11T22:54:34.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.810+0000 7f09123b3640 1 --2- 172.21.15.152:0/4127793609 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08f80a2e40 0x7f08f80a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:34.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.810+0000 7f09161ef740 1 -- 172.21.15.152:0/171873679 shutdown_connections 2024-09-11T22:54:34.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.810+0000 7f09161ef740 1 --2- 172.21.15.152:0/171873679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 0x55cac9f21760 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.810+0000 7f09161ef740 1 --2- 172.21.15.152:0/171873679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f145d0 0x55cac9f14a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.810+0000 7f09161ef740 1 --2- 172.21.15.152:0/171873679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f15b40 0x55cac9f14090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.810+0000 7f09161ef740 1 -- 172.21.15.152:0/171873679 >> 172.21.15.152:0/171873679 conn(0x55cac9edd310 msgr2=0x55cac9f10c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:34.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.811+0000 7f09161ef740 1 -- 172.21.15.152:0/171873679 shutdown_connections 2024-09-11T22:54:34.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.811+0000 7f09161ef740 1 -- 172.21.15.152:0/171873679 wait complete. 2024-09-11T22:54:34.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.811+0000 7f09161ef740 1 -- start start 2024-09-11T22:54:34.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.812+0000 7f09161ef740 1 Processor -- start 2024-09-11T22:54:34.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.813+0000 7f09123b3640 1 --2- 172.21.15.152:0/4127793609 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08f80a2e40 0x7f08f80a5300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f090c06b430 tx=0x7f0904012040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:34.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.813+0000 7f09161ef740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f145d0 0x55cac9f1edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.813+0000 7f09161ef740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f15b40 0x55cac9f1c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.813+0000 7f09161ef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 0x55cac9f1c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.813+0000 7f09161ef740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55cac9f22070 con 0x55cac9f145d0 2024-09-11T22:54:34.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.813+0000 7f09161ef740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55cac9e35060 con 0x55cac9f15b40 2024-09-11T22:54:34.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.813+0000 7f09161ef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55cac9f221f0 con 0x55cac9f1f370 2024-09-11T22:54:34.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.813+0000 7f09161ef740 10 client.0 _ll_register_callbacks cb 0x55cac9b888c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:34.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.814+0000 7f0911bb2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f145d0 0x55cac9f1edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:34.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.814+0000 7f0911bb2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f145d0 0x55cac9f1edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49180/0 (socket says 172.21.15.152:49180) 2024-09-11T22:54:34.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.814+0000 7f0911bb2640 1 -- 172.21.15.152:0/4183496877 learned_addr learned my addr 172.21.15.152:0/4183496877 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:34.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.814+0000 7f0911bb2640 1 -- 172.21.15.152:0/4183496877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 msgr2=0x55cac9f1c7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:54:34.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.814+0000 7f0911bb2640 1 --2- 172.21.15.152:0/4183496877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 0x55cac9f1c7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.814+0000 7f0911bb2640 1 -- 172.21.15.152:0/4183496877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f15b40 msgr2=0x55cac9f1c060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:54:34.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.814+0000 7f0911bb2640 1 --2- 172.21.15.152:0/4183496877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f15b40 0x55cac9f1c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.814+0000 7f0911bb2640 1 -- 172.21.15.152:0/4183496877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cac9beb3c0 con 0x55cac9f145d0 2024-09-11T22:54:34.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.815+0000 7f0911bb2640 1 --2- 172.21.15.152:0/4183496877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f145d0 0x55cac9f1edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f09080043c0 tx=0x7f09080bc260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebd9df16aa3a416 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:34.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.815+0000 7f08f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:34.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.815+0000 7f08f57fa640 1 -- 172.21.15.152:0/4183496877 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090804a040 con 0x55cac9f145d0 2024-09-11T22:54:34.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.816+0000 7f08f57fa640 1 -- 172.21.15.152:0/4183496877 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f09080bac00 con 0x55cac9f145d0 2024-09-11T22:54:34.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.816+0000 7f09161ef740 1 -- 172.21.15.152:0/4183496877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08fc015070 con 0x55cac9f145d0 2024-09-11T22:54:34.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.816+0000 7f08f57fa640 1 -- 172.21.15.152:0/4183496877 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f09080c54c0 con 0x55cac9f145d0 2024-09-11T22:54:34.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.816+0000 7f09161ef740 10 client.15945 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:34.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.816+0000 7f09161ef740 1 -- 172.21.15.152:0/4183496877 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cac9f36400 con 0x55cac9f145d0 2024-09-11T22:54:34.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.817+0000 7f08f57fa640 1 -- 172.21.15.152:0/4183496877 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08fc015070 con 0x55cac9f145d0 2024-09-11T22:54:34.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.817+0000 7f08f57fa640 1 -- 172.21.15.152:0/4183496877 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f09080bcb60 con 0x55cac9f145d0 2024-09-11T22:54:34.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.817+0000 7f09161ef740 10 client.15945 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:34.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.817+0000 7f09161ef740 10 client.15945 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:54:34.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.817+0000 7f09161ef740 20 client.15945 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:34.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.817+0000 7f09161ef740 1 --2- 172.21.15.152:0/4183496877 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55cac9f37810 0x55cac9f39c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.817+0000 7f09161ef740 4 client.15945 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:54:34.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.817+0000 7f09161ef740 1 -- 172.21.15.152:0/4183496877 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cac9e87860 con 0x55cac9f37810 2024-09-11T22:54:34.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.819+0000 7f09113b1640 1 --2- 172.21.15.152:0/4183496877 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55cac9f37810 0x55cac9f39c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:34.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.825+0000 7f09113b1640 1 --2- 172.21.15.152:0/4183496877 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55cac9f37810 0x55cac9f39c90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40eda5a2e07551ab server_cookie=abd83f8bd509bfee in_seq=0 out_seq=0 2024-09-11T22:54:34.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.825+0000 7f08f57fa640 10 client.15945 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:54:34.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.828+0000 7f08f57fa640 1 -- 172.21.15.152:0/4183496877 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cac9e87860 con 0x55cac9f37810 2024-09-11T22:54:34.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.828+0000 7f08f57fa640 10 client.15945 handle_command_reply: tid=0 2024-09-11T22:54:34.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.829+0000 7f09161ef740 1 client.15945 shutdown 2024-09-11T22:54:34.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.830+0000 7f09161ef740 1 -- 172.21.15.152:0/4183496877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f145d0 msgr2=0x55cac9f1edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:34.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.830+0000 7f09161ef740 1 --2- 172.21.15.152:0/4183496877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f145d0 0x55cac9f1edb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f09080043c0 tx=0x7f09080bc260 comp rx=0 tx=0).stop 2024-09-11T22:54:34.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.831+0000 7f09161ef740 1 -- 172.21.15.152:0/4183496877 shutdown_connections 2024-09-11T22:54:34.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.831+0000 7f09161ef740 1 --2- 172.21.15.152:0/4183496877 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55cac9f37810 0x55cac9f39c90 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.831+0000 7f09161ef740 1 --2- 172.21.15.152:0/4183496877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 0x55cac9f1c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.831+0000 7f08f57fa640 0 client.15945 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:54:34.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.831+0000 7f09161ef740 1 --2- 172.21.15.152:0/4183496877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f15b40 0x55cac9f1c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.831+0000 7f09161ef740 1 --2- 172.21.15.152:0/4183496877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f145d0 0x55cac9f1edb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.831+0000 7f09161ef740 1 -- 172.21.15.152:0/4183496877 >> 172.21.15.152:0/4183496877 conn(0x55cac9edd310 msgr2=0x55cac9f16930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:34.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.832+0000 7f09161ef740 1 -- 172.21.15.152:0/4183496877 shutdown_connections 2024-09-11T22:54:34.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.832+0000 7f09161ef740 1 -- 172.21.15.152:0/4183496877 wait complete. 2024-09-11T22:54:34.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.832+0000 7f09161ef740 20 client.15945 trim_cache size 0 max 16384 2024-09-11T22:54:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.852+0000 7f09161ef740 1 Processor -- start 2024-09-11T22:54:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.852+0000 7f09161ef740 1 -- start start 2024-09-11T22:54:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.853+0000 7f09161ef740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f145d0 0x55cac9f10d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.853+0000 7f09161ef740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f15b40 0x55cac9f1b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.853+0000 7f09161ef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 0x55cac9f3f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.853+0000 7f09161ef740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f08f000e580 con 0x55cac9f15b40 2024-09-11T22:54:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.853+0000 7f09161ef740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08f00100c0 con 0x55cac9f145d0 2024-09-11T22:54:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.853+0000 7f09161ef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f08f000ec50 con 0x55cac9f1f370 2024-09-11T22:54:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.853+0000 7f09113b1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f15b40 0x55cac9f1b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.853+0000 7f09113b1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f15b40 0x55cac9f1b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49184/0 (socket says 172.21.15.152:49184) 2024-09-11T22:54:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.853+0000 7f09113b1640 1 -- 172.21.15.152:0/2916847664 learned_addr learned my addr 172.21.15.152:0/2916847664 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:34.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.853+0000 7f0911bb2640 1 --2- 172.21.15.152:0/2916847664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f145d0 0x55cac9f10d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:34.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.854+0000 7f09113b1640 1 -- 172.21.15.152:0/2916847664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 msgr2=0x55cac9f3f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:34.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.854+0000 7f09113b1640 1 --2- 172.21.15.152:0/2916847664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 0x55cac9f3f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.854+0000 7f09113b1640 1 -- 172.21.15.152:0/2916847664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f145d0 msgr2=0x55cac9f10d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:34.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.854+0000 7f09113b1640 1 --2- 172.21.15.152:0/2916847664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f145d0 0x55cac9f10d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.854+0000 7f09113b1640 1 -- 172.21.15.152:0/2916847664 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cac9f104f0 con 0x55cac9f15b40 2024-09-11T22:54:34.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.854+0000 7f09113b1640 1 --2- 172.21.15.152:0/2916847664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f15b40 0x55cac9f1b040 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0045f0 tx=0x7f08fc003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa38724336cce19 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:34.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.854+0000 7f08f77fe640 1 -- 172.21.15.152:0/2916847664 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc003e50 con 0x55cac9f15b40 2024-09-11T22:54:34.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.855+0000 7f08f77fe640 1 -- 172.21.15.152:0/2916847664 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08fc02d020 con 0x55cac9f15b40 2024-09-11T22:54:34.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.855+0000 7f08f77fe640 1 -- 172.21.15.152:0/2916847664 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc004aa0 con 0x55cac9f15b40 2024-09-11T22:54:34.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.856+0000 7f09161ef740 1 -- 172.21.15.152:0/2916847664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f15b40 msgr2=0x55cac9f1b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:34.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.856+0000 7f09161ef740 1 --2- 172.21.15.152:0/2916847664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f15b40 0x55cac9f1b040 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0045f0 tx=0x7f08fc003d60 comp rx=0 tx=0).stop 2024-09-11T22:54:34.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.856+0000 7f09161ef740 1 -- 172.21.15.152:0/2916847664 shutdown_connections 2024-09-11T22:54:34.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.856+0000 7f09161ef740 1 --2- 172.21.15.152:0/2916847664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 0x55cac9f3f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.856+0000 7f09161ef740 1 --2- 172.21.15.152:0/2916847664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f15b40 0x55cac9f1b040 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.856+0000 7f09161ef740 1 --2- 172.21.15.152:0/2916847664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f145d0 0x55cac9f10d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.856+0000 7f09161ef740 1 -- 172.21.15.152:0/2916847664 >> 172.21.15.152:0/2916847664 conn(0x55cac9edd310 msgr2=0x55cac9f1a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:34.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.856+0000 7f09161ef740 1 -- 172.21.15.152:0/2916847664 shutdown_connections 2024-09-11T22:54:34.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.856+0000 7f09161ef740 1 -- 172.21.15.152:0/2916847664 wait complete. 2024-09-11T22:54:34.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.857+0000 7f09161ef740 1 -- start start 2024-09-11T22:54:34.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.857+0000 7f09161ef740 1 Processor -- start 2024-09-11T22:54:34.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.858+0000 7f09161ef740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f145d0 0x55cac9f2b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.858+0000 7f09161ef740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f15b40 0x55cac9f2bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.858+0000 7f09161ef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 0x55cac9f9a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.858+0000 7f09161ef740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f08f000f9f0 con 0x55cac9f145d0 2024-09-11T22:54:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.858+0000 7f09161ef740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08f000f320 con 0x55cac9f15b40 2024-09-11T22:54:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.858+0000 7f09161ef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f08f000c540 con 0x55cac9f1f370 2024-09-11T22:54:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.858+0000 7f09161ef740 10 client.0 _ll_register_callbacks cb 0x7f09080bc6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.858+0000 7f0911bb2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f145d0 0x55cac9f2b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.858+0000 7f0911bb2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f145d0 0x55cac9f2b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49194/0 (socket says 172.21.15.152:49194) 2024-09-11T22:54:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.858+0000 7f0911bb2640 1 -- 172.21.15.152:0/3567642009 learned_addr learned my addr 172.21.15.152:0/3567642009 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.859+0000 7f0911bb2640 1 -- 172.21.15.152:0/3567642009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 msgr2=0x55cac9f9a4f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:54:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.859+0000 7f0911bb2640 1 --2- 172.21.15.152:0/3567642009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 0x55cac9f9a4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.859+0000 7f0911bb2640 1 -- 172.21.15.152:0/3567642009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f15b40 msgr2=0x55cac9f2bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.859+0000 7f0911bb2640 1 --2- 172.21.15.152:0/3567642009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f15b40 0x55cac9f2bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.859+0000 7f0911bb2640 1 -- 172.21.15.152:0/3567642009 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cac9f9ad00 con 0x55cac9f145d0 2024-09-11T22:54:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.859+0000 7f0911bb2640 1 --2- 172.21.15.152:0/3567642009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f145d0 0x55cac9f2b670 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f09080c5080 tx=0x7f09080bc390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d2b3d4a2f11141 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.859+0000 7f08f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.859+0000 7f08f57fa640 1 -- 172.21.15.152:0/3567642009 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090804a070 con 0x55cac9f145d0 2024-09-11T22:54:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.859+0000 7f09161ef740 1 -- 172.21.15.152:0/3567642009 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08fc02d020 con 0x55cac9f145d0 2024-09-11T22:54:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.859+0000 7f08f57fa640 1 -- 172.21.15.152:0/3567642009 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f09080ef070 con 0x55cac9f145d0 2024-09-11T22:54:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.859+0000 7f08f57fa640 1 -- 172.21.15.152:0/3567642009 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f09080bacc0 con 0x55cac9f145d0 2024-09-11T22:54:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.859+0000 7f09161ef740 10 client.15951 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.859+0000 7f09161ef740 1 -- 172.21.15.152:0/3567642009 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cac9f9ad00 con 0x55cac9f145d0 2024-09-11T22:54:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.860+0000 7f08f57fa640 1 -- 172.21.15.152:0/3567642009 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08fc02d020 con 0x55cac9f145d0 2024-09-11T22:54:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.860+0000 7f08f57fa640 1 -- 172.21.15.152:0/3567642009 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f09080bbd50 con 0x55cac9f145d0 2024-09-11T22:54:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.860+0000 7f09161ef740 10 client.15951 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.860+0000 7f09161ef740 10 client.15951 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:54:34.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.860+0000 7f09161ef740 20 client.15951 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:34.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.860+0000 7f09161ef740 1 --2- 172.21.15.152:0/3567642009 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55cac9f37810 0x55cac9fa4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:34.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.860+0000 7f09161ef740 4 client.15951 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:54:34.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.860+0000 7f09161ef740 1 -- 172.21.15.152:0/3567642009 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cac9e81d70 con 0x55cac9f37810 2024-09-11T22:54:34.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.861+0000 7f09113b1640 1 --2- 172.21.15.152:0/3567642009 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55cac9f37810 0x55cac9fa4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:34.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.863+0000 7f09113b1640 1 --2- 172.21.15.152:0/3567642009 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55cac9f37810 0x55cac9fa4420 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3752aa33cafe8b14 server_cookie=179d1ab2381368f9 in_seq=0 out_seq=0 2024-09-11T22:54:34.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.863+0000 7f08f57fa640 10 client.15951 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:54:34.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.864+0000 7f08f57fa640 1 -- 172.21.15.152:0/3567642009 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cac9e81d70 con 0x55cac9f37810 2024-09-11T22:54:34.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.864+0000 7f08f57fa640 10 client.15951 handle_command_reply: tid=0 2024-09-11T22:54:34.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.864+0000 7f09161ef740 1 client.15951 shutdown 2024-09-11T22:54:34.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.864+0000 7f09161ef740 1 -- 172.21.15.152:0/3567642009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f145d0 msgr2=0x55cac9f2b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:34.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.864+0000 7f09161ef740 1 --2- 172.21.15.152:0/3567642009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f145d0 0x55cac9f2b670 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f09080c5080 tx=0x7f09080bc390 comp rx=0 tx=0).stop 2024-09-11T22:54:34.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.864+0000 7f09161ef740 1 -- 172.21.15.152:0/3567642009 shutdown_connections 2024-09-11T22:54:34.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.864+0000 7f09161ef740 1 --2- 172.21.15.152:0/3567642009 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55cac9f37810 0x55cac9fa4420 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.864+0000 7f09161ef740 1 --2- 172.21.15.152:0/3567642009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55cac9f1f370 0x55cac9f9a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.864+0000 7f09161ef740 1 --2- 172.21.15.152:0/3567642009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cac9f15b40 0x55cac9f2bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.864+0000 7f09161ef740 1 --2- 172.21.15.152:0/3567642009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cac9f145d0 0x55cac9f2b670 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.864+0000 7f09161ef740 1 -- 172.21.15.152:0/3567642009 >> 172.21.15.152:0/3567642009 conn(0x55cac9edd310 msgr2=0x55cac9f16110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:34.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.864+0000 7f08f57fa640 0 client.15951 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:54:34.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.864+0000 7f09161ef740 1 -- 172.21.15.152:0/3567642009 shutdown_connections 2024-09-11T22:54:34.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.864+0000 7f09161ef740 1 -- 172.21.15.152:0/3567642009 wait complete. 2024-09-11T22:54:34.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.864+0000 7f09161ef740 20 client.15951 trim_cache size 0 max 16384 2024-09-11T22:54:34.873 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:54:34.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.865+0000 7f0913e16640 1 -- 172.21.15.152:0/4127793609 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08f80a2e40 msgr2=0x7f08f80a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:34.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.865+0000 7f0913e16640 1 --2- 172.21.15.152:0/4127793609 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08f80a2e40 0x7f08f80a5300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f090c06b430 tx=0x7f0904012040 comp rx=0 tx=0).stop 2024-09-11T22:54:34.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.865+0000 7f0913e16640 1 -- 172.21.15.152:0/4127793609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f090c1091f0 msgr2=0x7f090c1a6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:34.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.865+0000 7f0913e16640 1 --2- 172.21.15.152:0/4127793609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f090c1091f0 0x7f090c1a6800 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0908009830 tx=0x7f0908005250 comp rx=0 tx=0).stop 2024-09-11T22:54:34.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.865+0000 7f0913e16640 1 -- 172.21.15.152:0/4127793609 shutdown_connections 2024-09-11T22:54:34.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.865+0000 7f0913e16640 1 --2- 172.21.15.152:0/4127793609 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08f80a2e40 0x7f08f80a5300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.865+0000 7f0913e16640 1 --2- 172.21.15.152:0/4127793609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f090c114ab0 0x7f090c1a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.865+0000 7f0913e16640 1 --2- 172.21.15.152:0/4127793609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f090c109b10 0x7f090c1a6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.865+0000 7f0913e16640 1 --2- 172.21.15.152:0/4127793609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f090c1091f0 0x7f090c1a6800 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:34.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.865+0000 7f0913e16640 1 -- 172.21.15.152:0/4127793609 >> 172.21.15.152:0/4127793609 conn(0x7f090c06e140 msgr2=0x7f090c0707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:34.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.866+0000 7f0913e16640 1 -- 172.21.15.152:0/4127793609 shutdown_connections 2024-09-11T22:54:34.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:34.866+0000 7f0913e16640 1 -- 172.21.15.152:0/4127793609 wait complete. 2024-09-11T22:54:35.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:34 smithi191 ceph-mon[35211]: pgmap v53: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-09-11T22:54:35.049 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:34 smithi204 ceph-mon[35454]: pgmap v53: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-09-11T22:54:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.422+0000 7f7b20c9b640 1 -- 172.21.15.152:0/3481660609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b1c107b80 msgr2=0x7f7b1c107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.422+0000 7f7b20c9b640 1 --2- 172.21.15.152:0/3481660609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b1c107b80 0x7f7b1c107f60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009930 tx=0x7f7b0c02a440 comp rx=0 tx=0).stop 2024-09-11T22:54:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.423+0000 7f7b20c9b640 1 -- 172.21.15.152:0/3481660609 shutdown_connections 2024-09-11T22:54:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.423+0000 7f7b20c9b640 1 --2- 172.21.15.152:0/3481660609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b1c102480 0x7f7b1c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.423+0000 7f7b20c9b640 1 --2- 172.21.15.152:0/3481660609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b1c101ae0 0x7f7b1c101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.423+0000 7f7b20c9b640 1 --2- 172.21.15.152:0/3481660609 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b1c107b80 0x7f7b1c107f60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.423+0000 7f7b20c9b640 1 -- 172.21.15.152:0/3481660609 >> 172.21.15.152:0/3481660609 conn(0x7f7b1c0fd7e0 msgr2=0x7f7b1c0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.423+0000 7f7b20c9b640 1 -- 172.21.15.152:0/3481660609 shutdown_connections 2024-09-11T22:54:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.423+0000 7f7b20c9b640 1 -- 172.21.15.152:0/3481660609 wait complete. 2024-09-11T22:54:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.424+0000 7f7b20c9b640 1 Processor -- start 2024-09-11T22:54:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.424+0000 7f7b20c9b640 1 -- start start 2024-09-11T22:54:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.424+0000 7f7b20c9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b1c101ae0 0x7f7b1c1a0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.424+0000 7f7b20c9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b1c102480 0x7f7b1c1a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.424+0000 7f7b20c9b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b1c107b80 0x7f7b1c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.424+0000 7f7b20c9b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c0748f0 con 0x7f7b1c101ae0 2024-09-11T22:54:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.424+0000 7f7b20c9b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c074770 con 0x7f7b1c107b80 2024-09-11T22:54:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.424+0000 7f7b20c9b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c074a70 con 0x7f7b1c102480 2024-09-11T22:54:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.424+0000 7f7b1a59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b1c101ae0 0x7f7b1c1a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.424+0000 7f7b1a59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b1c101ae0 0x7f7b1c1a0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49206/0 (socket says 172.21.15.152:49206) 2024-09-11T22:54:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.424+0000 7f7b1a59c640 1 -- 172.21.15.152:0/1462752871 learned_addr learned my addr 172.21.15.152:0/1462752871 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.424+0000 7f7b1a59c640 1 -- 172.21.15.152:0/1462752871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b1c102480 msgr2=0x7f7b1c1a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.424+0000 7f7b1a59c640 1 --2- 172.21.15.152:0/1462752871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b1c102480 0x7f7b1c1a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.424+0000 7f7b1a59c640 1 -- 172.21.15.152:0/1462752871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b1c107b80 msgr2=0x7f7b1c19ac50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:54:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.424+0000 7f7b1a59c640 1 --2- 172.21.15.152:0/1462752871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b1c107b80 0x7f7b1c19ac50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.424+0000 7f7b1a59c640 1 -- 172.21.15.152:0/1462752871 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1c19b450 con 0x7f7b1c101ae0 2024-09-11T22:54:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.425+0000 7f7b1a59c640 1 --2- 172.21.15.152:0/1462752871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b1c101ae0 0x7f7b1c1a0490 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c006010 tx=0x7f7b0c003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.425+0000 7f7b0b7fe640 1 -- 172.21.15.152:0/1462752871 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c03f070 con 0x7f7b1c101ae0 2024-09-11T22:54:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.425+0000 7f7b0b7fe640 1 -- 172.21.15.152:0/1462752871 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b0c031c90 con 0x7f7b1c101ae0 2024-09-11T22:54:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.425+0000 7f7b0b7fe640 1 -- 172.21.15.152:0/1462752871 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c035030 con 0x7f7b1c101ae0 2024-09-11T22:54:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.425+0000 7f7b20c9b640 1 -- 172.21.15.152:0/1462752871 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b1c19b6e0 con 0x7f7b1c101ae0 2024-09-11T22:54:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.425+0000 7f7b20c9b640 1 -- 172.21.15.152:0/1462752871 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b1c1a72a0 con 0x7f7b1c101ae0 2024-09-11T22:54:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.428+0000 7f7b0b7fe640 1 -- 172.21.15.152:0/1462752871 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7b0c042070 con 0x7f7b1c101ae0 2024-09-11T22:54:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.429+0000 7f7b23a34740 1 Processor -- start 2024-09-11T22:54:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.429+0000 7f7b23a34740 1 -- start start 2024-09-11T22:54:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.429+0000 7f7b23a34740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cf3af0 0x555b34cf3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.429+0000 7f7b23a34740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cf4410 0x555b34cfe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.429+0000 7f7b23a34740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cfecb0 0x555b34d010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.429+0000 7f7b23a34740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x555b34c582f0 con 0x555b34cf3af0 2024-09-11T22:54:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.429+0000 7f7b23a34740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555b34c00270 con 0x555b34cfecb0 2024-09-11T22:54:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.429+0000 7f7b23a34740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555b34c542b0 con 0x555b34cf4410 2024-09-11T22:54:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.430+0000 7f7b1ad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cf3af0 0x555b34cf3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.430+0000 7f7b1ad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cf3af0 0x555b34cf3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49220/0 (socket says 172.21.15.152:49220) 2024-09-11T22:54:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.430+0000 7f7b1ad9d640 1 -- 172.21.15.152:0/3979331879 learned_addr learned my addr 172.21.15.152:0/3979331879 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.430+0000 7f7b0b7fe640 1 --2- 172.21.15.152:0/1462752871 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7b040a2e20 0x7f7b040a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.430+0000 7f7b1a59c640 1 --2- 172.21.15.152:0/3979331879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cf4410 0x555b34cfe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.430+0000 7f7b19d9b640 1 --2- 172.21.15.152:0/3979331879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cfecb0 0x555b34d010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.430+0000 7f7b1ad9d640 1 -- 172.21.15.152:0/3979331879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cf4410 msgr2=0x555b34cfe770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.430+0000 7f7b1ad9d640 1 --2- 172.21.15.152:0/3979331879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cf4410 0x555b34cfe770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.430+0000 7f7b1ad9d640 1 -- 172.21.15.152:0/3979331879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cfecb0 msgr2=0x555b34d010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.430+0000 7f7b1ad9d640 1 --2- 172.21.15.152:0/3979331879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cfecb0 0x555b34d010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.430+0000 7f7b1ad9d640 1 -- 172.21.15.152:0/3979331879 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b34bff340 con 0x555b34cf3af0 2024-09-11T22:54:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.430+0000 7f7b0b7fe640 1 -- 172.21.15.152:0/1462752871 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7b0c0efcc0 con 0x7f7b1c101ae0 2024-09-11T22:54:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.430+0000 7f7b19d9b640 1 --2- 172.21.15.152:0/3979331879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cfecb0 0x555b34d010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:54:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.431+0000 7f7b1ad9d640 1 --2- 172.21.15.152:0/1462752871 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7b040a2e20 0x7f7b040a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.431+0000 7f7b1ad9d640 1 --2- 172.21.15.152:0/3979331879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cf3af0 0x555b34cf3ed0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009eb0 tx=0x7f7b1400b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8605a09d9f09354 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.431+0000 7f7b08ff9640 1 -- 172.21.15.152:0/3979331879 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14015e60 con 0x555b34cf3af0 2024-09-11T22:54:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.431+0000 7f7b1ad9d640 1 --2- 172.21.15.152:0/1462752871 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7b040a2e20 0x7f7b040a52e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7b1401d000 tx=0x7f7b1401e000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.431+0000 7f7b08ff9640 1 -- 172.21.15.152:0/3979331879 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b140148c0 con 0x555b34cf3af0 2024-09-11T22:54:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.431+0000 7f7b08ff9640 1 -- 172.21.15.152:0/3979331879 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b140138a0 con 0x555b34cf3af0 2024-09-11T22:54:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.432+0000 7f7b23a34740 1 -- 172.21.15.152:0/3979331879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cf3af0 msgr2=0x555b34cf3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.432+0000 7f7b23a34740 1 --2- 172.21.15.152:0/3979331879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cf3af0 0x555b34cf3ed0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009eb0 tx=0x7f7b1400b420 comp rx=0 tx=0).stop 2024-09-11T22:54:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.433+0000 7f7b23a34740 1 -- 172.21.15.152:0/3979331879 shutdown_connections 2024-09-11T22:54:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.433+0000 7f7b23a34740 1 --2- 172.21.15.152:0/3979331879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cfecb0 0x555b34d010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.433+0000 7f7b23a34740 1 --2- 172.21.15.152:0/3979331879 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cf4410 0x555b34cfe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.433+0000 7f7b23a34740 1 --2- 172.21.15.152:0/3979331879 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cf3af0 0x555b34cf3ed0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.433+0000 7f7b23a34740 1 -- 172.21.15.152:0/3979331879 >> 172.21.15.152:0/3979331879 conn(0x555b34cf0060 msgr2=0x555b34cf0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.433+0000 7f7b23a34740 1 -- 172.21.15.152:0/3979331879 shutdown_connections 2024-09-11T22:54:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.433+0000 7f7b23a34740 1 -- 172.21.15.152:0/3979331879 wait complete. 2024-09-11T22:54:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.433+0000 7f7b23a34740 1 -- start start 2024-09-11T22:54:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.433+0000 7f7b23a34740 1 Processor -- start 2024-09-11T22:54:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b23a34740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cf3af0 0x555b34cfbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b23a34740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cf4410 0x555b34cfc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b23a34740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cfecb0 0x555b34cfc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b23a34740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x555b34d01880 con 0x555b34cf3af0 2024-09-11T22:54:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b23a34740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555b34be1290 con 0x555b34cfecb0 2024-09-11T22:54:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b23a34740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555b34d01a00 con 0x555b34cf4410 2024-09-11T22:54:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b23a34740 10 client.0 _ll_register_callbacks cb 0x555b34c12140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b1a59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cf3af0 0x555b34cfbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b1a59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cf3af0 0x555b34cfbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49222/0 (socket says 172.21.15.152:49222) 2024-09-11T22:54:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b1a59c640 1 -- 172.21.15.152:0/41081468 learned_addr learned my addr 172.21.15.152:0/41081468 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b1a59c640 1 -- 172.21.15.152:0/41081468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cf4410 msgr2=0x555b34cfc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b1a59c640 1 --2- 172.21.15.152:0/41081468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cf4410 0x555b34cfc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b1a59c640 1 -- 172.21.15.152:0/41081468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cfecb0 msgr2=0x555b34cfc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b1a59c640 1 --2- 172.21.15.152:0/41081468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cfecb0 0x555b34cfc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b1a59c640 1 -- 172.21.15.152:0/41081468 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b3496b210 con 0x555b34cf3af0 2024-09-11T22:54:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b1a59c640 1 --2- 172.21.15.152:0/41081468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cf3af0 0x555b34cfbc60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009900 tx=0x7f7b0c02a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68c763245191c9a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.434+0000 7f7b01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.435+0000 7f7b01ffb640 1 -- 172.21.15.152:0/41081468 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c049050 con 0x555b34cf3af0 2024-09-11T22:54:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.435+0000 7f7b01ffb640 1 -- 172.21.15.152:0/41081468 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7b0c03b580 con 0x555b34cf3af0 2024-09-11T22:54:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.435+0000 7f7b23a34740 1 -- 172.21.15.152:0/41081468 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b140148c0 con 0x555b34cf3af0 2024-09-11T22:54:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.435+0000 7f7b01ffb640 1 -- 172.21.15.152:0/41081468 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b0c0d4030 con 0x555b34cf3af0 2024-09-11T22:54:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.435+0000 7f7b23a34740 10 client.15963 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.435+0000 7f7b23a34740 1 -- 172.21.15.152:0/41081468 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b34a37620 con 0x555b34cf3af0 2024-09-11T22:54:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.435+0000 7f7b01ffb640 1 -- 172.21.15.152:0/41081468 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b140148c0 con 0x555b34cf3af0 2024-09-11T22:54:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.435+0000 7f7b01ffb640 1 -- 172.21.15.152:0/41081468 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b0c0ed6e0 con 0x555b34cf3af0 2024-09-11T22:54:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.435+0000 7f7b23a34740 10 client.15963 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.435+0000 7f7b23a34740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:54:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.435+0000 7f7b23a34740 20 client.15963 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.435+0000 7f7b23a34740 1 --2- 172.21.15.152:0/41081468 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555b34d0ec40 0x555b34d110b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.435+0000 7f7b23a34740 4 client.15963 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:54:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.435+0000 7f7b23a34740 1 -- 172.21.15.152:0/41081468 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b34c65a20 con 0x555b34d0ec40 2024-09-11T22:54:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.438+0000 7f7b19d9b640 1 --2- 172.21.15.152:0/41081468 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555b34d0ec40 0x555b34d110b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.442+0000 7f7b01ffb640 10 client.15963 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.442+0000 7f7b19d9b640 1 --2- 172.21.15.152:0/41081468 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555b34d0ec40 0x555b34d110b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5959be88c0ad8c0 server_cookie=581169bcde25a891 in_seq=0 out_seq=0 2024-09-11T22:54:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.444+0000 7f7b01ffb640 1 -- 172.21.15.152:0/41081468 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b34c65a20 con 0x555b34d0ec40 2024-09-11T22:54:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.444+0000 7f7b01ffb640 10 client.15963 handle_command_reply: tid=0 2024-09-11T22:54:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.444+0000 7f7b23a34740 1 client.15963 shutdown 2024-09-11T22:54:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.444+0000 7f7b23a34740 1 -- 172.21.15.152:0/41081468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cf3af0 msgr2=0x555b34cfbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.444+0000 7f7b23a34740 1 --2- 172.21.15.152:0/41081468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cf3af0 0x555b34cfbc60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009900 tx=0x7f7b0c02a440 comp rx=0 tx=0).stop 2024-09-11T22:54:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.444+0000 7f7b23a34740 1 -- 172.21.15.152:0/41081468 shutdown_connections 2024-09-11T22:54:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.444+0000 7f7b23a34740 1 --2- 172.21.15.152:0/41081468 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555b34d0ec40 0x555b34d110b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.444+0000 7f7b23a34740 1 --2- 172.21.15.152:0/41081468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cfecb0 0x555b34cfc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.444+0000 7f7b23a34740 1 --2- 172.21.15.152:0/41081468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cf4410 0x555b34cfc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.444+0000 7f7b23a34740 1 --2- 172.21.15.152:0/41081468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cf3af0 0x555b34cfbc60 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.444+0000 7f7b01ffb640 0 client.15963 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.444+0000 7f7b23a34740 1 -- 172.21.15.152:0/41081468 >> 172.21.15.152:0/41081468 conn(0x555b34cf0060 msgr2=0x555b34cf5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.444+0000 7f7b23a34740 1 -- 172.21.15.152:0/41081468 shutdown_connections 2024-09-11T22:54:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.444+0000 7f7b23a34740 1 -- 172.21.15.152:0/41081468 wait complete. 2024-09-11T22:54:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.444+0000 7f7b23a34740 20 client.15963 trim_cache size 0 max 16384 2024-09-11T22:54:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.465+0000 7f7b23a34740 1 Processor -- start 2024-09-11T22:54:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.465+0000 7f7b23a34740 1 -- start start 2024-09-11T22:54:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.465+0000 7f7b23a34740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cf4410 0x555b34cfc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.465+0000 7f7b23a34740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cfecb0 0x555b34cfdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.465+0000 7f7b23a34740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cfad50 0x555b34d169c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.465+0000 7f7b23a34740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1af9e0 con 0x555b34cfecb0 2024-09-11T22:54:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.465+0000 7f7b23a34740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1b1520 con 0x555b34cf4410 2024-09-11T22:54:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.465+0000 7f7b23a34740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1b00b0 con 0x555b34cfad50 2024-09-11T22:54:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.465+0000 7f7b19d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cfecb0 0x555b34cfdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:35.467 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.465+0000 7f7b19d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cfecb0 0x555b34cfdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49224/0 (socket says 172.21.15.152:49224) 2024-09-11T22:54:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.465+0000 7f7b19d9b640 1 -- 172.21.15.152:0/2236638074 learned_addr learned my addr 172.21.15.152:0/2236638074 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.466+0000 7f7b19d9b640 1 -- 172.21.15.152:0/2236638074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cfad50 msgr2=0x555b34d169c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:54:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.466+0000 7f7b19d9b640 1 --2- 172.21.15.152:0/2236638074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cfad50 0x555b34d169c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.466+0000 7f7b19d9b640 1 -- 172.21.15.152:0/2236638074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cf4410 msgr2=0x555b34cfc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.466+0000 7f7b19d9b640 1 --2- 172.21.15.152:0/2236638074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cf4410 0x555b34cfc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.466+0000 7f7b19d9b640 1 -- 172.21.15.152:0/2236638074 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b34d01410 con 0x555b34cfecb0 2024-09-11T22:54:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.466+0000 7f7b19d9b640 1 --2- 172.21.15.152:0/2236638074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cfecb0 0x555b34cfdbd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000a770 tx=0x7f7b1000acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838a09b9a8e30e97 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.466+0000 7f7b03fff640 1 -- 172.21.15.152:0/2236638074 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b10021070 con 0x555b34cfecb0 2024-09-11T22:54:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.466+0000 7f7b03fff640 1 -- 172.21.15.152:0/2236638074 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b1001c430 con 0x555b34cfecb0 2024-09-11T22:54:35.468 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.466+0000 7f7b03fff640 1 -- 172.21.15.152:0/2236638074 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b10014670 con 0x555b34cfecb0 2024-09-11T22:54:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.467+0000 7f7b23a34740 1 -- 172.21.15.152:0/2236638074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cfecb0 msgr2=0x555b34cfdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.467+0000 7f7b23a34740 1 --2- 172.21.15.152:0/2236638074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cfecb0 0x555b34cfdbd0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7b1000a770 tx=0x7f7b1000acb0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.467+0000 7f7b23a34740 1 -- 172.21.15.152:0/2236638074 shutdown_connections 2024-09-11T22:54:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.467+0000 7f7b23a34740 1 --2- 172.21.15.152:0/2236638074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cfad50 0x555b34d169c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.467+0000 7f7b23a34740 1 --2- 172.21.15.152:0/2236638074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cfecb0 0x555b34cfdbd0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.467+0000 7f7b23a34740 1 --2- 172.21.15.152:0/2236638074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cf4410 0x555b34cfc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.467+0000 7f7b23a34740 1 -- 172.21.15.152:0/2236638074 >> 172.21.15.152:0/2236638074 conn(0x555b34cf0060 msgr2=0x555b34cfd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.467+0000 7f7b23a34740 1 -- 172.21.15.152:0/2236638074 shutdown_connections 2024-09-11T22:54:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.467+0000 7f7b23a34740 1 -- 172.21.15.152:0/2236638074 wait complete. 2024-09-11T22:54:35.472 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.467+0000 7f7b23a34740 1 -- start start 2024-09-11T22:54:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b23a34740 1 Processor -- start 2024-09-11T22:54:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b23a34740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cf4410 0x555b34d06c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b23a34740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cfad50 0x555b34d071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b23a34740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cfecb0 0x555b34d03c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b23a34740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1b0e50 con 0x555b34cfecb0 2024-09-11T22:54:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b23a34740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1b0780 con 0x555b34cfad50 2024-09-11T22:54:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b23a34740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c1ad9a0 con 0x555b34cf4410 2024-09-11T22:54:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b23a34740 10 client.0 _ll_register_callbacks cb 0x7f7b0c005cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b1ad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cfecb0 0x555b34d03c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b1ad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cfecb0 0x555b34d03c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49226/0 (socket says 172.21.15.152:49226) 2024-09-11T22:54:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b1ad9d640 1 -- 172.21.15.152:0/680996247 learned_addr learned my addr 172.21.15.152:0/680996247 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b1ad9d640 1 -- 172.21.15.152:0/680996247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cf4410 msgr2=0x555b34d06c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b1a59c640 1 --2- 172.21.15.152:0/680996247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cf4410 0x555b34d06c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b19d9b640 1 --2- 172.21.15.152:0/680996247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cfad50 0x555b34d071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b1ad9d640 1 --2- 172.21.15.152:0/680996247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cf4410 0x555b34d06c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b1ad9d640 1 -- 172.21.15.152:0/680996247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cfad50 msgr2=0x555b34d071d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.468+0000 7f7b1ad9d640 1 --2- 172.21.15.152:0/680996247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cfad50 0x555b34d071d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.469+0000 7f7b1ad9d640 1 -- 172.21.15.152:0/680996247 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b34d041a0 con 0x555b34cfecb0 2024-09-11T22:54:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.469+0000 7f7b1a59c640 1 --2- 172.21.15.152:0/680996247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cf4410 0x555b34d06c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:54:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.469+0000 7f7b19d9b640 1 --2- 172.21.15.152:0/680996247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cfad50 0x555b34d071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:54:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.469+0000 7f7b1ad9d640 1 --2- 172.21.15.152:0/680996247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cfecb0 0x555b34d03c60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400b920 tx=0x7f7b140135b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b1cbbfb4ab412e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.469+0000 7f7b01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.469+0000 7f7b01ffb640 1 -- 172.21.15.152:0/680996247 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1402d060 con 0x555b34cfecb0 2024-09-11T22:54:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.469+0000 7f7b01ffb640 1 -- 172.21.15.152:0/680996247 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7b14020440 con 0x555b34cfecb0 2024-09-11T22:54:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.469+0000 7f7b23a34740 1 -- 172.21.15.152:0/680996247 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b1001c430 con 0x555b34cfecb0 2024-09-11T22:54:35.474 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.469+0000 7f7b01ffb640 1 -- 172.21.15.152:0/680996247 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b14015e60 con 0x555b34cfecb0 2024-09-11T22:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.469+0000 7f7b23a34740 10 client.15969 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.469+0000 7f7b23a34740 1 -- 172.21.15.152:0/680996247 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b34d7a150 con 0x555b34cfecb0 2024-09-11T22:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.469+0000 7f7b01ffb640 1 -- 172.21.15.152:0/680996247 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b1001c430 con 0x555b34cfecb0 2024-09-11T22:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.469+0000 7f7b01ffb640 1 -- 172.21.15.152:0/680996247 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b14028410 con 0x555b34cfecb0 2024-09-11T22:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.470+0000 7f7b23a34740 10 client.15969 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.470+0000 7f7b23a34740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.470+0000 7f7b23a34740 20 client.15969 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.470+0000 7f7b23a34740 1 --2- 172.21.15.152:0/680996247 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555b34d7a760 0x555b34d7cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.470+0000 7f7b23a34740 4 client.15969 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:54:35.475 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.470+0000 7f7b23a34740 1 -- 172.21.15.152:0/680996247 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b34c5fd70 con 0x555b34d7a760 2024-09-11T22:54:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.471+0000 7f7b1a59c640 1 --2- 172.21.15.152:0/680996247 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555b34d7a760 0x555b34d7cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.474+0000 7f7b01ffb640 10 client.15969 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:35.476 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.474+0000 7f7b1a59c640 1 --2- 172.21.15.152:0/680996247 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555b34d7a760 0x555b34d7cb80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d46a348dc13f52 server_cookie=4638cf10de629ae5 in_seq=0 out_seq=0 2024-09-11T22:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.495+0000 7f7b01ffb640 1 -- 172.21.15.152:0/680996247 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b34c5fd70 con 0x555b34d7a760 2024-09-11T22:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.495+0000 7f7b01ffb640 10 client.15969 handle_command_reply: tid=0 2024-09-11T22:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.495+0000 7f7b23a34740 1 client.15969 shutdown 2024-09-11T22:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.496+0000 7f7b23a34740 1 -- 172.21.15.152:0/680996247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cfecb0 msgr2=0x555b34d03c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.496+0000 7f7b23a34740 1 --2- 172.21.15.152:0/680996247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cfecb0 0x555b34d03c60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400b920 tx=0x7f7b140135b0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.496+0000 7f7b23a34740 1 -- 172.21.15.152:0/680996247 shutdown_connections 2024-09-11T22:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.496+0000 7f7b23a34740 1 --2- 172.21.15.152:0/680996247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b34cfecb0 0x555b34d03c60 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.496+0000 7f7b23a34740 1 --2- 172.21.15.152:0/680996247 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555b34d7a760 0x555b34d7cb80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.496+0000 7f7b23a34740 1 --2- 172.21.15.152:0/680996247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b34cfad50 0x555b34d071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.496+0000 7f7b23a34740 1 --2- 172.21.15.152:0/680996247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b34cf4410 0x555b34d06c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.496+0000 7f7b01ffb640 0 client.15969 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.496+0000 7f7b23a34740 1 -- 172.21.15.152:0/680996247 >> 172.21.15.152:0/680996247 conn(0x555b34cf0060 msgr2=0x555b34cef1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.496+0000 7f7b23a34740 1 -- 172.21.15.152:0/680996247 shutdown_connections 2024-09-11T22:54:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.496+0000 7f7b23a34740 1 -- 172.21.15.152:0/680996247 wait complete. 2024-09-11T22:54:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.496+0000 7f7b23a34740 20 client.15969 trim_cache size 0 max 16384 2024-09-11T22:54:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:{ 2024-09-11T22:54:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "message": "", 2024-09-11T22:54:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "return_code": 0 2024-09-11T22:54:35.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:} 2024-09-11T22:54:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.497+0000 7f7b20c9b640 1 -- 172.21.15.152:0/1462752871 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7b040a2e20 msgr2=0x7f7b040a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.497+0000 7f7b20c9b640 1 --2- 172.21.15.152:0/1462752871 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7b040a2e20 0x7f7b040a52e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7b1401d000 tx=0x7f7b1401e000 comp rx=0 tx=0).stop 2024-09-11T22:54:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.497+0000 7f7b20c9b640 1 -- 172.21.15.152:0/1462752871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b1c101ae0 msgr2=0x7f7b1c1a0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.497+0000 7f7b20c9b640 1 --2- 172.21.15.152:0/1462752871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b1c101ae0 0x7f7b1c1a0490 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c006010 tx=0x7f7b0c003b60 comp rx=0 tx=0).stop 2024-09-11T22:54:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.497+0000 7f7b20c9b640 1 -- 172.21.15.152:0/1462752871 shutdown_connections 2024-09-11T22:54:35.499 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.497+0000 7f7b20c9b640 1 --2- 172.21.15.152:0/1462752871 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7b040a2e20 0x7f7b040a52e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.497+0000 7f7b20c9b640 1 --2- 172.21.15.152:0/1462752871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7b1c107b80 0x7f7b1c19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.497+0000 7f7b20c9b640 1 --2- 172.21.15.152:0/1462752871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b1c102480 0x7f7b1c1a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.497+0000 7f7b20c9b640 1 --2- 172.21.15.152:0/1462752871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7b1c101ae0 0x7f7b1c1a0490 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.497+0000 7f7b20c9b640 1 -- 172.21.15.152:0/1462752871 >> 172.21.15.152:0/1462752871 conn(0x7f7b1c0fd7e0 msgr2=0x7f7b1c075260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.497+0000 7f7b20c9b640 1 -- 172.21.15.152:0/1462752871 shutdown_connections 2024-09-11T22:54:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:35.497+0000 7f7b20c9b640 1 -- 172.21.15.152:0/1462752871 wait complete. 2024-09-11T22:54:35.590 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:35.590 INFO:tasks.workunit.client.0.smithi152.stdout: Children see throughput for 1 initial writers = 714336.56 kB/sec 2024-09-11T22:54:35.590 INFO:tasks.workunit.client.0.smithi152.stdout: Parent sees throughput for 1 initial writers = 714247.01 kB/sec 2024-09-11T22:54:35.590 INFO:tasks.workunit.client.0.smithi152.stdout: Min throughput per process = 714336.56 kB/sec 2024-09-11T22:54:35.590 INFO:tasks.workunit.client.0.smithi152.stdout: Max throughput per process = 714336.56 kB/sec 2024-09-11T22:54:35.590 INFO:tasks.workunit.client.0.smithi152.stdout: Avg throughput per process = 714336.56 kB/sec 2024-09-11T22:54:35.591 INFO:tasks.workunit.client.0.smithi152.stdout: Min xfer = 1048576.00 kB 2024-09-11T22:54:35.607 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:54:35.855 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:54:36.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.606+0000 7f08c37e4640 1 -- 172.21.15.152:0/3191719621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc102ef0 msgr2=0x7f08bc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:36.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.606+0000 7f08c37e4640 1 --2- 172.21.15.152:0/3191719621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc102ef0 0x7f08bc10da40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009ca0 tx=0x7f08b402a950 comp rx=0 tx=0).stop 2024-09-11T22:54:36.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.606+0000 7f08c37e4640 1 -- 172.21.15.152:0/3191719621 shutdown_connections 2024-09-11T22:54:36.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.606+0000 7f08c37e4640 1 --2- 172.21.15.152:0/3191719621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc102ef0 0x7f08bc10da40 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.606+0000 7f08c37e4640 1 --2- 172.21.15.152:0/3191719621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08bc102570 0x7f08bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.606+0000 7f08c37e4640 1 --2- 172.21.15.152:0/3191719621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08bc100e20 0x7f08bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.607+0000 7f08c37e4640 1 -- 172.21.15.152:0/3191719621 >> 172.21.15.152:0/3191719621 conn(0x7f08bc0fc7d0 msgr2=0x7f08bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:36.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.607+0000 7f08c37e4640 1 -- 172.21.15.152:0/3191719621 shutdown_connections 2024-09-11T22:54:36.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.607+0000 7f08c37e4640 1 -- 172.21.15.152:0/3191719621 wait complete. 2024-09-11T22:54:36.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.607+0000 7f08c37e4640 1 Processor -- start 2024-09-11T22:54:36.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.607+0000 7f08c37e4640 1 -- start start 2024-09-11T22:54:36.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.607+0000 7f08c37e4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08bc100e20 0x7f08bc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.607+0000 7f08c37e4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc102570 0x7f08bc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.607+0000 7f08c37e4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08bc102ef0 0x7f08bc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.607+0000 7f08c37e4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc110f80 con 0x7f08bc102570 2024-09-11T22:54:36.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.607+0000 7f08c37e4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc110e00 con 0x7f08bc102ef0 2024-09-11T22:54:36.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.607+0000 7f08c37e4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc111100 con 0x7f08bc100e20 2024-09-11T22:54:36.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.608+0000 7f08c0d7f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc102570 0x7f08bc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.608+0000 7f08c0d7f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc102570 0x7f08bc1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49254/0 (socket says 172.21.15.152:49254) 2024-09-11T22:54:36.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.608+0000 7f08c0d7f640 1 -- 172.21.15.152:0/3751038451 learned_addr learned my addr 172.21.15.152:0/3751038451 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:36.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.608+0000 7f08c1580640 1 --2- 172.21.15.152:0/3751038451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08bc100e20 0x7f08bc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.608+0000 7f08c1d81640 1 --2- 172.21.15.152:0/3751038451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08bc102ef0 0x7f08bc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.608+0000 7f08c0d7f640 1 -- 172.21.15.152:0/3751038451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08bc100e20 msgr2=0x7f08bc198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:36.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.608+0000 7f08c0d7f640 1 --2- 172.21.15.152:0/3751038451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08bc100e20 0x7f08bc198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.608+0000 7f08c0d7f640 1 -- 172.21.15.152:0/3751038451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08bc102ef0 msgr2=0x7f08bc19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:36.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.608+0000 7f08c0d7f640 1 --2- 172.21.15.152:0/3751038451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08bc102ef0 0x7f08bc19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.608+0000 7f08c0d7f640 1 -- 172.21.15.152:0/3751038451 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08bc19d440 con 0x7f08bc102570 2024-09-11T22:54:36.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.608+0000 7f08c0d7f640 1 --2- 172.21.15.152:0/3751038451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc102570 0x7f08bc1989d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f08b800d950 tx=0x7f08b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:36.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.608+0000 7f08ae7fc640 1 -- 172.21.15.152:0/3751038451 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8017070 con 0x7f08bc102570 2024-09-11T22:54:36.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.608+0000 7f08c37e4640 1 -- 172.21.15.152:0/3751038451 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08bc19d730 con 0x7f08bc102570 2024-09-11T22:54:36.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.608+0000 7f08ae7fc640 1 -- 172.21.15.152:0/3751038451 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b8009c30 con 0x7f08bc102570 2024-09-11T22:54:36.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.609+0000 7f08ae7fc640 1 -- 172.21.15.152:0/3751038451 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b800f6b0 con 0x7f08bc102570 2024-09-11T22:54:36.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.609+0000 7f08c37e4640 1 -- 172.21.15.152:0/3751038451 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08bc1a5010 con 0x7f08bc102570 2024-09-11T22:54:36.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.611+0000 7f08ae7fc640 1 -- 172.21.15.152:0/3751038451 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f08b80109d0 con 0x7f08bc102570 2024-09-11T22:54:36.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.611+0000 7f08ae7fc640 1 --2- 172.21.15.152:0/3751038451 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08a80a2cf0 0x7f08a80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.611+0000 7f08c1580640 1 --2- 172.21.15.152:0/3751038451 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08a80a2cf0 0x7f08a80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.611+0000 7f08ae7fc640 1 -- 172.21.15.152:0/3751038451 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f08b80cf2f0 con 0x7f08bc102570 2024-09-11T22:54:36.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c5bbd740 1 Processor -- start 2024-09-11T22:54:36.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c5bbd740 1 -- start start 2024-09-11T22:54:36.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c1580640 1 --2- 172.21.15.152:0/3751038451 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08a80a2cf0 0x7f08a80a51b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f08b0005fd0 tx=0x7f08b000b390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:36.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c5bbd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47db40 0x55641b47c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c5bbd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47c5d0 0x55641b47ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c5bbd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b487370 0x55641b489760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c5bbd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55641b3e22f0 con 0x55641b47c5d0 2024-09-11T22:54:36.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c5bbd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55641b389cf0 con 0x55641b487370 2024-09-11T22:54:36.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c5bbd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55641b3de2b0 con 0x55641b47db40 2024-09-11T22:54:36.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c0d7f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47c5d0 0x55641b47ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c0d7f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47c5d0 0x55641b47ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49262/0 (socket says 172.21.15.152:49262) 2024-09-11T22:54:36.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c0d7f640 1 -- 172.21.15.152:0/1044143734 learned_addr learned my addr 172.21.15.152:0/1044143734 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:36.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c1580640 1 --2- 172.21.15.152:0/1044143734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47db40 0x55641b47c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c1d81640 1 --2- 172.21.15.152:0/1044143734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b487370 0x55641b489760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c0d7f640 1 -- 172.21.15.152:0/1044143734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47db40 msgr2=0x55641b47c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:36.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c0d7f640 1 --2- 172.21.15.152:0/1044143734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47db40 0x55641b47c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c0d7f640 1 -- 172.21.15.152:0/1044143734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b487370 msgr2=0x55641b489760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:36.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c0d7f640 1 --2- 172.21.15.152:0/1044143734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b487370 0x55641b489760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c0d7f640 1 -- 172.21.15.152:0/1044143734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55641b36e3a0 con 0x55641b47c5d0 2024-09-11T22:54:36.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.612+0000 7f08c1d81640 1 --2- 172.21.15.152:0/1044143734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b487370 0x55641b489760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:54:36.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.613+0000 7f08c0d7f640 1 --2- 172.21.15.152:0/1044143734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47c5d0 0x55641b47ca10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f08b80a3fa0 tx=0x7f08b809b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32945e160b092b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:36.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.613+0000 7f08a77fe640 1 -- 172.21.15.152:0/1044143734 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8020070 con 0x55641b47c5d0 2024-09-11T22:54:36.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.613+0000 7f08a77fe640 1 -- 172.21.15.152:0/1044143734 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b8084e50 con 0x55641b47c5d0 2024-09-11T22:54:36.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.613+0000 7f08a77fe640 1 -- 172.21.15.152:0/1044143734 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8086310 con 0x55641b47c5d0 2024-09-11T22:54:36.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.614+0000 7f08c5bbd740 1 -- 172.21.15.152:0/1044143734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47c5d0 msgr2=0x55641b47ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:36.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.614+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/1044143734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47c5d0 0x55641b47ca10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f08b80a3fa0 tx=0x7f08b809b910 comp rx=0 tx=0).stop 2024-09-11T22:54:36.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.614+0000 7f08c5bbd740 1 -- 172.21.15.152:0/1044143734 shutdown_connections 2024-09-11T22:54:36.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.614+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/1044143734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b487370 0x55641b489760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.614+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/1044143734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47c5d0 0x55641b47ca10 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.614+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/1044143734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47db40 0x55641b47c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.614+0000 7f08c5bbd740 1 -- 172.21.15.152:0/1044143734 >> 172.21.15.152:0/1044143734 conn(0x55641b445310 msgr2=0x55641b478c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:36.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.614+0000 7f08c5bbd740 1 -- 172.21.15.152:0/1044143734 shutdown_connections 2024-09-11T22:54:36.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.614+0000 7f08c5bbd740 1 -- 172.21.15.152:0/1044143734 wait complete. 2024-09-11T22:54:36.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.614+0000 7f08c5bbd740 1 -- start start 2024-09-11T22:54:36.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.614+0000 7f08c5bbd740 1 Processor -- start 2024-09-11T22:54:36.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.614+0000 7f08c5bbd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b47c5d0 0x55641b486da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.614+0000 7f08c5bbd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47db40 0x55641b484050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c5bbd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b487370 0x55641b4847e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c5bbd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55641b48a010 con 0x55641b47db40 2024-09-11T22:54:36.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c5bbd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55641b39d060 con 0x55641b47c5d0 2024-09-11T22:54:36.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c5bbd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55641b48a190 con 0x55641b487370 2024-09-11T22:54:36.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c5bbd740 10 client.0 _ll_register_callbacks cb 0x55641b0f08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:36.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c0d7f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47db40 0x55641b484050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c0d7f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47db40 0x55641b484050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49276/0 (socket says 172.21.15.152:49276) 2024-09-11T22:54:36.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c0d7f640 1 -- 172.21.15.152:0/3702532140 learned_addr learned my addr 172.21.15.152:0/3702532140 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:36.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c1d81640 1 --2- 172.21.15.152:0/3702532140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b487370 0x55641b4847e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c1580640 1 --2- 172.21.15.152:0/3702532140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b47c5d0 0x55641b486da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c0d7f640 1 -- 172.21.15.152:0/3702532140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b487370 msgr2=0x55641b4847e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:36.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c0d7f640 1 --2- 172.21.15.152:0/3702532140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b487370 0x55641b4847e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c0d7f640 1 -- 172.21.15.152:0/3702532140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b47c5d0 msgr2=0x55641b486da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:36.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c0d7f640 1 --2- 172.21.15.152:0/3702532140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b47c5d0 0x55641b486da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c0d7f640 1 -- 172.21.15.152:0/3702532140 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55641b1533c0 con 0x55641b47db40 2024-09-11T22:54:36.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c1580640 1 --2- 172.21.15.152:0/3702532140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b47c5d0 0x55641b486da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:54:36.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c1d81640 1 --2- 172.21.15.152:0/3702532140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b487370 0x55641b4847e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:54:36.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c0d7f640 1 --2- 172.21.15.152:0/3702532140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47db40 0x55641b484050 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009610 tx=0x7f08b80a4610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6d5a6796e1d780 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:36.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:36.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08a4ff9640 1 -- 172.21.15.152:0/3702532140 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b80b9060 con 0x55641b47db40 2024-09-11T22:54:36.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08a4ff9640 1 -- 172.21.15.152:0/3702532140 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f08b8099c10 con 0x55641b47db40 2024-09-11T22:54:36.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3702532140 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b8084e50 con 0x55641b47db40 2024-09-11T22:54:36.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08a4ff9640 1 -- 172.21.15.152:0/3702532140 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f08b80b3030 con 0x55641b47db40 2024-09-11T22:54:36.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c5bbd740 10 client.15981 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:36.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.615+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3702532140 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55641b495af0 con 0x55641b47db40 2024-09-11T22:54:36.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.616+0000 7f08a4ff9640 1 -- 172.21.15.152:0/3702532140 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b8084e50 con 0x55641b47db40 2024-09-11T22:54:36.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.616+0000 7f08a4ff9640 1 -- 172.21.15.152:0/3702532140 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f08b809cb90 con 0x55641b47db40 2024-09-11T22:54:36.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.616+0000 7f08c5bbd740 10 client.15981 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:36.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.616+0000 7f08c5bbd740 10 client.15981 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:54:36.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.616+0000 7f08c5bbd740 20 client.15981 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:36.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.616+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/3702532140 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55641b496e40 0x55641b499230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.616+0000 7f08c5bbd740 4 client.15981 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:54:36.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.616+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3702532140 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55641b3ef860 con 0x55641b496e40 2024-09-11T22:54:36.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.616+0000 7f08c1580640 1 --2- 172.21.15.152:0/3702532140 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55641b496e40 0x55641b499230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.617+0000 7f08c1580640 1 --2- 172.21.15.152:0/3702532140 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55641b496e40 0x55641b499230 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef4161139d964e7 server_cookie=f7154cf8528f38ea in_seq=0 out_seq=0 2024-09-11T22:54:36.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.617+0000 7f08a4ff9640 10 client.15981 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:54:36.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.617+0000 7f08a4ff9640 1 -- 172.21.15.152:0/3702532140 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55641b3ef860 con 0x55641b496e40 2024-09-11T22:54:36.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.617+0000 7f08a4ff9640 10 client.15981 handle_command_reply: tid=0 2024-09-11T22:54:36.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.617+0000 7f08c5bbd740 1 client.15981 shutdown 2024-09-11T22:54:36.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.618+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3702532140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47db40 msgr2=0x55641b484050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:36.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.618+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/3702532140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47db40 0x55641b484050 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009610 tx=0x7f08b80a4610 comp rx=0 tx=0).stop 2024-09-11T22:54:36.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.618+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3702532140 shutdown_connections 2024-09-11T22:54:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.618+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/3702532140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b487370 0x55641b4847e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.618+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/3702532140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47db40 0x55641b484050 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.618+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/3702532140 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55641b496e40 0x55641b499230 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.618+0000 7f08a4ff9640 0 client.15981 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:54:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.618+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/3702532140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b47c5d0 0x55641b486da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.618+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3702532140 >> 172.21.15.152:0/3702532140 conn(0x55641b445310 msgr2=0x55641b47e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.618+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3702532140 shutdown_connections 2024-09-11T22:54:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.618+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3702532140 wait complete. 2024-09-11T22:54:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.618+0000 7f08c5bbd740 20 client.15981 trim_cache size 0 max 16384 2024-09-11T22:54:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.631+0000 7f08c5bbd740 1 Processor -- start 2024-09-11T22:54:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.631+0000 7f08c5bbd740 1 -- start start 2024-09-11T22:54:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.631+0000 7f08c5bbd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47c5d0 0x55641b47f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.631+0000 7f08c5bbd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47db40 0x55641b47f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.631+0000 7f08c5bbd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b487370 0x55641b4812b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.631+0000 7f08c5bbd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc1ad750 con 0x55641b47db40 2024-09-11T22:54:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.631+0000 7f08c5bbd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc1af290 con 0x55641b487370 2024-09-11T22:54:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.631+0000 7f08c5bbd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc1ade20 con 0x55641b47c5d0 2024-09-11T22:54:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.631+0000 7f08c1580640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47c5d0 0x55641b47f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.631+0000 7f08c0d7f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47db40 0x55641b47f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.631+0000 7f08c0d7f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47db40 0x55641b47f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49282/0 (socket says 172.21.15.152:49282) 2024-09-11T22:54:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.631+0000 7f08c0d7f640 1 -- 172.21.15.152:0/3349526001 learned_addr learned my addr 172.21.15.152:0/3349526001 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.631+0000 7f08c1d81640 1 --2- 172.21.15.152:0/3349526001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b487370 0x55641b4812b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.632+0000 7f08c1580640 1 -- 172.21.15.152:0/3349526001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b487370 msgr2=0x55641b4812b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.632+0000 7f08c1580640 1 --2- 172.21.15.152:0/3349526001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b487370 0x55641b4812b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.632+0000 7f08c1580640 1 -- 172.21.15.152:0/3349526001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47db40 msgr2=0x55641b47f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.632+0000 7f08c1580640 1 --2- 172.21.15.152:0/3349526001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47db40 0x55641b47f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.632+0000 7f08c1580640 1 -- 172.21.15.152:0/3349526001 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55641b479960 con 0x55641b47c5d0 2024-09-11T22:54:36.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.632+0000 7f08c1580640 1 --2- 172.21.15.152:0/3349526001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47c5d0 0x55641b47f120 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08b0029fd0 tx=0x7f08b0013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93c2c386605cbb7d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:36.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.635+0000 7f08a6ffd640 1 -- 172.21.15.152:0/3349526001 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b00293a0 con 0x55641b47c5d0 2024-09-11T22:54:36.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.635+0000 7f08a6ffd640 1 -- 172.21.15.152:0/3349526001 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b002f070 con 0x55641b47c5d0 2024-09-11T22:54:36.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.635+0000 7f08a6ffd640 1 -- 172.21.15.152:0/3349526001 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b0011ab0 con 0x55641b47c5d0 2024-09-11T22:54:36.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.636+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3349526001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47c5d0 msgr2=0x55641b47f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:36.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.636+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/3349526001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47c5d0 0x55641b47f120 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08b0029fd0 tx=0x7f08b0013000 comp rx=0 tx=0).stop 2024-09-11T22:54:36.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.636+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3349526001 shutdown_connections 2024-09-11T22:54:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.636+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/3349526001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b487370 0x55641b4812b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.636+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/3349526001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47db40 0x55641b47f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.636+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/3349526001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47c5d0 0x55641b47f120 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.636+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3349526001 >> 172.21.15.152:0/3349526001 conn(0x55641b445310 msgr2=0x55641b4860d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.636+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3349526001 shutdown_connections 2024-09-11T22:54:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.636+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3349526001 wait complete. 2024-09-11T22:54:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.637+0000 7f08c5bbd740 1 -- start start 2024-09-11T22:54:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.637+0000 7f08c5bbd740 1 Processor -- start 2024-09-11T22:54:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.637+0000 7f08c5bbd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47c5d0 0x55641b49a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.637+0000 7f08c5bbd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47db40 0x55641b49a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.637+0000 7f08c5bbd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b487370 0x55641b49ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.637+0000 7f08c5bbd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc1aebc0 con 0x55641b47c5d0 2024-09-11T22:54:36.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.637+0000 7f08c5bbd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc1ae4f0 con 0x55641b487370 2024-09-11T22:54:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.637+0000 7f08c5bbd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc1ab710 con 0x55641b47db40 2024-09-11T22:54:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.637+0000 7f08c5bbd740 10 client.0 _ll_register_callbacks cb 0x7f08b809b540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.637+0000 7f08c1580640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47c5d0 0x55641b49a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.637+0000 7f08c1580640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47c5d0 0x55641b49a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49290/0 (socket says 172.21.15.152:49290) 2024-09-11T22:54:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.637+0000 7f08c1580640 1 -- 172.21.15.152:0/3977496974 learned_addr learned my addr 172.21.15.152:0/3977496974 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.637+0000 7f08c0d7f640 1 --2- 172.21.15.152:0/3977496974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47db40 0x55641b49a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.637+0000 7f08c1d81640 1 --2- 172.21.15.152:0/3977496974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b487370 0x55641b49ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.638+0000 7f08c1580640 1 -- 172.21.15.152:0/3977496974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47db40 msgr2=0x55641b49a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.638+0000 7f08c1580640 1 --2- 172.21.15.152:0/3977496974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47db40 0x55641b49a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.638+0000 7f08c1580640 1 -- 172.21.15.152:0/3977496974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b487370 msgr2=0x55641b49ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.638+0000 7f08c1580640 1 --2- 172.21.15.152:0/3977496974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b487370 0x55641b49ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.638+0000 7f08c1580640 1 -- 172.21.15.152:0/3977496974 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55641b500880 con 0x55641b47c5d0 2024-09-11T22:54:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.638+0000 7f08c0d7f640 1 --2- 172.21.15.152:0/3977496974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47db40 0x55641b49a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:54:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.638+0000 7f08c1580640 1 --2- 172.21.15.152:0/3977496974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47c5d0 0x55641b49a0c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f08b0011750 tx=0x7f08b00057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d842cc3f31d6f72d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.638+0000 7f08a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.638+0000 7f08a4ff9640 1 -- 172.21.15.152:0/3977496974 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b00119f0 con 0x55641b47c5d0 2024-09-11T22:54:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.638+0000 7f08a4ff9640 1 -- 172.21.15.152:0/3977496974 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f08b0011b90 con 0x55641b47c5d0 2024-09-11T22:54:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.638+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3977496974 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b002f070 con 0x55641b47c5d0 2024-09-11T22:54:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.638+0000 7f08a4ff9640 1 -- 172.21.15.152:0/3977496974 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f08b0011d90 con 0x55641b47c5d0 2024-09-11T22:54:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.638+0000 7f08c5bbd740 10 client.15987 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.638+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3977496974 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55641b501b40 con 0x55641b47c5d0 2024-09-11T22:54:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.639+0000 7f08a4ff9640 1 -- 172.21.15.152:0/3977496974 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08b002f070 con 0x55641b47c5d0 2024-09-11T22:54:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.639+0000 7f08a4ff9640 1 -- 172.21.15.152:0/3977496974 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f08b001f760 con 0x55641b47c5d0 2024-09-11T22:54:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.639+0000 7f08c5bbd740 10 client.15987 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.639+0000 7f08c5bbd740 10 client.15987 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:54:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.639+0000 7f08c5bbd740 20 client.15987 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:36.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.639+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/3977496974 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55641b496e40 0x55641b504300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:36.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.639+0000 7f08c5bbd740 4 client.15987 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:54:36.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.639+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3977496974 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55641b3e9d70 con 0x55641b496e40 2024-09-11T22:54:36.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.639+0000 7f08c0d7f640 1 --2- 172.21.15.152:0/3977496974 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55641b496e40 0x55641b504300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:36.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08c0d7f640 1 --2- 172.21.15.152:0/3977496974 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55641b496e40 0x55641b504300 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1dcfdf69fd9e419 server_cookie=50d732957d18baf in_seq=0 out_seq=0 2024-09-11T22:54:36.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08a4ff9640 10 client.15987 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:54:36.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08a4ff9640 1 -- 172.21.15.152:0/3977496974 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55641b3e9d70 con 0x55641b496e40 2024-09-11T22:54:36.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08a4ff9640 10 client.15987 handle_command_reply: tid=0 2024-09-11T22:54:36.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08c5bbd740 1 client.15987 shutdown 2024-09-11T22:54:36.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3977496974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47c5d0 msgr2=0x55641b49a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:36.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/3977496974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47c5d0 0x55641b49a0c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f08b0011750 tx=0x7f08b00057e0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3977496974 shutdown_connections 2024-09-11T22:54:36.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/3977496974 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55641b496e40 0x55641b504300 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/3977496974 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55641b487370 0x55641b49ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/3977496974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55641b47db40 0x55641b49a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08c5bbd740 1 --2- 172.21.15.152:0/3977496974 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55641b47c5d0 0x55641b49a0c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3977496974 >> 172.21.15.152:0/3977496974 conn(0x55641b445310 msgr2=0x55641b4858b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:36.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08a4ff9640 0 client.15987 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:54:36.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3977496974 shutdown_connections 2024-09-11T22:54:36.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08c5bbd740 1 -- 172.21.15.152:0/3977496974 wait complete. 2024-09-11T22:54:36.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.640+0000 7f08c5bbd740 20 client.15987 trim_cache size 0 max 16384 2024-09-11T22:54:36.654 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:54:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.641+0000 7f08c37e4640 1 -- 172.21.15.152:0/3751038451 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08a80a2cf0 msgr2=0x7f08a80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.641+0000 7f08c37e4640 1 --2- 172.21.15.152:0/3751038451 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08a80a2cf0 0x7f08a80a51b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f08b0005fd0 tx=0x7f08b000b390 comp rx=0 tx=0).stop 2024-09-11T22:54:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.641+0000 7f08c37e4640 1 -- 172.21.15.152:0/3751038451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc102570 msgr2=0x7f08bc1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.641+0000 7f08c37e4640 1 --2- 172.21.15.152:0/3751038451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc102570 0x7f08bc1989d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f08b800d950 tx=0x7f08b800de10 comp rx=0 tx=0).stop 2024-09-11T22:54:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.641+0000 7f08c37e4640 1 -- 172.21.15.152:0/3751038451 shutdown_connections 2024-09-11T22:54:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.641+0000 7f08c37e4640 1 --2- 172.21.15.152:0/3751038451 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08a80a2cf0 0x7f08a80a51b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.641+0000 7f08c37e4640 1 --2- 172.21.15.152:0/3751038451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08bc102ef0 0x7f08bc19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.641+0000 7f08c37e4640 1 --2- 172.21.15.152:0/3751038451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08bc102570 0x7f08bc1989d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.641+0000 7f08c37e4640 1 --2- 172.21.15.152:0/3751038451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08bc100e20 0x7f08bc198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:36.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.641+0000 7f08c37e4640 1 -- 172.21.15.152:0/3751038451 >> 172.21.15.152:0/3751038451 conn(0x7f08bc0fc7d0 msgr2=0x7f08bc06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:36.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.641+0000 7f08c37e4640 1 -- 172.21.15.152:0/3751038451 shutdown_connections 2024-09-11T22:54:36.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:36.641+0000 7f08c37e4640 1 -- 172.21.15.152:0/3751038451 wait complete. 2024-09-11T22:54:36.794 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:36 smithi152 ceph-mon[31181]: pgmap v54: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 85 MiB/s wr, 38 op/s 2024-09-11T22:54:37.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:36 smithi191 ceph-mon[35211]: pgmap v54: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 85 MiB/s wr, 38 op/s 2024-09-11T22:54:37.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:36 smithi204 ceph-mon[35454]: pgmap v54: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 85 MiB/s wr, 38 op/s 2024-09-11T22:54:37.317 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:54:37.629 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:54:38.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.161+0000 7f346fffe640 1 -- 172.21.15.152:0/2266292063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3468106f40 msgr2=0x7f346810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:38.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.161+0000 7f346fffe640 1 --2- 172.21.15.152:0/2266292063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3468106f40 0x7f346810eac0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3460009ca0 tx=0x7f346002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.161+0000 7f346fffe640 1 -- 172.21.15.152:0/2266292063 shutdown_connections 2024-09-11T22:54:38.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.161+0000 7f346fffe640 1 --2- 172.21.15.152:0/2266292063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3468106f40 0x7f346810eac0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.161+0000 7f346fffe640 1 --2- 172.21.15.152:0/2266292063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34681027c0 0x7f3468106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.161+0000 7f346fffe640 1 --2- 172.21.15.152:0/2266292063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3468101e10 0x7f34681021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.161+0000 7f346fffe640 1 -- 172.21.15.152:0/2266292063 >> 172.21.15.152:0/2266292063 conn(0x7f34680fd8d0 msgr2=0x7f34680ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:38.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.162+0000 7f346fffe640 1 -- 172.21.15.152:0/2266292063 shutdown_connections 2024-09-11T22:54:38.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.162+0000 7f346fffe640 1 -- 172.21.15.152:0/2266292063 wait complete. 2024-09-11T22:54:38.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.162+0000 7f346fffe640 1 Processor -- start 2024-09-11T22:54:38.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.162+0000 7f346fffe640 1 -- start start 2024-09-11T22:54:38.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.162+0000 7f346fffe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3468101e10 0x7f346819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.162+0000 7f346fffe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34681027c0 0x7f346819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.163+0000 7f346fffe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3468106f40 0x7f346819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.163+0000 7f346fffe640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3468111fa0 con 0x7f3468101e10 2024-09-11T22:54:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.163+0000 7f346fffe640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3468111e20 con 0x7f34681027c0 2024-09-11T22:54:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.163+0000 7f346fffe640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3468112120 con 0x7f3468106f40 2024-09-11T22:54:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.163+0000 7f346dd9a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3468101e10 0x7f346819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.163+0000 7f346dd9a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3468101e10 0x7f346819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49318/0 (socket says 172.21.15.152:49318) 2024-09-11T22:54:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.163+0000 7f346dd9a640 1 -- 172.21.15.152:0/1874081128 learned_addr learned my addr 172.21.15.152:0/1874081128 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.163+0000 7f346e59b640 1 --2- 172.21.15.152:0/1874081128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3468106f40 0x7f346819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.163+0000 7f346d599640 1 --2- 172.21.15.152:0/1874081128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34681027c0 0x7f346819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.163+0000 7f346e59b640 1 -- 172.21.15.152:0/1874081128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34681027c0 msgr2=0x7f346819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.163+0000 7f346e59b640 1 --2- 172.21.15.152:0/1874081128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34681027c0 0x7f346819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.163+0000 7f346e59b640 1 -- 172.21.15.152:0/1874081128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3468101e10 msgr2=0x7f346819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.163+0000 7f346e59b640 1 --2- 172.21.15.152:0/1874081128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3468101e10 0x7f346819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.163+0000 7f346e59b640 1 -- 172.21.15.152:0/1874081128 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346819f570 con 0x7f3468106f40 2024-09-11T22:54:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.163+0000 7f346dd9a640 1 --2- 172.21.15.152:0/1874081128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3468101e10 0x7f346819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:54:38.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.163+0000 7f346e59b640 1 --2- 172.21.15.152:0/1874081128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3468106f40 0x7f346819ee90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f346002ade0 tx=0x7f34600319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.164+0000 7f345effd640 1 -- 172.21.15.152:0/1874081128 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460003b70 con 0x7f3468106f40 2024-09-11T22:54:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.164+0000 7f345effd640 1 -- 172.21.15.152:0/1874081128 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3460003d10 con 0x7f3468106f40 2024-09-11T22:54:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.164+0000 7f346fffe640 1 -- 172.21.15.152:0/1874081128 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346819f800 con 0x7f3468106f40 2024-09-11T22:54:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.164+0000 7f345effd640 1 -- 172.21.15.152:0/1874081128 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346002fcb0 con 0x7f3468106f40 2024-09-11T22:54:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.164+0000 7f346fffe640 1 -- 172.21.15.152:0/1874081128 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34681a7140 con 0x7f3468106f40 2024-09-11T22:54:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.166+0000 7f345effd640 1 -- 172.21.15.152:0/1874081128 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3460004050 con 0x7f3468106f40 2024-09-11T22:54:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.166+0000 7f34723d7740 1 Processor -- start 2024-09-11T22:54:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.167+0000 7f34723d7740 1 -- start start 2024-09-11T22:54:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.167+0000 7f345effd640 1 --2- 172.21.15.152:0/1874081128 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f34540a2cf0 0x7f34540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.167+0000 7f34723d7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5edb40 0x560fea5ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.167+0000 7f34723d7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5ec5d0 0x560fea5eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.167+0000 7f34723d7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5f7370 0x560fea5f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.167+0000 7f34723d7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560fea5522f0 con 0x560fea5f7370 2024-09-11T22:54:38.186 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.167+0000 7f34723d7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560fea4f9cf0 con 0x560fea5edb40 2024-09-11T22:54:38.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.167+0000 7f34723d7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560fea54e2b0 con 0x560fea5ec5d0 2024-09-11T22:54:38.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.167+0000 7f345effd640 1 -- 172.21.15.152:0/1874081128 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f34600f04b0 con 0x7f3468106f40 2024-09-11T22:54:38.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.167+0000 7f346d599640 1 --2- 172.21.15.152:0/1874081128 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f34540a2cf0 0x7f34540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.168+0000 7f346dd9a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5edb40 0x560fea5ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.168+0000 7f346dd9a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5edb40 0x560fea5ec090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:43296/0 (socket says 172.21.15.152:43296) 2024-09-11T22:54:38.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.168+0000 7f346dd9a640 1 -- 172.21.15.152:0/2754605228 learned_addr learned my addr 172.21.15.152:0/2754605228 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:38.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.168+0000 7f346e59b640 1 --2- 172.21.15.152:0/2754605228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5f7370 0x560fea5f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.168+0000 7f346dd9a640 1 -- 172.21.15.152:0/2754605228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5ec5d0 msgr2=0x560fea5eca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:38.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.168+0000 7f346d599640 1 --2- 172.21.15.152:0/2754605228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5ec5d0 0x560fea5eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.168+0000 7f346dd9a640 1 --2- 172.21.15.152:0/2754605228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5ec5d0 0x560fea5eca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.168+0000 7f346dd9a640 1 -- 172.21.15.152:0/2754605228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5f7370 msgr2=0x560fea5f9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:38.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.168+0000 7f346dd9a640 1 --2- 172.21.15.152:0/2754605228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5f7370 0x560fea5f9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.168+0000 7f346dd9a640 1 -- 172.21.15.152:0/2754605228 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fea4de3a0 con 0x560fea5edb40 2024-09-11T22:54:38.187 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.168+0000 7f346e59b640 1 --2- 172.21.15.152:0/2754605228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5f7370 0x560fea5f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:54:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.168+0000 7f346d599640 1 --2- 172.21.15.152:0/2754605228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5ec5d0 0x560fea5eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:54:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.168+0000 7f346d599640 1 --2- 172.21.15.152:0/1874081128 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f34540a2cf0 0x7f34540a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f346810eee0 tx=0x7f345800bf80 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.168+0000 7f346dd9a640 1 --2- 172.21.15.152:0/2754605228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5edb40 0x560fea5ec090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3464011650 tx=0x7f3464009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a3d99adbb73e9df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.169+0000 7f3453fff640 1 -- 172.21.15.152:0/2754605228 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346401e070 con 0x560fea5edb40 2024-09-11T22:54:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.169+0000 7f3453fff640 1 -- 172.21.15.152:0/2754605228 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f346400fba0 con 0x560fea5edb40 2024-09-11T22:54:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.169+0000 7f3453fff640 1 -- 172.21.15.152:0/2754605228 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464019cd0 con 0x560fea5edb40 2024-09-11T22:54:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.169+0000 7f34723d7740 1 -- 172.21.15.152:0/2754605228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5edb40 msgr2=0x560fea5ec090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.169+0000 7f34723d7740 1 --2- 172.21.15.152:0/2754605228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5edb40 0x560fea5ec090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3464011650 tx=0x7f3464009f90 comp rx=0 tx=0).stop 2024-09-11T22:54:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.169+0000 7f34723d7740 1 -- 172.21.15.152:0/2754605228 shutdown_connections 2024-09-11T22:54:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.169+0000 7f34723d7740 1 --2- 172.21.15.152:0/2754605228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5f7370 0x560fea5f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.169+0000 7f34723d7740 1 --2- 172.21.15.152:0/2754605228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5ec5d0 0x560fea5eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.169+0000 7f34723d7740 1 --2- 172.21.15.152:0/2754605228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5edb40 0x560fea5ec090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.169+0000 7f34723d7740 1 -- 172.21.15.152:0/2754605228 >> 172.21.15.152:0/2754605228 conn(0x560fea5b5310 msgr2=0x560fea5e8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:38.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.170+0000 7f34723d7740 1 -- 172.21.15.152:0/2754605228 shutdown_connections 2024-09-11T22:54:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.170+0000 7f34723d7740 1 -- 172.21.15.152:0/2754605228 wait complete. 2024-09-11T22:54:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.170+0000 7f34723d7740 1 -- start start 2024-09-11T22:54:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.170+0000 7f34723d7740 1 Processor -- start 2024-09-11T22:54:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.171+0000 7f34723d7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5ec5d0 0x560fea5f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.171+0000 7f34723d7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5edb40 0x560fea5f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.171+0000 7f34723d7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5f7370 0x560fea5f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.171+0000 7f346dd9a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5ec5d0 0x560fea5f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.171+0000 7f346dd9a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5ec5d0 0x560fea5f6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:50866/0 (socket says 172.21.15.152:50866) 2024-09-11T22:54:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.171+0000 7f346dd9a640 1 -- 172.21.15.152:0/2720882703 learned_addr learned my addr 172.21.15.152:0/2720882703 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.171+0000 7f34723d7740 1 -- 172.21.15.152:0/2720882703 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560fea602880 con 0x560fea5edb40 2024-09-11T22:54:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.171+0000 7f346d599640 1 --2- 172.21.15.152:0/2720882703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5edb40 0x560fea5f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.171+0000 7f346e59b640 1 --2- 172.21.15.152:0/2720882703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5f7370 0x560fea5f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.171+0000 7f34723d7740 1 -- 172.21.15.152:0/2720882703 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560fea50d060 con 0x560fea5f7370 2024-09-11T22:54:38.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.172+0000 7f34723d7740 1 -- 172.21.15.152:0/2720882703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560fea602a00 con 0x560fea5ec5d0 2024-09-11T22:54:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.172+0000 7f34723d7740 10 client.0 _ll_register_callbacks cb 0x560fea2608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.172+0000 7f346dd9a640 1 -- 172.21.15.152:0/2720882703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5f7370 msgr2=0x560fea5f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.172+0000 7f346dd9a640 1 --2- 172.21.15.152:0/2720882703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5f7370 0x560fea5f45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.172+0000 7f346dd9a640 1 -- 172.21.15.152:0/2720882703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5edb40 msgr2=0x560fea5f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.172+0000 7f346dd9a640 1 --2- 172.21.15.152:0/2720882703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5edb40 0x560fea5f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.172+0000 7f346dd9a640 1 -- 172.21.15.152:0/2720882703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fea2c33c0 con 0x560fea5ec5d0 2024-09-11T22:54:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.172+0000 7f346dd9a640 1 --2- 172.21.15.152:0/2720882703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5ec5d0 0x560fea5f6d30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3464019850 tx=0x7f346400fad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2fb614a9f76a7f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.173+0000 7f34517fa640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-11T22:54:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.173+0000 7f34517fa640 1 -- 172.21.15.152:0/2720882703 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346401e040 con 0x560fea5ec5d0 2024-09-11T22:54:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.173+0000 7f34723d7740 1 -- 172.21.15.152:0/2720882703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f346400fba0 con 0x560fea5ec5d0 2024-09-11T22:54:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.173+0000 7f34517fa640 1 -- 172.21.15.152:0/2720882703 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f3464009cc0 con 0x560fea5ec5d0 2024-09-11T22:54:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.173+0000 7f34517fa640 1 -- 172.21.15.152:0/2720882703 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f346400fd50 con 0x560fea5ec5d0 2024-09-11T22:54:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.173+0000 7f34723d7740 10 client.25381 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:38.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.173+0000 7f34723d7740 1 -- 172.21.15.152:0/2720882703 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560fea60e230 con 0x560fea5ec5d0 2024-09-11T22:54:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.173+0000 7f34517fa640 1 -- 172.21.15.152:0/2720882703 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f346400fba0 con 0x560fea5ec5d0 2024-09-11T22:54:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.174+0000 7f34517fa640 1 -- 172.21.15.152:0/2720882703 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3464021070 con 0x560fea5ec5d0 2024-09-11T22:54:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.174+0000 7f34723d7740 10 client.25381 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.174+0000 7f34723d7740 10 client.25381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:54:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.174+0000 7f34723d7740 20 client.25381 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.174+0000 7f34723d7740 1 --2- 172.21.15.152:0/2720882703 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560fea60f580 0x560fea611970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.174+0000 7f34723d7740 4 client.25381 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:54:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.174+0000 7f34723d7740 1 -- 172.21.15.152:0/2720882703 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fea55f860 con 0x560fea60f580 2024-09-11T22:54:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.174+0000 7f346d599640 1 --2- 172.21.15.152:0/2720882703 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560fea60f580 0x560fea611970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.175+0000 7f346d599640 1 --2- 172.21.15.152:0/2720882703 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560fea60f580 0x560fea611970 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=356b92707d08a01d server_cookie=1c90d4075e5d7f9f in_seq=0 out_seq=0 2024-09-11T22:54:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.175+0000 7f34517fa640 10 client.25381 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.175+0000 7f34517fa640 1 -- 172.21.15.152:0/2720882703 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560fea55f860 con 0x560fea60f580 2024-09-11T22:54:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.175+0000 7f34517fa640 10 client.25381 handle_command_reply: tid=0 2024-09-11T22:54:38.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.175+0000 7f34723d7740 1 client.25381 shutdown 2024-09-11T22:54:38.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.176+0000 7f34723d7740 1 -- 172.21.15.152:0/2720882703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5ec5d0 msgr2=0x560fea5f6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:38.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.176+0000 7f34723d7740 1 --2- 172.21.15.152:0/2720882703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5ec5d0 0x560fea5f6d30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3464019850 tx=0x7f346400fad0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.176+0000 7f34723d7740 1 -- 172.21.15.152:0/2720882703 shutdown_connections 2024-09-11T22:54:38.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.176+0000 7f34723d7740 1 --2- 172.21.15.152:0/2720882703 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560fea60f580 0x560fea611970 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.176+0000 7f34723d7740 1 --2- 172.21.15.152:0/2720882703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5f7370 0x560fea5f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.176+0000 7f34517fa640 0 client.25381 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:38.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.176+0000 7f34723d7740 1 --2- 172.21.15.152:0/2720882703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5edb40 0x560fea5f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.176+0000 7f34723d7740 1 --2- 172.21.15.152:0/2720882703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5ec5d0 0x560fea5f6d30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.176+0000 7f34723d7740 1 -- 172.21.15.152:0/2720882703 >> 172.21.15.152:0/2720882703 conn(0x560fea5b5310 msgr2=0x560fea5ee960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:38.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.176+0000 7f34723d7740 1 -- 172.21.15.152:0/2720882703 shutdown_connections 2024-09-11T22:54:38.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.176+0000 7f34723d7740 1 -- 172.21.15.152:0/2720882703 wait complete. 2024-09-11T22:54:38.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.176+0000 7f34723d7740 20 client.25381 trim_cache size 0 max 16384 2024-09-11T22:54:38.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.191+0000 7f34723d7740 1 Processor -- start 2024-09-11T22:54:38.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.191+0000 7f34723d7740 1 -- start start 2024-09-11T22:54:38.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.191+0000 7f34723d7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5ec5d0 0x560fea5e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.191+0000 7f34723d7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5edb40 0x560fea5ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.191+0000 7f34723d7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5f7370 0x560fea5ef660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.191+0000 7f34723d7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f34681ab5a0 con 0x560fea5ec5d0 2024-09-11T22:54:38.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.191+0000 7f34723d7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f34681ad0e0 con 0x560fea5edb40 2024-09-11T22:54:38.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.191+0000 7f34723d7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34681abc70 con 0x560fea5f7370 2024-09-11T22:54:38.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.191+0000 7f346dd9a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5ec5d0 0x560fea5e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.191+0000 7f346dd9a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5ec5d0 0x560fea5e7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49344/0 (socket says 172.21.15.152:49344) 2024-09-11T22:54:38.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.191+0000 7f346dd9a640 1 -- 172.21.15.152:0/3009835777 learned_addr learned my addr 172.21.15.152:0/3009835777 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:38.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.191+0000 7f346e59b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5f7370 0x560fea5ef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.192+0000 7f346d599640 1 --2- 172.21.15.152:0/3009835777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5edb40 0x560fea5ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.192+0000 7f346e59b640 1 -- 172.21.15.152:0/3009835777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5edb40 msgr2=0x560fea5ef120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:38.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.192+0000 7f346e59b640 1 --2- 172.21.15.152:0/3009835777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5edb40 0x560fea5ef120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.192+0000 7f346e59b640 1 -- 172.21.15.152:0/3009835777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5ec5d0 msgr2=0x560fea5e7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:38.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.192+0000 7f346e59b640 1 --2- 172.21.15.152:0/3009835777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5ec5d0 0x560fea5e7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.192+0000 7f346e59b640 1 -- 172.21.15.152:0/3009835777 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fea5f6950 con 0x560fea5f7370 2024-09-11T22:54:38.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.192+0000 7f346e59b640 1 --2- 172.21.15.152:0/3009835777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5f7370 0x560fea5ef660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f34600edd60 tx=0x7f346003d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=356453d1c5e7fb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:38.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.192+0000 7f34537fe640 1 -- 172.21.15.152:0/3009835777 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34600bde60 con 0x560fea5f7370 2024-09-11T22:54:38.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.192+0000 7f34537fe640 1 -- 172.21.15.152:0/3009835777 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3460036040 con 0x560fea5f7370 2024-09-11T22:54:38.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.192+0000 7f34537fe640 1 -- 172.21.15.152:0/3009835777 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34600bae30 con 0x560fea5f7370 2024-09-11T22:54:38.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.193+0000 7f34723d7740 1 -- 172.21.15.152:0/3009835777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5f7370 msgr2=0x560fea5ef660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:38.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.193+0000 7f34723d7740 1 --2- 172.21.15.152:0/3009835777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5f7370 0x560fea5ef660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f34600edd60 tx=0x7f346003d040 comp rx=0 tx=0).stop 2024-09-11T22:54:38.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.193+0000 7f34723d7740 1 -- 172.21.15.152:0/3009835777 shutdown_connections 2024-09-11T22:54:38.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.193+0000 7f34723d7740 1 --2- 172.21.15.152:0/3009835777 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5f7370 0x560fea5ef660 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.193+0000 7f34723d7740 1 --2- 172.21.15.152:0/3009835777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5edb40 0x560fea5ef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.193+0000 7f34723d7740 1 --2- 172.21.15.152:0/3009835777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5ec5d0 0x560fea5e7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.193+0000 7f34723d7740 1 -- 172.21.15.152:0/3009835777 >> 172.21.15.152:0/3009835777 conn(0x560fea5b5310 msgr2=0x560fea5e7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:38.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.193+0000 7f34723d7740 1 -- 172.21.15.152:0/3009835777 shutdown_connections 2024-09-11T22:54:38.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.193+0000 7f34723d7740 1 -- 172.21.15.152:0/3009835777 wait complete. 2024-09-11T22:54:38.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.194+0000 7f34723d7740 1 -- start start 2024-09-11T22:54:38.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.194+0000 7f34723d7740 1 Processor -- start 2024-09-11T22:54:38.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.194+0000 7f34723d7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5ec5d0 0x560fea6080d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f34723d7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5edb40 0x560fea603400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f34723d7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5f7370 0x560fea603940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f34723d7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f34681aca10 con 0x560fea5edb40 2024-09-11T22:54:38.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f34723d7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f34681ac340 con 0x560fea5ec5d0 2024-09-11T22:54:38.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f34723d7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34681a9580 con 0x560fea5f7370 2024-09-11T22:54:38.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f346d599640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5edb40 0x560fea603400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f346dd9a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5ec5d0 0x560fea6080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f346d599640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5edb40 0x560fea603400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:49348/0 (socket says 172.21.15.152:49348) 2024-09-11T22:54:38.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f346d599640 1 -- 172.21.15.152:0/1647994441 learned_addr learned my addr 172.21.15.152:0/1647994441 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:38.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f346dd9a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5ec5d0 0x560fea6080d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:43326/0 (socket says 172.21.15.152:43326) 2024-09-11T22:54:38.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f346e59b640 1 --2- 172.21.15.152:0/1647994441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5f7370 0x560fea603940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f34723d7740 10 client.0 _ll_register_callbacks cb 0x7f346401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:38.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f346dd9a640 1 -- 172.21.15.152:0/1647994441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5f7370 msgr2=0x560fea603940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:38.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f346dd9a640 1 --2- 172.21.15.152:0/1647994441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5f7370 0x560fea603940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f346dd9a640 1 -- 172.21.15.152:0/1647994441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5edb40 msgr2=0x560fea603400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:38.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f346dd9a640 1 --2- 172.21.15.152:0/1647994441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5edb40 0x560fea603400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f346dd9a640 1 -- 172.21.15.152:0/1647994441 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fea603e80 con 0x560fea5ec5d0 2024-09-11T22:54:38.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.195+0000 7f346d599640 1 --2- 172.21.15.152:0/1647994441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5edb40 0x560fea603400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:54:38.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.196+0000 7f346dd9a640 1 --2- 172.21.15.152:0/1647994441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5ec5d0 0x560fea6080d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3464001eb0 tx=0x7f346400fef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=283e447b7f0d766 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:38.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.196+0000 7f34517fa640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-11T22:54:38.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.196+0000 7f34517fa640 1 -- 172.21.15.152:0/1647994441 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346401e040 con 0x560fea5ec5d0 2024-09-11T22:54:38.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.196+0000 7f34517fa640 1 -- 172.21.15.152:0/1647994441 <== mon.2 v2:172.21.15.191:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f34640052b0 con 0x560fea5ec5d0 2024-09-11T22:54:38.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.196+0000 7f34723d7740 1 -- 172.21.15.152:0/1647994441 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3460036040 con 0x560fea5ec5d0 2024-09-11T22:54:38.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.196+0000 7f34517fa640 1 -- 172.21.15.152:0/1647994441 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3464038020 con 0x560fea5ec5d0 2024-09-11T22:54:38.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.196+0000 7f34723d7740 10 client.25337 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:38.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.196+0000 7f34723d7740 1 -- 172.21.15.152:0/1647994441 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560fea679e00 con 0x560fea5ec5d0 2024-09-11T22:54:38.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.196+0000 7f34517fa640 1 -- 172.21.15.152:0/1647994441 <== mon.2 v2:172.21.15.191:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34640054b0 con 0x560fea5ec5d0 2024-09-11T22:54:38.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.196+0000 7f34517fa640 1 -- 172.21.15.152:0/1647994441 <== mon.2 v2:172.21.15.191:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3464021040 con 0x560fea5ec5d0 2024-09-11T22:54:38.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.197+0000 7f34723d7740 10 client.25337 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:38.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.197+0000 7f34723d7740 10 client.25337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:54:38.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.197+0000 7f34723d7740 20 client.25337 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:38.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.197+0000 7f34723d7740 1 --2- 172.21.15.152:0/1647994441 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560fea60f580 0x560fea67c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:38.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.197+0000 7f34723d7740 4 client.25337 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:54:38.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.197+0000 7f34723d7740 1 -- 172.21.15.152:0/1647994441 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f3460036040 con 0x560fea60f580 2024-09-11T22:54:38.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.197+0000 7f346d599640 1 --2- 172.21.15.152:0/1647994441 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560fea60f580 0x560fea67c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:38.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.197+0000 7f34517fa640 10 client.25337 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:38.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.197+0000 7f346d599640 1 --2- 172.21.15.152:0/1647994441 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560fea60f580 0x560fea67c410 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a8ff39e174746b0 server_cookie=ddd6d18d0d5a41a in_seq=0 out_seq=0 2024-09-11T22:54:38.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.197+0000 7f34517fa640 1 -- 172.21.15.152:0/1647994441 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f3460036040 con 0x560fea60f580 2024-09-11T22:54:38.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.197+0000 7f34517fa640 10 client.25337 handle_command_reply: tid=0 2024-09-11T22:54:38.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.197+0000 7f34723d7740 1 client.25337 shutdown 2024-09-11T22:54:38.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.198+0000 7f34723d7740 1 -- 172.21.15.152:0/1647994441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5ec5d0 msgr2=0x560fea6080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:38.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.198+0000 7f34723d7740 1 --2- 172.21.15.152:0/1647994441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5ec5d0 0x560fea6080d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3464001eb0 tx=0x7f346400fef0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.198+0000 7f34723d7740 1 -- 172.21.15.152:0/1647994441 shutdown_connections 2024-09-11T22:54:38.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.198+0000 7f34723d7740 1 --2- 172.21.15.152:0/1647994441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560fea5f7370 0x560fea603940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.198+0000 7f34723d7740 1 --2- 172.21.15.152:0/1647994441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fea5edb40 0x560fea603400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.198+0000 7f34723d7740 1 --2- 172.21.15.152:0/1647994441 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560fea60f580 0x560fea67c410 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.198+0000 7f34723d7740 1 --2- 172.21.15.152:0/1647994441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560fea5ec5d0 0x560fea6080d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.198+0000 7f34517fa640 0 client.25337 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:38.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.198+0000 7f34723d7740 1 -- 172.21.15.152:0/1647994441 >> 172.21.15.152:0/1647994441 conn(0x560fea5b5310 msgr2=0x560fea5f6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:38.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.198+0000 7f34723d7740 1 -- 172.21.15.152:0/1647994441 shutdown_connections 2024-09-11T22:54:38.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.198+0000 7f34723d7740 1 -- 172.21.15.152:0/1647994441 wait complete. 2024-09-11T22:54:38.204 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.198+0000 7f34723d7740 20 client.25337 trim_cache size 0 max 16384 2024-09-11T22:54:38.204 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:54:38.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.199+0000 7f346fffe640 1 -- 172.21.15.152:0/1874081128 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f34540a2cf0 msgr2=0x7f34540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:38.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.199+0000 7f346fffe640 1 --2- 172.21.15.152:0/1874081128 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f34540a2cf0 0x7f34540a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f346810eee0 tx=0x7f345800bf80 comp rx=0 tx=0).stop 2024-09-11T22:54:38.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.199+0000 7f346fffe640 1 -- 172.21.15.152:0/1874081128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3468106f40 msgr2=0x7f346819ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:38.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.199+0000 7f346fffe640 1 --2- 172.21.15.152:0/1874081128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3468106f40 0x7f346819ee90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f346002ade0 tx=0x7f34600319f0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.199+0000 7f346fffe640 1 -- 172.21.15.152:0/1874081128 shutdown_connections 2024-09-11T22:54:38.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.199+0000 7f346fffe640 1 --2- 172.21.15.152:0/1874081128 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f34540a2cf0 0x7f34540a51b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.199+0000 7f346fffe640 1 --2- 172.21.15.152:0/1874081128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3468106f40 0x7f346819ee90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.199+0000 7f346fffe640 1 --2- 172.21.15.152:0/1874081128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34681027c0 0x7f346819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.199+0000 7f346fffe640 1 --2- 172.21.15.152:0/1874081128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3468101e10 0x7f346819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:38.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.199+0000 7f346fffe640 1 -- 172.21.15.152:0/1874081128 >> 172.21.15.152:0/1874081128 conn(0x7f34680fd8d0 msgr2=0x7f34680ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:38.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.199+0000 7f346fffe640 1 -- 172.21.15.152:0/1874081128 shutdown_connections 2024-09-11T22:54:38.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:38.199+0000 7f346fffe640 1 -- 172.21.15.152:0/1874081128 wait complete. 2024-09-11T22:54:38.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:38 smithi152 ceph-mon[31181]: pgmap v55: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-11T22:54:39.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:38 smithi191 ceph-mon[35211]: pgmap v55: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-11T22:54:39.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:38 smithi204 ceph-mon[35454]: pgmap v55: 97 pgs: 97 active+clean; 1.0 GiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-11T22:54:39.453 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:39.453 INFO:tasks.workunit.client.0.smithi152.stdout: Children see throughput for 1 rewriters = 748208.50 kB/sec 2024-09-11T22:54:39.453 INFO:tasks.workunit.client.0.smithi152.stdout: Parent sees throughput for 1 rewriters = 748101.60 kB/sec 2024-09-11T22:54:39.453 INFO:tasks.workunit.client.0.smithi152.stdout: Min throughput per process = 748208.50 kB/sec 2024-09-11T22:54:39.454 INFO:tasks.workunit.client.0.smithi152.stdout: Max throughput per process = 748208.50 kB/sec 2024-09-11T22:54:39.454 INFO:tasks.workunit.client.0.smithi152.stdout: Avg throughput per process = 748208.50 kB/sec 2024-09-11T22:54:39.454 INFO:tasks.workunit.client.0.smithi152.stdout: Min xfer = 1048576.00 kB 2024-09-11T22:54:39.936 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:54:39.937 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:54:40.200 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:54:40.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.809+0000 7f47ad2e7640 1 -- 172.21.15.152:0/4164139333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a8102ef0 msgr2=0x7f47a810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:40.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.809+0000 7f47ad2e7640 1 --2- 172.21.15.152:0/4164139333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a8102ef0 0x7f47a810da40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009ca0 tx=0x7f47a002a950 comp rx=0 tx=0).stop 2024-09-11T22:54:40.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.809+0000 7f47ad2e7640 1 -- 172.21.15.152:0/4164139333 shutdown_connections 2024-09-11T22:54:40.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.809+0000 7f47ad2e7640 1 --2- 172.21.15.152:0/4164139333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a8102ef0 0x7f47a810da40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.809+0000 7f47ad2e7640 1 --2- 172.21.15.152:0/4164139333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47a8102570 0x7f47a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.809+0000 7f47ad2e7640 1 --2- 172.21.15.152:0/4164139333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47a8100e20 0x7f47a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.809+0000 7f47ad2e7640 1 -- 172.21.15.152:0/4164139333 >> 172.21.15.152:0/4164139333 conn(0x7f47a80fc7d0 msgr2=0x7f47a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:40.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.809+0000 7f47ad2e7640 1 -- 172.21.15.152:0/4164139333 shutdown_connections 2024-09-11T22:54:40.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.809+0000 7f47ad2e7640 1 -- 172.21.15.152:0/4164139333 wait complete. 2024-09-11T22:54:40.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.810+0000 7f47ad2e7640 1 Processor -- start 2024-09-11T22:54:40.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.810+0000 7f47ad2e7640 1 -- start start 2024-09-11T22:54:40.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.810+0000 7f47ad2e7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a8100e20 0x7f47a8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.810+0000 7f47ad2e7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47a8102570 0x7f47a8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.810+0000 7f47ad2e7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47a8102ef0 0x7f47a819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.810+0000 7f47ad2e7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8110f80 con 0x7f47a8100e20 2024-09-11T22:54:40.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.810+0000 7f47ad2e7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8110e00 con 0x7f47a8102570 2024-09-11T22:54:40.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.810+0000 7f47ad2e7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8111100 con 0x7f47a8102ef0 2024-09-11T22:54:40.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.810+0000 7f47a6ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a8100e20 0x7f47a8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.810+0000 7f47a6ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a8100e20 0x7f47a8198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50200/0 (socket says 172.21.15.152:50200) 2024-09-11T22:54:40.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.811+0000 7f47a6ffd640 1 -- 172.21.15.152:0/91044571 learned_addr learned my addr 172.21.15.152:0/91044571 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:40.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.811+0000 7f47a77fe640 1 --2- 172.21.15.152:0/91044571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47a8102ef0 0x7f47a819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.811+0000 7f479e7fc640 1 --2- 172.21.15.152:0/91044571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47a8102570 0x7f47a8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.811+0000 7f47a6ffd640 1 -- 172.21.15.152:0/91044571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47a8102ef0 msgr2=0x7f47a819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:40.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.811+0000 7f47a6ffd640 1 --2- 172.21.15.152:0/91044571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47a8102ef0 0x7f47a819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.811+0000 7f47a6ffd640 1 -- 172.21.15.152:0/91044571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47a8102570 msgr2=0x7f47a8198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:40.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.811+0000 7f47a6ffd640 1 --2- 172.21.15.152:0/91044571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47a8102570 0x7f47a8198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.811+0000 7f47a6ffd640 1 -- 172.21.15.152:0/91044571 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a819d4d0 con 0x7f47a8100e20 2024-09-11T22:54:40.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.811+0000 7f47a6ffd640 1 --2- 172.21.15.152:0/91044571 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a8100e20 0x7f47a8198520 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f479800d950 tx=0x7f479800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:40.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.811+0000 7f47a4ff9640 1 -- 172.21.15.152:0/91044571 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798017030 con 0x7f47a8100e20 2024-09-11T22:54:40.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.811+0000 7f47ad2e7640 1 -- 172.21.15.152:0/91044571 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a819d7c0 con 0x7f47a8100e20 2024-09-11T22:54:40.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.811+0000 7f47a4ff9640 1 -- 172.21.15.152:0/91044571 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4798004020 con 0x7f47a8100e20 2024-09-11T22:54:40.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.811+0000 7f47a4ff9640 1 -- 172.21.15.152:0/91044571 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47980047a0 con 0x7f47a8100e20 2024-09-11T22:54:40.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.811+0000 7f47ad2e7640 1 -- 172.21.15.152:0/91044571 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a81a50a0 con 0x7f47a8100e20 2024-09-11T22:54:40.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.813+0000 7f47a4ff9640 1 -- 172.21.15.152:0/91044571 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f47980041d0 con 0x7f47a8100e20 2024-09-11T22:54:40.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.814+0000 7f47af6c0740 1 Processor -- start 2024-09-11T22:54:40.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.814+0000 7f47af6c0740 1 -- start start 2024-09-11T22:54:40.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.814+0000 7f47af6c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff3b60 0x557ea3ff3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.814+0000 7f47af6c0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ff4480 0x557ea3ffe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.814+0000 7f47af6c0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ffed40 0x557ea4001130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.814+0000 7f47af6c0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557ea3f59280 con 0x557ea3ffed40 2024-09-11T22:54:40.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.814+0000 7f47af6c0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557ea3f00b10 con 0x557ea3ff4480 2024-09-11T22:54:40.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.814+0000 7f47af6c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557ea3f55240 con 0x557ea3ff3b60 2024-09-11T22:54:40.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.814+0000 7f47a4ff9640 1 --2- 172.21.15.152:0/91044571 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f47940a2cf0 0x7f47940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.814+0000 7f479e7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ffed40 0x557ea4001130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.815+0000 7f479e7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ffed40 0x557ea4001130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50208/0 (socket says 172.21.15.152:50208) 2024-09-11T22:54:40.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.815+0000 7f479e7fc640 1 -- 172.21.15.152:0/1704663364 learned_addr learned my addr 172.21.15.152:0/1704663364 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:40.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.815+0000 7f47a77fe640 1 --2- 172.21.15.152:0/91044571 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f47940a2cf0 0x7f47940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.815+0000 7f47a4ff9640 1 -- 172.21.15.152:0/91044571 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f47980cedb0 con 0x7f47a8100e20 2024-09-11T22:54:40.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.815+0000 7f47a6ffd640 1 --2- 172.21.15.152:0/1704663364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ff4480 0x557ea3ffe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.815+0000 7f47a77fe640 1 --2- 172.21.15.152:0/1704663364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff3b60 0x557ea3ff3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.815+0000 7f479e7fc640 1 -- 172.21.15.152:0/1704663364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff3b60 msgr2=0x557ea3ff3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:40.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.815+0000 7f479e7fc640 1 --2- 172.21.15.152:0/1704663364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff3b60 0x557ea3ff3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.815+0000 7f479e7fc640 1 -- 172.21.15.152:0/1704663364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ff4480 msgr2=0x557ea3ffe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:40.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.815+0000 7f479e7fc640 1 --2- 172.21.15.152:0/1704663364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ff4480 0x557ea3ffe800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.815+0000 7f479e7fc640 1 -- 172.21.15.152:0/1704663364 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ea3f0d1f0 con 0x557ea3ffed40 2024-09-11T22:54:40.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.815+0000 7f479e7fc640 1 --2- 172.21.15.152:0/1704663364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ffed40 0x557ea4001130 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f479000b690 tx=0x7f479000bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a8178f1ea656b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:40.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.815+0000 7f47a77fe640 1 --2- 172.21.15.152:0/91044571 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f47940a2cf0 0x7f47940a51b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f47a0005e30 tx=0x7f47a002fe00 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:40.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.815+0000 7f479d7fa640 1 -- 172.21.15.152:0/1704663364 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790003bd0 con 0x557ea3ffed40 2024-09-11T22:54:40.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.815+0000 7f479d7fa640 1 -- 172.21.15.152:0/1704663364 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4790015070 con 0x557ea3ffed40 2024-09-11T22:54:40.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.815+0000 7f479d7fa640 1 -- 172.21.15.152:0/1704663364 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790004db0 con 0x557ea3ffed40 2024-09-11T22:54:40.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.816+0000 7f47af6c0740 1 -- 172.21.15.152:0/1704663364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ffed40 msgr2=0x557ea4001130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:40.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.816+0000 7f47af6c0740 1 --2- 172.21.15.152:0/1704663364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ffed40 0x557ea4001130 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f479000b690 tx=0x7f479000bb50 comp rx=0 tx=0).stop 2024-09-11T22:54:40.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.816+0000 7f47af6c0740 1 -- 172.21.15.152:0/1704663364 shutdown_connections 2024-09-11T22:54:40.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.816+0000 7f47af6c0740 1 --2- 172.21.15.152:0/1704663364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ffed40 0x557ea4001130 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.816+0000 7f47af6c0740 1 --2- 172.21.15.152:0/1704663364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ff4480 0x557ea3ffe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.816+0000 7f47af6c0740 1 --2- 172.21.15.152:0/1704663364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff3b60 0x557ea3ff3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.816+0000 7f47af6c0740 1 -- 172.21.15.152:0/1704663364 >> 172.21.15.152:0/1704663364 conn(0x557ea3ff00f0 msgr2=0x557ea3ff04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:40.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.816+0000 7f47af6c0740 1 -- 172.21.15.152:0/1704663364 shutdown_connections 2024-09-11T22:54:40.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.816+0000 7f47af6c0740 1 -- 172.21.15.152:0/1704663364 wait complete. 2024-09-11T22:54:40.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.816+0000 7f47af6c0740 1 -- start start 2024-09-11T22:54:40.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f47af6c0740 1 Processor -- start 2024-09-11T22:54:40.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f47af6c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff3b60 0x557ea3ffbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f47af6c0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ff4480 0x557ea3ffc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f47af6c0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ffed40 0x557ea3ffc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f47af6c0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557ea4001910 con 0x557ea3ff4480 2024-09-11T22:54:40.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f47af6c0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557ea3f13f10 con 0x557ea3ffed40 2024-09-11T22:54:40.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f47af6c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557ea4001a90 con 0x557ea3ff3b60 2024-09-11T22:54:40.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f47af6c0740 10 client.0 _ll_register_callbacks cb 0x557ea3edca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:40.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f479e7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ff4480 0x557ea3ffc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f479e7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ff4480 0x557ea3ffc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50218/0 (socket says 172.21.15.152:50218) 2024-09-11T22:54:40.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f479e7fc640 1 -- 172.21.15.152:0/2410492931 learned_addr learned my addr 172.21.15.152:0/2410492931 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:40.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f47a6ffd640 1 --2- 172.21.15.152:0/2410492931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff3b60 0x557ea3ffbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f47a77fe640 1 --2- 172.21.15.152:0/2410492931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ffed40 0x557ea3ffc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f479e7fc640 1 -- 172.21.15.152:0/2410492931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff3b60 msgr2=0x557ea3ffbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:40.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f479e7fc640 1 --2- 172.21.15.152:0/2410492931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff3b60 0x557ea3ffbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f479e7fc640 1 -- 172.21.15.152:0/2410492931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ffed40 msgr2=0x557ea3ffc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:40.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f479e7fc640 1 --2- 172.21.15.152:0/2410492931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ffed40 0x557ea3ffc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f479e7fc640 1 -- 172.21.15.152:0/2410492931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ea3cca6a0 con 0x557ea3ff4480 2024-09-11T22:54:40.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f479e7fc640 1 --2- 172.21.15.152:0/2410492931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ff4480 0x557ea3ffc110 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f479000bf60 tx=0x7f47900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aab07df5a795ac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:40.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f478e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:40.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.817+0000 7f478e7fc640 1 -- 172.21.15.152:0/2410492931 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790028060 con 0x557ea3ff4480 2024-09-11T22:54:40.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.818+0000 7f478e7fc640 1 -- 172.21.15.152:0/2410492931 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4790011640 con 0x557ea3ff4480 2024-09-11T22:54:40.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.818+0000 7f47af6c0740 1 -- 172.21.15.152:0/2410492931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4790015070 con 0x557ea3ff4480 2024-09-11T22:54:40.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.818+0000 7f478e7fc640 1 -- 172.21.15.152:0/2410492931 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4790003ea0 con 0x557ea3ff4480 2024-09-11T22:54:40.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.818+0000 7f47af6c0740 10 client.16017 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:40.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.818+0000 7f47af6c0740 1 -- 172.21.15.152:0/2410492931 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ea400d4c0 con 0x557ea3ff4480 2024-09-11T22:54:40.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.818+0000 7f478e7fc640 1 -- 172.21.15.152:0/2410492931 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4790015070 con 0x557ea3ff4480 2024-09-11T22:54:40.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.818+0000 7f478e7fc640 1 -- 172.21.15.152:0/2410492931 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4790022070 con 0x557ea3ff4480 2024-09-11T22:54:40.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.818+0000 7f47af6c0740 10 client.16017 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:40.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.818+0000 7f47af6c0740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:54:40.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.818+0000 7f47af6c0740 20 client.16017 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:40.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.818+0000 7f47af6c0740 1 --2- 172.21.15.152:0/2410492931 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557ea400e8d0 0x557ea4010cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.818+0000 7f47af6c0740 4 client.16017 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:54:40.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.818+0000 7f47af6c0740 1 -- 172.21.15.152:0/2410492931 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ea3f667f0 con 0x557ea400e8d0 2024-09-11T22:54:40.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.818+0000 7f47a6ffd640 1 --2- 172.21.15.152:0/2410492931 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557ea400e8d0 0x557ea4010cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.819+0000 7f47a6ffd640 1 --2- 172.21.15.152:0/2410492931 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557ea400e8d0 0x557ea4010cc0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dce7af39e9a6c7d9 server_cookie=1db313bab8dcc52c in_seq=0 out_seq=0 2024-09-11T22:54:40.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.819+0000 7f478e7fc640 10 client.16017 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:40.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.819+0000 7f478e7fc640 1 -- 172.21.15.152:0/2410492931 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ea3f667f0 con 0x557ea400e8d0 2024-09-11T22:54:40.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.819+0000 7f478e7fc640 10 client.16017 handle_command_reply: tid=0 2024-09-11T22:54:40.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.819+0000 7f47af6c0740 1 client.16017 shutdown 2024-09-11T22:54:40.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.820+0000 7f47af6c0740 1 -- 172.21.15.152:0/2410492931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ff4480 msgr2=0x557ea3ffc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:40.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.820+0000 7f47af6c0740 1 --2- 172.21.15.152:0/2410492931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ff4480 0x557ea3ffc110 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f479000bf60 tx=0x7f47900046c0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.820+0000 7f47af6c0740 1 -- 172.21.15.152:0/2410492931 shutdown_connections 2024-09-11T22:54:40.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.820+0000 7f47af6c0740 1 --2- 172.21.15.152:0/2410492931 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557ea400e8d0 0x557ea4010cc0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.820+0000 7f47af6c0740 1 --2- 172.21.15.152:0/2410492931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ffed40 0x557ea3ffc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.820+0000 7f47af6c0740 1 --2- 172.21.15.152:0/2410492931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ff4480 0x557ea3ffc110 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.820+0000 7f478e7fc640 0 client.16017 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:40.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.820+0000 7f47af6c0740 1 --2- 172.21.15.152:0/2410492931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff3b60 0x557ea3ffbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.820+0000 7f47af6c0740 1 -- 172.21.15.152:0/2410492931 >> 172.21.15.152:0/2410492931 conn(0x557ea3ff00f0 msgr2=0x557ea3fef270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:40.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.820+0000 7f47af6c0740 1 -- 172.21.15.152:0/2410492931 shutdown_connections 2024-09-11T22:54:40.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.820+0000 7f47af6c0740 1 -- 172.21.15.152:0/2410492931 wait complete. 2024-09-11T22:54:40.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.820+0000 7f47af6c0740 20 client.16017 trim_cache size 0 max 16384 2024-09-11T22:54:40.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.837+0000 7f47af6c0740 1 Processor -- start 2024-09-11T22:54:40.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.837+0000 7f47af6c0740 1 -- start start 2024-09-11T22:54:40.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f47af6c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff3b60 0x557ea4016260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f47af6c0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ff4480 0x557ea40167a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f47af6c0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ffed40 0x557ea3ff8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f47af6c0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81ad800 con 0x557ea3ffed40 2024-09-11T22:54:40.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f47af6c0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81af340 con 0x557ea3ff4480 2024-09-11T22:54:40.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f47af6c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81aded0 con 0x557ea3ff3b60 2024-09-11T22:54:40.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f47a77fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ffed40 0x557ea3ff8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f47a77fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ffed40 0x557ea3ff8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50222/0 (socket says 172.21.15.152:50222) 2024-09-11T22:54:40.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f47a77fe640 1 -- 172.21.15.152:0/3195171661 learned_addr learned my addr 172.21.15.152:0/3195171661 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:40.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f479e7fc640 1 --2- 172.21.15.152:0/3195171661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ff4480 0x557ea40167a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f47a6ffd640 1 --2- 172.21.15.152:0/3195171661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff3b60 0x557ea4016260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f47a77fe640 1 -- 172.21.15.152:0/3195171661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff3b60 msgr2=0x557ea4016260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:40.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f47a77fe640 1 --2- 172.21.15.152:0/3195171661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff3b60 0x557ea4016260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f47a77fe640 1 -- 172.21.15.152:0/3195171661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ff4480 msgr2=0x557ea40167a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:40.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f47a77fe640 1 --2- 172.21.15.152:0/3195171661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ff4480 0x557ea40167a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f47a77fe640 1 -- 172.21.15.152:0/3195171661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ea3ffa120 con 0x557ea3ffed40 2024-09-11T22:54:40.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f47a77fe640 1 --2- 172.21.15.152:0/3195171661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ffed40 0x557ea3ff8e30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f47a0035000 tx=0x7f47a0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5543335c0b96c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f479cff9640 1 -- 172.21.15.152:0/3195171661 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a0034040 con 0x557ea3ffed40 2024-09-11T22:54:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f479cff9640 1 -- 172.21.15.152:0/3195171661 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47a0004040 con 0x557ea3ffed40 2024-09-11T22:54:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.838+0000 7f479cff9640 1 -- 172.21.15.152:0/3195171661 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a003b6a0 con 0x557ea3ffed40 2024-09-11T22:54:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.839+0000 7f47af6c0740 1 -- 172.21.15.152:0/3195171661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ffed40 msgr2=0x557ea3ff8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.839+0000 7f47af6c0740 1 --2- 172.21.15.152:0/3195171661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ffed40 0x557ea3ff8e30 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f47a0035000 tx=0x7f47a0005c50 comp rx=0 tx=0).stop 2024-09-11T22:54:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.839+0000 7f47af6c0740 1 -- 172.21.15.152:0/3195171661 shutdown_connections 2024-09-11T22:54:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.839+0000 7f47af6c0740 1 --2- 172.21.15.152:0/3195171661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ffed40 0x557ea3ff8e30 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.839+0000 7f47af6c0740 1 --2- 172.21.15.152:0/3195171661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ff4480 0x557ea40167a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.839+0000 7f47af6c0740 1 --2- 172.21.15.152:0/3195171661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff3b60 0x557ea4016260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.839+0000 7f47af6c0740 1 -- 172.21.15.152:0/3195171661 >> 172.21.15.152:0/3195171661 conn(0x557ea3ff00f0 msgr2=0x557ea3ffdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.839+0000 7f47af6c0740 1 -- 172.21.15.152:0/3195171661 shutdown_connections 2024-09-11T22:54:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.839+0000 7f47af6c0740 1 -- 172.21.15.152:0/3195171661 wait complete. 2024-09-11T22:54:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47af6c0740 1 -- start start 2024-09-11T22:54:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47af6c0740 1 Processor -- start 2024-09-11T22:54:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47af6c0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ff3b60 0x557ea40032f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47af6c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff4480 0x557ea4003830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47af6c0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ffed40 0x557ea4078230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47af6c0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81aec70 con 0x557ea3ff3b60 2024-09-11T22:54:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47af6c0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81ae5a0 con 0x557ea3ffed40 2024-09-11T22:54:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47af6c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81ab7c0 con 0x557ea3ff4480 2024-09-11T22:54:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47af6c0740 10 client.0 _ll_register_callbacks cb 0x7f4798096f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:40.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47a6ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ff3b60 0x557ea40032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47a6ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ff3b60 0x557ea40032f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50228/0 (socket says 172.21.15.152:50228) 2024-09-11T22:54:40.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47a6ffd640 1 -- 172.21.15.152:0/619891188 learned_addr learned my addr 172.21.15.152:0/619891188 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:40.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f479e7fc640 1 --2- 172.21.15.152:0/619891188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff4480 0x557ea4003830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.844 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:40 smithi152 ceph-mon[31181]: pgmap v56: 97 pgs: 97 active+clean; 1.0 GiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 144 MiB/s wr, 77 op/s 2024-09-11T22:54:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47a77fe640 1 --2- 172.21.15.152:0/619891188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ffed40 0x557ea4078230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47a6ffd640 1 -- 172.21.15.152:0/619891188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff4480 msgr2=0x557ea4003830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47a6ffd640 1 --2- 172.21.15.152:0/619891188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff4480 0x557ea4003830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47a6ffd640 1 -- 172.21.15.152:0/619891188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ffed40 msgr2=0x557ea4078230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47a6ffd640 1 --2- 172.21.15.152:0/619891188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ffed40 0x557ea4078230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.840+0000 7f47a6ffd640 1 -- 172.21.15.152:0/619891188 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ea4078870 con 0x557ea3ff3b60 2024-09-11T22:54:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.841+0000 7f47a6ffd640 1 --2- 172.21.15.152:0/619891188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ff3b60 0x557ea40032f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4798097cb0 tx=0x7f47980bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbf25a33f870f18 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:40.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.841+0000 7f478e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.841+0000 7f478e7fc640 1 -- 172.21.15.152:0/619891188 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479809b2e0 con 0x557ea3ff3b60 2024-09-11T22:54:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.841+0000 7f478e7fc640 1 -- 172.21.15.152:0/619891188 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f47980cc790 con 0x557ea3ff3b60 2024-09-11T22:54:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.841+0000 7f47af6c0740 1 -- 172.21.15.152:0/619891188 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47a0004040 con 0x557ea3ff3b60 2024-09-11T22:54:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.841+0000 7f478e7fc640 1 -- 172.21.15.152:0/619891188 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f47980cc990 con 0x557ea3ff3b60 2024-09-11T22:54:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.841+0000 7f47af6c0740 10 client.16023 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.841+0000 7f47af6c0740 1 -- 172.21.15.152:0/619891188 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557ea4079bf0 con 0x557ea3ff3b60 2024-09-11T22:54:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.841+0000 7f478e7fc640 1 -- 172.21.15.152:0/619891188 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47a0004040 con 0x557ea3ff3b60 2024-09-11T22:54:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.841+0000 7f478e7fc640 1 -- 172.21.15.152:0/619891188 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f47980a8050 con 0x557ea3ff3b60 2024-09-11T22:54:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.841+0000 7f47af6c0740 10 client.16023 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.841+0000 7f47af6c0740 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:54:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.841+0000 7f47af6c0740 20 client.16023 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.841+0000 7f47af6c0740 1 --2- 172.21.15.152:0/619891188 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557ea407a2c0 0x557ea407c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:40.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.841+0000 7f47af6c0740 4 client.16023 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:54:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.842+0000 7f47af6c0740 1 -- 172.21.15.152:0/619891188 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557ea3f60d00 con 0x557ea407a2c0 2024-09-11T22:54:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.842+0000 7f479e7fc640 1 --2- 172.21.15.152:0/619891188 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557ea407a2c0 0x557ea407c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.842+0000 7f478e7fc640 10 client.16023 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.842+0000 7f479e7fc640 1 --2- 172.21.15.152:0/619891188 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557ea407a2c0 0x557ea407c6e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81311f817b54afff server_cookie=3f8720db82e0d80f in_seq=0 out_seq=0 2024-09-11T22:54:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.843+0000 7f478e7fc640 1 -- 172.21.15.152:0/619891188 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557ea3f60d00 con 0x557ea407a2c0 2024-09-11T22:54:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.843+0000 7f478e7fc640 10 client.16023 handle_command_reply: tid=0 2024-09-11T22:54:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.843+0000 7f47af6c0740 1 client.16023 shutdown 2024-09-11T22:54:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.843+0000 7f47af6c0740 1 -- 172.21.15.152:0/619891188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ff3b60 msgr2=0x557ea40032f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.843+0000 7f47af6c0740 1 --2- 172.21.15.152:0/619891188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ff3b60 0x557ea40032f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4798097cb0 tx=0x7f47980bb040 comp rx=0 tx=0).stop 2024-09-11T22:54:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.843+0000 7f47af6c0740 1 -- 172.21.15.152:0/619891188 shutdown_connections 2024-09-11T22:54:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.843+0000 7f47af6c0740 1 --2- 172.21.15.152:0/619891188 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557ea407a2c0 0x557ea407c6e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.843+0000 7f47af6c0740 1 --2- 172.21.15.152:0/619891188 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557ea3ffed40 0x557ea4078230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.843+0000 7f47af6c0740 1 --2- 172.21.15.152:0/619891188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557ea3ff4480 0x557ea4003830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.843+0000 7f47af6c0740 1 --2- 172.21.15.152:0/619891188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ea3ff3b60 0x557ea40032f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.843+0000 7f47af6c0740 1 -- 172.21.15.152:0/619891188 >> 172.21.15.152:0/619891188 conn(0x557ea3ff00f0 msgr2=0x557ea3feea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:40.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.843+0000 7f478e7fc640 0 client.16023 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:40.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.843+0000 7f47af6c0740 1 -- 172.21.15.152:0/619891188 shutdown_connections 2024-09-11T22:54:40.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.843+0000 7f47af6c0740 1 -- 172.21.15.152:0/619891188 wait complete. 2024-09-11T22:54:40.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.844+0000 7f47af6c0740 20 client.16023 trim_cache size 0 max 16384 2024-09-11T22:54:40.850 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:54:40.850 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T22:54:40.850 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "3bd2da69-9286-4eb5-a206-d65bad41197c", 2024-09-11T22:54:40.850 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T22:54:40.851 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:54:40.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.844+0000 7f47ad2e7640 1 -- 172.21.15.152:0/91044571 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f47940a2cf0 msgr2=0x7f47940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:40.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.844+0000 7f47ad2e7640 1 --2- 172.21.15.152:0/91044571 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f47940a2cf0 0x7f47940a51b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f47a0005e30 tx=0x7f47a002fe00 comp rx=0 tx=0).stop 2024-09-11T22:54:40.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.844+0000 7f47ad2e7640 1 -- 172.21.15.152:0/91044571 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a8100e20 msgr2=0x7f47a8198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:40.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.844+0000 7f47ad2e7640 1 --2- 172.21.15.152:0/91044571 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a8100e20 0x7f47a8198520 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f479800d950 tx=0x7f479800de10 comp rx=0 tx=0).stop 2024-09-11T22:54:40.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.844+0000 7f47ad2e7640 1 -- 172.21.15.152:0/91044571 shutdown_connections 2024-09-11T22:54:40.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.844+0000 7f47ad2e7640 1 --2- 172.21.15.152:0/91044571 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f47940a2cf0 0x7f47940a51b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.844+0000 7f47ad2e7640 1 --2- 172.21.15.152:0/91044571 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47a8102ef0 0x7f47a819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.844+0000 7f47ad2e7640 1 --2- 172.21.15.152:0/91044571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47a8102570 0x7f47a8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.844+0000 7f47ad2e7640 1 --2- 172.21.15.152:0/91044571 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f47a8100e20 0x7f47a8198520 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:40.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.844+0000 7f47ad2e7640 1 -- 172.21.15.152:0/91044571 >> 172.21.15.152:0/91044571 conn(0x7f47a80fc7d0 msgr2=0x7f47a806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:40.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.845+0000 7f47ad2e7640 1 -- 172.21.15.152:0/91044571 shutdown_connections 2024-09-11T22:54:40.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:40.845+0000 7f47ad2e7640 1 -- 172.21.15.152:0/91044571 wait complete. 2024-09-11T22:54:41.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:40 smithi191 ceph-mon[35211]: pgmap v56: 97 pgs: 97 active+clean; 1.0 GiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 144 MiB/s wr, 77 op/s 2024-09-11T22:54:41.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:40 smithi204 ceph-mon[35454]: pgmap v56: 97 pgs: 97 active+clean; 1.0 GiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 144 MiB/s wr, 77 op/s 2024-09-11T22:54:41.551 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:54:41.800 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:54:41.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:41 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T22:54:41.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:41 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:54:41.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:41 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T22:54:41.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:41 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:54:42.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:41 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T22:54:42.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:41 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:54:42.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:41 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T22:54:42.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:41 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:54:42.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:41 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T22:54:42.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:41 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:54:42.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:41 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T22:54:42.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:41 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:54:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.319+0000 7f02342db640 1 -- 172.21.15.152:0/1447074764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f022c102ef0 msgr2=0x7f022c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.319+0000 7f02342db640 1 --2- 172.21.15.152:0/1447074764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f022c102ef0 0x7f022c10da40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0224009ca0 tx=0x7f022402a950 comp rx=0 tx=0).stop 2024-09-11T22:54:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.319+0000 7f02342db640 1 -- 172.21.15.152:0/1447074764 shutdown_connections 2024-09-11T22:54:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.319+0000 7f02342db640 1 --2- 172.21.15.152:0/1447074764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f022c102ef0 0x7f022c10da40 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.319+0000 7f02342db640 1 --2- 172.21.15.152:0/1447074764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f022c102570 0x7f022c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.319+0000 7f02342db640 1 --2- 172.21.15.152:0/1447074764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f022c100e20 0x7f022c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.319+0000 7f02342db640 1 -- 172.21.15.152:0/1447074764 >> 172.21.15.152:0/1447074764 conn(0x7f022c0fc7d0 msgr2=0x7f022c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.319+0000 7f02342db640 1 -- 172.21.15.152:0/1447074764 shutdown_connections 2024-09-11T22:54:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.319+0000 7f02342db640 1 -- 172.21.15.152:0/1447074764 wait complete. 2024-09-11T22:54:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.320+0000 7f02342db640 1 Processor -- start 2024-09-11T22:54:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.320+0000 7f02342db640 1 -- start start 2024-09-11T22:54:42.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.320+0000 7f02342db640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f022c100e20 0x7f022c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.320+0000 7f02342db640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f022c102570 0x7f022c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.320+0000 7f02342db640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f022c102ef0 0x7f022c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.320+0000 7f02342db640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f022c1120a0 con 0x7f022c100e20 2024-09-11T22:54:42.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.320+0000 7f02342db640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f022c111f20 con 0x7f022c102570 2024-09-11T22:54:42.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.320+0000 7f02342db640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f022c112220 con 0x7f022c102ef0 2024-09-11T22:54:42.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.320+0000 7f0232077640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f022c100e20 0x7f022c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.320+0000 7f0232077640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f022c100e20 0x7f022c198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50254/0 (socket says 172.21.15.152:50254) 2024-09-11T22:54:42.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.320+0000 7f0232077640 1 -- 172.21.15.152:0/3204204026 learned_addr learned my addr 172.21.15.152:0/3204204026 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:42.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.320+0000 7f0232878640 1 --2- 172.21.15.152:0/3204204026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f022c102ef0 0x7f022c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.320+0000 7f0231876640 1 --2- 172.21.15.152:0/3204204026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f022c102570 0x7f022c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.321+0000 7f0232077640 1 -- 172.21.15.152:0/3204204026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f022c102ef0 msgr2=0x7f022c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:42.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.321+0000 7f0232077640 1 --2- 172.21.15.152:0/3204204026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f022c102ef0 0x7f022c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.321+0000 7f0232077640 1 -- 172.21.15.152:0/3204204026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f022c102570 msgr2=0x7f022c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:42.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.321+0000 7f0232077640 1 --2- 172.21.15.152:0/3204204026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f022c102570 0x7f022c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.321+0000 7f0232077640 1 -- 172.21.15.152:0/3204204026 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022c19d410 con 0x7f022c100e20 2024-09-11T22:54:42.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.321+0000 7f0232878640 1 --2- 172.21.15.152:0/3204204026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f022c102ef0 0x7f022c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:54:42.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.321+0000 7f0232077640 1 --2- 172.21.15.152:0/3204204026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f022c100e20 0x7f022c198460 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f022800d950 tx=0x7f022800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:42.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.321+0000 7f021f7fe640 1 -- 172.21.15.152:0/3204204026 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228017030 con 0x7f022c100e20 2024-09-11T22:54:42.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.321+0000 7f021f7fe640 1 -- 172.21.15.152:0/3204204026 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0228004020 con 0x7f022c100e20 2024-09-11T22:54:42.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.321+0000 7f021f7fe640 1 -- 172.21.15.152:0/3204204026 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280047a0 con 0x7f022c100e20 2024-09-11T22:54:42.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.321+0000 7f02342db640 1 -- 172.21.15.152:0/3204204026 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f022c19d700 con 0x7f022c100e20 2024-09-11T22:54:42.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.321+0000 7f02342db640 1 -- 172.21.15.152:0/3204204026 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f022c1a4fe0 con 0x7f022c100e20 2024-09-11T22:54:42.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.324+0000 7f02366b4740 1 Processor -- start 2024-09-11T22:54:42.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.324+0000 7f021f7fe640 1 -- 172.21.15.152:0/3204204026 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0228004940 con 0x7f022c100e20 2024-09-11T22:54:42.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.324+0000 7f02366b4740 1 -- start start 2024-09-11T22:54:42.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.324+0000 7f02366b4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 0x5646a0c55880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.324+0000 7f02366b4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 0x5646a0c60140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.324+0000 7f02366b4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c60680 0x5646a0c62a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.324+0000 7f02366b4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5646a0bbb2e0 con 0x5646a0c554a0 2024-09-11T22:54:42.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.325+0000 7f02366b4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5646a0b62ce0 con 0x5646a0c60680 2024-09-11T22:54:42.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.325+0000 7f02366b4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5646a0bb72a0 con 0x5646a0c55dc0 2024-09-11T22:54:42.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.325+0000 7f0232878640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 0x5646a0c55880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.325+0000 7f0232878640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 0x5646a0c55880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50258/0 (socket says 172.21.15.152:50258) 2024-09-11T22:54:42.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.325+0000 7f0232878640 1 -- 172.21.15.152:0/1540189834 learned_addr learned my addr 172.21.15.152:0/1540189834 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:42.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.325+0000 7f0232077640 1 --2- 172.21.15.152:0/1540189834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 0x5646a0c60140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.325+0000 7f0231876640 1 --2- 172.21.15.152:0/1540189834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c60680 0x5646a0c62a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.325+0000 7f021f7fe640 1 --2- 172.21.15.152:0/3204204026 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f02180a2cf0 0x7f02180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.325+0000 7f0231876640 1 -- 172.21.15.152:0/1540189834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 msgr2=0x5646a0c60140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:42.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.325+0000 7f0231876640 1 --2- 172.21.15.152:0/1540189834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 0x5646a0c60140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.325+0000 7f0231876640 1 -- 172.21.15.152:0/1540189834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 msgr2=0x5646a0c55880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:42.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.325+0000 7f0231876640 1 --2- 172.21.15.152:0/1540189834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 0x5646a0c55880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.325+0000 7f0231876640 1 -- 172.21.15.152:0/1540189834 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a0b47440 con 0x5646a0c60680 2024-09-11T22:54:42.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.325+0000 7f0232077640 1 --2- 172.21.15.152:0/1540189834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 0x5646a0c60140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:54:42.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.325+0000 7f021f7fe640 1 -- 172.21.15.152:0/3204204026 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f02280cee20 con 0x7f022c100e20 2024-09-11T22:54:42.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.325+0000 7f0232878640 1 --2- 172.21.15.152:0/3204204026 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f02180a2cf0 0x7f02180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.326+0000 7f0231876640 1 --2- 172.21.15.152:0/1540189834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c60680 0x5646a0c62a70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f022000bb40 tx=0x7f02200103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc065db65bceec6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:42.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.326+0000 7f021cdea640 1 -- 172.21.15.152:0/1540189834 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220004030 con 0x5646a0c60680 2024-09-11T22:54:42.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.326+0000 7f021cdea640 1 -- 172.21.15.152:0/1540189834 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02200041d0 con 0x5646a0c60680 2024-09-11T22:54:42.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.326+0000 7f021cdea640 1 -- 172.21.15.152:0/1540189834 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220011680 con 0x5646a0c60680 2024-09-11T22:54:42.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.327+0000 7f02366b4740 1 -- 172.21.15.152:0/1540189834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c60680 msgr2=0x5646a0c62a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:42.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.327+0000 7f02366b4740 1 --2- 172.21.15.152:0/1540189834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c60680 0x5646a0c62a70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f022000bb40 tx=0x7f02200103d0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.327+0000 7f0232878640 1 --2- 172.21.15.152:0/3204204026 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f02180a2cf0 0x7f02180a51b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f022402fd80 tx=0x7f022402a950 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:42.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.327+0000 7f02366b4740 1 -- 172.21.15.152:0/1540189834 shutdown_connections 2024-09-11T22:54:42.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.327+0000 7f02366b4740 1 --2- 172.21.15.152:0/1540189834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c60680 0x5646a0c62a70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.327+0000 7f02366b4740 1 --2- 172.21.15.152:0/1540189834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 0x5646a0c60140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.327+0000 7f02366b4740 1 --2- 172.21.15.152:0/1540189834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 0x5646a0c55880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.327+0000 7f02366b4740 1 -- 172.21.15.152:0/1540189834 >> 172.21.15.152:0/1540189834 conn(0x5646a0c51a30 msgr2=0x5646a0c51e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:42.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.327+0000 7f02366b4740 1 -- 172.21.15.152:0/1540189834 shutdown_connections 2024-09-11T22:54:42.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.327+0000 7f02366b4740 1 -- 172.21.15.152:0/1540189834 wait complete. 2024-09-11T22:54:42.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.327+0000 7f02366b4740 1 -- start start 2024-09-11T22:54:42.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.327+0000 7f02366b4740 1 Processor -- start 2024-09-11T22:54:42.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f02366b4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 0x5646a0c60000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f02366b4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 0x5646a0c5d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f02366b4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c60680 0x5646a0c5d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f02366b4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5646a0c63250 con 0x5646a0c554a0 2024-09-11T22:54:42.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f02366b4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5646a0b76050 con 0x5646a0c60680 2024-09-11T22:54:42.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f02366b4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5646a0c633d0 con 0x5646a0c55dc0 2024-09-11T22:54:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f02366b4740 10 client.0 _ll_register_callbacks cb 0x5646a0bb6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f0232077640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 0x5646a0c60000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f0232077640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 0x5646a0c60000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50270/0 (socket says 172.21.15.152:50270) 2024-09-11T22:54:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f0232077640 1 -- 172.21.15.152:0/916986073 learned_addr learned my addr 172.21.15.152:0/916986073 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f0232878640 1 --2- 172.21.15.152:0/916986073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c60680 0x5646a0c5d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f0231876640 1 --2- 172.21.15.152:0/916986073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 0x5646a0c5d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f0232077640 1 -- 172.21.15.152:0/916986073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 msgr2=0x5646a0c5d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f0232077640 1 --2- 172.21.15.152:0/916986073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 0x5646a0c5d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f0232077640 1 -- 172.21.15.152:0/916986073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c60680 msgr2=0x5646a0c5d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f0232077640 1 --2- 172.21.15.152:0/916986073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c60680 0x5646a0c5d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.328+0000 7f0232077640 1 -- 172.21.15.152:0/916986073 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a092c3c0 con 0x5646a0c554a0 2024-09-11T22:54:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.329+0000 7f0232077640 1 --2- 172.21.15.152:0/916986073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 0x5646a0c60000 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f022809b320 tx=0x7f022809b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2be6efd1ddb5743 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.329+0000 7f0215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.329+0000 7f0215ffb640 1 -- 172.21.15.152:0/916986073 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022809b390 con 0x5646a0c554a0 2024-09-11T22:54:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.329+0000 7f0215ffb640 1 -- 172.21.15.152:0/916986073 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f02280cc840 con 0x5646a0c554a0 2024-09-11T22:54:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.329+0000 7f02366b4740 1 -- 172.21.15.152:0/916986073 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02200041d0 con 0x5646a0c554a0 2024-09-11T22:54:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.329+0000 7f0215ffb640 1 -- 172.21.15.152:0/916986073 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f02280cca40 con 0x5646a0c554a0 2024-09-11T22:54:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.329+0000 7f02366b4740 10 client.16035 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.329+0000 7f02366b4740 1 -- 172.21.15.152:0/916986073 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5646a0c6ec40 con 0x5646a0c554a0 2024-09-11T22:54:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.329+0000 7f0215ffb640 1 -- 172.21.15.152:0/916986073 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02200041d0 con 0x5646a0c554a0 2024-09-11T22:54:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.329+0000 7f0215ffb640 1 -- 172.21.15.152:0/916986073 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f02280b41e0 con 0x5646a0c554a0 2024-09-11T22:54:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.329+0000 7f02366b4740 10 client.16035 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.329+0000 7f02366b4740 10 client.16035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:54:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.329+0000 7f02366b4740 20 client.16035 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.329+0000 7f02366b4740 1 --2- 172.21.15.152:0/916986073 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5646a0c70050 0x5646a0c72440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.329+0000 7f02366b4740 4 client.16035 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:54:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.329+0000 7f02366b4740 1 -- 172.21.15.152:0/916986073 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646a0bc8850 con 0x5646a0c70050 2024-09-11T22:54:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.330+0000 7f0231876640 1 --2- 172.21.15.152:0/916986073 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5646a0c70050 0x5646a0c72440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.330+0000 7f0231876640 1 --2- 172.21.15.152:0/916986073 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5646a0c70050 0x5646a0c72440 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b4130d4dd1357e server_cookie=fb4221767841239e in_seq=0 out_seq=0 2024-09-11T22:54:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.330+0000 7f0215ffb640 10 client.16035 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.331+0000 7f0215ffb640 1 -- 172.21.15.152:0/916986073 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646a0bc8850 con 0x5646a0c70050 2024-09-11T22:54:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.331+0000 7f0215ffb640 10 client.16035 handle_command_reply: tid=0 2024-09-11T22:54:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.331+0000 7f02366b4740 1 client.16035 shutdown 2024-09-11T22:54:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.331+0000 7f02366b4740 1 -- 172.21.15.152:0/916986073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 msgr2=0x5646a0c60000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.331+0000 7f02366b4740 1 --2- 172.21.15.152:0/916986073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 0x5646a0c60000 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f022809b320 tx=0x7f022809b1e0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.331+0000 7f02366b4740 1 -- 172.21.15.152:0/916986073 shutdown_connections 2024-09-11T22:54:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.331+0000 7f02366b4740 1 --2- 172.21.15.152:0/916986073 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5646a0c70050 0x5646a0c72440 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.331+0000 7f02366b4740 1 --2- 172.21.15.152:0/916986073 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c60680 0x5646a0c5d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.331+0000 7f02366b4740 1 --2- 172.21.15.152:0/916986073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 0x5646a0c5d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.331+0000 7f02366b4740 1 --2- 172.21.15.152:0/916986073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 0x5646a0c60000 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.331+0000 7f02366b4740 1 -- 172.21.15.152:0/916986073 >> 172.21.15.152:0/916986073 conn(0x5646a0c51a30 msgr2=0x5646a0c57760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.331+0000 7f0215ffb640 0 client.16035 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.331+0000 7f02366b4740 1 -- 172.21.15.152:0/916986073 shutdown_connections 2024-09-11T22:54:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.331+0000 7f02366b4740 1 -- 172.21.15.152:0/916986073 wait complete. 2024-09-11T22:54:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.331+0000 7f02366b4740 20 client.16035 trim_cache size 0 max 16384 2024-09-11T22:54:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f02366b4740 1 Processor -- start 2024-09-11T22:54:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f02366b4740 1 -- start start 2024-09-11T22:54:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f02366b4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 0x5646a0c6c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f02366b4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 0x5646a0c5bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f02366b4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c60680 0x5646a0c587f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f02366b4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f022c1ab560 con 0x5646a0c554a0 2024-09-11T22:54:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f02366b4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f022c1ad0a0 con 0x5646a0c60680 2024-09-11T22:54:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f02366b4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f022c1abc30 con 0x5646a0c55dc0 2024-09-11T22:54:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f0232077640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 0x5646a0c6c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f0231876640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 0x5646a0c5bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f0231876640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 0x5646a0c5bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:56424/0 (socket says 172.21.15.152:56424) 2024-09-11T22:54:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f0232077640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 0x5646a0c6c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50284/0 (socket says 172.21.15.152:50284) 2024-09-11T22:54:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f0231876640 1 -- 172.21.15.152:0/3348031702 learned_addr learned my addr 172.21.15.152:0/3348031702 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f0232878640 1 --2- 172.21.15.152:0/3348031702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c60680 0x5646a0c587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f0231876640 1 -- 172.21.15.152:0/3348031702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c60680 msgr2=0x5646a0c587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f0231876640 1 --2- 172.21.15.152:0/3348031702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c60680 0x5646a0c587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f0231876640 1 -- 172.21.15.152:0/3348031702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 msgr2=0x5646a0c6c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f0231876640 1 --2- 172.21.15.152:0/3348031702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 0x5646a0c6c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.349+0000 7f0231876640 1 -- 172.21.15.152:0/3348031702 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a0c5c260 con 0x5646a0c55dc0 2024-09-11T22:54:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.350+0000 7f0231876640 1 --2- 172.21.15.152:0/3348031702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 0x5646a0c5bb80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f022000bb40 tx=0x7f02200103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0837843cc8f5880 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:42.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.350+0000 7f0217fff640 1 -- 172.21.15.152:0/3348031702 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02200050b0 con 0x5646a0c55dc0 2024-09-11T22:54:42.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.350+0000 7f0217fff640 1 -- 172.21.15.152:0/3348031702 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02200035f0 con 0x5646a0c55dc0 2024-09-11T22:54:42.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.350+0000 7f0217fff640 1 -- 172.21.15.152:0/3348031702 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022002d610 con 0x5646a0c55dc0 2024-09-11T22:54:42.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.351+0000 7f02366b4740 1 -- 172.21.15.152:0/3348031702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 msgr2=0x5646a0c5bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:42.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.351+0000 7f02366b4740 1 --2- 172.21.15.152:0/3348031702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 0x5646a0c5bb80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f022000bb40 tx=0x7f02200103d0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.351+0000 7f02366b4740 1 -- 172.21.15.152:0/3348031702 shutdown_connections 2024-09-11T22:54:42.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.351+0000 7f02366b4740 1 --2- 172.21.15.152:0/3348031702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c60680 0x5646a0c587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.351+0000 7f02366b4740 1 --2- 172.21.15.152:0/3348031702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c55dc0 0x5646a0c5bb80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.351+0000 7f02366b4740 1 --2- 172.21.15.152:0/3348031702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c554a0 0x5646a0c6c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.351+0000 7f02366b4740 1 -- 172.21.15.152:0/3348031702 >> 172.21.15.152:0/3348031702 conn(0x5646a0c51a30 msgr2=0x5646a0c580e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:42.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.351+0000 7f02366b4740 1 -- 172.21.15.152:0/3348031702 shutdown_connections 2024-09-11T22:54:42.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.351+0000 7f02366b4740 1 -- 172.21.15.152:0/3348031702 wait complete. 2024-09-11T22:54:42.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.351+0000 7f02366b4740 1 -- start start 2024-09-11T22:54:42.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.351+0000 7f02366b4740 1 Processor -- start 2024-09-11T22:54:42.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f02366b4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c554a0 0x5646a0cd2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f02366b4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c55dc0 0x5646a0cd3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f02366b4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c60680 0x5646a0c63be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f02366b4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f022c1ac9d0 con 0x5646a0c60680 2024-09-11T22:54:42.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f02366b4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f022c1ac300 con 0x5646a0c55dc0 2024-09-11T22:54:42.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f02366b4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f022c1a9520 con 0x5646a0c554a0 2024-09-11T22:54:42.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f02366b4740 10 client.0 _ll_register_callbacks cb 0x7f0228005130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:42.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f0232878640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c60680 0x5646a0c63be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f0232077640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c554a0 0x5646a0cd2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f0232077640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c554a0 0x5646a0cd2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:56426/0 (socket says 172.21.15.152:56426) 2024-09-11T22:54:42.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f0232878640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c60680 0x5646a0c63be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50288/0 (socket says 172.21.15.152:50288) 2024-09-11T22:54:42.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f0232077640 1 -- 172.21.15.152:0/2905206614 learned_addr learned my addr 172.21.15.152:0/2905206614 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:42.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f0231876640 1 --2- 172.21.15.152:0/2905206614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c55dc0 0x5646a0cd3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f0232077640 1 -- 172.21.15.152:0/2905206614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c55dc0 msgr2=0x5646a0cd3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:42.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f0232077640 1 --2- 172.21.15.152:0/2905206614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c55dc0 0x5646a0cd3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f0232077640 1 -- 172.21.15.152:0/2905206614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c60680 msgr2=0x5646a0c63be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:42.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f0232077640 1 --2- 172.21.15.152:0/2905206614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c60680 0x5646a0c63be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f0232077640 1 -- 172.21.15.152:0/2905206614 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a0c642d0 con 0x5646a0c554a0 2024-09-11T22:54:42.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f0232077640 1 --2- 172.21.15.152:0/2905206614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c554a0 0x5646a0cd2ee0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f022809b550 tx=0x7f02280cca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a9197d20ec8b262 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:42.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.352+0000 7f0215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-11T22:54:42.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.353+0000 7f0215ffb640 1 -- 172.21.15.152:0/2905206614 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228004f60 con 0x5646a0c554a0 2024-09-11T22:54:42.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.353+0000 7f02366b4740 1 -- 172.21.15.152:0/2905206614 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02200035f0 con 0x5646a0c554a0 2024-09-11T22:54:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.353+0000 7f0215ffb640 1 -- 172.21.15.152:0/2905206614 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f0228004cd0 con 0x5646a0c554a0 2024-09-11T22:54:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.353+0000 7f0215ffb640 1 -- 172.21.15.152:0/2905206614 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f02280b4360 con 0x5646a0c554a0 2024-09-11T22:54:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.353+0000 7f02366b4740 10 client.25423 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.353+0000 7f02366b4740 1 -- 172.21.15.152:0/2905206614 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5646a0cda750 con 0x5646a0c554a0 2024-09-11T22:54:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.353+0000 7f0215ffb640 1 -- 172.21.15.152:0/2905206614 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02200035f0 con 0x5646a0c554a0 2024-09-11T22:54:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.353+0000 7f0215ffb640 1 -- 172.21.15.152:0/2905206614 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f02280d1070 con 0x5646a0c554a0 2024-09-11T22:54:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.353+0000 7f02366b4740 10 client.25423 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.353+0000 7f02366b4740 10 client.25423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:54:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.353+0000 7f02366b4740 20 client.25423 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:42.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.353+0000 7f02366b4740 1 --2- 172.21.15.152:0/2905206614 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5646a0cdaec0 0x5646a0cdd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:42.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.353+0000 7f02366b4740 4 client.25423 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:54:42.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.353+0000 7f02366b4740 1 -- 172.21.15.152:0/2905206614 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5646a0bc2d60 con 0x5646a0cdaec0 2024-09-11T22:54:42.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.354+0000 7f0231876640 1 --2- 172.21.15.152:0/2905206614 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5646a0cdaec0 0x5646a0cdd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:42.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.354+0000 7f0231876640 1 --2- 172.21.15.152:0/2905206614 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5646a0cdaec0 0x5646a0cdd2b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29f462de2f3cdc81 server_cookie=970bd70c3879f598 in_seq=0 out_seq=0 2024-09-11T22:54:42.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.354+0000 7f0215ffb640 10 client.25423 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:42.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.354+0000 7f0215ffb640 1 -- 172.21.15.152:0/2905206614 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5646a0bc2d60 con 0x5646a0cdaec0 2024-09-11T22:54:42.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.354+0000 7f0215ffb640 10 client.25423 handle_command_reply: tid=0 2024-09-11T22:54:42.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.354+0000 7f02366b4740 1 client.25423 shutdown 2024-09-11T22:54:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.354+0000 7f02366b4740 1 -- 172.21.15.152:0/2905206614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c554a0 msgr2=0x5646a0cd2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.354+0000 7f02366b4740 1 --2- 172.21.15.152:0/2905206614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c554a0 0x5646a0cd2ee0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f022809b550 tx=0x7f02280cca70 comp rx=0 tx=0).stop 2024-09-11T22:54:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.355+0000 7f02366b4740 1 -- 172.21.15.152:0/2905206614 shutdown_connections 2024-09-11T22:54:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.355+0000 7f02366b4740 1 --2- 172.21.15.152:0/2905206614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5646a0c60680 0x5646a0c63be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.355+0000 7f02366b4740 1 --2- 172.21.15.152:0/2905206614 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5646a0cdaec0 0x5646a0cdd2b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.355+0000 7f02366b4740 1 --2- 172.21.15.152:0/2905206614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5646a0c55dc0 0x5646a0cd3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.355+0000 7f02366b4740 1 --2- 172.21.15.152:0/2905206614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5646a0c554a0 0x5646a0cd2ee0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.355+0000 7f02366b4740 1 -- 172.21.15.152:0/2905206614 >> 172.21.15.152:0/2905206614 conn(0x5646a0c51a30 msgr2=0x5646a0c50b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.355+0000 7f0215ffb640 0 client.25423 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.355+0000 7f02366b4740 1 -- 172.21.15.152:0/2905206614 shutdown_connections 2024-09-11T22:54:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.355+0000 7f02366b4740 1 -- 172.21.15.152:0/2905206614 wait complete. 2024-09-11T22:54:42.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.355+0000 7f02366b4740 20 client.25423 trim_cache size 0 max 16384 2024-09-11T22:54:42.366 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:54:42.366 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:54:42.366 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": { 2024-09-11T22:54:42.366 INFO:teuthology.orchestra.run.smithi152.stdout: "3bd2da69-9286-4eb5-a206-d65bad41197c": { 2024-09-11T22:54:42.366 INFO:teuthology.orchestra.run.smithi152.stdout: "path": "/", 2024-09-11T22:54:42.366 INFO:teuthology.orchestra.run.smithi152.stdout: "tag": "3bd2da69-9286-4eb5-a206-d65bad41197c", 2024-09-11T22:54:42.367 INFO:teuthology.orchestra.run.smithi152.stdout: "options": "recursive,force" 2024-09-11T22:54:42.367 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:54:42.367 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:54:42.367 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:54:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.355+0000 7f02342db640 1 -- 172.21.15.152:0/3204204026 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f02180a2cf0 msgr2=0x7f02180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.355+0000 7f02342db640 1 --2- 172.21.15.152:0/3204204026 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f02180a2cf0 0x7f02180a51b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f022402fd80 tx=0x7f022402a950 comp rx=0 tx=0).stop 2024-09-11T22:54:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.355+0000 7f02342db640 1 -- 172.21.15.152:0/3204204026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f022c100e20 msgr2=0x7f022c198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.355+0000 7f02342db640 1 --2- 172.21.15.152:0/3204204026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f022c100e20 0x7f022c198460 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f022800d950 tx=0x7f022800de10 comp rx=0 tx=0).stop 2024-09-11T22:54:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.356+0000 7f02342db640 1 -- 172.21.15.152:0/3204204026 shutdown_connections 2024-09-11T22:54:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.356+0000 7f02342db640 1 --2- 172.21.15.152:0/3204204026 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f02180a2cf0 0x7f02180a51b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.356+0000 7f02342db640 1 --2- 172.21.15.152:0/3204204026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f022c102ef0 0x7f022c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.356+0000 7f02342db640 1 --2- 172.21.15.152:0/3204204026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f022c102570 0x7f022c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.356+0000 7f02342db640 1 --2- 172.21.15.152:0/3204204026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f022c100e20 0x7f022c198460 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.356+0000 7f02342db640 1 -- 172.21.15.152:0/3204204026 >> 172.21.15.152:0/3204204026 conn(0x7f022c0fc7d0 msgr2=0x7f022c106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:42.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.356+0000 7f02342db640 1 -- 172.21.15.152:0/3204204026 shutdown_connections 2024-09-11T22:54:42.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:42.356+0000 7f02342db640 1 -- 172.21.15.152:0/3204204026 wait complete. 2024-09-11T22:54:42.440 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:42.440 INFO:tasks.workunit.client.0.smithi152.stdout: Children see throughput for 1 readers = 7392199.00 kB/sec 2024-09-11T22:54:42.440 INFO:tasks.workunit.client.0.smithi152.stdout: Parent sees throughput for 1 readers = 7381529.17 kB/sec 2024-09-11T22:54:42.440 INFO:tasks.workunit.client.0.smithi152.stdout: Min throughput per process = 7392199.00 kB/sec 2024-09-11T22:54:42.440 INFO:tasks.workunit.client.0.smithi152.stdout: Max throughput per process = 7392199.00 kB/sec 2024-09-11T22:54:42.440 INFO:tasks.workunit.client.0.smithi152.stdout: Avg throughput per process = 7392199.00 kB/sec 2024-09-11T22:54:42.440 INFO:tasks.workunit.client.0.smithi152.stdout: Min xfer = 1048576.00 kB 2024-09-11T22:54:43.033 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:42 smithi152 ceph-mon[31181]: pgmap v57: 97 pgs: 97 active+clean; 1.0 GiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 144 MiB/s wr, 72 op/s 2024-09-11T22:54:43.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:42 smithi191 ceph-mon[35211]: pgmap v57: 97 pgs: 97 active+clean; 1.0 GiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 144 MiB/s wr, 72 op/s 2024-09-11T22:54:43.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:42 smithi204 ceph-mon[35454]: pgmap v57: 97 pgs: 97 active+clean; 1.0 GiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 144 MiB/s wr, 72 op/s 2024-09-11T22:54:43.385 INFO:tasks.cephfs.filesystem:scrub status for tag:3bd2da69-9286-4eb5-a206-d65bad41197c - {'path': '/', 'tag': '3bd2da69-9286-4eb5-a206-d65bad41197c', 'options': 'recursive,force'} 2024-09-11T22:54:44.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:43 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T22:54:44.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:43 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T22:54:44.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:43 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T22:54:44.890 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:44.891 INFO:tasks.workunit.client.0.smithi152.stdout: Children see throughput for 1 re-readers = 7052946.00 kB/sec 2024-09-11T22:54:44.891 INFO:tasks.workunit.client.0.smithi152.stdout: Parent sees throughput for 1 re-readers = 7042296.36 kB/sec 2024-09-11T22:54:44.891 INFO:tasks.workunit.client.0.smithi152.stdout: Min throughput per process = 7052946.00 kB/sec 2024-09-11T22:54:44.891 INFO:tasks.workunit.client.0.smithi152.stdout: Max throughput per process = 7052946.00 kB/sec 2024-09-11T22:54:44.891 INFO:tasks.workunit.client.0.smithi152.stdout: Avg throughput per process = 7052946.00 kB/sec 2024-09-11T22:54:44.891 INFO:tasks.workunit.client.0.smithi152.stdout: Min xfer = 1048576.00 kB 2024-09-11T22:54:45.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:44 smithi191 ceph-mon[35211]: pgmap v58: 97 pgs: 97 active+clean; 1.0 GiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 144 MiB/s wr, 72 op/s 2024-09-11T22:54:45.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:44 smithi204 ceph-mon[35454]: pgmap v58: 97 pgs: 97 active+clean; 1.0 GiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 144 MiB/s wr, 72 op/s 2024-09-11T22:54:45.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:44 smithi152 ceph-mon[31181]: pgmap v58: 97 pgs: 97 active+clean; 1.0 GiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 144 MiB/s wr, 72 op/s 2024-09-11T22:54:47.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:46 smithi191 ceph-mon[35211]: pgmap v59: 97 pgs: 97 active+clean; 1.0 GiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 171 MiB/s wr, 85 op/s 2024-09-11T22:54:47.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:46 smithi204 ceph-mon[35454]: pgmap v59: 97 pgs: 97 active+clean; 1.0 GiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 171 MiB/s wr, 85 op/s 2024-09-11T22:54:47.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:46 smithi152 ceph-mon[31181]: pgmap v59: 97 pgs: 97 active+clean; 1.0 GiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 171 MiB/s wr, 85 op/s 2024-09-11T22:54:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.598+0000 7fbb13d11640 1 -- 172.21.15.152:0/1869763504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c1009f0 msgr2=0x7fbb0c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.598+0000 7fbb13d11640 1 --2- 172.21.15.152:0/1869763504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c1009f0 0x7fbb0c100e50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009930 tx=0x7fbafc02a420 comp rx=0 tx=0).stop 2024-09-11T22:54:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.599+0000 7fbb13d11640 1 -- 172.21.15.152:0/1869763504 shutdown_connections 2024-09-11T22:54:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.599+0000 7fbb13d11640 1 --2- 172.21.15.152:0/1869763504 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb0c101390 0x7fbb0c10d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.599+0000 7fbb13d11640 1 --2- 172.21.15.152:0/1869763504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c1009f0 0x7fbb0c100e50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.599+0000 7fbb13d11640 1 --2- 172.21.15.152:0/1869763504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb0c106a00 0x7fbb0c106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.599+0000 7fbb13d11640 1 -- 172.21.15.152:0/1869763504 >> 172.21.15.152:0/1869763504 conn(0x7fbb0c0fc6b0 msgr2=0x7fbb0c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.599+0000 7fbb13d11640 1 -- 172.21.15.152:0/1869763504 shutdown_connections 2024-09-11T22:54:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.599+0000 7fbb13d11640 1 -- 172.21.15.152:0/1869763504 wait complete. 2024-09-11T22:54:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.599+0000 7fbb13d11640 1 Processor -- start 2024-09-11T22:54:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.599+0000 7fbb13d11640 1 -- start start 2024-09-11T22:54:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb13d11640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb0c1009f0 0x7fbb0c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb13d11640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb0c101390 0x7fbb0c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb13d11640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c106a00 0x7fbb0c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb13d11640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c111ef0 con 0x7fbb0c106a00 2024-09-11T22:54:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb13d11640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c111d70 con 0x7fbb0c101390 2024-09-11T22:54:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb13d11640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c112070 con 0x7fbb0c1009f0 2024-09-11T22:54:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb122ae640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c106a00 0x7fbb0c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb122ae640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c106a00 0x7fbb0c19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48178/0 (socket says 172.21.15.152:48178) 2024-09-11T22:54:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb122ae640 1 -- 172.21.15.152:0/1195666841 learned_addr learned my addr 172.21.15.152:0/1195666841 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb11aad640 1 --2- 172.21.15.152:0/1195666841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb0c1009f0 0x7fbb0c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb112ac640 1 --2- 172.21.15.152:0/1195666841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb0c101390 0x7fbb0c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb122ae640 1 -- 172.21.15.152:0/1195666841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb0c1009f0 msgr2=0x7fbb0c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb122ae640 1 --2- 172.21.15.152:0/1195666841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb0c1009f0 0x7fbb0c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb122ae640 1 -- 172.21.15.152:0/1195666841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb0c101390 msgr2=0x7fbb0c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb122ae640 1 --2- 172.21.15.152:0/1195666841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb0c101390 0x7fbb0c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb122ae640 1 -- 172.21.15.152:0/1195666841 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c19d390 con 0x7fbb0c106a00 2024-09-11T22:54:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb112ac640 1 --2- 172.21.15.152:0/1195666841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb0c101390 0x7fbb0c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:54:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb122ae640 1 --2- 172.21.15.152:0/1195666841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c106a00 0x7fbb0c19cc60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400eb10 tx=0x7fbb0400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb02ffd640 1 -- 172.21.15.152:0/1195666841 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04004b20 con 0x7fbb0c106a00 2024-09-11T22:54:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb02ffd640 1 -- 172.21.15.152:0/1195666841 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb04004cc0 con 0x7fbb0c106a00 2024-09-11T22:54:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb02ffd640 1 -- 172.21.15.152:0/1195666841 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04010760 con 0x7fbb0c106a00 2024-09-11T22:54:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb13d11640 1 -- 172.21.15.152:0/1195666841 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0c19d680 con 0x7fbb0c106a00 2024-09-11T22:54:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.600+0000 7fbb13d11640 1 -- 172.21.15.152:0/1195666841 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0c1a4f10 con 0x7fbb0c106a00 2024-09-11T22:54:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.604+0000 7fbb02ffd640 1 -- 172.21.15.152:0/1195666841 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fbb04005020 con 0x7fbb0c106a00 2024-09-11T22:54:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.604+0000 7fbb16aa9740 1 Processor -- start 2024-09-11T22:54:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.604+0000 7fbb16aa9740 1 -- start start 2024-09-11T22:54:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.604+0000 7fbb16aa9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068cca9b0 0x558068ccad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.604+0000 7fbb16aa9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068ccb2d0 0x558068cd9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.604+0000 7fbb16aa9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cdc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.604+0000 7fbb16aa9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558068c2f2f0 con 0x558068cda0c0 2024-09-11T22:54:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.604+0000 7fbb16aa9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558068bd7270 con 0x558068ccb2d0 2024-09-11T22:54:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.604+0000 7fbb16aa9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558068c2b2b0 con 0x558068cca9b0 2024-09-11T22:54:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.604+0000 7fbb112ac640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cdc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbb112ac640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cdc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48188/0 (socket says 172.21.15.152:48188) 2024-09-11T22:54:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbb112ac640 1 -- 172.21.15.152:0/201544171 learned_addr learned my addr 172.21.15.152:0/201544171 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbb11aad640 1 --2- 172.21.15.152:0/201544171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068ccb2d0 0x558068cd9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbb112ac640 1 --2- 172.21.15.152:0/201544171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068cca9b0 0x558068ccad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbb02ffd640 1 --2- 172.21.15.152:0/1195666841 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbaf80a2da0 0x7fbaf80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbb122ae640 1 --2- 172.21.15.152:0/1195666841 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbaf80a2da0 0x7fbaf80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbb112ac640 1 -- 172.21.15.152:0/201544171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068cca9b0 msgr2=0x558068ccad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbb112ac640 1 --2- 172.21.15.152:0/201544171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068cca9b0 0x558068ccad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbb112ac640 1 -- 172.21.15.152:0/201544171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068ccb2d0 msgr2=0x558068cd9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbb112ac640 1 --2- 172.21.15.152:0/201544171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068ccb2d0 0x558068cd9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbb112ac640 1 -- 172.21.15.152:0/201544171 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558068bd6340 con 0x558068cda0c0 2024-09-11T22:54:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbb02ffd640 1 -- 172.21.15.152:0/1195666841 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fbb040cfae0 con 0x7fbb0c106a00 2024-09-11T22:54:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbb112ac640 1 --2- 172.21.15.152:0/201544171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cdc4b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0057c0 tx=0x7fbafc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ba50e23d4213a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbb122ae640 1 --2- 172.21.15.152:0/1195666841 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbaf80a2da0 0x7fbaf80a5260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb040984b0 tx=0x7fbb0400efd0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbaf7fff640 1 -- 172.21.15.152:0/201544171 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc045070 con 0x558068cda0c0 2024-09-11T22:54:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbaf7fff640 1 -- 172.21.15.152:0/201544171 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbafc0035f0 con 0x558068cda0c0 2024-09-11T22:54:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.605+0000 7fbaf7fff640 1 -- 172.21.15.152:0/201544171 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc035040 con 0x558068cda0c0 2024-09-11T22:54:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.606+0000 7fbb16aa9740 1 -- 172.21.15.152:0/201544171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 msgr2=0x558068cdc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.606+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/201544171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cdc4b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0057c0 tx=0x7fbafc000bc0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.606+0000 7fbb16aa9740 1 -- 172.21.15.152:0/201544171 shutdown_connections 2024-09-11T22:54:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.606+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/201544171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cdc4b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.606+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/201544171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068ccb2d0 0x558068cd9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.606+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/201544171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068cca9b0 0x558068ccad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.606+0000 7fbb16aa9740 1 -- 172.21.15.152:0/201544171 >> 172.21.15.152:0/201544171 conn(0x558068cc7060 msgr2=0x558068cc7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.606+0000 7fbb16aa9740 1 -- 172.21.15.152:0/201544171 shutdown_connections 2024-09-11T22:54:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.606+0000 7fbb16aa9740 1 -- 172.21.15.152:0/201544171 wait complete. 2024-09-11T22:54:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.607+0000 7fbb16aa9740 1 -- start start 2024-09-11T22:54:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.607+0000 7fbb16aa9740 1 Processor -- start 2024-09-11T22:54:48.615 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.607+0000 7fbb16aa9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068cca9b0 0x558068cd7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.607+0000 7fbb16aa9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068ccb2d0 0x558068cd75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.607+0000 7fbb16aa9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068cda0c0 0x558068cd7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.607+0000 7fbb16aa9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558068cdcc30 con 0x558068ccb2d0 2024-09-11T22:54:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.607+0000 7fbb16aa9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558068bb8290 con 0x558068cca9b0 2024-09-11T22:54:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.607+0000 7fbb16aa9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558068cdcdb0 con 0x558068cda0c0 2024-09-11T22:54:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.607+0000 7fbb16aa9740 10 client.0 _ll_register_callbacks cb 0x558068be9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.607+0000 7fbb11aad640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068ccb2d0 0x558068cd75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.607+0000 7fbb11aad640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068ccb2d0 0x558068cd75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48190/0 (socket says 172.21.15.152:48190) 2024-09-11T22:54:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.607+0000 7fbb11aad640 1 -- 172.21.15.152:0/3193220712 learned_addr learned my addr 172.21.15.152:0/3193220712 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.608+0000 7fbb112ac640 1 --2- 172.21.15.152:0/3193220712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068cda0c0 0x558068cd7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.608+0000 7fbb11aad640 1 -- 172.21.15.152:0/3193220712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068cda0c0 msgr2=0x558068cd7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.608+0000 7fbb11aad640 1 --2- 172.21.15.152:0/3193220712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068cda0c0 0x558068cd7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.608+0000 7fbb11aad640 1 -- 172.21.15.152:0/3193220712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068cca9b0 msgr2=0x558068cd7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.608+0000 7fbb11aad640 1 --2- 172.21.15.152:0/3193220712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068cca9b0 0x558068cd7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.608+0000 7fbb11aad640 1 -- 172.21.15.152:0/3193220712 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558068942210 con 0x558068ccb2d0 2024-09-11T22:54:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.608+0000 7fbb112ac640 1 --2- 172.21.15.152:0/3193220712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068cda0c0 0x558068cd7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:54:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.608+0000 7fbb11aad640 1 --2- 172.21.15.152:0/3193220712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068ccb2d0 0x558068cd75b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800bf50 tx=0x7fbb080128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4333320a9f692210 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.608+0000 7fbaf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.608+0000 7fbaf57fa640 1 -- 172.21.15.152:0/3193220712 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0801f060 con 0x558068ccb2d0 2024-09-11T22:54:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.608+0000 7fbaf57fa640 1 -- 172.21.15.152:0/3193220712 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fbb080108a0 con 0x558068ccb2d0 2024-09-11T22:54:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.608+0000 7fbb16aa9740 1 -- 172.21.15.152:0/3193220712 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbafc0035f0 con 0x558068ccb2d0 2024-09-11T22:54:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.608+0000 7fbaf57fa640 1 -- 172.21.15.152:0/3193220712 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb08010aa0 con 0x558068ccb2d0 2024-09-11T22:54:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.608+0000 7fbb16aa9740 10 client.16053 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.608+0000 7fbb16aa9740 1 -- 172.21.15.152:0/3193220712 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558068a0e620 con 0x558068ccb2d0 2024-09-11T22:54:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.608+0000 7fbaf57fa640 1 -- 172.21.15.152:0/3193220712 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbafc0035f0 con 0x558068ccb2d0 2024-09-11T22:54:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.609+0000 7fbaf57fa640 1 -- 172.21.15.152:0/3193220712 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbb08019030 con 0x558068ccb2d0 2024-09-11T22:54:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.609+0000 7fbb16aa9740 10 client.16053 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.609+0000 7fbb16aa9740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:54:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.609+0000 7fbb16aa9740 20 client.16053 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.609+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/3193220712 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558068cea0c0 0x558068cec4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.609+0000 7fbb16aa9740 4 client.16053 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:54:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.609+0000 7fbb16aa9740 1 -- 172.21.15.152:0/3193220712 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558068c3ca20 con 0x558068cea0c0 2024-09-11T22:54:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.609+0000 7fbb112ac640 1 --2- 172.21.15.152:0/3193220712 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558068cea0c0 0x558068cec4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.609+0000 7fbaf57fa640 10 client.16053 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.609+0000 7fbb112ac640 1 --2- 172.21.15.152:0/3193220712 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558068cea0c0 0x558068cec4f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16fb15b0a0adefbe server_cookie=25489b746e8a36b8 in_seq=0 out_seq=0 2024-09-11T22:54:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.610+0000 7fbaf57fa640 1 -- 172.21.15.152:0/3193220712 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558068c3ca20 con 0x558068cea0c0 2024-09-11T22:54:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.610+0000 7fbaf57fa640 10 client.16053 handle_command_reply: tid=0 2024-09-11T22:54:48.618 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.610+0000 7fbb16aa9740 1 client.16053 shutdown 2024-09-11T22:54:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.610+0000 7fbb16aa9740 1 -- 172.21.15.152:0/3193220712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068ccb2d0 msgr2=0x558068cd75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.610+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/3193220712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068ccb2d0 0x558068cd75b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800bf50 tx=0x7fbb080128e0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.610+0000 7fbb16aa9740 1 -- 172.21.15.152:0/3193220712 shutdown_connections 2024-09-11T22:54:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.610+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/3193220712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068cda0c0 0x558068cd7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.610+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/3193220712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068ccb2d0 0x558068cd75b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.610+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/3193220712 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558068cea0c0 0x558068cec4f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.610+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/3193220712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068cca9b0 0x558068cd7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.610+0000 7fbaf57fa640 0 client.16053 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.610+0000 7fbb16aa9740 1 -- 172.21.15.152:0/3193220712 >> 172.21.15.152:0/3193220712 conn(0x558068cc7060 msgr2=0x558068cc61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.610+0000 7fbb16aa9740 1 -- 172.21.15.152:0/3193220712 shutdown_connections 2024-09-11T22:54:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.610+0000 7fbb16aa9740 1 -- 172.21.15.152:0/3193220712 wait complete. 2024-09-11T22:54:48.619 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.611+0000 7fbb16aa9740 20 client.16053 trim_cache size 0 max 16384 2024-09-11T22:54:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.625+0000 7fbb16aa9740 1 Processor -- start 2024-09-11T22:54:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.625+0000 7fbb16aa9740 1 -- start start 2024-09-11T22:54:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.625+0000 7fbb16aa9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068cca9b0 0x558068cf1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.625+0000 7fbb16aa9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068ccb2d0 0x558068cf2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.625+0000 7fbb16aa9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cd42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.625+0000 7fbb16aa9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1ab5f0 con 0x558068cda0c0 2024-09-11T22:54:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.625+0000 7fbb16aa9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1ad130 con 0x558068cca9b0 2024-09-11T22:54:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.625+0000 7fbb16aa9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1abcc0 con 0x558068ccb2d0 2024-09-11T22:54:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.625+0000 7fbb112ac640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cd42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.625+0000 7fbb112ac640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cd42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48202/0 (socket says 172.21.15.152:48202) 2024-09-11T22:54:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.625+0000 7fbb112ac640 1 -- 172.21.15.152:0/3428529744 learned_addr learned my addr 172.21.15.152:0/3428529744 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.626+0000 7fbb112ac640 1 --2- 172.21.15.152:0/3428529744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068cca9b0 0x558068cf1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.626+0000 7fbb11aad640 1 --2- 172.21.15.152:0/3428529744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068ccb2d0 0x558068cf2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.626+0000 7fbb112ac640 1 -- 172.21.15.152:0/3428529744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068ccb2d0 msgr2=0x558068cf2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.626+0000 7fbb112ac640 1 --2- 172.21.15.152:0/3428529744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068ccb2d0 0x558068cf2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.626+0000 7fbb112ac640 1 -- 172.21.15.152:0/3428529744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068cca9b0 msgr2=0x558068cf1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:48.632 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.626+0000 7fbb112ac640 1 --2- 172.21.15.152:0/3428529744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068cca9b0 0x558068cf1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.626+0000 7fbb112ac640 1 -- 172.21.15.152:0/3428529744 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558068cd55c0 con 0x558068cda0c0 2024-09-11T22:54:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.626+0000 7fbb112ac640 1 --2- 172.21.15.152:0/3428529744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cd42f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbafc006010 tx=0x7fbafc02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1f750545fa10f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.626+0000 7fbaf77fe640 1 -- 172.21.15.152:0/3428529744 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc035050 con 0x558068cda0c0 2024-09-11T22:54:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.626+0000 7fbaf77fe640 1 -- 172.21.15.152:0/3428529744 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbafc042040 con 0x558068cda0c0 2024-09-11T22:54:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.626+0000 7fbaf77fe640 1 -- 172.21.15.152:0/3428529744 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc045040 con 0x558068cda0c0 2024-09-11T22:54:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 -- 172.21.15.152:0/3428529744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 msgr2=0x558068cd42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/3428529744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cd42f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbafc006010 tx=0x7fbafc02a420 comp rx=0 tx=0).stop 2024-09-11T22:54:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 -- 172.21.15.152:0/3428529744 shutdown_connections 2024-09-11T22:54:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/3428529744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cd42f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/3428529744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068ccb2d0 0x558068cf2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/3428529744 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068cca9b0 0x558068cf1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 -- 172.21.15.152:0/3428529744 >> 172.21.15.152:0/3428529744 conn(0x558068cc7060 msgr2=0x558068cd90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 -- 172.21.15.152:0/3428529744 shutdown_connections 2024-09-11T22:54:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 -- 172.21.15.152:0/3428529744 wait complete. 2024-09-11T22:54:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 -- start start 2024-09-11T22:54:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 Processor -- start 2024-09-11T22:54:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068cca9b0 0x558068ce24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068ccb2d0 0x558068cdf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cdf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1aca60 con 0x558068cda0c0 2024-09-11T22:54:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1ac390 con 0x558068cca9b0 2024-09-11T22:54:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1a95b0 con 0x558068ccb2d0 2024-09-11T22:54:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.627+0000 7fbb16aa9740 10 client.0 _ll_register_callbacks cb 0x7fbb0800b790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:54:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbb112ac640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cdf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbb112ac640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cdf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48206/0 (socket says 172.21.15.152:48206) 2024-09-11T22:54:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbb112ac640 1 -- 172.21.15.152:0/1247983356 learned_addr learned my addr 172.21.15.152:0/1247983356 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbb112ac640 1 --2- 172.21.15.152:0/1247983356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068cca9b0 0x558068ce24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbb11aad640 1 --2- 172.21.15.152:0/1247983356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068ccb2d0 0x558068cdf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbb112ac640 1 -- 172.21.15.152:0/1247983356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068ccb2d0 msgr2=0x558068cdf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbb112ac640 1 --2- 172.21.15.152:0/1247983356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068ccb2d0 0x558068cdf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbb112ac640 1 -- 172.21.15.152:0/1247983356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068cca9b0 msgr2=0x558068ce24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:54:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbb112ac640 1 --2- 172.21.15.152:0/1247983356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068cca9b0 0x558068ce24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbb112ac640 1 -- 172.21.15.152:0/1247983356 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558068cdfac0 con 0x558068cda0c0 2024-09-11T22:54:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbb11aad640 1 --2- 172.21.15.152:0/1247983356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068ccb2d0 0x558068cdf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:54:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbb112ac640 1 --2- 172.21.15.152:0/1247983356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cdf550 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbafc04d040 tx=0x7fbafc0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f549d41daee6786a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbaf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:54:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbaf57fa640 1 -- 172.21.15.152:0/1247983356 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc035070 con 0x558068cda0c0 2024-09-11T22:54:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbaf57fa640 1 -- 172.21.15.152:0/1247983356 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fbafc04a680 con 0x558068cda0c0 2024-09-11T22:54:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbb16aa9740 1 -- 172.21.15.152:0/1247983356 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbafc042040 con 0x558068cda0c0 2024-09-11T22:54:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbaf57fa640 1 -- 172.21.15.152:0/1247983356 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbafc060030 con 0x558068cda0c0 2024-09-11T22:54:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbb16aa9740 10 client.16059 fetch_fsmap learned FSMap version 28 2024-09-11T22:54:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.628+0000 7fbb16aa9740 1 -- 172.21.15.152:0/1247983356 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558068d55b50 con 0x558068cda0c0 2024-09-11T22:54:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.629+0000 7fbaf57fa640 1 -- 172.21.15.152:0/1247983356 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbafc042040 con 0x558068cda0c0 2024-09-11T22:54:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.629+0000 7fbaf57fa640 1 -- 172.21.15.152:0/1247983356 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbafc052070 con 0x558068cda0c0 2024-09-11T22:54:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.629+0000 7fbb16aa9740 10 client.16059 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:54:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.629+0000 7fbb16aa9740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:54:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.629+0000 7fbb16aa9740 20 client.16059 populate_metadata read hostname 'smithi152' 2024-09-11T22:54:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.629+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/1247983356 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558068d55d80 0x558068d58170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.629+0000 7fbb16aa9740 4 client.16059 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:54:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.629+0000 7fbb16aa9740 1 -- 172.21.15.152:0/1247983356 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558068c36d70 con 0x558068d55d80 2024-09-11T22:54:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.629+0000 7fbb11aad640 1 --2- 172.21.15.152:0/1247983356 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558068d55d80 0x558068d58170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.629+0000 7fbaf57fa640 10 client.16059 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.629+0000 7fbb11aad640 1 --2- 172.21.15.152:0/1247983356 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558068d55d80 0x558068d58170 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bc2611efde7ebd server_cookie=b0292d537e1b9686 in_seq=0 out_seq=0 2024-09-11T22:54:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.639+0000 7fbaf57fa640 1 -- 172.21.15.152:0/1247983356 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558068c36d70 con 0x558068d55d80 2024-09-11T22:54:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.639+0000 7fbaf57fa640 10 client.16059 handle_command_reply: tid=0 2024-09-11T22:54:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.639+0000 7fbb16aa9740 1 client.16059 shutdown 2024-09-11T22:54:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.639+0000 7fbb16aa9740 1 -- 172.21.15.152:0/1247983356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 msgr2=0x558068cdf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.639+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/1247983356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cdf550 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbafc04d040 tx=0x7fbafc0092d0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb16aa9740 1 -- 172.21.15.152:0/1247983356 shutdown_connections 2024-09-11T22:54:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/1247983356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558068cda0c0 0x558068cdf550 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/1247983356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558068ccb2d0 0x558068cdf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/1247983356 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558068d55d80 0x558068d58170 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb16aa9740 1 --2- 172.21.15.152:0/1247983356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558068cca9b0 0x558068ce24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbaf57fa640 0 client.16059 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:54:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb16aa9740 1 -- 172.21.15.152:0/1247983356 >> 172.21.15.152:0/1247983356 conn(0x558068cc7060 msgr2=0x558068cc59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb16aa9740 1 -- 172.21.15.152:0/1247983356 shutdown_connections 2024-09-11T22:54:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb16aa9740 1 -- 172.21.15.152:0/1247983356 wait complete. 2024-09-11T22:54:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb16aa9740 20 client.16059 trim_cache size 0 max 16384 2024-09-11T22:54:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:{ 2024-09-11T22:54:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "message": "", 2024-09-11T22:54:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "return_code": 0 2024-09-11T22:54:48.643 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:} 2024-09-11T22:54:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb13d11640 1 -- 172.21.15.152:0/1195666841 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbaf80a2da0 msgr2=0x7fbaf80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb13d11640 1 --2- 172.21.15.152:0/1195666841 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbaf80a2da0 0x7fbaf80a5260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb040984b0 tx=0x7fbb0400efd0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb13d11640 1 -- 172.21.15.152:0/1195666841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c106a00 msgr2=0x7fbb0c19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb13d11640 1 --2- 172.21.15.152:0/1195666841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c106a00 0x7fbb0c19cc60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400eb10 tx=0x7fbb0400c2d0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb13d11640 1 -- 172.21.15.152:0/1195666841 shutdown_connections 2024-09-11T22:54:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb13d11640 1 --2- 172.21.15.152:0/1195666841 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbaf80a2da0 0x7fbaf80a5260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb13d11640 1 --2- 172.21.15.152:0/1195666841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0c106a00 0x7fbb0c19cc60 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb13d11640 1 --2- 172.21.15.152:0/1195666841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb0c101390 0x7fbb0c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.644 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb13d11640 1 --2- 172.21.15.152:0/1195666841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbb0c1009f0 0x7fbb0c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb13d11640 1 -- 172.21.15.152:0/1195666841 >> 172.21.15.152:0/1195666841 conn(0x7fbb0c0fc6b0 msgr2=0x7fbb0c0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb13d11640 1 -- 172.21.15.152:0/1195666841 shutdown_connections 2024-09-11T22:54:48.645 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:54:48.640+0000 7fbb13d11640 1 -- 172.21.15.152:0/1195666841 wait complete. 2024-09-11T22:54:49.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:48 smithi191 ceph-mon[35211]: pgmap v60: 97 pgs: 97 active+clean; 1.0 GiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 171 MiB/s wr, 80 op/s 2024-09-11T22:54:49.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:48 smithi204 ceph-mon[35454]: pgmap v60: 97 pgs: 97 active+clean; 1.0 GiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 171 MiB/s wr, 80 op/s 2024-09-11T22:54:49.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:48 smithi152 ceph-mon[31181]: pgmap v60: 97 pgs: 97 active+clean; 1.0 GiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 171 MiB/s wr, 80 op/s 2024-09-11T22:54:51.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:50 smithi191 ceph-mon[35211]: pgmap v61: 97 pgs: 97 active+clean; 1.0 GiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 171 MiB/s wr, 87 op/s 2024-09-11T22:54:51.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:50 smithi204 ceph-mon[35454]: pgmap v61: 97 pgs: 97 active+clean; 1.0 GiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 171 MiB/s wr, 87 op/s 2024-09-11T22:54:51.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:50 smithi152 ceph-mon[31181]: pgmap v61: 97 pgs: 97 active+clean; 1.0 GiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 171 MiB/s wr, 87 op/s 2024-09-11T22:54:51.230 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:51.230 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:51.230 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:51.231 INFO:tasks.workunit.client.0.smithi152.stdout:iozone test complete. 2024-09-11T22:54:51.231 INFO:tasks.workunit.client.0.smithi152.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-11T22:54:51.296 INFO:tasks.workunit.client.0.smithi152.stdout: Iozone: Performance Test of File I/O 2024-09-11T22:54:51.296 INFO:tasks.workunit.client.0.smithi152.stdout: Version $Revision: 3.494 $ 2024-09-11T22:54:51.296 INFO:tasks.workunit.client.0.smithi152.stdout: Compiled for 64 bit mode. 2024-09-11T22:54:51.296 INFO:tasks.workunit.client.0.smithi152.stdout: Build: linux-AMD64 2024-09-11T22:54:51.297 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:51.297 INFO:tasks.workunit.client.0.smithi152.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-09-11T22:54:51.297 INFO:tasks.workunit.client.0.smithi152.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-09-11T22:54:51.297 INFO:tasks.workunit.client.0.smithi152.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-09-11T22:54:51.297 INFO:tasks.workunit.client.0.smithi152.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-09-11T22:54:51.297 INFO:tasks.workunit.client.0.smithi152.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-09-11T22:54:51.297 INFO:tasks.workunit.client.0.smithi152.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-09-11T22:54:51.297 INFO:tasks.workunit.client.0.smithi152.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-09-11T22:54:51.297 INFO:tasks.workunit.client.0.smithi152.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-09-11T22:54:51.297 INFO:tasks.workunit.client.0.smithi152.stdout: Alexey Skidanov, Sudhir Kumar. 2024-09-11T22:54:51.297 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:51.297 INFO:tasks.workunit.client.0.smithi152.stdout: Run began: Wed Sep 11 22:54:51 2024 2024-09-11T22:54:51.298 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:54:51.298 INFO:tasks.workunit.client.0.smithi152.stdout: Include close in write timing 2024-09-11T22:54:51.298 INFO:tasks.workunit.client.0.smithi152.stdout: Include fsync in write timing 2024-09-11T22:54:51.298 INFO:tasks.workunit.client.0.smithi152.stdout: File size set to 10485760 kB 2024-09-11T22:54:51.298 INFO:tasks.workunit.client.0.smithi152.stdout: Record Size 1024 kB 2024-09-11T22:54:51.298 INFO:tasks.workunit.client.0.smithi152.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-09-11T22:54:51.298 INFO:tasks.workunit.client.0.smithi152.stdout: Output is in kBytes/sec 2024-09-11T22:54:51.298 INFO:tasks.workunit.client.0.smithi152.stdout: Time Resolution = 0.000001 seconds. 2024-09-11T22:54:51.298 INFO:tasks.workunit.client.0.smithi152.stdout: Processor cache size set to 1024 kBytes. 2024-09-11T22:54:51.298 INFO:tasks.workunit.client.0.smithi152.stdout: Processor cache line size set to 32 bytes. 2024-09-11T22:54:51.298 INFO:tasks.workunit.client.0.smithi152.stdout: File stride size set to 17 * record size. 2024-09-11T22:54:51.298 INFO:tasks.workunit.client.0.smithi152.stdout: Throughput test with 1 process 2024-09-11T22:54:51.299 INFO:tasks.workunit.client.0.smithi152.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-09-11T22:54:53.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:52 smithi191 ceph-mon[35211]: pgmap v62: 97 pgs: 97 active+clean; 1.0 GiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-11T22:54:53.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:52 smithi204 ceph-mon[35454]: pgmap v62: 97 pgs: 97 active+clean; 1.0 GiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-11T22:54:53.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:52 smithi152 ceph-mon[31181]: pgmap v62: 97 pgs: 97 active+clean; 1.0 GiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-11T22:54:53.991 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-11T22:54:53.992 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:54:54.503 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:54:55.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:54 smithi191 ceph-mon[35211]: pgmap v63: 97 pgs: 97 active+clean; 1.0 GiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-11T22:54:55.048 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:54 smithi204 ceph-mon[35454]: pgmap v63: 97 pgs: 97 active+clean; 1.0 GiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-11T22:54:55.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:54 smithi152 ceph-mon[31181]: pgmap v63: 97 pgs: 97 active+clean; 1.0 GiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 26 MiB/s wr, 19 op/s 2024-09-11T22:54:56.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.288+0000 7f8d3b9cd640 1 -- 172.21.15.152:0/3340625921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d34100de0 msgr2=0x7f8d34104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:56.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.288+0000 7f8d3b9cd640 1 --2- 172.21.15.152:0/3340625921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d34100de0 0x7f8d34104e30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8d24009950 tx=0x7f8d2402a480 comp rx=0 tx=0).stop 2024-09-11T22:54:56.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.288+0000 7f8d3b9cd640 1 -- 172.21.15.152:0/3340625921 shutdown_connections 2024-09-11T22:54:56.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.288+0000 7f8d3b9cd640 1 --2- 172.21.15.152:0/3340625921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d34105560 0x7f8d3410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:56.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.288+0000 7f8d3b9cd640 1 --2- 172.21.15.152:0/3340625921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d34100de0 0x7f8d34104e30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:56.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.288+0000 7f8d3b9cd640 1 --2- 172.21.15.152:0/3340625921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d34100430 0x7f8d34100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:56.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.288+0000 7f8d3b9cd640 1 -- 172.21.15.152:0/3340625921 >> 172.21.15.152:0/3340625921 conn(0x7f8d34077230 msgr2=0x7f8d340fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:56.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.289+0000 7f8d3b9cd640 1 -- 172.21.15.152:0/3340625921 shutdown_connections 2024-09-11T22:54:56.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.289+0000 7f8d3b9cd640 1 -- 172.21.15.152:0/3340625921 wait complete. 2024-09-11T22:54:56.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.290+0000 7f8d3b9cd640 1 Processor -- start 2024-09-11T22:54:56.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.290+0000 7f8d3b9cd640 1 -- start start 2024-09-11T22:54:56.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.290+0000 7f8d3b9cd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d34100430 0x7f8d3419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:56.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.290+0000 7f8d3b9cd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d34100de0 0x7f8d3419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:56.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.290+0000 7f8d3b9cd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d34105560 0x7f8d3419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:56.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.290+0000 7f8d3b9cd640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8d34110e80 con 0x7f8d34105560 2024-09-11T22:54:56.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.290+0000 7f8d3b9cd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8d34110d00 con 0x7f8d34100de0 2024-09-11T22:54:56.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.290+0000 7f8d3b9cd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d34111000 con 0x7f8d34100430 2024-09-11T22:54:56.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.290+0000 7f8d39f6a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d34105560 0x7f8d3419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:56.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.290+0000 7f8d39f6a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d34105560 0x7f8d3419efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48232/0 (socket says 172.21.15.152:48232) 2024-09-11T22:54:56.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.290+0000 7f8d39f6a640 1 -- 172.21.15.152:0/1844847313 learned_addr learned my addr 172.21.15.152:0/1844847313 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:54:56.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.290+0000 7f8d39769640 1 --2- 172.21.15.152:0/1844847313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d34100430 0x7f8d3419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:56.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.290+0000 7f8d39f6a640 1 -- 172.21.15.152:0/1844847313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d34100430 msgr2=0x7f8d3419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:56.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.290+0000 7f8d38f68640 1 --2- 172.21.15.152:0/1844847313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d34100de0 0x7f8d3419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:56.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.291+0000 7f8d39f6a640 1 --2- 172.21.15.152:0/1844847313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d34100430 0x7f8d3419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:56.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.291+0000 7f8d39f6a640 1 -- 172.21.15.152:0/1844847313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d34100de0 msgr2=0x7f8d3419ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:56.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.291+0000 7f8d39f6a640 1 --2- 172.21.15.152:0/1844847313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d34100de0 0x7f8d3419ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:56.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.291+0000 7f8d39f6a640 1 -- 172.21.15.152:0/1844847313 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d3419f6b0 con 0x7f8d34105560 2024-09-11T22:54:56.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.291+0000 7f8d38f68640 1 --2- 172.21.15.152:0/1844847313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d34100de0 0x7f8d3419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:54:56.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.291+0000 7f8d39769640 1 --2- 172.21.15.152:0/1844847313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d34100430 0x7f8d3419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:54:56.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.291+0000 7f8d39f6a640 1 --2- 172.21.15.152:0/1844847313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d34105560 0x7f8d3419efd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00cad0 tx=0x7f8d2c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:56.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.291+0000 7f8d2a7fc640 1 -- 172.21.15.152:0/1844847313 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c00ae20 con 0x7f8d34105560 2024-09-11T22:54:56.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.291+0000 7f8d2a7fc640 1 -- 172.21.15.152:0/1844847313 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d2c005020 con 0x7f8d34105560 2024-09-11T22:54:56.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.292+0000 7f8d2a7fc640 1 -- 172.21.15.152:0/1844847313 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c00f6f0 con 0x7f8d34105560 2024-09-11T22:54:56.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.292+0000 7f8d3b9cd640 1 -- 172.21.15.152:0/1844847313 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d3419f9a0 con 0x7f8d34105560 2024-09-11T22:54:56.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.292+0000 7f8d3b9cd640 1 -- 172.21.15.152:0/1844847313 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d341a7280 con 0x7f8d34105560 2024-09-11T22:54:56.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.293+0000 7f8d3b9cd640 1 -- 172.21.15.152:0/1844847313 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d34100810 con 0x7f8d34105560 2024-09-11T22:54:56.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.295+0000 7f8d2a7fc640 1 -- 172.21.15.152:0/1844847313 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8d2c0051d0 con 0x7f8d34105560 2024-09-11T22:54:56.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.296+0000 7f8d2a7fc640 1 --2- 172.21.15.152:0/1844847313 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8d200a2cf0 0x7f8d200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:54:56.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.296+0000 7f8d2a7fc640 1 -- 172.21.15.152:0/1844847313 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f8d2c0cf550 con 0x7f8d34105560 2024-09-11T22:54:56.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.296+0000 7f8d39769640 1 --2- 172.21.15.152:0/1844847313 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8d200a2cf0 0x7f8d200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:54:56.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.297+0000 7f8d39769640 1 --2- 172.21.15.152:0/1844847313 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8d200a2cf0 0x7f8d200a51b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000a8b0 tx=0x7f8d30008040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:54:56.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.303+0000 7f8d2a7fc640 1 -- 172.21.15.152:0/1844847313 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8d2c014030 con 0x7f8d34105560 2024-09-11T22:54:56.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.669+0000 7f8d3b9cd640 1 -- 172.21.15.152:0/1844847313 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d3402d7c0 con 0x7f8d34105560 2024-09-11T22:54:56.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.670+0000 7f8d2a7fc640 1 -- 172.21.15.152:0/1844847313 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f8d2c09c1c0 con 0x7f8d34105560 2024-09-11T22:54:56.755 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:54:56.755 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:54:56.755 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:54:56.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.676+0000 7f8d3b9cd640 1 -- 172.21.15.152:0/1844847313 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8d200a2cf0 msgr2=0x7f8d200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:56.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.676+0000 7f8d3b9cd640 1 --2- 172.21.15.152:0/1844847313 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8d200a2cf0 0x7f8d200a51b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000a8b0 tx=0x7f8d30008040 comp rx=0 tx=0).stop 2024-09-11T22:54:56.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.676+0000 7f8d3b9cd640 1 -- 172.21.15.152:0/1844847313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d34105560 msgr2=0x7f8d3419efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:54:56.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.676+0000 7f8d3b9cd640 1 --2- 172.21.15.152:0/1844847313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d34105560 0x7f8d3419efd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00cad0 tx=0x7f8d2c00cf90 comp rx=0 tx=0).stop 2024-09-11T22:54:56.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.676+0000 7f8d3b9cd640 1 -- 172.21.15.152:0/1844847313 shutdown_connections 2024-09-11T22:54:56.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.676+0000 7f8d3b9cd640 1 --2- 172.21.15.152:0/1844847313 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8d200a2cf0 0x7f8d200a51b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:56.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.676+0000 7f8d3b9cd640 1 --2- 172.21.15.152:0/1844847313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d34105560 0x7f8d3419efd0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:56.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.676+0000 7f8d3b9cd640 1 --2- 172.21.15.152:0/1844847313 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d34100de0 0x7f8d3419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:56.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.676+0000 7f8d3b9cd640 1 --2- 172.21.15.152:0/1844847313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d34100430 0x7f8d3419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:54:56.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.676+0000 7f8d3b9cd640 1 -- 172.21.15.152:0/1844847313 >> 172.21.15.152:0/1844847313 conn(0x7f8d34077230 msgr2=0x7f8d3410e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:54:56.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.676+0000 7f8d3b9cd640 1 -- 172.21.15.152:0/1844847313 shutdown_connections 2024-09-11T22:54:56.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:54:56.676+0000 7f8d3b9cd640 1 -- 172.21.15.152:0/1844847313 wait complete. 2024-09-11T22:54:56.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:56 smithi152 ceph-mon[31181]: pgmap v64: 97 pgs: 97 active+clean; 2.7 GiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 173 MiB/s wr, 150 op/s 2024-09-11T22:54:56.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:56 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1844847313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:54:57.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:56 smithi191 ceph-mon[35211]: pgmap v64: 97 pgs: 97 active+clean; 2.7 GiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 173 MiB/s wr, 150 op/s 2024-09-11T22:54:57.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:56 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1844847313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:54:57.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:56 smithi204 ceph-mon[35454]: pgmap v64: 97 pgs: 97 active+clean; 2.7 GiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 173 MiB/s wr, 150 op/s 2024-09-11T22:54:57.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:56 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1844847313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:54:59.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:54:58 smithi191 ceph-mon[35211]: pgmap v65: 97 pgs: 97 active+clean; 2.7 GiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 147 MiB/s wr, 138 op/s 2024-09-11T22:54:59.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:54:58 smithi204 ceph-mon[35454]: pgmap v65: 97 pgs: 97 active+clean; 2.7 GiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 147 MiB/s wr, 138 op/s 2024-09-11T22:54:59.059 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:54:58 smithi152 ceph-mon[31181]: pgmap v65: 97 pgs: 97 active+clean; 2.7 GiB data, 846 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 147 MiB/s wr, 138 op/s 2024-09-11T22:54:59.163 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24455 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-11T22:54:59.163 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24455 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-11T22:55:00.078 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:55:00.806 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:00 smithi152 ceph-mon[31181]: pgmap v66: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 507 MiB/s wr, 348 op/s 2024-09-11T22:55:00.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.828+0000 7f4b3614a640 1 -- 172.21.15.152:0/3251278766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30100e20 msgr2=0x7f4b30101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:00.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.828+0000 7f4b3614a640 1 --2- 172.21.15.152:0/3251278766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30100e20 0x7f4b30101200 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009900 tx=0x7f4b2002a4a0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.828+0000 7f4b3614a640 1 -- 172.21.15.152:0/3251278766 shutdown_connections 2024-09-11T22:55:00.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.828+0000 7f4b3614a640 1 --2- 172.21.15.152:0/3251278766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b30102ef0 0x7f4b3010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.828+0000 7f4b3614a640 1 --2- 172.21.15.152:0/3251278766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b30102570 0x7f4b301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.828+0000 7f4b3614a640 1 --2- 172.21.15.152:0/3251278766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30100e20 0x7f4b30101200 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.828+0000 7f4b3614a640 1 -- 172.21.15.152:0/3251278766 >> 172.21.15.152:0/3251278766 conn(0x7f4b300fc7d0 msgr2=0x7f4b300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:00.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.829+0000 7f4b3614a640 1 -- 172.21.15.152:0/3251278766 shutdown_connections 2024-09-11T22:55:00.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.829+0000 7f4b3614a640 1 -- 172.21.15.152:0/3251278766 wait complete. 2024-09-11T22:55:00.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.829+0000 7f4b3614a640 1 Processor -- start 2024-09-11T22:55:00.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.829+0000 7f4b3614a640 1 -- start start 2024-09-11T22:55:00.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.830+0000 7f4b3614a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30100e20 0x7f4b30198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.830+0000 7f4b3614a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b30102570 0x7f4b30198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.830+0000 7f4b3614a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b30102ef0 0x7f4b3019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.830+0000 7f4b3614a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30111040 con 0x7f4b30100e20 2024-09-11T22:55:00.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.830+0000 7f4b3614a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30110ec0 con 0x7f4b30102ef0 2024-09-11T22:55:00.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.830+0000 7f4b3614a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301111c0 con 0x7f4b30102570 2024-09-11T22:55:00.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.830+0000 7f4b2f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30100e20 0x7f4b30198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:00.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.830+0000 7f4b2f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30100e20 0x7f4b30198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46176/0 (socket says 172.21.15.152:46176) 2024-09-11T22:55:00.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.830+0000 7f4b2f7fe640 1 -- 172.21.15.152:0/3091549431 learned_addr learned my addr 172.21.15.152:0/3091549431 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:00.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.831+0000 7f4b2f7fe640 1 -- 172.21.15.152:0/3091549431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b30102570 msgr2=0x7f4b30198b80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:55:00.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.831+0000 7f4b2f7fe640 1 --2- 172.21.15.152:0/3091549431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b30102570 0x7f4b30198b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.831+0000 7f4b2f7fe640 1 -- 172.21.15.152:0/3091549431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b30102ef0 msgr2=0x7f4b3019cf10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:55:00.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.831+0000 7f4b2f7fe640 1 --2- 172.21.15.152:0/3091549431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b30102ef0 0x7f4b3019cf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.831+0000 7f4b2f7fe640 1 -- 172.21.15.152:0/3091549431 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b3019d5f0 con 0x7f4b30100e20 2024-09-11T22:55:00.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.831+0000 7f4b2f7fe640 1 --2- 172.21.15.152:0/3091549431 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30100e20 0x7f4b30198640 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4b20006010 tx=0x7f4b20003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:00.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.831+0000 7f4b2d7fa640 1 -- 172.21.15.152:0/3091549431 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2003f070 con 0x7f4b30100e20 2024-09-11T22:55:00.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.831+0000 7f4b2d7fa640 1 -- 172.21.15.152:0/3091549431 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b20031e30 con 0x7f4b30100e20 2024-09-11T22:55:00.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.831+0000 7f4b3614a640 1 -- 172.21.15.152:0/3091549431 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b3019d880 con 0x7f4b30100e20 2024-09-11T22:55:00.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.831+0000 7f4b2d7fa640 1 -- 172.21.15.152:0/3091549431 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20035030 con 0x7f4b30100e20 2024-09-11T22:55:00.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.831+0000 7f4b3614a640 1 -- 172.21.15.152:0/3091549431 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b301a51c0 con 0x7f4b30100e20 2024-09-11T22:55:00.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.834+0000 7f4b2d7fa640 1 -- 172.21.15.152:0/3091549431 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4b20042070 con 0x7f4b30100e20 2024-09-11T22:55:00.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.835+0000 7f4b2d7fa640 1 --2- 172.21.15.152:0/3091549431 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4b1c0a2d70 0x7f4b1c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.835+0000 7f4b38523740 1 Processor -- start 2024-09-11T22:55:00.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b38523740 1 -- start start 2024-09-11T22:55:00.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b2d7fa640 1 -- 172.21.15.152:0/3091549431 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4b200efe20 con 0x7f4b30100e20 2024-09-11T22:55:00.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b27fff640 1 --2- 172.21.15.152:0/3091549431 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4b1c0a2d70 0x7f4b1c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:00.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b38523740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b660760 0x557f6b660b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b38523740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f6b661080 0x557f6b66b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b38523740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f6b66b940 0x557f6b66dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b38523740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557f6b5c62a0 con 0x557f6b660760 2024-09-11T22:55:00.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b38523740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557f6b56db10 con 0x557f6b661080 2024-09-11T22:55:00.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b38523740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557f6b5c2260 con 0x557f6b66b940 2024-09-11T22:55:00.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b2f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b660760 0x557f6b660b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:00.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b2f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b660760 0x557f6b660b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46184/0 (socket says 172.21.15.152:46184) 2024-09-11T22:55:00.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b2f7fe640 1 -- 172.21.15.152:0/3797254351 learned_addr learned my addr 172.21.15.152:0/3797254351 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:00.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b27fff640 1 --2- 172.21.15.152:0/3091549431 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4b1c0a2d70 0x7f4b1c0a5230 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4b300fffa0 tx=0x7f4b18006c20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:00.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b2f7fe640 1 -- 172.21.15.152:0/3797254351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f6b66b940 msgr2=0x557f6b66dd30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:00.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b2f7fe640 1 --2- 172.21.15.152:0/3797254351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f6b66b940 0x557f6b66dd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b2f7fe640 1 -- 172.21.15.152:0/3797254351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f6b661080 msgr2=0x557f6b66b400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:00.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b2f7fe640 1 --2- 172.21.15.152:0/3797254351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f6b661080 0x557f6b66b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.836+0000 7f4b2f7fe640 1 -- 172.21.15.152:0/3797254351 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f6b337b90 con 0x557f6b660760 2024-09-11T22:55:00.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.837+0000 7f4b2f7fe640 1 --2- 172.21.15.152:0/3797254351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b660760 0x557f6b660b40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4b200dafd0 tx=0x7f4b200058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7f5b0d1e6bac0b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:00.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.837+0000 7f4b25ffb640 1 -- 172.21.15.152:0/3797254351 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b200d5070 con 0x557f6b660760 2024-09-11T22:55:00.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.837+0000 7f4b25ffb640 1 -- 172.21.15.152:0/3797254351 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b200bc390 con 0x557f6b660760 2024-09-11T22:55:00.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.837+0000 7f4b25ffb640 1 -- 172.21.15.152:0/3797254351 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20049040 con 0x557f6b660760 2024-09-11T22:55:00.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.838+0000 7f4b38523740 1 -- 172.21.15.152:0/3797254351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b660760 msgr2=0x557f6b660b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:00.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.838+0000 7f4b38523740 1 --2- 172.21.15.152:0/3797254351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b660760 0x557f6b660b40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4b200dafd0 tx=0x7f4b200058b0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.838+0000 7f4b38523740 1 -- 172.21.15.152:0/3797254351 shutdown_connections 2024-09-11T22:55:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.838+0000 7f4b38523740 1 --2- 172.21.15.152:0/3797254351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f6b66b940 0x557f6b66dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.838+0000 7f4b38523740 1 --2- 172.21.15.152:0/3797254351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f6b661080 0x557f6b66b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.838+0000 7f4b38523740 1 --2- 172.21.15.152:0/3797254351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b660760 0x557f6b660b40 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.838+0000 7f4b38523740 1 -- 172.21.15.152:0/3797254351 >> 172.21.15.152:0/3797254351 conn(0x557f6b65ccf0 msgr2=0x557f6b65d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.838+0000 7f4b38523740 1 -- 172.21.15.152:0/3797254351 shutdown_connections 2024-09-11T22:55:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.838+0000 7f4b38523740 1 -- 172.21.15.152:0/3797254351 wait complete. 2024-09-11T22:55:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.838+0000 7f4b38523740 1 -- start start 2024-09-11T22:55:00.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.839+0000 7f4b38523740 1 Processor -- start 2024-09-11T22:55:00.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.839+0000 7f4b38523740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f6b660760 0x557f6b668870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.839+0000 7f4b38523740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b661080 0x557f6b668db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.839+0000 7f4b38523740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f6b66b940 0x557f6b6692f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.839+0000 7f4b38523740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557f6b66e2d0 con 0x557f6b661080 2024-09-11T22:55:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.839+0000 7f4b38523740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557f6b54e6e0 con 0x557f6b660760 2024-09-11T22:55:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.839+0000 7f4b38523740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557f6b66e450 con 0x557f6b66b940 2024-09-11T22:55:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.839+0000 7f4b38523740 10 client.0 _ll_register_callbacks cb 0x557f6b572a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.839+0000 7f4b27fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b661080 0x557f6b668db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.839+0000 7f4b27fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b661080 0x557f6b668db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46188/0 (socket says 172.21.15.152:46188) 2024-09-11T22:55:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.839+0000 7f4b27fff640 1 -- 172.21.15.152:0/690256018 learned_addr learned my addr 172.21.15.152:0/690256018 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.840+0000 7f4b27fff640 1 -- 172.21.15.152:0/690256018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f6b66b940 msgr2=0x557f6b6692f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:00.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.840+0000 7f4b27fff640 1 --2- 172.21.15.152:0/690256018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f6b66b940 0x557f6b6692f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.840+0000 7f4b27fff640 1 -- 172.21.15.152:0/690256018 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f6b660760 msgr2=0x557f6b668870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:00.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.840+0000 7f4b27fff640 1 --2- 172.21.15.152:0/690256018 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f6b660760 0x557f6b668870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.840+0000 7f4b27fff640 1 -- 172.21.15.152:0/690256018 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f6b677780 con 0x557f6b661080 2024-09-11T22:55:00.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.840+0000 7f4b27fff640 1 --2- 172.21.15.152:0/690256018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b661080 0x557f6b668db0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800bff0 tx=0x7f4b180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7108e9643c2b1ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:00.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.840+0000 7f4b0effd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:00.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.840+0000 7f4b0effd640 1 -- 172.21.15.152:0/690256018 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18013e00 con 0x557f6b661080 2024-09-11T22:55:00.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.840+0000 7f4b0effd640 1 -- 172.21.15.152:0/690256018 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4b180125a0 con 0x557f6b661080 2024-09-11T22:55:00.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.840+0000 7f4b38523740 1 -- 172.21.15.152:0/690256018 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b200bc390 con 0x557f6b661080 2024-09-11T22:55:00.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.840+0000 7f4b0effd640 1 -- 172.21.15.152:0/690256018 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4b180127a0 con 0x557f6b661080 2024-09-11T22:55:00.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.840+0000 7f4b38523740 10 client.16077 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:00.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.840+0000 7f4b38523740 1 -- 172.21.15.152:0/690256018 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f6b67a1c0 con 0x557f6b661080 2024-09-11T22:55:00.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.841+0000 7f4b0effd640 1 -- 172.21.15.152:0/690256018 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b200bc390 con 0x557f6b661080 2024-09-11T22:55:00.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.841+0000 7f4b0effd640 1 -- 172.21.15.152:0/690256018 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4b18012070 con 0x557f6b661080 2024-09-11T22:55:00.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.841+0000 7f4b38523740 10 client.16077 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:00.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.841+0000 7f4b38523740 10 client.16077 resolve_mds: validated gid 24455 aka daemon mds.b 2024-09-11T22:55:00.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.841+0000 7f4b38523740 20 client.16077 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:00.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.841+0000 7f4b38523740 1 --2- 172.21.15.152:0/690256018 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557f6b67b670 0x557f6b67dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.841+0000 7f4b38523740 4 client.16077 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:55:00.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.841+0000 7f4b38523740 1 -- 172.21.15.152:0/690256018 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f6b5d39d0 con 0x557f6b67b670 2024-09-11T22:55:00.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.844+0000 7f4b2f7fe640 1 --2- 172.21.15.152:0/690256018 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557f6b67b670 0x557f6b67dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:00.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.847+0000 7f4b2f7fe640 1 --2- 172.21.15.152:0/690256018 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557f6b67b670 0x557f6b67dad0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90389d74a8f280a7 server_cookie=bc0663e91efc2c0e in_seq=0 out_seq=0 2024-09-11T22:55:00.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.847+0000 7f4b0effd640 10 client.16077 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:00.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.850+0000 7f4b0effd640 1 -- 172.21.15.152:0/690256018 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f6b5d39d0 con 0x557f6b67b670 2024-09-11T22:55:00.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.850+0000 7f4b0effd640 10 client.16077 handle_command_reply: tid=0 2024-09-11T22:55:00.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.850+0000 7f4b38523740 1 client.16077 shutdown 2024-09-11T22:55:00.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.851+0000 7f4b38523740 1 -- 172.21.15.152:0/690256018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b661080 msgr2=0x557f6b668db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:00.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.851+0000 7f4b38523740 1 --2- 172.21.15.152:0/690256018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b661080 0x557f6b668db0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800bff0 tx=0x7f4b180046c0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.851+0000 7f4b38523740 1 -- 172.21.15.152:0/690256018 shutdown_connections 2024-09-11T22:55:00.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.851+0000 7f4b38523740 1 --2- 172.21.15.152:0/690256018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f6b66b940 0x557f6b6692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.851+0000 7f4b38523740 1 --2- 172.21.15.152:0/690256018 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b661080 0x557f6b668db0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.851+0000 7f4b38523740 1 --2- 172.21.15.152:0/690256018 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557f6b67b670 0x557f6b67dad0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.851+0000 7f4b38523740 1 --2- 172.21.15.152:0/690256018 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f6b660760 0x557f6b668870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.851+0000 7f4b38523740 1 -- 172.21.15.152:0/690256018 >> 172.21.15.152:0/690256018 conn(0x557f6b65ccf0 msgr2=0x557f6b662a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:00.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.851+0000 7f4b0effd640 0 client.16077 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:00.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.851+0000 7f4b38523740 1 -- 172.21.15.152:0/690256018 shutdown_connections 2024-09-11T22:55:00.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.851+0000 7f4b38523740 1 -- 172.21.15.152:0/690256018 wait complete. 2024-09-11T22:55:00.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.851+0000 7f4b38523740 20 client.16077 trim_cache size 0 max 16384 2024-09-11T22:55:00.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.875+0000 7f4b38523740 1 Processor -- start 2024-09-11T22:55:00.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.875+0000 7f4b38523740 1 -- start start 2024-09-11T22:55:00.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.876+0000 7f4b38523740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f6b660760 0x557f6b663210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.876+0000 7f4b38523740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f6b66b940 0x557f6b668f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.876+0000 7f4b38523740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b669480 0x557f6b6833a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.876+0000 7f4b38523740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301afac0 con 0x557f6b669480 2024-09-11T22:55:00.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.876+0000 7f4b38523740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301b1600 con 0x557f6b66b940 2024-09-11T22:55:00.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.876+0000 7f4b38523740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301b0190 con 0x557f6b660760 2024-09-11T22:55:00.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.876+0000 7f4b2ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b669480 0x557f6b6833a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:00.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.876+0000 7f4b2ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b669480 0x557f6b6833a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46202/0 (socket says 172.21.15.152:46202) 2024-09-11T22:55:00.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.876+0000 7f4b2ffff640 1 -- 172.21.15.152:0/3732347737 learned_addr learned my addr 172.21.15.152:0/3732347737 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:00.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.876+0000 7f4b2ffff640 1 -- 172.21.15.152:0/3732347737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f6b660760 msgr2=0x557f6b663210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:00.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.876+0000 7f4b2ffff640 1 --2- 172.21.15.152:0/3732347737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f6b660760 0x557f6b663210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.876+0000 7f4b2ffff640 1 -- 172.21.15.152:0/3732347737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f6b66b940 msgr2=0x557f6b668f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:00.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.876+0000 7f4b2ffff640 1 --2- 172.21.15.152:0/3732347737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f6b66b940 0x557f6b668f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.876+0000 7f4b2ffff640 1 -- 172.21.15.152:0/3732347737 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f6b67e150 con 0x557f6b669480 2024-09-11T22:55:00.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.877+0000 7f4b2ffff640 1 --2- 172.21.15.152:0/3732347737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b669480 0x557f6b6833a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4b28009c20 tx=0x7f4b2800b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefac509a754360b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:00.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.877+0000 7f4b257fa640 1 -- 172.21.15.152:0/3732347737 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28009e30 con 0x557f6b669480 2024-09-11T22:55:00.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.877+0000 7f4b257fa640 1 -- 172.21.15.152:0/3732347737 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b28017070 con 0x557f6b669480 2024-09-11T22:55:00.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.877+0000 7f4b257fa640 1 -- 172.21.15.152:0/3732347737 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b280115f0 con 0x557f6b669480 2024-09-11T22:55:00.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.878+0000 7f4b38523740 1 -- 172.21.15.152:0/3732347737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b669480 msgr2=0x557f6b6833a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:00.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.878+0000 7f4b38523740 1 --2- 172.21.15.152:0/3732347737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b669480 0x557f6b6833a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4b28009c20 tx=0x7f4b2800b420 comp rx=0 tx=0).stop 2024-09-11T22:55:00.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.878+0000 7f4b38523740 1 -- 172.21.15.152:0/3732347737 shutdown_connections 2024-09-11T22:55:00.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.878+0000 7f4b38523740 1 --2- 172.21.15.152:0/3732347737 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b669480 0x557f6b6833a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.878+0000 7f4b38523740 1 --2- 172.21.15.152:0/3732347737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f6b66b940 0x557f6b668f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.878+0000 7f4b38523740 1 --2- 172.21.15.152:0/3732347737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f6b660760 0x557f6b663210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.878+0000 7f4b38523740 1 -- 172.21.15.152:0/3732347737 >> 172.21.15.152:0/3732347737 conn(0x557f6b65ccf0 msgr2=0x557f6b668830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:00.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.878+0000 7f4b38523740 1 -- 172.21.15.152:0/3732347737 shutdown_connections 2024-09-11T22:55:00.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.878+0000 7f4b38523740 1 -- 172.21.15.152:0/3732347737 wait complete. 2024-09-11T22:55:00.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.878+0000 7f4b38523740 1 -- start start 2024-09-11T22:55:00.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.879+0000 7f4b38523740 1 Processor -- start 2024-09-11T22:55:00.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.879+0000 7f4b38523740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f6b660760 0x557f6b673d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.879+0000 7f4b38523740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b669480 0x557f6b6700b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.879+0000 7f4b38523740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f6b66b940 0x557f6b670610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.879+0000 7f4b38523740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301b0f30 con 0x557f6b669480 2024-09-11T22:55:00.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.879+0000 7f4b38523740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301b0860 con 0x557f6b66b940 2024-09-11T22:55:00.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.879+0000 7f4b38523740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301ada80 con 0x557f6b660760 2024-09-11T22:55:00.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.879+0000 7f4b38523740 10 client.0 _ll_register_callbacks cb 0x7f4b1800b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:00.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.879+0000 7f4b27fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b669480 0x557f6b6700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:00.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.879+0000 7f4b27fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b669480 0x557f6b6700b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46206/0 (socket says 172.21.15.152:46206) 2024-09-11T22:55:00.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.879+0000 7f4b27fff640 1 -- 172.21.15.152:0/2298990525 learned_addr learned my addr 172.21.15.152:0/2298990525 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:00.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.880+0000 7f4b27fff640 1 -- 172.21.15.152:0/2298990525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f6b660760 msgr2=0x557f6b673d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:00.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.880+0000 7f4b27fff640 1 --2- 172.21.15.152:0/2298990525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f6b660760 0x557f6b673d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.880+0000 7f4b27fff640 1 -- 172.21.15.152:0/2298990525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f6b66b940 msgr2=0x557f6b670610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:00.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.880+0000 7f4b27fff640 1 --2- 172.21.15.152:0/2298990525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f6b66b940 0x557f6b670610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.880+0000 7f4b27fff640 1 -- 172.21.15.152:0/2298990525 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f6b670e40 con 0x557f6b669480 2024-09-11T22:55:00.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.880+0000 7f4b0effd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:00.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.880+0000 7f4b27fff640 1 --2- 172.21.15.152:0/2298990525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b669480 0x557f6b6700b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4b1801dc20 tx=0x7f4b18012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549cf74347e86167 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:00.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.880+0000 7f4b0effd640 1 -- 172.21.15.152:0/2298990525 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18035060 con 0x557f6b669480 2024-09-11T22:55:00.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.880+0000 7f4b0effd640 1 -- 172.21.15.152:0/2298990525 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4b180125a0 con 0x557f6b669480 2024-09-11T22:55:00.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.880+0000 7f4b38523740 1 -- 172.21.15.152:0/2298990525 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b28017070 con 0x557f6b669480 2024-09-11T22:55:00.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.881+0000 7f4b0effd640 1 -- 172.21.15.152:0/2298990525 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4b18012df0 con 0x557f6b669480 2024-09-11T22:55:00.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.881+0000 7f4b38523740 10 client.16083 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:00.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.881+0000 7f4b38523740 1 -- 172.21.15.152:0/2298990525 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f6b6efc00 con 0x557f6b669480 2024-09-11T22:55:00.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.881+0000 7f4b0effd640 1 -- 172.21.15.152:0/2298990525 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b28017070 con 0x557f6b669480 2024-09-11T22:55:00.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.881+0000 7f4b0effd640 1 -- 172.21.15.152:0/2298990525 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4b180259d0 con 0x557f6b669480 2024-09-11T22:55:00.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.881+0000 7f4b38523740 10 client.16083 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:00.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.881+0000 7f4b38523740 10 client.16083 resolve_mds: validated gid 24455 aka daemon mds.b 2024-09-11T22:55:00.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.881+0000 7f4b38523740 20 client.16083 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:00.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.881+0000 7f4b38523740 1 --2- 172.21.15.152:0/2298990525 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557f6b6f0360 0x557f6b6f2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:00.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.881+0000 7f4b38523740 4 client.16083 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-11T22:55:00.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.881+0000 7f4b38523740 1 -- 172.21.15.152:0/2298990525 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x557f6b5cdd20 con 0x557f6b6f0360 2024-09-11T22:55:00.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.885+0000 7f4b2f7fe640 1 --2- 172.21.15.152:0/2298990525 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557f6b6f0360 0x557f6b6f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:00.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.891+0000 7f4b0effd640 10 client.16083 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:00.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.891+0000 7f4b2f7fe640 1 --2- 172.21.15.152:0/2298990525 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557f6b6f0360 0x557f6b6f2780 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=478b848213242390 server_cookie=f96760477e2a3d21 in_seq=0 out_seq=0 2024-09-11T22:55:00.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.914+0000 7f4b0effd640 1 -- 172.21.15.152:0/2298990525 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x557f6b5cdd20 con 0x557f6b6f0360 2024-09-11T22:55:00.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.914+0000 7f4b0effd640 10 client.16083 handle_command_reply: tid=0 2024-09-11T22:55:00.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.914+0000 7f4b38523740 1 client.16083 shutdown 2024-09-11T22:55:00.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.914+0000 7f4b38523740 1 -- 172.21.15.152:0/2298990525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b669480 msgr2=0x557f6b6700b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:00.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.914+0000 7f4b38523740 1 --2- 172.21.15.152:0/2298990525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b669480 0x557f6b6700b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4b1801dc20 tx=0x7f4b18012b70 comp rx=0 tx=0).stop 2024-09-11T22:55:00.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.914+0000 7f4b38523740 1 -- 172.21.15.152:0/2298990525 shutdown_connections 2024-09-11T22:55:00.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.914+0000 7f4b38523740 1 --2- 172.21.15.152:0/2298990525 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557f6b6f0360 0x557f6b6f2780 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.914+0000 7f4b38523740 1 --2- 172.21.15.152:0/2298990525 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557f6b66b940 0x557f6b670610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.914+0000 7f4b38523740 1 --2- 172.21.15.152:0/2298990525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557f6b669480 0x557f6b6700b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.914+0000 7f4b0effd640 0 client.16083 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:00.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.914+0000 7f4b38523740 1 --2- 172.21.15.152:0/2298990525 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557f6b660760 0x557f6b673d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.914+0000 7f4b38523740 1 -- 172.21.15.152:0/2298990525 >> 172.21.15.152:0/2298990525 conn(0x557f6b65ccf0 msgr2=0x557f6b65b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:00.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.915+0000 7f4b38523740 1 -- 172.21.15.152:0/2298990525 shutdown_connections 2024-09-11T22:55:00.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.915+0000 7f4b38523740 1 -- 172.21.15.152:0/2298990525 wait complete. 2024-09-11T22:55:00.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.915+0000 7f4b38523740 20 client.16083 trim_cache size 0 max 16384 2024-09-11T22:55:00.918 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:55:00.918 INFO:teuthology.orchestra.run.smithi152.stdout: "epoch": 28, 2024-09-11T22:55:00.918 INFO:teuthology.orchestra.run.smithi152.stdout: "leader": 24455, 2024-09-11T22:55:00.918 INFO:teuthology.orchestra.run.smithi152.stdout: "set_version": 6, 2024-09-11T22:55:00.919 INFO:teuthology.orchestra.run.smithi152.stdout: "sets": { 2024-09-11T22:55:00.919 INFO:teuthology.orchestra.run.smithi152.stdout: "df9f6e3e": { 2024-09-11T22:55:00.919 INFO:teuthology.orchestra.run.smithi152.stdout: "version": 6, 2024-09-11T22:55:00.919 INFO:teuthology.orchestra.run.smithi152.stdout: "age_ref": 0.0, 2024-09-11T22:55:00.919 INFO:teuthology.orchestra.run.smithi152.stdout: "state": { 2024-09-11T22:55:00.919 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "QUIESCED", 2024-09-11T22:55:00.919 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.0 2024-09-11T22:55:00.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T22:55:00.919 INFO:teuthology.orchestra.run.smithi152.stdout: "timeout": 90.0, 2024-09-11T22:55:00.919 INFO:teuthology.orchestra.run.smithi152.stdout: "expiration": 133.4, 2024-09-11T22:55:00.919 INFO:teuthology.orchestra.run.smithi152.stdout: "members": { 2024-09-11T22:55:00.919 INFO:teuthology.orchestra.run.smithi152.stdout: "file:/": { 2024-09-11T22:55:00.919 INFO:teuthology.orchestra.run.smithi152.stdout: "excluded": false, 2024-09-11T22:55:00.919 INFO:teuthology.orchestra.run.smithi152.stdout: "state": { 2024-09-11T22:55:00.920 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "QUIESCED", 2024-09-11T22:55:00.920 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.0 2024-09-11T22:55:00.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:00.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:00.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:00.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:00.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:00.920 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:55:00.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.916+0000 7f4b3614a640 1 -- 172.21.15.152:0/3091549431 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4b1c0a2d70 msgr2=0x7f4b1c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:00.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.916+0000 7f4b3614a640 1 --2- 172.21.15.152:0/3091549431 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4b1c0a2d70 0x7f4b1c0a5230 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4b300fffa0 tx=0x7f4b18006c20 comp rx=0 tx=0).stop 2024-09-11T22:55:00.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.916+0000 7f4b3614a640 1 -- 172.21.15.152:0/3091549431 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30100e20 msgr2=0x7f4b30198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:00.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.916+0000 7f4b3614a640 1 --2- 172.21.15.152:0/3091549431 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30100e20 0x7f4b30198640 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4b20006010 tx=0x7f4b20003cd0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.916+0000 7f4b3614a640 1 -- 172.21.15.152:0/3091549431 shutdown_connections 2024-09-11T22:55:00.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.916+0000 7f4b3614a640 1 --2- 172.21.15.152:0/3091549431 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4b1c0a2d70 0x7f4b1c0a5230 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.916+0000 7f4b3614a640 1 --2- 172.21.15.152:0/3091549431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b30102ef0 0x7f4b3019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.916+0000 7f4b3614a640 1 --2- 172.21.15.152:0/3091549431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b30102570 0x7f4b30198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.916+0000 7f4b3614a640 1 --2- 172.21.15.152:0/3091549431 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b30100e20 0x7f4b30198640 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:00.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.916+0000 7f4b3614a640 1 -- 172.21.15.152:0/3091549431 >> 172.21.15.152:0/3091549431 conn(0x7f4b300fc7d0 msgr2=0x7f4b3006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:00.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.917+0000 7f4b3614a640 1 -- 172.21.15.152:0/3091549431 shutdown_connections 2024-09-11T22:55:00.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:00.917+0000 7f4b3614a640 1 -- 172.21.15.152:0/3091549431 wait complete. 2024-09-11T22:55:01.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:00 smithi191 ceph-mon[35211]: pgmap v66: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 507 MiB/s wr, 348 op/s 2024-09-11T22:55:01.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:00 smithi204 ceph-mon[35454]: pgmap v66: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 507 MiB/s wr, 348 op/s 2024-09-11T22:55:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.761+0000 7fa3c9653640 1 -- 172.21.15.152:0/1800351234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3c41069c0 msgr2=0x7fa3c4106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.761+0000 7fa3c9653640 1 --2- 172.21.15.152:0/1800351234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3c41069c0 0x7fa3c4106da0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009930 tx=0x7fa3b402a440 comp rx=0 tx=0).stop 2024-09-11T22:55:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.761+0000 7fa3c9653640 1 -- 172.21.15.152:0/1800351234 shutdown_connections 2024-09-11T22:55:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.761+0000 7fa3c9653640 1 --2- 172.21.15.152:0/1800351234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c4101350 0x7fa3c410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.761+0000 7fa3c9653640 1 --2- 172.21.15.152:0/1800351234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3c41009b0 0x7fa3c4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.761+0000 7fa3c9653640 1 --2- 172.21.15.152:0/1800351234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3c41069c0 0x7fa3c4106da0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.761+0000 7fa3c9653640 1 -- 172.21.15.152:0/1800351234 >> 172.21.15.152:0/1800351234 conn(0x7fa3c4077990 msgr2=0x7fa3c40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.761+0000 7fa3c9653640 1 -- 172.21.15.152:0/1800351234 shutdown_connections 2024-09-11T22:55:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.761+0000 7fa3c9653640 1 -- 172.21.15.152:0/1800351234 wait complete. 2024-09-11T22:55:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.762+0000 7fa3c9653640 1 Processor -- start 2024-09-11T22:55:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.762+0000 7fa3c9653640 1 -- start start 2024-09-11T22:55:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.762+0000 7fa3c9653640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c41009b0 0x7fa3c4198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.762+0000 7fa3c9653640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3c4101350 0x7fa3c41987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.762+0000 7fa3c9653640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3c41069c0 0x7fa3c419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.762+0000 7fa3c9653640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c4110dc0 con 0x7fa3c4101350 2024-09-11T22:55:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.762+0000 7fa3c9653640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c4110c40 con 0x7fa3c41069c0 2024-09-11T22:55:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.762+0000 7fa3c9653640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c4110f40 con 0x7fa3c41009b0 2024-09-11T22:55:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.763+0000 7fa3c27fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3c4101350 0x7fa3c41987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.763+0000 7fa3c27fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3c4101350 0x7fa3c41987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46232/0 (socket says 172.21.15.152:46232) 2024-09-11T22:55:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.763+0000 7fa3c27fc640 1 -- 172.21.15.152:0/2070130628 learned_addr learned my addr 172.21.15.152:0/2070130628 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:01.766 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.763+0000 7fa3c37fe640 1 --2- 172.21.15.152:0/2070130628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3c41069c0 0x7fa3c419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.763+0000 7fa3c27fc640 1 -- 172.21.15.152:0/2070130628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c41009b0 msgr2=0x7fa3c4198260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:55:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.763+0000 7fa3c27fc640 1 --2- 172.21.15.152:0/2070130628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c41009b0 0x7fa3c4198260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.763+0000 7fa3c27fc640 1 -- 172.21.15.152:0/2070130628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3c41069c0 msgr2=0x7fa3c419cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.763+0000 7fa3c27fc640 1 --2- 172.21.15.152:0/2070130628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3c41069c0 0x7fa3c419cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.763+0000 7fa3c27fc640 1 -- 172.21.15.152:0/2070130628 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c419d250 con 0x7fa3c4101350 2024-09-11T22:55:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.763+0000 7fa3c27fc640 1 --2- 172.21.15.152:0/2070130628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3c4101350 0x7fa3c41987a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800c920 tx=0x7fa3b800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.763+0000 7fa3abfff640 1 -- 172.21.15.152:0/2070130628 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8007c70 con 0x7fa3c4101350 2024-09-11T22:55:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.763+0000 7fa3c9653640 1 -- 172.21.15.152:0/2070130628 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c419d540 con 0x7fa3c4101350 2024-09-11T22:55:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.763+0000 7fa3abfff640 1 -- 172.21.15.152:0/2070130628 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b8007e10 con 0x7fa3c4101350 2024-09-11T22:55:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.764+0000 7fa3abfff640 1 -- 172.21.15.152:0/2070130628 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8018890 con 0x7fa3c4101350 2024-09-11T22:55:01.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.764+0000 7fa3c9653640 1 -- 172.21.15.152:0/2070130628 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c41a4dd0 con 0x7fa3c4101350 2024-09-11T22:55:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.767+0000 7fa3abfff640 1 -- 172.21.15.152:0/2070130628 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa3b8018a50 con 0x7fa3c4101350 2024-09-11T22:55:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.767+0000 7fa3cc3f2740 1 Processor -- start 2024-09-11T22:55:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.767+0000 7fa3cc3f2740 1 -- start start 2024-09-11T22:55:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.767+0000 7fa3cc3f2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f147950f00 0x55f1479512e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.767+0000 7fa3cc3f2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f147951820 0x55f14795bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.767+0000 7fa3cc3f2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f14795c0c0 0x55f14795e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.767+0000 7fa3cc3f2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f1478b12f0 con 0x55f14795c0c0 2024-09-11T22:55:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.767+0000 7fa3cc3f2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f147859270 con 0x55f147950f00 2024-09-11T22:55:01.770 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.767+0000 7fa3cc3f2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f1478ad2b0 con 0x55f147951820 2024-09-11T22:55:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.767+0000 7fa3c27fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f14795c0c0 0x55f14795e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.768+0000 7fa3c27fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f14795c0c0 0x55f14795e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46244/0 (socket says 172.21.15.152:46244) 2024-09-11T22:55:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.768+0000 7fa3c27fc640 1 -- 172.21.15.152:0/1427523691 learned_addr learned my addr 172.21.15.152:0/1427523691 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.768+0000 7fa3abfff640 1 --2- 172.21.15.152:0/2070130628 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa3ac0a2da0 0x7fa3ac0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.768+0000 7fa3c37fe640 1 --2- 172.21.15.152:0/2070130628 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa3ac0a2da0 0x7fa3ac0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.768+0000 7fa3abfff640 1 -- 172.21.15.152:0/2070130628 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa3b80cf620 con 0x7fa3c4101350 2024-09-11T22:55:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.768+0000 7fa3c27fc640 1 -- 172.21.15.152:0/1427523691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f147951820 msgr2=0x55f14795bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.768+0000 7fa3c27fc640 1 --2- 172.21.15.152:0/1427523691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f147951820 0x55f14795bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.768+0000 7fa3c27fc640 1 -- 172.21.15.152:0/1427523691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f147950f00 msgr2=0x55f1479512e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.768+0000 7fa3c27fc640 1 --2- 172.21.15.152:0/1427523691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f147950f00 0x55f1479512e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.768+0000 7fa3c27fc640 1 -- 172.21.15.152:0/1427523691 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f147858340 con 0x55f14795c0c0 2024-09-11T22:55:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.768+0000 7fa3c27fc640 1 --2- 172.21.15.152:0/1427523691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f14795c0c0 0x55f14795e4b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80ccd30 tx=0x7fa3b800ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6fa1baae07df43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.768+0000 7fa3c37fe640 1 --2- 172.21.15.152:0/2070130628 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa3ac0a2da0 0x7fa3ac0a5260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc007310 tx=0x7fa3bc0072a0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.769+0000 7fa3a97fa640 1 -- 172.21.15.152:0/1427523691 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b809bb90 con 0x55f14795c0c0 2024-09-11T22:55:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.769+0000 7fa3a97fa640 1 -- 172.21.15.152:0/1427523691 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b809ff40 con 0x55f14795c0c0 2024-09-11T22:55:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.769+0000 7fa3a97fa640 1 -- 172.21.15.152:0/1427523691 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80cd040 con 0x55f14795c0c0 2024-09-11T22:55:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.769+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/1427523691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f14795c0c0 msgr2=0x55f14795e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.769+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/1427523691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f14795c0c0 0x55f14795e4b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80ccd30 tx=0x7fa3b800ff30 comp rx=0 tx=0).stop 2024-09-11T22:55:01.773 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.770+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/1427523691 shutdown_connections 2024-09-11T22:55:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.770+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/1427523691 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f14795c0c0 0x55f14795e4b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.770+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/1427523691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f147951820 0x55f14795bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.770+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/1427523691 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f147950f00 0x55f1479512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.770+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/1427523691 >> 172.21.15.152:0/1427523691 conn(0x55f14794d470 msgr2=0x55f14794d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.770+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/1427523691 shutdown_connections 2024-09-11T22:55:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.770+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/1427523691 wait complete. 2024-09-11T22:55:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.770+0000 7fa3cc3f2740 1 -- start start 2024-09-11T22:55:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.770+0000 7fa3cc3f2740 1 Processor -- start 2024-09-11T22:55:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.771+0000 7fa3cc3f2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f147950f00 0x55f1479590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.771+0000 7fa3cc3f2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f147951820 0x55f147959600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.771+0000 7fa3cc3f2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f14795c0c0 0x55f147959b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.771+0000 7fa3cc3f2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f14795ec30 con 0x55f14795c0c0 2024-09-11T22:55:01.774 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.771+0000 7fa3cc3f2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f14783a290 con 0x55f147951820 2024-09-11T22:55:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.771+0000 7fa3cc3f2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f14795edb0 con 0x55f147950f00 2024-09-11T22:55:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.771+0000 7fa3cc3f2740 10 client.0 _ll_register_callbacks cb 0x55f14786b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.771+0000 7fa3c37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f14795c0c0 0x55f147959b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.771+0000 7fa3c37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f14795c0c0 0x55f147959b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46260/0 (socket says 172.21.15.152:46260) 2024-09-11T22:55:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.771+0000 7fa3c37fe640 1 -- 172.21.15.152:0/4081557994 learned_addr learned my addr 172.21.15.152:0/4081557994 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.772+0000 7fa3c37fe640 1 -- 172.21.15.152:0/4081557994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f147950f00 msgr2=0x55f1479590c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.772+0000 7fa3c37fe640 1 --2- 172.21.15.152:0/4081557994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f147950f00 0x55f1479590c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.772+0000 7fa3c37fe640 1 -- 172.21.15.152:0/4081557994 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f147951820 msgr2=0x55f147959600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.772+0000 7fa3c37fe640 1 --2- 172.21.15.152:0/4081557994 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f147951820 0x55f147959600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.772+0000 7fa3c37fe640 1 -- 172.21.15.152:0/4081557994 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1475c4210 con 0x55f14795c0c0 2024-09-11T22:55:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.772+0000 7fa3c37fe640 1 --2- 172.21.15.152:0/4081557994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f14795c0c0 0x55f147959b40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc008010 tx=0x7fa3bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229250bb5d9badd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.772+0000 7fa3a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.772+0000 7fa3a6ff5640 1 -- 172.21.15.152:0/4081557994 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc021070 con 0x55f14795c0c0 2024-09-11T22:55:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.772+0000 7fa3a6ff5640 1 -- 172.21.15.152:0/4081557994 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa3bc004d20 con 0x55f14795c0c0 2024-09-11T22:55:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.772+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/4081557994 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b809ff40 con 0x55f14795c0c0 2024-09-11T22:55:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.773+0000 7fa3a6ff5640 1 -- 172.21.15.152:0/4081557994 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa3bc02c050 con 0x55f14795c0c0 2024-09-11T22:55:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.773+0000 7fa3cc3f2740 10 client.16095 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.773+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/4081557994 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f147690620 con 0x55f14795c0c0 2024-09-11T22:55:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.773+0000 7fa3a6ff5640 1 -- 172.21.15.152:0/4081557994 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b809ff40 con 0x55f14795c0c0 2024-09-11T22:55:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.773+0000 7fa3a6ff5640 1 -- 172.21.15.152:0/4081557994 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa3bc005850 con 0x55f14795c0c0 2024-09-11T22:55:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.773+0000 7fa3cc3f2740 10 client.16095 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.773+0000 7fa3cc3f2740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.773+0000 7fa3cc3f2740 20 client.16095 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.774+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/4081557994 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f14796c470 0x55f14796e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.774+0000 7fa3cc3f2740 4 client.16095 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:55:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.774+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/4081557994 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1478bea20 con 0x55f14796c470 2024-09-11T22:55:01.778 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.776+0000 7fa3c2ffd640 1 --2- 172.21.15.152:0/4081557994 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f14796c470 0x55f14796e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.780+0000 7fa3a6ff5640 10 client.16095 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:01.782 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.780+0000 7fa3c2ffd640 1 --2- 172.21.15.152:0/4081557994 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f14796c470 0x55f14796e860 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e858a340854ecc4 server_cookie=d0d2c80f3950e806 in_seq=0 out_seq=0 2024-09-11T22:55:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.783+0000 7fa3a6ff5640 1 -- 172.21.15.152:0/4081557994 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1478bea20 con 0x55f14796c470 2024-09-11T22:55:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.783+0000 7fa3a6ff5640 10 client.16095 handle_command_reply: tid=0 2024-09-11T22:55:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.783+0000 7fa3cc3f2740 1 client.16095 shutdown 2024-09-11T22:55:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.784+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/4081557994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f14795c0c0 msgr2=0x55f147959b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.784+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/4081557994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f14795c0c0 0x55f147959b40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc008010 tx=0x7fa3bc004a40 comp rx=0 tx=0).stop 2024-09-11T22:55:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.784+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/4081557994 shutdown_connections 2024-09-11T22:55:01.786 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.784+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/4081557994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f14795c0c0 0x55f147959b40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.784+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/4081557994 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f14796c470 0x55f14796e860 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.784+0000 7fa3a6ff5640 0 client.16095 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.784+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/4081557994 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f147951820 0x55f147959600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.784+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/4081557994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f147950f00 0x55f1479590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.784+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/4081557994 >> 172.21.15.152:0/4081557994 conn(0x55f14794d470 msgr2=0x55f147953180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.784+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/4081557994 shutdown_connections 2024-09-11T22:55:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.784+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/4081557994 wait complete. 2024-09-11T22:55:01.787 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.785+0000 7fa3cc3f2740 20 client.16095 trim_cache size 0 max 16384 2024-09-11T22:55:01.795 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'df9f6e3e', quiesce took 7.8 seconds. Will release after: 5.6000000000000005 2024-09-11T22:55:01.805 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.804+0000 7fa3cc3f2740 1 Processor -- start 2024-09-11T22:55:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.804+0000 7fa3cc3f2740 1 -- start start 2024-09-11T22:55:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.804+0000 7fa3cc3f2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f147950f00 0x55f147973e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.804+0000 7fa3cc3f2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f147951820 0x55f1479743d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.806 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.804+0000 7fa3cc3f2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f14795c0c0 0x55f147956320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.804+0000 7fa3cc3f2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41ab510 con 0x55f147950f00 2024-09-11T22:55:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.804+0000 7fa3cc3f2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41ad050 con 0x55f147951820 2024-09-11T22:55:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.804+0000 7fa3cc3f2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41abbe0 con 0x55f14795c0c0 2024-09-11T22:55:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.804+0000 7fa3c2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f147950f00 0x55f147973e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.804+0000 7fa3c2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f147950f00 0x55f147973e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46276/0 (socket says 172.21.15.152:46276) 2024-09-11T22:55:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.804+0000 7fa3c2ffd640 1 -- 172.21.15.152:0/1675045123 learned_addr learned my addr 172.21.15.152:0/1675045123 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.805+0000 7fa3c2ffd640 1 -- 172.21.15.152:0/1675045123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f14795c0c0 msgr2=0x55f147956320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.805+0000 7fa3c2ffd640 1 --2- 172.21.15.152:0/1675045123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f14795c0c0 0x55f147956320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.805+0000 7fa3c2ffd640 1 -- 172.21.15.152:0/1675045123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f147951820 msgr2=0x55f1479743d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.805+0000 7fa3c2ffd640 1 --2- 172.21.15.152:0/1675045123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f147951820 0x55f1479743d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.805+0000 7fa3c2ffd640 1 -- 172.21.15.152:0/1675045123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f14794e100 con 0x55f147950f00 2024-09-11T22:55:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.805+0000 7fa3c2ffd640 1 --2- 172.21.15.152:0/1675045123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f147950f00 0x55f147973e90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009930 tx=0x7fa3b4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8916713a1e5b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.805+0000 7fa3a8ff9640 1 -- 172.21.15.152:0/1675045123 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4035040 con 0x55f147950f00 2024-09-11T22:55:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.805+0000 7fa3a8ff9640 1 -- 172.21.15.152:0/1675045123 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b4045030 con 0x55f147950f00 2024-09-11T22:55:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.805+0000 7fa3a8ff9640 1 -- 172.21.15.152:0/1675045123 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b403b430 con 0x55f147950f00 2024-09-11T22:55:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.806+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/1675045123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f147950f00 msgr2=0x55f147973e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.806+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/1675045123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f147950f00 0x55f147973e90 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009930 tx=0x7fa3b4004360 comp rx=0 tx=0).stop 2024-09-11T22:55:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.806+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/1675045123 shutdown_connections 2024-09-11T22:55:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.806+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/1675045123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f14795c0c0 0x55f147956320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.806+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/1675045123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f147951820 0x55f1479743d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.806+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/1675045123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f147950f00 0x55f147973e90 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.806+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/1675045123 >> 172.21.15.152:0/1675045123 conn(0x55f14794d470 msgr2=0x55f14795b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.806+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/1675045123 shutdown_connections 2024-09-11T22:55:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.806+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/1675045123 wait complete. 2024-09-11T22:55:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.806+0000 7fa3cc3f2740 1 -- start start 2024-09-11T22:55:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.806+0000 7fa3cc3f2740 1 Processor -- start 2024-09-11T22:55:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3cc3f2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f147950f00 0x55f147964070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3cc3f2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f147951820 0x55f1479645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.813 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3cc3f2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f14795c0c0 0x55f147961040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3cc3f2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41ac980 con 0x55f147950f00 2024-09-11T22:55:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3cc3f2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41ac2b0 con 0x55f14795c0c0 2024-09-11T22:55:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3cc3f2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41a94d0 con 0x55f147951820 2024-09-11T22:55:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3cc3f2740 10 client.0 _ll_register_callbacks cb 0x7fa3bc014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3c2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f147950f00 0x55f147964070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3c2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f147950f00 0x55f147964070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46278/0 (socket says 172.21.15.152:46278) 2024-09-11T22:55:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3c2ffd640 1 -- 172.21.15.152:0/2217845688 learned_addr learned my addr 172.21.15.152:0/2217845688 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3c2ffd640 1 -- 172.21.15.152:0/2217845688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f147951820 msgr2=0x55f1479645b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3c2ffd640 1 --2- 172.21.15.152:0/2217845688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f147951820 0x55f1479645b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3c2ffd640 1 -- 172.21.15.152:0/2217845688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f14795c0c0 msgr2=0x55f147961040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3c2ffd640 1 --2- 172.21.15.152:0/2217845688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f14795c0c0 0x55f147961040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3c2ffd640 1 -- 172.21.15.152:0/2217845688 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f147961580 con 0x55f147950f00 2024-09-11T22:55:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3c2ffd640 1 --2- 172.21.15.152:0/2217845688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f147950f00 0x55f147964070 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa3b402a960 tx=0x7fa3b4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa51638ce1a120f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.807+0000 7fa3a6ff5640 1 -- 172.21.15.152:0/2217845688 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4035040 con 0x55f147950f00 2024-09-11T22:55:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.808+0000 7fa3a6ff5640 1 -- 172.21.15.152:0/2217845688 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa3b4003c70 con 0x55f147950f00 2024-09-11T22:55:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.808+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/2217845688 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b4045030 con 0x55f147950f00 2024-09-11T22:55:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.808+0000 7fa3a6ff5640 1 -- 172.21.15.152:0/2217845688 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa3b4003e70 con 0x55f147950f00 2024-09-11T22:55:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.808+0000 7fa3cc3f2740 10 client.16101 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.808+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/2217845688 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1479d7560 con 0x55f147950f00 2024-09-11T22:55:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.808+0000 7fa3a6ff5640 1 -- 172.21.15.152:0/2217845688 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b4045030 con 0x55f147950f00 2024-09-11T22:55:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.808+0000 7fa3a6ff5640 1 -- 172.21.15.152:0/2217845688 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa3b40539b0 con 0x55f147950f00 2024-09-11T22:55:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.808+0000 7fa3cc3f2740 10 client.16101 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.808+0000 7fa3cc3f2740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.808+0000 7fa3cc3f2740 20 client.16101 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.808+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/2217845688 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f14796c470 0x55f1479d9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.808+0000 7fa3cc3f2740 4 client.16101 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:55:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.808+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/2217845688 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1478b8d70 con 0x55f14796c470 2024-09-11T22:55:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.809+0000 7fa3c27fc640 1 --2- 172.21.15.152:0/2217845688 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f14796c470 0x55f1479d9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.810+0000 7fa3c27fc640 1 --2- 172.21.15.152:0/2217845688 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f14796c470 0x55f1479d9be0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0249f7808333225 server_cookie=324fc8555e95252 in_seq=0 out_seq=0 2024-09-11T22:55:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:01.810+0000 7fa3a6ff5640 10 client.16101 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.185+0000 7fa3a6ff5640 1 -- 172.21.15.152:0/2217845688 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f1478b8d70 con 0x55f14796c470 2024-09-11T22:55:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.185+0000 7fa3a6ff5640 10 client.16101 handle_command_reply: tid=0 2024-09-11T22:55:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.185+0000 7fa3cc3f2740 1 client.16101 shutdown 2024-09-11T22:55:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.185+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/2217845688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f147950f00 msgr2=0x55f147964070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.185+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/2217845688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f147950f00 0x55f147964070 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa3b402a960 tx=0x7fa3b4004040 comp rx=0 tx=0).stop 2024-09-11T22:55:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.186+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/2217845688 shutdown_connections 2024-09-11T22:55:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.186+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/2217845688 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f14796c470 0x55f1479d9be0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.186+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/2217845688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f14795c0c0 0x55f147961040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.186+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/2217845688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f147951820 0x55f1479645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:02.188 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.186+0000 7fa3cc3f2740 1 --2- 172.21.15.152:0/2217845688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f147950f00 0x55f147964070 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.186+0000 7fa3a6ff5640 0 client.16101 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.186+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/2217845688 >> 172.21.15.152:0/2217845688 conn(0x55f14794d470 msgr2=0x55f14795b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.186+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/2217845688 shutdown_connections 2024-09-11T22:55:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.186+0000 7fa3cc3f2740 1 -- 172.21.15.152:0/2217845688 wait complete. 2024-09-11T22:55:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.186+0000 7fa3cc3f2740 20 client.16101 trim_cache size 0 max 16384 2024-09-11T22:55:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:{ 2024-09-11T22:55:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "message": "", 2024-09-11T22:55:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "return_code": 0 2024-09-11T22:55:02.189 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:} 2024-09-11T22:55:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.187+0000 7fa3c9653640 1 -- 172.21.15.152:0/2070130628 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa3ac0a2da0 msgr2=0x7fa3ac0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:02.190 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.187+0000 7fa3c9653640 1 --2- 172.21.15.152:0/2070130628 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa3ac0a2da0 0x7fa3ac0a5260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc007310 tx=0x7fa3bc0072a0 comp rx=0 tx=0).stop 2024-09-11T22:55:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.187+0000 7fa3c9653640 1 -- 172.21.15.152:0/2070130628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3c4101350 msgr2=0x7fa3c41987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.187+0000 7fa3c9653640 1 --2- 172.21.15.152:0/2070130628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3c4101350 0x7fa3c41987a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800c920 tx=0x7fa3b800cde0 comp rx=0 tx=0).stop 2024-09-11T22:55:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.187+0000 7fa3c9653640 1 -- 172.21.15.152:0/2070130628 shutdown_connections 2024-09-11T22:55:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.187+0000 7fa3c9653640 1 --2- 172.21.15.152:0/2070130628 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa3ac0a2da0 0x7fa3ac0a5260 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.187+0000 7fa3c9653640 1 --2- 172.21.15.152:0/2070130628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa3c41069c0 0x7fa3c419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.187+0000 7fa3c9653640 1 --2- 172.21.15.152:0/2070130628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3c4101350 0x7fa3c41987a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.187+0000 7fa3c9653640 1 --2- 172.21.15.152:0/2070130628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c41009b0 0x7fa3c4198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.187+0000 7fa3c9653640 1 -- 172.21.15.152:0/2070130628 >> 172.21.15.152:0/2070130628 conn(0x7fa3c4077990 msgr2=0x7fa3c40699a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.187+0000 7fa3c9653640 1 -- 172.21.15.152:0/2070130628 shutdown_connections 2024-09-11T22:55:02.191 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:02.187+0000 7fa3c9653640 1 -- 172.21.15.152:0/2070130628 wait complete. 2024-09-11T22:55:03.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:02 smithi191 ceph-mon[35211]: pgmap v67: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 507 MiB/s wr, 341 op/s 2024-09-11T22:55:03.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:02 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:55:03.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:02 smithi204 ceph-mon[35454]: pgmap v67: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 507 MiB/s wr, 341 op/s 2024-09-11T22:55:03.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:02 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:55:03.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:02 smithi152 ceph-mon[31181]: pgmap v67: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 507 MiB/s wr, 341 op/s 2024-09-11T22:55:03.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:02 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:55:03.671 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:55:03.671 INFO:tasks.workunit.client.0.smithi152.stdout: Children see throughput for 1 initial writers = 849586.06 kB/sec 2024-09-11T22:55:03.671 INFO:tasks.workunit.client.0.smithi152.stdout: Parent sees throughput for 1 initial writers = 849569.53 kB/sec 2024-09-11T22:55:03.671 INFO:tasks.workunit.client.0.smithi152.stdout: Min throughput per process = 849586.06 kB/sec 2024-09-11T22:55:03.671 INFO:tasks.workunit.client.0.smithi152.stdout: Max throughput per process = 849586.06 kB/sec 2024-09-11T22:55:03.671 INFO:tasks.workunit.client.0.smithi152.stdout: Avg throughput per process = 849586.06 kB/sec 2024-09-11T22:55:03.671 INFO:tasks.workunit.client.0.smithi152.stdout: Min xfer = 10485760.00 kB 2024-09-11T22:55:04.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:04 smithi191 ceph-mon[35211]: pgmap v68: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 507 MiB/s wr, 341 op/s 2024-09-11T22:55:04.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:04 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:55:04.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:04 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:55:04.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:04 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:55:04.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:04 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:55:04.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:04 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:55:04.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:04 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:55:04.538 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:04 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:55:04.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:04 smithi204 ceph-mon[35454]: pgmap v68: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 507 MiB/s wr, 341 op/s 2024-09-11T22:55:04.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:04 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:55:04.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:04 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:55:04.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:04 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:55:04.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:04 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:55:04.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:04 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:55:04.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:04 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:55:04.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:04 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:55:04.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:04 smithi152 ceph-mon[31181]: pgmap v68: 97 pgs: 97 active+clean; 5.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 507 MiB/s wr, 341 op/s 2024-09-11T22:55:04.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:04 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:55:04.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:04 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:55:04.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:04 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:55:04.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:04 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:55:04.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:04 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:55:04.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:04 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:55:04.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:04 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:55:06.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:06 smithi191 ceph-mon[35211]: pgmap v69: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 854 MiB/s wr, 451 op/s 2024-09-11T22:55:06.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:06 smithi204 ceph-mon[35454]: pgmap v69: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 854 MiB/s wr, 451 op/s 2024-09-11T22:55:06.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:06 smithi152 ceph-mon[31181]: pgmap v69: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 854 MiB/s wr, 451 op/s 2024-09-11T22:55:07.396 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:55:07.657 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:55:08.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.219+0000 7f33ea7d1640 1 -- 172.21.15.152:0/3644511527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e41082a0 msgr2=0x7f33e410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:08.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.219+0000 7f33ea7d1640 1 --2- 172.21.15.152:0/3644511527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e41082a0 0x7f33e410ec40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009cd0 tx=0x7f33dc02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:55:08.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.219+0000 7f33ea7d1640 1 -- 172.21.15.152:0/3644511527 shutdown_connections 2024-09-11T22:55:08.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.219+0000 7f33ea7d1640 1 --2- 172.21.15.152:0/3644511527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e41082a0 0x7f33e410ec40 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:08.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.219+0000 7f33ea7d1640 1 --2- 172.21.15.152:0/3644511527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33e41036e0 0x7f33e4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:08.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.219+0000 7f33ea7d1640 1 --2- 172.21.15.152:0/3644511527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33e4102d30 0x7f33e4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.219+0000 7f33ea7d1640 1 -- 172.21.15.152:0/3644511527 >> 172.21.15.152:0/3644511527 conn(0x7f33e40fc7d0 msgr2=0x7f33e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.219+0000 7f33ea7d1640 1 -- 172.21.15.152:0/3644511527 shutdown_connections 2024-09-11T22:55:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.219+0000 7f33ea7d1640 1 -- 172.21.15.152:0/3644511527 wait complete. 2024-09-11T22:55:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.220+0000 7f33ea7d1640 1 Processor -- start 2024-09-11T22:55:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.220+0000 7f33ea7d1640 1 -- start start 2024-09-11T22:55:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.220+0000 7f33ea7d1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33e4102d30 0x7f33e41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.220+0000 7f33ea7d1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33e41036e0 0x7f33e4198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.220+0000 7f33ea7d1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e41082a0 0x7f33e419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.220+0000 7f33ea7d1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f33e41120a0 con 0x7f33e41082a0 2024-09-11T22:55:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.220+0000 7f33ea7d1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4111f20 con 0x7f33e4102d30 2024-09-11T22:55:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.220+0000 7f33ea7d1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f33e4112220 con 0x7f33e41036e0 2024-09-11T22:55:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.220+0000 7f33e8d6e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e41082a0 0x7f33e419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.220+0000 7f33e8d6e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e41082a0 0x7f33e419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46294/0 (socket says 172.21.15.152:46294) 2024-09-11T22:55:08.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.220+0000 7f33e8d6e640 1 -- 172.21.15.152:0/3889207713 learned_addr learned my addr 172.21.15.152:0/3889207713 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.220+0000 7f33e3fff640 1 --2- 172.21.15.152:0/3889207713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33e4102d30 0x7f33e41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.220+0000 7f33dbfff640 1 --2- 172.21.15.152:0/3889207713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33e41036e0 0x7f33e4198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.221+0000 7f33e8d6e640 1 -- 172.21.15.152:0/3889207713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33e41036e0 msgr2=0x7f33e4198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.221+0000 7f33e8d6e640 1 --2- 172.21.15.152:0/3889207713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33e41036e0 0x7f33e4198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.221+0000 7f33e8d6e640 1 -- 172.21.15.152:0/3889207713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33e4102d30 msgr2=0x7f33e41984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.221+0000 7f33e8d6e640 1 --2- 172.21.15.152:0/3889207713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33e4102d30 0x7f33e41984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.221+0000 7f33e8d6e640 1 -- 172.21.15.152:0/3889207713 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e419d490 con 0x7f33e41082a0 2024-09-11T22:55:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.221+0000 7f33e8d6e640 1 --2- 172.21.15.152:0/3889207713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e41082a0 0x7f33e419cdb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f33dc005cc0 tx=0x7f33dc003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.221+0000 7f33e1ffb640 1 -- 172.21.15.152:0/3889207713 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc004040 con 0x7f33e41082a0 2024-09-11T22:55:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.221+0000 7f33ea7d1640 1 -- 172.21.15.152:0/3889207713 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e419d720 con 0x7f33e41082a0 2024-09-11T22:55:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.221+0000 7f33e1ffb640 1 -- 172.21.15.152:0/3889207713 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33dc0046c0 con 0x7f33e41082a0 2024-09-11T22:55:08.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.221+0000 7f33e1ffb640 1 -- 172.21.15.152:0/3889207713 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc030e70 con 0x7f33e41082a0 2024-09-11T22:55:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.221+0000 7f33ea7d1640 1 -- 172.21.15.152:0/3889207713 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e41a5060 con 0x7f33e41082a0 2024-09-11T22:55:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.222+0000 7f33ea7d1640 1 -- 172.21.15.152:0/3889207713 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33e4103110 con 0x7f33e41082a0 2024-09-11T22:55:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.223+0000 7f33e1ffb640 1 -- 172.21.15.152:0/3889207713 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f33dc0305a0 con 0x7f33e41082a0 2024-09-11T22:55:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.224+0000 7f33e1ffb640 1 --2- 172.21.15.152:0/3889207713 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f33d00a2d70 0x7f33d00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.224+0000 7f33e3fff640 1 --2- 172.21.15.152:0/3889207713 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f33d00a2d70 0x7f33d00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.224+0000 7f33e1ffb640 1 -- 172.21.15.152:0/3889207713 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f33dc02e030 con 0x7f33e41082a0 2024-09-11T22:55:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.225+0000 7f33e3fff640 1 --2- 172.21.15.152:0/3889207713 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f33d00a2d70 0x7f33d00a5230 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f33d400abe0 tx=0x7f33d400a2d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:08.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.231+0000 7f33e1ffb640 1 -- 172.21.15.152:0/3889207713 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f33dc0bd620 con 0x7f33e41082a0 2024-09-11T22:55:08.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.441+0000 7f33ea7d1640 1 -- 172.21.15.152:0/3889207713 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33e4199840 con 0x7f33e41082a0 2024-09-11T22:55:08.443 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:08 smithi152 ceph-mon[31181]: pgmap v70: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 707 MiB/s wr, 320 op/s 2024-09-11T22:55:08.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.442+0000 7f33e1ffb640 1 -- 172.21.15.152:0/3889207713 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f33e4199840 con 0x7f33e41082a0 2024-09-11T22:55:08.444 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:55:08.445 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:55:08.445 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:55:08.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.446+0000 7f33ea7d1640 1 -- 172.21.15.152:0/3889207713 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f33d00a2d70 msgr2=0x7f33d00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:08.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.446+0000 7f33ea7d1640 1 --2- 172.21.15.152:0/3889207713 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f33d00a2d70 0x7f33d00a5230 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f33d400abe0 tx=0x7f33d400a2d0 comp rx=0 tx=0).stop 2024-09-11T22:55:08.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.446+0000 7f33ea7d1640 1 -- 172.21.15.152:0/3889207713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e41082a0 msgr2=0x7f33e419cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:08.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.446+0000 7f33ea7d1640 1 --2- 172.21.15.152:0/3889207713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e41082a0 0x7f33e419cdb0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f33dc005cc0 tx=0x7f33dc003970 comp rx=0 tx=0).stop 2024-09-11T22:55:08.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.446+0000 7f33ea7d1640 1 -- 172.21.15.152:0/3889207713 shutdown_connections 2024-09-11T22:55:08.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.446+0000 7f33ea7d1640 1 --2- 172.21.15.152:0/3889207713 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f33d00a2d70 0x7f33d00a5230 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:08.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.446+0000 7f33ea7d1640 1 --2- 172.21.15.152:0/3889207713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33e41082a0 0x7f33e419cdb0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:08.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.446+0000 7f33ea7d1640 1 --2- 172.21.15.152:0/3889207713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33e41036e0 0x7f33e4198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:08.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.446+0000 7f33ea7d1640 1 --2- 172.21.15.152:0/3889207713 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33e4102d30 0x7f33e41984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:08.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.446+0000 7f33ea7d1640 1 -- 172.21.15.152:0/3889207713 >> 172.21.15.152:0/3889207713 conn(0x7f33e40fc7d0 msgr2=0x7f33e40fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:08.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.446+0000 7f33ea7d1640 1 -- 172.21.15.152:0/3889207713 shutdown_connections 2024-09-11T22:55:08.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:08.446+0000 7f33ea7d1640 1 -- 172.21.15.152:0/3889207713 wait complete. 2024-09-11T22:55:08.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:08 smithi191 ceph-mon[35211]: pgmap v70: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 707 MiB/s wr, 320 op/s 2024-09-11T22:55:08.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:08 smithi204 ceph-mon[35454]: pgmap v70: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 707 MiB/s wr, 320 op/s 2024-09-11T22:55:09.259 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24455 quiesce db --set-id df9f6e3e --release --await' 2024-09-11T22:55:09.260 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24455 quiesce db --set-id df9f6e3e --release --await 2024-09-11T22:55:09.510 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:55:09.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:09 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3889207713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:55:09.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:09 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3889207713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:55:09.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:09 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3889207713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:55:10.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.018+0000 7ff7df495640 1 -- 172.21.15.152:0/3247657400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7d8102790 msgr2=0x7ff7d81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:10.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.018+0000 7ff7df495640 1 --2- 172.21.15.152:0/3247657400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7d8102790 0x7ff7d81067e0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8009900 tx=0x7ff7c802a410 comp rx=0 tx=0).stop 2024-09-11T22:55:10.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.018+0000 7ff7df495640 1 -- 172.21.15.152:0/3247657400 shutdown_connections 2024-09-11T22:55:10.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.018+0000 7ff7df495640 1 --2- 172.21.15.152:0/3247657400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d8106f10 0x7ff7d810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.018+0000 7ff7df495640 1 --2- 172.21.15.152:0/3247657400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7d8102790 0x7ff7d81067e0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.018+0000 7ff7df495640 1 --2- 172.21.15.152:0/3247657400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7d8101de0 0x7ff7d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.018+0000 7ff7df495640 1 -- 172.21.15.152:0/3247657400 >> 172.21.15.152:0/3247657400 conn(0x7ff7d80fd8c0 msgr2=0x7ff7d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:10.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.018+0000 7ff7df495640 1 -- 172.21.15.152:0/3247657400 shutdown_connections 2024-09-11T22:55:10.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.018+0000 7ff7df495640 1 -- 172.21.15.152:0/3247657400 wait complete. 2024-09-11T22:55:10.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.019+0000 7ff7df495640 1 Processor -- start 2024-09-11T22:55:10.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.019+0000 7ff7df495640 1 -- start start 2024-09-11T22:55:10.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.019+0000 7ff7df495640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7d8101de0 0x7ff7d8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.019+0000 7ff7df495640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7d8102790 0x7ff7d8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.019+0000 7ff7df495640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d8106f10 0x7ff7d819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.019+0000 7ff7df495640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d8111f70 con 0x7ff7d8102790 2024-09-11T22:55:10.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.019+0000 7ff7df495640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d8111df0 con 0x7ff7d8106f10 2024-09-11T22:55:10.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.019+0000 7ff7df495640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d81120f0 con 0x7ff7d8101de0 2024-09-11T22:55:10.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.019+0000 7ff7dca30640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7d8102790 0x7ff7d8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.019+0000 7ff7dca30640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7d8102790 0x7ff7d8198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51326/0 (socket says 172.21.15.152:51326) 2024-09-11T22:55:10.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.019+0000 7ff7dca30640 1 -- 172.21.15.152:0/1657866123 learned_addr learned my addr 172.21.15.152:0/1657866123 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:10.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.019+0000 7ff7dda32640 1 --2- 172.21.15.152:0/1657866123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d8106f10 0x7ff7d819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.019+0000 7ff7dd231640 1 --2- 172.21.15.152:0/1657866123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7d8101de0 0x7ff7d8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.020+0000 7ff7dca30640 1 -- 172.21.15.152:0/1657866123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7d8101de0 msgr2=0x7ff7d8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:10.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.020+0000 7ff7dca30640 1 --2- 172.21.15.152:0/1657866123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7d8101de0 0x7ff7d8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.020+0000 7ff7dca30640 1 -- 172.21.15.152:0/1657866123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d8106f10 msgr2=0x7ff7d819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:10.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.020+0000 7ff7dca30640 1 --2- 172.21.15.152:0/1657866123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d8106f10 0x7ff7d819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.020+0000 7ff7dca30640 1 -- 172.21.15.152:0/1657866123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d819d3f0 con 0x7ff7d8102790 2024-09-11T22:55:10.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.020+0000 7ff7dca30640 1 --2- 172.21.15.152:0/1657866123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7d8102790 0x7ff7d8198980 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8006010 tx=0x7ff7c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:10.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.020+0000 7ff7ce7fc640 1 -- 172.21.15.152:0/1657866123 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c803f030 con 0x7ff7d8102790 2024-09-11T22:55:10.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.020+0000 7ff7ce7fc640 1 -- 172.21.15.152:0/1657866123 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7c8003ce0 con 0x7ff7d8102790 2024-09-11T22:55:10.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.020+0000 7ff7df495640 1 -- 172.21.15.152:0/1657866123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d819d680 con 0x7ff7d8102790 2024-09-11T22:55:10.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.020+0000 7ff7ce7fc640 1 -- 172.21.15.152:0/1657866123 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8035070 con 0x7ff7d8102790 2024-09-11T22:55:10.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.020+0000 7ff7df495640 1 -- 172.21.15.152:0/1657866123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d81a4fc0 con 0x7ff7d8102790 2024-09-11T22:55:10.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.022+0000 7ff7ce7fc640 1 -- 172.21.15.152:0/1657866123 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff7c8042030 con 0x7ff7d8102790 2024-09-11T22:55:10.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.023+0000 7ff7e186e740 1 Processor -- start 2024-09-11T22:55:10.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.023+0000 7ff7e186e740 1 -- start start 2024-09-11T22:55:10.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.023+0000 7ff7e186e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2a7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.023+0000 7ff7e186e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2a7980 0x555f4e2b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.023+0000 7ff7e186e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2b2240 0x555f4e2b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.023+0000 7ff7e186e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x555f4e20c620 con 0x555f4e2a7060 2024-09-11T22:55:10.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.023+0000 7ff7e186e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555f4e1b3e50 con 0x555f4e2b2240 2024-09-11T22:55:10.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.023+0000 7ff7e186e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555f4e2085e0 con 0x555f4e2a7980 2024-09-11T22:55:10.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.023+0000 7ff7ce7fc640 1 --2- 172.21.15.152:0/1657866123 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff7c40a2d70 0x7ff7c40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.023+0000 7ff7dda32640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.023+0000 7ff7dda32640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2a7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51336/0 (socket says 172.21.15.152:51336) 2024-09-11T22:55:10.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.023+0000 7ff7dda32640 1 -- 172.21.15.152:0/4163459278 learned_addr learned my addr 172.21.15.152:0/4163459278 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:10.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.024+0000 7ff7dd231640 1 --2- 172.21.15.152:0/4163459278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2a7980 0x555f4e2b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.024+0000 7ff7dda32640 1 --2- 172.21.15.152:0/1657866123 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff7c40a2d70 0x7ff7c40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.024+0000 7ff7ce7fc640 1 -- 172.21.15.152:0/1657866123 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7ff7c80ef8f0 con 0x7ff7d8102790 2024-09-11T22:55:10.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.024+0000 7ff7dca30640 1 --2- 172.21.15.152:0/4163459278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2b2240 0x555f4e2b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.024+0000 7ff7dda32640 1 -- 172.21.15.152:0/4163459278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2a7980 msgr2=0x555f4e2b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:10.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.024+0000 7ff7dda32640 1 --2- 172.21.15.152:0/4163459278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2a7980 0x555f4e2b1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.024+0000 7ff7dda32640 1 -- 172.21.15.152:0/4163459278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2b2240 msgr2=0x555f4e2b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:10.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.024+0000 7ff7dda32640 1 --2- 172.21.15.152:0/4163459278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2b2240 0x555f4e2b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.024+0000 7ff7dda32640 1 -- 172.21.15.152:0/4163459278 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f4e1a4d40 con 0x555f4e2a7060 2024-09-11T22:55:10.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.024+0000 7ff7dd231640 1 --2- 172.21.15.152:0/4163459278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2a7980 0x555f4e2b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:10.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.024+0000 7ff7dda32640 1 --2- 172.21.15.152:0/4163459278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2a7440 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0013940 tx=0x7ff7d0013e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb5a63cb69718f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:10.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.024+0000 7ff7c37fe640 1 -- 172.21.15.152:0/4163459278 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0021030 con 0x555f4e2a7060 2024-09-11T22:55:10.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.024+0000 7ff7c37fe640 1 -- 172.21.15.152:0/4163459278 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7d00043a0 con 0x555f4e2a7060 2024-09-11T22:55:10.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.024+0000 7ff7c37fe640 1 -- 172.21.15.152:0/4163459278 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d001d940 con 0x555f4e2a7060 2024-09-11T22:55:10.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.025+0000 7ff7e186e740 1 -- 172.21.15.152:0/4163459278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 msgr2=0x555f4e2a7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:10.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.025+0000 7ff7dda32640 1 --2- 172.21.15.152:0/1657866123 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff7c40a2d70 0x7ff7c40a5230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0021000 tx=0x7ff7d0020000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:10.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.025+0000 7ff7e186e740 1 --2- 172.21.15.152:0/4163459278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2a7440 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0013940 tx=0x7ff7d0013e00 comp rx=0 tx=0).stop 2024-09-11T22:55:10.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.025+0000 7ff7e186e740 1 -- 172.21.15.152:0/4163459278 shutdown_connections 2024-09-11T22:55:10.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.025+0000 7ff7e186e740 1 --2- 172.21.15.152:0/4163459278 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2b2240 0x555f4e2b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.025+0000 7ff7e186e740 1 --2- 172.21.15.152:0/4163459278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2a7980 0x555f4e2b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.025+0000 7ff7e186e740 1 --2- 172.21.15.152:0/4163459278 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2a7440 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.025+0000 7ff7e186e740 1 -- 172.21.15.152:0/4163459278 >> 172.21.15.152:0/4163459278 conn(0x555f4e2a35f0 msgr2=0x555f4e2a39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:10.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.025+0000 7ff7e186e740 1 -- 172.21.15.152:0/4163459278 shutdown_connections 2024-09-11T22:55:10.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.025+0000 7ff7e186e740 1 -- 172.21.15.152:0/4163459278 wait complete. 2024-09-11T22:55:10.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.025+0000 7ff7e186e740 1 -- start start 2024-09-11T22:55:10.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.025+0000 7ff7e186e740 1 Processor -- start 2024-09-11T22:55:10.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7e186e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2af170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7e186e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2a7980 0x555f4e2af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7e186e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2b2240 0x555f4e2afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7e186e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x555f4e2b4e10 con 0x555f4e2a7060 2024-09-11T22:55:10.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7e186e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555f4e194f10 con 0x555f4e2b2240 2024-09-11T22:55:10.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7e186e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555f4e2b4f90 con 0x555f4e2a7980 2024-09-11T22:55:10.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7e186e740 10 client.0 _ll_register_callbacks cb 0x555f4e1b8db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:10.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7dd231640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.057 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7dd231640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2af170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51338/0 (socket says 172.21.15.152:51338) 2024-09-11T22:55:10.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7dd231640 1 -- 172.21.15.152:0/1627116239 learned_addr learned my addr 172.21.15.152:0/1627116239 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:10.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7dda32640 1 --2- 172.21.15.152:0/1627116239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2b2240 0x555f4e2afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7dca30640 1 --2- 172.21.15.152:0/1627116239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2a7980 0x555f4e2af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7dd231640 1 -- 172.21.15.152:0/1627116239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2a7980 msgr2=0x555f4e2af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:10.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7dd231640 1 --2- 172.21.15.152:0/1627116239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2a7980 0x555f4e2af6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7dd231640 1 -- 172.21.15.152:0/1627116239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2b2240 msgr2=0x555f4e2afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:10.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7dd231640 1 --2- 172.21.15.152:0/1627116239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2b2240 0x555f4e2afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7dd231640 1 -- 172.21.15.152:0/1627116239 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f4df1b580 con 0x555f4e2a7060 2024-09-11T22:55:10.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7dd231640 1 --2- 172.21.15.152:0/1627116239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2af170 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4012730 tx=0x7ff7d4012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723fc240a6d6e594 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:10.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.026+0000 7ff7c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:10.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.027+0000 7ff7c0ff9640 1 -- 172.21.15.152:0/1627116239 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d4004020 con 0x555f4e2a7060 2024-09-11T22:55:10.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.027+0000 7ff7c0ff9640 1 -- 172.21.15.152:0/1627116239 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7ff7d40041c0 con 0x555f4e2a7060 2024-09-11T22:55:10.058 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.027+0000 7ff7e186e740 1 -- 172.21.15.152:0/1627116239 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7d00043a0 con 0x555f4e2a7060 2024-09-11T22:55:10.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.027+0000 7ff7c0ff9640 1 -- 172.21.15.152:0/1627116239 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff7d40043c0 con 0x555f4e2a7060 2024-09-11T22:55:10.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.027+0000 7ff7e186e740 10 client.16119 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:10.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.027+0000 7ff7e186e740 1 -- 172.21.15.152:0/1627116239 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f4dfe5960 con 0x555f4e2a7060 2024-09-11T22:55:10.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.027+0000 7ff7c0ff9640 1 -- 172.21.15.152:0/1627116239 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7d00043a0 con 0x555f4e2a7060 2024-09-11T22:55:10.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.027+0000 7ff7c0ff9640 1 -- 172.21.15.152:0/1627116239 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff7d4003b50 con 0x555f4e2a7060 2024-09-11T22:55:10.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.027+0000 7ff7e186e740 10 client.16119 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:10.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.027+0000 7ff7e186e740 10 client.16119 resolve_mds: validated gid 24455 aka daemon mds.b 2024-09-11T22:55:10.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.027+0000 7ff7e186e740 20 client.16119 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:10.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.027+0000 7ff7e186e740 1 --2- 172.21.15.152:0/1627116239 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555f4e2c1db0 0x555f4e2c4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.027+0000 7ff7e186e740 4 client.16119 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:55:10.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.027+0000 7ff7e186e740 1 -- 172.21.15.152:0/1627116239 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f4e219b90 con 0x555f4e2c1db0 2024-09-11T22:55:10.059 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.028+0000 7ff7dca30640 1 --2- 172.21.15.152:0/1627116239 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555f4e2c1db0 0x555f4e2c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.028+0000 7ff7c0ff9640 10 client.16119 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:10.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.028+0000 7ff7dca30640 1 --2- 172.21.15.152:0/1627116239 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555f4e2c1db0 0x555f4e2c4200 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e420d407be00dcce server_cookie=19d27facae2ecb26 in_seq=0 out_seq=0 2024-09-11T22:55:10.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.029+0000 7ff7c0ff9640 1 -- 172.21.15.152:0/1627116239 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f4e219b90 con 0x555f4e2c1db0 2024-09-11T22:55:10.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.029+0000 7ff7c0ff9640 10 client.16119 handle_command_reply: tid=0 2024-09-11T22:55:10.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.029+0000 7ff7e186e740 1 client.16119 shutdown 2024-09-11T22:55:10.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.029+0000 7ff7e186e740 1 -- 172.21.15.152:0/1627116239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 msgr2=0x555f4e2af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:10.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.029+0000 7ff7e186e740 1 --2- 172.21.15.152:0/1627116239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2af170 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4012730 tx=0x7ff7d4012bf0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.029+0000 7ff7e186e740 1 -- 172.21.15.152:0/1627116239 shutdown_connections 2024-09-11T22:55:10.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.029+0000 7ff7e186e740 1 --2- 172.21.15.152:0/1627116239 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555f4e2c1db0 0x555f4e2c4200 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.029+0000 7ff7e186e740 1 --2- 172.21.15.152:0/1627116239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2b2240 0x555f4e2afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.029+0000 7ff7e186e740 1 --2- 172.21.15.152:0/1627116239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2a7980 0x555f4e2af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.029+0000 7ff7c0ff9640 0 client.16119 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:10.060 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.029+0000 7ff7e186e740 1 --2- 172.21.15.152:0/1627116239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2af170 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.029+0000 7ff7e186e740 1 -- 172.21.15.152:0/1627116239 >> 172.21.15.152:0/1627116239 conn(0x555f4e2a35f0 msgr2=0x555f4e2a2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:10.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.029+0000 7ff7e186e740 1 -- 172.21.15.152:0/1627116239 shutdown_connections 2024-09-11T22:55:10.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.029+0000 7ff7e186e740 1 -- 172.21.15.152:0/1627116239 wait complete. 2024-09-11T22:55:10.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.029+0000 7ff7e186e740 20 client.16119 trim_cache size 0 max 16384 2024-09-11T22:55:10.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7e186e740 1 Processor -- start 2024-09-11T22:55:10.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7e186e740 1 -- start start 2024-09-11T22:55:10.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7e186e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2b1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7e186e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2a7980 0x555f4e2a9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7e186e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2b2240 0x555f4e2c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7e186e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d81ad700 con 0x555f4e2a7060 2024-09-11T22:55:10.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7e186e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d81af240 con 0x555f4e2a7980 2024-09-11T22:55:10.061 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7e186e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d81addd0 con 0x555f4e2b2240 2024-09-11T22:55:10.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7dd231640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7dd231640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2b1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51348/0 (socket says 172.21.15.152:51348) 2024-09-11T22:55:10.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7dd231640 1 -- 172.21.15.152:0/1040957821 learned_addr learned my addr 172.21.15.152:0/1040957821 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:10.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7dca30640 1 --2- 172.21.15.152:0/1040957821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2a7980 0x555f4e2a9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7dda32640 1 --2- 172.21.15.152:0/1040957821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2b2240 0x555f4e2c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7dd231640 1 -- 172.21.15.152:0/1040957821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2b2240 msgr2=0x555f4e2c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:10.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7dd231640 1 --2- 172.21.15.152:0/1040957821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2b2240 0x555f4e2c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7dd231640 1 -- 172.21.15.152:0/1040957821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2a7980 msgr2=0x555f4e2a9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:10.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7dd231640 1 --2- 172.21.15.152:0/1040957821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2a7980 0x555f4e2a9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7dd231640 1 -- 172.21.15.152:0/1040957821 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f4e2a77b0 con 0x555f4e2a7060 2024-09-11T22:55:10.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7dda32640 1 --2- 172.21.15.152:0/1040957821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2b2240 0x555f4e2c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:10.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7dca30640 1 --2- 172.21.15.152:0/1040957821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2a7980 0x555f4e2a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:10.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7dd231640 1 --2- 172.21.15.152:0/1040957821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2b1040 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4012420 tx=0x7ff7d4009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d2e318ad2c7eda server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:10.062 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.046+0000 7ff7c2ffd640 1 -- 172.21.15.152:0/1040957821 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d4004000 con 0x555f4e2a7060 2024-09-11T22:55:10.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.047+0000 7ff7c2ffd640 1 -- 172.21.15.152:0/1040957821 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7d402b030 con 0x555f4e2a7060 2024-09-11T22:55:10.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.047+0000 7ff7c2ffd640 1 -- 172.21.15.152:0/1040957821 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d400fea0 con 0x555f4e2a7060 2024-09-11T22:55:10.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.047+0000 7ff7e186e740 1 -- 172.21.15.152:0/1040957821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 msgr2=0x555f4e2b1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:10.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.047+0000 7ff7e186e740 1 --2- 172.21.15.152:0/1040957821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2b1040 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4012420 tx=0x7ff7d4009e20 comp rx=0 tx=0).stop 2024-09-11T22:55:10.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.047+0000 7ff7e186e740 1 -- 172.21.15.152:0/1040957821 shutdown_connections 2024-09-11T22:55:10.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.047+0000 7ff7e186e740 1 --2- 172.21.15.152:0/1040957821 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2b2240 0x555f4e2c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.047+0000 7ff7e186e740 1 --2- 172.21.15.152:0/1040957821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2a7980 0x555f4e2a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.047+0000 7ff7e186e740 1 --2- 172.21.15.152:0/1040957821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2a7060 0x555f4e2b1040 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.047+0000 7ff7e186e740 1 -- 172.21.15.152:0/1040957821 >> 172.21.15.152:0/1040957821 conn(0x555f4e2a35f0 msgr2=0x555f4e2a2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:10.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.047+0000 7ff7e186e740 1 -- 172.21.15.152:0/1040957821 shutdown_connections 2024-09-11T22:55:10.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.047+0000 7ff7e186e740 1 -- 172.21.15.152:0/1040957821 wait complete. 2024-09-11T22:55:10.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.047+0000 7ff7e186e740 1 -- start start 2024-09-11T22:55:10.063 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7e186e740 1 Processor -- start 2024-09-11T22:55:10.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7e186e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2a7060 0x555f4e2ba530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7e186e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2a7980 0x555f4e2baa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7e186e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2b2240 0x555f4e2b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7e186e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d81aeb70 con 0x555f4e2b2240 2024-09-11T22:55:10.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7e186e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d81ae4a0 con 0x555f4e2a7980 2024-09-11T22:55:10.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7e186e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d81ab6c0 con 0x555f4e2a7060 2024-09-11T22:55:10.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7e186e740 10 client.0 _ll_register_callbacks cb 0x7ff7d4009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:10.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7dda32640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2b2240 0x555f4e2b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7dda32640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2b2240 0x555f4e2b6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51352/0 (socket says 172.21.15.152:51352) 2024-09-11T22:55:10.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7dda32640 1 -- 172.21.15.152:0/3090030276 learned_addr learned my addr 172.21.15.152:0/3090030276 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:10.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7dd231640 1 --2- 172.21.15.152:0/3090030276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2a7060 0x555f4e2ba530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7dca30640 1 --2- 172.21.15.152:0/3090030276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2a7980 0x555f4e2baa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7dda32640 1 -- 172.21.15.152:0/3090030276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2a7060 msgr2=0x555f4e2ba530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:10.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7dda32640 1 --2- 172.21.15.152:0/3090030276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2a7060 0x555f4e2ba530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7dda32640 1 -- 172.21.15.152:0/3090030276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2a7980 msgr2=0x555f4e2baa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:10.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7dda32640 1 --2- 172.21.15.152:0/3090030276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2a7980 0x555f4e2baa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7dda32640 1 -- 172.21.15.152:0/3090030276 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f4e2b7170 con 0x555f4e2b2240 2024-09-11T22:55:10.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.048+0000 7ff7dca30640 1 --2- 172.21.15.152:0/3090030276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2a7980 0x555f4e2baa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:55:10.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.049+0000 7ff7dda32640 1 --2- 172.21.15.152:0/3090030276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2b2240 0x555f4e2b6940 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0012010 tx=0x7ff7d0013690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2c7c631284f9f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:10.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.049+0000 7ff7c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:10.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.049+0000 7ff7c0ff9640 1 -- 172.21.15.152:0/3090030276 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d000a540 con 0x555f4e2b2240 2024-09-11T22:55:10.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.049+0000 7ff7c0ff9640 1 -- 172.21.15.152:0/3090030276 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7ff7d0003ec0 con 0x555f4e2b2240 2024-09-11T22:55:10.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.049+0000 7ff7e186e740 1 -- 172.21.15.152:0/3090030276 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7d402b030 con 0x555f4e2b2240 2024-09-11T22:55:10.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.049+0000 7ff7c0ff9640 1 -- 172.21.15.152:0/3090030276 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff7d000a880 con 0x555f4e2b2240 2024-09-11T22:55:10.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.049+0000 7ff7e186e740 10 client.16125 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:10.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.049+0000 7ff7e186e740 1 -- 172.21.15.152:0/3090030276 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f4e32d6f0 con 0x555f4e2b2240 2024-09-11T22:55:10.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.049+0000 7ff7c0ff9640 1 -- 172.21.15.152:0/3090030276 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7d402b030 con 0x555f4e2b2240 2024-09-11T22:55:10.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.049+0000 7ff7c0ff9640 1 -- 172.21.15.152:0/3090030276 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff7d0043730 con 0x555f4e2b2240 2024-09-11T22:55:10.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.049+0000 7ff7e186e740 10 client.16125 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:10.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.049+0000 7ff7e186e740 10 client.16125 resolve_mds: validated gid 24455 aka daemon mds.b 2024-09-11T22:55:10.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.049+0000 7ff7e186e740 20 client.16125 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:10.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.049+0000 7ff7e186e740 1 --2- 172.21.15.152:0/3090030276 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555f4e32db40 0x555f4e32ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:10.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.049+0000 7ff7e186e740 4 client.16125 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "df9f6e3e", "release": true, "await": true}] 2024-09-11T22:55:10.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.049+0000 7ff7e186e740 1 -- 172.21.15.152:0/3090030276 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "quiesce db", "set_id": "df9f6e3e", "release": true, "await": true}) -- 0x555f4e2140a0 con 0x555f4e32db40 2024-09-11T22:55:10.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.050+0000 7ff7dd231640 1 --2- 172.21.15.152:0/3090030276 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555f4e32db40 0x555f4e32ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:10.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.050+0000 7ff7dd231640 1 --2- 172.21.15.152:0/3090030276 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555f4e32db40 0x555f4e32ff80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba7ef802619fc4b6 server_cookie=79c039a18f101250 in_seq=0 out_seq=0 2024-09-11T22:55:10.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.050+0000 7ff7c0ff9640 10 client.16125 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:10.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.052+0000 7ff7c0ff9640 1 -- 172.21.15.152:0/3090030276 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x555f4e2140a0 con 0x555f4e32db40 2024-09-11T22:55:10.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.052+0000 7ff7c0ff9640 10 client.16125 handle_command_reply: tid=0 2024-09-11T22:55:10.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.052+0000 7ff7e186e740 1 client.16125 shutdown 2024-09-11T22:55:10.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.052+0000 7ff7e186e740 1 -- 172.21.15.152:0/3090030276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2b2240 msgr2=0x555f4e2b6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:10.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.052+0000 7ff7e186e740 1 --2- 172.21.15.152:0/3090030276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2b2240 0x555f4e2b6940 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0012010 tx=0x7ff7d0013690 comp rx=0 tx=0).stop 2024-09-11T22:55:10.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.052+0000 7ff7e186e740 1 -- 172.21.15.152:0/3090030276 shutdown_connections 2024-09-11T22:55:10.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.052+0000 7ff7e186e740 1 --2- 172.21.15.152:0/3090030276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f4e2b2240 0x555f4e2b6940 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.052+0000 7ff7e186e740 1 --2- 172.21.15.152:0/3090030276 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555f4e32db40 0x555f4e32ff80 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.052+0000 7ff7e186e740 1 --2- 172.21.15.152:0/3090030276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f4e2a7980 0x555f4e2baa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.052+0000 7ff7e186e740 1 --2- 172.21.15.152:0/3090030276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f4e2a7060 0x555f4e2ba530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.052+0000 7ff7e186e740 1 -- 172.21.15.152:0/3090030276 >> 172.21.15.152:0/3090030276 conn(0x555f4e2a35f0 msgr2=0x555f4e2a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:10.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.053+0000 7ff7c0ff9640 0 client.16125 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:10.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.053+0000 7ff7e186e740 1 -- 172.21.15.152:0/3090030276 shutdown_connections 2024-09-11T22:55:10.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.053+0000 7ff7e186e740 1 -- 172.21.15.152:0/3090030276 wait complete. 2024-09-11T22:55:10.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.053+0000 7ff7e186e740 20 client.16125 trim_cache size 0 max 16384 2024-09-11T22:55:10.069 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:55:10.069 INFO:teuthology.orchestra.run.smithi152.stdout: "epoch": 28, 2024-09-11T22:55:10.069 INFO:teuthology.orchestra.run.smithi152.stdout: "leader": 24455, 2024-09-11T22:55:10.069 INFO:teuthology.orchestra.run.smithi152.stdout: "set_version": 8, 2024-09-11T22:55:10.070 INFO:teuthology.orchestra.run.smithi152.stdout: "sets": { 2024-09-11T22:55:10.070 INFO:teuthology.orchestra.run.smithi152.stdout: "df9f6e3e": { 2024-09-11T22:55:10.070 INFO:teuthology.orchestra.run.smithi152.stdout: "version": 8, 2024-09-11T22:55:10.070 INFO:teuthology.orchestra.run.smithi152.stdout: "age_ref": 0.0, 2024-09-11T22:55:10.070 INFO:teuthology.orchestra.run.smithi152.stdout: "state": { 2024-09-11T22:55:10.070 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RELEASED", 2024-09-11T22:55:10.070 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.0 2024-09-11T22:55:10.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T22:55:10.070 INFO:teuthology.orchestra.run.smithi152.stdout: "timeout": 90.0, 2024-09-11T22:55:10.070 INFO:teuthology.orchestra.run.smithi152.stdout: "expiration": 133.4, 2024-09-11T22:55:10.070 INFO:teuthology.orchestra.run.smithi152.stdout: "members": { 2024-09-11T22:55:10.070 INFO:teuthology.orchestra.run.smithi152.stdout: "file:/": { 2024-09-11T22:55:10.070 INFO:teuthology.orchestra.run.smithi152.stdout: "excluded": false, 2024-09-11T22:55:10.071 INFO:teuthology.orchestra.run.smithi152.stdout: "state": { 2024-09-11T22:55:10.071 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RELEASED", 2024-09-11T22:55:10.071 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.0 2024-09-11T22:55:10.071 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:10.071 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:10.071 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:10.071 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:10.071 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:10.071 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:55:10.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.053+0000 7ff7df495640 1 -- 172.21.15.152:0/1657866123 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff7c40a2d70 msgr2=0x7ff7c40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:10.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.053+0000 7ff7df495640 1 --2- 172.21.15.152:0/1657866123 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff7c40a2d70 0x7ff7c40a5230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff7d0021000 tx=0x7ff7d0020000 comp rx=0 tx=0).stop 2024-09-11T22:55:10.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.053+0000 7ff7df495640 1 -- 172.21.15.152:0/1657866123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7d8102790 msgr2=0x7ff7d8198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:10.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.053+0000 7ff7df495640 1 --2- 172.21.15.152:0/1657866123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7d8102790 0x7ff7d8198980 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8006010 tx=0x7ff7c80042f0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.053+0000 7ff7df495640 1 -- 172.21.15.152:0/1657866123 shutdown_connections 2024-09-11T22:55:10.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.053+0000 7ff7df495640 1 --2- 172.21.15.152:0/1657866123 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7ff7c40a2d70 0x7ff7c40a5230 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.053+0000 7ff7df495640 1 --2- 172.21.15.152:0/1657866123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7d8106f10 0x7ff7d819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.053+0000 7ff7df495640 1 --2- 172.21.15.152:0/1657866123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff7d8102790 0x7ff7d8198980 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.053+0000 7ff7df495640 1 --2- 172.21.15.152:0/1657866123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff7d8101de0 0x7ff7d8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:10.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.053+0000 7ff7df495640 1 -- 172.21.15.152:0/1657866123 >> 172.21.15.152:0/1657866123 conn(0x7ff7d80fd8c0 msgr2=0x7ff7d80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:10.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.053+0000 7ff7df495640 1 -- 172.21.15.152:0/1657866123 shutdown_connections 2024-09-11T22:55:10.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:10.053+0000 7ff7df495640 1 -- 172.21.15.152:0/1657866123 wait complete. 2024-09-11T22:55:10.530 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:10 smithi152 ceph-mon[31181]: pgmap v71: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 707 MiB/s wr, 325 op/s 2024-09-11T22:55:10.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:10 smithi191 ceph-mon[35211]: pgmap v71: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 707 MiB/s wr, 325 op/s 2024-09-11T22:55:10.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:10 smithi204 ceph-mon[35454]: pgmap v71: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 707 MiB/s wr, 325 op/s 2024-09-11T22:55:11.109 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'df9f6e3e', total seconds elapsed: 17.1 2024-09-11T22:55:12.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:12 smithi191 ceph-mon[35211]: pgmap v72: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 347 MiB/s wr, 114 op/s 2024-09-11T22:55:12.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:12 smithi204 ceph-mon[35454]: pgmap v72: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 347 MiB/s wr, 114 op/s 2024-09-11T22:55:12.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:12 smithi152 ceph-mon[31181]: pgmap v72: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 347 MiB/s wr, 114 op/s 2024-09-11T22:55:13.386 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:55:13.902 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:55:14.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:14 smithi152 ceph-mon[31181]: pgmap v73: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 347 MiB/s wr, 114 op/s 2024-09-11T22:55:14.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:14 smithi191 ceph-mon[35211]: pgmap v73: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 347 MiB/s wr, 114 op/s 2024-09-11T22:55:14.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:14 smithi204 ceph-mon[35454]: pgmap v73: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 347 MiB/s wr, 114 op/s 2024-09-11T22:55:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.334+0000 7f647ca23640 1 -- 172.21.15.152:0/4293910224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f647811b140 msgr2=0x7f647811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.334+0000 7f647ca23640 1 --2- 172.21.15.152:0/4293910224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f647811b140 0x7f647811d530 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6470009ca0 tx=0x7f647002a8b0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.335+0000 7f647ca23640 1 -- 172.21.15.152:0/4293910224 shutdown_connections 2024-09-11T22:55:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.335+0000 7f647ca23640 1 --2- 172.21.15.152:0/4293910224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f647811b140 0x7f647811d530 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.335+0000 7f647ca23640 1 --2- 172.21.15.152:0/4293910224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64781095d0 0x7f6478109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.335+0000 7f647ca23640 1 --2- 172.21.15.152:0/4293910224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64781091f0 0x7f6478110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.336 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.335+0000 7f647ca23640 1 -- 172.21.15.152:0/4293910224 >> 172.21.15.152:0/4293910224 conn(0x7f647806ce00 msgr2=0x7f647806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:15.340 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.339+0000 7f647ca23640 1 -- 172.21.15.152:0/4293910224 shutdown_connections 2024-09-11T22:55:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.340+0000 7f647ca23640 1 -- 172.21.15.152:0/4293910224 wait complete. 2024-09-11T22:55:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.343+0000 7f647ca23640 1 Processor -- start 2024-09-11T22:55:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.343+0000 7f647ca23640 1 -- start start 2024-09-11T22:55:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.343+0000 7f647ca23640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64781091f0 0x7f64781184e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.343+0000 7f647ca23640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64781095d0 0x7f6478118a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.343+0000 7f647ca23640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f647811b140 0x7f6478111580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.343+0000 7f647ca23640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6478120cf0 con 0x7f64781095d0 2024-09-11T22:55:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.343+0000 7f647ca23640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6478120b70 con 0x7f647811b140 2024-09-11T22:55:15.345 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.343+0000 7f647ca23640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6478120e70 con 0x7f64781091f0 2024-09-11T22:55:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.346+0000 7f6476d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f647811b140 0x7f6478111580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.346+0000 7f6476d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f647811b140 0x7f6478111580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:41916/0 (socket says 172.21.15.152:41916) 2024-09-11T22:55:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.346+0000 7f6476d9d640 1 -- 172.21.15.152:0/4183271230 learned_addr learned my addr 172.21.15.152:0/4183271230 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.347+0000 7f647659c640 1 --2- 172.21.15.152:0/4183271230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64781091f0 0x7f64781184e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.347+0000 7f6475d9b640 1 --2- 172.21.15.152:0/4183271230 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64781095d0 0x7f6478118a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.349+0000 7f6475d9b640 1 -- 172.21.15.152:0/4183271230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64781091f0 msgr2=0x7f64781184e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.349+0000 7f6475d9b640 1 --2- 172.21.15.152:0/4183271230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64781091f0 0x7f64781184e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.349+0000 7f6475d9b640 1 -- 172.21.15.152:0/4183271230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f647811b140 msgr2=0x7f6478111580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.349+0000 7f6475d9b640 1 --2- 172.21.15.152:0/4183271230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f647811b140 0x7f6478111580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.351 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.349+0000 7f6475d9b640 1 -- 172.21.15.152:0/4183271230 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6478111e40 con 0x7f64781095d0 2024-09-11T22:55:15.355 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.353+0000 7f6475d9b640 1 --2- 172.21.15.152:0/4183271230 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64781095d0 0x7f6478118a20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f646c00c970 tx=0x7f646c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.355+0000 7f64677fe640 1 -- 172.21.15.152:0/4183271230 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c007bb0 con 0x7f64781095d0 2024-09-11T22:55:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.356+0000 7f647ca23640 1 -- 172.21.15.152:0/4183271230 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6478112130 con 0x7f64781095d0 2024-09-11T22:55:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.356+0000 7f647ca23640 1 -- 172.21.15.152:0/4183271230 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64781161e0 con 0x7f64781095d0 2024-09-11T22:55:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.356+0000 7f64677fe640 1 -- 172.21.15.152:0/4183271230 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f646c007d50 con 0x7f64781095d0 2024-09-11T22:55:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.356+0000 7f64677fe640 1 -- 172.21.15.152:0/4183271230 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c00f450 con 0x7f64781095d0 2024-09-11T22:55:15.367 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.365+0000 7f647f7bd740 1 Processor -- start 2024-09-11T22:55:15.368 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.366+0000 7f64677fe640 1 -- 172.21.15.152:0/4183271230 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f646c00f5f0 con 0x7f64781095d0 2024-09-11T22:55:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.367+0000 7f64677fe640 1 --2- 172.21.15.152:0/4183271230 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f64600a2fc0 0x7f64600a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.368+0000 7f64677fe640 1 -- 172.21.15.152:0/4183271230 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f646c0cf0e0 con 0x7f64781095d0 2024-09-11T22:55:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.368+0000 7f647f7bd740 1 -- start start 2024-09-11T22:55:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.369+0000 7f647f7bd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e69af0 0x55f1b0e69ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.369+0000 7f647f7bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 0x55f1b0e74770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.369+0000 7f647f7bd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e770a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.369+0000 7f647f7bd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f1b0dce2f0 con 0x55f1b0e69af0 2024-09-11T22:55:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.369+0000 7f647f7bd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f1b0d76270 con 0x55f1b0e74cb0 2024-09-11T22:55:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.369+0000 7f647f7bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f1b0dca2b0 con 0x55f1b0e6a410 2024-09-11T22:55:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.373+0000 7f6475d9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 0x55f1b0e74770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.373+0000 7f6476d9d640 1 --2- 172.21.15.152:0/4183271230 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f64600a2fc0 0x7f64600a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.374+0000 7f647659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e69af0 0x55f1b0e69ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.374+0000 7f647659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e69af0 0x55f1b0e69ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51378/0 (socket says 172.21.15.152:51378) 2024-09-11T22:55:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.374+0000 7f647659c640 1 -- 172.21.15.152:0/3968016443 learned_addr learned my addr 172.21.15.152:0/3968016443 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.374+0000 7f6476d9d640 1 --2- 172.21.15.152:0/3968016443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e770a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.375+0000 7f647659c640 1 -- 172.21.15.152:0/3968016443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 msgr2=0x55f1b0e74770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.375+0000 7f647659c640 1 --2- 172.21.15.152:0/3968016443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 0x55f1b0e74770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.376+0000 7f647659c640 1 -- 172.21.15.152:0/3968016443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e74cb0 msgr2=0x55f1b0e770a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.376+0000 7f647659c640 1 --2- 172.21.15.152:0/3968016443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e770a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.376+0000 7f647659c640 1 -- 172.21.15.152:0/3968016443 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1b0d75340 con 0x55f1b0e69af0 2024-09-11T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.376+0000 7f647659c640 1 --2- 172.21.15.152:0/3968016443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e69af0 0x55f1b0e69ed0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f646800ac20 tx=0x7f646800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57202eeea7533e0e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.377+0000 7f6476d9d640 1 --2- 172.21.15.152:0/4183271230 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f64600a2fc0 0x7f64600a5480 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6478119100 tx=0x7f6470009940 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.379+0000 7f6464ff9640 1 -- 172.21.15.152:0/3968016443 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646801d070 con 0x55f1b0e69af0 2024-09-11T22:55:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.379+0000 7f6464ff9640 1 -- 172.21.15.152:0/3968016443 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6468018b20 con 0x55f1b0e69af0 2024-09-11T22:55:15.382 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.379+0000 7f6464ff9640 1 -- 172.21.15.152:0/3968016443 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468010bc0 con 0x55f1b0e69af0 2024-09-11T22:55:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.381+0000 7f647f7bd740 1 -- 172.21.15.152:0/3968016443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e69af0 msgr2=0x55f1b0e69ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.381+0000 7f647f7bd740 1 --2- 172.21.15.152:0/3968016443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e69af0 0x55f1b0e69ed0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f646800ac20 tx=0x7f646800f790 comp rx=0 tx=0).stop 2024-09-11T22:55:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.381+0000 7f647f7bd740 1 -- 172.21.15.152:0/3968016443 shutdown_connections 2024-09-11T22:55:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.381+0000 7f647f7bd740 1 --2- 172.21.15.152:0/3968016443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e770a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.381+0000 7f647f7bd740 1 --2- 172.21.15.152:0/3968016443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 0x55f1b0e74770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.381+0000 7f647f7bd740 1 --2- 172.21.15.152:0/3968016443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e69af0 0x55f1b0e69ed0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.384 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.381+0000 7f647f7bd740 1 -- 172.21.15.152:0/3968016443 >> 172.21.15.152:0/3968016443 conn(0x55f1b0e66060 msgr2=0x55f1b0e66440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.386+0000 7f647f7bd740 1 -- 172.21.15.152:0/3968016443 shutdown_connections 2024-09-11T22:55:15.390 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.389+0000 7f647f7bd740 1 -- 172.21.15.152:0/3968016443 wait complete. 2024-09-11T22:55:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.390+0000 7f647f7bd740 1 -- start start 2024-09-11T22:55:15.391 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.390+0000 7f647f7bd740 1 Processor -- start 2024-09-11T22:55:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.391+0000 7f647f7bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 0x55f1b0e71e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.391+0000 7f647f7bd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e72360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.391+0000 7f647f7bd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e728a0 0x55f1b0e80d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.391+0000 7f647f7bd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f1b0e77880 con 0x55f1b0e74cb0 2024-09-11T22:55:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.391+0000 7f647f7bd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f1b0d57290 con 0x55f1b0e728a0 2024-09-11T22:55:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.391+0000 7f647f7bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f1b0e77a00 con 0x55f1b0e6a410 2024-09-11T22:55:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.391+0000 7f647f7bd740 10 client.0 _ll_register_callbacks cb 0x55f1b0d88140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.393+0000 7f6475d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e72360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.393+0000 7f6475d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e72360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51388/0 (socket says 172.21.15.152:51388) 2024-09-11T22:55:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.393+0000 7f6475d9b640 1 -- 172.21.15.152:0/3465612491 learned_addr learned my addr 172.21.15.152:0/3465612491 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.394+0000 7f6476d9d640 1 --2- 172.21.15.152:0/3465612491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e728a0 0x55f1b0e80d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.394+0000 7f647659c640 1 --2- 172.21.15.152:0/3465612491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 0x55f1b0e71e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.397+0000 7f6475d9b640 1 -- 172.21.15.152:0/3465612491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 msgr2=0x55f1b0e71e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.397+0000 7f6475d9b640 1 --2- 172.21.15.152:0/3465612491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 0x55f1b0e71e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.397+0000 7f6475d9b640 1 -- 172.21.15.152:0/3465612491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e728a0 msgr2=0x55f1b0e80d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.397+0000 7f6475d9b640 1 --2- 172.21.15.152:0/3465612491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e728a0 0x55f1b0e80d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.397+0000 7f6475d9b640 1 -- 172.21.15.152:0/3465612491 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1b0ae1210 con 0x55f1b0e74cb0 2024-09-11T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.400+0000 7f6475d9b640 1 --2- 172.21.15.152:0/3465612491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e72360 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f646c00c940 tx=0x7f646c09b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c581b9a079889b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.401+0000 7f645dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.402+0000 7f645dffb640 1 -- 172.21.15.152:0/3465612491 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c0b3020 con 0x55f1b0e74cb0 2024-09-11T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.402+0000 7f647f7bd740 1 -- 172.21.15.152:0/3465612491 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6468018b20 con 0x55f1b0e74cb0 2024-09-11T22:55:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.402+0000 7f645dffb640 1 -- 172.21.15.152:0/3465612491 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f646c0a3df0 con 0x55f1b0e74cb0 2024-09-11T22:55:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.404+0000 7f645dffb640 1 -- 172.21.15.152:0/3465612491 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f646c01d040 con 0x55f1b0e74cb0 2024-09-11T22:55:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.406+0000 7f647f7bd740 10 client.16137 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.406+0000 7f647f7bd740 1 -- 172.21.15.152:0/3465612491 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1b0bad620 con 0x55f1b0e74cb0 2024-09-11T22:55:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.412+0000 7f645dffb640 1 -- 172.21.15.152:0/3465612491 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6468018b20 con 0x55f1b0e74cb0 2024-09-11T22:55:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.412+0000 7f645dffb640 1 -- 172.21.15.152:0/3465612491 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f646c09ab10 con 0x55f1b0e74cb0 2024-09-11T22:55:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.413+0000 7f647f7bd740 10 client.16137 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.413+0000 7f647f7bd740 10 client.16137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.413+0000 7f647f7bd740 20 client.16137 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.413+0000 7f647f7bd740 1 --2- 172.21.15.152:0/3465612491 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f1b0e85850 0x55f1b0e87c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.413+0000 7f647f7bd740 4 client.16137 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:55:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.413+0000 7f647f7bd740 1 -- 172.21.15.152:0/3465612491 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1b0ddba20 con 0x55f1b0e85850 2024-09-11T22:55:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.414+0000 7f647659c640 1 --2- 172.21.15.152:0/3465612491 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f1b0e85850 0x55f1b0e87c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.418+0000 7f647659c640 1 --2- 172.21.15.152:0/3465612491 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f1b0e85850 0x55f1b0e87c90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9eddf2157f424a0 server_cookie=eae5ec738e7936f8 in_seq=0 out_seq=0 2024-09-11T22:55:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.418+0000 7f645dffb640 10 client.16137 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.419+0000 7f645dffb640 1 -- 172.21.15.152:0/3465612491 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1b0ddba20 con 0x55f1b0e85850 2024-09-11T22:55:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.419+0000 7f645dffb640 10 client.16137 handle_command_reply: tid=0 2024-09-11T22:55:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.419+0000 7f647f7bd740 1 client.16137 shutdown 2024-09-11T22:55:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.419+0000 7f647f7bd740 1 -- 172.21.15.152:0/3465612491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e74cb0 msgr2=0x55f1b0e72360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.419+0000 7f647f7bd740 1 --2- 172.21.15.152:0/3465612491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e72360 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f646c00c940 tx=0x7f646c09b780 comp rx=0 tx=0).stop 2024-09-11T22:55:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.422+0000 7f647f7bd740 1 -- 172.21.15.152:0/3465612491 shutdown_connections 2024-09-11T22:55:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.422+0000 7f647f7bd740 1 --2- 172.21.15.152:0/3465612491 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f1b0e85850 0x55f1b0e87c90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.422+0000 7f647f7bd740 1 --2- 172.21.15.152:0/3465612491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e728a0 0x55f1b0e80d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.422+0000 7f647f7bd740 1 --2- 172.21.15.152:0/3465612491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e72360 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.422+0000 7f647f7bd740 1 --2- 172.21.15.152:0/3465612491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 0x55f1b0e71e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.422+0000 7f647f7bd740 1 -- 172.21.15.152:0/3465612491 >> 172.21.15.152:0/3465612491 conn(0x55f1b0e66060 msgr2=0x55f1b0e6bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.422+0000 7f645dffb640 0 client.16137 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:15.425 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.423+0000 7f647f7bd740 1 -- 172.21.15.152:0/3465612491 shutdown_connections 2024-09-11T22:55:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.425+0000 7f647f7bd740 1 -- 172.21.15.152:0/3465612491 wait complete. 2024-09-11T22:55:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.425+0000 7f647f7bd740 20 client.16137 trim_cache size 0 max 16384 2024-09-11T22:55:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.478+0000 7f647f7bd740 1 Processor -- start 2024-09-11T22:55:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.478+0000 7f647f7bd740 1 -- start start 2024-09-11T22:55:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.479+0000 7f647f7bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 0x55f1b0e706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.480+0000 7f647f7bd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e728a0 0x55f1b0e80b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.480+0000 7f647f7bd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e8d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.481+0000 7f647f7bd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f645800e7c0 con 0x55f1b0e74cb0 2024-09-11T22:55:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.481+0000 7f647f7bd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6458010300 con 0x55f1b0e728a0 2024-09-11T22:55:15.484 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.481+0000 7f647f7bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f645800ee90 con 0x55f1b0e6a410 2024-09-11T22:55:15.485 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.482+0000 7f6476d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e8d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.483+0000 7f6476d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e8d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51394/0 (socket says 172.21.15.152:51394) 2024-09-11T22:55:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.483+0000 7f6476d9d640 1 -- 172.21.15.152:0/760157772 learned_addr learned my addr 172.21.15.152:0/760157772 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.483+0000 7f6475d9b640 1 --2- 172.21.15.152:0/760157772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e728a0 0x55f1b0e80b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.484+0000 7f6476d9d640 1 -- 172.21.15.152:0/760157772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 msgr2=0x55f1b0e706d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.484+0000 7f6476d9d640 1 --2- 172.21.15.152:0/760157772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 0x55f1b0e706d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.484+0000 7f6476d9d640 1 -- 172.21.15.152:0/760157772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e728a0 msgr2=0x55f1b0e80b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.484+0000 7f6476d9d640 1 --2- 172.21.15.152:0/760157772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e728a0 0x55f1b0e80b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.484+0000 7f6476d9d640 1 -- 172.21.15.152:0/760157772 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1b0e70db0 con 0x55f1b0e74cb0 2024-09-11T22:55:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.485+0000 7f6476d9d640 1 --2- 172.21.15.152:0/760157772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e8d5a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6470030c10 tx=0x7f6470049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860aa94933b79f23 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.486+0000 7f645ffff640 1 -- 172.21.15.152:0/760157772 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470003d70 con 0x55f1b0e74cb0 2024-09-11T22:55:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.486+0000 7f645ffff640 1 -- 172.21.15.152:0/760157772 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6470050070 con 0x55f1b0e74cb0 2024-09-11T22:55:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.486+0000 7f645ffff640 1 -- 172.21.15.152:0/760157772 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470055a60 con 0x55f1b0e74cb0 2024-09-11T22:55:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.488+0000 7f647f7bd740 1 -- 172.21.15.152:0/760157772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e74cb0 msgr2=0x55f1b0e8d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.488+0000 7f647f7bd740 1 --2- 172.21.15.152:0/760157772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e8d5a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6470030c10 tx=0x7f6470049000 comp rx=0 tx=0).stop 2024-09-11T22:55:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.488+0000 7f647f7bd740 1 -- 172.21.15.152:0/760157772 shutdown_connections 2024-09-11T22:55:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.488+0000 7f647f7bd740 1 --2- 172.21.15.152:0/760157772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e8d5a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.488+0000 7f647f7bd740 1 --2- 172.21.15.152:0/760157772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e728a0 0x55f1b0e80b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.488+0000 7f647f7bd740 1 --2- 172.21.15.152:0/760157772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 0x55f1b0e706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.491 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.489+0000 7f647f7bd740 1 -- 172.21.15.152:0/760157772 >> 172.21.15.152:0/760157772 conn(0x55f1b0e66060 msgr2=0x55f1b0e73ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.489+0000 7f647f7bd740 1 -- 172.21.15.152:0/760157772 shutdown_connections 2024-09-11T22:55:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.489+0000 7f647f7bd740 1 -- 172.21.15.152:0/760157772 wait complete. 2024-09-11T22:55:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.490+0000 7f647f7bd740 1 -- start start 2024-09-11T22:55:15.492 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.490+0000 7f647f7bd740 1 Processor -- start 2024-09-11T22:55:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.491+0000 7f647f7bd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 0x55f1b0e7d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.491+0000 7f647f7bd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e728a0 0x55f1b0e7d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.491+0000 7f647f7bd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e6eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.491+0000 7f647f7bd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f645800fc30 con 0x55f1b0e728a0 2024-09-11T22:55:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.491+0000 7f647f7bd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f645800f560 con 0x55f1b0e74cb0 2024-09-11T22:55:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.491+0000 7f647f7bd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f645800c780 con 0x55f1b0e6a410 2024-09-11T22:55:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.491+0000 7f647f7bd740 10 client.0 _ll_register_callbacks cb 0x7f646c09b2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.491+0000 7f6475d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e728a0 0x55f1b0e7d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.491+0000 7f6475d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e728a0 0x55f1b0e7d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51402/0 (socket says 172.21.15.152:51402) 2024-09-11T22:55:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.491+0000 7f6475d9b640 1 -- 172.21.15.152:0/3693474237 learned_addr learned my addr 172.21.15.152:0/3693474237 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.492+0000 7f6475d9b640 1 -- 172.21.15.152:0/3693474237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 msgr2=0x55f1b0e7d090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.492+0000 7f6475d9b640 1 --2- 172.21.15.152:0/3693474237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 0x55f1b0e7d090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.492+0000 7f6475d9b640 1 -- 172.21.15.152:0/3693474237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e74cb0 msgr2=0x55f1b0e6eae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.492+0000 7f6475d9b640 1 --2- 172.21.15.152:0/3693474237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e6eae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.492+0000 7f6475d9b640 1 -- 172.21.15.152:0/3693474237 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1b0e6a240 con 0x55f1b0e728a0 2024-09-11T22:55:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.493+0000 7f6475d9b640 1 --2- 172.21.15.152:0/3693474237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e728a0 0x55f1b0e7d5d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f646c09b650 tx=0x7f646c00a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba1f62c1dcdc29d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.493+0000 7f645dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.493+0000 7f645dffb640 1 -- 172.21.15.152:0/3693474237 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c0b3070 con 0x55f1b0e728a0 2024-09-11T22:55:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.494+0000 7f645dffb640 1 -- 172.21.15.152:0/3693474237 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f646c0ccac0 con 0x55f1b0e728a0 2024-09-11T22:55:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.494+0000 7f647f7bd740 1 -- 172.21.15.152:0/3693474237 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6470050070 con 0x55f1b0e728a0 2024-09-11T22:55:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.494+0000 7f645dffb640 1 -- 172.21.15.152:0/3693474237 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f646c097960 con 0x55f1b0e728a0 2024-09-11T22:55:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.494+0000 7f647f7bd740 10 client.16143 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.494+0000 7f647f7bd740 1 -- 172.21.15.152:0/3693474237 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1b0ef0080 con 0x55f1b0e728a0 2024-09-11T22:55:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.495+0000 7f645dffb640 1 -- 172.21.15.152:0/3693474237 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6470050070 con 0x55f1b0e728a0 2024-09-11T22:55:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.496+0000 7f645dffb640 1 -- 172.21.15.152:0/3693474237 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f646c0a0b50 con 0x55f1b0e728a0 2024-09-11T22:55:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.496+0000 7f647f7bd740 10 client.16143 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.496+0000 7f647f7bd740 10 client.16143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.496+0000 7f647f7bd740 20 client.16143 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.496+0000 7f647f7bd740 1 --2- 172.21.15.152:0/3693474237 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f1b0ef0720 0x55f1b0ef2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.496+0000 7f647f7bd740 4 client.16143 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:55:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.496+0000 7f647f7bd740 1 -- 172.21.15.152:0/3693474237 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1b0dd5d70 con 0x55f1b0ef0720 2024-09-11T22:55:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.499+0000 7f647659c640 1 --2- 172.21.15.152:0/3693474237 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f1b0ef0720 0x55f1b0ef2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.507+0000 7f645dffb640 10 client.16143 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.507+0000 7f647659c640 1 --2- 172.21.15.152:0/3693474237 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f1b0ef0720 0x55f1b0ef2b40 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d79af3dca0bc35 server_cookie=73c341d4e122c58f in_seq=0 out_seq=0 2024-09-11T22:55:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.607+0000 7f645dffb640 1 -- 172.21.15.152:0/3693474237 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f1b0dd5d70 con 0x55f1b0ef0720 2024-09-11T22:55:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.607+0000 7f645dffb640 10 client.16143 handle_command_reply: tid=0 2024-09-11T22:55:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.607+0000 7f647f7bd740 1 client.16143 shutdown 2024-09-11T22:55:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.607+0000 7f647f7bd740 1 -- 172.21.15.152:0/3693474237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e728a0 msgr2=0x55f1b0e7d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.607+0000 7f647f7bd740 1 --2- 172.21.15.152:0/3693474237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e728a0 0x55f1b0e7d5d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f646c09b650 tx=0x7f646c00a470 comp rx=0 tx=0).stop 2024-09-11T22:55:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.607+0000 7f647f7bd740 1 -- 172.21.15.152:0/3693474237 shutdown_connections 2024-09-11T22:55:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.607+0000 7f647f7bd740 1 --2- 172.21.15.152:0/3693474237 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f1b0ef0720 0x55f1b0ef2b40 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.607+0000 7f647f7bd740 1 --2- 172.21.15.152:0/3693474237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1b0e74cb0 0x55f1b0e6eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.607+0000 7f647f7bd740 1 --2- 172.21.15.152:0/3693474237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1b0e728a0 0x55f1b0e7d5d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.607+0000 7f645dffb640 0 client.16143 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.607+0000 7f647f7bd740 1 --2- 172.21.15.152:0/3693474237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1b0e6a410 0x55f1b0e7d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.607+0000 7f647f7bd740 1 -- 172.21.15.152:0/3693474237 >> 172.21.15.152:0/3693474237 conn(0x55f1b0e66060 msgr2=0x55f1b0e6bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.607+0000 7f647f7bd740 1 -- 172.21.15.152:0/3693474237 shutdown_connections 2024-09-11T22:55:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.607+0000 7f647f7bd740 1 -- 172.21.15.152:0/3693474237 wait complete. 2024-09-11T22:55:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.607+0000 7f647f7bd740 20 client.16143 trim_cache size 0 max 16384 2024-09-11T22:55:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:{ 2024-09-11T22:55:15.611 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "message": "", 2024-09-11T22:55:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "return_code": 0 2024-09-11T22:55:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:} 2024-09-11T22:55:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.608+0000 7f647ca23640 1 -- 172.21.15.152:0/4183271230 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f64600a2fc0 msgr2=0x7f64600a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.608+0000 7f647ca23640 1 --2- 172.21.15.152:0/4183271230 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f64600a2fc0 0x7f64600a5480 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6478119100 tx=0x7f6470009940 comp rx=0 tx=0).stop 2024-09-11T22:55:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.608+0000 7f647ca23640 1 -- 172.21.15.152:0/4183271230 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64781095d0 msgr2=0x7f6478118a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.608+0000 7f647ca23640 1 --2- 172.21.15.152:0/4183271230 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64781095d0 0x7f6478118a20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f646c00c970 tx=0x7f646c00ce30 comp rx=0 tx=0).stop 2024-09-11T22:55:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.608+0000 7f647ca23640 1 -- 172.21.15.152:0/4183271230 shutdown_connections 2024-09-11T22:55:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.608+0000 7f647ca23640 1 --2- 172.21.15.152:0/4183271230 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f64600a2fc0 0x7f64600a5480 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.608+0000 7f647ca23640 1 --2- 172.21.15.152:0/4183271230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f647811b140 0x7f6478111580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.608+0000 7f647ca23640 1 --2- 172.21.15.152:0/4183271230 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64781095d0 0x7f6478118a20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.608+0000 7f647ca23640 1 --2- 172.21.15.152:0/4183271230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64781091f0 0x7f64781184e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.608+0000 7f647ca23640 1 -- 172.21.15.152:0/4183271230 >> 172.21.15.152:0/4183271230 conn(0x7f647806ce00 msgr2=0x7f647811b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.608+0000 7f647ca23640 1 -- 172.21.15.152:0/4183271230 shutdown_connections 2024-09-11T22:55:15.613 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:15.608+0000 7f647ca23640 1 -- 172.21.15.152:0/4183271230 wait complete. 2024-09-11T22:55:16.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:16 smithi191 ceph-mon[35211]: pgmap v74: 97 pgs: 97 active+clean; 13 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 575 MiB/s wr, 279 op/s 2024-09-11T22:55:16.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:16 smithi204 ceph-mon[35454]: pgmap v74: 97 pgs: 97 active+clean; 13 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 575 MiB/s wr, 279 op/s 2024-09-11T22:55:16.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:16 smithi152 ceph-mon[31181]: pgmap v74: 97 pgs: 97 active+clean; 13 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 575 MiB/s wr, 279 op/s 2024-09-11T22:55:18.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.207+0000 7fde6cd3a640 1 -- 172.21.15.152:0/2425293471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde68102570 msgr2=0x7fde681029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:18.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.207+0000 7fde6cd3a640 1 --2- 172.21.15.152:0/2425293471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde68102570 0x7fde681029b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009900 tx=0x7fde5c02a480 comp rx=0 tx=0).stop 2024-09-11T22:55:18.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.209+0000 7fde6cd3a640 1 -- 172.21.15.152:0/2425293471 shutdown_connections 2024-09-11T22:55:18.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.209+0000 7fde6cd3a640 1 --2- 172.21.15.152:0/2425293471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde68102ef0 0x7fde68069f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.209+0000 7fde6cd3a640 1 --2- 172.21.15.152:0/2425293471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde68102570 0x7fde681029b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.209+0000 7fde6cd3a640 1 --2- 172.21.15.152:0/2425293471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde68100e20 0x7fde68101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.209+0000 7fde6cd3a640 1 -- 172.21.15.152:0/2425293471 >> 172.21.15.152:0/2425293471 conn(0x7fde680fc7d0 msgr2=0x7fde680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:18.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.209+0000 7fde6cd3a640 1 -- 172.21.15.152:0/2425293471 shutdown_connections 2024-09-11T22:55:18.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.210+0000 7fde6cd3a640 1 -- 172.21.15.152:0/2425293471 wait complete. 2024-09-11T22:55:18.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.212+0000 7fde6cd3a640 1 Processor -- start 2024-09-11T22:55:18.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.214+0000 7fde6cd3a640 1 -- start start 2024-09-11T22:55:18.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.215+0000 7fde6cd3a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde68100e20 0x7fde681a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.215+0000 7fde6cd3a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde68102570 0x7fde681a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.215+0000 7fde6cd3a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde68102ef0 0x7fde6819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.215+0000 7fde6cd3a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fde681142e0 con 0x7fde68102570 2024-09-11T22:55:18.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.215+0000 7fde6cd3a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fde68114160 con 0x7fde68100e20 2024-09-11T22:55:18.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.215+0000 7fde6cd3a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fde68114460 con 0x7fde68102ef0 2024-09-11T22:55:18.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.216+0000 7fde6659c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde68100e20 0x7fde681a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:18.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.216+0000 7fde6659c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde68100e20 0x7fde681a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:41960/0 (socket says 172.21.15.152:41960) 2024-09-11T22:55:18.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.216+0000 7fde6659c640 1 -- 172.21.15.152:0/1565666528 learned_addr learned my addr 172.21.15.152:0/1565666528 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:18.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.216+0000 7fde66d9d640 1 --2- 172.21.15.152:0/1565666528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde68102ef0 0x7fde6819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:18.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.216+0000 7fde6659c640 1 -- 172.21.15.152:0/1565666528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde68102ef0 msgr2=0x7fde6819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:18.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.216+0000 7fde6659c640 1 --2- 172.21.15.152:0/1565666528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde68102ef0 0x7fde6819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.216+0000 7fde6659c640 1 -- 172.21.15.152:0/1565666528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde68102570 msgr2=0x7fde681a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:18.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.216+0000 7fde6659c640 1 --2- 172.21.15.152:0/1565666528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde68102570 0x7fde681a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.217+0000 7fde6659c640 1 -- 172.21.15.152:0/1565666528 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6819ace0 con 0x7fde68100e20 2024-09-11T22:55:18.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.217+0000 7fde6659c640 1 --2- 172.21.15.152:0/1565666528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde68100e20 0x7fde681a03f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fde5800d950 tx=0x7fde5800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:18.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.218+0000 7fde577fe640 1 -- 172.21.15.152:0/1565666528 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58017070 con 0x7fde68100e20 2024-09-11T22:55:18.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.218+0000 7fde577fe640 1 -- 172.21.15.152:0/1565666528 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde58009c30 con 0x7fde68100e20 2024-09-11T22:55:18.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.218+0000 7fde577fe640 1 -- 172.21.15.152:0/1565666528 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5800f6b0 con 0x7fde68100e20 2024-09-11T22:55:18.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.218+0000 7fde6cd3a640 1 -- 172.21.15.152:0/1565666528 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde6819afd0 con 0x7fde68100e20 2024-09-11T22:55:18.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.218+0000 7fde6cd3a640 1 -- 172.21.15.152:0/1565666528 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde681a72c0 con 0x7fde68100e20 2024-09-11T22:55:18.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.230+0000 7fde577fe640 1 -- 172.21.15.152:0/1565666528 <== mon.2 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fde5800f850 con 0x7fde68100e20 2024-09-11T22:55:18.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.231+0000 7fde577fe640 1 --2- 172.21.15.152:0/1565666528 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fde500a2f00 0x7fde500a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.232+0000 7fde577fe640 1 -- 172.21.15.152:0/1565666528 <== mon.2 v2:172.21.15.191:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fde580cf050 con 0x7fde68100e20 2024-09-11T22:55:18.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.233+0000 7fde6f113740 1 Processor -- start 2024-09-11T22:55:18.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.236+0000 7fde6f113740 1 -- start start 2024-09-11T22:55:18.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.237+0000 7fde6f113740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad8244a0 0x55d8ad824880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.237+0000 7fde6f113740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad824dc0 0x55d8ad82f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.237+0000 7fde6f113740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad82f680 0x55d8ad831a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.237+0000 7fde6f113740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d8ad78a2e0 con 0x55d8ad824dc0 2024-09-11T22:55:18.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.237+0000 7fde6f113740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d8ad731ce0 con 0x55d8ad82f680 2024-09-11T22:55:18.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.237+0000 7fde6f113740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d8ad7862a0 con 0x55d8ad8244a0 2024-09-11T22:55:18.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.239+0000 7fde66d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad82f680 0x55d8ad831a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:18.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.241+0000 7fde65d9b640 1 --2- 172.21.15.152:0/1565666528 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fde500a2f00 0x7fde500a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:18.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.241+0000 7fde65d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad824dc0 0x55d8ad82f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:18.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.243+0000 7fde66d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad82f680 0x55d8ad831a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:41964/0 (socket says 172.21.15.152:41964) 2024-09-11T22:55:18.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.243+0000 7fde66d9d640 1 -- 172.21.15.152:0/594247362 learned_addr learned my addr 172.21.15.152:0/594247362 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:18.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.244+0000 7fde6659c640 1 --2- 172.21.15.152:0/594247362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad8244a0 0x55d8ad824880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:18.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.247+0000 7fde65d9b640 1 --2- 172.21.15.152:0/1565666528 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fde500a2f00 0x7fde500a53c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fde680fec10 tx=0x7fde5c005d50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:18.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.250+0000 7fde65d9b640 1 -- 172.21.15.152:0/594247362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad8244a0 msgr2=0x55d8ad824880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:18.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.250+0000 7fde65d9b640 1 --2- 172.21.15.152:0/594247362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad8244a0 0x55d8ad824880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.250+0000 7fde65d9b640 1 -- 172.21.15.152:0/594247362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad82f680 msgr2=0x55d8ad831a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:18.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.250+0000 7fde65d9b640 1 --2- 172.21.15.152:0/594247362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad82f680 0x55d8ad831a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.250+0000 7fde65d9b640 1 -- 172.21.15.152:0/594247362 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8ad716440 con 0x55d8ad824dc0 2024-09-11T22:55:18.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.251+0000 7fde6659c640 1 --2- 172.21.15.152:0/594247362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad8244a0 0x55d8ad824880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:18.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.251+0000 7fde65d9b640 1 --2- 172.21.15.152:0/594247362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad824dc0 0x55d8ad82f140 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fde6819b6f0 tx=0x7fde5c043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8979619dda0c2895 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:18.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.251+0000 7fde54ff9640 1 -- 172.21.15.152:0/594247362 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c035040 con 0x55d8ad824dc0 2024-09-11T22:55:18.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.251+0000 7fde54ff9640 1 -- 172.21.15.152:0/594247362 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde5c004300 con 0x55d8ad824dc0 2024-09-11T22:55:18.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.253+0000 7fde6f113740 1 -- 172.21.15.152:0/594247362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad824dc0 msgr2=0x55d8ad82f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:18.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.253+0000 7fde6f113740 1 --2- 172.21.15.152:0/594247362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad824dc0 0x55d8ad82f140 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fde6819b6f0 tx=0x7fde5c043000 comp rx=0 tx=0).stop 2024-09-11T22:55:18.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.257+0000 7fde6f113740 1 -- 172.21.15.152:0/594247362 shutdown_connections 2024-09-11T22:55:18.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.257+0000 7fde6f113740 1 --2- 172.21.15.152:0/594247362 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad82f680 0x55d8ad831a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.257+0000 7fde6f113740 1 --2- 172.21.15.152:0/594247362 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad824dc0 0x55d8ad82f140 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.258+0000 7fde6f113740 1 --2- 172.21.15.152:0/594247362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad8244a0 0x55d8ad824880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.258+0000 7fde6f113740 1 -- 172.21.15.152:0/594247362 >> 172.21.15.152:0/594247362 conn(0x55d8ad820a30 msgr2=0x55d8ad820e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:18.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.262+0000 7fde6f113740 1 -- 172.21.15.152:0/594247362 shutdown_connections 2024-09-11T22:55:18.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.262+0000 7fde6f113740 1 -- 172.21.15.152:0/594247362 wait complete. 2024-09-11T22:55:18.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.263+0000 7fde6f113740 1 -- start start 2024-09-11T22:55:18.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.263+0000 7fde6f113740 1 Processor -- start 2024-09-11T22:55:18.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.263+0000 7fde6f113740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad8244a0 0x55d8ad82ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.264+0000 7fde6f113740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad824dc0 0x55d8ad82c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.264+0000 7fde6f113740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad82f680 0x55d8ad82c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.264+0000 7fde6f113740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d8ad8322a0 con 0x55d8ad8244a0 2024-09-11T22:55:18.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.264+0000 7fde6f113740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d8ad745050 con 0x55d8ad82f680 2024-09-11T22:55:18.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.264+0000 7fde6f113740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d8ad832420 con 0x55d8ad824dc0 2024-09-11T22:55:18.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.264+0000 7fde6f113740 10 client.0 _ll_register_callbacks cb 0x55d8ad785160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:18.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.264+0000 7fde6659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad8244a0 0x55d8ad82ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:18.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.264+0000 7fde6659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad8244a0 0x55d8ad82ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51432/0 (socket says 172.21.15.152:51432) 2024-09-11T22:55:18.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.264+0000 7fde6659c640 1 -- 172.21.15.152:0/1298560505 learned_addr learned my addr 172.21.15.152:0/1298560505 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:18.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.265+0000 7fde6659c640 1 -- 172.21.15.152:0/1298560505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad824dc0 msgr2=0x55d8ad82c1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:18.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.265+0000 7fde6659c640 1 --2- 172.21.15.152:0/1298560505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad824dc0 0x55d8ad82c1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.265+0000 7fde6659c640 1 -- 172.21.15.152:0/1298560505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad82f680 msgr2=0x55d8ad82c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:18.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.265+0000 7fde6659c640 1 --2- 172.21.15.152:0/1298560505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad82f680 0x55d8ad82c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.265+0000 7fde6659c640 1 -- 172.21.15.152:0/1298560505 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8ad4fb3c0 con 0x55d8ad8244a0 2024-09-11T22:55:18.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.265+0000 7fde6659c640 1 --2- 172.21.15.152:0/1298560505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad8244a0 0x55d8ad82ef10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fde58009610 tx=0x7fde5809cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9011a8b8b0b302 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:18.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.265+0000 7fde4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:18.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.265+0000 7fde4dffb640 1 -- 172.21.15.152:0/1298560505 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58020040 con 0x55d8ad8244a0 2024-09-11T22:55:18.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.265+0000 7fde6f113740 1 -- 172.21.15.152:0/1298560505 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde5c004300 con 0x55d8ad8244a0 2024-09-11T22:55:18.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.265+0000 7fde4dffb640 1 -- 172.21.15.152:0/1298560505 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fde58098c50 con 0x55d8ad8244a0 2024-09-11T22:55:18.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.266+0000 7fde4dffb640 1 -- 172.21.15.152:0/1298560505 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde580cca30 con 0x55d8ad8244a0 2024-09-11T22:55:18.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.268+0000 7fde6f113740 10 client.16149 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:18.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.268+0000 7fde6f113740 1 -- 172.21.15.152:0/1298560505 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8ad83dc90 con 0x55d8ad8244a0 2024-09-11T22:55:18.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.275+0000 7fde4dffb640 1 -- 172.21.15.152:0/1298560505 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde5c004300 con 0x55d8ad8244a0 2024-09-11T22:55:18.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.275+0000 7fde4dffb640 1 -- 172.21.15.152:0/1298560505 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fde5800fb40 con 0x55d8ad8244a0 2024-09-11T22:55:18.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.277+0000 7fde6f113740 10 client.16149 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:18.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.277+0000 7fde6f113740 10 client.16149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:18.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.277+0000 7fde6f113740 20 client.16149 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:18.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.277+0000 7fde6f113740 1 --2- 172.21.15.152:0/1298560505 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d8ad83f150 0x55d8ad8415e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.277+0000 7fde6f113740 4 client.16149 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:55:18.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.277+0000 7fde6f113740 1 -- 172.21.15.152:0/1298560505 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8ad797850 con 0x55d8ad83f150 2024-09-11T22:55:18.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.287+0000 7fde65d9b640 1 --2- 172.21.15.152:0/1298560505 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d8ad83f150 0x55d8ad8415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:18.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.303+0000 7fde65d9b640 1 --2- 172.21.15.152:0/1298560505 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d8ad83f150 0x55d8ad8415e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7733bd95ef31745 server_cookie=944a88850169a78 in_seq=0 out_seq=0 2024-09-11T22:55:18.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.303+0000 7fde4dffb640 10 client.16149 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:18.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.307+0000 7fde4dffb640 1 -- 172.21.15.152:0/1298560505 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8ad797850 con 0x55d8ad83f150 2024-09-11T22:55:18.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.307+0000 7fde4dffb640 10 client.16149 handle_command_reply: tid=0 2024-09-11T22:55:18.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.307+0000 7fde6f113740 1 client.16149 shutdown 2024-09-11T22:55:18.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.307+0000 7fde6f113740 1 -- 172.21.15.152:0/1298560505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad8244a0 msgr2=0x55d8ad82ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:18.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.307+0000 7fde6f113740 1 --2- 172.21.15.152:0/1298560505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad8244a0 0x55d8ad82ef10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fde58009610 tx=0x7fde5809cda0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.307+0000 7fde6f113740 1 -- 172.21.15.152:0/1298560505 shutdown_connections 2024-09-11T22:55:18.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.307+0000 7fde6f113740 1 --2- 172.21.15.152:0/1298560505 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d8ad83f150 0x55d8ad8415e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.307+0000 7fde6f113740 1 --2- 172.21.15.152:0/1298560505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad82f680 0x55d8ad82c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.307+0000 7fde6f113740 1 --2- 172.21.15.152:0/1298560505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad824dc0 0x55d8ad82c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.307+0000 7fde6f113740 1 --2- 172.21.15.152:0/1298560505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad8244a0 0x55d8ad82ef10 secure :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fde58009610 tx=0x7fde5809cda0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.307+0000 7fde6f113740 1 -- 172.21.15.152:0/1298560505 >> 172.21.15.152:0/1298560505 conn(0x55d8ad820a30 msgr2=0x55d8ad826760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:18.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.308+0000 7fde4dffb640 0 client.16149 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:18.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.310+0000 7fde6f113740 1 -- 172.21.15.152:0/1298560505 shutdown_connections 2024-09-11T22:55:18.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.311+0000 7fde6f113740 1 -- 172.21.15.152:0/1298560505 wait complete. 2024-09-11T22:55:18.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.311+0000 7fde6f113740 20 client.16149 trim_cache size 0 max 16384 2024-09-11T22:55:18.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.368+0000 7fde6f113740 1 Processor -- start 2024-09-11T22:55:18.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.368+0000 7fde6f113740 1 -- start start 2024-09-11T22:55:18.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.368+0000 7fde6f113740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad820a30 0x55d8ad81fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.369+0000 7fde6f113740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad824dc0 0x55d8ad846ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.369+0000 7fde6f113740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad82f680 0x55d8ad847430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.369+0000 7fde6f113740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fde48016f50 con 0x55d8ad820a30 2024-09-11T22:55:18.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.369+0000 7fde6f113740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fde48018a90 con 0x55d8ad82f680 2024-09-11T22:55:18.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.369+0000 7fde6f113740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fde48017620 con 0x55d8ad824dc0 2024-09-11T22:55:18.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.369+0000 7fde6659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad820a30 0x55d8ad81fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:18.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.369+0000 7fde6659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad820a30 0x55d8ad81fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:51442/0 (socket says 172.21.15.152:51442) 2024-09-11T22:55:18.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.369+0000 7fde6659c640 1 -- 172.21.15.152:0/31492281 learned_addr learned my addr 172.21.15.152:0/31492281 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:18.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.370+0000 7fde6659c640 1 -- 172.21.15.152:0/31492281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad824dc0 msgr2=0x55d8ad846ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:18.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.370+0000 7fde6659c640 1 --2- 172.21.15.152:0/31492281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad824dc0 0x55d8ad846ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.370+0000 7fde6659c640 1 -- 172.21.15.152:0/31492281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad82f680 msgr2=0x55d8ad847430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:55:18.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.370+0000 7fde6659c640 1 --2- 172.21.15.152:0/31492281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad82f680 0x55d8ad847430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.370+0000 7fde6659c640 1 -- 172.21.15.152:0/31492281 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8ad820860 con 0x55d8ad820a30 2024-09-11T22:55:18.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.370+0000 7fde6659c640 1 --2- 172.21.15.152:0/31492281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad820a30 0x55d8ad81fa80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fde5800ffb0 tx=0x7fde58005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31117a5f20acb217 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:18.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.372+0000 7fde4ffff640 1 -- 172.21.15.152:0/31492281 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde580b8020 con 0x55d8ad820a30 2024-09-11T22:55:18.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.372+0000 7fde4ffff640 1 -- 172.21.15.152:0/31492281 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde580ccbd0 con 0x55d8ad820a30 2024-09-11T22:55:18.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.372+0000 7fde4ffff640 1 -- 172.21.15.152:0/31492281 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58020070 con 0x55d8ad820a30 2024-09-11T22:55:18.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.378+0000 7fde6f113740 1 -- 172.21.15.152:0/31492281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad820a30 msgr2=0x55d8ad81fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:18.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.378+0000 7fde6f113740 1 --2- 172.21.15.152:0/31492281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad820a30 0x55d8ad81fa80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fde5800ffb0 tx=0x7fde58005bb0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.378+0000 7fde6f113740 1 -- 172.21.15.152:0/31492281 shutdown_connections 2024-09-11T22:55:18.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.378+0000 7fde6f113740 1 --2- 172.21.15.152:0/31492281 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad82f680 0x55d8ad847430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.378+0000 7fde6f113740 1 --2- 172.21.15.152:0/31492281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad824dc0 0x55d8ad846ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.379+0000 7fde6f113740 1 --2- 172.21.15.152:0/31492281 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad820a30 0x55d8ad81fa80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.379+0000 7fde6f113740 1 -- 172.21.15.152:0/31492281 >> 172.21.15.152:0/31492281 conn(0x55d8ad82c860 msgr2=0x55d8ad81f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:18.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.383+0000 7fde6f113740 1 -- 172.21.15.152:0/31492281 shutdown_connections 2024-09-11T22:55:18.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.383+0000 7fde6f113740 1 -- 172.21.15.152:0/31492281 wait complete. 2024-09-11T22:55:18.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.384+0000 7fde6f113740 1 -- start start 2024-09-11T22:55:18.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.385+0000 7fde6f113740 1 Processor -- start 2024-09-11T22:55:18.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.386+0000 7fde6f113740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad820a30 0x55d8ad837d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.387+0000 7fde6f113740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad824dc0 0x55d8ad835080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.387+0000 7fde6f113740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad82f680 0x55d8ad8355c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.387+0000 7fde6f113740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fde480183c0 con 0x55d8ad82f680 2024-09-11T22:55:18.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.387+0000 7fde6f113740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fde48017cf0 con 0x55d8ad820a30 2024-09-11T22:55:18.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.387+0000 7fde6f113740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fde48014f10 con 0x55d8ad824dc0 2024-09-11T22:55:18.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.387+0000 7fde6f113740 10 client.0 _ll_register_callbacks cb 0x7fde58098880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:18.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.389+0000 7fde66d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad82f680 0x55d8ad8355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:18.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.389+0000 7fde66d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad82f680 0x55d8ad8355c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38994/0 (socket says 172.21.15.152:38994) 2024-09-11T22:55:18.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.389+0000 7fde66d9d640 1 -- 172.21.15.152:0/1899109041 learned_addr learned my addr 172.21.15.152:0/1899109041 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:18.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.390+0000 7fde6659c640 1 --2- 172.21.15.152:0/1899109041 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad820a30 0x55d8ad837d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:18.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.390+0000 7fde65d9b640 1 --2- 172.21.15.152:0/1899109041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad824dc0 0x55d8ad835080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:18.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.390+0000 7fde66d9d640 1 -- 172.21.15.152:0/1899109041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad824dc0 msgr2=0x55d8ad835080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:18.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.390+0000 7fde66d9d640 1 --2- 172.21.15.152:0/1899109041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad824dc0 0x55d8ad835080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.390+0000 7fde66d9d640 1 -- 172.21.15.152:0/1899109041 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad820a30 msgr2=0x55d8ad837d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:18.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.390+0000 7fde66d9d640 1 --2- 172.21.15.152:0/1899109041 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad820a30 0x55d8ad837d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.390+0000 7fde66d9d640 1 -- 172.21.15.152:0/1899109041 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8ad835c60 con 0x55d8ad82f680 2024-09-11T22:55:18.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.391+0000 7fde66d9d640 1 --2- 172.21.15.152:0/1899109041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad82f680 0x55d8ad8355c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fde6000f800 tx=0x7fde6000fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3862262605e5361e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:18.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.391+0000 7fde4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:18.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.391+0000 7fde4dffb640 1 -- 172.21.15.152:0/1899109041 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60017070 con 0x55d8ad82f680 2024-09-11T22:55:18.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.391+0000 7fde6f113740 1 -- 172.21.15.152:0/1899109041 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d8ad791d60 con 0x55d8ad82f680 2024-09-11T22:55:18.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.392+0000 7fde4dffb640 1 -- 172.21.15.152:0/1899109041 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fde6000db70 con 0x55d8ad82f680 2024-09-11T22:55:18.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.392+0000 7fde4dffb640 1 -- 172.21.15.152:0/1899109041 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fde60023050 con 0x55d8ad82f680 2024-09-11T22:55:18.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.393+0000 7fde6f113740 10 client.16155 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:18.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.393+0000 7fde6f113740 1 -- 172.21.15.152:0/1899109041 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8ad8a9e00 con 0x55d8ad82f680 2024-09-11T22:55:18.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.394+0000 7fde4dffb640 1 -- 172.21.15.152:0/1899109041 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d8ad791d60 con 0x55d8ad82f680 2024-09-11T22:55:18.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.394+0000 7fde4dffb640 1 -- 172.21.15.152:0/1899109041 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fde60011040 con 0x55d8ad82f680 2024-09-11T22:55:18.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.394+0000 7fde6f113740 10 client.16155 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:18.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.394+0000 7fde6f113740 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:18.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.394+0000 7fde6f113740 20 client.16155 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:18.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.394+0000 7fde6f113740 1 --2- 172.21.15.152:0/1899109041 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d8ad8aa550 0x55d8ad8ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:18.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.395+0000 7fde6f113740 4 client.16155 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:55:18.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.395+0000 7fde6f113740 1 -- 172.21.15.152:0/1899109041 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d8ad70e210 con 0x55d8ad8aa550 2024-09-11T22:55:18.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.397+0000 7fde6659c640 1 --2- 172.21.15.152:0/1899109041 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d8ad8aa550 0x55d8ad8ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:18.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.409+0000 7fde6659c640 1 --2- 172.21.15.152:0/1899109041 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d8ad8aa550 0x55d8ad8ac970 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=becf7427d06adc7a server_cookie=ed02dcc079ce1665 in_seq=0 out_seq=0 2024-09-11T22:55:18.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.409+0000 7fde4dffb640 10 client.16155 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:18.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.411+0000 7fde4dffb640 1 -- 172.21.15.152:0/1899109041 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d8ad70e210 con 0x55d8ad8aa550 2024-09-11T22:55:18.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.411+0000 7fde4dffb640 10 client.16155 handle_command_reply: tid=0 2024-09-11T22:55:18.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.412+0000 7fde6f113740 1 client.16155 shutdown 2024-09-11T22:55:18.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.413+0000 7fde6f113740 1 -- 172.21.15.152:0/1899109041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad82f680 msgr2=0x55d8ad8355c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:18.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.413+0000 7fde6f113740 1 --2- 172.21.15.152:0/1899109041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad82f680 0x55d8ad8355c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fde6000f800 tx=0x7fde6000fcc0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.416+0000 7fde6f113740 1 -- 172.21.15.152:0/1899109041 shutdown_connections 2024-09-11T22:55:18.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.416+0000 7fde6f113740 1 --2- 172.21.15.152:0/1899109041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8ad82f680 0x55d8ad8355c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.416+0000 7fde6f113740 1 --2- 172.21.15.152:0/1899109041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8ad824dc0 0x55d8ad835080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.416+0000 7fde6f113740 1 --2- 172.21.15.152:0/1899109041 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d8ad8aa550 0x55d8ad8ac970 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.416+0000 7fde6f113740 1 --2- 172.21.15.152:0/1899109041 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8ad820a30 0x55d8ad837d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.416+0000 7fde6f113740 1 -- 172.21.15.152:0/1899109041 >> 172.21.15.152:0/1899109041 conn(0x55d8ad82c860 msgr2=0x55d8ad81f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:18.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.416+0000 7fde4dffb640 0 client.16155 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:18.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.419+0000 7fde6f113740 1 -- 172.21.15.152:0/1899109041 shutdown_connections 2024-09-11T22:55:18.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.421+0000 7fde6f113740 1 -- 172.21.15.152:0/1899109041 wait complete. 2024-09-11T22:55:18.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.421+0000 7fde6f113740 20 client.16155 trim_cache size 0 max 16384 2024-09-11T22:55:18.428 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:55:18.428 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T22:55:18.428 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T22:55:18.428 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:55:18.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.432+0000 7fde6cd3a640 1 -- 172.21.15.152:0/1565666528 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fde500a2f00 msgr2=0x7fde500a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:18.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.432+0000 7fde6cd3a640 1 --2- 172.21.15.152:0/1565666528 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fde500a2f00 0x7fde500a53c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fde680fec10 tx=0x7fde5c005d50 comp rx=0 tx=0).stop 2024-09-11T22:55:18.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.432+0000 7fde6cd3a640 1 -- 172.21.15.152:0/1565666528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde68100e20 msgr2=0x7fde681a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:18.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.432+0000 7fde6cd3a640 1 --2- 172.21.15.152:0/1565666528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde68100e20 0x7fde681a03f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fde5800d950 tx=0x7fde5800de10 comp rx=0 tx=0).stop 2024-09-11T22:55:18.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.436+0000 7fde6cd3a640 1 -- 172.21.15.152:0/1565666528 shutdown_connections 2024-09-11T22:55:18.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.436+0000 7fde6cd3a640 1 --2- 172.21.15.152:0/1565666528 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fde500a2f00 0x7fde500a53c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.436+0000 7fde6cd3a640 1 --2- 172.21.15.152:0/1565666528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fde68102ef0 0x7fde6819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.436+0000 7fde6cd3a640 1 --2- 172.21.15.152:0/1565666528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fde68102570 0x7fde681a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.436+0000 7fde6cd3a640 1 --2- 172.21.15.152:0/1565666528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde68100e20 0x7fde681a03f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:18.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.436+0000 7fde6cd3a640 1 -- 172.21.15.152:0/1565666528 >> 172.21.15.152:0/1565666528 conn(0x7fde680fc7d0 msgr2=0x7fde68104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:18.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.436+0000 7fde6cd3a640 1 -- 172.21.15.152:0/1565666528 shutdown_connections 2024-09-11T22:55:18.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:18.436+0000 7fde6cd3a640 1 -- 172.21.15.152:0/1565666528 wait complete. 2024-09-11T22:55:18.470 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:18 smithi152 ceph-mon[31181]: pgmap v75: 97 pgs: 97 active+clean; 13 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 228 MiB/s wr, 169 op/s 2024-09-11T22:55:18.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:18 smithi191 ceph-mon[35211]: pgmap v75: 97 pgs: 97 active+clean; 13 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 228 MiB/s wr, 169 op/s 2024-09-11T22:55:18.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:18 smithi204 ceph-mon[35454]: pgmap v75: 97 pgs: 97 active+clean; 13 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 228 MiB/s wr, 169 op/s 2024-09-11T22:55:19.429 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:55:19.429 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:55:19.986 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:55:20.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:20 smithi152 ceph-mon[31181]: pgmap v76: 97 pgs: 97 active+clean; 15 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 440 MiB/s wr, 316 op/s 2024-09-11T22:55:20.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:20 smithi191 ceph-mon[35211]: pgmap v76: 97 pgs: 97 active+clean; 15 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 440 MiB/s wr, 316 op/s 2024-09-11T22:55:20.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:20 smithi204 ceph-mon[35454]: pgmap v76: 97 pgs: 97 active+clean; 15 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 440 MiB/s wr, 316 op/s 2024-09-11T22:55:20.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.559+0000 7f15d9b70640 1 -- 172.21.15.152:0/2017003526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d4102570 msgr2=0x7f15d41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:20.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.559+0000 7f15d9b70640 1 --2- 172.21.15.152:0/2017003526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d4102570 0x7f15d41029b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009900 tx=0x7f15c802a4a0 comp rx=0 tx=0).stop 2024-09-11T22:55:20.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.559+0000 7f15d9b70640 1 -- 172.21.15.152:0/2017003526 shutdown_connections 2024-09-11T22:55:20.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.559+0000 7f15d9b70640 1 --2- 172.21.15.152:0/2017003526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15d4102ef0 0x7f15d410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:20.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.559+0000 7f15d9b70640 1 --2- 172.21.15.152:0/2017003526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d4102570 0x7f15d41029b0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:20.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.559+0000 7f15d9b70640 1 --2- 172.21.15.152:0/2017003526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15d4100e20 0x7f15d4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:20.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.559+0000 7f15d9b70640 1 -- 172.21.15.152:0/2017003526 >> 172.21.15.152:0/2017003526 conn(0x7f15d40fc7d0 msgr2=0x7f15d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:20.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.560+0000 7f15d9b70640 1 -- 172.21.15.152:0/2017003526 shutdown_connections 2024-09-11T22:55:20.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.560+0000 7f15d9b70640 1 -- 172.21.15.152:0/2017003526 wait complete. 2024-09-11T22:55:20.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.560+0000 7f15d9b70640 1 Processor -- start 2024-09-11T22:55:20.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.561+0000 7f15d9b70640 1 -- start start 2024-09-11T22:55:20.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.561+0000 7f15d9b70640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d4100e20 0x7f15d41975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:20.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.561+0000 7f15d9b70640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15d4102570 0x7f15d4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:20.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.561+0000 7f15d9b70640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15d4102ef0 0x7f15d419be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:20.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.561+0000 7f15d9b70640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4111040 con 0x7f15d4100e20 2024-09-11T22:55:20.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.561+0000 7f15d9b70640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4110ec0 con 0x7f15d4102ef0 2024-09-11T22:55:20.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.561+0000 7f15d9b70640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f15d41111c0 con 0x7f15d4102570 2024-09-11T22:55:20.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.561+0000 7f15d37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d4100e20 0x7f15d41975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:20.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.561+0000 7f15d37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d4100e20 0x7f15d41975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39008/0 (socket says 172.21.15.152:39008) 2024-09-11T22:55:20.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.561+0000 7f15d37fe640 1 -- 172.21.15.152:0/1332594002 learned_addr learned my addr 172.21.15.152:0/1332594002 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:20.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.562+0000 7f15d37fe640 1 -- 172.21.15.152:0/1332594002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15d4102570 msgr2=0x7f15d4197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:20.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.562+0000 7f15d37fe640 1 --2- 172.21.15.152:0/1332594002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15d4102570 0x7f15d4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:20.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.562+0000 7f15d3fff640 1 --2- 172.21.15.152:0/1332594002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15d4102ef0 0x7f15d419be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:20.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.562+0000 7f15d37fe640 1 -- 172.21.15.152:0/1332594002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15d4102ef0 msgr2=0x7f15d419be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:20.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.562+0000 7f15d37fe640 1 --2- 172.21.15.152:0/1332594002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15d4102ef0 0x7f15d419be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:20.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.562+0000 7f15d37fe640 1 -- 172.21.15.152:0/1332594002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d419c550 con 0x7f15d4100e20 2024-09-11T22:55:20.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.562+0000 7f15d3fff640 1 --2- 172.21.15.152:0/1332594002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15d4102ef0 0x7f15d419be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:55:20.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.562+0000 7f15d37fe640 1 --2- 172.21.15.152:0/1332594002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d4100e20 0x7f15d41975a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f15c400b4a0 tx=0x7f15c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:20.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.562+0000 7f15d0ff9640 1 -- 172.21.15.152:0/1332594002 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4014070 con 0x7f15d4100e20 2024-09-11T22:55:20.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.562+0000 7f15d0ff9640 1 -- 172.21.15.152:0/1332594002 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c4010af0 con 0x7f15d4100e20 2024-09-11T22:55:20.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.562+0000 7f15d9b70640 1 -- 172.21.15.152:0/1332594002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d419c840 con 0x7f15d4100e20 2024-09-11T22:55:20.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.562+0000 7f15d0ff9640 1 -- 172.21.15.152:0/1332594002 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4017d10 con 0x7f15d4100e20 2024-09-11T22:55:20.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.562+0000 7f15d9b70640 1 -- 172.21.15.152:0/1332594002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d41a4120 con 0x7f15d4100e20 2024-09-11T22:55:20.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.564+0000 7f15d9b70640 1 -- 172.21.15.152:0/1332594002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d4101280 con 0x7f15d4100e20 2024-09-11T22:55:20.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.565+0000 7f15d0ff9640 1 -- 172.21.15.152:0/1332594002 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f15c4017070 con 0x7f15d4100e20 2024-09-11T22:55:20.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.566+0000 7f15d0ff9640 1 --2- 172.21.15.152:0/1332594002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f15bc0a2cf0 0x7f15bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:20.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.566+0000 7f15d0ff9640 1 -- 172.21.15.152:0/1332594002 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f15c40ce7a0 con 0x7f15d4100e20 2024-09-11T22:55:20.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.566+0000 7f15d2ffd640 1 --2- 172.21.15.152:0/1332594002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f15bc0a2cf0 0x7f15bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:20.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.567+0000 7f15d2ffd640 1 --2- 172.21.15.152:0/1332594002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f15bc0a2cf0 0x7f15bc0a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f15c80096c0 tx=0x7f15c8000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:20.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.570+0000 7f15d0ff9640 1 -- 172.21.15.152:0/1332594002 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f15c409b520 con 0x7f15d4100e20 2024-09-11T22:55:20.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.864+0000 7f15d9b70640 1 -- 172.21.15.152:0/1332594002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15d4198900 con 0x7f15d4100e20 2024-09-11T22:55:20.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.864+0000 7f15d0ff9640 1 -- 172.21.15.152:0/1332594002 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f15c409ae20 con 0x7f15d4100e20 2024-09-11T22:55:20.865 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:55:20.866 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:55:20.866 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:55:20.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.867+0000 7f15d9b70640 1 -- 172.21.15.152:0/1332594002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f15bc0a2cf0 msgr2=0x7f15bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:20.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.867+0000 7f15d9b70640 1 --2- 172.21.15.152:0/1332594002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f15bc0a2cf0 0x7f15bc0a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f15c80096c0 tx=0x7f15c8000bc0 comp rx=0 tx=0).stop 2024-09-11T22:55:20.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.867+0000 7f15d9b70640 1 -- 172.21.15.152:0/1332594002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d4100e20 msgr2=0x7f15d41975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:20.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.867+0000 7f15d9b70640 1 --2- 172.21.15.152:0/1332594002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d4100e20 0x7f15d41975a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f15c400b4a0 tx=0x7f15c400b960 comp rx=0 tx=0).stop 2024-09-11T22:55:20.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.867+0000 7f15d9b70640 1 -- 172.21.15.152:0/1332594002 shutdown_connections 2024-09-11T22:55:20.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.867+0000 7f15d9b70640 1 --2- 172.21.15.152:0/1332594002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f15bc0a2cf0 0x7f15bc0a51b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:20.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.867+0000 7f15d9b70640 1 --2- 172.21.15.152:0/1332594002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15d4102ef0 0x7f15d419be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:20.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.867+0000 7f15d9b70640 1 --2- 172.21.15.152:0/1332594002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15d4102570 0x7f15d4197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.867+0000 7f15d9b70640 1 --2- 172.21.15.152:0/1332594002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15d4100e20 0x7f15d41975a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.867+0000 7f15d9b70640 1 -- 172.21.15.152:0/1332594002 >> 172.21.15.152:0/1332594002 conn(0x7f15d40fc7d0 msgr2=0x7f15d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.867+0000 7f15d9b70640 1 -- 172.21.15.152:0/1332594002 shutdown_connections 2024-09-11T22:55:20.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:20.868+0000 7f15d9b70640 1 -- 172.21.15.152:0/1332594002 wait complete. 2024-09-11T22:55:21.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:21 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1332594002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:55:21.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:21 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1332594002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:55:21.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:21 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1332594002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:55:21.685 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:55:22.059 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:55:22.520 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:22 smithi152 ceph-mon[31181]: pgmap v77: 97 pgs: 97 active+clean; 18 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 640 MiB/s wr, 363 op/s 2024-09-11T22:55:22.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:22 smithi191 ceph-mon[35211]: pgmap v77: 97 pgs: 97 active+clean; 18 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 640 MiB/s wr, 363 op/s 2024-09-11T22:55:22.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:22 smithi204 ceph-mon[35454]: pgmap v77: 97 pgs: 97 active+clean; 18 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 640 MiB/s wr, 363 op/s 2024-09-11T22:55:22.548 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:55:22.549 INFO:tasks.workunit.client.0.smithi152.stdout: Children see throughput for 1 rewriters = 845107.62 kB/sec 2024-09-11T22:55:22.549 INFO:tasks.workunit.client.0.smithi152.stdout: Parent sees throughput for 1 rewriters = 845089.98 kB/sec 2024-09-11T22:55:22.549 INFO:tasks.workunit.client.0.smithi152.stdout: Min throughput per process = 845107.62 kB/sec 2024-09-11T22:55:22.549 INFO:tasks.workunit.client.0.smithi152.stdout: Max throughput per process = 845107.62 kB/sec 2024-09-11T22:55:22.549 INFO:tasks.workunit.client.0.smithi152.stdout: Avg throughput per process = 845107.62 kB/sec 2024-09-11T22:55:22.549 INFO:tasks.workunit.client.0.smithi152.stdout: Min xfer = 10485760.00 kB 2024-09-11T22:55:24.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.682+0000 7f1be956c640 1 -- 172.21.15.152:0/1116634706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be4102570 msgr2=0x7f1be41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:24.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.682+0000 7f1be956c640 1 --2- 172.21.15.152:0/1116634706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be4102570 0x7f1be41029b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009900 tx=0x7f1bcc02a480 comp rx=0 tx=0).stop 2024-09-11T22:55:24.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.682+0000 7f1be956c640 1 -- 172.21.15.152:0/1116634706 shutdown_connections 2024-09-11T22:55:24.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.682+0000 7f1be956c640 1 --2- 172.21.15.152:0/1116634706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1be4102ef0 0x7f1be410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.682+0000 7f1be956c640 1 --2- 172.21.15.152:0/1116634706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be4102570 0x7f1be41029b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.682+0000 7f1be956c640 1 --2- 172.21.15.152:0/1116634706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1be4100e20 0x7f1be4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.682+0000 7f1be956c640 1 -- 172.21.15.152:0/1116634706 >> 172.21.15.152:0/1116634706 conn(0x7f1be40fc7d0 msgr2=0x7f1be40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:24.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.682+0000 7f1be956c640 1 -- 172.21.15.152:0/1116634706 shutdown_connections 2024-09-11T22:55:24.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.682+0000 7f1be956c640 1 -- 172.21.15.152:0/1116634706 wait complete. 2024-09-11T22:55:24.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.682+0000 7f1be956c640 1 Processor -- start 2024-09-11T22:55:24.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be956c640 1 -- start start 2024-09-11T22:55:24.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be956c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1be4100e20 0x7f1be41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be956c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be4102570 0x7f1be4198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be956c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1be4102ef0 0x7f1be419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be956c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1be41120a0 con 0x7f1be4102570 2024-09-11T22:55:24.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be956c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1be4111f20 con 0x7f1be4100e20 2024-09-11T22:55:24.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be956c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1be4112220 con 0x7f1be4102ef0 2024-09-11T22:55:24.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be27fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be4102570 0x7f1be4198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be2ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1be4100e20 0x7f1be41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be27fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be4102570 0x7f1be4198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39034/0 (socket says 172.21.15.152:39034) 2024-09-11T22:55:24.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be2ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1be4100e20 0x7f1be41984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:33718/0 (socket says 172.21.15.152:33718) 2024-09-11T22:55:24.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be27fc640 1 -- 172.21.15.152:0/49949602 learned_addr learned my addr 172.21.15.152:0/49949602 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:24.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be37fe640 1 --2- 172.21.15.152:0/49949602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1be4102ef0 0x7f1be419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be27fc640 1 -- 172.21.15.152:0/49949602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1be4102ef0 msgr2=0x7f1be419cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:24.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be27fc640 1 --2- 172.21.15.152:0/49949602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1be4102ef0 0x7f1be419cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be27fc640 1 -- 172.21.15.152:0/49949602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1be4100e20 msgr2=0x7f1be41984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:24.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be27fc640 1 --2- 172.21.15.152:0/49949602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1be4100e20 0x7f1be41984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be27fc640 1 -- 172.21.15.152:0/49949602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be419d490 con 0x7f1be4102570 2024-09-11T22:55:24.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be2ffd640 1 --2- 172.21.15.152:0/49949602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1be4100e20 0x7f1be41984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:55:24.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1be27fc640 1 --2- 172.21.15.152:0/49949602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be4102570 0x7f1be4198a20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc006010 tx=0x7f1bcc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:24.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.683+0000 7f1bdbfff640 1 -- 172.21.15.152:0/49949602 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc03f070 con 0x7f1be4102570 2024-09-11T22:55:24.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.684+0000 7f1bdbfff640 1 -- 172.21.15.152:0/49949602 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bcc031c30 con 0x7f1be4102570 2024-09-11T22:55:24.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.684+0000 7f1be956c640 1 -- 172.21.15.152:0/49949602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be419d720 con 0x7f1be4102570 2024-09-11T22:55:24.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.684+0000 7f1bdbfff640 1 -- 172.21.15.152:0/49949602 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc035030 con 0x7f1be4102570 2024-09-11T22:55:24.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.684+0000 7f1be956c640 1 -- 172.21.15.152:0/49949602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be41a5060 con 0x7f1be4102570 2024-09-11T22:55:24.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.686+0000 7f1bdbfff640 1 -- 172.21.15.152:0/49949602 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1bcc042070 con 0x7f1be4102570 2024-09-11T22:55:24.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.686+0000 7f1beb945740 1 Processor -- start 2024-09-11T22:55:24.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.686+0000 7f1beb945740 1 -- start start 2024-09-11T22:55:24.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.686+0000 7f1beb945740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067e8b40 0x55a5067e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1beb945740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 0x55a5067e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1beb945740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1bdbfff640 1 --2- 172.21.15.152:0/49949602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1bd00a2cf0 0x7f1bd00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1beb945740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a50674d2f0 con 0x55a5067f2370 2024-09-11T22:55:24.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1beb945740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a5066f4cf0 con 0x55a5067e75d0 2024-09-11T22:55:24.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1beb945740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a5067492b0 con 0x55a5067e8b40 2024-09-11T22:55:24.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1be27fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1be37fe640 1 --2- 172.21.15.152:0/49949602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1bd00a2cf0 0x7f1bd00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1be2ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 0x55a5067e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1be27fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067f4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39038/0 (socket says 172.21.15.152:39038) 2024-09-11T22:55:24.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1be27fc640 1 -- 172.21.15.152:0/2810802948 learned_addr learned my addr 172.21.15.152:0/2810802948 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:24.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1bdbfff640 1 -- 172.21.15.152:0/49949602 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f1bcc0efbb0 con 0x7f1be4102570 2024-09-11T22:55:24.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1be37fe640 1 --2- 172.21.15.152:0/2810802948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067e8b40 0x55a5067e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1be2ffd640 1 -- 172.21.15.152:0/2810802948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067e8b40 msgr2=0x55a5067e7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:24.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1be2ffd640 1 --2- 172.21.15.152:0/2810802948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067e8b40 0x55a5067e7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1be2ffd640 1 -- 172.21.15.152:0/2810802948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 msgr2=0x55a5067f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:24.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1be2ffd640 1 --2- 172.21.15.152:0/2810802948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1be2ffd640 1 -- 172.21.15.152:0/2810802948 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5066d93a0 con 0x55a5067e75d0 2024-09-11T22:55:24.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1be37fe640 1 --2- 172.21.15.152:0/49949602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1bd00a2cf0 0x7f1bd00a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc005fd0 tx=0x7f1bdc009b50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:24.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.687+0000 7f1be2ffd640 1 --2- 172.21.15.152:0/2810802948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 0x55a5067e7a10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40103d0 tx=0x7f1bd4010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea2dcc44a0c7dba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:24.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.688+0000 7f1bd97fa640 1 -- 172.21.15.152:0/2810802948 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4004220 con 0x55a5067e75d0 2024-09-11T22:55:24.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.688+0000 7f1bd97fa640 1 -- 172.21.15.152:0/2810802948 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd40043c0 con 0x55a5067e75d0 2024-09-11T22:55:24.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.688+0000 7f1bd97fa640 1 -- 172.21.15.152:0/2810802948 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4011a60 con 0x55a5067e75d0 2024-09-11T22:55:24.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.688+0000 7f1beb945740 1 -- 172.21.15.152:0/2810802948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 msgr2=0x55a5067e7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:24.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.688+0000 7f1beb945740 1 --2- 172.21.15.152:0/2810802948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 0x55a5067e7a10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40103d0 tx=0x7f1bd4010890 comp rx=0 tx=0).stop 2024-09-11T22:55:24.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.688+0000 7f1beb945740 1 -- 172.21.15.152:0/2810802948 shutdown_connections 2024-09-11T22:55:24.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.688+0000 7f1beb945740 1 --2- 172.21.15.152:0/2810802948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.688+0000 7f1beb945740 1 --2- 172.21.15.152:0/2810802948 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 0x55a5067e7a10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.688+0000 7f1beb945740 1 --2- 172.21.15.152:0/2810802948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067e8b40 0x55a5067e7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.689+0000 7f1beb945740 1 -- 172.21.15.152:0/2810802948 >> 172.21.15.152:0/2810802948 conn(0x55a5067b0310 msgr2=0x55a5067e3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:24.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.689+0000 7f1beb945740 1 -- 172.21.15.152:0/2810802948 shutdown_connections 2024-09-11T22:55:24.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.689+0000 7f1beb945740 1 -- 172.21.15.152:0/2810802948 wait complete. 2024-09-11T22:55:24.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.689+0000 7f1beb945740 1 -- start start 2024-09-11T22:55:24.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.689+0000 7f1beb945740 1 Processor -- start 2024-09-11T22:55:24.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.689+0000 7f1beb945740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067e75d0 0x55a5067f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.689+0000 7f1beb945740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e8b40 0x55a5067eef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.689+0000 7f1beb945740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067f2370 0x55a5067ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.689+0000 7f1beb945740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a5067f9480 con 0x55a5067e75d0 2024-09-11T22:55:24.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.689+0000 7f1beb945740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a506708060 con 0x55a5067e8b40 2024-09-11T22:55:24.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.689+0000 7f1beb945740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a5067f9600 con 0x55a5067f2370 2024-09-11T22:55:24.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.689+0000 7f1beb945740 10 client.0 _ll_register_callbacks cb 0x55a50645b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:24.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.689+0000 7f1be2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067e75d0 0x55a5067f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1be2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067e75d0 0x55a5067f1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39040/0 (socket says 172.21.15.152:39040) 2024-09-11T22:55:24.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1be2ffd640 1 -- 172.21.15.152:0/2833256176 learned_addr learned my addr 172.21.15.152:0/2833256176 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:24.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1be27fc640 1 --2- 172.21.15.152:0/2833256176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e8b40 0x55a5067eef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1be37fe640 1 --2- 172.21.15.152:0/2833256176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067f2370 0x55a5067ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1be2ffd640 1 -- 172.21.15.152:0/2833256176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067f2370 msgr2=0x55a5067ef4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:24.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1be2ffd640 1 --2- 172.21.15.152:0/2833256176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067f2370 0x55a5067ef4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1be2ffd640 1 -- 172.21.15.152:0/2833256176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e8b40 msgr2=0x55a5067eef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:24.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1be2ffd640 1 --2- 172.21.15.152:0/2833256176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e8b40 0x55a5067eef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1be2ffd640 1 -- 172.21.15.152:0/2833256176 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5064be3c0 con 0x55a5067e75d0 2024-09-11T22:55:24.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1be27fc640 1 --2- 172.21.15.152:0/2833256176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e8b40 0x55a5067eef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:24.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1be2ffd640 1 --2- 172.21.15.152:0/2833256176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067e75d0 0x55a5067f1ce0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40116a0 tx=0x7f1bd4005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c12ce361d2c2ad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:24.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1bca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:24.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1bca7fc640 1 -- 172.21.15.152:0/2833256176 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd402a060 con 0x55a5067e75d0 2024-09-11T22:55:24.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1bca7fc640 1 -- 172.21.15.152:0/2833256176 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f1bd40117e0 con 0x55a5067e75d0 2024-09-11T22:55:24.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1beb945740 1 -- 172.21.15.152:0/2833256176 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd40043c0 con 0x55a5067e75d0 2024-09-11T22:55:24.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1bca7fc640 1 -- 172.21.15.152:0/2833256176 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1bd40119e0 con 0x55a5067e75d0 2024-09-11T22:55:24.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1beb945740 10 client.16173 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:24.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1beb945740 1 -- 172.21.15.152:0/2833256176 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a506804e90 con 0x55a5067e75d0 2024-09-11T22:55:24.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.690+0000 7f1bca7fc640 1 -- 172.21.15.152:0/2833256176 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd40043c0 con 0x55a5067e75d0 2024-09-11T22:55:24.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.691+0000 7f1bca7fc640 1 -- 172.21.15.152:0/2833256176 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1bd401b430 con 0x55a5067e75d0 2024-09-11T22:55:24.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.691+0000 7f1beb945740 10 client.16173 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:24.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.691+0000 7f1beb945740 10 client.16173 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:55:24.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.691+0000 7f1beb945740 20 client.16173 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:24.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.691+0000 7f1beb945740 1 --2- 172.21.15.152:0/2833256176 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55a506806290 0x55a506808680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.691+0000 7f1beb945740 4 client.16173 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:55:24.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.691+0000 7f1beb945740 1 -- 172.21.15.152:0/2833256176 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a50675a860 con 0x55a506806290 2024-09-11T22:55:24.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.691+0000 7f1be27fc640 1 --2- 172.21.15.152:0/2833256176 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55a506806290 0x55a506808680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.691+0000 7f1be27fc640 1 --2- 172.21.15.152:0/2833256176 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55a506806290 0x55a506808680 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=89816c8079f1773c server_cookie=a7343b350f67f505 in_seq=0 out_seq=0 2024-09-11T22:55:24.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.691+0000 7f1bca7fc640 10 client.16173 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:55:24.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.692+0000 7f1bca7fc640 1 -- 172.21.15.152:0/2833256176 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a50675a860 con 0x55a506806290 2024-09-11T22:55:24.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.692+0000 7f1bca7fc640 10 client.16173 handle_command_reply: tid=0 2024-09-11T22:55:24.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.692+0000 7f1beb945740 1 client.16173 shutdown 2024-09-11T22:55:24.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.692+0000 7f1beb945740 1 -- 172.21.15.152:0/2833256176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067e75d0 msgr2=0x55a5067f1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:24.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.692+0000 7f1beb945740 1 --2- 172.21.15.152:0/2833256176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067e75d0 0x55a5067f1ce0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40116a0 tx=0x7f1bd4005270 comp rx=0 tx=0).stop 2024-09-11T22:55:24.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.692+0000 7f1beb945740 1 -- 172.21.15.152:0/2833256176 shutdown_connections 2024-09-11T22:55:24.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.692+0000 7f1beb945740 1 --2- 172.21.15.152:0/2833256176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067f2370 0x55a5067ef4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.692+0000 7f1beb945740 1 --2- 172.21.15.152:0/2833256176 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55a506806290 0x55a506808680 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.692+0000 7f1beb945740 1 --2- 172.21.15.152:0/2833256176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e8b40 0x55a5067eef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.692+0000 7f1beb945740 1 --2- 172.21.15.152:0/2833256176 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067e75d0 0x55a5067f1ce0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.692+0000 7f1bca7fc640 0 client.16173 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:55:24.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.692+0000 7f1beb945740 1 -- 172.21.15.152:0/2833256176 >> 172.21.15.152:0/2833256176 conn(0x55a5067b0310 msgr2=0x55a5067e9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:24.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.692+0000 7f1beb945740 1 -- 172.21.15.152:0/2833256176 shutdown_connections 2024-09-11T22:55:24.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.692+0000 7f1beb945740 1 -- 172.21.15.152:0/2833256176 wait complete. 2024-09-11T22:55:24.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.692+0000 7f1beb945740 20 client.16173 trim_cache size 0 max 16384 2024-09-11T22:55:24.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.705+0000 7f1beb945740 1 Processor -- start 2024-09-11T22:55:24.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.705+0000 7f1beb945740 1 -- start start 2024-09-11T22:55:24.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.705+0000 7f1beb945740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 0x55a5067f0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.705+0000 7f1beb945740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067e8b40 0x55a5067ea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.705+0000 7f1beb945740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067ea660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.705+0000 7f1beb945740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1be41ab570 con 0x55a5067f2370 2024-09-11T22:55:24.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.705+0000 7f1beb945740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1be41ad0b0 con 0x55a5067e75d0 2024-09-11T22:55:24.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.705+0000 7f1beb945740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1be41abc40 con 0x55a5067e8b40 2024-09-11T22:55:24.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.705+0000 7f1be37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067ea660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.705+0000 7f1be37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067ea660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39052/0 (socket says 172.21.15.152:39052) 2024-09-11T22:55:24.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.705+0000 7f1be37fe640 1 -- 172.21.15.152:0/2992449618 learned_addr learned my addr 172.21.15.152:0/2992449618 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:24.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.705+0000 7f1be2ffd640 1 --2- 172.21.15.152:0/2992449618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 0x55a5067f0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.706+0000 7f1be27fc640 1 --2- 172.21.15.152:0/2992449618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067e8b40 0x55a5067ea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.706+0000 7f1be2ffd640 1 -- 172.21.15.152:0/2992449618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067e8b40 msgr2=0x55a5067ea120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:24.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.706+0000 7f1be2ffd640 1 --2- 172.21.15.152:0/2992449618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067e8b40 0x55a5067ea120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.706+0000 7f1be2ffd640 1 -- 172.21.15.152:0/2992449618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 msgr2=0x55a5067ea660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:24.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.706+0000 7f1be2ffd640 1 --2- 172.21.15.152:0/2992449618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067ea660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.706+0000 7f1be2ffd640 1 -- 172.21.15.152:0/2992449618 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5067f1900 con 0x55a5067e75d0 2024-09-11T22:55:24.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.706+0000 7f1be37fe640 1 --2- 172.21.15.152:0/2992449618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067ea660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:55:24.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.706+0000 7f1be2ffd640 1 --2- 172.21.15.152:0/2992449618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 0x55a5067f0f00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1bd401b970 tx=0x7f1bd4010dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e9114a45381ef1f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:24.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.706+0000 7f1bd8ff9640 1 -- 172.21.15.152:0/2992449618 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd402a030 con 0x55a5067e75d0 2024-09-11T22:55:24.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.706+0000 7f1bd8ff9640 1 -- 172.21.15.152:0/2992449618 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd4005850 con 0x55a5067e75d0 2024-09-11T22:55:24.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.706+0000 7f1bd8ff9640 1 -- 172.21.15.152:0/2992449618 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4011b80 con 0x55a5067e75d0 2024-09-11T22:55:24.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.707+0000 7f1beb945740 1 -- 172.21.15.152:0/2992449618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 msgr2=0x55a5067f0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:24.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.707+0000 7f1beb945740 1 --2- 172.21.15.152:0/2992449618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 0x55a5067f0f00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1bd401b970 tx=0x7f1bd4010dd0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.707+0000 7f1beb945740 1 -- 172.21.15.152:0/2992449618 shutdown_connections 2024-09-11T22:55:24.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.707+0000 7f1beb945740 1 --2- 172.21.15.152:0/2992449618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067ea660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.707+0000 7f1beb945740 1 --2- 172.21.15.152:0/2992449618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067e8b40 0x55a5067ea120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.707+0000 7f1beb945740 1 --2- 172.21.15.152:0/2992449618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 0x55a5067f0f00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.707+0000 7f1beb945740 1 -- 172.21.15.152:0/2992449618 >> 172.21.15.152:0/2992449618 conn(0x55a5067b0310 msgr2=0x55a5067f07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:24.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.707+0000 7f1beb945740 1 -- 172.21.15.152:0/2992449618 shutdown_connections 2024-09-11T22:55:24.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.707+0000 7f1beb945740 1 -- 172.21.15.152:0/2992449618 wait complete. 2024-09-11T22:55:24.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.707+0000 7f1beb945740 1 -- start start 2024-09-11T22:55:24.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.707+0000 7f1beb945740 1 Processor -- start 2024-09-11T22:55:24.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1beb945740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 0x55a5067ff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1beb945740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067e8b40 0x55a5067ebe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1beb945740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067ec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1beb945740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1be41ac9e0 con 0x55a5067f2370 2024-09-11T22:55:24.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1beb945740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1be41ac310 con 0x55a5067e75d0 2024-09-11T22:55:24.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1beb945740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1be41a9570 con 0x55a5067e8b40 2024-09-11T22:55:24.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1beb945740 10 client.0 _ll_register_callbacks cb 0x7f1bcc005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:24.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1be37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1be2ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 0x55a5067ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1be27fc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067e8b40 0x55a5067ebe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1be37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067ec430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39064/0 (socket says 172.21.15.152:39064) 2024-09-11T22:55:24.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1be2ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 0x55a5067ff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:33770/0 (socket says 172.21.15.152:33770) 2024-09-11T22:55:24.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1be37fe640 1 -- 172.21.15.152:0/3589752893 learned_addr learned my addr 172.21.15.152:0/3589752893 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:24.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1be2ffd640 1 -- 172.21.15.152:0/3589752893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067e8b40 msgr2=0x55a5067ebe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:24.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1be2ffd640 1 --2- 172.21.15.152:0/3589752893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067e8b40 0x55a5067ebe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1be2ffd640 1 -- 172.21.15.152:0/3589752893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 msgr2=0x55a5067ec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:24.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1be2ffd640 1 --2- 172.21.15.152:0/3589752893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067ec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1be2ffd640 1 -- 172.21.15.152:0/3589752893 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5067ecc00 con 0x55a5067e75d0 2024-09-11T22:55:24.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1be37fe640 1 --2- 172.21.15.152:0/3589752893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:55:24.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1be27fc640 1 --2- 172.21.15.152:0/3589752893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067e8b40 0x55a5067ebe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:55:24.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1be2ffd640 1 --2- 172.21.15.152:0/3589752893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 0x55a5067ff080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4000c00 tx=0x7f1bd401b6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=488edc1c13ef451e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:24.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1bca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-11T22:55:24.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1bca7fc640 1 -- 172.21.15.152:0/3589752893 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd402a050 con 0x55a5067e75d0 2024-09-11T22:55:24.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1beb945740 1 -- 172.21.15.152:0/3589752893 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd4005850 con 0x55a5067e75d0 2024-09-11T22:55:24.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.708+0000 7f1bca7fc640 1 -- 172.21.15.152:0/3589752893 <== mon.2 v2:172.21.15.191:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f1bd4004660 con 0x55a5067e75d0 2024-09-11T22:55:24.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.709+0000 7f1bca7fc640 1 -- 172.21.15.152:0/3589752893 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1bd4047050 con 0x55a5067e75d0 2024-09-11T22:55:24.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.709+0000 7f1beb945740 10 client.25439 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:24.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.709+0000 7f1beb945740 1 -- 172.21.15.152:0/3589752893 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a506870cc0 con 0x55a5067e75d0 2024-09-11T22:55:24.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.709+0000 7f1bca7fc640 1 -- 172.21.15.152:0/3589752893 <== mon.2 v2:172.21.15.191:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd4005850 con 0x55a5067e75d0 2024-09-11T22:55:24.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.709+0000 7f1bca7fc640 1 -- 172.21.15.152:0/3589752893 <== mon.2 v2:172.21.15.191:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1bd4037070 con 0x55a5067e75d0 2024-09-11T22:55:24.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.709+0000 7f1beb945740 10 client.25439 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:24.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.709+0000 7f1beb945740 10 client.25439 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:55:24.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.709+0000 7f1beb945740 20 client.25439 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:24.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.709+0000 7f1beb945740 1 --2- 172.21.15.152:0/3589752893 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55a506871300 0x55a506873720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:24.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.709+0000 7f1beb945740 4 client.25439 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:55:24.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.709+0000 7f1beb945740 1 -- 172.21.15.152:0/3589752893 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a506754d70 con 0x55a506871300 2024-09-11T22:55:24.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.709+0000 7f1be27fc640 1 --2- 172.21.15.152:0/3589752893 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55a506871300 0x55a506873720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:24.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.709+0000 7f1bca7fc640 10 client.25439 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:55:24.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.709+0000 7f1be27fc640 1 --2- 172.21.15.152:0/3589752893 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55a506871300 0x55a506873720 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=95aa016141dd39b server_cookie=87269a2ed210b31d in_seq=0 out_seq=0 2024-09-11T22:55:24.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1bca7fc640 1 -- 172.21.15.152:0/3589752893 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a506754d70 con 0x55a506871300 2024-09-11T22:55:24.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1bca7fc640 10 client.25439 handle_command_reply: tid=0 2024-09-11T22:55:24.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1beb945740 1 client.25439 shutdown 2024-09-11T22:55:24.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1beb945740 1 -- 172.21.15.152:0/3589752893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 msgr2=0x55a5067ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:24.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1beb945740 1 --2- 172.21.15.152:0/3589752893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 0x55a5067ff080 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4000c00 tx=0x7f1bd401b6e0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1beb945740 1 -- 172.21.15.152:0/3589752893 shutdown_connections 2024-09-11T22:55:24.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1beb945740 1 --2- 172.21.15.152:0/3589752893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5067f2370 0x55a5067ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1beb945740 1 --2- 172.21.15.152:0/3589752893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5067e8b40 0x55a5067ebe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1beb945740 1 --2- 172.21.15.152:0/3589752893 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55a506871300 0x55a506873720 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1beb945740 1 --2- 172.21.15.152:0/3589752893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5067e75d0 0x55a5067ff080 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1beb945740 1 -- 172.21.15.152:0/3589752893 >> 172.21.15.152:0/3589752893 conn(0x55a5067b0310 msgr2=0x55a5067f1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:24.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1bca7fc640 0 client.25439 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:55:24.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1beb945740 1 -- 172.21.15.152:0/3589752893 shutdown_connections 2024-09-11T22:55:24.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1beb945740 1 -- 172.21.15.152:0/3589752893 wait complete. 2024-09-11T22:55:24.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1beb945740 20 client.25439 trim_cache size 0 max 16384 2024-09-11T22:55:24.247 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:55:24.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1be956c640 1 -- 172.21.15.152:0/49949602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1bd00a2cf0 msgr2=0x7f1bd00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:24.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1be956c640 1 --2- 172.21.15.152:0/49949602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1bd00a2cf0 0x7f1bd00a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc005fd0 tx=0x7f1bdc009b50 comp rx=0 tx=0).stop 2024-09-11T22:55:24.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1be956c640 1 -- 172.21.15.152:0/49949602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be4102570 msgr2=0x7f1be4198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:24.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1be956c640 1 --2- 172.21.15.152:0/49949602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be4102570 0x7f1be4198a20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc006010 tx=0x7f1bcc003c10 comp rx=0 tx=0).stop 2024-09-11T22:55:24.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1be956c640 1 -- 172.21.15.152:0/49949602 shutdown_connections 2024-09-11T22:55:24.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1be956c640 1 --2- 172.21.15.152:0/49949602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1bd00a2cf0 0x7f1bd00a51b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1be956c640 1 --2- 172.21.15.152:0/49949602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1be4102ef0 0x7f1be419cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1be956c640 1 --2- 172.21.15.152:0/49949602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1be4102570 0x7f1be4198a20 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1be956c640 1 --2- 172.21.15.152:0/49949602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1be4100e20 0x7f1be41984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:24.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.710+0000 7f1be956c640 1 -- 172.21.15.152:0/49949602 >> 172.21.15.152:0/49949602 conn(0x7f1be40fc7d0 msgr2=0x7f1be40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:24.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.711+0000 7f1be956c640 1 -- 172.21.15.152:0/49949602 shutdown_connections 2024-09-11T22:55:24.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:22.711+0000 7f1be956c640 1 -- 172.21.15.152:0/49949602 wait complete. 2024-09-11T22:55:24.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:24 smithi191 ceph-mon[35211]: pgmap v78: 97 pgs: 97 active+clean; 18 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 640 MiB/s wr, 363 op/s 2024-09-11T22:55:24.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:24 smithi204 ceph-mon[35454]: pgmap v78: 97 pgs: 97 active+clean; 18 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 640 MiB/s wr, 363 op/s 2024-09-11T22:55:24.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:24 smithi152 ceph-mon[31181]: pgmap v78: 97 pgs: 97 active+clean; 18 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 640 MiB/s wr, 363 op/s 2024-09-11T22:55:26.455 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:55:26.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:26 smithi191 ceph-mon[35211]: pgmap v79: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 853 MiB/s wr, 421 op/s 2024-09-11T22:55:26.618 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:26 smithi204 ceph-mon[35454]: pgmap v79: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 853 MiB/s wr, 421 op/s 2024-09-11T22:55:26.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:26 smithi152 ceph-mon[31181]: pgmap v79: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 853 MiB/s wr, 421 op/s 2024-09-11T22:55:26.706 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:55:27.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.205+0000 7f020dc3b640 1 -- 172.21.15.152:0/4032936781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0208100a60 msgr2=0x7f0208100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:27.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.205+0000 7f020dc3b640 1 --2- 172.21.15.152:0/4032936781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0208100a60 0x7f0208100ec0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009930 tx=0x7f01f002a420 comp rx=0 tx=0).stop 2024-09-11T22:55:27.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.205+0000 7f020dc3b640 1 -- 172.21.15.152:0/4032936781 shutdown_connections 2024-09-11T22:55:27.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.205+0000 7f020dc3b640 1 --2- 172.21.15.152:0/4032936781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0208101400 0x7f020810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.205+0000 7f020dc3b640 1 --2- 172.21.15.152:0/4032936781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0208100a60 0x7f0208100ec0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.205+0000 7f020dc3b640 1 --2- 172.21.15.152:0/4032936781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0208106a60 0x7f0208106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.205+0000 7f020dc3b640 1 -- 172.21.15.152:0/4032936781 >> 172.21.15.152:0/4032936781 conn(0x7f02080fc780 msgr2=0x7f02080feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:27.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.206+0000 7f020dc3b640 1 -- 172.21.15.152:0/4032936781 shutdown_connections 2024-09-11T22:55:27.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.206+0000 7f020dc3b640 1 -- 172.21.15.152:0/4032936781 wait complete. 2024-09-11T22:55:27.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.206+0000 7f020dc3b640 1 Processor -- start 2024-09-11T22:55:27.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.206+0000 7f020dc3b640 1 -- start start 2024-09-11T22:55:27.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f020dc3b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0208100a60 0x7f0208199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f020dc3b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0208101400 0x7f0208199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f020dc3b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0208106a60 0x7f02081a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f020dc3b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0208110e60 con 0x7f0208100a60 2024-09-11T22:55:27.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f020dc3b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0208110ce0 con 0x7f0208101400 2024-09-11T22:55:27.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f020dc3b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0208110fe0 con 0x7f0208106a60 2024-09-11T22:55:27.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f02077fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0208100a60 0x7f0208199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f02077fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0208100a60 0x7f0208199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39076/0 (socket says 172.21.15.152:39076) 2024-09-11T22:55:27.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f02077fe640 1 -- 172.21.15.152:0/3177820426 learned_addr learned my addr 172.21.15.152:0/3177820426 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:27.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f0207fff640 1 --2- 172.21.15.152:0/3177820426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0208106a60 0x7f02081a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f01feffd640 1 --2- 172.21.15.152:0/3177820426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0208101400 0x7f0208199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f02077fe640 1 -- 172.21.15.152:0/3177820426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0208106a60 msgr2=0x7f02081a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:27.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f02077fe640 1 --2- 172.21.15.152:0/3177820426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0208106a60 0x7f02081a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f02077fe640 1 -- 172.21.15.152:0/3177820426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0208101400 msgr2=0x7f0208199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:27.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f02077fe640 1 --2- 172.21.15.152:0/3177820426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0208101400 0x7f0208199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f02077fe640 1 -- 172.21.15.152:0/3177820426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02081a48d0 con 0x7f0208100a60 2024-09-11T22:55:27.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f02077fe640 1 --2- 172.21.15.152:0/3177820426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0208100a60 0x7f0208199310 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f01f800d950 tx=0x7f01f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:27.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f02057fa640 1 -- 172.21.15.152:0/3177820426 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8017030 con 0x7f0208100a60 2024-09-11T22:55:27.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f02057fa640 1 -- 172.21.15.152:0/3177820426 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f8004020 con 0x7f0208100a60 2024-09-11T22:55:27.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f02057fa640 1 -- 172.21.15.152:0/3177820426 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f80047a0 con 0x7f0208100a60 2024-09-11T22:55:27.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f020dc3b640 1 -- 172.21.15.152:0/3177820426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02081a4aa0 con 0x7f0208100a60 2024-09-11T22:55:27.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.207+0000 7f020dc3b640 1 -- 172.21.15.152:0/3177820426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02081a4f60 con 0x7f0208100a60 2024-09-11T22:55:27.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.210+0000 7f02057fa640 1 -- 172.21.15.152:0/3177820426 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f01f80041d0 con 0x7f0208100a60 2024-09-11T22:55:27.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.210+0000 7f0210014740 1 Processor -- start 2024-09-11T22:55:27.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.210+0000 7f02057fa640 1 --2- 172.21.15.152:0/3177820426 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f01f40a2cf0 0x7f01f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.210+0000 7f0210014740 1 -- start start 2024-09-11T22:55:27.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.210+0000 7f0210014740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5eb40 0x55e10fd5d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.210+0000 7f0207fff640 1 --2- 172.21.15.152:0/3177820426 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f01f40a2cf0 0x7f01f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.210+0000 7f0210014740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5d5d0 0x55e10fd5da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.210+0000 7f0210014740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd68370 0x55e10fd6a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.210+0000 7f0210014740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55e10fcc32f0 con 0x55e10fd5eb40 2024-09-11T22:55:27.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.210+0000 7f0210014740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e10fc6acf0 con 0x55e10fd5d5d0 2024-09-11T22:55:27.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.210+0000 7f02057fa640 1 -- 172.21.15.152:0/3177820426 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f01f80cecd0 con 0x7f0208100a60 2024-09-11T22:55:27.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.210+0000 7f0210014740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e10fcbf2b0 con 0x55e10fd68370 2024-09-11T22:55:27.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f02077fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5eb40 0x55e10fd5d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f01feffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5d5d0 0x55e10fd5da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f01feffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5d5d0 0x55e10fd5da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:33796/0 (socket says 172.21.15.152:33796) 2024-09-11T22:55:27.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f01feffd640 1 -- 172.21.15.152:0/1066615859 learned_addr learned my addr 172.21.15.152:0/1066615859 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:27.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f0207fff640 1 --2- 172.21.15.152:0/1066615859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd68370 0x55e10fd6a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f0207fff640 1 --2- 172.21.15.152:0/3177820426 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f01f40a2cf0 0x7f01f40a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0200005fd0 tx=0x7f0200010740 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f01feffd640 1 -- 172.21.15.152:0/1066615859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd68370 msgr2=0x55e10fd6a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f01feffd640 1 --2- 172.21.15.152:0/1066615859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd68370 0x55e10fd6a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f01feffd640 1 -- 172.21.15.152:0/1066615859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5eb40 msgr2=0x55e10fd5d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f01feffd640 1 --2- 172.21.15.152:0/1066615859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5eb40 0x55e10fd5d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f01feffd640 1 -- 172.21.15.152:0/1066615859 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e10fc4f3a0 con 0x55e10fd5d5d0 2024-09-11T22:55:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f02077fe640 1 --2- 172.21.15.152:0/1066615859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5eb40 0x55e10fd5d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:55:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f0207fff640 1 --2- 172.21.15.152:0/1066615859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd68370 0x55e10fd6a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:55:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f01feffd640 1 --2- 172.21.15.152:0/1066615859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5d5d0 0x55e10fd5da10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f01f002a940 tx=0x7f01f0005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99081a9b61f7abf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f01fdffb640 1 -- 172.21.15.152:0/1066615859 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0040070 con 0x55e10fd5d5d0 2024-09-11T22:55:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f01fdffb640 1 -- 172.21.15.152:0/1066615859 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f00319f0 con 0x55e10fd5d5d0 2024-09-11T22:55:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.211+0000 7f01fdffb640 1 -- 172.21.15.152:0/1066615859 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0035040 con 0x55e10fd5d5d0 2024-09-11T22:55:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.212+0000 7f0210014740 1 -- 172.21.15.152:0/1066615859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5d5d0 msgr2=0x55e10fd5da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:27.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.212+0000 7f0210014740 1 --2- 172.21.15.152:0/1066615859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5d5d0 0x55e10fd5da10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f01f002a940 tx=0x7f01f0005c00 comp rx=0 tx=0).stop 2024-09-11T22:55:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.212+0000 7f0210014740 1 -- 172.21.15.152:0/1066615859 shutdown_connections 2024-09-11T22:55:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.212+0000 7f0210014740 1 --2- 172.21.15.152:0/1066615859 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd68370 0x55e10fd6a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.212+0000 7f0210014740 1 --2- 172.21.15.152:0/1066615859 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5d5d0 0x55e10fd5da10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.212+0000 7f0210014740 1 --2- 172.21.15.152:0/1066615859 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5eb40 0x55e10fd5d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.212+0000 7f0210014740 1 -- 172.21.15.152:0/1066615859 >> 172.21.15.152:0/1066615859 conn(0x55e10fd26310 msgr2=0x55e10fd59c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.212+0000 7f0210014740 1 -- 172.21.15.152:0/1066615859 shutdown_connections 2024-09-11T22:55:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.212+0000 7f0210014740 1 -- 172.21.15.152:0/1066615859 wait complete. 2024-09-11T22:55:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.212+0000 7f0210014740 1 -- start start 2024-09-11T22:55:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f0210014740 1 Processor -- start 2024-09-11T22:55:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f0210014740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5d5d0 0x55e10fd67ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f0210014740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd5eb40 0x55e10fd64f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f0210014740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd68370 0x55e10fd65490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f0210014740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55e10fd6b010 con 0x55e10fd68370 2024-09-11T22:55:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f0210014740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e10fc7e060 con 0x55e10fd5d5d0 2024-09-11T22:55:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f0210014740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e10fd6b190 con 0x55e10fd5eb40 2024-09-11T22:55:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f0210014740 10 client.0 _ll_register_callbacks cb 0x55e10f9d18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f0207fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd68370 0x55e10fd65490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f0207fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd68370 0x55e10fd65490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39090/0 (socket says 172.21.15.152:39090) 2024-09-11T22:55:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f0207fff640 1 -- 172.21.15.152:0/3422671472 learned_addr learned my addr 172.21.15.152:0/3422671472 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f02077fe640 1 --2- 172.21.15.152:0/3422671472 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5d5d0 0x55e10fd67ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f01feffd640 1 --2- 172.21.15.152:0/3422671472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd5eb40 0x55e10fd64f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f0207fff640 1 -- 172.21.15.152:0/3422671472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd5eb40 msgr2=0x55e10fd64f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f0207fff640 1 --2- 172.21.15.152:0/3422671472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd5eb40 0x55e10fd64f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f0207fff640 1 -- 172.21.15.152:0/3422671472 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5d5d0 msgr2=0x55e10fd67ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f0207fff640 1 --2- 172.21.15.152:0/3422671472 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5d5d0 0x55e10fd67ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f0207fff640 1 -- 172.21.15.152:0/3422671472 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e10fa343c0 con 0x55e10fd68370 2024-09-11T22:55:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.213+0000 7f01feffd640 1 --2- 172.21.15.152:0/3422671472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd5eb40 0x55e10fd64f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:55:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.214+0000 7f0207fff640 1 --2- 172.21.15.152:0/3422671472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd68370 0x55e10fd65490 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0200013d50 tx=0x7f0200004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b4168d019a0c51 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.214+0000 7f01e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.214+0000 7f01e6ffd640 1 -- 172.21.15.152:0/3422671472 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0200023070 con 0x55e10fd68370 2024-09-11T22:55:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.214+0000 7f01e6ffd640 1 -- 172.21.15.152:0/3422671472 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f0200003b70 con 0x55e10fd68370 2024-09-11T22:55:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.214+0000 7f0210014740 1 -- 172.21.15.152:0/3422671472 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01f00319f0 con 0x55e10fd68370 2024-09-11T22:55:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.214+0000 7f01e6ffd640 1 -- 172.21.15.152:0/3422671472 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0200003db0 con 0x55e10fd68370 2024-09-11T22:55:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.214+0000 7f0210014740 10 client.16191 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:27.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.214+0000 7f0210014740 1 -- 172.21.15.152:0/3422671472 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e10fd76a30 con 0x55e10fd68370 2024-09-11T22:55:27.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.214+0000 7f01e6ffd640 1 -- 172.21.15.152:0/3422671472 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f00319f0 con 0x55e10fd68370 2024-09-11T22:55:27.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.214+0000 7f01e6ffd640 1 -- 172.21.15.152:0/3422671472 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f020000f040 con 0x55e10fd68370 2024-09-11T22:55:27.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.214+0000 7f0210014740 10 client.16191 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:27.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.214+0000 7f0210014740 10 client.16191 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:55:27.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.214+0000 7f0210014740 20 client.16191 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:27.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.214+0000 7f0210014740 1 --2- 172.21.15.152:0/3422671472 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55e10fd77e40 0x55e10fd7a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.214+0000 7f0210014740 4 client.16191 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:55:27.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.214+0000 7f0210014740 1 -- 172.21.15.152:0/3422671472 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e10fcd0860 con 0x55e10fd77e40 2024-09-11T22:55:27.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.215+0000 7f02077fe640 1 --2- 172.21.15.152:0/3422671472 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55e10fd77e40 0x55e10fd7a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.215+0000 7f02077fe640 1 --2- 172.21.15.152:0/3422671472 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55e10fd77e40 0x55e10fd7a2e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c1e63ec15eae3e2b server_cookie=d1d8d5419dd1912c in_seq=0 out_seq=0 2024-09-11T22:55:27.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.215+0000 7f01e6ffd640 10 client.16191 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:55:27.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.216+0000 7f01e6ffd640 1 -- 172.21.15.152:0/3422671472 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e10fcd0860 con 0x55e10fd77e40 2024-09-11T22:55:27.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.216+0000 7f01e6ffd640 10 client.16191 handle_command_reply: tid=0 2024-09-11T22:55:27.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.216+0000 7f0210014740 1 client.16191 shutdown 2024-09-11T22:55:27.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.216+0000 7f0210014740 1 -- 172.21.15.152:0/3422671472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd68370 msgr2=0x55e10fd65490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:27.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.216+0000 7f0210014740 1 --2- 172.21.15.152:0/3422671472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd68370 0x55e10fd65490 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0200013d50 tx=0x7f0200004ab0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.216+0000 7f0210014740 1 -- 172.21.15.152:0/3422671472 shutdown_connections 2024-09-11T22:55:27.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.216+0000 7f0210014740 1 --2- 172.21.15.152:0/3422671472 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55e10fd77e40 0x55e10fd7a2e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.216+0000 7f0210014740 1 --2- 172.21.15.152:0/3422671472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd68370 0x55e10fd65490 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.216+0000 7f0210014740 1 --2- 172.21.15.152:0/3422671472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd5eb40 0x55e10fd64f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.216+0000 7f0210014740 1 --2- 172.21.15.152:0/3422671472 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5d5d0 0x55e10fd67ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.216+0000 7f0210014740 1 -- 172.21.15.152:0/3422671472 >> 172.21.15.152:0/3422671472 conn(0x55e10fd26310 msgr2=0x55e10fd5f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.216+0000 7f01e6ffd640 0 client.16191 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:55:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.216+0000 7f0210014740 1 -- 172.21.15.152:0/3422671472 shutdown_connections 2024-09-11T22:55:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.216+0000 7f0210014740 1 -- 172.21.15.152:0/3422671472 wait complete. 2024-09-11T22:55:27.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.216+0000 7f0210014740 20 client.16191 trim_cache size 0 max 16384 2024-09-11T22:55:27.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.229+0000 7f0210014740 1 Processor -- start 2024-09-11T22:55:27.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.229+0000 7f0210014740 1 -- start start 2024-09-11T22:55:27.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.229+0000 7f0210014740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5d5d0 0x55e10fd74310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f0210014740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5eb40 0x55e10fd60120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f0210014740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd68370 0x55e10fd60660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f0210014740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f02081ab5b0 con 0x55e10fd5d5d0 2024-09-11T22:55:27.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f0210014740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f02081ad0f0 con 0x55e10fd5eb40 2024-09-11T22:55:27.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f0210014740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02081abc80 con 0x55e10fd68370 2024-09-11T22:55:27.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f02077fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5d5d0 0x55e10fd74310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f02077fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5d5d0 0x55e10fd74310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39094/0 (socket says 172.21.15.152:39094) 2024-09-11T22:55:27.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f02077fe640 1 -- 172.21.15.152:0/646488753 learned_addr learned my addr 172.21.15.152:0/646488753 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:27.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f01feffd640 1 --2- 172.21.15.152:0/646488753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5eb40 0x55e10fd60120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f0207fff640 1 --2- 172.21.15.152:0/646488753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd68370 0x55e10fd60660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f02077fe640 1 -- 172.21.15.152:0/646488753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd68370 msgr2=0x55e10fd60660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:27.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f02077fe640 1 --2- 172.21.15.152:0/646488753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd68370 0x55e10fd60660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f02077fe640 1 -- 172.21.15.152:0/646488753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5eb40 msgr2=0x55e10fd60120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:27.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f02077fe640 1 --2- 172.21.15.152:0/646488753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5eb40 0x55e10fd60120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f02077fe640 1 -- 172.21.15.152:0/646488753 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e10fd5e740 con 0x55e10fd5d5d0 2024-09-11T22:55:27.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f02077fe640 1 --2- 172.21.15.152:0/646488753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5d5d0 0x55e10fd74310 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f01f809af40 tx=0x7f01f80bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586e669f0e6e6844 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:27.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f01fd7fa640 1 -- 172.21.15.152:0/646488753 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f809f5f0 con 0x55e10fd5d5d0 2024-09-11T22:55:27.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f01fd7fa640 1 -- 172.21.15.152:0/646488753 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f80a3eb0 con 0x55e10fd5d5d0 2024-09-11T22:55:27.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.230+0000 7f01fd7fa640 1 -- 172.21.15.152:0/646488753 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f80a3c80 con 0x55e10fd5d5d0 2024-09-11T22:55:27.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.231+0000 7f0210014740 1 -- 172.21.15.152:0/646488753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5d5d0 msgr2=0x55e10fd74310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:27.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.231+0000 7f0210014740 1 --2- 172.21.15.152:0/646488753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5d5d0 0x55e10fd74310 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f01f809af40 tx=0x7f01f80bb040 comp rx=0 tx=0).stop 2024-09-11T22:55:27.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.231+0000 7f0210014740 1 -- 172.21.15.152:0/646488753 shutdown_connections 2024-09-11T22:55:27.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.231+0000 7f0210014740 1 --2- 172.21.15.152:0/646488753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd68370 0x55e10fd60660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.231+0000 7f0210014740 1 --2- 172.21.15.152:0/646488753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd5eb40 0x55e10fd60120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.231+0000 7f0210014740 1 --2- 172.21.15.152:0/646488753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5d5d0 0x55e10fd74310 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.231+0000 7f0210014740 1 -- 172.21.15.152:0/646488753 >> 172.21.15.152:0/646488753 conn(0x55e10fd26310 msgr2=0x55e10fd64f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:27.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.231+0000 7f0210014740 1 -- 172.21.15.152:0/646488753 shutdown_connections 2024-09-11T22:55:27.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.231+0000 7f0210014740 1 -- 172.21.15.152:0/646488753 wait complete. 2024-09-11T22:55:27.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.231+0000 7f0210014740 1 -- start start 2024-09-11T22:55:27.243 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f0210014740 1 Processor -- start 2024-09-11T22:55:27.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f0210014740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd5d5d0 0x55e10fd70980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f0210014740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5eb40 0x55e10fd7f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f0210014740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd68370 0x55e10fd7f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f0210014740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f02081aca20 con 0x55e10fd5eb40 2024-09-11T22:55:27.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f0210014740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f02081ac350 con 0x55e10fd68370 2024-09-11T22:55:27.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f0210014740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f02081a9590 con 0x55e10fd5d5d0 2024-09-11T22:55:27.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f0210014740 10 client.0 _ll_register_callbacks cb 0x7f0200009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:27.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f01feffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5eb40 0x55e10fd7f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f01feffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5eb40 0x55e10fd7f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39102/0 (socket says 172.21.15.152:39102) 2024-09-11T22:55:27.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f01feffd640 1 -- 172.21.15.152:0/2040060143 learned_addr learned my addr 172.21.15.152:0/2040060143 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:27.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f02077fe640 1 --2- 172.21.15.152:0/2040060143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd5d5d0 0x55e10fd70980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f0207fff640 1 --2- 172.21.15.152:0/2040060143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd68370 0x55e10fd7f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f01feffd640 1 -- 172.21.15.152:0/2040060143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd5d5d0 msgr2=0x55e10fd70980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:27.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f01feffd640 1 --2- 172.21.15.152:0/2040060143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd5d5d0 0x55e10fd70980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f01feffd640 1 -- 172.21.15.152:0/2040060143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd68370 msgr2=0x55e10fd7f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:27.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f01feffd640 1 --2- 172.21.15.152:0/2040060143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd68370 0x55e10fd7f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f01feffd640 1 -- 172.21.15.152:0/2040060143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e10fd7fd40 con 0x55e10fd5eb40 2024-09-11T22:55:27.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f02077fe640 1 --2- 172.21.15.152:0/2040060143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd5d5d0 0x55e10fd70980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:27.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f0207fff640 1 --2- 172.21.15.152:0/2040060143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd68370 0x55e10fd7f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:27.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f01feffd640 1 --2- 172.21.15.152:0/2040060143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5eb40 0x55e10fd7f240 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x55e10fd5aac0 tx=0x7f01f0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e3586357f2daa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:27.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.232+0000 7f01e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:27.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.233+0000 7f01e6ffd640 1 -- 172.21.15.152:0/2040060143 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0035070 con 0x55e10fd5eb40 2024-09-11T22:55:27.245 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.233+0000 7f01e6ffd640 1 -- 172.21.15.152:0/2040060143 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f01f0004450 con 0x55e10fd5eb40 2024-09-11T22:55:27.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.233+0000 7f0210014740 1 -- 172.21.15.152:0/2040060143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01f80a3eb0 con 0x55e10fd5eb40 2024-09-11T22:55:27.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.233+0000 7f01e6ffd640 1 -- 172.21.15.152:0/2040060143 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f01f0056050 con 0x55e10fd5eb40 2024-09-11T22:55:27.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.233+0000 7f0210014740 10 client.16197 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:27.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.233+0000 7f0210014740 1 -- 172.21.15.152:0/2040060143 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e10fd6f1e0 con 0x55e10fd5eb40 2024-09-11T22:55:27.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.233+0000 7f01e6ffd640 1 -- 172.21.15.152:0/2040060143 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f80a3eb0 con 0x55e10fd5eb40 2024-09-11T22:55:27.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.233+0000 7f01e6ffd640 1 -- 172.21.15.152:0/2040060143 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f01f00035f0 con 0x55e10fd5eb40 2024-09-11T22:55:27.246 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.233+0000 7f0210014740 10 client.16197 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:27.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.233+0000 7f0210014740 10 client.16197 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:55:27.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.233+0000 7f0210014740 20 client.16197 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:27.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.233+0000 7f0210014740 1 --2- 172.21.15.152:0/2040060143 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55e10fde2b80 0x55e10fde4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:27.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.233+0000 7f0210014740 4 client.16197 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:55:27.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.233+0000 7f0210014740 1 -- 172.21.15.152:0/2040060143 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e10fccad70 con 0x55e10fde2b80 2024-09-11T22:55:27.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.233+0000 7f02077fe640 1 --2- 172.21.15.152:0/2040060143 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55e10fde2b80 0x55e10fde4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:27.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.234+0000 7f01e6ffd640 10 client.16197 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:55:27.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.234+0000 7f02077fe640 1 --2- 172.21.15.152:0/2040060143 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55e10fde2b80 0x55e10fde4f70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e271dcac38d98821 server_cookie=aa5fa15b2f38f141 in_seq=0 out_seq=0 2024-09-11T22:55:27.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.234+0000 7f01e6ffd640 1 -- 172.21.15.152:0/2040060143 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e10fccad70 con 0x55e10fde2b80 2024-09-11T22:55:27.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.234+0000 7f01e6ffd640 10 client.16197 handle_command_reply: tid=0 2024-09-11T22:55:27.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.234+0000 7f0210014740 1 client.16197 shutdown 2024-09-11T22:55:27.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.234+0000 7f0210014740 1 -- 172.21.15.152:0/2040060143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5eb40 msgr2=0x55e10fd7f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:27.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.234+0000 7f0210014740 1 --2- 172.21.15.152:0/2040060143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5eb40 0x55e10fd7f240 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x55e10fd5aac0 tx=0x7f01f0005e80 comp rx=0 tx=0).stop 2024-09-11T22:55:27.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.234+0000 7f0210014740 1 -- 172.21.15.152:0/2040060143 shutdown_connections 2024-09-11T22:55:27.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.234+0000 7f0210014740 1 --2- 172.21.15.152:0/2040060143 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55e10fde2b80 0x55e10fde4f70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.234+0000 7f0210014740 1 --2- 172.21.15.152:0/2040060143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e10fd68370 0x55e10fd7f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.234+0000 7f0210014740 1 --2- 172.21.15.152:0/2040060143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e10fd5eb40 0x55e10fd7f240 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.234+0000 7f0210014740 1 --2- 172.21.15.152:0/2040060143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e10fd5d5d0 0x55e10fd70980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.234+0000 7f0210014740 1 -- 172.21.15.152:0/2040060143 >> 172.21.15.152:0/2040060143 conn(0x55e10fd26310 msgr2=0x55e10fd63780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:27.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.234+0000 7f01e6ffd640 0 client.16197 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:55:27.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.234+0000 7f0210014740 1 -- 172.21.15.152:0/2040060143 shutdown_connections 2024-09-11T22:55:27.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.234+0000 7f0210014740 1 -- 172.21.15.152:0/2040060143 wait complete. 2024-09-11T22:55:27.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.235+0000 7f0210014740 20 client.16197 trim_cache size 0 max 16384 2024-09-11T22:55:27.249 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:55:27.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.235+0000 7f020dc3b640 1 -- 172.21.15.152:0/3177820426 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f01f40a2cf0 msgr2=0x7f01f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:27.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.235+0000 7f020dc3b640 1 --2- 172.21.15.152:0/3177820426 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f01f40a2cf0 0x7f01f40a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0200005fd0 tx=0x7f0200010740 comp rx=0 tx=0).stop 2024-09-11T22:55:27.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.235+0000 7f020dc3b640 1 -- 172.21.15.152:0/3177820426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0208100a60 msgr2=0x7f0208199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:27.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.235+0000 7f020dc3b640 1 --2- 172.21.15.152:0/3177820426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0208100a60 0x7f0208199310 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f01f800d950 tx=0x7f01f800de10 comp rx=0 tx=0).stop 2024-09-11T22:55:27.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.235+0000 7f020dc3b640 1 -- 172.21.15.152:0/3177820426 shutdown_connections 2024-09-11T22:55:27.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.235+0000 7f020dc3b640 1 --2- 172.21.15.152:0/3177820426 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f01f40a2cf0 0x7f01f40a51b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.235+0000 7f020dc3b640 1 --2- 172.21.15.152:0/3177820426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0208106a60 0x7f02081a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.235+0000 7f020dc3b640 1 --2- 172.21.15.152:0/3177820426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0208101400 0x7f0208199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.235+0000 7f020dc3b640 1 --2- 172.21.15.152:0/3177820426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0208100a60 0x7f0208199310 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:27.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.235+0000 7f020dc3b640 1 -- 172.21.15.152:0/3177820426 >> 172.21.15.152:0/3177820426 conn(0x7f02080fc780 msgr2=0x7f02080fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:27.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.235+0000 7f020dc3b640 1 -- 172.21.15.152:0/3177820426 shutdown_connections 2024-09-11T22:55:27.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:27.235+0000 7f020dc3b640 1 -- 172.21.15.152:0/3177820426 wait complete. 2024-09-11T22:55:28.044 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:55:28.331 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:55:28.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:28 smithi152 ceph-mon[31181]: pgmap v80: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 625 MiB/s wr, 256 op/s 2024-09-11T22:55:28.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:28 smithi191 ceph-mon[35211]: pgmap v80: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 625 MiB/s wr, 256 op/s 2024-09-11T22:55:28.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:28 smithi204 ceph-mon[35454]: pgmap v80: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 625 MiB/s wr, 256 op/s 2024-09-11T22:55:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.711+0000 7fe7c2f26640 1 -- 172.21.15.152:0/3575483874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7bc101370 msgr2=0x7fe7bc10d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.711+0000 7fe7c2f26640 1 --2- 172.21.15.152:0/3575483874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7bc101370 0x7fe7bc10d8a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009930 tx=0x7fe7a402a420 comp rx=0 tx=0).stop 2024-09-11T22:55:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.711+0000 7fe7c2f26640 1 -- 172.21.15.152:0/3575483874 shutdown_connections 2024-09-11T22:55:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.711+0000 7fe7c2f26640 1 --2- 172.21.15.152:0/3575483874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7bc101370 0x7fe7bc10d8a0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.711+0000 7fe7c2f26640 1 --2- 172.21.15.152:0/3575483874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7bc1009d0 0x7fe7bc100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.711+0000 7fe7c2f26640 1 --2- 172.21.15.152:0/3575483874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7bc1069e0 0x7fe7bc106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.711+0000 7fe7c2f26640 1 -- 172.21.15.152:0/3575483874 >> 172.21.15.152:0/3575483874 conn(0x7fe7bc0fc6f0 msgr2=0x7fe7bc0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.711+0000 7fe7c2f26640 1 -- 172.21.15.152:0/3575483874 shutdown_connections 2024-09-11T22:55:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.711+0000 7fe7c2f26640 1 -- 172.21.15.152:0/3575483874 wait complete. 2024-09-11T22:55:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.712+0000 7fe7c2f26640 1 Processor -- start 2024-09-11T22:55:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.712+0000 7fe7c2f26640 1 -- start start 2024-09-11T22:55:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.712+0000 7fe7c2f26640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7bc1009d0 0x7fe7bc197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.712+0000 7fe7c2f26640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7bc101370 0x7fe7bc197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.712+0000 7fe7c2f26640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7bc1069e0 0x7fe7bc19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.712+0000 7fe7c2f26640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc110e20 con 0x7fe7bc1009d0 2024-09-11T22:55:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.712+0000 7fe7c2f26640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc110ca0 con 0x7fe7bc101370 2024-09-11T22:55:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.712+0000 7fe7c2f26640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc110fa0 con 0x7fe7bc1069e0 2024-09-11T22:55:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.712+0000 7fe7c0cc2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7bc1009d0 0x7fe7bc197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.712+0000 7fe7bbfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7bc101370 0x7fe7bc197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.712+0000 7fe7c0cc2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7bc1009d0 0x7fe7bc197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54898/0 (socket says 172.21.15.152:54898) 2024-09-11T22:55:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.712+0000 7fe7bbfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7bc101370 0x7fe7bc197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:46722/0 (socket says 172.21.15.152:46722) 2024-09-11T22:55:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.712+0000 7fe7c0cc2640 1 -- 172.21.15.152:0/3327851732 learned_addr learned my addr 172.21.15.152:0/3327851732 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.713+0000 7fe7c14c3640 1 --2- 172.21.15.152:0/3327851732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7bc1069e0 0x7fe7bc19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.713+0000 7fe7bbfff640 1 -- 172.21.15.152:0/3327851732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7bc1069e0 msgr2=0x7fe7bc19bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.713+0000 7fe7bbfff640 1 --2- 172.21.15.152:0/3327851732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7bc1069e0 0x7fe7bc19bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.713+0000 7fe7bbfff640 1 -- 172.21.15.152:0/3327851732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7bc1009d0 msgr2=0x7fe7bc197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.713+0000 7fe7bbfff640 1 --2- 172.21.15.152:0/3327851732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7bc1009d0 0x7fe7bc197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.713+0000 7fe7bbfff640 1 -- 172.21.15.152:0/3327851732 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7bc19c240 con 0x7fe7bc101370 2024-09-11T22:55:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.713+0000 7fe7c14c3640 1 --2- 172.21.15.152:0/3327851732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7bc1069e0 0x7fe7bc19bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.713+0000 7fe7c0cc2640 1 --2- 172.21.15.152:0/3327851732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7bc1009d0 0x7fe7bc197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:55:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.713+0000 7fe7bbfff640 1 --2- 172.21.15.152:0/3327851732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7bc101370 0x7fe7bc197790 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009fd0 tx=0x7fe7ac00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.713+0000 7fe7b9ffb640 1 -- 172.21.15.152:0/3327851732 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac004b20 con 0x7fe7bc101370 2024-09-11T22:55:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.713+0000 7fe7c2f26640 1 -- 172.21.15.152:0/3327851732 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7bc19c530 con 0x7fe7bc101370 2024-09-11T22:55:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.713+0000 7fe7b9ffb640 1 -- 172.21.15.152:0/3327851732 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7ac004cc0 con 0x7fe7bc101370 2024-09-11T22:55:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.713+0000 7fe7b9ffb640 1 -- 172.21.15.152:0/3327851732 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac010770 con 0x7fe7bc101370 2024-09-11T22:55:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.713+0000 7fe7c2f26640 1 -- 172.21.15.152:0/3327851732 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7bc1a3dc0 con 0x7fe7bc101370 2024-09-11T22:55:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.716+0000 7fe7b9ffb640 1 -- 172.21.15.152:0/3327851732 <== mon.2 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe7ac005000 con 0x7fe7bc101370 2024-09-11T22:55:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.716+0000 7fe7c5cb9740 1 Processor -- start 2024-09-11T22:55:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.716+0000 7fe7c5cb9740 1 -- start start 2024-09-11T22:55:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.716+0000 7fe7c5cb9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 0x558c313b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.716+0000 7fe7c5cb9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313b9410 0x558c313c3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.716+0000 7fe7c5cb9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313c3cb0 0x558c313c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.716+0000 7fe7c5cb9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558c3131d2f0 con 0x558c313c3cb0 2024-09-11T22:55:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.716+0000 7fe7c5cb9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558c312c5270 con 0x558c313b8af0 2024-09-11T22:55:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.716+0000 7fe7c5cb9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558c313192b0 con 0x558c313b9410 2024-09-11T22:55:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.716+0000 7fe7b9ffb640 1 --2- 172.21.15.152:0/3327851732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe7a80a2da0 0x7fe7a80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.717+0000 7fe7c14c3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 0x558c313b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.717+0000 7fe7bbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313c3cb0 0x558c313c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.717+0000 7fe7c14c3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 0x558c313b8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:46726/0 (socket says 172.21.15.152:46726) 2024-09-11T22:55:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.717+0000 7fe7c14c3640 1 -- 172.21.15.152:0/1272492149 learned_addr learned my addr 172.21.15.152:0/1272492149 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.717+0000 7fe7c0cc2640 1 --2- 172.21.15.152:0/1272492149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313b9410 0x558c313c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.717+0000 7fe7c14c3640 1 --2- 172.21.15.152:0/3327851732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe7a80a2da0 0x7fe7a80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.717+0000 7fe7b9ffb640 1 -- 172.21.15.152:0/3327851732 <== mon.2 v2:172.21.15.191:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fe7ac0cfb80 con 0x7fe7bc101370 2024-09-11T22:55:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.717+0000 7fe7c0cc2640 1 -- 172.21.15.152:0/1272492149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 msgr2=0x558c313b8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.717+0000 7fe7c0cc2640 1 --2- 172.21.15.152:0/1272492149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 0x558c313b8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.717+0000 7fe7c0cc2640 1 -- 172.21.15.152:0/1272492149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313c3cb0 msgr2=0x558c313c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.717+0000 7fe7c0cc2640 1 --2- 172.21.15.152:0/1272492149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313c3cb0 0x558c313c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.717+0000 7fe7c0cc2640 1 -- 172.21.15.152:0/1272492149 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c312c4340 con 0x558c313b9410 2024-09-11T22:55:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.717+0000 7fe7c14c3640 1 --2- 172.21.15.152:0/3327851732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe7a80a2da0 0x7fe7a80a5260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4032fd0 tx=0x7fe7a402a420 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.717+0000 7fe7c0cc2640 1 --2- 172.21.15.152:0/1272492149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313b9410 0x558c313c3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4011440 tx=0x7fe7b4011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df46fdddde7171b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.717+0000 7fe7b2ffd640 1 -- 172.21.15.152:0/1272492149 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4004030 con 0x558c313b9410 2024-09-11T22:55:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.717+0000 7fe7b2ffd640 1 -- 172.21.15.152:0/1272492149 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b40041d0 con 0x558c313b9410 2024-09-11T22:55:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.718+0000 7fe7b2ffd640 1 -- 172.21.15.152:0/1272492149 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4010960 con 0x558c313b9410 2024-09-11T22:55:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.719+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/1272492149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313b9410 msgr2=0x558c313c3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.719+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/1272492149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313b9410 0x558c313c3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4011440 tx=0x7fe7b4011900 comp rx=0 tx=0).stop 2024-09-11T22:55:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.719+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/1272492149 shutdown_connections 2024-09-11T22:55:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.719+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/1272492149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313c3cb0 0x558c313c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.719+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/1272492149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313b9410 0x558c313c3770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.719+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/1272492149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 0x558c313b8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.719+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/1272492149 >> 172.21.15.152:0/1272492149 conn(0x558c313b5060 msgr2=0x558c313b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.719+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/1272492149 shutdown_connections 2024-09-11T22:55:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.719+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/1272492149 wait complete. 2024-09-11T22:55:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.719+0000 7fe7c5cb9740 1 -- start start 2024-09-11T22:55:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.720+0000 7fe7c5cb9740 1 Processor -- start 2024-09-11T22:55:28.728 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.720+0000 7fe7c5cb9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 0x558c313c0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.720+0000 7fe7c5cb9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313b9410 0x558c313c1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.720+0000 7fe7c5cb9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313c3cb0 0x558c313c15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.720+0000 7fe7c5cb9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558c313c6880 con 0x558c313b9410 2024-09-11T22:55:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.720+0000 7fe7c5cb9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558c312a6290 con 0x558c313b8af0 2024-09-11T22:55:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.720+0000 7fe7c5cb9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558c313c6a00 con 0x558c313c3cb0 2024-09-11T22:55:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.720+0000 7fe7c5cb9740 10 client.0 _ll_register_callbacks cb 0x558c312d7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.720+0000 7fe7bbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313b9410 0x558c313c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.720+0000 7fe7c14c3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313c3cb0 0x558c313c15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.720+0000 7fe7c0cc2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 0x558c313c0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.720+0000 7fe7c0cc2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 0x558c313c0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:46740/0 (socket says 172.21.15.152:46740) 2024-09-11T22:55:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.720+0000 7fe7c14c3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313c3cb0 0x558c313c15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:57954/0 (socket says 172.21.15.152:57954) 2024-09-11T22:55:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.720+0000 7fe7c0cc2640 1 -- 172.21.15.152:0/3994095924 learned_addr learned my addr 172.21.15.152:0/3994095924 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.720+0000 7fe7bbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313b9410 0x558c313c1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54914/0 (socket says 172.21.15.152:54914) 2024-09-11T22:55:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.721+0000 7fe7c0cc2640 1 -- 172.21.15.152:0/3994095924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313c3cb0 msgr2=0x558c313c15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.721+0000 7fe7c0cc2640 1 --2- 172.21.15.152:0/3994095924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313c3cb0 0x558c313c15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.721+0000 7fe7c0cc2640 1 -- 172.21.15.152:0/3994095924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313b9410 msgr2=0x558c313c1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.721+0000 7fe7c0cc2640 1 --2- 172.21.15.152:0/3994095924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313b9410 0x558c313c1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.721+0000 7fe7c0cc2640 1 -- 172.21.15.152:0/3994095924 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c31030210 con 0x558c313b8af0 2024-09-11T22:55:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.721+0000 7fe7bbfff640 1 --2- 172.21.15.152:0/3994095924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313b9410 0x558c313c1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:55:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.721+0000 7fe7c0cc2640 1 --2- 172.21.15.152:0/3994095924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 0x558c313c0b30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4010680 tx=0x7fe7b4003c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cff1b3c2caf9d73 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.721+0000 7fe79bfff640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-11T22:55:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.721+0000 7fe79bfff640 1 -- 172.21.15.152:0/3994095924 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4027020 con 0x558c313b8af0 2024-09-11T22:55:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.721+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/3994095924 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7b40041d0 con 0x558c313b8af0 2024-09-11T22:55:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.721+0000 7fe79bfff640 1 -- 172.21.15.152:0/3994095924 <== mon.2 v2:172.21.15.191:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fe7b4016040 con 0x558c313b8af0 2024-09-11T22:55:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.721+0000 7fe79bfff640 1 -- 172.21.15.152:0/3994095924 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe7b4032050 con 0x558c313b8af0 2024-09-11T22:55:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.722+0000 7fe7c5cb9740 10 client.25469 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:28.730 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.722+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/3994095924 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c310fc620 con 0x558c313b8af0 2024-09-11T22:55:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.722+0000 7fe79bfff640 1 -- 172.21.15.152:0/3994095924 <== mon.2 v2:172.21.15.191:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b40041d0 con 0x558c313b8af0 2024-09-11T22:55:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.722+0000 7fe79bfff640 1 -- 172.21.15.152:0/3994095924 <== mon.2 v2:172.21.15.191:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe7b4010a80 con 0x558c313b8af0 2024-09-11T22:55:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.722+0000 7fe7c5cb9740 10 client.25469 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.722+0000 7fe7c5cb9740 10 client.25469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.722+0000 7fe7c5cb9740 20 client.25469 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.722+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/3994095924 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558c313d3ad0 0x558c313d5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.722+0000 7fe7c5cb9740 4 client.25469 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:55:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.722+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/3994095924 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c3132aa20 con 0x558c313d3ad0 2024-09-11T22:55:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.722+0000 7fe7bbfff640 1 --2- 172.21.15.152:0/3994095924 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558c313d3ad0 0x558c313d5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.723+0000 7fe79bfff640 10 client.25469 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.723+0000 7fe7bbfff640 1 --2- 172.21.15.152:0/3994095924 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558c313d3ad0 0x558c313d5ec0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c283dd9eb4898354 server_cookie=58626b8e697e63f3 in_seq=0 out_seq=0 2024-09-11T22:55:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.723+0000 7fe79bfff640 1 -- 172.21.15.152:0/3994095924 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c3132aa20 con 0x558c313d3ad0 2024-09-11T22:55:28.731 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.723+0000 7fe79bfff640 10 client.25469 handle_command_reply: tid=0 2024-09-11T22:55:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.724+0000 7fe7c5cb9740 1 client.25469 shutdown 2024-09-11T22:55:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.724+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/3994095924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 msgr2=0x558c313c0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.724+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/3994095924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 0x558c313c0b30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4010680 tx=0x7fe7b4003c90 comp rx=0 tx=0).stop 2024-09-11T22:55:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.724+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/3994095924 shutdown_connections 2024-09-11T22:55:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.724+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/3994095924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313c3cb0 0x558c313c15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.724+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/3994095924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313b9410 0x558c313c1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.724+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/3994095924 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558c313d3ad0 0x558c313d5ec0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.724+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/3994095924 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 0x558c313c0b30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.724+0000 7fe79bfff640 0 client.25469 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.724+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/3994095924 >> 172.21.15.152:0/3994095924 conn(0x558c313b5060 msgr2=0x558c313b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.724+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/3994095924 shutdown_connections 2024-09-11T22:55:28.732 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.724+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/3994095924 wait complete. 2024-09-11T22:55:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.724+0000 7fe7c5cb9740 20 client.25469 trim_cache size 0 max 16384 2024-09-11T22:55:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c5cb9740 1 Processor -- start 2024-09-11T22:55:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c5cb9740 1 -- start start 2024-09-11T22:55:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c5cb9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313b8af0 0x558c313c1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c5cb9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b9410 0x558c313c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c5cb9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313c3cb0 0x558c313db630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c5cb9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1ab4a0 con 0x558c313c3cb0 2024-09-11T22:55:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c5cb9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1acfe0 con 0x558c313b9410 2024-09-11T22:55:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c5cb9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1abb70 con 0x558c313b8af0 2024-09-11T22:55:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c14c3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313c3cb0 0x558c313db630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c14c3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313c3cb0 0x558c313db630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54930/0 (socket says 172.21.15.152:54930) 2024-09-11T22:55:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c14c3640 1 -- 172.21.15.152:0/3654959793 learned_addr learned my addr 172.21.15.152:0/3654959793 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7bbfff640 1 --2- 172.21.15.152:0/3654959793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b9410 0x558c313c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c0cc2640 1 --2- 172.21.15.152:0/3654959793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313b8af0 0x558c313c1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c14c3640 1 -- 172.21.15.152:0/3654959793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313b8af0 msgr2=0x558c313c1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c14c3640 1 --2- 172.21.15.152:0/3654959793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313b8af0 0x558c313c1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c14c3640 1 -- 172.21.15.152:0/3654959793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b9410 msgr2=0x558c313c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c14c3640 1 --2- 172.21.15.152:0/3654959793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b9410 0x558c313c15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7c14c3640 1 -- 172.21.15.152:0/3654959793 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c313bfac0 con 0x558c313c3cb0 2024-09-11T22:55:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.759+0000 7fe7bbfff640 1 --2- 172.21.15.152:0/3654959793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b9410 0x558c313c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.760+0000 7fe7c14c3640 1 --2- 172.21.15.152:0/3654959793 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313c3cb0 0x558c313db630 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4031550 tx=0x7fe7a4040070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f3ade5226e3053 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.760+0000 7fe7b27fc640 1 -- 172.21.15.152:0/3654959793 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a404f070 con 0x558c313c3cb0 2024-09-11T22:55:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.760+0000 7fe7b27fc640 1 -- 172.21.15.152:0/3654959793 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7a4003cd0 con 0x558c313c3cb0 2024-09-11T22:55:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.760+0000 7fe7b27fc640 1 -- 172.21.15.152:0/3654959793 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4035040 con 0x558c313c3cb0 2024-09-11T22:55:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.761+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/3654959793 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313c3cb0 msgr2=0x558c313db630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.761+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/3654959793 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313c3cb0 0x558c313db630 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4031550 tx=0x7fe7a4040070 comp rx=0 tx=0).stop 2024-09-11T22:55:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.761+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/3654959793 shutdown_connections 2024-09-11T22:55:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.761+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/3654959793 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313c3cb0 0x558c313db630 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.761+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/3654959793 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b9410 0x558c313c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.761+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/3654959793 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313b8af0 0x558c313c1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.761+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/3654959793 >> 172.21.15.152:0/3654959793 conn(0x558c313b5060 msgr2=0x558c313c2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.761+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/3654959793 shutdown_connections 2024-09-11T22:55:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.761+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/3654959793 wait complete. 2024-09-11T22:55:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.761+0000 7fe7c5cb9740 1 -- start start 2024-09-11T22:55:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.761+0000 7fe7c5cb9740 1 Processor -- start 2024-09-11T22:55:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7c5cb9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 0x558c313cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7c5cb9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313b9410 0x558c313bd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7c5cb9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313c3cb0 0x558c313bdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7c5cb9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1ac910 con 0x558c313b9410 2024-09-11T22:55:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7c5cb9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1ac240 con 0x558c313b8af0 2024-09-11T22:55:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7c5cb9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc1a9460 con 0x558c313c3cb0 2024-09-11T22:55:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7c5cb9740 10 client.0 _ll_register_callbacks cb 0x7fe7ac005b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7bbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313b9410 0x558c313bd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7bbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313b9410 0x558c313bd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54938/0 (socket says 172.21.15.152:54938) 2024-09-11T22:55:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7bbfff640 1 -- 172.21.15.152:0/2098597607 learned_addr learned my addr 172.21.15.152:0/2098597607 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7c14c3640 1 --2- 172.21.15.152:0/2098597607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313c3cb0 0x558c313bdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7c0cc2640 1 --2- 172.21.15.152:0/2098597607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 0x558c313cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7bbfff640 1 -- 172.21.15.152:0/2098597607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313c3cb0 msgr2=0x558c313bdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7bbfff640 1 --2- 172.21.15.152:0/2098597607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313c3cb0 0x558c313bdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7bbfff640 1 -- 172.21.15.152:0/2098597607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 msgr2=0x558c313cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7bbfff640 1 --2- 172.21.15.152:0/2098597607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 0x558c313cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7bbfff640 1 -- 172.21.15.152:0/2098597607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c313b9240 con 0x558c313b9410 2024-09-11T22:55:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7c0cc2640 1 --2- 172.21.15.152:0/2098597607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 0x558c313cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:55:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe7bbfff640 1 --2- 172.21.15.152:0/2098597607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313b9410 0x558c313bd800 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac098500 tx=0x7fe7ac005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea5deec5d0fa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe79bfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.762+0000 7fe79bfff640 1 -- 172.21.15.152:0/2098597607 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0a04a0 con 0x558c313b9410 2024-09-11T22:55:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.763+0000 7fe79bfff640 1 -- 172.21.15.152:0/2098597607 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fe7ac0670b0 con 0x558c313b9410 2024-09-11T22:55:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.763+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/2098597607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7a4003cd0 con 0x558c313b9410 2024-09-11T22:55:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.763+0000 7fe79bfff640 1 -- 172.21.15.152:0/2098597607 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe7ac0a4d60 con 0x558c313b9410 2024-09-11T22:55:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.763+0000 7fe7c5cb9740 10 client.16215 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.763+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/2098597607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558c3143e600 con 0x558c313b9410 2024-09-11T22:55:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.763+0000 7fe79bfff640 1 -- 172.21.15.152:0/2098597607 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7a4003cd0 con 0x558c313b9410 2024-09-11T22:55:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.763+0000 7fe79bfff640 1 -- 172.21.15.152:0/2098597607 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe7ac0b6060 con 0x558c313b9410 2024-09-11T22:55:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.763+0000 7fe7c5cb9740 10 client.16215 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.763+0000 7fe7c5cb9740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.763+0000 7fe7c5cb9740 20 client.16215 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.763+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/2098597607 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558c3143ec60 0x558c31441080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.763+0000 7fe7c5cb9740 4 client.16215 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:55:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.763+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/2098597607 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c31324d70 con 0x558c3143ec60 2024-09-11T22:55:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.763+0000 7fe7c0cc2640 1 --2- 172.21.15.152:0/2098597607 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558c3143ec60 0x558c31441080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.764+0000 7fe79bfff640 10 client.16215 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.764+0000 7fe7c0cc2640 1 --2- 172.21.15.152:0/2098597607 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558c3143ec60 0x558c31441080 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c759844fee988c19 server_cookie=649e086673fc9095 in_seq=0 out_seq=0 2024-09-11T22:55:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.777+0000 7fe79bfff640 1 -- 172.21.15.152:0/2098597607 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c31324d70 con 0x558c3143ec60 2024-09-11T22:55:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.777+0000 7fe79bfff640 10 client.16215 handle_command_reply: tid=0 2024-09-11T22:55:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.777+0000 7fe7c5cb9740 1 client.16215 shutdown 2024-09-11T22:55:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.777+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/2098597607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313b9410 msgr2=0x558c313bd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.778+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/2098597607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313b9410 0x558c313bd800 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac098500 tx=0x7fe7ac005990 comp rx=0 tx=0).stop 2024-09-11T22:55:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.778+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/2098597607 shutdown_connections 2024-09-11T22:55:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.778+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/2098597607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558c313c3cb0 0x558c313bdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.778+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/2098597607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558c313b9410 0x558c313bd800 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.778+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/2098597607 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558c3143ec60 0x558c31441080 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.778+0000 7fe7c5cb9740 1 --2- 172.21.15.152:0/2098597607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558c313b8af0 0x558c313cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.778+0000 7fe79bfff640 0 client.16215 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.778+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/2098597607 >> 172.21.15.152:0/2098597607 conn(0x558c313b5060 msgr2=0x558c313b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.778+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/2098597607 shutdown_connections 2024-09-11T22:55:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.778+0000 7fe7c5cb9740 1 -- 172.21.15.152:0/2098597607 wait complete. 2024-09-11T22:55:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.778+0000 7fe7c5cb9740 20 client.16215 trim_cache size 0 max 16384 2024-09-11T22:55:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:{ 2024-09-11T22:55:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "message": "", 2024-09-11T22:55:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "return_code": 0 2024-09-11T22:55:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:} 2024-09-11T22:55:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.779+0000 7fe7c2f26640 1 -- 172.21.15.152:0/3327851732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe7a80a2da0 msgr2=0x7fe7a80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.779+0000 7fe7c2f26640 1 --2- 172.21.15.152:0/3327851732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe7a80a2da0 0x7fe7a80a5260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4032fd0 tx=0x7fe7a402a420 comp rx=0 tx=0).stop 2024-09-11T22:55:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.779+0000 7fe7c2f26640 1 -- 172.21.15.152:0/3327851732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7bc101370 msgr2=0x7fe7bc197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.779+0000 7fe7c2f26640 1 --2- 172.21.15.152:0/3327851732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7bc101370 0x7fe7bc197790 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009fd0 tx=0x7fe7ac00ef90 comp rx=0 tx=0).stop 2024-09-11T22:55:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.779+0000 7fe7c2f26640 1 -- 172.21.15.152:0/3327851732 shutdown_connections 2024-09-11T22:55:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.779+0000 7fe7c2f26640 1 --2- 172.21.15.152:0/3327851732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe7a80a2da0 0x7fe7a80a5260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.779+0000 7fe7c2f26640 1 --2- 172.21.15.152:0/3327851732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7bc1069e0 0x7fe7bc19bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.779+0000 7fe7c2f26640 1 --2- 172.21.15.152:0/3327851732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7bc101370 0x7fe7bc197790 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.779+0000 7fe7c2f26640 1 --2- 172.21.15.152:0/3327851732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe7bc1009d0 0x7fe7bc197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.779+0000 7fe7c2f26640 1 -- 172.21.15.152:0/3327851732 >> 172.21.15.152:0/3327851732 conn(0x7fe7bc0fc6f0 msgr2=0x7fe7bc0fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.779+0000 7fe7c2f26640 1 -- 172.21.15.152:0/3327851732 shutdown_connections 2024-09-11T22:55:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:28.779+0000 7fe7c2f26640 1 -- 172.21.15.152:0/3327851732 wait complete. 2024-09-11T22:55:28.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.865+0000 7f4ac8624640 1 -- 172.21.15.152:0/188776472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ac0100100 msgr2=0x7f4ac0100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.865+0000 7f4ac8624640 1 --2- 172.21.15.152:0/188776472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ac0100100 0x7f4ac0100560 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009900 tx=0x7f4abc02a410 comp rx=0 tx=0).stop 2024-09-11T22:55:28.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.865+0000 7f4ac8624640 1 -- 172.21.15.152:0/188776472 shutdown_connections 2024-09-11T22:55:28.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.865+0000 7f4ac8624640 1 --2- 172.21.15.152:0/188776472 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ac0100aa0 0x7f4ac010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.865+0000 7f4ac8624640 1 --2- 172.21.15.152:0/188776472 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ac0100100 0x7f4ac0100560 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.865+0000 7f4ac8624640 1 --2- 172.21.15.152:0/188776472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ac0106b80 0x7f4ac0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.865+0000 7f4ac8624640 1 -- 172.21.15.152:0/188776472 >> 172.21.15.152:0/188776472 conn(0x7f4ac0077230 msgr2=0x7f4ac00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:28.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.865+0000 7f4ac8624640 1 -- 172.21.15.152:0/188776472 shutdown_connections 2024-09-11T22:55:28.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.865+0000 7f4ac8624640 1 -- 172.21.15.152:0/188776472 wait complete. 2024-09-11T22:55:28.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.866+0000 7f4ac8624640 1 Processor -- start 2024-09-11T22:55:28.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.866+0000 7f4ac8624640 1 -- start start 2024-09-11T22:55:28.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.866+0000 7f4ac8624640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ac0100100 0x7f4ac0198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.866+0000 7f4ac8624640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ac0100aa0 0x7f4ac0198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.866+0000 7f4ac8624640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ac0106b80 0x7f4ac019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.866+0000 7f4ac8624640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac0112070 con 0x7f4ac0100100 2024-09-11T22:55:28.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.866+0000 7f4ac8624640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac0111ef0 con 0x7f4ac0100aa0 2024-09-11T22:55:28.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.866+0000 7f4ac8624640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01121f0 con 0x7f4ac0106b80 2024-09-11T22:55:28.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.866+0000 7f4ac63c0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ac0100100 0x7f4ac0198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.866+0000 7f4ac63c0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ac0100100 0x7f4ac0198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54958/0 (socket says 172.21.15.152:54958) 2024-09-11T22:55:28.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.866+0000 7f4ac63c0640 1 -- 172.21.15.152:0/794412535 learned_addr learned my addr 172.21.15.152:0/794412535 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:28.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.866+0000 7f4ac5bbf640 1 --2- 172.21.15.152:0/794412535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ac0100aa0 0x7f4ac0198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.866+0000 7f4ac6bc1640 1 --2- 172.21.15.152:0/794412535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ac0106b80 0x7f4ac019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.867+0000 7f4ac63c0640 1 -- 172.21.15.152:0/794412535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ac0106b80 msgr2=0x7f4ac019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.867+0000 7f4ac63c0640 1 --2- 172.21.15.152:0/794412535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ac0106b80 0x7f4ac019cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.867+0000 7f4ac63c0640 1 -- 172.21.15.152:0/794412535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ac0100aa0 msgr2=0x7f4ac0198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.867+0000 7f4ac63c0640 1 --2- 172.21.15.152:0/794412535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ac0100aa0 0x7f4ac0198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.867+0000 7f4ac63c0640 1 -- 172.21.15.152:0/794412535 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac019d600 con 0x7f4ac0100100 2024-09-11T22:55:28.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.867+0000 7f4ac6bc1640 1 --2- 172.21.15.152:0/794412535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ac0106b80 0x7f4ac019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:28.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.867+0000 7f4ac63c0640 1 --2- 172.21.15.152:0/794412535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ac0100100 0x7f4ac0198650 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400b4a0 tx=0x7f4ab400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.867+0000 7f4ab37fe640 1 -- 172.21.15.152:0/794412535 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4014070 con 0x7f4ac0100100 2024-09-11T22:55:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.867+0000 7f4ac8624640 1 -- 172.21.15.152:0/794412535 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac019d8f0 con 0x7f4ac0100100 2024-09-11T22:55:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.867+0000 7f4ab37fe640 1 -- 172.21.15.152:0/794412535 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab4010af0 con 0x7f4ac0100100 2024-09-11T22:55:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.867+0000 7f4ab37fe640 1 -- 172.21.15.152:0/794412535 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4017d10 con 0x7f4ac0100100 2024-09-11T22:55:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.867+0000 7f4ac8624640 1 -- 172.21.15.152:0/794412535 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac01a51d0 con 0x7f4ac0100100 2024-09-11T22:55:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.869+0000 7f4ab37fe640 1 -- 172.21.15.152:0/794412535 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4ab4017070 con 0x7f4ac0100100 2024-09-11T22:55:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.870+0000 7f4ab37fe640 1 --2- 172.21.15.152:0/794412535 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4aac0a2cf0 0x7f4aac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.870+0000 7f4ab37fe640 1 -- 172.21.15.152:0/794412535 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4ab40ce6c0 con 0x7f4ac0100100 2024-09-11T22:55:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.870+0000 7f4ac5bbf640 1 --2- 172.21.15.152:0/794412535 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4aac0a2cf0 0x7f4aac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.870+0000 7f4aca9fd740 1 Processor -- start 2024-09-11T22:55:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4aca9fd740 1 -- start start 2024-09-11T22:55:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4aca9fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8866b40 0x560df8865090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4aca9fd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df88655d0 0x560df8865a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4aca9fd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8870370 0x560df8872760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4aca9fd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560df87cb2f0 con 0x560df8870370 2024-09-11T22:55:28.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4aca9fd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560df8772cf0 con 0x560df88655d0 2024-09-11T22:55:28.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4aca9fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560df87c72b0 con 0x560df8866b40 2024-09-11T22:55:28.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4ac6bc1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8870370 0x560df8872760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4ac6bc1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8870370 0x560df8872760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54966/0 (socket says 172.21.15.152:54966) 2024-09-11T22:55:28.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4ac6bc1640 1 -- 172.21.15.152:0/2474526887 learned_addr learned my addr 172.21.15.152:0/2474526887 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:28.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4ac5bbf640 1 --2- 172.21.15.152:0/794412535 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4aac0a2cf0 0x7f4aac0a51b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009230 tx=0x7f4abc0058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:28.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4ac63c0640 1 --2- 172.21.15.152:0/2474526887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8866b40 0x560df8865090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4ac5bbf640 1 --2- 172.21.15.152:0/2474526887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df88655d0 0x560df8865a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4ac6bc1640 1 -- 172.21.15.152:0/2474526887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8866b40 msgr2=0x560df8865090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:28.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4ac6bc1640 1 --2- 172.21.15.152:0/2474526887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8866b40 0x560df8865090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4ac6bc1640 1 -- 172.21.15.152:0/2474526887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df88655d0 msgr2=0x560df8865a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:28.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.871+0000 7f4ac6bc1640 1 --2- 172.21.15.152:0/2474526887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df88655d0 0x560df8865a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.872+0000 7f4ac6bc1640 1 -- 172.21.15.152:0/2474526887 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df87573a0 con 0x560df8870370 2024-09-11T22:55:28.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.872+0000 7f4ac5bbf640 1 --2- 172.21.15.152:0/2474526887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df88655d0 0x560df8865a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:55:28.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.872+0000 7f4ac6bc1640 1 --2- 172.21.15.152:0/2474526887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8870370 0x560df8872760 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009f60 tx=0x7f4ab8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f7376158d06623 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:28.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.872+0000 7f4ab0ff9640 1 -- 172.21.15.152:0/2474526887 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8016070 con 0x560df8870370 2024-09-11T22:55:28.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.872+0000 7f4ab0ff9640 1 -- 172.21.15.152:0/2474526887 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab801bb40 con 0x560df8870370 2024-09-11T22:55:28.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.872+0000 7f4ab0ff9640 1 -- 172.21.15.152:0/2474526887 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8012c70 con 0x560df8870370 2024-09-11T22:55:28.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.873+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2474526887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8870370 msgr2=0x560df8872760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.873+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/2474526887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8870370 0x560df8872760 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009f60 tx=0x7f4ab8009f90 comp rx=0 tx=0).stop 2024-09-11T22:55:28.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.873+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2474526887 shutdown_connections 2024-09-11T22:55:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.873+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/2474526887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8870370 0x560df8872760 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.873+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/2474526887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df88655d0 0x560df8865a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.873+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/2474526887 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8866b40 0x560df8865090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.873+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2474526887 >> 172.21.15.152:0/2474526887 conn(0x560df882e310 msgr2=0x560df8861c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.873+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2474526887 shutdown_connections 2024-09-11T22:55:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.873+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2474526887 wait complete. 2024-09-11T22:55:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.873+0000 7f4aca9fd740 1 -- start start 2024-09-11T22:55:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.873+0000 7f4aca9fd740 1 Processor -- start 2024-09-11T22:55:28.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4aca9fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df88655d0 0x560df886fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4aca9fd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8866b40 0x560df886d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4aca9fd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df8870370 0x560df886d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4aca9fd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560df8873070 con 0x560df8866b40 2024-09-11T22:55:28.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4aca9fd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560df8786060 con 0x560df8870370 2024-09-11T22:55:28.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4aca9fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560df88731f0 con 0x560df88655d0 2024-09-11T22:55:28.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4aca9fd740 10 client.0 _ll_register_callbacks cb 0x560df84d98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:28.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4ac5bbf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8866b40 0x560df886d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4ac6bc1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df8870370 0x560df886d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4ac6bc1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df8870370 0x560df886d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:46806/0 (socket says 172.21.15.152:46806) 2024-09-11T22:55:28.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4ac5bbf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8866b40 0x560df886d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:54982/0 (socket says 172.21.15.152:54982) 2024-09-11T22:55:28.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4ac6bc1640 1 -- 172.21.15.152:0/2525128009 learned_addr learned my addr 172.21.15.152:0/2525128009 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:28.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4ac63c0640 1 --2- 172.21.15.152:0/2525128009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df88655d0 0x560df886fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4ac6bc1640 1 -- 172.21.15.152:0/2525128009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df88655d0 msgr2=0x560df886fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:28.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4ac6bc1640 1 --2- 172.21.15.152:0/2525128009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df88655d0 0x560df886fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4ac6bc1640 1 -- 172.21.15.152:0/2525128009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8866b40 msgr2=0x560df886d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:28.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4ac6bc1640 1 --2- 172.21.15.152:0/2525128009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8866b40 0x560df886d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4ac6bc1640 1 -- 172.21.15.152:0/2525128009 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df853c3c0 con 0x560df8870370 2024-09-11T22:55:28.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4ac6bc1640 1 --2- 172.21.15.152:0/2525128009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df8870370 0x560df886d5d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8012830 tx=0x7f4ab8009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bba1ef701a71f8e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:28.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.874+0000 7f4aa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-11T22:55:28.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.875+0000 7f4aa9ffb640 1 -- 172.21.15.152:0/2525128009 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8016070 con 0x560df8870370 2024-09-11T22:55:28.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.875+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2525128009 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ab801bb40 con 0x560df8870370 2024-09-11T22:55:28.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.875+0000 7f4aa9ffb640 1 -- 172.21.15.152:0/2525128009 <== mon.2 v2:172.21.15.191:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4ab8004cd0 con 0x560df8870370 2024-09-11T22:55:28.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.875+0000 7f4aa9ffb640 1 -- 172.21.15.152:0/2525128009 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4ab8012a30 con 0x560df8870370 2024-09-11T22:55:28.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.875+0000 7f4aca9fd740 10 client.25484 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:28.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.875+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2525128009 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560df887ea60 con 0x560df8870370 2024-09-11T22:55:28.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.875+0000 7f4aa9ffb640 1 -- 172.21.15.152:0/2525128009 <== mon.2 v2:172.21.15.191:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab801bb40 con 0x560df8870370 2024-09-11T22:55:28.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.875+0000 7f4aa9ffb640 1 -- 172.21.15.152:0/2525128009 <== mon.2 v2:172.21.15.191:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4ab800f040 con 0x560df8870370 2024-09-11T22:55:28.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.875+0000 7f4aca9fd740 10 client.25484 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:28.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.875+0000 7f4aca9fd740 10 client.25484 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:55:28.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.875+0000 7f4aca9fd740 20 client.25484 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:28.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.875+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/2525128009 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560df887fe10 0x560df8882200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.875+0000 7f4aca9fd740 4 client.25484 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:55:28.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.875+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2525128009 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560df87d8860 con 0x560df887fe10 2024-09-11T22:55:28.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.876+0000 7f4ac63c0640 1 --2- 172.21.15.152:0/2525128009 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560df887fe10 0x560df8882200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.876+0000 7f4aa9ffb640 10 client.25484 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:55:28.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.876+0000 7f4ac63c0640 1 --2- 172.21.15.152:0/2525128009 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560df887fe10 0x560df8882200 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a5834b6407a0386 server_cookie=59e89c3e053246be in_seq=0 out_seq=0 2024-09-11T22:55:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.877+0000 7f4aa9ffb640 1 -- 172.21.15.152:0/2525128009 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560df87d8860 con 0x560df887fe10 2024-09-11T22:55:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.877+0000 7f4aa9ffb640 10 client.25484 handle_command_reply: tid=0 2024-09-11T22:55:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.877+0000 7f4aca9fd740 1 client.25484 shutdown 2024-09-11T22:55:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.877+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2525128009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df8870370 msgr2=0x560df886d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.877+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/2525128009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df8870370 0x560df886d5d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8012830 tx=0x7f4ab8009cd0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.877+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2525128009 shutdown_connections 2024-09-11T22:55:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.877+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/2525128009 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560df887fe10 0x560df8882200 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.877+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/2525128009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df8870370 0x560df886d5d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.877+0000 7f4aa9ffb640 0 client.25484 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:55:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.877+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/2525128009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8866b40 0x560df886d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.877+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/2525128009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df88655d0 0x560df886fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.877+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2525128009 >> 172.21.15.152:0/2525128009 conn(0x560df882e310 msgr2=0x560df8867960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.877+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2525128009 shutdown_connections 2024-09-11T22:55:28.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.877+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2525128009 wait complete. 2024-09-11T22:55:28.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.878+0000 7f4aca9fd740 20 client.25484 trim_cache size 0 max 16384 2024-09-11T22:55:28.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.891+0000 7f4aca9fd740 1 Processor -- start 2024-09-11T22:55:28.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.891+0000 7f4aca9fd740 1 -- start start 2024-09-11T22:55:28.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.891+0000 7f4aca9fd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df88655d0 0x560df887c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.891+0000 7f4aca9fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8866b40 0x560df8887990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.891+0000 7f4aca9fd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df8870370 0x560df8887ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.891+0000 7f4aca9fd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01ad8b0 con 0x560df88655d0 2024-09-11T22:55:28.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.891+0000 7f4aca9fd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01af3f0 con 0x560df8870370 2024-09-11T22:55:28.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.891+0000 7f4aca9fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01adf80 con 0x560df8866b40 2024-09-11T22:55:28.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.892+0000 7f4ac5bbf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8866b40 0x560df8887990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.892+0000 7f4ac5bbf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8866b40 0x560df8887990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:58026/0 (socket says 172.21.15.152:58026) 2024-09-11T22:55:28.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.892+0000 7f4ac63c0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df88655d0 0x560df887c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.892+0000 7f4ac5bbf640 1 -- 172.21.15.152:0/3731576080 learned_addr learned my addr 172.21.15.152:0/3731576080 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:28.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.892+0000 7f4ac6bc1640 1 --2- 172.21.15.152:0/3731576080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df8870370 0x560df8887ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.892+0000 7f4ac63c0640 1 -- 172.21.15.152:0/3731576080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8866b40 msgr2=0x560df8887990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:28.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.892+0000 7f4ac63c0640 1 --2- 172.21.15.152:0/3731576080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8866b40 0x560df8887990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.892+0000 7f4ac63c0640 1 -- 172.21.15.152:0/3731576080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df8870370 msgr2=0x560df8887ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:28.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.892+0000 7f4ac63c0640 1 --2- 172.21.15.152:0/3731576080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df8870370 0x560df8887ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.892+0000 7f4ac63c0640 1 -- 172.21.15.152:0/3731576080 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df886b550 con 0x560df88655d0 2024-09-11T22:55:28.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.892+0000 7f4ac5bbf640 1 --2- 172.21.15.152:0/3731576080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8866b40 0x560df8887990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:55:28.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.892+0000 7f4ac6bc1640 1 --2- 172.21.15.152:0/3731576080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df8870370 0x560df8887ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:55:28.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.892+0000 7f4ac63c0640 1 --2- 172.21.15.152:0/3731576080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df88655d0 0x560df887c310 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40cbf70 tx=0x7f4ab401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebe37d6398f48b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:28.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.892+0000 7f4aabfff640 1 -- 172.21.15.152:0/3731576080 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab401e040 con 0x560df88655d0 2024-09-11T22:55:28.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.892+0000 7f4aabfff640 1 -- 172.21.15.152:0/3731576080 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab4099040 con 0x560df88655d0 2024-09-11T22:55:28.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.892+0000 7f4aabfff640 1 -- 172.21.15.152:0/3731576080 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab40cc270 con 0x560df88655d0 2024-09-11T22:55:28.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.893+0000 7f4aca9fd740 1 -- 172.21.15.152:0/3731576080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df88655d0 msgr2=0x560df887c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.893+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/3731576080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df88655d0 0x560df887c310 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40cbf70 tx=0x7f4ab401b040 comp rx=0 tx=0).stop 2024-09-11T22:55:28.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.893+0000 7f4aca9fd740 1 -- 172.21.15.152:0/3731576080 shutdown_connections 2024-09-11T22:55:28.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.893+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/3731576080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df8870370 0x560df8887ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.893+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/3731576080 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8866b40 0x560df8887990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.893+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/3731576080 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df88655d0 0x560df887c310 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.893+0000 7f4aca9fd740 1 -- 172.21.15.152:0/3731576080 >> 172.21.15.152:0/3731576080 conn(0x560df882e310 msgr2=0x560df8868260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:28.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.893+0000 7f4aca9fd740 1 -- 172.21.15.152:0/3731576080 shutdown_connections 2024-09-11T22:55:28.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.893+0000 7f4aca9fd740 1 -- 172.21.15.152:0/3731576080 wait complete. 2024-09-11T22:55:28.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.894+0000 7f4aca9fd740 1 -- start start 2024-09-11T22:55:28.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.894+0000 7f4aca9fd740 1 Processor -- start 2024-09-11T22:55:28.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.894+0000 7f4aca9fd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df88655d0 0x560df88e2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.894+0000 7f4aca9fd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8866b40 0x560df88e3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.894+0000 7f4aca9fd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8870370 0x560df88e3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.894+0000 7f4aca9fd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01aed20 con 0x560df8866b40 2024-09-11T22:55:28.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.894+0000 7f4aca9fd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01ae650 con 0x560df88655d0 2024-09-11T22:55:28.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.894+0000 7f4aca9fd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac01ab870 con 0x560df8870370 2024-09-11T22:55:28.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.894+0000 7f4aca9fd740 10 client.0 _ll_register_callbacks cb 0x7f4ab40092d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:28.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.894+0000 7f4ac5bbf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8866b40 0x560df88e3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.894+0000 7f4ac5bbf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8866b40 0x560df88e3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55004/0 (socket says 172.21.15.152:55004) 2024-09-11T22:55:28.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.894+0000 7f4ac5bbf640 1 -- 172.21.15.152:0/2435233424 learned_addr learned my addr 172.21.15.152:0/2435233424 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:28.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.894+0000 7f4ac6bc1640 1 --2- 172.21.15.152:0/2435233424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8870370 0x560df88e3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.894+0000 7f4ac63c0640 1 --2- 172.21.15.152:0/2435233424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df88655d0 0x560df88e2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.895+0000 7f4ac5bbf640 1 -- 172.21.15.152:0/2435233424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8870370 msgr2=0x560df88e3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:28.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.895+0000 7f4ac5bbf640 1 --2- 172.21.15.152:0/2435233424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8870370 0x560df88e3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.895+0000 7f4ac5bbf640 1 -- 172.21.15.152:0/2435233424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df88655d0 msgr2=0x560df88e2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:28.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.895+0000 7f4ac5bbf640 1 --2- 172.21.15.152:0/2435233424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df88655d0 0x560df88e2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.895+0000 7f4ac5bbf640 1 -- 172.21.15.152:0/2435233424 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df8877760 con 0x560df8866b40 2024-09-11T22:55:28.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.895+0000 7f4ac5bbf640 1 --2- 172.21.15.152:0/2435233424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8866b40 0x560df88e3480 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005940 tx=0x7f4abc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55feeffd50817696 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:28.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.895+0000 7f4aa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:28.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.895+0000 7f4aa9ffb640 1 -- 172.21.15.152:0/2435233424 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc035040 con 0x560df8866b40 2024-09-11T22:55:28.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.895+0000 7f4aa9ffb640 1 -- 172.21.15.152:0/2435233424 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4abc003940 con 0x560df8866b40 2024-09-11T22:55:28.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.895+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2435233424 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ab4099040 con 0x560df8866b40 2024-09-11T22:55:28.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.895+0000 7f4aa9ffb640 1 -- 172.21.15.152:0/2435233424 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4abc003b40 con 0x560df8866b40 2024-09-11T22:55:28.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.895+0000 7f4aca9fd740 10 client.16233 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:28.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.895+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2435233424 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560df8878a90 con 0x560df8866b40 2024-09-11T22:55:28.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.895+0000 7f4aa9ffb640 1 -- 172.21.15.152:0/2435233424 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab4099040 con 0x560df8866b40 2024-09-11T22:55:28.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.896+0000 7f4aa9ffb640 1 -- 172.21.15.152:0/2435233424 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4abc04a430 con 0x560df8866b40 2024-09-11T22:55:28.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.896+0000 7f4aca9fd740 10 client.16233 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:28.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.896+0000 7f4aca9fd740 10 client.16233 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:55:28.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.896+0000 7f4aca9fd740 20 client.16233 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:28.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.896+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/2435233424 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560df88f3560 0x560df88f5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:28.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.896+0000 7f4aca9fd740 4 client.16233 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:55:28.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.896+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2435233424 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560df87d2d70 con 0x560df88f3560 2024-09-11T22:55:28.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.896+0000 7f4ac63c0640 1 --2- 172.21.15.152:0/2435233424 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560df88f3560 0x560df88f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:28.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.896+0000 7f4ac63c0640 1 --2- 172.21.15.152:0/2435233424 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560df88f3560 0x560df88f5950 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9edf79ec48360fe server_cookie=34d6eb674096bbeb in_seq=0 out_seq=0 2024-09-11T22:55:28.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.896+0000 7f4aa9ffb640 10 client.16233 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:55:28.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.896+0000 7f4aa9ffb640 1 -- 172.21.15.152:0/2435233424 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560df87d2d70 con 0x560df88f3560 2024-09-11T22:55:28.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.896+0000 7f4aa9ffb640 10 client.16233 handle_command_reply: tid=0 2024-09-11T22:55:28.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.896+0000 7f4aca9fd740 1 client.16233 shutdown 2024-09-11T22:55:28.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.897+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2435233424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8866b40 msgr2=0x560df88e3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.897+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/2435233424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8866b40 0x560df88e3480 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005940 tx=0x7f4abc009260 comp rx=0 tx=0).stop 2024-09-11T22:55:28.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.897+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2435233424 shutdown_connections 2024-09-11T22:55:28.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.897+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/2435233424 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560df88f3560 0x560df88f5950 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.897+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/2435233424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560df8870370 0x560df88e3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.897+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/2435233424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560df8866b40 0x560df88e3480 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.897+0000 7f4aca9fd740 1 --2- 172.21.15.152:0/2435233424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560df88655d0 0x560df88e2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.897+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2435233424 >> 172.21.15.152:0/2435233424 conn(0x560df882e310 msgr2=0x560df886b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:28.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.897+0000 7f4aa9ffb640 0 client.16233 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:55:28.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.897+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2435233424 shutdown_connections 2024-09-11T22:55:28.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.897+0000 7f4aca9fd740 1 -- 172.21.15.152:0/2435233424 wait complete. 2024-09-11T22:55:28.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.897+0000 7f4aca9fd740 20 client.16233 trim_cache size 0 max 16384 2024-09-11T22:55:28.914 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:55:28.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.897+0000 7f4ac8624640 1 -- 172.21.15.152:0/794412535 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4aac0a2cf0 msgr2=0x7f4aac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.897+0000 7f4ac8624640 1 --2- 172.21.15.152:0/794412535 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4aac0a2cf0 0x7f4aac0a51b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009230 tx=0x7f4abc0058d0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.897+0000 7f4ac8624640 1 -- 172.21.15.152:0/794412535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ac0100100 msgr2=0x7f4ac0198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:28.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.897+0000 7f4ac8624640 1 --2- 172.21.15.152:0/794412535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ac0100100 0x7f4ac0198650 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f4ab400b4a0 tx=0x7f4ab400b960 comp rx=0 tx=0).stop 2024-09-11T22:55:28.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.898+0000 7f4ac8624640 1 -- 172.21.15.152:0/794412535 shutdown_connections 2024-09-11T22:55:28.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.898+0000 7f4ac8624640 1 --2- 172.21.15.152:0/794412535 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4aac0a2cf0 0x7f4aac0a51b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.898+0000 7f4ac8624640 1 --2- 172.21.15.152:0/794412535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ac0106b80 0x7f4ac019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.898+0000 7f4ac8624640 1 --2- 172.21.15.152:0/794412535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4ac0100aa0 0x7f4ac0198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.898+0000 7f4ac8624640 1 --2- 172.21.15.152:0/794412535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4ac0100100 0x7f4ac0198650 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:28.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.898+0000 7f4ac8624640 1 -- 172.21.15.152:0/794412535 >> 172.21.15.152:0/794412535 conn(0x7f4ac0077230 msgr2=0x7f4ac0104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:28.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.898+0000 7f4ac8624640 1 -- 172.21.15.152:0/794412535 shutdown_connections 2024-09-11T22:55:28.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:28.898+0000 7f4ac8624640 1 -- 172.21.15.152:0/794412535 wait complete. 2024-09-11T22:55:29.047 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:55:29.047 INFO:tasks.workunit.client.0.smithi152.stdout: Children see throughput for 1 readers = 7528049.00 kB/sec 2024-09-11T22:55:29.047 INFO:tasks.workunit.client.0.smithi152.stdout: Parent sees throughput for 1 readers = 7526774.97 kB/sec 2024-09-11T22:55:29.047 INFO:tasks.workunit.client.0.smithi152.stdout: Min throughput per process = 7528049.00 kB/sec 2024-09-11T22:55:29.047 INFO:tasks.workunit.client.0.smithi152.stdout: Max throughput per process = 7528049.00 kB/sec 2024-09-11T22:55:29.047 INFO:tasks.workunit.client.0.smithi152.stdout: Avg throughput per process = 7528049.00 kB/sec 2024-09-11T22:55:29.048 INFO:tasks.workunit.client.0.smithi152.stdout: Min xfer = 10485760.00 kB 2024-09-11T22:55:29.905 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:55:30.152 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:55:30.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:30 smithi191 ceph-mon[35211]: pgmap v81: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 625 MiB/s wr, 261 op/s 2024-09-11T22:55:30.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:30 smithi204 ceph-mon[35454]: pgmap v81: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 625 MiB/s wr, 261 op/s 2024-09-11T22:55:30.572 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:30 smithi152 ceph-mon[31181]: pgmap v81: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 625 MiB/s wr, 261 op/s 2024-09-11T22:55:30.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.653+0000 7f9afa346640 1 -- 172.21.15.152:0/3466412226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9af4101420 msgr2=0x7f9af410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:30.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.653+0000 7f9afa346640 1 --2- 172.21.15.152:0/3466412226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9af4101420 0x7f9af410d950 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009cd0 tx=0x7f9aec02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.653+0000 7f9afa346640 1 -- 172.21.15.152:0/3466412226 shutdown_connections 2024-09-11T22:55:30.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.653+0000 7f9afa346640 1 --2- 172.21.15.152:0/3466412226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9af4101420 0x7f9af410d950 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.653+0000 7f9afa346640 1 --2- 172.21.15.152:0/3466412226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9af4100a80 0x7f9af4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.653+0000 7f9afa346640 1 --2- 172.21.15.152:0/3466412226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9af4106a80 0x7f9af4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.653+0000 7f9afa346640 1 -- 172.21.15.152:0/3466412226 >> 172.21.15.152:0/3466412226 conn(0x7f9af40fc7a0 msgr2=0x7f9af40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:30.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.653+0000 7f9afa346640 1 -- 172.21.15.152:0/3466412226 shutdown_connections 2024-09-11T22:55:30.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.653+0000 7f9afa346640 1 -- 172.21.15.152:0/3466412226 wait complete. 2024-09-11T22:55:30.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.654+0000 7f9afa346640 1 Processor -- start 2024-09-11T22:55:30.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.654+0000 7f9afa346640 1 -- start start 2024-09-11T22:55:30.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.654+0000 7f9afa346640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9af4100a80 0x7f9af41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.654+0000 7f9afa346640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9af4101420 0x7f9af4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.654+0000 7f9afa346640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9af4106a80 0x7f9af41a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.654+0000 7f9afa346640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4110ea0 con 0x7f9af4101420 2024-09-11T22:55:30.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.654+0000 7f9afa346640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4110d20 con 0x7f9af4100a80 2024-09-11T22:55:30.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.654+0000 7f9afa346640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9af4111020 con 0x7f9af4106a80 2024-09-11T22:55:30.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.654+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9af4101420 0x7f9af4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.654+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9af4101420 0x7f9af4199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55028/0 (socket says 172.21.15.152:55028) 2024-09-11T22:55:30.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.654+0000 7f9af37fe640 1 -- 172.21.15.152:0/1236679554 learned_addr learned my addr 172.21.15.152:0/1236679554 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:30.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.655+0000 7f9af3fff640 1 --2- 172.21.15.152:0/1236679554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9af4100a80 0x7f9af41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.655+0000 7f9af88e3640 1 --2- 172.21.15.152:0/1236679554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9af4106a80 0x7f9af41a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.655+0000 7f9af37fe640 1 -- 172.21.15.152:0/1236679554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9af4106a80 msgr2=0x7f9af41a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:30.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.655+0000 7f9af37fe640 1 --2- 172.21.15.152:0/1236679554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9af4106a80 0x7f9af41a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.655+0000 7f9af37fe640 1 -- 172.21.15.152:0/1236679554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9af4100a80 msgr2=0x7f9af41992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:30.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.655+0000 7f9af37fe640 1 --2- 172.21.15.152:0/1236679554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9af4100a80 0x7f9af41992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.655+0000 7f9af37fe640 1 -- 172.21.15.152:0/1236679554 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af41a48b0 con 0x7f9af4101420 2024-09-11T22:55:30.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.655+0000 7f9af88e3640 1 --2- 172.21.15.152:0/1236679554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9af4106a80 0x7f9af41a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:55:30.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.655+0000 7f9af37fe640 1 --2- 172.21.15.152:0/1236679554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9af4101420 0x7f9af4199830 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800b4a0 tx=0x7f9ae800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:30.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.655+0000 7f9af17fa640 1 -- 172.21.15.152:0/1236679554 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8014070 con 0x7f9af4101420 2024-09-11T22:55:30.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.655+0000 7f9afa346640 1 -- 172.21.15.152:0/1236679554 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af41a4a80 con 0x7f9af4101420 2024-09-11T22:55:30.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.655+0000 7f9af17fa640 1 -- 172.21.15.152:0/1236679554 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae8010af0 con 0x7f9af4101420 2024-09-11T22:55:30.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.655+0000 7f9af17fa640 1 -- 172.21.15.152:0/1236679554 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8017d10 con 0x7f9af4101420 2024-09-11T22:55:30.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.655+0000 7f9afa346640 1 -- 172.21.15.152:0/1236679554 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af41a4f90 con 0x7f9af4101420 2024-09-11T22:55:30.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.657+0000 7f9af17fa640 1 -- 172.21.15.152:0/1236679554 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9ae8017070 con 0x7f9af4101420 2024-09-11T22:55:30.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.657+0000 7f9afc71f740 1 Processor -- start 2024-09-11T22:55:30.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9afc71f740 1 -- start start 2024-09-11T22:55:30.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9afc71f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d94b40 0x5622a5d93090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9afc71f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d935d0 0x5622a5d93a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9afc71f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d9e370 0x5622a5da0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9afc71f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5622a5cf92f0 con 0x5622a5d94b40 2024-09-11T22:55:30.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9afc71f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5622a5ca0cf0 con 0x5622a5d9e370 2024-09-11T22:55:30.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9afc71f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5622a5cf52b0 con 0x5622a5d935d0 2024-09-11T22:55:30.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9af88e3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d94b40 0x5622a5d93090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9af88e3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d94b40 0x5622a5d93090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55036/0 (socket says 172.21.15.152:55036) 2024-09-11T22:55:30.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9af88e3640 1 -- 172.21.15.152:0/3715844916 learned_addr learned my addr 172.21.15.152:0/3715844916 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:30.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9af17fa640 1 --2- 172.21.15.152:0/1236679554 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9adc0a2cf0 0x7f9adc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9af3fff640 1 --2- 172.21.15.152:0/3715844916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d935d0 0x5622a5d93a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9af37fe640 1 --2- 172.21.15.152:0/3715844916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d9e370 0x5622a5da0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9af88e3640 1 --2- 172.21.15.152:0/1236679554 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9adc0a2cf0 0x7f9adc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9af17fa640 1 -- 172.21.15.152:0/1236679554 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f9ae80ce7a0 con 0x7f9af4101420 2024-09-11T22:55:30.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9af88e3640 1 -- 172.21.15.152:0/3715844916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d935d0 msgr2=0x5622a5d93a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:30.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9af88e3640 1 --2- 172.21.15.152:0/3715844916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d935d0 0x5622a5d93a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9af88e3640 1 -- 172.21.15.152:0/3715844916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d9e370 msgr2=0x5622a5da0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:30.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9af88e3640 1 --2- 172.21.15.152:0/3715844916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d9e370 0x5622a5da0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.658+0000 7f9af88e3640 1 -- 172.21.15.152:0/3715844916 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622a5c853a0 con 0x5622a5d94b40 2024-09-11T22:55:30.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.659+0000 7f9af88e3640 1 --2- 172.21.15.152:0/3715844916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d94b40 0x5622a5d93090 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009430 tx=0x7f9aec031c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d292f751930d9a8b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:30.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.659+0000 7f9ada7fc640 1 -- 172.21.15.152:0/3715844916 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec04f070 con 0x5622a5d94b40 2024-09-11T22:55:30.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.659+0000 7f9ada7fc640 1 -- 172.21.15.152:0/3715844916 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aec036040 con 0x5622a5d94b40 2024-09-11T22:55:30.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.659+0000 7f9ada7fc640 1 -- 172.21.15.152:0/3715844916 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec0309a0 con 0x5622a5d94b40 2024-09-11T22:55:30.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.659+0000 7f9af88e3640 1 --2- 172.21.15.152:0/1236679554 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9adc0a2cf0 0x7f9adc0a51b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9aec04c000 tx=0x7f9aec048000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:30.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.659+0000 7f9afc71f740 1 -- 172.21.15.152:0/3715844916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d94b40 msgr2=0x5622a5d93090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:30.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.659+0000 7f9afc71f740 1 --2- 172.21.15.152:0/3715844916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d94b40 0x5622a5d93090 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009430 tx=0x7f9aec031c00 comp rx=0 tx=0).stop 2024-09-11T22:55:30.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.659+0000 7f9afc71f740 1 -- 172.21.15.152:0/3715844916 shutdown_connections 2024-09-11T22:55:30.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.659+0000 7f9afc71f740 1 --2- 172.21.15.152:0/3715844916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d9e370 0x5622a5da0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.659+0000 7f9afc71f740 1 --2- 172.21.15.152:0/3715844916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d935d0 0x5622a5d93a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.659+0000 7f9afc71f740 1 --2- 172.21.15.152:0/3715844916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d94b40 0x5622a5d93090 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.659+0000 7f9afc71f740 1 -- 172.21.15.152:0/3715844916 >> 172.21.15.152:0/3715844916 conn(0x5622a5d5c310 msgr2=0x5622a5d8fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:30.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.660+0000 7f9afc71f740 1 -- 172.21.15.152:0/3715844916 shutdown_connections 2024-09-11T22:55:30.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.660+0000 7f9afc71f740 1 -- 172.21.15.152:0/3715844916 wait complete. 2024-09-11T22:55:30.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.660+0000 7f9afc71f740 1 -- start start 2024-09-11T22:55:30.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.660+0000 7f9afc71f740 1 Processor -- start 2024-09-11T22:55:30.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.660+0000 7f9afc71f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d935d0 0x5622a5d9dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.660+0000 7f9afc71f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d94b40 0x5622a5d9af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.660+0000 7f9afc71f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d9e370 0x5622a5d9b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.660+0000 7f9afc71f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5622a5da1070 con 0x5622a5d935d0 2024-09-11T22:55:30.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.660+0000 7f9afc71f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5622a5cb4060 con 0x5622a5d9e370 2024-09-11T22:55:30.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.660+0000 7f9afc71f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5622a5da11f0 con 0x5622a5d94b40 2024-09-11T22:55:30.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.660+0000 7f9afc71f740 10 client.0 _ll_register_callbacks cb 0x5622a5a078c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:30.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.660+0000 7f9af3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d935d0 0x5622a5d9dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.660+0000 7f9af3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d935d0 0x5622a5d9dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55044/0 (socket says 172.21.15.152:55044) 2024-09-11T22:55:30.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.660+0000 7f9af3fff640 1 -- 172.21.15.152:0/3681088084 learned_addr learned my addr 172.21.15.152:0/3681088084 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:30.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9af37fe640 1 --2- 172.21.15.152:0/3681088084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d94b40 0x5622a5d9af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9af88e3640 1 --2- 172.21.15.152:0/3681088084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d9e370 0x5622a5d9b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9af3fff640 1 -- 172.21.15.152:0/3681088084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d94b40 msgr2=0x5622a5d9af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:30.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9af3fff640 1 --2- 172.21.15.152:0/3681088084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d94b40 0x5622a5d9af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9af3fff640 1 -- 172.21.15.152:0/3681088084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d9e370 msgr2=0x5622a5d9b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:30.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9af3fff640 1 --2- 172.21.15.152:0/3681088084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d9e370 0x5622a5d9b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9af3fff640 1 -- 172.21.15.152:0/3681088084 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622a5a6a3c0 con 0x5622a5d935d0 2024-09-11T22:55:30.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9af37fe640 1 --2- 172.21.15.152:0/3681088084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d94b40 0x5622a5d9af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:30.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9af3fff640 1 --2- 172.21.15.152:0/3681088084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d935d0 0x5622a5d9dce0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400bee0 tx=0x7f9ae4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ec5b8a492bb445 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:30.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9ad37fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:30.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9ad37fe640 1 -- 172.21.15.152:0/3681088084 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4004230 con 0x5622a5d935d0 2024-09-11T22:55:30.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9ad37fe640 1 -- 172.21.15.152:0/3681088084 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f9ae40043d0 con 0x5622a5d935d0 2024-09-11T22:55:30.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9afc71f740 1 -- 172.21.15.152:0/3681088084 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aec036040 con 0x5622a5d935d0 2024-09-11T22:55:30.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9ad37fe640 1 -- 172.21.15.152:0/3681088084 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ae40106a0 con 0x5622a5d935d0 2024-09-11T22:55:30.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9afc71f740 10 client.16245 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:30.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9afc71f740 1 -- 172.21.15.152:0/3681088084 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622a5dacae0 con 0x5622a5d935d0 2024-09-11T22:55:30.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.661+0000 7f9ad37fe640 1 -- 172.21.15.152:0/3681088084 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aec036040 con 0x5622a5d935d0 2024-09-11T22:55:30.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.662+0000 7f9ad37fe640 1 -- 172.21.15.152:0/3681088084 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9ae4003bd0 con 0x5622a5d935d0 2024-09-11T22:55:30.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.662+0000 7f9afc71f740 10 client.16245 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:30.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.662+0000 7f9afc71f740 10 client.16245 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:55:30.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.662+0000 7f9afc71f740 20 client.16245 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:30.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.662+0000 7f9afc71f740 1 --2- 172.21.15.152:0/3681088084 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5622a5dadef0 0x5622a5db0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.662+0000 7f9afc71f740 4 client.16245 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:55:30.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.662+0000 7f9afc71f740 1 -- 172.21.15.152:0/3681088084 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622a5d06860 con 0x5622a5dadef0 2024-09-11T22:55:30.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.662+0000 7f9af37fe640 1 --2- 172.21.15.152:0/3681088084 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5622a5dadef0 0x5622a5db0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.662+0000 7f9ad37fe640 10 client.16245 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:55:30.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.662+0000 7f9af37fe640 1 --2- 172.21.15.152:0/3681088084 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5622a5dadef0 0x5622a5db0300 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8ec13efa27512c9 server_cookie=f8425692357dcea3 in_seq=0 out_seq=0 2024-09-11T22:55:30.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.663+0000 7f9ad37fe640 1 -- 172.21.15.152:0/3681088084 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622a5d06860 con 0x5622a5dadef0 2024-09-11T22:55:30.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.663+0000 7f9ad37fe640 10 client.16245 handle_command_reply: tid=0 2024-09-11T22:55:30.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.663+0000 7f9afc71f740 1 client.16245 shutdown 2024-09-11T22:55:30.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.663+0000 7f9afc71f740 1 -- 172.21.15.152:0/3681088084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d935d0 msgr2=0x5622a5d9dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:30.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.663+0000 7f9afc71f740 1 --2- 172.21.15.152:0/3681088084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d935d0 0x5622a5d9dce0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400bee0 tx=0x7f9ae4012820 comp rx=0 tx=0).stop 2024-09-11T22:55:30.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.663+0000 7f9afc71f740 1 -- 172.21.15.152:0/3681088084 shutdown_connections 2024-09-11T22:55:30.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.663+0000 7f9afc71f740 1 --2- 172.21.15.152:0/3681088084 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5622a5dadef0 0x5622a5db0300 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.663+0000 7f9afc71f740 1 --2- 172.21.15.152:0/3681088084 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d9e370 0x5622a5d9b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.663+0000 7f9afc71f740 1 --2- 172.21.15.152:0/3681088084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d94b40 0x5622a5d9af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.663+0000 7f9afc71f740 1 --2- 172.21.15.152:0/3681088084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d935d0 0x5622a5d9dce0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.663+0000 7f9ad37fe640 0 client.16245 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:55:30.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.663+0000 7f9afc71f740 1 -- 172.21.15.152:0/3681088084 >> 172.21.15.152:0/3681088084 conn(0x5622a5d5c310 msgr2=0x5622a5d95960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:30.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.663+0000 7f9afc71f740 1 -- 172.21.15.152:0/3681088084 shutdown_connections 2024-09-11T22:55:30.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.663+0000 7f9afc71f740 1 -- 172.21.15.152:0/3681088084 wait complete. 2024-09-11T22:55:30.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.664+0000 7f9afc71f740 20 client.16245 trim_cache size 0 max 16384 2024-09-11T22:55:30.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.676+0000 7f9afc71f740 1 Processor -- start 2024-09-11T22:55:30.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.676+0000 7f9afc71f740 1 -- start start 2024-09-11T22:55:30.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.676+0000 7f9afc71f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d935d0 0x5622a5daa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.676+0000 7f9afc71f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d94b40 0x5622a5d92de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9afc71f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d9e370 0x5622a5db5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9afc71f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41ad870 con 0x5622a5d9e370 2024-09-11T22:55:30.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9afc71f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41af3b0 con 0x5622a5d94b40 2024-09-11T22:55:30.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9afc71f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41adf40 con 0x5622a5d935d0 2024-09-11T22:55:30.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9af88e3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d9e370 0x5622a5db5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9af88e3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d9e370 0x5622a5db5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55060/0 (socket says 172.21.15.152:55060) 2024-09-11T22:55:30.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9af88e3640 1 -- 172.21.15.152:0/3590278424 learned_addr learned my addr 172.21.15.152:0/3590278424 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:30.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9af3fff640 1 --2- 172.21.15.152:0/3590278424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d935d0 0x5622a5daa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9af37fe640 1 --2- 172.21.15.152:0/3590278424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d94b40 0x5622a5d92de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9af88e3640 1 -- 172.21.15.152:0/3590278424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d935d0 msgr2=0x5622a5daa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:30.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9af88e3640 1 --2- 172.21.15.152:0/3590278424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d935d0 0x5622a5daa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9af88e3640 1 -- 172.21.15.152:0/3590278424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d94b40 msgr2=0x5622a5d92de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:30.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9af88e3640 1 --2- 172.21.15.152:0/3590278424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d94b40 0x5622a5d92de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9af88e3640 1 -- 172.21.15.152:0/3590278424 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622a5d9acb0 con 0x5622a5d9e370 2024-09-11T22:55:30.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9af3fff640 1 --2- 172.21.15.152:0/3590278424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d935d0 0x5622a5daa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:30.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9af88e3640 1 --2- 172.21.15.152:0/3590278424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d9e370 0x5622a5db5bd0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9aec004ae0 tx=0x7f9aec031550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d77bdd9eade8ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:30.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9ad9ffb640 1 -- 172.21.15.152:0/3590278424 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec04f040 con 0x5622a5d9e370 2024-09-11T22:55:30.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9ad9ffb640 1 -- 172.21.15.152:0/3590278424 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aec058070 con 0x5622a5d9e370 2024-09-11T22:55:30.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.677+0000 7f9ad9ffb640 1 -- 172.21.15.152:0/3590278424 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec004040 con 0x5622a5d9e370 2024-09-11T22:55:30.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.678+0000 7f9afc71f740 1 -- 172.21.15.152:0/3590278424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d9e370 msgr2=0x5622a5db5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:30.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.678+0000 7f9afc71f740 1 --2- 172.21.15.152:0/3590278424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d9e370 0x5622a5db5bd0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9aec004ae0 tx=0x7f9aec031550 comp rx=0 tx=0).stop 2024-09-11T22:55:30.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.678+0000 7f9afc71f740 1 -- 172.21.15.152:0/3590278424 shutdown_connections 2024-09-11T22:55:30.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.678+0000 7f9afc71f740 1 --2- 172.21.15.152:0/3590278424 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d9e370 0x5622a5db5bd0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.678+0000 7f9afc71f740 1 --2- 172.21.15.152:0/3590278424 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d94b40 0x5622a5d92de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.678+0000 7f9afc71f740 1 --2- 172.21.15.152:0/3590278424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d935d0 0x5622a5daa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.678+0000 7f9afc71f740 1 -- 172.21.15.152:0/3590278424 >> 172.21.15.152:0/3590278424 conn(0x5622a5d5c310 msgr2=0x5622a5d99860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:30.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.678+0000 7f9afc71f740 1 -- 172.21.15.152:0/3590278424 shutdown_connections 2024-09-11T22:55:30.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.678+0000 7f9afc71f740 1 -- 172.21.15.152:0/3590278424 wait complete. 2024-09-11T22:55:30.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.678+0000 7f9afc71f740 1 -- start start 2024-09-11T22:55:30.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.678+0000 7f9afc71f740 1 Processor -- start 2024-09-11T22:55:30.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9afc71f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d935d0 0x5622a5da6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9afc71f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d94b40 0x5622a5d97a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9afc71f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d9e370 0x5622a5d97fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9afc71f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41aece0 con 0x5622a5d94b40 2024-09-11T22:55:30.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9afc71f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41ae610 con 0x5622a5d935d0 2024-09-11T22:55:30.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9afc71f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9af41ab830 con 0x5622a5d9e370 2024-09-11T22:55:30.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9afc71f740 10 client.0 _ll_register_callbacks cb 0x7f9ae400b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:30.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d94b40 0x5622a5d97a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d94b40 0x5622a5d97a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55062/0 (socket says 172.21.15.152:55062) 2024-09-11T22:55:30.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9af37fe640 1 -- 172.21.15.152:0/2248756542 learned_addr learned my addr 172.21.15.152:0/2248756542 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:30.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9af88e3640 1 --2- 172.21.15.152:0/2248756542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d9e370 0x5622a5d97fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9af3fff640 1 --2- 172.21.15.152:0/2248756542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d935d0 0x5622a5da6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9af37fe640 1 -- 172.21.15.152:0/2248756542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d9e370 msgr2=0x5622a5d97fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:30.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9af37fe640 1 --2- 172.21.15.152:0/2248756542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d9e370 0x5622a5d97fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9af37fe640 1 -- 172.21.15.152:0/2248756542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d935d0 msgr2=0x5622a5da6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:30.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9af37fe640 1 --2- 172.21.15.152:0/2248756542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d935d0 0x5622a5da6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9af37fe640 1 -- 172.21.15.152:0/2248756542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622a5d98590 con 0x5622a5d94b40 2024-09-11T22:55:30.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9af37fe640 1 --2- 172.21.15.152:0/2248756542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d94b40 0x5622a5d97a60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80977f0 tx=0x7f9ae8009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82c56ed6d5079da server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:30.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9ad37fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:30.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9ad37fe640 1 -- 172.21.15.152:0/2248756542 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae801e040 con 0x5622a5d94b40 2024-09-11T22:55:30.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9ad37fe640 1 -- 172.21.15.152:0/2248756542 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f9ae8097490 con 0x5622a5d94b40 2024-09-11T22:55:30.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.679+0000 7f9afc71f740 1 -- 172.21.15.152:0/2248756542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aec058070 con 0x5622a5d94b40 2024-09-11T22:55:30.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.680+0000 7f9ad37fe640 1 -- 172.21.15.152:0/2248756542 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ae80991e0 con 0x5622a5d94b40 2024-09-11T22:55:30.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.680+0000 7f9afc71f740 10 client.16251 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:30.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.680+0000 7f9afc71f740 1 -- 172.21.15.152:0/2248756542 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622a5e1ce30 con 0x5622a5d94b40 2024-09-11T22:55:30.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.680+0000 7f9ad37fe640 1 -- 172.21.15.152:0/2248756542 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aec058070 con 0x5622a5d94b40 2024-09-11T22:55:30.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.680+0000 7f9ad37fe640 1 -- 172.21.15.152:0/2248756542 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9ae80a3ce0 con 0x5622a5d94b40 2024-09-11T22:55:30.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.680+0000 7f9afc71f740 10 client.16251 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:30.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.680+0000 7f9afc71f740 10 client.16251 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:55:30.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.680+0000 7f9afc71f740 20 client.16251 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:30.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.680+0000 7f9afc71f740 1 --2- 172.21.15.152:0/2248756542 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5622a5e1d560 0x5622a5e1f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:30.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.680+0000 7f9afc71f740 4 client.16251 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:55:30.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.680+0000 7f9afc71f740 1 -- 172.21.15.152:0/2248756542 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622a5d00d70 con 0x5622a5e1d560 2024-09-11T22:55:30.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.680+0000 7f9af3fff640 1 --2- 172.21.15.152:0/2248756542 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5622a5e1d560 0x5622a5e1f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:30.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9af3fff640 1 --2- 172.21.15.152:0/2248756542 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5622a5e1d560 0x5622a5e1f980 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53810b432c13709 server_cookie=24a785e051eefba5 in_seq=0 out_seq=0 2024-09-11T22:55:30.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9ad37fe640 10 client.16251 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:55:30.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9ad37fe640 1 -- 172.21.15.152:0/2248756542 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622a5d00d70 con 0x5622a5e1d560 2024-09-11T22:55:30.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9ad37fe640 10 client.16251 handle_command_reply: tid=0 2024-09-11T22:55:30.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9afc71f740 1 client.16251 shutdown 2024-09-11T22:55:30.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9afc71f740 1 -- 172.21.15.152:0/2248756542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d94b40 msgr2=0x5622a5d97a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:30.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9afc71f740 1 --2- 172.21.15.152:0/2248756542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d94b40 0x5622a5d97a60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80977f0 tx=0x7f9ae8009390 comp rx=0 tx=0).stop 2024-09-11T22:55:30.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9afc71f740 1 -- 172.21.15.152:0/2248756542 shutdown_connections 2024-09-11T22:55:30.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9afc71f740 1 --2- 172.21.15.152:0/2248756542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622a5d9e370 0x5622a5d97fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9afc71f740 1 --2- 172.21.15.152:0/2248756542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622a5d94b40 0x5622a5d97a60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9afc71f740 1 --2- 172.21.15.152:0/2248756542 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5622a5e1d560 0x5622a5e1f980 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9afc71f740 1 --2- 172.21.15.152:0/2248756542 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622a5d935d0 0x5622a5da6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9ad37fe640 0 client.16251 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:55:30.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9afc71f740 1 -- 172.21.15.152:0/2248756542 >> 172.21.15.152:0/2248756542 conn(0x5622a5d5c310 msgr2=0x5622a5d8e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:30.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9afc71f740 1 -- 172.21.15.152:0/2248756542 shutdown_connections 2024-09-11T22:55:30.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9afc71f740 1 -- 172.21.15.152:0/2248756542 wait complete. 2024-09-11T22:55:30.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.681+0000 7f9afc71f740 20 client.16251 trim_cache size 0 max 16384 2024-09-11T22:55:30.697 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:55:30.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.682+0000 7f9afa346640 1 -- 172.21.15.152:0/1236679554 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9adc0a2cf0 msgr2=0x7f9adc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:30.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.682+0000 7f9afa346640 1 --2- 172.21.15.152:0/1236679554 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9adc0a2cf0 0x7f9adc0a51b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9aec04c000 tx=0x7f9aec048000 comp rx=0 tx=0).stop 2024-09-11T22:55:30.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.682+0000 7f9afa346640 1 -- 172.21.15.152:0/1236679554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9af4101420 msgr2=0x7f9af4199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:30.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.682+0000 7f9afa346640 1 --2- 172.21.15.152:0/1236679554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9af4101420 0x7f9af4199830 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800b4a0 tx=0x7f9ae800b960 comp rx=0 tx=0).stop 2024-09-11T22:55:30.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.682+0000 7f9afa346640 1 -- 172.21.15.152:0/1236679554 shutdown_connections 2024-09-11T22:55:30.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.682+0000 7f9afa346640 1 --2- 172.21.15.152:0/1236679554 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9adc0a2cf0 0x7f9adc0a51b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.682+0000 7f9afa346640 1 --2- 172.21.15.152:0/1236679554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9af4106a80 0x7f9af41a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.682+0000 7f9afa346640 1 --2- 172.21.15.152:0/1236679554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9af4101420 0x7f9af4199830 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.682+0000 7f9afa346640 1 --2- 172.21.15.152:0/1236679554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9af4100a80 0x7f9af41992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:30.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.682+0000 7f9afa346640 1 -- 172.21.15.152:0/1236679554 >> 172.21.15.152:0/1236679554 conn(0x7f9af40fc7a0 msgr2=0x7f9af40fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:30.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.682+0000 7f9afa346640 1 -- 172.21.15.152:0/1236679554 shutdown_connections 2024-09-11T22:55:30.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:30.682+0000 7f9afa346640 1 -- 172.21.15.152:0/1236679554 wait complete. 2024-09-11T22:55:31.517 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:55:31.792 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:55:32.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.305+0000 7f757cdec640 1 -- 172.21.15.152:0/3069816639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7578102ef0 msgr2=0x7f757810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:32.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.305+0000 7f757cdec640 1 --2- 172.21.15.152:0/3069816639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7578102ef0 0x7f757810da40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f7570009ca0 tx=0x7f757002a950 comp rx=0 tx=0).stop 2024-09-11T22:55:32.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.305+0000 7f757cdec640 1 -- 172.21.15.152:0/3069816639 shutdown_connections 2024-09-11T22:55:32.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.305+0000 7f757cdec640 1 --2- 172.21.15.152:0/3069816639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7578102ef0 0x7f757810da40 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.305+0000 7f757cdec640 1 --2- 172.21.15.152:0/3069816639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7578102570 0x7f75781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.305+0000 7f757cdec640 1 --2- 172.21.15.152:0/3069816639 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7578100e20 0x7f7578101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.305+0000 7f757cdec640 1 -- 172.21.15.152:0/3069816639 >> 172.21.15.152:0/3069816639 conn(0x7f75780fc7d0 msgr2=0x7f75780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:32.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.305+0000 7f757cdec640 1 -- 172.21.15.152:0/3069816639 shutdown_connections 2024-09-11T22:55:32.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.305+0000 7f757cdec640 1 -- 172.21.15.152:0/3069816639 wait complete. 2024-09-11T22:55:32.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.306+0000 7f757cdec640 1 Processor -- start 2024-09-11T22:55:32.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.306+0000 7f757cdec640 1 -- start start 2024-09-11T22:55:32.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.306+0000 7f757cdec640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7578100e20 0x7f75781983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.306+0000 7f757cdec640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7578102570 0x7f7578198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.306+0000 7f757cdec640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7578102ef0 0x7f757819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.306+0000 7f757cdec640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7578110f80 con 0x7f7578102570 2024-09-11T22:55:32.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.306+0000 7f757cdec640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7578110e00 con 0x7f7578102ef0 2024-09-11T22:55:32.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.306+0000 7f757cdec640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7578111100 con 0x7f7578100e20 2024-09-11T22:55:32.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.306+0000 7f756ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7578102570 0x7f7578198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.306+0000 7f756ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7578102570 0x7f7578198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55080/0 (socket says 172.21.15.152:55080) 2024-09-11T22:55:32.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.306+0000 7f756ffff640 1 -- 172.21.15.152:0/2271258328 learned_addr learned my addr 172.21.15.152:0/2271258328 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:32.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.306+0000 7f7576d9d640 1 --2- 172.21.15.152:0/2271258328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7578102ef0 0x7f757819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.307+0000 7f757659c640 1 --2- 172.21.15.152:0/2271258328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7578100e20 0x7f75781983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.307+0000 7f756ffff640 1 -- 172.21.15.152:0/2271258328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7578100e20 msgr2=0x7f75781983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:32.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.307+0000 7f756ffff640 1 --2- 172.21.15.152:0/2271258328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7578100e20 0x7f75781983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.307+0000 7f756ffff640 1 -- 172.21.15.152:0/2271258328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7578102ef0 msgr2=0x7f757819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:32.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.307+0000 7f756ffff640 1 --2- 172.21.15.152:0/2271258328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7578102ef0 0x7f757819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.307+0000 7f756ffff640 1 -- 172.21.15.152:0/2271258328 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757819d380 con 0x7f7578102570 2024-09-11T22:55:32.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.307+0000 7f757659c640 1 --2- 172.21.15.152:0/2271258328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7578100e20 0x7f75781983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:32.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.307+0000 7f756ffff640 1 --2- 172.21.15.152:0/2271258328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7578102570 0x7f7578198910 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f756000c920 tx=0x7f756000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:32.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.307+0000 7f756f7fe640 1 -- 172.21.15.152:0/2271258328 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560007c70 con 0x7f7578102570 2024-09-11T22:55:32.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.307+0000 7f757cdec640 1 -- 172.21.15.152:0/2271258328 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757819d670 con 0x7f7578102570 2024-09-11T22:55:32.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.307+0000 7f756f7fe640 1 -- 172.21.15.152:0/2271258328 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7560007e10 con 0x7f7578102570 2024-09-11T22:55:32.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.307+0000 7f756f7fe640 1 -- 172.21.15.152:0/2271258328 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560018890 con 0x7f7578102570 2024-09-11T22:55:32.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.307+0000 7f757cdec640 1 -- 172.21.15.152:0/2271258328 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75781a4f50 con 0x7f7578102570 2024-09-11T22:55:32.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.310+0000 7f757f1c5740 1 Processor -- start 2024-09-11T22:55:32.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.310+0000 7f757f1c5740 1 -- start start 2024-09-11T22:55:32.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f757f1c5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a9b40 0x5628583a8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f757f1c5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583a85d0 0x5628583a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f757f1c5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583b3370 0x5628583b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f757f1c5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56285830e2f0 con 0x5628583b3370 2024-09-11T22:55:32.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f757f1c5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5628582b5cf0 con 0x5628583a9b40 2024-09-11T22:55:32.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f757f1c5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56285830a2b0 con 0x5628583a85d0 2024-09-11T22:55:32.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f757659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583a85d0 0x5628583a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f757659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583a85d0 0x5628583a8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:58106/0 (socket says 172.21.15.152:58106) 2024-09-11T22:55:32.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f757659c640 1 -- 172.21.15.152:0/2614056661 learned_addr learned my addr 172.21.15.152:0/2614056661 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:32.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f7576d9d640 1 --2- 172.21.15.152:0/2614056661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a9b40 0x5628583a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f756f7fe640 1 -- 172.21.15.152:0/2271258328 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7560018a50 con 0x7f7578102570 2024-09-11T22:55:32.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f7576d9d640 1 -- 172.21.15.152:0/2614056661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583a85d0 msgr2=0x5628583a8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:32.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f7576d9d640 1 --2- 172.21.15.152:0/2614056661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583a85d0 0x5628583a8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f7576d9d640 1 -- 172.21.15.152:0/2614056661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583b3370 msgr2=0x5628583b5760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:55:32.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f7576d9d640 1 --2- 172.21.15.152:0/2614056661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583b3370 0x5628583b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f7576d9d640 1 -- 172.21.15.152:0/2614056661 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56285829a3a0 con 0x5628583a9b40 2024-09-11T22:55:32.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.311+0000 7f757659c640 1 --2- 172.21.15.152:0/2614056661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583a85d0 0x5628583a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:55:32.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.312+0000 7f7576d9d640 1 --2- 172.21.15.152:0/2614056661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a9b40 0x5628583a8090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7570005cc0 tx=0x7f757002ff40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1e717fc8e4c354 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:32.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.312+0000 7f756cff9640 1 -- 172.21.15.152:0/2614056661 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570048070 con 0x5628583a9b40 2024-09-11T22:55:32.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.312+0000 7f756f7fe640 1 --2- 172.21.15.152:0/2271258328 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f75640a2cf0 0x7f75640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.312+0000 7f756cff9640 1 -- 172.21.15.152:0/2614056661 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f757002faf0 con 0x5628583a9b40 2024-09-11T22:55:32.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.312+0000 7f756cff9640 1 -- 172.21.15.152:0/2614056661 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570034040 con 0x5628583a9b40 2024-09-11T22:55:32.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.312+0000 7f757659c640 1 --2- 172.21.15.152:0/2271258328 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f75640a2cf0 0x7f75640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.312+0000 7f756f7fe640 1 -- 172.21.15.152:0/2271258328 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f75600cf570 con 0x7f7578102570 2024-09-11T22:55:32.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.313+0000 7f757f1c5740 1 -- 172.21.15.152:0/2614056661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a9b40 msgr2=0x5628583a8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:32.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.313+0000 7f757659c640 1 --2- 172.21.15.152:0/2271258328 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f75640a2cf0 0x7f75640a51b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x5628583a5ac0 tx=0x7f756800c590 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:32.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.313+0000 7f757f1c5740 1 --2- 172.21.15.152:0/2614056661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a9b40 0x5628583a8090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7570005cc0 tx=0x7f757002ff40 comp rx=0 tx=0).stop 2024-09-11T22:55:32.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.313+0000 7f757f1c5740 1 -- 172.21.15.152:0/2614056661 shutdown_connections 2024-09-11T22:55:32.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.313+0000 7f757f1c5740 1 --2- 172.21.15.152:0/2614056661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583b3370 0x5628583b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.313+0000 7f757f1c5740 1 --2- 172.21.15.152:0/2614056661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583a85d0 0x5628583a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.313+0000 7f757f1c5740 1 --2- 172.21.15.152:0/2614056661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a9b40 0x5628583a8090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.313+0000 7f757f1c5740 1 -- 172.21.15.152:0/2614056661 >> 172.21.15.152:0/2614056661 conn(0x562858371310 msgr2=0x5628583a4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:32.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.313+0000 7f757f1c5740 1 -- 172.21.15.152:0/2614056661 shutdown_connections 2024-09-11T22:55:32.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.313+0000 7f757f1c5740 1 -- 172.21.15.152:0/2614056661 wait complete. 2024-09-11T22:55:32.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.313+0000 7f757f1c5740 1 -- start start 2024-09-11T22:55:32.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f757f1c5740 1 Processor -- start 2024-09-11T22:55:32.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f757f1c5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a85d0 0x5628583b0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f757f1c5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a9b40 0x5628583b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f757f1c5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583b3370 0x5628583b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f757f1c5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5628583b6010 con 0x5628583a9b40 2024-09-11T22:55:32.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f757f1c5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5628582c9060 con 0x5628583a85d0 2024-09-11T22:55:32.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f757f1c5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5628583b6190 con 0x5628583b3370 2024-09-11T22:55:32.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f757f1c5740 10 client.0 _ll_register_callbacks cb 0x56285801c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:32.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f756ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a9b40 0x5628583b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f756ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a9b40 0x5628583b0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55102/0 (socket says 172.21.15.152:55102) 2024-09-11T22:55:32.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f756ffff640 1 -- 172.21.15.152:0/3229200986 learned_addr learned my addr 172.21.15.152:0/3229200986 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:32.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f7576d9d640 1 --2- 172.21.15.152:0/3229200986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583b3370 0x5628583b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f757659c640 1 --2- 172.21.15.152:0/3229200986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a85d0 0x5628583b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f756ffff640 1 -- 172.21.15.152:0/3229200986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583b3370 msgr2=0x5628583b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:32.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f756ffff640 1 --2- 172.21.15.152:0/3229200986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583b3370 0x5628583b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f756ffff640 1 -- 172.21.15.152:0/3229200986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a85d0 msgr2=0x5628583b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:32.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f756ffff640 1 --2- 172.21.15.152:0/3229200986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a85d0 0x5628583b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f756ffff640 1 -- 172.21.15.152:0/3229200986 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56285807f3c0 con 0x5628583a9b40 2024-09-11T22:55:32.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f7576d9d640 1 --2- 172.21.15.152:0/3229200986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583b3370 0x5628583b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:32.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.314+0000 7f757659c640 1 --2- 172.21.15.152:0/3229200986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a85d0 0x5628583b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:32.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.315+0000 7f756ffff640 1 --2- 172.21.15.152:0/3229200986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a9b40 0x5628583b0650 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f75600ccd70 tx=0x7f756009c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54be9471850e5ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:32.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.315+0000 7f755dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:32.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.315+0000 7f755dffb640 1 -- 172.21.15.152:0/3229200986 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75600a4750 con 0x5628583a9b40 2024-09-11T22:55:32.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.315+0000 7f755dffb640 1 -- 172.21.15.152:0/3229200986 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f75600ccf90 con 0x5628583a9b40 2024-09-11T22:55:32.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.315+0000 7f757f1c5740 1 -- 172.21.15.152:0/3229200986 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f757002faf0 con 0x5628583a9b40 2024-09-11T22:55:32.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.315+0000 7f755dffb640 1 -- 172.21.15.152:0/3229200986 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f75600a42c0 con 0x5628583a9b40 2024-09-11T22:55:32.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.315+0000 7f757f1c5740 10 client.16260 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:32.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.315+0000 7f757f1c5740 1 -- 172.21.15.152:0/3229200986 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5628583c1c70 con 0x5628583a9b40 2024-09-11T22:55:32.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.316+0000 7f755dffb640 1 -- 172.21.15.152:0/3229200986 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75600a4460 con 0x5628583a9b40 2024-09-11T22:55:32.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.316+0000 7f755dffb640 1 -- 172.21.15.152:0/3229200986 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f75600b4ce0 con 0x5628583a9b40 2024-09-11T22:55:32.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.316+0000 7f757f1c5740 10 client.16260 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:32.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.316+0000 7f757f1c5740 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:32.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.316+0000 7f757f1c5740 20 client.16260 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:32.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.316+0000 7f757f1c5740 1 --2- 172.21.15.152:0/3229200986 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5628583c3080 0x5628583c5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.316+0000 7f757f1c5740 4 client.16260 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:55:32.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.316+0000 7f757f1c5740 1 -- 172.21.15.152:0/3229200986 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f757002faf0 con 0x5628583c3080 2024-09-11T22:55:32.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.317+0000 7f757659c640 1 --2- 172.21.15.152:0/3229200986 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5628583c3080 0x5628583c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.317+0000 7f755dffb640 10 client.16260 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:32.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.317+0000 7f757659c640 1 --2- 172.21.15.152:0/3229200986 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5628583c3080 0x5628583c5470 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93a351113fb3c68b server_cookie=e6b195180ea41fb5 in_seq=0 out_seq=0 2024-09-11T22:55:32.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.318+0000 7f755dffb640 1 -- 172.21.15.152:0/3229200986 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f757002faf0 con 0x5628583c3080 2024-09-11T22:55:32.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.318+0000 7f755dffb640 10 client.16260 handle_command_reply: tid=0 2024-09-11T22:55:32.329 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.318+0000 7f757f1c5740 1 client.16260 shutdown 2024-09-11T22:55:32.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.318+0000 7f757f1c5740 1 -- 172.21.15.152:0/3229200986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a9b40 msgr2=0x5628583b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:32.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.318+0000 7f757f1c5740 1 --2- 172.21.15.152:0/3229200986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a9b40 0x5628583b0650 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f75600ccd70 tx=0x7f756009c2a0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.318+0000 7f757f1c5740 1 -- 172.21.15.152:0/3229200986 shutdown_connections 2024-09-11T22:55:32.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.318+0000 7f757f1c5740 1 --2- 172.21.15.152:0/3229200986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583b3370 0x5628583b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.318+0000 7f757f1c5740 1 --2- 172.21.15.152:0/3229200986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a9b40 0x5628583b0650 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.318+0000 7f757f1c5740 1 --2- 172.21.15.152:0/3229200986 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5628583c3080 0x5628583c5470 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.318+0000 7f757f1c5740 1 --2- 172.21.15.152:0/3229200986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a85d0 0x5628583b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.318+0000 7f755dffb640 0 client.16260 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:32.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.318+0000 7f757f1c5740 1 -- 172.21.15.152:0/3229200986 >> 172.21.15.152:0/3229200986 conn(0x562858371310 msgr2=0x5628583aa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:32.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.318+0000 7f757f1c5740 1 -- 172.21.15.152:0/3229200986 shutdown_connections 2024-09-11T22:55:32.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.318+0000 7f757f1c5740 1 -- 172.21.15.152:0/3229200986 wait complete. 2024-09-11T22:55:32.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.319+0000 7f757f1c5740 20 client.16260 trim_cache size 0 max 16384 2024-09-11T22:55:32.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.331+0000 7f757f1c5740 1 Processor -- start 2024-09-11T22:55:32.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f757f1c5740 1 -- start start 2024-09-11T22:55:32.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f757f1c5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a85d0 0x5628583beac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f757f1c5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583a9b40 0x5628583ab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f757f1c5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583b3370 0x5628583ab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f757f1c5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f75781ad5d0 con 0x5628583a85d0 2024-09-11T22:55:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f757f1c5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f75781af110 con 0x5628583b3370 2024-09-11T22:55:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f757f1c5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f75781adca0 con 0x5628583a9b40 2024-09-11T22:55:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f757659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a85d0 0x5628583beac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f757659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a85d0 0x5628583beac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55114/0 (socket says 172.21.15.152:55114) 2024-09-11T22:55:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f756ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583a9b40 0x5628583ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f757659c640 1 -- 172.21.15.152:0/4038502939 learned_addr learned my addr 172.21.15.152:0/4038502939 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f7576d9d640 1 --2- 172.21.15.152:0/4038502939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583b3370 0x5628583ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f757659c640 1 -- 172.21.15.152:0/4038502939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583a9b40 msgr2=0x5628583ab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f757659c640 1 --2- 172.21.15.152:0/4038502939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583a9b40 0x5628583ab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f757659c640 1 -- 172.21.15.152:0/4038502939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583b3370 msgr2=0x5628583ab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f757659c640 1 --2- 172.21.15.152:0/4038502939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583b3370 0x5628583ab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.332+0000 7f757659c640 1 -- 172.21.15.152:0/4038502939 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628583c57e0 con 0x5628583a85d0 2024-09-11T22:55:32.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.333+0000 7f757659c640 1 --2- 172.21.15.152:0/4038502939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a85d0 0x5628583beac0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f75680058c0 tx=0x7f7568014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd74dc1f906ed0c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:32.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.333+0000 7f755ffff640 1 -- 172.21.15.152:0/4038502939 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756800fe30 con 0x5628583a85d0 2024-09-11T22:55:32.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.333+0000 7f755ffff640 1 -- 172.21.15.152:0/4038502939 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75680035f0 con 0x5628583a85d0 2024-09-11T22:55:32.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.333+0000 7f755ffff640 1 -- 172.21.15.152:0/4038502939 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568027530 con 0x5628583a85d0 2024-09-11T22:55:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.333+0000 7f757f1c5740 1 -- 172.21.15.152:0/4038502939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a85d0 msgr2=0x5628583beac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.333+0000 7f757f1c5740 1 --2- 172.21.15.152:0/4038502939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a85d0 0x5628583beac0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f75680058c0 tx=0x7f7568014000 comp rx=0 tx=0).stop 2024-09-11T22:55:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.334+0000 7f757f1c5740 1 -- 172.21.15.152:0/4038502939 shutdown_connections 2024-09-11T22:55:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.334+0000 7f757f1c5740 1 --2- 172.21.15.152:0/4038502939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583b3370 0x5628583ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.334+0000 7f757f1c5740 1 --2- 172.21.15.152:0/4038502939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583a9b40 0x5628583ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.334+0000 7f757f1c5740 1 --2- 172.21.15.152:0/4038502939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a85d0 0x5628583beac0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.334+0000 7f757f1c5740 1 -- 172.21.15.152:0/4038502939 >> 172.21.15.152:0/4038502939 conn(0x562858371310 msgr2=0x5628583a3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.334+0000 7f757f1c5740 1 -- 172.21.15.152:0/4038502939 shutdown_connections 2024-09-11T22:55:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.334+0000 7f757f1c5740 1 -- 172.21.15.152:0/4038502939 wait complete. 2024-09-11T22:55:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.334+0000 7f757f1c5740 1 -- start start 2024-09-11T22:55:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.334+0000 7f757f1c5740 1 Processor -- start 2024-09-11T22:55:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.334+0000 7f757f1c5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a85d0 0x5628584271c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.334+0000 7f757f1c5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a9b40 0x562858427700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.334+0000 7f757f1c5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583b3370 0x5628583bb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.334+0000 7f757f1c5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f75781aea40 con 0x5628583a9b40 2024-09-11T22:55:32.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.334+0000 7f757f1c5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f75781ae370 con 0x5628583a85d0 2024-09-11T22:55:32.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.334+0000 7f757f1c5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f75781ab590 con 0x5628583b3370 2024-09-11T22:55:32.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.334+0000 7f757f1c5740 10 client.0 _ll_register_callbacks cb 0x7f756000cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:32.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.335+0000 7f756ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a9b40 0x562858427700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.335+0000 7f756ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a9b40 0x562858427700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55126/0 (socket says 172.21.15.152:55126) 2024-09-11T22:55:32.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.335+0000 7f756ffff640 1 -- 172.21.15.152:0/685043986 learned_addr learned my addr 172.21.15.152:0/685043986 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:32.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.335+0000 7f757659c640 1 --2- 172.21.15.152:0/685043986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a85d0 0x5628584271c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.335+0000 7f7576d9d640 1 --2- 172.21.15.152:0/685043986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583b3370 0x5628583bb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.335+0000 7f756ffff640 1 -- 172.21.15.152:0/685043986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583b3370 msgr2=0x5628583bb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:32.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.335+0000 7f756ffff640 1 --2- 172.21.15.152:0/685043986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583b3370 0x5628583bb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.335+0000 7f756ffff640 1 -- 172.21.15.152:0/685043986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a85d0 msgr2=0x5628584271c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:32.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.335+0000 7f756ffff640 1 --2- 172.21.15.152:0/685043986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a85d0 0x5628584271c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.335+0000 7f756ffff640 1 -- 172.21.15.152:0/685043986 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628583bb960 con 0x5628583a9b40 2024-09-11T22:55:32.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.335+0000 7f757659c640 1 --2- 172.21.15.152:0/685043986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a85d0 0x5628584271c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:55:32.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.335+0000 7f756ffff640 1 --2- 172.21.15.152:0/685043986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a9b40 0x562858427700 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7560007590 tx=0x7f75600cf450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b559b8bb55a0559 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:32.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.335+0000 7f755dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:32.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.335+0000 7f755dffb640 1 -- 172.21.15.152:0/685043986 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560064030 con 0x5628583a9b40 2024-09-11T22:55:32.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.336+0000 7f755dffb640 1 -- 172.21.15.152:0/685043986 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f756009b140 con 0x5628583a9b40 2024-09-11T22:55:32.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.336+0000 7f757f1c5740 1 -- 172.21.15.152:0/685043986 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75680035f0 con 0x5628583a9b40 2024-09-11T22:55:32.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.336+0000 7f755dffb640 1 -- 172.21.15.152:0/685043986 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f75600a4750 con 0x5628583a9b40 2024-09-11T22:55:32.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.336+0000 7f757f1c5740 10 client.16266 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:32.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.336+0000 7f757f1c5740 1 -- 172.21.15.152:0/685043986 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562858432980 con 0x5628583a9b40 2024-09-11T22:55:32.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.336+0000 7f755dffb640 1 -- 172.21.15.152:0/685043986 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75680035f0 con 0x5628583a9b40 2024-09-11T22:55:32.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.336+0000 7f755dffb640 1 -- 172.21.15.152:0/685043986 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f75600b4ce0 con 0x5628583a9b40 2024-09-11T22:55:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.336+0000 7f757f1c5740 10 client.16266 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.336+0000 7f757f1c5740 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.336+0000 7f757f1c5740 20 client.16266 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.336+0000 7f757f1c5740 1 --2- 172.21.15.152:0/685043986 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562858432ef0 0x562858435310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.336+0000 7f757f1c5740 4 client.16266 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:55:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.336+0000 7f757f1c5740 1 -- 172.21.15.152:0/685043986 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56285831b860 con 0x562858432ef0 2024-09-11T22:55:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.337+0000 7f757659c640 1 --2- 172.21.15.152:0/685043986 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562858432ef0 0x562858435310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.337+0000 7f755dffb640 10 client.16266 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.337+0000 7f757659c640 1 --2- 172.21.15.152:0/685043986 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562858432ef0 0x562858435310 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43bc2f9460bfc8db server_cookie=331df140991c75e0 in_seq=0 out_seq=0 2024-09-11T22:55:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.337+0000 7f755dffb640 1 -- 172.21.15.152:0/685043986 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56285831b860 con 0x562858432ef0 2024-09-11T22:55:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.337+0000 7f755dffb640 10 client.16266 handle_command_reply: tid=0 2024-09-11T22:55:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.337+0000 7f757f1c5740 1 client.16266 shutdown 2024-09-11T22:55:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.337+0000 7f757f1c5740 1 -- 172.21.15.152:0/685043986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a9b40 msgr2=0x562858427700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:32.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.337+0000 7f757f1c5740 1 --2- 172.21.15.152:0/685043986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a9b40 0x562858427700 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7560007590 tx=0x7f75600cf450 comp rx=0 tx=0).stop 2024-09-11T22:55:32.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.337+0000 7f757f1c5740 1 -- 172.21.15.152:0/685043986 shutdown_connections 2024-09-11T22:55:32.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.337+0000 7f757f1c5740 1 --2- 172.21.15.152:0/685043986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628583b3370 0x5628583bb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.337+0000 7f757f1c5740 1 --2- 172.21.15.152:0/685043986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5628583a9b40 0x562858427700 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.337+0000 7f757f1c5740 1 --2- 172.21.15.152:0/685043986 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562858432ef0 0x562858435310 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.337+0000 7f757f1c5740 1 --2- 172.21.15.152:0/685043986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5628583a85d0 0x5628584271c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.337+0000 7f755dffb640 0 client.16266 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:32.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.337+0000 7f757f1c5740 1 -- 172.21.15.152:0/685043986 >> 172.21.15.152:0/685043986 conn(0x562858371310 msgr2=0x5628583beac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:32.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.338+0000 7f757f1c5740 1 -- 172.21.15.152:0/685043986 shutdown_connections 2024-09-11T22:55:32.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.338+0000 7f757f1c5740 1 -- 172.21.15.152:0/685043986 wait complete. 2024-09-11T22:55:32.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.338+0000 7f757f1c5740 20 client.16266 trim_cache size 0 max 16384 2024-09-11T22:55:32.343 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:55:32.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.339+0000 7f757cdec640 1 -- 172.21.15.152:0/2271258328 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f75640a2cf0 msgr2=0x7f75640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:32.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.339+0000 7f757cdec640 1 --2- 172.21.15.152:0/2271258328 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f75640a2cf0 0x7f75640a51b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x5628583a5ac0 tx=0x7f756800c590 comp rx=0 tx=0).stop 2024-09-11T22:55:32.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.339+0000 7f757cdec640 1 -- 172.21.15.152:0/2271258328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7578102570 msgr2=0x7f7578198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:32.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.339+0000 7f757cdec640 1 --2- 172.21.15.152:0/2271258328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7578102570 0x7f7578198910 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f756000c920 tx=0x7f756000cde0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.339+0000 7f757cdec640 1 -- 172.21.15.152:0/2271258328 shutdown_connections 2024-09-11T22:55:32.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.339+0000 7f757cdec640 1 --2- 172.21.15.152:0/2271258328 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f75640a2cf0 0x7f75640a51b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.339+0000 7f757cdec640 1 --2- 172.21.15.152:0/2271258328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7578102ef0 0x7f757819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.339+0000 7f757cdec640 1 --2- 172.21.15.152:0/2271258328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7578102570 0x7f7578198910 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.339+0000 7f757cdec640 1 --2- 172.21.15.152:0/2271258328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7578100e20 0x7f75781983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:32.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.339+0000 7f757cdec640 1 -- 172.21.15.152:0/2271258328 >> 172.21.15.152:0/2271258328 conn(0x7f75780fc7d0 msgr2=0x7f757806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:32.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.339+0000 7f757cdec640 1 -- 172.21.15.152:0/2271258328 shutdown_connections 2024-09-11T22:55:32.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:32.339+0000 7f757cdec640 1 -- 172.21.15.152:0/2271258328 wait complete. 2024-09-11T22:55:32.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:32 smithi191 ceph-mon[35211]: pgmap v82: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 414 MiB/s wr, 116 op/s 2024-09-11T22:55:32.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:32 smithi204 ceph-mon[35454]: pgmap v82: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 414 MiB/s wr, 116 op/s 2024-09-11T22:55:32.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:32 smithi152 ceph-mon[31181]: pgmap v82: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 414 MiB/s wr, 116 op/s 2024-09-11T22:55:32.705 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:55:32.705 INFO:tasks.workunit.client.0.smithi152.stdout: Children see throughput for 1 re-readers = 7578185.00 kB/sec 2024-09-11T22:55:32.705 INFO:tasks.workunit.client.0.smithi152.stdout: Parent sees throughput for 1 re-readers = 7576942.11 kB/sec 2024-09-11T22:55:32.705 INFO:tasks.workunit.client.0.smithi152.stdout: Min throughput per process = 7578185.00 kB/sec 2024-09-11T22:55:32.705 INFO:tasks.workunit.client.0.smithi152.stdout: Max throughput per process = 7578185.00 kB/sec 2024-09-11T22:55:32.706 INFO:tasks.workunit.client.0.smithi152.stdout: Avg throughput per process = 7578185.00 kB/sec 2024-09-11T22:55:32.706 INFO:tasks.workunit.client.0.smithi152.stdout: Min xfer = 10485760.00 kB 2024-09-11T22:55:34.410 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:55:34.410 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:55:34.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:34 smithi191 ceph-mon[35211]: pgmap v83: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 213 MiB/s wr, 64 op/s 2024-09-11T22:55:34.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:34 smithi204 ceph-mon[35454]: pgmap v83: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 213 MiB/s wr, 64 op/s 2024-09-11T22:55:34.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:34 smithi152 ceph-mon[31181]: pgmap v83: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 213 MiB/s wr, 64 op/s 2024-09-11T22:55:34.671 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:55:35.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.184+0000 7f6bd5d59640 1 -- 172.21.15.152:0/2026821768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd0106a80 msgr2=0x7f6bd0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:35.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.184+0000 7f6bd5d59640 1 --2- 172.21.15.152:0/2026821768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd0106a80 0x7f6bd0106e60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009930 tx=0x7f6bc002a420 comp rx=0 tx=0).stop 2024-09-11T22:55:35.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.184+0000 7f6bd5d59640 1 -- 172.21.15.152:0/2026821768 shutdown_connections 2024-09-11T22:55:35.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.184+0000 7f6bd5d59640 1 --2- 172.21.15.152:0/2026821768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bd0101420 0x7f6bd010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.184+0000 7f6bd5d59640 1 --2- 172.21.15.152:0/2026821768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bd0100a80 0x7f6bd0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.184+0000 7f6bd5d59640 1 --2- 172.21.15.152:0/2026821768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd0106a80 0x7f6bd0106e60 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.184+0000 7f6bd5d59640 1 -- 172.21.15.152:0/2026821768 >> 172.21.15.152:0/2026821768 conn(0x7f6bd00fc7a0 msgr2=0x7f6bd00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:35.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.184+0000 7f6bd5d59640 1 -- 172.21.15.152:0/2026821768 shutdown_connections 2024-09-11T22:55:35.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.184+0000 7f6bd5d59640 1 -- 172.21.15.152:0/2026821768 wait complete. 2024-09-11T22:55:35.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.184+0000 7f6bd5d59640 1 Processor -- start 2024-09-11T22:55:35.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bd5d59640 1 -- start start 2024-09-11T22:55:35.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bd5d59640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd0100a80 0x7f6bd01984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bd5d59640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bd0101420 0x7f6bd01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bd5d59640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bd0106a80 0x7f6bd019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bd5d59640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd0110e40 con 0x7f6bd0100a80 2024-09-11T22:55:35.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bd5d59640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd0110cc0 con 0x7f6bd0101420 2024-09-11T22:55:35.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bd5d59640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd0110fc0 con 0x7f6bd0106a80 2024-09-11T22:55:35.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bcf7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd0100a80 0x7f6bd01984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bcf7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd0100a80 0x7f6bd01984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55140/0 (socket says 172.21.15.152:55140) 2024-09-11T22:55:35.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bcf7fe640 1 -- 172.21.15.152:0/1090739066 learned_addr learned my addr 172.21.15.152:0/1090739066 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:35.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bcffff640 1 --2- 172.21.15.152:0/1090739066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bd0106a80 0x7f6bd019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bceffd640 1 --2- 172.21.15.152:0/1090739066 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bd0101420 0x7f6bd01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bcf7fe640 1 -- 172.21.15.152:0/1090739066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bd0106a80 msgr2=0x7f6bd019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:35.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bcf7fe640 1 --2- 172.21.15.152:0/1090739066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bd0106a80 0x7f6bd019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bcf7fe640 1 -- 172.21.15.152:0/1090739066 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bd0101420 msgr2=0x7f6bd01989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:35.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bcf7fe640 1 --2- 172.21.15.152:0/1090739066 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bd0101420 0x7f6bd01989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bcf7fe640 1 -- 172.21.15.152:0/1090739066 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd019d450 con 0x7f6bd0100a80 2024-09-11T22:55:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.185+0000 7f6bcf7fe640 1 --2- 172.21.15.152:0/1090739066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd0100a80 0x7f6bd01984a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009230 tx=0x7f6bc0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.186+0000 7f6bccff9640 1 -- 172.21.15.152:0/1090739066 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc003f030 con 0x7f6bd0100a80 2024-09-11T22:55:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.186+0000 7f6bccff9640 1 -- 172.21.15.152:0/1090739066 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bc0003ad0 con 0x7f6bd0100a80 2024-09-11T22:55:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.186+0000 7f6bd5d59640 1 -- 172.21.15.152:0/1090739066 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd019d6e0 con 0x7f6bd0100a80 2024-09-11T22:55:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.186+0000 7f6bccff9640 1 -- 172.21.15.152:0/1090739066 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0035070 con 0x7f6bd0100a80 2024-09-11T22:55:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.186+0000 7f6bd5d59640 1 -- 172.21.15.152:0/1090739066 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd01a5020 con 0x7f6bd0100a80 2024-09-11T22:55:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.188+0000 7f6bccff9640 1 -- 172.21.15.152:0/1090739066 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6bc0042030 con 0x7f6bd0100a80 2024-09-11T22:55:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.188+0000 7f6bd8132740 1 Processor -- start 2024-09-11T22:55:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.188+0000 7f6bd8132740 1 -- start start 2024-09-11T22:55:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bd8132740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404778b60 0x563404778f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bd8132740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404779480 0x563404783800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bd8132740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404783d40 0x563404786130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bd8132740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5634046de280 con 0x563404783d40 2024-09-11T22:55:35.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bd8132740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563404685b10 con 0x563404779480 2024-09-11T22:55:35.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bd8132740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5634046da240 con 0x563404778b60 2024-09-11T22:55:35.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bceffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404783d40 0x563404786130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bccff9640 1 --2- 172.21.15.152:0/1090739066 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6bb80a2d70 0x7f6bb80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bceffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404783d40 0x563404786130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55146/0 (socket says 172.21.15.152:55146) 2024-09-11T22:55:35.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bceffd640 1 -- 172.21.15.152:0/2981733269 learned_addr learned my addr 172.21.15.152:0/2981733269 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:35.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bcf7fe640 1 --2- 172.21.15.152:0/2981733269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404779480 0x563404783800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bcffff640 1 --2- 172.21.15.152:0/2981733269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404778b60 0x563404778f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bcffff640 1 --2- 172.21.15.152:0/1090739066 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6bb80a2d70 0x7f6bb80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bceffd640 1 -- 172.21.15.152:0/2981733269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404778b60 msgr2=0x563404778f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:35.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bceffd640 1 --2- 172.21.15.152:0/2981733269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404778b60 0x563404778f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bceffd640 1 -- 172.21.15.152:0/2981733269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404779480 msgr2=0x563404783800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:35.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bceffd640 1 --2- 172.21.15.152:0/2981733269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404779480 0x563404783800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bceffd640 1 -- 172.21.15.152:0/2981733269 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634046921f0 con 0x563404783d40 2024-09-11T22:55:35.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bccff9640 1 -- 172.21.15.152:0/1090739066 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f6bc00ef8d0 con 0x7f6bd0100a80 2024-09-11T22:55:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bcffff640 1 --2- 172.21.15.152:0/2981733269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404778b60 0x563404778f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.189+0000 7f6bceffd640 1 --2- 172.21.15.152:0/2981733269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404783d40 0x563404786130 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400dc80 tx=0x7f6bc400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2a6b771941f80f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.190+0000 7f6bb5ffb640 1 -- 172.21.15.152:0/2981733269 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4015070 con 0x563404783d40 2024-09-11T22:55:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.190+0000 7f6bcffff640 1 --2- 172.21.15.152:0/1090739066 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6bb80a2d70 0x7f6bb80a5230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800afd0 tx=0x7f6bc800a4d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.190+0000 7f6bb5ffb640 1 -- 172.21.15.152:0/2981733269 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bc401a940 con 0x563404783d40 2024-09-11T22:55:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.190+0000 7f6bb5ffb640 1 -- 172.21.15.152:0/2981733269 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4011a40 con 0x563404783d40 2024-09-11T22:55:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.190+0000 7f6bd8132740 1 -- 172.21.15.152:0/2981733269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404783d40 msgr2=0x563404786130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.190+0000 7f6bd8132740 1 --2- 172.21.15.152:0/2981733269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404783d40 0x563404786130 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400dc80 tx=0x7f6bc400f730 comp rx=0 tx=0).stop 2024-09-11T22:55:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.190+0000 7f6bd8132740 1 -- 172.21.15.152:0/2981733269 shutdown_connections 2024-09-11T22:55:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.190+0000 7f6bd8132740 1 --2- 172.21.15.152:0/2981733269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404783d40 0x563404786130 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.190+0000 7f6bd8132740 1 --2- 172.21.15.152:0/2981733269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404779480 0x563404783800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.190+0000 7f6bd8132740 1 --2- 172.21.15.152:0/2981733269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404778b60 0x563404778f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.190+0000 7f6bd8132740 1 -- 172.21.15.152:0/2981733269 >> 172.21.15.152:0/2981733269 conn(0x5634047750f0 msgr2=0x5634047754d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:35.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.190+0000 7f6bd8132740 1 -- 172.21.15.152:0/2981733269 shutdown_connections 2024-09-11T22:55:35.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.190+0000 7f6bd8132740 1 -- 172.21.15.152:0/2981733269 wait complete. 2024-09-11T22:55:35.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.191+0000 7f6bd8132740 1 -- start start 2024-09-11T22:55:35.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.191+0000 7f6bd8132740 1 Processor -- start 2024-09-11T22:55:35.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.191+0000 7f6bd8132740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404778b60 0x563404780c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.191+0000 7f6bd8132740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404779480 0x563404781150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.191+0000 7f6bd8132740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404783d40 0x563404781690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.191+0000 7f6bd8132740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x563404786910 con 0x563404778b60 2024-09-11T22:55:35.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.191+0000 7f6bd8132740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563404698f10 con 0x563404779480 2024-09-11T22:55:35.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.191+0000 7f6bd8132740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563404786a90 con 0x563404783d40 2024-09-11T22:55:35.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.191+0000 7f6bd8132740 10 client.0 _ll_register_callbacks cb 0x563404661a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:35.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.191+0000 7f6bcf7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404778b60 0x563404780c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.191+0000 7f6bcf7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404778b60 0x563404780c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55152/0 (socket says 172.21.15.152:55152) 2024-09-11T22:55:35.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.191+0000 7f6bcf7fe640 1 -- 172.21.15.152:0/4081959763 learned_addr learned my addr 172.21.15.152:0/4081959763 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.191+0000 7f6bceffd640 1 --2- 172.21.15.152:0/4081959763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404779480 0x563404781150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6bcffff640 1 --2- 172.21.15.152:0/4081959763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404783d40 0x563404781690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6bcf7fe640 1 -- 172.21.15.152:0/4081959763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404783d40 msgr2=0x563404781690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6bcf7fe640 1 --2- 172.21.15.152:0/4081959763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404783d40 0x563404781690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6bcf7fe640 1 -- 172.21.15.152:0/4081959763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404779480 msgr2=0x563404781150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6bcf7fe640 1 --2- 172.21.15.152:0/4081959763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404779480 0x563404781150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6bcf7fe640 1 -- 172.21.15.152:0/4081959763 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56340444f6a0 con 0x563404778b60 2024-09-11T22:55:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6bceffd640 1 --2- 172.21.15.152:0/4081959763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404779480 0x563404781150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6bcf7fe640 1 --2- 172.21.15.152:0/4081959763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404778b60 0x563404780c10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00b8260 tx=0x7f6bc008ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef88c29d39226038 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6baeffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6baeffd640 1 -- 172.21.15.152:0/4081959763 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0049040 con 0x563404778b60 2024-09-11T22:55:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6baeffd640 1 -- 172.21.15.152:0/4081959763 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f6bc0003c80 con 0x563404778b60 2024-09-11T22:55:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6bd8132740 1 -- 172.21.15.152:0/4081959763 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bc401a940 con 0x563404778b60 2024-09-11T22:55:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6baeffd640 1 -- 172.21.15.152:0/4081959763 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6bc00bc500 con 0x563404778b60 2024-09-11T22:55:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6bd8132740 10 client.16278 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6bd8132740 1 -- 172.21.15.152:0/4081959763 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563404792510 con 0x563404778b60 2024-09-11T22:55:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6baeffd640 1 -- 172.21.15.152:0/4081959763 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bc401a940 con 0x563404778b60 2024-09-11T22:55:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6baeffd640 1 -- 172.21.15.152:0/4081959763 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6bc00b8470 con 0x563404778b60 2024-09-11T22:55:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6bd8132740 10 client.16278 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.192+0000 7f6bd8132740 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.193+0000 7f6bd8132740 20 client.16278 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.193+0000 7f6bd8132740 1 --2- 172.21.15.152:0/4081959763 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x563404793900 0x563404795d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.193+0000 7f6bd8132740 4 client.16278 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:55:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.193+0000 7f6bd8132740 1 -- 172.21.15.152:0/4081959763 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634046eb7f0 con 0x563404793900 2024-09-11T22:55:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.193+0000 7f6bceffd640 1 --2- 172.21.15.152:0/4081959763 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x563404793900 0x563404795d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.193+0000 7f6baeffd640 10 client.16278 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.193+0000 7f6bceffd640 1 --2- 172.21.15.152:0/4081959763 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x563404793900 0x563404795d30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c3b59c3e855dd7 server_cookie=b6f98ac6962b4bf in_seq=0 out_seq=0 2024-09-11T22:55:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.194+0000 7f6baeffd640 1 -- 172.21.15.152:0/4081959763 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634046eb7f0 con 0x563404793900 2024-09-11T22:55:35.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.194+0000 7f6baeffd640 10 client.16278 handle_command_reply: tid=0 2024-09-11T22:55:35.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.194+0000 7f6bd8132740 1 client.16278 shutdown 2024-09-11T22:55:35.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.194+0000 7f6bd8132740 1 -- 172.21.15.152:0/4081959763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404778b60 msgr2=0x563404780c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:35.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.194+0000 7f6bd8132740 1 --2- 172.21.15.152:0/4081959763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404778b60 0x563404780c10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00b8260 tx=0x7f6bc008ad30 comp rx=0 tx=0).stop 2024-09-11T22:55:35.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.194+0000 7f6bd8132740 1 -- 172.21.15.152:0/4081959763 shutdown_connections 2024-09-11T22:55:35.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.194+0000 7f6bd8132740 1 --2- 172.21.15.152:0/4081959763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404783d40 0x563404781690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.194+0000 7f6bd8132740 1 --2- 172.21.15.152:0/4081959763 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x563404793900 0x563404795d30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.194+0000 7f6bd8132740 1 --2- 172.21.15.152:0/4081959763 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404779480 0x563404781150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.194+0000 7f6bd8132740 1 --2- 172.21.15.152:0/4081959763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404778b60 0x563404780c10 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.194+0000 7f6baeffd640 0 client.16278 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:35.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.194+0000 7f6bd8132740 1 -- 172.21.15.152:0/4081959763 >> 172.21.15.152:0/4081959763 conn(0x5634047750f0 msgr2=0x56340477add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:35.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.194+0000 7f6bd8132740 1 -- 172.21.15.152:0/4081959763 shutdown_connections 2024-09-11T22:55:35.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.194+0000 7f6bd8132740 1 -- 172.21.15.152:0/4081959763 wait complete. 2024-09-11T22:55:35.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.194+0000 7f6bd8132740 20 client.16278 trim_cache size 0 max 16384 2024-09-11T22:55:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.211+0000 7f6bd8132740 1 Processor -- start 2024-09-11T22:55:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.211+0000 7f6bd8132740 1 -- start start 2024-09-11T22:55:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bd8132740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404778b60 0x56340477acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bd8132740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404779480 0x56340479b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bd8132740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404783d40 0x56340479bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bd8132740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01ad7f0 con 0x563404778b60 2024-09-11T22:55:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bd8132740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01af330 con 0x563404783d40 2024-09-11T22:55:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bd8132740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01adec0 con 0x563404779480 2024-09-11T22:55:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bcf7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404778b60 0x56340477acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bcf7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404778b60 0x56340477acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55164/0 (socket says 172.21.15.152:55164) 2024-09-11T22:55:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bcf7fe640 1 -- 172.21.15.152:0/1234151649 learned_addr learned my addr 172.21.15.152:0/1234151649 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bcf7fe640 1 -- 172.21.15.152:0/1234151649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404779480 msgr2=0x56340479b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bceffd640 1 --2- 172.21.15.152:0/1234151649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404779480 0x56340479b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bcf7fe640 1 --2- 172.21.15.152:0/1234151649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404779480 0x56340479b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bcf7fe640 1 -- 172.21.15.152:0/1234151649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404783d40 msgr2=0x56340479bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:35.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bcf7fe640 1 --2- 172.21.15.152:0/1234151649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404783d40 0x56340479bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bcf7fe640 1 -- 172.21.15.152:0/1234151649 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563404774f20 con 0x563404778b60 2024-09-11T22:55:35.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bcf7fe640 1 --2- 172.21.15.152:0/1234151649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404778b60 0x56340477acc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0000c00 tx=0x7f6bc002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d81d3dbd6f32e4a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:35.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bb57fa640 1 -- 172.21.15.152:0/1234151649 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0049070 con 0x563404778b60 2024-09-11T22:55:35.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bb57fa640 1 -- 172.21.15.152:0/1234151649 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bc00bbe00 con 0x563404778b60 2024-09-11T22:55:35.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.212+0000 7f6bb57fa640 1 -- 172.21.15.152:0/1234151649 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc008a040 con 0x563404778b60 2024-09-11T22:55:35.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.213+0000 7f6bd8132740 1 -- 172.21.15.152:0/1234151649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404778b60 msgr2=0x56340477acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:35.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.213+0000 7f6bd8132740 1 --2- 172.21.15.152:0/1234151649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404778b60 0x56340477acc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0000c00 tx=0x7f6bc002a420 comp rx=0 tx=0).stop 2024-09-11T22:55:35.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.213+0000 7f6bd8132740 1 -- 172.21.15.152:0/1234151649 shutdown_connections 2024-09-11T22:55:35.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.213+0000 7f6bd8132740 1 --2- 172.21.15.152:0/1234151649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404783d40 0x56340479bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.213+0000 7f6bd8132740 1 --2- 172.21.15.152:0/1234151649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404779480 0x56340479b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.213+0000 7f6bd8132740 1 --2- 172.21.15.152:0/1234151649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404778b60 0x56340477acc0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.213+0000 7f6bd8132740 1 -- 172.21.15.152:0/1234151649 >> 172.21.15.152:0/1234151649 conn(0x5634047750f0 msgr2=0x56340477a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.213+0000 7f6bd8132740 1 -- 172.21.15.152:0/1234151649 shutdown_connections 2024-09-11T22:55:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.213+0000 7f6bd8132740 1 -- 172.21.15.152:0/1234151649 wait complete. 2024-09-11T22:55:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.213+0000 7f6bd8132740 1 -- start start 2024-09-11T22:55:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bd8132740 1 Processor -- start 2024-09-11T22:55:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bd8132740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404778b60 0x5634047882a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bd8132740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404779480 0x5634047887e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bd8132740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404783d40 0x5634047f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bd8132740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01aec60 con 0x563404783d40 2024-09-11T22:55:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bd8132740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01ae590 con 0x563404778b60 2024-09-11T22:55:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bd8132740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd01ab7b0 con 0x563404779480 2024-09-11T22:55:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bd8132740 10 client.0 _ll_register_callbacks cb 0x7f6bc0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bcffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404783d40 0x5634047f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bcffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404783d40 0x5634047f75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55180/0 (socket says 172.21.15.152:55180) 2024-09-11T22:55:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bcffff640 1 -- 172.21.15.152:0/2246475610 learned_addr learned my addr 172.21.15.152:0/2246475610 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bcf7fe640 1 --2- 172.21.15.152:0/2246475610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404778b60 0x5634047882a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bceffd640 1 --2- 172.21.15.152:0/2246475610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404779480 0x5634047887e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bcffff640 1 -- 172.21.15.152:0/2246475610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404779480 msgr2=0x5634047887e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bcffff640 1 --2- 172.21.15.152:0/2246475610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404779480 0x5634047887e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bcffff640 1 -- 172.21.15.152:0/2246475610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404778b60 msgr2=0x5634047882a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bcffff640 1 --2- 172.21.15.152:0/2246475610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404778b60 0x5634047882a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bcffff640 1 -- 172.21.15.152:0/2246475610 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56340477b9d0 con 0x563404783d40 2024-09-11T22:55:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6bcffff640 1 --2- 172.21.15.152:0/2246475610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404783d40 0x5634047f75b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8011f90 tx=0x7f6bc800a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991202ae76967a7f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.214+0000 7f6baeffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.215+0000 7f6baeffd640 1 -- 172.21.15.152:0/2246475610 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8024070 con 0x563404783d40 2024-09-11T22:55:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.215+0000 7f6baeffd640 1 -- 172.21.15.152:0/2246475610 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f6bc80205f0 con 0x563404783d40 2024-09-11T22:55:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.215+0000 7f6bd8132740 1 -- 172.21.15.152:0/2246475610 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bc00bbe00 con 0x563404783d40 2024-09-11T22:55:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.215+0000 7f6baeffd640 1 -- 172.21.15.152:0/2246475610 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6bc8030050 con 0x563404783d40 2024-09-11T22:55:35.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.215+0000 7f6bd8132740 10 client.16284 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:35.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.215+0000 7f6bd8132740 1 -- 172.21.15.152:0/2246475610 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5634047fee20 con 0x563404783d40 2024-09-11T22:55:35.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.215+0000 7f6baeffd640 1 -- 172.21.15.152:0/2246475610 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bc00bbe00 con 0x563404783d40 2024-09-11T22:55:35.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.215+0000 7f6baeffd640 1 -- 172.21.15.152:0/2246475610 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6bc801f070 con 0x563404783d40 2024-09-11T22:55:35.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.215+0000 7f6bd8132740 10 client.16284 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:35.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.215+0000 7f6bd8132740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:35.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.215+0000 7f6bd8132740 20 client.16284 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:35.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.215+0000 7f6bd8132740 1 --2- 172.21.15.152:0/2246475610 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5634047ff5a0 0x5634048019c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:35.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.215+0000 7f6bd8132740 4 client.16284 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:55:35.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.215+0000 7f6bd8132740 1 -- 172.21.15.152:0/2246475610 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5634046e5d00 con 0x5634047ff5a0 2024-09-11T22:55:35.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.215+0000 7f6bcf7fe640 1 --2- 172.21.15.152:0/2246475610 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5634047ff5a0 0x5634048019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:35.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.216+0000 7f6baeffd640 10 client.16284 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:35.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.216+0000 7f6bcf7fe640 1 --2- 172.21.15.152:0/2246475610 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5634047ff5a0 0x5634048019c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a412d3e40fe1a165 server_cookie=803ed1353738a5d in_seq=0 out_seq=0 2024-09-11T22:55:35.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.217+0000 7f6baeffd640 1 -- 172.21.15.152:0/2246475610 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5634046e5d00 con 0x5634047ff5a0 2024-09-11T22:55:35.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.217+0000 7f6baeffd640 10 client.16284 handle_command_reply: tid=0 2024-09-11T22:55:35.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.217+0000 7f6bd8132740 1 client.16284 shutdown 2024-09-11T22:55:35.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.217+0000 7f6bd8132740 1 -- 172.21.15.152:0/2246475610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404783d40 msgr2=0x5634047f75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:35.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.217+0000 7f6bd8132740 1 --2- 172.21.15.152:0/2246475610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404783d40 0x5634047f75b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8011f90 tx=0x7f6bc800a9e0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.217+0000 7f6bd8132740 1 -- 172.21.15.152:0/2246475610 shutdown_connections 2024-09-11T22:55:35.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.217+0000 7f6bd8132740 1 --2- 172.21.15.152:0/2246475610 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563404783d40 0x5634047f75b0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.217+0000 7f6bd8132740 1 --2- 172.21.15.152:0/2246475610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563404779480 0x5634047887e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.217+0000 7f6bd8132740 1 --2- 172.21.15.152:0/2246475610 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5634047ff5a0 0x5634048019c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.217+0000 7f6bd8132740 1 --2- 172.21.15.152:0/2246475610 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563404778b60 0x5634047882a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.217+0000 7f6baeffd640 0 client.16284 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:35.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.217+0000 7f6bd8132740 1 -- 172.21.15.152:0/2246475610 >> 172.21.15.152:0/2246475610 conn(0x5634047750f0 msgr2=0x563404774270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:35.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.217+0000 7f6bd8132740 1 -- 172.21.15.152:0/2246475610 shutdown_connections 2024-09-11T22:55:35.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.217+0000 7f6bd8132740 1 -- 172.21.15.152:0/2246475610 wait complete. 2024-09-11T22:55:35.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.217+0000 7f6bd8132740 20 client.16284 trim_cache size 0 max 16384 2024-09-11T22:55:35.234 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:55:35.234 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T22:55:35.234 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "06fa1d57-e5b9-4c24-b7c1-1603eaf89e55", 2024-09-11T22:55:35.234 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T22:55:35.234 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:55:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.218+0000 7f6bd5d59640 1 -- 172.21.15.152:0/1090739066 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6bb80a2d70 msgr2=0x7f6bb80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.218+0000 7f6bd5d59640 1 --2- 172.21.15.152:0/1090739066 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6bb80a2d70 0x7f6bb80a5230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800afd0 tx=0x7f6bc800a4d0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.218+0000 7f6bd5d59640 1 -- 172.21.15.152:0/1090739066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd0100a80 msgr2=0x7f6bd01984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.218+0000 7f6bd5d59640 1 --2- 172.21.15.152:0/1090739066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd0100a80 0x7f6bd01984a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009230 tx=0x7f6bc0004270 comp rx=0 tx=0).stop 2024-09-11T22:55:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.218+0000 7f6bd5d59640 1 -- 172.21.15.152:0/1090739066 shutdown_connections 2024-09-11T22:55:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.218+0000 7f6bd5d59640 1 --2- 172.21.15.152:0/1090739066 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6bb80a2d70 0x7f6bb80a5230 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.218+0000 7f6bd5d59640 1 --2- 172.21.15.152:0/1090739066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bd0106a80 0x7f6bd019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.218+0000 7f6bd5d59640 1 --2- 172.21.15.152:0/1090739066 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6bd0101420 0x7f6bd01989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.218+0000 7f6bd5d59640 1 --2- 172.21.15.152:0/1090739066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6bd0100a80 0x7f6bd01984a0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.218+0000 7f6bd5d59640 1 -- 172.21.15.152:0/1090739066 >> 172.21.15.152:0/1090739066 conn(0x7f6bd00fc7a0 msgr2=0x7f6bd0104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.218+0000 7f6bd5d59640 1 -- 172.21.15.152:0/1090739066 shutdown_connections 2024-09-11T22:55:35.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:35.218+0000 7f6bd5d59640 1 -- 172.21.15.152:0/1090739066 wait complete. 2024-09-11T22:55:35.996 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-09-11T22:55:35.996 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:55:36.056 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:55:36.264 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:55:36.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:36 smithi152 ceph-mon[31181]: pgmap v84: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 213 MiB/s wr, 68 op/s 2024-09-11T22:55:36.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:36 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T22:55:36.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:36 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:55:36.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:36 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T22:55:36.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:36 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:55:36.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:36 smithi191 ceph-mon[35211]: pgmap v84: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 213 MiB/s wr, 68 op/s 2024-09-11T22:55:36.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:36 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T22:55:36.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:36 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:55:36.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:36 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T22:55:36.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:36 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:55:36.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:36 smithi204 ceph-mon[35454]: pgmap v84: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 213 MiB/s wr, 68 op/s 2024-09-11T22:55:36.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:36 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T22:55:36.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:36 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:55:36.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:36 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T22:55:36.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:36 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:55:36.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.806+0000 7fd6b1f0b640 1 -- 172.21.15.152:0/2550495320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6ac101420 msgr2=0x7fd6ac10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:36.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.806+0000 7fd6b1f0b640 1 --2- 172.21.15.152:0/2550495320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6ac101420 0x7fd6ac10d950 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009cd0 tx=0x7fd6a402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:55:36.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.807+0000 7fd6b1f0b640 1 -- 172.21.15.152:0/2550495320 shutdown_connections 2024-09-11T22:55:36.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.807+0000 7fd6b1f0b640 1 --2- 172.21.15.152:0/2550495320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6ac101420 0x7fd6ac10d950 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:36.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.807+0000 7fd6b1f0b640 1 --2- 172.21.15.152:0/2550495320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac100a80 0x7fd6ac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:36.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.807+0000 7fd6b1f0b640 1 --2- 172.21.15.152:0/2550495320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6ac106a80 0x7fd6ac106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:36.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.807+0000 7fd6b1f0b640 1 -- 172.21.15.152:0/2550495320 >> 172.21.15.152:0/2550495320 conn(0x7fd6ac0fc7a0 msgr2=0x7fd6ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:36.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.807+0000 7fd6b1f0b640 1 -- 172.21.15.152:0/2550495320 shutdown_connections 2024-09-11T22:55:36.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.807+0000 7fd6b1f0b640 1 -- 172.21.15.152:0/2550495320 wait complete. 2024-09-11T22:55:36.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.807+0000 7fd6b1f0b640 1 Processor -- start 2024-09-11T22:55:36.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.807+0000 7fd6b1f0b640 1 -- start start 2024-09-11T22:55:36.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6b1f0b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6ac100a80 0x7fd6ac1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:36.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6b1f0b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac101420 0x7fd6ac199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:36.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6b1f0b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6ac106a80 0x7fd6ac1a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:36.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6b1f0b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac110e80 con 0x7fd6ac100a80 2024-09-11T22:55:36.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6b1f0b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac110d00 con 0x7fd6ac106a80 2024-09-11T22:55:36.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6b1f0b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac111000 con 0x7fd6ac101420 2024-09-11T22:55:36.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6ab7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6ac100a80 0x7fd6ac1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:36.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6ab7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6ac100a80 0x7fd6ac1991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55192/0 (socket says 172.21.15.152:55192) 2024-09-11T22:55:36.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6ab7fe640 1 -- 172.21.15.152:0/1008181633 learned_addr learned my addr 172.21.15.152:0/1008181633 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:36.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6abfff640 1 --2- 172.21.15.152:0/1008181633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6ac106a80 0x7fd6ac1a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:36.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6a2ffd640 1 --2- 172.21.15.152:0/1008181633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac101420 0x7fd6ac199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:36.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6ab7fe640 1 -- 172.21.15.152:0/1008181633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac101420 msgr2=0x7fd6ac199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:36.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6ab7fe640 1 --2- 172.21.15.152:0/1008181633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac101420 0x7fd6ac199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:36.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6ab7fe640 1 -- 172.21.15.152:0/1008181633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6ac106a80 msgr2=0x7fd6ac1a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:36.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6ab7fe640 1 --2- 172.21.15.152:0/1008181633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6ac106a80 0x7fd6ac1a4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:36.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6ab7fe640 1 -- 172.21.15.152:0/1008181633 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ac1a4790 con 0x7fd6ac100a80 2024-09-11T22:55:36.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6abfff640 1 --2- 172.21.15.152:0/1008181633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6ac106a80 0x7fd6ac1a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:36.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6ab7fe640 1 --2- 172.21.15.152:0/1008181633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6ac100a80 0x7fd6ac1991d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00c920 tx=0x7fd69c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:36.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6a97fa640 1 -- 172.21.15.152:0/1008181633 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c007c70 con 0x7fd6ac100a80 2024-09-11T22:55:36.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6b1f0b640 1 -- 172.21.15.152:0/1008181633 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ac1a4960 con 0x7fd6ac100a80 2024-09-11T22:55:36.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6a97fa640 1 -- 172.21.15.152:0/1008181633 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd69c007e10 con 0x7fd6ac100a80 2024-09-11T22:55:36.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6a97fa640 1 -- 172.21.15.152:0/1008181633 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69c018890 con 0x7fd6ac100a80 2024-09-11T22:55:36.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.808+0000 7fd6b1f0b640 1 -- 172.21.15.152:0/1008181633 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6ac1a4e70 con 0x7fd6ac100a80 2024-09-11T22:55:36.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.809+0000 7fd6b1f0b640 1 -- 172.21.15.152:0/1008181633 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6ac100ee0 con 0x7fd6ac100a80 2024-09-11T22:55:36.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.811+0000 7fd6a97fa640 1 -- 172.21.15.152:0/1008181633 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd69c018a50 con 0x7fd6ac100a80 2024-09-11T22:55:36.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.812+0000 7fd6a97fa640 1 --2- 172.21.15.152:0/1008181633 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd6980a2cf0 0x7fd6980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:36.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.812+0000 7fd6a2ffd640 1 --2- 172.21.15.152:0/1008181633 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd6980a2cf0 0x7fd6980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:36.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.812+0000 7fd6a97fa640 1 -- 172.21.15.152:0/1008181633 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fd69c0cf7f0 con 0x7fd6ac100a80 2024-09-11T22:55:36.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.812+0000 7fd6a2ffd640 1 --2- 172.21.15.152:0/1008181633 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd6980a2cf0 0x7fd6980a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd694005fd0 tx=0x7fd694005850 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:36.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:36.815+0000 7fd6a97fa640 1 -- 172.21.15.152:0/1008181633 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd69c014030 con 0x7fd6ac100a80 2024-09-11T22:55:36.847 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:55:37.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.024+0000 7fd6b1f0b640 1 -- 172.21.15.152:0/1008181633 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6ac06ad80 con 0x7fd6ac100a80 2024-09-11T22:55:37.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.025+0000 7fd6a97fa640 1 -- 172.21.15.152:0/1008181633 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7fd69c00de20 con 0x7fd6ac100a80 2024-09-11T22:55:37.026 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:55:37.026 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:55:37.026 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:55:37.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.027+0000 7fd6b1f0b640 1 -- 172.21.15.152:0/1008181633 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd6980a2cf0 msgr2=0x7fd6980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:37.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.027+0000 7fd6b1f0b640 1 --2- 172.21.15.152:0/1008181633 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd6980a2cf0 0x7fd6980a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd694005fd0 tx=0x7fd694005850 comp rx=0 tx=0).stop 2024-09-11T22:55:37.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.027+0000 7fd6b1f0b640 1 -- 172.21.15.152:0/1008181633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6ac100a80 msgr2=0x7fd6ac1991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:37.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.027+0000 7fd6b1f0b640 1 --2- 172.21.15.152:0/1008181633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6ac100a80 0x7fd6ac1991d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00c920 tx=0x7fd69c00cde0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.027+0000 7fd6b1f0b640 1 -- 172.21.15.152:0/1008181633 shutdown_connections 2024-09-11T22:55:37.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.027+0000 7fd6b1f0b640 1 --2- 172.21.15.152:0/1008181633 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd6980a2cf0 0x7fd6980a51b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.027+0000 7fd6b1f0b640 1 --2- 172.21.15.152:0/1008181633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd6ac106a80 0x7fd6ac1a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.027+0000 7fd6b1f0b640 1 --2- 172.21.15.152:0/1008181633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6ac101420 0x7fd6ac199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.027+0000 7fd6b1f0b640 1 --2- 172.21.15.152:0/1008181633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd6ac100a80 0x7fd6ac1991d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.027+0000 7fd6b1f0b640 1 -- 172.21.15.152:0/1008181633 >> 172.21.15.152:0/1008181633 conn(0x7fd6ac0fc7a0 msgr2=0x7fd6ac0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:37.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.027+0000 7fd6b1f0b640 1 -- 172.21.15.152:0/1008181633 shutdown_connections 2024-09-11T22:55:37.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.027+0000 7fd6b1f0b640 1 -- 172.21.15.152:0/1008181633 wait complete. 2024-09-11T22:55:37.533 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:37 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1008181633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:55:37.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:37 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1008181633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:55:37.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:37 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1008181633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:55:37.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.532+0000 7f00c0bd8640 1 -- 172.21.15.152:0/2902624170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00bc102ef0 msgr2=0x7f00bc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:37.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.532+0000 7f00c0bd8640 1 --2- 172.21.15.152:0/2902624170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00bc102ef0 0x7f00bc10da40 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009ca0 tx=0x7f00b402a950 comp rx=0 tx=0).stop 2024-09-11T22:55:37.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.532+0000 7f00c0bd8640 1 -- 172.21.15.152:0/2902624170 shutdown_connections 2024-09-11T22:55:37.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.532+0000 7f00c0bd8640 1 --2- 172.21.15.152:0/2902624170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00bc102ef0 0x7f00bc10da40 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.532+0000 7f00c0bd8640 1 --2- 172.21.15.152:0/2902624170 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00bc102570 0x7f00bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.532+0000 7f00c0bd8640 1 --2- 172.21.15.152:0/2902624170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00bc100e20 0x7f00bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.532+0000 7f00c0bd8640 1 -- 172.21.15.152:0/2902624170 >> 172.21.15.152:0/2902624170 conn(0x7f00bc0fc7d0 msgr2=0x7f00bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:37.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.532+0000 7f00c0bd8640 1 -- 172.21.15.152:0/2902624170 shutdown_connections 2024-09-11T22:55:37.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.532+0000 7f00c0bd8640 1 -- 172.21.15.152:0/2902624170 wait complete. 2024-09-11T22:55:37.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.533+0000 7f00c0bd8640 1 Processor -- start 2024-09-11T22:55:37.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.533+0000 7f00c0bd8640 1 -- start start 2024-09-11T22:55:37.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.533+0000 7f00c0bd8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00bc100e20 0x7f00bc1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.533+0000 7f00c0bd8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00bc102570 0x7f00bc199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.533+0000 7f00c0bd8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00bc102ef0 0x7f00bc199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.533+0000 7f00c0bd8640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc110f80 con 0x7f00bc102ef0 2024-09-11T22:55:37.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.533+0000 7f00c0bd8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc110e00 con 0x7f00bc100e20 2024-09-11T22:55:37.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.533+0000 7f00c0bd8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc111100 con 0x7f00bc102570 2024-09-11T22:55:37.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.534+0000 7f00bad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00bc102ef0 0x7f00bc199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.534+0000 7f00bad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00bc102ef0 0x7f00bc199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55202/0 (socket says 172.21.15.152:55202) 2024-09-11T22:55:37.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.534+0000 7f00bad9d640 1 -- 172.21.15.152:0/4116994457 learned_addr learned my addr 172.21.15.152:0/4116994457 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:37.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.534+0000 7f00ba59c640 1 --2- 172.21.15.152:0/4116994457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00bc100e20 0x7f00bc1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.534+0000 7f00b3fff640 1 --2- 172.21.15.152:0/4116994457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00bc102570 0x7f00bc199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.534+0000 7f00bad9d640 1 -- 172.21.15.152:0/4116994457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00bc102570 msgr2=0x7f00bc199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.534+0000 7f00bad9d640 1 --2- 172.21.15.152:0/4116994457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00bc102570 0x7f00bc199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.534+0000 7f00bad9d640 1 -- 172.21.15.152:0/4116994457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00bc100e20 msgr2=0x7f00bc1993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.534+0000 7f00bad9d640 1 --2- 172.21.15.152:0/4116994457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00bc100e20 0x7f00bc1993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.534+0000 7f00bad9d640 1 -- 172.21.15.152:0/4116994457 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc1a47d0 con 0x7f00bc102ef0 2024-09-11T22:55:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.534+0000 7f00bad9d640 1 --2- 172.21.15.152:0/4116994457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00bc102ef0 0x7f00bc199e90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f00b402ae70 tx=0x7f00b402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.534+0000 7f00b37fe640 1 -- 172.21.15.152:0/4116994457 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4034040 con 0x7f00bc102ef0 2024-09-11T22:55:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.534+0000 7f00c0bd8640 1 -- 172.21.15.152:0/4116994457 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc1a4a60 con 0x7f00bc102ef0 2024-09-11T22:55:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.534+0000 7f00b37fe640 1 -- 172.21.15.152:0/4116994457 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b4030920 con 0x7f00bc102ef0 2024-09-11T22:55:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.534+0000 7f00b37fe640 1 -- 172.21.15.152:0/4116994457 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4045480 con 0x7f00bc102ef0 2024-09-11T22:55:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.534+0000 7f00c0bd8640 1 -- 172.21.15.152:0/4116994457 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc1971c0 con 0x7f00bc102ef0 2024-09-11T22:55:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.537+0000 7f00b37fe640 1 -- 172.21.15.152:0/4116994457 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f00b4037020 con 0x7f00bc102ef0 2024-09-11T22:55:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.537+0000 7f00b37fe640 1 --2- 172.21.15.152:0/4116994457 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f00a80a7100 0x7f00a80a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.537+0000 7f00b37fe640 1 -- 172.21.15.152:0/4116994457 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f00b40f01f0 con 0x7f00bc102ef0 2024-09-11T22:55:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.537+0000 7f00ba59c640 1 --2- 172.21.15.152:0/4116994457 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f00a80a7100 0x7f00a80a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.537+0000 7f00c2fb1740 1 Processor -- start 2024-09-11T22:55:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00c2fb1740 1 -- start start 2024-09-11T22:55:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00c2fb1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a604a0 0x558848a60880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00c2fb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a60dc0 0x558848a6b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00c2fb1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a6b680 0x558848a6da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00c2fb1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5588489c62e0 con 0x558848a6b680 2024-09-11T22:55:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00c2fb1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55884896dce0 con 0x558848a604a0 2024-09-11T22:55:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00c2fb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5588489c22a0 con 0x558848a60dc0 2024-09-11T22:55:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00ba59c640 1 --2- 172.21.15.152:0/4116994457 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f00a80a7100 0x7f00a80a95c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f00ac005fd0 tx=0x7f00ac00b390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00bad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a6b680 0x558848a6da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00bad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a6b680 0x558848a6da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55204/0 (socket says 172.21.15.152:55204) 2024-09-11T22:55:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00bad9d640 1 -- 172.21.15.152:0/3104778370 learned_addr learned my addr 172.21.15.152:0/3104778370 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00ba59c640 1 --2- 172.21.15.152:0/3104778370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a604a0 0x558848a60880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00b3fff640 1 --2- 172.21.15.152:0/3104778370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a60dc0 0x558848a6b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00bad9d640 1 -- 172.21.15.152:0/3104778370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a60dc0 msgr2=0x558848a6b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00bad9d640 1 --2- 172.21.15.152:0/3104778370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a60dc0 0x558848a6b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00bad9d640 1 -- 172.21.15.152:0/3104778370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a604a0 msgr2=0x558848a60880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.538+0000 7f00bad9d640 1 --2- 172.21.15.152:0/3104778370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a604a0 0x558848a60880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.539+0000 7f00bad9d640 1 -- 172.21.15.152:0/3104778370 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558848952440 con 0x558848a6b680 2024-09-11T22:55:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.539+0000 7f00bad9d640 1 --2- 172.21.15.152:0/3104778370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a6b680 0x558848a6da70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f00b40c4f70 tx=0x7f00b40098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c74a87d4bf762d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.539+0000 7f00b0ff9640 1 -- 172.21.15.152:0/3104778370 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b40bc830 con 0x558848a6b680 2024-09-11T22:55:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.539+0000 7f00b0ff9640 1 -- 172.21.15.152:0/3104778370 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b40c0be0 con 0x558848a6b680 2024-09-11T22:55:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.539+0000 7f00b0ff9640 1 -- 172.21.15.152:0/3104778370 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b40b8ea0 con 0x558848a6b680 2024-09-11T22:55:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 -- 172.21.15.152:0/3104778370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a6b680 msgr2=0x558848a6da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/3104778370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a6b680 0x558848a6da70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f00b40c4f70 tx=0x7f00b40098d0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 -- 172.21.15.152:0/3104778370 shutdown_connections 2024-09-11T22:55:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/3104778370 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a6b680 0x558848a6da70 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/3104778370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a60dc0 0x558848a6b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/3104778370 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a604a0 0x558848a60880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 -- 172.21.15.152:0/3104778370 >> 172.21.15.152:0/3104778370 conn(0x558848a5ca30 msgr2=0x558848a5ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 -- 172.21.15.152:0/3104778370 shutdown_connections 2024-09-11T22:55:37.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 -- 172.21.15.152:0/3104778370 wait complete. 2024-09-11T22:55:37.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 -- start start 2024-09-11T22:55:37.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 Processor -- start 2024-09-11T22:55:37.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a604a0 0x558848a6afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a60dc0 0x558848a68250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a6b680 0x558848a68790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558848a6e010 con 0x558848a6b680 2024-09-11T22:55:37.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558848981050 con 0x558848a604a0 2024-09-11T22:55:37.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558848a6e190 con 0x558848a60dc0 2024-09-11T22:55:37.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.540+0000 7f00c2fb1740 10 client.0 _ll_register_callbacks cb 0x5588489c1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:37.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00bad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a6b680 0x558848a68790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00bad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a6b680 0x558848a68790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55218/0 (socket says 172.21.15.152:55218) 2024-09-11T22:55:37.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00bad9d640 1 -- 172.21.15.152:0/636431577 learned_addr learned my addr 172.21.15.152:0/636431577 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:37.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00ba59c640 1 --2- 172.21.15.152:0/636431577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a604a0 0x558848a6afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00b3fff640 1 --2- 172.21.15.152:0/636431577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a60dc0 0x558848a68250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00bad9d640 1 -- 172.21.15.152:0/636431577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a60dc0 msgr2=0x558848a68250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:37.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00bad9d640 1 --2- 172.21.15.152:0/636431577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a60dc0 0x558848a68250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00bad9d640 1 -- 172.21.15.152:0/636431577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a604a0 msgr2=0x558848a6afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:37.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00bad9d640 1 --2- 172.21.15.152:0/636431577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a604a0 0x558848a6afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00bad9d640 1 -- 172.21.15.152:0/636431577 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588487373c0 con 0x558848a6b680 2024-09-11T22:55:37.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00bad9d640 1 --2- 172.21.15.152:0/636431577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a6b680 0x558848a68790 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f00b40b9200 tx=0x7f00b402f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209f5f6afbe807f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:37.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:37.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00a1ffb640 1 -- 172.21.15.152:0/636431577 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b40d9070 con 0x558848a6b680 2024-09-11T22:55:37.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00a1ffb640 1 -- 172.21.15.152:0/636431577 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f00b40a70b0 con 0x558848a6b680 2024-09-11T22:55:37.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00c2fb1740 1 -- 172.21.15.152:0/636431577 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00b40c0be0 con 0x558848a6b680 2024-09-11T22:55:37.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00a1ffb640 1 -- 172.21.15.152:0/636431577 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f00b4004040 con 0x558848a6b680 2024-09-11T22:55:37.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00c2fb1740 10 client.16302 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:37.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00c2fb1740 1 -- 172.21.15.152:0/636431577 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558848a6e010 con 0x558848a6b680 2024-09-11T22:55:37.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.541+0000 7f00a1ffb640 1 -- 172.21.15.152:0/636431577 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b40c0be0 con 0x558848a6b680 2024-09-11T22:55:37.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.542+0000 7f00a1ffb640 1 -- 172.21.15.152:0/636431577 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f00b404d030 con 0x558848a6b680 2024-09-11T22:55:37.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.542+0000 7f00c2fb1740 10 client.16302 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:37.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.542+0000 7f00c2fb1740 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:37.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.542+0000 7f00c2fb1740 20 client.16302 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:37.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.542+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/636431577 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558848a7aa80 0x558848a7ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.542+0000 7f00c2fb1740 4 client.16302 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:55:37.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.542+0000 7f00c2fb1740 1 -- 172.21.15.152:0/636431577 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588489d3850 con 0x558848a7aa80 2024-09-11T22:55:37.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.542+0000 7f00ba59c640 1 --2- 172.21.15.152:0/636431577 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558848a7aa80 0x558848a7ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.542+0000 7f00ba59c640 1 --2- 172.21.15.152:0/636431577 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558848a7aa80 0x558848a7ce70 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a51c87b053b0ff6d server_cookie=7d9c01c22400d2d0 in_seq=0 out_seq=0 2024-09-11T22:55:37.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.542+0000 7f00a1ffb640 10 client.16302 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:37.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.543+0000 7f00a1ffb640 1 -- 172.21.15.152:0/636431577 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588489d3850 con 0x558848a7aa80 2024-09-11T22:55:37.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.543+0000 7f00a1ffb640 10 client.16302 handle_command_reply: tid=0 2024-09-11T22:55:37.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.543+0000 7f00c2fb1740 1 client.16302 shutdown 2024-09-11T22:55:37.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.543+0000 7f00c2fb1740 1 -- 172.21.15.152:0/636431577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a6b680 msgr2=0x558848a68790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:37.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.543+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/636431577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a6b680 0x558848a68790 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f00b40b9200 tx=0x7f00b402f850 comp rx=0 tx=0).stop 2024-09-11T22:55:37.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.543+0000 7f00c2fb1740 1 -- 172.21.15.152:0/636431577 shutdown_connections 2024-09-11T22:55:37.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.543+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/636431577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a6b680 0x558848a68790 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.543+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/636431577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a60dc0 0x558848a68250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.543+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/636431577 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558848a7aa80 0x558848a7ce70 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.543+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/636431577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a604a0 0x558848a6afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.543+0000 7f00c2fb1740 1 -- 172.21.15.152:0/636431577 >> 172.21.15.152:0/636431577 conn(0x558848a5ca30 msgr2=0x558848a62760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:37.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.543+0000 7f00a1ffb640 0 client.16302 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:37.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.543+0000 7f00c2fb1740 1 -- 172.21.15.152:0/636431577 shutdown_connections 2024-09-11T22:55:37.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.543+0000 7f00c2fb1740 1 -- 172.21.15.152:0/636431577 wait complete. 2024-09-11T22:55:37.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.543+0000 7f00c2fb1740 20 client.16302 trim_cache size 0 max 16384 2024-09-11T22:55:37.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.561+0000 7f00c2fb1740 1 Processor -- start 2024-09-11T22:55:37.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.561+0000 7f00c2fb1740 1 -- start start 2024-09-11T22:55:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.561+0000 7f00c2fb1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a604a0 0x558848a77160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.561+0000 7f00c2fb1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a60dc0 0x558848a637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.561+0000 7f00c2fb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a6b680 0x558848a82600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.561+0000 7f00c2fb1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1ab580 con 0x558848a60dc0 2024-09-11T22:55:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.561+0000 7f00c2fb1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1ad0c0 con 0x558848a604a0 2024-09-11T22:55:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.561+0000 7f00c2fb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1abc50 con 0x558848a6b680 2024-09-11T22:55:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.561+0000 7f00b3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a60dc0 0x558848a637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.561+0000 7f00b3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a60dc0 0x558848a637f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55220/0 (socket says 172.21.15.152:55220) 2024-09-11T22:55:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.561+0000 7f00b3fff640 1 -- 172.21.15.152:0/463219941 learned_addr learned my addr 172.21.15.152:0/463219941 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.561+0000 7f00bad9d640 1 --2- 172.21.15.152:0/463219941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a6b680 0x558848a82600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.561+0000 7f00ba59c640 1 --2- 172.21.15.152:0/463219941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a604a0 0x558848a77160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.562+0000 7f00b3fff640 1 -- 172.21.15.152:0/463219941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a6b680 msgr2=0x558848a82600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.562+0000 7f00b3fff640 1 --2- 172.21.15.152:0/463219941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a6b680 0x558848a82600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.562+0000 7f00b3fff640 1 -- 172.21.15.152:0/463219941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a604a0 msgr2=0x558848a77160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.562+0000 7f00b3fff640 1 --2- 172.21.15.152:0/463219941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a604a0 0x558848a77160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.562+0000 7f00b3fff640 1 -- 172.21.15.152:0/463219941 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558848a667a0 con 0x558848a60dc0 2024-09-11T22:55:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.562+0000 7f00ba59c640 1 --2- 172.21.15.152:0/463219941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a604a0 0x558848a77160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.562+0000 7f00b3fff640 1 --2- 172.21.15.152:0/463219941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a60dc0 0x558848a637f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f00a400fba0 tx=0x7f00a4013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c860ba0ceb2dccc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.562+0000 7f00a3fff640 1 -- 172.21.15.152:0/463219941 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4004030 con 0x558848a60dc0 2024-09-11T22:55:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.562+0000 7f00a3fff640 1 -- 172.21.15.152:0/463219941 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a40041d0 con 0x558848a60dc0 2024-09-11T22:55:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.562+0000 7f00a3fff640 1 -- 172.21.15.152:0/463219941 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4011730 con 0x558848a60dc0 2024-09-11T22:55:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.563+0000 7f00c2fb1740 1 -- 172.21.15.152:0/463219941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a60dc0 msgr2=0x558848a637f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.563+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/463219941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a60dc0 0x558848a637f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f00a400fba0 tx=0x7f00a4013830 comp rx=0 tx=0).stop 2024-09-11T22:55:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.563+0000 7f00c2fb1740 1 -- 172.21.15.152:0/463219941 shutdown_connections 2024-09-11T22:55:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.563+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/463219941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a6b680 0x558848a82600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.563+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/463219941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a60dc0 0x558848a637f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.563+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/463219941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a604a0 0x558848a77160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.563+0000 7f00c2fb1740 1 -- 172.21.15.152:0/463219941 >> 172.21.15.152:0/463219941 conn(0x558848a5ca30 msgr2=0x558848a630e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.563+0000 7f00c2fb1740 1 -- 172.21.15.152:0/463219941 shutdown_connections 2024-09-11T22:55:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.563+0000 7f00c2fb1740 1 -- 172.21.15.152:0/463219941 wait complete. 2024-09-11T22:55:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.563+0000 7f00c2fb1740 1 -- start start 2024-09-11T22:55:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.563+0000 7f00c2fb1740 1 Processor -- start 2024-09-11T22:55:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.563+0000 7f00c2fb1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a604a0 0x558848a73840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.563+0000 7f00c2fb1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a60dc0 0x558848a6ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.563+0000 7f00c2fb1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a6b680 0x558848a6f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00c2fb1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1ac9f0 con 0x558848a604a0 2024-09-11T22:55:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00c2fb1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1ac320 con 0x558848a60dc0 2024-09-11T22:55:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00c2fb1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1a9560 con 0x558848a6b680 2024-09-11T22:55:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00c2fb1740 10 client.0 _ll_register_callbacks cb 0x7f00b40bc400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00ba59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a604a0 0x558848a73840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00ba59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a604a0 0x558848a73840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55234/0 (socket says 172.21.15.152:55234) 2024-09-11T22:55:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00ba59c640 1 -- 172.21.15.152:0/2551199347 learned_addr learned my addr 172.21.15.152:0/2551199347 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00bad9d640 1 --2- 172.21.15.152:0/2551199347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a6b680 0x558848a6f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00b3fff640 1 --2- 172.21.15.152:0/2551199347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a60dc0 0x558848a6ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00ba59c640 1 -- 172.21.15.152:0/2551199347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a6b680 msgr2=0x558848a6f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00ba59c640 1 --2- 172.21.15.152:0/2551199347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a6b680 0x558848a6f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00ba59c640 1 -- 172.21.15.152:0/2551199347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a60dc0 msgr2=0x558848a6ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:37.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00ba59c640 1 --2- 172.21.15.152:0/2551199347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a60dc0 0x558848a6ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00ba59c640 1 -- 172.21.15.152:0/2551199347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558848a6f740 con 0x558848a604a0 2024-09-11T22:55:37.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00ba59c640 1 --2- 172.21.15.152:0/2551199347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a604a0 0x558848a73840 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0039f0 tx=0x7f00ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3861d7e2602bd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:37.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:37.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00a1ffb640 1 -- 172.21.15.152:0/2551199347 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac011e30 con 0x558848a604a0 2024-09-11T22:55:37.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00a1ffb640 1 -- 172.21.15.152:0/2551199347 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f00ac005020 con 0x558848a604a0 2024-09-11T22:55:37.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.564+0000 7f00c2fb1740 1 -- 172.21.15.152:0/2551199347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00a40041d0 con 0x558848a604a0 2024-09-11T22:55:37.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.565+0000 7f00a1ffb640 1 -- 172.21.15.152:0/2551199347 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f00ac038050 con 0x558848a604a0 2024-09-11T22:55:37.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.565+0000 7f00c2fb1740 10 client.16308 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:37.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.565+0000 7f00c2fb1740 1 -- 172.21.15.152:0/2551199347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558848ae5450 con 0x558848a604a0 2024-09-11T22:55:37.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.565+0000 7f00a1ffb640 1 -- 172.21.15.152:0/2551199347 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a40041d0 con 0x558848a604a0 2024-09-11T22:55:37.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.565+0000 7f00a1ffb640 1 -- 172.21.15.152:0/2551199347 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f00ac0293b0 con 0x558848a604a0 2024-09-11T22:55:37.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.565+0000 7f00c2fb1740 10 client.16308 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:37.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.565+0000 7f00c2fb1740 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:37.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.565+0000 7f00c2fb1740 20 client.16308 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:37.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.565+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/2551199347 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558848ae5bd0 0x558848ae7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:37.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.565+0000 7f00c2fb1740 4 client.16308 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:55:37.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.565+0000 7f00c2fb1740 1 -- 172.21.15.152:0/2551199347 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5588489cdd60 con 0x558848ae5bd0 2024-09-11T22:55:37.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.565+0000 7f00b3fff640 1 --2- 172.21.15.152:0/2551199347 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558848ae5bd0 0x558848ae7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:37.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.566+0000 7f00a1ffb640 10 client.16308 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:37.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.566+0000 7f00b3fff640 1 --2- 172.21.15.152:0/2551199347 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558848ae5bd0 0x558848ae7ff0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7147e5ad0951a773 server_cookie=3c1d63c6ea8e6d7c in_seq=0 out_seq=0 2024-09-11T22:55:37.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.566+0000 7f00a1ffb640 1 -- 172.21.15.152:0/2551199347 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5588489cdd60 con 0x558848ae5bd0 2024-09-11T22:55:37.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.566+0000 7f00a1ffb640 10 client.16308 handle_command_reply: tid=0 2024-09-11T22:55:37.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.566+0000 7f00c2fb1740 1 client.16308 shutdown 2024-09-11T22:55:37.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.566+0000 7f00c2fb1740 1 -- 172.21.15.152:0/2551199347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a604a0 msgr2=0x558848a73840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:37.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.566+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/2551199347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a604a0 0x558848a73840 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0039f0 tx=0x7f00ac004730 comp rx=0 tx=0).stop 2024-09-11T22:55:37.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.566+0000 7f00c2fb1740 1 -- 172.21.15.152:0/2551199347 shutdown_connections 2024-09-11T22:55:37.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.566+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/2551199347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558848a6b680 0x558848a6f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.566+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/2551199347 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558848ae5bd0 0x558848ae7ff0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.566+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/2551199347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558848a60dc0 0x558848a6ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.566+0000 7f00c2fb1740 1 --2- 172.21.15.152:0/2551199347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558848a604a0 0x558848a73840 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.566+0000 7f00a1ffb640 0 client.16308 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:37.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.566+0000 7f00c2fb1740 1 -- 172.21.15.152:0/2551199347 >> 172.21.15.152:0/2551199347 conn(0x558848a5ca30 msgr2=0x558848a6a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:37.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.566+0000 7f00c2fb1740 1 -- 172.21.15.152:0/2551199347 shutdown_connections 2024-09-11T22:55:37.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.566+0000 7f00c2fb1740 1 -- 172.21.15.152:0/2551199347 wait complete. 2024-09-11T22:55:37.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.567+0000 7f00c2fb1740 20 client.16308 trim_cache size 0 max 16384 2024-09-11T22:55:37.573 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:55:37.573 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:55:37.573 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": { 2024-09-11T22:55:37.573 INFO:teuthology.orchestra.run.smithi152.stdout: "06fa1d57-e5b9-4c24-b7c1-1603eaf89e55": { 2024-09-11T22:55:37.573 INFO:teuthology.orchestra.run.smithi152.stdout: "path": "/", 2024-09-11T22:55:37.573 INFO:teuthology.orchestra.run.smithi152.stdout: "tag": "06fa1d57-e5b9-4c24-b7c1-1603eaf89e55", 2024-09-11T22:55:37.573 INFO:teuthology.orchestra.run.smithi152.stdout: "options": "recursive,force" 2024-09-11T22:55:37.573 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:37.573 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:37.573 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:55:37.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.567+0000 7f00c0bd8640 1 -- 172.21.15.152:0/4116994457 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f00a80a7100 msgr2=0x7f00a80a95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:37.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.567+0000 7f00c0bd8640 1 --2- 172.21.15.152:0/4116994457 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f00a80a7100 0x7f00a80a95c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f00ac005fd0 tx=0x7f00ac00b390 comp rx=0 tx=0).stop 2024-09-11T22:55:37.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.567+0000 7f00c0bd8640 1 -- 172.21.15.152:0/4116994457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00bc102ef0 msgr2=0x7f00bc199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:37.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.567+0000 7f00c0bd8640 1 --2- 172.21.15.152:0/4116994457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00bc102ef0 0x7f00bc199e90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f00b402ae70 tx=0x7f00b402f5d0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.567+0000 7f00c0bd8640 1 -- 172.21.15.152:0/4116994457 shutdown_connections 2024-09-11T22:55:37.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.567+0000 7f00c0bd8640 1 --2- 172.21.15.152:0/4116994457 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f00a80a7100 0x7f00a80a95c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.567+0000 7f00c0bd8640 1 --2- 172.21.15.152:0/4116994457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f00bc102ef0 0x7f00bc199e90 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.567+0000 7f00c0bd8640 1 --2- 172.21.15.152:0/4116994457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00bc102570 0x7f00bc199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.567+0000 7f00c0bd8640 1 --2- 172.21.15.152:0/4116994457 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00bc100e20 0x7f00bc1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:37.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.567+0000 7f00c0bd8640 1 -- 172.21.15.152:0/4116994457 >> 172.21.15.152:0/4116994457 conn(0x7f00bc0fc7d0 msgr2=0x7f00bc1068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:37.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.567+0000 7f00c0bd8640 1 -- 172.21.15.152:0/4116994457 shutdown_connections 2024-09-11T22:55:37.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:37.567+0000 7f00c0bd8640 1 -- 172.21.15.152:0/4116994457 wait complete. 2024-09-11T22:55:38.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:38 smithi191 ceph-mon[35211]: pgmap v85: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-09-11T22:55:38.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:38 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T22:55:38.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:38 smithi204 ceph-mon[35454]: pgmap v85: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-09-11T22:55:38.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:38 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T22:55:38.552 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24455 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0' 2024-09-11T22:55:38.553 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24455 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0 2024-09-11T22:55:38.553 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:38 smithi152 ceph-mon[31181]: pgmap v85: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-09-11T22:55:38.553 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:38 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T22:55:39.438 INFO:tasks.cephfs.filesystem:scrub status for tag:06fa1d57-e5b9-4c24-b7c1-1603eaf89e55 - {'path': '/', 'tag': '06fa1d57-e5b9-4c24-b7c1-1603eaf89e55', 'options': 'recursive,force'} 2024-09-11T22:55:39.551 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:55:40.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:40 smithi191 ceph-mon[35211]: pgmap v86: 97 pgs: 97 active+clean; 15 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.8 KiB/s wr, 124 op/s 2024-09-11T22:55:40.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:40 smithi204 ceph-mon[35454]: pgmap v86: 97 pgs: 97 active+clean; 15 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.8 KiB/s wr, 124 op/s 2024-09-11T22:55:40.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:40 smithi152 ceph-mon[31181]: pgmap v86: 97 pgs: 97 active+clean; 15 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.8 KiB/s wr, 124 op/s 2024-09-11T22:55:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.873+0000 7f883cf1f640 1 -- 172.21.15.152:0/682596853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8838101250 msgr2=0x7f8838107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.873+0000 7f883cf1f640 1 --2- 172.21.15.152:0/682596853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8838101250 0x7f8838107990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f882c009900 tx=0x7f882c02a410 comp rx=0 tx=0).stop 2024-09-11T22:55:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.873+0000 7f883cf1f640 1 -- 172.21.15.152:0/682596853 shutdown_connections 2024-09-11T22:55:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.873+0000 7f883cf1f640 1 --2- 172.21.15.152:0/682596853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88381080f0 0x7f883810ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.873+0000 7f883cf1f640 1 --2- 172.21.15.152:0/682596853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8838101250 0x7f8838107990 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.873+0000 7f883cf1f640 1 --2- 172.21.15.152:0/682596853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88381008a0 0x7f8838100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.873+0000 7f883cf1f640 1 -- 172.21.15.152:0/682596853 >> 172.21.15.152:0/682596853 conn(0x7f8838075ef0 msgr2=0x7f8838076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.873+0000 7f883cf1f640 1 -- 172.21.15.152:0/682596853 shutdown_connections 2024-09-11T22:55:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.873+0000 7f883cf1f640 1 -- 172.21.15.152:0/682596853 wait complete. 2024-09-11T22:55:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883cf1f640 1 Processor -- start 2024-09-11T22:55:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883cf1f640 1 -- start start 2024-09-11T22:55:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883cf1f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88381008a0 0x7f8838101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883cf1f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88381080f0 0x7f8838101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883cf1f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8838102290 0x7f88381a6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883cf1f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8838111f70 con 0x7f88381008a0 2024-09-11T22:55:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883cf1f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8838111df0 con 0x7f88381080f0 2024-09-11T22:55:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883cf1f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88381120f0 con 0x7f8838102290 2024-09-11T22:55:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88381008a0 0x7f8838101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88381008a0 0x7f8838101810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52982/0 (socket says 172.21.15.152:52982) 2024-09-11T22:55:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883659c640 1 -- 172.21.15.152:0/1040085416 learned_addr learned my addr 172.21.15.152:0/1040085416 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f8835d9b640 1 --2- 172.21.15.152:0/1040085416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88381080f0 0x7f8838101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f8836d9d640 1 --2- 172.21.15.152:0/1040085416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8838102290 0x7f88381a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883659c640 1 -- 172.21.15.152:0/1040085416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8838102290 msgr2=0x7f88381a6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:41.877 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883659c640 1 --2- 172.21.15.152:0/1040085416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8838102290 0x7f88381a6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883659c640 1 -- 172.21.15.152:0/1040085416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88381080f0 msgr2=0x7f8838101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883659c640 1 --2- 172.21.15.152:0/1040085416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88381080f0 0x7f8838101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883659c640 1 -- 172.21.15.152:0/1040085416 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88381a6b50 con 0x7f88381008a0 2024-09-11T22:55:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f8836d9d640 1 --2- 172.21.15.152:0/1040085416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8838102290 0x7f88381a6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.874+0000 7f883659c640 1 --2- 172.21.15.152:0/1040085416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88381008a0 0x7f8838101810 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f882800c890 tx=0x7f882800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.875+0000 7f88277fe640 1 -- 172.21.15.152:0/1040085416 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828004220 con 0x7f88381008a0 2024-09-11T22:55:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.875+0000 7f883cf1f640 1 -- 172.21.15.152:0/1040085416 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88381a6e40 con 0x7f88381008a0 2024-09-11T22:55:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.875+0000 7f88277fe640 1 -- 172.21.15.152:0/1040085416 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88280043c0 con 0x7f88381008a0 2024-09-11T22:55:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.875+0000 7f88277fe640 1 -- 172.21.15.152:0/1040085416 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828004a00 con 0x7f88381008a0 2024-09-11T22:55:41.878 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.875+0000 7f883cf1f640 1 -- 172.21.15.152:0/1040085416 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88381a7300 con 0x7f88381008a0 2024-09-11T22:55:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.877+0000 7f88277fe640 1 -- 172.21.15.152:0/1040085416 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f88280079b0 con 0x7f88381008a0 2024-09-11T22:55:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.877+0000 7f883fcb5740 1 Processor -- start 2024-09-11T22:55:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.878+0000 7f883fcb5740 1 -- start start 2024-09-11T22:55:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.878+0000 7f883fcb5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 0x559645454ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.878+0000 7f883fcb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645455410 0x55964545f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.878+0000 7f883fcb5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x5596454620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.878+0000 7f883fcb5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5596453b92f0 con 0x55964545fcb0 2024-09-11T22:55:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.878+0000 7f883fcb5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559645361270 con 0x559645454af0 2024-09-11T22:55:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.878+0000 7f883fcb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5596453b52b0 con 0x559645455410 2024-09-11T22:55:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.878+0000 7f8835d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x5596454620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.878+0000 7f8835d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x5596454620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52994/0 (socket says 172.21.15.152:52994) 2024-09-11T22:55:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.878+0000 7f8835d9b640 1 -- 172.21.15.152:0/4286201621 learned_addr learned my addr 172.21.15.152:0/4286201621 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.878+0000 7f883659c640 1 --2- 172.21.15.152:0/4286201621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645455410 0x55964545f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.878+0000 7f8836d9d640 1 --2- 172.21.15.152:0/4286201621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 0x559645454ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.878+0000 7f8835d9b640 1 -- 172.21.15.152:0/4286201621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645455410 msgr2=0x55964545f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.879+0000 7f8835d9b640 1 --2- 172.21.15.152:0/4286201621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645455410 0x55964545f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.879+0000 7f8835d9b640 1 -- 172.21.15.152:0/4286201621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 msgr2=0x559645454ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.879+0000 7f8835d9b640 1 --2- 172.21.15.152:0/4286201621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 0x559645454ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.879+0000 7f8835d9b640 1 -- 172.21.15.152:0/4286201621 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559645360340 con 0x55964545fcb0 2024-09-11T22:55:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.879+0000 7f8836d9d640 1 --2- 172.21.15.152:0/4286201621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 0x559645454ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.879+0000 7f88277fe640 1 --2- 172.21.15.152:0/1040085416 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f88200a2dd0 0x7f88200a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.879+0000 7f8836d9d640 1 --2- 172.21.15.152:0/1040085416 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f88200a2dd0 0x7f88200a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.879+0000 7f8835d9b640 1 --2- 172.21.15.152:0/4286201621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x5596454620a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f882c009900 tx=0x7f882c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86219e9a79e33d4a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.879+0000 7f88277fe640 1 -- 172.21.15.152:0/1040085416 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f88280cf460 con 0x7f88381008a0 2024-09-11T22:55:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.879+0000 7f8824ff9640 1 -- 172.21.15.152:0/4286201621 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c040070 con 0x55964545fcb0 2024-09-11T22:55:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.879+0000 7f8824ff9640 1 -- 172.21.15.152:0/4286201621 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f882c004490 con 0x55964545fcb0 2024-09-11T22:55:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.879+0000 7f8824ff9640 1 -- 172.21.15.152:0/4286201621 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c035040 con 0x55964545fcb0 2024-09-11T22:55:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.880+0000 7f8836d9d640 1 --2- 172.21.15.152:0/1040085416 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f88200a2dd0 0x7f88200a5290 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8830007f30 tx=0x7f8830007910 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.880+0000 7f883fcb5740 1 -- 172.21.15.152:0/4286201621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 msgr2=0x5596454620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.880+0000 7f883fcb5740 1 --2- 172.21.15.152:0/4286201621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x5596454620a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f882c009900 tx=0x7f882c005c00 comp rx=0 tx=0).stop 2024-09-11T22:55:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.880+0000 7f883fcb5740 1 -- 172.21.15.152:0/4286201621 shutdown_connections 2024-09-11T22:55:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.880+0000 7f883fcb5740 1 --2- 172.21.15.152:0/4286201621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x5596454620a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.880+0000 7f883fcb5740 1 --2- 172.21.15.152:0/4286201621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645455410 0x55964545f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.880+0000 7f883fcb5740 1 --2- 172.21.15.152:0/4286201621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 0x559645454ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.880+0000 7f883fcb5740 1 -- 172.21.15.152:0/4286201621 >> 172.21.15.152:0/4286201621 conn(0x559645451060 msgr2=0x559645451440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.880+0000 7f883fcb5740 1 -- 172.21.15.152:0/4286201621 shutdown_connections 2024-09-11T22:55:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.880+0000 7f883fcb5740 1 -- 172.21.15.152:0/4286201621 wait complete. 2024-09-11T22:55:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.880+0000 7f883fcb5740 1 -- start start 2024-09-11T22:55:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.880+0000 7f883fcb5740 1 Processor -- start 2024-09-11T22:55:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.880+0000 7f883fcb5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 0x55964545caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f883fcb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645455410 0x55964545d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f883fcb5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x55964545d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f883fcb5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x559645462880 con 0x55964545fcb0 2024-09-11T22:55:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f883fcb5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559645342290 con 0x559645454af0 2024-09-11T22:55:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f883fcb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559645462a00 con 0x559645455410 2024-09-11T22:55:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f883fcb5740 10 client.0 _ll_register_callbacks cb 0x559645373140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f8836d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x55964545d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f883659c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 0x55964545caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f883659c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 0x55964545caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:52992/0 (socket says 172.21.15.152:52992) 2024-09-11T22:55:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f883659c640 1 -- 172.21.15.152:0/1683060510 learned_addr learned my addr 172.21.15.152:0/1683060510 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f8836d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x55964545d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53010/0 (socket says 172.21.15.152:53010) 2024-09-11T22:55:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f8835d9b640 1 --2- 172.21.15.152:0/1683060510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645455410 0x55964545d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f883659c640 1 -- 172.21.15.152:0/1683060510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645455410 msgr2=0x55964545d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f883659c640 1 --2- 172.21.15.152:0/1683060510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645455410 0x55964545d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f883659c640 1 -- 172.21.15.152:0/1683060510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 msgr2=0x55964545d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f883659c640 1 --2- 172.21.15.152:0/1683060510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x55964545d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f883659c640 1 -- 172.21.15.152:0/1683060510 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596450cc210 con 0x559645454af0 2024-09-11T22:55:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f8836d9d640 1 --2- 172.21.15.152:0/1683060510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x55964545d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:55:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f883659c640 1 --2- 172.21.15.152:0/1683060510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 0x55964545caf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8828005fd0 tx=0x7f88280054a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b02b5c591cef5b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f881dffb640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-11T22:55:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f881dffb640 1 -- 172.21.15.152:0/1683060510 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882801d040 con 0x559645454af0 2024-09-11T22:55:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f883fcb5740 1 -- 172.21.15.152:0/1683060510 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f882c004490 con 0x559645454af0 2024-09-11T22:55:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f881dffb640 1 -- 172.21.15.152:0/1683060510 <== mon.2 v2:172.21.15.191:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f88280d2c00 con 0x559645454af0 2024-09-11T22:55:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.881+0000 7f881dffb640 1 -- 172.21.15.152:0/1683060510 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f882809b4d0 con 0x559645454af0 2024-09-11T22:55:41.889 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.882+0000 7f883fcb5740 10 client.25580 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.882+0000 7f883fcb5740 1 -- 172.21.15.152:0/1683060510 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559645198620 con 0x559645454af0 2024-09-11T22:55:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.882+0000 7f881dffb640 1 -- 172.21.15.152:0/1683060510 <== mon.2 v2:172.21.15.191:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f882c004490 con 0x559645454af0 2024-09-11T22:55:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.882+0000 7f881dffb640 1 -- 172.21.15.152:0/1683060510 <== mon.2 v2:172.21.15.191:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f88280a4170 con 0x559645454af0 2024-09-11T22:55:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.882+0000 7f883fcb5740 10 client.25580 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.882+0000 7f883fcb5740 10 client.25580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.882+0000 7f883fcb5740 20 client.25580 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.882+0000 7f883fcb5740 1 --2- 172.21.15.152:0/1683060510 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55964546fc40 0x559645472030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.882+0000 7f883fcb5740 4 client.25580 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:55:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.882+0000 7f883fcb5740 1 -- 172.21.15.152:0/1683060510 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596453c6a20 con 0x55964546fc40 2024-09-11T22:55:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.882+0000 7f8835d9b640 1 --2- 172.21.15.152:0/1683060510 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55964546fc40 0x559645472030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.883+0000 7f881dffb640 10 client.25580 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.883+0000 7f8835d9b640 1 --2- 172.21.15.152:0/1683060510 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55964546fc40 0x559645472030 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d15d4c0fc2e793df server_cookie=30b0ca5f0aa9d1b4 in_seq=0 out_seq=0 2024-09-11T22:55:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.883+0000 7f881dffb640 1 -- 172.21.15.152:0/1683060510 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596453c6a20 con 0x55964546fc40 2024-09-11T22:55:41.890 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.883+0000 7f881dffb640 10 client.25580 handle_command_reply: tid=0 2024-09-11T22:55:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.883+0000 7f883fcb5740 1 client.25580 shutdown 2024-09-11T22:55:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.883+0000 7f883fcb5740 1 -- 172.21.15.152:0/1683060510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 msgr2=0x55964545caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.883+0000 7f883fcb5740 1 --2- 172.21.15.152:0/1683060510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 0x55964545caf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8828005fd0 tx=0x7f88280054a0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.883+0000 7f883fcb5740 1 -- 172.21.15.152:0/1683060510 shutdown_connections 2024-09-11T22:55:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.883+0000 7f883fcb5740 1 --2- 172.21.15.152:0/1683060510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x55964545d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.883+0000 7f883fcb5740 1 --2- 172.21.15.152:0/1683060510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645455410 0x55964545d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.883+0000 7f883fcb5740 1 --2- 172.21.15.152:0/1683060510 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55964546fc40 0x559645472030 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.883+0000 7f883fcb5740 1 --2- 172.21.15.152:0/1683060510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 0x55964545caf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.891 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.883+0000 7f881dffb640 0 client.25580 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.883+0000 7f883fcb5740 1 -- 172.21.15.152:0/1683060510 >> 172.21.15.152:0/1683060510 conn(0x559645451060 msgr2=0x559645456d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.884+0000 7f883fcb5740 1 -- 172.21.15.152:0/1683060510 shutdown_connections 2024-09-11T22:55:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.884+0000 7f883fcb5740 1 -- 172.21.15.152:0/1683060510 wait complete. 2024-09-11T22:55:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.884+0000 7f883fcb5740 20 client.25580 trim_cache size 0 max 16384 2024-09-11T22:55:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f883fcb5740 1 Processor -- start 2024-09-11T22:55:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f883fcb5740 1 -- start start 2024-09-11T22:55:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f883fcb5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 0x55964545d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f883fcb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645455410 0x55964545d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f883fcb5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x5596454778d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f883fcb5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f88381af870 con 0x55964545fcb0 2024-09-11T22:55:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f883fcb5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f88381b13b0 con 0x559645454af0 2024-09-11T22:55:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f883fcb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88381aff40 con 0x559645455410 2024-09-11T22:55:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f8836d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x5596454778d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f8836d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x5596454778d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53022/0 (socket says 172.21.15.152:53022) 2024-09-11T22:55:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f8836d9d640 1 -- 172.21.15.152:0/3113372408 learned_addr learned my addr 172.21.15.152:0/3113372408 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f883659c640 1 --2- 172.21.15.152:0/3113372408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 0x55964545d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f8835d9b640 1 --2- 172.21.15.152:0/3113372408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645455410 0x55964545d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f8836d9d640 1 -- 172.21.15.152:0/3113372408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645455410 msgr2=0x55964545d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f8836d9d640 1 --2- 172.21.15.152:0/3113372408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645455410 0x55964545d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f8836d9d640 1 -- 172.21.15.152:0/3113372408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 msgr2=0x55964545d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f8836d9d640 1 --2- 172.21.15.152:0/3113372408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 0x55964545d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f8836d9d640 1 -- 172.21.15.152:0/3113372408 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559645455240 con 0x55964545fcb0 2024-09-11T22:55:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f8835d9b640 1 --2- 172.21.15.152:0/3113372408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645455410 0x55964545d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:55:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.898+0000 7f8836d9d640 1 --2- 172.21.15.152:0/3113372408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x5596454778d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8830011b10 tx=0x7f8830013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c98bdcc4d479622 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.899+0000 7f881ffff640 1 -- 172.21.15.152:0/3113372408 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883001fe50 con 0x55964545fcb0 2024-09-11T22:55:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.899+0000 7f881ffff640 1 -- 172.21.15.152:0/3113372408 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88300278c0 con 0x55964545fcb0 2024-09-11T22:55:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.899+0000 7f881ffff640 1 -- 172.21.15.152:0/3113372408 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883001aa80 con 0x55964545fcb0 2024-09-11T22:55:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.899+0000 7f883fcb5740 1 -- 172.21.15.152:0/3113372408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 msgr2=0x5596454778d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.899+0000 7f883fcb5740 1 --2- 172.21.15.152:0/3113372408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x5596454778d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8830011b10 tx=0x7f8830013000 comp rx=0 tx=0).stop 2024-09-11T22:55:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.899+0000 7f883fcb5740 1 -- 172.21.15.152:0/3113372408 shutdown_connections 2024-09-11T22:55:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.899+0000 7f883fcb5740 1 --2- 172.21.15.152:0/3113372408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x5596454778d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.899+0000 7f883fcb5740 1 --2- 172.21.15.152:0/3113372408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645455410 0x55964545d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.899+0000 7f883fcb5740 1 --2- 172.21.15.152:0/3113372408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645454af0 0x55964545d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.899+0000 7f883fcb5740 1 -- 172.21.15.152:0/3113372408 >> 172.21.15.152:0/3113372408 conn(0x559645451060 msgr2=0x55964545eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.899+0000 7f883fcb5740 1 -- 172.21.15.152:0/3113372408 shutdown_connections 2024-09-11T22:55:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.899+0000 7f883fcb5740 1 -- 172.21.15.152:0/3113372408 wait complete. 2024-09-11T22:55:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f883fcb5740 1 -- start start 2024-09-11T22:55:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f883fcb5740 1 Processor -- start 2024-09-11T22:55:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f883fcb5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645454af0 0x559645468540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f883fcb5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645455410 0x5596454598c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f883fcb5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x559645459e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f883fcb5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f88381b0ce0 con 0x55964545fcb0 2024-09-11T22:55:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f883fcb5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f88381b0610 con 0x559645455410 2024-09-11T22:55:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f883fcb5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f88381ad830 con 0x559645454af0 2024-09-11T22:55:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f883fcb5740 10 client.0 _ll_register_callbacks cb 0x7f88280057c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f8836d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x559645459e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f8836d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x559645459e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53030/0 (socket says 172.21.15.152:53030) 2024-09-11T22:55:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f8836d9d640 1 -- 172.21.15.152:0/270339453 learned_addr learned my addr 172.21.15.152:0/270339453 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f883659c640 1 --2- 172.21.15.152:0/270339453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645454af0 0x559645468540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f8835d9b640 1 --2- 172.21.15.152:0/270339453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645455410 0x5596454598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f8836d9d640 1 -- 172.21.15.152:0/270339453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645454af0 msgr2=0x559645468540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f8836d9d640 1 --2- 172.21.15.152:0/270339453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645454af0 0x559645468540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f8836d9d640 1 -- 172.21.15.152:0/270339453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645455410 msgr2=0x5596454598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f8836d9d640 1 --2- 172.21.15.152:0/270339453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645455410 0x5596454598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.900+0000 7f8836d9d640 1 -- 172.21.15.152:0/270339453 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55964545a6b0 con 0x55964545fcb0 2024-09-11T22:55:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.901+0000 7f8836d9d640 1 --2- 172.21.15.152:0/270339453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x559645459e80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f883001a4e0 tx=0x7f88300117f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348f38887b54ea2b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.901+0000 7f881dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.901+0000 7f881dffb640 1 -- 172.21.15.152:0/270339453 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8830036020 con 0x55964545fcb0 2024-09-11T22:55:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.901+0000 7f883fcb5740 1 -- 172.21.15.152:0/270339453 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88300278c0 con 0x55964545fcb0 2024-09-11T22:55:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.901+0000 7f881dffb640 1 -- 172.21.15.152:0/270339453 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f88300275f0 con 0x55964545fcb0 2024-09-11T22:55:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.901+0000 7f881dffb640 1 -- 172.21.15.152:0/270339453 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8830041050 con 0x55964545fcb0 2024-09-11T22:55:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.901+0000 7f883fcb5740 10 client.16344 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.901+0000 7f883fcb5740 1 -- 172.21.15.152:0/270339453 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5596454da900 con 0x55964545fcb0 2024-09-11T22:55:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.901+0000 7f881dffb640 1 -- 172.21.15.152:0/270339453 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88300278c0 con 0x55964545fcb0 2024-09-11T22:55:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.901+0000 7f881dffb640 1 -- 172.21.15.152:0/270339453 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8830026070 con 0x55964545fcb0 2024-09-11T22:55:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.901+0000 7f883fcb5740 10 client.16344 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.901+0000 7f883fcb5740 10 client.16344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.901+0000 7f883fcb5740 20 client.16344 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.901+0000 7f883fcb5740 1 --2- 172.21.15.152:0/270339453 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5596454daf60 0x5596454dd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.901+0000 7f883fcb5740 4 client.16344 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:55:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.901+0000 7f883fcb5740 1 -- 172.21.15.152:0/270339453 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596453c0d70 con 0x5596454daf60 2024-09-11T22:55:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.902+0000 7f883659c640 1 --2- 172.21.15.152:0/270339453 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5596454daf60 0x5596454dd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.902+0000 7f883659c640 1 --2- 172.21.15.152:0/270339453 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5596454daf60 0x5596454dd380 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=134e6736b37b505b server_cookie=5d3e81a6f06b5584 in_seq=0 out_seq=0 2024-09-11T22:55:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.902+0000 7f881dffb640 10 client.16344 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.913+0000 7f881dffb640 1 -- 172.21.15.152:0/270339453 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5596453c0d70 con 0x5596454daf60 2024-09-11T22:55:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.913+0000 7f881dffb640 10 client.16344 handle_command_reply: tid=0 2024-09-11T22:55:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.913+0000 7f883fcb5740 1 client.16344 shutdown 2024-09-11T22:55:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.913+0000 7f883fcb5740 1 -- 172.21.15.152:0/270339453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 msgr2=0x559645459e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.913+0000 7f883fcb5740 1 --2- 172.21.15.152:0/270339453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x559645459e80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f883001a4e0 tx=0x7f88300117f0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.913+0000 7f883fcb5740 1 -- 172.21.15.152:0/270339453 shutdown_connections 2024-09-11T22:55:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.913+0000 7f883fcb5740 1 --2- 172.21.15.152:0/270339453 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55964545fcb0 0x559645459e80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.913+0000 7f883fcb5740 1 --2- 172.21.15.152:0/270339453 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5596454daf60 0x5596454dd380 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.913+0000 7f883fcb5740 1 --2- 172.21.15.152:0/270339453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559645455410 0x5596454598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.913+0000 7f883fcb5740 1 --2- 172.21.15.152:0/270339453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559645454af0 0x559645468540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.913+0000 7f881dffb640 0 client.16344 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.913+0000 7f883fcb5740 1 -- 172.21.15.152:0/270339453 >> 172.21.15.152:0/270339453 conn(0x559645451060 msgr2=0x55964544f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.913+0000 7f883fcb5740 1 -- 172.21.15.152:0/270339453 shutdown_connections 2024-09-11T22:55:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.913+0000 7f883fcb5740 1 -- 172.21.15.152:0/270339453 wait complete. 2024-09-11T22:55:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.913+0000 7f883fcb5740 20 client.16344 trim_cache size 0 max 16384 2024-09-11T22:55:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:{ 2024-09-11T22:55:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "message": "", 2024-09-11T22:55:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "return_code": 0 2024-09-11T22:55:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:} 2024-09-11T22:55:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.914+0000 7f883cf1f640 1 -- 172.21.15.152:0/1040085416 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f88200a2dd0 msgr2=0x7f88200a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.914+0000 7f883cf1f640 1 --2- 172.21.15.152:0/1040085416 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f88200a2dd0 0x7f88200a5290 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8830007f30 tx=0x7f8830007910 comp rx=0 tx=0).stop 2024-09-11T22:55:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.914+0000 7f883cf1f640 1 -- 172.21.15.152:0/1040085416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88381008a0 msgr2=0x7f8838101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.914+0000 7f883cf1f640 1 --2- 172.21.15.152:0/1040085416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88381008a0 0x7f8838101810 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f882800c890 tx=0x7f882800cd50 comp rx=0 tx=0).stop 2024-09-11T22:55:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.914+0000 7f883cf1f640 1 -- 172.21.15.152:0/1040085416 shutdown_connections 2024-09-11T22:55:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.914+0000 7f883cf1f640 1 --2- 172.21.15.152:0/1040085416 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f88200a2dd0 0x7f88200a5290 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.914+0000 7f883cf1f640 1 --2- 172.21.15.152:0/1040085416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8838102290 0x7f88381a6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.914+0000 7f883cf1f640 1 --2- 172.21.15.152:0/1040085416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f88381080f0 0x7f8838101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.914+0000 7f883cf1f640 1 --2- 172.21.15.152:0/1040085416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f88381008a0 0x7f8838101810 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.914+0000 7f883cf1f640 1 -- 172.21.15.152:0/1040085416 >> 172.21.15.152:0/1040085416 conn(0x7f8838075ef0 msgr2=0x7f88380ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.914+0000 7f883cf1f640 1 -- 172.21.15.152:0/1040085416 shutdown_connections 2024-09-11T22:55:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:41.914+0000 7f883cf1f640 1 -- 172.21.15.152:0/1040085416 wait complete. 2024-09-11T22:55:42.255 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:55:42.255 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:55:42.255 INFO:tasks.workunit.client.0.smithi152.stdout: 2024-09-11T22:55:42.255 INFO:tasks.workunit.client.0.smithi152.stdout:iozone test complete. 2024-09-11T22:55:42.257 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T22:55:42.257 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T22:55:42.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.374+0000 7fc953895640 1 -- 172.21.15.152:0/1952113017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc94c102790 msgr2=0x7fc94c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:42.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.374+0000 7fc953895640 1 --2- 172.21.15.152:0/1952113017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc94c102790 0x7fc94c1067e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009900 tx=0x7fc93c02a410 comp rx=0 tx=0).stop 2024-09-11T22:55:42.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.375+0000 7fc953895640 1 -- 172.21.15.152:0/1952113017 shutdown_connections 2024-09-11T22:55:42.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.375+0000 7fc953895640 1 --2- 172.21.15.152:0/1952113017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc94c106f10 0x7fc94c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.375+0000 7fc953895640 1 --2- 172.21.15.152:0/1952113017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc94c102790 0x7fc94c1067e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.375+0000 7fc953895640 1 --2- 172.21.15.152:0/1952113017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc94c101de0 0x7fc94c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.375+0000 7fc953895640 1 -- 172.21.15.152:0/1952113017 >> 172.21.15.152:0/1952113017 conn(0x7fc94c0fd8c0 msgr2=0x7fc94c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.375+0000 7fc953895640 1 -- 172.21.15.152:0/1952113017 shutdown_connections 2024-09-11T22:55:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.375+0000 7fc953895640 1 -- 172.21.15.152:0/1952113017 wait complete. 2024-09-11T22:55:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.375+0000 7fc953895640 1 Processor -- start 2024-09-11T22:55:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.375+0000 7fc953895640 1 -- start start 2024-09-11T22:55:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc953895640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc94c101de0 0x7fc94c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc953895640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc94c102790 0x7fc94c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc953895640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc94c106f10 0x7fc94c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc953895640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c111f70 con 0x7fc94c102790 2024-09-11T22:55:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc953895640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c111df0 con 0x7fc94c101de0 2024-09-11T22:55:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc953895640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1120f0 con 0x7fc94c106f10 2024-09-11T22:55:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc950e30640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc94c102790 0x7fc94c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc950e30640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc94c102790 0x7fc94c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52916/0 (socket says 172.21.15.152:52916) 2024-09-11T22:55:42.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc950e30640 1 -- 172.21.15.152:0/1798138630 learned_addr learned my addr 172.21.15.152:0/1798138630 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc951631640 1 --2- 172.21.15.152:0/1798138630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc94c101de0 0x7fc94c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc951e32640 1 --2- 172.21.15.152:0/1798138630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc94c106f10 0x7fc94c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc950e30640 1 -- 172.21.15.152:0/1798138630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc94c106f10 msgr2=0x7fc94c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc950e30640 1 --2- 172.21.15.152:0/1798138630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc94c106f10 0x7fc94c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc950e30640 1 -- 172.21.15.152:0/1798138630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc94c101de0 msgr2=0x7fc94c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc950e30640 1 --2- 172.21.15.152:0/1798138630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc94c101de0 0x7fc94c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc950e30640 1 -- 172.21.15.152:0/1798138630 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94c19d3d0 con 0x7fc94c102790 2024-09-11T22:55:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.376+0000 7fc950e30640 1 --2- 172.21.15.152:0/1798138630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc94c102790 0x7fc94c198960 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc93c006010 tx=0x7fc93c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.377+0000 7fc9427fc640 1 -- 172.21.15.152:0/1798138630 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c03f030 con 0x7fc94c102790 2024-09-11T22:55:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.377+0000 7fc953895640 1 -- 172.21.15.152:0/1798138630 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94c19d660 con 0x7fc94c102790 2024-09-11T22:55:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.377+0000 7fc9427fc640 1 -- 172.21.15.152:0/1798138630 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc93c003ce0 con 0x7fc94c102790 2024-09-11T22:55:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.377+0000 7fc9427fc640 1 -- 172.21.15.152:0/1798138630 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c035070 con 0x7fc94c102790 2024-09-11T22:55:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.377+0000 7fc953895640 1 -- 172.21.15.152:0/1798138630 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc94c1a4fa0 con 0x7fc94c102790 2024-09-11T22:55:42.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.379+0000 7fc9427fc640 1 -- 172.21.15.152:0/1798138630 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc93c042030 con 0x7fc94c102790 2024-09-11T22:55:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.380+0000 7fc955c6e740 1 Processor -- start 2024-09-11T22:55:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.380+0000 7fc955c6e740 1 -- start start 2024-09-11T22:55:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.380+0000 7fc955c6e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0c9760 0x55fa0d0c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.380+0000 7fc955c6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.380+0000 7fc955c6e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0d4940 0x55fa0d0d6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.380+0000 7fc955c6e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55fa0d02f2a0 con 0x55fa0d0c9760 2024-09-11T22:55:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.380+0000 7fc955c6e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55fa0cfd6b10 con 0x55fa0d0d4940 2024-09-11T22:55:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.380+0000 7fc955c6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fa0d02b260 con 0x55fa0d0ca080 2024-09-11T22:55:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.380+0000 7fc9427fc640 1 --2- 172.21.15.152:0/1798138630 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9380a2cf0 0x7fc9380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.380+0000 7fc951e32640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0c9760 0x55fa0d0c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.380+0000 7fc951631640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.380+0000 7fc951e32640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0c9760 0x55fa0d0c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52926/0 (socket says 172.21.15.152:52926) 2024-09-11T22:55:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.380+0000 7fc951e32640 1 -- 172.21.15.152:0/3010942418 learned_addr learned my addr 172.21.15.152:0/3010942418 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:42.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.380+0000 7fc9427fc640 1 -- 172.21.15.152:0/1798138630 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc93c0efa10 con 0x7fc94c102790 2024-09-11T22:55:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.380+0000 7fc950e30640 1 --2- 172.21.15.152:0/3010942418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0d4940 0x55fa0d0d6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.380+0000 7fc951e32640 1 --2- 172.21.15.152:0/1798138630 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9380a2cf0 0x7fc9380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.381+0000 7fc951631640 1 -- 172.21.15.152:0/3010942418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0d4940 msgr2=0x55fa0d0d6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.381+0000 7fc951631640 1 --2- 172.21.15.152:0/3010942418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0d4940 0x55fa0d0d6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.381+0000 7fc951631640 1 -- 172.21.15.152:0/3010942418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0c9760 msgr2=0x55fa0d0c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.381+0000 7fc951631640 1 --2- 172.21.15.152:0/3010942418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0c9760 0x55fa0d0c9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.381+0000 7fc951631640 1 -- 172.21.15.152:0/3010942418 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa0cda0b90 con 0x55fa0d0ca080 2024-09-11T22:55:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.381+0000 7fc950e30640 1 --2- 172.21.15.152:0/3010942418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0d4940 0x55fa0d0d6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:55:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.381+0000 7fc951e32640 1 --2- 172.21.15.152:0/1798138630 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9380a2cf0 0x7fc9380a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc9440098b0 tx=0x7fc944009600 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.381+0000 7fc951631640 1 --2- 172.21.15.152:0/3010942418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d4400 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc948005fd0 tx=0x7fc9480107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=496edd43f5471394 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.381+0000 7fc9377fe640 1 -- 172.21.15.152:0/3010942418 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94801c480 con 0x55fa0d0ca080 2024-09-11T22:55:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.381+0000 7fc9377fe640 1 -- 172.21.15.152:0/3010942418 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc94801cb00 con 0x55fa0d0ca080 2024-09-11T22:55:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.381+0000 7fc9377fe640 1 -- 172.21.15.152:0/3010942418 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948011b80 con 0x55fa0d0ca080 2024-09-11T22:55:42.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.382+0000 7fc955c6e740 1 -- 172.21.15.152:0/3010942418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0ca080 msgr2=0x55fa0d0d4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:42.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.382+0000 7fc955c6e740 1 --2- 172.21.15.152:0/3010942418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d4400 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc948005fd0 tx=0x7fc9480107a0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.382+0000 7fc955c6e740 1 -- 172.21.15.152:0/3010942418 shutdown_connections 2024-09-11T22:55:42.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.382+0000 7fc955c6e740 1 --2- 172.21.15.152:0/3010942418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0d4940 0x55fa0d0d6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.382+0000 7fc955c6e740 1 --2- 172.21.15.152:0/3010942418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d4400 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.382+0000 7fc955c6e740 1 --2- 172.21.15.152:0/3010942418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0c9760 0x55fa0d0c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.382+0000 7fc955c6e740 1 -- 172.21.15.152:0/3010942418 >> 172.21.15.152:0/3010942418 conn(0x55fa0d0c5cf0 msgr2=0x55fa0d0c60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:42.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.382+0000 7fc955c6e740 1 -- 172.21.15.152:0/3010942418 shutdown_connections 2024-09-11T22:55:42.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.382+0000 7fc955c6e740 1 -- 172.21.15.152:0/3010942418 wait complete. 2024-09-11T22:55:42.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.382+0000 7fc955c6e740 1 -- start start 2024-09-11T22:55:42.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.382+0000 7fc955c6e740 1 Processor -- start 2024-09-11T22:55:42.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc955c6e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0c9760 0x55fa0d0d1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc955c6e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc955c6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0d4940 0x55fa0d0d21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc955c6e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55fa0d0d7510 con 0x55fa0d0ca080 2024-09-11T22:55:42.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc955c6e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55fa0cfb76e0 con 0x55fa0d0c9760 2024-09-11T22:55:42.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc955c6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fa0d0d7690 con 0x55fa0d0d4940 2024-09-11T22:55:42.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc955c6e740 10 client.0 _ll_register_callbacks cb 0x55fa0cfdba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:42.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc950e30640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc950e30640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52938/0 (socket says 172.21.15.152:52938) 2024-09-11T22:55:42.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc950e30640 1 -- 172.21.15.152:0/3429029845 learned_addr learned my addr 172.21.15.152:0/3429029845 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:42.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc951631640 1 --2- 172.21.15.152:0/3429029845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0c9760 0x55fa0d0d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc951e32640 1 --2- 172.21.15.152:0/3429029845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0d4940 0x55fa0d0d21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc950e30640 1 -- 172.21.15.152:0/3429029845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0d4940 msgr2=0x55fa0d0d21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:42.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc950e30640 1 --2- 172.21.15.152:0/3429029845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0d4940 0x55fa0d0d21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc950e30640 1 -- 172.21.15.152:0/3429029845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0c9760 msgr2=0x55fa0d0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:42.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc950e30640 1 --2- 172.21.15.152:0/3429029845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0c9760 0x55fa0d0d1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc950e30640 1 -- 172.21.15.152:0/3429029845 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa0d0e0980 con 0x55fa0d0ca080 2024-09-11T22:55:42.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc950e30640 1 --2- 172.21.15.152:0/3429029845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d1cb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0b8ac0 tx=0x7fc93c0bba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5c728cb559c227 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:42.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.383+0000 7fc934ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:42.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.384+0000 7fc934ff9640 1 -- 172.21.15.152:0/3429029845 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c086060 con 0x55fa0d0ca080 2024-09-11T22:55:42.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.384+0000 7fc934ff9640 1 -- 172.21.15.152:0/3429029845 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc93c0c1b80 con 0x55fa0d0ca080 2024-09-11T22:55:42.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.384+0000 7fc955c6e740 1 -- 172.21.15.152:0/3429029845 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc94801cb00 con 0x55fa0d0ca080 2024-09-11T22:55:42.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.384+0000 7fc934ff9640 1 -- 172.21.15.152:0/3429029845 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc93c049030 con 0x55fa0d0ca080 2024-09-11T22:55:42.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.384+0000 7fc955c6e740 10 client.16320 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:42.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.384+0000 7fc955c6e740 1 -- 172.21.15.152:0/3429029845 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa0d0e32e0 con 0x55fa0d0ca080 2024-09-11T22:55:42.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.384+0000 7fc934ff9640 1 -- 172.21.15.152:0/3429029845 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc93c0d6020 con 0x55fa0d0ca080 2024-09-11T22:55:42.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.384+0000 7fc934ff9640 1 -- 172.21.15.152:0/3429029845 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc93c0bc350 con 0x55fa0d0ca080 2024-09-11T22:55:42.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.384+0000 7fc955c6e740 10 client.16320 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:42.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.384+0000 7fc955c6e740 10 client.16320 resolve_mds: validated gid 24455 aka daemon mds.b 2024-09-11T22:55:42.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.384+0000 7fc955c6e740 20 client.16320 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:42.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.384+0000 7fc955c6e740 1 --2- 172.21.15.152:0/3429029845 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55fa0d0e4740 0x55fa0d0e6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.384+0000 7fc955c6e740 4 client.16320 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:55:42.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.384+0000 7fc955c6e740 1 -- 172.21.15.152:0/3429029845 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa0d03c9d0 con 0x55fa0d0e4740 2024-09-11T22:55:42.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.385+0000 7fc951631640 1 --2- 172.21.15.152:0/3429029845 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55fa0d0e4740 0x55fa0d0e6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.385+0000 7fc934ff9640 10 client.16320 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:42.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.385+0000 7fc951631640 1 --2- 172.21.15.152:0/3429029845 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55fa0d0e4740 0x55fa0d0e6bd0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec2a9c821292c43 server_cookie=f558428915932466 in_seq=0 out_seq=0 2024-09-11T22:55:42.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.386+0000 7fc934ff9640 1 -- 172.21.15.152:0/3429029845 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa0d03c9d0 con 0x55fa0d0e4740 2024-09-11T22:55:42.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.386+0000 7fc934ff9640 10 client.16320 handle_command_reply: tid=0 2024-09-11T22:55:42.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.386+0000 7fc955c6e740 1 client.16320 shutdown 2024-09-11T22:55:42.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.386+0000 7fc955c6e740 1 -- 172.21.15.152:0/3429029845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0ca080 msgr2=0x55fa0d0d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:42.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.386+0000 7fc955c6e740 1 --2- 172.21.15.152:0/3429029845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d1cb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0b8ac0 tx=0x7fc93c0bba50 comp rx=0 tx=0).stop 2024-09-11T22:55:42.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.386+0000 7fc955c6e740 1 -- 172.21.15.152:0/3429029845 shutdown_connections 2024-09-11T22:55:42.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.386+0000 7fc955c6e740 1 --2- 172.21.15.152:0/3429029845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0d4940 0x55fa0d0d21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.386+0000 7fc955c6e740 1 --2- 172.21.15.152:0/3429029845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d1cb0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.386+0000 7fc955c6e740 1 --2- 172.21.15.152:0/3429029845 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55fa0d0e4740 0x55fa0d0e6bd0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.386+0000 7fc955c6e740 1 --2- 172.21.15.152:0/3429029845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0c9760 0x55fa0d0d1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.386+0000 7fc934ff9640 0 client.16320 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:42.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.386+0000 7fc955c6e740 1 -- 172.21.15.152:0/3429029845 >> 172.21.15.152:0/3429029845 conn(0x55fa0d0c5cf0 msgr2=0x55fa0d0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:42.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.386+0000 7fc955c6e740 1 -- 172.21.15.152:0/3429029845 shutdown_connections 2024-09-11T22:55:42.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.386+0000 7fc955c6e740 1 -- 172.21.15.152:0/3429029845 wait complete. 2024-09-11T22:55:42.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.386+0000 7fc955c6e740 20 client.16320 trim_cache size 0 max 16384 2024-09-11T22:55:42.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.403+0000 7fc955c6e740 1 Processor -- start 2024-09-11T22:55:42.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.403+0000 7fc955c6e740 1 -- start start 2024-09-11T22:55:42.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.403+0000 7fc955c6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0c9760 0x55fa0d0d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.403+0000 7fc955c6e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0ca080 0x55fa0d0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.403+0000 7fc955c6e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0d4940 0x55fa0d0ec4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.403+0000 7fc955c6e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1ad7b0 con 0x55fa0d0d4940 2024-09-11T22:55:42.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.403+0000 7fc955c6e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1af2f0 con 0x55fa0d0ca080 2024-09-11T22:55:42.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.403+0000 7fc955c6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1ade80 con 0x55fa0d0c9760 2024-09-11T22:55:42.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.404+0000 7fc951e32640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0d4940 0x55fa0d0ec4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.404+0000 7fc951e32640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0d4940 0x55fa0d0ec4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52948/0 (socket says 172.21.15.152:52948) 2024-09-11T22:55:42.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.404+0000 7fc951e32640 1 -- 172.21.15.152:0/577963735 learned_addr learned my addr 172.21.15.152:0/577963735 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:42.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.404+0000 7fc951631640 1 --2- 172.21.15.152:0/577963735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0c9760 0x55fa0d0d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.404+0000 7fc950e30640 1 --2- 172.21.15.152:0/577963735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0ca080 0x55fa0d0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.404+0000 7fc951e32640 1 -- 172.21.15.152:0/577963735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0c9760 msgr2=0x55fa0d0d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:42.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.404+0000 7fc951e32640 1 --2- 172.21.15.152:0/577963735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0c9760 0x55fa0d0d06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.404+0000 7fc951e32640 1 -- 172.21.15.152:0/577963735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0ca080 msgr2=0x55fa0d0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:42.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.404+0000 7fc951e32640 1 --2- 172.21.15.152:0/577963735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0ca080 0x55fa0d0cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.404+0000 7fc951e32640 1 -- 172.21.15.152:0/577963735 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa0d0c5b20 con 0x55fa0d0d4940 2024-09-11T22:55:42.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.404+0000 7fc951631640 1 --2- 172.21.15.152:0/577963735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0c9760 0x55fa0d0d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:55:42.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.404+0000 7fc951e32640 1 --2- 172.21.15.152:0/577963735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0d4940 0x55fa0d0ec4e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc944009e20 tx=0x7fc94400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81339229bcb5cc91 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:42.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.404+0000 7fc936ffd640 1 -- 172.21.15.152:0/577963735 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944012d70 con 0x55fa0d0d4940 2024-09-11T22:55:42.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.404+0000 7fc936ffd640 1 -- 172.21.15.152:0/577963735 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc94401e970 con 0x55fa0d0d4940 2024-09-11T22:55:42.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.404+0000 7fc936ffd640 1 -- 172.21.15.152:0/577963735 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9440051a0 con 0x55fa0d0d4940 2024-09-11T22:55:42.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.405+0000 7fc955c6e740 1 -- 172.21.15.152:0/577963735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0d4940 msgr2=0x55fa0d0ec4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:42.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.405+0000 7fc955c6e740 1 --2- 172.21.15.152:0/577963735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0d4940 0x55fa0d0ec4e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc944009e20 tx=0x7fc94400c000 comp rx=0 tx=0).stop 2024-09-11T22:55:42.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.405+0000 7fc955c6e740 1 -- 172.21.15.152:0/577963735 shutdown_connections 2024-09-11T22:55:42.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.405+0000 7fc955c6e740 1 --2- 172.21.15.152:0/577963735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0d4940 0x55fa0d0ec4e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.405+0000 7fc955c6e740 1 --2- 172.21.15.152:0/577963735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0ca080 0x55fa0d0cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.405+0000 7fc955c6e740 1 --2- 172.21.15.152:0/577963735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0c9760 0x55fa0d0d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.405+0000 7fc955c6e740 1 -- 172.21.15.152:0/577963735 >> 172.21.15.152:0/577963735 conn(0x55fa0d0c5cf0 msgr2=0x55fa0d0d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:42.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.405+0000 7fc955c6e740 1 -- 172.21.15.152:0/577963735 shutdown_connections 2024-09-11T22:55:42.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.405+0000 7fc955c6e740 1 -- 172.21.15.152:0/577963735 wait complete. 2024-09-11T22:55:42.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.405+0000 7fc955c6e740 1 -- start start 2024-09-11T22:55:42.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.405+0000 7fc955c6e740 1 Processor -- start 2024-09-11T22:55:42.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc955c6e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0c9760 0x55fa0d0d91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc955c6e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc955c6e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0d4940 0x55fa0d147470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc955c6e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1aec20 con 0x55fa0d0ca080 2024-09-11T22:55:42.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc955c6e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1ae550 con 0x55fa0d0d4940 2024-09-11T22:55:42.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc955c6e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1ab770 con 0x55fa0d0c9760 2024-09-11T22:55:42.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc955c6e740 10 client.0 _ll_register_callbacks cb 0x7fc93c0bbc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:42.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc950e30640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc950e30640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52958/0 (socket says 172.21.15.152:52958) 2024-09-11T22:55:42.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc950e30640 1 -- 172.21.15.152:0/1482857131 learned_addr learned my addr 172.21.15.152:0/1482857131 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:42.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc951e32640 1 --2- 172.21.15.152:0/1482857131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0d4940 0x55fa0d147470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc951631640 1 --2- 172.21.15.152:0/1482857131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0c9760 0x55fa0d0d91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc950e30640 1 -- 172.21.15.152:0/1482857131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0c9760 msgr2=0x55fa0d0d91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:42.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc950e30640 1 --2- 172.21.15.152:0/1482857131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0c9760 0x55fa0d0d91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc950e30640 1 -- 172.21.15.152:0/1482857131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0d4940 msgr2=0x55fa0d147470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc950e30640 1 --2- 172.21.15.152:0/1482857131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0d4940 0x55fa0d147470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc950e30640 1 -- 172.21.15.152:0/1482857131 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa0d0c6980 con 0x55fa0d0ca080 2024-09-11T22:55:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc950e30640 1 --2- 172.21.15.152:0/1482857131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d96e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0bc080 tx=0x7fc93c0dc1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a4164402d5a1b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc934ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc934ff9640 1 -- 172.21.15.152:0/1482857131 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c03bcb0 con 0x55fa0d0ca080 2024-09-11T22:55:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc934ff9640 1 -- 172.21.15.152:0/1482857131 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc93c0bc350 con 0x55fa0d0ca080 2024-09-11T22:55:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.406+0000 7fc955c6e740 1 -- 172.21.15.152:0/1482857131 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc94401e970 con 0x55fa0d0ca080 2024-09-11T22:55:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.407+0000 7fc934ff9640 1 -- 172.21.15.152:0/1482857131 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc93c080050 con 0x55fa0d0ca080 2024-09-11T22:55:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.407+0000 7fc955c6e740 10 client.16326 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.407+0000 7fc955c6e740 1 -- 172.21.15.152:0/1482857131 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa0d0c1260 con 0x55fa0d0ca080 2024-09-11T22:55:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.407+0000 7fc934ff9640 1 -- 172.21.15.152:0/1482857131 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc94401e970 con 0x55fa0d0ca080 2024-09-11T22:55:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.407+0000 7fc934ff9640 1 -- 172.21.15.152:0/1482857131 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc93c0ed430 con 0x55fa0d0ca080 2024-09-11T22:55:42.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.407+0000 7fc955c6e740 10 client.16326 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.407+0000 7fc955c6e740 10 client.16326 resolve_mds: validated gid 24455 aka daemon mds.b 2024-09-11T22:55:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.407+0000 7fc955c6e740 20 client.16326 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.407+0000 7fc955c6e740 1 --2- 172.21.15.152:0/1482857131 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55fa0d150170 0x55fa0d152560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.407+0000 7fc955c6e740 4 client.16326 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}] 2024-09-11T22:55:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.407+0000 7fc955c6e740 1 -- 172.21.15.152:0/1482857131 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}) -- 0x55fa0d036d20 con 0x55fa0d150170 2024-09-11T22:55:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.407+0000 7fc951631640 1 --2- 172.21.15.152:0/1482857131 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55fa0d150170 0x55fa0d152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.408+0000 7fc951631640 1 --2- 172.21.15.152:0/1482857131 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55fa0d150170 0x55fa0d152560 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1f7c38bfab29255 server_cookie=81c7ea3050a2928a in_seq=0 out_seq=0 2024-09-11T22:55:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.408+0000 7fc934ff9640 10 client.16326 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.414+0000 7fc934ff9640 1 -- 172.21.15.152:0/1482857131 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fa0d036d20 con 0x55fa0d150170 2024-09-11T22:55:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.414+0000 7fc934ff9640 10 client.16326 handle_command_reply: tid=0 2024-09-11T22:55:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.414+0000 7fc955c6e740 1 client.16326 shutdown 2024-09-11T22:55:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.414+0000 7fc955c6e740 1 -- 172.21.15.152:0/1482857131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0ca080 msgr2=0x55fa0d0d96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:42.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.414+0000 7fc955c6e740 1 --2- 172.21.15.152:0/1482857131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d96e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc93c0bc080 tx=0x7fc93c0dc1a0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.414+0000 7fc955c6e740 1 -- 172.21.15.152:0/1482857131 shutdown_connections 2024-09-11T22:55:42.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.414+0000 7fc955c6e740 1 --2- 172.21.15.152:0/1482857131 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55fa0d150170 0x55fa0d152560 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.414+0000 7fc955c6e740 1 --2- 172.21.15.152:0/1482857131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fa0d0d4940 0x55fa0d147470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.414+0000 7fc955c6e740 1 --2- 172.21.15.152:0/1482857131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fa0d0ca080 0x55fa0d0d96e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.414+0000 7fc934ff9640 0 client.16326 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:42.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.414+0000 7fc955c6e740 1 --2- 172.21.15.152:0/1482857131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fa0d0c9760 0x55fa0d0d91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.414+0000 7fc955c6e740 1 -- 172.21.15.152:0/1482857131 >> 172.21.15.152:0/1482857131 conn(0x55fa0d0c5cf0 msgr2=0x55fa0d0c4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:42.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.414+0000 7fc955c6e740 1 -- 172.21.15.152:0/1482857131 shutdown_connections 2024-09-11T22:55:42.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.414+0000 7fc955c6e740 1 -- 172.21.15.152:0/1482857131 wait complete. 2024-09-11T22:55:42.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.414+0000 7fc955c6e740 20 client.16326 trim_cache size 0 max 16384 2024-09-11T22:55:42.354 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:55:42.354 INFO:teuthology.orchestra.run.smithi152.stdout: "epoch": 28, 2024-09-11T22:55:42.354 INFO:teuthology.orchestra.run.smithi152.stdout: "leader": 24455, 2024-09-11T22:55:42.354 INFO:teuthology.orchestra.run.smithi152.stdout: "set_version": 10, 2024-09-11T22:55:42.354 INFO:teuthology.orchestra.run.smithi152.stdout: "sets": { 2024-09-11T22:55:42.354 INFO:teuthology.orchestra.run.smithi152.stdout: "9adbcab6": { 2024-09-11T22:55:42.354 INFO:teuthology.orchestra.run.smithi152.stdout: "version": 10, 2024-09-11T22:55:42.354 INFO:teuthology.orchestra.run.smithi152.stdout: "age_ref": 0.0, 2024-09-11T22:55:42.354 INFO:teuthology.orchestra.run.smithi152.stdout: "state": { 2024-09-11T22:55:42.354 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "QUIESCED", 2024-09-11T22:55:42.354 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.0 2024-09-11T22:55:42.355 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T22:55:42.355 INFO:teuthology.orchestra.run.smithi152.stdout: "timeout": 90.0, 2024-09-11T22:55:42.355 INFO:teuthology.orchestra.run.smithi152.stdout: "expiration": 144.7, 2024-09-11T22:55:42.355 INFO:teuthology.orchestra.run.smithi152.stdout: "members": { 2024-09-11T22:55:42.355 INFO:teuthology.orchestra.run.smithi152.stdout: "file:/": { 2024-09-11T22:55:42.355 INFO:teuthology.orchestra.run.smithi152.stdout: "excluded": false, 2024-09-11T22:55:42.355 INFO:teuthology.orchestra.run.smithi152.stdout: "state": { 2024-09-11T22:55:42.355 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "QUIESCED", 2024-09-11T22:55:42.355 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.0 2024-09-11T22:55:42.355 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:42.355 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:42.355 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:42.355 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:42.355 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:55:42.356 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:55:42.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.415+0000 7fc953895640 1 -- 172.21.15.152:0/1798138630 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9380a2cf0 msgr2=0x7fc9380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:42.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.415+0000 7fc953895640 1 --2- 172.21.15.152:0/1798138630 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9380a2cf0 0x7fc9380a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc9440098b0 tx=0x7fc944009600 comp rx=0 tx=0).stop 2024-09-11T22:55:42.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.415+0000 7fc953895640 1 -- 172.21.15.152:0/1798138630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc94c102790 msgr2=0x7fc94c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:42.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.415+0000 7fc953895640 1 --2- 172.21.15.152:0/1798138630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc94c102790 0x7fc94c198960 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc93c006010 tx=0x7fc93c0042f0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.415+0000 7fc953895640 1 -- 172.21.15.152:0/1798138630 shutdown_connections 2024-09-11T22:55:42.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.415+0000 7fc953895640 1 --2- 172.21.15.152:0/1798138630 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9380a2cf0 0x7fc9380a51b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.415+0000 7fc953895640 1 --2- 172.21.15.152:0/1798138630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc94c106f10 0x7fc94c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.415+0000 7fc953895640 1 --2- 172.21.15.152:0/1798138630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc94c102790 0x7fc94c198960 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.415+0000 7fc953895640 1 --2- 172.21.15.152:0/1798138630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc94c101de0 0x7fc94c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:42.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.415+0000 7fc953895640 1 -- 172.21.15.152:0/1798138630 >> 172.21.15.152:0/1798138630 conn(0x7fc94c0fd8c0 msgr2=0x7fc94c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:42.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.415+0000 7fc953895640 1 -- 172.21.15.152:0/1798138630 shutdown_connections 2024-09-11T22:55:42.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:55:41.415+0000 7fc953895640 1 -- 172.21.15.152:0/1798138630 wait complete. 2024-09-11T22:55:42.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:42 smithi191 ceph-mon[35211]: pgmap v87: 97 pgs: 97 active+clean; 11 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 2.4 KiB/s wr, 201 op/s 2024-09-11T22:55:42.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:42 smithi204 ceph-mon[35454]: pgmap v87: 97 pgs: 97 active+clean; 11 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 2.4 KiB/s wr, 201 op/s 2024-09-11T22:55:42.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:42 smithi152 ceph-mon[31181]: pgmap v87: 97 pgs: 97 active+clean; 11 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 2.4 KiB/s wr, 201 op/s 2024-09-11T22:55:43.015 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9adbcab6', quiesce took 7.0 seconds. Will release after: 17.7 2024-09-11T22:55:44.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:44 smithi191 ceph-mon[35211]: pgmap v88: 97 pgs: 97 active+clean; 11 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 1.9 KiB/s wr, 199 op/s 2024-09-11T22:55:44.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:44 smithi204 ceph-mon[35454]: pgmap v88: 97 pgs: 97 active+clean; 11 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 1.9 KiB/s wr, 199 op/s 2024-09-11T22:55:44.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:44 smithi152 ceph-mon[31181]: pgmap v88: 97 pgs: 97 active+clean; 11 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 1.9 KiB/s wr, 199 op/s 2024-09-11T22:55:46.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:46 smithi191 ceph-mon[35211]: pgmap v89: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 KiB/s wr, 230 op/s 2024-09-11T22:55:46.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:46 smithi204 ceph-mon[35454]: pgmap v89: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 KiB/s wr, 230 op/s 2024-09-11T22:55:46.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:46 smithi152 ceph-mon[31181]: pgmap v89: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.7 KiB/s wr, 230 op/s 2024-09-11T22:55:48.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:48 smithi191 ceph-mon[35211]: pgmap v90: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.7 KiB/s wr, 226 op/s 2024-09-11T22:55:48.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:48 smithi204 ceph-mon[35454]: pgmap v90: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.7 KiB/s wr, 226 op/s 2024-09-11T22:55:48.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:48 smithi152 ceph-mon[31181]: pgmap v90: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.7 KiB/s wr, 226 op/s 2024-09-11T22:55:50.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:50 smithi191 ceph-mon[35211]: pgmap v91: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.7 KiB/s wr, 230 op/s 2024-09-11T22:55:50.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:50 smithi204 ceph-mon[35454]: pgmap v91: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.7 KiB/s wr, 230 op/s 2024-09-11T22:55:50.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:50 smithi152 ceph-mon[31181]: pgmap v91: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.7 KiB/s wr, 230 op/s 2024-09-11T22:55:52.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:52 smithi204 ceph-mon[35454]: pgmap v92: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 117 op/s 2024-09-11T22:55:52.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:52 smithi152 ceph-mon[31181]: pgmap v92: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 117 op/s 2024-09-11T22:55:52.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:52 smithi191 ceph-mon[35211]: pgmap v92: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.0 KiB/s wr, 117 op/s 2024-09-11T22:55:54.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:54 smithi204 ceph-mon[35454]: pgmap v93: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 35 op/s 2024-09-11T22:55:54.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:54 smithi152 ceph-mon[31181]: pgmap v93: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 35 op/s 2024-09-11T22:55:54.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:54 smithi191 ceph-mon[35211]: pgmap v93: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 35 op/s 2024-09-11T22:55:55.013 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.010+0000 7fad13554640 1 -- 172.21.15.152:0/1306281697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad0c102cc0 msgr2=0x7fad0c1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:55.013 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.010+0000 7fad13554640 1 --2- 172.21.15.152:0/1306281697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad0c102cc0 0x7fad0c1030a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fad00009900 tx=0x7fad0002a480 comp rx=0 tx=0).stop 2024-09-11T22:55:55.013 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.010+0000 7fad13554640 1 -- 172.21.15.152:0/1306281697 shutdown_connections 2024-09-11T22:55:55.013 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.010+0000 7fad13554640 1 --2- 172.21.15.152:0/1306281697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad0c10c110 0x7fad0c10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.013 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.010+0000 7fad13554640 1 --2- 172.21.15.152:0/1306281697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad0c1035e0 0x7fad0c10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.013 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.010+0000 7fad13554640 1 --2- 172.21.15.152:0/1306281697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad0c102cc0 0x7fad0c1030a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.010+0000 7fad13554640 1 -- 172.21.15.152:0/1306281697 >> 172.21.15.152:0/1306281697 conn(0x7fad0c0fc6b0 msgr2=0x7fad0c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.010+0000 7fad13554640 1 -- 172.21.15.152:0/1306281697 shutdown_connections 2024-09-11T22:55:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.011+0000 7fad13554640 1 -- 172.21.15.152:0/1306281697 wait complete. 2024-09-11T22:55:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.011+0000 7fad13554640 1 Processor -- start 2024-09-11T22:55:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.011+0000 7fad13554640 1 -- start start 2024-09-11T22:55:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.011+0000 7fad13554640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad0c102cc0 0x7fad0c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.011+0000 7fad13554640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad0c1035e0 0x7fad0c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.011+0000 7fad13554640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad0c10c110 0x7fad0c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.011+0000 7fad13554640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c111ef0 con 0x7fad0c1035e0 2024-09-11T22:55:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.011+0000 7fad13554640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c111d70 con 0x7fad0c102cc0 2024-09-11T22:55:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.011+0000 7fad13554640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c112070 con 0x7fad0c10c110 2024-09-11T22:55:55.014 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.011+0000 7fad10aef640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad0c1035e0 0x7fad0c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7fad10aef640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad0c1035e0 0x7fad0c1988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43278/0 (socket says 172.21.15.152:43278) 2024-09-11T22:55:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7fad10aef640 1 -- 172.21.15.152:0/4210539540 learned_addr learned my addr 172.21.15.152:0/4210539540 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7fad11af1640 1 --2- 172.21.15.152:0/4210539540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad0c10c110 0x7fad0c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7fad112f0640 1 --2- 172.21.15.152:0/4210539540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad0c102cc0 0x7fad0c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7fad10aef640 1 -- 172.21.15.152:0/4210539540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad0c10c110 msgr2=0x7fad0c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7fad10aef640 1 --2- 172.21.15.152:0/4210539540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad0c10c110 0x7fad0c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7fad10aef640 1 -- 172.21.15.152:0/4210539540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad0c102cc0 msgr2=0x7fad0c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7fad10aef640 1 --2- 172.21.15.152:0/4210539540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad0c102cc0 0x7fad0c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7fad10aef640 1 -- 172.21.15.152:0/4210539540 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0c19d350 con 0x7fad0c1035e0 2024-09-11T22:55:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7fad112f0640 1 --2- 172.21.15.152:0/4210539540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad0c102cc0 0x7fad0c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:55.015 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7fad11af1640 1 --2- 172.21.15.152:0/4210539540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad0c10c110 0x7fad0c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7fad10aef640 1 --2- 172.21.15.152:0/4210539540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad0c1035e0 0x7fad0c1988a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fad0800e9f0 tx=0x7fad0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7facfe7fc640 1 -- 172.21.15.152:0/4210539540 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0800ccc0 con 0x7fad0c1035e0 2024-09-11T22:55:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7facfe7fc640 1 -- 172.21.15.152:0/4210539540 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad08010910 con 0x7fad0c1035e0 2024-09-11T22:55:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7facfe7fc640 1 -- 172.21.15.152:0/4210539540 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad08018920 con 0x7fad0c1035e0 2024-09-11T22:55:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7fad13554640 1 -- 172.21.15.152:0/4210539540 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0c19d640 con 0x7fad0c1035e0 2024-09-11T22:55:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.012+0000 7fad13554640 1 -- 172.21.15.152:0/4210539540 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad0c1a4ed0 con 0x7fad0c1035e0 2024-09-11T22:55:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.015+0000 7facfe7fc640 1 -- 172.21.15.152:0/4210539540 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fad08010430 con 0x7fad0c1035e0 2024-09-11T22:55:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.015+0000 7fad163c8740 1 Processor -- start 2024-09-11T22:55:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.015+0000 7fad163c8740 1 -- start start 2024-09-11T22:55:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.015+0000 7fad163c8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2bcaf0 0x559b2d2bced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad163c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 0x559b2d2c7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad163c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2c7cb0 0x559b2d2ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad163c8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x559b2d2212f0 con 0x559b2d2bcaf0 2024-09-11T22:55:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad163c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559b2d1c9270 con 0x559b2d2c7cb0 2024-09-11T22:55:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad163c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559b2d21d2b0 con 0x559b2d2bd410 2024-09-11T22:55:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7facfe7fc640 1 --2- 172.21.15.152:0/4210539540 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7facf80a2da0 0x7facf80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad11af1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2bcaf0 0x559b2d2bced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad112f0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 0x559b2d2c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad10aef640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2c7cb0 0x559b2d2ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad112f0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 0x559b2d2c7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:44776/0 (socket says 172.21.15.152:44776) 2024-09-11T22:55:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad112f0640 1 -- 172.21.15.152:0/2404507702 learned_addr learned my addr 172.21.15.152:0/2404507702 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad11af1640 1 --2- 172.21.15.152:0/4210539540 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7facf80a2da0 0x7facf80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7facfe7fc640 1 -- 172.21.15.152:0/4210539540 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fad080cf780 con 0x7fad0c1035e0 2024-09-11T22:55:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad112f0640 1 -- 172.21.15.152:0/2404507702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2c7cb0 msgr2=0x559b2d2ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad112f0640 1 --2- 172.21.15.152:0/2404507702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2c7cb0 0x559b2d2ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad112f0640 1 -- 172.21.15.152:0/2404507702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2bcaf0 msgr2=0x559b2d2bced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad112f0640 1 --2- 172.21.15.152:0/2404507702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2bcaf0 0x559b2d2bced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad112f0640 1 -- 172.21.15.152:0/2404507702 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b2d1c8340 con 0x559b2d2bd410 2024-09-11T22:55:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad11af1640 1 --2- 172.21.15.152:0/4210539540 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7facf80a2da0 0x7facf80a5260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad04005c40 tx=0x7fad04004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.016+0000 7fad112f0640 1 --2- 172.21.15.152:0/2404507702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 0x559b2d2c7770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fad00033000 tx=0x7fad00005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f32f86e79ddb347 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.017+0000 7facf77fe640 1 -- 172.21.15.152:0/2404507702 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0003f070 con 0x559b2d2bd410 2024-09-11T22:55:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.017+0000 7facf77fe640 1 -- 172.21.15.152:0/2404507702 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad00004030 con 0x559b2d2bd410 2024-09-11T22:55:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.017+0000 7facf77fe640 1 -- 172.21.15.152:0/2404507702 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00035070 con 0x559b2d2bd410 2024-09-11T22:55:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.018+0000 7fad163c8740 1 -- 172.21.15.152:0/2404507702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 msgr2=0x559b2d2c7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.018+0000 7fad163c8740 1 --2- 172.21.15.152:0/2404507702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 0x559b2d2c7770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fad00033000 tx=0x7fad00005e60 comp rx=0 tx=0).stop 2024-09-11T22:55:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.018+0000 7fad163c8740 1 -- 172.21.15.152:0/2404507702 shutdown_connections 2024-09-11T22:55:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.018+0000 7fad163c8740 1 --2- 172.21.15.152:0/2404507702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2c7cb0 0x559b2d2ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.018+0000 7fad163c8740 1 --2- 172.21.15.152:0/2404507702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 0x559b2d2c7770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.018+0000 7fad163c8740 1 --2- 172.21.15.152:0/2404507702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2bcaf0 0x559b2d2bced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.018+0000 7fad163c8740 1 -- 172.21.15.152:0/2404507702 >> 172.21.15.152:0/2404507702 conn(0x559b2d2b9060 msgr2=0x559b2d2b9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.018+0000 7fad163c8740 1 -- 172.21.15.152:0/2404507702 shutdown_connections 2024-09-11T22:55:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.018+0000 7fad163c8740 1 -- 172.21.15.152:0/2404507702 wait complete. 2024-09-11T22:55:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.018+0000 7fad163c8740 1 -- start start 2024-09-11T22:55:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.018+0000 7fad163c8740 1 Processor -- start 2024-09-11T22:55:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad163c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 0x559b2d2c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad163c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 0x559b2d2c5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad163c8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2c7cb0 0x559b2d2c5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad163c8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x559b2d2ca880 con 0x559b2d2c7cb0 2024-09-11T22:55:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad163c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559b2d1aa290 con 0x559b2d2bcaf0 2024-09-11T22:55:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad163c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559b2d2caa00 con 0x559b2d2bd410 2024-09-11T22:55:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad163c8740 10 client.0 _ll_register_callbacks cb 0x559b2d1db140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad11af1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2c7cb0 0x559b2d2c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad112f0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 0x559b2d2c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad11af1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2c7cb0 0x559b2d2c5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43292/0 (socket says 172.21.15.152:43292) 2024-09-11T22:55:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad10aef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 0x559b2d2c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad11af1640 1 -- 172.21.15.152:0/1767132255 learned_addr learned my addr 172.21.15.152:0/1767132255 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad11af1640 1 -- 172.21.15.152:0/1767132255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 msgr2=0x559b2d2c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad11af1640 1 --2- 172.21.15.152:0/1767132255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 0x559b2d2c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad11af1640 1 -- 172.21.15.152:0/1767132255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 msgr2=0x559b2d2c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad11af1640 1 --2- 172.21.15.152:0/1767132255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 0x559b2d2c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad11af1640 1 -- 172.21.15.152:0/1767132255 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b2cf34210 con 0x559b2d2c7cb0 2024-09-11T22:55:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad112f0640 1 --2- 172.21.15.152:0/1767132255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 0x559b2d2c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:55:55.027 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad11af1640 1 --2- 172.21.15.152:0/1767132255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2c7cb0 0x559b2d2c5670 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fad0400cfb0 tx=0x7fad04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8451d8f021d7221 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7facf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7facf4ff9640 1 -- 172.21.15.152:0/1767132255 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04025070 con 0x559b2d2c7cb0 2024-09-11T22:55:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7facf4ff9640 1 -- 172.21.15.152:0/1767132255 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fad04021660 con 0x559b2d2c7cb0 2024-09-11T22:55:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.019+0000 7fad163c8740 1 -- 172.21.15.152:0/1767132255 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad00004030 con 0x559b2d2c7cb0 2024-09-11T22:55:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.020+0000 7facf4ff9640 1 -- 172.21.15.152:0/1767132255 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fad04021860 con 0x559b2d2c7cb0 2024-09-11T22:55:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.020+0000 7fad163c8740 10 client.16356 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.020+0000 7fad163c8740 1 -- 172.21.15.152:0/1767132255 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559b2d000620 con 0x559b2d2c7cb0 2024-09-11T22:55:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.020+0000 7facf4ff9640 1 -- 172.21.15.152:0/1767132255 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad00004030 con 0x559b2d2c7cb0 2024-09-11T22:55:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.020+0000 7facf4ff9640 1 -- 172.21.15.152:0/1767132255 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fad0401e040 con 0x559b2d2c7cb0 2024-09-11T22:55:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.020+0000 7fad163c8740 10 client.16356 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.020+0000 7fad163c8740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.020+0000 7fad163c8740 20 client.16356 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.020+0000 7fad163c8740 1 --2- 172.21.15.152:0/1767132255 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559b2d2d7cb0 0x559b2d2da0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.020+0000 7fad163c8740 4 client.16356 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:55:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.020+0000 7fad163c8740 1 -- 172.21.15.152:0/1767132255 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b2d22ea20 con 0x559b2d2d7cb0 2024-09-11T22:55:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.020+0000 7fad112f0640 1 --2- 172.21.15.152:0/1767132255 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559b2d2d7cb0 0x559b2d2da0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.021+0000 7facf4ff9640 10 client.16356 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.021+0000 7fad112f0640 1 --2- 172.21.15.152:0/1767132255 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559b2d2d7cb0 0x559b2d2da0a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=395c48fd4e078fea server_cookie=df5d22c0b9b84465 in_seq=0 out_seq=0 2024-09-11T22:55:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.021+0000 7facf4ff9640 1 -- 172.21.15.152:0/1767132255 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b2d22ea20 con 0x559b2d2d7cb0 2024-09-11T22:55:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.021+0000 7facf4ff9640 10 client.16356 handle_command_reply: tid=0 2024-09-11T22:55:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.021+0000 7fad163c8740 1 client.16356 shutdown 2024-09-11T22:55:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.022+0000 7fad163c8740 1 -- 172.21.15.152:0/1767132255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2c7cb0 msgr2=0x559b2d2c5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.022+0000 7fad163c8740 1 --2- 172.21.15.152:0/1767132255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2c7cb0 0x559b2d2c5670 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fad0400cfb0 tx=0x7fad04004a40 comp rx=0 tx=0).stop 2024-09-11T22:55:55.029 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.022+0000 7fad163c8740 1 -- 172.21.15.152:0/1767132255 shutdown_connections 2024-09-11T22:55:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.022+0000 7fad163c8740 1 --2- 172.21.15.152:0/1767132255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2c7cb0 0x559b2d2c5670 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.022+0000 7fad163c8740 1 --2- 172.21.15.152:0/1767132255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 0x559b2d2c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.022+0000 7fad163c8740 1 --2- 172.21.15.152:0/1767132255 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559b2d2d7cb0 0x559b2d2da0a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.022+0000 7fad163c8740 1 --2- 172.21.15.152:0/1767132255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 0x559b2d2c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.022+0000 7facf4ff9640 0 client.16356 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.022+0000 7fad163c8740 1 -- 172.21.15.152:0/1767132255 >> 172.21.15.152:0/1767132255 conn(0x559b2d2b9060 msgr2=0x559b2d2b81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.022+0000 7fad163c8740 1 -- 172.21.15.152:0/1767132255 shutdown_connections 2024-09-11T22:55:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.022+0000 7fad163c8740 1 -- 172.21.15.152:0/1767132255 wait complete. 2024-09-11T22:55:55.030 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.022+0000 7fad163c8740 20 client.16356 trim_cache size 0 max 16384 2024-09-11T22:55:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.036+0000 7fad163c8740 1 Processor -- start 2024-09-11T22:55:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.036+0000 7fad163c8740 1 -- start start 2024-09-11T22:55:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.036+0000 7fad163c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 0x559b2d2c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.036+0000 7fad163c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 0x559b2d2c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.036+0000 7fad163c8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2c7cb0 0x559b2d2d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.036+0000 7fad163c8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1ad690 con 0x559b2d2c7cb0 2024-09-11T22:55:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.036+0000 7fad163c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1af1d0 con 0x559b2d2bcaf0 2024-09-11T22:55:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.036+0000 7fad163c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1add60 con 0x559b2d2bd410 2024-09-11T22:55:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.037+0000 7fad11af1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2c7cb0 0x559b2d2d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.037+0000 7fad11af1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2c7cb0 0x559b2d2d0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43298/0 (socket says 172.21.15.152:43298) 2024-09-11T22:55:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.037+0000 7fad11af1640 1 -- 172.21.15.152:0/2839050925 learned_addr learned my addr 172.21.15.152:0/2839050925 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.037+0000 7fad112f0640 1 --2- 172.21.15.152:0/2839050925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 0x559b2d2c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.037+0000 7fad11af1640 1 -- 172.21.15.152:0/2839050925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 msgr2=0x559b2d2c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:55.042 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.037+0000 7fad10aef640 1 --2- 172.21.15.152:0/2839050925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 0x559b2d2c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.037+0000 7fad11af1640 1 --2- 172.21.15.152:0/2839050925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 0x559b2d2c6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.037+0000 7fad11af1640 1 -- 172.21.15.152:0/2839050925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 msgr2=0x559b2d2c54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.037+0000 7fad11af1640 1 --2- 172.21.15.152:0/2839050925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 0x559b2d2c54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.037+0000 7fad11af1640 1 -- 172.21.15.152:0/2839050925 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b2d2ca410 con 0x559b2d2c7cb0 2024-09-11T22:55:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.037+0000 7fad112f0640 1 --2- 172.21.15.152:0/2839050925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 0x559b2d2c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:55:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.037+0000 7fad10aef640 1 --2- 172.21.15.152:0/2839050925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 0x559b2d2c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:55:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.037+0000 7fad11af1640 1 --2- 172.21.15.152:0/2839050925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2c7cb0 0x559b2d2d0070 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fad04009e80 tx=0x7fad04009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb786a22cf226b33 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.037+0000 7facf6ffd640 1 -- 172.21.15.152:0/2839050925 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04025070 con 0x559b2d2c7cb0 2024-09-11T22:55:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.037+0000 7facf6ffd640 1 -- 172.21.15.152:0/2839050925 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad0400fc50 con 0x559b2d2c7cb0 2024-09-11T22:55:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.037+0000 7facf6ffd640 1 -- 172.21.15.152:0/2839050925 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04021e10 con 0x559b2d2c7cb0 2024-09-11T22:55:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.038+0000 7fad163c8740 1 -- 172.21.15.152:0/2839050925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2c7cb0 msgr2=0x559b2d2d0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.038+0000 7fad163c8740 1 --2- 172.21.15.152:0/2839050925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2c7cb0 0x559b2d2d0070 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fad04009e80 tx=0x7fad04009eb0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.043 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.038+0000 7fad163c8740 1 -- 172.21.15.152:0/2839050925 shutdown_connections 2024-09-11T22:55:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.038+0000 7fad163c8740 1 --2- 172.21.15.152:0/2839050925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2c7cb0 0x559b2d2d0070 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.038+0000 7fad163c8740 1 --2- 172.21.15.152:0/2839050925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2bd410 0x559b2d2c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.038+0000 7fad163c8740 1 --2- 172.21.15.152:0/2839050925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 0x559b2d2c54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.038+0000 7fad163c8740 1 -- 172.21.15.152:0/2839050925 >> 172.21.15.152:0/2839050925 conn(0x559b2d2b9060 msgr2=0x559b2d2c6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.038+0000 7fad163c8740 1 -- 172.21.15.152:0/2839050925 shutdown_connections 2024-09-11T22:55:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.038+0000 7fad163c8740 1 -- 172.21.15.152:0/2839050925 wait complete. 2024-09-11T22:55:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.038+0000 7fad163c8740 1 -- start start 2024-09-11T22:55:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.038+0000 7fad163c8740 1 Processor -- start 2024-09-11T22:55:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad163c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 0x559b2d2c26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad163c8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2bd410 0x559b2d2df3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad163c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2c7cb0 0x559b2d2df8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad163c8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1aeb00 con 0x559b2d2bd410 2024-09-11T22:55:55.044 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad163c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1ae430 con 0x559b2d2bcaf0 2024-09-11T22:55:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad163c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fad0c1ab650 con 0x559b2d2c7cb0 2024-09-11T22:55:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad163c8740 10 client.0 _ll_register_callbacks cb 0x7fad0400cee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:55:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad10aef640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2bd410 0x559b2d2df3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad10aef640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2bd410 0x559b2d2df3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43310/0 (socket says 172.21.15.152:43310) 2024-09-11T22:55:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad10aef640 1 -- 172.21.15.152:0/1338945019 learned_addr learned my addr 172.21.15.152:0/1338945019 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:55:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad112f0640 1 --2- 172.21.15.152:0/1338945019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 0x559b2d2c26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad11af1640 1 --2- 172.21.15.152:0/1338945019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2c7cb0 0x559b2d2df8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad10aef640 1 -- 172.21.15.152:0/1338945019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2c7cb0 msgr2=0x559b2d2df8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad10aef640 1 --2- 172.21.15.152:0/1338945019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2c7cb0 0x559b2d2df8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad10aef640 1 -- 172.21.15.152:0/1338945019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 msgr2=0x559b2d2c26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:55:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad10aef640 1 --2- 172.21.15.152:0/1338945019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 0x559b2d2c26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.045 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad10aef640 1 -- 172.21.15.152:0/1338945019 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b2d2e0110 con 0x559b2d2bd410 2024-09-11T22:55:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7fad10aef640 1 --2- 172.21.15.152:0/1338945019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2bd410 0x559b2d2df3a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fad08009780 tx=0x7fad0800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a9df996e888699 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:55:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.039+0000 7facf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:55:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.040+0000 7facf4ff9640 1 -- 172.21.15.152:0/1338945019 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad08026040 con 0x559b2d2bd410 2024-09-11T22:55:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.040+0000 7facf4ff9640 1 -- 172.21.15.152:0/1338945019 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fad080cfbb0 con 0x559b2d2bd410 2024-09-11T22:55:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.040+0000 7fad163c8740 1 -- 172.21.15.152:0/1338945019 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad0400fc50 con 0x559b2d2bd410 2024-09-11T22:55:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.040+0000 7facf4ff9640 1 -- 172.21.15.152:0/1338945019 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fad080cd3a0 con 0x559b2d2bd410 2024-09-11T22:55:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.040+0000 7fad163c8740 10 client.16362 fetch_fsmap learned FSMap version 28 2024-09-11T22:55:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.040+0000 7fad163c8740 1 -- 172.21.15.152:0/1338945019 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559b2d3468c0 con 0x559b2d2bd410 2024-09-11T22:55:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.040+0000 7facf4ff9640 1 -- 172.21.15.152:0/1338945019 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad0400fc50 con 0x559b2d2bd410 2024-09-11T22:55:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.040+0000 7facf4ff9640 1 -- 172.21.15.152:0/1338945019 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fad0809b260 con 0x559b2d2bd410 2024-09-11T22:55:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.040+0000 7fad163c8740 10 client.16362 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:55:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.040+0000 7fad163c8740 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:55:55.046 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.040+0000 7fad163c8740 20 client.16362 populate_metadata read hostname 'smithi152' 2024-09-11T22:55:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.040+0000 7fad163c8740 1 --2- 172.21.15.152:0/1338945019 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559b2d347010 0x559b2d349430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:55:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.040+0000 7fad163c8740 4 client.16362 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:55:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.040+0000 7fad163c8740 1 -- 172.21.15.152:0/1338945019 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b2d228d70 con 0x559b2d347010 2024-09-11T22:55:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.040+0000 7fad112f0640 1 --2- 172.21.15.152:0/1338945019 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559b2d347010 0x559b2d349430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:55:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.041+0000 7facf4ff9640 10 client.16362 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:55.047 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.041+0000 7fad112f0640 1 --2- 172.21.15.152:0/1338945019 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559b2d347010 0x559b2d349430 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ec802287eec1ce7 server_cookie=ed9fe0ac0032bb99 in_seq=0 out_seq=0 2024-09-11T22:55:55.048 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7facf4ff9640 1 -- 172.21.15.152:0/1338945019 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b2d228d70 con 0x559b2d347010 2024-09-11T22:55:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7facf4ff9640 10 client.16362 handle_command_reply: tid=0 2024-09-11T22:55:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad163c8740 1 client.16362 shutdown 2024-09-11T22:55:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad163c8740 1 -- 172.21.15.152:0/1338945019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2bd410 msgr2=0x559b2d2df3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad163c8740 1 --2- 172.21.15.152:0/1338945019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2bd410 0x559b2d2df3a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fad08009780 tx=0x7fad0800cf60 comp rx=0 tx=0).stop 2024-09-11T22:55:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad163c8740 1 -- 172.21.15.152:0/1338945019 shutdown_connections 2024-09-11T22:55:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad163c8740 1 --2- 172.21.15.152:0/1338945019 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559b2d2c7cb0 0x559b2d2df8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad163c8740 1 --2- 172.21.15.152:0/1338945019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b2d2bd410 0x559b2d2df3a0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad163c8740 1 --2- 172.21.15.152:0/1338945019 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559b2d347010 0x559b2d349430 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad163c8740 1 --2- 172.21.15.152:0/1338945019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559b2d2bcaf0 0x559b2d2c26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7facf4ff9640 0 client.16362 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:55:55.049 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad163c8740 1 -- 172.21.15.152:0/1338945019 >> 172.21.15.152:0/1338945019 conn(0x559b2d2b9060 msgr2=0x559b2d2bed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad163c8740 1 -- 172.21.15.152:0/1338945019 shutdown_connections 2024-09-11T22:55:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad163c8740 1 -- 172.21.15.152:0/1338945019 wait complete. 2024-09-11T22:55:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad163c8740 20 client.16362 trim_cache size 0 max 16384 2024-09-11T22:55:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:{ 2024-09-11T22:55:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "message": "", 2024-09-11T22:55:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout: "return_code": 0 2024-09-11T22:55:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stdout:} 2024-09-11T22:55:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad13554640 1 -- 172.21.15.152:0/4210539540 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7facf80a2da0 msgr2=0x7facf80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad13554640 1 --2- 172.21.15.152:0/4210539540 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7facf80a2da0 0x7facf80a5260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad04005c40 tx=0x7fad04004ab0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad13554640 1 -- 172.21.15.152:0/4210539540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad0c1035e0 msgr2=0x7fad0c1988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:55:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad13554640 1 --2- 172.21.15.152:0/4210539540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad0c1035e0 0x7fad0c1988a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fad0800e9f0 tx=0x7fad0800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad13554640 1 -- 172.21.15.152:0/4210539540 shutdown_connections 2024-09-11T22:55:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad13554640 1 --2- 172.21.15.152:0/4210539540 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7facf80a2da0 0x7facf80a5260 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad13554640 1 --2- 172.21.15.152:0/4210539540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad0c10c110 0x7fad0c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad13554640 1 --2- 172.21.15.152:0/4210539540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad0c1035e0 0x7fad0c1988a0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad13554640 1 --2- 172.21.15.152:0/4210539540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad0c102cc0 0x7fad0c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:55:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.048+0000 7fad13554640 1 -- 172.21.15.152:0/4210539540 >> 172.21.15.152:0/4210539540 conn(0x7fad0c0fc6b0 msgr2=0x7fad0c0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:55:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.049+0000 7fad13554640 1 -- 172.21.15.152:0/4210539540 shutdown_connections 2024-09-11T22:55:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi152.front.sepia.ceph.com.smithi152.stderr:2024-09-11T22:55:55.049+0000 7fad13554640 1 -- 172.21.15.152:0/4210539540 wait complete. 2024-09-11T22:55:56.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:56 smithi204 ceph-mon[35454]: pgmap v94: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.8 KiB/s wr, 39 op/s 2024-09-11T22:55:56.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:56 smithi152 ceph-mon[31181]: pgmap v94: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.8 KiB/s wr, 39 op/s 2024-09-11T22:55:56.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:56 smithi191 ceph-mon[35211]: pgmap v94: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.8 KiB/s wr, 39 op/s 2024-09-11T22:55:58.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:55:58 smithi152 ceph-mon[31181]: pgmap v95: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:55:58.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:55:58 smithi191 ceph-mon[35211]: pgmap v95: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:55:58.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:55:58 smithi204 ceph-mon[35454]: pgmap v95: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:56:00.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:00 smithi152 ceph-mon[31181]: pgmap v96: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-11T22:56:00.717 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:56:00.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:00 smithi191 ceph-mon[35211]: pgmap v96: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-11T22:56:00.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:00 smithi204 ceph-mon[35454]: pgmap v96: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-11T22:56:00.978 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:56:01.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.501+0000 7efcfde79640 1 -- 172.21.15.152:0/630921614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcf8105a90 msgr2=0x7efcf8105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:01.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.501+0000 7efcfde79640 1 --2- 172.21.15.152:0/630921614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcf8105a90 0x7efcf8105e70 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7efce8009900 tx=0x7efce802a3f0 comp rx=0 tx=0).stop 2024-09-11T22:56:01.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.501+0000 7efcfde79640 1 -- 172.21.15.152:0/630921614 shutdown_connections 2024-09-11T22:56:01.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.501+0000 7efcfde79640 1 --2- 172.21.15.152:0/630921614 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcf8101bc0 0x7efcf810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:01.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.501+0000 7efcfde79640 1 --2- 172.21.15.152:0/630921614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcf8101240 0x7efcf8101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:01.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.501+0000 7efcfde79640 1 --2- 172.21.15.152:0/630921614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcf8105a90 0x7efcf8105e70 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:01.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.501+0000 7efcfde79640 1 -- 172.21.15.152:0/630921614 >> 172.21.15.152:0/630921614 conn(0x7efcf8077210 msgr2=0x7efcf80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:01.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.501+0000 7efcfde79640 1 -- 172.21.15.152:0/630921614 shutdown_connections 2024-09-11T22:56:01.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.501+0000 7efcfde79640 1 -- 172.21.15.152:0/630921614 wait complete. 2024-09-11T22:56:01.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.501+0000 7efcfde79640 1 Processor -- start 2024-09-11T22:56:01.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcfde79640 1 -- start start 2024-09-11T22:56:01.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcfde79640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcf8101240 0x7efcf8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:01.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcfde79640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcf8101bc0 0x7efcf8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:01.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcfde79640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcf8105a90 0x7efcf819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:01.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcfde79640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efcf8112050 con 0x7efcf8105a90 2024-09-11T22:56:01.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcfde79640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efcf8111ed0 con 0x7efcf8101240 2024-09-11T22:56:01.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcfde79640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efcf81121d0 con 0x7efcf8101bc0 2024-09-11T22:56:01.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcf7fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcf8105a90 0x7efcf819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:01.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcf7fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcf8105a90 0x7efcf819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48410/0 (socket says 172.21.15.152:48410) 2024-09-11T22:56:01.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcf7fff640 1 -- 172.21.15.152:0/4112580957 learned_addr learned my addr 172.21.15.152:0/4112580957 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:01.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcf77fe640 1 --2- 172.21.15.152:0/4112580957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcf8101240 0x7efcf8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:01.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcf6ffd640 1 --2- 172.21.15.152:0/4112580957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcf8101bc0 0x7efcf8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:01.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcf7fff640 1 -- 172.21.15.152:0/4112580957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcf8101bc0 msgr2=0x7efcf8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:01.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcf7fff640 1 --2- 172.21.15.152:0/4112580957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcf8101bc0 0x7efcf8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:01.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcf7fff640 1 -- 172.21.15.152:0/4112580957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcf8101240 msgr2=0x7efcf8198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:01.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcf7fff640 1 --2- 172.21.15.152:0/4112580957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcf8101240 0x7efcf8198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:01.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.502+0000 7efcf7fff640 1 -- 172.21.15.152:0/4112580957 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf819d3e0 con 0x7efcf8105a90 2024-09-11T22:56:01.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.503+0000 7efcf7fff640 1 --2- 172.21.15.152:0/4112580957 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcf8105a90 0x7efcf819cd00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7efcf000eaa0 tx=0x7efcf000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:01.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.503+0000 7efcf4ff9640 1 -- 172.21.15.152:0/4112580957 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0004d20 con 0x7efcf8105a90 2024-09-11T22:56:01.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.503+0000 7efcf4ff9640 1 -- 172.21.15.152:0/4112580957 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf0005020 con 0x7efcf8105a90 2024-09-11T22:56:01.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.503+0000 7efcfde79640 1 -- 172.21.15.152:0/4112580957 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf819d6d0 con 0x7efcf8105a90 2024-09-11T22:56:01.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.503+0000 7efcf4ff9640 1 -- 172.21.15.152:0/4112580957 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00116d0 con 0x7efcf8105a90 2024-09-11T22:56:01.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.503+0000 7efcfde79640 1 -- 172.21.15.152:0/4112580957 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf81a4fb0 con 0x7efcf8105a90 2024-09-11T22:56:01.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.504+0000 7efcfde79640 1 -- 172.21.15.152:0/4112580957 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf8101680 con 0x7efcf8105a90 2024-09-11T22:56:01.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.505+0000 7efcf4ff9640 1 -- 172.21.15.152:0/4112580957 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7efcf00051d0 con 0x7efcf8105a90 2024-09-11T22:56:01.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.506+0000 7efcf4ff9640 1 --2- 172.21.15.152:0/4112580957 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efce00a2cf0 0x7efce00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:01.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.506+0000 7efcf77fe640 1 --2- 172.21.15.152:0/4112580957 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efce00a2cf0 0x7efce00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:01.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.506+0000 7efcf4ff9640 1 -- 172.21.15.152:0/4112580957 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7efcf00cf5c0 con 0x7efcf8105a90 2024-09-11T22:56:01.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.506+0000 7efcf77fe640 1 --2- 172.21.15.152:0/4112580957 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efce00a2cf0 0x7efce00a51b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efce8009230 tx=0x7efce80058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:01.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.510+0000 7efcf4ff9640 1 -- 172.21.15.152:0/4112580957 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efcf0016030 con 0x7efcf8105a90 2024-09-11T22:56:01.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.711+0000 7efcfde79640 1 -- 172.21.15.152:0/4112580957 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcf81a5180 con 0x7efcf8105a90 2024-09-11T22:56:01.712 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.712+0000 7efcf4ff9640 1 -- 172.21.15.152:0/4112580957 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7efcf009c230 con 0x7efcf8105a90 2024-09-11T22:56:01.713 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:56:01.713 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:56:01.713 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:56:01.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.714+0000 7efcfde79640 1 -- 172.21.15.152:0/4112580957 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efce00a2cf0 msgr2=0x7efce00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:01.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.714+0000 7efcfde79640 1 --2- 172.21.15.152:0/4112580957 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efce00a2cf0 0x7efce00a51b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efce8009230 tx=0x7efce80058d0 comp rx=0 tx=0).stop 2024-09-11T22:56:01.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.714+0000 7efcfde79640 1 -- 172.21.15.152:0/4112580957 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcf8105a90 msgr2=0x7efcf819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:01.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.714+0000 7efcfde79640 1 --2- 172.21.15.152:0/4112580957 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcf8105a90 0x7efcf819cd00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7efcf000eaa0 tx=0x7efcf000eea0 comp rx=0 tx=0).stop 2024-09-11T22:56:01.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.714+0000 7efcfde79640 1 -- 172.21.15.152:0/4112580957 shutdown_connections 2024-09-11T22:56:01.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.714+0000 7efcfde79640 1 --2- 172.21.15.152:0/4112580957 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efce00a2cf0 0x7efce00a51b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:01.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.714+0000 7efcfde79640 1 --2- 172.21.15.152:0/4112580957 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efcf8105a90 0x7efcf819cd00 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:01.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.714+0000 7efcfde79640 1 --2- 172.21.15.152:0/4112580957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcf8101bc0 0x7efcf8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:01.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.714+0000 7efcfde79640 1 --2- 172.21.15.152:0/4112580957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcf8101240 0x7efcf8198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:01.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.714+0000 7efcfde79640 1 -- 172.21.15.152:0/4112580957 >> 172.21.15.152:0/4112580957 conn(0x7efcf8077210 msgr2=0x7efcf8103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:01.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.714+0000 7efcfde79640 1 -- 172.21.15.152:0/4112580957 shutdown_connections 2024-09-11T22:56:01.715 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:01.714+0000 7efcfde79640 1 -- 172.21.15.152:0/4112580957 wait complete. 2024-09-11T22:56:02.542 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:02 smithi152 ceph-mon[31181]: pgmap v97: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-11T22:56:02.542 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:02 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4112580957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:56:02.576 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24455 quiesce db --set-id 9adbcab6 --release --await' 2024-09-11T22:56:02.576 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24455 quiesce db --set-id 9adbcab6 --release --await 2024-09-11T22:56:02.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:02 smithi191 ceph-mon[35211]: pgmap v97: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-11T22:56:02.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:02 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/4112580957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:56:02.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:02 smithi204 ceph-mon[35454]: pgmap v97: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-11T22:56:02.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:02 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/4112580957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:56:02.827 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:56:03.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.424+0000 7f36b617a640 1 -- 172.21.15.152:0/3620106549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36b0102d40 msgr2=0x7f36b0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:03.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.424+0000 7f36b617a640 1 --2- 172.21.15.152:0/3620106549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36b0102d40 0x7f36b0103120 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009ca0 tx=0x7f36a002a950 comp rx=0 tx=0).stop 2024-09-11T22:56:03.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.425+0000 7f36b617a640 1 -- 172.21.15.152:0/3620106549 shutdown_connections 2024-09-11T22:56:03.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.425+0000 7f36b617a640 1 --2- 172.21.15.152:0/3620106549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b010c170 0x7f36b010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.425+0000 7f36b617a640 1 --2- 172.21.15.152:0/3620106549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36b0103660 0x7f36b010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.425+0000 7f36b617a640 1 --2- 172.21.15.152:0/3620106549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36b0102d40 0x7f36b0103120 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.425+0000 7f36b617a640 1 -- 172.21.15.152:0/3620106549 >> 172.21.15.152:0/3620106549 conn(0x7f36b00fc7a0 msgr2=0x7f36b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:03.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.425+0000 7f36b617a640 1 -- 172.21.15.152:0/3620106549 shutdown_connections 2024-09-11T22:56:03.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.425+0000 7f36b617a640 1 -- 172.21.15.152:0/3620106549 wait complete. 2024-09-11T22:56:03.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.425+0000 7f36b617a640 1 Processor -- start 2024-09-11T22:56:03.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.425+0000 7f36b617a640 1 -- start start 2024-09-11T22:56:03.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36b617a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36b0102d40 0x7f36b0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36b617a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b0103660 0x7f36b01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.460 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36b617a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36b010c170 0x7f36b019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36b617a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f36b0111fb0 con 0x7f36b010c170 2024-09-11T22:56:03.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36b617a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f36b0111e30 con 0x7f36b0102d40 2024-09-11T22:56:03.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36b617a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36b0112130 con 0x7f36b0103660 2024-09-11T22:56:03.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36affff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36b010c170 0x7f36b019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:03.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36affff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36b010c170 0x7f36b019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48426/0 (socket says 172.21.15.152:48426) 2024-09-11T22:56:03.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36affff640 1 -- 172.21.15.152:0/7333679 learned_addr learned my addr 172.21.15.152:0/7333679 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:03.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36af7fe640 1 --2- 172.21.15.152:0/7333679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36b0102d40 0x7f36b0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:03.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36aeffd640 1 --2- 172.21.15.152:0/7333679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b0103660 0x7f36b01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:03.461 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36affff640 1 -- 172.21.15.152:0/7333679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b0103660 msgr2=0x7f36b01989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:03.462 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36affff640 1 --2- 172.21.15.152:0/7333679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b0103660 0x7f36b01989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36affff640 1 -- 172.21.15.152:0/7333679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36b0102d40 msgr2=0x7f36b0198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:03.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36affff640 1 --2- 172.21.15.152:0/7333679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36b0102d40 0x7f36b0198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36affff640 1 -- 172.21.15.152:0/7333679 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b019d420 con 0x7f36b010c170 2024-09-11T22:56:03.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36affff640 1 --2- 172.21.15.152:0/7333679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36b010c170 0x7f36b019cd40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009fd0 tx=0x7f36a800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:03.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36acff9640 1 -- 172.21.15.152:0/7333679 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a8011830 con 0x7f36b010c170 2024-09-11T22:56:03.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36b617a640 1 -- 172.21.15.152:0/7333679 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b019d710 con 0x7f36b010c170 2024-09-11T22:56:03.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36acff9640 1 -- 172.21.15.152:0/7333679 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a8016070 con 0x7f36b010c170 2024-09-11T22:56:03.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36acff9640 1 -- 172.21.15.152:0/7333679 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a800f2e0 con 0x7f36b010c170 2024-09-11T22:56:03.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.426+0000 7f36b617a640 1 -- 172.21.15.152:0/7333679 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b01a4ff0 con 0x7f36b010c170 2024-09-11T22:56:03.463 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.429+0000 7f36acff9640 1 -- 172.21.15.152:0/7333679 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f36a80119d0 con 0x7f36b010c170 2024-09-11T22:56:03.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.429+0000 7f36acff9640 1 --2- 172.21.15.152:0/7333679 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f36980a2cf0 0x7f36980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.429+0000 7f36b8553740 1 Processor -- start 2024-09-11T22:56:03.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.429+0000 7f36b8553740 1 -- start start 2024-09-11T22:56:03.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.429+0000 7f36af7fe640 1 --2- 172.21.15.152:0/7333679 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f36980a2cf0 0x7f36980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:03.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.429+0000 7f36acff9640 1 -- 172.21.15.152:0/7333679 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f36a80ce960 con 0x7f36b010c170 2024-09-11T22:56:03.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36b8553740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7815b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36b8553740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d7815b980 0x558d78165d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36b8553740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d78166240 0x558d78168630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36b8553740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558d780c0620 con 0x558d7815b060 2024-09-11T22:56:03.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36b8553740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558d78067e50 con 0x558d7815b980 2024-09-11T22:56:03.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36b8553740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558d780bc5e0 con 0x558d78166240 2024-09-11T22:56:03.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36af7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7815b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:03.464 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36af7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7815b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48430/0 (socket says 172.21.15.152:48430) 2024-09-11T22:56:03.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36af7fe640 1 -- 172.21.15.152:0/3449669241 learned_addr learned my addr 172.21.15.152:0/3449669241 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:03.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36aeffd640 1 --2- 172.21.15.152:0/3449669241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d7815b980 0x558d78165d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:03.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36affff640 1 --2- 172.21.15.152:0/3449669241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d78166240 0x558d78168630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:03.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36af7fe640 1 --2- 172.21.15.152:0/7333679 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f36980a2cf0 0x7f36980a51b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f36a0006010 tx=0x7f36a002faa0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:03.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36aeffd640 1 -- 172.21.15.152:0/3449669241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d78166240 msgr2=0x558d78168630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:03.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36aeffd640 1 --2- 172.21.15.152:0/3449669241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d78166240 0x558d78168630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36aeffd640 1 -- 172.21.15.152:0/3449669241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 msgr2=0x558d7815b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:03.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36aeffd640 1 --2- 172.21.15.152:0/3449669241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7815b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36aeffd640 1 -- 172.21.15.152:0/3449669241 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d78058d40 con 0x558d7815b980 2024-09-11T22:56:03.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36af7fe640 1 --2- 172.21.15.152:0/3449669241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7815b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:56:03.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.430+0000 7f36aeffd640 1 --2- 172.21.15.152:0/3449669241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d7815b980 0x558d78165d00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f36a400bbb0 tx=0x7f36a400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c52c2bc2192c46d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:03.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.431+0000 7f3695ffb640 1 -- 172.21.15.152:0/3449669241 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4015070 con 0x558d7815b980 2024-09-11T22:56:03.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.431+0000 7f3695ffb640 1 -- 172.21.15.152:0/3449669241 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a401a940 con 0x558d7815b980 2024-09-11T22:56:03.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.431+0000 7f3695ffb640 1 -- 172.21.15.152:0/3449669241 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4011900 con 0x558d7815b980 2024-09-11T22:56:03.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.431+0000 7f36b8553740 1 -- 172.21.15.152:0/3449669241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d7815b980 msgr2=0x558d78165d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:03.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.431+0000 7f36b8553740 1 --2- 172.21.15.152:0/3449669241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d7815b980 0x558d78165d00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f36a400bbb0 tx=0x7f36a400f730 comp rx=0 tx=0).stop 2024-09-11T22:56:03.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.431+0000 7f36b8553740 1 -- 172.21.15.152:0/3449669241 shutdown_connections 2024-09-11T22:56:03.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.431+0000 7f36b8553740 1 --2- 172.21.15.152:0/3449669241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d78166240 0x558d78168630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.431+0000 7f36b8553740 1 --2- 172.21.15.152:0/3449669241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d7815b980 0x558d78165d00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.431+0000 7f36b8553740 1 --2- 172.21.15.152:0/3449669241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7815b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.431+0000 7f36b8553740 1 -- 172.21.15.152:0/3449669241 >> 172.21.15.152:0/3449669241 conn(0x558d781575f0 msgr2=0x558d781579d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:03.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.431+0000 7f36b8553740 1 -- 172.21.15.152:0/3449669241 shutdown_connections 2024-09-11T22:56:03.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.431+0000 7f36b8553740 1 -- 172.21.15.152:0/3449669241 wait complete. 2024-09-11T22:56:03.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.432+0000 7f36b8553740 1 -- start start 2024-09-11T22:56:03.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.432+0000 7f36b8553740 1 Processor -- start 2024-09-11T22:56:03.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.432+0000 7f36b8553740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d781631b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.432+0000 7f36b8553740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d7815b980 0x558d781636f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.432+0000 7f36b8553740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d78166240 0x558d78163c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.432+0000 7f36b8553740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558d78168e10 con 0x558d7815b060 2024-09-11T22:56:03.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.432+0000 7f36b8553740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558d78048f10 con 0x558d78166240 2024-09-11T22:56:03.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.432+0000 7f36b8553740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558d78168f90 con 0x558d7815b980 2024-09-11T22:56:03.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.432+0000 7f36b8553740 10 client.0 _ll_register_callbacks cb 0x558d7806cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:03.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.432+0000 7f36af7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d781631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:03.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.432+0000 7f36af7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d781631b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48442/0 (socket says 172.21.15.152:48442) 2024-09-11T22:56:03.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.432+0000 7f36af7fe640 1 -- 172.21.15.152:0/3722933952 learned_addr learned my addr 172.21.15.152:0/3722933952 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:03.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.432+0000 7f36aeffd640 1 --2- 172.21.15.152:0/3722933952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d7815b980 0x558d781636f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:03.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.433+0000 7f36af7fe640 1 -- 172.21.15.152:0/3722933952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d7815b980 msgr2=0x558d781636f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:03.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.433+0000 7f36af7fe640 1 --2- 172.21.15.152:0/3722933952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d7815b980 0x558d781636f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.433+0000 7f36affff640 1 --2- 172.21.15.152:0/3722933952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d78166240 0x558d78163c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:03.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.433+0000 7f36af7fe640 1 -- 172.21.15.152:0/3722933952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d78166240 msgr2=0x558d78163c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:03.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.433+0000 7f36af7fe640 1 --2- 172.21.15.152:0/3722933952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d78166240 0x558d78163c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.433+0000 7f36af7fe640 1 -- 172.21.15.152:0/3722933952 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d77dcf580 con 0x558d7815b060 2024-09-11T22:56:03.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.433+0000 7f36affff640 1 --2- 172.21.15.152:0/3722933952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d78166240 0x558d78163c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:03.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.433+0000 7f36af7fe640 1 --2- 172.21.15.152:0/3722933952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d781631b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f36a0030040 tx=0x7f36a002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132bd73379cbe4b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:03.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.433+0000 7f368effd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:03.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.433+0000 7f368effd640 1 -- 172.21.15.152:0/3722933952 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0034040 con 0x558d7815b060 2024-09-11T22:56:03.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.433+0000 7f368effd640 1 -- 172.21.15.152:0/3722933952 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f36a0004710 con 0x558d7815b060 2024-09-11T22:56:03.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.433+0000 7f36b8553740 1 -- 172.21.15.152:0/3722933952 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36a401a940 con 0x558d7815b060 2024-09-11T22:56:03.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.433+0000 7f368effd640 1 -- 172.21.15.152:0/3722933952 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f36a004a450 con 0x558d7815b060 2024-09-11T22:56:03.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.433+0000 7f36b8553740 10 client.16380 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:03.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.433+0000 7f36b8553740 1 -- 172.21.15.152:0/3722933952 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d77e99960 con 0x558d7815b060 2024-09-11T22:56:03.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.433+0000 7f368effd640 1 -- 172.21.15.152:0/3722933952 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a401a940 con 0x558d7815b060 2024-09-11T22:56:03.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.434+0000 7f368effd640 1 -- 172.21.15.152:0/3722933952 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f36a0030810 con 0x558d7815b060 2024-09-11T22:56:03.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.434+0000 7f36b8553740 10 client.16380 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:03.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.434+0000 7f36b8553740 10 client.16380 resolve_mds: validated gid 24455 aka daemon mds.b 2024-09-11T22:56:03.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.434+0000 7f36b8553740 20 client.16380 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:03.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.434+0000 7f36b8553740 1 --2- 172.21.15.152:0/3722933952 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558d78175e30 0x558d78178280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.434+0000 7f36b8553740 4 client.16380 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:56:03.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.434+0000 7f36b8553740 1 -- 172.21.15.152:0/3722933952 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d780cdb90 con 0x558d78175e30 2024-09-11T22:56:03.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.434+0000 7f36aeffd640 1 --2- 172.21.15.152:0/3722933952 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558d78175e30 0x558d78178280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:03.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.434+0000 7f368effd640 10 client.16380 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:03.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.434+0000 7f36aeffd640 1 --2- 172.21.15.152:0/3722933952 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558d78175e30 0x558d78178280 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d9fe21c60ee5ef7 server_cookie=c52d5ab334892b7c in_seq=0 out_seq=0 2024-09-11T22:56:03.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.435+0000 7f368effd640 1 -- 172.21.15.152:0/3722933952 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d780cdb90 con 0x558d78175e30 2024-09-11T22:56:03.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.435+0000 7f368effd640 10 client.16380 handle_command_reply: tid=0 2024-09-11T22:56:03.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.435+0000 7f36b8553740 1 client.16380 shutdown 2024-09-11T22:56:03.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.435+0000 7f36b8553740 1 -- 172.21.15.152:0/3722933952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 msgr2=0x558d781631b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:03.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.435+0000 7f36b8553740 1 --2- 172.21.15.152:0/3722933952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d781631b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f36a0030040 tx=0x7f36a002a950 comp rx=0 tx=0).stop 2024-09-11T22:56:03.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.435+0000 7f36b8553740 1 -- 172.21.15.152:0/3722933952 shutdown_connections 2024-09-11T22:56:03.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.435+0000 7f36b8553740 1 --2- 172.21.15.152:0/3722933952 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558d78175e30 0x558d78178280 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.435+0000 7f36b8553740 1 --2- 172.21.15.152:0/3722933952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d78166240 0x558d78163c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.435+0000 7f36b8553740 1 --2- 172.21.15.152:0/3722933952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d7815b980 0x558d781636f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.435+0000 7f368effd640 0 client.16380 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:03.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.435+0000 7f36b8553740 1 --2- 172.21.15.152:0/3722933952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d781631b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.435+0000 7f36b8553740 1 -- 172.21.15.152:0/3722933952 >> 172.21.15.152:0/3722933952 conn(0x558d781575f0 msgr2=0x558d7815d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:03.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.435+0000 7f36b8553740 1 -- 172.21.15.152:0/3722933952 shutdown_connections 2024-09-11T22:56:03.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.436+0000 7f36b8553740 1 -- 172.21.15.152:0/3722933952 wait complete. 2024-09-11T22:56:03.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.436+0000 7f36b8553740 20 client.16380 trim_cache size 0 max 16384 2024-09-11T22:56:03.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.452+0000 7f36b8553740 1 Processor -- start 2024-09-11T22:56:03.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.452+0000 7f36b8553740 1 -- start start 2024-09-11T22:56:03.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36b8553740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7815e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36b8553740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d7815b980 0x558d78156640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36b8553740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d78166240 0x558d7817dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36b8553740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f36b01ad830 con 0x558d7815b060 2024-09-11T22:56:03.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36b8553740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f36b01af370 con 0x558d7815b980 2024-09-11T22:56:03.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36b8553740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36b01adf00 con 0x558d78166240 2024-09-11T22:56:03.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36af7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7815e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:03.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36af7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7815e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48444/0 (socket says 172.21.15.152:48444) 2024-09-11T22:56:03.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36af7fe640 1 -- 172.21.15.152:0/2312212186 learned_addr learned my addr 172.21.15.152:0/2312212186 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:03.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36af7fe640 1 -- 172.21.15.152:0/2312212186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d78166240 msgr2=0x558d7817dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:03.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36af7fe640 1 --2- 172.21.15.152:0/2312212186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d78166240 0x558d7817dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36af7fe640 1 -- 172.21.15.152:0/2312212186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d7815b980 msgr2=0x558d78156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:03.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36af7fe640 1 --2- 172.21.15.152:0/2312212186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d7815b980 0x558d78156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36af7fe640 1 -- 172.21.15.152:0/2312212186 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d781650f0 con 0x558d7815b060 2024-09-11T22:56:03.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36af7fe640 1 --2- 172.21.15.152:0/2312212186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7815e6b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f36a002ae70 tx=0x7f36a0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eed06819ada8415 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:03.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36957fa640 1 -- 172.21.15.152:0/2312212186 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0034040 con 0x558d7815b060 2024-09-11T22:56:03.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36957fa640 1 -- 172.21.15.152:0/2312212186 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a0030e50 con 0x558d7815b060 2024-09-11T22:56:03.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.453+0000 7f36957fa640 1 -- 172.21.15.152:0/2312212186 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a004abc0 con 0x558d7815b060 2024-09-11T22:56:03.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.454+0000 7f36b8553740 1 -- 172.21.15.152:0/2312212186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 msgr2=0x558d7815e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:03.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.454+0000 7f36b8553740 1 --2- 172.21.15.152:0/2312212186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7815e6b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f36a002ae70 tx=0x7f36a0009bd0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.454+0000 7f36b8553740 1 -- 172.21.15.152:0/2312212186 shutdown_connections 2024-09-11T22:56:03.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.454+0000 7f36b8553740 1 --2- 172.21.15.152:0/2312212186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d78166240 0x558d7817dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.454+0000 7f36b8553740 1 --2- 172.21.15.152:0/2312212186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d7815b980 0x558d78156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.454+0000 7f36b8553740 1 --2- 172.21.15.152:0/2312212186 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7815e6b0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.454+0000 7f36b8553740 1 -- 172.21.15.152:0/2312212186 >> 172.21.15.152:0/2312212186 conn(0x558d781575f0 msgr2=0x558d78155f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:03.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.454+0000 7f36b8553740 1 -- 172.21.15.152:0/2312212186 shutdown_connections 2024-09-11T22:56:03.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.454+0000 7f36b8553740 1 -- 172.21.15.152:0/2312212186 wait complete. 2024-09-11T22:56:03.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.454+0000 7f36b8553740 1 -- start start 2024-09-11T22:56:03.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36b8553740 1 Processor -- start 2024-09-11T22:56:03.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36b8553740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7816e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36b8553740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d7815b980 0x558d7816ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36b8553740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d78166240 0x558d7816a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36b8553740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f36b01aeca0 con 0x558d7815b060 2024-09-11T22:56:03.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36b8553740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f36b01ae5d0 con 0x558d78166240 2024-09-11T22:56:03.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36b8553740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f36b01ab7f0 con 0x558d7815b980 2024-09-11T22:56:03.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36b8553740 10 client.0 _ll_register_callbacks cb 0x7f36a002f860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:03.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36af7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7816e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:03.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36af7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7816e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48450/0 (socket says 172.21.15.152:48450) 2024-09-11T22:56:03.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36af7fe640 1 -- 172.21.15.152:0/445254089 learned_addr learned my addr 172.21.15.152:0/445254089 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:03.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36aeffd640 1 --2- 172.21.15.152:0/445254089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d7815b980 0x558d7816ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:03.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36affff640 1 --2- 172.21.15.152:0/445254089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d78166240 0x558d7816a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:03.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36af7fe640 1 -- 172.21.15.152:0/445254089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d7815b980 msgr2=0x558d7816ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:03.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36af7fe640 1 --2- 172.21.15.152:0/445254089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d7815b980 0x558d7816ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36af7fe640 1 -- 172.21.15.152:0/445254089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d78166240 msgr2=0x558d7816a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:03.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36af7fe640 1 --2- 172.21.15.152:0/445254089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d78166240 0x558d7816a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36af7fe640 1 -- 172.21.15.152:0/445254089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d7816b170 con 0x558d7815b060 2024-09-11T22:56:03.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36aeffd640 1 --2- 172.21.15.152:0/445254089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d7815b980 0x558d7816ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:56:03.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f36af7fe640 1 --2- 172.21.15.152:0/445254089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7816e530 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f36a0030040 tx=0x7f36a0030c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9323dab7e14bc3a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:03.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.455+0000 7f368effd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:03.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.456+0000 7f368effd640 1 -- 172.21.15.152:0/445254089 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0034040 con 0x558d7815b060 2024-09-11T22:56:03.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.456+0000 7f368effd640 1 -- 172.21.15.152:0/445254089 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f36a004ad50 con 0x558d7815b060 2024-09-11T22:56:03.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.456+0000 7f36b8553740 1 -- 172.21.15.152:0/445254089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36a0030e50 con 0x558d7815b060 2024-09-11T22:56:03.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.456+0000 7f368effd640 1 -- 172.21.15.152:0/445254089 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f36a0030440 con 0x558d7815b060 2024-09-11T22:56:03.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.456+0000 7f36b8553740 10 client.16386 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:03.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.456+0000 7f36b8553740 1 -- 172.21.15.152:0/445254089 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558d781e17c0 con 0x558d7815b060 2024-09-11T22:56:03.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.456+0000 7f368effd640 1 -- 172.21.15.152:0/445254089 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36a0030e50 con 0x558d7815b060 2024-09-11T22:56:03.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.456+0000 7f368effd640 1 -- 172.21.15.152:0/445254089 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f36a005cc90 con 0x558d7815b060 2024-09-11T22:56:03.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.456+0000 7f36b8553740 10 client.16386 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:03.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.456+0000 7f36b8553740 10 client.16386 resolve_mds: validated gid 24455 aka daemon mds.b 2024-09-11T22:56:03.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.456+0000 7f36b8553740 20 client.16386 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:03.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.456+0000 7f36b8553740 1 --2- 172.21.15.152:0/445254089 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558d781e1dc0 0x558d781e41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:03.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.456+0000 7f36b8553740 4 client.16386 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9adbcab6", "release": true, "await": true}] 2024-09-11T22:56:03.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.456+0000 7f36b8553740 1 -- 172.21.15.152:0/445254089 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9adbcab6", "release": true, "await": true}) -- 0x558d780c80a0 con 0x558d781e1dc0 2024-09-11T22:56:03.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.457+0000 7f36aeffd640 1 --2- 172.21.15.152:0/445254089 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558d781e1dc0 0x558d781e41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:03.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.457+0000 7f368effd640 10 client.16386 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:03.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.457+0000 7f36aeffd640 1 --2- 172.21.15.152:0/445254089 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558d781e1dc0 0x558d781e41b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31efdd8fadb73080 server_cookie=4aea7b1ba4c7fbe4 in_seq=0 out_seq=0 2024-09-11T22:56:03.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.461+0000 7f368effd640 1 -- 172.21.15.152:0/445254089 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558d780c80a0 con 0x558d781e1dc0 2024-09-11T22:56:03.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.461+0000 7f368effd640 10 client.16386 handle_command_reply: tid=0 2024-09-11T22:56:03.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.461+0000 7f36b8553740 1 client.16386 shutdown 2024-09-11T22:56:03.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.462+0000 7f36b8553740 1 -- 172.21.15.152:0/445254089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 msgr2=0x558d7816e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:03.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.462+0000 7f36b8553740 1 --2- 172.21.15.152:0/445254089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7816e530 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f36a0030040 tx=0x7f36a0030c60 comp rx=0 tx=0).stop 2024-09-11T22:56:03.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.462+0000 7f36b8553740 1 -- 172.21.15.152:0/445254089 shutdown_connections 2024-09-11T22:56:03.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.462+0000 7f36b8553740 1 --2- 172.21.15.152:0/445254089 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x558d781e1dc0 0x558d781e41b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.462+0000 7f36b8553740 1 --2- 172.21.15.152:0/445254089 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558d78166240 0x558d7816a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.462+0000 7f36b8553740 1 --2- 172.21.15.152:0/445254089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558d7815b980 0x558d7816ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.462+0000 7f36b8553740 1 --2- 172.21.15.152:0/445254089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d7815b060 0x558d7816e530 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.462+0000 7f36b8553740 1 -- 172.21.15.152:0/445254089 >> 172.21.15.152:0/445254089 conn(0x558d781575f0 msgr2=0x558d7815d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:03.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.462+0000 7f368effd640 0 client.16386 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:03.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.462+0000 7f36b8553740 1 -- 172.21.15.152:0/445254089 shutdown_connections 2024-09-11T22:56:03.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.462+0000 7f36b8553740 1 -- 172.21.15.152:0/445254089 wait complete. 2024-09-11T22:56:03.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.462+0000 7f36b8553740 20 client.16386 trim_cache size 0 max 16384 2024-09-11T22:56:03.478 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:56:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: "epoch": 28, 2024-09-11T22:56:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: "leader": 24455, 2024-09-11T22:56:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: "set_version": 12, 2024-09-11T22:56:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "sets": { 2024-09-11T22:56:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "9adbcab6": { 2024-09-11T22:56:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "version": 12, 2024-09-11T22:56:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "age_ref": 0.0, 2024-09-11T22:56:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "state": { 2024-09-11T22:56:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RELEASED", 2024-09-11T22:56:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.0 2024-09-11T22:56:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T22:56:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "timeout": 90.0, 2024-09-11T22:56:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "expiration": 144.7, 2024-09-11T22:56:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "members": { 2024-09-11T22:56:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "file:/": { 2024-09-11T22:56:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "excluded": false, 2024-09-11T22:56:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "state": { 2024-09-11T22:56:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RELEASED", 2024-09-11T22:56:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 0.0 2024-09-11T22:56:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:56:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:56:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:56:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:56:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:56:03.480 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:56:03.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.462+0000 7f36b617a640 1 -- 172.21.15.152:0/7333679 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f36980a2cf0 msgr2=0x7f36980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:03.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.462+0000 7f36b617a640 1 --2- 172.21.15.152:0/7333679 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f36980a2cf0 0x7f36980a51b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f36a0006010 tx=0x7f36a002faa0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.463+0000 7f36b617a640 1 -- 172.21.15.152:0/7333679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36b010c170 msgr2=0x7f36b019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:03.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.463+0000 7f36b617a640 1 --2- 172.21.15.152:0/7333679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36b010c170 0x7f36b019cd40 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009fd0 tx=0x7f36a800b8e0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.463+0000 7f36b617a640 1 -- 172.21.15.152:0/7333679 shutdown_connections 2024-09-11T22:56:03.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.463+0000 7f36b617a640 1 --2- 172.21.15.152:0/7333679 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f36980a2cf0 0x7f36980a51b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.463+0000 7f36b617a640 1 --2- 172.21.15.152:0/7333679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36b010c170 0x7f36b019cd40 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.463+0000 7f36b617a640 1 --2- 172.21.15.152:0/7333679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36b0103660 0x7f36b01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.463+0000 7f36b617a640 1 --2- 172.21.15.152:0/7333679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36b0102d40 0x7f36b0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:03.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.463+0000 7f36b617a640 1 -- 172.21.15.152:0/7333679 >> 172.21.15.152:0/7333679 conn(0x7f36b00fc7a0 msgr2=0x7f36b00fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:03.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.463+0000 7f36b617a640 1 -- 172.21.15.152:0/7333679 shutdown_connections 2024-09-11T22:56:03.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:03.463+0000 7f36b617a640 1 -- 172.21.15.152:0/7333679 wait complete. 2024-09-11T22:56:03.482 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-09-11T22:56:03.482 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-11T22:56:03.807 DEBUG:teuthology.parallel:result is None 2024-09-11T22:56:03.807 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T22:56:03.839 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T22:56:03.839 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-11T22:56:03.900 INFO:tasks.quiescer:joining Quiescers 2024-09-11T22:56:03.900 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-11T22:56:03.900 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:56:04.232 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9adbcab6', total seconds elapsed: 28.2 2024-09-11T22:56:04.298 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:56:04.451 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:04 smithi152 ceph-mon[31181]: pgmap v98: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-09-11T22:56:04.451 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:04 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:56:04.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:04 smithi191 ceph-mon[35211]: pgmap v98: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-09-11T22:56:04.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:04 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:56:04.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:04 smithi204 ceph-mon[35454]: pgmap v98: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-09-11T22:56:04.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:04 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:56:04.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.823+0000 7f87e913b640 1 -- 172.21.15.152:0/393019532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87e4102570 msgr2=0x7f87e41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:04.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.823+0000 7f87e913b640 1 --2- 172.21.15.152:0/393019532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87e4102570 0x7f87e41029b0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009900 tx=0x7f87d802a480 comp rx=0 tx=0).stop 2024-09-11T22:56:04.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.823+0000 7f87e913b640 1 -- 172.21.15.152:0/393019532 shutdown_connections 2024-09-11T22:56:04.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.823+0000 7f87e913b640 1 --2- 172.21.15.152:0/393019532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e4102ef0 0x7f87e410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:04.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.823+0000 7f87e913b640 1 --2- 172.21.15.152:0/393019532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87e4102570 0x7f87e41029b0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:04.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.824+0000 7f87e913b640 1 --2- 172.21.15.152:0/393019532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e4100e20 0x7f87e4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:04.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.824+0000 7f87e913b640 1 -- 172.21.15.152:0/393019532 >> 172.21.15.152:0/393019532 conn(0x7f87e40fc7d0 msgr2=0x7f87e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:04.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.824+0000 7f87e913b640 1 -- 172.21.15.152:0/393019532 shutdown_connections 2024-09-11T22:56:04.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.824+0000 7f87e913b640 1 -- 172.21.15.152:0/393019532 wait complete. 2024-09-11T22:56:04.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.824+0000 7f87e913b640 1 Processor -- start 2024-09-11T22:56:04.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e913b640 1 -- start start 2024-09-11T22:56:04.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e913b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e4100e20 0x7f87e41993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:04.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e913b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e4102570 0x7f87e4199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:04.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e913b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87e4102ef0 0x7f87e41a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:04.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e913b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f87e4110f50 con 0x7f87e4102ef0 2024-09-11T22:56:04.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e913b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f87e4110dd0 con 0x7f87e4100e20 2024-09-11T22:56:04.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e913b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f87e41110d0 con 0x7f87e4102570 2024-09-11T22:56:04.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e359e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87e4102ef0 0x7f87e41a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:04.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e2d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e4100e20 0x7f87e41993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:04.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e359e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87e4102ef0 0x7f87e41a4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48466/0 (socket says 172.21.15.152:48466) 2024-09-11T22:56:04.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e359e640 1 -- 172.21.15.152:0/3981267868 learned_addr learned my addr 172.21.15.152:0/3981267868 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:04.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e259c640 1 --2- 172.21.15.152:0/3981267868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e4102570 0x7f87e4199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:04.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e359e640 1 -- 172.21.15.152:0/3981267868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e4102570 msgr2=0x7f87e4199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:04.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e359e640 1 --2- 172.21.15.152:0/3981267868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e4102570 0x7f87e4199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:04.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e359e640 1 -- 172.21.15.152:0/3981267868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e4100e20 msgr2=0x7f87e41993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:04.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e359e640 1 --2- 172.21.15.152:0/3981267868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e4100e20 0x7f87e41993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:04.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e359e640 1 -- 172.21.15.152:0/3981267868 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e41a4980 con 0x7f87e4102ef0 2024-09-11T22:56:04.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e2d9d640 1 --2- 172.21.15.152:0/3981267868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e4100e20 0x7f87e41993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:56:04.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.825+0000 7f87e359e640 1 --2- 172.21.15.152:0/3981267868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87e4102ef0 0x7f87e41a4440 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00cad0 tx=0x7f87dc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:04.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.826+0000 7f87cbfff640 1 -- 172.21.15.152:0/3981267868 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc00ae20 con 0x7f87e4102ef0 2024-09-11T22:56:04.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.826+0000 7f87cbfff640 1 -- 172.21.15.152:0/3981267868 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87dc005020 con 0x7f87e4102ef0 2024-09-11T22:56:04.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.826+0000 7f87e913b640 1 -- 172.21.15.152:0/3981267868 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e41a4b50 con 0x7f87e4102ef0 2024-09-11T22:56:04.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.826+0000 7f87cbfff640 1 -- 172.21.15.152:0/3981267868 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc00f6f0 con 0x7f87e4102ef0 2024-09-11T22:56:04.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.826+0000 7f87e913b640 1 -- 172.21.15.152:0/3981267868 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e41a5010 con 0x7f87e4102ef0 2024-09-11T22:56:04.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.827+0000 7f87e913b640 1 -- 172.21.15.152:0/3981267868 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e4101200 con 0x7f87e4102ef0 2024-09-11T22:56:04.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.828+0000 7f87cbfff640 1 -- 172.21.15.152:0/3981267868 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f87dc0051d0 con 0x7f87e4102ef0 2024-09-11T22:56:04.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.829+0000 7f87cbfff640 1 --2- 172.21.15.152:0/3981267868 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f87cc0a2cf0 0x7f87cc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:04.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.829+0000 7f87e2d9d640 1 --2- 172.21.15.152:0/3981267868 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f87cc0a2cf0 0x7f87cc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:04.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.829+0000 7f87cbfff640 1 -- 172.21.15.152:0/3981267868 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f87dc0cf700 con 0x7f87e4102ef0 2024-09-11T22:56:04.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.829+0000 7f87e2d9d640 1 --2- 172.21.15.152:0/3981267868 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f87cc0a2cf0 0x7f87cc0a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f87d4005d00 tx=0x7f87d400b410 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:04.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:04.832+0000 7f87cbfff640 1 -- 172.21.15.152:0/3981267868 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f87dc014030 con 0x7f87e4102ef0 2024-09-11T22:56:05.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:05.033+0000 7f87e913b640 1 -- 172.21.15.152:0/3981267868 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87e406a560 con 0x7f87e4102ef0 2024-09-11T22:56:05.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:05.034+0000 7f87cbfff640 1 -- 172.21.15.152:0/3981267868 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f87e406a560 con 0x7f87e4102ef0 2024-09-11T22:56:05.037 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:56:05.037 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:56:05.037 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:56:05.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:05.036+0000 7f87e913b640 1 -- 172.21.15.152:0/3981267868 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f87cc0a2cf0 msgr2=0x7f87cc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:05.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:05.036+0000 7f87e913b640 1 --2- 172.21.15.152:0/3981267868 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f87cc0a2cf0 0x7f87cc0a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f87d4005d00 tx=0x7f87d400b410 comp rx=0 tx=0).stop 2024-09-11T22:56:05.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:05.036+0000 7f87e913b640 1 -- 172.21.15.152:0/3981267868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87e4102ef0 msgr2=0x7f87e41a4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:05.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:05.036+0000 7f87e913b640 1 --2- 172.21.15.152:0/3981267868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87e4102ef0 0x7f87e41a4440 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00cad0 tx=0x7f87dc00cf90 comp rx=0 tx=0).stop 2024-09-11T22:56:05.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:05.036+0000 7f87e913b640 1 -- 172.21.15.152:0/3981267868 shutdown_connections 2024-09-11T22:56:05.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:05.036+0000 7f87e913b640 1 --2- 172.21.15.152:0/3981267868 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f87cc0a2cf0 0x7f87cc0a51b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:05.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:05.036+0000 7f87e913b640 1 --2- 172.21.15.152:0/3981267868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f87e4102ef0 0x7f87e41a4440 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:05.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:05.036+0000 7f87e913b640 1 --2- 172.21.15.152:0/3981267868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87e4102570 0x7f87e4199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:05.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:05.036+0000 7f87e913b640 1 --2- 172.21.15.152:0/3981267868 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87e4100e20 0x7f87e41993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:05.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:05.036+0000 7f87e913b640 1 -- 172.21.15.152:0/3981267868 >> 172.21.15.152:0/3981267868 conn(0x7f87e40fc7d0 msgr2=0x7f87e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:05.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:05.037+0000 7f87e913b640 1 -- 172.21.15.152:0/3981267868 shutdown_connections 2024-09-11T22:56:05.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:05.037+0000 7f87e913b640 1 -- 172.21.15.152:0/3981267868 wait complete. 2024-09-11T22:56:05.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:05 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:56:05.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:05 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:56:05.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:05 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:56:05.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:05 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3981267868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:56:05.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:05 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:56:05.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:05 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:56:05.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:05 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:56:05.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:05 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3981267868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:56:05.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:05 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:56:05.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:05 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:56:05.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:05 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:56:05.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:05 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3981267868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:56:05.865 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24455 quiesce db --cancel --all' 2024-09-11T22:56:05.865 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24455 quiesce db --cancel --all 2024-09-11T22:56:06.114 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:56:06.538 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:06 smithi152 ceph-mon[31181]: pgmap v99: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-09-11T22:56:06.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.623+0000 7f2b74b92640 1 -- 172.21.15.152:0/1565246205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b70101220 msgr2=0x7f2b70101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:06.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.623+0000 7f2b74b92640 1 --2- 172.21.15.152:0/1565246205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b70101220 0x7f2b70101660 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009900 tx=0x7f2b6402a410 comp rx=0 tx=0).stop 2024-09-11T22:56:06.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.623+0000 7f2b74b92640 1 -- 172.21.15.152:0/1565246205 shutdown_connections 2024-09-11T22:56:06.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.623+0000 7f2b74b92640 1 --2- 172.21.15.152:0/1565246205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b70101ba0 0x7f2b7010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.623+0000 7f2b74b92640 1 --2- 172.21.15.152:0/1565246205 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b70101220 0x7f2b70101660 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.623+0000 7f2b74b92640 1 --2- 172.21.15.152:0/1565246205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b70105a70 0x7f2b70105e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.623+0000 7f2b74b92640 1 -- 172.21.15.152:0/1565246205 >> 172.21.15.152:0/1565246205 conn(0x7f2b700fbee0 msgr2=0x7f2b700fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:06.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.623+0000 7f2b74b92640 1 -- 172.21.15.152:0/1565246205 shutdown_connections 2024-09-11T22:56:06.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.623+0000 7f2b74b92640 1 -- 172.21.15.152:0/1565246205 wait complete. 2024-09-11T22:56:06.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.624+0000 7f2b74b92640 1 Processor -- start 2024-09-11T22:56:06.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.624+0000 7f2b74b92640 1 -- start start 2024-09-11T22:56:06.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.624+0000 7f2b74b92640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b70101220 0x7f2b701992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.624+0000 7f2b74b92640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b70101ba0 0x7f2b70199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.624+0000 7f2b74b92640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b70105a70 0x7f2b701a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.624+0000 7f2b74b92640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2b70110f50 con 0x7f2b70101220 2024-09-11T22:56:06.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.624+0000 7f2b74b92640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2b70110dd0 con 0x7f2b70101ba0 2024-09-11T22:56:06.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.624+0000 7f2b74b92640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2b701110d0 con 0x7f2b70105a70 2024-09-11T22:56:06.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.624+0000 7f2b6e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b70101220 0x7f2b701992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.624+0000 7f2b6e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b70101220 0x7f2b701992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48482/0 (socket says 172.21.15.152:48482) 2024-09-11T22:56:06.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.624+0000 7f2b6e59c640 1 -- 172.21.15.152:0/3146285853 learned_addr learned my addr 172.21.15.152:0/3146285853 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:06.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.625+0000 7f2b6dd9b640 1 --2- 172.21.15.152:0/3146285853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b70101ba0 0x7f2b70199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.625+0000 7f2b6e59c640 1 -- 172.21.15.152:0/3146285853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b70105a70 msgr2=0x7f2b701a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:06.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.625+0000 7f2b6e59c640 1 --2- 172.21.15.152:0/3146285853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b70105a70 0x7f2b701a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.625+0000 7f2b6e59c640 1 -- 172.21.15.152:0/3146285853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b70101ba0 msgr2=0x7f2b70199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:06.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.625+0000 7f2b6e59c640 1 --2- 172.21.15.152:0/3146285853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b70101ba0 0x7f2b70199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.625+0000 7f2b6e59c640 1 -- 172.21.15.152:0/3146285853 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b701a48b0 con 0x7f2b70101220 2024-09-11T22:56:06.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.625+0000 7f2b6dd9b640 1 --2- 172.21.15.152:0/3146285853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b70101ba0 0x7f2b70199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:06.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.625+0000 7f2b6e59c640 1 --2- 172.21.15.152:0/3146285853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b70101220 0x7f2b701992f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000c920 tx=0x7f2b6000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:06.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.625+0000 7f2b5f7fe640 1 -- 172.21.15.152:0/3146285853 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60007c70 con 0x7f2b70101220 2024-09-11T22:56:06.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.625+0000 7f2b5f7fe640 1 -- 172.21.15.152:0/3146285853 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b60007e10 con 0x7f2b70101220 2024-09-11T22:56:06.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.625+0000 7f2b74b92640 1 -- 172.21.15.152:0/3146285853 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b701a4a80 con 0x7f2b70101220 2024-09-11T22:56:06.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.625+0000 7f2b5f7fe640 1 -- 172.21.15.152:0/3146285853 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60018890 con 0x7f2b70101220 2024-09-11T22:56:06.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.625+0000 7f2b74b92640 1 -- 172.21.15.152:0/3146285853 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b701a4f90 con 0x7f2b70101220 2024-09-11T22:56:06.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.627+0000 7f2b5f7fe640 1 -- 172.21.15.152:0/3146285853 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2b60018a70 con 0x7f2b70101220 2024-09-11T22:56:06.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.628+0000 7f2b76f6b740 1 Processor -- start 2024-09-11T22:56:06.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.628+0000 7f2b76f6b740 1 -- start start 2024-09-11T22:56:06.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.628+0000 7f2b76f6b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 0x555f570dede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.628+0000 7f2b76f6b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570df320 0x555f570e96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.628+0000 7f2b76f6b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570e9be0 0x555f570ebfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.628+0000 7f2b76f6b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x555f57043200 con 0x555f570df320 2024-09-11T22:56:06.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.628+0000 7f2b76f6b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555f56fe9dc0 con 0x555f570dea00 2024-09-11T22:56:06.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.628+0000 7f2b76f6b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555f5703f1c0 con 0x555f570e9be0 2024-09-11T22:56:06.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.628+0000 7f2b5f7fe640 1 --2- 172.21.15.152:0/3146285853 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f2b580a2cf0 0x7f2b580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.628+0000 7f2b6e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570df320 0x555f570e96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.628+0000 7f2b6e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570df320 0x555f570e96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48484/0 (socket says 172.21.15.152:48484) 2024-09-11T22:56:06.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.628+0000 7f2b6e59c640 1 -- 172.21.15.152:0/2319300374 learned_addr learned my addr 172.21.15.152:0/2319300374 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:06.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.628+0000 7f2b6ed9d640 1 --2- 172.21.15.152:0/3146285853 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f2b580a2cf0 0x7f2b580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.628+0000 7f2b6dd9b640 1 --2- 172.21.15.152:0/2319300374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570e9be0 0x555f570ebfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.628+0000 7f2b6ed9d640 1 --2- 172.21.15.152:0/2319300374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 0x555f570dede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.629+0000 7f2b5f7fe640 1 -- 172.21.15.152:0/3146285853 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f2b600cf580 con 0x7f2b70101220 2024-09-11T22:56:06.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.629+0000 7f2b6e59c640 1 -- 172.21.15.152:0/2319300374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570e9be0 msgr2=0x555f570ebfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:06.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.629+0000 7f2b6e59c640 1 --2- 172.21.15.152:0/2319300374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570e9be0 0x555f570ebfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.629+0000 7f2b6e59c640 1 -- 172.21.15.152:0/2319300374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 msgr2=0x555f570dede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:06.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.629+0000 7f2b6e59c640 1 --2- 172.21.15.152:0/2319300374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 0x555f570dede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.629+0000 7f2b6e59c640 1 -- 172.21.15.152:0/2319300374 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f56db4620 con 0x555f570df320 2024-09-11T22:56:06.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.629+0000 7f2b6ed9d640 1 --2- 172.21.15.152:0/3146285853 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f2b580a2cf0 0x7f2b580a51b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2b68013970 tx=0x7f2b68009b10 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:06.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.629+0000 7f2b6e59c640 1 --2- 172.21.15.152:0/2319300374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570df320 0x555f570e96a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000fdd0 tx=0x7f2b60000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e03c505faeb62f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:06.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.629+0000 7f2b5cff9640 1 -- 172.21.15.152:0/2319300374 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6009fea0 con 0x555f570df320 2024-09-11T22:56:06.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.629+0000 7f2b5cff9640 1 -- 172.21.15.152:0/2319300374 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b60066040 con 0x555f570df320 2024-09-11T22:56:06.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.629+0000 7f2b5cff9640 1 -- 172.21.15.152:0/2319300374 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b600a42d0 con 0x555f570df320 2024-09-11T22:56:06.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.630+0000 7f2b76f6b740 1 -- 172.21.15.152:0/2319300374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570df320 msgr2=0x555f570e96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:06.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.630+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/2319300374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570df320 0x555f570e96a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000fdd0 tx=0x7f2b60000bc0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.630+0000 7f2b76f6b740 1 -- 172.21.15.152:0/2319300374 shutdown_connections 2024-09-11T22:56:06.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.630+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/2319300374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570e9be0 0x555f570ebfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.630+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/2319300374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570df320 0x555f570e96a0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.630+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/2319300374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 0x555f570dede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.630+0000 7f2b76f6b740 1 -- 172.21.15.152:0/2319300374 >> 172.21.15.152:0/2319300374 conn(0x555f570daf90 msgr2=0x555f570db370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:06.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.630+0000 7f2b76f6b740 1 -- 172.21.15.152:0/2319300374 shutdown_connections 2024-09-11T22:56:06.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.630+0000 7f2b76f6b740 1 -- 172.21.15.152:0/2319300374 wait complete. 2024-09-11T22:56:06.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.630+0000 7f2b76f6b740 1 -- start start 2024-09-11T22:56:06.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b76f6b740 1 Processor -- start 2024-09-11T22:56:06.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b76f6b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570dea00 0x555f570e9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b76f6b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570df320 0x555f570e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b76f6b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570e9be0 0x555f570e6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b76f6b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x555f570ec7b0 con 0x555f570dea00 2024-09-11T22:56:06.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b76f6b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555f56fddc50 con 0x555f570df320 2024-09-11T22:56:06.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b76f6b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555f570ec930 con 0x555f570e9be0 2024-09-11T22:56:06.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b76f6b740 10 client.0 _ll_register_callbacks cb 0x555f56fefba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:06.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b6e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570dea00 0x555f570e9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b6e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570dea00 0x555f570e9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48490/0 (socket says 172.21.15.152:48490) 2024-09-11T22:56:06.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b6e59c640 1 -- 172.21.15.152:0/62816544 learned_addr learned my addr 172.21.15.152:0/62816544 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:06.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b6ed9d640 1 --2- 172.21.15.152:0/62816544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570e9be0 0x555f570e6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b6dd9b640 1 --2- 172.21.15.152:0/62816544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570df320 0x555f570e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b6e59c640 1 -- 172.21.15.152:0/62816544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570e9be0 msgr2=0x555f570e6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:06.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b6e59c640 1 --2- 172.21.15.152:0/62816544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570e9be0 0x555f570e6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b6e59c640 1 -- 172.21.15.152:0/62816544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570df320 msgr2=0x555f570e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:06.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b6e59c640 1 --2- 172.21.15.152:0/62816544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570df320 0x555f570e67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b6e59c640 1 -- 172.21.15.152:0/62816544 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f570e7570 con 0x555f570dea00 2024-09-11T22:56:06.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b6e59c640 1 --2- 172.21.15.152:0/62816544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570dea00 0x555f570e9530 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2b60097f50 tx=0x7f2b6009c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700d746c4678e80c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:06.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.631+0000 7f2b55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:06.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.632+0000 7f2b55ffb640 1 -- 172.21.15.152:0/62816544 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b600a4760 con 0x555f570dea00 2024-09-11T22:56:06.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.632+0000 7f2b76f6b740 1 -- 172.21.15.152:0/62816544 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b60066040 con 0x555f570dea00 2024-09-11T22:56:06.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.632+0000 7f2b55ffb640 1 -- 172.21.15.152:0/62816544 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f2b600cf2d0 con 0x555f570dea00 2024-09-11T22:56:06.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.632+0000 7f2b55ffb640 1 -- 172.21.15.152:0/62816544 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b60018d60 con 0x555f570dea00 2024-09-11T22:56:06.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.632+0000 7f2b76f6b740 10 client.16404 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:06.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.632+0000 7f2b76f6b740 1 -- 172.21.15.152:0/62816544 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f570f8390 con 0x555f570dea00 2024-09-11T22:56:06.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.632+0000 7f2b55ffb640 1 -- 172.21.15.152:0/62816544 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b60066040 con 0x555f570dea00 2024-09-11T22:56:06.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.632+0000 7f2b55ffb640 1 -- 172.21.15.152:0/62816544 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2b6009af40 con 0x555f570dea00 2024-09-11T22:56:06.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.632+0000 7f2b76f6b740 10 client.16404 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:06.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.632+0000 7f2b76f6b740 10 client.16404 resolve_mds: validated gid 24455 aka daemon mds.b 2024-09-11T22:56:06.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.632+0000 7f2b76f6b740 20 client.16404 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:06.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.632+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/62816544 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555f570f96e0 0x555f570fbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.632+0000 7f2b76f6b740 4 client.16404 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:56:06.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.632+0000 7f2b76f6b740 1 -- 172.21.15.152:0/62816544 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f57050770 con 0x555f570f96e0 2024-09-11T22:56:06.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.632+0000 7f2b6dd9b640 1 --2- 172.21.15.152:0/62816544 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555f570f96e0 0x555f570fbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.633+0000 7f2b55ffb640 10 client.16404 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:06.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.633+0000 7f2b6dd9b640 1 --2- 172.21.15.152:0/62816544 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555f570f96e0 0x555f570fbad0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e65624c2a9cf70a server_cookie=d62c0ab7723a6bb8 in_seq=0 out_seq=0 2024-09-11T22:56:06.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.633+0000 7f2b55ffb640 1 -- 172.21.15.152:0/62816544 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f57050770 con 0x555f570f96e0 2024-09-11T22:56:06.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.633+0000 7f2b55ffb640 10 client.16404 handle_command_reply: tid=0 2024-09-11T22:56:06.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.634+0000 7f2b76f6b740 1 client.16404 shutdown 2024-09-11T22:56:06.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.634+0000 7f2b76f6b740 1 -- 172.21.15.152:0/62816544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570dea00 msgr2=0x555f570e9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:06.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.634+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/62816544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570dea00 0x555f570e9530 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2b60097f50 tx=0x7f2b6009c360 comp rx=0 tx=0).stop 2024-09-11T22:56:06.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.634+0000 7f2b76f6b740 1 -- 172.21.15.152:0/62816544 shutdown_connections 2024-09-11T22:56:06.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.634+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/62816544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570e9be0 0x555f570e6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.634+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/62816544 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555f570f96e0 0x555f570fbad0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.634+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/62816544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570df320 0x555f570e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.634+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/62816544 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570dea00 0x555f570e9530 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.634+0000 7f2b76f6b740 1 -- 172.21.15.152:0/62816544 >> 172.21.15.152:0/62816544 conn(0x555f570daf90 msgr2=0x555f570da1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:06.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.634+0000 7f2b55ffb640 0 client.16404 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:06.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.634+0000 7f2b76f6b740 1 -- 172.21.15.152:0/62816544 shutdown_connections 2024-09-11T22:56:06.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.634+0000 7f2b76f6b740 1 -- 172.21.15.152:0/62816544 wait complete. 2024-09-11T22:56:06.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.634+0000 7f2b76f6b740 20 client.16404 trim_cache size 0 max 16384 2024-09-11T22:56:06.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.651+0000 7f2b76f6b740 1 Processor -- start 2024-09-11T22:56:06.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.651+0000 7f2b76f6b740 1 -- start start 2024-09-11T22:56:06.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.651+0000 7f2b76f6b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 0x555f570f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.651+0000 7f2b76f6b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570df320 0x555f570e3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.651+0000 7f2b76f6b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570e9be0 0x555f570e3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.651+0000 7f2b76f6b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2b701ad7e0 con 0x555f570e9be0 2024-09-11T22:56:06.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.651+0000 7f2b76f6b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2b701af320 con 0x555f570dea00 2024-09-11T22:56:06.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.651+0000 7f2b76f6b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2b701adeb0 con 0x555f570df320 2024-09-11T22:56:06.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.651+0000 7f2b6ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570e9be0 0x555f570e3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.651+0000 7f2b6ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570e9be0 0x555f570e3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48506/0 (socket says 172.21.15.152:48506) 2024-09-11T22:56:06.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.651+0000 7f2b6ed9d640 1 -- 172.21.15.152:0/981588656 learned_addr learned my addr 172.21.15.152:0/981588656 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:06.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.652+0000 7f2b6e59c640 1 --2- 172.21.15.152:0/981588656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 0x555f570f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.652+0000 7f2b6dd9b640 1 --2- 172.21.15.152:0/981588656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570df320 0x555f570e3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.652+0000 7f2b6ed9d640 1 -- 172.21.15.152:0/981588656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570df320 msgr2=0x555f570e3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:06.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.652+0000 7f2b6ed9d640 1 --2- 172.21.15.152:0/981588656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570df320 0x555f570e3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.652+0000 7f2b6ed9d640 1 -- 172.21.15.152:0/981588656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 msgr2=0x555f570f5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:06.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.652+0000 7f2b6ed9d640 1 --2- 172.21.15.152:0/981588656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 0x555f570f5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.652+0000 7f2b6ed9d640 1 -- 172.21.15.152:0/981588656 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f570e70b0 con 0x555f570e9be0 2024-09-11T22:56:06.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.652+0000 7f2b6e59c640 1 --2- 172.21.15.152:0/981588656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 0x555f570f5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:56:06.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.652+0000 7f2b6ed9d640 1 --2- 172.21.15.152:0/981588656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570e9be0 0x555f570e3f80 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2b680097e0 tx=0x7f2b6801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c508555c3b45ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:06.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.652+0000 7f2b57fff640 1 -- 172.21.15.152:0/981588656 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68011cd0 con 0x555f570e9be0 2024-09-11T22:56:06.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.652+0000 7f2b57fff640 1 -- 172.21.15.152:0/981588656 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b68011e70 con 0x555f570e9be0 2024-09-11T22:56:06.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.652+0000 7f2b57fff640 1 -- 172.21.15.152:0/981588656 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6801fd40 con 0x555f570e9be0 2024-09-11T22:56:06.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.653+0000 7f2b76f6b740 1 -- 172.21.15.152:0/981588656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570e9be0 msgr2=0x555f570e3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:06.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.653+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/981588656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570e9be0 0x555f570e3f80 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2b680097e0 tx=0x7f2b6801b000 comp rx=0 tx=0).stop 2024-09-11T22:56:06.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.653+0000 7f2b76f6b740 1 -- 172.21.15.152:0/981588656 shutdown_connections 2024-09-11T22:56:06.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.653+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/981588656 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570e9be0 0x555f570e3f80 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.653+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/981588656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570df320 0x555f570e3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.653+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/981588656 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 0x555f570f5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.653+0000 7f2b76f6b740 1 -- 172.21.15.152:0/981588656 >> 172.21.15.152:0/981588656 conn(0x555f570daf90 msgr2=0x555f570e8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:06.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.653+0000 7f2b76f6b740 1 -- 172.21.15.152:0/981588656 shutdown_connections 2024-09-11T22:56:06.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.653+0000 7f2b76f6b740 1 -- 172.21.15.152:0/981588656 wait complete. 2024-09-11T22:56:06.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.653+0000 7f2b76f6b740 1 -- start start 2024-09-11T22:56:06.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.653+0000 7f2b76f6b740 1 Processor -- start 2024-09-11T22:56:06.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b76f6b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 0x555f5715d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b76f6b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570df320 0x555f5715d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b76f6b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570e9be0 0x555f570fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b76f6b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2b701aec50 con 0x555f570e9be0 2024-09-11T22:56:06.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b76f6b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2b701ae580 con 0x555f570dea00 2024-09-11T22:56:06.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b76f6b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2b701ab7a0 con 0x555f570df320 2024-09-11T22:56:06.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b76f6b740 10 client.0 _ll_register_callbacks cb 0x555f570e7740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:06.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b6ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570e9be0 0x555f570fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b6dd9b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570df320 0x555f5715d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b6ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570e9be0 0x555f570fdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48522/0 (socket says 172.21.15.152:48522) 2024-09-11T22:56:06.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b6ed9d640 1 -- 172.21.15.152:0/4132326157 learned_addr learned my addr 172.21.15.152:0/4132326157 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:06.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b6e59c640 1 --2- 172.21.15.152:0/4132326157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 0x555f5715d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b6ed9d640 1 -- 172.21.15.152:0/4132326157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570df320 msgr2=0x555f5715d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:06.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b6ed9d640 1 --2- 172.21.15.152:0/4132326157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570df320 0x555f5715d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b6ed9d640 1 -- 172.21.15.152:0/4132326157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 msgr2=0x555f5715d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:06.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b6ed9d640 1 --2- 172.21.15.152:0/4132326157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 0x555f5715d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b6ed9d640 1 -- 172.21.15.152:0/4132326157 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f570fe340 con 0x555f570e9be0 2024-09-11T22:56:06.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b6e59c640 1 --2- 172.21.15.152:0/4132326157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 0x555f5715d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:56:06.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b6dd9b640 1 --2- 172.21.15.152:0/4132326157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570df320 0x555f5715d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:56:06.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b6ed9d640 1 --2- 172.21.15.152:0/4132326157 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570e9be0 0x555f570fdc00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2b680099a0 tx=0x7f2b68005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d971515dd76ba5d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:06.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:06.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.654+0000 7f2b55ffb640 1 -- 172.21.15.152:0/4132326157 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6801f440 con 0x555f570e9be0 2024-09-11T22:56:06.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.655+0000 7f2b55ffb640 1 -- 172.21.15.152:0/4132326157 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f2b6802e070 con 0x555f570e9be0 2024-09-11T22:56:06.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.655+0000 7f2b76f6b740 1 -- 172.21.15.152:0/4132326157 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b68011e70 con 0x555f570e9be0 2024-09-11T22:56:06.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.655+0000 7f2b55ffb640 1 -- 172.21.15.152:0/4132326157 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b6801f5e0 con 0x555f570e9be0 2024-09-11T22:56:06.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.655+0000 7f2b76f6b740 10 client.16410 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:06.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.655+0000 7f2b76f6b740 1 -- 172.21.15.152:0/4132326157 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555f5715c9d0 con 0x555f570e9be0 2024-09-11T22:56:06.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.655+0000 7f2b55ffb640 1 -- 172.21.15.152:0/4132326157 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b68011e70 con 0x555f570e9be0 2024-09-11T22:56:06.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.655+0000 7f2b55ffb640 1 -- 172.21.15.152:0/4132326157 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2b6801d040 con 0x555f570e9be0 2024-09-11T22:56:06.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.655+0000 7f2b76f6b740 10 client.16410 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:06.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.655+0000 7f2b76f6b740 10 client.16410 resolve_mds: validated gid 24455 aka daemon mds.b 2024-09-11T22:56:06.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.655+0000 7f2b76f6b740 20 client.16410 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:06.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.655+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/4132326157 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555f57168560 0x555f5716a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:06.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.655+0000 7f2b76f6b740 4 client.16410 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-11T22:56:06.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.655+0000 7f2b76f6b740 1 -- 172.21.15.152:0/4132326157 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x555f5704ac80 con 0x555f57168560 2024-09-11T22:56:06.666 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.655+0000 7f2b6e59c640 1 --2- 172.21.15.152:0/4132326157 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555f57168560 0x555f5716a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:06.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.656+0000 7f2b55ffb640 10 client.16410 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:06.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.656+0000 7f2b6e59c640 1 --2- 172.21.15.152:0/4132326157 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555f57168560 0x555f5716a980 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5a627f2087b2978 server_cookie=903948d200d4e22f in_seq=0 out_seq=0 2024-09-11T22:56:06.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.656+0000 7f2b55ffb640 1 -- 172.21.15.152:0/4132326157 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x555f5704ac80 con 0x555f57168560 2024-09-11T22:56:06.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.656+0000 7f2b55ffb640 10 client.16410 handle_command_reply: tid=0 2024-09-11T22:56:06.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.656+0000 7f2b76f6b740 1 client.16410 shutdown 2024-09-11T22:56:06.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.656+0000 7f2b76f6b740 1 -- 172.21.15.152:0/4132326157 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570e9be0 msgr2=0x555f570fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:06.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.656+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/4132326157 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570e9be0 0x555f570fdc00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f2b680099a0 tx=0x7f2b68005e40 comp rx=0 tx=0).stop 2024-09-11T22:56:06.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.656+0000 7f2b76f6b740 1 -- 172.21.15.152:0/4132326157 shutdown_connections 2024-09-11T22:56:06.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.656+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/4132326157 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555f570e9be0 0x555f570fdc00 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.656+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/4132326157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555f570df320 0x555f5715d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.656+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/4132326157 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555f57168560 0x555f5716a980 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.656+0000 7f2b55ffb640 0 client.16410 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:06.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.656+0000 7f2b76f6b740 1 --2- 172.21.15.152:0/4132326157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555f570dea00 0x555f5715d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.656+0000 7f2b76f6b740 1 -- 172.21.15.152:0/4132326157 >> 172.21.15.152:0/4132326157 conn(0x555f570daf90 msgr2=0x555f570e4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:06.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.657+0000 7f2b76f6b740 1 -- 172.21.15.152:0/4132326157 shutdown_connections 2024-09-11T22:56:06.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.657+0000 7f2b76f6b740 1 -- 172.21.15.152:0/4132326157 wait complete. 2024-09-11T22:56:06.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.657+0000 7f2b76f6b740 20 client.16410 trim_cache size 0 max 16384 2024-09-11T22:56:06.668 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:56:06.668 INFO:teuthology.orchestra.run.smithi152.stdout: "epoch": 28, 2024-09-11T22:56:06.668 INFO:teuthology.orchestra.run.smithi152.stdout: "leader": 24455, 2024-09-11T22:56:06.668 INFO:teuthology.orchestra.run.smithi152.stdout: "set_version": 12, 2024-09-11T22:56:06.668 INFO:teuthology.orchestra.run.smithi152.stdout: "sets": {} 2024-09-11T22:56:06.668 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:56:06.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.657+0000 7f2b74b92640 1 -- 172.21.15.152:0/3146285853 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f2b580a2cf0 msgr2=0x7f2b580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:06.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.657+0000 7f2b74b92640 1 --2- 172.21.15.152:0/3146285853 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f2b580a2cf0 0x7f2b580a51b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2b68013970 tx=0x7f2b68009b10 comp rx=0 tx=0).stop 2024-09-11T22:56:06.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.657+0000 7f2b74b92640 1 -- 172.21.15.152:0/3146285853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b70101220 msgr2=0x7f2b701992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:06.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.657+0000 7f2b74b92640 1 --2- 172.21.15.152:0/3146285853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b70101220 0x7f2b701992f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000c920 tx=0x7f2b6000cde0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.657+0000 7f2b74b92640 1 -- 172.21.15.152:0/3146285853 shutdown_connections 2024-09-11T22:56:06.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.657+0000 7f2b74b92640 1 --2- 172.21.15.152:0/3146285853 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f2b580a2cf0 0x7f2b580a51b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.657+0000 7f2b74b92640 1 --2- 172.21.15.152:0/3146285853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b70105a70 0x7f2b701a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.657+0000 7f2b74b92640 1 --2- 172.21.15.152:0/3146285853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2b70101ba0 0x7f2b70199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.657+0000 7f2b74b92640 1 --2- 172.21.15.152:0/3146285853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2b70101220 0x7f2b701992f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:06.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.657+0000 7f2b74b92640 1 -- 172.21.15.152:0/3146285853 >> 172.21.15.152:0/3146285853 conn(0x7f2b700fbee0 msgr2=0x7f2b700fc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:06.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.657+0000 7f2b74b92640 1 -- 172.21.15.152:0/3146285853 shutdown_connections 2024-09-11T22:56:06.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:06.657+0000 7f2b74b92640 1 -- 172.21.15.152:0/3146285853 wait complete. 2024-09-11T22:56:06.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:06 smithi191 ceph-mon[35211]: pgmap v99: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-09-11T22:56:06.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:06 smithi204 ceph-mon[35454]: pgmap v99: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-09-11T22:56:07.537 INFO:tasks.quiescer:done joining Quiescers 2024-09-11T22:56:07.537 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-11T22:56:07.547 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi152.front.sepia.ceph.com 2024-09-11T22:56:07.556 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-11T22:56:07.583 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-11T22:56:07.583 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T22:56:07.583 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi152.front.sepia.ceph.com 2024-09-11T22:56:07.583 DEBUG:teuthology.orchestra.run.smithi152:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-11T22:56:07.628 DEBUG:teuthology.orchestra.run.smithi152:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-11T22:56:08.705 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:08 smithi152 ceph-mon[31181]: pgmap v100: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-11T22:56:08.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:08 smithi191 ceph-mon[35211]: pgmap v100: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-11T22:56:08.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:08 smithi204 ceph-mon[35454]: pgmap v100: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-11T22:56:09.440 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:56:09.692 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:56:10.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.206+0000 7fb9e405c640 1 -- 172.21.15.152:0/1441289201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc1027a0 msgr2=0x7fb9dc1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:10.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.206+0000 7fb9e405c640 1 --2- 172.21.15.152:0/1441289201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc1027a0 0x7fb9dc1067f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009900 tx=0x7fb9cc02a410 comp rx=0 tx=0).stop 2024-09-11T22:56:10.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.206+0000 7fb9e405c640 1 -- 172.21.15.152:0/1441289201 shutdown_connections 2024-09-11T22:56:10.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.206+0000 7fb9e405c640 1 --2- 172.21.15.152:0/1441289201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9dc106f20 0x7fb9dc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.206+0000 7fb9e405c640 1 --2- 172.21.15.152:0/1441289201 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc1027a0 0x7fb9dc1067f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.206+0000 7fb9e405c640 1 --2- 172.21.15.152:0/1441289201 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9dc101df0 0x7fb9dc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.206+0000 7fb9e405c640 1 -- 172.21.15.152:0/1441289201 >> 172.21.15.152:0/1441289201 conn(0x7fb9dc0fd8b0 msgr2=0x7fb9dc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:10.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.207+0000 7fb9e405c640 1 -- 172.21.15.152:0/1441289201 shutdown_connections 2024-09-11T22:56:10.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.207+0000 7fb9e405c640 1 -- 172.21.15.152:0/1441289201 wait complete. 2024-09-11T22:56:10.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.207+0000 7fb9e405c640 1 Processor -- start 2024-09-11T22:56:10.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.207+0000 7fb9e405c640 1 -- start start 2024-09-11T22:56:10.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.207+0000 7fb9e405c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9dc101df0 0x7fb9dc198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.207+0000 7fb9e405c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9dc1027a0 0x7fb9dc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.207+0000 7fb9e405c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc106f20 0x7fb9dc19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.207+0000 7fb9e405c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc111f80 con 0x7fb9dc106f20 2024-09-11T22:56:10.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.207+0000 7fb9e405c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc111e00 con 0x7fb9dc101df0 2024-09-11T22:56:10.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.207+0000 7fb9e405c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc112100 con 0x7fb9dc1027a0 2024-09-11T22:56:10.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.207+0000 7fb9e25f9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc106f20 0x7fb9dc19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9e25f9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc106f20 0x7fb9dc19ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56738/0 (socket says 172.21.15.152:56738) 2024-09-11T22:56:10.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9e25f9640 1 -- 172.21.15.152:0/3359083352 learned_addr learned my addr 172.21.15.152:0/3359083352 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:10.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9e1df8640 1 --2- 172.21.15.152:0/3359083352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9dc101df0 0x7fb9dc198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9e15f7640 1 --2- 172.21.15.152:0/3359083352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9dc1027a0 0x7fb9dc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9e25f9640 1 -- 172.21.15.152:0/3359083352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9dc1027a0 msgr2=0x7fb9dc198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:10.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9e25f9640 1 --2- 172.21.15.152:0/3359083352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9dc1027a0 0x7fb9dc198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9e25f9640 1 -- 172.21.15.152:0/3359083352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9dc101df0 msgr2=0x7fb9dc198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:10.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9e25f9640 1 --2- 172.21.15.152:0/3359083352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9dc101df0 0x7fb9dc198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9e25f9640 1 -- 172.21.15.152:0/3359083352 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9dc19d3d0 con 0x7fb9dc106f20 2024-09-11T22:56:10.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9e15f7640 1 --2- 172.21.15.152:0/3359083352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9dc1027a0 0x7fb9dc198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:10.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9e1df8640 1 --2- 172.21.15.152:0/3359083352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9dc101df0 0x7fb9dc198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:56:10.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9e25f9640 1 --2- 172.21.15.152:0/3359083352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc106f20 0x7fb9dc19ccf0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400eaa0 tx=0x7fb9d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:10.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9d2ffd640 1 -- 172.21.15.152:0/3359083352 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d400cdb0 con 0x7fb9dc106f20 2024-09-11T22:56:10.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9d2ffd640 1 -- 172.21.15.152:0/3359083352 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d4005020 con 0x7fb9dc106f20 2024-09-11T22:56:10.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9e405c640 1 -- 172.21.15.152:0/3359083352 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9dc19d6c0 con 0x7fb9dc106f20 2024-09-11T22:56:10.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9d2ffd640 1 -- 172.21.15.152:0/3359083352 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d40116b0 con 0x7fb9dc106f20 2024-09-11T22:56:10.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.208+0000 7fb9e405c640 1 -- 172.21.15.152:0/3359083352 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9dc1a4fa0 con 0x7fb9dc106f20 2024-09-11T22:56:10.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.211+0000 7fb9d2ffd640 1 -- 172.21.15.152:0/3359083352 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb9d4005560 con 0x7fb9dc106f20 2024-09-11T22:56:10.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.211+0000 7fb9e6435740 1 Processor -- start 2024-09-11T22:56:10.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.211+0000 7fb9e6435740 1 -- start start 2024-09-11T22:56:10.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.211+0000 7fb9e6435740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8efd360 0x5565e8efd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.211+0000 7fb9e6435740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efdc80 0x5565e8f0c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.211+0000 7fb9e6435740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8f0ca90 0x5565e8f0ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.211+0000 7fb9e6435740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5565e8e632e0 con 0x5565e8efdc80 2024-09-11T22:56:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.211+0000 7fb9e6435740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5565e8e0ace0 con 0x5565e8efd360 2024-09-11T22:56:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.211+0000 7fb9e6435740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5565e8e5f2a0 con 0x5565e8f0ca90 2024-09-11T22:56:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.211+0000 7fb9d2ffd640 1 --2- 172.21.15.152:0/3359083352 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9c80a2d70 0x7fb9c80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.211+0000 7fb9e1df8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efdc80 0x5565e8f0c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.211+0000 7fb9e1df8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efdc80 0x5565e8f0c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56754/0 (socket says 172.21.15.152:56754) 2024-09-11T22:56:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.211+0000 7fb9e1df8640 1 -- 172.21.15.152:0/2982987702 learned_addr learned my addr 172.21.15.152:0/2982987702 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.212+0000 7fb9e15f7640 1 --2- 172.21.15.152:0/2982987702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8efd360 0x5565e8efd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.212+0000 7fb9e25f9640 1 --2- 172.21.15.152:0/3359083352 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9c80a2d70 0x7fb9c80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.212+0000 7fb9e15f7640 1 --2- 172.21.15.152:0/2982987702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8f0ca90 0x5565e8f0ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.212+0000 7fb9e1df8640 1 -- 172.21.15.152:0/2982987702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8f0ca90 msgr2=0x5565e8f0ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.212+0000 7fb9e1df8640 1 --2- 172.21.15.152:0/2982987702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8f0ca90 0x5565e8f0ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.212+0000 7fb9e1df8640 1 -- 172.21.15.152:0/2982987702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8efd360 msgr2=0x5565e8efd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.212+0000 7fb9e1df8640 1 --2- 172.21.15.152:0/2982987702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8efd360 0x5565e8efd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.212+0000 7fb9d2ffd640 1 -- 172.21.15.152:0/3359083352 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb9d40cf5e0 con 0x7fb9dc106f20 2024-09-11T22:56:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.212+0000 7fb9e1df8640 1 -- 172.21.15.152:0/2982987702 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e8def440 con 0x5565e8efdc80 2024-09-11T22:56:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.212+0000 7fb9e1df8640 1 --2- 172.21.15.152:0/2982987702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efdc80 0x5565e8f0c550 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8008c60 tx=0x7fb9d800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5206aa291f3552a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.212+0000 7fb9c7fff640 1 -- 172.21.15.152:0/2982987702 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8014070 con 0x5565e8efdc80 2024-09-11T22:56:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.212+0000 7fb9e25f9640 1 --2- 172.21.15.152:0/3359083352 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9c80a2d70 0x7fb9c80a5230 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb9d409b850 tx=0x7fb9d4090fe0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.212+0000 7fb9c7fff640 1 -- 172.21.15.152:0/2982987702 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d80047a0 con 0x5565e8efdc80 2024-09-11T22:56:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.212+0000 7fb9c7fff640 1 -- 172.21.15.152:0/2982987702 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8010760 con 0x5565e8efdc80 2024-09-11T22:56:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.213+0000 7fb9e6435740 1 -- 172.21.15.152:0/2982987702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efdc80 msgr2=0x5565e8f0c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.213+0000 7fb9e6435740 1 --2- 172.21.15.152:0/2982987702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efdc80 0x5565e8f0c550 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8008c60 tx=0x7fb9d800e720 comp rx=0 tx=0).stop 2024-09-11T22:56:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.213+0000 7fb9e6435740 1 -- 172.21.15.152:0/2982987702 shutdown_connections 2024-09-11T22:56:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.213+0000 7fb9e6435740 1 --2- 172.21.15.152:0/2982987702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8f0ca90 0x5565e8f0ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.213+0000 7fb9e6435740 1 --2- 172.21.15.152:0/2982987702 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efdc80 0x5565e8f0c550 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.213+0000 7fb9e6435740 1 --2- 172.21.15.152:0/2982987702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8efd360 0x5565e8efd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.213+0000 7fb9e6435740 1 -- 172.21.15.152:0/2982987702 >> 172.21.15.152:0/2982987702 conn(0x5565e8ef9a30 msgr2=0x5565e8ef9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.213+0000 7fb9e6435740 1 -- 172.21.15.152:0/2982987702 shutdown_connections 2024-09-11T22:56:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.213+0000 7fb9e6435740 1 -- 172.21.15.152:0/2982987702 wait complete. 2024-09-11T22:56:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.213+0000 7fb9e6435740 1 -- start start 2024-09-11T22:56:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.213+0000 7fb9e6435740 1 Processor -- start 2024-09-11T22:56:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e6435740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efd360 0x5565e8f0c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e6435740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8efdc80 0x5565e8f096b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e6435740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8f0ca90 0x5565e8f09c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e6435740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5565e8f0f660 con 0x5565e8efd360 2024-09-11T22:56:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e6435740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5565e8e1e050 con 0x5565e8f0ca90 2024-09-11T22:56:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e6435740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5565e8f0f7e0 con 0x5565e8efdc80 2024-09-11T22:56:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e6435740 10 client.0 _ll_register_callbacks cb 0x5565e8e5e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e15f7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efd360 0x5565e8f0c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e15f7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efd360 0x5565e8f0c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56770/0 (socket says 172.21.15.152:56770) 2024-09-11T22:56:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e15f7640 1 -- 172.21.15.152:0/324602200 learned_addr learned my addr 172.21.15.152:0/324602200 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e1df8640 1 --2- 172.21.15.152:0/324602200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8efdc80 0x5565e8f096b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e15f7640 1 -- 172.21.15.152:0/324602200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8efdc80 msgr2=0x5565e8f096b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e15f7640 1 --2- 172.21.15.152:0/324602200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8efdc80 0x5565e8f096b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e15f7640 1 -- 172.21.15.152:0/324602200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8f0ca90 msgr2=0x5565e8f09c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e15f7640 1 --2- 172.21.15.152:0/324602200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8f0ca90 0x5565e8f09c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e15f7640 1 -- 172.21.15.152:0/324602200 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e8bd43c0 con 0x5565e8efd360 2024-09-11T22:56:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9e15f7640 1 --2- 172.21.15.152:0/324602200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efd360 0x5565e8f0c400 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009900 tx=0x7fb9cc0314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340ed86c250a8bea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.214+0000 7fb9c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.215+0000 7fb9c57fa640 1 -- 172.21.15.152:0/324602200 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc044070 con 0x5565e8efd360 2024-09-11T22:56:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.215+0000 7fb9c57fa640 1 -- 172.21.15.152:0/324602200 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb9cc004450 con 0x5565e8efd360 2024-09-11T22:56:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.215+0000 7fb9e6435740 1 -- 172.21.15.152:0/324602200 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9d80047a0 con 0x5565e8efd360 2024-09-11T22:56:10.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.215+0000 7fb9c57fa640 1 -- 172.21.15.152:0/324602200 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb9cc050050 con 0x5565e8efd360 2024-09-11T22:56:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.215+0000 7fb9e6435740 10 client.16422 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.215+0000 7fb9e6435740 1 -- 172.21.15.152:0/324602200 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565e8f1b070 con 0x5565e8efd360 2024-09-11T22:56:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.215+0000 7fb9c57fa640 1 -- 172.21.15.152:0/324602200 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d80047a0 con 0x5565e8efd360 2024-09-11T22:56:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.215+0000 7fb9c57fa640 1 -- 172.21.15.152:0/324602200 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb9cc03a040 con 0x5565e8efd360 2024-09-11T22:56:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.215+0000 7fb9e6435740 10 client.16422 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.215+0000 7fb9e6435740 10 client.16422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:56:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.215+0000 7fb9e6435740 20 client.16422 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.215+0000 7fb9e6435740 1 --2- 172.21.15.152:0/324602200 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5565e8f1c480 0x5565e8f1e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.215+0000 7fb9e6435740 4 client.16422 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:56:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.215+0000 7fb9e6435740 1 -- 172.21.15.152:0/324602200 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565e8e70850 con 0x5565e8f1c480 2024-09-11T22:56:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.216+0000 7fb9e1df8640 1 --2- 172.21.15.152:0/324602200 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5565e8f1c480 0x5565e8f1e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.216+0000 7fb9c57fa640 10 client.16422 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.216+0000 7fb9e1df8640 1 --2- 172.21.15.152:0/324602200 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5565e8f1c480 0x5565e8f1e8b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=593c9ba24f2a869b server_cookie=4b169a3fc5786879 in_seq=0 out_seq=0 2024-09-11T22:56:10.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.217+0000 7fb9c57fa640 1 -- 172.21.15.152:0/324602200 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565e8e70850 con 0x5565e8f1c480 2024-09-11T22:56:10.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.217+0000 7fb9c57fa640 10 client.16422 handle_command_reply: tid=0 2024-09-11T22:56:10.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.217+0000 7fb9e6435740 1 client.16422 shutdown 2024-09-11T22:56:10.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.217+0000 7fb9e6435740 1 -- 172.21.15.152:0/324602200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efd360 msgr2=0x5565e8f0c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:10.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.217+0000 7fb9e6435740 1 --2- 172.21.15.152:0/324602200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efd360 0x5565e8f0c400 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009900 tx=0x7fb9cc0314e0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.217+0000 7fb9e6435740 1 -- 172.21.15.152:0/324602200 shutdown_connections 2024-09-11T22:56:10.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.217+0000 7fb9e6435740 1 --2- 172.21.15.152:0/324602200 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5565e8f1c480 0x5565e8f1e8b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.217+0000 7fb9e6435740 1 --2- 172.21.15.152:0/324602200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8f0ca90 0x5565e8f09c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.217+0000 7fb9e6435740 1 --2- 172.21.15.152:0/324602200 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8efdc80 0x5565e8f096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.217+0000 7fb9c57fa640 0 client.16422 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:10.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.217+0000 7fb9e6435740 1 --2- 172.21.15.152:0/324602200 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efd360 0x5565e8f0c400 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.217+0000 7fb9e6435740 1 -- 172.21.15.152:0/324602200 >> 172.21.15.152:0/324602200 conn(0x5565e8ef9a30 msgr2=0x5565e8eff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:10.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.217+0000 7fb9e6435740 1 -- 172.21.15.152:0/324602200 shutdown_connections 2024-09-11T22:56:10.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.217+0000 7fb9e6435740 1 -- 172.21.15.152:0/324602200 wait complete. 2024-09-11T22:56:10.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.217+0000 7fb9e6435740 20 client.16422 trim_cache size 0 max 16384 2024-09-11T22:56:10.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.234+0000 7fb9e6435740 1 Processor -- start 2024-09-11T22:56:10.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.234+0000 7fb9e6435740 1 -- start start 2024-09-11T22:56:10.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.234+0000 7fb9e6435740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8efd360 0x5565e8f18900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.234+0000 7fb9e6435740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efdc80 0x5565e8f24180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9e6435740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8f0ca90 0x5565e8f246c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.247 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9e6435740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc1ab630 con 0x5565e8efdc80 2024-09-11T22:56:10.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9e6435740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc1ad170 con 0x5565e8f0ca90 2024-09-11T22:56:10.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9e6435740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc1abd00 con 0x5565e8efd360 2024-09-11T22:56:10.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9e1df8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efdc80 0x5565e8f24180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9e1df8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efdc80 0x5565e8f24180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56782/0 (socket says 172.21.15.152:56782) 2024-09-11T22:56:10.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9e1df8640 1 -- 172.21.15.152:0/3519962296 learned_addr learned my addr 172.21.15.152:0/3519962296 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:10.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9e15f7640 1 --2- 172.21.15.152:0/3519962296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8efd360 0x5565e8f18900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9e15f7640 1 --2- 172.21.15.152:0/3519962296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8f0ca90 0x5565e8f246c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9e1df8640 1 -- 172.21.15.152:0/3519962296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8efd360 msgr2=0x5565e8f18900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:10.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9e1df8640 1 --2- 172.21.15.152:0/3519962296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8efd360 0x5565e8f18900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9e1df8640 1 -- 172.21.15.152:0/3519962296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8f0ca90 msgr2=0x5565e8f246c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:10.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9e1df8640 1 --2- 172.21.15.152:0/3519962296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8f0ca90 0x5565e8f246c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9e1df8640 1 -- 172.21.15.152:0/3519962296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e8efe4b0 con 0x5565e8efdc80 2024-09-11T22:56:10.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9e15f7640 1 --2- 172.21.15.152:0/3519962296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8f0ca90 0x5565e8f246c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:10.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9e1df8640 1 --2- 172.21.15.152:0/3519962296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efdc80 0x5565e8f24180 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8000be0 tx=0x7fb9d8008550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b48b6eb78a0b665 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:10.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9c77fe640 1 -- 172.21.15.152:0/3519962296 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8014040 con 0x5565e8efdc80 2024-09-11T22:56:10.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9c77fe640 1 -- 172.21.15.152:0/3519962296 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d8010910 con 0x5565e8efdc80 2024-09-11T22:56:10.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.235+0000 7fb9c77fe640 1 -- 172.21.15.152:0/3519962296 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8024830 con 0x5565e8efdc80 2024-09-11T22:56:10.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.236+0000 7fb9e6435740 1 -- 172.21.15.152:0/3519962296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efdc80 msgr2=0x5565e8f24180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:10.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.236+0000 7fb9e6435740 1 --2- 172.21.15.152:0/3519962296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efdc80 0x5565e8f24180 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8000be0 tx=0x7fb9d8008550 comp rx=0 tx=0).stop 2024-09-11T22:56:10.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.236+0000 7fb9e6435740 1 -- 172.21.15.152:0/3519962296 shutdown_connections 2024-09-11T22:56:10.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.236+0000 7fb9e6435740 1 --2- 172.21.15.152:0/3519962296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8f0ca90 0x5565e8f246c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.236+0000 7fb9e6435740 1 --2- 172.21.15.152:0/3519962296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8efdc80 0x5565e8f24180 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.236+0000 7fb9e6435740 1 --2- 172.21.15.152:0/3519962296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8efd360 0x5565e8f18900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.236+0000 7fb9e6435740 1 -- 172.21.15.152:0/3519962296 >> 172.21.15.152:0/3519962296 conn(0x5565e8ef9a30 msgr2=0x5565e8f04d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:10.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.236+0000 7fb9e6435740 1 -- 172.21.15.152:0/3519962296 shutdown_connections 2024-09-11T22:56:10.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.236+0000 7fb9e6435740 1 -- 172.21.15.152:0/3519962296 wait complete. 2024-09-11T22:56:10.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e6435740 1 -- start start 2024-09-11T22:56:10.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e6435740 1 Processor -- start 2024-09-11T22:56:10.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e6435740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8efd360 0x5565e8f06720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e6435740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8efdc80 0x5565e8f06c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e6435740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8f0ca90 0x5565e8f071a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e6435740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc1acaa0 con 0x5565e8f0ca90 2024-09-11T22:56:10.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e6435740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc1ac3d0 con 0x5565e8efd360 2024-09-11T22:56:10.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e6435740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb9dc1a95f0 con 0x5565e8efdc80 2024-09-11T22:56:10.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e6435740 10 client.0 _ll_register_callbacks cb 0x7fb9cc031680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:10.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e15f7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8f0ca90 0x5565e8f071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e15f7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8f0ca90 0x5565e8f071a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56784/0 (socket says 172.21.15.152:56784) 2024-09-11T22:56:10.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e15f7640 1 -- 172.21.15.152:0/3733628700 learned_addr learned my addr 172.21.15.152:0/3733628700 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:10.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e15f7640 1 --2- 172.21.15.152:0/3733628700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8efd360 0x5565e8f06720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e1df8640 1 --2- 172.21.15.152:0/3733628700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8efdc80 0x5565e8f06c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e15f7640 1 -- 172.21.15.152:0/3733628700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8efdc80 msgr2=0x5565e8f06c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:10.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e15f7640 1 --2- 172.21.15.152:0/3733628700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8efdc80 0x5565e8f06c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e15f7640 1 -- 172.21.15.152:0/3733628700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8efd360 msgr2=0x5565e8f06720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:10.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e15f7640 1 --2- 172.21.15.152:0/3733628700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8efd360 0x5565e8f06720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.237+0000 7fb9e15f7640 1 -- 172.21.15.152:0/3733628700 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e8f861a0 con 0x5565e8f0ca90 2024-09-11T22:56:10.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.238+0000 7fb9e15f7640 1 --2- 172.21.15.152:0/3733628700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8f0ca90 0x5565e8f071a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x5565e8f06030 tx=0x7fb9cc0314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a728857a0e07c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:10.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.238+0000 7fb9c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:10.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.238+0000 7fb9c57fa640 1 -- 172.21.15.152:0/3733628700 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc044020 con 0x5565e8f0ca90 2024-09-11T22:56:10.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.238+0000 7fb9c57fa640 1 -- 172.21.15.152:0/3733628700 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb9cc03d990 con 0x5565e8f0ca90 2024-09-11T22:56:10.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.238+0000 7fb9e6435740 1 -- 172.21.15.152:0/3733628700 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9d8010910 con 0x5565e8f0ca90 2024-09-11T22:56:10.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.238+0000 7fb9c57fa640 1 -- 172.21.15.152:0/3733628700 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb9cc035040 con 0x5565e8f0ca90 2024-09-11T22:56:10.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.238+0000 7fb9e6435740 10 client.16428 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:10.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.238+0000 7fb9e6435740 1 -- 172.21.15.152:0/3733628700 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565e8f87480 con 0x5565e8f0ca90 2024-09-11T22:56:10.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.238+0000 7fb9c57fa640 1 -- 172.21.15.152:0/3733628700 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d8010910 con 0x5565e8f0ca90 2024-09-11T22:56:10.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.238+0000 7fb9c57fa640 1 -- 172.21.15.152:0/3733628700 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb9cc03a070 con 0x5565e8f0ca90 2024-09-11T22:56:10.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.238+0000 7fb9e6435740 10 client.16428 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:10.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.238+0000 7fb9e6435740 10 client.16428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:56:10.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.238+0000 7fb9e6435740 20 client.16428 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:10.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.238+0000 7fb9e6435740 1 --2- 172.21.15.152:0/3733628700 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5565e8f87c00 0x5565e8f8a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:10.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.238+0000 7fb9e6435740 4 client.16428 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:56:10.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.238+0000 7fb9e6435740 1 -- 172.21.15.152:0/3733628700 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5565e8e6ad60 con 0x5565e8f87c00 2024-09-11T22:56:10.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.239+0000 7fb9e1df8640 1 --2- 172.21.15.152:0/3733628700 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5565e8f87c00 0x5565e8f8a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:10.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.239+0000 7fb9e1df8640 1 --2- 172.21.15.152:0/3733628700 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5565e8f87c00 0x5565e8f8a020 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b833b21d33195f2a server_cookie=a35bf6e847f0cbd1 in_seq=0 out_seq=0 2024-09-11T22:56:10.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.239+0000 7fb9c57fa640 10 client.16428 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:10.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.239+0000 7fb9c57fa640 1 -- 172.21.15.152:0/3733628700 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5565e8e6ad60 con 0x5565e8f87c00 2024-09-11T22:56:10.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.239+0000 7fb9c57fa640 10 client.16428 handle_command_reply: tid=0 2024-09-11T22:56:10.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.239+0000 7fb9e6435740 1 client.16428 shutdown 2024-09-11T22:56:10.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e6435740 1 -- 172.21.15.152:0/3733628700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8f0ca90 msgr2=0x5565e8f071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:10.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e6435740 1 --2- 172.21.15.152:0/3733628700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8f0ca90 0x5565e8f071a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x5565e8f06030 tx=0x7fb9cc0314e0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e6435740 1 -- 172.21.15.152:0/3733628700 shutdown_connections 2024-09-11T22:56:10.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e6435740 1 --2- 172.21.15.152:0/3733628700 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565e8f0ca90 0x5565e8f071a0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e6435740 1 --2- 172.21.15.152:0/3733628700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565e8efdc80 0x5565e8f06c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e6435740 1 --2- 172.21.15.152:0/3733628700 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5565e8f87c00 0x5565e8f8a020 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e6435740 1 --2- 172.21.15.152:0/3733628700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565e8efd360 0x5565e8f06720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9c57fa640 0 client.16428 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:10.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e6435740 1 -- 172.21.15.152:0/3733628700 >> 172.21.15.152:0/3733628700 conn(0x5565e8ef9a30 msgr2=0x5565e8ef8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:10.253 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e6435740 1 -- 172.21.15.152:0/3733628700 shutdown_connections 2024-09-11T22:56:10.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e6435740 1 -- 172.21.15.152:0/3733628700 wait complete. 2024-09-11T22:56:10.254 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e6435740 20 client.16428 trim_cache size 0 max 16384 2024-09-11T22:56:10.254 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:56:10.254 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T22:56:10.254 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T22:56:10.254 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:56:10.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e405c640 1 -- 172.21.15.152:0/3359083352 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9c80a2d70 msgr2=0x7fb9c80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:10.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e405c640 1 --2- 172.21.15.152:0/3359083352 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9c80a2d70 0x7fb9c80a5230 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb9d409b850 tx=0x7fb9d4090fe0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e405c640 1 -- 172.21.15.152:0/3359083352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc106f20 msgr2=0x7fb9dc19ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:10.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e405c640 1 --2- 172.21.15.152:0/3359083352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc106f20 0x7fb9dc19ccf0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400eaa0 tx=0x7fb9d400ef60 comp rx=0 tx=0).stop 2024-09-11T22:56:10.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e405c640 1 -- 172.21.15.152:0/3359083352 shutdown_connections 2024-09-11T22:56:10.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e405c640 1 --2- 172.21.15.152:0/3359083352 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9c80a2d70 0x7fb9c80a5230 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.255 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e405c640 1 --2- 172.21.15.152:0/3359083352 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc106f20 0x7fb9dc19ccf0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e405c640 1 --2- 172.21.15.152:0/3359083352 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9dc1027a0 0x7fb9dc198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e405c640 1 --2- 172.21.15.152:0/3359083352 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb9dc101df0 0x7fb9dc198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:10.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.240+0000 7fb9e405c640 1 -- 172.21.15.152:0/3359083352 >> 172.21.15.152:0/3359083352 conn(0x7fb9dc0fd8b0 msgr2=0x7fb9dc0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:10.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.241+0000 7fb9e405c640 1 -- 172.21.15.152:0/3359083352 shutdown_connections 2024-09-11T22:56:10.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:10.241+0000 7fb9e405c640 1 -- 172.21.15.152:0/3359083352 wait complete. 2024-09-11T22:56:10.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:10 smithi191 ceph-mon[35211]: pgmap v101: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-09-11T22:56:10.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:10 smithi204 ceph-mon[35454]: pgmap v101: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-09-11T22:56:10.832 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:10 smithi152 ceph-mon[31181]: pgmap v101: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-09-11T22:56:11.083 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:56:11.083 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:56:11.334 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:56:11.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.830+0000 7f0636368640 1 -- 172.21.15.152:0/1430628664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0630101bc0 msgr2=0x7f063010d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:11.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.830+0000 7f0636368640 1 --2- 172.21.15.152:0/1430628664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0630101bc0 0x7f063010d910 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0628009ca0 tx=0x7f062802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:56:11.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.830+0000 7f0636368640 1 -- 172.21.15.152:0/1430628664 shutdown_connections 2024-09-11T22:56:11.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.830+0000 7f0636368640 1 --2- 172.21.15.152:0/1430628664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0630101bc0 0x7f063010d910 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:11.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.830+0000 7f0636368640 1 --2- 172.21.15.152:0/1430628664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0630101240 0x7f0630101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:11.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.830+0000 7f0636368640 1 --2- 172.21.15.152:0/1430628664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0630105a90 0x7f0630105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:11.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.830+0000 7f0636368640 1 -- 172.21.15.152:0/1430628664 >> 172.21.15.152:0/1430628664 conn(0x7f0630077210 msgr2=0x7f06300fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:11.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.830+0000 7f0636368640 1 -- 172.21.15.152:0/1430628664 shutdown_connections 2024-09-11T22:56:11.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.830+0000 7f0636368640 1 -- 172.21.15.152:0/1430628664 wait complete. 2024-09-11T22:56:11.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.830+0000 7f0636368640 1 Processor -- start 2024-09-11T22:56:11.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.830+0000 7f0636368640 1 -- start start 2024-09-11T22:56:11.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.830+0000 7f0636368640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0630101240 0x7f063019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:11.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0636368640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0630101bc0 0x7f063019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:11.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0636368640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0630105a90 0x7f063019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:11.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0636368640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0630110f70 con 0x7f0630101bc0 2024-09-11T22:56:11.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0636368640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0630110df0 con 0x7f0630101240 2024-09-11T22:56:11.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0636368640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f06301110f0 con 0x7f0630105a90 2024-09-11T22:56:11.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0627fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0630101bc0 0x7f063019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:11.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0627fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0630101bc0 0x7f063019ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56794/0 (socket says 172.21.15.152:56794) 2024-09-11T22:56:11.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0627fff640 1 -- 172.21.15.152:0/475673774 learned_addr learned my addr 172.21.15.152:0/475673774 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:11.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f062ffff640 1 --2- 172.21.15.152:0/475673774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0630101240 0x7f063019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:11.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0634905640 1 --2- 172.21.15.152:0/475673774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0630105a90 0x7f063019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:11.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0627fff640 1 -- 172.21.15.152:0/475673774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0630105a90 msgr2=0x7f063019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:11.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0627fff640 1 --2- 172.21.15.152:0/475673774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0630105a90 0x7f063019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:11.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0627fff640 1 -- 172.21.15.152:0/475673774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0630101240 msgr2=0x7f063019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:11.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0627fff640 1 --2- 172.21.15.152:0/475673774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0630101240 0x7f063019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:11.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0627fff640 1 -- 172.21.15.152:0/475673774 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063019f680 con 0x7f0630101bc0 2024-09-11T22:56:11.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0627fff640 1 --2- 172.21.15.152:0/475673774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0630101bc0 0x7f063019ac10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f061800c970 tx=0x7f061800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:11.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f062dffb640 1 -- 172.21.15.152:0/475673774 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618007bb0 con 0x7f0630101bc0 2024-09-11T22:56:11.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0636368640 1 -- 172.21.15.152:0/475673774 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063019f970 con 0x7f0630101bc0 2024-09-11T22:56:11.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f062dffb640 1 -- 172.21.15.152:0/475673774 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0618007d50 con 0x7f0630101bc0 2024-09-11T22:56:11.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f062dffb640 1 -- 172.21.15.152:0/475673774 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061800f450 con 0x7f0630101bc0 2024-09-11T22:56:11.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.831+0000 7f0636368640 1 -- 172.21.15.152:0/475673774 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06301a7250 con 0x7f0630101bc0 2024-09-11T22:56:11.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.832+0000 7f0636368640 1 -- 172.21.15.152:0/475673774 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0630101680 con 0x7f0630101bc0 2024-09-11T22:56:11.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.833+0000 7f062dffb640 1 -- 172.21.15.152:0/475673774 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0618005020 con 0x7f0630101bc0 2024-09-11T22:56:11.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.834+0000 7f062dffb640 1 --2- 172.21.15.152:0/475673774 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f061c0a2cf0 0x7f061c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:11.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.834+0000 7f062ffff640 1 --2- 172.21.15.152:0/475673774 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f061c0a2cf0 0x7f061c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:11.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.834+0000 7f062dffb640 1 -- 172.21.15.152:0/475673774 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f06180cf290 con 0x7f0630101bc0 2024-09-11T22:56:11.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.835+0000 7f062ffff640 1 --2- 172.21.15.152:0/475673774 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f061c0a2cf0 0x7f061c0a51b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0620005d90 tx=0x7f0620005d20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:11.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:11.838+0000 7f062dffb640 1 -- 172.21.15.152:0/475673774 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0618014030 con 0x7f0630101bc0 2024-09-11T22:56:12.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:12.038+0000 7f0636368640 1 -- 172.21.15.152:0/475673774 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06300fdd00 con 0x7f0630101bc0 2024-09-11T22:56:12.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:12.039+0000 7f062dffb640 1 -- 172.21.15.152:0/475673774 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f061809bf00 con 0x7f0630101bc0 2024-09-11T22:56:12.041 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:56:12.041 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:56:12.041 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:56:12.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:12.041+0000 7f0636368640 1 -- 172.21.15.152:0/475673774 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f061c0a2cf0 msgr2=0x7f061c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:12.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:12.041+0000 7f0636368640 1 --2- 172.21.15.152:0/475673774 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f061c0a2cf0 0x7f061c0a51b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0620005d90 tx=0x7f0620005d20 comp rx=0 tx=0).stop 2024-09-11T22:56:12.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:12.041+0000 7f0636368640 1 -- 172.21.15.152:0/475673774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0630101bc0 msgr2=0x7f063019ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:12.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:12.041+0000 7f0636368640 1 --2- 172.21.15.152:0/475673774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0630101bc0 0x7f063019ac10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f061800c970 tx=0x7f061800ce30 comp rx=0 tx=0).stop 2024-09-11T22:56:12.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:12.042+0000 7f0636368640 1 -- 172.21.15.152:0/475673774 shutdown_connections 2024-09-11T22:56:12.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:12.042+0000 7f0636368640 1 --2- 172.21.15.152:0/475673774 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f061c0a2cf0 0x7f061c0a51b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:12.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:12.042+0000 7f0636368640 1 --2- 172.21.15.152:0/475673774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0630105a90 0x7f063019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:12.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:12.042+0000 7f0636368640 1 --2- 172.21.15.152:0/475673774 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0630101bc0 0x7f063019ac10 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:12.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:12.042+0000 7f0636368640 1 --2- 172.21.15.152:0/475673774 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0630101240 0x7f063019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:12.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:12.042+0000 7f0636368640 1 -- 172.21.15.152:0/475673774 >> 172.21.15.152:0/475673774 conn(0x7f0630077210 msgr2=0x7f0630104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:12.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:12.042+0000 7f0636368640 1 -- 172.21.15.152:0/475673774 shutdown_connections 2024-09-11T22:56:12.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:12.042+0000 7f0636368640 1 -- 172.21.15.152:0/475673774 wait complete. 2024-09-11T22:56:12.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:12 smithi191 ceph-mon[35211]: pgmap v102: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-11T22:56:12.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:12 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/475673774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:56:12.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:12 smithi204 ceph-mon[35454]: pgmap v102: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-11T22:56:12.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:12 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/475673774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:56:12.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:12 smithi152 ceph-mon[31181]: pgmap v102: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-09-11T22:56:12.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:12 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/475673774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:56:12.959 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:56:13.211 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:56:13.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.732+0000 7f3dcd495640 1 -- 172.21.15.152:0/484268026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc8100e20 msgr2=0x7f3dc8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:13.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.732+0000 7f3dcd495640 1 --2- 172.21.15.152:0/484268026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc8100e20 0x7f3dc8101200 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009ca0 tx=0x7f3db802a950 comp rx=0 tx=0).stop 2024-09-11T22:56:13.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.732+0000 7f3dcd495640 1 -- 172.21.15.152:0/484268026 shutdown_connections 2024-09-11T22:56:13.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.732+0000 7f3dcd495640 1 --2- 172.21.15.152:0/484268026 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dc8102ef0 0x7f3dc810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.732+0000 7f3dcd495640 1 --2- 172.21.15.152:0/484268026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dc8102570 0x7f3dc81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.732+0000 7f3dcd495640 1 --2- 172.21.15.152:0/484268026 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc8100e20 0x7f3dc8101200 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.732+0000 7f3dcd495640 1 -- 172.21.15.152:0/484268026 >> 172.21.15.152:0/484268026 conn(0x7f3dc80fc7d0 msgr2=0x7f3dc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:13.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.732+0000 7f3dcd495640 1 -- 172.21.15.152:0/484268026 shutdown_connections 2024-09-11T22:56:13.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.732+0000 7f3dcd495640 1 -- 172.21.15.152:0/484268026 wait complete. 2024-09-11T22:56:13.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.733+0000 7f3dcd495640 1 Processor -- start 2024-09-11T22:56:13.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.733+0000 7f3dcd495640 1 -- start start 2024-09-11T22:56:13.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.733+0000 7f3dcd495640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc8100e20 0x7f3dc81970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.733+0000 7f3dcd495640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dc8102570 0x7f3dc8197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.733+0000 7f3dcd495640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dc8102ef0 0x7f3dc8197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.733+0000 7f3dcd495640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8110f80 con 0x7f3dc8100e20 2024-09-11T22:56:13.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.733+0000 7f3dcd495640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8110e00 con 0x7f3dc8102ef0 2024-09-11T22:56:13.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.733+0000 7f3dcd495640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8111100 con 0x7f3dc8102570 2024-09-11T22:56:13.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dc6ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc8100e20 0x7f3dc81970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dc6ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc8100e20 0x7f3dc81970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56806/0 (socket says 172.21.15.152:56806) 2024-09-11T22:56:13.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dc6ffd640 1 -- 172.21.15.152:0/2654307444 learned_addr learned my addr 172.21.15.152:0/2654307444 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:13.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dc77fe640 1 --2- 172.21.15.152:0/2654307444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dc8102ef0 0x7f3dc8197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dc67fc640 1 --2- 172.21.15.152:0/2654307444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dc8102570 0x7f3dc8197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dc6ffd640 1 -- 172.21.15.152:0/2654307444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dc8102570 msgr2=0x7f3dc8197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:13.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dc6ffd640 1 --2- 172.21.15.152:0/2654307444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dc8102570 0x7f3dc8197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dc6ffd640 1 -- 172.21.15.152:0/2654307444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dc8102ef0 msgr2=0x7f3dc8197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:13.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dc6ffd640 1 --2- 172.21.15.152:0/2654307444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dc8102ef0 0x7f3dc8197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dc6ffd640 1 -- 172.21.15.152:0/2654307444 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc81a6930 con 0x7f3dc8100e20 2024-09-11T22:56:13.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dc67fc640 1 --2- 172.21.15.152:0/2654307444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dc8102570 0x7f3dc8197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:56:13.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dc6ffd640 1 --2- 172.21.15.152:0/2654307444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc8100e20 0x7f3dc81970c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3db8005cc0 tx=0x7f3db802f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:13.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dbffff640 1 -- 172.21.15.152:0/2654307444 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8034040 con 0x7f3dc8100e20 2024-09-11T22:56:13.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dcd495640 1 -- 172.21.15.152:0/2654307444 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc81a6bc0 con 0x7f3dc8100e20 2024-09-11T22:56:13.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dbffff640 1 -- 172.21.15.152:0/2654307444 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3db8003e40 con 0x7f3dc8100e20 2024-09-11T22:56:13.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dbffff640 1 -- 172.21.15.152:0/2654307444 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db8045480 con 0x7f3dc8100e20 2024-09-11T22:56:13.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.734+0000 7f3dcd495640 1 -- 172.21.15.152:0/2654307444 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc81a70f0 con 0x7f3dc8100e20 2024-09-11T22:56:13.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.736+0000 7f3dbffff640 1 -- 172.21.15.152:0/2654307444 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3db8037020 con 0x7f3dc8100e20 2024-09-11T22:56:13.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.737+0000 7f3dcf86e740 1 Processor -- start 2024-09-11T22:56:13.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.737+0000 7f3dcf86e740 1 -- start start 2024-09-11T22:56:13.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.737+0000 7f3dcf86e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2bb40 0x5555dbf2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.737+0000 7f3dcf86e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf2a5d0 0x5555dbf2aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.737+0000 7f3dcf86e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf35370 0x5555dbf37760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.737+0000 7f3dcf86e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5555dbe902f0 con 0x5555dbf2bb40 2024-09-11T22:56:13.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.737+0000 7f3dcf86e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5555dbe37cf0 con 0x5555dbf2a5d0 2024-09-11T22:56:13.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.737+0000 7f3dcf86e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5555dbe8c2b0 con 0x5555dbf35370 2024-09-11T22:56:13.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.737+0000 7f3dbffff640 1 --2- 172.21.15.152:0/2654307444 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3db40a2cf0 0x7f3db40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.737+0000 7f3dc77fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2bb40 0x5555dbf2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.737+0000 7f3dc77fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2bb40 0x5555dbf2a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56816/0 (socket says 172.21.15.152:56816) 2024-09-11T22:56:13.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.737+0000 7f3dc77fe640 1 -- 172.21.15.152:0/2515226939 learned_addr learned my addr 172.21.15.152:0/2515226939 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:13.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.737+0000 7f3dc77fe640 1 --2- 172.21.15.152:0/2654307444 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3db40a2cf0 0x7f3db40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.738+0000 7f3dc67fc640 1 --2- 172.21.15.152:0/2515226939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf35370 0x5555dbf37760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.738+0000 7f3dc6ffd640 1 --2- 172.21.15.152:0/2515226939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf2a5d0 0x5555dbf2aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.738+0000 7f3dbffff640 1 -- 172.21.15.152:0/2654307444 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f3db80f04d0 con 0x7f3dc8100e20 2024-09-11T22:56:13.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.738+0000 7f3dc77fe640 1 -- 172.21.15.152:0/2515226939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf35370 msgr2=0x5555dbf37760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:13.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.738+0000 7f3dc77fe640 1 --2- 172.21.15.152:0/2515226939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf35370 0x5555dbf37760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.738+0000 7f3dc77fe640 1 -- 172.21.15.152:0/2515226939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf2a5d0 msgr2=0x5555dbf2aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:13.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.738+0000 7f3dc77fe640 1 --2- 172.21.15.152:0/2515226939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf2a5d0 0x5555dbf2aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.738+0000 7f3dc77fe640 1 -- 172.21.15.152:0/2515226939 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555dbe1c3a0 con 0x5555dbf2bb40 2024-09-11T22:56:13.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.738+0000 7f3dc6ffd640 1 --2- 172.21.15.152:0/2515226939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf2a5d0 0x5555dbf2aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:56:13.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.738+0000 7f3dc67fc640 1 --2- 172.21.15.152:0/2515226939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf35370 0x5555dbf37760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:56:13.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.738+0000 7f3dc77fe640 1 --2- 172.21.15.152:0/2515226939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2bb40 0x5555dbf2a090 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00119c0 tx=0x7f3dc0011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2ba63bc4b12e1b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:13.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.738+0000 7f3dc77fe640 1 --2- 172.21.15.152:0/2654307444 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3db40a2cf0 0x7f3db40a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000f790 tx=0x7f3dc001c000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:13.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.738+0000 7f3dbd7fa640 1 -- 172.21.15.152:0/2515226939 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0021070 con 0x5555dbf2bb40 2024-09-11T22:56:13.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.738+0000 7f3dbd7fa640 1 -- 172.21.15.152:0/2515226939 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dc000a960 con 0x5555dbf2bb40 2024-09-11T22:56:13.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.738+0000 7f3dbd7fa640 1 -- 172.21.15.152:0/2515226939 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00056b0 con 0x5555dbf2bb40 2024-09-11T22:56:13.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.739+0000 7f3dcf86e740 1 -- 172.21.15.152:0/2515226939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2bb40 msgr2=0x5555dbf2a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:13.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.739+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/2515226939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2bb40 0x5555dbf2a090 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00119c0 tx=0x7f3dc0011e80 comp rx=0 tx=0).stop 2024-09-11T22:56:13.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.739+0000 7f3dcf86e740 1 -- 172.21.15.152:0/2515226939 shutdown_connections 2024-09-11T22:56:13.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.739+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/2515226939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf35370 0x5555dbf37760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.739+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/2515226939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf2a5d0 0x5555dbf2aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.739+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/2515226939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2bb40 0x5555dbf2a090 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.739+0000 7f3dcf86e740 1 -- 172.21.15.152:0/2515226939 >> 172.21.15.152:0/2515226939 conn(0x5555dbef3310 msgr2=0x5555dbf26c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:13.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.739+0000 7f3dcf86e740 1 -- 172.21.15.152:0/2515226939 shutdown_connections 2024-09-11T22:56:13.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.739+0000 7f3dcf86e740 1 -- 172.21.15.152:0/2515226939 wait complete. 2024-09-11T22:56:13.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.739+0000 7f3dcf86e740 1 -- start start 2024-09-11T22:56:13.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.739+0000 7f3dcf86e740 1 Processor -- start 2024-09-11T22:56:13.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dcf86e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf2a5d0 0x5555dbf34d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dcf86e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2bb40 0x5555dbf31fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dcf86e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf35370 0x5555dbf32570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dcf86e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5555dbf38010 con 0x5555dbf2bb40 2024-09-11T22:56:13.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dcf86e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5555dbe4b060 con 0x5555dbf35370 2024-09-11T22:56:13.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dcf86e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5555dbf38190 con 0x5555dbf2a5d0 2024-09-11T22:56:13.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dcf86e740 10 client.0 _ll_register_callbacks cb 0x5555dbb9e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:13.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dc67fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2bb40 0x5555dbf31fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dc67fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2bb40 0x5555dbf31fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56820/0 (socket says 172.21.15.152:56820) 2024-09-11T22:56:13.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dc67fc640 1 -- 172.21.15.152:0/2893701854 learned_addr learned my addr 172.21.15.152:0/2893701854 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:13.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dc6ffd640 1 --2- 172.21.15.152:0/2893701854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf2a5d0 0x5555dbf34d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dc77fe640 1 --2- 172.21.15.152:0/2893701854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf35370 0x5555dbf32570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dc67fc640 1 -- 172.21.15.152:0/2893701854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf2a5d0 msgr2=0x5555dbf34d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:13.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dc67fc640 1 --2- 172.21.15.152:0/2893701854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf2a5d0 0x5555dbf34d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dc67fc640 1 -- 172.21.15.152:0/2893701854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf35370 msgr2=0x5555dbf32570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:13.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dc67fc640 1 --2- 172.21.15.152:0/2893701854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf35370 0x5555dbf32570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dc67fc640 1 -- 172.21.15.152:0/2893701854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555dbc013c0 con 0x5555dbf2bb40 2024-09-11T22:56:13.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dc67fc640 1 --2- 172.21.15.152:0/2893701854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2bb40 0x5555dbf31fd0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3db0004650 tx=0x7f3db0010bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea136f069b722d9e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:13.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:13.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dae7fc640 1 -- 172.21.15.152:0/2893701854 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db000eb40 con 0x5555dbf2bb40 2024-09-11T22:56:13.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dae7fc640 1 -- 172.21.15.152:0/2893701854 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f3db000ece0 con 0x5555dbf2bb40 2024-09-11T22:56:13.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.740+0000 7f3dcf86e740 1 -- 172.21.15.152:0/2893701854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5555dbe9d860 con 0x5555dbf2bb40 2024-09-11T22:56:13.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.741+0000 7f3dae7fc640 1 -- 172.21.15.152:0/2893701854 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3db0005020 con 0x5555dbf2bb40 2024-09-11T22:56:13.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.741+0000 7f3dcf86e740 10 client.16446 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:13.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.741+0000 7f3dcf86e740 1 -- 172.21.15.152:0/2893701854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555dbf43ac0 con 0x5555dbf2bb40 2024-09-11T22:56:13.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.741+0000 7f3dae7fc640 1 -- 172.21.15.152:0/2893701854 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5555dbe9d860 con 0x5555dbf2bb40 2024-09-11T22:56:13.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.741+0000 7f3dae7fc640 1 -- 172.21.15.152:0/2893701854 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3db0011440 con 0x5555dbf2bb40 2024-09-11T22:56:13.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.741+0000 7f3dcf86e740 10 client.16446 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:13.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.741+0000 7f3dcf86e740 10 client.16446 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:56:13.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.741+0000 7f3dcf86e740 20 client.16446 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:13.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.741+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/2893701854 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5555dbf44f20 0x5555dbf47340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.741+0000 7f3dcf86e740 4 client.16446 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:56:13.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.741+0000 7f3dcf86e740 1 -- 172.21.15.152:0/2893701854 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555dbe97d70 con 0x5555dbf44f20 2024-09-11T22:56:13.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.741+0000 7f3dc6ffd640 1 --2- 172.21.15.152:0/2893701854 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5555dbf44f20 0x5555dbf47340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dc6ffd640 1 --2- 172.21.15.152:0/2893701854 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5555dbf44f20 0x5555dbf47340 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5b5371aa194a7f10 server_cookie=5f2f2fb6a90fd822 in_seq=0 out_seq=0 2024-09-11T22:56:13.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dae7fc640 10 client.16446 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:56:13.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dae7fc640 1 -- 172.21.15.152:0/2893701854 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555dbe97d70 con 0x5555dbf44f20 2024-09-11T22:56:13.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dae7fc640 10 client.16446 handle_command_reply: tid=0 2024-09-11T22:56:13.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dcf86e740 1 client.16446 shutdown 2024-09-11T22:56:13.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dcf86e740 1 -- 172.21.15.152:0/2893701854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2bb40 msgr2=0x5555dbf31fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:13.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/2893701854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2bb40 0x5555dbf31fd0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3db0004650 tx=0x7f3db0010bf0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dcf86e740 1 -- 172.21.15.152:0/2893701854 shutdown_connections 2024-09-11T22:56:13.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/2893701854 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5555dbf44f20 0x5555dbf47340 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/2893701854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf35370 0x5555dbf32570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/2893701854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2bb40 0x5555dbf31fd0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/2893701854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf2a5d0 0x5555dbf34d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dcf86e740 1 -- 172.21.15.152:0/2893701854 >> 172.21.15.152:0/2893701854 conn(0x5555dbef3310 msgr2=0x5555dbf2c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:13.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dae7fc640 0 client.16446 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:56:13.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dcf86e740 1 -- 172.21.15.152:0/2893701854 shutdown_connections 2024-09-11T22:56:13.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dcf86e740 1 -- 172.21.15.152:0/2893701854 wait complete. 2024-09-11T22:56:13.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.742+0000 7f3dcf86e740 20 client.16446 trim_cache size 0 max 16384 2024-09-11T22:56:13.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.755+0000 7f3dcf86e740 1 Processor -- start 2024-09-11T22:56:13.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.755+0000 7f3dcf86e740 1 -- start start 2024-09-11T22:56:13.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dcf86e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2a5d0 0x5555dbf412b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dcf86e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf2bb40 0x5555dbf30fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dcf86e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf35370 0x5555dbf4cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dcf86e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81af900 con 0x5555dbf2a5d0 2024-09-11T22:56:13.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dcf86e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81b1440 con 0x5555dbf35370 2024-09-11T22:56:13.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dcf86e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81affd0 con 0x5555dbf2bb40 2024-09-11T22:56:13.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dc6ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2a5d0 0x5555dbf412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dc6ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2a5d0 0x5555dbf412b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56830/0 (socket says 172.21.15.152:56830) 2024-09-11T22:56:13.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dc6ffd640 1 -- 172.21.15.152:0/3137943992 learned_addr learned my addr 172.21.15.152:0/3137943992 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:13.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dc67fc640 1 --2- 172.21.15.152:0/3137943992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf2bb40 0x5555dbf30fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dc77fe640 1 --2- 172.21.15.152:0/3137943992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf35370 0x5555dbf4cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dc6ffd640 1 -- 172.21.15.152:0/3137943992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf2bb40 msgr2=0x5555dbf30fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:13.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dc6ffd640 1 --2- 172.21.15.152:0/3137943992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf2bb40 0x5555dbf30fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dc6ffd640 1 -- 172.21.15.152:0/3137943992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf35370 msgr2=0x5555dbf4cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:13.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dc6ffd640 1 --2- 172.21.15.152:0/3137943992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf35370 0x5555dbf4cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dc6ffd640 1 -- 172.21.15.152:0/3137943992 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555dbf26ed0 con 0x5555dbf2a5d0 2024-09-11T22:56:13.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dc67fc640 1 --2- 172.21.15.152:0/3137943992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf2bb40 0x5555dbf30fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:56:13.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dc6ffd640 1 --2- 172.21.15.152:0/3137943992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2a5d0 0x5555dbf412b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3db80edda0 tx=0x7f3db803c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac17d46c7099aa94 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:13.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.756+0000 7f3dbcff9640 1 -- 172.21.15.152:0/3137943992 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db80d9020 con 0x5555dbf2a5d0 2024-09-11T22:56:13.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.757+0000 7f3dbcff9640 1 -- 172.21.15.152:0/3137943992 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3db80c5780 con 0x5555dbf2a5d0 2024-09-11T22:56:13.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.757+0000 7f3dbcff9640 1 -- 172.21.15.152:0/3137943992 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db802c040 con 0x5555dbf2a5d0 2024-09-11T22:56:13.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.757+0000 7f3dcf86e740 1 -- 172.21.15.152:0/3137943992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2a5d0 msgr2=0x5555dbf412b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:13.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.757+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/3137943992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2a5d0 0x5555dbf412b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3db80edda0 tx=0x7f3db803c040 comp rx=0 tx=0).stop 2024-09-11T22:56:13.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.757+0000 7f3dcf86e740 1 -- 172.21.15.152:0/3137943992 shutdown_connections 2024-09-11T22:56:13.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.757+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/3137943992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf35370 0x5555dbf4cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.757+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/3137943992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf2bb40 0x5555dbf30fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.757+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/3137943992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2a5d0 0x5555dbf412b0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.757+0000 7f3dcf86e740 1 -- 172.21.15.152:0/3137943992 >> 172.21.15.152:0/3137943992 conn(0x5555dbef3310 msgr2=0x5555dbf308a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:13.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.757+0000 7f3dcf86e740 1 -- 172.21.15.152:0/3137943992 shutdown_connections 2024-09-11T22:56:13.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.757+0000 7f3dcf86e740 1 -- 172.21.15.152:0/3137943992 wait complete. 2024-09-11T22:56:13.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.758+0000 7f3dcf86e740 1 -- start start 2024-09-11T22:56:13.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.758+0000 7f3dcf86e740 1 Processor -- start 2024-09-11T22:56:13.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.758+0000 7f3dcf86e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2a5d0 0x5555dbf3da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.758+0000 7f3dcf86e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf2bb40 0x5555dbf2ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.758+0000 7f3dcf86e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf35370 0x5555dbf2f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.758+0000 7f3dcf86e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81b0d70 con 0x5555dbf2a5d0 2024-09-11T22:56:13.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.758+0000 7f3dcf86e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81b06a0 con 0x5555dbf2bb40 2024-09-11T22:56:13.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.758+0000 7f3dcf86e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81ad8c0 con 0x5555dbf35370 2024-09-11T22:56:13.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.758+0000 7f3dcf86e740 10 client.0 _ll_register_callbacks cb 0x7f3db000c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:13.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.758+0000 7f3dc6ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2a5d0 0x5555dbf3da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.758+0000 7f3dc6ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2a5d0 0x5555dbf3da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56844/0 (socket says 172.21.15.152:56844) 2024-09-11T22:56:13.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.758+0000 7f3dc6ffd640 1 -- 172.21.15.152:0/3803298628 learned_addr learned my addr 172.21.15.152:0/3803298628 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:13.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.758+0000 7f3dc67fc640 1 --2- 172.21.15.152:0/3803298628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf2bb40 0x5555dbf2ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.758+0000 7f3dc77fe640 1 --2- 172.21.15.152:0/3803298628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf35370 0x5555dbf2f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.759+0000 7f3dc6ffd640 1 -- 172.21.15.152:0/3803298628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf35370 msgr2=0x5555dbf2f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:13.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.759+0000 7f3dc6ffd640 1 --2- 172.21.15.152:0/3803298628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf35370 0x5555dbf2f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.759+0000 7f3dc6ffd640 1 -- 172.21.15.152:0/3803298628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf2bb40 msgr2=0x5555dbf2ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:13.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.759+0000 7f3dc6ffd640 1 --2- 172.21.15.152:0/3803298628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf2bb40 0x5555dbf2ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.759+0000 7f3dc6ffd640 1 -- 172.21.15.152:0/3803298628 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555dbf2f7f0 con 0x5555dbf2a5d0 2024-09-11T22:56:13.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.759+0000 7f3dc77fe640 1 --2- 172.21.15.152:0/3803298628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf35370 0x5555dbf2f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:56:13.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.759+0000 7f3dc6ffd640 1 --2- 172.21.15.152:0/3803298628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2a5d0 0x5555dbf3da80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3db8005fa0 tx=0x7f3db802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd7329d4b1caf3e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:13.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.759+0000 7f3dae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:13.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.759+0000 7f3dae7fc640 1 -- 172.21.15.152:0/3803298628 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db80d9070 con 0x5555dbf2a5d0 2024-09-11T22:56:13.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.759+0000 7f3dae7fc640 1 -- 172.21.15.152:0/3803298628 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f3db80db300 con 0x5555dbf2a5d0 2024-09-11T22:56:13.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.759+0000 7f3dcf86e740 1 -- 172.21.15.152:0/3803298628 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db80c5780 con 0x5555dbf2a5d0 2024-09-11T22:56:13.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.759+0000 7f3dae7fc640 1 -- 172.21.15.152:0/3803298628 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3db802c040 con 0x5555dbf2a5d0 2024-09-11T22:56:13.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.759+0000 7f3dcf86e740 10 client.16452 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:13.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.759+0000 7f3dcf86e740 1 -- 172.21.15.152:0/3803298628 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555dbfafa70 con 0x5555dbf2a5d0 2024-09-11T22:56:13.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.759+0000 7f3dae7fc640 1 -- 172.21.15.152:0/3803298628 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3db80c5780 con 0x5555dbf2a5d0 2024-09-11T22:56:13.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dae7fc640 1 -- 172.21.15.152:0/3803298628 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3db804d030 con 0x5555dbf2a5d0 2024-09-11T22:56:13.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dcf86e740 10 client.16452 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:13.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dcf86e740 10 client.16452 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:56:13.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dcf86e740 20 client.16452 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:13.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/3803298628 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5555dbfafef0 0x5555dbfb22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:13.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dcf86e740 4 client.16452 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:56:13.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dcf86e740 1 -- 172.21.15.152:0/3803298628 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5555dbe14170 con 0x5555dbfafef0 2024-09-11T22:56:13.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dc67fc640 1 --2- 172.21.15.152:0/3803298628 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5555dbfafef0 0x5555dbfb22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:13.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dae7fc640 10 client.16452 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:56:13.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dc67fc640 1 --2- 172.21.15.152:0/3803298628 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5555dbfafef0 0x5555dbfb22e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=880e989a4c51419e server_cookie=683ab9982df42118 in_seq=0 out_seq=0 2024-09-11T22:56:13.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dae7fc640 1 -- 172.21.15.152:0/3803298628 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5555dbe14170 con 0x5555dbfafef0 2024-09-11T22:56:13.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dae7fc640 10 client.16452 handle_command_reply: tid=0 2024-09-11T22:56:13.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dcf86e740 1 client.16452 shutdown 2024-09-11T22:56:13.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dcf86e740 1 -- 172.21.15.152:0/3803298628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2a5d0 msgr2=0x5555dbf3da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:13.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/3803298628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2a5d0 0x5555dbf3da80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3db8005fa0 tx=0x7f3db802a950 comp rx=0 tx=0).stop 2024-09-11T22:56:13.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dcf86e740 1 -- 172.21.15.152:0/3803298628 shutdown_connections 2024-09-11T22:56:13.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/3803298628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5555dbf35370 0x5555dbf2f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/3803298628 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5555dbfafef0 0x5555dbfb22e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/3803298628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5555dbf2bb40 0x5555dbf2ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dae7fc640 0 client.16452 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:56:13.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dcf86e740 1 --2- 172.21.15.152:0/3803298628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5555dbf2a5d0 0x5555dbf3da80 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.760+0000 7f3dcf86e740 1 -- 172.21.15.152:0/3803298628 >> 172.21.15.152:0/3803298628 conn(0x5555dbef3310 msgr2=0x5555dbf310c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:13.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.761+0000 7f3dcf86e740 1 -- 172.21.15.152:0/3803298628 shutdown_connections 2024-09-11T22:56:13.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.761+0000 7f3dcf86e740 1 -- 172.21.15.152:0/3803298628 wait complete. 2024-09-11T22:56:13.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.761+0000 7f3dcf86e740 20 client.16452 trim_cache size 0 max 16384 2024-09-11T22:56:13.777 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:56:13.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.761+0000 7f3dcd495640 1 -- 172.21.15.152:0/2654307444 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3db40a2cf0 msgr2=0x7f3db40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:13.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.761+0000 7f3dcd495640 1 --2- 172.21.15.152:0/2654307444 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3db40a2cf0 0x7f3db40a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000f790 tx=0x7f3dc001c000 comp rx=0 tx=0).stop 2024-09-11T22:56:13.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.761+0000 7f3dcd495640 1 -- 172.21.15.152:0/2654307444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc8100e20 msgr2=0x7f3dc81970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:13.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.761+0000 7f3dcd495640 1 --2- 172.21.15.152:0/2654307444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc8100e20 0x7f3dc81970c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3db8005cc0 tx=0x7f3db802f6c0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.761+0000 7f3dcd495640 1 -- 172.21.15.152:0/2654307444 shutdown_connections 2024-09-11T22:56:13.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.761+0000 7f3dcd495640 1 --2- 172.21.15.152:0/2654307444 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3db40a2cf0 0x7f3db40a51b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.761+0000 7f3dcd495640 1 --2- 172.21.15.152:0/2654307444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3dc8102ef0 0x7f3dc8197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.761+0000 7f3dcd495640 1 --2- 172.21.15.152:0/2654307444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dc8102570 0x7f3dc8197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.761+0000 7f3dcd495640 1 --2- 172.21.15.152:0/2654307444 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3dc8100e20 0x7f3dc81970c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:13.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.761+0000 7f3dcd495640 1 -- 172.21.15.152:0/2654307444 >> 172.21.15.152:0/2654307444 conn(0x7f3dc80fc7d0 msgr2=0x7f3dc8106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:13.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.761+0000 7f3dcd495640 1 -- 172.21.15.152:0/2654307444 shutdown_connections 2024-09-11T22:56:13.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:13.761+0000 7f3dcd495640 1 -- 172.21.15.152:0/2654307444 wait complete. 2024-09-11T22:56:14.611 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:56:14.633 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:14 smithi152 ceph-mon[31181]: pgmap v103: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-11T22:56:14.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:14 smithi191 ceph-mon[35211]: pgmap v103: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-11T22:56:14.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:14 smithi204 ceph-mon[35454]: pgmap v103: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-11T22:56:14.859 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:56:15.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.384+0000 7f685acd7640 1 -- 172.21.15.152:0/3262036918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6854101de0 msgr2=0x7f68541021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:15.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.384+0000 7f685acd7640 1 --2- 172.21.15.152:0/3262036918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6854101de0 0x7f68541021c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6848009900 tx=0x7f684802a3f0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.384+0000 7f685acd7640 1 -- 172.21.15.152:0/3262036918 shutdown_connections 2024-09-11T22:56:15.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.384+0000 7f685acd7640 1 --2- 172.21.15.152:0/3262036918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6854106f10 0x7f685410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.384+0000 7f685acd7640 1 --2- 172.21.15.152:0/3262036918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6854102790 0x7f68541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.384+0000 7f685acd7640 1 --2- 172.21.15.152:0/3262036918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6854101de0 0x7f68541021c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.384+0000 7f685acd7640 1 -- 172.21.15.152:0/3262036918 >> 172.21.15.152:0/3262036918 conn(0x7f68540fd8c0 msgr2=0x7f68540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:15.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.384+0000 7f685acd7640 1 -- 172.21.15.152:0/3262036918 shutdown_connections 2024-09-11T22:56:15.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.384+0000 7f685acd7640 1 -- 172.21.15.152:0/3262036918 wait complete. 2024-09-11T22:56:15.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.385+0000 7f685acd7640 1 Processor -- start 2024-09-11T22:56:15.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.385+0000 7f685acd7640 1 -- start start 2024-09-11T22:56:15.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.385+0000 7f685acd7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6854101de0 0x7f68540744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.385+0000 7f685acd7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6854102790 0x7f6854074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.385+0000 7f685acd7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6854106f10 0x7f6854074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.385+0000 7f685acd7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6854111f70 con 0x7f6854101de0 2024-09-11T22:56:15.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.385+0000 7f685acd7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6854111df0 con 0x7f6854102790 2024-09-11T22:56:15.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.385+0000 7f685acd7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f68541120f0 con 0x7f6854106f10 2024-09-11T22:56:15.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.385+0000 7f6858a73640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6854101de0 0x7f68540744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.385+0000 7f6858a73640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6854101de0 0x7f68540744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56864/0 (socket says 172.21.15.152:56864) 2024-09-11T22:56:15.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.385+0000 7f6858a73640 1 -- 172.21.15.152:0/1892856441 learned_addr learned my addr 172.21.15.152:0/1892856441 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:15.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.386+0000 7f6853fff640 1 --2- 172.21.15.152:0/1892856441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6854102790 0x7f6854074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.386+0000 7f6858a73640 1 -- 172.21.15.152:0/1892856441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6854106f10 msgr2=0x7f6854074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:15.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.386+0000 7f6859274640 1 --2- 172.21.15.152:0/1892856441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6854106f10 0x7f6854074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.386+0000 7f6858a73640 1 --2- 172.21.15.152:0/1892856441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6854106f10 0x7f6854074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.386+0000 7f6858a73640 1 -- 172.21.15.152:0/1892856441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6854102790 msgr2=0x7f6854074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:15.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.386+0000 7f6858a73640 1 --2- 172.21.15.152:0/1892856441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6854102790 0x7f6854074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.386+0000 7f6858a73640 1 -- 172.21.15.152:0/1892856441 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68541a6970 con 0x7f6854101de0 2024-09-11T22:56:15.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.386+0000 7f6853fff640 1 --2- 172.21.15.152:0/1892856441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6854102790 0x7f6854074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:15.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.386+0000 7f6859274640 1 --2- 172.21.15.152:0/1892856441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6854106f10 0x7f6854074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:15.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.386+0000 7f6858a73640 1 --2- 172.21.15.152:0/1892856441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6854101de0 0x7f68540744f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6848006010 tx=0x7f68480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:15.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.386+0000 7f6851ffb640 1 -- 172.21.15.152:0/1892856441 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684803f030 con 0x7f6854101de0 2024-09-11T22:56:15.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.386+0000 7f6851ffb640 1 -- 172.21.15.152:0/1892856441 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6848003ce0 con 0x7f6854101de0 2024-09-11T22:56:15.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.386+0000 7f685acd7640 1 -- 172.21.15.152:0/1892856441 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68541a6c00 con 0x7f6854101de0 2024-09-11T22:56:15.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.386+0000 7f6851ffb640 1 -- 172.21.15.152:0/1892856441 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848035070 con 0x7f6854101de0 2024-09-11T22:56:15.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.386+0000 7f685acd7640 1 -- 172.21.15.152:0/1892856441 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68541a70e0 con 0x7f6854101de0 2024-09-11T22:56:15.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.388+0000 7f6851ffb640 1 -- 172.21.15.152:0/1892856441 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6848042030 con 0x7f6854101de0 2024-09-11T22:56:15.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.389+0000 7f685d0b0740 1 Processor -- start 2024-09-11T22:56:15.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.389+0000 7f685d0b0740 1 -- start start 2024-09-11T22:56:15.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.389+0000 7f6851ffb640 1 --2- 172.21.15.152:0/1892856441 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f68440a2cf0 0x7f68440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.389+0000 7f685d0b0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb612b40 0x558ccb611090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.389+0000 7f685d0b0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb6115d0 0x558ccb611a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.389+0000 7f685d0b0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb61c370 0x558ccb61e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.389+0000 7f685d0b0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558ccb5772f0 con 0x558ccb612b40 2024-09-11T22:56:15.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.389+0000 7f685d0b0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558ccb51ecf0 con 0x558ccb61c370 2024-09-11T22:56:15.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.389+0000 7f685d0b0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558ccb5732b0 con 0x558ccb6115d0 2024-09-11T22:56:15.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.389+0000 7f6859274640 1 --2- 172.21.15.152:0/1892856441 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f68440a2cf0 0x7f68440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.389+0000 7f6851ffb640 1 -- 172.21.15.152:0/1892856441 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f68480efa00 con 0x7f6854101de0 2024-09-11T22:56:15.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.389+0000 7f6858a73640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb612b40 0x558ccb611090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.389+0000 7f6858a73640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb612b40 0x558ccb611090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56874/0 (socket says 172.21.15.152:56874) 2024-09-11T22:56:15.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.389+0000 7f6858a73640 1 -- 172.21.15.152:0/144674376 learned_addr learned my addr 172.21.15.152:0/144674376 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:15.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.389+0000 7f6853fff640 1 --2- 172.21.15.152:0/144674376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb6115d0 0x558ccb611a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.390+0000 7f6858a73640 1 -- 172.21.15.152:0/144674376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb6115d0 msgr2=0x558ccb611a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:15.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.390+0000 7f6858a73640 1 --2- 172.21.15.152:0/144674376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb6115d0 0x558ccb611a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.390+0000 7f6858a73640 1 -- 172.21.15.152:0/144674376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb61c370 msgr2=0x558ccb61e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:15.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.390+0000 7f6858a73640 1 --2- 172.21.15.152:0/144674376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb61c370 0x558ccb61e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.390+0000 7f6858a73640 1 -- 172.21.15.152:0/144674376 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ccb5033a0 con 0x558ccb612b40 2024-09-11T22:56:15.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.390+0000 7f6859274640 1 --2- 172.21.15.152:0/1892856441 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f68440a2cf0 0x7f68440a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f684c005fd0 tx=0x7f684c00f730 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:15.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.390+0000 7f6858a73640 1 --2- 172.21.15.152:0/144674376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb612b40 0x558ccb611090 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f68480c4730 tx=0x7f68480bbef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733a196389caffb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:15.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.390+0000 7f683effd640 1 -- 172.21.15.152:0/144674376 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848049070 con 0x558ccb612b40 2024-09-11T22:56:15.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.390+0000 7f683effd640 1 -- 172.21.15.152:0/144674376 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684803b580 con 0x558ccb612b40 2024-09-11T22:56:15.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.390+0000 7f683effd640 1 -- 172.21.15.152:0/144674376 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68480c4c60 con 0x558ccb612b40 2024-09-11T22:56:15.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.391+0000 7f685d0b0740 1 -- 172.21.15.152:0/144674376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb612b40 msgr2=0x558ccb611090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:15.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.391+0000 7f685d0b0740 1 --2- 172.21.15.152:0/144674376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb612b40 0x558ccb611090 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f68480c4730 tx=0x7f68480bbef0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.391+0000 7f685d0b0740 1 -- 172.21.15.152:0/144674376 shutdown_connections 2024-09-11T22:56:15.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.391+0000 7f685d0b0740 1 --2- 172.21.15.152:0/144674376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb61c370 0x558ccb61e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.391+0000 7f685d0b0740 1 --2- 172.21.15.152:0/144674376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb6115d0 0x558ccb611a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.391+0000 7f685d0b0740 1 --2- 172.21.15.152:0/144674376 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb612b40 0x558ccb611090 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.391+0000 7f685d0b0740 1 -- 172.21.15.152:0/144674376 >> 172.21.15.152:0/144674376 conn(0x558ccb5da310 msgr2=0x558ccb60dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:15.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.391+0000 7f685d0b0740 1 -- 172.21.15.152:0/144674376 shutdown_connections 2024-09-11T22:56:15.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.391+0000 7f685d0b0740 1 -- 172.21.15.152:0/144674376 wait complete. 2024-09-11T22:56:15.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.391+0000 7f685d0b0740 1 -- start start 2024-09-11T22:56:15.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.391+0000 7f685d0b0740 1 Processor -- start 2024-09-11T22:56:15.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f685d0b0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb6115d0 0x558ccb61bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f685d0b0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb612b40 0x558ccb618fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f685d0b0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb61c370 0x558ccb619570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f685d0b0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558ccb61ee30 con 0x558ccb6115d0 2024-09-11T22:56:15.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f685d0b0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558ccb532060 con 0x558ccb61c370 2024-09-11T22:56:15.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f685d0b0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558ccb61efb0 con 0x558ccb612b40 2024-09-11T22:56:15.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f685d0b0740 10 client.0 _ll_register_callbacks cb 0x558ccb2858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:15.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f6858a73640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb6115d0 0x558ccb61bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f6858a73640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb6115d0 0x558ccb61bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56886/0 (socket says 172.21.15.152:56886) 2024-09-11T22:56:15.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f6858a73640 1 -- 172.21.15.152:0/3630100954 learned_addr learned my addr 172.21.15.152:0/3630100954 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:15.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f6859274640 1 --2- 172.21.15.152:0/3630100954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb61c370 0x558ccb619570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f6853fff640 1 --2- 172.21.15.152:0/3630100954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb612b40 0x558ccb618fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f6858a73640 1 -- 172.21.15.152:0/3630100954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb612b40 msgr2=0x558ccb618fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:15.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f6858a73640 1 --2- 172.21.15.152:0/3630100954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb612b40 0x558ccb618fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f6858a73640 1 -- 172.21.15.152:0/3630100954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb61c370 msgr2=0x558ccb619570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:15.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f6858a73640 1 --2- 172.21.15.152:0/3630100954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb61c370 0x558ccb619570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f6858a73640 1 -- 172.21.15.152:0/3630100954 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ccb2e83c0 con 0x558ccb6115d0 2024-09-11T22:56:15.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f6859274640 1 --2- 172.21.15.152:0/3630100954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb61c370 0x558ccb619570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:56:15.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f6853fff640 1 --2- 172.21.15.152:0/3630100954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb612b40 0x558ccb618fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:56:15.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f6858a73640 1 --2- 172.21.15.152:0/3630100954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb6115d0 0x558ccb61bd20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6848000c00 tx=0x7f68480044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a901012fdde8a26c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:15.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f6833fff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:15.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f6833fff640 1 -- 172.21.15.152:0/3630100954 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848049030 con 0x558ccb6115d0 2024-09-11T22:56:15.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f6833fff640 1 -- 172.21.15.152:0/3630100954 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f68480c47d0 con 0x558ccb6115d0 2024-09-11T22:56:15.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.392+0000 7f685d0b0740 1 -- 172.21.15.152:0/3630100954 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558ccb584860 con 0x558ccb6115d0 2024-09-11T22:56:15.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.393+0000 7f6833fff640 1 -- 172.21.15.152:0/3630100954 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f68480ed5e0 con 0x558ccb6115d0 2024-09-11T22:56:15.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.393+0000 7f685d0b0740 10 client.16464 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:15.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.393+0000 7f685d0b0740 1 -- 172.21.15.152:0/3630100954 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ccb61ee30 con 0x558ccb6115d0 2024-09-11T22:56:15.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.393+0000 7f6833fff640 1 -- 172.21.15.152:0/3630100954 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558ccb584860 con 0x558ccb6115d0 2024-09-11T22:56:15.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.393+0000 7f6833fff640 1 -- 172.21.15.152:0/3630100954 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f68480dab70 con 0x558ccb6115d0 2024-09-11T22:56:15.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.393+0000 7f685d0b0740 10 client.16464 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:15.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.393+0000 7f685d0b0740 10 client.16464 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:56:15.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.393+0000 7f685d0b0740 20 client.16464 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:15.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.393+0000 7f685d0b0740 1 --2- 172.21.15.152:0/3630100954 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x558ccb62ba40 0x558ccb62de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.393+0000 7f685d0b0740 4 client.16464 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:56:15.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.393+0000 7f685d0b0740 1 -- 172.21.15.152:0/3630100954 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ccb57ed70 con 0x558ccb62ba40 2024-09-11T22:56:15.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.393+0000 7f6853fff640 1 --2- 172.21.15.152:0/3630100954 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x558ccb62ba40 0x558ccb62de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.394+0000 7f6853fff640 1 --2- 172.21.15.152:0/3630100954 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x558ccb62ba40 0x558ccb62de60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=47225693a23bd0b3 server_cookie=9aded5d1cbd6af1b in_seq=0 out_seq=0 2024-09-11T22:56:15.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.394+0000 7f6833fff640 10 client.16464 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:56:15.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.394+0000 7f6833fff640 1 -- 172.21.15.152:0/3630100954 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ccb57ed70 con 0x558ccb62ba40 2024-09-11T22:56:15.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.394+0000 7f6833fff640 10 client.16464 handle_command_reply: tid=0 2024-09-11T22:56:15.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.394+0000 7f685d0b0740 1 client.16464 shutdown 2024-09-11T22:56:15.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.394+0000 7f685d0b0740 1 -- 172.21.15.152:0/3630100954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb6115d0 msgr2=0x558ccb61bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:15.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.394+0000 7f685d0b0740 1 --2- 172.21.15.152:0/3630100954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb6115d0 0x558ccb61bd20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6848000c00 tx=0x7f68480044d0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.394+0000 7f685d0b0740 1 -- 172.21.15.152:0/3630100954 shutdown_connections 2024-09-11T22:56:15.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.394+0000 7f685d0b0740 1 --2- 172.21.15.152:0/3630100954 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x558ccb62ba40 0x558ccb62de60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.394+0000 7f685d0b0740 1 --2- 172.21.15.152:0/3630100954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb61c370 0x558ccb619570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.394+0000 7f6833fff640 0 client.16464 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:56:15.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.394+0000 7f685d0b0740 1 --2- 172.21.15.152:0/3630100954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb612b40 0x558ccb618fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.394+0000 7f685d0b0740 1 --2- 172.21.15.152:0/3630100954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb6115d0 0x558ccb61bd20 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.395+0000 7f685d0b0740 1 -- 172.21.15.152:0/3630100954 >> 172.21.15.152:0/3630100954 conn(0x558ccb5da310 msgr2=0x558ccb613990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:15.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.395+0000 7f685d0b0740 1 -- 172.21.15.152:0/3630100954 shutdown_connections 2024-09-11T22:56:15.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.395+0000 7f685d0b0740 1 -- 172.21.15.152:0/3630100954 wait complete. 2024-09-11T22:56:15.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.395+0000 7f685d0b0740 20 client.16464 trim_cache size 0 max 16384 2024-09-11T22:56:15.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.409+0000 7f685d0b0740 1 Processor -- start 2024-09-11T22:56:15.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.409+0000 7f685d0b0740 1 -- start start 2024-09-11T22:56:15.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.409+0000 7f685d0b0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb6115d0 0x558ccb614120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.409+0000 7f685d0b0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb612b40 0x558ccb614660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.409+0000 7f685d0b0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb61c370 0x558ccb633730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.409+0000 7f685d0b0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f68541af9b0 con 0x558ccb612b40 2024-09-11T22:56:15.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.409+0000 7f685d0b0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f68541b14f0 con 0x558ccb6115d0 2024-09-11T22:56:15.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.409+0000 7f685d0b0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f68541b0080 con 0x558ccb61c370 2024-09-11T22:56:15.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.410+0000 7f6853fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb612b40 0x558ccb614660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.410+0000 7f6853fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb612b40 0x558ccb614660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56894/0 (socket says 172.21.15.152:56894) 2024-09-11T22:56:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.410+0000 7f6853fff640 1 -- 172.21.15.152:0/1462117518 learned_addr learned my addr 172.21.15.152:0/1462117518 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.410+0000 7f6859274640 1 --2- 172.21.15.152:0/1462117518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb61c370 0x558ccb633730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.410+0000 7f6858a73640 1 --2- 172.21.15.152:0/1462117518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb6115d0 0x558ccb614120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.410+0000 7f6853fff640 1 -- 172.21.15.152:0/1462117518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb61c370 msgr2=0x558ccb633730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.410+0000 7f6853fff640 1 --2- 172.21.15.152:0/1462117518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb61c370 0x558ccb633730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.410+0000 7f6853fff640 1 -- 172.21.15.152:0/1462117518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb6115d0 msgr2=0x558ccb614120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.410+0000 7f6853fff640 1 --2- 172.21.15.152:0/1462117518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb6115d0 0x558ccb614120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.410+0000 7f6853fff640 1 -- 172.21.15.152:0/1462117518 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ccb61b940 con 0x558ccb612b40 2024-09-11T22:56:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.410+0000 7f6858a73640 1 --2- 172.21.15.152:0/1462117518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb6115d0 0x558ccb614120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.410+0000 7f6853fff640 1 --2- 172.21.15.152:0/1462117518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb612b40 0x558ccb614660 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f684000bc20 tx=0x7f684000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a345ea05fb4429 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.410+0000 7f683e7fc640 1 -- 172.21.15.152:0/1462117518 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684001b620 con 0x558ccb612b40 2024-09-11T22:56:15.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.410+0000 7f683e7fc640 1 -- 172.21.15.152:0/1462117518 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684000fe50 con 0x558ccb612b40 2024-09-11T22:56:15.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.410+0000 7f683e7fc640 1 -- 172.21.15.152:0/1462117518 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684001ac60 con 0x558ccb612b40 2024-09-11T22:56:15.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.411+0000 7f685d0b0740 1 -- 172.21.15.152:0/1462117518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb612b40 msgr2=0x558ccb614660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:15.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.411+0000 7f685d0b0740 1 --2- 172.21.15.152:0/1462117518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb612b40 0x558ccb614660 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f684000bc20 tx=0x7f684000f7f0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.411+0000 7f685d0b0740 1 -- 172.21.15.152:0/1462117518 shutdown_connections 2024-09-11T22:56:15.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.411+0000 7f685d0b0740 1 --2- 172.21.15.152:0/1462117518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb61c370 0x558ccb633730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.411+0000 7f685d0b0740 1 --2- 172.21.15.152:0/1462117518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb612b40 0x558ccb614660 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.411+0000 7f685d0b0740 1 --2- 172.21.15.152:0/1462117518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb6115d0 0x558ccb614120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.411+0000 7f685d0b0740 1 -- 172.21.15.152:0/1462117518 >> 172.21.15.152:0/1462117518 conn(0x558ccb5da310 msgr2=0x558ccb61b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:15.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.411+0000 7f685d0b0740 1 -- 172.21.15.152:0/1462117518 shutdown_connections 2024-09-11T22:56:15.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.411+0000 7f685d0b0740 1 -- 172.21.15.152:0/1462117518 wait complete. 2024-09-11T22:56:15.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.411+0000 7f685d0b0740 1 -- start start 2024-09-11T22:56:15.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f685d0b0740 1 Processor -- start 2024-09-11T22:56:15.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f685d0b0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb6115d0 0x558ccb6200f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f685d0b0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb612b40 0x558ccb620630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f685d0b0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb61c370 0x558ccb620b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f685d0b0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f68541b0e20 con 0x558ccb61c370 2024-09-11T22:56:15.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f685d0b0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f68541b0750 con 0x558ccb612b40 2024-09-11T22:56:15.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f685d0b0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f68541ad970 con 0x558ccb6115d0 2024-09-11T22:56:15.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f685d0b0740 10 client.0 _ll_register_callbacks cb 0x7f684000b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:15.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f6859274640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb61c370 0x558ccb620b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f6859274640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb61c370 0x558ccb620b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56898/0 (socket says 172.21.15.152:56898) 2024-09-11T22:56:15.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f6859274640 1 -- 172.21.15.152:0/1116234036 learned_addr learned my addr 172.21.15.152:0/1116234036 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:15.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f6858a73640 1 --2- 172.21.15.152:0/1116234036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb6115d0 0x558ccb6200f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f6853fff640 1 --2- 172.21.15.152:0/1116234036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb612b40 0x558ccb620630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f6859274640 1 -- 172.21.15.152:0/1116234036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb6115d0 msgr2=0x558ccb6200f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:15.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f6859274640 1 --2- 172.21.15.152:0/1116234036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb6115d0 0x558ccb6200f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f6859274640 1 -- 172.21.15.152:0/1116234036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb612b40 msgr2=0x558ccb620630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:15.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f6859274640 1 --2- 172.21.15.152:0/1116234036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb612b40 0x558ccb620630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.412+0000 7f6859274640 1 -- 172.21.15.152:0/1116234036 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ccb6958a0 con 0x558ccb61c370 2024-09-11T22:56:15.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.413+0000 7f6859274640 1 --2- 172.21.15.152:0/1116234036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb61c370 0x558ccb620b70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f684c0117e0 tx=0x7f684c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a989d0c487c2ea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:15.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.413+0000 7f6833fff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:15.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.413+0000 7f6833fff640 1 -- 172.21.15.152:0/1116234036 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c004d20 con 0x558ccb61c370 2024-09-11T22:56:15.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.413+0000 7f6833fff640 1 -- 172.21.15.152:0/1116234036 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f684c010950 con 0x558ccb61c370 2024-09-11T22:56:15.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.413+0000 7f685d0b0740 1 -- 172.21.15.152:0/1116234036 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f684000fe50 con 0x558ccb61c370 2024-09-11T22:56:15.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.413+0000 7f6833fff640 1 -- 172.21.15.152:0/1116234036 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f684c02d020 con 0x558ccb61c370 2024-09-11T22:56:15.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.413+0000 7f685d0b0740 10 client.16470 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:15.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.413+0000 7f685d0b0740 1 -- 172.21.15.152:0/1116234036 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558ccb696b80 con 0x558ccb61c370 2024-09-11T22:56:15.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.413+0000 7f6833fff640 1 -- 172.21.15.152:0/1116234036 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f684000fe50 con 0x558ccb61c370 2024-09-11T22:56:15.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.413+0000 7f6833fff640 1 -- 172.21.15.152:0/1116234036 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f684c021410 con 0x558ccb61c370 2024-09-11T22:56:15.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.414+0000 7f685d0b0740 10 client.16470 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:15.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.414+0000 7f685d0b0740 10 client.16470 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:56:15.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.414+0000 7f685d0b0740 20 client.16470 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:15.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.414+0000 7f685d0b0740 1 --2- 172.21.15.152:0/1116234036 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x558ccb697360 0x558ccb699780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:15.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.414+0000 7f685d0b0740 4 client.16470 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:56:15.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.414+0000 7f685d0b0740 1 -- 172.21.15.152:0/1116234036 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ccb4fb170 con 0x558ccb697360 2024-09-11T22:56:15.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.414+0000 7f6858a73640 1 --2- 172.21.15.152:0/1116234036 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x558ccb697360 0x558ccb699780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:15.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.414+0000 7f6858a73640 1 --2- 172.21.15.152:0/1116234036 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x558ccb697360 0x558ccb699780 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c0087dc75ad485fd server_cookie=e6643dd3e4a0d818 in_seq=0 out_seq=0 2024-09-11T22:56:15.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.414+0000 7f6833fff640 10 client.16470 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:56:15.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.414+0000 7f6833fff640 1 -- 172.21.15.152:0/1116234036 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ccb4fb170 con 0x558ccb697360 2024-09-11T22:56:15.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.414+0000 7f6833fff640 10 client.16470 handle_command_reply: tid=0 2024-09-11T22:56:15.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.414+0000 7f685d0b0740 1 client.16470 shutdown 2024-09-11T22:56:15.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.414+0000 7f685d0b0740 1 -- 172.21.15.152:0/1116234036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb61c370 msgr2=0x558ccb620b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:15.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.414+0000 7f685d0b0740 1 --2- 172.21.15.152:0/1116234036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb61c370 0x558ccb620b70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f684c0117e0 tx=0x7f684c004ab0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685d0b0740 1 -- 172.21.15.152:0/1116234036 shutdown_connections 2024-09-11T22:56:15.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685d0b0740 1 --2- 172.21.15.152:0/1116234036 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x558ccb697360 0x558ccb699780 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685d0b0740 1 --2- 172.21.15.152:0/1116234036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558ccb61c370 0x558ccb620b70 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f6833fff640 0 client.16470 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:56:15.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685d0b0740 1 --2- 172.21.15.152:0/1116234036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558ccb612b40 0x558ccb620630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685d0b0740 1 --2- 172.21.15.152:0/1116234036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558ccb6115d0 0x558ccb6200f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685d0b0740 1 -- 172.21.15.152:0/1116234036 >> 172.21.15.152:0/1116234036 conn(0x558ccb5da310 msgr2=0x558ccb60c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:15.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685d0b0740 1 -- 172.21.15.152:0/1116234036 shutdown_connections 2024-09-11T22:56:15.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685d0b0740 1 -- 172.21.15.152:0/1116234036 wait complete. 2024-09-11T22:56:15.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685d0b0740 20 client.16470 trim_cache size 0 max 16384 2024-09-11T22:56:15.433 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:56:15.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685acd7640 1 -- 172.21.15.152:0/1892856441 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f68440a2cf0 msgr2=0x7f68440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:15.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685acd7640 1 --2- 172.21.15.152:0/1892856441 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f68440a2cf0 0x7f68440a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f684c005fd0 tx=0x7f684c00f730 comp rx=0 tx=0).stop 2024-09-11T22:56:15.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685acd7640 1 -- 172.21.15.152:0/1892856441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6854101de0 msgr2=0x7f68540744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:15.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685acd7640 1 --2- 172.21.15.152:0/1892856441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6854101de0 0x7f68540744f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6848006010 tx=0x7f68480042f0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685acd7640 1 -- 172.21.15.152:0/1892856441 shutdown_connections 2024-09-11T22:56:15.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685acd7640 1 --2- 172.21.15.152:0/1892856441 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f68440a2cf0 0x7f68440a51b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685acd7640 1 --2- 172.21.15.152:0/1892856441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6854106f10 0x7f6854074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685acd7640 1 --2- 172.21.15.152:0/1892856441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6854102790 0x7f6854074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.415+0000 7f685acd7640 1 --2- 172.21.15.152:0/1892856441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6854101de0 0x7f68540744f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:15.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.416+0000 7f685acd7640 1 -- 172.21.15.152:0/1892856441 >> 172.21.15.152:0/1892856441 conn(0x7f68540fd8c0 msgr2=0x7f68540fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:15.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.416+0000 7f685acd7640 1 -- 172.21.15.152:0/1892856441 shutdown_connections 2024-09-11T22:56:15.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:15.416+0000 7f685acd7640 1 -- 172.21.15.152:0/1892856441 wait complete. 2024-09-11T22:56:16.251 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:56:16.541 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:56:16.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:16 smithi152 ceph-mon[31181]: pgmap v104: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-09-11T22:56:16.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:16 smithi191 ceph-mon[35211]: pgmap v104: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-09-11T22:56:16.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:16 smithi204 ceph-mon[35454]: pgmap v104: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-09-11T22:56:17.064 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.044+0000 7f7459def640 1 -- 172.21.15.152:0/2139206987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7454101de0 msgr2=0x7f74541021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:17.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.044+0000 7f7459def640 1 --2- 172.21.15.152:0/2139206987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7454101de0 0x7f74541021c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7444009900 tx=0x7f744402a410 comp rx=0 tx=0).stop 2024-09-11T22:56:17.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.044+0000 7f7459def640 1 -- 172.21.15.152:0/2139206987 shutdown_connections 2024-09-11T22:56:17.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.044+0000 7f7459def640 1 --2- 172.21.15.152:0/2139206987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7454106f10 0x7f745410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.044+0000 7f7459def640 1 --2- 172.21.15.152:0/2139206987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7454102790 0x7f74541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.044+0000 7f7459def640 1 --2- 172.21.15.152:0/2139206987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7454101de0 0x7f74541021c0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.044+0000 7f7459def640 1 -- 172.21.15.152:0/2139206987 >> 172.21.15.152:0/2139206987 conn(0x7f74540fd8c0 msgr2=0x7f74540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:17.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.044+0000 7f7459def640 1 -- 172.21.15.152:0/2139206987 shutdown_connections 2024-09-11T22:56:17.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.044+0000 7f7459def640 1 -- 172.21.15.152:0/2139206987 wait complete. 2024-09-11T22:56:17.065 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.045+0000 7f7459def640 1 Processor -- start 2024-09-11T22:56:17.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.045+0000 7f7459def640 1 -- start start 2024-09-11T22:56:17.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.045+0000 7f7459def640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7454101de0 0x7f74541983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.045+0000 7f7459def640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7454102790 0x7f7454198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.045+0000 7f7459def640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7454106f10 0x7f745419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.045+0000 7f7459def640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7454111f70 con 0x7f7454101de0 2024-09-11T22:56:17.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.045+0000 7f7459def640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7454111df0 con 0x7f7454106f10 2024-09-11T22:56:17.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.045+0000 7f7459def640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74541120f0 con 0x7f7454102790 2024-09-11T22:56:17.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.045+0000 7f74537fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7454101de0 0x7f74541983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.045+0000 7f74537fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7454101de0 0x7f74541983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56916/0 (socket says 172.21.15.152:56916) 2024-09-11T22:56:17.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.045+0000 7f74537fe640 1 -- 172.21.15.152:0/783324732 learned_addr learned my addr 172.21.15.152:0/783324732 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:17.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.045+0000 7f7453fff640 1 --2- 172.21.15.152:0/783324732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7454106f10 0x7f745419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.045+0000 7f7452ffd640 1 --2- 172.21.15.152:0/783324732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7454102790 0x7f7454198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.045+0000 7f74537fe640 1 -- 172.21.15.152:0/783324732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7454102790 msgr2=0x7f7454198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:17.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.045+0000 7f74537fe640 1 --2- 172.21.15.152:0/783324732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7454102790 0x7f7454198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.046+0000 7f74537fe640 1 -- 172.21.15.152:0/783324732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7454106f10 msgr2=0x7f745419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:17.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.046+0000 7f74537fe640 1 --2- 172.21.15.152:0/783324732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7454106f10 0x7f745419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.067 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.046+0000 7f74537fe640 1 -- 172.21.15.152:0/783324732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745419d380 con 0x7f7454101de0 2024-09-11T22:56:17.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.046+0000 7f74537fe640 1 --2- 172.21.15.152:0/783324732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7454101de0 0x7f74541983d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7444006010 tx=0x7f74440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:17.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.046+0000 7f7450ff9640 1 -- 172.21.15.152:0/783324732 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744403f030 con 0x7f7454101de0 2024-09-11T22:56:17.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.046+0000 7f7450ff9640 1 -- 172.21.15.152:0/783324732 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7444003ce0 con 0x7f7454101de0 2024-09-11T22:56:17.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.046+0000 7f7459def640 1 -- 172.21.15.152:0/783324732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745419d610 con 0x7f7454101de0 2024-09-11T22:56:17.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.046+0000 7f7450ff9640 1 -- 172.21.15.152:0/783324732 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444035070 con 0x7f7454101de0 2024-09-11T22:56:17.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.046+0000 7f7459def640 1 -- 172.21.15.152:0/783324732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74541a4f50 con 0x7f7454101de0 2024-09-11T22:56:17.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.048+0000 7f7450ff9640 1 -- 172.21.15.152:0/783324732 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7444042030 con 0x7f7454101de0 2024-09-11T22:56:17.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.048+0000 7f745c1c8740 1 Processor -- start 2024-09-11T22:56:17.068 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.048+0000 7f745c1c8740 1 -- start start 2024-09-11T22:56:17.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f745c1c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 0x556cf79ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f745c1c8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79ae5d0 0x556cf79aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f745c1c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79b9370 0x556cf79bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f745c1c8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x556cf79142f0 con 0x556cf79ae5d0 2024-09-11T22:56:17.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f745c1c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556cf78bbcf0 con 0x556cf79b9370 2024-09-11T22:56:17.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f745c1c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556cf79102b0 con 0x556cf79afb40 2024-09-11T22:56:17.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f74537fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79ae5d0 0x556cf79aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f74537fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79ae5d0 0x556cf79aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56924/0 (socket says 172.21.15.152:56924) 2024-09-11T22:56:17.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f74537fe640 1 -- 172.21.15.152:0/680374964 learned_addr learned my addr 172.21.15.152:0/680374964 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:17.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f7453fff640 1 --2- 172.21.15.152:0/680374964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 0x556cf79ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f7452ffd640 1 --2- 172.21.15.152:0/680374964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79b9370 0x556cf79bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f7450ff9640 1 --2- 172.21.15.152:0/783324732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f743c0a2d70 0x7f743c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f7453fff640 1 --2- 172.21.15.152:0/783324732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f743c0a2d70 0x7f743c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f7453fff640 1 -- 172.21.15.152:0/680374964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79b9370 msgr2=0x556cf79bb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:17.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f7453fff640 1 --2- 172.21.15.152:0/680374964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79b9370 0x556cf79bb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f7453fff640 1 -- 172.21.15.152:0/680374964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79ae5d0 msgr2=0x556cf79aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:17.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f7453fff640 1 --2- 172.21.15.152:0/680374964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79ae5d0 0x556cf79aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f7453fff640 1 -- 172.21.15.152:0/680374964 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cf78a03a0 con 0x556cf79afb40 2024-09-11T22:56:17.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f74537fe640 1 --2- 172.21.15.152:0/680374964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79ae5d0 0x556cf79aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:56:17.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.049+0000 7f7450ff9640 1 -- 172.21.15.152:0/783324732 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f74440ef810 con 0x7f7454101de0 2024-09-11T22:56:17.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.050+0000 7f7453fff640 1 --2- 172.21.15.152:0/680374964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 0x556cf79ae090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f744c0138f0 tx=0x7f744c013db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2f24f1bd88948e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:17.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.050+0000 7f7439ffb640 1 -- 172.21.15.152:0/680374964 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c013250 con 0x556cf79afb40 2024-09-11T22:56:17.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.050+0000 7f7453fff640 1 --2- 172.21.15.152:0/783324732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f743c0a2d70 0x7f743c0a5230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f744c021000 tx=0x7f744c01f000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:17.070 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.050+0000 7f7439ffb640 1 -- 172.21.15.152:0/680374964 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f744c022070 con 0x556cf79afb40 2024-09-11T22:56:17.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.050+0000 7f7439ffb640 1 -- 172.21.15.152:0/680374964 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c0126b0 con 0x556cf79afb40 2024-09-11T22:56:17.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.050+0000 7f745c1c8740 1 -- 172.21.15.152:0/680374964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 msgr2=0x556cf79ae090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:17.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.050+0000 7f745c1c8740 1 --2- 172.21.15.152:0/680374964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 0x556cf79ae090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f744c0138f0 tx=0x7f744c013db0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f745c1c8740 1 -- 172.21.15.152:0/680374964 shutdown_connections 2024-09-11T22:56:17.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f745c1c8740 1 --2- 172.21.15.152:0/680374964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79b9370 0x556cf79bb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f745c1c8740 1 --2- 172.21.15.152:0/680374964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79ae5d0 0x556cf79aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f745c1c8740 1 --2- 172.21.15.152:0/680374964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 0x556cf79ae090 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f745c1c8740 1 -- 172.21.15.152:0/680374964 >> 172.21.15.152:0/680374964 conn(0x556cf7977310 msgr2=0x556cf79aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:17.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f745c1c8740 1 -- 172.21.15.152:0/680374964 shutdown_connections 2024-09-11T22:56:17.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f745c1c8740 1 -- 172.21.15.152:0/680374964 wait complete. 2024-09-11T22:56:17.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f745c1c8740 1 -- start start 2024-09-11T22:56:17.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f745c1c8740 1 Processor -- start 2024-09-11T22:56:17.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f745c1c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79ae5d0 0x556cf79b8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.071 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f745c1c8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79afb40 0x556cf79b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f745c1c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79b9370 0x556cf79b6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f745c1c8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x556cf79c0480 con 0x556cf79afb40 2024-09-11T22:56:17.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f745c1c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556cf78cf060 con 0x556cf79ae5d0 2024-09-11T22:56:17.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f745c1c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556cf79c0600 con 0x556cf79b9370 2024-09-11T22:56:17.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f745c1c8740 10 client.0 _ll_register_callbacks cb 0x556cf76228c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:17.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f7452ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79afb40 0x556cf79b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f7452ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79afb40 0x556cf79b6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56926/0 (socket says 172.21.15.152:56926) 2024-09-11T22:56:17.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.051+0000 7f7452ffd640 1 -- 172.21.15.152:0/2284460779 learned_addr learned my addr 172.21.15.152:0/2284460779 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:17.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.052+0000 7f7453fff640 1 --2- 172.21.15.152:0/2284460779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79b9370 0x556cf79b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.052+0000 7f74537fe640 1 --2- 172.21.15.152:0/2284460779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79ae5d0 0x556cf79b8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.052+0000 7f7452ffd640 1 -- 172.21.15.152:0/2284460779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79b9370 msgr2=0x556cf79b6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:17.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.052+0000 7f7452ffd640 1 --2- 172.21.15.152:0/2284460779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79b9370 0x556cf79b6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.052+0000 7f7452ffd640 1 -- 172.21.15.152:0/2284460779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79ae5d0 msgr2=0x556cf79b8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:17.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.052+0000 7f7452ffd640 1 --2- 172.21.15.152:0/2284460779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79ae5d0 0x556cf79b8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.052+0000 7f7452ffd640 1 -- 172.21.15.152:0/2284460779 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cf76853c0 con 0x556cf79afb40 2024-09-11T22:56:17.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.052+0000 7f7452ffd640 1 --2- 172.21.15.152:0/2284460779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79afb40 0x556cf79b6020 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f744800bf00 tx=0x7f7448011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2eae781cc678f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:17.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.052+0000 7f7432ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:17.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.052+0000 7f7432ffd640 1 -- 172.21.15.152:0/2284460779 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448004020 con 0x556cf79afb40 2024-09-11T22:56:17.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.052+0000 7f7432ffd640 1 -- 172.21.15.152:0/2284460779 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f74480041c0 con 0x556cf79afb40 2024-09-11T22:56:17.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.052+0000 7f745c1c8740 1 -- 172.21.15.152:0/2284460779 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f744c022070 con 0x556cf79afb40 2024-09-11T22:56:17.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.052+0000 7f7432ffd640 1 -- 172.21.15.152:0/2284460779 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f74480043c0 con 0x556cf79afb40 2024-09-11T22:56:17.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.052+0000 7f745c1c8740 10 client.16482 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:17.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.052+0000 7f745c1c8740 1 -- 172.21.15.152:0/2284460779 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556cf79cbf50 con 0x556cf79afb40 2024-09-11T22:56:17.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.052+0000 7f7432ffd640 1 -- 172.21.15.152:0/2284460779 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f744c022070 con 0x556cf79afb40 2024-09-11T22:56:17.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.053+0000 7f7432ffd640 1 -- 172.21.15.152:0/2284460779 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7448010700 con 0x556cf79afb40 2024-09-11T22:56:17.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.053+0000 7f745c1c8740 10 client.16482 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:17.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.053+0000 7f745c1c8740 10 client.16482 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:56:17.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.053+0000 7f745c1c8740 20 client.16482 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:17.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.053+0000 7f745c1c8740 1 --2- 172.21.15.152:0/2284460779 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x556cf79cd4b0 0x556cf79cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.053+0000 7f745c1c8740 4 client.16482 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:56:17.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.053+0000 7f745c1c8740 1 -- 172.21.15.152:0/2284460779 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cf7921860 con 0x556cf79cd4b0 2024-09-11T22:56:17.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.053+0000 7f74537fe640 1 --2- 172.21.15.152:0/2284460779 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x556cf79cd4b0 0x556cf79cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.053+0000 7f7432ffd640 10 client.16482 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:56:17.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.053+0000 7f74537fe640 1 --2- 172.21.15.152:0/2284460779 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x556cf79cd4b0 0x556cf79cf8f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3a66f1f78fbf2a9 server_cookie=ff844c1a8f33677 in_seq=0 out_seq=0 2024-09-11T22:56:17.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.054+0000 7f7432ffd640 1 -- 172.21.15.152:0/2284460779 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556cf7921860 con 0x556cf79cd4b0 2024-09-11T22:56:17.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.054+0000 7f7432ffd640 10 client.16482 handle_command_reply: tid=0 2024-09-11T22:56:17.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.054+0000 7f745c1c8740 1 client.16482 shutdown 2024-09-11T22:56:17.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.054+0000 7f745c1c8740 1 -- 172.21.15.152:0/2284460779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79afb40 msgr2=0x556cf79b6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:17.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.054+0000 7f745c1c8740 1 --2- 172.21.15.152:0/2284460779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79afb40 0x556cf79b6020 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f744800bf00 tx=0x7f7448011810 comp rx=0 tx=0).stop 2024-09-11T22:56:17.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.054+0000 7f745c1c8740 1 -- 172.21.15.152:0/2284460779 shutdown_connections 2024-09-11T22:56:17.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.054+0000 7f745c1c8740 1 --2- 172.21.15.152:0/2284460779 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x556cf79cd4b0 0x556cf79cf8f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.054+0000 7f745c1c8740 1 --2- 172.21.15.152:0/2284460779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79b9370 0x556cf79b6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.054+0000 7f745c1c8740 1 --2- 172.21.15.152:0/2284460779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79afb40 0x556cf79b6020 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.054+0000 7f7432ffd640 0 client.16482 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:56:17.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.054+0000 7f745c1c8740 1 --2- 172.21.15.152:0/2284460779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79ae5d0 0x556cf79b8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.054+0000 7f745c1c8740 1 -- 172.21.15.152:0/2284460779 >> 172.21.15.152:0/2284460779 conn(0x556cf7977310 msgr2=0x556cf79b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:17.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.054+0000 7f745c1c8740 1 -- 172.21.15.152:0/2284460779 shutdown_connections 2024-09-11T22:56:17.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.054+0000 7f745c1c8740 1 -- 172.21.15.152:0/2284460779 wait complete. 2024-09-11T22:56:17.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.055+0000 7f745c1c8740 20 client.16482 trim_cache size 0 max 16384 2024-09-11T22:56:17.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.067+0000 7f745c1c8740 1 Processor -- start 2024-09-11T22:56:17.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.067+0000 7f745c1c8740 1 -- start start 2024-09-11T22:56:17.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.067+0000 7f745c1c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79ae5d0 0x556cf79c9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.067+0000 7f745c1c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 0x556cf79d51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.067+0000 7f745c1c8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79b9370 0x556cf79d5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.067+0000 7f745c1c8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f74541ab4d0 con 0x556cf79b9370 2024-09-11T22:56:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.067+0000 7f745c1c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f74541ad010 con 0x556cf79ae5d0 2024-09-11T22:56:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.067+0000 7f745c1c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74541abba0 con 0x556cf79afb40 2024-09-11T22:56:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.068+0000 7f7453fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79b9370 0x556cf79d5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.068+0000 7f7453fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79b9370 0x556cf79d5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56934/0 (socket says 172.21.15.152:56934) 2024-09-11T22:56:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.068+0000 7f7453fff640 1 -- 172.21.15.152:0/3584095111 learned_addr learned my addr 172.21.15.152:0/3584095111 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.068+0000 7f74537fe640 1 --2- 172.21.15.152:0/3584095111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79ae5d0 0x556cf79c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.068+0000 7f7452ffd640 1 --2- 172.21.15.152:0/3584095111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 0x556cf79d51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.068+0000 7f7453fff640 1 -- 172.21.15.152:0/3584095111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 msgr2=0x556cf79d51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.068+0000 7f7453fff640 1 --2- 172.21.15.152:0/3584095111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 0x556cf79d51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.068+0000 7f7453fff640 1 -- 172.21.15.152:0/3584095111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79ae5d0 msgr2=0x556cf79c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:17.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.068+0000 7f7453fff640 1 --2- 172.21.15.152:0/3584095111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79ae5d0 0x556cf79c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.068+0000 7f7453fff640 1 -- 172.21.15.152:0/3584095111 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cf79bbad0 con 0x556cf79b9370 2024-09-11T22:56:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.068+0000 7f74537fe640 1 --2- 172.21.15.152:0/3584095111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79ae5d0 0x556cf79c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.068+0000 7f7452ffd640 1 --2- 172.21.15.152:0/3584095111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 0x556cf79d51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.068+0000 7f7453fff640 1 --2- 172.21.15.152:0/3584095111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79b9370 0x556cf79d5700 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f744c01dd80 tx=0x7f744c0078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd385a23286d5a1c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.068+0000 7f74397fa640 1 -- 172.21.15.152:0/3584095111 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c035020 con 0x556cf79b9370 2024-09-11T22:56:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.068+0000 7f74397fa640 1 -- 172.21.15.152:0/3584095111 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f744c022070 con 0x556cf79b9370 2024-09-11T22:56:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.068+0000 7f74397fa640 1 -- 172.21.15.152:0/3584095111 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c003bd0 con 0x556cf79b9370 2024-09-11T22:56:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.069+0000 7f745c1c8740 1 -- 172.21.15.152:0/3584095111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79b9370 msgr2=0x556cf79d5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.069+0000 7f745c1c8740 1 --2- 172.21.15.152:0/3584095111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79b9370 0x556cf79d5700 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f744c01dd80 tx=0x7f744c0078e0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.069+0000 7f745c1c8740 1 -- 172.21.15.152:0/3584095111 shutdown_connections 2024-09-11T22:56:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.069+0000 7f745c1c8740 1 --2- 172.21.15.152:0/3584095111 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79b9370 0x556cf79d5700 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.069+0000 7f745c1c8740 1 --2- 172.21.15.152:0/3584095111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 0x556cf79d51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.069+0000 7f745c1c8740 1 --2- 172.21.15.152:0/3584095111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79ae5d0 0x556cf79c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.069+0000 7f745c1c8740 1 -- 172.21.15.152:0/3584095111 >> 172.21.15.152:0/3584095111 conn(0x556cf7977310 msgr2=0x556cf79b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.069+0000 7f745c1c8740 1 -- 172.21.15.152:0/3584095111 shutdown_connections 2024-09-11T22:56:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.069+0000 7f745c1c8740 1 -- 172.21.15.152:0/3584095111 wait complete. 2024-09-11T22:56:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.069+0000 7f745c1c8740 1 -- start start 2024-09-11T22:56:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.069+0000 7f745c1c8740 1 Processor -- start 2024-09-11T22:56:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f745c1c8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79ae5d0 0x556cf79c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f745c1c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 0x556cf79c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f745c1c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79b9370 0x556cf79c1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f745c1c8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f74541ac940 con 0x556cf79ae5d0 2024-09-11T22:56:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f745c1c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f74541ac270 con 0x556cf79b9370 2024-09-11T22:56:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f745c1c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f74541a9490 con 0x556cf79afb40 2024-09-11T22:56:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f74537fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79ae5d0 0x556cf79c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f745c1c8740 10 client.0 _ll_register_callbacks cb 0x7f744800bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:17.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f74537fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79ae5d0 0x556cf79c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56936/0 (socket says 172.21.15.152:56936) 2024-09-11T22:56:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f74537fe640 1 -- 172.21.15.152:0/3941837505 learned_addr learned my addr 172.21.15.152:0/3941837505 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f7452ffd640 1 --2- 172.21.15.152:0/3941837505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 0x556cf79c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f7453fff640 1 --2- 172.21.15.152:0/3941837505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79b9370 0x556cf79c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f74537fe640 1 -- 172.21.15.152:0/3941837505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 msgr2=0x556cf79c11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f74537fe640 1 --2- 172.21.15.152:0/3941837505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 0x556cf79c11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f74537fe640 1 -- 172.21.15.152:0/3941837505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79b9370 msgr2=0x556cf79c1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f74537fe640 1 --2- 172.21.15.152:0/3941837505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79b9370 0x556cf79c1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f74537fe640 1 -- 172.21.15.152:0/3941837505 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cf79c1db0 con 0x556cf79ae5d0 2024-09-11T22:56:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f7453fff640 1 --2- 172.21.15.152:0/3941837505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79b9370 0x556cf79c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:56:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f74537fe640 1 --2- 172.21.15.152:0/3941837505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79ae5d0 0x556cf79c5ea0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7444000c00 tx=0x7f7444004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe4abe77c150009 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f7432ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f7432ffd640 1 -- 172.21.15.152:0/3941837505 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74440be070 con 0x556cf79ae5d0 2024-09-11T22:56:17.084 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.070+0000 7f7432ffd640 1 -- 172.21.15.152:0/3941837505 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f74440c4620 con 0x556cf79ae5d0 2024-09-11T22:56:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.071+0000 7f745c1c8740 1 -- 172.21.15.152:0/3941837505 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f744c022070 con 0x556cf79ae5d0 2024-09-11T22:56:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.071+0000 7f7432ffd640 1 -- 172.21.15.152:0/3941837505 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f74440d8020 con 0x556cf79ae5d0 2024-09-11T22:56:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.071+0000 7f745c1c8740 10 client.16488 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.071+0000 7f745c1c8740 1 -- 172.21.15.152:0/3941837505 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556cf7a37ff0 con 0x556cf79ae5d0 2024-09-11T22:56:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.071+0000 7f7432ffd640 1 -- 172.21.15.152:0/3941837505 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f744c022070 con 0x556cf79ae5d0 2024-09-11T22:56:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.071+0000 7f7432ffd640 1 -- 172.21.15.152:0/3941837505 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f74440bc150 con 0x556cf79ae5d0 2024-09-11T22:56:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.071+0000 7f745c1c8740 10 client.16488 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.071+0000 7f745c1c8740 10 client.16488 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:56:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.071+0000 7f745c1c8740 20 client.16488 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.071+0000 7f745c1c8740 1 --2- 172.21.15.152:0/3941837505 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x556cf7a38760 0x556cf7a3ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.071+0000 7f745c1c8740 4 client.16488 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:56:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.071+0000 7f745c1c8740 1 -- 172.21.15.152:0/3941837505 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556cf791bd70 con 0x556cf7a38760 2024-09-11T22:56:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.071+0000 7f7452ffd640 1 --2- 172.21.15.152:0/3941837505 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x556cf7a38760 0x556cf7a3ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:17.085 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f7452ffd640 1 --2- 172.21.15.152:0/3941837505 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x556cf7a38760 0x556cf7a3ab80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e7d2c77fdf0f5ac server_cookie=8a63be820b53f113 in_seq=0 out_seq=0 2024-09-11T22:56:17.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f7432ffd640 10 client.16488 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:56:17.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f7432ffd640 1 -- 172.21.15.152:0/3941837505 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556cf791bd70 con 0x556cf7a38760 2024-09-11T22:56:17.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f7432ffd640 10 client.16488 handle_command_reply: tid=0 2024-09-11T22:56:17.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f745c1c8740 1 client.16488 shutdown 2024-09-11T22:56:17.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f745c1c8740 1 -- 172.21.15.152:0/3941837505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79ae5d0 msgr2=0x556cf79c5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:17.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f745c1c8740 1 --2- 172.21.15.152:0/3941837505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79ae5d0 0x556cf79c5ea0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7444000c00 tx=0x7f7444004270 comp rx=0 tx=0).stop 2024-09-11T22:56:17.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f745c1c8740 1 -- 172.21.15.152:0/3941837505 shutdown_connections 2024-09-11T22:56:17.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f745c1c8740 1 --2- 172.21.15.152:0/3941837505 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x556cf7a38760 0x556cf7a3ab80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f745c1c8740 1 --2- 172.21.15.152:0/3941837505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556cf79b9370 0x556cf79c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f745c1c8740 1 --2- 172.21.15.152:0/3941837505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cf79afb40 0x556cf79c11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f745c1c8740 1 --2- 172.21.15.152:0/3941837505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556cf79ae5d0 0x556cf79c5ea0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.086 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f745c1c8740 1 -- 172.21.15.152:0/3941837505 >> 172.21.15.152:0/3941837505 conn(0x556cf7977310 msgr2=0x556cf79b7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:17.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f7432ffd640 0 client.16488 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:56:17.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f745c1c8740 1 -- 172.21.15.152:0/3941837505 shutdown_connections 2024-09-11T22:56:17.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f745c1c8740 1 -- 172.21.15.152:0/3941837505 wait complete. 2024-09-11T22:56:17.087 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.072+0000 7f745c1c8740 20 client.16488 trim_cache size 0 max 16384 2024-09-11T22:56:17.087 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:56:17.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.073+0000 7f7459def640 1 -- 172.21.15.152:0/783324732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f743c0a2d70 msgr2=0x7f743c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:17.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.073+0000 7f7459def640 1 --2- 172.21.15.152:0/783324732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f743c0a2d70 0x7f743c0a5230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f744c021000 tx=0x7f744c01f000 comp rx=0 tx=0).stop 2024-09-11T22:56:17.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.073+0000 7f7459def640 1 -- 172.21.15.152:0/783324732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7454101de0 msgr2=0x7f74541983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:17.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.073+0000 7f7459def640 1 --2- 172.21.15.152:0/783324732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7454101de0 0x7f74541983d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7444006010 tx=0x7f74440042f0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.073+0000 7f7459def640 1 -- 172.21.15.152:0/783324732 shutdown_connections 2024-09-11T22:56:17.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.073+0000 7f7459def640 1 --2- 172.21.15.152:0/783324732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f743c0a2d70 0x7f743c0a5230 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.073+0000 7f7459def640 1 --2- 172.21.15.152:0/783324732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7454106f10 0x7f745419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.073+0000 7f7459def640 1 --2- 172.21.15.152:0/783324732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7454102790 0x7f7454198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.073+0000 7f7459def640 1 --2- 172.21.15.152:0/783324732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7454101de0 0x7f74541983d0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:17.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.073+0000 7f7459def640 1 -- 172.21.15.152:0/783324732 >> 172.21.15.152:0/783324732 conn(0x7f74540fd8c0 msgr2=0x7f74540febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:17.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.073+0000 7f7459def640 1 -- 172.21.15.152:0/783324732 shutdown_connections 2024-09-11T22:56:17.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:17.073+0000 7f7459def640 1 -- 172.21.15.152:0/783324732 wait complete. 2024-09-11T22:56:18.026 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:56:18.272 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:56:18.699 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:18 smithi152 ceph-mon[31181]: pgmap v105: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-11T22:56:18.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:18 smithi191 ceph-mon[35211]: pgmap v105: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-11T22:56:18.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:18 smithi204 ceph-mon[35454]: pgmap v105: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-11T22:56:18.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.779+0000 7fa329bdb640 1 -- 172.21.15.152:0/1819521905 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa324101de0 msgr2=0x7fa3241021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:18.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.779+0000 7fa329bdb640 1 --2- 172.21.15.152:0/1819521905 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa324101de0 0x7fa3241021c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa314009900 tx=0x7fa31402a410 comp rx=0 tx=0).stop 2024-09-11T22:56:18.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.779+0000 7fa329bdb640 1 -- 172.21.15.152:0/1819521905 shutdown_connections 2024-09-11T22:56:18.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.779+0000 7fa329bdb640 1 --2- 172.21.15.152:0/1819521905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa324106f10 0x7fa32410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.779+0000 7fa329bdb640 1 --2- 172.21.15.152:0/1819521905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa324102790 0x7fa3241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.779+0000 7fa329bdb640 1 --2- 172.21.15.152:0/1819521905 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa324101de0 0x7fa3241021c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.779+0000 7fa329bdb640 1 -- 172.21.15.152:0/1819521905 >> 172.21.15.152:0/1819521905 conn(0x7fa3240fd8c0 msgr2=0x7fa3240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:18.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.779+0000 7fa329bdb640 1 -- 172.21.15.152:0/1819521905 shutdown_connections 2024-09-11T22:56:18.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.779+0000 7fa329bdb640 1 -- 172.21.15.152:0/1819521905 wait complete. 2024-09-11T22:56:18.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.780+0000 7fa329bdb640 1 Processor -- start 2024-09-11T22:56:18.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.780+0000 7fa329bdb640 1 -- start start 2024-09-11T22:56:18.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.780+0000 7fa329bdb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa324101de0 0x7fa3241983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.780+0000 7fa329bdb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa324102790 0x7fa324198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.780+0000 7fa329bdb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa324106f10 0x7fa32419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.780+0000 7fa329bdb640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa324111f70 con 0x7fa324102790 2024-09-11T22:56:18.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.780+0000 7fa329bdb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa324111df0 con 0x7fa324106f10 2024-09-11T22:56:18.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.780+0000 7fa329bdb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241120f0 con 0x7fa324101de0 2024-09-11T22:56:18.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.780+0000 7fa322ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa324102790 0x7fa324198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.780+0000 7fa322ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa324102790 0x7fa324198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38704/0 (socket says 172.21.15.152:38704) 2024-09-11T22:56:18.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.780+0000 7fa322ffd640 1 -- 172.21.15.152:0/3725484074 learned_addr learned my addr 172.21.15.152:0/3725484074 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:18.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.780+0000 7fa3237fe640 1 --2- 172.21.15.152:0/3725484074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa324101de0 0x7fa3241983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.781+0000 7fa323fff640 1 --2- 172.21.15.152:0/3725484074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa324106f10 0x7fa32419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.781+0000 7fa322ffd640 1 -- 172.21.15.152:0/3725484074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa324101de0 msgr2=0x7fa3241983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:18.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.781+0000 7fa322ffd640 1 --2- 172.21.15.152:0/3725484074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa324101de0 0x7fa3241983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.781+0000 7fa322ffd640 1 -- 172.21.15.152:0/3725484074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa324106f10 msgr2=0x7fa32419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:18.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.781+0000 7fa322ffd640 1 --2- 172.21.15.152:0/3725484074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa324106f10 0x7fa32419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.781+0000 7fa322ffd640 1 -- 172.21.15.152:0/3725484074 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32419d380 con 0x7fa324102790 2024-09-11T22:56:18.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.781+0000 7fa323fff640 1 --2- 172.21.15.152:0/3725484074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa324106f10 0x7fa32419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:56:18.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.781+0000 7fa322ffd640 1 --2- 172.21.15.152:0/3725484074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa324102790 0x7fa324198910 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa31800d770 tx=0x7fa31800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:18.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.781+0000 7fa320ff9640 1 -- 172.21.15.152:0/3725484074 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318017070 con 0x7fa324102790 2024-09-11T22:56:18.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.781+0000 7fa329bdb640 1 -- 172.21.15.152:0/3725484074 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32419d670 con 0x7fa324102790 2024-09-11T22:56:18.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.781+0000 7fa320ff9640 1 -- 172.21.15.152:0/3725484074 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa318005500 con 0x7fa324102790 2024-09-11T22:56:18.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.781+0000 7fa320ff9640 1 -- 172.21.15.152:0/3725484074 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318004e00 con 0x7fa324102790 2024-09-11T22:56:18.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.781+0000 7fa329bdb640 1 -- 172.21.15.152:0/3725484074 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3241a4f50 con 0x7fa324102790 2024-09-11T22:56:18.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.783+0000 7fa320ff9640 1 -- 172.21.15.152:0/3725484074 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa318004fa0 con 0x7fa324102790 2024-09-11T22:56:18.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.784+0000 7fa32bfb4740 1 Processor -- start 2024-09-11T22:56:18.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.784+0000 7fa32bfb4740 1 -- start start 2024-09-11T22:56:18.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.784+0000 7fa320ff9640 1 --2- 172.21.15.152:0/3725484074 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa30c0a2cf0 0x7fa30c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.784+0000 7fa32bfb4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f713470b40 0x55f71346f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.784+0000 7fa32bfb4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f71346f5d0 0x55f71346fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.784+0000 7fa32bfb4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71347e780 0x55f713480b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.784+0000 7fa32bfb4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f7133d52f0 con 0x55f713470b40 2024-09-11T22:56:18.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.784+0000 7fa32bfb4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f71337ccf0 con 0x55f71346f5d0 2024-09-11T22:56:18.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.784+0000 7fa32bfb4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f7133d12b0 con 0x55f71347e780 2024-09-11T22:56:18.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.784+0000 7fa323fff640 1 --2- 172.21.15.152:0/3725484074 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa30c0a2cf0 0x7fa30c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.784+0000 7fa320ff9640 1 -- 172.21.15.152:0/3725484074 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa3180cee80 con 0x7fa324102790 2024-09-11T22:56:18.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.784+0000 7fa3237fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f713470b40 0x55f71346f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.784+0000 7fa3237fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f713470b40 0x55f71346f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38714/0 (socket says 172.21.15.152:38714) 2024-09-11T22:56:18.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.784+0000 7fa3237fe640 1 -- 172.21.15.152:0/3960476152 learned_addr learned my addr 172.21.15.152:0/3960476152 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:18.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.784+0000 7fa322ffd640 1 --2- 172.21.15.152:0/3960476152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f71346f5d0 0x55f71346fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.785+0000 7fa3237fe640 1 -- 172.21.15.152:0/3960476152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71347e780 msgr2=0x55f713480b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:18.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.785+0000 7fa3237fe640 1 --2- 172.21.15.152:0/3960476152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71347e780 0x55f713480b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.785+0000 7fa3237fe640 1 -- 172.21.15.152:0/3960476152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f71346f5d0 msgr2=0x55f71346fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:18.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.785+0000 7fa3237fe640 1 --2- 172.21.15.152:0/3960476152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f71346f5d0 0x55f71346fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.785+0000 7fa3237fe640 1 -- 172.21.15.152:0/3960476152 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7133613a0 con 0x55f713470b40 2024-09-11T22:56:18.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.785+0000 7fa323fff640 1 --2- 172.21.15.152:0/3725484074 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa30c0a2cf0 0x7fa30c0a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa31c005cc0 tx=0x7fa31c00e720 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:18.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.785+0000 7fa3237fe640 1 --2- 172.21.15.152:0/3960476152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f713470b40 0x55f71346f090 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa314009900 tx=0x7fa314031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa796cb289bc9628 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:18.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.785+0000 7fa309ffb640 1 -- 172.21.15.152:0/3960476152 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314040070 con 0x55f713470b40 2024-09-11T22:56:18.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.785+0000 7fa309ffb640 1 -- 172.21.15.152:0/3960476152 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa314005020 con 0x55f713470b40 2024-09-11T22:56:18.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.785+0000 7fa309ffb640 1 -- 172.21.15.152:0/3960476152 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314035040 con 0x55f713470b40 2024-09-11T22:56:18.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.786+0000 7fa32bfb4740 1 -- 172.21.15.152:0/3960476152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f713470b40 msgr2=0x55f71346f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:18.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.786+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/3960476152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f713470b40 0x55f71346f090 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa314009900 tx=0x7fa314031630 comp rx=0 tx=0).stop 2024-09-11T22:56:18.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.786+0000 7fa32bfb4740 1 -- 172.21.15.152:0/3960476152 shutdown_connections 2024-09-11T22:56:18.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.786+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/3960476152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71347e780 0x55f713480b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.786+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/3960476152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f71346f5d0 0x55f71346fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.786+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/3960476152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f713470b40 0x55f71346f090 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.786+0000 7fa32bfb4740 1 -- 172.21.15.152:0/3960476152 >> 172.21.15.152:0/3960476152 conn(0x55f713438310 msgr2=0x55f71346bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:18.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.786+0000 7fa32bfb4740 1 -- 172.21.15.152:0/3960476152 shutdown_connections 2024-09-11T22:56:18.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.786+0000 7fa32bfb4740 1 -- 172.21.15.152:0/3960476152 wait complete. 2024-09-11T22:56:18.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.786+0000 7fa32bfb4740 1 -- start start 2024-09-11T22:56:18.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.786+0000 7fa32bfb4740 1 Processor -- start 2024-09-11T22:56:18.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.786+0000 7fa32bfb4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71346f5d0 0x55f71347e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.786+0000 7fa32bfb4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f713470b40 0x55f71347b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.786+0000 7fa32bfb4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f71347e780 0x55f71347b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.786+0000 7fa32bfb4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f713481420 con 0x55f71347e780 2024-09-11T22:56:18.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa32bfb4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f713390060 con 0x55f713470b40 2024-09-11T22:56:18.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa32bfb4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f7134815a0 con 0x55f71346f5d0 2024-09-11T22:56:18.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa32bfb4740 10 client.0 _ll_register_callbacks cb 0x55f7130e38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:18.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa323fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f71347e780 0x55f71347b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa323fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f71347e780 0x55f71347b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38728/0 (socket says 172.21.15.152:38728) 2024-09-11T22:56:18.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa323fff640 1 -- 172.21.15.152:0/1025163128 learned_addr learned my addr 172.21.15.152:0/1025163128 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:18.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa322ffd640 1 --2- 172.21.15.152:0/1025163128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f713470b40 0x55f71347b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa3237fe640 1 --2- 172.21.15.152:0/1025163128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71346f5d0 0x55f71347e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa323fff640 1 -- 172.21.15.152:0/1025163128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71346f5d0 msgr2=0x55f71347e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:18.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa323fff640 1 --2- 172.21.15.152:0/1025163128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71346f5d0 0x55f71347e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa323fff640 1 -- 172.21.15.152:0/1025163128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f713470b40 msgr2=0x55f71347b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:18.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa323fff640 1 --2- 172.21.15.152:0/1025163128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f713470b40 0x55f71347b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa323fff640 1 -- 172.21.15.152:0/1025163128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7131463c0 con 0x55f71347e780 2024-09-11T22:56:18.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa322ffd640 1 --2- 172.21.15.152:0/1025163128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f713470b40 0x55f71347b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:56:18.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa323fff640 1 --2- 172.21.15.152:0/1025163128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f71347e780 0x55f71347b890 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa31c005c90 tx=0x7fa31c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c664d8d2f434931 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:18.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa302ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:18.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa302ffd640 1 -- 172.21.15.152:0/1025163128 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c022070 con 0x55f71347e780 2024-09-11T22:56:18.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa32bfb4740 1 -- 172.21.15.152:0/1025163128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa314005020 con 0x55f71347e780 2024-09-11T22:56:18.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.787+0000 7fa302ffd640 1 -- 172.21.15.152:0/1025163128 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa31c01e430 con 0x55f71347e780 2024-09-11T22:56:18.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.788+0000 7fa302ffd640 1 -- 172.21.15.152:0/1025163128 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa31c02e050 con 0x55f71347e780 2024-09-11T22:56:18.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.788+0000 7fa32bfb4740 10 client.16500 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:18.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.788+0000 7fa32bfb4740 1 -- 172.21.15.152:0/1025163128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f71348ce40 con 0x55f71347e780 2024-09-11T22:56:18.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.788+0000 7fa302ffd640 1 -- 172.21.15.152:0/1025163128 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa314005020 con 0x55f71347e780 2024-09-11T22:56:18.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.788+0000 7fa302ffd640 1 -- 172.21.15.152:0/1025163128 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa31c01d610 con 0x55f71347e780 2024-09-11T22:56:18.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.788+0000 7fa32bfb4740 10 client.16500 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:18.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.788+0000 7fa32bfb4740 10 client.16500 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:56:18.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.788+0000 7fa32bfb4740 20 client.16500 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:18.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.788+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/1025163128 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55f71348e2a0 0x55f7134906f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.788+0000 7fa32bfb4740 4 client.16500 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:56:18.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.788+0000 7fa32bfb4740 1 -- 172.21.15.152:0/1025163128 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7133e2860 con 0x55f71348e2a0 2024-09-11T22:56:18.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.788+0000 7fa3237fe640 1 --2- 172.21.15.152:0/1025163128 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55f71348e2a0 0x55f7134906f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.789+0000 7fa302ffd640 10 client.16500 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:56:18.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.789+0000 7fa3237fe640 1 --2- 172.21.15.152:0/1025163128 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55f71348e2a0 0x55f7134906f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f899995075de53e8 server_cookie=555e0f0ccee55eec in_seq=0 out_seq=0 2024-09-11T22:56:18.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.789+0000 7fa302ffd640 1 -- 172.21.15.152:0/1025163128 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7133e2860 con 0x55f71348e2a0 2024-09-11T22:56:18.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.789+0000 7fa302ffd640 10 client.16500 handle_command_reply: tid=0 2024-09-11T22:56:18.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.789+0000 7fa32bfb4740 1 client.16500 shutdown 2024-09-11T22:56:18.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.789+0000 7fa32bfb4740 1 -- 172.21.15.152:0/1025163128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f71347e780 msgr2=0x55f71347b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:18.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.789+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/1025163128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f71347e780 0x55f71347b890 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa31c005c90 tx=0x7fa31c004a40 comp rx=0 tx=0).stop 2024-09-11T22:56:18.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.790+0000 7fa32bfb4740 1 -- 172.21.15.152:0/1025163128 shutdown_connections 2024-09-11T22:56:18.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.790+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/1025163128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f71347e780 0x55f71347b890 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.790+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/1025163128 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55f71348e2a0 0x55f7134906f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.790+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/1025163128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f713470b40 0x55f71347b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.790+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/1025163128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71346f5d0 0x55f71347e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.790+0000 7fa32bfb4740 1 -- 172.21.15.152:0/1025163128 >> 172.21.15.152:0/1025163128 conn(0x55f713438310 msgr2=0x55f713471960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:18.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.790+0000 7fa302ffd640 0 client.16500 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:56:18.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.790+0000 7fa32bfb4740 1 -- 172.21.15.152:0/1025163128 shutdown_connections 2024-09-11T22:56:18.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.790+0000 7fa32bfb4740 1 -- 172.21.15.152:0/1025163128 wait complete. 2024-09-11T22:56:18.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.790+0000 7fa32bfb4740 20 client.16500 trim_cache size 0 max 16384 2024-09-11T22:56:18.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.802+0000 7fa32bfb4740 1 Processor -- start 2024-09-11T22:56:18.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.802+0000 7fa32bfb4740 1 -- start start 2024-09-11T22:56:18.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.802+0000 7fa32bfb4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71346f5d0 0x55f71348a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.802+0000 7fa32bfb4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f713470b40 0x55f713479c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.802+0000 7fa32bfb4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f71347e780 0x55f71347a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.802+0000 7fa32bfb4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241ab4c0 con 0x55f71347e780 2024-09-11T22:56:18.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.802+0000 7fa32bfb4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241ad000 con 0x55f713470b40 2024-09-11T22:56:18.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.802+0000 7fa32bfb4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241abb90 con 0x55f71346f5d0 2024-09-11T22:56:18.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.803+0000 7fa323fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f71347e780 0x55f71347a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.803+0000 7fa323fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f71347e780 0x55f71347a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38734/0 (socket says 172.21.15.152:38734) 2024-09-11T22:56:18.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.803+0000 7fa323fff640 1 -- 172.21.15.152:0/265638253 learned_addr learned my addr 172.21.15.152:0/265638253 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:18.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.803+0000 7fa323fff640 1 -- 172.21.15.152:0/265638253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71346f5d0 msgr2=0x55f71348a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:18.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.803+0000 7fa3237fe640 1 --2- 172.21.15.152:0/265638253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71346f5d0 0x55f71348a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.803+0000 7fa323fff640 1 --2- 172.21.15.152:0/265638253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71346f5d0 0x55f71348a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.803+0000 7fa323fff640 1 -- 172.21.15.152:0/265638253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f713470b40 msgr2=0x55f713479c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:18.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.803+0000 7fa322ffd640 1 --2- 172.21.15.152:0/265638253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f713470b40 0x55f713479c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.803+0000 7fa323fff640 1 --2- 172.21.15.152:0/265638253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f713470b40 0x55f713479c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.803+0000 7fa323fff640 1 -- 172.21.15.152:0/265638253 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f71346c960 con 0x55f71347e780 2024-09-11T22:56:18.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.803+0000 7fa3237fe640 1 --2- 172.21.15.152:0/265638253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71346f5d0 0x55f71348a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:18.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.803+0000 7fa322ffd640 1 --2- 172.21.15.152:0/265638253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f713470b40 0x55f713479c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:18.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.803+0000 7fa323fff640 1 --2- 172.21.15.152:0/265638253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f71347e780 0x55f71347a160 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0076b0 tx=0x7fa31c011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a911fa2192342ccc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:18.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.803+0000 7fa3097fa640 1 -- 172.21.15.152:0/265638253 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c037070 con 0x55f71347e780 2024-09-11T22:56:18.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.803+0000 7fa3097fa640 1 -- 172.21.15.152:0/265638253 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa31c01de50 con 0x55f71347e780 2024-09-11T22:56:18.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.803+0000 7fa3097fa640 1 -- 172.21.15.152:0/265638253 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c022040 con 0x55f71347e780 2024-09-11T22:56:18.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.804+0000 7fa32bfb4740 1 -- 172.21.15.152:0/265638253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f71347e780 msgr2=0x55f71347a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:18.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.804+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/265638253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f71347e780 0x55f71347a160 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa31c0076b0 tx=0x7fa31c011e30 comp rx=0 tx=0).stop 2024-09-11T22:56:18.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.804+0000 7fa32bfb4740 1 -- 172.21.15.152:0/265638253 shutdown_connections 2024-09-11T22:56:18.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.804+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/265638253 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f71347e780 0x55f71347a160 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.804+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/265638253 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f713470b40 0x55f713479c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.804+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/265638253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71346f5d0 0x55f71348a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.804+0000 7fa32bfb4740 1 -- 172.21.15.152:0/265638253 >> 172.21.15.152:0/265638253 conn(0x55f713438310 msgr2=0x55f71347b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:18.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.804+0000 7fa32bfb4740 1 -- 172.21.15.152:0/265638253 shutdown_connections 2024-09-11T22:56:18.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.804+0000 7fa32bfb4740 1 -- 172.21.15.152:0/265638253 wait complete. 2024-09-11T22:56:18.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.804+0000 7fa32bfb4740 1 -- start start 2024-09-11T22:56:18.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.804+0000 7fa32bfb4740 1 Processor -- start 2024-09-11T22:56:18.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa32bfb4740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f71346f5d0 0x55f713486b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa32bfb4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f713470b40 0x55f7134870d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa32bfb4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71347e780 0x55f713482160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa32bfb4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241ac930 con 0x55f713470b40 2024-09-11T22:56:18.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa32bfb4740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241ac260 con 0x55f71346f5d0 2024-09-11T22:56:18.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa32bfb4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241a9480 con 0x55f71347e780 2024-09-11T22:56:18.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa32bfb4740 10 client.0 _ll_register_callbacks cb 0x7fa31c016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:18.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa322ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f713470b40 0x55f7134870d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa322ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f713470b40 0x55f7134870d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38744/0 (socket says 172.21.15.152:38744) 2024-09-11T22:56:18.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa322ffd640 1 -- 172.21.15.152:0/1981099986 learned_addr learned my addr 172.21.15.152:0/1981099986 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:18.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa3237fe640 1 --2- 172.21.15.152:0/1981099986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f71346f5d0 0x55f713486b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa323fff640 1 --2- 172.21.15.152:0/1981099986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71347e780 0x55f713482160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa322ffd640 1 -- 172.21.15.152:0/1981099986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71347e780 msgr2=0x55f713482160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:18.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa322ffd640 1 --2- 172.21.15.152:0/1981099986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71347e780 0x55f713482160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa322ffd640 1 -- 172.21.15.152:0/1981099986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f71346f5d0 msgr2=0x55f713486b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:18.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa322ffd640 1 --2- 172.21.15.152:0/1981099986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f71346f5d0 0x55f713486b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa322ffd640 1 -- 172.21.15.152:0/1981099986 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f713482990 con 0x55f713470b40 2024-09-11T22:56:18.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa323fff640 1 --2- 172.21.15.152:0/1981099986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71347e780 0x55f713482160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:56:18.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa322ffd640 1 --2- 172.21.15.152:0/1981099986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f713470b40 0x55f7134870d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa318097dc0 tx=0x7fa3180047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852a7564b85ab0b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:18.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa302ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:18.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa302ffd640 1 -- 172.21.15.152:0/1981099986 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318020040 con 0x55f713470b40 2024-09-11T22:56:18.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa302ffd640 1 -- 172.21.15.152:0/1981099986 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa318097a60 con 0x55f713470b40 2024-09-11T22:56:18.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa32bfb4740 1 -- 172.21.15.152:0/1981099986 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa31c01de50 con 0x55f713470b40 2024-09-11T22:56:18.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa302ffd640 1 -- 172.21.15.152:0/1981099986 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa31809f7a0 con 0x55f713470b40 2024-09-11T22:56:18.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa32bfb4740 10 client.16506 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:18.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.805+0000 7fa32bfb4740 1 -- 172.21.15.152:0/1981099986 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f7134f8bd0 con 0x55f713470b40 2024-09-11T22:56:18.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.806+0000 7fa302ffd640 1 -- 172.21.15.152:0/1981099986 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa31c01de50 con 0x55f713470b40 2024-09-11T22:56:18.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.806+0000 7fa302ffd640 1 -- 172.21.15.152:0/1981099986 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa31809a840 con 0x55f713470b40 2024-09-11T22:56:18.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.806+0000 7fa32bfb4740 10 client.16506 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:18.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.806+0000 7fa32bfb4740 10 client.16506 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:56:18.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.806+0000 7fa32bfb4740 20 client.16506 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:18.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.806+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/1981099986 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55f7134f9250 0x55f7134fb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:18.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.806+0000 7fa32bfb4740 4 client.16506 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:56:18.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.806+0000 7fa32bfb4740 1 -- 172.21.15.152:0/1981099986 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f713490a80 con 0x55f7134f9250 2024-09-11T22:56:18.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.806+0000 7fa3237fe640 1 --2- 172.21.15.152:0/1981099986 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55f7134f9250 0x55f7134fb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:18.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa3237fe640 1 --2- 172.21.15.152:0/1981099986 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55f7134f9250 0x55f7134fb640 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b05f0d699f662fe server_cookie=a1e17efe2f2d62a9 in_seq=0 out_seq=0 2024-09-11T22:56:18.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa302ffd640 10 client.16506 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:56:18.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa302ffd640 1 -- 172.21.15.152:0/1981099986 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f713490a80 con 0x55f7134f9250 2024-09-11T22:56:18.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa302ffd640 10 client.16506 handle_command_reply: tid=0 2024-09-11T22:56:18.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa32bfb4740 1 client.16506 shutdown 2024-09-11T22:56:18.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa32bfb4740 1 -- 172.21.15.152:0/1981099986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f713470b40 msgr2=0x55f7134870d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:18.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/1981099986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f713470b40 0x55f7134870d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa318097dc0 tx=0x7fa3180047d0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa32bfb4740 1 -- 172.21.15.152:0/1981099986 shutdown_connections 2024-09-11T22:56:18.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/1981099986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f71347e780 0x55f713482160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/1981099986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f713470b40 0x55f7134870d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/1981099986 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55f7134f9250 0x55f7134fb640 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa32bfb4740 1 --2- 172.21.15.152:0/1981099986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f71346f5d0 0x55f713486b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa32bfb4740 1 -- 172.21.15.152:0/1981099986 >> 172.21.15.152:0/1981099986 conn(0x55f713438310 msgr2=0x55f713479b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:18.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa302ffd640 0 client.16506 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:56:18.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa32bfb4740 1 -- 172.21.15.152:0/1981099986 shutdown_connections 2024-09-11T22:56:18.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa32bfb4740 1 -- 172.21.15.152:0/1981099986 wait complete. 2024-09-11T22:56:18.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.807+0000 7fa32bfb4740 20 client.16506 trim_cache size 0 max 16384 2024-09-11T22:56:18.827 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:56:18.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.808+0000 7fa329bdb640 1 -- 172.21.15.152:0/3725484074 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa30c0a2cf0 msgr2=0x7fa30c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:18.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.808+0000 7fa329bdb640 1 --2- 172.21.15.152:0/3725484074 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa30c0a2cf0 0x7fa30c0a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa31c005cc0 tx=0x7fa31c00e720 comp rx=0 tx=0).stop 2024-09-11T22:56:18.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.808+0000 7fa329bdb640 1 -- 172.21.15.152:0/3725484074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa324102790 msgr2=0x7fa324198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:18.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.808+0000 7fa329bdb640 1 --2- 172.21.15.152:0/3725484074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa324102790 0x7fa324198910 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa31800d770 tx=0x7fa31800dc30 comp rx=0 tx=0).stop 2024-09-11T22:56:18.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.808+0000 7fa329bdb640 1 -- 172.21.15.152:0/3725484074 shutdown_connections 2024-09-11T22:56:18.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.808+0000 7fa329bdb640 1 --2- 172.21.15.152:0/3725484074 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa30c0a2cf0 0x7fa30c0a51b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.808+0000 7fa329bdb640 1 --2- 172.21.15.152:0/3725484074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa324106f10 0x7fa32419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.808+0000 7fa329bdb640 1 --2- 172.21.15.152:0/3725484074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa324102790 0x7fa324198910 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.808+0000 7fa329bdb640 1 --2- 172.21.15.152:0/3725484074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa324101de0 0x7fa3241983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:18.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.808+0000 7fa329bdb640 1 -- 172.21.15.152:0/3725484074 >> 172.21.15.152:0/3725484074 conn(0x7fa3240fd8c0 msgr2=0x7fa3240febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:18.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.808+0000 7fa329bdb640 1 -- 172.21.15.152:0/3725484074 shutdown_connections 2024-09-11T22:56:18.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:18.808+0000 7fa329bdb640 1 -- 172.21.15.152:0/3725484074 wait complete. 2024-09-11T22:56:19.704 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:56:19.953 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:56:20.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.473+0000 7f86c1c3e640 1 -- 172.21.15.152:0/74884536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86bc101420 msgr2=0x7f86bc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:20.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.473+0000 7f86c1c3e640 1 --2- 172.21.15.152:0/74884536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86bc101420 0x7f86bc10d950 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009cd0 tx=0x7f86b402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.473+0000 7f86c1c3e640 1 -- 172.21.15.152:0/74884536 shutdown_connections 2024-09-11T22:56:20.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.473+0000 7f86c1c3e640 1 --2- 172.21.15.152:0/74884536 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86bc101420 0x7f86bc10d950 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.473+0000 7f86c1c3e640 1 --2- 172.21.15.152:0/74884536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc100a80 0x7f86bc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.473+0000 7f86c1c3e640 1 --2- 172.21.15.152:0/74884536 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86bc106a80 0x7f86bc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.473+0000 7f86c1c3e640 1 -- 172.21.15.152:0/74884536 >> 172.21.15.152:0/74884536 conn(0x7f86bc0fc7a0 msgr2=0x7f86bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:20.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.473+0000 7f86c1c3e640 1 -- 172.21.15.152:0/74884536 shutdown_connections 2024-09-11T22:56:20.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.473+0000 7f86c1c3e640 1 -- 172.21.15.152:0/74884536 wait complete. 2024-09-11T22:56:20.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.474+0000 7f86c1c3e640 1 Processor -- start 2024-09-11T22:56:20.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.474+0000 7f86c1c3e640 1 -- start start 2024-09-11T22:56:20.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.474+0000 7f86c1c3e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86bc100a80 0x7f86bc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.474+0000 7f86c1c3e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc101420 0x7f86bc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.474+0000 7f86c1c3e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86bc106a80 0x7f86bc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.474+0000 7f86c1c3e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc110e80 con 0x7f86bc106a80 2024-09-11T22:56:20.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.474+0000 7f86c1c3e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc110d00 con 0x7f86bc101420 2024-09-11T22:56:20.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.474+0000 7f86c1c3e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc111000 con 0x7f86bc100a80 2024-09-11T22:56:20.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.474+0000 7f86bbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86bc106a80 0x7f86bc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.474+0000 7f86bbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86bc106a80 0x7f86bc19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38766/0 (socket says 172.21.15.152:38766) 2024-09-11T22:56:20.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.474+0000 7f86bbfff640 1 -- 172.21.15.152:0/2424157661 learned_addr learned my addr 172.21.15.152:0/2424157661 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:20.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.474+0000 7f86bb7fe640 1 --2- 172.21.15.152:0/2424157661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86bc100a80 0x7f86bc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.474+0000 7f86b2ffd640 1 --2- 172.21.15.152:0/2424157661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc101420 0x7f86bc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.475+0000 7f86bbfff640 1 -- 172.21.15.152:0/2424157661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86bc100a80 msgr2=0x7f86bc1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:20.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.475+0000 7f86bbfff640 1 --2- 172.21.15.152:0/2424157661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86bc100a80 0x7f86bc1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.475+0000 7f86bbfff640 1 -- 172.21.15.152:0/2424157661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc101420 msgr2=0x7f86bc1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:20.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.475+0000 7f86bbfff640 1 --2- 172.21.15.152:0/2424157661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc101420 0x7f86bc1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.475+0000 7f86bbfff640 1 -- 172.21.15.152:0/2424157661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc19d350 con 0x7f86bc106a80 2024-09-11T22:56:20.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.475+0000 7f86bbfff640 1 --2- 172.21.15.152:0/2424157661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86bc106a80 0x7f86bc19cc70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f86b4005cc0 tx=0x7f86b4032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:20.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.475+0000 7f86b97fa640 1 -- 172.21.15.152:0/2424157661 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4004040 con 0x7f86bc106a80 2024-09-11T22:56:20.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.475+0000 7f86b97fa640 1 -- 172.21.15.152:0/2424157661 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86b40046c0 con 0x7f86bc106a80 2024-09-11T22:56:20.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.475+0000 7f86c1c3e640 1 -- 172.21.15.152:0/2424157661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc19d5e0 con 0x7f86bc106a80 2024-09-11T22:56:20.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.475+0000 7f86b97fa640 1 -- 172.21.15.152:0/2424157661 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4030dc0 con 0x7f86bc106a80 2024-09-11T22:56:20.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.475+0000 7f86c1c3e640 1 -- 172.21.15.152:0/2424157661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc1a4f20 con 0x7f86bc106a80 2024-09-11T22:56:20.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.477+0000 7f86b97fa640 1 -- 172.21.15.152:0/2424157661 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f86b40041e0 con 0x7f86bc106a80 2024-09-11T22:56:20.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.477+0000 7f86c4017740 1 Processor -- start 2024-09-11T22:56:20.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.477+0000 7f86c4017740 1 -- start start 2024-09-11T22:56:20.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86c4017740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86c4017740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86c4017740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5de780 0x55a8cf5e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86c4017740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a8cf5352f0 con 0x55a8cf5de780 2024-09-11T22:56:20.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86c4017740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a8cf4dccf0 con 0x55a8cf5cf5d0 2024-09-11T22:56:20.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86c4017740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a8cf5312b0 con 0x55a8cf5d0b40 2024-09-11T22:56:20.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86b97fa640 1 --2- 172.21.15.152:0/2424157661 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f86a80a2cf0 0x7f86a80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86b2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5de780 0x55a8cf5e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86bb7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86b2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5de780 0x55a8cf5e0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38778/0 (socket says 172.21.15.152:38778) 2024-09-11T22:56:20.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86b2ffd640 1 -- 172.21.15.152:0/1177348321 learned_addr learned my addr 172.21.15.152:0/1177348321 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:20.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86bb7fe640 1 --2- 172.21.15.152:0/1177348321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5cfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:39388/0 (socket says 172.21.15.152:39388) 2024-09-11T22:56:20.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86b2ffd640 1 --2- 172.21.15.152:0/1177348321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86bbfff640 1 --2- 172.21.15.152:0/2424157661 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f86a80a2cf0 0x7f86a80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86b97fa640 1 -- 172.21.15.152:0/2424157661 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f86b402e030 con 0x7f86bc106a80 2024-09-11T22:56:20.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86bb7fe640 1 -- 172.21.15.152:0/1177348321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5d0b40 msgr2=0x55a8cf5cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:20.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86bb7fe640 1 --2- 172.21.15.152:0/1177348321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86bb7fe640 1 -- 172.21.15.152:0/1177348321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5de780 msgr2=0x55a8cf5e0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:20.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86bb7fe640 1 --2- 172.21.15.152:0/1177348321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5de780 0x55a8cf5e0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86bb7fe640 1 -- 172.21.15.152:0/1177348321 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8cf4c13a0 con 0x55a8cf5cf5d0 2024-09-11T22:56:20.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.478+0000 7f86bb7fe640 1 --2- 172.21.15.152:0/1177348321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5cfa10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00ab70 tx=0x7f86ac00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc5f3e5750722deb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:20.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.479+0000 7f86bbfff640 1 --2- 172.21.15.152:0/2424157661 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f86a80a2cf0 0x7f86a80a51b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f86b40bc6f0 tx=0x7f86b40aa040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:20.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.479+0000 7f86b1ffb640 1 -- 172.21.15.152:0/1177348321 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac005220 con 0x55a8cf5cf5d0 2024-09-11T22:56:20.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.479+0000 7f86b1ffb640 1 -- 172.21.15.152:0/1177348321 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ac004c80 con 0x55a8cf5cf5d0 2024-09-11T22:56:20.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.479+0000 7f86b1ffb640 1 -- 172.21.15.152:0/1177348321 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac018750 con 0x55a8cf5cf5d0 2024-09-11T22:56:20.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.479+0000 7f86c4017740 1 -- 172.21.15.152:0/1177348321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 msgr2=0x55a8cf5cfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:20.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.479+0000 7f86c4017740 1 --2- 172.21.15.152:0/1177348321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5cfa10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00ab70 tx=0x7f86ac00f790 comp rx=0 tx=0).stop 2024-09-11T22:56:20.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.479+0000 7f86c4017740 1 -- 172.21.15.152:0/1177348321 shutdown_connections 2024-09-11T22:56:20.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.479+0000 7f86c4017740 1 --2- 172.21.15.152:0/1177348321 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5de780 0x55a8cf5e0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.479+0000 7f86c4017740 1 --2- 172.21.15.152:0/1177348321 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5cfa10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.479+0000 7f86c4017740 1 --2- 172.21.15.152:0/1177348321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.480+0000 7f86c4017740 1 -- 172.21.15.152:0/1177348321 >> 172.21.15.152:0/1177348321 conn(0x55a8cf598310 msgr2=0x55a8cf5cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:20.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.480+0000 7f86c4017740 1 -- 172.21.15.152:0/1177348321 shutdown_connections 2024-09-11T22:56:20.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.480+0000 7f86c4017740 1 -- 172.21.15.152:0/1177348321 wait complete. 2024-09-11T22:56:20.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.480+0000 7f86c4017740 1 -- start start 2024-09-11T22:56:20.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.480+0000 7f86c4017740 1 Processor -- start 2024-09-11T22:56:20.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.480+0000 7f86c4017740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5de0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.480+0000 7f86c4017740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5db3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.480+0000 7f86c4017740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5de780 0x55a8cf5db900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.509 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.480+0000 7f86c4017740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a8cf5e1480 con 0x55a8cf5de780 2024-09-11T22:56:20.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.480+0000 7f86c4017740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a8cf4f0060 con 0x55a8cf5d0b40 2024-09-11T22:56:20.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.480+0000 7f86c4017740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a8cf5e1600 con 0x55a8cf5cf5d0 2024-09-11T22:56:20.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f86c4017740 10 client.0 _ll_register_callbacks cb 0x55a8cf2438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:20.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f86b2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5de780 0x55a8cf5db900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f86b2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5de780 0x55a8cf5db900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38784/0 (socket says 172.21.15.152:38784) 2024-09-11T22:56:20.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f86b2ffd640 1 -- 172.21.15.152:0/243369732 learned_addr learned my addr 172.21.15.152:0/243369732 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:20.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f86b2ffd640 1 --2- 172.21.15.152:0/243369732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5de0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f86bb7fe640 1 --2- 172.21.15.152:0/243369732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5db3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f86b2ffd640 1 -- 172.21.15.152:0/243369732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5cf5d0 msgr2=0x55a8cf5de0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:20.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f86b2ffd640 1 --2- 172.21.15.152:0/243369732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5de0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f86b2ffd640 1 -- 172.21.15.152:0/243369732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5d0b40 msgr2=0x55a8cf5db3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:20.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f86b2ffd640 1 --2- 172.21.15.152:0/243369732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5db3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.510 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f86b2ffd640 1 -- 172.21.15.152:0/243369732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8cf2a63c0 con 0x55a8cf5de780 2024-09-11T22:56:20.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f86b2ffd640 1 --2- 172.21.15.152:0/243369732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5de780 0x55a8cf5db900 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f86a4012d60 tx=0x7f86a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994874ac62a86823 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:20.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f869affd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:20.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f869affd640 1 -- 172.21.15.152:0/243369732 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4004030 con 0x55a8cf5de780 2024-09-11T22:56:20.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f869affd640 1 -- 172.21.15.152:0/243369732 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f86a401a040 con 0x55a8cf5de780 2024-09-11T22:56:20.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f86c4017740 1 -- 172.21.15.152:0/243369732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ac004c80 con 0x55a8cf5de780 2024-09-11T22:56:20.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f869affd640 1 -- 172.21.15.152:0/243369732 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f86a40041d0 con 0x55a8cf5de780 2024-09-11T22:56:20.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f86c4017740 10 client.16518 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:20.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.481+0000 7f86c4017740 1 -- 172.21.15.152:0/243369732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a8cf5ecf40 con 0x55a8cf5de780 2024-09-11T22:56:20.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.482+0000 7f869affd640 1 -- 172.21.15.152:0/243369732 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ac004c80 con 0x55a8cf5de780 2024-09-11T22:56:20.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.482+0000 7f869affd640 1 -- 172.21.15.152:0/243369732 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f86a401d070 con 0x55a8cf5de780 2024-09-11T22:56:20.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.482+0000 7f86c4017740 10 client.16518 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:20.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.482+0000 7f86c4017740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:56:20.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.482+0000 7f86c4017740 20 client.16518 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:20.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.482+0000 7f86c4017740 1 --2- 172.21.15.152:0/243369732 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a8cf5ee350 0x55a8cf5f0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.482+0000 7f86c4017740 4 client.16518 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:56:20.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.482+0000 7f86c4017740 1 -- 172.21.15.152:0/243369732 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8cf542860 con 0x55a8cf5ee350 2024-09-11T22:56:20.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.482+0000 7f86bb7fe640 1 --2- 172.21.15.152:0/243369732 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a8cf5ee350 0x55a8cf5f0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.483+0000 7f869affd640 10 client.16518 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:20.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.483+0000 7f86bb7fe640 1 --2- 172.21.15.152:0/243369732 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a8cf5ee350 0x55a8cf5f0740 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc884509dabb752e server_cookie=80fe414d64eb971d in_seq=0 out_seq=0 2024-09-11T22:56:20.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.483+0000 7f869affd640 1 -- 172.21.15.152:0/243369732 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8cf542860 con 0x55a8cf5ee350 2024-09-11T22:56:20.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.483+0000 7f869affd640 10 client.16518 handle_command_reply: tid=0 2024-09-11T22:56:20.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.483+0000 7f86c4017740 1 client.16518 shutdown 2024-09-11T22:56:20.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.483+0000 7f86c4017740 1 -- 172.21.15.152:0/243369732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5de780 msgr2=0x55a8cf5db900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:20.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.483+0000 7f86c4017740 1 --2- 172.21.15.152:0/243369732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5de780 0x55a8cf5db900 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f86a4012d60 tx=0x7f86a4004730 comp rx=0 tx=0).stop 2024-09-11T22:56:20.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.484+0000 7f86c4017740 1 -- 172.21.15.152:0/243369732 shutdown_connections 2024-09-11T22:56:20.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.484+0000 7f86c4017740 1 --2- 172.21.15.152:0/243369732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5de780 0x55a8cf5db900 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.484+0000 7f86c4017740 1 --2- 172.21.15.152:0/243369732 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a8cf5ee350 0x55a8cf5f0740 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.484+0000 7f86c4017740 1 --2- 172.21.15.152:0/243369732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5db3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.484+0000 7f86c4017740 1 --2- 172.21.15.152:0/243369732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5de0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.484+0000 7f86c4017740 1 -- 172.21.15.152:0/243369732 >> 172.21.15.152:0/243369732 conn(0x55a8cf598310 msgr2=0x55a8cf5d1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:20.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.484+0000 7f869affd640 0 client.16518 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:20.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.484+0000 7f86c4017740 1 -- 172.21.15.152:0/243369732 shutdown_connections 2024-09-11T22:56:20.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.484+0000 7f86c4017740 1 -- 172.21.15.152:0/243369732 wait complete. 2024-09-11T22:56:20.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.484+0000 7f86c4017740 20 client.16518 trim_cache size 0 max 16384 2024-09-11T22:56:20.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.496+0000 7f86c4017740 1 Processor -- start 2024-09-11T22:56:20.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86c4017740 1 -- start start 2024-09-11T22:56:20.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86c4017740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5ea720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86c4017740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5d9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86c4017740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5de780 0x55a8cf5da1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86c4017740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1ab470 con 0x55a8cf5d0b40 2024-09-11T22:56:20.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86c4017740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1acfb0 con 0x55a8cf5cf5d0 2024-09-11T22:56:20.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86c4017740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1abb40 con 0x55a8cf5de780 2024-09-11T22:56:20.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86bb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5d9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86bb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5d9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38800/0 (socket says 172.21.15.152:38800) 2024-09-11T22:56:20.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86bb7fe640 1 -- 172.21.15.152:0/869600450 learned_addr learned my addr 172.21.15.152:0/869600450 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:20.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86b2ffd640 1 --2- 172.21.15.152:0/869600450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5ea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86bb7fe640 1 -- 172.21.15.152:0/869600450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5de780 msgr2=0x55a8cf5da1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:20.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86bb7fe640 1 --2- 172.21.15.152:0/869600450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5de780 0x55a8cf5da1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86bb7fe640 1 -- 172.21.15.152:0/869600450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 msgr2=0x55a8cf5ea720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:20.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86bb7fe640 1 --2- 172.21.15.152:0/869600450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5ea720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86bb7fe640 1 -- 172.21.15.152:0/869600450 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8cf5cbed0 con 0x55a8cf5d0b40 2024-09-11T22:56:20.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86bb7fe640 1 --2- 172.21.15.152:0/869600450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5d9c70 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00ab70 tx=0x7f86ac005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4677cc83ac7c3c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:20.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86b17fa640 1 -- 172.21.15.152:0/869600450 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac033070 con 0x55a8cf5d0b40 2024-09-11T22:56:20.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86b17fa640 1 -- 172.21.15.152:0/869600450 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ac00a2d0 con 0x55a8cf5d0b40 2024-09-11T22:56:20.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.497+0000 7f86b17fa640 1 -- 172.21.15.152:0/869600450 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0051f0 con 0x55a8cf5d0b40 2024-09-11T22:56:20.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.498+0000 7f86c4017740 1 -- 172.21.15.152:0/869600450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5d0b40 msgr2=0x55a8cf5d9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:20.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.498+0000 7f86c4017740 1 --2- 172.21.15.152:0/869600450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5d9c70 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00ab70 tx=0x7f86ac005e40 comp rx=0 tx=0).stop 2024-09-11T22:56:20.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.498+0000 7f86c4017740 1 -- 172.21.15.152:0/869600450 shutdown_connections 2024-09-11T22:56:20.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.498+0000 7f86c4017740 1 --2- 172.21.15.152:0/869600450 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5de780 0x55a8cf5da1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.498+0000 7f86c4017740 1 --2- 172.21.15.152:0/869600450 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5d9c70 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.498+0000 7f86c4017740 1 --2- 172.21.15.152:0/869600450 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5ea720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.498+0000 7f86c4017740 1 -- 172.21.15.152:0/869600450 >> 172.21.15.152:0/869600450 conn(0x55a8cf598310 msgr2=0x55a8cf5db360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:20.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.498+0000 7f86c4017740 1 -- 172.21.15.152:0/869600450 shutdown_connections 2024-09-11T22:56:20.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.498+0000 7f86c4017740 1 -- 172.21.15.152:0/869600450 wait complete. 2024-09-11T22:56:20.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.498+0000 7f86c4017740 1 -- start start 2024-09-11T22:56:20.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86c4017740 1 Processor -- start 2024-09-11T22:56:20.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86c4017740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86c4017740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5e6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86c4017740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5de780 0x55a8cf5d8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86c4017740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1ac8e0 con 0x55a8cf5d0b40 2024-09-11T22:56:20.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86c4017740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1ac210 con 0x55a8cf5cf5d0 2024-09-11T22:56:20.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86c4017740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1a9430 con 0x55a8cf5de780 2024-09-11T22:56:20.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86c4017740 10 client.0 _ll_register_callbacks cb 0x7f86a4009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:20.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86bb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5e6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86bb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5e6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38802/0 (socket says 172.21.15.152:38802) 2024-09-11T22:56:20.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86bb7fe640 1 -- 172.21.15.152:0/1962711207 learned_addr learned my addr 172.21.15.152:0/1962711207 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:20.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86b2ffd640 1 --2- 172.21.15.152:0/1962711207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86bb7fe640 1 -- 172.21.15.152:0/1962711207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5de780 msgr2=0x55a8cf5d8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:20.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86b2ffd640 1 --2- 172.21.15.152:0/1962711207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5de780 0x55a8cf5d8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86bb7fe640 1 --2- 172.21.15.152:0/1962711207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5de780 0x55a8cf5d8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86bb7fe640 1 -- 172.21.15.152:0/1962711207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 msgr2=0x55a8cf5e6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:20.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86bb7fe640 1 --2- 172.21.15.152:0/1962711207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5e6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86bb7fe640 1 -- 172.21.15.152:0/1962711207 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8cf5d8ce0 con 0x55a8cf5d0b40 2024-09-11T22:56:20.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.499+0000 7f86b2ffd640 1 --2- 172.21.15.152:0/1962711207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5de780 0x55a8cf5d8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:20.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.500+0000 7f86bb7fe640 1 --2- 172.21.15.152:0/1962711207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5e6fb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00a900 tx=0x7f86ac0056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c700b65ee5f019 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:20.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.500+0000 7f869affd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:20.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.500+0000 7f869affd640 1 -- 172.21.15.152:0/1962711207 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac033030 con 0x55a8cf5d0b40 2024-09-11T22:56:20.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.500+0000 7f86c4017740 1 -- 172.21.15.152:0/1962711207 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ac00a2d0 con 0x55a8cf5d0b40 2024-09-11T22:56:20.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.500+0000 7f869affd640 1 -- 172.21.15.152:0/1962711207 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f86ac02b040 con 0x55a8cf5d0b40 2024-09-11T22:56:20.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.500+0000 7f869affd640 1 -- 172.21.15.152:0/1962711207 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f86ac004660 con 0x55a8cf5d0b40 2024-09-11T22:56:20.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.500+0000 7f86c4017740 10 client.16524 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:20.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.500+0000 7f86c4017740 1 -- 172.21.15.152:0/1962711207 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a8cf5f5da0 con 0x55a8cf5d0b40 2024-09-11T22:56:20.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.500+0000 7f869affd640 1 -- 172.21.15.152:0/1962711207 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ac00a2d0 con 0x55a8cf5d0b40 2024-09-11T22:56:20.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.500+0000 7f869affd640 1 -- 172.21.15.152:0/1962711207 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f86ac0428c0 con 0x55a8cf5d0b40 2024-09-11T22:56:20.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.500+0000 7f86c4017740 10 client.16524 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:20.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.500+0000 7f86c4017740 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:56:20.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.500+0000 7f86c4017740 20 client.16524 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:20.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.500+0000 7f86c4017740 1 --2- 172.21.15.152:0/1962711207 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a8cf658fc0 0x55a8cf65b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:20.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.500+0000 7f86c4017740 4 client.16524 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:56:20.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.500+0000 7f86c4017740 1 -- 172.21.15.152:0/1962711207 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8cf53cd70 con 0x55a8cf658fc0 2024-09-11T22:56:20.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.501+0000 7f86b2ffd640 1 --2- 172.21.15.152:0/1962711207 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a8cf658fc0 0x55a8cf65b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:20.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.501+0000 7f869affd640 10 client.16524 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:20.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.501+0000 7f86b2ffd640 1 --2- 172.21.15.152:0/1962711207 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a8cf658fc0 0x55a8cf65b3b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3692c09bdefc3c5f server_cookie=4a1ae4712eafbed in_seq=0 out_seq=0 2024-09-11T22:56:20.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.501+0000 7f869affd640 1 -- 172.21.15.152:0/1962711207 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8cf53cd70 con 0x55a8cf658fc0 2024-09-11T22:56:20.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.501+0000 7f869affd640 10 client.16524 handle_command_reply: tid=0 2024-09-11T22:56:20.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.501+0000 7f86c4017740 1 client.16524 shutdown 2024-09-11T22:56:20.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.502+0000 7f86c4017740 1 -- 172.21.15.152:0/1962711207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5d0b40 msgr2=0x55a8cf5e6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:20.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.502+0000 7f86c4017740 1 --2- 172.21.15.152:0/1962711207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5e6fb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00a900 tx=0x7f86ac0056b0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.502+0000 7f86c4017740 1 -- 172.21.15.152:0/1962711207 shutdown_connections 2024-09-11T22:56:20.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.502+0000 7f86c4017740 1 --2- 172.21.15.152:0/1962711207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a8cf5de780 0x55a8cf5d8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.502+0000 7f86c4017740 1 --2- 172.21.15.152:0/1962711207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a8cf5d0b40 0x55a8cf5e6fb0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.502+0000 7f86c4017740 1 --2- 172.21.15.152:0/1962711207 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a8cf658fc0 0x55a8cf65b3b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.502+0000 7f86c4017740 1 --2- 172.21.15.152:0/1962711207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a8cf5cf5d0 0x55a8cf5e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.502+0000 7f86c4017740 1 -- 172.21.15.152:0/1962711207 >> 172.21.15.152:0/1962711207 conn(0x55a8cf598310 msgr2=0x55a8cf5dcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:20.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.502+0000 7f869affd640 0 client.16524 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:20.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.502+0000 7f86c4017740 1 -- 172.21.15.152:0/1962711207 shutdown_connections 2024-09-11T22:56:20.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.502+0000 7f86c4017740 1 -- 172.21.15.152:0/1962711207 wait complete. 2024-09-11T22:56:20.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.502+0000 7f86c4017740 20 client.16524 trim_cache size 0 max 16384 2024-09-11T22:56:20.520 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:56:20.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.502+0000 7f86c1c3e640 1 -- 172.21.15.152:0/2424157661 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f86a80a2cf0 msgr2=0x7f86a80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:20.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.502+0000 7f86c1c3e640 1 --2- 172.21.15.152:0/2424157661 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f86a80a2cf0 0x7f86a80a51b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f86b40bc6f0 tx=0x7f86b40aa040 comp rx=0 tx=0).stop 2024-09-11T22:56:20.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.502+0000 7f86c1c3e640 1 -- 172.21.15.152:0/2424157661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86bc106a80 msgr2=0x7f86bc19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:20.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.502+0000 7f86c1c3e640 1 --2- 172.21.15.152:0/2424157661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86bc106a80 0x7f86bc19cc70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f86b4005cc0 tx=0x7f86b4032f40 comp rx=0 tx=0).stop 2024-09-11T22:56:20.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.503+0000 7f86c1c3e640 1 -- 172.21.15.152:0/2424157661 shutdown_connections 2024-09-11T22:56:20.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.503+0000 7f86c1c3e640 1 --2- 172.21.15.152:0/2424157661 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f86a80a2cf0 0x7f86a80a51b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.503+0000 7f86c1c3e640 1 --2- 172.21.15.152:0/2424157661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f86bc106a80 0x7f86bc19cc70 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.503+0000 7f86c1c3e640 1 --2- 172.21.15.152:0/2424157661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86bc101420 0x7f86bc1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.503+0000 7f86c1c3e640 1 --2- 172.21.15.152:0/2424157661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86bc100a80 0x7f86bc1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:20.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.503+0000 7f86c1c3e640 1 -- 172.21.15.152:0/2424157661 >> 172.21.15.152:0/2424157661 conn(0x7f86bc0fc7a0 msgr2=0x7f86bc06a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:20.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.503+0000 7f86c1c3e640 1 -- 172.21.15.152:0/2424157661 shutdown_connections 2024-09-11T22:56:20.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:20.503+0000 7f86c1c3e640 1 -- 172.21.15.152:0/2424157661 wait complete. 2024-09-11T22:56:20.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:20 smithi191 ceph-mon[35211]: pgmap v106: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-09-11T22:56:20.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:20 smithi204 ceph-mon[35454]: pgmap v106: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-09-11T22:56:20.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:20 smithi152 ceph-mon[31181]: pgmap v106: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-09-11T22:56:22.387 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:56:22.387 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:56:22.644 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:56:22.765 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:22 smithi152 ceph-mon[31181]: pgmap v107: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T22:56:22.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:22 smithi191 ceph-mon[35211]: pgmap v107: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T22:56:22.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:22 smithi204 ceph-mon[35454]: pgmap v107: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T22:56:23.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.161+0000 7f847b5c0640 1 -- 172.21.15.152:0/3926307196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8474101420 msgr2=0x7f847410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:23.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.161+0000 7f847b5c0640 1 --2- 172.21.15.152:0/3926307196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8474101420 0x7f847410d950 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f846c009cd0 tx=0x7f846c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.161+0000 7f847b5c0640 1 -- 172.21.15.152:0/3926307196 shutdown_connections 2024-09-11T22:56:23.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.161+0000 7f847b5c0640 1 --2- 172.21.15.152:0/3926307196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8474101420 0x7f847410d950 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.161+0000 7f847b5c0640 1 --2- 172.21.15.152:0/3926307196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8474100a80 0x7f8474100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.161+0000 7f847b5c0640 1 --2- 172.21.15.152:0/3926307196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8474106a80 0x7f8474106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.161+0000 7f847b5c0640 1 -- 172.21.15.152:0/3926307196 >> 172.21.15.152:0/3926307196 conn(0x7f84740fc7a0 msgr2=0x7f84740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:23.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.161+0000 7f847b5c0640 1 -- 172.21.15.152:0/3926307196 shutdown_connections 2024-09-11T22:56:23.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.161+0000 7f847b5c0640 1 -- 172.21.15.152:0/3926307196 wait complete. 2024-09-11T22:56:23.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.162+0000 7f847b5c0640 1 Processor -- start 2024-09-11T22:56:23.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.162+0000 7f847b5c0640 1 -- start start 2024-09-11T22:56:23.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.162+0000 7f847b5c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8474100a80 0x7f84741991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.162+0000 7f847b5c0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8474101420 0x7f8474199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.162+0000 7f847b5c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8474106a80 0x7f84741a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.162+0000 7f847b5c0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8474110e80 con 0x7f8474101420 2024-09-11T22:56:23.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.162+0000 7f847b5c0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8474110d00 con 0x7f8474100a80 2024-09-11T22:56:23.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.162+0000 7f847b5c0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8474111000 con 0x7f8474106a80 2024-09-11T22:56:23.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.162+0000 7f8478b5b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8474101420 0x7f8474199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.162+0000 7f8478b5b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8474101420 0x7f8474199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38826/0 (socket says 172.21.15.152:38826) 2024-09-11T22:56:23.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.162+0000 7f8478b5b640 1 -- 172.21.15.152:0/1772565002 learned_addr learned my addr 172.21.15.152:0/1772565002 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:23.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.162+0000 7f847935c640 1 --2- 172.21.15.152:0/1772565002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8474100a80 0x7f84741991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.163+0000 7f8479b5d640 1 --2- 172.21.15.152:0/1772565002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8474106a80 0x7f84741a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.163+0000 7f8478b5b640 1 -- 172.21.15.152:0/1772565002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8474106a80 msgr2=0x7f84741a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:23.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.163+0000 7f8478b5b640 1 --2- 172.21.15.152:0/1772565002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8474106a80 0x7f84741a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.163+0000 7f8478b5b640 1 -- 172.21.15.152:0/1772565002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8474100a80 msgr2=0x7f84741991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:23.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.163+0000 7f8478b5b640 1 --2- 172.21.15.152:0/1772565002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8474100a80 0x7f84741991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.163+0000 7f8478b5b640 1 -- 172.21.15.152:0/1772565002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84741a47b0 con 0x7f8474101420 2024-09-11T22:56:23.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.163+0000 7f8479b5d640 1 --2- 172.21.15.152:0/1772565002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8474106a80 0x7f84741a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:23.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.163+0000 7f8478b5b640 1 --2- 172.21.15.152:0/1772565002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8474101420 0x7f8474199730 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f846400c890 tx=0x7f846400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:23.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.163+0000 7f846a7fc640 1 -- 172.21.15.152:0/1772565002 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464004220 con 0x7f8474101420 2024-09-11T22:56:23.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.163+0000 7f847b5c0640 1 -- 172.21.15.152:0/1772565002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84741a4980 con 0x7f8474101420 2024-09-11T22:56:23.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.163+0000 7f846a7fc640 1 -- 172.21.15.152:0/1772565002 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84640043c0 con 0x7f8474101420 2024-09-11T22:56:23.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.163+0000 7f846a7fc640 1 -- 172.21.15.152:0/1772565002 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464004a00 con 0x7f8474101420 2024-09-11T22:56:23.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.163+0000 7f847b5c0640 1 -- 172.21.15.152:0/1772565002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84741a4ec0 con 0x7f8474101420 2024-09-11T22:56:23.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.165+0000 7f846a7fc640 1 -- 172.21.15.152:0/1772565002 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f84640079b0 con 0x7f8474101420 2024-09-11T22:56:23.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.166+0000 7f847d999740 1 Processor -- start 2024-09-11T22:56:23.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.166+0000 7f847d999740 1 -- start start 2024-09-11T22:56:23.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.166+0000 7f847d999740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e6866cb60 0x560e6866cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.166+0000 7f847d999740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e6866d480 0x560e68677800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.166+0000 7f847d999740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e68677d40 0x560e6867a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.166+0000 7f847d999740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560e685d2280 con 0x560e68677d40 2024-09-11T22:56:23.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.166+0000 7f847d999740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560e68579b10 con 0x560e6866cb60 2024-09-11T22:56:23.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.166+0000 7f847d999740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560e685ce240 con 0x560e6866d480 2024-09-11T22:56:23.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.166+0000 7f846a7fc640 1 --2- 172.21.15.152:0/1772565002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f84600a2cf0 0x7f84600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.166+0000 7f8478b5b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e68677d40 0x560e6867a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.166+0000 7f8479b5d640 1 --2- 172.21.15.152:0/1772565002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f84600a2cf0 0x7f84600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.166+0000 7f847935c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e6866d480 0x560e68677800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.166+0000 7f847935c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e6866d480 0x560e68677800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:48196/0 (socket says 172.21.15.152:48196) 2024-09-11T22:56:23.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.166+0000 7f847935c640 1 -- 172.21.15.152:0/2361377144 learned_addr learned my addr 172.21.15.152:0/2361377144 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:23.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.167+0000 7f8479b5d640 1 --2- 172.21.15.152:0/2361377144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e6866cb60 0x560e6866cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.167+0000 7f846a7fc640 1 -- 172.21.15.152:0/1772565002 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f84640cf0e0 con 0x7f8474101420 2024-09-11T22:56:23.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.167+0000 7f847935c640 1 -- 172.21.15.152:0/2361377144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e6866cb60 msgr2=0x560e6866cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:23.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.167+0000 7f847935c640 1 --2- 172.21.15.152:0/2361377144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e6866cb60 0x560e6866cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.167+0000 7f847935c640 1 -- 172.21.15.152:0/2361377144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e68677d40 msgr2=0x560e6867a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:23.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.167+0000 7f847935c640 1 --2- 172.21.15.152:0/2361377144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e68677d40 0x560e6867a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.167+0000 7f847935c640 1 -- 172.21.15.152:0/2361377144 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e685861f0 con 0x560e6866d480 2024-09-11T22:56:23.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.167+0000 7f8478b5b640 1 --2- 172.21.15.152:0/2361377144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e68677d40 0x560e6867a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:56:23.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.167+0000 7f8479b5d640 1 --2- 172.21.15.152:0/1772565002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f84600a2cf0 0x7f84600a51b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f846c009a60 tx=0x7f846c005da0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:23.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.167+0000 7f847935c640 1 --2- 172.21.15.152:0/2361377144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e6866d480 0x560e68677800 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f847000ac90 tx=0x7f847000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7584378ea0dd95b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:23.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.167+0000 7f845f7fe640 1 -- 172.21.15.152:0/2361377144 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847001d070 con 0x560e6866d480 2024-09-11T22:56:23.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.167+0000 7f845f7fe640 1 -- 172.21.15.152:0/2361377144 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8470018b20 con 0x560e6866d480 2024-09-11T22:56:23.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.167+0000 7f845f7fe640 1 -- 172.21.15.152:0/2361377144 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8470010af0 con 0x560e6866d480 2024-09-11T22:56:23.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.168+0000 7f847d999740 1 -- 172.21.15.152:0/2361377144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e6866d480 msgr2=0x560e68677800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:23.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.168+0000 7f847d999740 1 --2- 172.21.15.152:0/2361377144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e6866d480 0x560e68677800 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f847000ac90 tx=0x7f847000f790 comp rx=0 tx=0).stop 2024-09-11T22:56:23.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.168+0000 7f847d999740 1 -- 172.21.15.152:0/2361377144 shutdown_connections 2024-09-11T22:56:23.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.168+0000 7f847d999740 1 --2- 172.21.15.152:0/2361377144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e68677d40 0x560e6867a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.168+0000 7f847d999740 1 --2- 172.21.15.152:0/2361377144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e6866d480 0x560e68677800 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.168+0000 7f847d999740 1 --2- 172.21.15.152:0/2361377144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e6866cb60 0x560e6866cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.168+0000 7f847d999740 1 -- 172.21.15.152:0/2361377144 >> 172.21.15.152:0/2361377144 conn(0x560e686690f0 msgr2=0x560e686694d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:23.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.168+0000 7f847d999740 1 -- 172.21.15.152:0/2361377144 shutdown_connections 2024-09-11T22:56:23.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.168+0000 7f847d999740 1 -- 172.21.15.152:0/2361377144 wait complete. 2024-09-11T22:56:23.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.168+0000 7f847d999740 1 -- start start 2024-09-11T22:56:23.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.168+0000 7f847d999740 1 Processor -- start 2024-09-11T22:56:23.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.169+0000 7f847d999740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e6866cb60 0x560e68674bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.169+0000 7f847d999740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68675100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.169+0000 7f847d999740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e68677d40 0x560e68675640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.169+0000 7f847d999740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560e6867a910 con 0x560e6866d480 2024-09-11T22:56:23.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.169+0000 7f847d999740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560e6858cf10 con 0x560e6866cb60 2024-09-11T22:56:23.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.169+0000 7f847d999740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560e6867aa90 con 0x560e68677d40 2024-09-11T22:56:23.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.169+0000 7f847d999740 10 client.0 _ll_register_callbacks cb 0x560e68555a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:23.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.169+0000 7f8478b5b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68675100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.169+0000 7f8478b5b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68675100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38832/0 (socket says 172.21.15.152:38832) 2024-09-11T22:56:23.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.169+0000 7f8478b5b640 1 -- 172.21.15.152:0/916696599 learned_addr learned my addr 172.21.15.152:0/916696599 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:23.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.169+0000 7f8479b5d640 1 --2- 172.21.15.152:0/916696599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e68677d40 0x560e68675640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.169+0000 7f8478b5b640 1 -- 172.21.15.152:0/916696599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e68677d40 msgr2=0x560e68675640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:23.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.169+0000 7f8478b5b640 1 --2- 172.21.15.152:0/916696599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e68677d40 0x560e68675640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.169+0000 7f8478b5b640 1 -- 172.21.15.152:0/916696599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e6866cb60 msgr2=0x560e68674bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:23.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.169+0000 7f847935c640 1 --2- 172.21.15.152:0/916696599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e6866cb60 0x560e68674bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.169+0000 7f8478b5b640 1 --2- 172.21.15.152:0/916696599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e6866cb60 0x560e68674bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f8478b5b640 1 -- 172.21.15.152:0/916696599 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e683436a0 con 0x560e6866d480 2024-09-11T22:56:23.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f847935c640 1 --2- 172.21.15.152:0/916696599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e6866cb60 0x560e68674bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:23.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f8478b5b640 1 --2- 172.21.15.152:0/916696599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68675100 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f8464005fd0 tx=0x7f846406b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a41cebf43e2880 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:23.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f845cff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:23.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f845cff9640 1 -- 172.21.15.152:0/916696599 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846401d040 con 0x560e6866d480 2024-09-11T22:56:23.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f845cff9640 1 -- 172.21.15.152:0/916696599 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f84640ccb00 con 0x560e6866d480 2024-09-11T22:56:23.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f847d999740 1 -- 172.21.15.152:0/916696599 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8470018b20 con 0x560e6866d480 2024-09-11T22:56:23.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f845cff9640 1 -- 172.21.15.152:0/916696599 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f84640a0b90 con 0x560e6866d480 2024-09-11T22:56:23.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f847d999740 10 client.16536 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:23.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f847d999740 1 -- 172.21.15.152:0/916696599 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560e686864c0 con 0x560e6866d480 2024-09-11T22:56:23.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f845cff9640 1 -- 172.21.15.152:0/916696599 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8470018b20 con 0x560e6866d480 2024-09-11T22:56:23.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f845cff9640 1 -- 172.21.15.152:0/916696599 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f84640a3e30 con 0x560e6866d480 2024-09-11T22:56:23.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f847d999740 10 client.16536 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:23.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f847d999740 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:56:23.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f847d999740 20 client.16536 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:23.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f847d999740 1 --2- 172.21.15.152:0/916696599 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560e68687920 0x560e68689db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f847d999740 4 client.16536 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:56:23.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.170+0000 7f847d999740 1 -- 172.21.15.152:0/916696599 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e685df7f0 con 0x560e68687920 2024-09-11T22:56:23.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.171+0000 7f847935c640 1 --2- 172.21.15.152:0/916696599 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560e68687920 0x560e68689db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.171+0000 7f847935c640 1 --2- 172.21.15.152:0/916696599 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560e68687920 0x560e68689db0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8f313645932d374 server_cookie=b6c12cb3adb9d0af in_seq=0 out_seq=0 2024-09-11T22:56:23.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.171+0000 7f845cff9640 10 client.16536 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:23.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.172+0000 7f845cff9640 1 -- 172.21.15.152:0/916696599 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e685df7f0 con 0x560e68687920 2024-09-11T22:56:23.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.172+0000 7f845cff9640 10 client.16536 handle_command_reply: tid=0 2024-09-11T22:56:23.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.172+0000 7f847d999740 1 client.16536 shutdown 2024-09-11T22:56:23.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.172+0000 7f847d999740 1 -- 172.21.15.152:0/916696599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 msgr2=0x560e68675100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:23.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.172+0000 7f847d999740 1 --2- 172.21.15.152:0/916696599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68675100 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f8464005fd0 tx=0x7f846406b4b0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.172+0000 7f847d999740 1 -- 172.21.15.152:0/916696599 shutdown_connections 2024-09-11T22:56:23.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.172+0000 7f847d999740 1 --2- 172.21.15.152:0/916696599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e68677d40 0x560e68675640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.172+0000 7f847d999740 1 --2- 172.21.15.152:0/916696599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68675100 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.172+0000 7f847d999740 1 --2- 172.21.15.152:0/916696599 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560e68687920 0x560e68689db0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.172+0000 7f847d999740 1 --2- 172.21.15.152:0/916696599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e6866cb60 0x560e68674bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.172+0000 7f847d999740 1 -- 172.21.15.152:0/916696599 >> 172.21.15.152:0/916696599 conn(0x560e686690f0 msgr2=0x560e68668270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:23.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.172+0000 7f845cff9640 0 client.16536 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:23.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.172+0000 7f847d999740 1 -- 172.21.15.152:0/916696599 shutdown_connections 2024-09-11T22:56:23.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.172+0000 7f847d999740 1 -- 172.21.15.152:0/916696599 wait complete. 2024-09-11T22:56:23.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.172+0000 7f847d999740 20 client.16536 trim_cache size 0 max 16384 2024-09-11T22:56:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.189+0000 7f847d999740 1 Processor -- start 2024-09-11T22:56:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.189+0000 7f847d999740 1 -- start start 2024-09-11T22:56:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f847d999740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e6866cb60 0x560e6866f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f847d999740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68676a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f847d999740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e68677d40 0x560e6868f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f847d999740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f84741ad730 con 0x560e6866d480 2024-09-11T22:56:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f847d999740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f84741af270 con 0x560e68677d40 2024-09-11T22:56:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f847d999740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f84741ade00 con 0x560e6866cb60 2024-09-11T22:56:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f8478b5b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68676a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f8478b5b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68676a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38836/0 (socket says 172.21.15.152:38836) 2024-09-11T22:56:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f8478b5b640 1 -- 172.21.15.152:0/1140905518 learned_addr learned my addr 172.21.15.152:0/1140905518 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f8479b5d640 1 --2- 172.21.15.152:0/1140905518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e68677d40 0x560e6868f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f847935c640 1 --2- 172.21.15.152:0/1140905518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e6866cb60 0x560e6866f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f8478b5b640 1 -- 172.21.15.152:0/1140905518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e6866cb60 msgr2=0x560e6866f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f8478b5b640 1 --2- 172.21.15.152:0/1140905518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e6866cb60 0x560e6866f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f8478b5b640 1 -- 172.21.15.152:0/1140905518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e68677d40 msgr2=0x560e6868f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f8478b5b640 1 --2- 172.21.15.152:0/1140905518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e68677d40 0x560e6868f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f8478b5b640 1 -- 172.21.15.152:0/1140905518 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e68677490 con 0x560e6866d480 2024-09-11T22:56:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f8478b5b640 1 --2- 172.21.15.152:0/1140905518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68676a20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8464097fd0 tx=0x7f8464005380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec942bba35755d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f845effd640 1 -- 172.21.15.152:0/1140905518 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846401d070 con 0x560e6866d480 2024-09-11T22:56:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f845effd640 1 -- 172.21.15.152:0/1140905518 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f846409bd50 con 0x560e6866d480 2024-09-11T22:56:23.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.190+0000 7f845effd640 1 -- 172.21.15.152:0/1140905518 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846406b140 con 0x560e6866d480 2024-09-11T22:56:23.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.191+0000 7f847d999740 1 -- 172.21.15.152:0/1140905518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 msgr2=0x560e68676a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:23.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.191+0000 7f847d999740 1 --2- 172.21.15.152:0/1140905518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68676a20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8464097fd0 tx=0x7f8464005380 comp rx=0 tx=0).stop 2024-09-11T22:56:23.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.191+0000 7f847d999740 1 -- 172.21.15.152:0/1140905518 shutdown_connections 2024-09-11T22:56:23.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.191+0000 7f847d999740 1 --2- 172.21.15.152:0/1140905518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e68677d40 0x560e6868f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.191+0000 7f847d999740 1 --2- 172.21.15.152:0/1140905518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68676a20 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.191+0000 7f847d999740 1 --2- 172.21.15.152:0/1140905518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e6866cb60 0x560e6866f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.191+0000 7f847d999740 1 -- 172.21.15.152:0/1140905518 >> 172.21.15.152:0/1140905518 conn(0x560e686690f0 msgr2=0x560e68676310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:23.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.191+0000 7f847d999740 1 -- 172.21.15.152:0/1140905518 shutdown_connections 2024-09-11T22:56:23.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.191+0000 7f847d999740 1 -- 172.21.15.152:0/1140905518 wait complete. 2024-09-11T22:56:23.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.191+0000 7f847d999740 1 -- start start 2024-09-11T22:56:23.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f847d999740 1 Processor -- start 2024-09-11T22:56:23.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f847d999740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e6866cb60 0x560e686803d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f847d999740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68671820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f847d999740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e68677d40 0x560e68671d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f847d999740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f84741aeba0 con 0x560e6866d480 2024-09-11T22:56:23.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f847d999740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f84741ae4d0 con 0x560e6866cb60 2024-09-11T22:56:23.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f847d999740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f84741ab6f0 con 0x560e68677d40 2024-09-11T22:56:23.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f847d999740 10 client.0 _ll_register_callbacks cb 0x7f8464004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:23.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f8478b5b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68671820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f8478b5b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68671820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38852/0 (socket says 172.21.15.152:38852) 2024-09-11T22:56:23.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f8478b5b640 1 -- 172.21.15.152:0/2123537810 learned_addr learned my addr 172.21.15.152:0/2123537810 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:23.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f847935c640 1 --2- 172.21.15.152:0/2123537810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e6866cb60 0x560e686803d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f8479b5d640 1 --2- 172.21.15.152:0/2123537810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e68677d40 0x560e68671d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f8478b5b640 1 -- 172.21.15.152:0/2123537810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e68677d40 msgr2=0x560e68671d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:23.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f8478b5b640 1 --2- 172.21.15.152:0/2123537810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e68677d40 0x560e68671d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f8478b5b640 1 -- 172.21.15.152:0/2123537810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e6866cb60 msgr2=0x560e686803d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:23.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f8478b5b640 1 --2- 172.21.15.152:0/2123537810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e6866cb60 0x560e686803d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f8478b5b640 1 -- 172.21.15.152:0/2123537810 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e68672540 con 0x560e6866d480 2024-09-11T22:56:23.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.192+0000 7f8479b5d640 1 --2- 172.21.15.152:0/2123537810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e68677d40 0x560e68671d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:23.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.193+0000 7f8478b5b640 1 --2- 172.21.15.152:0/2123537810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68671820 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8464097ab0 tx=0x7f8464000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2fb0ab2d3456d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:23.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.193+0000 7f845cff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:23.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.193+0000 7f845cff9640 1 -- 172.21.15.152:0/2123537810 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846401d040 con 0x560e6866d480 2024-09-11T22:56:23.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.193+0000 7f845cff9640 1 -- 172.21.15.152:0/2123537810 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f8464084d90 con 0x560e6866d480 2024-09-11T22:56:23.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.193+0000 7f847d999740 1 -- 172.21.15.152:0/2123537810 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f846409bd50 con 0x560e6866d480 2024-09-11T22:56:23.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.193+0000 7f845cff9640 1 -- 172.21.15.152:0/2123537810 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f846409b190 con 0x560e6866d480 2024-09-11T22:56:23.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.193+0000 7f847d999740 10 client.16542 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:23.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.193+0000 7f847d999740 1 -- 172.21.15.152:0/2123537810 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560e686f2610 con 0x560e6866d480 2024-09-11T22:56:23.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.193+0000 7f845cff9640 1 -- 172.21.15.152:0/2123537810 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f846409bd50 con 0x560e6866d480 2024-09-11T22:56:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.193+0000 7f845cff9640 1 -- 172.21.15.152:0/2123537810 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8464060be0 con 0x560e6866d480 2024-09-11T22:56:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.193+0000 7f847d999740 10 client.16542 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.193+0000 7f847d999740 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:56:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.193+0000 7f847d999740 20 client.16542 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.193+0000 7f847d999740 1 --2- 172.21.15.152:0/2123537810 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560e686f2b30 0x560e686f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.193+0000 7f847d999740 4 client.16542 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:56:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.193+0000 7f847d999740 1 -- 172.21.15.152:0/2123537810 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560e685d9d00 con 0x560e686f2b30 2024-09-11T22:56:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.194+0000 7f847935c640 1 --2- 172.21.15.152:0/2123537810 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560e686f2b30 0x560e686f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.194+0000 7f845cff9640 10 client.16542 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.194+0000 7f847935c640 1 --2- 172.21.15.152:0/2123537810 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560e686f2b30 0x560e686f4f20 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7603a66dfb39e58 server_cookie=3569a87901c15810 in_seq=0 out_seq=0 2024-09-11T22:56:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.195+0000 7f845cff9640 1 -- 172.21.15.152:0/2123537810 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560e685d9d00 con 0x560e686f2b30 2024-09-11T22:56:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.195+0000 7f845cff9640 10 client.16542 handle_command_reply: tid=0 2024-09-11T22:56:23.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.195+0000 7f847d999740 1 client.16542 shutdown 2024-09-11T22:56:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.195+0000 7f847d999740 1 -- 172.21.15.152:0/2123537810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 msgr2=0x560e68671820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.195+0000 7f847d999740 1 --2- 172.21.15.152:0/2123537810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68671820 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8464097ab0 tx=0x7f8464000bc0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.195+0000 7f847d999740 1 -- 172.21.15.152:0/2123537810 shutdown_connections 2024-09-11T22:56:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.195+0000 7f847d999740 1 --2- 172.21.15.152:0/2123537810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e68677d40 0x560e68671d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.195+0000 7f847d999740 1 --2- 172.21.15.152:0/2123537810 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e6866d480 0x560e68671820 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.195+0000 7f847d999740 1 --2- 172.21.15.152:0/2123537810 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560e686f2b30 0x560e686f4f20 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.195+0000 7f847d999740 1 --2- 172.21.15.152:0/2123537810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e6866cb60 0x560e686803d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.195+0000 7f847d999740 1 -- 172.21.15.152:0/2123537810 >> 172.21.15.152:0/2123537810 conn(0x560e686690f0 msgr2=0x560e686753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.195+0000 7f845cff9640 0 client.16542 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.195+0000 7f847d999740 1 -- 172.21.15.152:0/2123537810 shutdown_connections 2024-09-11T22:56:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.195+0000 7f847d999740 1 -- 172.21.15.152:0/2123537810 wait complete. 2024-09-11T22:56:23.201 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.195+0000 7f847d999740 20 client.16542 trim_cache size 0 max 16384 2024-09-11T22:56:23.202 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:56:23.202 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T22:56:23.202 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "8fc712e6-71cc-4f11-b2aa-94fc461d32f5", 2024-09-11T22:56:23.202 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T22:56:23.202 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:56:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.196+0000 7f847b5c0640 1 -- 172.21.15.152:0/1772565002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f84600a2cf0 msgr2=0x7f84600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.196+0000 7f847b5c0640 1 --2- 172.21.15.152:0/1772565002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f84600a2cf0 0x7f84600a51b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f846c009a60 tx=0x7f846c005da0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.196+0000 7f847b5c0640 1 -- 172.21.15.152:0/1772565002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8474101420 msgr2=0x7f8474199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.196+0000 7f847b5c0640 1 --2- 172.21.15.152:0/1772565002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8474101420 0x7f8474199730 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f846400c890 tx=0x7f846400cc90 comp rx=0 tx=0).stop 2024-09-11T22:56:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.196+0000 7f847b5c0640 1 -- 172.21.15.152:0/1772565002 shutdown_connections 2024-09-11T22:56:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.196+0000 7f847b5c0640 1 --2- 172.21.15.152:0/1772565002 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f84600a2cf0 0x7f84600a51b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.196+0000 7f847b5c0640 1 --2- 172.21.15.152:0/1772565002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8474106a80 0x7f84741a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.196+0000 7f847b5c0640 1 --2- 172.21.15.152:0/1772565002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8474101420 0x7f8474199730 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.196+0000 7f847b5c0640 1 --2- 172.21.15.152:0/1772565002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8474100a80 0x7f84741991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.196+0000 7f847b5c0640 1 -- 172.21.15.152:0/1772565002 >> 172.21.15.152:0/1772565002 conn(0x7f84740fc7a0 msgr2=0x7f84740fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.196+0000 7f847b5c0640 1 -- 172.21.15.152:0/1772565002 shutdown_connections 2024-09-11T22:56:23.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:23.196+0000 7f847b5c0640 1 -- 172.21.15.152:0/1772565002 wait complete. 2024-09-11T22:56:24.035 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:56:24.289 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:56:24.730 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:24 smithi152 ceph-mon[31181]: pgmap v108: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-11T22:56:24.730 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:24 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T22:56:24.730 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:24 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:56:24.730 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:24 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T22:56:24.730 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:24 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:56:24.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:24 smithi191 ceph-mon[35211]: pgmap v108: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-11T22:56:24.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:24 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T22:56:24.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:24 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:56:24.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:24 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T22:56:24.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:24 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:56:24.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:24 smithi204 ceph-mon[35454]: pgmap v108: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-09-11T22:56:24.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:24 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T22:56:24.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:24 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:56:24.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:24 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T22:56:24.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:24 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:56:24.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.811+0000 7f7effcfa640 1 -- 172.21.15.152:0/1925457077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ef8106f10 msgr2=0x7f7ef810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:24.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.811+0000 7f7effcfa640 1 --2- 172.21.15.152:0/1925457077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ef8106f10 0x7f7ef810ea90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009ca0 tx=0x7f7ef002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.811+0000 7f7effcfa640 1 -- 172.21.15.152:0/1925457077 shutdown_connections 2024-09-11T22:56:24.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.811+0000 7f7effcfa640 1 --2- 172.21.15.152:0/1925457077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ef8106f10 0x7f7ef810ea90 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.811+0000 7f7effcfa640 1 --2- 172.21.15.152:0/1925457077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ef8102790 0x7f7ef81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.811+0000 7f7effcfa640 1 --2- 172.21.15.152:0/1925457077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef8101de0 0x7f7ef81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.811+0000 7f7effcfa640 1 -- 172.21.15.152:0/1925457077 >> 172.21.15.152:0/1925457077 conn(0x7f7ef80fd8c0 msgr2=0x7f7ef80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:24.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.811+0000 7f7effcfa640 1 -- 172.21.15.152:0/1925457077 shutdown_connections 2024-09-11T22:56:24.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.811+0000 7f7effcfa640 1 -- 172.21.15.152:0/1925457077 wait complete. 2024-09-11T22:56:24.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.812+0000 7f7effcfa640 1 Processor -- start 2024-09-11T22:56:24.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.812+0000 7f7effcfa640 1 -- start start 2024-09-11T22:56:24.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.812+0000 7f7effcfa640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ef8101de0 0x7f7ef8198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.812+0000 7f7effcfa640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ef8102790 0x7f7ef81987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.812+0000 7f7effcfa640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef8106f10 0x7f7ef819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.812+0000 7f7effcfa640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8111f70 con 0x7f7ef8101de0 2024-09-11T22:56:24.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.812+0000 7f7effcfa640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8111df0 con 0x7f7ef8102790 2024-09-11T22:56:24.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.812+0000 7f7effcfa640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81120f0 con 0x7f7ef8106f10 2024-09-11T22:56:24.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.812+0000 7f7efda96640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ef8101de0 0x7f7ef8198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.812+0000 7f7efda96640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ef8101de0 0x7f7ef8198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38866/0 (socket says 172.21.15.152:38866) 2024-09-11T22:56:24.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.812+0000 7f7efda96640 1 -- 172.21.15.152:0/1658879443 learned_addr learned my addr 172.21.15.152:0/1658879443 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:24.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.812+0000 7f7efd295640 1 --2- 172.21.15.152:0/1658879443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ef8102790 0x7f7ef81987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.812+0000 7f7efda96640 1 -- 172.21.15.152:0/1658879443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef8106f10 msgr2=0x7f7ef819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:24.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.812+0000 7f7efe297640 1 --2- 172.21.15.152:0/1658879443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef8106f10 0x7f7ef819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.813+0000 7f7efda96640 1 --2- 172.21.15.152:0/1658879443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef8106f10 0x7f7ef819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.813+0000 7f7efda96640 1 -- 172.21.15.152:0/1658879443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ef8102790 msgr2=0x7f7ef81987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:24.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.813+0000 7f7efda96640 1 --2- 172.21.15.152:0/1658879443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ef8102790 0x7f7ef81987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.813+0000 7f7efda96640 1 -- 172.21.15.152:0/1658879443 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef819d230 con 0x7f7ef8101de0 2024-09-11T22:56:24.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.813+0000 7f7efe297640 1 --2- 172.21.15.152:0/1658879443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef8106f10 0x7f7ef819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:24.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.813+0000 7f7efda96640 1 --2- 172.21.15.152:0/1658879443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ef8101de0 0x7f7ef8198280 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400c920 tx=0x7f7ef400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:24.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.813+0000 7f7eeeffd640 1 -- 172.21.15.152:0/1658879443 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4007c70 con 0x7f7ef8101de0 2024-09-11T22:56:24.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.813+0000 7f7eeeffd640 1 -- 172.21.15.152:0/1658879443 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ef4007e10 con 0x7f7ef8101de0 2024-09-11T22:56:24.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.813+0000 7f7effcfa640 1 -- 172.21.15.152:0/1658879443 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef819d520 con 0x7f7ef8101de0 2024-09-11T22:56:24.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.813+0000 7f7eeeffd640 1 -- 172.21.15.152:0/1658879443 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4018890 con 0x7f7ef8101de0 2024-09-11T22:56:24.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.813+0000 7f7effcfa640 1 -- 172.21.15.152:0/1658879443 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef81a4e00 con 0x7f7ef8101de0 2024-09-11T22:56:24.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.815+0000 7f7eeeffd640 1 -- 172.21.15.152:0/1658879443 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7ef4018a70 con 0x7f7ef8101de0 2024-09-11T22:56:24.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.816+0000 7f7f020d3740 1 Processor -- start 2024-09-11T22:56:24.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.816+0000 7f7f020d3740 1 -- start start 2024-09-11T22:56:24.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.816+0000 7f7f020d3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d116880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.816+0000 7f7f020d3740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d121140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.816+0000 7f7f020d3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d123a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.816+0000 7f7f020d3740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56479d07c2e0 con 0x56479d116dc0 2024-09-11T22:56:24.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.816+0000 7f7f020d3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56479d023ce0 con 0x56479d1164a0 2024-09-11T22:56:24.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.816+0000 7f7f020d3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56479d0782a0 con 0x56479d121680 2024-09-11T22:56:24.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.816+0000 7f7eeeffd640 1 --2- 172.21.15.152:0/1658879443 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7ee40a2d70 0x7f7ee40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.816+0000 7f7efda96640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d121140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.817+0000 7f7efda96640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d121140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38882/0 (socket says 172.21.15.152:38882) 2024-09-11T22:56:24.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.817+0000 7f7efda96640 1 -- 172.21.15.152:0/1096762211 learned_addr learned my addr 172.21.15.152:0/1096762211 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:24.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.817+0000 7f7efe297640 1 --2- 172.21.15.152:0/1096762211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d116880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.817+0000 7f7efe297640 1 --2- 172.21.15.152:0/1658879443 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7ee40a2d70 0x7f7ee40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.817+0000 7f7efd295640 1 --2- 172.21.15.152:0/1096762211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d123a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.817+0000 7f7eeeffd640 1 -- 172.21.15.152:0/1658879443 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7ef40cf610 con 0x7f7ef8101de0 2024-09-11T22:56:24.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.817+0000 7f7efda96640 1 -- 172.21.15.152:0/1096762211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 msgr2=0x56479d123a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:24.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.817+0000 7f7efda96640 1 --2- 172.21.15.152:0/1096762211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d123a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.817+0000 7f7efda96640 1 -- 172.21.15.152:0/1096762211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 msgr2=0x56479d116880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:24.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.817+0000 7f7efda96640 1 --2- 172.21.15.152:0/1096762211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d116880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.817+0000 7f7efda96640 1 -- 172.21.15.152:0/1096762211 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56479d008440 con 0x56479d116dc0 2024-09-11T22:56:24.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.817+0000 7f7efda96640 1 --2- 172.21.15.152:0/1096762211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d121140 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7ef401b000 tx=0x7f7ef4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f9ff4d1fe98df1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:24.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.817+0000 7f7efe297640 1 --2- 172.21.15.152:0/1658879443 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7ee40a2d70 0x7f7ee40a5230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009bf0 tx=0x7f7ef00095c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:24.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.817+0000 7f7ee3fff640 1 -- 172.21.15.152:0/1096762211 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef409fef0 con 0x56479d116dc0 2024-09-11T22:56:24.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.817+0000 7f7ee3fff640 1 -- 172.21.15.152:0/1096762211 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ef409c240 con 0x56479d116dc0 2024-09-11T22:56:24.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.817+0000 7f7ee3fff640 1 -- 172.21.15.152:0/1096762211 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40ccff0 con 0x56479d116dc0 2024-09-11T22:56:24.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.818+0000 7f7f020d3740 1 -- 172.21.15.152:0/1096762211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 msgr2=0x56479d121140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:24.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.818+0000 7f7f020d3740 1 --2- 172.21.15.152:0/1096762211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d121140 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f7ef401b000 tx=0x7f7ef4000bc0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.818+0000 7f7f020d3740 1 -- 172.21.15.152:0/1096762211 shutdown_connections 2024-09-11T22:56:24.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.818+0000 7f7f020d3740 1 --2- 172.21.15.152:0/1096762211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d123a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.818+0000 7f7f020d3740 1 --2- 172.21.15.152:0/1096762211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d121140 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.818+0000 7f7f020d3740 1 --2- 172.21.15.152:0/1096762211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d116880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.818+0000 7f7f020d3740 1 -- 172.21.15.152:0/1096762211 >> 172.21.15.152:0/1096762211 conn(0x56479d112a30 msgr2=0x56479d112e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:24.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.818+0000 7f7f020d3740 1 -- 172.21.15.152:0/1096762211 shutdown_connections 2024-09-11T22:56:24.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.818+0000 7f7f020d3740 1 -- 172.21.15.152:0/1096762211 wait complete. 2024-09-11T22:56:24.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.818+0000 7f7f020d3740 1 -- start start 2024-09-11T22:56:24.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7f020d3740 1 Processor -- start 2024-09-11T22:56:24.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7f020d3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d120fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7f020d3740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d11e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7f020d3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d11e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7f020d3740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56479d124250 con 0x56479d116dc0 2024-09-11T22:56:24.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7f020d3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56479d037050 con 0x56479d1164a0 2024-09-11T22:56:24.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7f020d3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56479d1243d0 con 0x56479d121680 2024-09-11T22:56:24.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7f020d3740 10 client.0 _ll_register_callbacks cb 0x56479d077160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:24.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7efd295640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d11e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7efd295640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d11e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38890/0 (socket says 172.21.15.152:38890) 2024-09-11T22:56:24.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7efd295640 1 -- 172.21.15.152:0/1175051628 learned_addr learned my addr 172.21.15.152:0/1175051628 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:24.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7efe297640 1 --2- 172.21.15.152:0/1175051628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d11e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7efda96640 1 --2- 172.21.15.152:0/1175051628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d120fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7efd295640 1 -- 172.21.15.152:0/1175051628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 msgr2=0x56479d11e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:24.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7efd295640 1 --2- 172.21.15.152:0/1175051628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d11e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7efd295640 1 -- 172.21.15.152:0/1175051628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 msgr2=0x56479d120fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:24.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7efd295640 1 --2- 172.21.15.152:0/1175051628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d120fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7efd295640 1 -- 172.21.15.152:0/1175051628 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56479cded3c0 con 0x56479d116dc0 2024-09-11T22:56:24.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.819+0000 7f7efe297640 1 --2- 172.21.15.152:0/1175051628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d11e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:56:24.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.820+0000 7f7efd295640 1 --2- 172.21.15.152:0/1175051628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d11e250 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8010810 tx=0x7f7ee8010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e3c94057d53cd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:24.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.820+0000 7f7ee17fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:24.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.820+0000 7f7ee17fa640 1 -- 172.21.15.152:0/1175051628 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee800cd20 con 0x56479d116dc0 2024-09-11T22:56:24.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.820+0000 7f7ee17fa640 1 -- 172.21.15.152:0/1175051628 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7ee8011440 con 0x56479d116dc0 2024-09-11T22:56:24.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.820+0000 7f7f020d3740 1 -- 172.21.15.152:0/1175051628 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ef409c240 con 0x56479d116dc0 2024-09-11T22:56:24.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.820+0000 7f7ee17fa640 1 -- 172.21.15.152:0/1175051628 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ee8020020 con 0x56479d116dc0 2024-09-11T22:56:24.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.820+0000 7f7f020d3740 10 client.16554 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:24.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.820+0000 7f7f020d3740 1 -- 172.21.15.152:0/1175051628 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56479d12fc30 con 0x56479d116dc0 2024-09-11T22:56:24.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.820+0000 7f7ee17fa640 1 -- 172.21.15.152:0/1175051628 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ef409c240 con 0x56479d116dc0 2024-09-11T22:56:24.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.820+0000 7f7ee17fa640 1 -- 172.21.15.152:0/1175051628 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7ee8025690 con 0x56479d116dc0 2024-09-11T22:56:24.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.820+0000 7f7f020d3740 10 client.16554 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:24.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.820+0000 7f7f020d3740 10 client.16554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:56:24.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.820+0000 7f7f020d3740 20 client.16554 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:24.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.820+0000 7f7f020d3740 1 --2- 172.21.15.152:0/1175051628 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56479d1310e0 0x56479d133520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.820+0000 7f7f020d3740 4 client.16554 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:56:24.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.820+0000 7f7f020d3740 1 -- 172.21.15.152:0/1175051628 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56479d089850 con 0x56479d1310e0 2024-09-11T22:56:24.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.821+0000 7f7efda96640 1 --2- 172.21.15.152:0/1175051628 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56479d1310e0 0x56479d133520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.821+0000 7f7efda96640 1 --2- 172.21.15.152:0/1175051628 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56479d1310e0 0x56479d133520 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0f383d9c2df6db7 server_cookie=6ae5a477698c66f0 in_seq=0 out_seq=0 2024-09-11T22:56:24.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.821+0000 7f7ee17fa640 10 client.16554 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:24.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.822+0000 7f7ee17fa640 1 -- 172.21.15.152:0/1175051628 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56479d089850 con 0x56479d1310e0 2024-09-11T22:56:24.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.822+0000 7f7ee17fa640 10 client.16554 handle_command_reply: tid=0 2024-09-11T22:56:24.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.822+0000 7f7f020d3740 1 client.16554 shutdown 2024-09-11T22:56:24.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.822+0000 7f7f020d3740 1 -- 172.21.15.152:0/1175051628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 msgr2=0x56479d11e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:24.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.822+0000 7f7f020d3740 1 --2- 172.21.15.152:0/1175051628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d11e250 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8010810 tx=0x7f7ee8010cd0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.822+0000 7f7f020d3740 1 -- 172.21.15.152:0/1175051628 shutdown_connections 2024-09-11T22:56:24.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.822+0000 7f7f020d3740 1 --2- 172.21.15.152:0/1175051628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d11e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.822+0000 7f7f020d3740 1 --2- 172.21.15.152:0/1175051628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d11e250 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.822+0000 7f7f020d3740 1 --2- 172.21.15.152:0/1175051628 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56479d1310e0 0x56479d133520 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.822+0000 7f7f020d3740 1 --2- 172.21.15.152:0/1175051628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d120fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.822+0000 7f7ee17fa640 0 client.16554 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:24.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.822+0000 7f7f020d3740 1 -- 172.21.15.152:0/1175051628 >> 172.21.15.152:0/1175051628 conn(0x56479d112a30 msgr2=0x56479d118760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:24.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.822+0000 7f7f020d3740 1 -- 172.21.15.152:0/1175051628 shutdown_connections 2024-09-11T22:56:24.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.822+0000 7f7f020d3740 1 -- 172.21.15.152:0/1175051628 wait complete. 2024-09-11T22:56:24.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.822+0000 7f7f020d3740 20 client.16554 trim_cache size 0 max 16384 2024-09-11T22:56:24.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.839+0000 7f7f020d3740 1 Processor -- start 2024-09-11T22:56:24.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.839+0000 7f7f020d3740 1 -- start start 2024-09-11T22:56:24.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.839+0000 7f7f020d3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d12d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.839+0000 7f7f020d3740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d138df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.839+0000 7f7f020d3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.839+0000 7f7f020d3740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81ad6b0 con 0x56479d116dc0 2024-09-11T22:56:24.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.839+0000 7f7f020d3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81af1f0 con 0x56479d1164a0 2024-09-11T22:56:24.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.839+0000 7f7f020d3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81add80 con 0x56479d121680 2024-09-11T22:56:24.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.839+0000 7f7efd295640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d138df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.839+0000 7f7efd295640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d138df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38896/0 (socket says 172.21.15.152:38896) 2024-09-11T22:56:24.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.839+0000 7f7efd295640 1 -- 172.21.15.152:0/3562846706 learned_addr learned my addr 172.21.15.152:0/3562846706 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:24.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.840+0000 7f7efe297640 1 --2- 172.21.15.152:0/3562846706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.840+0000 7f7efda96640 1 --2- 172.21.15.152:0/3562846706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d12d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.840+0000 7f7efd295640 1 -- 172.21.15.152:0/3562846706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 msgr2=0x56479d139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:24.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.840+0000 7f7efd295640 1 --2- 172.21.15.152:0/3562846706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.840+0000 7f7efd295640 1 -- 172.21.15.152:0/3562846706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 msgr2=0x56479d12d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:24.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.840+0000 7f7efd295640 1 --2- 172.21.15.152:0/3562846706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d12d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.840+0000 7f7efd295640 1 -- 172.21.15.152:0/3562846706 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56479d1136c0 con 0x56479d116dc0 2024-09-11T22:56:24.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.840+0000 7f7efe297640 1 --2- 172.21.15.152:0/3562846706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.840+0000 7f7efd295640 1 --2- 172.21.15.152:0/3562846706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d138df0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8025ab0 tx=0x7f7ee80103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a11d23747bfbef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.840+0000 7f7ee37fe640 1 -- 172.21.15.152:0/3562846706 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee802b070 con 0x56479d116dc0 2024-09-11T22:56:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.840+0000 7f7ee37fe640 1 -- 172.21.15.152:0/3562846706 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ee8015070 con 0x56479d116dc0 2024-09-11T22:56:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.840+0000 7f7ee37fe640 1 -- 172.21.15.152:0/3562846706 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee80035f0 con 0x56479d116dc0 2024-09-11T22:56:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.841+0000 7f7f020d3740 1 -- 172.21.15.152:0/3562846706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 msgr2=0x56479d138df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.841+0000 7f7f020d3740 1 --2- 172.21.15.152:0/3562846706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d138df0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8025ab0 tx=0x7f7ee80103d0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.841+0000 7f7f020d3740 1 -- 172.21.15.152:0/3562846706 shutdown_connections 2024-09-11T22:56:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.841+0000 7f7f020d3740 1 --2- 172.21.15.152:0/3562846706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.841+0000 7f7f020d3740 1 --2- 172.21.15.152:0/3562846706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d138df0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.841+0000 7f7f020d3740 1 --2- 172.21.15.152:0/3562846706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d12d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.841+0000 7f7f020d3740 1 -- 172.21.15.152:0/3562846706 >> 172.21.15.152:0/3562846706 conn(0x56479d112a30 msgr2=0x56479d11d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.841+0000 7f7f020d3740 1 -- 172.21.15.152:0/3562846706 shutdown_connections 2024-09-11T22:56:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.841+0000 7f7f020d3740 1 -- 172.21.15.152:0/3562846706 wait complete. 2024-09-11T22:56:24.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.841+0000 7f7f020d3740 1 -- start start 2024-09-11T22:56:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.841+0000 7f7f020d3740 1 Processor -- start 2024-09-11T22:56:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7f020d3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d129c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7f020d3740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d126f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7f020d3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d1274d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7f020d3740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81aeb20 con 0x56479d116dc0 2024-09-11T22:56:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7f020d3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81ae450 con 0x56479d1164a0 2024-09-11T22:56:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7f020d3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81ab670 con 0x56479d121680 2024-09-11T22:56:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7f020d3740 10 client.0 _ll_register_callbacks cb 0x7f7ee800e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7efd295640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d126f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7efd295640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d126f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38906/0 (socket says 172.21.15.152:38906) 2024-09-11T22:56:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7efd295640 1 -- 172.21.15.152:0/2044032611 learned_addr learned my addr 172.21.15.152:0/2044032611 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7efda96640 1 --2- 172.21.15.152:0/2044032611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d129c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7efe297640 1 --2- 172.21.15.152:0/2044032611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d1274d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7efd295640 1 -- 172.21.15.152:0/2044032611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 msgr2=0x56479d1274d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7efd295640 1 --2- 172.21.15.152:0/2044032611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d1274d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7efd295640 1 -- 172.21.15.152:0/2044032611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 msgr2=0x56479d129c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7efd295640 1 --2- 172.21.15.152:0/2044032611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d129c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7efd295640 1 -- 172.21.15.152:0/2044032611 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56479d127bc0 con 0x56479d116dc0 2024-09-11T22:56:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7efda96640 1 --2- 172.21.15.152:0/2044032611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d129c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:56:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7efd295640 1 --2- 172.21.15.152:0/2044032611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d126f90 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8025bd0 tx=0x7f7ee8010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697643e600efc55 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7ee17fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7ee17fa640 1 -- 172.21.15.152:0/2044032611 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee802b040 con 0x56479d116dc0 2024-09-11T22:56:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7ee17fa640 1 -- 172.21.15.152:0/2044032611 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7ee8025430 con 0x56479d116dc0 2024-09-11T22:56:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.842+0000 7f7f020d3740 1 -- 172.21.15.152:0/2044032611 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ee8015070 con 0x56479d116dc0 2024-09-11T22:56:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.843+0000 7f7ee17fa640 1 -- 172.21.15.152:0/2044032611 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ee8041050 con 0x56479d116dc0 2024-09-11T22:56:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.843+0000 7f7f020d3740 10 client.16560 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:24.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.843+0000 7f7f020d3740 1 -- 172.21.15.152:0/2044032611 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56479d19bc10 con 0x56479d116dc0 2024-09-11T22:56:24.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.843+0000 7f7ee17fa640 1 -- 172.21.15.152:0/2044032611 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ee8015070 con 0x56479d116dc0 2024-09-11T22:56:24.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.843+0000 7f7ee17fa640 1 -- 172.21.15.152:0/2044032611 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7ee8003c10 con 0x56479d116dc0 2024-09-11T22:56:24.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.843+0000 7f7f020d3740 10 client.16560 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:24.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.843+0000 7f7f020d3740 10 client.16560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:56:24.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.843+0000 7f7f020d3740 20 client.16560 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:24.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.843+0000 7f7f020d3740 1 --2- 172.21.15.152:0/2044032611 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56479d19c340 0x56479d19e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:24.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.843+0000 7f7f020d3740 4 client.16560 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:56:24.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.843+0000 7f7f020d3740 1 -- 172.21.15.152:0/2044032611 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56479d083d60 con 0x56479d19c340 2024-09-11T22:56:24.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.843+0000 7f7efda96640 1 --2- 172.21.15.152:0/2044032611 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56479d19c340 0x56479d19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:24.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7efda96640 1 --2- 172.21.15.152:0/2044032611 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56479d19c340 0x56479d19e760 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b0c1b2ba369982 server_cookie=4b26036eacbfdd1f in_seq=0 out_seq=0 2024-09-11T22:56:24.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7ee17fa640 10 client.16560 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:24.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7ee17fa640 1 -- 172.21.15.152:0/2044032611 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56479d083d60 con 0x56479d19c340 2024-09-11T22:56:24.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7ee17fa640 10 client.16560 handle_command_reply: tid=0 2024-09-11T22:56:24.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7f020d3740 1 client.16560 shutdown 2024-09-11T22:56:24.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7f020d3740 1 -- 172.21.15.152:0/2044032611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 msgr2=0x56479d126f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:24.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7f020d3740 1 --2- 172.21.15.152:0/2044032611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d126f90 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8025bd0 tx=0x7f7ee8010cd0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7f020d3740 1 -- 172.21.15.152:0/2044032611 shutdown_connections 2024-09-11T22:56:24.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7f020d3740 1 --2- 172.21.15.152:0/2044032611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56479d121680 0x56479d1274d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7f020d3740 1 --2- 172.21.15.152:0/2044032611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56479d116dc0 0x56479d126f90 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7f020d3740 1 --2- 172.21.15.152:0/2044032611 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56479d19c340 0x56479d19e760 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7f020d3740 1 --2- 172.21.15.152:0/2044032611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56479d1164a0 0x56479d129c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7ee17fa640 0 client.16560 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:24.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7f020d3740 1 -- 172.21.15.152:0/2044032611 >> 172.21.15.152:0/2044032611 conn(0x56479d112a30 msgr2=0x56479d11fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:24.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7f020d3740 1 -- 172.21.15.152:0/2044032611 shutdown_connections 2024-09-11T22:56:24.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7f020d3740 1 -- 172.21.15.152:0/2044032611 wait complete. 2024-09-11T22:56:24.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.844+0000 7f7f020d3740 20 client.16560 trim_cache size 0 max 16384 2024-09-11T22:56:24.860 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:56:24.860 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:56:24.860 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": { 2024-09-11T22:56:24.860 INFO:teuthology.orchestra.run.smithi152.stdout: "8fc712e6-71cc-4f11-b2aa-94fc461d32f5": { 2024-09-11T22:56:24.860 INFO:teuthology.orchestra.run.smithi152.stdout: "path": "/", 2024-09-11T22:56:24.860 INFO:teuthology.orchestra.run.smithi152.stdout: "tag": "8fc712e6-71cc-4f11-b2aa-94fc461d32f5", 2024-09-11T22:56:24.860 INFO:teuthology.orchestra.run.smithi152.stdout: "options": "recursive,force" 2024-09-11T22:56:24.860 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:56:24.860 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:56:24.860 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:56:24.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.845+0000 7f7effcfa640 1 -- 172.21.15.152:0/1658879443 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7ee40a2d70 msgr2=0x7f7ee40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:24.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.845+0000 7f7effcfa640 1 --2- 172.21.15.152:0/1658879443 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7ee40a2d70 0x7f7ee40a5230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009bf0 tx=0x7f7ef00095c0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.845+0000 7f7effcfa640 1 -- 172.21.15.152:0/1658879443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ef8101de0 msgr2=0x7f7ef8198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:24.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.845+0000 7f7effcfa640 1 --2- 172.21.15.152:0/1658879443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ef8101de0 0x7f7ef8198280 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400c920 tx=0x7f7ef400cde0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.845+0000 7f7effcfa640 1 -- 172.21.15.152:0/1658879443 shutdown_connections 2024-09-11T22:56:24.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.845+0000 7f7effcfa640 1 --2- 172.21.15.152:0/1658879443 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7ee40a2d70 0x7f7ee40a5230 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.845+0000 7f7effcfa640 1 --2- 172.21.15.152:0/1658879443 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ef8106f10 0x7f7ef819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.845+0000 7f7effcfa640 1 --2- 172.21.15.152:0/1658879443 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ef8102790 0x7f7ef81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.845+0000 7f7effcfa640 1 --2- 172.21.15.152:0/1658879443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ef8101de0 0x7f7ef8198280 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:24.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.845+0000 7f7effcfa640 1 -- 172.21.15.152:0/1658879443 >> 172.21.15.152:0/1658879443 conn(0x7f7ef80fd8c0 msgr2=0x7f7ef80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:24.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.845+0000 7f7effcfa640 1 -- 172.21.15.152:0/1658879443 shutdown_connections 2024-09-11T22:56:24.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:24.845+0000 7f7effcfa640 1 -- 172.21.15.152:0/1658879443 wait complete. 2024-09-11T22:56:25.761 INFO:tasks.cephfs.filesystem:scrub status for tag:8fc712e6-71cc-4f11-b2aa-94fc461d32f5 - {'path': '/', 'tag': '8fc712e6-71cc-4f11-b2aa-94fc461d32f5', 'options': 'recursive,force'} 2024-09-11T22:56:26.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:26 smithi191 ceph-mon[35211]: pgmap v109: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-11T22:56:26.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:26 smithi204 ceph-mon[35454]: pgmap v109: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-11T22:56:26.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:26 smithi152 ceph-mon[31181]: pgmap v109: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-11T22:56:28.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:28 smithi191 ceph-mon[35211]: pgmap v110: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:56:28.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:28 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T22:56:28.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:28 smithi204 ceph-mon[35454]: pgmap v110: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:56:28.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:28 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T22:56:28.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:28 smithi152 ceph-mon[31181]: pgmap v110: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:56:28.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:28 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T22:56:30.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:30 smithi191 ceph-mon[35211]: pgmap v111: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 12 op/s 2024-09-11T22:56:30.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:30 smithi204 ceph-mon[35454]: pgmap v111: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 12 op/s 2024-09-11T22:56:30.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:30 smithi152 ceph-mon[31181]: pgmap v111: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 12 op/s 2024-09-11T22:56:32.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:32 smithi191 ceph-mon[35211]: pgmap v112: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-11T22:56:32.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:32 smithi204 ceph-mon[35454]: pgmap v112: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-11T22:56:32.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:32 smithi152 ceph-mon[31181]: pgmap v112: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-11T22:56:34.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:34 smithi191 ceph-mon[35211]: pgmap v113: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:56:34.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:34 smithi204 ceph-mon[35454]: pgmap v113: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:56:34.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:34 smithi152 ceph-mon[31181]: pgmap v113: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:56:36.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:36 smithi191 ceph-mon[35211]: pgmap v114: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:56:36.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:36 smithi204 ceph-mon[35454]: pgmap v114: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:56:36.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:36 smithi152 ceph-mon[31181]: pgmap v114: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:56:38.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:38 smithi191 ceph-mon[35211]: pgmap v115: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:56:38.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:38 smithi204 ceph-mon[35454]: pgmap v115: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:56:38.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:38 smithi152 ceph-mon[31181]: pgmap v115: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:56:40.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:40 smithi191 ceph-mon[35211]: pgmap v116: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:56:40.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:40 smithi204 ceph-mon[35454]: pgmap v116: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:56:40.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:40 smithi152 ceph-mon[31181]: pgmap v116: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:56:42.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:42 smithi191 ceph-mon[35211]: pgmap v117: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:56:42.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:42 smithi204 ceph-mon[35454]: pgmap v117: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:56:42.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:42 smithi152 ceph-mon[31181]: pgmap v117: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:56:44.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:44 smithi191 ceph-mon[35211]: pgmap v118: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:56:44.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:44 smithi204 ceph-mon[35454]: pgmap v118: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:56:44.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:44 smithi152 ceph-mon[31181]: pgmap v118: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:56:46.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:46 smithi191 ceph-mon[35211]: pgmap v119: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:56:46.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:46 smithi204 ceph-mon[35454]: pgmap v119: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:56:46.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:46 smithi152 ceph-mon[31181]: pgmap v119: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:56:48.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:48 smithi191 ceph-mon[35211]: pgmap v120: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:56:48.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:48 smithi204 ceph-mon[35454]: pgmap v120: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:56:48.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:48 smithi152 ceph-mon[31181]: pgmap v120: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:56:50.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:50 smithi191 ceph-mon[35211]: pgmap v121: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:56:50.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:50 smithi204 ceph-mon[35454]: pgmap v121: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:56:50.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:50 smithi152 ceph-mon[31181]: pgmap v121: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:56:52.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:52 smithi191 ceph-mon[35211]: pgmap v122: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:56:52.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:52 smithi204 ceph-mon[35454]: pgmap v122: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:56:52.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:52 smithi152 ceph-mon[31181]: pgmap v122: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:56:54.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:54 smithi191 ceph-mon[35211]: pgmap v123: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:56:54.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:54 smithi204 ceph-mon[35454]: pgmap v123: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:56:54.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:54 smithi152 ceph-mon[31181]: pgmap v123: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:56:55.763 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:56:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:56:56.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.574+0000 7f30ae913640 1 -- 172.21.15.152:0/3484770260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a8102ef0 msgr2=0x7f30a810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:56.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.574+0000 7f30ae913640 1 --2- 172.21.15.152:0/3484770260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a8102ef0 0x7f30a810da40 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f30a0009ca0 tx=0x7f30a002a950 comp rx=0 tx=0).stop 2024-09-11T22:56:56.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.574+0000 7f30ae913640 1 -- 172.21.15.152:0/3484770260 shutdown_connections 2024-09-11T22:56:56.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.574+0000 7f30ae913640 1 --2- 172.21.15.152:0/3484770260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a8102ef0 0x7f30a810da40 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.574+0000 7f30ae913640 1 --2- 172.21.15.152:0/3484770260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30a8102570 0x7f30a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.574+0000 7f30ae913640 1 --2- 172.21.15.152:0/3484770260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a8100e20 0x7f30a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.574+0000 7f30ae913640 1 -- 172.21.15.152:0/3484770260 >> 172.21.15.152:0/3484770260 conn(0x7f30a80fc7d0 msgr2=0x7f30a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:56.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.574+0000 7f30ae913640 1 -- 172.21.15.152:0/3484770260 shutdown_connections 2024-09-11T22:56:56.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.574+0000 7f30ae913640 1 -- 172.21.15.152:0/3484770260 wait complete. 2024-09-11T22:56:56.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.575+0000 7f30ae913640 1 Processor -- start 2024-09-11T22:56:56.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.575+0000 7f30ae913640 1 -- start start 2024-09-11T22:56:56.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.575+0000 7f30ae913640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30a8100e20 0x7f30a8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.575+0000 7f30ae913640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a8102570 0x7f30a8199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.575+0000 7f30ae913640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a8102ef0 0x7f30a81a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.575+0000 7f30ae913640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f30a8110fa0 con 0x7f30a8102ef0 2024-09-11T22:56:56.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.575+0000 7f30ae913640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f30a8110e20 con 0x7f30a8100e20 2024-09-11T22:56:56.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.575+0000 7f30ae913640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f30a8111120 con 0x7f30a8102570 2024-09-11T22:56:56.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.575+0000 7f30aceb0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a8102ef0 0x7f30a81a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.575+0000 7f30aceb0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a8102ef0 0x7f30a81a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48210/0 (socket says 172.21.15.152:48210) 2024-09-11T22:56:56.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.575+0000 7f30aceb0640 1 -- 172.21.15.152:0/2099262642 learned_addr learned my addr 172.21.15.152:0/2099262642 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:56.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.575+0000 7f30a7fff640 1 --2- 172.21.15.152:0/2099262642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30a8100e20 0x7f30a8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.575+0000 7f30a77fe640 1 --2- 172.21.15.152:0/2099262642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a8102570 0x7f30a8199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.576+0000 7f30aceb0640 1 -- 172.21.15.152:0/2099262642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a8102570 msgr2=0x7f30a8199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:56.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.576+0000 7f30aceb0640 1 --2- 172.21.15.152:0/2099262642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a8102570 0x7f30a8199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.576+0000 7f30aceb0640 1 -- 172.21.15.152:0/2099262642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30a8100e20 msgr2=0x7f30a8199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:56.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.576+0000 7f30aceb0640 1 --2- 172.21.15.152:0/2099262642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30a8100e20 0x7f30a8199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.576+0000 7f30aceb0640 1 -- 172.21.15.152:0/2099262642 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a81a48d0 con 0x7f30a8102ef0 2024-09-11T22:56:56.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.576+0000 7f30aceb0640 1 --2- 172.21.15.152:0/2099262642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a8102ef0 0x7f30a81a4390 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f30a002ae70 tx=0x7f30a002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:56.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.576+0000 7f30a57fa640 1 -- 172.21.15.152:0/2099262642 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a0034040 con 0x7f30a8102ef0 2024-09-11T22:56:56.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.576+0000 7f30a57fa640 1 -- 172.21.15.152:0/2099262642 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a0030920 con 0x7f30a8102ef0 2024-09-11T22:56:56.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.576+0000 7f30ae913640 1 -- 172.21.15.152:0/2099262642 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a81a4aa0 con 0x7f30a8102ef0 2024-09-11T22:56:56.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.576+0000 7f30a57fa640 1 -- 172.21.15.152:0/2099262642 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a00047f0 con 0x7f30a8102ef0 2024-09-11T22:56:56.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.576+0000 7f30ae913640 1 -- 172.21.15.152:0/2099262642 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a81a4f80 con 0x7f30a8102ef0 2024-09-11T22:56:56.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.578+0000 7f30a57fa640 1 -- 172.21.15.152:0/2099262642 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f30a0037020 con 0x7f30a8102ef0 2024-09-11T22:56:56.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.578+0000 7f30b0cec740 1 Processor -- start 2024-09-11T22:56:56.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30b0cec740 1 -- start start 2024-09-11T22:56:56.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30b0cec740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc084a0 0x5564ecc08880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30b0cec740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc08dc0 0x5564ecc13140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30a57fa640 1 --2- 172.21.15.152:0/2099262642 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f30940a2d70 0x7f30940a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30b0cec740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc13680 0x5564ecc15a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30b0cec740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5564ecb6e2e0 con 0x5564ecc13680 2024-09-11T22:56:56.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30b0cec740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5564ecb15ce0 con 0x5564ecc08dc0 2024-09-11T22:56:56.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30b0cec740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5564ecb6a2a0 con 0x5564ecc084a0 2024-09-11T22:56:56.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30a77fe640 1 --2- 172.21.15.152:0/2099262642 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f30940a2d70 0x7f30940a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30aceb0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc13680 0x5564ecc15a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30a7fff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc08dc0 0x5564ecc13140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30a77fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc084a0 0x5564ecc08880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30a57fa640 1 -- 172.21.15.152:0/2099262642 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f30a00f0120 con 0x7f30a8102ef0 2024-09-11T22:56:56.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30aceb0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc13680 0x5564ecc15a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48220/0 (socket says 172.21.15.152:48220) 2024-09-11T22:56:56.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30aceb0640 1 -- 172.21.15.152:0/2191399976 learned_addr learned my addr 172.21.15.152:0/2191399976 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:56.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30a7fff640 1 -- 172.21.15.152:0/2191399976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc084a0 msgr2=0x5564ecc08880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:56.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30a7fff640 1 --2- 172.21.15.152:0/2191399976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc084a0 0x5564ecc08880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30a7fff640 1 -- 172.21.15.152:0/2191399976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc13680 msgr2=0x5564ecc15a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:56.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30a7fff640 1 --2- 172.21.15.152:0/2191399976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc13680 0x5564ecc15a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30a7fff640 1 -- 172.21.15.152:0/2191399976 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ecafa440 con 0x5564ecc08dc0 2024-09-11T22:56:56.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.579+0000 7f30aceb0640 1 --2- 172.21.15.152:0/2191399976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc13680 0x5564ecc15a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:56:56.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.580+0000 7f30a77fe640 1 --2- 172.21.15.152:0/2099262642 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f30940a2d70 0x7f30940a5230 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3090006e10 tx=0x7f3090009940 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:56.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.580+0000 7f30a7fff640 1 --2- 172.21.15.152:0/2191399976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc08dc0 0x5564ecc13140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f309800bbf0 tx=0x7f30980107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbffc33178d432f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:56.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.580+0000 7f309e7fc640 1 -- 172.21.15.152:0/2191399976 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098004030 con 0x5564ecc08dc0 2024-09-11T22:56:56.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.580+0000 7f309e7fc640 1 -- 172.21.15.152:0/2191399976 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30980041d0 con 0x5564ecc08dc0 2024-09-11T22:56:56.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.580+0000 7f309e7fc640 1 -- 172.21.15.152:0/2191399976 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30980116a0 con 0x5564ecc08dc0 2024-09-11T22:56:56.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.580+0000 7f30b0cec740 1 -- 172.21.15.152:0/2191399976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc08dc0 msgr2=0x5564ecc13140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:56.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.580+0000 7f30b0cec740 1 --2- 172.21.15.152:0/2191399976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc08dc0 0x5564ecc13140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f309800bbf0 tx=0x7f30980107a0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.580+0000 7f30b0cec740 1 -- 172.21.15.152:0/2191399976 shutdown_connections 2024-09-11T22:56:56.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.580+0000 7f30b0cec740 1 --2- 172.21.15.152:0/2191399976 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc13680 0x5564ecc15a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.580+0000 7f30b0cec740 1 --2- 172.21.15.152:0/2191399976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc08dc0 0x5564ecc13140 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.580+0000 7f30b0cec740 1 --2- 172.21.15.152:0/2191399976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc084a0 0x5564ecc08880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.581+0000 7f30b0cec740 1 -- 172.21.15.152:0/2191399976 >> 172.21.15.152:0/2191399976 conn(0x5564ecc04a30 msgr2=0x5564ecc04e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:56.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.581+0000 7f30b0cec740 1 -- 172.21.15.152:0/2191399976 shutdown_connections 2024-09-11T22:56:56.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.581+0000 7f30b0cec740 1 -- 172.21.15.152:0/2191399976 wait complete. 2024-09-11T22:56:56.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.581+0000 7f30b0cec740 1 -- start start 2024-09-11T22:56:56.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.581+0000 7f30b0cec740 1 Processor -- start 2024-09-11T22:56:56.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.581+0000 7f30b0cec740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc084a0 0x5564ecc12f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.581+0000 7f30b0cec740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc10200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.581+0000 7f30b0cec740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc13680 0x5564ecc10740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.581+0000 7f30b0cec740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5564ecc16250 con 0x5564ecc08dc0 2024-09-11T22:56:56.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.581+0000 7f30b0cec740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5564ecb29050 con 0x5564ecc13680 2024-09-11T22:56:56.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.581+0000 7f30b0cec740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5564ecc163d0 con 0x5564ecc084a0 2024-09-11T22:56:56.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.581+0000 7f30b0cec740 10 client.0 _ll_register_callbacks cb 0x5564ecb69160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:56.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.581+0000 7f30a77fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc10200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30a77fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc10200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48230/0 (socket says 172.21.15.152:48230) 2024-09-11T22:56:56.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30a77fe640 1 -- 172.21.15.152:0/3509913473 learned_addr learned my addr 172.21.15.152:0/3509913473 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:56.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30aceb0640 1 --2- 172.21.15.152:0/3509913473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc13680 0x5564ecc10740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30a7fff640 1 --2- 172.21.15.152:0/3509913473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc084a0 0x5564ecc12f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30a77fe640 1 -- 172.21.15.152:0/3509913473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc084a0 msgr2=0x5564ecc12f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:56.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30a77fe640 1 --2- 172.21.15.152:0/3509913473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc084a0 0x5564ecc12f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30a77fe640 1 -- 172.21.15.152:0/3509913473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc13680 msgr2=0x5564ecc10740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:56.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30a77fe640 1 --2- 172.21.15.152:0/3509913473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc13680 0x5564ecc10740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30a77fe640 1 -- 172.21.15.152:0/3509913473 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ec8df3c0 con 0x5564ecc08dc0 2024-09-11T22:56:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30aceb0640 1 --2- 172.21.15.152:0/3509913473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc13680 0x5564ecc10740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:56:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30a77fe640 1 --2- 172.21.15.152:0/3509913473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc10200 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3090017b90 tx=0x7f3090009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6bac13e02e1f242 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30877fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30877fe640 1 -- 172.21.15.152:0/3509913473 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3090018be0 con 0x5564ecc08dc0 2024-09-11T22:56:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30877fe640 1 -- 172.21.15.152:0/3509913473 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f3090018d80 con 0x5564ecc08dc0 2024-09-11T22:56:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30b0cec740 1 -- 172.21.15.152:0/3509913473 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30980041d0 con 0x5564ecc08dc0 2024-09-11T22:56:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30877fe640 1 -- 172.21.15.152:0/3509913473 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3090032050 con 0x5564ecc08dc0 2024-09-11T22:56:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30b0cec740 10 client.16572 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.582+0000 7f30b0cec740 1 -- 172.21.15.152:0/3509913473 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5564ecc21bd0 con 0x5564ecc08dc0 2024-09-11T22:56:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.583+0000 7f30877fe640 1 -- 172.21.15.152:0/3509913473 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30980041d0 con 0x5564ecc08dc0 2024-09-11T22:56:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.583+0000 7f30877fe640 1 -- 172.21.15.152:0/3509913473 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3090022530 con 0x5564ecc08dc0 2024-09-11T22:56:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.583+0000 7f30b0cec740 10 client.16572 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:56.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.583+0000 7f30b0cec740 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:56:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.583+0000 7f30b0cec740 20 client.16572 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.583+0000 7f30b0cec740 1 --2- 172.21.15.152:0/3509913473 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5564ecc230e0 0x5564ecc25540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.583+0000 7f30b0cec740 4 client.16572 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:56:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.583+0000 7f30b0cec740 1 -- 172.21.15.152:0/3509913473 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564ecb7b850 con 0x5564ecc230e0 2024-09-11T22:56:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.583+0000 7f30a7fff640 1 --2- 172.21.15.152:0/3509913473 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5564ecc230e0 0x5564ecc25540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.583+0000 7f30a7fff640 1 --2- 172.21.15.152:0/3509913473 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5564ecc230e0 0x5564ecc25540 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=657fa649a8ff5e1a server_cookie=5bbc58c1e5de52c6 in_seq=0 out_seq=0 2024-09-11T22:56:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.583+0000 7f30877fe640 10 client.16572 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.584+0000 7f30877fe640 1 -- 172.21.15.152:0/3509913473 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564ecb7b850 con 0x5564ecc230e0 2024-09-11T22:56:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.584+0000 7f30877fe640 10 client.16572 handle_command_reply: tid=0 2024-09-11T22:56:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.584+0000 7f30b0cec740 1 client.16572 shutdown 2024-09-11T22:56:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.584+0000 7f30b0cec740 1 -- 172.21.15.152:0/3509913473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 msgr2=0x5564ecc10200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.584+0000 7f30b0cec740 1 --2- 172.21.15.152:0/3509913473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc10200 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3090017b90 tx=0x7f3090009110 comp rx=0 tx=0).stop 2024-09-11T22:56:56.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.584+0000 7f30b0cec740 1 -- 172.21.15.152:0/3509913473 shutdown_connections 2024-09-11T22:56:56.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.584+0000 7f30b0cec740 1 --2- 172.21.15.152:0/3509913473 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5564ecc230e0 0x5564ecc25540 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.584+0000 7f30b0cec740 1 --2- 172.21.15.152:0/3509913473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc13680 0x5564ecc10740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.584+0000 7f30877fe640 0 client.16572 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:56.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.584+0000 7f30b0cec740 1 --2- 172.21.15.152:0/3509913473 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc10200 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.584+0000 7f30b0cec740 1 --2- 172.21.15.152:0/3509913473 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc084a0 0x5564ecc12f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.584+0000 7f30b0cec740 1 -- 172.21.15.152:0/3509913473 >> 172.21.15.152:0/3509913473 conn(0x5564ecc04a30 msgr2=0x5564ecc0a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:56.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.585+0000 7f30b0cec740 1 -- 172.21.15.152:0/3509913473 shutdown_connections 2024-09-11T22:56:56.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.585+0000 7f30b0cec740 1 -- 172.21.15.152:0/3509913473 wait complete. 2024-09-11T22:56:56.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.585+0000 7f30b0cec740 20 client.16572 trim_cache size 0 max 16384 2024-09-11T22:56:56.610 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.601+0000 7f30b0cec740 1 Processor -- start 2024-09-11T22:56:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.601+0000 7f30b0cec740 1 -- start start 2024-09-11T22:56:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30b0cec740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc04a30 0x5564ecc101c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30b0cec740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc084a0 0x5564ecc03a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30b0cec740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc2adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30b0cec740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f30a81ab550 con 0x5564ecc08dc0 2024-09-11T22:56:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30b0cec740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f30a81ad090 con 0x5564ecc084a0 2024-09-11T22:56:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30b0cec740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f30a81abc20 con 0x5564ecc04a30 2024-09-11T22:56:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30aceb0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc2adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30aceb0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc2adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48236/0 (socket says 172.21.15.152:48236) 2024-09-11T22:56:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30aceb0640 1 -- 172.21.15.152:0/3222144447 learned_addr learned my addr 172.21.15.152:0/3222144447 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30a7fff640 1 --2- 172.21.15.152:0/3222144447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc04a30 0x5564ecc101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30a77fe640 1 --2- 172.21.15.152:0/3222144447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc084a0 0x5564ecc03a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30aceb0640 1 -- 172.21.15.152:0/3222144447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc04a30 msgr2=0x5564ecc101c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:56.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30aceb0640 1 --2- 172.21.15.152:0/3222144447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc04a30 0x5564ecc101c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30aceb0640 1 -- 172.21.15.152:0/3222144447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc084a0 msgr2=0x5564ecc03a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:56.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30aceb0640 1 --2- 172.21.15.152:0/3222144447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc084a0 0x5564ecc03a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30aceb0640 1 -- 172.21.15.152:0/3222144447 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564ecc04860 con 0x5564ecc08dc0 2024-09-11T22:56:56.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30a77fe640 1 --2- 172.21.15.152:0/3222144447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc084a0 0x5564ecc03a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:56.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f30aceb0640 1 --2- 172.21.15.152:0/3222144447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc2adc0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f30a00b9130 tx=0x7f30a0004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333296cb2fca2f7e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:56.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f309dffb640 1 -- 172.21.15.152:0/3222144447 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a00c22a0 con 0x5564ecc08dc0 2024-09-11T22:56:56.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.602+0000 7f309dffb640 1 -- 172.21.15.152:0/3222144447 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a00098d0 con 0x5564ecc08dc0 2024-09-11T22:56:56.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.603+0000 7f309dffb640 1 -- 172.21.15.152:0/3222144447 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a00c53d0 con 0x5564ecc08dc0 2024-09-11T22:56:56.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.603+0000 7f30b0cec740 1 -- 172.21.15.152:0/3222144447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 msgr2=0x5564ecc2adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:56.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.603+0000 7f30b0cec740 1 --2- 172.21.15.152:0/3222144447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc2adc0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f30a00b9130 tx=0x7f30a0004290 comp rx=0 tx=0).stop 2024-09-11T22:56:56.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.603+0000 7f30b0cec740 1 -- 172.21.15.152:0/3222144447 shutdown_connections 2024-09-11T22:56:56.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.603+0000 7f30b0cec740 1 --2- 172.21.15.152:0/3222144447 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc2adc0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.603+0000 7f30b0cec740 1 --2- 172.21.15.152:0/3222144447 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc084a0 0x5564ecc03a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.603+0000 7f30b0cec740 1 --2- 172.21.15.152:0/3222144447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc04a30 0x5564ecc101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.603+0000 7f30b0cec740 1 -- 172.21.15.152:0/3222144447 >> 172.21.15.152:0/3222144447 conn(0x5564ecc108a0 msgr2=0x5564ecc03350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:56.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.603+0000 7f30b0cec740 1 -- 172.21.15.152:0/3222144447 shutdown_connections 2024-09-11T22:56:56.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.603+0000 7f30b0cec740 1 -- 172.21.15.152:0/3222144447 wait complete. 2024-09-11T22:56:56.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.603+0000 7f30b0cec740 1 -- start start 2024-09-11T22:56:56.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30b0cec740 1 Processor -- start 2024-09-11T22:56:56.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30b0cec740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc04a30 0x5564ecc1bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30b0cec740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc084a0 0x5564ecc19290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30b0cec740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc19a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30b0cec740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f30a81ac9c0 con 0x5564ecc08dc0 2024-09-11T22:56:56.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30b0cec740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f30a81ac2f0 con 0x5564ecc084a0 2024-09-11T22:56:56.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30b0cec740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f30a81a9550 con 0x5564ecc04a30 2024-09-11T22:56:56.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30b0cec740 10 client.0 _ll_register_callbacks cb 0x7f3090009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:56:56.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30aceb0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc19a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30aceb0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc19a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48244/0 (socket says 172.21.15.152:48244) 2024-09-11T22:56:56.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30aceb0640 1 -- 172.21.15.152:0/2961942287 learned_addr learned my addr 172.21.15.152:0/2961942287 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:56.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30a77fe640 1 --2- 172.21.15.152:0/2961942287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc084a0 0x5564ecc19290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30a7fff640 1 --2- 172.21.15.152:0/2961942287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc04a30 0x5564ecc1bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30aceb0640 1 -- 172.21.15.152:0/2961942287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc04a30 msgr2=0x5564ecc1bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:56.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30aceb0640 1 --2- 172.21.15.152:0/2961942287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc04a30 0x5564ecc1bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30aceb0640 1 -- 172.21.15.152:0/2961942287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc084a0 msgr2=0x5564ecc19290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:56:56.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30aceb0640 1 --2- 172.21.15.152:0/2961942287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc084a0 0x5564ecc19290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.604+0000 7f30aceb0640 1 -- 172.21.15.152:0/2961942287 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564ecc8cda0 con 0x5564ecc08dc0 2024-09-11T22:56:56.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.605+0000 7f30aceb0640 1 --2- 172.21.15.152:0/2961942287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc19a20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f30a002fdd0 tx=0x7f30a0004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188ba8a08fa93ab9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:56.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.605+0000 7f30877fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:56:56.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.605+0000 7f30877fe640 1 -- 172.21.15.152:0/2961942287 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a00bc760 con 0x5564ecc08dc0 2024-09-11T22:56:56.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.605+0000 7f30877fe640 1 -- 172.21.15.152:0/2961942287 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f30a00bc9c0 con 0x5564ecc08dc0 2024-09-11T22:56:56.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.605+0000 7f30b0cec740 1 -- 172.21.15.152:0/2961942287 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5564ecb75d60 con 0x5564ecc08dc0 2024-09-11T22:56:56.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.605+0000 7f30877fe640 1 -- 172.21.15.152:0/2961942287 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f30a00c0b10 con 0x5564ecc08dc0 2024-09-11T22:56:56.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.605+0000 7f30b0cec740 10 client.16578 fetch_fsmap learned FSMap version 28 2024-09-11T22:56:56.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.605+0000 7f30b0cec740 1 -- 172.21.15.152:0/2961942287 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5564ecc8de70 con 0x5564ecc08dc0 2024-09-11T22:56:56.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.605+0000 7f30877fe640 1 -- 172.21.15.152:0/2961942287 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5564ecb75d60 con 0x5564ecc08dc0 2024-09-11T22:56:56.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.605+0000 7f30877fe640 1 -- 172.21.15.152:0/2961942287 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f30a004d030 con 0x5564ecc08dc0 2024-09-11T22:56:56.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.605+0000 7f30b0cec740 10 client.16578 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:56:56.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.605+0000 7f30b0cec740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:56:56.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.605+0000 7f30b0cec740 20 client.16578 populate_metadata read hostname 'smithi152' 2024-09-11T22:56:56.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.605+0000 7f30b0cec740 1 --2- 172.21.15.152:0/2961942287 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5564ecc13680 0x5564ecc904f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:56.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.605+0000 7f30b0cec740 4 client.16578 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:56:56.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.605+0000 7f30b0cec740 1 -- 172.21.15.152:0/2961942287 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5564ecaf2210 con 0x5564ecc13680 2024-09-11T22:56:56.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.606+0000 7f30a7fff640 1 --2- 172.21.15.152:0/2961942287 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5564ecc13680 0x5564ecc904f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:56.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.606+0000 7f30a7fff640 1 --2- 172.21.15.152:0/2961942287 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5564ecc13680 0x5564ecc904f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39f468b3523add22 server_cookie=7b7d3decc8e6362 in_seq=0 out_seq=0 2024-09-11T22:56:56.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.606+0000 7f30877fe640 10 client.16578 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:56.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.606+0000 7f30877fe640 1 -- 172.21.15.152:0/2961942287 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5564ecaf2210 con 0x5564ecc13680 2024-09-11T22:56:56.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.606+0000 7f30877fe640 10 client.16578 handle_command_reply: tid=0 2024-09-11T22:56:56.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.606+0000 7f30b0cec740 1 client.16578 shutdown 2024-09-11T22:56:56.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.606+0000 7f30b0cec740 1 -- 172.21.15.152:0/2961942287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 msgr2=0x5564ecc19a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:56.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.606+0000 7f30b0cec740 1 --2- 172.21.15.152:0/2961942287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc19a20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f30a002fdd0 tx=0x7f30a0004990 comp rx=0 tx=0).stop 2024-09-11T22:56:56.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30b0cec740 1 -- 172.21.15.152:0/2961942287 shutdown_connections 2024-09-11T22:56:56.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30b0cec740 1 --2- 172.21.15.152:0/2961942287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564ecc08dc0 0x5564ecc19a20 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30b0cec740 1 --2- 172.21.15.152:0/2961942287 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5564ecc13680 0x5564ecc904f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30b0cec740 1 --2- 172.21.15.152:0/2961942287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564ecc084a0 0x5564ecc19290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30b0cec740 1 --2- 172.21.15.152:0/2961942287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564ecc04a30 0x5564ecc1bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30b0cec740 1 -- 172.21.15.152:0/2961942287 >> 172.21.15.152:0/2961942287 conn(0x5564ecc108a0 msgr2=0x5564ecc12280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:56.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30877fe640 0 client.16578 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:56:56.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30b0cec740 1 -- 172.21.15.152:0/2961942287 shutdown_connections 2024-09-11T22:56:56.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30b0cec740 1 -- 172.21.15.152:0/2961942287 wait complete. 2024-09-11T22:56:56.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30b0cec740 20 client.16578 trim_cache size 0 max 16384 2024-09-11T22:56:56.618 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:56:56.618 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T22:56:56.618 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T22:56:56.618 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:56:56.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30ae913640 1 -- 172.21.15.152:0/2099262642 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f30940a2d70 msgr2=0x7f30940a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30ae913640 1 --2- 172.21.15.152:0/2099262642 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f30940a2d70 0x7f30940a5230 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3090006e10 tx=0x7f3090009940 comp rx=0 tx=0).stop 2024-09-11T22:56:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30ae913640 1 -- 172.21.15.152:0/2099262642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a8102ef0 msgr2=0x7f30a81a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30ae913640 1 --2- 172.21.15.152:0/2099262642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a8102ef0 0x7f30a81a4390 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f30a002ae70 tx=0x7f30a002f5d0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30ae913640 1 -- 172.21.15.152:0/2099262642 shutdown_connections 2024-09-11T22:56:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30ae913640 1 --2- 172.21.15.152:0/2099262642 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f30940a2d70 0x7f30940a5230 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30ae913640 1 --2- 172.21.15.152:0/2099262642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30a8102ef0 0x7f30a81a4390 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30ae913640 1 --2- 172.21.15.152:0/2099262642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30a8102570 0x7f30a8199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30ae913640 1 --2- 172.21.15.152:0/2099262642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30a8100e20 0x7f30a8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.607+0000 7f30ae913640 1 -- 172.21.15.152:0/2099262642 >> 172.21.15.152:0/2099262642 conn(0x7f30a80fc7d0 msgr2=0x7f30a8106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.608+0000 7f30ae913640 1 -- 172.21.15.152:0/2099262642 shutdown_connections 2024-09-11T22:56:56.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:56.608+0000 7f30ae913640 1 -- 172.21.15.152:0/2099262642 wait complete. 2024-09-11T22:56:56.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:56 smithi191 ceph-mon[35211]: pgmap v124: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:56:56.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:56 smithi204 ceph-mon[35454]: pgmap v124: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:56:56.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:56 smithi152 ceph-mon[31181]: pgmap v124: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:56:57.535 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:56:57.535 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:56:57.791 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:56:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.309+0000 7f0554618640 1 -- 172.21.15.152:0/3246589877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f054c074510 msgr2=0x7f054c0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.309+0000 7f0554618640 1 --2- 172.21.15.152:0/3246589877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f054c074510 0x7f054c0748f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0548009900 tx=0x7f054802a480 comp rx=0 tx=0).stop 2024-09-11T22:56:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.310+0000 7f0554618640 1 -- 172.21.15.152:0/3246589877 shutdown_connections 2024-09-11T22:56:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.310+0000 7f0554618640 1 --2- 172.21.15.152:0/3246589877 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f054c10e440 0x7f054c110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.310+0000 7f0554618640 1 --2- 172.21.15.152:0/3246589877 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f054c074e30 0x7f054c10df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:58.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.310+0000 7f0554618640 1 --2- 172.21.15.152:0/3246589877 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f054c074510 0x7f054c0748f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.310+0000 7f0554618640 1 -- 172.21.15.152:0/3246589877 >> 172.21.15.152:0/3246589877 conn(0x7f054c0fd8c0 msgr2=0x7f054c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.310+0000 7f0554618640 1 -- 172.21.15.152:0/3246589877 shutdown_connections 2024-09-11T22:56:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.310+0000 7f0554618640 1 -- 172.21.15.152:0/3246589877 wait complete. 2024-09-11T22:56:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.310+0000 7f0554618640 1 Processor -- start 2024-09-11T22:56:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.310+0000 7f0554618640 1 -- start start 2024-09-11T22:56:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.310+0000 7f0554618640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f054c074510 0x7f054c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.310+0000 7f0554618640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f054c074e30 0x7f054c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.310+0000 7f0554618640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f054c10e440 0x7f054c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.310+0000 7f0554618640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f054c114170 con 0x7f054c074510 2024-09-11T22:56:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.310+0000 7f0554618640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f054c113ff0 con 0x7f054c10e440 2024-09-11T22:56:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.310+0000 7f0554618640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f054c1142f0 con 0x7f054c074e30 2024-09-11T22:56:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f05523b4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f054c074510 0x7f054c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:58.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f05523b4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f054c074510 0x7f054c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:48264/0 (socket says 172.21.15.152:48264) 2024-09-11T22:56:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f05523b4640 1 -- 172.21.15.152:0/2053065549 learned_addr learned my addr 172.21.15.152:0/2053065549 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:56:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f0551bb3640 1 --2- 172.21.15.152:0/2053065549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f054c074e30 0x7f054c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f0552bb5640 1 --2- 172.21.15.152:0/2053065549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f054c10e440 0x7f054c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f05523b4640 1 -- 172.21.15.152:0/2053065549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f054c074e30 msgr2=0x7f054c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f05523b4640 1 --2- 172.21.15.152:0/2053065549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f054c074e30 0x7f054c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f05523b4640 1 -- 172.21.15.152:0/2053065549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f054c10e440 msgr2=0x7f054c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f05523b4640 1 --2- 172.21.15.152:0/2053065549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f054c10e440 0x7f054c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f05523b4640 1 -- 172.21.15.152:0/2053065549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054c19f5f0 con 0x7f054c074510 2024-09-11T22:56:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f0552bb5640 1 --2- 172.21.15.152:0/2053065549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f054c10e440 0x7f054c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:56:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f05523b4640 1 --2- 172.21.15.152:0/2053065549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f054c074510 0x7f054c19a640 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0548006010 tx=0x7f05480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f05437fe640 1 -- 172.21.15.152:0/2053065549 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054803f030 con 0x7f054c074510 2024-09-11T22:56:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f05437fe640 1 -- 172.21.15.152:0/2053065549 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0548003ce0 con 0x7f054c074510 2024-09-11T22:56:58.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f05437fe640 1 -- 172.21.15.152:0/2053065549 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548035070 con 0x7f054c074510 2024-09-11T22:56:58.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f0554618640 1 -- 172.21.15.152:0/2053065549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054c19f880 con 0x7f054c074510 2024-09-11T22:56:58.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.311+0000 7f0554618640 1 -- 172.21.15.152:0/2053065549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f054c19fd10 con 0x7f054c074510 2024-09-11T22:56:58.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.312+0000 7f0554618640 1 -- 172.21.15.152:0/2053065549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f054c0748f0 con 0x7f054c074510 2024-09-11T22:56:58.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.313+0000 7f05437fe640 1 -- 172.21.15.152:0/2053065549 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0548042030 con 0x7f054c074510 2024-09-11T22:56:58.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.314+0000 7f05437fe640 1 --2- 172.21.15.152:0/2053065549 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f05380a2d70 0x7f05380a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:56:58.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.314+0000 7f0551bb3640 1 --2- 172.21.15.152:0/2053065549 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f05380a2d70 0x7f05380a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:56:58.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.314+0000 7f05437fe640 1 -- 172.21.15.152:0/2053065549 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f05480efc90 con 0x7f054c074510 2024-09-11T22:56:58.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.315+0000 7f0551bb3640 1 --2- 172.21.15.152:0/2053065549 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f05380a2d70 0x7f05380a5230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f053c005e10 tx=0x7f053c005da0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:56:58.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.318+0000 7f05437fe640 1 -- 172.21.15.152:0/2053065549 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f05480bc9c0 con 0x7f054c074510 2024-09-11T22:56:58.520 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:58 smithi152 ceph-mon[31181]: pgmap v125: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:56:58.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.520+0000 7f0554618640 1 -- 172.21.15.152:0/2053065549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f054c02d7c0 con 0x7f054c074510 2024-09-11T22:56:58.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.520+0000 7f05437fe640 1 -- 172.21.15.152:0/2053065549 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f05480bc2c0 con 0x7f054c074510 2024-09-11T22:56:58.522 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:56:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:56:58.522 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:56:58.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.522+0000 7f0554618640 1 -- 172.21.15.152:0/2053065549 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f05380a2d70 msgr2=0x7f05380a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:58.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.522+0000 7f0554618640 1 --2- 172.21.15.152:0/2053065549 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f05380a2d70 0x7f05380a5230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f053c005e10 tx=0x7f053c005da0 comp rx=0 tx=0).stop 2024-09-11T22:56:58.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.522+0000 7f0554618640 1 -- 172.21.15.152:0/2053065549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f054c074510 msgr2=0x7f054c19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:56:58.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.522+0000 7f0554618640 1 --2- 172.21.15.152:0/2053065549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f054c074510 0x7f054c19a640 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0548006010 tx=0x7f05480042f0 comp rx=0 tx=0).stop 2024-09-11T22:56:58.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.522+0000 7f0554618640 1 -- 172.21.15.152:0/2053065549 shutdown_connections 2024-09-11T22:56:58.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.522+0000 7f0554618640 1 --2- 172.21.15.152:0/2053065549 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f05380a2d70 0x7f05380a5230 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:58.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.522+0000 7f0554618640 1 --2- 172.21.15.152:0/2053065549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f054c10e440 0x7f054c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:58.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.522+0000 7f0554618640 1 --2- 172.21.15.152:0/2053065549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f054c074e30 0x7f054c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:58.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.522+0000 7f0554618640 1 --2- 172.21.15.152:0/2053065549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f054c074510 0x7f054c19a640 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:56:58.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.523+0000 7f0554618640 1 -- 172.21.15.152:0/2053065549 >> 172.21.15.152:0/2053065549 conn(0x7f054c0fd8c0 msgr2=0x7f054c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:56:58.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.523+0000 7f0554618640 1 -- 172.21.15.152:0/2053065549 shutdown_connections 2024-09-11T22:56:58.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:56:58.523+0000 7f0554618640 1 -- 172.21.15.152:0/2053065549 wait complete. 2024-09-11T22:56:58.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:58 smithi191 ceph-mon[35211]: pgmap v125: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:56:58.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:58 smithi204 ceph-mon[35454]: pgmap v125: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:56:59.383 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:56:59.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:56:59 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2053065549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:56:59.635 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:56:59.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:56:59 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2053065549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:56:59.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:56:59 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2053065549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:57:00.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.196+0000 7f4b160ef640 1 -- 172.21.15.152:0/2546963867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b10101de0 msgr2=0x7f4b101021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:00.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.196+0000 7f4b160ef640 1 --2- 172.21.15.152:0/2546963867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b10101de0 0x7f4b101021c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009900 tx=0x7f4b0002a410 comp rx=0 tx=0).stop 2024-09-11T22:57:00.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.196+0000 7f4b160ef640 1 -- 172.21.15.152:0/2546963867 shutdown_connections 2024-09-11T22:57:00.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.196+0000 7f4b160ef640 1 --2- 172.21.15.152:0/2546963867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b10106f10 0x7f4b1010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.216 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.196+0000 7f4b160ef640 1 --2- 172.21.15.152:0/2546963867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b10102790 0x7f4b101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.196+0000 7f4b160ef640 1 --2- 172.21.15.152:0/2546963867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b10101de0 0x7f4b101021c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.196+0000 7f4b160ef640 1 -- 172.21.15.152:0/2546963867 >> 172.21.15.152:0/2546963867 conn(0x7f4b100fd8c0 msgr2=0x7f4b100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:00.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.196+0000 7f4b160ef640 1 -- 172.21.15.152:0/2546963867 shutdown_connections 2024-09-11T22:57:00.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.196+0000 7f4b160ef640 1 -- 172.21.15.152:0/2546963867 wait complete. 2024-09-11T22:57:00.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.197+0000 7f4b160ef640 1 Processor -- start 2024-09-11T22:57:00.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.197+0000 7f4b160ef640 1 -- start start 2024-09-11T22:57:00.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.197+0000 7f4b160ef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b10101de0 0x7f4b101983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.197+0000 7f4b160ef640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b10102790 0x7f4b10198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.197+0000 7f4b160ef640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b10106f10 0x7f4b1019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.197+0000 7f4b160ef640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4b10111f70 con 0x7f4b10106f10 2024-09-11T22:57:00.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.197+0000 7f4b160ef640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4b10111df0 con 0x7f4b10102790 2024-09-11T22:57:00.217 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.197+0000 7f4b160ef640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101120f0 con 0x7f4b10101de0 2024-09-11T22:57:00.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.197+0000 7f4b0ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b10106f10 0x7f4b1019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.197+0000 7f4b0ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b10106f10 0x7f4b1019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39742/0 (socket says 172.21.15.152:39742) 2024-09-11T22:57:00.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.197+0000 7f4b0ffff640 1 -- 172.21.15.152:0/3838819296 learned_addr learned my addr 172.21.15.152:0/3838819296 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:00.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.197+0000 7f4b0effd640 1 --2- 172.21.15.152:0/3838819296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b10102790 0x7f4b10198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.197+0000 7f4b0f7fe640 1 --2- 172.21.15.152:0/3838819296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b10101de0 0x7f4b101983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.198+0000 7f4b0ffff640 1 -- 172.21.15.152:0/3838819296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b10101de0 msgr2=0x7f4b101983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:00.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.198+0000 7f4b0ffff640 1 --2- 172.21.15.152:0/3838819296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b10101de0 0x7f4b101983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.198+0000 7f4b0ffff640 1 -- 172.21.15.152:0/3838819296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b10102790 msgr2=0x7f4b10198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:00.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.198+0000 7f4b0ffff640 1 --2- 172.21.15.152:0/3838819296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b10102790 0x7f4b10198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.198+0000 7f4b0ffff640 1 -- 172.21.15.152:0/3838819296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1019d380 con 0x7f4b10106f10 2024-09-11T22:57:00.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.198+0000 7f4b0ffff640 1 --2- 172.21.15.152:0/3838819296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b10106f10 0x7f4b1019cca0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800cce0 tx=0x7f4b0800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:00.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.198+0000 7f4b0cff9640 1 -- 172.21.15.152:0/3838819296 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0800f450 con 0x7f4b10106f10 2024-09-11T22:57:00.218 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.198+0000 7f4b0cff9640 1 -- 172.21.15.152:0/3838819296 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b0800fad0 con 0x7f4b10106f10 2024-09-11T22:57:00.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.198+0000 7f4b160ef640 1 -- 172.21.15.152:0/3838819296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1019d670 con 0x7f4b10106f10 2024-09-11T22:57:00.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.198+0000 7f4b0cff9640 1 -- 172.21.15.152:0/3838819296 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b08018c10 con 0x7f4b10106f10 2024-09-11T22:57:00.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.198+0000 7f4b160ef640 1 -- 172.21.15.152:0/3838819296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b101a4f50 con 0x7f4b10106f10 2024-09-11T22:57:00.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.200+0000 7f4b0cff9640 1 -- 172.21.15.152:0/3838819296 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4b0800f5f0 con 0x7f4b10106f10 2024-09-11T22:57:00.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.200+0000 7f4b184c8740 1 Processor -- start 2024-09-11T22:57:00.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.200+0000 7f4b184c8740 1 -- start start 2024-09-11T22:57:00.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b184c8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e6f8b40 0x564f4e6f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b184c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f75d0 0x564f4e6f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b184c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e702370 0x564f4e704760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b184c8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x564f4e65d2f0 con 0x564f4e6f8b40 2024-09-11T22:57:00.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b184c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564f4e604cf0 con 0x564f4e702370 2024-09-11T22:57:00.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b184c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564f4e6592b0 con 0x564f4e6f75d0 2024-09-11T22:57:00.219 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b0cff9640 1 --2- 172.21.15.152:0/3838819296 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4af80a2cf0 0x7f4af80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b0effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e6f8b40 0x564f4e6f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b0effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e6f8b40 0x564f4e6f7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39750/0 (socket says 172.21.15.152:39750) 2024-09-11T22:57:00.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b0effd640 1 -- 172.21.15.152:0/2334091290 learned_addr learned my addr 172.21.15.152:0/2334091290 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:00.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b0f7fe640 1 --2- 172.21.15.152:0/2334091290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f75d0 0x564f4e6f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b0ffff640 1 --2- 172.21.15.152:0/3838819296 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4af80a2cf0 0x7f4af80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b0cff9640 1 -- 172.21.15.152:0/3838819296 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4b080cf980 con 0x7f4b10106f10 2024-09-11T22:57:00.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b0effd640 1 --2- 172.21.15.152:0/2334091290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e702370 0x564f4e704760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b0f7fe640 1 -- 172.21.15.152:0/2334091290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e702370 msgr2=0x564f4e704760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:00.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b0f7fe640 1 --2- 172.21.15.152:0/2334091290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e702370 0x564f4e704760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b0f7fe640 1 -- 172.21.15.152:0/2334091290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e6f8b40 msgr2=0x564f4e6f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:00.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b0f7fe640 1 --2- 172.21.15.152:0/2334091290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e6f8b40 0x564f4e6f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b0f7fe640 1 -- 172.21.15.152:0/2334091290 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f4e5e93a0 con 0x564f4e6f75d0 2024-09-11T22:57:00.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b0ffff640 1 --2- 172.21.15.152:0/3838819296 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4af80a2cf0 0x7f4af80a51b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4b0809bbf0 tx=0x7f4b0806bd50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:00.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.201+0000 7f4b0f7fe640 1 --2- 172.21.15.152:0/2334091290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f75d0 0x564f4e6f7a10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4b00005940 tx=0x7f4b000316e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=533b168a4b9ebf82 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:00.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.202+0000 7f4af5ffb640 1 -- 172.21.15.152:0/2334091290 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00040070 con 0x564f4e6f75d0 2024-09-11T22:57:00.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.202+0000 7f4af5ffb640 1 -- 172.21.15.152:0/2334091290 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b00005020 con 0x564f4e6f75d0 2024-09-11T22:57:00.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.202+0000 7f4af5ffb640 1 -- 172.21.15.152:0/2334091290 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00035040 con 0x564f4e6f75d0 2024-09-11T22:57:00.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.202+0000 7f4b184c8740 1 -- 172.21.15.152:0/2334091290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f75d0 msgr2=0x564f4e6f7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:00.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.202+0000 7f4b184c8740 1 --2- 172.21.15.152:0/2334091290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f75d0 0x564f4e6f7a10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4b00005940 tx=0x7f4b000316e0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.202+0000 7f4b184c8740 1 -- 172.21.15.152:0/2334091290 shutdown_connections 2024-09-11T22:57:00.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.202+0000 7f4b184c8740 1 --2- 172.21.15.152:0/2334091290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e702370 0x564f4e704760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.202+0000 7f4b184c8740 1 --2- 172.21.15.152:0/2334091290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f75d0 0x564f4e6f7a10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.202+0000 7f4b184c8740 1 --2- 172.21.15.152:0/2334091290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e6f8b40 0x564f4e6f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.202+0000 7f4b184c8740 1 -- 172.21.15.152:0/2334091290 >> 172.21.15.152:0/2334091290 conn(0x564f4e6c0310 msgr2=0x564f4e6f3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:00.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.202+0000 7f4b184c8740 1 -- 172.21.15.152:0/2334091290 shutdown_connections 2024-09-11T22:57:00.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.202+0000 7f4b184c8740 1 -- 172.21.15.152:0/2334091290 wait complete. 2024-09-11T22:57:00.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.203+0000 7f4b184c8740 1 -- start start 2024-09-11T22:57:00.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.203+0000 7f4b184c8740 1 Processor -- start 2024-09-11T22:57:00.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.203+0000 7f4b184c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f75d0 0x564f4e701d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.203+0000 7f4b184c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f8b40 0x564f4e6fefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.203+0000 7f4b184c8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e702370 0x564f4e6ff570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.203+0000 7f4b184c8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x564f4e705010 con 0x564f4e702370 2024-09-11T22:57:00.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.203+0000 7f4b184c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564f4e618060 con 0x564f4e6f75d0 2024-09-11T22:57:00.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.203+0000 7f4b184c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564f4e705190 con 0x564f4e6f8b40 2024-09-11T22:57:00.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.203+0000 7f4b184c8740 10 client.0 _ll_register_callbacks cb 0x564f4e36b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:00.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.203+0000 7f4b0effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e702370 0x564f4e6ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.203+0000 7f4b0effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e702370 0x564f4e6ff570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39766/0 (socket says 172.21.15.152:39766) 2024-09-11T22:57:00.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.203+0000 7f4b0effd640 1 -- 172.21.15.152:0/4179550303 learned_addr learned my addr 172.21.15.152:0/4179550303 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:00.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.203+0000 7f4b0effd640 1 --2- 172.21.15.152:0/4179550303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f75d0 0x564f4e701d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.203+0000 7f4b0f7fe640 1 --2- 172.21.15.152:0/4179550303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f8b40 0x564f4e6fefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4b0effd640 1 -- 172.21.15.152:0/4179550303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f8b40 msgr2=0x564f4e6fefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:00.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4b0effd640 1 --2- 172.21.15.152:0/4179550303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f8b40 0x564f4e6fefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4b0effd640 1 -- 172.21.15.152:0/4179550303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f75d0 msgr2=0x564f4e701d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:00.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4b0effd640 1 --2- 172.21.15.152:0/4179550303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f75d0 0x564f4e701d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4b0effd640 1 -- 172.21.15.152:0/4179550303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f4e3ce3c0 con 0x564f4e702370 2024-09-11T22:57:00.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4b0effd640 1 --2- 172.21.15.152:0/4179550303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e702370 0x564f4e6ff570 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4b04012ef0 tx=0x7f4b0400a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e65bfab7a80a89 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:00.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4aeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:00.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4aeeffd640 1 -- 172.21.15.152:0/4179550303 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b040110d0 con 0x564f4e702370 2024-09-11T22:57:00.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4aeeffd640 1 -- 172.21.15.152:0/4179550303 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4b0400f420 con 0x564f4e702370 2024-09-11T22:57:00.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4b184c8740 1 -- 172.21.15.152:0/4179550303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b00005020 con 0x564f4e702370 2024-09-11T22:57:00.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4aeeffd640 1 -- 172.21.15.152:0/4179550303 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4b04025020 con 0x564f4e702370 2024-09-11T22:57:00.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4b184c8740 10 client.16596 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:00.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4b184c8740 1 -- 172.21.15.152:0/4179550303 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f4e710a60 con 0x564f4e702370 2024-09-11T22:57:00.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4aeeffd640 1 -- 172.21.15.152:0/4179550303 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b00005020 con 0x564f4e702370 2024-09-11T22:57:00.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4aeeffd640 1 -- 172.21.15.152:0/4179550303 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4b040234b0 con 0x564f4e702370 2024-09-11T22:57:00.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4b184c8740 10 client.16596 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:00.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4b184c8740 10 client.16596 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:57:00.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.204+0000 7f4b184c8740 20 client.16596 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:00.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.205+0000 7f4b184c8740 1 --2- 172.21.15.152:0/4179550303 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x564f4e7122a0 0x564f4e714700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.205+0000 7f4b184c8740 4 client.16596 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:57:00.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.205+0000 7f4b184c8740 1 -- 172.21.15.152:0/4179550303 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f4e66a860 con 0x564f4e7122a0 2024-09-11T22:57:00.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.205+0000 7f4b0f7fe640 1 --2- 172.21.15.152:0/4179550303 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x564f4e7122a0 0x564f4e714700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.205+0000 7f4aeeffd640 10 client.16596 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:57:00.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.205+0000 7f4b0f7fe640 1 --2- 172.21.15.152:0/4179550303 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x564f4e7122a0 0x564f4e714700 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4ee1a4b7c4851cb3 server_cookie=3aae3dde20077d4f in_seq=0 out_seq=0 2024-09-11T22:57:00.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.206+0000 7f4aeeffd640 1 -- 172.21.15.152:0/4179550303 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f4e66a860 con 0x564f4e7122a0 2024-09-11T22:57:00.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.206+0000 7f4aeeffd640 10 client.16596 handle_command_reply: tid=0 2024-09-11T22:57:00.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.206+0000 7f4b184c8740 1 client.16596 shutdown 2024-09-11T22:57:00.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.206+0000 7f4b184c8740 1 -- 172.21.15.152:0/4179550303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e702370 msgr2=0x564f4e6ff570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:00.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.206+0000 7f4b184c8740 1 --2- 172.21.15.152:0/4179550303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e702370 0x564f4e6ff570 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4b04012ef0 tx=0x7f4b0400a2d0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.206+0000 7f4b184c8740 1 -- 172.21.15.152:0/4179550303 shutdown_connections 2024-09-11T22:57:00.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.206+0000 7f4b184c8740 1 --2- 172.21.15.152:0/4179550303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e702370 0x564f4e6ff570 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.206+0000 7f4b184c8740 1 --2- 172.21.15.152:0/4179550303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f8b40 0x564f4e6fefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.206+0000 7f4b184c8740 1 --2- 172.21.15.152:0/4179550303 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x564f4e7122a0 0x564f4e714700 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.206+0000 7f4b184c8740 1 --2- 172.21.15.152:0/4179550303 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f75d0 0x564f4e701d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.206+0000 7f4aeeffd640 0 client.16596 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:57:00.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.206+0000 7f4b184c8740 1 -- 172.21.15.152:0/4179550303 >> 172.21.15.152:0/4179550303 conn(0x564f4e6c0310 msgr2=0x564f4e6f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:00.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.206+0000 7f4b184c8740 1 -- 172.21.15.152:0/4179550303 shutdown_connections 2024-09-11T22:57:00.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.206+0000 7f4b184c8740 1 -- 172.21.15.152:0/4179550303 wait complete. 2024-09-11T22:57:00.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.206+0000 7f4b184c8740 20 client.16596 trim_cache size 0 max 16384 2024-09-11T22:57:00.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.219+0000 7f4b184c8740 1 Processor -- start 2024-09-11T22:57:00.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.219+0000 7f4b184c8740 1 -- start start 2024-09-11T22:57:00.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.219+0000 7f4b184c8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e6f75d0 0x564f4e6f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.219+0000 7f4b184c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f8b40 0x564f4e70e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.219+0000 7f4b184c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e702370 0x564f4e6fa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.219+0000 7f4b184c8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101ad6c0 con 0x564f4e6f75d0 2024-09-11T22:57:00.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.219+0000 7f4b184c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101af200 con 0x564f4e6f8b40 2024-09-11T22:57:00.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.219+0000 7f4b184c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101add90 con 0x564f4e702370 2024-09-11T22:57:00.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.220+0000 7f4b0effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e6f75d0 0x564f4e6f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.220+0000 7f4b0effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e6f75d0 0x564f4e6f3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39780/0 (socket says 172.21.15.152:39780) 2024-09-11T22:57:00.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.220+0000 7f4b0effd640 1 -- 172.21.15.152:0/2184002062 learned_addr learned my addr 172.21.15.152:0/2184002062 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:00.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.220+0000 7f4b0f7fe640 1 --2- 172.21.15.152:0/2184002062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f8b40 0x564f4e70e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.220+0000 7f4b0effd640 1 --2- 172.21.15.152:0/2184002062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e702370 0x564f4e6fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.220+0000 7f4b0effd640 1 -- 172.21.15.152:0/2184002062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e702370 msgr2=0x564f4e6fa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:00.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.220+0000 7f4b0effd640 1 --2- 172.21.15.152:0/2184002062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e702370 0x564f4e6fa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.220+0000 7f4b0effd640 1 -- 172.21.15.152:0/2184002062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f8b40 msgr2=0x564f4e70e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:00.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.220+0000 7f4b0effd640 1 --2- 172.21.15.152:0/2184002062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f8b40 0x564f4e70e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.220+0000 7f4b0effd640 1 -- 172.21.15.152:0/2184002062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f4e705180 con 0x564f4e6f75d0 2024-09-11T22:57:00.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.220+0000 7f4b0effd640 1 --2- 172.21.15.152:0/2184002062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e6f75d0 0x564f4e6f3e10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4b040129f0 tx=0x7f4b0400a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54a0c2a8ba4af0e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:00.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.220+0000 7f4af57fa640 1 -- 172.21.15.152:0/2184002062 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b0400a590 con 0x564f4e6f75d0 2024-09-11T22:57:00.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.220+0000 7f4af57fa640 1 -- 172.21.15.152:0/2184002062 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b04031030 con 0x564f4e6f75d0 2024-09-11T22:57:00.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.220+0000 7f4af57fa640 1 -- 172.21.15.152:0/2184002062 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04005490 con 0x564f4e6f75d0 2024-09-11T22:57:00.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.221+0000 7f4b184c8740 1 -- 172.21.15.152:0/2184002062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e6f75d0 msgr2=0x564f4e6f3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:00.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.221+0000 7f4b184c8740 1 --2- 172.21.15.152:0/2184002062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e6f75d0 0x564f4e6f3e10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4b040129f0 tx=0x7f4b0400a4b0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.221+0000 7f4b184c8740 1 -- 172.21.15.152:0/2184002062 shutdown_connections 2024-09-11T22:57:00.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.221+0000 7f4b184c8740 1 --2- 172.21.15.152:0/2184002062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e702370 0x564f4e6fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.221+0000 7f4b184c8740 1 --2- 172.21.15.152:0/2184002062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f8b40 0x564f4e70e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.221+0000 7f4b184c8740 1 --2- 172.21.15.152:0/2184002062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e6f75d0 0x564f4e6f3e10 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.221+0000 7f4b184c8740 1 -- 172.21.15.152:0/2184002062 >> 172.21.15.152:0/2184002062 conn(0x564f4e6c0310 msgr2=0x564f4e6f3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:00.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.221+0000 7f4b184c8740 1 -- 172.21.15.152:0/2184002062 shutdown_connections 2024-09-11T22:57:00.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.221+0000 7f4b184c8740 1 -- 172.21.15.152:0/2184002062 wait complete. 2024-09-11T22:57:00.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.221+0000 7f4b184c8740 1 -- start start 2024-09-11T22:57:00.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b184c8740 1 Processor -- start 2024-09-11T22:57:00.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b184c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f75d0 0x564f4e70ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b184c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f8b40 0x564f4e6fbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b184c8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e702370 0x564f4e6fc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b184c8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101aeb30 con 0x564f4e702370 2024-09-11T22:57:00.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b184c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101ae460 con 0x564f4e6f75d0 2024-09-11T22:57:00.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b184c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101ab6a0 con 0x564f4e6f8b40 2024-09-11T22:57:00.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b184c8740 10 client.0 _ll_register_callbacks cb 0x7f4b0400ade0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:00.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b0effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f75d0 0x564f4e70ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b0effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f75d0 0x564f4e70ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:37240/0 (socket says 172.21.15.152:37240) 2024-09-11T22:57:00.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b0effd640 1 -- 172.21.15.152:0/2490676683 learned_addr learned my addr 172.21.15.152:0/2490676683 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:00.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b0effd640 1 --2- 172.21.15.152:0/2490676683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e702370 0x564f4e6fc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b0f7fe640 1 --2- 172.21.15.152:0/2490676683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f8b40 0x564f4e6fbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b0effd640 1 -- 172.21.15.152:0/2490676683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f8b40 msgr2=0x564f4e6fbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:00.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b0effd640 1 --2- 172.21.15.152:0/2490676683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f8b40 0x564f4e6fbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b0effd640 1 -- 172.21.15.152:0/2490676683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e702370 msgr2=0x564f4e6fc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:00.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b0effd640 1 --2- 172.21.15.152:0/2490676683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e702370 0x564f4e6fc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b0effd640 1 -- 172.21.15.152:0/2490676683 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f4e6fcb70 con 0x564f4e6f75d0 2024-09-11T22:57:00.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.222+0000 7f4b0f7fe640 1 --2- 172.21.15.152:0/2490676683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f8b40 0x564f4e6fbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:57:00.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4b0effd640 1 --2- 172.21.15.152:0/2490676683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f75d0 0x564f4e70ac20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4b040058c0 tx=0x7f4b0400a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8197626f6d23918e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:00.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4aeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-11T22:57:00.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4aeeffd640 1 -- 172.21.15.152:0/2490676683 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04004220 con 0x564f4e6f75d0 2024-09-11T22:57:00.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4b184c8740 1 -- 172.21.15.152:0/2490676683 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b04031030 con 0x564f4e6f75d0 2024-09-11T22:57:00.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4aeeffd640 1 -- 172.21.15.152:0/2490676683 <== mon.2 v2:172.21.15.191:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4b040043c0 con 0x564f4e6f75d0 2024-09-11T22:57:00.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4aeeffd640 1 -- 172.21.15.152:0/2490676683 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4b04005110 con 0x564f4e6f75d0 2024-09-11T22:57:00.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4b184c8740 10 client.25772 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:00.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4b184c8740 1 -- 172.21.15.152:0/2490676683 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f4e77d080 con 0x564f4e6f75d0 2024-09-11T22:57:00.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4aeeffd640 1 -- 172.21.15.152:0/2490676683 <== mon.2 v2:172.21.15.191:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b04031030 con 0x564f4e6f75d0 2024-09-11T22:57:00.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4aeeffd640 1 -- 172.21.15.152:0/2490676683 <== mon.2 v2:172.21.15.191:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4b04003660 con 0x564f4e6f75d0 2024-09-11T22:57:00.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4b184c8740 10 client.25772 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:00.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4b184c8740 10 client.25772 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:57:00.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4b184c8740 20 client.25772 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:00.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4b184c8740 1 --2- 172.21.15.152:0/2490676683 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x564f4e77d6c0 0x564f4e77fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:00.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4b184c8740 4 client.25772 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:57:00.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4b184c8740 1 -- 172.21.15.152:0/2490676683 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f4e664d70 con 0x564f4e77d6c0 2024-09-11T22:57:00.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.223+0000 7f4b0f7fe640 1 --2- 172.21.15.152:0/2490676683 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x564f4e77d6c0 0x564f4e77fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:00.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4aeeffd640 10 client.25772 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:57:00.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4b0f7fe640 1 --2- 172.21.15.152:0/2490676683 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x564f4e77d6c0 0x564f4e77fae0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7e9abb0ec6748972 server_cookie=c65fff2ef7582c8f in_seq=0 out_seq=0 2024-09-11T22:57:00.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4aeeffd640 1 -- 172.21.15.152:0/2490676683 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f4e664d70 con 0x564f4e77d6c0 2024-09-11T22:57:00.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4aeeffd640 10 client.25772 handle_command_reply: tid=0 2024-09-11T22:57:00.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4b184c8740 1 client.25772 shutdown 2024-09-11T22:57:00.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4b184c8740 1 -- 172.21.15.152:0/2490676683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f75d0 msgr2=0x564f4e70ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:00.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4b184c8740 1 --2- 172.21.15.152:0/2490676683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f75d0 0x564f4e70ac20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4b040058c0 tx=0x7f4b0400a890 comp rx=0 tx=0).stop 2024-09-11T22:57:00.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4b184c8740 1 -- 172.21.15.152:0/2490676683 shutdown_connections 2024-09-11T22:57:00.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4b184c8740 1 --2- 172.21.15.152:0/2490676683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564f4e702370 0x564f4e6fc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4b184c8740 1 --2- 172.21.15.152:0/2490676683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564f4e6f8b40 0x564f4e6fbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4b184c8740 1 --2- 172.21.15.152:0/2490676683 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x564f4e77d6c0 0x564f4e77fae0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4b184c8740 1 --2- 172.21.15.152:0/2490676683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564f4e6f75d0 0x564f4e70ac20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4b184c8740 1 -- 172.21.15.152:0/2490676683 >> 172.21.15.152:0/2490676683 conn(0x564f4e6c0310 msgr2=0x564f4e700830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:00.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4aeeffd640 0 client.25772 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:57:00.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4b184c8740 1 -- 172.21.15.152:0/2490676683 shutdown_connections 2024-09-11T22:57:00.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4b184c8740 1 -- 172.21.15.152:0/2490676683 wait complete. 2024-09-11T22:57:00.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.224+0000 7f4b184c8740 20 client.25772 trim_cache size 0 max 16384 2024-09-11T22:57:00.238 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:57:00.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.225+0000 7f4b160ef640 1 -- 172.21.15.152:0/3838819296 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4af80a2cf0 msgr2=0x7f4af80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:00.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.225+0000 7f4b160ef640 1 --2- 172.21.15.152:0/3838819296 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4af80a2cf0 0x7f4af80a51b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4b0809bbf0 tx=0x7f4b0806bd50 comp rx=0 tx=0).stop 2024-09-11T22:57:00.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.225+0000 7f4b160ef640 1 -- 172.21.15.152:0/3838819296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b10106f10 msgr2=0x7f4b1019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:00.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.225+0000 7f4b160ef640 1 --2- 172.21.15.152:0/3838819296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b10106f10 0x7f4b1019cca0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800cce0 tx=0x7f4b0800a620 comp rx=0 tx=0).stop 2024-09-11T22:57:00.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.225+0000 7f4b160ef640 1 -- 172.21.15.152:0/3838819296 shutdown_connections 2024-09-11T22:57:00.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.225+0000 7f4b160ef640 1 --2- 172.21.15.152:0/3838819296 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4af80a2cf0 0x7f4af80a51b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.225+0000 7f4b160ef640 1 --2- 172.21.15.152:0/3838819296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b10106f10 0x7f4b1019cca0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.225+0000 7f4b160ef640 1 --2- 172.21.15.152:0/3838819296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b10102790 0x7f4b10198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.225+0000 7f4b160ef640 1 --2- 172.21.15.152:0/3838819296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b10101de0 0x7f4b101983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:00.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.225+0000 7f4b160ef640 1 -- 172.21.15.152:0/3838819296 >> 172.21.15.152:0/3838819296 conn(0x7f4b100fd8c0 msgr2=0x7f4b100febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:00.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.225+0000 7f4b160ef640 1 -- 172.21.15.152:0/3838819296 shutdown_connections 2024-09-11T22:57:00.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:00.225+0000 7f4b160ef640 1 -- 172.21.15.152:0/3838819296 wait complete. 2024-09-11T22:57:00.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:00 smithi191 ceph-mon[35211]: pgmap v126: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:57:00.789 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:00 smithi152 ceph-mon[31181]: pgmap v126: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:57:00.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:00 smithi204 ceph-mon[35454]: pgmap v126: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:57:01.076 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:57:01.325 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:57:01.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.876+0000 7f407c4bd640 1 -- 172.21.15.152:0/4281678083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4074100e20 msgr2=0x7f4074101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:01.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.876+0000 7f407c4bd640 1 --2- 172.21.15.152:0/4281678083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4074100e20 0x7f4074101200 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4070009900 tx=0x7f407002a480 comp rx=0 tx=0).stop 2024-09-11T22:57:01.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.876+0000 7f407c4bd640 1 -- 172.21.15.152:0/4281678083 shutdown_connections 2024-09-11T22:57:01.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.876+0000 7f407c4bd640 1 --2- 172.21.15.152:0/4281678083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4074102ef0 0x7f407410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.876+0000 7f407c4bd640 1 --2- 172.21.15.152:0/4281678083 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4074102570 0x7f40741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.876+0000 7f407c4bd640 1 --2- 172.21.15.152:0/4281678083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4074100e20 0x7f4074101200 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.876+0000 7f407c4bd640 1 -- 172.21.15.152:0/4281678083 >> 172.21.15.152:0/4281678083 conn(0x7f40740fc7d0 msgr2=0x7f40740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:01.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.876+0000 7f407c4bd640 1 -- 172.21.15.152:0/4281678083 shutdown_connections 2024-09-11T22:57:01.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.876+0000 7f407c4bd640 1 -- 172.21.15.152:0/4281678083 wait complete. 2024-09-11T22:57:01.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.876+0000 7f407c4bd640 1 Processor -- start 2024-09-11T22:57:01.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407c4bd640 1 -- start start 2024-09-11T22:57:01.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407c4bd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4074100e20 0x7f40741984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407c4bd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4074102570 0x7f4074198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407c4bd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4074102ef0 0x7f407419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407c4bd640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4074110f80 con 0x7f4074102ef0 2024-09-11T22:57:01.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407c4bd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4074110e00 con 0x7f4074100e20 2024-09-11T22:57:01.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407c4bd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4074111100 con 0x7f4074102570 2024-09-11T22:57:01.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407aa5a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4074102ef0 0x7f407419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:01.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407a259640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4074100e20 0x7f40741984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:01.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407aa5a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4074102ef0 0x7f407419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39802/0 (socket says 172.21.15.152:39802) 2024-09-11T22:57:01.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407aa5a640 1 -- 172.21.15.152:0/325107484 learned_addr learned my addr 172.21.15.152:0/325107484 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:01.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f4079a58640 1 --2- 172.21.15.152:0/325107484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4074102570 0x7f4074198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:01.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407aa5a640 1 -- 172.21.15.152:0/325107484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4074102570 msgr2=0x7f4074198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:01.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407aa5a640 1 --2- 172.21.15.152:0/325107484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4074102570 0x7f4074198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407aa5a640 1 -- 172.21.15.152:0/325107484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4074100e20 msgr2=0x7f40741984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:01.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407aa5a640 1 --2- 172.21.15.152:0/325107484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4074100e20 0x7f40741984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407aa5a640 1 -- 172.21.15.152:0/325107484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407419d490 con 0x7f4074102ef0 2024-09-11T22:57:01.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f4079a58640 1 --2- 172.21.15.152:0/325107484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4074102570 0x7f4074198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:01.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.877+0000 7f407aa5a640 1 --2- 172.21.15.152:0/325107484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4074102ef0 0x7f407419cdb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f406c00eaa0 tx=0x7f406c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:01.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.878+0000 7f406b7fe640 1 -- 172.21.15.152:0/325107484 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c00cdb0 con 0x7f4074102ef0 2024-09-11T22:57:01.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.878+0000 7f406b7fe640 1 -- 172.21.15.152:0/325107484 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f406c005020 con 0x7f4074102ef0 2024-09-11T22:57:01.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.878+0000 7f407c4bd640 1 -- 172.21.15.152:0/325107484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407419d780 con 0x7f4074102ef0 2024-09-11T22:57:01.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.878+0000 7f406b7fe640 1 -- 172.21.15.152:0/325107484 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c0116b0 con 0x7f4074102ef0 2024-09-11T22:57:01.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.878+0000 7f407c4bd640 1 -- 172.21.15.152:0/325107484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40741a5060 con 0x7f4074102ef0 2024-09-11T22:57:01.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.880+0000 7f406b7fe640 1 -- 172.21.15.152:0/325107484 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f406c005560 con 0x7f4074102ef0 2024-09-11T22:57:01.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.880+0000 7f407e896740 1 Processor -- start 2024-09-11T22:57:01.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.880+0000 7f407e896740 1 -- start start 2024-09-11T22:57:01.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f407e896740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502eb40 0x55f3d502d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f407e896740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d502d5d0 0x55f3d502da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f407e896740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d503c780 0x55f3d503eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f407e896740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f3d4f932f0 con 0x55f3d502eb40 2024-09-11T22:57:01.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f407e896740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f3d4f3acf0 con 0x55f3d503c780 2024-09-11T22:57:01.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f407e896740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f3d4f8f2b0 con 0x55f3d502d5d0 2024-09-11T22:57:01.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f4079a58640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502eb40 0x55f3d502d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:01.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f4079a58640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502eb40 0x55f3d502d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39808/0 (socket says 172.21.15.152:39808) 2024-09-11T22:57:01.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f4079a58640 1 -- 172.21.15.152:0/3184699302 learned_addr learned my addr 172.21.15.152:0/3184699302 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:01.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f406b7fe640 1 --2- 172.21.15.152:0/325107484 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f40600a2cf0 0x7f40600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f407a259640 1 --2- 172.21.15.152:0/3184699302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d502d5d0 0x55f3d502da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:01.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f4079a58640 1 --2- 172.21.15.152:0/3184699302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d503c780 0x55f3d503eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:01.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f407aa5a640 1 --2- 172.21.15.152:0/325107484 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f40600a2cf0 0x7f40600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:01.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f4079a58640 1 -- 172.21.15.152:0/3184699302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d502d5d0 msgr2=0x55f3d502da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:01.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f4079a58640 1 --2- 172.21.15.152:0/3184699302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d502d5d0 0x55f3d502da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f4079a58640 1 -- 172.21.15.152:0/3184699302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d503c780 msgr2=0x55f3d503eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:01.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f4079a58640 1 --2- 172.21.15.152:0/3184699302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d503c780 0x55f3d503eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f4079a58640 1 -- 172.21.15.152:0/3184699302 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3d4f1f3a0 con 0x55f3d502eb40 2024-09-11T22:57:01.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f406b7fe640 1 -- 172.21.15.152:0/325107484 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f406c0cf680 con 0x7f4074102ef0 2024-09-11T22:57:01.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.881+0000 7f4079a58640 1 --2- 172.21.15.152:0/3184699302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502eb40 0x55f3d502d090 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f406400cf90 tx=0x7f406400c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bc20ce4d2ce834 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:01.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.882+0000 7f4068ff9640 1 -- 172.21.15.152:0/3184699302 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064018030 con 0x55f3d502eb40 2024-09-11T22:57:01.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.882+0000 7f407aa5a640 1 --2- 172.21.15.152:0/325107484 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f40600a2cf0 0x7f40600a51b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f406c00efd0 tx=0x7f406c091080 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:01.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.882+0000 7f4068ff9640 1 -- 172.21.15.152:0/3184699302 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4064005500 con 0x55f3d502eb40 2024-09-11T22:57:01.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.882+0000 7f4068ff9640 1 -- 172.21.15.152:0/3184699302 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4064010840 con 0x55f3d502eb40 2024-09-11T22:57:01.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.882+0000 7f407e896740 1 -- 172.21.15.152:0/3184699302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502eb40 msgr2=0x55f3d502d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:01.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.882+0000 7f407e896740 1 --2- 172.21.15.152:0/3184699302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502eb40 0x55f3d502d090 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f406400cf90 tx=0x7f406400c640 comp rx=0 tx=0).stop 2024-09-11T22:57:01.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.882+0000 7f407e896740 1 -- 172.21.15.152:0/3184699302 shutdown_connections 2024-09-11T22:57:01.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.882+0000 7f407e896740 1 --2- 172.21.15.152:0/3184699302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d503c780 0x55f3d503eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.882+0000 7f407e896740 1 --2- 172.21.15.152:0/3184699302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d502d5d0 0x55f3d502da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.882+0000 7f407e896740 1 --2- 172.21.15.152:0/3184699302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502eb40 0x55f3d502d090 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.882+0000 7f407e896740 1 -- 172.21.15.152:0/3184699302 >> 172.21.15.152:0/3184699302 conn(0x55f3d4ff6310 msgr2=0x55f3d5029c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:01.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.882+0000 7f407e896740 1 -- 172.21.15.152:0/3184699302 shutdown_connections 2024-09-11T22:57:01.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.882+0000 7f407e896740 1 -- 172.21.15.152:0/3184699302 wait complete. 2024-09-11T22:57:01.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f407e896740 1 -- start start 2024-09-11T22:57:01.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f407e896740 1 Processor -- start 2024-09-11T22:57:01.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f407e896740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502d5d0 0x55f3d503c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f407e896740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d502eb40 0x55f3d50393f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f407e896740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d503c780 0x55f3d50399e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f407e896740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f3d503f480 con 0x55f3d502d5d0 2024-09-11T22:57:01.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f407e896740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f3d4f4e060 con 0x55f3d502eb40 2024-09-11T22:57:01.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f407e896740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f3d503f600 con 0x55f3d503c780 2024-09-11T22:57:01.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f407e896740 10 client.0 _ll_register_callbacks cb 0x55f3d4ca18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:01.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f4079a58640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502d5d0 0x55f3d503c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:01.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f4079a58640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502d5d0 0x55f3d503c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39824/0 (socket says 172.21.15.152:39824) 2024-09-11T22:57:01.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f4079a58640 1 -- 172.21.15.152:0/4016543654 learned_addr learned my addr 172.21.15.152:0/4016543654 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:01.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f407a259640 1 --2- 172.21.15.152:0/4016543654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d502eb40 0x55f3d50393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:01.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f4079a58640 1 -- 172.21.15.152:0/4016543654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d503c780 msgr2=0x55f3d50399e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:01.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f4079a58640 1 --2- 172.21.15.152:0/4016543654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d503c780 0x55f3d50399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f4079a58640 1 -- 172.21.15.152:0/4016543654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d502eb40 msgr2=0x55f3d50393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:01.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f4079a58640 1 --2- 172.21.15.152:0/4016543654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d502eb40 0x55f3d50393f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.883+0000 7f4079a58640 1 -- 172.21.15.152:0/4016543654 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3d4d043c0 con 0x55f3d502d5d0 2024-09-11T22:57:01.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.884+0000 7f4079a58640 1 --2- 172.21.15.152:0/4016543654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502d5d0 0x55f3d503c140 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4064005230 tx=0x7f4064005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=699912805ab8cf35 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:01.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.884+0000 7f405dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:01.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.884+0000 7f405dffb640 1 -- 172.21.15.152:0/4016543654 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406402a060 con 0x55f3d502d5d0 2024-09-11T22:57:01.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.884+0000 7f405dffb640 1 -- 172.21.15.152:0/4016543654 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4064010840 con 0x55f3d502d5d0 2024-09-11T22:57:01.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.884+0000 7f407e896740 1 -- 172.21.15.152:0/4016543654 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4064005500 con 0x55f3d502d5d0 2024-09-11T22:57:01.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.884+0000 7f405dffb640 1 -- 172.21.15.152:0/4016543654 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4064018040 con 0x55f3d502d5d0 2024-09-11T22:57:01.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.884+0000 7f407e896740 10 client.16614 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:01.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.884+0000 7f407e896740 1 -- 172.21.15.152:0/4016543654 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f3d504aee0 con 0x55f3d502d5d0 2024-09-11T22:57:01.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.884+0000 7f405dffb640 1 -- 172.21.15.152:0/4016543654 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4064005500 con 0x55f3d502d5d0 2024-09-11T22:57:01.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.884+0000 7f405dffb640 1 -- 172.21.15.152:0/4016543654 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f406401d5a0 con 0x55f3d502d5d0 2024-09-11T22:57:01.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.884+0000 7f407e896740 10 client.16614 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:01.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.884+0000 7f407e896740 10 client.16614 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:57:01.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.884+0000 7f407e896740 20 client.16614 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:01.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.884+0000 7f407e896740 1 --2- 172.21.15.152:0/4016543654 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55f3d504c2e0 0x55f3d504e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.884+0000 7f407e896740 4 client.16614 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:57:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.884+0000 7f407e896740 1 -- 172.21.15.152:0/4016543654 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3d4fa0860 con 0x55f3d504c2e0 2024-09-11T22:57:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.885+0000 7f407a259640 1 --2- 172.21.15.152:0/4016543654 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55f3d504c2e0 0x55f3d504e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.885+0000 7f407a259640 1 --2- 172.21.15.152:0/4016543654 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55f3d504c2e0 0x55f3d504e6d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=386449fc9eda9747 server_cookie=5c6544b88771ac34 in_seq=0 out_seq=0 2024-09-11T22:57:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.885+0000 7f405dffb640 10 client.16614 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:57:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.886+0000 7f405dffb640 1 -- 172.21.15.152:0/4016543654 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3d4fa0860 con 0x55f3d504c2e0 2024-09-11T22:57:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.886+0000 7f405dffb640 10 client.16614 handle_command_reply: tid=0 2024-09-11T22:57:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.886+0000 7f407e896740 1 client.16614 shutdown 2024-09-11T22:57:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.886+0000 7f407e896740 1 -- 172.21.15.152:0/4016543654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502d5d0 msgr2=0x55f3d503c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.886+0000 7f407e896740 1 --2- 172.21.15.152:0/4016543654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502d5d0 0x55f3d503c140 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4064005230 tx=0x7f4064005260 comp rx=0 tx=0).stop 2024-09-11T22:57:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.886+0000 7f407e896740 1 -- 172.21.15.152:0/4016543654 shutdown_connections 2024-09-11T22:57:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.886+0000 7f407e896740 1 --2- 172.21.15.152:0/4016543654 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55f3d504c2e0 0x55f3d504e6d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.886+0000 7f407e896740 1 --2- 172.21.15.152:0/4016543654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d503c780 0x55f3d50399e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.886+0000 7f407e896740 1 --2- 172.21.15.152:0/4016543654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d502eb40 0x55f3d50393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.886+0000 7f405dffb640 0 client.16614 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:57:01.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.886+0000 7f407e896740 1 --2- 172.21.15.152:0/4016543654 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502d5d0 0x55f3d503c140 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.886+0000 7f407e896740 1 -- 172.21.15.152:0/4016543654 >> 172.21.15.152:0/4016543654 conn(0x55f3d4ff6310 msgr2=0x55f3d502f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:01.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.886+0000 7f407e896740 1 -- 172.21.15.152:0/4016543654 shutdown_connections 2024-09-11T22:57:01.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.886+0000 7f407e896740 1 -- 172.21.15.152:0/4016543654 wait complete. 2024-09-11T22:57:01.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.886+0000 7f407e896740 20 client.16614 trim_cache size 0 max 16384 2024-09-11T22:57:01.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.899+0000 7f407e896740 1 Processor -- start 2024-09-11T22:57:01.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.899+0000 7f407e896740 1 -- start start 2024-09-11T22:57:01.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.899+0000 7f407e896740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502d5d0 0x55f3d5048720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.899+0000 7f407e896740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d502eb40 0x55f3d50383d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.899+0000 7f407e896740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d503c780 0x55f3d5053dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.899+0000 7f407e896740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f40741ab570 con 0x55f3d502d5d0 2024-09-11T22:57:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.899+0000 7f407e896740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f40741ad0b0 con 0x55f3d502eb40 2024-09-11T22:57:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.899+0000 7f407e896740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f40741abc40 con 0x55f3d503c780 2024-09-11T22:57:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.899+0000 7f4079a58640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502d5d0 0x55f3d5048720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.900+0000 7f4079a58640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502d5d0 0x55f3d5048720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39830/0 (socket says 172.21.15.152:39830) 2024-09-11T22:57:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.900+0000 7f4079a58640 1 -- 172.21.15.152:0/3128897051 learned_addr learned my addr 172.21.15.152:0/3128897051 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.900+0000 7f4079a58640 1 -- 172.21.15.152:0/3128897051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d503c780 msgr2=0x55f3d5053dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.900+0000 7f4079a58640 1 --2- 172.21.15.152:0/3128897051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d503c780 0x55f3d5053dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.900+0000 7f4079a58640 1 -- 172.21.15.152:0/3128897051 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d502eb40 msgr2=0x55f3d50383d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:01.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.900+0000 7f407a259640 1 --2- 172.21.15.152:0/3128897051 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d502eb40 0x55f3d50383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.900+0000 7f4079a58640 1 --2- 172.21.15.152:0/3128897051 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d502eb40 0x55f3d50383d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.900+0000 7f4079a58640 1 -- 172.21.15.152:0/3128897051 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3d502a960 con 0x55f3d502d5d0 2024-09-11T22:57:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.900+0000 7f407a259640 1 --2- 172.21.15.152:0/3128897051 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d502eb40 0x55f3d50383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:57:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.900+0000 7f4079a58640 1 --2- 172.21.15.152:0/3128897051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502d5d0 0x55f3d5048720 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4064001eb0 tx=0x7f4064004eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38b280a1bb9763f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.900+0000 7f405ffff640 1 -- 172.21.15.152:0/3128897051 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406402a070 con 0x55f3d502d5d0 2024-09-11T22:57:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.900+0000 7f405ffff640 1 -- 172.21.15.152:0/3128897051 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f406400cd70 con 0x55f3d502d5d0 2024-09-11T22:57:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.900+0000 7f405ffff640 1 -- 172.21.15.152:0/3128897051 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40640055c0 con 0x55f3d502d5d0 2024-09-11T22:57:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.901+0000 7f407e896740 1 -- 172.21.15.152:0/3128897051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502d5d0 msgr2=0x55f3d5048720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.901+0000 7f407e896740 1 --2- 172.21.15.152:0/3128897051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502d5d0 0x55f3d5048720 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4064001eb0 tx=0x7f4064004eb0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.901+0000 7f407e896740 1 -- 172.21.15.152:0/3128897051 shutdown_connections 2024-09-11T22:57:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.901+0000 7f407e896740 1 --2- 172.21.15.152:0/3128897051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d503c780 0x55f3d5053dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.901+0000 7f407e896740 1 --2- 172.21.15.152:0/3128897051 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d502eb40 0x55f3d50383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.901+0000 7f407e896740 1 --2- 172.21.15.152:0/3128897051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502d5d0 0x55f3d5048720 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.901+0000 7f407e896740 1 -- 172.21.15.152:0/3128897051 >> 172.21.15.152:0/3128897051 conn(0x55f3d4ff6310 msgr2=0x55f3d5037cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:01.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.901+0000 7f407e896740 1 -- 172.21.15.152:0/3128897051 shutdown_connections 2024-09-11T22:57:01.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.901+0000 7f407e896740 1 -- 172.21.15.152:0/3128897051 wait complete. 2024-09-11T22:57:01.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.901+0000 7f407e896740 1 -- start start 2024-09-11T22:57:01.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.901+0000 7f407e896740 1 Processor -- start 2024-09-11T22:57:01.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f407e896740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d502d5d0 0x55f3d503ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f407e896740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502eb40 0x55f3d5040510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f407e896740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d503c780 0x55f3d50aed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f407e896740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f40741ac9e0 con 0x55f3d502eb40 2024-09-11T22:57:01.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f407e896740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f40741ac310 con 0x55f3d503c780 2024-09-11T22:57:01.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f407e896740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f40741a9550 con 0x55f3d502d5d0 2024-09-11T22:57:01.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f407e896740 10 client.0 _ll_register_callbacks cb 0x7f4070009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:01.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f407a259640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502eb40 0x55f3d5040510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:01.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f407a259640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502eb40 0x55f3d5040510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39834/0 (socket says 172.21.15.152:39834) 2024-09-11T22:57:01.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f407a259640 1 -- 172.21.15.152:0/100302648 learned_addr learned my addr 172.21.15.152:0/100302648 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:01.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f4079a58640 1 --2- 172.21.15.152:0/100302648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d502d5d0 0x55f3d503ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:01.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f4079a58640 1 --2- 172.21.15.152:0/100302648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d503c780 0x55f3d50aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:01.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f407a259640 1 -- 172.21.15.152:0/100302648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d502d5d0 msgr2=0x55f3d503ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:01.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f407a259640 1 --2- 172.21.15.152:0/100302648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d502d5d0 0x55f3d503ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f407a259640 1 -- 172.21.15.152:0/100302648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d503c780 msgr2=0x55f3d50aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:01.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f407a259640 1 --2- 172.21.15.152:0/100302648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d503c780 0x55f3d50aed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f407a259640 1 -- 172.21.15.152:0/100302648 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3d50af390 con 0x55f3d502eb40 2024-09-11T22:57:01.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.902+0000 7f4079a58640 1 --2- 172.21.15.152:0/100302648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d503c780 0x55f3d50aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:01.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.903+0000 7f407a259640 1 --2- 172.21.15.152:0/100302648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502eb40 0x55f3d5040510 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f40700314e0 tx=0x7f407002aa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a90fb3a5e8b270 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:01.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.903+0000 7f405dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:01.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.903+0000 7f405dffb640 1 -- 172.21.15.152:0/100302648 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070049030 con 0x55f3d502eb40 2024-09-11T22:57:01.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.903+0000 7f405dffb640 1 -- 172.21.15.152:0/100302648 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f4070005020 con 0x55f3d502eb40 2024-09-11T22:57:01.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.903+0000 7f407e896740 1 -- 172.21.15.152:0/100302648 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f406400cd70 con 0x55f3d502eb40 2024-09-11T22:57:01.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.903+0000 7f405dffb640 1 -- 172.21.15.152:0/100302648 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4070053050 con 0x55f3d502eb40 2024-09-11T22:57:01.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.903+0000 7f407e896740 10 client.16620 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:01.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.903+0000 7f407e896740 1 -- 172.21.15.152:0/100302648 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f3d50b66e0 con 0x55f3d502eb40 2024-09-11T22:57:01.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.903+0000 7f405dffb640 1 -- 172.21.15.152:0/100302648 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f406400cd70 con 0x55f3d502eb40 2024-09-11T22:57:01.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.903+0000 7f405dffb640 1 -- 172.21.15.152:0/100302648 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4070004830 con 0x55f3d502eb40 2024-09-11T22:57:01.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.903+0000 7f407e896740 10 client.16620 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:01.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.903+0000 7f407e896740 10 client.16620 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:57:01.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.903+0000 7f407e896740 20 client.16620 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:01.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.903+0000 7f407e896740 1 --2- 172.21.15.152:0/100302648 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55f3d50b6e90 0x55f3d50b92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:01.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.903+0000 7f407e896740 4 client.16620 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:57:01.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.903+0000 7f407e896740 1 -- 172.21.15.152:0/100302648 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3d4f9ad70 con 0x55f3d50b6e90 2024-09-11T22:57:01.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.904+0000 7f4079a58640 1 --2- 172.21.15.152:0/100302648 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55f3d50b6e90 0x55f3d50b92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:01.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.904+0000 7f4079a58640 1 --2- 172.21.15.152:0/100302648 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55f3d50b6e90 0x55f3d50b92b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e477451392d6ae9f server_cookie=7326992c898bcab9 in_seq=0 out_seq=0 2024-09-11T22:57:01.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.904+0000 7f405dffb640 10 client.16620 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:57:01.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.904+0000 7f405dffb640 1 -- 172.21.15.152:0/100302648 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3d4f9ad70 con 0x55f3d50b6e90 2024-09-11T22:57:01.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.904+0000 7f405dffb640 10 client.16620 handle_command_reply: tid=0 2024-09-11T22:57:01.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.904+0000 7f407e896740 1 client.16620 shutdown 2024-09-11T22:57:01.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.904+0000 7f407e896740 1 -- 172.21.15.152:0/100302648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502eb40 msgr2=0x55f3d5040510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:01.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.904+0000 7f407e896740 1 --2- 172.21.15.152:0/100302648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502eb40 0x55f3d5040510 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f40700314e0 tx=0x7f407002aa20 comp rx=0 tx=0).stop 2024-09-11T22:57:01.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.904+0000 7f407e896740 1 -- 172.21.15.152:0/100302648 shutdown_connections 2024-09-11T22:57:01.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.904+0000 7f407e896740 1 --2- 172.21.15.152:0/100302648 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55f3d50b6e90 0x55f3d50b92b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407e896740 1 --2- 172.21.15.152:0/100302648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f3d503c780 0x55f3d50aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407e896740 1 --2- 172.21.15.152:0/100302648 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f3d502eb40 0x55f3d5040510 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407e896740 1 --2- 172.21.15.152:0/100302648 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f3d502d5d0 0x55f3d503ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407e896740 1 -- 172.21.15.152:0/100302648 >> 172.21.15.152:0/100302648 conn(0x55f3d4ff6310 msgr2=0x55f3d50384e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:01.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f405dffb640 0 client.16620 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:57:01.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407e896740 1 -- 172.21.15.152:0/100302648 shutdown_connections 2024-09-11T22:57:01.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407e896740 1 -- 172.21.15.152:0/100302648 wait complete. 2024-09-11T22:57:01.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407e896740 20 client.16620 trim_cache size 0 max 16384 2024-09-11T22:57:01.919 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:57:01.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407c4bd640 1 -- 172.21.15.152:0/325107484 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f40600a2cf0 msgr2=0x7f40600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:01.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407c4bd640 1 --2- 172.21.15.152:0/325107484 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f40600a2cf0 0x7f40600a51b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f406c00efd0 tx=0x7f406c091080 comp rx=0 tx=0).stop 2024-09-11T22:57:01.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407c4bd640 1 -- 172.21.15.152:0/325107484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4074102ef0 msgr2=0x7f407419cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:01.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407c4bd640 1 --2- 172.21.15.152:0/325107484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4074102ef0 0x7f407419cdb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f406c00eaa0 tx=0x7f406c00ef60 comp rx=0 tx=0).stop 2024-09-11T22:57:01.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407c4bd640 1 -- 172.21.15.152:0/325107484 shutdown_connections 2024-09-11T22:57:01.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407c4bd640 1 --2- 172.21.15.152:0/325107484 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f40600a2cf0 0x7f40600a51b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407c4bd640 1 --2- 172.21.15.152:0/325107484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4074102ef0 0x7f407419cdb0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407c4bd640 1 --2- 172.21.15.152:0/325107484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4074102570 0x7f4074198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407c4bd640 1 --2- 172.21.15.152:0/325107484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4074100e20 0x7f40741984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:01.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407c4bd640 1 -- 172.21.15.152:0/325107484 >> 172.21.15.152:0/325107484 conn(0x7f40740fc7d0 msgr2=0x7f4074105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:01.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407c4bd640 1 -- 172.21.15.152:0/325107484 shutdown_connections 2024-09-11T22:57:01.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:01.905+0000 7f407c4bd640 1 -- 172.21.15.152:0/325107484 wait complete. 2024-09-11T22:57:02.624 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:02 smithi152 ceph-mon[31181]: pgmap v127: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:57:02.748 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:57:02.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:02 smithi191 ceph-mon[35211]: pgmap v127: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:57:02.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:02 smithi204 ceph-mon[35454]: pgmap v127: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:57:02.998 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:57:03.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.528+0000 7fb74156d640 1 -- 172.21.15.152:0/4234420895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73c100a90 msgr2=0x7fb73c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:03.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.528+0000 7fb74156d640 1 --2- 172.21.15.152:0/4234420895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73c100a90 0x7fb73c100ef0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb724009930 tx=0x7fb72402a420 comp rx=0 tx=0).stop 2024-09-11T22:57:03.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.528+0000 7fb74156d640 1 -- 172.21.15.152:0/4234420895 shutdown_connections 2024-09-11T22:57:03.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.528+0000 7fb74156d640 1 --2- 172.21.15.152:0/4234420895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb73c101430 0x7fb73c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.528+0000 7fb74156d640 1 --2- 172.21.15.152:0/4234420895 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73c100a90 0x7fb73c100ef0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.528+0000 7fb74156d640 1 --2- 172.21.15.152:0/4234420895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb73c106a90 0x7fb73c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.528+0000 7fb74156d640 1 -- 172.21.15.152:0/4234420895 >> 172.21.15.152:0/4234420895 conn(0x7fb73c0fc7d0 msgr2=0x7fb73c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.528+0000 7fb74156d640 1 -- 172.21.15.152:0/4234420895 shutdown_connections 2024-09-11T22:57:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.528+0000 7fb74156d640 1 -- 172.21.15.152:0/4234420895 wait complete. 2024-09-11T22:57:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.529+0000 7fb74156d640 1 Processor -- start 2024-09-11T22:57:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.529+0000 7fb74156d640 1 -- start start 2024-09-11T22:57:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.529+0000 7fb74156d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb73c100a90 0x7fb73c197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.529+0000 7fb74156d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73c101430 0x7fb73c197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.529+0000 7fb74156d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb73c106a90 0x7fb73c19bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.529+0000 7fb74156d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c110ee0 con 0x7fb73c101430 2024-09-11T22:57:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.529+0000 7fb74156d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c110d60 con 0x7fb73c106a90 2024-09-11T22:57:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.529+0000 7fb74156d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c111060 con 0x7fb73c100a90 2024-09-11T22:57:03.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.529+0000 7fb733fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73c101430 0x7fb73c197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.529+0000 7fb733fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73c101430 0x7fb73c197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39852/0 (socket says 172.21.15.152:39852) 2024-09-11T22:57:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.529+0000 7fb733fff640 1 -- 172.21.15.152:0/733958651 learned_addr learned my addr 172.21.15.152:0/733958651 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.530+0000 7fb73b7fe640 1 --2- 172.21.15.152:0/733958651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb73c106a90 0x7fb73c19bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.530+0000 7fb73affd640 1 --2- 172.21.15.152:0/733958651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb73c100a90 0x7fb73c197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.530+0000 7fb733fff640 1 -- 172.21.15.152:0/733958651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb73c100a90 msgr2=0x7fb73c197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.530+0000 7fb733fff640 1 --2- 172.21.15.152:0/733958651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb73c100a90 0x7fb73c197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.530+0000 7fb733fff640 1 -- 172.21.15.152:0/733958651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb73c106a90 msgr2=0x7fb73c19bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.530+0000 7fb733fff640 1 --2- 172.21.15.152:0/733958651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb73c106a90 0x7fb73c19bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.530+0000 7fb733fff640 1 -- 172.21.15.152:0/733958651 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73c19c2f0 con 0x7fb73c101430 2024-09-11T22:57:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.530+0000 7fb73b7fe640 1 --2- 172.21.15.152:0/733958651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb73c106a90 0x7fb73c19bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.530+0000 7fb733fff640 1 --2- 172.21.15.152:0/733958651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73c101430 0x7fb73c197880 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb72402a940 tx=0x7fb724003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.530+0000 7fb738ff9640 1 -- 172.21.15.152:0/733958651 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72403f070 con 0x7fb73c101430 2024-09-11T22:57:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.530+0000 7fb74156d640 1 -- 172.21.15.152:0/733958651 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73c19c580 con 0x7fb73c101430 2024-09-11T22:57:03.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.530+0000 7fb738ff9640 1 -- 172.21.15.152:0/733958651 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb724031c90 con 0x7fb73c101430 2024-09-11T22:57:03.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.530+0000 7fb738ff9640 1 -- 172.21.15.152:0/733958651 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724035030 con 0x7fb73c101430 2024-09-11T22:57:03.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.530+0000 7fb74156d640 1 -- 172.21.15.152:0/733958651 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb73c1a3ec0 con 0x7fb73c101430 2024-09-11T22:57:03.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.532+0000 7fb738ff9640 1 -- 172.21.15.152:0/733958651 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb724042070 con 0x7fb73c101430 2024-09-11T22:57:03.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.533+0000 7fb743946740 1 Processor -- start 2024-09-11T22:57:03.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.533+0000 7fb743946740 1 -- start start 2024-09-11T22:57:03.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.533+0000 7fb743946740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f8573e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.533+0000 7fb743946740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8573e5d0 0x560f8573ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.533+0000 7fb743946740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8574b580 0x560f8574d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.533+0000 7fb743946740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560f856a42f0 con 0x560f8573fb40 2024-09-11T22:57:03.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.533+0000 7fb743946740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560f8564bcf0 con 0x560f8574b580 2024-09-11T22:57:03.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.533+0000 7fb743946740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560f856a02b0 con 0x560f8573e5d0 2024-09-11T22:57:03.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.533+0000 7fb73b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f8573e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.533+0000 7fb73b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f8573e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39854/0 (socket says 172.21.15.152:39854) 2024-09-11T22:57:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.533+0000 7fb738ff9640 1 --2- 172.21.15.152:0/733958651 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb7280a2cf0 0x7fb7280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.533+0000 7fb73b7fe640 1 -- 172.21.15.152:0/324904811 learned_addr learned my addr 172.21.15.152:0/324904811 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.534+0000 7fb73affd640 1 --2- 172.21.15.152:0/324904811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8573e5d0 0x560f8573ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.534+0000 7fb73b7fe640 1 --2- 172.21.15.152:0/733958651 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb7280a2cf0 0x7fb7280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.534+0000 7fb738ff9640 1 -- 172.21.15.152:0/733958651 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb7240efcb0 con 0x7fb73c101430 2024-09-11T22:57:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.534+0000 7fb733fff640 1 --2- 172.21.15.152:0/324904811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8574b580 0x560f8574d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.534+0000 7fb73b7fe640 1 -- 172.21.15.152:0/324904811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8573e5d0 msgr2=0x560f8573ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.534+0000 7fb73b7fe640 1 --2- 172.21.15.152:0/324904811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8573e5d0 0x560f8573ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.534+0000 7fb73b7fe640 1 -- 172.21.15.152:0/324904811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8574b580 msgr2=0x560f8574d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.534+0000 7fb73b7fe640 1 --2- 172.21.15.152:0/324904811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8574b580 0x560f8574d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.534+0000 7fb73b7fe640 1 -- 172.21.15.152:0/324904811 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f856303a0 con 0x560f8573fb40 2024-09-11T22:57:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.534+0000 7fb73b7fe640 1 --2- 172.21.15.152:0/324904811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f8573e090 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb734014a90 tx=0x7fb734014f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e08877e4bfa063 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.534+0000 7fb7317fa640 1 -- 172.21.15.152:0/324904811 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734014530 con 0x560f8573fb40 2024-09-11T22:57:03.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.534+0000 7fb7317fa640 1 -- 172.21.15.152:0/324904811 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb734023070 con 0x560f8573fb40 2024-09-11T22:57:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.534+0000 7fb7317fa640 1 -- 172.21.15.152:0/324904811 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73400ce40 con 0x560f8573fb40 2024-09-11T22:57:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.535+0000 7fb743946740 1 -- 172.21.15.152:0/324904811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 msgr2=0x560f8573e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.535+0000 7fb743946740 1 --2- 172.21.15.152:0/324904811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f8573e090 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fb734014a90 tx=0x7fb734014f50 comp rx=0 tx=0).stop 2024-09-11T22:57:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.535+0000 7fb73b7fe640 1 --2- 172.21.15.152:0/733958651 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb7280a2cf0 0x7fb7280a51b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb734022000 tx=0x7fb734020000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.535+0000 7fb743946740 1 -- 172.21.15.152:0/324904811 shutdown_connections 2024-09-11T22:57:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.535+0000 7fb743946740 1 --2- 172.21.15.152:0/324904811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8574b580 0x560f8574d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.535+0000 7fb743946740 1 --2- 172.21.15.152:0/324904811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8573e5d0 0x560f8573ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.535+0000 7fb743946740 1 --2- 172.21.15.152:0/324904811 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f8573e090 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.535+0000 7fb743946740 1 -- 172.21.15.152:0/324904811 >> 172.21.15.152:0/324904811 conn(0x560f85707310 msgr2=0x560f8573ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.535+0000 7fb743946740 1 -- 172.21.15.152:0/324904811 shutdown_connections 2024-09-11T22:57:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.535+0000 7fb743946740 1 -- 172.21.15.152:0/324904811 wait complete. 2024-09-11T22:57:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.535+0000 7fb743946740 1 -- start start 2024-09-11T22:57:03.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.535+0000 7fb743946740 1 Processor -- start 2024-09-11T22:57:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb743946740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8573e5d0 0x560f8574af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb743946740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8573fb40 0x560f85748230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb743946740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8574b580 0x560f85748820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb743946740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560f8574e280 con 0x560f8574b580 2024-09-11T22:57:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb743946740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560f8565f060 con 0x560f8573fb40 2024-09-11T22:57:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb743946740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560f8574e400 con 0x560f8573e5d0 2024-09-11T22:57:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb743946740 10 client.0 _ll_register_callbacks cb 0x560f853b28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb73b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8574b580 0x560f85748820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb73b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8574b580 0x560f85748820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39856/0 (socket says 172.21.15.152:39856) 2024-09-11T22:57:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb73b7fe640 1 -- 172.21.15.152:0/1905163177 learned_addr learned my addr 172.21.15.152:0/1905163177 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb73affd640 1 --2- 172.21.15.152:0/1905163177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8573e5d0 0x560f8574af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb733fff640 1 --2- 172.21.15.152:0/1905163177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8573fb40 0x560f85748230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb73b7fe640 1 -- 172.21.15.152:0/1905163177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8573e5d0 msgr2=0x560f8574af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb73b7fe640 1 --2- 172.21.15.152:0/1905163177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8573e5d0 0x560f8574af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb73b7fe640 1 -- 172.21.15.152:0/1905163177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8573fb40 msgr2=0x560f85748230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb73b7fe640 1 --2- 172.21.15.152:0/1905163177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8573fb40 0x560f85748230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb73b7fe640 1 -- 172.21.15.152:0/1905163177 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f854153c0 con 0x560f8574b580 2024-09-11T22:57:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb73b7fe640 1 --2- 172.21.15.152:0/1905163177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8574b580 0x560f85748820 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb73401ffd0 tx=0x7fb73400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25eef8b44058aa3d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb7227fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb7227fc640 1 -- 172.21.15.152:0/1905163177 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734037060 con 0x560f8574b580 2024-09-11T22:57:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb7227fc640 1 -- 172.21.15.152:0/1905163177 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb734012d70 con 0x560f8574b580 2024-09-11T22:57:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.536+0000 7fb743946740 1 -- 172.21.15.152:0/1905163177 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb734023070 con 0x560f8574b580 2024-09-11T22:57:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.537+0000 7fb7227fc640 1 -- 172.21.15.152:0/1905163177 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb73401ec50 con 0x560f8574b580 2024-09-11T22:57:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.537+0000 7fb743946740 10 client.16632 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.537+0000 7fb743946740 1 -- 172.21.15.152:0/1905163177 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f85759d20 con 0x560f8574b580 2024-09-11T22:57:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.537+0000 7fb7227fc640 1 -- 172.21.15.152:0/1905163177 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb734023070 con 0x560f8574b580 2024-09-11T22:57:03.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.537+0000 7fb7227fc640 1 -- 172.21.15.152:0/1905163177 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb73400f040 con 0x560f8574b580 2024-09-11T22:57:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.537+0000 7fb743946740 10 client.16632 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.537+0000 7fb743946740 10 client.16632 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:57:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.537+0000 7fb743946740 20 client.16632 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.537+0000 7fb743946740 1 --2- 172.21.15.152:0/1905163177 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560f8575b1e0 0x560f8575d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.537+0000 7fb743946740 4 client.16632 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:57:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.537+0000 7fb743946740 1 -- 172.21.15.152:0/1905163177 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f856b1860 con 0x560f8575b1e0 2024-09-11T22:57:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.537+0000 7fb73affd640 1 --2- 172.21.15.152:0/1905163177 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560f8575b1e0 0x560f8575d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.538+0000 7fb7227fc640 10 client.16632 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:57:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.538+0000 7fb73affd640 1 --2- 172.21.15.152:0/1905163177 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560f8575b1e0 0x560f8575d5d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=100f19a3de7ca788 server_cookie=7c8e1585b61792be in_seq=0 out_seq=0 2024-09-11T22:57:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.538+0000 7fb7227fc640 1 -- 172.21.15.152:0/1905163177 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f856b1860 con 0x560f8575b1e0 2024-09-11T22:57:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.538+0000 7fb7227fc640 10 client.16632 handle_command_reply: tid=0 2024-09-11T22:57:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.539+0000 7fb743946740 1 client.16632 shutdown 2024-09-11T22:57:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.539+0000 7fb743946740 1 -- 172.21.15.152:0/1905163177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8574b580 msgr2=0x560f85748820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.539+0000 7fb743946740 1 --2- 172.21.15.152:0/1905163177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8574b580 0x560f85748820 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb73401ffd0 tx=0x7fb73400cb00 comp rx=0 tx=0).stop 2024-09-11T22:57:03.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.539+0000 7fb743946740 1 -- 172.21.15.152:0/1905163177 shutdown_connections 2024-09-11T22:57:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.539+0000 7fb743946740 1 --2- 172.21.15.152:0/1905163177 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560f8575b1e0 0x560f8575d5d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.539+0000 7fb743946740 1 --2- 172.21.15.152:0/1905163177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8574b580 0x560f85748820 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.539+0000 7fb743946740 1 --2- 172.21.15.152:0/1905163177 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8573fb40 0x560f85748230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.539+0000 7fb7227fc640 0 client.16632 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:57:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.539+0000 7fb743946740 1 --2- 172.21.15.152:0/1905163177 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8573e5d0 0x560f8574af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.539+0000 7fb743946740 1 -- 172.21.15.152:0/1905163177 >> 172.21.15.152:0/1905163177 conn(0x560f85707310 msgr2=0x560f85740960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.539+0000 7fb743946740 1 -- 172.21.15.152:0/1905163177 shutdown_connections 2024-09-11T22:57:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.539+0000 7fb743946740 1 -- 172.21.15.152:0/1905163177 wait complete. 2024-09-11T22:57:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.539+0000 7fb743946740 20 client.16632 trim_cache size 0 max 16384 2024-09-11T22:57:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb743946740 1 Processor -- start 2024-09-11T22:57:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb743946740 1 -- start start 2024-09-11T22:57:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb743946740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8573e5d0 0x560f85757520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb743946740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f85747210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb743946740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8574b580 0x560f85762c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb743946740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c1ab600 con 0x560f8573fb40 2024-09-11T22:57:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb743946740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c1ad140 con 0x560f8573e5d0 2024-09-11T22:57:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb743946740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c1abcd0 con 0x560f8574b580 2024-09-11T22:57:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb733fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f85747210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb733fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f85747210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39862/0 (socket says 172.21.15.152:39862) 2024-09-11T22:57:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb733fff640 1 -- 172.21.15.152:0/1315813934 learned_addr learned my addr 172.21.15.152:0/1315813934 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb73affd640 1 --2- 172.21.15.152:0/1315813934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8573e5d0 0x560f85757520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb73b7fe640 1 --2- 172.21.15.152:0/1315813934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8574b580 0x560f85762c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb733fff640 1 -- 172.21.15.152:0/1315813934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8574b580 msgr2=0x560f85762c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb733fff640 1 --2- 172.21.15.152:0/1315813934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8574b580 0x560f85762c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb733fff640 1 -- 172.21.15.152:0/1315813934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8573e5d0 msgr2=0x560f85757520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:03.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb733fff640 1 --2- 172.21.15.152:0/1315813934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8573e5d0 0x560f85757520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.552+0000 7fb733fff640 1 -- 172.21.15.152:0/1315813934 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f85746780 con 0x560f8573fb40 2024-09-11T22:57:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.553+0000 7fb733fff640 1 --2- 172.21.15.152:0/1315813934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f85747210 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb72402a9e0 tx=0x7fb72403b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aeedf1205351bab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.553+0000 7fb730ff9640 1 -- 172.21.15.152:0/1315813934 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724049040 con 0x560f8573fb40 2024-09-11T22:57:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.553+0000 7fb730ff9640 1 -- 172.21.15.152:0/1315813934 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7240c4e50 con 0x560f8573fb40 2024-09-11T22:57:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.553+0000 7fb730ff9640 1 -- 172.21.15.152:0/1315813934 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7240ed690 con 0x560f8573fb40 2024-09-11T22:57:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.553+0000 7fb743946740 1 -- 172.21.15.152:0/1315813934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 msgr2=0x560f85747210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.553+0000 7fb743946740 1 --2- 172.21.15.152:0/1315813934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f85747210 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb72402a9e0 tx=0x7fb72403b990 comp rx=0 tx=0).stop 2024-09-11T22:57:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb743946740 1 -- 172.21.15.152:0/1315813934 shutdown_connections 2024-09-11T22:57:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb743946740 1 --2- 172.21.15.152:0/1315813934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8574b580 0x560f85762c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb743946740 1 --2- 172.21.15.152:0/1315813934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f85747210 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb743946740 1 --2- 172.21.15.152:0/1315813934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8573e5d0 0x560f85757520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb743946740 1 -- 172.21.15.152:0/1315813934 >> 172.21.15.152:0/1315813934 conn(0x560f85707310 msgr2=0x560f85746b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb743946740 1 -- 172.21.15.152:0/1315813934 shutdown_connections 2024-09-11T22:57:03.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb743946740 1 -- 172.21.15.152:0/1315813934 wait complete. 2024-09-11T22:57:03.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb743946740 1 -- start start 2024-09-11T22:57:03.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb743946740 1 Processor -- start 2024-09-11T22:57:03.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb743946740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8573e5d0 0x560f85753cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb743946740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f8574f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb743946740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8574b580 0x560f8574f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb743946740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c1aca70 con 0x560f8573fb40 2024-09-11T22:57:03.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb743946740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c1ac3a0 con 0x560f8574b580 2024-09-11T22:57:03.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb743946740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c1a95c0 con 0x560f8573e5d0 2024-09-11T22:57:03.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb743946740 10 client.0 _ll_register_callbacks cb 0x7fb734014860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:03.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb733fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f8574f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb733fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f8574f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39874/0 (socket says 172.21.15.152:39874) 2024-09-11T22:57:03.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb733fff640 1 -- 172.21.15.152:0/2131873518 learned_addr learned my addr 172.21.15.152:0/2131873518 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:03.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb73affd640 1 --2- 172.21.15.152:0/2131873518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8573e5d0 0x560f85753cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.554+0000 7fb73b7fe640 1 --2- 172.21.15.152:0/2131873518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8574b580 0x560f8574f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb733fff640 1 -- 172.21.15.152:0/2131873518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8573e5d0 msgr2=0x560f85753cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb733fff640 1 --2- 172.21.15.152:0/2131873518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8573e5d0 0x560f85753cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb733fff640 1 -- 172.21.15.152:0/2131873518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8574b580 msgr2=0x560f8574f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb733fff640 1 --2- 172.21.15.152:0/2131873518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8574b580 0x560f8574f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb733fff640 1 -- 172.21.15.152:0/2131873518 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f8574fc30 con 0x560f8573fb40 2024-09-11T22:57:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb73b7fe640 1 --2- 172.21.15.152:0/2131873518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8574b580 0x560f8574f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb73affd640 1 --2- 172.21.15.152:0/2131873518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8573e5d0 0x560f85753cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb733fff640 1 --2- 172.21.15.152:0/2131873518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f8574f000 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb7240b8640 tx=0x7fb724005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0424950dee9546 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb7227fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb7227fc640 1 -- 172.21.15.152:0/2131873518 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7240d4070 con 0x560f8573fb40 2024-09-11T22:57:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb7227fc640 1 -- 172.21.15.152:0/2131873518 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb7240ed690 con 0x560f8573fb40 2024-09-11T22:57:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb743946740 1 -- 172.21.15.152:0/2131873518 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7240c4e50 con 0x560f8573fb40 2024-09-11T22:57:03.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb7227fc640 1 -- 172.21.15.152:0/2131873518 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb724085070 con 0x560f8573fb40 2024-09-11T22:57:03.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb743946740 10 client.16638 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:03.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb743946740 1 -- 172.21.15.152:0/2131873518 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f857c5ab0 con 0x560f8573fb40 2024-09-11T22:57:03.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb7227fc640 1 -- 172.21.15.152:0/2131873518 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7240c4e50 con 0x560f8573fb40 2024-09-11T22:57:03.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb7227fc640 1 -- 172.21.15.152:0/2131873518 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb7240c0200 con 0x560f8573fb40 2024-09-11T22:57:03.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb743946740 10 client.16638 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:03.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb743946740 10 client.16638 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:57:03.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb743946740 20 client.16638 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:03.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb743946740 1 --2- 172.21.15.152:0/2131873518 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560f857c60d0 0x560f857c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:03.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb743946740 4 client.16638 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:57:03.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.555+0000 7fb743946740 1 -- 172.21.15.152:0/2131873518 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f856abd70 con 0x560f857c60d0 2024-09-11T22:57:03.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.556+0000 7fb73affd640 1 --2- 172.21.15.152:0/2131873518 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560f857c60d0 0x560f857c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:03.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.556+0000 7fb73affd640 1 --2- 172.21.15.152:0/2131873518 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560f857c60d0 0x560f857c84c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4301474585d875eb server_cookie=3d98c22ef986f111 in_seq=0 out_seq=0 2024-09-11T22:57:03.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.556+0000 7fb7227fc640 10 client.16638 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:57:03.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.556+0000 7fb7227fc640 1 -- 172.21.15.152:0/2131873518 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f856abd70 con 0x560f857c60d0 2024-09-11T22:57:03.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.556+0000 7fb7227fc640 10 client.16638 handle_command_reply: tid=0 2024-09-11T22:57:03.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.556+0000 7fb743946740 1 client.16638 shutdown 2024-09-11T22:57:03.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.556+0000 7fb743946740 1 -- 172.21.15.152:0/2131873518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 msgr2=0x560f8574f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:03.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.556+0000 7fb743946740 1 --2- 172.21.15.152:0/2131873518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f8574f000 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb7240b8640 tx=0x7fb724005c00 comp rx=0 tx=0).stop 2024-09-11T22:57:03.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.556+0000 7fb743946740 1 -- 172.21.15.152:0/2131873518 shutdown_connections 2024-09-11T22:57:03.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.556+0000 7fb743946740 1 --2- 172.21.15.152:0/2131873518 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560f857c60d0 0x560f857c84c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.556+0000 7fb743946740 1 --2- 172.21.15.152:0/2131873518 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560f8574b580 0x560f8574f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.556+0000 7fb743946740 1 --2- 172.21.15.152:0/2131873518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560f8573fb40 0x560f8574f000 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.556+0000 7fb7227fc640 0 client.16638 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:57:03.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.556+0000 7fb743946740 1 --2- 172.21.15.152:0/2131873518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560f8573e5d0 0x560f85753cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.556+0000 7fb743946740 1 -- 172.21.15.152:0/2131873518 >> 172.21.15.152:0/2131873518 conn(0x560f85707310 msgr2=0x560f85747320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:03.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.557+0000 7fb743946740 1 -- 172.21.15.152:0/2131873518 shutdown_connections 2024-09-11T22:57:03.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.557+0000 7fb743946740 1 -- 172.21.15.152:0/2131873518 wait complete. 2024-09-11T22:57:03.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.557+0000 7fb743946740 20 client.16638 trim_cache size 0 max 16384 2024-09-11T22:57:03.599 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:57:03.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.557+0000 7fb74156d640 1 -- 172.21.15.152:0/733958651 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb7280a2cf0 msgr2=0x7fb7280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.557+0000 7fb74156d640 1 --2- 172.21.15.152:0/733958651 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb7280a2cf0 0x7fb7280a51b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb734022000 tx=0x7fb734020000 comp rx=0 tx=0).stop 2024-09-11T22:57:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.557+0000 7fb74156d640 1 -- 172.21.15.152:0/733958651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73c101430 msgr2=0x7fb73c197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.557+0000 7fb74156d640 1 --2- 172.21.15.152:0/733958651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73c101430 0x7fb73c197880 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fb72402a940 tx=0x7fb724003b90 comp rx=0 tx=0).stop 2024-09-11T22:57:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.557+0000 7fb74156d640 1 -- 172.21.15.152:0/733958651 shutdown_connections 2024-09-11T22:57:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.557+0000 7fb74156d640 1 --2- 172.21.15.152:0/733958651 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb7280a2cf0 0x7fb7280a51b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.557+0000 7fb74156d640 1 --2- 172.21.15.152:0/733958651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb73c106a90 0x7fb73c19bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.557+0000 7fb74156d640 1 --2- 172.21.15.152:0/733958651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb73c101430 0x7fb73c197880 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.557+0000 7fb74156d640 1 --2- 172.21.15.152:0/733958651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb73c100a90 0x7fb73c197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.557+0000 7fb74156d640 1 -- 172.21.15.152:0/733958651 >> 172.21.15.152:0/733958651 conn(0x7fb73c0fc7d0 msgr2=0x7fb73c0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.557+0000 7fb74156d640 1 -- 172.21.15.152:0/733958651 shutdown_connections 2024-09-11T22:57:03.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:03.557+0000 7fb74156d640 1 -- 172.21.15.152:0/733958651 wait complete. 2024-09-11T22:57:04.483 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:57:04.749 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:57:04.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:04 smithi191 ceph-mon[35211]: pgmap v128: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:04.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:04 smithi204 ceph-mon[35454]: pgmap v128: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:04.868 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:04 smithi152 ceph-mon[31181]: pgmap v128: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:05.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.267+0000 7f2422a08640 1 -- 172.21.15.152:0/260155426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f241c074620 msgr2=0x7f241c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:05.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.268+0000 7f2422a08640 1 --2- 172.21.15.152:0/260155426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f241c074620 0x7f241c074a00 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f240c009900 tx=0x7f240c02a480 comp rx=0 tx=0).stop 2024-09-11T22:57:05.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.268+0000 7f2422a08640 1 -- 172.21.15.152:0/260155426 shutdown_connections 2024-09-11T22:57:05.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.268+0000 7f2422a08640 1 --2- 172.21.15.152:0/260155426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f241c10e3d0 0x7f241c110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.268+0000 7f2422a08640 1 --2- 172.21.15.152:0/260155426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f241c074f40 0x7f241c10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.268+0000 7f2422a08640 1 --2- 172.21.15.152:0/260155426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f241c074620 0x7f241c074a00 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.268+0000 7f2422a08640 1 -- 172.21.15.152:0/260155426 >> 172.21.15.152:0/260155426 conn(0x7f241c0fd8f0 msgr2=0x7f241c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:05.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.268+0000 7f2422a08640 1 -- 172.21.15.152:0/260155426 shutdown_connections 2024-09-11T22:57:05.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.268+0000 7f2422a08640 1 -- 172.21.15.152:0/260155426 wait complete. 2024-09-11T22:57:05.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.268+0000 7f2422a08640 1 Processor -- start 2024-09-11T22:57:05.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.268+0000 7f2422a08640 1 -- start start 2024-09-11T22:57:05.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.268+0000 7f2422a08640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f241c074620 0x7f241c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.268+0000 7f2422a08640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f241c074f40 0x7f241c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.268+0000 7f2422a08640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f241c10e3d0 0x7f241c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.268+0000 7f2422a08640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f241c114180 con 0x7f241c10e3d0 2024-09-11T22:57:05.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.268+0000 7f2422a08640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f241c114000 con 0x7f241c074f40 2024-09-11T22:57:05.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.268+0000 7f2422a08640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f241c114300 con 0x7f241c074620 2024-09-11T22:57:05.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f2420fa5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f241c10e3d0 0x7f241c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:05.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f2420fa5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f241c10e3d0 0x7f241c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39878/0 (socket says 172.21.15.152:39878) 2024-09-11T22:57:05.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f2420fa5640 1 -- 172.21.15.152:0/3614361499 learned_addr learned my addr 172.21.15.152:0/3614361499 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:05.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f241bfff640 1 --2- 172.21.15.152:0/3614361499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f241c074620 0x7f241c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:05.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f2413fff640 1 --2- 172.21.15.152:0/3614361499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f241c074f40 0x7f241c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:05.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f2420fa5640 1 -- 172.21.15.152:0/3614361499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f241c074620 msgr2=0x7f241c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:05.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f2420fa5640 1 --2- 172.21.15.152:0/3614361499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f241c074620 0x7f241c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f2420fa5640 1 -- 172.21.15.152:0/3614361499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f241c074f40 msgr2=0x7f241c19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:05.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f2420fa5640 1 --2- 172.21.15.152:0/3614361499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f241c074f40 0x7f241c19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f2420fa5640 1 -- 172.21.15.152:0/3614361499 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c19f610 con 0x7f241c10e3d0 2024-09-11T22:57:05.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f241bfff640 1 --2- 172.21.15.152:0/3614361499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f241c074620 0x7f241c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:05.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f2420fa5640 1 --2- 172.21.15.152:0/3614361499 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f241c10e3d0 0x7f241c19ef30 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f241400eaa0 tx=0x7f241400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:05.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f2419ffb640 1 -- 172.21.15.152:0/3614361499 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241400cdb0 con 0x7f241c10e3d0 2024-09-11T22:57:05.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f2419ffb640 1 -- 172.21.15.152:0/3614361499 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2414005020 con 0x7f241c10e3d0 2024-09-11T22:57:05.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f2422a08640 1 -- 172.21.15.152:0/3614361499 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241c19f900 con 0x7f241c10e3d0 2024-09-11T22:57:05.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f2419ffb640 1 -- 172.21.15.152:0/3614361499 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140116b0 con 0x7f241c10e3d0 2024-09-11T22:57:05.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.269+0000 7f2422a08640 1 -- 172.21.15.152:0/3614361499 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241c1a71e0 con 0x7f241c10e3d0 2024-09-11T22:57:05.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.271+0000 7f2419ffb640 1 -- 172.21.15.152:0/3614361499 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2414005560 con 0x7f241c10e3d0 2024-09-11T22:57:05.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.272+0000 7f2424de1740 1 Processor -- start 2024-09-11T22:57:05.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.272+0000 7f2424de1740 1 -- start start 2024-09-11T22:57:05.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.272+0000 7f2424de1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a674fb40 0x5641a674e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.272+0000 7f2424de1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a674ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.272+0000 7f2424de1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a6759370 0x5641a675b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.272+0000 7f2424de1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5641a66b42f0 con 0x5641a674e5d0 2024-09-11T22:57:05.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.272+0000 7f2424de1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5641a665bcf0 con 0x5641a6759370 2024-09-11T22:57:05.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.272+0000 7f2424de1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5641a66b02b0 con 0x5641a674fb40 2024-09-11T22:57:05.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.272+0000 7f2419ffb640 1 --2- 172.21.15.152:0/3614361499 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f24080a2d70 0x7f24080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.272+0000 7f241bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a674ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:05.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.272+0000 7f241bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a674ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39882/0 (socket says 172.21.15.152:39882) 2024-09-11T22:57:05.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.272+0000 7f241bfff640 1 -- 172.21.15.152:0/1740567836 learned_addr learned my addr 172.21.15.152:0/1740567836 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:05.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.272+0000 7f2413fff640 1 --2- 172.21.15.152:0/1740567836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a674fb40 0x5641a674e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:05.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.272+0000 7f2420fa5640 1 --2- 172.21.15.152:0/3614361499 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f24080a2d70 0x7f24080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:05.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.272+0000 7f2413fff640 1 --2- 172.21.15.152:0/1740567836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a6759370 0x5641a675b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:05.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.272+0000 7f2419ffb640 1 -- 172.21.15.152:0/3614361499 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f24140cf630 con 0x7f241c10e3d0 2024-09-11T22:57:05.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.273+0000 7f241bfff640 1 -- 172.21.15.152:0/1740567836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a674fb40 msgr2=0x5641a674e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:05.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.273+0000 7f241bfff640 1 --2- 172.21.15.152:0/1740567836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a674fb40 0x5641a674e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.273+0000 7f241bfff640 1 -- 172.21.15.152:0/1740567836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a6759370 msgr2=0x5641a675b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:05.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.273+0000 7f241bfff640 1 --2- 172.21.15.152:0/1740567836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a6759370 0x5641a675b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.273+0000 7f241bfff640 1 -- 172.21.15.152:0/1740567836 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a66403a0 con 0x5641a674e5d0 2024-09-11T22:57:05.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.273+0000 7f2413fff640 1 --2- 172.21.15.152:0/1740567836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a6759370 0x5641a675b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:05.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.273+0000 7f2420fa5640 1 --2- 172.21.15.152:0/3614361499 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f24080a2d70 0x7f24080a5230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f24140a42a0 tx=0x7f241406db20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:05.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.273+0000 7f241bfff640 1 --2- 172.21.15.152:0/1740567836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a674ea10 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f240c033000 tx=0x7f240c032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d9060a9cd46fdb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:05.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.273+0000 7f24127fc640 1 -- 172.21.15.152:0/1740567836 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c03f070 con 0x5641a674e5d0 2024-09-11T22:57:05.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.273+0000 7f24127fc640 1 -- 172.21.15.152:0/1740567836 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f240c0317b0 con 0x5641a674e5d0 2024-09-11T22:57:05.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.273+0000 7f24127fc640 1 -- 172.21.15.152:0/1740567836 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c035070 con 0x5641a674e5d0 2024-09-11T22:57:05.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.274+0000 7f2424de1740 1 -- 172.21.15.152:0/1740567836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 msgr2=0x5641a674ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:05.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.274+0000 7f2424de1740 1 --2- 172.21.15.152:0/1740567836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a674ea10 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f240c033000 tx=0x7f240c032d60 comp rx=0 tx=0).stop 2024-09-11T22:57:05.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.274+0000 7f2424de1740 1 -- 172.21.15.152:0/1740567836 shutdown_connections 2024-09-11T22:57:05.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.274+0000 7f2424de1740 1 --2- 172.21.15.152:0/1740567836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a6759370 0x5641a675b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.274+0000 7f2424de1740 1 --2- 172.21.15.152:0/1740567836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a674ea10 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.274+0000 7f2424de1740 1 --2- 172.21.15.152:0/1740567836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a674fb40 0x5641a674e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.274+0000 7f2424de1740 1 -- 172.21.15.152:0/1740567836 >> 172.21.15.152:0/1740567836 conn(0x5641a6717310 msgr2=0x5641a674ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:05.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.274+0000 7f2424de1740 1 -- 172.21.15.152:0/1740567836 shutdown_connections 2024-09-11T22:57:05.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.274+0000 7f2424de1740 1 -- 172.21.15.152:0/1740567836 wait complete. 2024-09-11T22:57:05.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.274+0000 7f2424de1740 1 -- start start 2024-09-11T22:57:05.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.274+0000 7f2424de1740 1 Processor -- start 2024-09-11T22:57:05.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2424de1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a674e5d0 0x5641a6758d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2424de1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a674fb40 0x5641a6756030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2424de1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a6759370 0x5641a6756640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2424de1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5641a675c010 con 0x5641a6759370 2024-09-11T22:57:05.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2424de1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5641a666f060 con 0x5641a674e5d0 2024-09-11T22:57:05.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2424de1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5641a675c190 con 0x5641a674fb40 2024-09-11T22:57:05.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2424de1740 10 client.0 _ll_register_callbacks cb 0x5641a63c28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:05.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2413fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a6759370 0x5641a6756640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:05.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f241bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a674fb40 0x5641a6756030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:05.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2413fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a6759370 0x5641a6756640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39898/0 (socket says 172.21.15.152:39898) 2024-09-11T22:57:05.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2413fff640 1 -- 172.21.15.152:0/2697831712 learned_addr learned my addr 172.21.15.152:0/2697831712 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:05.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2413fff640 1 --2- 172.21.15.152:0/2697831712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a674e5d0 0x5641a6758d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:05.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2413fff640 1 -- 172.21.15.152:0/2697831712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a674fb40 msgr2=0x5641a6756030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:05.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2413fff640 1 --2- 172.21.15.152:0/2697831712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a674fb40 0x5641a6756030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2413fff640 1 -- 172.21.15.152:0/2697831712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a674e5d0 msgr2=0x5641a6758d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:05.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2413fff640 1 --2- 172.21.15.152:0/2697831712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a674e5d0 0x5641a6758d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2413fff640 1 -- 172.21.15.152:0/2697831712 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a64253c0 con 0x5641a6759370 2024-09-11T22:57:05.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2413fff640 1 --2- 172.21.15.152:0/2697831712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a6759370 0x5641a6756640 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f240400d340 tx=0x7f240400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d73ca0423e5d4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:05.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f23fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:05.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f23fb7fe640 1 -- 172.21.15.152:0/2697831712 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240400de00 con 0x5641a6759370 2024-09-11T22:57:05.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f23fb7fe640 1 -- 172.21.15.152:0/2697831712 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f2404010640 con 0x5641a6759370 2024-09-11T22:57:05.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.275+0000 7f2424de1740 1 -- 172.21.15.152:0/2697831712 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f240c0317b0 con 0x5641a6759370 2024-09-11T22:57:05.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.276+0000 7f23fb7fe640 1 -- 172.21.15.152:0/2697831712 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2404023020 con 0x5641a6759370 2024-09-11T22:57:05.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.276+0000 7f2424de1740 10 client.16650 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:05.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.276+0000 7f2424de1740 1 -- 172.21.15.152:0/2697831712 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641a6767b40 con 0x5641a6759370 2024-09-11T22:57:05.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.276+0000 7f23fb7fe640 1 -- 172.21.15.152:0/2697831712 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f240c0317b0 con 0x5641a6759370 2024-09-11T22:57:05.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.276+0000 7f23fb7fe640 1 -- 172.21.15.152:0/2697831712 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2404028850 con 0x5641a6759370 2024-09-11T22:57:05.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.276+0000 7f2424de1740 10 client.16650 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:05.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.276+0000 7f2424de1740 10 client.16650 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:57:05.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.276+0000 7f2424de1740 20 client.16650 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:05.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.276+0000 7f2424de1740 1 --2- 172.21.15.152:0/2697831712 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5641a6768f50 0x5641a676b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.276+0000 7f2424de1740 4 client.16650 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:57:05.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.276+0000 7f2424de1740 1 -- 172.21.15.152:0/2697831712 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641a66c1860 con 0x5641a6768f50 2024-09-11T22:57:05.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.276+0000 7f241bfff640 1 --2- 172.21.15.152:0/2697831712 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5641a6768f50 0x5641a676b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:05.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.277+0000 7f241bfff640 1 --2- 172.21.15.152:0/2697831712 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5641a6768f50 0x5641a676b360 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af1bec5de80c41d server_cookie=be396aebde527c88 in_seq=0 out_seq=0 2024-09-11T22:57:05.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.277+0000 7f23fb7fe640 10 client.16650 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:57:05.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.277+0000 7f23fb7fe640 1 -- 172.21.15.152:0/2697831712 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641a66c1860 con 0x5641a6768f50 2024-09-11T22:57:05.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.277+0000 7f23fb7fe640 10 client.16650 handle_command_reply: tid=0 2024-09-11T22:57:05.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.277+0000 7f2424de1740 1 client.16650 shutdown 2024-09-11T22:57:05.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.277+0000 7f2424de1740 1 -- 172.21.15.152:0/2697831712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a6759370 msgr2=0x5641a6756640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:05.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.277+0000 7f2424de1740 1 --2- 172.21.15.152:0/2697831712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a6759370 0x5641a6756640 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f240400d340 tx=0x7f240400ec90 comp rx=0 tx=0).stop 2024-09-11T22:57:05.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.277+0000 7f2424de1740 1 -- 172.21.15.152:0/2697831712 shutdown_connections 2024-09-11T22:57:05.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.277+0000 7f2424de1740 1 --2- 172.21.15.152:0/2697831712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a6759370 0x5641a6756640 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.277+0000 7f2424de1740 1 --2- 172.21.15.152:0/2697831712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a674fb40 0x5641a6756030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.277+0000 7f2424de1740 1 --2- 172.21.15.152:0/2697831712 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5641a6768f50 0x5641a676b360 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.277+0000 7f2424de1740 1 --2- 172.21.15.152:0/2697831712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a674e5d0 0x5641a6758d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.277+0000 7f2424de1740 1 -- 172.21.15.152:0/2697831712 >> 172.21.15.152:0/2697831712 conn(0x5641a6717310 msgr2=0x5641a6750960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:05.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.277+0000 7f23fb7fe640 0 client.16650 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:57:05.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.277+0000 7f2424de1740 1 -- 172.21.15.152:0/2697831712 shutdown_connections 2024-09-11T22:57:05.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.277+0000 7f2424de1740 1 -- 172.21.15.152:0/2697831712 wait complete. 2024-09-11T22:57:05.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.278+0000 7f2424de1740 20 client.16650 trim_cache size 0 max 16384 2024-09-11T22:57:05.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.290+0000 7f2424de1740 1 Processor -- start 2024-09-11T22:57:05.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.290+0000 7f2424de1740 1 -- start start 2024-09-11T22:57:05.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.290+0000 7f2424de1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a6757fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.290+0000 7f2424de1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a674fb40 0x5641a6751120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.290+0000 7f2424de1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a6759370 0x5641a6751660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.290+0000 7f2424de1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1af930 con 0x5641a674e5d0 2024-09-11T22:57:05.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.290+0000 7f2424de1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1b1470 con 0x5641a6759370 2024-09-11T22:57:05.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.290+0000 7f2424de1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1b0000 con 0x5641a674fb40 2024-09-11T22:57:05.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.290+0000 7f2413fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a6757fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:05.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.290+0000 7f2413fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a6757fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39906/0 (socket says 172.21.15.152:39906) 2024-09-11T22:57:05.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.291+0000 7f2413fff640 1 -- 172.21.15.152:0/993270130 learned_addr learned my addr 172.21.15.152:0/993270130 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:05.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.291+0000 7f2413fff640 1 -- 172.21.15.152:0/993270130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a674fb40 msgr2=0x5641a6751120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:05.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.291+0000 7f2413fff640 1 --2- 172.21.15.152:0/993270130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a674fb40 0x5641a6751120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.291+0000 7f2413fff640 1 -- 172.21.15.152:0/993270130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a6759370 msgr2=0x5641a6751660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:05.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.291+0000 7f2413fff640 1 --2- 172.21.15.152:0/993270130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a6759370 0x5641a6751660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.291+0000 7f2413fff640 1 -- 172.21.15.152:0/993270130 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a674a4f0 con 0x5641a674e5d0 2024-09-11T22:57:05.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.291+0000 7f2413fff640 1 --2- 172.21.15.152:0/993270130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a6757fa0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2404011c80 tx=0x7f2404011840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124398005e528608 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:05.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.291+0000 7f2411ffb640 1 -- 172.21.15.152:0/993270130 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240401f410 con 0x5641a674e5d0 2024-09-11T22:57:05.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.291+0000 7f2411ffb640 1 -- 172.21.15.152:0/993270130 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f240400de00 con 0x5641a674e5d0 2024-09-11T22:57:05.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.291+0000 7f2411ffb640 1 -- 172.21.15.152:0/993270130 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2404005500 con 0x5641a674e5d0 2024-09-11T22:57:05.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.292+0000 7f2424de1740 1 -- 172.21.15.152:0/993270130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 msgr2=0x5641a6757fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:05.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.292+0000 7f2424de1740 1 --2- 172.21.15.152:0/993270130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a6757fa0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2404011c80 tx=0x7f2404011840 comp rx=0 tx=0).stop 2024-09-11T22:57:05.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.292+0000 7f2424de1740 1 -- 172.21.15.152:0/993270130 shutdown_connections 2024-09-11T22:57:05.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.292+0000 7f2424de1740 1 --2- 172.21.15.152:0/993270130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a6759370 0x5641a6751660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.292+0000 7f2424de1740 1 --2- 172.21.15.152:0/993270130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a674fb40 0x5641a6751120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.292+0000 7f2424de1740 1 --2- 172.21.15.152:0/993270130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a6757fa0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.292+0000 7f2424de1740 1 -- 172.21.15.152:0/993270130 >> 172.21.15.152:0/993270130 conn(0x5641a6717310 msgr2=0x5641a6757890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:05.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.292+0000 7f2424de1740 1 -- 172.21.15.152:0/993270130 shutdown_connections 2024-09-11T22:57:05.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.292+0000 7f2424de1740 1 -- 172.21.15.152:0/993270130 wait complete. 2024-09-11T22:57:05.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.292+0000 7f2424de1740 1 -- start start 2024-09-11T22:57:05.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.292+0000 7f2424de1740 1 Processor -- start 2024-09-11T22:57:05.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.292+0000 7f2424de1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a6761600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.292+0000 7f2424de1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a674fb40 0x5641a6761b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.292+0000 7f2424de1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a6759370 0x5641a6752680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2424de1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1b0da0 con 0x5641a674e5d0 2024-09-11T22:57:05.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2424de1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1b06d0 con 0x5641a674fb40 2024-09-11T22:57:05.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2424de1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1ad8f0 con 0x5641a6759370 2024-09-11T22:57:05.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2424de1740 10 client.0 _ll_register_callbacks cb 0x7f240400ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:05.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2413fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a6761600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:05.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2413fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a6761600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39918/0 (socket says 172.21.15.152:39918) 2024-09-11T22:57:05.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2413fff640 1 -- 172.21.15.152:0/2301502482 learned_addr learned my addr 172.21.15.152:0/2301502482 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:05.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f241bfff640 1 --2- 172.21.15.152:0/2301502482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a674fb40 0x5641a6761b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:05.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2413fff640 1 --2- 172.21.15.152:0/2301502482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a6759370 0x5641a6752680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:05.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2413fff640 1 -- 172.21.15.152:0/2301502482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a6759370 msgr2=0x5641a6752680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:05.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2413fff640 1 --2- 172.21.15.152:0/2301502482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a6759370 0x5641a6752680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2413fff640 1 -- 172.21.15.152:0/2301502482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a674fb40 msgr2=0x5641a6761b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:05.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2413fff640 1 --2- 172.21.15.152:0/2301502482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a674fb40 0x5641a6761b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2413fff640 1 -- 172.21.15.152:0/2301502482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a6752cc0 con 0x5641a674e5d0 2024-09-11T22:57:05.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2413fff640 1 --2- 172.21.15.152:0/2301502482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a6761600 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2404011d00 tx=0x7f24040050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab3f274cd185f0d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:05.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f23fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:05.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f23fb7fe640 1 -- 172.21.15.152:0/2301502482 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240403b020 con 0x5641a674e5d0 2024-09-11T22:57:05.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f23fb7fe640 1 -- 172.21.15.152:0/2301502482 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f2404028840 con 0x5641a674e5d0 2024-09-11T22:57:05.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2424de1740 1 -- 172.21.15.152:0/2301502482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f240400de00 con 0x5641a674e5d0 2024-09-11T22:57:05.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f23fb7fe640 1 -- 172.21.15.152:0/2301502482 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2404028a40 con 0x5641a674e5d0 2024-09-11T22:57:05.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2424de1740 10 client.16656 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:05.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.293+0000 7f2424de1740 1 -- 172.21.15.152:0/2301502482 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5641a67d3500 con 0x5641a674e5d0 2024-09-11T22:57:05.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.294+0000 7f23fb7fe640 1 -- 172.21.15.152:0/2301502482 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f240400de00 con 0x5641a674e5d0 2024-09-11T22:57:05.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.294+0000 7f23fb7fe640 1 -- 172.21.15.152:0/2301502482 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2404010640 con 0x5641a674e5d0 2024-09-11T22:57:05.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.294+0000 7f2424de1740 10 client.16656 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:05.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.294+0000 7f2424de1740 10 client.16656 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:57:05.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.294+0000 7f2424de1740 20 client.16656 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:05.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.294+0000 7f2424de1740 1 --2- 172.21.15.152:0/2301502482 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5641a67d3a10 0x5641a67d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:05.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.294+0000 7f2424de1740 4 client.16656 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:57:05.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.294+0000 7f2424de1740 1 -- 172.21.15.152:0/2301502482 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641a66bbd70 con 0x5641a67d3a10 2024-09-11T22:57:05.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.294+0000 7f241bfff640 1 --2- 172.21.15.152:0/2301502482 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5641a67d3a10 0x5641a67d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:05.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.294+0000 7f23fb7fe640 10 client.16656 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:57:05.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.294+0000 7f241bfff640 1 --2- 172.21.15.152:0/2301502482 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5641a67d3a10 0x5641a67d5e00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1896412afec8324d server_cookie=53d0ec9923780bcc in_seq=0 out_seq=0 2024-09-11T22:57:05.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.295+0000 7f23fb7fe640 1 -- 172.21.15.152:0/2301502482 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641a66bbd70 con 0x5641a67d3a10 2024-09-11T22:57:05.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.295+0000 7f23fb7fe640 10 client.16656 handle_command_reply: tid=0 2024-09-11T22:57:05.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.295+0000 7f2424de1740 1 client.16656 shutdown 2024-09-11T22:57:05.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.295+0000 7f2424de1740 1 -- 172.21.15.152:0/2301502482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 msgr2=0x5641a6761600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:05.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.295+0000 7f2424de1740 1 --2- 172.21.15.152:0/2301502482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a6761600 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2404011d00 tx=0x7f24040050e0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.295+0000 7f2424de1740 1 -- 172.21.15.152:0/2301502482 shutdown_connections 2024-09-11T22:57:05.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.295+0000 7f2424de1740 1 --2- 172.21.15.152:0/2301502482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5641a6759370 0x5641a6752680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.295+0000 7f2424de1740 1 --2- 172.21.15.152:0/2301502482 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5641a67d3a10 0x5641a67d5e00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.295+0000 7f2424de1740 1 --2- 172.21.15.152:0/2301502482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5641a674fb40 0x5641a6761b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.295+0000 7f2424de1740 1 --2- 172.21.15.152:0/2301502482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5641a674e5d0 0x5641a6761600 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.295+0000 7f2424de1740 1 -- 172.21.15.152:0/2301502482 >> 172.21.15.152:0/2301502482 conn(0x5641a6717310 msgr2=0x5641a67580b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:05.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.295+0000 7f23fb7fe640 0 client.16656 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:57:05.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.295+0000 7f2424de1740 1 -- 172.21.15.152:0/2301502482 shutdown_connections 2024-09-11T22:57:05.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.295+0000 7f2424de1740 1 -- 172.21.15.152:0/2301502482 wait complete. 2024-09-11T22:57:05.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.295+0000 7f2424de1740 20 client.16656 trim_cache size 0 max 16384 2024-09-11T22:57:05.314 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:57:05.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.296+0000 7f2422a08640 1 -- 172.21.15.152:0/3614361499 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f24080a2d70 msgr2=0x7f24080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:05.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.296+0000 7f2422a08640 1 --2- 172.21.15.152:0/3614361499 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f24080a2d70 0x7f24080a5230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f24140a42a0 tx=0x7f241406db20 comp rx=0 tx=0).stop 2024-09-11T22:57:05.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.296+0000 7f2422a08640 1 -- 172.21.15.152:0/3614361499 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f241c10e3d0 msgr2=0x7f241c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:05.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.296+0000 7f2422a08640 1 --2- 172.21.15.152:0/3614361499 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f241c10e3d0 0x7f241c19ef30 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f241400eaa0 tx=0x7f241400ef60 comp rx=0 tx=0).stop 2024-09-11T22:57:05.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.296+0000 7f2422a08640 1 -- 172.21.15.152:0/3614361499 shutdown_connections 2024-09-11T22:57:05.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.296+0000 7f2422a08640 1 --2- 172.21.15.152:0/3614361499 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f24080a2d70 0x7f24080a5230 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.296+0000 7f2422a08640 1 --2- 172.21.15.152:0/3614361499 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f241c10e3d0 0x7f241c19ef30 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.296+0000 7f2422a08640 1 --2- 172.21.15.152:0/3614361499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f241c074f40 0x7f241c19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.296+0000 7f2422a08640 1 --2- 172.21.15.152:0/3614361499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f241c074620 0x7f241c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:05.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.296+0000 7f2422a08640 1 -- 172.21.15.152:0/3614361499 >> 172.21.15.152:0/3614361499 conn(0x7f241c0fd8f0 msgr2=0x7f241c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:05.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.296+0000 7f2422a08640 1 -- 172.21.15.152:0/3614361499 shutdown_connections 2024-09-11T22:57:05.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:05.296+0000 7f2422a08640 1 -- 172.21.15.152:0/3614361499 wait complete. 2024-09-11T22:57:05.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:05 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:57:05.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:05 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:57:05.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:05 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:57:05.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:05 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:57:05.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:05 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:57:05.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:05 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:57:05.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:05 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:57:05.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:05 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:57:05.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:05 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:57:05.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:05 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:57:05.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:05 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:57:05.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:05 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:57:06.101 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:57:06.350 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:57:06.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.860+0000 7f93e353e640 1 -- 172.21.15.152:0/841428931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93dc102570 msgr2=0x7f93dc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:06.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.860+0000 7f93e353e640 1 --2- 172.21.15.152:0/841428931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93dc102570 0x7f93dc1029b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009900 tx=0x7f93cc02a480 comp rx=0 tx=0).stop 2024-09-11T22:57:06.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.860+0000 7f93e353e640 1 -- 172.21.15.152:0/841428931 shutdown_connections 2024-09-11T22:57:06.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.860+0000 7f93e353e640 1 --2- 172.21.15.152:0/841428931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93dc102ef0 0x7f93dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.860+0000 7f93e353e640 1 --2- 172.21.15.152:0/841428931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93dc102570 0x7f93dc1029b0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.860+0000 7f93e353e640 1 --2- 172.21.15.152:0/841428931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93dc100e20 0x7f93dc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.860+0000 7f93e353e640 1 -- 172.21.15.152:0/841428931 >> 172.21.15.152:0/841428931 conn(0x7f93dc0fc7d0 msgr2=0x7f93dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:06.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.860+0000 7f93e353e640 1 -- 172.21.15.152:0/841428931 shutdown_connections 2024-09-11T22:57:06.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.860+0000 7f93e353e640 1 -- 172.21.15.152:0/841428931 wait complete. 2024-09-11T22:57:06.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e353e640 1 Processor -- start 2024-09-11T22:57:06.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e353e640 1 -- start start 2024-09-11T22:57:06.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e353e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93dc100e20 0x7f93dc1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e353e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93dc102570 0x7f93dc1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e353e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93dc102ef0 0x7f93dc1a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e353e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc111020 con 0x7f93dc102570 2024-09-11T22:57:06.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e353e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc110ea0 con 0x7f93dc102ef0 2024-09-11T22:57:06.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e353e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1111a0 con 0x7f93dc100e20 2024-09-11T22:57:06.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e0ad9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93dc102570 0x7f93dc1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e0ad9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93dc102570 0x7f93dc1999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39938/0 (socket says 172.21.15.152:39938) 2024-09-11T22:57:06.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e0ad9640 1 -- 172.21.15.152:0/2830757316 learned_addr learned my addr 172.21.15.152:0/2830757316 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:06.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e12da640 1 --2- 172.21.15.152:0/2830757316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93dc100e20 0x7f93dc1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e1adb640 1 --2- 172.21.15.152:0/2830757316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93dc102ef0 0x7f93dc1a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e0ad9640 1 -- 172.21.15.152:0/2830757316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93dc100e20 msgr2=0x7f93dc1994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:06.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e0ad9640 1 --2- 172.21.15.152:0/2830757316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93dc100e20 0x7f93dc1994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e0ad9640 1 -- 172.21.15.152:0/2830757316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93dc102ef0 msgr2=0x7f93dc1a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:06.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e0ad9640 1 --2- 172.21.15.152:0/2830757316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93dc102ef0 0x7f93dc1a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e0ad9640 1 -- 172.21.15.152:0/2830757316 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93dc1a4a70 con 0x7f93dc102570 2024-09-11T22:57:06.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.861+0000 7f93e1adb640 1 --2- 172.21.15.152:0/2830757316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93dc102ef0 0x7f93dc1a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:57:06.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.862+0000 7f93e0ad9640 1 --2- 172.21.15.152:0/2830757316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93dc102570 0x7f93dc1999f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f93cc006010 tx=0x7f93cc003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:06.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.862+0000 7f93d27fc640 1 -- 172.21.15.152:0/2830757316 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc03f070 con 0x7f93dc102570 2024-09-11T22:57:06.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.862+0000 7f93e353e640 1 -- 172.21.15.152:0/2830757316 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93dc1a4c40 con 0x7f93dc102570 2024-09-11T22:57:06.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.862+0000 7f93d27fc640 1 -- 172.21.15.152:0/2830757316 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93cc031e30 con 0x7f93dc102570 2024-09-11T22:57:06.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.862+0000 7f93d27fc640 1 -- 172.21.15.152:0/2830757316 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc035030 con 0x7f93dc102570 2024-09-11T22:57:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.862+0000 7f93e353e640 1 -- 172.21.15.152:0/2830757316 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93dc1a5090 con 0x7f93dc102570 2024-09-11T22:57:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.864+0000 7f93d27fc640 1 -- 172.21.15.152:0/2830757316 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f93cc042070 con 0x7f93dc102570 2024-09-11T22:57:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e5917740 1 Processor -- start 2024-09-11T22:57:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e5917740 1 -- start start 2024-09-11T22:57:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e5917740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 0x55ea57daa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e5917740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57daaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e5917740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 0x55ea57dbbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e5917740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ea57d102f0 con 0x55ea57daa5d0 2024-09-11T22:57:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e5917740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ea57cb7cf0 con 0x55ea57dabb40 2024-09-11T22:57:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e5917740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea57d0c2b0 con 0x55ea57db9780 2024-09-11T22:57:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93d27fc640 1 --2- 172.21.15.152:0/2830757316 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f93c80a2cf0 0x7f93c80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e12da640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57daaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e12da640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57daaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39944/0 (socket says 172.21.15.152:39944) 2024-09-11T22:57:06.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e12da640 1 -- 172.21.15.152:0/3487598818 learned_addr learned my addr 172.21.15.152:0/3487598818 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:06.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e0ad9640 1 --2- 172.21.15.152:0/3487598818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 0x55ea57dbbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e1adb640 1 --2- 172.21.15.152:0/2830757316 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f93c80a2cf0 0x7f93c80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e1adb640 1 --2- 172.21.15.152:0/3487598818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 0x55ea57daa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e12da640 1 -- 172.21.15.152:0/3487598818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 msgr2=0x55ea57dbbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:06.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e12da640 1 --2- 172.21.15.152:0/3487598818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 0x55ea57dbbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e12da640 1 -- 172.21.15.152:0/3487598818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 msgr2=0x55ea57daa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:06.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e12da640 1 --2- 172.21.15.152:0/3487598818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 0x55ea57daa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93d27fc640 1 -- 172.21.15.152:0/2830757316 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f93cc0efd30 con 0x7f93dc102570 2024-09-11T22:57:06.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e12da640 1 -- 172.21.15.152:0/3487598818 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea57c9c3a0 con 0x55ea57daa5d0 2024-09-11T22:57:06.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.865+0000 7f93e1adb640 1 --2- 172.21.15.152:0/3487598818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 0x55ea57daa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:57:06.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.866+0000 7f93e12da640 1 --2- 172.21.15.152:0/3487598818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57daaa10 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009a70 tx=0x7f93d8009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb605f6dbadddc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:06.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.866+0000 7f93c77fe640 1 -- 172.21.15.152:0/3487598818 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8017070 con 0x55ea57daa5d0 2024-09-11T22:57:06.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.866+0000 7f93e1adb640 1 --2- 172.21.15.152:0/2830757316 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f93c80a2cf0 0x7f93c80a51b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f93d400e510 tx=0x7f93d40094d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.866+0000 7f93c77fe640 1 -- 172.21.15.152:0/3487598818 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93d8013470 con 0x55ea57daa5d0 2024-09-11T22:57:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.866+0000 7f93c77fe640 1 -- 172.21.15.152:0/3487598818 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8012630 con 0x55ea57daa5d0 2024-09-11T22:57:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.866+0000 7f93e5917740 1 -- 172.21.15.152:0/3487598818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 msgr2=0x55ea57daaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.866+0000 7f93e5917740 1 --2- 172.21.15.152:0/3487598818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57daaa10 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009a70 tx=0x7f93d8009f30 comp rx=0 tx=0).stop 2024-09-11T22:57:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.867+0000 7f93e5917740 1 -- 172.21.15.152:0/3487598818 shutdown_connections 2024-09-11T22:57:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.867+0000 7f93e5917740 1 --2- 172.21.15.152:0/3487598818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 0x55ea57dbbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.867+0000 7f93e5917740 1 --2- 172.21.15.152:0/3487598818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57daaa10 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.867+0000 7f93e5917740 1 --2- 172.21.15.152:0/3487598818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 0x55ea57daa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.867+0000 7f93e5917740 1 -- 172.21.15.152:0/3487598818 >> 172.21.15.152:0/3487598818 conn(0x55ea57d73310 msgr2=0x55ea57da6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.867+0000 7f93e5917740 1 -- 172.21.15.152:0/3487598818 shutdown_connections 2024-09-11T22:57:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.867+0000 7f93e5917740 1 -- 172.21.15.152:0/3487598818 wait complete. 2024-09-11T22:57:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.867+0000 7f93e5917740 1 -- start start 2024-09-11T22:57:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.867+0000 7f93e5917740 1 Processor -- start 2024-09-11T22:57:06.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.867+0000 7f93e5917740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57daa5d0 0x55ea57db90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.867+0000 7f93e5917740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57dabb40 0x55ea57db6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.867+0000 7f93e5917740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57db9780 0x55ea57db68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.867+0000 7f93e5917740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ea57dbc480 con 0x55ea57db9780 2024-09-11T22:57:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.867+0000 7f93e5917740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ea57ccb060 con 0x55ea57daa5d0 2024-09-11T22:57:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.867+0000 7f93e5917740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea57dbc600 con 0x55ea57dabb40 2024-09-11T22:57:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93e5917740 10 client.0 _ll_register_callbacks cb 0x55ea57a1e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93e1adb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57db9780 0x55ea57db68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93e1adb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57db9780 0x55ea57db68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39946/0 (socket says 172.21.15.152:39946) 2024-09-11T22:57:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93e1adb640 1 -- 172.21.15.152:0/1158536853 learned_addr learned my addr 172.21.15.152:0/1158536853 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:06.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93e12da640 1 --2- 172.21.15.152:0/1158536853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57daa5d0 0x55ea57db90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93e0ad9640 1 --2- 172.21.15.152:0/1158536853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57dabb40 0x55ea57db6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93e1adb640 1 -- 172.21.15.152:0/1158536853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57dabb40 msgr2=0x55ea57db6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:06.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93e1adb640 1 --2- 172.21.15.152:0/1158536853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57dabb40 0x55ea57db6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93e1adb640 1 -- 172.21.15.152:0/1158536853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57daa5d0 msgr2=0x55ea57db90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:06.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93e1adb640 1 --2- 172.21.15.152:0/1158536853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57daa5d0 0x55ea57db90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93e1adb640 1 -- 172.21.15.152:0/1158536853 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea57a813c0 con 0x55ea57db9780 2024-09-11T22:57:06.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93e1adb640 1 --2- 172.21.15.152:0/1158536853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57db9780 0x55ea57db68e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f93d400dd10 tx=0x7f93d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a3f373c8f9104e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:06.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:06.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93c4ff9640 1 -- 172.21.15.152:0/1158536853 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4023400 con 0x55ea57db9780 2024-09-11T22:57:06.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93c4ff9640 1 -- 172.21.15.152:0/1158536853 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f93d40235a0 con 0x55ea57db9780 2024-09-11T22:57:06.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93e5917740 1 -- 172.21.15.152:0/1158536853 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93d8013470 con 0x55ea57db9780 2024-09-11T22:57:06.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93c4ff9640 1 -- 172.21.15.152:0/1158536853 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f93d40237a0 con 0x55ea57db9780 2024-09-11T22:57:06.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93e5917740 10 client.16668 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:06.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.868+0000 7f93e5917740 1 -- 172.21.15.152:0/1158536853 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ea57dc7e90 con 0x55ea57db9780 2024-09-11T22:57:06.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.869+0000 7f93c4ff9640 1 -- 172.21.15.152:0/1158536853 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93d8013470 con 0x55ea57db9780 2024-09-11T22:57:06.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.869+0000 7f93c4ff9640 1 -- 172.21.15.152:0/1158536853 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f93d402db40 con 0x55ea57db9780 2024-09-11T22:57:06.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.869+0000 7f93e5917740 10 client.16668 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:06.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.869+0000 7f93e5917740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:57:06.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.869+0000 7f93e5917740 20 client.16668 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:06.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.869+0000 7f93e5917740 1 --2- 172.21.15.152:0/1158536853 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ea57dc92a0 0x55ea57dcb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.869+0000 7f93e5917740 4 client.16668 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:57:06.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.869+0000 7f93e5917740 1 -- 172.21.15.152:0/1158536853 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea57d1d860 con 0x55ea57dc92a0 2024-09-11T22:57:06.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.869+0000 7f93e12da640 1 --2- 172.21.15.152:0/1158536853 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ea57dc92a0 0x55ea57dcb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.870+0000 7f93e12da640 1 --2- 172.21.15.152:0/1158536853 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ea57dc92a0 0x55ea57dcb6f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=353788d74d148eae server_cookie=d85869dac7bd18b6 in_seq=0 out_seq=0 2024-09-11T22:57:06.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.870+0000 7f93c4ff9640 10 client.16668 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:06.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.870+0000 7f93c4ff9640 1 -- 172.21.15.152:0/1158536853 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea57d1d860 con 0x55ea57dc92a0 2024-09-11T22:57:06.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.870+0000 7f93c4ff9640 10 client.16668 handle_command_reply: tid=0 2024-09-11T22:57:06.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.870+0000 7f93e5917740 1 client.16668 shutdown 2024-09-11T22:57:06.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.871+0000 7f93e5917740 1 -- 172.21.15.152:0/1158536853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57db9780 msgr2=0x55ea57db68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:06.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.871+0000 7f93e5917740 1 --2- 172.21.15.152:0/1158536853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57db9780 0x55ea57db68e0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f93d400dd10 tx=0x7f93d4004a40 comp rx=0 tx=0).stop 2024-09-11T22:57:06.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.871+0000 7f93e5917740 1 -- 172.21.15.152:0/1158536853 shutdown_connections 2024-09-11T22:57:06.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.871+0000 7f93e5917740 1 --2- 172.21.15.152:0/1158536853 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57db9780 0x55ea57db68e0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.871+0000 7f93e5917740 1 --2- 172.21.15.152:0/1158536853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57dabb40 0x55ea57db6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.871+0000 7f93e5917740 1 --2- 172.21.15.152:0/1158536853 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ea57dc92a0 0x55ea57dcb6f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.871+0000 7f93e5917740 1 --2- 172.21.15.152:0/1158536853 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57daa5d0 0x55ea57db90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.871+0000 7f93e5917740 1 -- 172.21.15.152:0/1158536853 >> 172.21.15.152:0/1158536853 conn(0x55ea57d73310 msgr2=0x55ea57dac930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.871+0000 7f93c4ff9640 0 client.16668 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.871+0000 7f93e5917740 1 -- 172.21.15.152:0/1158536853 shutdown_connections 2024-09-11T22:57:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.871+0000 7f93e5917740 1 -- 172.21.15.152:0/1158536853 wait complete. 2024-09-11T22:57:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.871+0000 7f93e5917740 20 client.16668 trim_cache size 0 max 16384 2024-09-11T22:57:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.883+0000 7f93e5917740 1 Processor -- start 2024-09-11T22:57:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e5917740 1 -- start start 2024-09-11T22:57:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e5917740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57dc5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e5917740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 0x55ea57da6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e5917740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 0x55ea57da6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e5917740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1ab660 con 0x55ea57daa5d0 2024-09-11T22:57:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e5917740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1ad1a0 con 0x55ea57dabb40 2024-09-11T22:57:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e5917740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1abd30 con 0x55ea57db9780 2024-09-11T22:57:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e12da640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57dc5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e12da640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57dc5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39962/0 (socket says 172.21.15.152:39962) 2024-09-11T22:57:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e12da640 1 -- 172.21.15.152:0/3333437752 learned_addr learned my addr 172.21.15.152:0/3333437752 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e0ad9640 1 --2- 172.21.15.152:0/3333437752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 0x55ea57da6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e1adb640 1 --2- 172.21.15.152:0/3333437752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 0x55ea57da6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e12da640 1 -- 172.21.15.152:0/3333437752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 msgr2=0x55ea57da6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e12da640 1 --2- 172.21.15.152:0/3333437752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 0x55ea57da6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e12da640 1 -- 172.21.15.152:0/3333437752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 msgr2=0x55ea57da6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e12da640 1 --2- 172.21.15.152:0/3333437752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 0x55ea57da6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e12da640 1 -- 172.21.15.152:0/3333437752 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea57da7180 con 0x55ea57daa5d0 2024-09-11T22:57:06.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e1adb640 1 --2- 172.21.15.152:0/3333437752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 0x55ea57da6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:57:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e0ad9640 1 --2- 172.21.15.152:0/3333437752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 0x55ea57da6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93e12da640 1 --2- 172.21.15.152:0/3333437752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57dc5750 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f93d8013c20 tx=0x7f93d8009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca545a33a68d8bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93c6ffd640 1 -- 172.21.15.152:0/3333437752 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8017040 con 0x55ea57daa5d0 2024-09-11T22:57:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93c6ffd640 1 -- 172.21.15.152:0/3333437752 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93d8013950 con 0x55ea57daa5d0 2024-09-11T22:57:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.884+0000 7f93c6ffd640 1 -- 172.21.15.152:0/3333437752 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d80268a0 con 0x55ea57daa5d0 2024-09-11T22:57:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.885+0000 7f93e5917740 1 -- 172.21.15.152:0/3333437752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 msgr2=0x55ea57dc5750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.885+0000 7f93e5917740 1 --2- 172.21.15.152:0/3333437752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57dc5750 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f93d8013c20 tx=0x7f93d8009110 comp rx=0 tx=0).stop 2024-09-11T22:57:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.885+0000 7f93e5917740 1 -- 172.21.15.152:0/3333437752 shutdown_connections 2024-09-11T22:57:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.885+0000 7f93e5917740 1 --2- 172.21.15.152:0/3333437752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 0x55ea57da6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.885+0000 7f93e5917740 1 --2- 172.21.15.152:0/3333437752 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 0x55ea57da6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.885+0000 7f93e5917740 1 --2- 172.21.15.152:0/3333437752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57dc5750 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.885+0000 7f93e5917740 1 -- 172.21.15.152:0/3333437752 >> 172.21.15.152:0/3333437752 conn(0x55ea57d73310 msgr2=0x55ea57db6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.885+0000 7f93e5917740 1 -- 172.21.15.152:0/3333437752 shutdown_connections 2024-09-11T22:57:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.885+0000 7f93e5917740 1 -- 172.21.15.152:0/3333437752 wait complete. 2024-09-11T22:57:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.885+0000 7f93e5917740 1 -- start start 2024-09-11T22:57:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.885+0000 7f93e5917740 1 Processor -- start 2024-09-11T22:57:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e5917740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57dc1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e5917740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 0x55ea57db2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e5917740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 0x55ea57db3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e5917740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1acad0 con 0x55ea57daa5d0 2024-09-11T22:57:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e5917740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1ac400 con 0x55ea57dabb40 2024-09-11T22:57:06.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e5917740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1a9660 con 0x55ea57db9780 2024-09-11T22:57:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e5917740 10 client.0 _ll_register_callbacks cb 0x7f93d4009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e12da640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57dc1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e1adb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 0x55ea57db3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e12da640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57dc1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39966/0 (socket says 172.21.15.152:39966) 2024-09-11T22:57:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e12da640 1 -- 172.21.15.152:0/752116003 learned_addr learned my addr 172.21.15.152:0/752116003 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e0ad9640 1 --2- 172.21.15.152:0/752116003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 0x55ea57db2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e1adb640 1 -- 172.21.15.152:0/752116003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 msgr2=0x55ea57db2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e1adb640 1 --2- 172.21.15.152:0/752116003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 0x55ea57db2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e1adb640 1 -- 172.21.15.152:0/752116003 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 msgr2=0x55ea57dc1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e1adb640 1 --2- 172.21.15.152:0/752116003 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57dc1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e1adb640 1 -- 172.21.15.152:0/752116003 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea57db3ad0 con 0x55ea57db9780 2024-09-11T22:57:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e0ad9640 1 --2- 172.21.15.152:0/752116003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 0x55ea57db2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:06.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.886+0000 7f93e12da640 1 --2- 172.21.15.152:0/752116003 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57dc1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:57:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.887+0000 7f93e1adb640 1 --2- 172.21.15.152:0/752116003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 0x55ea57db3490 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f93d401b490 tx=0x7f93d4009860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f918a8679f51b7b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.887+0000 7f93c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-09-11T22:57:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.887+0000 7f93c4ff9640 1 -- 172.21.15.152:0/752116003 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d401bda0 con 0x55ea57db9780 2024-09-11T22:57:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.887+0000 7f93c4ff9640 1 -- 172.21.15.152:0/752116003 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f93d4023940 con 0x55ea57db9780 2024-09-11T22:57:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.887+0000 7f93e5917740 1 -- 172.21.15.152:0/752116003 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93d8013950 con 0x55ea57db9780 2024-09-11T22:57:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.888+0000 7f93c4ff9640 1 -- 172.21.15.152:0/752116003 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f93d4044020 con 0x55ea57db9780 2024-09-11T22:57:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.888+0000 7f93e5917740 10 client.25855 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.888+0000 7f93e5917740 1 -- 172.21.15.152:0/752116003 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ea57dad310 con 0x55ea57db9780 2024-09-11T22:57:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.888+0000 7f93c4ff9640 1 -- 172.21.15.152:0/752116003 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93d8013950 con 0x55ea57db9780 2024-09-11T22:57:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.888+0000 7f93c4ff9640 1 -- 172.21.15.152:0/752116003 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f93d400cb80 con 0x55ea57db9780 2024-09-11T22:57:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.888+0000 7f93e5917740 10 client.25855 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.888+0000 7f93e5917740 10 client.25855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:57:06.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.888+0000 7f93e5917740 20 client.25855 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:06.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.888+0000 7f93e5917740 1 --2- 172.21.15.152:0/752116003 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ea57dada30 0x55ea57dafe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:06.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.888+0000 7f93e5917740 4 client.25855 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:57:06.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.888+0000 7f93e5917740 1 -- 172.21.15.152:0/752116003 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea57d17d70 con 0x55ea57dada30 2024-09-11T22:57:06.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.888+0000 7f93e12da640 1 --2- 172.21.15.152:0/752116003 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ea57dada30 0x55ea57dafe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:06.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.889+0000 7f93c4ff9640 10 client.25855 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:06.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.889+0000 7f93e12da640 1 --2- 172.21.15.152:0/752116003 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ea57dada30 0x55ea57dafe20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a505774effa4a04c server_cookie=5cad00d3516fdd4c in_seq=0 out_seq=0 2024-09-11T22:57:06.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.889+0000 7f93c4ff9640 1 -- 172.21.15.152:0/752116003 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea57d17d70 con 0x55ea57dada30 2024-09-11T22:57:06.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.889+0000 7f93c4ff9640 10 client.25855 handle_command_reply: tid=0 2024-09-11T22:57:06.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.889+0000 7f93e5917740 1 client.25855 shutdown 2024-09-11T22:57:06.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.890+0000 7f93e5917740 1 -- 172.21.15.152:0/752116003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 msgr2=0x55ea57db3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:06.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.890+0000 7f93e5917740 1 --2- 172.21.15.152:0/752116003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 0x55ea57db3490 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f93d401b490 tx=0x7f93d4009860 comp rx=0 tx=0).stop 2024-09-11T22:57:06.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.890+0000 7f93e5917740 1 -- 172.21.15.152:0/752116003 shutdown_connections 2024-09-11T22:57:06.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.890+0000 7f93e5917740 1 --2- 172.21.15.152:0/752116003 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea57db9780 0x55ea57db3490 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.890+0000 7f93e5917740 1 --2- 172.21.15.152:0/752116003 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ea57dada30 0x55ea57dafe20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.890+0000 7f93e5917740 1 --2- 172.21.15.152:0/752116003 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea57dabb40 0x55ea57db2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.890+0000 7f93e5917740 1 --2- 172.21.15.152:0/752116003 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea57daa5d0 0x55ea57dc1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.890+0000 7f93c4ff9640 0 client.25855 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:06.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.890+0000 7f93e5917740 1 -- 172.21.15.152:0/752116003 >> 172.21.15.152:0/752116003 conn(0x55ea57d73310 msgr2=0x55ea57db6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:06.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.890+0000 7f93e5917740 1 -- 172.21.15.152:0/752116003 shutdown_connections 2024-09-11T22:57:06.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.890+0000 7f93e5917740 1 -- 172.21.15.152:0/752116003 wait complete. 2024-09-11T22:57:06.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.890+0000 7f93e5917740 20 client.25855 trim_cache size 0 max 16384 2024-09-11T22:57:06.911 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:57:06.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.891+0000 7f93e353e640 1 -- 172.21.15.152:0/2830757316 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f93c80a2cf0 msgr2=0x7f93c80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:06.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.891+0000 7f93e353e640 1 --2- 172.21.15.152:0/2830757316 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f93c80a2cf0 0x7f93c80a51b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f93d400e510 tx=0x7f93d40094d0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.891+0000 7f93e353e640 1 -- 172.21.15.152:0/2830757316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93dc102570 msgr2=0x7f93dc1999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:06.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.891+0000 7f93e353e640 1 --2- 172.21.15.152:0/2830757316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93dc102570 0x7f93dc1999f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f93cc006010 tx=0x7f93cc003c80 comp rx=0 tx=0).stop 2024-09-11T22:57:06.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.891+0000 7f93e353e640 1 -- 172.21.15.152:0/2830757316 shutdown_connections 2024-09-11T22:57:06.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.891+0000 7f93e353e640 1 --2- 172.21.15.152:0/2830757316 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f93c80a2cf0 0x7f93c80a51b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.891+0000 7f93e353e640 1 --2- 172.21.15.152:0/2830757316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93dc102ef0 0x7f93dc1a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.891+0000 7f93e353e640 1 --2- 172.21.15.152:0/2830757316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93dc102570 0x7f93dc1999f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.891+0000 7f93e353e640 1 --2- 172.21.15.152:0/2830757316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93dc100e20 0x7f93dc1994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:06.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.891+0000 7f93e353e640 1 -- 172.21.15.152:0/2830757316 >> 172.21.15.152:0/2830757316 conn(0x7f93dc0fc7d0 msgr2=0x7f93dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:06.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.891+0000 7f93e353e640 1 -- 172.21.15.152:0/2830757316 shutdown_connections 2024-09-11T22:57:06.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:06.891+0000 7f93e353e640 1 -- 172.21.15.152:0/2830757316 wait complete. 2024-09-11T22:57:07.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:06 smithi152 ceph-mon[31181]: pgmap v129: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-11T22:57:07.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:06 smithi191 ceph-mon[35211]: pgmap v129: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-11T22:57:07.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:06 smithi204 ceph-mon[35454]: pgmap v129: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-11T22:57:08.696 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:57:08.697 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:57:08.966 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:57:09.261 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:08 smithi152 ceph-mon[31181]: pgmap v130: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-11T22:57:09.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:08 smithi191 ceph-mon[35211]: pgmap v130: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-11T22:57:09.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:08 smithi204 ceph-mon[35454]: pgmap v130: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-11T22:57:09.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.500+0000 7f6d92574640 1 -- 172.21.15.152:0/1532238128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d8c103660 msgr2=0x7f6d8c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:09.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.500+0000 7f6d92574640 1 --2- 172.21.15.152:0/1532238128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d8c103660 0x7f6d8c10bb50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009900 tx=0x7f6d7402a480 comp rx=0 tx=0).stop 2024-09-11T22:57:09.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.500+0000 7f6d92574640 1 -- 172.21.15.152:0/1532238128 shutdown_connections 2024-09-11T22:57:09.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.500+0000 7f6d92574640 1 --2- 172.21.15.152:0/1532238128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d8c10c170 0x7f6d8c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.500+0000 7f6d92574640 1 --2- 172.21.15.152:0/1532238128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d8c103660 0x7f6d8c10bb50 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.500+0000 7f6d92574640 1 --2- 172.21.15.152:0/1532238128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d8c102d40 0x7f6d8c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.500+0000 7f6d92574640 1 -- 172.21.15.152:0/1532238128 >> 172.21.15.152:0/1532238128 conn(0x7f6d8c0fc7a0 msgr2=0x7f6d8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:09.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.500+0000 7f6d92574640 1 -- 172.21.15.152:0/1532238128 shutdown_connections 2024-09-11T22:57:09.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.500+0000 7f6d92574640 1 -- 172.21.15.152:0/1532238128 wait complete. 2024-09-11T22:57:09.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.501+0000 7f6d92574640 1 Processor -- start 2024-09-11T22:57:09.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.501+0000 7f6d92574640 1 -- start start 2024-09-11T22:57:09.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.501+0000 7f6d92574640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d8c102d40 0x7f6d8c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.501+0000 7f6d92574640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d8c103660 0x7f6d8c19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.501+0000 7f6d92574640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d8c10c170 0x7f6d8c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.501+0000 7f6d92574640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c111fb0 con 0x7f6d8c102d40 2024-09-11T22:57:09.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.501+0000 7f6d92574640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c111e30 con 0x7f6d8c103660 2024-09-11T22:57:09.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.501+0000 7f6d92574640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c112130 con 0x7f6d8c10c170 2024-09-11T22:57:09.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.501+0000 7f6d8bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d8c102d40 0x7f6d8c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:09.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.501+0000 7f6d8bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d8c102d40 0x7f6d8c19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43632/0 (socket says 172.21.15.152:43632) 2024-09-11T22:57:09.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.501+0000 7f6d8bfff640 1 -- 172.21.15.152:0/4028129403 learned_addr learned my addr 172.21.15.152:0/4028129403 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:09.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.502+0000 7f6d8b7fe640 1 --2- 172.21.15.152:0/4028129403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d8c103660 0x7f6d8c19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:09.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.502+0000 7f6d90b11640 1 --2- 172.21.15.152:0/4028129403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d8c10c170 0x7f6d8c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:09.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.502+0000 7f6d8bfff640 1 -- 172.21.15.152:0/4028129403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d8c10c170 msgr2=0x7f6d8c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:09.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.502+0000 7f6d8bfff640 1 --2- 172.21.15.152:0/4028129403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d8c10c170 0x7f6d8c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.502+0000 7f6d8bfff640 1 -- 172.21.15.152:0/4028129403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d8c103660 msgr2=0x7f6d8c19ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:09.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.502+0000 7f6d8bfff640 1 --2- 172.21.15.152:0/4028129403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d8c103660 0x7f6d8c19ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.502+0000 7f6d8bfff640 1 -- 172.21.15.152:0/4028129403 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8c19f670 con 0x7f6d8c102d40 2024-09-11T22:57:09.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.502+0000 7f6d8bfff640 1 --2- 172.21.15.152:0/4028129403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d8c102d40 0x7f6d8c19a6c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00d950 tx=0x7f6d7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:09.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.502+0000 7f6d897fa640 1 -- 172.21.15.152:0/4028129403 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c017070 con 0x7f6d8c102d40 2024-09-11T22:57:09.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.502+0000 7f6d897fa640 1 -- 172.21.15.152:0/4028129403 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d7c009c30 con 0x7f6d8c102d40 2024-09-11T22:57:09.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.502+0000 7f6d92574640 1 -- 172.21.15.152:0/4028129403 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8c19f960 con 0x7f6d8c102d40 2024-09-11T22:57:09.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.502+0000 7f6d897fa640 1 -- 172.21.15.152:0/4028129403 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c00f6b0 con 0x7f6d8c102d40 2024-09-11T22:57:09.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.502+0000 7f6d92574640 1 -- 172.21.15.152:0/4028129403 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8c1a7240 con 0x7f6d8c102d40 2024-09-11T22:57:09.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d9494d740 1 Processor -- start 2024-09-11T22:57:09.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d9494d740 1 -- start start 2024-09-11T22:57:09.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d9494d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03de0b60 0x555d03de0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d9494d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de1480 0x555d03deb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d9494d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03debd40 0x555d03dee130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d9494d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x555d03d46280 con 0x555d03de0b60 2024-09-11T22:57:09.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d9494d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555d03cedb10 con 0x555d03debd40 2024-09-11T22:57:09.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d9494d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555d03d42240 con 0x555d03de1480 2024-09-11T22:57:09.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d90b11640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03de0b60 0x555d03de0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:09.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d90b11640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03de0b60 0x555d03de0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43646/0 (socket says 172.21.15.152:43646) 2024-09-11T22:57:09.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d90b11640 1 -- 172.21.15.152:0/225730529 learned_addr learned my addr 172.21.15.152:0/225730529 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:09.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d90b11640 1 -- 172.21.15.152:0/225730529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de1480 msgr2=0x555d03deb800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:57:09.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d90b11640 1 --2- 172.21.15.152:0/225730529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de1480 0x555d03deb800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d90b11640 1 -- 172.21.15.152:0/225730529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03debd40 msgr2=0x555d03dee130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:09.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d897fa640 1 -- 172.21.15.152:0/4028129403 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6d7c00f850 con 0x7f6d8c102d40 2024-09-11T22:57:09.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d90b11640 1 --2- 172.21.15.152:0/225730529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03debd40 0x555d03dee130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.505+0000 7f6d90b11640 1 -- 172.21.15.152:0/225730529 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d03cfa1f0 con 0x555d03de0b60 2024-09-11T22:57:09.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.506+0000 7f6d90b11640 1 --2- 172.21.15.152:0/225730529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03de0b60 0x555d03de0f40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6d84007d90 tx=0x7f6d8400c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff47cef789e7400 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:09.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.506+0000 7f6d827fc640 1 -- 172.21.15.152:0/225730529 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84015020 con 0x555d03de0b60 2024-09-11T22:57:09.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.506+0000 7f6d827fc640 1 -- 172.21.15.152:0/225730529 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d84011b10 con 0x555d03de0b60 2024-09-11T22:57:09.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.506+0000 7f6d827fc640 1 -- 172.21.15.152:0/225730529 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84010b10 con 0x555d03de0b60 2024-09-11T22:57:09.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.506+0000 7f6d897fa640 1 --2- 172.21.15.152:0/4028129403 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6d780a2d70 0x7f6d780a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.506+0000 7f6d8bfff640 1 --2- 172.21.15.152:0/4028129403 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6d780a2d70 0x7f6d780a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:09.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.506+0000 7f6d897fa640 1 -- 172.21.15.152:0/4028129403 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f6d7c0cf090 con 0x7f6d8c102d40 2024-09-11T22:57:09.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.507+0000 7f6d9494d740 1 -- 172.21.15.152:0/225730529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03de0b60 msgr2=0x555d03de0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:09.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.507+0000 7f6d9494d740 1 --2- 172.21.15.152:0/225730529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03de0b60 0x555d03de0f40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6d84007d90 tx=0x7f6d8400c560 comp rx=0 tx=0).stop 2024-09-11T22:57:09.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.507+0000 7f6d8bfff640 1 --2- 172.21.15.152:0/4028129403 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6d780a2d70 0x7f6d780a5230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x555d03dddf50 tx=0x7f6d7c086040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:09.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.507+0000 7f6d9494d740 1 -- 172.21.15.152:0/225730529 shutdown_connections 2024-09-11T22:57:09.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.507+0000 7f6d9494d740 1 --2- 172.21.15.152:0/225730529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03debd40 0x555d03dee130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.507+0000 7f6d9494d740 1 --2- 172.21.15.152:0/225730529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de1480 0x555d03deb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.507+0000 7f6d9494d740 1 --2- 172.21.15.152:0/225730529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03de0b60 0x555d03de0f40 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.507+0000 7f6d9494d740 1 -- 172.21.15.152:0/225730529 >> 172.21.15.152:0/225730529 conn(0x555d03ddd0f0 msgr2=0x555d03ddd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:09.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.507+0000 7f6d9494d740 1 -- 172.21.15.152:0/225730529 shutdown_connections 2024-09-11T22:57:09.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.507+0000 7f6d9494d740 1 -- 172.21.15.152:0/225730529 wait complete. 2024-09-11T22:57:09.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.507+0000 7f6d9494d740 1 -- start start 2024-09-11T22:57:09.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d9494d740 1 Processor -- start 2024-09-11T22:57:09.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d9494d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de0b60 0x555d03de8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d9494d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03de1480 0x555d03de9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d9494d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03debd40 0x555d03de9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d9494d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x555d03dee910 con 0x555d03debd40 2024-09-11T22:57:09.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d9494d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555d03d00f10 con 0x555d03de1480 2024-09-11T22:57:09.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d9494d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555d03deea90 con 0x555d03de0b60 2024-09-11T22:57:09.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d9494d740 10 client.0 _ll_register_callbacks cb 0x555d03cc9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:09.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d90b11640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03debd40 0x555d03de9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:09.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d90b11640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03debd40 0x555d03de9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43660/0 (socket says 172.21.15.152:43660) 2024-09-11T22:57:09.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d90b11640 1 -- 172.21.15.152:0/679192567 learned_addr learned my addr 172.21.15.152:0/679192567 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:09.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d8b7fe640 1 --2- 172.21.15.152:0/679192567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03de1480 0x555d03de9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:09.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d90b11640 1 --2- 172.21.15.152:0/679192567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de0b60 0x555d03de8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:09.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d90b11640 1 -- 172.21.15.152:0/679192567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de0b60 msgr2=0x555d03de8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:09.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d90b11640 1 --2- 172.21.15.152:0/679192567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de0b60 0x555d03de8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d90b11640 1 -- 172.21.15.152:0/679192567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03de1480 msgr2=0x555d03de9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:09.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d90b11640 1 --2- 172.21.15.152:0/679192567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03de1480 0x555d03de9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d90b11640 1 -- 172.21.15.152:0/679192567 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d03ab76a0 con 0x555d03debd40 2024-09-11T22:57:09.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.508+0000 7f6d8b7fe640 1 --2- 172.21.15.152:0/679192567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03de1480 0x555d03de9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:09.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.509+0000 7f6d90b11640 1 --2- 172.21.15.152:0/679192567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03debd40 0x555d03de9640 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6d840075a0 tx=0x7f6d84010670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b296523c8e127f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:09.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.509+0000 7f6d6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:09.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.509+0000 7f6d6b7fe640 1 -- 172.21.15.152:0/679192567 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84015040 con 0x555d03debd40 2024-09-11T22:57:09.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.509+0000 7f6d6b7fe640 1 -- 172.21.15.152:0/679192567 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f6d84011630 con 0x555d03debd40 2024-09-11T22:57:09.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.509+0000 7f6d9494d740 1 -- 172.21.15.152:0/679192567 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d84011b10 con 0x555d03debd40 2024-09-11T22:57:09.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.509+0000 7f6d6b7fe640 1 -- 172.21.15.152:0/679192567 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d8402d020 con 0x555d03debd40 2024-09-11T22:57:09.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.509+0000 7f6d9494d740 10 client.16686 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:09.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.509+0000 7f6d9494d740 1 -- 172.21.15.152:0/679192567 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555d03dfa4c0 con 0x555d03debd40 2024-09-11T22:57:09.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.509+0000 7f6d6b7fe640 1 -- 172.21.15.152:0/679192567 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d84011b10 con 0x555d03debd40 2024-09-11T22:57:09.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.509+0000 7f6d6b7fe640 1 -- 172.21.15.152:0/679192567 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6d8402b4b0 con 0x555d03debd40 2024-09-11T22:57:09.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.509+0000 7f6d9494d740 10 client.16686 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:09.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.509+0000 7f6d9494d740 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:57:09.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.509+0000 7f6d9494d740 20 client.16686 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:09.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.509+0000 7f6d9494d740 1 --2- 172.21.15.152:0/679192567 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555d03dfb810 0x555d03dfdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.509+0000 7f6d9494d740 4 client.16686 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:57:09.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.509+0000 7f6d9494d740 1 -- 172.21.15.152:0/679192567 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d03d537f0 con 0x555d03dfb810 2024-09-11T22:57:09.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.510+0000 7f6d8b7fe640 1 --2- 172.21.15.152:0/679192567 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555d03dfb810 0x555d03dfdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:09.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.510+0000 7f6d6b7fe640 10 client.16686 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:09.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.510+0000 7f6d8b7fe640 1 --2- 172.21.15.152:0/679192567 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555d03dfb810 0x555d03dfdc00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d9191964994db75 server_cookie=39da64157a58ad5b in_seq=0 out_seq=0 2024-09-11T22:57:09.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.511+0000 7f6d6b7fe640 1 -- 172.21.15.152:0/679192567 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d03d537f0 con 0x555d03dfb810 2024-09-11T22:57:09.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.511+0000 7f6d6b7fe640 10 client.16686 handle_command_reply: tid=0 2024-09-11T22:57:09.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.511+0000 7f6d9494d740 1 client.16686 shutdown 2024-09-11T22:57:09.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.511+0000 7f6d9494d740 1 -- 172.21.15.152:0/679192567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03debd40 msgr2=0x555d03de9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:09.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.511+0000 7f6d9494d740 1 --2- 172.21.15.152:0/679192567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03debd40 0x555d03de9640 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6d840075a0 tx=0x7f6d84010670 comp rx=0 tx=0).stop 2024-09-11T22:57:09.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.511+0000 7f6d9494d740 1 -- 172.21.15.152:0/679192567 shutdown_connections 2024-09-11T22:57:09.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.511+0000 7f6d9494d740 1 --2- 172.21.15.152:0/679192567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03debd40 0x555d03de9640 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.511+0000 7f6d9494d740 1 --2- 172.21.15.152:0/679192567 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555d03dfb810 0x555d03dfdc00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.511+0000 7f6d6b7fe640 0 client.16686 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:09.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.511+0000 7f6d9494d740 1 --2- 172.21.15.152:0/679192567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03de1480 0x555d03de9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.511+0000 7f6d9494d740 1 --2- 172.21.15.152:0/679192567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de0b60 0x555d03de8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.511+0000 7f6d9494d740 1 -- 172.21.15.152:0/679192567 >> 172.21.15.152:0/679192567 conn(0x555d03ddd0f0 msgr2=0x555d03de2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:09.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.511+0000 7f6d9494d740 1 -- 172.21.15.152:0/679192567 shutdown_connections 2024-09-11T22:57:09.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.511+0000 7f6d9494d740 1 -- 172.21.15.152:0/679192567 wait complete. 2024-09-11T22:57:09.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.511+0000 7f6d9494d740 20 client.16686 trim_cache size 0 max 16384 2024-09-11T22:57:09.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.543+0000 7f6d9494d740 1 Processor -- start 2024-09-11T22:57:09.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.543+0000 7f6d9494d740 1 -- start start 2024-09-11T22:57:09.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.543+0000 7f6d9494d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03de0b60 0x555d03de7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.543+0000 7f6d9494d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de1480 0x555d03e033c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.543+0000 7f6d9494d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03debd40 0x555d03e03900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.543+0000 7f6d9494d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1afa90 con 0x555d03de0b60 2024-09-11T22:57:09.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.543+0000 7f6d9494d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1b15d0 con 0x555d03debd40 2024-09-11T22:57:09.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.543+0000 7f6d9494d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1b0160 con 0x555d03de1480 2024-09-11T22:57:09.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.544+0000 7f6d90b11640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03de0b60 0x555d03de7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:09.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.544+0000 7f6d90b11640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03de0b60 0x555d03de7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43668/0 (socket says 172.21.15.152:43668) 2024-09-11T22:57:09.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.544+0000 7f6d90b11640 1 -- 172.21.15.152:0/4003883340 learned_addr learned my addr 172.21.15.152:0/4003883340 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:09.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.544+0000 7f6d8b7fe640 1 --2- 172.21.15.152:0/4003883340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de1480 0x555d03e033c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:09.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.544+0000 7f6d90b11640 1 -- 172.21.15.152:0/4003883340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de1480 msgr2=0x555d03e033c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:09.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.544+0000 7f6d90b11640 1 --2- 172.21.15.152:0/4003883340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de1480 0x555d03e033c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.544+0000 7f6d90b11640 1 -- 172.21.15.152:0/4003883340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03debd40 msgr2=0x555d03e03900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:09.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.544+0000 7f6d90b11640 1 --2- 172.21.15.152:0/4003883340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03debd40 0x555d03e03900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.544+0000 7f6d90b11640 1 -- 172.21.15.152:0/4003883340 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d03de12b0 con 0x555d03de0b60 2024-09-11T22:57:09.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.544+0000 7f6d90b11640 1 --2- 172.21.15.152:0/4003883340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03de0b60 0x555d03de7a70 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6d8402b8d0 tx=0x7f6d8400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c5dfb5f6af84de server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:09.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.544+0000 7f6d81ffb640 1 -- 172.21.15.152:0/4003883340 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84037070 con 0x555d03de0b60 2024-09-11T22:57:09.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.544+0000 7f6d81ffb640 1 -- 172.21.15.152:0/4003883340 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d84010870 con 0x555d03de0b60 2024-09-11T22:57:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.544+0000 7f6d81ffb640 1 -- 172.21.15.152:0/4003883340 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84015070 con 0x555d03de0b60 2024-09-11T22:57:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.545+0000 7f6d9494d740 1 -- 172.21.15.152:0/4003883340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03de0b60 msgr2=0x555d03de7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.545+0000 7f6d9494d740 1 --2- 172.21.15.152:0/4003883340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03de0b60 0x555d03de7a70 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6d8402b8d0 tx=0x7f6d8400cb90 comp rx=0 tx=0).stop 2024-09-11T22:57:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.545+0000 7f6d9494d740 1 -- 172.21.15.152:0/4003883340 shutdown_connections 2024-09-11T22:57:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.545+0000 7f6d9494d740 1 --2- 172.21.15.152:0/4003883340 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03debd40 0x555d03e03900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.545+0000 7f6d9494d740 1 --2- 172.21.15.152:0/4003883340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de1480 0x555d03e033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.545+0000 7f6d9494d740 1 --2- 172.21.15.152:0/4003883340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03de0b60 0x555d03de7a70 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.545+0000 7f6d9494d740 1 -- 172.21.15.152:0/4003883340 >> 172.21.15.152:0/4003883340 conn(0x555d03ddd0f0 msgr2=0x555d03de7360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.545+0000 7f6d9494d740 1 -- 172.21.15.152:0/4003883340 shutdown_connections 2024-09-11T22:57:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.545+0000 7f6d9494d740 1 -- 172.21.15.152:0/4003883340 wait complete. 2024-09-11T22:57:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.546+0000 7f6d9494d740 1 -- start start 2024-09-11T22:57:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.546+0000 7f6d9494d740 1 Processor -- start 2024-09-11T22:57:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.546+0000 7f6d9494d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03de0b60 0x555d03de5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.546+0000 7f6d9494d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de1480 0x555d03de62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.546+0000 7f6d9494d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03debd40 0x555d03de6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.546+0000 7f6d9494d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1b0f00 con 0x555d03debd40 2024-09-11T22:57:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.546+0000 7f6d9494d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1b0830 con 0x555d03de0b60 2024-09-11T22:57:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.546+0000 7f6d9494d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8c1ada50 con 0x555d03de1480 2024-09-11T22:57:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.546+0000 7f6d9494d740 10 client.0 _ll_register_callbacks cb 0x7f6d84007be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.546+0000 7f6d90b11640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03debd40 0x555d03de6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.546+0000 7f6d90b11640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03debd40 0x555d03de6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43678/0 (socket says 172.21.15.152:43678) 2024-09-11T22:57:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.546+0000 7f6d90b11640 1 -- 172.21.15.152:0/1947498343 learned_addr learned my addr 172.21.15.152:0/1947498343 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.546+0000 7f6d8b7fe640 1 --2- 172.21.15.152:0/1947498343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de1480 0x555d03de62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.546+0000 7f6d90b11640 1 --2- 172.21.15.152:0/1947498343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03de0b60 0x555d03de5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.547+0000 7f6d90b11640 1 -- 172.21.15.152:0/1947498343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de1480 msgr2=0x555d03de62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.547+0000 7f6d90b11640 1 --2- 172.21.15.152:0/1947498343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de1480 0x555d03de62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.547+0000 7f6d90b11640 1 -- 172.21.15.152:0/1947498343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03de0b60 msgr2=0x555d03de5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.547+0000 7f6d90b11640 1 --2- 172.21.15.152:0/1947498343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03de0b60 0x555d03de5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.547+0000 7f6d90b11640 1 -- 172.21.15.152:0/1947498343 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d03e65590 con 0x555d03debd40 2024-09-11T22:57:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.547+0000 7f6d90b11640 1 --2- 172.21.15.152:0/1947498343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03debd40 0x555d03de6800 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6d8402b800 tx=0x7f6d84034040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260cdc1214557d88 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.547+0000 7f6d6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.547+0000 7f6d6b7fe640 1 -- 172.21.15.152:0/1947498343 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84043060 con 0x555d03debd40 2024-09-11T22:57:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.547+0000 7f6d6b7fe640 1 -- 172.21.15.152:0/1947498343 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f6d84010550 con 0x555d03debd40 2024-09-11T22:57:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.547+0000 7f6d9494d740 1 -- 172.21.15.152:0/1947498343 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555d03d4dd00 con 0x555d03debd40 2024-09-11T22:57:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.547+0000 7f6d6b7fe640 1 -- 172.21.15.152:0/1947498343 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d84037040 con 0x555d03debd40 2024-09-11T22:57:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.547+0000 7f6d9494d740 10 client.16692 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.547+0000 7f6d9494d740 1 -- 172.21.15.152:0/1947498343 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555d03e66870 con 0x555d03debd40 2024-09-11T22:57:09.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.547+0000 7f6d6b7fe640 1 -- 172.21.15.152:0/1947498343 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555d03d4dd00 con 0x555d03debd40 2024-09-11T22:57:09.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.548+0000 7f6d6b7fe640 1 -- 172.21.15.152:0/1947498343 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6d8401c670 con 0x555d03debd40 2024-09-11T22:57:09.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.548+0000 7f6d9494d740 10 client.16692 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:09.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.548+0000 7f6d9494d740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:57:09.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.548+0000 7f6d9494d740 20 client.16692 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:09.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.548+0000 7f6d9494d740 1 --2- 172.21.15.152:0/1947498343 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555d03dfb810 0x555d03e68e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:09.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.548+0000 7f6d9494d740 4 client.16692 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:57:09.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.548+0000 7f6d9494d740 1 -- 172.21.15.152:0/1947498343 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555d03cca110 con 0x555d03dfb810 2024-09-11T22:57:09.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.548+0000 7f6d8b7fe640 1 --2- 172.21.15.152:0/1947498343 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555d03dfb810 0x555d03e68e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:09.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.548+0000 7f6d8b7fe640 1 --2- 172.21.15.152:0/1947498343 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555d03dfb810 0x555d03e68e80 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58bce5414fec2b0e server_cookie=2d2a447d2f4474c2 in_seq=0 out_seq=0 2024-09-11T22:57:09.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.548+0000 7f6d6b7fe640 10 client.16692 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:09.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.549+0000 7f6d6b7fe640 1 -- 172.21.15.152:0/1947498343 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555d03cca110 con 0x555d03dfb810 2024-09-11T22:57:09.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.549+0000 7f6d6b7fe640 10 client.16692 handle_command_reply: tid=0 2024-09-11T22:57:09.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.549+0000 7f6d9494d740 1 client.16692 shutdown 2024-09-11T22:57:09.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.549+0000 7f6d9494d740 1 -- 172.21.15.152:0/1947498343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03debd40 msgr2=0x555d03de6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:09.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.549+0000 7f6d9494d740 1 --2- 172.21.15.152:0/1947498343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03debd40 0x555d03de6800 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6d8402b800 tx=0x7f6d84034040 comp rx=0 tx=0).stop 2024-09-11T22:57:09.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.550+0000 7f6d9494d740 1 -- 172.21.15.152:0/1947498343 shutdown_connections 2024-09-11T22:57:09.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.550+0000 7f6d9494d740 1 --2- 172.21.15.152:0/1947498343 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555d03debd40 0x555d03de6800 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.550+0000 7f6d9494d740 1 --2- 172.21.15.152:0/1947498343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555d03de1480 0x555d03de62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.550+0000 7f6d9494d740 1 --2- 172.21.15.152:0/1947498343 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555d03dfb810 0x555d03e68e80 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.550+0000 7f6d9494d740 1 --2- 172.21.15.152:0/1947498343 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555d03de0b60 0x555d03de5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.550+0000 7f6d9494d740 1 -- 172.21.15.152:0/1947498343 >> 172.21.15.152:0/1947498343 conn(0x555d03ddd0f0 msgr2=0x555d03de93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:09.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.550+0000 7f6d6b7fe640 0 client.16692 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:09.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.550+0000 7f6d9494d740 1 -- 172.21.15.152:0/1947498343 shutdown_connections 2024-09-11T22:57:09.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.550+0000 7f6d9494d740 1 -- 172.21.15.152:0/1947498343 wait complete. 2024-09-11T22:57:09.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.550+0000 7f6d9494d740 20 client.16692 trim_cache size 0 max 16384 2024-09-11T22:57:09.555 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:57:09.555 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T22:57:09.555 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "a6c9993c-c126-4270-9467-4ce8b93e2083", 2024-09-11T22:57:09.555 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T22:57:09.555 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:57:09.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.551+0000 7f6d92574640 1 -- 172.21.15.152:0/4028129403 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6d780a2d70 msgr2=0x7f6d780a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:09.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.551+0000 7f6d92574640 1 --2- 172.21.15.152:0/4028129403 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6d780a2d70 0x7f6d780a5230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x555d03dddf50 tx=0x7f6d7c086040 comp rx=0 tx=0).stop 2024-09-11T22:57:09.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.551+0000 7f6d92574640 1 -- 172.21.15.152:0/4028129403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d8c102d40 msgr2=0x7f6d8c19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:09.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.551+0000 7f6d92574640 1 --2- 172.21.15.152:0/4028129403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d8c102d40 0x7f6d8c19a6c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00d950 tx=0x7f6d7c00de10 comp rx=0 tx=0).stop 2024-09-11T22:57:09.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.551+0000 7f6d92574640 1 -- 172.21.15.152:0/4028129403 shutdown_connections 2024-09-11T22:57:09.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.551+0000 7f6d92574640 1 --2- 172.21.15.152:0/4028129403 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6d780a2d70 0x7f6d780a5230 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.551+0000 7f6d92574640 1 --2- 172.21.15.152:0/4028129403 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d8c10c170 0x7f6d8c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.551+0000 7f6d92574640 1 --2- 172.21.15.152:0/4028129403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d8c103660 0x7f6d8c19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.551+0000 7f6d92574640 1 --2- 172.21.15.152:0/4028129403 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d8c102d40 0x7f6d8c19a6c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:09.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.551+0000 7f6d92574640 1 -- 172.21.15.152:0/4028129403 >> 172.21.15.152:0/4028129403 conn(0x7f6d8c0fc7a0 msgr2=0x7f6d8c1008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:09.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.551+0000 7f6d92574640 1 -- 172.21.15.152:0/4028129403 shutdown_connections 2024-09-11T22:57:09.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:09.551+0000 7f6d92574640 1 -- 172.21.15.152:0/4028129403 wait complete. 2024-09-11T22:57:10.477 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:57:10.727 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:57:11.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:10 smithi152 ceph-mon[31181]: pgmap v131: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-11T22:57:11.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:10 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T22:57:11.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:10 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:57:11.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:10 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T22:57:11.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:10 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:57:11.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.258+0000 7f34e2fde640 1 -- 172.21.15.152:0/3937154032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34dc102570 msgr2=0x7f34dc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:11.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.258+0000 7f34e2fde640 1 --2- 172.21.15.152:0/3937154032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34dc102570 0x7f34dc1029b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009900 tx=0x7f34c802a480 comp rx=0 tx=0).stop 2024-09-11T22:57:11.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.258+0000 7f34e2fde640 1 -- 172.21.15.152:0/3937154032 shutdown_connections 2024-09-11T22:57:11.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.258+0000 7f34e2fde640 1 --2- 172.21.15.152:0/3937154032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34dc102ef0 0x7f34dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.258+0000 7f34e2fde640 1 --2- 172.21.15.152:0/3937154032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34dc102570 0x7f34dc1029b0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.258+0000 7f34e2fde640 1 --2- 172.21.15.152:0/3937154032 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34dc100e20 0x7f34dc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.258+0000 7f34e2fde640 1 -- 172.21.15.152:0/3937154032 >> 172.21.15.152:0/3937154032 conn(0x7f34dc0fc7d0 msgr2=0x7f34dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:11.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.258+0000 7f34e2fde640 1 -- 172.21.15.152:0/3937154032 shutdown_connections 2024-09-11T22:57:11.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.258+0000 7f34e2fde640 1 -- 172.21.15.152:0/3937154032 wait complete. 2024-09-11T22:57:11.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e2fde640 1 Processor -- start 2024-09-11T22:57:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e2fde640 1 -- start start 2024-09-11T22:57:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e2fde640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34dc100e20 0x7f34dc198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e2fde640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34dc102570 0x7f34dc198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e2fde640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34dc102ef0 0x7f34dc19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e2fde640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc110fa0 con 0x7f34dc100e20 2024-09-11T22:57:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e2fde640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc110e20 con 0x7f34dc102ef0 2024-09-11T22:57:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e2fde640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc111120 con 0x7f34dc102570 2024-09-11T22:57:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e0d7a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34dc100e20 0x7f34dc198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e0d7a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34dc100e20 0x7f34dc198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43708/0 (socket says 172.21.15.152:43708) 2024-09-11T22:57:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e0d7a640 1 -- 172.21.15.152:0/1174145987 learned_addr learned my addr 172.21.15.152:0/1174145987 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e157b640 1 --2- 172.21.15.152:0/1174145987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34dc102ef0 0x7f34dc19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34dbfff640 1 --2- 172.21.15.152:0/1174145987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34dc102570 0x7f34dc198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e0d7a640 1 -- 172.21.15.152:0/1174145987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34dc102570 msgr2=0x7f34dc198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:11.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e0d7a640 1 --2- 172.21.15.152:0/1174145987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34dc102570 0x7f34dc198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e0d7a640 1 -- 172.21.15.152:0/1174145987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34dc102ef0 msgr2=0x7f34dc19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:11.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e0d7a640 1 --2- 172.21.15.152:0/1174145987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34dc102ef0 0x7f34dc19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e0d7a640 1 -- 172.21.15.152:0/1174145987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc19d4f0 con 0x7f34dc100e20 2024-09-11T22:57:11.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.259+0000 7f34e157b640 1 --2- 172.21.15.152:0/1174145987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34dc102ef0 0x7f34dc19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:11.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.260+0000 7f34e0d7a640 1 --2- 172.21.15.152:0/1174145987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34dc100e20 0x7f34dc198540 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f34d000b4a0 tx=0x7f34d000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:11.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.260+0000 7f34d9ffb640 1 -- 172.21.15.152:0/1174145987 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0014070 con 0x7f34dc100e20 2024-09-11T22:57:11.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.260+0000 7f34d9ffb640 1 -- 172.21.15.152:0/1174145987 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d0010af0 con 0x7f34dc100e20 2024-09-11T22:57:11.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.260+0000 7f34e2fde640 1 -- 172.21.15.152:0/1174145987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34dc19d7e0 con 0x7f34dc100e20 2024-09-11T22:57:11.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.260+0000 7f34d9ffb640 1 -- 172.21.15.152:0/1174145987 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0017d10 con 0x7f34dc100e20 2024-09-11T22:57:11.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.260+0000 7f34e2fde640 1 -- 172.21.15.152:0/1174145987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34dc1a50c0 con 0x7f34dc100e20 2024-09-11T22:57:11.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.263+0000 7f34d9ffb640 1 -- 172.21.15.152:0/1174145987 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f34d0017070 con 0x7f34dc100e20 2024-09-11T22:57:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.263+0000 7f34e53b7740 1 Processor -- start 2024-09-11T22:57:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.263+0000 7f34e53b7740 1 -- start start 2024-09-11T22:57:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.263+0000 7f34d9ffb640 1 --2- 172.21.15.152:0/1174145987 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f34cc0a2cf0 0x7f34cc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.263+0000 7f34e53b7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651955404a0 0x565195540880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.263+0000 7f34e53b7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565195540dc0 0x56519554b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.263+0000 7f34e53b7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519554b680 0x56519554da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.263+0000 7f34e53b7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5651954a62e0 con 0x565195540dc0 2024-09-11T22:57:11.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.263+0000 7f34e53b7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56519544dce0 con 0x5651955404a0 2024-09-11T22:57:11.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.263+0000 7f34e53b7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5651954a22a0 con 0x56519554b680 2024-09-11T22:57:11.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34e157b640 1 --2- 172.21.15.152:0/1174145987 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f34cc0a2cf0 0x7f34cc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34d9ffb640 1 -- 172.21.15.152:0/1174145987 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f34d00ce770 con 0x7f34dc100e20 2024-09-11T22:57:11.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34dbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565195540dc0 0x56519554b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34dbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565195540dc0 0x56519554b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43714/0 (socket says 172.21.15.152:43714) 2024-09-11T22:57:11.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34dbfff640 1 -- 172.21.15.152:0/3354778480 learned_addr learned my addr 172.21.15.152:0/3354778480 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:11.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34e0d7a640 1 --2- 172.21.15.152:0/3354778480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651955404a0 0x565195540880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34e157b640 1 --2- 172.21.15.152:0/3354778480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519554b680 0x56519554da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34dbfff640 1 -- 172.21.15.152:0/3354778480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519554b680 msgr2=0x56519554da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:11.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34dbfff640 1 --2- 172.21.15.152:0/3354778480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519554b680 0x56519554da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34dbfff640 1 -- 172.21.15.152:0/3354778480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651955404a0 msgr2=0x565195540880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:11.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34dbfff640 1 --2- 172.21.15.152:0/3354778480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651955404a0 0x565195540880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34dbfff640 1 -- 172.21.15.152:0/3354778480 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565195432440 con 0x565195540dc0 2024-09-11T22:57:11.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34e157b640 1 --2- 172.21.15.152:0/1174145987 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f34cc0a2cf0 0x7f34cc0a51b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f34d40126c0 tx=0x7f34d4012ac0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:11.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34dbfff640 1 --2- 172.21.15.152:0/3354778480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565195540dc0 0x56519554b140 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f34c80314e0 tx=0x7f34c8004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e1a789a3f93205 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:11.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34c6ffd640 1 -- 172.21.15.152:0/3354778480 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8035030 con 0x565195540dc0 2024-09-11T22:57:11.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34c6ffd640 1 -- 172.21.15.152:0/3354778480 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c8003ad0 con 0x565195540dc0 2024-09-11T22:57:11.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.264+0000 7f34c6ffd640 1 -- 172.21.15.152:0/3354778480 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8031bc0 con 0x565195540dc0 2024-09-11T22:57:11.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.265+0000 7f34e53b7740 1 -- 172.21.15.152:0/3354778480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565195540dc0 msgr2=0x56519554b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:11.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.265+0000 7f34e53b7740 1 --2- 172.21.15.152:0/3354778480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565195540dc0 0x56519554b140 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f34c80314e0 tx=0x7f34c8004170 comp rx=0 tx=0).stop 2024-09-11T22:57:11.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.265+0000 7f34e53b7740 1 -- 172.21.15.152:0/3354778480 shutdown_connections 2024-09-11T22:57:11.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.265+0000 7f34e53b7740 1 --2- 172.21.15.152:0/3354778480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56519554b680 0x56519554da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.265+0000 7f34e53b7740 1 --2- 172.21.15.152:0/3354778480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565195540dc0 0x56519554b140 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.265+0000 7f34e53b7740 1 --2- 172.21.15.152:0/3354778480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651955404a0 0x565195540880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.265+0000 7f34e53b7740 1 -- 172.21.15.152:0/3354778480 >> 172.21.15.152:0/3354778480 conn(0x56519553ca30 msgr2=0x56519553ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:11.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.265+0000 7f34e53b7740 1 -- 172.21.15.152:0/3354778480 shutdown_connections 2024-09-11T22:57:11.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.265+0000 7f34e53b7740 1 -- 172.21.15.152:0/3354778480 wait complete. 2024-09-11T22:57:11.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.265+0000 7f34e53b7740 1 -- start start 2024-09-11T22:57:11.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e53b7740 1 Processor -- start 2024-09-11T22:57:11.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e53b7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5651955404a0 0x56519554af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e53b7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565195540dc0 0x565195548220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e53b7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56519554b680 0x565195548760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e53b7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56519554e250 con 0x5651955404a0 2024-09-11T22:57:11.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e53b7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x565195461050 con 0x56519554b680 2024-09-11T22:57:11.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e53b7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56519554e3d0 con 0x565195540dc0 2024-09-11T22:57:11.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e53b7740 10 client.0 _ll_register_callbacks cb 0x5651954a1160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:11.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e0d7a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5651955404a0 0x56519554af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e0d7a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5651955404a0 0x56519554af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43718/0 (socket says 172.21.15.152:43718) 2024-09-11T22:57:11.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e0d7a640 1 -- 172.21.15.152:0/3784195168 learned_addr learned my addr 172.21.15.152:0/3784195168 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:11.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e157b640 1 --2- 172.21.15.152:0/3784195168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56519554b680 0x565195548760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34dbfff640 1 --2- 172.21.15.152:0/3784195168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565195540dc0 0x565195548220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e0d7a640 1 -- 172.21.15.152:0/3784195168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565195540dc0 msgr2=0x565195548220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:11.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e0d7a640 1 --2- 172.21.15.152:0/3784195168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565195540dc0 0x565195548220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e0d7a640 1 -- 172.21.15.152:0/3784195168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56519554b680 msgr2=0x565195548760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:11.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e0d7a640 1 --2- 172.21.15.152:0/3784195168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56519554b680 0x565195548760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34e0d7a640 1 -- 172.21.15.152:0/3784195168 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651952173c0 con 0x5651955404a0 2024-09-11T22:57:11.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.266+0000 7f34dbfff640 1 --2- 172.21.15.152:0/3784195168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565195540dc0 0x565195548220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:11.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.267+0000 7f34e0d7a640 1 --2- 172.21.15.152:0/3784195168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5651955404a0 0x56519554af70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f34d009ad80 tx=0x7f34d00b41d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a22cf58be354a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:11.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.267+0000 7f34bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:11.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.267+0000 7f34bbfff640 1 -- 172.21.15.152:0/3784195168 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d001e070 con 0x5651955404a0 2024-09-11T22:57:11.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.267+0000 7f34bbfff640 1 -- 172.21.15.152:0/3784195168 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f34d0097460 con 0x5651955404a0 2024-09-11T22:57:11.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.267+0000 7f34e53b7740 1 -- 172.21.15.152:0/3784195168 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34c8003ad0 con 0x5651955404a0 2024-09-11T22:57:11.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.267+0000 7f34bbfff640 1 -- 172.21.15.152:0/3784195168 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f34d009adf0 con 0x5651955404a0 2024-09-11T22:57:11.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.267+0000 7f34e53b7740 10 client.16704 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:11.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.267+0000 7f34e53b7740 1 -- 172.21.15.152:0/3784195168 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565195559bc0 con 0x5651955404a0 2024-09-11T22:57:11.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.267+0000 7f34bbfff640 1 -- 172.21.15.152:0/3784195168 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c8003ad0 con 0x5651955404a0 2024-09-11T22:57:11.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.267+0000 7f34bbfff640 1 -- 172.21.15.152:0/3784195168 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f34d0061020 con 0x5651955404a0 2024-09-11T22:57:11.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.267+0000 7f34e53b7740 10 client.16704 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:11.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.267+0000 7f34e53b7740 10 client.16704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:57:11.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.267+0000 7f34e53b7740 20 client.16704 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:11.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.267+0000 7f34e53b7740 1 --2- 172.21.15.152:0/3784195168 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56519555afd0 0x56519555d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.267+0000 7f34e53b7740 4 client.16704 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:57:11.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.267+0000 7f34e53b7740 1 -- 172.21.15.152:0/3784195168 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651954b3850 con 0x56519555afd0 2024-09-11T22:57:11.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.268+0000 7f34dbfff640 1 --2- 172.21.15.152:0/3784195168 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56519555afd0 0x56519555d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.268+0000 7f34bbfff640 10 client.16704 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:11.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.268+0000 7f34dbfff640 1 --2- 172.21.15.152:0/3784195168 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56519555afd0 0x56519555d440 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a4eaf7e5e8cd2b4 server_cookie=d29ad9471df58c0a in_seq=0 out_seq=0 2024-09-11T22:57:11.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.269+0000 7f34bbfff640 1 -- 172.21.15.152:0/3784195168 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651954b3850 con 0x56519555afd0 2024-09-11T22:57:11.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.269+0000 7f34bbfff640 10 client.16704 handle_command_reply: tid=0 2024-09-11T22:57:11.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.269+0000 7f34e53b7740 1 client.16704 shutdown 2024-09-11T22:57:11.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.269+0000 7f34e53b7740 1 -- 172.21.15.152:0/3784195168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5651955404a0 msgr2=0x56519554af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:11.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.269+0000 7f34e53b7740 1 --2- 172.21.15.152:0/3784195168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5651955404a0 0x56519554af70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f34d009ad80 tx=0x7f34d00b41d0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.269+0000 7f34e53b7740 1 -- 172.21.15.152:0/3784195168 shutdown_connections 2024-09-11T22:57:11.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.269+0000 7f34e53b7740 1 --2- 172.21.15.152:0/3784195168 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56519555afd0 0x56519555d440 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.269+0000 7f34e53b7740 1 --2- 172.21.15.152:0/3784195168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56519554b680 0x565195548760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.269+0000 7f34e53b7740 1 --2- 172.21.15.152:0/3784195168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565195540dc0 0x565195548220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.269+0000 7f34bbfff640 0 client.16704 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:11.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.269+0000 7f34e53b7740 1 --2- 172.21.15.152:0/3784195168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5651955404a0 0x56519554af70 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.269+0000 7f34e53b7740 1 -- 172.21.15.152:0/3784195168 >> 172.21.15.152:0/3784195168 conn(0x56519553ca30 msgr2=0x56519553bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:11.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.269+0000 7f34e53b7740 1 -- 172.21.15.152:0/3784195168 shutdown_connections 2024-09-11T22:57:11.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.269+0000 7f34e53b7740 1 -- 172.21.15.152:0/3784195168 wait complete. 2024-09-11T22:57:11.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.269+0000 7f34e53b7740 20 client.16704 trim_cache size 0 max 16384 2024-09-11T22:57:11.304 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:10 smithi204 ceph-mon[35454]: pgmap v131: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-11T22:57:11.304 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:10 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T22:57:11.304 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:10 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:57:11.304 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:10 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T22:57:11.304 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:10 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:57:11.304 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:10 smithi191 ceph-mon[35211]: pgmap v131: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-11T22:57:11.304 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:10 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T22:57:11.305 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:10 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:57:11.305 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:10 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T22:57:11.305 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:10 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:57:11.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.286+0000 7f34e53b7740 1 Processor -- start 2024-09-11T22:57:11.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.286+0000 7f34e53b7740 1 -- start start 2024-09-11T22:57:11.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.286+0000 7f34e53b7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651955404a0 0x5651955574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.286+0000 7f34e53b7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565195540dc0 0x5651955437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.286+0000 7f34e53b7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56519554b680 0x565195562d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.286+0000 7f34e53b7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc1ad6b0 con 0x56519554b680 2024-09-11T22:57:11.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.286+0000 7f34e53b7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc1af1f0 con 0x5651955404a0 2024-09-11T22:57:11.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.286+0000 7f34e53b7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc1add80 con 0x565195540dc0 2024-09-11T22:57:11.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.286+0000 7f34e157b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56519554b680 0x565195562d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.286+0000 7f34e157b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56519554b680 0x565195562d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43722/0 (socket says 172.21.15.152:43722) 2024-09-11T22:57:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.286+0000 7f34e157b640 1 -- 172.21.15.152:0/1833670467 learned_addr learned my addr 172.21.15.152:0/1833670467 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.287+0000 7f34e0d7a640 1 --2- 172.21.15.152:0/1833670467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651955404a0 0x5651955574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.287+0000 7f34dbfff640 1 --2- 172.21.15.152:0/1833670467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565195540dc0 0x5651955437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.287+0000 7f34e157b640 1 -- 172.21.15.152:0/1833670467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565195540dc0 msgr2=0x5651955437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.287+0000 7f34e157b640 1 --2- 172.21.15.152:0/1833670467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565195540dc0 0x5651955437f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.287+0000 7f34e157b640 1 -- 172.21.15.152:0/1833670467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651955404a0 msgr2=0x5651955574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.287+0000 7f34e157b640 1 --2- 172.21.15.152:0/1833670467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651955404a0 0x5651955574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.287+0000 7f34e157b640 1 -- 172.21.15.152:0/1833670467 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56519553d6c0 con 0x56519554b680 2024-09-11T22:57:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.287+0000 7f34e157b640 1 --2- 172.21.15.152:0/1833670467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56519554b680 0x565195562d00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f34d4010710 tx=0x7f34d401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3fd8464c236524 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.287+0000 7f34c67fc640 1 -- 172.21.15.152:0/1833670467 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4004d20 con 0x56519554b680 2024-09-11T22:57:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.287+0000 7f34c67fc640 1 -- 172.21.15.152:0/1833670467 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d4027430 con 0x56519554b680 2024-09-11T22:57:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.287+0000 7f34c67fc640 1 -- 172.21.15.152:0/1833670467 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4005280 con 0x56519554b680 2024-09-11T22:57:11.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.288+0000 7f34e53b7740 1 -- 172.21.15.152:0/1833670467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56519554b680 msgr2=0x565195562d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.288+0000 7f34e53b7740 1 --2- 172.21.15.152:0/1833670467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56519554b680 0x565195562d00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f34d4010710 tx=0x7f34d401c000 comp rx=0 tx=0).stop 2024-09-11T22:57:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.288+0000 7f34e53b7740 1 -- 172.21.15.152:0/1833670467 shutdown_connections 2024-09-11T22:57:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.288+0000 7f34e53b7740 1 --2- 172.21.15.152:0/1833670467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56519554b680 0x565195562d00 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.288+0000 7f34e53b7740 1 --2- 172.21.15.152:0/1833670467 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565195540dc0 0x5651955437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.288+0000 7f34e53b7740 1 --2- 172.21.15.152:0/1833670467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651955404a0 0x5651955574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.288+0000 7f34e53b7740 1 -- 172.21.15.152:0/1833670467 >> 172.21.15.152:0/1833670467 conn(0x56519553ca30 msgr2=0x5651955430e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.288+0000 7f34e53b7740 1 -- 172.21.15.152:0/1833670467 shutdown_connections 2024-09-11T22:57:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.288+0000 7f34e53b7740 1 -- 172.21.15.152:0/1833670467 wait complete. 2024-09-11T22:57:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.288+0000 7f34e53b7740 1 -- start start 2024-09-11T22:57:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.288+0000 7f34e53b7740 1 Processor -- start 2024-09-11T22:57:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e53b7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651955404a0 0x565195553a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e53b7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565195540dc0 0x56519554ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e53b7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56519554b680 0x56519554f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e53b7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc1aeb20 con 0x56519554b680 2024-09-11T22:57:11.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e53b7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc1ae450 con 0x5651955404a0 2024-09-11T22:57:11.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e53b7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc1ab670 con 0x565195540dc0 2024-09-11T22:57:11.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e53b7740 10 client.0 _ll_register_callbacks cb 0x7f34d0097110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:11.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e157b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56519554b680 0x56519554f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e157b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56519554b680 0x56519554f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43732/0 (socket says 172.21.15.152:43732) 2024-09-11T22:57:11.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e157b640 1 -- 172.21.15.152:0/3932619769 learned_addr learned my addr 172.21.15.152:0/3932619769 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:11.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34dbfff640 1 --2- 172.21.15.152:0/3932619769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565195540dc0 0x56519554ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e0d7a640 1 --2- 172.21.15.152:0/3932619769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651955404a0 0x565195553a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e157b640 1 -- 172.21.15.152:0/3932619769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565195540dc0 msgr2=0x56519554ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:11.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e157b640 1 --2- 172.21.15.152:0/3932619769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565195540dc0 0x56519554ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e157b640 1 -- 172.21.15.152:0/3932619769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651955404a0 msgr2=0x565195553a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:11.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e157b640 1 --2- 172.21.15.152:0/3932619769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651955404a0 0x565195553a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e157b640 1 -- 172.21.15.152:0/3932619769 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56519554f800 con 0x56519554b680 2024-09-11T22:57:11.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34e157b640 1 --2- 172.21.15.152:0/3932619769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56519554b680 0x56519554f2c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f34d401fc40 tx=0x7f34d401fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cf698daef93caf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:11.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:11.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.289+0000 7f34bbfff640 1 -- 172.21.15.152:0/3932619769 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d402b050 con 0x56519554b680 2024-09-11T22:57:11.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.290+0000 7f34bbfff640 1 -- 172.21.15.152:0/3932619769 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f34d40279f0 con 0x56519554b680 2024-09-11T22:57:11.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.290+0000 7f34e53b7740 1 -- 172.21.15.152:0/3932619769 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34d4027430 con 0x56519554b680 2024-09-11T22:57:11.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.290+0000 7f34bbfff640 1 -- 172.21.15.152:0/3932619769 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f34d4010800 con 0x56519554b680 2024-09-11T22:57:11.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.290+0000 7f34e53b7740 10 client.16710 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:11.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.290+0000 7f34e53b7740 1 -- 172.21.15.152:0/3932619769 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5651955c59d0 con 0x56519554b680 2024-09-11T22:57:11.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.290+0000 7f34bbfff640 1 -- 172.21.15.152:0/3932619769 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d4027430 con 0x56519554b680 2024-09-11T22:57:11.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.290+0000 7f34bbfff640 1 -- 172.21.15.152:0/3932619769 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f34d40305a0 con 0x56519554b680 2024-09-11T22:57:11.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.290+0000 7f34e53b7740 10 client.16710 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:11.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.290+0000 7f34e53b7740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:57:11.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.290+0000 7f34e53b7740 20 client.16710 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:11.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.290+0000 7f34e53b7740 1 --2- 172.21.15.152:0/3932619769 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5651955c60e0 0x5651955c8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:11.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.290+0000 7f34e53b7740 4 client.16710 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:57:11.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.290+0000 7f34e53b7740 1 -- 172.21.15.152:0/3932619769 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565195548af0 con 0x5651955c60e0 2024-09-11T22:57:11.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.290+0000 7f34e0d7a640 1 --2- 172.21.15.152:0/3932619769 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5651955c60e0 0x5651955c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:11.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.291+0000 7f34bbfff640 10 client.16710 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:11.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.291+0000 7f34e0d7a640 1 --2- 172.21.15.152:0/3932619769 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5651955c60e0 0x5651955c8500 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=676a1e2132194adc server_cookie=83b79a2986f81f26 in_seq=0 out_seq=0 2024-09-11T22:57:11.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.291+0000 7f34bbfff640 1 -- 172.21.15.152:0/3932619769 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x565195548af0 con 0x5651955c60e0 2024-09-11T22:57:11.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.291+0000 7f34bbfff640 10 client.16710 handle_command_reply: tid=0 2024-09-11T22:57:11.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.291+0000 7f34e53b7740 1 client.16710 shutdown 2024-09-11T22:57:11.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.291+0000 7f34e53b7740 1 -- 172.21.15.152:0/3932619769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56519554b680 msgr2=0x56519554f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:11.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.291+0000 7f34e53b7740 1 --2- 172.21.15.152:0/3932619769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56519554b680 0x56519554f2c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f34d401fc40 tx=0x7f34d401fc70 comp rx=0 tx=0).stop 2024-09-11T22:57:11.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.291+0000 7f34e53b7740 1 -- 172.21.15.152:0/3932619769 shutdown_connections 2024-09-11T22:57:11.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.291+0000 7f34e53b7740 1 --2- 172.21.15.152:0/3932619769 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56519554b680 0x56519554f2c0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.291+0000 7f34e53b7740 1 --2- 172.21.15.152:0/3932619769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565195540dc0 0x56519554ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.291+0000 7f34e53b7740 1 --2- 172.21.15.152:0/3932619769 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5651955c60e0 0x5651955c8500 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.291+0000 7f34e53b7740 1 --2- 172.21.15.152:0/3932619769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5651955404a0 0x565195553a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.291+0000 7f34e53b7740 1 -- 172.21.15.152:0/3932619769 >> 172.21.15.152:0/3932619769 conn(0x56519553ca30 msgr2=0x565195542710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:11.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.291+0000 7f34bbfff640 0 client.16710 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:11.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.291+0000 7f34e53b7740 1 -- 172.21.15.152:0/3932619769 shutdown_connections 2024-09-11T22:57:11.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.291+0000 7f34e53b7740 1 -- 172.21.15.152:0/3932619769 wait complete. 2024-09-11T22:57:11.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.292+0000 7f34e53b7740 20 client.16710 trim_cache size 0 max 16384 2024-09-11T22:57:11.318 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:57:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:57:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": { 2024-09-11T22:57:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "a6c9993c-c126-4270-9467-4ce8b93e2083": { 2024-09-11T22:57:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "path": "/", 2024-09-11T22:57:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "tag": "a6c9993c-c126-4270-9467-4ce8b93e2083", 2024-09-11T22:57:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "options": "recursive,force" 2024-09-11T22:57:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:57:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:57:11.318 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:57:11.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.292+0000 7f34e2fde640 1 -- 172.21.15.152:0/1174145987 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f34cc0a2cf0 msgr2=0x7f34cc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:11.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.292+0000 7f34e2fde640 1 --2- 172.21.15.152:0/1174145987 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f34cc0a2cf0 0x7f34cc0a51b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f34d40126c0 tx=0x7f34d4012ac0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.292+0000 7f34e2fde640 1 -- 172.21.15.152:0/1174145987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34dc100e20 msgr2=0x7f34dc198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:11.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.292+0000 7f34e2fde640 1 --2- 172.21.15.152:0/1174145987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34dc100e20 0x7f34dc198540 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f34d000b4a0 tx=0x7f34d000b960 comp rx=0 tx=0).stop 2024-09-11T22:57:11.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.292+0000 7f34e2fde640 1 -- 172.21.15.152:0/1174145987 shutdown_connections 2024-09-11T22:57:11.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.292+0000 7f34e2fde640 1 --2- 172.21.15.152:0/1174145987 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f34cc0a2cf0 0x7f34cc0a51b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.292+0000 7f34e2fde640 1 --2- 172.21.15.152:0/1174145987 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34dc102ef0 0x7f34dc19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.292+0000 7f34e2fde640 1 --2- 172.21.15.152:0/1174145987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34dc102570 0x7f34dc198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.292+0000 7f34e2fde640 1 --2- 172.21.15.152:0/1174145987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f34dc100e20 0x7f34dc198540 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:11.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.292+0000 7f34e2fde640 1 -- 172.21.15.152:0/1174145987 >> 172.21.15.152:0/1174145987 conn(0x7f34dc0fc7d0 msgr2=0x7f34dc069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:11.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.292+0000 7f34e2fde640 1 -- 172.21.15.152:0/1174145987 shutdown_connections 2024-09-11T22:57:11.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:11.292+0000 7f34e2fde640 1 -- 172.21.15.152:0/1174145987 wait complete. 2024-09-11T22:57:12.103 INFO:tasks.cephfs.filesystem:scrub status for tag:a6c9993c-c126-4270-9467-4ce8b93e2083 - {'path': '/', 'tag': 'a6c9993c-c126-4270-9467-4ce8b93e2083', 'options': 'recursive,force'} 2024-09-11T22:57:13.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:12 smithi191 ceph-mon[35211]: pgmap v132: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-11T22:57:13.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:12 smithi204 ceph-mon[35454]: pgmap v132: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-11T22:57:13.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:12 smithi152 ceph-mon[31181]: pgmap v132: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-11T22:57:14.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:13 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T22:57:14.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:13 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T22:57:14.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:13 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T22:57:15.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:14 smithi191 ceph-mon[35211]: pgmap v133: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-11T22:57:15.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:14 smithi204 ceph-mon[35454]: pgmap v133: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-11T22:57:15.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:14 smithi152 ceph-mon[31181]: pgmap v133: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-11T22:57:17.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:16 smithi191 ceph-mon[35211]: pgmap v134: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-11T22:57:17.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:16 smithi204 ceph-mon[35454]: pgmap v134: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-11T22:57:17.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:16 smithi152 ceph-mon[31181]: pgmap v134: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-11T22:57:19.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:18 smithi191 ceph-mon[35211]: pgmap v135: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-11T22:57:19.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:18 smithi204 ceph-mon[35454]: pgmap v135: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-11T22:57:19.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:18 smithi152 ceph-mon[31181]: pgmap v135: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-11T22:57:21.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:20 smithi191 ceph-mon[35211]: pgmap v136: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-11T22:57:21.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:20 smithi204 ceph-mon[35454]: pgmap v136: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-11T22:57:21.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:20 smithi152 ceph-mon[31181]: pgmap v136: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-11T22:57:23.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:22 smithi191 ceph-mon[35211]: pgmap v137: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-11T22:57:23.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:22 smithi204 ceph-mon[35454]: pgmap v137: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-11T22:57:23.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:22 smithi152 ceph-mon[31181]: pgmap v137: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-11T22:57:25.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:24 smithi191 ceph-mon[35211]: pgmap v138: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:57:25.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:24 smithi204 ceph-mon[35454]: pgmap v138: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:57:25.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:24 smithi152 ceph-mon[31181]: pgmap v138: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:57:27.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:26 smithi191 ceph-mon[35211]: pgmap v139: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:57:27.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:26 smithi204 ceph-mon[35454]: pgmap v139: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:57:27.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:26 smithi152 ceph-mon[31181]: pgmap v139: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:57:29.302 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:28 smithi191 ceph-mon[35211]: pgmap v140: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:29.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:28 smithi152 ceph-mon[31181]: pgmap v140: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:29.501 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:28 smithi204 ceph-mon[35454]: pgmap v140: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:31.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:30 smithi191 ceph-mon[35211]: pgmap v141: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:57:31.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:30 smithi204 ceph-mon[35454]: pgmap v141: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:57:31.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:30 smithi152 ceph-mon[31181]: pgmap v141: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:57:33.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:32 smithi191 ceph-mon[35211]: pgmap v142: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:57:33.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:32 smithi204 ceph-mon[35454]: pgmap v142: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:57:33.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:32 smithi152 ceph-mon[31181]: pgmap v142: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:57:35.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:34 smithi191 ceph-mon[35211]: pgmap v143: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:35.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:34 smithi204 ceph-mon[35454]: pgmap v143: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:35.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:34 smithi152 ceph-mon[31181]: pgmap v143: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:37.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:36 smithi191 ceph-mon[35211]: pgmap v144: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:57:37.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:36 smithi204 ceph-mon[35454]: pgmap v144: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:57:37.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:36 smithi152 ceph-mon[31181]: pgmap v144: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:57:39.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:38 smithi191 ceph-mon[35211]: pgmap v145: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:39.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:38 smithi204 ceph-mon[35454]: pgmap v145: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:39.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:38 smithi152 ceph-mon[31181]: pgmap v145: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:41.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:40 smithi191 ceph-mon[35211]: pgmap v146: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:57:41.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:40 smithi204 ceph-mon[35454]: pgmap v146: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:57:41.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:40 smithi152 ceph-mon[31181]: pgmap v146: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:57:42.104 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:57:42.375 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:57:43.002 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:42 smithi152 ceph-mon[31181]: pgmap v147: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:57:43.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.994+0000 7fbc72dc6640 1 -- 172.21.15.152:0/2138326696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc6c100480 msgr2=0x7fbc6c100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:43.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.994+0000 7fbc72dc6640 1 --2- 172.21.15.152:0/2138326696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc6c100480 0x7fbc6c100860 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000aa10 tx=0x7fbc6002b720 comp rx=0 tx=0).stop 2024-09-11T22:57:43.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.994+0000 7fbc72dc6640 1 -- 172.21.15.152:0/2138326696 shutdown_connections 2024-09-11T22:57:43.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.994+0000 7fbc72dc6640 1 --2- 172.21.15.152:0/2138326696 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc6c1055b0 0x7fbc6c10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.994+0000 7fbc72dc6640 1 --2- 172.21.15.152:0/2138326696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc6c100e30 0x7fbc6c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.994+0000 7fbc72dc6640 1 --2- 172.21.15.152:0/2138326696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc6c100480 0x7fbc6c100860 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.994+0000 7fbc72dc6640 1 -- 172.21.15.152:0/2138326696 >> 172.21.15.152:0/2138326696 conn(0x7fbc6c077210 msgr2=0x7fbc6c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:43.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.994+0000 7fbc72dc6640 1 -- 172.21.15.152:0/2138326696 shutdown_connections 2024-09-11T22:57:43.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.994+0000 7fbc72dc6640 1 -- 172.21.15.152:0/2138326696 wait complete. 2024-09-11T22:57:43.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc72dc6640 1 Processor -- start 2024-09-11T22:57:43.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc72dc6640 1 -- start start 2024-09-11T22:57:43.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc72dc6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc6c100480 0x7fbc6c1963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc72dc6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc6c100e30 0x7fbc6c196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc72dc6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc6c1055b0 0x7fbc6c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc72dc6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c10fe30 con 0x7fbc6c1055b0 2024-09-11T22:57:43.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc72dc6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c10fcb0 con 0x7fbc6c100e30 2024-09-11T22:57:43.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc72dc6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c10ffb0 con 0x7fbc6c100480 2024-09-11T22:57:43.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc71363640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc6c1055b0 0x7fbc6c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc71363640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc6c1055b0 0x7fbc6c19aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45656/0 (socket says 172.21.15.152:45656) 2024-09-11T22:57:43.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc71363640 1 -- 172.21.15.152:0/1670017131 learned_addr learned my addr 172.21.15.152:0/1670017131 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:43.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc70b62640 1 --2- 172.21.15.152:0/1670017131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc6c100480 0x7fbc6c1963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/1670017131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc6c100e30 0x7fbc6c196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc71363640 1 -- 172.21.15.152:0/1670017131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc6c100480 msgr2=0x7fbc6c1963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:43.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc71363640 1 --2- 172.21.15.152:0/1670017131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc6c100480 0x7fbc6c1963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc71363640 1 -- 172.21.15.152:0/1670017131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc6c100e30 msgr2=0x7fbc6c196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:43.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc71363640 1 --2- 172.21.15.152:0/1670017131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc6c100e30 0x7fbc6c196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc71363640 1 -- 172.21.15.152:0/1670017131 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c19b380 con 0x7fbc6c1055b0 2024-09-11T22:57:43.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/1670017131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc6c100e30 0x7fbc6c196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:57:43.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.995+0000 7fbc70b62640 1 --2- 172.21.15.152:0/1670017131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc6c100480 0x7fbc6c1963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:43.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.996+0000 7fbc71363640 1 --2- 172.21.15.152:0/1670017131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc6c1055b0 0x7fbc6c19aca0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400b970 tx=0x7fbc6400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:43.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.996+0000 7fbc69ffb640 1 -- 172.21.15.152:0/1670017131 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64019070 con 0x7fbc6c1055b0 2024-09-11T22:57:43.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.996+0000 7fbc72dc6640 1 -- 172.21.15.152:0/1670017131 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6c19b670 con 0x7fbc6c1055b0 2024-09-11T22:57:43.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.996+0000 7fbc69ffb640 1 -- 172.21.15.152:0/1670017131 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc6400cb40 con 0x7fbc6c1055b0 2024-09-11T22:57:43.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.996+0000 7fbc69ffb640 1 -- 172.21.15.152:0/1670017131 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64007650 con 0x7fbc6c1055b0 2024-09-11T22:57:43.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.996+0000 7fbc72dc6640 1 -- 172.21.15.152:0/1670017131 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6c1a2f50 con 0x7fbc6c1055b0 2024-09-11T22:57:43.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.998+0000 7fbc69ffb640 1 -- 172.21.15.152:0/1670017131 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fbc6401c070 con 0x7fbc6c1055b0 2024-09-11T22:57:43.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc7519f740 1 Processor -- start 2024-09-11T22:57:43.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc7519f740 1 -- start start 2024-09-11T22:57:43.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc7519f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1df360 0x55e62b1df740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc7519f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1dfc80 0x55e62b1ec350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc7519f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1ec890 0x55e62b1eec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc7519f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55e62b1452e0 con 0x55e62b1ec890 2024-09-11T22:57:43.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc7519f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e62b0ecce0 con 0x55e62b1dfc80 2024-09-11T22:57:43.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc7519f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e62b1412a0 con 0x55e62b1df360 2024-09-11T22:57:43.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc6bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1ec890 0x55e62b1eec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc6bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1df360 0x55e62b1df740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc69ffb640 1 --2- 172.21.15.152:0/1670017131 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbc5c0a2cf0 0x7fbc5c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc6bfff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1df360 0x55e62b1df740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:56346/0 (socket says 172.21.15.152:56346) 2024-09-11T22:57:43.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc6bfff640 1 -- 172.21.15.152:0/2125047152 learned_addr learned my addr 172.21.15.152:0/2125047152 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:43.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc70b62640 1 --2- 172.21.15.152:0/2125047152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1dfc80 0x55e62b1ec350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc71363640 1 --2- 172.21.15.152:0/1670017131 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbc5c0a2cf0 0x7fbc5c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc69ffb640 1 -- 172.21.15.152:0/1670017131 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fbc640ce980 con 0x7fbc6c1055b0 2024-09-11T22:57:43.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc6bfff640 1 -- 172.21.15.152:0/2125047152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1dfc80 msgr2=0x55e62b1ec350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:43.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/2125047152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1dfc80 0x55e62b1ec350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc6bfff640 1 -- 172.21.15.152:0/2125047152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1ec890 msgr2=0x55e62b1eec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:43.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/2125047152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1ec890 0x55e62b1eec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:42.999+0000 7fbc6bfff640 1 -- 172.21.15.152:0/2125047152 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e62b0d1440 con 0x55e62b1df360 2024-09-11T22:57:43.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.000+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/2125047152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1df360 0x55e62b1df740 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbc58015620 tx=0x7fbc58006bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eedd04db25633f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:43.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.000+0000 7fbc71363640 1 --2- 172.21.15.152:0/1670017131 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbc5c0a2cf0 0x7fbc5c0a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbc6409ad00 tx=0x7fbc64090490 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:43.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.000+0000 7fbc56ffd640 1 -- 172.21.15.152:0/2125047152 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5800f040 con 0x55e62b1df360 2024-09-11T22:57:43.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.000+0000 7fbc56ffd640 1 -- 172.21.15.152:0/2125047152 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc5801c070 con 0x55e62b1df360 2024-09-11T22:57:43.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.000+0000 7fbc56ffd640 1 -- 172.21.15.152:0/2125047152 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc580116c0 con 0x55e62b1df360 2024-09-11T22:57:43.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.001+0000 7fbc7519f740 1 -- 172.21.15.152:0/2125047152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1df360 msgr2=0x55e62b1df740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:43.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.001+0000 7fbc7519f740 1 --2- 172.21.15.152:0/2125047152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1df360 0x55e62b1df740 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbc58015620 tx=0x7fbc58006bf0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.001+0000 7fbc7519f740 1 -- 172.21.15.152:0/2125047152 shutdown_connections 2024-09-11T22:57:43.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.001+0000 7fbc7519f740 1 --2- 172.21.15.152:0/2125047152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1ec890 0x55e62b1eec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.001+0000 7fbc7519f740 1 --2- 172.21.15.152:0/2125047152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1dfc80 0x55e62b1ec350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.001+0000 7fbc7519f740 1 --2- 172.21.15.152:0/2125047152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1df360 0x55e62b1df740 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.001+0000 7fbc7519f740 1 -- 172.21.15.152:0/2125047152 >> 172.21.15.152:0/2125047152 conn(0x55e62b1dba30 msgr2=0x55e62b1dbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:43.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.001+0000 7fbc7519f740 1 -- 172.21.15.152:0/2125047152 shutdown_connections 2024-09-11T22:57:43.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.001+0000 7fbc7519f740 1 -- 172.21.15.152:0/2125047152 wait complete. 2024-09-11T22:57:43.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.001+0000 7fbc7519f740 1 -- start start 2024-09-11T22:57:43.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.001+0000 7fbc7519f740 1 Processor -- start 2024-09-11T22:57:43.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc7519f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1df360 0x55e62b1ec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc7519f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1dfc80 0x55e62b1e9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc7519f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1ec890 0x55e62b1e99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc7519f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55e62b1ef400 con 0x55e62b1dfc80 2024-09-11T22:57:43.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc7519f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e62b100050 con 0x55e62b1df360 2024-09-11T22:57:43.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc7519f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e62b1ef580 con 0x55e62b1ec890 2024-09-11T22:57:43.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc7519f740 10 client.0 _ll_register_callbacks cb 0x55e62b140160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:43.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc70b62640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1dfc80 0x55e62b1e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc70b62640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1dfc80 0x55e62b1e9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45684/0 (socket says 172.21.15.152:45684) 2024-09-11T22:57:43.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc70b62640 1 -- 172.21.15.152:0/1462862522 learned_addr learned my addr 172.21.15.152:0/1462862522 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:43.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/1462862522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1df360 0x55e62b1ec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/1462862522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1ec890 0x55e62b1e99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc70b62640 1 -- 172.21.15.152:0/1462862522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1ec890 msgr2=0x55e62b1e99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:43.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc70b62640 1 --2- 172.21.15.152:0/1462862522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1ec890 0x55e62b1e99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc70b62640 1 -- 172.21.15.152:0/1462862522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1df360 msgr2=0x55e62b1ec1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:43.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc70b62640 1 --2- 172.21.15.152:0/1462862522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1df360 0x55e62b1ec1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc70b62640 1 -- 172.21.15.152:0/1462862522 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e62aeb63c0 con 0x55e62b1dfc80 2024-09-11T22:57:43.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/1462862522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1df360 0x55e62b1ec1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:43.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc70b62640 1 --2- 172.21.15.152:0/1462862522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1dfc80 0x55e62b1e9490 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbc6002bc40 tx=0x7fbc600308f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f62dc84296beda6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:43.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.002+0000 7fbc4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:43.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.003+0000 7fbc4bfff640 1 -- 172.21.15.152:0/1462862522 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60041070 con 0x55e62b1dfc80 2024-09-11T22:57:43.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.003+0000 7fbc4bfff640 1 -- 172.21.15.152:0/1462862522 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fbc60004250 con 0x55e62b1dfc80 2024-09-11T22:57:43.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.003+0000 7fbc7519f740 1 -- 172.21.15.152:0/1462862522 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc5801c070 con 0x55e62b1dfc80 2024-09-11T22:57:43.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.003+0000 7fbc4bfff640 1 -- 172.21.15.152:0/1462862522 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc60049020 con 0x55e62b1dfc80 2024-09-11T22:57:43.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.003+0000 7fbc7519f740 10 client.16722 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:43.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.003+0000 7fbc7519f740 1 -- 172.21.15.152:0/1462862522 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e62b1fae10 con 0x55e62b1dfc80 2024-09-11T22:57:43.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.003+0000 7fbc4bfff640 1 -- 172.21.15.152:0/1462862522 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc5801c070 con 0x55e62b1dfc80 2024-09-11T22:57:43.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.003+0000 7fbc4bfff640 1 -- 172.21.15.152:0/1462862522 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbc6000b040 con 0x55e62b1dfc80 2024-09-11T22:57:43.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.003+0000 7fbc7519f740 10 client.16722 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:43.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.003+0000 7fbc7519f740 10 client.16722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:57:43.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.003+0000 7fbc7519f740 20 client.16722 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:43.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.003+0000 7fbc7519f740 1 --2- 172.21.15.152:0/1462862522 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55e62b1fc270 0x55e62b1fe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.003+0000 7fbc7519f740 4 client.16722 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:57:43.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.003+0000 7fbc7519f740 1 -- 172.21.15.152:0/1462862522 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e62b152850 con 0x55e62b1fc270 2024-09-11T22:57:43.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.004+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/1462862522 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55e62b1fc270 0x55e62b1fe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.004+0000 7fbc4bfff640 10 client.16722 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:43.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.004+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/1462862522 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55e62b1fc270 0x55e62b1fe6b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d997fe7a2d2bae99 server_cookie=bbe9a21e5f814642 in_seq=0 out_seq=0 2024-09-11T22:57:43.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.005+0000 7fbc4bfff640 1 -- 172.21.15.152:0/1462862522 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e62b152850 con 0x55e62b1fc270 2024-09-11T22:57:43.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.005+0000 7fbc4bfff640 10 client.16722 handle_command_reply: tid=0 2024-09-11T22:57:43.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.005+0000 7fbc7519f740 1 client.16722 shutdown 2024-09-11T22:57:43.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.005+0000 7fbc7519f740 1 -- 172.21.15.152:0/1462862522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1dfc80 msgr2=0x55e62b1e9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:43.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.005+0000 7fbc7519f740 1 --2- 172.21.15.152:0/1462862522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1dfc80 0x55e62b1e9490 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbc6002bc40 tx=0x7fbc600308f0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.005+0000 7fbc7519f740 1 -- 172.21.15.152:0/1462862522 shutdown_connections 2024-09-11T22:57:43.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.005+0000 7fbc7519f740 1 --2- 172.21.15.152:0/1462862522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1ec890 0x55e62b1e99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.005+0000 7fbc7519f740 1 --2- 172.21.15.152:0/1462862522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1dfc80 0x55e62b1e9490 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.005+0000 7fbc7519f740 1 --2- 172.21.15.152:0/1462862522 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55e62b1fc270 0x55e62b1fe6b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.005+0000 7fbc7519f740 1 --2- 172.21.15.152:0/1462862522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1df360 0x55e62b1ec1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.005+0000 7fbc4bfff640 0 client.16722 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:43.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.005+0000 7fbc7519f740 1 -- 172.21.15.152:0/1462862522 >> 172.21.15.152:0/1462862522 conn(0x55e62b1dba30 msgr2=0x55e62b1e1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:43.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.005+0000 7fbc7519f740 1 -- 172.21.15.152:0/1462862522 shutdown_connections 2024-09-11T22:57:43.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.005+0000 7fbc7519f740 1 -- 172.21.15.152:0/1462862522 wait complete. 2024-09-11T22:57:43.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.005+0000 7fbc7519f740 20 client.16722 trim_cache size 0 max 16384 2024-09-11T22:57:43.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc7519f740 1 Processor -- start 2024-09-11T22:57:43.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc7519f740 1 -- start start 2024-09-11T22:57:43.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc7519f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1df360 0x55e62b1f86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc7519f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1dfc80 0x55e62b203f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc7519f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1ec890 0x55e62b2044c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc7519f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1ab6c0 con 0x55e62b1df360 2024-09-11T22:57:43.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc7519f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1ad200 con 0x55e62b1ec890 2024-09-11T22:57:43.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc7519f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1abd90 con 0x55e62b1dfc80 2024-09-11T22:57:43.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc6bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1df360 0x55e62b1f86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc6bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1df360 0x55e62b1f86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45688/0 (socket says 172.21.15.152:45688) 2024-09-11T22:57:43.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc6bfff640 1 -- 172.21.15.152:0/3665315773 learned_addr learned my addr 172.21.15.152:0/3665315773 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:43.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc70b62640 1 --2- 172.21.15.152:0/3665315773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1dfc80 0x55e62b203f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/3665315773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1ec890 0x55e62b2044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc6bfff640 1 -- 172.21.15.152:0/3665315773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1dfc80 msgr2=0x55e62b203f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:43.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/3665315773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1dfc80 0x55e62b203f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc6bfff640 1 -- 172.21.15.152:0/3665315773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1ec890 msgr2=0x55e62b2044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:43.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/3665315773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1ec890 0x55e62b2044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.023+0000 7fbc6bfff640 1 -- 172.21.15.152:0/3665315773 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e62b1ef020 con 0x55e62b1df360 2024-09-11T22:57:43.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.024+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/3665315773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1df360 0x55e62b1f86a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbc58017fd0 tx=0x7fbc5800b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6194d4cb5e1d94d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:43.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.024+0000 7fbc567fc640 1 -- 172.21.15.152:0/3665315773 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58017820 con 0x55e62b1df360 2024-09-11T22:57:43.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.024+0000 7fbc567fc640 1 -- 172.21.15.152:0/3665315773 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc5801c040 con 0x55e62b1df360 2024-09-11T22:57:43.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.024+0000 7fbc567fc640 1 -- 172.21.15.152:0/3665315773 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58022d80 con 0x55e62b1df360 2024-09-11T22:57:43.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.024+0000 7fbc7519f740 1 -- 172.21.15.152:0/3665315773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1df360 msgr2=0x55e62b1f86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:43.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.024+0000 7fbc7519f740 1 --2- 172.21.15.152:0/3665315773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1df360 0x55e62b1f86a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbc58017fd0 tx=0x7fbc5800b930 comp rx=0 tx=0).stop 2024-09-11T22:57:43.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.024+0000 7fbc7519f740 1 -- 172.21.15.152:0/3665315773 shutdown_connections 2024-09-11T22:57:43.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.024+0000 7fbc7519f740 1 --2- 172.21.15.152:0/3665315773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1ec890 0x55e62b2044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.024+0000 7fbc7519f740 1 --2- 172.21.15.152:0/3665315773 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1dfc80 0x55e62b203f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.024+0000 7fbc7519f740 1 --2- 172.21.15.152:0/3665315773 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1df360 0x55e62b1f86a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.024+0000 7fbc7519f740 1 -- 172.21.15.152:0/3665315773 >> 172.21.15.152:0/3665315773 conn(0x55e62b1dba30 msgr2=0x55e62b1e4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:43.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.024+0000 7fbc7519f740 1 -- 172.21.15.152:0/3665315773 shutdown_connections 2024-09-11T22:57:43.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.024+0000 7fbc7519f740 1 -- 172.21.15.152:0/3665315773 wait complete. 2024-09-11T22:57:43.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.025+0000 7fbc7519f740 1 -- start start 2024-09-11T22:57:43.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.025+0000 7fbc7519f740 1 Processor -- start 2024-09-11T22:57:43.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.025+0000 7fbc7519f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1df360 0x55e62b1f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.025+0000 7fbc7519f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1dfc80 0x55e62b1effc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.025+0000 7fbc7519f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1ec890 0x55e62b1f0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.025+0000 7fbc7519f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1acb30 con 0x55e62b1dfc80 2024-09-11T22:57:43.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.025+0000 7fbc7519f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1ac460 con 0x55e62b1ec890 2024-09-11T22:57:43.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.025+0000 7fbc7519f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c1a9680 con 0x55e62b1df360 2024-09-11T22:57:43.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.025+0000 7fbc7519f740 10 client.0 _ll_register_callbacks cb 0x7fbc60030e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:43.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.025+0000 7fbc70b62640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1dfc80 0x55e62b1effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.025+0000 7fbc70b62640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1dfc80 0x55e62b1effc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45694/0 (socket says 172.21.15.152:45694) 2024-09-11T22:57:43.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.025+0000 7fbc70b62640 1 -- 172.21.15.152:0/2872015226 learned_addr learned my addr 172.21.15.152:0/2872015226 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:43.049 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:57:43.049 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T22:57:43.049 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T22:57:43.049 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:57:43.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.025+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/2872015226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1df360 0x55e62b1f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/2872015226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1ec890 0x55e62b1f0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc70b62640 1 -- 172.21.15.152:0/2872015226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1df360 msgr2=0x55e62b1f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:43.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc70b62640 1 --2- 172.21.15.152:0/2872015226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1df360 0x55e62b1f4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc70b62640 1 -- 172.21.15.152:0/2872015226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1ec890 msgr2=0x55e62b1f0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:43.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc70b62640 1 --2- 172.21.15.152:0/2872015226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1ec890 0x55e62b1f0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc70b62640 1 -- 172.21.15.152:0/2872015226 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e62b1f0a40 con 0x55e62b1dfc80 2024-09-11T22:57:43.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/2872015226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1ec890 0x55e62b1f0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:57:43.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc70b62640 1 --2- 172.21.15.152:0/2872015226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1dfc80 0x55e62b1effc0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000a960 tx=0x7fbc6000c2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b888b5cd5070b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:43.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:43.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc4bfff640 1 -- 172.21.15.152:0/2872015226 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60041070 con 0x55e62b1dfc80 2024-09-11T22:57:43.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc4bfff640 1 -- 172.21.15.152:0/2872015226 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fbc60004250 con 0x55e62b1dfc80 2024-09-11T22:57:43.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc7519f740 1 -- 172.21.15.152:0/2872015226 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc5801c040 con 0x55e62b1dfc80 2024-09-11T22:57:43.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc4bfff640 1 -- 172.21.15.152:0/2872015226 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc60060050 con 0x55e62b1dfc80 2024-09-11T22:57:43.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc7519f740 10 client.16728 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:43.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc7519f740 1 -- 172.21.15.152:0/2872015226 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e62b266d20 con 0x55e62b1dfc80 2024-09-11T22:57:43.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc4bfff640 1 -- 172.21.15.152:0/2872015226 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc5801c040 con 0x55e62b1dfc80 2024-09-11T22:57:43.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc4bfff640 1 -- 172.21.15.152:0/2872015226 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbc60003a00 con 0x55e62b1dfc80 2024-09-11T22:57:43.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc7519f740 10 client.16728 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:43.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc7519f740 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:57:43.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc7519f740 20 client.16728 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:43.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc7519f740 1 --2- 172.21.15.152:0/2872015226 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55e62b2674a0 0x55e62b2698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:43.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc7519f740 4 client.16728 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:57:43.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.026+0000 7fbc7519f740 1 -- 172.21.15.152:0/2872015226 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e62b14cd60 con 0x55e62b2674a0 2024-09-11T22:57:43.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.027+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/2872015226 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55e62b2674a0 0x55e62b2698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:43.051 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.027+0000 7fbc4bfff640 10 client.16728 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:43.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.027+0000 7fbc6bfff640 1 --2- 172.21.15.152:0/2872015226 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55e62b2674a0 0x55e62b2698c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34e4f35cf2b7a970 server_cookie=a69ffc9a8884cc44 in_seq=0 out_seq=0 2024-09-11T22:57:43.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.027+0000 7fbc4bfff640 1 -- 172.21.15.152:0/2872015226 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e62b14cd60 con 0x55e62b2674a0 2024-09-11T22:57:43.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.027+0000 7fbc4bfff640 10 client.16728 handle_command_reply: tid=0 2024-09-11T22:57:43.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.027+0000 7fbc7519f740 1 client.16728 shutdown 2024-09-11T22:57:43.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.027+0000 7fbc7519f740 1 -- 172.21.15.152:0/2872015226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1dfc80 msgr2=0x55e62b1effc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:43.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.027+0000 7fbc7519f740 1 --2- 172.21.15.152:0/2872015226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1dfc80 0x55e62b1effc0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbc6000a960 tx=0x7fbc6000c2c0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.027+0000 7fbc7519f740 1 -- 172.21.15.152:0/2872015226 shutdown_connections 2024-09-11T22:57:43.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.027+0000 7fbc7519f740 1 --2- 172.21.15.152:0/2872015226 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55e62b2674a0 0x55e62b2698c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.027+0000 7fbc7519f740 1 --2- 172.21.15.152:0/2872015226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e62b1ec890 0x55e62b1f0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.027+0000 7fbc7519f740 1 --2- 172.21.15.152:0/2872015226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e62b1dfc80 0x55e62b1effc0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.027+0000 7fbc4bfff640 0 client.16728 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:43.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.027+0000 7fbc7519f740 1 --2- 172.21.15.152:0/2872015226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e62b1df360 0x55e62b1f4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.052 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.027+0000 7fbc7519f740 1 -- 172.21.15.152:0/2872015226 >> 172.21.15.152:0/2872015226 conn(0x55e62b1dba30 msgr2=0x55e62b1e7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:43.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.028+0000 7fbc7519f740 1 -- 172.21.15.152:0/2872015226 shutdown_connections 2024-09-11T22:57:43.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.028+0000 7fbc7519f740 1 -- 172.21.15.152:0/2872015226 wait complete. 2024-09-11T22:57:43.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.028+0000 7fbc7519f740 20 client.16728 trim_cache size 0 max 16384 2024-09-11T22:57:43.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.028+0000 7fbc72dc6640 1 -- 172.21.15.152:0/1670017131 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbc5c0a2cf0 msgr2=0x7fbc5c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:43.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.028+0000 7fbc72dc6640 1 --2- 172.21.15.152:0/1670017131 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbc5c0a2cf0 0x7fbc5c0a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbc6409ad00 tx=0x7fbc64090490 comp rx=0 tx=0).stop 2024-09-11T22:57:43.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.028+0000 7fbc72dc6640 1 -- 172.21.15.152:0/1670017131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc6c1055b0 msgr2=0x7fbc6c19aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:43.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.028+0000 7fbc72dc6640 1 --2- 172.21.15.152:0/1670017131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc6c1055b0 0x7fbc6c19aca0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400b970 tx=0x7fbc6400be30 comp rx=0 tx=0).stop 2024-09-11T22:57:43.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.028+0000 7fbc72dc6640 1 -- 172.21.15.152:0/1670017131 shutdown_connections 2024-09-11T22:57:43.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.028+0000 7fbc72dc6640 1 --2- 172.21.15.152:0/1670017131 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbc5c0a2cf0 0x7fbc5c0a51b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.053 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.028+0000 7fbc72dc6640 1 --2- 172.21.15.152:0/1670017131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc6c1055b0 0x7fbc6c19aca0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.028+0000 7fbc72dc6640 1 --2- 172.21.15.152:0/1670017131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc6c100e30 0x7fbc6c196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.028+0000 7fbc72dc6640 1 --2- 172.21.15.152:0/1670017131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc6c100480 0x7fbc6c1963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:43.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.028+0000 7fbc72dc6640 1 -- 172.21.15.152:0/1670017131 >> 172.21.15.152:0/1670017131 conn(0x7fbc6c077210 msgr2=0x7fbc6c103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:43.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.028+0000 7fbc72dc6640 1 -- 172.21.15.152:0/1670017131 shutdown_connections 2024-09-11T22:57:43.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:43.028+0000 7fbc72dc6640 1 -- 172.21.15.152:0/1670017131 wait complete. 2024-09-11T22:57:43.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:42 smithi191 ceph-mon[35211]: pgmap v147: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:57:43.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:42 smithi204 ceph-mon[35454]: pgmap v147: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:57:43.932 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:57:43.933 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:57:44.184 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:57:44.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.718+0000 7f44daf95640 1 -- 172.21.15.152:0/1190241159 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44d4100a80 msgr2=0x7f44d4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:44.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.718+0000 7f44daf95640 1 --2- 172.21.15.152:0/1190241159 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44d4100a80 0x7f44d4100ee0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f44c0009930 tx=0x7f44c002a420 comp rx=0 tx=0).stop 2024-09-11T22:57:44.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.718+0000 7f44daf95640 1 -- 172.21.15.152:0/1190241159 shutdown_connections 2024-09-11T22:57:44.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.718+0000 7f44daf95640 1 --2- 172.21.15.152:0/1190241159 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44d4101420 0x7f44d410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:44.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.718+0000 7f44daf95640 1 --2- 172.21.15.152:0/1190241159 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44d4100a80 0x7f44d4100ee0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:44.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.718+0000 7f44daf95640 1 --2- 172.21.15.152:0/1190241159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44d4106a80 0x7f44d4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:44.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.718+0000 7f44daf95640 1 -- 172.21.15.152:0/1190241159 >> 172.21.15.152:0/1190241159 conn(0x7f44d40fc7a0 msgr2=0x7f44d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:44.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.718+0000 7f44daf95640 1 -- 172.21.15.152:0/1190241159 shutdown_connections 2024-09-11T22:57:44.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.718+0000 7f44daf95640 1 -- 172.21.15.152:0/1190241159 wait complete. 2024-09-11T22:57:44.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.719+0000 7f44daf95640 1 Processor -- start 2024-09-11T22:57:44.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.719+0000 7f44daf95640 1 -- start start 2024-09-11T22:57:44.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.719+0000 7f44daf95640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44d4100a80 0x7f44d4197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:44.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.719+0000 7f44daf95640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44d4101420 0x7f44d4197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:44.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.719+0000 7f44daf95640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44d4106a80 0x7f44d419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:44.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.719+0000 7f44daf95640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f44d4110ec0 con 0x7f44d4100a80 2024-09-11T22:57:44.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.719+0000 7f44daf95640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f44d4110d40 con 0x7f44d4106a80 2024-09-11T22:57:44.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.719+0000 7f44daf95640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f44d4111040 con 0x7f44d4101420 2024-09-11T22:57:44.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.719+0000 7f44d8d31640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44d4100a80 0x7f44d4197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:44.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.719+0000 7f44d8d31640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44d4100a80 0x7f44d4197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45714/0 (socket says 172.21.15.152:45714) 2024-09-11T22:57:44.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.719+0000 7f44d8d31640 1 -- 172.21.15.152:0/3988972918 learned_addr learned my addr 172.21.15.152:0/3988972918 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:44.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.719+0000 7f44d9532640 1 --2- 172.21.15.152:0/3988972918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44d4106a80 0x7f44d419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:44.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.719+0000 7f44d3fff640 1 --2- 172.21.15.152:0/3988972918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44d4101420 0x7f44d4197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:44.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.719+0000 7f44d8d31640 1 -- 172.21.15.152:0/3988972918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44d4101420 msgr2=0x7f44d4197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:44.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.720+0000 7f44d8d31640 1 --2- 172.21.15.152:0/3988972918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44d4101420 0x7f44d4197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:44.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.720+0000 7f44d8d31640 1 -- 172.21.15.152:0/3988972918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44d4106a80 msgr2=0x7f44d419bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:44.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.720+0000 7f44d8d31640 1 --2- 172.21.15.152:0/3988972918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44d4106a80 0x7f44d419bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:44.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.720+0000 7f44d8d31640 1 -- 172.21.15.152:0/3988972918 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d419c3b0 con 0x7f44d4100a80 2024-09-11T22:57:44.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.720+0000 7f44d9532640 1 --2- 172.21.15.152:0/3988972918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44d4106a80 0x7f44d419bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:44.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.720+0000 7f44d8d31640 1 --2- 172.21.15.152:0/3988972918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44d4100a80 0x7f44d4197400 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009860 tx=0x7f44c8009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:44.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.720+0000 7f44d1ffb640 1 -- 172.21.15.152:0/3988972918 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8005020 con 0x7f44d4100a80 2024-09-11T22:57:44.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.720+0000 7f44d1ffb640 1 -- 172.21.15.152:0/3988972918 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44c80047a0 con 0x7f44d4100a80 2024-09-11T22:57:44.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.720+0000 7f44daf95640 1 -- 172.21.15.152:0/3988972918 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d419c6a0 con 0x7f44d4100a80 2024-09-11T22:57:44.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.720+0000 7f44d1ffb640 1 -- 172.21.15.152:0/3988972918 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c8007ba0 con 0x7f44d4100a80 2024-09-11T22:57:44.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.720+0000 7f44daf95640 1 -- 172.21.15.152:0/3988972918 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d406a790 con 0x7f44d4100a80 2024-09-11T22:57:44.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.721+0000 7f44daf95640 1 -- 172.21.15.152:0/3988972918 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d41021c0 con 0x7f44d4100a80 2024-09-11T22:57:44.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.722+0000 7f44d1ffb640 1 -- 172.21.15.152:0/3988972918 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f44c8004950 con 0x7f44d4100a80 2024-09-11T22:57:44.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.723+0000 7f44d1ffb640 1 --2- 172.21.15.152:0/3988972918 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f44c40a2cf0 0x7f44c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:44.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.723+0000 7f44d3fff640 1 --2- 172.21.15.152:0/3988972918 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f44c40a2cf0 0x7f44c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:44.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.723+0000 7f44d1ffb640 1 -- 172.21.15.152:0/3988972918 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f44c80d11b0 con 0x7f44d4100a80 2024-09-11T22:57:44.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.723+0000 7f44d3fff640 1 --2- 172.21.15.152:0/3988972918 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f44c40a2cf0 0x7f44c40a51b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f44c00096c0 tx=0x7f44c0000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:44.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.726+0000 7f44d1ffb640 1 -- 172.21.15.152:0/3988972918 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44c809de20 con 0x7f44d4100a80 2024-09-11T22:57:44.930 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.929+0000 7f44daf95640 1 -- 172.21.15.152:0/3988972918 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44d4100ee0 con 0x7f44d4100a80 2024-09-11T22:57:44.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.929+0000 7f44d1ffb640 1 -- 172.21.15.152:0/3988972918 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f44c809d720 con 0x7f44d4100a80 2024-09-11T22:57:44.933 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:57:44.934 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:57:44.934 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:57:44.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.932+0000 7f44daf95640 1 -- 172.21.15.152:0/3988972918 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f44c40a2cf0 msgr2=0x7f44c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:44.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.932+0000 7f44daf95640 1 --2- 172.21.15.152:0/3988972918 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f44c40a2cf0 0x7f44c40a51b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f44c00096c0 tx=0x7f44c0000bc0 comp rx=0 tx=0).stop 2024-09-11T22:57:44.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.932+0000 7f44daf95640 1 -- 172.21.15.152:0/3988972918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44d4100a80 msgr2=0x7f44d4197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:44.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.932+0000 7f44daf95640 1 --2- 172.21.15.152:0/3988972918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44d4100a80 0x7f44d4197400 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009860 tx=0x7f44c8009d20 comp rx=0 tx=0).stop 2024-09-11T22:57:44.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.932+0000 7f44daf95640 1 -- 172.21.15.152:0/3988972918 shutdown_connections 2024-09-11T22:57:44.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.932+0000 7f44daf95640 1 --2- 172.21.15.152:0/3988972918 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f44c40a2cf0 0x7f44c40a51b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:44.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.932+0000 7f44daf95640 1 --2- 172.21.15.152:0/3988972918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f44d4106a80 0x7f44d419bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:44.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.932+0000 7f44daf95640 1 --2- 172.21.15.152:0/3988972918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44d4101420 0x7f44d4197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:44.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.932+0000 7f44daf95640 1 --2- 172.21.15.152:0/3988972918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f44d4100a80 0x7f44d4197400 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:44.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.932+0000 7f44daf95640 1 -- 172.21.15.152:0/3988972918 >> 172.21.15.152:0/3988972918 conn(0x7f44d40fc7a0 msgr2=0x7f44d40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:44.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.932+0000 7f44daf95640 1 -- 172.21.15.152:0/3988972918 shutdown_connections 2024-09-11T22:57:44.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:44.932+0000 7f44daf95640 1 -- 172.21.15.152:0/3988972918 wait complete. 2024-09-11T22:57:45.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:44 smithi191 ceph-mon[35211]: pgmap v148: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:45.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:44 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3988972918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:57:45.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:44 smithi204 ceph-mon[35454]: pgmap v148: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:45.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:44 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3988972918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:57:45.304 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:44 smithi152 ceph-mon[31181]: pgmap v148: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:45.304 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:44 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3988972918' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:57:45.800 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:57:46.051 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:57:46.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.597+0000 7f5ce32d8640 1 -- 172.21.15.152:0/1150786484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cdc106f20 msgr2=0x7f5cdc10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:46.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.597+0000 7f5ce32d8640 1 --2- 172.21.15.152:0/1150786484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cdc106f20 0x7f5cdc10eaa0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009ca0 tx=0x7f5cd402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.598+0000 7f5ce32d8640 1 -- 172.21.15.152:0/1150786484 shutdown_connections 2024-09-11T22:57:46.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.598+0000 7f5ce32d8640 1 --2- 172.21.15.152:0/1150786484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cdc106f20 0x7f5cdc10eaa0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.598+0000 7f5ce32d8640 1 --2- 172.21.15.152:0/1150786484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cdc1027a0 0x7f5cdc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.598+0000 7f5ce32d8640 1 --2- 172.21.15.152:0/1150786484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cdc101df0 0x7f5cdc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.598+0000 7f5ce32d8640 1 -- 172.21.15.152:0/1150786484 >> 172.21.15.152:0/1150786484 conn(0x7f5cdc0fd8f0 msgr2=0x7f5cdc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:46.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.598+0000 7f5ce32d8640 1 -- 172.21.15.152:0/1150786484 shutdown_connections 2024-09-11T22:57:46.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.598+0000 7f5ce32d8640 1 -- 172.21.15.152:0/1150786484 wait complete. 2024-09-11T22:57:46.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.598+0000 7f5ce32d8640 1 Processor -- start 2024-09-11T22:57:46.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.598+0000 7f5ce32d8640 1 -- start start 2024-09-11T22:57:46.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.598+0000 7f5ce32d8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cdc101df0 0x7f5cdc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.598+0000 7f5ce32d8640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cdc1027a0 0x7f5cdc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.598+0000 7f5ce32d8640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cdc106f20 0x7f5cdc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.598+0000 7f5ce32d8640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc111f80 con 0x7f5cdc101df0 2024-09-11T22:57:46.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.598+0000 7f5ce32d8640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc111e00 con 0x7f5cdc106f20 2024-09-11T22:57:46.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.598+0000 7f5ce32d8640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc112100 con 0x7f5cdc1027a0 2024-09-11T22:57:46.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5ce1074640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cdc101df0 0x7f5cdc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5ce1074640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cdc101df0 0x7f5cdc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45732/0 (socket says 172.21.15.152:45732) 2024-09-11T22:57:46.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5ce1074640 1 -- 172.21.15.152:0/3241194514 learned_addr learned my addr 172.21.15.152:0/3241194514 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:46.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5ce1875640 1 --2- 172.21.15.152:0/3241194514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cdc106f20 0x7f5cdc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5ce0873640 1 --2- 172.21.15.152:0/3241194514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cdc1027a0 0x7f5cdc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5ce1074640 1 -- 172.21.15.152:0/3241194514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cdc1027a0 msgr2=0x7f5cdc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:46.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5ce1074640 1 --2- 172.21.15.152:0/3241194514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cdc1027a0 0x7f5cdc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5ce1074640 1 -- 172.21.15.152:0/3241194514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cdc106f20 msgr2=0x7f5cdc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:46.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5ce1074640 1 --2- 172.21.15.152:0/3241194514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cdc106f20 0x7f5cdc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5ce1074640 1 -- 172.21.15.152:0/3241194514 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cdc1a6890 con 0x7f5cdc101df0 2024-09-11T22:57:46.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5ce0873640 1 --2- 172.21.15.152:0/3241194514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cdc1027a0 0x7f5cdc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:57:46.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5ce1074640 1 --2- 172.21.15.152:0/3241194514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cdc101df0 0x7f5cdc0744f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800c890 tx=0x7f5cd800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:46.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5cd27fc640 1 -- 172.21.15.152:0/3241194514 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8004220 con 0x7f5cdc101df0 2024-09-11T22:57:46.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5ce32d8640 1 -- 172.21.15.152:0/3241194514 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cdc1a6b80 con 0x7f5cdc101df0 2024-09-11T22:57:46.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5cd27fc640 1 -- 172.21.15.152:0/3241194514 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd80043c0 con 0x7f5cdc101df0 2024-09-11T22:57:46.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5cd27fc640 1 -- 172.21.15.152:0/3241194514 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80049e0 con 0x7f5cdc101df0 2024-09-11T22:57:46.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.599+0000 7f5ce32d8640 1 -- 172.21.15.152:0/3241194514 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cdc1a70c0 con 0x7f5cdc101df0 2024-09-11T22:57:46.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.602+0000 7f5cd27fc640 1 -- 172.21.15.152:0/3241194514 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5cd80079b0 con 0x7f5cdc101df0 2024-09-11T22:57:46.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.602+0000 7f5ce56b1740 1 Processor -- start 2024-09-11T22:57:46.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.602+0000 7f5ce56b1740 1 -- start start 2024-09-11T22:57:46.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.602+0000 7f5ce56b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c28b40 0x5623e0c27090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.602+0000 7f5ce56b1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c275d0 0x5623e0c27a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.602+0000 7f5ce56b1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c32370 0x5623e0c34760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.602+0000 7f5ce56b1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5623e0b8d2f0 con 0x5623e0c275d0 2024-09-11T22:57:46.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.602+0000 7f5ce56b1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5623e0b34cf0 con 0x5623e0c32370 2024-09-11T22:57:46.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.602+0000 7f5ce56b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5623e0b892b0 con 0x5623e0c28b40 2024-09-11T22:57:46.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.602+0000 7f5ce1074640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c275d0 0x5623e0c27a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.602+0000 7f5ce1074640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c275d0 0x5623e0c27a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45738/0 (socket says 172.21.15.152:45738) 2024-09-11T22:57:46.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.602+0000 7f5ce1074640 1 -- 172.21.15.152:0/2374767730 learned_addr learned my addr 172.21.15.152:0/2374767730 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:46.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.602+0000 7f5cd27fc640 1 --2- 172.21.15.152:0/3241194514 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f5cc80a2cf0 0x7f5cc80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.603+0000 7f5ce0873640 1 --2- 172.21.15.152:0/2374767730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c32370 0x5623e0c34760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.603+0000 7f5ce1875640 1 --2- 172.21.15.152:0/2374767730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c28b40 0x5623e0c27090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.603+0000 7f5ce1875640 1 --2- 172.21.15.152:0/3241194514 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f5cc80a2cf0 0x7f5cc80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.603+0000 7f5cd27fc640 1 -- 172.21.15.152:0/3241194514 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f5cd80cf0c0 con 0x7f5cdc101df0 2024-09-11T22:57:46.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.603+0000 7f5ce1074640 1 -- 172.21.15.152:0/2374767730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c28b40 msgr2=0x5623e0c27090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:46.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.603+0000 7f5ce1074640 1 --2- 172.21.15.152:0/2374767730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c28b40 0x5623e0c27090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.603+0000 7f5ce1074640 1 -- 172.21.15.152:0/2374767730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c32370 msgr2=0x5623e0c34760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:46.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.603+0000 7f5ce1074640 1 --2- 172.21.15.152:0/2374767730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c32370 0x5623e0c34760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.603+0000 7f5ce1074640 1 -- 172.21.15.152:0/2374767730 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e0b193a0 con 0x5623e0c275d0 2024-09-11T22:57:46.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.603+0000 7f5ce1074640 1 --2- 172.21.15.152:0/2374767730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c275d0 0x5623e0c27a10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5cd809b3d0 tx=0x7f5cd8005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9569c71d517a0bd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:46.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.603+0000 7f5ce1875640 1 --2- 172.21.15.152:0/3241194514 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f5cc80a2cf0 0x7f5cc80a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4031f80 tx=0x7f5cd40095c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:46.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.603+0000 7f5cc77fe640 1 -- 172.21.15.152:0/2374767730 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd801d050 con 0x5623e0c275d0 2024-09-11T22:57:46.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.603+0000 7f5cc77fe640 1 -- 172.21.15.152:0/2374767730 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd809bdd0 con 0x5623e0c275d0 2024-09-11T22:57:46.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.603+0000 7f5cc77fe640 1 -- 172.21.15.152:0/2374767730 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80a4340 con 0x5623e0c275d0 2024-09-11T22:57:46.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.604+0000 7f5ce56b1740 1 -- 172.21.15.152:0/2374767730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c275d0 msgr2=0x5623e0c27a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:46.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.604+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/2374767730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c275d0 0x5623e0c27a10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5cd809b3d0 tx=0x7f5cd8005300 comp rx=0 tx=0).stop 2024-09-11T22:57:46.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.604+0000 7f5ce56b1740 1 -- 172.21.15.152:0/2374767730 shutdown_connections 2024-09-11T22:57:46.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.604+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/2374767730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c32370 0x5623e0c34760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.604+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/2374767730 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c275d0 0x5623e0c27a10 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.604+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/2374767730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c28b40 0x5623e0c27090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.604+0000 7f5ce56b1740 1 -- 172.21.15.152:0/2374767730 >> 172.21.15.152:0/2374767730 conn(0x5623e0bf0310 msgr2=0x5623e0c23c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:46.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.604+0000 7f5ce56b1740 1 -- 172.21.15.152:0/2374767730 shutdown_connections 2024-09-11T22:57:46.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.604+0000 7f5ce56b1740 1 -- 172.21.15.152:0/2374767730 wait complete. 2024-09-11T22:57:46.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.604+0000 7f5ce56b1740 1 -- start start 2024-09-11T22:57:46.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce56b1740 1 Processor -- start 2024-09-11T22:57:46.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce56b1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c275d0 0x5623e0c31e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce56b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c28b40 0x5623e0c2f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce56b1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c32370 0x5623e0c2f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce56b1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5623e0c34fb0 con 0x5623e0c32370 2024-09-11T22:57:46.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce56b1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5623e0b48060 con 0x5623e0c275d0 2024-09-11T22:57:46.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce56b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5623e0c35130 con 0x5623e0c28b40 2024-09-11T22:57:46.650 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:57:46.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce56b1740 10 client.0 _ll_register_callbacks cb 0x5623e089b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:46.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce1875640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c32370 0x5623e0c2f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce1875640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c32370 0x5623e0c2f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45748/0 (socket says 172.21.15.152:45748) 2024-09-11T22:57:46.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce1875640 1 -- 172.21.15.152:0/3072915630 learned_addr learned my addr 172.21.15.152:0/3072915630 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:46.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce0873640 1 --2- 172.21.15.152:0/3072915630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c28b40 0x5623e0c2f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce1074640 1 --2- 172.21.15.152:0/3072915630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c275d0 0x5623e0c31e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce1875640 1 -- 172.21.15.152:0/3072915630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c28b40 msgr2=0x5623e0c2f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:46.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce1875640 1 --2- 172.21.15.152:0/3072915630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c28b40 0x5623e0c2f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce1875640 1 -- 172.21.15.152:0/3072915630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c275d0 msgr2=0x5623e0c31e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:46.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce1875640 1 --2- 172.21.15.152:0/3072915630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c275d0 0x5623e0c31e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.605+0000 7f5ce1875640 1 -- 172.21.15.152:0/3072915630 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e08fe3c0 con 0x5623e0c32370 2024-09-11T22:57:46.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5ce1875640 1 --2- 172.21.15.152:0/3072915630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c32370 0x5623e0c2f850 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009ca0 tx=0x7f5cd402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569f5f3097c8b4f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:46.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5cc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:46.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5cc4ff9640 1 -- 172.21.15.152:0/3072915630 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd40305c0 con 0x5623e0c32370 2024-09-11T22:57:46.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5cc4ff9640 1 -- 172.21.15.152:0/3072915630 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f5cd4030760 con 0x5623e0c32370 2024-09-11T22:57:46.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5ce56b1740 1 -- 172.21.15.152:0/3072915630 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cd809bdd0 con 0x5623e0c32370 2024-09-11T22:57:46.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5cc4ff9640 1 -- 172.21.15.152:0/3072915630 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5cd4030960 con 0x5623e0c32370 2024-09-11T22:57:46.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5ce56b1740 10 client.16746 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:46.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5ce56b1740 1 -- 172.21.15.152:0/3072915630 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5623e0c40b30 con 0x5623e0c32370 2024-09-11T22:57:46.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5cc4ff9640 1 -- 172.21.15.152:0/3072915630 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cd809bdd0 con 0x5623e0c32370 2024-09-11T22:57:46.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5cc4ff9640 1 -- 172.21.15.152:0/3072915630 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5cd403ab80 con 0x5623e0c32370 2024-09-11T22:57:46.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5ce56b1740 10 client.16746 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:46.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5ce56b1740 10 client.16746 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:57:46.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5ce56b1740 20 client.16746 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:46.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/3072915630 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5623e0c42040 0x5623e0c44480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5ce56b1740 4 client.16746 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:57:46.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5ce56b1740 1 -- 172.21.15.152:0/3072915630 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623e0b9a860 con 0x5623e0c42040 2024-09-11T22:57:46.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.606+0000 7f5ce1074640 1 --2- 172.21.15.152:0/3072915630 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5623e0c42040 0x5623e0c44480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.607+0000 7f5cc4ff9640 10 client.16746 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:57:46.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.607+0000 7f5ce1074640 1 --2- 172.21.15.152:0/3072915630 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5623e0c42040 0x5623e0c44480 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d2e7286a4083971c server_cookie=eae6eb9511511b28 in_seq=0 out_seq=0 2024-09-11T22:57:46.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.607+0000 7f5cc4ff9640 1 -- 172.21.15.152:0/3072915630 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623e0b9a860 con 0x5623e0c42040 2024-09-11T22:57:46.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.607+0000 7f5cc4ff9640 10 client.16746 handle_command_reply: tid=0 2024-09-11T22:57:46.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.607+0000 7f5ce56b1740 1 client.16746 shutdown 2024-09-11T22:57:46.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.607+0000 7f5ce56b1740 1 -- 172.21.15.152:0/3072915630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c32370 msgr2=0x5623e0c2f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:46.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.607+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/3072915630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c32370 0x5623e0c2f850 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009ca0 tx=0x7f5cd402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.607+0000 7f5ce56b1740 1 -- 172.21.15.152:0/3072915630 shutdown_connections 2024-09-11T22:57:46.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.607+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/3072915630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c32370 0x5623e0c2f850 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.607+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/3072915630 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c28b40 0x5623e0c2f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.607+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/3072915630 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5623e0c42040 0x5623e0c44480 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.607+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/3072915630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c275d0 0x5623e0c31e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.607+0000 7f5ce56b1740 1 -- 172.21.15.152:0/3072915630 >> 172.21.15.152:0/3072915630 conn(0x5623e0bf0310 msgr2=0x5623e0c29990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:46.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.608+0000 7f5cc4ff9640 0 client.16746 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:57:46.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.608+0000 7f5ce56b1740 1 -- 172.21.15.152:0/3072915630 shutdown_connections 2024-09-11T22:57:46.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.608+0000 7f5ce56b1740 1 -- 172.21.15.152:0/3072915630 wait complete. 2024-09-11T22:57:46.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.608+0000 7f5ce56b1740 20 client.16746 trim_cache size 0 max 16384 2024-09-11T22:57:46.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.620+0000 7f5ce56b1740 1 Processor -- start 2024-09-11T22:57:46.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce56b1740 1 -- start start 2024-09-11T22:57:46.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce56b1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c275d0 0x5623e0c23d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce56b1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c28b40 0x5623e0c2e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce56b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c32370 0x5623e0c49d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce56b1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1afa60 con 0x5623e0c28b40 2024-09-11T22:57:46.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce56b1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1b15a0 con 0x5623e0c275d0 2024-09-11T22:57:46.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce56b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1b0130 con 0x5623e0c32370 2024-09-11T22:57:46.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce0873640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c28b40 0x5623e0c2e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce0873640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c28b40 0x5623e0c2e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45750/0 (socket says 172.21.15.152:45750) 2024-09-11T22:57:46.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce0873640 1 -- 172.21.15.152:0/3160021665 learned_addr learned my addr 172.21.15.152:0/3160021665 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:46.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce1875640 1 --2- 172.21.15.152:0/3160021665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c32370 0x5623e0c49d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce1074640 1 --2- 172.21.15.152:0/3160021665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c275d0 0x5623e0c23d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce0873640 1 -- 172.21.15.152:0/3160021665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c32370 msgr2=0x5623e0c49d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:46.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce0873640 1 --2- 172.21.15.152:0/3160021665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c32370 0x5623e0c49d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce0873640 1 -- 172.21.15.152:0/3160021665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c275d0 msgr2=0x5623e0c23d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:46.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce0873640 1 --2- 172.21.15.152:0/3160021665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c275d0 0x5623e0c23d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce0873640 1 -- 172.21.15.152:0/3160021665 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e0c3e280 con 0x5623e0c28b40 2024-09-11T22:57:46.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5ce0873640 1 --2- 172.21.15.152:0/3160021665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c28b40 0x5623e0c2e0a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00bc20 tx=0x7f5ccc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62d9e862151c081 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:46.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.621+0000 7f5cc6ffd640 1 -- 172.21.15.152:0/3160021665 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0110d0 con 0x5623e0c28b40 2024-09-11T22:57:46.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.622+0000 7f5cc6ffd640 1 -- 172.21.15.152:0/3160021665 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ccc011270 con 0x5623e0c28b40 2024-09-11T22:57:46.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.622+0000 7f5cc6ffd640 1 -- 172.21.15.152:0/3160021665 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc01a740 con 0x5623e0c28b40 2024-09-11T22:57:46.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.622+0000 7f5ce56b1740 1 -- 172.21.15.152:0/3160021665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c28b40 msgr2=0x5623e0c2e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:46.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.622+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/3160021665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c28b40 0x5623e0c2e0a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00bc20 tx=0x7f5ccc011810 comp rx=0 tx=0).stop 2024-09-11T22:57:46.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.622+0000 7f5ce56b1740 1 -- 172.21.15.152:0/3160021665 shutdown_connections 2024-09-11T22:57:46.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.622+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/3160021665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c32370 0x5623e0c49d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.622+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/3160021665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c28b40 0x5623e0c2e0a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.622+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/3160021665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c275d0 0x5623e0c23d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.622+0000 7f5ce56b1740 1 -- 172.21.15.152:0/3160021665 >> 172.21.15.152:0/3160021665 conn(0x5623e0bf0310 msgr2=0x5623e0c2d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:46.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.622+0000 7f5ce56b1740 1 -- 172.21.15.152:0/3160021665 shutdown_connections 2024-09-11T22:57:46.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.622+0000 7f5ce56b1740 1 -- 172.21.15.152:0/3160021665 wait complete. 2024-09-11T22:57:46.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.622+0000 7f5ce56b1740 1 -- start start 2024-09-11T22:57:46.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce56b1740 1 Processor -- start 2024-09-11T22:57:46.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce56b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c275d0 0x5623e0c3ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce56b1740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c28b40 0x5623e0c36060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce56b1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c32370 0x5623e0c367a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce56b1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1b0ed0 con 0x5623e0c32370 2024-09-11T22:57:46.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce56b1740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1b0800 con 0x5623e0c28b40 2024-09-11T22:57:46.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce56b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc1ada20 con 0x5623e0c275d0 2024-09-11T22:57:46.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce56b1740 10 client.0 _ll_register_callbacks cb 0x7f5cd402f330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:46.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce1875640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c32370 0x5623e0c367a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce1875640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c32370 0x5623e0c367a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45754/0 (socket says 172.21.15.152:45754) 2024-09-11T22:57:46.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce1875640 1 -- 172.21.15.152:0/2702278519 learned_addr learned my addr 172.21.15.152:0/2702278519 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:46.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce1074640 1 --2- 172.21.15.152:0/2702278519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c275d0 0x5623e0c3ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce0873640 1 --2- 172.21.15.152:0/2702278519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c28b40 0x5623e0c36060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce1875640 1 -- 172.21.15.152:0/2702278519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c275d0 msgr2=0x5623e0c3ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:46.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce1875640 1 --2- 172.21.15.152:0/2702278519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c275d0 0x5623e0c3ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce1875640 1 -- 172.21.15.152:0/2702278519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c28b40 msgr2=0x5623e0c36060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:46.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce1875640 1 --2- 172.21.15.152:0/2702278519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c28b40 0x5623e0c36060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce1875640 1 -- 172.21.15.152:0/2702278519 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e0cabd20 con 0x5623e0c32370 2024-09-11T22:57:46.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce0873640 1 --2- 172.21.15.152:0/2702278519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c28b40 0x5623e0c36060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:46.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.623+0000 7f5ce1074640 1 --2- 172.21.15.152:0/2702278519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c275d0 0x5623e0c3ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:46.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.624+0000 7f5ce1875640 1 --2- 172.21.15.152:0/2702278519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c32370 0x5623e0c367a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4033ff0 tx=0x7f5cd402ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4935cf13b28627d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:46.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.624+0000 7f5cc4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:46.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.624+0000 7f5cc4ff9640 1 -- 172.21.15.152:0/2702278519 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4062060 con 0x5623e0c32370 2024-09-11T22:57:46.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.624+0000 7f5ce56b1740 1 -- 172.21.15.152:0/2702278519 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ccc011270 con 0x5623e0c32370 2024-09-11T22:57:46.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.624+0000 7f5cc4ff9640 1 -- 172.21.15.152:0/2702278519 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f5cd4031980 con 0x5623e0c32370 2024-09-11T22:57:46.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.624+0000 7f5cc4ff9640 1 -- 172.21.15.152:0/2702278519 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5cd4030d10 con 0x5623e0c32370 2024-09-11T22:57:46.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.624+0000 7f5ce56b1740 10 client.16752 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:46.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.624+0000 7f5ce56b1740 1 -- 172.21.15.152:0/2702278519 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5623e0cacec0 con 0x5623e0c32370 2024-09-11T22:57:46.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.624+0000 7f5cc4ff9640 1 -- 172.21.15.152:0/2702278519 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ccc011270 con 0x5623e0c32370 2024-09-11T22:57:46.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.624+0000 7f5cc4ff9640 1 -- 172.21.15.152:0/2702278519 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5cd4003970 con 0x5623e0c32370 2024-09-11T22:57:46.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.624+0000 7f5ce56b1740 10 client.16752 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:46.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.624+0000 7f5ce56b1740 10 client.16752 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:57:46.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.624+0000 7f5ce56b1740 20 client.16752 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:46.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.624+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/2702278519 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5623e0cad5b0 0x5623e0caf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:46.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.624+0000 7f5ce56b1740 4 client.16752 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:57:46.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.624+0000 7f5ce56b1740 1 -- 172.21.15.152:0/2702278519 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623e0b94d70 con 0x5623e0cad5b0 2024-09-11T22:57:46.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5ce1074640 1 --2- 172.21.15.152:0/2702278519 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5623e0cad5b0 0x5623e0caf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:46.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5cc4ff9640 10 client.16752 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:57:46.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5ce1074640 1 --2- 172.21.15.152:0/2702278519 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5623e0cad5b0 0x5623e0caf9d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=afd1664079dd51ae server_cookie=d0f947fe77e971e9 in_seq=0 out_seq=0 2024-09-11T22:57:46.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5cc4ff9640 1 -- 172.21.15.152:0/2702278519 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623e0b94d70 con 0x5623e0cad5b0 2024-09-11T22:57:46.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5cc4ff9640 10 client.16752 handle_command_reply: tid=0 2024-09-11T22:57:46.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5ce56b1740 1 client.16752 shutdown 2024-09-11T22:57:46.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5ce56b1740 1 -- 172.21.15.152:0/2702278519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c32370 msgr2=0x5623e0c367a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:46.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/2702278519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c32370 0x5623e0c367a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4033ff0 tx=0x7f5cd402ffa0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5ce56b1740 1 -- 172.21.15.152:0/2702278519 shutdown_connections 2024-09-11T22:57:46.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/2702278519 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5623e0c32370 0x5623e0c367a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/2702278519 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5623e0cad5b0 0x5623e0caf9d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/2702278519 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5623e0c28b40 0x5623e0c36060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5ce56b1740 1 --2- 172.21.15.152:0/2702278519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5623e0c275d0 0x5623e0c3ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5cc4ff9640 0 client.16752 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:57:46.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5ce56b1740 1 -- 172.21.15.152:0/2702278519 >> 172.21.15.152:0/2702278519 conn(0x5623e0bf0310 msgr2=0x5623e0c2e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:46.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5ce56b1740 1 -- 172.21.15.152:0/2702278519 shutdown_connections 2024-09-11T22:57:46.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5ce56b1740 1 -- 172.21.15.152:0/2702278519 wait complete. 2024-09-11T22:57:46.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.625+0000 7f5ce56b1740 20 client.16752 trim_cache size 0 max 16384 2024-09-11T22:57:46.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.626+0000 7f5ce32d8640 1 -- 172.21.15.152:0/3241194514 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f5cc80a2cf0 msgr2=0x7f5cc80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:46.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.626+0000 7f5ce32d8640 1 --2- 172.21.15.152:0/3241194514 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f5cc80a2cf0 0x7f5cc80a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4031f80 tx=0x7f5cd40095c0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.626+0000 7f5ce32d8640 1 -- 172.21.15.152:0/3241194514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cdc101df0 msgr2=0x7f5cdc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:46.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.626+0000 7f5ce32d8640 1 --2- 172.21.15.152:0/3241194514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cdc101df0 0x7f5cdc0744f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800c890 tx=0x7f5cd800cd50 comp rx=0 tx=0).stop 2024-09-11T22:57:46.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.626+0000 7f5ce32d8640 1 -- 172.21.15.152:0/3241194514 shutdown_connections 2024-09-11T22:57:46.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.626+0000 7f5ce32d8640 1 --2- 172.21.15.152:0/3241194514 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f5cc80a2cf0 0x7f5cc80a51b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.626+0000 7f5ce32d8640 1 --2- 172.21.15.152:0/3241194514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5cdc106f20 0x7f5cdc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.626+0000 7f5ce32d8640 1 --2- 172.21.15.152:0/3241194514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5cdc1027a0 0x7f5cdc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.626+0000 7f5ce32d8640 1 --2- 172.21.15.152:0/3241194514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5cdc101df0 0x7f5cdc0744f0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:46.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.626+0000 7f5ce32d8640 1 -- 172.21.15.152:0/3241194514 >> 172.21.15.152:0/3241194514 conn(0x7f5cdc0fd8f0 msgr2=0x7f5cdc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:46.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.626+0000 7f5ce32d8640 1 -- 172.21.15.152:0/3241194514 shutdown_connections 2024-09-11T22:57:46.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:46.626+0000 7f5ce32d8640 1 -- 172.21.15.152:0/3241194514 wait complete. 2024-09-11T22:57:47.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:46 smithi191 ceph-mon[35211]: pgmap v149: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:57:47.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:46 smithi204 ceph-mon[35454]: pgmap v149: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:57:47.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:46 smithi152 ceph-mon[31181]: pgmap v149: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:57:47.478 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:57:47.723 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:57:48.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.253+0000 7f42e2daf640 1 -- 172.21.15.152:0/207092602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42dc101de0 msgr2=0x7f42dc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:48.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.253+0000 7f42e2daf640 1 --2- 172.21.15.152:0/207092602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42dc101de0 0x7f42dc1021c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009900 tx=0x7f42d002a410 comp rx=0 tx=0).stop 2024-09-11T22:57:48.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.253+0000 7f42e2daf640 1 -- 172.21.15.152:0/207092602 shutdown_connections 2024-09-11T22:57:48.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.253+0000 7f42e2daf640 1 --2- 172.21.15.152:0/207092602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42dc106f10 0x7f42dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.253+0000 7f42e2daf640 1 --2- 172.21.15.152:0/207092602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42dc102790 0x7f42dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.253+0000 7f42e2daf640 1 --2- 172.21.15.152:0/207092602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42dc101de0 0x7f42dc1021c0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.253+0000 7f42e2daf640 1 -- 172.21.15.152:0/207092602 >> 172.21.15.152:0/207092602 conn(0x7f42dc0fd8c0 msgr2=0x7f42dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:48.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.253+0000 7f42e2daf640 1 -- 172.21.15.152:0/207092602 shutdown_connections 2024-09-11T22:57:48.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.253+0000 7f42e2daf640 1 -- 172.21.15.152:0/207092602 wait complete. 2024-09-11T22:57:48.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.254+0000 7f42e2daf640 1 Processor -- start 2024-09-11T22:57:48.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.254+0000 7f42e2daf640 1 -- start start 2024-09-11T22:57:48.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.254+0000 7f42e2daf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42dc101de0 0x7f42dc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.254+0000 7f42e2daf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42dc102790 0x7f42dc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.254+0000 7f42e2daf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42dc106f10 0x7f42dc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.254+0000 7f42e2daf640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc111f70 con 0x7f42dc106f10 2024-09-11T22:57:48.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.254+0000 7f42e2daf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc111df0 con 0x7f42dc102790 2024-09-11T22:57:48.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.254+0000 7f42e2daf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc1120f0 con 0x7f42dc101de0 2024-09-11T22:57:48.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.254+0000 7f42e134c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42dc106f10 0x7f42dc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.254+0000 7f42e134c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42dc106f10 0x7f42dc19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45776/0 (socket says 172.21.15.152:45776) 2024-09-11T22:57:48.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.254+0000 7f42e134c640 1 -- 172.21.15.152:0/1279037523 learned_addr learned my addr 172.21.15.152:0/1279037523 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:48.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.254+0000 7f42e0b4b640 1 --2- 172.21.15.152:0/1279037523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42dc101de0 0x7f42dc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.254+0000 7f42dbfff640 1 --2- 172.21.15.152:0/1279037523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42dc102790 0x7f42dc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.255+0000 7f42e134c640 1 -- 172.21.15.152:0/1279037523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42dc101de0 msgr2=0x7f42dc198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:48.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.255+0000 7f42e134c640 1 --2- 172.21.15.152:0/1279037523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42dc101de0 0x7f42dc198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.255+0000 7f42e134c640 1 -- 172.21.15.152:0/1279037523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42dc102790 msgr2=0x7f42dc198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:48.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.255+0000 7f42e134c640 1 --2- 172.21.15.152:0/1279037523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42dc102790 0x7f42dc198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.255+0000 7f42e134c640 1 -- 172.21.15.152:0/1279037523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42dc19d3e0 con 0x7f42dc106f10 2024-09-11T22:57:48.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.255+0000 7f42e134c640 1 --2- 172.21.15.152:0/1279037523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42dc106f10 0x7f42dc19cd00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f42d400eaa0 tx=0x7f42d400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:48.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.255+0000 7f42d9ffb640 1 -- 172.21.15.152:0/1279037523 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d400cdb0 con 0x7f42dc106f10 2024-09-11T22:57:48.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.255+0000 7f42d9ffb640 1 -- 172.21.15.152:0/1279037523 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d4005020 con 0x7f42dc106f10 2024-09-11T22:57:48.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.255+0000 7f42e2daf640 1 -- 172.21.15.152:0/1279037523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42dc19d6d0 con 0x7f42dc106f10 2024-09-11T22:57:48.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.255+0000 7f42d9ffb640 1 -- 172.21.15.152:0/1279037523 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d40116b0 con 0x7f42dc106f10 2024-09-11T22:57:48.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.255+0000 7f42e2daf640 1 -- 172.21.15.152:0/1279037523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42dc1a4fb0 con 0x7f42dc106f10 2024-09-11T22:57:48.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.257+0000 7f42d9ffb640 1 -- 172.21.15.152:0/1279037523 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f42d4005560 con 0x7f42dc106f10 2024-09-11T22:57:48.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e5188740 1 Processor -- start 2024-09-11T22:57:48.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e5188740 1 -- start start 2024-09-11T22:57:48.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42d9ffb640 1 --2- 172.21.15.152:0/1279037523 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f42cc0a2cf0 0x7f42cc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e5188740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42dbfff640 1 --2- 172.21.15.152:0/1279037523 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f42cc0a2cf0 0x7f42cc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e5188740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15cd5d0 0x55d9b15cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e5188740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b15da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42d9ffb640 1 -- 172.21.15.152:0/1279037523 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f42d40cf620 con 0x7f42dc106f10 2024-09-11T22:57:48.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e5188740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d9b15332f0 con 0x55d9b15ceb40 2024-09-11T22:57:48.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e5188740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d9b14dacf0 con 0x55d9b15d8370 2024-09-11T22:57:48.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e5188740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d9b152f2b0 con 0x55d9b15cd5d0 2024-09-11T22:57:48.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e0b4b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e0b4b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45790/0 (socket says 172.21.15.152:45790) 2024-09-11T22:57:48.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e0b4b640 1 -- 172.21.15.152:0/2120234565 learned_addr learned my addr 172.21.15.152:0/2120234565 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:48.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e0b4b640 1 -- 172.21.15.152:0/2120234565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15cd5d0 msgr2=0x55d9b15cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:48.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42dbfff640 1 --2- 172.21.15.152:0/2120234565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15cd5d0 0x55d9b15cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e134c640 1 --2- 172.21.15.152:0/2120234565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b15da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e0b4b640 1 --2- 172.21.15.152:0/2120234565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15cd5d0 0x55d9b15cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e0b4b640 1 -- 172.21.15.152:0/2120234565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 msgr2=0x55d9b15da760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:48.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e0b4b640 1 --2- 172.21.15.152:0/2120234565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b15da760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42dbfff640 1 --2- 172.21.15.152:0/1279037523 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f42cc0a2cf0 0x7f42cc0a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f42c8005fd0 tx=0x7f42c800f360 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:48.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e0b4b640 1 -- 172.21.15.152:0/2120234565 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9b14bf3a0 con 0x55d9b15ceb40 2024-09-11T22:57:48.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.258+0000 7f42e134c640 1 --2- 172.21.15.152:0/2120234565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b15da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:57:48.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.259+0000 7f42e0b4b640 1 --2- 172.21.15.152:0/2120234565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15cd090 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009900 tx=0x7f42d0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c57798094779805 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:48.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.259+0000 7f42c6ffd640 1 -- 172.21.15.152:0/2120234565 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0040070 con 0x55d9b15ceb40 2024-09-11T22:57:48.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.259+0000 7f42c6ffd640 1 -- 172.21.15.152:0/2120234565 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d00319f0 con 0x55d9b15ceb40 2024-09-11T22:57:48.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.259+0000 7f42c6ffd640 1 -- 172.21.15.152:0/2120234565 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0035040 con 0x55d9b15ceb40 2024-09-11T22:57:48.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 -- 172.21.15.152:0/2120234565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 msgr2=0x55d9b15cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:48.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 --2- 172.21.15.152:0/2120234565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15cd090 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009900 tx=0x7f42d0005e80 comp rx=0 tx=0).stop 2024-09-11T22:57:48.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 -- 172.21.15.152:0/2120234565 shutdown_connections 2024-09-11T22:57:48.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 --2- 172.21.15.152:0/2120234565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b15da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 --2- 172.21.15.152:0/2120234565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15cd5d0 0x55d9b15cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 --2- 172.21.15.152:0/2120234565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15cd090 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 -- 172.21.15.152:0/2120234565 >> 172.21.15.152:0/2120234565 conn(0x55d9b1596310 msgr2=0x55d9b15c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:48.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 -- 172.21.15.152:0/2120234565 shutdown_connections 2024-09-11T22:57:48.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 -- 172.21.15.152:0/2120234565 wait complete. 2024-09-11T22:57:48.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 -- start start 2024-09-11T22:57:48.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 Processor -- start 2024-09-11T22:57:48.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15cd5d0 0x55d9b15d7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15ceb40 0x55d9b15d4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b15d54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d9b15db010 con 0x55d9b15cd5d0 2024-09-11T22:57:48.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d9b14ee060 con 0x55d9b15d8370 2024-09-11T22:57:48.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.260+0000 7f42e5188740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d9b15db190 con 0x55d9b15ceb40 2024-09-11T22:57:48.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42e5188740 10 client.0 _ll_register_callbacks cb 0x55d9b12418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:48.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42e0b4b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15cd5d0 0x55d9b15d7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42e0b4b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15cd5d0 0x55d9b15d7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45802/0 (socket says 172.21.15.152:45802) 2024-09-11T22:57:48.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42e0b4b640 1 -- 172.21.15.152:0/128457962 learned_addr learned my addr 172.21.15.152:0/128457962 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:48.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42e134c640 1 --2- 172.21.15.152:0/128457962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b15d54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42dbfff640 1 --2- 172.21.15.152:0/128457962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15ceb40 0x55d9b15d4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42e0b4b640 1 -- 172.21.15.152:0/128457962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15ceb40 msgr2=0x55d9b15d4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:48.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42e0b4b640 1 --2- 172.21.15.152:0/128457962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15ceb40 0x55d9b15d4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42e0b4b640 1 -- 172.21.15.152:0/128457962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 msgr2=0x55d9b15d54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42e0b4b640 1 --2- 172.21.15.152:0/128457962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b15d54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42e0b4b640 1 -- 172.21.15.152:0/128457962 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9b12a43c0 con 0x55d9b15cd5d0 2024-09-11T22:57:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42e134c640 1 --2- 172.21.15.152:0/128457962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b15d54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42e0b4b640 1 --2- 172.21.15.152:0/128457962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15cd5d0 0x55d9b15d7cc0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f42d0006010 tx=0x7f42d00052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669b6c31b14c4c4d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42bbfff640 1 -- 172.21.15.152:0/128457962 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d0040030 con 0x55d9b15cd5d0 2024-09-11T22:57:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42bbfff640 1 -- 172.21.15.152:0/128457962 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f42d0004460 con 0x55d9b15cd5d0 2024-09-11T22:57:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42e5188740 1 -- 172.21.15.152:0/128457962 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42d00319f0 con 0x55d9b15cd5d0 2024-09-11T22:57:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42bbfff640 1 -- 172.21.15.152:0/128457962 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42d0056050 con 0x55d9b15cd5d0 2024-09-11T22:57:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42e5188740 10 client.16764 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.261+0000 7f42e5188740 1 -- 172.21.15.152:0/128457962 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9b15e6990 con 0x55d9b15cd5d0 2024-09-11T22:57:48.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.262+0000 7f42bbfff640 1 -- 172.21.15.152:0/128457962 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d00319f0 con 0x55d9b15cd5d0 2024-09-11T22:57:48.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.262+0000 7f42bbfff640 1 -- 172.21.15.152:0/128457962 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f42d0003960 con 0x55d9b15cd5d0 2024-09-11T22:57:48.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.262+0000 7f42e5188740 10 client.16764 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:48.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.262+0000 7f42e5188740 10 client.16764 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:57:48.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.262+0000 7f42e5188740 20 client.16764 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:48.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.262+0000 7f42e5188740 1 --2- 172.21.15.152:0/128457962 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55d9b15e7ce0 0x55d9b15ea0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.262+0000 7f42e5188740 4 client.16764 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:57:48.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.262+0000 7f42e5188740 1 -- 172.21.15.152:0/128457962 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9b1540860 con 0x55d9b15e7ce0 2024-09-11T22:57:48.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.262+0000 7f42dbfff640 1 --2- 172.21.15.152:0/128457962 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55d9b15e7ce0 0x55d9b15ea0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.262+0000 7f42dbfff640 1 --2- 172.21.15.152:0/128457962 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55d9b15e7ce0 0x55d9b15ea0d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d9186dc9c1418991 server_cookie=c23e9c62b022f3b5 in_seq=0 out_seq=0 2024-09-11T22:57:48.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.262+0000 7f42bbfff640 10 client.16764 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:57:48.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.263+0000 7f42bbfff640 1 -- 172.21.15.152:0/128457962 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9b1540860 con 0x55d9b15e7ce0 2024-09-11T22:57:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.263+0000 7f42bbfff640 10 client.16764 handle_command_reply: tid=0 2024-09-11T22:57:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.263+0000 7f42e5188740 1 client.16764 shutdown 2024-09-11T22:57:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.263+0000 7f42e5188740 1 -- 172.21.15.152:0/128457962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15cd5d0 msgr2=0x55d9b15d7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.263+0000 7f42e5188740 1 --2- 172.21.15.152:0/128457962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15cd5d0 0x55d9b15d7cc0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f42d0006010 tx=0x7f42d00052d0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.263+0000 7f42e5188740 1 -- 172.21.15.152:0/128457962 shutdown_connections 2024-09-11T22:57:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.263+0000 7f42e5188740 1 --2- 172.21.15.152:0/128457962 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55d9b15e7ce0 0x55d9b15ea0d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.263+0000 7f42e5188740 1 --2- 172.21.15.152:0/128457962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b15d54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.263+0000 7f42bbfff640 0 client.16764 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:57:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.263+0000 7f42e5188740 1 --2- 172.21.15.152:0/128457962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15ceb40 0x55d9b15d4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.263+0000 7f42e5188740 1 --2- 172.21.15.152:0/128457962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15cd5d0 0x55d9b15d7cc0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.263+0000 7f42e5188740 1 -- 172.21.15.152:0/128457962 >> 172.21.15.152:0/128457962 conn(0x55d9b1596310 msgr2=0x55d9b15cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:48.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.264+0000 7f42e5188740 1 -- 172.21.15.152:0/128457962 shutdown_connections 2024-09-11T22:57:48.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.264+0000 7f42e5188740 1 -- 172.21.15.152:0/128457962 wait complete. 2024-09-11T22:57:48.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.264+0000 7f42e5188740 20 client.16764 trim_cache size 0 max 16384 2024-09-11T22:57:48.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42e5188740 1 Processor -- start 2024-09-11T22:57:48.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42e5188740 1 -- start start 2024-09-11T22:57:48.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42e5188740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15cd5d0 0x55d9b15c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42e5188740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42e5188740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b15d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42e5188740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc1ab5e0 con 0x55d9b15ceb40 2024-09-11T22:57:48.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42e5188740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc1ad120 con 0x55d9b15d8370 2024-09-11T22:57:48.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42e5188740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc1abcb0 con 0x55d9b15cd5d0 2024-09-11T22:57:48.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42dbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42dbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15d0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45816/0 (socket says 172.21.15.152:45816) 2024-09-11T22:57:48.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42dbfff640 1 -- 172.21.15.152:0/4268125314 learned_addr learned my addr 172.21.15.152:0/4268125314 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:48.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42e0b4b640 1 --2- 172.21.15.152:0/4268125314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15cd5d0 0x55d9b15c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42e134c640 1 --2- 172.21.15.152:0/4268125314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b15d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42dbfff640 1 -- 172.21.15.152:0/4268125314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15cd5d0 msgr2=0x55d9b15c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:48.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42dbfff640 1 --2- 172.21.15.152:0/4268125314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15cd5d0 0x55d9b15c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42dbfff640 1 -- 172.21.15.152:0/4268125314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 msgr2=0x55d9b15d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:48.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.277+0000 7f42dbfff640 1 --2- 172.21.15.152:0/4268125314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b15d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.278+0000 7f42dbfff640 1 -- 172.21.15.152:0/4268125314 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9b15d62f0 con 0x55d9b15ceb40 2024-09-11T22:57:48.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.278+0000 7f42dbfff640 1 --2- 172.21.15.152:0/4268125314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15d0120 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f42c80058c0 tx=0x7f42c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefb07ecc9a1f64f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:48.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.278+0000 7f42c67fc640 1 -- 172.21.15.152:0/4268125314 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8012810 con 0x55d9b15ceb40 2024-09-11T22:57:48.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.278+0000 7f42c67fc640 1 -- 172.21.15.152:0/4268125314 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42c802a430 con 0x55d9b15ceb40 2024-09-11T22:57:48.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.278+0000 7f42c67fc640 1 -- 172.21.15.152:0/4268125314 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8029610 con 0x55d9b15ceb40 2024-09-11T22:57:48.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 -- 172.21.15.152:0/4268125314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 msgr2=0x55d9b15d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:48.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 --2- 172.21.15.152:0/4268125314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15d0120 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f42c80058c0 tx=0x7f42c8014000 comp rx=0 tx=0).stop 2024-09-11T22:57:48.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 -- 172.21.15.152:0/4268125314 shutdown_connections 2024-09-11T22:57:48.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 --2- 172.21.15.152:0/4268125314 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b15d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 --2- 172.21.15.152:0/4268125314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15d0120 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 --2- 172.21.15.152:0/4268125314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15cd5d0 0x55d9b15c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 -- 172.21.15.152:0/4268125314 >> 172.21.15.152:0/4268125314 conn(0x55d9b1596310 msgr2=0x55d9b15c8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:48.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 -- 172.21.15.152:0/4268125314 shutdown_connections 2024-09-11T22:57:48.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 -- 172.21.15.152:0/4268125314 wait complete. 2024-09-11T22:57:48.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 -- start start 2024-09-11T22:57:48.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 Processor -- start 2024-09-11T22:57:48.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15cd5d0 0x55d9b15dc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b1651730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc1aca50 con 0x55d9b15ceb40 2024-09-11T22:57:48.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc1ac380 con 0x55d9b15d8370 2024-09-11T22:57:48.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc1a95a0 con 0x55d9b15cd5d0 2024-09-11T22:57:48.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.279+0000 7f42e5188740 10 client.0 _ll_register_callbacks cb 0x7f42d0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:48.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42e0b4b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15cd5d0 0x55d9b15dc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42dbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42e0b4b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15cd5d0 0x55d9b15dc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:56492/0 (socket says 172.21.15.152:56492) 2024-09-11T22:57:48.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42e0b4b640 1 -- 172.21.15.152:0/1593623612 learned_addr learned my addr 172.21.15.152:0/1593623612 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:48.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42e134c640 1 --2- 172.21.15.152:0/1593623612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b1651730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42dbfff640 1 -- 172.21.15.152:0/1593623612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15cd5d0 msgr2=0x55d9b15dc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:48.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42dbfff640 1 --2- 172.21.15.152:0/1593623612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15cd5d0 0x55d9b15dc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42dbfff640 1 -- 172.21.15.152:0/1593623612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 msgr2=0x55d9b1651730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:48.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42dbfff640 1 --2- 172.21.15.152:0/1593623612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b1651730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42dbfff640 1 -- 172.21.15.152:0/1593623612 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9b1651c70 con 0x55d9b15ceb40 2024-09-11T22:57:48.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42e134c640 1 --2- 172.21.15.152:0/1593623612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b1651730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:48.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42dbfff640 1 --2- 172.21.15.152:0/1593623612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15dcda0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x55d9b15c9690 tx=0x7f42c800fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0485847da4152 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:48.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:48.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42bbfff640 1 -- 172.21.15.152:0/1593623612 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c802ab30 con 0x55d9b15ceb40 2024-09-11T22:57:48.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42e5188740 1 -- 172.21.15.152:0/1593623612 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42c802a430 con 0x55d9b15ceb40 2024-09-11T22:57:48.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42bbfff640 1 -- 172.21.15.152:0/1593623612 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f42c8005030 con 0x55d9b15ceb40 2024-09-11T22:57:48.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42bbfff640 1 -- 172.21.15.152:0/1593623612 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f42c8005230 con 0x55d9b15ceb40 2024-09-11T22:57:48.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.280+0000 7f42e5188740 10 client.16770 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:48.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.281+0000 7f42e5188740 1 -- 172.21.15.152:0/1593623612 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9b1652f10 con 0x55d9b15ceb40 2024-09-11T22:57:48.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.281+0000 7f42bbfff640 1 -- 172.21.15.152:0/1593623612 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42c802a430 con 0x55d9b15ceb40 2024-09-11T22:57:48.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.281+0000 7f42bbfff640 1 -- 172.21.15.152:0/1593623612 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f42c8031030 con 0x55d9b15ceb40 2024-09-11T22:57:48.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.281+0000 7f42e5188740 10 client.16770 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:48.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.281+0000 7f42e5188740 10 client.16770 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:57:48.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.281+0000 7f42e5188740 20 client.16770 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:48.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.281+0000 7f42e5188740 1 --2- 172.21.15.152:0/1593623612 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55d9b16535c0 0x55d9b16559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:48.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.281+0000 7f42e5188740 4 client.16770 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:57:48.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.281+0000 7f42e5188740 1 -- 172.21.15.152:0/1593623612 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d9b153ad70 con 0x55d9b16535c0 2024-09-11T22:57:48.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.281+0000 7f42e0b4b640 1 --2- 172.21.15.152:0/1593623612 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55d9b16535c0 0x55d9b16559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:48.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42e0b4b640 1 --2- 172.21.15.152:0/1593623612 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55d9b16535c0 0x55d9b16559e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=569a3b3ef4e7bca4 server_cookie=a8ea4ed63a150650 in_seq=0 out_seq=0 2024-09-11T22:57:48.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42bbfff640 10 client.16770 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:57:48.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42bbfff640 1 -- 172.21.15.152:0/1593623612 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9b153ad70 con 0x55d9b16535c0 2024-09-11T22:57:48.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42bbfff640 10 client.16770 handle_command_reply: tid=0 2024-09-11T22:57:48.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42e5188740 1 client.16770 shutdown 2024-09-11T22:57:48.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42e5188740 1 -- 172.21.15.152:0/1593623612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 msgr2=0x55d9b15dcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:48.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42e5188740 1 --2- 172.21.15.152:0/1593623612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15dcda0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x55d9b15c9690 tx=0x7f42c800fd10 comp rx=0 tx=0).stop 2024-09-11T22:57:48.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42e5188740 1 -- 172.21.15.152:0/1593623612 shutdown_connections 2024-09-11T22:57:48.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42e5188740 1 --2- 172.21.15.152:0/1593623612 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55d9b16535c0 0x55d9b16559e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42e5188740 1 --2- 172.21.15.152:0/1593623612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d9b15d8370 0x55d9b1651730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42e5188740 1 --2- 172.21.15.152:0/1593623612 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d9b15ceb40 0x55d9b15dcda0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42bbfff640 0 client.16770 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:57:48.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42e5188740 1 --2- 172.21.15.152:0/1593623612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d9b15cd5d0 0x55d9b15dc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42e5188740 1 -- 172.21.15.152:0/1593623612 >> 172.21.15.152:0/1593623612 conn(0x55d9b1596310 msgr2=0x55d9b15c8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:48.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42e5188740 1 -- 172.21.15.152:0/1593623612 shutdown_connections 2024-09-11T22:57:48.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42e5188740 1 -- 172.21.15.152:0/1593623612 wait complete. 2024-09-11T22:57:48.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.282+0000 7f42e5188740 20 client.16770 trim_cache size 0 max 16384 2024-09-11T22:57:48.297 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:57:48.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.283+0000 7f42e2daf640 1 -- 172.21.15.152:0/1279037523 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f42cc0a2cf0 msgr2=0x7f42cc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:48.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.283+0000 7f42e2daf640 1 --2- 172.21.15.152:0/1279037523 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f42cc0a2cf0 0x7f42cc0a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f42c8005fd0 tx=0x7f42c800f360 comp rx=0 tx=0).stop 2024-09-11T22:57:48.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.283+0000 7f42e2daf640 1 -- 172.21.15.152:0/1279037523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42dc106f10 msgr2=0x7f42dc19cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:48.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.283+0000 7f42e2daf640 1 --2- 172.21.15.152:0/1279037523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42dc106f10 0x7f42dc19cd00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f42d400eaa0 tx=0x7f42d400ef60 comp rx=0 tx=0).stop 2024-09-11T22:57:48.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.283+0000 7f42e2daf640 1 -- 172.21.15.152:0/1279037523 shutdown_connections 2024-09-11T22:57:48.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.283+0000 7f42e2daf640 1 --2- 172.21.15.152:0/1279037523 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f42cc0a2cf0 0x7f42cc0a51b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.283+0000 7f42e2daf640 1 --2- 172.21.15.152:0/1279037523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42dc106f10 0x7f42dc19cd00 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.283+0000 7f42e2daf640 1 --2- 172.21.15.152:0/1279037523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f42dc102790 0x7f42dc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.283+0000 7f42e2daf640 1 --2- 172.21.15.152:0/1279037523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42dc101de0 0x7f42dc198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:48.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.283+0000 7f42e2daf640 1 -- 172.21.15.152:0/1279037523 >> 172.21.15.152:0/1279037523 conn(0x7f42dc0fd8c0 msgr2=0x7f42dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:48.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.283+0000 7f42e2daf640 1 -- 172.21.15.152:0/1279037523 shutdown_connections 2024-09-11T22:57:48.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:48.283+0000 7f42e2daf640 1 -- 172.21.15.152:0/1279037523 wait complete. 2024-09-11T22:57:49.074 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:57:49.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:48 smithi191 ceph-mon[35211]: pgmap v150: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:49.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:48 smithi204 ceph-mon[35454]: pgmap v150: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:49.325 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:57:49.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:48 smithi152 ceph-mon[31181]: pgmap v150: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:49.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.854+0000 7f1a694d1640 1 -- 172.21.15.152:0/588773551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a64101de0 msgr2=0x7f1a641021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:49.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.854+0000 7f1a694d1640 1 --2- 172.21.15.152:0/588773551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a64101de0 0x7f1a641021c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009900 tx=0x7f1a5402a410 comp rx=0 tx=0).stop 2024-09-11T22:57:49.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.854+0000 7f1a694d1640 1 -- 172.21.15.152:0/588773551 shutdown_connections 2024-09-11T22:57:49.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.854+0000 7f1a694d1640 1 --2- 172.21.15.152:0/588773551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a64106f10 0x7f1a6410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.854+0000 7f1a694d1640 1 --2- 172.21.15.152:0/588773551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a64102790 0x7f1a641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.854+0000 7f1a694d1640 1 --2- 172.21.15.152:0/588773551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a64101de0 0x7f1a641021c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.854+0000 7f1a694d1640 1 -- 172.21.15.152:0/588773551 >> 172.21.15.152:0/588773551 conn(0x7f1a640fd8c0 msgr2=0x7f1a640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:49.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.854+0000 7f1a694d1640 1 -- 172.21.15.152:0/588773551 shutdown_connections 2024-09-11T22:57:49.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.854+0000 7f1a694d1640 1 -- 172.21.15.152:0/588773551 wait complete. 2024-09-11T22:57:49.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.855+0000 7f1a694d1640 1 Processor -- start 2024-09-11T22:57:49.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.855+0000 7f1a694d1640 1 -- start start 2024-09-11T22:57:49.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.855+0000 7f1a694d1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a64101de0 0x7f1a641983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.855+0000 7f1a694d1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a64102790 0x7f1a64198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.855+0000 7f1a694d1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a64106f10 0x7f1a6419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.855+0000 7f1a694d1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1a64111f70 con 0x7f1a64106f10 2024-09-11T22:57:49.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.855+0000 7f1a694d1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1a64111df0 con 0x7f1a64102790 2024-09-11T22:57:49.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.855+0000 7f1a694d1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641120f0 con 0x7f1a64101de0 2024-09-11T22:57:49.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.855+0000 7f1a637fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a64106f10 0x7f1a6419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a64101de0 0x7f1a641983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a64101de0 0x7f1a641983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:60714/0 (socket says 172.21.15.152:60714) 2024-09-11T22:57:49.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a62ffd640 1 -- 172.21.15.152:0/2559925462 learned_addr learned my addr 172.21.15.152:0/2559925462 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:49.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a637fe640 1 --2- 172.21.15.152:0/2559925462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a64106f10 0x7f1a6419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43150/0 (socket says 172.21.15.152:43150) 2024-09-11T22:57:49.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a5a7fc640 1 --2- 172.21.15.152:0/2559925462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a64102790 0x7f1a64198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a637fe640 1 -- 172.21.15.152:0/2559925462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a64101de0 msgr2=0x7f1a641983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:49.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a637fe640 1 --2- 172.21.15.152:0/2559925462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a64101de0 0x7f1a641983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a637fe640 1 -- 172.21.15.152:0/2559925462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a64102790 msgr2=0x7f1a64198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:49.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a637fe640 1 --2- 172.21.15.152:0/2559925462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a64102790 0x7f1a64198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a637fe640 1 -- 172.21.15.152:0/2559925462 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6419d380 con 0x7f1a64106f10 2024-09-11T22:57:49.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a5a7fc640 1 --2- 172.21.15.152:0/2559925462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a64102790 0x7f1a64198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:57:49.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a637fe640 1 --2- 172.21.15.152:0/2559925462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a64106f10 0x7f1a6419cca0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00ca30 tx=0x7f1a5c00cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:49.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a60ff9640 1 -- 172.21.15.152:0/2559925462 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c007d90 con 0x7f1a64106f10 2024-09-11T22:57:49.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a60ff9640 1 -- 172.21.15.152:0/2559925462 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a5c005020 con 0x7f1a64106f10 2024-09-11T22:57:49.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a694d1640 1 -- 172.21.15.152:0/2559925462 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6419d670 con 0x7f1a64106f10 2024-09-11T22:57:49.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a60ff9640 1 -- 172.21.15.152:0/2559925462 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c00f780 con 0x7f1a64106f10 2024-09-11T22:57:49.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.856+0000 7f1a694d1640 1 -- 172.21.15.152:0/2559925462 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a641a4f50 con 0x7f1a64106f10 2024-09-11T22:57:49.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.859+0000 7f1a60ff9640 1 -- 172.21.15.152:0/2559925462 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1a5c0051d0 con 0x7f1a64106f10 2024-09-11T22:57:49.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.859+0000 7f1a6b8aa740 1 Processor -- start 2024-09-11T22:57:49.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.859+0000 7f1a6b8aa740 1 -- start start 2024-09-11T22:57:49.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.859+0000 7f1a6b8aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c21560b40 0x564c2155f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.859+0000 7f1a6b8aa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2155f5d0 0x564c2155fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.859+0000 7f1a6b8aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c2156a370 0x564c2156c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.859+0000 7f1a6b8aa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x564c214c52f0 con 0x564c2155f5d0 2024-09-11T22:57:49.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.859+0000 7f1a6b8aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564c2146ccf0 con 0x564c21560b40 2024-09-11T22:57:49.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.859+0000 7f1a6b8aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564c214c12b0 con 0x564c2156a370 2024-09-11T22:57:49.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2155f5d0 0x564c2155fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2155f5d0 0x564c2155fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43166/0 (socket says 172.21.15.152:43166) 2024-09-11T22:57:49.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a62ffd640 1 -- 172.21.15.152:0/1632841593 learned_addr learned my addr 172.21.15.152:0/1632841593 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:49.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a60ff9640 1 --2- 172.21.15.152:0/2559925462 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1a500a2cf0 0x7f1a500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a5a7fc640 1 --2- 172.21.15.152:0/1632841593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c21560b40 0x564c2155f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a637fe640 1 --2- 172.21.15.152:0/2559925462 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1a500a2cf0 0x7f1a500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a5a7fc640 1 --2- 172.21.15.152:0/1632841593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c2156a370 0x564c2156c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a62ffd640 1 -- 172.21.15.152:0/1632841593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c2156a370 msgr2=0x564c2156c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:49.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a62ffd640 1 --2- 172.21.15.152:0/1632841593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c2156a370 0x564c2156c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a62ffd640 1 -- 172.21.15.152:0/1632841593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c21560b40 msgr2=0x564c2155f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:49.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a62ffd640 1 --2- 172.21.15.152:0/1632841593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c21560b40 0x564c2155f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a62ffd640 1 -- 172.21.15.152:0/1632841593 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c214513a0 con 0x564c2155f5d0 2024-09-11T22:57:49.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a60ff9640 1 -- 172.21.15.152:0/2559925462 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f1a5c0cf9d0 con 0x7f1a64106f10 2024-09-11T22:57:49.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a62ffd640 1 --2- 172.21.15.152:0/1632841593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2155f5d0 0x564c2155fa10 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009a20 tx=0x7f1a54031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285cb2fa9a62989c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:49.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a637fe640 1 --2- 172.21.15.152:0/2559925462 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1a500a2cf0 0x7f1a500a51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0a4640 tx=0x7f1a5c06dec0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:49.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a597fa640 1 -- 172.21.15.152:0/1632841593 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54040070 con 0x564c2155f5d0 2024-09-11T22:57:49.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a597fa640 1 -- 172.21.15.152:0/1632841593 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a54005500 con 0x564c2155f5d0 2024-09-11T22:57:49.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.860+0000 7f1a597fa640 1 -- 172.21.15.152:0/1632841593 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54035040 con 0x564c2155f5d0 2024-09-11T22:57:49.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.861+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/1632841593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2155f5d0 msgr2=0x564c2155fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:49.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.861+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/1632841593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2155f5d0 0x564c2155fa10 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009a20 tx=0x7f1a54031770 comp rx=0 tx=0).stop 2024-09-11T22:57:49.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.861+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/1632841593 shutdown_connections 2024-09-11T22:57:49.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.861+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/1632841593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c2156a370 0x564c2156c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.861+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/1632841593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2155f5d0 0x564c2155fa10 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.861+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/1632841593 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c21560b40 0x564c2155f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.861+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/1632841593 >> 172.21.15.152:0/1632841593 conn(0x564c21528310 msgr2=0x564c2155bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:49.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.861+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/1632841593 shutdown_connections 2024-09-11T22:57:49.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.861+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/1632841593 wait complete. 2024-09-11T22:57:49.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a6b8aa740 1 -- start start 2024-09-11T22:57:49.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a6b8aa740 1 Processor -- start 2024-09-11T22:57:49.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a6b8aa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2155f5d0 0x564c21569ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a6b8aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c21560b40 0x564c21566f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a6b8aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c2156a370 0x564c215674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a6b8aa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x564c2156d010 con 0x564c2155f5d0 2024-09-11T22:57:49.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a6b8aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564c21480060 con 0x564c2156a370 2024-09-11T22:57:49.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a6b8aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564c2156d190 con 0x564c21560b40 2024-09-11T22:57:49.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a6b8aa740 10 client.0 _ll_register_callbacks cb 0x564c211d38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:49.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a5a7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2155f5d0 0x564c21569ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a5a7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2155f5d0 0x564c21569ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43182/0 (socket says 172.21.15.152:43182) 2024-09-11T22:57:49.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a5a7fc640 1 -- 172.21.15.152:0/4268437127 learned_addr learned my addr 172.21.15.152:0/4268437127 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:49.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a62ffd640 1 --2- 172.21.15.152:0/4268437127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c21560b40 0x564c21566f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a5a7fc640 1 -- 172.21.15.152:0/4268437127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c21560b40 msgr2=0x564c21566f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:49.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a5a7fc640 1 --2- 172.21.15.152:0/4268437127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c21560b40 0x564c21566f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a5a7fc640 1 -- 172.21.15.152:0/4268437127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c2156a370 msgr2=0x564c215674f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:49.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a5a7fc640 1 --2- 172.21.15.152:0/4268437127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c2156a370 0x564c215674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a5a7fc640 1 -- 172.21.15.152:0/4268437127 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c212363c0 con 0x564c2155f5d0 2024-09-11T22:57:49.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a5a7fc640 1 --2- 172.21.15.152:0/4268437127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2155f5d0 0x564c21569ce0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c013850 tx=0x7f1a4c013d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dacbce738c5ade server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:49.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.862+0000 7f1a4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:49.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.863+0000 7f1a4a7fc640 1 -- 172.21.15.152:0/4268437127 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c011e10 con 0x564c2155f5d0 2024-09-11T22:57:49.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.863+0000 7f1a4a7fc640 1 -- 172.21.15.152:0/4268437127 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f1a4c01d420 con 0x564c2155f5d0 2024-09-11T22:57:49.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.863+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/4268437127 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a54005500 con 0x564c2155f5d0 2024-09-11T22:57:49.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.863+0000 7f1a4a7fc640 1 -- 172.21.15.152:0/4268437127 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1a4c021020 con 0x564c2155f5d0 2024-09-11T22:57:49.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.863+0000 7f1a6b8aa740 10 client.16782 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:49.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.863+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/4268437127 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c21578a80 con 0x564c2155f5d0 2024-09-11T22:57:49.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.863+0000 7f1a4a7fc640 1 -- 172.21.15.152:0/4268437127 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a54005500 con 0x564c2155f5d0 2024-09-11T22:57:49.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.863+0000 7f1a4a7fc640 1 -- 172.21.15.152:0/4268437127 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1a4c026840 con 0x564c2155f5d0 2024-09-11T22:57:49.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.863+0000 7f1a6b8aa740 10 client.16782 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:49.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.863+0000 7f1a6b8aa740 10 client.16782 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:57:49.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.863+0000 7f1a6b8aa740 20 client.16782 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:49.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.863+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/4268437127 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x564c21579e90 0x564c2157c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.863+0000 7f1a6b8aa740 4 client.16782 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:57:49.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.863+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/4268437127 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c214d2860 con 0x564c21579e90 2024-09-11T22:57:49.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.864+0000 7f1a62ffd640 1 --2- 172.21.15.152:0/4268437127 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x564c21579e90 0x564c2157c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.864+0000 7f1a4a7fc640 10 client.16782 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:57:49.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.864+0000 7f1a62ffd640 1 --2- 172.21.15.152:0/4268437127 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x564c21579e90 0x564c2157c280 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5916ff0e8567cb1 server_cookie=66400528512978e in_seq=0 out_seq=0 2024-09-11T22:57:49.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.865+0000 7f1a4a7fc640 1 -- 172.21.15.152:0/4268437127 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c214d2860 con 0x564c21579e90 2024-09-11T22:57:49.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.865+0000 7f1a4a7fc640 10 client.16782 handle_command_reply: tid=0 2024-09-11T22:57:49.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.865+0000 7f1a6b8aa740 1 client.16782 shutdown 2024-09-11T22:57:49.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.865+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/4268437127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2155f5d0 msgr2=0x564c21569ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:49.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.865+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/4268437127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2155f5d0 0x564c21569ce0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c013850 tx=0x7f1a4c013d10 comp rx=0 tx=0).stop 2024-09-11T22:57:49.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.865+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/4268437127 shutdown_connections 2024-09-11T22:57:49.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.865+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/4268437127 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x564c21579e90 0x564c2157c280 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.865+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/4268437127 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c2156a370 0x564c215674f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.865+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/4268437127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c21560b40 0x564c21566f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.865+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/4268437127 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c2155f5d0 0x564c21569ce0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.865+0000 7f1a4a7fc640 0 client.16782 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:57:49.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.865+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/4268437127 >> 172.21.15.152:0/4268437127 conn(0x564c21528310 msgr2=0x564c21561930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:49.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.865+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/4268437127 shutdown_connections 2024-09-11T22:57:49.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.865+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/4268437127 wait complete. 2024-09-11T22:57:49.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.865+0000 7f1a6b8aa740 20 client.16782 trim_cache size 0 max 16384 2024-09-11T22:57:49.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.878+0000 7f1a6b8aa740 1 Processor -- start 2024-09-11T22:57:49.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.878+0000 7f1a6b8aa740 1 -- start start 2024-09-11T22:57:49.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a6b8aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c2155f5d0 0x564c215762b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a6b8aa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c21560b40 0x564c21566850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a6b8aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c2156a370 0x564c2155b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a6b8aa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641ad800 con 0x564c21560b40 2024-09-11T22:57:49.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a6b8aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641af340 con 0x564c2155f5d0 2024-09-11T22:57:49.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a6b8aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641aded0 con 0x564c2156a370 2024-09-11T22:57:49.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c21560b40 0x564c21566850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c21560b40 0x564c21566850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43184/0 (socket says 172.21.15.152:43184) 2024-09-11T22:57:49.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a62ffd640 1 -- 172.21.15.152:0/2801217228 learned_addr learned my addr 172.21.15.152:0/2801217228 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:49.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a5a7fc640 1 --2- 172.21.15.152:0/2801217228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c2156a370 0x564c2155b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a5a7fc640 1 --2- 172.21.15.152:0/2801217228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c2155f5d0 0x564c215762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a62ffd640 1 -- 172.21.15.152:0/2801217228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c2156a370 msgr2=0x564c2155b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:49.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a62ffd640 1 --2- 172.21.15.152:0/2801217228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c2156a370 0x564c2155b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a62ffd640 1 -- 172.21.15.152:0/2801217228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c2155f5d0 msgr2=0x564c215762b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:49.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a62ffd640 1 --2- 172.21.15.152:0/2801217228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c2155f5d0 0x564c215762b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a62ffd640 1 -- 172.21.15.152:0/2801217228 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c2155ee70 con 0x564c21560b40 2024-09-11T22:57:49.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a5a7fc640 1 --2- 172.21.15.152:0/2801217228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c2155f5d0 0x564c215762b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:57:49.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a62ffd640 1 --2- 172.21.15.152:0/2801217228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c21560b40 0x564c21566850 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1a540317e0 tx=0x7f1a54005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e83589b4850f977 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:49.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a58ff9640 1 -- 172.21.15.152:0/2801217228 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54040040 con 0x564c21560b40 2024-09-11T22:57:49.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a58ff9640 1 -- 172.21.15.152:0/2801217228 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a5404fb20 con 0x564c21560b40 2024-09-11T22:57:49.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.879+0000 7f1a58ff9640 1 -- 172.21.15.152:0/2801217228 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54035040 con 0x564c21560b40 2024-09-11T22:57:49.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.880+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/2801217228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c21560b40 msgr2=0x564c21566850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:49.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.880+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/2801217228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c21560b40 0x564c21566850 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1a540317e0 tx=0x7f1a54005a20 comp rx=0 tx=0).stop 2024-09-11T22:57:49.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.880+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/2801217228 shutdown_connections 2024-09-11T22:57:49.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.880+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/2801217228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c2156a370 0x564c2155b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.880+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/2801217228 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c21560b40 0x564c21566850 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.880+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/2801217228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c2155f5d0 0x564c215762b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.880+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/2801217228 >> 172.21.15.152:0/2801217228 conn(0x564c21528310 msgr2=0x564c215622c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:49.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.881+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/2801217228 shutdown_connections 2024-09-11T22:57:49.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.881+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/2801217228 wait complete. 2024-09-11T22:57:49.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.881+0000 7f1a6b8aa740 1 -- start start 2024-09-11T22:57:49.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.881+0000 7f1a6b8aa740 1 Processor -- start 2024-09-11T22:57:49.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.881+0000 7f1a6b8aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c2155f5d0 0x564c21572420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.881+0000 7f1a6b8aa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c21560b40 0x564c21572960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.881+0000 7f1a6b8aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c2156a370 0x564c215635c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.881+0000 7f1a6b8aa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641aec70 con 0x564c21560b40 2024-09-11T22:57:49.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.881+0000 7f1a6b8aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641ae5a0 con 0x564c2156a370 2024-09-11T22:57:49.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.881+0000 7f1a6b8aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1a641ab7c0 con 0x564c2155f5d0 2024-09-11T22:57:49.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.881+0000 7f1a6b8aa740 10 client.0 _ll_register_callbacks cb 0x7f1a4c013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:49.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.881+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c21560b40 0x564c21572960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a62ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c21560b40 0x564c21572960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43192/0 (socket says 172.21.15.152:43192) 2024-09-11T22:57:49.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a62ffd640 1 -- 172.21.15.152:0/3891198708 learned_addr learned my addr 172.21.15.152:0/3891198708 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:49.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a5a7fc640 1 --2- 172.21.15.152:0/3891198708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c2155f5d0 0x564c21572420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a5a7fc640 1 --2- 172.21.15.152:0/3891198708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c2156a370 0x564c215635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a62ffd640 1 -- 172.21.15.152:0/3891198708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c2155f5d0 msgr2=0x564c21572420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:49.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a62ffd640 1 --2- 172.21.15.152:0/3891198708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c2155f5d0 0x564c21572420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a62ffd640 1 -- 172.21.15.152:0/3891198708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c2156a370 msgr2=0x564c215635c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:49.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a62ffd640 1 --2- 172.21.15.152:0/3891198708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c2156a370 0x564c215635c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a62ffd640 1 -- 172.21.15.152:0/3891198708 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c21563c00 con 0x564c21560b40 2024-09-11T22:57:49.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a62ffd640 1 --2- 172.21.15.152:0/3891198708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c21560b40 0x564c21572960 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1a54005c40 tx=0x7f1a54031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677b4935001afab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:49.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:49.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a4a7fc640 1 -- 172.21.15.152:0/3891198708 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5405e070 con 0x564c21560b40 2024-09-11T22:57:49.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a4a7fc640 1 -- 172.21.15.152:0/3891198708 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f1a54004240 con 0x564c21560b40 2024-09-11T22:57:49.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/3891198708 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a5404fb20 con 0x564c21560b40 2024-09-11T22:57:49.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a4a7fc640 1 -- 172.21.15.152:0/3891198708 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1a54067050 con 0x564c21560b40 2024-09-11T22:57:49.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a6b8aa740 10 client.16788 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:49.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.882+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/3891198708 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564c215e4340 con 0x564c21560b40 2024-09-11T22:57:49.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.883+0000 7f1a4a7fc640 1 -- 172.21.15.152:0/3891198708 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a5404fb20 con 0x564c21560b40 2024-09-11T22:57:49.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.883+0000 7f1a4a7fc640 1 -- 172.21.15.152:0/3891198708 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1a54061020 con 0x564c21560b40 2024-09-11T22:57:49.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.883+0000 7f1a6b8aa740 10 client.16788 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:49.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.883+0000 7f1a6b8aa740 10 client.16788 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:57:49.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.883+0000 7f1a6b8aa740 20 client.16788 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:49.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.883+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/3891198708 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x564c215e4560 0x564c215e6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:49.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.883+0000 7f1a6b8aa740 4 client.16788 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:57:49.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.883+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/3891198708 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c214ccd70 con 0x564c215e4560 2024-09-11T22:57:49.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.883+0000 7f1a5a7fc640 1 --2- 172.21.15.152:0/3891198708 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x564c215e4560 0x564c215e6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:49.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a5a7fc640 1 --2- 172.21.15.152:0/3891198708 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x564c215e4560 0x564c215e6950 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d0fbb4fd36ac04 server_cookie=bb25362d237dbecf in_seq=0 out_seq=0 2024-09-11T22:57:49.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a4a7fc640 10 client.16788 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:57:49.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a4a7fc640 1 -- 172.21.15.152:0/3891198708 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c214ccd70 con 0x564c215e4560 2024-09-11T22:57:49.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a4a7fc640 10 client.16788 handle_command_reply: tid=0 2024-09-11T22:57:49.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a6b8aa740 1 client.16788 shutdown 2024-09-11T22:57:49.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/3891198708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c21560b40 msgr2=0x564c21572960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:49.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/3891198708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c21560b40 0x564c21572960 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1a54005c40 tx=0x7f1a54031770 comp rx=0 tx=0).stop 2024-09-11T22:57:49.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/3891198708 shutdown_connections 2024-09-11T22:57:49.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/3891198708 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x564c215e4560 0x564c215e6950 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/3891198708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564c2156a370 0x564c215635c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/3891198708 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564c21560b40 0x564c21572960 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a6b8aa740 1 --2- 172.21.15.152:0/3891198708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564c2155f5d0 0x564c21572420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/3891198708 >> 172.21.15.152:0/3891198708 conn(0x564c21528310 msgr2=0x564c2155b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:49.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a4a7fc640 0 client.16788 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:57:49.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/3891198708 shutdown_connections 2024-09-11T22:57:49.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a6b8aa740 1 -- 172.21.15.152:0/3891198708 wait complete. 2024-09-11T22:57:49.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.884+0000 7f1a6b8aa740 20 client.16788 trim_cache size 0 max 16384 2024-09-11T22:57:49.897 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:57:49.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.885+0000 7f1a694d1640 1 -- 172.21.15.152:0/2559925462 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1a500a2cf0 msgr2=0x7f1a500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:49.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.885+0000 7f1a694d1640 1 --2- 172.21.15.152:0/2559925462 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1a500a2cf0 0x7f1a500a51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0a4640 tx=0x7f1a5c06dec0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.885+0000 7f1a694d1640 1 -- 172.21.15.152:0/2559925462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a64106f10 msgr2=0x7f1a6419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:49.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.885+0000 7f1a694d1640 1 --2- 172.21.15.152:0/2559925462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a64106f10 0x7f1a6419cca0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00ca30 tx=0x7f1a5c00cef0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.885+0000 7f1a694d1640 1 -- 172.21.15.152:0/2559925462 shutdown_connections 2024-09-11T22:57:49.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.885+0000 7f1a694d1640 1 --2- 172.21.15.152:0/2559925462 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1a500a2cf0 0x7f1a500a51b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.885+0000 7f1a694d1640 1 --2- 172.21.15.152:0/2559925462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a64106f10 0x7f1a6419cca0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.885+0000 7f1a694d1640 1 --2- 172.21.15.152:0/2559925462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a64102790 0x7f1a64198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.885+0000 7f1a694d1640 1 --2- 172.21.15.152:0/2559925462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a64101de0 0x7f1a641983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:49.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.885+0000 7f1a694d1640 1 -- 172.21.15.152:0/2559925462 >> 172.21.15.152:0/2559925462 conn(0x7f1a640fd8c0 msgr2=0x7f1a640feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:49.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.885+0000 7f1a694d1640 1 -- 172.21.15.152:0/2559925462 shutdown_connections 2024-09-11T22:57:49.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:49.885+0000 7f1a694d1640 1 -- 172.21.15.152:0/2559925462 wait complete. 2024-09-11T22:57:50.724 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:57:50.972 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:57:51.255 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:50 smithi152 ceph-mon[31181]: pgmap v151: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:57:51.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:50 smithi191 ceph-mon[35211]: pgmap v151: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:57:51.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:50 smithi204 ceph-mon[35454]: pgmap v151: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:57:51.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.507+0000 7f108f2d1640 1 -- 172.21.15.152:0/2305616355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1088101420 msgr2=0x7f108810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:51.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.507+0000 7f108f2d1640 1 --2- 172.21.15.152:0/2305616355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1088101420 0x7f108810d950 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1080009cd0 tx=0x7f108002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.507+0000 7f108f2d1640 1 -- 172.21.15.152:0/2305616355 shutdown_connections 2024-09-11T22:57:51.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.507+0000 7f108f2d1640 1 --2- 172.21.15.152:0/2305616355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1088101420 0x7f108810d950 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.507+0000 7f108f2d1640 1 --2- 172.21.15.152:0/2305616355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1088100a80 0x7f1088100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.507+0000 7f108f2d1640 1 --2- 172.21.15.152:0/2305616355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1088106a80 0x7f1088106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.507+0000 7f108f2d1640 1 -- 172.21.15.152:0/2305616355 >> 172.21.15.152:0/2305616355 conn(0x7f10880fc7a0 msgr2=0x7f10880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:51.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.507+0000 7f108f2d1640 1 -- 172.21.15.152:0/2305616355 shutdown_connections 2024-09-11T22:57:51.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.507+0000 7f108f2d1640 1 -- 172.21.15.152:0/2305616355 wait complete. 2024-09-11T22:57:51.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.508+0000 7f108f2d1640 1 Processor -- start 2024-09-11T22:57:51.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.508+0000 7f108f2d1640 1 -- start start 2024-09-11T22:57:51.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.508+0000 7f108f2d1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1088100a80 0x7f1088198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.508+0000 7f108f2d1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1088101420 0x7f10881989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.508+0000 7f108f2d1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1088106a80 0x7f108819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.508+0000 7f108f2d1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1088110ea0 con 0x7f1088106a80 2024-09-11T22:57:51.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.508+0000 7f108f2d1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1088110d20 con 0x7f1088101420 2024-09-11T22:57:51.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.508+0000 7f108f2d1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1088111020 con 0x7f1088100a80 2024-09-11T22:57:51.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.508+0000 7f108d86e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1088106a80 0x7f108819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.508+0000 7f108d86e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1088106a80 0x7f108819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43210/0 (socket says 172.21.15.152:43210) 2024-09-11T22:57:51.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.508+0000 7f108d86e640 1 -- 172.21.15.152:0/428291132 learned_addr learned my addr 172.21.15.152:0/428291132 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:51.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.508+0000 7f108c86c640 1 --2- 172.21.15.152:0/428291132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1088101420 0x7f10881989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.508+0000 7f108d06d640 1 --2- 172.21.15.152:0/428291132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1088100a80 0x7f1088198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.509+0000 7f108d86e640 1 -- 172.21.15.152:0/428291132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1088100a80 msgr2=0x7f1088198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:51.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.509+0000 7f108d86e640 1 --2- 172.21.15.152:0/428291132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1088100a80 0x7f1088198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.509+0000 7f108d86e640 1 -- 172.21.15.152:0/428291132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1088101420 msgr2=0x7f10881989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:51.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.509+0000 7f108d86e640 1 --2- 172.21.15.152:0/428291132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1088101420 0x7f10881989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.509+0000 7f108d86e640 1 -- 172.21.15.152:0/428291132 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108819d430 con 0x7f1088106a80 2024-09-11T22:57:51.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.509+0000 7f108c86c640 1 --2- 172.21.15.152:0/428291132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1088101420 0x7f10881989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:51.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.509+0000 7f108d86e640 1 --2- 172.21.15.152:0/428291132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1088106a80 0x7f108819cd50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1080006010 tx=0x7f1080003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:51.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.509+0000 7f107e7fc640 1 -- 172.21.15.152:0/428291132 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1080004450 con 0x7f1088106a80 2024-09-11T22:57:51.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.509+0000 7f108f2d1640 1 -- 172.21.15.152:0/428291132 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108819d6c0 con 0x7f1088106a80 2024-09-11T22:57:51.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.509+0000 7f107e7fc640 1 -- 172.21.15.152:0/428291132 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1080003e10 con 0x7f1088106a80 2024-09-11T22:57:51.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.509+0000 7f107e7fc640 1 -- 172.21.15.152:0/428291132 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10800466e0 con 0x7f1088106a80 2024-09-11T22:57:51.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.509+0000 7f108f2d1640 1 -- 172.21.15.152:0/428291132 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10881a5000 con 0x7f1088106a80 2024-09-11T22:57:51.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.511+0000 7f107e7fc640 1 -- 172.21.15.152:0/428291132 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f10800305a0 con 0x7f1088106a80 2024-09-11T22:57:51.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.512+0000 7f10916aa740 1 Processor -- start 2024-09-11T22:57:51.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.512+0000 7f10916aa740 1 -- start start 2024-09-11T22:57:51.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.512+0000 7f10916aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b0929ab40 0x563b09299090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.512+0000 7f10916aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 0x563b09299a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.512+0000 7f10916aa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b092a4370 0x563b092a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.512+0000 7f10916aa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x563b091ff2f0 con 0x563b092a4370 2024-09-11T22:57:51.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.512+0000 7f10916aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563b091a6cf0 con 0x563b092995d0 2024-09-11T22:57:51.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.512+0000 7f10916aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563b091fb2b0 con 0x563b0929ab40 2024-09-11T22:57:51.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.512+0000 7f107e7fc640 1 --2- 172.21.15.152:0/428291132 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f10740a2cf0 0x7f10740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.512+0000 7f108c86c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b092a4370 0x563b092a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.512+0000 7f108c86c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b092a4370 0x563b092a6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43218/0 (socket says 172.21.15.152:43218) 2024-09-11T22:57:51.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.512+0000 7f108d86e640 1 --2- 172.21.15.152:0/428291132 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f10740a2cf0 0x7f10740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.512+0000 7f108c86c640 1 -- 172.21.15.152:0/3872091374 learned_addr learned my addr 172.21.15.152:0/3872091374 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:51.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.512+0000 7f108d06d640 1 --2- 172.21.15.152:0/3872091374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 0x563b09299a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.512+0000 7f108c86c640 1 --2- 172.21.15.152:0/3872091374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b0929ab40 0x563b09299090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.512+0000 7f107e7fc640 1 -- 172.21.15.152:0/428291132 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f108002e030 con 0x7f1088106a80 2024-09-11T22:57:51.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.513+0000 7f108c86c640 1 -- 172.21.15.152:0/3872091374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b0929ab40 msgr2=0x563b09299090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:51.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.513+0000 7f108c86c640 1 --2- 172.21.15.152:0/3872091374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b0929ab40 0x563b09299090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.513+0000 7f108c86c640 1 -- 172.21.15.152:0/3872091374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 msgr2=0x563b09299a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:51.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.513+0000 7f108c86c640 1 --2- 172.21.15.152:0/3872091374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 0x563b09299a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.513+0000 7f108c86c640 1 -- 172.21.15.152:0/3872091374 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b0918b3a0 con 0x563b092a4370 2024-09-11T22:57:51.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.513+0000 7f108d86e640 1 --2- 172.21.15.152:0/428291132 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f10740a2cf0 0x7f10740a51b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f10800bd0d0 tx=0x7f1080087f50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:51.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.513+0000 7f108c86c640 1 --2- 172.21.15.152:0/3872091374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b092a4370 0x563b092a6760 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1078016560 tx=0x7f107800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28e76ad83ece197 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:51.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.513+0000 7f10737fe640 1 -- 172.21.15.152:0/3872091374 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078018430 con 0x563b092a4370 2024-09-11T22:57:51.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.513+0000 7f10737fe640 1 -- 172.21.15.152:0/3872091374 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1078018ab0 con 0x563b092a4370 2024-09-11T22:57:51.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.513+0000 7f10737fe640 1 -- 172.21.15.152:0/3872091374 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1078017870 con 0x563b092a4370 2024-09-11T22:57:51.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.514+0000 7f10916aa740 1 -- 172.21.15.152:0/3872091374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b092a4370 msgr2=0x563b092a6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:51.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.514+0000 7f10916aa740 1 --2- 172.21.15.152:0/3872091374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b092a4370 0x563b092a6760 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1078016560 tx=0x7f107800a340 comp rx=0 tx=0).stop 2024-09-11T22:57:51.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.514+0000 7f10916aa740 1 -- 172.21.15.152:0/3872091374 shutdown_connections 2024-09-11T22:57:51.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.514+0000 7f10916aa740 1 --2- 172.21.15.152:0/3872091374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b092a4370 0x563b092a6760 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.514+0000 7f10916aa740 1 --2- 172.21.15.152:0/3872091374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 0x563b09299a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.514+0000 7f10916aa740 1 --2- 172.21.15.152:0/3872091374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b0929ab40 0x563b09299090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.514+0000 7f10916aa740 1 -- 172.21.15.152:0/3872091374 >> 172.21.15.152:0/3872091374 conn(0x563b09262310 msgr2=0x563b09295c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:51.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.514+0000 7f10916aa740 1 -- 172.21.15.152:0/3872091374 shutdown_connections 2024-09-11T22:57:51.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.514+0000 7f10916aa740 1 -- 172.21.15.152:0/3872091374 wait complete. 2024-09-11T22:57:51.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.514+0000 7f10916aa740 1 -- start start 2024-09-11T22:57:51.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.514+0000 7f10916aa740 1 Processor -- start 2024-09-11T22:57:51.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f10916aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 0x563b092a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f10916aa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092a1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f10916aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b092a4370 0x563b092a1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f10916aa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x563b092a7010 con 0x563b0929ab40 2024-09-11T22:57:51.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f10916aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563b091ba060 con 0x563b092995d0 2024-09-11T22:57:51.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f10916aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563b092a7190 con 0x563b092a4370 2024-09-11T22:57:51.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f10916aa740 10 client.0 _ll_register_callbacks cb 0x563b08f0d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:51.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f108d06d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f108d06d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092a1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43232/0 (socket says 172.21.15.152:43232) 2024-09-11T22:57:51.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f108d06d640 1 -- 172.21.15.152:0/2571520663 learned_addr learned my addr 172.21.15.152:0/2571520663 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:51.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f108c86c640 1 --2- 172.21.15.152:0/2571520663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b092a4370 0x563b092a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f108c86c640 1 --2- 172.21.15.152:0/2571520663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 0x563b092a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f108d06d640 1 -- 172.21.15.152:0/2571520663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b092a4370 msgr2=0x563b092a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:51.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f108d06d640 1 --2- 172.21.15.152:0/2571520663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b092a4370 0x563b092a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f108d06d640 1 -- 172.21.15.152:0/2571520663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 msgr2=0x563b092a3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:51.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f108d06d640 1 --2- 172.21.15.152:0/2571520663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 0x563b092a3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f108d06d640 1 -- 172.21.15.152:0/2571520663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b08f703c0 con 0x563b0929ab40 2024-09-11T22:57:51.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f108c86c640 1 --2- 172.21.15.152:0/2571520663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 0x563b092a3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:57:51.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f108d06d640 1 --2- 172.21.15.152:0/2571520663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092a1020 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f108400aa70 tx=0x7f108400af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539a769cb559fcbd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:51.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f1070ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:51.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f1070ff9640 1 -- 172.21.15.152:0/2571520663 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084009d90 con 0x563b0929ab40 2024-09-11T22:57:51.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f10916aa740 1 -- 172.21.15.152:0/2571520663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1078018ab0 con 0x563b0929ab40 2024-09-11T22:57:51.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.515+0000 7f1070ff9640 1 -- 172.21.15.152:0/2571520663 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f1084005020 con 0x563b0929ab40 2024-09-11T22:57:51.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.516+0000 7f1070ff9640 1 -- 172.21.15.152:0/2571520663 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1084005220 con 0x563b0929ab40 2024-09-11T22:57:51.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.516+0000 7f10916aa740 10 client.16800 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:51.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.516+0000 7f10916aa740 1 -- 172.21.15.152:0/2571520663 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563b092b2b60 con 0x563b0929ab40 2024-09-11T22:57:51.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.516+0000 7f1070ff9640 1 -- 172.21.15.152:0/2571520663 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1078018ab0 con 0x563b0929ab40 2024-09-11T22:57:51.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.516+0000 7f1070ff9640 1 -- 172.21.15.152:0/2571520663 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f108401c5e0 con 0x563b0929ab40 2024-09-11T22:57:51.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.516+0000 7f10916aa740 10 client.16800 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:51.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.516+0000 7f10916aa740 10 client.16800 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:57:51.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.516+0000 7f10916aa740 20 client.16800 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:51.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.516+0000 7f10916aa740 1 --2- 172.21.15.152:0/2571520663 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x563b092b40c0 0x563b092b6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.516+0000 7f10916aa740 4 client.16800 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:57:51.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.516+0000 7f10916aa740 1 -- 172.21.15.152:0/2571520663 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b0920c860 con 0x563b092b40c0 2024-09-11T22:57:51.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.516+0000 7f108c86c640 1 --2- 172.21.15.152:0/2571520663 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x563b092b40c0 0x563b092b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.517+0000 7f108c86c640 1 --2- 172.21.15.152:0/2571520663 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x563b092b40c0 0x563b092b6500 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c3bec9f4b681d773 server_cookie=2d32aab77e01f134 in_seq=0 out_seq=0 2024-09-11T22:57:51.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.517+0000 7f1070ff9640 10 client.16800 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:57:51.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.517+0000 7f1070ff9640 1 -- 172.21.15.152:0/2571520663 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b0920c860 con 0x563b092b40c0 2024-09-11T22:57:51.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.517+0000 7f1070ff9640 10 client.16800 handle_command_reply: tid=0 2024-09-11T22:57:51.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.517+0000 7f10916aa740 1 client.16800 shutdown 2024-09-11T22:57:51.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.517+0000 7f10916aa740 1 -- 172.21.15.152:0/2571520663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 msgr2=0x563b092a1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:51.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.517+0000 7f10916aa740 1 --2- 172.21.15.152:0/2571520663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092a1020 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f108400aa70 tx=0x7f108400af30 comp rx=0 tx=0).stop 2024-09-11T22:57:51.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.518+0000 7f10916aa740 1 -- 172.21.15.152:0/2571520663 shutdown_connections 2024-09-11T22:57:51.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.518+0000 7f10916aa740 1 --2- 172.21.15.152:0/2571520663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b092a4370 0x563b092a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.518+0000 7f10916aa740 1 --2- 172.21.15.152:0/2571520663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092a1020 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.518+0000 7f10916aa740 1 --2- 172.21.15.152:0/2571520663 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x563b092b40c0 0x563b092b6500 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.518+0000 7f10916aa740 1 --2- 172.21.15.152:0/2571520663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 0x563b092a3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.518+0000 7f1070ff9640 0 client.16800 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:57:51.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.518+0000 7f10916aa740 1 -- 172.21.15.152:0/2571520663 >> 172.21.15.152:0/2571520663 conn(0x563b09262310 msgr2=0x563b0929b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:51.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.518+0000 7f10916aa740 1 -- 172.21.15.152:0/2571520663 shutdown_connections 2024-09-11T22:57:51.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.518+0000 7f10916aa740 1 -- 172.21.15.152:0/2571520663 wait complete. 2024-09-11T22:57:51.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.518+0000 7f10916aa740 20 client.16800 trim_cache size 0 max 16384 2024-09-11T22:57:51.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.531+0000 7f10916aa740 1 Processor -- start 2024-09-11T22:57:51.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.531+0000 7f10916aa740 1 -- start start 2024-09-11T22:57:51.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f10916aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 0x563b092b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f10916aa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f10916aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b092a4370 0x563b09295700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f10916aa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f10881ab500 con 0x563b0929ab40 2024-09-11T22:57:51.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f10916aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f10881ad040 con 0x563b092995d0 2024-09-11T22:57:51.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f10916aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f10881abbd0 con 0x563b092a4370 2024-09-11T22:57:51.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f108d06d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f108c86c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 0x563b092b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f108d06d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43234/0 (socket says 172.21.15.152:43234) 2024-09-11T22:57:51.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f108d06d640 1 -- 172.21.15.152:0/1252651345 learned_addr learned my addr 172.21.15.152:0/1252651345 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:51.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f108c86c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 0x563b092b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:46520/0 (socket says 172.21.15.152:46520) 2024-09-11T22:57:51.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f108c86c640 1 --2- 172.21.15.152:0/1252651345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b092a4370 0x563b09295700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f108d06d640 1 -- 172.21.15.152:0/1252651345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b092a4370 msgr2=0x563b09295700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:51.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f108d06d640 1 --2- 172.21.15.152:0/1252651345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b092a4370 0x563b09295700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f108d06d640 1 -- 172.21.15.152:0/1252651345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 msgr2=0x563b092b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:51.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f108d06d640 1 --2- 172.21.15.152:0/1252651345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 0x563b092b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f108d06d640 1 -- 172.21.15.152:0/1252651345 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b09296960 con 0x563b0929ab40 2024-09-11T22:57:51.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f108c86c640 1 --2- 172.21.15.152:0/1252651345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 0x563b092b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:57:51.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.532+0000 7f108d06d640 1 --2- 172.21.15.152:0/1252651345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092a08e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f108401ca00 tx=0x7f10840097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2069a95370310f89 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:51.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.533+0000 7f1072ffd640 1 -- 172.21.15.152:0/1252651345 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10840053c0 con 0x563b0929ab40 2024-09-11T22:57:51.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.533+0000 7f1072ffd640 1 -- 172.21.15.152:0/1252651345 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1084005560 con 0x563b0929ab40 2024-09-11T22:57:51.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.533+0000 7f1072ffd640 1 -- 172.21.15.152:0/1252651345 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084004660 con 0x563b0929ab40 2024-09-11T22:57:51.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.533+0000 7f10916aa740 1 -- 172.21.15.152:0/1252651345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 msgr2=0x563b092a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:51.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.533+0000 7f10916aa740 1 --2- 172.21.15.152:0/1252651345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092a08e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f108401ca00 tx=0x7f10840097f0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.533+0000 7f10916aa740 1 -- 172.21.15.152:0/1252651345 shutdown_connections 2024-09-11T22:57:51.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.533+0000 7f10916aa740 1 --2- 172.21.15.152:0/1252651345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b092a4370 0x563b09295700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.533+0000 7f10916aa740 1 --2- 172.21.15.152:0/1252651345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092a08e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.533+0000 7f10916aa740 1 --2- 172.21.15.152:0/1252651345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092995d0 0x563b092b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.533+0000 7f10916aa740 1 -- 172.21.15.152:0/1252651345 >> 172.21.15.152:0/1252651345 conn(0x563b09262310 msgr2=0x563b0929c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:51.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.533+0000 7f10916aa740 1 -- 172.21.15.152:0/1252651345 shutdown_connections 2024-09-11T22:57:51.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.534+0000 7f10916aa740 1 -- 172.21.15.152:0/1252651345 wait complete. 2024-09-11T22:57:51.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.534+0000 7f10916aa740 1 -- start start 2024-09-11T22:57:51.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.534+0000 7f10916aa740 1 Processor -- start 2024-09-11T22:57:51.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.534+0000 7f10916aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b092995d0 0x563b092ac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.534+0000 7f10916aa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092ac960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.534+0000 7f10916aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092a4370 0x563b0929d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.534+0000 7f10916aa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f10881ac970 con 0x563b0929ab40 2024-09-11T22:57:51.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.534+0000 7f10916aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f10881ac2a0 con 0x563b092a4370 2024-09-11T22:57:51.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.534+0000 7f10916aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f10881a94c0 con 0x563b092995d0 2024-09-11T22:57:51.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.534+0000 7f10916aa740 10 client.0 _ll_register_callbacks cb 0x7f1084010ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:51.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.534+0000 7f108d06d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.534+0000 7f108d06d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092ac960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43250/0 (socket says 172.21.15.152:43250) 2024-09-11T22:57:51.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.534+0000 7f108d06d640 1 -- 172.21.15.152:0/3612058099 learned_addr learned my addr 172.21.15.152:0/3612058099 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:51.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.535+0000 7f108c86c640 1 --2- 172.21.15.152:0/3612058099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b092995d0 0x563b092ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.535+0000 7f108c86c640 1 --2- 172.21.15.152:0/3612058099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092a4370 0x563b0929d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.535+0000 7f108d06d640 1 -- 172.21.15.152:0/3612058099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b092995d0 msgr2=0x563b092ac420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:51.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.535+0000 7f108d06d640 1 --2- 172.21.15.152:0/3612058099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b092995d0 0x563b092ac420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.535+0000 7f108d06d640 1 -- 172.21.15.152:0/3612058099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092a4370 msgr2=0x563b0929d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:51.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.535+0000 7f108d06d640 1 --2- 172.21.15.152:0/3612058099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092a4370 0x563b0929d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.535+0000 7f108d06d640 1 -- 172.21.15.152:0/3612058099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b0929dbb0 con 0x563b0929ab40 2024-09-11T22:57:51.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.535+0000 7f108c86c640 1 --2- 172.21.15.152:0/3612058099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092a4370 0x563b0929d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:51.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.535+0000 7f108d06d640 1 --2- 172.21.15.152:0/3612058099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092ac960 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x563b092b07f0 tx=0x7f108400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bceab038b40fa28 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:51.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.535+0000 7f1070ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:51.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.535+0000 7f1070ff9640 1 -- 172.21.15.152:0/3612058099 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10840107c0 con 0x563b0929ab40 2024-09-11T22:57:51.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.535+0000 7f1070ff9640 1 -- 172.21.15.152:0/3612058099 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f108401c440 con 0x563b0929ab40 2024-09-11T22:57:51.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.535+0000 7f10916aa740 1 -- 172.21.15.152:0/3612058099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1084005560 con 0x563b0929ab40 2024-09-11T22:57:51.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.535+0000 7f1070ff9640 1 -- 172.21.15.152:0/3612058099 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f108403e050 con 0x563b0929ab40 2024-09-11T22:57:51.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.535+0000 7f10916aa740 10 client.16806 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:51.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.535+0000 7f10916aa740 1 -- 172.21.15.152:0/3612058099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563b0931e580 con 0x563b0929ab40 2024-09-11T22:57:51.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.536+0000 7f1070ff9640 1 -- 172.21.15.152:0/3612058099 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1084005560 con 0x563b0929ab40 2024-09-11T22:57:51.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.536+0000 7f1070ff9640 1 -- 172.21.15.152:0/3612058099 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1084037060 con 0x563b0929ab40 2024-09-11T22:57:51.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.536+0000 7f10916aa740 10 client.16806 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:51.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.536+0000 7f10916aa740 10 client.16806 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:57:51.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.536+0000 7f10916aa740 20 client.16806 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:51.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.536+0000 7f10916aa740 1 --2- 172.21.15.152:0/3612058099 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x563b0931ea10 0x563b09320e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:51.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.536+0000 7f10916aa740 4 client.16806 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:57:51.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.536+0000 7f10916aa740 1 -- 172.21.15.152:0/3612058099 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b09206d70 con 0x563b0931ea10 2024-09-11T22:57:51.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.536+0000 7f108c86c640 1 --2- 172.21.15.152:0/3612058099 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x563b0931ea10 0x563b09320e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:51.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.536+0000 7f108c86c640 1 --2- 172.21.15.152:0/3612058099 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x563b0931ea10 0x563b09320e30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c335e70d82d40b21 server_cookie=36b107e66637f787 in_seq=0 out_seq=0 2024-09-11T22:57:51.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.536+0000 7f1070ff9640 10 client.16806 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:57:51.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.537+0000 7f1070ff9640 1 -- 172.21.15.152:0/3612058099 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b09206d70 con 0x563b0931ea10 2024-09-11T22:57:51.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.537+0000 7f1070ff9640 10 client.16806 handle_command_reply: tid=0 2024-09-11T22:57:51.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.537+0000 7f10916aa740 1 client.16806 shutdown 2024-09-11T22:57:51.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.537+0000 7f10916aa740 1 -- 172.21.15.152:0/3612058099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 msgr2=0x563b092ac960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:51.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.537+0000 7f10916aa740 1 --2- 172.21.15.152:0/3612058099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092ac960 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x563b092b07f0 tx=0x7f108400a710 comp rx=0 tx=0).stop 2024-09-11T22:57:51.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.537+0000 7f10916aa740 1 -- 172.21.15.152:0/3612058099 shutdown_connections 2024-09-11T22:57:51.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.537+0000 7f10916aa740 1 --2- 172.21.15.152:0/3612058099 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x563b0931ea10 0x563b09320e30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.537+0000 7f10916aa740 1 --2- 172.21.15.152:0/3612058099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b092a4370 0x563b0929d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.537+0000 7f10916aa740 1 --2- 172.21.15.152:0/3612058099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563b0929ab40 0x563b092ac960 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.537+0000 7f10916aa740 1 --2- 172.21.15.152:0/3612058099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563b092995d0 0x563b092ac420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.537+0000 7f1070ff9640 0 client.16806 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:57:51.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.537+0000 7f10916aa740 1 -- 172.21.15.152:0/3612058099 >> 172.21.15.152:0/3612058099 conn(0x563b09262310 msgr2=0x563b09295700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:51.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.537+0000 7f10916aa740 1 -- 172.21.15.152:0/3612058099 shutdown_connections 2024-09-11T22:57:51.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.537+0000 7f10916aa740 1 -- 172.21.15.152:0/3612058099 wait complete. 2024-09-11T22:57:51.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.537+0000 7f10916aa740 20 client.16806 trim_cache size 0 max 16384 2024-09-11T22:57:51.552 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:57:51.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.538+0000 7f108f2d1640 1 -- 172.21.15.152:0/428291132 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f10740a2cf0 msgr2=0x7f10740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:51.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.538+0000 7f108f2d1640 1 --2- 172.21.15.152:0/428291132 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f10740a2cf0 0x7f10740a51b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f10800bd0d0 tx=0x7f1080087f50 comp rx=0 tx=0).stop 2024-09-11T22:57:51.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.538+0000 7f108f2d1640 1 -- 172.21.15.152:0/428291132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1088106a80 msgr2=0x7f108819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:51.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.538+0000 7f108f2d1640 1 --2- 172.21.15.152:0/428291132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1088106a80 0x7f108819cd50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1080006010 tx=0x7f1080003c20 comp rx=0 tx=0).stop 2024-09-11T22:57:51.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.538+0000 7f108f2d1640 1 -- 172.21.15.152:0/428291132 shutdown_connections 2024-09-11T22:57:51.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.538+0000 7f108f2d1640 1 --2- 172.21.15.152:0/428291132 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f10740a2cf0 0x7f10740a51b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.538+0000 7f108f2d1640 1 --2- 172.21.15.152:0/428291132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1088106a80 0x7f108819cd50 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.538+0000 7f108f2d1640 1 --2- 172.21.15.152:0/428291132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1088101420 0x7f10881989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.538+0000 7f108f2d1640 1 --2- 172.21.15.152:0/428291132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1088100a80 0x7f1088198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:51.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.538+0000 7f108f2d1640 1 -- 172.21.15.152:0/428291132 >> 172.21.15.152:0/428291132 conn(0x7f10880fc7a0 msgr2=0x7f10881047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:51.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.538+0000 7f108f2d1640 1 -- 172.21.15.152:0/428291132 shutdown_connections 2024-09-11T22:57:51.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:51.538+0000 7f108f2d1640 1 -- 172.21.15.152:0/428291132 wait complete. 2024-09-11T22:57:52.286 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:57:52.744 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:57:53.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:52 smithi152 ceph-mon[31181]: pgmap v152: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:57:53.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:53 smithi191 ceph-mon[35211]: pgmap v152: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:57:53.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:52 smithi204 ceph-mon[35454]: pgmap v152: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:57:53.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.284+0000 7f141c3d6640 1 -- 172.21.15.152:0/4164573831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1414106f10 msgr2=0x7f141410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:53.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.284+0000 7f141c3d6640 1 --2- 172.21.15.152:0/4164573831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1414106f10 0x7f141410ea90 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f140c009ca0 tx=0x7f140c02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.284+0000 7f141c3d6640 1 -- 172.21.15.152:0/4164573831 shutdown_connections 2024-09-11T22:57:53.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.284+0000 7f141c3d6640 1 --2- 172.21.15.152:0/4164573831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1414106f10 0x7f141410ea90 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.284+0000 7f141c3d6640 1 --2- 172.21.15.152:0/4164573831 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1414102790 0x7f14141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.284+0000 7f141c3d6640 1 --2- 172.21.15.152:0/4164573831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1414101de0 0x7f14141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.284+0000 7f141c3d6640 1 -- 172.21.15.152:0/4164573831 >> 172.21.15.152:0/4164573831 conn(0x7f14140fd8c0 msgr2=0x7f14140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:53.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.284+0000 7f141c3d6640 1 -- 172.21.15.152:0/4164573831 shutdown_connections 2024-09-11T22:57:53.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.284+0000 7f141c3d6640 1 -- 172.21.15.152:0/4164573831 wait complete. 2024-09-11T22:57:53.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.285+0000 7f141c3d6640 1 Processor -- start 2024-09-11T22:57:53.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.285+0000 7f141c3d6640 1 -- start start 2024-09-11T22:57:53.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.285+0000 7f141c3d6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1414101de0 0x7f14141983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.285+0000 7f141c3d6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1414102790 0x7f1414198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.285+0000 7f141c3d6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1414106f10 0x7f141419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.285+0000 7f141c3d6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1414111f70 con 0x7f1414101de0 2024-09-11T22:57:53.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.285+0000 7f141c3d6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1414111df0 con 0x7f1414106f10 2024-09-11T22:57:53.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.285+0000 7f141c3d6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14141120f0 con 0x7f1414102790 2024-09-11T22:57:53.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.285+0000 7f141a172640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1414101de0 0x7f14141983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.285+0000 7f141a172640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1414101de0 0x7f14141983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43262/0 (socket says 172.21.15.152:43262) 2024-09-11T22:57:53.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.285+0000 7f141a172640 1 -- 172.21.15.152:0/618633576 learned_addr learned my addr 172.21.15.152:0/618633576 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:53.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.285+0000 7f141a973640 1 --2- 172.21.15.152:0/618633576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1414106f10 0x7f141419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.286+0000 7f141a172640 1 -- 172.21.15.152:0/618633576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1414102790 msgr2=0x7f1414198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:53.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.286+0000 7f1419971640 1 --2- 172.21.15.152:0/618633576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1414102790 0x7f1414198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.286+0000 7f141a172640 1 --2- 172.21.15.152:0/618633576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1414102790 0x7f1414198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.286+0000 7f141a172640 1 -- 172.21.15.152:0/618633576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1414106f10 msgr2=0x7f141419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:53.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.286+0000 7f141a172640 1 --2- 172.21.15.152:0/618633576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1414106f10 0x7f141419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.286+0000 7f141a172640 1 -- 172.21.15.152:0/618633576 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141419d3a0 con 0x7f1414101de0 2024-09-11T22:57:53.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.286+0000 7f141a973640 1 --2- 172.21.15.152:0/618633576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1414106f10 0x7f141419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:53.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.286+0000 7f141a172640 1 --2- 172.21.15.152:0/618633576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1414101de0 0x7f14141983f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f141000d950 tx=0x7f141000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:53.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.286+0000 7f140b7fe640 1 -- 172.21.15.152:0/618633576 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1410017070 con 0x7f1414101de0 2024-09-11T22:57:53.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.286+0000 7f141c3d6640 1 -- 172.21.15.152:0/618633576 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141419d690 con 0x7f1414101de0 2024-09-11T22:57:53.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.286+0000 7f140b7fe640 1 -- 172.21.15.152:0/618633576 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1410009c30 con 0x7f1414101de0 2024-09-11T22:57:53.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.286+0000 7f140b7fe640 1 -- 172.21.15.152:0/618633576 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141000f6b0 con 0x7f1414101de0 2024-09-11T22:57:53.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.286+0000 7f141c3d6640 1 -- 172.21.15.152:0/618633576 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14141a4f70 con 0x7f1414101de0 2024-09-11T22:57:53.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.288+0000 7f140b7fe640 1 -- 172.21.15.152:0/618633576 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f14100109d0 con 0x7f1414101de0 2024-09-11T22:57:53.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.289+0000 7f141e7af740 1 Processor -- start 2024-09-11T22:57:53.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.289+0000 7f141e7af740 1 -- start start 2024-09-11T22:57:53.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.289+0000 7f141e7af740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b23283090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.289+0000 7f141e7af740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b232835d0 0x555b23283a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.289+0000 7f141e7af740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b23292780 0x555b23294b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.289+0000 7f141e7af740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x555b231e92f0 con 0x555b23292780 2024-09-11T22:57:53.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.289+0000 7f141e7af740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555b23190cf0 con 0x555b23284b40 2024-09-11T22:57:53.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.289+0000 7f141e7af740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555b231e52b0 con 0x555b232835d0 2024-09-11T22:57:53.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.289+0000 7f1419971640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b23292780 0x555b23294b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.289+0000 7f1419971640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b23292780 0x555b23294b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43276/0 (socket says 172.21.15.152:43276) 2024-09-11T22:57:53.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.289+0000 7f1419971640 1 -- 172.21.15.152:0/140374945 learned_addr learned my addr 172.21.15.152:0/140374945 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:53.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.289+0000 7f140b7fe640 1 --2- 172.21.15.152:0/618633576 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f14000a2cf0 0x7f14000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.290+0000 7f141a172640 1 --2- 172.21.15.152:0/140374945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b232835d0 0x555b23283a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.290+0000 7f141a973640 1 --2- 172.21.15.152:0/140374945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b23283090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.290+0000 7f1419971640 1 -- 172.21.15.152:0/140374945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b232835d0 msgr2=0x555b23283a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:53.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.290+0000 7f1419971640 1 --2- 172.21.15.152:0/140374945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b232835d0 0x555b23283a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.290+0000 7f1419971640 1 -- 172.21.15.152:0/140374945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 msgr2=0x555b23283090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:53.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.290+0000 7f1419971640 1 --2- 172.21.15.152:0/140374945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b23283090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.290+0000 7f140b7fe640 1 -- 172.21.15.152:0/618633576 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f14100cf140 con 0x7f1414101de0 2024-09-11T22:57:53.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.290+0000 7f1419971640 1 -- 172.21.15.152:0/140374945 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b231753a0 con 0x555b23292780 2024-09-11T22:57:53.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.290+0000 7f141a973640 1 --2- 172.21.15.152:0/140374945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b23283090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:57:53.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.290+0000 7f141a973640 1 --2- 172.21.15.152:0/618633576 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f14000a2cf0 0x7f14000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.290+0000 7f1419971640 1 --2- 172.21.15.152:0/140374945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b23292780 0x555b23294b70 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1404009ae0 tx=0x7f140400e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d81dbdb3b0d6df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:53.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.290+0000 7f1408dea640 1 -- 172.21.15.152:0/140374945 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404016070 con 0x555b23292780 2024-09-11T22:57:53.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.290+0000 7f1408dea640 1 -- 172.21.15.152:0/140374945 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f140400ecc0 con 0x555b23292780 2024-09-11T22:57:53.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.290+0000 7f1408dea640 1 -- 172.21.15.152:0/140374945 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404010750 con 0x555b23292780 2024-09-11T22:57:53.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.291+0000 7f141e7af740 1 -- 172.21.15.152:0/140374945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b23292780 msgr2=0x555b23294b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:53.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.291+0000 7f141a973640 1 --2- 172.21.15.152:0/618633576 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f14000a2cf0 0x7f14000a51b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f140c005cc0 tx=0x7f140c005da0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:53.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.291+0000 7f141e7af740 1 --2- 172.21.15.152:0/140374945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b23292780 0x555b23294b70 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1404009ae0 tx=0x7f140400e2f0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.291+0000 7f141e7af740 1 -- 172.21.15.152:0/140374945 shutdown_connections 2024-09-11T22:57:53.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.291+0000 7f141e7af740 1 --2- 172.21.15.152:0/140374945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b23292780 0x555b23294b70 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.291+0000 7f141e7af740 1 --2- 172.21.15.152:0/140374945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b232835d0 0x555b23283a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.291+0000 7f141e7af740 1 --2- 172.21.15.152:0/140374945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b23283090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.291+0000 7f141e7af740 1 -- 172.21.15.152:0/140374945 >> 172.21.15.152:0/140374945 conn(0x555b2324c310 msgr2=0x555b2327fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:53.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.291+0000 7f141e7af740 1 -- 172.21.15.152:0/140374945 shutdown_connections 2024-09-11T22:57:53.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.291+0000 7f141e7af740 1 -- 172.21.15.152:0/140374945 wait complete. 2024-09-11T22:57:53.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.291+0000 7f141e7af740 1 -- start start 2024-09-11T22:57:53.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.291+0000 7f141e7af740 1 Processor -- start 2024-09-11T22:57:53.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141e7af740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b232835d0 0x555b23292060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141e7af740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b2328f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141e7af740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b23292780 0x555b2328f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141e7af740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x555b23295480 con 0x555b232835d0 2024-09-11T22:57:53.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141e7af740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555b231a4060 con 0x555b23284b40 2024-09-11T22:57:53.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141e7af740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555b23295600 con 0x555b23292780 2024-09-11T22:57:53.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141e7af740 10 client.0 _ll_register_callbacks cb 0x555b22ef78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:53.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141a172640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b232835d0 0x555b23292060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141a172640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b232835d0 0x555b23292060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43290/0 (socket says 172.21.15.152:43290) 2024-09-11T22:57:53.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141a172640 1 -- 172.21.15.152:0/4222727149 learned_addr learned my addr 172.21.15.152:0/4222727149 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:53.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141a973640 1 --2- 172.21.15.152:0/4222727149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b23292780 0x555b2328f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f1419971640 1 --2- 172.21.15.152:0/4222727149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b2328f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141a172640 1 -- 172.21.15.152:0/4222727149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b23292780 msgr2=0x555b2328f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:53.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141a172640 1 --2- 172.21.15.152:0/4222727149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b23292780 0x555b2328f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141a172640 1 -- 172.21.15.152:0/4222727149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 msgr2=0x555b2328f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:53.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141a172640 1 --2- 172.21.15.152:0/4222727149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b2328f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141a172640 1 -- 172.21.15.152:0/4222727149 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b22f5a3c0 con 0x555b232835d0 2024-09-11T22:57:53.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141a973640 1 --2- 172.21.15.152:0/4222727149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b23292780 0x555b2328f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:53.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141a172640 1 --2- 172.21.15.152:0/4222727149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b232835d0 0x555b23292060 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1410009610 tx=0x7f141006a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a8ecc6d3feaece server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:53.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f13fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:53.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f13fdffb640 1 -- 172.21.15.152:0/4222727149 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1410020050 con 0x555b232835d0 2024-09-11T22:57:53.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f13fdffb640 1 -- 172.21.15.152:0/4222727149 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f1410097ce0 con 0x555b232835d0 2024-09-11T22:57:53.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.292+0000 7f141e7af740 1 -- 172.21.15.152:0/4222727149 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f140400ecc0 con 0x555b232835d0 2024-09-11T22:57:53.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.293+0000 7f13fdffb640 1 -- 172.21.15.152:0/4222727149 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f141009bd70 con 0x555b232835d0 2024-09-11T22:57:53.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.293+0000 7f141e7af740 10 client.16818 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:53.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.293+0000 7f141e7af740 1 -- 172.21.15.152:0/4222727149 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b232a0df0 con 0x555b232835d0 2024-09-11T22:57:53.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.293+0000 7f13fdffb640 1 -- 172.21.15.152:0/4222727149 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f140400ecc0 con 0x555b232835d0 2024-09-11T22:57:53.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.293+0000 7f13fdffb640 1 -- 172.21.15.152:0/4222727149 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f14100ccb20 con 0x555b232835d0 2024-09-11T22:57:53.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.293+0000 7f141e7af740 10 client.16818 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:53.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.293+0000 7f141e7af740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:57:53.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.293+0000 7f141e7af740 20 client.16818 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:53.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.293+0000 7f141e7af740 1 --2- 172.21.15.152:0/4222727149 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555b232a22b0 0x555b232a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.293+0000 7f141e7af740 4 client.16818 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:57:53.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.293+0000 7f141e7af740 1 -- 172.21.15.152:0/4222727149 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b231f6860 con 0x555b232a22b0 2024-09-11T22:57:53.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.293+0000 7f1419971640 1 --2- 172.21.15.152:0/4222727149 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555b232a22b0 0x555b232a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.294+0000 7f13fdffb640 10 client.16818 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:53.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.294+0000 7f1419971640 1 --2- 172.21.15.152:0/4222727149 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555b232a22b0 0x555b232a46a0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7be67884404c6b23 server_cookie=3e8c378d54f63adc in_seq=0 out_seq=0 2024-09-11T22:57:53.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.294+0000 7f13fdffb640 1 -- 172.21.15.152:0/4222727149 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b231f6860 con 0x555b232a22b0 2024-09-11T22:57:53.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.294+0000 7f13fdffb640 10 client.16818 handle_command_reply: tid=0 2024-09-11T22:57:53.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.294+0000 7f141e7af740 1 client.16818 shutdown 2024-09-11T22:57:53.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.294+0000 7f141e7af740 1 -- 172.21.15.152:0/4222727149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b232835d0 msgr2=0x555b23292060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:53.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.294+0000 7f141e7af740 1 --2- 172.21.15.152:0/4222727149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b232835d0 0x555b23292060 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1410009610 tx=0x7f141006a5a0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.295+0000 7f141e7af740 1 -- 172.21.15.152:0/4222727149 shutdown_connections 2024-09-11T22:57:53.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.295+0000 7f141e7af740 1 --2- 172.21.15.152:0/4222727149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b23292780 0x555b2328f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.295+0000 7f141e7af740 1 --2- 172.21.15.152:0/4222727149 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555b232a22b0 0x555b232a46a0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.295+0000 7f141e7af740 1 --2- 172.21.15.152:0/4222727149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b2328f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.295+0000 7f141e7af740 1 --2- 172.21.15.152:0/4222727149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b232835d0 0x555b23292060 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.295+0000 7f141e7af740 1 -- 172.21.15.152:0/4222727149 >> 172.21.15.152:0/4222727149 conn(0x555b2324c310 msgr2=0x555b23285960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:53.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.295+0000 7f13fdffb640 0 client.16818 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:53.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.295+0000 7f141e7af740 1 -- 172.21.15.152:0/4222727149 shutdown_connections 2024-09-11T22:57:53.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.295+0000 7f141e7af740 1 -- 172.21.15.152:0/4222727149 wait complete. 2024-09-11T22:57:53.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.295+0000 7f141e7af740 20 client.16818 trim_cache size 0 max 16384 2024-09-11T22:57:53.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141e7af740 1 Processor -- start 2024-09-11T22:57:53.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141e7af740 1 -- start start 2024-09-11T22:57:53.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141e7af740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b232835d0 0x555b2329e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141e7af740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b2328a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141e7af740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b23292780 0x555b232a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141e7af740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f14141ab550 con 0x555b232835d0 2024-09-11T22:57:53.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141e7af740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f14141ad090 con 0x555b23284b40 2024-09-11T22:57:53.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141e7af740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14141abc20 con 0x555b23292780 2024-09-11T22:57:53.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141a172640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b232835d0 0x555b2329e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141a172640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b232835d0 0x555b2329e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43302/0 (socket says 172.21.15.152:43302) 2024-09-11T22:57:53.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141a172640 1 -- 172.21.15.152:0/3591928027 learned_addr learned my addr 172.21.15.152:0/3591928027 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:53.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f1419971640 1 --2- 172.21.15.152:0/3591928027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b2328a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141a973640 1 --2- 172.21.15.152:0/3591928027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b23292780 0x555b232a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141a172640 1 -- 172.21.15.152:0/3591928027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b23292780 msgr2=0x555b232a9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:53.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141a172640 1 --2- 172.21.15.152:0/3591928027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b23292780 0x555b232a9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141a172640 1 -- 172.21.15.152:0/3591928027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 msgr2=0x555b2328a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:53.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141a172640 1 --2- 172.21.15.152:0/3591928027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b2328a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141a172640 1 -- 172.21.15.152:0/3591928027 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b2327e310 con 0x555b232835d0 2024-09-11T22:57:53.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f1419971640 1 --2- 172.21.15.152:0/3591928027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b2328a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:53.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.308+0000 7f141a172640 1 --2- 172.21.15.152:0/3591928027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b232835d0 0x555b2329e720 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1410097ad0 tx=0x7f141000b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c86d6efdef7a259 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:53.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.309+0000 7f13fffff640 1 -- 172.21.15.152:0/3591928027 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1410020070 con 0x555b232835d0 2024-09-11T22:57:53.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.309+0000 7f13fffff640 1 -- 172.21.15.152:0/3591928027 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f141009b670 con 0x555b232835d0 2024-09-11T22:57:53.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.309+0000 7f13fffff640 1 -- 172.21.15.152:0/3591928027 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141006a260 con 0x555b232835d0 2024-09-11T22:57:53.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.309+0000 7f141e7af740 1 -- 172.21.15.152:0/3591928027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b232835d0 msgr2=0x555b2329e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:53.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.309+0000 7f141e7af740 1 --2- 172.21.15.152:0/3591928027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b232835d0 0x555b2329e720 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1410097ad0 tx=0x7f141000b530 comp rx=0 tx=0).stop 2024-09-11T22:57:53.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.309+0000 7f141e7af740 1 -- 172.21.15.152:0/3591928027 shutdown_connections 2024-09-11T22:57:53.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.309+0000 7f141e7af740 1 --2- 172.21.15.152:0/3591928027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b23292780 0x555b232a9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.309+0000 7f141e7af740 1 --2- 172.21.15.152:0/3591928027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b2328a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.309+0000 7f141e7af740 1 --2- 172.21.15.152:0/3591928027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b232835d0 0x555b2329e720 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.309+0000 7f141e7af740 1 -- 172.21.15.152:0/3591928027 >> 172.21.15.152:0/3591928027 conn(0x555b2324c310 msgr2=0x555b2328dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:53.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.309+0000 7f141e7af740 1 -- 172.21.15.152:0/3591928027 shutdown_connections 2024-09-11T22:57:53.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.309+0000 7f141e7af740 1 -- 172.21.15.152:0/3591928027 wait complete. 2024-09-11T22:57:53.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141e7af740 1 -- start start 2024-09-11T22:57:53.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141e7af740 1 Processor -- start 2024-09-11T22:57:53.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141e7af740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b232835d0 0x555b2329a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141e7af740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b2329add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141e7af740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b23292780 0x555b23297860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141e7af740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f14141ac9c0 con 0x555b23292780 2024-09-11T22:57:53.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141e7af740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f14141ac2f0 con 0x555b23284b40 2024-09-11T22:57:53.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141e7af740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f14141a9510 con 0x555b232835d0 2024-09-11T22:57:53.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141e7af740 10 client.0 _ll_register_callbacks cb 0x7f141000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:53.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141a973640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b23292780 0x555b23297860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141a973640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b23292780 0x555b23297860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43316/0 (socket says 172.21.15.152:43316) 2024-09-11T22:57:53.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141a973640 1 -- 172.21.15.152:0/2357641621 learned_addr learned my addr 172.21.15.152:0/2357641621 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:53.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f1419971640 1 --2- 172.21.15.152:0/2357641621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b2329add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141a172640 1 --2- 172.21.15.152:0/2357641621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b232835d0 0x555b2329a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141a973640 1 -- 172.21.15.152:0/2357641621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b232835d0 msgr2=0x555b2329a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:53.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141a973640 1 --2- 172.21.15.152:0/2357641621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b232835d0 0x555b2329a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141a973640 1 -- 172.21.15.152:0/2357641621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 msgr2=0x555b2329add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:53.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141a973640 1 --2- 172.21.15.152:0/2357641621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b2329add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.310+0000 7f141a973640 1 -- 172.21.15.152:0/2357641621 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b23280960 con 0x555b23292780 2024-09-11T22:57:53.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.311+0000 7f141a973640 1 --2- 172.21.15.152:0/2357641621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b23292780 0x555b23297860 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f140c02ade0 tx=0x7f140c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34621a25d7bdba04 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:53.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.311+0000 7f13fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:53.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.311+0000 7f13fdffb640 1 -- 172.21.15.152:0/2357641621 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140c0319a0 con 0x555b23292780 2024-09-11T22:57:53.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.311+0000 7f13fdffb640 1 -- 172.21.15.152:0/2357641621 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f140c02f610 con 0x555b23292780 2024-09-11T22:57:53.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.311+0000 7f141e7af740 1 -- 172.21.15.152:0/2357641621 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f141009b670 con 0x555b23292780 2024-09-11T22:57:53.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.311+0000 7f13fdffb640 1 -- 172.21.15.152:0/2357641621 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f140c05a050 con 0x555b23292780 2024-09-11T22:57:53.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.311+0000 7f141e7af740 10 client.16824 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:53.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.311+0000 7f141e7af740 1 -- 172.21.15.152:0/2357641621 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555b2330d330 con 0x555b23292780 2024-09-11T22:57:53.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.311+0000 7f13fdffb640 1 -- 172.21.15.152:0/2357641621 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f141009b670 con 0x555b23292780 2024-09-11T22:57:53.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.311+0000 7f13fdffb640 1 -- 172.21.15.152:0/2357641621 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f140c030a30 con 0x555b23292780 2024-09-11T22:57:53.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.311+0000 7f141e7af740 10 client.16824 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:53.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.311+0000 7f141e7af740 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:57:53.324 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.311+0000 7f141e7af740 20 client.16824 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:53.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.311+0000 7f141e7af740 1 --2- 172.21.15.152:0/2357641621 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555b2330db00 0x555b2330ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:53.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.311+0000 7f141e7af740 4 client.16824 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:57:53.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.311+0000 7f141e7af740 1 -- 172.21.15.152:0/2357641621 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b231f0d70 con 0x555b2330db00 2024-09-11T22:57:53.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f141a172640 1 --2- 172.21.15.152:0/2357641621 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555b2330db00 0x555b2330ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:53.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f141a172640 1 --2- 172.21.15.152:0/2357641621 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555b2330db00 0x555b2330ff20 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ff95b0497a8a35e server_cookie=104f4062c4db9800 in_seq=0 out_seq=0 2024-09-11T22:57:53.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f13fdffb640 10 client.16824 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:53.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f13fdffb640 1 -- 172.21.15.152:0/2357641621 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b231f0d70 con 0x555b2330db00 2024-09-11T22:57:53.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f13fdffb640 10 client.16824 handle_command_reply: tid=0 2024-09-11T22:57:53.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f141e7af740 1 client.16824 shutdown 2024-09-11T22:57:53.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f141e7af740 1 -- 172.21.15.152:0/2357641621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b23292780 msgr2=0x555b23297860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:53.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f141e7af740 1 --2- 172.21.15.152:0/2357641621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b23292780 0x555b23297860 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f140c02ade0 tx=0x7f140c0095c0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.325 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f141e7af740 1 -- 172.21.15.152:0/2357641621 shutdown_connections 2024-09-11T22:57:53.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f141e7af740 1 --2- 172.21.15.152:0/2357641621 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b23292780 0x555b23297860 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f141e7af740 1 --2- 172.21.15.152:0/2357641621 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555b2330db00 0x555b2330ff20 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f141e7af740 1 --2- 172.21.15.152:0/2357641621 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555b23284b40 0x555b2329add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f141e7af740 1 --2- 172.21.15.152:0/2357641621 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555b232835d0 0x555b2329a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f141e7af740 1 -- 172.21.15.152:0/2357641621 >> 172.21.15.152:0/2357641621 conn(0x555b2324c310 msgr2=0x555b2327e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:53.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f13fdffb640 0 client.16824 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:53.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f141e7af740 1 -- 172.21.15.152:0/2357641621 shutdown_connections 2024-09-11T22:57:53.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.312+0000 7f141e7af740 1 -- 172.21.15.152:0/2357641621 wait complete. 2024-09-11T22:57:53.326 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.313+0000 7f141e7af740 20 client.16824 trim_cache size 0 max 16384 2024-09-11T22:57:53.326 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:57:53.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.313+0000 7f141c3d6640 1 -- 172.21.15.152:0/618633576 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f14000a2cf0 msgr2=0x7f14000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:53.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.313+0000 7f141c3d6640 1 --2- 172.21.15.152:0/618633576 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f14000a2cf0 0x7f14000a51b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f140c005cc0 tx=0x7f140c005da0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.313+0000 7f141c3d6640 1 -- 172.21.15.152:0/618633576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1414101de0 msgr2=0x7f14141983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:53.327 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.313+0000 7f141c3d6640 1 --2- 172.21.15.152:0/618633576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1414101de0 0x7f14141983f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f141000d950 tx=0x7f141000de10 comp rx=0 tx=0).stop 2024-09-11T22:57:53.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.313+0000 7f141c3d6640 1 -- 172.21.15.152:0/618633576 shutdown_connections 2024-09-11T22:57:53.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.313+0000 7f141c3d6640 1 --2- 172.21.15.152:0/618633576 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f14000a2cf0 0x7f14000a51b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.313+0000 7f141c3d6640 1 --2- 172.21.15.152:0/618633576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1414106f10 0x7f141419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.313+0000 7f141c3d6640 1 --2- 172.21.15.152:0/618633576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1414102790 0x7f1414198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.313+0000 7f141c3d6640 1 --2- 172.21.15.152:0/618633576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1414101de0 0x7f14141983f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:53.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.313+0000 7f141c3d6640 1 -- 172.21.15.152:0/618633576 >> 172.21.15.152:0/618633576 conn(0x7f14140fd8c0 msgr2=0x7f14140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:53.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.313+0000 7f141c3d6640 1 -- 172.21.15.152:0/618633576 shutdown_connections 2024-09-11T22:57:53.328 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:53.313+0000 7f141c3d6640 1 -- 172.21.15.152:0/618633576 wait complete. 2024-09-11T22:57:55.082 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:57:55.082 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:57:55.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:55 smithi191 ceph-mon[35211]: pgmap v153: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:55.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:55 smithi204 ceph-mon[35454]: pgmap v153: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:55.346 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:57:55.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:55 smithi152 ceph-mon[31181]: pgmap v153: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:55.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.907+0000 7f5a14ee3640 1 -- 172.21.15.152:0/3771977642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a10102570 msgr2=0x7f5a101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:55.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.907+0000 7f5a14ee3640 1 --2- 172.21.15.152:0/3771977642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a10102570 0x7f5a101029b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009900 tx=0x7f59f802a480 comp rx=0 tx=0).stop 2024-09-11T22:57:55.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.907+0000 7f5a14ee3640 1 -- 172.21.15.152:0/3771977642 shutdown_connections 2024-09-11T22:57:55.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.907+0000 7f5a14ee3640 1 --2- 172.21.15.152:0/3771977642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a10102ef0 0x7f5a1010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.907+0000 7f5a14ee3640 1 --2- 172.21.15.152:0/3771977642 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a10102570 0x7f5a101029b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.907+0000 7f5a14ee3640 1 --2- 172.21.15.152:0/3771977642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a10100e20 0x7f5a10101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.907+0000 7f5a14ee3640 1 -- 172.21.15.152:0/3771977642 >> 172.21.15.152:0/3771977642 conn(0x7f5a100fc7d0 msgr2=0x7f5a100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:55.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.907+0000 7f5a14ee3640 1 -- 172.21.15.152:0/3771977642 shutdown_connections 2024-09-11T22:57:55.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.907+0000 7f5a14ee3640 1 -- 172.21.15.152:0/3771977642 wait complete. 2024-09-11T22:57:55.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a14ee3640 1 Processor -- start 2024-09-11T22:57:55.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a14ee3640 1 -- start start 2024-09-11T22:57:55.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a14ee3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a10100e20 0x7f5a101992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a14ee3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a10102570 0x7f5a10199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a14ee3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a10102ef0 0x7f5a101a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a14ee3640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10110f80 con 0x7f5a10100e20 2024-09-11T22:57:55.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a14ee3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10110e00 con 0x7f5a10102570 2024-09-11T22:57:55.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a14ee3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10111100 con 0x7f5a10102ef0 2024-09-11T22:57:55.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a0e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a10100e20 0x7f5a101992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a0e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a10100e20 0x7f5a101992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43342/0 (socket says 172.21.15.152:43342) 2024-09-11T22:57:55.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a0e59c640 1 -- 172.21.15.152:0/1529448207 learned_addr learned my addr 172.21.15.152:0/1529448207 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:55.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a0dd9b640 1 --2- 172.21.15.152:0/1529448207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a10102570 0x7f5a10199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a0ed9d640 1 --2- 172.21.15.152:0/1529448207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a10102ef0 0x7f5a101a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a0e59c640 1 -- 172.21.15.152:0/1529448207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a10102ef0 msgr2=0x7f5a101a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:55.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a0e59c640 1 --2- 172.21.15.152:0/1529448207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a10102ef0 0x7f5a101a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a0e59c640 1 -- 172.21.15.152:0/1529448207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a10102570 msgr2=0x7f5a10199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:55.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a0e59c640 1 --2- 172.21.15.152:0/1529448207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a10102570 0x7f5a10199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a0e59c640 1 -- 172.21.15.152:0/1529448207 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a101a48b0 con 0x7f5a10100e20 2024-09-11T22:57:55.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a0dd9b640 1 --2- 172.21.15.152:0/1529448207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a10102570 0x7f5a10199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:57:55.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.908+0000 7f5a0e59c640 1 --2- 172.21.15.152:0/1529448207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a10100e20 0x7f5a101992f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000c970 tx=0x7f5a0000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:55.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.909+0000 7f5a077fe640 1 -- 172.21.15.152:0/1529448207 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00007bb0 con 0x7f5a10100e20 2024-09-11T22:57:55.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.909+0000 7f5a14ee3640 1 -- 172.21.15.152:0/1529448207 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a101a4a80 con 0x7f5a10100e20 2024-09-11T22:57:55.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.909+0000 7f5a077fe640 1 -- 172.21.15.152:0/1529448207 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a00007d50 con 0x7f5a10100e20 2024-09-11T22:57:55.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.909+0000 7f5a077fe640 1 -- 172.21.15.152:0/1529448207 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000f450 con 0x7f5a10100e20 2024-09-11T22:57:55.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.909+0000 7f5a14ee3640 1 -- 172.21.15.152:0/1529448207 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a101a4f90 con 0x7f5a10100e20 2024-09-11T22:57:55.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.911+0000 7f5a077fe640 1 -- 172.21.15.152:0/1529448207 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5a00005020 con 0x7f5a10100e20 2024-09-11T22:57:55.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.911+0000 7f5a172bc740 1 Processor -- start 2024-09-11T22:57:55.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a172bc740 1 -- start start 2024-09-11T22:57:55.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a172bc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f7db60 0x562d86f7df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a172bc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7e480 0x562d86f88800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a172bc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f88d40 0x562d86f8b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a172bc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x562d86ee3280 con 0x562d86f7e480 2024-09-11T22:57:55.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a172bc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562d86e8ab10 con 0x562d86f88d40 2024-09-11T22:57:55.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a172bc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562d86edf240 con 0x562d86f7db60 2024-09-11T22:57:55.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a0e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7e480 0x562d86f88800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a0e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7e480 0x562d86f88800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43354/0 (socket says 172.21.15.152:43354) 2024-09-11T22:57:55.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a0e59c640 1 -- 172.21.15.152:0/3277464709 learned_addr learned my addr 172.21.15.152:0/3277464709 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:55.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a0ed9d640 1 --2- 172.21.15.152:0/3277464709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f7db60 0x562d86f7df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a0dd9b640 1 --2- 172.21.15.152:0/3277464709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f88d40 0x562d86f8b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a077fe640 1 --2- 172.21.15.152:0/1529448207 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f59fc0a2cf0 0x7f59fc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a0e59c640 1 -- 172.21.15.152:0/3277464709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f7db60 msgr2=0x562d86f7df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:55.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a0e59c640 1 --2- 172.21.15.152:0/3277464709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f7db60 0x562d86f7df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a0e59c640 1 -- 172.21.15.152:0/3277464709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f88d40 msgr2=0x562d86f8b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:55.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a0e59c640 1 --2- 172.21.15.152:0/3277464709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f88d40 0x562d86f8b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a0e59c640 1 -- 172.21.15.152:0/3277464709 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d86e971f0 con 0x562d86f7e480 2024-09-11T22:57:55.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.912+0000 7f5a0ed9d640 1 --2- 172.21.15.152:0/1529448207 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f59fc0a2cf0 0x7f59fc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.913+0000 7f5a077fe640 1 -- 172.21.15.152:0/1529448207 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f5a000cf290 con 0x7f5a10100e20 2024-09-11T22:57:55.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.913+0000 7f5a0e59c640 1 --2- 172.21.15.152:0/3277464709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7e480 0x562d86f88800 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5a000cc9a0 tx=0x7f5a00066380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b69da4cf760c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:55.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.913+0000 7f5a04ff9640 1 -- 172.21.15.152:0/3277464709 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0001d020 con 0x562d86f7e480 2024-09-11T22:57:55.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.913+0000 7f5a04ff9640 1 -- 172.21.15.152:0/3277464709 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a00066890 con 0x562d86f7e480 2024-09-11T22:57:55.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.913+0000 7f5a0ed9d640 1 --2- 172.21.15.152:0/1529448207 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f59fc0a2cf0 0x7f59fc0a51b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5a08005cc0 tx=0x7f5a08004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:55.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.913+0000 7f5a04ff9640 1 -- 172.21.15.152:0/3277464709 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a00064940 con 0x562d86f7e480 2024-09-11T22:57:55.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 -- 172.21.15.152:0/3277464709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7e480 msgr2=0x562d86f88800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:55.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 --2- 172.21.15.152:0/3277464709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7e480 0x562d86f88800 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5a000cc9a0 tx=0x7f5a00066380 comp rx=0 tx=0).stop 2024-09-11T22:57:55.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 -- 172.21.15.152:0/3277464709 shutdown_connections 2024-09-11T22:57:55.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 --2- 172.21.15.152:0/3277464709 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f88d40 0x562d86f8b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 --2- 172.21.15.152:0/3277464709 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7e480 0x562d86f88800 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 --2- 172.21.15.152:0/3277464709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f7db60 0x562d86f7df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 -- 172.21.15.152:0/3277464709 >> 172.21.15.152:0/3277464709 conn(0x562d86f7a0f0 msgr2=0x562d86f7a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:55.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 -- 172.21.15.152:0/3277464709 shutdown_connections 2024-09-11T22:57:55.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 -- 172.21.15.152:0/3277464709 wait complete. 2024-09-11T22:57:55.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 -- start start 2024-09-11T22:57:55.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 Processor -- start 2024-09-11T22:57:55.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7db60 0x562d86f85c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f7e480 0x562d86f86140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f88d40 0x562d86f86680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x562d86f8b8b0 con 0x562d86f7db60 2024-09-11T22:57:55.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562d86e9df10 con 0x562d86f7e480 2024-09-11T22:57:55.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562d86f8ba30 con 0x562d86f88d40 2024-09-11T22:57:55.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a172bc740 10 client.0 _ll_register_callbacks cb 0x562d86e66a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:55.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a0e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7db60 0x562d86f85c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a0e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7db60 0x562d86f85c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43370/0 (socket says 172.21.15.152:43370) 2024-09-11T22:57:55.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.914+0000 7f5a0e59c640 1 -- 172.21.15.152:0/343554122 learned_addr learned my addr 172.21.15.152:0/343554122 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:55.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f5a0ed9d640 1 --2- 172.21.15.152:0/343554122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f88d40 0x562d86f86680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f5a0dd9b640 1 --2- 172.21.15.152:0/343554122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f7e480 0x562d86f86140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f5a0e59c640 1 -- 172.21.15.152:0/343554122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f88d40 msgr2=0x562d86f86680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:55.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f5a0e59c640 1 --2- 172.21.15.152:0/343554122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f88d40 0x562d86f86680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f5a0e59c640 1 -- 172.21.15.152:0/343554122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f7e480 msgr2=0x562d86f86140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:55.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f5a0e59c640 1 --2- 172.21.15.152:0/343554122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f7e480 0x562d86f86140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f5a0e59c640 1 -- 172.21.15.152:0/343554122 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d86c546a0 con 0x562d86f7db60 2024-09-11T22:57:55.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f5a0ed9d640 1 --2- 172.21.15.152:0/343554122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f88d40 0x562d86f86680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:55.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f5a0dd9b640 1 --2- 172.21.15.152:0/343554122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f7e480 0x562d86f86140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:57:55.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f5a0e59c640 1 --2- 172.21.15.152:0/343554122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7db60 0x562d86f85c00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f5a0009b2d0 tx=0x7f5a00005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c516bb099a57556 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:55.939 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f59f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:55.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f59f5ffb640 1 -- 172.21.15.152:0/343554122 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0001d040 con 0x562d86f7db60 2024-09-11T22:57:55.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f59f5ffb640 1 -- 172.21.15.152:0/343554122 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f5a00099b00 con 0x562d86f7db60 2024-09-11T22:57:55.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f5a172bc740 1 -- 172.21.15.152:0/343554122 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a00066890 con 0x562d86f7db60 2024-09-11T22:57:55.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f59f5ffb640 1 -- 172.21.15.152:0/343554122 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5a000d2a90 con 0x562d86f7db60 2024-09-11T22:57:55.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f5a172bc740 10 client.16836 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:55.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f5a172bc740 1 -- 172.21.15.152:0/343554122 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562d86f97490 con 0x562d86f7db60 2024-09-11T22:57:55.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.915+0000 7f59f5ffb640 1 -- 172.21.15.152:0/343554122 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a00066890 con 0x562d86f7db60 2024-09-11T22:57:55.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.916+0000 7f59f5ffb640 1 -- 172.21.15.152:0/343554122 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5a00005310 con 0x562d86f7db60 2024-09-11T22:57:55.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.916+0000 7f5a172bc740 10 client.16836 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:55.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.916+0000 7f5a172bc740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:57:55.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.916+0000 7f5a172bc740 20 client.16836 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:55.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.916+0000 7f5a172bc740 1 --2- 172.21.15.152:0/343554122 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562d86f987e0 0x562d86f9abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.916+0000 7f5a172bc740 4 client.16836 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:57:55.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.916+0000 7f5a172bc740 1 -- 172.21.15.152:0/343554122 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d86ef07f0 con 0x562d86f987e0 2024-09-11T22:57:55.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.916+0000 7f5a0dd9b640 1 --2- 172.21.15.152:0/343554122 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562d86f987e0 0x562d86f9abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.916+0000 7f59f5ffb640 10 client.16836 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:55.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.916+0000 7f5a0dd9b640 1 --2- 172.21.15.152:0/343554122 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562d86f987e0 0x562d86f9abd0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e09035683507608b server_cookie=b5e3c5f12224526d in_seq=0 out_seq=0 2024-09-11T22:57:55.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.917+0000 7f59f5ffb640 1 -- 172.21.15.152:0/343554122 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d86ef07f0 con 0x562d86f987e0 2024-09-11T22:57:55.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.917+0000 7f59f5ffb640 10 client.16836 handle_command_reply: tid=0 2024-09-11T22:57:55.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.917+0000 7f5a172bc740 1 client.16836 shutdown 2024-09-11T22:57:55.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.917+0000 7f5a172bc740 1 -- 172.21.15.152:0/343554122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7db60 msgr2=0x562d86f85c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:55.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.917+0000 7f5a172bc740 1 --2- 172.21.15.152:0/343554122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7db60 0x562d86f85c00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f5a0009b2d0 tx=0x7f5a00005700 comp rx=0 tx=0).stop 2024-09-11T22:57:55.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.917+0000 7f5a172bc740 1 -- 172.21.15.152:0/343554122 shutdown_connections 2024-09-11T22:57:55.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.917+0000 7f5a172bc740 1 --2- 172.21.15.152:0/343554122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f88d40 0x562d86f86680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.917+0000 7f5a172bc740 1 --2- 172.21.15.152:0/343554122 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562d86f987e0 0x562d86f9abd0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.917+0000 7f5a172bc740 1 --2- 172.21.15.152:0/343554122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f7e480 0x562d86f86140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.917+0000 7f5a172bc740 1 --2- 172.21.15.152:0/343554122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7db60 0x562d86f85c00 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.917+0000 7f59f5ffb640 0 client.16836 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:55.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.917+0000 7f5a172bc740 1 -- 172.21.15.152:0/343554122 >> 172.21.15.152:0/343554122 conn(0x562d86f7a0f0 msgr2=0x562d86f7fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:55.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.917+0000 7f5a172bc740 1 -- 172.21.15.152:0/343554122 shutdown_connections 2024-09-11T22:57:55.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.917+0000 7f5a172bc740 1 -- 172.21.15.152:0/343554122 wait complete. 2024-09-11T22:57:55.942 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.917+0000 7f5a172bc740 20 client.16836 trim_cache size 0 max 16384 2024-09-11T22:57:55.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.934+0000 7f5a172bc740 1 Processor -- start 2024-09-11T22:57:55.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.934+0000 7f5a172bc740 1 -- start start 2024-09-11T22:57:55.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.934+0000 7f5a172bc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f7db60 0x562d86f87a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f5a172bc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f7e480 0x562d86f9dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f5a172bc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f88d40 0x562d86f9e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f5a172bc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101ad800 con 0x562d86f88d40 2024-09-11T22:57:55.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f5a172bc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101af340 con 0x562d86f7db60 2024-09-11T22:57:55.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f5a172bc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101aded0 con 0x562d86f7e480 2024-09-11T22:57:55.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f5a0ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f88d40 0x562d86f9e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f5a0ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f88d40 0x562d86f9e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43382/0 (socket says 172.21.15.152:43382) 2024-09-11T22:57:55.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f5a0ed9d640 1 -- 172.21.15.152:0/2673174451 learned_addr learned my addr 172.21.15.152:0/2673174451 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:55.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f5a0e59c640 1 --2- 172.21.15.152:0/2673174451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f7db60 0x562d86f87a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f5a0dd9b640 1 --2- 172.21.15.152:0/2673174451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f7e480 0x562d86f9dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f5a0ed9d640 1 -- 172.21.15.152:0/2673174451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f7e480 msgr2=0x562d86f9dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:55.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f5a0ed9d640 1 --2- 172.21.15.152:0/2673174451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f7e480 0x562d86f9dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f5a0ed9d640 1 -- 172.21.15.152:0/2673174451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f7db60 msgr2=0x562d86f87a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:55.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f5a0ed9d640 1 --2- 172.21.15.152:0/2673174451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f7db60 0x562d86f87a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f5a0ed9d640 1 -- 172.21.15.152:0/2673174451 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d86f79f20 con 0x562d86f88d40 2024-09-11T22:57:55.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f5a0ed9d640 1 --2- 172.21.15.152:0/2673174451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f88d40 0x562d86f9e130 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800b350 tx=0x7f5a08013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91632bff4630d708 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:55.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f59f7fff640 1 -- 172.21.15.152:0/2673174451 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a080127a0 con 0x562d86f88d40 2024-09-11T22:57:55.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f59f7fff640 1 -- 172.21.15.152:0/2673174451 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a08012e20 con 0x562d86f88d40 2024-09-11T22:57:55.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.935+0000 7f59f7fff640 1 -- 172.21.15.152:0/2673174451 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08025400 con 0x562d86f88d40 2024-09-11T22:57:55.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.936+0000 7f5a172bc740 1 -- 172.21.15.152:0/2673174451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f88d40 msgr2=0x562d86f9e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:55.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.936+0000 7f5a172bc740 1 --2- 172.21.15.152:0/2673174451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f88d40 0x562d86f9e130 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800b350 tx=0x7f5a08013000 comp rx=0 tx=0).stop 2024-09-11T22:57:55.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.936+0000 7f5a172bc740 1 -- 172.21.15.152:0/2673174451 shutdown_connections 2024-09-11T22:57:55.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.936+0000 7f5a172bc740 1 --2- 172.21.15.152:0/2673174451 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f88d40 0x562d86f9e130 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.936+0000 7f5a172bc740 1 --2- 172.21.15.152:0/2673174451 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f7e480 0x562d86f9dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.936+0000 7f5a172bc740 1 --2- 172.21.15.152:0/2673174451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f7db60 0x562d86f87a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.936+0000 7f5a172bc740 1 -- 172.21.15.152:0/2673174451 >> 172.21.15.152:0/2673174451 conn(0x562d86f7a0f0 msgr2=0x562d86f87350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:55.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.936+0000 7f5a172bc740 1 -- 172.21.15.152:0/2673174451 shutdown_connections 2024-09-11T22:57:55.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.936+0000 7f5a172bc740 1 -- 172.21.15.152:0/2673174451 wait complete. 2024-09-11T22:57:55.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.936+0000 7f5a172bc740 1 -- start start 2024-09-11T22:57:55.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.936+0000 7f5a172bc740 1 Processor -- start 2024-09-11T22:57:55.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a172bc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f7db60 0x562d86f9b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a172bc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7e480 0x562d86f9bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a172bc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f88d40 0x562d86f9c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a172bc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101aec70 con 0x562d86f7e480 2024-09-11T22:57:55.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a172bc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101ae5a0 con 0x562d86f88d40 2024-09-11T22:57:55.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a172bc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101ab7e0 con 0x562d86f7db60 2024-09-11T22:57:55.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a172bc740 10 client.0 _ll_register_callbacks cb 0x7f59f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:55.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a0dd9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7e480 0x562d86f9bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a0dd9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7e480 0x562d86f9bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43390/0 (socket says 172.21.15.152:43390) 2024-09-11T22:57:55.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a0dd9b640 1 -- 172.21.15.152:0/2237997526 learned_addr learned my addr 172.21.15.152:0/2237997526 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:55.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a0ed9d640 1 --2- 172.21.15.152:0/2237997526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f88d40 0x562d86f9c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a0e59c640 1 --2- 172.21.15.152:0/2237997526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f7db60 0x562d86f9b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a0dd9b640 1 -- 172.21.15.152:0/2237997526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f7db60 msgr2=0x562d86f9b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:55.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a0dd9b640 1 --2- 172.21.15.152:0/2237997526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f7db60 0x562d86f9b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a0dd9b640 1 -- 172.21.15.152:0/2237997526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f88d40 msgr2=0x562d86f9c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:55.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a0dd9b640 1 --2- 172.21.15.152:0/2237997526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f88d40 0x562d86f9c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a0dd9b640 1 -- 172.21.15.152:0/2237997526 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d87002150 con 0x562d86f7e480 2024-09-11T22:57:55.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f5a0dd9b640 1 --2- 172.21.15.152:0/2237997526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7e480 0x562d86f9bed0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009900 tx=0x7f59f8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e58ecc14716d06 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:55.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f59f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:55.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f59f5ffb640 1 -- 172.21.15.152:0/2237997526 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8035040 con 0x562d86f7e480 2024-09-11T22:57:55.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.937+0000 7f59f5ffb640 1 -- 172.21.15.152:0/2237997526 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f59f8005510 con 0x562d86f7e480 2024-09-11T22:57:55.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.938+0000 7f5a172bc740 1 -- 172.21.15.152:0/2237997526 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a08012e20 con 0x562d86f7e480 2024-09-11T22:57:55.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.938+0000 7f59f5ffb640 1 -- 172.21.15.152:0/2237997526 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f59f80046c0 con 0x562d86f7e480 2024-09-11T22:57:55.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.938+0000 7f5a172bc740 10 client.16842 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:55.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.938+0000 7f5a172bc740 1 -- 172.21.15.152:0/2237997526 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562d870034e0 con 0x562d86f7e480 2024-09-11T22:57:55.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.938+0000 7f59f5ffb640 1 -- 172.21.15.152:0/2237997526 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a08012e20 con 0x562d86f7e480 2024-09-11T22:57:55.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.938+0000 7f59f5ffb640 1 -- 172.21.15.152:0/2237997526 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f59f8046070 con 0x562d86f7e480 2024-09-11T22:57:55.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.938+0000 7f5a172bc740 10 client.16842 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:55.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.938+0000 7f5a172bc740 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:57:55.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.938+0000 7f5a172bc740 20 client.16842 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:55.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.938+0000 7f5a172bc740 1 --2- 172.21.15.152:0/2237997526 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562d87003cc0 0x562d870060e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:55.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.938+0000 7f5a172bc740 4 client.16842 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:57:55.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.938+0000 7f5a172bc740 1 -- 172.21.15.152:0/2237997526 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562d86eead00 con 0x562d87003cc0 2024-09-11T22:57:55.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.938+0000 7f5a0e59c640 1 --2- 172.21.15.152:0/2237997526 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562d87003cc0 0x562d870060e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:55.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.939+0000 7f5a0e59c640 1 --2- 172.21.15.152:0/2237997526 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562d87003cc0 0x562d870060e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80b2c184acb50d5b server_cookie=48f95088ac487268 in_seq=0 out_seq=0 2024-09-11T22:57:55.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.939+0000 7f59f5ffb640 10 client.16842 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:55.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.940+0000 7f59f5ffb640 1 -- 172.21.15.152:0/2237997526 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562d86eead00 con 0x562d87003cc0 2024-09-11T22:57:55.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.940+0000 7f59f5ffb640 10 client.16842 handle_command_reply: tid=0 2024-09-11T22:57:55.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.940+0000 7f5a172bc740 1 client.16842 shutdown 2024-09-11T22:57:55.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.940+0000 7f5a172bc740 1 -- 172.21.15.152:0/2237997526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7e480 msgr2=0x562d86f9bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:55.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.940+0000 7f5a172bc740 1 --2- 172.21.15.152:0/2237997526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7e480 0x562d86f9bed0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009900 tx=0x7f59f8004610 comp rx=0 tx=0).stop 2024-09-11T22:57:55.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.940+0000 7f5a172bc740 1 -- 172.21.15.152:0/2237997526 shutdown_connections 2024-09-11T22:57:55.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.940+0000 7f5a172bc740 1 --2- 172.21.15.152:0/2237997526 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562d87003cc0 0x562d870060e0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.940+0000 7f5a172bc740 1 --2- 172.21.15.152:0/2237997526 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562d86f88d40 0x562d86f9c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.940+0000 7f5a172bc740 1 --2- 172.21.15.152:0/2237997526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562d86f7e480 0x562d86f9bed0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.940+0000 7f5a172bc740 1 --2- 172.21.15.152:0/2237997526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562d86f7db60 0x562d86f9b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.940+0000 7f5a172bc740 1 -- 172.21.15.152:0/2237997526 >> 172.21.15.152:0/2237997526 conn(0x562d86f7a0f0 msgr2=0x562d86f944a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:55.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.940+0000 7f59f5ffb640 0 client.16842 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:55.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.940+0000 7f5a172bc740 1 -- 172.21.15.152:0/2237997526 shutdown_connections 2024-09-11T22:57:55.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.940+0000 7f5a172bc740 1 -- 172.21.15.152:0/2237997526 wait complete. 2024-09-11T22:57:55.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.940+0000 7f5a172bc740 20 client.16842 trim_cache size 0 max 16384 2024-09-11T22:57:55.960 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:57:55.960 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T22:57:55.960 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "acdb25fd-cdd9-41a6-b7a6-9f440f1f69ae", 2024-09-11T22:57:55.960 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T22:57:55.960 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:57:55.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.941+0000 7f5a14ee3640 1 -- 172.21.15.152:0/1529448207 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f59fc0a2cf0 msgr2=0x7f59fc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:55.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.941+0000 7f5a14ee3640 1 --2- 172.21.15.152:0/1529448207 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f59fc0a2cf0 0x7f59fc0a51b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5a08005cc0 tx=0x7f5a08004a40 comp rx=0 tx=0).stop 2024-09-11T22:57:55.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.941+0000 7f5a14ee3640 1 -- 172.21.15.152:0/1529448207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a10100e20 msgr2=0x7f5a101992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:55.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.941+0000 7f5a14ee3640 1 --2- 172.21.15.152:0/1529448207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a10100e20 0x7f5a101992f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000c970 tx=0x7f5a0000ce30 comp rx=0 tx=0).stop 2024-09-11T22:57:55.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.941+0000 7f5a14ee3640 1 -- 172.21.15.152:0/1529448207 shutdown_connections 2024-09-11T22:57:55.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.941+0000 7f5a14ee3640 1 --2- 172.21.15.152:0/1529448207 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f59fc0a2cf0 0x7f59fc0a51b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.941+0000 7f5a14ee3640 1 --2- 172.21.15.152:0/1529448207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a10102ef0 0x7f5a101a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.941+0000 7f5a14ee3640 1 --2- 172.21.15.152:0/1529448207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5a10102570 0x7f5a10199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.941+0000 7f5a14ee3640 1 --2- 172.21.15.152:0/1529448207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5a10100e20 0x7f5a101992f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:55.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.941+0000 7f5a14ee3640 1 -- 172.21.15.152:0/1529448207 >> 172.21.15.152:0/1529448207 conn(0x7f5a100fc7d0 msgr2=0x7f5a10105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:55.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.941+0000 7f5a14ee3640 1 -- 172.21.15.152:0/1529448207 shutdown_connections 2024-09-11T22:57:55.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:55.941+0000 7f5a14ee3640 1 -- 172.21.15.152:0/1529448207 wait complete. 2024-09-11T22:57:56.767 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:57:57.015 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:57:57.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:57 smithi152 ceph-mon[31181]: pgmap v154: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:57:57.124 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:57 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T22:57:57.125 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:57 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:57:57.125 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:57 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T22:57:57.125 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:57 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:57:57.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:57 smithi191 ceph-mon[35211]: pgmap v154: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:57:57.288 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:57 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T22:57:57.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:57 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:57:57.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:57 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T22:57:57.289 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:57 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:57:57.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:57 smithi204 ceph-mon[35454]: pgmap v154: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:57:57.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:57 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T22:57:57.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:57 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:57:57.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:57 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T22:57:57.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:57 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:57:57.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.539+0000 7f9fd1b8b640 1 -- 172.21.15.152:0/3605819989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fcc102790 msgr2=0x7f9fcc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:57.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.539+0000 7f9fd1b8b640 1 --2- 172.21.15.152:0/3605819989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fcc102790 0x7f9fcc1067e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4009900 tx=0x7f9fb402a410 comp rx=0 tx=0).stop 2024-09-11T22:57:57.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.539+0000 7f9fd1b8b640 1 -- 172.21.15.152:0/3605819989 shutdown_connections 2024-09-11T22:57:57.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.539+0000 7f9fd1b8b640 1 --2- 172.21.15.152:0/3605819989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fcc106f10 0x7f9fcc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.539+0000 7f9fd1b8b640 1 --2- 172.21.15.152:0/3605819989 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fcc102790 0x7f9fcc1067e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.539+0000 7f9fd1b8b640 1 --2- 172.21.15.152:0/3605819989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fcc101de0 0x7f9fcc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.539+0000 7f9fd1b8b640 1 -- 172.21.15.152:0/3605819989 >> 172.21.15.152:0/3605819989 conn(0x7f9fcc0fd8c0 msgr2=0x7f9fcc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:57.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.540+0000 7f9fd1b8b640 1 -- 172.21.15.152:0/3605819989 shutdown_connections 2024-09-11T22:57:57.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.540+0000 7f9fd1b8b640 1 -- 172.21.15.152:0/3605819989 wait complete. 2024-09-11T22:57:57.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.540+0000 7f9fd1b8b640 1 Processor -- start 2024-09-11T22:57:57.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.540+0000 7f9fd1b8b640 1 -- start start 2024-09-11T22:57:57.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fd1b8b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fcc101de0 0x7f9fcc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fd1b8b640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fcc102790 0x7f9fcc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fd1b8b640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fcc106f10 0x7f9fcc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fd1b8b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc111f70 con 0x7f9fcc101de0 2024-09-11T22:57:57.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fd1b8b640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc111df0 con 0x7f9fcc102790 2024-09-11T22:57:57.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fd1b8b640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1120f0 con 0x7f9fcc106f10 2024-09-11T22:57:57.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fcb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fcc101de0 0x7f9fcc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fcb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fcc101de0 0x7f9fcc198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43400/0 (socket says 172.21.15.152:43400) 2024-09-11T22:57:57.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fcb7fe640 1 -- 172.21.15.152:0/2644747094 learned_addr learned my addr 172.21.15.152:0/2644747094 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:57.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fcbfff640 1 --2- 172.21.15.152:0/2644747094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fcc106f10 0x7f9fcc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fc3fff640 1 --2- 172.21.15.152:0/2644747094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fcc102790 0x7f9fcc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fcb7fe640 1 -- 172.21.15.152:0/2644747094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fcc106f10 msgr2=0x7f9fcc19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:57.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fcb7fe640 1 --2- 172.21.15.152:0/2644747094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fcc106f10 0x7f9fcc19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fcb7fe640 1 -- 172.21.15.152:0/2644747094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fcc102790 msgr2=0x7f9fcc1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:57.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fcb7fe640 1 --2- 172.21.15.152:0/2644747094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fcc102790 0x7f9fcc1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fcb7fe640 1 -- 172.21.15.152:0/2644747094 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc19d440 con 0x7f9fcc101de0 2024-09-11T22:57:57.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fcbfff640 1 --2- 172.21.15.152:0/2644747094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fcc106f10 0x7f9fcc19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:57:57.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fc3fff640 1 --2- 172.21.15.152:0/2644747094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fcc102790 0x7f9fcc1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:57:57.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fcb7fe640 1 --2- 172.21.15.152:0/2644747094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fcc101de0 0x7f9fcc198490 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00d950 tx=0x7f9fbc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:57.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fc97fa640 1 -- 172.21.15.152:0/2644747094 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc017070 con 0x7f9fcc101de0 2024-09-11T22:57:57.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fc97fa640 1 -- 172.21.15.152:0/2644747094 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fbc009c30 con 0x7f9fcc101de0 2024-09-11T22:57:57.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fc97fa640 1 -- 172.21.15.152:0/2644747094 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc00f6b0 con 0x7f9fcc101de0 2024-09-11T22:57:57.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fd1b8b640 1 -- 172.21.15.152:0/2644747094 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fcc19d730 con 0x7f9fcc101de0 2024-09-11T22:57:57.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.541+0000 7f9fd1b8b640 1 -- 172.21.15.152:0/2644747094 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fcc1a5010 con 0x7f9fcc101de0 2024-09-11T22:57:57.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.544+0000 7f9fc97fa640 1 -- 172.21.15.152:0/2644747094 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9fbc0109d0 con 0x7f9fcc101de0 2024-09-11T22:57:57.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.544+0000 7f9fd3f64740 1 Processor -- start 2024-09-11T22:57:57.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.544+0000 7f9fd3f64740 1 -- start start 2024-09-11T22:57:57.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fc97fa640 1 --2- 172.21.15.152:0/2644747094 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9fb80a2cf0 0x7f9fb80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fd3f64740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b24a0 0x55d47c7b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fd3f64740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7b2dc0 0x55d47c7bd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fd3f64740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7bd680 0x55d47c7bfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fd3f64740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d47c7182e0 con 0x55d47c7b2dc0 2024-09-11T22:57:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fd3f64740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d47c6bfce0 con 0x55d47c7bd680 2024-09-11T22:57:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fd3f64740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d47c7142a0 con 0x55d47c7b24a0 2024-09-11T22:57:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fcbfff640 1 --2- 172.21.15.152:0/2644747094 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9fb80a2cf0 0x7f9fb80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fcb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b24a0 0x55d47c7b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fcb7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b24a0 0x55d47c7b2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:60950/0 (socket says 172.21.15.152:60950) 2024-09-11T22:57:57.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fcb7fe640 1 -- 172.21.15.152:0/2439471250 learned_addr learned my addr 172.21.15.152:0/2439471250 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:57.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fc3fff640 1 --2- 172.21.15.152:0/2439471250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7b2dc0 0x55d47c7bd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fcbfff640 1 --2- 172.21.15.152:0/2439471250 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7bd680 0x55d47c7bfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fc97fa640 1 -- 172.21.15.152:0/2644747094 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f9fbc0cf140 con 0x7f9fcc101de0 2024-09-11T22:57:57.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fc3fff640 1 -- 172.21.15.152:0/2439471250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b24a0 msgr2=0x55d47c7b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:57.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fc3fff640 1 --2- 172.21.15.152:0/2439471250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b24a0 0x55d47c7b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fc3fff640 1 -- 172.21.15.152:0/2439471250 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7bd680 msgr2=0x55d47c7bfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:57.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fc3fff640 1 --2- 172.21.15.152:0/2439471250 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7bd680 0x55d47c7bfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fc3fff640 1 -- 172.21.15.152:0/2439471250 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d47c6a4440 con 0x55d47c7b2dc0 2024-09-11T22:57:57.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.545+0000 7f9fcbfff640 1 --2- 172.21.15.152:0/2644747094 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9fb80a2cf0 0x7f9fb80a51b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4005fd0 tx=0x7f9fc40139f0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:57.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.546+0000 7f9fc3fff640 1 --2- 172.21.15.152:0/2439471250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7b2dc0 0x55d47c7bd140 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4000c00 tx=0x7f9fb4032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50915393a80bf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:57.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.546+0000 7f9fc1ffb640 1 -- 172.21.15.152:0/2439471250 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb403f070 con 0x55d47c7b2dc0 2024-09-11T22:57:57.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.546+0000 7f9fc1ffb640 1 -- 172.21.15.152:0/2439471250 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb4032e20 con 0x55d47c7b2dc0 2024-09-11T22:57:57.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.546+0000 7f9fc1ffb640 1 -- 172.21.15.152:0/2439471250 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4035070 con 0x55d47c7b2dc0 2024-09-11T22:57:57.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.546+0000 7f9fd3f64740 1 -- 172.21.15.152:0/2439471250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7b2dc0 msgr2=0x55d47c7bd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:57.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.546+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/2439471250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7b2dc0 0x55d47c7bd140 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4000c00 tx=0x7f9fb4032cb0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.546+0000 7f9fd3f64740 1 -- 172.21.15.152:0/2439471250 shutdown_connections 2024-09-11T22:57:57.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.546+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/2439471250 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7bd680 0x55d47c7bfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.546+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/2439471250 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7b2dc0 0x55d47c7bd140 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.546+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/2439471250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b24a0 0x55d47c7b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.546+0000 7f9fd3f64740 1 -- 172.21.15.152:0/2439471250 >> 172.21.15.152:0/2439471250 conn(0x55d47c7aea30 msgr2=0x55d47c7aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:57.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.546+0000 7f9fd3f64740 1 -- 172.21.15.152:0/2439471250 shutdown_connections 2024-09-11T22:57:57.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.547+0000 7f9fd3f64740 1 -- 172.21.15.152:0/2439471250 wait complete. 2024-09-11T22:57:57.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.547+0000 7f9fd3f64740 1 -- start start 2024-09-11T22:57:57.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.547+0000 7f9fd3f64740 1 Processor -- start 2024-09-11T22:57:57.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.547+0000 7f9fd3f64740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7b24a0 0x55d47c7bd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.547+0000 7f9fd3f64740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b2dc0 0x55d47c7ba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.547+0000 7f9fd3f64740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7bd680 0x55d47c7ba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.547+0000 7f9fd3f64740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d47c7c4660 con 0x55d47c7bd680 2024-09-11T22:57:57.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.547+0000 7f9fd3f64740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d47c6d3050 con 0x55d47c7b24a0 2024-09-11T22:57:57.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.547+0000 7f9fd3f64740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d47c7c47e0 con 0x55d47c7b2dc0 2024-09-11T22:57:57.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.547+0000 7f9fd3f64740 10 client.0 _ll_register_callbacks cb 0x55d47c713160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:57.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9fcbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7bd680 0x55d47c7ba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9fcbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7bd680 0x55d47c7ba8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43420/0 (socket says 172.21.15.152:43420) 2024-09-11T22:57:57.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9fcbfff640 1 -- 172.21.15.152:0/3382189006 learned_addr learned my addr 172.21.15.152:0/3382189006 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:57.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9fcb7fe640 1 --2- 172.21.15.152:0/3382189006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7b24a0 0x55d47c7bd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9fc3fff640 1 --2- 172.21.15.152:0/3382189006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b2dc0 0x55d47c7ba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9fcbfff640 1 -- 172.21.15.152:0/3382189006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b2dc0 msgr2=0x55d47c7ba2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:57.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9fcbfff640 1 --2- 172.21.15.152:0/3382189006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b2dc0 0x55d47c7ba2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9fcbfff640 1 -- 172.21.15.152:0/3382189006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7b24a0 msgr2=0x55d47c7bd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:57.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9fcbfff640 1 --2- 172.21.15.152:0/3382189006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7b24a0 0x55d47c7bd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9fcbfff640 1 -- 172.21.15.152:0/3382189006 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d47c4893c0 con 0x55d47c7bd680 2024-09-11T22:57:57.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9fc3fff640 1 --2- 172.21.15.152:0/3382189006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b2dc0 0x55d47c7ba2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:57:57.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9fcbfff640 1 --2- 172.21.15.152:0/3382189006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7bd680 0x55d47c7ba8e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400f750 tx=0x7f9fc4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357021a7d3a36cad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:57.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9faaffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:57.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9faaffd640 1 -- 172.21.15.152:0/3382189006 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4023070 con 0x55d47c7bd680 2024-09-11T22:57:57.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9faaffd640 1 -- 172.21.15.152:0/3382189006 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f9fc4010790 con 0x55d47c7bd680 2024-09-11T22:57:57.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9fd3f64740 1 -- 172.21.15.152:0/3382189006 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb4032e20 con 0x55d47c7bd680 2024-09-11T22:57:57.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9faaffd640 1 -- 172.21.15.152:0/3382189006 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9fc402f050 con 0x55d47c7bd680 2024-09-11T22:57:57.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9fd3f64740 10 client.16854 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:57.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.548+0000 7f9fd3f64740 1 -- 172.21.15.152:0/3382189006 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d47c7d00d0 con 0x55d47c7bd680 2024-09-11T22:57:57.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.549+0000 7f9faaffd640 1 -- 172.21.15.152:0/3382189006 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb4032e20 con 0x55d47c7bd680 2024-09-11T22:57:57.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.549+0000 7f9faaffd640 1 -- 172.21.15.152:0/3382189006 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9fc4011bb0 con 0x55d47c7bd680 2024-09-11T22:57:57.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.549+0000 7f9fd3f64740 10 client.16854 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:57.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.549+0000 7f9fd3f64740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:57:57.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.549+0000 7f9fd3f64740 20 client.16854 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:57.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.549+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/3382189006 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d47c7d1530 0x55d47c7d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.549+0000 7f9fd3f64740 4 client.16854 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:57:57.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.549+0000 7f9fd3f64740 1 -- 172.21.15.152:0/3382189006 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d47c725850 con 0x55d47c7d1530 2024-09-11T22:57:57.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.549+0000 7f9fcb7fe640 1 --2- 172.21.15.152:0/3382189006 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d47c7d1530 0x55d47c7d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.549+0000 7f9fcb7fe640 1 --2- 172.21.15.152:0/3382189006 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d47c7d1530 0x55d47c7d39b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd2f19b909e11ef server_cookie=a29132a94ae7d3cb in_seq=0 out_seq=0 2024-09-11T22:57:57.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.549+0000 7f9faaffd640 10 client.16854 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:57.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.550+0000 7f9faaffd640 1 -- 172.21.15.152:0/3382189006 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d47c725850 con 0x55d47c7d1530 2024-09-11T22:57:57.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.550+0000 7f9faaffd640 10 client.16854 handle_command_reply: tid=0 2024-09-11T22:57:57.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.550+0000 7f9fd3f64740 1 client.16854 shutdown 2024-09-11T22:57:57.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.550+0000 7f9fd3f64740 1 -- 172.21.15.152:0/3382189006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7bd680 msgr2=0x55d47c7ba8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:57.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.550+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/3382189006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7bd680 0x55d47c7ba8e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400f750 tx=0x7f9fc4004ab0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.550+0000 7f9fd3f64740 1 -- 172.21.15.152:0/3382189006 shutdown_connections 2024-09-11T22:57:57.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.550+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/3382189006 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7bd680 0x55d47c7ba8e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.550+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/3382189006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b2dc0 0x55d47c7ba2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.550+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/3382189006 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d47c7d1530 0x55d47c7d39b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.550+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/3382189006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7b24a0 0x55d47c7bd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.550+0000 7f9fd3f64740 1 -- 172.21.15.152:0/3382189006 >> 172.21.15.152:0/3382189006 conn(0x55d47c7aea30 msgr2=0x55d47c7b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:57.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.550+0000 7f9faaffd640 0 client.16854 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:57.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.551+0000 7f9fd3f64740 1 -- 172.21.15.152:0/3382189006 shutdown_connections 2024-09-11T22:57:57.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.551+0000 7f9fd3f64740 1 -- 172.21.15.152:0/3382189006 wait complete. 2024-09-11T22:57:57.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.551+0000 7f9fd3f64740 20 client.16854 trim_cache size 0 max 16384 2024-09-11T22:57:57.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.567+0000 7f9fd3f64740 1 Processor -- start 2024-09-11T22:57:57.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.567+0000 7f9fd3f64740 1 -- start start 2024-09-11T22:57:57.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.567+0000 7f9fd3f64740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b24a0 0x55d47c7cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.567+0000 7f9fd3f64740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7b2dc0 0x55d47c7d9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.567+0000 7f9fd3f64740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7bd680 0x55d47c7d97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.567+0000 7f9fd3f64740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1ab560 con 0x55d47c7bd680 2024-09-11T22:57:57.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.568+0000 7f9fd3f64740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1ad0a0 con 0x55d47c7b2dc0 2024-09-11T22:57:57.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.568+0000 7f9fd3f64740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1abc30 con 0x55d47c7b24a0 2024-09-11T22:57:57.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.568+0000 7f9fcbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7bd680 0x55d47c7d97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.568+0000 7f9fcbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7bd680 0x55d47c7d97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43428/0 (socket says 172.21.15.152:43428) 2024-09-11T22:57:57.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.568+0000 7f9fcbfff640 1 -- 172.21.15.152:0/3013048348 learned_addr learned my addr 172.21.15.152:0/3013048348 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:57.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.568+0000 7f9fc3fff640 1 --2- 172.21.15.152:0/3013048348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7b2dc0 0x55d47c7d9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.568+0000 7f9fcb7fe640 1 --2- 172.21.15.152:0/3013048348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b24a0 0x55d47c7cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.568+0000 7f9fcbfff640 1 -- 172.21.15.152:0/3013048348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b24a0 msgr2=0x55d47c7cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:57.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.568+0000 7f9fcbfff640 1 --2- 172.21.15.152:0/3013048348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b24a0 0x55d47c7cd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.568+0000 7f9fcbfff640 1 -- 172.21.15.152:0/3013048348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7b2dc0 msgr2=0x55d47c7d9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:57.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.568+0000 7f9fcbfff640 1 --2- 172.21.15.152:0/3013048348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7b2dc0 0x55d47c7d9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.568+0000 7f9fcbfff640 1 -- 172.21.15.152:0/3013048348 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d47c7b8840 con 0x55d47c7bd680 2024-09-11T22:57:57.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.568+0000 7f9fcbfff640 1 --2- 172.21.15.152:0/3013048348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7bd680 0x55d47c7d97c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4013530 tx=0x7f9fc401f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce44c0c6cec36a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:57.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.568+0000 7f9fc17fa640 1 -- 172.21.15.152:0/3013048348 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4038070 con 0x55d47c7bd680 2024-09-11T22:57:57.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.568+0000 7f9fc17fa640 1 -- 172.21.15.152:0/3013048348 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fc402c070 con 0x55d47c7bd680 2024-09-11T22:57:57.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.568+0000 7f9fc17fa640 1 -- 172.21.15.152:0/3013048348 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4023040 con 0x55d47c7bd680 2024-09-11T22:57:57.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.569+0000 7f9fd3f64740 1 -- 172.21.15.152:0/3013048348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7bd680 msgr2=0x55d47c7d97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:57.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.569+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/3013048348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7bd680 0x55d47c7d97c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4013530 tx=0x7f9fc401f840 comp rx=0 tx=0).stop 2024-09-11T22:57:57.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.569+0000 7f9fd3f64740 1 -- 172.21.15.152:0/3013048348 shutdown_connections 2024-09-11T22:57:57.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.569+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/3013048348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7bd680 0x55d47c7d97c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.569+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/3013048348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7b2dc0 0x55d47c7d9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.569+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/3013048348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7b24a0 0x55d47c7cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.569+0000 7f9fd3f64740 1 -- 172.21.15.152:0/3013048348 >> 172.21.15.152:0/3013048348 conn(0x55d47c7aea30 msgr2=0x55d47c7b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:57.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.569+0000 7f9fd3f64740 1 -- 172.21.15.152:0/3013048348 shutdown_connections 2024-09-11T22:57:57.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.569+0000 7f9fd3f64740 1 -- 172.21.15.152:0/3013048348 wait complete. 2024-09-11T22:57:57.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.569+0000 7f9fd3f64740 1 -- start start 2024-09-11T22:57:57.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.569+0000 7f9fd3f64740 1 Processor -- start 2024-09-11T22:57:57.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fd3f64740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7b24a0 0x55d47c7c6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fd3f64740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7b2dc0 0x55d47c7c6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fd3f64740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7bd680 0x55d47c835220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fd3f64740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1ac9d0 con 0x55d47c7b24a0 2024-09-11T22:57:57.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fd3f64740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1ac300 con 0x55d47c7b2dc0 2024-09-11T22:57:57.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fd3f64740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc1a9520 con 0x55d47c7bd680 2024-09-11T22:57:57.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fd3f64740 10 client.0 _ll_register_callbacks cb 0x7f9fc4009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:57:57.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fcb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7b24a0 0x55d47c7c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fcb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7b24a0 0x55d47c7c6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43444/0 (socket says 172.21.15.152:43444) 2024-09-11T22:57:57.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fcb7fe640 1 -- 172.21.15.152:0/2579725057 learned_addr learned my addr 172.21.15.152:0/2579725057 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:57:57.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fcbfff640 1 --2- 172.21.15.152:0/2579725057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7bd680 0x55d47c835220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fc3fff640 1 --2- 172.21.15.152:0/2579725057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7b2dc0 0x55d47c7c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fcb7fe640 1 -- 172.21.15.152:0/2579725057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7bd680 msgr2=0x55d47c835220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:57.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fcb7fe640 1 --2- 172.21.15.152:0/2579725057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7bd680 0x55d47c835220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fcb7fe640 1 -- 172.21.15.152:0/2579725057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7b2dc0 msgr2=0x55d47c7c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:57:57.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fcb7fe640 1 --2- 172.21.15.152:0/2579725057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7b2dc0 0x55d47c7c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fcb7fe640 1 -- 172.21.15.152:0/2579725057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d47c835810 con 0x55d47c7b24a0 2024-09-11T22:57:57.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fcb7fe640 1 --2- 172.21.15.152:0/2579725057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7b24a0 0x55d47c7c6020 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009610 tx=0x7f9fbc010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cc7a8865d10502 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:57:57.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9faaffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:57:57.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9faaffd640 1 -- 172.21.15.152:0/2579725057 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc0ba060 con 0x55d47c7b24a0 2024-09-11T22:57:57.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9faaffd640 1 -- 172.21.15.152:0/2579725057 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f9fbc097d20 con 0x55d47c7b24a0 2024-09-11T22:57:57.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.570+0000 7f9fd3f64740 1 -- 172.21.15.152:0/2579725057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fc402c070 con 0x55d47c7b24a0 2024-09-11T22:57:57.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.571+0000 7f9faaffd640 1 -- 172.21.15.152:0/2579725057 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9fbc020040 con 0x55d47c7b24a0 2024-09-11T22:57:57.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.571+0000 7f9fd3f64740 10 client.16860 fetch_fsmap learned FSMap version 28 2024-09-11T22:57:57.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.571+0000 7f9fd3f64740 1 -- 172.21.15.152:0/2579725057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d47c83cc00 con 0x55d47c7b24a0 2024-09-11T22:57:57.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.571+0000 7f9faaffd640 1 -- 172.21.15.152:0/2579725057 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fc402c070 con 0x55d47c7b24a0 2024-09-11T22:57:57.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.571+0000 7f9faaffd640 1 -- 172.21.15.152:0/2579725057 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9fbc0bb040 con 0x55d47c7b24a0 2024-09-11T22:57:57.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.571+0000 7f9fd3f64740 10 client.16860 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:57:57.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.571+0000 7f9fd3f64740 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:57:57.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.571+0000 7f9fd3f64740 20 client.16860 populate_metadata read hostname 'smithi152' 2024-09-11T22:57:57.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.571+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/2579725057 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d47c83d2d0 0x55d47c83f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:57:57.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.571+0000 7f9fd3f64740 4 client.16860 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:57:57.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.571+0000 7f9fd3f64740 1 -- 172.21.15.152:0/2579725057 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d47c71fd60 con 0x55d47c83d2d0 2024-09-11T22:57:57.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.571+0000 7f9fc3fff640 1 --2- 172.21.15.152:0/2579725057 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d47c83d2d0 0x55d47c83f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:57:57.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.572+0000 7f9faaffd640 10 client.16860 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:57.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.572+0000 7f9fc3fff640 1 --2- 172.21.15.152:0/2579725057 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d47c83d2d0 0x55d47c83f6f0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=180b285ca8e19c33 server_cookie=efa15dbd57a0d937 in_seq=0 out_seq=0 2024-09-11T22:57:57.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.572+0000 7f9faaffd640 1 -- 172.21.15.152:0/2579725057 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d47c71fd60 con 0x55d47c83d2d0 2024-09-11T22:57:57.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.572+0000 7f9faaffd640 10 client.16860 handle_command_reply: tid=0 2024-09-11T22:57:57.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.572+0000 7f9fd3f64740 1 client.16860 shutdown 2024-09-11T22:57:57.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.572+0000 7f9fd3f64740 1 -- 172.21.15.152:0/2579725057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7b24a0 msgr2=0x55d47c7c6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:57.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.572+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/2579725057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7b24a0 0x55d47c7c6020 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc009610 tx=0x7f9fbc010d70 comp rx=0 tx=0).stop 2024-09-11T22:57:57.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.572+0000 7f9fd3f64740 1 -- 172.21.15.152:0/2579725057 shutdown_connections 2024-09-11T22:57:57.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.572+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/2579725057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d47c7bd680 0x55d47c835220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.572+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/2579725057 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d47c83d2d0 0x55d47c83f6f0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.572+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/2579725057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d47c7b2dc0 0x55d47c7c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.572+0000 7f9fd3f64740 1 --2- 172.21.15.152:0/2579725057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d47c7b24a0 0x55d47c7c6020 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.572+0000 7f9faaffd640 0 client.16860 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:57:57.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.572+0000 7f9fd3f64740 1 -- 172.21.15.152:0/2579725057 >> 172.21.15.152:0/2579725057 conn(0x55d47c7aea30 msgr2=0x55d47c7ad350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:57.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.572+0000 7f9fd3f64740 1 -- 172.21.15.152:0/2579725057 shutdown_connections 2024-09-11T22:57:57.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.572+0000 7f9fd3f64740 1 -- 172.21.15.152:0/2579725057 wait complete. 2024-09-11T22:57:57.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.573+0000 7f9fd3f64740 20 client.16860 trim_cache size 0 max 16384 2024-09-11T22:57:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:57:57.580 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:57:57.580 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": { 2024-09-11T22:57:57.580 INFO:teuthology.orchestra.run.smithi152.stdout: "acdb25fd-cdd9-41a6-b7a6-9f440f1f69ae": { 2024-09-11T22:57:57.580 INFO:teuthology.orchestra.run.smithi152.stdout: "path": "/", 2024-09-11T22:57:57.580 INFO:teuthology.orchestra.run.smithi152.stdout: "tag": "acdb25fd-cdd9-41a6-b7a6-9f440f1f69ae", 2024-09-11T22:57:57.580 INFO:teuthology.orchestra.run.smithi152.stdout: "options": "recursive,force" 2024-09-11T22:57:57.580 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:57:57.580 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:57:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:57:57.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.573+0000 7f9fd1b8b640 1 -- 172.21.15.152:0/2644747094 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9fb80a2cf0 msgr2=0x7f9fb80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:57.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.573+0000 7f9fd1b8b640 1 --2- 172.21.15.152:0/2644747094 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9fb80a2cf0 0x7f9fb80a51b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4005fd0 tx=0x7f9fc40139f0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.573+0000 7f9fd1b8b640 1 -- 172.21.15.152:0/2644747094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fcc101de0 msgr2=0x7f9fcc198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:57:57.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.573+0000 7f9fd1b8b640 1 --2- 172.21.15.152:0/2644747094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fcc101de0 0x7f9fcc198490 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00d950 tx=0x7f9fbc00de10 comp rx=0 tx=0).stop 2024-09-11T22:57:57.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.573+0000 7f9fd1b8b640 1 -- 172.21.15.152:0/2644747094 shutdown_connections 2024-09-11T22:57:57.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.573+0000 7f9fd1b8b640 1 --2- 172.21.15.152:0/2644747094 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f9fb80a2cf0 0x7f9fb80a51b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.573+0000 7f9fd1b8b640 1 --2- 172.21.15.152:0/2644747094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fcc106f10 0x7f9fcc19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.573+0000 7f9fd1b8b640 1 --2- 172.21.15.152:0/2644747094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9fcc102790 0x7f9fcc1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.573+0000 7f9fd1b8b640 1 --2- 172.21.15.152:0/2644747094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9fcc101de0 0x7f9fcc198490 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:57:57.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.573+0000 7f9fd1b8b640 1 -- 172.21.15.152:0/2644747094 >> 172.21.15.152:0/2644747094 conn(0x7f9fcc0fd8c0 msgr2=0x7f9fcc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:57:57.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.573+0000 7f9fd1b8b640 1 -- 172.21.15.152:0/2644747094 shutdown_connections 2024-09-11T22:57:57.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:57:57.573+0000 7f9fd1b8b640 1 -- 172.21.15.152:0/2644747094 wait complete. 2024-09-11T22:57:58.495 INFO:tasks.cephfs.filesystem:scrub status for tag:acdb25fd-cdd9-41a6-b7a6-9f440f1f69ae - {'path': '/', 'tag': 'acdb25fd-cdd9-41a6-b7a6-9f440f1f69ae', 'options': 'recursive,force'} 2024-09-11T22:57:59.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:59 smithi191 ceph-mon[35211]: pgmap v155: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:59.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:57:59 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T22:57:59.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:59 smithi204 ceph-mon[35454]: pgmap v155: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:59.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:57:59 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T22:57:59.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:59 smithi152 ceph-mon[31181]: pgmap v155: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:57:59.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:57:59 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T22:58:01.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:01 smithi191 ceph-mon[35211]: pgmap v156: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-09-11T22:58:01.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:01 smithi204 ceph-mon[35454]: pgmap v156: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-09-11T22:58:01.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:01 smithi152 ceph-mon[31181]: pgmap v156: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-09-11T22:58:03.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:03 smithi191 ceph-mon[35211]: pgmap v157: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-11T22:58:03.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:03 smithi204 ceph-mon[35454]: pgmap v157: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-11T22:58:03.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:03 smithi152 ceph-mon[31181]: pgmap v157: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-11T22:58:05.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:05 smithi191 ceph-mon[35211]: pgmap v158: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:58:05.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:05 smithi204 ceph-mon[35454]: pgmap v158: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:58:05.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:05 smithi152 ceph-mon[31181]: pgmap v158: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:58:06.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:06 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:58:06.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:06 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:58:06.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:06 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:58:06.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:06 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:58:06.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:06 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:58:06.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:06 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:58:06.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:06 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:58:06.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:06 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:58:06.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:06 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:58:06.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:06 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:58:06.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:06 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:58:06.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:06 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:58:07.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:07 smithi191 ceph-mon[35211]: pgmap v159: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-11T22:58:07.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:07 smithi204 ceph-mon[35454]: pgmap v159: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-11T22:58:07.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:07 smithi152 ceph-mon[31181]: pgmap v159: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-11T22:58:09.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:09 smithi191 ceph-mon[35211]: pgmap v160: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-11T22:58:09.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:09 smithi204 ceph-mon[35454]: pgmap v160: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-11T22:58:09.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:09 smithi152 ceph-mon[31181]: pgmap v160: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-11T22:58:11.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:11 smithi191 ceph-mon[35211]: pgmap v161: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-11T22:58:11.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:11 smithi204 ceph-mon[35454]: pgmap v161: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-11T22:58:11.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:11 smithi152 ceph-mon[31181]: pgmap v161: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-11T22:58:13.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:13 smithi191 ceph-mon[35211]: pgmap v162: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-11T22:58:13.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:13 smithi204 ceph-mon[35454]: pgmap v162: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-11T22:58:13.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:13 smithi152 ceph-mon[31181]: pgmap v162: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-11T22:58:15.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:15 smithi191 ceph-mon[35211]: pgmap v163: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-11T22:58:15.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:15 smithi204 ceph-mon[35454]: pgmap v163: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-11T22:58:15.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:15 smithi152 ceph-mon[31181]: pgmap v163: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-11T22:58:17.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:17 smithi204 ceph-mon[35454]: pgmap v164: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-11T22:58:17.747 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:17 smithi191 ceph-mon[35211]: pgmap v164: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-11T22:58:17.747 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:17 smithi152 ceph-mon[31181]: pgmap v164: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-11T22:58:19.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:19 smithi152 ceph-mon[31181]: pgmap v165: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T22:58:19.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:19 smithi191 ceph-mon[35211]: pgmap v165: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T22:58:19.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:19 smithi204 ceph-mon[35454]: pgmap v165: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T22:58:21.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:21 smithi152 ceph-mon[31181]: pgmap v166: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-11T22:58:21.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:21 smithi191 ceph-mon[35211]: pgmap v166: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-11T22:58:21.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:21 smithi204 ceph-mon[35454]: pgmap v166: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-11T22:58:23.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:23 smithi152 ceph-mon[31181]: pgmap v167: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:58:23.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:23 smithi191 ceph-mon[35211]: pgmap v167: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:58:23.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:23 smithi204 ceph-mon[35454]: pgmap v167: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:58:25.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:25 smithi152 ceph-mon[31181]: pgmap v168: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:25.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:25 smithi191 ceph-mon[35211]: pgmap v168: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:25.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:25 smithi204 ceph-mon[35454]: pgmap v168: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:27.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:27 smithi152 ceph-mon[31181]: pgmap v169: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:58:27.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:27 smithi191 ceph-mon[35211]: pgmap v169: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:58:27.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:27 smithi204 ceph-mon[35454]: pgmap v169: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:58:28.497 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:58:28.763 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:29.205 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:29 smithi152 ceph-mon[31181]: pgmap v170: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:29.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.288+0000 7f3aace06640 1 -- 172.21.15.152:0/4003113757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8101de0 msgr2=0x7f3aa81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:29.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.288+0000 7f3aace06640 1 --2- 172.21.15.152:0/4003113757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8101de0 0x7f3aa81021c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009ca0 tx=0x7f3a9802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.288+0000 7f3aace06640 1 -- 172.21.15.152:0/4003113757 shutdown_connections 2024-09-11T22:58:29.330 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.288+0000 7f3aace06640 1 --2- 172.21.15.152:0/4003113757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3aa8106f10 0x7f3aa810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.288+0000 7f3aace06640 1 --2- 172.21.15.152:0/4003113757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3aa8102790 0x7f3aa81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.288+0000 7f3aace06640 1 --2- 172.21.15.152:0/4003113757 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8101de0 0x7f3aa81021c0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.288+0000 7f3aace06640 1 -- 172.21.15.152:0/4003113757 >> 172.21.15.152:0/4003113757 conn(0x7f3aa80fd8e0 msgr2=0x7f3aa80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:29.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.288+0000 7f3aace06640 1 -- 172.21.15.152:0/4003113757 shutdown_connections 2024-09-11T22:58:29.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.288+0000 7f3aace06640 1 -- 172.21.15.152:0/4003113757 wait complete. 2024-09-11T22:58:29.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.289+0000 7f3aace06640 1 Processor -- start 2024-09-11T22:58:29.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.289+0000 7f3aace06640 1 -- start start 2024-09-11T22:58:29.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.289+0000 7f3aace06640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3aa8101de0 0x7f3aa8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.289+0000 7f3aace06640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8102790 0x7f3aa8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.289+0000 7f3aace06640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3aa8106f10 0x7f3aa819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.289+0000 7f3aace06640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa8111f70 con 0x7f3aa8102790 2024-09-11T22:58:29.331 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.289+0000 7f3aace06640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa8111df0 con 0x7f3aa8101de0 2024-09-11T22:58:29.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.289+0000 7f3aace06640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81120f0 con 0x7f3aa8106f10 2024-09-11T22:58:29.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.289+0000 7f3aa5d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8102790 0x7f3aa8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.289+0000 7f3aa5d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8102790 0x7f3aa8198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41608/0 (socket says 172.21.15.152:41608) 2024-09-11T22:58:29.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.289+0000 7f3aa5d9b640 1 -- 172.21.15.152:0/499227106 learned_addr learned my addr 172.21.15.152:0/499227106 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:29.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.289+0000 7f3aa659c640 1 --2- 172.21.15.152:0/499227106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3aa8101de0 0x7f3aa8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.289+0000 7f3aa6d9d640 1 --2- 172.21.15.152:0/499227106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3aa8106f10 0x7f3aa819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.290+0000 7f3aa5d9b640 1 -- 172.21.15.152:0/499227106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3aa8106f10 msgr2=0x7f3aa819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:29.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.290+0000 7f3aa5d9b640 1 --2- 172.21.15.152:0/499227106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3aa8106f10 0x7f3aa819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.290+0000 7f3aa5d9b640 1 -- 172.21.15.152:0/499227106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3aa8101de0 msgr2=0x7f3aa8198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:29.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.290+0000 7f3aa5d9b640 1 --2- 172.21.15.152:0/499227106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3aa8101de0 0x7f3aa8198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.290+0000 7f3aa5d9b640 1 -- 172.21.15.152:0/499227106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa819d4b0 con 0x7f3aa8102790 2024-09-11T22:58:29.332 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.290+0000 7f3aa5d9b640 1 --2- 172.21.15.152:0/499227106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8102790 0x7f3aa8198a40 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00d950 tx=0x7f3a9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:29.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.290+0000 7f3a977fe640 1 -- 172.21.15.152:0/499227106 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c017070 con 0x7f3aa8102790 2024-09-11T22:58:29.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.290+0000 7f3a977fe640 1 -- 172.21.15.152:0/499227106 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a9c009c30 con 0x7f3aa8102790 2024-09-11T22:58:29.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.290+0000 7f3aace06640 1 -- 172.21.15.152:0/499227106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa819d7a0 con 0x7f3aa8102790 2024-09-11T22:58:29.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.290+0000 7f3a977fe640 1 -- 172.21.15.152:0/499227106 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c00f6b0 con 0x7f3aa8102790 2024-09-11T22:58:29.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.290+0000 7f3aace06640 1 -- 172.21.15.152:0/499227106 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa81a5080 con 0x7f3aa8102790 2024-09-11T22:58:29.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.292+0000 7f3a977fe640 1 -- 172.21.15.152:0/499227106 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3a9c0109d0 con 0x7f3aa8102790 2024-09-11T22:58:29.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.293+0000 7f3aaf1df740 1 Processor -- start 2024-09-11T22:58:29.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.293+0000 7f3aaf1df740 1 -- start start 2024-09-11T22:58:29.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.293+0000 7f3aaf1df740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bf4a0 0x56053b0bf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.293+0000 7f3aaf1df740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bfdc0 0x56053b0ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.293+0000 7f3aaf1df740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0ca680 0x56053b0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.293+0000 7f3aaf1df740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56053b0252e0 con 0x56053b0bf4a0 2024-09-11T22:58:29.333 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.293+0000 7f3aaf1df740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56053afccce0 con 0x56053b0ca680 2024-09-11T22:58:29.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.293+0000 7f3aaf1df740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56053b0212a0 con 0x56053b0bfdc0 2024-09-11T22:58:29.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.293+0000 7f3aa6d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bf4a0 0x56053b0bf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.293+0000 7f3aa6d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bf4a0 0x56053b0bf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41616/0 (socket says 172.21.15.152:41616) 2024-09-11T22:58:29.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.293+0000 7f3aa6d9d640 1 -- 172.21.15.152:0/573620277 learned_addr learned my addr 172.21.15.152:0/573620277 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:29.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.293+0000 7f3a977fe640 1 --2- 172.21.15.152:0/499227106 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3a900a2cf0 0x7f3a900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.293+0000 7f3aa5d9b640 1 --2- 172.21.15.152:0/573620277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0ca680 0x56053b0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.293+0000 7f3aa659c640 1 --2- 172.21.15.152:0/573620277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bfdc0 0x56053b0ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.293+0000 7f3aa6d9d640 1 --2- 172.21.15.152:0/499227106 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3a900a2cf0 0x7f3a900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.294+0000 7f3aa6d9d640 1 -- 172.21.15.152:0/573620277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bfdc0 msgr2=0x56053b0ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:29.334 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.294+0000 7f3aa6d9d640 1 --2- 172.21.15.152:0/573620277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bfdc0 0x56053b0ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.294+0000 7f3aa6d9d640 1 -- 172.21.15.152:0/573620277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0ca680 msgr2=0x56053b0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:29.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.294+0000 7f3aa6d9d640 1 --2- 172.21.15.152:0/573620277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0ca680 0x56053b0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.294+0000 7f3a977fe640 1 -- 172.21.15.152:0/499227106 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f3a9c0cf100 con 0x7f3aa8102790 2024-09-11T22:58:29.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.294+0000 7f3aa6d9d640 1 -- 172.21.15.152:0/573620277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56053afb1440 con 0x56053b0bf4a0 2024-09-11T22:58:29.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.294+0000 7f3aa6d9d640 1 --2- 172.21.15.152:0/573620277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bf4a0 0x56053b0bf880 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00137c0 tx=0x7f3aa0013c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d155496127374040 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:29.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.294+0000 7f3aa6d9d640 1 --2- 172.21.15.152:0/499227106 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3a900a2cf0 0x7f3a900a51b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3aa001b000 tx=0x7f3aa001d000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:29.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.294+0000 7f3a94ff9640 1 -- 172.21.15.152:0/573620277 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0022070 con 0x56053b0bf4a0 2024-09-11T22:58:29.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.294+0000 7f3a94ff9640 1 -- 172.21.15.152:0/573620277 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa001cc00 con 0x56053b0bf4a0 2024-09-11T22:58:29.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.294+0000 7f3a94ff9640 1 -- 172.21.15.152:0/573620277 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa001db20 con 0x56053b0bf4a0 2024-09-11T22:58:29.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.295+0000 7f3aaf1df740 1 -- 172.21.15.152:0/573620277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bf4a0 msgr2=0x56053b0bf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:29.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.295+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/573620277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bf4a0 0x56053b0bf880 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00137c0 tx=0x7f3aa0013c80 comp rx=0 tx=0).stop 2024-09-11T22:58:29.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.295+0000 7f3aaf1df740 1 -- 172.21.15.152:0/573620277 shutdown_connections 2024-09-11T22:58:29.335 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.295+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/573620277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0ca680 0x56053b0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.295+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/573620277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bfdc0 0x56053b0ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.295+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/573620277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bf4a0 0x56053b0bf880 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.295+0000 7f3aaf1df740 1 -- 172.21.15.152:0/573620277 >> 172.21.15.152:0/573620277 conn(0x56053b0bba30 msgr2=0x56053b0bbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:29.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.295+0000 7f3aaf1df740 1 -- 172.21.15.152:0/573620277 shutdown_connections 2024-09-11T22:58:29.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.295+0000 7f3aaf1df740 1 -- 172.21.15.152:0/573620277 wait complete. 2024-09-11T22:58:29.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.295+0000 7f3aaf1df740 1 -- start start 2024-09-11T22:58:29.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.295+0000 7f3aaf1df740 1 Processor -- start 2024-09-11T22:58:29.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aaf1df740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bf4a0 0x56053b0c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aaf1df740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aaf1df740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0ca680 0x56053b0c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aaf1df740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56053b0cd250 con 0x56053b0bfdc0 2024-09-11T22:58:29.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aaf1df740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56053afe0050 con 0x56053b0ca680 2024-09-11T22:58:29.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aaf1df740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56053b0cd3d0 con 0x56053b0bf4a0 2024-09-11T22:58:29.336 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aaf1df740 10 client.0 _ll_register_callbacks cb 0x56053b020160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:29.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aa5d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aa5d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0c7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41628/0 (socket says 172.21.15.152:41628) 2024-09-11T22:58:29.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aa659c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bf4a0 0x56053b0c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aa5d9b640 1 -- 172.21.15.152:0/3048871249 learned_addr learned my addr 172.21.15.152:0/3048871249 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:29.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aa6d9d640 1 --2- 172.21.15.152:0/3048871249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0ca680 0x56053b0c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aa5d9b640 1 -- 172.21.15.152:0/3048871249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bf4a0 msgr2=0x56053b0c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:29.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aa5d9b640 1 --2- 172.21.15.152:0/3048871249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bf4a0 0x56053b0c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aa5d9b640 1 -- 172.21.15.152:0/3048871249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0ca680 msgr2=0x56053b0c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:29.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aa5d9b640 1 --2- 172.21.15.152:0/3048871249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0ca680 0x56053b0c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aa5d9b640 1 -- 172.21.15.152:0/3048871249 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56053ad963c0 con 0x56053b0bfdc0 2024-09-11T22:58:29.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aa659c640 1 --2- 172.21.15.152:0/3048871249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bf4a0 0x56053b0c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:58:29.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3aa5d9b640 1 --2- 172.21.15.152:0/3048871249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0c7270 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c010fd0 tx=0x7f3a9c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92aad2cbcf795ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:29.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.296+0000 7f3a8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:29.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.297+0000 7f3a8dffb640 1 -- 172.21.15.152:0/3048871249 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c020040 con 0x56053b0bfdc0 2024-09-11T22:58:29.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.297+0000 7f3a8dffb640 1 -- 172.21.15.152:0/3048871249 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f3a9c09f170 con 0x56053b0bfdc0 2024-09-11T22:58:29.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.297+0000 7f3aaf1df740 1 -- 172.21.15.152:0/3048871249 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aa001cc00 con 0x56053b0bfdc0 2024-09-11T22:58:29.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.297+0000 7f3a8dffb640 1 -- 172.21.15.152:0/3048871249 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3a9c0ccd20 con 0x56053b0bfdc0 2024-09-11T22:58:29.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.297+0000 7f3aaf1df740 10 client.16872 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:29.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.297+0000 7f3aaf1df740 1 -- 172.21.15.152:0/3048871249 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56053b0d8c60 con 0x56053b0bfdc0 2024-09-11T22:58:29.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.297+0000 7f3a8dffb640 1 -- 172.21.15.152:0/3048871249 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa001cc00 con 0x56053b0bfdc0 2024-09-11T22:58:29.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.297+0000 7f3a8dffb640 1 -- 172.21.15.152:0/3048871249 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3a9c0a3e50 con 0x56053b0bfdc0 2024-09-11T22:58:29.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.297+0000 7f3aaf1df740 10 client.16872 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:29.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.297+0000 7f3aaf1df740 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:58:29.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.297+0000 7f3aaf1df740 20 client.16872 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:29.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.297+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/3048871249 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56053b0da120 0x56053b0dc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.297+0000 7f3aaf1df740 4 client.16872 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:58:29.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.297+0000 7f3aaf1df740 1 -- 172.21.15.152:0/3048871249 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56053b032850 con 0x56053b0da120 2024-09-11T22:58:29.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.297+0000 7f3aa659c640 1 --2- 172.21.15.152:0/3048871249 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56053b0da120 0x56053b0dc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.298+0000 7f3aa659c640 1 --2- 172.21.15.152:0/3048871249 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56053b0da120 0x56053b0dc510 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42fdc1fcb79f93cd server_cookie=663d60e67655a73f in_seq=0 out_seq=0 2024-09-11T22:58:29.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.298+0000 7f3a8dffb640 10 client.16872 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:29.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.298+0000 7f3a8dffb640 1 -- 172.21.15.152:0/3048871249 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56053b032850 con 0x56053b0da120 2024-09-11T22:58:29.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.298+0000 7f3a8dffb640 10 client.16872 handle_command_reply: tid=0 2024-09-11T22:58:29.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.298+0000 7f3aaf1df740 1 client.16872 shutdown 2024-09-11T22:58:29.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.299+0000 7f3aaf1df740 1 -- 172.21.15.152:0/3048871249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 msgr2=0x56053b0c7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:29.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.299+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/3048871249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0c7270 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c010fd0 tx=0x7f3a9c005bd0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.299+0000 7f3aaf1df740 1 -- 172.21.15.152:0/3048871249 shutdown_connections 2024-09-11T22:58:29.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.299+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/3048871249 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56053b0da120 0x56053b0dc510 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.299+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/3048871249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0ca680 0x56053b0c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.299+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/3048871249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0c7270 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.299+0000 7f3a8dffb640 0 client.16872 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:29.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.299+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/3048871249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bf4a0 0x56053b0c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.299+0000 7f3aaf1df740 1 -- 172.21.15.152:0/3048871249 >> 172.21.15.152:0/3048871249 conn(0x56053b0bba30 msgr2=0x56053b0c1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:29.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.299+0000 7f3aaf1df740 1 -- 172.21.15.152:0/3048871249 shutdown_connections 2024-09-11T22:58:29.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.299+0000 7f3aaf1df740 1 -- 172.21.15.152:0/3048871249 wait complete. 2024-09-11T22:58:29.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.299+0000 7f3aaf1df740 20 client.16872 trim_cache size 0 max 16384 2024-09-11T22:58:29.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aaf1df740 1 Processor -- start 2024-09-11T22:58:29.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aaf1df740 1 -- start start 2024-09-11T22:58:29.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aaf1df740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0bba30 0x56053b0d64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aaf1df740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bf4a0 0x56053b0c7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aaf1df740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aaf1df740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81ad840 con 0x56053b0bfdc0 2024-09-11T22:58:29.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aaf1df740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81af380 con 0x56053b0bba30 2024-09-11T22:58:29.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aaf1df740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81adf10 con 0x56053b0bf4a0 2024-09-11T22:58:29.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aa6d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aa6d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0e1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41630/0 (socket says 172.21.15.152:41630) 2024-09-11T22:58:29.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aa6d9d640 1 -- 172.21.15.152:0/306190637 learned_addr learned my addr 172.21.15.152:0/306190637 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:29.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aa659c640 1 --2- 172.21.15.152:0/306190637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0bba30 0x56053b0d64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aa6d9d640 1 -- 172.21.15.152:0/306190637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bf4a0 msgr2=0x56053b0c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:29.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aa6d9d640 1 --2- 172.21.15.152:0/306190637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bf4a0 0x56053b0c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aa6d9d640 1 -- 172.21.15.152:0/306190637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0bba30 msgr2=0x56053b0d64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:29.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aa6d9d640 1 --2- 172.21.15.152:0/306190637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0bba30 0x56053b0d64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aa6d9d640 1 -- 172.21.15.152:0/306190637 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56053b0c57c0 con 0x56053b0bfdc0 2024-09-11T22:58:29.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.316+0000 7f3aa659c640 1 --2- 172.21.15.152:0/306190637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0bba30 0x56053b0d64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:29.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.317+0000 7f3aa6d9d640 1 --2- 172.21.15.152:0/306190637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0e1d20 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0013790 tx=0x7f3aa000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5345c2db44e201f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:29.342 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:58:29.342 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T22:58:29.342 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T22:58:29.342 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:58:29.342 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.317+0000 7f3a8ffff640 1 -- 172.21.15.152:0/306190637 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0022030 con 0x56053b0bfdc0 2024-09-11T22:58:29.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.317+0000 7f3a8ffff640 1 -- 172.21.15.152:0/306190637 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa0005050 con 0x56053b0bfdc0 2024-09-11T22:58:29.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.317+0000 7f3a8ffff640 1 -- 172.21.15.152:0/306190637 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa001ca10 con 0x56053b0bfdc0 2024-09-11T22:58:29.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.317+0000 7f3aaf1df740 1 -- 172.21.15.152:0/306190637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 msgr2=0x56053b0e1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:29.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.317+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/306190637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0e1d20 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0013790 tx=0x7f3aa000b410 comp rx=0 tx=0).stop 2024-09-11T22:58:29.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.317+0000 7f3aaf1df740 1 -- 172.21.15.152:0/306190637 shutdown_connections 2024-09-11T22:58:29.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.317+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/306190637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0e1d20 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.317+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/306190637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bf4a0 0x56053b0c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.317+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/306190637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0bba30 0x56053b0d64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.317+0000 7f3aaf1df740 1 -- 172.21.15.152:0/306190637 >> 172.21.15.152:0/306190637 conn(0x56053b0c7910 msgr2=0x56053b0c6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:29.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.317+0000 7f3aaf1df740 1 -- 172.21.15.152:0/306190637 shutdown_connections 2024-09-11T22:58:29.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.317+0000 7f3aaf1df740 1 -- 172.21.15.152:0/306190637 wait complete. 2024-09-11T22:58:29.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aaf1df740 1 -- start start 2024-09-11T22:58:29.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aaf1df740 1 Processor -- start 2024-09-11T22:58:29.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aaf1df740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0bba30 0x56053b0d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aaf1df740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bf4a0 0x56053b0d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aaf1df740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0d09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aaf1df740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81aecb0 con 0x56053b0bfdc0 2024-09-11T22:58:29.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aaf1df740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81ae5e0 con 0x56053b0bba30 2024-09-11T22:58:29.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aaf1df740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa81ab800 con 0x56053b0bf4a0 2024-09-11T22:58:29.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aaf1df740 10 client.0 _ll_register_callbacks cb 0x7f3a9c09b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:29.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aa6d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0d09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aa6d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0d09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41644/0 (socket says 172.21.15.152:41644) 2024-09-11T22:58:29.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aa6d9d640 1 -- 172.21.15.152:0/2936542683 learned_addr learned my addr 172.21.15.152:0/2936542683 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:29.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aa659c640 1 --2- 172.21.15.152:0/2936542683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0bba30 0x56053b0d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aa5d9b640 1 --2- 172.21.15.152:0/2936542683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bf4a0 0x56053b0d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aa6d9d640 1 -- 172.21.15.152:0/2936542683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bf4a0 msgr2=0x56053b0d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:29.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aa6d9d640 1 --2- 172.21.15.152:0/2936542683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bf4a0 0x56053b0d0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aa6d9d640 1 -- 172.21.15.152:0/2936542683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0bba30 msgr2=0x56053b0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:29.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aa6d9d640 1 --2- 172.21.15.152:0/2936542683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0bba30 0x56053b0d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aa6d9d640 1 -- 172.21.15.152:0/2936542683 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56053b143d00 con 0x56053b0bfdc0 2024-09-11T22:58:29.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.318+0000 7f3aa5d9b640 1 --2- 172.21.15.152:0/2936542683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bf4a0 0x56053b0d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:29.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.319+0000 7f3aa6d9d640 1 --2- 172.21.15.152:0/2936542683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0d09b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00059f0 tx=0x7f3aa000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96068d5cb864924 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:29.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.319+0000 7f3a8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:29.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.319+0000 7f3a8dffb640 1 -- 172.21.15.152:0/2936542683 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0022030 con 0x56053b0bfdc0 2024-09-11T22:58:29.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.319+0000 7f3a8dffb640 1 -- 172.21.15.152:0/2936542683 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f3aa0013320 con 0x56053b0bfdc0 2024-09-11T22:58:29.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.319+0000 7f3aaf1df740 1 -- 172.21.15.152:0/2936542683 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aa0005050 con 0x56053b0bfdc0 2024-09-11T22:58:29.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.319+0000 7f3a8dffb640 1 -- 172.21.15.152:0/2936542683 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3aa0005200 con 0x56053b0bfdc0 2024-09-11T22:58:29.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.319+0000 7f3aaf1df740 10 client.16878 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:29.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.319+0000 7f3aaf1df740 1 -- 172.21.15.152:0/2936542683 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56053b144e70 con 0x56053b0bfdc0 2024-09-11T22:58:29.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.319+0000 7f3a8dffb640 1 -- 172.21.15.152:0/2936542683 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa0005050 con 0x56053b0bfdc0 2024-09-11T22:58:29.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.319+0000 7f3a8dffb640 1 -- 172.21.15.152:0/2936542683 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3aa0004640 con 0x56053b0bfdc0 2024-09-11T22:58:29.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.319+0000 7f3aaf1df740 10 client.16878 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:29.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.319+0000 7f3aaf1df740 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:58:29.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.319+0000 7f3aaf1df740 20 client.16878 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:29.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.319+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/2936542683 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56053b0ca680 0x56053b1472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:29.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.319+0000 7f3aaf1df740 4 client.16878 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:58:29.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.319+0000 7f3aaf1df740 1 -- 172.21.15.152:0/2936542683 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56053b02cd60 con 0x56053b0ca680 2024-09-11T22:58:29.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3aa659c640 1 --2- 172.21.15.152:0/2936542683 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56053b0ca680 0x56053b1472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:29.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3a8dffb640 10 client.16878 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:29.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3aa659c640 1 --2- 172.21.15.152:0/2936542683 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56053b0ca680 0x56053b1472a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6fc6174cad677d5 server_cookie=1af36fffdcaebd4c in_seq=0 out_seq=0 2024-09-11T22:58:29.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3a8dffb640 1 -- 172.21.15.152:0/2936542683 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56053b02cd60 con 0x56053b0ca680 2024-09-11T22:58:29.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3a8dffb640 10 client.16878 handle_command_reply: tid=0 2024-09-11T22:58:29.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3aaf1df740 1 client.16878 shutdown 2024-09-11T22:58:29.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3aaf1df740 1 -- 172.21.15.152:0/2936542683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 msgr2=0x56053b0d09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:29.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/2936542683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0d09b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00059f0 tx=0x7f3aa000b4f0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3aaf1df740 1 -- 172.21.15.152:0/2936542683 shutdown_connections 2024-09-11T22:58:29.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/2936542683 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56053b0bfdc0 0x56053b0d09b0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/2936542683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56053b0bf4a0 0x56053b0d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/2936542683 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56053b0ca680 0x56053b1472a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3aaf1df740 1 --2- 172.21.15.152:0/2936542683 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56053b0bba30 0x56053b0d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3a8dffb640 0 client.16878 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:29.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3aaf1df740 1 -- 172.21.15.152:0/2936542683 >> 172.21.15.152:0/2936542683 conn(0x56053b0c7910 msgr2=0x56053b0c8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:29.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3aaf1df740 1 -- 172.21.15.152:0/2936542683 shutdown_connections 2024-09-11T22:58:29.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.320+0000 7f3aaf1df740 1 -- 172.21.15.152:0/2936542683 wait complete. 2024-09-11T22:58:29.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.321+0000 7f3aaf1df740 20 client.16878 trim_cache size 0 max 16384 2024-09-11T22:58:29.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.321+0000 7f3aace06640 1 -- 172.21.15.152:0/499227106 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3a900a2cf0 msgr2=0x7f3a900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:29.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.321+0000 7f3aace06640 1 --2- 172.21.15.152:0/499227106 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3a900a2cf0 0x7f3a900a51b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3aa001b000 tx=0x7f3aa001d000 comp rx=0 tx=0).stop 2024-09-11T22:58:29.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.321+0000 7f3aace06640 1 -- 172.21.15.152:0/499227106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8102790 msgr2=0x7f3aa8198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:29.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.321+0000 7f3aace06640 1 --2- 172.21.15.152:0/499227106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8102790 0x7f3aa8198a40 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00d950 tx=0x7f3a9c00de10 comp rx=0 tx=0).stop 2024-09-11T22:58:29.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.321+0000 7f3aace06640 1 -- 172.21.15.152:0/499227106 shutdown_connections 2024-09-11T22:58:29.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.321+0000 7f3aace06640 1 --2- 172.21.15.152:0/499227106 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3a900a2cf0 0x7f3a900a51b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.321+0000 7f3aace06640 1 --2- 172.21.15.152:0/499227106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3aa8106f10 0x7f3aa819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.321+0000 7f3aace06640 1 --2- 172.21.15.152:0/499227106 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3aa8102790 0x7f3aa8198a40 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.321+0000 7f3aace06640 1 --2- 172.21.15.152:0/499227106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3aa8101de0 0x7f3aa8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:29.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.321+0000 7f3aace06640 1 -- 172.21.15.152:0/499227106 >> 172.21.15.152:0/499227106 conn(0x7f3aa80fd8e0 msgr2=0x7f3aa80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:29.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.321+0000 7f3aace06640 1 -- 172.21.15.152:0/499227106 shutdown_connections 2024-09-11T22:58:29.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:29.321+0000 7f3aace06640 1 -- 172.21.15.152:0/499227106 wait complete. 2024-09-11T22:58:29.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:29 smithi191 ceph-mon[35211]: pgmap v170: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:29.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:29 smithi204 ceph-mon[35454]: pgmap v170: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:30.234 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:58:30.234 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:58:30.488 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:31.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.980+0000 7f1d1084d640 1 -- 172.21.15.152:0/3188267527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d0c10c170 msgr2=0x7f1d0c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:31.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.980+0000 7f1d1084d640 1 --2- 172.21.15.152:0/3188267527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d0c10c170 0x7f1d0c10e610 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009ca0 tx=0x7f1d0402a950 comp rx=0 tx=0).stop 2024-09-11T22:58:31.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.980+0000 7f1d1084d640 1 -- 172.21.15.152:0/3188267527 shutdown_connections 2024-09-11T22:58:31.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.980+0000 7f1d1084d640 1 --2- 172.21.15.152:0/3188267527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d0c10c170 0x7f1d0c10e610 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:31.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.980+0000 7f1d1084d640 1 --2- 172.21.15.152:0/3188267527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d0c103660 0x7f1d0c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:31.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.980+0000 7f1d1084d640 1 --2- 172.21.15.152:0/3188267527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d0c102d40 0x7f1d0c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:31.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.980+0000 7f1d1084d640 1 -- 172.21.15.152:0/3188267527 >> 172.21.15.152:0/3188267527 conn(0x7f1d0c0fc7a0 msgr2=0x7f1d0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:31.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.980+0000 7f1d1084d640 1 -- 172.21.15.152:0/3188267527 shutdown_connections 2024-09-11T22:58:31.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.980+0000 7f1d1084d640 1 -- 172.21.15.152:0/3188267527 wait complete. 2024-09-11T22:58:31.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.981+0000 7f1d1084d640 1 Processor -- start 2024-09-11T22:58:31.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.981+0000 7f1d1084d640 1 -- start start 2024-09-11T22:58:31.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.981+0000 7f1d1084d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d0c102d40 0x7f1d0c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:31.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.981+0000 7f1d1084d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d0c103660 0x7f1d0c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:31.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.981+0000 7f1d1084d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d0c10c170 0x7f1d0c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:31.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.981+0000 7f1d1084d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c111f70 con 0x7f1d0c103660 2024-09-11T22:58:31.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.981+0000 7f1d1084d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c111df0 con 0x7f1d0c10c170 2024-09-11T22:58:31.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.981+0000 7f1d1084d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0c1120f0 con 0x7f1d0c102d40 2024-09-11T22:58:31.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.981+0000 7f1d09d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d0c103660 0x7f1d0c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:31.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.982+0000 7f1d09d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d0c103660 0x7f1d0c100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41656/0 (socket says 172.21.15.152:41656) 2024-09-11T22:58:31.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.982+0000 7f1d09d9b640 1 -- 172.21.15.152:0/3685040791 learned_addr learned my addr 172.21.15.152:0/3685040791 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:31.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.982+0000 7f1d0ad9d640 1 --2- 172.21.15.152:0/3685040791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d0c10c170 0x7f1d0c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:31.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.982+0000 7f1d0a59c640 1 --2- 172.21.15.152:0/3685040791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d0c102d40 0x7f1d0c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:31.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.982+0000 7f1d09d9b640 1 -- 172.21.15.152:0/3685040791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d0c102d40 msgr2=0x7f1d0c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:31.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.982+0000 7f1d09d9b640 1 --2- 172.21.15.152:0/3685040791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d0c102d40 0x7f1d0c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:31.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.982+0000 7f1d09d9b640 1 -- 172.21.15.152:0/3685040791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d0c10c170 msgr2=0x7f1d0c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:31.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.982+0000 7f1d09d9b640 1 --2- 172.21.15.152:0/3685040791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d0c10c170 0x7f1d0c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:31.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.982+0000 7f1d09d9b640 1 -- 172.21.15.152:0/3685040791 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0c1a6890 con 0x7f1d0c103660 2024-09-11T22:58:31.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.982+0000 7f1d09d9b640 1 --2- 172.21.15.152:0/3685040791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d0c103660 0x7f1d0c100ee0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00d740 tx=0x7f1cfc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:31.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.982+0000 7f1d037fe640 1 -- 172.21.15.152:0/3685040791 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc017070 con 0x7f1d0c103660 2024-09-11T22:58:31.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.982+0000 7f1d1084d640 1 -- 172.21.15.152:0/3685040791 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0c1a6b80 con 0x7f1d0c103660 2024-09-11T22:58:31.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.982+0000 7f1d037fe640 1 -- 172.21.15.152:0/3685040791 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cfc00de50 con 0x7f1d0c103660 2024-09-11T22:58:31.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.982+0000 7f1d037fe640 1 -- 172.21.15.152:0/3685040791 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc00f620 con 0x7f1d0c103660 2024-09-11T22:58:31.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.982+0000 7f1d1084d640 1 -- 172.21.15.152:0/3685040791 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0c1a7110 con 0x7f1d0c103660 2024-09-11T22:58:31.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.983+0000 7f1d1084d640 1 -- 172.21.15.152:0/3685040791 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d0c103120 con 0x7f1d0c103660 2024-09-11T22:58:31.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.984+0000 7f1d037fe640 1 -- 172.21.15.152:0/3685040791 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1cfc009c30 con 0x7f1d0c103660 2024-09-11T22:58:31.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.985+0000 7f1d037fe640 1 --2- 172.21.15.152:0/3685040791 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1cf80a2cf0 0x7f1cf80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:31.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.985+0000 7f1d0a59c640 1 --2- 172.21.15.152:0/3685040791 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1cf80a2cf0 0x7f1cf80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:31.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.985+0000 7f1d037fe640 1 -- 172.21.15.152:0/3685040791 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f1cfc0cef50 con 0x7f1d0c103660 2024-09-11T22:58:31.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.986+0000 7f1d0a59c640 1 --2- 172.21.15.152:0/3685040791 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1cf80a2cf0 0x7f1cf80a51b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4005fd0 tx=0x7f1cf4005850 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:31.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:30.988+0000 7f1d037fe640 1 -- 172.21.15.152:0/3685040791 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1cfc09bbc0 con 0x7f1d0c103660 2024-09-11T22:58:31.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:31.192+0000 7f1d1084d640 1 -- 172.21.15.152:0/3685040791 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d0c107210 con 0x7f1d0c103660 2024-09-11T22:58:31.192 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:31 smithi152 ceph-mon[31181]: pgmap v171: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:58:31.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:31.193+0000 7f1d037fe640 1 -- 172.21.15.152:0/3685040791 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f1cfc09b4c0 con 0x7f1d0c103660 2024-09-11T22:58:31.193 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:58:31.194 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:58:31.194 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:58:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:31.195+0000 7f1d1084d640 1 -- 172.21.15.152:0/3685040791 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1cf80a2cf0 msgr2=0x7f1cf80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:31.195+0000 7f1d1084d640 1 --2- 172.21.15.152:0/3685040791 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1cf80a2cf0 0x7f1cf80a51b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4005fd0 tx=0x7f1cf4005850 comp rx=0 tx=0).stop 2024-09-11T22:58:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:31.195+0000 7f1d1084d640 1 -- 172.21.15.152:0/3685040791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d0c103660 msgr2=0x7f1d0c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:31.195+0000 7f1d1084d640 1 --2- 172.21.15.152:0/3685040791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d0c103660 0x7f1d0c100ee0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00d740 tx=0x7f1cfc00dc00 comp rx=0 tx=0).stop 2024-09-11T22:58:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:31.195+0000 7f1d1084d640 1 -- 172.21.15.152:0/3685040791 shutdown_connections 2024-09-11T22:58:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:31.195+0000 7f1d1084d640 1 --2- 172.21.15.152:0/3685040791 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1cf80a2cf0 0x7f1cf80a51b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:31.195+0000 7f1d1084d640 1 --2- 172.21.15.152:0/3685040791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d0c10c170 0x7f1d0c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:31.195+0000 7f1d1084d640 1 --2- 172.21.15.152:0/3685040791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d0c103660 0x7f1d0c100ee0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:31.195+0000 7f1d1084d640 1 --2- 172.21.15.152:0/3685040791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d0c102d40 0x7f1d0c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:31.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:31.195+0000 7f1d1084d640 1 -- 172.21.15.152:0/3685040791 >> 172.21.15.152:0/3685040791 conn(0x7f1d0c0fc7a0 msgr2=0x7f1d0c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:31.195+0000 7f1d1084d640 1 -- 172.21.15.152:0/3685040791 shutdown_connections 2024-09-11T22:58:31.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:31.196+0000 7f1d1084d640 1 -- 172.21.15.152:0/3685040791 wait complete. 2024-09-11T22:58:31.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:31 smithi191 ceph-mon[35211]: pgmap v171: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:58:31.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:31 smithi204 ceph-mon[35454]: pgmap v171: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:58:31.902 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:58:32.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:32 smithi152 ceph-mon[31181]: pgmap v172: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:58:32.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:32 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3685040791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:58:32.154 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:32.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:32 smithi191 ceph-mon[35211]: pgmap v172: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:58:32.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:32 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3685040791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:58:32.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:32 smithi204 ceph-mon[35454]: pgmap v172: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:58:32.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:32 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3685040791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:58:32.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.640+0000 7f7a1b834640 1 -- 172.21.15.152:0/3652426005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a1410c180 msgr2=0x7f7a1410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:32.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.640+0000 7f7a1b834640 1 --2- 172.21.15.152:0/3652426005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a1410c180 0x7f7a1410e600 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009ca0 tx=0x7f7a0c02a950 comp rx=0 tx=0).stop 2024-09-11T22:58:32.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.640+0000 7f7a1b834640 1 -- 172.21.15.152:0/3652426005 shutdown_connections 2024-09-11T22:58:32.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.640+0000 7f7a1b834640 1 --2- 172.21.15.152:0/3652426005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a1410c180 0x7f7a1410e600 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.640+0000 7f7a1b834640 1 --2- 172.21.15.152:0/3652426005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a141036c0 0x7f7a1410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.640+0000 7f7a1b834640 1 --2- 172.21.15.152:0/3652426005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a14102da0 0x7f7a14103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.640+0000 7f7a1b834640 1 -- 172.21.15.152:0/3652426005 >> 172.21.15.152:0/3652426005 conn(0x7f7a140fc7a0 msgr2=0x7f7a140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:32.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.640+0000 7f7a1b834640 1 -- 172.21.15.152:0/3652426005 shutdown_connections 2024-09-11T22:58:32.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.640+0000 7f7a1b834640 1 -- 172.21.15.152:0/3652426005 wait complete. 2024-09-11T22:58:32.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.640+0000 7f7a1b834640 1 Processor -- start 2024-09-11T22:58:32.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.640+0000 7f7a1b834640 1 -- start start 2024-09-11T22:58:32.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a1b834640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a14102da0 0x7f7a141983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a1b834640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a141036c0 0x7f7a14198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a1b834640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a1410c180 0x7f7a1419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a1b834640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7a14111f40 con 0x7f7a141036c0 2024-09-11T22:58:32.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a1b834640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7a14111dc0 con 0x7f7a1410c180 2024-09-11T22:58:32.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a1b834640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141120c0 con 0x7f7a14102da0 2024-09-11T22:58:32.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a18dcf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a141036c0 0x7f7a14198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a18dcf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a141036c0 0x7f7a14198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41680/0 (socket says 172.21.15.152:41680) 2024-09-11T22:58:32.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a18dcf640 1 -- 172.21.15.152:0/599409400 learned_addr learned my addr 172.21.15.152:0/599409400 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:32.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a195d0640 1 --2- 172.21.15.152:0/599409400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a14102da0 0x7f7a141983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a19dd1640 1 --2- 172.21.15.152:0/599409400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a1410c180 0x7f7a1419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a18dcf640 1 -- 172.21.15.152:0/599409400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a14102da0 msgr2=0x7f7a141983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:32.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a18dcf640 1 --2- 172.21.15.152:0/599409400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a14102da0 0x7f7a141983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a18dcf640 1 -- 172.21.15.152:0/599409400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a1410c180 msgr2=0x7f7a1419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:32.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a18dcf640 1 --2- 172.21.15.152:0/599409400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a1410c180 0x7f7a1419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a18dcf640 1 -- 172.21.15.152:0/599409400 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1419d380 con 0x7f7a141036c0 2024-09-11T22:58:32.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a19dd1640 1 --2- 172.21.15.152:0/599409400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a1410c180 0x7f7a1419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:32.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a18dcf640 1 --2- 172.21.15.152:0/599409400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a141036c0 0x7f7a14198910 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400d770 tx=0x7f7a0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:32.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a0a7fc640 1 -- 172.21.15.152:0/599409400 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04017070 con 0x7f7a141036c0 2024-09-11T22:58:32.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a1b834640 1 -- 172.21.15.152:0/599409400 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1419d670 con 0x7f7a141036c0 2024-09-11T22:58:32.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a0a7fc640 1 -- 172.21.15.152:0/599409400 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a04005500 con 0x7f7a141036c0 2024-09-11T22:58:32.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.641+0000 7f7a0a7fc640 1 -- 172.21.15.152:0/599409400 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04004e00 con 0x7f7a141036c0 2024-09-11T22:58:32.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.642+0000 7f7a1b834640 1 -- 172.21.15.152:0/599409400 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a141a4f50 con 0x7f7a141036c0 2024-09-11T22:58:32.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.644+0000 7f7a0a7fc640 1 -- 172.21.15.152:0/599409400 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7a04004fa0 con 0x7f7a141036c0 2024-09-11T22:58:32.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.644+0000 7f7a1dc0d740 1 Processor -- start 2024-09-11T22:58:32.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.644+0000 7f7a1dc0d740 1 -- start start 2024-09-11T22:58:32.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.644+0000 7f7a1dc0d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd22b40 0x55ea1fd21090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.644+0000 7f7a1dc0d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd215d0 0x55ea1fd21a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.644+0000 7f7a1dc0d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd2c370 0x55ea1fd2e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.644+0000 7f7a1dc0d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ea1fc872f0 con 0x55ea1fd2c370 2024-09-11T22:58:32.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.644+0000 7f7a1dc0d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ea1fc2ecf0 con 0x55ea1fd22b40 2024-09-11T22:58:32.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.644+0000 7f7a1dc0d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea1fc832b0 con 0x55ea1fd215d0 2024-09-11T22:58:32.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.644+0000 7f7a18dcf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd2c370 0x55ea1fd2e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.644+0000 7f7a18dcf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd2c370 0x55ea1fd2e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41692/0 (socket says 172.21.15.152:41692) 2024-09-11T22:58:32.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.644+0000 7f7a18dcf640 1 -- 172.21.15.152:0/2051294407 learned_addr learned my addr 172.21.15.152:0/2051294407 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:32.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.645+0000 7f7a195d0640 1 --2- 172.21.15.152:0/2051294407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd215d0 0x55ea1fd21a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.645+0000 7f7a19dd1640 1 --2- 172.21.15.152:0/2051294407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd22b40 0x55ea1fd21090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.645+0000 7f7a0a7fc640 1 --2- 172.21.15.152:0/599409400 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7a000a2cf0 0x7f7a000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.645+0000 7f7a18dcf640 1 -- 172.21.15.152:0/2051294407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd215d0 msgr2=0x55ea1fd21a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.645+0000 7f7a18dcf640 1 --2- 172.21.15.152:0/2051294407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd215d0 0x55ea1fd21a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.645+0000 7f7a18dcf640 1 -- 172.21.15.152:0/2051294407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd22b40 msgr2=0x55ea1fd21090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.645+0000 7f7a18dcf640 1 --2- 172.21.15.152:0/2051294407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd22b40 0x55ea1fd21090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.645+0000 7f7a18dcf640 1 -- 172.21.15.152:0/2051294407 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea1fc133a0 con 0x55ea1fd2c370 2024-09-11T22:58:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.645+0000 7f7a19dd1640 1 --2- 172.21.15.152:0/599409400 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7a000a2cf0 0x7f7a000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.645+0000 7f7a18dcf640 1 --2- 172.21.15.152:0/2051294407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd2c370 0x55ea1fd2e760 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7a040cc500 tx=0x7f7a040d2f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a162df973f019961 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.645+0000 7f7a0a7fc640 1 -- 172.21.15.152:0/599409400 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7a040cee60 con 0x7f7a141036c0 2024-09-11T22:58:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.645+0000 7f79ff7fe640 1 -- 172.21.15.152:0/2051294407 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04020070 con 0x55ea1fd2c370 2024-09-11T22:58:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.645+0000 7f79ff7fe640 1 -- 172.21.15.152:0/2051294407 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a040046c0 con 0x55ea1fd2c370 2024-09-11T22:58:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.645+0000 7f79ff7fe640 1 -- 172.21.15.152:0/2051294407 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0409aed0 con 0x55ea1fd2c370 2024-09-11T22:58:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.646+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2051294407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd2c370 msgr2=0x55ea1fd2e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:32.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.646+0000 7f7a19dd1640 1 --2- 172.21.15.152:0/599409400 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7a000a2cf0 0x7f7a000a51b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c030040 tx=0x7f7a0c03d040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:32.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.646+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/2051294407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd2c370 0x55ea1fd2e760 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7a040cc500 tx=0x7f7a040d2f70 comp rx=0 tx=0).stop 2024-09-11T22:58:32.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.646+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2051294407 shutdown_connections 2024-09-11T22:58:32.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.646+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/2051294407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd2c370 0x55ea1fd2e760 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.646+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/2051294407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd215d0 0x55ea1fd21a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.646+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/2051294407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd22b40 0x55ea1fd21090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.646+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2051294407 >> 172.21.15.152:0/2051294407 conn(0x55ea1fcea310 msgr2=0x55ea1fd1dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:32.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.646+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2051294407 shutdown_connections 2024-09-11T22:58:32.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.646+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2051294407 wait complete. 2024-09-11T22:58:32.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.646+0000 7f7a1dc0d740 1 -- start start 2024-09-11T22:58:32.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a1dc0d740 1 Processor -- start 2024-09-11T22:58:32.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a1dc0d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd215d0 0x55ea1fd2bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a1dc0d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd22b40 0x55ea1fd28fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a1dc0d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd2c370 0x55ea1fd295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a1dc0d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ea1fd2f070 con 0x55ea1fd215d0 2024-09-11T22:58:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a1dc0d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ea1fc42060 con 0x55ea1fd22b40 2024-09-11T22:58:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a1dc0d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ea1fd2f1f0 con 0x55ea1fd2c370 2024-09-11T22:58:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a1dc0d740 10 client.0 _ll_register_callbacks cb 0x55ea1f9958c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a195d0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd215d0 0x55ea1fd2bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a195d0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd215d0 0x55ea1fd2bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41704/0 (socket says 172.21.15.152:41704) 2024-09-11T22:58:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a195d0640 1 -- 172.21.15.152:0/2257625207 learned_addr learned my addr 172.21.15.152:0/2257625207 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a19dd1640 1 --2- 172.21.15.152:0/2257625207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd2c370 0x55ea1fd295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a18dcf640 1 --2- 172.21.15.152:0/2257625207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd22b40 0x55ea1fd28fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a195d0640 1 -- 172.21.15.152:0/2257625207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd2c370 msgr2=0x55ea1fd295d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a195d0640 1 --2- 172.21.15.152:0/2257625207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd2c370 0x55ea1fd295d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a195d0640 1 -- 172.21.15.152:0/2257625207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd22b40 msgr2=0x55ea1fd28fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a195d0640 1 --2- 172.21.15.152:0/2257625207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd22b40 0x55ea1fd28fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a195d0640 1 -- 172.21.15.152:0/2257625207 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea1f9f83c0 con 0x55ea1fd215d0 2024-09-11T22:58:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f7a195d0640 1 --2- 172.21.15.152:0/2257625207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd215d0 0x55ea1fd2bd30 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000bad0 tx=0x7f7a1000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1c1cb3ecd22032 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.647+0000 7f79fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.648+0000 7f79fcff9640 1 -- 172.21.15.152:0/2257625207 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10005020 con 0x55ea1fd215d0 2024-09-11T22:58:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.648+0000 7f79fcff9640 1 -- 172.21.15.152:0/2257625207 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7a100051c0 con 0x55ea1fd215d0 2024-09-11T22:58:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.648+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2257625207 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ea1fc94860 con 0x55ea1fd215d0 2024-09-11T22:58:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.648+0000 7f79fcff9640 1 -- 172.21.15.152:0/2257625207 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a10005400 con 0x55ea1fd215d0 2024-09-11T22:58:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.648+0000 7f7a1dc0d740 10 client.16896 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.648+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2257625207 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ea1fd3ab20 con 0x55ea1fd215d0 2024-09-11T22:58:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.648+0000 7f79fcff9640 1 -- 172.21.15.152:0/2257625207 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55ea1fc94860 con 0x55ea1fd215d0 2024-09-11T22:58:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.648+0000 7f79fcff9640 1 -- 172.21.15.152:0/2257625207 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7a1001b670 con 0x55ea1fd215d0 2024-09-11T22:58:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.648+0000 7f7a1dc0d740 10 client.16896 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.648+0000 7f7a1dc0d740 10 client.16896 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:58:32.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.648+0000 7f7a1dc0d740 20 client.16896 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.648+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/2257625207 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55ea1fd3bf30 0x55ea1fd3e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.648+0000 7f7a1dc0d740 4 client.16896 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:58:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.648+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2257625207 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea1fc8ed70 con 0x55ea1fd3bf30 2024-09-11T22:58:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.648+0000 7f7a18dcf640 1 --2- 172.21.15.152:0/2257625207 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55ea1fd3bf30 0x55ea1fd3e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.649+0000 7f7a18dcf640 1 --2- 172.21.15.152:0/2257625207 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55ea1fd3bf30 0x55ea1fd3e340 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2cca6b6107765e48 server_cookie=77122b8c378b3e7f in_seq=0 out_seq=0 2024-09-11T22:58:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.649+0000 7f79fcff9640 10 client.16896 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:58:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.649+0000 7f79fcff9640 1 -- 172.21.15.152:0/2257625207 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea1fc8ed70 con 0x55ea1fd3bf30 2024-09-11T22:58:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.649+0000 7f79fcff9640 10 client.16896 handle_command_reply: tid=0 2024-09-11T22:58:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.649+0000 7f7a1dc0d740 1 client.16896 shutdown 2024-09-11T22:58:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.649+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2257625207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd215d0 msgr2=0x55ea1fd2bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.649+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/2257625207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd215d0 0x55ea1fd2bd30 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000bad0 tx=0x7f7a1000bed0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.649+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2257625207 shutdown_connections 2024-09-11T22:58:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.649+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/2257625207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd2c370 0x55ea1fd295d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.649+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/2257625207 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55ea1fd3bf30 0x55ea1fd3e340 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.649+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/2257625207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd22b40 0x55ea1fd28fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.649+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/2257625207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd215d0 0x55ea1fd2bd30 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.649+0000 7f79fcff9640 0 client.16896 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:58:32.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.649+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2257625207 >> 172.21.15.152:0/2257625207 conn(0x55ea1fcea310 msgr2=0x55ea1fd23990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:32.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.650+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2257625207 shutdown_connections 2024-09-11T22:58:32.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.650+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2257625207 wait complete. 2024-09-11T22:58:32.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.650+0000 7f7a1dc0d740 20 client.16896 trim_cache size 0 max 16384 2024-09-11T22:58:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.662+0000 7f7a1dc0d740 1 Processor -- start 2024-09-11T22:58:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a1dc0d740 1 -- start start 2024-09-11T22:58:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a1dc0d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd215d0 0x55ea1fd2af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a1dc0d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd22b40 0x55ea1fd24120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a1dc0d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd2c370 0x55ea1fd24660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a1dc0d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141ad6e0 con 0x55ea1fd22b40 2024-09-11T22:58:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a1dc0d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141af220 con 0x55ea1fd2c370 2024-09-11T22:58:32.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a1dc0d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141addb0 con 0x55ea1fd215d0 2024-09-11T22:58:32.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a18dcf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd22b40 0x55ea1fd24120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a18dcf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd22b40 0x55ea1fd24120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41706/0 (socket says 172.21.15.152:41706) 2024-09-11T22:58:32.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a18dcf640 1 -- 172.21.15.152:0/1329732698 learned_addr learned my addr 172.21.15.152:0/1329732698 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:32.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a195d0640 1 --2- 172.21.15.152:0/1329732698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd215d0 0x55ea1fd2af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a19dd1640 1 --2- 172.21.15.152:0/1329732698 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd2c370 0x55ea1fd24660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a18dcf640 1 -- 172.21.15.152:0/1329732698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd215d0 msgr2=0x55ea1fd2af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:32.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a18dcf640 1 --2- 172.21.15.152:0/1329732698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd215d0 0x55ea1fd2af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a18dcf640 1 -- 172.21.15.152:0/1329732698 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd2c370 msgr2=0x55ea1fd24660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:32.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a18dcf640 1 --2- 172.21.15.152:0/1329732698 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd2c370 0x55ea1fd24660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a18dcf640 1 -- 172.21.15.152:0/1329732698 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea1fd1e960 con 0x55ea1fd22b40 2024-09-11T22:58:32.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f7a18dcf640 1 --2- 172.21.15.152:0/1329732698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd22b40 0x55ea1fd24120 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7a04099640 tx=0x7f7a040bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b006146400b2df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:32.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.663+0000 7f79feffd640 1 -- 172.21.15.152:0/1329732698 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04020040 con 0x55ea1fd22b40 2024-09-11T22:58:32.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.664+0000 7f79feffd640 1 -- 172.21.15.152:0/1329732698 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a0409ba90 con 0x55ea1fd22b40 2024-09-11T22:58:32.669 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.664+0000 7f79feffd640 1 -- 172.21.15.152:0/1329732698 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a040bde00 con 0x55ea1fd22b40 2024-09-11T22:58:32.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.664+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/1329732698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd22b40 msgr2=0x55ea1fd24120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:32.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.664+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/1329732698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd22b40 0x55ea1fd24120 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7a04099640 tx=0x7f7a040bb040 comp rx=0 tx=0).stop 2024-09-11T22:58:32.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.664+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/1329732698 shutdown_connections 2024-09-11T22:58:32.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.664+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/1329732698 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd2c370 0x55ea1fd24660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.664+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/1329732698 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd22b40 0x55ea1fd24120 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.664+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/1329732698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd215d0 0x55ea1fd2af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.664+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/1329732698 >> 172.21.15.152:0/1329732698 conn(0x55ea1fcea310 msgr2=0x55ea1fd2a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:32.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.664+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/1329732698 shutdown_connections 2024-09-11T22:58:32.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.664+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/1329732698 wait complete. 2024-09-11T22:58:32.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.665+0000 7f7a1dc0d740 1 -- start start 2024-09-11T22:58:32.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.665+0000 7f7a1dc0d740 1 Processor -- start 2024-09-11T22:58:32.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.665+0000 7f7a1dc0d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd215d0 0x55ea1fd34480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.665+0000 7f7a1dc0d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd22b40 0x55ea1fd349c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.665+0000 7f7a1dc0d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd2c370 0x55ea1fd31450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.665+0000 7f7a1dc0d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141aeb50 con 0x55ea1fd215d0 2024-09-11T22:58:32.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.665+0000 7f7a1dc0d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141ae480 con 0x55ea1fd2c370 2024-09-11T22:58:32.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.665+0000 7f7a1dc0d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7a141ab6a0 con 0x55ea1fd22b40 2024-09-11T22:58:32.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.665+0000 7f7a1dc0d740 10 client.0 _ll_register_callbacks cb 0x7f7a100040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:32.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.665+0000 7f7a195d0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd215d0 0x55ea1fd34480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.665+0000 7f7a195d0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd215d0 0x55ea1fd34480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41722/0 (socket says 172.21.15.152:41722) 2024-09-11T22:58:32.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.665+0000 7f7a195d0640 1 -- 172.21.15.152:0/2040013898 learned_addr learned my addr 172.21.15.152:0/2040013898 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:32.671 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.665+0000 7f7a18dcf640 1 --2- 172.21.15.152:0/2040013898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd22b40 0x55ea1fd349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.665+0000 7f7a19dd1640 1 --2- 172.21.15.152:0/2040013898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd2c370 0x55ea1fd31450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.666+0000 7f7a195d0640 1 -- 172.21.15.152:0/2040013898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd22b40 msgr2=0x55ea1fd349c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:32.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.666+0000 7f7a195d0640 1 --2- 172.21.15.152:0/2040013898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd22b40 0x55ea1fd349c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.666+0000 7f7a195d0640 1 -- 172.21.15.152:0/2040013898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd2c370 msgr2=0x55ea1fd31450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:32.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.666+0000 7f7a195d0640 1 --2- 172.21.15.152:0/2040013898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd2c370 0x55ea1fd31450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.666+0000 7f7a195d0640 1 -- 172.21.15.152:0/2040013898 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea1fd31990 con 0x55ea1fd215d0 2024-09-11T22:58:32.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.666+0000 7f7a195d0640 1 --2- 172.21.15.152:0/2040013898 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd215d0 0x55ea1fd34480 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7a1001ba90 tx=0x7f7a10003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c46e3dd733af22 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:32.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.666+0000 7f79fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:32.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.666+0000 7f79fcff9640 1 -- 172.21.15.152:0/2040013898 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1002a040 con 0x55ea1fd215d0 2024-09-11T22:58:32.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.666+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2040013898 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a0409ba90 con 0x55ea1fd215d0 2024-09-11T22:58:32.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.666+0000 7f79fcff9640 1 -- 172.21.15.152:0/2040013898 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7a100055a0 con 0x55ea1fd215d0 2024-09-11T22:58:32.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.666+0000 7f79fcff9640 1 -- 172.21.15.152:0/2040013898 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a1000b400 con 0x55ea1fd215d0 2024-09-11T22:58:32.672 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.666+0000 7f7a1dc0d740 10 client.16902 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:32.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.666+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2040013898 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ea1fda7300 con 0x55ea1fd215d0 2024-09-11T22:58:32.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.666+0000 7f79fcff9640 1 -- 172.21.15.152:0/2040013898 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a0409ba90 con 0x55ea1fd215d0 2024-09-11T22:58:32.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.666+0000 7f79fcff9640 1 -- 172.21.15.152:0/2040013898 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7a1002f5f0 con 0x55ea1fd215d0 2024-09-11T22:58:32.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a1dc0d740 10 client.16902 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:32.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a1dc0d740 10 client.16902 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:58:32.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a1dc0d740 20 client.16902 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:32.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/2040013898 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55ea1fda7ad0 0x55ea1fda9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:32.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a1dc0d740 4 client.16902 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:58:32.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2040013898 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea1fc0b170 con 0x55ea1fda7ad0 2024-09-11T22:58:32.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a18dcf640 1 --2- 172.21.15.152:0/2040013898 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55ea1fda7ad0 0x55ea1fda9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:32.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a18dcf640 1 --2- 172.21.15.152:0/2040013898 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55ea1fda7ad0 0x55ea1fda9ef0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6f1c781a2fb1bfbb server_cookie=80dc7da3140909d7 in_seq=0 out_seq=0 2024-09-11T22:58:32.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f79fcff9640 10 client.16902 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:58:32.673 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f79fcff9640 1 -- 172.21.15.152:0/2040013898 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea1fc0b170 con 0x55ea1fda7ad0 2024-09-11T22:58:32.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f79fcff9640 10 client.16902 handle_command_reply: tid=0 2024-09-11T22:58:32.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a1dc0d740 1 client.16902 shutdown 2024-09-11T22:58:32.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2040013898 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd215d0 msgr2=0x55ea1fd34480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:32.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/2040013898 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd215d0 0x55ea1fd34480 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7a1001ba90 tx=0x7f7a10003a30 comp rx=0 tx=0).stop 2024-09-11T22:58:32.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2040013898 shutdown_connections 2024-09-11T22:58:32.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/2040013898 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55ea1fda7ad0 0x55ea1fda9ef0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/2040013898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ea1fd2c370 0x55ea1fd31450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/2040013898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea1fd22b40 0x55ea1fd349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a1dc0d740 1 --2- 172.21.15.152:0/2040013898 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ea1fd215d0 0x55ea1fd34480 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2040013898 >> 172.21.15.152:0/2040013898 conn(0x55ea1fcea310 msgr2=0x55ea1fd2b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:32.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.667+0000 7f79fcff9640 0 client.16902 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:58:32.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.668+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2040013898 shutdown_connections 2024-09-11T22:58:32.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.668+0000 7f7a1dc0d740 1 -- 172.21.15.152:0/2040013898 wait complete. 2024-09-11T22:58:32.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.668+0000 7f7a1dc0d740 20 client.16902 trim_cache size 0 max 16384 2024-09-11T22:58:32.675 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:58:32.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.668+0000 7f7a1b834640 1 -- 172.21.15.152:0/599409400 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7a000a2cf0 msgr2=0x7f7a000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:32.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.668+0000 7f7a1b834640 1 --2- 172.21.15.152:0/599409400 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7a000a2cf0 0x7f7a000a51b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c030040 tx=0x7f7a0c03d040 comp rx=0 tx=0).stop 2024-09-11T22:58:32.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.668+0000 7f7a1b834640 1 -- 172.21.15.152:0/599409400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a141036c0 msgr2=0x7f7a14198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:32.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.668+0000 7f7a1b834640 1 --2- 172.21.15.152:0/599409400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a141036c0 0x7f7a14198910 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400d770 tx=0x7f7a0400dc30 comp rx=0 tx=0).stop 2024-09-11T22:58:32.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.668+0000 7f7a1b834640 1 -- 172.21.15.152:0/599409400 shutdown_connections 2024-09-11T22:58:32.676 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.668+0000 7f7a1b834640 1 --2- 172.21.15.152:0/599409400 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7a000a2cf0 0x7f7a000a51b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.668+0000 7f7a1b834640 1 --2- 172.21.15.152:0/599409400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a1410c180 0x7f7a1419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.668+0000 7f7a1b834640 1 --2- 172.21.15.152:0/599409400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a141036c0 0x7f7a14198910 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.668+0000 7f7a1b834640 1 --2- 172.21.15.152:0/599409400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a14102da0 0x7f7a141983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:32.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.668+0000 7f7a1b834640 1 -- 172.21.15.152:0/599409400 >> 172.21.15.152:0/599409400 conn(0x7f7a140fc7a0 msgr2=0x7f7a140fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:32.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.668+0000 7f7a1b834640 1 -- 172.21.15.152:0/599409400 shutdown_connections 2024-09-11T22:58:32.677 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:32.669+0000 7f7a1b834640 1 -- 172.21.15.152:0/599409400 wait complete. 2024-09-11T22:58:33.385 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:58:33.632 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:34.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.144+0000 7f8543dc5640 1 -- 172.21.15.152:0/2734264849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f853c101420 msgr2=0x7f853c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:34.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.144+0000 7f8543dc5640 1 --2- 172.21.15.152:0/2734264849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f853c101420 0x7f853c10d950 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8534009cd0 tx=0x7f853402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.144+0000 7f8543dc5640 1 -- 172.21.15.152:0/2734264849 shutdown_connections 2024-09-11T22:58:34.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.144+0000 7f8543dc5640 1 --2- 172.21.15.152:0/2734264849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f853c101420 0x7f853c10d950 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.144+0000 7f8543dc5640 1 --2- 172.21.15.152:0/2734264849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f853c100a80 0x7f853c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.144+0000 7f8543dc5640 1 --2- 172.21.15.152:0/2734264849 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f853c106a80 0x7f853c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.144+0000 7f8543dc5640 1 -- 172.21.15.152:0/2734264849 >> 172.21.15.152:0/2734264849 conn(0x7f853c0fc7a0 msgr2=0x7f853c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:34.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.144+0000 7f8543dc5640 1 -- 172.21.15.152:0/2734264849 shutdown_connections 2024-09-11T22:58:34.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.144+0000 7f8543dc5640 1 -- 172.21.15.152:0/2734264849 wait complete. 2024-09-11T22:58:34.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8543dc5640 1 Processor -- start 2024-09-11T22:58:34.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8543dc5640 1 -- start start 2024-09-11T22:58:34.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8543dc5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f853c100a80 0x7f853c1972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8543dc5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f853c101420 0x7f853c197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8543dc5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f853c106a80 0x7f853c19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8543dc5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f853c110e80 con 0x7f853c100a80 2024-09-11T22:58:34.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8543dc5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f853c110d00 con 0x7f853c101420 2024-09-11T22:58:34.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8543dc5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f853c111000 con 0x7f853c106a80 2024-09-11T22:58:34.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8541b61640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f853c100a80 0x7f853c1972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8541b61640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f853c100a80 0x7f853c1972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41740/0 (socket says 172.21.15.152:41740) 2024-09-11T22:58:34.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8541b61640 1 -- 172.21.15.152:0/3568187848 learned_addr learned my addr 172.21.15.152:0/3568187848 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:34.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8542362640 1 --2- 172.21.15.152:0/3568187848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f853c106a80 0x7f853c19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8541360640 1 --2- 172.21.15.152:0/3568187848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f853c101420 0x7f853c197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8541b61640 1 -- 172.21.15.152:0/3568187848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f853c106a80 msgr2=0x7f853c19bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:34.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8541b61640 1 --2- 172.21.15.152:0/3568187848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f853c106a80 0x7f853c19bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8541b61640 1 -- 172.21.15.152:0/3568187848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f853c101420 msgr2=0x7f853c197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:34.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8541b61640 1 --2- 172.21.15.152:0/3568187848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f853c101420 0x7f853c197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.145+0000 7f8541b61640 1 -- 172.21.15.152:0/3568187848 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853c19c290 con 0x7f853c100a80 2024-09-11T22:58:34.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.146+0000 7f8541b61640 1 --2- 172.21.15.152:0/3568187848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f853c100a80 0x7f853c1972e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f852c00c890 tx=0x7f852c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:34.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.146+0000 7f8532ffd640 1 -- 172.21.15.152:0/3568187848 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c004220 con 0x7f853c100a80 2024-09-11T22:58:34.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.146+0000 7f8532ffd640 1 -- 172.21.15.152:0/3568187848 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f852c0043c0 con 0x7f853c100a80 2024-09-11T22:58:34.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.146+0000 7f8543dc5640 1 -- 172.21.15.152:0/3568187848 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853c19c580 con 0x7f853c100a80 2024-09-11T22:58:34.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.146+0000 7f8532ffd640 1 -- 172.21.15.152:0/3568187848 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c0049e0 con 0x7f853c100a80 2024-09-11T22:58:34.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.146+0000 7f8543dc5640 1 -- 172.21.15.152:0/3568187848 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f853c06a790 con 0x7f853c100a80 2024-09-11T22:58:34.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.148+0000 7f8532ffd640 1 -- 172.21.15.152:0/3568187848 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f852c0079b0 con 0x7f853c100a80 2024-09-11T22:58:34.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.149+0000 7f854619e740 1 Processor -- start 2024-09-11T22:58:34.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.149+0000 7f8532ffd640 1 --2- 172.21.15.152:0/3568187848 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f85280a2cf0 0x7f85280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.149+0000 7f854619e740 1 -- start start 2024-09-11T22:58:34.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.149+0000 7f8542362640 1 --2- 172.21.15.152:0/3568187848 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f85280a2cf0 0x7f85280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.149+0000 7f8532ffd640 1 -- 172.21.15.152:0/3568187848 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f852c0cf390 con 0x7f853c100a80 2024-09-11T22:58:34.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.149+0000 7f854619e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906e9a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.149+0000 7f854619e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9a5d0 0x557906e9aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.149+0000 7f854619e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906ea7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.149+0000 7f854619e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557906e002f0 con 0x557906e9bb40 2024-09-11T22:58:34.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.149+0000 7f854619e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557906da7cf0 con 0x557906ea5370 2024-09-11T22:58:34.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.149+0000 7f854619e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557906dfc2b0 con 0x557906e9a5d0 2024-09-11T22:58:34.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.149+0000 7f8541b61640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906e9a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.149+0000 7f8541b61640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906e9a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41748/0 (socket says 172.21.15.152:41748) 2024-09-11T22:58:34.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.149+0000 7f8541b61640 1 -- 172.21.15.152:0/1333631801 learned_addr learned my addr 172.21.15.152:0/1333631801 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:34.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.150+0000 7f8542362640 1 --2- 172.21.15.152:0/1333631801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906ea7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.150+0000 7f8541360640 1 --2- 172.21.15.152:0/1333631801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9a5d0 0x557906e9aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.150+0000 7f8541b61640 1 -- 172.21.15.152:0/1333631801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9a5d0 msgr2=0x557906e9aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:34.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.150+0000 7f8541b61640 1 --2- 172.21.15.152:0/1333631801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9a5d0 0x557906e9aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.150+0000 7f8541b61640 1 -- 172.21.15.152:0/1333631801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 msgr2=0x557906ea7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:34.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.150+0000 7f8541b61640 1 --2- 172.21.15.152:0/1333631801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906ea7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.150+0000 7f8541b61640 1 -- 172.21.15.152:0/1333631801 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557906d8c3a0 con 0x557906e9bb40 2024-09-11T22:58:34.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.150+0000 7f8541360640 1 --2- 172.21.15.152:0/1333631801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9a5d0 0x557906e9aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:34.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.150+0000 7f8542362640 1 --2- 172.21.15.152:0/1333631801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906ea7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:34.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.150+0000 7f8542362640 1 --2- 172.21.15.152:0/3568187848 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f85280a2cf0 0x7f85280a51b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8534004ab0 tx=0x7f8534031510 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:34.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.150+0000 7f8541b61640 1 --2- 172.21.15.152:0/1333631801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906e9a090 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f852c0a4000 tx=0x7f852c09b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd48094213938aaa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:34.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.150+0000 7f8527fff640 1 -- 172.21.15.152:0/1333631801 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c01d040 con 0x557906e9bb40 2024-09-11T22:58:34.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.150+0000 7f8527fff640 1 -- 172.21.15.152:0/1333631801 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f852c0ccd70 con 0x557906e9bb40 2024-09-11T22:58:34.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.150+0000 7f8527fff640 1 -- 172.21.15.152:0/1333631801 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c0a42a0 con 0x557906e9bb40 2024-09-11T22:58:34.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.151+0000 7f854619e740 1 -- 172.21.15.152:0/1333631801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 msgr2=0x557906e9a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:34.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.151+0000 7f854619e740 1 --2- 172.21.15.152:0/1333631801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906e9a090 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f852c0a4000 tx=0x7f852c09b970 comp rx=0 tx=0).stop 2024-09-11T22:58:34.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.151+0000 7f854619e740 1 -- 172.21.15.152:0/1333631801 shutdown_connections 2024-09-11T22:58:34.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.151+0000 7f854619e740 1 --2- 172.21.15.152:0/1333631801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906ea7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.151+0000 7f854619e740 1 --2- 172.21.15.152:0/1333631801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9a5d0 0x557906e9aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.151+0000 7f854619e740 1 --2- 172.21.15.152:0/1333631801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906e9a090 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.151+0000 7f854619e740 1 -- 172.21.15.152:0/1333631801 >> 172.21.15.152:0/1333631801 conn(0x557906e63310 msgr2=0x557906e96c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:34.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.151+0000 7f854619e740 1 -- 172.21.15.152:0/1333631801 shutdown_connections 2024-09-11T22:58:34.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.151+0000 7f854619e740 1 -- 172.21.15.152:0/1333631801 wait complete. 2024-09-11T22:58:34.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.151+0000 7f854619e740 1 -- start start 2024-09-11T22:58:34.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.151+0000 7f854619e740 1 Processor -- start 2024-09-11T22:58:34.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f854619e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9a5d0 0x557906ea4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f854619e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906ea2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f854619e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906ea26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f854619e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557906ea8070 con 0x557906e9bb40 2024-09-11T22:58:34.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f854619e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557906dbb060 con 0x557906ea5370 2024-09-11T22:58:34.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f854619e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557906ea81f0 con 0x557906e9a5d0 2024-09-11T22:58:34.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f854619e740 10 client.0 _ll_register_callbacks cb 0x557906b0e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:34.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f8541360640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906ea2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f8541360640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906ea2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41750/0 (socket says 172.21.15.152:41750) 2024-09-11T22:58:34.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f8541360640 1 -- 172.21.15.152:0/2422563888 learned_addr learned my addr 172.21.15.152:0/2422563888 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:34.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f8541b61640 1 --2- 172.21.15.152:0/2422563888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9a5d0 0x557906ea4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f8542362640 1 --2- 172.21.15.152:0/2422563888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906ea26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f8541360640 1 -- 172.21.15.152:0/2422563888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9a5d0 msgr2=0x557906ea4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:34.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f8541360640 1 --2- 172.21.15.152:0/2422563888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9a5d0 0x557906ea4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f8541360640 1 -- 172.21.15.152:0/2422563888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 msgr2=0x557906ea26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:34.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f8541360640 1 --2- 172.21.15.152:0/2422563888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906ea26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f8541360640 1 -- 172.21.15.152:0/2422563888 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557906b713c0 con 0x557906e9bb40 2024-09-11T22:58:34.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f8541360640 1 --2- 172.21.15.152:0/2422563888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906ea2090 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f853800b8f0 tx=0x7f853800bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e6100ee4b9bf6e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:34.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.152+0000 7f85257fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:34.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.153+0000 7f85257fa640 1 -- 172.21.15.152:0/2422563888 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853801b470 con 0x557906e9bb40 2024-09-11T22:58:34.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.153+0000 7f854619e740 1 -- 172.21.15.152:0/2422563888 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f852c0ccd70 con 0x557906e9bb40 2024-09-11T22:58:34.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.153+0000 7f85257fa640 1 -- 172.21.15.152:0/2422563888 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f853801b610 con 0x557906e9bb40 2024-09-11T22:58:34.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.153+0000 7f85257fa640 1 -- 172.21.15.152:0/2422563888 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f853801b810 con 0x557906e9bb40 2024-09-11T22:58:34.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.153+0000 7f854619e740 10 client.16914 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:34.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.153+0000 7f854619e740 1 -- 172.21.15.152:0/2422563888 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557906eb3b50 con 0x557906e9bb40 2024-09-11T22:58:34.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.153+0000 7f85257fa640 1 -- 172.21.15.152:0/2422563888 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f852c0ccd70 con 0x557906e9bb40 2024-09-11T22:58:34.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.153+0000 7f85257fa640 1 -- 172.21.15.152:0/2422563888 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8538025c60 con 0x557906e9bb40 2024-09-11T22:58:34.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.153+0000 7f854619e740 10 client.16914 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:34.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.153+0000 7f854619e740 10 client.16914 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:58:34.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.153+0000 7f854619e740 20 client.16914 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:34.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.153+0000 7f854619e740 1 --2- 172.21.15.152:0/2422563888 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x557906eb4fb0 0x557906eb73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.153+0000 7f854619e740 4 client.16914 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:58:34.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.153+0000 7f854619e740 1 -- 172.21.15.152:0/2422563888 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557906e0d860 con 0x557906eb4fb0 2024-09-11T22:58:34.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.154+0000 7f8541b61640 1 --2- 172.21.15.152:0/2422563888 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x557906eb4fb0 0x557906eb73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.154+0000 7f8541b61640 1 --2- 172.21.15.152:0/2422563888 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x557906eb4fb0 0x557906eb73f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=753e2928b30bf04 server_cookie=9f11f6702c4f3cbb in_seq=0 out_seq=0 2024-09-11T22:58:34.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.154+0000 7f85257fa640 10 client.16914 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:58:34.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.155+0000 7f85257fa640 1 -- 172.21.15.152:0/2422563888 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557906e0d860 con 0x557906eb4fb0 2024-09-11T22:58:34.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.155+0000 7f85257fa640 10 client.16914 handle_command_reply: tid=0 2024-09-11T22:58:34.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.155+0000 7f854619e740 1 client.16914 shutdown 2024-09-11T22:58:34.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.155+0000 7f854619e740 1 -- 172.21.15.152:0/2422563888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 msgr2=0x557906ea2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:34.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.155+0000 7f854619e740 1 --2- 172.21.15.152:0/2422563888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906ea2090 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f853800b8f0 tx=0x7f853800bdb0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.155+0000 7f854619e740 1 -- 172.21.15.152:0/2422563888 shutdown_connections 2024-09-11T22:58:34.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.155+0000 7f854619e740 1 --2- 172.21.15.152:0/2422563888 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x557906eb4fb0 0x557906eb73f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.155+0000 7f854619e740 1 --2- 172.21.15.152:0/2422563888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906ea26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.155+0000 7f854619e740 1 --2- 172.21.15.152:0/2422563888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906ea2090 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.155+0000 7f85257fa640 0 client.16914 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:58:34.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.155+0000 7f854619e740 1 --2- 172.21.15.152:0/2422563888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9a5d0 0x557906ea4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.155+0000 7f854619e740 1 -- 172.21.15.152:0/2422563888 >> 172.21.15.152:0/2422563888 conn(0x557906e63310 msgr2=0x557906e9c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:34.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.155+0000 7f854619e740 1 -- 172.21.15.152:0/2422563888 shutdown_connections 2024-09-11T22:58:34.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.155+0000 7f854619e740 1 -- 172.21.15.152:0/2422563888 wait complete. 2024-09-11T22:58:34.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.155+0000 7f854619e740 20 client.16914 trim_cache size 0 max 16384 2024-09-11T22:58:34.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f854619e740 1 Processor -- start 2024-09-11T22:58:34.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f854619e740 1 -- start start 2024-09-11T22:58:34.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f854619e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9a5d0 0x557906eb1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f854619e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906e9d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f854619e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906e9d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f854619e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1afa00 con 0x557906e9bb40 2024-09-11T22:58:34.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f854619e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1b1540 con 0x557906ea5370 2024-09-11T22:58:34.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f854619e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1b00d0 con 0x557906e9a5d0 2024-09-11T22:58:34.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f8541360640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906e9d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f8541360640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906e9d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41766/0 (socket says 172.21.15.152:41766) 2024-09-11T22:58:34.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f8541360640 1 -- 172.21.15.152:0/1758002802 learned_addr learned my addr 172.21.15.152:0/1758002802 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:34.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f8542362640 1 --2- 172.21.15.152:0/1758002802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906e9d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f8541b61640 1 --2- 172.21.15.152:0/1758002802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9a5d0 0x557906eb1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f8541360640 1 -- 172.21.15.152:0/1758002802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9a5d0 msgr2=0x557906eb1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:34.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f8541360640 1 --2- 172.21.15.152:0/1758002802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9a5d0 0x557906eb1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f8541360640 1 -- 172.21.15.152:0/1758002802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 msgr2=0x557906e9d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:34.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f8541360640 1 --2- 172.21.15.152:0/1758002802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906e9d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.168+0000 7f8541360640 1 -- 172.21.15.152:0/1758002802 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557906e97960 con 0x557906e9bb40 2024-09-11T22:58:34.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.169+0000 7f8541360640 1 --2- 172.21.15.152:0/1758002802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906e9d180 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x557906e97ce0 tx=0x7f8538004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a646318ffdef57 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:34.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.169+0000 7f85277fe640 1 -- 172.21.15.152:0/1758002802 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853802b040 con 0x557906e9bb40 2024-09-11T22:58:34.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.169+0000 7f85277fe640 1 -- 172.21.15.152:0/1758002802 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8538016070 con 0x557906e9bb40 2024-09-11T22:58:34.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.169+0000 7f85277fe640 1 -- 172.21.15.152:0/1758002802 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538012c40 con 0x557906e9bb40 2024-09-11T22:58:34.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.169+0000 7f854619e740 1 -- 172.21.15.152:0/1758002802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 msgr2=0x557906e9d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:34.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.169+0000 7f854619e740 1 --2- 172.21.15.152:0/1758002802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906e9d180 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x557906e97ce0 tx=0x7f8538004190 comp rx=0 tx=0).stop 2024-09-11T22:58:34.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.169+0000 7f854619e740 1 -- 172.21.15.152:0/1758002802 shutdown_connections 2024-09-11T22:58:34.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.169+0000 7f854619e740 1 --2- 172.21.15.152:0/1758002802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906e9d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.169+0000 7f854619e740 1 --2- 172.21.15.152:0/1758002802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9bb40 0x557906e9d180 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.169+0000 7f854619e740 1 --2- 172.21.15.152:0/1758002802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9a5d0 0x557906eb1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.169+0000 7f854619e740 1 -- 172.21.15.152:0/1758002802 >> 172.21.15.152:0/1758002802 conn(0x557906e63310 msgr2=0x557906ea1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:34.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.170+0000 7f854619e740 1 -- 172.21.15.152:0/1758002802 shutdown_connections 2024-09-11T22:58:34.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.170+0000 7f854619e740 1 -- 172.21.15.152:0/1758002802 wait complete. 2024-09-11T22:58:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.170+0000 7f854619e740 1 -- start start 2024-09-11T22:58:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.170+0000 7f854619e740 1 Processor -- start 2024-09-11T22:58:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.170+0000 7f854619e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9a5d0 0x557906ead8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.170+0000 7f854619e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9bb40 0x557906e9ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.170+0000 7f854619e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906e9efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.170+0000 7f854619e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1b0e70 con 0x557906e9a5d0 2024-09-11T22:58:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.170+0000 7f854619e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1b07a0 con 0x557906ea5370 2024-09-11T22:58:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.170+0000 7f854619e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f853c1ad9c0 con 0x557906e9bb40 2024-09-11T22:58:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.170+0000 7f8541b61640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9a5d0 0x557906ead8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.170+0000 7f854619e740 10 client.0 _ll_register_callbacks cb 0x7f8538004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.170+0000 7f8541b61640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9a5d0 0x557906ead8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41772/0 (socket says 172.21.15.152:41772) 2024-09-11T22:58:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.170+0000 7f8541b61640 1 -- 172.21.15.152:0/2868488301 learned_addr learned my addr 172.21.15.152:0/2868488301 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f8541360640 1 --2- 172.21.15.152:0/2868488301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9bb40 0x557906e9ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f8542362640 1 --2- 172.21.15.152:0/2868488301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906e9efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f8541b61640 1 -- 172.21.15.152:0/2868488301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9bb40 msgr2=0x557906e9ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:34.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f8541b61640 1 --2- 172.21.15.152:0/2868488301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9bb40 0x557906e9ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f8541b61640 1 -- 172.21.15.152:0/2868488301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 msgr2=0x557906e9efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:34.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f8541b61640 1 --2- 172.21.15.152:0/2868488301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906e9efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f8541b61640 1 -- 172.21.15.152:0/2868488301 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557906e9f530 con 0x557906e9a5d0 2024-09-11T22:58:34.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f8542362640 1 --2- 172.21.15.152:0/2868488301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906e9efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:34.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f8541b61640 1 --2- 172.21.15.152:0/2868488301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9a5d0 0x557906ead8e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f852c005fd0 tx=0x7f852c097c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4caa71b7e1775c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:34.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f85257fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:34.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f85257fa640 1 -- 172.21.15.152:0/2868488301 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852c07f060 con 0x557906e9a5d0 2024-09-11T22:58:34.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f85257fa640 1 -- 172.21.15.152:0/2868488301 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f852c007ca0 con 0x557906e9a5d0 2024-09-11T22:58:34.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f854619e740 1 -- 172.21.15.152:0/2868488301 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8538016070 con 0x557906e9a5d0 2024-09-11T22:58:34.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f85257fa640 1 -- 172.21.15.152:0/2868488301 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f852c01d040 con 0x557906e9a5d0 2024-09-11T22:58:34.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f854619e740 10 client.16920 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:34.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f854619e740 1 -- 172.21.15.152:0/2868488301 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557906f1f9d0 con 0x557906e9a5d0 2024-09-11T22:58:34.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.171+0000 7f85257fa640 1 -- 172.21.15.152:0/2868488301 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8538016070 con 0x557906e9a5d0 2024-09-11T22:58:34.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.172+0000 7f85257fa640 1 -- 172.21.15.152:0/2868488301 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f852c0860c0 con 0x557906e9a5d0 2024-09-11T22:58:34.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.172+0000 7f854619e740 10 client.16920 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:34.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.172+0000 7f854619e740 10 client.16920 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:58:34.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.172+0000 7f854619e740 20 client.16920 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:34.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.172+0000 7f854619e740 1 --2- 172.21.15.152:0/2868488301 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x557906f1fc50 0x557906f22040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:34.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.172+0000 7f854619e740 4 client.16920 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:58:34.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.172+0000 7f854619e740 1 -- 172.21.15.152:0/2868488301 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557906e07d70 con 0x557906f1fc50 2024-09-11T22:58:34.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.172+0000 7f8541360640 1 --2- 172.21.15.152:0/2868488301 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x557906f1fc50 0x557906f22040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:34.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.172+0000 7f85257fa640 10 client.16920 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:58:34.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.172+0000 7f8541360640 1 --2- 172.21.15.152:0/2868488301 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x557906f1fc50 0x557906f22040 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ede3a59c4195dbe server_cookie=cced93a14f3bb3e1 in_seq=0 out_seq=0 2024-09-11T22:58:34.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.172+0000 7f85257fa640 1 -- 172.21.15.152:0/2868488301 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557906e07d70 con 0x557906f1fc50 2024-09-11T22:58:34.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.172+0000 7f85257fa640 10 client.16920 handle_command_reply: tid=0 2024-09-11T22:58:34.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.172+0000 7f854619e740 1 client.16920 shutdown 2024-09-11T22:58:34.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f854619e740 1 -- 172.21.15.152:0/2868488301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9a5d0 msgr2=0x557906ead8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:34.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f854619e740 1 --2- 172.21.15.152:0/2868488301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9a5d0 0x557906ead8e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f852c005fd0 tx=0x7f852c097c10 comp rx=0 tx=0).stop 2024-09-11T22:58:34.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f854619e740 1 -- 172.21.15.152:0/2868488301 shutdown_connections 2024-09-11T22:58:34.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f854619e740 1 --2- 172.21.15.152:0/2868488301 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x557906f1fc50 0x557906f22040 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f854619e740 1 --2- 172.21.15.152:0/2868488301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557906ea5370 0x557906e9efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f854619e740 1 --2- 172.21.15.152:0/2868488301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557906e9bb40 0x557906e9ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f854619e740 1 --2- 172.21.15.152:0/2868488301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557906e9a5d0 0x557906ead8e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f85257fa640 0 client.16920 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:58:34.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f854619e740 1 -- 172.21.15.152:0/2868488301 >> 172.21.15.152:0/2868488301 conn(0x557906e63310 msgr2=0x557906e954e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:34.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f854619e740 1 -- 172.21.15.152:0/2868488301 shutdown_connections 2024-09-11T22:58:34.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f854619e740 1 -- 172.21.15.152:0/2868488301 wait complete. 2024-09-11T22:58:34.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f854619e740 20 client.16920 trim_cache size 0 max 16384 2024-09-11T22:58:34.180 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:58:34.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f8543dc5640 1 -- 172.21.15.152:0/3568187848 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f85280a2cf0 msgr2=0x7f85280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:34.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f8543dc5640 1 --2- 172.21.15.152:0/3568187848 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f85280a2cf0 0x7f85280a51b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8534004ab0 tx=0x7f8534031510 comp rx=0 tx=0).stop 2024-09-11T22:58:34.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f8543dc5640 1 -- 172.21.15.152:0/3568187848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f853c100a80 msgr2=0x7f853c1972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:34.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f8543dc5640 1 --2- 172.21.15.152:0/3568187848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f853c100a80 0x7f853c1972e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f852c00c890 tx=0x7f852c00cd50 comp rx=0 tx=0).stop 2024-09-11T22:58:34.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f8543dc5640 1 -- 172.21.15.152:0/3568187848 shutdown_connections 2024-09-11T22:58:34.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f8543dc5640 1 --2- 172.21.15.152:0/3568187848 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f85280a2cf0 0x7f85280a51b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f8543dc5640 1 --2- 172.21.15.152:0/3568187848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f853c106a80 0x7f853c19bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f8543dc5640 1 --2- 172.21.15.152:0/3568187848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f853c101420 0x7f853c197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f8543dc5640 1 --2- 172.21.15.152:0/3568187848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f853c100a80 0x7f853c1972e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:34.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.173+0000 7f8543dc5640 1 -- 172.21.15.152:0/3568187848 >> 172.21.15.152:0/3568187848 conn(0x7f853c0fc7a0 msgr2=0x7f853c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:34.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.174+0000 7f8543dc5640 1 -- 172.21.15.152:0/3568187848 shutdown_connections 2024-09-11T22:58:34.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:34.174+0000 7f8543dc5640 1 -- 172.21.15.152:0/3568187848 wait complete. 2024-09-11T22:58:34.373 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:34 smithi152 ceph-mon[31181]: pgmap v173: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:34.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:34 smithi191 ceph-mon[35211]: pgmap v173: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:34.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:34 smithi204 ceph-mon[35454]: pgmap v173: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:35.013 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:58:35.260 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.803+0000 7f0974ca1640 1 -- 172.21.15.152:0/2204880222 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0970100a80 msgr2=0x7f0970100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.803+0000 7f0974ca1640 1 --2- 172.21.15.152:0/2204880222 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0970100a80 0x7f0970100ee0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0964009930 tx=0x7f096402a420 comp rx=0 tx=0).stop 2024-09-11T22:58:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.803+0000 7f0974ca1640 1 -- 172.21.15.152:0/2204880222 shutdown_connections 2024-09-11T22:58:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.803+0000 7f0974ca1640 1 --2- 172.21.15.152:0/2204880222 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0970101420 0x7f097010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.803+0000 7f0974ca1640 1 --2- 172.21.15.152:0/2204880222 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0970100a80 0x7f0970100ee0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.803+0000 7f0974ca1640 1 --2- 172.21.15.152:0/2204880222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0970106a80 0x7f0970106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.803+0000 7f0974ca1640 1 -- 172.21.15.152:0/2204880222 >> 172.21.15.152:0/2204880222 conn(0x7f09700fc7a0 msgr2=0x7f09700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.803+0000 7f0974ca1640 1 -- 172.21.15.152:0/2204880222 shutdown_connections 2024-09-11T22:58:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.803+0000 7f0974ca1640 1 -- 172.21.15.152:0/2204880222 wait complete. 2024-09-11T22:58:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.804+0000 7f0974ca1640 1 Processor -- start 2024-09-11T22:58:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.804+0000 7f0974ca1640 1 -- start start 2024-09-11T22:58:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.804+0000 7f0974ca1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0970100a80 0x7f0970198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.804+0000 7f0974ca1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0970101420 0x7f0970198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.804+0000 7f0974ca1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0970106a80 0x7f097019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.804+0000 7f0974ca1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0970111fa0 con 0x7f0970106a80 2024-09-11T22:58:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.804+0000 7f0974ca1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0970111e20 con 0x7f0970101420 2024-09-11T22:58:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.804+0000 7f0974ca1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0970112120 con 0x7f0970100a80 2024-09-11T22:58:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.805+0000 7f096ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0970106a80 0x7f097019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.805+0000 7f096ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0970106a80 0x7f097019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41790/0 (socket says 172.21.15.152:41790) 2024-09-11T22:58:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.805+0000 7f096ed9d640 1 -- 172.21.15.152:0/4224727153 learned_addr learned my addr 172.21.15.152:0/4224727153 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.805+0000 7f096dd9b640 1 --2- 172.21.15.152:0/4224727153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0970101420 0x7f0970198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.805+0000 7f096e59c640 1 --2- 172.21.15.152:0/4224727153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0970100a80 0x7f0970198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.805+0000 7f096ed9d640 1 -- 172.21.15.152:0/4224727153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0970100a80 msgr2=0x7f0970198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.805+0000 7f096ed9d640 1 --2- 172.21.15.152:0/4224727153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0970100a80 0x7f0970198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.805+0000 7f096ed9d640 1 -- 172.21.15.152:0/4224727153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0970101420 msgr2=0x7f0970198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.805+0000 7f096ed9d640 1 --2- 172.21.15.152:0/4224727153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0970101420 0x7f0970198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.805+0000 7f096ed9d640 1 -- 172.21.15.152:0/4224727153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097019d3d0 con 0x7f0970106a80 2024-09-11T22:58:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.805+0000 7f096ed9d640 1 --2- 172.21.15.152:0/4224727153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0970106a80 0x7f097019ccf0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f096800b970 tx=0x7f096800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.805+0000 7f095f7fe640 1 -- 172.21.15.152:0/4224727153 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968019070 con 0x7f0970106a80 2024-09-11T22:58:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.805+0000 7f0974ca1640 1 -- 172.21.15.152:0/4224727153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097019d6c0 con 0x7f0970106a80 2024-09-11T22:58:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.805+0000 7f0974ca1640 1 -- 172.21.15.152:0/4224727153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09701a4fa0 con 0x7f0970106a80 2024-09-11T22:58:35.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.805+0000 7f095f7fe640 1 -- 172.21.15.152:0/4224727153 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f096800cb40 con 0x7f0970106a80 2024-09-11T22:58:35.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.805+0000 7f095f7fe640 1 -- 172.21.15.152:0/4224727153 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968007610 con 0x7f0970106a80 2024-09-11T22:58:35.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.807+0000 7f095f7fe640 1 -- 172.21.15.152:0/4224727153 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f096801c070 con 0x7f0970106a80 2024-09-11T22:58:35.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.808+0000 7f097707a740 1 Processor -- start 2024-09-11T22:58:35.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.808+0000 7f097707a740 1 -- start start 2024-09-11T22:58:35.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.808+0000 7f097707a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96bdb40 0x562ee96bc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.808+0000 7f097707a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96bc5d0 0x562ee96bca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.808+0000 7f097707a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96c7370 0x562ee96c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.808+0000 7f097707a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x562ee96222f0 con 0x562ee96bdb40 2024-09-11T22:58:35.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.808+0000 7f097707a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562ee95c9cf0 con 0x562ee96c7370 2024-09-11T22:58:35.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.808+0000 7f097707a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562ee961e2b0 con 0x562ee96bc5d0 2024-09-11T22:58:35.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.808+0000 7f096dd9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96bdb40 0x562ee96bc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.808+0000 7f096dd9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96bdb40 0x562ee96bc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41794/0 (socket says 172.21.15.152:41794) 2024-09-11T22:58:35.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.808+0000 7f096dd9b640 1 -- 172.21.15.152:0/4079519060 learned_addr learned my addr 172.21.15.152:0/4079519060 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:35.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.808+0000 7f095f7fe640 1 --2- 172.21.15.152:0/4224727153 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f09580a2cf0 0x7f09580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.808+0000 7f096dd9b640 1 --2- 172.21.15.152:0/4079519060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96c7370 0x562ee96c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.809+0000 7f096e59c640 1 --2- 172.21.15.152:0/4079519060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96bc5d0 0x562ee96bca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.809+0000 7f096ed9d640 1 --2- 172.21.15.152:0/4224727153 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f09580a2cf0 0x7f09580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.809+0000 7f095f7fe640 1 -- 172.21.15.152:0/4224727153 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f09680ce7e0 con 0x7f0970106a80 2024-09-11T22:58:35.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.809+0000 7f096dd9b640 1 -- 172.21.15.152:0/4079519060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96bc5d0 msgr2=0x562ee96bca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:35.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.809+0000 7f096dd9b640 1 --2- 172.21.15.152:0/4079519060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96bc5d0 0x562ee96bca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.809+0000 7f096dd9b640 1 -- 172.21.15.152:0/4079519060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96c7370 msgr2=0x562ee96c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:35.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.809+0000 7f096dd9b640 1 --2- 172.21.15.152:0/4079519060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96c7370 0x562ee96c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.809+0000 7f096dd9b640 1 -- 172.21.15.152:0/4079519060 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ee95ae3a0 con 0x562ee96bdb40 2024-09-11T22:58:35.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.809+0000 7f096dd9b640 1 --2- 172.21.15.152:0/4079519060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96bdb40 0x562ee96bc090 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0964031550 tx=0x7f09640058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efdee59e66162a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:35.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.809+0000 7f096ed9d640 1 --2- 172.21.15.152:0/4224727153 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f09580a2cf0 0x7f09580a51b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f096809adb0 tx=0x7f096806cde0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:35.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.809+0000 7f095cff9640 1 -- 172.21.15.152:0/4079519060 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964035030 con 0x562ee96bdb40 2024-09-11T22:58:35.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.809+0000 7f095cff9640 1 -- 172.21.15.152:0/4079519060 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0964003ad0 con 0x562ee96bdb40 2024-09-11T22:58:35.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.809+0000 7f095cff9640 1 -- 172.21.15.152:0/4079519060 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096403a860 con 0x562ee96bdb40 2024-09-11T22:58:35.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.810+0000 7f097707a740 1 -- 172.21.15.152:0/4079519060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96bdb40 msgr2=0x562ee96bc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:35.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.810+0000 7f097707a740 1 --2- 172.21.15.152:0/4079519060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96bdb40 0x562ee96bc090 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0964031550 tx=0x7f09640058b0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.810+0000 7f097707a740 1 -- 172.21.15.152:0/4079519060 shutdown_connections 2024-09-11T22:58:35.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.810+0000 7f097707a740 1 --2- 172.21.15.152:0/4079519060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96c7370 0x562ee96c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.810+0000 7f097707a740 1 --2- 172.21.15.152:0/4079519060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96bc5d0 0x562ee96bca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.810+0000 7f097707a740 1 --2- 172.21.15.152:0/4079519060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96bdb40 0x562ee96bc090 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.810+0000 7f097707a740 1 -- 172.21.15.152:0/4079519060 >> 172.21.15.152:0/4079519060 conn(0x562ee9685310 msgr2=0x562ee96b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:35.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.810+0000 7f097707a740 1 -- 172.21.15.152:0/4079519060 shutdown_connections 2024-09-11T22:58:35.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.810+0000 7f097707a740 1 -- 172.21.15.152:0/4079519060 wait complete. 2024-09-11T22:58:35.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.810+0000 7f097707a740 1 -- start start 2024-09-11T22:58:35.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f097707a740 1 Processor -- start 2024-09-11T22:58:35.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f097707a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96bc5d0 0x562ee96c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f097707a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96bdb40 0x562ee96c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f097707a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96c7370 0x562ee96c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f097707a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x562ee96ca010 con 0x562ee96c7370 2024-09-11T22:58:35.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f097707a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562ee95dd060 con 0x562ee96bdb40 2024-09-11T22:58:35.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f097707a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562ee96ca190 con 0x562ee96bc5d0 2024-09-11T22:58:35.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f097707a740 10 client.0 _ll_register_callbacks cb 0x562ee93308c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:35.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f096dd9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96c7370 0x562ee96c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f096dd9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96c7370 0x562ee96c44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41800/0 (socket says 172.21.15.152:41800) 2024-09-11T22:58:35.816 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f096dd9b640 1 -- 172.21.15.152:0/3671462168 learned_addr learned my addr 172.21.15.152:0/3671462168 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:35.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f096dd9b640 1 --2- 172.21.15.152:0/3671462168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96bc5d0 0x562ee96c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f096e59c640 1 --2- 172.21.15.152:0/3671462168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96bdb40 0x562ee96c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f096dd9b640 1 -- 172.21.15.152:0/3671462168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96bc5d0 msgr2=0x562ee96c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:35.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f096dd9b640 1 --2- 172.21.15.152:0/3671462168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96bc5d0 0x562ee96c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f096dd9b640 1 -- 172.21.15.152:0/3671462168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96bdb40 msgr2=0x562ee96c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:35.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f096dd9b640 1 --2- 172.21.15.152:0/3671462168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96bdb40 0x562ee96c3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f096dd9b640 1 -- 172.21.15.152:0/3671462168 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ee93933c0 con 0x562ee96c7370 2024-09-11T22:58:35.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.811+0000 7f096e59c640 1 --2- 172.21.15.152:0/3671462168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96bdb40 0x562ee96c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:35.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.812+0000 7f096dd9b640 1 --2- 172.21.15.152:0/3671462168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96c7370 0x562ee96c44f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0964009900 tx=0x7f096403a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf51f336dd55fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:35.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.812+0000 7f0955ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:35.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.812+0000 7f0955ffb640 1 -- 172.21.15.152:0/3671462168 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964035040 con 0x562ee96c7370 2024-09-11T22:58:35.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.812+0000 7f0955ffb640 1 -- 172.21.15.152:0/3671462168 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f09640035f0 con 0x562ee96c7370 2024-09-11T22:58:35.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.812+0000 7f097707a740 1 -- 172.21.15.152:0/3671462168 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0964003ad0 con 0x562ee96c7370 2024-09-11T22:58:35.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.812+0000 7f0955ffb640 1 -- 172.21.15.152:0/3671462168 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f096405b030 con 0x562ee96c7370 2024-09-11T22:58:35.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.812+0000 7f097707a740 10 client.16932 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:35.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.812+0000 7f097707a740 1 -- 172.21.15.152:0/3671462168 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ee96d5a80 con 0x562ee96c7370 2024-09-11T22:58:35.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.812+0000 7f0955ffb640 1 -- 172.21.15.152:0/3671462168 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0964003ad0 con 0x562ee96c7370 2024-09-11T22:58:35.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.812+0000 7f0955ffb640 1 -- 172.21.15.152:0/3671462168 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f09640594b0 con 0x562ee96c7370 2024-09-11T22:58:35.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.812+0000 7f097707a740 10 client.16932 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:35.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.812+0000 7f097707a740 10 client.16932 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:58:35.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.812+0000 7f097707a740 20 client.16932 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:35.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.812+0000 7f097707a740 1 --2- 172.21.15.152:0/3671462168 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x562ee96d6dd0 0x562ee96d91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.812+0000 7f097707a740 4 client.16932 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:58:35.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.812+0000 7f097707a740 1 -- 172.21.15.152:0/3671462168 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ee962f860 con 0x562ee96d6dd0 2024-09-11T22:58:35.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.813+0000 7f096e59c640 1 --2- 172.21.15.152:0/3671462168 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x562ee96d6dd0 0x562ee96d91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.813+0000 7f0955ffb640 10 client.16932 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:58:35.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.813+0000 7f096e59c640 1 --2- 172.21.15.152:0/3671462168 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x562ee96d6dd0 0x562ee96d91c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9d1d7f818302813 server_cookie=bdbf2680a9f7f41d in_seq=0 out_seq=0 2024-09-11T22:58:35.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.814+0000 7f0955ffb640 1 -- 172.21.15.152:0/3671462168 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ee962f860 con 0x562ee96d6dd0 2024-09-11T22:58:35.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.814+0000 7f0955ffb640 10 client.16932 handle_command_reply: tid=0 2024-09-11T22:58:35.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.814+0000 7f097707a740 1 client.16932 shutdown 2024-09-11T22:58:35.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.814+0000 7f097707a740 1 -- 172.21.15.152:0/3671462168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96c7370 msgr2=0x562ee96c44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:35.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.814+0000 7f097707a740 1 --2- 172.21.15.152:0/3671462168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96c7370 0x562ee96c44f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0964009900 tx=0x7f096403a440 comp rx=0 tx=0).stop 2024-09-11T22:58:35.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.814+0000 7f097707a740 1 -- 172.21.15.152:0/3671462168 shutdown_connections 2024-09-11T22:58:35.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.814+0000 7f097707a740 1 --2- 172.21.15.152:0/3671462168 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x562ee96d6dd0 0x562ee96d91c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.814+0000 7f097707a740 1 --2- 172.21.15.152:0/3671462168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96c7370 0x562ee96c44f0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.814+0000 7f097707a740 1 --2- 172.21.15.152:0/3671462168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96bdb40 0x562ee96c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.814+0000 7f0955ffb640 0 client.16932 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:58:35.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.814+0000 7f097707a740 1 --2- 172.21.15.152:0/3671462168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96bc5d0 0x562ee96c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.814+0000 7f097707a740 1 -- 172.21.15.152:0/3671462168 >> 172.21.15.152:0/3671462168 conn(0x562ee9685310 msgr2=0x562ee96be9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:35.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.814+0000 7f097707a740 1 -- 172.21.15.152:0/3671462168 shutdown_connections 2024-09-11T22:58:35.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.814+0000 7f097707a740 1 -- 172.21.15.152:0/3671462168 wait complete. 2024-09-11T22:58:35.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.814+0000 7f097707a740 20 client.16932 trim_cache size 0 max 16384 2024-09-11T22:58:35.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.827+0000 7f097707a740 1 Processor -- start 2024-09-11T22:58:35.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.827+0000 7f097707a740 1 -- start start 2024-09-11T22:58:35.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.827+0000 7f097707a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96bc5d0 0x562ee96dc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.827+0000 7f097707a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96bdb40 0x562ee96dc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.827+0000 7f097707a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96c7370 0x562ee96c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.827+0000 7f097707a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f095000e670 con 0x562ee96c7370 2024-09-11T22:58:35.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.827+0000 7f097707a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f09500101b0 con 0x562ee96bc5d0 2024-09-11T22:58:35.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.827+0000 7f097707a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f095000ed40 con 0x562ee96bdb40 2024-09-11T22:58:35.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.827+0000 7f096dd9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96c7370 0x562ee96c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.827+0000 7f096dd9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96c7370 0x562ee96c1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41802/0 (socket says 172.21.15.152:41802) 2024-09-11T22:58:35.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.827+0000 7f096dd9b640 1 -- 172.21.15.152:0/4102449270 learned_addr learned my addr 172.21.15.152:0/4102449270 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:35.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.828+0000 7f096dd9b640 1 --2- 172.21.15.152:0/4102449270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96bc5d0 0x562ee96dc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.828+0000 7f096e59c640 1 --2- 172.21.15.152:0/4102449270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96bdb40 0x562ee96dc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.828+0000 7f096dd9b640 1 -- 172.21.15.152:0/4102449270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96bdb40 msgr2=0x562ee96dc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:35.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.828+0000 7f096dd9b640 1 --2- 172.21.15.152:0/4102449270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96bdb40 0x562ee96dc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.828+0000 7f096dd9b640 1 -- 172.21.15.152:0/4102449270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96bc5d0 msgr2=0x562ee96dc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:35.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.828+0000 7f096dd9b640 1 --2- 172.21.15.152:0/4102449270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96bc5d0 0x562ee96dc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.828+0000 7f096dd9b640 1 -- 172.21.15.152:0/4102449270 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ee96b8510 con 0x562ee96c7370 2024-09-11T22:58:35.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.828+0000 7f096e59c640 1 --2- 172.21.15.152:0/4102449270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96bdb40 0x562ee96dc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:35.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.828+0000 7f096dd9b640 1 --2- 172.21.15.152:0/4102449270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96c7370 0x562ee96c1210 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0964031550 tx=0x7f0964004160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd44f58a3fa8f83 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:35.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.828+0000 7f0957fff640 1 -- 172.21.15.152:0/4102449270 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964064070 con 0x562ee96c7370 2024-09-11T22:58:35.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.828+0000 7f0957fff640 1 -- 172.21.15.152:0/4102449270 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09640315b0 con 0x562ee96c7370 2024-09-11T22:58:35.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.828+0000 7f0957fff640 1 -- 172.21.15.152:0/4102449270 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964035040 con 0x562ee96c7370 2024-09-11T22:58:35.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 -- 172.21.15.152:0/4102449270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96c7370 msgr2=0x562ee96c1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:35.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 --2- 172.21.15.152:0/4102449270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96c7370 0x562ee96c1210 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0964031550 tx=0x7f0964004160 comp rx=0 tx=0).stop 2024-09-11T22:58:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 -- 172.21.15.152:0/4102449270 shutdown_connections 2024-09-11T22:58:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 --2- 172.21.15.152:0/4102449270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96c7370 0x562ee96c1210 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 --2- 172.21.15.152:0/4102449270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96bdb40 0x562ee96dc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 --2- 172.21.15.152:0/4102449270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96bc5d0 0x562ee96dc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 -- 172.21.15.152:0/4102449270 >> 172.21.15.152:0/4102449270 conn(0x562ee9685310 msgr2=0x562ee96c2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 -- 172.21.15.152:0/4102449270 shutdown_connections 2024-09-11T22:58:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 -- 172.21.15.152:0/4102449270 wait complete. 2024-09-11T22:58:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 -- start start 2024-09-11T22:58:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 Processor -- start 2024-09-11T22:58:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96bc5d0 0x562ee96da280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96bdb40 0x562ee96da7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96c7370 0x562ee96dad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f095000fae0 con 0x562ee96bc5d0 2024-09-11T22:58:35.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f095000f410 con 0x562ee96bdb40 2024-09-11T22:58:35.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f095000c630 con 0x562ee96c7370 2024-09-11T22:58:35.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.829+0000 7f097707a740 10 client.0 _ll_register_callbacks cb 0x7f0964031840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:35.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f096dd9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96bc5d0 0x562ee96da280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f096dd9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96bc5d0 0x562ee96da280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41818/0 (socket says 172.21.15.152:41818) 2024-09-11T22:58:35.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f096dd9b640 1 -- 172.21.15.152:0/290920291 learned_addr learned my addr 172.21.15.152:0/290920291 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:35.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f096e59c640 1 --2- 172.21.15.152:0/290920291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96bdb40 0x562ee96da7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f096dd9b640 1 -- 172.21.15.152:0/290920291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96c7370 msgr2=0x562ee96dad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:35.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f096dd9b640 1 --2- 172.21.15.152:0/290920291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96c7370 0x562ee96dad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f096dd9b640 1 -- 172.21.15.152:0/290920291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96bdb40 msgr2=0x562ee96da7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:35.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f096dd9b640 1 --2- 172.21.15.152:0/290920291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96bdb40 0x562ee96da7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f096dd9b640 1 -- 172.21.15.152:0/290920291 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ee9740a00 con 0x562ee96bc5d0 2024-09-11T22:58:35.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f096dd9b640 1 --2- 172.21.15.152:0/290920291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96bc5d0 0x562ee96da280 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f0964004650 tx=0x7f0964031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9aac038abf4858 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:35.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f0955ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:35.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f0955ffb640 1 -- 172.21.15.152:0/290920291 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964064060 con 0x562ee96bc5d0 2024-09-11T22:58:35.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f0955ffb640 1 -- 172.21.15.152:0/290920291 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f096405c040 con 0x562ee96bc5d0 2024-09-11T22:58:35.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f097707a740 1 -- 172.21.15.152:0/290920291 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09640315b0 con 0x562ee96bc5d0 2024-09-11T22:58:35.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f0955ffb640 1 -- 172.21.15.152:0/290920291 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0964041020 con 0x562ee96bc5d0 2024-09-11T22:58:35.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f097707a740 10 client.16938 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:35.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f097707a740 1 -- 172.21.15.152:0/290920291 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ee9741c70 con 0x562ee96bc5d0 2024-09-11T22:58:35.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.830+0000 7f0955ffb640 1 -- 172.21.15.152:0/290920291 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09640315b0 con 0x562ee96bc5d0 2024-09-11T22:58:35.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.831+0000 7f0955ffb640 1 -- 172.21.15.152:0/290920291 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0964005020 con 0x562ee96bc5d0 2024-09-11T22:58:35.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.831+0000 7f097707a740 10 client.16938 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:35.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.831+0000 7f097707a740 10 client.16938 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:58:35.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.831+0000 7f097707a740 20 client.16938 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:35.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.831+0000 7f097707a740 1 --2- 172.21.15.152:0/290920291 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x562ee97422d0 0x562ee97446f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:35.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.831+0000 7f097707a740 4 client.16938 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:58:35.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.831+0000 7f097707a740 1 -- 172.21.15.152:0/290920291 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ee9629d70 con 0x562ee97422d0 2024-09-11T22:58:35.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.831+0000 7f096e59c640 1 --2- 172.21.15.152:0/290920291 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x562ee97422d0 0x562ee97446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:35.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.831+0000 7f0955ffb640 10 client.16938 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:58:35.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.831+0000 7f096e59c640 1 --2- 172.21.15.152:0/290920291 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x562ee97422d0 0x562ee97446f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e82e51ecd167f81 server_cookie=3d5ba723a3067ea5 in_seq=0 out_seq=0 2024-09-11T22:58:35.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.832+0000 7f0955ffb640 1 -- 172.21.15.152:0/290920291 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ee9629d70 con 0x562ee97422d0 2024-09-11T22:58:35.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.832+0000 7f0955ffb640 10 client.16938 handle_command_reply: tid=0 2024-09-11T22:58:35.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.832+0000 7f097707a740 1 client.16938 shutdown 2024-09-11T22:58:35.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.832+0000 7f097707a740 1 -- 172.21.15.152:0/290920291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96bc5d0 msgr2=0x562ee96da280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:35.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.832+0000 7f097707a740 1 --2- 172.21.15.152:0/290920291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96bc5d0 0x562ee96da280 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f0964004650 tx=0x7f0964031970 comp rx=0 tx=0).stop 2024-09-11T22:58:35.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.832+0000 7f097707a740 1 -- 172.21.15.152:0/290920291 shutdown_connections 2024-09-11T22:58:35.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.832+0000 7f097707a740 1 --2- 172.21.15.152:0/290920291 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x562ee97422d0 0x562ee97446f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.832+0000 7f097707a740 1 --2- 172.21.15.152:0/290920291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562ee96c7370 0x562ee96dad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.832+0000 7f097707a740 1 --2- 172.21.15.152:0/290920291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562ee96bdb40 0x562ee96da7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.832+0000 7f097707a740 1 --2- 172.21.15.152:0/290920291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562ee96bc5d0 0x562ee96da280 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.832+0000 7f0955ffb640 0 client.16938 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:58:35.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.832+0000 7f097707a740 1 -- 172.21.15.152:0/290920291 >> 172.21.15.152:0/290920291 conn(0x562ee9685310 msgr2=0x562ee96be110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:35.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.832+0000 7f097707a740 1 -- 172.21.15.152:0/290920291 shutdown_connections 2024-09-11T22:58:35.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.832+0000 7f097707a740 1 -- 172.21.15.152:0/290920291 wait complete. 2024-09-11T22:58:35.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.832+0000 7f097707a740 20 client.16938 trim_cache size 0 max 16384 2024-09-11T22:58:35.838 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:58:35.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.833+0000 7f0974ca1640 1 -- 172.21.15.152:0/4224727153 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f09580a2cf0 msgr2=0x7f09580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:35.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.833+0000 7f0974ca1640 1 --2- 172.21.15.152:0/4224727153 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f09580a2cf0 0x7f09580a51b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f096809adb0 tx=0x7f096806cde0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.833+0000 7f0974ca1640 1 -- 172.21.15.152:0/4224727153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0970106a80 msgr2=0x7f097019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:35.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.833+0000 7f0974ca1640 1 --2- 172.21.15.152:0/4224727153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0970106a80 0x7f097019ccf0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f096800b970 tx=0x7f096800be30 comp rx=0 tx=0).stop 2024-09-11T22:58:35.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.833+0000 7f0974ca1640 1 -- 172.21.15.152:0/4224727153 shutdown_connections 2024-09-11T22:58:35.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.833+0000 7f0974ca1640 1 --2- 172.21.15.152:0/4224727153 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f09580a2cf0 0x7f09580a51b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.833+0000 7f0974ca1640 1 --2- 172.21.15.152:0/4224727153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0970106a80 0x7f097019ccf0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.833+0000 7f0974ca1640 1 --2- 172.21.15.152:0/4224727153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0970101420 0x7f0970198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.833+0000 7f0974ca1640 1 --2- 172.21.15.152:0/4224727153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0970100a80 0x7f0970198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:35.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.833+0000 7f0974ca1640 1 -- 172.21.15.152:0/4224727153 >> 172.21.15.152:0/4224727153 conn(0x7f09700fc7a0 msgr2=0x7f09700fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:35.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.833+0000 7f0974ca1640 1 -- 172.21.15.152:0/4224727153 shutdown_connections 2024-09-11T22:58:35.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:35.833+0000 7f0974ca1640 1 -- 172.21.15.152:0/4224727153 wait complete. 2024-09-11T22:58:36.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:36 smithi191 ceph-mon[35211]: pgmap v174: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:58:36.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:36 smithi204 ceph-mon[35454]: pgmap v174: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:58:36.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:36 smithi152 ceph-mon[31181]: pgmap v174: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:58:36.725 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:58:37.033 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.548+0000 7fe24c5a0640 1 -- 172.21.15.152:0/2204285384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe244106f10 msgr2=0x7fe24410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.548+0000 7fe24c5a0640 1 --2- 172.21.15.152:0/2204285384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe244106f10 0x7fe24410ea90 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009ca0 tx=0x7fe23c02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.548+0000 7fe24c5a0640 1 -- 172.21.15.152:0/2204285384 shutdown_connections 2024-09-11T22:58:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.548+0000 7fe24c5a0640 1 --2- 172.21.15.152:0/2204285384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe244106f10 0x7fe24410ea90 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.548+0000 7fe24c5a0640 1 --2- 172.21.15.152:0/2204285384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe244102790 0x7fe2441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.548+0000 7fe24c5a0640 1 --2- 172.21.15.152:0/2204285384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe244101de0 0x7fe2441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.548+0000 7fe24c5a0640 1 -- 172.21.15.152:0/2204285384 >> 172.21.15.152:0/2204285384 conn(0x7fe2440fd8c0 msgr2=0x7fe2440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.548+0000 7fe24c5a0640 1 -- 172.21.15.152:0/2204285384 shutdown_connections 2024-09-11T22:58:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.548+0000 7fe24c5a0640 1 -- 172.21.15.152:0/2204285384 wait complete. 2024-09-11T22:58:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24c5a0640 1 Processor -- start 2024-09-11T22:58:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24c5a0640 1 -- start start 2024-09-11T22:58:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24c5a0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe244101de0 0x7fe2440744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24c5a0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe244102790 0x7fe244074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24c5a0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe244106f10 0x7fe244074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24c5a0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe244111f70 con 0x7fe244101de0 2024-09-11T22:58:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24c5a0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe244111df0 con 0x7fe244106f10 2024-09-11T22:58:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24c5a0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441120f0 con 0x7fe244102790 2024-09-11T22:58:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24a33c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe244101de0 0x7fe2440744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24a33c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe244101de0 0x7fe2440744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41832/0 (socket says 172.21.15.152:41832) 2024-09-11T22:58:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24a33c640 1 -- 172.21.15.152:0/4087522280 learned_addr learned my addr 172.21.15.152:0/4087522280 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24ab3d640 1 --2- 172.21.15.152:0/4087522280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe244106f10 0x7fe244074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe249b3b640 1 --2- 172.21.15.152:0/4087522280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe244102790 0x7fe244074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24a33c640 1 -- 172.21.15.152:0/4087522280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe244102790 msgr2=0x7fe244074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24a33c640 1 --2- 172.21.15.152:0/4087522280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe244102790 0x7fe244074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24a33c640 1 -- 172.21.15.152:0/4087522280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe244106f10 msgr2=0x7fe244074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24a33c640 1 --2- 172.21.15.152:0/4087522280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe244106f10 0x7fe244074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.549+0000 7fe24a33c640 1 -- 172.21.15.152:0/4087522280 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2441a6920 con 0x7fe244101de0 2024-09-11T22:58:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.550+0000 7fe24a33c640 1 --2- 172.21.15.152:0/4087522280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe244101de0 0x7fe2440744f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe24000d950 tx=0x7fe24000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.550+0000 7fe2377fe640 1 -- 172.21.15.152:0/4087522280 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240017070 con 0x7fe244101de0 2024-09-11T22:58:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.550+0000 7fe24c5a0640 1 -- 172.21.15.152:0/4087522280 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2441a6c10 con 0x7fe244101de0 2024-09-11T22:58:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.550+0000 7fe2377fe640 1 -- 172.21.15.152:0/4087522280 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe240009c30 con 0x7fe244101de0 2024-09-11T22:58:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.550+0000 7fe2377fe640 1 -- 172.21.15.152:0/4087522280 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24000f6b0 con 0x7fe244101de0 2024-09-11T22:58:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.550+0000 7fe24c5a0640 1 -- 172.21.15.152:0/4087522280 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2441a7150 con 0x7fe244101de0 2024-09-11T22:58:37.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.552+0000 7fe2377fe640 1 -- 172.21.15.152:0/4087522280 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe2400109d0 con 0x7fe244101de0 2024-09-11T22:58:37.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.552+0000 7fe24e979740 1 Processor -- start 2024-09-11T22:58:37.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe24e979740 1 -- start start 2024-09-11T22:58:37.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe24e979740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557802029b40 0x557802028090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe24e979740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578020285d0 0x557802028a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe24e979740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x557802035760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe24e979740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557801f8e2f0 con 0x557802033370 2024-09-11T22:58:37.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe24e979740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557801f35cf0 con 0x5578020285d0 2024-09-11T22:58:37.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe24e979740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557801f8a2b0 con 0x557802029b40 2024-09-11T22:58:37.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe2377fe640 1 --2- 172.21.15.152:0/4087522280 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe2300a2cf0 0x7fe2300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe24ab3d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557802029b40 0x557802028090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe249b3b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x557802035760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe24ab3d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557802029b40 0x557802028090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:33540/0 (socket says 172.21.15.152:33540) 2024-09-11T22:58:37.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe24ab3d640 1 -- 172.21.15.152:0/1769327633 learned_addr learned my addr 172.21.15.152:0/1769327633 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:37.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe24a33c640 1 --2- 172.21.15.152:0/1769327633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578020285d0 0x557802028a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe24ab3d640 1 --2- 172.21.15.152:0/4087522280 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe2300a2cf0 0x7fe2300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe2377fe640 1 -- 172.21.15.152:0/4087522280 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fe2400cf140 con 0x7fe244101de0 2024-09-11T22:58:37.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe249b3b640 1 -- 172.21.15.152:0/1769327633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557802029b40 msgr2=0x557802028090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe249b3b640 1 --2- 172.21.15.152:0/1769327633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557802029b40 0x557802028090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe249b3b640 1 -- 172.21.15.152:0/1769327633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578020285d0 msgr2=0x557802028a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe249b3b640 1 --2- 172.21.15.152:0/1769327633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578020285d0 0x557802028a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.553+0000 7fe249b3b640 1 -- 172.21.15.152:0/1769327633 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557801f1a3a0 con 0x557802033370 2024-09-11T22:58:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.554+0000 7fe249b3b640 1 --2- 172.21.15.152:0/1769327633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x557802035760 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe23800bb30 tx=0x7fe23800f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681d6ec52832c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.554+0000 7fe24ab3d640 1 --2- 172.21.15.152:0/4087522280 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe2300a2cf0 0x7fe2300a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe23c004af0 tx=0x7fe23c0095c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.554+0000 7fe234ff9640 1 -- 172.21.15.152:0/1769327633 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238015070 con 0x557802033370 2024-09-11T22:58:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.554+0000 7fe234ff9640 1 -- 172.21.15.152:0/1769327633 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe23800fd90 con 0x557802033370 2024-09-11T22:58:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.554+0000 7fe234ff9640 1 -- 172.21.15.152:0/1769327633 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238011440 con 0x557802033370 2024-09-11T22:58:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 -- 172.21.15.152:0/1769327633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 msgr2=0x557802035760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 --2- 172.21.15.152:0/1769327633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x557802035760 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe23800bb30 tx=0x7fe23800f360 comp rx=0 tx=0).stop 2024-09-11T22:58:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 -- 172.21.15.152:0/1769327633 shutdown_connections 2024-09-11T22:58:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 --2- 172.21.15.152:0/1769327633 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x557802035760 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 --2- 172.21.15.152:0/1769327633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578020285d0 0x557802028a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 --2- 172.21.15.152:0/1769327633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557802029b40 0x557802028090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 -- 172.21.15.152:0/1769327633 >> 172.21.15.152:0/1769327633 conn(0x557801ff1310 msgr2=0x557802024c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 -- 172.21.15.152:0/1769327633 shutdown_connections 2024-09-11T22:58:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 -- 172.21.15.152:0/1769327633 wait complete. 2024-09-11T22:58:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 -- start start 2024-09-11T22:58:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 Processor -- start 2024-09-11T22:58:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578020285d0 0x557802032ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557802029b40 0x55780202ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x5578020304f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557802036070 con 0x557802033370 2024-09-11T22:58:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557801f49060 con 0x557802029b40 2024-09-11T22:58:37.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5578020361f0 con 0x5578020285d0 2024-09-11T22:58:37.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.555+0000 7fe24e979740 10 client.0 _ll_register_callbacks cb 0x557801c9c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:37.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe24ab3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x5578020304f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe24ab3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x5578020304f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41846/0 (socket says 172.21.15.152:41846) 2024-09-11T22:58:37.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe24ab3d640 1 -- 172.21.15.152:0/3485507690 learned_addr learned my addr 172.21.15.152:0/3485507690 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:37.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe24a33c640 1 --2- 172.21.15.152:0/3485507690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578020285d0 0x557802032ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe249b3b640 1 --2- 172.21.15.152:0/3485507690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557802029b40 0x55780202ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe24ab3d640 1 -- 172.21.15.152:0/3485507690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578020285d0 msgr2=0x557802032ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:37.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe24ab3d640 1 --2- 172.21.15.152:0/3485507690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578020285d0 0x557802032ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe24ab3d640 1 -- 172.21.15.152:0/3485507690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557802029b40 msgr2=0x55780202ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:37.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe24ab3d640 1 --2- 172.21.15.152:0/3485507690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557802029b40 0x55780202ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe24ab3d640 1 -- 172.21.15.152:0/3485507690 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557801cff3c0 con 0x557802033370 2024-09-11T22:58:37.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe24a33c640 1 --2- 172.21.15.152:0/3485507690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578020285d0 0x557802032ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:37.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe249b3b640 1 --2- 172.21.15.152:0/3485507690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557802029b40 0x55780202ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:37.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe24ab3d640 1 --2- 172.21.15.152:0/3485507690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x5578020304f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009660 tx=0x7fe23c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3acfc2f95108d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:37.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe22dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:37.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe22dffb640 1 -- 172.21.15.152:0/3485507690 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c02fd90 con 0x557802033370 2024-09-11T22:58:37.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe22dffb640 1 -- 172.21.15.152:0/3485507690 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fe23c0305c0 con 0x557802033370 2024-09-11T22:58:37.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe24e979740 1 -- 172.21.15.152:0/3485507690 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe23800fd90 con 0x557802033370 2024-09-11T22:58:37.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe22dffb640 1 -- 172.21.15.152:0/3485507690 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe23c0307c0 con 0x557802033370 2024-09-11T22:58:37.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe24e979740 10 client.16950 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:37.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.556+0000 7fe24e979740 1 -- 172.21.15.152:0/3485507690 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557802041a80 con 0x557802033370 2024-09-11T22:58:37.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.557+0000 7fe22dffb640 1 -- 172.21.15.152:0/3485507690 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe23800fd90 con 0x557802033370 2024-09-11T22:58:37.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.557+0000 7fe22dffb640 1 -- 172.21.15.152:0/3485507690 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe23c047070 con 0x557802033370 2024-09-11T22:58:37.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.557+0000 7fe24e979740 10 client.16950 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:37.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.557+0000 7fe24e979740 10 client.16950 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:58:37.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.557+0000 7fe24e979740 20 client.16950 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:37.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.557+0000 7fe24e979740 1 --2- 172.21.15.152:0/3485507690 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x557802042e40 0x5578020452e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.557+0000 7fe24e979740 4 client.16950 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:58:37.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.557+0000 7fe24e979740 1 -- 172.21.15.152:0/3485507690 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557801f9b860 con 0x557802042e40 2024-09-11T22:58:37.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.557+0000 7fe24a33c640 1 --2- 172.21.15.152:0/3485507690 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x557802042e40 0x5578020452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.558+0000 7fe22dffb640 10 client.16950 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:58:37.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.558+0000 7fe24a33c640 1 --2- 172.21.15.152:0/3485507690 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x557802042e40 0x5578020452e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55ebb242f25b2f19 server_cookie=ed2efe161b9e79c9 in_seq=0 out_seq=0 2024-09-11T22:58:37.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.558+0000 7fe22dffb640 1 -- 172.21.15.152:0/3485507690 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557801f9b860 con 0x557802042e40 2024-09-11T22:58:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.558+0000 7fe22dffb640 10 client.16950 handle_command_reply: tid=0 2024-09-11T22:58:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.558+0000 7fe24e979740 1 client.16950 shutdown 2024-09-11T22:58:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.558+0000 7fe24e979740 1 -- 172.21.15.152:0/3485507690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 msgr2=0x5578020304f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.559+0000 7fe24e979740 1 --2- 172.21.15.152:0/3485507690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x5578020304f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009660 tx=0x7fe23c02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.559+0000 7fe24e979740 1 -- 172.21.15.152:0/3485507690 shutdown_connections 2024-09-11T22:58:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.559+0000 7fe24e979740 1 --2- 172.21.15.152:0/3485507690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x5578020304f0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.559+0000 7fe24e979740 1 --2- 172.21.15.152:0/3485507690 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x557802042e40 0x5578020452e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.559+0000 7fe24e979740 1 --2- 172.21.15.152:0/3485507690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557802029b40 0x55780202ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.559+0000 7fe24e979740 1 --2- 172.21.15.152:0/3485507690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5578020285d0 0x557802032ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.559+0000 7fe24e979740 1 -- 172.21.15.152:0/3485507690 >> 172.21.15.152:0/3485507690 conn(0x557801ff1310 msgr2=0x55780202a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.559+0000 7fe22dffb640 0 client.16950 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:58:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.559+0000 7fe24e979740 1 -- 172.21.15.152:0/3485507690 shutdown_connections 2024-09-11T22:58:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.559+0000 7fe24e979740 1 -- 172.21.15.152:0/3485507690 wait complete. 2024-09-11T22:58:37.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.559+0000 7fe24e979740 20 client.16950 trim_cache size 0 max 16384 2024-09-11T22:58:37.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.571+0000 7fe24e979740 1 Processor -- start 2024-09-11T22:58:37.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.571+0000 7fe24e979740 1 -- start start 2024-09-11T22:58:37.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe24e979740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578020285d0 0x55780203f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe24e979740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557802029b40 0x55780202b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe24e979740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557802033370 0x55780202b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe24e979740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441af970 con 0x5578020285d0 2024-09-11T22:58:37.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe24e979740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441b14b0 con 0x557802033370 2024-09-11T22:58:37.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe24e979740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441b0040 con 0x557802029b40 2024-09-11T22:58:37.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe24a33c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578020285d0 0x55780203f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe24a33c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578020285d0 0x55780203f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41848/0 (socket says 172.21.15.152:41848) 2024-09-11T22:58:37.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe24a33c640 1 -- 172.21.15.152:0/1726700226 learned_addr learned my addr 172.21.15.152:0/1726700226 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:37.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe249b3b640 1 --2- 172.21.15.152:0/1726700226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557802029b40 0x55780202b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe24ab3d640 1 --2- 172.21.15.152:0/1726700226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557802033370 0x55780202b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe24a33c640 1 -- 172.21.15.152:0/1726700226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557802029b40 msgr2=0x55780202b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:37.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe24a33c640 1 --2- 172.21.15.152:0/1726700226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557802029b40 0x55780202b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe24a33c640 1 -- 172.21.15.152:0/1726700226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557802033370 msgr2=0x55780202b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:37.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe24a33c640 1 --2- 172.21.15.152:0/1726700226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557802033370 0x55780202b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe24a33c640 1 -- 172.21.15.152:0/1726700226 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55780202bba0 con 0x5578020285d0 2024-09-11T22:58:37.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe24a33c640 1 --2- 172.21.15.152:0/1726700226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578020285d0 0x55780203f310 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe24000bfa0 tx=0x7fe2400168c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692c679b9fdc3993 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:37.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe22ffff640 1 -- 172.21.15.152:0/1726700226 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240020040 con 0x5578020285d0 2024-09-11T22:58:37.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe22ffff640 1 -- 172.21.15.152:0/1726700226 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2400071c0 con 0x5578020285d0 2024-09-11T22:58:37.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.572+0000 7fe22ffff640 1 -- 172.21.15.152:0/1726700226 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240099a00 con 0x5578020285d0 2024-09-11T22:58:37.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.573+0000 7fe24e979740 1 -- 172.21.15.152:0/1726700226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578020285d0 msgr2=0x55780203f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:37.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.573+0000 7fe24e979740 1 --2- 172.21.15.152:0/1726700226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578020285d0 0x55780203f310 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe24000bfa0 tx=0x7fe2400168c0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.573+0000 7fe24e979740 1 -- 172.21.15.152:0/1726700226 shutdown_connections 2024-09-11T22:58:37.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.573+0000 7fe24e979740 1 --2- 172.21.15.152:0/1726700226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557802033370 0x55780202b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.573+0000 7fe24e979740 1 --2- 172.21.15.152:0/1726700226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557802029b40 0x55780202b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.573+0000 7fe24e979740 1 --2- 172.21.15.152:0/1726700226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5578020285d0 0x55780203f310 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.573+0000 7fe24e979740 1 -- 172.21.15.152:0/1726700226 >> 172.21.15.152:0/1726700226 conn(0x557801ff1310 msgr2=0x55780202ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:37.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.573+0000 7fe24e979740 1 -- 172.21.15.152:0/1726700226 shutdown_connections 2024-09-11T22:58:37.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.573+0000 7fe24e979740 1 -- 172.21.15.152:0/1726700226 wait complete. 2024-09-11T22:58:37.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.573+0000 7fe24e979740 1 -- start start 2024-09-11T22:58:37.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.573+0000 7fe24e979740 1 Processor -- start 2024-09-11T22:58:37.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24e979740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578020285d0 0x55780202cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24e979740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557802029b40 0x55780202d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24e979740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x55780202d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24e979740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441b0de0 con 0x557802033370 2024-09-11T22:58:37.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24e979740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441b0710 con 0x5578020285d0 2024-09-11T22:58:37.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24e979740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe2441ad930 con 0x557802029b40 2024-09-11T22:58:37.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24e979740 10 client.0 _ll_register_callbacks cb 0x7fe23c031830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:37.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24ab3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x55780202d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24a33c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578020285d0 0x55780202cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24ab3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x55780202d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:41862/0 (socket says 172.21.15.152:41862) 2024-09-11T22:58:37.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24ab3d640 1 -- 172.21.15.152:0/717583227 learned_addr learned my addr 172.21.15.152:0/717583227 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:37.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe249b3b640 1 --2- 172.21.15.152:0/717583227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557802029b40 0x55780202d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24ab3d640 1 -- 172.21.15.152:0/717583227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557802029b40 msgr2=0x55780202d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:37.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24ab3d640 1 --2- 172.21.15.152:0/717583227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557802029b40 0x55780202d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24ab3d640 1 -- 172.21.15.152:0/717583227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578020285d0 msgr2=0x55780202cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:37.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24ab3d640 1 --2- 172.21.15.152:0/717583227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578020285d0 0x55780202cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24ab3d640 1 -- 172.21.15.152:0/717583227 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578020acb80 con 0x557802033370 2024-09-11T22:58:37.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24ab3d640 1 --2- 172.21.15.152:0/717583227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x55780202d9f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe23c031d00 tx=0x7fe23c054f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffd0fb87e35cd8c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:37.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe22dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:37.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe22dffb640 1 -- 172.21.15.152:0/717583227 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23c003970 con 0x557802033370 2024-09-11T22:58:37.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe22dffb640 1 -- 172.21.15.152:0/717583227 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fe23c003b10 con 0x557802033370 2024-09-11T22:58:37.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.574+0000 7fe24e979740 1 -- 172.21.15.152:0/717583227 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2400071c0 con 0x557802033370 2024-09-11T22:58:37.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.575+0000 7fe22dffb640 1 -- 172.21.15.152:0/717583227 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe23c003d50 con 0x557802033370 2024-09-11T22:58:37.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.575+0000 7fe24e979740 10 client.16956 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:37.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.575+0000 7fe24e979740 1 -- 172.21.15.152:0/717583227 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578020ade40 con 0x557802033370 2024-09-11T22:58:37.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.575+0000 7fe22dffb640 1 -- 172.21.15.152:0/717583227 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2400071c0 con 0x557802033370 2024-09-11T22:58:37.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.575+0000 7fe22dffb640 1 -- 172.21.15.152:0/717583227 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe23c047070 con 0x557802033370 2024-09-11T22:58:37.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.575+0000 7fe24e979740 10 client.16956 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:37.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.575+0000 7fe24e979740 10 client.16956 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:58:37.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.575+0000 7fe24e979740 20 client.16956 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:37.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.575+0000 7fe24e979740 1 --2- 172.21.15.152:0/717583227 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5578020ae500 0x5578020b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:37.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.575+0000 7fe24e979740 4 client.16956 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:58:37.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.575+0000 7fe24e979740 1 -- 172.21.15.152:0/717583227 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557801f95d70 con 0x5578020ae500 2024-09-11T22:58:37.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.575+0000 7fe24a33c640 1 --2- 172.21.15.152:0/717583227 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5578020ae500 0x5578020b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:37.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe24a33c640 1 --2- 172.21.15.152:0/717583227 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5578020ae500 0x5578020b08f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c7a8d7ed1d500ecc server_cookie=c86ff757bbf2a3e1 in_seq=0 out_seq=0 2024-09-11T22:58:37.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe22dffb640 10 client.16956 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:58:37.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe22dffb640 1 -- 172.21.15.152:0/717583227 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557801f95d70 con 0x5578020ae500 2024-09-11T22:58:37.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe22dffb640 10 client.16956 handle_command_reply: tid=0 2024-09-11T22:58:37.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe24e979740 1 client.16956 shutdown 2024-09-11T22:58:37.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe24e979740 1 -- 172.21.15.152:0/717583227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 msgr2=0x55780202d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:37.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe24e979740 1 --2- 172.21.15.152:0/717583227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x55780202d9f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe23c031d00 tx=0x7fe23c054f30 comp rx=0 tx=0).stop 2024-09-11T22:58:37.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe24e979740 1 -- 172.21.15.152:0/717583227 shutdown_connections 2024-09-11T22:58:37.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe24e979740 1 --2- 172.21.15.152:0/717583227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557802033370 0x55780202d9f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe24e979740 1 --2- 172.21.15.152:0/717583227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557802029b40 0x55780202d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe24e979740 1 --2- 172.21.15.152:0/717583227 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5578020ae500 0x5578020b08f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe22dffb640 0 client.16956 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:58:37.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe24e979740 1 --2- 172.21.15.152:0/717583227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5578020285d0 0x55780202cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe24e979740 1 -- 172.21.15.152:0/717583227 >> 172.21.15.152:0/717583227 conn(0x557801ff1310 msgr2=0x5578020317f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:37.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe24e979740 1 -- 172.21.15.152:0/717583227 shutdown_connections 2024-09-11T22:58:37.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe24e979740 1 -- 172.21.15.152:0/717583227 wait complete. 2024-09-11T22:58:37.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.576+0000 7fe24e979740 20 client.16956 trim_cache size 0 max 16384 2024-09-11T22:58:37.584 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:58:37.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.577+0000 7fe24c5a0640 1 -- 172.21.15.152:0/4087522280 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe2300a2cf0 msgr2=0x7fe2300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:37.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.577+0000 7fe24c5a0640 1 --2- 172.21.15.152:0/4087522280 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe2300a2cf0 0x7fe2300a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe23c004af0 tx=0x7fe23c0095c0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.577+0000 7fe24c5a0640 1 -- 172.21.15.152:0/4087522280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe244101de0 msgr2=0x7fe2440744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:37.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.577+0000 7fe24c5a0640 1 --2- 172.21.15.152:0/4087522280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe244101de0 0x7fe2440744f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe24000d950 tx=0x7fe24000de10 comp rx=0 tx=0).stop 2024-09-11T22:58:37.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.577+0000 7fe24c5a0640 1 -- 172.21.15.152:0/4087522280 shutdown_connections 2024-09-11T22:58:37.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.577+0000 7fe24c5a0640 1 --2- 172.21.15.152:0/4087522280 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe2300a2cf0 0x7fe2300a51b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.577+0000 7fe24c5a0640 1 --2- 172.21.15.152:0/4087522280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe244106f10 0x7fe244074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.577+0000 7fe24c5a0640 1 --2- 172.21.15.152:0/4087522280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe244102790 0x7fe244074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.577+0000 7fe24c5a0640 1 --2- 172.21.15.152:0/4087522280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe244101de0 0x7fe2440744f0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:37.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.577+0000 7fe24c5a0640 1 -- 172.21.15.152:0/4087522280 >> 172.21.15.152:0/4087522280 conn(0x7fe2440fd8c0 msgr2=0x7fe2440fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:37.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.577+0000 7fe24c5a0640 1 -- 172.21.15.152:0/4087522280 shutdown_connections 2024-09-11T22:58:37.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:37.577+0000 7fe24c5a0640 1 -- 172.21.15.152:0/4087522280 wait complete. 2024-09-11T22:58:38.461 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:58:38.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:38 smithi191 ceph-mon[35211]: pgmap v175: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:38.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:38 smithi204 ceph-mon[35454]: pgmap v175: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:38.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:38 smithi152 ceph-mon[31181]: pgmap v175: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:38.707 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:39.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.222+0000 7fbe2fdcf640 1 -- 172.21.15.152:0/1268935072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe28100a80 msgr2=0x7fbe28100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:39.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.222+0000 7fbe2fdcf640 1 --2- 172.21.15.152:0/1268935072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe28100a80 0x7fbe28100ee0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009930 tx=0x7fbe1802a420 comp rx=0 tx=0).stop 2024-09-11T22:58:39.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.222+0000 7fbe2fdcf640 1 -- 172.21.15.152:0/1268935072 shutdown_connections 2024-09-11T22:58:39.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.222+0000 7fbe2fdcf640 1 --2- 172.21.15.152:0/1268935072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe28101420 0x7fbe2810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.222+0000 7fbe2fdcf640 1 --2- 172.21.15.152:0/1268935072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe28100a80 0x7fbe28100ee0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.222+0000 7fbe2fdcf640 1 --2- 172.21.15.152:0/1268935072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe28106a80 0x7fbe28106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.222+0000 7fbe2fdcf640 1 -- 172.21.15.152:0/1268935072 >> 172.21.15.152:0/1268935072 conn(0x7fbe280fc7a0 msgr2=0x7fbe280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:39.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.222+0000 7fbe2fdcf640 1 -- 172.21.15.152:0/1268935072 shutdown_connections 2024-09-11T22:58:39.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.222+0000 7fbe2fdcf640 1 -- 172.21.15.152:0/1268935072 wait complete. 2024-09-11T22:58:39.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2fdcf640 1 Processor -- start 2024-09-11T22:58:39.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2fdcf640 1 -- start start 2024-09-11T22:58:39.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2fdcf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe28100a80 0x7fbe281970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2fdcf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe28101420 0x7fbe28197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2fdcf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe28106a80 0x7fbe28197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2fdcf640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28110ea0 con 0x7fbe28101420 2024-09-11T22:58:39.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2fdcf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28110d20 con 0x7fbe28106a80 2024-09-11T22:58:39.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2fdcf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28111020 con 0x7fbe28100a80 2024-09-11T22:58:39.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2d36a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe28101420 0x7fbe28197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2d36a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe28101420 0x7fbe28197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53156/0 (socket says 172.21.15.152:53156) 2024-09-11T22:58:39.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2d36a640 1 -- 172.21.15.152:0/740158547 learned_addr learned my addr 172.21.15.152:0/740158547 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:39.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2db6b640 1 --2- 172.21.15.152:0/740158547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe28100a80 0x7fbe281970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2e36c640 1 --2- 172.21.15.152:0/740158547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe28106a80 0x7fbe28197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2d36a640 1 -- 172.21.15.152:0/740158547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe28100a80 msgr2=0x7fbe281970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:39.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2d36a640 1 --2- 172.21.15.152:0/740158547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe28100a80 0x7fbe281970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2d36a640 1 -- 172.21.15.152:0/740158547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe28106a80 msgr2=0x7fbe28197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:39.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2d36a640 1 --2- 172.21.15.152:0/740158547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe28106a80 0x7fbe28197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.223+0000 7fbe2d36a640 1 -- 172.21.15.152:0/740158547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe281a6930 con 0x7fbe28101420 2024-09-11T22:58:39.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.224+0000 7fbe2d36a640 1 --2- 172.21.15.152:0/740158547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe28101420 0x7fbe28197600 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009a50 tx=0x7fbe18004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:39.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.224+0000 7fbe1effd640 1 -- 172.21.15.152:0/740158547 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1803f030 con 0x7fbe28101420 2024-09-11T22:58:39.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.224+0000 7fbe1effd640 1 -- 172.21.15.152:0/740158547 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe18003ce0 con 0x7fbe28101420 2024-09-11T22:58:39.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.224+0000 7fbe1effd640 1 -- 172.21.15.152:0/740158547 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe18035070 con 0x7fbe28101420 2024-09-11T22:58:39.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.224+0000 7fbe2fdcf640 1 -- 172.21.15.152:0/740158547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe281a6bc0 con 0x7fbe28101420 2024-09-11T22:58:39.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.224+0000 7fbe2fdcf640 1 -- 172.21.15.152:0/740158547 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe281a7050 con 0x7fbe28101420 2024-09-11T22:58:39.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.226+0000 7fbe1effd640 1 -- 172.21.15.152:0/740158547 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fbe18042030 con 0x7fbe28101420 2024-09-11T22:58:39.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.226+0000 7fbe321a8740 1 Processor -- start 2024-09-11T22:58:39.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe321a8740 1 -- start start 2024-09-11T22:58:39.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe321a8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e51b40 0x557d26e50090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe321a8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e505d0 0x557d26e50a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe321a8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e5b370 0x557d26e5d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe321a8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557d26db62f0 con 0x557d26e51b40 2024-09-11T22:58:39.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe321a8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557d26d5dcf0 con 0x557d26e5b370 2024-09-11T22:58:39.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe321a8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557d26db22b0 con 0x557d26e505d0 2024-09-11T22:58:39.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe2e36c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e51b40 0x557d26e50090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe2e36c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e51b40 0x557d26e50090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53158/0 (socket says 172.21.15.152:53158) 2024-09-11T22:58:39.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe2e36c640 1 -- 172.21.15.152:0/3210770512 learned_addr learned my addr 172.21.15.152:0/3210770512 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:39.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe2db6b640 1 --2- 172.21.15.152:0/3210770512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e505d0 0x557d26e50a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe1effd640 1 --2- 172.21.15.152:0/740158547 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbe140a2cf0 0x7fbe140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe2d36a640 1 --2- 172.21.15.152:0/3210770512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e5b370 0x557d26e5d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe2e36c640 1 -- 172.21.15.152:0/3210770512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e505d0 msgr2=0x557d26e50a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:39.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe2e36c640 1 --2- 172.21.15.152:0/3210770512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e505d0 0x557d26e50a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe2e36c640 1 -- 172.21.15.152:0/3210770512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e5b370 msgr2=0x557d26e5d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:39.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe2e36c640 1 --2- 172.21.15.152:0/3210770512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e5b370 0x557d26e5d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.227+0000 7fbe2e36c640 1 -- 172.21.15.152:0/3210770512 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d26d423a0 con 0x557d26e51b40 2024-09-11T22:58:39.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.228+0000 7fbe2e36c640 1 --2- 172.21.15.152:0/740158547 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbe140a2cf0 0x7fbe140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.228+0000 7fbe1effd640 1 -- 172.21.15.152:0/740158547 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fbe180efa70 con 0x7fbe28101420 2024-09-11T22:58:39.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.228+0000 7fbe2e36c640 1 --2- 172.21.15.152:0/3210770512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e51b40 0x557d26e50090 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbe20007fd0 tx=0x7fbe2000e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2831e4d32865df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:39.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.228+0000 7fbe13fff640 1 -- 172.21.15.152:0/3210770512 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20022070 con 0x557d26e51b40 2024-09-11T22:58:39.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.228+0000 7fbe13fff640 1 -- 172.21.15.152:0/3210770512 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe20011c30 con 0x557d26e51b40 2024-09-11T22:58:39.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.228+0000 7fbe13fff640 1 -- 172.21.15.152:0/3210770512 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2001a400 con 0x557d26e51b40 2024-09-11T22:58:39.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.229+0000 7fbe321a8740 1 -- 172.21.15.152:0/3210770512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e51b40 msgr2=0x557d26e50090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:39.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.229+0000 7fbe321a8740 1 --2- 172.21.15.152:0/3210770512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e51b40 0x557d26e50090 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbe20007fd0 tx=0x7fbe2000e7e0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.229+0000 7fbe2e36c640 1 --2- 172.21.15.152:0/740158547 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbe140a2cf0 0x7fbe140a51b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbe2001f000 tx=0x7fbe20020000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:39.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.229+0000 7fbe321a8740 1 -- 172.21.15.152:0/3210770512 shutdown_connections 2024-09-11T22:58:39.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.229+0000 7fbe321a8740 1 --2- 172.21.15.152:0/3210770512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e5b370 0x557d26e5d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.229+0000 7fbe321a8740 1 --2- 172.21.15.152:0/3210770512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e505d0 0x557d26e50a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.229+0000 7fbe321a8740 1 --2- 172.21.15.152:0/3210770512 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e51b40 0x557d26e50090 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.229+0000 7fbe321a8740 1 -- 172.21.15.152:0/3210770512 >> 172.21.15.152:0/3210770512 conn(0x557d26e19310 msgr2=0x557d26e4cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:39.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.229+0000 7fbe321a8740 1 -- 172.21.15.152:0/3210770512 shutdown_connections 2024-09-11T22:58:39.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.229+0000 7fbe321a8740 1 -- 172.21.15.152:0/3210770512 wait complete. 2024-09-11T22:58:39.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.229+0000 7fbe321a8740 1 -- start start 2024-09-11T22:58:39.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.229+0000 7fbe321a8740 1 Processor -- start 2024-09-11T22:58:39.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe321a8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e5ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe321a8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e51b40 0x557d26e57fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe321a8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e5b370 0x557d26e585d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe321a8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557d26e5e070 con 0x557d26e505d0 2024-09-11T22:58:39.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe321a8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557d26d71060 con 0x557d26e51b40 2024-09-11T22:58:39.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe321a8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x557d26e5e1f0 con 0x557d26e5b370 2024-09-11T22:58:39.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe321a8740 10 client.0 _ll_register_callbacks cb 0x557d26ac48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:39.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe2db6b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e5ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe2db6b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e5ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53168/0 (socket says 172.21.15.152:53168) 2024-09-11T22:58:39.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe2db6b640 1 -- 172.21.15.152:0/1937917721 learned_addr learned my addr 172.21.15.152:0/1937917721 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:39.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe2e36c640 1 --2- 172.21.15.152:0/1937917721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e5b370 0x557d26e585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe2d36a640 1 --2- 172.21.15.152:0/1937917721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e51b40 0x557d26e57fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe2db6b640 1 -- 172.21.15.152:0/1937917721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e5b370 msgr2=0x557d26e585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:39.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe2db6b640 1 --2- 172.21.15.152:0/1937917721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e5b370 0x557d26e585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe2db6b640 1 -- 172.21.15.152:0/1937917721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e51b40 msgr2=0x557d26e57fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:39.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe2db6b640 1 --2- 172.21.15.152:0/1937917721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e51b40 0x557d26e57fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe2db6b640 1 -- 172.21.15.152:0/1937917721 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d26b273c0 con 0x557d26e505d0 2024-09-11T22:58:39.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe2e36c640 1 --2- 172.21.15.152:0/1937917721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e5b370 0x557d26e585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:39.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe2d36a640 1 --2- 172.21.15.152:0/1937917721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e51b40 0x557d26e57fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:39.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe2db6b640 1 --2- 172.21.15.152:0/1937917721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e5ad30 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbe240104d0 tx=0x7fbe24010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17f1ed25b87639b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:39.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe117fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:39.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe117fa640 1 -- 172.21.15.152:0/1937917721 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2401a020 con 0x557d26e505d0 2024-09-11T22:58:39.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe321a8740 1 -- 172.21.15.152:0/1937917721 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe20011c30 con 0x557d26e505d0 2024-09-11T22:58:39.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe117fa640 1 -- 172.21.15.152:0/1937917721 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fbe24016690 con 0x557d26e505d0 2024-09-11T22:58:39.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe117fa640 1 -- 172.21.15.152:0/1937917721 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbe24025050 con 0x557d26e505d0 2024-09-11T22:58:39.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe321a8740 10 client.16968 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:39.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.230+0000 7fbe321a8740 1 -- 172.21.15.152:0/1937917721 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d26e69b30 con 0x557d26e505d0 2024-09-11T22:58:39.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.231+0000 7fbe117fa640 1 -- 172.21.15.152:0/1937917721 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe20011c30 con 0x557d26e505d0 2024-09-11T22:58:39.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.231+0000 7fbe117fa640 1 -- 172.21.15.152:0/1937917721 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbe24014ae0 con 0x557d26e505d0 2024-09-11T22:58:39.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.231+0000 7fbe321a8740 10 client.16968 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:39.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.231+0000 7fbe321a8740 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:58:39.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.231+0000 7fbe321a8740 20 client.16968 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:39.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.231+0000 7fbe321a8740 1 --2- 172.21.15.152:0/1937917721 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557d26e6aff0 0x557d26e6d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.231+0000 7fbe321a8740 4 client.16968 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:58:39.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.231+0000 7fbe321a8740 1 -- 172.21.15.152:0/1937917721 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d26dc3860 con 0x557d26e6aff0 2024-09-11T22:58:39.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.231+0000 7fbe2d36a640 1 --2- 172.21.15.152:0/1937917721 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557d26e6aff0 0x557d26e6d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.232+0000 7fbe117fa640 10 client.16968 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:39.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.232+0000 7fbe2d36a640 1 --2- 172.21.15.152:0/1937917721 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557d26e6aff0 0x557d26e6d450 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43634ab6c9f047f9 server_cookie=fc5306d46c828be2 in_seq=0 out_seq=0 2024-09-11T22:58:39.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.232+0000 7fbe117fa640 1 -- 172.21.15.152:0/1937917721 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d26dc3860 con 0x557d26e6aff0 2024-09-11T22:58:39.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.232+0000 7fbe117fa640 10 client.16968 handle_command_reply: tid=0 2024-09-11T22:58:39.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.232+0000 7fbe321a8740 1 client.16968 shutdown 2024-09-11T22:58:39.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.232+0000 7fbe321a8740 1 -- 172.21.15.152:0/1937917721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 msgr2=0x557d26e5ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:39.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.232+0000 7fbe321a8740 1 --2- 172.21.15.152:0/1937917721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e5ad30 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbe240104d0 tx=0x7fbe24010990 comp rx=0 tx=0).stop 2024-09-11T22:58:39.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.232+0000 7fbe321a8740 1 -- 172.21.15.152:0/1937917721 shutdown_connections 2024-09-11T22:58:39.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.232+0000 7fbe321a8740 1 --2- 172.21.15.152:0/1937917721 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e5b370 0x557d26e585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.232+0000 7fbe321a8740 1 --2- 172.21.15.152:0/1937917721 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557d26e6aff0 0x557d26e6d450 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.232+0000 7fbe321a8740 1 --2- 172.21.15.152:0/1937917721 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e51b40 0x557d26e57fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.232+0000 7fbe321a8740 1 --2- 172.21.15.152:0/1937917721 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e5ad30 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.232+0000 7fbe321a8740 1 -- 172.21.15.152:0/1937917721 >> 172.21.15.152:0/1937917721 conn(0x557d26e19310 msgr2=0x557d26e52960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:39.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.232+0000 7fbe117fa640 0 client.16968 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:39.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.233+0000 7fbe321a8740 1 -- 172.21.15.152:0/1937917721 shutdown_connections 2024-09-11T22:58:39.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.233+0000 7fbe321a8740 1 -- 172.21.15.152:0/1937917721 wait complete. 2024-09-11T22:58:39.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.233+0000 7fbe321a8740 20 client.16968 trim_cache size 0 max 16384 2024-09-11T22:58:39.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.245+0000 7fbe321a8740 1 Processor -- start 2024-09-11T22:58:39.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.245+0000 7fbe321a8740 1 -- start start 2024-09-11T22:58:39.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.245+0000 7fbe321a8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e67310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.277 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:58:39.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.245+0000 7fbe321a8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e51b40 0x557d26e578a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.245+0000 7fbe321a8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e5b370 0x557d26e72d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.245+0000 7fbe321a8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbe281af8f0 con 0x557d26e505d0 2024-09-11T22:58:39.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.245+0000 7fbe321a8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbe281b1430 con 0x557d26e51b40 2024-09-11T22:58:39.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.246+0000 7fbe321a8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe281affc0 con 0x557d26e5b370 2024-09-11T22:58:39.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.246+0000 7fbe2db6b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e67310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.246+0000 7fbe2db6b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e67310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53184/0 (socket says 172.21.15.152:53184) 2024-09-11T22:58:39.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.246+0000 7fbe2db6b640 1 -- 172.21.15.152:0/2974683622 learned_addr learned my addr 172.21.15.152:0/2974683622 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:39.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.246+0000 7fbe2e36c640 1 --2- 172.21.15.152:0/2974683622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e5b370 0x557d26e72d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.246+0000 7fbe2d36a640 1 --2- 172.21.15.152:0/2974683622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e51b40 0x557d26e578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.246+0000 7fbe2db6b640 1 -- 172.21.15.152:0/2974683622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e5b370 msgr2=0x557d26e72d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:39.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.246+0000 7fbe2db6b640 1 --2- 172.21.15.152:0/2974683622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e5b370 0x557d26e72d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.246+0000 7fbe2db6b640 1 -- 172.21.15.152:0/2974683622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e51b40 msgr2=0x557d26e578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:39.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.246+0000 7fbe2db6b640 1 --2- 172.21.15.152:0/2974683622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e51b40 0x557d26e578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.246+0000 7fbe2db6b640 1 -- 172.21.15.152:0/2974683622 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d26e4d960 con 0x557d26e505d0 2024-09-11T22:58:39.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.246+0000 7fbe2d36a640 1 --2- 172.21.15.152:0/2974683622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e51b40 0x557d26e578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:39.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.246+0000 7fbe2db6b640 1 --2- 172.21.15.152:0/2974683622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e67310 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbe24004810 tx=0x7fbe2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17710aa59f2124b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:39.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.246+0000 7fbe137fe640 1 -- 172.21.15.152:0/2974683622 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2401a070 con 0x557d26e505d0 2024-09-11T22:58:39.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.246+0000 7fbe137fe640 1 -- 172.21.15.152:0/2974683622 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe2400bd80 con 0x557d26e505d0 2024-09-11T22:58:39.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.246+0000 7fbe137fe640 1 -- 172.21.15.152:0/2974683622 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24000e60 con 0x557d26e505d0 2024-09-11T22:58:39.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.247+0000 7fbe321a8740 1 -- 172.21.15.152:0/2974683622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 msgr2=0x557d26e67310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:39.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.247+0000 7fbe321a8740 1 --2- 172.21.15.152:0/2974683622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e67310 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbe24004810 tx=0x7fbe2400b330 comp rx=0 tx=0).stop 2024-09-11T22:58:39.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.247+0000 7fbe321a8740 1 -- 172.21.15.152:0/2974683622 shutdown_connections 2024-09-11T22:58:39.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.247+0000 7fbe321a8740 1 --2- 172.21.15.152:0/2974683622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e5b370 0x557d26e72d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.247+0000 7fbe321a8740 1 --2- 172.21.15.152:0/2974683622 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e51b40 0x557d26e578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.247+0000 7fbe321a8740 1 --2- 172.21.15.152:0/2974683622 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e67310 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.247+0000 7fbe321a8740 1 -- 172.21.15.152:0/2974683622 >> 172.21.15.152:0/2974683622 conn(0x557d26e19310 msgr2=0x557d26e568b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:39.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.247+0000 7fbe321a8740 1 -- 172.21.15.152:0/2974683622 shutdown_connections 2024-09-11T22:58:39.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.247+0000 7fbe321a8740 1 -- 172.21.15.152:0/2974683622 wait complete. 2024-09-11T22:58:39.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.247+0000 7fbe321a8740 1 -- start start 2024-09-11T22:58:39.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.247+0000 7fbe321a8740 1 Processor -- start 2024-09-11T22:58:39.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe321a8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e63920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe321a8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e51b40 0x557d26e5ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe321a8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e5b370 0x557d26e5f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe321a8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbe281b0d60 con 0x557d26e505d0 2024-09-11T22:58:39.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe321a8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbe281b0690 con 0x557d26e5b370 2024-09-11T22:58:39.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe321a8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbe281ad8b0 con 0x557d26e51b40 2024-09-11T22:58:39.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe321a8740 10 client.0 _ll_register_callbacks cb 0x7fbe2400b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:39.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe2db6b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e63920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe2db6b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e63920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53200/0 (socket says 172.21.15.152:53200) 2024-09-11T22:58:39.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe2db6b640 1 -- 172.21.15.152:0/2782105010 learned_addr learned my addr 172.21.15.152:0/2782105010 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:39.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe2d36a640 1 --2- 172.21.15.152:0/2782105010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e51b40 0x557d26e5ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe2e36c640 1 --2- 172.21.15.152:0/2782105010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e5b370 0x557d26e5f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe2db6b640 1 -- 172.21.15.152:0/2782105010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e51b40 msgr2=0x557d26e5ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:39.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe2db6b640 1 --2- 172.21.15.152:0/2782105010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e51b40 0x557d26e5ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe2db6b640 1 -- 172.21.15.152:0/2782105010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e5b370 msgr2=0x557d26e5f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe2db6b640 1 --2- 172.21.15.152:0/2782105010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e5b370 0x557d26e5f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.248+0000 7fbe2db6b640 1 -- 172.21.15.152:0/2782105010 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d26e5f6d0 con 0x557d26e505d0 2024-09-11T22:58:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.249+0000 7fbe2db6b640 1 --2- 172.21.15.152:0/2782105010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e63920 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400b4c0 tx=0x7fbe24012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538755ec66183824 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.249+0000 7fbe117fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.249+0000 7fbe117fa640 1 -- 172.21.15.152:0/2782105010 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2401a040 con 0x557d26e505d0 2024-09-11T22:58:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.249+0000 7fbe117fa640 1 -- 172.21.15.152:0/2782105010 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fbe240168a0 con 0x557d26e505d0 2024-09-11T22:58:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.249+0000 7fbe321a8740 1 -- 172.21.15.152:0/2782105010 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe2400bd80 con 0x557d26e505d0 2024-09-11T22:58:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.249+0000 7fbe117fa640 1 -- 172.21.15.152:0/2782105010 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbe24016aa0 con 0x557d26e505d0 2024-09-11T22:58:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.249+0000 7fbe321a8740 10 client.16974 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.249+0000 7fbe321a8740 1 -- 172.21.15.152:0/2782105010 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d26ed5a90 con 0x557d26e505d0 2024-09-11T22:58:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.249+0000 7fbe117fa640 1 -- 172.21.15.152:0/2782105010 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe2400bd80 con 0x557d26e505d0 2024-09-11T22:58:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.250+0000 7fbe117fa640 1 -- 172.21.15.152:0/2782105010 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbe24032400 con 0x557d26e505d0 2024-09-11T22:58:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.250+0000 7fbe321a8740 10 client.16974 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:39.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.250+0000 7fbe321a8740 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:58:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.250+0000 7fbe321a8740 20 client.16974 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.250+0000 7fbe321a8740 1 --2- 172.21.15.152:0/2782105010 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557d26ed60f0 0x557d26ed84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.250+0000 7fbe321a8740 4 client.16974 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:58:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.250+0000 7fbe321a8740 1 -- 172.21.15.152:0/2782105010 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d26dbdd70 con 0x557d26ed60f0 2024-09-11T22:58:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.250+0000 7fbe2d36a640 1 --2- 172.21.15.152:0/2782105010 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557d26ed60f0 0x557d26ed84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.250+0000 7fbe117fa640 10 client.16974 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.250+0000 7fbe2d36a640 1 --2- 172.21.15.152:0/2782105010 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557d26ed60f0 0x557d26ed84e0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ad9a7bb03c0806e server_cookie=4f799f5bb5b53b11 in_seq=0 out_seq=0 2024-09-11T22:58:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.250+0000 7fbe117fa640 1 -- 172.21.15.152:0/2782105010 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d26dbdd70 con 0x557d26ed60f0 2024-09-11T22:58:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.250+0000 7fbe117fa640 10 client.16974 handle_command_reply: tid=0 2024-09-11T22:58:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.250+0000 7fbe321a8740 1 client.16974 shutdown 2024-09-11T22:58:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe321a8740 1 -- 172.21.15.152:0/2782105010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 msgr2=0x557d26e63920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe321a8740 1 --2- 172.21.15.152:0/2782105010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e63920 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400b4c0 tx=0x7fbe24012910 comp rx=0 tx=0).stop 2024-09-11T22:58:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe321a8740 1 -- 172.21.15.152:0/2782105010 shutdown_connections 2024-09-11T22:58:39.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe321a8740 1 --2- 172.21.15.152:0/2782105010 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x557d26ed60f0 0x557d26ed84e0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe321a8740 1 --2- 172.21.15.152:0/2782105010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557d26e5b370 0x557d26e5f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe321a8740 1 --2- 172.21.15.152:0/2782105010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557d26e51b40 0x557d26e5ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe117fa640 0 client.16974 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:39.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe321a8740 1 --2- 172.21.15.152:0/2782105010 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557d26e505d0 0x557d26e63920 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe321a8740 1 -- 172.21.15.152:0/2782105010 >> 172.21.15.152:0/2782105010 conn(0x557d26e19310 msgr2=0x557d26e52110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:39.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe321a8740 1 -- 172.21.15.152:0/2782105010 shutdown_connections 2024-09-11T22:58:39.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe321a8740 1 -- 172.21.15.152:0/2782105010 wait complete. 2024-09-11T22:58:39.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe321a8740 20 client.16974 trim_cache size 0 max 16384 2024-09-11T22:58:39.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe2fdcf640 1 -- 172.21.15.152:0/740158547 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbe140a2cf0 msgr2=0x7fbe140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:39.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe2fdcf640 1 --2- 172.21.15.152:0/740158547 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbe140a2cf0 0x7fbe140a51b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbe2001f000 tx=0x7fbe20020000 comp rx=0 tx=0).stop 2024-09-11T22:58:39.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe2fdcf640 1 -- 172.21.15.152:0/740158547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe28101420 msgr2=0x7fbe28197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:39.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe2fdcf640 1 --2- 172.21.15.152:0/740158547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe28101420 0x7fbe28197600 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fbe18009a50 tx=0x7fbe18004270 comp rx=0 tx=0).stop 2024-09-11T22:58:39.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe2fdcf640 1 -- 172.21.15.152:0/740158547 shutdown_connections 2024-09-11T22:58:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe2fdcf640 1 --2- 172.21.15.152:0/740158547 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbe140a2cf0 0x7fbe140a51b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe2fdcf640 1 --2- 172.21.15.152:0/740158547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe28106a80 0x7fbe28197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe2fdcf640 1 --2- 172.21.15.152:0/740158547 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe28101420 0x7fbe28197600 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe2fdcf640 1 --2- 172.21.15.152:0/740158547 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe28100a80 0x7fbe281970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe2fdcf640 1 -- 172.21.15.152:0/740158547 >> 172.21.15.152:0/740158547 conn(0x7fbe280fc7a0 msgr2=0x7fbe280fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe2fdcf640 1 -- 172.21.15.152:0/740158547 shutdown_connections 2024-09-11T22:58:39.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:39.251+0000 7fbe2fdcf640 1 -- 172.21.15.152:0/740158547 wait complete. 2024-09-11T22:58:40.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:40 smithi191 ceph-mon[35211]: pgmap v176: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:58:40.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:40 smithi204 ceph-mon[35454]: pgmap v176: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:58:40.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:40 smithi152 ceph-mon[31181]: pgmap v176: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s 2024-09-11T22:58:41.156 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:58:41.157 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:58:41.423 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:41.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.920+0000 7fc029258640 1 -- 172.21.15.152:0/1675065181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0241027a0 msgr2=0x7fc0241067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:41.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.920+0000 7fc029258640 1 --2- 172.21.15.152:0/1675065181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0241027a0 0x7fc0241067f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc018009900 tx=0x7fc01802a410 comp rx=0 tx=0).stop 2024-09-11T22:58:41.945 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.920+0000 7fc029258640 1 -- 172.21.15.152:0/1675065181 shutdown_connections 2024-09-11T22:58:41.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.920+0000 7fc029258640 1 --2- 172.21.15.152:0/1675065181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc024106f20 0x7fc02410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.920+0000 7fc029258640 1 --2- 172.21.15.152:0/1675065181 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0241027a0 0x7fc0241067f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.920+0000 7fc029258640 1 --2- 172.21.15.152:0/1675065181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc024101df0 0x7fc0241021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.920+0000 7fc029258640 1 -- 172.21.15.152:0/1675065181 >> 172.21.15.152:0/1675065181 conn(0x7fc0240fd8f0 msgr2=0x7fc0240ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:41.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.920+0000 7fc029258640 1 -- 172.21.15.152:0/1675065181 shutdown_connections 2024-09-11T22:58:41.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.920+0000 7fc029258640 1 -- 172.21.15.152:0/1675065181 wait complete. 2024-09-11T22:58:41.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.921+0000 7fc029258640 1 Processor -- start 2024-09-11T22:58:41.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.921+0000 7fc029258640 1 -- start start 2024-09-11T22:58:41.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.921+0000 7fc029258640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc024101df0 0x7fc024198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.921+0000 7fc029258640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0241027a0 0x7fc024198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.921+0000 7fc029258640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc024106f20 0x7fc02419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.921+0000 7fc029258640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc024111fa0 con 0x7fc0241027a0 2024-09-11T22:58:41.946 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.921+0000 7fc029258640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc024111e20 con 0x7fc024101df0 2024-09-11T22:58:41.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.921+0000 7fc029258640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc024112120 con 0x7fc024106f20 2024-09-11T22:58:41.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.921+0000 7fc02259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0241027a0 0x7fc024198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.921+0000 7fc02259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0241027a0 0x7fc024198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53218/0 (socket says 172.21.15.152:53218) 2024-09-11T22:58:41.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.921+0000 7fc02259c640 1 -- 172.21.15.152:0/3120523134 learned_addr learned my addr 172.21.15.152:0/3120523134 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:41.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.921+0000 7fc022d9d640 1 --2- 172.21.15.152:0/3120523134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc024101df0 0x7fc024198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.921+0000 7fc02359e640 1 --2- 172.21.15.152:0/3120523134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc024106f20 0x7fc02419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.922+0000 7fc02259c640 1 -- 172.21.15.152:0/3120523134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc024106f20 msgr2=0x7fc02419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:41.947 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.922+0000 7fc02259c640 1 --2- 172.21.15.152:0/3120523134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc024106f20 0x7fc02419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.922+0000 7fc02259c640 1 -- 172.21.15.152:0/3120523134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc024101df0 msgr2=0x7fc024198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:41.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.922+0000 7fc02259c640 1 --2- 172.21.15.152:0/3120523134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc024101df0 0x7fc024198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.922+0000 7fc02259c640 1 -- 172.21.15.152:0/3120523134 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02419d4d0 con 0x7fc0241027a0 2024-09-11T22:58:41.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.922+0000 7fc022d9d640 1 --2- 172.21.15.152:0/3120523134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc024101df0 0x7fc024198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:41.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.922+0000 7fc02259c640 1 --2- 172.21.15.152:0/3120523134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0241027a0 0x7fc024198a60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc018005ab0 tx=0x7fc0180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:41.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.922+0000 7fc00bfff640 1 -- 172.21.15.152:0/3120523134 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01803f030 con 0x7fc0241027a0 2024-09-11T22:58:41.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.922+0000 7fc029258640 1 -- 172.21.15.152:0/3120523134 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02419d760 con 0x7fc0241027a0 2024-09-11T22:58:41.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.922+0000 7fc00bfff640 1 -- 172.21.15.152:0/3120523134 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc018003ce0 con 0x7fc0241027a0 2024-09-11T22:58:41.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.922+0000 7fc00bfff640 1 -- 172.21.15.152:0/3120523134 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018035070 con 0x7fc0241027a0 2024-09-11T22:58:41.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.922+0000 7fc029258640 1 -- 172.21.15.152:0/3120523134 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0241a50a0 con 0x7fc0241027a0 2024-09-11T22:58:41.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.924+0000 7fc00bfff640 1 -- 172.21.15.152:0/3120523134 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc018042030 con 0x7fc0241027a0 2024-09-11T22:58:41.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.925+0000 7fc02b631740 1 Processor -- start 2024-09-11T22:58:41.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.925+0000 7fc02b631740 1 -- start start 2024-09-11T22:58:41.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.925+0000 7fc02b631740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56232574fb60 0x56232574ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.925+0000 7fc02b631740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562325750480 0x56232575a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.925+0000 7fc02b631740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232575ad40 0x56232575d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.925+0000 7fc02b631740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5623256b5280 con 0x56232575ad40 2024-09-11T22:58:41.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.925+0000 7fc02b631740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56232565cb10 con 0x56232574fb60 2024-09-11T22:58:41.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.925+0000 7fc02b631740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5623256b1240 con 0x562325750480 2024-09-11T22:58:41.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.925+0000 7fc02259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232575ad40 0x56232575d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.925+0000 7fc00bfff640 1 --2- 172.21.15.152:0/3120523134 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc00c0a2d70 0x7fc00c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.925+0000 7fc02359e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56232574fb60 0x56232574ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.925+0000 7fc02259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232575ad40 0x56232575d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53222/0 (socket says 172.21.15.152:53222) 2024-09-11T22:58:41.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.925+0000 7fc02259c640 1 -- 172.21.15.152:0/2629054583 learned_addr learned my addr 172.21.15.152:0/2629054583 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:41.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.925+0000 7fc022d9d640 1 --2- 172.21.15.152:0/2629054583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562325750480 0x56232575a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.925+0000 7fc02359e640 1 --2- 172.21.15.152:0/3120523134 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc00c0a2d70 0x7fc00c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.925+0000 7fc00bfff640 1 -- 172.21.15.152:0/3120523134 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc0180efd40 con 0x7fc0241027a0 2024-09-11T22:58:41.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.926+0000 7fc02259c640 1 -- 172.21.15.152:0/2629054583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562325750480 msgr2=0x56232575a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:41.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.926+0000 7fc02259c640 1 --2- 172.21.15.152:0/2629054583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562325750480 0x56232575a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.926+0000 7fc02259c640 1 -- 172.21.15.152:0/2629054583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56232574fb60 msgr2=0x56232574ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:41.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.926+0000 7fc02259c640 1 --2- 172.21.15.152:0/2629054583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56232574fb60 0x56232574ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.926+0000 7fc02259c640 1 -- 172.21.15.152:0/2629054583 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623256691f0 con 0x56232575ad40 2024-09-11T22:58:41.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.926+0000 7fc02359e640 1 --2- 172.21.15.152:0/2629054583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56232574fb60 0x56232574ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:58:41.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.926+0000 7fc02259c640 1 --2- 172.21.15.152:0/2629054583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232575ad40 0x56232575d130 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc0180ed420 tx=0x7fc01802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9235004362a8bf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:41.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.926+0000 7fc0097fa640 1 -- 172.21.15.152:0/2629054583 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0180f1070 con 0x56232575ad40 2024-09-11T22:58:41.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.926+0000 7fc0097fa640 1 -- 172.21.15.152:0/2629054583 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0180bc9b0 con 0x56232575ad40 2024-09-11T22:58:41.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.926+0000 7fc0097fa640 1 -- 172.21.15.152:0/2629054583 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018049070 con 0x56232575ad40 2024-09-11T22:58:41.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 -- 172.21.15.152:0/2629054583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232575ad40 msgr2=0x56232575d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:41.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02359e640 1 --2- 172.21.15.152:0/3120523134 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc00c0a2d70 0x7fc00c0a5230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc01c012940 tx=0x7fc01c004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:41.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 --2- 172.21.15.152:0/2629054583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232575ad40 0x56232575d130 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc0180ed420 tx=0x7fc01802a410 comp rx=0 tx=0).stop 2024-09-11T22:58:41.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 -- 172.21.15.152:0/2629054583 shutdown_connections 2024-09-11T22:58:41.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 --2- 172.21.15.152:0/2629054583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232575ad40 0x56232575d130 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 --2- 172.21.15.152:0/2629054583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562325750480 0x56232575a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 --2- 172.21.15.152:0/2629054583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56232574fb60 0x56232574ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 -- 172.21.15.152:0/2629054583 >> 172.21.15.152:0/2629054583 conn(0x56232574c0f0 msgr2=0x56232574c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:41.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 -- 172.21.15.152:0/2629054583 shutdown_connections 2024-09-11T22:58:41.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 -- 172.21.15.152:0/2629054583 wait complete. 2024-09-11T22:58:41.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 -- start start 2024-09-11T22:58:41.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 Processor -- start 2024-09-11T22:58:41.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56232574fb60 0x562325757c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562325750480 0x562325758180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56232575ad40 0x5623257586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56232575d6d0 con 0x562325750480 2024-09-11T22:58:41.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56232566ff10 con 0x56232575ad40 2024-09-11T22:58:41.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56232575d850 con 0x56232574fb60 2024-09-11T22:58:41.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02b631740 10 client.0 _ll_register_callbacks cb 0x562325638a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:41.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.927+0000 7fc02259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562325750480 0x562325758180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc02259c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562325750480 0x562325758180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53236/0 (socket says 172.21.15.152:53236) 2024-09-11T22:58:41.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc02259c640 1 -- 172.21.15.152:0/441407182 learned_addr learned my addr 172.21.15.152:0/441407182 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:41.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc022d9d640 1 --2- 172.21.15.152:0/441407182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56232574fb60 0x562325757c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc02359e640 1 --2- 172.21.15.152:0/441407182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56232575ad40 0x5623257586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc02259c640 1 -- 172.21.15.152:0/441407182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56232574fb60 msgr2=0x562325757c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:41.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc02259c640 1 --2- 172.21.15.152:0/441407182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56232574fb60 0x562325757c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc02259c640 1 -- 172.21.15.152:0/441407182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56232575ad40 msgr2=0x5623257586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:41.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc02259c640 1 --2- 172.21.15.152:0/441407182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56232575ad40 0x5623257586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc02259c640 1 -- 172.21.15.152:0/441407182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623254266a0 con 0x562325750480 2024-09-11T22:58:41.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc02259c640 1 --2- 172.21.15.152:0/441407182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562325750480 0x562325758180 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fc018003fd0 tx=0x7fc01803bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfeaffabfad1272 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:41.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc006ff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:41.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc006ff5640 1 -- 172.21.15.152:0/441407182 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018049070 con 0x562325750480 2024-09-11T22:58:41.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc006ff5640 1 -- 172.21.15.152:0/441407182 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc01803b460 con 0x562325750480 2024-09-11T22:58:41.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc02b631740 1 -- 172.21.15.152:0/441407182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0180bc9b0 con 0x562325750480 2024-09-11T22:58:41.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc006ff5640 1 -- 172.21.15.152:0/441407182 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc0180c6050 con 0x562325750480 2024-09-11T22:58:41.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc02b631740 10 client.16986 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:41.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc02b631740 1 -- 172.21.15.152:0/441407182 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56232575d6d0 con 0x562325750480 2024-09-11T22:58:41.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.928+0000 7fc006ff5640 1 -- 172.21.15.152:0/441407182 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0180bc9b0 con 0x562325750480 2024-09-11T22:58:41.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.929+0000 7fc006ff5640 1 -- 172.21.15.152:0/441407182 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc0180bc510 con 0x562325750480 2024-09-11T22:58:41.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.929+0000 7fc02b631740 10 client.16986 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:41.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.929+0000 7fc02b631740 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:58:41.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.929+0000 7fc02b631740 20 client.16986 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:41.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.929+0000 7fc02b631740 1 --2- 172.21.15.152:0/441407182 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56232576a4d0 0x56232576c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.929+0000 7fc02b631740 4 client.16986 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:58:41.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.929+0000 7fc02b631740 1 -- 172.21.15.152:0/441407182 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623256c27f0 con 0x56232576a4d0 2024-09-11T22:58:41.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.929+0000 7fc022d9d640 1 --2- 172.21.15.152:0/441407182 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56232576a4d0 0x56232576c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.929+0000 7fc006ff5640 10 client.16986 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:41.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.929+0000 7fc022d9d640 1 --2- 172.21.15.152:0/441407182 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56232576a4d0 0x56232576c8f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6302631ed76e874 server_cookie=d694fac6c367bf1f in_seq=0 out_seq=0 2024-09-11T22:58:41.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.930+0000 7fc006ff5640 1 -- 172.21.15.152:0/441407182 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623256c27f0 con 0x56232576a4d0 2024-09-11T22:58:41.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.930+0000 7fc006ff5640 10 client.16986 handle_command_reply: tid=0 2024-09-11T22:58:41.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.930+0000 7fc02b631740 1 client.16986 shutdown 2024-09-11T22:58:41.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.930+0000 7fc02b631740 1 -- 172.21.15.152:0/441407182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562325750480 msgr2=0x562325758180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:41.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.930+0000 7fc02b631740 1 --2- 172.21.15.152:0/441407182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562325750480 0x562325758180 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fc018003fd0 tx=0x7fc01803bde0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.930+0000 7fc02b631740 1 -- 172.21.15.152:0/441407182 shutdown_connections 2024-09-11T22:58:41.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.930+0000 7fc02b631740 1 --2- 172.21.15.152:0/441407182 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56232576a4d0 0x56232576c8f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.930+0000 7fc02b631740 1 --2- 172.21.15.152:0/441407182 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56232575ad40 0x5623257586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.930+0000 7fc02b631740 1 --2- 172.21.15.152:0/441407182 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562325750480 0x562325758180 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.930+0000 7fc006ff5640 0 client.16986 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:41.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.930+0000 7fc02b631740 1 --2- 172.21.15.152:0/441407182 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56232574fb60 0x562325757c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.930+0000 7fc02b631740 1 -- 172.21.15.152:0/441407182 >> 172.21.15.152:0/441407182 conn(0x56232574c0f0 msgr2=0x562325751dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:41.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.930+0000 7fc02b631740 1 -- 172.21.15.152:0/441407182 shutdown_connections 2024-09-11T22:58:41.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.930+0000 7fc02b631740 1 -- 172.21.15.152:0/441407182 wait complete. 2024-09-11T22:58:41.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.930+0000 7fc02b631740 20 client.16986 trim_cache size 0 max 16384 2024-09-11T22:58:41.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc02b631740 1 Processor -- start 2024-09-11T22:58:41.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc02b631740 1 -- start start 2024-09-11T22:58:41.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc02b631740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232574fb60 0x562325751dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc02b631740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562325750480 0x562325758510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc02b631740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56232575ad40 0x5623257721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc02b631740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241ad690 con 0x56232574fb60 2024-09-11T22:58:41.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc02b631740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241af1d0 con 0x56232575ad40 2024-09-11T22:58:41.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc02b631740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241add60 con 0x562325750480 2024-09-11T22:58:41.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc022d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232574fb60 0x562325751dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc022d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232574fb60 0x562325751dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53244/0 (socket says 172.21.15.152:53244) 2024-09-11T22:58:41.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc022d9d640 1 -- 172.21.15.152:0/2574102600 learned_addr learned my addr 172.21.15.152:0/2574102600 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:41.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc02359e640 1 --2- 172.21.15.152:0/2574102600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56232575ad40 0x5623257721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc022d9d640 1 -- 172.21.15.152:0/2574102600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562325750480 msgr2=0x562325758510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:41.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc022d9d640 1 --2- 172.21.15.152:0/2574102600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562325750480 0x562325758510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc022d9d640 1 -- 172.21.15.152:0/2574102600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56232575ad40 msgr2=0x5623257721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:41.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc022d9d640 1 --2- 172.21.15.152:0/2574102600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56232575ad40 0x5623257721a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc022d9d640 1 -- 172.21.15.152:0/2574102600 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562325756190 con 0x56232574fb60 2024-09-11T22:58:41.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc022d9d640 1 --2- 172.21.15.152:0/2574102600 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232574fb60 0x562325751dd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x56232576d110 tx=0x7fc014008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aeb5a42d73b1cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:41.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc008ff9640 1 -- 172.21.15.152:0/2574102600 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014016dd0 con 0x56232574fb60 2024-09-11T22:58:41.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.948+0000 7fc008ff9640 1 -- 172.21.15.152:0/2574102600 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc014020030 con 0x56232574fb60 2024-09-11T22:58:41.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.949+0000 7fc008ff9640 1 -- 172.21.15.152:0/2574102600 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014024dc0 con 0x56232574fb60 2024-09-11T22:58:41.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.949+0000 7fc02b631740 1 -- 172.21.15.152:0/2574102600 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232574fb60 msgr2=0x562325751dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:41.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.949+0000 7fc02b631740 1 --2- 172.21.15.152:0/2574102600 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232574fb60 0x562325751dd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x56232576d110 tx=0x7fc014008040 comp rx=0 tx=0).stop 2024-09-11T22:58:41.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.949+0000 7fc02b631740 1 -- 172.21.15.152:0/2574102600 shutdown_connections 2024-09-11T22:58:41.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.949+0000 7fc02b631740 1 --2- 172.21.15.152:0/2574102600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56232575ad40 0x5623257721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.949+0000 7fc02b631740 1 --2- 172.21.15.152:0/2574102600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562325750480 0x562325758510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.949+0000 7fc02b631740 1 --2- 172.21.15.152:0/2574102600 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232574fb60 0x562325751dd0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.949+0000 7fc02b631740 1 -- 172.21.15.152:0/2574102600 >> 172.21.15.152:0/2574102600 conn(0x56232574c0f0 msgr2=0x562325759c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:41.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.949+0000 7fc02b631740 1 -- 172.21.15.152:0/2574102600 shutdown_connections 2024-09-11T22:58:41.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.949+0000 7fc02b631740 1 -- 172.21.15.152:0/2574102600 wait complete. 2024-09-11T22:58:41.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.949+0000 7fc02b631740 1 -- start start 2024-09-11T22:58:41.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.949+0000 7fc02b631740 1 Processor -- start 2024-09-11T22:58:41.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc02b631740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232574fb60 0x562325763240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc02b631740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562325750480 0x56232575e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc02b631740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56232575ad40 0x56232575eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc02b631740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241aeb00 con 0x56232574fb60 2024-09-11T22:58:41.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc02b631740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241ae430 con 0x562325750480 2024-09-11T22:58:41.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc02b631740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241ab650 con 0x56232575ad40 2024-09-11T22:58:41.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc02b631740 10 client.0 _ll_register_callbacks cb 0x7fc01c0075e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:41.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc022d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232574fb60 0x562325763240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc022d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232574fb60 0x562325763240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53252/0 (socket says 172.21.15.152:53252) 2024-09-11T22:58:41.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc022d9d640 1 -- 172.21.15.152:0/4037311172 learned_addr learned my addr 172.21.15.152:0/4037311172 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:41.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc02259c640 1 --2- 172.21.15.152:0/4037311172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562325750480 0x56232575e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc02359e640 1 --2- 172.21.15.152:0/4037311172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56232575ad40 0x56232575eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc022d9d640 1 -- 172.21.15.152:0/4037311172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56232575ad40 msgr2=0x56232575eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:41.964 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc022d9d640 1 --2- 172.21.15.152:0/4037311172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56232575ad40 0x56232575eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc022d9d640 1 -- 172.21.15.152:0/4037311172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562325750480 msgr2=0x56232575e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:41.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc022d9d640 1 --2- 172.21.15.152:0/4037311172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562325750480 0x56232575e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc022d9d640 1 -- 172.21.15.152:0/4037311172 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56232575f390 con 0x56232574fb60 2024-09-11T22:58:41.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc022d9d640 1 --2- 172.21.15.152:0/4037311172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232574fb60 0x562325763240 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fc014004810 tx=0x7fc014007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb3b085b346ae6a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:41.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc006ff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:41.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc006ff5640 1 -- 172.21.15.152:0/4037311172 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014029070 con 0x56232574fb60 2024-09-11T22:58:41.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc006ff5640 1 -- 172.21.15.152:0/4037311172 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc0140247e0 con 0x56232574fb60 2024-09-11T22:58:41.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.950+0000 7fc02b631740 1 -- 172.21.15.152:0/4037311172 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc014020030 con 0x56232574fb60 2024-09-11T22:58:41.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.951+0000 7fc006ff5640 1 -- 172.21.15.152:0/4037311172 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc014024e80 con 0x56232574fb60 2024-09-11T22:58:41.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.951+0000 7fc02b631740 10 client.16992 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:41.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.951+0000 7fc02b631740 1 -- 172.21.15.152:0/4037311172 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5623257d5290 con 0x56232574fb60 2024-09-11T22:58:41.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.951+0000 7fc006ff5640 1 -- 172.21.15.152:0/4037311172 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc014020030 con 0x56232574fb60 2024-09-11T22:58:41.965 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.951+0000 7fc006ff5640 1 -- 172.21.15.152:0/4037311172 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc0140143d0 con 0x56232574fb60 2024-09-11T22:58:41.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.951+0000 7fc02b631740 10 client.16992 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:41.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.951+0000 7fc02b631740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:58:41.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.951+0000 7fc02b631740 20 client.16992 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:41.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.951+0000 7fc02b631740 1 --2- 172.21.15.152:0/4037311172 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5623257d5980 0x5623257d7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:41.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.951+0000 7fc02b631740 4 client.16992 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:58:41.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.951+0000 7fc02b631740 1 -- 172.21.15.152:0/4037311172 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5623256bcd00 con 0x5623257d5980 2024-09-11T22:58:41.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.951+0000 7fc02259c640 1 --2- 172.21.15.152:0/4037311172 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5623257d5980 0x5623257d7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:41.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.952+0000 7fc02259c640 1 --2- 172.21.15.152:0/4037311172 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5623257d5980 0x5623257d7da0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f58e5fead657186 server_cookie=26c14dc79eae3d38 in_seq=0 out_seq=0 2024-09-11T22:58:41.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.952+0000 7fc006ff5640 10 client.16992 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:41.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.953+0000 7fc006ff5640 1 -- 172.21.15.152:0/4037311172 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5623256bcd00 con 0x5623257d5980 2024-09-11T22:58:41.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.953+0000 7fc006ff5640 10 client.16992 handle_command_reply: tid=0 2024-09-11T22:58:41.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.953+0000 7fc02b631740 1 client.16992 shutdown 2024-09-11T22:58:41.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.953+0000 7fc02b631740 1 -- 172.21.15.152:0/4037311172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232574fb60 msgr2=0x562325763240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:41.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.953+0000 7fc02b631740 1 --2- 172.21.15.152:0/4037311172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232574fb60 0x562325763240 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fc014004810 tx=0x7fc014007f50 comp rx=0 tx=0).stop 2024-09-11T22:58:41.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.953+0000 7fc02b631740 1 -- 172.21.15.152:0/4037311172 shutdown_connections 2024-09-11T22:58:41.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.953+0000 7fc02b631740 1 --2- 172.21.15.152:0/4037311172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56232575ad40 0x56232575eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.953+0000 7fc02b631740 1 --2- 172.21.15.152:0/4037311172 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5623257d5980 0x5623257d7da0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.953+0000 7fc02b631740 1 --2- 172.21.15.152:0/4037311172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562325750480 0x56232575e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.953+0000 7fc006ff5640 0 client.16992 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:41.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.953+0000 7fc02b631740 1 --2- 172.21.15.152:0/4037311172 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56232574fb60 0x562325763240 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.953+0000 7fc02b631740 1 -- 172.21.15.152:0/4037311172 >> 172.21.15.152:0/4037311172 conn(0x56232574c0f0 msgr2=0x562325756c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:41.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.953+0000 7fc02b631740 1 -- 172.21.15.152:0/4037311172 shutdown_connections 2024-09-11T22:58:41.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.953+0000 7fc02b631740 1 -- 172.21.15.152:0/4037311172 wait complete. 2024-09-11T22:58:41.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.953+0000 7fc02b631740 20 client.16992 trim_cache size 0 max 16384 2024-09-11T22:58:41.968 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:58:41.968 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T22:58:41.968 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "5f65c69f-031d-472b-ada9-374bdf0b4315", 2024-09-11T22:58:41.968 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T22:58:41.968 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:58:41.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.954+0000 7fc029258640 1 -- 172.21.15.152:0/3120523134 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc00c0a2d70 msgr2=0x7fc00c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:41.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.954+0000 7fc029258640 1 --2- 172.21.15.152:0/3120523134 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc00c0a2d70 0x7fc00c0a5230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc01c012940 tx=0x7fc01c004ab0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.954+0000 7fc029258640 1 -- 172.21.15.152:0/3120523134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0241027a0 msgr2=0x7fc024198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:41.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.954+0000 7fc029258640 1 --2- 172.21.15.152:0/3120523134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0241027a0 0x7fc024198a60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc018005ab0 tx=0x7fc0180042f0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.954+0000 7fc029258640 1 -- 172.21.15.152:0/3120523134 shutdown_connections 2024-09-11T22:58:41.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.954+0000 7fc029258640 1 --2- 172.21.15.152:0/3120523134 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc00c0a2d70 0x7fc00c0a5230 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.954+0000 7fc029258640 1 --2- 172.21.15.152:0/3120523134 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc024106f20 0x7fc02419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.954+0000 7fc029258640 1 --2- 172.21.15.152:0/3120523134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc0241027a0 0x7fc024198a60 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.954+0000 7fc029258640 1 --2- 172.21.15.152:0/3120523134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc024101df0 0x7fc024198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:41.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.954+0000 7fc029258640 1 -- 172.21.15.152:0/3120523134 >> 172.21.15.152:0/3120523134 conn(0x7fc0240fd8f0 msgr2=0x7fc0240fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:41.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.954+0000 7fc029258640 1 -- 172.21.15.152:0/3120523134 shutdown_connections 2024-09-11T22:58:41.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:41.954+0000 7fc029258640 1 -- 172.21.15.152:0/3120523134 wait complete. 2024-09-11T22:58:42.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:42 smithi191 ceph-mon[35211]: pgmap v177: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:58:42.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:42 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T22:58:42.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:42 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:58:42.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:42 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T22:58:42.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:42 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:58:42.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:42 smithi204 ceph-mon[35454]: pgmap v177: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:58:42.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:42 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T22:58:42.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:42 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:58:42.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:42 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T22:58:42.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:42 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:58:42.587 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:42 smithi152 ceph-mon[31181]: pgmap v177: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T22:58:42.587 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:42 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T22:58:42.587 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:42 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:58:42.587 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:42 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T22:58:42.587 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:42 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:58:42.705 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:58:42.951 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:43.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:43 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T22:58:43.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.488+0000 7f30b11a3640 1 -- 172.21.15.152:0/1249395168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ac106f50 msgr2=0x7f30ac10eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:43.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.488+0000 7f30b11a3640 1 --2- 172.21.15.152:0/1249395168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ac106f50 0x7f30ac10eaf0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f30a4009cd0 tx=0x7f30a402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.489 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.488+0000 7f30b11a3640 1 -- 172.21.15.152:0/1249395168 shutdown_connections 2024-09-11T22:58:43.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.488+0000 7f30b11a3640 1 --2- 172.21.15.152:0/1249395168 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ac106f50 0x7f30ac10eaf0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.488+0000 7f30b11a3640 1 --2- 172.21.15.152:0/1249395168 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30ac100ca0 0x7f30ac106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.488+0000 7f30b11a3640 1 --2- 172.21.15.152:0/1249395168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30ac1002f0 0x7f30ac1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.488+0000 7f30b11a3640 1 -- 172.21.15.152:0/1249395168 >> 172.21.15.152:0/1249395168 conn(0x7f30ac077210 msgr2=0x7f30ac0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:43.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.488+0000 7f30b11a3640 1 -- 172.21.15.152:0/1249395168 shutdown_connections 2024-09-11T22:58:43.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.488+0000 7f30b11a3640 1 -- 172.21.15.152:0/1249395168 wait complete. 2024-09-11T22:58:43.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.489+0000 7f30b11a3640 1 Processor -- start 2024-09-11T22:58:43.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.489+0000 7f30b11a3640 1 -- start start 2024-09-11T22:58:43.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.489+0000 7f30b11a3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30ac1002f0 0x7f30ac1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.489+0000 7f30b11a3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30ac100ca0 0x7f30ac1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.489+0000 7f30b11a3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ac106f50 0x7f30ac19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.489+0000 7f30b11a3640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac111fa0 con 0x7f30ac106f50 2024-09-11T22:58:43.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.489+0000 7f30b11a3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac111e20 con 0x7f30ac100ca0 2024-09-11T22:58:43.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.489+0000 7f30b11a3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac112120 con 0x7f30ac1002f0 2024-09-11T22:58:43.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f30ab59e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ac106f50 0x7f30ac19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f30ab59e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ac106f50 0x7f30ac19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53282/0 (socket says 172.21.15.152:53282) 2024-09-11T22:58:43.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f30ab59e640 1 -- 172.21.15.152:0/4196928277 learned_addr learned my addr 172.21.15.152:0/4196928277 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:43.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f30aad9d640 1 --2- 172.21.15.152:0/4196928277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30ac1002f0 0x7f30ac1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f30aa59c640 1 --2- 172.21.15.152:0/4196928277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30ac100ca0 0x7f30ac1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f30ab59e640 1 -- 172.21.15.152:0/4196928277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30ac1002f0 msgr2=0x7f30ac1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:43.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f30ab59e640 1 --2- 172.21.15.152:0/4196928277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30ac1002f0 0x7f30ac1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f30ab59e640 1 -- 172.21.15.152:0/4196928277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30ac100ca0 msgr2=0x7f30ac1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:43.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f30ab59e640 1 --2- 172.21.15.152:0/4196928277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30ac100ca0 0x7f30ac1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f30ab59e640 1 -- 172.21.15.152:0/4196928277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ac19d350 con 0x7f30ac106f50 2024-09-11T22:58:43.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f30aa59c640 1 --2- 172.21.15.152:0/4196928277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30ac100ca0 0x7f30ac1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:43.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f30ab59e640 1 --2- 172.21.15.152:0/4196928277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ac106f50 0x7f30ac19cc70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f30a4006010 tx=0x7f30a4003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:43.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f3093fff640 1 -- 172.21.15.152:0/4196928277 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4004450 con 0x7f30ac106f50 2024-09-11T22:58:43.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f30b11a3640 1 -- 172.21.15.152:0/4196928277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ac19d5e0 con 0x7f30ac106f50 2024-09-11T22:58:43.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f3093fff640 1 -- 172.21.15.152:0/4196928277 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a4003e10 con 0x7f30ac106f50 2024-09-11T22:58:43.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f3093fff640 1 -- 172.21.15.152:0/4196928277 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a40466e0 con 0x7f30ac106f50 2024-09-11T22:58:43.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.490+0000 7f30b11a3640 1 -- 172.21.15.152:0/4196928277 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ac1a4f20 con 0x7f30ac106f50 2024-09-11T22:58:43.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.493+0000 7f3093fff640 1 -- 172.21.15.152:0/4196928277 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f30a40305a0 con 0x7f30ac106f50 2024-09-11T22:58:43.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.493+0000 7f30b357c740 1 Processor -- start 2024-09-11T22:58:43.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.493+0000 7f30b357c740 1 -- start start 2024-09-11T22:58:43.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.493+0000 7f3093fff640 1 --2- 172.21.15.152:0/4196928277 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f30940a2cf0 0x7f30940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.493+0000 7f30b357c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb907880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30b357c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb907dc0 0x55d8fb912140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30b357c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb912680 0x55d8fb914a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30b357c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d8fb86d2e0 con 0x55d8fb9074a0 2024-09-11T22:58:43.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30b357c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d8fb814ce0 con 0x55d8fb907dc0 2024-09-11T22:58:43.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30b357c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d8fb8692a0 con 0x55d8fb912680 2024-09-11T22:58:43.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30aad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb907880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30aad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb907880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53298/0 (socket says 172.21.15.152:53298) 2024-09-11T22:58:43.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30aad9d640 1 -- 172.21.15.152:0/542873255 learned_addr learned my addr 172.21.15.152:0/542873255 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:43.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f3093fff640 1 -- 172.21.15.152:0/4196928277 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f30a402e030 con 0x7f30ac106f50 2024-09-11T22:58:43.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30aa59c640 1 --2- 172.21.15.152:0/4196928277 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f30940a2cf0 0x7f30940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30ab59e640 1 --2- 172.21.15.152:0/542873255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb912680 0x55d8fb914a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30aad9d640 1 -- 172.21.15.152:0/542873255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb912680 msgr2=0x55d8fb914a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:43.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30aad9d640 1 --2- 172.21.15.152:0/542873255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb912680 0x55d8fb914a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30aad9d640 1 -- 172.21.15.152:0/542873255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb907dc0 msgr2=0x55d8fb912140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:43.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30aad9d640 1 --2- 172.21.15.152:0/542873255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb907dc0 0x55d8fb912140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30aad9d640 1 -- 172.21.15.152:0/542873255 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8fb7f9440 con 0x55d8fb9074a0 2024-09-11T22:58:43.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30aad9d640 1 --2- 172.21.15.152:0/542873255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb907880 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f309c00ac20 tx=0x7f309c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf16c98745f30b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:43.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30aa59c640 1 --2- 172.21.15.152:0/4196928277 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f30940a2cf0 0x7f30940a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55d8fb904890 tx=0x7f30a000e350 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:43.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30917fa640 1 -- 172.21.15.152:0/542873255 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c021070 con 0x55d8fb9074a0 2024-09-11T22:58:43.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30917fa640 1 -- 172.21.15.152:0/542873255 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f309c01c440 con 0x55d8fb9074a0 2024-09-11T22:58:43.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.494+0000 7f30917fa640 1 -- 172.21.15.152:0/542873255 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c01a580 con 0x55d8fb9074a0 2024-09-11T22:58:43.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.495+0000 7f30b357c740 1 -- 172.21.15.152:0/542873255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 msgr2=0x55d8fb907880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:43.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.495+0000 7f30b357c740 1 --2- 172.21.15.152:0/542873255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb907880 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f309c00ac20 tx=0x7f309c00f790 comp rx=0 tx=0).stop 2024-09-11T22:58:43.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.495+0000 7f30b357c740 1 -- 172.21.15.152:0/542873255 shutdown_connections 2024-09-11T22:58:43.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.495+0000 7f30b357c740 1 --2- 172.21.15.152:0/542873255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb912680 0x55d8fb914a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.495+0000 7f30b357c740 1 --2- 172.21.15.152:0/542873255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb907dc0 0x55d8fb912140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.495+0000 7f30b357c740 1 --2- 172.21.15.152:0/542873255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb907880 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.495+0000 7f30b357c740 1 -- 172.21.15.152:0/542873255 >> 172.21.15.152:0/542873255 conn(0x55d8fb903a30 msgr2=0x55d8fb903e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:43.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.495+0000 7f30b357c740 1 -- 172.21.15.152:0/542873255 shutdown_connections 2024-09-11T22:58:43.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.495+0000 7f30b357c740 1 -- 172.21.15.152:0/542873255 wait complete. 2024-09-11T22:58:43.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30b357c740 1 -- start start 2024-09-11T22:58:43.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30b357c740 1 Processor -- start 2024-09-11T22:58:43.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30b357c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb911ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30b357c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb907dc0 0x55d8fb90f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30b357c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb912680 0x55d8fb90f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30b357c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d8fb915250 con 0x55d8fb9074a0 2024-09-11T22:58:43.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30b357c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d8fb828050 con 0x55d8fb907dc0 2024-09-11T22:58:43.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30b357c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d8fb9153d0 con 0x55d8fb912680 2024-09-11T22:58:43.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30b357c740 10 client.0 _ll_register_callbacks cb 0x55d8fb868160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:43.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30aad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb911ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30aad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb911ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53302/0 (socket says 172.21.15.152:53302) 2024-09-11T22:58:43.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30aad9d640 1 -- 172.21.15.152:0/476117539 learned_addr learned my addr 172.21.15.152:0/476117539 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:43.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30ab59e640 1 --2- 172.21.15.152:0/476117539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb912680 0x55d8fb90f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30aa59c640 1 --2- 172.21.15.152:0/476117539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb907dc0 0x55d8fb90f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30aad9d640 1 -- 172.21.15.152:0/476117539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb912680 msgr2=0x55d8fb90f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:43.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30aad9d640 1 --2- 172.21.15.152:0/476117539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb912680 0x55d8fb90f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30aad9d640 1 -- 172.21.15.152:0/476117539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb907dc0 msgr2=0x55d8fb90f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:43.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30aad9d640 1 --2- 172.21.15.152:0/476117539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb907dc0 0x55d8fb90f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.496+0000 7f30aad9d640 1 -- 172.21.15.152:0/476117539 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8fb5de3c0 con 0x55d8fb9074a0 2024-09-11T22:58:43.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.497+0000 7f30aad9d640 1 --2- 172.21.15.152:0/476117539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb911ff0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f309c01ce40 tx=0x7f309c00a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa40cc10df2a6bbe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:43.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.497+0000 7f308eff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:43.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.497+0000 7f308eff5640 1 -- 172.21.15.152:0/476117539 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c021050 con 0x55d8fb9074a0 2024-09-11T22:58:43.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.497+0000 7f308eff5640 1 -- 172.21.15.152:0/476117539 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f309c005450 con 0x55d8fb9074a0 2024-09-11T22:58:43.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.497+0000 7f30b357c740 1 -- 172.21.15.152:0/476117539 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f309c01c440 con 0x55d8fb9074a0 2024-09-11T22:58:43.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.497+0000 7f308eff5640 1 -- 172.21.15.152:0/476117539 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f309c036050 con 0x55d8fb9074a0 2024-09-11T22:58:43.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.497+0000 7f30b357c740 10 client.17004 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:43.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.497+0000 7f30b357c740 1 -- 172.21.15.152:0/476117539 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8fb920c20 con 0x55d8fb9074a0 2024-09-11T22:58:43.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.497+0000 7f308eff5640 1 -- 172.21.15.152:0/476117539 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f309c01c440 con 0x55d8fb9074a0 2024-09-11T22:58:43.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.497+0000 7f308eff5640 1 -- 172.21.15.152:0/476117539 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f309c0049c0 con 0x55d8fb9074a0 2024-09-11T22:58:43.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.497+0000 7f30b357c740 10 client.17004 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:43.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.497+0000 7f30b357c740 10 client.17004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:58:43.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.497+0000 7f30b357c740 20 client.17004 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:43.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.497+0000 7f30b357c740 1 --2- 172.21.15.152:0/476117539 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d8fb922080 0x55d8fb924470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.497+0000 7f30b357c740 4 client.17004 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:58:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.497+0000 7f30b357c740 1 -- 172.21.15.152:0/476117539 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8fb87a850 con 0x55d8fb922080 2024-09-11T22:58:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.498+0000 7f30aa59c640 1 --2- 172.21.15.152:0/476117539 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d8fb922080 0x55d8fb924470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.498+0000 7f30aa59c640 1 --2- 172.21.15.152:0/476117539 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d8fb922080 0x55d8fb924470 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e61bd268c17efc server_cookie=a1b9bbf8cae33269 in_seq=0 out_seq=0 2024-09-11T22:58:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.498+0000 7f308eff5640 10 client.17004 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.499+0000 7f308eff5640 1 -- 172.21.15.152:0/476117539 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8fb87a850 con 0x55d8fb922080 2024-09-11T22:58:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.499+0000 7f308eff5640 10 client.17004 handle_command_reply: tid=0 2024-09-11T22:58:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.499+0000 7f30b357c740 1 client.17004 shutdown 2024-09-11T22:58:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.499+0000 7f30b357c740 1 -- 172.21.15.152:0/476117539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 msgr2=0x55d8fb911ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.499+0000 7f30b357c740 1 --2- 172.21.15.152:0/476117539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb911ff0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f309c01ce40 tx=0x7f309c00a950 comp rx=0 tx=0).stop 2024-09-11T22:58:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.499+0000 7f30b357c740 1 -- 172.21.15.152:0/476117539 shutdown_connections 2024-09-11T22:58:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.499+0000 7f30b357c740 1 --2- 172.21.15.152:0/476117539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb912680 0x55d8fb90f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.499+0000 7f30b357c740 1 --2- 172.21.15.152:0/476117539 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d8fb922080 0x55d8fb924470 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.499+0000 7f30b357c740 1 --2- 172.21.15.152:0/476117539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb907dc0 0x55d8fb90f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.499+0000 7f30b357c740 1 --2- 172.21.15.152:0/476117539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb911ff0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.499+0000 7f30b357c740 1 -- 172.21.15.152:0/476117539 >> 172.21.15.152:0/476117539 conn(0x55d8fb903a30 msgr2=0x55d8fb902b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:43.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.499+0000 7f308eff5640 0 client.17004 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:43.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.499+0000 7f30b357c740 1 -- 172.21.15.152:0/476117539 shutdown_connections 2024-09-11T22:58:43.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.499+0000 7f30b357c740 1 -- 172.21.15.152:0/476117539 wait complete. 2024-09-11T22:58:43.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.499+0000 7f30b357c740 20 client.17004 trim_cache size 0 max 16384 2024-09-11T22:58:43.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30b357c740 1 Processor -- start 2024-09-11T22:58:43.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30b357c740 1 -- start start 2024-09-11T22:58:43.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30b357c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb9074a0 0x55d8fb929ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30b357c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb907dc0 0x55d8fb92a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30b357c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb912680 0x55d8fb90c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30b357c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1ab490 con 0x55d8fb912680 2024-09-11T22:58:43.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30b357c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1acfd0 con 0x55d8fb907dc0 2024-09-11T22:58:43.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30b357c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1abb60 con 0x55d8fb9074a0 2024-09-11T22:58:43.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30ab59e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb912680 0x55d8fb90c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30ab59e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb912680 0x55d8fb90c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53310/0 (socket says 172.21.15.152:53310) 2024-09-11T22:58:43.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30ab59e640 1 -- 172.21.15.152:0/1890487394 learned_addr learned my addr 172.21.15.152:0/1890487394 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:43.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30aad9d640 1 --2- 172.21.15.152:0/1890487394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb9074a0 0x55d8fb929ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30aa59c640 1 --2- 172.21.15.152:0/1890487394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb907dc0 0x55d8fb92a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30ab59e640 1 -- 172.21.15.152:0/1890487394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb9074a0 msgr2=0x55d8fb929ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:43.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30ab59e640 1 --2- 172.21.15.152:0/1890487394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb9074a0 0x55d8fb929ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30ab59e640 1 -- 172.21.15.152:0/1890487394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb907dc0 msgr2=0x55d8fb92a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:43.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30ab59e640 1 --2- 172.21.15.152:0/1890487394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb907dc0 0x55d8fb92a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30ab59e640 1 -- 172.21.15.152:0/1890487394 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8fb915380 con 0x55d8fb912680 2024-09-11T22:58:43.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30aad9d640 1 --2- 172.21.15.152:0/1890487394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb9074a0 0x55d8fb929ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:43.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.516+0000 7f30aa59c640 1 --2- 172.21.15.152:0/1890487394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb907dc0 0x55d8fb92a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:43.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.517+0000 7f30ab59e640 1 --2- 172.21.15.152:0/1890487394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb912680 0x55d8fb90c520 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f30a40b9d10 tx=0x7f30a40dd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39731b2d022ea03c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:43.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.517+0000 7f3090ff9640 1 -- 172.21.15.152:0/1890487394 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a40c16f0 con 0x55d8fb912680 2024-09-11T22:58:43.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.517+0000 7f3090ff9640 1 -- 172.21.15.152:0/1890487394 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a4036040 con 0x55d8fb912680 2024-09-11T22:58:43.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.517+0000 7f3090ff9640 1 -- 172.21.15.152:0/1890487394 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a40b99b0 con 0x55d8fb912680 2024-09-11T22:58:43.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.517+0000 7f30b357c740 1 -- 172.21.15.152:0/1890487394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb912680 msgr2=0x55d8fb90c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:43.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.517+0000 7f30b357c740 1 --2- 172.21.15.152:0/1890487394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb912680 0x55d8fb90c520 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f30a40b9d10 tx=0x7f30a40dd040 comp rx=0 tx=0).stop 2024-09-11T22:58:43.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.517+0000 7f30b357c740 1 -- 172.21.15.152:0/1890487394 shutdown_connections 2024-09-11T22:58:43.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.517+0000 7f30b357c740 1 --2- 172.21.15.152:0/1890487394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb912680 0x55d8fb90c520 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.517+0000 7f30b357c740 1 --2- 172.21.15.152:0/1890487394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb907dc0 0x55d8fb92a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.517+0000 7f30b357c740 1 --2- 172.21.15.152:0/1890487394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb9074a0 0x55d8fb929ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.517+0000 7f30b357c740 1 -- 172.21.15.152:0/1890487394 >> 172.21.15.152:0/1890487394 conn(0x55d8fb903a30 msgr2=0x55d8fb902b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:43.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.518+0000 7f30b357c740 1 -- 172.21.15.152:0/1890487394 shutdown_connections 2024-09-11T22:58:43.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.518+0000 7f30b357c740 1 -- 172.21.15.152:0/1890487394 wait complete. 2024-09-11T22:58:43.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.518+0000 7f30b357c740 1 -- start start 2024-09-11T22:58:43.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.518+0000 7f30b357c740 1 Processor -- start 2024-09-11T22:58:43.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.518+0000 7f30b357c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb91ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.524 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.518+0000 7f30b357c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb907dc0 0x55d8fb9180f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.518+0000 7f30b357c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb912680 0x55d8fb918630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.518+0000 7f30b357c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1ac900 con 0x55d8fb9074a0 2024-09-11T22:58:43.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.518+0000 7f30b357c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1ac230 con 0x55d8fb912680 2024-09-11T22:58:43.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.518+0000 7f30b357c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac1a9470 con 0x55d8fb907dc0 2024-09-11T22:58:43.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.518+0000 7f30b357c740 10 client.0 _ll_register_callbacks cb 0x7f309c01f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:43.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.518+0000 7f30aad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb91ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.518+0000 7f30aad9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb91ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53320/0 (socket says 172.21.15.152:53320) 2024-09-11T22:58:43.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.518+0000 7f30aad9d640 1 -- 172.21.15.152:0/3876572576 learned_addr learned my addr 172.21.15.152:0/3876572576 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:43.525 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f30ab59e640 1 --2- 172.21.15.152:0/3876572576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb912680 0x55d8fb918630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f30aa59c640 1 --2- 172.21.15.152:0/3876572576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb907dc0 0x55d8fb9180f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f30aad9d640 1 -- 172.21.15.152:0/3876572576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb907dc0 msgr2=0x55d8fb9180f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:43.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f30aad9d640 1 --2- 172.21.15.152:0/3876572576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb907dc0 0x55d8fb9180f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f30aad9d640 1 -- 172.21.15.152:0/3876572576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb912680 msgr2=0x55d8fb918630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:43.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f30aad9d640 1 --2- 172.21.15.152:0/3876572576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb912680 0x55d8fb918630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f30aad9d640 1 -- 172.21.15.152:0/3876572576 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8fb918e30 con 0x55d8fb9074a0 2024-09-11T22:58:43.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f30aa59c640 1 --2- 172.21.15.152:0/3876572576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb907dc0 0x55d8fb9180f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:43.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f30aad9d640 1 --2- 172.21.15.152:0/3876572576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb91ada0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f309c0058c0 tx=0x7f309c00afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d468284cb4e654 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:43.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f308eff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:43.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f308eff5640 1 -- 172.21.15.152:0/3876572576 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c021050 con 0x55d8fb9074a0 2024-09-11T22:58:43.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f308eff5640 1 -- 172.21.15.152:0/3876572576 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f309c028450 con 0x55d8fb9074a0 2024-09-11T22:58:43.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f30b357c740 1 -- 172.21.15.152:0/3876572576 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30a4036040 con 0x55d8fb9074a0 2024-09-11T22:58:43.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f308eff5640 1 -- 172.21.15.152:0/3876572576 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f309c04a050 con 0x55d8fb9074a0 2024-09-11T22:58:43.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f30b357c740 10 client.17010 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:43.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f30b357c740 1 -- 172.21.15.152:0/3876572576 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d8fb98ca60 con 0x55d8fb9074a0 2024-09-11T22:58:43.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.519+0000 7f308eff5640 1 -- 172.21.15.152:0/3876572576 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a4036040 con 0x55d8fb9074a0 2024-09-11T22:58:43.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.520+0000 7f308eff5640 1 -- 172.21.15.152:0/3876572576 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f309c004660 con 0x55d8fb9074a0 2024-09-11T22:58:43.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.520+0000 7f30b357c740 10 client.17010 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:43.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.520+0000 7f30b357c740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:58:43.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.520+0000 7f30b357c740 20 client.17010 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:43.527 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.520+0000 7f30b357c740 1 --2- 172.21.15.152:0/3876572576 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d8fb98d170 0x55d8fb98f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:43.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.520+0000 7f30b357c740 4 client.17010 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:58:43.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.520+0000 7f30b357c740 1 -- 172.21.15.152:0/3876572576 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d8fb874d60 con 0x55d8fb98d170 2024-09-11T22:58:43.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.520+0000 7f30aa59c640 1 --2- 172.21.15.152:0/3876572576 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d8fb98d170 0x55d8fb98f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:43.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.520+0000 7f308eff5640 10 client.17010 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:43.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.520+0000 7f30aa59c640 1 --2- 172.21.15.152:0/3876572576 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d8fb98d170 0x55d8fb98f560 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b2cfb9d5fc2b18e server_cookie=6c70ddf21a34b56e in_seq=0 out_seq=0 2024-09-11T22:58:43.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f308eff5640 1 -- 172.21.15.152:0/3876572576 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d8fb874d60 con 0x55d8fb98d170 2024-09-11T22:58:43.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f308eff5640 10 client.17010 handle_command_reply: tid=0 2024-09-11T22:58:43.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b357c740 1 client.17010 shutdown 2024-09-11T22:58:43.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b357c740 1 -- 172.21.15.152:0/3876572576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 msgr2=0x55d8fb91ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:43.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b357c740 1 --2- 172.21.15.152:0/3876572576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb91ada0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f309c0058c0 tx=0x7f309c00afa0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b357c740 1 -- 172.21.15.152:0/3876572576 shutdown_connections 2024-09-11T22:58:43.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b357c740 1 --2- 172.21.15.152:0/3876572576 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d8fb98d170 0x55d8fb98f560 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f308eff5640 0 client.17010 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:43.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b357c740 1 --2- 172.21.15.152:0/3876572576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d8fb912680 0x55d8fb918630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b357c740 1 --2- 172.21.15.152:0/3876572576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d8fb907dc0 0x55d8fb9180f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b357c740 1 --2- 172.21.15.152:0/3876572576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d8fb9074a0 0x55d8fb91ada0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b357c740 1 -- 172.21.15.152:0/3876572576 >> 172.21.15.152:0/3876572576 conn(0x55d8fb903a30 msgr2=0x55d8fb910b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:43.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b357c740 1 -- 172.21.15.152:0/3876572576 shutdown_connections 2024-09-11T22:58:43.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b357c740 1 -- 172.21.15.152:0/3876572576 wait complete. 2024-09-11T22:58:43.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b357c740 20 client.17010 trim_cache size 0 max 16384 2024-09-11T22:58:43.530 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:58:43.530 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T22:58:43.530 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T22:58:43.530 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:58:43.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b11a3640 1 -- 172.21.15.152:0/4196928277 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f30940a2cf0 msgr2=0x7f30940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:43.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b11a3640 1 --2- 172.21.15.152:0/4196928277 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f30940a2cf0 0x7f30940a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55d8fb904890 tx=0x7f30a000e350 comp rx=0 tx=0).stop 2024-09-11T22:58:43.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b11a3640 1 -- 172.21.15.152:0/4196928277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ac106f50 msgr2=0x7f30ac19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:43.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b11a3640 1 --2- 172.21.15.152:0/4196928277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ac106f50 0x7f30ac19cc70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f30a4006010 tx=0x7f30a4003c20 comp rx=0 tx=0).stop 2024-09-11T22:58:43.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b11a3640 1 -- 172.21.15.152:0/4196928277 shutdown_connections 2024-09-11T22:58:43.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b11a3640 1 --2- 172.21.15.152:0/4196928277 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f30940a2cf0 0x7f30940a51b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b11a3640 1 --2- 172.21.15.152:0/4196928277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f30ac106f50 0x7f30ac19cc70 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b11a3640 1 --2- 172.21.15.152:0/4196928277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30ac100ca0 0x7f30ac1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b11a3640 1 --2- 172.21.15.152:0/4196928277 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30ac1002f0 0x7f30ac1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:43.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.521+0000 7f30b11a3640 1 -- 172.21.15.152:0/4196928277 >> 172.21.15.152:0/4196928277 conn(0x7f30ac077210 msgr2=0x7f30ac105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:43.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.522+0000 7f30b11a3640 1 -- 172.21.15.152:0/4196928277 shutdown_connections 2024-09-11T22:58:43.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:43.522+0000 7f30b11a3640 1 -- 172.21.15.152:0/4196928277 wait complete. 2024-09-11T22:58:43.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:43 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T22:58:43.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:43 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T22:58:44.334 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:58:44.334 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:58:44.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:44 smithi191 ceph-mon[35211]: pgmap v178: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:44.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:44 smithi204 ceph-mon[35454]: pgmap v178: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:44.581 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:44.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:44 smithi152 ceph-mon[31181]: pgmap v178: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:45.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.090+0000 7fdbe5349640 1 -- 172.21.15.152:0/1544338235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe0101430 msgr2=0x7fdbe010d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:45.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.090+0000 7fdbe5349640 1 --2- 172.21.15.152:0/1544338235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe0101430 0x7fdbe010d960 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009cd0 tx=0x7fdbd802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:58:45.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.090+0000 7fdbe5349640 1 -- 172.21.15.152:0/1544338235 shutdown_connections 2024-09-11T22:58:45.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.090+0000 7fdbe5349640 1 --2- 172.21.15.152:0/1544338235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe0101430 0x7fdbe010d960 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:45.119 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.090+0000 7fdbe5349640 1 --2- 172.21.15.152:0/1544338235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbe0100a90 0x7fdbe0100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:45.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.090+0000 7fdbe5349640 1 --2- 172.21.15.152:0/1544338235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbe0106a90 0x7fdbe0106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:45.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.090+0000 7fdbe5349640 1 -- 172.21.15.152:0/1544338235 >> 172.21.15.152:0/1544338235 conn(0x7fdbe00fc7d0 msgr2=0x7fdbe00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:45.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.090+0000 7fdbe5349640 1 -- 172.21.15.152:0/1544338235 shutdown_connections 2024-09-11T22:58:45.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.090+0000 7fdbe5349640 1 -- 172.21.15.152:0/1544338235 wait complete. 2024-09-11T22:58:45.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.091+0000 7fdbe5349640 1 Processor -- start 2024-09-11T22:58:45.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.091+0000 7fdbe5349640 1 -- start start 2024-09-11T22:58:45.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.091+0000 7fdbe5349640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbe0100a90 0x7fdbe0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:45.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.091+0000 7fdbe5349640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbe0101430 0x7fdbe0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:45.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.091+0000 7fdbe5349640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe0106a90 0x7fdbe019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:45.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.091+0000 7fdbe5349640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe0111fb0 con 0x7fdbe0106a90 2024-09-11T22:58:45.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.091+0000 7fdbe5349640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe0111e30 con 0x7fdbe0100a90 2024-09-11T22:58:45.120 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.091+0000 7fdbe5349640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe0112130 con 0x7fdbe0101430 2024-09-11T22:58:45.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.091+0000 7fdbdf7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe0106a90 0x7fdbe019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:45.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.091+0000 7fdbdf7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe0106a90 0x7fdbe019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53324/0 (socket says 172.21.15.152:53324) 2024-09-11T22:58:45.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.091+0000 7fdbdf7fe640 1 -- 172.21.15.152:0/2890613254 learned_addr learned my addr 172.21.15.152:0/2890613254 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:45.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.091+0000 7fdbdeffd640 1 --2- 172.21.15.152:0/2890613254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbe0100a90 0x7fdbe0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:45.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.091+0000 7fdbde7fc640 1 --2- 172.21.15.152:0/2890613254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbe0101430 0x7fdbe0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:45.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.092+0000 7fdbdf7fe640 1 -- 172.21.15.152:0/2890613254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbe0101430 msgr2=0x7fdbe0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:45.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.092+0000 7fdbdf7fe640 1 --2- 172.21.15.152:0/2890613254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbe0101430 0x7fdbe0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:45.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.092+0000 7fdbdf7fe640 1 -- 172.21.15.152:0/2890613254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbe0100a90 msgr2=0x7fdbe0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:45.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.092+0000 7fdbdf7fe640 1 --2- 172.21.15.152:0/2890613254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbe0100a90 0x7fdbe0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:45.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.092+0000 7fdbdf7fe640 1 -- 172.21.15.152:0/2890613254 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe019d3e0 con 0x7fdbe0106a90 2024-09-11T22:58:45.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.092+0000 7fdbdf7fe640 1 --2- 172.21.15.152:0/2890613254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe0106a90 0x7fdbe019cd00 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8005cc0 tx=0x7fdbd8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:45.121 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.092+0000 7fdbc7fff640 1 -- 172.21.15.152:0/2890613254 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8004040 con 0x7fdbe0106a90 2024-09-11T22:58:45.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.092+0000 7fdbc7fff640 1 -- 172.21.15.152:0/2890613254 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd80046c0 con 0x7fdbe0106a90 2024-09-11T22:58:45.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.092+0000 7fdbe5349640 1 -- 172.21.15.152:0/2890613254 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe019d670 con 0x7fdbe0106a90 2024-09-11T22:58:45.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.092+0000 7fdbc7fff640 1 -- 172.21.15.152:0/2890613254 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8046480 con 0x7fdbe0106a90 2024-09-11T22:58:45.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.092+0000 7fdbe5349640 1 -- 172.21.15.152:0/2890613254 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe01a4fb0 con 0x7fdbe0106a90 2024-09-11T22:58:45.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.093+0000 7fdbe5349640 1 -- 172.21.15.152:0/2890613254 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe0100ef0 con 0x7fdbe0106a90 2024-09-11T22:58:45.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.094+0000 7fdbc7fff640 1 -- 172.21.15.152:0/2890613254 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fdbd80305a0 con 0x7fdbe0106a90 2024-09-11T22:58:45.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.095+0000 7fdbc7fff640 1 --2- 172.21.15.152:0/2890613254 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fdbc80a2cf0 0x7fdbc80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:45.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.095+0000 7fdbdeffd640 1 --2- 172.21.15.152:0/2890613254 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fdbc80a2cf0 0x7fdbc80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:45.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.095+0000 7fdbc7fff640 1 -- 172.21.15.152:0/2890613254 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fdbd802e030 con 0x7fdbe0106a90 2024-09-11T22:58:45.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.095+0000 7fdbdeffd640 1 --2- 172.21.15.152:0/2890613254 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fdbc80a2cf0 0x7fdbc80a51b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000bb60 tx=0x7fdbd000b390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:45.122 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.099+0000 7fdbc7fff640 1 -- 172.21.15.152:0/2890613254 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdbd804b060 con 0x7fdbe0106a90 2024-09-11T22:58:45.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.301+0000 7fdbe5349640 1 -- 172.21.15.152:0/2890613254 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbe0109080 con 0x7fdbe0106a90 2024-09-11T22:58:45.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.302+0000 7fdbc7fff640 1 -- 172.21.15.152:0/2890613254 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7fdbd80bd7e0 con 0x7fdbe0106a90 2024-09-11T22:58:45.303 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:58:45.303 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:58:45.303 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:58:45.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.304+0000 7fdbe5349640 1 -- 172.21.15.152:0/2890613254 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fdbc80a2cf0 msgr2=0x7fdbc80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:45.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.304+0000 7fdbe5349640 1 --2- 172.21.15.152:0/2890613254 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fdbc80a2cf0 0x7fdbc80a51b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000bb60 tx=0x7fdbd000b390 comp rx=0 tx=0).stop 2024-09-11T22:58:45.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.304+0000 7fdbe5349640 1 -- 172.21.15.152:0/2890613254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe0106a90 msgr2=0x7fdbe019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:45.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.304+0000 7fdbe5349640 1 --2- 172.21.15.152:0/2890613254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe0106a90 0x7fdbe019cd00 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8005cc0 tx=0x7fdbd8003970 comp rx=0 tx=0).stop 2024-09-11T22:58:45.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.304+0000 7fdbe5349640 1 -- 172.21.15.152:0/2890613254 shutdown_connections 2024-09-11T22:58:45.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.304+0000 7fdbe5349640 1 --2- 172.21.15.152:0/2890613254 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fdbc80a2cf0 0x7fdbc80a51b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:45.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.304+0000 7fdbe5349640 1 --2- 172.21.15.152:0/2890613254 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbe0106a90 0x7fdbe019cd00 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:45.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.304+0000 7fdbe5349640 1 --2- 172.21.15.152:0/2890613254 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbe0101430 0x7fdbe0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:45.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.304+0000 7fdbe5349640 1 --2- 172.21.15.152:0/2890613254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbe0100a90 0x7fdbe0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:45.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.304+0000 7fdbe5349640 1 -- 172.21.15.152:0/2890613254 >> 172.21.15.152:0/2890613254 conn(0x7fdbe00fc7d0 msgr2=0x7fdbe00fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:45.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.304+0000 7fdbe5349640 1 -- 172.21.15.152:0/2890613254 shutdown_connections 2024-09-11T22:58:45.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:45.304+0000 7fdbe5349640 1 -- 172.21.15.152:0/2890613254 wait complete. 2024-09-11T22:58:46.162 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:58:46.414 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:46.535 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:46 smithi152 ceph-mon[31181]: pgmap v179: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:58:46.535 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:46 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2890613254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:58:46.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:46 smithi191 ceph-mon[35211]: pgmap v179: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:58:46.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:46 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2890613254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:58:46.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:46 smithi204 ceph-mon[35454]: pgmap v179: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:58:46.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:46 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2890613254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:58:46.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.937+0000 7f7982dd6640 1 -- 172.21.15.152:0/1716170386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f797c100e20 msgr2=0x7f797c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:46.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.937+0000 7f7982dd6640 1 --2- 172.21.15.152:0/1716170386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f797c100e20 0x7f797c101200 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f796c009900 tx=0x7f796c02a480 comp rx=0 tx=0).stop 2024-09-11T22:58:46.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.937+0000 7f7982dd6640 1 -- 172.21.15.152:0/1716170386 shutdown_connections 2024-09-11T22:58:46.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.937+0000 7f7982dd6640 1 --2- 172.21.15.152:0/1716170386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f797c102ef0 0x7f797c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.937+0000 7f7982dd6640 1 --2- 172.21.15.152:0/1716170386 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f797c102570 0x7f797c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.937+0000 7f7982dd6640 1 --2- 172.21.15.152:0/1716170386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f797c100e20 0x7f797c101200 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.937+0000 7f7982dd6640 1 -- 172.21.15.152:0/1716170386 >> 172.21.15.152:0/1716170386 conn(0x7f797c0fc7d0 msgr2=0x7f797c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:46.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.937+0000 7f7982dd6640 1 -- 172.21.15.152:0/1716170386 shutdown_connections 2024-09-11T22:58:46.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.937+0000 7f7982dd6640 1 -- 172.21.15.152:0/1716170386 wait complete. 2024-09-11T22:58:46.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.937+0000 7f7982dd6640 1 Processor -- start 2024-09-11T22:58:46.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.937+0000 7f7982dd6640 1 -- start start 2024-09-11T22:58:46.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.937+0000 7f7982dd6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f797c100e20 0x7f797c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.937+0000 7f7982dd6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f797c102570 0x7f797c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7982dd6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f797c102ef0 0x7f797c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7982dd6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f797c110fa0 con 0x7f797c102ef0 2024-09-11T22:58:46.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7982dd6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f797c110e20 con 0x7f797c102570 2024-09-11T22:58:46.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7982dd6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f797c111120 con 0x7f797c100e20 2024-09-11T22:58:46.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7981373640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f797c102ef0 0x7f797c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7981373640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f797c102ef0 0x7f797c19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53332/0 (socket says 172.21.15.152:53332) 2024-09-11T22:58:46.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7981373640 1 -- 172.21.15.152:0/1044294535 learned_addr learned my addr 172.21.15.152:0/1044294535 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:46.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7980b72640 1 --2- 172.21.15.152:0/1044294535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f797c100e20 0x7f797c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f797bfff640 1 --2- 172.21.15.152:0/1044294535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f797c102570 0x7f797c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7981373640 1 -- 172.21.15.152:0/1044294535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f797c100e20 msgr2=0x7f797c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:46.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7981373640 1 --2- 172.21.15.152:0/1044294535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f797c100e20 0x7f797c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7981373640 1 -- 172.21.15.152:0/1044294535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f797c102570 msgr2=0x7f797c198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:46.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7981373640 1 --2- 172.21.15.152:0/1044294535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f797c102570 0x7f797c198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7981373640 1 -- 172.21.15.152:0/1044294535 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797c19d490 con 0x7f797c102ef0 2024-09-11T22:58:46.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7981373640 1 --2- 172.21.15.152:0/1044294535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f797c102ef0 0x7f797c19cdb0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f797400c970 tx=0x7f797400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:46.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7979ffb640 1 -- 172.21.15.152:0/1044294535 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974007bb0 con 0x7f797c102ef0 2024-09-11T22:58:46.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7979ffb640 1 -- 172.21.15.152:0/1044294535 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7974007d50 con 0x7f797c102ef0 2024-09-11T22:58:46.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7982dd6640 1 -- 172.21.15.152:0/1044294535 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797c19d780 con 0x7f797c102ef0 2024-09-11T22:58:46.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.938+0000 7f7979ffb640 1 -- 172.21.15.152:0/1044294535 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974018800 con 0x7f797c102ef0 2024-09-11T22:58:46.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.939+0000 7f7982dd6640 1 -- 172.21.15.152:0/1044294535 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f797c1a5060 con 0x7f797c102ef0 2024-09-11T22:58:46.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.941+0000 7f7979ffb640 1 -- 172.21.15.152:0/1044294535 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f797400fa50 con 0x7f797c102ef0 2024-09-11T22:58:46.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.941+0000 7f79851af740 1 Processor -- start 2024-09-11T22:58:46.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.941+0000 7f79851af740 1 -- start start 2024-09-11T22:58:46.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.941+0000 7f79851af740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cc7b40 0x55ca75cc6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.941+0000 7f7979ffb640 1 --2- 172.21.15.152:0/1044294535 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f79640a2cf0 0x7f79640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.941+0000 7f79851af740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cc65d0 0x55ca75cc6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.941+0000 7f79851af740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cd1370 0x55ca75cd3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f79851af740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ca75c2c2f0 con 0x55ca75cd1370 2024-09-11T22:58:46.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f79851af740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ca75bd3cf0 con 0x55ca75cc65d0 2024-09-11T22:58:46.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f79851af740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ca75c282b0 con 0x55ca75cc7b40 2024-09-11T22:58:46.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f7980b72640 1 --2- 172.21.15.152:0/1044294535 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f79640a2cf0 0x7f79640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f7981373640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cd1370 0x55ca75cd3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f7981373640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cd1370 0x55ca75cd3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53344/0 (socket says 172.21.15.152:53344) 2024-09-11T22:58:46.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f7981373640 1 -- 172.21.15.152:0/4219279406 learned_addr learned my addr 172.21.15.152:0/4219279406 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:46.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f7979ffb640 1 -- 172.21.15.152:0/1044294535 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f79740cf3e0 con 0x7f797c102ef0 2024-09-11T22:58:46.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f797bfff640 1 --2- 172.21.15.152:0/4219279406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cc7b40 0x55ca75cc6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f797bfff640 1 --2- 172.21.15.152:0/4219279406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cc65d0 0x55ca75cc6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f7981373640 1 -- 172.21.15.152:0/4219279406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cc7b40 msgr2=0x55ca75cc6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:46.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f7981373640 1 --2- 172.21.15.152:0/4219279406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cc7b40 0x55ca75cc6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f7981373640 1 -- 172.21.15.152:0/4219279406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cc65d0 msgr2=0x55ca75cc6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:46.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f7981373640 1 --2- 172.21.15.152:0/4219279406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cc65d0 0x55ca75cc6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f7981373640 1 -- 172.21.15.152:0/4219279406 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca75bb83a0 con 0x55ca75cd1370 2024-09-11T22:58:46.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f7980b72640 1 --2- 172.21.15.152:0/1044294535 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f79640a2cf0 0x7f79640a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f796c000c00 tx=0x7f796c0058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:46.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f7981373640 1 --2- 172.21.15.152:0/4219279406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cd1370 0x55ca75cd3760 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f797400afd0 tx=0x7f797401baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfc6e657778997f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:46.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f7962ffd640 1 -- 172.21.15.152:0/4219279406 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797409c010 con 0x55ca75cd1370 2024-09-11T22:58:46.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f7962ffd640 1 -- 172.21.15.152:0/4219279406 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7974084e50 con 0x55ca75cd1370 2024-09-11T22:58:46.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.942+0000 7f7962ffd640 1 -- 172.21.15.152:0/4219279406 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974099ca0 con 0x55ca75cd1370 2024-09-11T22:58:46.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.943+0000 7f79851af740 1 -- 172.21.15.152:0/4219279406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cd1370 msgr2=0x55ca75cd3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:46.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.943+0000 7f79851af740 1 --2- 172.21.15.152:0/4219279406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cd1370 0x55ca75cd3760 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f797400afd0 tx=0x7f797401baf0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.943+0000 7f79851af740 1 -- 172.21.15.152:0/4219279406 shutdown_connections 2024-09-11T22:58:46.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.943+0000 7f79851af740 1 --2- 172.21.15.152:0/4219279406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cd1370 0x55ca75cd3760 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.943+0000 7f79851af740 1 --2- 172.21.15.152:0/4219279406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cc65d0 0x55ca75cc6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.943+0000 7f79851af740 1 --2- 172.21.15.152:0/4219279406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cc7b40 0x55ca75cc6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.943+0000 7f79851af740 1 -- 172.21.15.152:0/4219279406 >> 172.21.15.152:0/4219279406 conn(0x55ca75c8f310 msgr2=0x55ca75cc2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.943+0000 7f79851af740 1 -- 172.21.15.152:0/4219279406 shutdown_connections 2024-09-11T22:58:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.943+0000 7f79851af740 1 -- 172.21.15.152:0/4219279406 wait complete. 2024-09-11T22:58:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.943+0000 7f79851af740 1 -- start start 2024-09-11T22:58:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f79851af740 1 Processor -- start 2024-09-11T22:58:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f79851af740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cc65d0 0x55ca75cd0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f79851af740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75cce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f79851af740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cd1370 0x55ca75cce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f79851af740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ca75cd4010 con 0x55ca75cc7b40 2024-09-11T22:58:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f79851af740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ca75be7060 con 0x55ca75cc65d0 2024-09-11T22:58:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f79851af740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ca75cd4190 con 0x55ca75cd1370 2024-09-11T22:58:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f79851af740 10 client.0 _ll_register_callbacks cb 0x55ca7593a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f797bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75cce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f797bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75cce080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53360/0 (socket says 172.21.15.152:53360) 2024-09-11T22:58:46.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f797bfff640 1 -- 172.21.15.152:0/3580754145 learned_addr learned my addr 172.21.15.152:0/3580754145 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:46.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f7980b72640 1 --2- 172.21.15.152:0/3580754145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cc65d0 0x55ca75cd0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f797bfff640 1 -- 172.21.15.152:0/3580754145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cd1370 msgr2=0x55ca75cce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:46.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f797bfff640 1 --2- 172.21.15.152:0/3580754145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cd1370 0x55ca75cce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f797bfff640 1 -- 172.21.15.152:0/3580754145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cc65d0 msgr2=0x55ca75cd0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:46.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f797bfff640 1 --2- 172.21.15.152:0/3580754145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cc65d0 0x55ca75cd0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.944+0000 7f797bfff640 1 -- 172.21.15.152:0/3580754145 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca7599d3c0 con 0x55ca75cc7b40 2024-09-11T22:58:46.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f797bfff640 1 --2- 172.21.15.152:0/3580754145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75cce080 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f79700071a0 tx=0x7f79700072b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1e5ef449626800 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:46.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f795bfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:46.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f795bfff640 1 -- 172.21.15.152:0/3580754145 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970020060 con 0x55ca75cc7b40 2024-09-11T22:58:46.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f795bfff640 1 -- 172.21.15.152:0/3580754145 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f797000bdf0 con 0x55ca75cc7b40 2024-09-11T22:58:46.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f79851af740 1 -- 172.21.15.152:0/3580754145 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7974084e50 con 0x55ca75cc7b40 2024-09-11T22:58:46.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f795bfff640 1 -- 172.21.15.152:0/3580754145 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7970014040 con 0x55ca75cc7b40 2024-09-11T22:58:46.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f79851af740 10 client.17028 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:46.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f79851af740 1 -- 172.21.15.152:0/3580754145 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca75cdfb90 con 0x55ca75cc7b40 2024-09-11T22:58:46.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f795bfff640 1 -- 172.21.15.152:0/3580754145 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7974084e50 con 0x55ca75cc7b40 2024-09-11T22:58:46.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f795bfff640 1 -- 172.21.15.152:0/3580754145 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f797001aaa0 con 0x55ca75cc7b40 2024-09-11T22:58:46.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f79851af740 10 client.17028 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:46.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f79851af740 10 client.17028 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:58:46.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f79851af740 20 client.17028 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:46.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f79851af740 1 --2- 172.21.15.152:0/3580754145 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55ca75ce10a0 0x55ca75ce3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f79851af740 4 client.17028 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:58:46.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f79851af740 1 -- 172.21.15.152:0/3580754145 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca75c39860 con 0x55ca75ce10a0 2024-09-11T22:58:46.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.945+0000 7f7980b72640 1 --2- 172.21.15.152:0/3580754145 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55ca75ce10a0 0x55ca75ce3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.946+0000 7f795bfff640 10 client.17028 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:58:46.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.946+0000 7f7980b72640 1 --2- 172.21.15.152:0/3580754145 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55ca75ce10a0 0x55ca75ce3540 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b6e651e0ba7c96d7 server_cookie=2b6279dfeed921b in_seq=0 out_seq=0 2024-09-11T22:58:46.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.946+0000 7f795bfff640 1 -- 172.21.15.152:0/3580754145 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca75c39860 con 0x55ca75ce10a0 2024-09-11T22:58:46.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.946+0000 7f795bfff640 10 client.17028 handle_command_reply: tid=0 2024-09-11T22:58:46.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.946+0000 7f79851af740 1 client.17028 shutdown 2024-09-11T22:58:46.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.947+0000 7f79851af740 1 -- 172.21.15.152:0/3580754145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 msgr2=0x55ca75cce080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:46.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.947+0000 7f79851af740 1 --2- 172.21.15.152:0/3580754145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75cce080 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f79700071a0 tx=0x7f79700072b0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.947+0000 7f79851af740 1 -- 172.21.15.152:0/3580754145 shutdown_connections 2024-09-11T22:58:46.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.947+0000 7f79851af740 1 --2- 172.21.15.152:0/3580754145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cd1370 0x55ca75cce810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.947+0000 7f79851af740 1 --2- 172.21.15.152:0/3580754145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75cce080 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.947+0000 7f79851af740 1 --2- 172.21.15.152:0/3580754145 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55ca75ce10a0 0x55ca75ce3540 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.947+0000 7f79851af740 1 --2- 172.21.15.152:0/3580754145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cc65d0 0x55ca75cd0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.947+0000 7f79851af740 1 -- 172.21.15.152:0/3580754145 >> 172.21.15.152:0/3580754145 conn(0x55ca75c8f310 msgr2=0x55ca75cc8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:46.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.947+0000 7f795bfff640 0 client.17028 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:58:46.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.947+0000 7f79851af740 1 -- 172.21.15.152:0/3580754145 shutdown_connections 2024-09-11T22:58:46.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.947+0000 7f79851af740 1 -- 172.21.15.152:0/3580754145 wait complete. 2024-09-11T22:58:46.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.947+0000 7f79851af740 20 client.17028 trim_cache size 0 max 16384 2024-09-11T22:58:46.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f79851af740 1 Processor -- start 2024-09-11T22:58:46.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f79851af740 1 -- start start 2024-09-11T22:58:46.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f79851af740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cc65d0 0x55ca75cc2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f79851af740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75cd9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f79851af740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cd1370 0x55ca75cd9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f79851af740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f797c1ad7f0 con 0x55ca75cc7b40 2024-09-11T22:58:46.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f79851af740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f797c1af330 con 0x55ca75cd1370 2024-09-11T22:58:46.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f79851af740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f797c1adec0 con 0x55ca75cc65d0 2024-09-11T22:58:46.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f797bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75cd9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f797bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75cd9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53376/0 (socket says 172.21.15.152:53376) 2024-09-11T22:58:46.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f797bfff640 1 -- 172.21.15.152:0/739251009 learned_addr learned my addr 172.21.15.152:0/739251009 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:46.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f7980b72640 1 --2- 172.21.15.152:0/739251009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cc65d0 0x55ca75cc2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f7981373640 1 --2- 172.21.15.152:0/739251009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cd1370 0x55ca75cd9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f797bfff640 1 -- 172.21.15.152:0/739251009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cc65d0 msgr2=0x55ca75cc2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:46.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f797bfff640 1 --2- 172.21.15.152:0/739251009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cc65d0 0x55ca75cc2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f797bfff640 1 -- 172.21.15.152:0/739251009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cd1370 msgr2=0x55ca75cd9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:46.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f797bfff640 1 --2- 172.21.15.152:0/739251009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cd1370 0x55ca75cd9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f797bfff640 1 -- 172.21.15.152:0/739251009 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca75cc73a0 con 0x55ca75cc7b40 2024-09-11T22:58:46.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f7981373640 1 --2- 172.21.15.152:0/739251009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cd1370 0x55ca75cd9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:46.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.960+0000 7f797bfff640 1 --2- 172.21.15.152:0/739251009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75cd9800 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f797001ae70 tx=0x7f7970004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904bee1910cbfdc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:46.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.961+0000 7f79627fc640 1 -- 172.21.15.152:0/739251009 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797000f070 con 0x55ca75cc7b40 2024-09-11T22:58:46.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.961+0000 7f79627fc640 1 -- 172.21.15.152:0/739251009 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7970007c90 con 0x55ca75cc7b40 2024-09-11T22:58:46.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.961+0000 7f79627fc640 1 -- 172.21.15.152:0/739251009 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797001a620 con 0x55ca75cc7b40 2024-09-11T22:58:46.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.961+0000 7f79851af740 1 -- 172.21.15.152:0/739251009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 msgr2=0x55ca75cd9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:46.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.961+0000 7f79851af740 1 --2- 172.21.15.152:0/739251009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75cd9800 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f797001ae70 tx=0x7f7970004a40 comp rx=0 tx=0).stop 2024-09-11T22:58:46.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.961+0000 7f79851af740 1 -- 172.21.15.152:0/739251009 shutdown_connections 2024-09-11T22:58:46.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.961+0000 7f79851af740 1 --2- 172.21.15.152:0/739251009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cd1370 0x55ca75cd9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.961+0000 7f79851af740 1 --2- 172.21.15.152:0/739251009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75cd9800 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.961+0000 7f79851af740 1 --2- 172.21.15.152:0/739251009 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cc65d0 0x55ca75cc2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.961+0000 7f79851af740 1 -- 172.21.15.152:0/739251009 >> 172.21.15.152:0/739251009 conn(0x55ca75c8f310 msgr2=0x55ca75cc2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:46.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.961+0000 7f79851af740 1 -- 172.21.15.152:0/739251009 shutdown_connections 2024-09-11T22:58:46.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.961+0000 7f79851af740 1 -- 172.21.15.152:0/739251009 wait complete. 2024-09-11T22:58:46.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.962+0000 7f79851af740 1 -- start start 2024-09-11T22:58:46.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.962+0000 7f79851af740 1 Processor -- start 2024-09-11T22:58:46.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.962+0000 7f79851af740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cc65d0 0x55ca75d43da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.962+0000 7f79851af740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75d442e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.962+0000 7f79851af740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cd1370 0x55ca75d44820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.962+0000 7f79851af740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f797c1aec60 con 0x55ca75cc7b40 2024-09-11T22:58:46.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.962+0000 7f79851af740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f797c1ae590 con 0x55ca75cc65d0 2024-09-11T22:58:46.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.962+0000 7f79851af740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f797c1ab7b0 con 0x55ca75cd1370 2024-09-11T22:58:46.987 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:58:46.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.962+0000 7f79851af740 10 client.0 _ll_register_callbacks cb 0x7f797000b560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:46.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.962+0000 7f797bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75d442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.962+0000 7f797bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75d442e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53384/0 (socket says 172.21.15.152:53384) 2024-09-11T22:58:46.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.962+0000 7f797bfff640 1 -- 172.21.15.152:0/3278756268 learned_addr learned my addr 172.21.15.152:0/3278756268 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:46.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.962+0000 7f7980b72640 1 --2- 172.21.15.152:0/3278756268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cc65d0 0x55ca75d43da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.962+0000 7f7981373640 1 --2- 172.21.15.152:0/3278756268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cd1370 0x55ca75d44820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f797bfff640 1 -- 172.21.15.152:0/3278756268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cd1370 msgr2=0x55ca75d44820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:46.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f797bfff640 1 --2- 172.21.15.152:0/3278756268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cd1370 0x55ca75d44820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f797bfff640 1 -- 172.21.15.152:0/3278756268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cc65d0 msgr2=0x55ca75d43da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:46.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f797bfff640 1 --2- 172.21.15.152:0/3278756268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cc65d0 0x55ca75d43da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f797bfff640 1 -- 172.21.15.152:0/3278756268 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca75d4efc0 con 0x55ca75cc7b40 2024-09-11T22:58:46.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f797bfff640 1 --2- 172.21.15.152:0/3278756268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75d442e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7970019840 tx=0x7f797000b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3659a18b13938e3a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:46.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f795bfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:46.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f795bfff640 1 -- 172.21.15.152:0/3278756268 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970039060 con 0x55ca75cc7b40 2024-09-11T22:58:46.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f795bfff640 1 -- 172.21.15.152:0/3278756268 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f797000b240 con 0x55ca75cc7b40 2024-09-11T22:58:46.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f79851af740 1 -- 172.21.15.152:0/3278756268 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7970007c90 con 0x55ca75cc7b40 2024-09-11T22:58:46.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f795bfff640 1 -- 172.21.15.152:0/3278756268 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f797000f040 con 0x55ca75cc7b40 2024-09-11T22:58:46.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f79851af740 10 client.17034 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:46.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f79851af740 1 -- 172.21.15.152:0/3278756268 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca75d50210 con 0x55ca75cc7b40 2024-09-11T22:58:46.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f795bfff640 1 -- 172.21.15.152:0/3278756268 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7970007c90 con 0x55ca75cc7b40 2024-09-11T22:58:46.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f795bfff640 1 -- 172.21.15.152:0/3278756268 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f797001b480 con 0x55ca75cc7b40 2024-09-11T22:58:46.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f79851af740 10 client.17034 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:46.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f79851af740 10 client.17034 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:58:46.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.963+0000 7f79851af740 20 client.17034 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:46.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f79851af740 1 --2- 172.21.15.152:0/3278756268 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55ca75d50870 0x55ca75d52c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:46.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f79851af740 4 client.17034 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:58:46.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f79851af740 1 -- 172.21.15.152:0/3278756268 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca75c33d70 con 0x55ca75d50870 2024-09-11T22:58:46.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f7980b72640 1 --2- 172.21.15.152:0/3278756268 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55ca75d50870 0x55ca75d52c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:46.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f795bfff640 10 client.17034 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:58:46.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f7980b72640 1 --2- 172.21.15.152:0/3278756268 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55ca75d50870 0x55ca75d52c90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e7c6af27ab1397cc server_cookie=73601524049946ff in_seq=0 out_seq=0 2024-09-11T22:58:46.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f795bfff640 1 -- 172.21.15.152:0/3278756268 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca75c33d70 con 0x55ca75d50870 2024-09-11T22:58:46.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f795bfff640 10 client.17034 handle_command_reply: tid=0 2024-09-11T22:58:46.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f79851af740 1 client.17034 shutdown 2024-09-11T22:58:46.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f79851af740 1 -- 172.21.15.152:0/3278756268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 msgr2=0x55ca75d442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:46.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f79851af740 1 --2- 172.21.15.152:0/3278756268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75d442e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7970019840 tx=0x7f797000b770 comp rx=0 tx=0).stop 2024-09-11T22:58:46.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f79851af740 1 -- 172.21.15.152:0/3278756268 shutdown_connections 2024-09-11T22:58:46.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f79851af740 1 --2- 172.21.15.152:0/3278756268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca75cd1370 0x55ca75d44820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f79851af740 1 --2- 172.21.15.152:0/3278756268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca75cc7b40 0x55ca75d442e0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f79851af740 1 --2- 172.21.15.152:0/3278756268 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55ca75d50870 0x55ca75d52c90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f79851af740 1 --2- 172.21.15.152:0/3278756268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca75cc65d0 0x55ca75d43da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f795bfff640 0 client.17034 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:58:46.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f79851af740 1 -- 172.21.15.152:0/3278756268 >> 172.21.15.152:0/3278756268 conn(0x55ca75c8f310 msgr2=0x55ca75cc14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:46.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f79851af740 1 -- 172.21.15.152:0/3278756268 shutdown_connections 2024-09-11T22:58:46.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.964+0000 7f79851af740 1 -- 172.21.15.152:0/3278756268 wait complete. 2024-09-11T22:58:46.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.965+0000 7f79851af740 20 client.17034 trim_cache size 0 max 16384 2024-09-11T22:58:46.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.965+0000 7f7982dd6640 1 -- 172.21.15.152:0/1044294535 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f79640a2cf0 msgr2=0x7f79640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:46.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.965+0000 7f7982dd6640 1 --2- 172.21.15.152:0/1044294535 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f79640a2cf0 0x7f79640a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f796c000c00 tx=0x7f796c0058d0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.965+0000 7f7982dd6640 1 -- 172.21.15.152:0/1044294535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f797c102ef0 msgr2=0x7f797c19cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:46.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.965+0000 7f7982dd6640 1 --2- 172.21.15.152:0/1044294535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f797c102ef0 0x7f797c19cdb0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f797400c970 tx=0x7f797400ce30 comp rx=0 tx=0).stop 2024-09-11T22:58:46.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.965+0000 7f7982dd6640 1 -- 172.21.15.152:0/1044294535 shutdown_connections 2024-09-11T22:58:46.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.965+0000 7f7982dd6640 1 --2- 172.21.15.152:0/1044294535 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f79640a2cf0 0x7f79640a51b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.965+0000 7f7982dd6640 1 --2- 172.21.15.152:0/1044294535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f797c102ef0 0x7f797c19cdb0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.995 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.965+0000 7f7982dd6640 1 --2- 172.21.15.152:0/1044294535 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f797c102570 0x7f797c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.965+0000 7f7982dd6640 1 --2- 172.21.15.152:0/1044294535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f797c100e20 0x7f797c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:46.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.965+0000 7f7982dd6640 1 -- 172.21.15.152:0/1044294535 >> 172.21.15.152:0/1044294535 conn(0x7f797c0fc7d0 msgr2=0x7f797c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:46.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.965+0000 7f7982dd6640 1 -- 172.21.15.152:0/1044294535 shutdown_connections 2024-09-11T22:58:46.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:46.965+0000 7f7982dd6640 1 -- 172.21.15.152:0/1044294535 wait complete. 2024-09-11T22:58:47.873 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:58:48.121 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:48.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:48 smithi191 ceph-mon[35211]: pgmap v180: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:58:48.544 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:48 smithi152 ceph-mon[31181]: pgmap v180: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:58:48.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:48 smithi204 ceph-mon[35454]: pgmap v180: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:58:48.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.623+0000 7fa67d3c6640 1 -- 172.21.15.152:0/447781510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6781036e0 msgr2=0x7fa678107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:48.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.623+0000 7fa67d3c6640 1 --2- 172.21.15.152:0/447781510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6781036e0 0x7fa678107b90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009930 tx=0x7fa66c02a420 comp rx=0 tx=0).stop 2024-09-11T22:58:48.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.623+0000 7fa67d3c6640 1 -- 172.21.15.152:0/447781510 shutdown_connections 2024-09-11T22:58:48.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.623+0000 7fa67d3c6640 1 --2- 172.21.15.152:0/447781510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6781082a0 0x7fa67810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.623+0000 7fa67d3c6640 1 --2- 172.21.15.152:0/447781510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6781036e0 0x7fa678107b90 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.623+0000 7fa67d3c6640 1 --2- 172.21.15.152:0/447781510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa678102d30 0x7fa678103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.623+0000 7fa67d3c6640 1 -- 172.21.15.152:0/447781510 >> 172.21.15.152:0/447781510 conn(0x7fa6780fc7d0 msgr2=0x7fa6780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:48.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.623+0000 7fa67d3c6640 1 -- 172.21.15.152:0/447781510 shutdown_connections 2024-09-11T22:58:48.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.623+0000 7fa67d3c6640 1 -- 172.21.15.152:0/447781510 wait complete. 2024-09-11T22:58:48.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.623+0000 7fa67d3c6640 1 Processor -- start 2024-09-11T22:58:48.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.623+0000 7fa67d3c6640 1 -- start start 2024-09-11T22:58:48.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa67d3c6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa678102d30 0x7fa67819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa67d3c6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6781036e0 0x7fa67819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa67d3c6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6781082a0 0x7fa67819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa67d3c6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa678112060 con 0x7fa678102d30 2024-09-11T22:58:48.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa67d3c6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa678111ee0 con 0x7fa6781036e0 2024-09-11T22:58:48.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa67d3c6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6781121e0 con 0x7fa6781082a0 2024-09-11T22:58:48.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa676ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa678102d30 0x7fa67819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:48.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa676ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa678102d30 0x7fa67819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52086/0 (socket says 172.21.15.152:52086) 2024-09-11T22:58:48.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa676ffd640 1 -- 172.21.15.152:0/3345895454 learned_addr learned my addr 172.21.15.152:0/3345895454 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:48.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa6767fc640 1 --2- 172.21.15.152:0/3345895454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6781036e0 0x7fa67819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:48.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa6777fe640 1 --2- 172.21.15.152:0/3345895454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6781082a0 0x7fa67819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:48.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa676ffd640 1 -- 172.21.15.152:0/3345895454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6781082a0 msgr2=0x7fa67819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:48.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa676ffd640 1 --2- 172.21.15.152:0/3345895454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6781082a0 0x7fa67819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa676ffd640 1 -- 172.21.15.152:0/3345895454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6781036e0 msgr2=0x7fa67819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:48.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa676ffd640 1 --2- 172.21.15.152:0/3345895454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6781036e0 0x7fa67819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa676ffd640 1 -- 172.21.15.152:0/3345895454 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67819f700 con 0x7fa678102d30 2024-09-11T22:58:48.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa6777fe640 1 --2- 172.21.15.152:0/3345895454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6781082a0 0x7fa67819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:48.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa676ffd640 1 --2- 172.21.15.152:0/3345895454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa678102d30 0x7fa67819a750 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa66800cce0 tx=0x7fa66800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:48.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa65ffff640 1 -- 172.21.15.152:0/3345895454 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66800f650 con 0x7fa678102d30 2024-09-11T22:58:48.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.624+0000 7fa65ffff640 1 -- 172.21.15.152:0/3345895454 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa66800fcd0 con 0x7fa678102d30 2024-09-11T22:58:48.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.625+0000 7fa67d3c6640 1 -- 172.21.15.152:0/3345895454 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67819f9f0 con 0x7fa678102d30 2024-09-11T22:58:48.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.625+0000 7fa65ffff640 1 -- 172.21.15.152:0/3345895454 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa668018c70 con 0x7fa678102d30 2024-09-11T22:58:48.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.625+0000 7fa67d3c6640 1 -- 172.21.15.152:0/3345895454 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6781a72d0 con 0x7fa678102d30 2024-09-11T22:58:48.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.627+0000 7fa65ffff640 1 -- 172.21.15.152:0/3345895454 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa66800f7f0 con 0x7fa678102d30 2024-09-11T22:58:48.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.627+0000 7fa67f79f740 1 Processor -- start 2024-09-11T22:58:48.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.627+0000 7fa67f79f740 1 -- start start 2024-09-11T22:58:48.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.627+0000 7fa67f79f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19ad0b40 0x55fb19acf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.627+0000 7fa67f79f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19acf5d0 0x55fb19acfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.627+0000 7fa67f79f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ada370 0x55fb19adc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.627+0000 7fa67f79f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55fb19a352f0 con 0x55fb19acf5d0 2024-09-11T22:58:48.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.627+0000 7fa67f79f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55fb199dccf0 con 0x55fb19ada370 2024-09-11T22:58:48.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.627+0000 7fa67f79f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fb19a312b0 con 0x55fb19ad0b40 2024-09-11T22:58:48.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.627+0000 7fa676ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19acf5d0 0x55fb19acfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:48.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.627+0000 7fa676ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19acf5d0 0x55fb19acfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52098/0 (socket says 172.21.15.152:52098) 2024-09-11T22:58:48.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.627+0000 7fa676ffd640 1 -- 172.21.15.152:0/3467323428 learned_addr learned my addr 172.21.15.152:0/3467323428 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:48.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.628+0000 7fa6767fc640 1 --2- 172.21.15.152:0/3467323428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ada370 0x55fb19adc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:48.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.628+0000 7fa6777fe640 1 --2- 172.21.15.152:0/3467323428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19ad0b40 0x55fb19acf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:48.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.628+0000 7fa65ffff640 1 --2- 172.21.15.152:0/3345895454 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6600a2cf0 0x7fa6600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.628+0000 7fa676ffd640 1 -- 172.21.15.152:0/3467323428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19ad0b40 msgr2=0x55fb19acf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:48.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.628+0000 7fa676ffd640 1 --2- 172.21.15.152:0/3467323428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19ad0b40 0x55fb19acf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.628+0000 7fa676ffd640 1 -- 172.21.15.152:0/3467323428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ada370 msgr2=0x55fb19adc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:48.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.628+0000 7fa676ffd640 1 --2- 172.21.15.152:0/3467323428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ada370 0x55fb19adc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.628+0000 7fa65ffff640 1 -- 172.21.15.152:0/3345895454 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa6680cf8b0 con 0x7fa678102d30 2024-09-11T22:58:48.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.628+0000 7fa676ffd640 1 -- 172.21.15.152:0/3467323428 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb199c13a0 con 0x55fb19acf5d0 2024-09-11T22:58:48.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.628+0000 7fa6777fe640 1 --2- 172.21.15.152:0/3467323428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19ad0b40 0x55fb19acf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:48.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.628+0000 7fa6777fe640 1 --2- 172.21.15.152:0/3345895454 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6600a2cf0 0x7fa6600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:48.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.628+0000 7fa676ffd640 1 --2- 172.21.15.152:0/3467323428 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19acf5d0 0x55fb19acfa10 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fa6680ccf90 tx=0x7fa668015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985baba0735525b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:48.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.628+0000 7fa65d7fa640 1 -- 172.21.15.152:0/3467323428 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66809bbf0 con 0x55fb19acf5d0 2024-09-11T22:58:48.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.628+0000 7fa65d7fa640 1 -- 172.21.15.152:0/3467323428 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa66809c520 con 0x55fb19acf5d0 2024-09-11T22:58:48.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.628+0000 7fa65d7fa640 1 -- 172.21.15.152:0/3467323428 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66809b780 con 0x55fb19acf5d0 2024-09-11T22:58:48.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.629+0000 7fa67f79f740 1 -- 172.21.15.152:0/3467323428 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19acf5d0 msgr2=0x55fb19acfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:48.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.629+0000 7fa6777fe640 1 --2- 172.21.15.152:0/3345895454 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6600a2cf0 0x7fa6600a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa670009b30 tx=0x7fa6700046c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:48.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.629+0000 7fa67f79f740 1 --2- 172.21.15.152:0/3467323428 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19acf5d0 0x55fb19acfa10 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fa6680ccf90 tx=0x7fa668015000 comp rx=0 tx=0).stop 2024-09-11T22:58:48.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.629+0000 7fa67f79f740 1 -- 172.21.15.152:0/3467323428 shutdown_connections 2024-09-11T22:58:48.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.629+0000 7fa67f79f740 1 --2- 172.21.15.152:0/3467323428 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ada370 0x55fb19adc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.629+0000 7fa67f79f740 1 --2- 172.21.15.152:0/3467323428 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19acf5d0 0x55fb19acfa10 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.629+0000 7fa67f79f740 1 --2- 172.21.15.152:0/3467323428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19ad0b40 0x55fb19acf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.629+0000 7fa67f79f740 1 -- 172.21.15.152:0/3467323428 >> 172.21.15.152:0/3467323428 conn(0x55fb19a98310 msgr2=0x55fb19acbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:48.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.629+0000 7fa67f79f740 1 -- 172.21.15.152:0/3467323428 shutdown_connections 2024-09-11T22:58:48.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.629+0000 7fa67f79f740 1 -- 172.21.15.152:0/3467323428 wait complete. 2024-09-11T22:58:48.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.629+0000 7fa67f79f740 1 -- start start 2024-09-11T22:58:48.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.629+0000 7fa67f79f740 1 Processor -- start 2024-09-11T22:58:48.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa67f79f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19acf5d0 0x55fb19ad9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa67f79f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ad0b40 0x55fb19ad7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa67f79f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ada370 0x55fb19ad7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa67f79f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55fb19add010 con 0x55fb19ad0b40 2024-09-11T22:58:48.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa67f79f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55fb199f0060 con 0x55fb19ada370 2024-09-11T22:58:48.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa67f79f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55fb19add190 con 0x55fb19acf5d0 2024-09-11T22:58:48.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa67f79f740 10 client.0 _ll_register_callbacks cb 0x55fb197438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:48.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa6767fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ad0b40 0x55fb19ad7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:48.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa6767fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ad0b40 0x55fb19ad7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52114/0 (socket says 172.21.15.152:52114) 2024-09-11T22:58:48.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa6767fc640 1 -- 172.21.15.152:0/1983167900 learned_addr learned my addr 172.21.15.152:0/1983167900 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:48.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa676ffd640 1 --2- 172.21.15.152:0/1983167900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19acf5d0 0x55fb19ad9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:48.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa6777fe640 1 --2- 172.21.15.152:0/1983167900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ada370 0x55fb19ad7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:48.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa6767fc640 1 -- 172.21.15.152:0/1983167900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19acf5d0 msgr2=0x55fb19ad9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:48.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa6767fc640 1 --2- 172.21.15.152:0/1983167900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19acf5d0 0x55fb19ad9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa6767fc640 1 -- 172.21.15.152:0/1983167900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ada370 msgr2=0x55fb19ad7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:48.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa6767fc640 1 --2- 172.21.15.152:0/1983167900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ada370 0x55fb19ad7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa6767fc640 1 -- 172.21.15.152:0/1983167900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb197a63c0 con 0x55fb19ad0b40 2024-09-11T22:58:48.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa6777fe640 1 --2- 172.21.15.152:0/1983167900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ada370 0x55fb19ad7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:48.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa6767fc640 1 --2- 172.21.15.152:0/1983167900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ad0b40 0x55fb19ad7070 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa66c000c00 tx=0x7fa66c0315d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abf908ee25e019 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:48.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa65aff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:48.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa65aff5640 1 -- 172.21.15.152:0/1983167900 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c035040 con 0x55fb19ad0b40 2024-09-11T22:58:48.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa65aff5640 1 -- 172.21.15.152:0/1983167900 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa66c004040 con 0x55fb19ad0b40 2024-09-11T22:58:48.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.630+0000 7fa67f79f740 1 -- 172.21.15.152:0/1983167900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa66809c520 con 0x55fb19ad0b40 2024-09-11T22:58:48.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.631+0000 7fa65aff5640 1 -- 172.21.15.152:0/1983167900 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa66c004240 con 0x55fb19ad0b40 2024-09-11T22:58:48.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.631+0000 7fa67f79f740 10 client.17046 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:48.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.631+0000 7fa67f79f740 1 -- 172.21.15.152:0/1983167900 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fb19ae8a90 con 0x55fb19ad0b40 2024-09-11T22:58:48.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.631+0000 7fa65aff5640 1 -- 172.21.15.152:0/1983167900 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa66809c520 con 0x55fb19ad0b40 2024-09-11T22:58:48.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.631+0000 7fa65aff5640 1 -- 172.21.15.152:0/1983167900 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa66c004a70 con 0x55fb19ad0b40 2024-09-11T22:58:48.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.631+0000 7fa67f79f740 10 client.17046 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:48.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.631+0000 7fa67f79f740 10 client.17046 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:58:48.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.631+0000 7fa67f79f740 20 client.17046 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:48.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.631+0000 7fa67f79f740 1 --2- 172.21.15.152:0/1983167900 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55fb19ae9ea0 0x55fb19aec2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.631+0000 7fa67f79f740 4 client.17046 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:58:48.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.631+0000 7fa67f79f740 1 -- 172.21.15.152:0/1983167900 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb19a42860 con 0x55fb19ae9ea0 2024-09-11T22:58:48.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.631+0000 7fa676ffd640 1 --2- 172.21.15.152:0/1983167900 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55fb19ae9ea0 0x55fb19aec2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:48.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.632+0000 7fa65aff5640 10 client.17046 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:58:48.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.632+0000 7fa676ffd640 1 --2- 172.21.15.152:0/1983167900 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55fb19ae9ea0 0x55fb19aec2c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=aefbf65d03e98e90 server_cookie=eb73c53d4d68fe48 in_seq=0 out_seq=0 2024-09-11T22:58:48.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.632+0000 7fa65aff5640 1 -- 172.21.15.152:0/1983167900 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb19a42860 con 0x55fb19ae9ea0 2024-09-11T22:58:48.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.632+0000 7fa65aff5640 10 client.17046 handle_command_reply: tid=0 2024-09-11T22:58:48.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.632+0000 7fa67f79f740 1 client.17046 shutdown 2024-09-11T22:58:48.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.633+0000 7fa67f79f740 1 -- 172.21.15.152:0/1983167900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ad0b40 msgr2=0x55fb19ad7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:48.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.633+0000 7fa67f79f740 1 --2- 172.21.15.152:0/1983167900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ad0b40 0x55fb19ad7070 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa66c000c00 tx=0x7fa66c0315d0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.633+0000 7fa67f79f740 1 -- 172.21.15.152:0/1983167900 shutdown_connections 2024-09-11T22:58:48.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.633+0000 7fa67f79f740 1 --2- 172.21.15.152:0/1983167900 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55fb19ae9ea0 0x55fb19aec2c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.633+0000 7fa67f79f740 1 --2- 172.21.15.152:0/1983167900 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ada370 0x55fb19ad7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.633+0000 7fa67f79f740 1 --2- 172.21.15.152:0/1983167900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ad0b40 0x55fb19ad7070 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.633+0000 7fa65aff5640 0 client.17046 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:58:48.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.633+0000 7fa67f79f740 1 --2- 172.21.15.152:0/1983167900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19acf5d0 0x55fb19ad9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.633+0000 7fa67f79f740 1 -- 172.21.15.152:0/1983167900 >> 172.21.15.152:0/1983167900 conn(0x55fb19a98310 msgr2=0x55fb19ad1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:48.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.633+0000 7fa67f79f740 1 -- 172.21.15.152:0/1983167900 shutdown_connections 2024-09-11T22:58:48.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.633+0000 7fa67f79f740 1 -- 172.21.15.152:0/1983167900 wait complete. 2024-09-11T22:58:48.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.633+0000 7fa67f79f740 20 client.17046 trim_cache size 0 max 16384 2024-09-11T22:58:48.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa67f79f740 1 Processor -- start 2024-09-11T22:58:48.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa67f79f740 1 -- start start 2024-09-11T22:58:48.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa67f79f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19acf5d0 0x55fb19ad0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa67f79f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ad0b40 0x55fb19ad6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa67f79f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ada370 0x55fb19af1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa67f79f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa6781af970 con 0x55fb19ad0b40 2024-09-11T22:58:48.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa67f79f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa6781b14b0 con 0x55fb19ada370 2024-09-11T22:58:48.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa67f79f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6781b0040 con 0x55fb19acf5d0 2024-09-11T22:58:48.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa6767fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ad0b40 0x55fb19ad6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:48.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa6767fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ad0b40 0x55fb19ad6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52122/0 (socket says 172.21.15.152:52122) 2024-09-11T22:58:48.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa6767fc640 1 -- 172.21.15.152:0/1674031085 learned_addr learned my addr 172.21.15.152:0/1674031085 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:48.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa6767fc640 1 -- 172.21.15.152:0/1674031085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19acf5d0 msgr2=0x55fb19ad0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:48.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa6767fc640 1 --2- 172.21.15.152:0/1674031085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19acf5d0 0x55fb19ad0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa6767fc640 1 -- 172.21.15.152:0/1674031085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ada370 msgr2=0x55fb19af1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:48.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa6767fc640 1 --2- 172.21.15.152:0/1674031085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ada370 0x55fb19af1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa6767fc640 1 -- 172.21.15.152:0/1674031085 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb19ada130 con 0x55fb19ad0b40 2024-09-11T22:58:48.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.646+0000 7fa6767fc640 1 --2- 172.21.15.152:0/1674031085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ad0b40 0x55fb19ad6050 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa66c005940 tx=0x7fa66c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ba05c8f0bfc42 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:48.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.647+0000 7fa65cff9640 1 -- 172.21.15.152:0/1674031085 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c055030 con 0x55fb19ad0b40 2024-09-11T22:58:48.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.647+0000 7fa65cff9640 1 -- 172.21.15.152:0/1674031085 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa66c004c50 con 0x55fb19ad0b40 2024-09-11T22:58:48.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.647+0000 7fa65cff9640 1 -- 172.21.15.152:0/1674031085 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c035070 con 0x55fb19ad0b40 2024-09-11T22:58:48.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.647+0000 7fa67f79f740 1 -- 172.21.15.152:0/1674031085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ad0b40 msgr2=0x55fb19ad6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:48.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.647+0000 7fa67f79f740 1 --2- 172.21.15.152:0/1674031085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ad0b40 0x55fb19ad6050 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa66c005940 tx=0x7fa66c02a420 comp rx=0 tx=0).stop 2024-09-11T22:58:48.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.647+0000 7fa67f79f740 1 -- 172.21.15.152:0/1674031085 shutdown_connections 2024-09-11T22:58:48.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.647+0000 7fa67f79f740 1 --2- 172.21.15.152:0/1674031085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ada370 0x55fb19af1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.647+0000 7fa67f79f740 1 --2- 172.21.15.152:0/1674031085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ad0b40 0x55fb19ad6050 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.647+0000 7fa67f79f740 1 --2- 172.21.15.152:0/1674031085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19acf5d0 0x55fb19ad0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.647+0000 7fa67f79f740 1 -- 172.21.15.152:0/1674031085 >> 172.21.15.152:0/1674031085 conn(0x55fb19a98310 msgr2=0x55fb19ad5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:48.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.647+0000 7fa67f79f740 1 -- 172.21.15.152:0/1674031085 shutdown_connections 2024-09-11T22:58:48.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.647+0000 7fa67f79f740 1 -- 172.21.15.152:0/1674031085 wait complete. 2024-09-11T22:58:48.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa67f79f740 1 -- start start 2024-09-11T22:58:48.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa67f79f740 1 Processor -- start 2024-09-11T22:58:48.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa67f79f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19acf5d0 0x55fb19ae2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa67f79f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ad0b40 0x55fb19ade020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa67f79f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ada370 0x55fb19ade7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa67f79f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa6781b0de0 con 0x55fb19ada370 2024-09-11T22:58:48.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa67f79f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa6781b0710 con 0x55fb19ad0b40 2024-09-11T22:58:48.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa67f79f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6781ad930 con 0x55fb19acf5d0 2024-09-11T22:58:48.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa67f79f740 10 client.0 _ll_register_callbacks cb 0x7fa66c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:48.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa6777fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ada370 0x55fb19ade7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:48.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa6777fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ada370 0x55fb19ade7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52134/0 (socket says 172.21.15.152:52134) 2024-09-11T22:58:48.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa6777fe640 1 -- 172.21.15.152:0/169887634 learned_addr learned my addr 172.21.15.152:0/169887634 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:48.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa6767fc640 1 --2- 172.21.15.152:0/169887634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ad0b40 0x55fb19ade020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:48.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa676ffd640 1 --2- 172.21.15.152:0/169887634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19acf5d0 0x55fb19ae2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:48.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa6777fe640 1 -- 172.21.15.152:0/169887634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19acf5d0 msgr2=0x55fb19ae2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:48.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa6777fe640 1 --2- 172.21.15.152:0/169887634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19acf5d0 0x55fb19ae2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa6777fe640 1 -- 172.21.15.152:0/169887634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ad0b40 msgr2=0x55fb19ade020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:48.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa6777fe640 1 --2- 172.21.15.152:0/169887634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ad0b40 0x55fb19ade020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.648+0000 7fa6777fe640 1 -- 172.21.15.152:0/169887634 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb19b5c370 con 0x55fb19ada370 2024-09-11T22:58:48.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.649+0000 7fa676ffd640 1 --2- 172.21.15.152:0/169887634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19acf5d0 0x55fb19ae2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:48.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.649+0000 7fa6777fe640 1 --2- 172.21.15.152:0/169887634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ada370 0x55fb19ade7b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa6700114e0 tx=0x7fa6700118e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9bb75171d26631 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:48.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.649+0000 7fa65aff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:48.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.649+0000 7fa65aff5640 1 -- 172.21.15.152:0/169887634 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67001e910 con 0x55fb19ada370 2024-09-11T22:58:48.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.649+0000 7fa67f79f740 1 -- 172.21.15.152:0/169887634 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa66c004c50 con 0x55fb19ada370 2024-09-11T22:58:48.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.649+0000 7fa65aff5640 1 -- 172.21.15.152:0/169887634 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa67001eab0 con 0x55fb19ada370 2024-09-11T22:58:48.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.649+0000 7fa65aff5640 1 -- 172.21.15.152:0/169887634 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa67002b020 con 0x55fb19ada370 2024-09-11T22:58:48.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.649+0000 7fa67f79f740 10 client.17052 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:48.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.649+0000 7fa67f79f740 1 -- 172.21.15.152:0/169887634 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fb19b5d560 con 0x55fb19ada370 2024-09-11T22:58:48.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.649+0000 7fa65aff5640 1 -- 172.21.15.152:0/169887634 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa66c004c50 con 0x55fb19ada370 2024-09-11T22:58:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.649+0000 7fa65aff5640 1 -- 172.21.15.152:0/169887634 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa670027420 con 0x55fb19ada370 2024-09-11T22:58:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.649+0000 7fa67f79f740 10 client.17052 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.649+0000 7fa67f79f740 10 client.17052 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:58:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.649+0000 7fa67f79f740 20 client.17052 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.650+0000 7fa67f79f740 1 --2- 172.21.15.152:0/169887634 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55fb19b5dce0 0x55fb19b60100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.650+0000 7fa67f79f740 4 client.17052 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:58:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.650+0000 7fa67f79f740 1 -- 172.21.15.152:0/169887634 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb19a3cd70 con 0x55fb19b5dce0 2024-09-11T22:58:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.650+0000 7fa676ffd640 1 --2- 172.21.15.152:0/169887634 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55fb19b5dce0 0x55fb19b60100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.650+0000 7fa65aff5640 10 client.17052 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:58:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.650+0000 7fa676ffd640 1 --2- 172.21.15.152:0/169887634 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55fb19b5dce0 0x55fb19b60100 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e0ac158100c2f2e2 server_cookie=421d14c5fe0a9da6 in_seq=0 out_seq=0 2024-09-11T22:58:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.650+0000 7fa65aff5640 1 -- 172.21.15.152:0/169887634 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb19a3cd70 con 0x55fb19b5dce0 2024-09-11T22:58:48.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.650+0000 7fa65aff5640 10 client.17052 handle_command_reply: tid=0 2024-09-11T22:58:48.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67f79f740 1 client.17052 shutdown 2024-09-11T22:58:48.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67f79f740 1 -- 172.21.15.152:0/169887634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ada370 msgr2=0x55fb19ade7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:48.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67f79f740 1 --2- 172.21.15.152:0/169887634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ada370 0x55fb19ade7b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fa6700114e0 tx=0x7fa6700118e0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67f79f740 1 -- 172.21.15.152:0/169887634 shutdown_connections 2024-09-11T22:58:48.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67f79f740 1 --2- 172.21.15.152:0/169887634 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55fb19b5dce0 0x55fb19b60100 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67f79f740 1 --2- 172.21.15.152:0/169887634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb19ada370 0x55fb19ade7b0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67f79f740 1 --2- 172.21.15.152:0/169887634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fb19ad0b40 0x55fb19ade020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa65aff5640 0 client.17052 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:58:48.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67f79f740 1 --2- 172.21.15.152:0/169887634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb19acf5d0 0x55fb19ae2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67f79f740 1 -- 172.21.15.152:0/169887634 >> 172.21.15.152:0/169887634 conn(0x55fb19a98310 msgr2=0x55fb19ad1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:48.658 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67f79f740 1 -- 172.21.15.152:0/169887634 shutdown_connections 2024-09-11T22:58:48.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67f79f740 1 -- 172.21.15.152:0/169887634 wait complete. 2024-09-11T22:58:48.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67f79f740 20 client.17052 trim_cache size 0 max 16384 2024-09-11T22:58:48.659 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:58:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67d3c6640 1 -- 172.21.15.152:0/3345895454 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6600a2cf0 msgr2=0x7fa6600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67d3c6640 1 --2- 172.21.15.152:0/3345895454 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6600a2cf0 0x7fa6600a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa670009b30 tx=0x7fa6700046c0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67d3c6640 1 -- 172.21.15.152:0/3345895454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa678102d30 msgr2=0x7fa67819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67d3c6640 1 --2- 172.21.15.152:0/3345895454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa678102d30 0x7fa67819a750 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa66800cce0 tx=0x7fa66800a620 comp rx=0 tx=0).stop 2024-09-11T22:58:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67d3c6640 1 -- 172.21.15.152:0/3345895454 shutdown_connections 2024-09-11T22:58:48.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67d3c6640 1 --2- 172.21.15.152:0/3345895454 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6600a2cf0 0x7fa6600a51b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67d3c6640 1 --2- 172.21.15.152:0/3345895454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6781082a0 0x7fa67819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67d3c6640 1 --2- 172.21.15.152:0/3345895454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6781036e0 0x7fa67819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:48.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67d3c6640 1 --2- 172.21.15.152:0/3345895454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa678102d30 0x7fa67819a750 secure :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa66800cce0 tx=0x7fa66800a620 comp rx=0 tx=0).stop 2024-09-11T22:58:48.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.651+0000 7fa67d3c6640 1 -- 172.21.15.152:0/3345895454 >> 172.21.15.152:0/3345895454 conn(0x7fa6780fc7d0 msgr2=0x7fa678100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:48.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.652+0000 7fa67d3c6640 1 -- 172.21.15.152:0/3345895454 shutdown_connections 2024-09-11T22:58:48.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:48.652+0000 7fa67d3c6640 1 -- 172.21.15.152:0/3345895454 wait complete. 2024-09-11T22:58:49.477 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:58:49.723 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:50.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.261+0000 7fa69f542640 1 -- 172.21.15.152:0/7314979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa698102570 msgr2=0x7fa6981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:50.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.261+0000 7fa69f542640 1 --2- 172.21.15.152:0/7314979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa698102570 0x7fa6981029b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa688009900 tx=0x7fa68802a480 comp rx=0 tx=0).stop 2024-09-11T22:58:50.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.261+0000 7fa69f542640 1 -- 172.21.15.152:0/7314979 shutdown_connections 2024-09-11T22:58:50.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.261+0000 7fa69f542640 1 --2- 172.21.15.152:0/7314979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa698102ef0 0x7fa69810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.261+0000 7fa69f542640 1 --2- 172.21.15.152:0/7314979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa698102570 0x7fa6981029b0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.261+0000 7fa69f542640 1 --2- 172.21.15.152:0/7314979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa698100e20 0x7fa698101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.261+0000 7fa69f542640 1 -- 172.21.15.152:0/7314979 >> 172.21.15.152:0/7314979 conn(0x7fa6980fc7d0 msgr2=0x7fa6980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:50.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.261+0000 7fa69f542640 1 -- 172.21.15.152:0/7314979 shutdown_connections 2024-09-11T22:58:50.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.261+0000 7fa69f542640 1 -- 172.21.15.152:0/7314979 wait complete. 2024-09-11T22:58:50.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.261+0000 7fa69f542640 1 Processor -- start 2024-09-11T22:58:50.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.261+0000 7fa69f542640 1 -- start start 2024-09-11T22:58:50.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69f542640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa698100e20 0x7fa698197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69f542640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa698102570 0x7fa6981978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69f542640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa698102ef0 0x7fa69819bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69f542640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa698110f50 con 0x7fa698102570 2024-09-11T22:58:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69f542640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa698110dd0 con 0x7fa698102ef0 2024-09-11T22:58:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69f542640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981110d0 con 0x7fa698100e20 2024-09-11T22:58:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69cadd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa698102570 0x7fa6981978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69cadd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa698102570 0x7fa6981978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52156/0 (socket says 172.21.15.152:52156) 2024-09-11T22:58:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69cadd640 1 -- 172.21.15.152:0/662583711 learned_addr learned my addr 172.21.15.152:0/662583711 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69d2de640 1 --2- 172.21.15.152:0/662583711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa698100e20 0x7fa698197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69dadf640 1 --2- 172.21.15.152:0/662583711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa698102ef0 0x7fa69819bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69cadd640 1 -- 172.21.15.152:0/662583711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa698100e20 msgr2=0x7fa698197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69cadd640 1 --2- 172.21.15.152:0/662583711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa698100e20 0x7fa698197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69cadd640 1 -- 172.21.15.152:0/662583711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa698102ef0 msgr2=0x7fa69819bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:50.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69cadd640 1 --2- 172.21.15.152:0/662583711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa698102ef0 0x7fa69819bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69cadd640 1 -- 172.21.15.152:0/662583711 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa69819c340 con 0x7fa698102570 2024-09-11T22:58:50.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69d2de640 1 --2- 172.21.15.152:0/662583711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa698100e20 0x7fa698197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:50.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69dadf640 1 --2- 172.21.15.152:0/662583711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa698102ef0 0x7fa69819bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:50.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.262+0000 7fa69cadd640 1 --2- 172.21.15.152:0/662583711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa698102570 0x7fa6981978d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa688006010 tx=0x7fa688003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:50.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.263+0000 7fa68e7fc640 1 -- 172.21.15.152:0/662583711 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68803f070 con 0x7fa698102570 2024-09-11T22:58:50.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.263+0000 7fa68e7fc640 1 -- 172.21.15.152:0/662583711 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa688031e30 con 0x7fa698102570 2024-09-11T22:58:50.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.263+0000 7fa69f542640 1 -- 172.21.15.152:0/662583711 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa69819c5d0 con 0x7fa698102570 2024-09-11T22:58:50.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.263+0000 7fa68e7fc640 1 -- 172.21.15.152:0/662583711 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa688035030 con 0x7fa698102570 2024-09-11T22:58:50.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.263+0000 7fa69f542640 1 -- 172.21.15.152:0/662583711 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6981a3f10 con 0x7fa698102570 2024-09-11T22:58:50.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.265+0000 7fa68e7fc640 1 -- 172.21.15.152:0/662583711 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa688042070 con 0x7fa698102570 2024-09-11T22:58:50.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.265+0000 7fa6a191b740 1 Processor -- start 2024-09-11T22:58:50.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.265+0000 7fa6a191b740 1 -- start start 2024-09-11T22:58:50.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa6a191b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd321db40 0x555fd321c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa6a191b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321c5d0 0x555fd321ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa6a191b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd3227370 0x555fd3229760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa6a191b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x555fd31822f0 con 0x555fd321db40 2024-09-11T22:58:50.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa6a191b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555fd3129cf0 con 0x555fd3227370 2024-09-11T22:58:50.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa6a191b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555fd317e2b0 con 0x555fd321c5d0 2024-09-11T22:58:50.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa69dadf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd321db40 0x555fd321c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa69dadf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd321db40 0x555fd321c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52158/0 (socket says 172.21.15.152:52158) 2024-09-11T22:58:50.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa69dadf640 1 -- 172.21.15.152:0/705837017 learned_addr learned my addr 172.21.15.152:0/705837017 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:50.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa69cadd640 1 --2- 172.21.15.152:0/705837017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd3227370 0x555fd3229760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa69d2de640 1 --2- 172.21.15.152:0/705837017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321c5d0 0x555fd321ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa69dadf640 1 -- 172.21.15.152:0/705837017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321c5d0 msgr2=0x555fd321ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:50.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa69dadf640 1 --2- 172.21.15.152:0/705837017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321c5d0 0x555fd321ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa69dadf640 1 -- 172.21.15.152:0/705837017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd3227370 msgr2=0x555fd3229760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:50.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa69dadf640 1 --2- 172.21.15.152:0/705837017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd3227370 0x555fd3229760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa69dadf640 1 -- 172.21.15.152:0/705837017 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd310e3a0 con 0x555fd321db40 2024-09-11T22:58:50.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa68e7fc640 1 --2- 172.21.15.152:0/662583711 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6840a2cf0 0x7fa6840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa69dadf640 1 --2- 172.21.15.152:0/705837017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd321db40 0x555fd321c090 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa69000af40 tx=0x7fa69000e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f632408ee193a502 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:50.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa69d2de640 1 --2- 172.21.15.152:0/662583711 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6840a2cf0 0x7fa6840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.266+0000 7fa68e7fc640 1 -- 172.21.15.152:0/662583711 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa6880efe00 con 0x7fa698102570 2024-09-11T22:58:50.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.267+0000 7fa69d2de640 1 --2- 172.21.15.152:0/662583711 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6840a2cf0 0x7fa6840a51b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x555fd3219ac0 tx=0x7fa6940084c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:50.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.269+0000 7fa6837fe640 1 -- 172.21.15.152:0/705837017 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690003b70 con 0x555fd321db40 2024-09-11T22:58:50.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.269+0000 7fa6837fe640 1 -- 172.21.15.152:0/705837017 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa690010b20 con 0x555fd321db40 2024-09-11T22:58:50.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.269+0000 7fa6837fe640 1 -- 172.21.15.152:0/705837017 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69000fbb0 con 0x555fd321db40 2024-09-11T22:58:50.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.269+0000 7fa6a191b740 1 -- 172.21.15.152:0/705837017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd321db40 msgr2=0x555fd321c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:50.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.269+0000 7fa6a191b740 1 --2- 172.21.15.152:0/705837017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd321db40 0x555fd321c090 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fa69000af40 tx=0x7fa69000e7e0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.269+0000 7fa6a191b740 1 -- 172.21.15.152:0/705837017 shutdown_connections 2024-09-11T22:58:50.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.269+0000 7fa6a191b740 1 --2- 172.21.15.152:0/705837017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd3227370 0x555fd3229760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.269+0000 7fa6a191b740 1 --2- 172.21.15.152:0/705837017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321c5d0 0x555fd321ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.269+0000 7fa6a191b740 1 --2- 172.21.15.152:0/705837017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd321db40 0x555fd321c090 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.269+0000 7fa6a191b740 1 -- 172.21.15.152:0/705837017 >> 172.21.15.152:0/705837017 conn(0x555fd31e5310 msgr2=0x555fd3218c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:50.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.269+0000 7fa6a191b740 1 -- 172.21.15.152:0/705837017 shutdown_connections 2024-09-11T22:58:50.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.269+0000 7fa6a191b740 1 -- 172.21.15.152:0/705837017 wait complete. 2024-09-11T22:58:50.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.270+0000 7fa6a191b740 1 -- start start 2024-09-11T22:58:50.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.270+0000 7fa6a191b740 1 Processor -- start 2024-09-11T22:58:50.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.270+0000 7fa6a191b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd321c5d0 0x555fd3226d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.270+0000 7fa6a191b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321db40 0x555fd3224000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.270+0000 7fa6a191b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd3227370 0x555fd32245d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.270+0000 7fa6a191b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x555fd322c280 con 0x555fd321c5d0 2024-09-11T22:58:50.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.270+0000 7fa6a191b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555fd313d060 con 0x555fd3227370 2024-09-11T22:58:50.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.270+0000 7fa6a191b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555fd322c400 con 0x555fd321db40 2024-09-11T22:58:50.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.270+0000 7fa6a191b740 10 client.0 _ll_register_callbacks cb 0x555fd2e908c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:50.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.270+0000 7fa69d2de640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd321c5d0 0x555fd3226d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.270+0000 7fa69d2de640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd321c5d0 0x555fd3226d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52168/0 (socket says 172.21.15.152:52168) 2024-09-11T22:58:50.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.270+0000 7fa69d2de640 1 -- 172.21.15.152:0/1133223062 learned_addr learned my addr 172.21.15.152:0/1133223062 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:50.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.270+0000 7fa69cadd640 1 --2- 172.21.15.152:0/1133223062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321db40 0x555fd3224000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.271+0000 7fa69dadf640 1 --2- 172.21.15.152:0/1133223062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd3227370 0x555fd32245d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.271+0000 7fa69d2de640 1 -- 172.21.15.152:0/1133223062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321db40 msgr2=0x555fd3224000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:50.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.271+0000 7fa69d2de640 1 --2- 172.21.15.152:0/1133223062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321db40 0x555fd3224000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.271+0000 7fa69d2de640 1 -- 172.21.15.152:0/1133223062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd3227370 msgr2=0x555fd32245d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:50.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.271+0000 7fa69d2de640 1 --2- 172.21.15.152:0/1133223062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd3227370 0x555fd32245d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.271+0000 7fa69d2de640 1 -- 172.21.15.152:0/1133223062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fd2ef33c0 con 0x555fd321c5d0 2024-09-11T22:58:50.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.271+0000 7fa69dadf640 1 --2- 172.21.15.152:0/1133223062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd3227370 0x555fd32245d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:50.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.271+0000 7fa69d2de640 1 --2- 172.21.15.152:0/1133223062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd321c5d0 0x555fd3226d50 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa694003ba0 tx=0x7fa6940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d504bfaf01796094 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:50.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.271+0000 7fa680ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:50.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.271+0000 7fa680ff9640 1 -- 172.21.15.152:0/1133223062 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69401cac0 con 0x555fd321c5d0 2024-09-11T22:58:50.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.271+0000 7fa680ff9640 1 -- 172.21.15.152:0/1133223062 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa69401cc60 con 0x555fd321c5d0 2024-09-11T22:58:50.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.271+0000 7fa6a191b740 1 -- 172.21.15.152:0/1133223062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa690010b20 con 0x555fd321c5d0 2024-09-11T22:58:50.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.271+0000 7fa680ff9640 1 -- 172.21.15.152:0/1133223062 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa694026470 con 0x555fd321c5d0 2024-09-11T22:58:50.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.271+0000 7fa6a191b740 10 client.17064 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:50.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.271+0000 7fa6a191b740 1 -- 172.21.15.152:0/1133223062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555fd3237db0 con 0x555fd321c5d0 2024-09-11T22:58:50.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.271+0000 7fa680ff9640 1 -- 172.21.15.152:0/1133223062 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa690010b20 con 0x555fd321c5d0 2024-09-11T22:58:50.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.272+0000 7fa680ff9640 1 -- 172.21.15.152:0/1133223062 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa694030880 con 0x555fd321c5d0 2024-09-11T22:58:50.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.272+0000 7fa6a191b740 10 client.17064 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:50.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.272+0000 7fa6a191b740 10 client.17064 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:58:50.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.272+0000 7fa6a191b740 20 client.17064 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:50.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.272+0000 7fa6a191b740 1 --2- 172.21.15.152:0/1133223062 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x555fd32391c0 0x555fd323b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.272+0000 7fa6a191b740 4 client.17064 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:58:50.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.272+0000 7fa6a191b740 1 -- 172.21.15.152:0/1133223062 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fd318f860 con 0x555fd32391c0 2024-09-11T22:58:50.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.272+0000 7fa69cadd640 1 --2- 172.21.15.152:0/1133223062 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x555fd32391c0 0x555fd323b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.272+0000 7fa69cadd640 1 --2- 172.21.15.152:0/1133223062 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x555fd32391c0 0x555fd323b5b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d0350af15418dd5 server_cookie=60cd539a0d23a40e in_seq=0 out_seq=0 2024-09-11T22:58:50.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.272+0000 7fa680ff9640 10 client.17064 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:58:50.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.273+0000 7fa680ff9640 1 -- 172.21.15.152:0/1133223062 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555fd318f860 con 0x555fd32391c0 2024-09-11T22:58:50.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.273+0000 7fa680ff9640 10 client.17064 handle_command_reply: tid=0 2024-09-11T22:58:50.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.273+0000 7fa6a191b740 1 client.17064 shutdown 2024-09-11T22:58:50.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.273+0000 7fa6a191b740 1 -- 172.21.15.152:0/1133223062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd321c5d0 msgr2=0x555fd3226d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:50.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.273+0000 7fa6a191b740 1 --2- 172.21.15.152:0/1133223062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd321c5d0 0x555fd3226d50 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa694003ba0 tx=0x7fa6940046c0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.273+0000 7fa6a191b740 1 -- 172.21.15.152:0/1133223062 shutdown_connections 2024-09-11T22:58:50.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.273+0000 7fa6a191b740 1 --2- 172.21.15.152:0/1133223062 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x555fd32391c0 0x555fd323b5b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.273+0000 7fa6a191b740 1 --2- 172.21.15.152:0/1133223062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd3227370 0x555fd32245d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.273+0000 7fa6a191b740 1 --2- 172.21.15.152:0/1133223062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321db40 0x555fd3224000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.273+0000 7fa680ff9640 0 client.17064 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:58:50.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.273+0000 7fa6a191b740 1 --2- 172.21.15.152:0/1133223062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd321c5d0 0x555fd3226d50 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.273+0000 7fa6a191b740 1 -- 172.21.15.152:0/1133223062 >> 172.21.15.152:0/1133223062 conn(0x555fd31e5310 msgr2=0x555fd321e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:50.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.273+0000 7fa6a191b740 1 -- 172.21.15.152:0/1133223062 shutdown_connections 2024-09-11T22:58:50.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.273+0000 7fa6a191b740 1 -- 172.21.15.152:0/1133223062 wait complete. 2024-09-11T22:58:50.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.274+0000 7fa6a191b740 20 client.17064 trim_cache size 0 max 16384 2024-09-11T22:58:50.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.286+0000 7fa6a191b740 1 Processor -- start 2024-09-11T22:58:50.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.286+0000 7fa6a191b740 1 -- start start 2024-09-11T22:58:50.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.286+0000 7fa6a191b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd321c5d0 0x555fd3217c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa6a191b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321db40 0x555fd321f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa6a191b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd3227370 0x555fd321f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa6a191b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981ab480 con 0x555fd3227370 2024-09-11T22:58:50.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa6a191b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981acfc0 con 0x555fd321c5d0 2024-09-11T22:58:50.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa6a191b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981abb50 con 0x555fd321db40 2024-09-11T22:58:50.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa69dadf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd3227370 0x555fd321f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa69dadf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd3227370 0x555fd321f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52182/0 (socket says 172.21.15.152:52182) 2024-09-11T22:58:50.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa69dadf640 1 -- 172.21.15.152:0/3322741925 learned_addr learned my addr 172.21.15.152:0/3322741925 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:50.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa69d2de640 1 --2- 172.21.15.152:0/3322741925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd321c5d0 0x555fd3217c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa69cadd640 1 --2- 172.21.15.152:0/3322741925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321db40 0x555fd321f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa69dadf640 1 -- 172.21.15.152:0/3322741925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321db40 msgr2=0x555fd321f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:50.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa69dadf640 1 --2- 172.21.15.152:0/3322741925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321db40 0x555fd321f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa69dadf640 1 -- 172.21.15.152:0/3322741925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd321c5d0 msgr2=0x555fd3217c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:50.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa69dadf640 1 --2- 172.21.15.152:0/3322741925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd321c5d0 0x555fd3217c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa69dadf640 1 -- 172.21.15.152:0/3322741925 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd3224940 con 0x555fd3227370 2024-09-11T22:58:50.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa69dadf640 1 --2- 172.21.15.152:0/3322741925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd3227370 0x555fd321f660 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fa69000ae60 tx=0x7fa69000ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a4164d60d876db server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:50.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa682ffd640 1 -- 172.21.15.152:0/3322741925 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69002a070 con 0x555fd3227370 2024-09-11T22:58:50.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa682ffd640 1 -- 172.21.15.152:0/3322741925 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa690004c80 con 0x555fd3227370 2024-09-11T22:58:50.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.287+0000 7fa682ffd640 1 -- 172.21.15.152:0/3322741925 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6900055a0 con 0x555fd3227370 2024-09-11T22:58:50.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.288+0000 7fa6a191b740 1 -- 172.21.15.152:0/3322741925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd3227370 msgr2=0x555fd321f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:50.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.288+0000 7fa6a191b740 1 --2- 172.21.15.152:0/3322741925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd3227370 0x555fd321f660 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fa69000ae60 tx=0x7fa69000ac60 comp rx=0 tx=0).stop 2024-09-11T22:58:50.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.288+0000 7fa6a191b740 1 -- 172.21.15.152:0/3322741925 shutdown_connections 2024-09-11T22:58:50.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.288+0000 7fa6a191b740 1 --2- 172.21.15.152:0/3322741925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd3227370 0x555fd321f660 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.288+0000 7fa6a191b740 1 --2- 172.21.15.152:0/3322741925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321db40 0x555fd321f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.288+0000 7fa6a191b740 1 --2- 172.21.15.152:0/3322741925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd321c5d0 0x555fd3217c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.288+0000 7fa6a191b740 1 -- 172.21.15.152:0/3322741925 >> 172.21.15.152:0/3322741925 conn(0x555fd31e5310 msgr2=0x555fd3217500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:50.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.288+0000 7fa6a191b740 1 -- 172.21.15.152:0/3322741925 shutdown_connections 2024-09-11T22:58:50.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.288+0000 7fa6a191b740 1 -- 172.21.15.152:0/3322741925 wait complete. 2024-09-11T22:58:50.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.288+0000 7fa6a191b740 1 -- start start 2024-09-11T22:58:50.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.288+0000 7fa6a191b740 1 Processor -- start 2024-09-11T22:58:50.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa6a191b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd321c5d0 0x555fd322db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa6a191b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321db40 0x555fd322e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa6a191b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd3227370 0x555fd32a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa6a191b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981ac8f0 con 0x555fd3227370 2024-09-11T22:58:50.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa6a191b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981ac220 con 0x555fd321c5d0 2024-09-11T22:58:50.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa6a191b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6981a9460 con 0x555fd321db40 2024-09-11T22:58:50.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa6a191b740 10 client.0 _ll_register_callbacks cb 0x7fa6940089a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:50.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa69dadf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd3227370 0x555fd32a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa69dadf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd3227370 0x555fd32a2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52188/0 (socket says 172.21.15.152:52188) 2024-09-11T22:58:50.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa69d2de640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd321c5d0 0x555fd322db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa69dadf640 1 -- 172.21.15.152:0/2790352834 learned_addr learned my addr 172.21.15.152:0/2790352834 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:50.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa69cadd640 1 --2- 172.21.15.152:0/2790352834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321db40 0x555fd322e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa69dadf640 1 -- 172.21.15.152:0/2790352834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321db40 msgr2=0x555fd322e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:50.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa69dadf640 1 --2- 172.21.15.152:0/2790352834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321db40 0x555fd322e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa69dadf640 1 -- 172.21.15.152:0/2790352834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd321c5d0 msgr2=0x555fd322db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:50.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa69dadf640 1 --2- 172.21.15.152:0/2790352834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd321c5d0 0x555fd322db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa69dadf640 1 -- 172.21.15.152:0/2790352834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fd32a3320 con 0x555fd3227370 2024-09-11T22:58:50.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa69cadd640 1 --2- 172.21.15.152:0/2790352834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321db40 0x555fd322e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:50.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa69dadf640 1 --2- 172.21.15.152:0/2790352834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd3227370 0x555fd32a2de0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fa6900051c0 tx=0x7fa690004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a11d50fc955277 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:50.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa680ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:50.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.289+0000 7fa680ff9640 1 -- 172.21.15.152:0/2790352834 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69002a040 con 0x555fd3227370 2024-09-11T22:58:50.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.290+0000 7fa680ff9640 1 -- 172.21.15.152:0/2790352834 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa6900054e0 con 0x555fd3227370 2024-09-11T22:58:50.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.290+0000 7fa6a191b740 1 -- 172.21.15.152:0/2790352834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa690004c80 con 0x555fd3227370 2024-09-11T22:58:50.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.290+0000 7fa680ff9640 1 -- 172.21.15.152:0/2790352834 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa6900056e0 con 0x555fd3227370 2024-09-11T22:58:50.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.290+0000 7fa6a191b740 10 client.17070 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:50.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.290+0000 7fa6a191b740 1 -- 172.21.15.152:0/2790352834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555fd32a45a0 con 0x555fd3227370 2024-09-11T22:58:50.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.290+0000 7fa680ff9640 1 -- 172.21.15.152:0/2790352834 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa690004c80 con 0x555fd3227370 2024-09-11T22:58:50.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.290+0000 7fa680ff9640 1 -- 172.21.15.152:0/2790352834 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa69002d070 con 0x555fd3227370 2024-09-11T22:58:50.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.290+0000 7fa6a191b740 10 client.17070 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:50.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.290+0000 7fa6a191b740 10 client.17070 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:58:50.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.290+0000 7fa6a191b740 20 client.17070 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:50.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.290+0000 7fa6a191b740 1 --2- 172.21.15.152:0/2790352834 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x555fd32a4a70 0x555fd32a6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:50.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.290+0000 7fa6a191b740 4 client.17070 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:58:50.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.290+0000 7fa6a191b740 1 -- 172.21.15.152:0/2790352834 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555fd3189d70 con 0x555fd32a4a70 2024-09-11T22:58:50.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.290+0000 7fa69d2de640 1 --2- 172.21.15.152:0/2790352834 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x555fd32a4a70 0x555fd32a6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:50.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa680ff9640 10 client.17070 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:58:50.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa69d2de640 1 --2- 172.21.15.152:0/2790352834 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x555fd32a4a70 0x555fd32a6e60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a13e48197b0ab1c server_cookie=803d6c9f805953b1 in_seq=0 out_seq=0 2024-09-11T22:58:50.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa680ff9640 1 -- 172.21.15.152:0/2790352834 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555fd3189d70 con 0x555fd32a4a70 2024-09-11T22:58:50.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa680ff9640 10 client.17070 handle_command_reply: tid=0 2024-09-11T22:58:50.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa6a191b740 1 client.17070 shutdown 2024-09-11T22:58:50.317 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:58:50.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa6a191b740 1 -- 172.21.15.152:0/2790352834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd3227370 msgr2=0x555fd32a2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:50.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa6a191b740 1 --2- 172.21.15.152:0/2790352834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd3227370 0x555fd32a2de0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fa6900051c0 tx=0x7fa690004a80 comp rx=0 tx=0).stop 2024-09-11T22:58:50.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa6a191b740 1 -- 172.21.15.152:0/2790352834 shutdown_connections 2024-09-11T22:58:50.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa6a191b740 1 --2- 172.21.15.152:0/2790352834 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x555fd32a4a70 0x555fd32a6e60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa6a191b740 1 --2- 172.21.15.152:0/2790352834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fd3227370 0x555fd32a2de0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa6a191b740 1 --2- 172.21.15.152:0/2790352834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fd321db40 0x555fd322e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa6a191b740 1 --2- 172.21.15.152:0/2790352834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fd321c5d0 0x555fd322db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa6a191b740 1 -- 172.21.15.152:0/2790352834 >> 172.21.15.152:0/2790352834 conn(0x555fd31e5310 msgr2=0x555fd3217d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:50.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa680ff9640 0 client.17070 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:58:50.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa6a191b740 1 -- 172.21.15.152:0/2790352834 shutdown_connections 2024-09-11T22:58:50.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa6a191b740 1 -- 172.21.15.152:0/2790352834 wait complete. 2024-09-11T22:58:50.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.291+0000 7fa6a191b740 20 client.17070 trim_cache size 0 max 16384 2024-09-11T22:58:50.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.292+0000 7fa69f542640 1 -- 172.21.15.152:0/662583711 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6840a2cf0 msgr2=0x7fa6840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:50.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.292+0000 7fa69f542640 1 --2- 172.21.15.152:0/662583711 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6840a2cf0 0x7fa6840a51b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x555fd3219ac0 tx=0x7fa6940084c0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.292+0000 7fa69f542640 1 -- 172.21.15.152:0/662583711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa698102570 msgr2=0x7fa6981978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:50.318 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.292+0000 7fa69f542640 1 --2- 172.21.15.152:0/662583711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa698102570 0x7fa6981978d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fa688006010 tx=0x7fa688003c80 comp rx=0 tx=0).stop 2024-09-11T22:58:50.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.292+0000 7fa69f542640 1 -- 172.21.15.152:0/662583711 shutdown_connections 2024-09-11T22:58:50.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.292+0000 7fa69f542640 1 --2- 172.21.15.152:0/662583711 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6840a2cf0 0x7fa6840a51b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.292+0000 7fa69f542640 1 --2- 172.21.15.152:0/662583711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa698102ef0 0x7fa69819bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.292+0000 7fa69f542640 1 --2- 172.21.15.152:0/662583711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa698102570 0x7fa6981978d0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.292+0000 7fa69f542640 1 --2- 172.21.15.152:0/662583711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa698100e20 0x7fa698197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:50.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.292+0000 7fa69f542640 1 -- 172.21.15.152:0/662583711 >> 172.21.15.152:0/662583711 conn(0x7fa6980fc7d0 msgr2=0x7fa6981068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:50.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.292+0000 7fa69f542640 1 -- 172.21.15.152:0/662583711 shutdown_connections 2024-09-11T22:58:50.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:50.292+0000 7fa69f542640 1 -- 172.21.15.152:0/662583711 wait complete. 2024-09-11T22:58:50.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:50 smithi191 ceph-mon[35211]: pgmap v181: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 12 op/s 2024-09-11T22:58:50.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:50 smithi204 ceph-mon[35454]: pgmap v181: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 12 op/s 2024-09-11T22:58:50.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:50 smithi152 ceph-mon[31181]: pgmap v181: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 12 op/s 2024-09-11T22:58:51.031 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:58:51.282 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:51.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.779+0000 7fc64b349640 1 -- 172.21.15.152:0/3352221867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc644106a80 msgr2=0x7fc644106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:51.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.779+0000 7fc64b349640 1 --2- 172.21.15.152:0/3352221867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc644106a80 0x7fc644106e60 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fc640009930 tx=0x7fc64002a420 comp rx=0 tx=0).stop 2024-09-11T22:58:51.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.779+0000 7fc64b349640 1 -- 172.21.15.152:0/3352221867 shutdown_connections 2024-09-11T22:58:51.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.779+0000 7fc64b349640 1 --2- 172.21.15.152:0/3352221867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc644101420 0x7fc64410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.779+0000 7fc64b349640 1 --2- 172.21.15.152:0/3352221867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc644100a80 0x7fc644100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.817 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.779+0000 7fc64b349640 1 --2- 172.21.15.152:0/3352221867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc644106a80 0x7fc644106e60 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.779+0000 7fc64b349640 1 -- 172.21.15.152:0/3352221867 >> 172.21.15.152:0/3352221867 conn(0x7fc6440fc7a0 msgr2=0x7fc6440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.779+0000 7fc64b349640 1 -- 172.21.15.152:0/3352221867 shutdown_connections 2024-09-11T22:58:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.779+0000 7fc64b349640 1 -- 172.21.15.152:0/3352221867 wait complete. 2024-09-11T22:58:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc64b349640 1 Processor -- start 2024-09-11T22:58:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc64b349640 1 -- start start 2024-09-11T22:58:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc64b349640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc644100a80 0x7fc644199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc64b349640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc644101420 0x7fc644199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc64b349640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc644106a80 0x7fc6441a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc64b349640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc644110f20 con 0x7fc644101420 2024-09-11T22:58:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc64b349640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc644110da0 con 0x7fc644100a80 2024-09-11T22:58:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc64b349640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc6441110a0 con 0x7fc644106a80 2024-09-11T22:58:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc6488e4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc644101420 0x7fc644199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc6488e4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc644101420 0x7fc644199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52196/0 (socket says 172.21.15.152:52196) 2024-09-11T22:58:51.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc6488e4640 1 -- 172.21.15.152:0/3949068347 learned_addr learned my addr 172.21.15.152:0/3949068347 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:51.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc6490e5640 1 --2- 172.21.15.152:0/3949068347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc644100a80 0x7fc644199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc6498e6640 1 --2- 172.21.15.152:0/3949068347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc644106a80 0x7fc6441a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc6488e4640 1 -- 172.21.15.152:0/3949068347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc644106a80 msgr2=0x7fc6441a44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:51.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc6488e4640 1 --2- 172.21.15.152:0/3949068347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc644106a80 0x7fc6441a44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc6488e4640 1 -- 172.21.15.152:0/3949068347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc644100a80 msgr2=0x7fc644199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:51.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc6488e4640 1 --2- 172.21.15.152:0/3949068347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc644100a80 0x7fc644199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc6488e4640 1 -- 172.21.15.152:0/3949068347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6441a4a10 con 0x7fc644101420 2024-09-11T22:58:51.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc6490e5640 1 --2- 172.21.15.152:0/3949068347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc644100a80 0x7fc644199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:51.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.780+0000 7fc6488e4640 1 --2- 172.21.15.152:0/3949068347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc644101420 0x7fc644199990 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc63800b4a0 tx=0x7fc63800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:51.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.781+0000 7fc6367fc640 1 -- 172.21.15.152:0/3949068347 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638014070 con 0x7fc644101420 2024-09-11T22:58:51.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.781+0000 7fc6367fc640 1 -- 172.21.15.152:0/3949068347 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc638010af0 con 0x7fc644101420 2024-09-11T22:58:51.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.781+0000 7fc6367fc640 1 -- 172.21.15.152:0/3949068347 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638017d10 con 0x7fc644101420 2024-09-11T22:58:51.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.781+0000 7fc64b349640 1 -- 172.21.15.152:0/3949068347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6441a4be0 con 0x7fc644101420 2024-09-11T22:58:51.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.781+0000 7fc64b349640 1 -- 172.21.15.152:0/3949068347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6441a50f0 con 0x7fc644101420 2024-09-11T22:58:51.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.783+0000 7fc6367fc640 1 -- 172.21.15.152:0/3949068347 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc638017070 con 0x7fc644101420 2024-09-11T22:58:51.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.783+0000 7fc64d722740 1 Processor -- start 2024-09-11T22:58:51.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.783+0000 7fc64d722740 1 -- start start 2024-09-11T22:58:51.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc64d722740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae3b40 0x5586b0ae2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc64d722740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae25d0 0x5586b0ae2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc64d722740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0af3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc6367fc640 1 --2- 172.21.15.152:0/3949068347 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc6300a2cf0 0x7fc6300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc64d722740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5586b0a482f0 con 0x5586b0af1780 2024-09-11T22:58:51.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc64d722740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5586b09efcf0 con 0x5586b0ae25d0 2024-09-11T22:58:51.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc64d722740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5586b0a442b0 con 0x5586b0ae3b40 2024-09-11T22:58:51.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc6488e4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0af3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc6498e6640 1 --2- 172.21.15.152:0/3949068347 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc6300a2cf0 0x7fc6300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc6488e4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0af3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52200/0 (socket says 172.21.15.152:52200) 2024-09-11T22:58:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc6488e4640 1 -- 172.21.15.152:0/284835816 learned_addr learned my addr 172.21.15.152:0/284835816 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc6490e5640 1 --2- 172.21.15.152:0/284835816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae25d0 0x5586b0ae2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc6367fc640 1 -- 172.21.15.152:0/3949068347 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc6380ce710 con 0x7fc644101420 2024-09-11T22:58:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc6498e6640 1 --2- 172.21.15.152:0/284835816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae3b40 0x5586b0ae2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc6488e4640 1 -- 172.21.15.152:0/284835816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae3b40 msgr2=0x5586b0ae2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc6488e4640 1 --2- 172.21.15.152:0/284835816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae3b40 0x5586b0ae2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc6488e4640 1 -- 172.21.15.152:0/284835816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae25d0 msgr2=0x5586b0ae2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc6488e4640 1 --2- 172.21.15.152:0/284835816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae25d0 0x5586b0ae2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc6488e4640 1 -- 172.21.15.152:0/284835816 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586b09d43a0 con 0x5586b0af1780 2024-09-11T22:58:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc6498e6640 1 --2- 172.21.15.152:0/3949068347 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc6300a2cf0 0x7fc6300a51b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc63c014390 tx=0x7fc63c014790 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc6488e4640 1 --2- 172.21.15.152:0/284835816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0af3b70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc638097760 tx=0x7fc638065db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b1f21940567182 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:51.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.784+0000 7fc62f7fe640 1 -- 172.21.15.152:0/284835816 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63801e070 con 0x5586b0af1780 2024-09-11T22:58:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.785+0000 7fc62f7fe640 1 -- 172.21.15.152:0/284835816 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc63809b490 con 0x5586b0af1780 2024-09-11T22:58:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.785+0000 7fc62f7fe640 1 -- 172.21.15.152:0/284835816 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63800c990 con 0x5586b0af1780 2024-09-11T22:58:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.785+0000 7fc64d722740 1 -- 172.21.15.152:0/284835816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 msgr2=0x5586b0af3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.785+0000 7fc64d722740 1 --2- 172.21.15.152:0/284835816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0af3b70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fc638097760 tx=0x7fc638065db0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.785+0000 7fc64d722740 1 -- 172.21.15.152:0/284835816 shutdown_connections 2024-09-11T22:58:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.785+0000 7fc64d722740 1 --2- 172.21.15.152:0/284835816 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0af3b70 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.785+0000 7fc64d722740 1 --2- 172.21.15.152:0/284835816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae25d0 0x5586b0ae2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.785+0000 7fc64d722740 1 --2- 172.21.15.152:0/284835816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae3b40 0x5586b0ae2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.785+0000 7fc64d722740 1 -- 172.21.15.152:0/284835816 >> 172.21.15.152:0/284835816 conn(0x5586b0aab310 msgr2=0x5586b0adec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.785+0000 7fc64d722740 1 -- 172.21.15.152:0/284835816 shutdown_connections 2024-09-11T22:58:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.785+0000 7fc64d722740 1 -- 172.21.15.152:0/284835816 wait complete. 2024-09-11T22:58:51.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.786+0000 7fc64d722740 1 -- start start 2024-09-11T22:58:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.786+0000 7fc64d722740 1 Processor -- start 2024-09-11T22:58:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.786+0000 7fc64d722740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae25d0 0x5586b0af1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.786+0000 7fc64d722740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae3b40 0x5586b0aee420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.786+0000 7fc64d722740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0aeea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.786+0000 7fc64d722740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5586b0af4480 con 0x5586b0af1780 2024-09-11T22:58:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.786+0000 7fc64d722740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5586b0a03060 con 0x5586b0ae25d0 2024-09-11T22:58:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.786+0000 7fc64d722740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5586b0af4600 con 0x5586b0ae3b40 2024-09-11T22:58:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.786+0000 7fc64d722740 10 client.0 _ll_register_callbacks cb 0x5586b07568c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.786+0000 7fc6498e6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0aeea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.786+0000 7fc6498e6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0aeea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52204/0 (socket says 172.21.15.152:52204) 2024-09-11T22:58:51.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.786+0000 7fc6498e6640 1 -- 172.21.15.152:0/1548158128 learned_addr learned my addr 172.21.15.152:0/1548158128 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.786+0000 7fc6490e5640 1 --2- 172.21.15.152:0/1548158128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae25d0 0x5586b0af1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.787+0000 7fc6488e4640 1 --2- 172.21.15.152:0/1548158128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae3b40 0x5586b0aee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.787+0000 7fc6498e6640 1 -- 172.21.15.152:0/1548158128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae3b40 msgr2=0x5586b0aee420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.787+0000 7fc6498e6640 1 --2- 172.21.15.152:0/1548158128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae3b40 0x5586b0aee420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.787+0000 7fc6498e6640 1 -- 172.21.15.152:0/1548158128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae25d0 msgr2=0x5586b0af1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.787+0000 7fc6498e6640 1 --2- 172.21.15.152:0/1548158128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae25d0 0x5586b0af1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.787+0000 7fc6498e6640 1 -- 172.21.15.152:0/1548158128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586b07b93c0 con 0x5586b0af1780 2024-09-11T22:58:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.787+0000 7fc6498e6640 1 --2- 172.21.15.152:0/1548158128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0aeea10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc63c016d90 tx=0x7fc63c001300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd35bf2b3295f38 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.787+0000 7fc62cff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.787+0000 7fc62cff9640 1 -- 172.21.15.152:0/1548158128 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c027060 con 0x5586b0af1780 2024-09-11T22:58:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.787+0000 7fc64d722740 1 -- 172.21.15.152:0/1548158128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc63809b490 con 0x5586b0af1780 2024-09-11T22:58:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.787+0000 7fc62cff9640 1 -- 172.21.15.152:0/1548158128 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc63c022600 con 0x5586b0af1780 2024-09-11T22:58:51.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.787+0000 7fc62cff9640 1 -- 172.21.15.152:0/1548158128 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc63c022800 con 0x5586b0af1780 2024-09-11T22:58:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.787+0000 7fc64d722740 10 client.17082 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.787+0000 7fc64d722740 1 -- 172.21.15.152:0/1548158128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5586b0afff20 con 0x5586b0af1780 2024-09-11T22:58:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.787+0000 7fc62cff9640 1 -- 172.21.15.152:0/1548158128 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc63809b490 con 0x5586b0af1780 2024-09-11T22:58:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.788+0000 7fc62cff9640 1 -- 172.21.15.152:0/1548158128 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc63c02c960 con 0x5586b0af1780 2024-09-11T22:58:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.788+0000 7fc64d722740 10 client.17082 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.788+0000 7fc64d722740 10 client.17082 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:58:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.788+0000 7fc64d722740 20 client.17082 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.788+0000 7fc64d722740 1 --2- 172.21.15.152:0/1548158128 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5586b0b013e0 0x5586b0b037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.788+0000 7fc64d722740 4 client.17082 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:58:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.788+0000 7fc64d722740 1 -- 172.21.15.152:0/1548158128 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586b0a55860 con 0x5586b0b013e0 2024-09-11T22:58:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.788+0000 7fc6490e5640 1 --2- 172.21.15.152:0/1548158128 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5586b0b013e0 0x5586b0b037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.788+0000 7fc62cff9640 10 client.17082 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:58:51.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.788+0000 7fc6490e5640 1 --2- 172.21.15.152:0/1548158128 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5586b0b013e0 0x5586b0b037d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61afedc24386ac86 server_cookie=682c9217eba6360a in_seq=0 out_seq=0 2024-09-11T22:58:51.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.789+0000 7fc62cff9640 1 -- 172.21.15.152:0/1548158128 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586b0a55860 con 0x5586b0b013e0 2024-09-11T22:58:51.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.789+0000 7fc62cff9640 10 client.17082 handle_command_reply: tid=0 2024-09-11T22:58:51.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.789+0000 7fc64d722740 1 client.17082 shutdown 2024-09-11T22:58:51.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.789+0000 7fc64d722740 1 -- 172.21.15.152:0/1548158128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 msgr2=0x5586b0aeea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:51.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.789+0000 7fc64d722740 1 --2- 172.21.15.152:0/1548158128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0aeea10 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc63c016d90 tx=0x7fc63c001300 comp rx=0 tx=0).stop 2024-09-11T22:58:51.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.789+0000 7fc64d722740 1 -- 172.21.15.152:0/1548158128 shutdown_connections 2024-09-11T22:58:51.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.789+0000 7fc64d722740 1 --2- 172.21.15.152:0/1548158128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0aeea10 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.789+0000 7fc64d722740 1 --2- 172.21.15.152:0/1548158128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae3b40 0x5586b0aee420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.789+0000 7fc64d722740 1 --2- 172.21.15.152:0/1548158128 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5586b0b013e0 0x5586b0b037d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.789+0000 7fc64d722740 1 --2- 172.21.15.152:0/1548158128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae25d0 0x5586b0af1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.789+0000 7fc62cff9640 0 client.17082 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:58:51.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.789+0000 7fc64d722740 1 -- 172.21.15.152:0/1548158128 >> 172.21.15.152:0/1548158128 conn(0x5586b0aab310 msgr2=0x5586b0ae4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:51.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.789+0000 7fc64d722740 1 -- 172.21.15.152:0/1548158128 shutdown_connections 2024-09-11T22:58:51.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.789+0000 7fc64d722740 1 -- 172.21.15.152:0/1548158128 wait complete. 2024-09-11T22:58:51.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.790+0000 7fc64d722740 20 client.17082 trim_cache size 0 max 16384 2024-09-11T22:58:51.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.802+0000 7fc64d722740 1 Processor -- start 2024-09-11T22:58:51.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.802+0000 7fc64d722740 1 -- start start 2024-09-11T22:58:51.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc64d722740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae25d0 0x5586b0adee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc64d722740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0ae3b40 0x5586b0afd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc64d722740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0af1780 0x5586b0aedcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc64d722740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc6441ab7a0 con 0x5586b0ae3b40 2024-09-11T22:58:51.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc64d722740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc6441ad2e0 con 0x5586b0af1780 2024-09-11T22:58:51.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc64d722740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc6441abe70 con 0x5586b0ae25d0 2024-09-11T22:58:51.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc6488e4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0ae3b40 0x5586b0afd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc6488e4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0ae3b40 0x5586b0afd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52214/0 (socket says 172.21.15.152:52214) 2024-09-11T22:58:51.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc6488e4640 1 -- 172.21.15.152:0/1154970361 learned_addr learned my addr 172.21.15.152:0/1154970361 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:51.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc6498e6640 1 --2- 172.21.15.152:0/1154970361 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0af1780 0x5586b0aedcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc6490e5640 1 --2- 172.21.15.152:0/1154970361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae25d0 0x5586b0adee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc6488e4640 1 -- 172.21.15.152:0/1154970361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae25d0 msgr2=0x5586b0adee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:51.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc6488e4640 1 --2- 172.21.15.152:0/1154970361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae25d0 0x5586b0adee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc6488e4640 1 -- 172.21.15.152:0/1154970361 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0af1780 msgr2=0x5586b0aedcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:51.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc6488e4640 1 --2- 172.21.15.152:0/1154970361 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0af1780 0x5586b0aedcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc6488e4640 1 -- 172.21.15.152:0/1154970361 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586b0aed3d0 con 0x5586b0ae3b40 2024-09-11T22:58:51.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc6488e4640 1 --2- 172.21.15.152:0/1154970361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0ae3b40 0x5586b0afd720 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc63809ad20 tx=0x7fc6380609f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4463f52aeceab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:51.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc62effd640 1 -- 172.21.15.152:0/1154970361 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63801e050 con 0x5586b0ae3b40 2024-09-11T22:58:51.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc62effd640 1 -- 172.21.15.152:0/1154970361 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc638065e20 con 0x5586b0ae3b40 2024-09-11T22:58:51.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.803+0000 7fc62effd640 1 -- 172.21.15.152:0/1154970361 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638097400 con 0x5586b0ae3b40 2024-09-11T22:58:51.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.804+0000 7fc64d722740 1 -- 172.21.15.152:0/1154970361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0ae3b40 msgr2=0x5586b0afd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:51.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.804+0000 7fc64d722740 1 --2- 172.21.15.152:0/1154970361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0ae3b40 0x5586b0afd720 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc63809ad20 tx=0x7fc6380609f0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.804+0000 7fc64d722740 1 -- 172.21.15.152:0/1154970361 shutdown_connections 2024-09-11T22:58:51.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.804+0000 7fc64d722740 1 --2- 172.21.15.152:0/1154970361 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0af1780 0x5586b0aedcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.804+0000 7fc64d722740 1 --2- 172.21.15.152:0/1154970361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0ae3b40 0x5586b0afd720 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.804+0000 7fc64d722740 1 --2- 172.21.15.152:0/1154970361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae25d0 0x5586b0adee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.804+0000 7fc64d722740 1 -- 172.21.15.152:0/1154970361 >> 172.21.15.152:0/1154970361 conn(0x5586b0aab310 msgr2=0x5586b0ade700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:51.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.804+0000 7fc64d722740 1 -- 172.21.15.152:0/1154970361 shutdown_connections 2024-09-11T22:58:51.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.804+0000 7fc64d722740 1 -- 172.21.15.152:0/1154970361 wait complete. 2024-09-11T22:58:51.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.804+0000 7fc64d722740 1 -- start start 2024-09-11T22:58:51.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.804+0000 7fc64d722740 1 Processor -- start 2024-09-11T22:58:51.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc64d722740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae25d0 0x5586b0afa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc64d722740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae3b40 0x5586b0af5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc64d722740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0af5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc64d722740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc6441acc10 con 0x5586b0af1780 2024-09-11T22:58:51.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc64d722740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc6441ac540 con 0x5586b0ae3b40 2024-09-11T22:58:51.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc64d722740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc6441a9780 con 0x5586b0ae25d0 2024-09-11T22:58:51.831 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc64d722740 10 client.0 _ll_register_callbacks cb 0x7fc63c008070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:51.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc6498e6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0af5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc6488e4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae3b40 0x5586b0af5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc6498e6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0af5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52216/0 (socket says 172.21.15.152:52216) 2024-09-11T22:58:51.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc6498e6640 1 -- 172.21.15.152:0/1563369514 learned_addr learned my addr 172.21.15.152:0/1563369514 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:51.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc6488e4640 1 --2- 172.21.15.152:0/1563369514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae3b40 0x5586b0af5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:44644/0 (socket says 172.21.15.152:44644) 2024-09-11T22:58:51.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc6490e5640 1 --2- 172.21.15.152:0/1563369514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae25d0 0x5586b0afa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc6498e6640 1 -- 172.21.15.152:0/1563369514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae25d0 msgr2=0x5586b0afa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:51.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc6498e6640 1 --2- 172.21.15.152:0/1563369514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae25d0 0x5586b0afa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc6498e6640 1 -- 172.21.15.152:0/1563369514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae3b40 msgr2=0x5586b0af5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:51.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc6498e6640 1 --2- 172.21.15.152:0/1563369514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae3b40 0x5586b0af5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc6498e6640 1 -- 172.21.15.152:0/1563369514 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586b0af6090 con 0x5586b0af1780 2024-09-11T22:58:51.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.805+0000 7fc6488e4640 1 --2- 172.21.15.152:0/1563369514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae3b40 0x5586b0af5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:58:51.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.806+0000 7fc6498e6640 1 --2- 172.21.15.152:0/1563369514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0af5890 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc63c017040 tx=0x7fc63c014d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1630b7af5850c4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:51.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.806+0000 7fc62cff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:51.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.806+0000 7fc62cff9640 1 -- 172.21.15.152:0/1563369514 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c027040 con 0x5586b0af1780 2024-09-11T22:58:51.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.806+0000 7fc64d722740 1 -- 172.21.15.152:0/1563369514 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc638065e20 con 0x5586b0af1780 2024-09-11T22:58:51.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.806+0000 7fc62cff9640 1 -- 172.21.15.152:0/1563369514 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc63c022ba0 con 0x5586b0af1780 2024-09-11T22:58:51.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.806+0000 7fc62cff9640 1 -- 172.21.15.152:0/1563369514 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc63c022610 con 0x5586b0af1780 2024-09-11T22:58:51.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.806+0000 7fc64d722740 10 client.17088 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:51.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.806+0000 7fc64d722740 1 -- 172.21.15.152:0/1563369514 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5586b0b6c080 con 0x5586b0af1780 2024-09-11T22:58:51.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.806+0000 7fc62cff9640 1 -- 172.21.15.152:0/1563369514 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc638065e20 con 0x5586b0af1780 2024-09-11T22:58:51.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.806+0000 7fc62cff9640 1 -- 172.21.15.152:0/1563369514 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc63c02a7f0 con 0x5586b0af1780 2024-09-11T22:58:51.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.806+0000 7fc64d722740 10 client.17088 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:51.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.806+0000 7fc64d722740 10 client.17088 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:58:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.806+0000 7fc64d722740 20 client.17088 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.806+0000 7fc64d722740 1 --2- 172.21.15.152:0/1563369514 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5586b0b6c790 0x5586b0b6eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.806+0000 7fc64d722740 4 client.17088 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:58:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.806+0000 7fc64d722740 1 -- 172.21.15.152:0/1563369514 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586b0a4fd70 con 0x5586b0b6c790 2024-09-11T22:58:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.807+0000 7fc6490e5640 1 --2- 172.21.15.152:0/1563369514 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5586b0b6c790 0x5586b0b6eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.807+0000 7fc62cff9640 10 client.17088 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:58:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.807+0000 7fc6490e5640 1 --2- 172.21.15.152:0/1563369514 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5586b0b6c790 0x5586b0b6eb80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e9cdbdedddfcb1 server_cookie=2f9a106f18f24136 in_seq=0 out_seq=0 2024-09-11T22:58:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.807+0000 7fc62cff9640 1 -- 172.21.15.152:0/1563369514 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586b0a4fd70 con 0x5586b0b6c790 2024-09-11T22:58:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.807+0000 7fc62cff9640 10 client.17088 handle_command_reply: tid=0 2024-09-11T22:58:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.807+0000 7fc64d722740 1 client.17088 shutdown 2024-09-11T22:58:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.807+0000 7fc64d722740 1 -- 172.21.15.152:0/1563369514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 msgr2=0x5586b0af5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.807+0000 7fc64d722740 1 --2- 172.21.15.152:0/1563369514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0af5890 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc63c017040 tx=0x7fc63c014d60 comp rx=0 tx=0).stop 2024-09-11T22:58:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.807+0000 7fc64d722740 1 -- 172.21.15.152:0/1563369514 shutdown_connections 2024-09-11T22:58:51.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.807+0000 7fc64d722740 1 --2- 172.21.15.152:0/1563369514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5586b0af1780 0x5586b0af5890 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.807+0000 7fc64d722740 1 --2- 172.21.15.152:0/1563369514 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5586b0b6c790 0x5586b0b6eb80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.807+0000 7fc64d722740 1 --2- 172.21.15.152:0/1563369514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5586b0ae3b40 0x5586b0af5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.807+0000 7fc64d722740 1 --2- 172.21.15.152:0/1563369514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5586b0ae25d0 0x5586b0afa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.807+0000 7fc62cff9640 0 client.17088 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:58:51.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.807+0000 7fc64d722740 1 -- 172.21.15.152:0/1563369514 >> 172.21.15.152:0/1563369514 conn(0x5586b0aab310 msgr2=0x5586b0ade680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:51.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.808+0000 7fc64d722740 1 -- 172.21.15.152:0/1563369514 shutdown_connections 2024-09-11T22:58:51.835 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:58:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.808+0000 7fc64d722740 1 -- 172.21.15.152:0/1563369514 wait complete. 2024-09-11T22:58:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.808+0000 7fc64d722740 20 client.17088 trim_cache size 0 max 16384 2024-09-11T22:58:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.808+0000 7fc64b349640 1 -- 172.21.15.152:0/3949068347 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc6300a2cf0 msgr2=0x7fc6300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.808+0000 7fc64b349640 1 --2- 172.21.15.152:0/3949068347 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc6300a2cf0 0x7fc6300a51b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc63c014390 tx=0x7fc63c014790 comp rx=0 tx=0).stop 2024-09-11T22:58:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.808+0000 7fc64b349640 1 -- 172.21.15.152:0/3949068347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc644101420 msgr2=0x7fc644199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.808+0000 7fc64b349640 1 --2- 172.21.15.152:0/3949068347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc644101420 0x7fc644199990 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc63800b4a0 tx=0x7fc63800b960 comp rx=0 tx=0).stop 2024-09-11T22:58:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.808+0000 7fc64b349640 1 -- 172.21.15.152:0/3949068347 shutdown_connections 2024-09-11T22:58:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.808+0000 7fc64b349640 1 --2- 172.21.15.152:0/3949068347 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc6300a2cf0 0x7fc6300a51b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.808+0000 7fc64b349640 1 --2- 172.21.15.152:0/3949068347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc644106a80 0x7fc6441a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.808+0000 7fc64b349640 1 --2- 172.21.15.152:0/3949068347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc644101420 0x7fc644199990 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.808+0000 7fc64b349640 1 --2- 172.21.15.152:0/3949068347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc644100a80 0x7fc644199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:51.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.808+0000 7fc64b349640 1 -- 172.21.15.152:0/3949068347 >> 172.21.15.152:0/3949068347 conn(0x7fc6440fc7a0 msgr2=0x7fc6440fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:51.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.808+0000 7fc64b349640 1 -- 172.21.15.152:0/3949068347 shutdown_connections 2024-09-11T22:58:51.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:51.808+0000 7fc64b349640 1 -- 172.21.15.152:0/3949068347 wait complete. 2024-09-11T22:58:52.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:52 smithi191 ceph-mon[35211]: pgmap v182: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-11T22:58:52.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:52 smithi204 ceph-mon[35454]: pgmap v182: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-11T22:58:52.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:52 smithi152 ceph-mon[31181]: pgmap v182: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-11T22:58:52.702 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:58:52.948 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:53.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 -- 172.21.15.152:0/1774980255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a41011a0 msgr2=0x7f15a4101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:53.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 --2- 172.21.15.152:0/1774980255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a41011a0 0x7f15a4101600 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f1598009900 tx=0x7f159802a410 comp rx=0 tx=0).stop 2024-09-11T22:58:53.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 -- 172.21.15.152:0/1774980255 shutdown_connections 2024-09-11T22:58:53.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 --2- 172.21.15.152:0/1774980255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a4101b40 0x7f15a410da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 --2- 172.21.15.152:0/1774980255 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a41011a0 0x7f15a4101600 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 --2- 172.21.15.152:0/1774980255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a4106c30 0x7f15a4107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 -- 172.21.15.152:0/1774980255 >> 172.21.15.152:0/1774980255 conn(0x7f15a4077210 msgr2=0x7f15a40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:53.465 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 -- 172.21.15.152:0/1774980255 shutdown_connections 2024-09-11T22:58:53.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 -- 172.21.15.152:0/1774980255 wait complete. 2024-09-11T22:58:53.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 Processor -- start 2024-09-11T22:58:53.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 -- start start 2024-09-11T22:58:53.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a41011a0 0x7f15a4197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a4101b40 0x7f15a4197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a4106c30 0x7f15a419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4110fa0 con 0x7f15a4101b40 2024-09-11T22:58:53.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4110e20 con 0x7f15a41011a0 2024-09-11T22:58:53.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.435+0000 7f15aaf7e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4111120 con 0x7f15a4106c30 2024-09-11T22:58:53.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15a3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a4101b40 0x7f15a4197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15a3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a4101b40 0x7f15a4197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52230/0 (socket says 172.21.15.152:52230) 2024-09-11T22:58:53.466 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15a3fff640 1 -- 172.21.15.152:0/2057045804 learned_addr learned my addr 172.21.15.152:0/2057045804 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:53.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15a8d1a640 1 --2- 172.21.15.152:0/2057045804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a41011a0 0x7f15a4197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15a951b640 1 --2- 172.21.15.152:0/2057045804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a4106c30 0x7f15a419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15a3fff640 1 -- 172.21.15.152:0/2057045804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a4106c30 msgr2=0x7f15a419bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:53.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15a3fff640 1 --2- 172.21.15.152:0/2057045804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a4106c30 0x7f15a419bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15a3fff640 1 -- 172.21.15.152:0/2057045804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a41011a0 msgr2=0x7f15a4197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:53.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15a3fff640 1 --2- 172.21.15.152:0/2057045804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a41011a0 0x7f15a4197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15a3fff640 1 -- 172.21.15.152:0/2057045804 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a419c3b0 con 0x7f15a4101b40 2024-09-11T22:58:53.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15a951b640 1 --2- 172.21.15.152:0/2057045804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a4106c30 0x7f15a419bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:53.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15a8d1a640 1 --2- 172.21.15.152:0/2057045804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a41011a0 0x7f15a4197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:53.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15a3fff640 1 --2- 172.21.15.152:0/2057045804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a4101b40 0x7f15a4197940 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1598006010 tx=0x7f15980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:53.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15a1ffb640 1 -- 172.21.15.152:0/2057045804 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159803f030 con 0x7f15a4101b40 2024-09-11T22:58:53.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15a1ffb640 1 -- 172.21.15.152:0/2057045804 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1598003ce0 con 0x7f15a4101b40 2024-09-11T22:58:53.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15aaf7e640 1 -- 172.21.15.152:0/2057045804 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a419c640 con 0x7f15a4101b40 2024-09-11T22:58:53.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.436+0000 7f15a1ffb640 1 -- 172.21.15.152:0/2057045804 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598035070 con 0x7f15a4101b40 2024-09-11T22:58:53.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.437+0000 7f15aaf7e640 1 -- 172.21.15.152:0/2057045804 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a41a3f80 con 0x7f15a4101b40 2024-09-11T22:58:53.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.439+0000 7f15a1ffb640 1 -- 172.21.15.152:0/2057045804 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1598042030 con 0x7f15a4101b40 2024-09-11T22:58:53.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.439+0000 7f15ad357740 1 Processor -- start 2024-09-11T22:58:53.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.439+0000 7f15ad357740 1 -- start start 2024-09-11T22:58:53.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.439+0000 7f15ad357740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c64b40 0x559a30c63090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.439+0000 7f15ad357740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c635d0 0x559a30c63a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.439+0000 7f15ad357740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c72970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.439+0000 7f15ad357740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x559a30bc92f0 con 0x559a30c70580 2024-09-11T22:58:53.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.439+0000 7f15ad357740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559a30b70cf0 con 0x559a30c635d0 2024-09-11T22:58:53.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.439+0000 7f15ad357740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559a30bc52b0 con 0x559a30c64b40 2024-09-11T22:58:53.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.439+0000 7f15a1ffb640 1 --2- 172.21.15.152:0/2057045804 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f158c0a2cf0 0x7f158c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.468 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.439+0000 7f15a3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c72970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.440+0000 7f15a3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c72970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52234/0 (socket says 172.21.15.152:52234) 2024-09-11T22:58:53.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.440+0000 7f15a3fff640 1 -- 172.21.15.152:0/3435385603 learned_addr learned my addr 172.21.15.152:0/3435385603 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:53.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.440+0000 7f15a951b640 1 --2- 172.21.15.152:0/3435385603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c64b40 0x559a30c63090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.440+0000 7f15a8d1a640 1 --2- 172.21.15.152:0/3435385603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c635d0 0x559a30c63a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.440+0000 7f15a951b640 1 --2- 172.21.15.152:0/2057045804 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f158c0a2cf0 0x7f158c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.440+0000 7f15a1ffb640 1 -- 172.21.15.152:0/2057045804 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f15980efbb0 con 0x7f15a4101b40 2024-09-11T22:58:53.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.440+0000 7f15a3fff640 1 -- 172.21.15.152:0/3435385603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c64b40 msgr2=0x559a30c63090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:53.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.440+0000 7f15a3fff640 1 --2- 172.21.15.152:0/3435385603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c64b40 0x559a30c63090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.440+0000 7f15a3fff640 1 -- 172.21.15.152:0/3435385603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c635d0 msgr2=0x559a30c63a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:53.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.440+0000 7f15a3fff640 1 --2- 172.21.15.152:0/3435385603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c635d0 0x559a30c63a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.440+0000 7f15a3fff640 1 -- 172.21.15.152:0/3435385603 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a30b553a0 con 0x559a30c70580 2024-09-11T22:58:53.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.440+0000 7f15a3fff640 1 --2- 172.21.15.152:0/3435385603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c72970 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f15980ed370 tx=0x7f1598003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312c4b10e1381ba7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:53.469 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.440+0000 7f15a951b640 1 --2- 172.21.15.152:0/2057045804 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f158c0a2cf0 0x7f158c0a51b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f159c009270 tx=0x7f159c005bd0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:53.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.440+0000 7f158affd640 1 -- 172.21.15.152:0/3435385603 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1598049070 con 0x559a30c70580 2024-09-11T22:58:53.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.440+0000 7f158affd640 1 -- 172.21.15.152:0/3435385603 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f159803b580 con 0x559a30c70580 2024-09-11T22:58:53.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.440+0000 7f158affd640 1 -- 172.21.15.152:0/3435385603 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15980c0550 con 0x559a30c70580 2024-09-11T22:58:53.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.441+0000 7f15ad357740 1 -- 172.21.15.152:0/3435385603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 msgr2=0x559a30c72970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:53.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.441+0000 7f15ad357740 1 --2- 172.21.15.152:0/3435385603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c72970 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f15980ed370 tx=0x7f1598003f60 comp rx=0 tx=0).stop 2024-09-11T22:58:53.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.441+0000 7f15ad357740 1 -- 172.21.15.152:0/3435385603 shutdown_connections 2024-09-11T22:58:53.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.441+0000 7f15ad357740 1 --2- 172.21.15.152:0/3435385603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c72970 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.441+0000 7f15ad357740 1 --2- 172.21.15.152:0/3435385603 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c635d0 0x559a30c63a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.441+0000 7f15ad357740 1 --2- 172.21.15.152:0/3435385603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c64b40 0x559a30c63090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.441+0000 7f15ad357740 1 -- 172.21.15.152:0/3435385603 >> 172.21.15.152:0/3435385603 conn(0x559a30c2c310 msgr2=0x559a30c5fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:53.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.441+0000 7f15ad357740 1 -- 172.21.15.152:0/3435385603 shutdown_connections 2024-09-11T22:58:53.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.441+0000 7f15ad357740 1 -- 172.21.15.152:0/3435385603 wait complete. 2024-09-11T22:58:53.470 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.441+0000 7f15ad357740 1 -- start start 2024-09-11T22:58:53.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.441+0000 7f15ad357740 1 Processor -- start 2024-09-11T22:58:53.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15ad357740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c635d0 0x559a30c6ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15ad357740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c64b40 0x559a30c6d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15ad357740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c6da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15ad357740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x559a30c73040 con 0x559a30c70580 2024-09-11T22:58:53.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15ad357740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559a30b84060 con 0x559a30c64b40 2024-09-11T22:58:53.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15ad357740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559a30c731c0 con 0x559a30c635d0 2024-09-11T22:58:53.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15ad357740 10 client.0 _ll_register_callbacks cb 0x559a308d78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:53.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15a951b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c6da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15a8d1a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c635d0 0x559a30c6ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15a8d1a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c635d0 0x559a30c6ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:55574/0 (socket says 172.21.15.152:55574) 2024-09-11T22:58:53.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15a8d1a640 1 -- 172.21.15.152:0/2950966652 learned_addr learned my addr 172.21.15.152:0/2950966652 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:53.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15a951b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c6da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52244/0 (socket says 172.21.15.152:52244) 2024-09-11T22:58:53.471 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15a3fff640 1 --2- 172.21.15.152:0/2950966652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c64b40 0x559a30c6d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15a951b640 1 -- 172.21.15.152:0/2950966652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c635d0 msgr2=0x559a30c6ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:53.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15a951b640 1 --2- 172.21.15.152:0/2950966652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c635d0 0x559a30c6ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15a951b640 1 -- 172.21.15.152:0/2950966652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c64b40 msgr2=0x559a30c6d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:53.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15a951b640 1 --2- 172.21.15.152:0/2950966652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c64b40 0x559a30c6d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15a951b640 1 -- 172.21.15.152:0/2950966652 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a3093a3c0 con 0x559a30c70580 2024-09-11T22:58:53.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15a8d1a640 1 --2- 172.21.15.152:0/2950966652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c635d0 0x559a30c6ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:58:53.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f15a951b640 1 --2- 172.21.15.152:0/2950966652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c6da20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f159c009fd0 tx=0x7f159c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528309bc67dda139 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:53.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.442+0000 7f1583fff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:53.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.443+0000 7f1583fff640 1 -- 172.21.15.152:0/2950966652 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c004d20 con 0x559a30c70580 2024-09-11T22:58:53.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.443+0000 7f1583fff640 1 -- 172.21.15.152:0/2950966652 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f159c00fb40 con 0x559a30c70580 2024-09-11T22:58:53.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.443+0000 7f15ad357740 1 -- 172.21.15.152:0/2950966652 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a30bd6860 con 0x559a30c70580 2024-09-11T22:58:53.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.443+0000 7f1583fff640 1 -- 172.21.15.152:0/2950966652 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f159c00fd40 con 0x559a30c70580 2024-09-11T22:58:53.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.443+0000 7f15ad357740 10 client.17100 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:53.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.443+0000 7f15ad357740 1 -- 172.21.15.152:0/2950966652 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559a30c7eb00 con 0x559a30c70580 2024-09-11T22:58:53.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.443+0000 7f1583fff640 1 -- 172.21.15.152:0/2950966652 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559a30bd6860 con 0x559a30c70580 2024-09-11T22:58:53.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.443+0000 7f1583fff640 1 -- 172.21.15.152:0/2950966652 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f159c01d040 con 0x559a30c70580 2024-09-11T22:58:53.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.443+0000 7f15ad357740 10 client.17100 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:53.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.443+0000 7f15ad357740 10 client.17100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:58:53.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.443+0000 7f15ad357740 20 client.17100 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:53.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.443+0000 7f15ad357740 1 --2- 172.21.15.152:0/2950966652 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559a30c7ff10 0x559a30c82300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.443+0000 7f15ad357740 4 client.17100 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:58:53.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.443+0000 7f15ad357740 1 -- 172.21.15.152:0/2950966652 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a30bd0d70 con 0x559a30c7ff10 2024-09-11T22:58:53.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.443+0000 7f15a8d1a640 1 --2- 172.21.15.152:0/2950966652 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559a30c7ff10 0x559a30c82300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.444+0000 7f1583fff640 10 client.17100 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:53.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.444+0000 7f15a8d1a640 1 --2- 172.21.15.152:0/2950966652 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559a30c7ff10 0x559a30c82300 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92e7818aee37f569 server_cookie=2c842c8400fd1b1e in_seq=0 out_seq=0 2024-09-11T22:58:53.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.445+0000 7f1583fff640 1 -- 172.21.15.152:0/2950966652 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a30bd0d70 con 0x559a30c7ff10 2024-09-11T22:58:53.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.445+0000 7f1583fff640 10 client.17100 handle_command_reply: tid=0 2024-09-11T22:58:53.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.445+0000 7f15ad357740 1 client.17100 shutdown 2024-09-11T22:58:53.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.445+0000 7f15ad357740 1 -- 172.21.15.152:0/2950966652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 msgr2=0x559a30c6da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:53.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.445+0000 7f15ad357740 1 --2- 172.21.15.152:0/2950966652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c6da20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f159c009fd0 tx=0x7f159c004a40 comp rx=0 tx=0).stop 2024-09-11T22:58:53.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.445+0000 7f15ad357740 1 -- 172.21.15.152:0/2950966652 shutdown_connections 2024-09-11T22:58:53.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.445+0000 7f15ad357740 1 --2- 172.21.15.152:0/2950966652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c6da20 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.445+0000 7f15ad357740 1 --2- 172.21.15.152:0/2950966652 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559a30c7ff10 0x559a30c82300 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.445+0000 7f1583fff640 0 client.17100 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:53.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.445+0000 7f15ad357740 1 --2- 172.21.15.152:0/2950966652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c64b40 0x559a30c6d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.445+0000 7f15ad357740 1 --2- 172.21.15.152:0/2950966652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c635d0 0x559a30c6ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.445+0000 7f15ad357740 1 -- 172.21.15.152:0/2950966652 >> 172.21.15.152:0/2950966652 conn(0x559a30c2c310 msgr2=0x559a30c65990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:53.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.445+0000 7f15ad357740 1 -- 172.21.15.152:0/2950966652 shutdown_connections 2024-09-11T22:58:53.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.445+0000 7f15ad357740 1 -- 172.21.15.152:0/2950966652 wait complete. 2024-09-11T22:58:53.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.445+0000 7f15ad357740 20 client.17100 trim_cache size 0 max 16384 2024-09-11T22:58:53.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15ad357740 1 Processor -- start 2024-09-11T22:58:53.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15ad357740 1 -- start start 2024-09-11T22:58:53.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15ad357740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c635d0 0x559a30c5fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15ad357740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c64b40 0x559a30c6cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15ad357740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c87bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15ad357740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41ab5d0 con 0x559a30c70580 2024-09-11T22:58:53.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15ad357740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41ad110 con 0x559a30c635d0 2024-09-11T22:58:53.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15ad357740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41abca0 con 0x559a30c64b40 2024-09-11T22:58:53.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15a951b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c87bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15a951b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c87bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52252/0 (socket says 172.21.15.152:52252) 2024-09-11T22:58:53.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15a951b640 1 -- 172.21.15.152:0/3445514096 learned_addr learned my addr 172.21.15.152:0/3445514096 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:53.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15a3fff640 1 --2- 172.21.15.152:0/3445514096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c64b40 0x559a30c6cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15a8d1a640 1 --2- 172.21.15.152:0/3445514096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c635d0 0x559a30c5fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15a951b640 1 -- 172.21.15.152:0/3445514096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c64b40 msgr2=0x559a30c6cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:53.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15a951b640 1 --2- 172.21.15.152:0/3445514096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c64b40 0x559a30c6cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15a951b640 1 -- 172.21.15.152:0/3445514096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c635d0 msgr2=0x559a30c5fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:53.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15a951b640 1 --2- 172.21.15.152:0/3445514096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c635d0 0x559a30c5fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15a951b640 1 -- 172.21.15.152:0/3445514096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a30c82670 con 0x559a30c70580 2024-09-11T22:58:53.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.458+0000 7f15a951b640 1 --2- 172.21.15.152:0/3445514096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c87bd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f159c009c20 tx=0x7f159c0098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63a7601174b6766 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:53.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.459+0000 7f158a7fc640 1 -- 172.21.15.152:0/3445514096 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c0294e0 con 0x559a30c70580 2024-09-11T22:58:53.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.459+0000 7f158a7fc640 1 -- 172.21.15.152:0/3445514096 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f159c029680 con 0x559a30c70580 2024-09-11T22:58:53.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.459+0000 7f158a7fc640 1 -- 172.21.15.152:0/3445514096 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c01fa40 con 0x559a30c70580 2024-09-11T22:58:53.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.459+0000 7f15ad357740 1 -- 172.21.15.152:0/3445514096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 msgr2=0x559a30c87bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:53.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.459+0000 7f15ad357740 1 --2- 172.21.15.152:0/3445514096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c87bd0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f159c009c20 tx=0x7f159c0098f0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.459+0000 7f15ad357740 1 -- 172.21.15.152:0/3445514096 shutdown_connections 2024-09-11T22:58:53.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.459+0000 7f15ad357740 1 --2- 172.21.15.152:0/3445514096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c87bd0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.459+0000 7f15ad357740 1 --2- 172.21.15.152:0/3445514096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c64b40 0x559a30c6cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.459+0000 7f15ad357740 1 --2- 172.21.15.152:0/3445514096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c635d0 0x559a30c5fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.459+0000 7f15ad357740 1 -- 172.21.15.152:0/3445514096 >> 172.21.15.152:0/3445514096 conn(0x559a30c2c310 msgr2=0x559a30c5f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:53.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.459+0000 7f15ad357740 1 -- 172.21.15.152:0/3445514096 shutdown_connections 2024-09-11T22:58:53.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.459+0000 7f15ad357740 1 -- 172.21.15.152:0/3445514096 wait complete. 2024-09-11T22:58:53.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.460+0000 7f15ad357740 1 -- start start 2024-09-11T22:58:53.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.460+0000 7f15ad357740 1 Processor -- start 2024-09-11T22:58:53.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.460+0000 7f15ad357740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c635d0 0x559a30c78a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.460+0000 7f15ad357740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c64b40 0x559a30c69ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.460+0000 7f15ad357740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c6a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.460+0000 7f15ad357740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41aca40 con 0x559a30c70580 2024-09-11T22:58:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.460+0000 7f15ad357740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41ac370 con 0x559a30c64b40 2024-09-11T22:58:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.460+0000 7f15ad357740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41a9590 con 0x559a30c635d0 2024-09-11T22:58:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.460+0000 7f15ad357740 10 client.0 _ll_register_callbacks cb 0x7f159c009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.460+0000 7f15a951b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c6a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.460+0000 7f15a8d1a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c635d0 0x559a30c78a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.460+0000 7f15a951b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c6a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52260/0 (socket says 172.21.15.152:52260) 2024-09-11T22:58:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.460+0000 7f15a951b640 1 -- 172.21.15.152:0/2891310549 learned_addr learned my addr 172.21.15.152:0/2891310549 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.460+0000 7f15a3fff640 1 --2- 172.21.15.152:0/2891310549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c64b40 0x559a30c69ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.461+0000 7f15a951b640 1 -- 172.21.15.152:0/2891310549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c635d0 msgr2=0x559a30c78a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.461+0000 7f15a951b640 1 --2- 172.21.15.152:0/2891310549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c635d0 0x559a30c78a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.461+0000 7f15a951b640 1 -- 172.21.15.152:0/2891310549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c64b40 msgr2=0x559a30c69ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:53.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.461+0000 7f15a951b640 1 --2- 172.21.15.152:0/2891310549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c64b40 0x559a30c69ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.461+0000 7f15a951b640 1 -- 172.21.15.152:0/2891310549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a30c6aad0 con 0x559a30c70580 2024-09-11T22:58:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.461+0000 7f15a3fff640 1 --2- 172.21.15.152:0/2891310549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c64b40 0x559a30c69ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.461+0000 7f15a951b640 1 --2- 172.21.15.152:0/2891310549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c6a3e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f159c03d040 tx=0x7f159c009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3101ea4e27fb18 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.461+0000 7f1583fff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.461+0000 7f1583fff640 1 -- 172.21.15.152:0/2891310549 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c04a060 con 0x559a30c70580 2024-09-11T22:58:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.461+0000 7f1583fff640 1 -- 172.21.15.152:0/2891310549 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f159c00fb40 con 0x559a30c70580 2024-09-11T22:58:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.461+0000 7f15ad357740 1 -- 172.21.15.152:0/2891310549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f159c029680 con 0x559a30c70580 2024-09-11T22:58:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.461+0000 7f1583fff640 1 -- 172.21.15.152:0/2891310549 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f159c0294e0 con 0x559a30c70580 2024-09-11T22:58:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.461+0000 7f15ad357740 10 client.17106 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.461+0000 7f15ad357740 1 -- 172.21.15.152:0/2891310549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559a30ceaa60 con 0x559a30c70580 2024-09-11T22:58:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.461+0000 7f1583fff640 1 -- 172.21.15.152:0/2891310549 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f159c029680 con 0x559a30c70580 2024-09-11T22:58:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.462+0000 7f1583fff640 1 -- 172.21.15.152:0/2891310549 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f159c0325f0 con 0x559a30c70580 2024-09-11T22:58:53.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.462+0000 7f15ad357740 10 client.17106 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:53.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.462+0000 7f15ad357740 10 client.17106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:58:53.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.462+0000 7f15ad357740 20 client.17106 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:53.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.462+0000 7f15ad357740 1 --2- 172.21.15.152:0/2891310549 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559a30c7ff10 0x559a30ced0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:53.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.462+0000 7f15ad357740 4 client.17106 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:58:53.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.462+0000 7f15ad357740 1 -- 172.21.15.152:0/2891310549 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a30b4d170 con 0x559a30c7ff10 2024-09-11T22:58:53.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.462+0000 7f15a8d1a640 1 --2- 172.21.15.152:0/2891310549 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559a30c7ff10 0x559a30ced0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:53.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.462+0000 7f1583fff640 10 client.17106 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:53.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.462+0000 7f15a8d1a640 1 --2- 172.21.15.152:0/2891310549 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559a30c7ff10 0x559a30ced0b0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d267e50ffc4458e server_cookie=dea91b2ee5ce5d78 in_seq=0 out_seq=0 2024-09-11T22:58:53.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.462+0000 7f1583fff640 1 -- 172.21.15.152:0/2891310549 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a30b4d170 con 0x559a30c7ff10 2024-09-11T22:58:53.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.462+0000 7f1583fff640 10 client.17106 handle_command_reply: tid=0 2024-09-11T22:58:53.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.462+0000 7f15ad357740 1 client.17106 shutdown 2024-09-11T22:58:53.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.463+0000 7f15ad357740 1 -- 172.21.15.152:0/2891310549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 msgr2=0x559a30c6a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:53.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.463+0000 7f15ad357740 1 --2- 172.21.15.152:0/2891310549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c6a3e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f159c03d040 tx=0x7f159c009d90 comp rx=0 tx=0).stop 2024-09-11T22:58:53.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.463+0000 7f15ad357740 1 -- 172.21.15.152:0/2891310549 shutdown_connections 2024-09-11T22:58:53.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.463+0000 7f15ad357740 1 --2- 172.21.15.152:0/2891310549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a30c70580 0x559a30c6a3e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.463+0000 7f15ad357740 1 --2- 172.21.15.152:0/2891310549 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x559a30c7ff10 0x559a30ced0b0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.463+0000 7f15ad357740 1 --2- 172.21.15.152:0/2891310549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a30c64b40 0x559a30c69ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.463+0000 7f15ad357740 1 --2- 172.21.15.152:0/2891310549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a30c635d0 0x559a30c78a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.463+0000 7f15ad357740 1 -- 172.21.15.152:0/2891310549 >> 172.21.15.152:0/2891310549 conn(0x559a30c2c310 msgr2=0x559a30c5f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:53.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.463+0000 7f1583fff640 0 client.17106 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:53.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.463+0000 7f15ad357740 1 -- 172.21.15.152:0/2891310549 shutdown_connections 2024-09-11T22:58:53.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.463+0000 7f15ad357740 1 -- 172.21.15.152:0/2891310549 wait complete. 2024-09-11T22:58:53.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.463+0000 7f15ad357740 20 client.17106 trim_cache size 0 max 16384 2024-09-11T22:58:53.481 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:58:53.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.463+0000 7f15aaf7e640 1 -- 172.21.15.152:0/2057045804 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f158c0a2cf0 msgr2=0x7f158c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:53.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.463+0000 7f15aaf7e640 1 --2- 172.21.15.152:0/2057045804 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f158c0a2cf0 0x7f158c0a51b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f159c009270 tx=0x7f159c005bd0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.463+0000 7f15aaf7e640 1 -- 172.21.15.152:0/2057045804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a4101b40 msgr2=0x7f15a4197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:53.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.463+0000 7f15aaf7e640 1 --2- 172.21.15.152:0/2057045804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a4101b40 0x7f15a4197940 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f1598006010 tx=0x7f15980042f0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.464+0000 7f15aaf7e640 1 -- 172.21.15.152:0/2057045804 shutdown_connections 2024-09-11T22:58:53.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.464+0000 7f15aaf7e640 1 --2- 172.21.15.152:0/2057045804 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f158c0a2cf0 0x7f158c0a51b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.464+0000 7f15aaf7e640 1 --2- 172.21.15.152:0/2057045804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15a4106c30 0x7f15a419bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.464+0000 7f15aaf7e640 1 --2- 172.21.15.152:0/2057045804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15a4101b40 0x7f15a4197940 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.464+0000 7f15aaf7e640 1 --2- 172.21.15.152:0/2057045804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15a41011a0 0x7f15a4197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:53.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.464+0000 7f15aaf7e640 1 -- 172.21.15.152:0/2057045804 >> 172.21.15.152:0/2057045804 conn(0x7f15a4077210 msgr2=0x7f15a4103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:53.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.464+0000 7f15aaf7e640 1 -- 172.21.15.152:0/2057045804 shutdown_connections 2024-09-11T22:58:53.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:53.464+0000 7f15aaf7e640 1 -- 172.21.15.152:0/2057045804 wait complete. 2024-09-11T22:58:54.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:54 smithi152 ceph-mon[31181]: pgmap v183: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:58:54.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:54 smithi191 ceph-mon[35211]: pgmap v183: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:58:54.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:54 smithi204 ceph-mon[35454]: pgmap v183: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:58:55.200 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:58:55.200 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:58:55.459 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:55.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.987+0000 7fa6a65e7640 1 -- 172.21.15.152:0/483480481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6a0102d40 msgr2=0x7fa6a0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:55.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.987+0000 7fa6a65e7640 1 --2- 172.21.15.152:0/483480481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6a0102d40 0x7fa6a0103120 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fa690009900 tx=0x7fa69002a480 comp rx=0 tx=0).stop 2024-09-11T22:58:55.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.987+0000 7fa6a65e7640 1 -- 172.21.15.152:0/483480481 shutdown_connections 2024-09-11T22:58:55.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.987+0000 7fa6a65e7640 1 --2- 172.21.15.152:0/483480481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6a010c170 0x7fa6a010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:55.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.987+0000 7fa6a65e7640 1 --2- 172.21.15.152:0/483480481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6a0103660 0x7fa6a010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:55.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.987+0000 7fa6a65e7640 1 --2- 172.21.15.152:0/483480481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6a0102d40 0x7fa6a0103120 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:55.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.987+0000 7fa6a65e7640 1 -- 172.21.15.152:0/483480481 >> 172.21.15.152:0/483480481 conn(0x7fa6a00fc7a0 msgr2=0x7fa6a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:55.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.987+0000 7fa6a65e7640 1 -- 172.21.15.152:0/483480481 shutdown_connections 2024-09-11T22:58:55.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.987+0000 7fa6a65e7640 1 -- 172.21.15.152:0/483480481 wait complete. 2024-09-11T22:58:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.988+0000 7fa6a65e7640 1 Processor -- start 2024-09-11T22:58:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.988+0000 7fa6a65e7640 1 -- start start 2024-09-11T22:58:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa6a65e7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6a0102d40 0x7fa6a01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa6a65e7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6a0103660 0x7fa6a0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa6a65e7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6a010c170 0x7fa6a019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:55.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa6a65e7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a0111fd0 con 0x7fa6a010c170 2024-09-11T22:58:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa6a65e7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a0111e50 con 0x7fa6a0102d40 2024-09-11T22:58:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa6a65e7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a0112150 con 0x7fa6a0103660 2024-09-11T22:58:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa6a4b84640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6a010c170 0x7fa6a019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa6a4b84640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6a010c170 0x7fa6a019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52270/0 (socket says 172.21.15.152:52270) 2024-09-11T22:58:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa6a4b84640 1 -- 172.21.15.152:0/187652949 learned_addr learned my addr 172.21.15.152:0/187652949 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa69ffff640 1 --2- 172.21.15.152:0/187652949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6a0102d40 0x7fa6a01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa69f7fe640 1 --2- 172.21.15.152:0/187652949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6a0103660 0x7fa6a0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa6a4b84640 1 -- 172.21.15.152:0/187652949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6a0103660 msgr2=0x7fa6a0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa6a4b84640 1 --2- 172.21.15.152:0/187652949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6a0103660 0x7fa6a0198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa6a4b84640 1 -- 172.21.15.152:0/187652949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6a0102d40 msgr2=0x7fa6a01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa6a4b84640 1 --2- 172.21.15.152:0/187652949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6a0102d40 0x7fa6a01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa6a4b84640 1 -- 172.21.15.152:0/187652949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a019d480 con 0x7fa6a010c170 2024-09-11T22:58:55.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.989+0000 7fa6a4b84640 1 --2- 172.21.15.152:0/187652949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6a010c170 0x7fa6a019cda0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa69800eaa0 tx=0x7fa69800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:55.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.990+0000 7fa69d7fa640 1 -- 172.21.15.152:0/187652949 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69800cdb0 con 0x7fa6a010c170 2024-09-11T22:58:55.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.990+0000 7fa6a65e7640 1 -- 172.21.15.152:0/187652949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a019d770 con 0x7fa6a010c170 2024-09-11T22:58:55.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.990+0000 7fa69d7fa640 1 -- 172.21.15.152:0/187652949 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa698005020 con 0x7fa6a010c170 2024-09-11T22:58:55.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.990+0000 7fa69d7fa640 1 -- 172.21.15.152:0/187652949 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6980116b0 con 0x7fa6a010c170 2024-09-11T22:58:55.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.990+0000 7fa6a65e7640 1 -- 172.21.15.152:0/187652949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a01a5050 con 0x7fa6a010c170 2024-09-11T22:58:55.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.992+0000 7fa69d7fa640 1 -- 172.21.15.152:0/187652949 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa698005560 con 0x7fa6a010c170 2024-09-11T22:58:55.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.992+0000 7fa6a89c0740 1 Processor -- start 2024-09-11T22:58:55.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.992+0000 7fa6a89c0740 1 -- start start 2024-09-11T22:58:55.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa6a89c0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e48b60 0x5577b1e48f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:55.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa6a89c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e49480 0x5577b1e53800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:55.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa6a89c0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e53d40 0x5577b1e56130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:55.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa6a89c0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5577b1dae280 con 0x5577b1e48b60 2024-09-11T22:58:55.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa6a89c0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5577b1d55b10 con 0x5577b1e53d40 2024-09-11T22:58:55.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa6a89c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5577b1daa240 con 0x5577b1e49480 2024-09-11T22:58:55.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa69f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e48b60 0x5577b1e48f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:55.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa69d7fa640 1 --2- 172.21.15.152:0/187652949 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6880a2cf0 0x7fa6880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa69f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e48b60 0x5577b1e48f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52284/0 (socket says 172.21.15.152:52284) 2024-09-11T22:58:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa69f7fe640 1 -- 172.21.15.152:0/2612048401 learned_addr learned my addr 172.21.15.152:0/2612048401 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa69ffff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e49480 0x5577b1e53800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa6a4b84640 1 --2- 172.21.15.152:0/187652949 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6880a2cf0 0x7fa6880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa69f7fe640 1 --2- 172.21.15.152:0/2612048401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e53d40 0x5577b1e56130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa69d7fa640 1 -- 172.21.15.152:0/187652949 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa6980cf600 con 0x7fa6a010c170 2024-09-11T22:58:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa69f7fe640 1 -- 172.21.15.152:0/2612048401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e49480 msgr2=0x5577b1e53800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa69f7fe640 1 --2- 172.21.15.152:0/2612048401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e49480 0x5577b1e53800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa69f7fe640 1 -- 172.21.15.152:0/2612048401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e53d40 msgr2=0x5577b1e56130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa69f7fe640 1 --2- 172.21.15.152:0/2612048401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e53d40 0x5577b1e56130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa69f7fe640 1 -- 172.21.15.152:0/2612048401 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b1d621f0 con 0x5577b1e48b60 2024-09-11T22:58:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa69ffff640 1 --2- 172.21.15.152:0/2612048401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e49480 0x5577b1e53800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:58:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.993+0000 7fa69f7fe640 1 --2- 172.21.15.152:0/2612048401 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e48b60 0x5577b1e48f40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa694006aa0 tx=0x7fa6940095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b1c835bcd9fa31 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:55.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.994+0000 7fa6a4b84640 1 --2- 172.21.15.152:0/187652949 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6880a2cf0 0x7fa6880a51b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa6980985b0 tx=0x7fa698000f40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.994+0000 7fa6867fc640 1 -- 172.21.15.152:0/2612048401 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694017610 con 0x5577b1e48b60 2024-09-11T22:58:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.994+0000 7fa6867fc640 1 -- 172.21.15.152:0/2612048401 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa69401c070 con 0x5577b1e48b60 2024-09-11T22:58:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.994+0000 7fa6867fc640 1 -- 172.21.15.152:0/2612048401 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694015910 con 0x5577b1e48b60 2024-09-11T22:58:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.994+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2612048401 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e48b60 msgr2=0x5577b1e48f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.994+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/2612048401 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e48b60 0x5577b1e48f40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa694006aa0 tx=0x7fa6940095c0 comp rx=0 tx=0).stop 2024-09-11T22:58:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.994+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2612048401 shutdown_connections 2024-09-11T22:58:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.994+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/2612048401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e53d40 0x5577b1e56130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.994+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/2612048401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e49480 0x5577b1e53800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.994+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/2612048401 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e48b60 0x5577b1e48f40 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.994+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2612048401 >> 172.21.15.152:0/2612048401 conn(0x5577b1e450f0 msgr2=0x5577b1e454d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.995+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2612048401 shutdown_connections 2024-09-11T22:58:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.995+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2612048401 wait complete. 2024-09-11T22:58:55.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.995+0000 7fa6a89c0740 1 -- start start 2024-09-11T22:58:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.995+0000 7fa6a89c0740 1 Processor -- start 2024-09-11T22:58:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.995+0000 7fa6a89c0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e48b60 0x5577b1e53800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:56.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.995+0000 7fa6a89c0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e49480 0x5577b1e50b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.995+0000 7fa6a89c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e53d40 0x5577b1e512e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.995+0000 7fa6a89c0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5577b1e56910 con 0x5577b1e48b60 2024-09-11T22:58:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.995+0000 7fa6a89c0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5577b1d68f10 con 0x5577b1e49480 2024-09-11T22:58:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.995+0000 7fa6a89c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5577b1e56a90 con 0x5577b1e53d40 2024-09-11T22:58:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.995+0000 7fa6a89c0740 10 client.0 _ll_register_callbacks cb 0x5577b1d31a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.995+0000 7fa69f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e48b60 0x5577b1e53800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.995+0000 7fa69f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e48b60 0x5577b1e53800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52294/0 (socket says 172.21.15.152:52294) 2024-09-11T22:58:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.995+0000 7fa69f7fe640 1 -- 172.21.15.152:0/2250888945 learned_addr learned my addr 172.21.15.152:0/2250888945 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.996+0000 7fa69f7fe640 1 --2- 172.21.15.152:0/2250888945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e53d40 0x5577b1e512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.996+0000 7fa69ffff640 1 --2- 172.21.15.152:0/2250888945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e49480 0x5577b1e50b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.996+0000 7fa69f7fe640 1 -- 172.21.15.152:0/2250888945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e53d40 msgr2=0x5577b1e512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:56.002 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.996+0000 7fa69f7fe640 1 --2- 172.21.15.152:0/2250888945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e53d40 0x5577b1e512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.996+0000 7fa69f7fe640 1 -- 172.21.15.152:0/2250888945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e49480 msgr2=0x5577b1e50b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.996+0000 7fa69f7fe640 1 --2- 172.21.15.152:0/2250888945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e49480 0x5577b1e50b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.996+0000 7fa69f7fe640 1 -- 172.21.15.152:0/2250888945 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b1b1f6a0 con 0x5577b1e48b60 2024-09-11T22:58:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.996+0000 7fa69ffff640 1 --2- 172.21.15.152:0/2250888945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e49480 0x5577b1e50b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.996+0000 7fa69f7fe640 1 --2- 172.21.15.152:0/2250888945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e48b60 0x5577b1e53800 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa694006a70 tx=0x7fa6940154f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19d90e0d8b1755c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.996+0000 7fa67f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.996+0000 7fa67f7fe640 1 -- 172.21.15.152:0/2250888945 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69402c060 con 0x5577b1e48b60 2024-09-11T22:58:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.996+0000 7fa67f7fe640 1 -- 172.21.15.152:0/2250888945 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa694015e20 con 0x5577b1e48b60 2024-09-11T22:58:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.996+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2250888945 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa69401c070 con 0x5577b1e48b60 2024-09-11T22:58:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.996+0000 7fa67f7fe640 1 -- 172.21.15.152:0/2250888945 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa694017410 con 0x5577b1e48b60 2024-09-11T22:58:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.996+0000 7fa6a89c0740 10 client.17118 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.996+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2250888945 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5577b1e62440 con 0x5577b1e48b60 2024-09-11T22:58:56.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.997+0000 7fa67f7fe640 1 -- 172.21.15.152:0/2250888945 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa69401c070 con 0x5577b1e48b60 2024-09-11T22:58:56.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.997+0000 7fa67f7fe640 1 -- 172.21.15.152:0/2250888945 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa694020930 con 0x5577b1e48b60 2024-09-11T22:58:56.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.997+0000 7fa6a89c0740 10 client.17118 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:56.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.997+0000 7fa6a89c0740 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:58:56.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.997+0000 7fa6a89c0740 20 client.17118 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:56.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.997+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/2250888945 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5577b1e638a0 0x5577b1e65c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:56.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.997+0000 7fa6a89c0740 4 client.17118 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:58:56.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.997+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2250888945 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577b1dbb7f0 con 0x5577b1e638a0 2024-09-11T22:58:56.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.997+0000 7fa69ffff640 1 --2- 172.21.15.152:0/2250888945 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5577b1e638a0 0x5577b1e65c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:56.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.997+0000 7fa69ffff640 1 --2- 172.21.15.152:0/2250888945 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5577b1e638a0 0x5577b1e65c90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc72b14f29e5bdf9 server_cookie=360955ec65ab9697 in_seq=0 out_seq=0 2024-09-11T22:58:56.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.997+0000 7fa67f7fe640 10 client.17118 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:56.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.998+0000 7fa67f7fe640 1 -- 172.21.15.152:0/2250888945 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577b1dbb7f0 con 0x5577b1e638a0 2024-09-11T22:58:56.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.998+0000 7fa67f7fe640 10 client.17118 handle_command_reply: tid=0 2024-09-11T22:58:56.004 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.998+0000 7fa6a89c0740 1 client.17118 shutdown 2024-09-11T22:58:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.998+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2250888945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e48b60 msgr2=0x5577b1e53800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.998+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/2250888945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e48b60 0x5577b1e53800 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa694006a70 tx=0x7fa6940154f0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.998+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2250888945 shutdown_connections 2024-09-11T22:58:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.998+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/2250888945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e53d40 0x5577b1e512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.998+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/2250888945 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5577b1e638a0 0x5577b1e65c90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.998+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/2250888945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e49480 0x5577b1e50b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.998+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/2250888945 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e48b60 0x5577b1e53800 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.998+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2250888945 >> 172.21.15.152:0/2250888945 conn(0x5577b1e450f0 msgr2=0x5577b1e44270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.998+0000 7fa67f7fe640 0 client.17118 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.998+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2250888945 shutdown_connections 2024-09-11T22:58:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.998+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2250888945 wait complete. 2024-09-11T22:58:56.005 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:55.999+0000 7fa6a89c0740 20 client.17118 trim_cache size 0 max 16384 2024-09-11T22:58:56.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.015+0000 7fa6a89c0740 1 Processor -- start 2024-09-11T22:58:56.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.015+0000 7fa6a89c0740 1 -- start start 2024-09-11T22:58:56.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa6a89c0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e48b60 0x5577b1e4fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:56.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa6a89c0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e49480 0x5577b1e68cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:56.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa6a89c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e53d40 0x5577b1e69200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:56.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa6a89c0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a01ad860 con 0x5577b1e49480 2024-09-11T22:58:56.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa6a89c0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a01af3a0 con 0x5577b1e48b60 2024-09-11T22:58:56.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa6a89c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a01adf30 con 0x5577b1e53d40 2024-09-11T22:58:56.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa69ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e49480 0x5577b1e68cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:56.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa69ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e49480 0x5577b1e68cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52304/0 (socket says 172.21.15.152:52304) 2024-09-11T22:58:56.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa69ffff640 1 -- 172.21.15.152:0/3150618035 learned_addr learned my addr 172.21.15.152:0/3150618035 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:56.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa69f7fe640 1 --2- 172.21.15.152:0/3150618035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e53d40 0x5577b1e69200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:56.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa69ffff640 1 -- 172.21.15.152:0/3150618035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e53d40 msgr2=0x5577b1e69200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:56.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa69ffff640 1 --2- 172.21.15.152:0/3150618035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e53d40 0x5577b1e69200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa69ffff640 1 -- 172.21.15.152:0/3150618035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e48b60 msgr2=0x5577b1e4fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:56.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa69ffff640 1 --2- 172.21.15.152:0/3150618035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e48b60 0x5577b1e4fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa69ffff640 1 -- 172.21.15.152:0/3150618035 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b1e40830 con 0x5577b1e49480 2024-09-11T22:58:56.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa69f7fe640 1 --2- 172.21.15.152:0/3150618035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e53d40 0x5577b1e69200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:56.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa69ffff640 1 --2- 172.21.15.152:0/3150618035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e49480 0x5577b1e68cc0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa69002a9a0 tx=0x7fa6900058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d3d279c7f2b324 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:56.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa685ffb640 1 -- 172.21.15.152:0/3150618035 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690049070 con 0x5577b1e49480 2024-09-11T22:58:56.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa685ffb640 1 -- 172.21.15.152:0/3150618035 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa690005500 con 0x5577b1e49480 2024-09-11T22:58:56.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.016+0000 7fa685ffb640 1 -- 172.21.15.152:0/3150618035 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690035040 con 0x5577b1e49480 2024-09-11T22:58:56.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.017+0000 7fa6a89c0740 1 -- 172.21.15.152:0/3150618035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e49480 msgr2=0x5577b1e68cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:56.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.017+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/3150618035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e49480 0x5577b1e68cc0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa69002a9a0 tx=0x7fa6900058d0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.017+0000 7fa6a89c0740 1 -- 172.21.15.152:0/3150618035 shutdown_connections 2024-09-11T22:58:56.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.017+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/3150618035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e53d40 0x5577b1e69200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.017+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/3150618035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e49480 0x5577b1e68cc0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.017+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/3150618035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e48b60 0x5577b1e4fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.017+0000 7fa6a89c0740 1 -- 172.21.15.152:0/3150618035 >> 172.21.15.152:0/3150618035 conn(0x5577b1e450f0 msgr2=0x5577b1e4f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.017+0000 7fa6a89c0740 1 -- 172.21.15.152:0/3150618035 shutdown_connections 2024-09-11T22:58:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.017+0000 7fa6a89c0740 1 -- 172.21.15.152:0/3150618035 wait complete. 2024-09-11T22:58:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.017+0000 7fa6a89c0740 1 -- start start 2024-09-11T22:58:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.017+0000 7fa6a89c0740 1 Processor -- start 2024-09-11T22:58:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa6a89c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e48b60 0x5577b1ec5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa6a89c0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e49480 0x5577b1ec58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa6a89c0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e53d40 0x5577b1ec9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa6a89c0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a01aecd0 con 0x5577b1e49480 2024-09-11T22:58:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa6a89c0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a01ae600 con 0x5577b1e53d40 2024-09-11T22:58:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa6a89c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a01ab840 con 0x5577b1e48b60 2024-09-11T22:58:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa6a89c0740 10 client.0 _ll_register_callbacks cb 0x7fa690009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa69ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e49480 0x5577b1ec58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:56.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa69ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e49480 0x5577b1ec58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52318/0 (socket says 172.21.15.152:52318) 2024-09-11T22:58:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa69ffff640 1 -- 172.21.15.152:0/2491983455 learned_addr learned my addr 172.21.15.152:0/2491983455 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa69f7fe640 1 --2- 172.21.15.152:0/2491983455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e48b60 0x5577b1ec5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa69f7fe640 1 --2- 172.21.15.152:0/2491983455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e53d40 0x5577b1ec9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa69ffff640 1 -- 172.21.15.152:0/2491983455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e48b60 msgr2=0x5577b1ec5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa69ffff640 1 --2- 172.21.15.152:0/2491983455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e48b60 0x5577b1ec5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa69ffff640 1 -- 172.21.15.152:0/2491983455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e53d40 msgr2=0x5577b1ec9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa69ffff640 1 --2- 172.21.15.152:0/2491983455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e53d40 0x5577b1ec9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa69ffff640 1 -- 172.21.15.152:0/2491983455 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b1e492b0 con 0x5577b1e49480 2024-09-11T22:58:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa69f7fe640 1 --2- 172.21.15.152:0/2491983455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e53d40 0x5577b1ec9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa69ffff640 1 --2- 172.21.15.152:0/2491983455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e49480 0x5577b1ec58c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa6900096c0 tx=0x7fa690005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e617c885ae80053e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.018+0000 7fa67f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.019+0000 7fa67f7fe640 1 -- 172.21.15.152:0/2491983455 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690049070 con 0x5577b1e49480 2024-09-11T22:58:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.019+0000 7fa67f7fe640 1 -- 172.21.15.152:0/2491983455 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fa690005220 con 0x5577b1e49480 2024-09-11T22:58:56.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.019+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2491983455 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa690005500 con 0x5577b1e49480 2024-09-11T22:58:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.019+0000 7fa67f7fe640 1 -- 172.21.15.152:0/2491983455 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa69005a030 con 0x5577b1e49480 2024-09-11T22:58:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.019+0000 7fa6a89c0740 10 client.17124 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.019+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2491983455 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5577b1ed2390 con 0x5577b1e49480 2024-09-11T22:58:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.019+0000 7fa67f7fe640 1 -- 172.21.15.152:0/2491983455 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa690005500 con 0x5577b1e49480 2024-09-11T22:58:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.019+0000 7fa67f7fe640 1 -- 172.21.15.152:0/2491983455 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa690003bf0 con 0x5577b1e49480 2024-09-11T22:58:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.019+0000 7fa6a89c0740 10 client.17124 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.019+0000 7fa6a89c0740 10 client.17124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:58:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.019+0000 7fa6a89c0740 20 client.17124 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.019+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/2491983455 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5577b1ed2ab0 0x5577b1ed4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.019+0000 7fa6a89c0740 4 client.17124 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:58:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.019+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2491983455 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5577b1db5d00 con 0x5577b1ed2ab0 2024-09-11T22:58:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.020+0000 7fa69f7fe640 1 --2- 172.21.15.152:0/2491983455 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5577b1ed2ab0 0x5577b1ed4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:56.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.020+0000 7fa67f7fe640 10 client.17124 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.020+0000 7fa69f7fe640 1 --2- 172.21.15.152:0/2491983455 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5577b1ed2ab0 0x5577b1ed4ed0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2e6c5606db88628 server_cookie=f940730d8784fc35 in_seq=0 out_seq=0 2024-09-11T22:58:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.021+0000 7fa67f7fe640 1 -- 172.21.15.152:0/2491983455 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5577b1db5d00 con 0x5577b1ed2ab0 2024-09-11T22:58:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.021+0000 7fa67f7fe640 10 client.17124 handle_command_reply: tid=0 2024-09-11T22:58:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.021+0000 7fa6a89c0740 1 client.17124 shutdown 2024-09-11T22:58:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.021+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2491983455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e49480 msgr2=0x5577b1ec58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.021+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/2491983455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e49480 0x5577b1ec58c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fa6900096c0 tx=0x7fa690005780 comp rx=0 tx=0).stop 2024-09-11T22:58:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.021+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2491983455 shutdown_connections 2024-09-11T22:58:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.021+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/2491983455 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5577b1ed2ab0 0x5577b1ed4ed0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.021+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/2491983455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5577b1e53d40 0x5577b1ec9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.021+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/2491983455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5577b1e49480 0x5577b1ec58c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.021+0000 7fa6a89c0740 1 --2- 172.21.15.152:0/2491983455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5577b1e48b60 0x5577b1ec5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.021+0000 7fa67f7fe640 0 client.17124 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.021+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2491983455 >> 172.21.15.152:0/2491983455 conn(0x5577b1e450f0 msgr2=0x5577b1e4fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:56.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.021+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2491983455 shutdown_connections 2024-09-11T22:58:56.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.021+0000 7fa6a89c0740 1 -- 172.21.15.152:0/2491983455 wait complete. 2024-09-11T22:58:56.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.022+0000 7fa6a89c0740 20 client.17124 trim_cache size 0 max 16384 2024-09-11T22:58:56.028 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:58:56.028 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T22:58:56.028 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "406a6121-75cd-408d-8cc4-ed1a80d12721", 2024-09-11T22:58:56.028 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T22:58:56.028 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:58:56.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.022+0000 7fa6a65e7640 1 -- 172.21.15.152:0/187652949 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6880a2cf0 msgr2=0x7fa6880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:56.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.022+0000 7fa6a65e7640 1 --2- 172.21.15.152:0/187652949 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6880a2cf0 0x7fa6880a51b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa6980985b0 tx=0x7fa698000f40 comp rx=0 tx=0).stop 2024-09-11T22:58:56.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.022+0000 7fa6a65e7640 1 -- 172.21.15.152:0/187652949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6a010c170 msgr2=0x7fa6a019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:56.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.022+0000 7fa6a65e7640 1 --2- 172.21.15.152:0/187652949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6a010c170 0x7fa6a019cda0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa69800eaa0 tx=0x7fa69800ef60 comp rx=0 tx=0).stop 2024-09-11T22:58:56.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.022+0000 7fa6a65e7640 1 -- 172.21.15.152:0/187652949 shutdown_connections 2024-09-11T22:58:56.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.022+0000 7fa6a65e7640 1 --2- 172.21.15.152:0/187652949 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6880a2cf0 0x7fa6880a51b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.022+0000 7fa6a65e7640 1 --2- 172.21.15.152:0/187652949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6a010c170 0x7fa6a019cda0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.022+0000 7fa6a65e7640 1 --2- 172.21.15.152:0/187652949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6a0103660 0x7fa6a0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.022+0000 7fa6a65e7640 1 --2- 172.21.15.152:0/187652949 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa6a0102d40 0x7fa6a01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:56.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.022+0000 7fa6a65e7640 1 -- 172.21.15.152:0/187652949 >> 172.21.15.152:0/187652949 conn(0x7fa6a00fc7a0 msgr2=0x7fa6a00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:56.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.022+0000 7fa6a65e7640 1 -- 172.21.15.152:0/187652949 shutdown_connections 2024-09-11T22:58:56.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:56.022+0000 7fa6a65e7640 1 -- 172.21.15.152:0/187652949 wait complete. 2024-09-11T22:58:56.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:56 smithi191 ceph-mon[35211]: pgmap v184: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:58:56.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:56 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T22:58:56.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:56 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:58:56.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:56 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T22:58:56.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:56 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:58:56.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:56 smithi204 ceph-mon[35454]: pgmap v184: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:58:56.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:56 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T22:58:56.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:56 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:58:56.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:56 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T22:58:56.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:56 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:58:56.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:56 smithi152 ceph-mon[31181]: pgmap v184: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:58:56.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:56 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T22:58:56.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:56 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:58:56.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:56 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T22:58:56.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:56 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:58:56.885 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:58:57.132 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:58:57.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.612+0000 7f35bf1c3640 1 -- 172.21.15.152:0/2721413269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b8106a80 msgr2=0x7f35b8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:57.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.612+0000 7f35bf1c3640 1 --2- 172.21.15.152:0/2721413269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b8106a80 0x7f35b8106e60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009930 tx=0x7f35ac02a420 comp rx=0 tx=0).stop 2024-09-11T22:58:57.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.612+0000 7f35bf1c3640 1 -- 172.21.15.152:0/2721413269 shutdown_connections 2024-09-11T22:58:57.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.612+0000 7f35bf1c3640 1 --2- 172.21.15.152:0/2721413269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35b8101420 0x7f35b810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.612+0000 7f35bf1c3640 1 --2- 172.21.15.152:0/2721413269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b8100a80 0x7f35b8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.612+0000 7f35bf1c3640 1 --2- 172.21.15.152:0/2721413269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b8106a80 0x7f35b8106e60 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.612+0000 7f35bf1c3640 1 -- 172.21.15.152:0/2721413269 >> 172.21.15.152:0/2721413269 conn(0x7f35b80fc7a0 msgr2=0x7f35b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:57.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.612+0000 7f35bf1c3640 1 -- 172.21.15.152:0/2721413269 shutdown_connections 2024-09-11T22:58:57.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.612+0000 7f35bf1c3640 1 -- 172.21.15.152:0/2721413269 wait complete. 2024-09-11T22:58:57.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bf1c3640 1 Processor -- start 2024-09-11T22:58:57.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bf1c3640 1 -- start start 2024-09-11T22:58:57.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bf1c3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b8100a80 0x7f35b8197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bf1c3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b8101420 0x7f35b8197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bf1c3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35b8106a80 0x7f35b819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bf1c3640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f35b8110e80 con 0x7f35b8100a80 2024-09-11T22:58:57.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bf1c3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f35b8110d00 con 0x7f35b8101420 2024-09-11T22:58:57.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bf1c3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f35b8111000 con 0x7f35b8106a80 2024-09-11T22:58:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bcf5f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b8100a80 0x7f35b8197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bcf5f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b8100a80 0x7f35b8197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52332/0 (socket says 172.21.15.152:52332) 2024-09-11T22:58:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bcf5f640 1 -- 172.21.15.152:0/690439462 learned_addr learned my addr 172.21.15.152:0/690439462 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35b7fff640 1 --2- 172.21.15.152:0/690439462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b8101420 0x7f35b8197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bd760640 1 --2- 172.21.15.152:0/690439462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35b8106a80 0x7f35b819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bcf5f640 1 -- 172.21.15.152:0/690439462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35b8106a80 msgr2=0x7f35b819bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bcf5f640 1 --2- 172.21.15.152:0/690439462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35b8106a80 0x7f35b819bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bcf5f640 1 -- 172.21.15.152:0/690439462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b8101420 msgr2=0x7f35b8197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bcf5f640 1 --2- 172.21.15.152:0/690439462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b8101420 0x7f35b8197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.613+0000 7f35bcf5f640 1 -- 172.21.15.152:0/690439462 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b819c2f0 con 0x7f35b8100a80 2024-09-11T22:58:57.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.614+0000 7f35bcf5f640 1 --2- 172.21.15.152:0/690439462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b8100a80 0x7f35b8197340 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009230 tx=0x7f35ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:57.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.614+0000 7f35b5ffb640 1 -- 172.21.15.152:0/690439462 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac03f030 con 0x7f35b8100a80 2024-09-11T22:58:57.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.614+0000 7f35bf1c3640 1 -- 172.21.15.152:0/690439462 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b819c580 con 0x7f35b8100a80 2024-09-11T22:58:57.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.614+0000 7f35b5ffb640 1 -- 172.21.15.152:0/690439462 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35ac003ad0 con 0x7f35b8100a80 2024-09-11T22:58:57.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.614+0000 7f35b5ffb640 1 -- 172.21.15.152:0/690439462 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac035070 con 0x7f35b8100a80 2024-09-11T22:58:57.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.614+0000 7f35bf1c3640 1 -- 172.21.15.152:0/690439462 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b81a3ec0 con 0x7f35b8100a80 2024-09-11T22:58:57.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.617+0000 7f35b5ffb640 1 -- 172.21.15.152:0/690439462 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f35ac042030 con 0x7f35b8100a80 2024-09-11T22:58:57.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.617+0000 7f35c159c740 1 Processor -- start 2024-09-11T22:58:57.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.617+0000 7f35c159c740 1 -- start start 2024-09-11T22:58:57.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.617+0000 7f35c159c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7b7320 0x55f1af7b7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.617+0000 7f35c159c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7c40 0x55f1af7c4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.617+0000 7f35c159c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7c4890 0x55f1af7c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.617+0000 7f35c159c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f1af71d2e0 con 0x55f1af7b7320 2024-09-11T22:58:57.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.617+0000 7f35c159c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f1af6c4ce0 con 0x55f1af7b7c40 2024-09-11T22:58:57.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.617+0000 7f35c159c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f1af7192a0 con 0x55f1af7c4890 2024-09-11T22:58:57.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.618+0000 7f35bd760640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7b7320 0x55f1af7b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.618+0000 7f35bd760640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7b7320 0x55f1af7b7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52346/0 (socket says 172.21.15.152:52346) 2024-09-11T22:58:57.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.618+0000 7f35bd760640 1 -- 172.21.15.152:0/3034278604 learned_addr learned my addr 172.21.15.152:0/3034278604 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:57.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.618+0000 7f35bcf5f640 1 --2- 172.21.15.152:0/3034278604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7c40 0x55f1af7c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.618+0000 7f35b5ffb640 1 --2- 172.21.15.152:0/690439462 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f35a80a2cf0 0x7f35a80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.618+0000 7f35b7fff640 1 --2- 172.21.15.152:0/3034278604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7c4890 0x55f1af7c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.618+0000 7f35bd760640 1 --2- 172.21.15.152:0/690439462 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f35a80a2cf0 0x7f35a80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.618+0000 7f35bd760640 1 -- 172.21.15.152:0/3034278604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7c4890 msgr2=0x55f1af7c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:57.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.618+0000 7f35bd760640 1 --2- 172.21.15.152:0/3034278604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7c4890 0x55f1af7c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.618+0000 7f35bd760640 1 -- 172.21.15.152:0/3034278604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7c40 msgr2=0x55f1af7c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:57.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.618+0000 7f35bd760640 1 --2- 172.21.15.152:0/3034278604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7c40 0x55f1af7c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.618+0000 7f35bd760640 1 -- 172.21.15.152:0/3034278604 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1af6a9440 con 0x55f1af7b7320 2024-09-11T22:58:57.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.618+0000 7f35b5ffb640 1 -- 172.21.15.152:0/690439462 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f35ac0efb30 con 0x7f35b8100a80 2024-09-11T22:58:57.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.618+0000 7f35bd760640 1 --2- 172.21.15.152:0/3034278604 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7b7320 0x55f1af7b7700 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f35b0012420 tx=0x7f35b00128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fb442ee4867c3b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:57.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.618+0000 7f35a2ffd640 1 -- 172.21.15.152:0/3034278604 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0021070 con 0x55f1af7b7320 2024-09-11T22:58:57.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.619+0000 7f35a2ffd640 1 -- 172.21.15.152:0/3034278604 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b0012110 con 0x55f1af7b7320 2024-09-11T22:58:57.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.619+0000 7f35bd760640 1 --2- 172.21.15.152:0/690439462 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f35a80a2cf0 0x7f35a80a51b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f35b001e000 tx=0x7f35b0018000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:57.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.619+0000 7f35c159c740 1 -- 172.21.15.152:0/3034278604 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7b7320 msgr2=0x55f1af7b7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:57.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.619+0000 7f35c159c740 1 --2- 172.21.15.152:0/3034278604 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7b7320 0x55f1af7b7700 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f35b0012420 tx=0x7f35b00128e0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.619+0000 7f35c159c740 1 -- 172.21.15.152:0/3034278604 shutdown_connections 2024-09-11T22:58:57.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.619+0000 7f35c159c740 1 --2- 172.21.15.152:0/3034278604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7c4890 0x55f1af7c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.619+0000 7f35c159c740 1 --2- 172.21.15.152:0/3034278604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7c40 0x55f1af7c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.619+0000 7f35c159c740 1 --2- 172.21.15.152:0/3034278604 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7b7320 0x55f1af7b7700 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.619+0000 7f35c159c740 1 -- 172.21.15.152:0/3034278604 >> 172.21.15.152:0/3034278604 conn(0x55f1af7b3a30 msgr2=0x55f1af7b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:57.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.619+0000 7f35c159c740 1 -- 172.21.15.152:0/3034278604 shutdown_connections 2024-09-11T22:58:57.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.620+0000 7f35c159c740 1 -- 172.21.15.152:0/3034278604 wait complete. 2024-09-11T22:58:57.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.620+0000 7f35c159c740 1 -- start start 2024-09-11T22:58:57.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.620+0000 7f35c159c740 1 Processor -- start 2024-09-11T22:58:57.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35c159c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7b7320 0x55f1af7c4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35c159c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7c40 0x55f1af7c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35c159c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7c4890 0x55f1af7c1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35c159c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f1af7c74c0 con 0x55f1af7c4890 2024-09-11T22:58:57.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35c159c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f1af6d8050 con 0x55f1af7b7c40 2024-09-11T22:58:57.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35c159c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f1af7c7640 con 0x55f1af7b7320 2024-09-11T22:58:57.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35c159c740 10 client.0 _ll_register_callbacks cb 0x55f1af718160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:57.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35bd760640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7c4890 0x55f1af7c1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35bd760640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7c4890 0x55f1af7c1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52350/0 (socket says 172.21.15.152:52350) 2024-09-11T22:58:57.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35bd760640 1 -- 172.21.15.152:0/4162037354 learned_addr learned my addr 172.21.15.152:0/4162037354 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:57.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35bcf5f640 1 --2- 172.21.15.152:0/4162037354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7b7320 0x55f1af7c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35b7fff640 1 --2- 172.21.15.152:0/4162037354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7c40 0x55f1af7c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35bd760640 1 -- 172.21.15.152:0/4162037354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7b7320 msgr2=0x55f1af7c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:57.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35bd760640 1 --2- 172.21.15.152:0/4162037354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7b7320 0x55f1af7c4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35bd760640 1 -- 172.21.15.152:0/4162037354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7c40 msgr2=0x55f1af7c13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:57.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35bd760640 1 --2- 172.21.15.152:0/4162037354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7c40 0x55f1af7c13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35bd760640 1 -- 172.21.15.152:0/4162037354 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1af48e3c0 con 0x55f1af7c4890 2024-09-11T22:58:57.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35b7fff640 1 --2- 172.21.15.152:0/4162037354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7c40 0x55f1af7c13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:58:57.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f35bd760640 1 --2- 172.21.15.152:0/4162037354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7c4890 0x55f1af7c1920 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f35b0008670 tx=0x7f35b00083b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c117f2ba1224ae52 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:57.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.621+0000 7f3597fff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:57.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.622+0000 7f3597fff640 1 -- 172.21.15.152:0/4162037354 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0021040 con 0x55f1af7c4890 2024-09-11T22:58:57.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.622+0000 7f3597fff640 1 -- 172.21.15.152:0/4162037354 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f35b0003970 con 0x55f1af7c4890 2024-09-11T22:58:57.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.622+0000 7f35c159c740 1 -- 172.21.15.152:0/4162037354 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b0012110 con 0x55f1af7c4890 2024-09-11T22:58:57.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.622+0000 7f3597fff640 1 -- 172.21.15.152:0/4162037354 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f35b000e5f0 con 0x55f1af7c4890 2024-09-11T22:58:57.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.622+0000 7f35c159c740 10 client.17136 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:57.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.622+0000 7f35c159c740 1 -- 172.21.15.152:0/4162037354 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1af7d2e70 con 0x55f1af7c4890 2024-09-11T22:58:57.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.622+0000 7f3597fff640 1 -- 172.21.15.152:0/4162037354 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b0012110 con 0x55f1af7c4890 2024-09-11T22:58:57.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.622+0000 7f3597fff640 1 -- 172.21.15.152:0/4162037354 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f35b0025440 con 0x55f1af7c4890 2024-09-11T22:58:57.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.622+0000 7f35c159c740 10 client.17136 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:57.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.622+0000 7f35c159c740 10 client.17136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:58:57.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.622+0000 7f35c159c740 20 client.17136 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:57.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.622+0000 7f35c159c740 1 --2- 172.21.15.152:0/4162037354 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f1af7d4280 0x55f1af7d6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.622+0000 7f35c159c740 4 client.17136 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:58:57.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.622+0000 7f35c159c740 1 -- 172.21.15.152:0/4162037354 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1af72a850 con 0x55f1af7d4280 2024-09-11T22:58:57.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.623+0000 7f35bcf5f640 1 --2- 172.21.15.152:0/4162037354 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f1af7d4280 0x55f1af7d6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.623+0000 7f3597fff640 10 client.17136 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:57.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.623+0000 7f35bcf5f640 1 --2- 172.21.15.152:0/4162037354 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f1af7d4280 0x55f1af7d6670 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ba41617b4bd6b6a server_cookie=33dd5022bdfbcb9e in_seq=0 out_seq=0 2024-09-11T22:58:57.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.624+0000 7f3597fff640 1 -- 172.21.15.152:0/4162037354 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1af72a850 con 0x55f1af7d4280 2024-09-11T22:58:57.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.624+0000 7f3597fff640 10 client.17136 handle_command_reply: tid=0 2024-09-11T22:58:57.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.624+0000 7f35c159c740 1 client.17136 shutdown 2024-09-11T22:58:57.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.624+0000 7f35c159c740 1 -- 172.21.15.152:0/4162037354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7c4890 msgr2=0x55f1af7c1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:57.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.624+0000 7f35c159c740 1 --2- 172.21.15.152:0/4162037354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7c4890 0x55f1af7c1920 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f35b0008670 tx=0x7f35b00083b0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.624+0000 7f35c159c740 1 -- 172.21.15.152:0/4162037354 shutdown_connections 2024-09-11T22:58:57.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.624+0000 7f35c159c740 1 --2- 172.21.15.152:0/4162037354 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7c4890 0x55f1af7c1920 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.624+0000 7f35c159c740 1 --2- 172.21.15.152:0/4162037354 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f1af7d4280 0x55f1af7d6670 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.624+0000 7f3597fff640 0 client.17136 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:57.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.624+0000 7f35c159c740 1 --2- 172.21.15.152:0/4162037354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7c40 0x55f1af7c13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.624+0000 7f35c159c740 1 --2- 172.21.15.152:0/4162037354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7b7320 0x55f1af7c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.624+0000 7f35c159c740 1 -- 172.21.15.152:0/4162037354 >> 172.21.15.152:0/4162037354 conn(0x55f1af7b3a30 msgr2=0x55f1af7b96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:57.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.624+0000 7f35c159c740 1 -- 172.21.15.152:0/4162037354 shutdown_connections 2024-09-11T22:58:57.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.624+0000 7f35c159c740 1 -- 172.21.15.152:0/4162037354 wait complete. 2024-09-11T22:58:57.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.624+0000 7f35c159c740 20 client.17136 trim_cache size 0 max 16384 2024-09-11T22:58:57.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.641+0000 7f35c159c740 1 Processor -- start 2024-09-11T22:58:57.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.641+0000 7f35c159c740 1 -- start start 2024-09-11T22:58:57.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.641+0000 7f35c159c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7320 0x55f1af7d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.641+0000 7f35c159c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7b7c40 0x55f1af7dbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.641+0000 7f35c159c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7c4890 0x55f1af7dc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.641+0000 7f35c159c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81ab5b0 con 0x55f1af7c4890 2024-09-11T22:58:57.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.641+0000 7f35c159c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81ad0f0 con 0x55f1af7b7320 2024-09-11T22:58:57.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.641+0000 7f35c159c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81abc80 con 0x55f1af7b7c40 2024-09-11T22:58:57.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.642+0000 7f35bd760640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7c4890 0x55f1af7dc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.642+0000 7f35bd760640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7c4890 0x55f1af7dc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52358/0 (socket says 172.21.15.152:52358) 2024-09-11T22:58:57.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.642+0000 7f35bd760640 1 -- 172.21.15.152:0/789122779 learned_addr learned my addr 172.21.15.152:0/789122779 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:57.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.642+0000 7f35b7fff640 1 --2- 172.21.15.152:0/789122779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7b7c40 0x55f1af7dbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.642+0000 7f35bcf5f640 1 --2- 172.21.15.152:0/789122779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7320 0x55f1af7d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.642+0000 7f35bd760640 1 -- 172.21.15.152:0/789122779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7b7c40 msgr2=0x55f1af7dbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:57.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.642+0000 7f35bd760640 1 --2- 172.21.15.152:0/789122779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7b7c40 0x55f1af7dbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.642+0000 7f35bd760640 1 -- 172.21.15.152:0/789122779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7320 msgr2=0x55f1af7d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:57.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.642+0000 7f35bd760640 1 --2- 172.21.15.152:0/789122779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7320 0x55f1af7d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.642+0000 7f35bd760640 1 -- 172.21.15.152:0/789122779 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1af7d6c50 con 0x55f1af7c4890 2024-09-11T22:58:57.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.642+0000 7f35bcf5f640 1 --2- 172.21.15.152:0/789122779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7320 0x55f1af7d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:57.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.642+0000 7f35bd760640 1 --2- 172.21.15.152:0/789122779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7c4890 0x55f1af7dc3c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f35b0000f80 tx=0x7f35b001afa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2ddf7a729f0bfd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:57.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.642+0000 7f35a27fc640 1 -- 172.21.15.152:0/789122779 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b003a020 con 0x55f1af7c4890 2024-09-11T22:58:57.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.642+0000 7f35a27fc640 1 -- 172.21.15.152:0/789122779 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b0016040 con 0x55f1af7c4890 2024-09-11T22:58:57.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.642+0000 7f35a27fc640 1 -- 172.21.15.152:0/789122779 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0021070 con 0x55f1af7c4890 2024-09-11T22:58:57.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.643+0000 7f35c159c740 1 -- 172.21.15.152:0/789122779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7c4890 msgr2=0x55f1af7dc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:57.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.643+0000 7f35c159c740 1 --2- 172.21.15.152:0/789122779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7c4890 0x55f1af7dc3c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f35b0000f80 tx=0x7f35b001afa0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.643+0000 7f35c159c740 1 -- 172.21.15.152:0/789122779 shutdown_connections 2024-09-11T22:58:57.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.643+0000 7f35c159c740 1 --2- 172.21.15.152:0/789122779 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7c4890 0x55f1af7dc3c0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.643+0000 7f35c159c740 1 --2- 172.21.15.152:0/789122779 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7b7c40 0x55f1af7dbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.643+0000 7f35c159c740 1 --2- 172.21.15.152:0/789122779 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7b7320 0x55f1af7d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.643+0000 7f35c159c740 1 -- 172.21.15.152:0/789122779 >> 172.21.15.152:0/789122779 conn(0x55f1af7b3a30 msgr2=0x55f1af7c3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:57.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.643+0000 7f35c159c740 1 -- 172.21.15.152:0/789122779 shutdown_connections 2024-09-11T22:58:57.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.643+0000 7f35c159c740 1 -- 172.21.15.152:0/789122779 wait complete. 2024-09-11T22:58:57.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.643+0000 7f35c159c740 1 -- start start 2024-09-11T22:58:57.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.643+0000 7f35c159c740 1 Processor -- start 2024-09-11T22:58:57.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35c159c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7b7320 0x55f1af7cd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35c159c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7b7c40 0x55f1af7c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35c159c740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7c4890 0x55f1af7c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35c159c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81aca20 con 0x55f1af7b7320 2024-09-11T22:58:57.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35c159c740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81ac350 con 0x55f1af7c4890 2024-09-11T22:58:57.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35c159c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81a9570 con 0x55f1af7b7c40 2024-09-11T22:58:57.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35c159c740 10 client.0 _ll_register_callbacks cb 0x7f35b001f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:58:57.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35bcf5f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7b7320 0x55f1af7cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35bcf5f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7b7320 0x55f1af7cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52362/0 (socket says 172.21.15.152:52362) 2024-09-11T22:58:57.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35bcf5f640 1 -- 172.21.15.152:0/408181923 learned_addr learned my addr 172.21.15.152:0/408181923 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:58:57.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35b7fff640 1 --2- 172.21.15.152:0/408181923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7b7c40 0x55f1af7c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35bd760640 1 --2- 172.21.15.152:0/408181923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7c4890 0x55f1af7c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35bcf5f640 1 -- 172.21.15.152:0/408181923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7b7c40 msgr2=0x55f1af7c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:57.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35bcf5f640 1 --2- 172.21.15.152:0/408181923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7b7c40 0x55f1af7c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35bcf5f640 1 -- 172.21.15.152:0/408181923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7c4890 msgr2=0x55f1af7c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:58:57.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35bcf5f640 1 --2- 172.21.15.152:0/408181923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7c4890 0x55f1af7c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35bcf5f640 1 -- 172.21.15.152:0/408181923 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1af7c92a0 con 0x55f1af7b7320 2024-09-11T22:58:57.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35bd760640 1 --2- 172.21.15.152:0/408181923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7c4890 0x55f1af7c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:58:57.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35bcf5f640 1 --2- 172.21.15.152:0/408181923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7b7320 0x55f1af7cd170 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0b8a70 tx=0x7f35ac0f1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a7a3fc3961063b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:58:57.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f3597fff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:58:57.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f3597fff640 1 -- 172.21.15.152:0/408181923 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac049040 con 0x55f1af7b7320 2024-09-11T22:58:57.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f3597fff640 1 -- 172.21.15.152:0/408181923 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f35ac0ba3a0 con 0x55f1af7b7320 2024-09-11T22:58:57.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.644+0000 7f35c159c740 1 -- 172.21.15.152:0/408181923 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b0016040 con 0x55f1af7b7320 2024-09-11T22:58:57.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.645+0000 7f3597fff640 1 -- 172.21.15.152:0/408181923 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f35ac0bc0a0 con 0x55f1af7b7320 2024-09-11T22:58:57.654 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.645+0000 7f35c159c740 10 client.17142 fetch_fsmap learned FSMap version 28 2024-09-11T22:58:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.645+0000 7f35c159c740 1 -- 172.21.15.152:0/408181923 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1af83ef30 con 0x55f1af7b7320 2024-09-11T22:58:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.645+0000 7f3597fff640 1 -- 172.21.15.152:0/408181923 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b0016040 con 0x55f1af7b7320 2024-09-11T22:58:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.645+0000 7f3597fff640 1 -- 172.21.15.152:0/408181923 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f35ac0d7050 con 0x55f1af7b7320 2024-09-11T22:58:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.645+0000 7f35c159c740 10 client.17142 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:58:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.645+0000 7f35c159c740 10 client.17142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:58:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.645+0000 7f35c159c740 20 client.17142 populate_metadata read hostname 'smithi152' 2024-09-11T22:58:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.645+0000 7f35c159c740 1 --2- 172.21.15.152:0/408181923 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f1af7d4280 0x55f1af841610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:58:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.645+0000 7f35c159c740 4 client.17142 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:58:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.645+0000 7f35c159c740 1 -- 172.21.15.152:0/408181923 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f1af724d60 con 0x55f1af7d4280 2024-09-11T22:58:57.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.645+0000 7f35b7fff640 1 --2- 172.21.15.152:0/408181923 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f1af7d4280 0x55f1af841610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:58:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f3597fff640 10 client.17142 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f35b7fff640 1 --2- 172.21.15.152:0/408181923 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f1af7d4280 0x55f1af841610 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc84ea8bc274e328 server_cookie=d3225ee79e851a1 in_seq=0 out_seq=0 2024-09-11T22:58:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f3597fff640 1 -- 172.21.15.152:0/408181923 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f1af724d60 con 0x55f1af7d4280 2024-09-11T22:58:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f3597fff640 10 client.17142 handle_command_reply: tid=0 2024-09-11T22:58:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f35c159c740 1 client.17142 shutdown 2024-09-11T22:58:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f35c159c740 1 -- 172.21.15.152:0/408181923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7b7320 msgr2=0x55f1af7cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f35c159c740 1 --2- 172.21.15.152:0/408181923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7b7320 0x55f1af7cd170 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0b8a70 tx=0x7f35ac0f1040 comp rx=0 tx=0).stop 2024-09-11T22:58:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f35c159c740 1 -- 172.21.15.152:0/408181923 shutdown_connections 2024-09-11T22:58:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f35c159c740 1 --2- 172.21.15.152:0/408181923 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f1af7d4280 0x55f1af841610 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f35c159c740 1 --2- 172.21.15.152:0/408181923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f1af7c4890 0x55f1af7c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f3597fff640 0 client.17142 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:58:57.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f35c159c740 1 --2- 172.21.15.152:0/408181923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f1af7b7c40 0x55f1af7c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f35c159c740 1 --2- 172.21.15.152:0/408181923 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f1af7b7320 0x55f1af7cd170 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f35c159c740 1 -- 172.21.15.152:0/408181923 >> 172.21.15.152:0/408181923 conn(0x55f1af7b3a30 msgr2=0x55f1af7bfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:57.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f35c159c740 1 -- 172.21.15.152:0/408181923 shutdown_connections 2024-09-11T22:58:57.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f35c159c740 1 -- 172.21.15.152:0/408181923 wait complete. 2024-09-11T22:58:57.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.646+0000 7f35c159c740 20 client.17142 trim_cache size 0 max 16384 2024-09-11T22:58:57.657 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:58:57.657 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:58:57.657 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": { 2024-09-11T22:58:57.657 INFO:teuthology.orchestra.run.smithi152.stdout: "406a6121-75cd-408d-8cc4-ed1a80d12721": { 2024-09-11T22:58:57.658 INFO:teuthology.orchestra.run.smithi152.stdout: "path": "/", 2024-09-11T22:58:57.658 INFO:teuthology.orchestra.run.smithi152.stdout: "tag": "406a6121-75cd-408d-8cc4-ed1a80d12721", 2024-09-11T22:58:57.658 INFO:teuthology.orchestra.run.smithi152.stdout: "options": "recursive,force" 2024-09-11T22:58:57.658 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:58:57.658 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T22:58:57.658 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:58:57.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.647+0000 7f35bf1c3640 1 -- 172.21.15.152:0/690439462 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f35a80a2cf0 msgr2=0x7f35a80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:57.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.647+0000 7f35bf1c3640 1 --2- 172.21.15.152:0/690439462 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f35a80a2cf0 0x7f35a80a51b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f35b001e000 tx=0x7f35b0018000 comp rx=0 tx=0).stop 2024-09-11T22:58:57.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.647+0000 7f35bf1c3640 1 -- 172.21.15.152:0/690439462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b8100a80 msgr2=0x7f35b8197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:58:57.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.647+0000 7f35bf1c3640 1 --2- 172.21.15.152:0/690439462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b8100a80 0x7f35b8197340 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009230 tx=0x7f35ac004270 comp rx=0 tx=0).stop 2024-09-11T22:58:57.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.647+0000 7f35bf1c3640 1 -- 172.21.15.152:0/690439462 shutdown_connections 2024-09-11T22:58:57.659 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.647+0000 7f35bf1c3640 1 --2- 172.21.15.152:0/690439462 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f35a80a2cf0 0x7f35a80a51b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.647+0000 7f35bf1c3640 1 --2- 172.21.15.152:0/690439462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35b8106a80 0x7f35b819bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.647+0000 7f35bf1c3640 1 --2- 172.21.15.152:0/690439462 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f35b8101420 0x7f35b8197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.647+0000 7f35bf1c3640 1 --2- 172.21.15.152:0/690439462 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f35b8100a80 0x7f35b8197340 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:58:57.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.647+0000 7f35bf1c3640 1 -- 172.21.15.152:0/690439462 >> 172.21.15.152:0/690439462 conn(0x7f35b80fc7a0 msgr2=0x7f35b80fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:58:57.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.647+0000 7f35bf1c3640 1 -- 172.21.15.152:0/690439462 shutdown_connections 2024-09-11T22:58:57.660 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:58:57.647+0000 7f35bf1c3640 1 -- 172.21.15.152:0/690439462 wait complete. 2024-09-11T22:58:58.363 INFO:tasks.cephfs.filesystem:scrub status for tag:406a6121-75cd-408d-8cc4-ed1a80d12721 - {'path': '/', 'tag': '406a6121-75cd-408d-8cc4-ed1a80d12721', 'options': 'recursive,force'} 2024-09-11T22:58:58.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:58 smithi191 ceph-mon[35211]: pgmap v185: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:58.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:58:58 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T22:58:58.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:58 smithi204 ceph-mon[35454]: pgmap v185: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:58.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:58:58 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T22:58:58.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:58 smithi152 ceph-mon[31181]: pgmap v185: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:58:58.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:58:58 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T22:59:00.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:00 smithi191 ceph-mon[35211]: pgmap v186: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-09-11T22:59:00.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:00 smithi204 ceph-mon[35454]: pgmap v186: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-09-11T22:59:00.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:00 smithi152 ceph-mon[31181]: pgmap v186: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 12 op/s 2024-09-11T22:59:02.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:02 smithi191 ceph-mon[35211]: pgmap v187: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-11T22:59:02.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:02 smithi204 ceph-mon[35454]: pgmap v187: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-11T22:59:02.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:02 smithi152 ceph-mon[31181]: pgmap v187: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s 2024-09-11T22:59:04.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:04 smithi191 ceph-mon[35211]: pgmap v188: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:59:04.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:04 smithi204 ceph-mon[35454]: pgmap v188: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:59:04.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:04 smithi152 ceph-mon[31181]: pgmap v188: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:59:06.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:06 smithi204 ceph-mon[35454]: pgmap v189: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-11T22:59:06.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:06 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:59:06.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:06 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:59:06.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:06 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:59:06.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:06 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:59:06.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:06 smithi152 ceph-mon[31181]: pgmap v189: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-11T22:59:06.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:06 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:59:06.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:06 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:59:06.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:06 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:59:06.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:06 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:59:06.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:06 smithi191 ceph-mon[35211]: pgmap v189: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-11T22:59:06.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:06 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:59:06.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:06 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:59:06.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:06 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:59:06.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:06 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T22:59:08.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:08 smithi152 ceph-mon[31181]: pgmap v190: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-11T22:59:08.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:08 smithi191 ceph-mon[35211]: pgmap v190: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-11T22:59:08.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:08 smithi204 ceph-mon[35454]: pgmap v190: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-11T22:59:10.590 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:10 smithi191 ceph-mon[35211]: pgmap v191: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-11T22:59:10.597 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:10 smithi204 ceph-mon[35454]: pgmap v191: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-11T22:59:10.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:10 smithi152 ceph-mon[31181]: pgmap v191: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-09-11T22:59:12.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:12 smithi152 ceph-mon[31181]: pgmap v192: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-11T22:59:12.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:12 smithi191 ceph-mon[35211]: pgmap v192: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-11T22:59:12.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:12 smithi204 ceph-mon[35454]: pgmap v192: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-11T22:59:14.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:14 smithi152 ceph-mon[31181]: pgmap v193: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-11T22:59:14.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:14 smithi191 ceph-mon[35211]: pgmap v193: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-11T22:59:14.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:14 smithi204 ceph-mon[35454]: pgmap v193: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-11T22:59:16.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:16 smithi152 ceph-mon[31181]: pgmap v194: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-11T22:59:16.789 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:16 smithi191 ceph-mon[35211]: pgmap v194: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-11T22:59:16.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:16 smithi204 ceph-mon[35454]: pgmap v194: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-09-11T22:59:18.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:18 smithi152 ceph-mon[31181]: pgmap v195: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T22:59:18.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:18 smithi191 ceph-mon[35211]: pgmap v195: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T22:59:18.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:18 smithi204 ceph-mon[35454]: pgmap v195: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T22:59:20.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:20 smithi152 ceph-mon[31181]: pgmap v196: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-11T22:59:20.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:20 smithi191 ceph-mon[35211]: pgmap v196: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-11T22:59:20.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:20 smithi204 ceph-mon[35454]: pgmap v196: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-11T22:59:22.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:22 smithi152 ceph-mon[31181]: pgmap v197: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T22:59:22.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:22 smithi191 ceph-mon[35211]: pgmap v197: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T22:59:22.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:22 smithi204 ceph-mon[35454]: pgmap v197: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T22:59:24.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:24 smithi152 ceph-mon[31181]: pgmap v198: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:24.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:24 smithi191 ceph-mon[35211]: pgmap v198: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:24.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:24 smithi204 ceph-mon[35454]: pgmap v198: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:26.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:26 smithi152 ceph-mon[31181]: pgmap v199: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:59:26.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:26 smithi191 ceph-mon[35211]: pgmap v199: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:59:26.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:26 smithi204 ceph-mon[35454]: pgmap v199: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:59:28.365 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:59:28.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:28 smithi152 ceph-mon[31181]: pgmap v200: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:28.628 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:28.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:28 smithi191 ceph-mon[35211]: pgmap v200: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:28.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:28 smithi204 ceph-mon[35454]: pgmap v200: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:29.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.198+0000 7f81635e9640 1 -- 172.21.15.152:0/1537799355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f815c102790 msgr2=0x7f815c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:29.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.198+0000 7f81635e9640 1 --2- 172.21.15.152:0/1537799355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f815c102790 0x7f815c1067e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f814c009900 tx=0x7f814c02a410 comp rx=0 tx=0).stop 2024-09-11T22:59:29.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.198+0000 7f81635e9640 1 -- 172.21.15.152:0/1537799355 shutdown_connections 2024-09-11T22:59:29.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.198+0000 7f81635e9640 1 --2- 172.21.15.152:0/1537799355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f815c106f10 0x7f815c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.198+0000 7f81635e9640 1 --2- 172.21.15.152:0/1537799355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f815c102790 0x7f815c1067e0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.198+0000 7f81635e9640 1 --2- 172.21.15.152:0/1537799355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f815c101de0 0x7f815c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.198+0000 7f81635e9640 1 -- 172.21.15.152:0/1537799355 >> 172.21.15.152:0/1537799355 conn(0x7f815c0fd8c0 msgr2=0x7f815c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:29.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.199+0000 7f81635e9640 1 -- 172.21.15.152:0/1537799355 shutdown_connections 2024-09-11T22:59:29.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.199+0000 7f81635e9640 1 -- 172.21.15.152:0/1537799355 wait complete. 2024-09-11T22:59:29.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.199+0000 7f81635e9640 1 Processor -- start 2024-09-11T22:59:29.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.199+0000 7f81635e9640 1 -- start start 2024-09-11T22:59:29.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.199+0000 7f81635e9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f815c101de0 0x7f815c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.199+0000 7f81635e9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f815c102790 0x7f815c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.199+0000 7f81635e9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f815c106f10 0x7f815c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.199+0000 7f81635e9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f815c111f70 con 0x7f815c101de0 2024-09-11T22:59:29.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.199+0000 7f81635e9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f815c111df0 con 0x7f815c102790 2024-09-11T22:59:29.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.199+0000 7f81635e9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1120f0 con 0x7f815c106f10 2024-09-11T22:59:29.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.199+0000 7f8161385640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f815c101de0 0x7f815c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.199+0000 7f8161385640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f815c101de0 0x7f815c198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46290/0 (socket says 172.21.15.152:46290) 2024-09-11T22:59:29.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.200+0000 7f8161385640 1 -- 172.21.15.152:0/3424975802 learned_addr learned my addr 172.21.15.152:0/3424975802 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:29.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.200+0000 7f8161b86640 1 --2- 172.21.15.152:0/3424975802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f815c106f10 0x7f815c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.200+0000 7f8160b84640 1 --2- 172.21.15.152:0/3424975802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f815c102790 0x7f815c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.200+0000 7f8161385640 1 -- 172.21.15.152:0/3424975802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f815c106f10 msgr2=0x7f815c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:29.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.200+0000 7f8161385640 1 --2- 172.21.15.152:0/3424975802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f815c106f10 0x7f815c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.200+0000 7f8161385640 1 -- 172.21.15.152:0/3424975802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f815c102790 msgr2=0x7f815c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:29.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.200+0000 7f8161385640 1 --2- 172.21.15.152:0/3424975802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f815c102790 0x7f815c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.200+0000 7f8161385640 1 -- 172.21.15.152:0/3424975802 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c19d440 con 0x7f815c101de0 2024-09-11T22:59:29.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.200+0000 7f8160b84640 1 --2- 172.21.15.152:0/3424975802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f815c102790 0x7f815c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:59:29.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.200+0000 7f8161385640 1 --2- 172.21.15.152:0/3424975802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f815c101de0 0x7f815c198490 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f815800d950 tx=0x7f815800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:29.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.200+0000 7f81527fc640 1 -- 172.21.15.152:0/3424975802 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158017070 con 0x7f815c101de0 2024-09-11T22:59:29.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.200+0000 7f81527fc640 1 -- 172.21.15.152:0/3424975802 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8158009c30 con 0x7f815c101de0 2024-09-11T22:59:29.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.200+0000 7f81635e9640 1 -- 172.21.15.152:0/3424975802 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815c19d730 con 0x7f815c101de0 2024-09-11T22:59:29.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.200+0000 7f81527fc640 1 -- 172.21.15.152:0/3424975802 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815800f6b0 con 0x7f815c101de0 2024-09-11T22:59:29.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.200+0000 7f81635e9640 1 -- 172.21.15.152:0/3424975802 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f815c1a5010 con 0x7f815c101de0 2024-09-11T22:59:29.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.202+0000 7f81527fc640 1 -- 172.21.15.152:0/3424975802 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f81580109d0 con 0x7f815c101de0 2024-09-11T22:59:29.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.203+0000 7f81659c2740 1 Processor -- start 2024-09-11T22:59:29.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.203+0000 7f81659c2740 1 -- start start 2024-09-11T22:59:29.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.203+0000 7f81659c2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8607753a0 0x55b860775780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.203+0000 7f81659c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b860775cc0 0x55b860784550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.203+0000 7f81659c2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860784a90 0x55b860786e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.203+0000 7f81527fc640 1 --2- 172.21.15.152:0/3424975802 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f81480a2d70 0x7f81480a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.203+0000 7f81659c2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55b8606db2e0 con 0x55b860784a90 2024-09-11T22:59:29.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.203+0000 7f81659c2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b860682ce0 con 0x55b8607753a0 2024-09-11T22:59:29.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.203+0000 7f81659c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b8606d72a0 con 0x55b860775cc0 2024-09-11T22:59:29.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.203+0000 7f8161b86640 1 --2- 172.21.15.152:0/3424975802 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f81480a2d70 0x7f81480a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.203+0000 7f8160b84640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860784a90 0x55b860786e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.203+0000 7f8160b84640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860784a90 0x55b860786e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46296/0 (socket says 172.21.15.152:46296) 2024-09-11T22:59:29.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.203+0000 7f8160b84640 1 -- 172.21.15.152:0/252480780 learned_addr learned my addr 172.21.15.152:0/252480780 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:29.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.203+0000 7f81527fc640 1 -- 172.21.15.152:0/3424975802 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f81580ceff0 con 0x7f815c101de0 2024-09-11T22:59:29.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.203+0000 7f8161385640 1 --2- 172.21.15.152:0/252480780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b860775cc0 0x55b860784550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.204+0000 7f8161b86640 1 --2- 172.21.15.152:0/252480780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8607753a0 0x55b860775780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.204+0000 7f8160b84640 1 -- 172.21.15.152:0/252480780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b860775cc0 msgr2=0x55b860784550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:29.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.204+0000 7f8160b84640 1 --2- 172.21.15.152:0/252480780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b860775cc0 0x55b860784550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.204+0000 7f8160b84640 1 -- 172.21.15.152:0/252480780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8607753a0 msgr2=0x55b860775780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:29.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.204+0000 7f8160b84640 1 --2- 172.21.15.152:0/252480780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8607753a0 0x55b860775780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.204+0000 7f8160b84640 1 -- 172.21.15.152:0/252480780 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b860667440 con 0x55b860784a90 2024-09-11T22:59:29.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.204+0000 7f8161b86640 1 --2- 172.21.15.152:0/3424975802 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f81480a2d70 0x7f81480a5230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8154005cc0 tx=0x7f8154013ac0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:29.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.204+0000 7f8160b84640 1 --2- 172.21.15.152:0/252480780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860784a90 0x55b860786e80 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f814c005c40 tx=0x7f814c032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1feeb50061a38d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:29.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.204+0000 7f81477fe640 1 -- 172.21.15.152:0/252480780 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c03f070 con 0x55b860784a90 2024-09-11T22:59:29.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.204+0000 7f81477fe640 1 -- 172.21.15.152:0/252480780 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f814c032e20 con 0x55b860784a90 2024-09-11T22:59:29.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.204+0000 7f81477fe640 1 -- 172.21.15.152:0/252480780 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c035070 con 0x55b860784a90 2024-09-11T22:59:29.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.205+0000 7f81659c2740 1 -- 172.21.15.152:0/252480780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860784a90 msgr2=0x55b860786e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:29.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.205+0000 7f81659c2740 1 --2- 172.21.15.152:0/252480780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860784a90 0x55b860786e80 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f814c005c40 tx=0x7f814c032cb0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.205+0000 7f81659c2740 1 -- 172.21.15.152:0/252480780 shutdown_connections 2024-09-11T22:59:29.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.205+0000 7f81659c2740 1 --2- 172.21.15.152:0/252480780 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860784a90 0x55b860786e80 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.205+0000 7f81659c2740 1 --2- 172.21.15.152:0/252480780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b860775cc0 0x55b860784550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.205+0000 7f81659c2740 1 --2- 172.21.15.152:0/252480780 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8607753a0 0x55b860775780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.205+0000 7f81659c2740 1 -- 172.21.15.152:0/252480780 >> 172.21.15.152:0/252480780 conn(0x55b860771a30 msgr2=0x55b860771e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:29.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.205+0000 7f81659c2740 1 -- 172.21.15.152:0/252480780 shutdown_connections 2024-09-11T22:59:29.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.205+0000 7f81659c2740 1 -- 172.21.15.152:0/252480780 wait complete. 2024-09-11T22:59:29.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.205+0000 7f81659c2740 1 -- start start 2024-09-11T22:59:29.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.205+0000 7f81659c2740 1 Processor -- start 2024-09-11T22:59:29.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f81659c2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8607753a0 0x55b860784410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f81659c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b860775cc0 0x55b8607816c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f81659c2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860784a90 0x55b860781c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f81659c2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55b860787660 con 0x55b860784a90 2024-09-11T22:59:29.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f81659c2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b860696050 con 0x55b8607753a0 2024-09-11T22:59:29.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f81659c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b8607877e0 con 0x55b860775cc0 2024-09-11T22:59:29.264 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:59:29.264 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T22:59:29.264 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T22:59:29.264 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:59:29.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f81659c2740 10 client.0 _ll_register_callbacks cb 0x55b8606d6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:29.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f8161b86640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860784a90 0x55b860781c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f8161b86640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860784a90 0x55b860781c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46304/0 (socket says 172.21.15.152:46304) 2024-09-11T22:59:29.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f8161b86640 1 -- 172.21.15.152:0/1259115565 learned_addr learned my addr 172.21.15.152:0/1259115565 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:29.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f8160b84640 1 --2- 172.21.15.152:0/1259115565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b860775cc0 0x55b8607816c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f8161385640 1 --2- 172.21.15.152:0/1259115565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8607753a0 0x55b860784410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f8161b86640 1 -- 172.21.15.152:0/1259115565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b860775cc0 msgr2=0x55b8607816c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:29.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f8161b86640 1 --2- 172.21.15.152:0/1259115565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b860775cc0 0x55b8607816c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f8161b86640 1 -- 172.21.15.152:0/1259115565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8607753a0 msgr2=0x55b860784410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:29.265 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f8161b86640 1 --2- 172.21.15.152:0/1259115565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8607753a0 0x55b860784410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f8161b86640 1 -- 172.21.15.152:0/1259115565 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b86044c3c0 con 0x55b860784a90 2024-09-11T22:59:29.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f8161b86640 1 --2- 172.21.15.152:0/1259115565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860784a90 0x55b860781c40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f8154011ce0 tx=0x7f815400a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cbae935536025dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:29.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.206+0000 7f8144ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:29.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.207+0000 7f8144ff9640 1 -- 172.21.15.152:0/1259115565 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154023070 con 0x55b860784a90 2024-09-11T22:59:29.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.207+0000 7f8144ff9640 1 -- 172.21.15.152:0/1259115565 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f8154003970 con 0x55b860784a90 2024-09-11T22:59:29.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.207+0000 7f81659c2740 1 -- 172.21.15.152:0/1259115565 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f814c032e20 con 0x55b860784a90 2024-09-11T22:59:29.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.207+0000 7f8144ff9640 1 -- 172.21.15.152:0/1259115565 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f815402e050 con 0x55b860784a90 2024-09-11T22:59:29.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.207+0000 7f81659c2740 10 client.17154 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:29.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.207+0000 7f81659c2740 1 -- 172.21.15.152:0/1259115565 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b8607930e0 con 0x55b860784a90 2024-09-11T22:59:29.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.207+0000 7f8144ff9640 1 -- 172.21.15.152:0/1259115565 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f814c032e20 con 0x55b860784a90 2024-09-11T22:59:29.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.207+0000 7f8144ff9640 1 -- 172.21.15.152:0/1259115565 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f815400f4f0 con 0x55b860784a90 2024-09-11T22:59:29.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.207+0000 7f81659c2740 10 client.17154 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:29.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.207+0000 7f81659c2740 10 client.17154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:59:29.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.207+0000 7f81659c2740 20 client.17154 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:29.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.207+0000 7f81659c2740 1 --2- 172.21.15.152:0/1259115565 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b8607944f0 0x55b860796990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.207+0000 7f81659c2740 4 client.17154 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:59:29.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.207+0000 7f81659c2740 1 -- 172.21.15.152:0/1259115565 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8606e8850 con 0x55b8607944f0 2024-09-11T22:59:29.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.208+0000 7f8161385640 1 --2- 172.21.15.152:0/1259115565 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b8607944f0 0x55b860796990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.208+0000 7f8144ff9640 10 client.17154 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:29.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.208+0000 7f8161385640 1 --2- 172.21.15.152:0/1259115565 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b8607944f0 0x55b860796990 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85a40fae95752891 server_cookie=e8d04786e0d282f8 in_seq=0 out_seq=0 2024-09-11T22:59:29.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.209+0000 7f8144ff9640 1 -- 172.21.15.152:0/1259115565 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8606e8850 con 0x55b8607944f0 2024-09-11T22:59:29.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.209+0000 7f8144ff9640 10 client.17154 handle_command_reply: tid=0 2024-09-11T22:59:29.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.209+0000 7f81659c2740 1 client.17154 shutdown 2024-09-11T22:59:29.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.209+0000 7f81659c2740 1 -- 172.21.15.152:0/1259115565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860784a90 msgr2=0x55b860781c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:29.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.209+0000 7f81659c2740 1 --2- 172.21.15.152:0/1259115565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860784a90 0x55b860781c40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f8154011ce0 tx=0x7f815400a4d0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.209+0000 7f81659c2740 1 -- 172.21.15.152:0/1259115565 shutdown_connections 2024-09-11T22:59:29.267 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.209+0000 7f81659c2740 1 --2- 172.21.15.152:0/1259115565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860784a90 0x55b860781c40 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.209+0000 7f81659c2740 1 --2- 172.21.15.152:0/1259115565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b860775cc0 0x55b8607816c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.209+0000 7f81659c2740 1 --2- 172.21.15.152:0/1259115565 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b8607944f0 0x55b860796990 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.209+0000 7f81659c2740 1 --2- 172.21.15.152:0/1259115565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b8607753a0 0x55b860784410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.209+0000 7f81659c2740 1 -- 172.21.15.152:0/1259115565 >> 172.21.15.152:0/1259115565 conn(0x55b860771a30 msgr2=0x55b860777790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:29.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.209+0000 7f8144ff9640 0 client.17154 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:29.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.209+0000 7f81659c2740 1 -- 172.21.15.152:0/1259115565 shutdown_connections 2024-09-11T22:59:29.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.209+0000 7f81659c2740 1 -- 172.21.15.152:0/1259115565 wait complete. 2024-09-11T22:59:29.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.209+0000 7f81659c2740 20 client.17154 trim_cache size 0 max 16384 2024-09-11T22:59:29.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.226+0000 7f81659c2740 1 Processor -- start 2024-09-11T22:59:29.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.226+0000 7f81659c2740 1 -- start start 2024-09-11T22:59:29.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.226+0000 7f81659c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8607753a0 0x55b860790900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.226+0000 7f81659c2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860775cc0 0x55b86079c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.226+0000 7f81659c2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b860784a90 0x55b86079c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.226+0000 7f81659c2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1ab5a0 con 0x55b860775cc0 2024-09-11T22:59:29.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.226+0000 7f81659c2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1ad0e0 con 0x55b860784a90 2024-09-11T22:59:29.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.226+0000 7f81659c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1abc70 con 0x55b8607753a0 2024-09-11T22:59:29.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.226+0000 7f8160b84640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860775cc0 0x55b86079c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.227+0000 7f8160b84640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860775cc0 0x55b86079c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46310/0 (socket says 172.21.15.152:46310) 2024-09-11T22:59:29.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.227+0000 7f8160b84640 1 -- 172.21.15.152:0/1800721882 learned_addr learned my addr 172.21.15.152:0/1800721882 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:29.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.227+0000 7f8161385640 1 --2- 172.21.15.152:0/1800721882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8607753a0 0x55b860790900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.227+0000 7f8161b86640 1 --2- 172.21.15.152:0/1800721882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b860784a90 0x55b86079c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.227+0000 7f8160b84640 1 -- 172.21.15.152:0/1800721882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8607753a0 msgr2=0x55b860790900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:29.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.227+0000 7f8160b84640 1 --2- 172.21.15.152:0/1800721882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8607753a0 0x55b860790900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.227+0000 7f8160b84640 1 -- 172.21.15.152:0/1800721882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b860784a90 msgr2=0x55b86079c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:29.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.227+0000 7f8160b84640 1 --2- 172.21.15.152:0/1800721882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b860784a90 0x55b86079c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.227+0000 7f8160b84640 1 -- 172.21.15.152:0/1800721882 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b860780670 con 0x55b860775cc0 2024-09-11T22:59:29.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.227+0000 7f8160b84640 1 --2- 172.21.15.152:0/1800721882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860775cc0 0x55b86079c2b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x55b860776f40 tx=0x7f814c0052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d427411169af1a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:29.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.227+0000 7f8146ffd640 1 -- 172.21.15.152:0/1800721882 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c03f040 con 0x55b860775cc0 2024-09-11T22:59:29.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.227+0000 7f8146ffd640 1 -- 172.21.15.152:0/1800721882 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f814c005510 con 0x55b860775cc0 2024-09-11T22:59:29.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.227+0000 7f8146ffd640 1 -- 172.21.15.152:0/1800721882 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c035040 con 0x55b860775cc0 2024-09-11T22:59:29.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.228+0000 7f81659c2740 1 -- 172.21.15.152:0/1800721882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860775cc0 msgr2=0x55b86079c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:29.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.228+0000 7f81659c2740 1 --2- 172.21.15.152:0/1800721882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860775cc0 0x55b86079c2b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x55b860776f40 tx=0x7f814c0052a0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.228+0000 7f81659c2740 1 -- 172.21.15.152:0/1800721882 shutdown_connections 2024-09-11T22:59:29.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.228+0000 7f81659c2740 1 --2- 172.21.15.152:0/1800721882 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b860784a90 0x55b86079c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.228+0000 7f81659c2740 1 --2- 172.21.15.152:0/1800721882 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860775cc0 0x55b86079c2b0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.228+0000 7f81659c2740 1 --2- 172.21.15.152:0/1800721882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8607753a0 0x55b860790900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.228+0000 7f81659c2740 1 -- 172.21.15.152:0/1800721882 >> 172.21.15.152:0/1800721882 conn(0x55b860771a30 msgr2=0x55b86077c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:29.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.228+0000 7f81659c2740 1 -- 172.21.15.152:0/1800721882 shutdown_connections 2024-09-11T22:59:29.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.228+0000 7f81659c2740 1 -- 172.21.15.152:0/1800721882 wait complete. 2024-09-11T22:59:29.270 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.228+0000 7f81659c2740 1 -- start start 2024-09-11T22:59:29.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.228+0000 7f81659c2740 1 Processor -- start 2024-09-11T22:59:29.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f81659c2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8607753a0 0x55b860788aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f81659c2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860775cc0 0x55b860788fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f81659c2740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b860784a90 0x55b860789520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f81659c2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1aca10 con 0x55b860775cc0 2024-09-11T22:59:29.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f81659c2740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1ac340 con 0x55b860784a90 2024-09-11T22:59:29.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f81659c2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1a9560 con 0x55b8607753a0 2024-09-11T22:59:29.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f81659c2740 10 client.0 _ll_register_callbacks cb 0x7f815400a900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:29.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f8160b84640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860775cc0 0x55b860788fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f8160b84640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860775cc0 0x55b860788fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46322/0 (socket says 172.21.15.152:46322) 2024-09-11T22:59:29.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f8160b84640 1 -- 172.21.15.152:0/2910372272 learned_addr learned my addr 172.21.15.152:0/2910372272 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:29.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f8161385640 1 --2- 172.21.15.152:0/2910372272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8607753a0 0x55b860788aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f8161b86640 1 --2- 172.21.15.152:0/2910372272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b860784a90 0x55b860789520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f8160b84640 1 -- 172.21.15.152:0/2910372272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8607753a0 msgr2=0x55b860788aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:29.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f8160b84640 1 --2- 172.21.15.152:0/2910372272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8607753a0 0x55b860788aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f8160b84640 1 -- 172.21.15.152:0/2910372272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b860784a90 msgr2=0x55b860789520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:29.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f8160b84640 1 --2- 172.21.15.152:0/2910372272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b860784a90 0x55b860789520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f8160b84640 1 -- 172.21.15.152:0/2910372272 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b860784780 con 0x55b860775cc0 2024-09-11T22:59:29.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f8160b84640 1 --2- 172.21.15.152:0/2910372272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860775cc0 0x55b860788fe0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f814c033000 tx=0x7f814c032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d16f7ea49821fc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:29.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.229+0000 7f8144ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:29.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.230+0000 7f8144ff9640 1 -- 172.21.15.152:0/2910372272 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c035050 con 0x55b860775cc0 2024-09-11T22:59:29.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.230+0000 7f8144ff9640 1 -- 172.21.15.152:0/2910372272 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f814c03b460 con 0x55b860775cc0 2024-09-11T22:59:29.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.230+0000 7f81659c2740 1 -- 172.21.15.152:0/2910372272 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f814c005510 con 0x55b860775cc0 2024-09-11T22:59:29.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.230+0000 7f8144ff9640 1 -- 172.21.15.152:0/2910372272 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f814c05f050 con 0x55b860775cc0 2024-09-11T22:59:29.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.230+0000 7f81659c2740 10 client.17160 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:29.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.230+0000 7f81659c2740 1 -- 172.21.15.152:0/2910372272 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b8607ff630 con 0x55b860775cc0 2024-09-11T22:59:29.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.230+0000 7f8144ff9640 1 -- 172.21.15.152:0/2910372272 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f814c005510 con 0x55b860775cc0 2024-09-11T22:59:29.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.230+0000 7f8144ff9640 1 -- 172.21.15.152:0/2910372272 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f814c044720 con 0x55b860775cc0 2024-09-11T22:59:29.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.230+0000 7f81659c2740 10 client.17160 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:29.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.230+0000 7f81659c2740 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:59:29.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.230+0000 7f81659c2740 20 client.17160 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:29.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.230+0000 7f81659c2740 1 --2- 172.21.15.152:0/2910372272 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b8607ff8b0 0x55b860801ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:29.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.230+0000 7f81659c2740 4 client.17160 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:59:29.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.230+0000 7f81659c2740 1 -- 172.21.15.152:0/2910372272 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b8606e2d60 con 0x55b8607ff8b0 2024-09-11T22:59:29.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f8161385640 1 --2- 172.21.15.152:0/2910372272 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b8607ff8b0 0x55b860801ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:29.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f8144ff9640 10 client.17160 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:29.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f8161385640 1 --2- 172.21.15.152:0/2910372272 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b8607ff8b0 0x55b860801ca0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21476735a874db66 server_cookie=4df8d5e95f65a893 in_seq=0 out_seq=0 2024-09-11T22:59:29.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f8144ff9640 1 -- 172.21.15.152:0/2910372272 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b8606e2d60 con 0x55b8607ff8b0 2024-09-11T22:59:29.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f8144ff9640 10 client.17160 handle_command_reply: tid=0 2024-09-11T22:59:29.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f81659c2740 1 client.17160 shutdown 2024-09-11T22:59:29.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f81659c2740 1 -- 172.21.15.152:0/2910372272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860775cc0 msgr2=0x55b860788fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:29.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f81659c2740 1 --2- 172.21.15.152:0/2910372272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860775cc0 0x55b860788fe0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f814c033000 tx=0x7f814c032cb0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f81659c2740 1 -- 172.21.15.152:0/2910372272 shutdown_connections 2024-09-11T22:59:29.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f81659c2740 1 --2- 172.21.15.152:0/2910372272 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b8607ff8b0 0x55b860801ca0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f81659c2740 1 --2- 172.21.15.152:0/2910372272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b860784a90 0x55b860789520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f8144ff9640 0 client.17160 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:29.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f81659c2740 1 --2- 172.21.15.152:0/2910372272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b860775cc0 0x55b860788fe0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f81659c2740 1 --2- 172.21.15.152:0/2910372272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b8607753a0 0x55b860788aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f81659c2740 1 -- 172.21.15.152:0/2910372272 >> 172.21.15.152:0/2910372272 conn(0x55b860771a30 msgr2=0x55b860776f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:29.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f81659c2740 1 -- 172.21.15.152:0/2910372272 shutdown_connections 2024-09-11T22:59:29.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.231+0000 7f81659c2740 1 -- 172.21.15.152:0/2910372272 wait complete. 2024-09-11T22:59:29.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.232+0000 7f81659c2740 20 client.17160 trim_cache size 0 max 16384 2024-09-11T22:59:29.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.232+0000 7f81635e9640 1 -- 172.21.15.152:0/3424975802 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f81480a2d70 msgr2=0x7f81480a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:29.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.232+0000 7f81635e9640 1 --2- 172.21.15.152:0/3424975802 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f81480a2d70 0x7f81480a5230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8154005cc0 tx=0x7f8154013ac0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.232+0000 7f81635e9640 1 -- 172.21.15.152:0/3424975802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f815c101de0 msgr2=0x7f815c198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:29.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.232+0000 7f81635e9640 1 --2- 172.21.15.152:0/3424975802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f815c101de0 0x7f815c198490 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f815800d950 tx=0x7f815800de10 comp rx=0 tx=0).stop 2024-09-11T22:59:29.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.232+0000 7f81635e9640 1 -- 172.21.15.152:0/3424975802 shutdown_connections 2024-09-11T22:59:29.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.232+0000 7f81635e9640 1 --2- 172.21.15.152:0/3424975802 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f81480a2d70 0x7f81480a5230 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.232+0000 7f81635e9640 1 --2- 172.21.15.152:0/3424975802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f815c106f10 0x7f815c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.232+0000 7f81635e9640 1 --2- 172.21.15.152:0/3424975802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f815c102790 0x7f815c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.232+0000 7f81635e9640 1 --2- 172.21.15.152:0/3424975802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f815c101de0 0x7f815c198490 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:29.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.232+0000 7f81635e9640 1 -- 172.21.15.152:0/3424975802 >> 172.21.15.152:0/3424975802 conn(0x7f815c0fd8c0 msgr2=0x7f815c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:29.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.232+0000 7f81635e9640 1 -- 172.21.15.152:0/3424975802 shutdown_connections 2024-09-11T22:59:29.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:29.232+0000 7f81635e9640 1 -- 172.21.15.152:0/3424975802 wait complete. 2024-09-11T22:59:30.177 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:59:30.177 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:59:30.435 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:30.509 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:30 smithi152 ceph-mon[31181]: pgmap v201: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-11T22:59:30.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:30 smithi191 ceph-mon[35211]: pgmap v201: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-11T22:59:30.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:30 smithi204 ceph-mon[35454]: pgmap v201: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-11T22:59:30.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.958+0000 7f23dc0ba640 1 -- 172.21.15.152:0/481125817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d4106a80 msgr2=0x7f23d4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:30.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.958+0000 7f23dc0ba640 1 --2- 172.21.15.152:0/481125817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d4106a80 0x7f23d4106e60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009930 tx=0x7f23d002a420 comp rx=0 tx=0).stop 2024-09-11T22:59:30.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.958+0000 7f23dc0ba640 1 -- 172.21.15.152:0/481125817 shutdown_connections 2024-09-11T22:59:30.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.958+0000 7f23dc0ba640 1 --2- 172.21.15.152:0/481125817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23d4101420 0x7f23d410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:30.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.958+0000 7f23dc0ba640 1 --2- 172.21.15.152:0/481125817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23d4100a80 0x7f23d4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:30.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.958+0000 7f23dc0ba640 1 --2- 172.21.15.152:0/481125817 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d4106a80 0x7f23d4106e60 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:30.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.958+0000 7f23dc0ba640 1 -- 172.21.15.152:0/481125817 >> 172.21.15.152:0/481125817 conn(0x7f23d40fc7a0 msgr2=0x7f23d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.958+0000 7f23dc0ba640 1 -- 172.21.15.152:0/481125817 shutdown_connections 2024-09-11T22:59:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.958+0000 7f23dc0ba640 1 -- 172.21.15.152:0/481125817 wait complete. 2024-09-11T22:59:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.959+0000 7f23dc0ba640 1 Processor -- start 2024-09-11T22:59:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.959+0000 7f23dc0ba640 1 -- start start 2024-09-11T22:59:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.959+0000 7f23dc0ba640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23d4100a80 0x7f23d4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.959+0000 7f23dc0ba640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d4101420 0x7f23d4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.959+0000 7f23dc0ba640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23d4106a80 0x7f23d419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.959+0000 7f23dc0ba640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4110e40 con 0x7f23d4101420 2024-09-11T22:59:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.959+0000 7f23dc0ba640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4110cc0 con 0x7f23d4106a80 2024-09-11T22:59:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.959+0000 7f23dc0ba640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f23d4110fc0 con 0x7f23d4100a80 2024-09-11T22:59:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.959+0000 7f23d9655640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d4101420 0x7f23d4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.959+0000 7f23d9655640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d4101420 0x7f23d4198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46342/0 (socket says 172.21.15.152:46342) 2024-09-11T22:59:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.959+0000 7f23d9655640 1 -- 172.21.15.152:0/3085937577 learned_addr learned my addr 172.21.15.152:0/3085937577 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.960+0000 7f23da657640 1 --2- 172.21.15.152:0/3085937577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23d4106a80 0x7f23d419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:30.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.960+0000 7f23d9e56640 1 --2- 172.21.15.152:0/3085937577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23d4100a80 0x7f23d4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:30.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.960+0000 7f23d9655640 1 -- 172.21.15.152:0/3085937577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23d4100a80 msgr2=0x7f23d4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:30.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.960+0000 7f23d9655640 1 --2- 172.21.15.152:0/3085937577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23d4100a80 0x7f23d4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:30.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.960+0000 7f23d9655640 1 -- 172.21.15.152:0/3085937577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23d4106a80 msgr2=0x7f23d419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:30.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.960+0000 7f23d9655640 1 --2- 172.21.15.152:0/3085937577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23d4106a80 0x7f23d419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:30.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.960+0000 7f23d9655640 1 -- 172.21.15.152:0/3085937577 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d419d3e0 con 0x7f23d4101420 2024-09-11T22:59:30.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.960+0000 7f23da657640 1 --2- 172.21.15.152:0/3085937577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23d4106a80 0x7f23d419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:59:30.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.960+0000 7f23d9e56640 1 --2- 172.21.15.152:0/3085937577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23d4100a80 0x7f23d4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:59:30.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.960+0000 7f23d9655640 1 --2- 172.21.15.152:0/3085937577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d4101420 0x7f23d4198970 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f23c400c890 tx=0x7f23c400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:30.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.960+0000 7f23caffd640 1 -- 172.21.15.152:0/3085937577 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4004220 con 0x7f23d4101420 2024-09-11T22:59:30.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.960+0000 7f23dc0ba640 1 -- 172.21.15.152:0/3085937577 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d419d6d0 con 0x7f23d4101420 2024-09-11T22:59:30.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.960+0000 7f23caffd640 1 -- 172.21.15.152:0/3085937577 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23c40043c0 con 0x7f23d4101420 2024-09-11T22:59:30.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.960+0000 7f23caffd640 1 -- 172.21.15.152:0/3085937577 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c40049e0 con 0x7f23d4101420 2024-09-11T22:59:30.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.960+0000 7f23dc0ba640 1 -- 172.21.15.152:0/3085937577 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d41a4fb0 con 0x7f23d4101420 2024-09-11T22:59:30.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.961+0000 7f23dc0ba640 1 -- 172.21.15.152:0/3085937577 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d4100ee0 con 0x7f23d4101420 2024-09-11T22:59:30.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.962+0000 7f23caffd640 1 -- 172.21.15.152:0/3085937577 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f23c40079b0 con 0x7f23d4101420 2024-09-11T22:59:30.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.963+0000 7f23caffd640 1 --2- 172.21.15.152:0/3085937577 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f23c00a2cf0 0x7f23c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:30.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.963+0000 7f23d9e56640 1 --2- 172.21.15.152:0/3085937577 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f23c00a2cf0 0x7f23c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:30.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.963+0000 7f23caffd640 1 -- 172.21.15.152:0/3085937577 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f23c40cf560 con 0x7f23d4101420 2024-09-11T22:59:30.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.963+0000 7f23d9e56640 1 --2- 172.21.15.152:0/3085937577 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f23c00a2cf0 0x7f23c00a51b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f23d0000c00 tx=0x7f23d0032c70 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:30.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:30.966+0000 7f23caffd640 1 -- 172.21.15.152:0/3085937577 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f23c4014030 con 0x7f23d4101420 2024-09-11T22:59:31.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:31.168+0000 7f23dc0ba640 1 -- 172.21.15.152:0/3085937577 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23d4199790 con 0x7f23d4101420 2024-09-11T22:59:31.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:31.169+0000 7f23caffd640 1 -- 172.21.15.152:0/3085937577 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f23d4199790 con 0x7f23d4101420 2024-09-11T22:59:31.170 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:59:31.170 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:59:31.170 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:59:31.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:31.171+0000 7f23dc0ba640 1 -- 172.21.15.152:0/3085937577 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f23c00a2cf0 msgr2=0x7f23c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:31.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:31.171+0000 7f23dc0ba640 1 --2- 172.21.15.152:0/3085937577 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f23c00a2cf0 0x7f23c00a51b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f23d0000c00 tx=0x7f23d0032c70 comp rx=0 tx=0).stop 2024-09-11T22:59:31.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:31.171+0000 7f23dc0ba640 1 -- 172.21.15.152:0/3085937577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d4101420 msgr2=0x7f23d4198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:31.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:31.171+0000 7f23dc0ba640 1 --2- 172.21.15.152:0/3085937577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d4101420 0x7f23d4198970 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f23c400c890 tx=0x7f23c400cd50 comp rx=0 tx=0).stop 2024-09-11T22:59:31.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:31.171+0000 7f23dc0ba640 1 -- 172.21.15.152:0/3085937577 shutdown_connections 2024-09-11T22:59:31.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:31.171+0000 7f23dc0ba640 1 --2- 172.21.15.152:0/3085937577 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f23c00a2cf0 0x7f23c00a51b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:31.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:31.171+0000 7f23dc0ba640 1 --2- 172.21.15.152:0/3085937577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f23d4106a80 0x7f23d419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:31.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:31.171+0000 7f23dc0ba640 1 --2- 172.21.15.152:0/3085937577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23d4101420 0x7f23d4198970 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:31.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:31.171+0000 7f23dc0ba640 1 --2- 172.21.15.152:0/3085937577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23d4100a80 0x7f23d4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:31.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:31.171+0000 7f23dc0ba640 1 -- 172.21.15.152:0/3085937577 >> 172.21.15.152:0/3085937577 conn(0x7f23d40fc7a0 msgr2=0x7f23d4104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:31.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:31.171+0000 7f23dc0ba640 1 -- 172.21.15.152:0/3085937577 shutdown_connections 2024-09-11T22:59:31.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:31.172+0000 7f23dc0ba640 1 -- 172.21.15.152:0/3085937577 wait complete. 2024-09-11T22:59:31.548 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:31 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3085937577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:59:31.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:31 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3085937577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:59:31.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:31 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3085937577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:59:31.998 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:59:32.245 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:32.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:32 smithi152 ceph-mon[31181]: pgmap v202: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T22:59:32.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.746+0000 7f7d35780640 1 -- 172.21.15.152:0/3538565252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d30106f20 msgr2=0x7f7d3010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:32.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.746+0000 7f7d35780640 1 --2- 172.21.15.152:0/3538565252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d30106f20 0x7f7d3010eaa0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009900 tx=0x7f7d2802a410 comp rx=0 tx=0).stop 2024-09-11T22:59:32.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.746+0000 7f7d35780640 1 -- 172.21.15.152:0/3538565252 shutdown_connections 2024-09-11T22:59:32.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.746+0000 7f7d35780640 1 --2- 172.21.15.152:0/3538565252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d30106f20 0x7f7d3010eaa0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.746+0000 7f7d35780640 1 --2- 172.21.15.152:0/3538565252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d301027a0 0x7f7d301067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.746+0000 7f7d35780640 1 --2- 172.21.15.152:0/3538565252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d30101df0 0x7f7d301021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.746+0000 7f7d35780640 1 -- 172.21.15.152:0/3538565252 >> 172.21.15.152:0/3538565252 conn(0x7f7d300fd8b0 msgr2=0x7f7d300ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:32.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.746+0000 7f7d35780640 1 -- 172.21.15.152:0/3538565252 shutdown_connections 2024-09-11T22:59:32.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.746+0000 7f7d35780640 1 -- 172.21.15.152:0/3538565252 wait complete. 2024-09-11T22:59:32.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.746+0000 7f7d35780640 1 Processor -- start 2024-09-11T22:59:32.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.746+0000 7f7d35780640 1 -- start start 2024-09-11T22:59:32.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d35780640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d30101df0 0x7f7d300744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d35780640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d301027a0 0x7f7d30074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d35780640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d30106f20 0x7f7d30074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d35780640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7d30111f80 con 0x7f7d30101df0 2024-09-11T22:59:32.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d35780640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7d30111e00 con 0x7f7d301027a0 2024-09-11T22:59:32.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d35780640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d30112100 con 0x7f7d30106f20 2024-09-11T22:59:32.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d2effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d30101df0 0x7f7d300744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d2effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d30101df0 0x7f7d300744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46354/0 (socket says 172.21.15.152:46354) 2024-09-11T22:59:32.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d2effd640 1 -- 172.21.15.152:0/2697075815 learned_addr learned my addr 172.21.15.152:0/2697075815 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:32.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d2e7fc640 1 --2- 172.21.15.152:0/2697075815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d301027a0 0x7f7d30074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d2f7fe640 1 --2- 172.21.15.152:0/2697075815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d30106f20 0x7f7d30074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d2effd640 1 -- 172.21.15.152:0/2697075815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d30106f20 msgr2=0x7f7d30074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:32.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d2effd640 1 --2- 172.21.15.152:0/2697075815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d30106f20 0x7f7d30074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d2effd640 1 -- 172.21.15.152:0/2697075815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d301027a0 msgr2=0x7f7d30074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:32.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d2effd640 1 --2- 172.21.15.152:0/2697075815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d301027a0 0x7f7d30074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d2effd640 1 -- 172.21.15.152:0/2697075815 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d301a6840 con 0x7f7d30101df0 2024-09-11T22:59:32.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d2effd640 1 --2- 172.21.15.152:0/2697075815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d30101df0 0x7f7d300744f0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000eab0 tx=0x7f7d2000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:32.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.747+0000 7f7d27fff640 1 -- 172.21.15.152:0/2697075815 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2000ce40 con 0x7f7d30101df0 2024-09-11T22:59:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.748+0000 7f7d35780640 1 -- 172.21.15.152:0/2697075815 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d301a6b30 con 0x7f7d30101df0 2024-09-11T22:59:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.748+0000 7f7d27fff640 1 -- 172.21.15.152:0/2697075815 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d20011950 con 0x7f7d30101df0 2024-09-11T22:59:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.748+0000 7f7d27fff640 1 -- 172.21.15.152:0/2697075815 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20010ae0 con 0x7f7d30101df0 2024-09-11T22:59:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.748+0000 7f7d35780640 1 -- 172.21.15.152:0/2697075815 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d301a70c0 con 0x7f7d30101df0 2024-09-11T22:59:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.750+0000 7f7d27fff640 1 -- 172.21.15.152:0/2697075815 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7d20011b00 con 0x7f7d30101df0 2024-09-11T22:59:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.750+0000 7f7d37b59740 1 Processor -- start 2024-09-11T22:59:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.750+0000 7f7d37b59740 1 -- start start 2024-09-11T22:59:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d27fff640 1 --2- 172.21.15.152:0/2697075815 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7d1c0a2cf0 0x7f7d1c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d37b59740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557011264b40 0x557011263090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d37b59740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570112635d0 0x557011263a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d37b59740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701126e370 0x557011270760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d37b59740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5570111c92f0 con 0x5570112635d0 2024-09-11T22:59:32.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d37b59740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557011170cf0 con 0x557011264b40 2024-09-11T22:59:32.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d37b59740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5570111c52b0 con 0x55701126e370 2024-09-11T22:59:32.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d27fff640 1 -- 172.21.15.152:0/2697075815 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7d200cfa20 con 0x7f7d30101df0 2024-09-11T22:59:32.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d2f7fe640 1 --2- 172.21.15.152:0/2697075815 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7d1c0a2cf0 0x7f7d1c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d2e7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570112635d0 0x557011263a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d2e7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570112635d0 0x557011263a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46360/0 (socket says 172.21.15.152:46360) 2024-09-11T22:59:32.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d2e7fc640 1 -- 172.21.15.152:0/3788090270 learned_addr learned my addr 172.21.15.152:0/3788090270 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:32.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d2effd640 1 --2- 172.21.15.152:0/3788090270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557011264b40 0x557011263090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d2f7fe640 1 --2- 172.21.15.152:0/3788090270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701126e370 0x557011270760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d2e7fc640 1 -- 172.21.15.152:0/3788090270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701126e370 msgr2=0x557011270760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:32.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d2e7fc640 1 --2- 172.21.15.152:0/3788090270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701126e370 0x557011270760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d2e7fc640 1 -- 172.21.15.152:0/3788090270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557011264b40 msgr2=0x557011263090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:32.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d2e7fc640 1 --2- 172.21.15.152:0/3788090270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557011264b40 0x557011263090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d2e7fc640 1 -- 172.21.15.152:0/3788090270 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570111553a0 con 0x5570112635d0 2024-09-11T22:59:32.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d2f7fe640 1 --2- 172.21.15.152:0/2697075815 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7d1c0a2cf0 0x7f7d1c0a51b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7d280096c0 tx=0x7f7d280317a0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:32.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.751+0000 7f7d2e7fc640 1 --2- 172.21.15.152:0/3788090270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570112635d0 0x557011263a10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800bc10 tx=0x7f7d180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6bc9a0dd576613 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:32.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.752+0000 7f7d257fa640 1 -- 172.21.15.152:0/3788090270 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18005020 con 0x5570112635d0 2024-09-11T22:59:32.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.752+0000 7f7d257fa640 1 -- 172.21.15.152:0/3788090270 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d180047a0 con 0x5570112635d0 2024-09-11T22:59:32.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.752+0000 7f7d257fa640 1 -- 172.21.15.152:0/3788090270 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d180116a0 con 0x5570112635d0 2024-09-11T22:59:32.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.752+0000 7f7d37b59740 1 -- 172.21.15.152:0/3788090270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570112635d0 msgr2=0x557011263a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:32.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.752+0000 7f7d37b59740 1 --2- 172.21.15.152:0/3788090270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570112635d0 0x557011263a10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800bc10 tx=0x7f7d180107a0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.752+0000 7f7d37b59740 1 -- 172.21.15.152:0/3788090270 shutdown_connections 2024-09-11T22:59:32.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.752+0000 7f7d37b59740 1 --2- 172.21.15.152:0/3788090270 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701126e370 0x557011270760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.752+0000 7f7d37b59740 1 --2- 172.21.15.152:0/3788090270 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570112635d0 0x557011263a10 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.752+0000 7f7d37b59740 1 --2- 172.21.15.152:0/3788090270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557011264b40 0x557011263090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.752+0000 7f7d37b59740 1 -- 172.21.15.152:0/3788090270 >> 172.21.15.152:0/3788090270 conn(0x55701122c310 msgr2=0x55701125fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:32.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.752+0000 7f7d37b59740 1 -- 172.21.15.152:0/3788090270 shutdown_connections 2024-09-11T22:59:32.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.752+0000 7f7d37b59740 1 -- 172.21.15.152:0/3788090270 wait complete. 2024-09-11T22:59:32.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.753+0000 7f7d37b59740 1 -- start start 2024-09-11T22:59:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.753+0000 7f7d37b59740 1 Processor -- start 2024-09-11T22:59:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.753+0000 7f7d37b59740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570112635d0 0x55701126dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.753+0000 7f7d37b59740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557011264b40 0x55701126b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.753+0000 7f7d37b59740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55701126e370 0x55701126b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.753+0000 7f7d37b59740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x557011271070 con 0x55701126e370 2024-09-11T22:59:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.753+0000 7f7d37b59740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x557011184060 con 0x5570112635d0 2024-09-11T22:59:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.753+0000 7f7d37b59740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5570112711f0 con 0x557011264b40 2024-09-11T22:59:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.753+0000 7f7d37b59740 10 client.0 _ll_register_callbacks cb 0x557010ed78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.753+0000 7f7d2f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55701126e370 0x55701126b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.753+0000 7f7d2f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55701126e370 0x55701126b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46374/0 (socket says 172.21.15.152:46374) 2024-09-11T22:59:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.753+0000 7f7d2f7fe640 1 -- 172.21.15.152:0/1884492620 learned_addr learned my addr 172.21.15.152:0/1884492620 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.753+0000 7f7d2effd640 1 --2- 172.21.15.152:0/1884492620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570112635d0 0x55701126dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.753+0000 7f7d2e7fc640 1 --2- 172.21.15.152:0/1884492620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557011264b40 0x55701126b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d2f7fe640 1 -- 172.21.15.152:0/1884492620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557011264b40 msgr2=0x55701126b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d2f7fe640 1 --2- 172.21.15.152:0/1884492620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557011264b40 0x55701126b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d2f7fe640 1 -- 172.21.15.152:0/1884492620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570112635d0 msgr2=0x55701126dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d2f7fe640 1 --2- 172.21.15.152:0/1884492620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570112635d0 0x55701126dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d2f7fe640 1 -- 172.21.15.152:0/1884492620 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557010f3a3c0 con 0x55701126e370 2024-09-11T22:59:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d2f7fe640 1 --2- 172.21.15.152:0/1884492620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55701126e370 0x55701126b610 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009900 tx=0x7f7d2802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9525d8c97f45f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d167fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d167fc640 1 -- 172.21.15.152:0/1884492620 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2804e070 con 0x55701126e370 2024-09-11T22:59:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d167fc640 1 -- 172.21.15.152:0/1884492620 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7d280037f0 con 0x55701126e370 2024-09-11T22:59:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d37b59740 1 -- 172.21.15.152:0/1884492620 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d180047a0 con 0x55701126e370 2024-09-11T22:59:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d167fc640 1 -- 172.21.15.152:0/1884492620 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7d2805a050 con 0x55701126e370 2024-09-11T22:59:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d37b59740 10 client.17178 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d37b59740 1 -- 172.21.15.152:0/1884492620 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55701127cb40 con 0x55701126e370 2024-09-11T22:59:32.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d167fc640 1 -- 172.21.15.152:0/1884492620 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d180047a0 con 0x55701126e370 2024-09-11T22:59:32.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d167fc640 1 -- 172.21.15.152:0/1884492620 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7d2804aa50 con 0x55701126e370 2024-09-11T22:59:32.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d37b59740 10 client.17178 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:32.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.754+0000 7f7d37b59740 10 client.17178 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:59:32.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.755+0000 7f7d37b59740 20 client.17178 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:32.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.755+0000 7f7d37b59740 1 --2- 172.21.15.152:0/1884492620 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55701127df50 0x5570112803d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.755+0000 7f7d37b59740 4 client.17178 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:59:32.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.755+0000 7f7d37b59740 1 -- 172.21.15.152:0/1884492620 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570111d6860 con 0x55701127df50 2024-09-11T22:59:32.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.755+0000 7f7d2effd640 1 --2- 172.21.15.152:0/1884492620 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55701127df50 0x5570112803d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.755+0000 7f7d167fc640 10 client.17178 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:59:32.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.755+0000 7f7d2effd640 1 --2- 172.21.15.152:0/1884492620 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55701127df50 0x5570112803d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6c0f6b6c206f084e server_cookie=5aeaaaa05d56cd in_seq=0 out_seq=0 2024-09-11T22:59:32.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.755+0000 7f7d167fc640 1 -- 172.21.15.152:0/1884492620 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570111d6860 con 0x55701127df50 2024-09-11T22:59:32.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.755+0000 7f7d167fc640 10 client.17178 handle_command_reply: tid=0 2024-09-11T22:59:32.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.756+0000 7f7d37b59740 1 client.17178 shutdown 2024-09-11T22:59:32.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.756+0000 7f7d37b59740 1 -- 172.21.15.152:0/1884492620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55701126e370 msgr2=0x55701126b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.756+0000 7f7d37b59740 1 --2- 172.21.15.152:0/1884492620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55701126e370 0x55701126b610 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009900 tx=0x7f7d2802a410 comp rx=0 tx=0).stop 2024-09-11T22:59:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.756+0000 7f7d37b59740 1 -- 172.21.15.152:0/1884492620 shutdown_connections 2024-09-11T22:59:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.756+0000 7f7d37b59740 1 --2- 172.21.15.152:0/1884492620 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55701126e370 0x55701126b610 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.756+0000 7f7d37b59740 1 --2- 172.21.15.152:0/1884492620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557011264b40 0x55701126b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.756+0000 7f7d37b59740 1 --2- 172.21.15.152:0/1884492620 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55701127df50 0x5570112803d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.756+0000 7f7d37b59740 1 --2- 172.21.15.152:0/1884492620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570112635d0 0x55701126dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.756+0000 7f7d167fc640 0 client.17178 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:59:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.756+0000 7f7d37b59740 1 -- 172.21.15.152:0/1884492620 >> 172.21.15.152:0/1884492620 conn(0x55701122c310 msgr2=0x557011265930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.756+0000 7f7d37b59740 1 -- 172.21.15.152:0/1884492620 shutdown_connections 2024-09-11T22:59:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.756+0000 7f7d37b59740 1 -- 172.21.15.152:0/1884492620 wait complete. 2024-09-11T22:59:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.756+0000 7f7d37b59740 20 client.17178 trim_cache size 0 max 16384 2024-09-11T22:59:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d37b59740 1 Processor -- start 2024-09-11T22:59:32.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d37b59740 1 -- start start 2024-09-11T22:59:32.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d37b59740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570112635d0 0x55701127a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d37b59740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557011264b40 0x55701126a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d37b59740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701126e370 0x55701125f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d37b59740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301af900 con 0x557011264b40 2024-09-11T22:59:32.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d37b59740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301b1440 con 0x5570112635d0 2024-09-11T22:59:32.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d37b59740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301affd0 con 0x55701126e370 2024-09-11T22:59:32.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d2e7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557011264b40 0x55701126a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d2e7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557011264b40 0x55701126a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46390/0 (socket says 172.21.15.152:46390) 2024-09-11T22:59:32.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d2e7fc640 1 -- 172.21.15.152:0/1617504225 learned_addr learned my addr 172.21.15.152:0/1617504225 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:32.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d2f7fe640 1 --2- 172.21.15.152:0/1617504225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701126e370 0x55701125f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d2e7fc640 1 -- 172.21.15.152:0/1617504225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701126e370 msgr2=0x55701125f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:32.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d2e7fc640 1 --2- 172.21.15.152:0/1617504225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701126e370 0x55701125f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d2e7fc640 1 -- 172.21.15.152:0/1617504225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570112635d0 msgr2=0x55701127a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:32.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d2effd640 1 --2- 172.21.15.152:0/1617504225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570112635d0 0x55701127a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d2e7fc640 1 --2- 172.21.15.152:0/1617504225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570112635d0 0x55701127a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.769+0000 7f7d2e7fc640 1 -- 172.21.15.152:0/1617504225 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55701126d990 con 0x557011264b40 2024-09-11T22:59:32.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.770+0000 7f7d2e7fc640 1 --2- 172.21.15.152:0/1617504225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557011264b40 0x55701126a8e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800bb30 tx=0x7f7d18005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21d6582c774c8e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:32.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.770+0000 7f7d24ff9640 1 -- 172.21.15.152:0/1617504225 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d1802b070 con 0x557011264b40 2024-09-11T22:59:32.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.770+0000 7f7d24ff9640 1 -- 172.21.15.152:0/1617504225 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d18004660 con 0x557011264b40 2024-09-11T22:59:32.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.770+0000 7f7d24ff9640 1 -- 172.21.15.152:0/1617504225 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d18011680 con 0x557011264b40 2024-09-11T22:59:32.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.770+0000 7f7d37b59740 1 -- 172.21.15.152:0/1617504225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557011264b40 msgr2=0x55701126a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:32.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.770+0000 7f7d37b59740 1 --2- 172.21.15.152:0/1617504225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557011264b40 0x55701126a8e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800bb30 tx=0x7f7d18005fd0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d37b59740 1 -- 172.21.15.152:0/1617504225 shutdown_connections 2024-09-11T22:59:32.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d37b59740 1 --2- 172.21.15.152:0/1617504225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701126e370 0x55701125f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d37b59740 1 --2- 172.21.15.152:0/1617504225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557011264b40 0x55701126a8e0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d37b59740 1 --2- 172.21.15.152:0/1617504225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570112635d0 0x55701127a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d37b59740 1 -- 172.21.15.152:0/1617504225 >> 172.21.15.152:0/1617504225 conn(0x55701122c310 msgr2=0x5570112698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d37b59740 1 -- 172.21.15.152:0/1617504225 shutdown_connections 2024-09-11T22:59:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d37b59740 1 -- 172.21.15.152:0/1617504225 wait complete. 2024-09-11T22:59:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d37b59740 1 -- start start 2024-09-11T22:59:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d37b59740 1 Processor -- start 2024-09-11T22:59:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d37b59740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570112635d0 0x5570112e0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d37b59740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557011264b40 0x5570112e14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d37b59740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55701126e370 0x557011271a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d37b59740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301b0d70 con 0x55701126e370 2024-09-11T22:59:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d37b59740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301b06a0 con 0x557011264b40 2024-09-11T22:59:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d37b59740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7d301ad8c0 con 0x5570112635d0 2024-09-11T22:59:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d37b59740 10 client.0 _ll_register_callbacks cb 0x7f7d28031af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d2f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55701126e370 0x557011271a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d2f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55701126e370 0x557011271a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46398/0 (socket says 172.21.15.152:46398) 2024-09-11T22:59:32.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d2f7fe640 1 -- 172.21.15.152:0/2422553563 learned_addr learned my addr 172.21.15.152:0/2422553563 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.771+0000 7f7d2effd640 1 --2- 172.21.15.152:0/2422553563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570112635d0 0x5570112e0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.772+0000 7f7d2e7fc640 1 --2- 172.21.15.152:0/2422553563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557011264b40 0x5570112e14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.772+0000 7f7d2f7fe640 1 -- 172.21.15.152:0/2422553563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570112635d0 msgr2=0x5570112e0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.772+0000 7f7d2f7fe640 1 --2- 172.21.15.152:0/2422553563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570112635d0 0x5570112e0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.772+0000 7f7d2f7fe640 1 -- 172.21.15.152:0/2422553563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557011264b40 msgr2=0x5570112e14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.772+0000 7f7d2f7fe640 1 --2- 172.21.15.152:0/2422553563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557011264b40 0x5570112e14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.772+0000 7f7d2f7fe640 1 -- 172.21.15.152:0/2422553563 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557011272260 con 0x55701126e370 2024-09-11T22:59:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.772+0000 7f7d2f7fe640 1 --2- 172.21.15.152:0/2422553563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55701126e370 0x557011271a60 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f7d28004770 tx=0x7f7d280058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f040507a5be04c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.772+0000 7f7d167fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.772+0000 7f7d167fc640 1 -- 172.21.15.152:0/2422553563 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2804e040 con 0x55701126e370 2024-09-11T22:59:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.772+0000 7f7d37b59740 1 -- 172.21.15.152:0/2422553563 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d18004660 con 0x55701126e370 2024-09-11T22:59:32.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.772+0000 7f7d167fc640 1 -- 172.21.15.152:0/2422553563 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f7d2803dc70 con 0x55701126e370 2024-09-11T22:59:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.772+0000 7f7d167fc640 1 -- 172.21.15.152:0/2422553563 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7d28035040 con 0x55701126e370 2024-09-11T22:59:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.772+0000 7f7d37b59740 10 client.17184 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.772+0000 7f7d37b59740 1 -- 172.21.15.152:0/2422553563 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570112e86f0 con 0x55701126e370 2024-09-11T22:59:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.772+0000 7f7d167fc640 1 -- 172.21.15.152:0/2422553563 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d18004660 con 0x55701126e370 2024-09-11T22:59:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d167fc640 1 -- 172.21.15.152:0/2422553563 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7d280035f0 con 0x55701126e370 2024-09-11T22:59:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d37b59740 10 client.17184 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d37b59740 10 client.17184 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:59:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d37b59740 20 client.17184 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d37b59740 1 --2- 172.21.15.152:0/2422553563 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5570112e8da0 0x5570112eb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d37b59740 4 client.17184 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:59:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d37b59740 1 -- 172.21.15.152:0/2422553563 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570111d0d70 con 0x5570112e8da0 2024-09-11T22:59:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d2effd640 1 --2- 172.21.15.152:0/2422553563 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5570112e8da0 0x5570112eb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d167fc640 10 client.17184 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:59:32.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d2effd640 1 --2- 172.21.15.152:0/2422553563 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5570112e8da0 0x5570112eb190 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c447b2516cfc583c server_cookie=33d816969006853 in_seq=0 out_seq=0 2024-09-11T22:59:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d167fc640 1 -- 172.21.15.152:0/2422553563 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570111d0d70 con 0x5570112e8da0 2024-09-11T22:59:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d167fc640 10 client.17184 handle_command_reply: tid=0 2024-09-11T22:59:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d37b59740 1 client.17184 shutdown 2024-09-11T22:59:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d37b59740 1 -- 172.21.15.152:0/2422553563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55701126e370 msgr2=0x557011271a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d37b59740 1 --2- 172.21.15.152:0/2422553563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55701126e370 0x557011271a60 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f7d28004770 tx=0x7f7d280058d0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d37b59740 1 -- 172.21.15.152:0/2422553563 shutdown_connections 2024-09-11T22:59:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d37b59740 1 --2- 172.21.15.152:0/2422553563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55701126e370 0x557011271a60 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d37b59740 1 --2- 172.21.15.152:0/2422553563 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5570112e8da0 0x5570112eb190 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d37b59740 1 --2- 172.21.15.152:0/2422553563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x557011264b40 0x5570112e14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d37b59740 1 --2- 172.21.15.152:0/2422553563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570112635d0 0x5570112e0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d37b59740 1 -- 172.21.15.152:0/2422553563 >> 172.21.15.152:0/2422553563 conn(0x55701122c310 msgr2=0x55701126c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.773+0000 7f7d167fc640 0 client.17184 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:59:32.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.774+0000 7f7d37b59740 1 -- 172.21.15.152:0/2422553563 shutdown_connections 2024-09-11T22:59:32.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.774+0000 7f7d37b59740 1 -- 172.21.15.152:0/2422553563 wait complete. 2024-09-11T22:59:32.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.774+0000 7f7d37b59740 20 client.17184 trim_cache size 0 max 16384 2024-09-11T22:59:32.790 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:59:32.791 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:32 smithi191 ceph-mon[35211]: pgmap v202: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T22:59:32.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.774+0000 7f7d35780640 1 -- 172.21.15.152:0/2697075815 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7d1c0a2cf0 msgr2=0x7f7d1c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:32.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.774+0000 7f7d35780640 1 --2- 172.21.15.152:0/2697075815 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7d1c0a2cf0 0x7f7d1c0a51b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7d280096c0 tx=0x7f7d280317a0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.774+0000 7f7d35780640 1 -- 172.21.15.152:0/2697075815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d30101df0 msgr2=0x7f7d300744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:32.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.774+0000 7f7d35780640 1 --2- 172.21.15.152:0/2697075815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d30101df0 0x7f7d300744f0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000eab0 tx=0x7f7d2000ef70 comp rx=0 tx=0).stop 2024-09-11T22:59:32.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.774+0000 7f7d35780640 1 -- 172.21.15.152:0/2697075815 shutdown_connections 2024-09-11T22:59:32.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.774+0000 7f7d35780640 1 --2- 172.21.15.152:0/2697075815 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f7d1c0a2cf0 0x7f7d1c0a51b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.774+0000 7f7d35780640 1 --2- 172.21.15.152:0/2697075815 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d30106f20 0x7f7d30074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.774+0000 7f7d35780640 1 --2- 172.21.15.152:0/2697075815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d301027a0 0x7f7d30074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.774+0000 7f7d35780640 1 --2- 172.21.15.152:0/2697075815 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d30101df0 0x7f7d300744f0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:32.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.774+0000 7f7d35780640 1 -- 172.21.15.152:0/2697075815 >> 172.21.15.152:0/2697075815 conn(0x7f7d300fd8b0 msgr2=0x7f7d300ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:32.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.774+0000 7f7d35780640 1 -- 172.21.15.152:0/2697075815 shutdown_connections 2024-09-11T22:59:32.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:32.775+0000 7f7d35780640 1 -- 172.21.15.152:0/2697075815 wait complete. 2024-09-11T22:59:32.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:32 smithi204 ceph-mon[35454]: pgmap v202: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T22:59:33.709 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:59:33.957 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:34.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.558+0000 7f593ca69640 1 -- 172.21.15.152:0/2101858712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59381027a0 msgr2=0x7f59381067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:34.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.558+0000 7f593ca69640 1 --2- 172.21.15.152:0/2101858712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59381027a0 0x7f59381067f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f592c009900 tx=0x7f592c02a410 comp rx=0 tx=0).stop 2024-09-11T22:59:34.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.558+0000 7f593ca69640 1 -- 172.21.15.152:0/2101858712 shutdown_connections 2024-09-11T22:59:34.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.558+0000 7f593ca69640 1 --2- 172.21.15.152:0/2101858712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5938106f20 0x7f593810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.558+0000 7f593ca69640 1 --2- 172.21.15.152:0/2101858712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59381027a0 0x7f59381067f0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.558+0000 7f593ca69640 1 --2- 172.21.15.152:0/2101858712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5938101df0 0x7f59381021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.558+0000 7f593ca69640 1 -- 172.21.15.152:0/2101858712 >> 172.21.15.152:0/2101858712 conn(0x7f59380fd8f0 msgr2=0x7f59380ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:34.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.558+0000 7f593ca69640 1 -- 172.21.15.152:0/2101858712 shutdown_connections 2024-09-11T22:59:34.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.558+0000 7f593ca69640 1 -- 172.21.15.152:0/2101858712 wait complete. 2024-09-11T22:59:34.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.558+0000 7f593ca69640 1 Processor -- start 2024-09-11T22:59:34.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f593ca69640 1 -- start start 2024-09-11T22:59:34.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f593ca69640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5938101df0 0x7f59381984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f593ca69640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59381027a0 0x7f5938198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f593ca69640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5938106f20 0x7f593819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f593ca69640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5938111fa0 con 0x7f5938106f20 2024-09-11T22:59:34.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f593ca69640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5938111e20 con 0x7f59381027a0 2024-09-11T22:59:34.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f593ca69640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5938112120 con 0x7f5938101df0 2024-09-11T22:59:34.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f5936d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5938106f20 0x7f593819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f5936d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5938106f20 0x7f593819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46428/0 (socket says 172.21.15.152:46428) 2024-09-11T22:59:34.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f5936d9d640 1 -- 172.21.15.152:0/2051099689 learned_addr learned my addr 172.21.15.152:0/2051099689 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:34.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f593659c640 1 --2- 172.21.15.152:0/2051099689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5938101df0 0x7f59381984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f5935d9b640 1 --2- 172.21.15.152:0/2051099689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59381027a0 0x7f5938198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f5936d9d640 1 -- 172.21.15.152:0/2051099689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5938101df0 msgr2=0x7f59381984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:34.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f5936d9d640 1 --2- 172.21.15.152:0/2051099689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5938101df0 0x7f59381984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f5936d9d640 1 -- 172.21.15.152:0/2051099689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59381027a0 msgr2=0x7f5938198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:34.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f5936d9d640 1 --2- 172.21.15.152:0/2051099689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59381027a0 0x7f5938198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f5936d9d640 1 -- 172.21.15.152:0/2051099689 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593819d470 con 0x7f5938106f20 2024-09-11T22:59:34.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.559+0000 7f5936d9d640 1 --2- 172.21.15.152:0/2051099689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5938106f20 0x7f593819cd90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f593000b970 tx=0x7f593000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:34.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.560+0000 7f59277fe640 1 -- 172.21.15.152:0/2051099689 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930019070 con 0x7f5938106f20 2024-09-11T22:59:34.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.560+0000 7f593ca69640 1 -- 172.21.15.152:0/2051099689 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593819d760 con 0x7f5938106f20 2024-09-11T22:59:34.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.560+0000 7f59277fe640 1 -- 172.21.15.152:0/2051099689 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f593000cb40 con 0x7f5938106f20 2024-09-11T22:59:34.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.560+0000 7f59277fe640 1 -- 172.21.15.152:0/2051099689 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930007650 con 0x7f5938106f20 2024-09-11T22:59:34.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.560+0000 7f593ca69640 1 -- 172.21.15.152:0/2051099689 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59381a5040 con 0x7f5938106f20 2024-09-11T22:59:34.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.562+0000 7f59277fe640 1 -- 172.21.15.152:0/2051099689 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f593001c070 con 0x7f5938106f20 2024-09-11T22:59:34.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.562+0000 7f593ee42740 1 Processor -- start 2024-09-11T22:59:34.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f593ee42740 1 -- start start 2024-09-11T22:59:34.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f593ee42740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62486b40 0x55bf62485090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f593ee42740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf624855d0 0x55bf62485a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f593ee42740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf62490370 0x55bf62492760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f593ee42740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55bf623eb2f0 con 0x55bf62486b40 2024-09-11T22:59:34.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f593ee42740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bf62392cf0 con 0x55bf624855d0 2024-09-11T22:59:34.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f593ee42740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bf623e72b0 con 0x55bf62490370 2024-09-11T22:59:34.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f59277fe640 1 --2- 172.21.15.152:0/2051099689 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f59200a2cf0 0x7f59200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f5935d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62486b40 0x55bf62485090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f5935d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62486b40 0x55bf62485090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46444/0 (socket says 172.21.15.152:46444) 2024-09-11T22:59:34.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f5935d9b640 1 -- 172.21.15.152:0/3882082522 learned_addr learned my addr 172.21.15.152:0/3882082522 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:34.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f593659c640 1 --2- 172.21.15.152:0/3882082522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf624855d0 0x55bf62485a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f5936d9d640 1 --2- 172.21.15.152:0/2051099689 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f59200a2cf0 0x7f59200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f5935d9b640 1 --2- 172.21.15.152:0/3882082522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf62490370 0x55bf62492760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f59277fe640 1 -- 172.21.15.152:0/2051099689 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f59300ce960 con 0x7f5938106f20 2024-09-11T22:59:34.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f5935d9b640 1 -- 172.21.15.152:0/3882082522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf62490370 msgr2=0x55bf62492760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:34.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f5935d9b640 1 --2- 172.21.15.152:0/3882082522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf62490370 0x55bf62492760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f5935d9b640 1 -- 172.21.15.152:0/3882082522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf624855d0 msgr2=0x55bf62485a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:34.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f5935d9b640 1 --2- 172.21.15.152:0/3882082522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf624855d0 0x55bf62485a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.563+0000 7f5935d9b640 1 -- 172.21.15.152:0/3882082522 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf623773a0 con 0x55bf62486b40 2024-09-11T22:59:34.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.564+0000 7f5935d9b640 1 --2- 172.21.15.152:0/3882082522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62486b40 0x55bf62485090 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f592c0096c0 tx=0x7f592c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a5990d01333ce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:34.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.564+0000 7f5936d9d640 1 --2- 172.21.15.152:0/2051099689 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f59200a2cf0 0x7f59200a51b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f59300974b0 tx=0x7f593006beb0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:34.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.564+0000 7f5924ff9640 1 -- 172.21.15.152:0/3882082522 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c044030 con 0x55bf62486b40 2024-09-11T22:59:34.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.564+0000 7f5924ff9640 1 -- 172.21.15.152:0/3882082522 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f592c005020 con 0x55bf62486b40 2024-09-11T22:59:34.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.564+0000 7f5924ff9640 1 -- 172.21.15.152:0/3882082522 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c035040 con 0x55bf62486b40 2024-09-11T22:59:34.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.564+0000 7f593ee42740 1 -- 172.21.15.152:0/3882082522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62486b40 msgr2=0x55bf62485090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:34.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.564+0000 7f593ee42740 1 --2- 172.21.15.152:0/3882082522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62486b40 0x55bf62485090 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f592c0096c0 tx=0x7f592c000bc0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.564+0000 7f593ee42740 1 -- 172.21.15.152:0/3882082522 shutdown_connections 2024-09-11T22:59:34.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.564+0000 7f593ee42740 1 --2- 172.21.15.152:0/3882082522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf62490370 0x55bf62492760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593ee42740 1 --2- 172.21.15.152:0/3882082522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf624855d0 0x55bf62485a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593ee42740 1 --2- 172.21.15.152:0/3882082522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62486b40 0x55bf62485090 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593ee42740 1 -- 172.21.15.152:0/3882082522 >> 172.21.15.152:0/3882082522 conn(0x55bf6244e310 msgr2=0x55bf62481c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:34.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593ee42740 1 -- 172.21.15.152:0/3882082522 shutdown_connections 2024-09-11T22:59:34.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593ee42740 1 -- 172.21.15.152:0/3882082522 wait complete. 2024-09-11T22:59:34.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593ee42740 1 -- start start 2024-09-11T22:59:34.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593ee42740 1 Processor -- start 2024-09-11T22:59:34.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593ee42740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf624855d0 0x55bf6248fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593ee42740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62486b40 0x55bf6248cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593ee42740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf62490370 0x55bf6248d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593ee42740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55bf62493010 con 0x55bf62486b40 2024-09-11T22:59:34.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593ee42740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55bf623a6060 con 0x55bf62490370 2024-09-11T22:59:34.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593ee42740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55bf62493190 con 0x55bf624855d0 2024-09-11T22:59:34.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593ee42740 10 client.0 _ll_register_callbacks cb 0x55bf620f98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:34.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62486b40 0x55bf6248cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62486b40 0x55bf6248cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46458/0 (socket says 172.21.15.152:46458) 2024-09-11T22:59:34.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f593659c640 1 -- 172.21.15.152:0/1195506734 learned_addr learned my addr 172.21.15.152:0/1195506734 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:34.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f5935d9b640 1 --2- 172.21.15.152:0/1195506734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf62490370 0x55bf6248d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.565+0000 7f5935d9b640 1 --2- 172.21.15.152:0/1195506734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf624855d0 0x55bf6248fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.566+0000 7f593659c640 1 -- 172.21.15.152:0/1195506734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf624855d0 msgr2=0x55bf6248fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:34.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.566+0000 7f593659c640 1 --2- 172.21.15.152:0/1195506734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf624855d0 0x55bf6248fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.566+0000 7f593659c640 1 -- 172.21.15.152:0/1195506734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf62490370 msgr2=0x55bf6248d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:34.583 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.566+0000 7f593659c640 1 --2- 172.21.15.152:0/1195506734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf62490370 0x55bf6248d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.566+0000 7f593659c640 1 -- 172.21.15.152:0/1195506734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf6215c3c0 con 0x55bf62486b40 2024-09-11T22:59:34.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.566+0000 7f5935d9b640 1 --2- 172.21.15.152:0/1195506734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf62490370 0x55bf6248d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:34.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.566+0000 7f593659c640 1 --2- 172.21.15.152:0/1195506734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62486b40 0x55bf6248cfb0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f592800bfc0 tx=0x7f5928014c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71167f7333e2af5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:34.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.566+0000 7f591dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:34.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.566+0000 7f591dffb640 1 -- 172.21.15.152:0/1195506734 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5928021020 con 0x55bf62486b40 2024-09-11T22:59:34.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.566+0000 7f591dffb640 1 -- 172.21.15.152:0/1195506734 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f592800eb80 con 0x55bf62486b40 2024-09-11T22:59:34.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.566+0000 7f593ee42740 1 -- 172.21.15.152:0/1195506734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f592c005020 con 0x55bf62486b40 2024-09-11T22:59:34.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.566+0000 7f591dffb640 1 -- 172.21.15.152:0/1195506734 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f592800ed80 con 0x55bf62486b40 2024-09-11T22:59:34.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.566+0000 7f593ee42740 10 client.17196 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:34.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.566+0000 7f593ee42740 1 -- 172.21.15.152:0/1195506734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bf6249eaa0 con 0x55bf62486b40 2024-09-11T22:59:34.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.566+0000 7f591dffb640 1 -- 172.21.15.152:0/1195506734 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f592c005020 con 0x55bf62486b40 2024-09-11T22:59:34.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.567+0000 7f591dffb640 1 -- 172.21.15.152:0/1195506734 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f592801c070 con 0x55bf62486b40 2024-09-11T22:59:34.584 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.567+0000 7f593ee42740 10 client.17196 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:34.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.567+0000 7f593ee42740 10 client.17196 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:59:34.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.567+0000 7f593ee42740 20 client.17196 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:34.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.567+0000 7f593ee42740 1 --2- 172.21.15.152:0/1195506734 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55bf624a0000 0x55bf624a2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.567+0000 7f593ee42740 4 client.17196 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:59:34.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.567+0000 7f593ee42740 1 -- 172.21.15.152:0/1195506734 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf623f8860 con 0x55bf624a0000 2024-09-11T22:59:34.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.567+0000 7f5935d9b640 1 --2- 172.21.15.152:0/1195506734 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55bf624a0000 0x55bf624a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.567+0000 7f591dffb640 10 client.17196 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:59:34.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.567+0000 7f5935d9b640 1 --2- 172.21.15.152:0/1195506734 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55bf624a0000 0x55bf624a2420 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2fed1029a9cc9cf4 server_cookie=5a20a68242a64c45 in_seq=0 out_seq=0 2024-09-11T22:59:34.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.568+0000 7f591dffb640 1 -- 172.21.15.152:0/1195506734 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf623f8860 con 0x55bf624a0000 2024-09-11T22:59:34.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.568+0000 7f591dffb640 10 client.17196 handle_command_reply: tid=0 2024-09-11T22:59:34.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.568+0000 7f593ee42740 1 client.17196 shutdown 2024-09-11T22:59:34.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.568+0000 7f593ee42740 1 -- 172.21.15.152:0/1195506734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62486b40 msgr2=0x55bf6248cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:34.585 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.568+0000 7f593ee42740 1 --2- 172.21.15.152:0/1195506734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62486b40 0x55bf6248cfb0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f592800bfc0 tx=0x7f5928014c70 comp rx=0 tx=0).stop 2024-09-11T22:59:34.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.568+0000 7f593ee42740 1 -- 172.21.15.152:0/1195506734 shutdown_connections 2024-09-11T22:59:34.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.568+0000 7f593ee42740 1 --2- 172.21.15.152:0/1195506734 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55bf624a0000 0x55bf624a2420 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.568+0000 7f593ee42740 1 --2- 172.21.15.152:0/1195506734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf62490370 0x55bf6248d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.568+0000 7f593ee42740 1 --2- 172.21.15.152:0/1195506734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62486b40 0x55bf6248cfb0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.568+0000 7f591dffb640 0 client.17196 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:59:34.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.568+0000 7f593ee42740 1 --2- 172.21.15.152:0/1195506734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf624855d0 0x55bf6248fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.568+0000 7f593ee42740 1 -- 172.21.15.152:0/1195506734 >> 172.21.15.152:0/1195506734 conn(0x55bf6244e310 msgr2=0x55bf62487930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:34.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.568+0000 7f593ee42740 1 -- 172.21.15.152:0/1195506734 shutdown_connections 2024-09-11T22:59:34.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.568+0000 7f593ee42740 1 -- 172.21.15.152:0/1195506734 wait complete. 2024-09-11T22:59:34.586 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.568+0000 7f593ee42740 20 client.17196 trim_cache size 0 max 16384 2024-09-11T22:59:34.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.581+0000 7f593ee42740 1 Processor -- start 2024-09-11T22:59:34.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.581+0000 7f593ee42740 1 -- start start 2024-09-11T22:59:34.587 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:34 smithi152 ceph-mon[31181]: pgmap v203: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:34.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f593ee42740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf624855d0 0x55bf6248cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f593ee42740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf62486b40 0x55bf6249c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f593ee42740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62490370 0x55bf624a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f593ee42740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f59381ab4c0 con 0x55bf62490370 2024-09-11T22:59:34.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f593ee42740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f59381ad000 con 0x55bf624855d0 2024-09-11T22:59:34.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f593ee42740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f59381abb90 con 0x55bf62486b40 2024-09-11T22:59:34.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f5935d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62490370 0x55bf624a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f5935d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62490370 0x55bf624a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46470/0 (socket says 172.21.15.152:46470) 2024-09-11T22:59:34.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f5935d9b640 1 -- 172.21.15.152:0/1328741212 learned_addr learned my addr 172.21.15.152:0/1328741212 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:34.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f5935d9b640 1 --2- 172.21.15.152:0/1328741212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf624855d0 0x55bf6248cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f593659c640 1 --2- 172.21.15.152:0/1328741212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf62486b40 0x55bf6249c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f5935d9b640 1 -- 172.21.15.152:0/1328741212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf62486b40 msgr2=0x55bf6249c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:34.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f5935d9b640 1 --2- 172.21.15.152:0/1328741212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf62486b40 0x55bf6249c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f5935d9b640 1 -- 172.21.15.152:0/1328741212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf624855d0 msgr2=0x55bf6248cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:34.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f5935d9b640 1 --2- 172.21.15.152:0/1328741212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf624855d0 0x55bf6248cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f5935d9b640 1 -- 172.21.15.152:0/1328741212 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf62481ed0 con 0x55bf62490370 2024-09-11T22:59:34.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.582+0000 7f5935d9b640 1 --2- 172.21.15.152:0/1328741212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62490370 0x55bf624a7cd0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f592c006010 tx=0x7f592c005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e8dd3a8c853568 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:34.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.583+0000 7f591ffff640 1 -- 172.21.15.152:0/1328741212 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c035040 con 0x55bf62490370 2024-09-11T22:59:34.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.583+0000 7f591ffff640 1 -- 172.21.15.152:0/1328741212 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f592c0055c0 con 0x55bf62490370 2024-09-11T22:59:34.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.583+0000 7f591ffff640 1 -- 172.21.15.152:0/1328741212 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c04a7d0 con 0x55bf62490370 2024-09-11T22:59:34.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.583+0000 7f593ee42740 1 -- 172.21.15.152:0/1328741212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62490370 msgr2=0x55bf624a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:34.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.583+0000 7f593ee42740 1 --2- 172.21.15.152:0/1328741212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62490370 0x55bf624a7cd0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f592c006010 tx=0x7f592c005e30 comp rx=0 tx=0).stop 2024-09-11T22:59:34.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.583+0000 7f593ee42740 1 -- 172.21.15.152:0/1328741212 shutdown_connections 2024-09-11T22:59:34.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.583+0000 7f593ee42740 1 --2- 172.21.15.152:0/1328741212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62490370 0x55bf624a7cd0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.583+0000 7f593ee42740 1 --2- 172.21.15.152:0/1328741212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf62486b40 0x55bf6249c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.583+0000 7f593ee42740 1 --2- 172.21.15.152:0/1328741212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf624855d0 0x55bf6248cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.583+0000 7f593ee42740 1 -- 172.21.15.152:0/1328741212 >> 172.21.15.152:0/1328741212 conn(0x55bf6244e310 msgr2=0x55bf6248c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:34.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.583+0000 7f593ee42740 1 -- 172.21.15.152:0/1328741212 shutdown_connections 2024-09-11T22:59:34.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.583+0000 7f593ee42740 1 -- 172.21.15.152:0/1328741212 wait complete. 2024-09-11T22:59:34.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.584+0000 7f593ee42740 1 -- start start 2024-09-11T22:59:34.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.584+0000 7f593ee42740 1 Processor -- start 2024-09-11T22:59:34.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.584+0000 7f593ee42740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf624855d0 0x55bf62493b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.584+0000 7f593ee42740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf62486b40 0x55bf62494070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.584+0000 7f593ee42740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62490370 0x55bf6250b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.584+0000 7f593ee42740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f59381ac930 con 0x55bf62490370 2024-09-11T22:59:34.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.584+0000 7f593ee42740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f59381ac260 con 0x55bf62486b40 2024-09-11T22:59:34.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.584+0000 7f593ee42740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f59381a94a0 con 0x55bf624855d0 2024-09-11T22:59:34.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.584+0000 7f593ee42740 10 client.0 _ll_register_callbacks cb 0x7f592800bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:34.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.584+0000 7f5935d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62490370 0x55bf6250b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.584+0000 7f5935d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62490370 0x55bf6250b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46480/0 (socket says 172.21.15.152:46480) 2024-09-11T22:59:34.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.584+0000 7f5935d9b640 1 -- 172.21.15.152:0/2830043502 learned_addr learned my addr 172.21.15.152:0/2830043502 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:34.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.584+0000 7f5935d9b640 1 --2- 172.21.15.152:0/2830043502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf624855d0 0x55bf62493b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.584+0000 7f593659c640 1 --2- 172.21.15.152:0/2830043502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf62486b40 0x55bf62494070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.585+0000 7f5935d9b640 1 -- 172.21.15.152:0/2830043502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf624855d0 msgr2=0x55bf62493b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:34.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.585+0000 7f5935d9b640 1 --2- 172.21.15.152:0/2830043502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf624855d0 0x55bf62493b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.585+0000 7f5935d9b640 1 -- 172.21.15.152:0/2830043502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf62486b40 msgr2=0x55bf62494070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:34.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.585+0000 7f5935d9b640 1 --2- 172.21.15.152:0/2830043502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf62486b40 0x55bf62494070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.585+0000 7f5935d9b640 1 -- 172.21.15.152:0/2830043502 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf6250bab0 con 0x55bf62490370 2024-09-11T22:59:34.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.585+0000 7f593659c640 1 --2- 172.21.15.152:0/2830043502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf62486b40 0x55bf62494070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:34.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.585+0000 7f5935d9b640 1 --2- 172.21.15.152:0/2830043502 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62490370 0x55bf6250b470 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f592c02a930 tx=0x7f592c02ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a966ec4922ac630e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:34.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.585+0000 7f591dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:34.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.585+0000 7f591dffb640 1 -- 172.21.15.152:0/2830043502 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c035040 con 0x55bf62490370 2024-09-11T22:59:34.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.585+0000 7f591dffb640 1 -- 172.21.15.152:0/2830043502 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f592c04a890 con 0x55bf62490370 2024-09-11T22:59:34.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.585+0000 7f593ee42740 1 -- 172.21.15.152:0/2830043502 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f592c0055c0 con 0x55bf62490370 2024-09-11T22:59:34.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.585+0000 7f591dffb640 1 -- 172.21.15.152:0/2830043502 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f592c04aa90 con 0x55bf62490370 2024-09-11T22:59:34.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.585+0000 7f593ee42740 10 client.17202 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:34.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.585+0000 7f593ee42740 1 -- 172.21.15.152:0/2830043502 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bf62512e60 con 0x55bf62490370 2024-09-11T22:59:34.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.585+0000 7f591dffb640 1 -- 172.21.15.152:0/2830043502 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f592c0055c0 con 0x55bf62490370 2024-09-11T22:59:34.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.586+0000 7f591dffb640 1 -- 172.21.15.152:0/2830043502 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f592c0562d0 con 0x55bf62490370 2024-09-11T22:59:34.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.586+0000 7f593ee42740 10 client.17202 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:34.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.586+0000 7f593ee42740 10 client.17202 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:59:34.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.586+0000 7f593ee42740 20 client.17202 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:34.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.586+0000 7f593ee42740 1 --2- 172.21.15.152:0/2830043502 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55bf62513030 0x55bf62515420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:34.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.586+0000 7f593ee42740 4 client.17202 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:59:34.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.586+0000 7f593ee42740 1 -- 172.21.15.152:0/2830043502 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf623f2d70 con 0x55bf62513030 2024-09-11T22:59:34.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.586+0000 7f593659c640 1 --2- 172.21.15.152:0/2830043502 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55bf62513030 0x55bf62515420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:34.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.586+0000 7f593659c640 1 --2- 172.21.15.152:0/2830043502 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55bf62513030 0x55bf62515420 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c24792374885c0ab server_cookie=125f7d96acdea580 in_seq=0 out_seq=0 2024-09-11T22:59:34.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.586+0000 7f591dffb640 10 client.17202 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:59:34.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.586+0000 7f591dffb640 1 -- 172.21.15.152:0/2830043502 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf623f2d70 con 0x55bf62513030 2024-09-11T22:59:34.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.587+0000 7f591dffb640 10 client.17202 handle_command_reply: tid=0 2024-09-11T22:59:34.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.587+0000 7f593ee42740 1 client.17202 shutdown 2024-09-11T22:59:34.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.587+0000 7f593ee42740 1 -- 172.21.15.152:0/2830043502 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62490370 msgr2=0x55bf6250b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:34.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.587+0000 7f593ee42740 1 --2- 172.21.15.152:0/2830043502 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62490370 0x55bf6250b470 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f592c02a930 tx=0x7f592c02ab10 comp rx=0 tx=0).stop 2024-09-11T22:59:34.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.587+0000 7f593ee42740 1 -- 172.21.15.152:0/2830043502 shutdown_connections 2024-09-11T22:59:34.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.587+0000 7f593ee42740 1 --2- 172.21.15.152:0/2830043502 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55bf62513030 0x55bf62515420 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.587+0000 7f591dffb640 0 client.17202 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:59:34.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.587+0000 7f593ee42740 1 --2- 172.21.15.152:0/2830043502 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bf62490370 0x55bf6250b470 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.587+0000 7f593ee42740 1 --2- 172.21.15.152:0/2830043502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55bf62486b40 0x55bf62494070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.587+0000 7f593ee42740 1 --2- 172.21.15.152:0/2830043502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bf624855d0 0x55bf62493b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.587+0000 7f593ee42740 1 -- 172.21.15.152:0/2830043502 >> 172.21.15.152:0/2830043502 conn(0x55bf6244e310 msgr2=0x55bf6248c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:34.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.587+0000 7f593ee42740 1 -- 172.21.15.152:0/2830043502 shutdown_connections 2024-09-11T22:59:34.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.587+0000 7f593ee42740 1 -- 172.21.15.152:0/2830043502 wait complete. 2024-09-11T22:59:34.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.587+0000 7f593ee42740 20 client.17202 trim_cache size 0 max 16384 2024-09-11T22:59:34.598 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:59:34.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.588+0000 7f593ca69640 1 -- 172.21.15.152:0/2051099689 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f59200a2cf0 msgr2=0x7f59200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:34.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.588+0000 7f593ca69640 1 --2- 172.21.15.152:0/2051099689 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f59200a2cf0 0x7f59200a51b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f59300974b0 tx=0x7f593006beb0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.588+0000 7f593ca69640 1 -- 172.21.15.152:0/2051099689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5938106f20 msgr2=0x7f593819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:34.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.588+0000 7f593ca69640 1 --2- 172.21.15.152:0/2051099689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5938106f20 0x7f593819cd90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f593000b970 tx=0x7f593000be30 comp rx=0 tx=0).stop 2024-09-11T22:59:34.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.588+0000 7f593ca69640 1 -- 172.21.15.152:0/2051099689 shutdown_connections 2024-09-11T22:59:34.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.588+0000 7f593ca69640 1 --2- 172.21.15.152:0/2051099689 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f59200a2cf0 0x7f59200a51b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.588+0000 7f593ca69640 1 --2- 172.21.15.152:0/2051099689 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5938106f20 0x7f593819cd90 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.588+0000 7f593ca69640 1 --2- 172.21.15.152:0/2051099689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59381027a0 0x7f5938198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.588+0000 7f593ca69640 1 --2- 172.21.15.152:0/2051099689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5938101df0 0x7f59381984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:34.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.588+0000 7f593ca69640 1 -- 172.21.15.152:0/2051099689 >> 172.21.15.152:0/2051099689 conn(0x7f59380fd8f0 msgr2=0x7f59380feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:34.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.588+0000 7f593ca69640 1 -- 172.21.15.152:0/2051099689 shutdown_connections 2024-09-11T22:59:34.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:34.588+0000 7f593ca69640 1 -- 172.21.15.152:0/2051099689 wait complete. 2024-09-11T22:59:34.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:34 smithi191 ceph-mon[35211]: pgmap v203: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:34.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:34 smithi204 ceph-mon[35454]: pgmap v203: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:35.520 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:59:35.775 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:36.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.270+0000 7f254669d640 1 -- 172.21.15.152:0/3560968193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2540101df0 msgr2=0x7f25401021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:36.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.270+0000 7f254669d640 1 --2- 172.21.15.152:0/3560968193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2540101df0 0x7f25401021d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2530009900 tx=0x7f253002a410 comp rx=0 tx=0).stop 2024-09-11T22:59:36.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.270+0000 7f254669d640 1 -- 172.21.15.152:0/3560968193 shutdown_connections 2024-09-11T22:59:36.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.270+0000 7f254669d640 1 --2- 172.21.15.152:0/3560968193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2540106f20 0x7f254010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.270+0000 7f254669d640 1 --2- 172.21.15.152:0/3560968193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25401027a0 0x7f25401067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.270+0000 7f254669d640 1 --2- 172.21.15.152:0/3560968193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2540101df0 0x7f25401021d0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.270+0000 7f254669d640 1 -- 172.21.15.152:0/3560968193 >> 172.21.15.152:0/3560968193 conn(0x7f25400fd8f0 msgr2=0x7f25400ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:36.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.270+0000 7f254669d640 1 -- 172.21.15.152:0/3560968193 shutdown_connections 2024-09-11T22:59:36.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.270+0000 7f254669d640 1 -- 172.21.15.152:0/3560968193 wait complete. 2024-09-11T22:59:36.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.270+0000 7f254669d640 1 Processor -- start 2024-09-11T22:59:36.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f254669d640 1 -- start start 2024-09-11T22:59:36.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f254669d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2540101df0 0x7f2540198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f254669d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25401027a0 0x7f2540198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f254669d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2540106f20 0x7f254019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f254669d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f2540111fa0 con 0x7f2540101df0 2024-09-11T22:59:36.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f254669d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f2540111e20 con 0x7f25401027a0 2024-09-11T22:59:36.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f254669d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f2540112120 con 0x7f2540106f20 2024-09-11T22:59:36.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f253ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2540101df0 0x7f2540198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f253ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2540101df0 0x7f2540198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46510/0 (socket says 172.21.15.152:46510) 2024-09-11T22:59:36.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f253ffff640 1 -- 172.21.15.152:0/3288257681 learned_addr learned my addr 172.21.15.152:0/3288257681 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:36.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f2537fff640 1 --2- 172.21.15.152:0/3288257681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25401027a0 0x7f2540198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f2544c3a640 1 --2- 172.21.15.152:0/3288257681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2540106f20 0x7f254019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f253ffff640 1 -- 172.21.15.152:0/3288257681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2540106f20 msgr2=0x7f254019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:36.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f253ffff640 1 --2- 172.21.15.152:0/3288257681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2540106f20 0x7f254019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f253ffff640 1 -- 172.21.15.152:0/3288257681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25401027a0 msgr2=0x7f2540198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:36.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f253ffff640 1 --2- 172.21.15.152:0/3288257681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25401027a0 0x7f2540198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f253ffff640 1 -- 172.21.15.152:0/3288257681 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254019d4e0 con 0x7f2540101df0 2024-09-11T22:59:36.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f2537fff640 1 --2- 172.21.15.152:0/3288257681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25401027a0 0x7f2540198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:36.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.271+0000 7f253ffff640 1 --2- 172.21.15.152:0/3288257681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2540101df0 0x7f2540198530 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2530005ab0 tx=0x7f25300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:36.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.272+0000 7f253dffb640 1 -- 172.21.15.152:0/3288257681 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253003f030 con 0x7f2540101df0 2024-09-11T22:59:36.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.272+0000 7f253dffb640 1 -- 172.21.15.152:0/3288257681 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2530003ce0 con 0x7f2540101df0 2024-09-11T22:59:36.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.272+0000 7f254669d640 1 -- 172.21.15.152:0/3288257681 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254019d770 con 0x7f2540101df0 2024-09-11T22:59:36.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.272+0000 7f253dffb640 1 -- 172.21.15.152:0/3288257681 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530035070 con 0x7f2540101df0 2024-09-11T22:59:36.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.272+0000 7f254669d640 1 -- 172.21.15.152:0/3288257681 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25401a50b0 con 0x7f2540101df0 2024-09-11T22:59:36.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.274+0000 7f253dffb640 1 -- 172.21.15.152:0/3288257681 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2530042030 con 0x7f2540101df0 2024-09-11T22:59:36.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.274+0000 7f2548a76740 1 Processor -- start 2024-09-11T22:59:36.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.274+0000 7f2548a76740 1 -- start start 2024-09-11T22:59:36.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.274+0000 7f2548a76740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c1b40 0x56069f6c0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.274+0000 7f2548a76740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6c05d0 0x56069f6c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2548a76740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6cb370 0x56069f6cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2548a76740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56069f6262f0 con 0x56069f6cb370 2024-09-11T22:59:36.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2548a76740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56069f5cdcf0 con 0x56069f6c1b40 2024-09-11T22:59:36.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2548a76740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56069f6222b0 con 0x56069f6c05d0 2024-09-11T22:59:36.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f253dffb640 1 --2- 172.21.15.152:0/3288257681 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f252c0a2d70 0x7f252c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2537fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6cb370 0x56069f6cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2537fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6cb370 0x56069f6cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46522/0 (socket says 172.21.15.152:46522) 2024-09-11T22:59:36.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2537fff640 1 -- 172.21.15.152:0/1775949764 learned_addr learned my addr 172.21.15.152:0/1775949764 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:36.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f253ffff640 1 --2- 172.21.15.152:0/1775949764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6c05d0 0x56069f6c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2544c3a640 1 --2- 172.21.15.152:0/3288257681 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f252c0a2d70 0x7f252c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2544c3a640 1 --2- 172.21.15.152:0/1775949764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c1b40 0x56069f6c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f253dffb640 1 -- 172.21.15.152:0/3288257681 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f25300efd40 con 0x7f2540101df0 2024-09-11T22:59:36.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2537fff640 1 -- 172.21.15.152:0/1775949764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6c05d0 msgr2=0x56069f6c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:36.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2537fff640 1 --2- 172.21.15.152:0/1775949764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6c05d0 0x56069f6c0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2537fff640 1 -- 172.21.15.152:0/1775949764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c1b40 msgr2=0x56069f6c0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:36.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2537fff640 1 --2- 172.21.15.152:0/1775949764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c1b40 0x56069f6c0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2537fff640 1 -- 172.21.15.152:0/1775949764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56069f5b23a0 con 0x56069f6cb370 2024-09-11T22:59:36.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2544c3a640 1 --2- 172.21.15.152:0/1775949764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c1b40 0x56069f6c0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:36.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2537fff640 1 --2- 172.21.15.152:0/1775949764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6cb370 0x56069f6cd760 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f252800cc50 tx=0x7f252800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fafbbc575e8d0f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:36.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f2544c3a640 1 --2- 172.21.15.152:0/3288257681 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f252c0a2d70 0x7f252c0a5230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2538009fd0 tx=0x7f2538009540 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:36.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.275+0000 7f25367fc640 1 -- 172.21.15.152:0/1775949764 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528014070 con 0x56069f6cb370 2024-09-11T22:59:36.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.276+0000 7f25367fc640 1 -- 172.21.15.152:0/1775949764 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2528019b40 con 0x56069f6cb370 2024-09-11T22:59:36.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.276+0000 7f25367fc640 1 -- 172.21.15.152:0/1775949764 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528010b60 con 0x56069f6cb370 2024-09-11T22:59:36.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.276+0000 7f2548a76740 1 -- 172.21.15.152:0/1775949764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6cb370 msgr2=0x56069f6cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:36.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.276+0000 7f2548a76740 1 --2- 172.21.15.152:0/1775949764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6cb370 0x56069f6cd760 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f252800cc50 tx=0x7f252800e720 comp rx=0 tx=0).stop 2024-09-11T22:59:36.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.276+0000 7f2548a76740 1 -- 172.21.15.152:0/1775949764 shutdown_connections 2024-09-11T22:59:36.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.276+0000 7f2548a76740 1 --2- 172.21.15.152:0/1775949764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6cb370 0x56069f6cd760 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.276+0000 7f2548a76740 1 --2- 172.21.15.152:0/1775949764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6c05d0 0x56069f6c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.276+0000 7f2548a76740 1 --2- 172.21.15.152:0/1775949764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c1b40 0x56069f6c0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.276+0000 7f2548a76740 1 -- 172.21.15.152:0/1775949764 >> 172.21.15.152:0/1775949764 conn(0x56069f689310 msgr2=0x56069f6bcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:36.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.276+0000 7f2548a76740 1 -- 172.21.15.152:0/1775949764 shutdown_connections 2024-09-11T22:59:36.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.276+0000 7f2548a76740 1 -- 172.21.15.152:0/1775949764 wait complete. 2024-09-11T22:59:36.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.277+0000 7f2548a76740 1 -- start start 2024-09-11T22:59:36.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.277+0000 7f2548a76740 1 Processor -- start 2024-09-11T22:59:36.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.277+0000 7f2548a76740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6c05d0 0x56069f6cada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.277+0000 7f2548a76740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c1b40 0x56069f6c8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.277+0000 7f2548a76740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6cb370 0x56069f6c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.277+0000 7f2548a76740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56069f6ce070 con 0x56069f6c05d0 2024-09-11T22:59:36.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.277+0000 7f2548a76740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56069f5e1060 con 0x56069f6c1b40 2024-09-11T22:59:36.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.277+0000 7f2548a76740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56069f6ce1f0 con 0x56069f6cb370 2024-09-11T22:59:36.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.277+0000 7f2548a76740 10 client.0 _ll_register_callbacks cb 0x56069f3348c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:36.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.277+0000 7f253ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6c05d0 0x56069f6cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.277+0000 7f253ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6c05d0 0x56069f6cada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46536/0 (socket says 172.21.15.152:46536) 2024-09-11T22:59:36.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.277+0000 7f253ffff640 1 -- 172.21.15.152:0/778671681 learned_addr learned my addr 172.21.15.152:0/778671681 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:36.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f2537fff640 1 --2- 172.21.15.152:0/778671681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c1b40 0x56069f6c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f253ffff640 1 -- 172.21.15.152:0/778671681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6cb370 msgr2=0x56069f6c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:36.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f253ffff640 1 --2- 172.21.15.152:0/778671681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6cb370 0x56069f6c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f253ffff640 1 -- 172.21.15.152:0/778671681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c1b40 msgr2=0x56069f6c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:36.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f253ffff640 1 --2- 172.21.15.152:0/778671681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c1b40 0x56069f6c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f253ffff640 1 -- 172.21.15.152:0/778671681 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56069f3973c0 con 0x56069f6c05d0 2024-09-11T22:59:36.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f253ffff640 1 --2- 172.21.15.152:0/778671681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6c05d0 0x56069f6cada0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f25300bc240 tx=0x7f253008c110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcae9a5863af208 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:36.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f251f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:36.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f251f7fe640 1 -- 172.21.15.152:0/778671681 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25300f1040 con 0x56069f6c05d0 2024-09-11T22:59:36.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f251f7fe640 1 -- 172.21.15.152:0/778671681 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f253003b460 con 0x56069f6c05d0 2024-09-11T22:59:36.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f2548a76740 1 -- 172.21.15.152:0/778671681 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2528019b40 con 0x56069f6c05d0 2024-09-11T22:59:36.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f251f7fe640 1 -- 172.21.15.152:0/778671681 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2530049040 con 0x56069f6c05d0 2024-09-11T22:59:36.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f2548a76740 10 client.17214 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:36.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f2548a76740 1 -- 172.21.15.152:0/778671681 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56069f6d9b50 con 0x56069f6c05d0 2024-09-11T22:59:36.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f251f7fe640 1 -- 172.21.15.152:0/778671681 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2528019b40 con 0x56069f6c05d0 2024-09-11T22:59:36.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f251f7fe640 1 -- 172.21.15.152:0/778671681 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f25300ed760 con 0x56069f6c05d0 2024-09-11T22:59:36.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f2548a76740 10 client.17214 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:36.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.278+0000 7f2548a76740 10 client.17214 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:59:36.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.279+0000 7f2548a76740 20 client.17214 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:36.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.279+0000 7f2548a76740 1 --2- 172.21.15.152:0/778671681 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x56069f6dafb0 0x56069f6dd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.279+0000 7f2548a76740 4 client.17214 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:59:36.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.279+0000 7f2548a76740 1 -- 172.21.15.152:0/778671681 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56069f633860 con 0x56069f6dafb0 2024-09-11T22:59:36.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.279+0000 7f2537fff640 1 --2- 172.21.15.152:0/778671681 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x56069f6dafb0 0x56069f6dd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.279+0000 7f251f7fe640 10 client.17214 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:59:36.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.279+0000 7f2537fff640 1 --2- 172.21.15.152:0/778671681 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x56069f6dafb0 0x56069f6dd420 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6d1f9bfab81e829 server_cookie=a35f20d5e74f72fb in_seq=0 out_seq=0 2024-09-11T22:59:36.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.280+0000 7f251f7fe640 1 -- 172.21.15.152:0/778671681 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56069f633860 con 0x56069f6dafb0 2024-09-11T22:59:36.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.280+0000 7f251f7fe640 10 client.17214 handle_command_reply: tid=0 2024-09-11T22:59:36.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.280+0000 7f2548a76740 1 client.17214 shutdown 2024-09-11T22:59:36.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.280+0000 7f2548a76740 1 -- 172.21.15.152:0/778671681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6c05d0 msgr2=0x56069f6cada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:36.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.280+0000 7f2548a76740 1 --2- 172.21.15.152:0/778671681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6c05d0 0x56069f6cada0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f25300bc240 tx=0x7f253008c110 comp rx=0 tx=0).stop 2024-09-11T22:59:36.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.280+0000 7f2548a76740 1 -- 172.21.15.152:0/778671681 shutdown_connections 2024-09-11T22:59:36.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.280+0000 7f2548a76740 1 --2- 172.21.15.152:0/778671681 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x56069f6dafb0 0x56069f6dd420 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.280+0000 7f2548a76740 1 --2- 172.21.15.152:0/778671681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6cb370 0x56069f6c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.280+0000 7f2548a76740 1 --2- 172.21.15.152:0/778671681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c1b40 0x56069f6c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.280+0000 7f2548a76740 1 --2- 172.21.15.152:0/778671681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6c05d0 0x56069f6cada0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.280+0000 7f251f7fe640 0 client.17214 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:59:36.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.280+0000 7f2548a76740 1 -- 172.21.15.152:0/778671681 >> 172.21.15.152:0/778671681 conn(0x56069f689310 msgr2=0x56069f6c2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:36.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.280+0000 7f2548a76740 1 -- 172.21.15.152:0/778671681 shutdown_connections 2024-09-11T22:59:36.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.280+0000 7f2548a76740 1 -- 172.21.15.152:0/778671681 wait complete. 2024-09-11T22:59:36.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.280+0000 7f2548a76740 20 client.17214 trim_cache size 0 max 16384 2024-09-11T22:59:36.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2548a76740 1 Processor -- start 2024-09-11T22:59:36.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2548a76740 1 -- start start 2024-09-11T22:59:36.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2548a76740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c05d0 0x56069f6bbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2548a76740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6c1b40 0x56069f6bc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2548a76740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6cb370 0x56069f6c3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2548a76740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f25401ad730 con 0x56069f6c1b40 2024-09-11T22:59:36.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2548a76740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f25401af270 con 0x56069f6c05d0 2024-09-11T22:59:36.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2548a76740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f25401ade00 con 0x56069f6cb370 2024-09-11T22:59:36.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2537fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6c1b40 0x56069f6bc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2537fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6c1b40 0x56069f6bc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46550/0 (socket says 172.21.15.152:46550) 2024-09-11T22:59:36.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2537fff640 1 -- 172.21.15.152:0/3496375093 learned_addr learned my addr 172.21.15.152:0/3496375093 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:36.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f253ffff640 1 --2- 172.21.15.152:0/3496375093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c05d0 0x56069f6bbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2544c3a640 1 --2- 172.21.15.152:0/3496375093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6cb370 0x56069f6c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2537fff640 1 -- 172.21.15.152:0/3496375093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6cb370 msgr2=0x56069f6c3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:36.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2537fff640 1 --2- 172.21.15.152:0/3496375093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6cb370 0x56069f6c3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2537fff640 1 -- 172.21.15.152:0/3496375093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c05d0 msgr2=0x56069f6bbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:36.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2537fff640 1 --2- 172.21.15.152:0/3496375093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c05d0 0x56069f6bbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f2537fff640 1 -- 172.21.15.152:0/3496375093 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56069f6d77f0 con 0x56069f6c1b40 2024-09-11T22:59:36.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.293+0000 7f253ffff640 1 --2- 172.21.15.152:0/3496375093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c05d0 0x56069f6bbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:36.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.294+0000 7f2537fff640 1 --2- 172.21.15.152:0/3496375093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6c1b40 0x56069f6bc130 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f2528010730 tx=0x7f2528010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ba269deb95cec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:36.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.294+0000 7f2535ffb640 1 -- 172.21.15.152:0/3496375093 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528014040 con 0x56069f6c1b40 2024-09-11T22:59:36.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.294+0000 7f2535ffb640 1 -- 172.21.15.152:0/3496375093 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25280104f0 con 0x56069f6c1b40 2024-09-11T22:59:36.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.294+0000 7f2535ffb640 1 -- 172.21.15.152:0/3496375093 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528005330 con 0x56069f6c1b40 2024-09-11T22:59:36.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.294+0000 7f2548a76740 1 -- 172.21.15.152:0/3496375093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6c1b40 msgr2=0x56069f6bc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:36.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.294+0000 7f2548a76740 1 --2- 172.21.15.152:0/3496375093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6c1b40 0x56069f6bc130 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f2528010730 tx=0x7f2528010800 comp rx=0 tx=0).stop 2024-09-11T22:59:36.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.294+0000 7f2548a76740 1 -- 172.21.15.152:0/3496375093 shutdown_connections 2024-09-11T22:59:36.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.294+0000 7f2548a76740 1 --2- 172.21.15.152:0/3496375093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6cb370 0x56069f6c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.294+0000 7f2548a76740 1 --2- 172.21.15.152:0/3496375093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6c1b40 0x56069f6bc130 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.294+0000 7f2548a76740 1 --2- 172.21.15.152:0/3496375093 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c05d0 0x56069f6bbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.294+0000 7f2548a76740 1 -- 172.21.15.152:0/3496375093 >> 172.21.15.152:0/3496375093 conn(0x56069f689310 msgr2=0x56069f6bb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:36.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.294+0000 7f2548a76740 1 -- 172.21.15.152:0/3496375093 shutdown_connections 2024-09-11T22:59:36.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.294+0000 7f2548a76740 1 -- 172.21.15.152:0/3496375093 wait complete. 2024-09-11T22:59:36.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.295+0000 7f2548a76740 1 -- start start 2024-09-11T22:59:36.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.295+0000 7f2548a76740 1 Processor -- start 2024-09-11T22:59:36.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.295+0000 7f2548a76740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c05d0 0x56069f6c5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.295+0000 7f2548a76740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6c1b40 0x56069f6c5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.295+0000 7f2548a76740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6cb370 0x56069f6c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.295+0000 7f2548a76740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f25401aeba0 con 0x56069f6cb370 2024-09-11T22:59:36.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.295+0000 7f2548a76740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f25401ae4d0 con 0x56069f6c05d0 2024-09-11T22:59:36.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.295+0000 7f2548a76740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f25401ab6f0 con 0x56069f6c1b40 2024-09-11T22:59:36.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.295+0000 7f2548a76740 10 client.0 _ll_register_callbacks cb 0x7f2530005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:36.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.295+0000 7f2544c3a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6cb370 0x56069f6c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.295+0000 7f2544c3a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6cb370 0x56069f6c5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46566/0 (socket says 172.21.15.152:46566) 2024-09-11T22:59:36.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.295+0000 7f2544c3a640 1 -- 172.21.15.152:0/1274940426 learned_addr learned my addr 172.21.15.152:0/1274940426 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:36.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2537fff640 1 --2- 172.21.15.152:0/1274940426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6c1b40 0x56069f6c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f253ffff640 1 --2- 172.21.15.152:0/1274940426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c05d0 0x56069f6c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2544c3a640 1 -- 172.21.15.152:0/1274940426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6c1b40 msgr2=0x56069f6c5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:36.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2544c3a640 1 --2- 172.21.15.152:0/1274940426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6c1b40 0x56069f6c5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2544c3a640 1 -- 172.21.15.152:0/1274940426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c05d0 msgr2=0x56069f6c5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:36.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2544c3a640 1 --2- 172.21.15.152:0/1274940426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c05d0 0x56069f6c5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2544c3a640 1 -- 172.21.15.152:0/1274940426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56069f744e60 con 0x56069f6cb370 2024-09-11T22:59:36.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2537fff640 1 --2- 172.21.15.152:0/1274940426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6c1b40 0x56069f6c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:59:36.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f253ffff640 1 --2- 172.21.15.152:0/1274940426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c05d0 0x56069f6c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:59:36.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2544c3a640 1 --2- 172.21.15.152:0/1274940426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6cb370 0x56069f6c5c90 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f253800af20 tx=0x7f2538009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3c041a15742358 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:36.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f251f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:36.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f251f7fe640 1 -- 172.21.15.152:0/1274940426 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538013040 con 0x56069f6cb370 2024-09-11T22:59:36.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f251f7fe640 1 -- 172.21.15.152:0/1274940426 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f2538018a70 con 0x56069f6cb370 2024-09-11T22:59:36.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2548a76740 1 -- 172.21.15.152:0/1274940426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25280104f0 con 0x56069f6cb370 2024-09-11T22:59:36.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f251f7fe640 1 -- 172.21.15.152:0/1274940426 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2538018c70 con 0x56069f6cb370 2024-09-11T22:59:36.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2548a76740 10 client.17220 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:36.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2548a76740 1 -- 172.21.15.152:0/1274940426 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56069f7461a0 con 0x56069f6cb370 2024-09-11T22:59:36.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f251f7fe640 1 -- 172.21.15.152:0/1274940426 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25280104f0 con 0x56069f6cb370 2024-09-11T22:59:36.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f251f7fe640 1 -- 172.21.15.152:0/1274940426 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f253802c4e0 con 0x56069f6cb370 2024-09-11T22:59:36.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2548a76740 10 client.17220 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:36.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2548a76740 10 client.17220 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:59:36.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2548a76740 20 client.17220 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:36.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2548a76740 1 --2- 172.21.15.152:0/1274940426 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x56069f7469a0 0x56069f748dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:36.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2548a76740 4 client.17220 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:59:36.313 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.296+0000 7f2548a76740 1 -- 172.21.15.152:0/1274940426 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56069f62dd70 con 0x56069f7469a0 2024-09-11T22:59:36.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.297+0000 7f253ffff640 1 --2- 172.21.15.152:0/1274940426 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x56069f7469a0 0x56069f748dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:36.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f251f7fe640 10 client.17220 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:59:36.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f253ffff640 1 --2- 172.21.15.152:0/1274940426 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x56069f7469a0 0x56069f748dc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a554781c03e8e3e server_cookie=5d75a706b87c437d in_seq=0 out_seq=0 2024-09-11T22:59:36.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f251f7fe640 1 -- 172.21.15.152:0/1274940426 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56069f62dd70 con 0x56069f7469a0 2024-09-11T22:59:36.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f251f7fe640 10 client.17220 handle_command_reply: tid=0 2024-09-11T22:59:36.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f2548a76740 1 client.17220 shutdown 2024-09-11T22:59:36.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f2548a76740 1 -- 172.21.15.152:0/1274940426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6cb370 msgr2=0x56069f6c5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:36.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f2548a76740 1 --2- 172.21.15.152:0/1274940426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6cb370 0x56069f6c5c90 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f253800af20 tx=0x7f2538009d60 comp rx=0 tx=0).stop 2024-09-11T22:59:36.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f2548a76740 1 -- 172.21.15.152:0/1274940426 shutdown_connections 2024-09-11T22:59:36.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f2548a76740 1 --2- 172.21.15.152:0/1274940426 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x56069f7469a0 0x56069f748dc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f2548a76740 1 --2- 172.21.15.152:0/1274940426 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56069f6cb370 0x56069f6c5c90 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f2548a76740 1 --2- 172.21.15.152:0/1274940426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56069f6c1b40 0x56069f6c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.314 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f251f7fe640 0 client.17220 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:59:36.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f2548a76740 1 --2- 172.21.15.152:0/1274940426 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56069f6c05d0 0x56069f6c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f2548a76740 1 -- 172.21.15.152:0/1274940426 >> 172.21.15.152:0/1274940426 conn(0x56069f689310 msgr2=0x56069f6bbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:36.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f2548a76740 1 -- 172.21.15.152:0/1274940426 shutdown_connections 2024-09-11T22:59:36.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f2548a76740 1 -- 172.21.15.152:0/1274940426 wait complete. 2024-09-11T22:59:36.315 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.298+0000 7f2548a76740 20 client.17220 trim_cache size 0 max 16384 2024-09-11T22:59:36.315 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:59:36.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.299+0000 7f254669d640 1 -- 172.21.15.152:0/3288257681 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f252c0a2d70 msgr2=0x7f252c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:36.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.299+0000 7f254669d640 1 --2- 172.21.15.152:0/3288257681 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f252c0a2d70 0x7f252c0a5230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2538009fd0 tx=0x7f2538009540 comp rx=0 tx=0).stop 2024-09-11T22:59:36.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.299+0000 7f254669d640 1 -- 172.21.15.152:0/3288257681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2540101df0 msgr2=0x7f2540198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:36.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.299+0000 7f254669d640 1 --2- 172.21.15.152:0/3288257681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2540101df0 0x7f2540198530 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2530005ab0 tx=0x7f25300042f0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.299+0000 7f254669d640 1 -- 172.21.15.152:0/3288257681 shutdown_connections 2024-09-11T22:59:36.316 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.299+0000 7f254669d640 1 --2- 172.21.15.152:0/3288257681 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f252c0a2d70 0x7f252c0a5230 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.299+0000 7f254669d640 1 --2- 172.21.15.152:0/3288257681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2540106f20 0x7f254019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.299+0000 7f254669d640 1 --2- 172.21.15.152:0/3288257681 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25401027a0 0x7f2540198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.299+0000 7f254669d640 1 --2- 172.21.15.152:0/3288257681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2540101df0 0x7f2540198530 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:36.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.299+0000 7f254669d640 1 -- 172.21.15.152:0/3288257681 >> 172.21.15.152:0/3288257681 conn(0x7f25400fd8f0 msgr2=0x7f25400fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:36.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.299+0000 7f254669d640 1 -- 172.21.15.152:0/3288257681 shutdown_connections 2024-09-11T22:59:36.317 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:36.299+0000 7f254669d640 1 -- 172.21.15.152:0/3288257681 wait complete. 2024-09-11T22:59:36.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:36 smithi152 ceph-mon[31181]: pgmap v204: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:59:36.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:36 smithi191 ceph-mon[35211]: pgmap v204: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:59:36.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:36 smithi204 ceph-mon[35454]: pgmap v204: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T22:59:37.221 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:59:37.471 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:37.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.982+0000 7f376f96a640 1 -- 172.21.15.152:0/3867895733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3768106f20 msgr2=0x7f376810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:37.984 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.982+0000 7f376f96a640 1 --2- 172.21.15.152:0/3867895733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3768106f20 0x7f376810eaa0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f3760009900 tx=0x7f376002a410 comp rx=0 tx=0).stop 2024-09-11T22:59:37.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.982+0000 7f376f96a640 1 -- 172.21.15.152:0/3867895733 shutdown_connections 2024-09-11T22:59:37.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.982+0000 7f376f96a640 1 --2- 172.21.15.152:0/3867895733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3768106f20 0x7f376810eaa0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:37.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.982+0000 7f376f96a640 1 --2- 172.21.15.152:0/3867895733 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f37681027a0 0x7f37681067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:37.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.982+0000 7f376f96a640 1 --2- 172.21.15.152:0/3867895733 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3768101df0 0x7f37681021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:37.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.982+0000 7f376f96a640 1 -- 172.21.15.152:0/3867895733 >> 172.21.15.152:0/3867895733 conn(0x7f37680fd8f0 msgr2=0x7f37680ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:37.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.982+0000 7f376f96a640 1 -- 172.21.15.152:0/3867895733 shutdown_connections 2024-09-11T22:59:37.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.982+0000 7f376f96a640 1 -- 172.21.15.152:0/3867895733 wait complete. 2024-09-11T22:59:37.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.982+0000 7f376f96a640 1 Processor -- start 2024-09-11T22:59:37.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.982+0000 7f376f96a640 1 -- start start 2024-09-11T22:59:37.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376f96a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3768101df0 0x7f3768198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:37.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376f96a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37681027a0 0x7f37681988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:37.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376f96a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3768106f20 0x7f376819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376f96a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3768111f80 con 0x7f37681027a0 2024-09-11T22:59:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376f96a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3768111e00 con 0x7f3768101df0 2024-09-11T22:59:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376f96a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3768112100 con 0x7f3768106f20 2024-09-11T22:59:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376cf05640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37681027a0 0x7f37681988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376cf05640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37681027a0 0x7f37681988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46594/0 (socket says 172.21.15.152:46594) 2024-09-11T22:59:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376cf05640 1 -- 172.21.15.152:0/3024274764 learned_addr learned my addr 172.21.15.152:0/3024274764 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376d706640 1 --2- 172.21.15.152:0/3024274764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3768101df0 0x7f3768198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376df07640 1 --2- 172.21.15.152:0/3024274764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3768106f20 0x7f376819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376cf05640 1 -- 172.21.15.152:0/3024274764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3768106f20 msgr2=0x7f376819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376cf05640 1 --2- 172.21.15.152:0/3024274764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3768106f20 0x7f376819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376cf05640 1 -- 172.21.15.152:0/3024274764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3768101df0 msgr2=0x7f3768198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376cf05640 1 --2- 172.21.15.152:0/3024274764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3768101df0 0x7f3768198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:37.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376cf05640 1 -- 172.21.15.152:0/3024274764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f376819d320 con 0x7f37681027a0 2024-09-11T22:59:37.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376cf05640 1 --2- 172.21.15.152:0/3024274764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37681027a0 0x7f37681988b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f375c00c920 tx=0x7f375c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:37.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f37667fc640 1 -- 172.21.15.152:0/3024274764 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c007c70 con 0x7f37681027a0 2024-09-11T22:59:37.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376f96a640 1 -- 172.21.15.152:0/3024274764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f376819d610 con 0x7f37681027a0 2024-09-11T22:59:37.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f37667fc640 1 -- 172.21.15.152:0/3024274764 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375c007e10 con 0x7f37681027a0 2024-09-11T22:59:37.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f37667fc640 1 -- 172.21.15.152:0/3024274764 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c018890 con 0x7f37681027a0 2024-09-11T22:59:37.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.983+0000 7f376f96a640 1 -- 172.21.15.152:0/3024274764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37681a4ef0 con 0x7f37681027a0 2024-09-11T22:59:37.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.986+0000 7f37667fc640 1 -- 172.21.15.152:0/3024274764 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f375c018a70 con 0x7f37681027a0 2024-09-11T22:59:37.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.986+0000 7f3771d43740 1 Processor -- start 2024-09-11T22:59:37.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f3771d43740 1 -- start start 2024-09-11T22:59:37.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f3771d43740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564351431b40 0x564351430090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:37.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f3771d43740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643514305d0 0x564351430a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:37.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f3771d43740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x564351441b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:37.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f3771d43740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5643513962f0 con 0x5643514305d0 2024-09-11T22:59:37.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f3771d43740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56435133dcf0 con 0x564351431b40 2024-09-11T22:59:37.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f3771d43740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5643513922b0 con 0x56435143f780 2024-09-11T22:59:37.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f376d706640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643514305d0 0x564351430a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:37.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f376d706640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643514305d0 0x564351430a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46600/0 (socket says 172.21.15.152:46600) 2024-09-11T22:59:37.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f376d706640 1 -- 172.21.15.152:0/4046511894 learned_addr learned my addr 172.21.15.152:0/4046511894 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:37.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f37667fc640 1 --2- 172.21.15.152:0/3024274764 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f37500a2cf0 0x7f37500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:37.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f376df07640 1 --2- 172.21.15.152:0/4046511894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564351431b40 0x564351430090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:37.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f376cf05640 1 --2- 172.21.15.152:0/4046511894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x564351441b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:37.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f376df07640 1 --2- 172.21.15.152:0/3024274764 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f37500a2cf0 0x7f37500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:37.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f376d706640 1 -- 172.21.15.152:0/4046511894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 msgr2=0x564351441b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f376d706640 1 --2- 172.21.15.152:0/4046511894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x564351441b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f376d706640 1 -- 172.21.15.152:0/4046511894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564351431b40 msgr2=0x564351430090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f376d706640 1 --2- 172.21.15.152:0/4046511894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564351431b40 0x564351430090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f37667fc640 1 -- 172.21.15.152:0/3024274764 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f375c0cf420 con 0x7f37681027a0 2024-09-11T22:59:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.987+0000 7f376d706640 1 -- 172.21.15.152:0/4046511894 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643513223a0 con 0x5643514305d0 2024-09-11T22:59:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.988+0000 7f376d706640 1 --2- 172.21.15.152:0/4046511894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643514305d0 0x564351430a10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f37580103f0 tx=0x7f37580108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92d8583527eb43e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.988+0000 7f376df07640 1 --2- 172.21.15.152:0/3024274764 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f37500a2cf0 0x7f37500a51b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f37600096c0 tx=0x7f3760009260 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.988+0000 7f374f7fe640 1 -- 172.21.15.152:0/4046511894 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758016070 con 0x5643514305d0 2024-09-11T22:59:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.988+0000 7f374f7fe640 1 -- 172.21.15.152:0/4046511894 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3758004d20 con 0x5643514305d0 2024-09-11T22:59:37.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.988+0000 7f374f7fe640 1 -- 172.21.15.152:0/4046511894 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37580053a0 con 0x5643514305d0 2024-09-11T22:59:37.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.989+0000 7f3771d43740 1 -- 172.21.15.152:0/4046511894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643514305d0 msgr2=0x564351430a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:37.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.989+0000 7f3771d43740 1 --2- 172.21.15.152:0/4046511894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643514305d0 0x564351430a10 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f37580103f0 tx=0x7f37580108b0 comp rx=0 tx=0).stop 2024-09-11T22:59:37.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.989+0000 7f3771d43740 1 -- 172.21.15.152:0/4046511894 shutdown_connections 2024-09-11T22:59:37.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.989+0000 7f3771d43740 1 --2- 172.21.15.152:0/4046511894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x564351441b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:37.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.989+0000 7f3771d43740 1 --2- 172.21.15.152:0/4046511894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643514305d0 0x564351430a10 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:37.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.989+0000 7f3771d43740 1 --2- 172.21.15.152:0/4046511894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564351431b40 0x564351430090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:37.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.989+0000 7f3771d43740 1 -- 172.21.15.152:0/4046511894 >> 172.21.15.152:0/4046511894 conn(0x5643513f9310 msgr2=0x56435142cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:37.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.989+0000 7f3771d43740 1 -- 172.21.15.152:0/4046511894 shutdown_connections 2024-09-11T22:59:37.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.989+0000 7f3771d43740 1 -- 172.21.15.152:0/4046511894 wait complete. 2024-09-11T22:59:37.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.989+0000 7f3771d43740 1 -- start start 2024-09-11T22:59:37.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.989+0000 7f3771d43740 1 Processor -- start 2024-09-11T22:59:37.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f3771d43740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643514305d0 0x56435143f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:37.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f3771d43740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564351431b40 0x56435143c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:37.996 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f3771d43740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x56435143c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:37.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f3771d43740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x564351442480 con 0x564351431b40 2024-09-11T22:59:37.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f3771d43740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x564351351060 con 0x5643514305d0 2024-09-11T22:59:37.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f3771d43740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x564351442600 con 0x56435143f780 2024-09-11T22:59:37.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f3771d43740 10 client.0 _ll_register_callbacks cb 0x5643510a48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:37.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f376cf05640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564351431b40 0x56435143c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:37.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f376cf05640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564351431b40 0x56435143c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46604/0 (socket says 172.21.15.152:46604) 2024-09-11T22:59:37.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f376cf05640 1 -- 172.21.15.152:0/2930542686 learned_addr learned my addr 172.21.15.152:0/2930542686 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:37.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f376df07640 1 --2- 172.21.15.152:0/2930542686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x56435143c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:37.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f376d706640 1 --2- 172.21.15.152:0/2930542686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643514305d0 0x56435143f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:37.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f376cf05640 1 -- 172.21.15.152:0/2930542686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 msgr2=0x56435143c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:37.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f376cf05640 1 --2- 172.21.15.152:0/2930542686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x56435143c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:37.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f376cf05640 1 -- 172.21.15.152:0/2930542686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643514305d0 msgr2=0x56435143f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:37.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f376cf05640 1 --2- 172.21.15.152:0/2930542686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643514305d0 0x56435143f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:37.997 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f376cf05640 1 -- 172.21.15.152:0/2930542686 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643511073c0 con 0x564351431b40 2024-09-11T22:59:37.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f376cf05640 1 --2- 172.21.15.152:0/2930542686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564351431b40 0x56435143c3e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f375c09b8d0 tx=0x7f375c06b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fbc34183b1b228 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:37.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f374cff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:37.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f374cff9640 1 -- 172.21.15.152:0/2930542686 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0a4600 con 0x564351431b40 2024-09-11T22:59:37.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f374cff9640 1 -- 172.21.15.152:0/2930542686 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f375c0cce40 con 0x564351431b40 2024-09-11T22:59:37.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.990+0000 7f3771d43740 1 -- 172.21.15.152:0/2930542686 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3758004d20 con 0x564351431b40 2024-09-11T22:59:37.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.991+0000 7f374cff9640 1 -- 172.21.15.152:0/2930542686 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f375c0cd040 con 0x564351431b40 2024-09-11T22:59:37.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.991+0000 7f3771d43740 10 client.17232 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:37.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.991+0000 7f3771d43740 1 -- 172.21.15.152:0/2930542686 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56435144df30 con 0x564351431b40 2024-09-11T22:59:37.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.991+0000 7f374cff9640 1 -- 172.21.15.152:0/2930542686 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3758004d20 con 0x564351431b40 2024-09-11T22:59:37.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.991+0000 7f374cff9640 1 -- 172.21.15.152:0/2930542686 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f375c0860b0 con 0x564351431b40 2024-09-11T22:59:37.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.991+0000 7f3771d43740 10 client.17232 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:37.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.991+0000 7f3771d43740 10 client.17232 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:59:37.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.991+0000 7f3771d43740 20 client.17232 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:37.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.991+0000 7f3771d43740 1 --2- 172.21.15.152:0/2930542686 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x56435144f340 0x564351451770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:37.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.991+0000 7f3771d43740 4 client.17232 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:59:37.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.991+0000 7f3771d43740 1 -- 172.21.15.152:0/2930542686 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643513a3860 con 0x56435144f340 2024-09-11T22:59:37.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.991+0000 7f376d706640 1 --2- 172.21.15.152:0/2930542686 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x56435144f340 0x564351451770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:37.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.992+0000 7f376d706640 1 --2- 172.21.15.152:0/2930542686 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x56435144f340 0x564351451770 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b791c386273321de server_cookie=8cc8becfebadf414 in_seq=0 out_seq=0 2024-09-11T22:59:38.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.992+0000 7f374cff9640 10 client.17232 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:59:38.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.992+0000 7f374cff9640 1 -- 172.21.15.152:0/2930542686 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643513a3860 con 0x56435144f340 2024-09-11T22:59:38.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.992+0000 7f374cff9640 10 client.17232 handle_command_reply: tid=0 2024-09-11T22:59:38.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.992+0000 7f3771d43740 1 client.17232 shutdown 2024-09-11T22:59:38.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.993+0000 7f3771d43740 1 -- 172.21.15.152:0/2930542686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564351431b40 msgr2=0x56435143c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:38.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.993+0000 7f3771d43740 1 --2- 172.21.15.152:0/2930542686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564351431b40 0x56435143c3e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f375c09b8d0 tx=0x7f375c06b7f0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.993+0000 7f3771d43740 1 -- 172.21.15.152:0/2930542686 shutdown_connections 2024-09-11T22:59:38.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.993+0000 7f3771d43740 1 --2- 172.21.15.152:0/2930542686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x56435143c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.993+0000 7f3771d43740 1 --2- 172.21.15.152:0/2930542686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564351431b40 0x56435143c3e0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.993+0000 7f3771d43740 1 --2- 172.21.15.152:0/2930542686 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x56435144f340 0x564351451770 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.993+0000 7f3771d43740 1 --2- 172.21.15.152:0/2930542686 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643514305d0 0x56435143f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.993+0000 7f3771d43740 1 -- 172.21.15.152:0/2930542686 >> 172.21.15.152:0/2930542686 conn(0x5643513f9310 msgr2=0x564351432960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:38.000 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.993+0000 7f374cff9640 0 client.17232 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:59:38.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.993+0000 7f3771d43740 1 -- 172.21.15.152:0/2930542686 shutdown_connections 2024-09-11T22:59:38.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.993+0000 7f3771d43740 1 -- 172.21.15.152:0/2930542686 wait complete. 2024-09-11T22:59:38.001 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:37.993+0000 7f3771d43740 20 client.17232 trim_cache size 0 max 16384 2024-09-11T22:59:38.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.005+0000 7f3771d43740 1 Processor -- start 2024-09-11T22:59:38.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f3771d43740 1 -- start start 2024-09-11T22:59:38.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f3771d43740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643514305d0 0x56435142ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:38.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f3771d43740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564351431b40 0x56435144b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:38.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f3771d43740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x564351457040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:38.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f3771d43740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f37681ab4e0 con 0x5643514305d0 2024-09-11T22:59:38.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f3771d43740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f37681ad020 con 0x564351431b40 2024-09-11T22:59:38.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f3771d43740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37681abbb0 con 0x56435143f780 2024-09-11T22:59:38.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f376d706640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643514305d0 0x56435142ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:38.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f376d706640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643514305d0 0x56435142ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46606/0 (socket says 172.21.15.152:46606) 2024-09-11T22:59:38.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f376d706640 1 -- 172.21.15.152:0/2315456533 learned_addr learned my addr 172.21.15.152:0/2315456533 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:38.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f376df07640 1 --2- 172.21.15.152:0/2315456533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x564351457040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:38.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f376cf05640 1 --2- 172.21.15.152:0/2315456533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564351431b40 0x56435144b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:38.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f376d706640 1 -- 172.21.15.152:0/2315456533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 msgr2=0x564351457040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:38.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f376d706640 1 --2- 172.21.15.152:0/2315456533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x564351457040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f376d706640 1 -- 172.21.15.152:0/2315456533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564351431b40 msgr2=0x56435144b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:38.008 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f376d706640 1 --2- 172.21.15.152:0/2315456533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564351431b40 0x56435144b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f376d706640 1 -- 172.21.15.152:0/2315456533 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564351431760 con 0x5643514305d0 2024-09-11T22:59:38.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.006+0000 7f376d706640 1 --2- 172.21.15.152:0/2315456533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643514305d0 0x56435142ce10 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f37580103f0 tx=0x7f3758009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd56c91764088df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:38.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.007+0000 7f374effd640 1 -- 172.21.15.152:0/2315456533 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758016040 con 0x5643514305d0 2024-09-11T22:59:38.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.007+0000 7f374effd640 1 -- 172.21.15.152:0/2315456533 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375802c070 con 0x5643514305d0 2024-09-11T22:59:38.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.007+0000 7f374effd640 1 -- 172.21.15.152:0/2315456533 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3758005050 con 0x5643514305d0 2024-09-11T22:59:38.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.007+0000 7f3771d43740 1 -- 172.21.15.152:0/2315456533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643514305d0 msgr2=0x56435142ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:38.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.007+0000 7f3771d43740 1 --2- 172.21.15.152:0/2315456533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643514305d0 0x56435142ce10 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f37580103f0 tx=0x7f3758009680 comp rx=0 tx=0).stop 2024-09-11T22:59:38.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.007+0000 7f3771d43740 1 -- 172.21.15.152:0/2315456533 shutdown_connections 2024-09-11T22:59:38.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.007+0000 7f3771d43740 1 --2- 172.21.15.152:0/2315456533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x564351457040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.007+0000 7f3771d43740 1 --2- 172.21.15.152:0/2315456533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564351431b40 0x56435144b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.007+0000 7f3771d43740 1 --2- 172.21.15.152:0/2315456533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5643514305d0 0x56435142ce10 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.007+0000 7f3771d43740 1 -- 172.21.15.152:0/2315456533 >> 172.21.15.152:0/2315456533 conn(0x5643513f9310 msgr2=0x56435142c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:38.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.007+0000 7f3771d43740 1 -- 172.21.15.152:0/2315456533 shutdown_connections 2024-09-11T22:59:38.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.007+0000 7f3771d43740 1 -- 172.21.15.152:0/2315456533 wait complete. 2024-09-11T22:59:38.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f3771d43740 1 -- start start 2024-09-11T22:59:38.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f3771d43740 1 Processor -- start 2024-09-11T22:59:38.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f3771d43740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643514305d0 0x564351447c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:38.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f3771d43740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564351431b40 0x564351438e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:38.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f3771d43740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x564351439350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:38.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f3771d43740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f37681ac950 con 0x564351431b40 2024-09-11T22:59:38.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f3771d43740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f37681ac280 con 0x5643514305d0 2024-09-11T22:59:38.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f3771d43740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f37681a94a0 con 0x56435143f780 2024-09-11T22:59:38.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f3771d43740 10 client.0 _ll_register_callbacks cb 0x7f375c00c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:38.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f376cf05640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564351431b40 0x564351438e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:38.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f376cf05640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564351431b40 0x564351438e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46612/0 (socket says 172.21.15.152:46612) 2024-09-11T22:59:38.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f376cf05640 1 -- 172.21.15.152:0/2865604422 learned_addr learned my addr 172.21.15.152:0/2865604422 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:38.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f376d706640 1 --2- 172.21.15.152:0/2865604422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643514305d0 0x564351447c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:38.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f376df07640 1 --2- 172.21.15.152:0/2865604422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x564351439350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:38.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f376cf05640 1 -- 172.21.15.152:0/2865604422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 msgr2=0x564351439350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:38.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f376cf05640 1 --2- 172.21.15.152:0/2865604422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x564351439350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f376cf05640 1 -- 172.21.15.152:0/2865604422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643514305d0 msgr2=0x564351447c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:38.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f376cf05640 1 --2- 172.21.15.152:0/2865604422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643514305d0 0x564351447c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.008+0000 7f376cf05640 1 -- 172.21.15.152:0/2865604422 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564351439890 con 0x564351431b40 2024-09-11T22:59:38.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f376df07640 1 --2- 172.21.15.152:0/2865604422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x564351439350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:59:38.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f376cf05640 1 --2- 172.21.15.152:0/2865604422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564351431b40 0x564351438e10 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f375c0a40d0 tx=0x7f375c09b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e453b5a0d7d31c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:38.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f374cff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:38.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f374cff9640 1 -- 172.21.15.152:0/2865604422 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0cd040 con 0x564351431b40 2024-09-11T22:59:38.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f374cff9640 1 -- 172.21.15.152:0/2865604422 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f375c0a4170 con 0x564351431b40 2024-09-11T22:59:38.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f3771d43740 1 -- 172.21.15.152:0/2865604422 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375802c070 con 0x564351431b40 2024-09-11T22:59:38.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f374cff9640 1 -- 172.21.15.152:0/2865604422 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f375c0a4370 con 0x564351431b40 2024-09-11T22:59:38.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f3771d43740 10 client.17238 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:38.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f3771d43740 1 -- 172.21.15.152:0/2865604422 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643514b9d20 con 0x564351431b40 2024-09-11T22:59:38.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f374cff9640 1 -- 172.21.15.152:0/2865604422 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375802c070 con 0x564351431b40 2024-09-11T22:59:38.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f374cff9640 1 -- 172.21.15.152:0/2865604422 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f375c09a850 con 0x564351431b40 2024-09-11T22:59:38.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f3771d43740 10 client.17238 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:38.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f3771d43740 10 client.17238 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:59:38.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f3771d43740 20 client.17238 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:38.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f3771d43740 1 --2- 172.21.15.152:0/2865604422 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5643514ba4a0 0x5643514bc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:38.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f3771d43740 4 client.17238 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:59:38.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.009+0000 7f3771d43740 1 -- 172.21.15.152:0/2865604422 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56435139dd70 con 0x5643514ba4a0 2024-09-11T22:59:38.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.010+0000 7f376d706640 1 --2- 172.21.15.152:0/2865604422 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5643514ba4a0 0x5643514bc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:38.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.010+0000 7f374cff9640 10 client.17238 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:59:38.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.010+0000 7f376d706640 1 --2- 172.21.15.152:0/2865604422 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5643514ba4a0 0x5643514bc8c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8a9b78719fbdc38 server_cookie=59f0ef814a35e2ca in_seq=0 out_seq=0 2024-09-11T22:59:38.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.010+0000 7f374cff9640 1 -- 172.21.15.152:0/2865604422 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56435139dd70 con 0x5643514ba4a0 2024-09-11T22:59:38.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.010+0000 7f374cff9640 10 client.17238 handle_command_reply: tid=0 2024-09-11T22:59:38.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.010+0000 7f3771d43740 1 client.17238 shutdown 2024-09-11T22:59:38.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.010+0000 7f3771d43740 1 -- 172.21.15.152:0/2865604422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564351431b40 msgr2=0x564351438e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:38.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.010+0000 7f3771d43740 1 --2- 172.21.15.152:0/2865604422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564351431b40 0x564351438e10 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f375c0a40d0 tx=0x7f375c09b420 comp rx=0 tx=0).stop 2024-09-11T22:59:38.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.010+0000 7f3771d43740 1 -- 172.21.15.152:0/2865604422 shutdown_connections 2024-09-11T22:59:38.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.010+0000 7f3771d43740 1 --2- 172.21.15.152:0/2865604422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56435143f780 0x564351439350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f3771d43740 1 --2- 172.21.15.152:0/2865604422 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564351431b40 0x564351438e10 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f3771d43740 1 --2- 172.21.15.152:0/2865604422 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5643514ba4a0 0x5643514bc8c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f3771d43740 1 --2- 172.21.15.152:0/2865604422 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5643514305d0 0x564351447c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f3771d43740 1 -- 172.21.15.152:0/2865604422 >> 172.21.15.152:0/2865604422 conn(0x5643513f9310 msgr2=0x56435143ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:38.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f374cff9640 0 client.17238 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:59:38.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f3771d43740 1 -- 172.21.15.152:0/2865604422 shutdown_connections 2024-09-11T22:59:38.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f3771d43740 1 -- 172.21.15.152:0/2865604422 wait complete. 2024-09-11T22:59:38.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f3771d43740 20 client.17238 trim_cache size 0 max 16384 2024-09-11T22:59:38.021 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:59:38.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f376f96a640 1 -- 172.21.15.152:0/3024274764 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f37500a2cf0 msgr2=0x7f37500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:38.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f376f96a640 1 --2- 172.21.15.152:0/3024274764 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f37500a2cf0 0x7f37500a51b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f37600096c0 tx=0x7f3760009260 comp rx=0 tx=0).stop 2024-09-11T22:59:38.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f376f96a640 1 -- 172.21.15.152:0/3024274764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37681027a0 msgr2=0x7f37681988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:38.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f376f96a640 1 --2- 172.21.15.152:0/3024274764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37681027a0 0x7f37681988b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f375c00c920 tx=0x7f375c00cde0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f376f96a640 1 -- 172.21.15.152:0/3024274764 shutdown_connections 2024-09-11T22:59:38.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f376f96a640 1 --2- 172.21.15.152:0/3024274764 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f37500a2cf0 0x7f37500a51b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f376f96a640 1 --2- 172.21.15.152:0/3024274764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3768106f20 0x7f376819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f376f96a640 1 --2- 172.21.15.152:0/3024274764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f37681027a0 0x7f37681988b0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f376f96a640 1 --2- 172.21.15.152:0/3024274764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3768101df0 0x7f3768198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:38.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f376f96a640 1 -- 172.21.15.152:0/3024274764 >> 172.21.15.152:0/3024274764 conn(0x7f37680fd8f0 msgr2=0x7f37680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:38.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f376f96a640 1 -- 172.21.15.152:0/3024274764 shutdown_connections 2024-09-11T22:59:38.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:38.011+0000 7f376f96a640 1 -- 172.21.15.152:0/3024274764 wait complete. 2024-09-11T22:59:38.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:38 smithi152 ceph-mon[31181]: pgmap v205: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:38.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:38 smithi191 ceph-mon[35211]: pgmap v205: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:38.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:38 smithi204 ceph-mon[35454]: pgmap v205: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:38.934 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:59:39.184 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:39.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.755+0000 7f984e832640 1 -- 172.21.15.152:0/2111731809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9848101de0 msgr2=0x7f98481021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:39.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.755+0000 7f984e832640 1 --2- 172.21.15.152:0/2111731809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9848101de0 0x7f98481021c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9838009900 tx=0x7f983802a410 comp rx=0 tx=0).stop 2024-09-11T22:59:39.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.755+0000 7f984e832640 1 -- 172.21.15.152:0/2111731809 shutdown_connections 2024-09-11T22:59:39.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.755+0000 7f984e832640 1 --2- 172.21.15.152:0/2111731809 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9848106f10 0x7f984810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.755+0000 7f984e832640 1 --2- 172.21.15.152:0/2111731809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9848102790 0x7f98481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.755+0000 7f984e832640 1 --2- 172.21.15.152:0/2111731809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9848101de0 0x7f98481021c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.755+0000 7f984e832640 1 -- 172.21.15.152:0/2111731809 >> 172.21.15.152:0/2111731809 conn(0x7f98480fd8c0 msgr2=0x7f98480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:39.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.755+0000 7f984e832640 1 -- 172.21.15.152:0/2111731809 shutdown_connections 2024-09-11T22:59:39.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.755+0000 7f984e832640 1 -- 172.21.15.152:0/2111731809 wait complete. 2024-09-11T22:59:39.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.755+0000 7f984e832640 1 Processor -- start 2024-09-11T22:59:39.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f984e832640 1 -- start start 2024-09-11T22:59:39.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f984e832640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9848101de0 0x7f98481984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f984e832640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9848102790 0x7f9848198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f984e832640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9848106f10 0x7f984819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f984e832640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9848111f90 con 0x7f9848106f10 2024-09-11T22:59:39.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f984e832640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9848111e10 con 0x7f9848102790 2024-09-11T22:59:39.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f984e832640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9848112110 con 0x7f9848101de0 2024-09-11T22:59:39.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f984cdcf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9848106f10 0x7f984819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f984cdcf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9848106f10 0x7f984819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38540/0 (socket says 172.21.15.152:38540) 2024-09-11T22:59:39.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f984cdcf640 1 -- 172.21.15.152:0/752630449 learned_addr learned my addr 172.21.15.152:0/752630449 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:39.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f9847fff640 1 --2- 172.21.15.152:0/752630449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9848101de0 0x7f98481984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f98477fe640 1 --2- 172.21.15.152:0/752630449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9848102790 0x7f9848198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f984cdcf640 1 -- 172.21.15.152:0/752630449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9848101de0 msgr2=0x7f98481984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:39.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f984cdcf640 1 --2- 172.21.15.152:0/752630449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9848101de0 0x7f98481984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f984cdcf640 1 -- 172.21.15.152:0/752630449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9848102790 msgr2=0x7f9848198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:39.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f984cdcf640 1 --2- 172.21.15.152:0/752630449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9848102790 0x7f9848198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f984cdcf640 1 -- 172.21.15.152:0/752630449 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984819d480 con 0x7f9848106f10 2024-09-11T22:59:39.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.756+0000 7f98477fe640 1 --2- 172.21.15.152:0/752630449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9848102790 0x7f9848198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:59:39.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.757+0000 7f984cdcf640 1 --2- 172.21.15.152:0/752630449 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9848106f10 0x7f984819cda0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f984000cad0 tx=0x7f984000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:39.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.757+0000 7f98457fa640 1 -- 172.21.15.152:0/752630449 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984000ae20 con 0x7f9848106f10 2024-09-11T22:59:39.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.757+0000 7f984e832640 1 -- 172.21.15.152:0/752630449 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984819d770 con 0x7f9848106f10 2024-09-11T22:59:39.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.757+0000 7f98457fa640 1 -- 172.21.15.152:0/752630449 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9840005020 con 0x7f9848106f10 2024-09-11T22:59:39.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.757+0000 7f98457fa640 1 -- 172.21.15.152:0/752630449 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984000f6f0 con 0x7f9848106f10 2024-09-11T22:59:39.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.757+0000 7f984e832640 1 -- 172.21.15.152:0/752630449 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98481a5050 con 0x7f9848106f10 2024-09-11T22:59:39.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.759+0000 7f98457fa640 1 -- 172.21.15.152:0/752630449 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f98400051d0 con 0x7f9848106f10 2024-09-11T22:59:39.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.759+0000 7f9850c0b740 1 Processor -- start 2024-09-11T22:59:39.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.759+0000 7f9850c0b740 1 -- start start 2024-09-11T22:59:39.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.759+0000 7f9850c0b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacdb40 0x55d6bcacc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.759+0000 7f9850c0b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcacc5d0 0x55d6bcacca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.759+0000 7f9850c0b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcadb780 0x55d6bcaddb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.759+0000 7f9850c0b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d6bca322f0 con 0x55d6bcacc5d0 2024-09-11T22:59:39.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.759+0000 7f9850c0b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d6bc9d9cf0 con 0x55d6bcacdb40 2024-09-11T22:59:39.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.759+0000 7f9850c0b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d6bca2e2b0 con 0x55d6bcadb780 2024-09-11T22:59:39.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f9847fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcacc5d0 0x55d6bcacca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f9847fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcacc5d0 0x55d6bcacca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38542/0 (socket says 172.21.15.152:38542) 2024-09-11T22:59:39.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f9847fff640 1 -- 172.21.15.152:0/1659266002 learned_addr learned my addr 172.21.15.152:0/1659266002 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:39.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f98477fe640 1 --2- 172.21.15.152:0/1659266002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacdb40 0x55d6bcacc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f98457fa640 1 --2- 172.21.15.152:0/752630449 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f98300a2cf0 0x7f98300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f9847fff640 1 -- 172.21.15.152:0/1659266002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcadb780 msgr2=0x55d6bcaddb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:39.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f98477fe640 1 --2- 172.21.15.152:0/1659266002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcadb780 0x55d6bcaddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f9847fff640 1 --2- 172.21.15.152:0/1659266002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcadb780 0x55d6bcaddb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f9847fff640 1 -- 172.21.15.152:0/1659266002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacdb40 msgr2=0x55d6bcacc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:39.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f9847fff640 1 --2- 172.21.15.152:0/1659266002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacdb40 0x55d6bcacc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f9847fff640 1 -- 172.21.15.152:0/1659266002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6bc9be3a0 con 0x55d6bcacc5d0 2024-09-11T22:59:39.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f984cdcf640 1 --2- 172.21.15.152:0/752630449 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f98300a2cf0 0x7f98300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f98457fa640 1 -- 172.21.15.152:0/752630449 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f98400cf550 con 0x7f9848106f10 2024-09-11T22:59:39.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f9847fff640 1 --2- 172.21.15.152:0/1659266002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcacc5d0 0x55d6bcacca10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f9838005ab0 tx=0x7f9838031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8318364a48c9e167 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:39.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f982e7fc640 1 -- 172.21.15.152:0/1659266002 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983803f070 con 0x55d6bcacc5d0 2024-09-11T22:59:39.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f982e7fc640 1 -- 172.21.15.152:0/1659266002 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9838031a20 con 0x55d6bcacc5d0 2024-09-11T22:59:39.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f982e7fc640 1 -- 172.21.15.152:0/1659266002 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838035070 con 0x55d6bcacc5d0 2024-09-11T22:59:39.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.760+0000 7f984cdcf640 1 --2- 172.21.15.152:0/752630449 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f98300a2cf0 0x7f98300a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f98400a4200 tx=0x7f98400c6650 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:39.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.761+0000 7f9850c0b740 1 -- 172.21.15.152:0/1659266002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcacc5d0 msgr2=0x55d6bcacca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:39.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.761+0000 7f9850c0b740 1 --2- 172.21.15.152:0/1659266002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcacc5d0 0x55d6bcacca10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f9838005ab0 tx=0x7f9838031630 comp rx=0 tx=0).stop 2024-09-11T22:59:39.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.761+0000 7f9850c0b740 1 -- 172.21.15.152:0/1659266002 shutdown_connections 2024-09-11T22:59:39.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.761+0000 7f9850c0b740 1 --2- 172.21.15.152:0/1659266002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcadb780 0x55d6bcaddb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.761+0000 7f9850c0b740 1 --2- 172.21.15.152:0/1659266002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcacc5d0 0x55d6bcacca10 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.761+0000 7f9850c0b740 1 --2- 172.21.15.152:0/1659266002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacdb40 0x55d6bcacc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.761+0000 7f9850c0b740 1 -- 172.21.15.152:0/1659266002 >> 172.21.15.152:0/1659266002 conn(0x55d6bca95310 msgr2=0x55d6bcac8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:39.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.761+0000 7f9850c0b740 1 -- 172.21.15.152:0/1659266002 shutdown_connections 2024-09-11T22:59:39.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.761+0000 7f9850c0b740 1 -- 172.21.15.152:0/1659266002 wait complete. 2024-09-11T22:59:39.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.762+0000 7f9850c0b740 1 -- start start 2024-09-11T22:59:39.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.762+0000 7f9850c0b740 1 Processor -- start 2024-09-11T22:59:39.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.762+0000 7f9850c0b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcacc5d0 0x55d6bcadb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.762+0000 7f9850c0b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacdb40 0x55d6bcad83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.762+0000 7f9850c0b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcadb780 0x55d6bcad8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.762+0000 7f9850c0b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d6bcade420 con 0x55d6bcadb780 2024-09-11T22:59:39.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.762+0000 7f9850c0b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d6bc9ed060 con 0x55d6bcacdb40 2024-09-11T22:59:39.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.762+0000 7f9850c0b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d6bcade5a0 con 0x55d6bcacc5d0 2024-09-11T22:59:39.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.762+0000 7f9850c0b740 10 client.0 _ll_register_callbacks cb 0x55d6bc7408c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:39.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.762+0000 7f98477fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcadb780 0x55d6bcad8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.762+0000 7f98477fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcadb780 0x55d6bcad8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38548/0 (socket says 172.21.15.152:38548) 2024-09-11T22:59:39.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.762+0000 7f98477fe640 1 -- 172.21.15.152:0/3826082885 learned_addr learned my addr 172.21.15.152:0/3826082885 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:39.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.762+0000 7f98477fe640 1 --2- 172.21.15.152:0/3826082885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcacc5d0 0x55d6bcadb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.762+0000 7f9847fff640 1 --2- 172.21.15.152:0/3826082885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacdb40 0x55d6bcad83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.762+0000 7f98477fe640 1 -- 172.21.15.152:0/3826082885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcacc5d0 msgr2=0x55d6bcadb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:39.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.762+0000 7f98477fe640 1 --2- 172.21.15.152:0/3826082885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcacc5d0 0x55d6bcadb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f98477fe640 1 -- 172.21.15.152:0/3826082885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacdb40 msgr2=0x55d6bcad83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:39.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f98477fe640 1 --2- 172.21.15.152:0/3826082885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacdb40 0x55d6bcad83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f98477fe640 1 -- 172.21.15.152:0/3826082885 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6bc7a33c0 con 0x55d6bcadb780 2024-09-11T22:59:39.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f9847fff640 1 --2- 172.21.15.152:0/3826082885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacdb40 0x55d6bcad83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:39.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f98477fe640 1 --2- 172.21.15.152:0/3826082885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcadb780 0x55d6bcad8900 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f983c018c60 tx=0x7f983c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d5d00c9b45408e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:39.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f98277fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:39.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f98277fe640 1 -- 172.21.15.152:0/3826082885 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c00f040 con 0x55d6bcadb780 2024-09-11T22:59:39.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f98277fe640 1 -- 172.21.15.152:0/3826082885 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f983c018230 con 0x55d6bcadb780 2024-09-11T22:59:39.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f9850c0b740 1 -- 172.21.15.152:0/3826082885 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9838031a20 con 0x55d6bcadb780 2024-09-11T22:59:39.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f98277fe640 1 -- 172.21.15.152:0/3826082885 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f983c011af0 con 0x55d6bcadb780 2024-09-11T22:59:39.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f9850c0b740 10 client.17250 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:39.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f9850c0b740 1 -- 172.21.15.152:0/3826082885 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d6bcae9ee0 con 0x55d6bcadb780 2024-09-11T22:59:39.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f98277fe640 1 -- 172.21.15.152:0/3826082885 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9838031a20 con 0x55d6bcadb780 2024-09-11T22:59:39.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f98277fe640 1 -- 172.21.15.152:0/3826082885 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f983c015ba0 con 0x55d6bcadb780 2024-09-11T22:59:39.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f9850c0b740 10 client.17250 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:39.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f9850c0b740 10 client.17250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:59:39.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.763+0000 7f9850c0b740 20 client.17250 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:39.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.764+0000 7f9850c0b740 1 --2- 172.21.15.152:0/3826082885 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d6bcaeb3a0 0x55d6bcaed790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.764+0000 7f9850c0b740 4 client.17250 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:59:39.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.764+0000 7f9850c0b740 1 -- 172.21.15.152:0/3826082885 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6bca3f860 con 0x55d6bcaeb3a0 2024-09-11T22:59:39.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.764+0000 7f9847fff640 1 --2- 172.21.15.152:0/3826082885 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d6bcaeb3a0 0x55d6bcaed790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.764+0000 7f9847fff640 1 --2- 172.21.15.152:0/3826082885 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d6bcaeb3a0 0x55d6bcaed790 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61ca41e84432719d server_cookie=bdea92176e93114a in_seq=0 out_seq=0 2024-09-11T22:59:39.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.764+0000 7f98277fe640 10 client.17250 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:39.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.765+0000 7f98277fe640 1 -- 172.21.15.152:0/3826082885 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d6bca3f860 con 0x55d6bcaeb3a0 2024-09-11T22:59:39.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.765+0000 7f98277fe640 10 client.17250 handle_command_reply: tid=0 2024-09-11T22:59:39.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.765+0000 7f9850c0b740 1 client.17250 shutdown 2024-09-11T22:59:39.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.765+0000 7f9850c0b740 1 -- 172.21.15.152:0/3826082885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcadb780 msgr2=0x55d6bcad8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:39.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.765+0000 7f9850c0b740 1 --2- 172.21.15.152:0/3826082885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcadb780 0x55d6bcad8900 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f983c018c60 tx=0x7f983c0046c0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.765+0000 7f9850c0b740 1 -- 172.21.15.152:0/3826082885 shutdown_connections 2024-09-11T22:59:39.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.765+0000 7f9850c0b740 1 --2- 172.21.15.152:0/3826082885 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcadb780 0x55d6bcad8900 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.765+0000 7f9850c0b740 1 --2- 172.21.15.152:0/3826082885 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d6bcaeb3a0 0x55d6bcaed790 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.765+0000 7f98277fe640 0 client.17250 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:39.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.765+0000 7f9850c0b740 1 --2- 172.21.15.152:0/3826082885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacdb40 0x55d6bcad83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.765+0000 7f9850c0b740 1 --2- 172.21.15.152:0/3826082885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcacc5d0 0x55d6bcadb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.765+0000 7f9850c0b740 1 -- 172.21.15.152:0/3826082885 >> 172.21.15.152:0/3826082885 conn(0x55d6bca95310 msgr2=0x55d6bcace930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:39.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.766+0000 7f9850c0b740 1 -- 172.21.15.152:0/3826082885 shutdown_connections 2024-09-11T22:59:39.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.766+0000 7f9850c0b740 1 -- 172.21.15.152:0/3826082885 wait complete. 2024-09-11T22:59:39.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.766+0000 7f9850c0b740 20 client.17250 trim_cache size 0 max 16384 2024-09-11T22:59:39.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.778+0000 7f9850c0b740 1 Processor -- start 2024-09-11T22:59:39.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.778+0000 7f9850c0b740 1 -- start start 2024-09-11T22:59:39.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.778+0000 7f9850c0b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcacc5d0 0x55d6bcae76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.778+0000 7f9850c0b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacdb40 0x55d6bcadcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.778+0000 7f9850c0b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcadb780 0x55d6bcadd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f9850c0b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f98481ab520 con 0x55d6bcadb780 2024-09-11T22:59:39.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f9850c0b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f98481ad060 con 0x55d6bcacdb40 2024-09-11T22:59:39.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f9850c0b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f98481abbf0 con 0x55d6bcacc5d0 2024-09-11T22:59:39.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f98477fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcadb780 0x55d6bcadd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f98477fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcadb780 0x55d6bcadd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38556/0 (socket says 172.21.15.152:38556) 2024-09-11T22:59:39.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f98477fe640 1 -- 172.21.15.152:0/2239623539 learned_addr learned my addr 172.21.15.152:0/2239623539 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:39.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f9847fff640 1 --2- 172.21.15.152:0/2239623539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacdb40 0x55d6bcadcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f98477fe640 1 --2- 172.21.15.152:0/2239623539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcacc5d0 0x55d6bcae76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f98477fe640 1 -- 172.21.15.152:0/2239623539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcacc5d0 msgr2=0x55d6bcae76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:39.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f98477fe640 1 --2- 172.21.15.152:0/2239623539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcacc5d0 0x55d6bcae76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f98477fe640 1 -- 172.21.15.152:0/2239623539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacdb40 msgr2=0x55d6bcadcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:39.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f98477fe640 1 --2- 172.21.15.152:0/2239623539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacdb40 0x55d6bcadcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f98477fe640 1 -- 172.21.15.152:0/2239623539 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6bcac9730 con 0x55d6bcadb780 2024-09-11T22:59:39.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f98477fe640 1 --2- 172.21.15.152:0/2239623539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcadb780 0x55d6bcadd0b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f983c0185a0 tx=0x7f983c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa887f1044b66a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:39.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f982dffb640 1 -- 172.21.15.152:0/2239623539 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c030070 con 0x55d6bcadb780 2024-09-11T22:59:39.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f982dffb640 1 -- 172.21.15.152:0/2239623539 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983c00ca50 con 0x55d6bcadb780 2024-09-11T22:59:39.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.779+0000 7f982dffb640 1 -- 172.21.15.152:0/2239623539 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c017ed0 con 0x55d6bcadb780 2024-09-11T22:59:39.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.780+0000 7f9850c0b740 1 -- 172.21.15.152:0/2239623539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcadb780 msgr2=0x55d6bcadd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:39.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.780+0000 7f9850c0b740 1 --2- 172.21.15.152:0/2239623539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcadb780 0x55d6bcadd0b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f983c0185a0 tx=0x7f983c00cf60 comp rx=0 tx=0).stop 2024-09-11T22:59:39.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.780+0000 7f9850c0b740 1 -- 172.21.15.152:0/2239623539 shutdown_connections 2024-09-11T22:59:39.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.780+0000 7f9850c0b740 1 --2- 172.21.15.152:0/2239623539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcadb780 0x55d6bcadd0b0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.780+0000 7f9850c0b740 1 --2- 172.21.15.152:0/2239623539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacdb40 0x55d6bcadcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.780+0000 7f9850c0b740 1 --2- 172.21.15.152:0/2239623539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcacc5d0 0x55d6bcae76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.780+0000 7f9850c0b740 1 -- 172.21.15.152:0/2239623539 >> 172.21.15.152:0/2239623539 conn(0x55d6bca95310 msgr2=0x55d6bcad6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:39.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.780+0000 7f9850c0b740 1 -- 172.21.15.152:0/2239623539 shutdown_connections 2024-09-11T22:59:39.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.780+0000 7f9850c0b740 1 -- 172.21.15.152:0/2239623539 wait complete. 2024-09-11T22:59:39.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.780+0000 7f9850c0b740 1 -- start start 2024-09-11T22:59:39.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9850c0b740 1 Processor -- start 2024-09-11T22:59:39.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9850c0b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacc5d0 0x55d6bcad5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9850c0b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcacdb40 0x55d6bcae3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9850c0b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcadb780 0x55d6bcae3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.785 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9850c0b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f98481ac990 con 0x55d6bcacdb40 2024-09-11T22:59:39.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9850c0b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f98481ac2c0 con 0x55d6bcacc5d0 2024-09-11T22:59:39.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9850c0b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f98481a9500 con 0x55d6bcadb780 2024-09-11T22:59:39.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9850c0b740 10 client.0 _ll_register_callbacks cb 0x7f983c00c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:39.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9847fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcacdb40 0x55d6bcae3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9847fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcacdb40 0x55d6bcae3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38572/0 (socket says 172.21.15.152:38572) 2024-09-11T22:59:39.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9847fff640 1 -- 172.21.15.152:0/961204529 learned_addr learned my addr 172.21.15.152:0/961204529 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:39.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f98477fe640 1 --2- 172.21.15.152:0/961204529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacc5d0 0x55d6bcad5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f98477fe640 1 --2- 172.21.15.152:0/961204529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcadb780 0x55d6bcae3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9847fff640 1 -- 172.21.15.152:0/961204529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcadb780 msgr2=0x55d6bcae3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:39.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9847fff640 1 --2- 172.21.15.152:0/961204529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcadb780 0x55d6bcae3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9847fff640 1 -- 172.21.15.152:0/961204529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacc5d0 msgr2=0x55d6bcad5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:39.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9847fff640 1 --2- 172.21.15.152:0/961204529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacc5d0 0x55d6bcad5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9847fff640 1 -- 172.21.15.152:0/961204529 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6bcae42e0 con 0x55d6bcacdb40 2024-09-11T22:59:39.786 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f9847fff640 1 --2- 172.21.15.152:0/961204529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcacdb40 0x55d6bcae3830 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f9838005ab0 tx=0x7f9838005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950e7c397fc3b98f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:39.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.781+0000 7f98277fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:39.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.782+0000 7f98277fe640 1 -- 172.21.15.152:0/961204529 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838035040 con 0x55d6bcacdb40 2024-09-11T22:59:39.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.782+0000 7f98277fe640 1 -- 172.21.15.152:0/961204529 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f98380047a0 con 0x55d6bcacdb40 2024-09-11T22:59:39.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.782+0000 7f9850c0b740 1 -- 172.21.15.152:0/961204529 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f983c00ca50 con 0x55d6bcacdb40 2024-09-11T22:59:39.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.782+0000 7f98277fe640 1 -- 172.21.15.152:0/961204529 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f983805c050 con 0x55d6bcacdb40 2024-09-11T22:59:39.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.782+0000 7f9850c0b740 10 client.17256 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:39.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.782+0000 7f9850c0b740 1 -- 172.21.15.152:0/961204529 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d6bcb55ff0 con 0x55d6bcacdb40 2024-09-11T22:59:39.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.782+0000 7f98277fe640 1 -- 172.21.15.152:0/961204529 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983c00ca50 con 0x55d6bcacdb40 2024-09-11T22:59:39.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.782+0000 7f98277fe640 1 -- 172.21.15.152:0/961204529 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f983804eab0 con 0x55d6bcacdb40 2024-09-11T22:59:39.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.782+0000 7f9850c0b740 10 client.17256 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:39.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.782+0000 7f9850c0b740 10 client.17256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:59:39.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.782+0000 7f9850c0b740 20 client.17256 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:39.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.782+0000 7f9850c0b740 1 --2- 172.21.15.152:0/961204529 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d6bcaeb3a0 0x55d6bcb58270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:39.787 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.782+0000 7f9850c0b740 4 client.17256 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:59:39.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.782+0000 7f9850c0b740 1 -- 172.21.15.152:0/961204529 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d6bca39d70 con 0x55d6bcaeb3a0 2024-09-11T22:59:39.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f98477fe640 1 --2- 172.21.15.152:0/961204529 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d6bcaeb3a0 0x55d6bcb58270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:39.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f98477fe640 1 --2- 172.21.15.152:0/961204529 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d6bcaeb3a0 0x55d6bcb58270 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c73122a038ef3f0f server_cookie=a72946b1f11e3e1c in_seq=0 out_seq=0 2024-09-11T22:59:39.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f98277fe640 10 client.17256 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:39.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f98277fe640 1 -- 172.21.15.152:0/961204529 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d6bca39d70 con 0x55d6bcaeb3a0 2024-09-11T22:59:39.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f98277fe640 10 client.17256 handle_command_reply: tid=0 2024-09-11T22:59:39.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f9850c0b740 1 client.17256 shutdown 2024-09-11T22:59:39.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f9850c0b740 1 -- 172.21.15.152:0/961204529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcacdb40 msgr2=0x55d6bcae3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:39.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f9850c0b740 1 --2- 172.21.15.152:0/961204529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcacdb40 0x55d6bcae3830 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f9838005ab0 tx=0x7f9838005630 comp rx=0 tx=0).stop 2024-09-11T22:59:39.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f9850c0b740 1 -- 172.21.15.152:0/961204529 shutdown_connections 2024-09-11T22:59:39.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f9850c0b740 1 --2- 172.21.15.152:0/961204529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d6bcadb780 0x55d6bcae3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f9850c0b740 1 --2- 172.21.15.152:0/961204529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6bcacdb40 0x55d6bcae3830 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.788 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f9850c0b740 1 --2- 172.21.15.152:0/961204529 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d6bcaeb3a0 0x55d6bcb58270 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f9850c0b740 1 --2- 172.21.15.152:0/961204529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d6bcacc5d0 0x55d6bcad5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f98277fe640 0 client.17256 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:39.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f9850c0b740 1 -- 172.21.15.152:0/961204529 >> 172.21.15.152:0/961204529 conn(0x55d6bca95310 msgr2=0x55d6bcad70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:39.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f9850c0b740 1 -- 172.21.15.152:0/961204529 shutdown_connections 2024-09-11T22:59:39.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f9850c0b740 1 -- 172.21.15.152:0/961204529 wait complete. 2024-09-11T22:59:39.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.783+0000 7f9850c0b740 20 client.17256 trim_cache size 0 max 16384 2024-09-11T22:59:39.789 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:59:39.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.784+0000 7f984e832640 1 -- 172.21.15.152:0/752630449 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f98300a2cf0 msgr2=0x7f98300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:39.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.784+0000 7f984e832640 1 --2- 172.21.15.152:0/752630449 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f98300a2cf0 0x7f98300a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f98400a4200 tx=0x7f98400c6650 comp rx=0 tx=0).stop 2024-09-11T22:59:39.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.784+0000 7f984e832640 1 -- 172.21.15.152:0/752630449 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9848106f10 msgr2=0x7f984819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:39.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.784+0000 7f984e832640 1 --2- 172.21.15.152:0/752630449 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9848106f10 0x7f984819cda0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f984000cad0 tx=0x7f984000cf90 comp rx=0 tx=0).stop 2024-09-11T22:59:39.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.784+0000 7f984e832640 1 -- 172.21.15.152:0/752630449 shutdown_connections 2024-09-11T22:59:39.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.784+0000 7f984e832640 1 --2- 172.21.15.152:0/752630449 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f98300a2cf0 0x7f98300a51b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.784+0000 7f984e832640 1 --2- 172.21.15.152:0/752630449 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9848106f10 0x7f984819cda0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.784+0000 7f984e832640 1 --2- 172.21.15.152:0/752630449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9848102790 0x7f9848198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.784+0000 7f984e832640 1 --2- 172.21.15.152:0/752630449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9848101de0 0x7f98481984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:39.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.784+0000 7f984e832640 1 -- 172.21.15.152:0/752630449 >> 172.21.15.152:0/752630449 conn(0x7f98480fd8c0 msgr2=0x7f98480fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:39.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.784+0000 7f984e832640 1 -- 172.21.15.152:0/752630449 shutdown_connections 2024-09-11T22:59:39.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:39.784+0000 7f984e832640 1 -- 172.21.15.152:0/752630449 wait complete. 2024-09-11T22:59:40.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:40 smithi152 ceph-mon[31181]: pgmap v206: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-11T22:59:40.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:40 smithi191 ceph-mon[35211]: pgmap v206: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-11T22:59:40.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:40 smithi204 ceph-mon[35454]: pgmap v206: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-11T22:59:41.607 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:59:41.752 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:59:42.024 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:42.517 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:42 smithi152 ceph-mon[31181]: pgmap v207: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T22:59:42.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.605+0000 7fb739e57640 1 -- 172.21.15.152:0/1888274157 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb734102570 msgr2=0x7fb7341029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:42.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.605+0000 7fb739e57640 1 --2- 172.21.15.152:0/1888274157 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb734102570 0x7fb7341029b0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb724009900 tx=0x7fb72402a480 comp rx=0 tx=0).stop 2024-09-11T22:59:42.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.605+0000 7fb739e57640 1 -- 172.21.15.152:0/1888274157 shutdown_connections 2024-09-11T22:59:42.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.605+0000 7fb739e57640 1 --2- 172.21.15.152:0/1888274157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb734102ef0 0x7fb73410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.605+0000 7fb739e57640 1 --2- 172.21.15.152:0/1888274157 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb734102570 0x7fb7341029b0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.605+0000 7fb739e57640 1 --2- 172.21.15.152:0/1888274157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb734100e20 0x7fb734101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.605+0000 7fb739e57640 1 -- 172.21.15.152:0/1888274157 >> 172.21.15.152:0/1888274157 conn(0x7fb7340fc7d0 msgr2=0x7fb7340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:42.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.605+0000 7fb739e57640 1 -- 172.21.15.152:0/1888274157 shutdown_connections 2024-09-11T22:59:42.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.605+0000 7fb739e57640 1 -- 172.21.15.152:0/1888274157 wait complete. 2024-09-11T22:59:42.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.606+0000 7fb739e57640 1 Processor -- start 2024-09-11T22:59:42.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.606+0000 7fb739e57640 1 -- start start 2024-09-11T22:59:42.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.606+0000 7fb739e57640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb734100e20 0x7fb734198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.606+0000 7fb739e57640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb734102570 0x7fb734198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.606+0000 7fb739e57640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb734102ef0 0x7fb73419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.606+0000 7fb739e57640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb734110f80 con 0x7fb734102ef0 2024-09-11T22:59:42.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.606+0000 7fb739e57640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb734110e00 con 0x7fb734100e20 2024-09-11T22:59:42.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.606+0000 7fb739e57640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb734111100 con 0x7fb734102570 2024-09-11T22:59:42.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.606+0000 7fb733fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb734102ef0 0x7fb73419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.606+0000 7fb733fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb734102ef0 0x7fb73419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38596/0 (socket says 172.21.15.152:38596) 2024-09-11T22:59:42.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.606+0000 7fb733fff640 1 -- 172.21.15.152:0/2607185272 learned_addr learned my addr 172.21.15.152:0/2607185272 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:42.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.606+0000 7fb732ffd640 1 --2- 172.21.15.152:0/2607185272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb734102570 0x7fb734198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.606+0000 7fb7337fe640 1 --2- 172.21.15.152:0/2607185272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb734100e20 0x7fb734198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.607+0000 7fb733fff640 1 -- 172.21.15.152:0/2607185272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb734102570 msgr2=0x7fb734198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:42.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.607+0000 7fb733fff640 1 --2- 172.21.15.152:0/2607185272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb734102570 0x7fb734198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.607+0000 7fb733fff640 1 -- 172.21.15.152:0/2607185272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb734100e20 msgr2=0x7fb734198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:42.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.607+0000 7fb733fff640 1 --2- 172.21.15.152:0/2607185272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb734100e20 0x7fb734198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.607+0000 7fb733fff640 1 -- 172.21.15.152:0/2607185272 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73419d500 con 0x7fb734102ef0 2024-09-11T22:59:42.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.607+0000 7fb733fff640 1 --2- 172.21.15.152:0/2607185272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb734102ef0 0x7fb73419ce20 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00eb10 tx=0x7fb72c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:42.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.607+0000 7fb730ff9640 1 -- 172.21.15.152:0/2607185272 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c004b20 con 0x7fb734102ef0 2024-09-11T22:59:42.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.607+0000 7fb730ff9640 1 -- 172.21.15.152:0/2607185272 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb72c004cc0 con 0x7fb734102ef0 2024-09-11T22:59:42.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.607+0000 7fb739e57640 1 -- 172.21.15.152:0/2607185272 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73419d7f0 con 0x7fb734102ef0 2024-09-11T22:59:42.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.607+0000 7fb730ff9640 1 -- 172.21.15.152:0/2607185272 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c010760 con 0x7fb734102ef0 2024-09-11T22:59:42.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.607+0000 7fb739e57640 1 -- 172.21.15.152:0/2607185272 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7341a50d0 con 0x7fb734102ef0 2024-09-11T22:59:42.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.609+0000 7fb730ff9640 1 -- 172.21.15.152:0/2607185272 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb72c005020 con 0x7fb734102ef0 2024-09-11T22:59:42.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.610+0000 7fb73c230740 1 Processor -- start 2024-09-11T22:59:42.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.610+0000 7fb730ff9640 1 --2- 172.21.15.152:0/2607185272 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb71c0a2cf0 0x7fb71c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.610+0000 7fb73c230740 1 -- start start 2024-09-11T22:59:42.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.610+0000 7fb732ffd640 1 --2- 172.21.15.152:0/2607185272 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb71c0a2cf0 0x7fb71c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.610+0000 7fb73c230740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd18db60 0x5564cd18df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.610+0000 7fb730ff9640 1 -- 172.21.15.152:0/2607185272 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb72c0cf9e0 con 0x7fb734102ef0 2024-09-11T22:59:42.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.610+0000 7fb73c230740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd18e480 0x5564cd198800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.610+0000 7fb73c230740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd198d40 0x5564cd19b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.610+0000 7fb73c230740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5564cd0f3280 con 0x5564cd198d40 2024-09-11T22:59:42.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.610+0000 7fb73c230740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5564cd09ab10 con 0x5564cd18e480 2024-09-11T22:59:42.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.610+0000 7fb73c230740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5564cd0ef240 con 0x5564cd18db60 2024-09-11T22:59:42.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.610+0000 7fb733fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd198d40 0x5564cd19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.610+0000 7fb733fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd198d40 0x5564cd19b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38600/0 (socket says 172.21.15.152:38600) 2024-09-11T22:59:42.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.610+0000 7fb733fff640 1 -- 172.21.15.152:0/3615699117 learned_addr learned my addr 172.21.15.152:0/3615699117 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:42.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.610+0000 7fb7337fe640 1 --2- 172.21.15.152:0/3615699117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd18db60 0x5564cd18df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.610+0000 7fb732ffd640 1 --2- 172.21.15.152:0/3615699117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd18e480 0x5564cd198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.611+0000 7fb733fff640 1 -- 172.21.15.152:0/3615699117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd18db60 msgr2=0x5564cd18df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:42.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.611+0000 7fb733fff640 1 --2- 172.21.15.152:0/3615699117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd18db60 0x5564cd18df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.611+0000 7fb733fff640 1 -- 172.21.15.152:0/3615699117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd18e480 msgr2=0x5564cd198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:42.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.611+0000 7fb733fff640 1 --2- 172.21.15.152:0/3615699117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd18e480 0x5564cd198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.611+0000 7fb733fff640 1 -- 172.21.15.152:0/3615699117 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564cd0a71f0 con 0x5564cd198d40 2024-09-11T22:59:42.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.611+0000 7fb7337fe640 1 --2- 172.21.15.152:0/3615699117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd18db60 0x5564cd18df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:42.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.611+0000 7fb732ffd640 1 --2- 172.21.15.152:0/3615699117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd18e480 0x5564cd198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:42.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.611+0000 7fb732ffd640 1 --2- 172.21.15.152:0/2607185272 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb71c0a2cf0 0x7fb71c0a51b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb724009230 tx=0x7fb724005e50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:42.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.611+0000 7fb733fff640 1 --2- 172.21.15.152:0/3615699117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd198d40 0x5564cd19b130 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0a4650 tx=0x7fb72c09ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137800d1891bc6ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:42.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.611+0000 7fb719ffb640 1 -- 172.21.15.152:0/3615699117 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c09bf10 con 0x5564cd198d40 2024-09-11T22:59:42.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.611+0000 7fb719ffb640 1 -- 172.21.15.152:0/3615699117 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb72c0a02c0 con 0x5564cd198d40 2024-09-11T22:59:42.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.611+0000 7fb719ffb640 1 -- 172.21.15.152:0/3615699117 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c0a46f0 con 0x5564cd198d40 2024-09-11T22:59:42.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.612+0000 7fb73c230740 1 -- 172.21.15.152:0/3615699117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd198d40 msgr2=0x5564cd19b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:42.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.612+0000 7fb73c230740 1 --2- 172.21.15.152:0/3615699117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd198d40 0x5564cd19b130 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0a4650 tx=0x7fb72c09ba50 comp rx=0 tx=0).stop 2024-09-11T22:59:42.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.612+0000 7fb73c230740 1 -- 172.21.15.152:0/3615699117 shutdown_connections 2024-09-11T22:59:42.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.612+0000 7fb73c230740 1 --2- 172.21.15.152:0/3615699117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd198d40 0x5564cd19b130 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.612+0000 7fb73c230740 1 --2- 172.21.15.152:0/3615699117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd18e480 0x5564cd198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.612+0000 7fb73c230740 1 --2- 172.21.15.152:0/3615699117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd18db60 0x5564cd18df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.612+0000 7fb73c230740 1 -- 172.21.15.152:0/3615699117 >> 172.21.15.152:0/3615699117 conn(0x5564cd18a0f0 msgr2=0x5564cd18a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:42.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.612+0000 7fb73c230740 1 -- 172.21.15.152:0/3615699117 shutdown_connections 2024-09-11T22:59:42.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.612+0000 7fb73c230740 1 -- 172.21.15.152:0/3615699117 wait complete. 2024-09-11T22:59:42.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.612+0000 7fb73c230740 1 -- start start 2024-09-11T22:59:42.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb73c230740 1 Processor -- start 2024-09-11T22:59:42.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb73c230740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd18db60 0x5564cd195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb73c230740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd196130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb73c230740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd198d40 0x5564cd196670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb73c230740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5564cd19fd20 con 0x5564cd18e480 2024-09-11T22:59:42.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb73c230740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5564cd0adf10 con 0x5564cd18db60 2024-09-11T22:59:42.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb73c230740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5564cd19fea0 con 0x5564cd198d40 2024-09-11T22:59:42.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb73c230740 10 client.0 _ll_register_callbacks cb 0x5564cd076a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:42.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb732ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb732ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd196130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38612/0 (socket says 172.21.15.152:38612) 2024-09-11T22:59:42.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb732ffd640 1 -- 172.21.15.152:0/1610336784 learned_addr learned my addr 172.21.15.152:0/1610336784 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:42.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb7337fe640 1 --2- 172.21.15.152:0/1610336784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd18db60 0x5564cd195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb732ffd640 1 -- 172.21.15.152:0/1610336784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd198d40 msgr2=0x5564cd196670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:42.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb733fff640 1 --2- 172.21.15.152:0/1610336784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd198d40 0x5564cd196670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb732ffd640 1 --2- 172.21.15.152:0/1610336784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd198d40 0x5564cd196670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb732ffd640 1 -- 172.21.15.152:0/1610336784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd18db60 msgr2=0x5564cd195bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:42.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb732ffd640 1 --2- 172.21.15.152:0/1610336784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd18db60 0x5564cd195bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.613+0000 7fb732ffd640 1 -- 172.21.15.152:0/1610336784 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564cce646a0 con 0x5564cd18e480 2024-09-11T22:59:42.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.614+0000 7fb732ffd640 1 --2- 172.21.15.152:0/1610336784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd196130 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fb724009900 tx=0x7fb72402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe6d2cab698ed2f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:42.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.614+0000 7fb712ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:42.626 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.614+0000 7fb712ffd640 1 -- 172.21.15.152:0/1610336784 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724035050 con 0x5564cd18e480 2024-09-11T22:59:42.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.614+0000 7fb712ffd640 1 -- 172.21.15.152:0/1610336784 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb724031740 con 0x5564cd18e480 2024-09-11T22:59:42.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.614+0000 7fb73c230740 1 -- 172.21.15.152:0/1610336784 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb72c0a02c0 con 0x5564cd18e480 2024-09-11T22:59:42.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.614+0000 7fb712ffd640 1 -- 172.21.15.152:0/1610336784 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb724031940 con 0x5564cd18e480 2024-09-11T22:59:42.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.614+0000 7fb73c230740 10 client.17268 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:42.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.614+0000 7fb73c230740 1 -- 172.21.15.152:0/1610336784 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5564cd1ab8f0 con 0x5564cd18e480 2024-09-11T22:59:42.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.614+0000 7fb712ffd640 1 -- 172.21.15.152:0/1610336784 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb72c0a02c0 con 0x5564cd18e480 2024-09-11T22:59:42.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.614+0000 7fb712ffd640 1 -- 172.21.15.152:0/1610336784 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb72403c440 con 0x5564cd18e480 2024-09-11T22:59:42.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.614+0000 7fb73c230740 10 client.17268 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:42.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.614+0000 7fb73c230740 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:59:42.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.614+0000 7fb73c230740 20 client.17268 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:42.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.614+0000 7fb73c230740 1 --2- 172.21.15.152:0/1610336784 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5564cd1acd00 0x5564cd1af110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.615+0000 7fb73c230740 4 client.17268 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:59:42.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.615+0000 7fb73c230740 1 -- 172.21.15.152:0/1610336784 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564cd1007f0 con 0x5564cd1acd00 2024-09-11T22:59:42.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.615+0000 7fb7337fe640 1 --2- 172.21.15.152:0/1610336784 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5564cd1acd00 0x5564cd1af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.615+0000 7fb7337fe640 1 --2- 172.21.15.152:0/1610336784 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5564cd1acd00 0x5564cd1af110 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5cb6c2fa52d627d server_cookie=37e80c0be73e5c1d in_seq=0 out_seq=0 2024-09-11T22:59:42.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.615+0000 7fb712ffd640 10 client.17268 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:42.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.616+0000 7fb712ffd640 1 -- 172.21.15.152:0/1610336784 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564cd1007f0 con 0x5564cd1acd00 2024-09-11T22:59:42.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.616+0000 7fb712ffd640 10 client.17268 handle_command_reply: tid=0 2024-09-11T22:59:42.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.616+0000 7fb73c230740 1 client.17268 shutdown 2024-09-11T22:59:42.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.616+0000 7fb73c230740 1 -- 172.21.15.152:0/1610336784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 msgr2=0x5564cd196130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:42.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.616+0000 7fb73c230740 1 --2- 172.21.15.152:0/1610336784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd196130 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fb724009900 tx=0x7fb72402a480 comp rx=0 tx=0).stop 2024-09-11T22:59:42.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.616+0000 7fb73c230740 1 -- 172.21.15.152:0/1610336784 shutdown_connections 2024-09-11T22:59:42.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.616+0000 7fb73c230740 1 --2- 172.21.15.152:0/1610336784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd198d40 0x5564cd196670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.616+0000 7fb73c230740 1 --2- 172.21.15.152:0/1610336784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd196130 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.616+0000 7fb73c230740 1 --2- 172.21.15.152:0/1610336784 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5564cd1acd00 0x5564cd1af110 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.616+0000 7fb73c230740 1 --2- 172.21.15.152:0/1610336784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd18db60 0x5564cd195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.616+0000 7fb712ffd640 0 client.17268 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:42.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.616+0000 7fb73c230740 1 -- 172.21.15.152:0/1610336784 >> 172.21.15.152:0/1610336784 conn(0x5564cd18a0f0 msgr2=0x5564cd189270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:42.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.616+0000 7fb73c230740 1 -- 172.21.15.152:0/1610336784 shutdown_connections 2024-09-11T22:59:42.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.616+0000 7fb73c230740 1 -- 172.21.15.152:0/1610336784 wait complete. 2024-09-11T22:59:42.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.616+0000 7fb73c230740 20 client.17268 trim_cache size 0 max 16384 2024-09-11T22:59:42.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.634+0000 7fb73c230740 1 Processor -- start 2024-09-11T22:59:42.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.634+0000 7fb73c230740 1 -- start start 2024-09-11T22:59:42.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.634+0000 7fb73c230740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd18db60 0x5564cd18fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.634+0000 7fb73c230740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd1b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.634+0000 7fb73c230740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd198d40 0x5564cd1b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.634+0000 7fb73c230740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb7341ab5f0 con 0x5564cd18e480 2024-09-11T22:59:42.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.634+0000 7fb73c230740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb7341ad130 con 0x5564cd198d40 2024-09-11T22:59:42.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.634+0000 7fb73c230740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7341abcc0 con 0x5564cd18db60 2024-09-11T22:59:42.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.634+0000 7fb732ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd1b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.634+0000 7fb732ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd1b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38618/0 (socket says 172.21.15.152:38618) 2024-09-11T22:59:42.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.634+0000 7fb732ffd640 1 -- 172.21.15.152:0/488759490 learned_addr learned my addr 172.21.15.152:0/488759490 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:42.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.635+0000 7fb7337fe640 1 --2- 172.21.15.152:0/488759490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd18db60 0x5564cd18fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.635+0000 7fb733fff640 1 --2- 172.21.15.152:0/488759490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd198d40 0x5564cd1b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.635+0000 7fb732ffd640 1 -- 172.21.15.152:0/488759490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd18db60 msgr2=0x5564cd18fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:42.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.635+0000 7fb732ffd640 1 --2- 172.21.15.152:0/488759490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd18db60 0x5564cd18fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.635+0000 7fb732ffd640 1 -- 172.21.15.152:0/488759490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd198d40 msgr2=0x5564cd1b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:42.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.635+0000 7fb732ffd640 1 --2- 172.21.15.152:0/488759490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd198d40 0x5564cd1b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.635+0000 7fb732ffd640 1 -- 172.21.15.152:0/488759490 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564cd18e2b0 con 0x5564cd18e480 2024-09-11T22:59:42.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.635+0000 7fb732ffd640 1 --2- 172.21.15.152:0/488759490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd1b49e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fb72403c860 tx=0x7fb72403c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42e4fce883d2753 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:42.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.635+0000 7fb7197fa640 1 -- 172.21.15.152:0/488759490 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724061070 con 0x5564cd18e480 2024-09-11T22:59:42.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.635+0000 7fb7197fa640 1 -- 172.21.15.152:0/488759490 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb72404e070 con 0x5564cd18e480 2024-09-11T22:59:42.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.635+0000 7fb7197fa640 1 -- 172.21.15.152:0/488759490 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724035070 con 0x5564cd18e480 2024-09-11T22:59:42.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.636+0000 7fb73c230740 1 -- 172.21.15.152:0/488759490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 msgr2=0x5564cd1b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:42.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.636+0000 7fb73c230740 1 --2- 172.21.15.152:0/488759490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd1b49e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fb72403c860 tx=0x7fb72403c650 comp rx=0 tx=0).stop 2024-09-11T22:59:42.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.636+0000 7fb73c230740 1 -- 172.21.15.152:0/488759490 shutdown_connections 2024-09-11T22:59:42.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.636+0000 7fb73c230740 1 --2- 172.21.15.152:0/488759490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd198d40 0x5564cd1b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.636+0000 7fb73c230740 1 --2- 172.21.15.152:0/488759490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd1b49e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.636+0000 7fb73c230740 1 --2- 172.21.15.152:0/488759490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd18db60 0x5564cd18fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.636+0000 7fb73c230740 1 -- 172.21.15.152:0/488759490 >> 172.21.15.152:0/488759490 conn(0x5564cd18a0f0 msgr2=0x5564cd18f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:42.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.636+0000 7fb73c230740 1 -- 172.21.15.152:0/488759490 shutdown_connections 2024-09-11T22:59:42.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.636+0000 7fb73c230740 1 -- 172.21.15.152:0/488759490 wait complete. 2024-09-11T22:59:42.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.636+0000 7fb73c230740 1 -- start start 2024-09-11T22:59:42.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb73c230740 1 Processor -- start 2024-09-11T22:59:42.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb73c230740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd18db60 0x5564cd1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb73c230740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb73c230740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd198d40 0x5564cd1a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb73c230740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb7341aca60 con 0x5564cd18e480 2024-09-11T22:59:42.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb73c230740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb7341ac390 con 0x5564cd198d40 2024-09-11T22:59:42.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb73c230740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb7341a95b0 con 0x5564cd18db60 2024-09-11T22:59:42.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb73c230740 10 client.0 _ll_register_callbacks cb 0x7fb724004350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:42.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb732ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb732ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd1a12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38622/0 (socket says 172.21.15.152:38622) 2024-09-11T22:59:42.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb732ffd640 1 -- 172.21.15.152:0/3019618583 learned_addr learned my addr 172.21.15.152:0/3019618583 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:42.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb7337fe640 1 --2- 172.21.15.152:0/3019618583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd18db60 0x5564cd1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb733fff640 1 --2- 172.21.15.152:0/3019618583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd198d40 0x5564cd1a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb732ffd640 1 -- 172.21.15.152:0/3019618583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd18db60 msgr2=0x5564cd1a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:42.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb732ffd640 1 --2- 172.21.15.152:0/3019618583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd18db60 0x5564cd1a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb732ffd640 1 -- 172.21.15.152:0/3019618583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd198d40 msgr2=0x5564cd1a17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:42.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb732ffd640 1 --2- 172.21.15.152:0/3019618583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd198d40 0x5564cd1a17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb732ffd640 1 -- 172.21.15.152:0/3019618583 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564cd2169b0 con 0x5564cd18e480 2024-09-11T22:59:42.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb732ffd640 1 --2- 172.21.15.152:0/3019618583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd1a12a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb724005ec0 tx=0x7fb724031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2392f6877ca34175 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:42.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.637+0000 7fb712ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:42.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.638+0000 7fb712ffd640 1 -- 172.21.15.152:0/3019618583 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb724061070 con 0x5564cd18e480 2024-09-11T22:59:42.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.638+0000 7fb712ffd640 1 -- 172.21.15.152:0/3019618583 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb724005040 con 0x5564cd18e480 2024-09-11T22:59:42.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.638+0000 7fb73c230740 1 -- 172.21.15.152:0/3019618583 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb72404e070 con 0x5564cd18e480 2024-09-11T22:59:42.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.638+0000 7fb712ffd640 1 -- 172.21.15.152:0/3019618583 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb724035040 con 0x5564cd18e480 2024-09-11T22:59:42.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.638+0000 7fb73c230740 10 client.17274 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:42.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.638+0000 7fb73c230740 1 -- 172.21.15.152:0/3019618583 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5564cd217bf0 con 0x5564cd18e480 2024-09-11T22:59:42.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.638+0000 7fb712ffd640 1 -- 172.21.15.152:0/3019618583 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb72404e070 con 0x5564cd18e480 2024-09-11T22:59:42.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.638+0000 7fb712ffd640 1 -- 172.21.15.152:0/3019618583 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb724004a80 con 0x5564cd18e480 2024-09-11T22:59:42.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.638+0000 7fb73c230740 10 client.17274 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:42.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.638+0000 7fb73c230740 10 client.17274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:59:42.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.638+0000 7fb73c230740 20 client.17274 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:42.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.638+0000 7fb73c230740 1 --2- 172.21.15.152:0/3019618583 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5564cd1acd00 0x5564cd21a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:42.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.638+0000 7fb73c230740 4 client.17274 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:59:42.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.638+0000 7fb73c230740 1 -- 172.21.15.152:0/3019618583 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5564cd0fad00 con 0x5564cd1acd00 2024-09-11T22:59:42.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.639+0000 7fb7337fe640 1 --2- 172.21.15.152:0/3019618583 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5564cd1acd00 0x5564cd21a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:42.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.639+0000 7fb7337fe640 1 --2- 172.21.15.152:0/3019618583 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5564cd1acd00 0x5564cd21a080 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2154bb0ffe7d26 server_cookie=b35435496d9ec8e2 in_seq=0 out_seq=0 2024-09-11T22:59:42.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.639+0000 7fb712ffd640 10 client.17274 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:42.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.640+0000 7fb712ffd640 1 -- 172.21.15.152:0/3019618583 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5564cd0fad00 con 0x5564cd1acd00 2024-09-11T22:59:42.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.640+0000 7fb712ffd640 10 client.17274 handle_command_reply: tid=0 2024-09-11T22:59:42.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.640+0000 7fb73c230740 1 client.17274 shutdown 2024-09-11T22:59:42.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.640+0000 7fb73c230740 1 -- 172.21.15.152:0/3019618583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 msgr2=0x5564cd1a12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:42.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.640+0000 7fb73c230740 1 --2- 172.21.15.152:0/3019618583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd1a12a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb724005ec0 tx=0x7fb724031770 comp rx=0 tx=0).stop 2024-09-11T22:59:42.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.640+0000 7fb73c230740 1 -- 172.21.15.152:0/3019618583 shutdown_connections 2024-09-11T22:59:42.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.640+0000 7fb73c230740 1 --2- 172.21.15.152:0/3019618583 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5564cd1acd00 0x5564cd21a080 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.640+0000 7fb73c230740 1 --2- 172.21.15.152:0/3019618583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564cd198d40 0x5564cd1a17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.640+0000 7fb73c230740 1 --2- 172.21.15.152:0/3019618583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564cd18e480 0x5564cd1a12a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.640+0000 7fb73c230740 1 --2- 172.21.15.152:0/3019618583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564cd18db60 0x5564cd1a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.640+0000 7fb73c230740 1 -- 172.21.15.152:0/3019618583 >> 172.21.15.152:0/3019618583 conn(0x5564cd18a0f0 msgr2=0x5564cd1963d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:42.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.640+0000 7fb712ffd640 0 client.17274 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:42.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.640+0000 7fb73c230740 1 -- 172.21.15.152:0/3019618583 shutdown_connections 2024-09-11T22:59:42.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.640+0000 7fb73c230740 1 -- 172.21.15.152:0/3019618583 wait complete. 2024-09-11T22:59:42.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.641+0000 7fb73c230740 20 client.17274 trim_cache size 0 max 16384 2024-09-11T22:59:42.646 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:59:42.646 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T22:59:42.646 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "92b8d226-caa2-4770-8494-e9357f43ab29", 2024-09-11T22:59:42.646 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T22:59:42.646 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:59:42.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.641+0000 7fb739e57640 1 -- 172.21.15.152:0/2607185272 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb71c0a2cf0 msgr2=0x7fb71c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:42.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.641+0000 7fb739e57640 1 --2- 172.21.15.152:0/2607185272 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb71c0a2cf0 0x7fb71c0a51b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb724009230 tx=0x7fb724005e50 comp rx=0 tx=0).stop 2024-09-11T22:59:42.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.641+0000 7fb739e57640 1 -- 172.21.15.152:0/2607185272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb734102ef0 msgr2=0x7fb73419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:42.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.641+0000 7fb739e57640 1 --2- 172.21.15.152:0/2607185272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb734102ef0 0x7fb73419ce20 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00eb10 tx=0x7fb72c00c2d0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.641+0000 7fb739e57640 1 -- 172.21.15.152:0/2607185272 shutdown_connections 2024-09-11T22:59:42.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.641+0000 7fb739e57640 1 --2- 172.21.15.152:0/2607185272 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb71c0a2cf0 0x7fb71c0a51b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.641+0000 7fb739e57640 1 --2- 172.21.15.152:0/2607185272 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb734102ef0 0x7fb73419ce20 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.641+0000 7fb739e57640 1 --2- 172.21.15.152:0/2607185272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb734102570 0x7fb734198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.641+0000 7fb739e57640 1 --2- 172.21.15.152:0/2607185272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb734100e20 0x7fb734198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:42.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.641+0000 7fb739e57640 1 -- 172.21.15.152:0/2607185272 >> 172.21.15.152:0/2607185272 conn(0x7fb7340fc7d0 msgr2=0x7fb73406a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:42.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.641+0000 7fb739e57640 1 -- 172.21.15.152:0/2607185272 shutdown_connections 2024-09-11T22:59:42.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:42.641+0000 7fb739e57640 1 -- 172.21.15.152:0/2607185272 wait complete. 2024-09-11T22:59:42.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:42 smithi191 ceph-mon[35211]: pgmap v207: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T22:59:42.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:42 smithi204 ceph-mon[35454]: pgmap v207: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T22:59:43.466 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:43 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T22:59:43.466 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:43 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:59:43.466 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:43 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T22:59:43.466 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:43 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:59:43.467 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:43 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T22:59:43.495 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:59:43.746 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:43.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:43 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T22:59:43.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:43 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:59:43.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:43 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T22:59:43.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:43 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:59:43.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:43 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T22:59:43.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:43 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T22:59:43.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:43 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:59:43.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:43 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T22:59:43.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:43 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:59:43.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:43 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T22:59:44.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.270+0000 7fbd31b2a640 1 -- 172.21.15.152:0/330508983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c102790 msgr2=0x7fbd2c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.270+0000 7fbd31b2a640 1 --2- 172.21.15.152:0/330508983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c102790 0x7fbd2c1067e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009900 tx=0x7fbd2002a410 comp rx=0 tx=0).stop 2024-09-11T22:59:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.270+0000 7fbd31b2a640 1 -- 172.21.15.152:0/330508983 shutdown_connections 2024-09-11T22:59:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.270+0000 7fbd31b2a640 1 --2- 172.21.15.152:0/330508983 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd2c106f10 0x7fbd2c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.270+0000 7fbd31b2a640 1 --2- 172.21.15.152:0/330508983 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c102790 0x7fbd2c1067e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.270+0000 7fbd31b2a640 1 --2- 172.21.15.152:0/330508983 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd2c101de0 0x7fbd2c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.270+0000 7fbd31b2a640 1 -- 172.21.15.152:0/330508983 >> 172.21.15.152:0/330508983 conn(0x7fbd2c0fd8c0 msgr2=0x7fbd2c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.270+0000 7fbd31b2a640 1 -- 172.21.15.152:0/330508983 shutdown_connections 2024-09-11T22:59:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.270+0000 7fbd31b2a640 1 -- 172.21.15.152:0/330508983 wait complete. 2024-09-11T22:59:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.271+0000 7fbd31b2a640 1 Processor -- start 2024-09-11T22:59:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.271+0000 7fbd31b2a640 1 -- start start 2024-09-11T22:59:44.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.271+0000 7fbd31b2a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c101de0 0x7fbd2c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.271+0000 7fbd31b2a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd2c102790 0x7fbd2c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.271+0000 7fbd31b2a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd2c106f10 0x7fbd2c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.271+0000 7fbd31b2a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c111f70 con 0x7fbd2c101de0 2024-09-11T22:59:44.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.271+0000 7fbd31b2a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c111df0 con 0x7fbd2c102790 2024-09-11T22:59:44.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.271+0000 7fbd31b2a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1120f0 con 0x7fbd2c106f10 2024-09-11T22:59:44.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.271+0000 7fbd2b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c101de0 0x7fbd2c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.271+0000 7fbd2b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c101de0 0x7fbd2c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38638/0 (socket says 172.21.15.152:38638) 2024-09-11T22:59:44.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.271+0000 7fbd2b7fe640 1 -- 172.21.15.152:0/20601355 learned_addr learned my addr 172.21.15.152:0/20601355 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:44.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.271+0000 7fbd2affd640 1 --2- 172.21.15.152:0/20601355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd2c102790 0x7fbd2c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.271+0000 7fbd2bfff640 1 --2- 172.21.15.152:0/20601355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd2c106f10 0x7fbd2c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.272+0000 7fbd2b7fe640 1 -- 172.21.15.152:0/20601355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd2c106f10 msgr2=0x7fbd2c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:44.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.272+0000 7fbd2b7fe640 1 --2- 172.21.15.152:0/20601355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd2c106f10 0x7fbd2c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.272+0000 7fbd2b7fe640 1 -- 172.21.15.152:0/20601355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd2c102790 msgr2=0x7fbd2c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:44.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.272+0000 7fbd2b7fe640 1 --2- 172.21.15.152:0/20601355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd2c102790 0x7fbd2c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.272+0000 7fbd2b7fe640 1 -- 172.21.15.152:0/20601355 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2c1a69d0 con 0x7fbd2c101de0 2024-09-11T22:59:44.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.272+0000 7fbd2bfff640 1 --2- 172.21.15.152:0/20601355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd2c106f10 0x7fbd2c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:59:44.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.272+0000 7fbd2b7fe640 1 --2- 172.21.15.152:0/20601355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c101de0 0x7fbd2c0744f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00c970 tx=0x7fbd1c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:44.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.272+0000 7fbd28ff9640 1 -- 172.21.15.152:0/20601355 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c007bb0 con 0x7fbd2c101de0 2024-09-11T22:59:44.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.272+0000 7fbd28ff9640 1 -- 172.21.15.152:0/20601355 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd1c007d50 con 0x7fbd2c101de0 2024-09-11T22:59:44.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.272+0000 7fbd31b2a640 1 -- 172.21.15.152:0/20601355 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2c1a6cc0 con 0x7fbd2c101de0 2024-09-11T22:59:44.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.272+0000 7fbd28ff9640 1 -- 172.21.15.152:0/20601355 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c00f450 con 0x7fbd2c101de0 2024-09-11T22:59:44.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.272+0000 7fbd31b2a640 1 -- 172.21.15.152:0/20601355 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd2c1a7180 con 0x7fbd2c101de0 2024-09-11T22:59:44.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.274+0000 7fbd28ff9640 1 -- 172.21.15.152:0/20601355 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fbd1c005020 con 0x7fbd2c101de0 2024-09-11T22:59:44.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.274+0000 7fbd33f03740 1 Processor -- start 2024-09-11T22:59:44.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd33f03740 1 -- start start 2024-09-11T22:59:44.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd33f03740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4427794a0 0x55e442779880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd33f03740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442779dc0 0x55e442784140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd33f03740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442784680 0x55e442786a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd33f03740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55e4426df2e0 con 0x55e442784680 2024-09-11T22:59:44.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd33f03740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e442686ce0 con 0x55e4427794a0 2024-09-11T22:59:44.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd33f03740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e4426db2a0 con 0x55e442779dc0 2024-09-11T22:59:44.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd2affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442784680 0x55e442786a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd28ff9640 1 --2- 172.21.15.152:0/20601355 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbd140a2cf0 0x7fbd140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd2affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442784680 0x55e442786a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38648/0 (socket says 172.21.15.152:38648) 2024-09-11T22:59:44.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd2affd640 1 -- 172.21.15.152:0/2351166585 learned_addr learned my addr 172.21.15.152:0/2351166585 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:44.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd2b7fe640 1 --2- 172.21.15.152:0/2351166585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442779dc0 0x55e442784140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd2bfff640 1 --2- 172.21.15.152:0/2351166585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4427794a0 0x55e442779880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd2bfff640 1 --2- 172.21.15.152:0/20601355 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbd140a2cf0 0x7fbd140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd28ff9640 1 -- 172.21.15.152:0/20601355 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fbd1c0cf230 con 0x7fbd2c101de0 2024-09-11T22:59:44.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd2affd640 1 -- 172.21.15.152:0/2351166585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442779dc0 msgr2=0x55e442784140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:44.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd2affd640 1 --2- 172.21.15.152:0/2351166585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442779dc0 0x55e442784140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd2affd640 1 -- 172.21.15.152:0/2351166585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4427794a0 msgr2=0x55e442779880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:44.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd2affd640 1 --2- 172.21.15.152:0/2351166585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4427794a0 0x55e442779880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.275+0000 7fbd2affd640 1 -- 172.21.15.152:0/2351166585 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e44266b440 con 0x55e442784680 2024-09-11T22:59:44.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.276+0000 7fbd2affd640 1 --2- 172.21.15.152:0/2351166585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442784680 0x55e442786a70 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009900 tx=0x7fbd200316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c42a200c44e201 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:44.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.276+0000 7fbd2bfff640 1 --2- 172.21.15.152:0/20601355 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbd140a2cf0 0x7fbd140a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbd24005fd0 tx=0x7fbd24005c80 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:44.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.276+0000 7fbd11ffb640 1 -- 172.21.15.152:0/2351166585 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20040070 con 0x55e442784680 2024-09-11T22:59:44.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.276+0000 7fbd11ffb640 1 -- 172.21.15.152:0/2351166585 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd20005020 con 0x55e442784680 2024-09-11T22:59:44.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.276+0000 7fbd11ffb640 1 -- 172.21.15.152:0/2351166585 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20035040 con 0x55e442784680 2024-09-11T22:59:44.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.277+0000 7fbd33f03740 1 -- 172.21.15.152:0/2351166585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442784680 msgr2=0x55e442786a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.277+0000 7fbd33f03740 1 --2- 172.21.15.152:0/2351166585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442784680 0x55e442786a70 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009900 tx=0x7fbd200316e0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.277+0000 7fbd33f03740 1 -- 172.21.15.152:0/2351166585 shutdown_connections 2024-09-11T22:59:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.277+0000 7fbd33f03740 1 --2- 172.21.15.152:0/2351166585 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442784680 0x55e442786a70 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.277+0000 7fbd33f03740 1 --2- 172.21.15.152:0/2351166585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442779dc0 0x55e442784140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.277+0000 7fbd33f03740 1 --2- 172.21.15.152:0/2351166585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4427794a0 0x55e442779880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.277+0000 7fbd33f03740 1 -- 172.21.15.152:0/2351166585 >> 172.21.15.152:0/2351166585 conn(0x55e442775a30 msgr2=0x55e442775e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.277+0000 7fbd33f03740 1 -- 172.21.15.152:0/2351166585 shutdown_connections 2024-09-11T22:59:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.277+0000 7fbd33f03740 1 -- 172.21.15.152:0/2351166585 wait complete. 2024-09-11T22:59:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.277+0000 7fbd33f03740 1 -- start start 2024-09-11T22:59:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.277+0000 7fbd33f03740 1 Processor -- start 2024-09-11T22:59:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.277+0000 7fbd33f03740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4427794a0 0x55e442783f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.277+0000 7fbd33f03740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442779dc0 0x55e4427811b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.277+0000 7fbd33f03740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442784680 0x55e4427816f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd33f03740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55e442787250 con 0x55e442784680 2024-09-11T22:59:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd33f03740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e44269a050 con 0x55e4427794a0 2024-09-11T22:59:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd33f03740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e4427873d0 con 0x55e442779dc0 2024-09-11T22:59:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd33f03740 10 client.0 _ll_register_callbacks cb 0x55e4426da160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd2bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442784680 0x55e4427816f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd2bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442784680 0x55e4427816f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38656/0 (socket says 172.21.15.152:38656) 2024-09-11T22:59:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd2bfff640 1 -- 172.21.15.152:0/986745212 learned_addr learned my addr 172.21.15.152:0/986745212 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd2b7fe640 1 --2- 172.21.15.152:0/986745212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4427794a0 0x55e442783f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd2bfff640 1 -- 172.21.15.152:0/986745212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442779dc0 msgr2=0x55e4427811b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd2bfff640 1 --2- 172.21.15.152:0/986745212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442779dc0 0x55e4427811b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd2bfff640 1 -- 172.21.15.152:0/986745212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4427794a0 msgr2=0x55e442783f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd2bfff640 1 --2- 172.21.15.152:0/986745212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4427794a0 0x55e442783f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd2bfff640 1 -- 172.21.15.152:0/986745212 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4424503c0 con 0x55e442784680 2024-09-11T22:59:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd2bfff640 1 --2- 172.21.15.152:0/986745212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442784680 0x55e4427816f0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400fb80 tx=0x7fbd24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d324172e8068ce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd0affd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd0affd640 1 -- 172.21.15.152:0/986745212 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2400de50 con 0x55e442784680 2024-09-11T22:59:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd0affd640 1 -- 172.21.15.152:0/986745212 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fbd2401e450 con 0x55e442784680 2024-09-11T22:59:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd33f03740 1 -- 172.21.15.152:0/986745212 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd20005020 con 0x55e442784680 2024-09-11T22:59:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd0affd640 1 -- 172.21.15.152:0/986745212 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbd2401e650 con 0x55e442784680 2024-09-11T22:59:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd33f03740 10 client.17286 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.278+0000 7fbd33f03740 1 -- 172.21.15.152:0/986745212 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e442792bc0 con 0x55e442784680 2024-09-11T22:59:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.279+0000 7fbd0affd640 1 -- 172.21.15.152:0/986745212 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd20005020 con 0x55e442784680 2024-09-11T22:59:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.279+0000 7fbd0affd640 1 -- 172.21.15.152:0/986745212 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbd2401b040 con 0x55e442784680 2024-09-11T22:59:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.279+0000 7fbd33f03740 10 client.17286 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.279+0000 7fbd33f03740 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:59:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.279+0000 7fbd33f03740 20 client.17286 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.279+0000 7fbd33f03740 1 --2- 172.21.15.152:0/986745212 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55e442794040 0x55e442796500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.279+0000 7fbd33f03740 4 client.17286 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:59:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.279+0000 7fbd33f03740 1 -- 172.21.15.152:0/986745212 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4426ec850 con 0x55e442794040 2024-09-11T22:59:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.279+0000 7fbd2b7fe640 1 --2- 172.21.15.152:0/986745212 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55e442794040 0x55e442796500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.279+0000 7fbd0affd640 10 client.17286 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.279+0000 7fbd2b7fe640 1 --2- 172.21.15.152:0/986745212 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55e442794040 0x55e442796500 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee0f137e87592464 server_cookie=88183d35be56691b in_seq=0 out_seq=0 2024-09-11T22:59:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.280+0000 7fbd0affd640 1 -- 172.21.15.152:0/986745212 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4426ec850 con 0x55e442794040 2024-09-11T22:59:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.280+0000 7fbd0affd640 10 client.17286 handle_command_reply: tid=0 2024-09-11T22:59:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.280+0000 7fbd33f03740 1 client.17286 shutdown 2024-09-11T22:59:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.280+0000 7fbd33f03740 1 -- 172.21.15.152:0/986745212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442784680 msgr2=0x55e4427816f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.280+0000 7fbd33f03740 1 --2- 172.21.15.152:0/986745212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442784680 0x55e4427816f0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400fb80 tx=0x7fbd24004a40 comp rx=0 tx=0).stop 2024-09-11T22:59:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.281+0000 7fbd33f03740 1 -- 172.21.15.152:0/986745212 shutdown_connections 2024-09-11T22:59:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.281+0000 7fbd33f03740 1 --2- 172.21.15.152:0/986745212 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442784680 0x55e4427816f0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.281+0000 7fbd33f03740 1 --2- 172.21.15.152:0/986745212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442779dc0 0x55e4427811b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.281+0000 7fbd33f03740 1 --2- 172.21.15.152:0/986745212 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55e442794040 0x55e442796500 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.281+0000 7fbd33f03740 1 --2- 172.21.15.152:0/986745212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4427794a0 0x55e442783f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.281+0000 7fbd33f03740 1 -- 172.21.15.152:0/986745212 >> 172.21.15.152:0/986745212 conn(0x55e442775a30 msgr2=0x55e44277b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.281+0000 7fbd0affd640 0 client.17286 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.281+0000 7fbd33f03740 1 -- 172.21.15.152:0/986745212 shutdown_connections 2024-09-11T22:59:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.281+0000 7fbd33f03740 1 -- 172.21.15.152:0/986745212 wait complete. 2024-09-11T22:59:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.281+0000 7fbd33f03740 20 client.17286 trim_cache size 0 max 16384 2024-09-11T22:59:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd33f03740 1 Processor -- start 2024-09-11T22:59:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd33f03740 1 -- start start 2024-09-11T22:59:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd33f03740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4427794a0 0x55e442783120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd33f03740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442779dc0 0x55e44279bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd33f03740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442784680 0x55e44279c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd33f03740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1af970 con 0x55e442779dc0 2024-09-11T22:59:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd33f03740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1b14b0 con 0x55e4427794a0 2024-09-11T22:59:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd33f03740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1b0040 con 0x55e442784680 2024-09-11T22:59:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd2affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442779dc0 0x55e44279bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd2affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442779dc0 0x55e44279bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38658/0 (socket says 172.21.15.152:38658) 2024-09-11T22:59:44.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd2affd640 1 -- 172.21.15.152:0/128075402 learned_addr learned my addr 172.21.15.152:0/128075402 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:44.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd2b7fe640 1 --2- 172.21.15.152:0/128075402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4427794a0 0x55e442783120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd2affd640 1 -- 172.21.15.152:0/128075402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442784680 msgr2=0x55e44279c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:44.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd2bfff640 1 --2- 172.21.15.152:0/128075402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442784680 0x55e44279c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd2affd640 1 --2- 172.21.15.152:0/128075402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442784680 0x55e44279c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd2affd640 1 -- 172.21.15.152:0/128075402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4427794a0 msgr2=0x55e442783120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:44.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd2affd640 1 --2- 172.21.15.152:0/128075402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4427794a0 0x55e442783120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd2affd640 1 -- 172.21.15.152:0/128075402 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e442775860 con 0x55e442779dc0 2024-09-11T22:59:44.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd2b7fe640 1 --2- 172.21.15.152:0/128075402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4427794a0 0x55e442783120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:44.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.298+0000 7fbd2bfff640 1 --2- 172.21.15.152:0/128075402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442784680 0x55e44279c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:59:44.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.299+0000 7fbd2affd640 1 --2- 172.21.15.152:0/128075402 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442779dc0 0x55e44279bdd0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fbd2002a930 tx=0x7fbd20000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94239c290b698a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:44.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.299+0000 7fbd117fa640 1 -- 172.21.15.152:0/128075402 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20040050 con 0x55e442779dc0 2024-09-11T22:59:44.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.299+0000 7fbd117fa640 1 -- 172.21.15.152:0/128075402 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd20048070 con 0x55e442779dc0 2024-09-11T22:59:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.299+0000 7fbd117fa640 1 -- 172.21.15.152:0/128075402 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20035040 con 0x55e442779dc0 2024-09-11T22:59:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.299+0000 7fbd33f03740 1 -- 172.21.15.152:0/128075402 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442779dc0 msgr2=0x55e44279bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.299+0000 7fbd33f03740 1 --2- 172.21.15.152:0/128075402 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442779dc0 0x55e44279bdd0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fbd2002a930 tx=0x7fbd20000bc0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.300+0000 7fbd33f03740 1 -- 172.21.15.152:0/128075402 shutdown_connections 2024-09-11T22:59:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.300+0000 7fbd33f03740 1 --2- 172.21.15.152:0/128075402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442784680 0x55e44279c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.300+0000 7fbd33f03740 1 --2- 172.21.15.152:0/128075402 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e442779dc0 0x55e44279bdd0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.300+0000 7fbd33f03740 1 --2- 172.21.15.152:0/128075402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e4427794a0 0x55e442783120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.300+0000 7fbd33f03740 1 -- 172.21.15.152:0/128075402 >> 172.21.15.152:0/128075402 conn(0x55e442775a30 msgr2=0x55e442782a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.300+0000 7fbd33f03740 1 -- 172.21.15.152:0/128075402 shutdown_connections 2024-09-11T22:59:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.300+0000 7fbd33f03740 1 -- 172.21.15.152:0/128075402 wait complete. 2024-09-11T22:59:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.300+0000 7fbd33f03740 1 -- start start 2024-09-11T22:59:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.300+0000 7fbd33f03740 1 Processor -- start 2024-09-11T22:59:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.300+0000 7fbd33f03740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e4427794a0 0x55e44278cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.300+0000 7fbd33f03740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e442779dc0 0x55e442789f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd33f03740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442784680 0x55e44278a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd33f03740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1b0de0 con 0x55e4427794a0 2024-09-11T22:59:44.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd33f03740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1b0710 con 0x55e442779dc0 2024-09-11T22:59:44.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd33f03740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1ad930 con 0x55e442784680 2024-09-11T22:59:44.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd33f03740 10 client.0 _ll_register_callbacks cb 0x7fbd2400f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:44.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd2b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e4427794a0 0x55e44278cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd2b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e4427794a0 0x55e44278cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38660/0 (socket says 172.21.15.152:38660) 2024-09-11T22:59:44.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd2b7fe640 1 -- 172.21.15.152:0/3374181661 learned_addr learned my addr 172.21.15.152:0/3374181661 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:44.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd2affd640 1 --2- 172.21.15.152:0/3374181661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e442779dc0 0x55e442789f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd2bfff640 1 --2- 172.21.15.152:0/3374181661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442784680 0x55e44278a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd2b7fe640 1 -- 172.21.15.152:0/3374181661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442784680 msgr2=0x55e44278a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:44.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd2b7fe640 1 --2- 172.21.15.152:0/3374181661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442784680 0x55e44278a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd2b7fe640 1 -- 172.21.15.152:0/3374181661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e442779dc0 msgr2=0x55e442789f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:44.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd2b7fe640 1 --2- 172.21.15.152:0/3374181661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e442779dc0 0x55e442789f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd2b7fe640 1 -- 172.21.15.152:0/3374181661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e44278abc0 con 0x55e4427794a0 2024-09-11T22:59:44.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd2b7fe640 1 --2- 172.21.15.152:0/3374181661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e4427794a0 0x55e44278cc40 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c098130 tx=0x7fbd1c004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e9fe286ab38bd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:44.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd0affd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:44.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd0affd640 1 -- 172.21.15.152:0/3374181661 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c01d040 con 0x55e4427794a0 2024-09-11T22:59:44.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd0affd640 1 -- 172.21.15.152:0/3374181661 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fbd1c099a60 con 0x55e4427794a0 2024-09-11T22:59:44.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd33f03740 1 -- 172.21.15.152:0/3374181661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd20048070 con 0x55e4427794a0 2024-09-11T22:59:44.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd0affd640 1 -- 172.21.15.152:0/3374181661 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbd1c09fb10 con 0x55e4427794a0 2024-09-11T22:59:44.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd33f03740 10 client.17292 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:44.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.301+0000 7fbd33f03740 1 -- 172.21.15.152:0/3374181661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e4427febf0 con 0x55e4427794a0 2024-09-11T22:59:44.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.302+0000 7fbd0affd640 1 -- 172.21.15.152:0/3374181661 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd20048070 con 0x55e4427794a0 2024-09-11T22:59:44.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.302+0000 7fbd0affd640 1 -- 172.21.15.152:0/3374181661 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbd1c0ccc10 con 0x55e4427794a0 2024-09-11T22:59:44.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.302+0000 7fbd33f03740 10 client.17292 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:44.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.302+0000 7fbd33f03740 10 client.17292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:59:44.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.302+0000 7fbd33f03740 20 client.17292 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:44.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.302+0000 7fbd33f03740 1 --2- 172.21.15.152:0/3374181661 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55e4427ff270 0x55e442801660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:44.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.302+0000 7fbd33f03740 4 client.17292 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:59:44.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.302+0000 7fbd33f03740 1 -- 172.21.15.152:0/3374181661 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e442781a80 con 0x55e4427ff270 2024-09-11T22:59:44.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.302+0000 7fbd2affd640 1 --2- 172.21.15.152:0/3374181661 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55e4427ff270 0x55e442801660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:44.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.302+0000 7fbd2affd640 1 --2- 172.21.15.152:0/3374181661 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55e4427ff270 0x55e442801660 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39af4a985fd8580d server_cookie=f042a9a41017bdfd in_seq=0 out_seq=0 2024-09-11T22:59:44.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.302+0000 7fbd0affd640 10 client.17292 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:44.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.303+0000 7fbd0affd640 1 -- 172.21.15.152:0/3374181661 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e442781a80 con 0x55e4427ff270 2024-09-11T22:59:44.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.303+0000 7fbd0affd640 10 client.17292 handle_command_reply: tid=0 2024-09-11T22:59:44.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.303+0000 7fbd33f03740 1 client.17292 shutdown 2024-09-11T22:59:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.303+0000 7fbd33f03740 1 -- 172.21.15.152:0/3374181661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e4427794a0 msgr2=0x55e44278cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.303+0000 7fbd33f03740 1 --2- 172.21.15.152:0/3374181661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e4427794a0 0x55e44278cc40 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c098130 tx=0x7fbd1c004a20 comp rx=0 tx=0).stop 2024-09-11T22:59:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.303+0000 7fbd33f03740 1 -- 172.21.15.152:0/3374181661 shutdown_connections 2024-09-11T22:59:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.303+0000 7fbd33f03740 1 --2- 172.21.15.152:0/3374181661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e442784680 0x55e44278a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.309 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.303+0000 7fbd33f03740 1 --2- 172.21.15.152:0/3374181661 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55e4427ff270 0x55e442801660 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.303+0000 7fbd33f03740 1 --2- 172.21.15.152:0/3374181661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e442779dc0 0x55e442789f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.303+0000 7fbd33f03740 1 --2- 172.21.15.152:0/3374181661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e4427794a0 0x55e44278cc40 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.303+0000 7fbd0affd640 0 client.17292 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:44.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.303+0000 7fbd33f03740 1 -- 172.21.15.152:0/3374181661 >> 172.21.15.152:0/3374181661 conn(0x55e442775a30 msgr2=0x55e442774350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:44.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.303+0000 7fbd33f03740 1 -- 172.21.15.152:0/3374181661 shutdown_connections 2024-09-11T22:59:44.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.303+0000 7fbd33f03740 1 -- 172.21.15.152:0/3374181661 wait complete. 2024-09-11T22:59:44.310 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.303+0000 7fbd33f03740 20 client.17292 trim_cache size 0 max 16384 2024-09-11T22:59:44.310 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:59:44.310 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T22:59:44.310 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T22:59:44.311 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:59:44.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.304+0000 7fbd31b2a640 1 -- 172.21.15.152:0/20601355 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbd140a2cf0 msgr2=0x7fbd140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:44.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.304+0000 7fbd31b2a640 1 --2- 172.21.15.152:0/20601355 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbd140a2cf0 0x7fbd140a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbd24005fd0 tx=0x7fbd24005c80 comp rx=0 tx=0).stop 2024-09-11T22:59:44.311 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.304+0000 7fbd31b2a640 1 -- 172.21.15.152:0/20601355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c101de0 msgr2=0x7fbd2c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:44.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.304+0000 7fbd31b2a640 1 --2- 172.21.15.152:0/20601355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c101de0 0x7fbd2c0744f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00c970 tx=0x7fbd1c00ce30 comp rx=0 tx=0).stop 2024-09-11T22:59:44.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.304+0000 7fbd31b2a640 1 -- 172.21.15.152:0/20601355 shutdown_connections 2024-09-11T22:59:44.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.304+0000 7fbd31b2a640 1 --2- 172.21.15.152:0/20601355 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbd140a2cf0 0x7fbd140a51b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.304+0000 7fbd31b2a640 1 --2- 172.21.15.152:0/20601355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd2c106f10 0x7fbd2c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.304+0000 7fbd31b2a640 1 --2- 172.21.15.152:0/20601355 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd2c102790 0x7fbd2c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.304+0000 7fbd31b2a640 1 --2- 172.21.15.152:0/20601355 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbd2c101de0 0x7fbd2c0744f0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:44.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.304+0000 7fbd31b2a640 1 -- 172.21.15.152:0/20601355 >> 172.21.15.152:0/20601355 conn(0x7fbd2c0fd8c0 msgr2=0x7fbd2c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:44.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.304+0000 7fbd31b2a640 1 -- 172.21.15.152:0/20601355 shutdown_connections 2024-09-11T22:59:44.312 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:44.304+0000 7fbd31b2a640 1 -- 172.21.15.152:0/20601355 wait complete. 2024-09-11T22:59:44.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:44 smithi152 ceph-mon[31181]: pgmap v208: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:44.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:44 smithi191 ceph-mon[35211]: pgmap v208: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:44.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:44 smithi204 ceph-mon[35454]: pgmap v208: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:45.081 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:59:45.082 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:59:45.329 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:45.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.830+0000 7fdab7e72640 1 -- 172.21.15.152:0/19913517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab01036c0 msgr2=0x7fdab010bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:45.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.830+0000 7fdab7e72640 1 --2- 172.21.15.152:0/19913517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab01036c0 0x7fdab010bbb0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009900 tx=0x7fdaa002a480 comp rx=0 tx=0).stop 2024-09-11T22:59:45.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.830+0000 7fdab7e72640 1 -- 172.21.15.152:0/19913517 shutdown_connections 2024-09-11T22:59:45.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.830+0000 7fdab7e72640 1 --2- 172.21.15.152:0/19913517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdab010c180 0x7fdab010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:45.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.830+0000 7fdab7e72640 1 --2- 172.21.15.152:0/19913517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab01036c0 0x7fdab010bbb0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:45.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.830+0000 7fdab7e72640 1 --2- 172.21.15.152:0/19913517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdab0102da0 0x7fdab0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:45.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.830+0000 7fdab7e72640 1 -- 172.21.15.152:0/19913517 >> 172.21.15.152:0/19913517 conn(0x7fdab00fc7a0 msgr2=0x7fdab00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:45.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.830+0000 7fdab7e72640 1 -- 172.21.15.152:0/19913517 shutdown_connections 2024-09-11T22:59:45.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.830+0000 7fdab7e72640 1 -- 172.21.15.152:0/19913517 wait complete. 2024-09-11T22:59:45.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.831+0000 7fdab7e72640 1 Processor -- start 2024-09-11T22:59:45.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.831+0000 7fdab7e72640 1 -- start start 2024-09-11T22:59:45.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.831+0000 7fdab7e72640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdab0102da0 0x7fdab0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:45.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.831+0000 7fdab7e72640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdab01036c0 0x7fdab0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:45.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.831+0000 7fdab7e72640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab010c180 0x7fdab019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:45.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.831+0000 7fdab7e72640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdab0111f60 con 0x7fdab010c180 2024-09-11T22:59:45.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.831+0000 7fdab7e72640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdab0111de0 con 0x7fdab01036c0 2024-09-11T22:59:45.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.831+0000 7fdab7e72640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01120e0 con 0x7fdab0102da0 2024-09-11T22:59:45.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.831+0000 7fdab640f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab010c180 0x7fdab019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:45.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.831+0000 7fdab640f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab010c180 0x7fdab019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38676/0 (socket says 172.21.15.152:38676) 2024-09-11T22:59:45.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.831+0000 7fdab640f640 1 -- 172.21.15.152:0/1349421541 learned_addr learned my addr 172.21.15.152:0/1349421541 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:45.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.831+0000 7fdab5c0e640 1 --2- 172.21.15.152:0/1349421541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdab0102da0 0x7fdab0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:45.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.832+0000 7fdab540d640 1 --2- 172.21.15.152:0/1349421541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdab01036c0 0x7fdab0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:45.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.832+0000 7fdab640f640 1 -- 172.21.15.152:0/1349421541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdab0102da0 msgr2=0x7fdab0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:45.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.832+0000 7fdab640f640 1 --2- 172.21.15.152:0/1349421541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdab0102da0 0x7fdab0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:45.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.832+0000 7fdab640f640 1 -- 172.21.15.152:0/1349421541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdab01036c0 msgr2=0x7fdab0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:45.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.832+0000 7fdab640f640 1 --2- 172.21.15.152:0/1349421541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdab01036c0 0x7fdab0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:45.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.832+0000 7fdab640f640 1 -- 172.21.15.152:0/1349421541 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab019d3e0 con 0x7fdab010c180 2024-09-11T22:59:45.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.832+0000 7fdab640f640 1 --2- 172.21.15.152:0/1349421541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab010c180 0x7fdab019cd00 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800cad0 tx=0x7fdaa800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:45.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.832+0000 7fdaa6ffd640 1 -- 172.21.15.152:0/1349421541 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa800ae20 con 0x7fdab010c180 2024-09-11T22:59:45.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.832+0000 7fdaa6ffd640 1 -- 172.21.15.152:0/1349421541 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaa8005020 con 0x7fdab010c180 2024-09-11T22:59:45.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.832+0000 7fdaa6ffd640 1 -- 172.21.15.152:0/1349421541 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa800f6f0 con 0x7fdab010c180 2024-09-11T22:59:45.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.832+0000 7fdab7e72640 1 -- 172.21.15.152:0/1349421541 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab019d6d0 con 0x7fdab010c180 2024-09-11T22:59:45.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.832+0000 7fdab7e72640 1 -- 172.21.15.152:0/1349421541 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab01a4fb0 con 0x7fdab010c180 2024-09-11T22:59:45.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.833+0000 7fdab7e72640 1 -- 172.21.15.152:0/1349421541 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab0103180 con 0x7fdab010c180 2024-09-11T22:59:45.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.834+0000 7fdaa6ffd640 1 -- 172.21.15.152:0/1349421541 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fdaa80051d0 con 0x7fdab010c180 2024-09-11T22:59:45.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.835+0000 7fdaa6ffd640 1 --2- 172.21.15.152:0/1349421541 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fda9c0a2cf0 0x7fda9c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:45.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.835+0000 7fdab5c0e640 1 --2- 172.21.15.152:0/1349421541 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fda9c0a2cf0 0x7fda9c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:45.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.835+0000 7fdaa6ffd640 1 -- 172.21.15.152:0/1349421541 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fdaa80cf790 con 0x7fdab010c180 2024-09-11T22:59:45.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.835+0000 7fdab5c0e640 1 --2- 172.21.15.152:0/1349421541 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fda9c0a2cf0 0x7fda9c0a51b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009bd0 tx=0x7fdaac005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:45.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:45.839+0000 7fdaa6ffd640 1 -- 172.21.15.152:0/1349421541 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdaa8014030 con 0x7fdab010c180 2024-09-11T22:59:46.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:46.039+0000 7fdab7e72640 1 -- 172.21.15.152:0/1349421541 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdab0107220 con 0x7fdab010c180 2024-09-11T22:59:46.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:46.040+0000 7fdaa6ffd640 1 -- 172.21.15.152:0/1349421541 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7fdaa809c400 con 0x7fdab010c180 2024-09-11T22:59:46.040 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T22:59:46.041 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T22:59:46.041 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T22:59:46.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:46.042+0000 7fdab7e72640 1 -- 172.21.15.152:0/1349421541 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fda9c0a2cf0 msgr2=0x7fda9c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:46.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:46.042+0000 7fdab7e72640 1 --2- 172.21.15.152:0/1349421541 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fda9c0a2cf0 0x7fda9c0a51b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009bd0 tx=0x7fdaac005f90 comp rx=0 tx=0).stop 2024-09-11T22:59:46.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:46.042+0000 7fdab7e72640 1 -- 172.21.15.152:0/1349421541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab010c180 msgr2=0x7fdab019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:46.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:46.042+0000 7fdab7e72640 1 --2- 172.21.15.152:0/1349421541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab010c180 0x7fdab019cd00 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800cad0 tx=0x7fdaa800cf90 comp rx=0 tx=0).stop 2024-09-11T22:59:46.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:46.042+0000 7fdab7e72640 1 -- 172.21.15.152:0/1349421541 shutdown_connections 2024-09-11T22:59:46.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:46.042+0000 7fdab7e72640 1 --2- 172.21.15.152:0/1349421541 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fda9c0a2cf0 0x7fda9c0a51b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:46.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:46.042+0000 7fdab7e72640 1 --2- 172.21.15.152:0/1349421541 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdab010c180 0x7fdab019cd00 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:46.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:46.042+0000 7fdab7e72640 1 --2- 172.21.15.152:0/1349421541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdab01036c0 0x7fdab0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:46.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:46.042+0000 7fdab7e72640 1 --2- 172.21.15.152:0/1349421541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdab0102da0 0x7fdab0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:46.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:46.042+0000 7fdab7e72640 1 -- 172.21.15.152:0/1349421541 >> 172.21.15.152:0/1349421541 conn(0x7fdab00fc7a0 msgr2=0x7fdab00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:46.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:46.042+0000 7fdab7e72640 1 -- 172.21.15.152:0/1349421541 shutdown_connections 2024-09-11T22:59:46.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:46.042+0000 7fdab7e72640 1 -- 172.21.15.152:0/1349421541 wait complete. 2024-09-11T22:59:46.781 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:46 smithi152 ceph-mon[31181]: pgmap v209: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:59:46.782 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:46 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1349421541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:59:46.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:46 smithi191 ceph-mon[35211]: pgmap v209: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:59:46.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:46 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1349421541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:59:46.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:46 smithi204 ceph-mon[35454]: pgmap v209: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:59:46.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:46 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1349421541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:59:46.816 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:59:47.070 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:47.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.575+0000 7fe5216d7640 1 -- 172.21.15.152:0/3256208387 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe51c106a80 msgr2=0x7fe51c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:47.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.575+0000 7fe5216d7640 1 --2- 172.21.15.152:0/3256208387 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe51c106a80 0x7fe51c106e60 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009930 tx=0x7fe50c02a420 comp rx=0 tx=0).stop 2024-09-11T22:59:47.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.575+0000 7fe5216d7640 1 -- 172.21.15.152:0/3256208387 shutdown_connections 2024-09-11T22:59:47.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.575+0000 7fe5216d7640 1 --2- 172.21.15.152:0/3256208387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe51c101420 0x7fe51c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.575+0000 7fe5216d7640 1 --2- 172.21.15.152:0/3256208387 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c100a80 0x7fe51c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.575+0000 7fe5216d7640 1 --2- 172.21.15.152:0/3256208387 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe51c106a80 0x7fe51c106e60 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.575+0000 7fe5216d7640 1 -- 172.21.15.152:0/3256208387 >> 172.21.15.152:0/3256208387 conn(0x7fe51c0fc7a0 msgr2=0x7fe51c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:47.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.576+0000 7fe5216d7640 1 -- 172.21.15.152:0/3256208387 shutdown_connections 2024-09-11T22:59:47.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.576+0000 7fe5216d7640 1 -- 172.21.15.152:0/3256208387 wait complete. 2024-09-11T22:59:47.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.576+0000 7fe5216d7640 1 Processor -- start 2024-09-11T22:59:47.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.576+0000 7fe5216d7640 1 -- start start 2024-09-11T22:59:47.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.576+0000 7fe5216d7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c100a80 0x7fe51c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe5216d7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe51c101420 0x7fe51c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe5216d7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe51c106a80 0x7fe51c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.595 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe5216d7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c110e80 con 0x7fe51c106a80 2024-09-11T22:59:47.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe5216d7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c110d00 con 0x7fe51c100a80 2024-09-11T22:59:47.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe5216d7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c111000 con 0x7fe51c101420 2024-09-11T22:59:47.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe51b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe51c106a80 0x7fe51c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:47.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe51b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe51c106a80 0x7fe51c19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38692/0 (socket says 172.21.15.152:38692) 2024-09-11T22:59:47.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe51b7fe640 1 -- 172.21.15.152:0/175333245 learned_addr learned my addr 172.21.15.152:0/175333245 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:47.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe51a7fc640 1 --2- 172.21.15.152:0/175333245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe51c101420 0x7fe51c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:47.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe51affd640 1 --2- 172.21.15.152:0/175333245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c100a80 0x7fe51c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:47.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe51b7fe640 1 -- 172.21.15.152:0/175333245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe51c101420 msgr2=0x7fe51c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:47.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe51b7fe640 1 --2- 172.21.15.152:0/175333245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe51c101420 0x7fe51c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe51b7fe640 1 -- 172.21.15.152:0/175333245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c100a80 msgr2=0x7fe51c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:47.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe51b7fe640 1 --2- 172.21.15.152:0/175333245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c100a80 0x7fe51c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe51b7fe640 1 -- 172.21.15.152:0/175333245 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c19d430 con 0x7fe51c106a80 2024-09-11T22:59:47.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe51affd640 1 --2- 172.21.15.152:0/175333245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c100a80 0x7fe51c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:59:47.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe51a7fc640 1 --2- 172.21.15.152:0/175333245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe51c101420 0x7fe51c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:47.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe51b7fe640 1 --2- 172.21.15.152:0/175333245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe51c106a80 0x7fe51c19cd50 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe51400eaa0 tx=0x7fe51400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:47.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe503fff640 1 -- 172.21.15.152:0/175333245 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51400cdb0 con 0x7fe51c106a80 2024-09-11T22:59:47.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe503fff640 1 -- 172.21.15.152:0/175333245 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe514005020 con 0x7fe51c106a80 2024-09-11T22:59:47.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe503fff640 1 -- 172.21.15.152:0/175333245 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5140116b0 con 0x7fe51c106a80 2024-09-11T22:59:47.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe5216d7640 1 -- 172.21.15.152:0/175333245 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51c19d720 con 0x7fe51c106a80 2024-09-11T22:59:47.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.577+0000 7fe5216d7640 1 -- 172.21.15.152:0/175333245 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51c1a5000 con 0x7fe51c106a80 2024-09-11T22:59:47.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.580+0000 7fe503fff640 1 -- 172.21.15.152:0/175333245 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe514005560 con 0x7fe51c106a80 2024-09-11T22:59:47.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.580+0000 7fe523ab0740 1 Processor -- start 2024-09-11T22:59:47.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.580+0000 7fe523ab0740 1 -- start start 2024-09-11T22:59:47.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.580+0000 7fe523ab0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126acb40 0x5571126ab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.580+0000 7fe523ab0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126ab5d0 0x5571126aba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.580+0000 7fe523ab0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126b6370 0x5571126b8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.580+0000 7fe523ab0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5571126112f0 con 0x5571126acb40 2024-09-11T22:59:47.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.580+0000 7fe523ab0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5571125b8cf0 con 0x5571126b6370 2024-09-11T22:59:47.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.580+0000 7fe523ab0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55711260d2b0 con 0x5571126ab5d0 2024-09-11T22:59:47.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe51a7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126acb40 0x5571126ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:47.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe51a7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126acb40 0x5571126ab090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38698/0 (socket says 172.21.15.152:38698) 2024-09-11T22:59:47.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe51a7fc640 1 -- 172.21.15.152:0/448081397 learned_addr learned my addr 172.21.15.152:0/448081397 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:47.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe51affd640 1 --2- 172.21.15.152:0/448081397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126ab5d0 0x5571126aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:47.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe503fff640 1 --2- 172.21.15.152:0/175333245 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5040a2d70 0x7fe5040a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe51a7fc640 1 --2- 172.21.15.152:0/448081397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126b6370 0x5571126b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:47.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe51a7fc640 1 -- 172.21.15.152:0/448081397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126ab5d0 msgr2=0x5571126aba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:47.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe51a7fc640 1 --2- 172.21.15.152:0/448081397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126ab5d0 0x5571126aba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe51a7fc640 1 -- 172.21.15.152:0/448081397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126b6370 msgr2=0x5571126b8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:47.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe51a7fc640 1 --2- 172.21.15.152:0/448081397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126b6370 0x5571126b8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe51a7fc640 1 -- 172.21.15.152:0/448081397 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55711259d3a0 con 0x5571126acb40 2024-09-11T22:59:47.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe51b7fe640 1 --2- 172.21.15.152:0/175333245 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5040a2d70 0x7fe5040a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:47.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe503fff640 1 -- 172.21.15.152:0/175333245 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fe5140cf670 con 0x7fe51c106a80 2024-09-11T22:59:47.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe51a7fc640 1 --2- 172.21.15.152:0/448081397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126acb40 0x5571126ab090 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe5100058c0 tx=0x7fe51000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2471495482cf38 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:47.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe5017fa640 1 -- 172.21.15.152:0/448081397 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510019030 con 0x5571126acb40 2024-09-11T22:59:47.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe5017fa640 1 -- 172.21.15.152:0/448081397 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe510011a20 con 0x5571126acb40 2024-09-11T22:59:47.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.581+0000 7fe5017fa640 1 -- 172.21.15.152:0/448081397 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51001da00 con 0x5571126acb40 2024-09-11T22:59:47.599 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.582+0000 7fe523ab0740 1 -- 172.21.15.152:0/448081397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126acb40 msgr2=0x5571126ab090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:47.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.582+0000 7fe51b7fe640 1 --2- 172.21.15.152:0/175333245 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5040a2d70 0x7fe5040a5230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe51400efd0 tx=0x7fe5140b6040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:47.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.582+0000 7fe523ab0740 1 --2- 172.21.15.152:0/448081397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126acb40 0x5571126ab090 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe5100058c0 tx=0x7fe51000b730 comp rx=0 tx=0).stop 2024-09-11T22:59:47.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.582+0000 7fe523ab0740 1 -- 172.21.15.152:0/448081397 shutdown_connections 2024-09-11T22:59:47.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.582+0000 7fe523ab0740 1 --2- 172.21.15.152:0/448081397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126b6370 0x5571126b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.582+0000 7fe523ab0740 1 --2- 172.21.15.152:0/448081397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126ab5d0 0x5571126aba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.582+0000 7fe523ab0740 1 --2- 172.21.15.152:0/448081397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126acb40 0x5571126ab090 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.582+0000 7fe523ab0740 1 -- 172.21.15.152:0/448081397 >> 172.21.15.152:0/448081397 conn(0x557112674310 msgr2=0x5571126a7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:47.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.582+0000 7fe523ab0740 1 -- 172.21.15.152:0/448081397 shutdown_connections 2024-09-11T22:59:47.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.582+0000 7fe523ab0740 1 -- 172.21.15.152:0/448081397 wait complete. 2024-09-11T22:59:47.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.582+0000 7fe523ab0740 1 -- start start 2024-09-11T22:59:47.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe523ab0740 1 Processor -- start 2024-09-11T22:59:47.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe523ab0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126ab5d0 0x5571126b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe523ab0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126acb40 0x5571126b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe523ab0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126b6370 0x5571126b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe523ab0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5571126b9070 con 0x5571126ab5d0 2024-09-11T22:59:47.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe523ab0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5571125cc060 con 0x5571126acb40 2024-09-11T22:59:47.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe523ab0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5571126b91f0 con 0x5571126b6370 2024-09-11T22:59:47.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe523ab0740 10 client.0 _ll_register_callbacks cb 0x55711231f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:47.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe51a7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126ab5d0 0x5571126b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:47.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe51a7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126ab5d0 0x5571126b5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38710/0 (socket says 172.21.15.152:38710) 2024-09-11T22:59:47.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe51a7fc640 1 -- 172.21.15.152:0/1477045863 learned_addr learned my addr 172.21.15.152:0/1477045863 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:47.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe51a7fc640 1 -- 172.21.15.152:0/1477045863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126b6370 msgr2=0x5571126b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:47.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe51a7fc640 1 --2- 172.21.15.152:0/1477045863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126b6370 0x5571126b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe51a7fc640 1 -- 172.21.15.152:0/1477045863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126acb40 msgr2=0x5571126b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:47.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe51a7fc640 1 --2- 172.21.15.152:0/1477045863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126acb40 0x5571126b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe51a7fc640 1 -- 172.21.15.152:0/1477045863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571123823c0 con 0x5571126ab5d0 2024-09-11T22:59:47.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe51a7fc640 1 --2- 172.21.15.152:0/1477045863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126ab5d0 0x5571126b5d30 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe51000bc90 tx=0x7fe51000b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f575de3448bc3f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:47.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.583+0000 7fe4feff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:47.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.584+0000 7fe4feff5640 1 -- 172.21.15.152:0/1477045863 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510019040 con 0x5571126ab5d0 2024-09-11T22:59:47.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.584+0000 7fe523ab0740 1 -- 172.21.15.152:0/1477045863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe510011a20 con 0x5571126ab5d0 2024-09-11T22:59:47.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.584+0000 7fe4feff5640 1 -- 172.21.15.152:0/1477045863 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fe510011740 con 0x5571126ab5d0 2024-09-11T22:59:47.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.584+0000 7fe4feff5640 1 -- 172.21.15.152:0/1477045863 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe510013040 con 0x5571126ab5d0 2024-09-11T22:59:47.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.584+0000 7fe523ab0740 10 client.17310 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:47.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.584+0000 7fe523ab0740 1 -- 172.21.15.152:0/1477045863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5571126c4ad0 con 0x5571126ab5d0 2024-09-11T22:59:47.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.584+0000 7fe4feff5640 1 -- 172.21.15.152:0/1477045863 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe510011a20 con 0x5571126ab5d0 2024-09-11T22:59:47.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.584+0000 7fe4feff5640 1 -- 172.21.15.152:0/1477045863 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe51001d9e0 con 0x5571126ab5d0 2024-09-11T22:59:47.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.584+0000 7fe523ab0740 10 client.17310 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:47.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.584+0000 7fe523ab0740 10 client.17310 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:59:47.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.584+0000 7fe523ab0740 20 client.17310 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:47.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.584+0000 7fe523ab0740 1 --2- 172.21.15.152:0/1477045863 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5571126c5ed0 0x5571126c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.584+0000 7fe523ab0740 4 client.17310 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:59:47.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.584+0000 7fe523ab0740 1 -- 172.21.15.152:0/1477045863 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55711261e860 con 0x5571126c5ed0 2024-09-11T22:59:47.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.584+0000 7fe51affd640 1 --2- 172.21.15.152:0/1477045863 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5571126c5ed0 0x5571126c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:47.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.585+0000 7fe4feff5640 10 client.17310 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:59:47.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.585+0000 7fe51affd640 1 --2- 172.21.15.152:0/1477045863 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5571126c5ed0 0x5571126c82c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eaf4fadcc7331c61 server_cookie=26f2dbca88acd349 in_seq=0 out_seq=0 2024-09-11T22:59:47.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.585+0000 7fe4feff5640 1 -- 172.21.15.152:0/1477045863 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55711261e860 con 0x5571126c5ed0 2024-09-11T22:59:47.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.585+0000 7fe4feff5640 10 client.17310 handle_command_reply: tid=0 2024-09-11T22:59:47.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.585+0000 7fe523ab0740 1 client.17310 shutdown 2024-09-11T22:59:47.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.585+0000 7fe523ab0740 1 -- 172.21.15.152:0/1477045863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126ab5d0 msgr2=0x5571126b5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:47.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.585+0000 7fe523ab0740 1 --2- 172.21.15.152:0/1477045863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126ab5d0 0x5571126b5d30 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe51000bc90 tx=0x7fe51000b3a0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.585+0000 7fe523ab0740 1 -- 172.21.15.152:0/1477045863 shutdown_connections 2024-09-11T22:59:47.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.585+0000 7fe523ab0740 1 --2- 172.21.15.152:0/1477045863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126b6370 0x5571126b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.585+0000 7fe523ab0740 1 --2- 172.21.15.152:0/1477045863 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5571126c5ed0 0x5571126c82c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.585+0000 7fe523ab0740 1 --2- 172.21.15.152:0/1477045863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126acb40 0x5571126b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.585+0000 7fe4feff5640 0 client.17310 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:59:47.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.585+0000 7fe523ab0740 1 --2- 172.21.15.152:0/1477045863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126ab5d0 0x5571126b5d30 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.585+0000 7fe523ab0740 1 -- 172.21.15.152:0/1477045863 >> 172.21.15.152:0/1477045863 conn(0x557112674310 msgr2=0x5571126ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:47.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.586+0000 7fe523ab0740 1 -- 172.21.15.152:0/1477045863 shutdown_connections 2024-09-11T22:59:47.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.586+0000 7fe523ab0740 1 -- 172.21.15.152:0/1477045863 wait complete. 2024-09-11T22:59:47.605 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.586+0000 7fe523ab0740 20 client.17310 trim_cache size 0 max 16384 2024-09-11T22:59:47.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.598+0000 7fe523ab0740 1 Processor -- start 2024-09-11T22:59:47.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.598+0000 7fe523ab0740 1 -- start start 2024-09-11T22:59:47.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe523ab0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126ab5d0 0x5571126ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe523ab0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126acb40 0x5571126ae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe523ab0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126b6370 0x5571126cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe523ab0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1ad8b0 con 0x5571126acb40 2024-09-11T22:59:47.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe523ab0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1af3f0 con 0x5571126b6370 2024-09-11T22:59:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe523ab0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1adf80 con 0x5571126ab5d0 2024-09-11T22:59:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe51affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126acb40 0x5571126ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe51affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126acb40 0x5571126ae660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38722/0 (socket says 172.21.15.152:38722) 2024-09-11T22:59:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe51affd640 1 -- 172.21.15.152:0/1975080776 learned_addr learned my addr 172.21.15.152:0/1975080776 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe51a7fc640 1 --2- 172.21.15.152:0/1975080776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126ab5d0 0x5571126ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe51a7fc640 1 --2- 172.21.15.152:0/1975080776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126b6370 0x5571126cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe51affd640 1 -- 172.21.15.152:0/1975080776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126ab5d0 msgr2=0x5571126ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe51affd640 1 --2- 172.21.15.152:0/1975080776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126ab5d0 0x5571126ae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.612 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe51affd640 1 -- 172.21.15.152:0/1975080776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126b6370 msgr2=0x5571126cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe51affd640 1 --2- 172.21.15.152:0/1975080776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126b6370 0x5571126cd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe51affd640 1 -- 172.21.15.152:0/1975080776 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571126a8960 con 0x5571126acb40 2024-09-11T22:59:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe51affd640 1 --2- 172.21.15.152:0/1975080776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126acb40 0x5571126ae660 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009900 tx=0x7fe50c032e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38de3680264b96e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.599+0000 7fe500ff9640 1 -- 172.21.15.152:0/1975080776 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c035040 con 0x5571126acb40 2024-09-11T22:59:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.600+0000 7fe500ff9640 1 -- 172.21.15.152:0/1975080776 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe50c046020 con 0x5571126acb40 2024-09-11T22:59:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.600+0000 7fe500ff9640 1 -- 172.21.15.152:0/1975080776 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c0046c0 con 0x5571126acb40 2024-09-11T22:59:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.600+0000 7fe523ab0740 1 -- 172.21.15.152:0/1975080776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126acb40 msgr2=0x5571126ae660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.600+0000 7fe523ab0740 1 --2- 172.21.15.152:0/1975080776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126acb40 0x5571126ae660 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009900 tx=0x7fe50c032e00 comp rx=0 tx=0).stop 2024-09-11T22:59:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.600+0000 7fe523ab0740 1 -- 172.21.15.152:0/1975080776 shutdown_connections 2024-09-11T22:59:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.600+0000 7fe523ab0740 1 --2- 172.21.15.152:0/1975080776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126b6370 0x5571126cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.600+0000 7fe523ab0740 1 --2- 172.21.15.152:0/1975080776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126acb40 0x5571126ae660 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.613 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.600+0000 7fe523ab0740 1 --2- 172.21.15.152:0/1975080776 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126ab5d0 0x5571126ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.600+0000 7fe523ab0740 1 -- 172.21.15.152:0/1975080776 >> 172.21.15.152:0/1975080776 conn(0x557112674310 msgr2=0x5571126a6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.600+0000 7fe523ab0740 1 -- 172.21.15.152:0/1975080776 shutdown_connections 2024-09-11T22:59:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.600+0000 7fe523ab0740 1 -- 172.21.15.152:0/1975080776 wait complete. 2024-09-11T22:59:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe523ab0740 1 -- start start 2024-09-11T22:59:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe523ab0740 1 Processor -- start 2024-09-11T22:59:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe523ab0740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126ab5d0 0x5571126ba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe523ab0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126acb40 0x5571126ba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe523ab0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126b6370 0x5571126bae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe523ab0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1aed20 con 0x5571126b6370 2024-09-11T22:59:47.614 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe523ab0740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1ae650 con 0x5571126ab5d0 2024-09-11T22:59:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe523ab0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1ab870 con 0x5571126acb40 2024-09-11T22:59:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe523ab0740 10 client.0 _ll_register_callbacks cb 0x7fe50c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe51a7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126b6370 0x5571126bae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe51a7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126b6370 0x5571126bae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:38726/0 (socket says 172.21.15.152:38726) 2024-09-11T22:59:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe51a7fc640 1 -- 172.21.15.152:0/1715920408 learned_addr learned my addr 172.21.15.152:0/1715920408 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe51a7fc640 1 --2- 172.21.15.152:0/1715920408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126ab5d0 0x5571126ba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe51affd640 1 --2- 172.21.15.152:0/1715920408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126acb40 0x5571126ba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe51a7fc640 1 -- 172.21.15.152:0/1715920408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126acb40 msgr2=0x5571126ba8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:47.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe51a7fc640 1 --2- 172.21.15.152:0/1715920408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126acb40 0x5571126ba8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe51a7fc640 1 -- 172.21.15.152:0/1715920408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126ab5d0 msgr2=0x5571126ba3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:47.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe51a7fc640 1 --2- 172.21.15.152:0/1715920408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126ab5d0 0x5571126ba3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.601+0000 7fe51a7fc640 1 -- 172.21.15.152:0/1715920408 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55711272fa90 con 0x5571126b6370 2024-09-11T22:59:47.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.602+0000 7fe51a7fc640 1 --2- 172.21.15.152:0/1715920408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126b6370 0x5571126bae20 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe51000bc60 tx=0x7fe510000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9662ecac97b7152a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:47.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.602+0000 7fe4feff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:47.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.602+0000 7fe4feff5640 1 -- 172.21.15.152:0/1715920408 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510022070 con 0x5571126b6370 2024-09-11T22:59:47.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.602+0000 7fe4feff5640 1 -- 172.21.15.152:0/1715920408 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fe510011750 con 0x5571126b6370 2024-09-11T22:59:47.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.602+0000 7fe523ab0740 1 -- 172.21.15.152:0/1715920408 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe50c046020 con 0x5571126b6370 2024-09-11T22:59:47.616 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.602+0000 7fe4feff5640 1 -- 172.21.15.152:0/1715920408 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe510048050 con 0x5571126b6370 2024-09-11T22:59:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.602+0000 7fe523ab0740 10 client.17316 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.602+0000 7fe523ab0740 1 -- 172.21.15.152:0/1715920408 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557112730d30 con 0x5571126b6370 2024-09-11T22:59:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.602+0000 7fe4feff5640 1 -- 172.21.15.152:0/1715920408 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe50c046020 con 0x5571126b6370 2024-09-11T22:59:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.602+0000 7fe4feff5640 1 -- 172.21.15.152:0/1715920408 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe51001e530 con 0x5571126b6370 2024-09-11T22:59:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.602+0000 7fe523ab0740 10 client.17316 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.602+0000 7fe523ab0740 10 client.17316 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T22:59:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.602+0000 7fe523ab0740 20 client.17316 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.602+0000 7fe523ab0740 1 --2- 172.21.15.152:0/1715920408 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5571127313f0 0x5571127337e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.602+0000 7fe523ab0740 4 client.17316 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:59:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.602+0000 7fe523ab0740 1 -- 172.21.15.152:0/1715920408 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557112618d70 con 0x5571127313f0 2024-09-11T22:59:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe51affd640 1 --2- 172.21.15.152:0/1715920408 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5571127313f0 0x5571127337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe4feff5640 10 client.17316 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T22:59:47.617 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe51affd640 1 --2- 172.21.15.152:0/1715920408 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5571127313f0 0x5571127337e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e65c4ba5d1674cc4 server_cookie=fb6f921f460a2fa7 in_seq=0 out_seq=0 2024-09-11T22:59:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe4feff5640 1 -- 172.21.15.152:0/1715920408 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557112618d70 con 0x5571127313f0 2024-09-11T22:59:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe4feff5640 10 client.17316 handle_command_reply: tid=0 2024-09-11T22:59:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe523ab0740 1 client.17316 shutdown 2024-09-11T22:59:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe523ab0740 1 -- 172.21.15.152:0/1715920408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126b6370 msgr2=0x5571126bae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe523ab0740 1 --2- 172.21.15.152:0/1715920408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126b6370 0x5571126bae20 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe51000bc60 tx=0x7fe510000bc0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe523ab0740 1 -- 172.21.15.152:0/1715920408 shutdown_connections 2024-09-11T22:59:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe523ab0740 1 --2- 172.21.15.152:0/1715920408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5571126b6370 0x5571126bae20 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe523ab0740 1 --2- 172.21.15.152:0/1715920408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5571126acb40 0x5571126ba8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe523ab0740 1 --2- 172.21.15.152:0/1715920408 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5571127313f0 0x5571127337e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe523ab0740 1 --2- 172.21.15.152:0/1715920408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5571126ab5d0 0x5571126ba3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe523ab0740 1 -- 172.21.15.152:0/1715920408 >> 172.21.15.152:0/1715920408 conn(0x557112674310 msgr2=0x5571126b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe4feff5640 0 client.17316 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T22:59:47.618 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe523ab0740 1 -- 172.21.15.152:0/1715920408 shutdown_connections 2024-09-11T22:59:47.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe523ab0740 1 -- 172.21.15.152:0/1715920408 wait complete. 2024-09-11T22:59:47.619 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.603+0000 7fe523ab0740 20 client.17316 trim_cache size 0 max 16384 2024-09-11T22:59:47.619 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:59:47.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.604+0000 7fe5216d7640 1 -- 172.21.15.152:0/175333245 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5040a2d70 msgr2=0x7fe5040a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:47.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.604+0000 7fe5216d7640 1 --2- 172.21.15.152:0/175333245 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5040a2d70 0x7fe5040a5230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe51400efd0 tx=0x7fe5140b6040 comp rx=0 tx=0).stop 2024-09-11T22:59:47.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.604+0000 7fe5216d7640 1 -- 172.21.15.152:0/175333245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe51c106a80 msgr2=0x7fe51c19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:47.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.604+0000 7fe5216d7640 1 --2- 172.21.15.152:0/175333245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe51c106a80 0x7fe51c19cd50 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe51400eaa0 tx=0x7fe51400ef60 comp rx=0 tx=0).stop 2024-09-11T22:59:47.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.604+0000 7fe5216d7640 1 -- 172.21.15.152:0/175333245 shutdown_connections 2024-09-11T22:59:47.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.604+0000 7fe5216d7640 1 --2- 172.21.15.152:0/175333245 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5040a2d70 0x7fe5040a5230 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.604+0000 7fe5216d7640 1 --2- 172.21.15.152:0/175333245 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe51c106a80 0x7fe51c19cd50 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.604+0000 7fe5216d7640 1 --2- 172.21.15.152:0/175333245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe51c101420 0x7fe51c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.604+0000 7fe5216d7640 1 --2- 172.21.15.152:0/175333245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c100a80 0x7fe51c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:47.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.604+0000 7fe5216d7640 1 -- 172.21.15.152:0/175333245 >> 172.21.15.152:0/175333245 conn(0x7fe51c0fc7a0 msgr2=0x7fe51c06a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:47.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.604+0000 7fe5216d7640 1 -- 172.21.15.152:0/175333245 shutdown_connections 2024-09-11T22:59:47.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:47.604+0000 7fe5216d7640 1 -- 172.21.15.152:0/175333245 wait complete. 2024-09-11T22:59:48.414 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:59:48.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:48 smithi152 ceph-mon[31181]: pgmap v210: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:59:48.661 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:48.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:48 smithi191 ceph-mon[35211]: pgmap v210: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:59:48.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:48 smithi204 ceph-mon[35454]: pgmap v210: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:59:49.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.190+0000 7f71c0f84640 1 -- 172.21.15.152:0/3564570164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71bc102ef0 msgr2=0x7f71bc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:49.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.190+0000 7f71c0f84640 1 --2- 172.21.15.152:0/3564570164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71bc102ef0 0x7f71bc10da40 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009ca0 tx=0x7f71b402a950 comp rx=0 tx=0).stop 2024-09-11T22:59:49.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.190+0000 7f71c0f84640 1 -- 172.21.15.152:0/3564570164 shutdown_connections 2024-09-11T22:59:49.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.190+0000 7f71c0f84640 1 --2- 172.21.15.152:0/3564570164 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71bc102ef0 0x7f71bc10da40 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.190+0000 7f71c0f84640 1 --2- 172.21.15.152:0/3564570164 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71bc102570 0x7f71bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.190+0000 7f71c0f84640 1 --2- 172.21.15.152:0/3564570164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71bc100e20 0x7f71bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.190+0000 7f71c0f84640 1 -- 172.21.15.152:0/3564570164 >> 172.21.15.152:0/3564570164 conn(0x7f71bc0fc7d0 msgr2=0x7f71bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:49.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.190+0000 7f71c0f84640 1 -- 172.21.15.152:0/3564570164 shutdown_connections 2024-09-11T22:59:49.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.190+0000 7f71c0f84640 1 -- 172.21.15.152:0/3564570164 wait complete. 2024-09-11T22:59:49.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71c0f84640 1 Processor -- start 2024-09-11T22:59:49.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71c0f84640 1 -- start start 2024-09-11T22:59:49.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71c0f84640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71bc100e20 0x7f71bc197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71c0f84640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71bc102570 0x7f71bc197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71c0f84640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71bc102ef0 0x7f71bc19bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71c0f84640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc110f80 con 0x7f71bc102570 2024-09-11T22:59:49.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71c0f84640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc110e00 con 0x7f71bc100e20 2024-09-11T22:59:49.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71c0f84640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc111100 con 0x7f71bc102ef0 2024-09-11T22:59:49.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71bc102570 0x7f71bc197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71bc102570 0x7f71bc197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57454/0 (socket says 172.21.15.152:57454) 2024-09-11T22:59:49.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71b9d9b640 1 -- 172.21.15.152:0/2124304707 learned_addr learned my addr 172.21.15.152:0/2124304707 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:49.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71bad9d640 1 --2- 172.21.15.152:0/2124304707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71bc102ef0 0x7f71bc19bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71ba59c640 1 --2- 172.21.15.152:0/2124304707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71bc100e20 0x7f71bc197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71b9d9b640 1 -- 172.21.15.152:0/2124304707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71bc102ef0 msgr2=0x7f71bc19bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:49.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71b9d9b640 1 --2- 172.21.15.152:0/2124304707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71bc102ef0 0x7f71bc19bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71b9d9b640 1 -- 172.21.15.152:0/2124304707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71bc100e20 msgr2=0x7f71bc197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:49.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.191+0000 7f71b9d9b640 1 --2- 172.21.15.152:0/2124304707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71bc100e20 0x7f71bc197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.192+0000 7f71b9d9b640 1 -- 172.21.15.152:0/2124304707 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc19c2b0 con 0x7f71bc102570 2024-09-11T22:59:49.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.192+0000 7f71ba59c640 1 --2- 172.21.15.152:0/2124304707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71bc100e20 0x7f71bc197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:59:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.192+0000 7f71b9d9b640 1 --2- 172.21.15.152:0/2124304707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71bc102570 0x7f71bc197840 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f71b000c920 tx=0x7f71b000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.192+0000 7f71ab7fe640 1 -- 172.21.15.152:0/2124304707 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0007c70 con 0x7f71bc102570 2024-09-11T22:59:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.192+0000 7f71ab7fe640 1 -- 172.21.15.152:0/2124304707 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b0007e10 con 0x7f71bc102570 2024-09-11T22:59:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.192+0000 7f71c0f84640 1 -- 172.21.15.152:0/2124304707 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71bc19c5a0 con 0x7f71bc102570 2024-09-11T22:59:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.192+0000 7f71ab7fe640 1 -- 172.21.15.152:0/2124304707 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0018890 con 0x7f71bc102570 2024-09-11T22:59:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.192+0000 7f71c0f84640 1 -- 172.21.15.152:0/2124304707 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71bc1a3e80 con 0x7f71bc102570 2024-09-11T22:59:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.194+0000 7f71ab7fe640 1 -- 172.21.15.152:0/2124304707 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f71b0018a70 con 0x7f71bc102570 2024-09-11T22:59:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.194+0000 7f71c335d740 1 Processor -- start 2024-09-11T22:59:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71c335d740 1 -- start start 2024-09-11T22:59:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71c335d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce3eb40 0x5570fce3d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71c335d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3d5d0 0x5570fce3da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71c335d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce4a580 0x5570fce4c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71c335d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5570fcda32f0 con 0x5570fce3d5d0 2024-09-11T22:59:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71c335d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5570fcd4acf0 con 0x5570fce3eb40 2024-09-11T22:59:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71c335d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5570fcd9f2b0 con 0x5570fce4a580 2024-09-11T22:59:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71ab7fe640 1 --2- 172.21.15.152:0/2124304707 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f71a40a2cf0 0x7f71a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71ba59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3d5d0 0x5570fce3da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71ba59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3d5d0 0x5570fce3da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57460/0 (socket says 172.21.15.152:57460) 2024-09-11T22:59:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71ba59c640 1 -- 172.21.15.152:0/822893607 learned_addr learned my addr 172.21.15.152:0/822893607 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71bad9d640 1 --2- 172.21.15.152:0/2124304707 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f71a40a2cf0 0x7f71a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71b9d9b640 1 --2- 172.21.15.152:0/822893607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce4a580 0x5570fce4c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71bad9d640 1 --2- 172.21.15.152:0/822893607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce3eb40 0x5570fce3d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71ab7fe640 1 -- 172.21.15.152:0/2124304707 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f71b00cf600 con 0x7f71bc102570 2024-09-11T22:59:49.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71ba59c640 1 -- 172.21.15.152:0/822893607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce4a580 msgr2=0x5570fce4c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:49.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71ba59c640 1 --2- 172.21.15.152:0/822893607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce4a580 0x5570fce4c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71ba59c640 1 -- 172.21.15.152:0/822893607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce3eb40 msgr2=0x5570fce3d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:49.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71ba59c640 1 --2- 172.21.15.152:0/822893607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce3eb40 0x5570fce3d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71ba59c640 1 -- 172.21.15.152:0/822893607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570fcd2f3a0 con 0x5570fce3d5d0 2024-09-11T22:59:49.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71b9d9b640 1 --2- 172.21.15.152:0/822893607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce4a580 0x5570fce4c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:49.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.195+0000 7f71ba59c640 1 --2- 172.21.15.152:0/822893607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3d5d0 0x5570fce3da10 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00bb70 tx=0x7f71ac00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba7b57f505d2090 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:49.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.196+0000 7f71bad9d640 1 --2- 172.21.15.152:0/2124304707 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f71a40a2cf0 0x7f71a40a51b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f71b4005fa0 tx=0x7f71b4005ef0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:49.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.196+0000 7f71a8ff9640 1 -- 172.21.15.152:0/822893607 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac017620 con 0x5570fce3d5d0 2024-09-11T22:59:49.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.196+0000 7f71a8ff9640 1 -- 172.21.15.152:0/822893607 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac017ca0 con 0x5570fce3d5d0 2024-09-11T22:59:49.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.196+0000 7f71a8ff9640 1 -- 172.21.15.152:0/822893607 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac00fc00 con 0x5570fce3d5d0 2024-09-11T22:59:49.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.196+0000 7f71c335d740 1 -- 172.21.15.152:0/822893607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3d5d0 msgr2=0x5570fce3da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:49.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.196+0000 7f71c335d740 1 --2- 172.21.15.152:0/822893607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3d5d0 0x5570fce3da10 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00bb70 tx=0x7f71ac00d710 comp rx=0 tx=0).stop 2024-09-11T22:59:49.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.196+0000 7f71c335d740 1 -- 172.21.15.152:0/822893607 shutdown_connections 2024-09-11T22:59:49.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.196+0000 7f71c335d740 1 --2- 172.21.15.152:0/822893607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce4a580 0x5570fce4c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71c335d740 1 --2- 172.21.15.152:0/822893607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3d5d0 0x5570fce3da10 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71c335d740 1 --2- 172.21.15.152:0/822893607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce3eb40 0x5570fce3d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71c335d740 1 -- 172.21.15.152:0/822893607 >> 172.21.15.152:0/822893607 conn(0x5570fce06310 msgr2=0x5570fce39c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:49.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71c335d740 1 -- 172.21.15.152:0/822893607 shutdown_connections 2024-09-11T22:59:49.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71c335d740 1 -- 172.21.15.152:0/822893607 wait complete. 2024-09-11T22:59:49.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71c335d740 1 -- start start 2024-09-11T22:59:49.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71c335d740 1 Processor -- start 2024-09-11T22:59:49.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71c335d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce3d5d0 0x5570fce49ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71c335d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71c335d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce4a580 0x5570fce47700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71c335d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5570fce4d280 con 0x5570fce3eb40 2024-09-11T22:59:49.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71c335d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5570fcd5e060 con 0x5570fce4a580 2024-09-11T22:59:49.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71c335d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5570fce4d400 con 0x5570fce3d5d0 2024-09-11T22:59:49.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71c335d740 10 client.0 _ll_register_callbacks cb 0x5570fcab18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:49.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57462/0 (socket says 172.21.15.152:57462) 2024-09-11T22:59:49.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71b9d9b640 1 -- 172.21.15.152:0/1011450925 learned_addr learned my addr 172.21.15.152:0/1011450925 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:49.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.197+0000 7f71ba59c640 1 --2- 172.21.15.152:0/1011450925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce3d5d0 0x5570fce49ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71bad9d640 1 --2- 172.21.15.152:0/1011450925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce4a580 0x5570fce47700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71b9d9b640 1 -- 172.21.15.152:0/1011450925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce3d5d0 msgr2=0x5570fce49ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:49.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71b9d9b640 1 --2- 172.21.15.152:0/1011450925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce3d5d0 0x5570fce49ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71b9d9b640 1 -- 172.21.15.152:0/1011450925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce4a580 msgr2=0x5570fce47700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:49.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71b9d9b640 1 --2- 172.21.15.152:0/1011450925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce4a580 0x5570fce47700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71b9d9b640 1 -- 172.21.15.152:0/1011450925 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570fcb143c0 con 0x5570fce3eb40 2024-09-11T22:59:49.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71bad9d640 1 --2- 172.21.15.152:0/1011450925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce4a580 0x5570fce47700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:49.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71b9d9b640 1 --2- 172.21.15.152:0/1011450925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce471a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f71b0097fd0 tx=0x7f71b0066b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181096f9720daaf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:49.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:49.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71a1ffb640 1 -- 172.21.15.152:0/1011450925 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b006a030 con 0x5570fce3eb40 2024-09-11T22:59:49.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71a1ffb640 1 -- 172.21.15.152:0/1011450925 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f71b00cd020 con 0x5570fce3eb40 2024-09-11T22:59:49.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71c335d740 1 -- 172.21.15.152:0/1011450925 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ac017ca0 con 0x5570fce3eb40 2024-09-11T22:59:49.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71a1ffb640 1 -- 172.21.15.152:0/1011450925 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71b00cd220 con 0x5570fce3eb40 2024-09-11T22:59:49.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71c335d740 10 client.17328 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:49.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71c335d740 1 -- 172.21.15.152:0/1011450925 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570fce58c60 con 0x5570fce3eb40 2024-09-11T22:59:49.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71a1ffb640 1 -- 172.21.15.152:0/1011450925 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac017ca0 con 0x5570fce3eb40 2024-09-11T22:59:49.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71a1ffb640 1 -- 172.21.15.152:0/1011450925 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f71b00662d0 con 0x5570fce3eb40 2024-09-11T22:59:49.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71c335d740 10 client.17328 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:49.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.198+0000 7f71c335d740 10 client.17328 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:59:49.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.199+0000 7f71c335d740 20 client.17328 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:49.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.199+0000 7f71c335d740 1 --2- 172.21.15.152:0/1011450925 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5570fce5a120 0x5570fce5c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.199+0000 7f71c335d740 4 client.17328 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:59:49.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.199+0000 7f71c335d740 1 -- 172.21.15.152:0/1011450925 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570fcdb0860 con 0x5570fce5a120 2024-09-11T22:59:49.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.199+0000 7f71ba59c640 1 --2- 172.21.15.152:0/1011450925 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5570fce5a120 0x5570fce5c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.199+0000 7f71a1ffb640 10 client.17328 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:59:49.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.199+0000 7f71ba59c640 1 --2- 172.21.15.152:0/1011450925 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5570fce5a120 0x5570fce5c510 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=22be61f4f72956c2 server_cookie=b025e47fcee44ee0 in_seq=0 out_seq=0 2024-09-11T22:59:49.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.200+0000 7f71a1ffb640 1 -- 172.21.15.152:0/1011450925 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570fcdb0860 con 0x5570fce5a120 2024-09-11T22:59:49.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.200+0000 7f71a1ffb640 10 client.17328 handle_command_reply: tid=0 2024-09-11T22:59:49.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.200+0000 7f71c335d740 1 client.17328 shutdown 2024-09-11T22:59:49.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.200+0000 7f71c335d740 1 -- 172.21.15.152:0/1011450925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 msgr2=0x5570fce471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:49.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.200+0000 7f71c335d740 1 --2- 172.21.15.152:0/1011450925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce471a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f71b0097fd0 tx=0x7f71b0066b90 comp rx=0 tx=0).stop 2024-09-11T22:59:49.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.200+0000 7f71c335d740 1 -- 172.21.15.152:0/1011450925 shutdown_connections 2024-09-11T22:59:49.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.200+0000 7f71c335d740 1 --2- 172.21.15.152:0/1011450925 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5570fce5a120 0x5570fce5c510 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.200+0000 7f71c335d740 1 --2- 172.21.15.152:0/1011450925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce4a580 0x5570fce47700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.200+0000 7f71c335d740 1 --2- 172.21.15.152:0/1011450925 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce471a0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.200+0000 7f71c335d740 1 --2- 172.21.15.152:0/1011450925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce3d5d0 0x5570fce49ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.200+0000 7f71c335d740 1 -- 172.21.15.152:0/1011450925 >> 172.21.15.152:0/1011450925 conn(0x5570fce06310 msgr2=0x5570fce3f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:49.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.200+0000 7f71a1ffb640 0 client.17328 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:59:49.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.200+0000 7f71c335d740 1 -- 172.21.15.152:0/1011450925 shutdown_connections 2024-09-11T22:59:49.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.200+0000 7f71c335d740 1 -- 172.21.15.152:0/1011450925 wait complete. 2024-09-11T22:59:49.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.200+0000 7f71c335d740 20 client.17328 trim_cache size 0 max 16384 2024-09-11T22:59:49.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71c335d740 1 Processor -- start 2024-09-11T22:59:49.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71c335d740 1 -- start start 2024-09-11T22:59:49.230 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71c335d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce3d5d0 0x5570fce56520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71c335d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce39700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71c335d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce4a580 0x5570fce39c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71c335d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1ab4a0 con 0x5570fce3eb40 2024-09-11T22:59:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71c335d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1acfe0 con 0x5570fce3d5d0 2024-09-11T22:59:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71c335d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1abb70 con 0x5570fce4a580 2024-09-11T22:59:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce39700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce39700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57476/0 (socket says 172.21.15.152:57476) 2024-09-11T22:59:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71b9d9b640 1 -- 172.21.15.152:0/1198035484 learned_addr learned my addr 172.21.15.152:0/1198035484 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71ba59c640 1 --2- 172.21.15.152:0/1198035484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce3d5d0 0x5570fce56520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71bad9d640 1 --2- 172.21.15.152:0/1198035484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce4a580 0x5570fce39c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71b9d9b640 1 -- 172.21.15.152:0/1198035484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce4a580 msgr2=0x5570fce39c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71b9d9b640 1 --2- 172.21.15.152:0/1198035484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce4a580 0x5570fce39c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71b9d9b640 1 -- 172.21.15.152:0/1198035484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce3d5d0 msgr2=0x5570fce56520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:49.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71b9d9b640 1 --2- 172.21.15.152:0/1198035484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce3d5d0 0x5570fce56520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71b9d9b640 1 -- 172.21.15.152:0/1198035484 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570fce40120 con 0x5570fce3eb40 2024-09-11T22:59:49.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.214+0000 7f71ba59c640 1 --2- 172.21.15.152:0/1198035484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce3d5d0 0x5570fce56520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:49.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.215+0000 7f71b9d9b640 1 --2- 172.21.15.152:0/1198035484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce39700 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f71b00a48d0 tx=0x7f71b0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a7c238d0e4ba19 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:49.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.215+0000 7f71a3fff640 1 -- 172.21.15.152:0/1198035484 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b000de20 con 0x5570fce3eb40 2024-09-11T22:59:49.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.215+0000 7f71a3fff640 1 -- 172.21.15.152:0/1198035484 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b0064030 con 0x5570fce3eb40 2024-09-11T22:59:49.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.215+0000 7f71a3fff640 1 -- 172.21.15.152:0/1198035484 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b00cd020 con 0x5570fce3eb40 2024-09-11T22:59:49.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.216+0000 7f71c335d740 1 -- 172.21.15.152:0/1198035484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 msgr2=0x5570fce39700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:49.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.216+0000 7f71c335d740 1 --2- 172.21.15.152:0/1198035484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce39700 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f71b00a48d0 tx=0x7f71b0020000 comp rx=0 tx=0).stop 2024-09-11T22:59:49.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.216+0000 7f71c335d740 1 -- 172.21.15.152:0/1198035484 shutdown_connections 2024-09-11T22:59:49.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.216+0000 7f71c335d740 1 --2- 172.21.15.152:0/1198035484 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce4a580 0x5570fce39c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.216+0000 7f71c335d740 1 --2- 172.21.15.152:0/1198035484 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce39700 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.216+0000 7f71c335d740 1 --2- 172.21.15.152:0/1198035484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce3d5d0 0x5570fce56520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.216+0000 7f71c335d740 1 -- 172.21.15.152:0/1198035484 >> 172.21.15.152:0/1198035484 conn(0x5570fce06310 msgr2=0x5570fce47160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.216+0000 7f71c335d740 1 -- 172.21.15.152:0/1198035484 shutdown_connections 2024-09-11T22:59:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.216+0000 7f71c335d740 1 -- 172.21.15.152:0/1198035484 wait complete. 2024-09-11T22:59:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.216+0000 7f71c335d740 1 -- start start 2024-09-11T22:59:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.216+0000 7f71c335d740 1 Processor -- start 2024-09-11T22:59:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71c335d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce3d5d0 0x5570fce52d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71c335d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce43e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71c335d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce4a580 0x5570fce44340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71c335d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1ac910 con 0x5570fce3eb40 2024-09-11T22:59:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71c335d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1ac240 con 0x5570fce4a580 2024-09-11T22:59:49.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71c335d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc1a9480 con 0x5570fce3d5d0 2024-09-11T22:59:49.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71c335d740 10 client.0 _ll_register_callbacks cb 0x7f71b000c870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:49.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce43e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71b9d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce43e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57492/0 (socket says 172.21.15.152:57492) 2024-09-11T22:59:49.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71b9d9b640 1 -- 172.21.15.152:0/238104329 learned_addr learned my addr 172.21.15.152:0/238104329 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:49.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71ba59c640 1 --2- 172.21.15.152:0/238104329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce3d5d0 0x5570fce52d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71bad9d640 1 --2- 172.21.15.152:0/238104329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce4a580 0x5570fce44340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71b9d9b640 1 -- 172.21.15.152:0/238104329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce3d5d0 msgr2=0x5570fce52d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:49.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71b9d9b640 1 --2- 172.21.15.152:0/238104329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce3d5d0 0x5570fce52d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71b9d9b640 1 -- 172.21.15.152:0/238104329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce4a580 msgr2=0x5570fce44340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:49.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71b9d9b640 1 --2- 172.21.15.152:0/238104329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce4a580 0x5570fce44340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71b9d9b640 1 -- 172.21.15.152:0/238104329 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570fce44b20 con 0x5570fce3eb40 2024-09-11T22:59:49.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71b9d9b640 1 --2- 172.21.15.152:0/238104329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce43e00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f71b0099e20 tx=0x7f71b009b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ca2447f103981 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:49.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.217+0000 7f71a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.218+0000 7f71a1ffb640 1 -- 172.21.15.152:0/238104329 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b000de20 con 0x5570fce3eb40 2024-09-11T22:59:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.218+0000 7f71a1ffb640 1 -- 172.21.15.152:0/238104329 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f71b006b050 con 0x5570fce3eb40 2024-09-11T22:59:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.218+0000 7f71c335d740 1 -- 172.21.15.152:0/238104329 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71b0064030 con 0x5570fce3eb40 2024-09-11T22:59:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.218+0000 7f71a1ffb640 1 -- 172.21.15.152:0/238104329 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f71b000fb30 con 0x5570fce3eb40 2024-09-11T22:59:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.218+0000 7f71c335d740 10 client.17334 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.218+0000 7f71c335d740 1 -- 172.21.15.152:0/238104329 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5570fcec4d60 con 0x5570fce3eb40 2024-09-11T22:59:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.218+0000 7f71a1ffb640 1 -- 172.21.15.152:0/238104329 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71b0064030 con 0x5570fce3eb40 2024-09-11T22:59:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.218+0000 7f71a1ffb640 1 -- 172.21.15.152:0/238104329 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f71b00ba2e0 con 0x5570fce3eb40 2024-09-11T22:59:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.218+0000 7f71c335d740 10 client.17334 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.218+0000 7f71c335d740 10 client.17334 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T22:59:49.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.218+0000 7f71c335d740 20 client.17334 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:49.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.218+0000 7f71c335d740 1 --2- 172.21.15.152:0/238104329 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5570fcec5170 0x5570fcec7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:49.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.218+0000 7f71c335d740 4 client.17334 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:59:49.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.218+0000 7f71c335d740 1 -- 172.21.15.152:0/238104329 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570fcdaad70 con 0x5570fcec5170 2024-09-11T22:59:49.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.218+0000 7f71ba59c640 1 --2- 172.21.15.152:0/238104329 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5570fcec5170 0x5570fcec7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:49.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71a1ffb640 10 client.17334 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T22:59:49.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71ba59c640 1 --2- 172.21.15.152:0/238104329 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5570fcec5170 0x5570fcec7590 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ff8785e536b4fd16 server_cookie=251c935159a5b1c7 in_seq=0 out_seq=0 2024-09-11T22:59:49.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71a1ffb640 1 -- 172.21.15.152:0/238104329 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570fcdaad70 con 0x5570fcec5170 2024-09-11T22:59:49.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71a1ffb640 10 client.17334 handle_command_reply: tid=0 2024-09-11T22:59:49.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71c335d740 1 client.17334 shutdown 2024-09-11T22:59:49.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71c335d740 1 -- 172.21.15.152:0/238104329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 msgr2=0x5570fce43e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:49.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71c335d740 1 --2- 172.21.15.152:0/238104329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce43e00 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f71b0099e20 tx=0x7f71b009b4d0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71c335d740 1 -- 172.21.15.152:0/238104329 shutdown_connections 2024-09-11T22:59:49.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71c335d740 1 --2- 172.21.15.152:0/238104329 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5570fcec5170 0x5570fcec7590 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71c335d740 1 --2- 172.21.15.152:0/238104329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5570fce4a580 0x5570fce44340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71c335d740 1 --2- 172.21.15.152:0/238104329 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5570fce3eb40 0x5570fce43e00 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71a1ffb640 0 client.17334 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T22:59:49.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71c335d740 1 --2- 172.21.15.152:0/238104329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570fce3d5d0 0x5570fce52d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71c335d740 1 -- 172.21.15.152:0/238104329 >> 172.21.15.152:0/238104329 conn(0x5570fce06310 msgr2=0x5570fce39700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:49.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71c335d740 1 -- 172.21.15.152:0/238104329 shutdown_connections 2024-09-11T22:59:49.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71c335d740 1 -- 172.21.15.152:0/238104329 wait complete. 2024-09-11T22:59:49.237 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.219+0000 7f71c335d740 20 client.17334 trim_cache size 0 max 16384 2024-09-11T22:59:49.237 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:59:49.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.220+0000 7f71c0f84640 1 -- 172.21.15.152:0/2124304707 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f71a40a2cf0 msgr2=0x7f71a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:49.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.220+0000 7f71c0f84640 1 --2- 172.21.15.152:0/2124304707 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f71a40a2cf0 0x7f71a40a51b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f71b4005fa0 tx=0x7f71b4005ef0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.220+0000 7f71c0f84640 1 -- 172.21.15.152:0/2124304707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71bc102570 msgr2=0x7f71bc197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:49.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.220+0000 7f71c0f84640 1 --2- 172.21.15.152:0/2124304707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71bc102570 0x7f71bc197840 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f71b000c920 tx=0x7f71b000cde0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.238 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.220+0000 7f71c0f84640 1 -- 172.21.15.152:0/2124304707 shutdown_connections 2024-09-11T22:59:49.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.220+0000 7f71c0f84640 1 --2- 172.21.15.152:0/2124304707 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f71a40a2cf0 0x7f71a40a51b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.220+0000 7f71c0f84640 1 --2- 172.21.15.152:0/2124304707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71bc102ef0 0x7f71bc19bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.220+0000 7f71c0f84640 1 --2- 172.21.15.152:0/2124304707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f71bc102570 0x7f71bc197840 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.220+0000 7f71c0f84640 1 --2- 172.21.15.152:0/2124304707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71bc100e20 0x7f71bc197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:49.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.220+0000 7f71c0f84640 1 -- 172.21.15.152:0/2124304707 >> 172.21.15.152:0/2124304707 conn(0x7f71bc0fc7d0 msgr2=0x7f71bc1067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:49.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.220+0000 7f71c0f84640 1 -- 172.21.15.152:0/2124304707 shutdown_connections 2024-09-11T22:59:49.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:49.220+0000 7f71c0f84640 1 -- 172.21.15.152:0/2124304707 wait complete. 2024-09-11T22:59:49.944 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:59:50.192 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:50.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:50 smithi152 ceph-mon[31181]: pgmap v211: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 12 op/s 2024-09-11T22:59:50.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.710+0000 7fbff57dd640 1 -- 172.21.15.152:0/1075763655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbff0106f50 msgr2=0x7fbff010eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:50.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.710+0000 7fbff57dd640 1 --2- 172.21.15.152:0/1075763655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbff0106f50 0x7fbff010eaf0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009cd0 tx=0x7fbfe802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.710+0000 7fbff57dd640 1 -- 172.21.15.152:0/1075763655 shutdown_connections 2024-09-11T22:59:50.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.710+0000 7fbff57dd640 1 --2- 172.21.15.152:0/1075763655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbff0106f50 0x7fbff010eaf0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.710+0000 7fbff57dd640 1 --2- 172.21.15.152:0/1075763655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbff0100ca0 0x7fbff0106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.710+0000 7fbff57dd640 1 --2- 172.21.15.152:0/1075763655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbff01002f0 0x7fbff01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.710+0000 7fbff57dd640 1 -- 172.21.15.152:0/1075763655 >> 172.21.15.152:0/1075763655 conn(0x7fbff0077210 msgr2=0x7fbff00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:50.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.710+0000 7fbff57dd640 1 -- 172.21.15.152:0/1075763655 shutdown_connections 2024-09-11T22:59:50.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.710+0000 7fbff57dd640 1 -- 172.21.15.152:0/1075763655 wait complete. 2024-09-11T22:59:50.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.710+0000 7fbff57dd640 1 Processor -- start 2024-09-11T22:59:50.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.710+0000 7fbff57dd640 1 -- start start 2024-09-11T22:59:50.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbff57dd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbff01002f0 0x7fbff019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbff57dd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbff0100ca0 0x7fbff019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbff57dd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbff0106f50 0x7fbff019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbff57dd640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0111fa0 con 0x7fbff01002f0 2024-09-11T22:59:50.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbff57dd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0111e20 con 0x7fbff0106f50 2024-09-11T22:59:50.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbff57dd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0112120 con 0x7fbff0100ca0 2024-09-11T22:59:50.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbfeeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbff01002f0 0x7fbff019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbfeeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbff01002f0 0x7fbff019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57508/0 (socket says 172.21.15.152:57508) 2024-09-11T22:59:50.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbfeeffd640 1 -- 172.21.15.152:0/3978738119 learned_addr learned my addr 172.21.15.152:0/3978738119 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:50.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbfef7fe640 1 --2- 172.21.15.152:0/3978738119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbff0106f50 0x7fbff019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbfee7fc640 1 --2- 172.21.15.152:0/3978738119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbff0100ca0 0x7fbff019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbfeeffd640 1 -- 172.21.15.152:0/3978738119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbff0100ca0 msgr2=0x7fbff019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:50.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbfeeffd640 1 --2- 172.21.15.152:0/3978738119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbff0100ca0 0x7fbff019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbfeeffd640 1 -- 172.21.15.152:0/3978738119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbff0106f50 msgr2=0x7fbff019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:50.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbfeeffd640 1 --2- 172.21.15.152:0/3978738119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbff0106f50 0x7fbff019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbfeeffd640 1 -- 172.21.15.152:0/3978738119 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff019f630 con 0x7fbff01002f0 2024-09-11T22:59:50.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbfef7fe640 1 --2- 172.21.15.152:0/3978738119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbff0106f50 0x7fbff019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:50.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbfeeffd640 1 --2- 172.21.15.152:0/3978738119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbff01002f0 0x7fbff019a680 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000c890 tx=0x7fbfe000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:50.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbfd7fff640 1 -- 172.21.15.152:0/3978738119 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0004220 con 0x7fbff01002f0 2024-09-11T22:59:50.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbff57dd640 1 -- 172.21.15.152:0/3978738119 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff019f920 con 0x7fbff01002f0 2024-09-11T22:59:50.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbfd7fff640 1 -- 172.21.15.152:0/3978738119 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe00043c0 con 0x7fbff01002f0 2024-09-11T22:59:50.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.711+0000 7fbfd7fff640 1 -- 172.21.15.152:0/3978738119 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0004a00 con 0x7fbff01002f0 2024-09-11T22:59:50.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.712+0000 7fbff57dd640 1 -- 172.21.15.152:0/3978738119 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff01a7200 con 0x7fbff01002f0 2024-09-11T22:59:50.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.714+0000 7fbfd7fff640 1 -- 172.21.15.152:0/3978738119 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fbfe00079b0 con 0x7fbff01002f0 2024-09-11T22:59:50.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.714+0000 7fbff7bb6740 1 Processor -- start 2024-09-11T22:59:50.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.714+0000 7fbff7bb6740 1 -- start start 2024-09-11T22:59:50.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.714+0000 7fbfd7fff640 1 --2- 172.21.15.152:0/3978738119 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbfd80a2cf0 0x7fbfd80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.714+0000 7fbff7bb6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282efb40 0x5609282ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.714+0000 7fbff7bb6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282ee5d0 0x5609282eea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.714+0000 7fbff7bb6740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 0x5609282fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.714+0000 7fbff7bb6740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5609282542f0 con 0x5609282f9370 2024-09-11T22:59:50.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.714+0000 7fbff7bb6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5609281fbcf0 con 0x5609282ee5d0 2024-09-11T22:59:50.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.714+0000 7fbff7bb6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5609282502b0 con 0x5609282efb40 2024-09-11T22:59:50.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfef7fe640 1 --2- 172.21.15.152:0/3978738119 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbfd80a2cf0 0x7fbfd80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfeeffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282efb40 0x5609282ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfd7fff640 1 -- 172.21.15.152:0/3978738119 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fbfe00cf1f0 con 0x7fbff01002f0 2024-09-11T22:59:50.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfeeffd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282efb40 0x5609282ee090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:33584/0 (socket says 172.21.15.152:33584) 2024-09-11T22:59:50.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfeeffd640 1 -- 172.21.15.152:0/1887516778 learned_addr learned my addr 172.21.15.152:0/1887516778 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfef7fe640 1 --2- 172.21.15.152:0/1887516778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 0x5609282fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfee7fc640 1 --2- 172.21.15.152:0/1887516778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282ee5d0 0x5609282eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfeeffd640 1 -- 172.21.15.152:0/1887516778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282ee5d0 msgr2=0x5609282eea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfeeffd640 1 --2- 172.21.15.152:0/1887516778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282ee5d0 0x5609282eea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfeeffd640 1 -- 172.21.15.152:0/1887516778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 msgr2=0x5609282fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfeeffd640 1 --2- 172.21.15.152:0/1887516778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 0x5609282fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfeeffd640 1 -- 172.21.15.152:0/1887516778 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609281e03a0 con 0x5609282efb40 2024-09-11T22:59:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfef7fe640 1 --2- 172.21.15.152:0/3978738119 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbfd80a2cf0 0x7fbfd80a51b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009430 tx=0x7fbfe8031430 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfeeffd640 1 --2- 172.21.15.152:0/1887516778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282efb40 0x5609282ee090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0000c00 tx=0x7fbfe001cb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51de955f10c82509 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfd57fa640 1 -- 172.21.15.152:0/1887516778 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe009b080 con 0x5609282efb40 2024-09-11T22:59:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfd57fa640 1 -- 172.21.15.152:0/1887516778 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe009be20 con 0x5609282efb40 2024-09-11T22:59:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.715+0000 7fbfd57fa640 1 -- 172.21.15.152:0/1887516778 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0098d70 con 0x5609282efb40 2024-09-11T22:59:50.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.716+0000 7fbff7bb6740 1 -- 172.21.15.152:0/1887516778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282efb40 msgr2=0x5609282ee090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.716+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/1887516778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282efb40 0x5609282ee090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0000c00 tx=0x7fbfe001cb40 comp rx=0 tx=0).stop 2024-09-11T22:59:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.716+0000 7fbff7bb6740 1 -- 172.21.15.152:0/1887516778 shutdown_connections 2024-09-11T22:59:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.716+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/1887516778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 0x5609282fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.716+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/1887516778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282ee5d0 0x5609282eea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.716+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/1887516778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282efb40 0x5609282ee090 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.716+0000 7fbff7bb6740 1 -- 172.21.15.152:0/1887516778 >> 172.21.15.152:0/1887516778 conn(0x5609282b7310 msgr2=0x5609282eac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.716+0000 7fbff7bb6740 1 -- 172.21.15.152:0/1887516778 shutdown_connections 2024-09-11T22:59:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.716+0000 7fbff7bb6740 1 -- 172.21.15.152:0/1887516778 wait complete. 2024-09-11T22:59:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.716+0000 7fbff7bb6740 1 -- start start 2024-09-11T22:59:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.716+0000 7fbff7bb6740 1 Processor -- start 2024-09-11T22:59:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbff7bb6740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282ee5d0 0x5609282f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbff7bb6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282efb40 0x5609282f6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbff7bb6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282f9370 0x5609282f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbff7bb6740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5609282fc070 con 0x5609282ee5d0 2024-09-11T22:59:50.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbff7bb6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56092820f060 con 0x5609282f9370 2024-09-11T22:59:50.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbff7bb6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5609282fc1f0 con 0x5609282efb40 2024-09-11T22:59:50.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbff7bb6740 10 client.0 _ll_register_callbacks cb 0x560927f628c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:50.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbfeeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282ee5d0 0x5609282f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbfeeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282ee5d0 0x5609282f8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57532/0 (socket says 172.21.15.152:57532) 2024-09-11T22:59:50.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbfeeffd640 1 -- 172.21.15.152:0/3555350460 learned_addr learned my addr 172.21.15.152:0/3555350460 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:50.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbfef7fe640 1 --2- 172.21.15.152:0/3555350460 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282f9370 0x5609282f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbfee7fc640 1 --2- 172.21.15.152:0/3555350460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282efb40 0x5609282f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbfeeffd640 1 -- 172.21.15.152:0/3555350460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282efb40 msgr2=0x5609282f6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:50.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbfeeffd640 1 --2- 172.21.15.152:0/3555350460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282efb40 0x5609282f6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbfeeffd640 1 -- 172.21.15.152:0/3555350460 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282f9370 msgr2=0x5609282f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:50.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbfeeffd640 1 --2- 172.21.15.152:0/3555350460 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282f9370 0x5609282f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbfeeffd640 1 -- 172.21.15.152:0/3555350460 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560927fc53c0 con 0x5609282ee5d0 2024-09-11T22:59:50.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbfef7fe640 1 --2- 172.21.15.152:0/3555350460 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282f9370 0x5609282f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:50.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbfeeffd640 1 --2- 172.21.15.152:0/3555350460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282ee5d0 0x5609282f8d50 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0097b80 tx=0x7fbfe009a0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff248f7d762c15b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:50.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbfd2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:50.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbfd2ff5640 1 -- 172.21.15.152:0/3555350460 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0098f50 con 0x5609282ee5d0 2024-09-11T22:59:50.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbfd2ff5640 1 -- 172.21.15.152:0/3555350460 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fbfe00a3f00 con 0x5609282ee5d0 2024-09-11T22:59:50.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.717+0000 7fbff7bb6740 1 -- 172.21.15.152:0/3555350460 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfe009be20 con 0x5609282ee5d0 2024-09-11T22:59:50.761 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.718+0000 7fbfd2ff5640 1 -- 172.21.15.152:0/3555350460 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbfe0099170 con 0x5609282ee5d0 2024-09-11T22:59:50.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.718+0000 7fbff7bb6740 10 client.17346 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:50.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.718+0000 7fbff7bb6740 1 -- 172.21.15.152:0/3555350460 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560928307af0 con 0x5609282ee5d0 2024-09-11T22:59:50.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.718+0000 7fbfd2ff5640 1 -- 172.21.15.152:0/3555350460 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe009be20 con 0x5609282ee5d0 2024-09-11T22:59:50.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.718+0000 7fbfd2ff5640 1 -- 172.21.15.152:0/3555350460 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbfe00a3b00 con 0x5609282ee5d0 2024-09-11T22:59:50.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.718+0000 7fbff7bb6740 10 client.17346 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:50.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.718+0000 7fbff7bb6740 10 client.17346 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:59:50.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.718+0000 7fbff7bb6740 20 client.17346 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:50.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.718+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/3555350460 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560928308e40 0x56092830b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.718+0000 7fbff7bb6740 4 client.17346 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:59:50.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.718+0000 7fbff7bb6740 1 -- 172.21.15.152:0/3555350460 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560928261860 con 0x560928308e40 2024-09-11T22:59:50.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.718+0000 7fbfee7fc640 1 --2- 172.21.15.152:0/3555350460 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560928308e40 0x56092830b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.762 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.719+0000 7fbfd2ff5640 10 client.17346 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:59:50.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.719+0000 7fbfee7fc640 1 --2- 172.21.15.152:0/3555350460 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560928308e40 0x56092830b230 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=83602d2030e6bd8d server_cookie=9301fc4d02b0b798 in_seq=0 out_seq=0 2024-09-11T22:59:50.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.719+0000 7fbfd2ff5640 1 -- 172.21.15.152:0/3555350460 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560928261860 con 0x560928308e40 2024-09-11T22:59:50.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.719+0000 7fbfd2ff5640 10 client.17346 handle_command_reply: tid=0 2024-09-11T22:59:50.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.719+0000 7fbff7bb6740 1 client.17346 shutdown 2024-09-11T22:59:50.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.720+0000 7fbff7bb6740 1 -- 172.21.15.152:0/3555350460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282ee5d0 msgr2=0x5609282f8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:50.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.720+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/3555350460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282ee5d0 0x5609282f8d50 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0097b80 tx=0x7fbfe009a0e0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.720+0000 7fbff7bb6740 1 -- 172.21.15.152:0/3555350460 shutdown_connections 2024-09-11T22:59:50.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.720+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/3555350460 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560928308e40 0x56092830b230 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.720+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/3555350460 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282f9370 0x5609282f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.720+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/3555350460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282efb40 0x5609282f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.720+0000 7fbfd2ff5640 0 client.17346 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:59:50.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.720+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/3555350460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282ee5d0 0x5609282f8d50 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.720+0000 7fbff7bb6740 1 -- 172.21.15.152:0/3555350460 >> 172.21.15.152:0/3555350460 conn(0x5609282b7310 msgr2=0x5609282f0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:50.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.720+0000 7fbff7bb6740 1 -- 172.21.15.152:0/3555350460 shutdown_connections 2024-09-11T22:59:50.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.720+0000 7fbff7bb6740 1 -- 172.21.15.152:0/3555350460 wait complete. 2024-09-11T22:59:50.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.720+0000 7fbff7bb6740 20 client.17346 trim_cache size 0 max 16384 2024-09-11T22:59:50.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.732+0000 7fbff7bb6740 1 Processor -- start 2024-09-11T22:59:50.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.732+0000 7fbff7bb6740 1 -- start start 2024-09-11T22:59:50.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbff7bb6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282ee5d0 0x5609282f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbff7bb6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282efb40 0x5609282f7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbff7bb6740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 0x5609282f1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbff7bb6740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbff01afb40 con 0x5609282f9370 2024-09-11T22:59:50.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbff7bb6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbff01b1680 con 0x5609282efb40 2024-09-11T22:59:50.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbff7bb6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbff01b0210 con 0x5609282ee5d0 2024-09-11T22:59:50.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbfef7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 0x5609282f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbfef7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 0x5609282f1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57534/0 (socket says 172.21.15.152:57534) 2024-09-11T22:59:50.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbfef7fe640 1 -- 172.21.15.152:0/17144096 learned_addr learned my addr 172.21.15.152:0/17144096 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:50.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbfee7fc640 1 --2- 172.21.15.152:0/17144096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282efb40 0x5609282f7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbfeeffd640 1 --2- 172.21.15.152:0/17144096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282ee5d0 0x5609282f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbfef7fe640 1 -- 172.21.15.152:0/17144096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282ee5d0 msgr2=0x5609282f58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:50.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbfef7fe640 1 --2- 172.21.15.152:0/17144096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282ee5d0 0x5609282f58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbfef7fe640 1 -- 172.21.15.152:0/17144096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282efb40 msgr2=0x5609282f7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:50.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbfef7fe640 1 --2- 172.21.15.152:0/17144096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282efb40 0x5609282f7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbfef7fe640 1 -- 172.21.15.152:0/17144096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609282f8970 con 0x5609282f9370 2024-09-11T22:59:50.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbfee7fc640 1 --2- 172.21.15.152:0/17144096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282efb40 0x5609282f7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:59:50.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbfef7fe640 1 --2- 172.21.15.152:0/17144096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 0x5609282f1120 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009cd0 tx=0x7fbfe802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf2c6a179d46b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:50.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbfd4ff9640 1 -- 172.21.15.152:0/17144096 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe804f070 con 0x5609282f9370 2024-09-11T22:59:50.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbfd4ff9640 1 -- 172.21.15.152:0/17144096 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe8036040 con 0x5609282f9370 2024-09-11T22:59:50.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.733+0000 7fbfd4ff9640 1 -- 172.21.15.152:0/17144096 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8030bb0 con 0x5609282f9370 2024-09-11T22:59:50.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.734+0000 7fbff7bb6740 1 -- 172.21.15.152:0/17144096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 msgr2=0x5609282f1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:50.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.734+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/17144096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 0x5609282f1120 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009cd0 tx=0x7fbfe802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.734+0000 7fbff7bb6740 1 -- 172.21.15.152:0/17144096 shutdown_connections 2024-09-11T22:59:50.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.734+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/17144096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 0x5609282f1120 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.734+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/17144096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282efb40 0x5609282f7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.734+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/17144096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282ee5d0 0x5609282f58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.734+0000 7fbff7bb6740 1 -- 172.21.15.152:0/17144096 >> 172.21.15.152:0/17144096 conn(0x5609282b7310 msgr2=0x5609282f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:50.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.734+0000 7fbff7bb6740 1 -- 172.21.15.152:0/17144096 shutdown_connections 2024-09-11T22:59:50.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.734+0000 7fbff7bb6740 1 -- 172.21.15.152:0/17144096 wait complete. 2024-09-11T22:59:50.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.734+0000 7fbff7bb6740 1 -- start start 2024-09-11T22:59:50.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.734+0000 7fbff7bb6740 1 Processor -- start 2024-09-11T22:59:50.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbff7bb6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282ee5d0 0x56092830e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbff7bb6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282efb40 0x56092830ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbff7bb6740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 0x56092830bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbff7bb6740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbff01b0fb0 con 0x5609282f9370 2024-09-11T22:59:50.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbff7bb6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbff01b08e0 con 0x5609282efb40 2024-09-11T22:59:50.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbff7bb6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbff01adb00 con 0x5609282ee5d0 2024-09-11T22:59:50.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbff7bb6740 10 client.0 _ll_register_callbacks cb 0x7fbfe8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:50.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbfef7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 0x56092830bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbfef7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 0x56092830bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57550/0 (socket says 172.21.15.152:57550) 2024-09-11T22:59:50.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbfef7fe640 1 -- 172.21.15.152:0/2918330202 learned_addr learned my addr 172.21.15.152:0/2918330202 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:50.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbfee7fc640 1 --2- 172.21.15.152:0/2918330202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282efb40 0x56092830ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbfeeffd640 1 --2- 172.21.15.152:0/2918330202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282ee5d0 0x56092830e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbfef7fe640 1 -- 172.21.15.152:0/2918330202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282ee5d0 msgr2=0x56092830e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:50.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbfef7fe640 1 --2- 172.21.15.152:0/2918330202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282ee5d0 0x56092830e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbfef7fe640 1 -- 172.21.15.152:0/2918330202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282efb40 msgr2=0x56092830ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:50.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbfef7fe640 1 --2- 172.21.15.152:0/2918330202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282efb40 0x56092830ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbfef7fe640 1 -- 172.21.15.152:0/2918330202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56092830c580 con 0x5609282f9370 2024-09-11T22:59:50.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbfee7fc640 1 --2- 172.21.15.152:0/2918330202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282efb40 0x56092830ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:50.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbfef7fe640 1 --2- 172.21.15.152:0/2918330202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 0x56092830bfc0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8031eb0 tx=0x7fbfe80306b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8e5aa60389d803 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:50.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbfd2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:50.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbfd2ff5640 1 -- 172.21.15.152:0/2918330202 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe804f040 con 0x5609282f9370 2024-09-11T22:59:50.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbfd2ff5640 1 -- 172.21.15.152:0/2918330202 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fbfe8030c00 con 0x5609282f9370 2024-09-11T22:59:50.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.735+0000 7fbff7bb6740 1 -- 172.21.15.152:0/2918330202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfe8036040 con 0x5609282f9370 2024-09-11T22:59:50.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.736+0000 7fbfd2ff5640 1 -- 172.21.15.152:0/2918330202 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbfe8030e00 con 0x5609282f9370 2024-09-11T22:59:50.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.736+0000 7fbff7bb6740 10 client.17352 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:50.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.736+0000 7fbff7bb6740 1 -- 172.21.15.152:0/2918330202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560928373460 con 0x5609282f9370 2024-09-11T22:59:50.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.736+0000 7fbfd2ff5640 1 -- 172.21.15.152:0/2918330202 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe8036040 con 0x5609282f9370 2024-09-11T22:59:50.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.736+0000 7fbfd2ff5640 1 -- 172.21.15.152:0/2918330202 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fbfe8030070 con 0x5609282f9370 2024-09-11T22:59:50.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.736+0000 7fbff7bb6740 10 client.17352 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:50.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.736+0000 7fbff7bb6740 10 client.17352 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T22:59:50.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.736+0000 7fbff7bb6740 20 client.17352 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:50.769 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:59:50.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.736+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/2918330202 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560928373a20 0x560928375e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:50.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.736+0000 7fbff7bb6740 4 client.17352 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:59:50.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.736+0000 7fbff7bb6740 1 -- 172.21.15.152:0/2918330202 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56092825bd70 con 0x560928373a20 2024-09-11T22:59:50.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.736+0000 7fbfeeffd640 1 --2- 172.21.15.152:0/2918330202 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560928373a20 0x560928375e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:50.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbfd2ff5640 10 client.17352 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T22:59:50.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbfeeffd640 1 --2- 172.21.15.152:0/2918330202 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560928373a20 0x560928375e10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e7f5e517e17662e server_cookie=aed4e6cefba9655c in_seq=0 out_seq=0 2024-09-11T22:59:50.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbfd2ff5640 1 -- 172.21.15.152:0/2918330202 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56092825bd70 con 0x560928373a20 2024-09-11T22:59:50.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbfd2ff5640 10 client.17352 handle_command_reply: tid=0 2024-09-11T22:59:50.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbff7bb6740 1 client.17352 shutdown 2024-09-11T22:59:50.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbff7bb6740 1 -- 172.21.15.152:0/2918330202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 msgr2=0x56092830bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:50.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/2918330202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 0x56092830bfc0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8031eb0 tx=0x7fbfe80306b0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbff7bb6740 1 -- 172.21.15.152:0/2918330202 shutdown_connections 2024-09-11T22:59:50.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/2918330202 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560928373a20 0x560928375e10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/2918330202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5609282f9370 0x56092830bfc0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbfd2ff5640 0 client.17352 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T22:59:50.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/2918330202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5609282efb40 0x56092830ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbff7bb6740 1 --2- 172.21.15.152:0/2918330202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5609282ee5d0 0x56092830e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbff7bb6740 1 -- 172.21.15.152:0/2918330202 >> 172.21.15.152:0/2918330202 conn(0x5609282b7310 msgr2=0x5609282f8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:50.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbff7bb6740 1 -- 172.21.15.152:0/2918330202 shutdown_connections 2024-09-11T22:59:50.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbff7bb6740 1 -- 172.21.15.152:0/2918330202 wait complete. 2024-09-11T22:59:50.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.737+0000 7fbff7bb6740 20 client.17352 trim_cache size 0 max 16384 2024-09-11T22:59:50.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.738+0000 7fbff57dd640 1 -- 172.21.15.152:0/3978738119 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbfd80a2cf0 msgr2=0x7fbfd80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:50.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.738+0000 7fbff57dd640 1 --2- 172.21.15.152:0/3978738119 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbfd80a2cf0 0x7fbfd80a51b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009430 tx=0x7fbfe8031430 comp rx=0 tx=0).stop 2024-09-11T22:59:50.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.738+0000 7fbff57dd640 1 -- 172.21.15.152:0/3978738119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbff01002f0 msgr2=0x7fbff019a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:50.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.738+0000 7fbff57dd640 1 --2- 172.21.15.152:0/3978738119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbff01002f0 0x7fbff019a680 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000c890 tx=0x7fbfe000cd50 comp rx=0 tx=0).stop 2024-09-11T22:59:50.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.738+0000 7fbff57dd640 1 -- 172.21.15.152:0/3978738119 shutdown_connections 2024-09-11T22:59:50.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.738+0000 7fbff57dd640 1 --2- 172.21.15.152:0/3978738119 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbfd80a2cf0 0x7fbfd80a51b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.738+0000 7fbff57dd640 1 --2- 172.21.15.152:0/3978738119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbff0106f50 0x7fbff019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.738+0000 7fbff57dd640 1 --2- 172.21.15.152:0/3978738119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbff0100ca0 0x7fbff019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.738+0000 7fbff57dd640 1 --2- 172.21.15.152:0/3978738119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbff01002f0 0x7fbff019a680 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:50.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.738+0000 7fbff57dd640 1 -- 172.21.15.152:0/3978738119 >> 172.21.15.152:0/3978738119 conn(0x7fbff0077210 msgr2=0x7fbff0105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:50.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.738+0000 7fbff57dd640 1 -- 172.21.15.152:0/3978738119 shutdown_connections 2024-09-11T22:59:50.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:50.738+0000 7fbff57dd640 1 -- 172.21.15.152:0/3978738119 wait complete. 2024-09-11T22:59:50.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:50 smithi191 ceph-mon[35211]: pgmap v211: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 12 op/s 2024-09-11T22:59:50.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:50 smithi204 ceph-mon[35454]: pgmap v211: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 12 op/s 2024-09-11T22:59:51.506 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:59:51.754 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:52.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.262+0000 7fcd95ac1640 1 -- 172.21.15.152:0/3945351823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd901027a0 msgr2=0x7fcd901067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:52.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.262+0000 7fcd95ac1640 1 --2- 172.21.15.152:0/3945351823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd901027a0 0x7fcd901067f0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009900 tx=0x7fcd8402a410 comp rx=0 tx=0).stop 2024-09-11T22:59:52.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.263+0000 7fcd95ac1640 1 -- 172.21.15.152:0/3945351823 shutdown_connections 2024-09-11T22:59:52.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.263+0000 7fcd95ac1640 1 --2- 172.21.15.152:0/3945351823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd90106f20 0x7fcd9010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.263+0000 7fcd95ac1640 1 --2- 172.21.15.152:0/3945351823 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd901027a0 0x7fcd901067f0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.263+0000 7fcd95ac1640 1 --2- 172.21.15.152:0/3945351823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd90101df0 0x7fcd901021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.263+0000 7fcd95ac1640 1 -- 172.21.15.152:0/3945351823 >> 172.21.15.152:0/3945351823 conn(0x7fcd900fd8f0 msgr2=0x7fcd900ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:52.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.263+0000 7fcd95ac1640 1 -- 172.21.15.152:0/3945351823 shutdown_connections 2024-09-11T22:59:52.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.263+0000 7fcd95ac1640 1 -- 172.21.15.152:0/3945351823 wait complete. 2024-09-11T22:59:52.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.263+0000 7fcd95ac1640 1 Processor -- start 2024-09-11T22:59:52.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.263+0000 7fcd95ac1640 1 -- start start 2024-09-11T22:59:52.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd95ac1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd90101df0 0x7fcd9019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd95ac1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd901027a0 0x7fcd9019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd95ac1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd90106f20 0x7fcd9019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd95ac1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90111f80 con 0x7fcd90101df0 2024-09-11T22:59:52.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd95ac1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90111e00 con 0x7fcd901027a0 2024-09-11T22:59:52.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd95ac1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90112100 con 0x7fcd90106f20 2024-09-11T22:59:52.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd8f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd90101df0 0x7fcd9019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd8f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd90101df0 0x7fcd9019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57570/0 (socket says 172.21.15.152:57570) 2024-09-11T22:59:52.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd8f7fe640 1 -- 172.21.15.152:0/207527967 learned_addr learned my addr 172.21.15.152:0/207527967 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:52.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd8effd640 1 --2- 172.21.15.152:0/207527967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd901027a0 0x7fcd9019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd8ffff640 1 --2- 172.21.15.152:0/207527967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd90106f20 0x7fcd9019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd8f7fe640 1 -- 172.21.15.152:0/207527967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd90106f20 msgr2=0x7fcd9019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:52.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd8f7fe640 1 --2- 172.21.15.152:0/207527967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd90106f20 0x7fcd9019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd8f7fe640 1 -- 172.21.15.152:0/207527967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd901027a0 msgr2=0x7fcd9019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:52.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd8f7fe640 1 --2- 172.21.15.152:0/207527967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd901027a0 0x7fcd9019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd8f7fe640 1 -- 172.21.15.152:0/207527967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9019f6b0 con 0x7fcd90101df0 2024-09-11T22:59:52.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd8f7fe640 1 --2- 172.21.15.152:0/207527967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd90101df0 0x7fcd9019a700 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000d770 tx=0x7fcd8000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:52.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd8cff9640 1 -- 172.21.15.152:0/207527967 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80017070 con 0x7fcd90101df0 2024-09-11T22:59:52.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd95ac1640 1 -- 172.21.15.152:0/207527967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9019f9a0 con 0x7fcd90101df0 2024-09-11T22:59:52.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd8cff9640 1 -- 172.21.15.152:0/207527967 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd80005500 con 0x7fcd90101df0 2024-09-11T22:59:52.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd8cff9640 1 -- 172.21.15.152:0/207527967 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80004e00 con 0x7fcd90101df0 2024-09-11T22:59:52.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.264+0000 7fcd95ac1640 1 -- 172.21.15.152:0/207527967 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd901a7280 con 0x7fcd90101df0 2024-09-11T22:59:52.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.266+0000 7fcd8cff9640 1 -- 172.21.15.152:0/207527967 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fcd80004fa0 con 0x7fcd90101df0 2024-09-11T22:59:52.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.267+0000 7fcd97e9a740 1 Processor -- start 2024-09-11T22:59:52.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.267+0000 7fcd97e9a740 1 -- start start 2024-09-11T22:59:52.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.267+0000 7fcd97e9a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65fb40 0x5574bb65e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.267+0000 7fcd8cff9640 1 --2- 172.21.15.152:0/207527967 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcd780a2cf0 0x7fcd780a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.267+0000 7fcd97e9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb65e5d0 0x5574bb65ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.267+0000 7fcd97e9a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb669370 0x5574bb66b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.267+0000 7fcd97e9a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5574bb5c42f0 con 0x5574bb669370 2024-09-11T22:59:52.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.267+0000 7fcd97e9a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5574bb56bcf0 con 0x5574bb65fb40 2024-09-11T22:59:52.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.267+0000 7fcd97e9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5574bb5c02b0 con 0x5574bb65e5d0 2024-09-11T22:59:52.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.267+0000 7fcd8ffff640 1 --2- 172.21.15.152:0/207527967 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcd780a2cf0 0x7fcd780a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.267+0000 7fcd8ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb669370 0x5574bb66b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.267+0000 7fcd8ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb669370 0x5574bb66b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57586/0 (socket says 172.21.15.152:57586) 2024-09-11T22:59:52.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.267+0000 7fcd8ffff640 1 -- 172.21.15.152:0/1491924975 learned_addr learned my addr 172.21.15.152:0/1491924975 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:52.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.267+0000 7fcd8cff9640 1 -- 172.21.15.152:0/207527967 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fcd800cef30 con 0x7fcd90101df0 2024-09-11T22:59:52.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.267+0000 7fcd8f7fe640 1 --2- 172.21.15.152:0/1491924975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65fb40 0x5574bb65e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.267+0000 7fcd8effd640 1 --2- 172.21.15.152:0/1491924975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb65e5d0 0x5574bb65ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.268+0000 7fcd8ffff640 1 -- 172.21.15.152:0/1491924975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb65e5d0 msgr2=0x5574bb65ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:52.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.268+0000 7fcd8ffff640 1 --2- 172.21.15.152:0/1491924975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb65e5d0 0x5574bb65ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.268+0000 7fcd8ffff640 1 -- 172.21.15.152:0/1491924975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65fb40 msgr2=0x5574bb65e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:52.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.268+0000 7fcd8ffff640 1 --2- 172.21.15.152:0/1491924975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65fb40 0x5574bb65e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.268+0000 7fcd8ffff640 1 -- 172.21.15.152:0/1491924975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574bb5503a0 con 0x5574bb669370 2024-09-11T22:59:52.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.268+0000 7fcd8ffff640 1 --2- 172.21.15.152:0/1491924975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb669370 0x5574bb66b760 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fcd88013de0 tx=0x7fcd880107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12bce72e94e7f3d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:52.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.268+0000 7fcd8ffff640 1 --2- 172.21.15.152:0/207527967 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcd780a2cf0 0x7fcd780a51b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcd88010ca0 tx=0x7fcd8801b000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:52.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.268+0000 7fcd75ffb640 1 -- 172.21.15.152:0/1491924975 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88021070 con 0x5574bb669370 2024-09-11T22:59:52.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.268+0000 7fcd75ffb640 1 -- 172.21.15.152:0/1491924975 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd8800bdc0 con 0x5574bb669370 2024-09-11T22:59:52.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.268+0000 7fcd75ffb640 1 -- 172.21.15.152:0/1491924975 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8800e610 con 0x5574bb669370 2024-09-11T22:59:52.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.269+0000 7fcd97e9a740 1 -- 172.21.15.152:0/1491924975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb669370 msgr2=0x5574bb66b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:52.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.269+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/1491924975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb669370 0x5574bb66b760 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fcd88013de0 tx=0x7fcd880107a0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.269+0000 7fcd97e9a740 1 -- 172.21.15.152:0/1491924975 shutdown_connections 2024-09-11T22:59:52.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.269+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/1491924975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb669370 0x5574bb66b760 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.269+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/1491924975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb65e5d0 0x5574bb65ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.269+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/1491924975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65fb40 0x5574bb65e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.269+0000 7fcd97e9a740 1 -- 172.21.15.152:0/1491924975 >> 172.21.15.152:0/1491924975 conn(0x5574bb627310 msgr2=0x5574bb65ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:52.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.269+0000 7fcd97e9a740 1 -- 172.21.15.152:0/1491924975 shutdown_connections 2024-09-11T22:59:52.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.269+0000 7fcd97e9a740 1 -- 172.21.15.152:0/1491924975 wait complete. 2024-09-11T22:59:52.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.269+0000 7fcd97e9a740 1 -- start start 2024-09-11T22:59:52.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.269+0000 7fcd97e9a740 1 Processor -- start 2024-09-11T22:59:52.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd97e9a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65e5d0 0x5574bb668d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd97e9a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb65fb40 0x5574bb666040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd97e9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb669370 0x5574bb666650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd97e9a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5574bb66c010 con 0x5574bb65fb40 2024-09-11T22:59:52.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd97e9a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5574bb57f060 con 0x5574bb65e5d0 2024-09-11T22:59:52.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd97e9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5574bb66c190 con 0x5574bb669370 2024-09-11T22:59:52.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd97e9a740 10 client.0 _ll_register_callbacks cb 0x5574bb2d28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:52.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd8effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb65fb40 0x5574bb666040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd8effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb65fb40 0x5574bb666040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57588/0 (socket says 172.21.15.152:57588) 2024-09-11T22:59:52.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd8effd640 1 -- 172.21.15.152:0/2440517764 learned_addr learned my addr 172.21.15.152:0/2440517764 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:52.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd8ffff640 1 --2- 172.21.15.152:0/2440517764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb669370 0x5574bb666650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd8f7fe640 1 --2- 172.21.15.152:0/2440517764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65e5d0 0x5574bb668d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd8effd640 1 -- 172.21.15.152:0/2440517764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb669370 msgr2=0x5574bb666650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:52.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd8effd640 1 --2- 172.21.15.152:0/2440517764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb669370 0x5574bb666650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd8effd640 1 -- 172.21.15.152:0/2440517764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65e5d0 msgr2=0x5574bb668d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:52.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd8effd640 1 --2- 172.21.15.152:0/2440517764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65e5d0 0x5574bb668d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd8effd640 1 -- 172.21.15.152:0/2440517764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574bb3353c0 con 0x5574bb65fb40 2024-09-11T22:59:52.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd8f7fe640 1 --2- 172.21.15.152:0/2440517764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65e5d0 0x5574bb668d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:52.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd8effd640 1 --2- 172.21.15.152:0/2440517764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb65fb40 0x5574bb666040 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009900 tx=0x7fcd840319b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3893dcf804d18505 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:52.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd6effd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:52.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.270+0000 7fcd6effd640 1 -- 172.21.15.152:0/2440517764 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84035040 con 0x5574bb65fb40 2024-09-11T22:59:52.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.271+0000 7fcd6effd640 1 -- 172.21.15.152:0/2440517764 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fcd84004240 con 0x5574bb65fb40 2024-09-11T22:59:52.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.271+0000 7fcd97e9a740 1 -- 172.21.15.152:0/2440517764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd8800bdc0 con 0x5574bb65fb40 2024-09-11T22:59:52.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.271+0000 7fcd6effd640 1 -- 172.21.15.152:0/2440517764 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcd84004440 con 0x5574bb65fb40 2024-09-11T22:59:52.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.271+0000 7fcd97e9a740 10 client.17364 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:52.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.271+0000 7fcd97e9a740 1 -- 172.21.15.152:0/2440517764 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574bb677ae0 con 0x5574bb65fb40 2024-09-11T22:59:52.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.271+0000 7fcd6effd640 1 -- 172.21.15.152:0/2440517764 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd8800bdc0 con 0x5574bb65fb40 2024-09-11T22:59:52.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.271+0000 7fcd6effd640 1 -- 172.21.15.152:0/2440517764 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcd84004b90 con 0x5574bb65fb40 2024-09-11T22:59:52.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.271+0000 7fcd97e9a740 10 client.17364 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:52.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.271+0000 7fcd97e9a740 10 client.17364 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:59:52.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.271+0000 7fcd97e9a740 20 client.17364 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:52.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.271+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/2440517764 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5574bb678ef0 0x5574bb67b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.271+0000 7fcd97e9a740 4 client.17364 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:59:52.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.271+0000 7fcd97e9a740 1 -- 172.21.15.152:0/2440517764 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574bb5d1860 con 0x5574bb678ef0 2024-09-11T22:59:52.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.272+0000 7fcd8f7fe640 1 --2- 172.21.15.152:0/2440517764 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5574bb678ef0 0x5574bb67b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.272+0000 7fcd6effd640 10 client.17364 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:59:52.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.272+0000 7fcd8f7fe640 1 --2- 172.21.15.152:0/2440517764 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5574bb678ef0 0x5574bb67b320 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=83795269163fdfc1 server_cookie=5da70faf8f52ff8a in_seq=0 out_seq=0 2024-09-11T22:59:52.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.273+0000 7fcd6effd640 1 -- 172.21.15.152:0/2440517764 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574bb5d1860 con 0x5574bb678ef0 2024-09-11T22:59:52.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.273+0000 7fcd6effd640 10 client.17364 handle_command_reply: tid=0 2024-09-11T22:59:52.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.273+0000 7fcd97e9a740 1 client.17364 shutdown 2024-09-11T22:59:52.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.273+0000 7fcd97e9a740 1 -- 172.21.15.152:0/2440517764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb65fb40 msgr2=0x5574bb666040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:52.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.273+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/2440517764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb65fb40 0x5574bb666040 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009900 tx=0x7fcd840319b0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.273+0000 7fcd97e9a740 1 -- 172.21.15.152:0/2440517764 shutdown_connections 2024-09-11T22:59:52.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.273+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/2440517764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb669370 0x5574bb666650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.273+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/2440517764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb65fb40 0x5574bb666040 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.273+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/2440517764 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5574bb678ef0 0x5574bb67b320 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.273+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/2440517764 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65e5d0 0x5574bb668d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.273+0000 7fcd6effd640 0 client.17364 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:59:52.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.273+0000 7fcd97e9a740 1 -- 172.21.15.152:0/2440517764 >> 172.21.15.152:0/2440517764 conn(0x5574bb627310 msgr2=0x5574bb660960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:52.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.273+0000 7fcd97e9a740 1 -- 172.21.15.152:0/2440517764 shutdown_connections 2024-09-11T22:59:52.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.273+0000 7fcd97e9a740 1 -- 172.21.15.152:0/2440517764 wait complete. 2024-09-11T22:59:52.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.273+0000 7fcd97e9a740 20 client.17364 trim_cache size 0 max 16384 2024-09-11T22:59:52.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd97e9a740 1 Processor -- start 2024-09-11T22:59:52.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd97e9a740 1 -- start start 2024-09-11T22:59:52.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd97e9a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb65e5d0 0x5574bb661120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd97e9a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65fb40 0x5574bb661660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd97e9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb669370 0x5574bb680bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd97e9a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901af9d0 con 0x5574bb65e5d0 2024-09-11T22:59:52.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd97e9a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901b1510 con 0x5574bb65fb40 2024-09-11T22:59:52.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd97e9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901b00a0 con 0x5574bb669370 2024-09-11T22:59:52.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd8f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb65e5d0 0x5574bb661120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd8f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb65e5d0 0x5574bb661120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57598/0 (socket says 172.21.15.152:57598) 2024-09-11T22:59:52.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd8f7fe640 1 -- 172.21.15.152:0/2267403930 learned_addr learned my addr 172.21.15.152:0/2267403930 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:52.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd8effd640 1 --2- 172.21.15.152:0/2267403930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65fb40 0x5574bb661660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd8ffff640 1 --2- 172.21.15.152:0/2267403930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb669370 0x5574bb680bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd8f7fe640 1 -- 172.21.15.152:0/2267403930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb669370 msgr2=0x5574bb680bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:52.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd8f7fe640 1 --2- 172.21.15.152:0/2267403930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb669370 0x5574bb680bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd8f7fe640 1 -- 172.21.15.152:0/2267403930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65fb40 msgr2=0x5574bb661660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:52.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd8f7fe640 1 --2- 172.21.15.152:0/2267403930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65fb40 0x5574bb661660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd8f7fe640 1 -- 172.21.15.152:0/2267403930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574bb65b960 con 0x5574bb65e5d0 2024-09-11T22:59:52.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.286+0000 7fcd8effd640 1 --2- 172.21.15.152:0/2267403930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65fb40 0x5574bb661660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:52.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.287+0000 7fcd8f7fe640 1 --2- 172.21.15.152:0/2267403930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb65e5d0 0x5574bb661120 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009610 tx=0x7fcd8001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04d5e2e8d91c6f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:52.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.287+0000 7fcd757fa640 1 -- 172.21.15.152:0/2267403930 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80082040 con 0x5574bb65e5d0 2024-09-11T22:59:52.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.287+0000 7fcd757fa640 1 -- 172.21.15.152:0/2267403930 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd80015070 con 0x5574bb65e5d0 2024-09-11T22:59:52.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.287+0000 7fcd757fa640 1 -- 172.21.15.152:0/2267403930 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80020040 con 0x5574bb65e5d0 2024-09-11T22:59:52.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.287+0000 7fcd97e9a740 1 -- 172.21.15.152:0/2267403930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb65e5d0 msgr2=0x5574bb661120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:52.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.287+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/2267403930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb65e5d0 0x5574bb661120 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009610 tx=0x7fcd8001d040 comp rx=0 tx=0).stop 2024-09-11T22:59:52.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.287+0000 7fcd97e9a740 1 -- 172.21.15.152:0/2267403930 shutdown_connections 2024-09-11T22:59:52.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.287+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/2267403930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb669370 0x5574bb680bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.287+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/2267403930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65fb40 0x5574bb661660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.287+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/2267403930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb65e5d0 0x5574bb661120 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.287+0000 7fcd97e9a740 1 -- 172.21.15.152:0/2267403930 >> 172.21.15.152:0/2267403930 conn(0x5574bb627310 msgr2=0x5574bb6680c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:52.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.288+0000 7fcd97e9a740 1 -- 172.21.15.152:0/2267403930 shutdown_connections 2024-09-11T22:59:52.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.288+0000 7fcd97e9a740 1 -- 172.21.15.152:0/2267403930 wait complete. 2024-09-11T22:59:52.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.288+0000 7fcd97e9a740 1 -- start start 2024-09-11T22:59:52.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.288+0000 7fcd97e9a740 1 Processor -- start 2024-09-11T22:59:52.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.288+0000 7fcd97e9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb65e5d0 0x5574bb66d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.288+0000 7fcd97e9a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65fb40 0x5574bb66daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.288+0000 7fcd97e9a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb669370 0x5574bb6e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.288+0000 7fcd97e9a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901b0e40 con 0x5574bb669370 2024-09-11T22:59:52.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.288+0000 7fcd97e9a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901b0770 con 0x5574bb65fb40 2024-09-11T22:59:52.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.288+0000 7fcd97e9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcd901ad990 con 0x5574bb65e5d0 2024-09-11T22:59:52.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.288+0000 7fcd97e9a740 10 client.0 _ll_register_callbacks cb 0x7fcd840314d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:52.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.288+0000 7fcd8ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb669370 0x5574bb6e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.288+0000 7fcd8ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb669370 0x5574bb6e2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57600/0 (socket says 172.21.15.152:57600) 2024-09-11T22:59:52.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.288+0000 7fcd8ffff640 1 -- 172.21.15.152:0/3307774718 learned_addr learned my addr 172.21.15.152:0/3307774718 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:52.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.289+0000 7fcd8effd640 1 --2- 172.21.15.152:0/3307774718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65fb40 0x5574bb66daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.289+0000 7fcd8f7fe640 1 --2- 172.21.15.152:0/3307774718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb65e5d0 0x5574bb66d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.289+0000 7fcd8ffff640 1 -- 172.21.15.152:0/3307774718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb65e5d0 msgr2=0x5574bb66d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:52.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.289+0000 7fcd8ffff640 1 --2- 172.21.15.152:0/3307774718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb65e5d0 0x5574bb66d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.289+0000 7fcd8ffff640 1 -- 172.21.15.152:0/3307774718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65fb40 msgr2=0x5574bb66daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:52.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.289+0000 7fcd8ffff640 1 --2- 172.21.15.152:0/3307774718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65fb40 0x5574bb66daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.289+0000 7fcd8ffff640 1 -- 172.21.15.152:0/3307774718 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574bb6e3100 con 0x5574bb669370 2024-09-11T22:59:52.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.289+0000 7fcd8ffff640 1 --2- 172.21.15.152:0/3307774718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb669370 0x5574bb6e2bc0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fcd8801dd40 tx=0x7fcd8801dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e39a725bd3530d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:52.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.289+0000 7fcd6effd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:52.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.289+0000 7fcd6effd640 1 -- 172.21.15.152:0/3307774718 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88021040 con 0x5574bb669370 2024-09-11T22:59:52.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.289+0000 7fcd6effd640 1 -- 172.21.15.152:0/3307774718 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fcd88013020 con 0x5574bb669370 2024-09-11T22:59:52.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.289+0000 7fcd97e9a740 1 -- 172.21.15.152:0/3307774718 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd80015070 con 0x5574bb669370 2024-09-11T22:59:52.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.289+0000 7fcd6effd640 1 -- 172.21.15.152:0/3307774718 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcd88034020 con 0x5574bb669370 2024-09-11T22:59:52.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.289+0000 7fcd97e9a740 10 client.17370 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:52.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.289+0000 7fcd97e9a740 1 -- 172.21.15.152:0/3307774718 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5574bb6e42e0 con 0x5574bb669370 2024-09-11T22:59:52.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.290+0000 7fcd6effd640 1 -- 172.21.15.152:0/3307774718 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd80015070 con 0x5574bb669370 2024-09-11T22:59:52.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.290+0000 7fcd6effd640 1 -- 172.21.15.152:0/3307774718 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcd8801c070 con 0x5574bb669370 2024-09-11T22:59:52.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.290+0000 7fcd97e9a740 10 client.17370 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:52.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.290+0000 7fcd97e9a740 10 client.17370 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T22:59:52.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.290+0000 7fcd97e9a740 20 client.17370 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:52.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.290+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/3307774718 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5574bb6e4720 0x5574bb6e6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:52.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.290+0000 7fcd97e9a740 4 client.17370 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:59:52.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.290+0000 7fcd97e9a740 1 -- 172.21.15.152:0/3307774718 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574bb5cbd70 con 0x5574bb6e4720 2024-09-11T22:59:52.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.290+0000 7fcd8f7fe640 1 --2- 172.21.15.152:0/3307774718 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5574bb6e4720 0x5574bb6e6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:52.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.290+0000 7fcd6effd640 10 client.17370 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T22:59:52.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.290+0000 7fcd8f7fe640 1 --2- 172.21.15.152:0/3307774718 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5574bb6e4720 0x5574bb6e6b40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f361f5ca7b1b77d2 server_cookie=602994f4933fd70f in_seq=0 out_seq=0 2024-09-11T22:59:52.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.291+0000 7fcd6effd640 1 -- 172.21.15.152:0/3307774718 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574bb5cbd70 con 0x5574bb6e4720 2024-09-11T22:59:52.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.291+0000 7fcd6effd640 10 client.17370 handle_command_reply: tid=0 2024-09-11T22:59:52.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.291+0000 7fcd97e9a740 1 client.17370 shutdown 2024-09-11T22:59:52.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.291+0000 7fcd97e9a740 1 -- 172.21.15.152:0/3307774718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb669370 msgr2=0x5574bb6e2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:52.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.291+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/3307774718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb669370 0x5574bb6e2bc0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fcd8801dd40 tx=0x7fcd8801dcd0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.291+0000 7fcd97e9a740 1 -- 172.21.15.152:0/3307774718 shutdown_connections 2024-09-11T22:59:52.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.291+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/3307774718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5574bb669370 0x5574bb6e2bc0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.291+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/3307774718 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5574bb6e4720 0x5574bb6e6b40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.291+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/3307774718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5574bb65fb40 0x5574bb66daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.291+0000 7fcd97e9a740 1 --2- 172.21.15.152:0/3307774718 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574bb65e5d0 0x5574bb66d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.291+0000 7fcd97e9a740 1 -- 172.21.15.152:0/3307774718 >> 172.21.15.152:0/3307774718 conn(0x5574bb627310 msgr2=0x5574bb6594e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:52.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.291+0000 7fcd6effd640 0 client.17370 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T22:59:52.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.291+0000 7fcd97e9a740 1 -- 172.21.15.152:0/3307774718 shutdown_connections 2024-09-11T22:59:52.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.291+0000 7fcd97e9a740 1 -- 172.21.15.152:0/3307774718 wait complete. 2024-09-11T22:59:52.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.291+0000 7fcd97e9a740 20 client.17370 trim_cache size 0 max 16384 2024-09-11T22:59:52.302 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:59:52.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.292+0000 7fcd95ac1640 1 -- 172.21.15.152:0/207527967 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcd780a2cf0 msgr2=0x7fcd780a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:52.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.292+0000 7fcd95ac1640 1 --2- 172.21.15.152:0/207527967 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcd780a2cf0 0x7fcd780a51b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcd88010ca0 tx=0x7fcd8801b000 comp rx=0 tx=0).stop 2024-09-11T22:59:52.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.292+0000 7fcd95ac1640 1 -- 172.21.15.152:0/207527967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd90101df0 msgr2=0x7fcd9019a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:52.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.292+0000 7fcd95ac1640 1 --2- 172.21.15.152:0/207527967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd90101df0 0x7fcd9019a700 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000d770 tx=0x7fcd8000dc30 comp rx=0 tx=0).stop 2024-09-11T22:59:52.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.292+0000 7fcd95ac1640 1 -- 172.21.15.152:0/207527967 shutdown_connections 2024-09-11T22:59:52.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.292+0000 7fcd95ac1640 1 --2- 172.21.15.152:0/207527967 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcd780a2cf0 0x7fcd780a51b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.292+0000 7fcd95ac1640 1 --2- 172.21.15.152:0/207527967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd90106f20 0x7fcd9019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.292+0000 7fcd95ac1640 1 --2- 172.21.15.152:0/207527967 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd901027a0 0x7fcd9019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.292+0000 7fcd95ac1640 1 --2- 172.21.15.152:0/207527967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd90101df0 0x7fcd9019a700 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:52.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.292+0000 7fcd95ac1640 1 -- 172.21.15.152:0/207527967 >> 172.21.15.152:0/207527967 conn(0x7fcd900fd8f0 msgr2=0x7fcd900fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:52.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.292+0000 7fcd95ac1640 1 -- 172.21.15.152:0/207527967 shutdown_connections 2024-09-11T22:59:52.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:52.292+0000 7fcd95ac1640 1 -- 172.21.15.152:0/207527967 wait complete. 2024-09-11T22:59:52.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:52 smithi152 ceph-mon[31181]: pgmap v212: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-09-11T22:59:52.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:52 smithi191 ceph-mon[35211]: pgmap v212: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-09-11T22:59:52.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:52 smithi204 ceph-mon[35454]: pgmap v212: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-09-11T22:59:53.108 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:59:53.362 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:53.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.883+0000 7fe65b6c1640 1 -- 172.21.15.152:0/545822563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe654100a80 msgr2=0x7fe654100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:53.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.883+0000 7fe65b6c1640 1 --2- 172.21.15.152:0/545822563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe654100a80 0x7fe654100ee0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fe648009cd0 tx=0x7fe64802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.883+0000 7fe65b6c1640 1 -- 172.21.15.152:0/545822563 shutdown_connections 2024-09-11T22:59:53.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.883+0000 7fe65b6c1640 1 --2- 172.21.15.152:0/545822563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe654101420 0x7fe65410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.883+0000 7fe65b6c1640 1 --2- 172.21.15.152:0/545822563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe654100a80 0x7fe654100ee0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.883+0000 7fe65b6c1640 1 --2- 172.21.15.152:0/545822563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe654106a80 0x7fe654106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.883+0000 7fe65b6c1640 1 -- 172.21.15.152:0/545822563 >> 172.21.15.152:0/545822563 conn(0x7fe6540fc7a0 msgr2=0x7fe6540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:53.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.883+0000 7fe65b6c1640 1 -- 172.21.15.152:0/545822563 shutdown_connections 2024-09-11T22:59:53.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.883+0000 7fe65b6c1640 1 -- 172.21.15.152:0/545822563 wait complete. 2024-09-11T22:59:53.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.883+0000 7fe65b6c1640 1 Processor -- start 2024-09-11T22:59:53.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.883+0000 7fe65b6c1640 1 -- start start 2024-09-11T22:59:53.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe65b6c1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe654100a80 0x7fe6541985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe65b6c1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe654101420 0x7fe654198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe65b6c1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe654106a80 0x7fe65419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe65b6c1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe654112020 con 0x7fe654101420 2024-09-11T22:59:53.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe65b6c1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe654111ea0 con 0x7fe654106a80 2024-09-11T22:59:53.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe65b6c1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe6541121a0 con 0x7fe654100a80 2024-09-11T22:59:53.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe658c5c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe654101420 0x7fe654198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe658c5c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe654101420 0x7fe654198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57624/0 (socket says 172.21.15.152:57624) 2024-09-11T22:59:53.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe658c5c640 1 -- 172.21.15.152:0/2288623099 learned_addr learned my addr 172.21.15.152:0/2288623099 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:53.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe65945d640 1 --2- 172.21.15.152:0/2288623099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe654100a80 0x7fe6541985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe659c5e640 1 --2- 172.21.15.152:0/2288623099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe654106a80 0x7fe65419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe658c5c640 1 -- 172.21.15.152:0/2288623099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe654100a80 msgr2=0x7fe6541985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:53.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe658c5c640 1 --2- 172.21.15.152:0/2288623099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe654100a80 0x7fe6541985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe658c5c640 1 -- 172.21.15.152:0/2288623099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe654106a80 msgr2=0x7fe65419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:53.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe658c5c640 1 --2- 172.21.15.152:0/2288623099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe654106a80 0x7fe65419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe658c5c640 1 -- 172.21.15.152:0/2288623099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65419d570 con 0x7fe654101420 2024-09-11T22:59:53.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe659c5e640 1 --2- 172.21.15.152:0/2288623099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe654106a80 0x7fe65419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:53.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe658c5c640 1 --2- 172.21.15.152:0/2288623099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe654101420 0x7fe654198b00 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe64802ae00 tx=0x7fe648003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:53.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe6527fc640 1 -- 172.21.15.152:0/2288623099 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648004240 con 0x7fe654101420 2024-09-11T22:59:53.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe65b6c1640 1 -- 172.21.15.152:0/2288623099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65419d800 con 0x7fe654101420 2024-09-11T22:59:53.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe6527fc640 1 -- 172.21.15.152:0/2288623099 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6480305a0 con 0x7fe654101420 2024-09-11T22:59:53.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe6527fc640 1 -- 172.21.15.152:0/2288623099 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64803a660 con 0x7fe654101420 2024-09-11T22:59:53.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.884+0000 7fe65b6c1640 1 -- 172.21.15.152:0/2288623099 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6541a5140 con 0x7fe654101420 2024-09-11T22:59:53.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.887+0000 7fe6527fc640 1 -- 172.21.15.152:0/2288623099 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe648030ac0 con 0x7fe654101420 2024-09-11T22:59:53.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.887+0000 7fe65da9a740 1 Processor -- start 2024-09-11T22:59:53.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.887+0000 7fe65da9a740 1 -- start start 2024-09-11T22:59:53.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.887+0000 7fe65da9a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c045090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.887+0000 7fe65da9a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c0455d0 0x56104c045a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.887+0000 7fe65da9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c050370 0x56104c052760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.887+0000 7fe65da9a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56104bfab2f0 con 0x56104c046b40 2024-09-11T22:59:53.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.887+0000 7fe65da9a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56104bf52cf0 con 0x56104c0455d0 2024-09-11T22:59:53.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.887+0000 7fe65da9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56104bfa72b0 con 0x56104c050370 2024-09-11T22:59:53.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe659c5e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c045090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe659c5e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c045090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57640/0 (socket says 172.21.15.152:57640) 2024-09-11T22:59:53.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe659c5e640 1 -- 172.21.15.152:0/4101201820 learned_addr learned my addr 172.21.15.152:0/4101201820 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:53.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe6527fc640 1 --2- 172.21.15.152:0/2288623099 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe63c0a2cf0 0x7fe63c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe65945d640 1 --2- 172.21.15.152:0/4101201820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c0455d0 0x56104c045a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe658c5c640 1 --2- 172.21.15.152:0/4101201820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c050370 0x56104c052760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe659c5e640 1 --2- 172.21.15.152:0/2288623099 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe63c0a2cf0 0x7fe63c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe6527fc640 1 -- 172.21.15.152:0/2288623099 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fe64802e030 con 0x7fe654101420 2024-09-11T22:59:53.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe659c5e640 1 -- 172.21.15.152:0/4101201820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c050370 msgr2=0x56104c052760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:53.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe659c5e640 1 --2- 172.21.15.152:0/4101201820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c050370 0x56104c052760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe659c5e640 1 -- 172.21.15.152:0/4101201820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c0455d0 msgr2=0x56104c045a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:53.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe659c5e640 1 --2- 172.21.15.152:0/4101201820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c0455d0 0x56104c045a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe659c5e640 1 -- 172.21.15.152:0/4101201820 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104bf373a0 con 0x56104c046b40 2024-09-11T22:59:53.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe659c5e640 1 --2- 172.21.15.152:0/4101201820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c045090 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe6440143c0 tx=0x7fe644014880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68364485e4c213e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:53.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe63b7fe640 1 -- 172.21.15.152:0/4101201820 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64401b030 con 0x56104c046b40 2024-09-11T22:59:53.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe63b7fe640 1 -- 172.21.15.152:0/4101201820 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe644010040 con 0x56104c046b40 2024-09-11T22:59:53.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe659c5e640 1 --2- 172.21.15.152:0/2288623099 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe63c0a2cf0 0x7fe63c0a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe64401e000 tx=0x7fe64401a000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:53.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.888+0000 7fe63b7fe640 1 -- 172.21.15.152:0/4101201820 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64400f050 con 0x56104c046b40 2024-09-11T22:59:53.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.889+0000 7fe65da9a740 1 -- 172.21.15.152:0/4101201820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 msgr2=0x56104c045090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:53.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.889+0000 7fe65da9a740 1 --2- 172.21.15.152:0/4101201820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c045090 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe6440143c0 tx=0x7fe644014880 comp rx=0 tx=0).stop 2024-09-11T22:59:53.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.889+0000 7fe65da9a740 1 -- 172.21.15.152:0/4101201820 shutdown_connections 2024-09-11T22:59:53.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.889+0000 7fe65da9a740 1 --2- 172.21.15.152:0/4101201820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c050370 0x56104c052760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.889+0000 7fe65da9a740 1 --2- 172.21.15.152:0/4101201820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c0455d0 0x56104c045a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.889+0000 7fe65da9a740 1 --2- 172.21.15.152:0/4101201820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c045090 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.889+0000 7fe65da9a740 1 -- 172.21.15.152:0/4101201820 >> 172.21.15.152:0/4101201820 conn(0x56104c00e310 msgr2=0x56104c041c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:53.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.889+0000 7fe65da9a740 1 -- 172.21.15.152:0/4101201820 shutdown_connections 2024-09-11T22:59:53.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.889+0000 7fe65da9a740 1 -- 172.21.15.152:0/4101201820 wait complete. 2024-09-11T22:59:53.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.889+0000 7fe65da9a740 1 -- start start 2024-09-11T22:59:53.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe65da9a740 1 Processor -- start 2024-09-11T22:59:53.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe65da9a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c0455d0 0x56104c04fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe65da9a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c04d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe65da9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c050370 0x56104c04d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe65da9a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56104c053070 con 0x56104c046b40 2024-09-11T22:59:53.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe65da9a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56104bf66060 con 0x56104c0455d0 2024-09-11T22:59:53.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe65da9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56104c0531f0 con 0x56104c050370 2024-09-11T22:59:53.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe65da9a740 10 client.0 _ll_register_callbacks cb 0x56104bcb98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:53.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe658c5c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c04d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe658c5c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c04d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57652/0 (socket says 172.21.15.152:57652) 2024-09-11T22:59:53.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe658c5c640 1 -- 172.21.15.152:0/1245737119 learned_addr learned my addr 172.21.15.152:0/1245737119 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:53.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe65945d640 1 --2- 172.21.15.152:0/1245737119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c0455d0 0x56104c04fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe659c5e640 1 --2- 172.21.15.152:0/1245737119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c050370 0x56104c04d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe658c5c640 1 -- 172.21.15.152:0/1245737119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c050370 msgr2=0x56104c04d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:53.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe658c5c640 1 --2- 172.21.15.152:0/1245737119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c050370 0x56104c04d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe658c5c640 1 -- 172.21.15.152:0/1245737119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c0455d0 msgr2=0x56104c04fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:53.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe658c5c640 1 --2- 172.21.15.152:0/1245737119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c0455d0 0x56104c04fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe658c5c640 1 -- 172.21.15.152:0/1245737119 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56104bd1c3c0 con 0x56104c046b40 2024-09-11T22:59:53.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.890+0000 7fe65945d640 1 --2- 172.21.15.152:0/1245737119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c0455d0 0x56104c04fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:53.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.891+0000 7fe658c5c640 1 --2- 172.21.15.152:0/1245737119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c04d030 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe6480bc900 tx=0x7fe648042f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314d54145261814c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:53.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.891+0000 7fe638ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:53.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.891+0000 7fe638ff9640 1 -- 172.21.15.152:0/1245737119 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6480bcc00 con 0x56104c046b40 2024-09-11T22:59:53.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.891+0000 7fe638ff9640 1 -- 172.21.15.152:0/1245737119 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fe648030750 con 0x56104c046b40 2024-09-11T22:59:53.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.891+0000 7fe65da9a740 1 -- 172.21.15.152:0/1245737119 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe644010040 con 0x56104c046b40 2024-09-11T22:59:53.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.891+0000 7fe638ff9640 1 -- 172.21.15.152:0/1245737119 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe6480c0fb0 con 0x56104c046b40 2024-09-11T22:59:53.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.891+0000 7fe65da9a740 10 client.17382 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:53.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.891+0000 7fe65da9a740 1 -- 172.21.15.152:0/1245737119 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56104c05eab0 con 0x56104c046b40 2024-09-11T22:59:53.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.891+0000 7fe638ff9640 1 -- 172.21.15.152:0/1245737119 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe644010040 con 0x56104c046b40 2024-09-11T22:59:53.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.891+0000 7fe638ff9640 1 -- 172.21.15.152:0/1245737119 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe64804d020 con 0x56104c046b40 2024-09-11T22:59:53.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.891+0000 7fe65da9a740 10 client.17382 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:53.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.891+0000 7fe65da9a740 10 client.17382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:59:53.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.891+0000 7fe65da9a740 20 client.17382 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:53.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.891+0000 7fe65da9a740 1 --2- 172.21.15.152:0/1245737119 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56104c05fea0 0x56104c0622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.891+0000 7fe65da9a740 4 client.17382 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:59:53.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.891+0000 7fe65da9a740 1 -- 172.21.15.152:0/1245737119 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56104bfb8860 con 0x56104c05fea0 2024-09-11T22:59:53.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.892+0000 7fe65945d640 1 --2- 172.21.15.152:0/1245737119 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56104c05fea0 0x56104c0622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.892+0000 7fe638ff9640 10 client.17382 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:53.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.892+0000 7fe65945d640 1 --2- 172.21.15.152:0/1245737119 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56104c05fea0 0x56104c0622b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc077710cc193e0a server_cookie=4034f5ab2906cce3 in_seq=0 out_seq=0 2024-09-11T22:59:53.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.893+0000 7fe638ff9640 1 -- 172.21.15.152:0/1245737119 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56104bfb8860 con 0x56104c05fea0 2024-09-11T22:59:53.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.893+0000 7fe638ff9640 10 client.17382 handle_command_reply: tid=0 2024-09-11T22:59:53.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.893+0000 7fe65da9a740 1 client.17382 shutdown 2024-09-11T22:59:53.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.893+0000 7fe65da9a740 1 -- 172.21.15.152:0/1245737119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 msgr2=0x56104c04d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:53.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.893+0000 7fe65da9a740 1 --2- 172.21.15.152:0/1245737119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c04d030 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe6480bc900 tx=0x7fe648042f30 comp rx=0 tx=0).stop 2024-09-11T22:59:53.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.893+0000 7fe65da9a740 1 -- 172.21.15.152:0/1245737119 shutdown_connections 2024-09-11T22:59:53.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.893+0000 7fe65da9a740 1 --2- 172.21.15.152:0/1245737119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c050370 0x56104c04d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.893+0000 7fe65da9a740 1 --2- 172.21.15.152:0/1245737119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c04d030 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.893+0000 7fe65da9a740 1 --2- 172.21.15.152:0/1245737119 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56104c05fea0 0x56104c0622b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.893+0000 7fe65da9a740 1 --2- 172.21.15.152:0/1245737119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c0455d0 0x56104c04fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.893+0000 7fe638ff9640 0 client.17382 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:53.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.893+0000 7fe65da9a740 1 -- 172.21.15.152:0/1245737119 >> 172.21.15.152:0/1245737119 conn(0x56104c00e310 msgr2=0x56104c047930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:53.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.893+0000 7fe65da9a740 1 -- 172.21.15.152:0/1245737119 shutdown_connections 2024-09-11T22:59:53.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.893+0000 7fe65da9a740 1 -- 172.21.15.152:0/1245737119 wait complete. 2024-09-11T22:59:53.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.893+0000 7fe65da9a740 20 client.17382 trim_cache size 0 max 16384 2024-09-11T22:59:53.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.906+0000 7fe65da9a740 1 Processor -- start 2024-09-11T22:59:53.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.906+0000 7fe65da9a740 1 -- start start 2024-09-11T22:59:53.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.906+0000 7fe65da9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c0455d0 0x56104c048120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.906+0000 7fe65da9a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c048660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.906+0000 7fe65da9a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c050370 0x56104c067b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.906+0000 7fe65da9a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe6541ad810 con 0x56104c046b40 2024-09-11T22:59:53.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.906+0000 7fe65da9a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe6541af350 con 0x56104c050370 2024-09-11T22:59:53.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.906+0000 7fe65da9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe6541adee0 con 0x56104c0455d0 2024-09-11T22:59:53.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.906+0000 7fe658c5c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c048660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.906+0000 7fe658c5c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c048660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57658/0 (socket says 172.21.15.152:57658) 2024-09-11T22:59:53.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.906+0000 7fe658c5c640 1 -- 172.21.15.152:0/1371545232 learned_addr learned my addr 172.21.15.152:0/1371545232 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:53.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.907+0000 7fe65945d640 1 --2- 172.21.15.152:0/1371545232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c0455d0 0x56104c048120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.907+0000 7fe659c5e640 1 --2- 172.21.15.152:0/1371545232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c050370 0x56104c067b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.907+0000 7fe658c5c640 1 -- 172.21.15.152:0/1371545232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c0455d0 msgr2=0x56104c048120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:53.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.907+0000 7fe658c5c640 1 --2- 172.21.15.152:0/1371545232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c0455d0 0x56104c048120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.907+0000 7fe658c5c640 1 -- 172.21.15.152:0/1371545232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c050370 msgr2=0x56104c067b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:53.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.907+0000 7fe658c5c640 1 --2- 172.21.15.152:0/1371545232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c050370 0x56104c067b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.907+0000 7fe658c5c640 1 -- 172.21.15.152:0/1371545232 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104c062900 con 0x56104c046b40 2024-09-11T22:59:53.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.907+0000 7fe658c5c640 1 --2- 172.21.15.152:0/1371545232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c048660 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe6480c5340 tx=0x7fe6480093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b461db7fb79eb938 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:53.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.907+0000 7fe63affd640 1 -- 172.21.15.152:0/1371545232 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6480db070 con 0x56104c046b40 2024-09-11T22:59:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.907+0000 7fe63affd640 1 -- 172.21.15.152:0/1371545232 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe648036040 con 0x56104c046b40 2024-09-11T22:59:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.907+0000 7fe63affd640 1 -- 172.21.15.152:0/1371545232 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6480b73c0 con 0x56104c046b40 2024-09-11T22:59:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.908+0000 7fe65da9a740 1 -- 172.21.15.152:0/1371545232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 msgr2=0x56104c048660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.908+0000 7fe65da9a740 1 --2- 172.21.15.152:0/1371545232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c048660 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe6480c5340 tx=0x7fe6480093f0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.908+0000 7fe65da9a740 1 -- 172.21.15.152:0/1371545232 shutdown_connections 2024-09-11T22:59:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.908+0000 7fe65da9a740 1 --2- 172.21.15.152:0/1371545232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c050370 0x56104c067b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.908+0000 7fe65da9a740 1 --2- 172.21.15.152:0/1371545232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c046b40 0x56104c048660 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.908+0000 7fe65da9a740 1 --2- 172.21.15.152:0/1371545232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c0455d0 0x56104c048120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.908+0000 7fe65da9a740 1 -- 172.21.15.152:0/1371545232 >> 172.21.15.152:0/1371545232 conn(0x56104c00e310 msgr2=0x56104c04f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.908+0000 7fe65da9a740 1 -- 172.21.15.152:0/1371545232 shutdown_connections 2024-09-11T22:59:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.908+0000 7fe65da9a740 1 -- 172.21.15.152:0/1371545232 wait complete. 2024-09-11T22:59:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.908+0000 7fe65da9a740 1 -- start start 2024-09-11T22:59:53.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.908+0000 7fe65da9a740 1 Processor -- start 2024-09-11T22:59:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.908+0000 7fe65da9a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c0455d0 0x56104c054a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65da9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c046b40 0x56104c054f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65da9a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c050370 0x56104c0c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65da9a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe6541aec80 con 0x56104c0455d0 2024-09-11T22:59:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65da9a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe6541ae5b0 con 0x56104c050370 2024-09-11T22:59:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65da9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe6541ab7f0 con 0x56104c046b40 2024-09-11T22:59:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65da9a740 10 client.0 _ll_register_callbacks cb 0x7fe648009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65945d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c0455d0 0x56104c054a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65945d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c0455d0 0x56104c054a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57670/0 (socket says 172.21.15.152:57670) 2024-09-11T22:59:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65945d640 1 -- 172.21.15.152:0/3524416543 learned_addr learned my addr 172.21.15.152:0/3524416543 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe658c5c640 1 --2- 172.21.15.152:0/3524416543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c046b40 0x56104c054f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe659c5e640 1 --2- 172.21.15.152:0/3524416543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c050370 0x56104c0c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65945d640 1 -- 172.21.15.152:0/3524416543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c046b40 msgr2=0x56104c054f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65945d640 1 --2- 172.21.15.152:0/3524416543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c046b40 0x56104c054f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65945d640 1 -- 172.21.15.152:0/3524416543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c050370 msgr2=0x56104c0c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65945d640 1 --2- 172.21.15.152:0/3524416543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c050370 0x56104c0c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65945d640 1 -- 172.21.15.152:0/3524416543 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56104c0452d0 con 0x56104c0455d0 2024-09-11T22:59:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe659c5e640 1 --2- 172.21.15.152:0/3524416543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c050370 0x56104c0c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:59:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65945d640 1 --2- 172.21.15.152:0/3524416543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c0455d0 0x56104c054a40 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe64c004810 tx=0x7fe64c016800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926ae6e987129dba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe638ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe638ff9640 1 -- 172.21.15.152:0/3524416543 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c0155d0 con 0x56104c0455d0 2024-09-11T22:59:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe638ff9640 1 -- 172.21.15.152:0/3524416543 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fe64c015770 con 0x56104c0455d0 2024-09-11T22:59:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65da9a740 1 -- 172.21.15.152:0/3524416543 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe648036040 con 0x56104c0455d0 2024-09-11T22:59:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe638ff9640 1 -- 172.21.15.152:0/3524416543 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe64c015970 con 0x56104c0455d0 2024-09-11T22:59:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65da9a740 10 client.17388 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:53.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.909+0000 7fe65da9a740 1 -- 172.21.15.152:0/3524416543 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56104c0cb360 con 0x56104c0455d0 2024-09-11T22:59:53.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.910+0000 7fe638ff9640 1 -- 172.21.15.152:0/3524416543 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe648036040 con 0x56104c0455d0 2024-09-11T22:59:53.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.910+0000 7fe638ff9640 1 -- 172.21.15.152:0/3524416543 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe64c02fc60 con 0x56104c0455d0 2024-09-11T22:59:53.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.910+0000 7fe65da9a740 10 client.17388 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:53.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.910+0000 7fe65da9a740 10 client.17388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:59:53.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.910+0000 7fe65da9a740 20 client.17388 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:53.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.910+0000 7fe65da9a740 1 --2- 172.21.15.152:0/3524416543 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56104c0cba10 0x56104c0cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:53.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.910+0000 7fe65da9a740 4 client.17388 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:59:53.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.910+0000 7fe65da9a740 1 -- 172.21.15.152:0/3524416543 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56104bfb2d70 con 0x56104c0cba10 2024-09-11T22:59:53.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.910+0000 7fe658c5c640 1 --2- 172.21.15.152:0/3524416543 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56104c0cba10 0x56104c0cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:53.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe658c5c640 1 --2- 172.21.15.152:0/3524416543 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56104c0cba10 0x56104c0cde30 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3df8f80a50e5d567 server_cookie=f123564ac9631c2a in_seq=0 out_seq=0 2024-09-11T22:59:53.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe638ff9640 10 client.17388 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:53.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe638ff9640 1 -- 172.21.15.152:0/3524416543 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56104bfb2d70 con 0x56104c0cba10 2024-09-11T22:59:53.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe638ff9640 10 client.17388 handle_command_reply: tid=0 2024-09-11T22:59:53.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe65da9a740 1 client.17388 shutdown 2024-09-11T22:59:53.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe65da9a740 1 -- 172.21.15.152:0/3524416543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c0455d0 msgr2=0x56104c054a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:53.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe65da9a740 1 --2- 172.21.15.152:0/3524416543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c0455d0 0x56104c054a40 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe64c004810 tx=0x7fe64c016800 comp rx=0 tx=0).stop 2024-09-11T22:59:53.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe65da9a740 1 -- 172.21.15.152:0/3524416543 shutdown_connections 2024-09-11T22:59:53.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe65da9a740 1 --2- 172.21.15.152:0/3524416543 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56104c0cba10 0x56104c0cde30 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe65da9a740 1 --2- 172.21.15.152:0/3524416543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56104c050370 0x56104c0c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe65da9a740 1 --2- 172.21.15.152:0/3524416543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56104c046b40 0x56104c054f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe65da9a740 1 --2- 172.21.15.152:0/3524416543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56104c0455d0 0x56104c054a40 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe65da9a740 1 -- 172.21.15.152:0/3524416543 >> 172.21.15.152:0/3524416543 conn(0x56104c00e310 msgr2=0x56104c0404e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:53.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe638ff9640 0 client.17388 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:53.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe65da9a740 1 -- 172.21.15.152:0/3524416543 shutdown_connections 2024-09-11T22:59:53.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe65da9a740 1 -- 172.21.15.152:0/3524416543 wait complete. 2024-09-11T22:59:53.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.911+0000 7fe65da9a740 20 client.17388 trim_cache size 0 max 16384 2024-09-11T22:59:53.923 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T22:59:53.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.912+0000 7fe65b6c1640 1 -- 172.21.15.152:0/2288623099 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe63c0a2cf0 msgr2=0x7fe63c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:53.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.912+0000 7fe65b6c1640 1 --2- 172.21.15.152:0/2288623099 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe63c0a2cf0 0x7fe63c0a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe64401e000 tx=0x7fe64401a000 comp rx=0 tx=0).stop 2024-09-11T22:59:53.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.912+0000 7fe65b6c1640 1 -- 172.21.15.152:0/2288623099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe654101420 msgr2=0x7fe654198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:53.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.912+0000 7fe65b6c1640 1 --2- 172.21.15.152:0/2288623099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe654101420 0x7fe654198b00 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fe64802ae00 tx=0x7fe648003ba0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.912+0000 7fe65b6c1640 1 -- 172.21.15.152:0/2288623099 shutdown_connections 2024-09-11T22:59:53.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.912+0000 7fe65b6c1640 1 --2- 172.21.15.152:0/2288623099 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe63c0a2cf0 0x7fe63c0a51b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.912+0000 7fe65b6c1640 1 --2- 172.21.15.152:0/2288623099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe654106a80 0x7fe65419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.912+0000 7fe65b6c1640 1 --2- 172.21.15.152:0/2288623099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe654101420 0x7fe654198b00 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.912+0000 7fe65b6c1640 1 --2- 172.21.15.152:0/2288623099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe654100a80 0x7fe6541985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:53.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.912+0000 7fe65b6c1640 1 -- 172.21.15.152:0/2288623099 >> 172.21.15.152:0/2288623099 conn(0x7fe6540fc7a0 msgr2=0x7fe6540fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:53.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.912+0000 7fe65b6c1640 1 -- 172.21.15.152:0/2288623099 shutdown_connections 2024-09-11T22:59:53.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:53.912+0000 7fe65b6c1640 1 -- 172.21.15.152:0/2288623099 wait complete. 2024-09-11T22:59:54.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:54 smithi152 ceph-mon[31181]: pgmap v213: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:59:54.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:54 smithi191 ceph-mon[35211]: pgmap v213: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:59:54.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:54 smithi204 ceph-mon[35454]: pgmap v213: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T22:59:55.745 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:59:55.746 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:59:56.006 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:56.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.515+0000 7fb414c3c640 1 -- 172.21.15.152:0/2589122020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb410100460 msgr2=0x7fb410100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:56.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.515+0000 7fb414c3c640 1 --2- 172.21.15.152:0/2589122020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb410100460 0x7fb410100840 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb400009900 tx=0x7fb40002a3f0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.515+0000 7fb414c3c640 1 -- 172.21.15.152:0/2589122020 shutdown_connections 2024-09-11T22:59:56.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.515+0000 7fb414c3c640 1 --2- 172.21.15.152:0/2589122020 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb410105590 0x7fb41010d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.515+0000 7fb414c3c640 1 --2- 172.21.15.152:0/2589122020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb410100e10 0x7fb410104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.515+0000 7fb414c3c640 1 --2- 172.21.15.152:0/2589122020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb410100460 0x7fb410100840 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.515+0000 7fb414c3c640 1 -- 172.21.15.152:0/2589122020 >> 172.21.15.152:0/2589122020 conn(0x7fb4100fbee0 msgr2=0x7fb4100fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.515+0000 7fb414c3c640 1 -- 172.21.15.152:0/2589122020 shutdown_connections 2024-09-11T22:59:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.515+0000 7fb414c3c640 1 -- 172.21.15.152:0/2589122020 wait complete. 2024-09-11T22:59:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb414c3c640 1 Processor -- start 2024-09-11T22:59:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb414c3c640 1 -- start start 2024-09-11T22:59:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb414c3c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb410100460 0x7fb41010e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb414c3c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb410100e10 0x7fb41010ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb414c3c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb410105590 0x7fb41010f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb414c3c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb410111f70 con 0x7fb410100e10 2024-09-11T22:59:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb414c3c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb410111df0 con 0x7fb410100460 2024-09-11T22:59:56.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb414c3c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101120f0 con 0x7fb410105590 2024-09-11T22:59:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb407fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb410100e10 0x7fb41010ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb407fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb410100e10 0x7fb41010ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57674/0 (socket says 172.21.15.152:57674) 2024-09-11T22:59:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb407fff640 1 -- 172.21.15.152:0/3940885914 learned_addr learned my addr 172.21.15.152:0/3940885914 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb40e59c640 1 --2- 172.21.15.152:0/3940885914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb410100460 0x7fb41010e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb40ed9d640 1 --2- 172.21.15.152:0/3940885914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb410105590 0x7fb41010f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb407fff640 1 -- 172.21.15.152:0/3940885914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb410105590 msgr2=0x7fb41010f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb407fff640 1 --2- 172.21.15.152:0/3940885914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb410105590 0x7fb41010f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb407fff640 1 -- 172.21.15.152:0/3940885914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb410100460 msgr2=0x7fb41010e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb407fff640 1 --2- 172.21.15.152:0/3940885914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb410100460 0x7fb41010e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb407fff640 1 -- 172.21.15.152:0/3940885914 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4101a6830 con 0x7fb410100e10 2024-09-11T22:59:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.516+0000 7fb40e59c640 1 --2- 172.21.15.152:0/3940885914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb410100460 0x7fb41010e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.517+0000 7fb407fff640 1 --2- 172.21.15.152:0/3940885914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb410100e10 0x7fb41010ed70 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800c920 tx=0x7fb3f800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.517+0000 7fb4077fe640 1 -- 172.21.15.152:0/3940885914 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8007c70 con 0x7fb410100e10 2024-09-11T22:59:56.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.517+0000 7fb414c3c640 1 -- 172.21.15.152:0/3940885914 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4101a6b20 con 0x7fb410100e10 2024-09-11T22:59:56.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.517+0000 7fb4077fe640 1 -- 172.21.15.152:0/3940885914 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3f8007e10 con 0x7fb410100e10 2024-09-11T22:59:56.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.517+0000 7fb4077fe640 1 -- 172.21.15.152:0/3940885914 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8018890 con 0x7fb410100e10 2024-09-11T22:59:56.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.517+0000 7fb414c3c640 1 -- 172.21.15.152:0/3940885914 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4101a7060 con 0x7fb410100e10 2024-09-11T22:59:56.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.519+0000 7fb4077fe640 1 -- 172.21.15.152:0/3940885914 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb3f8018a70 con 0x7fb410100e10 2024-09-11T22:59:56.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.519+0000 7fb417015740 1 Processor -- start 2024-09-11T22:59:56.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb417015740 1 -- start start 2024-09-11T22:59:56.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb417015740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258cb60 0x55a5b258cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb417015740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b258d480 0x55a5b2597800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb417015740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b2597d40 0x55a5b259a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb417015740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a5b24f2280 con 0x55a5b258d480 2024-09-11T22:59:56.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb417015740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a5b2499b10 con 0x55a5b2597d40 2024-09-11T22:59:56.540 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb417015740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a5b24ee240 con 0x55a5b258cb60 2024-09-11T22:59:56.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb40e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b258d480 0x55a5b2597800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb40ed9d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258cb60 0x55a5b258cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb40e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b258d480 0x55a5b2597800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57682/0 (socket says 172.21.15.152:57682) 2024-09-11T22:59:56.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb40e59c640 1 -- 172.21.15.152:0/1042957468 learned_addr learned my addr 172.21.15.152:0/1042957468 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:56.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb4077fe640 1 --2- 172.21.15.152:0/3940885914 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb3fc0a2cf0 0x7fb3fc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb407fff640 1 --2- 172.21.15.152:0/1042957468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b2597d40 0x55a5b259a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb40ed9d640 1 --2- 172.21.15.152:0/3940885914 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb3fc0a2cf0 0x7fb3fc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb40e59c640 1 -- 172.21.15.152:0/1042957468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258cb60 msgr2=0x55a5b258cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:56.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb40e59c640 1 --2- 172.21.15.152:0/1042957468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258cb60 0x55a5b258cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb4077fe640 1 -- 172.21.15.152:0/3940885914 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb3f80cf550 con 0x7fb410100e10 2024-09-11T22:59:56.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb40e59c640 1 -- 172.21.15.152:0/1042957468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b2597d40 msgr2=0x55a5b259a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:56.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.520+0000 7fb40e59c640 1 --2- 172.21.15.152:0/1042957468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b2597d40 0x55a5b259a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.541 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.521+0000 7fb40e59c640 1 -- 172.21.15.152:0/1042957468 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b24a61f0 con 0x55a5b258d480 2024-09-11T22:59:56.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.521+0000 7fb40ed9d640 1 --2- 172.21.15.152:0/1042957468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258cb60 0x55a5b258cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:59:56.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.521+0000 7fb407fff640 1 --2- 172.21.15.152:0/1042957468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b2597d40 0x55a5b259a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:59:56.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.521+0000 7fb40e59c640 1 --2- 172.21.15.152:0/1042957468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b258d480 0x55a5b2597800 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fb400009900 tx=0x7fb4000316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eba184cbc3be56f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:56.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.521+0000 7fb40ed9d640 1 --2- 172.21.15.152:0/3940885914 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb3fc0a2cf0 0x7fb3fc0a51b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb408007310 tx=0x7fb408004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:56.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.521+0000 7fb404ff9640 1 -- 172.21.15.152:0/1042957468 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40003f070 con 0x55a5b258d480 2024-09-11T22:59:56.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.521+0000 7fb404ff9640 1 -- 172.21.15.152:0/1042957468 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4000035f0 con 0x55a5b258d480 2024-09-11T22:59:56.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.521+0000 7fb404ff9640 1 -- 172.21.15.152:0/1042957468 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400035070 con 0x55a5b258d480 2024-09-11T22:59:56.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.522+0000 7fb417015740 1 -- 172.21.15.152:0/1042957468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b258d480 msgr2=0x55a5b2597800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:56.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.522+0000 7fb417015740 1 --2- 172.21.15.152:0/1042957468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b258d480 0x55a5b2597800 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fb400009900 tx=0x7fb4000316e0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.522+0000 7fb417015740 1 -- 172.21.15.152:0/1042957468 shutdown_connections 2024-09-11T22:59:56.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.522+0000 7fb417015740 1 --2- 172.21.15.152:0/1042957468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b2597d40 0x55a5b259a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.522+0000 7fb417015740 1 --2- 172.21.15.152:0/1042957468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b258d480 0x55a5b2597800 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.522+0000 7fb417015740 1 --2- 172.21.15.152:0/1042957468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258cb60 0x55a5b258cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.522+0000 7fb417015740 1 -- 172.21.15.152:0/1042957468 >> 172.21.15.152:0/1042957468 conn(0x55a5b25890f0 msgr2=0x55a5b25894d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:56.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.522+0000 7fb417015740 1 -- 172.21.15.152:0/1042957468 shutdown_connections 2024-09-11T22:59:56.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.522+0000 7fb417015740 1 -- 172.21.15.152:0/1042957468 wait complete. 2024-09-11T22:59:56.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.522+0000 7fb417015740 1 -- start start 2024-09-11T22:59:56.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.522+0000 7fb417015740 1 Processor -- start 2024-09-11T22:59:56.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.522+0000 7fb417015740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b258cb60 0x55a5b2594b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb417015740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258d480 0x55a5b25950d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb417015740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b2597d40 0x55a5b2595610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb417015740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a5b259a910 con 0x55a5b2597d40 2024-09-11T22:59:56.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb417015740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a5b24acf10 con 0x55a5b258cb60 2024-09-11T22:59:56.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb417015740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a5b259aa90 con 0x55a5b258d480 2024-09-11T22:59:56.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb417015740 10 client.0 _ll_register_callbacks cb 0x55a5b2475a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:56.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb40ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b2597d40 0x55a5b2595610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb40ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b2597d40 0x55a5b2595610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57686/0 (socket says 172.21.15.152:57686) 2024-09-11T22:59:56.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb40ed9d640 1 -- 172.21.15.152:0/284853863 learned_addr learned my addr 172.21.15.152:0/284853863 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:56.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb40e59c640 1 --2- 172.21.15.152:0/284853863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b258cb60 0x55a5b2594b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb407fff640 1 --2- 172.21.15.152:0/284853863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258d480 0x55a5b25950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb40ed9d640 1 -- 172.21.15.152:0/284853863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258d480 msgr2=0x55a5b25950d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:56.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb40ed9d640 1 --2- 172.21.15.152:0/284853863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258d480 0x55a5b25950d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb40ed9d640 1 -- 172.21.15.152:0/284853863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b258cb60 msgr2=0x55a5b2594b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:56.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb40ed9d640 1 --2- 172.21.15.152:0/284853863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b258cb60 0x55a5b2594b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb40ed9d640 1 -- 172.21.15.152:0/284853863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b22636a0 con 0x55a5b2597d40 2024-09-11T22:59:56.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb40e59c640 1 --2- 172.21.15.152:0/284853863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b258cb60 0x55a5b2594b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:56.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb40ed9d640 1 --2- 172.21.15.152:0/284853863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b2597d40 0x55a5b2595610 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fb408011c90 tx=0x7fb408004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5d498aa39be194 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:56.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb3f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:56.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb3f5ffb640 1 -- 172.21.15.152:0/284853863 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408025020 con 0x55a5b2597d40 2024-09-11T22:59:56.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb3f5ffb640 1 -- 172.21.15.152:0/284853863 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb408021610 con 0x55a5b2597d40 2024-09-11T22:59:56.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb417015740 1 -- 172.21.15.152:0/284853863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4000035f0 con 0x55a5b2597d40 2024-09-11T22:59:56.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.523+0000 7fb3f5ffb640 1 -- 172.21.15.152:0/284853863 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb408030050 con 0x55a5b2597d40 2024-09-11T22:59:56.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.524+0000 7fb417015740 10 client.17400 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:56.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.524+0000 7fb417015740 1 -- 172.21.15.152:0/284853863 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5b25a64a0 con 0x55a5b2597d40 2024-09-11T22:59:56.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.524+0000 7fb3f5ffb640 1 -- 172.21.15.152:0/284853863 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4000035f0 con 0x55a5b2597d40 2024-09-11T22:59:56.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.524+0000 7fb3f5ffb640 1 -- 172.21.15.152:0/284853863 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb40800f850 con 0x55a5b2597d40 2024-09-11T22:59:56.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.524+0000 7fb417015740 10 client.17400 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:56.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.524+0000 7fb417015740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:59:56.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.524+0000 7fb417015740 20 client.17400 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:56.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.524+0000 7fb417015740 1 --2- 172.21.15.152:0/284853863 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a5b25a7960 0x55a5b25a9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.524+0000 7fb417015740 4 client.17400 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:59:56.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.524+0000 7fb417015740 1 -- 172.21.15.152:0/284853863 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5b24ff7f0 con 0x55a5b25a7960 2024-09-11T22:59:56.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.524+0000 7fb40e59c640 1 --2- 172.21.15.152:0/284853863 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a5b25a7960 0x55a5b25a9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.525+0000 7fb3f5ffb640 10 client.17400 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:56.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.525+0000 7fb40e59c640 1 --2- 172.21.15.152:0/284853863 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a5b25a7960 0x55a5b25a9e00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dffdaffd10670e07 server_cookie=7eb2471c59ededf1 in_seq=0 out_seq=0 2024-09-11T22:59:56.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.525+0000 7fb3f5ffb640 1 -- 172.21.15.152:0/284853863 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5b24ff7f0 con 0x55a5b25a7960 2024-09-11T22:59:56.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.525+0000 7fb3f5ffb640 10 client.17400 handle_command_reply: tid=0 2024-09-11T22:59:56.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.525+0000 7fb417015740 1 client.17400 shutdown 2024-09-11T22:59:56.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.525+0000 7fb417015740 1 -- 172.21.15.152:0/284853863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b2597d40 msgr2=0x55a5b2595610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:56.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.525+0000 7fb417015740 1 --2- 172.21.15.152:0/284853863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b2597d40 0x55a5b2595610 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fb408011c90 tx=0x7fb408004a40 comp rx=0 tx=0).stop 2024-09-11T22:59:56.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.525+0000 7fb417015740 1 -- 172.21.15.152:0/284853863 shutdown_connections 2024-09-11T22:59:56.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.525+0000 7fb417015740 1 --2- 172.21.15.152:0/284853863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b2597d40 0x55a5b2595610 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.525+0000 7fb417015740 1 --2- 172.21.15.152:0/284853863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258d480 0x55a5b25950d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.525+0000 7fb417015740 1 --2- 172.21.15.152:0/284853863 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a5b25a7960 0x55a5b25a9e00 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.525+0000 7fb3f5ffb640 0 client.17400 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:56.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.525+0000 7fb417015740 1 --2- 172.21.15.152:0/284853863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b258cb60 0x55a5b2594b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.525+0000 7fb417015740 1 -- 172.21.15.152:0/284853863 >> 172.21.15.152:0/284853863 conn(0x55a5b25890f0 msgr2=0x55a5b2588270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:56.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.526+0000 7fb417015740 1 -- 172.21.15.152:0/284853863 shutdown_connections 2024-09-11T22:59:56.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.526+0000 7fb417015740 1 -- 172.21.15.152:0/284853863 wait complete. 2024-09-11T22:59:56.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.526+0000 7fb417015740 20 client.17400 trim_cache size 0 max 16384 2024-09-11T22:59:56.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb417015740 1 Processor -- start 2024-09-11T22:59:56.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb417015740 1 -- start start 2024-09-11T22:59:56.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb417015740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b258cb60 0x55a5b25af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb417015740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258d480 0x55a5b25afc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb417015740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b2597d40 0x55a5b25962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb417015740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101af7b0 con 0x55a5b2597d40 2024-09-11T22:59:56.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb417015740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101b12f0 con 0x55a5b258cb60 2024-09-11T22:59:56.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb417015740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101afe80 con 0x55a5b258d480 2024-09-11T22:59:56.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb40ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b2597d40 0x55a5b25962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb40ed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b2597d40 0x55a5b25962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57698/0 (socket says 172.21.15.152:57698) 2024-09-11T22:59:56.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb40ed9d640 1 -- 172.21.15.152:0/279531419 learned_addr learned my addr 172.21.15.152:0/279531419 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:56.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb40e59c640 1 --2- 172.21.15.152:0/279531419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b258cb60 0x55a5b25af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb407fff640 1 --2- 172.21.15.152:0/279531419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258d480 0x55a5b25afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb40ed9d640 1 -- 172.21.15.152:0/279531419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258d480 msgr2=0x55a5b25afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:56.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb40ed9d640 1 --2- 172.21.15.152:0/279531419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258d480 0x55a5b25afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb40ed9d640 1 -- 172.21.15.152:0/279531419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b258cb60 msgr2=0x55a5b25af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:56.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb40ed9d640 1 --2- 172.21.15.152:0/279531419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b258cb60 0x55a5b25af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.543+0000 7fb40ed9d640 1 -- 172.21.15.152:0/279531419 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b2583680 con 0x55a5b2597d40 2024-09-11T22:59:56.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.544+0000 7fb40ed9d640 1 --2- 172.21.15.152:0/279531419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b2597d40 0x55a5b25962e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb408007620 tx=0x7fb408007fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f098ed15cd8de8c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:56.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.544+0000 7fb3f7fff640 1 -- 172.21.15.152:0/279531419 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408025030 con 0x55a5b2597d40 2024-09-11T22:59:56.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.544+0000 7fb3f7fff640 1 -- 172.21.15.152:0/279531419 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb40801a040 con 0x55a5b2597d40 2024-09-11T22:59:56.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.544+0000 7fb3f7fff640 1 -- 172.21.15.152:0/279531419 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408019070 con 0x55a5b2597d40 2024-09-11T22:59:56.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.544+0000 7fb417015740 1 -- 172.21.15.152:0/279531419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b2597d40 msgr2=0x55a5b25962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:56.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.544+0000 7fb417015740 1 --2- 172.21.15.152:0/279531419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b2597d40 0x55a5b25962e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fb408007620 tx=0x7fb408007fc0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.544+0000 7fb417015740 1 -- 172.21.15.152:0/279531419 shutdown_connections 2024-09-11T22:59:56.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.544+0000 7fb417015740 1 --2- 172.21.15.152:0/279531419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b2597d40 0x55a5b25962e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.544+0000 7fb417015740 1 --2- 172.21.15.152:0/279531419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258d480 0x55a5b25afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.544+0000 7fb417015740 1 --2- 172.21.15.152:0/279531419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b258cb60 0x55a5b25af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.544+0000 7fb417015740 1 -- 172.21.15.152:0/279531419 >> 172.21.15.152:0/279531419 conn(0x55a5b25890f0 msgr2=0x55a5b2593c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:56.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.544+0000 7fb417015740 1 -- 172.21.15.152:0/279531419 shutdown_connections 2024-09-11T22:59:56.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.544+0000 7fb417015740 1 -- 172.21.15.152:0/279531419 wait complete. 2024-09-11T22:59:56.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.545+0000 7fb417015740 1 -- start start 2024-09-11T22:59:56.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.545+0000 7fb417015740 1 Processor -- start 2024-09-11T22:59:56.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.545+0000 7fb417015740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b258cb60 0x55a5b2591bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.545+0000 7fb417015740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258d480 0x55a5b2592100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.545+0000 7fb417015740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b2597d40 0x55a5b2592640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.545+0000 7fb417015740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101b0c20 con 0x55a5b258cb60 2024-09-11T22:59:56.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.545+0000 7fb417015740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101b0550 con 0x55a5b2597d40 2024-09-11T22:59:56.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.545+0000 7fb417015740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb4101ad770 con 0x55a5b258d480 2024-09-11T22:59:56.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.545+0000 7fb417015740 10 client.0 _ll_register_callbacks cb 0x7fb408013050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:56.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.545+0000 7fb40e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b258cb60 0x55a5b2591bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.545+0000 7fb40e59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b258cb60 0x55a5b2591bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57710/0 (socket says 172.21.15.152:57710) 2024-09-11T22:59:56.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.545+0000 7fb40e59c640 1 -- 172.21.15.152:0/202423193 learned_addr learned my addr 172.21.15.152:0/202423193 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:56.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.545+0000 7fb407fff640 1 --2- 172.21.15.152:0/202423193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258d480 0x55a5b2592100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.545+0000 7fb40ed9d640 1 --2- 172.21.15.152:0/202423193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b2597d40 0x55a5b2592640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.545+0000 7fb40e59c640 1 -- 172.21.15.152:0/202423193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258d480 msgr2=0x55a5b2592100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:56.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.546+0000 7fb40e59c640 1 --2- 172.21.15.152:0/202423193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258d480 0x55a5b2592100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.546+0000 7fb40e59c640 1 -- 172.21.15.152:0/202423193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b2597d40 msgr2=0x55a5b2592640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:56.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.546+0000 7fb40e59c640 1 --2- 172.21.15.152:0/202423193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b2597d40 0x55a5b2592640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.546+0000 7fb40e59c640 1 -- 172.21.15.152:0/202423193 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b26116f0 con 0x55a5b258cb60 2024-09-11T22:59:56.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.546+0000 7fb40e59c640 1 --2- 172.21.15.152:0/202423193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b258cb60 0x55a5b2591bc0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb400042000 tx=0x7fb400031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0a3ce95622d224 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:56.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.546+0000 7fb3f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:56.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.546+0000 7fb3f5ffb640 1 -- 172.21.15.152:0/202423193 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400052070 con 0x55a5b258cb60 2024-09-11T22:59:56.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.546+0000 7fb3f5ffb640 1 -- 172.21.15.152:0/202423193 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fb400003f70 con 0x55a5b258cb60 2024-09-11T22:59:56.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.546+0000 7fb417015740 1 -- 172.21.15.152:0/202423193 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb40801a040 con 0x55a5b258cb60 2024-09-11T22:59:56.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.546+0000 7fb3f5ffb640 1 -- 172.21.15.152:0/202423193 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb40005b050 con 0x55a5b258cb60 2024-09-11T22:59:56.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.546+0000 7fb417015740 10 client.17406 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:56.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.546+0000 7fb417015740 1 -- 172.21.15.152:0/202423193 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a5b2612a50 con 0x55a5b258cb60 2024-09-11T22:59:56.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.546+0000 7fb3f5ffb640 1 -- 172.21.15.152:0/202423193 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb40801a040 con 0x55a5b258cb60 2024-09-11T22:59:56.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.546+0000 7fb3f5ffb640 1 -- 172.21.15.152:0/202423193 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb400055020 con 0x55a5b258cb60 2024-09-11T22:59:56.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.547+0000 7fb417015740 10 client.17406 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:56.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.547+0000 7fb417015740 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:59:56.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.547+0000 7fb417015740 20 client.17406 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:56.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.547+0000 7fb417015740 1 --2- 172.21.15.152:0/202423193 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a5b25a7960 0x55a5b2614dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:56.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.547+0000 7fb417015740 4 client.17406 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:59:56.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.547+0000 7fb417015740 1 -- 172.21.15.152:0/202423193 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a5b24f9d00 con 0x55a5b25a7960 2024-09-11T22:59:56.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.547+0000 7fb407fff640 1 --2- 172.21.15.152:0/202423193 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a5b25a7960 0x55a5b2614dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:56.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.547+0000 7fb3f5ffb640 10 client.17406 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:56.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.547+0000 7fb407fff640 1 --2- 172.21.15.152:0/202423193 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a5b25a7960 0x55a5b2614dc0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=767b59b5de91d263 server_cookie=83cba092cc09b494 in_seq=0 out_seq=0 2024-09-11T22:59:56.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.548+0000 7fb3f5ffb640 1 -- 172.21.15.152:0/202423193 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a5b24f9d00 con 0x55a5b25a7960 2024-09-11T22:59:56.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.548+0000 7fb3f5ffb640 10 client.17406 handle_command_reply: tid=0 2024-09-11T22:59:56.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.548+0000 7fb417015740 1 client.17406 shutdown 2024-09-11T22:59:56.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.549+0000 7fb417015740 1 -- 172.21.15.152:0/202423193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b258cb60 msgr2=0x55a5b2591bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:56.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.549+0000 7fb417015740 1 --2- 172.21.15.152:0/202423193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b258cb60 0x55a5b2591bc0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fb400042000 tx=0x7fb400031660 comp rx=0 tx=0).stop 2024-09-11T22:59:56.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.549+0000 7fb417015740 1 -- 172.21.15.152:0/202423193 shutdown_connections 2024-09-11T22:59:56.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.549+0000 7fb417015740 1 --2- 172.21.15.152:0/202423193 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55a5b25a7960 0x55a5b2614dc0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.549+0000 7fb3f5ffb640 0 client.17406 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:56.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.549+0000 7fb417015740 1 --2- 172.21.15.152:0/202423193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5b2597d40 0x55a5b2592640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.549+0000 7fb417015740 1 --2- 172.21.15.152:0/202423193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a5b258d480 0x55a5b2592100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.549+0000 7fb417015740 1 --2- 172.21.15.152:0/202423193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a5b258cb60 0x55a5b2591bc0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.549+0000 7fb417015740 1 -- 172.21.15.152:0/202423193 >> 172.21.15.152:0/202423193 conn(0x55a5b25890f0 msgr2=0x55a5b2594b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:56.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.549+0000 7fb417015740 1 -- 172.21.15.152:0/202423193 shutdown_connections 2024-09-11T22:59:56.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.549+0000 7fb417015740 1 -- 172.21.15.152:0/202423193 wait complete. 2024-09-11T22:59:56.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.549+0000 7fb417015740 20 client.17406 trim_cache size 0 max 16384 2024-09-11T22:59:56.558 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:59:56.558 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T22:59:56.558 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "d252c8ef-c9c0-4267-bc31-f19f907e6e88", 2024-09-11T22:59:56.558 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T22:59:56.558 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:59:56.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.549+0000 7fb414c3c640 1 -- 172.21.15.152:0/3940885914 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb3fc0a2cf0 msgr2=0x7fb3fc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:56.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.549+0000 7fb414c3c640 1 --2- 172.21.15.152:0/3940885914 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb3fc0a2cf0 0x7fb3fc0a51b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb408007310 tx=0x7fb408004ab0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.549+0000 7fb414c3c640 1 -- 172.21.15.152:0/3940885914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb410100e10 msgr2=0x7fb41010ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:56.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.550+0000 7fb414c3c640 1 --2- 172.21.15.152:0/3940885914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb410100e10 0x7fb41010ed70 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800c920 tx=0x7fb3f800cde0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.550+0000 7fb414c3c640 1 -- 172.21.15.152:0/3940885914 shutdown_connections 2024-09-11T22:59:56.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.550+0000 7fb414c3c640 1 --2- 172.21.15.152:0/3940885914 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb3fc0a2cf0 0x7fb3fc0a51b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.550+0000 7fb414c3c640 1 --2- 172.21.15.152:0/3940885914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb410105590 0x7fb41010f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.550+0000 7fb414c3c640 1 --2- 172.21.15.152:0/3940885914 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb410100e10 0x7fb41010ed70 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.550+0000 7fb414c3c640 1 --2- 172.21.15.152:0/3940885914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb410100460 0x7fb41010e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:56.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.550+0000 7fb414c3c640 1 -- 172.21.15.152:0/3940885914 >> 172.21.15.152:0/3940885914 conn(0x7fb4100fbee0 msgr2=0x7fb4100fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:56.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.550+0000 7fb414c3c640 1 -- 172.21.15.152:0/3940885914 shutdown_connections 2024-09-11T22:59:56.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:56.550+0000 7fb414c3c640 1 -- 172.21.15.152:0/3940885914 wait complete. 2024-09-11T22:59:56.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:56 smithi191 ceph-mon[35211]: pgmap v214: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:59:56.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:56 smithi204 ceph-mon[35454]: pgmap v214: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:59:56.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:56 smithi152 ceph-mon[31181]: pgmap v214: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T22:59:57.466 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:59:57.712 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:57 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T22:59:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:57 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:59:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:57 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T22:59:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:57 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T22:59:57.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:57 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T22:59:57.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:57 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:59:57.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:57 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T22:59:57.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:57 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T22:59:57.833 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:57 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T22:59:57.834 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:57 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:59:57.834 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:57 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T22:59:57.834 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:57 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T22:59:58.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.259+0000 7fcac48d3640 1 -- 172.21.15.152:0/3293425813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcac01036c0 msgr2=0x7fcac010bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:58.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.259+0000 7fcac48d3640 1 --2- 172.21.15.152:0/3293425813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcac01036c0 0x7fcac010bbb0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009900 tx=0x7fcab402a480 comp rx=0 tx=0).stop 2024-09-11T22:59:58.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.259+0000 7fcac48d3640 1 -- 172.21.15.152:0/3293425813 shutdown_connections 2024-09-11T22:59:58.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.259+0000 7fcac48d3640 1 --2- 172.21.15.152:0/3293425813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac010c180 0x7fcac010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.259+0000 7fcac48d3640 1 --2- 172.21.15.152:0/3293425813 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcac01036c0 0x7fcac010bbb0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.259+0000 7fcac48d3640 1 --2- 172.21.15.152:0/3293425813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcac0102da0 0x7fcac0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.259+0000 7fcac48d3640 1 -- 172.21.15.152:0/3293425813 >> 172.21.15.152:0/3293425813 conn(0x7fcac00fc7a0 msgr2=0x7fcac00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:58.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.259+0000 7fcac48d3640 1 -- 172.21.15.152:0/3293425813 shutdown_connections 2024-09-11T22:59:58.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.259+0000 7fcac48d3640 1 -- 172.21.15.152:0/3293425813 wait complete. 2024-09-11T22:59:58.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.260+0000 7fcac48d3640 1 Processor -- start 2024-09-11T22:59:58.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.260+0000 7fcac48d3640 1 -- start start 2024-09-11T22:59:58.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.260+0000 7fcac48d3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcac0102da0 0x7fcac01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.276 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.260+0000 7fcac48d3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcac01036c0 0x7fcac0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.260+0000 7fcac48d3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac010c180 0x7fcac019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.260+0000 7fcac48d3640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0111f60 con 0x7fcac01036c0 2024-09-11T22:59:58.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.260+0000 7fcac48d3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcac0111de0 con 0x7fcac010c180 2024-09-11T22:59:58.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.260+0000 7fcac48d3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01120e0 con 0x7fcac0102da0 2024-09-11T22:59:58.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.261+0000 7fcabdd9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcac01036c0 0x7fcac0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.261+0000 7fcabdd9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcac01036c0 0x7fcac0198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57742/0 (socket says 172.21.15.152:57742) 2024-09-11T22:59:58.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.261+0000 7fcabdd9b640 1 -- 172.21.15.152:0/2675003791 learned_addr learned my addr 172.21.15.152:0/2675003791 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:58.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.261+0000 7fcabed9d640 1 --2- 172.21.15.152:0/2675003791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac010c180 0x7fcac019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.261+0000 7fcabe59c640 1 --2- 172.21.15.152:0/2675003791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcac0102da0 0x7fcac01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.261+0000 7fcabdd9b640 1 -- 172.21.15.152:0/2675003791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcac0102da0 msgr2=0x7fcac01983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:58.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.261+0000 7fcabdd9b640 1 --2- 172.21.15.152:0/2675003791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcac0102da0 0x7fcac01983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.261+0000 7fcabdd9b640 1 -- 172.21.15.152:0/2675003791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac010c180 msgr2=0x7fcac019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:58.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.261+0000 7fcabdd9b640 1 --2- 172.21.15.152:0/2675003791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac010c180 0x7fcac019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.261+0000 7fcabdd9b640 1 -- 172.21.15.152:0/2675003791 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac019d370 con 0x7fcac01036c0 2024-09-11T22:59:58.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.261+0000 7fcabdd9b640 1 --2- 172.21.15.152:0/2675003791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcac01036c0 0x7fcac0198900 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fcab4005ab0 tx=0x7fcab40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:58.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.261+0000 7fcaaf7fe640 1 -- 172.21.15.152:0/2675003791 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab403f030 con 0x7fcac01036c0 2024-09-11T22:59:58.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.261+0000 7fcaaf7fe640 1 -- 172.21.15.152:0/2675003791 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab4003ce0 con 0x7fcac01036c0 2024-09-11T22:59:58.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.261+0000 7fcac48d3640 1 -- 172.21.15.152:0/2675003791 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac019d600 con 0x7fcac01036c0 2024-09-11T22:59:58.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.261+0000 7fcaaf7fe640 1 -- 172.21.15.152:0/2675003791 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4035070 con 0x7fcac01036c0 2024-09-11T22:59:58.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.261+0000 7fcac48d3640 1 -- 172.21.15.152:0/2675003791 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac01a4f40 con 0x7fcac01036c0 2024-09-11T22:59:58.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.264+0000 7fcaaf7fe640 1 -- 172.21.15.152:0/2675003791 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fcab4042030 con 0x7fcac01036c0 2024-09-11T22:59:58.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.264+0000 7fcac6cac740 1 Processor -- start 2024-09-11T22:59:58.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcac6cac740 1 -- start start 2024-09-11T22:59:58.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcac6cac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 0x560e9b82b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcac6cac740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b82bca0 0x560e9b83a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcac6cac740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b83aa90 0x560e9b83ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcac6cac740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560e9b7912e0 con 0x560e9b82bca0 2024-09-11T22:59:58.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcac6cac740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560e9b738ce0 con 0x560e9b83aa90 2024-09-11T22:59:58.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcac6cac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560e9b78d2a0 con 0x560e9b82b380 2024-09-11T22:59:58.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcabe59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b82bca0 0x560e9b83a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcabe59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b82bca0 0x560e9b83a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57748/0 (socket says 172.21.15.152:57748) 2024-09-11T22:59:58.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcabe59c640 1 -- 172.21.15.152:0/4199672161 learned_addr learned my addr 172.21.15.152:0/4199672161 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:58.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcabed9d640 1 --2- 172.21.15.152:0/4199672161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 0x560e9b82b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcaaf7fe640 1 --2- 172.21.15.152:0/2675003791 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcaa80a2cf0 0x7fcaa80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcabed9d640 1 --2- 172.21.15.152:0/2675003791 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcaa80a2cf0 0x7fcaa80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcaaf7fe640 1 -- 172.21.15.152:0/2675003791 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fcab40efcf0 con 0x7fcac01036c0 2024-09-11T22:59:58.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcabdd9b640 1 --2- 172.21.15.152:0/4199672161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b83aa90 0x560e9b83ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcabe59c640 1 -- 172.21.15.152:0/4199672161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 msgr2=0x560e9b82b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:58.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcabe59c640 1 --2- 172.21.15.152:0/4199672161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 0x560e9b82b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcabe59c640 1 -- 172.21.15.152:0/4199672161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b83aa90 msgr2=0x560e9b83ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:58.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcabe59c640 1 --2- 172.21.15.152:0/4199672161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b83aa90 0x560e9b83ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcabe59c640 1 -- 172.21.15.152:0/4199672161 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e9b71d440 con 0x560e9b82bca0 2024-09-11T22:59:58.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcabdd9b640 1 --2- 172.21.15.152:0/4199672161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b83aa90 0x560e9b83ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:59:58.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.265+0000 7fcabed9d640 1 --2- 172.21.15.152:0/4199672161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 0x560e9b82b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:59:58.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.266+0000 7fcabe59c640 1 --2- 172.21.15.152:0/4199672161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b82bca0 0x560e9b83a550 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fcab000cc60 tx=0x7fcab000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37ccdfb1f7eb709 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:58.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.266+0000 7fcabed9d640 1 --2- 172.21.15.152:0/2675003791 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcaa80a2cf0 0x7fcaa80a51b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcab800a2c0 tx=0x7fcab8004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:58.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.266+0000 7fcaacff9640 1 -- 172.21.15.152:0/4199672161 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0014070 con 0x560e9b82bca0 2024-09-11T22:59:58.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.266+0000 7fcaacff9640 1 -- 172.21.15.152:0/4199672161 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab00047a0 con 0x560e9b82bca0 2024-09-11T22:59:58.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.266+0000 7fcaacff9640 1 -- 172.21.15.152:0/4199672161 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0010760 con 0x560e9b82bca0 2024-09-11T22:59:58.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 -- 172.21.15.152:0/4199672161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b82bca0 msgr2=0x560e9b83a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:58.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 --2- 172.21.15.152:0/4199672161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b82bca0 0x560e9b83a550 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fcab000cc60 tx=0x7fcab000e720 comp rx=0 tx=0).stop 2024-09-11T22:59:58.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 -- 172.21.15.152:0/4199672161 shutdown_connections 2024-09-11T22:59:58.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 --2- 172.21.15.152:0/4199672161 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b83aa90 0x560e9b83ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 --2- 172.21.15.152:0/4199672161 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b82bca0 0x560e9b83a550 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 --2- 172.21.15.152:0/4199672161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 0x560e9b82b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 -- 172.21.15.152:0/4199672161 >> 172.21.15.152:0/4199672161 conn(0x560e9b827a30 msgr2=0x560e9b827e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:58.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 -- 172.21.15.152:0/4199672161 shutdown_connections 2024-09-11T22:59:58.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 -- 172.21.15.152:0/4199672161 wait complete. 2024-09-11T22:59:58.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 -- start start 2024-09-11T22:59:58.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 Processor -- start 2024-09-11T22:59:58.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 0x560e9b83a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b82bca0 0x560e9b8376d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b83aa90 0x560e9b837c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560e9b83d660 con 0x560e9b83aa90 2024-09-11T22:59:58.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560e9b74c050 con 0x560e9b82bca0 2024-09-11T22:59:58.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560e9b83d7e0 con 0x560e9b82b380 2024-09-11T22:59:58.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.267+0000 7fcac6cac740 10 client.0 _ll_register_callbacks cb 0x560e9b78c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:58.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcabed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b83aa90 0x560e9b837c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcabed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b83aa90 0x560e9b837c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57764/0 (socket says 172.21.15.152:57764) 2024-09-11T22:59:58.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcabed9d640 1 -- 172.21.15.152:0/2751085220 learned_addr learned my addr 172.21.15.152:0/2751085220 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:58.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcabdd9b640 1 --2- 172.21.15.152:0/2751085220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b82bca0 0x560e9b8376d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcabe59c640 1 --2- 172.21.15.152:0/2751085220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 0x560e9b83a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcabed9d640 1 -- 172.21.15.152:0/2751085220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 msgr2=0x560e9b83a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:58.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcabed9d640 1 --2- 172.21.15.152:0/2751085220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 0x560e9b83a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcabed9d640 1 -- 172.21.15.152:0/2751085220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b82bca0 msgr2=0x560e9b8376d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:58.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcabed9d640 1 --2- 172.21.15.152:0/2751085220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b82bca0 0x560e9b8376d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcabed9d640 1 -- 172.21.15.152:0/2751085220 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e9b5023c0 con 0x560e9b83aa90 2024-09-11T22:59:58.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcabe59c640 1 --2- 172.21.15.152:0/2751085220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 0x560e9b83a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:59:58.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcabed9d640 1 --2- 172.21.15.152:0/2751085220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b83aa90 0x560e9b837c50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fcab8011f10 tx=0x7fcab8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752f4519100784ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:58.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcaa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:58.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcaa5ffb640 1 -- 172.21.15.152:0/2751085220 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8025070 con 0x560e9b83aa90 2024-09-11T22:59:58.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcaa5ffb640 1 -- 172.21.15.152:0/2751085220 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fcab8021610 con 0x560e9b83aa90 2024-09-11T22:59:58.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcac6cac740 1 -- 172.21.15.152:0/2751085220 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab00047a0 con 0x560e9b83aa90 2024-09-11T22:59:58.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcaa5ffb640 1 -- 172.21.15.152:0/2751085220 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcab8031050 con 0x560e9b83aa90 2024-09-11T22:59:58.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcac6cac740 10 client.17418 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:58.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcac6cac740 1 -- 172.21.15.152:0/2751085220 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560e9b849060 con 0x560e9b83aa90 2024-09-11T22:59:58.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.268+0000 7fcaa5ffb640 1 -- 172.21.15.152:0/2751085220 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab00047a0 con 0x560e9b83aa90 2024-09-11T22:59:58.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.269+0000 7fcaa5ffb640 1 -- 172.21.15.152:0/2751085220 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcab801d070 con 0x560e9b83aa90 2024-09-11T22:59:58.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.269+0000 7fcac6cac740 10 client.17418 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:58.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.269+0000 7fcac6cac740 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:59:58.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.269+0000 7fcac6cac740 20 client.17418 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:58.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.269+0000 7fcac6cac740 1 --2- 172.21.15.152:0/2751085220 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560e9b84a470 0x560e9b84c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.269+0000 7fcac6cac740 4 client.17418 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:59:58.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.269+0000 7fcac6cac740 1 -- 172.21.15.152:0/2751085220 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e9b79e850 con 0x560e9b84a470 2024-09-11T22:59:58.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.269+0000 7fcabe59c640 1 --2- 172.21.15.152:0/2751085220 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560e9b84a470 0x560e9b84c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.270+0000 7fcaa5ffb640 10 client.17418 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:58.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.270+0000 7fcabe59c640 1 --2- 172.21.15.152:0/2751085220 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560e9b84a470 0x560e9b84c8a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d727e8d79db4cd9 server_cookie=b64fbeaa81a124fb in_seq=0 out_seq=0 2024-09-11T22:59:58.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.270+0000 7fcaa5ffb640 1 -- 172.21.15.152:0/2751085220 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e9b79e850 con 0x560e9b84a470 2024-09-11T22:59:58.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.270+0000 7fcaa5ffb640 10 client.17418 handle_command_reply: tid=0 2024-09-11T22:59:58.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.270+0000 7fcac6cac740 1 client.17418 shutdown 2024-09-11T22:59:58.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.270+0000 7fcac6cac740 1 -- 172.21.15.152:0/2751085220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b83aa90 msgr2=0x560e9b837c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:58.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.270+0000 7fcac6cac740 1 --2- 172.21.15.152:0/2751085220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b83aa90 0x560e9b837c50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fcab8011f10 tx=0x7fcab8004a40 comp rx=0 tx=0).stop 2024-09-11T22:59:58.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.270+0000 7fcac6cac740 1 -- 172.21.15.152:0/2751085220 shutdown_connections 2024-09-11T22:59:58.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.270+0000 7fcac6cac740 1 --2- 172.21.15.152:0/2751085220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b83aa90 0x560e9b837c50 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.270+0000 7fcac6cac740 1 --2- 172.21.15.152:0/2751085220 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560e9b84a470 0x560e9b84c8a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.270+0000 7fcaa5ffb640 0 client.17418 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:58.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.270+0000 7fcac6cac740 1 --2- 172.21.15.152:0/2751085220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b82bca0 0x560e9b8376d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.271+0000 7fcac6cac740 1 --2- 172.21.15.152:0/2751085220 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 0x560e9b83a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.271+0000 7fcac6cac740 1 -- 172.21.15.152:0/2751085220 >> 172.21.15.152:0/2751085220 conn(0x560e9b827a30 msgr2=0x560e9b82d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:58.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.271+0000 7fcac6cac740 1 -- 172.21.15.152:0/2751085220 shutdown_connections 2024-09-11T22:59:58.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.271+0000 7fcac6cac740 1 -- 172.21.15.152:0/2751085220 wait complete. 2024-09-11T22:59:58.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.271+0000 7fcac6cac740 20 client.17418 trim_cache size 0 max 16384 2024-09-11T22:59:58.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcac6cac740 1 Processor -- start 2024-09-11T22:59:58.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcac6cac740 1 -- start start 2024-09-11T22:59:58.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcac6cac740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b82b380 0x560e9b846900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcac6cac740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b82bca0 0x560e9b832c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcac6cac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b83aa90 0x560e9b852170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcac6cac740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01ab520 con 0x560e9b82bca0 2024-09-11T22:59:58.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcac6cac740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01ad060 con 0x560e9b82b380 2024-09-11T22:59:58.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcac6cac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01abbf0 con 0x560e9b83aa90 2024-09-11T22:59:58.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcabdd9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b82bca0 0x560e9b832c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcabdd9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b82bca0 0x560e9b832c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57770/0 (socket says 172.21.15.152:57770) 2024-09-11T22:59:58.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcabdd9b640 1 -- 172.21.15.152:0/999909968 learned_addr learned my addr 172.21.15.152:0/999909968 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:58.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcabe59c640 1 --2- 172.21.15.152:0/999909968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b82b380 0x560e9b846900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcabed9d640 1 --2- 172.21.15.152:0/999909968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b83aa90 0x560e9b852170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcabdd9b640 1 -- 172.21.15.152:0/999909968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b83aa90 msgr2=0x560e9b852170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:58.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcabdd9b640 1 --2- 172.21.15.152:0/999909968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b83aa90 0x560e9b852170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcabdd9b640 1 -- 172.21.15.152:0/999909968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b82b380 msgr2=0x560e9b846900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:58.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcabdd9b640 1 --2- 172.21.15.152:0/999909968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b82b380 0x560e9b846900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcabdd9b640 1 -- 172.21.15.152:0/999909968 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e9b83a790 con 0x560e9b82bca0 2024-09-11T22:59:58.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.288+0000 7fcabe59c640 1 --2- 172.21.15.152:0/999909968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b82b380 0x560e9b846900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:59:58.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.289+0000 7fcabdd9b640 1 --2- 172.21.15.152:0/999909968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b82bca0 0x560e9b832c00 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fcab4000c00 tx=0x7fcab402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f695dda9f90639 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:58.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.289+0000 7fcaa7fff640 1 -- 172.21.15.152:0/999909968 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4049040 con 0x560e9b82bca0 2024-09-11T22:59:58.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.289+0000 7fcaa7fff640 1 -- 172.21.15.152:0/999909968 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab40c0610 con 0x560e9b82bca0 2024-09-11T22:59:58.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.289+0000 7fcaa7fff640 1 -- 172.21.15.152:0/999909968 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab40ed710 con 0x560e9b82bca0 2024-09-11T22:59:58.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.289+0000 7fcac6cac740 1 -- 172.21.15.152:0/999909968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b82bca0 msgr2=0x560e9b832c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:58.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.289+0000 7fcac6cac740 1 --2- 172.21.15.152:0/999909968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b82bca0 0x560e9b832c00 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fcab4000c00 tx=0x7fcab402a480 comp rx=0 tx=0).stop 2024-09-11T22:59:58.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.289+0000 7fcac6cac740 1 -- 172.21.15.152:0/999909968 shutdown_connections 2024-09-11T22:59:58.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcac6cac740 1 --2- 172.21.15.152:0/999909968 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b83aa90 0x560e9b852170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcac6cac740 1 --2- 172.21.15.152:0/999909968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b82bca0 0x560e9b832c00 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcac6cac740 1 --2- 172.21.15.152:0/999909968 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b82b380 0x560e9b846900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcac6cac740 1 -- 172.21.15.152:0/999909968 >> 172.21.15.152:0/999909968 conn(0x560e9b827a30 msgr2=0x560e9b8324f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcac6cac740 1 -- 172.21.15.152:0/999909968 shutdown_connections 2024-09-11T22:59:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcac6cac740 1 -- 172.21.15.152:0/999909968 wait complete. 2024-09-11T22:59:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcac6cac740 1 -- start start 2024-09-11T22:59:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcac6cac740 1 Processor -- start 2024-09-11T22:59:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcac6cac740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 0x560e9b843330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcac6cac740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b82bca0 0x560e9b834560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcac6cac740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b83aa90 0x560e9b834ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcac6cac740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01ac990 con 0x560e9b83aa90 2024-09-11T22:59:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcac6cac740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01ac2c0 con 0x560e9b82bca0 2024-09-11T22:59:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcac6cac740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcac01a9520 con 0x560e9b82b380 2024-09-11T22:59:58.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcac6cac740 10 client.0 _ll_register_callbacks cb 0x7fcab8013050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:59:58.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcabed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b83aa90 0x560e9b834ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.290+0000 7fcabed9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b83aa90 0x560e9b834ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57780/0 (socket says 172.21.15.152:57780) 2024-09-11T22:59:58.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcabed9d640 1 -- 172.21.15.152:0/2630682120 learned_addr learned my addr 172.21.15.152:0/2630682120 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:58.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcabe59c640 1 --2- 172.21.15.152:0/2630682120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 0x560e9b843330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcabdd9b640 1 --2- 172.21.15.152:0/2630682120 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b82bca0 0x560e9b834560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcabed9d640 1 -- 172.21.15.152:0/2630682120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 msgr2=0x560e9b843330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:58.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcabed9d640 1 --2- 172.21.15.152:0/2630682120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 0x560e9b843330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcabed9d640 1 -- 172.21.15.152:0/2630682120 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b82bca0 msgr2=0x560e9b834560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:59:58.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcabed9d640 1 --2- 172.21.15.152:0/2630682120 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b82bca0 0x560e9b834560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcabed9d640 1 -- 172.21.15.152:0/2630682120 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e9b8b4140 con 0x560e9b83aa90 2024-09-11T22:59:58.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcabe59c640 1 --2- 172.21.15.152:0/2630682120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 0x560e9b843330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:59:58.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcabed9d640 1 --2- 172.21.15.152:0/2630682120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b83aa90 0x560e9b834ca0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fcab8011930 tx=0x7fcab800aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4815061a9834a3b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:58.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcaa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T22:59:58.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcaa5ffb640 1 -- 172.21.15.152:0/2630682120 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8039070 con 0x560e9b83aa90 2024-09-11T22:59:58.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcaa5ffb640 1 -- 172.21.15.152:0/2630682120 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fcab800fa70 con 0x560e9b83aa90 2024-09-11T22:59:58.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcac6cac740 1 -- 172.21.15.152:0/2630682120 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab40c0610 con 0x560e9b83aa90 2024-09-11T22:59:58.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcaa5ffb640 1 -- 172.21.15.152:0/2630682120 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcab8042050 con 0x560e9b83aa90 2024-09-11T22:59:58.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcac6cac740 10 client.17424 fetch_fsmap learned FSMap version 28 2024-09-11T22:59:58.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.291+0000 7fcac6cac740 1 -- 172.21.15.152:0/2630682120 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560e9b8b5290 con 0x560e9b83aa90 2024-09-11T22:59:58.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.292+0000 7fcaa5ffb640 1 -- 172.21.15.152:0/2630682120 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab40c0610 con 0x560e9b83aa90 2024-09-11T22:59:58.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.292+0000 7fcaa5ffb640 1 -- 172.21.15.152:0/2630682120 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcab803c020 con 0x560e9b83aa90 2024-09-11T22:59:58.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.292+0000 7fcac6cac740 10 client.17424 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:59:58.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.292+0000 7fcac6cac740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:59:58.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.292+0000 7fcac6cac740 20 client.17424 populate_metadata read hostname 'smithi152' 2024-09-11T22:59:58.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.292+0000 7fcac6cac740 1 --2- 172.21.15.152:0/2630682120 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560e9b84a470 0x560e9b8b7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:58.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.292+0000 7fcac6cac740 4 client.17424 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:59:58.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.292+0000 7fcac6cac740 1 -- 172.21.15.152:0/2630682120 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e9b798d60 con 0x560e9b84a470 2024-09-11T22:59:58.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.292+0000 7fcabe59c640 1 --2- 172.21.15.152:0/2630682120 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560e9b84a470 0x560e9b8b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:58.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.292+0000 7fcaa5ffb640 10 client.17424 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:58.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.292+0000 7fcabe59c640 1 --2- 172.21.15.152:0/2630682120 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560e9b84a470 0x560e9b8b7950 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=837d8ef4c274c298 server_cookie=6634cb1e0f34c1c1 in_seq=0 out_seq=0 2024-09-11T22:59:58.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcaa5ffb640 1 -- 172.21.15.152:0/2630682120 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560e9b798d60 con 0x560e9b84a470 2024-09-11T22:59:58.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcaa5ffb640 10 client.17424 handle_command_reply: tid=0 2024-09-11T22:59:58.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac6cac740 1 client.17424 shutdown 2024-09-11T22:59:58.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac6cac740 1 -- 172.21.15.152:0/2630682120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b83aa90 msgr2=0x560e9b834ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:58.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac6cac740 1 --2- 172.21.15.152:0/2630682120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b83aa90 0x560e9b834ca0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fcab8011930 tx=0x7fcab800aed0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac6cac740 1 -- 172.21.15.152:0/2630682120 shutdown_connections 2024-09-11T22:59:58.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac6cac740 1 --2- 172.21.15.152:0/2630682120 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e9b83aa90 0x560e9b834ca0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac6cac740 1 --2- 172.21.15.152:0/2630682120 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x560e9b84a470 0x560e9b8b7950 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac6cac740 1 --2- 172.21.15.152:0/2630682120 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e9b82bca0 0x560e9b834560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac6cac740 1 --2- 172.21.15.152:0/2630682120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e9b82b380 0x560e9b843330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcaa5ffb640 0 client.17424 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T22:59:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac6cac740 1 -- 172.21.15.152:0/2630682120 >> 172.21.15.152:0/2630682120 conn(0x560e9b827a30 msgr2=0x560e9b832d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac6cac740 1 -- 172.21.15.152:0/2630682120 shutdown_connections 2024-09-11T22:59:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac6cac740 1 -- 172.21.15.152:0/2630682120 wait complete. 2024-09-11T22:59:58.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac6cac740 20 client.17424 trim_cache size 0 max 16384 2024-09-11T22:59:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T22:59:58.300 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T22:59:58.300 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T22:59:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T22:59:58.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac48d3640 1 -- 172.21.15.152:0/2675003791 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcaa80a2cf0 msgr2=0x7fcaa80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:58.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac48d3640 1 --2- 172.21.15.152:0/2675003791 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcaa80a2cf0 0x7fcaa80a51b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcab800a2c0 tx=0x7fcab8004ab0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac48d3640 1 -- 172.21.15.152:0/2675003791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcac01036c0 msgr2=0x7fcac0198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:58.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac48d3640 1 --2- 172.21.15.152:0/2675003791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcac01036c0 0x7fcac0198900 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fcab4005ab0 tx=0x7fcab40042f0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac48d3640 1 -- 172.21.15.152:0/2675003791 shutdown_connections 2024-09-11T22:59:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac48d3640 1 --2- 172.21.15.152:0/2675003791 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcaa80a2cf0 0x7fcaa80a51b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac48d3640 1 --2- 172.21.15.152:0/2675003791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac010c180 0x7fcac019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac48d3640 1 --2- 172.21.15.152:0/2675003791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcac01036c0 0x7fcac0198900 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac48d3640 1 --2- 172.21.15.152:0/2675003791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcac0102da0 0x7fcac01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.293+0000 7fcac48d3640 1 -- 172.21.15.152:0/2675003791 >> 172.21.15.152:0/2675003791 conn(0x7fcac00fc7a0 msgr2=0x7fcac00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.294+0000 7fcac48d3640 1 -- 172.21.15.152:0/2675003791 shutdown_connections 2024-09-11T22:59:58.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:58.294+0000 7fcac48d3640 1 -- 172.21.15.152:0/2675003791 wait complete. 2024-09-11T22:59:58.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:58 smithi152 ceph-mon[31181]: pgmap v215: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:58.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 22:59:58 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T22:59:58.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:58 smithi191 ceph-mon[35211]: pgmap v215: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:58.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 22:59:58 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T22:59:58.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:58 smithi204 ceph-mon[35454]: pgmap v215: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:59:58.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 22:59:58 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T22:59:59.093 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:59:59.094 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:59:59.345 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:59:59.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.866+0000 7f3937ef7640 1 -- 172.21.15.152:0/1348851077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39301002f0 msgr2=0x7f39301006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:59.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.866+0000 7f3937ef7640 1 --2- 172.21.15.152:0/1348851077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39301002f0 0x7f39301006d0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f3924009930 tx=0x7f392402a420 comp rx=0 tx=0).stop 2024-09-11T22:59:59.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.866+0000 7f3937ef7640 1 -- 172.21.15.152:0/1348851077 shutdown_connections 2024-09-11T22:59:59.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.866+0000 7f3937ef7640 1 --2- 172.21.15.152:0/1348851077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3930106f50 0x7f393010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:59.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.866+0000 7f3937ef7640 1 --2- 172.21.15.152:0/1348851077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3930100ca0 0x7f3930106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:59.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.866+0000 7f3937ef7640 1 --2- 172.21.15.152:0/1348851077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39301002f0 0x7f39301006d0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:59.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.866+0000 7f3937ef7640 1 -- 172.21.15.152:0/1348851077 >> 172.21.15.152:0/1348851077 conn(0x7f3930077210 msgr2=0x7f39300fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:59:59.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.866+0000 7f3937ef7640 1 -- 172.21.15.152:0/1348851077 shutdown_connections 2024-09-11T22:59:59.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.866+0000 7f3937ef7640 1 -- 172.21.15.152:0/1348851077 wait complete. 2024-09-11T22:59:59.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3937ef7640 1 Processor -- start 2024-09-11T22:59:59.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3937ef7640 1 -- start start 2024-09-11T22:59:59.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3937ef7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39301002f0 0x7f3930198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:59.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3937ef7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3930100ca0 0x7f3930198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:59.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3937ef7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3930106f50 0x7f393019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:59.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3937ef7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3930111fa0 con 0x7f3930106f50 2024-09-11T22:59:59.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3937ef7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3930111e20 con 0x7f39301002f0 2024-09-11T22:59:59.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3937ef7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3930112120 con 0x7f3930100ca0 2024-09-11T22:59:59.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3936494640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3930106f50 0x7f393019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:59.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3936494640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3930106f50 0x7f393019cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56404/0 (socket says 172.21.15.152:56404) 2024-09-11T22:59:59.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3936494640 1 -- 172.21.15.152:0/2353793781 learned_addr learned my addr 172.21.15.152:0/2353793781 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T22:59:59.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3935492640 1 --2- 172.21.15.152:0/2353793781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3930100ca0 0x7f3930198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:59.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3935c93640 1 --2- 172.21.15.152:0/2353793781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39301002f0 0x7f3930198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:59.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3936494640 1 -- 172.21.15.152:0/2353793781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3930100ca0 msgr2=0x7f3930198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:59.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3936494640 1 --2- 172.21.15.152:0/2353793781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3930100ca0 0x7f3930198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:59.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3936494640 1 -- 172.21.15.152:0/2353793781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39301002f0 msgr2=0x7f3930198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:59:59.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3936494640 1 --2- 172.21.15.152:0/2353793781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39301002f0 0x7f3930198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:59:59.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3936494640 1 -- 172.21.15.152:0/2353793781 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393019d4d0 con 0x7f3930106f50 2024-09-11T22:59:59.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.867+0000 7f3936494640 1 --2- 172.21.15.152:0/2353793781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3930106f50 0x7f393019cdf0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f392800cce0 tx=0x7f392800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:59.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.868+0000 7f3922ffd640 1 -- 172.21.15.152:0/2353793781 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392800f450 con 0x7f3930106f50 2024-09-11T22:59:59.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.868+0000 7f3922ffd640 1 -- 172.21.15.152:0/2353793781 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f392800fad0 con 0x7f3930106f50 2024-09-11T22:59:59.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.868+0000 7f3937ef7640 1 -- 172.21.15.152:0/2353793781 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393019d7c0 con 0x7f3930106f50 2024-09-11T22:59:59.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.868+0000 7f3922ffd640 1 -- 172.21.15.152:0/2353793781 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928018c10 con 0x7f3930106f50 2024-09-11T22:59:59.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.868+0000 7f3937ef7640 1 -- 172.21.15.152:0/2353793781 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39301a50a0 con 0x7f3930106f50 2024-09-11T22:59:59.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.869+0000 7f3937ef7640 1 -- 172.21.15.152:0/2353793781 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39301006d0 con 0x7f3930106f50 2024-09-11T22:59:59.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.870+0000 7f3922ffd640 1 -- 172.21.15.152:0/2353793781 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f392800f5f0 con 0x7f3930106f50 2024-09-11T22:59:59.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.870+0000 7f3922ffd640 1 --2- 172.21.15.152:0/2353793781 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f391c0a2cf0 0x7f391c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:59:59.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.870+0000 7f3935c93640 1 --2- 172.21.15.152:0/2353793781 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f391c0a2cf0 0x7f391c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:59:59.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.870+0000 7f3922ffd640 1 -- 172.21.15.152:0/2353793781 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f39280cfa30 con 0x7f3930106f50 2024-09-11T22:59:59.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.871+0000 7f3935c93640 1 --2- 172.21.15.152:0/2353793781 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f391c0a2cf0 0x7f391c0a51b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3924000c00 tx=0x7f3924005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:59:59.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T22:59:59.874+0000 7f3922ffd640 1 -- 172.21.15.152:0/2353793781 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3928014030 con 0x7f3930106f50 2024-09-11T23:00:00.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:00.075+0000 7f3937ef7640 1 -- 172.21.15.152:0/2353793781 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3930109bb0 con 0x7f3930106f50 2024-09-11T23:00:00.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:00.076+0000 7f3922ffd640 1 -- 172.21.15.152:0/2353793781 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f392809c6a0 con 0x7f3930106f50 2024-09-11T23:00:00.077 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T23:00:00.077 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T23:00:00.077 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T23:00:00.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:00.078+0000 7f3937ef7640 1 -- 172.21.15.152:0/2353793781 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f391c0a2cf0 msgr2=0x7f391c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:00.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:00.078+0000 7f3937ef7640 1 --2- 172.21.15.152:0/2353793781 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f391c0a2cf0 0x7f391c0a51b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3924000c00 tx=0x7f3924005a20 comp rx=0 tx=0).stop 2024-09-11T23:00:00.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:00.078+0000 7f3937ef7640 1 -- 172.21.15.152:0/2353793781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3930106f50 msgr2=0x7f393019cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:00.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:00.078+0000 7f3937ef7640 1 --2- 172.21.15.152:0/2353793781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3930106f50 0x7f393019cdf0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f392800cce0 tx=0x7f392800a620 comp rx=0 tx=0).stop 2024-09-11T23:00:00.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:00.078+0000 7f3937ef7640 1 -- 172.21.15.152:0/2353793781 shutdown_connections 2024-09-11T23:00:00.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:00.078+0000 7f3937ef7640 1 --2- 172.21.15.152:0/2353793781 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f391c0a2cf0 0x7f391c0a51b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:00.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:00.078+0000 7f3937ef7640 1 --2- 172.21.15.152:0/2353793781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3930106f50 0x7f393019cdf0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:00.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:00.078+0000 7f3937ef7640 1 --2- 172.21.15.152:0/2353793781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3930100ca0 0x7f3930198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:00.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:00.078+0000 7f3937ef7640 1 --2- 172.21.15.152:0/2353793781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f39301002f0 0x7f3930198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:00.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:00.078+0000 7f3937ef7640 1 -- 172.21.15.152:0/2353793781 >> 172.21.15.152:0/2353793781 conn(0x7f3930077210 msgr2=0x7f39300fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:00.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:00.078+0000 7f3937ef7640 1 -- 172.21.15.152:0/2353793781 shutdown_connections 2024-09-11T23:00:00.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:00.078+0000 7f3937ef7640 1 -- 172.21.15.152:0/2353793781 wait complete. 2024-09-11T23:00:00.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:00 smithi152 ceph-mon[31181]: pgmap v216: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 12 op/s 2024-09-11T23:00:00.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:00 smithi152 ceph-mon[31181]: overall HEALTH_OK 2024-09-11T23:00:00.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:00 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2353793781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:00:00.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:00 smithi191 ceph-mon[35211]: pgmap v216: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 12 op/s 2024-09-11T23:00:00.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:00 smithi191 ceph-mon[35211]: overall HEALTH_OK 2024-09-11T23:00:00.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:00 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2353793781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:00:00.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:00 smithi204 ceph-mon[35454]: pgmap v216: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 12 op/s 2024-09-11T23:00:00.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:00 smithi204 ceph-mon[35454]: overall HEALTH_OK 2024-09-11T23:00:00.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:00 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2353793781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:00:00.958 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:00:01.214 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:01.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.732+0000 7f65040ca640 1 -- 172.21.15.152:0/1722899455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64fc103660 msgr2=0x7f64fc10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:01.763 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.732+0000 7f65040ca640 1 --2- 172.21.15.152:0/1722899455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64fc103660 0x7f64fc10bb50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009900 tx=0x7f64ec02a480 comp rx=0 tx=0).stop 2024-09-11T23:00:01.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.732+0000 7f65040ca640 1 -- 172.21.15.152:0/1722899455 shutdown_connections 2024-09-11T23:00:01.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.732+0000 7f65040ca640 1 --2- 172.21.15.152:0/1722899455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64fc10c170 0x7f64fc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.732+0000 7f65040ca640 1 --2- 172.21.15.152:0/1722899455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64fc103660 0x7f64fc10bb50 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.732+0000 7f65040ca640 1 --2- 172.21.15.152:0/1722899455 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64fc102d40 0x7f64fc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.732+0000 7f65040ca640 1 -- 172.21.15.152:0/1722899455 >> 172.21.15.152:0/1722899455 conn(0x7f64fc0fc7a0 msgr2=0x7f64fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:01.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.732+0000 7f65040ca640 1 -- 172.21.15.152:0/1722899455 shutdown_connections 2024-09-11T23:00:01.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.732+0000 7f65040ca640 1 -- 172.21.15.152:0/1722899455 wait complete. 2024-09-11T23:00:01.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.732+0000 7f65040ca640 1 Processor -- start 2024-09-11T23:00:01.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.732+0000 7f65040ca640 1 -- start start 2024-09-11T23:00:01.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f65040ca640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64fc102d40 0x7f64fc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f65040ca640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64fc103660 0x7f64fc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f65040ca640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64fc10c170 0x7f64fc19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f65040ca640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc111f70 con 0x7f64fc10c170 2024-09-11T23:00:01.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f65040ca640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc111df0 con 0x7f64fc103660 2024-09-11T23:00:01.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f65040ca640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1120f0 con 0x7f64fc102d40 2024-09-11T23:00:01.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f6502667640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64fc10c170 0x7f64fc19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f6502667640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64fc10c170 0x7f64fc19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56418/0 (socket says 172.21.15.152:56418) 2024-09-11T23:00:01.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f6502667640 1 -- 172.21.15.152:0/2205241393 learned_addr learned my addr 172.21.15.152:0/2205241393 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:01.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f6501665640 1 --2- 172.21.15.152:0/2205241393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64fc103660 0x7f64fc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f6501e66640 1 --2- 172.21.15.152:0/2205241393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64fc102d40 0x7f64fc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f6502667640 1 -- 172.21.15.152:0/2205241393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64fc102d40 msgr2=0x7f64fc198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:01.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f6502667640 1 --2- 172.21.15.152:0/2205241393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64fc102d40 0x7f64fc198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f6502667640 1 -- 172.21.15.152:0/2205241393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64fc103660 msgr2=0x7f64fc1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:01.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f6502667640 1 --2- 172.21.15.152:0/2205241393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64fc103660 0x7f64fc1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f6502667640 1 -- 172.21.15.152:0/2205241393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc19d420 con 0x7f64fc10c170 2024-09-11T23:00:01.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f6502667640 1 --2- 172.21.15.152:0/2205241393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64fc10c170 0x7f64fc19cd40 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f64f400cad0 tx=0x7f64f400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:01.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f64f2ffd640 1 -- 172.21.15.152:0/2205241393 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f400ae20 con 0x7f64fc10c170 2024-09-11T23:00:01.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f65040ca640 1 -- 172.21.15.152:0/2205241393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc19d710 con 0x7f64fc10c170 2024-09-11T23:00:01.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f64f2ffd640 1 -- 172.21.15.152:0/2205241393 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f4005020 con 0x7f64fc10c170 2024-09-11T23:00:01.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f64f2ffd640 1 -- 172.21.15.152:0/2205241393 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f400f6f0 con 0x7f64fc10c170 2024-09-11T23:00:01.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.733+0000 7f65040ca640 1 -- 172.21.15.152:0/2205241393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64fc1a4ff0 con 0x7f64fc10c170 2024-09-11T23:00:01.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.735+0000 7f64f2ffd640 1 -- 172.21.15.152:0/2205241393 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f64f40051d0 con 0x7f64fc10c170 2024-09-11T23:00:01.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.736+0000 7f65064a3740 1 Processor -- start 2024-09-11T23:00:01.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.736+0000 7f65064a3740 1 -- start start 2024-09-11T23:00:01.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.736+0000 7f65064a3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e0352b40 0x5591e0351090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.736+0000 7f65064a3740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e0351a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f65064a3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e035c370 0x5591e035e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f65064a3740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5591e02b72f0 con 0x5591e03515d0 2024-09-11T23:00:01.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f65064a3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5591e025ecf0 con 0x5591e035c370 2024-09-11T23:00:01.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f65064a3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5591e02b32b0 con 0x5591e0352b40 2024-09-11T23:00:01.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f64f2ffd640 1 --2- 172.21.15.152:0/2205241393 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f64e80a2cf0 0x7f64e80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f6501e66640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e0351a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f6501e66640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e0351a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56422/0 (socket says 172.21.15.152:56422) 2024-09-11T23:00:01.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f6501e66640 1 -- 172.21.15.152:0/935282145 learned_addr learned my addr 172.21.15.152:0/935282145 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:01.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f6502667640 1 --2- 172.21.15.152:0/2205241393 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f64e80a2cf0 0x7f64e80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f6501665640 1 --2- 172.21.15.152:0/935282145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e035c370 0x5591e035e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f6501665640 1 --2- 172.21.15.152:0/935282145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e0352b40 0x5591e0351090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f64f2ffd640 1 -- 172.21.15.152:0/2205241393 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f64f40cf470 con 0x7f64fc10c170 2024-09-11T23:00:01.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f6501e66640 1 -- 172.21.15.152:0/935282145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e0352b40 msgr2=0x5591e0351090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:01.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f6501e66640 1 --2- 172.21.15.152:0/935282145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e0352b40 0x5591e0351090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f6501e66640 1 -- 172.21.15.152:0/935282145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e035c370 msgr2=0x5591e035e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:01.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f6501e66640 1 --2- 172.21.15.152:0/935282145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e035c370 0x5591e035e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f6501e66640 1 -- 172.21.15.152:0/935282145 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591e02433a0 con 0x5591e03515d0 2024-09-11T23:00:01.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f6501e66640 1 --2- 172.21.15.152:0/935282145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e0351a10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f64f800bcb0 tx=0x7f64f800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876d5adc1b1d6fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:01.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f6502667640 1 --2- 172.21.15.152:0/2205241393 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f64e80a2cf0 0x7f64e80a51b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f64f4097e00 tx=0x7f64f40c6530 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:01.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f64e7fff640 1 -- 172.21.15.152:0/935282145 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8015070 con 0x5591e03515d0 2024-09-11T23:00:01.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f64e7fff640 1 -- 172.21.15.152:0/935282145 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f801a940 con 0x5591e03515d0 2024-09-11T23:00:01.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.737+0000 7f64e7fff640 1 -- 172.21.15.152:0/935282145 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8011a10 con 0x5591e03515d0 2024-09-11T23:00:01.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.738+0000 7f65064a3740 1 -- 172.21.15.152:0/935282145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 msgr2=0x5591e0351a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:01.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.738+0000 7f65064a3740 1 --2- 172.21.15.152:0/935282145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e0351a10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f64f800bcb0 tx=0x7f64f800f730 comp rx=0 tx=0).stop 2024-09-11T23:00:01.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.738+0000 7f65064a3740 1 -- 172.21.15.152:0/935282145 shutdown_connections 2024-09-11T23:00:01.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.738+0000 7f65064a3740 1 --2- 172.21.15.152:0/935282145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e035c370 0x5591e035e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.738+0000 7f65064a3740 1 --2- 172.21.15.152:0/935282145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e0351a10 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.738+0000 7f65064a3740 1 --2- 172.21.15.152:0/935282145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e0352b40 0x5591e0351090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.738+0000 7f65064a3740 1 -- 172.21.15.152:0/935282145 >> 172.21.15.152:0/935282145 conn(0x5591e031a310 msgr2=0x5591e034dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:01.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.738+0000 7f65064a3740 1 -- 172.21.15.152:0/935282145 shutdown_connections 2024-09-11T23:00:01.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.738+0000 7f65064a3740 1 -- 172.21.15.152:0/935282145 wait complete. 2024-09-11T23:00:01.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.739+0000 7f65064a3740 1 -- start start 2024-09-11T23:00:01.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.739+0000 7f65064a3740 1 Processor -- start 2024-09-11T23:00:01.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.739+0000 7f65064a3740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e035bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.739+0000 7f65064a3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e0352b40 0x5591e0358ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.739+0000 7f65064a3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e035c370 0x5591e03595e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.739+0000 7f65064a3740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5591e035f070 con 0x5591e03515d0 2024-09-11T23:00:01.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.739+0000 7f65064a3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5591e0272060 con 0x5591e0352b40 2024-09-11T23:00:01.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.739+0000 7f65064a3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5591e035f1f0 con 0x5591e035c370 2024-09-11T23:00:01.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.739+0000 7f65064a3740 10 client.0 _ll_register_callbacks cb 0x5591dffc58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:01.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.739+0000 7f6501665640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e035bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.739+0000 7f6501665640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e035bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56426/0 (socket says 172.21.15.152:56426) 2024-09-11T23:00:01.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.739+0000 7f6501665640 1 -- 172.21.15.152:0/3066584119 learned_addr learned my addr 172.21.15.152:0/3066584119 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:01.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.739+0000 7f6501e66640 1 --2- 172.21.15.152:0/3066584119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e0352b40 0x5591e0358ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.739+0000 7f6501665640 1 --2- 172.21.15.152:0/3066584119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e035c370 0x5591e03595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f6501665640 1 -- 172.21.15.152:0/3066584119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e035c370 msgr2=0x5591e03595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:01.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f6501665640 1 --2- 172.21.15.152:0/3066584119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e035c370 0x5591e03595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f6501665640 1 -- 172.21.15.152:0/3066584119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e0352b40 msgr2=0x5591e0358ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:01.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f6501665640 1 --2- 172.21.15.152:0/3066584119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e0352b40 0x5591e0358ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f6501665640 1 -- 172.21.15.152:0/3066584119 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591e00283c0 con 0x5591e03515d0 2024-09-11T23:00:01.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f6501665640 1 --2- 172.21.15.152:0/3066584119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e035bd40 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f64ec02a9a0 tx=0x7f64ec009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d603649699778304 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:01.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f64e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:01.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f64e57fa640 1 -- 172.21.15.152:0/3066584119 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec035040 con 0x5591e03515d0 2024-09-11T23:00:01.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f64e57fa640 1 -- 172.21.15.152:0/3066584119 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f64ec004250 con 0x5591e03515d0 2024-09-11T23:00:01.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f65064a3740 1 -- 172.21.15.152:0/3066584119 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64f801a940 con 0x5591e03515d0 2024-09-11T23:00:01.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f64e57fa640 1 -- 172.21.15.152:0/3066584119 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f64ec053050 con 0x5591e03515d0 2024-09-11T23:00:01.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f65064a3740 10 client.17442 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:01.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f65064a3740 1 -- 172.21.15.152:0/3066584119 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5591e036eef0 con 0x5591e03515d0 2024-09-11T23:00:01.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f64e57fa640 1 -- 172.21.15.152:0/3066584119 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64f801a940 con 0x5591e03515d0 2024-09-11T23:00:01.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f64e57fa640 1 -- 172.21.15.152:0/3066584119 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f64ec004b30 con 0x5591e03515d0 2024-09-11T23:00:01.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f65064a3740 10 client.17442 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:01.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f65064a3740 10 client.17442 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T23:00:01.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.740+0000 7f65064a3740 20 client.17442 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:01.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.741+0000 7f65064a3740 1 --2- 172.21.15.152:0/3066584119 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5591e0370300 0x5591e0372730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.741+0000 7f65064a3740 4 client.17442 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:00:01.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.741+0000 7f65064a3740 1 -- 172.21.15.152:0/3066584119 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591e02c4860 con 0x5591e0370300 2024-09-11T23:00:01.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.741+0000 7f6501e66640 1 --2- 172.21.15.152:0/3066584119 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5591e0370300 0x5591e0372730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.741+0000 7f64e57fa640 10 client.17442 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T23:00:01.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.741+0000 7f6501e66640 1 --2- 172.21.15.152:0/3066584119 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5591e0370300 0x5591e0372730 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9abb8eb2b43f4301 server_cookie=6e3338021078efdc in_seq=0 out_seq=0 2024-09-11T23:00:01.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.741+0000 7f64e57fa640 1 -- 172.21.15.152:0/3066584119 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5591e02c4860 con 0x5591e0370300 2024-09-11T23:00:01.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.741+0000 7f64e57fa640 10 client.17442 handle_command_reply: tid=0 2024-09-11T23:00:01.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.742+0000 7f65064a3740 1 client.17442 shutdown 2024-09-11T23:00:01.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.742+0000 7f65064a3740 1 -- 172.21.15.152:0/3066584119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 msgr2=0x5591e035bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:01.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.742+0000 7f65064a3740 1 --2- 172.21.15.152:0/3066584119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e035bd40 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f64ec02a9a0 tx=0x7f64ec009260 comp rx=0 tx=0).stop 2024-09-11T23:00:01.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.742+0000 7f65064a3740 1 -- 172.21.15.152:0/3066584119 shutdown_connections 2024-09-11T23:00:01.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.742+0000 7f65064a3740 1 --2- 172.21.15.152:0/3066584119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e035c370 0x5591e03595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.742+0000 7f65064a3740 1 --2- 172.21.15.152:0/3066584119 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5591e0370300 0x5591e0372730 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.742+0000 7f65064a3740 1 --2- 172.21.15.152:0/3066584119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e0352b40 0x5591e0358ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.742+0000 7f65064a3740 1 --2- 172.21.15.152:0/3066584119 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e035bd40 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.742+0000 7f64e57fa640 0 client.17442 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T23:00:01.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.742+0000 7f65064a3740 1 -- 172.21.15.152:0/3066584119 >> 172.21.15.152:0/3066584119 conn(0x5591e031a310 msgr2=0x5591e0353960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:01.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.742+0000 7f65064a3740 1 -- 172.21.15.152:0/3066584119 shutdown_connections 2024-09-11T23:00:01.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.742+0000 7f65064a3740 1 -- 172.21.15.152:0/3066584119 wait complete. 2024-09-11T23:00:01.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.742+0000 7f65064a3740 20 client.17442 trim_cache size 0 max 16384 2024-09-11T23:00:01.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f65064a3740 1 Processor -- start 2024-09-11T23:00:01.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f65064a3740 1 -- start start 2024-09-11T23:00:01.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f65064a3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e03515d0 0x5591e034cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f65064a3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e0352b40 0x5591e0354120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f65064a3740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e035c370 0x5591e0354660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f65064a3740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1ab540 con 0x5591e035c370 2024-09-11T23:00:01.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f65064a3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1ad080 con 0x5591e0352b40 2024-09-11T23:00:01.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f65064a3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1abc10 con 0x5591e03515d0 2024-09-11T23:00:01.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f6501665640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e035c370 0x5591e0354660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f6501665640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e035c370 0x5591e0354660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56428/0 (socket says 172.21.15.152:56428) 2024-09-11T23:00:01.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f6501665640 1 -- 172.21.15.152:0/2020969564 learned_addr learned my addr 172.21.15.152:0/2020969564 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:01.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f6501e66640 1 --2- 172.21.15.152:0/2020969564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e0352b40 0x5591e0354120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f6501665640 1 --2- 172.21.15.152:0/2020969564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e03515d0 0x5591e034cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f6501665640 1 -- 172.21.15.152:0/2020969564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e03515d0 msgr2=0x5591e034cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:01.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f6501665640 1 --2- 172.21.15.152:0/2020969564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e03515d0 0x5591e034cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f6501665640 1 -- 172.21.15.152:0/2020969564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e0352b40 msgr2=0x5591e0354120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:01.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f6501665640 1 --2- 172.21.15.152:0/2020969564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e0352b40 0x5591e0354120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f6501665640 1 -- 172.21.15.152:0/2020969564 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591e0357540 con 0x5591e035c370 2024-09-11T23:00:01.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.755+0000 7f6501665640 1 --2- 172.21.15.152:0/2020969564 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e035c370 0x5591e0354660 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009230 tx=0x7f64ec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d4386e2a3b4e02 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:01.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.756+0000 7f64e77fe640 1 -- 172.21.15.152:0/2020969564 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec035070 con 0x5591e035c370 2024-09-11T23:00:01.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.756+0000 7f64e77fe640 1 -- 172.21.15.152:0/2020969564 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64ec0315b0 con 0x5591e035c370 2024-09-11T23:00:01.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.756+0000 7f64e77fe640 1 -- 172.21.15.152:0/2020969564 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec041040 con 0x5591e035c370 2024-09-11T23:00:01.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.756+0000 7f65064a3740 1 -- 172.21.15.152:0/2020969564 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e035c370 msgr2=0x5591e0354660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:01.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.756+0000 7f65064a3740 1 --2- 172.21.15.152:0/2020969564 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e035c370 0x5591e0354660 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009230 tx=0x7f64ec000bc0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.756+0000 7f65064a3740 1 -- 172.21.15.152:0/2020969564 shutdown_connections 2024-09-11T23:00:01.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.756+0000 7f65064a3740 1 --2- 172.21.15.152:0/2020969564 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e035c370 0x5591e0354660 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.756+0000 7f65064a3740 1 --2- 172.21.15.152:0/2020969564 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e0352b40 0x5591e0354120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.756+0000 7f65064a3740 1 --2- 172.21.15.152:0/2020969564 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e03515d0 0x5591e034cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.756+0000 7f65064a3740 1 -- 172.21.15.152:0/2020969564 >> 172.21.15.152:0/2020969564 conn(0x5591e031a310 msgr2=0x5591e034c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:01.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.756+0000 7f65064a3740 1 -- 172.21.15.152:0/2020969564 shutdown_connections 2024-09-11T23:00:01.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.756+0000 7f65064a3740 1 -- 172.21.15.152:0/2020969564 wait complete. 2024-09-11T23:00:01.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.757+0000 7f65064a3740 1 -- start start 2024-09-11T23:00:01.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.757+0000 7f65064a3740 1 Processor -- start 2024-09-11T23:00:01.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.757+0000 7f65064a3740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e0368f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.757+0000 7f65064a3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e0352b40 0x5591e0355c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.757+0000 7f65064a3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e035c370 0x5591e0356140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.757+0000 7f65064a3740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1ac9b0 con 0x5591e03515d0 2024-09-11T23:00:01.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.757+0000 7f65064a3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1ac2e0 con 0x5591e035c370 2024-09-11T23:00:01.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.757+0000 7f65064a3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc1a9500 con 0x5591e0352b40 2024-09-11T23:00:01.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.757+0000 7f65064a3740 10 client.0 _ll_register_callbacks cb 0x7f64ec009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:01.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.758+0000 7f6501665640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e0368f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.758+0000 7f6501e66640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e0352b40 0x5591e0355c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.758+0000 7f6501665640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e0368f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56440/0 (socket says 172.21.15.152:56440) 2024-09-11T23:00:01.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.758+0000 7f6501665640 1 -- 172.21.15.152:0/2616700225 learned_addr learned my addr 172.21.15.152:0/2616700225 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:01.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.758+0000 7f6501665640 1 --2- 172.21.15.152:0/2616700225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e035c370 0x5591e0356140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.758+0000 7f6501665640 1 -- 172.21.15.152:0/2616700225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e0352b40 msgr2=0x5591e0355c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:01.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.758+0000 7f6501665640 1 --2- 172.21.15.152:0/2616700225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e0352b40 0x5591e0355c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.758+0000 7f6501665640 1 -- 172.21.15.152:0/2616700225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e035c370 msgr2=0x5591e0356140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:01.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.758+0000 7f6501665640 1 --2- 172.21.15.152:0/2616700225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e035c370 0x5591e0356140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.758+0000 7f6501665640 1 -- 172.21.15.152:0/2616700225 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591e0356830 con 0x5591e03515d0 2024-09-11T23:00:01.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.758+0000 7f6501e66640 1 --2- 172.21.15.152:0/2616700225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e0352b40 0x5591e0355c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:00:01.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.758+0000 7f6501665640 1 --2- 172.21.15.152:0/2616700225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e0368f30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f64ec031bf0 tx=0x7f64ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c32fe253ed1b0d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:01.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.758+0000 7f64e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:01.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.759+0000 7f64e57fa640 1 -- 172.21.15.152:0/2616700225 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec035070 con 0x5591e03515d0 2024-09-11T23:00:01.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.759+0000 7f64e57fa640 1 -- 172.21.15.152:0/2616700225 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f64ec003ca0 con 0x5591e03515d0 2024-09-11T23:00:01.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.759+0000 7f65064a3740 1 -- 172.21.15.152:0/2616700225 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64ec0315b0 con 0x5591e03515d0 2024-09-11T23:00:01.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.759+0000 7f64e57fa640 1 -- 172.21.15.152:0/2616700225 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f64ec068030 con 0x5591e03515d0 2024-09-11T23:00:01.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.759+0000 7f65064a3740 10 client.17448 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:01.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.759+0000 7f65064a3740 1 -- 172.21.15.152:0/2616700225 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5591e03dae90 con 0x5591e03515d0 2024-09-11T23:00:01.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.759+0000 7f64e57fa640 1 -- 172.21.15.152:0/2616700225 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64ec0315b0 con 0x5591e03515d0 2024-09-11T23:00:01.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.759+0000 7f64e57fa640 1 -- 172.21.15.152:0/2616700225 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f64ec004850 con 0x5591e03515d0 2024-09-11T23:00:01.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.759+0000 7f65064a3740 10 client.17448 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:01.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.759+0000 7f65064a3740 10 client.17448 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T23:00:01.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.759+0000 7f65064a3740 20 client.17448 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:01.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.759+0000 7f65064a3740 1 --2- 172.21.15.152:0/2616700225 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5591e03db3f0 0x5591e03dd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:01.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.759+0000 7f65064a3740 4 client.17448 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:00:01.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.759+0000 7f65064a3740 1 -- 172.21.15.152:0/2616700225 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591e02bed70 con 0x5591e03db3f0 2024-09-11T23:00:01.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.759+0000 7f6501e66640 1 --2- 172.21.15.152:0/2616700225 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5591e03db3f0 0x5591e03dd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:01.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.760+0000 7f64e57fa640 10 client.17448 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T23:00:01.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.760+0000 7f6501e66640 1 --2- 172.21.15.152:0/2616700225 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5591e03db3f0 0x5591e03dd7e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=299f35b0007f42de server_cookie=21898ddd155bbe95 in_seq=0 out_seq=0 2024-09-11T23:00:01.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.760+0000 7f64e57fa640 1 -- 172.21.15.152:0/2616700225 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591e02bed70 con 0x5591e03db3f0 2024-09-11T23:00:01.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.760+0000 7f64e57fa640 10 client.17448 handle_command_reply: tid=0 2024-09-11T23:00:01.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.760+0000 7f65064a3740 1 client.17448 shutdown 2024-09-11T23:00:01.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.760+0000 7f65064a3740 1 -- 172.21.15.152:0/2616700225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 msgr2=0x5591e0368f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:01.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.760+0000 7f65064a3740 1 --2- 172.21.15.152:0/2616700225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e0368f30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f64ec031bf0 tx=0x7f64ec0042f0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.760+0000 7f65064a3740 1 -- 172.21.15.152:0/2616700225 shutdown_connections 2024-09-11T23:00:01.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.760+0000 7f65064a3740 1 --2- 172.21.15.152:0/2616700225 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5591e03db3f0 0x5591e03dd7e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.760+0000 7f65064a3740 1 --2- 172.21.15.152:0/2616700225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5591e035c370 0x5591e0356140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.760+0000 7f65064a3740 1 --2- 172.21.15.152:0/2616700225 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5591e0352b40 0x5591e0355c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.760+0000 7f65064a3740 1 --2- 172.21.15.152:0/2616700225 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591e03515d0 0x5591e0368f30 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.760+0000 7f64e57fa640 0 client.17448 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T23:00:01.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.760+0000 7f65064a3740 1 -- 172.21.15.152:0/2616700225 >> 172.21.15.152:0/2616700225 conn(0x5591e031a310 msgr2=0x5591e034cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:01.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.760+0000 7f65064a3740 1 -- 172.21.15.152:0/2616700225 shutdown_connections 2024-09-11T23:00:01.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.760+0000 7f65064a3740 1 -- 172.21.15.152:0/2616700225 wait complete. 2024-09-11T23:00:01.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.761+0000 7f65064a3740 20 client.17448 trim_cache size 0 max 16384 2024-09-11T23:00:01.782 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:00:01.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.761+0000 7f65040ca640 1 -- 172.21.15.152:0/2205241393 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f64e80a2cf0 msgr2=0x7f64e80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:01.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.761+0000 7f65040ca640 1 --2- 172.21.15.152:0/2205241393 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f64e80a2cf0 0x7f64e80a51b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f64f4097e00 tx=0x7f64f40c6530 comp rx=0 tx=0).stop 2024-09-11T23:00:01.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.761+0000 7f65040ca640 1 -- 172.21.15.152:0/2205241393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64fc10c170 msgr2=0x7f64fc19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:01.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.761+0000 7f65040ca640 1 --2- 172.21.15.152:0/2205241393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64fc10c170 0x7f64fc19cd40 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f64f400cad0 tx=0x7f64f400cf90 comp rx=0 tx=0).stop 2024-09-11T23:00:01.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.761+0000 7f65040ca640 1 -- 172.21.15.152:0/2205241393 shutdown_connections 2024-09-11T23:00:01.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.761+0000 7f65040ca640 1 --2- 172.21.15.152:0/2205241393 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f64e80a2cf0 0x7f64e80a51b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.761+0000 7f65040ca640 1 --2- 172.21.15.152:0/2205241393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64fc10c170 0x7f64fc19cd40 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.761+0000 7f65040ca640 1 --2- 172.21.15.152:0/2205241393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64fc103660 0x7f64fc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.761+0000 7f65040ca640 1 --2- 172.21.15.152:0/2205241393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64fc102d40 0x7f64fc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:01.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.761+0000 7f65040ca640 1 -- 172.21.15.152:0/2205241393 >> 172.21.15.152:0/2205241393 conn(0x7f64fc0fc7a0 msgr2=0x7f64fc0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:01.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.761+0000 7f65040ca640 1 -- 172.21.15.152:0/2205241393 shutdown_connections 2024-09-11T23:00:01.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:01.761+0000 7f65040ca640 1 -- 172.21.15.152:0/2205241393 wait complete. 2024-09-11T23:00:02.608 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:00:02.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:02 smithi191 ceph-mon[35211]: pgmap v217: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-09-11T23:00:02.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:02 smithi204 ceph-mon[35454]: pgmap v217: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-09-11T23:00:02.857 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:02.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:02 smithi152 ceph-mon[31181]: pgmap v217: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-09-11T23:00:03.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.384+0000 7f135fca1640 1 -- 172.21.15.152:0/3625222724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13581002f0 msgr2=0x7f13581006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:03.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.384+0000 7f135fca1640 1 --2- 172.21.15.152:0/3625222724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13581002f0 0x7f13581006d0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f1354009930 tx=0x7f135402a420 comp rx=0 tx=0).stop 2024-09-11T23:00:03.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.384+0000 7f135fca1640 1 -- 172.21.15.152:0/3625222724 shutdown_connections 2024-09-11T23:00:03.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.384+0000 7f135fca1640 1 --2- 172.21.15.152:0/3625222724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1358106f50 0x7f135810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.384+0000 7f135fca1640 1 --2- 172.21.15.152:0/3625222724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1358100ca0 0x7f1358106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.384+0000 7f135fca1640 1 --2- 172.21.15.152:0/3625222724 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13581002f0 0x7f13581006d0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.384+0000 7f135fca1640 1 -- 172.21.15.152:0/3625222724 >> 172.21.15.152:0/3625222724 conn(0x7f1358077210 msgr2=0x7f13580fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:03.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.384+0000 7f135fca1640 1 -- 172.21.15.152:0/3625222724 shutdown_connections 2024-09-11T23:00:03.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.384+0000 7f135fca1640 1 -- 172.21.15.152:0/3625222724 wait complete. 2024-09-11T23:00:03.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.384+0000 7f135fca1640 1 Processor -- start 2024-09-11T23:00:03.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135fca1640 1 -- start start 2024-09-11T23:00:03.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135fca1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13581002f0 0x7f1358198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135fca1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1358100ca0 0x7f1358198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135fca1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1358106f50 0x7f135819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135fca1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1358111fa0 con 0x7f13581002f0 2024-09-11T23:00:03.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135fca1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1358111e20 con 0x7f1358100ca0 2024-09-11T23:00:03.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135fca1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1358112120 con 0x7f1358106f50 2024-09-11T23:00:03.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135da3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13581002f0 0x7f1358198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135da3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13581002f0 0x7f1358198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56450/0 (socket says 172.21.15.152:56450) 2024-09-11T23:00:03.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135da3d640 1 -- 172.21.15.152:0/659472602 learned_addr learned my addr 172.21.15.152:0/659472602 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:03.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135e23e640 1 --2- 172.21.15.152:0/659472602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1358106f50 0x7f135819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135d23c640 1 --2- 172.21.15.152:0/659472602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1358100ca0 0x7f1358198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135da3d640 1 -- 172.21.15.152:0/659472602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1358106f50 msgr2=0x7f135819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:03.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135da3d640 1 --2- 172.21.15.152:0/659472602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1358106f50 0x7f135819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135da3d640 1 -- 172.21.15.152:0/659472602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1358100ca0 msgr2=0x7f1358198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:03.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135da3d640 1 --2- 172.21.15.152:0/659472602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1358100ca0 0x7f1358198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135da3d640 1 -- 172.21.15.152:0/659472602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135819d510 con 0x7f13581002f0 2024-09-11T23:00:03.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.385+0000 7f135da3d640 1 --2- 172.21.15.152:0/659472602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13581002f0 0x7f1358198560 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1354009230 tx=0x7f1354004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:03.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.386+0000 7f134effd640 1 -- 172.21.15.152:0/659472602 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135403f030 con 0x7f13581002f0 2024-09-11T23:00:03.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.386+0000 7f135fca1640 1 -- 172.21.15.152:0/659472602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135819d7a0 con 0x7f13581002f0 2024-09-11T23:00:03.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.386+0000 7f134effd640 1 -- 172.21.15.152:0/659472602 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1354003ad0 con 0x7f13581002f0 2024-09-11T23:00:03.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.386+0000 7f134effd640 1 -- 172.21.15.152:0/659472602 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354035070 con 0x7f13581002f0 2024-09-11T23:00:03.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.386+0000 7f135fca1640 1 -- 172.21.15.152:0/659472602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13581a50e0 con 0x7f13581002f0 2024-09-11T23:00:03.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.388+0000 7f134effd640 1 -- 172.21.15.152:0/659472602 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1354042030 con 0x7f13581002f0 2024-09-11T23:00:03.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.388+0000 7f136207a740 1 Processor -- start 2024-09-11T23:00:03.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.388+0000 7f136207a740 1 -- start start 2024-09-11T23:00:03.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f134effd640 1 --2- 172.21.15.152:0/659472602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f13440a2cf0 0x7f13440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f136207a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce5bb40 0x55ab1ce5a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f136207a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce5aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f136207a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce69780 0x55ab1ce6bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f136207a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ab1cdc02f0 con 0x55ab1ce69780 2024-09-11T23:00:03.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f136207a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ab1cd67cf0 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f136207a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ab1cdbc2b0 con 0x55ab1ce5bb40 2024-09-11T23:00:03.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f135e23e640 1 --2- 172.21.15.152:0/659472602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f13440a2cf0 0x7f13440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f134effd640 1 -- 172.21.15.152:0/659472602 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f13540ef860 con 0x7f13581002f0 2024-09-11T23:00:03.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f135d23c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce5aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f135da3d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce5bb40 0x55ab1ce5a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f135d23c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce5aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:40778/0 (socket says 172.21.15.152:40778) 2024-09-11T23:00:03.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f135d23c640 1 -- 172.21.15.152:0/3074665000 learned_addr learned my addr 172.21.15.152:0/3074665000 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:03.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f135e23e640 1 --2- 172.21.15.152:0/3074665000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce69780 0x55ab1ce6bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f135d23c640 1 -- 172.21.15.152:0/3074665000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce5bb40 msgr2=0x55ab1ce5a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:03.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f135d23c640 1 --2- 172.21.15.152:0/3074665000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce5bb40 0x55ab1ce5a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f135d23c640 1 -- 172.21.15.152:0/3074665000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce69780 msgr2=0x55ab1ce6bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:03.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f135d23c640 1 --2- 172.21.15.152:0/3074665000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce69780 0x55ab1ce6bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f135d23c640 1 -- 172.21.15.152:0/3074665000 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab1cd4c3a0 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f135da3d640 1 --2- 172.21.15.152:0/3074665000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce5bb40 0x55ab1ce5a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:00:03.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f135e23e640 1 --2- 172.21.15.152:0/3074665000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce69780 0x55ab1ce6bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:00:03.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f135e23e640 1 --2- 172.21.15.152:0/659472602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f13440a2cf0 0x7f13440a51b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1350009fd0 tx=0x7f1350009da0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:03.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.389+0000 7f135d23c640 1 --2- 172.21.15.152:0/3074665000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce5aa10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f134800bc90 tx=0x7f13480107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62628202590247bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:03.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.390+0000 7f1343fff640 1 -- 172.21.15.152:0/3074665000 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134801c480 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.390+0000 7f1343fff640 1 -- 172.21.15.152:0/3074665000 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134801cb00 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.390+0000 7f1343fff640 1 -- 172.21.15.152:0/3074665000 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348011b80 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.390+0000 7f136207a740 1 -- 172.21.15.152:0/3074665000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5a5d0 msgr2=0x55ab1ce5aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:03.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.390+0000 7f136207a740 1 --2- 172.21.15.152:0/3074665000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce5aa10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f134800bc90 tx=0x7f13480107a0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.390+0000 7f136207a740 1 -- 172.21.15.152:0/3074665000 shutdown_connections 2024-09-11T23:00:03.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.390+0000 7f136207a740 1 --2- 172.21.15.152:0/3074665000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce69780 0x55ab1ce6bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.390+0000 7f136207a740 1 --2- 172.21.15.152:0/3074665000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce5aa10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.390+0000 7f136207a740 1 --2- 172.21.15.152:0/3074665000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce5bb40 0x55ab1ce5a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.390+0000 7f136207a740 1 -- 172.21.15.152:0/3074665000 >> 172.21.15.152:0/3074665000 conn(0x55ab1ce23310 msgr2=0x55ab1ce56c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:03.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.391+0000 7f136207a740 1 -- 172.21.15.152:0/3074665000 shutdown_connections 2024-09-11T23:00:03.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.391+0000 7f136207a740 1 -- 172.21.15.152:0/3074665000 wait complete. 2024-09-11T23:00:03.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.391+0000 7f136207a740 1 -- start start 2024-09-11T23:00:03.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.391+0000 7f136207a740 1 Processor -- start 2024-09-11T23:00:03.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.391+0000 7f136207a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce69160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.391+0000 7f136207a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce5bb40 0x55ab1ce66410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.391+0000 7f136207a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce69780 0x55ab1ce669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.391+0000 7f136207a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ab1ce6c480 con 0x55ab1ce69780 2024-09-11T23:00:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.391+0000 7f136207a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ab1cd7b060 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.391+0000 7f136207a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ab1ce6c600 con 0x55ab1ce5bb40 2024-09-11T23:00:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.391+0000 7f136207a740 10 client.0 _ll_register_callbacks cb 0x55ab1cace8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.391+0000 7f135e23e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce69780 0x55ab1ce669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.391+0000 7f135e23e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce69780 0x55ab1ce669e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56470/0 (socket says 172.21.15.152:56470) 2024-09-11T23:00:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.391+0000 7f135e23e640 1 -- 172.21.15.152:0/3125836819 learned_addr learned my addr 172.21.15.152:0/3125836819 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:03.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.392+0000 7f135da3d640 1 --2- 172.21.15.152:0/3125836819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce69160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.392+0000 7f135e23e640 1 -- 172.21.15.152:0/3125836819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce5bb40 msgr2=0x55ab1ce66410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:03.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.392+0000 7f135e23e640 1 --2- 172.21.15.152:0/3125836819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce5bb40 0x55ab1ce66410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.392+0000 7f135e23e640 1 -- 172.21.15.152:0/3125836819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5a5d0 msgr2=0x55ab1ce69160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:03.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.392+0000 7f135e23e640 1 --2- 172.21.15.152:0/3125836819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce69160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.392+0000 7f135e23e640 1 -- 172.21.15.152:0/3125836819 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab1cb313c0 con 0x55ab1ce69780 2024-09-11T23:00:03.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.392+0000 7f135e23e640 1 --2- 172.21.15.152:0/3125836819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce69780 0x55ab1ce669e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1350010e40 tx=0x7f1350009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9462155e954bec90 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:03.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.392+0000 7f13417fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:03.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.392+0000 7f13417fa640 1 -- 172.21.15.152:0/3125836819 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350003e00 con 0x55ab1ce69780 2024-09-11T23:00:03.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.392+0000 7f136207a740 1 -- 172.21.15.152:0/3125836819 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f134801cb00 con 0x55ab1ce69780 2024-09-11T23:00:03.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.392+0000 7f13417fa640 1 -- 172.21.15.152:0/3125836819 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f13500046c0 con 0x55ab1ce69780 2024-09-11T23:00:03.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.392+0000 7f13417fa640 1 -- 172.21.15.152:0/3125836819 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1350029480 con 0x55ab1ce69780 2024-09-11T23:00:03.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.392+0000 7f136207a740 10 client.17460 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:03.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.392+0000 7f136207a740 1 -- 172.21.15.152:0/3125836819 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab1ce77f50 con 0x55ab1ce69780 2024-09-11T23:00:03.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.392+0000 7f13417fa640 1 -- 172.21.15.152:0/3125836819 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134801cb00 con 0x55ab1ce69780 2024-09-11T23:00:03.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.393+0000 7f13417fa640 1 -- 172.21.15.152:0/3125836819 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f135001f070 con 0x55ab1ce69780 2024-09-11T23:00:03.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.393+0000 7f136207a740 10 client.17460 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:03.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.393+0000 7f136207a740 10 client.17460 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:00:03.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.393+0000 7f136207a740 20 client.17460 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:03.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.393+0000 7f136207a740 1 --2- 172.21.15.152:0/3125836819 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55ab1ce793b0 0x55ab1ce7b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.393+0000 7f136207a740 4 client.17460 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:00:03.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.393+0000 7f136207a740 1 -- 172.21.15.152:0/3125836819 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab1cdcd860 con 0x55ab1ce793b0 2024-09-11T23:00:03.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.393+0000 7f135da3d640 1 --2- 172.21.15.152:0/3125836819 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55ab1ce793b0 0x55ab1ce7b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.393+0000 7f13417fa640 10 client.17460 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T23:00:03.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.393+0000 7f135da3d640 1 --2- 172.21.15.152:0/3125836819 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55ab1ce793b0 0x55ab1ce7b870 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=87e9ba3b63de9ea8 server_cookie=382cbc9e21745e3d in_seq=0 out_seq=0 2024-09-11T23:00:03.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.394+0000 7f13417fa640 1 -- 172.21.15.152:0/3125836819 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab1cdcd860 con 0x55ab1ce793b0 2024-09-11T23:00:03.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.394+0000 7f13417fa640 10 client.17460 handle_command_reply: tid=0 2024-09-11T23:00:03.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.394+0000 7f136207a740 1 client.17460 shutdown 2024-09-11T23:00:03.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.394+0000 7f136207a740 1 -- 172.21.15.152:0/3125836819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce69780 msgr2=0x55ab1ce669e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:03.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.394+0000 7f136207a740 1 --2- 172.21.15.152:0/3125836819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce69780 0x55ab1ce669e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f1350010e40 tx=0x7f1350009600 comp rx=0 tx=0).stop 2024-09-11T23:00:03.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.394+0000 7f136207a740 1 -- 172.21.15.152:0/3125836819 shutdown_connections 2024-09-11T23:00:03.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.394+0000 7f136207a740 1 --2- 172.21.15.152:0/3125836819 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55ab1ce793b0 0x55ab1ce7b870 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.394+0000 7f136207a740 1 --2- 172.21.15.152:0/3125836819 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce69780 0x55ab1ce669e0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.394+0000 7f136207a740 1 --2- 172.21.15.152:0/3125836819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce5bb40 0x55ab1ce66410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.394+0000 7f13417fa640 0 client.17460 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T23:00:03.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.394+0000 7f136207a740 1 --2- 172.21.15.152:0/3125836819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce69160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.394+0000 7f136207a740 1 -- 172.21.15.152:0/3125836819 >> 172.21.15.152:0/3125836819 conn(0x55ab1ce23310 msgr2=0x55ab1ce5c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:03.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.394+0000 7f136207a740 1 -- 172.21.15.152:0/3125836819 shutdown_connections 2024-09-11T23:00:03.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.394+0000 7f136207a740 1 -- 172.21.15.152:0/3125836819 wait complete. 2024-09-11T23:00:03.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.394+0000 7f136207a740 20 client.17460 trim_cache size 0 max 16384 2024-09-11T23:00:03.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.407+0000 7f136207a740 1 Processor -- start 2024-09-11T23:00:03.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.407+0000 7f136207a740 1 -- start start 2024-09-11T23:00:03.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.407+0000 7f136207a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce56e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.407+0000 7f136207a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5bb40 0x55ab1ce75720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.407+0000 7f136207a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce69780 0x55ab1ce81140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.407+0000 7f136207a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f13581ab640 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.407+0000 7f136207a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f13581ad180 con 0x55ab1ce5bb40 2024-09-11T23:00:03.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.407+0000 7f136207a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f13581abd10 con 0x55ab1ce69780 2024-09-11T23:00:03.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.408+0000 7f135da3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce56e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.408+0000 7f135da3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce56e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56472/0 (socket says 172.21.15.152:56472) 2024-09-11T23:00:03.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.408+0000 7f135da3d640 1 -- 172.21.15.152:0/2860878704 learned_addr learned my addr 172.21.15.152:0/2860878704 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:03.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.408+0000 7f135d23c640 1 --2- 172.21.15.152:0/2860878704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5bb40 0x55ab1ce75720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.408+0000 7f135e23e640 1 --2- 172.21.15.152:0/2860878704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce69780 0x55ab1ce81140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.408+0000 7f135da3d640 1 -- 172.21.15.152:0/2860878704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce69780 msgr2=0x55ab1ce81140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:03.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.408+0000 7f135da3d640 1 --2- 172.21.15.152:0/2860878704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce69780 0x55ab1ce81140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.408+0000 7f135da3d640 1 -- 172.21.15.152:0/2860878704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5bb40 msgr2=0x55ab1ce75720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:03.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.408+0000 7f135da3d640 1 --2- 172.21.15.152:0/2860878704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5bb40 0x55ab1ce75720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.408+0000 7f135da3d640 1 -- 172.21.15.152:0/2860878704 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab1ce564f0 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.408+0000 7f135da3d640 1 --2- 172.21.15.152:0/2860878704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce56e10 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f13540ed020 tx=0x7f135402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166f8f389289a87f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:03.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.408+0000 7f13437fe640 1 -- 172.21.15.152:0/2860878704 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354049040 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.408+0000 7f13437fe640 1 -- 172.21.15.152:0/2860878704 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13540ba140 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.408+0000 7f13437fe640 1 -- 172.21.15.152:0/2860878704 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13540bc0a0 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.409+0000 7f136207a740 1 -- 172.21.15.152:0/2860878704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce5a5d0 msgr2=0x55ab1ce56e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:03.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.409+0000 7f136207a740 1 --2- 172.21.15.152:0/2860878704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce56e10 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f13540ed020 tx=0x7f135402a420 comp rx=0 tx=0).stop 2024-09-11T23:00:03.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.409+0000 7f136207a740 1 -- 172.21.15.152:0/2860878704 shutdown_connections 2024-09-11T23:00:03.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.409+0000 7f136207a740 1 --2- 172.21.15.152:0/2860878704 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce69780 0x55ab1ce81140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.409+0000 7f136207a740 1 --2- 172.21.15.152:0/2860878704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce5bb40 0x55ab1ce75720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.409+0000 7f136207a740 1 --2- 172.21.15.152:0/2860878704 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce56e10 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.409+0000 7f136207a740 1 -- 172.21.15.152:0/2860878704 >> 172.21.15.152:0/2860878704 conn(0x55ab1ce23310 msgr2=0x55ab1ce56700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:03.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.409+0000 7f136207a740 1 -- 172.21.15.152:0/2860878704 shutdown_connections 2024-09-11T23:00:03.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.409+0000 7f136207a740 1 -- 172.21.15.152:0/2860878704 wait complete. 2024-09-11T23:00:03.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.409+0000 7f136207a740 1 -- start start 2024-09-11T23:00:03.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.409+0000 7f136207a740 1 Processor -- start 2024-09-11T23:00:03.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f136207a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce71a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f136207a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce5bb40 0x55ab1ce71fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f136207a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce69780 0x55ab1ce6e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f136207a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f13581acab0 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f136207a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f13581ac3e0 con 0x55ab1ce69780 2024-09-11T23:00:03.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f136207a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f13581a9620 con 0x55ab1ce5bb40 2024-09-11T23:00:03.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f136207a740 10 client.0 _ll_register_callbacks cb 0x7f13500124f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:03.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f135da3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce71a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f135da3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce71a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56484/0 (socket says 172.21.15.152:56484) 2024-09-11T23:00:03.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f135da3d640 1 -- 172.21.15.152:0/2831144912 learned_addr learned my addr 172.21.15.152:0/2831144912 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:03.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f135d23c640 1 --2- 172.21.15.152:0/2831144912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce5bb40 0x55ab1ce71fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f135e23e640 1 --2- 172.21.15.152:0/2831144912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce69780 0x55ab1ce6e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f135da3d640 1 -- 172.21.15.152:0/2831144912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce5bb40 msgr2=0x55ab1ce71fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:03.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f135da3d640 1 --2- 172.21.15.152:0/2831144912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce5bb40 0x55ab1ce71fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f135da3d640 1 -- 172.21.15.152:0/2831144912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce69780 msgr2=0x55ab1ce6e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:03.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f135da3d640 1 --2- 172.21.15.152:0/2831144912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce69780 0x55ab1ce6e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f135da3d640 1 -- 172.21.15.152:0/2831144912 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab1ce6f090 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f135da3d640 1 --2- 172.21.15.152:0/2831144912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce71a70 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f13540b7b20 tx=0x7f13540045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f135187de83a02 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:03.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.410+0000 7f13417fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:03.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.412+0000 7f13417fa640 1 -- 172.21.15.152:0/2831144912 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13540eb020 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.412+0000 7f13417fa640 1 -- 172.21.15.152:0/2831144912 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f13540ed240 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.412+0000 7f136207a740 1 -- 172.21.15.152:0/2831144912 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13540ba140 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.412+0000 7f13417fa640 1 -- 172.21.15.152:0/2831144912 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1354049040 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.413+0000 7f136207a740 10 client.17466 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:03.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.413+0000 7f136207a740 1 -- 172.21.15.152:0/2831144912 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ab1cee4c40 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.413+0000 7f13417fa640 1 -- 172.21.15.152:0/2831144912 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13540ba140 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.413+0000 7f13417fa640 1 -- 172.21.15.152:0/2831144912 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1354031bd0 con 0x55ab1ce5a5d0 2024-09-11T23:00:03.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.413+0000 7f136207a740 10 client.17466 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:03.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.413+0000 7f136207a740 10 client.17466 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:00:03.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.413+0000 7f136207a740 20 client.17466 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:03.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.413+0000 7f136207a740 1 --2- 172.21.15.152:0/2831144912 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55ab1cee5300 0x55ab1cee76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:03.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.413+0000 7f136207a740 4 client.17466 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:00:03.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.413+0000 7f136207a740 1 -- 172.21.15.152:0/2831144912 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab1cdc7d70 con 0x55ab1cee5300 2024-09-11T23:00:03.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.413+0000 7f135d23c640 1 --2- 172.21.15.152:0/2831144912 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55ab1cee5300 0x55ab1cee76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:03.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f135d23c640 1 --2- 172.21.15.152:0/2831144912 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55ab1cee5300 0x55ab1cee76f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=681fe439eb0359d1 server_cookie=fd1db674def87297 in_seq=0 out_seq=0 2024-09-11T23:00:03.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f13417fa640 10 client.17466 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T23:00:03.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f13417fa640 1 -- 172.21.15.152:0/2831144912 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab1cdc7d70 con 0x55ab1cee5300 2024-09-11T23:00:03.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f13417fa640 10 client.17466 handle_command_reply: tid=0 2024-09-11T23:00:03.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f136207a740 1 client.17466 shutdown 2024-09-11T23:00:03.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f136207a740 1 -- 172.21.15.152:0/2831144912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce5a5d0 msgr2=0x55ab1ce71a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:03.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f136207a740 1 --2- 172.21.15.152:0/2831144912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce71a70 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f13540b7b20 tx=0x7f13540045f0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f136207a740 1 -- 172.21.15.152:0/2831144912 shutdown_connections 2024-09-11T23:00:03.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f136207a740 1 --2- 172.21.15.152:0/2831144912 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55ab1cee5300 0x55ab1cee76f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f136207a740 1 --2- 172.21.15.152:0/2831144912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ab1ce69780 0x55ab1ce6e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f136207a740 1 --2- 172.21.15.152:0/2831144912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ab1ce5bb40 0x55ab1ce71fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f13417fa640 0 client.17466 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T23:00:03.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f136207a740 1 --2- 172.21.15.152:0/2831144912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ab1ce5a5d0 0x55ab1ce71a70 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f136207a740 1 -- 172.21.15.152:0/2831144912 >> 172.21.15.152:0/2831144912 conn(0x55ab1ce23310 msgr2=0x55ab1ce554e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:03.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f136207a740 1 -- 172.21.15.152:0/2831144912 shutdown_connections 2024-09-11T23:00:03.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f136207a740 1 -- 172.21.15.152:0/2831144912 wait complete. 2024-09-11T23:00:03.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.414+0000 7f136207a740 20 client.17466 trim_cache size 0 max 16384 2024-09-11T23:00:03.437 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:00:03.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.415+0000 7f135fca1640 1 -- 172.21.15.152:0/659472602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f13440a2cf0 msgr2=0x7f13440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:03.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.415+0000 7f135fca1640 1 --2- 172.21.15.152:0/659472602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f13440a2cf0 0x7f13440a51b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1350009fd0 tx=0x7f1350009da0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.415+0000 7f135fca1640 1 -- 172.21.15.152:0/659472602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13581002f0 msgr2=0x7f1358198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:03.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.415+0000 7f135fca1640 1 --2- 172.21.15.152:0/659472602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13581002f0 0x7f1358198560 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1354009230 tx=0x7f1354004270 comp rx=0 tx=0).stop 2024-09-11T23:00:03.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.415+0000 7f135fca1640 1 -- 172.21.15.152:0/659472602 shutdown_connections 2024-09-11T23:00:03.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.415+0000 7f135fca1640 1 --2- 172.21.15.152:0/659472602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f13440a2cf0 0x7f13440a51b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.415+0000 7f135fca1640 1 --2- 172.21.15.152:0/659472602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1358106f50 0x7f135819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.415+0000 7f135fca1640 1 --2- 172.21.15.152:0/659472602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1358100ca0 0x7f1358198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.415+0000 7f135fca1640 1 --2- 172.21.15.152:0/659472602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13581002f0 0x7f1358198560 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:03.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.415+0000 7f135fca1640 1 -- 172.21.15.152:0/659472602 >> 172.21.15.152:0/659472602 conn(0x7f1358077210 msgr2=0x7f13580fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:03.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.415+0000 7f135fca1640 1 -- 172.21.15.152:0/659472602 shutdown_connections 2024-09-11T23:00:03.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:03.415+0000 7f135fca1640 1 -- 172.21.15.152:0/659472602 wait complete. 2024-09-11T23:00:04.103 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:00:04.353 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:04.665 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:04 smithi152 ceph-mon[31181]: pgmap v218: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T23:00:04.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:04 smithi191 ceph-mon[35211]: pgmap v218: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T23:00:04.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:04 smithi204 ceph-mon[35454]: pgmap v218: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T23:00:04.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.870+0000 7f828c4a2640 1 -- 172.21.15.152:0/2755712247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8284100a80 msgr2=0x7f8284100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:04.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.870+0000 7f828c4a2640 1 --2- 172.21.15.152:0/2755712247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8284100a80 0x7f8284100ee0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8278009930 tx=0x7f827802a420 comp rx=0 tx=0).stop 2024-09-11T23:00:04.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.870+0000 7f828c4a2640 1 -- 172.21.15.152:0/2755712247 shutdown_connections 2024-09-11T23:00:04.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.870+0000 7f828c4a2640 1 --2- 172.21.15.152:0/2755712247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8284101420 0x7f828410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.870+0000 7f828c4a2640 1 --2- 172.21.15.152:0/2755712247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8284100a80 0x7f8284100ee0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.870+0000 7f828c4a2640 1 --2- 172.21.15.152:0/2755712247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8284106a80 0x7f8284106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.870+0000 7f828c4a2640 1 -- 172.21.15.152:0/2755712247 >> 172.21.15.152:0/2755712247 conn(0x7f82840fc7a0 msgr2=0x7f82840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:04.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.870+0000 7f828c4a2640 1 -- 172.21.15.152:0/2755712247 shutdown_connections 2024-09-11T23:00:04.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.870+0000 7f828c4a2640 1 -- 172.21.15.152:0/2755712247 wait complete. 2024-09-11T23:00:04.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.871+0000 7f828c4a2640 1 Processor -- start 2024-09-11T23:00:04.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.871+0000 7f828c4a2640 1 -- start start 2024-09-11T23:00:04.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.871+0000 7f828c4a2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8284100a80 0x7f82841973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.871+0000 7f828c4a2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8284101420 0x7f8284197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.871+0000 7f828c4a2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8284106a80 0x7f828419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.871+0000 7f828c4a2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8284110e80 con 0x7f8284106a80 2024-09-11T23:00:04.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.871+0000 7f828c4a2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8284110d00 con 0x7f8284100a80 2024-09-11T23:00:04.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.871+0000 7f828c4a2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8284111000 con 0x7f8284101420 2024-09-11T23:00:04.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.871+0000 7f828aa3f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8284106a80 0x7f828419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.871+0000 7f828aa3f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8284106a80 0x7f828419bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56502/0 (socket says 172.21.15.152:56502) 2024-09-11T23:00:04.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.871+0000 7f828aa3f640 1 -- 172.21.15.152:0/3737626325 learned_addr learned my addr 172.21.15.152:0/3737626325 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:04.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.872+0000 7f828a23e640 1 --2- 172.21.15.152:0/3737626325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8284100a80 0x7f82841973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.872+0000 7f8289a3d640 1 --2- 172.21.15.152:0/3737626325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8284101420 0x7f8284197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.872+0000 7f828aa3f640 1 -- 172.21.15.152:0/3737626325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8284101420 msgr2=0x7f8284197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:04.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.872+0000 7f828aa3f640 1 --2- 172.21.15.152:0/3737626325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8284101420 0x7f8284197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.872+0000 7f828aa3f640 1 -- 172.21.15.152:0/3737626325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8284100a80 msgr2=0x7f82841973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:04.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.872+0000 7f828aa3f640 1 --2- 172.21.15.152:0/3737626325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8284100a80 0x7f82841973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.872+0000 7f828aa3f640 1 -- 172.21.15.152:0/3737626325 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828419c370 con 0x7f8284106a80 2024-09-11T23:00:04.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.872+0000 7f828aa3f640 1 --2- 172.21.15.152:0/3737626325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8284106a80 0x7f828419bc90 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f827c009fd0 tx=0x7f827c007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:04.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.872+0000 7f82777fe640 1 -- 172.21.15.152:0/3737626325 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c00de10 con 0x7f8284106a80 2024-09-11T23:00:04.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.872+0000 7f82777fe640 1 -- 172.21.15.152:0/3737626325 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f827c003e50 con 0x7f8284106a80 2024-09-11T23:00:04.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.872+0000 7f82777fe640 1 -- 172.21.15.152:0/3737626325 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c018420 con 0x7f8284106a80 2024-09-11T23:00:04.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.872+0000 7f828c4a2640 1 -- 172.21.15.152:0/3737626325 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f828419c660 con 0x7f8284106a80 2024-09-11T23:00:04.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.872+0000 7f828c4a2640 1 -- 172.21.15.152:0/3737626325 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f828406a790 con 0x7f8284106a80 2024-09-11T23:00:04.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.875+0000 7f82777fe640 1 -- 172.21.15.152:0/3737626325 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f827c003970 con 0x7f8284106a80 2024-09-11T23:00:04.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.875+0000 7f828e87b740 1 Processor -- start 2024-09-11T23:00:04.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.875+0000 7f828e87b740 1 -- start start 2024-09-11T23:00:04.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f828e87b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9aeb40 0x555efe9ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f828e87b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9ad5d0 0x555efe9ada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f828e87b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9b8370 0x555efe9ba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f828e87b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x555efe9132f0 con 0x555efe9b8370 2024-09-11T23:00:04.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f828e87b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555efe8bacf0 con 0x555efe9ad5d0 2024-09-11T23:00:04.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f828e87b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555efe90f2b0 con 0x555efe9aeb40 2024-09-11T23:00:04.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f82777fe640 1 --2- 172.21.15.152:0/3737626325 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f82700a2d70 0x7f82700a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f8289a3d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9aeb40 0x555efe9ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f8289a3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9b8370 0x555efe9ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f828a23e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9ad5d0 0x555efe9ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f8289a3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9b8370 0x555efe9ba760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56512/0 (socket says 172.21.15.152:56512) 2024-09-11T23:00:04.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f828a23e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9ad5d0 0x555efe9ada10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:40826/0 (socket says 172.21.15.152:40826) 2024-09-11T23:00:04.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f8289a3d640 1 -- 172.21.15.152:0/2393199348 learned_addr learned my addr 172.21.15.152:0/2393199348 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:04.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f828aa3f640 1 --2- 172.21.15.152:0/3737626325 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f82700a2d70 0x7f82700a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f82777fe640 1 -- 172.21.15.152:0/3737626325 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f827c0d0ba0 con 0x7f8284106a80 2024-09-11T23:00:04.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f828a23e640 1 -- 172.21.15.152:0/2393199348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9aeb40 msgr2=0x555efe9ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:04.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f828a23e640 1 --2- 172.21.15.152:0/2393199348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9aeb40 0x555efe9ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f828a23e640 1 -- 172.21.15.152:0/2393199348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9b8370 msgr2=0x555efe9ba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:04.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f828a23e640 1 --2- 172.21.15.152:0/2393199348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9b8370 0x555efe9ba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f828a23e640 1 -- 172.21.15.152:0/2393199348 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555efe89f3a0 con 0x555efe9ad5d0 2024-09-11T23:00:04.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f8289a3d640 1 --2- 172.21.15.152:0/2393199348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9b8370 0x555efe9ba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:00:04.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f828a23e640 1 --2- 172.21.15.152:0/2393199348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9ad5d0 0x555efe9ada10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f828000bb60 tx=0x7f828000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d122e56b4f8499f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:04.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.876+0000 7f828aa3f640 1 --2- 172.21.15.152:0/3737626325 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f82700a2d70 0x7f82700a5230 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f827c09ce10 tx=0x7f827c06f0d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:04.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.877+0000 7f8274ff9640 1 -- 172.21.15.152:0/2393199348 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280015070 con 0x555efe9ad5d0 2024-09-11T23:00:04.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.877+0000 7f8274ff9640 1 -- 172.21.15.152:0/2393199348 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f828000be50 con 0x555efe9ad5d0 2024-09-11T23:00:04.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.877+0000 7f8274ff9640 1 -- 172.21.15.152:0/2393199348 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8280011440 con 0x555efe9ad5d0 2024-09-11T23:00:04.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.877+0000 7f828e87b740 1 -- 172.21.15.152:0/2393199348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9ad5d0 msgr2=0x555efe9ada10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:04.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.877+0000 7f828e87b740 1 --2- 172.21.15.152:0/2393199348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9ad5d0 0x555efe9ada10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f828000bb60 tx=0x7f828000f730 comp rx=0 tx=0).stop 2024-09-11T23:00:04.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.877+0000 7f828e87b740 1 -- 172.21.15.152:0/2393199348 shutdown_connections 2024-09-11T23:00:04.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.877+0000 7f828e87b740 1 --2- 172.21.15.152:0/2393199348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9b8370 0x555efe9ba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.877+0000 7f828e87b740 1 --2- 172.21.15.152:0/2393199348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9ad5d0 0x555efe9ada10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.877+0000 7f828e87b740 1 --2- 172.21.15.152:0/2393199348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9aeb40 0x555efe9ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.877+0000 7f828e87b740 1 -- 172.21.15.152:0/2393199348 >> 172.21.15.152:0/2393199348 conn(0x555efe976310 msgr2=0x555efe9a9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:04.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.878+0000 7f828e87b740 1 -- 172.21.15.152:0/2393199348 shutdown_connections 2024-09-11T23:00:04.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.878+0000 7f828e87b740 1 -- 172.21.15.152:0/2393199348 wait complete. 2024-09-11T23:00:04.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.878+0000 7f828e87b740 1 -- start start 2024-09-11T23:00:04.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.878+0000 7f828e87b740 1 Processor -- start 2024-09-11T23:00:04.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.878+0000 7f828e87b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9ad5d0 0x555efe9b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.878+0000 7f828e87b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9aeb40 0x555efe9b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.878+0000 7f828e87b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9b8370 0x555efe9b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.878+0000 7f828e87b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x555efe9bb070 con 0x555efe9b8370 2024-09-11T23:00:04.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.878+0000 7f828e87b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555efe8ce060 con 0x555efe9ad5d0 2024-09-11T23:00:04.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.878+0000 7f828e87b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555efe9bb1f0 con 0x555efe9aeb40 2024-09-11T23:00:04.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.878+0000 7f828e87b740 10 client.0 _ll_register_callbacks cb 0x555efe6218c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:04.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.878+0000 7f8289a3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9b8370 0x555efe9b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.878+0000 7f8289a3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9b8370 0x555efe9b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56518/0 (socket says 172.21.15.152:56518) 2024-09-11T23:00:04.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f8289a3d640 1 -- 172.21.15.152:0/1812495405 learned_addr learned my addr 172.21.15.152:0/1812495405 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:04.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f8289a3d640 1 --2- 172.21.15.152:0/1812495405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9ad5d0 0x555efe9b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f828a23e640 1 --2- 172.21.15.152:0/1812495405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9aeb40 0x555efe9b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f8289a3d640 1 -- 172.21.15.152:0/1812495405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9aeb40 msgr2=0x555efe9b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:04.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f8289a3d640 1 --2- 172.21.15.152:0/1812495405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9aeb40 0x555efe9b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f8289a3d640 1 -- 172.21.15.152:0/1812495405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9ad5d0 msgr2=0x555efe9b7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:04.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f8289a3d640 1 --2- 172.21.15.152:0/1812495405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9ad5d0 0x555efe9b7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f8289a3d640 1 -- 172.21.15.152:0/1812495405 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555efe6843c0 con 0x555efe9b8370 2024-09-11T23:00:04.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f8289a3d640 1 --2- 172.21.15.152:0/1812495405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9b8370 0x555efe9b55d0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8278009900 tx=0x7f8278009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca25a77eb39b85e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:04.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f826dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:04.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f826dffb640 1 -- 172.21.15.152:0/1812495405 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278035040 con 0x555efe9b8370 2024-09-11T23:00:04.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f828e87b740 1 -- 172.21.15.152:0/1812495405 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f828000be50 con 0x555efe9b8370 2024-09-11T23:00:04.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f826dffb640 1 -- 172.21.15.152:0/1812495405 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f8278004260 con 0x555efe9b8370 2024-09-11T23:00:04.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f826dffb640 1 -- 172.21.15.152:0/1812495405 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8278051050 con 0x555efe9b8370 2024-09-11T23:00:04.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f828e87b740 10 client.17478 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:04.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f828e87b740 1 -- 172.21.15.152:0/1812495405 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555efe9c6b20 con 0x555efe9b8370 2024-09-11T23:00:04.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.879+0000 7f826dffb640 1 -- 172.21.15.152:0/1812495405 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f828000be50 con 0x555efe9b8370 2024-09-11T23:00:04.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.880+0000 7f826dffb640 1 -- 172.21.15.152:0/1812495405 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f827803d5a0 con 0x555efe9b8370 2024-09-11T23:00:04.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.880+0000 7f828e87b740 10 client.17478 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:04.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.880+0000 7f828e87b740 10 client.17478 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:00:04.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.880+0000 7f828e87b740 20 client.17478 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:04.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.880+0000 7f828e87b740 1 --2- 172.21.15.152:0/1812495405 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x555efe9c7fe0 0x555efe9ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.880+0000 7f828e87b740 4 client.17478 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:00:04.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.880+0000 7f828e87b740 1 -- 172.21.15.152:0/1812495405 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555efe920860 con 0x555efe9c7fe0 2024-09-11T23:00:04.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.880+0000 7f828a23e640 1 --2- 172.21.15.152:0/1812495405 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x555efe9c7fe0 0x555efe9ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.880+0000 7f828a23e640 1 --2- 172.21.15.152:0/1812495405 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x555efe9c7fe0 0x555efe9ca3d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e98882f85c85f53 server_cookie=627cfa6fec491fff in_seq=0 out_seq=0 2024-09-11T23:00:04.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.880+0000 7f826dffb640 10 client.17478 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T23:00:04.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.881+0000 7f826dffb640 1 -- 172.21.15.152:0/1812495405 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555efe920860 con 0x555efe9c7fe0 2024-09-11T23:00:04.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.881+0000 7f826dffb640 10 client.17478 handle_command_reply: tid=0 2024-09-11T23:00:04.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.881+0000 7f828e87b740 1 client.17478 shutdown 2024-09-11T23:00:04.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.881+0000 7f828e87b740 1 -- 172.21.15.152:0/1812495405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9b8370 msgr2=0x555efe9b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:04.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.881+0000 7f828e87b740 1 --2- 172.21.15.152:0/1812495405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9b8370 0x555efe9b55d0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8278009900 tx=0x7f8278009660 comp rx=0 tx=0).stop 2024-09-11T23:00:04.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.881+0000 7f828e87b740 1 -- 172.21.15.152:0/1812495405 shutdown_connections 2024-09-11T23:00:04.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.881+0000 7f828e87b740 1 --2- 172.21.15.152:0/1812495405 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x555efe9c7fe0 0x555efe9ca3d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.881+0000 7f828e87b740 1 --2- 172.21.15.152:0/1812495405 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9b8370 0x555efe9b55d0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.881+0000 7f828e87b740 1 --2- 172.21.15.152:0/1812495405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9aeb40 0x555efe9b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.881+0000 7f828e87b740 1 --2- 172.21.15.152:0/1812495405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9ad5d0 0x555efe9b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.881+0000 7f828e87b740 1 -- 172.21.15.152:0/1812495405 >> 172.21.15.152:0/1812495405 conn(0x555efe976310 msgr2=0x555efe9af960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:04.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.881+0000 7f826dffb640 0 client.17478 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T23:00:04.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.881+0000 7f828e87b740 1 -- 172.21.15.152:0/1812495405 shutdown_connections 2024-09-11T23:00:04.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.881+0000 7f828e87b740 1 -- 172.21.15.152:0/1812495405 wait complete. 2024-09-11T23:00:04.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.881+0000 7f828e87b740 20 client.17478 trim_cache size 0 max 16384 2024-09-11T23:00:04.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.894+0000 7f828e87b740 1 Processor -- start 2024-09-11T23:00:04.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.894+0000 7f828e87b740 1 -- start start 2024-09-11T23:00:04.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.894+0000 7f828e87b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9ad5d0 0x555efe9a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.894+0000 7f828e87b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9aeb40 0x555efe9c4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.894+0000 7f828e87b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9b8370 0x555efe9b0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.894+0000 7f828e87b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f82841ab590 con 0x555efe9ad5d0 2024-09-11T23:00:04.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.894+0000 7f828e87b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f82841ad0d0 con 0x555efe9b8370 2024-09-11T23:00:04.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.894+0000 7f828e87b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82841abc60 con 0x555efe9aeb40 2024-09-11T23:00:04.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.894+0000 7f8289a3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9ad5d0 0x555efe9a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.894+0000 7f8289a3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9ad5d0 0x555efe9a9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56522/0 (socket says 172.21.15.152:56522) 2024-09-11T23:00:04.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.894+0000 7f8289a3d640 1 -- 172.21.15.152:0/1918922088 learned_addr learned my addr 172.21.15.152:0/1918922088 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:04.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.895+0000 7f8289a3d640 1 --2- 172.21.15.152:0/1918922088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9b8370 0x555efe9b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.895+0000 7f828a23e640 1 --2- 172.21.15.152:0/1918922088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9aeb40 0x555efe9c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.895+0000 7f8289a3d640 1 -- 172.21.15.152:0/1918922088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9aeb40 msgr2=0x555efe9c4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:04.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.895+0000 7f8289a3d640 1 --2- 172.21.15.152:0/1918922088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9aeb40 0x555efe9c4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.895+0000 7f8289a3d640 1 -- 172.21.15.152:0/1918922088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9b8370 msgr2=0x555efe9b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.895+0000 7f8289a3d640 1 --2- 172.21.15.152:0/1918922088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9b8370 0x555efe9b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.895+0000 7f8289a3d640 1 -- 172.21.15.152:0/1918922088 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555efe9baf50 con 0x555efe9ad5d0 2024-09-11T23:00:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.895+0000 7f8289a3d640 1 --2- 172.21.15.152:0/1918922088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9ad5d0 0x555efe9a9e10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f82780057c0 tx=0x7f82780318f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731c57a8470e4e87 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.895+0000 7f826ffff640 1 -- 172.21.15.152:0/1918922088 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278035070 con 0x555efe9ad5d0 2024-09-11T23:00:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.895+0000 7f826ffff640 1 -- 172.21.15.152:0/1918922088 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8278004460 con 0x555efe9ad5d0 2024-09-11T23:00:04.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.895+0000 7f826ffff640 1 -- 172.21.15.152:0/1918922088 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278003d50 con 0x555efe9ad5d0 2024-09-11T23:00:04.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 -- 172.21.15.152:0/1918922088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9ad5d0 msgr2=0x555efe9a9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:04.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 --2- 172.21.15.152:0/1918922088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9ad5d0 0x555efe9a9e10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f82780057c0 tx=0x7f82780318f0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 -- 172.21.15.152:0/1918922088 shutdown_connections 2024-09-11T23:00:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 --2- 172.21.15.152:0/1918922088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9b8370 0x555efe9b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 --2- 172.21.15.152:0/1918922088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9aeb40 0x555efe9c4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 --2- 172.21.15.152:0/1918922088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9ad5d0 0x555efe9a9e10 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 -- 172.21.15.152:0/1918922088 >> 172.21.15.152:0/1918922088 conn(0x555efe976310 msgr2=0x555efe9a9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 -- 172.21.15.152:0/1918922088 shutdown_connections 2024-09-11T23:00:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 -- 172.21.15.152:0/1918922088 wait complete. 2024-09-11T23:00:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 -- start start 2024-09-11T23:00:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 Processor -- start 2024-09-11T23:00:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9ad5d0 0x555efe9c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9aeb40 0x555efe9b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9b8370 0x555efe9b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f82841aca00 con 0x555efe9ad5d0 2024-09-11T23:00:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f82841ac330 con 0x555efe9aeb40 2024-09-11T23:00:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f82841a9550 con 0x555efe9b8370 2024-09-11T23:00:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f828e87b740 10 client.0 _ll_register_callbacks cb 0x7f8278031840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.896+0000 7f8289a3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9ad5d0 0x555efe9c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.897+0000 7f8289a3d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9ad5d0 0x555efe9c0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56524/0 (socket says 172.21.15.152:56524) 2024-09-11T23:00:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.897+0000 7f8289a3d640 1 -- 172.21.15.152:0/2226455149 learned_addr learned my addr 172.21.15.152:0/2226455149 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.897+0000 7f828a23e640 1 --2- 172.21.15.152:0/2226455149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9aeb40 0x555efe9b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.897+0000 7f8289a3d640 1 -- 172.21.15.152:0/2226455149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9b8370 msgr2=0x555efe9b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.897+0000 7f8289a3d640 1 --2- 172.21.15.152:0/2226455149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9b8370 0x555efe9b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.897+0000 7f8289a3d640 1 -- 172.21.15.152:0/2226455149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9aeb40 msgr2=0x555efe9b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.897+0000 7f8289a3d640 1 --2- 172.21.15.152:0/2226455149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9aeb40 0x555efe9b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.897+0000 7f8289a3d640 1 -- 172.21.15.152:0/2226455149 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555efe9b2bd0 con 0x555efe9ad5d0 2024-09-11T23:00:04.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.897+0000 7f8289a3d640 1 --2- 172.21.15.152:0/2226455149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9ad5d0 0x555efe9c0c50 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f827803d8f0 tx=0x7f827802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0278d27c8e6892 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:04.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.897+0000 7f826dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:04.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.897+0000 7f826dffb640 1 -- 172.21.15.152:0/2226455149 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278035070 con 0x555efe9ad5d0 2024-09-11T23:00:04.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.897+0000 7f826dffb640 1 -- 172.21.15.152:0/2226455149 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7f82780035f0 con 0x555efe9ad5d0 2024-09-11T23:00:04.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.897+0000 7f828e87b740 1 -- 172.21.15.152:0/2226455149 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8278004460 con 0x555efe9ad5d0 2024-09-11T23:00:04.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.897+0000 7f826dffb640 1 -- 172.21.15.152:0/2226455149 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f827803d480 con 0x555efe9ad5d0 2024-09-11T23:00:04.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.897+0000 7f828e87b740 10 client.17484 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:04.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.897+0000 7f828e87b740 1 -- 172.21.15.152:0/2226455149 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555efea37110 con 0x555efe9ad5d0 2024-09-11T23:00:04.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.898+0000 7f826dffb640 1 -- 172.21.15.152:0/2226455149 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8278004460 con 0x555efe9ad5d0 2024-09-11T23:00:04.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.898+0000 7f826dffb640 1 -- 172.21.15.152:0/2226455149 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8278068730 con 0x555efe9ad5d0 2024-09-11T23:00:04.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.898+0000 7f828e87b740 10 client.17484 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:04.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.898+0000 7f828e87b740 10 client.17484 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:00:04.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.898+0000 7f828e87b740 20 client.17484 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.898+0000 7f828e87b740 1 --2- 172.21.15.152:0/2226455149 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x555efea376c0 0x555efea39ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.898+0000 7f828e87b740 4 client.17484 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:00:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.898+0000 7f828e87b740 1 -- 172.21.15.152:0/2226455149 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555efe91ad70 con 0x555efea376c0 2024-09-11T23:00:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.898+0000 7f828a23e640 1 --2- 172.21.15.152:0/2226455149 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x555efea376c0 0x555efea39ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f826dffb640 10 client.17484 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T23:00:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f828a23e640 1 --2- 172.21.15.152:0/2226455149 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x555efea376c0 0x555efea39ae0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68acca3cad711e9a server_cookie=461df2229790f3fe in_seq=0 out_seq=0 2024-09-11T23:00:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f826dffb640 1 -- 172.21.15.152:0/2226455149 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555efe91ad70 con 0x555efea376c0 2024-09-11T23:00:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f826dffb640 10 client.17484 handle_command_reply: tid=0 2024-09-11T23:00:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f828e87b740 1 client.17484 shutdown 2024-09-11T23:00:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f828e87b740 1 -- 172.21.15.152:0/2226455149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9ad5d0 msgr2=0x555efe9c0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f828e87b740 1 --2- 172.21.15.152:0/2226455149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9ad5d0 0x555efe9c0c50 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f827803d8f0 tx=0x7f827802a420 comp rx=0 tx=0).stop 2024-09-11T23:00:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f828e87b740 1 -- 172.21.15.152:0/2226455149 shutdown_connections 2024-09-11T23:00:04.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f828e87b740 1 --2- 172.21.15.152:0/2226455149 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x555efea376c0 0x555efea39ae0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f828e87b740 1 --2- 172.21.15.152:0/2226455149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555efe9b8370 0x555efe9b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f828e87b740 1 --2- 172.21.15.152:0/2226455149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555efe9aeb40 0x555efe9b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f828e87b740 1 --2- 172.21.15.152:0/2226455149 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555efe9ad5d0 0x555efe9c0c50 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f828e87b740 1 -- 172.21.15.152:0/2226455149 >> 172.21.15.152:0/2226455149 conn(0x555efe976310 msgr2=0x555efe9b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:04.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f826dffb640 0 client.17484 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T23:00:04.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f828e87b740 1 -- 172.21.15.152:0/2226455149 shutdown_connections 2024-09-11T23:00:04.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f828e87b740 1 -- 172.21.15.152:0/2226455149 wait complete. 2024-09-11T23:00:04.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.899+0000 7f828e87b740 20 client.17484 trim_cache size 0 max 16384 2024-09-11T23:00:04.905 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:00:04.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.900+0000 7f828c4a2640 1 -- 172.21.15.152:0/3737626325 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f82700a2d70 msgr2=0x7f82700a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:04.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.900+0000 7f828c4a2640 1 --2- 172.21.15.152:0/3737626325 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f82700a2d70 0x7f82700a5230 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f827c09ce10 tx=0x7f827c06f0d0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.900+0000 7f828c4a2640 1 -- 172.21.15.152:0/3737626325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8284106a80 msgr2=0x7f828419bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:04.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.900+0000 7f828c4a2640 1 --2- 172.21.15.152:0/3737626325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8284106a80 0x7f828419bc90 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f827c009fd0 tx=0x7f827c007590 comp rx=0 tx=0).stop 2024-09-11T23:00:04.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.900+0000 7f828c4a2640 1 -- 172.21.15.152:0/3737626325 shutdown_connections 2024-09-11T23:00:04.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.900+0000 7f828c4a2640 1 --2- 172.21.15.152:0/3737626325 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f82700a2d70 0x7f82700a5230 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.900+0000 7f828c4a2640 1 --2- 172.21.15.152:0/3737626325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8284106a80 0x7f828419bc90 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.900+0000 7f828c4a2640 1 --2- 172.21.15.152:0/3737626325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8284101420 0x7f8284197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.900+0000 7f828c4a2640 1 --2- 172.21.15.152:0/3737626325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8284100a80 0x7f82841973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:04.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.900+0000 7f828c4a2640 1 -- 172.21.15.152:0/3737626325 >> 172.21.15.152:0/3737626325 conn(0x7f82840fc7a0 msgr2=0x7f82840fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:04.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.900+0000 7f828c4a2640 1 -- 172.21.15.152:0/3737626325 shutdown_connections 2024-09-11T23:00:04.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:04.900+0000 7f828c4a2640 1 -- 172.21.15.152:0/3737626325 wait complete. 2024-09-11T23:00:05.642 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:00:05.891 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:06.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.401+0000 7fc5a1299640 1 -- 172.21.15.152:0/1251855798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc59c10c190 msgr2=0x7fc59c10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:06.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.401+0000 7fc5a1299640 1 --2- 172.21.15.152:0/1251855798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc59c10c190 0x7fc59c10e600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc594009be0 tx=0x7fc59402a750 comp rx=0 tx=0).stop 2024-09-11T23:00:06.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.401+0000 7fc5a1299640 1 -- 172.21.15.152:0/1251855798 shutdown_connections 2024-09-11T23:00:06.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.401+0000 7fc5a1299640 1 --2- 172.21.15.152:0/1251855798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc59c10c190 0x7fc59c10e600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.401+0000 7fc5a1299640 1 --2- 172.21.15.152:0/1251855798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc59c1036d0 0x7fc59c10bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.401+0000 7fc5a1299640 1 --2- 172.21.15.152:0/1251855798 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc59c102db0 0x7fc59c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.402+0000 7fc5a1299640 1 -- 172.21.15.152:0/1251855798 >> 172.21.15.152:0/1251855798 conn(0x7fc59c0fc790 msgr2=0x7fc59c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:06.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.402+0000 7fc5a1299640 1 -- 172.21.15.152:0/1251855798 shutdown_connections 2024-09-11T23:00:06.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.402+0000 7fc5a1299640 1 -- 172.21.15.152:0/1251855798 wait complete. 2024-09-11T23:00:06.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.402+0000 7fc5a1299640 1 Processor -- start 2024-09-11T23:00:06.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.402+0000 7fc5a1299640 1 -- start start 2024-09-11T23:00:06.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.402+0000 7fc5a1299640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc59c102db0 0x7fc59c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc5a1299640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc59c1036d0 0x7fc59c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc5a1299640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc59c10c190 0x7fc59c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc5a1299640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c111f10 con 0x7fc59c102db0 2024-09-11T23:00:06.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc5a1299640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c111d90 con 0x7fc59c1036d0 2024-09-11T23:00:06.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc5a1299640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c112090 con 0x7fc59c10c190 2024-09-11T23:00:06.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc59affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc59c102db0 0x7fc59c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc59affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc59c102db0 0x7fc59c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56530/0 (socket says 172.21.15.152:56530) 2024-09-11T23:00:06.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc59affd640 1 -- 172.21.15.152:0/2171183122 learned_addr learned my addr 172.21.15.152:0/2171183122 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:06.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc59a7fc640 1 --2- 172.21.15.152:0/2171183122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc59c1036d0 0x7fc59c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc59b7fe640 1 --2- 172.21.15.152:0/2171183122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc59c10c190 0x7fc59c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc59affd640 1 -- 172.21.15.152:0/2171183122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc59c10c190 msgr2=0x7fc59c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:06.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc59affd640 1 --2- 172.21.15.152:0/2171183122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc59c10c190 0x7fc59c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc59affd640 1 -- 172.21.15.152:0/2171183122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc59c1036d0 msgr2=0x7fc59c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:06.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc59affd640 1 --2- 172.21.15.152:0/2171183122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc59c1036d0 0x7fc59c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc59affd640 1 -- 172.21.15.152:0/2171183122 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59c19d3a0 con 0x7fc59c102db0 2024-09-11T23:00:06.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc59a7fc640 1 --2- 172.21.15.152:0/2171183122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc59c1036d0 0x7fc59c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:00:06.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc59b7fe640 1 --2- 172.21.15.152:0/2171183122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc59c10c190 0x7fc59c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:06.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc59affd640 1 --2- 172.21.15.152:0/2171183122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc59c102db0 0x7fc59c1983f0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00e9c0 tx=0x7fc58c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:06.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc583fff640 1 -- 172.21.15.152:0/2171183122 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c00cde0 con 0x7fc59c102db0 2024-09-11T23:00:06.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc5a1299640 1 -- 172.21.15.152:0/2171183122 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc59c19d690 con 0x7fc59c102db0 2024-09-11T23:00:06.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc583fff640 1 -- 172.21.15.152:0/2171183122 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc58c0035f0 con 0x7fc59c102db0 2024-09-11T23:00:06.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc583fff640 1 -- 172.21.15.152:0/2171183122 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c010650 con 0x7fc59c102db0 2024-09-11T23:00:06.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.403+0000 7fc5a1299640 1 -- 172.21.15.152:0/2171183122 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc59c1a4f70 con 0x7fc59c102db0 2024-09-11T23:00:06.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.406+0000 7fc583fff640 1 -- 172.21.15.152:0/2171183122 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc58c0037a0 con 0x7fc59c102db0 2024-09-11T23:00:06.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc5a3672740 1 Processor -- start 2024-09-11T23:00:06.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc5a3672740 1 -- start start 2024-09-11T23:00:06.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc5a3672740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8db40 0x55d7feb8c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc5a3672740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb8ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc5a3672740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb9b780 0x55d7feb9db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc5a3672740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d7feaf22f0 con 0x55d7feb8c5d0 2024-09-11T23:00:06.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc5a3672740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d7fea99cf0 con 0x55d7feb9b780 2024-09-11T23:00:06.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc5a3672740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d7feaee2b0 con 0x55d7feb8db40 2024-09-11T23:00:06.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc59affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb8ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc59b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8db40 0x55d7feb8c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc59affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb8ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56544/0 (socket says 172.21.15.152:56544) 2024-09-11T23:00:06.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc59affd640 1 -- 172.21.15.152:0/2590681667 learned_addr learned my addr 172.21.15.152:0/2590681667 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:06.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc583fff640 1 --2- 172.21.15.152:0/2171183122 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc5840a2cf0 0x7fc5840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc59a7fc640 1 --2- 172.21.15.152:0/2590681667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb9b780 0x55d7feb9db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc59b7fe640 1 --2- 172.21.15.152:0/2171183122 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc5840a2cf0 0x7fc5840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc59a7fc640 1 -- 172.21.15.152:0/2590681667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8db40 msgr2=0x55d7feb8c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:06.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc59a7fc640 1 --2- 172.21.15.152:0/2590681667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8db40 0x55d7feb8c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc583fff640 1 -- 172.21.15.152:0/2171183122 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc58c0cf7d0 con 0x7fc59c102db0 2024-09-11T23:00:06.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc59a7fc640 1 -- 172.21.15.152:0/2590681667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 msgr2=0x55d7feb8ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:06.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc59a7fc640 1 --2- 172.21.15.152:0/2590681667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb8ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc59a7fc640 1 -- 172.21.15.152:0/2590681667 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7fea7e3a0 con 0x55d7feb9b780 2024-09-11T23:00:06.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.407+0000 7fc59affd640 1 --2- 172.21.15.152:0/2590681667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb8ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:00:06.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.408+0000 7fc59a7fc640 1 --2- 172.21.15.152:0/2590681667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb9b780 0x55d7feb9db70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc590009f20 tx=0x7fc5900118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f27affb00868c3bc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:06.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.408+0000 7fc59b7fe640 1 --2- 172.21.15.152:0/2171183122 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc5840a2cf0 0x7fc5840a51b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc594005cc0 tx=0x7fc59402a750 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:06.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.408+0000 7fc5817fa640 1 -- 172.21.15.152:0/2590681667 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc590004020 con 0x55d7feb9b780 2024-09-11T23:00:06.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.408+0000 7fc5817fa640 1 -- 172.21.15.152:0/2590681667 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc590005500 con 0x55d7feb9b780 2024-09-11T23:00:06.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.408+0000 7fc5817fa640 1 -- 172.21.15.152:0/2590681667 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc590004db0 con 0x55d7feb9b780 2024-09-11T23:00:06.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 -- 172.21.15.152:0/2590681667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb9b780 msgr2=0x55d7feb9db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:06.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 --2- 172.21.15.152:0/2590681667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb9b780 0x55d7feb9db70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc590009f20 tx=0x7fc5900118d0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 -- 172.21.15.152:0/2590681667 shutdown_connections 2024-09-11T23:00:06.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 --2- 172.21.15.152:0/2590681667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb9b780 0x55d7feb9db70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 --2- 172.21.15.152:0/2590681667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb8ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 --2- 172.21.15.152:0/2590681667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8db40 0x55d7feb8c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 -- 172.21.15.152:0/2590681667 >> 172.21.15.152:0/2590681667 conn(0x55d7feb55310 msgr2=0x55d7feb88c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:06.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 -- 172.21.15.152:0/2590681667 shutdown_connections 2024-09-11T23:00:06.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 -- 172.21.15.152:0/2590681667 wait complete. 2024-09-11T23:00:06.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 -- start start 2024-09-11T23:00:06.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 Processor -- start 2024-09-11T23:00:06.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8c5d0 0x55d7feb9b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8db40 0x55d7feb98350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb9b780 0x55d7feb98890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d7feb9e480 con 0x55d7feb8db40 2024-09-11T23:00:06.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d7feaad060 con 0x55d7feb9b780 2024-09-11T23:00:06.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.409+0000 7fc5a3672740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d7feb9e600 con 0x55d7feb8c5d0 2024-09-11T23:00:06.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc5a3672740 10 client.0 _ll_register_callbacks cb 0x55d7fe8008c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:06.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc59a7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8db40 0x55d7feb98350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc59a7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8db40 0x55d7feb98350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56554/0 (socket says 172.21.15.152:56554) 2024-09-11T23:00:06.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc59a7fc640 1 -- 172.21.15.152:0/1171108523 learned_addr learned my addr 172.21.15.152:0/1171108523 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:06.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc59b7fe640 1 --2- 172.21.15.152:0/1171108523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb9b780 0x55d7feb98890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc59affd640 1 --2- 172.21.15.152:0/1171108523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8c5d0 0x55d7feb9b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc59a7fc640 1 -- 172.21.15.152:0/1171108523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8c5d0 msgr2=0x55d7feb9b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:06.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc59a7fc640 1 --2- 172.21.15.152:0/1171108523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8c5d0 0x55d7feb9b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc59a7fc640 1 -- 172.21.15.152:0/1171108523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb9b780 msgr2=0x55d7feb98890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:06.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc59a7fc640 1 --2- 172.21.15.152:0/1171108523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb9b780 0x55d7feb98890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc59a7fc640 1 -- 172.21.15.152:0/1171108523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7fe8633c0 con 0x55d7feb8db40 2024-09-11T23:00:06.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc59a7fc640 1 --2- 172.21.15.152:0/1171108523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8db40 0x55d7feb98350 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fc590004510 tx=0x7fc5900051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ff1e5b3449aa3e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:06.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc57eff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:06.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc57eff5640 1 -- 172.21.15.152:0/1171108523 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc590028020 con 0x55d7feb8db40 2024-09-11T23:00:06.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc57eff5640 1 -- 172.21.15.152:0/1171108523 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc59000f540 con 0x55d7feb8db40 2024-09-11T23:00:06.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc5a3672740 1 -- 172.21.15.152:0/1171108523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc590005500 con 0x55d7feb8db40 2024-09-11T23:00:06.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc57eff5640 1 -- 172.21.15.152:0/1171108523 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc590033050 con 0x55d7feb8db40 2024-09-11T23:00:06.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc5a3672740 10 client.17496 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:06.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.410+0000 7fc5a3672740 1 -- 172.21.15.152:0/1171108523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7feba9e70 con 0x55d7feb8db40 2024-09-11T23:00:06.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.411+0000 7fc57eff5640 1 -- 172.21.15.152:0/1171108523 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc590005500 con 0x55d7feb8db40 2024-09-11T23:00:06.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.411+0000 7fc57eff5640 1 -- 172.21.15.152:0/1171108523 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc59000fd20 con 0x55d7feb8db40 2024-09-11T23:00:06.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.411+0000 7fc5a3672740 10 client.17496 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:06.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.411+0000 7fc5a3672740 10 client.17496 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:00:06.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.411+0000 7fc5a3672740 20 client.17496 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:06.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.411+0000 7fc5a3672740 1 --2- 172.21.15.152:0/1171108523 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55d7febab2d0 0x55d7febad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.411+0000 7fc5a3672740 4 client.17496 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:00:06.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.411+0000 7fc5a3672740 1 -- 172.21.15.152:0/1171108523 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7feaff860 con 0x55d7febab2d0 2024-09-11T23:00:06.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.411+0000 7fc59affd640 1 --2- 172.21.15.152:0/1171108523 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55d7febab2d0 0x55d7febad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.411+0000 7fc57eff5640 10 client.17496 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T23:00:06.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.411+0000 7fc59affd640 1 --2- 172.21.15.152:0/1171108523 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55d7febab2d0 0x55d7febad6c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b5bb323152eb98b server_cookie=e11ca752bbeb4e64 in_seq=0 out_seq=0 2024-09-11T23:00:06.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.412+0000 7fc57eff5640 1 -- 172.21.15.152:0/1171108523 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7feaff860 con 0x55d7febab2d0 2024-09-11T23:00:06.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.412+0000 7fc57eff5640 10 client.17496 handle_command_reply: tid=0 2024-09-11T23:00:06.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.412+0000 7fc5a3672740 1 client.17496 shutdown 2024-09-11T23:00:06.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.412+0000 7fc5a3672740 1 -- 172.21.15.152:0/1171108523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8db40 msgr2=0x55d7feb98350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:06.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.412+0000 7fc5a3672740 1 --2- 172.21.15.152:0/1171108523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8db40 0x55d7feb98350 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fc590004510 tx=0x7fc5900051a0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.412+0000 7fc5a3672740 1 -- 172.21.15.152:0/1171108523 shutdown_connections 2024-09-11T23:00:06.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.412+0000 7fc5a3672740 1 --2- 172.21.15.152:0/1171108523 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55d7febab2d0 0x55d7febad6c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.412+0000 7fc5a3672740 1 --2- 172.21.15.152:0/1171108523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb9b780 0x55d7feb98890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.412+0000 7fc5a3672740 1 --2- 172.21.15.152:0/1171108523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8db40 0x55d7feb98350 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.412+0000 7fc5a3672740 1 --2- 172.21.15.152:0/1171108523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8c5d0 0x55d7feb9b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.412+0000 7fc5a3672740 1 -- 172.21.15.152:0/1171108523 >> 172.21.15.152:0/1171108523 conn(0x55d7feb55310 msgr2=0x55d7feb8e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:06.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.412+0000 7fc57eff5640 0 client.17496 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T23:00:06.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.412+0000 7fc5a3672740 1 -- 172.21.15.152:0/1171108523 shutdown_connections 2024-09-11T23:00:06.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.412+0000 7fc5a3672740 1 -- 172.21.15.152:0/1171108523 wait complete. 2024-09-11T23:00:06.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.413+0000 7fc5a3672740 20 client.17496 trim_cache size 0 max 16384 2024-09-11T23:00:06.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.425+0000 7fc5a3672740 1 Processor -- start 2024-09-11T23:00:06.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.425+0000 7fc5a3672740 1 -- start start 2024-09-11T23:00:06.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc5a3672740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb55310 0x55d7feba7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc5a3672740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb9a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc5a3672740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8db40 0x55d7febb2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc5a3672740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c1ab680 con 0x55d7feb8c5d0 2024-09-11T23:00:06.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc5a3672740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c1ad1c0 con 0x55d7feb55310 2024-09-11T23:00:06.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc5a3672740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c1abd50 con 0x55d7feb8db40 2024-09-11T23:00:06.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc59a7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb9a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc59b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8db40 0x55d7febb2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc59a7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb9a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56558/0 (socket says 172.21.15.152:56558) 2024-09-11T23:00:06.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc59affd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb55310 0x55d7feba7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc59b7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8db40 0x55d7febb2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:49898/0 (socket says 172.21.15.152:49898) 2024-09-11T23:00:06.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc59a7fc640 1 -- 172.21.15.152:0/2771781332 learned_addr learned my addr 172.21.15.152:0/2771781332 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:06.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc59a7fc640 1 -- 172.21.15.152:0/2771781332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8db40 msgr2=0x55d7febb2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:06.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc59a7fc640 1 --2- 172.21.15.152:0/2771781332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8db40 0x55d7febb2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc59a7fc640 1 -- 172.21.15.152:0/2771781332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb55310 msgr2=0x55d7feba7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:06.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc59a7fc640 1 --2- 172.21.15.152:0/2771781332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb55310 0x55d7feba7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc59a7fc640 1 -- 172.21.15.152:0/2771781332 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7feb8f120 con 0x55d7feb8c5d0 2024-09-11T23:00:06.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc59b7fe640 1 --2- 172.21.15.152:0/2771781332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8db40 0x55d7febb2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:00:06.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc59affd640 1 --2- 172.21.15.152:0/2771781332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb55310 0x55d7feba7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:00:06.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc59a7fc640 1 --2- 172.21.15.152:0/2771781332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb9a2c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fc590004940 tx=0x7fc590005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484a1308cd282ab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:06.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.426+0000 7fc580ff9640 1 -- 172.21.15.152:0/2771781332 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc590028070 con 0x55d7feb8c5d0 2024-09-11T23:00:06.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.427+0000 7fc580ff9640 1 -- 172.21.15.152:0/2771781332 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc590011dd0 con 0x55d7feb8c5d0 2024-09-11T23:00:06.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.427+0000 7fc580ff9640 1 -- 172.21.15.152:0/2771781332 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59001d040 con 0x55d7feb8c5d0 2024-09-11T23:00:06.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.428+0000 7fc5a3672740 1 -- 172.21.15.152:0/2771781332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 msgr2=0x55d7feb9a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:06.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.428+0000 7fc5a3672740 1 --2- 172.21.15.152:0/2771781332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb9a2c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fc590004940 tx=0x7fc590005d30 comp rx=0 tx=0).stop 2024-09-11T23:00:06.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.428+0000 7fc5a3672740 1 -- 172.21.15.152:0/2771781332 shutdown_connections 2024-09-11T23:00:06.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.428+0000 7fc5a3672740 1 --2- 172.21.15.152:0/2771781332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb8db40 0x55d7febb2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.428+0000 7fc5a3672740 1 --2- 172.21.15.152:0/2771781332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb9a2c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.428+0000 7fc5a3672740 1 --2- 172.21.15.152:0/2771781332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb55310 0x55d7feba7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.428+0000 7fc5a3672740 1 -- 172.21.15.152:0/2771781332 >> 172.21.15.152:0/2771781332 conn(0x55d7febadab0 msgr2=0x55d7feb99bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:06.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.428+0000 7fc5a3672740 1 -- 172.21.15.152:0/2771781332 shutdown_connections 2024-09-11T23:00:06.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.428+0000 7fc5a3672740 1 -- 172.21.15.152:0/2771781332 wait complete. 2024-09-11T23:00:06.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.428+0000 7fc5a3672740 1 -- start start 2024-09-11T23:00:06.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.428+0000 7fc5a3672740 1 Processor -- start 2024-09-11T23:00:06.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.428+0000 7fc5a3672740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb55310 0x55d7feb9ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc5a3672740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb9f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc5a3672740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb8db40 0x55d7fec13eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc5a3672740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c1acaf0 con 0x55d7feb8c5d0 2024-09-11T23:00:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc5a3672740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c1ac420 con 0x55d7feb8db40 2024-09-11T23:00:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc5a3672740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc59c1a9640 con 0x55d7feb55310 2024-09-11T23:00:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc5a3672740 10 client.0 _ll_register_callbacks cb 0x7fc58c097e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc59a7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb9f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc59a7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb9f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56564/0 (socket says 172.21.15.152:56564) 2024-09-11T23:00:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc59a7fc640 1 -- 172.21.15.152:0/2380227537 learned_addr learned my addr 172.21.15.152:0/2380227537 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc59b7fe640 1 --2- 172.21.15.152:0/2380227537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb8db40 0x55d7fec13eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc59affd640 1 --2- 172.21.15.152:0/2380227537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb55310 0x55d7feb9ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc59a7fc640 1 -- 172.21.15.152:0/2380227537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb55310 msgr2=0x55d7feb9ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:06.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc59a7fc640 1 --2- 172.21.15.152:0/2380227537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb55310 0x55d7feb9ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc59a7fc640 1 -- 172.21.15.152:0/2380227537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb8db40 msgr2=0x55d7fec13eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:06.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc59a7fc640 1 --2- 172.21.15.152:0/2380227537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb8db40 0x55d7fec13eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc59a7fc640 1 -- 172.21.15.152:0/2380227537 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7fec143f0 con 0x55d7feb8c5d0 2024-09-11T23:00:06.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc59b7fe640 1 --2- 172.21.15.152:0/2380227537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb8db40 0x55d7fec13eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:00:06.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc59a7fc640 1 --2- 172.21.15.152:0/2380227537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb9f360 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fc590023510 tx=0x7fc590009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5f632f5755c41a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:06.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc57eff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:06.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc57eff5640 1 -- 172.21.15.152:0/2380227537 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc590028040 con 0x55d7feb8c5d0 2024-09-11T23:00:06.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc57eff5640 1 -- 172.21.15.152:0/2380227537 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fc590023d30 con 0x55d7feb8c5d0 2024-09-11T23:00:06.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.429+0000 7fc5a3672740 1 -- 172.21.15.152:0/2380227537 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc590011dd0 con 0x55d7feb8c5d0 2024-09-11T23:00:06.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.430+0000 7fc57eff5640 1 -- 172.21.15.152:0/2380227537 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc5900051a0 con 0x55d7feb8c5d0 2024-09-11T23:00:06.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.430+0000 7fc5a3672740 10 client.17502 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:06.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.430+0000 7fc5a3672740 1 -- 172.21.15.152:0/2380227537 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7fec15630 con 0x55d7feb8c5d0 2024-09-11T23:00:06.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.430+0000 7fc57eff5640 1 -- 172.21.15.152:0/2380227537 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc590011dd0 con 0x55d7feb8c5d0 2024-09-11T23:00:06.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.430+0000 7fc57eff5640 1 -- 172.21.15.152:0/2380227537 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc59000fd20 con 0x55d7feb8c5d0 2024-09-11T23:00:06.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.430+0000 7fc5a3672740 10 client.17502 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:06.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.430+0000 7fc5a3672740 10 client.17502 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:00:06.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.430+0000 7fc5a3672740 20 client.17502 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:06.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.430+0000 7fc5a3672740 1 --2- 172.21.15.152:0/2380227537 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55d7feb9b780 0x55d7fec17c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:06.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.430+0000 7fc5a3672740 4 client.17502 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:00:06.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.430+0000 7fc5a3672740 1 -- 172.21.15.152:0/2380227537 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7feb98c20 con 0x55d7feb9b780 2024-09-11T23:00:06.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.430+0000 7fc59affd640 1 --2- 172.21.15.152:0/2380227537 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55d7feb9b780 0x55d7fec17c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:06.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc59affd640 1 --2- 172.21.15.152:0/2380227537 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55d7feb9b780 0x55d7fec17c40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c18fc02882fd759 server_cookie=dbb97e07ef1ff82f in_seq=0 out_seq=0 2024-09-11T23:00:06.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc57eff5640 10 client.17502 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T23:00:06.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc57eff5640 1 -- 172.21.15.152:0/2380227537 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7feb98c20 con 0x55d7feb9b780 2024-09-11T23:00:06.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc57eff5640 10 client.17502 handle_command_reply: tid=0 2024-09-11T23:00:06.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc5a3672740 1 client.17502 shutdown 2024-09-11T23:00:06.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc5a3672740 1 -- 172.21.15.152:0/2380227537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 msgr2=0x55d7feb9f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:06.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc5a3672740 1 --2- 172.21.15.152:0/2380227537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb9f360 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fc590023510 tx=0x7fc590009c80 comp rx=0 tx=0).stop 2024-09-11T23:00:06.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc5a3672740 1 -- 172.21.15.152:0/2380227537 shutdown_connections 2024-09-11T23:00:06.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc5a3672740 1 --2- 172.21.15.152:0/2380227537 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55d7feb9b780 0x55d7fec17c40 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc5a3672740 1 --2- 172.21.15.152:0/2380227537 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7feb8db40 0x55d7fec13eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc5a3672740 1 --2- 172.21.15.152:0/2380227537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7feb8c5d0 0x55d7feb9f360 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc57eff5640 0 client.17502 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T23:00:06.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc5a3672740 1 --2- 172.21.15.152:0/2380227537 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7feb55310 0x55d7feb9ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc5a3672740 1 -- 172.21.15.152:0/2380227537 >> 172.21.15.152:0/2380227537 conn(0x55d7febadab0 msgr2=0x55d7feb9a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:06.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc5a3672740 1 -- 172.21.15.152:0/2380227537 shutdown_connections 2024-09-11T23:00:06.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc5a3672740 1 -- 172.21.15.152:0/2380227537 wait complete. 2024-09-11T23:00:06.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.431+0000 7fc5a3672740 20 client.17502 trim_cache size 0 max 16384 2024-09-11T23:00:06.439 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:00:06.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.432+0000 7fc5a1299640 1 -- 172.21.15.152:0/2171183122 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc5840a2cf0 msgr2=0x7fc5840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:06.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.432+0000 7fc5a1299640 1 --2- 172.21.15.152:0/2171183122 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc5840a2cf0 0x7fc5840a51b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc594005cc0 tx=0x7fc59402a750 comp rx=0 tx=0).stop 2024-09-11T23:00:06.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.432+0000 7fc5a1299640 1 -- 172.21.15.152:0/2171183122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc59c102db0 msgr2=0x7fc59c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:06.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.432+0000 7fc5a1299640 1 --2- 172.21.15.152:0/2171183122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc59c102db0 0x7fc59c1983f0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00e9c0 tx=0x7fc58c00ee80 comp rx=0 tx=0).stop 2024-09-11T23:00:06.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.432+0000 7fc5a1299640 1 -- 172.21.15.152:0/2171183122 shutdown_connections 2024-09-11T23:00:06.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.432+0000 7fc5a1299640 1 --2- 172.21.15.152:0/2171183122 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc5840a2cf0 0x7fc5840a51b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.432+0000 7fc5a1299640 1 --2- 172.21.15.152:0/2171183122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc59c10c190 0x7fc59c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.432+0000 7fc5a1299640 1 --2- 172.21.15.152:0/2171183122 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc59c1036d0 0x7fc59c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.432+0000 7fc5a1299640 1 --2- 172.21.15.152:0/2171183122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc59c102db0 0x7fc59c1983f0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:06.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.432+0000 7fc5a1299640 1 -- 172.21.15.152:0/2171183122 >> 172.21.15.152:0/2171183122 conn(0x7fc59c0fc790 msgr2=0x7fc59c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:06.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.432+0000 7fc5a1299640 1 -- 172.21.15.152:0/2171183122 shutdown_connections 2024-09-11T23:00:06.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:06.432+0000 7fc5a1299640 1 -- 172.21.15.152:0/2171183122 wait complete. 2024-09-11T23:00:06.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:06 smithi152 ceph-mon[31181]: pgmap v219: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-11T23:00:06.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:06 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:00:06.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:06 smithi191 ceph-mon[35211]: pgmap v219: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-11T23:00:06.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:06 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:00:06.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:06 smithi204 ceph-mon[35454]: pgmap v219: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-11T23:00:06.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:06 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:00:07.251 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:00:07.500 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:07.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:07 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:00:07.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:07 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:00:07.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:07 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:00:07.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:07 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:00:07.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:07 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:00:07.788 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:07 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:00:07.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:07 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:00:07.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:07 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:00:07.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:07 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:00:08.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.059+0000 7fca89d4a640 1 -- 172.21.15.152:0/2446872502 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca84102d40 msgr2=0x7fca84103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:08.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.059+0000 7fca89d4a640 1 --2- 172.21.15.152:0/2446872502 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca84102d40 0x7fca84103120 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fca74009900 tx=0x7fca7402a480 comp rx=0 tx=0).stop 2024-09-11T23:00:08.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.059+0000 7fca89d4a640 1 -- 172.21.15.152:0/2446872502 shutdown_connections 2024-09-11T23:00:08.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.059+0000 7fca89d4a640 1 --2- 172.21.15.152:0/2446872502 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca8410c170 0x7fca8410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.059+0000 7fca89d4a640 1 --2- 172.21.15.152:0/2446872502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca84103660 0x7fca8410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.059+0000 7fca89d4a640 1 --2- 172.21.15.152:0/2446872502 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca84102d40 0x7fca84103120 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.059+0000 7fca89d4a640 1 -- 172.21.15.152:0/2446872502 >> 172.21.15.152:0/2446872502 conn(0x7fca840fc7a0 msgr2=0x7fca840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:08.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.060+0000 7fca89d4a640 1 -- 172.21.15.152:0/2446872502 shutdown_connections 2024-09-11T23:00:08.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.060+0000 7fca89d4a640 1 -- 172.21.15.152:0/2446872502 wait complete. 2024-09-11T23:00:08.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.060+0000 7fca89d4a640 1 Processor -- start 2024-09-11T23:00:08.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.060+0000 7fca89d4a640 1 -- start start 2024-09-11T23:00:08.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.060+0000 7fca89d4a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca84102d40 0x7fca841009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca89d4a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca84103660 0x7fca84100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca89d4a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8410c170 0x7fca84101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca89d4a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fca84111fb0 con 0x7fca84103660 2024-09-11T23:00:08.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca89d4a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fca84111e30 con 0x7fca84102d40 2024-09-11T23:00:08.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca89d4a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fca84112130 con 0x7fca8410c170 2024-09-11T23:00:08.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca82ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca84103660 0x7fca84100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.074 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca82ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca84103660 0x7fca84100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56578/0 (socket says 172.21.15.152:56578) 2024-09-11T23:00:08.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca82ffd640 1 -- 172.21.15.152:0/3418881578 learned_addr learned my addr 172.21.15.152:0/3418881578 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:08.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca83fff640 1 --2- 172.21.15.152:0/3418881578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8410c170 0x7fca84101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca837fe640 1 --2- 172.21.15.152:0/3418881578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca84102d40 0x7fca841009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca82ffd640 1 -- 172.21.15.152:0/3418881578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8410c170 msgr2=0x7fca84101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:08.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca82ffd640 1 --2- 172.21.15.152:0/3418881578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8410c170 0x7fca84101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca82ffd640 1 -- 172.21.15.152:0/3418881578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca84102d40 msgr2=0x7fca841009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:08.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca82ffd640 1 --2- 172.21.15.152:0/3418881578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca84102d40 0x7fca841009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca82ffd640 1 -- 172.21.15.152:0/3418881578 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca841a6970 con 0x7fca84103660 2024-09-11T23:00:08.075 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca837fe640 1 --2- 172.21.15.152:0/3418881578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca84102d40 0x7fca841009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:00:08.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca82ffd640 1 --2- 172.21.15.152:0/3418881578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca84103660 0x7fca84100ee0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fca7800b3f0 tx=0x7fca7800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:08.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca80ff9640 1 -- 172.21.15.152:0/3418881578 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78015020 con 0x7fca84103660 2024-09-11T23:00:08.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca89d4a640 1 -- 172.21.15.152:0/3418881578 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca841a6c60 con 0x7fca84103660 2024-09-11T23:00:08.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca80ff9640 1 -- 172.21.15.152:0/3418881578 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca78011d10 con 0x7fca84103660 2024-09-11T23:00:08.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.061+0000 7fca80ff9640 1 -- 172.21.15.152:0/3418881578 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7800f2e0 con 0x7fca84103660 2024-09-11T23:00:08.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.062+0000 7fca89d4a640 1 -- 172.21.15.152:0/3418881578 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca841a71a0 con 0x7fca84103660 2024-09-11T23:00:08.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.064+0000 7fca80ff9640 1 -- 172.21.15.152:0/3418881578 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fca78018020 con 0x7fca84103660 2024-09-11T23:00:08.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.064+0000 7fca8c123740 1 Processor -- start 2024-09-11T23:00:08.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.064+0000 7fca8c123740 1 -- start start 2024-09-11T23:00:08.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.064+0000 7fca8c123740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723139b40 0x55f723138090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.064+0000 7fca8c123740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7231385d0 0x55f723138a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.076 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.064+0000 7fca8c123740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723143370 0x55f723145760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.064+0000 7fca8c123740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f72309e2f0 con 0x55f723143370 2024-09-11T23:00:08.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.064+0000 7fca8c123740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f723045cf0 con 0x55f7231385d0 2024-09-11T23:00:08.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.064+0000 7fca8c123740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f72309a2b0 con 0x55f723139b40 2024-09-11T23:00:08.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.064+0000 7fca80ff9640 1 --2- 172.21.15.152:0/3418881578 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fca6c0a2cf0 0x7fca6c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.064+0000 7fca82ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723143370 0x55f723145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.064+0000 7fca83fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723139b40 0x55f723138090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.065+0000 7fca83fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723139b40 0x55f723138090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:49928/0 (socket says 172.21.15.152:49928) 2024-09-11T23:00:08.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.065+0000 7fca83fff640 1 -- 172.21.15.152:0/4183318933 learned_addr learned my addr 172.21.15.152:0/4183318933 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:08.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.065+0000 7fca83fff640 1 --2- 172.21.15.152:0/3418881578 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fca6c0a2cf0 0x7fca6c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.065+0000 7fca837fe640 1 --2- 172.21.15.152:0/4183318933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7231385d0 0x55f723138a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.065+0000 7fca80ff9640 1 -- 172.21.15.152:0/3418881578 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fca780cec10 con 0x7fca84103660 2024-09-11T23:00:08.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.065+0000 7fca83fff640 1 -- 172.21.15.152:0/4183318933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7231385d0 msgr2=0x55f723138a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:08.077 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.065+0000 7fca83fff640 1 --2- 172.21.15.152:0/4183318933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7231385d0 0x55f723138a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.065+0000 7fca83fff640 1 -- 172.21.15.152:0/4183318933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723143370 msgr2=0x55f723145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:08.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.065+0000 7fca83fff640 1 --2- 172.21.15.152:0/4183318933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723143370 0x55f723145760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.065+0000 7fca83fff640 1 -- 172.21.15.152:0/4183318933 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72302a3a0 con 0x55f723139b40 2024-09-11T23:00:08.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.065+0000 7fca82ffd640 1 --2- 172.21.15.152:0/4183318933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723143370 0x55f723145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:00:08.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.065+0000 7fca83fff640 1 --2- 172.21.15.152:0/3418881578 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fca6c0a2cf0 0x7fca6c0a51b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca7c009fd0 tx=0x7fca7c017000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:08.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.065+0000 7fca83fff640 1 --2- 172.21.15.152:0/4183318933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723139b40 0x55f723138090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fca7c012d00 tx=0x7fca7c010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=844083ccbe1a11d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:08.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.065+0000 7fca69ffb640 1 -- 172.21.15.152:0/4183318933 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c022070 con 0x55f723139b40 2024-09-11T23:00:08.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.065+0000 7fca69ffb640 1 -- 172.21.15.152:0/4183318933 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca7c0124f0 con 0x55f723139b40 2024-09-11T23:00:08.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.065+0000 7fca69ffb640 1 -- 172.21.15.152:0/4183318933 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c01e900 con 0x55f723139b40 2024-09-11T23:00:08.078 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.066+0000 7fca8c123740 1 -- 172.21.15.152:0/4183318933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723139b40 msgr2=0x55f723138090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.066+0000 7fca8c123740 1 --2- 172.21.15.152:0/4183318933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723139b40 0x55f723138090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fca7c012d00 tx=0x7fca7c010740 comp rx=0 tx=0).stop 2024-09-11T23:00:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.066+0000 7fca8c123740 1 -- 172.21.15.152:0/4183318933 shutdown_connections 2024-09-11T23:00:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.066+0000 7fca8c123740 1 --2- 172.21.15.152:0/4183318933 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723143370 0x55f723145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.066+0000 7fca8c123740 1 --2- 172.21.15.152:0/4183318933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7231385d0 0x55f723138a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.066+0000 7fca8c123740 1 --2- 172.21.15.152:0/4183318933 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723139b40 0x55f723138090 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.066+0000 7fca8c123740 1 -- 172.21.15.152:0/4183318933 >> 172.21.15.152:0/4183318933 conn(0x55f723101310 msgr2=0x55f723134c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.066+0000 7fca8c123740 1 -- 172.21.15.152:0/4183318933 shutdown_connections 2024-09-11T23:00:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.066+0000 7fca8c123740 1 -- 172.21.15.152:0/4183318933 wait complete. 2024-09-11T23:00:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.067+0000 7fca8c123740 1 -- start start 2024-09-11T23:00:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.067+0000 7fca8c123740 1 Processor -- start 2024-09-11T23:00:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.067+0000 7fca8c123740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7231385d0 0x55f723142d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.067+0000 7fca8c123740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f72313ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.079 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.067+0000 7fca8c123740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f723143370 0x55f7231405d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.067+0000 7fca8c123740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f723146010 con 0x55f723139b40 2024-09-11T23:00:08.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.067+0000 7fca8c123740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f723059060 con 0x55f723143370 2024-09-11T23:00:08.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.067+0000 7fca8c123740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f723146190 con 0x55f7231385d0 2024-09-11T23:00:08.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.067+0000 7fca8c123740 10 client.0 _ll_register_callbacks cb 0x55f722dac8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:08.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.067+0000 7fca82ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f72313ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.067+0000 7fca82ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f72313ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56598/0 (socket says 172.21.15.152:56598) 2024-09-11T23:00:08.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.067+0000 7fca82ffd640 1 -- 172.21.15.152:0/1986015131 learned_addr learned my addr 172.21.15.152:0/1986015131 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:08.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.067+0000 7fca837fe640 1 --2- 172.21.15.152:0/1986015131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7231385d0 0x55f723142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.067+0000 7fca83fff640 1 --2- 172.21.15.152:0/1986015131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f723143370 0x55f7231405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca82ffd640 1 -- 172.21.15.152:0/1986015131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7231385d0 msgr2=0x55f723142d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:08.080 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca82ffd640 1 --2- 172.21.15.152:0/1986015131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7231385d0 0x55f723142d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca82ffd640 1 -- 172.21.15.152:0/1986015131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f723143370 msgr2=0x55f7231405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:08.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca82ffd640 1 --2- 172.21.15.152:0/1986015131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f723143370 0x55f7231405d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca82ffd640 1 -- 172.21.15.152:0/1986015131 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f722e0f3c0 con 0x55f723139b40 2024-09-11T23:00:08.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca82ffd640 1 --2- 172.21.15.152:0/1986015131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f72313ffe0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fca78097c60 tx=0x7fca780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59e101641ee9f6e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:08.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:08.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca62ffd640 1 -- 172.21.15.152:0/1986015131 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7809b990 con 0x55f723139b40 2024-09-11T23:00:08.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca62ffd640 1 -- 172.21.15.152:0/1986015131 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fca780cc0f0 con 0x55f723139b40 2024-09-11T23:00:08.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca8c123740 1 -- 172.21.15.152:0/1986015131 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca7c0124f0 con 0x55f723139b40 2024-09-11T23:00:08.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca62ffd640 1 -- 172.21.15.152:0/1986015131 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fca780b3030 con 0x55f723139b40 2024-09-11T23:00:08.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca8c123740 10 client.17514 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:08.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca8c123740 1 -- 172.21.15.152:0/1986015131 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f723151a70 con 0x55f723139b40 2024-09-11T23:00:08.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca62ffd640 1 -- 172.21.15.152:0/1986015131 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca7c0124f0 con 0x55f723139b40 2024-09-11T23:00:08.081 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca62ffd640 1 -- 172.21.15.152:0/1986015131 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fca78097900 con 0x55f723139b40 2024-09-11T23:00:08.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca8c123740 10 client.17514 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:08.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca8c123740 10 client.17514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:00:08.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca8c123740 20 client.17514 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:08.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca8c123740 1 --2- 172.21.15.152:0/1986015131 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f723152f20 0x55f723155330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca8c123740 4 client.17514 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:00:08.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.068+0000 7fca8c123740 1 -- 172.21.15.152:0/1986015131 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7230ab860 con 0x55f723152f20 2024-09-11T23:00:08.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.069+0000 7fca837fe640 1 --2- 172.21.15.152:0/1986015131 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f723152f20 0x55f723155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.069+0000 7fca62ffd640 10 client.17514 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:08.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.069+0000 7fca837fe640 1 --2- 172.21.15.152:0/1986015131 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f723152f20 0x55f723155330 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e81de128657b5b server_cookie=4cafc6619d5491f1 in_seq=0 out_seq=0 2024-09-11T23:00:08.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.070+0000 7fca62ffd640 1 -- 172.21.15.152:0/1986015131 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7230ab860 con 0x55f723152f20 2024-09-11T23:00:08.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.070+0000 7fca62ffd640 10 client.17514 handle_command_reply: tid=0 2024-09-11T23:00:08.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.070+0000 7fca8c123740 1 client.17514 shutdown 2024-09-11T23:00:08.082 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.070+0000 7fca8c123740 1 -- 172.21.15.152:0/1986015131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 msgr2=0x55f72313ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:08.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.070+0000 7fca8c123740 1 --2- 172.21.15.152:0/1986015131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f72313ffe0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fca78097c60 tx=0x7fca780046c0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.070+0000 7fca8c123740 1 -- 172.21.15.152:0/1986015131 shutdown_connections 2024-09-11T23:00:08.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.070+0000 7fca8c123740 1 --2- 172.21.15.152:0/1986015131 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f723152f20 0x55f723155330 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.070+0000 7fca8c123740 1 --2- 172.21.15.152:0/1986015131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f723143370 0x55f7231405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.070+0000 7fca8c123740 1 --2- 172.21.15.152:0/1986015131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f72313ffe0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.070+0000 7fca8c123740 1 --2- 172.21.15.152:0/1986015131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f7231385d0 0x55f723142d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.070+0000 7fca8c123740 1 -- 172.21.15.152:0/1986015131 >> 172.21.15.152:0/1986015131 conn(0x55f723101310 msgr2=0x55f72313a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:08.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.070+0000 7fca62ffd640 0 client.17514 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:08.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.070+0000 7fca8c123740 1 -- 172.21.15.152:0/1986015131 shutdown_connections 2024-09-11T23:00:08.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.070+0000 7fca8c123740 1 -- 172.21.15.152:0/1986015131 wait complete. 2024-09-11T23:00:08.083 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.070+0000 7fca8c123740 20 client.17514 trim_cache size 0 max 16384 2024-09-11T23:00:08.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca8c123740 1 Processor -- start 2024-09-11T23:00:08.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca8c123740 1 -- start start 2024-09-11T23:00:08.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca8c123740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7231385d0 0x55f72313ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.088 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca8c123740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f723141f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca8c123740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723143370 0x55f72313b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca8c123740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fca841afa80 con 0x55f723139b40 2024-09-11T23:00:08.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca8c123740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fca841b15c0 con 0x55f7231385d0 2024-09-11T23:00:08.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca8c123740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fca841b0150 con 0x55f723143370 2024-09-11T23:00:08.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca82ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f723141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca82ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f723141f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56602/0 (socket says 172.21.15.152:56602) 2024-09-11T23:00:08.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca82ffd640 1 -- 172.21.15.152:0/2550911650 learned_addr learned my addr 172.21.15.152:0/2550911650 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:08.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca83fff640 1 --2- 172.21.15.152:0/2550911650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723143370 0x55f72313b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca837fe640 1 --2- 172.21.15.152:0/2550911650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7231385d0 0x55f72313ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca82ffd640 1 -- 172.21.15.152:0/2550911650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723143370 msgr2=0x55f72313b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:08.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca82ffd640 1 --2- 172.21.15.152:0/2550911650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723143370 0x55f72313b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca82ffd640 1 -- 172.21.15.152:0/2550911650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7231385d0 msgr2=0x55f72313ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:08.089 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca82ffd640 1 --2- 172.21.15.152:0/2550911650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7231385d0 0x55f72313ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca82ffd640 1 -- 172.21.15.152:0/2550911650 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f723134510 con 0x55f723139b40 2024-09-11T23:00:08.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.083+0000 7fca837fe640 1 --2- 172.21.15.152:0/2550911650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7231385d0 0x55f72313ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:00:08.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.084+0000 7fca82ffd640 1 --2- 172.21.15.152:0/2550911650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f723141f50 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fca780976f0 tx=0x7fca78010720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da122a4bd4e88e25 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:08.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.084+0000 7fca697fa640 1 -- 172.21.15.152:0/2550911650 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78097360 con 0x55f723139b40 2024-09-11T23:00:08.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.084+0000 7fca697fa640 1 -- 172.21.15.152:0/2550911650 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca780b4270 con 0x55f723139b40 2024-09-11T23:00:08.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.084+0000 7fca697fa640 1 -- 172.21.15.152:0/2550911650 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7809abf0 con 0x55f723139b40 2024-09-11T23:00:08.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.084+0000 7fca8c123740 1 -- 172.21.15.152:0/2550911650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 msgr2=0x55f723141f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:08.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.084+0000 7fca8c123740 1 --2- 172.21.15.152:0/2550911650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f723141f50 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fca780976f0 tx=0x7fca78010720 comp rx=0 tx=0).stop 2024-09-11T23:00:08.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.084+0000 7fca8c123740 1 -- 172.21.15.152:0/2550911650 shutdown_connections 2024-09-11T23:00:08.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.084+0000 7fca8c123740 1 --2- 172.21.15.152:0/2550911650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723143370 0x55f72313b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.084+0000 7fca8c123740 1 --2- 172.21.15.152:0/2550911650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f723141f50 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.084+0000 7fca8c123740 1 --2- 172.21.15.152:0/2550911650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7231385d0 0x55f72313ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.084+0000 7fca8c123740 1 -- 172.21.15.152:0/2550911650 >> 172.21.15.152:0/2550911650 conn(0x55f723101310 msgr2=0x55f723141840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:08.090 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.084+0000 7fca8c123740 1 -- 172.21.15.152:0/2550911650 shutdown_connections 2024-09-11T23:00:08.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.085+0000 7fca8c123740 1 -- 172.21.15.152:0/2550911650 wait complete. 2024-09-11T23:00:08.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.085+0000 7fca8c123740 1 -- start start 2024-09-11T23:00:08.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.085+0000 7fca8c123740 1 Processor -- start 2024-09-11T23:00:08.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.085+0000 7fca8c123740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7231385d0 0x55f7231b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.085+0000 7fca8c123740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f7231b6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.085+0000 7fca8c123740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723143370 0x55f723146940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.085+0000 7fca8c123740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fca841b0ef0 con 0x55f723139b40 2024-09-11T23:00:08.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.085+0000 7fca8c123740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fca841b0820 con 0x55f7231385d0 2024-09-11T23:00:08.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.085+0000 7fca8c123740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fca841ada40 con 0x55f723143370 2024-09-11T23:00:08.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.085+0000 7fca8c123740 10 client.0 _ll_register_callbacks cb 0x7fca78009180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:08.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.085+0000 7fca82ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f7231b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.085+0000 7fca82ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f7231b6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:56616/0 (socket says 172.21.15.152:56616) 2024-09-11T23:00:08.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.085+0000 7fca82ffd640 1 -- 172.21.15.152:0/4080383383 learned_addr learned my addr 172.21.15.152:0/4080383383 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:08.091 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.085+0000 7fca837fe640 1 --2- 172.21.15.152:0/4080383383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7231385d0 0x55f7231b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.086+0000 7fca83fff640 1 --2- 172.21.15.152:0/4080383383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723143370 0x55f723146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.086+0000 7fca82ffd640 1 -- 172.21.15.152:0/4080383383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723143370 msgr2=0x55f723146940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:08.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.086+0000 7fca82ffd640 1 --2- 172.21.15.152:0/4080383383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723143370 0x55f723146940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.086+0000 7fca82ffd640 1 -- 172.21.15.152:0/4080383383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7231385d0 msgr2=0x55f7231b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:08.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.086+0000 7fca82ffd640 1 --2- 172.21.15.152:0/4080383383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7231385d0 0x55f7231b5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.086+0000 7fca82ffd640 1 -- 172.21.15.152:0/4080383383 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f723147140 con 0x55f723139b40 2024-09-11T23:00:08.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.086+0000 7fca82ffd640 1 --2- 172.21.15.152:0/4080383383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f7231b6420 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fca7809b220 tx=0x7fca78097530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66023f50230dee01 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:08.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.086+0000 7fca62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:08.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.086+0000 7fca62ffd640 1 -- 172.21.15.152:0/4080383383 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7809abf0 con 0x55f723139b40 2024-09-11T23:00:08.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.086+0000 7fca62ffd640 1 -- 172.21.15.152:0/4080383383 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8448+0+0 (secure 0 0 0) 0x7fca780b4040 con 0x55f723139b40 2024-09-11T23:00:08.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.086+0000 7fca8c123740 1 -- 172.21.15.152:0/4080383383 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f7230a5d70 con 0x55f723139b40 2024-09-11T23:00:08.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.086+0000 7fca62ffd640 1 -- 172.21.15.152:0/4080383383 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fca78065050 con 0x55f723139b40 2024-09-11T23:00:08.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.086+0000 7fca8c123740 10 client.17520 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:08.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.086+0000 7fca8c123740 1 -- 172.21.15.152:0/4080383383 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f7231bd5f0 con 0x55f723139b40 2024-09-11T23:00:08.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.086+0000 7fca62ffd640 1 -- 172.21.15.152:0/4080383383 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f7230a5d70 con 0x55f723139b40 2024-09-11T23:00:08.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.087+0000 7fca62ffd640 1 -- 172.21.15.152:0/4080383383 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fca780cc0f0 con 0x55f723139b40 2024-09-11T23:00:08.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.087+0000 7fca8c123740 10 client.17520 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:08.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.087+0000 7fca8c123740 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:00:08.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.087+0000 7fca8c123740 20 client.17520 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:08.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.087+0000 7fca8c123740 1 --2- 172.21.15.152:0/4080383383 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f7231bdb00 0x55f7231bff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:08.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.087+0000 7fca8c123740 4 client.17520 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:00:08.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.087+0000 7fca8c123740 1 -- 172.21.15.152:0/4080383383 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f723022170 con 0x55f7231bdb00 2024-09-11T23:00:08.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.087+0000 7fca837fe640 1 --2- 172.21.15.152:0/4080383383 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f7231bdb00 0x55f7231bff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:08.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.087+0000 7fca837fe640 1 --2- 172.21.15.152:0/4080383383 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f7231bdb00 0x55f7231bff20 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a0dea782702d34e server_cookie=b55c63f8b3e01833 in_seq=0 out_seq=0 2024-09-11T23:00:08.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.087+0000 7fca62ffd640 10 client.17520 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:08.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca62ffd640 1 -- 172.21.15.152:0/4080383383 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f723022170 con 0x55f7231bdb00 2024-09-11T23:00:08.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca62ffd640 10 client.17520 handle_command_reply: tid=0 2024-09-11T23:00:08.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca8c123740 1 client.17520 shutdown 2024-09-11T23:00:08.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca8c123740 1 -- 172.21.15.152:0/4080383383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 msgr2=0x55f7231b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:08.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca8c123740 1 --2- 172.21.15.152:0/4080383383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f7231b6420 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fca7809b220 tx=0x7fca78097530 comp rx=0 tx=0).stop 2024-09-11T23:00:08.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca8c123740 1 -- 172.21.15.152:0/4080383383 shutdown_connections 2024-09-11T23:00:08.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca8c123740 1 --2- 172.21.15.152:0/4080383383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f723143370 0x55f723146940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca8c123740 1 --2- 172.21.15.152:0/4080383383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f723139b40 0x55f7231b6420 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca8c123740 1 --2- 172.21.15.152:0/4080383383 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f7231bdb00 0x55f7231bff20 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca8c123740 1 --2- 172.21.15.152:0/4080383383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f7231385d0 0x55f7231b5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca62ffd640 0 client.17520 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:08.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca8c123740 1 -- 172.21.15.152:0/4080383383 >> 172.21.15.152:0/4080383383 conn(0x55f723101310 msgr2=0x55f723142060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:08.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca8c123740 1 -- 172.21.15.152:0/4080383383 shutdown_connections 2024-09-11T23:00:08.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca8c123740 1 -- 172.21.15.152:0/4080383383 wait complete. 2024-09-11T23:00:08.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca8c123740 20 client.17520 trim_cache size 0 max 16384 2024-09-11T23:00:08.095 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:00:08.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca89d4a640 1 -- 172.21.15.152:0/3418881578 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fca6c0a2cf0 msgr2=0x7fca6c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:08.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca89d4a640 1 --2- 172.21.15.152:0/3418881578 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fca6c0a2cf0 0x7fca6c0a51b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fca7c009fd0 tx=0x7fca7c017000 comp rx=0 tx=0).stop 2024-09-11T23:00:08.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca89d4a640 1 -- 172.21.15.152:0/3418881578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca84103660 msgr2=0x7fca84100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:08.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.088+0000 7fca89d4a640 1 --2- 172.21.15.152:0/3418881578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca84103660 0x7fca84100ee0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fca7800b3f0 tx=0x7fca7800b8b0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.089+0000 7fca89d4a640 1 -- 172.21.15.152:0/3418881578 shutdown_connections 2024-09-11T23:00:08.096 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.089+0000 7fca89d4a640 1 --2- 172.21.15.152:0/3418881578 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fca6c0a2cf0 0x7fca6c0a51b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.089+0000 7fca89d4a640 1 --2- 172.21.15.152:0/3418881578 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca8410c170 0x7fca84101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.089+0000 7fca89d4a640 1 --2- 172.21.15.152:0/3418881578 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca84103660 0x7fca84100ee0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.089+0000 7fca89d4a640 1 --2- 172.21.15.152:0/3418881578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca84102d40 0x7fca841009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:08.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.089+0000 7fca89d4a640 1 -- 172.21.15.152:0/3418881578 >> 172.21.15.152:0/3418881578 conn(0x7fca840fc7a0 msgr2=0x7fca840fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:08.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.089+0000 7fca89d4a640 1 -- 172.21.15.152:0/3418881578 shutdown_connections 2024-09-11T23:00:08.097 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:08.089+0000 7fca89d4a640 1 -- 172.21.15.152:0/3418881578 wait complete. 2024-09-11T23:00:08.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:08 smithi152 ceph-mon[31181]: pgmap v220: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-11T23:00:09.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:08 smithi191 ceph-mon[35211]: pgmap v220: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-11T23:00:09.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:08 smithi204 ceph-mon[35454]: pgmap v220: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-11T23:00:09.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:09 smithi152 ceph-mon[31181]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T23:00:10.007 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:00:10.007 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:00:10.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:09 smithi191 ceph-mon[35211]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T23:00:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:09 smithi204 ceph-mon[35454]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T23:00:10.268 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:10.665 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:10 smithi152 ceph-mon[31181]: pgmap v222: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:00:10.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.892+0000 7f5913bee640 1 -- 172.21.15.152:0/1232324187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f590c102790 msgr2=0x7f590c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:10.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.892+0000 7f5913bee640 1 --2- 172.21.15.152:0/1232324187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f590c102790 0x7f590c1067e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009900 tx=0x7f58fc02a410 comp rx=0 tx=0).stop 2024-09-11T23:00:10.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.892+0000 7f5913bee640 1 -- 172.21.15.152:0/1232324187 shutdown_connections 2024-09-11T23:00:10.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.892+0000 7f5913bee640 1 --2- 172.21.15.152:0/1232324187 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f590c106f10 0x7f590c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.892+0000 7f5913bee640 1 --2- 172.21.15.152:0/1232324187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f590c102790 0x7f590c1067e0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.917 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.892+0000 7f5913bee640 1 --2- 172.21.15.152:0/1232324187 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590c101de0 0x7f590c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.892+0000 7f5913bee640 1 -- 172.21.15.152:0/1232324187 >> 172.21.15.152:0/1232324187 conn(0x7f590c0fd8c0 msgr2=0x7f590c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:10.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.892+0000 7f5913bee640 1 -- 172.21.15.152:0/1232324187 shutdown_connections 2024-09-11T23:00:10.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.892+0000 7f5913bee640 1 -- 172.21.15.152:0/1232324187 wait complete. 2024-09-11T23:00:10.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f5913bee640 1 Processor -- start 2024-09-11T23:00:10.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f5913bee640 1 -- start start 2024-09-11T23:00:10.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f5913bee640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f590c101de0 0x7f590c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f5913bee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f590c102790 0x7f590c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f5913bee640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590c106f10 0x7f590c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f5913bee640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f590c111f90 con 0x7f590c102790 2024-09-11T23:00:10.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f5913bee640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f590c111e10 con 0x7f590c106f10 2024-09-11T23:00:10.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f5913bee640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f590c112110 con 0x7f590c101de0 2024-09-11T23:00:10.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f5911189640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f590c102790 0x7f590c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.918 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f5911189640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f590c102790 0x7f590c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36584/0 (socket says 172.21.15.152:36584) 2024-09-11T23:00:10.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f5911189640 1 -- 172.21.15.152:0/3460158930 learned_addr learned my addr 172.21.15.152:0/3460158930 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:10.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f591198a640 1 --2- 172.21.15.152:0/3460158930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f590c101de0 0x7f590c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f591218b640 1 --2- 172.21.15.152:0/3460158930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590c106f10 0x7f590c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f5911189640 1 -- 172.21.15.152:0/3460158930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f590c101de0 msgr2=0x7f590c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:10.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f5911189640 1 --2- 172.21.15.152:0/3460158930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f590c101de0 0x7f590c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f5911189640 1 -- 172.21.15.152:0/3460158930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590c106f10 msgr2=0x7f590c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:10.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f5911189640 1 --2- 172.21.15.152:0/3460158930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590c106f10 0x7f590c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f5911189640 1 -- 172.21.15.152:0/3460158930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590c1a68e0 con 0x7f590c102790 2024-09-11T23:00:10.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f591198a640 1 --2- 172.21.15.152:0/3460158930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f590c101de0 0x7f590c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:10.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.893+0000 7f591218b640 1 --2- 172.21.15.152:0/3460158930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590c106f10 0x7f590c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:00:10.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.894+0000 7f5911189640 1 --2- 172.21.15.152:0/3460158930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f590c102790 0x7f590c074a30 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f58fc005ab0 tx=0x7f58fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:10.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.894+0000 7f5902ffd640 1 -- 172.21.15.152:0/3460158930 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc03f030 con 0x7f590c102790 2024-09-11T23:00:10.919 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.894+0000 7f5913bee640 1 -- 172.21.15.152:0/3460158930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590c1a6b70 con 0x7f590c102790 2024-09-11T23:00:10.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.894+0000 7f5902ffd640 1 -- 172.21.15.152:0/3460158930 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58fc003ce0 con 0x7f590c102790 2024-09-11T23:00:10.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.894+0000 7f5902ffd640 1 -- 172.21.15.152:0/3460158930 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc035070 con 0x7f590c102790 2024-09-11T23:00:10.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.894+0000 7f5913bee640 1 -- 172.21.15.152:0/3460158930 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f590c1a7050 con 0x7f590c102790 2024-09-11T23:00:10.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.896+0000 7f5902ffd640 1 -- 172.21.15.152:0/3460158930 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f58fc042030 con 0x7f590c102790 2024-09-11T23:00:10.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.896+0000 7f5915fc7740 1 Processor -- start 2024-09-11T23:00:10.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.896+0000 7f5915fc7740 1 -- start start 2024-09-11T23:00:10.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f5915fc7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c6bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f5915fc7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6c480 0x55f2f4c76800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f5915fc7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c76d40 0x55f2f4c79130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f5915fc7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f2f4bd1280 con 0x55f2f4c6c480 2024-09-11T23:00:10.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f5915fc7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f2f4b78b10 con 0x55f2f4c6bb60 2024-09-11T23:00:10.920 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f5915fc7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f2f4bcd240 con 0x55f2f4c76d40 2024-09-11T23:00:10.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f591198a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6c480 0x55f2f4c76800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f591198a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6c480 0x55f2f4c76800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36598/0 (socket says 172.21.15.152:36598) 2024-09-11T23:00:10.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f591198a640 1 -- 172.21.15.152:0/4256445890 learned_addr learned my addr 172.21.15.152:0/4256445890 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:10.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f5902ffd640 1 --2- 172.21.15.152:0/3460158930 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58f80a2cf0 0x7f58f80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f591218b640 1 --2- 172.21.15.152:0/4256445890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c6bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f5911189640 1 --2- 172.21.15.152:0/4256445890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c76d40 0x55f2f4c79130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f591198a640 1 -- 172.21.15.152:0/4256445890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c76d40 msgr2=0x55f2f4c79130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:10.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f591218b640 1 --2- 172.21.15.152:0/3460158930 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58f80a2cf0 0x7f58f80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f591198a640 1 --2- 172.21.15.152:0/4256445890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c76d40 0x55f2f4c79130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f591198a640 1 -- 172.21.15.152:0/4256445890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6bb60 msgr2=0x55f2f4c6bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:10.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f591198a640 1 --2- 172.21.15.152:0/4256445890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c6bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f591198a640 1 -- 172.21.15.152:0/4256445890 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2f4b851f0 con 0x55f2f4c6c480 2024-09-11T23:00:10.921 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f5911189640 1 --2- 172.21.15.152:0/4256445890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c76d40 0x55f2f4c79130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:10.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f5902ffd640 1 -- 172.21.15.152:0/3460158930 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8536+0+0 (secure 0 0 0) 0x7f58fc0efa80 con 0x7f590c102790 2024-09-11T23:00:10.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f591198a640 1 --2- 172.21.15.152:0/4256445890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6c480 0x55f2f4c76800 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f5908008c60 tx=0x7f590800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710eeba850e9146c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:10.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.897+0000 7f58f7fff640 1 -- 172.21.15.152:0/4256445890 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908014070 con 0x55f2f4c6c480 2024-09-11T23:00:10.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.898+0000 7f58f7fff640 1 -- 172.21.15.152:0/4256445890 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59080047a0 con 0x55f2f4c6c480 2024-09-11T23:00:10.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.898+0000 7f58f7fff640 1 -- 172.21.15.152:0/4256445890 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908010740 con 0x55f2f4c6c480 2024-09-11T23:00:10.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.898+0000 7f5915fc7740 1 -- 172.21.15.152:0/4256445890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6c480 msgr2=0x55f2f4c76800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:10.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.898+0000 7f591218b640 1 --2- 172.21.15.152:0/3460158930 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58f80a2cf0 0x7f58f80a51b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5904010500 tx=0x7f5904004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:10.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.898+0000 7f5915fc7740 1 --2- 172.21.15.152:0/4256445890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6c480 0x55f2f4c76800 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f5908008c60 tx=0x7f590800e720 comp rx=0 tx=0).stop 2024-09-11T23:00:10.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.898+0000 7f5915fc7740 1 -- 172.21.15.152:0/4256445890 shutdown_connections 2024-09-11T23:00:10.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.898+0000 7f5915fc7740 1 --2- 172.21.15.152:0/4256445890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c76d40 0x55f2f4c79130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.898+0000 7f5915fc7740 1 --2- 172.21.15.152:0/4256445890 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6c480 0x55f2f4c76800 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.898+0000 7f5915fc7740 1 --2- 172.21.15.152:0/4256445890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c6bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.922 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.898+0000 7f5915fc7740 1 -- 172.21.15.152:0/4256445890 >> 172.21.15.152:0/4256445890 conn(0x55f2f4c680f0 msgr2=0x55f2f4c684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:10.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.898+0000 7f5915fc7740 1 -- 172.21.15.152:0/4256445890 shutdown_connections 2024-09-11T23:00:10.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.898+0000 7f5915fc7740 1 -- 172.21.15.152:0/4256445890 wait complete. 2024-09-11T23:00:10.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.899+0000 7f5915fc7740 1 -- start start 2024-09-11T23:00:10.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.899+0000 7f5915fc7740 1 Processor -- start 2024-09-11T23:00:10.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.899+0000 7f5915fc7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c76800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.899+0000 7f5915fc7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6c480 0x55f2f4c73b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.899+0000 7f5915fc7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c76d40 0x55f2f4c742e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.899+0000 7f5915fc7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f2f4c79910 con 0x55f2f4c6c480 2024-09-11T23:00:10.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.899+0000 7f5915fc7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f2f4b8bf10 con 0x55f2f4c6bb60 2024-09-11T23:00:10.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.899+0000 7f5915fc7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f2f4c79a90 con 0x55f2f4c76d40 2024-09-11T23:00:10.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.899+0000 7f5915fc7740 10 client.0 _ll_register_callbacks cb 0x55f2f4b54a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:10.923 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.899+0000 7f5911189640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6c480 0x55f2f4c73b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.899+0000 7f5911189640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6c480 0x55f2f4c73b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36610/0 (socket says 172.21.15.152:36610) 2024-09-11T23:00:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.899+0000 7f5911189640 1 -- 172.21.15.152:0/3929761304 learned_addr learned my addr 172.21.15.152:0/3929761304 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.899+0000 7f591198a640 1 --2- 172.21.15.152:0/3929761304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c76800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.900+0000 7f591218b640 1 --2- 172.21.15.152:0/3929761304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c76d40 0x55f2f4c742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.900+0000 7f5911189640 1 -- 172.21.15.152:0/3929761304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c76d40 msgr2=0x55f2f4c742e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.900+0000 7f5911189640 1 --2- 172.21.15.152:0/3929761304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c76d40 0x55f2f4c742e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.900+0000 7f5911189640 1 -- 172.21.15.152:0/3929761304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6bb60 msgr2=0x55f2f4c76800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.900+0000 7f5911189640 1 --2- 172.21.15.152:0/3929761304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c76800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.900+0000 7f5911189640 1 -- 172.21.15.152:0/3929761304 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2f49426a0 con 0x55f2f4c6c480 2024-09-11T23:00:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.900+0000 7f5911189640 1 --2- 172.21.15.152:0/3929761304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6c480 0x55f2f4c73b50 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f58fc006010 tx=0x7f58fc009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca66e27b27b1113 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.900+0000 7f58f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.900+0000 7f58f57fa640 1 -- 172.21.15.152:0/3929761304 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc049070 con 0x55f2f4c6c480 2024-09-11T23:00:10.924 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.900+0000 7f58f57fa640 1 -- 172.21.15.152:0/3929761304 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8536+0+0 (secure 0 0 0) 0x7f58fc0d0130 con 0x55f2f4c6c480 2024-09-11T23:00:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.900+0000 7f5915fc7740 1 -- 172.21.15.152:0/3929761304 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59080047a0 con 0x55f2f4c6c480 2024-09-11T23:00:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.900+0000 7f58f57fa640 1 -- 172.21.15.152:0/3929761304 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f58fc0bb8f0 con 0x55f2f4c6c480 2024-09-11T23:00:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.900+0000 7f5915fc7740 10 client.17532 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.900+0000 7f5915fc7740 1 -- 172.21.15.152:0/3929761304 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2f4c853e0 con 0x55f2f4c6c480 2024-09-11T23:00:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.900+0000 7f58f57fa640 1 -- 172.21.15.152:0/3929761304 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59080047a0 con 0x55f2f4c6c480 2024-09-11T23:00:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.900+0000 7f58f57fa640 1 -- 172.21.15.152:0/3929761304 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f58fc0ed440 con 0x55f2f4c6c480 2024-09-11T23:00:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.901+0000 7f5915fc7740 10 client.17532 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.901+0000 7f5915fc7740 10 client.17532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:00:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.901+0000 7f5915fc7740 20 client.17532 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.901+0000 7f5915fc7740 1 --2- 172.21.15.152:0/3929761304 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f2f4c86730 0x55f2f4c88b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.901+0000 7f5915fc7740 4 client.17532 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:00:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.901+0000 7f5915fc7740 1 -- 172.21.15.152:0/3929761304 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2f4bde7f0 con 0x55f2f4c86730 2024-09-11T23:00:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.901+0000 7f591198a640 1 --2- 172.21.15.152:0/3929761304 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f2f4c86730 0x55f2f4c88b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.925 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.901+0000 7f58f57fa640 10 client.17532 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:10.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.901+0000 7f591198a640 1 --2- 172.21.15.152:0/3929761304 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f2f4c86730 0x55f2f4c88b20 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b1b0d13ad8467d5 server_cookie=f163476b14422fab in_seq=0 out_seq=0 2024-09-11T23:00:10.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.902+0000 7f58f57fa640 1 -- 172.21.15.152:0/3929761304 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f2f4bde7f0 con 0x55f2f4c86730 2024-09-11T23:00:10.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.902+0000 7f58f57fa640 10 client.17532 handle_command_reply: tid=0 2024-09-11T23:00:10.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.902+0000 7f5915fc7740 1 client.17532 shutdown 2024-09-11T23:00:10.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.902+0000 7f5915fc7740 1 -- 172.21.15.152:0/3929761304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6c480 msgr2=0x55f2f4c73b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:10.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.902+0000 7f5915fc7740 1 --2- 172.21.15.152:0/3929761304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6c480 0x55f2f4c73b50 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f58fc006010 tx=0x7f58fc009620 comp rx=0 tx=0).stop 2024-09-11T23:00:10.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.902+0000 7f5915fc7740 1 -- 172.21.15.152:0/3929761304 shutdown_connections 2024-09-11T23:00:10.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.902+0000 7f5915fc7740 1 --2- 172.21.15.152:0/3929761304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c76d40 0x55f2f4c742e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.902+0000 7f5915fc7740 1 --2- 172.21.15.152:0/3929761304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6c480 0x55f2f4c73b50 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.902+0000 7f5915fc7740 1 --2- 172.21.15.152:0/3929761304 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f2f4c86730 0x55f2f4c88b20 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.902+0000 7f5915fc7740 1 --2- 172.21.15.152:0/3929761304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c76800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.902+0000 7f58f57fa640 0 client.17532 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:10.926 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.902+0000 7f5915fc7740 1 -- 172.21.15.152:0/3929761304 >> 172.21.15.152:0/3929761304 conn(0x55f2f4c680f0 msgr2=0x55f2f4c6de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:10.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.902+0000 7f5915fc7740 1 -- 172.21.15.152:0/3929761304 shutdown_connections 2024-09-11T23:00:10.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.902+0000 7f5915fc7740 1 -- 172.21.15.152:0/3929761304 wait complete. 2024-09-11T23:00:10.927 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.902+0000 7f5915fc7740 20 client.17532 trim_cache size 0 max 16384 2024-09-11T23:00:10.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.923+0000 7f5915fc7740 1 Processor -- start 2024-09-11T23:00:10.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.923+0000 7f5915fc7740 1 -- start start 2024-09-11T23:00:10.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.923+0000 7f5915fc7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c8e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.923+0000 7f5915fc7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6c480 0x55f2f4c8e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.923+0000 7f5915fc7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c76d40 0x55f2f4c70db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.923+0000 7f5915fc7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1afa30 con 0x55f2f4c76d40 2024-09-11T23:00:10.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.923+0000 7f5915fc7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1b1570 con 0x55f2f4c6c480 2024-09-11T23:00:10.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.923+0000 7f5915fc7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1b0100 con 0x55f2f4c6bb60 2024-09-11T23:00:10.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.923+0000 7f591218b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c76d40 0x55f2f4c70db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.932 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.923+0000 7f591218b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c76d40 0x55f2f4c70db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36612/0 (socket says 172.21.15.152:36612) 2024-09-11T23:00:10.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.923+0000 7f591218b640 1 -- 172.21.15.152:0/2558147062 learned_addr learned my addr 172.21.15.152:0/2558147062 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:10.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.923+0000 7f591198a640 1 --2- 172.21.15.152:0/2558147062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c8e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.923+0000 7f5911189640 1 --2- 172.21.15.152:0/2558147062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6c480 0x55f2f4c8e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.923+0000 7f591218b640 1 -- 172.21.15.152:0/2558147062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c6bb60 msgr2=0x55f2f4c8e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:10.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.924+0000 7f591218b640 1 --2- 172.21.15.152:0/2558147062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c8e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.924+0000 7f591218b640 1 -- 172.21.15.152:0/2558147062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6c480 msgr2=0x55f2f4c8e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:10.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.924+0000 7f591218b640 1 --2- 172.21.15.152:0/2558147062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6c480 0x55f2f4c8e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.924+0000 7f591218b640 1 -- 172.21.15.152:0/2558147062 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2f4c76b70 con 0x55f2f4c76d40 2024-09-11T23:00:10.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.924+0000 7f591218b640 1 --2- 172.21.15.152:0/2558147062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c76d40 0x55f2f4c70db0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5904011880 tx=0x7f5904012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10913d2c1c49855d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:10.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.924+0000 7f58f77fe640 1 -- 172.21.15.152:0/2558147062 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904012d80 con 0x55f2f4c76d40 2024-09-11T23:00:10.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.924+0000 7f58f77fe640 1 -- 172.21.15.152:0/2558147062 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5904025070 con 0x55f2f4c76d40 2024-09-11T23:00:10.933 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.924+0000 7f58f77fe640 1 -- 172.21.15.152:0/2558147062 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590401f5f0 con 0x55f2f4c76d40 2024-09-11T23:00:10.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.925+0000 7f5915fc7740 1 -- 172.21.15.152:0/2558147062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c76d40 msgr2=0x55f2f4c70db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:10.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.925+0000 7f5915fc7740 1 --2- 172.21.15.152:0/2558147062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c76d40 0x55f2f4c70db0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5904011880 tx=0x7f5904012000 comp rx=0 tx=0).stop 2024-09-11T23:00:10.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.925+0000 7f5915fc7740 1 -- 172.21.15.152:0/2558147062 shutdown_connections 2024-09-11T23:00:10.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.925+0000 7f5915fc7740 1 --2- 172.21.15.152:0/2558147062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c76d40 0x55f2f4c70db0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.925+0000 7f5915fc7740 1 --2- 172.21.15.152:0/2558147062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6c480 0x55f2f4c8e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.925+0000 7f5915fc7740 1 --2- 172.21.15.152:0/2558147062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c8e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.925+0000 7f5915fc7740 1 -- 172.21.15.152:0/2558147062 >> 172.21.15.152:0/2558147062 conn(0x55f2f4c680f0 msgr2=0x55f2f4c72420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:10.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.925+0000 7f5915fc7740 1 -- 172.21.15.152:0/2558147062 shutdown_connections 2024-09-11T23:00:10.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.925+0000 7f5915fc7740 1 -- 172.21.15.152:0/2558147062 wait complete. 2024-09-11T23:00:10.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.925+0000 7f5915fc7740 1 -- start start 2024-09-11T23:00:10.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f5915fc7740 1 Processor -- start 2024-09-11T23:00:10.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f5915fc7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c7ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.934 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f5915fc7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6c480 0x55f2f4c7f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f5915fc7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c76d40 0x55f2f4c79cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f5915fc7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1b0ea0 con 0x55f2f4c6bb60 2024-09-11T23:00:10.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f5915fc7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1b07d0 con 0x55f2f4c6c480 2024-09-11T23:00:10.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f5915fc7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f590c1ada30 con 0x55f2f4c76d40 2024-09-11T23:00:10.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f5915fc7740 10 client.0 _ll_register_callbacks cb 0x7f58fc046fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:10.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f591198a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c7ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f591198a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c7ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36622/0 (socket says 172.21.15.152:36622) 2024-09-11T23:00:10.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f591198a640 1 -- 172.21.15.152:0/2061466333 learned_addr learned my addr 172.21.15.152:0/2061466333 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:10.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f5911189640 1 --2- 172.21.15.152:0/2061466333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6c480 0x55f2f4c7f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f591218b640 1 --2- 172.21.15.152:0/2061466333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c76d40 0x55f2f4c79cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f591198a640 1 -- 172.21.15.152:0/2061466333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c76d40 msgr2=0x55f2f4c79cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:10.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f591198a640 1 --2- 172.21.15.152:0/2061466333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c76d40 0x55f2f4c79cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.935 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f591198a640 1 -- 172.21.15.152:0/2061466333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6c480 msgr2=0x55f2f4c7f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:10.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f591198a640 1 --2- 172.21.15.152:0/2061466333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6c480 0x55f2f4c7f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.926+0000 7f591198a640 1 -- 172.21.15.152:0/2061466333 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2f4c7a210 con 0x55f2f4c6bb60 2024-09-11T23:00:10.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.927+0000 7f591198a640 1 --2- 172.21.15.152:0/2061466333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c7ed20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5908008c60 tx=0x7f5908008310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd40ba84942b6ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:10.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.927+0000 7f58f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:10.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.927+0000 7f58f57fa640 1 -- 172.21.15.152:0/2061466333 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908014040 con 0x55f2f4c6bb60 2024-09-11T23:00:10.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.927+0000 7f58f57fa640 1 -- 172.21.15.152:0/2061466333 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8536+0+0 (secure 0 0 0) 0x7f59080054d0 con 0x55f2f4c6bb60 2024-09-11T23:00:10.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.927+0000 7f5915fc7740 1 -- 172.21.15.152:0/2061466333 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5904025070 con 0x55f2f4c6bb60 2024-09-11T23:00:10.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.927+0000 7f58f57fa640 1 -- 172.21.15.152:0/2061466333 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f59080035f0 con 0x55f2f4c6bb60 2024-09-11T23:00:10.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.927+0000 7f5915fc7740 10 client.17538 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:10.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.927+0000 7f5915fc7740 1 -- 172.21.15.152:0/2061466333 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f2f4cf1c60 con 0x55f2f4c6bb60 2024-09-11T23:00:10.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.927+0000 7f58f57fa640 1 -- 172.21.15.152:0/2061466333 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5904025070 con 0x55f2f4c6bb60 2024-09-11T23:00:10.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.928+0000 7f58f57fa640 1 -- 172.21.15.152:0/2061466333 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f59080259c0 con 0x55f2f4c6bb60 2024-09-11T23:00:10.936 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.928+0000 7f5915fc7740 10 client.17538 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:10.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.928+0000 7f5915fc7740 10 client.17538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:00:10.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.928+0000 7f5915fc7740 20 client.17538 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:10.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.928+0000 7f5915fc7740 1 --2- 172.21.15.152:0/2061466333 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f2f4cf2130 0x55f2f4cf4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:10.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.928+0000 7f5915fc7740 4 client.17538 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:00:10.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.928+0000 7f5915fc7740 1 -- 172.21.15.152:0/2061466333 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f2f4bd8d00 con 0x55f2f4cf2130 2024-09-11T23:00:10.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.928+0000 7f5911189640 1 --2- 172.21.15.152:0/2061466333 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f2f4cf2130 0x55f2f4cf4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:10.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.928+0000 7f5911189640 1 --2- 172.21.15.152:0/2061466333 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f2f4cf2130 0x55f2f4cf4550 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80da162ad864777f server_cookie=88f77bd4bbe9e0f6 in_seq=0 out_seq=0 2024-09-11T23:00:10.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.928+0000 7f58f57fa640 10 client.17538 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:10.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.929+0000 7f58f57fa640 1 -- 172.21.15.152:0/2061466333 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f2f4bd8d00 con 0x55f2f4cf2130 2024-09-11T23:00:10.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.929+0000 7f58f57fa640 10 client.17538 handle_command_reply: tid=0 2024-09-11T23:00:10.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.929+0000 7f5915fc7740 1 client.17538 shutdown 2024-09-11T23:00:10.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.929+0000 7f5915fc7740 1 -- 172.21.15.152:0/2061466333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6bb60 msgr2=0x55f2f4c7ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:10.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.929+0000 7f5915fc7740 1 --2- 172.21.15.152:0/2061466333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c7ed20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f5908008c60 tx=0x7f5908008310 comp rx=0 tx=0).stop 2024-09-11T23:00:10.937 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.930+0000 7f5915fc7740 1 -- 172.21.15.152:0/2061466333 shutdown_connections 2024-09-11T23:00:10.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.930+0000 7f5915fc7740 1 --2- 172.21.15.152:0/2061466333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f2f4c76d40 0x55f2f4c79cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.930+0000 7f5915fc7740 1 --2- 172.21.15.152:0/2061466333 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55f2f4cf2130 0x55f2f4cf4550 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.930+0000 7f5915fc7740 1 --2- 172.21.15.152:0/2061466333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f2f4c6c480 0x55f2f4c7f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.930+0000 7f58f57fa640 0 client.17538 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:10.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.930+0000 7f5915fc7740 1 --2- 172.21.15.152:0/2061466333 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f2f4c6bb60 0x55f2f4c7ed20 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.930+0000 7f5915fc7740 1 -- 172.21.15.152:0/2061466333 >> 172.21.15.152:0/2061466333 conn(0x55f2f4c680f0 msgr2=0x55f2f4c6de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:10.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.930+0000 7f5915fc7740 1 -- 172.21.15.152:0/2061466333 shutdown_connections 2024-09-11T23:00:10.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.930+0000 7f5915fc7740 1 -- 172.21.15.152:0/2061466333 wait complete. 2024-09-11T23:00:10.938 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.930+0000 7f5915fc7740 20 client.17538 trim_cache size 0 max 16384 2024-09-11T23:00:10.938 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:00:10.939 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T23:00:10.939 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "615605be-1220-4876-b6aa-55e860123578", 2024-09-11T23:00:10.939 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T23:00:10.939 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:00:10.939 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:10 smithi191 ceph-mon[35211]: pgmap v222: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:00:10.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.931+0000 7f5913bee640 1 -- 172.21.15.152:0/3460158930 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58f80a2cf0 msgr2=0x7f58f80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:10.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.931+0000 7f5913bee640 1 --2- 172.21.15.152:0/3460158930 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58f80a2cf0 0x7f58f80a51b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5904010500 tx=0x7f5904004a40 comp rx=0 tx=0).stop 2024-09-11T23:00:10.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.931+0000 7f5913bee640 1 -- 172.21.15.152:0/3460158930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f590c102790 msgr2=0x7f590c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:10.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.931+0000 7f5913bee640 1 --2- 172.21.15.152:0/3460158930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f590c102790 0x7f590c074a30 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f58fc005ab0 tx=0x7f58fc0042f0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.931+0000 7f5913bee640 1 -- 172.21.15.152:0/3460158930 shutdown_connections 2024-09-11T23:00:10.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.931+0000 7f5913bee640 1 --2- 172.21.15.152:0/3460158930 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58f80a2cf0 0x7f58f80a51b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.931+0000 7f5913bee640 1 --2- 172.21.15.152:0/3460158930 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f590c106f10 0x7f590c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.940 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.931+0000 7f5913bee640 1 --2- 172.21.15.152:0/3460158930 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f590c102790 0x7f590c074a30 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.931+0000 7f5913bee640 1 --2- 172.21.15.152:0/3460158930 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f590c101de0 0x7f590c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:10.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.931+0000 7f5913bee640 1 -- 172.21.15.152:0/3460158930 >> 172.21.15.152:0/3460158930 conn(0x7f590c0fd8c0 msgr2=0x7f590c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:10.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.931+0000 7f5913bee640 1 -- 172.21.15.152:0/3460158930 shutdown_connections 2024-09-11T23:00:10.941 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:10.931+0000 7f5913bee640 1 -- 172.21.15.152:0/3460158930 wait complete. 2024-09-11T23:00:10.947 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:10 smithi204 ceph-mon[35454]: pgmap v222: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:00:11.752 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:11 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T23:00:11.752 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:11 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T23:00:11.752 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:11 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T23:00:11.752 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:11 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T23:00:11.753 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:00:12.003 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:12.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:11 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T23:00:12.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:11 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T23:00:12.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:11 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T23:00:12.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:11 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T23:00:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:11 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T23:00:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:11 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T23:00:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:11 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T23:00:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:11 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T23:00:12.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.508+0000 7f33d1eef640 1 -- 172.21.15.152:0/3640114275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33cc10c180 msgr2=0x7f33cc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:12.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.508+0000 7f33d1eef640 1 --2- 172.21.15.152:0/3640114275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33cc10c180 0x7f33cc10e610 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009900 tx=0x7f33c402a480 comp rx=0 tx=0).stop 2024-09-11T23:00:12.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.509+0000 7f33d1eef640 1 -- 172.21.15.152:0/3640114275 shutdown_connections 2024-09-11T23:00:12.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.509+0000 7f33d1eef640 1 --2- 172.21.15.152:0/3640114275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33cc10c180 0x7f33cc10e610 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.509+0000 7f33d1eef640 1 --2- 172.21.15.152:0/3640114275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33cc103700 0x7f33cc10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.509+0000 7f33d1eef640 1 --2- 172.21.15.152:0/3640114275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33cc102de0 0x7f33cc1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.509+0000 7f33d1eef640 1 -- 172.21.15.152:0/3640114275 >> 172.21.15.152:0/3640114275 conn(0x7f33cc0fc7a0 msgr2=0x7f33cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.509+0000 7f33d1eef640 1 -- 172.21.15.152:0/3640114275 shutdown_connections 2024-09-11T23:00:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.509+0000 7f33d1eef640 1 -- 172.21.15.152:0/3640114275 wait complete. 2024-09-11T23:00:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.509+0000 7f33d1eef640 1 Processor -- start 2024-09-11T23:00:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.509+0000 7f33d1eef640 1 -- start start 2024-09-11T23:00:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33d1eef640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33cc102de0 0x7f33cc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33d1eef640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33cc103700 0x7f33cc198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33d1eef640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33cc10c180 0x7f33cc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33d1eef640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc111fb0 con 0x7f33cc10c180 2024-09-11T23:00:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33d1eef640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc111e30 con 0x7f33cc102de0 2024-09-11T23:00:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33d1eef640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc112130 con 0x7f33cc103700 2024-09-11T23:00:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33cbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33cc10c180 0x7f33cc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33cbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33cc10c180 0x7f33cc19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36646/0 (socket says 172.21.15.152:36646) 2024-09-11T23:00:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33cbfff640 1 -- 172.21.15.152:0/1070369492 learned_addr learned my addr 172.21.15.152:0/1070369492 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33cb7fe640 1 --2- 172.21.15.152:0/1070369492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33cc102de0 0x7f33cc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33caffd640 1 --2- 172.21.15.152:0/1070369492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33cc103700 0x7f33cc198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33cbfff640 1 -- 172.21.15.152:0/1070369492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33cc103700 msgr2=0x7f33cc198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33cbfff640 1 --2- 172.21.15.152:0/1070369492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33cc103700 0x7f33cc198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33cbfff640 1 -- 172.21.15.152:0/1070369492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33cc102de0 msgr2=0x7f33cc1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33cbfff640 1 --2- 172.21.15.152:0/1070369492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33cc102de0 0x7f33cc1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33cbfff640 1 -- 172.21.15.152:0/1070369492 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc19d3a0 con 0x7f33cc10c180 2024-09-11T23:00:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33cb7fe640 1 --2- 172.21.15.152:0/1070369492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33cc102de0 0x7f33cc1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33cbfff640 1 --2- 172.21.15.152:0/1070369492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33cc10c180 0x7f33cc19ccc0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009230 tx=0x7f33c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33c8ff9640 1 -- 172.21.15.152:0/1070369492 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c403f030 con 0x7f33cc10c180 2024-09-11T23:00:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33d1eef640 1 -- 172.21.15.152:0/1070369492 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33cc19d630 con 0x7f33cc10c180 2024-09-11T23:00:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.510+0000 7f33c8ff9640 1 -- 172.21.15.152:0/1070369492 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33c4003ce0 con 0x7f33cc10c180 2024-09-11T23:00:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.511+0000 7f33c8ff9640 1 -- 172.21.15.152:0/1070369492 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4035070 con 0x7f33cc10c180 2024-09-11T23:00:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.511+0000 7f33d1eef640 1 -- 172.21.15.152:0/1070369492 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33cc1a4f70 con 0x7f33cc10c180 2024-09-11T23:00:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.513+0000 7f33c8ff9640 1 -- 172.21.15.152:0/1070369492 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f33c4042030 con 0x7f33cc10c180 2024-09-11T23:00:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.513+0000 7f33d42c8740 1 Processor -- start 2024-09-11T23:00:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.513+0000 7f33d42c8740 1 -- start start 2024-09-11T23:00:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.513+0000 7f33c8ff9640 1 --2- 172.21.15.152:0/1070369492 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f33b40a2cf0 0x7f33b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.513+0000 7f33d42c8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce32665880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.513+0000 7f33d42c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce32665dc0 0x55ce32670140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.513+0000 7f33d42c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32670680 0x55ce32672a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.513+0000 7f33d42c8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ce325cb2e0 con 0x55ce326654a0 2024-09-11T23:00:12.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.513+0000 7f33d42c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ce32572ce0 con 0x55ce32670680 2024-09-11T23:00:12.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.513+0000 7f33d42c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ce325c72a0 con 0x55ce32665dc0 2024-09-11T23:00:12.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.513+0000 7f33caffd640 1 --2- 172.21.15.152:0/1070369492 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f33b40a2cf0 0x7f33b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.514+0000 7f33cb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce32665880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.514+0000 7f33c8ff9640 1 -- 172.21.15.152:0/1070369492 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8536+0+0 (secure 0 0 0) 0x7f33c40efe20 con 0x7f33cc10c180 2024-09-11T23:00:12.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.514+0000 7f33cb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce32665880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36650/0 (socket says 172.21.15.152:36650) 2024-09-11T23:00:12.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.514+0000 7f33cb7fe640 1 -- 172.21.15.152:0/1667934002 learned_addr learned my addr 172.21.15.152:0/1667934002 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:12.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.514+0000 7f33cbfff640 1 --2- 172.21.15.152:0/1667934002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32670680 0x55ce32672a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.514+0000 7f33caffd640 1 --2- 172.21.15.152:0/1667934002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce32665dc0 0x55ce32670140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.514+0000 7f33cb7fe640 1 -- 172.21.15.152:0/1667934002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce32665dc0 msgr2=0x55ce32670140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:12.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.514+0000 7f33cb7fe640 1 --2- 172.21.15.152:0/1667934002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce32665dc0 0x55ce32670140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.514+0000 7f33cb7fe640 1 -- 172.21.15.152:0/1667934002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32670680 msgr2=0x55ce32672a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:12.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.514+0000 7f33cb7fe640 1 --2- 172.21.15.152:0/1667934002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32670680 0x55ce32672a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.514+0000 7f33cb7fe640 1 -- 172.21.15.152:0/1667934002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce32557440 con 0x55ce326654a0 2024-09-11T23:00:12.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.514+0000 7f33caffd640 1 --2- 172.21.15.152:0/1070369492 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f33b40a2cf0 0x7f33b40a51b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f33c0006e10 tx=0x7f33c0009e50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:12.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.514+0000 7f33cb7fe640 1 --2- 172.21.15.152:0/1667934002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce32665880 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f33bc007f70 tx=0x7f33bc00e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d0a58ed10917fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:12.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.514+0000 7f33b1ffb640 1 -- 172.21.15.152:0/1667934002 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc014070 con 0x55ce326654a0 2024-09-11T23:00:12.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.514+0000 7f33b1ffb640 1 -- 172.21.15.152:0/1667934002 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33bc004d20 con 0x55ce326654a0 2024-09-11T23:00:12.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.514+0000 7f33b1ffb640 1 -- 172.21.15.152:0/1667934002 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc005560 con 0x55ce326654a0 2024-09-11T23:00:12.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.515+0000 7f33d42c8740 1 -- 172.21.15.152:0/1667934002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 msgr2=0x55ce32665880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:12.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.515+0000 7f33d42c8740 1 --2- 172.21.15.152:0/1667934002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce32665880 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f33bc007f70 tx=0x7f33bc00e7e0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.515+0000 7f33d42c8740 1 -- 172.21.15.152:0/1667934002 shutdown_connections 2024-09-11T23:00:12.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.515+0000 7f33d42c8740 1 --2- 172.21.15.152:0/1667934002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32670680 0x55ce32672a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.515+0000 7f33d42c8740 1 --2- 172.21.15.152:0/1667934002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce32665dc0 0x55ce32670140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.515+0000 7f33d42c8740 1 --2- 172.21.15.152:0/1667934002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce32665880 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.515+0000 7f33d42c8740 1 -- 172.21.15.152:0/1667934002 >> 172.21.15.152:0/1667934002 conn(0x55ce32661a30 msgr2=0x55ce32661e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.515+0000 7f33d42c8740 1 -- 172.21.15.152:0/1667934002 shutdown_connections 2024-09-11T23:00:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.515+0000 7f33d42c8740 1 -- 172.21.15.152:0/1667934002 wait complete. 2024-09-11T23:00:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.515+0000 7f33d42c8740 1 -- start start 2024-09-11T23:00:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33d42c8740 1 Processor -- start 2024-09-11T23:00:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33d42c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce326654a0 0x55ce3266ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33d42c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32665dc0 0x55ce3266d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33d42c8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce32670680 0x55ce3266d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33d42c8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ce32673250 con 0x55ce32670680 2024-09-11T23:00:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33d42c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ce32586050 con 0x55ce32665dc0 2024-09-11T23:00:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33d42c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ce326733d0 con 0x55ce326654a0 2024-09-11T23:00:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33d42c8740 10 client.0 _ll_register_callbacks cb 0x55ce325c6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33cbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce32670680 0x55ce3266d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33cbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce32670680 0x55ce3266d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36658/0 (socket says 172.21.15.152:36658) 2024-09-11T23:00:12.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33cbfff640 1 -- 172.21.15.152:0/961276522 learned_addr learned my addr 172.21.15.152:0/961276522 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:12.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33cb7fe640 1 --2- 172.21.15.152:0/961276522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce326654a0 0x55ce3266ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33caffd640 1 --2- 172.21.15.152:0/961276522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32665dc0 0x55ce3266d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33cbfff640 1 -- 172.21.15.152:0/961276522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce326654a0 msgr2=0x55ce3266ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:12.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33cbfff640 1 --2- 172.21.15.152:0/961276522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce326654a0 0x55ce3266ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33cbfff640 1 -- 172.21.15.152:0/961276522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32665dc0 msgr2=0x55ce3266d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:12.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33cbfff640 1 --2- 172.21.15.152:0/961276522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32665dc0 0x55ce3266d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33cbfff640 1 -- 172.21.15.152:0/961276522 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce3233c3c0 con 0x55ce32670680 2024-09-11T23:00:12.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33cbfff640 1 --2- 172.21.15.152:0/961276522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce32670680 0x55ce3266d710 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f33c40b8d00 tx=0x7f33c4003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d67a778caaa1643 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:12.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.516+0000 7f33aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:12.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.517+0000 7f33aaffd640 1 -- 172.21.15.152:0/961276522 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4049040 con 0x55ce32670680 2024-09-11T23:00:12.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.517+0000 7f33aaffd640 1 -- 172.21.15.152:0/961276522 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8536+0+0 (secure 0 0 0) 0x7f33c40ed7e0 con 0x55ce32670680 2024-09-11T23:00:12.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.517+0000 7f33d42c8740 1 -- 172.21.15.152:0/961276522 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33bc004d20 con 0x55ce32670680 2024-09-11T23:00:12.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.517+0000 7f33aaffd640 1 -- 172.21.15.152:0/961276522 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33c40bbc90 con 0x55ce32670680 2024-09-11T23:00:12.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.517+0000 7f33d42c8740 10 client.17550 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:12.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.517+0000 7f33d42c8740 1 -- 172.21.15.152:0/961276522 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ce3267eb70 con 0x55ce32670680 2024-09-11T23:00:12.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.517+0000 7f33aaffd640 1 -- 172.21.15.152:0/961276522 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33bc004d20 con 0x55ce32670680 2024-09-11T23:00:12.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.517+0000 7f33aaffd640 1 -- 172.21.15.152:0/961276522 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f33c40b89a0 con 0x55ce32670680 2024-09-11T23:00:12.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.517+0000 7f33d42c8740 10 client.17550 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:12.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.517+0000 7f33d42c8740 10 client.17550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:00:12.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.517+0000 7f33d42c8740 20 client.17550 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:12.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.517+0000 7f33d42c8740 1 --2- 172.21.15.152:0/961276522 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ce32680020 0x55ce32682490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.517+0000 7f33d42c8740 4 client.17550 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:00:12.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.517+0000 7f33d42c8740 1 -- 172.21.15.152:0/961276522 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce325d8850 con 0x55ce32680020 2024-09-11T23:00:12.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.518+0000 7f33cb7fe640 1 --2- 172.21.15.152:0/961276522 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ce32680020 0x55ce32682490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.518+0000 7f33cb7fe640 1 --2- 172.21.15.152:0/961276522 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ce32680020 0x55ce32682490 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f52f053def6d2d7 server_cookie=8fe341bde2579a0f in_seq=0 out_seq=0 2024-09-11T23:00:12.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.518+0000 7f33aaffd640 10 client.17550 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:12.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.519+0000 7f33aaffd640 1 -- 172.21.15.152:0/961276522 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce325d8850 con 0x55ce32680020 2024-09-11T23:00:12.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.519+0000 7f33aaffd640 10 client.17550 handle_command_reply: tid=0 2024-09-11T23:00:12.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.519+0000 7f33d42c8740 1 client.17550 shutdown 2024-09-11T23:00:12.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.519+0000 7f33d42c8740 1 -- 172.21.15.152:0/961276522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce32670680 msgr2=0x55ce3266d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:12.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.519+0000 7f33d42c8740 1 --2- 172.21.15.152:0/961276522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce32670680 0x55ce3266d710 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f33c40b8d00 tx=0x7f33c4003f40 comp rx=0 tx=0).stop 2024-09-11T23:00:12.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.519+0000 7f33d42c8740 1 -- 172.21.15.152:0/961276522 shutdown_connections 2024-09-11T23:00:12.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.519+0000 7f33d42c8740 1 --2- 172.21.15.152:0/961276522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce32670680 0x55ce3266d710 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.519+0000 7f33d42c8740 1 --2- 172.21.15.152:0/961276522 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ce32680020 0x55ce32682490 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.519+0000 7f33d42c8740 1 --2- 172.21.15.152:0/961276522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32665dc0 0x55ce3266d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.519+0000 7f33d42c8740 1 --2- 172.21.15.152:0/961276522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce326654a0 0x55ce3266ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.519+0000 7f33d42c8740 1 -- 172.21.15.152:0/961276522 >> 172.21.15.152:0/961276522 conn(0x55ce32661a30 msgr2=0x55ce32660b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.519+0000 7f33aaffd640 0 client.17550 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.519+0000 7f33d42c8740 1 -- 172.21.15.152:0/961276522 shutdown_connections 2024-09-11T23:00:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.519+0000 7f33d42c8740 1 -- 172.21.15.152:0/961276522 wait complete. 2024-09-11T23:00:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.519+0000 7f33d42c8740 20 client.17550 trim_cache size 0 max 16384 2024-09-11T23:00:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33d42c8740 1 Processor -- start 2024-09-11T23:00:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33d42c8740 1 -- start start 2024-09-11T23:00:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33d42c8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce3267c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33d42c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce32665dc0 0x55ce32668610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33d42c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32670680 0x55ce32687da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33d42c8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1ad7e0 con 0x55ce326654a0 2024-09-11T23:00:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33d42c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1af320 con 0x55ce32670680 2024-09-11T23:00:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33d42c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1adeb0 con 0x55ce32665dc0 2024-09-11T23:00:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33cb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce3267c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33cb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce3267c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36668/0 (socket says 172.21.15.152:36668) 2024-09-11T23:00:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33cb7fe640 1 -- 172.21.15.152:0/1316665836 learned_addr learned my addr 172.21.15.152:0/1316665836 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33cbfff640 1 --2- 172.21.15.152:0/1316665836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32670680 0x55ce32687da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33cb7fe640 1 -- 172.21.15.152:0/1316665836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce32665dc0 msgr2=0x55ce32668610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33cb7fe640 1 --2- 172.21.15.152:0/1316665836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce32665dc0 0x55ce32668610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33cb7fe640 1 -- 172.21.15.152:0/1316665836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32670680 msgr2=0x55ce32687da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33cb7fe640 1 --2- 172.21.15.152:0/1316665836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32670680 0x55ce32687da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33cb7fe640 1 -- 172.21.15.152:0/1316665836 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce3266b720 con 0x55ce326654a0 2024-09-11T23:00:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33cbfff640 1 --2- 172.21.15.152:0/1316665836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32670680 0x55ce32687da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:00:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.536+0000 7f33cb7fe640 1 --2- 172.21.15.152:0/1316665836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce3267c4b0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f33bc007f70 tx=0x7f33bc005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace518c6b5e97b6a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.537+0000 7f33b17fa640 1 -- 172.21.15.152:0/1316665836 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc029020 con 0x55ce326654a0 2024-09-11T23:00:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.537+0000 7f33b17fa640 1 -- 172.21.15.152:0/1316665836 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33bc003dd0 con 0x55ce326654a0 2024-09-11T23:00:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.537+0000 7f33b17fa640 1 -- 172.21.15.152:0/1316665836 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc014050 con 0x55ce326654a0 2024-09-11T23:00:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.537+0000 7f33d42c8740 1 -- 172.21.15.152:0/1316665836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 msgr2=0x55ce3267c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.537+0000 7f33d42c8740 1 --2- 172.21.15.152:0/1316665836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce3267c4b0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f33bc007f70 tx=0x7f33bc005160 comp rx=0 tx=0).stop 2024-09-11T23:00:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.537+0000 7f33d42c8740 1 -- 172.21.15.152:0/1316665836 shutdown_connections 2024-09-11T23:00:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.537+0000 7f33d42c8740 1 --2- 172.21.15.152:0/1316665836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32670680 0x55ce32687da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.537+0000 7f33d42c8740 1 --2- 172.21.15.152:0/1316665836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce32665dc0 0x55ce32668610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.537+0000 7f33d42c8740 1 --2- 172.21.15.152:0/1316665836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce3267c4b0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.537+0000 7f33d42c8740 1 -- 172.21.15.152:0/1316665836 >> 172.21.15.152:0/1316665836 conn(0x55ce32661a30 msgr2=0x55ce32666ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.537+0000 7f33d42c8740 1 -- 172.21.15.152:0/1316665836 shutdown_connections 2024-09-11T23:00:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.537+0000 7f33d42c8740 1 -- 172.21.15.152:0/1316665836 wait complete. 2024-09-11T23:00:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.538+0000 7f33d42c8740 1 -- start start 2024-09-11T23:00:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.538+0000 7f33d42c8740 1 Processor -- start 2024-09-11T23:00:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.538+0000 7f33d42c8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce32678e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.538+0000 7f33d42c8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce32665dc0 0x55ce32674130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.538+0000 7f33d42c8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32670680 0x55ce32674690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.538+0000 7f33d42c8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1aec50 con 0x55ce326654a0 2024-09-11T23:00:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.538+0000 7f33d42c8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1ae580 con 0x55ce32670680 2024-09-11T23:00:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.538+0000 7f33d42c8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc1ab7e0 con 0x55ce32665dc0 2024-09-11T23:00:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.538+0000 7f33d42c8740 10 client.0 _ll_register_callbacks cb 0x7f33c4046fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.538+0000 7f33cb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce32678e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.538+0000 7f33cb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce32678e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36676/0 (socket says 172.21.15.152:36676) 2024-09-11T23:00:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.538+0000 7f33cb7fe640 1 -- 172.21.15.152:0/1407671818 learned_addr learned my addr 172.21.15.152:0/1407671818 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.538+0000 7f33caffd640 1 --2- 172.21.15.152:0/1407671818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce32665dc0 0x55ce32674130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.538+0000 7f33cbfff640 1 --2- 172.21.15.152:0/1407671818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32670680 0x55ce32674690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.538+0000 7f33cb7fe640 1 -- 172.21.15.152:0/1407671818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce32665dc0 msgr2=0x55ce32674130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.538+0000 7f33cb7fe640 1 --2- 172.21.15.152:0/1407671818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce32665dc0 0x55ce32674130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33cb7fe640 1 -- 172.21.15.152:0/1407671818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32670680 msgr2=0x55ce32674690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33cb7fe640 1 --2- 172.21.15.152:0/1407671818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32670680 0x55ce32674690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33cb7fe640 1 -- 172.21.15.152:0/1407671818 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce32674ec0 con 0x55ce326654a0 2024-09-11T23:00:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33caffd640 1 --2- 172.21.15.152:0/1407671818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce32665dc0 0x55ce32674130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33cb7fe640 1 --2- 172.21.15.152:0/1407671818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce32678e00 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f33bc007c80 tx=0x7f33bc003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2027494167bd43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33aaffd640 1 -- 172.21.15.152:0/1407671818 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc029040 con 0x55ce326654a0 2024-09-11T23:00:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33aaffd640 1 -- 172.21.15.152:0/1407671818 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8536+0+0 (secure 0 0 0) 0x7f33bc0058c0 con 0x55ce326654a0 2024-09-11T23:00:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33d42c8740 1 -- 172.21.15.152:0/1407671818 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33bc003dd0 con 0x55ce326654a0 2024-09-11T23:00:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33aaffd640 1 -- 172.21.15.152:0/1407671818 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33bc014020 con 0x55ce326654a0 2024-09-11T23:00:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33d42c8740 10 client.17556 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33d42c8740 1 -- 172.21.15.152:0/1407671818 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ce326ead80 con 0x55ce326654a0 2024-09-11T23:00:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33aaffd640 1 -- 172.21.15.152:0/1407671818 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33bc003dd0 con 0x55ce326654a0 2024-09-11T23:00:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33aaffd640 1 -- 172.21.15.152:0/1407671818 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f33bc02e7b0 con 0x55ce326654a0 2024-09-11T23:00:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33d42c8740 10 client.17556 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33d42c8740 10 client.17556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:00:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33d42c8740 20 client.17556 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33d42c8740 1 --2- 172.21.15.152:0/1407671818 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ce326eb240 0x55ce326ed630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33d42c8740 4 client.17556 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:00:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.539+0000 7f33d42c8740 1 -- 172.21.15.152:0/1407671818 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ce325d2d60 con 0x55ce326eb240 2024-09-11T23:00:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.540+0000 7f33caffd640 1 --2- 172.21.15.152:0/1407671818 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ce326eb240 0x55ce326ed630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.540+0000 7f33caffd640 1 --2- 172.21.15.152:0/1407671818 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ce326eb240 0x55ce326ed630 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fddcab2fcd66e269 server_cookie=65dc29899ae70ef9 in_seq=0 out_seq=0 2024-09-11T23:00:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.540+0000 7f33aaffd640 10 client.17556 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.540+0000 7f33aaffd640 1 -- 172.21.15.152:0/1407671818 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ce325d2d60 con 0x55ce326eb240 2024-09-11T23:00:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.540+0000 7f33aaffd640 10 client.17556 handle_command_reply: tid=0 2024-09-11T23:00:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.540+0000 7f33d42c8740 1 client.17556 shutdown 2024-09-11T23:00:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d42c8740 1 -- 172.21.15.152:0/1407671818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 msgr2=0x55ce32678e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d42c8740 1 --2- 172.21.15.152:0/1407671818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce32678e00 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f33bc007c80 tx=0x7f33bc003a00 comp rx=0 tx=0).stop 2024-09-11T23:00:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d42c8740 1 -- 172.21.15.152:0/1407671818 shutdown_connections 2024-09-11T23:00:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d42c8740 1 --2- 172.21.15.152:0/1407671818 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ce326eb240 0x55ce326ed630 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d42c8740 1 --2- 172.21.15.152:0/1407671818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ce32670680 0x55ce32674690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d42c8740 1 --2- 172.21.15.152:0/1407671818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce32665dc0 0x55ce32674130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33aaffd640 0 client.17556 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d42c8740 1 --2- 172.21.15.152:0/1407671818 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ce326654a0 0x55ce32678e00 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d42c8740 1 -- 172.21.15.152:0/1407671818 >> 172.21.15.152:0/1407671818 conn(0x55ce32661a30 msgr2=0x55ce3266f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d42c8740 1 -- 172.21.15.152:0/1407671818 shutdown_connections 2024-09-11T23:00:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d42c8740 1 -- 172.21.15.152:0/1407671818 wait complete. 2024-09-11T23:00:12.578 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:00:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:00:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": { 2024-09-11T23:00:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "615605be-1220-4876-b6aa-55e860123578": { 2024-09-11T23:00:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "path": "/", 2024-09-11T23:00:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "tag": "615605be-1220-4876-b6aa-55e860123578", 2024-09-11T23:00:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "options": "recursive,force" 2024-09-11T23:00:12.579 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:00:12.579 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:00:12.579 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:00:12.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d42c8740 20 client.17556 trim_cache size 0 max 16384 2024-09-11T23:00:12.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d1eef640 1 -- 172.21.15.152:0/1070369492 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f33b40a2cf0 msgr2=0x7f33b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:12.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d1eef640 1 --2- 172.21.15.152:0/1070369492 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f33b40a2cf0 0x7f33b40a51b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f33c0006e10 tx=0x7f33c0009e50 comp rx=0 tx=0).stop 2024-09-11T23:00:12.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d1eef640 1 -- 172.21.15.152:0/1070369492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33cc10c180 msgr2=0x7f33cc19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:12.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d1eef640 1 --2- 172.21.15.152:0/1070369492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33cc10c180 0x7f33cc19ccc0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009230 tx=0x7f33c40042f0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d1eef640 1 -- 172.21.15.152:0/1070369492 shutdown_connections 2024-09-11T23:00:12.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d1eef640 1 --2- 172.21.15.152:0/1070369492 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f33b40a2cf0 0x7f33b40a51b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d1eef640 1 --2- 172.21.15.152:0/1070369492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f33cc10c180 0x7f33cc19ccc0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d1eef640 1 --2- 172.21.15.152:0/1070369492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33cc103700 0x7f33cc198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d1eef640 1 --2- 172.21.15.152:0/1070369492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33cc102de0 0x7f33cc1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:12.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.541+0000 7f33d1eef640 1 -- 172.21.15.152:0/1070369492 >> 172.21.15.152:0/1070369492 conn(0x7f33cc0fc7a0 msgr2=0x7f33cc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:12.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.542+0000 7f33d1eef640 1 -- 172.21.15.152:0/1070369492 shutdown_connections 2024-09-11T23:00:12.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:12.542+0000 7f33d1eef640 1 -- 172.21.15.152:0/1070369492 wait complete. 2024-09-11T23:00:12.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:12 smithi152 ceph-mon[31181]: pgmap v223: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:00:13.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:12 smithi191 ceph-mon[35211]: pgmap v223: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:00:13.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:12 smithi204 ceph-mon[35454]: pgmap v223: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:00:13.324 INFO:tasks.cephfs.filesystem:scrub status for tag:615605be-1220-4876-b6aa-55e860123578 - {'path': '/', 'tag': '615605be-1220-4876-b6aa-55e860123578', 'options': 'recursive,force'} 2024-09-11T23:00:13.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:13 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T23:00:14.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:13 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T23:00:14.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:13 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T23:00:14.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:14 smithi152 ceph-mon[31181]: pgmap v224: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:00:15.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:14 smithi191 ceph-mon[35211]: pgmap v224: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:00:15.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:14 smithi204 ceph-mon[35454]: pgmap v224: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:00:16.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:16 smithi191 ceph-mon[35211]: pgmap v225: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:00:16.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:16 smithi191 ceph-mon[35211]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T23:00:16.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:16 smithi204 ceph-mon[35454]: pgmap v225: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:00:16.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:16 smithi204 ceph-mon[35454]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T23:00:16.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:16 smithi152 ceph-mon[31181]: pgmap v225: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:00:16.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:16 smithi152 ceph-mon[31181]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T23:00:18.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:18 smithi191 ceph-mon[35211]: pgmap v227: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 237 B/s wr, 13 op/s 2024-09-11T23:00:18.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:18 smithi204 ceph-mon[35454]: pgmap v227: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 237 B/s wr, 13 op/s 2024-09-11T23:00:18.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:18 smithi152 ceph-mon[31181]: pgmap v227: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 237 B/s wr, 13 op/s 2024-09-11T23:00:20.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:20 smithi191 ceph-mon[35211]: pgmap v228: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:00:20.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:20 smithi204 ceph-mon[35454]: pgmap v228: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:00:20.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:20 smithi152 ceph-mon[31181]: pgmap v228: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:00:22.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:22 smithi191 ceph-mon[35211]: pgmap v229: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:00:22.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:22 smithi204 ceph-mon[35454]: pgmap v229: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:00:22.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:22 smithi152 ceph-mon[31181]: pgmap v229: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:00:24.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:24 smithi191 ceph-mon[35211]: pgmap v230: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:00:24.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:24 smithi204 ceph-mon[35454]: pgmap v230: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:00:24.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:24 smithi152 ceph-mon[31181]: pgmap v230: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:00:26.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:26 smithi191 ceph-mon[35211]: pgmap v231: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:00:26.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:26 smithi204 ceph-mon[35454]: pgmap v231: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:00:26.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:26 smithi152 ceph-mon[31181]: pgmap v231: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:00:28.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:28 smithi191 ceph-mon[35211]: pgmap v232: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:28.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:28 smithi204 ceph-mon[35454]: pgmap v232: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:28.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:28 smithi152 ceph-mon[31181]: pgmap v232: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:30.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:30 smithi191 ceph-mon[35211]: pgmap v233: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-11T23:00:30.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:30 smithi204 ceph-mon[35454]: pgmap v233: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-11T23:00:30.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:30 smithi152 ceph-mon[31181]: pgmap v233: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-11T23:00:32.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:32 smithi191 ceph-mon[35211]: pgmap v234: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:00:32.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:32 smithi204 ceph-mon[35454]: pgmap v234: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:00:32.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:32 smithi152 ceph-mon[31181]: pgmap v234: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:00:34.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:34 smithi191 ceph-mon[35211]: pgmap v235: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:34.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:34 smithi204 ceph-mon[35454]: pgmap v235: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:34.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:34 smithi152 ceph-mon[31181]: pgmap v235: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:36.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:36 smithi191 ceph-mon[35211]: pgmap v236: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:00:36.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:36 smithi204 ceph-mon[35454]: pgmap v236: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:00:36.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:36 smithi152 ceph-mon[31181]: pgmap v236: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:00:38.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:38 smithi204 ceph-mon[35454]: pgmap v237: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:38.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:38 smithi152 ceph-mon[31181]: pgmap v237: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:38.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:38 smithi191 ceph-mon[35211]: pgmap v237: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:40.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:40 smithi204 ceph-mon[35454]: pgmap v238: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-11T23:00:40.589 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:40 smithi191 ceph-mon[35211]: pgmap v238: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-11T23:00:40.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:40 smithi152 ceph-mon[31181]: pgmap v238: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-11T23:00:42.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:42 smithi204 ceph-mon[35454]: pgmap v239: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:00:42.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:42 smithi152 ceph-mon[31181]: pgmap v239: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:00:42.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:42 smithi191 ceph-mon[35211]: pgmap v239: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:00:43.326 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:00:43.583 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:44.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.156+0000 7f0c1ecd1640 1 -- 172.21.15.152:0/2344859657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c18106a80 msgr2=0x7f0c18106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:44.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.156+0000 7f0c1ecd1640 1 --2- 172.21.15.152:0/2344859657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c18106a80 0x7f0c18106e60 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009cd0 tx=0x7f0c1002a8e0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.156+0000 7f0c1ecd1640 1 -- 172.21.15.152:0/2344859657 shutdown_connections 2024-09-11T23:00:44.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.156+0000 7f0c1ecd1640 1 --2- 172.21.15.152:0/2344859657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c18101420 0x7f0c1810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.156+0000 7f0c1ecd1640 1 --2- 172.21.15.152:0/2344859657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c18100a80 0x7f0c18100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.156+0000 7f0c1ecd1640 1 --2- 172.21.15.152:0/2344859657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c18106a80 0x7f0c18106e60 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.156+0000 7f0c1ecd1640 1 -- 172.21.15.152:0/2344859657 >> 172.21.15.152:0/2344859657 conn(0x7f0c180fc7a0 msgr2=0x7f0c180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:44.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.156+0000 7f0c1ecd1640 1 -- 172.21.15.152:0/2344859657 shutdown_connections 2024-09-11T23:00:44.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.156+0000 7f0c1ecd1640 1 -- 172.21.15.152:0/2344859657 wait complete. 2024-09-11T23:00:44.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.156+0000 7f0c1ecd1640 1 Processor -- start 2024-09-11T23:00:44.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.156+0000 7f0c1ecd1640 1 -- start start 2024-09-11T23:00:44.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c1ecd1640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c18100a80 0x7f0c181992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c1ecd1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c18101420 0x7f0c18199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c1ecd1640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c18106a80 0x7f0c181a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c1ecd1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0c18110e80 con 0x7f0c18101420 2024-09-11T23:00:44.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c1ecd1640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0c18110d00 con 0x7f0c18106a80 2024-09-11T23:00:44.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c1ecd1640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c18111000 con 0x7f0c18100a80 2024-09-11T23:00:44.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c17fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c18101420 0x7f0c18199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c17fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c18101420 0x7f0c18199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40366/0 (socket says 172.21.15.152:40366) 2024-09-11T23:00:44.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c17fff640 1 -- 172.21.15.152:0/540015503 learned_addr learned my addr 172.21.15.152:0/540015503 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:44.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c1ca6d640 1 --2- 172.21.15.152:0/540015503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c18100a80 0x7f0c181992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c1d26e640 1 --2- 172.21.15.152:0/540015503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c18106a80 0x7f0c181a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c17fff640 1 -- 172.21.15.152:0/540015503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c18100a80 msgr2=0x7f0c181992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:44.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c17fff640 1 --2- 172.21.15.152:0/540015503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c18100a80 0x7f0c181992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c17fff640 1 -- 172.21.15.152:0/540015503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c18106a80 msgr2=0x7f0c181a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:44.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c17fff640 1 --2- 172.21.15.152:0/540015503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c18106a80 0x7f0c181a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c17fff640 1 -- 172.21.15.152:0/540015503 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c181a48b0 con 0x7f0c18101420 2024-09-11T23:00:44.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c1d26e640 1 --2- 172.21.15.152:0/540015503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c18106a80 0x7f0c181a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:44.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c17fff640 1 --2- 172.21.15.152:0/540015503 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c18101420 0x7f0c18199830 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00e9f0 tx=0x7f0c0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:44.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.157+0000 7f0c15ffb640 1 -- 172.21.15.152:0/540015503 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c00cd80 con 0x7f0c18101420 2024-09-11T23:00:44.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.158+0000 7f0c1ecd1640 1 -- 172.21.15.152:0/540015503 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c181a4a80 con 0x7f0c18101420 2024-09-11T23:00:44.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.158+0000 7f0c15ffb640 1 -- 172.21.15.152:0/540015503 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c0c010910 con 0x7f0c18101420 2024-09-11T23:00:44.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.158+0000 7f0c15ffb640 1 -- 172.21.15.152:0/540015503 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0188f0 con 0x7f0c18101420 2024-09-11T23:00:44.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.158+0000 7f0c1ecd1640 1 -- 172.21.15.152:0/540015503 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c181a4fc0 con 0x7f0c18101420 2024-09-11T23:00:44.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.160+0000 7f0c15ffb640 1 -- 172.21.15.152:0/540015503 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0c0c010ac0 con 0x7f0c18101420 2024-09-11T23:00:44.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c210aa740 1 Processor -- start 2024-09-11T23:00:44.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c210aa740 1 -- start start 2024-09-11T23:00:44.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c15ffb640 1 --2- 172.21.15.152:0/540015503 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c000a2cf0 0x7f0c000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c210aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7007344a0 0x55d700734880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c210aa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d700734dc0 0x55d70073f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c210aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d70073f680 0x55d700741a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c210aa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d70069a2e0 con 0x55d700734dc0 2024-09-11T23:00:44.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c210aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d700641ce0 con 0x55d7007344a0 2024-09-11T23:00:44.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c210aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d7006962a0 con 0x55d70073f680 2024-09-11T23:00:44.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c1ca6d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7007344a0 0x55d700734880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c1d26e640 1 --2- 172.21.15.152:0/540015503 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c000a2cf0 0x7f0c000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c1ca6d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7007344a0 0x55d700734880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:54132/0 (socket says 172.21.15.152:54132) 2024-09-11T23:00:44.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c1ca6d640 1 -- 172.21.15.152:0/4201122690 learned_addr learned my addr 172.21.15.152:0/4201122690 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:44.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c1d26e640 1 --2- 172.21.15.152:0/4201122690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d70073f680 0x55d700741a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c15ffb640 1 -- 172.21.15.152:0/540015503 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f0c0c0cf870 con 0x7f0c18101420 2024-09-11T23:00:44.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c17fff640 1 --2- 172.21.15.152:0/4201122690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d700734dc0 0x55d70073f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c1ca6d640 1 -- 172.21.15.152:0/4201122690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d70073f680 msgr2=0x55d700741a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:44.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c1ca6d640 1 --2- 172.21.15.152:0/4201122690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d70073f680 0x55d700741a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c1ca6d640 1 -- 172.21.15.152:0/4201122690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d700734dc0 msgr2=0x55d70073f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:44.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c1ca6d640 1 --2- 172.21.15.152:0/4201122690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d700734dc0 0x55d70073f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c1ca6d640 1 -- 172.21.15.152:0/4201122690 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d700626440 con 0x55d7007344a0 2024-09-11T23:00:44.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.161+0000 7f0c1d26e640 1 --2- 172.21.15.152:0/4201122690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d70073f680 0x55d700741a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:00:44.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.162+0000 7f0c1ca6d640 1 --2- 172.21.15.152:0/4201122690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7007344a0 0x55d700734880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009cd0 tx=0x7f0c10005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eaf8c0253657a01 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:44.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.162+0000 7f0c1d26e640 1 --2- 172.21.15.152:0/540015503 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c000a2cf0 0x7f0c000a51b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55d700731db0 tx=0x7f0c08005e00 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:44.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.162+0000 7f0bfeffd640 1 -- 172.21.15.152:0/4201122690 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10048070 con 0x55d7007344a0 2024-09-11T23:00:44.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.162+0000 7f0bfeffd640 1 -- 172.21.15.152:0/4201122690 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c10036070 con 0x55d7007344a0 2024-09-11T23:00:44.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.162+0000 7f0bfeffd640 1 -- 172.21.15.152:0/4201122690 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10004580 con 0x55d7007344a0 2024-09-11T23:00:44.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.163+0000 7f0c210aa740 1 -- 172.21.15.152:0/4201122690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7007344a0 msgr2=0x55d700734880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:44.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.163+0000 7f0c210aa740 1 --2- 172.21.15.152:0/4201122690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7007344a0 0x55d700734880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009cd0 tx=0x7f0c10005fa0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.163+0000 7f0c210aa740 1 -- 172.21.15.152:0/4201122690 shutdown_connections 2024-09-11T23:00:44.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.163+0000 7f0c210aa740 1 --2- 172.21.15.152:0/4201122690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d70073f680 0x55d700741a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.163+0000 7f0c210aa740 1 --2- 172.21.15.152:0/4201122690 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d700734dc0 0x55d70073f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.163+0000 7f0c210aa740 1 --2- 172.21.15.152:0/4201122690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d7007344a0 0x55d700734880 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.163+0000 7f0c210aa740 1 -- 172.21.15.152:0/4201122690 >> 172.21.15.152:0/4201122690 conn(0x55d700730a30 msgr2=0x55d700730e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:44.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.163+0000 7f0c210aa740 1 -- 172.21.15.152:0/4201122690 shutdown_connections 2024-09-11T23:00:44.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.163+0000 7f0c210aa740 1 -- 172.21.15.152:0/4201122690 wait complete. 2024-09-11T23:00:44.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.163+0000 7f0c210aa740 1 -- start start 2024-09-11T23:00:44.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.163+0000 7f0c210aa740 1 Processor -- start 2024-09-11T23:00:44.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c210aa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7007344a0 0x55d70073eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c210aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d700734dc0 0x55d70073c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c210aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d70073f680 0x55d70073c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c210aa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d700742250 con 0x55d7007344a0 2024-09-11T23:00:44.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c210aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d700655050 con 0x55d70073f680 2024-09-11T23:00:44.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c210aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d7007423d0 con 0x55d700734dc0 2024-09-11T23:00:44.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c210aa740 10 client.0 _ll_register_callbacks cb 0x55d700695160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:44.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c1ca6d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7007344a0 0x55d70073eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c1ca6d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7007344a0 0x55d70073eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40374/0 (socket says 172.21.15.152:40374) 2024-09-11T23:00:44.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c1ca6d640 1 -- 172.21.15.152:0/3106670441 learned_addr learned my addr 172.21.15.152:0/3106670441 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:44.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c1d26e640 1 --2- 172.21.15.152:0/3106670441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d70073f680 0x55d70073c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c17fff640 1 --2- 172.21.15.152:0/3106670441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d700734dc0 0x55d70073c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c1ca6d640 1 -- 172.21.15.152:0/3106670441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d700734dc0 msgr2=0x55d70073c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:44.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c1ca6d640 1 --2- 172.21.15.152:0/3106670441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d700734dc0 0x55d70073c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c1ca6d640 1 -- 172.21.15.152:0/3106670441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d70073f680 msgr2=0x55d70073c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:44.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c1ca6d640 1 --2- 172.21.15.152:0/3106670441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d70073f680 0x55d70073c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c1ca6d640 1 -- 172.21.15.152:0/3106670441 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d70040b3c0 con 0x55d7007344a0 2024-09-11T23:00:44.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c1ca6d640 1 --2- 172.21.15.152:0/3106670441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7007344a0 0x55d70073eec0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0c10005e30 tx=0x7f0c100040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62bbf9f0bb3dfe8a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:44.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0bf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:44.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0bf7fff640 1 -- 172.21.15.152:0/3106670441 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10048040 con 0x55d7007344a0 2024-09-11T23:00:44.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0bf7fff640 1 -- 172.21.15.152:0/3106670441 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f0c10004760 con 0x55d7007344a0 2024-09-11T23:00:44.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.164+0000 7f0c210aa740 1 -- 172.21.15.152:0/3106670441 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c10036070 con 0x55d7007344a0 2024-09-11T23:00:44.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.165+0000 7f0bf7fff640 1 -- 172.21.15.152:0/3106670441 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c1005c050 con 0x55d7007344a0 2024-09-11T23:00:44.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.165+0000 7f0c210aa740 10 client.17568 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:44.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.165+0000 7f0c210aa740 1 -- 172.21.15.152:0/3106670441 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d700742250 con 0x55d7007344a0 2024-09-11T23:00:44.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.165+0000 7f0bf7fff640 1 -- 172.21.15.152:0/3106670441 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c10036070 con 0x55d7007344a0 2024-09-11T23:00:44.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.165+0000 7f0bf7fff640 1 -- 172.21.15.152:0/3106670441 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0c1004b030 con 0x55d7007344a0 2024-09-11T23:00:44.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.165+0000 7f0c210aa740 10 client.17568 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:44.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.165+0000 7f0c210aa740 10 client.17568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:00:44.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.165+0000 7f0c210aa740 20 client.17568 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:44.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.165+0000 7f0c210aa740 1 --2- 172.21.15.152:0/3106670441 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d70074ecc0 0x55d7007510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.165+0000 7f0c210aa740 4 client.17568 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:00:44.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.165+0000 7f0c210aa740 1 -- 172.21.15.152:0/3106670441 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7006a7850 con 0x55d70074ecc0 2024-09-11T23:00:44.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.165+0000 7f0c17fff640 1 --2- 172.21.15.152:0/3106670441 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d70074ecc0 0x55d7007510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.166+0000 7f0bf7fff640 10 client.17568 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:44.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.166+0000 7f0c17fff640 1 --2- 172.21.15.152:0/3106670441 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d70074ecc0 0x55d7007510b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61fd8fff93fd0b1d server_cookie=439491230f382909 in_seq=0 out_seq=0 2024-09-11T23:00:44.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.166+0000 7f0bf7fff640 1 -- 172.21.15.152:0/3106670441 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7006a7850 con 0x55d70074ecc0 2024-09-11T23:00:44.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.166+0000 7f0bf7fff640 10 client.17568 handle_command_reply: tid=0 2024-09-11T23:00:44.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.166+0000 7f0c210aa740 1 client.17568 shutdown 2024-09-11T23:00:44.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.167+0000 7f0c210aa740 1 -- 172.21.15.152:0/3106670441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7007344a0 msgr2=0x55d70073eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:44.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.167+0000 7f0c210aa740 1 --2- 172.21.15.152:0/3106670441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7007344a0 0x55d70073eec0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f0c10005e30 tx=0x7f0c100040e0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.167+0000 7f0c210aa740 1 -- 172.21.15.152:0/3106670441 shutdown_connections 2024-09-11T23:00:44.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.167+0000 7f0c210aa740 1 --2- 172.21.15.152:0/3106670441 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d70074ecc0 0x55d7007510b0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.167+0000 7f0c210aa740 1 --2- 172.21.15.152:0/3106670441 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d70073f680 0x55d70073c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.167+0000 7f0c210aa740 1 --2- 172.21.15.152:0/3106670441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d700734dc0 0x55d70073c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.167+0000 7f0c210aa740 1 --2- 172.21.15.152:0/3106670441 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d7007344a0 0x55d70073eec0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.167+0000 7f0bf7fff640 0 client.17568 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:44.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.167+0000 7f0c210aa740 1 -- 172.21.15.152:0/3106670441 >> 172.21.15.152:0/3106670441 conn(0x55d700730a30 msgr2=0x55d700736760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:44.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.167+0000 7f0c210aa740 1 -- 172.21.15.152:0/3106670441 shutdown_connections 2024-09-11T23:00:44.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.167+0000 7f0c210aa740 1 -- 172.21.15.152:0/3106670441 wait complete. 2024-09-11T23:00:44.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.167+0000 7f0c210aa740 20 client.17568 trim_cache size 0 max 16384 2024-09-11T23:00:44.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.184+0000 7f0c210aa740 1 Processor -- start 2024-09-11T23:00:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.184+0000 7f0c210aa740 1 -- start start 2024-09-11T23:00:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.184+0000 7f0c210aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7007344a0 0x55d700756870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.184+0000 7f0c210aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d700734dc0 0x55d700756db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.184+0000 7f0c210aa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d70073f680 0x55d700747640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.184+0000 7f0c210aa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0c181ab550 con 0x55d70073f680 2024-09-11T23:00:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.184+0000 7f0c210aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0c181ad090 con 0x55d700734dc0 2024-09-11T23:00:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.184+0000 7f0c210aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c181abc20 con 0x55d7007344a0 2024-09-11T23:00:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.185+0000 7f0c1d26e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d70073f680 0x55d700747640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.185+0000 7f0c1d26e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d70073f680 0x55d700747640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40382/0 (socket says 172.21.15.152:40382) 2024-09-11T23:00:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.185+0000 7f0c1d26e640 1 -- 172.21.15.152:0/3131733596 learned_addr learned my addr 172.21.15.152:0/3131733596 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.185+0000 7f0c1ca6d640 1 --2- 172.21.15.152:0/3131733596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7007344a0 0x55d700756870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.185+0000 7f0c17fff640 1 --2- 172.21.15.152:0/3131733596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d700734dc0 0x55d700756db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.185+0000 7f0c1d26e640 1 -- 172.21.15.152:0/3131733596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7007344a0 msgr2=0x55d700756870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:44.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.185+0000 7f0c1d26e640 1 --2- 172.21.15.152:0/3131733596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7007344a0 0x55d700756870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.185+0000 7f0c1d26e640 1 -- 172.21.15.152:0/3131733596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d700734dc0 msgr2=0x55d700756db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:44.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.185+0000 7f0c1d26e640 1 --2- 172.21.15.152:0/3131733596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d700734dc0 0x55d700756db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.185+0000 7f0c1d26e640 1 -- 172.21.15.152:0/3131733596 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d700751640 con 0x55d70073f680 2024-09-11T23:00:44.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.185+0000 7f0c1ca6d640 1 --2- 172.21.15.152:0/3131733596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7007344a0 0x55d700756870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:44.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.185+0000 7f0c1d26e640 1 --2- 172.21.15.152:0/3131733596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d70073f680 0x55d700747640 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0c080149e0 tx=0x7f0c08005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d2f0a7d422f66f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:44.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.185+0000 7f0bfe7fc640 1 -- 172.21.15.152:0/3131733596 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08004020 con 0x55d70073f680 2024-09-11T23:00:44.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.185+0000 7f0bfe7fc640 1 -- 172.21.15.152:0/3131733596 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c08012ca0 con 0x55d70073f680 2024-09-11T23:00:44.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.185+0000 7f0bfe7fc640 1 -- 172.21.15.152:0/3131733596 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c080039e0 con 0x55d70073f680 2024-09-11T23:00:44.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.186+0000 7f0c210aa740 1 -- 172.21.15.152:0/3131733596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d70073f680 msgr2=0x55d700747640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:44.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.186+0000 7f0c210aa740 1 --2- 172.21.15.152:0/3131733596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d70073f680 0x55d700747640 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f0c080149e0 tx=0x7f0c08005890 comp rx=0 tx=0).stop 2024-09-11T23:00:44.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.186+0000 7f0c210aa740 1 -- 172.21.15.152:0/3131733596 shutdown_connections 2024-09-11T23:00:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.186+0000 7f0c210aa740 1 --2- 172.21.15.152:0/3131733596 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d70073f680 0x55d700747640 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.186+0000 7f0c210aa740 1 --2- 172.21.15.152:0/3131733596 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d700734dc0 0x55d700756db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.186+0000 7f0c210aa740 1 --2- 172.21.15.152:0/3131733596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7007344a0 0x55d700756870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.186+0000 7f0c210aa740 1 -- 172.21.15.152:0/3131733596 >> 172.21.15.152:0/3131733596 conn(0x55d700730a30 msgr2=0x55d70073b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.186+0000 7f0c210aa740 1 -- 172.21.15.152:0/3131733596 shutdown_connections 2024-09-11T23:00:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.186+0000 7f0c210aa740 1 -- 172.21.15.152:0/3131733596 wait complete. 2024-09-11T23:00:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.186+0000 7f0c210aa740 1 -- start start 2024-09-11T23:00:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.186+0000 7f0c210aa740 1 Processor -- start 2024-09-11T23:00:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c210aa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7007344a0 0x55d700747d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c210aa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d700734dc0 0x55d7007450d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c210aa740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d70073f680 0x55d700745610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c210aa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0c181ac9c0 con 0x55d700734dc0 2024-09-11T23:00:44.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c210aa740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0c181ac2f0 con 0x55d70073f680 2024-09-11T23:00:44.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c210aa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c181a9530 con 0x55d7007344a0 2024-09-11T23:00:44.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c210aa740 10 client.0 _ll_register_callbacks cb 0x7f0c10046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:44.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c17fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d700734dc0 0x55d7007450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c17fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d700734dc0 0x55d7007450d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40394/0 (socket says 172.21.15.152:40394) 2024-09-11T23:00:44.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c17fff640 1 -- 172.21.15.152:0/694451935 learned_addr learned my addr 172.21.15.152:0/694451935 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:44.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c1ca6d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7007344a0 0x55d700747d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c1d26e640 1 --2- 172.21.15.152:0/694451935 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d70073f680 0x55d700745610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c17fff640 1 -- 172.21.15.152:0/694451935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7007344a0 msgr2=0x55d700747d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:44.195 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c17fff640 1 --2- 172.21.15.152:0/694451935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7007344a0 0x55d700747d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c17fff640 1 -- 172.21.15.152:0/694451935 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d70073f680 msgr2=0x55d700745610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:44.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c17fff640 1 --2- 172.21.15.152:0/694451935 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d70073f680 0x55d700745610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c17fff640 1 -- 172.21.15.152:0/694451935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d700730860 con 0x55d700734dc0 2024-09-11T23:00:44.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c1ca6d640 1 --2- 172.21.15.152:0/694451935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7007344a0 0x55d700747d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:00:44.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0c17fff640 1 --2- 172.21.15.152:0/694451935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d700734dc0 0x55d7007450d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00c6a0 tx=0x7f0c0c0cfa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683aec6d4805d1aa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:44.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.187+0000 7f0bf7fff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:44.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.188+0000 7f0bf7fff640 1 -- 172.21.15.152:0/694451935 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0a4a20 con 0x55d700734dc0 2024-09-11T23:00:44.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.188+0000 7f0bf7fff640 1 -- 172.21.15.152:0/694451935 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f0c0c098420 con 0x55d700734dc0 2024-09-11T23:00:44.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.188+0000 7f0c210aa740 1 -- 172.21.15.152:0/694451935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c08012ca0 con 0x55d700734dc0 2024-09-11T23:00:44.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.188+0000 7f0bf7fff640 1 -- 172.21.15.152:0/694451935 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c0c09b200 con 0x55d700734dc0 2024-09-11T23:00:44.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.188+0000 7f0c210aa740 10 client.17574 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:44.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.188+0000 7f0c210aa740 1 -- 172.21.15.152:0/694451935 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d7007b9630 con 0x55d700734dc0 2024-09-11T23:00:44.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.188+0000 7f0bf7fff640 1 -- 172.21.15.152:0/694451935 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c08012ca0 con 0x55d700734dc0 2024-09-11T23:00:44.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.188+0000 7f0bf7fff640 1 -- 172.21.15.152:0/694451935 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0c0c086040 con 0x55d700734dc0 2024-09-11T23:00:44.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.188+0000 7f0c210aa740 10 client.17574 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:44.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.188+0000 7f0c210aa740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:00:44.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.188+0000 7f0c210aa740 20 client.17574 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:44.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.188+0000 7f0c210aa740 1 --2- 172.21.15.152:0/694451935 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d7007b9db0 0x55d7007bc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:44.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.188+0000 7f0c210aa740 4 client.17574 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:00:44.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.188+0000 7f0c210aa740 1 -- 172.21.15.152:0/694451935 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7006a1d60 con 0x55d7007b9db0 2024-09-11T23:00:44.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.189+0000 7f0c1ca6d640 1 --2- 172.21.15.152:0/694451935 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d7007b9db0 0x55d7007bc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:44.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.189+0000 7f0bf7fff640 10 client.17574 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:44.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.189+0000 7f0c1ca6d640 1 --2- 172.21.15.152:0/694451935 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d7007b9db0 0x55d7007bc1d0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81fbb3305f75acbf server_cookie=c54954a20d84b7af in_seq=0 out_seq=0 2024-09-11T23:00:44.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.189+0000 7f0bf7fff640 1 -- 172.21.15.152:0/694451935 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d7006a1d60 con 0x55d7007b9db0 2024-09-11T23:00:44.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.189+0000 7f0bf7fff640 10 client.17574 handle_command_reply: tid=0 2024-09-11T23:00:44.198 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.189+0000 7f0c210aa740 1 client.17574 shutdown 2024-09-11T23:00:44.199 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.189+0000 7f0c210aa740 1 -- 172.21.15.152:0/694451935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d700734dc0 msgr2=0x55d7007450d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:44.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.189+0000 7f0c210aa740 1 --2- 172.21.15.152:0/694451935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d700734dc0 0x55d7007450d0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00c6a0 tx=0x7f0c0c0cfa70 comp rx=0 tx=0).stop 2024-09-11T23:00:44.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.189+0000 7f0c210aa740 1 -- 172.21.15.152:0/694451935 shutdown_connections 2024-09-11T23:00:44.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.189+0000 7f0c210aa740 1 --2- 172.21.15.152:0/694451935 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d7007b9db0 0x55d7007bc1d0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.189+0000 7f0c210aa740 1 --2- 172.21.15.152:0/694451935 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d70073f680 0x55d700745610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.189+0000 7f0c210aa740 1 --2- 172.21.15.152:0/694451935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d700734dc0 0x55d7007450d0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.189+0000 7f0bf7fff640 0 client.17574 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:44.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.189+0000 7f0c210aa740 1 --2- 172.21.15.152:0/694451935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d7007344a0 0x55d700747d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.189+0000 7f0c210aa740 1 -- 172.21.15.152:0/694451935 >> 172.21.15.152:0/694451935 conn(0x55d700730a30 msgr2=0x55d70073a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:44.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.190+0000 7f0c210aa740 1 -- 172.21.15.152:0/694451935 shutdown_connections 2024-09-11T23:00:44.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.190+0000 7f0c210aa740 1 -- 172.21.15.152:0/694451935 wait complete. 2024-09-11T23:00:44.200 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.190+0000 7f0c210aa740 20 client.17574 trim_cache size 0 max 16384 2024-09-11T23:00:44.201 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:00:44.201 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T23:00:44.201 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T23:00:44.201 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:00:44.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.190+0000 7f0c1ecd1640 1 -- 172.21.15.152:0/540015503 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c000a2cf0 msgr2=0x7f0c000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:44.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.190+0000 7f0c1ecd1640 1 --2- 172.21.15.152:0/540015503 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c000a2cf0 0x7f0c000a51b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55d700731db0 tx=0x7f0c08005e00 comp rx=0 tx=0).stop 2024-09-11T23:00:44.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.190+0000 7f0c1ecd1640 1 -- 172.21.15.152:0/540015503 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c18101420 msgr2=0x7f0c18199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:44.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.190+0000 7f0c1ecd1640 1 --2- 172.21.15.152:0/540015503 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c18101420 0x7f0c18199830 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00e9f0 tx=0x7f0c0c00eeb0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.190+0000 7f0c1ecd1640 1 -- 172.21.15.152:0/540015503 shutdown_connections 2024-09-11T23:00:44.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.190+0000 7f0c1ecd1640 1 --2- 172.21.15.152:0/540015503 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c000a2cf0 0x7f0c000a51b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.190+0000 7f0c1ecd1640 1 --2- 172.21.15.152:0/540015503 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c18106a80 0x7f0c181a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.190+0000 7f0c1ecd1640 1 --2- 172.21.15.152:0/540015503 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c18101420 0x7f0c18199830 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.190+0000 7f0c1ecd1640 1 --2- 172.21.15.152:0/540015503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c18100a80 0x7f0c181992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:44.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.190+0000 7f0c1ecd1640 1 -- 172.21.15.152:0/540015503 >> 172.21.15.152:0/540015503 conn(0x7f0c180fc7a0 msgr2=0x7f0c180fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:44.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.190+0000 7f0c1ecd1640 1 -- 172.21.15.152:0/540015503 shutdown_connections 2024-09-11T23:00:44.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:44.190+0000 7f0c1ecd1640 1 -- 172.21.15.152:0/540015503 wait complete. 2024-09-11T23:00:44.560 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:44 smithi152 ceph-mon[31181]: pgmap v240: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:44.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:44 smithi191 ceph-mon[35211]: pgmap v240: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:44.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:44 smithi204 ceph-mon[35454]: pgmap v240: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:45.049 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:00:45.049 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:00:45.295 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:45.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.779+0000 7f08ec0f0640 1 -- 172.21.15.152:0/2063877964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e4102ef0 msgr2=0x7f08e410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:45.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.779+0000 7f08ec0f0640 1 --2- 172.21.15.152:0/2063877964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e4102ef0 0x7f08e410da40 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009ca0 tx=0x7f08dc02a950 comp rx=0 tx=0).stop 2024-09-11T23:00:45.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.779+0000 7f08ec0f0640 1 -- 172.21.15.152:0/2063877964 shutdown_connections 2024-09-11T23:00:45.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.779+0000 7f08ec0f0640 1 --2- 172.21.15.152:0/2063877964 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e4102ef0 0x7f08e410da40 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:45.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.779+0000 7f08ec0f0640 1 --2- 172.21.15.152:0/2063877964 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08e4102570 0x7f08e41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:45.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.779+0000 7f08ec0f0640 1 --2- 172.21.15.152:0/2063877964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e4100e20 0x7f08e4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:45.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.779+0000 7f08ec0f0640 1 -- 172.21.15.152:0/2063877964 >> 172.21.15.152:0/2063877964 conn(0x7f08e40fc7d0 msgr2=0x7f08e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:45.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.779+0000 7f08ec0f0640 1 -- 172.21.15.152:0/2063877964 shutdown_connections 2024-09-11T23:00:45.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.779+0000 7f08ec0f0640 1 -- 172.21.15.152:0/2063877964 wait complete. 2024-09-11T23:00:45.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.779+0000 7f08ec0f0640 1 Processor -- start 2024-09-11T23:00:45.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.779+0000 7f08ec0f0640 1 -- start start 2024-09-11T23:00:45.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08ec0f0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e4100e20 0x7f08e4198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:45.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08ec0f0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e4102570 0x7f08e4198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:45.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08ec0f0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08e4102ef0 0x7f08e419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:45.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08ec0f0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4110f80 con 0x7f08e4102570 2024-09-11T23:00:45.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08ec0f0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4110e00 con 0x7f08e4102ef0 2024-09-11T23:00:45.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08ec0f0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f08e4111100 con 0x7f08e4100e20 2024-09-11T23:00:45.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08e968b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e4102570 0x7f08e4198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:45.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08e968b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e4102570 0x7f08e4198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40416/0 (socket says 172.21.15.152:40416) 2024-09-11T23:00:45.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08e968b640 1 -- 172.21.15.152:0/2954277855 learned_addr learned my addr 172.21.15.152:0/2954277855 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:45.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08ea68d640 1 --2- 172.21.15.152:0/2954277855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08e4102ef0 0x7f08e419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:45.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08e9e8c640 1 --2- 172.21.15.152:0/2954277855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e4100e20 0x7f08e4198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:45.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08e968b640 1 -- 172.21.15.152:0/2954277855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e4100e20 msgr2=0x7f08e4198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:45.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08e968b640 1 --2- 172.21.15.152:0/2954277855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e4100e20 0x7f08e4198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:45.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08e968b640 1 -- 172.21.15.152:0/2954277855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08e4102ef0 msgr2=0x7f08e419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:45.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08e968b640 1 --2- 172.21.15.152:0/2954277855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08e4102ef0 0x7f08e419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:45.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08e968b640 1 -- 172.21.15.152:0/2954277855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08e419d2c0 con 0x7f08e4102570 2024-09-11T23:00:45.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08e9e8c640 1 --2- 172.21.15.152:0/2954277855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e4100e20 0x7f08e4198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:45.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08e968b640 1 --2- 172.21.15.152:0/2954277855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e4102570 0x7f08e4198850 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f08e000c920 tx=0x7f08e000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:45.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08d6ffd640 1 -- 172.21.15.152:0/2954277855 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0007c70 con 0x7f08e4102570 2024-09-11T23:00:45.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.780+0000 7f08ec0f0640 1 -- 172.21.15.152:0/2954277855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08e419d5b0 con 0x7f08e4102570 2024-09-11T23:00:45.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.781+0000 7f08d6ffd640 1 -- 172.21.15.152:0/2954277855 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08e0007e10 con 0x7f08e4102570 2024-09-11T23:00:45.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.781+0000 7f08d6ffd640 1 -- 172.21.15.152:0/2954277855 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0018890 con 0x7f08e4102570 2024-09-11T23:00:45.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.781+0000 7f08ec0f0640 1 -- 172.21.15.152:0/2954277855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08e41a4e90 con 0x7f08e4102570 2024-09-11T23:00:45.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.782+0000 7f08ec0f0640 1 -- 172.21.15.152:0/2954277855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08e4101200 con 0x7f08e4102570 2024-09-11T23:00:45.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.783+0000 7f08d6ffd640 1 -- 172.21.15.152:0/2954277855 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f08e0018a50 con 0x7f08e4102570 2024-09-11T23:00:45.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.783+0000 7f08d6ffd640 1 --2- 172.21.15.152:0/2954277855 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08d00a2cf0 0x7f08d00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:45.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.784+0000 7f08e9e8c640 1 --2- 172.21.15.152:0/2954277855 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08d00a2cf0 0x7f08d00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:45.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.784+0000 7f08d6ffd640 1 -- 172.21.15.152:0/2954277855 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f08e00cf870 con 0x7f08e4102570 2024-09-11T23:00:45.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.784+0000 7f08e9e8c640 1 --2- 172.21.15.152:0/2954277855 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08d00a2cf0 0x7f08d00a51b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f08d800abe0 tx=0x7f08d800a5c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:45.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.787+0000 7f08d6ffd640 1 -- 172.21.15.152:0/2954277855 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f08e0014030 con 0x7f08e4102570 2024-09-11T23:00:45.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.988+0000 7f08ec0f0640 1 -- 172.21.15.152:0/2954277855 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08e4199670 con 0x7f08e4102570 2024-09-11T23:00:45.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.989+0000 7f08d6ffd640 1 -- 172.21.15.152:0/2954277855 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f08e4199670 con 0x7f08e4102570 2024-09-11T23:00:45.990 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T23:00:45.990 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T23:00:45.990 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T23:00:45.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.991+0000 7f08ec0f0640 1 -- 172.21.15.152:0/2954277855 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08d00a2cf0 msgr2=0x7f08d00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:45.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.991+0000 7f08ec0f0640 1 --2- 172.21.15.152:0/2954277855 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08d00a2cf0 0x7f08d00a51b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f08d800abe0 tx=0x7f08d800a5c0 comp rx=0 tx=0).stop 2024-09-11T23:00:45.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.991+0000 7f08ec0f0640 1 -- 172.21.15.152:0/2954277855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e4102570 msgr2=0x7f08e4198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:45.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.991+0000 7f08ec0f0640 1 --2- 172.21.15.152:0/2954277855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e4102570 0x7f08e4198850 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f08e000c920 tx=0x7f08e000cde0 comp rx=0 tx=0).stop 2024-09-11T23:00:45.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.991+0000 7f08ec0f0640 1 -- 172.21.15.152:0/2954277855 shutdown_connections 2024-09-11T23:00:45.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.991+0000 7f08ec0f0640 1 --2- 172.21.15.152:0/2954277855 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f08d00a2cf0 0x7f08d00a51b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:45.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.991+0000 7f08ec0f0640 1 --2- 172.21.15.152:0/2954277855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08e4102ef0 0x7f08e419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:45.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.991+0000 7f08ec0f0640 1 --2- 172.21.15.152:0/2954277855 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08e4102570 0x7f08e4198850 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:45.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.991+0000 7f08ec0f0640 1 --2- 172.21.15.152:0/2954277855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08e4100e20 0x7f08e4198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:45.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.991+0000 7f08ec0f0640 1 -- 172.21.15.152:0/2954277855 >> 172.21.15.152:0/2954277855 conn(0x7f08e40fc7d0 msgr2=0x7f08e406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:45.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.992+0000 7f08ec0f0640 1 -- 172.21.15.152:0/2954277855 shutdown_connections 2024-09-11T23:00:45.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:45.992+0000 7f08ec0f0640 1 -- 172.21.15.152:0/2954277855 wait complete. 2024-09-11T23:00:46.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:46 smithi152 ceph-mon[31181]: pgmap v241: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:00:46.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:46 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2954277855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:00:46.718 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:00:47.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:46 smithi204 ceph-mon[35454]: pgmap v241: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:00:47.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:46 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2954277855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:00:47.123 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:46 smithi191 ceph-mon[35211]: pgmap v241: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:00:47.123 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:46 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2954277855' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:00:47.371 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:47.896 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:00:47.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.859+0000 7fa66e2f0640 1 -- 172.21.15.152:0/461336038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6681055b0 msgr2=0x7fa66810d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:47.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.859+0000 7fa66e2f0640 1 --2- 172.21.15.152:0/461336038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6681055b0 0x7fa66810d130 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa660009ca0 tx=0x7fa66002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.859+0000 7fa66e2f0640 1 -- 172.21.15.152:0/461336038 shutdown_connections 2024-09-11T23:00:47.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.859+0000 7fa66e2f0640 1 --2- 172.21.15.152:0/461336038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6681055b0 0x7fa66810d130 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.859+0000 7fa66e2f0640 1 --2- 172.21.15.152:0/461336038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa668100e30 0x7fa668104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.859+0000 7fa66e2f0640 1 --2- 172.21.15.152:0/461336038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa668100480 0x7fa668100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.859+0000 7fa66e2f0640 1 -- 172.21.15.152:0/461336038 >> 172.21.15.152:0/461336038 conn(0x7fa668077210 msgr2=0x7fa6680fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:47.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.859+0000 7fa66e2f0640 1 -- 172.21.15.152:0/461336038 shutdown_connections 2024-09-11T23:00:47.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.859+0000 7fa66e2f0640 1 -- 172.21.15.152:0/461336038 wait complete. 2024-09-11T23:00:47.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.860+0000 7fa66e2f0640 1 Processor -- start 2024-09-11T23:00:47.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.860+0000 7fa66e2f0640 1 -- start start 2024-09-11T23:00:47.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa66e2f0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa668100480 0x7fa66810e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa66e2f0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa668100e30 0x7fa66810ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa66e2f0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6681055b0 0x7fa66810f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa66e2f0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa668111f90 con 0x7fa6681055b0 2024-09-11T23:00:47.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa66e2f0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa668111e10 con 0x7fa668100e30 2024-09-11T23:00:47.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa66e2f0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa668112110 con 0x7fa668100480 2024-09-11T23:00:47.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa66c88d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6681055b0 0x7fa66810f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa66c88d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6681055b0 0x7fa66810f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40432/0 (socket says 172.21.15.152:40432) 2024-09-11T23:00:47.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa667fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa668100480 0x7fa66810e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa66c88d640 1 -- 172.21.15.152:0/117457873 learned_addr learned my addr 172.21.15.152:0/117457873 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:47.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa6677fe640 1 --2- 172.21.15.152:0/117457873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa668100e30 0x7fa66810ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa66c88d640 1 -- 172.21.15.152:0/117457873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa668100480 msgr2=0x7fa66810e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:47.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa66c88d640 1 --2- 172.21.15.152:0/117457873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa668100480 0x7fa66810e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa66c88d640 1 -- 172.21.15.152:0/117457873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa668100e30 msgr2=0x7fa66810ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:47.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa66c88d640 1 --2- 172.21.15.152:0/117457873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa668100e30 0x7fa66810ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa66c88d640 1 -- 172.21.15.152:0/117457873 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6681aac80 con 0x7fa6681055b0 2024-09-11T23:00:47.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa66c88d640 1 --2- 172.21.15.152:0/117457873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6681055b0 0x7fa66810f2d0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa660031a60 tx=0x7fa660031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:47.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.861+0000 7fa6657fa640 1 -- 172.21.15.152:0/117457873 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa660003b70 con 0x7fa6681055b0 2024-09-11T23:00:47.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.862+0000 7fa66e2f0640 1 -- 172.21.15.152:0/117457873 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6681aaeb0 con 0x7fa6681055b0 2024-09-11T23:00:47.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.862+0000 7fa6657fa640 1 -- 172.21.15.152:0/117457873 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa660003d10 con 0x7fa6681055b0 2024-09-11T23:00:47.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.862+0000 7fa6657fa640 1 -- 172.21.15.152:0/117457873 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa660004770 con 0x7fa6681055b0 2024-09-11T23:00:47.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.862+0000 7fa66e2f0640 1 -- 172.21.15.152:0/117457873 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6681ab360 con 0x7fa6681055b0 2024-09-11T23:00:47.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.864+0000 7fa6657fa640 1 -- 172.21.15.152:0/117457873 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa6600305c0 con 0x7fa6681055b0 2024-09-11T23:00:47.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.864+0000 7fa6706c9740 1 Processor -- start 2024-09-11T23:00:47.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.864+0000 7fa6706c9740 1 -- start start 2024-09-11T23:00:47.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.864+0000 7fa6706c9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145ab40 0x562401459090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.864+0000 7fa6706c9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5624014595d0 0x562401459a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.864+0000 7fa6706c9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 0x562401466760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.864+0000 7fa6706c9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5624013bf2f0 con 0x5624014595d0 2024-09-11T23:00:47.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.864+0000 7fa6706c9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562401366cf0 con 0x562401464370 2024-09-11T23:00:47.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.864+0000 7fa6706c9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5624013bb2b0 con 0x56240145ab40 2024-09-11T23:00:47.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.864+0000 7fa667fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5624014595d0 0x562401459a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.900 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.864+0000 7fa667fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5624014595d0 0x562401459a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40446/0 (socket says 172.21.15.152:40446) 2024-09-11T23:00:47.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.864+0000 7fa667fff640 1 -- 172.21.15.152:0/4268646548 learned_addr learned my addr 172.21.15.152:0/4268646548 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:47.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.864+0000 7fa6677fe640 1 --2- 172.21.15.152:0/4268646548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145ab40 0x562401459090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.865+0000 7fa6677fe640 1 --2- 172.21.15.152:0/4268646548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 0x562401466760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.865+0000 7fa667fff640 1 -- 172.21.15.152:0/4268646548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145ab40 msgr2=0x562401459090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:47.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.865+0000 7fa667fff640 1 --2- 172.21.15.152:0/4268646548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145ab40 0x562401459090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.865+0000 7fa667fff640 1 -- 172.21.15.152:0/4268646548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 msgr2=0x562401466760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:47.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.865+0000 7fa667fff640 1 --2- 172.21.15.152:0/4268646548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 0x562401466760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.865+0000 7fa6657fa640 1 --2- 172.21.15.152:0/117457873 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6500a2cf0 0x7fa6500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.865+0000 7fa667fff640 1 -- 172.21.15.152:0/4268646548 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56240134b3a0 con 0x5624014595d0 2024-09-11T23:00:47.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.865+0000 7fa667fff640 1 --2- 172.21.15.152:0/4268646548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5624014595d0 0x562401459a10 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa65800ad00 tx=0x7fa65800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19568adb9c13133d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:47.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.865+0000 7fa6677fe640 1 --2- 172.21.15.152:0/117457873 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6500a2cf0 0x7fa6500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.901 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.865+0000 7fa6657fa640 1 -- 172.21.15.152:0/117457873 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7fa6600f05b0 con 0x7fa6681055b0 2024-09-11T23:00:47.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.865+0000 7fa64e7fc640 1 -- 172.21.15.152:0/4268646548 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65801d070 con 0x5624014595d0 2024-09-11T23:00:47.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.865+0000 7fa64e7fc640 1 -- 172.21.15.152:0/4268646548 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa658018b20 con 0x5624014595d0 2024-09-11T23:00:47.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.865+0000 7fa64e7fc640 1 -- 172.21.15.152:0/4268646548 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658010bc0 con 0x5624014595d0 2024-09-11T23:00:47.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.866+0000 7fa6706c9740 1 -- 172.21.15.152:0/4268646548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5624014595d0 msgr2=0x562401459a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:47.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.866+0000 7fa6677fe640 1 --2- 172.21.15.152:0/117457873 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6500a2cf0 0x7fa6500a51b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x56240145a070 tx=0x7fa65c012040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:47.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.866+0000 7fa6706c9740 1 --2- 172.21.15.152:0/4268646548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5624014595d0 0x562401459a10 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa65800ad00 tx=0x7fa65800f790 comp rx=0 tx=0).stop 2024-09-11T23:00:47.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.866+0000 7fa6706c9740 1 -- 172.21.15.152:0/4268646548 shutdown_connections 2024-09-11T23:00:47.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.866+0000 7fa6706c9740 1 --2- 172.21.15.152:0/4268646548 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 0x562401466760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.866+0000 7fa6706c9740 1 --2- 172.21.15.152:0/4268646548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5624014595d0 0x562401459a10 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.866+0000 7fa6706c9740 1 --2- 172.21.15.152:0/4268646548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145ab40 0x562401459090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.866+0000 7fa6706c9740 1 -- 172.21.15.152:0/4268646548 >> 172.21.15.152:0/4268646548 conn(0x562401422310 msgr2=0x562401455c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:47.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.866+0000 7fa6706c9740 1 -- 172.21.15.152:0/4268646548 shutdown_connections 2024-09-11T23:00:47.902 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.866+0000 7fa6706c9740 1 -- 172.21.15.152:0/4268646548 wait complete. 2024-09-11T23:00:47.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.866+0000 7fa6706c9740 1 -- start start 2024-09-11T23:00:47.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.866+0000 7fa6706c9740 1 Processor -- start 2024-09-11T23:00:47.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa6706c9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5624014595d0 0x562401463d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa6706c9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145ab40 0x562401461010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa6706c9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562401464370 0x562401461600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa6706c9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x562401467070 con 0x562401464370 2024-09-11T23:00:47.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa6706c9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56240137a060 con 0x5624014595d0 2024-09-11T23:00:47.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa6706c9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5624014671f0 con 0x56240145ab40 2024-09-11T23:00:47.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa6706c9740 10 client.0 _ll_register_callbacks cb 0x5624010cd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:47.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa66c88d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562401464370 0x562401461600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa66c88d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562401464370 0x562401461600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40452/0 (socket says 172.21.15.152:40452) 2024-09-11T23:00:47.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa66c88d640 1 -- 172.21.15.152:0/2038003035 learned_addr learned my addr 172.21.15.152:0/2038003035 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:47.903 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa6677fe640 1 --2- 172.21.15.152:0/2038003035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145ab40 0x562401461010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa667fff640 1 --2- 172.21.15.152:0/2038003035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5624014595d0 0x562401463d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa66c88d640 1 -- 172.21.15.152:0/2038003035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145ab40 msgr2=0x562401461010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:47.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa66c88d640 1 --2- 172.21.15.152:0/2038003035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145ab40 0x562401461010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa66c88d640 1 -- 172.21.15.152:0/2038003035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5624014595d0 msgr2=0x562401463d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:47.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa66c88d640 1 --2- 172.21.15.152:0/2038003035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5624014595d0 0x562401463d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa66c88d640 1 -- 172.21.15.152:0/2038003035 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624011303c0 con 0x562401464370 2024-09-11T23:00:47.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa66c88d640 1 --2- 172.21.15.152:0/2038003035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562401464370 0x562401461600 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa6600b95d0 tx=0x7fa66002ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b641d091dcc3c858 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:47.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa6477fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:47.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa6477fe640 1 -- 172.21.15.152:0/2038003035 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6600c0fb0 con 0x562401464370 2024-09-11T23:00:47.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa6477fe640 1 -- 172.21.15.152:0/2038003035 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7fa660003ec0 con 0x562401464370 2024-09-11T23:00:47.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.867+0000 7fa6706c9740 1 -- 172.21.15.152:0/2038003035 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa658018b20 con 0x562401464370 2024-09-11T23:00:47.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.868+0000 7fa6477fe640 1 -- 172.21.15.152:0/2038003035 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa6600da050 con 0x562401464370 2024-09-11T23:00:47.904 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.868+0000 7fa6706c9740 10 client.17592 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:47.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.868+0000 7fa6706c9740 1 -- 172.21.15.152:0/2038003035 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562401467070 con 0x562401464370 2024-09-11T23:00:47.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.868+0000 7fa6477fe640 1 -- 172.21.15.152:0/2038003035 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa658018b20 con 0x562401464370 2024-09-11T23:00:47.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.868+0000 7fa6477fe640 1 -- 172.21.15.152:0/2038003035 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa6600c53e0 con 0x562401464370 2024-09-11T23:00:47.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.868+0000 7fa6706c9740 10 client.17592 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:47.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.868+0000 7fa6706c9740 10 client.17592 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T23:00:47.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.868+0000 7fa6706c9740 20 client.17592 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:47.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.868+0000 7fa6706c9740 1 --2- 172.21.15.152:0/2038003035 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x562401473f30 0x5624014763b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.868+0000 7fa6706c9740 4 client.17592 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:00:47.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.868+0000 7fa6706c9740 1 -- 172.21.15.152:0/2038003035 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624013cc860 con 0x562401473f30 2024-09-11T23:00:47.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.868+0000 7fa667fff640 1 --2- 172.21.15.152:0/2038003035 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x562401473f30 0x5624014763b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.905 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.868+0000 7fa6477fe640 10 client.17592 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T23:00:47.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.868+0000 7fa667fff640 1 --2- 172.21.15.152:0/2038003035 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x562401473f30 0x5624014763b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e2938afa7ec81675 server_cookie=8d9fd6d2bb242d29 in_seq=0 out_seq=0 2024-09-11T23:00:47.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.869+0000 7fa6477fe640 1 -- 172.21.15.152:0/2038003035 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624013cc860 con 0x562401473f30 2024-09-11T23:00:47.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.869+0000 7fa6477fe640 10 client.17592 handle_command_reply: tid=0 2024-09-11T23:00:47.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.869+0000 7fa6706c9740 1 client.17592 shutdown 2024-09-11T23:00:47.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.869+0000 7fa6706c9740 1 -- 172.21.15.152:0/2038003035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562401464370 msgr2=0x562401461600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:47.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.869+0000 7fa6706c9740 1 --2- 172.21.15.152:0/2038003035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562401464370 0x562401461600 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa6600b95d0 tx=0x7fa66002ff50 comp rx=0 tx=0).stop 2024-09-11T23:00:47.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.869+0000 7fa6706c9740 1 -- 172.21.15.152:0/2038003035 shutdown_connections 2024-09-11T23:00:47.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.869+0000 7fa6706c9740 1 --2- 172.21.15.152:0/2038003035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562401464370 0x562401461600 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.869+0000 7fa6706c9740 1 --2- 172.21.15.152:0/2038003035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145ab40 0x562401461010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.906 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.869+0000 7fa6706c9740 1 --2- 172.21.15.152:0/2038003035 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x562401473f30 0x5624014763b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.869+0000 7fa6706c9740 1 --2- 172.21.15.152:0/2038003035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5624014595d0 0x562401463d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.869+0000 7fa6706c9740 1 -- 172.21.15.152:0/2038003035 >> 172.21.15.152:0/2038003035 conn(0x562401422310 msgr2=0x56240145b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:47.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.869+0000 7fa6477fe640 0 client.17592 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T23:00:47.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.869+0000 7fa6706c9740 1 -- 172.21.15.152:0/2038003035 shutdown_connections 2024-09-11T23:00:47.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.869+0000 7fa6706c9740 1 -- 172.21.15.152:0/2038003035 wait complete. 2024-09-11T23:00:47.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.870+0000 7fa6706c9740 20 client.17592 trim_cache size 0 max 16384 2024-09-11T23:00:47.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.882+0000 7fa6706c9740 1 Processor -- start 2024-09-11T23:00:47.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.882+0000 7fa6706c9740 1 -- start start 2024-09-11T23:00:47.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.882+0000 7fa6706c9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624014595d0 0x56240145ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.907 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.882+0000 7fa6706c9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56240145ab40 0x562401462f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.882+0000 7fa6706c9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 0x56240145c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.882+0000 7fa6706c9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa6681b3d40 con 0x56240145ab40 2024-09-11T23:00:47.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.882+0000 7fa6706c9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa6681b5880 con 0x562401464370 2024-09-11T23:00:47.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.882+0000 7fa6706c9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6681b4410 con 0x5624014595d0 2024-09-11T23:00:47.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.882+0000 7fa6677fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56240145ab40 0x562401462f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.882+0000 7fa6677fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56240145ab40 0x562401462f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40456/0 (socket says 172.21.15.152:40456) 2024-09-11T23:00:47.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.882+0000 7fa6677fe640 1 -- 172.21.15.152:0/729236822 learned_addr learned my addr 172.21.15.152:0/729236822 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:47.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.883+0000 7fa66c88d640 1 --2- 172.21.15.152:0/729236822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 0x56240145c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.883+0000 7fa667fff640 1 --2- 172.21.15.152:0/729236822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624014595d0 0x56240145ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.883+0000 7fa6677fe640 1 -- 172.21.15.152:0/729236822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624014595d0 msgr2=0x56240145ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:47.908 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.883+0000 7fa6677fe640 1 --2- 172.21.15.152:0/729236822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624014595d0 0x56240145ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.883+0000 7fa6677fe640 1 -- 172.21.15.152:0/729236822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 msgr2=0x56240145c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:47.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.883+0000 7fa6677fe640 1 --2- 172.21.15.152:0/729236822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 0x56240145c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.883+0000 7fa6677fe640 1 -- 172.21.15.152:0/729236822 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562401473c90 con 0x56240145ab40 2024-09-11T23:00:47.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.883+0000 7fa66c88d640 1 --2- 172.21.15.152:0/729236822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 0x56240145c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:00:47.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.883+0000 7fa667fff640 1 --2- 172.21.15.152:0/729236822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624014595d0 0x56240145ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:00:47.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.883+0000 7fa6677fe640 1 --2- 172.21.15.152:0/729236822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56240145ab40 0x562401462f80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa6681100a0 tx=0x7fa65c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d97fa0872fb7a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:47.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.883+0000 7fa64dffb640 1 -- 172.21.15.152:0/729236822 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c02a070 con 0x56240145ab40 2024-09-11T23:00:47.909 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.883+0000 7fa64dffb640 1 -- 172.21.15.152:0/729236822 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65c01ab10 con 0x56240145ab40 2024-09-11T23:00:47.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.883+0000 7fa64dffb640 1 -- 172.21.15.152:0/729236822 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c025b30 con 0x56240145ab40 2024-09-11T23:00:47.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 -- 172.21.15.152:0/729236822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56240145ab40 msgr2=0x562401462f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:47.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 --2- 172.21.15.152:0/729236822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56240145ab40 0x562401462f80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa6681100a0 tx=0x7fa65c014000 comp rx=0 tx=0).stop 2024-09-11T23:00:47.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 -- 172.21.15.152:0/729236822 shutdown_connections 2024-09-11T23:00:47.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 --2- 172.21.15.152:0/729236822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 0x56240145c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 --2- 172.21.15.152:0/729236822 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56240145ab40 0x562401462f80 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 --2- 172.21.15.152:0/729236822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5624014595d0 0x56240145ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 -- 172.21.15.152:0/729236822 >> 172.21.15.152:0/729236822 conn(0x562401422310 msgr2=0x562401462870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:47.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 -- 172.21.15.152:0/729236822 shutdown_connections 2024-09-11T23:00:47.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 -- 172.21.15.152:0/729236822 wait complete. 2024-09-11T23:00:47.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 -- start start 2024-09-11T23:00:47.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 Processor -- start 2024-09-11T23:00:47.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5624014595d0 0x56240146ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.910 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145ab40 0x56240145dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 0x56240145e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa6681b51b0 con 0x5624014595d0 2024-09-11T23:00:47.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa6681b4ae0 con 0x562401464370 2024-09-11T23:00:47.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa6681b1d40 con 0x56240145ab40 2024-09-11T23:00:47.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.884+0000 7fa6706c9740 10 client.0 _ll_register_callbacks cb 0x7fa660004910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:47.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa667fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5624014595d0 0x56240146ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa667fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5624014595d0 0x56240146ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40472/0 (socket says 172.21.15.152:40472) 2024-09-11T23:00:47.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa667fff640 1 -- 172.21.15.152:0/189379666 learned_addr learned my addr 172.21.15.152:0/189379666 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:47.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa6677fe640 1 --2- 172.21.15.152:0/189379666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145ab40 0x56240145dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa66c88d640 1 --2- 172.21.15.152:0/189379666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 0x56240145e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.911 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa667fff640 1 -- 172.21.15.152:0/189379666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145ab40 msgr2=0x56240145dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:47.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa667fff640 1 --2- 172.21.15.152:0/189379666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145ab40 0x56240145dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa667fff640 1 -- 172.21.15.152:0/189379666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 msgr2=0x56240145e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:47.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa667fff640 1 --2- 172.21.15.152:0/189379666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 0x56240145e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa667fff640 1 -- 172.21.15.152:0/189379666 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56240145ecc0 con 0x5624014595d0 2024-09-11T23:00:47.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa66c88d640 1 --2- 172.21.15.152:0/189379666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 0x56240145e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:00:47.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa667fff640 1 --2- 172.21.15.152:0/189379666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5624014595d0 0x56240146ccd0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa65800ad00 tx=0x7fa65800a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3f00931d840fed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:47.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa6477fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:47.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa6477fe640 1 -- 172.21.15.152:0/189379666 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658032060 con 0x5624014595d0 2024-09-11T23:00:47.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa6477fe640 1 -- 172.21.15.152:0/189379666 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7fa658018680 con 0x5624014595d0 2024-09-11T23:00:47.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa6706c9740 1 -- 172.21.15.152:0/189379666 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa65c01ab10 con 0x5624014595d0 2024-09-11T23:00:47.912 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa6477fe640 1 -- 172.21.15.152:0/189379666 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa65801d040 con 0x5624014595d0 2024-09-11T23:00:47.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa6706c9740 10 client.17598 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:47.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.885+0000 7fa6706c9740 1 -- 172.21.15.152:0/189379666 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5624014ded00 con 0x5624014595d0 2024-09-11T23:00:47.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.886+0000 7fa6477fe640 1 -- 172.21.15.152:0/189379666 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65c01ab10 con 0x5624014595d0 2024-09-11T23:00:47.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.886+0000 7fa6477fe640 1 -- 172.21.15.152:0/189379666 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa6580106d0 con 0x5624014595d0 2024-09-11T23:00:47.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.886+0000 7fa6706c9740 10 client.17598 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:47.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.886+0000 7fa6706c9740 10 client.17598 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T23:00:47.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.886+0000 7fa6706c9740 20 client.17598 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:47.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.886+0000 7fa6706c9740 1 --2- 172.21.15.152:0/189379666 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5624014df150 0x5624014e1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:47.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.886+0000 7fa6706c9740 4 client.17598 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:00:47.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.886+0000 7fa6706c9740 1 -- 172.21.15.152:0/189379666 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5624013c6d70 con 0x5624014df150 2024-09-11T23:00:47.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.886+0000 7fa6677fe640 1 --2- 172.21.15.152:0/189379666 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5624014df150 0x5624014e1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:47.913 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.886+0000 7fa6477fe640 10 client.17598 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T23:00:47.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.886+0000 7fa6677fe640 1 --2- 172.21.15.152:0/189379666 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5624014df150 0x5624014e1570 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b8f39b3b1edd1c9b server_cookie=6ed1fe47fdd412bb in_seq=0 out_seq=0 2024-09-11T23:00:47.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa6477fe640 1 -- 172.21.15.152:0/189379666 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5624013c6d70 con 0x5624014df150 2024-09-11T23:00:47.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa6477fe640 10 client.17598 handle_command_reply: tid=0 2024-09-11T23:00:47.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa6706c9740 1 client.17598 shutdown 2024-09-11T23:00:47.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa6706c9740 1 -- 172.21.15.152:0/189379666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5624014595d0 msgr2=0x56240146ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:47.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa6706c9740 1 --2- 172.21.15.152:0/189379666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5624014595d0 0x56240146ccd0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa65800ad00 tx=0x7fa65800a550 comp rx=0 tx=0).stop 2024-09-11T23:00:47.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa6706c9740 1 -- 172.21.15.152:0/189379666 shutdown_connections 2024-09-11T23:00:47.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa6706c9740 1 --2- 172.21.15.152:0/189379666 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x5624014df150 0x5624014e1570 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa6706c9740 1 --2- 172.21.15.152:0/189379666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562401464370 0x56240145e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.914 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa6706c9740 1 --2- 172.21.15.152:0/189379666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56240145ab40 0x56240145dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa6706c9740 1 --2- 172.21.15.152:0/189379666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5624014595d0 0x56240146ccd0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa6477fe640 0 client.17598 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T23:00:47.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa6706c9740 1 -- 172.21.15.152:0/189379666 >> 172.21.15.152:0/189379666 conn(0x562401422310 msgr2=0x562401463090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:47.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa6706c9740 1 -- 172.21.15.152:0/189379666 shutdown_connections 2024-09-11T23:00:47.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa6706c9740 1 -- 172.21.15.152:0/189379666 wait complete. 2024-09-11T23:00:47.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa6706c9740 20 client.17598 trim_cache size 0 max 16384 2024-09-11T23:00:47.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa66e2f0640 1 -- 172.21.15.152:0/117457873 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6500a2cf0 msgr2=0x7fa6500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:47.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa66e2f0640 1 --2- 172.21.15.152:0/117457873 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6500a2cf0 0x7fa6500a51b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x56240145a070 tx=0x7fa65c012040 comp rx=0 tx=0).stop 2024-09-11T23:00:47.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa66e2f0640 1 -- 172.21.15.152:0/117457873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6681055b0 msgr2=0x7fa66810f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:47.915 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.887+0000 7fa66e2f0640 1 --2- 172.21.15.152:0/117457873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6681055b0 0x7fa66810f2d0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa660031a60 tx=0x7fa660031a90 comp rx=0 tx=0).stop 2024-09-11T23:00:47.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.888+0000 7fa66e2f0640 1 -- 172.21.15.152:0/117457873 shutdown_connections 2024-09-11T23:00:47.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.888+0000 7fa66e2f0640 1 --2- 172.21.15.152:0/117457873 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa6500a2cf0 0x7fa6500a51b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.888+0000 7fa66e2f0640 1 --2- 172.21.15.152:0/117457873 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa6681055b0 0x7fa66810f2d0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.888+0000 7fa66e2f0640 1 --2- 172.21.15.152:0/117457873 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa668100e30 0x7fa66810ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.888+0000 7fa66e2f0640 1 --2- 172.21.15.152:0/117457873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa668100480 0x7fa66810e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:47.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.888+0000 7fa66e2f0640 1 -- 172.21.15.152:0/117457873 >> 172.21.15.152:0/117457873 conn(0x7fa668077210 msgr2=0x7fa668103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:47.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.888+0000 7fa66e2f0640 1 -- 172.21.15.152:0/117457873 shutdown_connections 2024-09-11T23:00:47.916 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:47.888+0000 7fa66e2f0640 1 -- 172.21.15.152:0/117457873 wait complete. 2024-09-11T23:00:48.576 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:48 smithi152 ceph-mon[31181]: pgmap v242: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:48.603 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:00:48.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:48 smithi191 ceph-mon[35211]: pgmap v242: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:48.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:48 smithi204 ceph-mon[35454]: pgmap v242: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:48.853 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:49.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.322+0000 7fa508ff5640 1 -- 172.21.15.152:0/324509916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa504100a80 msgr2=0x7fa504100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:49.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.322+0000 7fa508ff5640 1 --2- 172.21.15.152:0/324509916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa504100a80 0x7fa504100ee0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009930 tx=0x7fa4f802a420 comp rx=0 tx=0).stop 2024-09-11T23:00:49.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.322+0000 7fa508ff5640 1 -- 172.21.15.152:0/324509916 shutdown_connections 2024-09-11T23:00:49.343 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.322+0000 7fa508ff5640 1 --2- 172.21.15.152:0/324509916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa504101420 0x7fa50410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.322+0000 7fa508ff5640 1 --2- 172.21.15.152:0/324509916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa504100a80 0x7fa504100ee0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.322+0000 7fa508ff5640 1 --2- 172.21.15.152:0/324509916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa504106a80 0x7fa504106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.322+0000 7fa508ff5640 1 -- 172.21.15.152:0/324509916 >> 172.21.15.152:0/324509916 conn(0x7fa5040fc7a0 msgr2=0x7fa5040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:49.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.322+0000 7fa508ff5640 1 -- 172.21.15.152:0/324509916 shutdown_connections 2024-09-11T23:00:49.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.322+0000 7fa508ff5640 1 -- 172.21.15.152:0/324509916 wait complete. 2024-09-11T23:00:49.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.322+0000 7fa508ff5640 1 Processor -- start 2024-09-11T23:00:49.344 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa508ff5640 1 -- start start 2024-09-11T23:00:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa508ff5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa504100a80 0x7fa5040fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa508ff5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa504101420 0x7fa5040fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa508ff5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa504106a80 0x7fa5040fd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa508ff5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa504110e40 con 0x7fa504106a80 2024-09-11T23:00:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa508ff5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa504110cc0 con 0x7fa504101420 2024-09-11T23:00:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa508ff5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa504110fc0 con 0x7fa504100a80 2024-09-11T23:00:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa502d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa504106a80 0x7fa5040fd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa502d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa504106a80 0x7fa5040fd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33298/0 (socket says 172.21.15.152:33298) 2024-09-11T23:00:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa502d9d640 1 -- 172.21.15.152:0/2364446574 learned_addr learned my addr 172.21.15.152:0/2364446574 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa50259c640 1 --2- 172.21.15.152:0/2364446574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa504100a80 0x7fa5040fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa501d9b640 1 --2- 172.21.15.152:0/2364446574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa504101420 0x7fa5040fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:49.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa502d9d640 1 -- 172.21.15.152:0/2364446574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa504100a80 msgr2=0x7fa5040fcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa502d9d640 1 --2- 172.21.15.152:0/2364446574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa504100a80 0x7fa5040fcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa502d9d640 1 -- 172.21.15.152:0/2364446574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa504101420 msgr2=0x7fa5040fd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa502d9d640 1 --2- 172.21.15.152:0/2364446574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa504101420 0x7fa5040fd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa502d9d640 1 -- 172.21.15.152:0/2364446574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5041a68a0 con 0x7fa504106a80 2024-09-11T23:00:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa501d9b640 1 --2- 172.21.15.152:0/2364446574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa504101420 0x7fa5040fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa50259c640 1 --2- 172.21.15.152:0/2364446574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa504100a80 0x7fa5040fcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:49.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa502d9d640 1 --2- 172.21.15.152:0/2364446574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa504106a80 0x7fa5040fd7b0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00d750 tx=0x7fa4fc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:49.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.323+0000 7fa4f37fe640 1 -- 172.21.15.152:0/2364446574 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc017070 con 0x7fa504106a80 2024-09-11T23:00:49.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.324+0000 7fa508ff5640 1 -- 172.21.15.152:0/2364446574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5041a6b90 con 0x7fa504106a80 2024-09-11T23:00:49.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.324+0000 7fa4f37fe640 1 -- 172.21.15.152:0/2364446574 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4fc005500 con 0x7fa504106a80 2024-09-11T23:00:49.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.324+0000 7fa4f37fe640 1 -- 172.21.15.152:0/2364446574 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc004d90 con 0x7fa504106a80 2024-09-11T23:00:49.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.324+0000 7fa508ff5640 1 -- 172.21.15.152:0/2364446574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5041a70a0 con 0x7fa504106a80 2024-09-11T23:00:49.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.326+0000 7fa4f37fe640 1 -- 172.21.15.152:0/2364446574 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa4fc004f30 con 0x7fa504106a80 2024-09-11T23:00:49.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.326+0000 7fa50b3ce740 1 Processor -- start 2024-09-11T23:00:49.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.326+0000 7fa50b3ce740 1 -- start start 2024-09-11T23:00:49.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa50b3ce740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b5b40 0x55f4545b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa50b3ce740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545b45d0 0x55f4545b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa50b3ce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545bf370 0x55f4545c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa50b3ce740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f45451a2f0 con 0x55f4545b5b40 2024-09-11T23:00:49.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa50b3ce740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f4544c1cf0 con 0x55f4545b45d0 2024-09-11T23:00:49.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa50b3ce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f4545162b0 con 0x55f4545bf370 2024-09-11T23:00:49.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa4f37fe640 1 --2- 172.21.15.152:0/2364446574 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa4ec0a2cf0 0x7fa4ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa501d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b5b40 0x55f4545b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa501d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b5b40 0x55f4545b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33306/0 (socket says 172.21.15.152:33306) 2024-09-11T23:00:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa501d9b640 1 -- 172.21.15.152:0/928698331 learned_addr learned my addr 172.21.15.152:0/928698331 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa502d9d640 1 --2- 172.21.15.152:0/2364446574 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa4ec0a2cf0 0x7fa4ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa4f37fe640 1 -- 172.21.15.152:0/2364446574 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7fa4fc0cef00 con 0x7fa504106a80 2024-09-11T23:00:49.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa50259c640 1 --2- 172.21.15.152:0/928698331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545b45d0 0x55f4545b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa501d9b640 1 --2- 172.21.15.152:0/928698331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545bf370 0x55f4545c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa501d9b640 1 -- 172.21.15.152:0/928698331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545bf370 msgr2=0x55f4545c1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa501d9b640 1 --2- 172.21.15.152:0/928698331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545bf370 0x55f4545c1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa501d9b640 1 -- 172.21.15.152:0/928698331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545b45d0 msgr2=0x55f4545b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa501d9b640 1 --2- 172.21.15.152:0/928698331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545b45d0 0x55f4545b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa501d9b640 1 -- 172.21.15.152:0/928698331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4544a63a0 con 0x55f4545b5b40 2024-09-11T23:00:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa502d9d640 1 --2- 172.21.15.152:0/2364446574 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa4ec0a2cf0 0x7fa4ec0a51b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00de60 tx=0x7fa4fc0908d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa501d9b640 1 --2- 172.21.15.152:0/928698331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b5b40 0x55f4545b4090 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x55f4545b5070 tx=0x7fa4f8031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee6f09e9be66018 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.327+0000 7fa4f0ff9640 1 -- 172.21.15.152:0/928698331 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8035050 con 0x55f4545b5b40 2024-09-11T23:00:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.328+0000 7fa4f0ff9640 1 -- 172.21.15.152:0/928698331 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f8031890 con 0x55f4545b5b40 2024-09-11T23:00:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.328+0000 7fa4f0ff9640 1 -- 172.21.15.152:0/928698331 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f80046c0 con 0x55f4545b5b40 2024-09-11T23:00:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.328+0000 7fa50b3ce740 1 -- 172.21.15.152:0/928698331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b5b40 msgr2=0x55f4545b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:49.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.328+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/928698331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b5b40 0x55f4545b4090 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x55f4545b5070 tx=0x7fa4f8031480 comp rx=0 tx=0).stop 2024-09-11T23:00:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.328+0000 7fa50b3ce740 1 -- 172.21.15.152:0/928698331 shutdown_connections 2024-09-11T23:00:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.328+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/928698331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545bf370 0x55f4545c1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.328+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/928698331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545b45d0 0x55f4545b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.328+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/928698331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b5b40 0x55f4545b4090 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.328+0000 7fa50b3ce740 1 -- 172.21.15.152:0/928698331 >> 172.21.15.152:0/928698331 conn(0x55f45457d310 msgr2=0x55f4545b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.328+0000 7fa50b3ce740 1 -- 172.21.15.152:0/928698331 shutdown_connections 2024-09-11T23:00:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.328+0000 7fa50b3ce740 1 -- 172.21.15.152:0/928698331 wait complete. 2024-09-11T23:00:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.329+0000 7fa50b3ce740 1 -- start start 2024-09-11T23:00:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.329+0000 7fa50b3ce740 1 Processor -- start 2024-09-11T23:00:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.329+0000 7fa50b3ce740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545b45d0 0x55f4545bed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.329+0000 7fa50b3ce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545b5b40 0x55f4545bbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.329+0000 7fa50b3ce740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545bf370 0x55f4545bc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.329+0000 7fa50b3ce740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55f4545c2010 con 0x55f4545bf370 2024-09-11T23:00:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.329+0000 7fa50b3ce740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55f4544d5060 con 0x55f4545b45d0 2024-09-11T23:00:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.329+0000 7fa50b3ce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55f4545c2190 con 0x55f4545b5b40 2024-09-11T23:00:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.329+0000 7fa50b3ce740 10 client.0 _ll_register_callbacks cb 0x55f4542288c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.329+0000 7fa501d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545bf370 0x55f4545bc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.329+0000 7fa501d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545bf370 0x55f4545bc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33316/0 (socket says 172.21.15.152:33316) 2024-09-11T23:00:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.329+0000 7fa501d9b640 1 -- 172.21.15.152:0/2375189874 learned_addr learned my addr 172.21.15.152:0/2375189874 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.329+0000 7fa501d9b640 1 --2- 172.21.15.152:0/2375189874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545b45d0 0x55f4545bed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.329+0000 7fa50259c640 1 --2- 172.21.15.152:0/2375189874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545b5b40 0x55f4545bbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.329+0000 7fa501d9b640 1 -- 172.21.15.152:0/2375189874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545b5b40 msgr2=0x55f4545bbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.329+0000 7fa501d9b640 1 --2- 172.21.15.152:0/2375189874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545b5b40 0x55f4545bbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa501d9b640 1 -- 172.21.15.152:0/2375189874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545b45d0 msgr2=0x55f4545bed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:49.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa501d9b640 1 --2- 172.21.15.152:0/2375189874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545b45d0 0x55f4545bed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa501d9b640 1 -- 172.21.15.152:0/2375189874 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f45428b3c0 con 0x55f4545bf370 2024-09-11T23:00:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa501d9b640 1 --2- 172.21.15.152:0/2375189874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545bf370 0x55f4545bc530 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa4f802a940 tx=0x7fa4f8005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91fe1c903c149491 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa4e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa4e9ffb640 1 -- 172.21.15.152:0/2375189874 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8035030 con 0x55f4545bf370 2024-09-11T23:00:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa4e9ffb640 1 -- 172.21.15.152:0/2375189874 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7fa4f8044040 con 0x55f4545bf370 2024-09-11T23:00:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa50b3ce740 1 -- 172.21.15.152:0/2375189874 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f8031890 con 0x55f4545bf370 2024-09-11T23:00:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa4e9ffb640 1 -- 172.21.15.152:0/2375189874 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4f8057050 con 0x55f4545bf370 2024-09-11T23:00:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa50b3ce740 10 client.17610 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:49.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa50b3ce740 1 -- 172.21.15.152:0/2375189874 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f4545cdaa0 con 0x55f4545bf370 2024-09-11T23:00:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa4e9ffb640 1 -- 172.21.15.152:0/2375189874 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f8031890 con 0x55f4545bf370 2024-09-11T23:00:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa4e9ffb640 1 -- 172.21.15.152:0/2375189874 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa4f8003680 con 0x55f4545bf370 2024-09-11T23:00:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa50b3ce740 10 client.17610 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa50b3ce740 10 client.17610 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:00:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa50b3ce740 20 client.17610 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/2375189874 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55f4545cedf0 0x55f4545d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa50b3ce740 4 client.17610 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:00:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.330+0000 7fa50b3ce740 1 -- 172.21.15.152:0/2375189874 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f454527860 con 0x55f4545cedf0 2024-09-11T23:00:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.331+0000 7fa50259c640 1 --2- 172.21.15.152:0/2375189874 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55f4545cedf0 0x55f4545d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:49.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.331+0000 7fa4e9ffb640 10 client.17610 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T23:00:49.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.331+0000 7fa50259c640 1 --2- 172.21.15.152:0/2375189874 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55f4545cedf0 0x55f4545d11e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=55b877e1acacabdb server_cookie=659cfaf8b3cf65ce in_seq=0 out_seq=0 2024-09-11T23:00:49.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.332+0000 7fa4e9ffb640 1 -- 172.21.15.152:0/2375189874 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f454527860 con 0x55f4545cedf0 2024-09-11T23:00:49.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.332+0000 7fa4e9ffb640 10 client.17610 handle_command_reply: tid=0 2024-09-11T23:00:49.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.332+0000 7fa50b3ce740 1 client.17610 shutdown 2024-09-11T23:00:49.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.332+0000 7fa50b3ce740 1 -- 172.21.15.152:0/2375189874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545bf370 msgr2=0x55f4545bc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:49.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.332+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/2375189874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545bf370 0x55f4545bc530 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fa4f802a940 tx=0x7fa4f8005500 comp rx=0 tx=0).stop 2024-09-11T23:00:49.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.332+0000 7fa50b3ce740 1 -- 172.21.15.152:0/2375189874 shutdown_connections 2024-09-11T23:00:49.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.332+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/2375189874 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55f4545cedf0 0x55f4545d11e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.332+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/2375189874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545bf370 0x55f4545bc530 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.332+0000 7fa4e9ffb640 0 client.17610 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T23:00:49.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.332+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/2375189874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545b5b40 0x55f4545bbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.332+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/2375189874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545b45d0 0x55f4545bed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.332+0000 7fa50b3ce740 1 -- 172.21.15.152:0/2375189874 >> 172.21.15.152:0/2375189874 conn(0x55f45457d310 msgr2=0x55f4545b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:49.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.332+0000 7fa50b3ce740 1 -- 172.21.15.152:0/2375189874 shutdown_connections 2024-09-11T23:00:49.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.332+0000 7fa50b3ce740 1 -- 172.21.15.152:0/2375189874 wait complete. 2024-09-11T23:00:49.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.332+0000 7fa50b3ce740 20 client.17610 trim_cache size 0 max 16384 2024-09-11T23:00:49.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.345+0000 7fa50b3ce740 1 Processor -- start 2024-09-11T23:00:49.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.345+0000 7fa50b3ce740 1 -- start start 2024-09-11T23:00:49.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.345+0000 7fa50b3ce740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b45d0 0x55f4545afc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.345+0000 7fa50b3ce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545b5b40 0x55f4545b7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.345+0000 7fa50b3ce740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545bf370 0x55f4545b7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.345+0000 7fa50b3ce740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041af870 con 0x55f4545b45d0 2024-09-11T23:00:49.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.345+0000 7fa50b3ce740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041b13b0 con 0x55f4545bf370 2024-09-11T23:00:49.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.345+0000 7fa50b3ce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041aff40 con 0x55f4545b5b40 2024-09-11T23:00:49.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.345+0000 7fa501d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b45d0 0x55f4545afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:49.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.345+0000 7fa501d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b45d0 0x55f4545afc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33330/0 (socket says 172.21.15.152:33330) 2024-09-11T23:00:49.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.345+0000 7fa501d9b640 1 -- 172.21.15.152:0/3139829735 learned_addr learned my addr 172.21.15.152:0/3139829735 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:49.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.346+0000 7fa50259c640 1 --2- 172.21.15.152:0/3139829735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545b5b40 0x55f4545b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:49.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.346+0000 7fa501d9b640 1 -- 172.21.15.152:0/3139829735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545b5b40 msgr2=0x55f4545b7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:49.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.346+0000 7fa501d9b640 1 --2- 172.21.15.152:0/3139829735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545b5b40 0x55f4545b7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.346+0000 7fa501d9b640 1 -- 172.21.15.152:0/3139829735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545bf370 msgr2=0x55f4545b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:49.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.346+0000 7fa501d9b640 1 --2- 172.21.15.152:0/3139829735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545bf370 0x55f4545b7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.346+0000 7fa501d9b640 1 -- 172.21.15.152:0/3139829735 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4545baf60 con 0x55f4545b45d0 2024-09-11T23:00:49.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.346+0000 7fa50259c640 1 --2- 172.21.15.152:0/3139829735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545b5b40 0x55f4545b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:00:49.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.346+0000 7fa501d9b640 1 --2- 172.21.15.152:0/3139829735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b45d0 0x55f4545afc10 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8032c00 tx=0x7fa4f802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a186095e4b6850 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:49.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.346+0000 7fa4ebfff640 1 -- 172.21.15.152:0/3139829735 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8035070 con 0x55f4545b45d0 2024-09-11T23:00:49.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.346+0000 7fa4ebfff640 1 -- 172.21.15.152:0/3139829735 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f8004340 con 0x55f4545b45d0 2024-09-11T23:00:49.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.346+0000 7fa4ebfff640 1 -- 172.21.15.152:0/3139829735 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8004ba0 con 0x55f4545b45d0 2024-09-11T23:00:49.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 -- 172.21.15.152:0/3139829735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b45d0 msgr2=0x55f4545afc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:49.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/3139829735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b45d0 0x55f4545afc10 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8032c00 tx=0x7fa4f802a420 comp rx=0 tx=0).stop 2024-09-11T23:00:49.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 -- 172.21.15.152:0/3139829735 shutdown_connections 2024-09-11T23:00:49.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/3139829735 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545bf370 0x55f4545b7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/3139829735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545b5b40 0x55f4545b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/3139829735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b45d0 0x55f4545afc10 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 -- 172.21.15.152:0/3139829735 >> 172.21.15.152:0/3139829735 conn(0x55f45457d310 msgr2=0x55f4545af500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:49.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 -- 172.21.15.152:0/3139829735 shutdown_connections 2024-09-11T23:00:49.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 -- 172.21.15.152:0/3139829735 wait complete. 2024-09-11T23:00:49.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 -- start start 2024-09-11T23:00:49.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 Processor -- start 2024-09-11T23:00:49.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b45d0 0x55f4545c3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545b5b40 0x55f4545c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545bf370 0x55f4545c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041b0ce0 con 0x55f4545b45d0 2024-09-11T23:00:49.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041b0610 con 0x55f4545b5b40 2024-09-11T23:00:49.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa5041ad870 con 0x55f4545bf370 2024-09-11T23:00:49.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa50b3ce740 10 client.0 _ll_register_callbacks cb 0x7fa4f40075e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:49.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.347+0000 7fa501d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b45d0 0x55f4545c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:49.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa501d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b45d0 0x55f4545c3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33336/0 (socket says 172.21.15.152:33336) 2024-09-11T23:00:49.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa501d9b640 1 -- 172.21.15.152:0/1144310287 learned_addr learned my addr 172.21.15.152:0/1144310287 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:49.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa50259c640 1 --2- 172.21.15.152:0/1144310287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545b5b40 0x55f4545c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:49.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa501d9b640 1 -- 172.21.15.152:0/1144310287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545bf370 msgr2=0x55f4545c3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:49.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa501d9b640 1 --2- 172.21.15.152:0/1144310287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545bf370 0x55f4545c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa501d9b640 1 -- 172.21.15.152:0/1144310287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545b5b40 msgr2=0x55f4545c37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:49.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa501d9b640 1 --2- 172.21.15.152:0/1144310287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545b5b40 0x55f4545c37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa501d9b640 1 -- 172.21.15.152:0/1144310287 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f454638890 con 0x55f4545b45d0 2024-09-11T23:00:49.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa501d9b640 1 --2- 172.21.15.152:0/1144310287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b45d0 0x55f4545c3290 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fa4f802a940 tx=0x7fa4f8032d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6843f0bbe7d57d8a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:49.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa4e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:49.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa4e9ffb640 1 -- 172.21.15.152:0/1144310287 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8035040 con 0x55f4545b45d0 2024-09-11T23:00:49.370 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa4e9ffb640 1 -- 172.21.15.152:0/1144310287 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7fa4f804f040 con 0x55f4545b45d0 2024-09-11T23:00:49.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa50b3ce740 1 -- 172.21.15.152:0/1144310287 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f8004340 con 0x55f4545b45d0 2024-09-11T23:00:49.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa4e9ffb640 1 -- 172.21.15.152:0/1144310287 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4f80316f0 con 0x55f4545b45d0 2024-09-11T23:00:49.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa50b3ce740 10 client.17616 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:49.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa50b3ce740 1 -- 172.21.15.152:0/1144310287 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f454639ae0 con 0x55f4545b45d0 2024-09-11T23:00:49.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.348+0000 7fa4e9ffb640 1 -- 172.21.15.152:0/1144310287 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f8004340 con 0x55f4545b45d0 2024-09-11T23:00:49.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.349+0000 7fa4e9ffb640 1 -- 172.21.15.152:0/1144310287 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa4f806b030 con 0x55f4545b45d0 2024-09-11T23:00:49.371 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.349+0000 7fa50b3ce740 10 client.17616 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:49.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.349+0000 7fa50b3ce740 10 client.17616 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:00:49.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.349+0000 7fa50b3ce740 20 client.17616 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:49.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.349+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/1144310287 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55f45463a0e0 0x55f45463c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:49.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.349+0000 7fa50b3ce740 4 client.17616 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:00:49.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.349+0000 7fa50b3ce740 1 -- 172.21.15.152:0/1144310287 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f454521d70 con 0x55f45463a0e0 2024-09-11T23:00:49.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.349+0000 7fa50259c640 1 --2- 172.21.15.152:0/1144310287 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55f45463a0e0 0x55f45463c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:49.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.349+0000 7fa50259c640 1 --2- 172.21.15.152:0/1144310287 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55f45463a0e0 0x55f45463c4d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eac2bab581db2c7 server_cookie=30e5eea44b95de82 in_seq=0 out_seq=0 2024-09-11T23:00:49.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.349+0000 7fa4e9ffb640 10 client.17616 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T23:00:49.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa4e9ffb640 1 -- 172.21.15.152:0/1144310287 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f454521d70 con 0x55f45463a0e0 2024-09-11T23:00:49.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa4e9ffb640 10 client.17616 handle_command_reply: tid=0 2024-09-11T23:00:49.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa50b3ce740 1 client.17616 shutdown 2024-09-11T23:00:49.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa50b3ce740 1 -- 172.21.15.152:0/1144310287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b45d0 msgr2=0x55f4545c3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:49.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/1144310287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b45d0 0x55f4545c3290 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fa4f802a940 tx=0x7fa4f8032d20 comp rx=0 tx=0).stop 2024-09-11T23:00:49.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa50b3ce740 1 -- 172.21.15.152:0/1144310287 shutdown_connections 2024-09-11T23:00:49.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/1144310287 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55f45463a0e0 0x55f45463c4d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/1144310287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f4545bf370 0x55f4545c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa4e9ffb640 0 client.17616 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T23:00:49.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/1144310287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55f4545b5b40 0x55f4545c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa50b3ce740 1 --2- 172.21.15.152:0/1144310287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f4545b45d0 0x55f4545c3290 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa50b3ce740 1 -- 172.21.15.152:0/1144310287 >> 172.21.15.152:0/1144310287 conn(0x55f45457d310 msgr2=0x55f4545be030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:49.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa50b3ce740 1 -- 172.21.15.152:0/1144310287 shutdown_connections 2024-09-11T23:00:49.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa50b3ce740 1 -- 172.21.15.152:0/1144310287 wait complete. 2024-09-11T23:00:49.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa50b3ce740 20 client.17616 trim_cache size 0 max 16384 2024-09-11T23:00:49.375 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:00:49.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa508ff5640 1 -- 172.21.15.152:0/2364446574 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa4ec0a2cf0 msgr2=0x7fa4ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:49.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa508ff5640 1 --2- 172.21.15.152:0/2364446574 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa4ec0a2cf0 0x7fa4ec0a51b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00de60 tx=0x7fa4fc0908d0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa508ff5640 1 -- 172.21.15.152:0/2364446574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa504106a80 msgr2=0x7fa5040fd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:49.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa508ff5640 1 --2- 172.21.15.152:0/2364446574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa504106a80 0x7fa5040fd7b0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00d750 tx=0x7fa4fc00dc10 comp rx=0 tx=0).stop 2024-09-11T23:00:49.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa508ff5640 1 -- 172.21.15.152:0/2364446574 shutdown_connections 2024-09-11T23:00:49.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa508ff5640 1 --2- 172.21.15.152:0/2364446574 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa4ec0a2cf0 0x7fa4ec0a51b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa508ff5640 1 --2- 172.21.15.152:0/2364446574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa504106a80 0x7fa5040fd7b0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa508ff5640 1 --2- 172.21.15.152:0/2364446574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa504101420 0x7fa5040fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa508ff5640 1 --2- 172.21.15.152:0/2364446574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa504100a80 0x7fa5040fcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:49.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.350+0000 7fa508ff5640 1 -- 172.21.15.152:0/2364446574 >> 172.21.15.152:0/2364446574 conn(0x7fa5040fc7a0 msgr2=0x7fa50406a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:49.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.351+0000 7fa508ff5640 1 -- 172.21.15.152:0/2364446574 shutdown_connections 2024-09-11T23:00:49.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:49.351+0000 7fa508ff5640 1 -- 172.21.15.152:0/2364446574 wait complete. 2024-09-11T23:00:50.123 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:00:50.372 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:50.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:50 smithi152 ceph-mon[31181]: pgmap v243: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-11T23:00:50.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:50 smithi191 ceph-mon[35211]: pgmap v243: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-11T23:00:50.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:50 smithi204 ceph-mon[35454]: pgmap v243: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 12 op/s 2024-09-11T23:00:50.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.857+0000 7f4742fe5640 1 -- 172.21.15.152:0/957231635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f473c100e20 msgr2=0x7f473c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:50.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.857+0000 7f4742fe5640 1 --2- 172.21.15.152:0/957231635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f473c100e20 0x7f473c101200 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f472c009ca0 tx=0x7f472c02a950 comp rx=0 tx=0).stop 2024-09-11T23:00:50.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.858+0000 7f4742fe5640 1 -- 172.21.15.152:0/957231635 shutdown_connections 2024-09-11T23:00:50.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.858+0000 7f4742fe5640 1 --2- 172.21.15.152:0/957231635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c102ef0 0x7f473c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.858+0000 7f4742fe5640 1 --2- 172.21.15.152:0/957231635 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f473c102570 0x7f473c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.858+0000 7f4742fe5640 1 --2- 172.21.15.152:0/957231635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f473c100e20 0x7f473c101200 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.858+0000 7f4742fe5640 1 -- 172.21.15.152:0/957231635 >> 172.21.15.152:0/957231635 conn(0x7f473c0fc7d0 msgr2=0x7f473c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:50.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.858+0000 7f4742fe5640 1 -- 172.21.15.152:0/957231635 shutdown_connections 2024-09-11T23:00:50.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.858+0000 7f4742fe5640 1 -- 172.21.15.152:0/957231635 wait complete. 2024-09-11T23:00:50.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.858+0000 7f4742fe5640 1 Processor -- start 2024-09-11T23:00:50.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.858+0000 7f4742fe5640 1 -- start start 2024-09-11T23:00:50.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.858+0000 7f4742fe5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c100e20 0x7f473c1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.858+0000 7f4742fe5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f473c102570 0x7f473c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.858+0000 7f4742fe5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f473c102ef0 0x7f473c1a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.858+0000 7f4742fe5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f473c110fa0 con 0x7f473c102570 2024-09-11T23:00:50.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.858+0000 7f4742fe5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f473c110e20 con 0x7f473c102ef0 2024-09-11T23:00:50.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.858+0000 7f4742fe5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f473c111120 con 0x7f473c100e20 2024-09-11T23:00:50.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.858+0000 7f473bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f473c102570 0x7f473c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.859+0000 7f473bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f473c102570 0x7f473c199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33344/0 (socket says 172.21.15.152:33344) 2024-09-11T23:00:50.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.859+0000 7f473bfff640 1 -- 172.21.15.152:0/28160495 learned_addr learned my addr 172.21.15.152:0/28160495 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:50.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.859+0000 7f4740d81640 1 --2- 172.21.15.152:0/28160495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c100e20 0x7f473c1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.859+0000 7f4741582640 1 --2- 172.21.15.152:0/28160495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f473c102ef0 0x7f473c1a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.859+0000 7f473bfff640 1 -- 172.21.15.152:0/28160495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c100e20 msgr2=0x7f473c1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:50.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.859+0000 7f473bfff640 1 --2- 172.21.15.152:0/28160495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c100e20 0x7f473c1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.859+0000 7f473bfff640 1 -- 172.21.15.152:0/28160495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f473c102ef0 msgr2=0x7f473c1a4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:50.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.859+0000 7f473bfff640 1 --2- 172.21.15.152:0/28160495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f473c102ef0 0x7f473c1a4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.859+0000 7f473bfff640 1 -- 172.21.15.152:0/28160495 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c1a49b0 con 0x7f473c102570 2024-09-11T23:00:50.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.859+0000 7f4741582640 1 --2- 172.21.15.152:0/28160495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f473c102ef0 0x7f473c1a4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:00:50.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.859+0000 7f473bfff640 1 --2- 172.21.15.152:0/28160495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f473c102570 0x7f473c199930 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f472400b4a0 tx=0x7f472400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:50.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.859+0000 7f4739ffb640 1 -- 172.21.15.152:0/28160495 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724014070 con 0x7f473c102570 2024-09-11T23:00:50.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.859+0000 7f4742fe5640 1 -- 172.21.15.152:0/28160495 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473c1a4b80 con 0x7f473c102570 2024-09-11T23:00:50.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.859+0000 7f4739ffb640 1 -- 172.21.15.152:0/28160495 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4724010af0 con 0x7f473c102570 2024-09-11T23:00:50.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.859+0000 7f4739ffb640 1 -- 172.21.15.152:0/28160495 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724017d10 con 0x7f473c102570 2024-09-11T23:00:50.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.859+0000 7f4742fe5640 1 -- 172.21.15.152:0/28160495 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473c1a50c0 con 0x7f473c102570 2024-09-11T23:00:50.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.861+0000 7f4739ffb640 1 -- 172.21.15.152:0/28160495 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4724017070 con 0x7f473c102570 2024-09-11T23:00:50.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.862+0000 7f47453be740 1 Processor -- start 2024-09-11T23:00:50.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.862+0000 7f47453be740 1 -- start start 2024-09-11T23:00:50.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.862+0000 7f47453be740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473cb40 0x560de473b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.862+0000 7f47453be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473b5d0 0x560de473ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.862+0000 7f47453be740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 0x560de4748760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.862+0000 7f47453be740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560de46a12f0 con 0x560de473cb40 2024-09-11T23:00:50.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.862+0000 7f47453be740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560de4648cf0 con 0x560de4746370 2024-09-11T23:00:50.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.862+0000 7f47453be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560de469d2b0 con 0x560de473b5d0 2024-09-11T23:00:50.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.862+0000 7f4739ffb640 1 --2- 172.21.15.152:0/28160495 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f47280a2cf0 0x7f47280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.862+0000 7f4741582640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473cb40 0x560de473b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.862+0000 7f4741582640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473cb40 0x560de473b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33352/0 (socket says 172.21.15.152:33352) 2024-09-11T23:00:50.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.862+0000 7f4741582640 1 -- 172.21.15.152:0/2088571703 learned_addr learned my addr 172.21.15.152:0/2088571703 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:50.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.862+0000 7f4740d81640 1 --2- 172.21.15.152:0/2088571703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473b5d0 0x560de473ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.862+0000 7f4741582640 1 --2- 172.21.15.152:0/28160495 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f47280a2cf0 0x7f47280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.862+0000 7f4739ffb640 1 -- 172.21.15.152:0/28160495 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f47240ce7a0 con 0x7f473c102570 2024-09-11T23:00:50.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.862+0000 7f473bfff640 1 --2- 172.21.15.152:0/2088571703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 0x560de4748760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.863+0000 7f4741582640 1 -- 172.21.15.152:0/2088571703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473b5d0 msgr2=0x560de473ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:50.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.863+0000 7f4741582640 1 --2- 172.21.15.152:0/2088571703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473b5d0 0x560de473ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.863+0000 7f4741582640 1 -- 172.21.15.152:0/2088571703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 msgr2=0x560de4748760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:50.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.863+0000 7f4741582640 1 --2- 172.21.15.152:0/2088571703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 0x560de4748760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.863+0000 7f4741582640 1 -- 172.21.15.152:0/2088571703 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560de462d3a0 con 0x560de473cb40 2024-09-11T23:00:50.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.863+0000 7f473bfff640 1 --2- 172.21.15.152:0/2088571703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 0x560de4748760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:50.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.863+0000 7f4741582640 1 --2- 172.21.15.152:0/2088571703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473cb40 0x560de473b090 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f4734012ac0 tx=0x7f4734012f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4c0a242d2db4d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:50.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.863+0000 7f4741582640 1 --2- 172.21.15.152:0/28160495 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f47280a2cf0 0x7f47280a51b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f473400f7f0 tx=0x7f473401a000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:50.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.863+0000 7f4732ffd640 1 -- 172.21.15.152:0/2088571703 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473400ab80 con 0x560de473cb40 2024-09-11T23:00:50.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.863+0000 7f4732ffd640 1 -- 172.21.15.152:0/2088571703 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473400de50 con 0x560de473cb40 2024-09-11T23:00:50.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.863+0000 7f4732ffd640 1 -- 172.21.15.152:0/2088571703 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473401c410 con 0x560de473cb40 2024-09-11T23:00:50.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.864+0000 7f47453be740 1 -- 172.21.15.152:0/2088571703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473cb40 msgr2=0x560de473b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:50.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.864+0000 7f47453be740 1 --2- 172.21.15.152:0/2088571703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473cb40 0x560de473b090 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f4734012ac0 tx=0x7f4734012f80 comp rx=0 tx=0).stop 2024-09-11T23:00:50.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.864+0000 7f47453be740 1 -- 172.21.15.152:0/2088571703 shutdown_connections 2024-09-11T23:00:50.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.864+0000 7f47453be740 1 --2- 172.21.15.152:0/2088571703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 0x560de4748760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.864+0000 7f47453be740 1 --2- 172.21.15.152:0/2088571703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473b5d0 0x560de473ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.864+0000 7f47453be740 1 --2- 172.21.15.152:0/2088571703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473cb40 0x560de473b090 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.864+0000 7f47453be740 1 -- 172.21.15.152:0/2088571703 >> 172.21.15.152:0/2088571703 conn(0x560de4704310 msgr2=0x560de4737c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:50.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.864+0000 7f47453be740 1 -- 172.21.15.152:0/2088571703 shutdown_connections 2024-09-11T23:00:50.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.864+0000 7f47453be740 1 -- 172.21.15.152:0/2088571703 wait complete. 2024-09-11T23:00:50.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.864+0000 7f47453be740 1 -- start start 2024-09-11T23:00:50.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.864+0000 7f47453be740 1 Processor -- start 2024-09-11T23:00:50.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f47453be740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de473b5d0 0x560de4745d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f47453be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473cb40 0x560de4742fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f47453be740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de4746370 0x560de47435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f47453be740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560de4749010 con 0x560de4746370 2024-09-11T23:00:50.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f47453be740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560de465c060 con 0x560de473b5d0 2024-09-11T23:00:50.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f47453be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560de4749190 con 0x560de473cb40 2024-09-11T23:00:50.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f47453be740 10 client.0 _ll_register_callbacks cb 0x560de43af8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:50.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f4741582640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de4746370 0x560de47435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f4741582640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de4746370 0x560de47435d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33362/0 (socket says 172.21.15.152:33362) 2024-09-11T23:00:50.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f4741582640 1 -- 172.21.15.152:0/4282661659 learned_addr learned my addr 172.21.15.152:0/4282661659 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:50.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f473bfff640 1 --2- 172.21.15.152:0/4282661659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473cb40 0x560de4742fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f4740d81640 1 --2- 172.21.15.152:0/4282661659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de473b5d0 0x560de4745d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f4741582640 1 -- 172.21.15.152:0/4282661659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473cb40 msgr2=0x560de4742fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:50.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f4741582640 1 --2- 172.21.15.152:0/4282661659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473cb40 0x560de4742fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f4741582640 1 -- 172.21.15.152:0/4282661659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de473b5d0 msgr2=0x560de4745d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:50.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f4741582640 1 --2- 172.21.15.152:0/4282661659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de473b5d0 0x560de4745d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f4741582640 1 -- 172.21.15.152:0/4282661659 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560de44123c0 con 0x560de4746370 2024-09-11T23:00:50.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f4740d81640 1 --2- 172.21.15.152:0/4282661659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de473b5d0 0x560de4745d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:50.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f4741582640 1 --2- 172.21.15.152:0/4282661659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de4746370 0x560de47435d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f4734012740 tx=0x7f473400af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417aa9ba32508787 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:50.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f471bfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:50.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f471bfff640 1 -- 172.21.15.152:0/4282661659 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734033060 con 0x560de4746370 2024-09-11T23:00:50.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f471bfff640 1 -- 172.21.15.152:0/4282661659 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f4734024450 con 0x560de4746370 2024-09-11T23:00:50.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.865+0000 7f47453be740 1 -- 172.21.15.152:0/4282661659 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f473400de50 con 0x560de4746370 2024-09-11T23:00:50.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.866+0000 7f471bfff640 1 -- 172.21.15.152:0/4282661659 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f473402a050 con 0x560de4746370 2024-09-11T23:00:50.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.866+0000 7f47453be740 10 client.17628 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:50.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.866+0000 7f47453be740 1 -- 172.21.15.152:0/4282661659 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560de4754a70 con 0x560de4746370 2024-09-11T23:00:50.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.866+0000 7f471bfff640 1 -- 172.21.15.152:0/4282661659 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f473400de50 con 0x560de4746370 2024-09-11T23:00:50.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.866+0000 7f471bfff640 1 -- 172.21.15.152:0/4282661659 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f473401b070 con 0x560de4746370 2024-09-11T23:00:50.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.866+0000 7f47453be740 10 client.17628 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:50.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.866+0000 7f47453be740 10 client.17628 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:00:50.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.866+0000 7f47453be740 20 client.17628 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:50.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.866+0000 7f47453be740 1 --2- 172.21.15.152:0/4282661659 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560de4755f30 0x560de4758320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.866+0000 7f47453be740 4 client.17628 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:00:50.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.866+0000 7f47453be740 1 -- 172.21.15.152:0/4282661659 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560de46ae860 con 0x560de4755f30 2024-09-11T23:00:50.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.866+0000 7f4740d81640 1 --2- 172.21.15.152:0/4282661659 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560de4755f30 0x560de4758320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.867+0000 7f4740d81640 1 --2- 172.21.15.152:0/4282661659 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560de4755f30 0x560de4758320 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6b2ba774518dc3b server_cookie=b255a0ad5dff544 in_seq=0 out_seq=0 2024-09-11T23:00:50.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.867+0000 7f471bfff640 10 client.17628 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T23:00:50.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.867+0000 7f471bfff640 1 -- 172.21.15.152:0/4282661659 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560de46ae860 con 0x560de4755f30 2024-09-11T23:00:50.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.867+0000 7f471bfff640 10 client.17628 handle_command_reply: tid=0 2024-09-11T23:00:50.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.867+0000 7f47453be740 1 client.17628 shutdown 2024-09-11T23:00:50.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.868+0000 7f47453be740 1 -- 172.21.15.152:0/4282661659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de4746370 msgr2=0x560de47435d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:50.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.868+0000 7f47453be740 1 --2- 172.21.15.152:0/4282661659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de4746370 0x560de47435d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f4734012740 tx=0x7f473400af20 comp rx=0 tx=0).stop 2024-09-11T23:00:50.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.868+0000 7f47453be740 1 -- 172.21.15.152:0/4282661659 shutdown_connections 2024-09-11T23:00:50.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.868+0000 7f47453be740 1 --2- 172.21.15.152:0/4282661659 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560de4755f30 0x560de4758320 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.868+0000 7f47453be740 1 --2- 172.21.15.152:0/4282661659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de4746370 0x560de47435d0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.868+0000 7f47453be740 1 --2- 172.21.15.152:0/4282661659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473cb40 0x560de4742fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.868+0000 7f471bfff640 0 client.17628 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T23:00:50.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.868+0000 7f47453be740 1 --2- 172.21.15.152:0/4282661659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de473b5d0 0x560de4745d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.868+0000 7f47453be740 1 -- 172.21.15.152:0/4282661659 >> 172.21.15.152:0/4282661659 conn(0x560de4704310 msgr2=0x560de473d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:50.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.868+0000 7f47453be740 1 -- 172.21.15.152:0/4282661659 shutdown_connections 2024-09-11T23:00:50.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.868+0000 7f47453be740 1 -- 172.21.15.152:0/4282661659 wait complete. 2024-09-11T23:00:50.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.868+0000 7f47453be740 20 client.17628 trim_cache size 0 max 16384 2024-09-11T23:00:50.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f47453be740 1 Processor -- start 2024-09-11T23:00:50.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f47453be740 1 -- start start 2024-09-11T23:00:50.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f47453be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473b5d0 0x560de4745310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f47453be740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473cb40 0x560de47522b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f47453be740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 0x560de473e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f47453be740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1ad8b0 con 0x560de473cb40 2024-09-11T23:00:50.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f47453be740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1af3f0 con 0x560de4746370 2024-09-11T23:00:50.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f47453be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1adf80 con 0x560de473b5d0 2024-09-11T23:00:50.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f473bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473cb40 0x560de47522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f473bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473cb40 0x560de47522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33374/0 (socket says 172.21.15.152:33374) 2024-09-11T23:00:50.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f473bfff640 1 -- 172.21.15.152:0/2817867276 learned_addr learned my addr 172.21.15.152:0/2817867276 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:50.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f4741582640 1 --2- 172.21.15.152:0/2817867276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 0x560de473e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f4740d81640 1 --2- 172.21.15.152:0/2817867276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473b5d0 0x560de4745310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.891 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f473bfff640 1 -- 172.21.15.152:0/2817867276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473b5d0 msgr2=0x560de4745310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:50.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f473bfff640 1 --2- 172.21.15.152:0/2817867276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473b5d0 0x560de4745310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f473bfff640 1 -- 172.21.15.152:0/2817867276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 msgr2=0x560de473e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:50.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f473bfff640 1 --2- 172.21.15.152:0/2817867276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 0x560de473e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f473bfff640 1 -- 172.21.15.152:0/2817867276 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560de473c780 con 0x560de473cb40 2024-09-11T23:00:50.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.881+0000 7f4741582640 1 --2- 172.21.15.152:0/2817867276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 0x560de473e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:00:50.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.882+0000 7f473bfff640 1 --2- 172.21.15.152:0/2817867276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473cb40 0x560de47522b0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f47240cc020 tx=0x7f4724012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fdf59ab8180e97 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:50.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.882+0000 7f47327fc640 1 -- 172.21.15.152:0/2817867276 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472401e040 con 0x560de473cb40 2024-09-11T23:00:50.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.882+0000 7f47327fc640 1 -- 172.21.15.152:0/2817867276 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f472409f140 con 0x560de473cb40 2024-09-11T23:00:50.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.882+0000 7f47327fc640 1 -- 172.21.15.152:0/2817867276 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47240cc130 con 0x560de473cb40 2024-09-11T23:00:50.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.882+0000 7f47453be740 1 -- 172.21.15.152:0/2817867276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473cb40 msgr2=0x560de47522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:50.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.882+0000 7f47453be740 1 --2- 172.21.15.152:0/2817867276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473cb40 0x560de47522b0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f47240cc020 tx=0x7f4724012040 comp rx=0 tx=0).stop 2024-09-11T23:00:50.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.882+0000 7f47453be740 1 -- 172.21.15.152:0/2817867276 shutdown_connections 2024-09-11T23:00:50.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.882+0000 7f47453be740 1 --2- 172.21.15.152:0/2817867276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 0x560de473e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.882+0000 7f47453be740 1 --2- 172.21.15.152:0/2817867276 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473cb40 0x560de47522b0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.882+0000 7f47453be740 1 --2- 172.21.15.152:0/2817867276 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473b5d0 0x560de4745310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.882+0000 7f47453be740 1 -- 172.21.15.152:0/2817867276 >> 172.21.15.152:0/2817867276 conn(0x560de4704310 msgr2=0x560de4736d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:50.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.882+0000 7f47453be740 1 -- 172.21.15.152:0/2817867276 shutdown_connections 2024-09-11T23:00:50.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.882+0000 7f47453be740 1 -- 172.21.15.152:0/2817867276 wait complete. 2024-09-11T23:00:50.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.883+0000 7f47453be740 1 -- start start 2024-09-11T23:00:50.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.883+0000 7f47453be740 1 Processor -- start 2024-09-11T23:00:50.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.883+0000 7f47453be740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473b5d0 0x560de474e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.883+0000 7f47453be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473cb40 0x560de473fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.883+0000 7f47453be740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 0x560de4740090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.883+0000 7f47453be740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1aed20 con 0x560de473b5d0 2024-09-11T23:00:50.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.883+0000 7f47453be740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1ae650 con 0x560de4746370 2024-09-11T23:00:50.893 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.883+0000 7f47453be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1ab8b0 con 0x560de473cb40 2024-09-11T23:00:50.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.883+0000 7f47453be740 10 client.0 _ll_register_callbacks cb 0x7f472c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:50.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.883+0000 7f4740d81640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473b5d0 0x560de474e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.883+0000 7f4740d81640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473b5d0 0x560de474e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33376/0 (socket says 172.21.15.152:33376) 2024-09-11T23:00:50.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.883+0000 7f4740d81640 1 -- 172.21.15.152:0/1541903267 learned_addr learned my addr 172.21.15.152:0/1541903267 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:50.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.883+0000 7f473bfff640 1 --2- 172.21.15.152:0/1541903267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473cb40 0x560de473fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f4741582640 1 --2- 172.21.15.152:0/1541903267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 0x560de4740090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f4740d81640 1 -- 172.21.15.152:0/1541903267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473cb40 msgr2=0x560de473fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:50.894 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f4740d81640 1 --2- 172.21.15.152:0/1541903267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473cb40 0x560de473fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f4740d81640 1 -- 172.21.15.152:0/1541903267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 msgr2=0x560de4740090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:50.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f4740d81640 1 --2- 172.21.15.152:0/1541903267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 0x560de4740090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f4740d81640 1 -- 172.21.15.152:0/1541903267 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560de4740680 con 0x560de473b5d0 2024-09-11T23:00:50.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f4741582640 1 --2- 172.21.15.152:0/1541903267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 0x560de4740090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:00:50.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f4740d81640 1 --2- 172.21.15.152:0/1541903267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473b5d0 0x560de474e9a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f472c005e30 tx=0x7f472c02fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52df15081c74d84 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:50.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f471bfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:50.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f471bfff640 1 -- 172.21.15.152:0/1541903267 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c04b070 con 0x560de473b5d0 2024-09-11T23:00:50.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f471bfff640 1 -- 172.21.15.152:0/1541903267 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f472c003970 con 0x560de473b5d0 2024-09-11T23:00:50.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f47453be740 1 -- 172.21.15.152:0/1541903267 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f472409f140 con 0x560de473b5d0 2024-09-11T23:00:50.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f471bfff640 1 -- 172.21.15.152:0/1541903267 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f472c057050 con 0x560de473b5d0 2024-09-11T23:00:50.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f47453be740 10 client.17634 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:50.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f47453be740 1 -- 172.21.15.152:0/1541903267 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560de47c0670 con 0x560de473b5d0 2024-09-11T23:00:50.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f471bfff640 1 -- 172.21.15.152:0/1541903267 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f472409f140 con 0x560de473b5d0 2024-09-11T23:00:50.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.884+0000 7f471bfff640 1 -- 172.21.15.152:0/1541903267 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f472c0046b0 con 0x560de473b5d0 2024-09-11T23:00:50.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.885+0000 7f47453be740 10 client.17634 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:50.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.885+0000 7f47453be740 10 client.17634 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:00:50.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.885+0000 7f47453be740 20 client.17634 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:50.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.885+0000 7f47453be740 1 --2- 172.21.15.152:0/1541903267 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560de47c0e20 0x560de47c3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:50.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.885+0000 7f47453be740 4 client.17634 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:00:50.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.885+0000 7f47453be740 1 -- 172.21.15.152:0/1541903267 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560de46a8d70 con 0x560de47c0e20 2024-09-11T23:00:50.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.885+0000 7f473bfff640 1 --2- 172.21.15.152:0/1541903267 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560de47c0e20 0x560de47c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:50.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.885+0000 7f471bfff640 10 client.17634 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T23:00:50.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.885+0000 7f473bfff640 1 --2- 172.21.15.152:0/1541903267 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560de47c0e20 0x560de47c3240 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b8692ab91f52a14 server_cookie=ac799e6f173d5731 in_seq=0 out_seq=0 2024-09-11T23:00:50.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.885+0000 7f471bfff640 1 -- 172.21.15.152:0/1541903267 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560de46a8d70 con 0x560de47c0e20 2024-09-11T23:00:50.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.885+0000 7f471bfff640 10 client.17634 handle_command_reply: tid=0 2024-09-11T23:00:50.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.885+0000 7f47453be740 1 client.17634 shutdown 2024-09-11T23:00:50.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f47453be740 1 -- 172.21.15.152:0/1541903267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473b5d0 msgr2=0x560de474e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:50.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f47453be740 1 --2- 172.21.15.152:0/1541903267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473b5d0 0x560de474e9a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f472c005e30 tx=0x7f472c02fb10 comp rx=0 tx=0).stop 2024-09-11T23:00:50.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f47453be740 1 -- 172.21.15.152:0/1541903267 shutdown_connections 2024-09-11T23:00:50.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f47453be740 1 --2- 172.21.15.152:0/1541903267 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x560de47c0e20 0x560de47c3240 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f47453be740 1 --2- 172.21.15.152:0/1541903267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560de4746370 0x560de4740090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f471bfff640 0 client.17634 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T23:00:50.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f47453be740 1 --2- 172.21.15.152:0/1541903267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560de473cb40 0x560de473fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f47453be740 1 --2- 172.21.15.152:0/1541903267 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560de473b5d0 0x560de474e9a0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f47453be740 1 -- 172.21.15.152:0/1541903267 >> 172.21.15.152:0/1541903267 conn(0x560de4704310 msgr2=0x560de47512a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:50.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f47453be740 1 -- 172.21.15.152:0/1541903267 shutdown_connections 2024-09-11T23:00:50.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f47453be740 1 -- 172.21.15.152:0/1541903267 wait complete. 2024-09-11T23:00:50.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f47453be740 20 client.17634 trim_cache size 0 max 16384 2024-09-11T23:00:50.898 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:00:50.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f4742fe5640 1 -- 172.21.15.152:0/28160495 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f47280a2cf0 msgr2=0x7f47280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:50.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f4742fe5640 1 --2- 172.21.15.152:0/28160495 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f47280a2cf0 0x7f47280a51b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f473400f7f0 tx=0x7f473401a000 comp rx=0 tx=0).stop 2024-09-11T23:00:50.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f4742fe5640 1 -- 172.21.15.152:0/28160495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f473c102570 msgr2=0x7f473c199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:50.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f4742fe5640 1 --2- 172.21.15.152:0/28160495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f473c102570 0x7f473c199930 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f472400b4a0 tx=0x7f472400b960 comp rx=0 tx=0).stop 2024-09-11T23:00:50.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f4742fe5640 1 -- 172.21.15.152:0/28160495 shutdown_connections 2024-09-11T23:00:50.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f4742fe5640 1 --2- 172.21.15.152:0/28160495 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f47280a2cf0 0x7f47280a51b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f4742fe5640 1 --2- 172.21.15.152:0/28160495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f473c102ef0 0x7f473c1a4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f4742fe5640 1 --2- 172.21.15.152:0/28160495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f473c102570 0x7f473c199930 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f4742fe5640 1 --2- 172.21.15.152:0/28160495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f473c100e20 0x7f473c1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:50.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.886+0000 7f4742fe5640 1 -- 172.21.15.152:0/28160495 >> 172.21.15.152:0/28160495 conn(0x7f473c0fc7d0 msgr2=0x7f473c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:50.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.887+0000 7f4742fe5640 1 -- 172.21.15.152:0/28160495 shutdown_connections 2024-09-11T23:00:50.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:50.887+0000 7f4742fe5640 1 -- 172.21.15.152:0/28160495 wait complete. 2024-09-11T23:00:51.662 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:00:51.909 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:52.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.454+0000 7fc8ec5ce640 1 -- 172.21.15.152:0/738069203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8e4100480 msgr2=0x7fc8e4100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:52.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.454+0000 7fc8ec5ce640 1 --2- 172.21.15.152:0/738069203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8e4100480 0x7fc8e4100860 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009900 tx=0x7fc8e002a410 comp rx=0 tx=0).stop 2024-09-11T23:00:52.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.454+0000 7fc8ec5ce640 1 -- 172.21.15.152:0/738069203 shutdown_connections 2024-09-11T23:00:52.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.454+0000 7fc8ec5ce640 1 --2- 172.21.15.152:0/738069203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e41055b0 0x7fc8e410d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.454+0000 7fc8ec5ce640 1 --2- 172.21.15.152:0/738069203 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8e4100e30 0x7fc8e4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.454+0000 7fc8ec5ce640 1 --2- 172.21.15.152:0/738069203 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8e4100480 0x7fc8e4100860 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.454+0000 7fc8ec5ce640 1 -- 172.21.15.152:0/738069203 >> 172.21.15.152:0/738069203 conn(0x7fc8e4077210 msgr2=0x7fc8e40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:52.472 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.454+0000 7fc8ec5ce640 1 -- 172.21.15.152:0/738069203 shutdown_connections 2024-09-11T23:00:52.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.454+0000 7fc8ec5ce640 1 -- 172.21.15.152:0/738069203 wait complete. 2024-09-11T23:00:52.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.455+0000 7fc8ec5ce640 1 Processor -- start 2024-09-11T23:00:52.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.455+0000 7fc8ec5ce640 1 -- start start 2024-09-11T23:00:52.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.455+0000 7fc8ec5ce640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e4100480 0x7fc8e41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.455+0000 7fc8ec5ce640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8e4100e30 0x7fc8e4198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.455+0000 7fc8ec5ce640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8e41055b0 0x7fc8e419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.455+0000 7fc8ec5ce640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e410fdb0 con 0x7fc8e41055b0 2024-09-11T23:00:52.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.455+0000 7fc8ec5ce640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e410fc30 con 0x7fc8e4100e30 2024-09-11T23:00:52.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.455+0000 7fc8ec5ce640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e410ff30 con 0x7fc8e4100480 2024-09-11T23:00:52.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.455+0000 7fc8eab6b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8e41055b0 0x7fc8e419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.455+0000 7fc8ea36a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e4100480 0x7fc8e41984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.455+0000 7fc8eab6b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8e41055b0 0x7fc8e419cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33408/0 (socket says 172.21.15.152:33408) 2024-09-11T23:00:52.473 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.455+0000 7fc8eab6b640 1 -- 172.21.15.152:0/3582497554 learned_addr learned my addr 172.21.15.152:0/3582497554 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:52.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.455+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/3582497554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8e4100e30 0x7fc8e4198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.455+0000 7fc8eab6b640 1 -- 172.21.15.152:0/3582497554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e4100480 msgr2=0x7fc8e41984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:52.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.455+0000 7fc8eab6b640 1 --2- 172.21.15.152:0/3582497554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e4100480 0x7fc8e41984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.456+0000 7fc8eab6b640 1 -- 172.21.15.152:0/3582497554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8e4100e30 msgr2=0x7fc8e4198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:52.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.456+0000 7fc8eab6b640 1 --2- 172.21.15.152:0/3582497554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8e4100e30 0x7fc8e4198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.456+0000 7fc8eab6b640 1 -- 172.21.15.152:0/3582497554 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e419d4a0 con 0x7fc8e41055b0 2024-09-11T23:00:52.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.456+0000 7fc8eab6b640 1 --2- 172.21.15.152:0/3582497554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8e41055b0 0x7fc8e419cdc0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00dd70 tx=0x7fc8dc00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:52.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.456+0000 7fc8d77fe640 1 -- 172.21.15.152:0/3582497554 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc004d20 con 0x7fc8e41055b0 2024-09-11T23:00:52.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.456+0000 7fc8d77fe640 1 -- 172.21.15.152:0/3582497554 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8dc005500 con 0x7fc8e41055b0 2024-09-11T23:00:52.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.456+0000 7fc8ec5ce640 1 -- 172.21.15.152:0/3582497554 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e419d790 con 0x7fc8e41055b0 2024-09-11T23:00:52.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.456+0000 7fc8d77fe640 1 -- 172.21.15.152:0/3582497554 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc00ea80 con 0x7fc8e41055b0 2024-09-11T23:00:52.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.456+0000 7fc8ec5ce640 1 -- 172.21.15.152:0/3582497554 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e41a5070 con 0x7fc8e41055b0 2024-09-11T23:00:52.474 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.458+0000 7fc8d77fe640 1 -- 172.21.15.152:0/3582497554 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc8dc0056b0 con 0x7fc8e41055b0 2024-09-11T23:00:52.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.458+0000 7fc8ee9a7740 1 Processor -- start 2024-09-11T23:00:52.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8ee9a7740 1 -- start start 2024-09-11T23:00:52.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8ee9a7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47ea99b40 0x55b47ea98090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8ee9a7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea985d0 0x55b47ea98a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8ee9a7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47eaa3370 0x55b47eaa5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8ee9a7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55b47e9fe2f0 con 0x55b47ea99b40 2024-09-11T23:00:52.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8ee9a7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b47e9a5cf0 con 0x55b47ea985d0 2024-09-11T23:00:52.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8ee9a7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b47e9fa2b0 con 0x55b47eaa3370 2024-09-11T23:00:52.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8e9b69640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47ea99b40 0x55b47ea98090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8e9b69640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47ea99b40 0x55b47ea98090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33416/0 (socket says 172.21.15.152:33416) 2024-09-11T23:00:52.475 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8e9b69640 1 -- 172.21.15.152:0/609192481 learned_addr learned my addr 172.21.15.152:0/609192481 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:52.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8d77fe640 1 --2- 172.21.15.152:0/3582497554 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8d00a2cf0 0x7fc8d00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8ea36a640 1 --2- 172.21.15.152:0/609192481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea985d0 0x55b47ea98a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/609192481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47eaa3370 0x55b47eaa5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8eab6b640 1 --2- 172.21.15.152:0/3582497554 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8d00a2cf0 0x7fc8d00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8e9b69640 1 -- 172.21.15.152:0/609192481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47eaa3370 msgr2=0x55b47eaa5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:52.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/609192481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47eaa3370 0x55b47eaa5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8e9b69640 1 -- 172.21.15.152:0/609192481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea985d0 msgr2=0x55b47ea98a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:52.476 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/609192481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea985d0 0x55b47ea98a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8e9b69640 1 -- 172.21.15.152:0/609192481 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b47e98a3a0 con 0x55b47ea99b40 2024-09-11T23:00:52.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.459+0000 7fc8d77fe640 1 -- 172.21.15.152:0/3582497554 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7fc8dc0cf870 con 0x7fc8e41055b0 2024-09-11T23:00:52.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.460+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/609192481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47ea99b40 0x55b47ea98090 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8006aa0 tx=0x7fc8d80095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe5e73a1dcb5d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:52.477 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.460+0000 7fc8d4ff9640 1 -- 172.21.15.152:0/609192481 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8017610 con 0x55b47ea99b40 2024-09-11T23:00:52.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.460+0000 7fc8eab6b640 1 --2- 172.21.15.152:0/3582497554 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8d00a2cf0 0x7fc8d00a51b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc098890 tx=0x7fc8dc0a0040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:52.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.460+0000 7fc8d4ff9640 1 -- 172.21.15.152:0/609192481 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d801c070 con 0x55b47ea99b40 2024-09-11T23:00:52.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.460+0000 7fc8d4ff9640 1 -- 172.21.15.152:0/609192481 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8015940 con 0x55b47ea99b40 2024-09-11T23:00:52.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.460+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/609192481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47ea99b40 msgr2=0x55b47ea98090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:52.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.460+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/609192481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47ea99b40 0x55b47ea98090 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8006aa0 tx=0x7fc8d80095d0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/609192481 shutdown_connections 2024-09-11T23:00:52.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/609192481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47eaa3370 0x55b47eaa5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/609192481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea985d0 0x55b47ea98a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.478 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/609192481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47ea99b40 0x55b47ea98090 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/609192481 >> 172.21.15.152:0/609192481 conn(0x55b47ea61310 msgr2=0x55b47ea94c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:52.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/609192481 shutdown_connections 2024-09-11T23:00:52.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/609192481 wait complete. 2024-09-11T23:00:52.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8ee9a7740 1 -- start start 2024-09-11T23:00:52.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8ee9a7740 1 Processor -- start 2024-09-11T23:00:52.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8ee9a7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea985d0 0x55b47eaa2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8ee9a7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47ea99b40 0x55b47ea9ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8ee9a7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47eaa3370 0x55b47eaa04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8ee9a7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55b47eaa6010 con 0x55b47eaa3370 2024-09-11T23:00:52.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8ee9a7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b47e9b9060 con 0x55b47ea985d0 2024-09-11T23:00:52.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8ee9a7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b47eaa6190 con 0x55b47ea99b40 2024-09-11T23:00:52.479 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8ee9a7740 10 client.0 _ll_register_callbacks cb 0x55b47e70c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:52.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8e9b69640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47eaa3370 0x55b47eaa04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8e9b69640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47eaa3370 0x55b47eaa04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33426/0 (socket says 172.21.15.152:33426) 2024-09-11T23:00:52.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.461+0000 7fc8e9b69640 1 -- 172.21.15.152:0/2901604166 learned_addr learned my addr 172.21.15.152:0/2901604166 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:52.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.462+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/2901604166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea985d0 0x55b47eaa2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.462+0000 7fc8ea36a640 1 --2- 172.21.15.152:0/2901604166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47ea99b40 0x55b47ea9ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.462+0000 7fc8e9b69640 1 -- 172.21.15.152:0/2901604166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47ea99b40 msgr2=0x55b47ea9ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:52.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.462+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/2901604166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47ea99b40 0x55b47ea9ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.462+0000 7fc8e9b69640 1 -- 172.21.15.152:0/2901604166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea985d0 msgr2=0x55b47eaa2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:52.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.462+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/2901604166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea985d0 0x55b47eaa2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.462+0000 7fc8e9b69640 1 -- 172.21.15.152:0/2901604166 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b47e76f3c0 con 0x55b47eaa3370 2024-09-11T23:00:52.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.462+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/2901604166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47eaa3370 0x55b47eaa04e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8009cf0 tx=0x7fc8d8015520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2bb52320555174 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:52.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.462+0000 7fc8cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:52.480 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.462+0000 7fc8cdffb640 1 -- 172.21.15.152:0/2901604166 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8015d20 con 0x55b47eaa3370 2024-09-11T23:00:52.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.462+0000 7fc8cdffb640 1 -- 172.21.15.152:0/2901604166 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7fc8d8017410 con 0x55b47eaa3370 2024-09-11T23:00:52.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.462+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/2901604166 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d801c070 con 0x55b47eaa3370 2024-09-11T23:00:52.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.462+0000 7fc8cdffb640 1 -- 172.21.15.152:0/2901604166 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc8d8035050 con 0x55b47eaa3370 2024-09-11T23:00:52.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.462+0000 7fc8ee9a7740 10 client.17646 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:52.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.462+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/2901604166 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b47eab1a20 con 0x55b47eaa3370 2024-09-11T23:00:52.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.463+0000 7fc8cdffb640 1 -- 172.21.15.152:0/2901604166 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d801c070 con 0x55b47eaa3370 2024-09-11T23:00:52.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.463+0000 7fc8cdffb640 1 -- 172.21.15.152:0/2901604166 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc8d8021660 con 0x55b47eaa3370 2024-09-11T23:00:52.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.463+0000 7fc8ee9a7740 10 client.17646 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:52.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.463+0000 7fc8ee9a7740 10 client.17646 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:00:52.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.463+0000 7fc8ee9a7740 20 client.17646 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:52.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.463+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/2901604166 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55b47eab2d70 0x55b47eab5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.463+0000 7fc8ee9a7740 4 client.17646 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:00:52.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.463+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/2901604166 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b47ea0b860 con 0x55b47eab2d70 2024-09-11T23:00:52.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.463+0000 7fc8ea36a640 1 --2- 172.21.15.152:0/2901604166 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55b47eab2d70 0x55b47eab5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.464+0000 7fc8cdffb640 10 client.17646 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T23:00:52.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.464+0000 7fc8ea36a640 1 --2- 172.21.15.152:0/2901604166 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55b47eab2d70 0x55b47eab5160 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=831fcb362c2fb1a7 server_cookie=c23dff71dfa6430a in_seq=0 out_seq=0 2024-09-11T23:00:52.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.464+0000 7fc8cdffb640 1 -- 172.21.15.152:0/2901604166 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b47ea0b860 con 0x55b47eab2d70 2024-09-11T23:00:52.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.464+0000 7fc8cdffb640 10 client.17646 handle_command_reply: tid=0 2024-09-11T23:00:52.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.464+0000 7fc8ee9a7740 1 client.17646 shutdown 2024-09-11T23:00:52.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.464+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/2901604166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47eaa3370 msgr2=0x55b47eaa04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:52.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.464+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/2901604166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47eaa3370 0x55b47eaa04e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8009cf0 tx=0x7fc8d8015520 comp rx=0 tx=0).stop 2024-09-11T23:00:52.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.465+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/2901604166 shutdown_connections 2024-09-11T23:00:52.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.465+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/2901604166 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47eaa3370 0x55b47eaa04e0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.465+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/2901604166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47ea99b40 0x55b47ea9ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.485 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.465+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/2901604166 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55b47eab2d70 0x55b47eab5160 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.465+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/2901604166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea985d0 0x55b47eaa2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.465+0000 7fc8cdffb640 0 client.17646 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T23:00:52.486 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.465+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/2901604166 >> 172.21.15.152:0/2901604166 conn(0x55b47ea61310 msgr2=0x55b47ea9a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:52.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.465+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/2901604166 shutdown_connections 2024-09-11T23:00:52.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.465+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/2901604166 wait complete. 2024-09-11T23:00:52.487 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.465+0000 7fc8ee9a7740 20 client.17646 trim_cache size 0 max 16384 2024-09-11T23:00:52.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.477+0000 7fc8ee9a7740 1 Processor -- start 2024-09-11T23:00:52.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.477+0000 7fc8ee9a7740 1 -- start start 2024-09-11T23:00:52.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8ee9a7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47ea985d0 0x55b47eaaf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8ee9a7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea99b40 0x55b47ea9b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8ee9a7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47eaa3370 0x55b47ea9b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8ee9a7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e41ad850 con 0x55b47ea985d0 2024-09-11T23:00:52.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8ee9a7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e41af390 con 0x55b47ea99b40 2024-09-11T23:00:52.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8ee9a7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e41adf20 con 0x55b47eaa3370 2024-09-11T23:00:52.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8e9b69640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47ea985d0 0x55b47eaaf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8e9b69640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47ea985d0 0x55b47eaaf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33430/0 (socket says 172.21.15.152:33430) 2024-09-11T23:00:52.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8e9b69640 1 -- 172.21.15.152:0/3528735523 learned_addr learned my addr 172.21.15.152:0/3528735523 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:52.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8ea36a640 1 --2- 172.21.15.152:0/3528735523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea99b40 0x55b47ea9b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/3528735523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47eaa3370 0x55b47ea9b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8e9b69640 1 -- 172.21.15.152:0/3528735523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47eaa3370 msgr2=0x55b47ea9b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:52.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/3528735523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47eaa3370 0x55b47ea9b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8e9b69640 1 -- 172.21.15.152:0/3528735523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea99b40 msgr2=0x55b47ea9b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:52.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/3528735523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea99b40 0x55b47ea9b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8e9b69640 1 -- 172.21.15.152:0/3528735523 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b47eaa05b0 con 0x55b47ea985d0 2024-09-11T23:00:52.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/3528735523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47ea985d0 0x55b47eaaf2b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8021a30 tx=0x7fc8d80093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2fab198a57ed9f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:52.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8cffff640 1 -- 172.21.15.152:0/3528735523 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8015620 con 0x55b47ea985d0 2024-09-11T23:00:52.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8cffff640 1 -- 172.21.15.152:0/3528735523 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d801c040 con 0x55b47ea985d0 2024-09-11T23:00:52.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.478+0000 7fc8cffff640 1 -- 172.21.15.152:0/3528735523 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8017610 con 0x55b47ea985d0 2024-09-11T23:00:52.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.479+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/3528735523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47ea985d0 msgr2=0x55b47eaaf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:52.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.479+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/3528735523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47ea985d0 0x55b47eaaf2b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8021a30 tx=0x7fc8d80093a0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.479+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/3528735523 shutdown_connections 2024-09-11T23:00:52.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.479+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/3528735523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47eaa3370 0x55b47ea9b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.479+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/3528735523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea99b40 0x55b47ea9b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.479+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/3528735523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47ea985d0 0x55b47eaaf2b0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.479+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/3528735523 >> 172.21.15.152:0/3528735523 conn(0x55b47ea61310 msgr2=0x55b47ea9f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:52.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.479+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/3528735523 shutdown_connections 2024-09-11T23:00:52.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.479+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/3528735523 wait complete. 2024-09-11T23:00:52.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.479+0000 7fc8ee9a7740 1 -- start start 2024-09-11T23:00:52.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.479+0000 7fc8ee9a7740 1 Processor -- start 2024-09-11T23:00:52.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8ee9a7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea985d0 0x55b47eaabb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8ee9a7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47ea99b40 0x55b47eaa6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8ee9a7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47eaa3370 0x55b47eaa7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8ee9a7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e41aecc0 con 0x55b47eaa3370 2024-09-11T23:00:52.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8ee9a7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e41ae5f0 con 0x55b47ea985d0 2024-09-11T23:00:52.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8ee9a7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e41ab850 con 0x55b47ea99b40 2024-09-11T23:00:52.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8ee9a7740 10 client.0 _ll_register_callbacks cb 0x7fc8e0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:52.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8e9b69640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47eaa3370 0x55b47eaa7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8e9b69640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47eaa3370 0x55b47eaa7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33440/0 (socket says 172.21.15.152:33440) 2024-09-11T23:00:52.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8e9b69640 1 -- 172.21.15.152:0/1568114510 learned_addr learned my addr 172.21.15.152:0/1568114510 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:52.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8ea36a640 1 --2- 172.21.15.152:0/1568114510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47ea99b40 0x55b47eaa6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/1568114510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea985d0 0x55b47eaabb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8e9b69640 1 -- 172.21.15.152:0/1568114510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47ea99b40 msgr2=0x55b47eaa6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:52.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/1568114510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47ea99b40 0x55b47eaa6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8e9b69640 1 -- 172.21.15.152:0/1568114510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea985d0 msgr2=0x55b47eaabb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:52.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/1568114510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea985d0 0x55b47eaabb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8e9b69640 1 -- 172.21.15.152:0/1568114510 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b47eaa7c50 con 0x55b47eaa3370 2024-09-11T23:00:52.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8e9b69640 1 --2- 172.21.15.152:0/1568114510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47eaa3370 0x55b47eaa7400 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8011010 tx=0x7fc8d8023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b583deb3f3174369 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:52.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.480+0000 7fc8cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:52.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.481+0000 7fc8cdffb640 1 -- 172.21.15.152:0/1568114510 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8040020 con 0x55b47eaa3370 2024-09-11T23:00:52.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.481+0000 7fc8cdffb640 1 -- 172.21.15.152:0/1568114510 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7fc8d8021400 con 0x55b47eaa3370 2024-09-11T23:00:52.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.481+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/1568114510 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d801c040 con 0x55b47eaa3370 2024-09-11T23:00:52.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.481+0000 7fc8cdffb640 1 -- 172.21.15.152:0/1568114510 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc8d8015660 con 0x55b47eaa3370 2024-09-11T23:00:52.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.481+0000 7fc8ee9a7740 10 client.17652 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:52.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.481+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/1568114510 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b47eb1d7f0 con 0x55b47eaa3370 2024-09-11T23:00:52.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.481+0000 7fc8cdffb640 1 -- 172.21.15.152:0/1568114510 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d801c040 con 0x55b47eaa3370 2024-09-11T23:00:52.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.481+0000 7fc8cdffb640 1 -- 172.21.15.152:0/1568114510 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc8d8006360 con 0x55b47eaa3370 2024-09-11T23:00:52.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.481+0000 7fc8ee9a7740 10 client.17652 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:52.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.481+0000 7fc8ee9a7740 10 client.17652 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:00:52.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.481+0000 7fc8ee9a7740 20 client.17652 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:52.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.481+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/1568114510 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55b47eb1dda0 0x55b47eb20190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:52.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.481+0000 7fc8ee9a7740 4 client.17652 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:00:52.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.481+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/1568114510 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b47ea05d70 con 0x55b47eb1dda0 2024-09-11T23:00:52.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8ea36a640 1 --2- 172.21.15.152:0/1568114510 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55b47eb1dda0 0x55b47eb20190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:52.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8cdffb640 10 client.17652 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T23:00:52.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8ea36a640 1 --2- 172.21.15.152:0/1568114510 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55b47eb1dda0 0x55b47eb20190 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f948b9d646224d58 server_cookie=7c445196e181a25c in_seq=0 out_seq=0 2024-09-11T23:00:52.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8cdffb640 1 -- 172.21.15.152:0/1568114510 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b47ea05d70 con 0x55b47eb1dda0 2024-09-11T23:00:52.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8cdffb640 10 client.17652 handle_command_reply: tid=0 2024-09-11T23:00:52.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8ee9a7740 1 client.17652 shutdown 2024-09-11T23:00:52.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/1568114510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47eaa3370 msgr2=0x55b47eaa7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:52.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/1568114510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47eaa3370 0x55b47eaa7400 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8011010 tx=0x7fc8d8023040 comp rx=0 tx=0).stop 2024-09-11T23:00:52.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/1568114510 shutdown_connections 2024-09-11T23:00:52.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/1568114510 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b47eaa3370 0x55b47eaa7400 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.505 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/1568114510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b47ea99b40 0x55b47eaa6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/1568114510 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55b47eb1dda0 0x55b47eb20190 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8ee9a7740 1 --2- 172.21.15.152:0/1568114510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b47ea985d0 0x55b47eaabb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/1568114510 >> 172.21.15.152:0/1568114510 conn(0x55b47ea61310 msgr2=0x55b47ea934e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:52.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8cdffb640 0 client.17652 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T23:00:52.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/1568114510 shutdown_connections 2024-09-11T23:00:52.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.482+0000 7fc8ee9a7740 1 -- 172.21.15.152:0/1568114510 wait complete. 2024-09-11T23:00:52.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.483+0000 7fc8ee9a7740 20 client.17652 trim_cache size 0 max 16384 2024-09-11T23:00:52.506 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:00:52.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.483+0000 7fc8ec5ce640 1 -- 172.21.15.152:0/3582497554 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8d00a2cf0 msgr2=0x7fc8d00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:52.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.483+0000 7fc8ec5ce640 1 --2- 172.21.15.152:0/3582497554 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8d00a2cf0 0x7fc8d00a51b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc098890 tx=0x7fc8dc0a0040 comp rx=0 tx=0).stop 2024-09-11T23:00:52.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.483+0000 7fc8ec5ce640 1 -- 172.21.15.152:0/3582497554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8e41055b0 msgr2=0x7fc8e419cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:52.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.483+0000 7fc8ec5ce640 1 --2- 172.21.15.152:0/3582497554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8e41055b0 0x7fc8e419cdc0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00dd70 tx=0x7fc8dc00b730 comp rx=0 tx=0).stop 2024-09-11T23:00:52.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.483+0000 7fc8ec5ce640 1 -- 172.21.15.152:0/3582497554 shutdown_connections 2024-09-11T23:00:52.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.483+0000 7fc8ec5ce640 1 --2- 172.21.15.152:0/3582497554 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8d00a2cf0 0x7fc8d00a51b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.483+0000 7fc8ec5ce640 1 --2- 172.21.15.152:0/3582497554 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc8e41055b0 0x7fc8e419cdc0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.483+0000 7fc8ec5ce640 1 --2- 172.21.15.152:0/3582497554 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8e4100e30 0x7fc8e4198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.483+0000 7fc8ec5ce640 1 --2- 172.21.15.152:0/3582497554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8e4100480 0x7fc8e41984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:52.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.483+0000 7fc8ec5ce640 1 -- 172.21.15.152:0/3582497554 >> 172.21.15.152:0/3582497554 conn(0x7fc8e4077210 msgr2=0x7fc8e40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:52.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.483+0000 7fc8ec5ce640 1 -- 172.21.15.152:0/3582497554 shutdown_connections 2024-09-11T23:00:52.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:52.483+0000 7fc8ec5ce640 1 -- 172.21.15.152:0/3582497554 wait complete. 2024-09-11T23:00:52.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:52 smithi152 ceph-mon[31181]: pgmap v244: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:00:52.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:52 smithi191 ceph-mon[35211]: pgmap v244: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:00:52.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:52 smithi204 ceph-mon[35454]: pgmap v244: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:00:53.189 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:00:53.445 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:53.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.930+0000 7fa260640640 1 -- 172.21.15.152:0/1124287394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa258102ef0 msgr2=0x7fa25810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:53.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.930+0000 7fa260640640 1 --2- 172.21.15.152:0/1124287394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa258102ef0 0x7fa25810da40 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fa250009900 tx=0x7fa25002a480 comp rx=0 tx=0).stop 2024-09-11T23:00:53.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.930+0000 7fa260640640 1 -- 172.21.15.152:0/1124287394 shutdown_connections 2024-09-11T23:00:53.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.930+0000 7fa260640640 1 --2- 172.21.15.152:0/1124287394 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa258102ef0 0x7fa25810da40 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.930+0000 7fa260640640 1 --2- 172.21.15.152:0/1124287394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa258102570 0x7fa2581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.930+0000 7fa260640640 1 --2- 172.21.15.152:0/1124287394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa258100e20 0x7fa258101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.930+0000 7fa260640640 1 -- 172.21.15.152:0/1124287394 >> 172.21.15.152:0/1124287394 conn(0x7fa2580fc7d0 msgr2=0x7fa2580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:53.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.930+0000 7fa260640640 1 -- 172.21.15.152:0/1124287394 shutdown_connections 2024-09-11T23:00:53.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.930+0000 7fa260640640 1 -- 172.21.15.152:0/1124287394 wait complete. 2024-09-11T23:00:53.950 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.930+0000 7fa260640640 1 Processor -- start 2024-09-11T23:00:53.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.930+0000 7fa260640640 1 -- start start 2024-09-11T23:00:53.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa260640640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa258100e20 0x7fa2580fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa260640640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa258102570 0x7fa2580fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa260640640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa258102ef0 0x7fa2580fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa260640640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa258110fa0 con 0x7fa258100e20 2024-09-11T23:00:53.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa260640640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa258110e20 con 0x7fa258102ef0 2024-09-11T23:00:53.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa260640640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa258111120 con 0x7fa258102570 2024-09-11T23:00:53.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa25e3dc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa258100e20 0x7fa2580fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:53.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa25e3dc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa258100e20 0x7fa2580fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33466/0 (socket says 172.21.15.152:33466) 2024-09-11T23:00:53.951 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa25e3dc640 1 -- 172.21.15.152:0/1632764202 learned_addr learned my addr 172.21.15.152:0/1632764202 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:53.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa25ebdd640 1 --2- 172.21.15.152:0/1632764202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa258102ef0 0x7fa2580fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:53.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa25dbdb640 1 --2- 172.21.15.152:0/1632764202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa258102570 0x7fa2580fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:53.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa25e3dc640 1 -- 172.21.15.152:0/1632764202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa258102570 msgr2=0x7fa2580fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:53.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa25e3dc640 1 --2- 172.21.15.152:0/1632764202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa258102570 0x7fa2580fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa25e3dc640 1 -- 172.21.15.152:0/1632764202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa258102ef0 msgr2=0x7fa2580fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:53.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa25e3dc640 1 --2- 172.21.15.152:0/1632764202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa258102ef0 0x7fa2580fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa25e3dc640 1 -- 172.21.15.152:0/1632764202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2581a6990 con 0x7fa258100e20 2024-09-11T23:00:53.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa25ebdd640 1 --2- 172.21.15.152:0/1632764202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa258102ef0 0x7fa2580fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:53.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa25e3dc640 1 --2- 172.21.15.152:0/1632764202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa258100e20 0x7fa2580fcc80 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fa24800eab0 tx=0x7fa24800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:53.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.931+0000 7fa2577fe640 1 -- 172.21.15.152:0/1632764202 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24800ce40 con 0x7fa258100e20 2024-09-11T23:00:53.952 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.932+0000 7fa260640640 1 -- 172.21.15.152:0/1632764202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2581a6c80 con 0x7fa258100e20 2024-09-11T23:00:53.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.932+0000 7fa2577fe640 1 -- 172.21.15.152:0/1632764202 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa248011950 con 0x7fa258100e20 2024-09-11T23:00:53.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.932+0000 7fa2577fe640 1 -- 172.21.15.152:0/1632764202 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248010ae0 con 0x7fa258100e20 2024-09-11T23:00:53.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.932+0000 7fa260640640 1 -- 172.21.15.152:0/1632764202 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2581a71c0 con 0x7fa258100e20 2024-09-11T23:00:53.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.934+0000 7fa2577fe640 1 -- 172.21.15.152:0/1632764202 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa248011b00 con 0x7fa258100e20 2024-09-11T23:00:53.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.934+0000 7fa262a19740 1 Processor -- start 2024-09-11T23:00:53.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.934+0000 7fa262a19740 1 -- start start 2024-09-11T23:00:53.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa262a19740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db704090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa262a19740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db7045d0 0x5565db704a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa262a19740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db70f370 0x5565db711760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa262a19740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5565db66a2f0 con 0x5565db705b40 2024-09-11T23:00:53.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa262a19740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5565db611cf0 con 0x5565db70f370 2024-09-11T23:00:53.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa262a19740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5565db6662b0 con 0x5565db7045d0 2024-09-11T23:00:53.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa25ebdd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db704090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:53.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa2577fe640 1 --2- 172.21.15.152:0/1632764202 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa2400a2cf0 0x7fa2400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa25ebdd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db704090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33482/0 (socket says 172.21.15.152:33482) 2024-09-11T23:00:53.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa25ebdd640 1 -- 172.21.15.152:0/1923539197 learned_addr learned my addr 172.21.15.152:0/1923539197 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:53.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa25e3dc640 1 --2- 172.21.15.152:0/1923539197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db7045d0 0x5565db704a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:53.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa25dbdb640 1 --2- 172.21.15.152:0/1923539197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db70f370 0x5565db711760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:53.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa25ebdd640 1 --2- 172.21.15.152:0/1632764202 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa2400a2cf0 0x7fa2400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:53.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa2577fe640 1 -- 172.21.15.152:0/1632764202 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7fa2480cfc10 con 0x7fa258100e20 2024-09-11T23:00:53.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa25ebdd640 1 -- 172.21.15.152:0/1923539197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db7045d0 msgr2=0x5565db704a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:53.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa25ebdd640 1 --2- 172.21.15.152:0/1923539197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db7045d0 0x5565db704a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa25ebdd640 1 -- 172.21.15.152:0/1923539197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db70f370 msgr2=0x5565db711760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:53.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa25ebdd640 1 --2- 172.21.15.152:0/1923539197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db70f370 0x5565db711760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa25ebdd640 1 -- 172.21.15.152:0/1923539197 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565db5f63a0 con 0x5565db705b40 2024-09-11T23:00:53.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa25e3dc640 1 --2- 172.21.15.152:0/1923539197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db7045d0 0x5565db704a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:53.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.935+0000 7fa25ebdd640 1 --2- 172.21.15.152:0/1923539197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db704090 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa250005940 tx=0x7fa2500051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f73bc5930d754d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:53.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.936+0000 7fa254ff9640 1 -- 172.21.15.152:0/1923539197 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25004d070 con 0x5565db705b40 2024-09-11T23:00:53.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.936+0000 7fa254ff9640 1 -- 172.21.15.152:0/1923539197 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa250004c90 con 0x5565db705b40 2024-09-11T23:00:53.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.936+0000 7fa254ff9640 1 -- 172.21.15.152:0/1923539197 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250035040 con 0x5565db705b40 2024-09-11T23:00:53.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.936+0000 7fa262a19740 1 -- 172.21.15.152:0/1923539197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 msgr2=0x5565db704090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:53.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.936+0000 7fa25ebdd640 1 --2- 172.21.15.152:0/1632764202 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa2400a2cf0 0x7fa2400a51b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa25004a000 tx=0x7fa250048000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:53.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.936+0000 7fa262a19740 1 --2- 172.21.15.152:0/1923539197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db704090 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa250005940 tx=0x7fa2500051f0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.936+0000 7fa262a19740 1 -- 172.21.15.152:0/1923539197 shutdown_connections 2024-09-11T23:00:53.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.936+0000 7fa262a19740 1 --2- 172.21.15.152:0/1923539197 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db70f370 0x5565db711760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.936+0000 7fa262a19740 1 --2- 172.21.15.152:0/1923539197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db7045d0 0x5565db704a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.936+0000 7fa262a19740 1 --2- 172.21.15.152:0/1923539197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db704090 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.936+0000 7fa262a19740 1 -- 172.21.15.152:0/1923539197 >> 172.21.15.152:0/1923539197 conn(0x5565db6cd310 msgr2=0x5565db700c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:53.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.936+0000 7fa262a19740 1 -- 172.21.15.152:0/1923539197 shutdown_connections 2024-09-11T23:00:53.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.936+0000 7fa262a19740 1 -- 172.21.15.152:0/1923539197 wait complete. 2024-09-11T23:00:53.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa262a19740 1 -- start start 2024-09-11T23:00:53.956 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa262a19740 1 Processor -- start 2024-09-11T23:00:53.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa262a19740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db7045d0 0x5565db70ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa262a19740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db70bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa262a19740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db70f370 0x5565db70c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa262a19740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5565db712070 con 0x5565db705b40 2024-09-11T23:00:53.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa262a19740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5565db625060 con 0x5565db70f370 2024-09-11T23:00:53.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa262a19740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5565db7121f0 con 0x5565db7045d0 2024-09-11T23:00:53.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa262a19740 10 client.0 _ll_register_callbacks cb 0x5565db3788c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:53.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa25dbdb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db70bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:53.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa25dbdb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db70bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33492/0 (socket says 172.21.15.152:33492) 2024-09-11T23:00:53.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa25dbdb640 1 -- 172.21.15.152:0/890611382 learned_addr learned my addr 172.21.15.152:0/890611382 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:53.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa25e3dc640 1 --2- 172.21.15.152:0/890611382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db7045d0 0x5565db70ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:53.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa25dbdb640 1 -- 172.21.15.152:0/890611382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db7045d0 msgr2=0x5565db70ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:53.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa25dbdb640 1 --2- 172.21.15.152:0/890611382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db7045d0 0x5565db70ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa25dbdb640 1 -- 172.21.15.152:0/890611382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db70f370 msgr2=0x5565db70c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:53.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa25dbdb640 1 --2- 172.21.15.152:0/890611382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db70f370 0x5565db70c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.937+0000 7fa25dbdb640 1 -- 172.21.15.152:0/890611382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565db3db3c0 con 0x5565db705b40 2024-09-11T23:00:53.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.938+0000 7fa25dbdb640 1 --2- 172.21.15.152:0/890611382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db70bf70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa24c010500 tx=0x7fa24c0109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd697a9bba3f1b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:53.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.938+0000 7fa23dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:53.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.938+0000 7fa23dffb640 1 -- 172.21.15.152:0/890611382 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c01a020 con 0x5565db705b40 2024-09-11T23:00:53.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.938+0000 7fa23dffb640 1 -- 172.21.15.152:0/890611382 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7fa24c016750 con 0x5565db705b40 2024-09-11T23:00:53.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.938+0000 7fa262a19740 1 -- 172.21.15.152:0/890611382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa250004c90 con 0x5565db705b40 2024-09-11T23:00:53.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.938+0000 7fa23dffb640 1 -- 172.21.15.152:0/890611382 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa24c025050 con 0x5565db705b40 2024-09-11T23:00:53.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.938+0000 7fa262a19740 10 client.17664 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:53.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.938+0000 7fa262a19740 1 -- 172.21.15.152:0/890611382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565db71db40 con 0x5565db705b40 2024-09-11T23:00:53.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.938+0000 7fa23dffb640 1 -- 172.21.15.152:0/890611382 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa250004c90 con 0x5565db705b40 2024-09-11T23:00:53.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.938+0000 7fa23dffb640 1 -- 172.21.15.152:0/890611382 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa24c014bc0 con 0x5565db705b40 2024-09-11T23:00:53.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.938+0000 7fa262a19740 10 client.17664 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:53.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.938+0000 7fa262a19740 10 client.17664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:00:53.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.938+0000 7fa262a19740 20 client.17664 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:53.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.938+0000 7fa262a19740 1 --2- 172.21.15.152:0/890611382 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5565db71f0a0 0x5565db721500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.938+0000 7fa262a19740 4 client.17664 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:00:53.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.938+0000 7fa262a19740 1 -- 172.21.15.152:0/890611382 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565db677860 con 0x5565db71f0a0 2024-09-11T23:00:53.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.939+0000 7fa25e3dc640 1 --2- 172.21.15.152:0/890611382 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5565db71f0a0 0x5565db721500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:53.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.939+0000 7fa23dffb640 10 client.17664 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:53.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.939+0000 7fa25e3dc640 1 --2- 172.21.15.152:0/890611382 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5565db71f0a0 0x5565db721500 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58d7c3b79786db45 server_cookie=b87c0e47cf2dc33d in_seq=0 out_seq=0 2024-09-11T23:00:53.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.940+0000 7fa23dffb640 1 -- 172.21.15.152:0/890611382 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565db677860 con 0x5565db71f0a0 2024-09-11T23:00:53.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.940+0000 7fa23dffb640 10 client.17664 handle_command_reply: tid=0 2024-09-11T23:00:53.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.940+0000 7fa262a19740 1 client.17664 shutdown 2024-09-11T23:00:53.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.940+0000 7fa262a19740 1 -- 172.21.15.152:0/890611382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 msgr2=0x5565db70bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:53.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.940+0000 7fa262a19740 1 --2- 172.21.15.152:0/890611382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db70bf70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa24c010500 tx=0x7fa24c0109c0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.940+0000 7fa262a19740 1 -- 172.21.15.152:0/890611382 shutdown_connections 2024-09-11T23:00:53.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.940+0000 7fa262a19740 1 --2- 172.21.15.152:0/890611382 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5565db71f0a0 0x5565db721500 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.940+0000 7fa262a19740 1 --2- 172.21.15.152:0/890611382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db70f370 0x5565db70c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.940+0000 7fa262a19740 1 --2- 172.21.15.152:0/890611382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db70bf70 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.940+0000 7fa262a19740 1 --2- 172.21.15.152:0/890611382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db7045d0 0x5565db70ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.940+0000 7fa23dffb640 0 client.17664 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:53.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.940+0000 7fa262a19740 1 -- 172.21.15.152:0/890611382 >> 172.21.15.152:0/890611382 conn(0x5565db6cd310 msgr2=0x5565db706930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:53.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.940+0000 7fa262a19740 1 -- 172.21.15.152:0/890611382 shutdown_connections 2024-09-11T23:00:53.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.940+0000 7fa262a19740 1 -- 172.21.15.152:0/890611382 wait complete. 2024-09-11T23:00:53.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.940+0000 7fa262a19740 20 client.17664 trim_cache size 0 max 16384 2024-09-11T23:00:53.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.953+0000 7fa262a19740 1 Processor -- start 2024-09-11T23:00:53.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.953+0000 7fa262a19740 1 -- start start 2024-09-11T23:00:53.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.953+0000 7fa262a19740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db7045d0 0x5565db6fbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.953+0000 7fa262a19740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db71b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.953+0000 7fa262a19740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db70f370 0x5565db726dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.953+0000 7fa262a19740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581afa70 con 0x5565db705b40 2024-09-11T23:00:53.966 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.953+0000 7fa262a19740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581b15b0 con 0x5565db70f370 2024-09-11T23:00:53.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.953+0000 7fa262a19740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581b0140 con 0x5565db7045d0 2024-09-11T23:00:53.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.953+0000 7fa25dbdb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db71b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:53.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.953+0000 7fa25dbdb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db71b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33500/0 (socket says 172.21.15.152:33500) 2024-09-11T23:00:53.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.953+0000 7fa25dbdb640 1 -- 172.21.15.152:0/1959780660 learned_addr learned my addr 172.21.15.152:0/1959780660 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:53.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.954+0000 7fa25ebdd640 1 --2- 172.21.15.152:0/1959780660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db70f370 0x5565db726dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:53.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.954+0000 7fa25dbdb640 1 -- 172.21.15.152:0/1959780660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db7045d0 msgr2=0x5565db6fbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:53.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.954+0000 7fa25dbdb640 1 --2- 172.21.15.152:0/1959780660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db7045d0 0x5565db6fbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.954+0000 7fa25dbdb640 1 -- 172.21.15.152:0/1959780660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db70f370 msgr2=0x5565db726dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:53.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.954+0000 7fa25dbdb640 1 --2- 172.21.15.152:0/1959780660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db70f370 0x5565db726dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.954+0000 7fa25dbdb640 1 -- 172.21.15.152:0/1959780660 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565db701960 con 0x5565db705b40 2024-09-11T23:00:53.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.954+0000 7fa25dbdb640 1 --2- 172.21.15.152:0/1959780660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db71b370 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa24c014f60 tx=0x7fa24c006a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43772a9dfb427779 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:53.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.954+0000 7fa23ffff640 1 -- 172.21.15.152:0/1959780660 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c02d020 con 0x5565db705b40 2024-09-11T23:00:53.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.954+0000 7fa23ffff640 1 -- 172.21.15.152:0/1959780660 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa24c00bd20 con 0x5565db705b40 2024-09-11T23:00:53.967 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.954+0000 7fa23ffff640 1 -- 172.21.15.152:0/1959780660 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c01a040 con 0x5565db705b40 2024-09-11T23:00:53.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 -- 172.21.15.152:0/1959780660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 msgr2=0x5565db71b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:53.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 --2- 172.21.15.152:0/1959780660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db71b370 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa24c014f60 tx=0x7fa24c006a30 comp rx=0 tx=0).stop 2024-09-11T23:00:53.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 -- 172.21.15.152:0/1959780660 shutdown_connections 2024-09-11T23:00:53.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 --2- 172.21.15.152:0/1959780660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db70f370 0x5565db726dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 --2- 172.21.15.152:0/1959780660 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db71b370 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 --2- 172.21.15.152:0/1959780660 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db7045d0 0x5565db6fbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 -- 172.21.15.152:0/1959780660 >> 172.21.15.152:0/1959780660 conn(0x5565db6cd310 msgr2=0x5565db7072c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:53.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 -- 172.21.15.152:0/1959780660 shutdown_connections 2024-09-11T23:00:53.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 -- 172.21.15.152:0/1959780660 wait complete. 2024-09-11T23:00:53.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 -- start start 2024-09-11T23:00:53.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 Processor -- start 2024-09-11T23:00:53.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db7045d0 0x5565db7134d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db713a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.968 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db70f370 0x5565db713f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581b0ee0 con 0x5565db705b40 2024-09-11T23:00:53.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581b0810 con 0x5565db7045d0 2024-09-11T23:00:53.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.955+0000 7fa262a19740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581ada70 con 0x5565db70f370 2024-09-11T23:00:53.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa262a19740 10 client.0 _ll_register_callbacks cb 0x7fa24c00b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:53.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa25dbdb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db713a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:53.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa25dbdb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db713a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33510/0 (socket says 172.21.15.152:33510) 2024-09-11T23:00:53.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa25dbdb640 1 -- 172.21.15.152:0/1672995507 learned_addr learned my addr 172.21.15.152:0/1672995507 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:53.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa25e3dc640 1 --2- 172.21.15.152:0/1672995507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db7045d0 0x5565db7134d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:53.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa25ebdd640 1 --2- 172.21.15.152:0/1672995507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db70f370 0x5565db713f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:53.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa25dbdb640 1 -- 172.21.15.152:0/1672995507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db70f370 msgr2=0x5565db713f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:53.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa25dbdb640 1 --2- 172.21.15.152:0/1672995507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db70f370 0x5565db713f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa25dbdb640 1 -- 172.21.15.152:0/1672995507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db7045d0 msgr2=0x5565db7134d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:53.969 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa25dbdb640 1 --2- 172.21.15.152:0/1672995507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db7045d0 0x5565db7134d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa25dbdb640 1 -- 172.21.15.152:0/1672995507 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565db788ff0 con 0x5565db705b40 2024-09-11T23:00:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa25dbdb640 1 --2- 172.21.15.152:0/1672995507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db713a10 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00b4f0 tx=0x7fa24c012810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa8b3d51accbbf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa23dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa23dffb640 1 -- 172.21.15.152:0/1672995507 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c02d030 con 0x5565db705b40 2024-09-11T23:00:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa23dffb640 1 -- 172.21.15.152:0/1672995507 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7fa24c016750 con 0x5565db705b40 2024-09-11T23:00:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa262a19740 1 -- 172.21.15.152:0/1672995507 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa24c00bd20 con 0x5565db705b40 2024-09-11T23:00:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa23dffb640 1 -- 172.21.15.152:0/1672995507 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa24c01a040 con 0x5565db705b40 2024-09-11T23:00:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa262a19740 10 client.17670 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.956+0000 7fa262a19740 1 -- 172.21.15.152:0/1672995507 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565db78a2d0 con 0x5565db705b40 2024-09-11T23:00:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.957+0000 7fa23dffb640 1 -- 172.21.15.152:0/1672995507 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa24c014430 con 0x5565db705b40 2024-09-11T23:00:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.957+0000 7fa23dffb640 1 -- 172.21.15.152:0/1672995507 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa24c039020 con 0x5565db705b40 2024-09-11T23:00:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.957+0000 7fa262a19740 10 client.17670 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.957+0000 7fa262a19740 10 client.17670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:00:53.970 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.957+0000 7fa262a19740 20 client.17670 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:53.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.957+0000 7fa262a19740 1 --2- 172.21.15.152:0/1672995507 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5565db78a920 0x5565db78cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:53.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.957+0000 7fa262a19740 4 client.17670 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:00:53.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.957+0000 7fa262a19740 1 -- 172.21.15.152:0/1672995507 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa24c00bd20 con 0x5565db78a920 2024-09-11T23:00:53.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.957+0000 7fa25e3dc640 1 --2- 172.21.15.152:0/1672995507 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5565db78a920 0x5565db78cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:53.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.957+0000 7fa23dffb640 10 client.17670 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:53.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.957+0000 7fa25e3dc640 1 --2- 172.21.15.152:0/1672995507 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5565db78a920 0x5565db78cd10 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1909d4b344afb1b9 server_cookie=7e7cf39e80ca514e in_seq=0 out_seq=0 2024-09-11T23:00:53.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa23dffb640 1 -- 172.21.15.152:0/1672995507 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa24c00bd20 con 0x5565db78a920 2024-09-11T23:00:53.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa23dffb640 10 client.17670 handle_command_reply: tid=0 2024-09-11T23:00:53.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa262a19740 1 client.17670 shutdown 2024-09-11T23:00:53.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa262a19740 1 -- 172.21.15.152:0/1672995507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 msgr2=0x5565db713a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:53.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa262a19740 1 --2- 172.21.15.152:0/1672995507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db713a10 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa24c00b4f0 tx=0x7fa24c012810 comp rx=0 tx=0).stop 2024-09-11T23:00:53.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa262a19740 1 -- 172.21.15.152:0/1672995507 shutdown_connections 2024-09-11T23:00:53.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa262a19740 1 --2- 172.21.15.152:0/1672995507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5565db70f370 0x5565db713f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa262a19740 1 --2- 172.21.15.152:0/1672995507 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5565db705b40 0x5565db713a10 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa262a19740 1 --2- 172.21.15.152:0/1672995507 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5565db78a920 0x5565db78cd10 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa262a19740 1 --2- 172.21.15.152:0/1672995507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5565db7045d0 0x5565db7134d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa262a19740 1 -- 172.21.15.152:0/1672995507 >> 172.21.15.152:0/1672995507 conn(0x5565db6cd310 msgr2=0x5565db70d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:53.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa23dffb640 0 client.17670 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:53.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa262a19740 1 -- 172.21.15.152:0/1672995507 shutdown_connections 2024-09-11T23:00:53.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa262a19740 1 -- 172.21.15.152:0/1672995507 wait complete. 2024-09-11T23:00:53.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa262a19740 20 client.17670 trim_cache size 0 max 16384 2024-09-11T23:00:53.973 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:00:53.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa260640640 1 -- 172.21.15.152:0/1632764202 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa2400a2cf0 msgr2=0x7fa2400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:53.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa260640640 1 --2- 172.21.15.152:0/1632764202 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa2400a2cf0 0x7fa2400a51b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa25004a000 tx=0x7fa250048000 comp rx=0 tx=0).stop 2024-09-11T23:00:53.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa260640640 1 -- 172.21.15.152:0/1632764202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa258100e20 msgr2=0x7fa2580fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:53.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa260640640 1 --2- 172.21.15.152:0/1632764202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa258100e20 0x7fa2580fcc80 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fa24800eab0 tx=0x7fa24800ef70 comp rx=0 tx=0).stop 2024-09-11T23:00:53.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa260640640 1 -- 172.21.15.152:0/1632764202 shutdown_connections 2024-09-11T23:00:53.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa260640640 1 --2- 172.21.15.152:0/1632764202 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa2400a2cf0 0x7fa2400a51b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa260640640 1 --2- 172.21.15.152:0/1632764202 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa258102ef0 0x7fa2580fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa260640640 1 --2- 172.21.15.152:0/1632764202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa258102570 0x7fa2580fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.958+0000 7fa260640640 1 --2- 172.21.15.152:0/1632764202 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa258100e20 0x7fa2580fcc80 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:53.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.959+0000 7fa260640640 1 -- 172.21.15.152:0/1632764202 >> 172.21.15.152:0/1632764202 conn(0x7fa2580fc7d0 msgr2=0x7fa25806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:53.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.959+0000 7fa260640640 1 -- 172.21.15.152:0/1632764202 shutdown_connections 2024-09-11T23:00:53.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:53.959+0000 7fa260640640 1 -- 172.21.15.152:0/1632764202 wait complete. 2024-09-11T23:00:54.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:54 smithi152 ceph-mon[31181]: pgmap v245: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:54.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:54 smithi191 ceph-mon[35211]: pgmap v245: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:54.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:54 smithi204 ceph-mon[35454]: pgmap v245: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:55.693 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:00:55.694 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:00:55.950 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:56.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:56 smithi152 ceph-mon[31181]: pgmap v246: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:00:56.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.480+0000 7f38f15de640 1 -- 172.21.15.152:0/1710126151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38ec101420 msgr2=0x7f38ec101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:56.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.480+0000 7f38f15de640 1 --2- 172.21.15.152:0/1710126151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38ec101420 0x7f38ec101800 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009900 tx=0x7f38dc02a410 comp rx=0 tx=0).stop 2024-09-11T23:00:56.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.480+0000 7f38f15de640 1 -- 172.21.15.152:0/1710126151 shutdown_connections 2024-09-11T23:00:56.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.480+0000 7f38f15de640 1 --2- 172.21.15.152:0/1710126151 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38ec106030 0x7f38ec10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.480+0000 7f38f15de640 1 --2- 172.21.15.152:0/1710126151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ec101dd0 0x7f38ec105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.480+0000 7f38f15de640 1 --2- 172.21.15.152:0/1710126151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38ec101420 0x7f38ec101800 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.480+0000 7f38f15de640 1 -- 172.21.15.152:0/1710126151 >> 172.21.15.152:0/1710126151 conn(0x7f38ec0fbee0 msgr2=0x7f38ec0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.480+0000 7f38f15de640 1 -- 172.21.15.152:0/1710126151 shutdown_connections 2024-09-11T23:00:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.480+0000 7f38f15de640 1 -- 172.21.15.152:0/1710126151 wait complete. 2024-09-11T23:00:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.481+0000 7f38f15de640 1 Processor -- start 2024-09-11T23:00:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.481+0000 7f38f15de640 1 -- start start 2024-09-11T23:00:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.481+0000 7f38f15de640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38ec101420 0x7f38ec1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.481+0000 7f38f15de640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ec101dd0 0x7f38ec198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.481+0000 7f38f15de640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38ec106030 0x7f38ec19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.481+0000 7f38f15de640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec111050 con 0x7f38ec106030 2024-09-11T23:00:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.481+0000 7f38f15de640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec110ed0 con 0x7f38ec101420 2024-09-11T23:00:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.481+0000 7f38f15de640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec1111d0 con 0x7f38ec101dd0 2024-09-11T23:00:56.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.481+0000 7f38eb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38ec106030 0x7f38ec19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.481+0000 7f38eb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38ec106030 0x7f38ec19cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33526/0 (socket says 172.21.15.152:33526) 2024-09-11T23:00:56.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.481+0000 7f38eb7fe640 1 -- 172.21.15.152:0/2118336315 learned_addr learned my addr 172.21.15.152:0/2118336315 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:56.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.481+0000 7f38eaffd640 1 --2- 172.21.15.152:0/2118336315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38ec101420 0x7f38ec1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.482+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/2118336315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ec101dd0 0x7f38ec198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.482+0000 7f38eb7fe640 1 -- 172.21.15.152:0/2118336315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ec101dd0 msgr2=0x7f38ec198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:56.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.482+0000 7f38eb7fe640 1 --2- 172.21.15.152:0/2118336315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ec101dd0 0x7f38ec198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.482+0000 7f38eb7fe640 1 -- 172.21.15.152:0/2118336315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38ec101420 msgr2=0x7f38ec1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:56.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.482+0000 7f38eb7fe640 1 --2- 172.21.15.152:0/2118336315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38ec101420 0x7f38ec1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.482+0000 7f38eb7fe640 1 -- 172.21.15.152:0/2118336315 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec19d670 con 0x7f38ec106030 2024-09-11T23:00:56.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.482+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/2118336315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ec101dd0 0x7f38ec198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:00:56.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.482+0000 7f38eb7fe640 1 --2- 172.21.15.152:0/2118336315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38ec106030 0x7f38ec19cf90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f38e400eaa0 tx=0x7f38e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:56.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.482+0000 7f38d3fff640 1 -- 172.21.15.152:0/2118336315 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e400cdb0 con 0x7f38ec106030 2024-09-11T23:00:56.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.482+0000 7f38d3fff640 1 -- 172.21.15.152:0/2118336315 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38e4005020 con 0x7f38ec106030 2024-09-11T23:00:56.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.482+0000 7f38f15de640 1 -- 172.21.15.152:0/2118336315 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ec19d960 con 0x7f38ec106030 2024-09-11T23:00:56.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.482+0000 7f38d3fff640 1 -- 172.21.15.152:0/2118336315 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e40116b0 con 0x7f38ec106030 2024-09-11T23:00:56.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.482+0000 7f38f15de640 1 -- 172.21.15.152:0/2118336315 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ec1a5240 con 0x7f38ec106030 2024-09-11T23:00:56.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.484+0000 7f38d3fff640 1 -- 172.21.15.152:0/2118336315 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f38e4005560 con 0x7f38ec106030 2024-09-11T23:00:56.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38f39b7740 1 Processor -- start 2024-09-11T23:00:56.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38f39b7740 1 -- start start 2024-09-11T23:00:56.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38f39b7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dab7b60 0x55975dab7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38f39b7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dab8480 0x55975dac2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38f39b7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dac2d40 0x55975dac5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38f39b7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55975da1d280 con 0x55975dac2d40 2024-09-11T23:00:56.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38f39b7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55975d9c4b10 con 0x55975dab7b60 2024-09-11T23:00:56.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38f39b7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55975da19240 con 0x55975dab8480 2024-09-11T23:00:56.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38ea7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dac2d40 0x55975dac5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38ea7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dac2d40 0x55975dac5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33534/0 (socket says 172.21.15.152:33534) 2024-09-11T23:00:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38ea7fc640 1 -- 172.21.15.152:0/2108853520 learned_addr learned my addr 172.21.15.152:0/2108853520 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38eaffd640 1 --2- 172.21.15.152:0/2108853520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dab8480 0x55975dac2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/2108853520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dab7b60 0x55975dab7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38ea7fc640 1 -- 172.21.15.152:0/2108853520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dab8480 msgr2=0x55975dac2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/2108853520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dab8480 0x55975dac2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38d3fff640 1 --2- 172.21.15.152:0/2118336315 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f38d40a2cf0 0x7f38d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38ea7fc640 1 -- 172.21.15.152:0/2108853520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dab7b60 msgr2=0x55975dab7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:56.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/2108853520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dab7b60 0x55975dab7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38ea7fc640 1 -- 172.21.15.152:0/2108853520 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55975d9d11f0 con 0x55975dac2d40 2024-09-11T23:00:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.485+0000 7f38eaffd640 1 --2- 172.21.15.152:0/2108853520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dab8480 0x55975dac2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.486+0000 7f38eb7fe640 1 --2- 172.21.15.152:0/2118336315 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f38d40a2cf0 0x7f38d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.486+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/2108853520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dac2d40 0x55975dac5130 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f38e00124f0 tx=0x7f38e000b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5df6337226db58 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.486+0000 7f38d3fff640 1 -- 172.21.15.152:0/2118336315 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f38e40cf620 con 0x7f38ec106030 2024-09-11T23:00:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.486+0000 7f38d17fa640 1 -- 172.21.15.152:0/2108853520 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e0018070 con 0x55975dac2d40 2024-09-11T23:00:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.486+0000 7f38d17fa640 1 -- 172.21.15.152:0/2108853520 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38e001db40 con 0x55975dac2d40 2024-09-11T23:00:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.486+0000 7f38d17fa640 1 -- 172.21.15.152:0/2108853520 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e0010b50 con 0x55975dac2d40 2024-09-11T23:00:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 -- 172.21.15.152:0/2108853520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dac2d40 msgr2=0x55975dac5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38eb7fe640 1 --2- 172.21.15.152:0/2118336315 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f38d40a2cf0 0x7f38d40a51b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f38e40cce40 tx=0x7f38e406ebd0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 --2- 172.21.15.152:0/2108853520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dac2d40 0x55975dac5130 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f38e00124f0 tx=0x7f38e000b600 comp rx=0 tx=0).stop 2024-09-11T23:00:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 -- 172.21.15.152:0/2108853520 shutdown_connections 2024-09-11T23:00:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 --2- 172.21.15.152:0/2108853520 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dac2d40 0x55975dac5130 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 --2- 172.21.15.152:0/2108853520 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dab8480 0x55975dac2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 --2- 172.21.15.152:0/2108853520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dab7b60 0x55975dab7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 -- 172.21.15.152:0/2108853520 >> 172.21.15.152:0/2108853520 conn(0x55975dab40f0 msgr2=0x55975dab44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 -- 172.21.15.152:0/2108853520 shutdown_connections 2024-09-11T23:00:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 -- 172.21.15.152:0/2108853520 wait complete. 2024-09-11T23:00:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 -- start start 2024-09-11T23:00:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 Processor -- start 2024-09-11T23:00:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dab7b60 0x55975dabfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dab8480 0x55975dac01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dac2d40 0x55975dac0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55975dac5910 con 0x55975dac2d40 2024-09-11T23:00:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55975d9d7f10 con 0x55975dab8480 2024-09-11T23:00:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55975dac5a90 con 0x55975dab7b60 2024-09-11T23:00:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38f39b7740 10 client.0 _ll_register_callbacks cb 0x55975d9a0a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:56.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38ea7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dac2d40 0x55975dac0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38ea7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dac2d40 0x55975dac0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33542/0 (socket says 172.21.15.152:33542) 2024-09-11T23:00:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.487+0000 7f38ea7fc640 1 -- 172.21.15.152:0/4097609287 learned_addr learned my addr 172.21.15.152:0/4097609287 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.488+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/4097609287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dab7b60 0x55975dabfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.488+0000 7f38ea7fc640 1 -- 172.21.15.152:0/4097609287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dab7b60 msgr2=0x55975dabfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.488+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/4097609287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dab7b60 0x55975dabfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.488+0000 7f38ea7fc640 1 -- 172.21.15.152:0/4097609287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dab8480 msgr2=0x55975dac01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.488+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/4097609287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dab8480 0x55975dac01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.488+0000 7f38ea7fc640 1 -- 172.21.15.152:0/4097609287 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55975d78e6a0 con 0x55975dac2d40 2024-09-11T23:00:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.488+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/4097609287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dac2d40 0x55975dac0700 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f38e0012e40 tx=0x7f38e0012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3261f602b3a0ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.488+0000 7f38ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.488+0000 7f38ceff5640 1 -- 172.21.15.152:0/4097609287 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e0018040 con 0x55975dac2d40 2024-09-11T23:00:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.488+0000 7f38ceff5640 1 -- 172.21.15.152:0/4097609287 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f38e0004020 con 0x55975dac2d40 2024-09-11T23:00:56.501 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.488+0000 7f38f39b7740 1 -- 172.21.15.152:0/4097609287 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38e001db40 con 0x55975dac2d40 2024-09-11T23:00:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.488+0000 7f38ceff5640 1 -- 172.21.15.152:0/4097609287 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f38e0030050 con 0x55975dac2d40 2024-09-11T23:00:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.488+0000 7f38f39b7740 10 client.17682 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.488+0000 7f38f39b7740 1 -- 172.21.15.152:0/4097609287 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55975dad1570 con 0x55975dac2d40 2024-09-11T23:00:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.488+0000 7f38ceff5640 1 -- 172.21.15.152:0/4097609287 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38e001db40 con 0x55975dac2d40 2024-09-11T23:00:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.489+0000 7f38ceff5640 1 -- 172.21.15.152:0/4097609287 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f38e000f040 con 0x55975dac2d40 2024-09-11T23:00:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.489+0000 7f38f39b7740 10 client.17682 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.489+0000 7f38f39b7740 10 client.17682 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:00:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.489+0000 7f38f39b7740 20 client.17682 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.489+0000 7f38f39b7740 1 --2- 172.21.15.152:0/4097609287 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55975dad2980 0x55975dad4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.489+0000 7f38f39b7740 4 client.17682 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:00:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.489+0000 7f38f39b7740 1 -- 172.21.15.152:0/4097609287 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55975da2a7f0 con 0x55975dad2980 2024-09-11T23:00:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.489+0000 7f38eaffd640 1 --2- 172.21.15.152:0/4097609287 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55975dad2980 0x55975dad4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.489+0000 7f38ceff5640 10 client.17682 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:56.502 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.489+0000 7f38eaffd640 1 --2- 172.21.15.152:0/4097609287 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55975dad2980 0x55975dad4d70 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d996dcdd98a4a035 server_cookie=d344e79e54747884 in_seq=0 out_seq=0 2024-09-11T23:00:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.490+0000 7f38ceff5640 1 -- 172.21.15.152:0/4097609287 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55975da2a7f0 con 0x55975dad2980 2024-09-11T23:00:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.490+0000 7f38ceff5640 10 client.17682 handle_command_reply: tid=0 2024-09-11T23:00:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.490+0000 7f38f39b7740 1 client.17682 shutdown 2024-09-11T23:00:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.490+0000 7f38f39b7740 1 -- 172.21.15.152:0/4097609287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dac2d40 msgr2=0x55975dac0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.490+0000 7f38f39b7740 1 --2- 172.21.15.152:0/4097609287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dac2d40 0x55975dac0700 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f38e0012e40 tx=0x7f38e0012f90 comp rx=0 tx=0).stop 2024-09-11T23:00:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.490+0000 7f38f39b7740 1 -- 172.21.15.152:0/4097609287 shutdown_connections 2024-09-11T23:00:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.490+0000 7f38f39b7740 1 --2- 172.21.15.152:0/4097609287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dac2d40 0x55975dac0700 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.490+0000 7f38f39b7740 1 --2- 172.21.15.152:0/4097609287 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55975dad2980 0x55975dad4d70 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.490+0000 7f38f39b7740 1 --2- 172.21.15.152:0/4097609287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dab8480 0x55975dac01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.490+0000 7f38f39b7740 1 --2- 172.21.15.152:0/4097609287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dab7b60 0x55975dabfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.490+0000 7f38f39b7740 1 -- 172.21.15.152:0/4097609287 >> 172.21.15.152:0/4097609287 conn(0x55975dab40f0 msgr2=0x55975dab9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.490+0000 7f38ceff5640 0 client.17682 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.490+0000 7f38f39b7740 1 -- 172.21.15.152:0/4097609287 shutdown_connections 2024-09-11T23:00:56.503 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.490+0000 7f38f39b7740 1 -- 172.21.15.152:0/4097609287 wait complete. 2024-09-11T23:00:56.504 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.490+0000 7f38f39b7740 20 client.17682 trim_cache size 0 max 16384 2024-09-11T23:00:56.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.507+0000 7f38f39b7740 1 Processor -- start 2024-09-11T23:00:56.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.507+0000 7f38f39b7740 1 -- start start 2024-09-11T23:00:56.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.507+0000 7f38f39b7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dab7b60 0x55975dab3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.507+0000 7f38f39b7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dab8480 0x55975dada370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.507+0000 7f38f39b7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dac2d40 0x55975dada8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.507+0000 7f38f39b7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec1ad8b0 con 0x55975dab8480 2024-09-11T23:00:56.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.507+0000 7f38f39b7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec1af3f0 con 0x55975dab7b60 2024-09-11T23:00:56.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.507+0000 7f38f39b7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec1adf80 con 0x55975dac2d40 2024-09-11T23:00:56.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.507+0000 7f38eaffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dab8480 0x55975dada370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.507+0000 7f38eaffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dab8480 0x55975dada370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33550/0 (socket says 172.21.15.152:33550) 2024-09-11T23:00:56.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.507+0000 7f38eaffd640 1 -- 172.21.15.152:0/3243933156 learned_addr learned my addr 172.21.15.152:0/3243933156 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:56.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.507+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/3243933156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dab7b60 0x55975dab3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.512 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.508+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/3243933156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dac2d40 0x55975dada8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.508+0000 7f38eaffd640 1 -- 172.21.15.152:0/3243933156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dac2d40 msgr2=0x55975dada8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:56.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.508+0000 7f38eaffd640 1 --2- 172.21.15.152:0/3243933156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dac2d40 0x55975dada8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.508+0000 7f38eaffd640 1 -- 172.21.15.152:0/3243933156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dab7b60 msgr2=0x55975dab3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:56.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.508+0000 7f38eaffd640 1 --2- 172.21.15.152:0/3243933156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dab7b60 0x55975dab3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.508+0000 7f38eaffd640 1 -- 172.21.15.152:0/3243933156 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55975dac2260 con 0x55975dab8480 2024-09-11T23:00:56.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.508+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/3243933156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dab7b60 0x55975dab3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:56.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.508+0000 7f38eaffd640 1 --2- 172.21.15.152:0/3243933156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dab8480 0x55975dada370 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009900 tx=0x7f38dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c65480aa95f9ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:56.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.508+0000 7f38d0ff9640 1 -- 172.21.15.152:0/3243933156 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc035040 con 0x55975dab8480 2024-09-11T23:00:56.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.508+0000 7f38d0ff9640 1 -- 172.21.15.152:0/3243933156 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38dc048070 con 0x55975dab8480 2024-09-11T23:00:56.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.508+0000 7f38d0ff9640 1 -- 172.21.15.152:0/3243933156 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc003d30 con 0x55975dab8480 2024-09-11T23:00:56.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 -- 172.21.15.152:0/3243933156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dab8480 msgr2=0x55975dada370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:56.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 --2- 172.21.15.152:0/3243933156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dab8480 0x55975dada370 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009900 tx=0x7f38dc000bc0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.513 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 -- 172.21.15.152:0/3243933156 shutdown_connections 2024-09-11T23:00:56.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 --2- 172.21.15.152:0/3243933156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dac2d40 0x55975dada8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 --2- 172.21.15.152:0/3243933156 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dab8480 0x55975dada370 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 --2- 172.21.15.152:0/3243933156 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dab7b60 0x55975dab3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 -- 172.21.15.152:0/3243933156 >> 172.21.15.152:0/3243933156 conn(0x55975dab40f0 msgr2=0x55975dab2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:56.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 -- 172.21.15.152:0/3243933156 shutdown_connections 2024-09-11T23:00:56.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 -- 172.21.15.152:0/3243933156 wait complete. 2024-09-11T23:00:56.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 -- start start 2024-09-11T23:00:56.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 Processor -- start 2024-09-11T23:00:56.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dab7b60 0x55975dac6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dab8480 0x55975dac7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dac2d40 0x55975dac7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec1aed20 con 0x55975dab7b60 2024-09-11T23:00:56.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec1ae650 con 0x55975dac2d40 2024-09-11T23:00:56.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec1ab8b0 con 0x55975dab8480 2024-09-11T23:00:56.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38f39b7740 10 client.0 _ll_register_callbacks cb 0x7f38e000bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:56.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38ea7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dab7b60 0x55975dac6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38ea7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dab7b60 0x55975dac6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33562/0 (socket says 172.21.15.152:33562) 2024-09-11T23:00:56.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.509+0000 7f38ea7fc640 1 -- 172.21.15.152:0/1065387468 learned_addr learned my addr 172.21.15.152:0/1065387468 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:56.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.510+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/1065387468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dac2d40 0x55975dac7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.510+0000 7f38eaffd640 1 --2- 172.21.15.152:0/1065387468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dab8480 0x55975dac7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.510+0000 7f38ea7fc640 1 -- 172.21.15.152:0/1065387468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dab8480 msgr2=0x55975dac7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:56.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.510+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/1065387468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dab8480 0x55975dac7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.510+0000 7f38ea7fc640 1 -- 172.21.15.152:0/1065387468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dac2d40 msgr2=0x55975dac7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:56.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.510+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/1065387468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dac2d40 0x55975dac7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.510+0000 7f38ea7fc640 1 -- 172.21.15.152:0/1065387468 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55975db3c4e0 con 0x55975dab7b60 2024-09-11T23:00:56.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.510+0000 7f38ea7fc640 1 --2- 172.21.15.152:0/1065387468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dab7b60 0x55975dac6be0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f38e000b640 tx=0x7f38e000bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e2eafb3fd961d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:56.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.510+0000 7f38ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:56.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.510+0000 7f38ceff5640 1 -- 172.21.15.152:0/1065387468 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e0018040 con 0x55975dab7b60 2024-09-11T23:00:56.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.510+0000 7f38ceff5640 1 -- 172.21.15.152:0/1065387468 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f38e001ed40 con 0x55975dab7b60 2024-09-11T23:00:56.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.510+0000 7f38f39b7740 1 -- 172.21.15.152:0/1065387468 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38dc048070 con 0x55975dab7b60 2024-09-11T23:00:56.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.510+0000 7f38ceff5640 1 -- 172.21.15.152:0/1065387468 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f38e0048050 con 0x55975dab7b60 2024-09-11T23:00:56.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.510+0000 7f38f39b7740 10 client.17688 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:56.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.510+0000 7f38f39b7740 1 -- 172.21.15.152:0/1065387468 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55975db3d800 con 0x55975dab7b60 2024-09-11T23:00:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.510+0000 7f38ceff5640 1 -- 172.21.15.152:0/1065387468 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38dc048070 con 0x55975dab7b60 2024-09-11T23:00:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.511+0000 7f38ceff5640 1 -- 172.21.15.152:0/1065387468 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f38e00048e0 con 0x55975dab7b60 2024-09-11T23:00:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.511+0000 7f38f39b7740 10 client.17688 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.511+0000 7f38f39b7740 10 client.17688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:00:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.511+0000 7f38f39b7740 20 client.17688 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.511+0000 7f38f39b7740 1 --2- 172.21.15.152:0/1065387468 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55975db3de10 0x55975db40230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.511+0000 7f38f39b7740 4 client.17688 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:00:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.511+0000 7f38f39b7740 1 -- 172.21.15.152:0/1065387468 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55975dad5160 con 0x55975db3de10 2024-09-11T23:00:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.511+0000 7f38eaffd640 1 --2- 172.21.15.152:0/1065387468 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55975db3de10 0x55975db40230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.511+0000 7f38ceff5640 10 client.17688 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:56.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.511+0000 7f38eaffd640 1 --2- 172.21.15.152:0/1065387468 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55975db3de10 0x55975db40230 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae22cc59a4a39e2e server_cookie=7f6dabd6f4a47c1a in_seq=0 out_seq=0 2024-09-11T23:00:56.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.512+0000 7f38ceff5640 1 -- 172.21.15.152:0/1065387468 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55975dad5160 con 0x55975db3de10 2024-09-11T23:00:56.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.512+0000 7f38ceff5640 10 client.17688 handle_command_reply: tid=0 2024-09-11T23:00:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.512+0000 7f38f39b7740 1 client.17688 shutdown 2024-09-11T23:00:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.512+0000 7f38f39b7740 1 -- 172.21.15.152:0/1065387468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dab7b60 msgr2=0x55975dac6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.512+0000 7f38f39b7740 1 --2- 172.21.15.152:0/1065387468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dab7b60 0x55975dac6be0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f38e000b640 tx=0x7f38e000bc70 comp rx=0 tx=0).stop 2024-09-11T23:00:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.512+0000 7f38f39b7740 1 -- 172.21.15.152:0/1065387468 shutdown_connections 2024-09-11T23:00:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f39b7740 1 --2- 172.21.15.152:0/1065387468 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55975db3de10 0x55975db40230 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f39b7740 1 --2- 172.21.15.152:0/1065387468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55975dac2d40 0x55975dac7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f39b7740 1 --2- 172.21.15.152:0/1065387468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55975dab8480 0x55975dac7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f39b7740 1 --2- 172.21.15.152:0/1065387468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55975dab7b60 0x55975dac6be0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f39b7740 1 -- 172.21.15.152:0/1065387468 >> 172.21.15.152:0/1065387468 conn(0x55975dab40f0 msgr2=0x55975dac0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38ceff5640 0 client.17688 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f39b7740 1 -- 172.21.15.152:0/1065387468 shutdown_connections 2024-09-11T23:00:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f39b7740 1 -- 172.21.15.152:0/1065387468 wait complete. 2024-09-11T23:00:56.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f39b7740 20 client.17688 trim_cache size 0 max 16384 2024-09-11T23:00:56.520 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:00:56.520 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T23:00:56.520 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "ee0d42de-f4a4-4722-9f4d-753db10080f5", 2024-09-11T23:00:56.520 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T23:00:56.520 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:00:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f15de640 1 -- 172.21.15.152:0/2118336315 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f38d40a2cf0 msgr2=0x7f38d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f15de640 1 --2- 172.21.15.152:0/2118336315 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f38d40a2cf0 0x7f38d40a51b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f38e40cce40 tx=0x7f38e406ebd0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f15de640 1 -- 172.21.15.152:0/2118336315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38ec106030 msgr2=0x7f38ec19cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f15de640 1 --2- 172.21.15.152:0/2118336315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38ec106030 0x7f38ec19cf90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f38e400eaa0 tx=0x7f38e400ef60 comp rx=0 tx=0).stop 2024-09-11T23:00:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f15de640 1 -- 172.21.15.152:0/2118336315 shutdown_connections 2024-09-11T23:00:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f15de640 1 --2- 172.21.15.152:0/2118336315 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f38d40a2cf0 0x7f38d40a51b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f15de640 1 --2- 172.21.15.152:0/2118336315 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38ec106030 0x7f38ec19cf90 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f15de640 1 --2- 172.21.15.152:0/2118336315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38ec101dd0 0x7f38ec198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f15de640 1 --2- 172.21.15.152:0/2118336315 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38ec101420 0x7f38ec1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.513+0000 7f38f15de640 1 -- 172.21.15.152:0/2118336315 >> 172.21.15.152:0/2118336315 conn(0x7f38ec0fbee0 msgr2=0x7f38ec1003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.514+0000 7f38f15de640 1 -- 172.21.15.152:0/2118336315 shutdown_connections 2024-09-11T23:00:56.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:56.514+0000 7f38f15de640 1 -- 172.21.15.152:0/2118336315 wait complete. 2024-09-11T23:00:56.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:56 smithi191 ceph-mon[35211]: pgmap v246: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:00:56.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:56 smithi204 ceph-mon[35454]: pgmap v246: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:00:57.346 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:00:57.368 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:57 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T23:00:57.368 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:57 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T23:00:57.369 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:57 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T23:00:57.369 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:57 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T23:00:57.591 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:57 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T23:00:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:57 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T23:00:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:57 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T23:00:57.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:57 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T23:00:57.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:57 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T23:00:57.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:57 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T23:00:57.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:57 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T23:00:57.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:57 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T23:00:58.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.094+0000 7f3c2e16f640 1 -- 172.21.15.152:0/2076216667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c28101dd0 msgr2=0x7f3c281021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:58.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.094+0000 7f3c2e16f640 1 --2- 172.21.15.152:0/2076216667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c28101dd0 0x7f3c281021b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009900 tx=0x7f3c1802a410 comp rx=0 tx=0).stop 2024-09-11T23:00:58.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.094+0000 7f3c2e16f640 1 -- 172.21.15.152:0/2076216667 shutdown_connections 2024-09-11T23:00:58.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.094+0000 7f3c2e16f640 1 --2- 172.21.15.152:0/2076216667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c28106f00 0x7f3c2810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.094+0000 7f3c2e16f640 1 --2- 172.21.15.152:0/2076216667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c28102780 0x7f3c281067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.094+0000 7f3c2e16f640 1 --2- 172.21.15.152:0/2076216667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c28101dd0 0x7f3c281021b0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.094+0000 7f3c2e16f640 1 -- 172.21.15.152:0/2076216667 >> 172.21.15.152:0/2076216667 conn(0x7f3c280fd890 msgr2=0x7f3c280ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:58.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.094+0000 7f3c2e16f640 1 -- 172.21.15.152:0/2076216667 shutdown_connections 2024-09-11T23:00:58.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.094+0000 7f3c2e16f640 1 -- 172.21.15.152:0/2076216667 wait complete. 2024-09-11T23:00:58.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.095+0000 7f3c2e16f640 1 Processor -- start 2024-09-11T23:00:58.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.095+0000 7f3c2e16f640 1 -- start start 2024-09-11T23:00:58.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.095+0000 7f3c2e16f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c28101dd0 0x7f3c28198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.095+0000 7f3c2e16f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c28102780 0x7f3c281989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.095+0000 7f3c2e16f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c28106f00 0x7f3c2819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.095+0000 7f3c2e16f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3c28111f60 con 0x7f3c28102780 2024-09-11T23:00:58.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.095+0000 7f3c2e16f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3c28111de0 con 0x7f3c28106f00 2024-09-11T23:00:58.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.095+0000 7f3c2e16f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281120e0 con 0x7f3c28101dd0 2024-09-11T23:00:58.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.095+0000 7f3c26ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c28102780 0x7f3c281989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.096+0000 7f3c26ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c28102780 0x7f3c281989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33580/0 (socket says 172.21.15.152:33580) 2024-09-11T23:00:58.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.096+0000 7f3c26ffd640 1 -- 172.21.15.152:0/2799404854 learned_addr learned my addr 172.21.15.152:0/2799404854 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:58.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.096+0000 7f3c27fff640 1 --2- 172.21.15.152:0/2799404854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c28106f00 0x7f3c2819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.096+0000 7f3c277fe640 1 --2- 172.21.15.152:0/2799404854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c28101dd0 0x7f3c28198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.096+0000 7f3c26ffd640 1 -- 172.21.15.152:0/2799404854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c28101dd0 msgr2=0x7f3c28198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:58.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.096+0000 7f3c26ffd640 1 --2- 172.21.15.152:0/2799404854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c28101dd0 0x7f3c28198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.096+0000 7f3c26ffd640 1 -- 172.21.15.152:0/2799404854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c28106f00 msgr2=0x7f3c2819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:58.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.096+0000 7f3c26ffd640 1 --2- 172.21.15.152:0/2799404854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c28106f00 0x7f3c2819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.096+0000 7f3c26ffd640 1 -- 172.21.15.152:0/2799404854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2819d410 con 0x7f3c28102780 2024-09-11T23:00:58.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.096+0000 7f3c26ffd640 1 --2- 172.21.15.152:0/2799404854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c28102780 0x7f3c281989a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00d950 tx=0x7f3c1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:58.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.096+0000 7f3c24ff9640 1 -- 172.21.15.152:0/2799404854 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c017070 con 0x7f3c28102780 2024-09-11T23:00:58.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.096+0000 7f3c24ff9640 1 -- 172.21.15.152:0/2799404854 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c1c009c30 con 0x7f3c28102780 2024-09-11T23:00:58.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.096+0000 7f3c2e16f640 1 -- 172.21.15.152:0/2799404854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2819d700 con 0x7f3c28102780 2024-09-11T23:00:58.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.096+0000 7f3c24ff9640 1 -- 172.21.15.152:0/2799404854 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c00f6b0 con 0x7f3c28102780 2024-09-11T23:00:58.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.096+0000 7f3c2e16f640 1 -- 172.21.15.152:0/2799404854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c281a4fe0 con 0x7f3c28102780 2024-09-11T23:00:58.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.098+0000 7f3c24ff9640 1 -- 172.21.15.152:0/2799404854 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3c1c0109d0 con 0x7f3c28102780 2024-09-11T23:00:58.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.099+0000 7f3c30548740 1 Processor -- start 2024-09-11T23:00:58.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.099+0000 7f3c30548740 1 -- start start 2024-09-11T23:00:58.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.099+0000 7f3c30548740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f8884a0 0x56241f888880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.099+0000 7f3c30548740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f888dc0 0x56241f893140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.099+0000 7f3c30548740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f893680 0x56241f895a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.099+0000 7f3c24ff9640 1 --2- 172.21.15.152:0/2799404854 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3c100a2cf0 0x7f3c100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.099+0000 7f3c30548740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56241f7ee2e0 con 0x56241f893680 2024-09-11T23:00:58.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.099+0000 7f3c30548740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56241f795ce0 con 0x56241f8884a0 2024-09-11T23:00:58.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.099+0000 7f3c30548740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56241f7ea2a0 con 0x56241f888dc0 2024-09-11T23:00:58.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.099+0000 7f3c26ffd640 1 --2- 172.21.15.152:0/2799404854 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3c100a2cf0 0x7f3c100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.099+0000 7f3c27fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f893680 0x56241f895a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.099+0000 7f3c27fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f893680 0x56241f895a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33586/0 (socket says 172.21.15.152:33586) 2024-09-11T23:00:58.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.099+0000 7f3c27fff640 1 -- 172.21.15.152:0/1734837997 learned_addr learned my addr 172.21.15.152:0/1734837997 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:58.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.099+0000 7f3c27fff640 1 --2- 172.21.15.152:0/1734837997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f8884a0 0x56241f888880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.099+0000 7f3c24ff9640 1 -- 172.21.15.152:0/2799404854 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f3c1c0cf010 con 0x7f3c28102780 2024-09-11T23:00:58.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.099+0000 7f3c277fe640 1 --2- 172.21.15.152:0/1734837997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f888dc0 0x56241f893140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.100+0000 7f3c27fff640 1 -- 172.21.15.152:0/1734837997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f888dc0 msgr2=0x56241f893140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:58.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.100+0000 7f3c27fff640 1 --2- 172.21.15.152:0/1734837997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f888dc0 0x56241f893140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.100+0000 7f3c27fff640 1 -- 172.21.15.152:0/1734837997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f8884a0 msgr2=0x56241f888880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:58.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.100+0000 7f3c27fff640 1 --2- 172.21.15.152:0/1734837997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f8884a0 0x56241f888880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.100+0000 7f3c27fff640 1 -- 172.21.15.152:0/1734837997 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56241f77a440 con 0x56241f893680 2024-09-11T23:00:58.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.100+0000 7f3c277fe640 1 --2- 172.21.15.152:0/1734837997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f888dc0 0x56241f893140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:58.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.100+0000 7f3c27fff640 1 --2- 172.21.15.152:0/1734837997 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f893680 0x56241f895a70 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f3c20005f40 tx=0x7f3c20005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c2ae44f3777e32 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:58.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.100+0000 7f3c26ffd640 1 --2- 172.21.15.152:0/2799404854 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3c100a2cf0 0x7f3c100a51b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c097fd0 tx=0x7f3c1c005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:58.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.100+0000 7f3c0dffb640 1 -- 172.21.15.152:0/1734837997 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c20011130 con 0x56241f893680 2024-09-11T23:00:58.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.100+0000 7f3c0dffb640 1 -- 172.21.15.152:0/1734837997 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c200112d0 con 0x56241f893680 2024-09-11T23:00:58.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.100+0000 7f3c0dffb640 1 -- 172.21.15.152:0/1734837997 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000fc80 con 0x56241f893680 2024-09-11T23:00:58.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.101+0000 7f3c30548740 1 -- 172.21.15.152:0/1734837997 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f893680 msgr2=0x56241f895a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:58.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.101+0000 7f3c30548740 1 --2- 172.21.15.152:0/1734837997 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f893680 0x56241f895a70 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f3c20005f40 tx=0x7f3c20005bd0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.101+0000 7f3c30548740 1 -- 172.21.15.152:0/1734837997 shutdown_connections 2024-09-11T23:00:58.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.101+0000 7f3c30548740 1 --2- 172.21.15.152:0/1734837997 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f893680 0x56241f895a70 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.101+0000 7f3c30548740 1 --2- 172.21.15.152:0/1734837997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f888dc0 0x56241f893140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.101+0000 7f3c30548740 1 --2- 172.21.15.152:0/1734837997 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f8884a0 0x56241f888880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.101+0000 7f3c30548740 1 -- 172.21.15.152:0/1734837997 >> 172.21.15.152:0/1734837997 conn(0x56241f884a30 msgr2=0x56241f884e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:58.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.101+0000 7f3c30548740 1 -- 172.21.15.152:0/1734837997 shutdown_connections 2024-09-11T23:00:58.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.101+0000 7f3c30548740 1 -- 172.21.15.152:0/1734837997 wait complete. 2024-09-11T23:00:58.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.101+0000 7f3c30548740 1 -- start start 2024-09-11T23:00:58.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.101+0000 7f3c30548740 1 Processor -- start 2024-09-11T23:00:58.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c30548740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f8884a0 0x56241f893080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c30548740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f888dc0 0x56241f890330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c30548740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f893680 0x56241f890920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c30548740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56241f896250 con 0x56241f8884a0 2024-09-11T23:00:58.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c30548740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56241f7a9050 con 0x56241f893680 2024-09-11T23:00:58.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c30548740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56241f8963d0 con 0x56241f888dc0 2024-09-11T23:00:58.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c30548740 10 client.0 _ll_register_callbacks cb 0x56241f7e9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:58.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c27fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f8884a0 0x56241f893080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c27fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f8884a0 0x56241f893080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33594/0 (socket says 172.21.15.152:33594) 2024-09-11T23:00:58.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c27fff640 1 -- 172.21.15.152:0/1485368789 learned_addr learned my addr 172.21.15.152:0/1485368789 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:58.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c277fe640 1 --2- 172.21.15.152:0/1485368789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f888dc0 0x56241f890330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c27fff640 1 --2- 172.21.15.152:0/1485368789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f893680 0x56241f890920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c27fff640 1 -- 172.21.15.152:0/1485368789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f888dc0 msgr2=0x56241f890330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:58.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c27fff640 1 --2- 172.21.15.152:0/1485368789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f888dc0 0x56241f890330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c27fff640 1 -- 172.21.15.152:0/1485368789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f893680 msgr2=0x56241f890920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:58.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c27fff640 1 --2- 172.21.15.152:0/1485368789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f893680 0x56241f890920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c27fff640 1 -- 172.21.15.152:0/1485368789 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56241f55f3c0 con 0x56241f8884a0 2024-09-11T23:00:58.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c27fff640 1 --2- 172.21.15.152:0/1485368789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f8884a0 0x56241f893080 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3c20009270 tx=0x7f3c20005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824a6835bcd7d127 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:58.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:58.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c06ffd640 1 -- 172.21.15.152:0/1485368789 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000fa00 con 0x56241f8884a0 2024-09-11T23:00:58.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c06ffd640 1 -- 172.21.15.152:0/1485368789 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f3c2000fba0 con 0x56241f8884a0 2024-09-11T23:00:58.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.102+0000 7f3c30548740 1 -- 172.21.15.152:0/1485368789 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c200112d0 con 0x56241f8884a0 2024-09-11T23:00:58.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.103+0000 7f3c06ffd640 1 -- 172.21.15.152:0/1485368789 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c20033050 con 0x56241f8884a0 2024-09-11T23:00:58.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.103+0000 7f3c30548740 10 client.17700 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:58.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.103+0000 7f3c30548740 1 -- 172.21.15.152:0/1485368789 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56241f8a1cb0 con 0x56241f8884a0 2024-09-11T23:00:58.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.103+0000 7f3c06ffd640 1 -- 172.21.15.152:0/1485368789 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c200112d0 con 0x56241f8884a0 2024-09-11T23:00:58.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.103+0000 7f3c06ffd640 1 -- 172.21.15.152:0/1485368789 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3c2001d020 con 0x56241f8884a0 2024-09-11T23:00:58.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.103+0000 7f3c30548740 10 client.17700 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:58.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.103+0000 7f3c30548740 10 client.17700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:00:58.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.103+0000 7f3c30548740 20 client.17700 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:58.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.103+0000 7f3c30548740 1 --2- 172.21.15.152:0/1485368789 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56241f8a30b0 0x56241f8a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.103+0000 7f3c30548740 4 client.17700 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:00:58.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.103+0000 7f3c30548740 1 -- 172.21.15.152:0/1485368789 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56241f7fb850 con 0x56241f8a30b0 2024-09-11T23:00:58.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.103+0000 7f3c277fe640 1 --2- 172.21.15.152:0/1485368789 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56241f8a30b0 0x56241f8a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.104+0000 7f3c277fe640 1 --2- 172.21.15.152:0/1485368789 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56241f8a30b0 0x56241f8a54a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fd8f44239b5050b server_cookie=bc30691aabca0108 in_seq=0 out_seq=0 2024-09-11T23:00:58.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.104+0000 7f3c06ffd640 10 client.17700 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:58.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.104+0000 7f3c06ffd640 1 -- 172.21.15.152:0/1485368789 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56241f7fb850 con 0x56241f8a30b0 2024-09-11T23:00:58.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.104+0000 7f3c06ffd640 10 client.17700 handle_command_reply: tid=0 2024-09-11T23:00:58.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.104+0000 7f3c30548740 1 client.17700 shutdown 2024-09-11T23:00:58.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.104+0000 7f3c30548740 1 -- 172.21.15.152:0/1485368789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f8884a0 msgr2=0x56241f893080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:58.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.104+0000 7f3c30548740 1 --2- 172.21.15.152:0/1485368789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f8884a0 0x56241f893080 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3c20009270 tx=0x7f3c20005c40 comp rx=0 tx=0).stop 2024-09-11T23:00:58.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.105+0000 7f3c30548740 1 -- 172.21.15.152:0/1485368789 shutdown_connections 2024-09-11T23:00:58.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.105+0000 7f3c30548740 1 --2- 172.21.15.152:0/1485368789 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56241f8a30b0 0x56241f8a54a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.105+0000 7f3c30548740 1 --2- 172.21.15.152:0/1485368789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f893680 0x56241f890920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.105+0000 7f3c30548740 1 --2- 172.21.15.152:0/1485368789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f888dc0 0x56241f890330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.105+0000 7f3c30548740 1 --2- 172.21.15.152:0/1485368789 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f8884a0 0x56241f893080 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.105+0000 7f3c30548740 1 -- 172.21.15.152:0/1485368789 >> 172.21.15.152:0/1485368789 conn(0x56241f884a30 msgr2=0x56241f88a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:58.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.105+0000 7f3c06ffd640 0 client.17700 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:58.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.105+0000 7f3c30548740 1 -- 172.21.15.152:0/1485368789 shutdown_connections 2024-09-11T23:00:58.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.105+0000 7f3c30548740 1 -- 172.21.15.152:0/1485368789 wait complete. 2024-09-11T23:00:58.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.105+0000 7f3c30548740 20 client.17700 trim_cache size 0 max 16384 2024-09-11T23:00:58.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c30548740 1 Processor -- start 2024-09-11T23:00:58.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c30548740 1 -- start start 2024-09-11T23:00:58.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c30548740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f8884a0 0x56241f89f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c30548740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f888dc0 0x56241f8aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c30548740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f893680 0x56241f8ab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c30548740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281ad880 con 0x56241f8884a0 2024-09-11T23:00:58.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c30548740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281af3c0 con 0x56241f888dc0 2024-09-11T23:00:58.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c30548740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281adf50 con 0x56241f893680 2024-09-11T23:00:58.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c27fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f8884a0 0x56241f89f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c27fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f8884a0 0x56241f89f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33606/0 (socket says 172.21.15.152:33606) 2024-09-11T23:00:58.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c27fff640 1 -- 172.21.15.152:0/1840735787 learned_addr learned my addr 172.21.15.152:0/1840735787 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:58.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c27fff640 1 -- 172.21.15.152:0/1840735787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f893680 msgr2=0x56241f8ab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:58.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c27fff640 1 --2- 172.21.15.152:0/1840735787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f893680 0x56241f8ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c27fff640 1 -- 172.21.15.152:0/1840735787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f888dc0 msgr2=0x56241f8aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:58.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c277fe640 1 --2- 172.21.15.152:0/1840735787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f888dc0 0x56241f8aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c27fff640 1 --2- 172.21.15.152:0/1840735787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f888dc0 0x56241f8aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c27fff640 1 -- 172.21.15.152:0/1840735787 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56241f8933f0 con 0x56241f8884a0 2024-09-11T23:00:58.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.122+0000 7f3c277fe640 1 --2- 172.21.15.152:0/1840735787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f888dc0 0x56241f8aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:00:58.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.123+0000 7f3c27fff640 1 --2- 172.21.15.152:0/1840735787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f8884a0 0x56241f89f4b0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3c200116f0 tx=0x7f3c20005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ced8ada6aba5f2c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:58.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.123+0000 7f3c0d7fa640 1 -- 172.21.15.152:0/1840735787 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2003c070 con 0x56241f8884a0 2024-09-11T23:00:58.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.123+0000 7f3c0d7fa640 1 -- 172.21.15.152:0/1840735787 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c20029040 con 0x56241f8884a0 2024-09-11T23:00:58.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.123+0000 7f3c0d7fa640 1 -- 172.21.15.152:0/1840735787 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000f850 con 0x56241f8884a0 2024-09-11T23:00:58.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.123+0000 7f3c30548740 1 -- 172.21.15.152:0/1840735787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f8884a0 msgr2=0x56241f89f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:58.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.123+0000 7f3c30548740 1 --2- 172.21.15.152:0/1840735787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f8884a0 0x56241f89f4b0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3c200116f0 tx=0x7f3c20005240 comp rx=0 tx=0).stop 2024-09-11T23:00:58.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.123+0000 7f3c30548740 1 -- 172.21.15.152:0/1840735787 shutdown_connections 2024-09-11T23:00:58.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.123+0000 7f3c30548740 1 --2- 172.21.15.152:0/1840735787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f893680 0x56241f8ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.123+0000 7f3c30548740 1 --2- 172.21.15.152:0/1840735787 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f888dc0 0x56241f8aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.123+0000 7f3c30548740 1 --2- 172.21.15.152:0/1840735787 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f8884a0 0x56241f89f4b0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.123+0000 7f3c30548740 1 -- 172.21.15.152:0/1840735787 >> 172.21.15.152:0/1840735787 conn(0x56241f884a30 msgr2=0x56241f88b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:58.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.123+0000 7f3c30548740 1 -- 172.21.15.152:0/1840735787 shutdown_connections 2024-09-11T23:00:58.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.123+0000 7f3c30548740 1 -- 172.21.15.152:0/1840735787 wait complete. 2024-09-11T23:00:58.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.124+0000 7f3c30548740 1 -- start start 2024-09-11T23:00:58.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.124+0000 7f3c30548740 1 Processor -- start 2024-09-11T23:00:58.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.124+0000 7f3c30548740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f8884a0 0x56241f8965e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.124+0000 7f3c30548740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f888dc0 0x56241f896b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.124+0000 7f3c30548740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f893680 0x56241f897060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.124+0000 7f3c30548740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281aecf0 con 0x56241f888dc0 2024-09-11T23:00:58.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.124+0000 7f3c30548740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281ae620 con 0x56241f893680 2024-09-11T23:00:58.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.124+0000 7f3c30548740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f3c281ab880 con 0x56241f8884a0 2024-09-11T23:00:58.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.124+0000 7f3c30548740 10 client.0 _ll_register_callbacks cb 0x7f3c200115d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:00:58.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.124+0000 7f3c277fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f888dc0 0x56241f896b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.124+0000 7f3c277fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f888dc0 0x56241f896b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:33622/0 (socket says 172.21.15.152:33622) 2024-09-11T23:00:58.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.124+0000 7f3c277fe640 1 -- 172.21.15.152:0/2411250053 learned_addr learned my addr 172.21.15.152:0/2411250053 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:58.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.124+0000 7f3c27fff640 1 --2- 172.21.15.152:0/2411250053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f8884a0 0x56241f8965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.124+0000 7f3c27fff640 1 --2- 172.21.15.152:0/2411250053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f893680 0x56241f897060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c277fe640 1 -- 172.21.15.152:0/2411250053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f8884a0 msgr2=0x56241f8965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:58.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c277fe640 1 --2- 172.21.15.152:0/2411250053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f8884a0 0x56241f8965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c277fe640 1 -- 172.21.15.152:0/2411250053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f893680 msgr2=0x56241f897060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:00:58.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c277fe640 1 --2- 172.21.15.152:0/2411250053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f893680 0x56241f897060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c277fe640 1 -- 172.21.15.152:0/2411250053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56241f90bba0 con 0x56241f888dc0 2024-09-11T23:00:58.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c27fff640 1 --2- 172.21.15.152:0/2411250053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f8884a0 0x56241f8965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:00:58.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c277fe640 1 --2- 172.21.15.152:0/2411250053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f888dc0 0x56241f896b20 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009900 tx=0x7f3c180318b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60b0c5bc6876e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:58.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:00:58.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c06ffd640 1 -- 172.21.15.152:0/2411250053 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1804c030 con 0x56241f888dc0 2024-09-11T23:00:58.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c06ffd640 1 -- 172.21.15.152:0/2411250053 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f3c18004250 con 0x56241f888dc0 2024-09-11T23:00:58.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c30548740 1 -- 172.21.15.152:0/2411250053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c20029040 con 0x56241f888dc0 2024-09-11T23:00:58.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c06ffd640 1 -- 172.21.15.152:0/2411250053 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3c18056050 con 0x56241f888dc0 2024-09-11T23:00:58.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c30548740 10 client.17706 fetch_fsmap learned FSMap version 28 2024-09-11T23:00:58.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c30548740 1 -- 172.21.15.152:0/2411250053 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56241f90cdf0 con 0x56241f888dc0 2024-09-11T23:00:58.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c06ffd640 1 -- 172.21.15.152:0/2411250053 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c20029040 con 0x56241f888dc0 2024-09-11T23:00:58.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c06ffd640 1 -- 172.21.15.152:0/2411250053 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3c18047540 con 0x56241f888dc0 2024-09-11T23:00:58.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c30548740 10 client.17706 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:00:58.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c30548740 10 client.17706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:00:58.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c30548740 20 client.17706 populate_metadata read hostname 'smithi152' 2024-09-11T23:00:58.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c30548740 1 --2- 172.21.15.152:0/2411250053 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56241f90d5a0 0x56241f90f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:58.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c30548740 4 client.17706 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:00:58.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.125+0000 7f3c30548740 1 -- 172.21.15.152:0/2411250053 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56241f7f5d60 con 0x56241f90d5a0 2024-09-11T23:00:58.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.126+0000 7f3c27fff640 1 --2- 172.21.15.152:0/2411250053 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56241f90d5a0 0x56241f90f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:58.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.126+0000 7f3c27fff640 1 --2- 172.21.15.152:0/2411250053 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56241f90d5a0 0x56241f90f9c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ed35d86120bcec server_cookie=1f054974ac1e7c4b in_seq=0 out_seq=0 2024-09-11T23:00:58.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.126+0000 7f3c06ffd640 10 client.17706 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:58.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.126+0000 7f3c06ffd640 1 -- 172.21.15.152:0/2411250053 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56241f7f5d60 con 0x56241f90d5a0 2024-09-11T23:00:58.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.126+0000 7f3c06ffd640 10 client.17706 handle_command_reply: tid=0 2024-09-11T23:00:58.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.126+0000 7f3c30548740 1 client.17706 shutdown 2024-09-11T23:00:58.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.126+0000 7f3c30548740 1 -- 172.21.15.152:0/2411250053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f888dc0 msgr2=0x56241f896b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:58.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.126+0000 7f3c30548740 1 --2- 172.21.15.152:0/2411250053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f888dc0 0x56241f896b20 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009900 tx=0x7f3c180318b0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.126+0000 7f3c30548740 1 -- 172.21.15.152:0/2411250053 shutdown_connections 2024-09-11T23:00:58.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.126+0000 7f3c30548740 1 --2- 172.21.15.152:0/2411250053 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x56241f90d5a0 0x56241f90f9c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.126+0000 7f3c30548740 1 --2- 172.21.15.152:0/2411250053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56241f893680 0x56241f897060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.126+0000 7f3c30548740 1 --2- 172.21.15.152:0/2411250053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56241f888dc0 0x56241f896b20 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.126+0000 7f3c06ffd640 0 client.17706 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:00:58.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.126+0000 7f3c30548740 1 --2- 172.21.15.152:0/2411250053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56241f8884a0 0x56241f8965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.126+0000 7f3c30548740 1 -- 172.21.15.152:0/2411250053 >> 172.21.15.152:0/2411250053 conn(0x56241f884a30 msgr2=0x56241f889ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:58.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.127+0000 7f3c30548740 1 -- 172.21.15.152:0/2411250053 shutdown_connections 2024-09-11T23:00:58.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.127+0000 7f3c30548740 1 -- 172.21.15.152:0/2411250053 wait complete. 2024-09-11T23:00:58.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.127+0000 7f3c30548740 20 client.17706 trim_cache size 0 max 16384 2024-09-11T23:00:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:00:58.167 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T23:00:58.167 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T23:00:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:00:58.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.127+0000 7f3c2e16f640 1 -- 172.21.15.152:0/2799404854 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3c100a2cf0 msgr2=0x7f3c100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:58.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.127+0000 7f3c2e16f640 1 --2- 172.21.15.152:0/2799404854 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3c100a2cf0 0x7f3c100a51b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c097fd0 tx=0x7f3c1c005f90 comp rx=0 tx=0).stop 2024-09-11T23:00:58.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.127+0000 7f3c2e16f640 1 -- 172.21.15.152:0/2799404854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c28102780 msgr2=0x7f3c281989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:58.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.127+0000 7f3c2e16f640 1 --2- 172.21.15.152:0/2799404854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c28102780 0x7f3c281989a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00d950 tx=0x7f3c1c00de10 comp rx=0 tx=0).stop 2024-09-11T23:00:58.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.127+0000 7f3c2e16f640 1 -- 172.21.15.152:0/2799404854 shutdown_connections 2024-09-11T23:00:58.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.127+0000 7f3c2e16f640 1 --2- 172.21.15.152:0/2799404854 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f3c100a2cf0 0x7f3c100a51b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.127+0000 7f3c2e16f640 1 --2- 172.21.15.152:0/2799404854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c28106f00 0x7f3c2819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.127+0000 7f3c2e16f640 1 --2- 172.21.15.152:0/2799404854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c28102780 0x7f3c281989a0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.127+0000 7f3c2e16f640 1 --2- 172.21.15.152:0/2799404854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c28101dd0 0x7f3c28198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:58.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.127+0000 7f3c2e16f640 1 -- 172.21.15.152:0/2799404854 >> 172.21.15.152:0/2799404854 conn(0x7f3c280fd890 msgr2=0x7f3c280fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:58.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.127+0000 7f3c2e16f640 1 -- 172.21.15.152:0/2799404854 shutdown_connections 2024-09-11T23:00:58.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:58.127+0000 7f3c2e16f640 1 -- 172.21.15.152:0/2799404854 wait complete. 2024-09-11T23:00:58.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:58 smithi152 ceph-mon[31181]: pgmap v247: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:58.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:00:58 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T23:00:58.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:58 smithi191 ceph-mon[35211]: pgmap v247: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:58.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:00:58 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T23:00:58.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:58 smithi204 ceph-mon[35454]: pgmap v247: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:00:58.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:00:58 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T23:00:58.856 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:00:58.856 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:00:59.143 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:00:59.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.684+0000 7fe810ce7640 1 -- 172.21.15.152:0/77456679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe80c101420 msgr2=0x7fe80c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:59.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.684+0000 7fe810ce7640 1 --2- 172.21.15.152:0/77456679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe80c101420 0x7fe80c10d950 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe804009cd0 tx=0x7fe80402a8e0 comp rx=0 tx=0).stop 2024-09-11T23:00:59.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.684+0000 7fe810ce7640 1 -- 172.21.15.152:0/77456679 shutdown_connections 2024-09-11T23:00:59.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.684+0000 7fe810ce7640 1 --2- 172.21.15.152:0/77456679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe80c101420 0x7fe80c10d950 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:59.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.684+0000 7fe810ce7640 1 --2- 172.21.15.152:0/77456679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe80c100a80 0x7fe80c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:59.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.684+0000 7fe810ce7640 1 --2- 172.21.15.152:0/77456679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe80c106a80 0x7fe80c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:59.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.684+0000 7fe810ce7640 1 -- 172.21.15.152:0/77456679 >> 172.21.15.152:0/77456679 conn(0x7fe80c0fc7a0 msgr2=0x7fe80c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:59.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.684+0000 7fe810ce7640 1 -- 172.21.15.152:0/77456679 shutdown_connections 2024-09-11T23:00:59.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.684+0000 7fe810ce7640 1 -- 172.21.15.152:0/77456679 wait complete. 2024-09-11T23:00:59.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.684+0000 7fe810ce7640 1 Processor -- start 2024-09-11T23:00:59.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.684+0000 7fe810ce7640 1 -- start start 2024-09-11T23:00:59.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe810ce7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe80c100a80 0x7fe80c0fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:59.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe810ce7640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe80c101420 0x7fe80c0fd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:59.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe810ce7640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe80c106a80 0x7fe80c0fd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:59.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe810ce7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c110ea0 con 0x7fe80c100a80 2024-09-11T23:00:59.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe810ce7640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c110d20 con 0x7fe80c101420 2024-09-11T23:00:59.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe810ce7640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c111020 con 0x7fe80c106a80 2024-09-11T23:00:59.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe80a59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe80c100a80 0x7fe80c0fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:59.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe80a59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe80c100a80 0x7fe80c0fcdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43360/0 (socket says 172.21.15.152:43360) 2024-09-11T23:00:59.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe80a59c640 1 -- 172.21.15.152:0/2295838437 learned_addr learned my addr 172.21.15.152:0/2295838437 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:00:59.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe803fff640 1 --2- 172.21.15.152:0/2295838437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe80c101420 0x7fe80c0fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:59.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe80ad9d640 1 --2- 172.21.15.152:0/2295838437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe80c106a80 0x7fe80c0fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:59.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe80a59c640 1 -- 172.21.15.152:0/2295838437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe80c106a80 msgr2=0x7fe80c0fd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:59.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe80a59c640 1 --2- 172.21.15.152:0/2295838437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe80c106a80 0x7fe80c0fd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:59.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe80a59c640 1 -- 172.21.15.152:0/2295838437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe80c101420 msgr2=0x7fe80c0fd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:59.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe80a59c640 1 --2- 172.21.15.152:0/2295838437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe80c101420 0x7fe80c0fd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:59.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe80a59c640 1 -- 172.21.15.152:0/2295838437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c1a69c0 con 0x7fe80c100a80 2024-09-11T23:00:59.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe80a59c640 1 --2- 172.21.15.152:0/2295838437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe80c100a80 0x7fe80c0fcdc0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00d950 tx=0x7fe7fc00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:59.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.685+0000 7fe8037fe640 1 -- 172.21.15.152:0/2295838437 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc017070 con 0x7fe80c100a80 2024-09-11T23:00:59.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.686+0000 7fe8037fe640 1 -- 172.21.15.152:0/2295838437 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7fc00de80 con 0x7fe80c100a80 2024-09-11T23:00:59.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.686+0000 7fe8037fe640 1 -- 172.21.15.152:0/2295838437 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc00f620 con 0x7fe80c100a80 2024-09-11T23:00:59.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.686+0000 7fe810ce7640 1 -- 172.21.15.152:0/2295838437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80c1a6cb0 con 0x7fe80c100a80 2024-09-11T23:00:59.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.686+0000 7fe810ce7640 1 -- 172.21.15.152:0/2295838437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80c1a71a0 con 0x7fe80c100a80 2024-09-11T23:00:59.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.687+0000 7fe810ce7640 1 -- 172.21.15.152:0/2295838437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80c100ee0 con 0x7fe80c100a80 2024-09-11T23:00:59.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.688+0000 7fe8037fe640 1 -- 172.21.15.152:0/2295838437 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe7fc009c30 con 0x7fe80c100a80 2024-09-11T23:00:59.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.688+0000 7fe8037fe640 1 --2- 172.21.15.152:0/2295838437 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe7f80a2cf0 0x7fe7f80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:00:59.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.688+0000 7fe803fff640 1 --2- 172.21.15.152:0/2295838437 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe7f80a2cf0 0x7fe7f80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:00:59.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.688+0000 7fe8037fe640 1 -- 172.21.15.152:0/2295838437 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7fe7fc0cf0a0 con 0x7fe80c100a80 2024-09-11T23:00:59.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.689+0000 7fe803fff640 1 --2- 172.21.15.152:0/2295838437 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe7f80a2cf0 0x7fe7f80a51b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0fe850 tx=0x7fe7f4009110 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:00:59.729 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.692+0000 7fe8037fe640 1 -- 172.21.15.152:0/2295838437 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7fc09bce0 con 0x7fe80c100a80 2024-09-11T23:00:59.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.891+0000 7fe810ce7640 1 -- 172.21.15.152:0/2295838437 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe80c10a020 con 0x7fe80c100a80 2024-09-11T23:00:59.892 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.892+0000 7fe8037fe640 1 -- 172.21.15.152:0/2295838437 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7fe7fc09b5e0 con 0x7fe80c100a80 2024-09-11T23:00:59.893 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T23:00:59.893 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T23:00:59.893 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T23:00:59.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.894+0000 7fe810ce7640 1 -- 172.21.15.152:0/2295838437 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe7f80a2cf0 msgr2=0x7fe7f80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:59.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.894+0000 7fe810ce7640 1 --2- 172.21.15.152:0/2295838437 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe7f80a2cf0 0x7fe7f80a51b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0fe850 tx=0x7fe7f4009110 comp rx=0 tx=0).stop 2024-09-11T23:00:59.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.894+0000 7fe810ce7640 1 -- 172.21.15.152:0/2295838437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe80c100a80 msgr2=0x7fe80c0fcdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:00:59.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.894+0000 7fe810ce7640 1 --2- 172.21.15.152:0/2295838437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe80c100a80 0x7fe80c0fcdc0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00d950 tx=0x7fe7fc00dd50 comp rx=0 tx=0).stop 2024-09-11T23:00:59.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.895+0000 7fe810ce7640 1 -- 172.21.15.152:0/2295838437 shutdown_connections 2024-09-11T23:00:59.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.895+0000 7fe810ce7640 1 --2- 172.21.15.152:0/2295838437 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe7f80a2cf0 0x7fe7f80a51b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:59.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.895+0000 7fe810ce7640 1 --2- 172.21.15.152:0/2295838437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe80c106a80 0x7fe80c0fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:59.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.895+0000 7fe810ce7640 1 --2- 172.21.15.152:0/2295838437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe80c101420 0x7fe80c0fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:59.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.895+0000 7fe810ce7640 1 --2- 172.21.15.152:0/2295838437 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe80c100a80 0x7fe80c0fcdc0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:00:59.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.895+0000 7fe810ce7640 1 -- 172.21.15.152:0/2295838437 >> 172.21.15.152:0/2295838437 conn(0x7fe80c0fc7a0 msgr2=0x7fe80c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:00:59.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.895+0000 7fe810ce7640 1 -- 172.21.15.152:0/2295838437 shutdown_connections 2024-09-11T23:00:59.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:00:59.895+0000 7fe810ce7640 1 -- 172.21.15.152:0/2295838437 wait complete. 2024-09-11T23:01:00.593 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:00 smithi152 ceph-mon[31181]: pgmap v248: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 13 op/s 2024-09-11T23:01:00.593 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:00 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2295838437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:01:00.621 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:01:00.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:00 smithi191 ceph-mon[35211]: pgmap v248: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 13 op/s 2024-09-11T23:01:00.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:00 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2295838437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:01:00.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:00 smithi204 ceph-mon[35454]: pgmap v248: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 13 op/s 2024-09-11T23:01:00.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:00 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2295838437' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:01:00.867 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:01:01.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.378+0000 7f61970c5640 1 -- 172.21.15.152:0/305169341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6190102570 msgr2=0x7f61901029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:01.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.378+0000 7f61970c5640 1 --2- 172.21.15.152:0/305169341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6190102570 0x7f61901029b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f617c009900 tx=0x7f617c02a480 comp rx=0 tx=0).stop 2024-09-11T23:01:01.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.378+0000 7f61970c5640 1 -- 172.21.15.152:0/305169341 shutdown_connections 2024-09-11T23:01:01.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.378+0000 7f61970c5640 1 --2- 172.21.15.152:0/305169341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6190102ef0 0x7f619010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.378+0000 7f61970c5640 1 --2- 172.21.15.152:0/305169341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6190102570 0x7f61901029b0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.378+0000 7f61970c5640 1 --2- 172.21.15.152:0/305169341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6190100e20 0x7f6190101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.378+0000 7f61970c5640 1 -- 172.21.15.152:0/305169341 >> 172.21.15.152:0/305169341 conn(0x7f61900fc7d0 msgr2=0x7f61900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:01.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.378+0000 7f61970c5640 1 -- 172.21.15.152:0/305169341 shutdown_connections 2024-09-11T23:01:01.407 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.378+0000 7f61970c5640 1 -- 172.21.15.152:0/305169341 wait complete. 2024-09-11T23:01:01.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f61970c5640 1 Processor -- start 2024-09-11T23:01:01.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f61970c5640 1 -- start start 2024-09-11T23:01:01.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f61970c5640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6190100e20 0x7f619019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f61970c5640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6190102570 0x7f619019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f61970c5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6190102ef0 0x7f619019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f61970c5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6190110f50 con 0x7f6190102ef0 2024-09-11T23:01:01.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f61970c5640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6190110dd0 con 0x7f6190102570 2024-09-11T23:01:01.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f61970c5640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f61901110d0 con 0x7f6190100e20 2024-09-11T23:01:01.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f6195662640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6190102ef0 0x7f619019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:01.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f6195662640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6190102ef0 0x7f619019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43388/0 (socket says 172.21.15.152:43388) 2024-09-11T23:01:01.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f6195662640 1 -- 172.21.15.152:0/3166992163 learned_addr learned my addr 172.21.15.152:0/3166992163 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:01.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f618ffff640 1 --2- 172.21.15.152:0/3166992163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6190102570 0x7f619019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:01.408 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f6194e61640 1 --2- 172.21.15.152:0/3166992163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6190100e20 0x7f619019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:01.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f6195662640 1 -- 172.21.15.152:0/3166992163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6190100e20 msgr2=0x7f619019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:01.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f6195662640 1 --2- 172.21.15.152:0/3166992163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6190100e20 0x7f619019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f6195662640 1 -- 172.21.15.152:0/3166992163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6190102570 msgr2=0x7f619019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:01.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f6195662640 1 --2- 172.21.15.152:0/3166992163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6190102570 0x7f619019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f6195662640 1 -- 172.21.15.152:0/3166992163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619019f690 con 0x7f6190102ef0 2024-09-11T23:01:01.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f6194e61640 1 --2- 172.21.15.152:0/3166992163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6190100e20 0x7f619019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:01:01.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f618ffff640 1 --2- 172.21.15.152:0/3166992163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6190102570 0x7f619019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:01:01.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.379+0000 7f6195662640 1 --2- 172.21.15.152:0/3166992163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6190102ef0 0x7f619019efb0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f618800eaa0 tx=0x7f618800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:01.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.380+0000 7f618dffb640 1 -- 172.21.15.152:0/3166992163 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618800cdb0 con 0x7f6190102ef0 2024-09-11T23:01:01.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.380+0000 7f618dffb640 1 -- 172.21.15.152:0/3166992163 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6188005020 con 0x7f6190102ef0 2024-09-11T23:01:01.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.380+0000 7f61970c5640 1 -- 172.21.15.152:0/3166992163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619019f980 con 0x7f6190102ef0 2024-09-11T23:01:01.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.380+0000 7f618dffb640 1 -- 172.21.15.152:0/3166992163 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61880116b0 con 0x7f6190102ef0 2024-09-11T23:01:01.409 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.380+0000 7f61970c5640 1 -- 172.21.15.152:0/3166992163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61901a7260 con 0x7f6190102ef0 2024-09-11T23:01:01.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.383+0000 7f618dffb640 1 -- 172.21.15.152:0/3166992163 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6188005560 con 0x7f6190102ef0 2024-09-11T23:01:01.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.383+0000 7f619949e740 1 Processor -- start 2024-09-11T23:01:01.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.383+0000 7f619949e740 1 -- start start 2024-09-11T23:01:01.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.383+0000 7f619949e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e233b40 0x558a6e232090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.383+0000 7f619949e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e2325d0 0x558a6e232a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.383+0000 7f619949e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e23f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.383+0000 7f619949e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558a6e1982f0 con 0x558a6e23d370 2024-09-11T23:01:01.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.383+0000 7f619949e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558a6e13fcf0 con 0x558a6e233b40 2024-09-11T23:01:01.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.383+0000 7f619949e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558a6e1942b0 con 0x558a6e2325d0 2024-09-11T23:01:01.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.383+0000 7f618dffb640 1 --2- 172.21.15.152:0/3166992163 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f61800a2cf0 0x7f61800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.383+0000 7f618ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e23f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:01.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.383+0000 7f618ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e23f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43400/0 (socket says 172.21.15.152:43400) 2024-09-11T23:01:01.410 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.383+0000 7f618ffff640 1 -- 172.21.15.152:0/1161140496 learned_addr learned my addr 172.21.15.152:0/1161140496 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:01.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.383+0000 7f6195662640 1 --2- 172.21.15.152:0/3166992163 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f61800a2cf0 0x7f61800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:01.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.383+0000 7f6194e61640 1 --2- 172.21.15.152:0/1161140496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e2325d0 0x558a6e232a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:01.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.383+0000 7f618dffb640 1 -- 172.21.15.152:0/3166992163 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f61880cf760 con 0x7f6190102ef0 2024-09-11T23:01:01.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.384+0000 7f618ffff640 1 --2- 172.21.15.152:0/1161140496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e233b40 0x558a6e232090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:01.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.384+0000 7f618ffff640 1 -- 172.21.15.152:0/1161140496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e2325d0 msgr2=0x558a6e232a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:01.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.384+0000 7f618ffff640 1 --2- 172.21.15.152:0/1161140496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e2325d0 0x558a6e232a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.384+0000 7f618ffff640 1 -- 172.21.15.152:0/1161140496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e233b40 msgr2=0x558a6e232090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:01.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.384+0000 7f618ffff640 1 --2- 172.21.15.152:0/1161140496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e233b40 0x558a6e232090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.384+0000 7f618ffff640 1 -- 172.21.15.152:0/1161140496 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a6e1243a0 con 0x558a6e23d370 2024-09-11T23:01:01.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.384+0000 7f6195662640 1 --2- 172.21.15.152:0/3166992163 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f61800a2cf0 0x7f61800a51b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6188009270 tx=0x7f618806bb00 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:01.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.384+0000 7f618ffff640 1 --2- 172.21.15.152:0/1161140496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e23f760 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f617c005770 tx=0x7f617c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd97310dabd22e4e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:01.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.384+0000 7f617affd640 1 -- 172.21.15.152:0/1161140496 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c043020 con 0x558a6e23d370 2024-09-11T23:01:01.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.384+0000 7f617affd640 1 -- 172.21.15.152:0/1161140496 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f617c0046c0 con 0x558a6e23d370 2024-09-11T23:01:01.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.384+0000 7f617affd640 1 -- 172.21.15.152:0/1161140496 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c035040 con 0x558a6e23d370 2024-09-11T23:01:01.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.385+0000 7f619949e740 1 -- 172.21.15.152:0/1161140496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 msgr2=0x558a6e23f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:01.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.385+0000 7f619949e740 1 --2- 172.21.15.152:0/1161140496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e23f760 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f617c005770 tx=0x7f617c005b50 comp rx=0 tx=0).stop 2024-09-11T23:01:01.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.385+0000 7f619949e740 1 -- 172.21.15.152:0/1161140496 shutdown_connections 2024-09-11T23:01:01.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.385+0000 7f619949e740 1 --2- 172.21.15.152:0/1161140496 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e23f760 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.385+0000 7f619949e740 1 --2- 172.21.15.152:0/1161140496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e2325d0 0x558a6e232a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.385+0000 7f619949e740 1 --2- 172.21.15.152:0/1161140496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e233b40 0x558a6e232090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.385+0000 7f619949e740 1 -- 172.21.15.152:0/1161140496 >> 172.21.15.152:0/1161140496 conn(0x558a6e1fb310 msgr2=0x558a6e22ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:01.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.385+0000 7f619949e740 1 -- 172.21.15.152:0/1161140496 shutdown_connections 2024-09-11T23:01:01.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.385+0000 7f619949e740 1 -- 172.21.15.152:0/1161140496 wait complete. 2024-09-11T23:01:01.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.385+0000 7f619949e740 1 -- start start 2024-09-11T23:01:01.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.385+0000 7f619949e740 1 Processor -- start 2024-09-11T23:01:01.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f619949e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e2325d0 0x558a6e23cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f619949e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e233b40 0x558a6e23a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f619949e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e23a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f619949e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558a6e240010 con 0x558a6e23d370 2024-09-11T23:01:01.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f619949e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558a6e153060 con 0x558a6e2325d0 2024-09-11T23:01:01.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f619949e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558a6e240190 con 0x558a6e233b40 2024-09-11T23:01:01.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f619949e740 10 client.0 _ll_register_callbacks cb 0x558a6dea68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:01.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f618ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e23a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:01.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f618ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e23a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43402/0 (socket says 172.21.15.152:43402) 2024-09-11T23:01:01.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f618ffff640 1 -- 172.21.15.152:0/129168478 learned_addr learned my addr 172.21.15.152:0/129168478 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:01.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f618ffff640 1 --2- 172.21.15.152:0/129168478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e2325d0 0x558a6e23cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:01.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f6194e61640 1 --2- 172.21.15.152:0/129168478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e233b40 0x558a6e23a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:01.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f618ffff640 1 -- 172.21.15.152:0/129168478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e233b40 msgr2=0x558a6e23a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:01.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f618ffff640 1 --2- 172.21.15.152:0/129168478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e233b40 0x558a6e23a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f618ffff640 1 -- 172.21.15.152:0/129168478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e2325d0 msgr2=0x558a6e23cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:01.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f618ffff640 1 --2- 172.21.15.152:0/129168478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e2325d0 0x558a6e23cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f618ffff640 1 -- 172.21.15.152:0/129168478 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a6df093c0 con 0x558a6e23d370 2024-09-11T23:01:01.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f618ffff640 1 --2- 172.21.15.152:0/129168478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e23a610 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f617c005e60 tx=0x7f617c005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b565ca8386276e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:01.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f616ffff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:01.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f616ffff640 1 -- 172.21.15.152:0/129168478 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c043070 con 0x558a6e23d370 2024-09-11T23:01:01.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f616ffff640 1 -- 172.21.15.152:0/129168478 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f617c003770 con 0x558a6e23d370 2024-09-11T23:01:01.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.386+0000 7f619949e740 1 -- 172.21.15.152:0/129168478 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f617c0046c0 con 0x558a6e23d370 2024-09-11T23:01:01.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.387+0000 7f616ffff640 1 -- 172.21.15.152:0/129168478 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f617c003970 con 0x558a6e23d370 2024-09-11T23:01:01.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.387+0000 7f619949e740 10 client.17724 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:01.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.387+0000 7f619949e740 1 -- 172.21.15.152:0/129168478 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a6e24bab0 con 0x558a6e23d370 2024-09-11T23:01:01.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.387+0000 7f616ffff640 1 -- 172.21.15.152:0/129168478 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f617c0046c0 con 0x558a6e23d370 2024-09-11T23:01:01.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.387+0000 7f616ffff640 1 -- 172.21.15.152:0/129168478 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f617c03a040 con 0x558a6e23d370 2024-09-11T23:01:01.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.387+0000 7f619949e740 10 client.17724 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:01.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.387+0000 7f619949e740 10 client.17724 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T23:01:01.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.387+0000 7f619949e740 20 client.17724 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:01.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.387+0000 7f619949e740 1 --2- 172.21.15.152:0/129168478 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x558a6e24ce00 0x558a6e24f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.387+0000 7f619949e740 4 client.17724 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:01:01.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.387+0000 7f619949e740 1 -- 172.21.15.152:0/129168478 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a6e1a5860 con 0x558a6e24ce00 2024-09-11T23:01:01.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.387+0000 7f6194e61640 1 --2- 172.21.15.152:0/129168478 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x558a6e24ce00 0x558a6e24f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:01.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.388+0000 7f616ffff640 10 client.17724 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T23:01:01.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.388+0000 7f6194e61640 1 --2- 172.21.15.152:0/129168478 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x558a6e24ce00 0x558a6e24f1f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2ad4dce5ce91fa12 server_cookie=d1d89fb8025cbaed in_seq=0 out_seq=0 2024-09-11T23:01:01.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.388+0000 7f616ffff640 1 -- 172.21.15.152:0/129168478 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a6e1a5860 con 0x558a6e24ce00 2024-09-11T23:01:01.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.388+0000 7f616ffff640 10 client.17724 handle_command_reply: tid=0 2024-09-11T23:01:01.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.388+0000 7f619949e740 1 client.17724 shutdown 2024-09-11T23:01:01.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.388+0000 7f619949e740 1 -- 172.21.15.152:0/129168478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 msgr2=0x558a6e23a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:01.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.388+0000 7f619949e740 1 --2- 172.21.15.152:0/129168478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e23a610 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f617c005e60 tx=0x7f617c005bc0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.388+0000 7f619949e740 1 -- 172.21.15.152:0/129168478 shutdown_connections 2024-09-11T23:01:01.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.388+0000 7f619949e740 1 --2- 172.21.15.152:0/129168478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e23a610 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.388+0000 7f619949e740 1 --2- 172.21.15.152:0/129168478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e233b40 0x558a6e23a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.388+0000 7f619949e740 1 --2- 172.21.15.152:0/129168478 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x558a6e24ce00 0x558a6e24f1f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.388+0000 7f619949e740 1 --2- 172.21.15.152:0/129168478 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e2325d0 0x558a6e23cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.388+0000 7f616ffff640 0 client.17724 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T23:01:01.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.388+0000 7f619949e740 1 -- 172.21.15.152:0/129168478 >> 172.21.15.152:0/129168478 conn(0x558a6e1fb310 msgr2=0x558a6e234960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:01.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.388+0000 7f619949e740 1 -- 172.21.15.152:0/129168478 shutdown_connections 2024-09-11T23:01:01.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.388+0000 7f619949e740 1 -- 172.21.15.152:0/129168478 wait complete. 2024-09-11T23:01:01.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.389+0000 7f619949e740 20 client.17724 trim_cache size 0 max 16384 2024-09-11T23:01:01.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.401+0000 7f619949e740 1 Processor -- start 2024-09-11T23:01:01.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.401+0000 7f619949e740 1 -- start start 2024-09-11T23:01:01.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.401+0000 7f619949e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e2325d0 0x558a6e235120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.401+0000 7f619949e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e233b40 0x558a6e235660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.401+0000 7f619949e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e23d370 0x558a6e252210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.401+0000 7f619949e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f61901af920 con 0x558a6e233b40 2024-09-11T23:01:01.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.401+0000 7f619949e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f61901b1460 con 0x558a6e23d370 2024-09-11T23:01:01.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.401+0000 7f619949e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f61901afff0 con 0x558a6e2325d0 2024-09-11T23:01:01.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.401+0000 7f6194e61640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e233b40 0x558a6e235660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:01.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.401+0000 7f6194e61640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e233b40 0x558a6e235660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43406/0 (socket says 172.21.15.152:43406) 2024-09-11T23:01:01.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.401+0000 7f6194e61640 1 -- 172.21.15.152:0/1939178015 learned_addr learned my addr 172.21.15.152:0/1939178015 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:01.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.402+0000 7f618ffff640 1 --2- 172.21.15.152:0/1939178015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e23d370 0x558a6e252210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:01.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.402+0000 7f6194e61640 1 -- 172.21.15.152:0/1939178015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e2325d0 msgr2=0x558a6e235120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:01.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.402+0000 7f6194e61640 1 --2- 172.21.15.152:0/1939178015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e2325d0 0x558a6e235120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.402+0000 7f6194e61640 1 -- 172.21.15.152:0/1939178015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e23d370 msgr2=0x558a6e252210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:01.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.402+0000 7f6194e61640 1 --2- 172.21.15.152:0/1939178015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e23d370 0x558a6e252210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.402+0000 7f6194e61640 1 -- 172.21.15.152:0/1939178015 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a6e22fd60 con 0x558a6e233b40 2024-09-11T23:01:01.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.402+0000 7f6194e61640 1 --2- 172.21.15.152:0/1939178015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e233b40 0x558a6e235660 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x558a6e2398b0 tx=0x7f618400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2730b895428a4591 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:01.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.402+0000 7f617a7fc640 1 -- 172.21.15.152:0/1939178015 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618401b620 con 0x558a6e233b40 2024-09-11T23:01:01.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.402+0000 7f617a7fc640 1 -- 172.21.15.152:0/1939178015 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f618401bca0 con 0x558a6e233b40 2024-09-11T23:01:01.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.402+0000 7f617a7fc640 1 -- 172.21.15.152:0/1939178015 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618401ad30 con 0x558a6e233b40 2024-09-11T23:01:01.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.402+0000 7f619949e740 1 -- 172.21.15.152:0/1939178015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e233b40 msgr2=0x558a6e235660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:01.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.402+0000 7f619949e740 1 --2- 172.21.15.152:0/1939178015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e233b40 0x558a6e235660 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x558a6e2398b0 tx=0x7f618400f7f0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.403+0000 7f619949e740 1 -- 172.21.15.152:0/1939178015 shutdown_connections 2024-09-11T23:01:01.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.403+0000 7f619949e740 1 --2- 172.21.15.152:0/1939178015 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e23d370 0x558a6e252210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.403+0000 7f619949e740 1 --2- 172.21.15.152:0/1939178015 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e233b40 0x558a6e235660 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.403+0000 7f619949e740 1 --2- 172.21.15.152:0/1939178015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e2325d0 0x558a6e235120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.403+0000 7f619949e740 1 -- 172.21.15.152:0/1939178015 >> 172.21.15.152:0/1939178015 conn(0x558a6e1fb310 msgr2=0x558a6e239130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.403+0000 7f619949e740 1 -- 172.21.15.152:0/1939178015 shutdown_connections 2024-09-11T23:01:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.403+0000 7f619949e740 1 -- 172.21.15.152:0/1939178015 wait complete. 2024-09-11T23:01:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.403+0000 7f619949e740 1 -- start start 2024-09-11T23:01:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.403+0000 7f619949e740 1 Processor -- start 2024-09-11T23:01:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.403+0000 7f619949e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e2325d0 0x558a6e252600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.403+0000 7f619949e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e233b40 0x558a6e252b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.403+0000 7f619949e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e24fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.403+0000 7f619949e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f61901b0d90 con 0x558a6e23d370 2024-09-11T23:01:01.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.403+0000 7f619949e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f61901b06c0 con 0x558a6e233b40 2024-09-11T23:01:01.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.403+0000 7f619949e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f61901ad920 con 0x558a6e2325d0 2024-09-11T23:01:01.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.403+0000 7f619949e740 10 client.0 _ll_register_callbacks cb 0x7f618400b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:01.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f618ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e24fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:01.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f618ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e24fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43418/0 (socket says 172.21.15.152:43418) 2024-09-11T23:01:01.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f618ffff640 1 -- 172.21.15.152:0/1134650316 learned_addr learned my addr 172.21.15.152:0/1134650316 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:01.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f6194e61640 1 --2- 172.21.15.152:0/1134650316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e233b40 0x558a6e252b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:01.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f618ffff640 1 -- 172.21.15.152:0/1134650316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e2325d0 msgr2=0x558a6e252600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:01.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f618ffff640 1 --2- 172.21.15.152:0/1134650316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e2325d0 0x558a6e252600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f618ffff640 1 -- 172.21.15.152:0/1134650316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e233b40 msgr2=0x558a6e252b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:01.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f618ffff640 1 --2- 172.21.15.152:0/1134650316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e233b40 0x558a6e252b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f618ffff640 1 -- 172.21.15.152:0/1134650316 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a6e2503d0 con 0x558a6e23d370 2024-09-11T23:01:01.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f6194e61640 1 --2- 172.21.15.152:0/1134650316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e233b40 0x558a6e252b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:01:01.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f618ffff640 1 --2- 172.21.15.152:0/1134650316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e24fba0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f617c0059d0 tx=0x7f617c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5486cf7b6be8b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:01.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f616ffff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:01.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f616ffff640 1 -- 172.21.15.152:0/1134650316 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c043070 con 0x558a6e23d370 2024-09-11T23:01:01.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f616ffff640 1 -- 172.21.15.152:0/1134650316 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f617c0048d0 con 0x558a6e23d370 2024-09-11T23:01:01.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f619949e740 1 -- 172.21.15.152:0/1134650316 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f618401bca0 con 0x558a6e23d370 2024-09-11T23:01:01.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f616ffff640 1 -- 172.21.15.152:0/1134650316 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f617c035040 con 0x558a6e23d370 2024-09-11T23:01:01.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f619949e740 10 client.17730 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:01.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.404+0000 7f619949e740 1 -- 172.21.15.152:0/1134650316 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a6e2b73f0 con 0x558a6e23d370 2024-09-11T23:01:01.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.405+0000 7f616ffff640 1 -- 172.21.15.152:0/1134650316 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f618401bca0 con 0x558a6e23d370 2024-09-11T23:01:01.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.405+0000 7f616ffff640 1 -- 172.21.15.152:0/1134650316 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f617c065070 con 0x558a6e23d370 2024-09-11T23:01:01.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.405+0000 7f619949e740 10 client.17730 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:01.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.405+0000 7f619949e740 10 client.17730 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T23:01:01.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.405+0000 7f619949e740 20 client.17730 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:01.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.405+0000 7f619949e740 1 --2- 172.21.15.152:0/1134650316 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x558a6e2b78c0 0x558a6e2b9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:01.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.405+0000 7f619949e740 4 client.17730 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:01:01.422 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.405+0000 7f619949e740 1 -- 172.21.15.152:0/1134650316 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a6e19fd70 con 0x558a6e2b78c0 2024-09-11T23:01:01.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.405+0000 7f6194e61640 1 --2- 172.21.15.152:0/1134650316 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x558a6e2b78c0 0x558a6e2b9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:01.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.405+0000 7f616ffff640 10 client.17730 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T23:01:01.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.405+0000 7f6194e61640 1 --2- 172.21.15.152:0/1134650316 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x558a6e2b78c0 0x558a6e2b9cb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=85280625f8b98d1d server_cookie=5a9cfa14855a2af8 in_seq=0 out_seq=0 2024-09-11T23:01:01.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.405+0000 7f616ffff640 1 -- 172.21.15.152:0/1134650316 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a6e19fd70 con 0x558a6e2b78c0 2024-09-11T23:01:01.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.405+0000 7f616ffff640 10 client.17730 handle_command_reply: tid=0 2024-09-11T23:01:01.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.406+0000 7f619949e740 1 client.17730 shutdown 2024-09-11T23:01:01.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.406+0000 7f619949e740 1 -- 172.21.15.152:0/1134650316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 msgr2=0x558a6e24fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:01.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.406+0000 7f619949e740 1 --2- 172.21.15.152:0/1134650316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e24fba0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f617c0059d0 tx=0x7f617c02a480 comp rx=0 tx=0).stop 2024-09-11T23:01:01.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.406+0000 7f619949e740 1 -- 172.21.15.152:0/1134650316 shutdown_connections 2024-09-11T23:01:01.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.406+0000 7f619949e740 1 --2- 172.21.15.152:0/1134650316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558a6e23d370 0x558a6e24fba0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.423 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.406+0000 7f619949e740 1 --2- 172.21.15.152:0/1134650316 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x558a6e2b78c0 0x558a6e2b9cb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.406+0000 7f619949e740 1 --2- 172.21.15.152:0/1134650316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558a6e233b40 0x558a6e252b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.406+0000 7f619949e740 1 --2- 172.21.15.152:0/1134650316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558a6e2325d0 0x558a6e252600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.406+0000 7f616ffff640 0 client.17730 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T23:01:01.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.406+0000 7f619949e740 1 -- 172.21.15.152:0/1134650316 >> 172.21.15.152:0/1134650316 conn(0x558a6e1fb310 msgr2=0x558a6e23b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:01.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.406+0000 7f619949e740 1 -- 172.21.15.152:0/1134650316 shutdown_connections 2024-09-11T23:01:01.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.406+0000 7f619949e740 1 -- 172.21.15.152:0/1134650316 wait complete. 2024-09-11T23:01:01.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.406+0000 7f619949e740 20 client.17730 trim_cache size 0 max 16384 2024-09-11T23:01:01.425 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:01:01.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.406+0000 7f61970c5640 1 -- 172.21.15.152:0/3166992163 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f61800a2cf0 msgr2=0x7f61800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:01.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.407+0000 7f61970c5640 1 --2- 172.21.15.152:0/3166992163 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f61800a2cf0 0x7f61800a51b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6188009270 tx=0x7f618806bb00 comp rx=0 tx=0).stop 2024-09-11T23:01:01.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.407+0000 7f61970c5640 1 -- 172.21.15.152:0/3166992163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6190102ef0 msgr2=0x7f619019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:01.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.407+0000 7f61970c5640 1 --2- 172.21.15.152:0/3166992163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6190102ef0 0x7f619019efb0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f618800eaa0 tx=0x7f618800ef60 comp rx=0 tx=0).stop 2024-09-11T23:01:01.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.407+0000 7f61970c5640 1 -- 172.21.15.152:0/3166992163 shutdown_connections 2024-09-11T23:01:01.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.407+0000 7f61970c5640 1 --2- 172.21.15.152:0/3166992163 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f61800a2cf0 0x7f61800a51b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.407+0000 7f61970c5640 1 --2- 172.21.15.152:0/3166992163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6190102ef0 0x7f619019efb0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.407+0000 7f61970c5640 1 --2- 172.21.15.152:0/3166992163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6190102570 0x7f619019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.407+0000 7f61970c5640 1 --2- 172.21.15.152:0/3166992163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6190100e20 0x7f619019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:01.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.407+0000 7f61970c5640 1 -- 172.21.15.152:0/3166992163 >> 172.21.15.152:0/3166992163 conn(0x7f61900fc7d0 msgr2=0x7f619006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:01.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.407+0000 7f61970c5640 1 -- 172.21.15.152:0/3166992163 shutdown_connections 2024-09-11T23:01:01.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:01.407+0000 7f61970c5640 1 -- 172.21.15.152:0/3166992163 wait complete. 2024-09-11T23:01:02.308 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:01:02.558 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:01:02.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:02 smithi152 ceph-mon[31181]: pgmap v249: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-09-11T23:01:02.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:02 smithi191 ceph-mon[35211]: pgmap v249: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-09-11T23:01:02.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:02 smithi204 ceph-mon[35454]: pgmap v249: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-09-11T23:01:03.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.115+0000 7fe55f5bc640 1 -- 172.21.15.152:0/3899975913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5581082a0 msgr2=0x7fe55810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:03.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.115+0000 7fe55f5bc640 1 --2- 172.21.15.152:0/3899975913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5581082a0 0x7fe55810ec40 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe550009930 tx=0x7fe55002a440 comp rx=0 tx=0).stop 2024-09-11T23:01:03.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.115+0000 7fe55f5bc640 1 -- 172.21.15.152:0/3899975913 shutdown_connections 2024-09-11T23:01:03.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.115+0000 7fe55f5bc640 1 --2- 172.21.15.152:0/3899975913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5581082a0 0x7fe55810ec40 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.115+0000 7fe55f5bc640 1 --2- 172.21.15.152:0/3899975913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5581036e0 0x7fe558107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.115+0000 7fe55f5bc640 1 --2- 172.21.15.152:0/3899975913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe558102d30 0x7fe558103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.115+0000 7fe55f5bc640 1 -- 172.21.15.152:0/3899975913 >> 172.21.15.152:0/3899975913 conn(0x7fe5580fc7d0 msgr2=0x7fe5580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:03.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.116+0000 7fe55f5bc640 1 -- 172.21.15.152:0/3899975913 shutdown_connections 2024-09-11T23:01:03.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.116+0000 7fe55f5bc640 1 -- 172.21.15.152:0/3899975913 wait complete. 2024-09-11T23:01:03.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.116+0000 7fe55f5bc640 1 Processor -- start 2024-09-11T23:01:03.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.116+0000 7fe55f5bc640 1 -- start start 2024-09-11T23:01:03.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.116+0000 7fe55f5bc640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe558102d30 0x7fe5581a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.116+0000 7fe55f5bc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5581036e0 0x7fe5581a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.116+0000 7fe55f5bc640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5581082a0 0x7fe55819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.116+0000 7fe55f5bc640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe558112160 con 0x7fe5581036e0 2024-09-11T23:01:03.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.116+0000 7fe55f5bc640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe558111fe0 con 0x7fe5581082a0 2024-09-11T23:01:03.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.116+0000 7fe55f5bc640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581122e0 con 0x7fe558102d30 2024-09-11T23:01:03.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.117+0000 7fe55cb57640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5581036e0 0x7fe5581a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.117+0000 7fe55cb57640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5581036e0 0x7fe5581a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43436/0 (socket says 172.21.15.152:43436) 2024-09-11T23:01:03.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.117+0000 7fe55cb57640 1 -- 172.21.15.152:0/537711216 learned_addr learned my addr 172.21.15.152:0/537711216 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:03.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.117+0000 7fe55d358640 1 --2- 172.21.15.152:0/537711216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe558102d30 0x7fe5581a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.117+0000 7fe55db59640 1 --2- 172.21.15.152:0/537711216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5581082a0 0x7fe55819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.117+0000 7fe55cb57640 1 -- 172.21.15.152:0/537711216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe558102d30 msgr2=0x7fe5581a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:03.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.117+0000 7fe55cb57640 1 --2- 172.21.15.152:0/537711216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe558102d30 0x7fe5581a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.117+0000 7fe55cb57640 1 -- 172.21.15.152:0/537711216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5581082a0 msgr2=0x7fe55819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:03.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.117+0000 7fe55cb57640 1 --2- 172.21.15.152:0/537711216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5581082a0 0x7fe55819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.117+0000 7fe55cb57640 1 -- 172.21.15.152:0/537711216 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55819af40 con 0x7fe5581036e0 2024-09-11T23:01:03.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.117+0000 7fe55cb57640 1 --2- 172.21.15.152:0/537711216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5581036e0 0x7fe5581a0b40 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fe54c006fd0 tx=0x7fe54c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:03.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.117+0000 7fe5567fc640 1 -- 172.21.15.152:0/537711216 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c011830 con 0x7fe5581036e0 2024-09-11T23:01:03.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.117+0000 7fe5567fc640 1 -- 172.21.15.152:0/537711216 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54c016070 con 0x7fe5581036e0 2024-09-11T23:01:03.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.117+0000 7fe5567fc640 1 -- 172.21.15.152:0/537711216 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c00f2e0 con 0x7fe5581036e0 2024-09-11T23:01:03.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.117+0000 7fe55f5bc640 1 -- 172.21.15.152:0/537711216 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55819b230 con 0x7fe5581036e0 2024-09-11T23:01:03.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.117+0000 7fe55f5bc640 1 -- 172.21.15.152:0/537711216 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5581a73b0 con 0x7fe5581036e0 2024-09-11T23:01:03.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.120+0000 7fe5567fc640 1 -- 172.21.15.152:0/537711216 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe54c01b460 con 0x7fe5581036e0 2024-09-11T23:01:03.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe561995740 1 Processor -- start 2024-09-11T23:01:03.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe561995740 1 -- start start 2024-09-11T23:01:03.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe561995740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d597b40 0x55a68d596090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe561995740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d5965d0 0x55a68d596a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe561995740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5a1370 0x55a68d5a3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe561995740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a68d4fc2f0 con 0x55a68d5a1370 2024-09-11T23:01:03.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe561995740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a68d4a3cf0 con 0x55a68d597b40 2024-09-11T23:01:03.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe561995740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a68d4f82b0 con 0x55a68d5965d0 2024-09-11T23:01:03.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe55cb57640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5a1370 0x55a68d5a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe55cb57640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5a1370 0x55a68d5a3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43440/0 (socket says 172.21.15.152:43440) 2024-09-11T23:01:03.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe55cb57640 1 -- 172.21.15.152:0/2990068954 learned_addr learned my addr 172.21.15.152:0/2990068954 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:03.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe55d358640 1 --2- 172.21.15.152:0/2990068954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d5965d0 0x55a68d596a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe55db59640 1 --2- 172.21.15.152:0/2990068954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d597b40 0x55a68d596090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe5567fc640 1 --2- 172.21.15.152:0/537711216 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5400a2cf0 0x7fe5400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe55db59640 1 --2- 172.21.15.152:0/537711216 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5400a2cf0 0x7fe5400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe55cb57640 1 -- 172.21.15.152:0/2990068954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d5965d0 msgr2=0x55a68d596a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:03.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe55cb57640 1 --2- 172.21.15.152:0/2990068954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d5965d0 0x55a68d596a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe55cb57640 1 -- 172.21.15.152:0/2990068954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d597b40 msgr2=0x55a68d596090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:03.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe55cb57640 1 --2- 172.21.15.152:0/2990068954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d597b40 0x55a68d596090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe55cb57640 1 -- 172.21.15.152:0/2990068954 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a68d4883a0 con 0x55a68d5a1370 2024-09-11T23:01:03.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.121+0000 7fe55d358640 1 --2- 172.21.15.152:0/2990068954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d5965d0 0x55a68d596a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:01:03.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.122+0000 7fe5567fc640 1 -- 172.21.15.152:0/537711216 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7fe54c0ce950 con 0x7fe5581036e0 2024-09-11T23:01:03.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.122+0000 7fe55cb57640 1 --2- 172.21.15.152:0/2990068954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5a1370 0x55a68d5a3760 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0cc170 tx=0x7fe54c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e6de92d0a3f0b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:03.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.122+0000 7fe55db59640 1 --2- 172.21.15.152:0/537711216 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5400a2cf0 0x7fe5400a51b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe550031550 tx=0x7fe55002a440 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:03.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.122+0000 7fe53f7fe640 1 -- 172.21.15.152:0/2990068954 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c0d0e60 con 0x55a68d5a1370 2024-09-11T23:01:03.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.122+0000 7fe53f7fe640 1 -- 172.21.15.152:0/2990068954 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54c09b6a0 con 0x55a68d5a1370 2024-09-11T23:01:03.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.122+0000 7fe53f7fe640 1 -- 172.21.15.152:0/2990068954 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c01e100 con 0x55a68d5a1370 2024-09-11T23:01:03.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.123+0000 7fe561995740 1 -- 172.21.15.152:0/2990068954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5a1370 msgr2=0x55a68d5a3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:03.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.123+0000 7fe561995740 1 --2- 172.21.15.152:0/2990068954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5a1370 0x55a68d5a3760 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0cc170 tx=0x7fe54c020000 comp rx=0 tx=0).stop 2024-09-11T23:01:03.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.123+0000 7fe561995740 1 -- 172.21.15.152:0/2990068954 shutdown_connections 2024-09-11T23:01:03.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.123+0000 7fe561995740 1 --2- 172.21.15.152:0/2990068954 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5a1370 0x55a68d5a3760 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.123+0000 7fe561995740 1 --2- 172.21.15.152:0/2990068954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d5965d0 0x55a68d596a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.123+0000 7fe561995740 1 --2- 172.21.15.152:0/2990068954 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d597b40 0x55a68d596090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.123+0000 7fe561995740 1 -- 172.21.15.152:0/2990068954 >> 172.21.15.152:0/2990068954 conn(0x55a68d55f310 msgr2=0x55a68d592c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:03.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.123+0000 7fe561995740 1 -- 172.21.15.152:0/2990068954 shutdown_connections 2024-09-11T23:01:03.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.123+0000 7fe561995740 1 -- 172.21.15.152:0/2990068954 wait complete. 2024-09-11T23:01:03.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.123+0000 7fe561995740 1 -- start start 2024-09-11T23:01:03.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.123+0000 7fe561995740 1 Processor -- start 2024-09-11T23:01:03.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.123+0000 7fe561995740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5965d0 0x55a68d5a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.123+0000 7fe561995740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d597b40 0x55a68d59df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.123+0000 7fe561995740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d5a1370 0x55a68d59e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe561995740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a68d5a4010 con 0x55a68d5965d0 2024-09-11T23:01:03.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe561995740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a68d4b7060 con 0x55a68d597b40 2024-09-11T23:01:03.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe561995740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a68d5a4190 con 0x55a68d5a1370 2024-09-11T23:01:03.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe561995740 10 client.0 _ll_register_callbacks cb 0x55a68d20a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:03.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe55d358640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5965d0 0x55a68d5a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe55d358640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5965d0 0x55a68d5a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43452/0 (socket says 172.21.15.152:43452) 2024-09-11T23:01:03.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe55d358640 1 -- 172.21.15.152:0/3997165740 learned_addr learned my addr 172.21.15.152:0/3997165740 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:03.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe55cb57640 1 --2- 172.21.15.152:0/3997165740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d597b40 0x55a68d59df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe55db59640 1 --2- 172.21.15.152:0/3997165740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d5a1370 0x55a68d59e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe55d358640 1 -- 172.21.15.152:0/3997165740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d5a1370 msgr2=0x55a68d59e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:03.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe55d358640 1 --2- 172.21.15.152:0/3997165740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d5a1370 0x55a68d59e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe55d358640 1 -- 172.21.15.152:0/3997165740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d597b40 msgr2=0x55a68d59df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:03.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe55d358640 1 --2- 172.21.15.152:0/3997165740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d597b40 0x55a68d59df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe55d358640 1 -- 172.21.15.152:0/3997165740 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a68d26d3c0 con 0x55a68d5965d0 2024-09-11T23:01:03.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe55db59640 1 --2- 172.21.15.152:0/3997165740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d5a1370 0x55a68d59e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:01:03.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe55d358640 1 --2- 172.21.15.152:0/3997165740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5965d0 0x55a68d5a0ce0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fe54800ae70 tx=0x7fe548011580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5729571cd9886563 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:03.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe53cff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:03.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe53cff9640 1 -- 172.21.15.152:0/3997165740 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54801a070 con 0x55a68d5965d0 2024-09-11T23:01:03.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe53cff9640 1 -- 172.21.15.152:0/3997165740 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7fe548016450 con 0x55a68d5965d0 2024-09-11T23:01:03.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe561995740 1 -- 172.21.15.152:0/3997165740 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe54c09b6a0 con 0x55a68d5965d0 2024-09-11T23:01:03.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe53cff9640 1 -- 172.21.15.152:0/3997165740 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe548026050 con 0x55a68d5965d0 2024-09-11T23:01:03.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe561995740 10 client.17742 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:03.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.124+0000 7fe561995740 1 -- 172.21.15.152:0/3997165740 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a68d5afab0 con 0x55a68d5965d0 2024-09-11T23:01:03.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.125+0000 7fe53cff9640 1 -- 172.21.15.152:0/3997165740 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54c09b6a0 con 0x55a68d5965d0 2024-09-11T23:01:03.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.125+0000 7fe53cff9640 1 -- 172.21.15.152:0/3997165740 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe54801e710 con 0x55a68d5965d0 2024-09-11T23:01:03.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.125+0000 7fe561995740 10 client.17742 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:03.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.125+0000 7fe561995740 10 client.17742 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:01:03.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.125+0000 7fe561995740 20 client.17742 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:03.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.125+0000 7fe561995740 1 --2- 172.21.15.152:0/3997165740 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55a68d5b0ec0 0x55a68d5b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.125+0000 7fe561995740 4 client.17742 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:01:03.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.125+0000 7fe561995740 1 -- 172.21.15.152:0/3997165740 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a68d509860 con 0x55a68d5b0ec0 2024-09-11T23:01:03.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.125+0000 7fe55cb57640 1 --2- 172.21.15.152:0/3997165740 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55a68d5b0ec0 0x55a68d5b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.126+0000 7fe53cff9640 10 client.17742 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T23:01:03.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.126+0000 7fe55cb57640 1 --2- 172.21.15.152:0/3997165740 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55a68d5b0ec0 0x55a68d5b32d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9110f23d9addde9f server_cookie=b016236dba8a07f6 in_seq=0 out_seq=0 2024-09-11T23:01:03.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.126+0000 7fe53cff9640 1 -- 172.21.15.152:0/3997165740 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a68d509860 con 0x55a68d5b0ec0 2024-09-11T23:01:03.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.126+0000 7fe53cff9640 10 client.17742 handle_command_reply: tid=0 2024-09-11T23:01:03.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.126+0000 7fe561995740 1 client.17742 shutdown 2024-09-11T23:01:03.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.126+0000 7fe561995740 1 -- 172.21.15.152:0/3997165740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5965d0 msgr2=0x55a68d5a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:03.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.126+0000 7fe561995740 1 --2- 172.21.15.152:0/3997165740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5965d0 0x55a68d5a0ce0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fe54800ae70 tx=0x7fe548011580 comp rx=0 tx=0).stop 2024-09-11T23:01:03.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.126+0000 7fe561995740 1 -- 172.21.15.152:0/3997165740 shutdown_connections 2024-09-11T23:01:03.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.126+0000 7fe561995740 1 --2- 172.21.15.152:0/3997165740 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55a68d5b0ec0 0x55a68d5b32d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.126+0000 7fe561995740 1 --2- 172.21.15.152:0/3997165740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d5a1370 0x55a68d59e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.126+0000 7fe561995740 1 --2- 172.21.15.152:0/3997165740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d597b40 0x55a68d59df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.126+0000 7fe561995740 1 --2- 172.21.15.152:0/3997165740 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5965d0 0x55a68d5a0ce0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.126+0000 7fe53cff9640 0 client.17742 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T23:01:03.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.126+0000 7fe561995740 1 -- 172.21.15.152:0/3997165740 >> 172.21.15.152:0/3997165740 conn(0x55a68d55f310 msgr2=0x55a68d598990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:03.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.127+0000 7fe561995740 1 -- 172.21.15.152:0/3997165740 shutdown_connections 2024-09-11T23:01:03.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.127+0000 7fe561995740 1 -- 172.21.15.152:0/3997165740 wait complete. 2024-09-11T23:01:03.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.127+0000 7fe561995740 20 client.17742 trim_cache size 0 max 16384 2024-09-11T23:01:03.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.139+0000 7fe561995740 1 Processor -- start 2024-09-11T23:01:03.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe561995740 1 -- start start 2024-09-11T23:01:03.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe561995740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5965d0 0x55a68d5ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe561995740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d597b40 0x55a68d5b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe561995740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d5a1370 0x55a68d5b90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe561995740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581afb20 con 0x55a68d5965d0 2024-09-11T23:01:03.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe561995740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581b1660 con 0x55a68d5a1370 2024-09-11T23:01:03.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe561995740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581b01f0 con 0x55a68d597b40 2024-09-11T23:01:03.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe55d358640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5965d0 0x55a68d5ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe55d358640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5965d0 0x55a68d5ad2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43466/0 (socket says 172.21.15.152:43466) 2024-09-11T23:01:03.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe55d358640 1 -- 172.21.15.152:0/3854316199 learned_addr learned my addr 172.21.15.152:0/3854316199 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:03.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe55cb57640 1 --2- 172.21.15.152:0/3854316199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d597b40 0x55a68d5b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe55db59640 1 --2- 172.21.15.152:0/3854316199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d5a1370 0x55a68d5b90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe55d358640 1 -- 172.21.15.152:0/3854316199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d597b40 msgr2=0x55a68d5b8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:03.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe55d358640 1 --2- 172.21.15.152:0/3854316199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d597b40 0x55a68d5b8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe55d358640 1 -- 172.21.15.152:0/3854316199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d5a1370 msgr2=0x55a68d5b90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:03.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe55d358640 1 --2- 172.21.15.152:0/3854316199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d5a1370 0x55a68d5b90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe55d358640 1 -- 172.21.15.152:0/3854316199 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a68d5a0900 con 0x55a68d5965d0 2024-09-11T23:01:03.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe55d358640 1 --2- 172.21.15.152:0/3854316199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5965d0 0x55a68d5ad2e0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fe54801eae0 tx=0x7fe548015430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fb4508f0f8a672 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:03.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.140+0000 7fe53effd640 1 -- 172.21.15.152:0/3854316199 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54802f070 con 0x55a68d5965d0 2024-09-11T23:01:03.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.141+0000 7fe53effd640 1 -- 172.21.15.152:0/3854316199 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5480097c0 con 0x55a68d5965d0 2024-09-11T23:01:03.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.141+0000 7fe53effd640 1 -- 172.21.15.152:0/3854316199 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54801a070 con 0x55a68d5965d0 2024-09-11T23:01:03.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.141+0000 7fe561995740 1 -- 172.21.15.152:0/3854316199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5965d0 msgr2=0x55a68d5ad2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:03.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.141+0000 7fe561995740 1 --2- 172.21.15.152:0/3854316199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5965d0 0x55a68d5ad2e0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fe54801eae0 tx=0x7fe548015430 comp rx=0 tx=0).stop 2024-09-11T23:01:03.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.141+0000 7fe561995740 1 -- 172.21.15.152:0/3854316199 shutdown_connections 2024-09-11T23:01:03.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.141+0000 7fe561995740 1 --2- 172.21.15.152:0/3854316199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d5a1370 0x55a68d5b90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.141+0000 7fe561995740 1 --2- 172.21.15.152:0/3854316199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d597b40 0x55a68d5b8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.141+0000 7fe561995740 1 --2- 172.21.15.152:0/3854316199 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5965d0 0x55a68d5ad2e0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.141+0000 7fe561995740 1 -- 172.21.15.152:0/3854316199 >> 172.21.15.152:0/3854316199 conn(0x55a68d55f310 msgr2=0x55a68d599260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:03.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.141+0000 7fe561995740 1 -- 172.21.15.152:0/3854316199 shutdown_connections 2024-09-11T23:01:03.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.141+0000 7fe561995740 1 -- 172.21.15.152:0/3854316199 wait complete. 2024-09-11T23:01:03.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.142+0000 7fe561995740 1 -- start start 2024-09-11T23:01:03.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.142+0000 7fe561995740 1 Processor -- start 2024-09-11T23:01:03.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.142+0000 7fe561995740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d5965d0 0x55a68d59b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.142+0000 7fe561995740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d597b40 0x55a68d59b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.142+0000 7fe561995740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5a1370 0x55a68d59ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.142+0000 7fe561995740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581b0f90 con 0x55a68d5a1370 2024-09-11T23:01:03.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.142+0000 7fe561995740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581b08c0 con 0x55a68d5965d0 2024-09-11T23:01:03.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.142+0000 7fe561995740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe5581adb20 con 0x55a68d597b40 2024-09-11T23:01:03.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.142+0000 7fe561995740 10 client.0 _ll_register_callbacks cb 0x7fe548009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:03.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.142+0000 7fe55db59640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5a1370 0x55a68d59ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.142+0000 7fe55db59640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5a1370 0x55a68d59ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43470/0 (socket says 172.21.15.152:43470) 2024-09-11T23:01:03.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.142+0000 7fe55db59640 1 -- 172.21.15.152:0/2785813834 learned_addr learned my addr 172.21.15.152:0/2785813834 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:03.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.142+0000 7fe55d358640 1 --2- 172.21.15.152:0/2785813834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d5965d0 0x55a68d59b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.142+0000 7fe55cb57640 1 --2- 172.21.15.152:0/2785813834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d597b40 0x55a68d59b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe55db59640 1 -- 172.21.15.152:0/2785813834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d597b40 msgr2=0x55a68d59b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:03.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe55db59640 1 --2- 172.21.15.152:0/2785813834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d597b40 0x55a68d59b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe55db59640 1 -- 172.21.15.152:0/2785813834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d5965d0 msgr2=0x55a68d59b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:03.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe55db59640 1 --2- 172.21.15.152:0/2785813834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d5965d0 0x55a68d59b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe55db59640 1 -- 172.21.15.152:0/2785813834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a68d61abc0 con 0x55a68d5a1370 2024-09-11T23:01:03.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe55db59640 1 --2- 172.21.15.152:0/2785813834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5a1370 0x55a68d59ba90 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fe550009930 tx=0x7fe550009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34ab3ae0d8dc5fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:03.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe53cff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:03.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe53cff9640 1 -- 172.21.15.152:0/2785813834 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550050030 con 0x55a68d5a1370 2024-09-11T23:01:03.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe561995740 1 -- 172.21.15.152:0/2785813834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5480097c0 con 0x55a68d5a1370 2024-09-11T23:01:03.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe53cff9640 1 -- 172.21.15.152:0/2785813834 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7fe550003c50 con 0x55a68d5a1370 2024-09-11T23:01:03.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe53cff9640 1 -- 172.21.15.152:0/2785813834 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe55005c050 con 0x55a68d5a1370 2024-09-11T23:01:03.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe561995740 10 client.17748 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:03.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe561995740 1 -- 172.21.15.152:0/2785813834 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a68d61bec0 con 0x55a68d5a1370 2024-09-11T23:01:03.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe53cff9640 1 -- 172.21.15.152:0/2785813834 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5480097c0 con 0x55a68d5a1370 2024-09-11T23:01:03.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe53cff9640 1 -- 172.21.15.152:0/2785813834 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe550004c10 con 0x55a68d5a1370 2024-09-11T23:01:03.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe561995740 10 client.17748 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:03.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe561995740 10 client.17748 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:01:03.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe561995740 20 client.17748 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:03.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe561995740 1 --2- 172.21.15.152:0/2785813834 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55a68d61c6c0 0x55a68d61eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:03.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe561995740 4 client.17748 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:01:03.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.143+0000 7fe561995740 1 -- 172.21.15.152:0/2785813834 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a68d503d70 con 0x55a68d61c6c0 2024-09-11T23:01:03.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.144+0000 7fe55d358640 1 --2- 172.21.15.152:0/2785813834 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55a68d61c6c0 0x55a68d61eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:03.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.144+0000 7fe55d358640 1 --2- 172.21.15.152:0/2785813834 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55a68d61c6c0 0x55a68d61eae0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=501dd9b06f07974c server_cookie=4dc41537d18b208e in_seq=0 out_seq=0 2024-09-11T23:01:03.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.144+0000 7fe53cff9640 10 client.17748 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T23:01:03.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.144+0000 7fe53cff9640 1 -- 172.21.15.152:0/2785813834 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a68d503d70 con 0x55a68d61c6c0 2024-09-11T23:01:03.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.144+0000 7fe53cff9640 10 client.17748 handle_command_reply: tid=0 2024-09-11T23:01:03.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.144+0000 7fe561995740 1 client.17748 shutdown 2024-09-11T23:01:03.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.144+0000 7fe561995740 1 -- 172.21.15.152:0/2785813834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5a1370 msgr2=0x55a68d59ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:03.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.144+0000 7fe561995740 1 --2- 172.21.15.152:0/2785813834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5a1370 0x55a68d59ba90 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fe550009930 tx=0x7fe550009260 comp rx=0 tx=0).stop 2024-09-11T23:01:03.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.144+0000 7fe561995740 1 -- 172.21.15.152:0/2785813834 shutdown_connections 2024-09-11T23:01:03.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.144+0000 7fe561995740 1 --2- 172.21.15.152:0/2785813834 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x55a68d61c6c0 0x55a68d61eae0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.144+0000 7fe561995740 1 --2- 172.21.15.152:0/2785813834 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a68d5a1370 0x55a68d59ba90 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.144+0000 7fe53cff9640 0 client.17748 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T23:01:03.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.144+0000 7fe561995740 1 --2- 172.21.15.152:0/2785813834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a68d597b40 0x55a68d59b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.144+0000 7fe561995740 1 --2- 172.21.15.152:0/2785813834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a68d5965d0 0x55a68d59b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.144+0000 7fe561995740 1 -- 172.21.15.152:0/2785813834 >> 172.21.15.152:0/2785813834 conn(0x55a68d55f310 msgr2=0x55a68d59f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:03.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.145+0000 7fe561995740 1 -- 172.21.15.152:0/2785813834 shutdown_connections 2024-09-11T23:01:03.168 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:01:03.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.145+0000 7fe561995740 1 -- 172.21.15.152:0/2785813834 wait complete. 2024-09-11T23:01:03.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.145+0000 7fe561995740 20 client.17748 trim_cache size 0 max 16384 2024-09-11T23:01:03.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.145+0000 7fe55f5bc640 1 -- 172.21.15.152:0/537711216 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5400a2cf0 msgr2=0x7fe5400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:03.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.145+0000 7fe55f5bc640 1 --2- 172.21.15.152:0/537711216 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5400a2cf0 0x7fe5400a51b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe550031550 tx=0x7fe55002a440 comp rx=0 tx=0).stop 2024-09-11T23:01:03.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.145+0000 7fe55f5bc640 1 -- 172.21.15.152:0/537711216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5581036e0 msgr2=0x7fe5581a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:03.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.145+0000 7fe55f5bc640 1 --2- 172.21.15.152:0/537711216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5581036e0 0x7fe5581a0b40 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fe54c006fd0 tx=0x7fe54c00b8e0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.145+0000 7fe55f5bc640 1 -- 172.21.15.152:0/537711216 shutdown_connections 2024-09-11T23:01:03.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.145+0000 7fe55f5bc640 1 --2- 172.21.15.152:0/537711216 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5400a2cf0 0x7fe5400a51b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.145+0000 7fe55f5bc640 1 --2- 172.21.15.152:0/537711216 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5581082a0 0x7fe55819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.145+0000 7fe55f5bc640 1 --2- 172.21.15.152:0/537711216 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5581036e0 0x7fe5581a0b40 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.145+0000 7fe55f5bc640 1 --2- 172.21.15.152:0/537711216 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe558102d30 0x7fe5581a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:03.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.145+0000 7fe55f5bc640 1 -- 172.21.15.152:0/537711216 >> 172.21.15.152:0/537711216 conn(0x7fe5580fc7d0 msgr2=0x7fe558101470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:03.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.145+0000 7fe55f5bc640 1 -- 172.21.15.152:0/537711216 shutdown_connections 2024-09-11T23:01:03.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:03.145+0000 7fe55f5bc640 1 -- 172.21.15.152:0/537711216 wait complete. 2024-09-11T23:01:03.978 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:01:04.227 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:01:04.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:04 smithi152 ceph-mon[31181]: pgmap v250: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T23:01:04.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:04 smithi191 ceph-mon[35211]: pgmap v250: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T23:01:04.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.757+0000 7efe0eb38640 1 -- 172.21.15.152:0/4265059224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe08106a80 msgr2=0x7efe08106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:04.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.757+0000 7efe0eb38640 1 --2- 172.21.15.152:0/4265059224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe08106a80 0x7efe08106e60 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7efe04009930 tx=0x7efe0402a420 comp rx=0 tx=0).stop 2024-09-11T23:01:04.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.757+0000 7efe0eb38640 1 -- 172.21.15.152:0/4265059224 shutdown_connections 2024-09-11T23:01:04.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.757+0000 7efe0eb38640 1 --2- 172.21.15.152:0/4265059224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe08101420 0x7efe0810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.757+0000 7efe0eb38640 1 --2- 172.21.15.152:0/4265059224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe08100a80 0x7efe08100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.757+0000 7efe0eb38640 1 --2- 172.21.15.152:0/4265059224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe08106a80 0x7efe08106e60 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.757+0000 7efe0eb38640 1 -- 172.21.15.152:0/4265059224 >> 172.21.15.152:0/4265059224 conn(0x7efe080fc7a0 msgr2=0x7efe080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:04.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.757+0000 7efe0eb38640 1 -- 172.21.15.152:0/4265059224 shutdown_connections 2024-09-11T23:01:04.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.757+0000 7efe0eb38640 1 -- 172.21.15.152:0/4265059224 wait complete. 2024-09-11T23:01:04.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.758+0000 7efe0eb38640 1 Processor -- start 2024-09-11T23:01:04.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.758+0000 7efe0eb38640 1 -- start start 2024-09-11T23:01:04.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.758+0000 7efe0eb38640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe08100a80 0x7efe08198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.758+0000 7efe0eb38640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe08101420 0x7efe08198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.758+0000 7efe0eb38640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe08106a80 0x7efe0819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.758+0000 7efe0eb38640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efe08110e80 con 0x7efe08106a80 2024-09-11T23:01:04.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.758+0000 7efe0eb38640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efe08110d00 con 0x7efe08100a80 2024-09-11T23:01:04.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.758+0000 7efe0eb38640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efe08111000 con 0x7efe08101420 2024-09-11T23:01:04.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.758+0000 7efe0d0d5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe08106a80 0x7efe0819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.758+0000 7efe0d0d5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe08106a80 0x7efe0819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43496/0 (socket says 172.21.15.152:43496) 2024-09-11T23:01:04.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.758+0000 7efe0d0d5640 1 -- 172.21.15.152:0/1712271521 learned_addr learned my addr 172.21.15.152:0/1712271521 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:04.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.758+0000 7efdfffff640 1 --2- 172.21.15.152:0/1712271521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe08101420 0x7efe08198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.759+0000 7efe0c8d4640 1 --2- 172.21.15.152:0/1712271521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe08100a80 0x7efe08198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.759+0000 7efe0d0d5640 1 -- 172.21.15.152:0/1712271521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe08101420 msgr2=0x7efe08198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:04.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.759+0000 7efe0d0d5640 1 --2- 172.21.15.152:0/1712271521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe08101420 0x7efe08198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.759+0000 7efe0d0d5640 1 -- 172.21.15.152:0/1712271521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe08100a80 msgr2=0x7efe08198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:04.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.759+0000 7efe0d0d5640 1 --2- 172.21.15.152:0/1712271521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe08100a80 0x7efe08198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.759+0000 7efe0d0d5640 1 -- 172.21.15.152:0/1712271521 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0819d3c0 con 0x7efe08106a80 2024-09-11T23:01:04.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.759+0000 7efe0d0d5640 1 --2- 172.21.15.152:0/1712271521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe08106a80 0x7efe0819cce0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7efe0000eaa0 tx=0x7efe0000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:04.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.759+0000 7efdfdffb640 1 -- 172.21.15.152:0/1712271521 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0000cdb0 con 0x7efe08106a80 2024-09-11T23:01:04.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.759+0000 7efdfdffb640 1 -- 172.21.15.152:0/1712271521 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe00005020 con 0x7efe08106a80 2024-09-11T23:01:04.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.759+0000 7efe0eb38640 1 -- 172.21.15.152:0/1712271521 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0819d6b0 con 0x7efe08106a80 2024-09-11T23:01:04.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.759+0000 7efdfdffb640 1 -- 172.21.15.152:0/1712271521 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe000116b0 con 0x7efe08106a80 2024-09-11T23:01:04.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.759+0000 7efe0eb38640 1 -- 172.21.15.152:0/1712271521 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe081a4f90 con 0x7efe08106a80 2024-09-11T23:01:04.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.761+0000 7efdfdffb640 1 -- 172.21.15.152:0/1712271521 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7efe00005560 con 0x7efe08106a80 2024-09-11T23:01:04.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe10f11740 1 Processor -- start 2024-09-11T23:01:04.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe10f11740 1 -- start start 2024-09-11T23:01:04.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe10f11740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dc7b40 0x55afd1dc6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe10f11740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dc65d0 0x55afd1dc6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe10f11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dd1370 0x55afd1dd3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe10f11740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55afd1d2c2f0 con 0x55afd1dc65d0 2024-09-11T23:01:04.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe10f11740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55afd1cd3cf0 con 0x55afd1dc7b40 2024-09-11T23:01:04.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe10f11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55afd1d282b0 con 0x55afd1dd1370 2024-09-11T23:01:04.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe0c8d4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dc65d0 0x55afd1dc6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe0c8d4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dc65d0 0x55afd1dc6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43500/0 (socket says 172.21.15.152:43500) 2024-09-11T23:01:04.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe0c8d4640 1 -- 172.21.15.152:0/1170486561 learned_addr learned my addr 172.21.15.152:0/1170486561 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:04.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efdfdffb640 1 --2- 172.21.15.152:0/1712271521 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efdf00a2cf0 0x7efdf00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efdfffff640 1 --2- 172.21.15.152:0/1170486561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dd1370 0x55afd1dd3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efdfffff640 1 --2- 172.21.15.152:0/1170486561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dc7b40 0x55afd1dc6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efdfdffb640 1 -- 172.21.15.152:0/1712271521 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7efe000cf620 con 0x7efe08106a80 2024-09-11T23:01:04.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe0c8d4640 1 -- 172.21.15.152:0/1170486561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dd1370 msgr2=0x55afd1dd3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:04.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe0d0d5640 1 --2- 172.21.15.152:0/1712271521 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efdf00a2cf0 0x7efdf00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe0c8d4640 1 --2- 172.21.15.152:0/1170486561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dd1370 0x55afd1dd3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe0c8d4640 1 -- 172.21.15.152:0/1170486561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dc7b40 msgr2=0x55afd1dc6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:04.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe0c8d4640 1 --2- 172.21.15.152:0/1170486561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dc7b40 0x55afd1dc6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.762+0000 7efe0c8d4640 1 -- 172.21.15.152:0/1170486561 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afd1cb83a0 con 0x55afd1dc65d0 2024-09-11T23:01:04.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.763+0000 7efdfffff640 1 --2- 172.21.15.152:0/1170486561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dd1370 0x55afd1dd3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:01:04.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.763+0000 7efe0c8d4640 1 --2- 172.21.15.152:0/1170486561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dc65d0 0x55afd1dc6a10 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7efe0402a940 tx=0x7efe04005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd9a6bdf53ec56d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:04.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.763+0000 7efdeeffd640 1 -- 172.21.15.152:0/1170486561 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0403f070 con 0x55afd1dc65d0 2024-09-11T23:01:04.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.763+0000 7efe0d0d5640 1 --2- 172.21.15.152:0/1712271521 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efdf00a2cf0 0x7efdf00a51b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efe0009baf0 tx=0x7efe0006b9c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:04.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.763+0000 7efdeeffd640 1 -- 172.21.15.152:0/1170486561 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe04004030 con 0x55afd1dc65d0 2024-09-11T23:01:04.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.763+0000 7efdeeffd640 1 -- 172.21.15.152:0/1170486561 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04035070 con 0x55afd1dc65d0 2024-09-11T23:01:04.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.764+0000 7efe10f11740 1 -- 172.21.15.152:0/1170486561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dc65d0 msgr2=0x55afd1dc6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:04.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.764+0000 7efe10f11740 1 --2- 172.21.15.152:0/1170486561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dc65d0 0x55afd1dc6a10 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7efe0402a940 tx=0x7efe04005e80 comp rx=0 tx=0).stop 2024-09-11T23:01:04.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.764+0000 7efe10f11740 1 -- 172.21.15.152:0/1170486561 shutdown_connections 2024-09-11T23:01:04.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.764+0000 7efe10f11740 1 --2- 172.21.15.152:0/1170486561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dd1370 0x55afd1dd3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.764+0000 7efe10f11740 1 --2- 172.21.15.152:0/1170486561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dc65d0 0x55afd1dc6a10 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.764+0000 7efe10f11740 1 --2- 172.21.15.152:0/1170486561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dc7b40 0x55afd1dc6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.764+0000 7efe10f11740 1 -- 172.21.15.152:0/1170486561 >> 172.21.15.152:0/1170486561 conn(0x55afd1d8f310 msgr2=0x55afd1dc2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:04.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.764+0000 7efe10f11740 1 -- 172.21.15.152:0/1170486561 shutdown_connections 2024-09-11T23:01:04.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.764+0000 7efe10f11740 1 -- 172.21.15.152:0/1170486561 wait complete. 2024-09-11T23:01:04.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.764+0000 7efe10f11740 1 -- start start 2024-09-11T23:01:04.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.764+0000 7efe10f11740 1 Processor -- start 2024-09-11T23:01:04.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.764+0000 7efe10f11740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dc65d0 0x55afd1dd0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.764+0000 7efe10f11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dc7b40 0x55afd1dcdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efe10f11740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dd1370 0x55afd1dce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efe10f11740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55afd1dd4010 con 0x55afd1dd1370 2024-09-11T23:01:04.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efe10f11740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55afd1ce7060 con 0x55afd1dc65d0 2024-09-11T23:01:04.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efe10f11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55afd1dd4190 con 0x55afd1dc7b40 2024-09-11T23:01:04.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efe10f11740 10 client.0 _ll_register_callbacks cb 0x55afd1a3a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:04.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efdfffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dd1370 0x55afd1dce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efdfffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dd1370 0x55afd1dce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43510/0 (socket says 172.21.15.152:43510) 2024-09-11T23:01:04.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efdfffff640 1 -- 172.21.15.152:0/2730170712 learned_addr learned my addr 172.21.15.152:0/2730170712 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:04.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efdfffff640 1 --2- 172.21.15.152:0/2730170712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dc65d0 0x55afd1dd0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efe0c8d4640 1 --2- 172.21.15.152:0/2730170712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dc7b40 0x55afd1dcdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efdfffff640 1 -- 172.21.15.152:0/2730170712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dc7b40 msgr2=0x55afd1dcdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:04.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efdfffff640 1 --2- 172.21.15.152:0/2730170712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dc7b40 0x55afd1dcdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efdfffff640 1 -- 172.21.15.152:0/2730170712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dc65d0 msgr2=0x55afd1dd0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:04.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efdfffff640 1 --2- 172.21.15.152:0/2730170712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dc65d0 0x55afd1dd0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efdfffff640 1 -- 172.21.15.152:0/2730170712 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afd1a9d3c0 con 0x55afd1dd1370 2024-09-11T23:01:04.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efdfffff640 1 --2- 172.21.15.152:0/2730170712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dd1370 0x55afd1dce5e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7efdf8012cd0 tx=0x7efdf8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f94d5016a060b0a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:04.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efde7fff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:04.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efde7fff640 1 -- 172.21.15.152:0/2730170712 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf801b040 con 0x55afd1dd1370 2024-09-11T23:01:04.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efde7fff640 1 -- 172.21.15.152:0/2730170712 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7efdf8005020 con 0x55afd1dd1370 2024-09-11T23:01:04.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efe10f11740 1 -- 172.21.15.152:0/2730170712 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe04004030 con 0x55afd1dd1370 2024-09-11T23:01:04.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efde7fff640 1 -- 172.21.15.152:0/2730170712 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efdf8010e60 con 0x55afd1dd1370 2024-09-11T23:01:04.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efe10f11740 10 client.17760 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:04.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.765+0000 7efe10f11740 1 -- 172.21.15.152:0/2730170712 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55afd1ddfae0 con 0x55afd1dd1370 2024-09-11T23:01:04.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.766+0000 7efde7fff640 1 -- 172.21.15.152:0/2730170712 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe04004030 con 0x55afd1dd1370 2024-09-11T23:01:04.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.766+0000 7efde7fff640 1 -- 172.21.15.152:0/2730170712 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efdf8020400 con 0x55afd1dd1370 2024-09-11T23:01:04.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.766+0000 7efe10f11740 10 client.17760 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:04.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.766+0000 7efe10f11740 10 client.17760 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:01:04.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.766+0000 7efe10f11740 20 client.17760 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:04.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.766+0000 7efe10f11740 1 --2- 172.21.15.152:0/2730170712 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55afd1de0fa0 0x55afd1de3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.766+0000 7efe10f11740 4 client.17760 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:01:04.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.766+0000 7efe10f11740 1 -- 172.21.15.152:0/2730170712 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afd1d39860 con 0x55afd1de0fa0 2024-09-11T23:01:04.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.766+0000 7efe0c8d4640 1 --2- 172.21.15.152:0/2730170712 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55afd1de0fa0 0x55afd1de3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.767+0000 7efe0c8d4640 1 --2- 172.21.15.152:0/2730170712 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55afd1de0fa0 0x55afd1de3390 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=80c55025310e6953 server_cookie=d01449140745c1b5 in_seq=0 out_seq=0 2024-09-11T23:01:04.802 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.767+0000 7efde7fff640 10 client.17760 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T23:01:04.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.767+0000 7efde7fff640 1 -- 172.21.15.152:0/2730170712 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55afd1d39860 con 0x55afd1de0fa0 2024-09-11T23:01:04.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.767+0000 7efde7fff640 10 client.17760 handle_command_reply: tid=0 2024-09-11T23:01:04.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.767+0000 7efe10f11740 1 client.17760 shutdown 2024-09-11T23:01:04.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.767+0000 7efe10f11740 1 -- 172.21.15.152:0/2730170712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dd1370 msgr2=0x55afd1dce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:04.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.767+0000 7efe10f11740 1 --2- 172.21.15.152:0/2730170712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dd1370 0x55afd1dce5e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7efdf8012cd0 tx=0x7efdf8004730 comp rx=0 tx=0).stop 2024-09-11T23:01:04.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.767+0000 7efe10f11740 1 -- 172.21.15.152:0/2730170712 shutdown_connections 2024-09-11T23:01:04.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.767+0000 7efe10f11740 1 --2- 172.21.15.152:0/2730170712 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55afd1de0fa0 0x55afd1de3390 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.767+0000 7efe10f11740 1 --2- 172.21.15.152:0/2730170712 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dd1370 0x55afd1dce5e0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.767+0000 7efe10f11740 1 --2- 172.21.15.152:0/2730170712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dc7b40 0x55afd1dcdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.767+0000 7efe10f11740 1 --2- 172.21.15.152:0/2730170712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dc65d0 0x55afd1dd0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.767+0000 7efe10f11740 1 -- 172.21.15.152:0/2730170712 >> 172.21.15.152:0/2730170712 conn(0x55afd1d8f310 msgr2=0x55afd1dc8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:04.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.767+0000 7efde7fff640 0 client.17760 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T23:01:04.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.768+0000 7efe10f11740 1 -- 172.21.15.152:0/2730170712 shutdown_connections 2024-09-11T23:01:04.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.768+0000 7efe10f11740 1 -- 172.21.15.152:0/2730170712 wait complete. 2024-09-11T23:01:04.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.768+0000 7efe10f11740 20 client.17760 trim_cache size 0 max 16384 2024-09-11T23:01:04.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.780+0000 7efe10f11740 1 Processor -- start 2024-09-11T23:01:04.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.780+0000 7efe10f11740 1 -- start start 2024-09-11T23:01:04.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.780+0000 7efe10f11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dc65d0 0x55afd1dc2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.780+0000 7efe10f11740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dc7b40 0x55afd1ddd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.780+0000 7efe10f11740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dd1370 0x55afd1dc9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.780+0000 7efe10f11740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efe081ad820 con 0x55afd1dd1370 2024-09-11T23:01:04.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.780+0000 7efe10f11740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efe081af360 con 0x55afd1dc7b40 2024-09-11T23:01:04.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.780+0000 7efe10f11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efe081adef0 con 0x55afd1dc65d0 2024-09-11T23:01:04.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.781+0000 7efdfffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dd1370 0x55afd1dc9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.781+0000 7efdfffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dd1370 0x55afd1dc9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43522/0 (socket says 172.21.15.152:43522) 2024-09-11T23:01:04.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.781+0000 7efdfffff640 1 -- 172.21.15.152:0/1953627645 learned_addr learned my addr 172.21.15.152:0/1953627645 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:04.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.781+0000 7efdfffff640 1 -- 172.21.15.152:0/1953627645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dc65d0 msgr2=0x55afd1dc2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:04.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.781+0000 7efdfffff640 1 --2- 172.21.15.152:0/1953627645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dc65d0 0x55afd1dc2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.781+0000 7efe0c8d4640 1 --2- 172.21.15.152:0/1953627645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dc7b40 0x55afd1ddd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.781+0000 7efdfffff640 1 -- 172.21.15.152:0/1953627645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dc7b40 msgr2=0x55afd1ddd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:04.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.781+0000 7efdfffff640 1 --2- 172.21.15.152:0/1953627645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dc7b40 0x55afd1ddd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.781+0000 7efdfffff640 1 -- 172.21.15.152:0/1953627645 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afd1dcc580 con 0x55afd1dd1370 2024-09-11T23:01:04.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.781+0000 7efe0c8d4640 1 --2- 172.21.15.152:0/1953627645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dc7b40 0x55afd1ddd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:01:04.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.781+0000 7efdfffff640 1 --2- 172.21.15.152:0/1953627645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dd1370 0x55afd1dc9120 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7efdf8004700 tx=0x7efdf8012630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7824a5d6fb76b8a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:04.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.781+0000 7efdee7fc640 1 -- 172.21.15.152:0/1953627645 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf802d040 con 0x55afd1dd1370 2024-09-11T23:01:04.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.781+0000 7efdee7fc640 1 -- 172.21.15.152:0/1953627645 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdf8005850 con 0x55afd1dd1370 2024-09-11T23:01:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.781+0000 7efdee7fc640 1 -- 172.21.15.152:0/1953627645 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8020a10 con 0x55afd1dd1370 2024-09-11T23:01:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 -- 172.21.15.152:0/1953627645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dd1370 msgr2=0x55afd1dc9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 --2- 172.21.15.152:0/1953627645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dd1370 0x55afd1dc9120 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7efdf8004700 tx=0x7efdf8012630 comp rx=0 tx=0).stop 2024-09-11T23:01:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 -- 172.21.15.152:0/1953627645 shutdown_connections 2024-09-11T23:01:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 --2- 172.21.15.152:0/1953627645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dd1370 0x55afd1dc9120 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 --2- 172.21.15.152:0/1953627645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dc7b40 0x55afd1ddd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 --2- 172.21.15.152:0/1953627645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dc65d0 0x55afd1dc2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 -- 172.21.15.152:0/1953627645 >> 172.21.15.152:0/1953627645 conn(0x55afd1d8f310 msgr2=0x55afd1dc2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 -- 172.21.15.152:0/1953627645 shutdown_connections 2024-09-11T23:01:04.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 -- 172.21.15.152:0/1953627645 wait complete. 2024-09-11T23:01:04.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 -- start start 2024-09-11T23:01:04.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 Processor -- start 2024-09-11T23:01:04.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dc65d0 0x55afd1dd9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dc7b40 0x55afd1dd9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dd1370 0x55afd1dcaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7efe081aec90 con 0x55afd1dc65d0 2024-09-11T23:01:04.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7efe081ae5c0 con 0x55afd1dd1370 2024-09-11T23:01:04.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.782+0000 7efe10f11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7efe081ab820 con 0x55afd1dc7b40 2024-09-11T23:01:04.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efe10f11740 10 client.0 _ll_register_callbacks cb 0x7efdf800bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:04.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efdfffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dc65d0 0x55afd1dd9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efdfffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dc65d0 0x55afd1dd9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43534/0 (socket says 172.21.15.152:43534) 2024-09-11T23:01:04.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efdfffff640 1 -- 172.21.15.152:0/4080113116 learned_addr learned my addr 172.21.15.152:0/4080113116 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:04.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efe0c8d4640 1 --2- 172.21.15.152:0/4080113116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dc7b40 0x55afd1dd9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efdfffff640 1 --2- 172.21.15.152:0/4080113116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dd1370 0x55afd1dcaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efdfffff640 1 -- 172.21.15.152:0/4080113116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dc7b40 msgr2=0x55afd1dd9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:04.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efdfffff640 1 --2- 172.21.15.152:0/4080113116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dc7b40 0x55afd1dd9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efdfffff640 1 -- 172.21.15.152:0/4080113116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dd1370 msgr2=0x55afd1dcaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:04.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efdfffff640 1 --2- 172.21.15.152:0/4080113116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dd1370 0x55afd1dcaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efdfffff640 1 -- 172.21.15.152:0/4080113116 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afd1dcb240 con 0x55afd1dc65d0 2024-09-11T23:01:04.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efdfffff640 1 --2- 172.21.15.152:0/4080113116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dc65d0 0x55afd1dd9720 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7efdf8020820 tx=0x7efdf8012360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c504daa06f638e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:04.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efde7fff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:04.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efde7fff640 1 -- 172.21.15.152:0/4080113116 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8038030 con 0x55afd1dc65d0 2024-09-11T23:01:04.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efde7fff640 1 -- 172.21.15.152:0/4080113116 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7efdf8020400 con 0x55afd1dc65d0 2024-09-11T23:01:04.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efe10f11740 1 -- 172.21.15.152:0/4080113116 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdf8005850 con 0x55afd1dc65d0 2024-09-11T23:01:04.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efde7fff640 1 -- 172.21.15.152:0/4080113116 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efdf802d040 con 0x55afd1dc65d0 2024-09-11T23:01:04.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efe10f11740 10 client.17766 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:04.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.783+0000 7efe10f11740 1 -- 172.21.15.152:0/4080113116 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55afd1e4fb70 con 0x55afd1dc65d0 2024-09-11T23:01:04.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.784+0000 7efde7fff640 1 -- 172.21.15.152:0/4080113116 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdf8005850 con 0x55afd1dc65d0 2024-09-11T23:01:04.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.784+0000 7efde7fff640 1 -- 172.21.15.152:0/4080113116 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efdf8044020 con 0x55afd1dc65d0 2024-09-11T23:01:04.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.784+0000 7efe10f11740 10 client.17766 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:04.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.784+0000 7efe10f11740 10 client.17766 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:01:04.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.784+0000 7efe10f11740 20 client.17766 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:04.809 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:01:04.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.784+0000 7efe10f11740 1 --2- 172.21.15.152:0/4080113116 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55afd1e50180 0x55afd1e52570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:04.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.784+0000 7efe10f11740 4 client.17766 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:01:04.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.784+0000 7efe10f11740 1 -- 172.21.15.152:0/4080113116 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55afd1d33d70 con 0x55afd1e50180 2024-09-11T23:01:04.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.784+0000 7efe0c8d4640 1 --2- 172.21.15.152:0/4080113116 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55afd1e50180 0x55afd1e52570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:04.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.784+0000 7efde7fff640 10 client.17766 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T23:01:04.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.784+0000 7efe0c8d4640 1 --2- 172.21.15.152:0/4080113116 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55afd1e50180 0x55afd1e52570 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b8e2b83975e329c server_cookie=678f011c1d1b5b in_seq=0 out_seq=0 2024-09-11T23:01:04.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efde7fff640 1 -- 172.21.15.152:0/4080113116 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55afd1d33d70 con 0x55afd1e50180 2024-09-11T23:01:04.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efde7fff640 10 client.17766 handle_command_reply: tid=0 2024-09-11T23:01:04.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efe10f11740 1 client.17766 shutdown 2024-09-11T23:01:04.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efe10f11740 1 -- 172.21.15.152:0/4080113116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dc65d0 msgr2=0x55afd1dd9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:04.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efe10f11740 1 --2- 172.21.15.152:0/4080113116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dc65d0 0x55afd1dd9720 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7efdf8020820 tx=0x7efdf8012360 comp rx=0 tx=0).stop 2024-09-11T23:01:04.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efe10f11740 1 -- 172.21.15.152:0/4080113116 shutdown_connections 2024-09-11T23:01:04.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efe10f11740 1 --2- 172.21.15.152:0/4080113116 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55afd1e50180 0x55afd1e52570 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efe10f11740 1 --2- 172.21.15.152:0/4080113116 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55afd1dd1370 0x55afd1dcaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efe10f11740 1 --2- 172.21.15.152:0/4080113116 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55afd1dc7b40 0x55afd1dd9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efde7fff640 0 client.17766 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T23:01:04.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efe10f11740 1 --2- 172.21.15.152:0/4080113116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55afd1dc65d0 0x55afd1dd9720 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efe10f11740 1 -- 172.21.15.152:0/4080113116 >> 172.21.15.152:0/4080113116 conn(0x55afd1d8f310 msgr2=0x55afd1dc14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:04.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efe10f11740 1 -- 172.21.15.152:0/4080113116 shutdown_connections 2024-09-11T23:01:04.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efe10f11740 1 -- 172.21.15.152:0/4080113116 wait complete. 2024-09-11T23:01:04.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efe10f11740 20 client.17766 trim_cache size 0 max 16384 2024-09-11T23:01:04.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efe0eb38640 1 -- 172.21.15.152:0/1712271521 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efdf00a2cf0 msgr2=0x7efdf00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:04.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efe0eb38640 1 --2- 172.21.15.152:0/1712271521 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efdf00a2cf0 0x7efdf00a51b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7efe0009baf0 tx=0x7efe0006b9c0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efe0eb38640 1 -- 172.21.15.152:0/1712271521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe08106a80 msgr2=0x7efe0819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:04.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.785+0000 7efe0eb38640 1 --2- 172.21.15.152:0/1712271521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe08106a80 0x7efe0819cce0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7efe0000eaa0 tx=0x7efe0000ef60 comp rx=0 tx=0).stop 2024-09-11T23:01:04.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.786+0000 7efe0eb38640 1 -- 172.21.15.152:0/1712271521 shutdown_connections 2024-09-11T23:01:04.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.786+0000 7efe0eb38640 1 --2- 172.21.15.152:0/1712271521 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7efdf00a2cf0 0x7efdf00a51b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.786+0000 7efe0eb38640 1 --2- 172.21.15.152:0/1712271521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efe08106a80 0x7efe0819cce0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.786+0000 7efe0eb38640 1 --2- 172.21.15.152:0/1712271521 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe08101420 0x7efe08198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.786+0000 7efe0eb38640 1 --2- 172.21.15.152:0/1712271521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe08100a80 0x7efe08198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:04.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.786+0000 7efe0eb38640 1 -- 172.21.15.152:0/1712271521 >> 172.21.15.152:0/1712271521 conn(0x7efe080fc7a0 msgr2=0x7efe0806a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:04.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.786+0000 7efe0eb38640 1 -- 172.21.15.152:0/1712271521 shutdown_connections 2024-09-11T23:01:04.812 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:04.786+0000 7efe0eb38640 1 -- 172.21.15.152:0/1712271521 wait complete. 2024-09-11T23:01:04.812 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:04 smithi204 ceph-mon[35454]: pgmap v250: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T23:01:05.708 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:01:05.958 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:01:06.481 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.479+0000 7f58437b9640 1 -- 172.21.15.152:0/1576348307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f583c101de0 msgr2=0x7f583c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:06.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.479+0000 7f58437b9640 1 --2- 172.21.15.152:0/1576348307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f583c101de0 0x7f583c1021c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f582c009900 tx=0x7f582c02a410 comp rx=0 tx=0).stop 2024-09-11T23:01:06.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.479+0000 7f58437b9640 1 -- 172.21.15.152:0/1576348307 shutdown_connections 2024-09-11T23:01:06.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.479+0000 7f58437b9640 1 --2- 172.21.15.152:0/1576348307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c106f10 0x7f583c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.479+0000 7f58437b9640 1 --2- 172.21.15.152:0/1576348307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583c102790 0x7f583c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.479+0000 7f58437b9640 1 --2- 172.21.15.152:0/1576348307 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f583c101de0 0x7f583c1021c0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.479+0000 7f58437b9640 1 -- 172.21.15.152:0/1576348307 >> 172.21.15.152:0/1576348307 conn(0x7f583c0fd8c0 msgr2=0x7f583c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:06.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.479+0000 7f58437b9640 1 -- 172.21.15.152:0/1576348307 shutdown_connections 2024-09-11T23:01:06.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.479+0000 7f58437b9640 1 -- 172.21.15.152:0/1576348307 wait complete. 2024-09-11T23:01:06.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.479+0000 7f58437b9640 1 Processor -- start 2024-09-11T23:01:06.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.479+0000 7f58437b9640 1 -- start start 2024-09-11T23:01:06.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f58437b9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c101de0 0x7f583c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f58437b9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583c102790 0x7f583c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f58437b9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f583c106f10 0x7f583c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f58437b9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f583c111f70 con 0x7f583c106f10 2024-09-11T23:01:06.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f58437b9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f583c111df0 con 0x7f583c102790 2024-09-11T23:01:06.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f58437b9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f583c1120f0 con 0x7f583c101de0 2024-09-11T23:01:06.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f5841d56640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f583c106f10 0x7f583c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f5841d56640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f583c106f10 0x7f583c19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43548/0 (socket says 172.21.15.152:43548) 2024-09-11T23:01:06.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f5841d56640 1 -- 172.21.15.152:0/242865147 learned_addr learned my addr 172.21.15.152:0/242865147 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:06.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f5840d54640 1 --2- 172.21.15.152:0/242865147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583c102790 0x7f583c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f5841555640 1 --2- 172.21.15.152:0/242865147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c101de0 0x7f583c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.483 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f5841d56640 1 -- 172.21.15.152:0/242865147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c101de0 msgr2=0x7f583c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:06.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f5841d56640 1 --2- 172.21.15.152:0/242865147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c101de0 0x7f583c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f5841d56640 1 -- 172.21.15.152:0/242865147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583c102790 msgr2=0x7f583c198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:06.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f5841d56640 1 --2- 172.21.15.152:0/242865147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583c102790 0x7f583c198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f5841d56640 1 -- 172.21.15.152:0/242865147 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583c19d480 con 0x7f583c106f10 2024-09-11T23:01:06.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.480+0000 7f5841d56640 1 --2- 172.21.15.152:0/242865147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f583c106f10 0x7f583c19cda0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f583400eb10 tx=0x7f583400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:06.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.481+0000 7f58327fc640 1 -- 172.21.15.152:0/242865147 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834004b20 con 0x7f583c106f10 2024-09-11T23:01:06.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.481+0000 7f58327fc640 1 -- 172.21.15.152:0/242865147 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5834004cc0 con 0x7f583c106f10 2024-09-11T23:01:06.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.481+0000 7f58437b9640 1 -- 172.21.15.152:0/242865147 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583c19d770 con 0x7f583c106f10 2024-09-11T23:01:06.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.481+0000 7f58327fc640 1 -- 172.21.15.152:0/242865147 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834010760 con 0x7f583c106f10 2024-09-11T23:01:06.484 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.481+0000 7f58437b9640 1 -- 172.21.15.152:0/242865147 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f583c1a5050 con 0x7f583c106f10 2024-09-11T23:01:06.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.483+0000 7f58327fc640 1 -- 172.21.15.152:0/242865147 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5834005020 con 0x7f583c106f10 2024-09-11T23:01:06.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.484+0000 7f5845b92740 1 Processor -- start 2024-09-11T23:01:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.484+0000 7f5845b92740 1 -- start start 2024-09-11T23:01:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.484+0000 7f5845b92740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476d9b40 0x5564476d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.484+0000 7f5845b92740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d85d0 0x5564476d8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.484+0000 7f5845b92740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476e7780 0x5564476e9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.484+0000 7f5845b92740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55644763e2f0 con 0x5564476e7780 2024-09-11T23:01:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.484+0000 7f5845b92740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5564475e5cf0 con 0x5564476d85d0 2024-09-11T23:01:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.484+0000 7f5845b92740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55644763a2b0 con 0x5564476d9b40 2024-09-11T23:01:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.484+0000 7f58327fc640 1 --2- 172.21.15.152:0/242865147 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58280a2cf0 0x7f58280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.484+0000 7f5840d54640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476e7780 0x5564476e9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.484+0000 7f5840d54640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476e7780 0x5564476e9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43562/0 (socket says 172.21.15.152:43562) 2024-09-11T23:01:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.484+0000 7f5840d54640 1 -- 172.21.15.152:0/3251106263 learned_addr learned my addr 172.21.15.152:0/3251106263 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.484+0000 7f5841555640 1 --2- 172.21.15.152:0/3251106263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d85d0 0x5564476d8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.484+0000 7f5840d54640 1 --2- 172.21.15.152:0/3251106263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476d9b40 0x5564476d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.484+0000 7f5841d56640 1 --2- 172.21.15.152:0/242865147 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58280a2cf0 0x7f58280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.484+0000 7f58327fc640 1 -- 172.21.15.152:0/242865147 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f58340cfad0 con 0x7f583c106f10 2024-09-11T23:01:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.485+0000 7f5841555640 1 -- 172.21.15.152:0/3251106263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476d9b40 msgr2=0x5564476d8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.485+0000 7f5841555640 1 --2- 172.21.15.152:0/3251106263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476d9b40 0x5564476d8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.485+0000 7f5841555640 1 -- 172.21.15.152:0/3251106263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476e7780 msgr2=0x5564476e9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.485+0000 7f5841555640 1 --2- 172.21.15.152:0/3251106263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476e7780 0x5564476e9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.493 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.485+0000 7f5841555640 1 -- 172.21.15.152:0/3251106263 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564475ca3a0 con 0x5564476d85d0 2024-09-11T23:01:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.485+0000 7f5840d54640 1 --2- 172.21.15.152:0/3251106263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476e7780 0x5564476e9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:01:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.485+0000 7f5841d56640 1 --2- 172.21.15.152:0/242865147 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58280a2cf0 0x7f58280a51b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5834098980 tx=0x7f5834091440 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.485+0000 7f5841555640 1 --2- 172.21.15.152:0/3251106263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d85d0 0x5564476d8a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f582c005940 tx=0x7f582c005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aa81b3c1a8a2ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.485+0000 7f58277fe640 1 -- 172.21.15.152:0/3251106263 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c040070 con 0x5564476d85d0 2024-09-11T23:01:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.485+0000 7f58277fe640 1 -- 172.21.15.152:0/3251106263 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582c0318f0 con 0x5564476d85d0 2024-09-11T23:01:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.485+0000 7f58277fe640 1 -- 172.21.15.152:0/3251106263 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c035040 con 0x5564476d85d0 2024-09-11T23:01:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.486+0000 7f5845b92740 1 -- 172.21.15.152:0/3251106263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d85d0 msgr2=0x5564476d8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.486+0000 7f5845b92740 1 --2- 172.21.15.152:0/3251106263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d85d0 0x5564476d8a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f582c005940 tx=0x7f582c005e00 comp rx=0 tx=0).stop 2024-09-11T23:01:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.486+0000 7f5845b92740 1 -- 172.21.15.152:0/3251106263 shutdown_connections 2024-09-11T23:01:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.486+0000 7f5845b92740 1 --2- 172.21.15.152:0/3251106263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476e7780 0x5564476e9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.486+0000 7f5845b92740 1 --2- 172.21.15.152:0/3251106263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d85d0 0x5564476d8a10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.486+0000 7f5845b92740 1 --2- 172.21.15.152:0/3251106263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476d9b40 0x5564476d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.486+0000 7f5845b92740 1 -- 172.21.15.152:0/3251106263 >> 172.21.15.152:0/3251106263 conn(0x5564476a1310 msgr2=0x5564476d4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.486+0000 7f5845b92740 1 -- 172.21.15.152:0/3251106263 shutdown_connections 2024-09-11T23:01:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.486+0000 7f5845b92740 1 -- 172.21.15.152:0/3251106263 wait complete. 2024-09-11T23:01:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.486+0000 7f5845b92740 1 -- start start 2024-09-11T23:01:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.486+0000 7f5845b92740 1 Processor -- start 2024-09-11T23:01:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5845b92740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d85d0 0x5564476e7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5845b92740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d9b40 0x5564476e4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5845b92740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476e7780 0x5564476e49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5845b92740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5564476ea420 con 0x5564476d85d0 2024-09-11T23:01:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5845b92740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5564475f9060 con 0x5564476d9b40 2024-09-11T23:01:06.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5845b92740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5564476ea5a0 con 0x5564476e7780 2024-09-11T23:01:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5845b92740 10 client.0 _ll_register_callbacks cb 0x55644734c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5840d54640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d85d0 0x5564476e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5840d54640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d85d0 0x5564476e7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43568/0 (socket says 172.21.15.152:43568) 2024-09-11T23:01:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5840d54640 1 -- 172.21.15.152:0/22174057 learned_addr learned my addr 172.21.15.152:0/22174057 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5841555640 1 --2- 172.21.15.152:0/22174057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d9b40 0x5564476e4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5840d54640 1 --2- 172.21.15.152:0/22174057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476e7780 0x5564476e49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5840d54640 1 -- 172.21.15.152:0/22174057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476e7780 msgr2=0x5564476e49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5840d54640 1 --2- 172.21.15.152:0/22174057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476e7780 0x5564476e49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5840d54640 1 -- 172.21.15.152:0/22174057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d9b40 msgr2=0x5564476e4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5840d54640 1 --2- 172.21.15.152:0/22174057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d9b40 0x5564476e4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5840d54640 1 -- 172.21.15.152:0/22174057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564473af3c0 con 0x5564476d85d0 2024-09-11T23:01:06.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5841555640 1 --2- 172.21.15.152:0/22174057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d9b40 0x5564476e4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:01:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5840d54640 1 --2- 172.21.15.152:0/22174057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d85d0 0x5564476e7150 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f5838004610 tx=0x7f5838014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96203c236419f24c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.487+0000 7f5824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.488+0000 7f5824ff9640 1 -- 172.21.15.152:0/22174057 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838012ab0 con 0x5564476d85d0 2024-09-11T23:01:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.488+0000 7f5824ff9640 1 -- 172.21.15.152:0/22174057 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f5838012c50 con 0x5564476d85d0 2024-09-11T23:01:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.488+0000 7f5845b92740 1 -- 172.21.15.152:0/22174057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f582c0318f0 con 0x5564476d85d0 2024-09-11T23:01:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.488+0000 7f5824ff9640 1 -- 172.21.15.152:0/22174057 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5838029050 con 0x5564476d85d0 2024-09-11T23:01:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.488+0000 7f5845b92740 10 client.17778 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.488+0000 7f5845b92740 1 -- 172.21.15.152:0/22174057 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5564476f5ef0 con 0x5564476d85d0 2024-09-11T23:01:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.488+0000 7f5824ff9640 1 -- 172.21.15.152:0/22174057 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582c0318f0 con 0x5564476d85d0 2024-09-11T23:01:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.488+0000 7f5824ff9640 1 -- 172.21.15.152:0/22174057 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f58380243d0 con 0x5564476d85d0 2024-09-11T23:01:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.488+0000 7f5845b92740 10 client.17778 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:06.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.488+0000 7f5845b92740 10 client.17778 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:01:06.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.488+0000 7f5845b92740 20 client.17778 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:06.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.488+0000 7f5845b92740 1 --2- 172.21.15.152:0/22174057 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5564476f73b0 0x5564476f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.488+0000 7f5845b92740 4 client.17778 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:01:06.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.488+0000 7f5845b92740 1 -- 172.21.15.152:0/22174057 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55644764b860 con 0x5564476f73b0 2024-09-11T23:01:06.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.489+0000 7f5841555640 1 --2- 172.21.15.152:0/22174057 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5564476f73b0 0x5564476f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.489+0000 7f5824ff9640 10 client.17778 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T23:01:06.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.489+0000 7f5841555640 1 --2- 172.21.15.152:0/22174057 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5564476f73b0 0x5564476f97c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ba215716dd5b27e server_cookie=7f5c5bc893ac2dfd in_seq=0 out_seq=0 2024-09-11T23:01:06.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.490+0000 7f5824ff9640 1 -- 172.21.15.152:0/22174057 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55644764b860 con 0x5564476f73b0 2024-09-11T23:01:06.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.490+0000 7f5824ff9640 10 client.17778 handle_command_reply: tid=0 2024-09-11T23:01:06.498 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.490+0000 7f5845b92740 1 client.17778 shutdown 2024-09-11T23:01:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.490+0000 7f5845b92740 1 -- 172.21.15.152:0/22174057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d85d0 msgr2=0x5564476e7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.490+0000 7f5845b92740 1 --2- 172.21.15.152:0/22174057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d85d0 0x5564476e7150 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f5838004610 tx=0x7f5838014040 comp rx=0 tx=0).stop 2024-09-11T23:01:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.490+0000 7f5845b92740 1 -- 172.21.15.152:0/22174057 shutdown_connections 2024-09-11T23:01:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.490+0000 7f5845b92740 1 --2- 172.21.15.152:0/22174057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476e7780 0x5564476e49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.490+0000 7f5845b92740 1 --2- 172.21.15.152:0/22174057 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5564476f73b0 0x5564476f97c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.490+0000 7f5845b92740 1 --2- 172.21.15.152:0/22174057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d9b40 0x5564476e4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.490+0000 7f5845b92740 1 --2- 172.21.15.152:0/22174057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d85d0 0x5564476e7150 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.490+0000 7f5824ff9640 0 client.17778 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T23:01:06.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.490+0000 7f5845b92740 1 -- 172.21.15.152:0/22174057 >> 172.21.15.152:0/22174057 conn(0x5564476a1310 msgr2=0x5564476da960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:06.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.490+0000 7f5845b92740 1 -- 172.21.15.152:0/22174057 shutdown_connections 2024-09-11T23:01:06.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.490+0000 7f5845b92740 1 -- 172.21.15.152:0/22174057 wait complete. 2024-09-11T23:01:06.500 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.490+0000 7f5845b92740 20 client.17778 trim_cache size 0 max 16384 2024-09-11T23:01:06.500 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:06 smithi152 ceph-mon[31181]: pgmap v251: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-11T23:01:06.506 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.505+0000 7f5845b92740 1 Processor -- start 2024-09-11T23:01:06.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.505+0000 7f5845b92740 1 -- start start 2024-09-11T23:01:06.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.505+0000 7f5845b92740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d85d0 0x5564476d4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.505+0000 7f5845b92740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d9b40 0x5564476f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.505+0000 7f5845b92740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476e7780 0x5564476ff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.505+0000 7f5845b92740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f583c1ab6b0 con 0x5564476d85d0 2024-09-11T23:01:06.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.505+0000 7f5845b92740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f583c1ad1f0 con 0x5564476d9b40 2024-09-11T23:01:06.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.505+0000 7f5845b92740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f583c1abd80 con 0x5564476e7780 2024-09-11T23:01:06.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.505+0000 7f5840d54640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d85d0 0x5564476d4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.505+0000 7f5840d54640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d85d0 0x5564476d4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43572/0 (socket says 172.21.15.152:43572) 2024-09-11T23:01:06.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.505+0000 7f5840d54640 1 -- 172.21.15.152:0/1563906492 learned_addr learned my addr 172.21.15.152:0/1563906492 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:06.507 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.505+0000 7f5841555640 1 --2- 172.21.15.152:0/1563906492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d9b40 0x5564476f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.506+0000 7f5840d54640 1 -- 172.21.15.152:0/1563906492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476e7780 msgr2=0x5564476ff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:06.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.506+0000 7f5840d54640 1 --2- 172.21.15.152:0/1563906492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476e7780 0x5564476ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.506+0000 7f5840d54640 1 -- 172.21.15.152:0/1563906492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d9b40 msgr2=0x5564476f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:06.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.506+0000 7f5840d54640 1 --2- 172.21.15.152:0/1563906492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d9b40 0x5564476f36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.506+0000 7f5840d54640 1 -- 172.21.15.152:0/1563906492 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564476d5730 con 0x5564476d85d0 2024-09-11T23:01:06.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.506+0000 7f5840d54640 1 --2- 172.21.15.152:0/1563906492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d85d0 0x5564476d4e10 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f583800b6b0 tx=0x7f58380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e65779895c328d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:06.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.506+0000 7f5826ffd640 1 -- 172.21.15.152:0/1563906492 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838031020 con 0x5564476d85d0 2024-09-11T23:01:06.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.506+0000 7f5826ffd640 1 -- 172.21.15.152:0/1563906492 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58380243f0 con 0x5564476d85d0 2024-09-11T23:01:06.508 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.506+0000 7f5826ffd640 1 -- 172.21.15.152:0/1563906492 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838004a00 con 0x5564476d85d0 2024-09-11T23:01:06.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.507+0000 7f5845b92740 1 -- 172.21.15.152:0/1563906492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d85d0 msgr2=0x5564476d4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:06.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.507+0000 7f5845b92740 1 --2- 172.21.15.152:0/1563906492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d85d0 0x5564476d4e10 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f583800b6b0 tx=0x7f58380046c0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.507+0000 7f5845b92740 1 -- 172.21.15.152:0/1563906492 shutdown_connections 2024-09-11T23:01:06.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.507+0000 7f5845b92740 1 --2- 172.21.15.152:0/1563906492 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476e7780 0x5564476ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.507+0000 7f5845b92740 1 --2- 172.21.15.152:0/1563906492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476d9b40 0x5564476f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.507+0000 7f5845b92740 1 --2- 172.21.15.152:0/1563906492 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d85d0 0x5564476d4e10 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.507+0000 7f5845b92740 1 -- 172.21.15.152:0/1563906492 >> 172.21.15.152:0/1563906492 conn(0x5564476a1310 msgr2=0x5564476d4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:06.514 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.507+0000 7f5845b92740 1 -- 172.21.15.152:0/1563906492 shutdown_connections 2024-09-11T23:01:06.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.507+0000 7f5845b92740 1 -- 172.21.15.152:0/1563906492 wait complete. 2024-09-11T23:01:06.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.507+0000 7f5845b92740 1 -- start start 2024-09-11T23:01:06.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.507+0000 7f5845b92740 1 Processor -- start 2024-09-11T23:01:06.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5845b92740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476d85d0 0x5564476eff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5845b92740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d9b40 0x5564476e10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5845b92740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476e7780 0x5564476e1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5845b92740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f583c1acb20 con 0x5564476d9b40 2024-09-11T23:01:06.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5845b92740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f583c1ac450 con 0x5564476e7780 2024-09-11T23:01:06.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5845b92740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f583c1a96b0 con 0x5564476d85d0 2024-09-11T23:01:06.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5845b92740 10 client.0 _ll_register_callbacks cb 0x7f5838004290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:06.515 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5841555640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d9b40 0x5564476e10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5841555640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d9b40 0x5564476e10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43586/0 (socket says 172.21.15.152:43586) 2024-09-11T23:01:06.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5841555640 1 -- 172.21.15.152:0/2704441082 learned_addr learned my addr 172.21.15.152:0/2704441082 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:06.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5840d54640 1 --2- 172.21.15.152:0/2704441082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476e7780 0x5564476e1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5840d54640 1 --2- 172.21.15.152:0/2704441082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476d85d0 0x5564476eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5841555640 1 -- 172.21.15.152:0/2704441082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476d85d0 msgr2=0x5564476eff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:06.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5841555640 1 --2- 172.21.15.152:0/2704441082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476d85d0 0x5564476eff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5841555640 1 -- 172.21.15.152:0/2704441082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476e7780 msgr2=0x5564476e1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:06.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5841555640 1 --2- 172.21.15.152:0/2704441082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476e7780 0x5564476e1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5841555640 1 -- 172.21.15.152:0/2704441082 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564476e1e20 con 0x5564476d9b40 2024-09-11T23:01:06.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5841555640 1 --2- 172.21.15.152:0/2704441082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d9b40 0x5564476e10e0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f582c031470 tx=0x7f582c0054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1323ae354a761140 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:06.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:06.516 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.508+0000 7f5824ff9640 1 -- 172.21.15.152:0/2704441082 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c040040 con 0x5564476d9b40 2024-09-11T23:01:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.509+0000 7f5824ff9640 1 -- 172.21.15.152:0/2704441082 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7f582c004030 con 0x5564476d9b40 2024-09-11T23:01:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.509+0000 7f5845b92740 1 -- 172.21.15.152:0/2704441082 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58380243f0 con 0x5564476d9b40 2024-09-11T23:01:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.509+0000 7f5824ff9640 1 -- 172.21.15.152:0/2704441082 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f582c061050 con 0x5564476d9b40 2024-09-11T23:01:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.509+0000 7f5845b92740 10 client.17784 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.509+0000 7f5845b92740 1 -- 172.21.15.152:0/2704441082 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556447762090 con 0x5564476d9b40 2024-09-11T23:01:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.509+0000 7f5824ff9640 1 -- 172.21.15.152:0/2704441082 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58380243f0 con 0x5564476d9b40 2024-09-11T23:01:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.509+0000 7f5824ff9640 1 -- 172.21.15.152:0/2704441082 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f582c058070 con 0x5564476d9b40 2024-09-11T23:01:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.509+0000 7f5845b92740 10 client.17784 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.509+0000 7f5845b92740 10 client.17784 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:01:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.509+0000 7f5845b92740 20 client.17784 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:06.517 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.509+0000 7f5845b92740 1 --2- 172.21.15.152:0/2704441082 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5564477623d0 0x5564477647c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:06.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.509+0000 7f5845b92740 4 client.17784 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:01:06.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.509+0000 7f5845b92740 1 -- 172.21.15.152:0/2704441082 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556447645d70 con 0x5564477623d0 2024-09-11T23:01:06.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.509+0000 7f5840d54640 1 --2- 172.21.15.152:0/2704441082 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5564477623d0 0x5564477647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:06.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.510+0000 7f5840d54640 1 --2- 172.21.15.152:0/2704441082 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5564477623d0 0x5564477647c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d8f26614c0769d8 server_cookie=51ab105da4d3ede5 in_seq=0 out_seq=0 2024-09-11T23:01:06.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.510+0000 7f5824ff9640 10 client.17784 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T23:01:06.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.510+0000 7f5824ff9640 1 -- 172.21.15.152:0/2704441082 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556447645d70 con 0x5564477623d0 2024-09-11T23:01:06.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.510+0000 7f5824ff9640 10 client.17784 handle_command_reply: tid=0 2024-09-11T23:01:06.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.510+0000 7f5845b92740 1 client.17784 shutdown 2024-09-11T23:01:06.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.510+0000 7f5845b92740 1 -- 172.21.15.152:0/2704441082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d9b40 msgr2=0x5564476e10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:06.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.510+0000 7f5845b92740 1 --2- 172.21.15.152:0/2704441082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d9b40 0x5564476e10e0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f582c031470 tx=0x7f582c0054d0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.510+0000 7f5845b92740 1 -- 172.21.15.152:0/2704441082 shutdown_connections 2024-09-11T23:01:06.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.510+0000 7f5845b92740 1 --2- 172.21.15.152:0/2704441082 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5564477623d0 0x5564477647c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.510+0000 7f5845b92740 1 --2- 172.21.15.152:0/2704441082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5564476e7780 0x5564476e1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.510+0000 7f5845b92740 1 --2- 172.21.15.152:0/2704441082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5564476d9b40 0x5564476e10e0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.510+0000 7f5845b92740 1 --2- 172.21.15.152:0/2704441082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5564476d85d0 0x5564476eff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.510+0000 7f5845b92740 1 -- 172.21.15.152:0/2704441082 >> 172.21.15.152:0/2704441082 conn(0x5564476a1310 msgr2=0x5564476d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:06.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.510+0000 7f5824ff9640 0 client.17784 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T23:01:06.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.510+0000 7f5845b92740 1 -- 172.21.15.152:0/2704441082 shutdown_connections 2024-09-11T23:01:06.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.510+0000 7f5845b92740 1 -- 172.21.15.152:0/2704441082 wait complete. 2024-09-11T23:01:06.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.511+0000 7f5845b92740 20 client.17784 trim_cache size 0 max 16384 2024-09-11T23:01:06.520 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:01:06.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.511+0000 7f58437b9640 1 -- 172.21.15.152:0/242865147 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58280a2cf0 msgr2=0x7f58280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:06.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.511+0000 7f58437b9640 1 --2- 172.21.15.152:0/242865147 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58280a2cf0 0x7f58280a51b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5834098980 tx=0x7f5834091440 comp rx=0 tx=0).stop 2024-09-11T23:01:06.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.511+0000 7f58437b9640 1 -- 172.21.15.152:0/242865147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f583c106f10 msgr2=0x7f583c19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:06.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.511+0000 7f58437b9640 1 --2- 172.21.15.152:0/242865147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f583c106f10 0x7f583c19cda0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f583400eb10 tx=0x7f583400c2d0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.511+0000 7f58437b9640 1 -- 172.21.15.152:0/242865147 shutdown_connections 2024-09-11T23:01:06.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.511+0000 7f58437b9640 1 --2- 172.21.15.152:0/242865147 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58280a2cf0 0x7f58280a51b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.511+0000 7f58437b9640 1 --2- 172.21.15.152:0/242865147 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f583c106f10 0x7f583c19cda0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.511+0000 7f58437b9640 1 --2- 172.21.15.152:0/242865147 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f583c102790 0x7f583c198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.511+0000 7f58437b9640 1 --2- 172.21.15.152:0/242865147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f583c101de0 0x7f583c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:06.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.511+0000 7f58437b9640 1 -- 172.21.15.152:0/242865147 >> 172.21.15.152:0/242865147 conn(0x7f583c0fd8c0 msgr2=0x7f583c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:06.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.511+0000 7f58437b9640 1 -- 172.21.15.152:0/242865147 shutdown_connections 2024-09-11T23:01:06.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:06.511+0000 7f58437b9640 1 -- 172.21.15.152:0/242865147 wait complete. 2024-09-11T23:01:06.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:06 smithi191 ceph-mon[35211]: pgmap v251: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-11T23:01:06.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:06 smithi204 ceph-mon[35454]: pgmap v251: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-11T23:01:07.468 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:01:07.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:07 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:01:07.726 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:01:07.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:07 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:01:07.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:07 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:01:08.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.225+0000 7faaea2da640 1 -- 172.21.15.152:0/2581229528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae41027a0 msgr2=0x7faae41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:08.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.225+0000 7faaea2da640 1 --2- 172.21.15.152:0/2581229528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae41027a0 0x7faae41067f0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7faacc009900 tx=0x7faacc02a410 comp rx=0 tx=0).stop 2024-09-11T23:01:08.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.225+0000 7faaea2da640 1 -- 172.21.15.152:0/2581229528 shutdown_connections 2024-09-11T23:01:08.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.225+0000 7faaea2da640 1 --2- 172.21.15.152:0/2581229528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faae4106f20 0x7faae410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.225+0000 7faaea2da640 1 --2- 172.21.15.152:0/2581229528 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae41027a0 0x7faae41067f0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.225+0000 7faaea2da640 1 --2- 172.21.15.152:0/2581229528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faae4101df0 0x7faae41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.225+0000 7faaea2da640 1 -- 172.21.15.152:0/2581229528 >> 172.21.15.152:0/2581229528 conn(0x7faae40fd8f0 msgr2=0x7faae40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:08.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.225+0000 7faaea2da640 1 -- 172.21.15.152:0/2581229528 shutdown_connections 2024-09-11T23:01:08.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.225+0000 7faaea2da640 1 -- 172.21.15.152:0/2581229528 wait complete. 2024-09-11T23:01:08.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.225+0000 7faaea2da640 1 Processor -- start 2024-09-11T23:01:08.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.225+0000 7faaea2da640 1 -- start start 2024-09-11T23:01:08.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.225+0000 7faaea2da640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faae4101df0 0x7faae4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.225+0000 7faaea2da640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faae41027a0 0x7faae4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faaea2da640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae4106f20 0x7faae419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faaea2da640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faae4111f80 con 0x7faae4106f20 2024-09-11T23:01:08.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faaea2da640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faae4111e00 con 0x7faae41027a0 2024-09-11T23:01:08.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faaea2da640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faae4112100 con 0x7faae4101df0 2024-09-11T23:01:08.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faae8877640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae4106f20 0x7faae419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faae8877640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae4106f20 0x7faae419cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43596/0 (socket says 172.21.15.152:43596) 2024-09-11T23:01:08.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faadbfff640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faae41027a0 0x7faae4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faae8877640 1 -- 172.21.15.152:0/3796254889 learned_addr learned my addr 172.21.15.152:0/3796254889 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:08.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faae3fff640 1 --2- 172.21.15.152:0/3796254889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faae4101df0 0x7faae4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faae8877640 1 -- 172.21.15.152:0/3796254889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faae4101df0 msgr2=0x7faae4198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:08.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faae8877640 1 --2- 172.21.15.152:0/3796254889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faae4101df0 0x7faae4198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faae8877640 1 -- 172.21.15.152:0/3796254889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faae41027a0 msgr2=0x7faae4198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:08.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faae8877640 1 --2- 172.21.15.152:0/3796254889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faae41027a0 0x7faae4198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faae8877640 1 -- 172.21.15.152:0/3796254889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae419d4b0 con 0x7faae4106f20 2024-09-11T23:01:08.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faae8877640 1 --2- 172.21.15.152:0/3796254889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae4106f20 0x7faae419cdd0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7faadc009fd0 tx=0x7faadc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:08.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faae1ffb640 1 -- 172.21.15.152:0/3796254889 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc017070 con 0x7faae4106f20 2024-09-11T23:01:08.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faaea2da640 1 -- 172.21.15.152:0/3796254889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae419d7a0 con 0x7faae4106f20 2024-09-11T23:01:08.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faae1ffb640 1 -- 172.21.15.152:0/3796254889 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faadc003e50 con 0x7faae4106f20 2024-09-11T23:01:08.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faae1ffb640 1 -- 172.21.15.152:0/3796254889 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc0106c0 con 0x7faae4106f20 2024-09-11T23:01:08.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.226+0000 7faaea2da640 1 -- 172.21.15.152:0/3796254889 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae41a5080 con 0x7faae4106f20 2024-09-11T23:01:08.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.229+0000 7faae1ffb640 1 -- 172.21.15.152:0/3796254889 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7faadc003970 con 0x7faae4106f20 2024-09-11T23:01:08.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.229+0000 7faaec6b3740 1 Processor -- start 2024-09-11T23:01:08.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.229+0000 7faaec6b3740 1 -- start start 2024-09-11T23:01:08.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.229+0000 7faaec6b3740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd82b40 0x55957cd81090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.229+0000 7faaec6b3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd815d0 0x55957cd81a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.229+0000 7faaec6b3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd90780 0x55957cd92b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faaec6b3740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55957cce72f0 con 0x55957cd82b40 2024-09-11T23:01:08.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faaec6b3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55957cc8ecf0 con 0x55957cd90780 2024-09-11T23:01:08.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faaec6b3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55957cce32b0 con 0x55957cd815d0 2024-09-11T23:01:08.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faadbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd82b40 0x55957cd81090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faadbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd82b40 0x55957cd81090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43608/0 (socket says 172.21.15.152:43608) 2024-09-11T23:01:08.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faae1ffb640 1 --2- 172.21.15.152:0/3796254889 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7faad00a2cf0 0x7faad00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.233 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faadbfff640 1 -- 172.21.15.152:0/2134632045 learned_addr learned my addr 172.21.15.152:0/2134632045 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faae3fff640 1 --2- 172.21.15.152:0/2134632045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd815d0 0x55957cd81a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faae8877640 1 --2- 172.21.15.152:0/3796254889 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7faad00a2cf0 0x7faad00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faadbfff640 1 --2- 172.21.15.152:0/2134632045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd90780 0x55957cd92b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faadbfff640 1 -- 172.21.15.152:0/2134632045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd815d0 msgr2=0x55957cd81a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faadbfff640 1 --2- 172.21.15.152:0/2134632045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd815d0 0x55957cd81a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faadbfff640 1 -- 172.21.15.152:0/2134632045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd90780 msgr2=0x55957cd92b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faadbfff640 1 --2- 172.21.15.152:0/2134632045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd90780 0x55957cd92b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faae1ffb640 1 -- 172.21.15.152:0/3796254889 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7faadc0cf150 con 0x7faae4106f20 2024-09-11T23:01:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faadbfff640 1 -- 172.21.15.152:0/2134632045 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55957cc733a0 con 0x55957cd82b40 2024-09-11T23:01:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faadbfff640 1 --2- 172.21.15.152:0/2134632045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd82b40 0x55957cd81090 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x55957cd82070 tx=0x7faacc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a1ce709b6a9753 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:08.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faae8877640 1 --2- 172.21.15.152:0/3796254889 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7faad00a2cf0 0x7faad00a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faadc0a3d70 tx=0x7faadc097980 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:08.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.230+0000 7faada7fc640 1 -- 172.21.15.152:0/2134632045 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc044030 con 0x55957cd82b40 2024-09-11T23:01:08.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.231+0000 7faada7fc640 1 -- 172.21.15.152:0/2134632045 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faacc005020 con 0x55957cd82b40 2024-09-11T23:01:08.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.231+0000 7faada7fc640 1 -- 172.21.15.152:0/2134632045 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc035040 con 0x55957cd82b40 2024-09-11T23:01:08.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.231+0000 7faaec6b3740 1 -- 172.21.15.152:0/2134632045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd82b40 msgr2=0x55957cd81090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:08.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.231+0000 7faaec6b3740 1 --2- 172.21.15.152:0/2134632045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd82b40 0x55957cd81090 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x55957cd82070 tx=0x7faacc000bc0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.231+0000 7faaec6b3740 1 -- 172.21.15.152:0/2134632045 shutdown_connections 2024-09-11T23:01:08.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.231+0000 7faaec6b3740 1 --2- 172.21.15.152:0/2134632045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd90780 0x55957cd92b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.231+0000 7faaec6b3740 1 --2- 172.21.15.152:0/2134632045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd815d0 0x55957cd81a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.231+0000 7faaec6b3740 1 --2- 172.21.15.152:0/2134632045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd82b40 0x55957cd81090 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.231+0000 7faaec6b3740 1 -- 172.21.15.152:0/2134632045 >> 172.21.15.152:0/2134632045 conn(0x55957cd4a310 msgr2=0x55957cd7dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:08.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.231+0000 7faaec6b3740 1 -- 172.21.15.152:0/2134632045 shutdown_connections 2024-09-11T23:01:08.235 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.231+0000 7faaec6b3740 1 -- 172.21.15.152:0/2134632045 wait complete. 2024-09-11T23:01:08.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.232+0000 7faaec6b3740 1 -- start start 2024-09-11T23:01:08.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.232+0000 7faaec6b3740 1 Processor -- start 2024-09-11T23:01:08.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.232+0000 7faaec6b3740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd815d0 0x55957cd90160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.232+0000 7faaec6b3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd82b40 0x55957cd8d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.232+0000 7faaec6b3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd90780 0x55957cd8d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.232+0000 7faaec6b3740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55957cd93420 con 0x55957cd815d0 2024-09-11T23:01:08.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.232+0000 7faaec6b3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55957cca2060 con 0x55957cd90780 2024-09-11T23:01:08.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.232+0000 7faaec6b3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55957cd935a0 con 0x55957cd82b40 2024-09-11T23:01:08.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.232+0000 7faaec6b3740 10 client.0 _ll_register_callbacks cb 0x55957c9f58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:08.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.232+0000 7faadbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd815d0 0x55957cd90160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.239 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.233+0000 7faadbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd815d0 0x55957cd90160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43624/0 (socket says 172.21.15.152:43624) 2024-09-11T23:01:08.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.233+0000 7faadbfff640 1 -- 172.21.15.152:0/3380170296 learned_addr learned my addr 172.21.15.152:0/3380170296 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:08.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.233+0000 7faae3fff640 1 --2- 172.21.15.152:0/3380170296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd82b40 0x55957cd8d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.233+0000 7faadbfff640 1 -- 172.21.15.152:0/3380170296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd82b40 msgr2=0x55957cd8d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:08.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.233+0000 7faadbfff640 1 --2- 172.21.15.152:0/3380170296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd82b40 0x55957cd8d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.233+0000 7faadbfff640 1 -- 172.21.15.152:0/3380170296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd90780 msgr2=0x55957cd8d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:08.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.233+0000 7faadbfff640 1 --2- 172.21.15.152:0/3380170296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd90780 0x55957cd8d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.233+0000 7faadbfff640 1 -- 172.21.15.152:0/3380170296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957ca583c0 con 0x55957cd815d0 2024-09-11T23:01:08.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.233+0000 7faadbfff640 1 --2- 172.21.15.152:0/3380170296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd815d0 0x55957cd90160 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7faacc02aba0 tx=0x7faacc0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af65a96bb917fe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:08.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.233+0000 7faac37fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:08.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.233+0000 7faac37fe640 1 -- 172.21.15.152:0/3380170296 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc035030 con 0x55957cd815d0 2024-09-11T23:01:08.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.233+0000 7faac37fe640 1 -- 172.21.15.152:0/3380170296 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7faacc004050 con 0x55957cd815d0 2024-09-11T23:01:08.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.233+0000 7faaec6b3740 1 -- 172.21.15.152:0/3380170296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faacc005020 con 0x55957cd815d0 2024-09-11T23:01:08.240 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.233+0000 7faac37fe640 1 -- 172.21.15.152:0/3380170296 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faacc057050 con 0x55957cd815d0 2024-09-11T23:01:08.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.233+0000 7faaec6b3740 10 client.17796 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:08.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.233+0000 7faaec6b3740 1 -- 172.21.15.152:0/3380170296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55957cd9eea0 con 0x55957cd815d0 2024-09-11T23:01:08.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.234+0000 7faac37fe640 1 -- 172.21.15.152:0/3380170296 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faacc005020 con 0x55957cd815d0 2024-09-11T23:01:08.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.234+0000 7faac37fe640 1 -- 172.21.15.152:0/3380170296 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faacc003660 con 0x55957cd815d0 2024-09-11T23:01:08.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.234+0000 7faaec6b3740 10 client.17796 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:08.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.234+0000 7faaec6b3740 10 client.17796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:01:08.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.234+0000 7faaec6b3740 20 client.17796 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:08.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.234+0000 7faaec6b3740 1 --2- 172.21.15.152:0/3380170296 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55957cda01f0 0x55957cda25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.234+0000 7faaec6b3740 4 client.17796 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:01:08.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.234+0000 7faaec6b3740 1 -- 172.21.15.152:0/3380170296 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55957ccf4860 con 0x55957cda01f0 2024-09-11T23:01:08.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.234+0000 7faae3fff640 1 --2- 172.21.15.152:0/3380170296 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55957cda01f0 0x55957cda25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.234+0000 7faac37fe640 10 client.17796 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:08.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.234+0000 7faae3fff640 1 --2- 172.21.15.152:0/3380170296 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55957cda01f0 0x55957cda25e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4632e9f458a935d server_cookie=c345311e5a25e575 in_seq=0 out_seq=0 2024-09-11T23:01:08.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.235+0000 7faac37fe640 1 -- 172.21.15.152:0/3380170296 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55957ccf4860 con 0x55957cda01f0 2024-09-11T23:01:08.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.235+0000 7faac37fe640 10 client.17796 handle_command_reply: tid=0 2024-09-11T23:01:08.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.235+0000 7faaec6b3740 1 client.17796 shutdown 2024-09-11T23:01:08.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.235+0000 7faaec6b3740 1 -- 172.21.15.152:0/3380170296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd815d0 msgr2=0x55957cd90160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:08.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.235+0000 7faaec6b3740 1 --2- 172.21.15.152:0/3380170296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd815d0 0x55957cd90160 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7faacc02aba0 tx=0x7faacc0315b0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.235+0000 7faaec6b3740 1 -- 172.21.15.152:0/3380170296 shutdown_connections 2024-09-11T23:01:08.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.235+0000 7faaec6b3740 1 --2- 172.21.15.152:0/3380170296 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55957cda01f0 0x55957cda25e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.235+0000 7faaec6b3740 1 --2- 172.21.15.152:0/3380170296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd90780 0x55957cd8d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.235+0000 7faac37fe640 0 client.17796 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:08.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.235+0000 7faaec6b3740 1 --2- 172.21.15.152:0/3380170296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd82b40 0x55957cd8d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.235+0000 7faaec6b3740 1 --2- 172.21.15.152:0/3380170296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd815d0 0x55957cd90160 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.235+0000 7faaec6b3740 1 -- 172.21.15.152:0/3380170296 >> 172.21.15.152:0/3380170296 conn(0x55957cd4a310 msgr2=0x55957cd83960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:08.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.235+0000 7faaec6b3740 1 -- 172.21.15.152:0/3380170296 shutdown_connections 2024-09-11T23:01:08.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.235+0000 7faaec6b3740 1 -- 172.21.15.152:0/3380170296 wait complete. 2024-09-11T23:01:08.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.235+0000 7faaec6b3740 20 client.17796 trim_cache size 0 max 16384 2024-09-11T23:01:08.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.248+0000 7faaec6b3740 1 Processor -- start 2024-09-11T23:01:08.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.248+0000 7faaec6b3740 1 -- start start 2024-09-11T23:01:08.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faaec6b3740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd815d0 0x55957cd7de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faaec6b3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd82b40 0x55957cd9c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faaec6b3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd90780 0x55957cd8ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faaec6b3740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faae41ab500 con 0x55957cd815d0 2024-09-11T23:01:08.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faaec6b3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faae41ad040 con 0x55957cd82b40 2024-09-11T23:01:08.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faaec6b3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faae41abbd0 con 0x55957cd90780 2024-09-11T23:01:08.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faadbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd815d0 0x55957cd7de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faadbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd815d0 0x55957cd7de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43632/0 (socket says 172.21.15.152:43632) 2024-09-11T23:01:08.250 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faadbfff640 1 -- 172.21.15.152:0/3207214604 learned_addr learned my addr 172.21.15.152:0/3207214604 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:08.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faae3fff640 1 --2- 172.21.15.152:0/3207214604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd82b40 0x55957cd9c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faadbfff640 1 --2- 172.21.15.152:0/3207214604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd90780 0x55957cd8ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faadbfff640 1 -- 172.21.15.152:0/3207214604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd90780 msgr2=0x55957cd8ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:08.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faadbfff640 1 --2- 172.21.15.152:0/3207214604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd90780 0x55957cd8ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faadbfff640 1 -- 172.21.15.152:0/3207214604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd82b40 msgr2=0x55957cd9c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:08.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faadbfff640 1 --2- 172.21.15.152:0/3207214604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd82b40 0x55957cd9c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faadbfff640 1 -- 172.21.15.152:0/3207214604 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55957cd8c3c0 con 0x55957cd815d0 2024-09-11T23:01:08.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faadbfff640 1 --2- 172.21.15.152:0/3207214604 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd815d0 0x55957cd7de30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7faacc02a450 tx=0x7faacc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8bd60c6cf716c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:08.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faad9ffb640 1 -- 172.21.15.152:0/3207214604 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc035070 con 0x55957cd815d0 2024-09-11T23:01:08.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faad9ffb640 1 -- 172.21.15.152:0/3207214604 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faacc04e070 con 0x55957cd815d0 2024-09-11T23:01:08.251 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.249+0000 7faad9ffb640 1 -- 172.21.15.152:0/3207214604 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc044040 con 0x55957cd815d0 2024-09-11T23:01:08.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.250+0000 7faaec6b3740 1 -- 172.21.15.152:0/3207214604 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd815d0 msgr2=0x55957cd7de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:08.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.250+0000 7faaec6b3740 1 --2- 172.21.15.152:0/3207214604 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd815d0 0x55957cd7de30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7faacc02a450 tx=0x7faacc005fd0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.250+0000 7faaec6b3740 1 -- 172.21.15.152:0/3207214604 shutdown_connections 2024-09-11T23:01:08.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.250+0000 7faaec6b3740 1 --2- 172.21.15.152:0/3207214604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd90780 0x55957cd8ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.250+0000 7faaec6b3740 1 --2- 172.21.15.152:0/3207214604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd82b40 0x55957cd9c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.250+0000 7faaec6b3740 1 --2- 172.21.15.152:0/3207214604 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd815d0 0x55957cd7de30 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.250+0000 7faaec6b3740 1 -- 172.21.15.152:0/3207214604 >> 172.21.15.152:0/3207214604 conn(0x55957cd4a310 msgr2=0x55957cd7c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:08.256 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.250+0000 7faaec6b3740 1 -- 172.21.15.152:0/3207214604 shutdown_connections 2024-09-11T23:01:08.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.250+0000 7faaec6b3740 1 -- 172.21.15.152:0/3207214604 wait complete. 2024-09-11T23:01:08.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.250+0000 7faaec6b3740 1 -- start start 2024-09-11T23:01:08.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faaec6b3740 1 Processor -- start 2024-09-11T23:01:08.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faaec6b3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd815d0 0x55957cd944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faaec6b3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd82b40 0x55957cd949f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faaec6b3740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd90780 0x55957cd94f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faaec6b3740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7faae41ac970 con 0x55957cd90780 2024-09-11T23:01:08.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faaec6b3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7faae41ac2a0 con 0x55957cd815d0 2024-09-11T23:01:08.257 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faaec6b3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7faae41a9500 con 0x55957cd82b40 2024-09-11T23:01:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faaec6b3740 10 client.0 _ll_register_callbacks cb 0x7faad4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faadbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd90780 0x55957cd94f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faadbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd90780 0x55957cd94f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43646/0 (socket says 172.21.15.152:43646) 2024-09-11T23:01:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faadbfff640 1 -- 172.21.15.152:0/2489348711 learned_addr learned my addr 172.21.15.152:0/2489348711 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faae3fff640 1 --2- 172.21.15.152:0/2489348711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd82b40 0x55957cd949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faadbfff640 1 --2- 172.21.15.152:0/2489348711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd815d0 0x55957cd944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faadbfff640 1 -- 172.21.15.152:0/2489348711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd82b40 msgr2=0x55957cd949f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faadbfff640 1 --2- 172.21.15.152:0/2489348711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd82b40 0x55957cd949f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faadbfff640 1 -- 172.21.15.152:0/2489348711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd815d0 msgr2=0x55957cd944b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faadbfff640 1 --2- 172.21.15.152:0/2489348711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd815d0 0x55957cd944b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.251+0000 7faadbfff640 1 -- 172.21.15.152:0/2489348711 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55957cd8dd50 con 0x55957cd90780 2024-09-11T23:01:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.252+0000 7faadbfff640 1 --2- 172.21.15.152:0/2489348711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd90780 0x55957cd94f30 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7faacc031470 tx=0x7faacc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1102bd3951a2861 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:08.258 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.252+0000 7faac37fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:08.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.252+0000 7faac37fe640 1 -- 172.21.15.152:0/2489348711 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc035070 con 0x55957cd90780 2024-09-11T23:01:08.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.252+0000 7faac37fe640 1 -- 172.21.15.152:0/2489348711 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8492+0+0 (secure 0 0 0) 0x7faacc0037f0 con 0x55957cd90780 2024-09-11T23:01:08.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.252+0000 7faaec6b3740 1 -- 172.21.15.152:0/2489348711 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faacc04e070 con 0x55957cd90780 2024-09-11T23:01:08.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.252+0000 7faac37fe640 1 -- 172.21.15.152:0/2489348711 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faacc070050 con 0x55957cd90780 2024-09-11T23:01:08.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.252+0000 7faaec6b3740 10 client.17802 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:08.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.252+0000 7faaec6b3740 1 -- 172.21.15.152:0/2489348711 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55957ce0abb0 con 0x55957cd90780 2024-09-11T23:01:08.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.252+0000 7faac37fe640 1 -- 172.21.15.152:0/2489348711 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faacc04e070 con 0x55957cd90780 2024-09-11T23:01:08.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.252+0000 7faac37fe640 1 -- 172.21.15.152:0/2489348711 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7faacc049bb0 con 0x55957cd90780 2024-09-11T23:01:08.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.252+0000 7faaec6b3740 10 client.17802 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:08.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.252+0000 7faaec6b3740 10 client.17802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:01:08.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.252+0000 7faaec6b3740 20 client.17802 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:08.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.252+0000 7faaec6b3740 1 --2- 172.21.15.152:0/2489348711 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55957cda01f0 0x55957ce0d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:08.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.252+0000 7faaec6b3740 4 client.17802 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:01:08.259 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.252+0000 7faaec6b3740 1 -- 172.21.15.152:0/2489348711 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55957cceed70 con 0x55957cda01f0 2024-09-11T23:01:08.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.253+0000 7faae3fff640 1 --2- 172.21.15.152:0/2489348711 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55957cda01f0 0x55957ce0d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:08.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.253+0000 7faae3fff640 1 --2- 172.21.15.152:0/2489348711 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55957cda01f0 0x55957ce0d1d0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4cea6d1c31d197b server_cookie=963b441dc0ff7051 in_seq=0 out_seq=0 2024-09-11T23:01:08.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.253+0000 7faac37fe640 10 client.17802 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:08.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.253+0000 7faac37fe640 1 -- 172.21.15.152:0/2489348711 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55957cceed70 con 0x55957cda01f0 2024-09-11T23:01:08.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.253+0000 7faac37fe640 10 client.17802 handle_command_reply: tid=0 2024-09-11T23:01:08.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.253+0000 7faaec6b3740 1 client.17802 shutdown 2024-09-11T23:01:08.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaec6b3740 1 -- 172.21.15.152:0/2489348711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd90780 msgr2=0x55957cd94f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:08.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaec6b3740 1 --2- 172.21.15.152:0/2489348711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd90780 0x55957cd94f30 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7faacc031470 tx=0x7faacc005a20 comp rx=0 tx=0).stop 2024-09-11T23:01:08.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaec6b3740 1 -- 172.21.15.152:0/2489348711 shutdown_connections 2024-09-11T23:01:08.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaec6b3740 1 --2- 172.21.15.152:0/2489348711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55957cd90780 0x55957cd94f30 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaec6b3740 1 --2- 172.21.15.152:0/2489348711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55957cd82b40 0x55957cd949f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaec6b3740 1 --2- 172.21.15.152:0/2489348711 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55957cda01f0 0x55957ce0d1d0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.260 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaec6b3740 1 --2- 172.21.15.152:0/2489348711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55957cd815d0 0x55957cd944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaec6b3740 1 -- 172.21.15.152:0/2489348711 >> 172.21.15.152:0/2489348711 conn(0x55957cd4a310 msgr2=0x55957cd8f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:08.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faac37fe640 0 client.17802 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:08.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaec6b3740 1 -- 172.21.15.152:0/2489348711 shutdown_connections 2024-09-11T23:01:08.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaec6b3740 1 -- 172.21.15.152:0/2489348711 wait complete. 2024-09-11T23:01:08.261 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaec6b3740 20 client.17802 trim_cache size 0 max 16384 2024-09-11T23:01:08.261 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:01:08.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaea2da640 1 -- 172.21.15.152:0/3796254889 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7faad00a2cf0 msgr2=0x7faad00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:08.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaea2da640 1 --2- 172.21.15.152:0/3796254889 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7faad00a2cf0 0x7faad00a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faadc0a3d70 tx=0x7faadc097980 comp rx=0 tx=0).stop 2024-09-11T23:01:08.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaea2da640 1 -- 172.21.15.152:0/3796254889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae4106f20 msgr2=0x7faae419cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:08.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaea2da640 1 --2- 172.21.15.152:0/3796254889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae4106f20 0x7faae419cdd0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7faadc009fd0 tx=0x7faadc00b260 comp rx=0 tx=0).stop 2024-09-11T23:01:08.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaea2da640 1 -- 172.21.15.152:0/3796254889 shutdown_connections 2024-09-11T23:01:08.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaea2da640 1 --2- 172.21.15.152:0/3796254889 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7faad00a2cf0 0x7faad00a51b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaea2da640 1 --2- 172.21.15.152:0/3796254889 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faae4106f20 0x7faae419cdd0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaea2da640 1 --2- 172.21.15.152:0/3796254889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faae41027a0 0x7faae4198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.262 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaea2da640 1 --2- 172.21.15.152:0/3796254889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faae4101df0 0x7faae4198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:08.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.254+0000 7faaea2da640 1 -- 172.21.15.152:0/3796254889 >> 172.21.15.152:0/3796254889 conn(0x7faae40fd8f0 msgr2=0x7faae40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:08.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.255+0000 7faaea2da640 1 -- 172.21.15.152:0/3796254889 shutdown_connections 2024-09-11T23:01:08.263 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:08.255+0000 7faaea2da640 1 -- 172.21.15.152:0/3796254889 wait complete. 2024-09-11T23:01:08.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:08 smithi152 ceph-mon[31181]: pgmap v252: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-11T23:01:08.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:08 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:01:08.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:08 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:01:08.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:08 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:01:08.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:08 smithi191 ceph-mon[35211]: pgmap v252: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-11T23:01:08.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:08 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:01:08.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:08 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:01:08.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:08 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:01:08.795 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:08 smithi204 ceph-mon[35454]: pgmap v252: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-11T23:01:08.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:08 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:01:08.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:08 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:01:08.796 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:08 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:01:09.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:09 smithi152 ceph-mon[31181]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T23:01:10.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:09 smithi191 ceph-mon[35211]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T23:01:10.041 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:01:10.041 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:01:10.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:09 smithi204 ceph-mon[35454]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T23:01:10.293 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:01:10.648 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:10 smithi152 ceph-mon[31181]: pgmap v254: 97 pgs: 15 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 58 active+clean; 6.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:01:10.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.808+0000 7fb8dc60c640 1 -- 172.21.15.152:0/1605772606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8d4100e20 msgr2=0x7fb8d4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:10.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.808+0000 7fb8dc60c640 1 --2- 172.21.15.152:0/1605772606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8d4100e20 0x7fb8d4101200 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009ca0 tx=0x7fb8c402a950 comp rx=0 tx=0).stop 2024-09-11T23:01:10.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.808+0000 7fb8dc60c640 1 -- 172.21.15.152:0/1605772606 shutdown_connections 2024-09-11T23:01:10.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.808+0000 7fb8dc60c640 1 --2- 172.21.15.152:0/1605772606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d4102ef0 0x7fb8d410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.808+0000 7fb8dc60c640 1 --2- 172.21.15.152:0/1605772606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8d4102570 0x7fb8d41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.808+0000 7fb8dc60c640 1 --2- 172.21.15.152:0/1605772606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8d4100e20 0x7fb8d4101200 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.808+0000 7fb8dc60c640 1 -- 172.21.15.152:0/1605772606 >> 172.21.15.152:0/1605772606 conn(0x7fb8d40fc7d0 msgr2=0x7fb8d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:10.813 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.808+0000 7fb8dc60c640 1 -- 172.21.15.152:0/1605772606 shutdown_connections 2024-09-11T23:01:10.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.808+0000 7fb8dc60c640 1 -- 172.21.15.152:0/1605772606 wait complete. 2024-09-11T23:01:10.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8dc60c640 1 Processor -- start 2024-09-11T23:01:10.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8dc60c640 1 -- start start 2024-09-11T23:01:10.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8dc60c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8d4100e20 0x7fb8d4199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8dc60c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8d4102570 0x7fb8d4199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8dc60c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d4102ef0 0x7fb8d41a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8dc60c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d4110fa0 con 0x7fb8d4100e20 2024-09-11T23:01:10.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8dc60c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d4110e20 con 0x7fb8d4102570 2024-09-11T23:01:10.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8dc60c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d4111120 con 0x7fb8d4102ef0 2024-09-11T23:01:10.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8da3a8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8d4100e20 0x7fb8d4199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8da3a8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8d4100e20 0x7fb8d4199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47012/0 (socket says 172.21.15.152:47012) 2024-09-11T23:01:10.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8da3a8640 1 -- 172.21.15.152:0/1610856775 learned_addr learned my addr 172.21.15.152:0/1610856775 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:10.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8d9ba7640 1 --2- 172.21.15.152:0/1610856775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8d4102570 0x7fb8d4199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.814 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8daba9640 1 --2- 172.21.15.152:0/1610856775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d4102ef0 0x7fb8d41a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8da3a8640 1 -- 172.21.15.152:0/1610856775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d4102ef0 msgr2=0x7fb8d41a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:10.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8da3a8640 1 --2- 172.21.15.152:0/1610856775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d4102ef0 0x7fb8d41a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8da3a8640 1 -- 172.21.15.152:0/1610856775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8d4102570 msgr2=0x7fb8d4199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:10.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8da3a8640 1 --2- 172.21.15.152:0/1610856775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8d4102570 0x7fb8d4199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8da3a8640 1 -- 172.21.15.152:0/1610856775 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d41a48d0 con 0x7fb8d4100e20 2024-09-11T23:01:10.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.809+0000 7fb8daba9640 1 --2- 172.21.15.152:0/1610856775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d4102ef0 0x7fb8d41a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:01:10.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.810+0000 7fb8da3a8640 1 --2- 172.21.15.152:0/1610856775 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8d4100e20 0x7fb8d4199310 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4005cc0 tx=0x7fb8c402f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:10.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.810+0000 7fb8d37fe640 1 -- 172.21.15.152:0/1610856775 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4034040 con 0x7fb8d4100e20 2024-09-11T23:01:10.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.810+0000 7fb8dc60c640 1 -- 172.21.15.152:0/1610856775 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d41a4aa0 con 0x7fb8d4100e20 2024-09-11T23:01:10.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.810+0000 7fb8d37fe640 1 -- 172.21.15.152:0/1610856775 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c4030920 con 0x7fb8d4100e20 2024-09-11T23:01:10.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.810+0000 7fb8d37fe640 1 -- 172.21.15.152:0/1610856775 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4045480 con 0x7fb8d4100e20 2024-09-11T23:01:10.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.810+0000 7fb8dc60c640 1 -- 172.21.15.152:0/1610856775 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d41a4f50 con 0x7fb8d4100e20 2024-09-11T23:01:10.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.813+0000 7fb8de9e5740 1 Processor -- start 2024-09-11T23:01:10.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.813+0000 7fb8de9e5740 1 -- start start 2024-09-11T23:01:10.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.813+0000 7fb8d37fe640 1 -- 172.21.15.152:0/1610856775 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb8c4037020 con 0x7fb8d4100e20 2024-09-11T23:01:10.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.813+0000 7fb8de9e5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab233b60 0x55b6ab233f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.813+0000 7fb8de9e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab234480 0x55b6ab23e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.813+0000 7fb8de9e5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab23ed40 0x55b6ab241130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.813+0000 7fb8de9e5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55b6ab199280 con 0x55b6ab233b60 2024-09-11T23:01:10.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.813+0000 7fb8de9e5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b6ab140b10 con 0x55b6ab23ed40 2024-09-11T23:01:10.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.813+0000 7fb8de9e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b6ab195240 con 0x55b6ab234480 2024-09-11T23:01:10.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.813+0000 7fb8daba9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab233b60 0x55b6ab233f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.813+0000 7fb8daba9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab233b60 0x55b6ab233f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47014/0 (socket says 172.21.15.152:47014) 2024-09-11T23:01:10.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.813+0000 7fb8daba9640 1 -- 172.21.15.152:0/4037188975 learned_addr learned my addr 172.21.15.152:0/4037188975 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:10.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.814+0000 7fb8d9ba7640 1 --2- 172.21.15.152:0/4037188975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab23ed40 0x55b6ab241130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.814+0000 7fb8da3a8640 1 --2- 172.21.15.152:0/4037188975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab234480 0x55b6ab23e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.814+0000 7fb8daba9640 1 -- 172.21.15.152:0/4037188975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab234480 msgr2=0x55b6ab23e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:10.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.814+0000 7fb8daba9640 1 --2- 172.21.15.152:0/4037188975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab234480 0x55b6ab23e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.814+0000 7fb8daba9640 1 -- 172.21.15.152:0/4037188975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab23ed40 msgr2=0x55b6ab241130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:10.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.814+0000 7fb8daba9640 1 --2- 172.21.15.152:0/4037188975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab23ed40 0x55b6ab241130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.822 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.814+0000 7fb8daba9640 1 -- 172.21.15.152:0/4037188975 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ab14d1f0 con 0x55b6ab233b60 2024-09-11T23:01:10.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.814+0000 7fb8da3a8640 1 --2- 172.21.15.152:0/4037188975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab234480 0x55b6ab23e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:01:10.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.815+0000 7fb8daba9640 1 --2- 172.21.15.152:0/4037188975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab233b60 0x55b6ab233f40 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00a990 tx=0x7fb8cc00ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34e647b5216b141 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:10.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.815+0000 7fb8d37fe640 1 --2- 172.21.15.152:0/1610856775 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb8bc0a2cf0 0x7fb8bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.815+0000 7fb8da3a8640 1 --2- 172.21.15.152:0/1610856775 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb8bc0a2cf0 0x7fb8bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.815+0000 7fb8d0ff9640 1 -- 172.21.15.152:0/4037188975 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc015070 con 0x55b6ab233b60 2024-09-11T23:01:10.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.815+0000 7fb8d0ff9640 1 -- 172.21.15.152:0/4037188975 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8cc00cbe0 con 0x55b6ab233b60 2024-09-11T23:01:10.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.815+0000 7fb8d0ff9640 1 -- 172.21.15.152:0/4037188975 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc0186d0 con 0x55b6ab233b60 2024-09-11T23:01:10.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.815+0000 7fb8d37fe640 1 -- 172.21.15.152:0/1610856775 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7fb8c40f0580 con 0x7fb8d4100e20 2024-09-11T23:01:10.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.816+0000 7fb8de9e5740 1 -- 172.21.15.152:0/4037188975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab233b60 msgr2=0x55b6ab233f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:10.823 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.816+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/4037188975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab233b60 0x55b6ab233f40 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00a990 tx=0x7fb8cc00ae50 comp rx=0 tx=0).stop 2024-09-11T23:01:10.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.816+0000 7fb8da3a8640 1 --2- 172.21.15.152:0/1610856775 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb8bc0a2cf0 0x7fb8bc0a51b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40ede80 tx=0x7fb8c40f5040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:10.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.816+0000 7fb8de9e5740 1 -- 172.21.15.152:0/4037188975 shutdown_connections 2024-09-11T23:01:10.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.816+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/4037188975 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab23ed40 0x55b6ab241130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.816+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/4037188975 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab234480 0x55b6ab23e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.816+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/4037188975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab233b60 0x55b6ab233f40 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.816+0000 7fb8de9e5740 1 -- 172.21.15.152:0/4037188975 >> 172.21.15.152:0/4037188975 conn(0x55b6ab2300f0 msgr2=0x55b6ab2304d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:10.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.816+0000 7fb8de9e5740 1 -- 172.21.15.152:0/4037188975 shutdown_connections 2024-09-11T23:01:10.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.816+0000 7fb8de9e5740 1 -- 172.21.15.152:0/4037188975 wait complete. 2024-09-11T23:01:10.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.816+0000 7fb8de9e5740 1 -- start start 2024-09-11T23:01:10.824 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.816+0000 7fb8de9e5740 1 Processor -- start 2024-09-11T23:01:10.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8de9e5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab233b60 0x55b6ab23bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8de9e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab234480 0x55b6ab23c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8de9e5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab23c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8de9e5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55b6ab241910 con 0x55b6ab23ed40 2024-09-11T23:01:10.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8de9e5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b6ab153f10 con 0x55b6ab233b60 2024-09-11T23:01:10.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8de9e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b6ab241a90 con 0x55b6ab234480 2024-09-11T23:01:10.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8de9e5740 10 client.0 _ll_register_callbacks cb 0x55b6ab11ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:10.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8daba9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab23c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8daba9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab23c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47016/0 (socket says 172.21.15.152:47016) 2024-09-11T23:01:10.825 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8daba9640 1 -- 172.21.15.152:0/3711732778 learned_addr learned my addr 172.21.15.152:0/3711732778 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:10.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8daba9640 1 --2- 172.21.15.152:0/3711732778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab233b60 0x55b6ab23bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8d9ba7640 1 --2- 172.21.15.152:0/3711732778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab234480 0x55b6ab23c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8daba9640 1 -- 172.21.15.152:0/3711732778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab234480 msgr2=0x55b6ab23c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:10.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8daba9640 1 --2- 172.21.15.152:0/3711732778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab234480 0x55b6ab23c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8daba9640 1 -- 172.21.15.152:0/3711732778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab233b60 msgr2=0x55b6ab23bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:10.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8daba9640 1 --2- 172.21.15.152:0/3711732778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab233b60 0x55b6ab23bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8daba9640 1 -- 172.21.15.152:0/3711732778 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6aaf0a6a0 con 0x55b6ab23ed40 2024-09-11T23:01:10.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.817+0000 7fb8daba9640 1 --2- 172.21.15.152:0/3711732778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab23c630 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc011d70 tx=0x7fb8cc011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f738ce1387e7f99 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:10.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.818+0000 7fb8b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:10.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.818+0000 7fb8b9ffb640 1 -- 172.21.15.152:0/3711732778 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc015070 con 0x55b6ab23ed40 2024-09-11T23:01:10.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.818+0000 7fb8b9ffb640 1 -- 172.21.15.152:0/3711732778 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7fb8cc018b00 con 0x55b6ab23ed40 2024-09-11T23:01:10.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.818+0000 7fb8de9e5740 1 -- 172.21.15.152:0/3711732778 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8cc00cbe0 con 0x55b6ab23ed40 2024-09-11T23:01:10.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.818+0000 7fb8b9ffb640 1 -- 172.21.15.152:0/3711732778 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb8cc018d00 con 0x55b6ab23ed40 2024-09-11T23:01:10.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.818+0000 7fb8de9e5740 10 client.17814 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:10.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.818+0000 7fb8de9e5740 1 -- 172.21.15.152:0/3711732778 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6ab24d4a0 con 0x55b6ab23ed40 2024-09-11T23:01:10.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.818+0000 7fb8b9ffb640 1 -- 172.21.15.152:0/3711732778 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8cc00cbe0 con 0x55b6ab23ed40 2024-09-11T23:01:10.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.818+0000 7fb8b9ffb640 1 -- 172.21.15.152:0/3711732778 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb8cc026020 con 0x55b6ab23ed40 2024-09-11T23:01:10.827 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.818+0000 7fb8de9e5740 10 client.17814 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:10.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.818+0000 7fb8de9e5740 10 client.17814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:01:10.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.818+0000 7fb8de9e5740 20 client.17814 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:10.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.818+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/3711732778 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b6ab24e7f0 0x55b6ab250be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.818+0000 7fb8de9e5740 4 client.17814 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:01:10.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.818+0000 7fb8de9e5740 1 -- 172.21.15.152:0/3711732778 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6ab1a67f0 con 0x55b6ab24e7f0 2024-09-11T23:01:10.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.819+0000 7fb8d9ba7640 1 --2- 172.21.15.152:0/3711732778 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b6ab24e7f0 0x55b6ab250be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.819+0000 7fb8d9ba7640 1 --2- 172.21.15.152:0/3711732778 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b6ab24e7f0 0x55b6ab250be0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7971f7d86cfafff server_cookie=a1a28d3ce7457bf9 in_seq=0 out_seq=0 2024-09-11T23:01:10.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.819+0000 7fb8b9ffb640 10 client.17814 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:10.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.820+0000 7fb8b9ffb640 1 -- 172.21.15.152:0/3711732778 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6ab1a67f0 con 0x55b6ab24e7f0 2024-09-11T23:01:10.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.820+0000 7fb8b9ffb640 10 client.17814 handle_command_reply: tid=0 2024-09-11T23:01:10.828 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.820+0000 7fb8de9e5740 1 client.17814 shutdown 2024-09-11T23:01:10.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.820+0000 7fb8de9e5740 1 -- 172.21.15.152:0/3711732778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 msgr2=0x55b6ab23c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:10.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.820+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/3711732778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab23c630 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc011d70 tx=0x7fb8cc011e70 comp rx=0 tx=0).stop 2024-09-11T23:01:10.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.820+0000 7fb8de9e5740 1 -- 172.21.15.152:0/3711732778 shutdown_connections 2024-09-11T23:01:10.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.820+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/3711732778 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab23c630 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.820+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/3711732778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab234480 0x55b6ab23c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.820+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/3711732778 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b6ab24e7f0 0x55b6ab250be0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.820+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/3711732778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab233b60 0x55b6ab23bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.820+0000 7fb8b9ffb640 0 client.17814 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:10.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.820+0000 7fb8de9e5740 1 -- 172.21.15.152:0/3711732778 >> 172.21.15.152:0/3711732778 conn(0x55b6ab2300f0 msgr2=0x55b6ab22f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:10.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.820+0000 7fb8de9e5740 1 -- 172.21.15.152:0/3711732778 shutdown_connections 2024-09-11T23:01:10.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.820+0000 7fb8de9e5740 1 -- 172.21.15.152:0/3711732778 wait complete. 2024-09-11T23:01:10.830 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.820+0000 7fb8de9e5740 20 client.17814 trim_cache size 0 max 16384 2024-09-11T23:01:10.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.837+0000 7fb8de9e5740 1 Processor -- start 2024-09-11T23:01:10.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.837+0000 7fb8de9e5740 1 -- start start 2024-09-11T23:01:10.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8de9e5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab233b60 0x55b6ab2365c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8de9e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab234480 0x55b6ab23aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8de9e5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab2562d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8de9e5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d41ad820 con 0x55b6ab23ed40 2024-09-11T23:01:10.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8de9e5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d41af360 con 0x55b6ab233b60 2024-09-11T23:01:10.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8de9e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d41adef0 con 0x55b6ab234480 2024-09-11T23:01:10.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8daba9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab2562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8daba9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab2562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47022/0 (socket says 172.21.15.152:47022) 2024-09-11T23:01:10.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8daba9640 1 -- 172.21.15.152:0/2763418000 learned_addr learned my addr 172.21.15.152:0/2763418000 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:10.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8daba9640 1 --2- 172.21.15.152:0/2763418000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab233b60 0x55b6ab2365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8d9ba7640 1 --2- 172.21.15.152:0/2763418000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab234480 0x55b6ab23aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8daba9640 1 -- 172.21.15.152:0/2763418000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab234480 msgr2=0x55b6ab23aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:10.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8daba9640 1 --2- 172.21.15.152:0/2763418000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab234480 0x55b6ab23aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8daba9640 1 -- 172.21.15.152:0/2763418000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab233b60 msgr2=0x55b6ab2365c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:10.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8daba9640 1 --2- 172.21.15.152:0/2763418000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab233b60 0x55b6ab2365c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8daba9640 1 -- 172.21.15.152:0/2763418000 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ab23e190 con 0x55b6ab23ed40 2024-09-11T23:01:10.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.838+0000 7fb8daba9640 1 --2- 172.21.15.152:0/2763418000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab2562d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc001eb0 tx=0x7fb8cc011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfec0c7284a4100 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:10.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.839+0000 7fb8bbfff640 1 -- 172.21.15.152:0/2763418000 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc034070 con 0x55b6ab23ed40 2024-09-11T23:01:10.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.839+0000 7fb8bbfff640 1 -- 172.21.15.152:0/2763418000 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8cc00a560 con 0x55b6ab23ed40 2024-09-11T23:01:10.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.839+0000 7fb8bbfff640 1 -- 172.21.15.152:0/2763418000 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc015040 con 0x55b6ab23ed40 2024-09-11T23:01:10.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.839+0000 7fb8de9e5740 1 -- 172.21.15.152:0/2763418000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 msgr2=0x55b6ab2562d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:10.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.839+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/2763418000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab2562d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc001eb0 tx=0x7fb8cc011dd0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.839+0000 7fb8de9e5740 1 -- 172.21.15.152:0/2763418000 shutdown_connections 2024-09-11T23:01:10.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.839+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/2763418000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab2562d0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.839+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/2763418000 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab234480 0x55b6ab23aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.839+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/2763418000 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab233b60 0x55b6ab2365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.839+0000 7fb8de9e5740 1 -- 172.21.15.152:0/2763418000 >> 172.21.15.152:0/2763418000 conn(0x55b6ab2300f0 msgr2=0x55b6ab23a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:10.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.839+0000 7fb8de9e5740 1 -- 172.21.15.152:0/2763418000 shutdown_connections 2024-09-11T23:01:10.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.839+0000 7fb8de9e5740 1 -- 172.21.15.152:0/2763418000 wait complete. 2024-09-11T23:01:10.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.840+0000 7fb8de9e5740 1 -- start start 2024-09-11T23:01:10.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.840+0000 7fb8de9e5740 1 Processor -- start 2024-09-11T23:01:10.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.840+0000 7fb8de9e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab233b60 0x55b6ab2473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.840+0000 7fb8de9e5740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab234480 0x55b6ab2426d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.840+0000 7fb8de9e5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab242c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.840+0000 7fb8de9e5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d41aec90 con 0x55b6ab23ed40 2024-09-11T23:01:10.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.840+0000 7fb8de9e5740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d41ae5c0 con 0x55b6ab234480 2024-09-11T23:01:10.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.840+0000 7fb8de9e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d41ab820 con 0x55b6ab233b60 2024-09-11T23:01:10.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.840+0000 7fb8de9e5740 10 client.0 _ll_register_callbacks cb 0x7fb8c800a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:10.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.840+0000 7fb8daba9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab242c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.840+0000 7fb8daba9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab242c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47028/0 (socket says 172.21.15.152:47028) 2024-09-11T23:01:10.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.840+0000 7fb8daba9640 1 -- 172.21.15.152:0/2062265218 learned_addr learned my addr 172.21.15.152:0/2062265218 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:10.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.840+0000 7fb8d9ba7640 1 --2- 172.21.15.152:0/2062265218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab234480 0x55b6ab2426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.840+0000 7fb8daba9640 1 --2- 172.21.15.152:0/2062265218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab233b60 0x55b6ab2473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8daba9640 1 -- 172.21.15.152:0/2062265218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab233b60 msgr2=0x55b6ab2473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:10.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8daba9640 1 --2- 172.21.15.152:0/2062265218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab233b60 0x55b6ab2473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8daba9640 1 -- 172.21.15.152:0/2062265218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab234480 msgr2=0x55b6ab2426d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:10.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8daba9640 1 --2- 172.21.15.152:0/2062265218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab234480 0x55b6ab2426d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8daba9640 1 -- 172.21.15.152:0/2062265218 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ab243420 con 0x55b6ab23ed40 2024-09-11T23:01:10.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8daba9640 1 --2- 172.21.15.152:0/2062265218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab242c10 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc018ff0 tx=0x7fb8cc011530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df31cf7f01a9e3e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:10.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:10.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8b9ffb640 1 -- 172.21.15.152:0/2062265218 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc015070 con 0x55b6ab23ed40 2024-09-11T23:01:10.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8b9ffb640 1 -- 172.21.15.152:0/2062265218 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7fb8cc018b00 con 0x55b6ab23ed40 2024-09-11T23:01:10.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8de9e5740 1 -- 172.21.15.152:0/2062265218 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8cc00a560 con 0x55b6ab23ed40 2024-09-11T23:01:10.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8b9ffb640 1 -- 172.21.15.152:0/2062265218 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb8cc041050 con 0x55b6ab23ed40 2024-09-11T23:01:10.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8de9e5740 10 client.17820 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:10.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8de9e5740 1 -- 172.21.15.152:0/2062265218 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6ab2bd660 con 0x55b6ab23ed40 2024-09-11T23:01:10.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8b9ffb640 1 -- 172.21.15.152:0/2062265218 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8cc00a560 con 0x55b6ab23ed40 2024-09-11T23:01:10.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8b9ffb640 1 -- 172.21.15.152:0/2062265218 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb8cc026070 con 0x55b6ab23ed40 2024-09-11T23:01:10.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8de9e5740 10 client.17820 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:10.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8de9e5740 10 client.17820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:01:10.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.841+0000 7fb8de9e5740 20 client.17820 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:10.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.842+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/2062265218 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b6ab2bdd20 0x55b6ab2c0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:10.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.842+0000 7fb8de9e5740 4 client.17820 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:01:10.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.842+0000 7fb8de9e5740 1 -- 172.21.15.152:0/2062265218 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b6ab1a0d00 con 0x55b6ab2bdd20 2024-09-11T23:01:10.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.842+0000 7fb8d9ba7640 1 --2- 172.21.15.152:0/2062265218 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b6ab2bdd20 0x55b6ab2c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:10.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.842+0000 7fb8d9ba7640 1 --2- 172.21.15.152:0/2062265218 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b6ab2bdd20 0x55b6ab2c0140 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e64807136c74c47c server_cookie=dcee6c415e54428b in_seq=0 out_seq=0 2024-09-11T23:01:10.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.842+0000 7fb8b9ffb640 10 client.17820 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:10.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.843+0000 7fb8b9ffb640 1 -- 172.21.15.152:0/2062265218 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b6ab1a0d00 con 0x55b6ab2bdd20 2024-09-11T23:01:10.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.843+0000 7fb8b9ffb640 10 client.17820 handle_command_reply: tid=0 2024-09-11T23:01:10.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.843+0000 7fb8de9e5740 1 client.17820 shutdown 2024-09-11T23:01:10.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.843+0000 7fb8de9e5740 1 -- 172.21.15.152:0/2062265218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 msgr2=0x55b6ab242c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:10.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.843+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/2062265218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab242c10 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc018ff0 tx=0x7fb8cc011530 comp rx=0 tx=0).stop 2024-09-11T23:01:10.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.843+0000 7fb8de9e5740 1 -- 172.21.15.152:0/2062265218 shutdown_connections 2024-09-11T23:01:10.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.843+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/2062265218 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b6ab23ed40 0x55b6ab242c10 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.843+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/2062265218 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b6ab2bdd20 0x55b6ab2c0140 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.843+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/2062265218 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b6ab234480 0x55b6ab2426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.843+0000 7fb8b9ffb640 0 client.17820 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:10.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.843+0000 7fb8de9e5740 1 --2- 172.21.15.152:0/2062265218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b6ab233b60 0x55b6ab2473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.843+0000 7fb8de9e5740 1 -- 172.21.15.152:0/2062265218 >> 172.21.15.152:0/2062265218 conn(0x55b6ab2300f0 msgr2=0x55b6ab22f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:10.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.844+0000 7fb8de9e5740 1 -- 172.21.15.152:0/2062265218 shutdown_connections 2024-09-11T23:01:10.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.844+0000 7fb8de9e5740 1 -- 172.21.15.152:0/2062265218 wait complete. 2024-09-11T23:01:10.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.844+0000 7fb8de9e5740 20 client.17820 trim_cache size 0 max 16384 2024-09-11T23:01:10.851 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:01:10.852 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T23:01:10.852 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "393211ce-1de4-451c-a606-4e11e171f698", 2024-09-11T23:01:10.852 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T23:01:10.852 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:01:10.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.844+0000 7fb8dc60c640 1 -- 172.21.15.152:0/1610856775 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb8bc0a2cf0 msgr2=0x7fb8bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:10.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.844+0000 7fb8dc60c640 1 --2- 172.21.15.152:0/1610856775 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb8bc0a2cf0 0x7fb8bc0a51b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40ede80 tx=0x7fb8c40f5040 comp rx=0 tx=0).stop 2024-09-11T23:01:10.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.844+0000 7fb8dc60c640 1 -- 172.21.15.152:0/1610856775 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8d4100e20 msgr2=0x7fb8d4199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:10.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.844+0000 7fb8dc60c640 1 --2- 172.21.15.152:0/1610856775 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8d4100e20 0x7fb8d4199310 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4005cc0 tx=0x7fb8c402f6c0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.844+0000 7fb8dc60c640 1 -- 172.21.15.152:0/1610856775 shutdown_connections 2024-09-11T23:01:10.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.844+0000 7fb8dc60c640 1 --2- 172.21.15.152:0/1610856775 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb8bc0a2cf0 0x7fb8bc0a51b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.844+0000 7fb8dc60c640 1 --2- 172.21.15.152:0/1610856775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8d4102ef0 0x7fb8d41a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.844+0000 7fb8dc60c640 1 --2- 172.21.15.152:0/1610856775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8d4102570 0x7fb8d4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.844+0000 7fb8dc60c640 1 --2- 172.21.15.152:0/1610856775 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8d4100e20 0x7fb8d4199310 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:10.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.844+0000 7fb8dc60c640 1 -- 172.21.15.152:0/1610856775 >> 172.21.15.152:0/1610856775 conn(0x7fb8d40fc7d0 msgr2=0x7fb8d4106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:10.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.845+0000 7fb8dc60c640 1 -- 172.21.15.152:0/1610856775 shutdown_connections 2024-09-11T23:01:10.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:10.845+0000 7fb8dc60c640 1 -- 172.21.15.152:0/1610856775 wait complete. 2024-09-11T23:01:10.938 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:10 smithi191 ceph-mon[35211]: pgmap v254: 97 pgs: 15 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 58 active+clean; 6.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:01:10.946 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:10 smithi204 ceph-mon[35454]: pgmap v254: 97 pgs: 15 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 58 active+clean; 6.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:01:11.729 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:11 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T23:01:11.729 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:11 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T23:01:11.729 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:11 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T23:01:11.729 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:11 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T23:01:11.755 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:01:12.003 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:01:12.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:11 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T23:01:12.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:11 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T23:01:12.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:11 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T23:01:12.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:11 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T23:01:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:11 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T23:01:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:11 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T23:01:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:11 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T23:01:12.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:11 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T23:01:12.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.524+0000 7f6b9ddb2640 1 -- 172.21.15.152:0/626946244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b98102570 msgr2=0x7f6b981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.524+0000 7f6b9ddb2640 1 --2- 172.21.15.152:0/626946244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b98102570 0x7f6b981029b0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009900 tx=0x7f6b8002a480 comp rx=0 tx=0).stop 2024-09-11T23:01:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.524+0000 7f6b9ddb2640 1 -- 172.21.15.152:0/626946244 shutdown_connections 2024-09-11T23:01:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.524+0000 7f6b9ddb2640 1 --2- 172.21.15.152:0/626946244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b98102ef0 0x7f6b9810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.524+0000 7f6b9ddb2640 1 --2- 172.21.15.152:0/626946244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b98102570 0x7f6b981029b0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.524+0000 7f6b9ddb2640 1 --2- 172.21.15.152:0/626946244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b98100e20 0x7f6b98101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.524+0000 7f6b9ddb2640 1 -- 172.21.15.152:0/626946244 >> 172.21.15.152:0/626946244 conn(0x7f6b980fc7d0 msgr2=0x7f6b980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.524+0000 7f6b9ddb2640 1 -- 172.21.15.152:0/626946244 shutdown_connections 2024-09-11T23:01:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.524+0000 7f6b9ddb2640 1 -- 172.21.15.152:0/626946244 wait complete. 2024-09-11T23:01:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.525+0000 7f6b9ddb2640 1 Processor -- start 2024-09-11T23:01:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.525+0000 7f6b9ddb2640 1 -- start start 2024-09-11T23:01:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.525+0000 7f6b9ddb2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b98100e20 0x7f6b98197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.525+0000 7f6b9ddb2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b98102570 0x7f6b98197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.525+0000 7f6b9ddb2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b98102ef0 0x7f6b9819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.525+0000 7f6b9ddb2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6b98110f50 con 0x7f6b98102570 2024-09-11T23:01:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.525+0000 7f6b9ddb2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6b98110dd0 con 0x7f6b98102ef0 2024-09-11T23:01:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.525+0000 7f6b9ddb2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b981110d0 con 0x7f6b98100e20 2024-09-11T23:01:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b8effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b98102570 0x7f6b98197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b8effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b98102570 0x7f6b98197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47034/0 (socket says 172.21.15.152:47034) 2024-09-11T23:01:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b8effd640 1 -- 172.21.15.152:0/4071925393 learned_addr learned my addr 172.21.15.152:0/4071925393 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b97fff640 1 --2- 172.21.15.152:0/4071925393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b98102ef0 0x7f6b9819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b977fe640 1 --2- 172.21.15.152:0/4071925393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b98100e20 0x7f6b98197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b8effd640 1 -- 172.21.15.152:0/4071925393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b98100e20 msgr2=0x7f6b98197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b8effd640 1 --2- 172.21.15.152:0/4071925393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b98100e20 0x7f6b98197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b8effd640 1 -- 172.21.15.152:0/4071925393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b98102ef0 msgr2=0x7f6b9819bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b8effd640 1 --2- 172.21.15.152:0/4071925393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b98102ef0 0x7f6b9819bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b8effd640 1 -- 172.21.15.152:0/4071925393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9819c3e0 con 0x7f6b98102570 2024-09-11T23:01:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b977fe640 1 --2- 172.21.15.152:0/4071925393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b98100e20 0x7f6b98197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:01:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b97fff640 1 --2- 172.21.15.152:0/4071925393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b98102ef0 0x7f6b9819bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:01:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b8effd640 1 --2- 172.21.15.152:0/4071925393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b98102570 0x7f6b98197970 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f6b80006010 tx=0x7f6b80003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b957fa640 1 -- 172.21.15.152:0/4071925393 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8003f070 con 0x7f6b98102570 2024-09-11T23:01:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b957fa640 1 -- 172.21.15.152:0/4071925393 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b80031c30 con 0x7f6b98102570 2024-09-11T23:01:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b957fa640 1 -- 172.21.15.152:0/4071925393 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b80035030 con 0x7f6b98102570 2024-09-11T23:01:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b9ddb2640 1 -- 172.21.15.152:0/4071925393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9819c670 con 0x7f6b98102570 2024-09-11T23:01:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.526+0000 7f6b9ddb2640 1 -- 172.21.15.152:0/4071925393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b981a3fb0 con 0x7f6b98102570 2024-09-11T23:01:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.528+0000 7f6b957fa640 1 -- 172.21.15.152:0/4071925393 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6b80042070 con 0x7f6b98102570 2024-09-11T23:01:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.529+0000 7f6ba018b740 1 Processor -- start 2024-09-11T23:01:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.529+0000 7f6ba018b740 1 -- start start 2024-09-11T23:01:12.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.529+0000 7f6ba018b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8cf2e0 0x55c21a8cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.529+0000 7f6ba018b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8cfc00 0x55c21a8dc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.529+0000 7f6ba018b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8dc890 0x55c21a8dec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.529+0000 7f6ba018b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55c21a8352e0 con 0x55c21a8dc890 2024-09-11T23:01:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.529+0000 7f6ba018b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c21a7dcce0 con 0x55c21a8cfc00 2024-09-11T23:01:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.529+0000 7f6ba018b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c21a8312a0 con 0x55c21a8cf2e0 2024-09-11T23:01:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.529+0000 7f6b957fa640 1 --2- 172.21.15.152:0/4071925393 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6b840a2cf0 0x7f6b840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.529+0000 7f6b8effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8dc890 0x55c21a8dec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.529+0000 7f6b8effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8dc890 0x55c21a8dec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47046/0 (socket says 172.21.15.152:47046) 2024-09-11T23:01:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.529+0000 7f6b8effd640 1 -- 172.21.15.152:0/1943011598 learned_addr learned my addr 172.21.15.152:0/1943011598 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.529+0000 7f6b97fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8cf2e0 0x55c21a8cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.529+0000 7f6b97fff640 1 --2- 172.21.15.152:0/4071925393 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6b840a2cf0 0x7f6b840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.529+0000 7f6b977fe640 1 --2- 172.21.15.152:0/1943011598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8cfc00 0x55c21a8dc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.529+0000 7f6b957fa640 1 -- 172.21.15.152:0/4071925393 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7f6b800efd70 con 0x7f6b98102570 2024-09-11T23:01:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.530+0000 7f6b8effd640 1 -- 172.21.15.152:0/1943011598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8cf2e0 msgr2=0x55c21a8cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.530+0000 7f6b8effd640 1 --2- 172.21.15.152:0/1943011598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8cf2e0 0x55c21a8cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.530+0000 7f6b8effd640 1 -- 172.21.15.152:0/1943011598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8cfc00 msgr2=0x55c21a8dc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.530+0000 7f6b8effd640 1 --2- 172.21.15.152:0/1943011598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8cfc00 0x55c21a8dc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.530+0000 7f6b8effd640 1 -- 172.21.15.152:0/1943011598 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c21a7c1440 con 0x55c21a8dc890 2024-09-11T23:01:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.530+0000 7f6b8effd640 1 --2- 172.21.15.152:0/1943011598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8dc890 0x55c21a8dec80 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f6b800ed510 tx=0x7f6b8003b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a56db496f5b876 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.530+0000 7f6b97fff640 1 --2- 172.21.15.152:0/4071925393 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6b840a2cf0 0x7f6b840a51b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6b90005c40 tx=0x7f6b90009280 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.530+0000 7f6b8dffb640 1 -- 172.21.15.152:0/1943011598 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b80049070 con 0x55c21a8dc890 2024-09-11T23:01:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.530+0000 7f6b8dffb640 1 -- 172.21.15.152:0/1943011598 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b800c0630 con 0x55c21a8dc890 2024-09-11T23:01:12.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.530+0000 7f6b8dffb640 1 -- 172.21.15.152:0/1943011598 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b800b88f0 con 0x55c21a8dc890 2024-09-11T23:01:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.531+0000 7f6ba018b740 1 -- 172.21.15.152:0/1943011598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8dc890 msgr2=0x55c21a8dec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.531+0000 7f6ba018b740 1 --2- 172.21.15.152:0/1943011598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8dc890 0x55c21a8dec80 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f6b800ed510 tx=0x7f6b8003b870 comp rx=0 tx=0).stop 2024-09-11T23:01:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.531+0000 7f6ba018b740 1 -- 172.21.15.152:0/1943011598 shutdown_connections 2024-09-11T23:01:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.531+0000 7f6ba018b740 1 --2- 172.21.15.152:0/1943011598 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8dc890 0x55c21a8dec80 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.531+0000 7f6ba018b740 1 --2- 172.21.15.152:0/1943011598 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8cfc00 0x55c21a8dc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.531+0000 7f6ba018b740 1 --2- 172.21.15.152:0/1943011598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8cf2e0 0x55c21a8cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.531+0000 7f6ba018b740 1 -- 172.21.15.152:0/1943011598 >> 172.21.15.152:0/1943011598 conn(0x55c21a8cba30 msgr2=0x55c21a8cbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.531+0000 7f6ba018b740 1 -- 172.21.15.152:0/1943011598 shutdown_connections 2024-09-11T23:01:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.531+0000 7f6ba018b740 1 -- 172.21.15.152:0/1943011598 wait complete. 2024-09-11T23:01:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.531+0000 7f6ba018b740 1 -- start start 2024-09-11T23:01:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.531+0000 7f6ba018b740 1 Processor -- start 2024-09-11T23:01:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.531+0000 7f6ba018b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8cf2e0 0x55c21a8dc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6ba018b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8cfc00 0x55c21a8d9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6ba018b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8dc890 0x55c21a8d9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6ba018b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55c21a8352e0 con 0x55c21a8cf2e0 2024-09-11T23:01:12.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6ba018b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c21a7f0050 con 0x55c21a8dc890 2024-09-11T23:01:12.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6ba018b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c21a8df220 con 0x55c21a8cfc00 2024-09-11T23:01:12.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6ba018b740 10 client.0 _ll_register_callbacks cb 0x55c21a830160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:12.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6b977fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8cf2e0 0x55c21a8dc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6b977fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8cf2e0 0x55c21a8dc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47062/0 (socket says 172.21.15.152:47062) 2024-09-11T23:01:12.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6b977fe640 1 -- 172.21.15.152:0/1000350916 learned_addr learned my addr 172.21.15.152:0/1000350916 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:12.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6b8effd640 1 --2- 172.21.15.152:0/1000350916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8cfc00 0x55c21a8d9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6b97fff640 1 --2- 172.21.15.152:0/1000350916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8dc890 0x55c21a8d9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6b977fe640 1 -- 172.21.15.152:0/1000350916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8cfc00 msgr2=0x55c21a8d9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:12.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6b977fe640 1 --2- 172.21.15.152:0/1000350916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8cfc00 0x55c21a8d9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6b977fe640 1 -- 172.21.15.152:0/1000350916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8dc890 msgr2=0x55c21a8d9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:12.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6b977fe640 1 --2- 172.21.15.152:0/1000350916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8dc890 0x55c21a8d9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6b977fe640 1 -- 172.21.15.152:0/1000350916 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c21a5a63c0 con 0x55c21a8cf2e0 2024-09-11T23:01:12.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6b977fe640 1 --2- 172.21.15.152:0/1000350916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8cf2e0 0x55c21a8dc200 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800bad0 tx=0x7f6b8800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a48bfbc1347bc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:12.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6b76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:12.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6b76ffd640 1 -- 172.21.15.152:0/1000350916 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88005220 con 0x55c21a8cf2e0 2024-09-11T23:01:12.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6b76ffd640 1 -- 172.21.15.152:0/1000350916 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7f6b880053c0 con 0x55c21a8cf2e0 2024-09-11T23:01:12.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.532+0000 7f6ba018b740 1 -- 172.21.15.152:0/1000350916 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c21a842850 con 0x55c21a8cf2e0 2024-09-11T23:01:12.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.533+0000 7f6b76ffd640 1 -- 172.21.15.152:0/1000350916 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b880055c0 con 0x55c21a8cf2e0 2024-09-11T23:01:12.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.533+0000 7f6ba018b740 10 client.17832 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:12.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.533+0000 7f6ba018b740 1 -- 172.21.15.152:0/1000350916 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c21a8eaba0 con 0x55c21a8cf2e0 2024-09-11T23:01:12.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.533+0000 7f6b76ffd640 1 -- 172.21.15.152:0/1000350916 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c21a842850 con 0x55c21a8cf2e0 2024-09-11T23:01:12.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.533+0000 7f6b76ffd640 1 -- 172.21.15.152:0/1000350916 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6b88019030 con 0x55c21a8cf2e0 2024-09-11T23:01:12.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.533+0000 7f6ba018b740 10 client.17832 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:12.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.533+0000 7f6ba018b740 10 client.17832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:01:12.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.533+0000 7f6ba018b740 20 client.17832 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:12.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.533+0000 7f6ba018b740 1 --2- 172.21.15.152:0/1000350916 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c21a8ec0b0 0x55c21a8ee520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.533+0000 7f6ba018b740 4 client.17832 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:01:12.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.533+0000 7f6ba018b740 1 -- 172.21.15.152:0/1000350916 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c21a83cd60 con 0x55c21a8ec0b0 2024-09-11T23:01:12.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.533+0000 7f6b8effd640 1 --2- 172.21.15.152:0/1000350916 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c21a8ec0b0 0x55c21a8ee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.534+0000 7f6b76ffd640 10 client.17832 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:12.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.534+0000 7f6b8effd640 1 --2- 172.21.15.152:0/1000350916 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c21a8ec0b0 0x55c21a8ee520 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=941134551ac302fe server_cookie=342e7e2ce00238d5 in_seq=0 out_seq=0 2024-09-11T23:01:12.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.534+0000 7f6b76ffd640 1 -- 172.21.15.152:0/1000350916 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c21a83cd60 con 0x55c21a8ec0b0 2024-09-11T23:01:12.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.534+0000 7f6b76ffd640 10 client.17832 handle_command_reply: tid=0 2024-09-11T23:01:12.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.534+0000 7f6ba018b740 1 client.17832 shutdown 2024-09-11T23:01:12.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.535+0000 7f6ba018b740 1 -- 172.21.15.152:0/1000350916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8cf2e0 msgr2=0x55c21a8dc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:12.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.535+0000 7f6ba018b740 1 --2- 172.21.15.152:0/1000350916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8cf2e0 0x55c21a8dc200 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800bad0 tx=0x7f6b8800bed0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.535+0000 7f6ba018b740 1 -- 172.21.15.152:0/1000350916 shutdown_connections 2024-09-11T23:01:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.535+0000 7f6ba018b740 1 --2- 172.21.15.152:0/1000350916 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c21a8ec0b0 0x55c21a8ee520 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.535+0000 7f6ba018b740 1 --2- 172.21.15.152:0/1000350916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8dc890 0x55c21a8d9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.535+0000 7f6ba018b740 1 --2- 172.21.15.152:0/1000350916 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8cfc00 0x55c21a8d9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.535+0000 7f6b76ffd640 0 client.17832 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.535+0000 7f6ba018b740 1 --2- 172.21.15.152:0/1000350916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8cf2e0 0x55c21a8dc200 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.535+0000 7f6ba018b740 1 -- 172.21.15.152:0/1000350916 >> 172.21.15.152:0/1000350916 conn(0x55c21a8cba30 msgr2=0x55c21a8cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:12.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.535+0000 7f6ba018b740 1 -- 172.21.15.152:0/1000350916 shutdown_connections 2024-09-11T23:01:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.535+0000 7f6ba018b740 1 -- 172.21.15.152:0/1000350916 wait complete. 2024-09-11T23:01:12.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.535+0000 7f6ba018b740 20 client.17832 trim_cache size 0 max 16384 2024-09-11T23:01:12.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.552+0000 7f6ba018b740 1 Processor -- start 2024-09-11T23:01:12.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.552+0000 7f6ba018b740 1 -- start start 2024-09-11T23:01:12.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.552+0000 7f6ba018b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8cf2e0 0x55c21a8e8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.552+0000 7f6ba018b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8cfc00 0x55c21a8f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.552+0000 7f6ba018b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8dc890 0x55c21a8f4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.552+0000 7f6ba018b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6b981ab620 con 0x55c21a8dc890 2024-09-11T23:01:12.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.552+0000 7f6ba018b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6b981ad160 con 0x55c21a8cfc00 2024-09-11T23:01:12.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.552+0000 7f6ba018b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b981abcf0 con 0x55c21a8cf2e0 2024-09-11T23:01:12.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.552+0000 7f6b97fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8dc890 0x55c21a8f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.552+0000 7f6b97fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8dc890 0x55c21a8f4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47066/0 (socket says 172.21.15.152:47066) 2024-09-11T23:01:12.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.552+0000 7f6b97fff640 1 -- 172.21.15.152:0/1891112927 learned_addr learned my addr 172.21.15.152:0/1891112927 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:12.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.552+0000 7f6b977fe640 1 --2- 172.21.15.152:0/1891112927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8cf2e0 0x55c21a8e8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.552+0000 7f6b8effd640 1 --2- 172.21.15.152:0/1891112927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8cfc00 0x55c21a8f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.553+0000 7f6b97fff640 1 -- 172.21.15.152:0/1891112927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8cf2e0 msgr2=0x55c21a8e8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.553+0000 7f6b97fff640 1 --2- 172.21.15.152:0/1891112927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8cf2e0 0x55c21a8e8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.553+0000 7f6b97fff640 1 -- 172.21.15.152:0/1891112927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8cfc00 msgr2=0x55c21a8f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.553+0000 7f6b97fff640 1 --2- 172.21.15.152:0/1891112927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8cfc00 0x55c21a8f3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.553+0000 7f6b97fff640 1 -- 172.21.15.152:0/1891112927 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c21a8da880 con 0x55c21a8dc890 2024-09-11T23:01:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.553+0000 7f6b8effd640 1 --2- 172.21.15.152:0/1891112927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8cfc00 0x55c21a8f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:01:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.553+0000 7f6b97fff640 1 --2- 172.21.15.152:0/1891112927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8dc890 0x55c21a8f4330 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f6b900129c0 tx=0x7f6b90009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b658f9adbd7efe88 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.553+0000 7f6b8d7fa640 1 -- 172.21.15.152:0/1891112927 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90004b20 con 0x55c21a8dc890 2024-09-11T23:01:12.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.553+0000 7f6b8d7fa640 1 -- 172.21.15.152:0/1891112927 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b90004cc0 con 0x55c21a8dc890 2024-09-11T23:01:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.553+0000 7f6b8d7fa640 1 -- 172.21.15.152:0/1891112927 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b900056e0 con 0x55c21a8dc890 2024-09-11T23:01:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.554+0000 7f6ba018b740 1 -- 172.21.15.152:0/1891112927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8dc890 msgr2=0x55c21a8f4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.554+0000 7f6ba018b740 1 --2- 172.21.15.152:0/1891112927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8dc890 0x55c21a8f4330 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f6b900129c0 tx=0x7f6b90009700 comp rx=0 tx=0).stop 2024-09-11T23:01:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.554+0000 7f6ba018b740 1 -- 172.21.15.152:0/1891112927 shutdown_connections 2024-09-11T23:01:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.554+0000 7f6ba018b740 1 --2- 172.21.15.152:0/1891112927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8dc890 0x55c21a8f4330 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.554+0000 7f6ba018b740 1 --2- 172.21.15.152:0/1891112927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8cfc00 0x55c21a8f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.554+0000 7f6ba018b740 1 --2- 172.21.15.152:0/1891112927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8cf2e0 0x55c21a8e8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.554+0000 7f6ba018b740 1 -- 172.21.15.152:0/1891112927 >> 172.21.15.152:0/1891112927 conn(0x55c21a8cba30 msgr2=0x55c21a8d84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.554+0000 7f6ba018b740 1 -- 172.21.15.152:0/1891112927 shutdown_connections 2024-09-11T23:01:12.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.554+0000 7f6ba018b740 1 -- 172.21.15.152:0/1891112927 wait complete. 2024-09-11T23:01:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.554+0000 7f6ba018b740 1 -- start start 2024-09-11T23:01:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.554+0000 7f6ba018b740 1 Processor -- start 2024-09-11T23:01:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6ba018b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8cf2e0 0x55c21a8e4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6ba018b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8cfc00 0x55c21a8dfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6ba018b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8dc890 0x55c21a8e01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6ba018b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6b981aca90 con 0x55c21a8cf2e0 2024-09-11T23:01:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6ba018b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6b981ac3c0 con 0x55c21a8cfc00 2024-09-11T23:01:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6ba018b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6b981a9620 con 0x55c21a8dc890 2024-09-11T23:01:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6ba018b740 10 client.0 _ll_register_callbacks cb 0x7f6b88004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6b977fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8cf2e0 0x55c21a8e4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6b977fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8cf2e0 0x55c21a8e4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47082/0 (socket says 172.21.15.152:47082) 2024-09-11T23:01:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6b977fe640 1 -- 172.21.15.152:0/432436229 learned_addr learned my addr 172.21.15.152:0/432436229 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6b8effd640 1 --2- 172.21.15.152:0/432436229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8cfc00 0x55c21a8dfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6b97fff640 1 --2- 172.21.15.152:0/432436229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8dc890 0x55c21a8e01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6b977fe640 1 -- 172.21.15.152:0/432436229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8dc890 msgr2=0x55c21a8e01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6b977fe640 1 --2- 172.21.15.152:0/432436229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8dc890 0x55c21a8e01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6b977fe640 1 -- 172.21.15.152:0/432436229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8cfc00 msgr2=0x55c21a8dfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6b977fe640 1 --2- 172.21.15.152:0/432436229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8cfc00 0x55c21a8dfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6b977fe640 1 -- 172.21.15.152:0/432436229 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c21a8e07b0 con 0x55c21a8cf2e0 2024-09-11T23:01:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6b977fe640 1 --2- 172.21.15.152:0/432436229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8cf2e0 0x55c21a8e4980 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f6b880043a0 tx=0x7f6b880116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a638c23297cab1ca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6b76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6b76ffd640 1 -- 172.21.15.152:0/432436229 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88026070 con 0x55c21a8cf2e0 2024-09-11T23:01:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.555+0000 7f6b76ffd640 1 -- 172.21.15.152:0/432436229 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7f6b8802b440 con 0x55c21a8cf2e0 2024-09-11T23:01:12.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.556+0000 7f6ba018b740 1 -- 172.21.15.152:0/432436229 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b90004cc0 con 0x55c21a8cf2e0 2024-09-11T23:01:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.556+0000 7f6b76ffd640 1 -- 172.21.15.152:0/432436229 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6b88038050 con 0x55c21a8cf2e0 2024-09-11T23:01:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.556+0000 7f6ba018b740 10 client.17838 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.556+0000 7f6ba018b740 1 -- 172.21.15.152:0/432436229 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c21a956b40 con 0x55c21a8cf2e0 2024-09-11T23:01:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.556+0000 7f6b76ffd640 1 -- 172.21.15.152:0/432436229 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b90004cc0 con 0x55c21a8cf2e0 2024-09-11T23:01:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.556+0000 7f6b76ffd640 1 -- 172.21.15.152:0/432436229 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6b88033030 con 0x55c21a8cf2e0 2024-09-11T23:01:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.556+0000 7f6ba018b740 10 client.17838 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.556+0000 7f6ba018b740 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:01:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.556+0000 7f6ba018b740 20 client.17838 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.556+0000 7f6ba018b740 1 --2- 172.21.15.152:0/432436229 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c21a9572b0 0x55c21a9596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.556+0000 7f6ba018b740 4 client.17838 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:01:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.556+0000 7f6ba018b740 1 -- 172.21.15.152:0/432436229 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c21a7b9210 con 0x55c21a9572b0 2024-09-11T23:01:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.556+0000 7f6b8effd640 1 --2- 172.21.15.152:0/432436229 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c21a9572b0 0x55c21a9596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:12.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6b76ffd640 10 client.17838 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6b8effd640 1 --2- 172.21.15.152:0/432436229 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c21a9572b0 0x55c21a9596a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c82f377c0722e2e server_cookie=127f9c8fa601049b in_seq=0 out_seq=0 2024-09-11T23:01:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6b76ffd640 1 -- 172.21.15.152:0/432436229 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c21a7b9210 con 0x55c21a9572b0 2024-09-11T23:01:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6b76ffd640 10 client.17838 handle_command_reply: tid=0 2024-09-11T23:01:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6ba018b740 1 client.17838 shutdown 2024-09-11T23:01:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6ba018b740 1 -- 172.21.15.152:0/432436229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8cf2e0 msgr2=0x55c21a8e4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6ba018b740 1 --2- 172.21.15.152:0/432436229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8cf2e0 0x55c21a8e4980 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f6b880043a0 tx=0x7f6b880116f0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6ba018b740 1 -- 172.21.15.152:0/432436229 shutdown_connections 2024-09-11T23:01:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6ba018b740 1 --2- 172.21.15.152:0/432436229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c21a8dc890 0x55c21a8e01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6ba018b740 1 --2- 172.21.15.152:0/432436229 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c21a9572b0 0x55c21a9596a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6ba018b740 1 --2- 172.21.15.152:0/432436229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c21a8cfc00 0x55c21a8dfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6ba018b740 1 --2- 172.21.15.152:0/432436229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c21a8cf2e0 0x55c21a8e4980 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6ba018b740 1 -- 172.21.15.152:0/432436229 >> 172.21.15.152:0/432436229 conn(0x55c21a8cba30 msgr2=0x55c21a8d7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6b76ffd640 0 client.17838 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:12.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6ba018b740 1 -- 172.21.15.152:0/432436229 shutdown_connections 2024-09-11T23:01:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6ba018b740 1 -- 172.21.15.152:0/432436229 wait complete. 2024-09-11T23:01:12.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.557+0000 7f6ba018b740 20 client.17838 trim_cache size 0 max 16384 2024-09-11T23:01:12.577 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:01:12.577 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:01:12.577 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": { 2024-09-11T23:01:12.577 INFO:teuthology.orchestra.run.smithi152.stdout: "393211ce-1de4-451c-a606-4e11e171f698": { 2024-09-11T23:01:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "path": "/", 2024-09-11T23:01:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "tag": "393211ce-1de4-451c-a606-4e11e171f698", 2024-09-11T23:01:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: "options": "recursive,force" 2024-09-11T23:01:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:01:12.578 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:01:12.578 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:01:12.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.558+0000 7f6b9ddb2640 1 -- 172.21.15.152:0/4071925393 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6b840a2cf0 msgr2=0x7f6b840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:12.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.558+0000 7f6b9ddb2640 1 --2- 172.21.15.152:0/4071925393 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6b840a2cf0 0x7f6b840a51b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6b90005c40 tx=0x7f6b90009280 comp rx=0 tx=0).stop 2024-09-11T23:01:12.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.558+0000 7f6b9ddb2640 1 -- 172.21.15.152:0/4071925393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b98102570 msgr2=0x7f6b98197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:12.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.558+0000 7f6b9ddb2640 1 --2- 172.21.15.152:0/4071925393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b98102570 0x7f6b98197970 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f6b80006010 tx=0x7f6b80003c10 comp rx=0 tx=0).stop 2024-09-11T23:01:12.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.558+0000 7f6b9ddb2640 1 -- 172.21.15.152:0/4071925393 shutdown_connections 2024-09-11T23:01:12.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.558+0000 7f6b9ddb2640 1 --2- 172.21.15.152:0/4071925393 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6b840a2cf0 0x7f6b840a51b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.558+0000 7f6b9ddb2640 1 --2- 172.21.15.152:0/4071925393 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b98102ef0 0x7f6b9819bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.558+0000 7f6b9ddb2640 1 --2- 172.21.15.152:0/4071925393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6b98102570 0x7f6b98197970 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.558+0000 7f6b9ddb2640 1 --2- 172.21.15.152:0/4071925393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b98100e20 0x7f6b98197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:12.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.558+0000 7f6b9ddb2640 1 -- 172.21.15.152:0/4071925393 >> 172.21.15.152:0/4071925393 conn(0x7f6b980fc7d0 msgr2=0x7f6b980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:12.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.558+0000 7f6b9ddb2640 1 -- 172.21.15.152:0/4071925393 shutdown_connections 2024-09-11T23:01:12.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:12.558+0000 7f6b9ddb2640 1 -- 172.21.15.152:0/4071925393 wait complete. 2024-09-11T23:01:12.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:12 smithi152 ceph-mon[31181]: pgmap v255: 97 pgs: 18 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 51 active+clean; 3.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:01:13.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:12 smithi191 ceph-mon[35211]: pgmap v255: 97 pgs: 18 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 51 active+clean; 3.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:01:13.205 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:12 smithi204 ceph-mon[35454]: pgmap v255: 97 pgs: 18 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 51 active+clean; 3.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:01:13.451 INFO:tasks.cephfs.filesystem:scrub status for tag:393211ce-1de4-451c-a606-4e11e171f698 - {'path': '/', 'tag': '393211ce-1de4-451c-a606-4e11e171f698', 'options': 'recursive,force'} 2024-09-11T23:01:13.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:13 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T23:01:14.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:13 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T23:01:14.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:13 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T23:01:14.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:14 smithi152 ceph-mon[31181]: pgmap v256: 97 pgs: 18 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 51 active+clean; 3.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:01:15.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:14 smithi191 ceph-mon[35211]: pgmap v256: 97 pgs: 18 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 51 active+clean; 3.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:01:15.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:14 smithi204 ceph-mon[35454]: pgmap v256: 97 pgs: 18 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 51 active+clean; 3.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:01:16.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:16 smithi152 ceph-mon[31181]: pgmap v257: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:01:17.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:16 smithi191 ceph-mon[35211]: pgmap v257: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:01:17.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:16 smithi204 ceph-mon[35454]: pgmap v257: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:01:19.044 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:18 smithi191 ceph-mon[35211]: pgmap v258: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:01:19.069 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:18 smithi204 ceph-mon[35454]: pgmap v258: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:01:19.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:18 smithi152 ceph-mon[31181]: pgmap v258: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:01:21.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:20 smithi191 ceph-mon[35211]: pgmap v259: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 192 B/s wr, 15 op/s 2024-09-11T23:01:21.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:20 smithi191 ceph-mon[35211]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T23:01:21.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:20 smithi204 ceph-mon[35454]: pgmap v259: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 192 B/s wr, 15 op/s 2024-09-11T23:01:21.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:20 smithi204 ceph-mon[35454]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T23:01:21.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:20 smithi152 ceph-mon[31181]: pgmap v259: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 192 B/s wr, 15 op/s 2024-09-11T23:01:21.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:20 smithi152 ceph-mon[31181]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T23:01:23.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:22 smithi191 ceph-mon[35211]: pgmap v261: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:01:23.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:22 smithi204 ceph-mon[35454]: pgmap v261: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:01:23.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:22 smithi152 ceph-mon[31181]: pgmap v261: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:01:25.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:24 smithi191 ceph-mon[35211]: pgmap v262: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:01:25.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:24 smithi204 ceph-mon[35454]: pgmap v262: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:01:25.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:24 smithi152 ceph-mon[31181]: pgmap v262: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:01:27.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:26 smithi191 ceph-mon[35211]: pgmap v263: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:01:27.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:26 smithi204 ceph-mon[35454]: pgmap v263: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:01:27.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:26 smithi152 ceph-mon[31181]: pgmap v263: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:01:29.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:28 smithi191 ceph-mon[35211]: pgmap v264: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:01:29.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:28 smithi204 ceph-mon[35454]: pgmap v264: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:01:29.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:28 smithi152 ceph-mon[31181]: pgmap v264: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:01:31.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:30 smithi191 ceph-mon[35211]: pgmap v265: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:01:31.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:30 smithi204 ceph-mon[35454]: pgmap v265: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:01:31.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:30 smithi152 ceph-mon[31181]: pgmap v265: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:01:33.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:32 smithi191 ceph-mon[35211]: pgmap v266: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:01:33.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:32 smithi204 ceph-mon[35454]: pgmap v266: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:01:33.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:32 smithi152 ceph-mon[31181]: pgmap v266: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:01:35.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:34 smithi191 ceph-mon[35211]: pgmap v267: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:35.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:34 smithi204 ceph-mon[35454]: pgmap v267: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:35.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:34 smithi152 ceph-mon[31181]: pgmap v267: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:37.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:36 smithi191 ceph-mon[35211]: pgmap v268: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:01:37.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:36 smithi204 ceph-mon[35454]: pgmap v268: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:01:37.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:36 smithi152 ceph-mon[31181]: pgmap v268: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:01:39.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:38 smithi191 ceph-mon[35211]: pgmap v269: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:39.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:38 smithi204 ceph-mon[35454]: pgmap v269: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:39.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:38 smithi152 ceph-mon[31181]: pgmap v269: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:40.924 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:40 smithi152 ceph-mon[31181]: pgmap v270: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T23:01:41.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:40 smithi191 ceph-mon[35211]: pgmap v270: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T23:01:41.049 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:40 smithi204 ceph-mon[35454]: pgmap v270: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T23:01:43.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:42 smithi191 ceph-mon[35211]: pgmap v271: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:01:43.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:42 smithi204 ceph-mon[35454]: pgmap v271: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:01:43.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:42 smithi152 ceph-mon[31181]: pgmap v271: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:01:43.453 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:01:43.718 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:01:44.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.259+0000 7f8d5263c640 1 -- 172.21.15.152:0/457687361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d4c1036e0 msgr2=0x7f8d4c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:44.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.259+0000 7f8d5263c640 1 --2- 172.21.15.152:0/457687361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d4c1036e0 0x7f8d4c107b90 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009930 tx=0x7f8d4002a420 comp rx=0 tx=0).stop 2024-09-11T23:01:44.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.259+0000 7f8d5263c640 1 -- 172.21.15.152:0/457687361 shutdown_connections 2024-09-11T23:01:44.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.259+0000 7f8d5263c640 1 --2- 172.21.15.152:0/457687361 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d4c1082a0 0x7f8d4c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.259+0000 7f8d5263c640 1 --2- 172.21.15.152:0/457687361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d4c1036e0 0x7f8d4c107b90 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.259+0000 7f8d5263c640 1 --2- 172.21.15.152:0/457687361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d4c102d30 0x7f8d4c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.259+0000 7f8d5263c640 1 -- 172.21.15.152:0/457687361 >> 172.21.15.152:0/457687361 conn(0x7f8d4c0fc7d0 msgr2=0x7f8d4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:44.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.260+0000 7f8d5263c640 1 -- 172.21.15.152:0/457687361 shutdown_connections 2024-09-11T23:01:44.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.260+0000 7f8d5263c640 1 -- 172.21.15.152:0/457687361 wait complete. 2024-09-11T23:01:44.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.260+0000 7f8d5263c640 1 Processor -- start 2024-09-11T23:01:44.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.260+0000 7f8d5263c640 1 -- start start 2024-09-11T23:01:44.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.260+0000 7f8d5263c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d4c102d30 0x7f8d4c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.260+0000 7f8d5263c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d4c1036e0 0x7f8d4c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.260+0000 7f8d5263c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d4c1082a0 0x7f8d4c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d5263c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c1120a0 con 0x7f8d4c102d30 2024-09-11T23:01:44.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d5263c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c111f20 con 0x7f8d4c1082a0 2024-09-11T23:01:44.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d5263c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c112220 con 0x7f8d4c1036e0 2024-09-11T23:01:44.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d4bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d4c102d30 0x7f8d4c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d4bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d4c102d30 0x7f8d4c1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39368/0 (socket says 172.21.15.152:39368) 2024-09-11T23:01:44.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d4bfff640 1 -- 172.21.15.152:0/1550124094 learned_addr learned my addr 172.21.15.152:0/1550124094 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:44.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d50bd9640 1 --2- 172.21.15.152:0/1550124094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d4c1082a0 0x7f8d4c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d4b7fe640 1 --2- 172.21.15.152:0/1550124094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d4c1036e0 0x7f8d4c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d4bfff640 1 -- 172.21.15.152:0/1550124094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d4c1036e0 msgr2=0x7f8d4c198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:44.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d4bfff640 1 --2- 172.21.15.152:0/1550124094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d4c1036e0 0x7f8d4c198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d4bfff640 1 -- 172.21.15.152:0/1550124094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d4c1082a0 msgr2=0x7f8d4c19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:44.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d4bfff640 1 --2- 172.21.15.152:0/1550124094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d4c1082a0 0x7f8d4c19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d4bfff640 1 -- 172.21.15.152:0/1550124094 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4c19d550 con 0x7f8d4c102d30 2024-09-11T23:01:44.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d4b7fe640 1 --2- 172.21.15.152:0/1550124094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d4c1036e0 0x7f8d4c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:01:44.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d4bfff640 1 --2- 172.21.15.152:0/1550124094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d4c102d30 0x7f8d4c1985a0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00d950 tx=0x7f8d3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:44.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d497fa640 1 -- 172.21.15.152:0/1550124094 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c017070 con 0x7f8d4c102d30 2024-09-11T23:01:44.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d5263c640 1 -- 172.21.15.152:0/1550124094 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4c19d840 con 0x7f8d4c102d30 2024-09-11T23:01:44.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d497fa640 1 -- 172.21.15.152:0/1550124094 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3c009c30 con 0x7f8d4c102d30 2024-09-11T23:01:44.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d497fa640 1 -- 172.21.15.152:0/1550124094 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c00f6b0 con 0x7f8d4c102d30 2024-09-11T23:01:44.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.261+0000 7f8d5263c640 1 -- 172.21.15.152:0/1550124094 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4c1a5120 con 0x7f8d4c102d30 2024-09-11T23:01:44.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.264+0000 7f8d497fa640 1 -- 172.21.15.152:0/1550124094 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8d3c0109d0 con 0x7f8d4c102d30 2024-09-11T23:01:44.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.264+0000 7f8d54a15740 1 Processor -- start 2024-09-11T23:01:44.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.264+0000 7f8d54a15740 1 -- start start 2024-09-11T23:01:44.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.264+0000 7f8d54a15740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651a380 0x5622d651a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.264+0000 7f8d54a15740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651aca0 0x5622d6529550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.264+0000 7f8d54a15740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d6529a90 0x5622d652be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.264+0000 7f8d54a15740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5622d64802e0 con 0x5622d651a380 2024-09-11T23:01:44.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.264+0000 7f8d54a15740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5622d6427ce0 con 0x5622d6529a90 2024-09-11T23:01:44.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.264+0000 7f8d54a15740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5622d647c2a0 con 0x5622d651aca0 2024-09-11T23:01:44.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.264+0000 7f8d497fa640 1 --2- 172.21.15.152:0/1550124094 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8d340a2cf0 0x7f8d340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.264+0000 7f8d50bd9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651a380 0x5622d651a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.264+0000 7f8d50bd9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651a380 0x5622d651a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39370/0 (socket says 172.21.15.152:39370) 2024-09-11T23:01:44.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.264+0000 7f8d50bd9640 1 -- 172.21.15.152:0/944795703 learned_addr learned my addr 172.21.15.152:0/944795703 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:44.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.264+0000 7f8d50bd9640 1 --2- 172.21.15.152:0/1550124094 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8d340a2cf0 0x7f8d340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.264+0000 7f8d497fa640 1 -- 172.21.15.152:0/1550124094 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f8d3c0cf0c0 con 0x7f8d4c102d30 2024-09-11T23:01:44.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.265+0000 7f8d4bfff640 1 --2- 172.21.15.152:0/944795703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651aca0 0x5622d6529550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.265+0000 7f8d4b7fe640 1 --2- 172.21.15.152:0/944795703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d6529a90 0x5622d652be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.265+0000 7f8d50bd9640 1 -- 172.21.15.152:0/944795703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651aca0 msgr2=0x5622d6529550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:44.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.265+0000 7f8d50bd9640 1 --2- 172.21.15.152:0/944795703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651aca0 0x5622d6529550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.265+0000 7f8d50bd9640 1 -- 172.21.15.152:0/944795703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d6529a90 msgr2=0x5622d652be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:44.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.265+0000 7f8d50bd9640 1 --2- 172.21.15.152:0/944795703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d6529a90 0x5622d652be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.265+0000 7f8d50bd9640 1 -- 172.21.15.152:0/944795703 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622d640c440 con 0x5622d651a380 2024-09-11T23:01:44.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.265+0000 7f8d4b7fe640 1 --2- 172.21.15.152:0/944795703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d6529a90 0x5622d652be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:01:44.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.265+0000 7f8d4bfff640 1 --2- 172.21.15.152:0/944795703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651aca0 0x5622d6529550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:01:44.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.265+0000 7f8d50bd9640 1 --2- 172.21.15.152:0/944795703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651a380 0x5622d651a760 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f8d44012d00 tx=0x7f8d44010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7abdeb0aaa2fa0a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:44.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.265+0000 7f8d50bd9640 1 --2- 172.21.15.152:0/1550124094 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8d340a2cf0 0x7f8d340a51b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8d44010c40 tx=0x7f8d44017000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:44.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.265+0000 7f8d327fc640 1 -- 172.21.15.152:0/944795703 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d44024070 con 0x5622d651a380 2024-09-11T23:01:44.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.265+0000 7f8d327fc640 1 -- 172.21.15.152:0/944795703 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d44012520 con 0x5622d651a380 2024-09-11T23:01:44.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.265+0000 7f8d327fc640 1 -- 172.21.15.152:0/944795703 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d44005220 con 0x5622d651a380 2024-09-11T23:01:44.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.266+0000 7f8d54a15740 1 -- 172.21.15.152:0/944795703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651a380 msgr2=0x5622d651a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:44.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.266+0000 7f8d54a15740 1 --2- 172.21.15.152:0/944795703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651a380 0x5622d651a760 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f8d44012d00 tx=0x7f8d44010740 comp rx=0 tx=0).stop 2024-09-11T23:01:44.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.266+0000 7f8d54a15740 1 -- 172.21.15.152:0/944795703 shutdown_connections 2024-09-11T23:01:44.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.266+0000 7f8d54a15740 1 --2- 172.21.15.152:0/944795703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d6529a90 0x5622d652be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.266+0000 7f8d54a15740 1 --2- 172.21.15.152:0/944795703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651aca0 0x5622d6529550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.266+0000 7f8d54a15740 1 --2- 172.21.15.152:0/944795703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651a380 0x5622d651a760 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.266+0000 7f8d54a15740 1 -- 172.21.15.152:0/944795703 >> 172.21.15.152:0/944795703 conn(0x5622d6516a30 msgr2=0x5622d6516e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:44.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.266+0000 7f8d54a15740 1 -- 172.21.15.152:0/944795703 shutdown_connections 2024-09-11T23:01:44.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.266+0000 7f8d54a15740 1 -- 172.21.15.152:0/944795703 wait complete. 2024-09-11T23:01:44.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.266+0000 7f8d54a15740 1 -- start start 2024-09-11T23:01:44.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d54a15740 1 Processor -- start 2024-09-11T23:01:44.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d54a15740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d651a380 0x5622d6529460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d54a15740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651aca0 0x5622d6526710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d54a15740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d6529a90 0x5622d6526d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d54a15740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5622d652c660 con 0x5622d6529a90 2024-09-11T23:01:44.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d54a15740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5622d643b050 con 0x5622d651a380 2024-09-11T23:01:44.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d54a15740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5622d652c7e0 con 0x5622d651aca0 2024-09-11T23:01:44.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d54a15740 10 client.0 _ll_register_callbacks cb 0x5622d647b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:44.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d50bd9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d6529a90 0x5622d6526d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d50bd9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d6529a90 0x5622d6526d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39374/0 (socket says 172.21.15.152:39374) 2024-09-11T23:01:44.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d50bd9640 1 -- 172.21.15.152:0/16957841 learned_addr learned my addr 172.21.15.152:0/16957841 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:44.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d4bfff640 1 --2- 172.21.15.152:0/16957841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d651a380 0x5622d6529460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d4b7fe640 1 --2- 172.21.15.152:0/16957841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651aca0 0x5622d6526710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d50bd9640 1 -- 172.21.15.152:0/16957841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651aca0 msgr2=0x5622d6526710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:44.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d50bd9640 1 --2- 172.21.15.152:0/16957841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651aca0 0x5622d6526710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d50bd9640 1 -- 172.21.15.152:0/16957841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d651a380 msgr2=0x5622d6529460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:44.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d50bd9640 1 --2- 172.21.15.152:0/16957841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d651a380 0x5622d6529460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d50bd9640 1 -- 172.21.15.152:0/16957841 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622d61f13c0 con 0x5622d6529a90 2024-09-11T23:01:44.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d4bfff640 1 --2- 172.21.15.152:0/16957841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d651a380 0x5622d6529460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:01:44.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d50bd9640 1 --2- 172.21.15.152:0/16957841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d6529a90 0x5622d6526d00 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400e010 tx=0x7f8d44009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbcb6225dc41585 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:44.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.267+0000 7f8d2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:44.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.268+0000 7f8d2b7fe640 1 -- 172.21.15.152:0/16957841 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d4402e070 con 0x5622d6529a90 2024-09-11T23:01:44.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.268+0000 7f8d2b7fe640 1 -- 172.21.15.152:0/16957841 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f8d440056a0 con 0x5622d6529a90 2024-09-11T23:01:44.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.268+0000 7f8d54a15740 1 -- 172.21.15.152:0/16957841 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d44012520 con 0x5622d6529a90 2024-09-11T23:01:44.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.268+0000 7f8d2b7fe640 1 -- 172.21.15.152:0/16957841 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d44024040 con 0x5622d6529a90 2024-09-11T23:01:44.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.268+0000 7f8d54a15740 10 client.17850 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:44.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.268+0000 7f8d54a15740 1 -- 172.21.15.152:0/16957841 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622d6538040 con 0x5622d6529a90 2024-09-11T23:01:44.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.268+0000 7f8d2b7fe640 1 -- 172.21.15.152:0/16957841 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d44012520 con 0x5622d6529a90 2024-09-11T23:01:44.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.268+0000 7f8d2b7fe640 1 -- 172.21.15.152:0/16957841 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8d440204c0 con 0x5622d6529a90 2024-09-11T23:01:44.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.268+0000 7f8d54a15740 10 client.17850 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:44.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.268+0000 7f8d54a15740 10 client.17850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:01:44.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.268+0000 7f8d54a15740 20 client.17850 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:44.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.268+0000 7f8d54a15740 1 --2- 172.21.15.152:0/16957841 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5622d65393f0 0x5622d653b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.268+0000 7f8d54a15740 4 client.17850 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:01:44.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.268+0000 7f8d54a15740 1 -- 172.21.15.152:0/16957841 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622d648d850 con 0x5622d65393f0 2024-09-11T23:01:44.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.268+0000 7f8d4bfff640 1 --2- 172.21.15.152:0/16957841 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5622d65393f0 0x5622d653b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.269+0000 7f8d4bfff640 1 --2- 172.21.15.152:0/16957841 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5622d65393f0 0x5622d653b7e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efe91b61a2691a4 server_cookie=9f236ba6a056020e in_seq=0 out_seq=0 2024-09-11T23:01:44.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.269+0000 7f8d2b7fe640 10 client.17850 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:44.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.270+0000 7f8d2b7fe640 1 -- 172.21.15.152:0/16957841 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622d648d850 con 0x5622d65393f0 2024-09-11T23:01:44.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.270+0000 7f8d2b7fe640 10 client.17850 handle_command_reply: tid=0 2024-09-11T23:01:44.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.270+0000 7f8d54a15740 1 client.17850 shutdown 2024-09-11T23:01:44.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.270+0000 7f8d54a15740 1 -- 172.21.15.152:0/16957841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d6529a90 msgr2=0x5622d6526d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:44.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.270+0000 7f8d54a15740 1 --2- 172.21.15.152:0/16957841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d6529a90 0x5622d6526d00 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400e010 tx=0x7f8d44009e00 comp rx=0 tx=0).stop 2024-09-11T23:01:44.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.270+0000 7f8d54a15740 1 -- 172.21.15.152:0/16957841 shutdown_connections 2024-09-11T23:01:44.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.270+0000 7f8d54a15740 1 --2- 172.21.15.152:0/16957841 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d6529a90 0x5622d6526d00 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.270+0000 7f8d54a15740 1 --2- 172.21.15.152:0/16957841 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651aca0 0x5622d6526710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.270+0000 7f8d54a15740 1 --2- 172.21.15.152:0/16957841 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5622d65393f0 0x5622d653b7e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.270+0000 7f8d54a15740 1 --2- 172.21.15.152:0/16957841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d651a380 0x5622d6529460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.270+0000 7f8d2b7fe640 0 client.17850 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.270+0000 7f8d54a15740 1 -- 172.21.15.152:0/16957841 >> 172.21.15.152:0/16957841 conn(0x5622d6516a30 msgr2=0x5622d651c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.270+0000 7f8d54a15740 1 -- 172.21.15.152:0/16957841 shutdown_connections 2024-09-11T23:01:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.270+0000 7f8d54a15740 1 -- 172.21.15.152:0/16957841 wait complete. 2024-09-11T23:01:44.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.270+0000 7f8d54a15740 20 client.17850 trim_cache size 0 max 16384 2024-09-11T23:01:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d54a15740 1 Processor -- start 2024-09-11T23:01:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d54a15740 1 -- start start 2024-09-11T23:01:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d54a15740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651a380 0x5622d6535900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d54a15740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d651aca0 0x5622d6540db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d54a15740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d6529a90 0x5622d65412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d54a15740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c1ab7a0 con 0x5622d651a380 2024-09-11T23:01:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d54a15740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c1ad2e0 con 0x5622d651aca0 2024-09-11T23:01:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d54a15740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c1abe70 con 0x5622d6529a90 2024-09-11T23:01:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d4bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651a380 0x5622d6535900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d4bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651a380 0x5622d6535900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39388/0 (socket says 172.21.15.152:39388) 2024-09-11T23:01:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d4bfff640 1 -- 172.21.15.152:0/2961176402 learned_addr learned my addr 172.21.15.152:0/2961176402 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:44.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d4b7fe640 1 --2- 172.21.15.152:0/2961176402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d651aca0 0x5622d6540db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d4bfff640 1 -- 172.21.15.152:0/2961176402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d6529a90 msgr2=0x5622d65412f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d4bfff640 1 --2- 172.21.15.152:0/2961176402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d6529a90 0x5622d65412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d4bfff640 1 -- 172.21.15.152:0/2961176402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d651aca0 msgr2=0x5622d6540db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d4bfff640 1 --2- 172.21.15.152:0/2961176402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d651aca0 0x5622d6540db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.287+0000 7f8d4bfff640 1 -- 172.21.15.152:0/2961176402 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622d65297d0 con 0x5622d651a380 2024-09-11T23:01:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.288+0000 7f8d4bfff640 1 --2- 172.21.15.152:0/2961176402 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651a380 0x5622d6535900 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c010fd0 tx=0x7f8d3c090a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794690042a19b372 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.288+0000 7f8d31ffb640 1 -- 172.21.15.152:0/2961176402 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c095070 con 0x5622d651a380 2024-09-11T23:01:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.288+0000 7f8d31ffb640 1 -- 172.21.15.152:0/2961176402 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3c09f9b0 con 0x5622d651a380 2024-09-11T23:01:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.288+0000 7f8d31ffb640 1 -- 172.21.15.152:0/2961176402 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c014030 con 0x5622d651a380 2024-09-11T23:01:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.288+0000 7f8d54a15740 1 -- 172.21.15.152:0/2961176402 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651a380 msgr2=0x5622d6535900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.288+0000 7f8d54a15740 1 --2- 172.21.15.152:0/2961176402 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651a380 0x5622d6535900 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c010fd0 tx=0x7f8d3c090a90 comp rx=0 tx=0).stop 2024-09-11T23:01:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.288+0000 7f8d54a15740 1 -- 172.21.15.152:0/2961176402 shutdown_connections 2024-09-11T23:01:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.288+0000 7f8d54a15740 1 --2- 172.21.15.152:0/2961176402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d6529a90 0x5622d65412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.288+0000 7f8d54a15740 1 --2- 172.21.15.152:0/2961176402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d651aca0 0x5622d6540db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.288+0000 7f8d54a15740 1 --2- 172.21.15.152:0/2961176402 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651a380 0x5622d6535900 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.288+0000 7f8d54a15740 1 -- 172.21.15.152:0/2961176402 >> 172.21.15.152:0/2961176402 conn(0x5622d6516a30 msgr2=0x5622d6528790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.288+0000 7f8d54a15740 1 -- 172.21.15.152:0/2961176402 shutdown_connections 2024-09-11T23:01:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.288+0000 7f8d54a15740 1 -- 172.21.15.152:0/2961176402 wait complete. 2024-09-11T23:01:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.289+0000 7f8d54a15740 1 -- start start 2024-09-11T23:01:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.289+0000 7f8d54a15740 1 Processor -- start 2024-09-11T23:01:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.289+0000 7f8d54a15740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651a380 0x5622d65322a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.289+0000 7f8d54a15740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651aca0 0x5622d652d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.289+0000 7f8d54a15740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d6529a90 0x5622d652dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.289+0000 7f8d54a15740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c1acc10 con 0x5622d651aca0 2024-09-11T23:01:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.289+0000 7f8d54a15740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c1ac540 con 0x5622d6529a90 2024-09-11T23:01:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.289+0000 7f8d54a15740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c1a97a0 con 0x5622d651a380 2024-09-11T23:01:44.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.289+0000 7f8d54a15740 10 client.0 _ll_register_callbacks cb 0x7f8d44012b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.289+0000 7f8d4b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651aca0 0x5622d652d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.289+0000 7f8d4b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651aca0 0x5622d652d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39396/0 (socket says 172.21.15.152:39396) 2024-09-11T23:01:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.289+0000 7f8d4b7fe640 1 -- 172.21.15.152:0/3765582539 learned_addr learned my addr 172.21.15.152:0/3765582539 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d4bfff640 1 --2- 172.21.15.152:0/3765582539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651a380 0x5622d65322a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d50bd9640 1 --2- 172.21.15.152:0/3765582539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d6529a90 0x5622d652dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d4b7fe640 1 -- 172.21.15.152:0/3765582539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651a380 msgr2=0x5622d65322a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d4b7fe640 1 --2- 172.21.15.152:0/3765582539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651a380 0x5622d65322a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d4b7fe640 1 -- 172.21.15.152:0/3765582539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d6529a90 msgr2=0x5622d652dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d4b7fe640 1 --2- 172.21.15.152:0/3765582539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d6529a90 0x5622d652dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d4b7fe640 1 -- 172.21.15.152:0/3765582539 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622d652e3f0 con 0x5622d651aca0 2024-09-11T23:01:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d50bd9640 1 --2- 172.21.15.152:0/3765582539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d6529a90 0x5622d652dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:01:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d4bfff640 1 --2- 172.21.15.152:0/3765582539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651a380 0x5622d65322a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:01:44.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d4b7fe640 1 --2- 172.21.15.152:0/3765582539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651aca0 0x5622d652d5d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f8d40031550 tx=0x7f8d40031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f8a8cbf6449e04 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d2b7fe640 1 -- 172.21.15.152:0/3765582539 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d40035040 con 0x5622d651aca0 2024-09-11T23:01:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d2b7fe640 1 -- 172.21.15.152:0/3765582539 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f8d40005030 con 0x5622d651aca0 2024-09-11T23:01:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d54a15740 1 -- 172.21.15.152:0/3765582539 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d3c09f9b0 con 0x5622d651aca0 2024-09-11T23:01:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d2b7fe640 1 -- 172.21.15.152:0/3765582539 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8d40005230 con 0x5622d651aca0 2024-09-11T23:01:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d54a15740 10 client.17856 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d54a15740 1 -- 172.21.15.152:0/3765582539 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5622d65a3e70 con 0x5622d651aca0 2024-09-11T23:01:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.290+0000 7f8d2b7fe640 1 -- 172.21.15.152:0/3765582539 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d3c09f9b0 con 0x5622d651aca0 2024-09-11T23:01:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.291+0000 7f8d2b7fe640 1 -- 172.21.15.152:0/3765582539 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8d400035f0 con 0x5622d651aca0 2024-09-11T23:01:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.291+0000 7f8d54a15740 10 client.17856 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.291+0000 7f8d54a15740 10 client.17856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:01:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.291+0000 7f8d54a15740 20 client.17856 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.291+0000 7f8d54a15740 1 --2- 172.21.15.152:0/3765582539 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5622d65a4620 0x5622d65a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:44.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.291+0000 7f8d54a15740 4 client.17856 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:01:44.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.291+0000 7f8d54a15740 1 -- 172.21.15.152:0/3765582539 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5622d6487d60 con 0x5622d65a4620 2024-09-11T23:01:44.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.291+0000 7f8d4bfff640 1 --2- 172.21.15.152:0/3765582539 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5622d65a4620 0x5622d65a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:44.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.291+0000 7f8d4bfff640 1 --2- 172.21.15.152:0/3765582539 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5622d65a4620 0x5622d65a6a40 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6638bc9814e9fd09 server_cookie=507d3dcddad5a0b5 in_seq=0 out_seq=0 2024-09-11T23:01:44.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.291+0000 7f8d2b7fe640 10 client.17856 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:44.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.291+0000 7f8d2b7fe640 1 -- 172.21.15.152:0/3765582539 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5622d6487d60 con 0x5622d65a4620 2024-09-11T23:01:44.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.291+0000 7f8d2b7fe640 10 client.17856 handle_command_reply: tid=0 2024-09-11T23:01:44.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d54a15740 1 client.17856 shutdown 2024-09-11T23:01:44.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d54a15740 1 -- 172.21.15.152:0/3765582539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651aca0 msgr2=0x5622d652d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:44.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d54a15740 1 --2- 172.21.15.152:0/3765582539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651aca0 0x5622d652d5d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f8d40031550 tx=0x7f8d40031b10 comp rx=0 tx=0).stop 2024-09-11T23:01:44.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d54a15740 1 -- 172.21.15.152:0/3765582539 shutdown_connections 2024-09-11T23:01:44.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d54a15740 1 --2- 172.21.15.152:0/3765582539 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5622d65a4620 0x5622d65a6a40 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d2b7fe640 0 client.17856 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:44.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d54a15740 1 --2- 172.21.15.152:0/3765582539 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5622d6529a90 0x5622d652dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d54a15740 1 --2- 172.21.15.152:0/3765582539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5622d651aca0 0x5622d652d5d0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d54a15740 1 --2- 172.21.15.152:0/3765582539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5622d651a380 0x5622d65322a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d54a15740 1 -- 172.21.15.152:0/3765582539 >> 172.21.15.152:0/3765582539 conn(0x5622d6516a30 msgr2=0x5622d6515350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:44.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d54a15740 1 -- 172.21.15.152:0/3765582539 shutdown_connections 2024-09-11T23:01:44.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d54a15740 1 -- 172.21.15.152:0/3765582539 wait complete. 2024-09-11T23:01:44.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d54a15740 20 client.17856 trim_cache size 0 max 16384 2024-09-11T23:01:44.300 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:01:44.300 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T23:01:44.300 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T23:01:44.300 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:01:44.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d5263c640 1 -- 172.21.15.152:0/1550124094 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8d340a2cf0 msgr2=0x7f8d340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:44.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d5263c640 1 --2- 172.21.15.152:0/1550124094 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8d340a2cf0 0x7f8d340a51b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8d44010c40 tx=0x7f8d44017000 comp rx=0 tx=0).stop 2024-09-11T23:01:44.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d5263c640 1 -- 172.21.15.152:0/1550124094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d4c102d30 msgr2=0x7f8d4c1985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:44.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.292+0000 7f8d5263c640 1 --2- 172.21.15.152:0/1550124094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d4c102d30 0x7f8d4c1985a0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00d950 tx=0x7f8d3c00de10 comp rx=0 tx=0).stop 2024-09-11T23:01:44.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.293+0000 7f8d5263c640 1 -- 172.21.15.152:0/1550124094 shutdown_connections 2024-09-11T23:01:44.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.293+0000 7f8d5263c640 1 --2- 172.21.15.152:0/1550124094 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8d340a2cf0 0x7f8d340a51b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.293+0000 7f8d5263c640 1 --2- 172.21.15.152:0/1550124094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d4c1082a0 0x7f8d4c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.293+0000 7f8d5263c640 1 --2- 172.21.15.152:0/1550124094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d4c1036e0 0x7f8d4c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.293+0000 7f8d5263c640 1 --2- 172.21.15.152:0/1550124094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d4c102d30 0x7f8d4c1985a0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.293+0000 7f8d5263c640 1 -- 172.21.15.152:0/1550124094 >> 172.21.15.152:0/1550124094 conn(0x7f8d4c0fc7d0 msgr2=0x7f8d4c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.293+0000 7f8d5263c640 1 -- 172.21.15.152:0/1550124094 shutdown_connections 2024-09-11T23:01:44.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:44.293+0000 7f8d5263c640 1 -- 172.21.15.152:0/1550124094 wait complete. 2024-09-11T23:01:45.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:44 smithi191 ceph-mon[35211]: pgmap v272: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:45.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:44 smithi204 ceph-mon[35454]: pgmap v272: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:45.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:44 smithi152 ceph-mon[31181]: pgmap v272: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:45.222 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:01:45.223 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:01:45.474 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:01:46.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.991+0000 7f099d769640 1 -- 172.21.15.152:0/2848314481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0998106a70 msgr2=0x7f0998106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:46.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.991+0000 7f099d769640 1 --2- 172.21.15.152:0/2848314481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0998106a70 0x7f0998106e50 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f0988009930 tx=0x7f098802a420 comp rx=0 tx=0).stop 2024-09-11T23:01:46.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.991+0000 7f099d769640 1 -- 172.21.15.152:0/2848314481 shutdown_connections 2024-09-11T23:01:46.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.991+0000 7f099d769640 1 --2- 172.21.15.152:0/2848314481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0998101410 0x7f099810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:46.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.991+0000 7f099d769640 1 --2- 172.21.15.152:0/2848314481 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0998100a70 0x7f0998100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:46.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.991+0000 7f099d769640 1 --2- 172.21.15.152:0/2848314481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0998106a70 0x7f0998106e50 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:46.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.991+0000 7f099d769640 1 -- 172.21.15.152:0/2848314481 >> 172.21.15.152:0/2848314481 conn(0x7f09980fc790 msgr2=0x7f09980febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:46.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.991+0000 7f099d769640 1 -- 172.21.15.152:0/2848314481 shutdown_connections 2024-09-11T23:01:46.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.991+0000 7f099d769640 1 -- 172.21.15.152:0/2848314481 wait complete. 2024-09-11T23:01:46.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.992+0000 7f099d769640 1 Processor -- start 2024-09-11T23:01:46.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.992+0000 7f099d769640 1 -- start start 2024-09-11T23:01:46.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.992+0000 7f099d769640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0998100a70 0x7f0998198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:46.015 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.992+0000 7f099d769640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0998101410 0x7f0998198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:46.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.992+0000 7f099d769640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0998106a70 0x7f099819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:46.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.992+0000 7f099d769640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0998110e70 con 0x7f0998100a70 2024-09-11T23:01:46.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.992+0000 7f099d769640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0998110cf0 con 0x7f0998106a70 2024-09-11T23:01:46.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.992+0000 7f099d769640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0998110ff0 con 0x7f0998101410 2024-09-11T23:01:46.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.992+0000 7f0996ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0998100a70 0x7f0998198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:46.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.992+0000 7f0996ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0998100a70 0x7f0998198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39420/0 (socket says 172.21.15.152:39420) 2024-09-11T23:01:46.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.992+0000 7f0996ffd640 1 -- 172.21.15.152:0/3998545586 learned_addr learned my addr 172.21.15.152:0/3998545586 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:46.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.992+0000 7f09977fe640 1 --2- 172.21.15.152:0/3998545586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0998106a70 0x7f099819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:46.016 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.992+0000 7f0996ffd640 1 -- 172.21.15.152:0/3998545586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0998101410 msgr2=0x7f0998198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:46.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.992+0000 7f09967fc640 1 --2- 172.21.15.152:0/3998545586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0998101410 0x7f0998198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:46.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.993+0000 7f0996ffd640 1 --2- 172.21.15.152:0/3998545586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0998101410 0x7f0998198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:46.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.993+0000 7f0996ffd640 1 -- 172.21.15.152:0/3998545586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0998106a70 msgr2=0x7f099819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:46.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.993+0000 7f0996ffd640 1 --2- 172.21.15.152:0/3998545586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0998106a70 0x7f099819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:46.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.993+0000 7f0996ffd640 1 -- 172.21.15.152:0/3998545586 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099819d3f0 con 0x7f0998100a70 2024-09-11T23:01:46.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.993+0000 7f09977fe640 1 --2- 172.21.15.152:0/3998545586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0998106a70 0x7f099819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:01:46.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.993+0000 7f0996ffd640 1 --2- 172.21.15.152:0/3998545586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0998100a70 0x7f0998198440 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f0988009230 tx=0x7f0988004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:46.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.993+0000 7f097ffff640 1 -- 172.21.15.152:0/3998545586 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098803f030 con 0x7f0998100a70 2024-09-11T23:01:46.017 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.993+0000 7f099d769640 1 -- 172.21.15.152:0/3998545586 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f099819d680 con 0x7f0998100a70 2024-09-11T23:01:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.993+0000 7f097ffff640 1 -- 172.21.15.152:0/3998545586 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0988003ad0 con 0x7f0998100a70 2024-09-11T23:01:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.993+0000 7f097ffff640 1 -- 172.21.15.152:0/3998545586 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988035070 con 0x7f0998100a70 2024-09-11T23:01:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.993+0000 7f099d769640 1 -- 172.21.15.152:0/3998545586 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09981a4fc0 con 0x7f0998100a70 2024-09-11T23:01:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.994+0000 7f099d769640 1 -- 172.21.15.152:0/3998545586 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0998100ed0 con 0x7f0998100a70 2024-09-11T23:01:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.995+0000 7f097ffff640 1 -- 172.21.15.152:0/3998545586 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0988042030 con 0x7f0998100a70 2024-09-11T23:01:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.996+0000 7f097ffff640 1 --2- 172.21.15.152:0/3998545586 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f09800a2cf0 0x7f09800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.996+0000 7f097ffff640 1 -- 172.21.15.152:0/3998545586 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f09880ef9e0 con 0x7f0998100a70 2024-09-11T23:01:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.996+0000 7f09967fc640 1 --2- 172.21.15.152:0/3998545586 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f09800a2cf0 0x7f09800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:46.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:45.996+0000 7f09967fc640 1 --2- 172.21.15.152:0/3998545586 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f09800a2cf0 0x7f09800a51b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0998104a50 tx=0x7f098c005c60 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:46.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:46.000+0000 7f097ffff640 1 -- 172.21.15.152:0/3998545586 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09880bc620 con 0x7f0998100a70 2024-09-11T23:01:46.202 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:46.201+0000 7f099d769640 1 -- 172.21.15.152:0/3998545586 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0998109ff0 con 0x7f0998100a70 2024-09-11T23:01:46.203 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:46.202+0000 7f097ffff640 1 -- 172.21.15.152:0/3998545586 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f09880bbf20 con 0x7f0998100a70 2024-09-11T23:01:46.204 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T23:01:46.204 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T23:01:46.204 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T23:01:46.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:46.204+0000 7f099d769640 1 -- 172.21.15.152:0/3998545586 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f09800a2cf0 msgr2=0x7f09800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:46.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:46.204+0000 7f099d769640 1 --2- 172.21.15.152:0/3998545586 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f09800a2cf0 0x7f09800a51b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0998104a50 tx=0x7f098c005c60 comp rx=0 tx=0).stop 2024-09-11T23:01:46.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:46.204+0000 7f099d769640 1 -- 172.21.15.152:0/3998545586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0998100a70 msgr2=0x7f0998198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:46.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:46.204+0000 7f099d769640 1 --2- 172.21.15.152:0/3998545586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0998100a70 0x7f0998198440 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f0988009230 tx=0x7f0988004270 comp rx=0 tx=0).stop 2024-09-11T23:01:46.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:46.204+0000 7f099d769640 1 -- 172.21.15.152:0/3998545586 shutdown_connections 2024-09-11T23:01:46.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:46.204+0000 7f099d769640 1 --2- 172.21.15.152:0/3998545586 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f09800a2cf0 0x7f09800a51b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:46.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:46.204+0000 7f099d769640 1 --2- 172.21.15.152:0/3998545586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0998106a70 0x7f099819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:46.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:46.204+0000 7f099d769640 1 --2- 172.21.15.152:0/3998545586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0998101410 0x7f0998198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:46.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:46.204+0000 7f099d769640 1 --2- 172.21.15.152:0/3998545586 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0998100a70 0x7f0998198440 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:46.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:46.204+0000 7f099d769640 1 -- 172.21.15.152:0/3998545586 >> 172.21.15.152:0/3998545586 conn(0x7f09980fc790 msgr2=0x7f099806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:46.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:46.205+0000 7f099d769640 1 -- 172.21.15.152:0/3998545586 shutdown_connections 2024-09-11T23:01:46.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:46.205+0000 7f099d769640 1 -- 172.21.15.152:0/3998545586 wait complete. 2024-09-11T23:01:46.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:46 smithi152 ceph-mon[31181]: pgmap v273: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:01:46.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:46 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3998545586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:01:46.935 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:01:47.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:46 smithi191 ceph-mon[35211]: pgmap v273: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:01:47.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:46 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3998545586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:01:47.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:46 smithi204 ceph-mon[35454]: pgmap v273: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:01:47.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:46 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3998545586' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:01:47.188 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:01:47.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.731+0000 7fcf5e352640 1 -- 172.21.15.152:0/1572734871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf58106f10 msgr2=0x7fcf5810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:47.749 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.731+0000 7fcf5e352640 1 --2- 172.21.15.152:0/1572734871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf58106f10 0x7fcf5810ea90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009ca0 tx=0x7fcf5002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.731+0000 7fcf5e352640 1 -- 172.21.15.152:0/1572734871 shutdown_connections 2024-09-11T23:01:47.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.731+0000 7fcf5e352640 1 --2- 172.21.15.152:0/1572734871 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf58106f10 0x7fcf5810ea90 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.731+0000 7fcf5e352640 1 --2- 172.21.15.152:0/1572734871 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf58102790 0x7fcf581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.731+0000 7fcf5e352640 1 --2- 172.21.15.152:0/1572734871 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf58101de0 0x7fcf581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.731+0000 7fcf5e352640 1 -- 172.21.15.152:0/1572734871 >> 172.21.15.152:0/1572734871 conn(0x7fcf580fd8c0 msgr2=0x7fcf580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:47.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.731+0000 7fcf5e352640 1 -- 172.21.15.152:0/1572734871 shutdown_connections 2024-09-11T23:01:47.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.731+0000 7fcf5e352640 1 -- 172.21.15.152:0/1572734871 wait complete. 2024-09-11T23:01:47.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.732+0000 7fcf5e352640 1 Processor -- start 2024-09-11T23:01:47.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.732+0000 7fcf5e352640 1 -- start start 2024-09-11T23:01:47.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.732+0000 7fcf5e352640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf58101de0 0x7fcf58198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.732+0000 7fcf5e352640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf58102790 0x7fcf581989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.750 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.732+0000 7fcf5e352640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf58106f10 0x7fcf5819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.732+0000 7fcf5e352640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcf58111fb0 con 0x7fcf58102790 2024-09-11T23:01:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.732+0000 7fcf5e352640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcf58111e30 con 0x7fcf58101de0 2024-09-11T23:01:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.732+0000 7fcf5e352640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcf58112130 con 0x7fcf58106f10 2024-09-11T23:01:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.732+0000 7fcf577fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf58102790 0x7fcf581989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.732+0000 7fcf577fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf58102790 0x7fcf581989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39426/0 (socket says 172.21.15.152:39426) 2024-09-11T23:01:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.732+0000 7fcf577fe640 1 -- 172.21.15.152:0/2900916495 learned_addr learned my addr 172.21.15.152:0/2900916495 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.732+0000 7fcf5c8ef640 1 --2- 172.21.15.152:0/2900916495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf58106f10 0x7fcf5819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.732+0000 7fcf57fff640 1 --2- 172.21.15.152:0/2900916495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf58101de0 0x7fcf58198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.733+0000 7fcf577fe640 1 -- 172.21.15.152:0/2900916495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf58106f10 msgr2=0x7fcf5819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.733+0000 7fcf577fe640 1 --2- 172.21.15.152:0/2900916495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf58106f10 0x7fcf5819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.733+0000 7fcf577fe640 1 -- 172.21.15.152:0/2900916495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf58101de0 msgr2=0x7fcf58198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:47.751 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.733+0000 7fcf577fe640 1 --2- 172.21.15.152:0/2900916495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf58101de0 0x7fcf58198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.733+0000 7fcf577fe640 1 -- 172.21.15.152:0/2900916495 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5819d440 con 0x7fcf58102790 2024-09-11T23:01:47.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.733+0000 7fcf57fff640 1 --2- 172.21.15.152:0/2900916495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf58101de0 0x7fcf58198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:01:47.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.733+0000 7fcf577fe640 1 --2- 172.21.15.152:0/2900916495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf58102790 0x7fcf581989d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00b4a0 tx=0x7fcf4c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:47.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.733+0000 7fcf557fa640 1 -- 172.21.15.152:0/2900916495 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c014070 con 0x7fcf58102790 2024-09-11T23:01:47.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.733+0000 7fcf5e352640 1 -- 172.21.15.152:0/2900916495 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5819d730 con 0x7fcf58102790 2024-09-11T23:01:47.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.733+0000 7fcf557fa640 1 -- 172.21.15.152:0/2900916495 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf4c010af0 con 0x7fcf58102790 2024-09-11T23:01:47.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.733+0000 7fcf557fa640 1 -- 172.21.15.152:0/2900916495 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c017d10 con 0x7fcf58102790 2024-09-11T23:01:47.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.733+0000 7fcf5e352640 1 -- 172.21.15.152:0/2900916495 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf581a5010 con 0x7fcf58102790 2024-09-11T23:01:47.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.735+0000 7fcf557fa640 1 -- 172.21.15.152:0/2900916495 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fcf4c017070 con 0x7fcf58102790 2024-09-11T23:01:47.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.735+0000 7fcf6072b740 1 Processor -- start 2024-09-11T23:01:47.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.735+0000 7fcf6072b740 1 -- start start 2024-09-11T23:01:47.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf6072b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a06b40 0x563d88a05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf6072b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a055d0 0x563d88a05a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf6072b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a10370 0x563d88a12760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf6072b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x563d8896b2f0 con 0x563d88a055d0 2024-09-11T23:01:47.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf6072b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563d88912cf0 con 0x563d88a10370 2024-09-11T23:01:47.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf6072b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563d889672b0 con 0x563d88a06b40 2024-09-11T23:01:47.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf57fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a055d0 0x563d88a05a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf557fa640 1 --2- 172.21.15.152:0/2900916495 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcf400a2cf0 0x7fcf400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.753 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf57fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a055d0 0x563d88a05a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39440/0 (socket says 172.21.15.152:39440) 2024-09-11T23:01:47.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf57fff640 1 -- 172.21.15.152:0/1546794678 learned_addr learned my addr 172.21.15.152:0/1546794678 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:47.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf5c8ef640 1 --2- 172.21.15.152:0/1546794678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a06b40 0x563d88a05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf5c8ef640 1 --2- 172.21.15.152:0/2900916495 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcf400a2cf0 0x7fcf400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf577fe640 1 --2- 172.21.15.152:0/1546794678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a10370 0x563d88a12760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf557fa640 1 -- 172.21.15.152:0/2900916495 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7fcf4c0ce6f0 con 0x7fcf58102790 2024-09-11T23:01:47.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf57fff640 1 -- 172.21.15.152:0/1546794678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a06b40 msgr2=0x563d88a05090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:47.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf57fff640 1 --2- 172.21.15.152:0/1546794678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a06b40 0x563d88a05090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf57fff640 1 -- 172.21.15.152:0/1546794678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a10370 msgr2=0x563d88a12760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:47.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf57fff640 1 --2- 172.21.15.152:0/1546794678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a10370 0x563d88a12760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf57fff640 1 -- 172.21.15.152:0/1546794678 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d888f73a0 con 0x563d88a055d0 2024-09-11T23:01:47.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf57fff640 1 --2- 172.21.15.152:0/1546794678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a055d0 0x563d88a05a10 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800bb70 tx=0x7fcf4800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9708b29c1bebdb8f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:47.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf5c8ef640 1 --2- 172.21.15.152:0/2900916495 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcf400a2cf0 0x7fcf400a51b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcf50005fe0 tx=0x7fcf50009380 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:47.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.736+0000 7fcf3e7fc640 1 -- 172.21.15.152:0/1546794678 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48017620 con 0x563d88a055d0 2024-09-11T23:01:47.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.737+0000 7fcf3e7fc640 1 -- 172.21.15.152:0/1546794678 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf48017ca0 con 0x563d88a055d0 2024-09-11T23:01:47.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.737+0000 7fcf3e7fc640 1 -- 172.21.15.152:0/1546794678 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4800fca0 con 0x563d88a055d0 2024-09-11T23:01:47.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.737+0000 7fcf6072b740 1 -- 172.21.15.152:0/1546794678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a055d0 msgr2=0x563d88a05a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:47.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.737+0000 7fcf6072b740 1 --2- 172.21.15.152:0/1546794678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a055d0 0x563d88a05a10 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800bb70 tx=0x7fcf4800d710 comp rx=0 tx=0).stop 2024-09-11T23:01:47.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.737+0000 7fcf6072b740 1 -- 172.21.15.152:0/1546794678 shutdown_connections 2024-09-11T23:01:47.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.737+0000 7fcf6072b740 1 --2- 172.21.15.152:0/1546794678 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a10370 0x563d88a12760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.737+0000 7fcf6072b740 1 --2- 172.21.15.152:0/1546794678 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a055d0 0x563d88a05a10 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.737+0000 7fcf6072b740 1 --2- 172.21.15.152:0/1546794678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a06b40 0x563d88a05090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.737+0000 7fcf6072b740 1 -- 172.21.15.152:0/1546794678 >> 172.21.15.152:0/1546794678 conn(0x563d889ce310 msgr2=0x563d88a01c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:47.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.737+0000 7fcf6072b740 1 -- 172.21.15.152:0/1546794678 shutdown_connections 2024-09-11T23:01:47.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.737+0000 7fcf6072b740 1 -- 172.21.15.152:0/1546794678 wait complete. 2024-09-11T23:01:47.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf6072b740 1 -- start start 2024-09-11T23:01:47.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf6072b740 1 Processor -- start 2024-09-11T23:01:47.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf6072b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a055d0 0x563d88a0fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf6072b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 0x563d88a0cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf6072b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 0x563d88a0d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf6072b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x563d88a13070 con 0x563d88a10370 2024-09-11T23:01:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf6072b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x563d88926060 con 0x563d88a06b40 2024-09-11T23:01:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf6072b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x563d88a131f0 con 0x563d88a055d0 2024-09-11T23:01:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf6072b740 10 client.0 _ll_register_callbacks cb 0x563d886798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf5c8ef640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 0x563d88a0d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf5c8ef640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 0x563d88a0d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39444/0 (socket says 172.21.15.152:39444) 2024-09-11T23:01:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf5c8ef640 1 -- 172.21.15.152:0/1611719589 learned_addr learned my addr 172.21.15.152:0/1611719589 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf57fff640 1 --2- 172.21.15.152:0/1611719589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a055d0 0x563d88a0fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf5c8ef640 1 -- 172.21.15.152:0/1611719589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a055d0 msgr2=0x563d88a0fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf5c8ef640 1 --2- 172.21.15.152:0/1611719589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a055d0 0x563d88a0fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf5c8ef640 1 -- 172.21.15.152:0/1611719589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 msgr2=0x563d88a0cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:47.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.738+0000 7fcf577fe640 1 --2- 172.21.15.152:0/1611719589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 0x563d88a0cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.739+0000 7fcf5c8ef640 1 --2- 172.21.15.152:0/1611719589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 0x563d88a0cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.739+0000 7fcf5c8ef640 1 -- 172.21.15.152:0/1611719589 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d886dc3c0 con 0x563d88a10370 2024-09-11T23:01:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.739+0000 7fcf5c8ef640 1 --2- 172.21.15.152:0/1611719589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 0x563d88a0d4f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009ca0 tx=0x7fcf500095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459439fdf6870b64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.739+0000 7fcf377fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.739+0000 7fcf377fe640 1 -- 172.21.15.152:0/1611719589 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf500309d0 con 0x563d88a10370 2024-09-11T23:01:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.739+0000 7fcf377fe640 1 -- 172.21.15.152:0/1611719589 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7fcf50030b70 con 0x563d88a10370 2024-09-11T23:01:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.739+0000 7fcf6072b740 1 -- 172.21.15.152:0/1611719589 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf48017ca0 con 0x563d88a10370 2024-09-11T23:01:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.739+0000 7fcf377fe640 1 -- 172.21.15.152:0/1611719589 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcf50030d70 con 0x563d88a10370 2024-09-11T23:01:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.739+0000 7fcf6072b740 10 client.17874 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.739+0000 7fcf6072b740 1 -- 172.21.15.152:0/1611719589 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563d88a1ea80 con 0x563d88a10370 2024-09-11T23:01:47.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.739+0000 7fcf377fe640 1 -- 172.21.15.152:0/1611719589 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf48017ca0 con 0x563d88a10370 2024-09-11T23:01:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.739+0000 7fcf377fe640 1 -- 172.21.15.152:0/1611719589 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcf50048040 con 0x563d88a10370 2024-09-11T23:01:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.739+0000 7fcf6072b740 10 client.17874 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.739+0000 7fcf6072b740 10 client.17874 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T23:01:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.739+0000 7fcf6072b740 20 client.17874 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.740+0000 7fcf6072b740 1 --2- 172.21.15.152:0/1611719589 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x563d88a1fe90 0x563d88a22330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.740+0000 7fcf6072b740 4 client.17874 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:01:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.740+0000 7fcf6072b740 1 -- 172.21.15.152:0/1611719589 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d88978860 con 0x563d88a1fe90 2024-09-11T23:01:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.740+0000 7fcf57fff640 1 --2- 172.21.15.152:0/1611719589 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x563d88a1fe90 0x563d88a22330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.740+0000 7fcf377fe640 10 client.17874 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T23:01:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.740+0000 7fcf57fff640 1 --2- 172.21.15.152:0/1611719589 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x563d88a1fe90 0x563d88a22330 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1086538e8396654b server_cookie=81a45d6995524d85 in_seq=0 out_seq=0 2024-09-11T23:01:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.740+0000 7fcf377fe640 1 -- 172.21.15.152:0/1611719589 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d88978860 con 0x563d88a1fe90 2024-09-11T23:01:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.740+0000 7fcf377fe640 10 client.17874 handle_command_reply: tid=0 2024-09-11T23:01:47.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.741+0000 7fcf6072b740 1 client.17874 shutdown 2024-09-11T23:01:47.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.741+0000 7fcf6072b740 1 -- 172.21.15.152:0/1611719589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 msgr2=0x563d88a0d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:47.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.741+0000 7fcf6072b740 1 --2- 172.21.15.152:0/1611719589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 0x563d88a0d4f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009ca0 tx=0x7fcf500095c0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.741+0000 7fcf6072b740 1 -- 172.21.15.152:0/1611719589 shutdown_connections 2024-09-11T23:01:47.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.741+0000 7fcf6072b740 1 --2- 172.21.15.152:0/1611719589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 0x563d88a0d4f0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.741+0000 7fcf6072b740 1 --2- 172.21.15.152:0/1611719589 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x563d88a1fe90 0x563d88a22330 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.741+0000 7fcf6072b740 1 --2- 172.21.15.152:0/1611719589 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 0x563d88a0cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.741+0000 7fcf377fe640 0 client.17874 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T23:01:47.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.741+0000 7fcf6072b740 1 --2- 172.21.15.152:0/1611719589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a055d0 0x563d88a0fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.741+0000 7fcf6072b740 1 -- 172.21.15.152:0/1611719589 >> 172.21.15.152:0/1611719589 conn(0x563d889ce310 msgr2=0x563d88a07960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:47.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.741+0000 7fcf6072b740 1 -- 172.21.15.152:0/1611719589 shutdown_connections 2024-09-11T23:01:47.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.741+0000 7fcf6072b740 1 -- 172.21.15.152:0/1611719589 wait complete. 2024-09-11T23:01:47.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.741+0000 7fcf6072b740 20 client.17874 trim_cache size 0 max 16384 2024-09-11T23:01:47.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf6072b740 1 Processor -- start 2024-09-11T23:01:47.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf6072b740 1 -- start start 2024-09-11T23:01:47.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf6072b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a055d0 0x563d88a0c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf6072b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 0x563d88a0ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf6072b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 0x563d88a08120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf6072b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581ad830 con 0x563d88a10370 2024-09-11T23:01:47.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf6072b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581af370 con 0x563d88a06b40 2024-09-11T23:01:47.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf6072b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581adf00 con 0x563d88a055d0 2024-09-11T23:01:47.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf5c8ef640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 0x563d88a08120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf5c8ef640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 0x563d88a08120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:39454/0 (socket says 172.21.15.152:39454) 2024-09-11T23:01:47.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf5c8ef640 1 -- 172.21.15.152:0/3390417577 learned_addr learned my addr 172.21.15.152:0/3390417577 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:47.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf57fff640 1 --2- 172.21.15.152:0/3390417577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a055d0 0x563d88a0c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf577fe640 1 --2- 172.21.15.152:0/3390417577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 0x563d88a0ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf5c8ef640 1 -- 172.21.15.152:0/3390417577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a055d0 msgr2=0x563d88a0c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:47.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf5c8ef640 1 --2- 172.21.15.152:0/3390417577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a055d0 0x563d88a0c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf5c8ef640 1 -- 172.21.15.152:0/3390417577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 msgr2=0x563d88a0ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:47.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf5c8ef640 1 --2- 172.21.15.152:0/3390417577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 0x563d88a0ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.754+0000 7fcf5c8ef640 1 -- 172.21.15.152:0/3390417577 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d88a0bf40 con 0x563d88a10370 2024-09-11T23:01:47.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.755+0000 7fcf5c8ef640 1 --2- 172.21.15.152:0/3390417577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 0x563d88a08120 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcf500098d0 tx=0x7fcf50005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d768fa34ff612 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:47.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.755+0000 7fcf3dffb640 1 -- 172.21.15.152:0/3390417577 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50060070 con 0x563d88a10370 2024-09-11T23:01:47.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.755+0000 7fcf3dffb640 1 -- 172.21.15.152:0/3390417577 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf50036070 con 0x563d88a10370 2024-09-11T23:01:47.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.755+0000 7fcf3dffb640 1 -- 172.21.15.152:0/3390417577 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5004a440 con 0x563d88a10370 2024-09-11T23:01:47.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.755+0000 7fcf6072b740 1 -- 172.21.15.152:0/3390417577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 msgr2=0x563d88a08120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:47.767 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.755+0000 7fcf6072b740 1 --2- 172.21.15.152:0/3390417577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 0x563d88a08120 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcf500098d0 tx=0x7fcf50005f50 comp rx=0 tx=0).stop 2024-09-11T23:01:47.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.755+0000 7fcf6072b740 1 -- 172.21.15.152:0/3390417577 shutdown_connections 2024-09-11T23:01:47.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.755+0000 7fcf6072b740 1 --2- 172.21.15.152:0/3390417577 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 0x563d88a08120 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.755+0000 7fcf6072b740 1 --2- 172.21.15.152:0/3390417577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 0x563d88a0ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.755+0000 7fcf6072b740 1 --2- 172.21.15.152:0/3390417577 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a055d0 0x563d88a0c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.755+0000 7fcf6072b740 1 -- 172.21.15.152:0/3390417577 >> 172.21.15.152:0/3390417577 conn(0x563d889ce310 msgr2=0x563d88a0e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:47.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.756+0000 7fcf6072b740 1 -- 172.21.15.152:0/3390417577 shutdown_connections 2024-09-11T23:01:47.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.756+0000 7fcf6072b740 1 -- 172.21.15.152:0/3390417577 wait complete. 2024-09-11T23:01:47.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.756+0000 7fcf6072b740 1 -- start start 2024-09-11T23:01:47.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.757+0000 7fcf6072b740 1 Processor -- start 2024-09-11T23:01:47.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.757+0000 7fcf6072b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a055d0 0x563d88a09fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.757+0000 7fcf6072b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 0x563d88a0a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.757+0000 7fcf6072b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 0x563d88a0aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.757+0000 7fcf6072b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581aeca0 con 0x563d88a10370 2024-09-11T23:01:47.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.757+0000 7fcf6072b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581ae5d0 con 0x563d88a06b40 2024-09-11T23:01:47.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.757+0000 7fcf6072b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581ab830 con 0x563d88a055d0 2024-09-11T23:01:47.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.757+0000 7fcf6072b740 10 client.0 _ll_register_callbacks cb 0x7fcf50031ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:47.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.757+0000 7fcf577fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 0x563d88a0a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.757+0000 7fcf577fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 0x563d88a0a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:48956/0 (socket says 172.21.15.152:48956) 2024-09-11T23:01:47.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.757+0000 7fcf577fe640 1 -- 172.21.15.152:0/1240225522 learned_addr learned my addr 172.21.15.152:0/1240225522 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:47.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.757+0000 7fcf5c8ef640 1 --2- 172.21.15.152:0/1240225522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 0x563d88a0aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.757+0000 7fcf57fff640 1 --2- 172.21.15.152:0/1240225522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a055d0 0x563d88a09fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.758+0000 7fcf577fe640 1 -- 172.21.15.152:0/1240225522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a055d0 msgr2=0x563d88a09fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:47.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.758+0000 7fcf577fe640 1 --2- 172.21.15.152:0/1240225522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a055d0 0x563d88a09fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.758+0000 7fcf577fe640 1 -- 172.21.15.152:0/1240225522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 msgr2=0x563d88a0aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:47.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.758+0000 7fcf577fe640 1 --2- 172.21.15.152:0/1240225522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 0x563d88a0aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.758+0000 7fcf577fe640 1 -- 172.21.15.152:0/1240225522 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d88a8e030 con 0x563d88a06b40 2024-09-11T23:01:47.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.758+0000 7fcf577fe640 1 --2- 172.21.15.152:0/1240225522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 0x563d88a0a510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009420 tx=0x7fcf4c00c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5410b894db4af6e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:47.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.758+0000 7fcf377fe640 10 client.0 ms_handle_connect on v2:172.21.15.191:3300/0 2024-09-11T23:01:47.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.758+0000 7fcf377fe640 1 -- 172.21.15.152:0/1240225522 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c01e040 con 0x563d88a06b40 2024-09-11T23:01:47.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.758+0000 7fcf377fe640 1 -- 172.21.15.152:0/1240225522 <== mon.2 v2:172.21.15.191:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7fcf4c0cc0e0 con 0x563d88a06b40 2024-09-11T23:01:47.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.758+0000 7fcf6072b740 1 -- 172.21.15.152:0/1240225522 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf50036070 con 0x563d88a06b40 2024-09-11T23:01:47.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.758+0000 7fcf377fe640 1 -- 172.21.15.152:0/1240225522 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fcf4c01c120 con 0x563d88a06b40 2024-09-11T23:01:47.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.758+0000 7fcf6072b740 10 client.26720 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:47.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.758+0000 7fcf6072b740 1 -- 172.21.15.152:0/1240225522 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563d88a8f3a0 con 0x563d88a06b40 2024-09-11T23:01:47.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.759+0000 7fcf377fe640 1 -- 172.21.15.152:0/1240225522 <== mon.2 v2:172.21.15.191:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf4c09ad40 con 0x563d88a06b40 2024-09-11T23:01:47.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.759+0000 7fcf377fe640 1 -- 172.21.15.152:0/1240225522 <== mon.2 v2:172.21.15.191:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fcf4c01ecf0 con 0x563d88a06b40 2024-09-11T23:01:47.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.759+0000 7fcf6072b740 10 client.26720 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:47.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.759+0000 7fcf6072b740 10 client.26720 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T23:01:47.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.759+0000 7fcf6072b740 20 client.26720 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:47.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.759+0000 7fcf6072b740 1 --2- 172.21.15.152:0/1240225522 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x563d88a1fe90 0x563d88a91b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:47.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.759+0000 7fcf6072b740 4 client.26720 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:01:47.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.759+0000 7fcf6072b740 1 -- 172.21.15.152:0/1240225522 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fcf50036070 con 0x563d88a1fe90 2024-09-11T23:01:47.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.759+0000 7fcf57fff640 1 --2- 172.21.15.152:0/1240225522 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x563d88a1fe90 0x563d88a91b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:47.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.760+0000 7fcf377fe640 10 client.26720 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T23:01:47.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.760+0000 7fcf57fff640 1 --2- 172.21.15.152:0/1240225522 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x563d88a1fe90 0x563d88a91b60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e9115282bc3e626d server_cookie=1df8bb84f1dba3d7 in_seq=0 out_seq=0 2024-09-11T23:01:47.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.760+0000 7fcf377fe640 1 -- 172.21.15.152:0/1240225522 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fcf50036070 con 0x563d88a1fe90 2024-09-11T23:01:47.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.760+0000 7fcf377fe640 10 client.26720 handle_command_reply: tid=0 2024-09-11T23:01:47.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.760+0000 7fcf6072b740 1 client.26720 shutdown 2024-09-11T23:01:47.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.760+0000 7fcf6072b740 1 -- 172.21.15.152:0/1240225522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 msgr2=0x563d88a0a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:47.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.760+0000 7fcf6072b740 1 --2- 172.21.15.152:0/1240225522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 0x563d88a0a510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009420 tx=0x7fcf4c00c9e0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.760+0000 7fcf6072b740 1 -- 172.21.15.152:0/1240225522 shutdown_connections 2024-09-11T23:01:47.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.760+0000 7fcf6072b740 1 --2- 172.21.15.152:0/1240225522 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d88a10370 0x563d88a0aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.760+0000 7fcf6072b740 1 --2- 172.21.15.152:0/1240225522 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x563d88a1fe90 0x563d88a91b60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.760+0000 7fcf6072b740 1 --2- 172.21.15.152:0/1240225522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563d88a06b40 0x563d88a0a510 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.760+0000 7fcf6072b740 1 --2- 172.21.15.152:0/1240225522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563d88a055d0 0x563d88a09fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.772 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.760+0000 7fcf6072b740 1 -- 172.21.15.152:0/1240225522 >> 172.21.15.152:0/1240225522 conn(0x563d889ce310 msgr2=0x563d88a0f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:47.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.760+0000 7fcf377fe640 0 client.26720 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T23:01:47.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.760+0000 7fcf6072b740 1 -- 172.21.15.152:0/1240225522 shutdown_connections 2024-09-11T23:01:47.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.760+0000 7fcf6072b740 1 -- 172.21.15.152:0/1240225522 wait complete. 2024-09-11T23:01:47.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.761+0000 7fcf6072b740 20 client.26720 trim_cache size 0 max 16384 2024-09-11T23:01:47.774 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:01:47.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.761+0000 7fcf5e352640 1 -- 172.21.15.152:0/2900916495 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcf400a2cf0 msgr2=0x7fcf400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:47.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.761+0000 7fcf5e352640 1 --2- 172.21.15.152:0/2900916495 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcf400a2cf0 0x7fcf400a51b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcf50005fe0 tx=0x7fcf50009380 comp rx=0 tx=0).stop 2024-09-11T23:01:47.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.761+0000 7fcf5e352640 1 -- 172.21.15.152:0/2900916495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf58102790 msgr2=0x7fcf581989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:47.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.761+0000 7fcf5e352640 1 --2- 172.21.15.152:0/2900916495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf58102790 0x7fcf581989d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00b4a0 tx=0x7fcf4c00b960 comp rx=0 tx=0).stop 2024-09-11T23:01:47.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.761+0000 7fcf5e352640 1 -- 172.21.15.152:0/2900916495 shutdown_connections 2024-09-11T23:01:47.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.761+0000 7fcf5e352640 1 --2- 172.21.15.152:0/2900916495 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcf400a2cf0 0x7fcf400a51b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.761+0000 7fcf5e352640 1 --2- 172.21.15.152:0/2900916495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf58106f10 0x7fcf5819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.761+0000 7fcf5e352640 1 --2- 172.21.15.152:0/2900916495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf58102790 0x7fcf581989d0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.761+0000 7fcf5e352640 1 --2- 172.21.15.152:0/2900916495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf58101de0 0x7fcf58198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:47.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.761+0000 7fcf5e352640 1 -- 172.21.15.152:0/2900916495 >> 172.21.15.152:0/2900916495 conn(0x7fcf580fd8c0 msgr2=0x7fcf580fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:47.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.761+0000 7fcf5e352640 1 -- 172.21.15.152:0/2900916495 shutdown_connections 2024-09-11T23:01:47.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:47.761+0000 7fcf5e352640 1 -- 172.21.15.152:0/2900916495 wait complete. 2024-09-11T23:01:48.491 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:01:48.738 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:01:49.016 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:48 smithi152 ceph-mon[31181]: pgmap v274: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:49.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:48 smithi191 ceph-mon[35211]: pgmap v274: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:49.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:48 smithi204 ceph-mon[35454]: pgmap v274: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:49.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.332+0000 7f0c58bad640 1 -- 172.21.15.152:0/2449428927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c54101de0 msgr2=0x7f0c541021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:49.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.332+0000 7f0c58bad640 1 --2- 172.21.15.152:0/2449428927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c54101de0 0x7f0c541021c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009900 tx=0x7f0c4402a410 comp rx=0 tx=0).stop 2024-09-11T23:01:49.411 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.332+0000 7f0c58bad640 1 -- 172.21.15.152:0/2449428927 shutdown_connections 2024-09-11T23:01:49.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.332+0000 7f0c58bad640 1 --2- 172.21.15.152:0/2449428927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c54106f10 0x7f0c5410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.332+0000 7f0c58bad640 1 --2- 172.21.15.152:0/2449428927 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c54102790 0x7f0c541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.332+0000 7f0c58bad640 1 --2- 172.21.15.152:0/2449428927 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c54101de0 0x7f0c541021c0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.332+0000 7f0c58bad640 1 -- 172.21.15.152:0/2449428927 >> 172.21.15.152:0/2449428927 conn(0x7f0c540fd8c0 msgr2=0x7f0c540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:49.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.332+0000 7f0c58bad640 1 -- 172.21.15.152:0/2449428927 shutdown_connections 2024-09-11T23:01:49.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.332+0000 7f0c58bad640 1 -- 172.21.15.152:0/2449428927 wait complete. 2024-09-11T23:01:49.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c58bad640 1 Processor -- start 2024-09-11T23:01:49.412 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c58bad640 1 -- start start 2024-09-11T23:01:49.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c58bad640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c54101de0 0x7f0c54198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c58bad640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c54102790 0x7f0c54198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c58bad640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c54106f10 0x7f0c5419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c58bad640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0c54111f70 con 0x7f0c54102790 2024-09-11T23:01:49.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c58bad640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0c54111df0 con 0x7f0c54101de0 2024-09-11T23:01:49.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c58bad640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c541120f0 con 0x7f0c54106f10 2024-09-11T23:01:49.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c54102790 0x7f0c54198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:49.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c54102790 0x7f0c54198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36884/0 (socket says 172.21.15.152:36884) 2024-09-11T23:01:49.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c4bfff640 1 -- 172.21.15.152:0/623698553 learned_addr learned my addr 172.21.15.152:0/623698553 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:49.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c5259c640 1 --2- 172.21.15.152:0/623698553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c54101de0 0x7f0c54198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:49.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c52d9d640 1 --2- 172.21.15.152:0/623698553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c54106f10 0x7f0c5419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:49.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c4bfff640 1 -- 172.21.15.152:0/623698553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c54106f10 msgr2=0x7f0c5419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:49.413 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c4bfff640 1 --2- 172.21.15.152:0/623698553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c54106f10 0x7f0c5419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c4bfff640 1 -- 172.21.15.152:0/623698553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c54101de0 msgr2=0x7f0c54198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:49.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c4bfff640 1 --2- 172.21.15.152:0/623698553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c54101de0 0x7f0c54198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.333+0000 7f0c4bfff640 1 -- 172.21.15.152:0/623698553 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5419d3c0 con 0x7f0c54102790 2024-09-11T23:01:49.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.334+0000 7f0c4bfff640 1 --2- 172.21.15.152:0/623698553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c54102790 0x7f0c54198950 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00d950 tx=0x7f0c3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:49.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.334+0000 7f0c4b7fe640 1 -- 172.21.15.152:0/623698553 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c017070 con 0x7f0c54102790 2024-09-11T23:01:49.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.334+0000 7f0c58bad640 1 -- 172.21.15.152:0/623698553 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5419d6b0 con 0x7f0c54102790 2024-09-11T23:01:49.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.334+0000 7f0c4b7fe640 1 -- 172.21.15.152:0/623698553 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c3c009c30 con 0x7f0c54102790 2024-09-11T23:01:49.414 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.334+0000 7f0c4b7fe640 1 -- 172.21.15.152:0/623698553 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c00f6b0 con 0x7f0c54102790 2024-09-11T23:01:49.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.334+0000 7f0c58bad640 1 -- 172.21.15.152:0/623698553 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c541a4f90 con 0x7f0c54102790 2024-09-11T23:01:49.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.336+0000 7f0c4b7fe640 1 -- 172.21.15.152:0/623698553 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0c3c0109d0 con 0x7f0c54102790 2024-09-11T23:01:49.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.336+0000 7f0c5af86740 1 Processor -- start 2024-09-11T23:01:49.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c5af86740 1 -- start start 2024-09-11T23:01:49.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c5af86740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c5af86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783a65d0 0x560e783a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c5af86740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783b1370 0x560e783b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.415 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c5af86740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560e7830c2f0 con 0x560e783a7b40 2024-09-11T23:01:49.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c5af86740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560e782b3cf0 con 0x560e783b1370 2024-09-11T23:01:49.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c5af86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560e783082b0 con 0x560e783a65d0 2024-09-11T23:01:49.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c52d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:49.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c4b7fe640 1 --2- 172.21.15.152:0/623698553 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c400a2cf0 0x7f0c400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c52d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783a6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36888/0 (socket says 172.21.15.152:36888) 2024-09-11T23:01:49.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c52d9d640 1 -- 172.21.15.152:0/503223017 learned_addr learned my addr 172.21.15.152:0/503223017 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:49.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c4bfff640 1 --2- 172.21.15.152:0/503223017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783b1370 0x560e783b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:49.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c5259c640 1 --2- 172.21.15.152:0/503223017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783a65d0 0x560e783a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:49.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c52d9d640 1 --2- 172.21.15.152:0/623698553 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c400a2cf0 0x7f0c400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:49.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c4b7fe640 1 -- 172.21.15.152:0/623698553 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f0c3c0cf150 con 0x7f0c54102790 2024-09-11T23:01:49.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c52d9d640 1 -- 172.21.15.152:0/503223017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783a65d0 msgr2=0x560e783a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:49.416 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c52d9d640 1 --2- 172.21.15.152:0/503223017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783a65d0 0x560e783a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c52d9d640 1 -- 172.21.15.152:0/503223017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783b1370 msgr2=0x560e783b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:49.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c52d9d640 1 --2- 172.21.15.152:0/503223017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783b1370 0x560e783b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c52d9d640 1 -- 172.21.15.152:0/503223017 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e782983a0 con 0x560e783a7b40 2024-09-11T23:01:49.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.337+0000 7f0c4bfff640 1 --2- 172.21.15.152:0/503223017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783b1370 0x560e783b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:01:49.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.338+0000 7f0c52d9d640 1 --2- 172.21.15.152:0/503223017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783a6090 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c013820 tx=0x7f0c4c013ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace350e0cc1b00ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:49.417 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.338+0000 7f0c48ff9640 1 -- 172.21.15.152:0/503223017 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c003a30 con 0x560e783a7b40 2024-09-11T23:01:49.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.338+0000 7f0c48ff9640 1 -- 172.21.15.152:0/503223017 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c4c009da0 con 0x560e783a7b40 2024-09-11T23:01:49.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.338+0000 7f0c48ff9640 1 -- 172.21.15.152:0/503223017 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c0123f0 con 0x560e783a7b40 2024-09-11T23:01:49.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.338+0000 7f0c52d9d640 1 --2- 172.21.15.152:0/623698553 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c400a2cf0 0x7f0c400a51b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c020000 tx=0x7f0c4c01f000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:49.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 -- 172.21.15.152:0/503223017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 msgr2=0x560e783a6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:49.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 --2- 172.21.15.152:0/503223017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783a6090 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c013820 tx=0x7f0c4c013ce0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.418 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 -- 172.21.15.152:0/503223017 shutdown_connections 2024-09-11T23:01:49.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 --2- 172.21.15.152:0/503223017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783b1370 0x560e783b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 --2- 172.21.15.152:0/503223017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783a65d0 0x560e783a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 --2- 172.21.15.152:0/503223017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783a6090 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 -- 172.21.15.152:0/503223017 >> 172.21.15.152:0/503223017 conn(0x560e7836f310 msgr2=0x560e783a2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:49.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 -- 172.21.15.152:0/503223017 shutdown_connections 2024-09-11T23:01:49.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 -- 172.21.15.152:0/503223017 wait complete. 2024-09-11T23:01:49.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 -- start start 2024-09-11T23:01:49.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 Processor -- start 2024-09-11T23:01:49.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783a65d0 0x560e783b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783ae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783b1370 0x560e783ae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x560e783b4070 con 0x560e783a7b40 2024-09-11T23:01:49.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x560e782c7060 con 0x560e783a65d0 2024-09-11T23:01:49.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x560e783b41f0 con 0x560e783b1370 2024-09-11T23:01:49.421 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.339+0000 7f0c5af86740 10 client.0 _ll_register_callbacks cb 0x560e7801a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:49.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:49.424 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36894/0 (socket says 172.21.15.152:36894) 2024-09-11T23:01:49.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c4bfff640 1 -- 172.21.15.152:0/302014290 learned_addr learned my addr 172.21.15.152:0/302014290 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:49.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c5259c640 1 --2- 172.21.15.152:0/302014290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783a65d0 0x560e783b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:49.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c4bfff640 1 -- 172.21.15.152:0/302014290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783b1370 msgr2=0x560e783ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:49.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c4bfff640 1 --2- 172.21.15.152:0/302014290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783b1370 0x560e783ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c4bfff640 1 -- 172.21.15.152:0/302014290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783a65d0 msgr2=0x560e783b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:49.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c4bfff640 1 --2- 172.21.15.152:0/302014290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783a65d0 0x560e783b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c4bfff640 1 -- 172.21.15.152:0/302014290 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e7807d3c0 con 0x560e783a7b40 2024-09-11T23:01:49.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c4bfff640 1 --2- 172.21.15.152:0/302014290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783ae030 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c097af0 tx=0x7f0c3c01e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d652947bb5ddcc2d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:49.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:49.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c39ffb640 1 -- 172.21.15.152:0/302014290 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c020030 con 0x560e783a7b40 2024-09-11T23:01:49.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c39ffb640 1 -- 172.21.15.152:0/302014290 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f0c3c06a1b0 con 0x560e783a7b40 2024-09-11T23:01:49.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c5af86740 1 -- 172.21.15.152:0/302014290 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c4c009da0 con 0x560e783a7b40 2024-09-11T23:01:49.425 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c39ffb640 1 -- 172.21.15.152:0/302014290 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c3c066720 con 0x560e783a7b40 2024-09-11T23:01:49.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c5af86740 10 client.17892 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:49.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c5af86740 1 -- 172.21.15.152:0/302014290 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560e783bfab0 con 0x560e783a7b40 2024-09-11T23:01:49.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.340+0000 7f0c39ffb640 1 -- 172.21.15.152:0/302014290 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c4c009da0 con 0x560e783a7b40 2024-09-11T23:01:49.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.341+0000 7f0c39ffb640 1 -- 172.21.15.152:0/302014290 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0c3c084bd0 con 0x560e783a7b40 2024-09-11T23:01:49.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.341+0000 7f0c5af86740 10 client.17892 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:49.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.341+0000 7f0c5af86740 10 client.17892 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:01:49.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.341+0000 7f0c5af86740 20 client.17892 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:49.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.341+0000 7f0c5af86740 1 --2- 172.21.15.152:0/302014290 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x560e783c0e50 0x560e783c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.341+0000 7f0c5af86740 4 client.17892 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:01:49.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.341+0000 7f0c5af86740 1 -- 172.21.15.152:0/302014290 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e78319860 con 0x560e783c0e50 2024-09-11T23:01:49.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.341+0000 7f0c5259c640 1 --2- 172.21.15.152:0/302014290 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x560e783c0e50 0x560e783c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:49.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.341+0000 7f0c39ffb640 10 client.17892 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T23:01:49.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.341+0000 7f0c5259c640 1 --2- 172.21.15.152:0/302014290 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x560e783c0e50 0x560e783c32e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=241b2af756d778bb server_cookie=10b19d6f36e7a1dc in_seq=0 out_seq=0 2024-09-11T23:01:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.342+0000 7f0c39ffb640 1 -- 172.21.15.152:0/302014290 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e78319860 con 0x560e783c0e50 2024-09-11T23:01:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.342+0000 7f0c39ffb640 10 client.17892 handle_command_reply: tid=0 2024-09-11T23:01:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.342+0000 7f0c5af86740 1 client.17892 shutdown 2024-09-11T23:01:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.342+0000 7f0c5af86740 1 -- 172.21.15.152:0/302014290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 msgr2=0x560e783ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.342+0000 7f0c5af86740 1 --2- 172.21.15.152:0/302014290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783ae030 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c097af0 tx=0x7f0c3c01e740 comp rx=0 tx=0).stop 2024-09-11T23:01:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.342+0000 7f0c5af86740 1 -- 172.21.15.152:0/302014290 shutdown_connections 2024-09-11T23:01:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.342+0000 7f0c5af86740 1 --2- 172.21.15.152:0/302014290 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x560e783c0e50 0x560e783c32e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.342+0000 7f0c5af86740 1 --2- 172.21.15.152:0/302014290 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783b1370 0x560e783ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.342+0000 7f0c39ffb640 0 client.17892 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T23:01:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.342+0000 7f0c5af86740 1 --2- 172.21.15.152:0/302014290 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783ae030 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.342+0000 7f0c5af86740 1 --2- 172.21.15.152:0/302014290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783a65d0 0x560e783b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.427 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.342+0000 7f0c5af86740 1 -- 172.21.15.152:0/302014290 >> 172.21.15.152:0/302014290 conn(0x560e7836f310 msgr2=0x560e783a8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:49.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.342+0000 7f0c5af86740 1 -- 172.21.15.152:0/302014290 shutdown_connections 2024-09-11T23:01:49.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.342+0000 7f0c5af86740 1 -- 172.21.15.152:0/302014290 wait complete. 2024-09-11T23:01:49.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.342+0000 7f0c5af86740 20 client.17892 trim_cache size 0 max 16384 2024-09-11T23:01:49.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c5af86740 1 Processor -- start 2024-09-11T23:01:49.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c5af86740 1 -- start start 2024-09-11T23:01:49.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c5af86740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783a65d0 0x560e783a7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c5af86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783a7b40 0x560e783bd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c5af86740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783b1370 0x560e783a2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c5af86740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0c541ad7b0 con 0x560e783b1370 2024-09-11T23:01:49.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c5af86740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0c541af2f0 con 0x560e783a65d0 2024-09-11T23:01:49.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c5af86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c541ade80 con 0x560e783a7b40 2024-09-11T23:01:49.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c52d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783b1370 0x560e783a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:49.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c52d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783b1370 0x560e783a2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36896/0 (socket says 172.21.15.152:36896) 2024-09-11T23:01:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c52d9d640 1 -- 172.21.15.152:0/2415763607 learned_addr learned my addr 172.21.15.152:0/2415763607 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c5259c640 1 --2- 172.21.15.152:0/2415763607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783a65d0 0x560e783a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c4bfff640 1 --2- 172.21.15.152:0/2415763607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783a7b40 0x560e783bd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c52d9d640 1 -- 172.21.15.152:0/2415763607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783a7b40 msgr2=0x560e783bd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c52d9d640 1 --2- 172.21.15.152:0/2415763607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783a7b40 0x560e783bd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c52d9d640 1 -- 172.21.15.152:0/2415763607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783a65d0 msgr2=0x560e783a7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c52d9d640 1 --2- 172.21.15.152:0/2415763607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783a65d0 0x560e783a7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.356+0000 7f0c52d9d640 1 -- 172.21.15.152:0/2415763607 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e783a62d0 con 0x560e783b1370 2024-09-11T23:01:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.357+0000 7f0c52d9d640 1 --2- 172.21.15.152:0/2415763607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783b1370 0x560e783a2700 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c012010 tx=0x7f0c4c009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dafc35c25c84559 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.357+0000 7f0c3bfff640 1 -- 172.21.15.152:0/2415763607 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c0123f0 con 0x560e783b1370 2024-09-11T23:01:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.357+0000 7f0c3bfff640 1 -- 172.21.15.152:0/2415763607 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c4c0096a0 con 0x560e783b1370 2024-09-11T23:01:49.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.357+0000 7f0c3bfff640 1 -- 172.21.15.152:0/2415763607 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c005110 con 0x560e783b1370 2024-09-11T23:01:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.357+0000 7f0c5af86740 1 -- 172.21.15.152:0/2415763607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783b1370 msgr2=0x560e783a2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.357+0000 7f0c5af86740 1 --2- 172.21.15.152:0/2415763607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783b1370 0x560e783a2700 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c012010 tx=0x7f0c4c009510 comp rx=0 tx=0).stop 2024-09-11T23:01:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.357+0000 7f0c5af86740 1 -- 172.21.15.152:0/2415763607 shutdown_connections 2024-09-11T23:01:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.357+0000 7f0c5af86740 1 --2- 172.21.15.152:0/2415763607 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783b1370 0x560e783a2700 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.357+0000 7f0c5af86740 1 --2- 172.21.15.152:0/2415763607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783a7b40 0x560e783bd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.357+0000 7f0c5af86740 1 --2- 172.21.15.152:0/2415763607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783a65d0 0x560e783a7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.357+0000 7f0c5af86740 1 -- 172.21.15.152:0/2415763607 >> 172.21.15.152:0/2415763607 conn(0x560e7836f310 msgr2=0x560e7839da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.357+0000 7f0c5af86740 1 -- 172.21.15.152:0/2415763607 shutdown_connections 2024-09-11T23:01:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.357+0000 7f0c5af86740 1 -- 172.21.15.152:0/2415763607 wait complete. 2024-09-11T23:01:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c5af86740 1 -- start start 2024-09-11T23:01:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c5af86740 1 Processor -- start 2024-09-11T23:01:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c5af86740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783a65d0 0x560e783b9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c5af86740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c5af86740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783b1370 0x560e783b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c5af86740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0c541aec20 con 0x560e783a7b40 2024-09-11T23:01:49.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c5af86740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0c541ae550 con 0x560e783b1370 2024-09-11T23:01:49.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c5af86740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c541ab7b0 con 0x560e783a65d0 2024-09-11T23:01:49.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c5af86740 10 client.0 _ll_register_callbacks cb 0x7f0c3c097270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:49.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:49.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c4bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783b50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36902/0 (socket says 172.21.15.152:36902) 2024-09-11T23:01:49.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c4bfff640 1 -- 172.21.15.152:0/2368803275 learned_addr learned my addr 172.21.15.152:0/2368803275 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:49.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c4bfff640 1 -- 172.21.15.152:0/2368803275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783a65d0 msgr2=0x560e783b9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:49.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c4bfff640 1 --2- 172.21.15.152:0/2368803275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783a65d0 0x560e783b9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c4bfff640 1 -- 172.21.15.152:0/2368803275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783b1370 msgr2=0x560e783b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:49.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c4bfff640 1 --2- 172.21.15.152:0/2368803275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783b1370 0x560e783b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c4bfff640 1 -- 172.21.15.152:0/2368803275 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e783a9120 con 0x560e783a7b40 2024-09-11T23:01:49.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.358+0000 7f0c4bfff640 1 --2- 172.21.15.152:0/2368803275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783b50d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c09b390 tx=0x7f0c3c097fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed697691986a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:49.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.359+0000 7f0c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:49.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.359+0000 7f0c39ffb640 1 -- 172.21.15.152:0/2368803275 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c020070 con 0x560e783a7b40 2024-09-11T23:01:49.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.359+0000 7f0c39ffb640 1 -- 172.21.15.152:0/2368803275 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f0c3c0a3e70 con 0x560e783a7b40 2024-09-11T23:01:49.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.359+0000 7f0c5af86740 1 -- 172.21.15.152:0/2368803275 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c4c0096a0 con 0x560e783a7b40 2024-09-11T23:01:49.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.359+0000 7f0c39ffb640 1 -- 172.21.15.152:0/2368803275 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c3c09bd90 con 0x560e783a7b40 2024-09-11T23:01:49.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.359+0000 7f0c5af86740 10 client.17898 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:49.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.359+0000 7f0c5af86740 1 -- 172.21.15.152:0/2368803275 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560e783a3960 con 0x560e783a7b40 2024-09-11T23:01:49.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.359+0000 7f0c39ffb640 1 -- 172.21.15.152:0/2368803275 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c4c0096a0 con 0x560e783a7b40 2024-09-11T23:01:49.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.359+0000 7f0c39ffb640 1 -- 172.21.15.152:0/2368803275 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0c3c0afbb0 con 0x560e783a7b40 2024-09-11T23:01:49.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.359+0000 7f0c5af86740 10 client.17898 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:49.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.359+0000 7f0c5af86740 10 client.17898 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:01:49.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.359+0000 7f0c5af86740 20 client.17898 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:49.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.359+0000 7f0c5af86740 1 --2- 172.21.15.152:0/2368803275 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x560e7842bfb0 0x560e7842e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:49.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.359+0000 7f0c5af86740 4 client.17898 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:01:49.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.359+0000 7f0c5af86740 1 -- 172.21.15.152:0/2368803275 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e78313d70 con 0x560e7842bfb0 2024-09-11T23:01:49.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c5259c640 1 --2- 172.21.15.152:0/2368803275 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x560e7842bfb0 0x560e7842e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:49.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c39ffb640 10 client.17898 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T23:01:49.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c5259c640 1 --2- 172.21.15.152:0/2368803275 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x560e7842bfb0 0x560e7842e3d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6e763c07eee72bdd server_cookie=23a7081047692c7e in_seq=0 out_seq=0 2024-09-11T23:01:49.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c39ffb640 1 -- 172.21.15.152:0/2368803275 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e78313d70 con 0x560e7842bfb0 2024-09-11T23:01:49.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c39ffb640 10 client.17898 handle_command_reply: tid=0 2024-09-11T23:01:49.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c5af86740 1 client.17898 shutdown 2024-09-11T23:01:49.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c5af86740 1 -- 172.21.15.152:0/2368803275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 msgr2=0x560e783b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:49.433 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c5af86740 1 --2- 172.21.15.152:0/2368803275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783b50d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c09b390 tx=0x7f0c3c097fd0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.434 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:01:49.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c5af86740 1 -- 172.21.15.152:0/2368803275 shutdown_connections 2024-09-11T23:01:49.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c5af86740 1 --2- 172.21.15.152:0/2368803275 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x560e7842bfb0 0x560e7842e3d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c5af86740 1 --2- 172.21.15.152:0/2368803275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x560e783b1370 0x560e783b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c39ffb640 0 client.17898 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T23:01:49.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c5af86740 1 --2- 172.21.15.152:0/2368803275 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560e783a7b40 0x560e783b50d0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c5af86740 1 --2- 172.21.15.152:0/2368803275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x560e783a65d0 0x560e783b9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c5af86740 1 -- 172.21.15.152:0/2368803275 >> 172.21.15.152:0/2368803275 conn(0x560e7836f310 msgr2=0x560e783a71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:49.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c5af86740 1 -- 172.21.15.152:0/2368803275 shutdown_connections 2024-09-11T23:01:49.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.360+0000 7f0c5af86740 1 -- 172.21.15.152:0/2368803275 wait complete. 2024-09-11T23:01:49.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.361+0000 7f0c5af86740 20 client.17898 trim_cache size 0 max 16384 2024-09-11T23:01:49.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.361+0000 7f0c58bad640 1 -- 172.21.15.152:0/623698553 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c400a2cf0 msgr2=0x7f0c400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:49.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.361+0000 7f0c58bad640 1 --2- 172.21.15.152:0/623698553 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c400a2cf0 0x7f0c400a51b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c020000 tx=0x7f0c4c01f000 comp rx=0 tx=0).stop 2024-09-11T23:01:49.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.361+0000 7f0c58bad640 1 -- 172.21.15.152:0/623698553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c54102790 msgr2=0x7f0c54198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:49.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.361+0000 7f0c58bad640 1 --2- 172.21.15.152:0/623698553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c54102790 0x7f0c54198950 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00d950 tx=0x7f0c3c00de10 comp rx=0 tx=0).stop 2024-09-11T23:01:49.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.361+0000 7f0c58bad640 1 -- 172.21.15.152:0/623698553 shutdown_connections 2024-09-11T23:01:49.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.361+0000 7f0c58bad640 1 --2- 172.21.15.152:0/623698553 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c400a2cf0 0x7f0c400a51b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.435 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.361+0000 7f0c58bad640 1 --2- 172.21.15.152:0/623698553 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c54106f10 0x7f0c5419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.361+0000 7f0c58bad640 1 --2- 172.21.15.152:0/623698553 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c54102790 0x7f0c54198950 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.361+0000 7f0c58bad640 1 --2- 172.21.15.152:0/623698553 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c54101de0 0x7f0c54198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:49.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.361+0000 7f0c58bad640 1 -- 172.21.15.152:0/623698553 >> 172.21.15.152:0/623698553 conn(0x7f0c540fd8c0 msgr2=0x7f0c540fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:49.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.361+0000 7f0c58bad640 1 -- 172.21.15.152:0/623698553 shutdown_connections 2024-09-11T23:01:49.436 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:49.361+0000 7f0c58bad640 1 -- 172.21.15.152:0/623698553 wait complete. 2024-09-11T23:01:50.185 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:01:50.438 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:01:50.871 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:50 smithi152 ceph-mon[31181]: pgmap v275: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T23:01:50.971 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.950+0000 7f24b2c95640 1 -- 172.21.15.152:0/1003447852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24ac102790 msgr2=0x7f24ac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:50.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.950+0000 7f24b2c95640 1 --2- 172.21.15.152:0/1003447852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24ac102790 0x7f24ac1067e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2498009900 tx=0x7f249802a410 comp rx=0 tx=0).stop 2024-09-11T23:01:50.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.950+0000 7f24b2c95640 1 -- 172.21.15.152:0/1003447852 shutdown_connections 2024-09-11T23:01:50.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.950+0000 7f24b2c95640 1 --2- 172.21.15.152:0/1003447852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24ac106f10 0x7f24ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.950+0000 7f24b2c95640 1 --2- 172.21.15.152:0/1003447852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24ac102790 0x7f24ac1067e0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.950+0000 7f24b2c95640 1 --2- 172.21.15.152:0/1003447852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24ac101de0 0x7f24ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.950+0000 7f24b2c95640 1 -- 172.21.15.152:0/1003447852 >> 172.21.15.152:0/1003447852 conn(0x7f24ac0fd8c0 msgr2=0x7f24ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:50.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.950+0000 7f24b2c95640 1 -- 172.21.15.152:0/1003447852 shutdown_connections 2024-09-11T23:01:50.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.950+0000 7f24b2c95640 1 -- 172.21.15.152:0/1003447852 wait complete. 2024-09-11T23:01:50.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.951+0000 7f24b2c95640 1 Processor -- start 2024-09-11T23:01:50.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.951+0000 7f24b2c95640 1 -- start start 2024-09-11T23:01:50.972 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.951+0000 7f24b2c95640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24ac101de0 0x7f24ac198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.951+0000 7f24b2c95640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24ac102790 0x7f24ac1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.951+0000 7f24b2c95640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24ac106f10 0x7f24ac19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.951+0000 7f24b2c95640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac111f70 con 0x7f24ac102790 2024-09-11T23:01:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.951+0000 7f24b2c95640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac111df0 con 0x7f24ac106f10 2024-09-11T23:01:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.951+0000 7f24b2c95640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac1120f0 con 0x7f24ac101de0 2024-09-11T23:01:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.951+0000 7f24abfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24ac102790 0x7f24ac1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.951+0000 7f24abfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24ac102790 0x7f24ac1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36914/0 (socket says 172.21.15.152:36914) 2024-09-11T23:01:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.951+0000 7f24abfff640 1 -- 172.21.15.152:0/535921835 learned_addr learned my addr 172.21.15.152:0/535921835 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.951+0000 7f24b1232640 1 --2- 172.21.15.152:0/535921835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24ac106f10 0x7f24ac19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.952+0000 7f24b0a31640 1 --2- 172.21.15.152:0/535921835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24ac101de0 0x7f24ac198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.952+0000 7f24abfff640 1 -- 172.21.15.152:0/535921835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24ac101de0 msgr2=0x7f24ac198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.952+0000 7f24abfff640 1 --2- 172.21.15.152:0/535921835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24ac101de0 0x7f24ac198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.973 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.952+0000 7f24abfff640 1 -- 172.21.15.152:0/535921835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24ac106f10 msgr2=0x7f24ac19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.952+0000 7f24abfff640 1 --2- 172.21.15.152:0/535921835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24ac106f10 0x7f24ac19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.952+0000 7f24abfff640 1 -- 172.21.15.152:0/535921835 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24ac19d420 con 0x7f24ac102790 2024-09-11T23:01:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.952+0000 7f24abfff640 1 --2- 172.21.15.152:0/535921835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24ac102790 0x7f24ac1989b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2498006010 tx=0x7f24980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.952+0000 7f24a9ffb640 1 -- 172.21.15.152:0/535921835 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249803f030 con 0x7f24ac102790 2024-09-11T23:01:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.952+0000 7f24b2c95640 1 -- 172.21.15.152:0/535921835 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24ac19d6b0 con 0x7f24ac102790 2024-09-11T23:01:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.952+0000 7f24a9ffb640 1 -- 172.21.15.152:0/535921835 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2498003ce0 con 0x7f24ac102790 2024-09-11T23:01:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.952+0000 7f24a9ffb640 1 -- 172.21.15.152:0/535921835 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498035070 con 0x7f24ac102790 2024-09-11T23:01:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.952+0000 7f24b2c95640 1 -- 172.21.15.152:0/535921835 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24ac1a4ff0 con 0x7f24ac102790 2024-09-11T23:01:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.955+0000 7f24b506e740 1 Processor -- start 2024-09-11T23:01:50.974 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.955+0000 7f24b506e740 1 -- start start 2024-09-11T23:01:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.955+0000 7f24a9ffb640 1 -- 172.21.15.152:0/535921835 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2498042030 con 0x7f24ac102790 2024-09-11T23:01:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.955+0000 7f24b506e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559488177b40 0x559488176090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.955+0000 7f24b506e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594881765d0 0x559488176a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.955+0000 7f24b506e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488181370 0x559488183760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.955+0000 7f24b506e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5594880dc2f0 con 0x559488181370 2024-09-11T23:01:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.955+0000 7f24b506e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559488083cf0 con 0x5594881765d0 2024-09-11T23:01:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.955+0000 7f24b506e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5594880d82b0 con 0x559488177b40 2024-09-11T23:01:50.975 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.955+0000 7f24abfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488181370 0x559488183760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.955+0000 7f24abfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488181370 0x559488183760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36922/0 (socket says 172.21.15.152:36922) 2024-09-11T23:01:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.955+0000 7f24abfff640 1 -- 172.21.15.152:0/2485334483 learned_addr learned my addr 172.21.15.152:0/2485334483 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.955+0000 7f24b1232640 1 --2- 172.21.15.152:0/2485334483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559488177b40 0x559488176090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.955+0000 7f24b0a31640 1 --2- 172.21.15.152:0/2485334483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594881765d0 0x559488176a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.956+0000 7f24abfff640 1 -- 172.21.15.152:0/2485334483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559488177b40 msgr2=0x559488176090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.956+0000 7f24abfff640 1 --2- 172.21.15.152:0/2485334483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559488177b40 0x559488176090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.956+0000 7f24abfff640 1 -- 172.21.15.152:0/2485334483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594881765d0 msgr2=0x559488176a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.956+0000 7f24abfff640 1 --2- 172.21.15.152:0/2485334483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594881765d0 0x559488176a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.976 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.956+0000 7f24abfff640 1 -- 172.21.15.152:0/2485334483 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594880683a0 con 0x559488181370 2024-09-11T23:01:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.956+0000 7f24abfff640 1 --2- 172.21.15.152:0/2485334483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488181370 0x559488183760 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f24980ed140 tx=0x7f24980f3c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd35486935785e64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.956+0000 7f2496ffd640 1 -- 172.21.15.152:0/2485334483 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498103070 con 0x559488181370 2024-09-11T23:01:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.956+0000 7f2496ffd640 1 -- 172.21.15.152:0/2485334483 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24980faa60 con 0x559488181370 2024-09-11T23:01:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.956+0000 7f2496ffd640 1 -- 172.21.15.152:0/2485334483 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24980fc960 con 0x559488181370 2024-09-11T23:01:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.956+0000 7f24a9ffb640 1 --2- 172.21.15.152:0/535921835 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f249c0a2cf0 0x7f249c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.956+0000 7f24b1232640 1 --2- 172.21.15.152:0/535921835 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f249c0a2cf0 0x7f249c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.956+0000 7f24a9ffb640 1 -- 172.21.15.152:0/535921835 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f24980efa60 con 0x7f24ac102790 2024-09-11T23:01:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.957+0000 7f24b506e740 1 -- 172.21.15.152:0/2485334483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488181370 msgr2=0x559488183760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.957+0000 7f24b506e740 1 --2- 172.21.15.152:0/2485334483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488181370 0x559488183760 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f24980ed140 tx=0x7f24980f3c70 comp rx=0 tx=0).stop 2024-09-11T23:01:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.957+0000 7f24b1232640 1 --2- 172.21.15.152:0/535921835 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f249c0a2cf0 0x7f249c0a51b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f24a400ae90 tx=0x7f24a400a4d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:50.977 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.957+0000 7f24b506e740 1 -- 172.21.15.152:0/2485334483 shutdown_connections 2024-09-11T23:01:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.957+0000 7f24b506e740 1 --2- 172.21.15.152:0/2485334483 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488181370 0x559488183760 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.957+0000 7f24b506e740 1 --2- 172.21.15.152:0/2485334483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5594881765d0 0x559488176a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.957+0000 7f24b506e740 1 --2- 172.21.15.152:0/2485334483 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559488177b40 0x559488176090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.957+0000 7f24b506e740 1 -- 172.21.15.152:0/2485334483 >> 172.21.15.152:0/2485334483 conn(0x55948813f310 msgr2=0x559488172c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.957+0000 7f24b506e740 1 -- 172.21.15.152:0/2485334483 shutdown_connections 2024-09-11T23:01:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.957+0000 7f24b506e740 1 -- 172.21.15.152:0/2485334483 wait complete. 2024-09-11T23:01:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.957+0000 7f24b506e740 1 -- start start 2024-09-11T23:01:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.957+0000 7f24b506e740 1 Processor -- start 2024-09-11T23:01:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24b506e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594881765d0 0x559488180dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24b506e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488177b40 0x55948817e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24b506e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559488181370 0x55948817e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24b506e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5594880dc2f0 con 0x559488177b40 2024-09-11T23:01:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24b506e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559488097060 con 0x559488181370 2024-09-11T23:01:50.978 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24b506e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559488183e30 con 0x5594881765d0 2024-09-11T23:01:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24b506e740 10 client.0 _ll_register_callbacks cb 0x559487dea8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24abfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488177b40 0x55948817e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24abfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488177b40 0x55948817e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36932/0 (socket says 172.21.15.152:36932) 2024-09-11T23:01:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24abfff640 1 -- 172.21.15.152:0/1026460232 learned_addr learned my addr 172.21.15.152:0/1026460232 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24b0a31640 1 --2- 172.21.15.152:0/1026460232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594881765d0 0x559488180dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24b1232640 1 --2- 172.21.15.152:0/1026460232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559488181370 0x55948817e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24abfff640 1 -- 172.21.15.152:0/1026460232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594881765d0 msgr2=0x559488180dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24abfff640 1 --2- 172.21.15.152:0/1026460232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594881765d0 0x559488180dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24abfff640 1 -- 172.21.15.152:0/1026460232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559488181370 msgr2=0x55948817e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24abfff640 1 --2- 172.21.15.152:0/1026460232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559488181370 0x55948817e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24abfff640 1 -- 172.21.15.152:0/1026460232 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559487e4d3c0 con 0x559488177b40 2024-09-11T23:01:50.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24abfff640 1 --2- 172.21.15.152:0/1026460232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488177b40 0x55948817e080 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f24980ed7d0 tx=0x7f24980f37f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a552fc15411461a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:50.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f248bfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:50.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f248bfff640 1 -- 172.21.15.152:0/1026460232 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498049030 con 0x559488177b40 2024-09-11T23:01:50.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f248bfff640 1 -- 172.21.15.152:0/1026460232 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f24980ba360 con 0x559488177b40 2024-09-11T23:01:50.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.958+0000 7f24b506e740 1 -- 172.21.15.152:0/1026460232 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24980faa60 con 0x559488177b40 2024-09-11T23:01:50.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.959+0000 7f248bfff640 1 -- 172.21.15.152:0/1026460232 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f24980b8300 con 0x559488177b40 2024-09-11T23:01:50.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.959+0000 7f24b506e740 10 client.17910 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:50.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.959+0000 7f24b506e740 1 -- 172.21.15.152:0/1026460232 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55948818f750 con 0x559488177b40 2024-09-11T23:01:50.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.959+0000 7f248bfff640 1 -- 172.21.15.152:0/1026460232 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24980faa60 con 0x559488177b40 2024-09-11T23:01:50.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.959+0000 7f248bfff640 1 -- 172.21.15.152:0/1026460232 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f24980ed450 con 0x559488177b40 2024-09-11T23:01:50.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.959+0000 7f24b506e740 10 client.17910 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:50.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.959+0000 7f24b506e740 10 client.17910 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:01:50.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.959+0000 7f24b506e740 20 client.17910 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:50.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.959+0000 7f24b506e740 1 --2- 172.21.15.152:0/1026460232 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x559488190970 0x559488192d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.959+0000 7f24b506e740 4 client.17910 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:01:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.959+0000 7f24b506e740 1 -- 172.21.15.152:0/1026460232 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594880e9860 con 0x559488190970 2024-09-11T23:01:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.959+0000 7f24b0a31640 1 --2- 172.21.15.152:0/1026460232 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x559488190970 0x559488192d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.960+0000 7f248bfff640 10 client.17910 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T23:01:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.960+0000 7f24b0a31640 1 --2- 172.21.15.152:0/1026460232 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x559488190970 0x559488192d60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f84d7a256bea502c server_cookie=81515d708457e436 in_seq=0 out_seq=0 2024-09-11T23:01:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.960+0000 7f248bfff640 1 -- 172.21.15.152:0/1026460232 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594880e9860 con 0x559488190970 2024-09-11T23:01:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.960+0000 7f248bfff640 10 client.17910 handle_command_reply: tid=0 2024-09-11T23:01:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.960+0000 7f24b506e740 1 client.17910 shutdown 2024-09-11T23:01:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.961+0000 7f24b506e740 1 -- 172.21.15.152:0/1026460232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488177b40 msgr2=0x55948817e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.961+0000 7f24b506e740 1 --2- 172.21.15.152:0/1026460232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488177b40 0x55948817e080 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f24980ed7d0 tx=0x7f24980f37f0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.961+0000 7f24b506e740 1 -- 172.21.15.152:0/1026460232 shutdown_connections 2024-09-11T23:01:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.961+0000 7f24b506e740 1 --2- 172.21.15.152:0/1026460232 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x559488190970 0x559488192d60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.961+0000 7f24b506e740 1 --2- 172.21.15.152:0/1026460232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559488181370 0x55948817e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.961+0000 7f248bfff640 0 client.17910 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T23:01:50.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.961+0000 7f24b506e740 1 --2- 172.21.15.152:0/1026460232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488177b40 0x55948817e080 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.961+0000 7f24b506e740 1 --2- 172.21.15.152:0/1026460232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594881765d0 0x559488180dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.961+0000 7f24b506e740 1 -- 172.21.15.152:0/1026460232 >> 172.21.15.152:0/1026460232 conn(0x55948813f310 msgr2=0x559488178960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:50.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.961+0000 7f24b506e740 1 -- 172.21.15.152:0/1026460232 shutdown_connections 2024-09-11T23:01:50.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.961+0000 7f24b506e740 1 -- 172.21.15.152:0/1026460232 wait complete. 2024-09-11T23:01:50.982 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.961+0000 7f24b506e740 20 client.17910 trim_cache size 0 max 16384 2024-09-11T23:01:50.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b506e740 1 Processor -- start 2024-09-11T23:01:50.986 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b506e740 1 -- start start 2024-09-11T23:01:50.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b506e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5594881765d0 0x559488172d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b506e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559488177b40 0x55948817cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b506e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559488181370 0x5594881985f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b506e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac1ad810 con 0x5594881765d0 2024-09-11T23:01:50.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b506e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac1af350 con 0x559488181370 2024-09-11T23:01:50.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b506e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac1adee0 con 0x559488177b40 2024-09-11T23:01:50.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b0a31640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5594881765d0 0x559488172d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b0a31640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5594881765d0 0x559488172d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36946/0 (socket says 172.21.15.152:36946) 2024-09-11T23:01:50.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b0a31640 1 -- 172.21.15.152:0/3741505031 learned_addr learned my addr 172.21.15.152:0/3741505031 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:50.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b1232640 1 --2- 172.21.15.152:0/3741505031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559488181370 0x5594881985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b0a31640 1 -- 172.21.15.152:0/3741505031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559488177b40 msgr2=0x55948817cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:50.987 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b0a31640 1 --2- 172.21.15.152:0/3741505031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559488177b40 0x55948817cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b0a31640 1 -- 172.21.15.152:0/3741505031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559488181370 msgr2=0x5594881985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:50.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b0a31640 1 --2- 172.21.15.152:0/3741505031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559488181370 0x5594881985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b0a31640 1 -- 172.21.15.152:0/3741505031 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594881808f0 con 0x5594881765d0 2024-09-11T23:01:50.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.974+0000 7f24b0a31640 1 --2- 172.21.15.152:0/3741505031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5594881765d0 0x559488172d70 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f24a000bd20 tx=0x7f24a0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93215b7299aff4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:50.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.975+0000 7f24967fc640 1 -- 172.21.15.152:0/3741505031 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a0004230 con 0x5594881765d0 2024-09-11T23:01:50.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.975+0000 7f24967fc640 1 -- 172.21.15.152:0/3741505031 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a00043d0 con 0x5594881765d0 2024-09-11T23:01:50.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.975+0000 7f24967fc640 1 -- 172.21.15.152:0/3741505031 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a001d8b0 con 0x5594881765d0 2024-09-11T23:01:50.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.975+0000 7f24b506e740 1 -- 172.21.15.152:0/3741505031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5594881765d0 msgr2=0x559488172d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:50.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.975+0000 7f24b506e740 1 --2- 172.21.15.152:0/3741505031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5594881765d0 0x559488172d70 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f24a000bd20 tx=0x7f24a0011810 comp rx=0 tx=0).stop 2024-09-11T23:01:50.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.975+0000 7f24b506e740 1 -- 172.21.15.152:0/3741505031 shutdown_connections 2024-09-11T23:01:50.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.975+0000 7f24b506e740 1 --2- 172.21.15.152:0/3741505031 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559488181370 0x5594881985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.975+0000 7f24b506e740 1 --2- 172.21.15.152:0/3741505031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559488177b40 0x55948817cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.975+0000 7f24b506e740 1 --2- 172.21.15.152:0/3741505031 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5594881765d0 0x559488172d70 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.975+0000 7f24b506e740 1 -- 172.21.15.152:0/3741505031 >> 172.21.15.152:0/3741505031 conn(0x55948813f310 msgr2=0x55948817c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:50.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.975+0000 7f24b506e740 1 -- 172.21.15.152:0/3741505031 shutdown_connections 2024-09-11T23:01:50.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.975+0000 7f24b506e740 1 -- 172.21.15.152:0/3741505031 wait complete. 2024-09-11T23:01:50.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.976+0000 7f24b506e740 1 -- start start 2024-09-11T23:01:50.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.976+0000 7f24b506e740 1 Processor -- start 2024-09-11T23:01:50.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.976+0000 7f24b506e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594881765d0 0x559488189540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.976+0000 7f24b506e740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559488177b40 0x559488184870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.976+0000 7f24b506e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488181370 0x559488184db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.976+0000 7f24b506e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac1aec80 con 0x559488181370 2024-09-11T23:01:50.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.976+0000 7f24b506e740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac1ae5b0 con 0x559488177b40 2024-09-11T23:01:50.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.976+0000 7f24b506e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f24ac1ab810 con 0x5594881765d0 2024-09-11T23:01:50.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.976+0000 7f24b506e740 10 client.0 _ll_register_callbacks cb 0x7f2498101070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:50.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.976+0000 7f24b1232640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488181370 0x559488184db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.989 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.976+0000 7f24b1232640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488181370 0x559488184db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36960/0 (socket says 172.21.15.152:36960) 2024-09-11T23:01:50.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.976+0000 7f24b1232640 1 -- 172.21.15.152:0/282252883 learned_addr learned my addr 172.21.15.152:0/282252883 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:50.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.976+0000 7f24abfff640 1 --2- 172.21.15.152:0/282252883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559488177b40 0x559488184870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.976+0000 7f24b0a31640 1 --2- 172.21.15.152:0/282252883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594881765d0 0x559488189540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.977+0000 7f24b1232640 1 -- 172.21.15.152:0/282252883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594881765d0 msgr2=0x559488189540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:50.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.977+0000 7f24b1232640 1 --2- 172.21.15.152:0/282252883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594881765d0 0x559488189540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.977+0000 7f24b1232640 1 -- 172.21.15.152:0/282252883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559488177b40 msgr2=0x559488184870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:50.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.977+0000 7f24b1232640 1 --2- 172.21.15.152:0/282252883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559488177b40 0x559488184870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.977+0000 7f24b1232640 1 -- 172.21.15.152:0/282252883 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559488185320 con 0x559488181370 2024-09-11T23:01:50.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.977+0000 7f24b1232640 1 --2- 172.21.15.152:0/282252883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488181370 0x559488184db0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f24a401bbe0 tx=0x7f24a4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639ca69cf48c2ece server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:50.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.977+0000 7f248bfff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:50.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.977+0000 7f248bfff640 1 -- 172.21.15.152:0/282252883 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a402c060 con 0x559488181370 2024-09-11T23:01:50.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.977+0000 7f24b506e740 1 -- 172.21.15.152:0/282252883 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24a00043d0 con 0x559488181370 2024-09-11T23:01:50.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.977+0000 7f248bfff640 1 -- 172.21.15.152:0/282252883 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f24a400f920 con 0x559488181370 2024-09-11T23:01:50.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.977+0000 7f248bfff640 1 -- 172.21.15.152:0/282252883 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f24a400fb20 con 0x559488181370 2024-09-11T23:01:50.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.977+0000 7f24b506e740 10 client.17916 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:50.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.977+0000 7f24b506e740 1 -- 172.21.15.152:0/282252883 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5594881fb380 con 0x559488181370 2024-09-11T23:01:50.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.977+0000 7f248bfff640 1 -- 172.21.15.152:0/282252883 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24a00043d0 con 0x559488181370 2024-09-11T23:01:50.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.978+0000 7f248bfff640 1 -- 172.21.15.152:0/282252883 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f24a4026070 con 0x559488181370 2024-09-11T23:01:50.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.978+0000 7f24b506e740 10 client.17916 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:50.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.978+0000 7f24b506e740 10 client.17916 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:01:50.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.978+0000 7f24b506e740 20 client.17916 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:50.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.978+0000 7f24b506e740 1 --2- 172.21.15.152:0/282252883 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x5594881fbaa0 0x5594881fde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:50.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.978+0000 7f24b506e740 4 client.17916 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:01:50.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.978+0000 7f24b506e740 1 -- 172.21.15.152:0/282252883 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594880e3d70 con 0x5594881fbaa0 2024-09-11T23:01:50.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.978+0000 7f24b0a31640 1 --2- 172.21.15.152:0/282252883 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x5594881fbaa0 0x5594881fde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:50.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.978+0000 7f248bfff640 10 client.17916 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T23:01:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.978+0000 7f24b0a31640 1 --2- 172.21.15.152:0/282252883 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x5594881fbaa0 0x5594881fde90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48758f9e9a7af03d server_cookie=b864be5a5a8a1ffe in_seq=0 out_seq=0 2024-09-11T23:01:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f248bfff640 1 -- 172.21.15.152:0/282252883 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594880e3d70 con 0x5594881fbaa0 2024-09-11T23:01:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f248bfff640 10 client.17916 handle_command_reply: tid=0 2024-09-11T23:01:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f24b506e740 1 client.17916 shutdown 2024-09-11T23:01:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f24b506e740 1 -- 172.21.15.152:0/282252883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488181370 msgr2=0x559488184db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f24b506e740 1 --2- 172.21.15.152:0/282252883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488181370 0x559488184db0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f24a401bbe0 tx=0x7f24a4004ab0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f24b506e740 1 -- 172.21.15.152:0/282252883 shutdown_connections 2024-09-11T23:01:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f24b506e740 1 --2- 172.21.15.152:0/282252883 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x5594881fbaa0 0x5594881fde90 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f24b506e740 1 --2- 172.21.15.152:0/282252883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559488181370 0x559488184db0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f24b506e740 1 --2- 172.21.15.152:0/282252883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559488177b40 0x559488184870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f24b506e740 1 --2- 172.21.15.152:0/282252883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5594881765d0 0x559488189540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f24b506e740 1 -- 172.21.15.152:0/282252883 >> 172.21.15.152:0/282252883 conn(0x55948813f310 msgr2=0x559488180100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f248bfff640 0 client.17916 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T23:01:50.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f24b506e740 1 -- 172.21.15.152:0/282252883 shutdown_connections 2024-09-11T23:01:50.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f24b506e740 1 -- 172.21.15.152:0/282252883 wait complete. 2024-09-11T23:01:50.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f24b506e740 20 client.17916 trim_cache size 0 max 16384 2024-09-11T23:01:50.993 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:01:50.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f24b2c95640 1 -- 172.21.15.152:0/535921835 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f249c0a2cf0 msgr2=0x7f249c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:50.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f24b2c95640 1 --2- 172.21.15.152:0/535921835 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f249c0a2cf0 0x7f249c0a51b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f24a400ae90 tx=0x7f24a400a4d0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f24b2c95640 1 -- 172.21.15.152:0/535921835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24ac102790 msgr2=0x7f24ac1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:50.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.979+0000 7f24b2c95640 1 --2- 172.21.15.152:0/535921835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24ac102790 0x7f24ac1989b0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2498006010 tx=0x7f24980042f0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.980+0000 7f24b2c95640 1 -- 172.21.15.152:0/535921835 shutdown_connections 2024-09-11T23:01:50.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.980+0000 7f24b2c95640 1 --2- 172.21.15.152:0/535921835 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f249c0a2cf0 0x7f249c0a51b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.980+0000 7f24b2c95640 1 --2- 172.21.15.152:0/535921835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f24ac106f10 0x7f24ac19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.980+0000 7f24b2c95640 1 --2- 172.21.15.152:0/535921835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f24ac102790 0x7f24ac1989b0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.980+0000 7f24b2c95640 1 --2- 172.21.15.152:0/535921835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24ac101de0 0x7f24ac198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:50.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.980+0000 7f24b2c95640 1 -- 172.21.15.152:0/535921835 >> 172.21.15.152:0/535921835 conn(0x7f24ac0fd8c0 msgr2=0x7f24ac0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:50.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.980+0000 7f24b2c95640 1 -- 172.21.15.152:0/535921835 shutdown_connections 2024-09-11T23:01:50.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:50.980+0000 7f24b2c95640 1 -- 172.21.15.152:0/535921835 wait complete. 2024-09-11T23:01:51.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:50 smithi191 ceph-mon[35211]: pgmap v275: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T23:01:51.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:50 smithi204 ceph-mon[35454]: pgmap v275: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T23:01:51.814 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:01:52.060 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:01:52.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.511+0000 7f966595d640 1 -- 172.21.15.152:0/2775937962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601055b0 msgr2=0x7f966010d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:52.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.511+0000 7f966595d640 1 --2- 172.21.15.152:0/2775937962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601055b0 0x7f966010d130 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f9658009900 tx=0x7f965802a410 comp rx=0 tx=0).stop 2024-09-11T23:01:52.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.511+0000 7f966595d640 1 -- 172.21.15.152:0/2775937962 shutdown_connections 2024-09-11T23:01:52.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.511+0000 7f966595d640 1 --2- 172.21.15.152:0/2775937962 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601055b0 0x7f966010d130 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.511+0000 7f966595d640 1 --2- 172.21.15.152:0/2775937962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9660100e30 0x7f9660104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.511+0000 7f966595d640 1 --2- 172.21.15.152:0/2775937962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9660100480 0x7f9660100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.511+0000 7f966595d640 1 -- 172.21.15.152:0/2775937962 >> 172.21.15.152:0/2775937962 conn(0x7f9660077210 msgr2=0x7f96600fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:52.542 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.511+0000 7f966595d640 1 -- 172.21.15.152:0/2775937962 shutdown_connections 2024-09-11T23:01:52.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.511+0000 7f966595d640 1 -- 172.21.15.152:0/2775937962 wait complete. 2024-09-11T23:01:52.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.512+0000 7f966595d640 1 Processor -- start 2024-09-11T23:01:52.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.512+0000 7f966595d640 1 -- start start 2024-09-11T23:01:52.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.512+0000 7f966595d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9660100480 0x7f9660196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.512+0000 7f966595d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9660100e30 0x7f9660196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.512+0000 7f966595d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601055b0 0x7f966019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.512+0000 7f966595d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f966010fd90 con 0x7f96601055b0 2024-09-11T23:01:52.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.512+0000 7f966595d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f966010fc10 con 0x7f9660100480 2024-09-11T23:01:52.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.512+0000 7f966595d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f966010ff10 con 0x7f9660100e30 2024-09-11T23:01:52.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.512+0000 7f965f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601055b0 0x7f966019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.512+0000 7f965f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601055b0 0x7f966019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36976/0 (socket says 172.21.15.152:36976) 2024-09-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.512+0000 7f965f7fe640 1 -- 172.21.15.152:0/3515005468 learned_addr learned my addr 172.21.15.152:0/3515005468 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.512+0000 7f965effd640 1 --2- 172.21.15.152:0/3515005468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9660100480 0x7f9660196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.512+0000 7f965e7fc640 1 --2- 172.21.15.152:0/3515005468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9660100e30 0x7f9660196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.513+0000 7f965f7fe640 1 -- 172.21.15.152:0/3515005468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9660100e30 msgr2=0x7f9660196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.513+0000 7f965f7fe640 1 --2- 172.21.15.152:0/3515005468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9660100e30 0x7f9660196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.513+0000 7f965f7fe640 1 -- 172.21.15.152:0/3515005468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9660100480 msgr2=0x7f9660196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.513+0000 7f965f7fe640 1 --2- 172.21.15.152:0/3515005468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9660100480 0x7f9660196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.513+0000 7f965f7fe640 1 -- 172.21.15.152:0/3515005468 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966019b200 con 0x7f96601055b0 2024-09-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.513+0000 7f965f7fe640 1 --2- 172.21.15.152:0/3515005468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601055b0 0x7f966019ab20 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9658005ab0 tx=0x7f96580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.513+0000 7f966495b640 1 -- 172.21.15.152:0/3515005468 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965803f030 con 0x7f96601055b0 2024-09-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.513+0000 7f966495b640 1 -- 172.21.15.152:0/3515005468 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9658003ce0 con 0x7f96601055b0 2024-09-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.513+0000 7f966595d640 1 -- 172.21.15.152:0/3515005468 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966019b490 con 0x7f96601055b0 2024-09-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.513+0000 7f966495b640 1 -- 172.21.15.152:0/3515005468 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658035070 con 0x7f96601055b0 2024-09-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.513+0000 7f966595d640 1 -- 172.21.15.152:0/3515005468 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96601a2dd0 con 0x7f96601055b0 2024-09-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.515+0000 7f966495b640 1 -- 172.21.15.152:0/3515005468 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9658042030 con 0x7f96601055b0 2024-09-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f9667d36740 1 Processor -- start 2024-09-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f9667d36740 1 -- start start 2024-09-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f966495b640 1 --2- 172.21.15.152:0/3515005468 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f964c0a2cf0 0x7f964c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f9667d36740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fd040 0x5629215fd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f9667d36740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fddc0 0x5629215fe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f9667d36740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fe760 0x562921604f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f965e7fc640 1 --2- 172.21.15.152:0/3515005468 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f964c0a2cf0 0x7f964c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f9667d36740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5629215592f0 con 0x5629215fddc0 2024-09-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f9667d36740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562921500cf0 con 0x5629215fd040 2024-09-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f9667d36740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5629215552b0 con 0x5629215fe760 2024-09-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f966495b640 1 -- 172.21.15.152:0/3515005468 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f96580efbf0 con 0x7f96601055b0 2024-09-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f965e7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fddc0 0x5629215fe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f965e7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fddc0 0x5629215fe220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36984/0 (socket says 172.21.15.152:36984) 2024-09-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f965f7fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fe760 0x562921604f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f965e7fc640 1 -- 172.21.15.152:0/3873005345 learned_addr learned my addr 172.21.15.152:0/3873005345 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f965effd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fd040 0x5629215fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f965f7fe640 1 -- 172.21.15.152:0/3873005345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fd040 msgr2=0x5629215fd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f965f7fe640 1 --2- 172.21.15.152:0/3873005345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fd040 0x5629215fd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f965f7fe640 1 -- 172.21.15.152:0/3873005345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fddc0 msgr2=0x5629215fe220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f965f7fe640 1 --2- 172.21.15.152:0/3873005345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fddc0 0x5629215fe220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.516+0000 7f965e7fc640 1 --2- 172.21.15.152:0/3515005468 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f964c0a2cf0 0x7f964c0a51b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9648013460 tx=0x7f9648013860 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.517+0000 7f965f7fe640 1 -- 172.21.15.152:0/3873005345 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629214e53a0 con 0x5629215fe760 2024-09-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.517+0000 7f965effd640 1 --2- 172.21.15.152:0/3873005345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fd040 0x5629215fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.517+0000 7f965f7fe640 1 --2- 172.21.15.152:0/3873005345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fe760 0x562921604f20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f96580ed300 tx=0x7f96580b84e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e2e30dcbc5d529f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.517+0000 7f9655ffb640 1 -- 172.21.15.152:0/3873005345 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9658049030 con 0x5629215fe760 2024-09-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.517+0000 7f9655ffb640 1 -- 172.21.15.152:0/3873005345 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96580bc180 con 0x5629215fe760 2024-09-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.517+0000 7f9655ffb640 1 -- 172.21.15.152:0/3873005345 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96580ed580 con 0x5629215fe760 2024-09-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.518+0000 7f9667d36740 1 -- 172.21.15.152:0/3873005345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fe760 msgr2=0x562921604f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.518+0000 7f9667d36740 1 --2- 172.21.15.152:0/3873005345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fe760 0x562921604f20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f96580ed300 tx=0x7f96580b84e0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.518+0000 7f9667d36740 1 -- 172.21.15.152:0/3873005345 shutdown_connections 2024-09-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.518+0000 7f9667d36740 1 --2- 172.21.15.152:0/3873005345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fe760 0x562921604f20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.518+0000 7f9667d36740 1 --2- 172.21.15.152:0/3873005345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fddc0 0x5629215fe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.518+0000 7f9667d36740 1 --2- 172.21.15.152:0/3873005345 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fd040 0x5629215fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.518+0000 7f9667d36740 1 -- 172.21.15.152:0/3873005345 >> 172.21.15.152:0/3873005345 conn(0x5629215bc310 msgr2=0x5629215ef720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.518+0000 7f9667d36740 1 -- 172.21.15.152:0/3873005345 shutdown_connections 2024-09-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.518+0000 7f9667d36740 1 -- 172.21.15.152:0/3873005345 wait complete. 2024-09-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.518+0000 7f9667d36740 1 -- start start 2024-09-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.518+0000 7f9667d36740 1 Processor -- start 2024-09-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f9667d36740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fd040 0x5629215fc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f9667d36740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fddc0 0x5629215f9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f9667d36740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fe760 0x5629215f9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f9667d36740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5629215fdb40 con 0x5629215fd040 2024-09-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f9667d36740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562921514060 con 0x5629215fe760 2024-09-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f9667d36740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562921605460 con 0x5629215fddc0 2024-09-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f9667d36740 10 client.0 _ll_register_callbacks cb 0x5629212678c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f965effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fd040 0x5629215fc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f965effd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fd040 0x5629215fc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36996/0 (socket says 172.21.15.152:36996) 2024-09-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f965effd640 1 -- 172.21.15.152:0/2706156418 learned_addr learned my addr 172.21.15.152:0/2706156418 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f965e7fc640 1 --2- 172.21.15.152:0/2706156418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fddc0 0x5629215f9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f965f7fe640 1 --2- 172.21.15.152:0/2706156418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fe760 0x5629215f9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f965effd640 1 -- 172.21.15.152:0/2706156418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fddc0 msgr2=0x5629215f9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f965effd640 1 --2- 172.21.15.152:0/2706156418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fddc0 0x5629215f9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f965effd640 1 -- 172.21.15.152:0/2706156418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fe760 msgr2=0x5629215f9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f965effd640 1 --2- 172.21.15.152:0/2706156418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fe760 0x5629215f9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f965effd640 1 -- 172.21.15.152:0/2706156418 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629212ca3c0 con 0x5629215fd040 2024-09-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f965effd640 1 --2- 172.21.15.152:0/2706156418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fd040 0x5629215fc6b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f9650004360 tx=0x7f9650004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a9b35cb7da4116 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f963effd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f963effd640 1 -- 172.21.15.152:0/2706156418 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96500193f0 con 0x5629215fd040 2024-09-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f963effd640 1 -- 172.21.15.152:0/2706156418 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f965000e3f0 con 0x5629215fd040 2024-09-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f9667d36740 1 -- 172.21.15.152:0/2706156418 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96580bc180 con 0x5629215fd040 2024-09-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.519+0000 7f963effd640 1 -- 172.21.15.152:0/2706156418 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f965000e5f0 con 0x5629215fd040 2024-09-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.520+0000 7f9667d36740 10 client.17928 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.520+0000 7f9667d36740 1 -- 172.21.15.152:0/2706156418 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562921610cb0 con 0x5629215fd040 2024-09-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.520+0000 7f963effd640 1 -- 172.21.15.152:0/2706156418 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96580bc180 con 0x5629215fd040 2024-09-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.520+0000 7f963effd640 1 -- 172.21.15.152:0/2706156418 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9650020070 con 0x5629215fd040 2024-09-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.520+0000 7f9667d36740 10 client.17928 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.520+0000 7f9667d36740 10 client.17928 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.520+0000 7f9667d36740 20 client.17928 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.520+0000 7f9667d36740 1 --2- 172.21.15.152:0/2706156418 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x562921612210 0x562921614600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.520+0000 7f9667d36740 4 client.17928 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.520+0000 7f9667d36740 1 -- 172.21.15.152:0/2706156418 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562921566860 con 0x562921612210 2024-09-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.520+0000 7f965e7fc640 1 --2- 172.21.15.152:0/2706156418 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x562921612210 0x562921614600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.521+0000 7f963effd640 10 client.17928 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.521+0000 7f965e7fc640 1 --2- 172.21.15.152:0/2706156418 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x562921612210 0x562921614600 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d98ad637edc403d2 server_cookie=6afb0811e1b3412f in_seq=0 out_seq=0 2024-09-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.521+0000 7f963effd640 1 -- 172.21.15.152:0/2706156418 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562921566860 con 0x562921612210 2024-09-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.521+0000 7f963effd640 10 client.17928 handle_command_reply: tid=0 2024-09-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.521+0000 7f9667d36740 1 client.17928 shutdown 2024-09-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.522+0000 7f9667d36740 1 -- 172.21.15.152:0/2706156418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fd040 msgr2=0x5629215fc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.522+0000 7f9667d36740 1 --2- 172.21.15.152:0/2706156418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fd040 0x5629215fc6b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f9650004360 tx=0x7f9650004820 comp rx=0 tx=0).stop 2024-09-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.522+0000 7f9667d36740 1 -- 172.21.15.152:0/2706156418 shutdown_connections 2024-09-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.522+0000 7f9667d36740 1 --2- 172.21.15.152:0/2706156418 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x562921612210 0x562921614600 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.522+0000 7f9667d36740 1 --2- 172.21.15.152:0/2706156418 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fe760 0x5629215f9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.522+0000 7f9667d36740 1 --2- 172.21.15.152:0/2706156418 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fddc0 0x5629215f9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.522+0000 7f9667d36740 1 --2- 172.21.15.152:0/2706156418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fd040 0x5629215fc6b0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.522+0000 7f963effd640 0 client.17928 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.522+0000 7f9667d36740 1 -- 172.21.15.152:0/2706156418 >> 172.21.15.152:0/2706156418 conn(0x5629215bc310 msgr2=0x5629215eed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.522+0000 7f9667d36740 1 -- 172.21.15.152:0/2706156418 shutdown_connections 2024-09-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.522+0000 7f9667d36740 1 -- 172.21.15.152:0/2706156418 wait complete. 2024-09-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.522+0000 7f9667d36740 20 client.17928 trim_cache size 0 max 16384 2024-09-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.534+0000 7f9667d36740 1 Processor -- start 2024-09-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.534+0000 7f9667d36740 1 -- start start 2024-09-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f9667d36740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fd040 0x562921619f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f9667d36740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fddc0 0x56292161a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f9667d36740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fe760 0x5629215f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f9667d36740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f96601ab630 con 0x5629215fddc0 2024-09-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f9667d36740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f96601ad170 con 0x5629215fd040 2024-09-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f9667d36740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96601abd00 con 0x5629215fe760 2024-09-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f965e7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fddc0 0x56292161a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f965e7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fddc0 0x56292161a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:36998/0 (socket says 172.21.15.152:36998) 2024-09-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f965e7fc640 1 -- 172.21.15.152:0/1239084738 learned_addr learned my addr 172.21.15.152:0/1239084738 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f965e7fc640 1 -- 172.21.15.152:0/1239084738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fe760 msgr2=0x5629215f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f965e7fc640 1 --2- 172.21.15.152:0/1239084738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fe760 0x5629215f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f965e7fc640 1 -- 172.21.15.152:0/1239084738 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fd040 msgr2=0x562921619f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f965effd640 1 --2- 172.21.15.152:0/1239084738 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fd040 0x562921619f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f965e7fc640 1 --2- 172.21.15.152:0/1239084738 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fd040 0x562921619f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f965e7fc640 1 -- 172.21.15.152:0/1239084738 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629215fe3d0 con 0x5629215fddc0 2024-09-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f965effd640 1 --2- 172.21.15.152:0/1239084738 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fd040 0x562921619f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f965e7fc640 1 --2- 172.21.15.152:0/1239084738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fddc0 0x56292161a450 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f964800a520 tx=0x7f964801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff756d7aef1c002 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f96557fa640 1 -- 172.21.15.152:0/1239084738 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648030030 con 0x5629215fddc0 2024-09-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f96557fa640 1 -- 172.21.15.152:0/1239084738 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9648011e60 con 0x5629215fddc0 2024-09-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.535+0000 7f96557fa640 1 -- 172.21.15.152:0/1239084738 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96480047a0 con 0x5629215fddc0 2024-09-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.536+0000 7f9667d36740 1 -- 172.21.15.152:0/1239084738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fddc0 msgr2=0x56292161a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.536+0000 7f9667d36740 1 --2- 172.21.15.152:0/1239084738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fddc0 0x56292161a450 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f964800a520 tx=0x7f964801e000 comp rx=0 tx=0).stop 2024-09-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.536+0000 7f9667d36740 1 -- 172.21.15.152:0/1239084738 shutdown_connections 2024-09-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.536+0000 7f9667d36740 1 --2- 172.21.15.152:0/1239084738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fe760 0x5629215f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.536+0000 7f9667d36740 1 --2- 172.21.15.152:0/1239084738 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fddc0 0x56292161a450 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.536+0000 7f9667d36740 1 --2- 172.21.15.152:0/1239084738 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fd040 0x562921619f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.536+0000 7f9667d36740 1 -- 172.21.15.152:0/1239084738 >> 172.21.15.152:0/1239084738 conn(0x5629215bc310 msgr2=0x5629215f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.536+0000 7f9667d36740 1 -- 172.21.15.152:0/1239084738 shutdown_connections 2024-09-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.536+0000 7f9667d36740 1 -- 172.21.15.152:0/1239084738 wait complete. 2024-09-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.536+0000 7f9667d36740 1 -- start start 2024-09-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f9667d36740 1 Processor -- start 2024-09-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f9667d36740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fd040 0x5629216046f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f9667d36740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fddc0 0x562921601a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f9667d36740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fe760 0x562921601fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f9667d36740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f96601acaa0 con 0x5629215fe760 2024-09-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f9667d36740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f96601ac3d0 con 0x5629215fddc0 2024-09-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f9667d36740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f96601a9630 con 0x5629215fd040 2024-09-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f9667d36740 10 client.0 _ll_register_callbacks cb 0x7f965000ad40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f965f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fe760 0x562921601fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f965f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fe760 0x562921601fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37006/0 (socket says 172.21.15.152:37006) 2024-09-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f965f7fe640 1 -- 172.21.15.152:0/2517339791 learned_addr learned my addr 172.21.15.152:0/2517339791 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f965e7fc640 1 --2- 172.21.15.152:0/2517339791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fddc0 0x562921601a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f965effd640 1 --2- 172.21.15.152:0/2517339791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fd040 0x5629216046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f965f7fe640 1 -- 172.21.15.152:0/2517339791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fd040 msgr2=0x5629216046f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f965f7fe640 1 --2- 172.21.15.152:0/2517339791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fd040 0x5629216046f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f965f7fe640 1 -- 172.21.15.152:0/2517339791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fddc0 msgr2=0x562921601a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f965f7fe640 1 --2- 172.21.15.152:0/2517339791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fddc0 0x562921601a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f965f7fe640 1 -- 172.21.15.152:0/2517339791 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562921602800 con 0x5629215fe760 2024-09-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f965effd640 1 --2- 172.21.15.152:0/2517339791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fd040 0x5629216046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f965f7fe640 1 --2- 172.21.15.152:0/2517339791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fe760 0x562921601fd0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f96580b85e0 tx=0x7f9658004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e238cd1f7565e224 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.537+0000 7f963effd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.538+0000 7f963effd640 1 -- 172.21.15.152:0/2517339791 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96580d4070 con 0x5629215fe760 2024-09-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.538+0000 7f963effd640 1 -- 172.21.15.152:0/2517339791 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f96580c0120 con 0x5629215fe760 2024-09-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.538+0000 7f9667d36740 1 -- 172.21.15.152:0/2517339791 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9648011e60 con 0x5629215fe760 2024-09-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.538+0000 7f963effd640 1 -- 172.21.15.152:0/2517339791 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9658049040 con 0x5629215fe760 2024-09-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.538+0000 7f9667d36740 10 client.17934 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.538+0000 7f9667d36740 1 -- 172.21.15.152:0/2517339791 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56292167cd90 con 0x5629215fe760 2024-09-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.538+0000 7f963effd640 1 -- 172.21.15.152:0/2517339791 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9648011e60 con 0x5629215fe760 2024-09-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.538+0000 7f963effd640 1 -- 172.21.15.152:0/2517339791 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f96580b87f0 con 0x5629215fe760 2024-09-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.538+0000 7f9667d36740 10 client.17934 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.538+0000 7f9667d36740 10 client.17934 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.538+0000 7f9667d36740 20 client.17934 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.538+0000 7f9667d36740 1 --2- 172.21.15.152:0/2517339791 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x56292167d130 0x56292167f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.538+0000 7f9667d36740 4 client.17934 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.538+0000 7f9667d36740 1 -- 172.21.15.152:0/2517339791 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562921560d70 con 0x56292167d130 2024-09-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.539+0000 7f965effd640 1 --2- 172.21.15.152:0/2517339791 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x56292167d130 0x56292167f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f965effd640 1 --2- 172.21.15.152:0/2517339791 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x56292167d130 0x56292167f520 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33acd570612efc53 server_cookie=dfb4e8c5bd6c1131 in_seq=0 out_seq=0 2024-09-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f963effd640 10 client.17934 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f963effd640 1 -- 172.21.15.152:0/2517339791 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562921560d70 con 0x56292167d130 2024-09-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f963effd640 10 client.17934 handle_command_reply: tid=0 2024-09-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f9667d36740 1 client.17934 shutdown 2024-09-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f9667d36740 1 -- 172.21.15.152:0/2517339791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fe760 msgr2=0x562921601fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f9667d36740 1 --2- 172.21.15.152:0/2517339791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fe760 0x562921601fd0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f96580b85e0 tx=0x7f9658004590 comp rx=0 tx=0).stop 2024-09-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f9667d36740 1 -- 172.21.15.152:0/2517339791 shutdown_connections 2024-09-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f9667d36740 1 --2- 172.21.15.152:0/2517339791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629215fe760 0x562921601fd0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f9667d36740 1 --2- 172.21.15.152:0/2517339791 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x56292167d130 0x56292167f520 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f963effd640 0 client.17934 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f9667d36740 1 --2- 172.21.15.152:0/2517339791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5629215fddc0 0x562921601a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f9667d36740 1 --2- 172.21.15.152:0/2517339791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5629215fd040 0x5629216046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f9667d36740 1 -- 172.21.15.152:0/2517339791 >> 172.21.15.152:0/2517339791 conn(0x5629215bc310 msgr2=0x5629215fb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f9667d36740 1 -- 172.21.15.152:0/2517339791 shutdown_connections 2024-09-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f9667d36740 1 -- 172.21.15.152:0/2517339791 wait complete. 2024-09-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.540+0000 7f9667d36740 20 client.17934 trim_cache size 0 max 16384 2024-09-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.541+0000 7f966595d640 1 -- 172.21.15.152:0/3515005468 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f964c0a2cf0 msgr2=0x7f964c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.541+0000 7f966595d640 1 --2- 172.21.15.152:0/3515005468 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f964c0a2cf0 0x7f964c0a51b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9648013460 tx=0x7f9648013860 comp rx=0 tx=0).stop 2024-09-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.541+0000 7f966595d640 1 -- 172.21.15.152:0/3515005468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601055b0 msgr2=0x7f966019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.541+0000 7f966595d640 1 --2- 172.21.15.152:0/3515005468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601055b0 0x7f966019ab20 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9658005ab0 tx=0x7f96580042f0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.541+0000 7f966595d640 1 -- 172.21.15.152:0/3515005468 shutdown_connections 2024-09-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.541+0000 7f966595d640 1 --2- 172.21.15.152:0/3515005468 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f964c0a2cf0 0x7f964c0a51b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.541+0000 7f966595d640 1 --2- 172.21.15.152:0/3515005468 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96601055b0 0x7f966019ab20 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.541+0000 7f966595d640 1 --2- 172.21.15.152:0/3515005468 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9660100e30 0x7f9660196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.541+0000 7f966595d640 1 --2- 172.21.15.152:0/3515005468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9660100480 0x7f9660196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.541+0000 7f966595d640 1 -- 172.21.15.152:0/3515005468 >> 172.21.15.152:0/3515005468 conn(0x7f9660077210 msgr2=0x7f9660103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.541+0000 7f966595d640 1 -- 172.21.15.152:0/3515005468 shutdown_connections 2024-09-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:52.541+0000 7f966595d640 1 -- 172.21.15.152:0/3515005468 wait complete. 2024-09-11T23:01:52.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:52 smithi152 ceph-mon[31181]: pgmap v276: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:01:53.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:52 smithi191 ceph-mon[35211]: pgmap v276: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:01:53.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:52 smithi204 ceph-mon[35454]: pgmap v276: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T23:01:53.317 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:01:53.564 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:01:54.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.141+0000 7fd1bba62640 1 -- 172.21.15.152:0/3433390419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b4112640 msgr2=0x7fd1b4112a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:54.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.141+0000 7fd1bba62640 1 --2- 172.21.15.152:0/3433390419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b4112640 0x7fd1b4112a20 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009900 tx=0x7fd1b002a410 comp rx=0 tx=0).stop 2024-09-11T23:01:54.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.141+0000 7fd1bba62640 1 -- 172.21.15.152:0/3433390419 shutdown_connections 2024-09-11T23:01:54.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.141+0000 7fd1bba62640 1 --2- 172.21.15.152:0/3433390419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1b410cf30 0x7fd1b4119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.141+0000 7fd1bba62640 1 --2- 172.21.15.152:0/3433390419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1b410c590 0x7fd1b410c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.142+0000 7fd1bba62640 1 --2- 172.21.15.152:0/3433390419 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b4112640 0x7fd1b4112a20 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.142+0000 7fd1bba62640 1 -- 172.21.15.152:0/3433390419 >> 172.21.15.152:0/3433390419 conn(0x7fd1b406cdc0 msgr2=0x7fd1b406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:54.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.142+0000 7fd1bba62640 1 -- 172.21.15.152:0/3433390419 shutdown_connections 2024-09-11T23:01:54.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.142+0000 7fd1bba62640 1 -- 172.21.15.152:0/3433390419 wait complete. 2024-09-11T23:01:54.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.142+0000 7fd1bba62640 1 Processor -- start 2024-09-11T23:01:54.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1bba62640 1 -- start start 2024-09-11T23:01:54.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1bba62640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b410c590 0x7fd1b41a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1bba62640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1b410cf30 0x7fd1b41a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1bba62640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1b4112640 0x7fd1b41a7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1bba62640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b411c9a0 con 0x7fd1b410c590 2024-09-11T23:01:54.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1bba62640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b411c820 con 0x7fd1b4112640 2024-09-11T23:01:54.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1bba62640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b411cb20 con 0x7fd1b410cf30 2024-09-11T23:01:54.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1b97fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b410c590 0x7fd1b41a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1b97fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b410c590 0x7fd1b41a2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37028/0 (socket says 172.21.15.152:37028) 2024-09-11T23:01:54.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1b97fe640 1 -- 172.21.15.152:0/3227709081 learned_addr learned my addr 172.21.15.152:0/3227709081 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:54.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1b9fff640 1 --2- 172.21.15.152:0/3227709081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1b4112640 0x7fd1b41a7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1b8ffd640 1 --2- 172.21.15.152:0/3227709081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1b410cf30 0x7fd1b41a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1b97fe640 1 -- 172.21.15.152:0/3227709081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1b410cf30 msgr2=0x7fd1b41a3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:54.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1b97fe640 1 --2- 172.21.15.152:0/3227709081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1b410cf30 0x7fd1b41a3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1b97fe640 1 -- 172.21.15.152:0/3227709081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1b4112640 msgr2=0x7fd1b41a7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:54.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1b97fe640 1 --2- 172.21.15.152:0/3227709081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1b4112640 0x7fd1b41a7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.143+0000 7fd1b97fe640 1 -- 172.21.15.152:0/3227709081 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b41a7e70 con 0x7fd1b410c590 2024-09-11T23:01:54.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.144+0000 7fd1b97fe640 1 --2- 172.21.15.152:0/3227709081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b410c590 0x7fd1b41a2ec0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0006010 tx=0x7fd1b0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:54.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.144+0000 7fd1aa7fc640 1 -- 172.21.15.152:0/3227709081 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b003f070 con 0x7fd1b410c590 2024-09-11T23:01:54.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.144+0000 7fd1bba62640 1 -- 172.21.15.152:0/3227709081 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b41a8100 con 0x7fd1b410c590 2024-09-11T23:01:54.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.144+0000 7fd1aa7fc640 1 -- 172.21.15.152:0/3227709081 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b0031c30 con 0x7fd1b410c590 2024-09-11T23:01:54.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.144+0000 7fd1aa7fc640 1 -- 172.21.15.152:0/3227709081 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0035030 con 0x7fd1b410c590 2024-09-11T23:01:54.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.144+0000 7fd1bba62640 1 -- 172.21.15.152:0/3227709081 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b41afa40 con 0x7fd1b410c590 2024-09-11T23:01:54.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.146+0000 7fd1aa7fc640 1 -- 172.21.15.152:0/3227709081 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd1b0042070 con 0x7fd1b410c590 2024-09-11T23:01:54.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.147+0000 7fd1bde3b740 1 Processor -- start 2024-09-11T23:01:54.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.147+0000 7fd1bde3b740 1 -- start start 2024-09-11T23:01:54.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.147+0000 7fd1aa7fc640 1 --2- 172.21.15.152:0/3227709081 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd1a00a2cf0 0x7fd1a00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.147+0000 7fd1bde3b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556889973b40 0x556889972090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.147+0000 7fd1bde3b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568899725d0 0x556889972a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.147+0000 7fd1bde3b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55688997d370 0x55688997f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.147+0000 7fd1bde3b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5568898d82f0 con 0x55688997d370 2024-09-11T23:01:54.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.147+0000 7fd1bde3b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55688987fcf0 con 0x5568899725d0 2024-09-11T23:01:54.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.147+0000 7fd1bde3b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5568898d42b0 con 0x556889973b40 2024-09-11T23:01:54.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.147+0000 7fd1aa7fc640 1 -- 172.21.15.152:0/3227709081 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7fd1b00efb30 con 0x7fd1b410c590 2024-09-11T23:01:54.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.147+0000 7fd1b9fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55688997d370 0x55688997f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.147+0000 7fd1b8ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568899725d0 0x556889972a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.147+0000 7fd1b97fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556889973b40 0x556889972090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.148+0000 7fd1b9fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55688997d370 0x55688997f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37042/0 (socket says 172.21.15.152:37042) 2024-09-11T23:01:54.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.148+0000 7fd1b9fff640 1 -- 172.21.15.152:0/1037049096 learned_addr learned my addr 172.21.15.152:0/1037049096 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:54.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.148+0000 7fd1b9fff640 1 --2- 172.21.15.152:0/3227709081 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd1a00a2cf0 0x7fd1a00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.148+0000 7fd1b8ffd640 1 -- 172.21.15.152:0/1037049096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556889973b40 msgr2=0x556889972090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:54.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.148+0000 7fd1b8ffd640 1 --2- 172.21.15.152:0/1037049096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556889973b40 0x556889972090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.148+0000 7fd1b8ffd640 1 -- 172.21.15.152:0/1037049096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55688997d370 msgr2=0x55688997f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:54.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.148+0000 7fd1b8ffd640 1 --2- 172.21.15.152:0/1037049096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55688997d370 0x55688997f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.148+0000 7fd1b8ffd640 1 -- 172.21.15.152:0/1037049096 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568898643a0 con 0x5568899725d0 2024-09-11T23:01:54.176 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.148+0000 7fd1b97fe640 1 --2- 172.21.15.152:0/1037049096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556889973b40 0x556889972090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:01:54.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.148+0000 7fd1b9fff640 1 --2- 172.21.15.152:0/1037049096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55688997d370 0x55688997f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:01:54.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.148+0000 7fd1b8ffd640 1 --2- 172.21.15.152:0/1037049096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568899725d0 0x556889972a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4005d00 tx=0x7fd1a40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c92209fa0ef52f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:54.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.148+0000 7fd1b9fff640 1 --2- 172.21.15.152:0/3227709081 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd1a00a2cf0 0x7fd1a00a51b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x556889973070 tx=0x7fd1ac009600 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:54.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.148+0000 7fd19f7fe640 1 -- 172.21.15.152:0/1037049096 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a400be60 con 0x5568899725d0 2024-09-11T23:01:54.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.148+0000 7fd19f7fe640 1 -- 172.21.15.152:0/1037049096 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1a401c960 con 0x5568899725d0 2024-09-11T23:01:54.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.148+0000 7fd19f7fe640 1 -- 172.21.15.152:0/1037049096 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4011a90 con 0x5568899725d0 2024-09-11T23:01:54.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.149+0000 7fd1bde3b740 1 -- 172.21.15.152:0/1037049096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568899725d0 msgr2=0x556889972a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:54.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.149+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/1037049096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568899725d0 0x556889972a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4005d00 tx=0x7fd1a40107a0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.149+0000 7fd1bde3b740 1 -- 172.21.15.152:0/1037049096 shutdown_connections 2024-09-11T23:01:54.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.149+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/1037049096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55688997d370 0x55688997f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.149+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/1037049096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568899725d0 0x556889972a10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.177 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.149+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/1037049096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556889973b40 0x556889972090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.149+0000 7fd1bde3b740 1 -- 172.21.15.152:0/1037049096 >> 172.21.15.152:0/1037049096 conn(0x55688993b310 msgr2=0x55688996ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:54.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.150+0000 7fd1bde3b740 1 -- 172.21.15.152:0/1037049096 shutdown_connections 2024-09-11T23:01:54.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.150+0000 7fd1bde3b740 1 -- 172.21.15.152:0/1037049096 wait complete. 2024-09-11T23:01:54.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.150+0000 7fd1bde3b740 1 -- start start 2024-09-11T23:01:54.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.150+0000 7fd1bde3b740 1 Processor -- start 2024-09-11T23:01:54.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1bde3b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5568899725d0 0x55688997cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1bde3b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 0x556889979fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1bde3b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55688997d370 0x55688997a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1bde3b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x556889980070 con 0x5568899725d0 2024-09-11T23:01:54.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1bde3b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556889893060 con 0x556889973b40 2024-09-11T23:01:54.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1bde3b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5568899801f0 con 0x55688997d370 2024-09-11T23:01:54.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1bde3b740 10 client.0 _ll_register_callbacks cb 0x5568895e68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:54.178 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1b97fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5568899725d0 0x55688997cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1b8ffd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 0x556889979fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1b97fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5568899725d0 0x55688997cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37048/0 (socket says 172.21.15.152:37048) 2024-09-11T23:01:54.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1b97fe640 1 -- 172.21.15.152:0/1726442210 learned_addr learned my addr 172.21.15.152:0/1726442210 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:54.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1b9fff640 1 --2- 172.21.15.152:0/1726442210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55688997d370 0x55688997a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1b97fe640 1 -- 172.21.15.152:0/1726442210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55688997d370 msgr2=0x55688997a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:54.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1b97fe640 1 --2- 172.21.15.152:0/1726442210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55688997d370 0x55688997a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1b97fe640 1 -- 172.21.15.152:0/1726442210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 msgr2=0x556889979fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:54.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1b97fe640 1 --2- 172.21.15.152:0/1726442210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 0x556889979fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1b97fe640 1 -- 172.21.15.152:0/1726442210 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568896493c0 con 0x5568899725d0 2024-09-11T23:01:54.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1b9fff640 1 --2- 172.21.15.152:0/1726442210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55688997d370 0x55688997a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:01:54.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1b8ffd640 1 --2- 172.21.15.152:0/1726442210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 0x556889979fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:01:54.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd1b97fe640 1 --2- 172.21.15.152:0/1726442210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5568899725d0 0x55688997cd20 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0005c40 tx=0x7fd1b003cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf37720aa59c4a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:54.179 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd19cff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:54.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.151+0000 7fd19cff9640 1 -- 172.21.15.152:0/1726442210 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b00c03c0 con 0x5568899725d0 2024-09-11T23:01:54.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.152+0000 7fd19cff9640 1 -- 172.21.15.152:0/1726442210 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7fd1b00ed4c0 con 0x5568899725d0 2024-09-11T23:01:54.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.152+0000 7fd1bde3b740 1 -- 172.21.15.152:0/1726442210 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1a401c960 con 0x5568899725d0 2024-09-11T23:01:54.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.152+0000 7fd19cff9640 1 -- 172.21.15.152:0/1726442210 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1b00ed6c0 con 0x5568899725d0 2024-09-11T23:01:54.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.152+0000 7fd1bde3b740 10 client.17946 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:54.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.152+0000 7fd1bde3b740 1 -- 172.21.15.152:0/1726442210 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55688998bac0 con 0x5568899725d0 2024-09-11T23:01:54.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.152+0000 7fd19cff9640 1 -- 172.21.15.152:0/1726442210 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1a401c960 con 0x5568899725d0 2024-09-11T23:01:54.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.152+0000 7fd19cff9640 1 -- 172.21.15.152:0/1726442210 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd1b00bc320 con 0x5568899725d0 2024-09-11T23:01:54.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.152+0000 7fd1bde3b740 10 client.17946 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:54.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.152+0000 7fd1bde3b740 10 client.17946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:01:54.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.152+0000 7fd1bde3b740 20 client.17946 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:54.180 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.152+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/1726442210 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55688998cfc0 0x55688998f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.152+0000 7fd1bde3b740 4 client.17946 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:01:54.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.152+0000 7fd1bde3b740 1 -- 172.21.15.152:0/1726442210 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568898e5860 con 0x55688998cfc0 2024-09-11T23:01:54.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.153+0000 7fd1b8ffd640 1 --2- 172.21.15.152:0/1726442210 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55688998cfc0 0x55688998f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.153+0000 7fd1b8ffd640 1 --2- 172.21.15.152:0/1726442210 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55688998cfc0 0x55688998f410 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25c5a7528fc23f64 server_cookie=8c329366e80eac20 in_seq=0 out_seq=0 2024-09-11T23:01:54.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.153+0000 7fd19cff9640 10 client.17946 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:54.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.154+0000 7fd19cff9640 1 -- 172.21.15.152:0/1726442210 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5568898e5860 con 0x55688998cfc0 2024-09-11T23:01:54.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.154+0000 7fd19cff9640 10 client.17946 handle_command_reply: tid=0 2024-09-11T23:01:54.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.154+0000 7fd1bde3b740 1 client.17946 shutdown 2024-09-11T23:01:54.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.154+0000 7fd1bde3b740 1 -- 172.21.15.152:0/1726442210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5568899725d0 msgr2=0x55688997cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:54.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.154+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/1726442210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5568899725d0 0x55688997cd20 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0005c40 tx=0x7fd1b003cf30 comp rx=0 tx=0).stop 2024-09-11T23:01:54.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.154+0000 7fd1bde3b740 1 -- 172.21.15.152:0/1726442210 shutdown_connections 2024-09-11T23:01:54.181 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.154+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/1726442210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55688997d370 0x55688997a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.154+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/1726442210 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55688998cfc0 0x55688998f410 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.154+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/1726442210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 0x556889979fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.154+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/1726442210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5568899725d0 0x55688997cd20 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.154+0000 7fd1bde3b740 1 -- 172.21.15.152:0/1726442210 >> 172.21.15.152:0/1726442210 conn(0x55688993b310 msgr2=0x556889974960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:54.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.154+0000 7fd19cff9640 0 client.17946 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:54.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.154+0000 7fd1bde3b740 1 -- 172.21.15.152:0/1726442210 shutdown_connections 2024-09-11T23:01:54.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.154+0000 7fd1bde3b740 1 -- 172.21.15.152:0/1726442210 wait complete. 2024-09-11T23:01:54.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.154+0000 7fd1bde3b740 20 client.17946 trim_cache size 0 max 16384 2024-09-11T23:01:54.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.168+0000 7fd1bde3b740 1 Processor -- start 2024-09-11T23:01:54.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.168+0000 7fd1bde3b740 1 -- start start 2024-09-11T23:01:54.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.168+0000 7fd1bde3b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568899725d0 0x556889989310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.168+0000 7fd1bde3b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 0x55688997e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.182 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.168+0000 7fd1bde3b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55688997d370 0x55688997eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.168+0000 7fd1bde3b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41b8260 con 0x55688997d370 2024-09-11T23:01:54.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.168+0000 7fd1bde3b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41b9da0 con 0x556889973b40 2024-09-11T23:01:54.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.168+0000 7fd1bde3b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41b8930 con 0x5568899725d0 2024-09-11T23:01:54.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.168+0000 7fd1b9fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55688997d370 0x55688997eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.168+0000 7fd1b97fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568899725d0 0x556889989310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.168+0000 7fd1b9fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55688997d370 0x55688997eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37056/0 (socket says 172.21.15.152:37056) 2024-09-11T23:01:54.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.168+0000 7fd1b97fe640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568899725d0 0x556889989310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:57074/0 (socket says 172.21.15.152:57074) 2024-09-11T23:01:54.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.168+0000 7fd1b9fff640 1 -- 172.21.15.152:0/3875184247 learned_addr learned my addr 172.21.15.152:0/3875184247 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:54.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.168+0000 7fd1b8ffd640 1 --2- 172.21.15.152:0/3875184247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 0x55688997e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.169+0000 7fd1b8ffd640 1 -- 172.21.15.152:0/3875184247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568899725d0 msgr2=0x556889989310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:54.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.169+0000 7fd1b8ffd640 1 --2- 172.21.15.152:0/3875184247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568899725d0 0x556889989310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.169+0000 7fd1b8ffd640 1 -- 172.21.15.152:0/3875184247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55688997d370 msgr2=0x55688997eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:54.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.169+0000 7fd1b8ffd640 1 --2- 172.21.15.152:0/3875184247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55688997d370 0x55688997eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.183 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.169+0000 7fd1b8ffd640 1 -- 172.21.15.152:0/3875184247 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55688997a8e0 con 0x556889973b40 2024-09-11T23:01:54.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.169+0000 7fd1b97fe640 1 --2- 172.21.15.152:0/3875184247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568899725d0 0x556889989310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:01:54.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.169+0000 7fd1b8ffd640 1 --2- 172.21.15.152:0/3875184247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 0x55688997e760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4000c00 tx=0x7fd1a4011550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d85cca2a06495c90 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:54.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.169+0000 7fd19effd640 1 -- 172.21.15.152:0/3875184247 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4004020 con 0x556889973b40 2024-09-11T23:01:54.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.169+0000 7fd19effd640 1 -- 172.21.15.152:0/3875184247 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1a4005520 con 0x556889973b40 2024-09-11T23:01:54.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.169+0000 7fd19effd640 1 -- 172.21.15.152:0/3875184247 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a40048c0 con 0x556889973b40 2024-09-11T23:01:54.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.170+0000 7fd1bde3b740 1 -- 172.21.15.152:0/3875184247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 msgr2=0x55688997e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:54.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.170+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/3875184247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 0x55688997e760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4000c00 tx=0x7fd1a4011550 comp rx=0 tx=0).stop 2024-09-11T23:01:54.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.170+0000 7fd1bde3b740 1 -- 172.21.15.152:0/3875184247 shutdown_connections 2024-09-11T23:01:54.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.170+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/3875184247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55688997d370 0x55688997eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.170+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/3875184247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 0x55688997e760 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.170+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/3875184247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568899725d0 0x556889989310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.170+0000 7fd1bde3b740 1 -- 172.21.15.152:0/3875184247 >> 172.21.15.152:0/3875184247 conn(0x55688993b310 msgr2=0x5568899788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:54.184 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.170+0000 7fd1bde3b740 1 -- 172.21.15.152:0/3875184247 shutdown_connections 2024-09-11T23:01:54.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.170+0000 7fd1bde3b740 1 -- 172.21.15.152:0/3875184247 wait complete. 2024-09-11T23:01:54.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.171+0000 7fd1bde3b740 1 -- start start 2024-09-11T23:01:54.185 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.171+0000 7fd1bde3b740 1 Processor -- start 2024-09-11T23:01:54.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.171+0000 7fd1bde3b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5568899725d0 0x556889985720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.171+0000 7fd1bde3b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 0x556889985c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.171+0000 7fd1bde3b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55688997d370 0x55688997e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.171+0000 7fd1bde3b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41b96d0 con 0x5568899725d0 2024-09-11T23:01:54.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.171+0000 7fd1bde3b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41b9000 con 0x556889973b40 2024-09-11T23:01:54.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.171+0000 7fd1bde3b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b41b6260 con 0x55688997d370 2024-09-11T23:01:54.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.172+0000 7fd1bde3b740 10 client.0 _ll_register_callbacks cb 0x7fd1b00b7cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:54.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.172+0000 7fd1b97fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5568899725d0 0x556889985720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.188 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.172+0000 7fd1b9fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55688997d370 0x55688997e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.172+0000 7fd1b97fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5568899725d0 0x556889985720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37068/0 (socket says 172.21.15.152:37068) 2024-09-11T23:01:54.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.172+0000 7fd1b97fe640 1 -- 172.21.15.152:0/2164613167 learned_addr learned my addr 172.21.15.152:0/2164613167 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:54.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.172+0000 7fd1b8ffd640 1 --2- 172.21.15.152:0/2164613167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 0x556889985c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.172+0000 7fd1b97fe640 1 -- 172.21.15.152:0/2164613167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55688997d370 msgr2=0x55688997e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:54.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.172+0000 7fd1b97fe640 1 --2- 172.21.15.152:0/2164613167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55688997d370 0x55688997e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.172+0000 7fd1b97fe640 1 -- 172.21.15.152:0/2164613167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 msgr2=0x556889985c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:54.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.172+0000 7fd1b97fe640 1 --2- 172.21.15.152:0/2164613167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 0x556889985c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.172+0000 7fd1b97fe640 1 -- 172.21.15.152:0/2164613167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55688997e9d0 con 0x5568899725d0 2024-09-11T23:01:54.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.172+0000 7fd1b97fe640 1 --2- 172.21.15.152:0/2164613167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5568899725d0 0x556889985720 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0005820 tx=0x7fd1b0031f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eeb563a1d27a040 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:54.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.172+0000 7fd19cff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:54.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.173+0000 7fd19cff9640 1 -- 172.21.15.152:0/2164613167 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b00ed6d0 con 0x5568899725d0 2024-09-11T23:01:54.189 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.173+0000 7fd19cff9640 1 -- 172.21.15.152:0/2164613167 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7fd1b00c47f0 con 0x5568899725d0 2024-09-11T23:01:54.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.173+0000 7fd1bde3b740 1 -- 172.21.15.152:0/2164613167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1a4005520 con 0x5568899725d0 2024-09-11T23:01:54.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.173+0000 7fd19cff9640 1 -- 172.21.15.152:0/2164613167 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd1b00c4c80 con 0x5568899725d0 2024-09-11T23:01:54.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.173+0000 7fd1bde3b740 10 client.17952 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:54.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.173+0000 7fd1bde3b740 1 -- 172.21.15.152:0/2164613167 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556889984fc0 con 0x5568899725d0 2024-09-11T23:01:54.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.173+0000 7fd19cff9640 1 -- 172.21.15.152:0/2164613167 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b0032000 con 0x5568899725d0 2024-09-11T23:01:54.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.173+0000 7fd19cff9640 1 -- 172.21.15.152:0/2164613167 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd1b00bb5a0 con 0x5568899725d0 2024-09-11T23:01:54.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.173+0000 7fd1bde3b740 10 client.17952 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:54.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.173+0000 7fd1bde3b740 10 client.17952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:01:54.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.173+0000 7fd1bde3b740 20 client.17952 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:54.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.173+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/2164613167 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5568899f7cd0 0x5568899fa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:54.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.173+0000 7fd1bde3b740 4 client.17952 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:01:54.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.173+0000 7fd1bde3b740 1 -- 172.21.15.152:0/2164613167 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd1a4005520 con 0x5568899f7cd0 2024-09-11T23:01:54.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.174+0000 7fd1b8ffd640 1 --2- 172.21.15.152:0/2164613167 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5568899f7cd0 0x5568899fa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:54.190 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.174+0000 7fd1b8ffd640 1 --2- 172.21.15.152:0/2164613167 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5568899f7cd0 0x5568899fa0c0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d23dd1946835c02 server_cookie=ccfb1f7141b14c69 in_seq=0 out_seq=0 2024-09-11T23:01:54.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.174+0000 7fd19cff9640 10 client.17952 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:54.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.174+0000 7fd19cff9640 1 -- 172.21.15.152:0/2164613167 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd1a4005520 con 0x5568899f7cd0 2024-09-11T23:01:54.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.174+0000 7fd19cff9640 10 client.17952 handle_command_reply: tid=0 2024-09-11T23:01:54.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.174+0000 7fd1bde3b740 1 client.17952 shutdown 2024-09-11T23:01:54.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.174+0000 7fd1bde3b740 1 -- 172.21.15.152:0/2164613167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5568899725d0 msgr2=0x556889985720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:54.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.174+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/2164613167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5568899725d0 0x556889985720 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0005820 tx=0x7fd1b0031f20 comp rx=0 tx=0).stop 2024-09-11T23:01:54.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.174+0000 7fd1bde3b740 1 -- 172.21.15.152:0/2164613167 shutdown_connections 2024-09-11T23:01:54.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.174+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/2164613167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55688997d370 0x55688997e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.174+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/2164613167 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5568899f7cd0 0x5568899fa0c0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.174+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/2164613167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556889973b40 0x556889985c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.174+0000 7fd1bde3b740 1 --2- 172.21.15.152:0/2164613167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5568899725d0 0x556889985720 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.174+0000 7fd19cff9640 0 client.17952 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:54.191 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.174+0000 7fd1bde3b740 1 -- 172.21.15.152:0/2164613167 >> 172.21.15.152:0/2164613167 conn(0x55688993b310 msgr2=0x55688997c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:54.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.175+0000 7fd1bde3b740 1 -- 172.21.15.152:0/2164613167 shutdown_connections 2024-09-11T23:01:54.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.175+0000 7fd1bde3b740 1 -- 172.21.15.152:0/2164613167 wait complete. 2024-09-11T23:01:54.192 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.175+0000 7fd1bde3b740 20 client.17952 trim_cache size 0 max 16384 2024-09-11T23:01:54.192 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:01:54.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.175+0000 7fd1bba62640 1 -- 172.21.15.152:0/3227709081 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd1a00a2cf0 msgr2=0x7fd1a00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:54.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.175+0000 7fd1bba62640 1 --2- 172.21.15.152:0/3227709081 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd1a00a2cf0 0x7fd1a00a51b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x556889973070 tx=0x7fd1ac009600 comp rx=0 tx=0).stop 2024-09-11T23:01:54.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.175+0000 7fd1bba62640 1 -- 172.21.15.152:0/3227709081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b410c590 msgr2=0x7fd1b41a2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:54.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.175+0000 7fd1bba62640 1 --2- 172.21.15.152:0/3227709081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b410c590 0x7fd1b41a2ec0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0006010 tx=0x7fd1b0003c10 comp rx=0 tx=0).stop 2024-09-11T23:01:54.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.175+0000 7fd1bba62640 1 -- 172.21.15.152:0/3227709081 shutdown_connections 2024-09-11T23:01:54.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.175+0000 7fd1bba62640 1 --2- 172.21.15.152:0/3227709081 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd1a00a2cf0 0x7fd1a00a51b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.175+0000 7fd1bba62640 1 --2- 172.21.15.152:0/3227709081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd1b4112640 0x7fd1b41a7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.175+0000 7fd1bba62640 1 --2- 172.21.15.152:0/3227709081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1b410cf30 0x7fd1b41a3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.175+0000 7fd1bba62640 1 --2- 172.21.15.152:0/3227709081 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1b410c590 0x7fd1b41a2ec0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:54.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.175+0000 7fd1bba62640 1 -- 172.21.15.152:0/3227709081 >> 172.21.15.152:0/3227709081 conn(0x7fd1b406cdc0 msgr2=0x7fd1b410af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:54.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.176+0000 7fd1bba62640 1 -- 172.21.15.152:0/3227709081 shutdown_connections 2024-09-11T23:01:54.194 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:54.176+0000 7fd1bba62640 1 -- 172.21.15.152:0/3227709081 wait complete. 2024-09-11T23:01:54.848 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:54 smithi152 ceph-mon[31181]: pgmap v277: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:55.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:54 smithi191 ceph-mon[35211]: pgmap v277: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:55.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:54 smithi204 ceph-mon[35454]: pgmap v277: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:55.878 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:01:55.879 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:01:56.140 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:01:56.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.660+0000 7f15f6cd2640 1 -- 172.21.15.152:0/1627398076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f0102570 msgr2=0x7f15f01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:56.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.660+0000 7f15f6cd2640 1 --2- 172.21.15.152:0/1627398076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f0102570 0x7f15f01029b0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009900 tx=0x7f15dc02a480 comp rx=0 tx=0).stop 2024-09-11T23:01:56.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.660+0000 7f15f6cd2640 1 -- 172.21.15.152:0/1627398076 shutdown_connections 2024-09-11T23:01:56.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.660+0000 7f15f6cd2640 1 --2- 172.21.15.152:0/1627398076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15f0102ef0 0x7f15f010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.660+0000 7f15f6cd2640 1 --2- 172.21.15.152:0/1627398076 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f0102570 0x7f15f01029b0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.660+0000 7f15f6cd2640 1 --2- 172.21.15.152:0/1627398076 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f0100e20 0x7f15f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.660+0000 7f15f6cd2640 1 -- 172.21.15.152:0/1627398076 >> 172.21.15.152:0/1627398076 conn(0x7f15f00fc7d0 msgr2=0x7f15f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:56.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.660+0000 7f15f6cd2640 1 -- 172.21.15.152:0/1627398076 shutdown_connections 2024-09-11T23:01:56.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.660+0000 7f15f6cd2640 1 -- 172.21.15.152:0/1627398076 wait complete. 2024-09-11T23:01:56.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.660+0000 7f15f6cd2640 1 Processor -- start 2024-09-11T23:01:56.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f6cd2640 1 -- start start 2024-09-11T23:01:56.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f6cd2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f0100e20 0x7f15f01984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f6cd2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f0102570 0x7f15f0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f6cd2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15f0102ef0 0x7f15f019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f6cd2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f15f0110fa0 con 0x7f15f0100e20 2024-09-11T23:01:56.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f6cd2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f15f0110e20 con 0x7f15f0102570 2024-09-11T23:01:56.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f6cd2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f15f0111120 con 0x7f15f0102ef0 2024-09-11T23:01:56.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f4a6e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f0100e20 0x7f15f01984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f4a6e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f0100e20 0x7f15f01984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37088/0 (socket says 172.21.15.152:37088) 2024-09-11T23:01:56.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f4a6e640 1 -- 172.21.15.152:0/276791001 learned_addr learned my addr 172.21.15.152:0/276791001 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:56.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15effff640 1 --2- 172.21.15.152:0/276791001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f0102570 0x7f15f0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f526f640 1 --2- 172.21.15.152:0/276791001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15f0102ef0 0x7f15f019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f4a6e640 1 -- 172.21.15.152:0/276791001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15f0102ef0 msgr2=0x7f15f019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:56.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f4a6e640 1 --2- 172.21.15.152:0/276791001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15f0102ef0 0x7f15f019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f4a6e640 1 -- 172.21.15.152:0/276791001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f0102570 msgr2=0x7f15f0198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:56.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f4a6e640 1 --2- 172.21.15.152:0/276791001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f0102570 0x7f15f0198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f4a6e640 1 -- 172.21.15.152:0/276791001 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f019d470 con 0x7f15f0100e20 2024-09-11T23:01:56.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.661+0000 7f15f4a6e640 1 --2- 172.21.15.152:0/276791001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f0100e20 0x7f15f01984c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f15e400c920 tx=0x7f15e400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:56.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.662+0000 7f15edffb640 1 -- 172.21.15.152:0/276791001 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4007c70 con 0x7f15f0100e20 2024-09-11T23:01:56.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.662+0000 7f15f6cd2640 1 -- 172.21.15.152:0/276791001 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f019d760 con 0x7f15f0100e20 2024-09-11T23:01:56.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.662+0000 7f15edffb640 1 -- 172.21.15.152:0/276791001 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15e4007e10 con 0x7f15f0100e20 2024-09-11T23:01:56.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.662+0000 7f15edffb640 1 -- 172.21.15.152:0/276791001 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4018890 con 0x7f15f0100e20 2024-09-11T23:01:56.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.662+0000 7f15f6cd2640 1 -- 172.21.15.152:0/276791001 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f01a5040 con 0x7f15f0100e20 2024-09-11T23:01:56.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.664+0000 7f15edffb640 1 -- 172.21.15.152:0/276791001 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f15e4018a70 con 0x7f15f0100e20 2024-09-11T23:01:56.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.664+0000 7f15f90ab740 1 Processor -- start 2024-09-11T23:01:56.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15f90ab740 1 -- start start 2024-09-11T23:01:56.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15f90ab740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f066bb60 0x55d5f066bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15f90ab740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f066c480 0x55d5f0676800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15f90ab740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f0676d40 0x55d5f0679130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15f90ab740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d5f05d1280 con 0x55d5f0676d40 2024-09-11T23:01:56.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15f90ab740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d5f0578b10 con 0x55d5f066c480 2024-09-11T23:01:56.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15f90ab740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d5f05cd240 con 0x55d5f066bb60 2024-09-11T23:01:56.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15edffb640 1 --2- 172.21.15.152:0/276791001 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f15e00a2cf0 0x7f15e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15effff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f0676d40 0x55d5f0679130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15effff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f0676d40 0x55d5f0679130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37102/0 (socket says 172.21.15.152:37102) 2024-09-11T23:01:56.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15effff640 1 -- 172.21.15.152:0/4171407941 learned_addr learned my addr 172.21.15.152:0/4171407941 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:56.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15f526f640 1 --2- 172.21.15.152:0/4171407941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f066bb60 0x55d5f066bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15f4a6e640 1 --2- 172.21.15.152:0/4171407941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f066c480 0x55d5f0676800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15f526f640 1 --2- 172.21.15.152:0/276791001 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f15e00a2cf0 0x7f15e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15edffb640 1 -- 172.21.15.152:0/276791001 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f15e40cf580 con 0x7f15f0100e20 2024-09-11T23:01:56.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15effff640 1 -- 172.21.15.152:0/4171407941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f066bb60 msgr2=0x55d5f066bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:56.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15effff640 1 --2- 172.21.15.152:0/4171407941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f066bb60 0x55d5f066bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15effff640 1 -- 172.21.15.152:0/4171407941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f066c480 msgr2=0x55d5f0676800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:56.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15effff640 1 --2- 172.21.15.152:0/4171407941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f066c480 0x55d5f0676800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15effff640 1 -- 172.21.15.152:0/4171407941 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5f05851f0 con 0x55d5f0676d40 2024-09-11T23:01:56.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.665+0000 7f15effff640 1 --2- 172.21.15.152:0/4171407941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f0676d40 0x55d5f0679130 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009900 tx=0x7f15dc004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2ead1e52171317 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:56.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.666+0000 7f15f526f640 1 --2- 172.21.15.152:0/276791001 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f15e00a2cf0 0x7f15e00a51b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f15e80070b0 tx=0x7f15e80093d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:56.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.666+0000 7f15daffd640 1 -- 172.21.15.152:0/4171407941 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc035030 con 0x55d5f0676d40 2024-09-11T23:01:56.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.666+0000 7f15daffd640 1 -- 172.21.15.152:0/4171407941 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15dc040070 con 0x55d5f0676d40 2024-09-11T23:01:56.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.666+0000 7f15daffd640 1 -- 172.21.15.152:0/4171407941 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc031bb0 con 0x55d5f0676d40 2024-09-11T23:01:56.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.666+0000 7f15f90ab740 1 -- 172.21.15.152:0/4171407941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f0676d40 msgr2=0x55d5f0679130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:56.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.666+0000 7f15f90ab740 1 --2- 172.21.15.152:0/4171407941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f0676d40 0x55d5f0679130 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009900 tx=0x7f15dc004200 comp rx=0 tx=0).stop 2024-09-11T23:01:56.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.666+0000 7f15f90ab740 1 -- 172.21.15.152:0/4171407941 shutdown_connections 2024-09-11T23:01:56.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.666+0000 7f15f90ab740 1 --2- 172.21.15.152:0/4171407941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f0676d40 0x55d5f0679130 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.666+0000 7f15f90ab740 1 --2- 172.21.15.152:0/4171407941 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f066c480 0x55d5f0676800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.666+0000 7f15f90ab740 1 --2- 172.21.15.152:0/4171407941 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f066bb60 0x55d5f066bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.666+0000 7f15f90ab740 1 -- 172.21.15.152:0/4171407941 >> 172.21.15.152:0/4171407941 conn(0x55d5f06680f0 msgr2=0x55d5f06684d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:56.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.666+0000 7f15f90ab740 1 -- 172.21.15.152:0/4171407941 shutdown_connections 2024-09-11T23:01:56.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.666+0000 7f15f90ab740 1 -- 172.21.15.152:0/4171407941 wait complete. 2024-09-11T23:01:56.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.667+0000 7f15f90ab740 1 -- start start 2024-09-11T23:01:56.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.667+0000 7f15f90ab740 1 Processor -- start 2024-09-11T23:01:56.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.667+0000 7f15f90ab740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f066bb60 0x55d5f0673c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.667+0000 7f15f90ab740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f066c480 0x55d5f0674170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.667+0000 7f15f90ab740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f0676d40 0x55d5f06746b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.667+0000 7f15f90ab740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55d5f0679910 con 0x55d5f066bb60 2024-09-11T23:01:56.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.667+0000 7f15f90ab740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55d5f058bf10 con 0x55d5f066c480 2024-09-11T23:01:56.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.667+0000 7f15f90ab740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55d5f0679a90 con 0x55d5f0676d40 2024-09-11T23:01:56.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.667+0000 7f15f90ab740 10 client.0 _ll_register_callbacks cb 0x55d5f0554a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:56.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15f4a6e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f066bb60 0x55d5f0673c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15f4a6e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f066bb60 0x55d5f0673c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37112/0 (socket says 172.21.15.152:37112) 2024-09-11T23:01:56.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15f4a6e640 1 -- 172.21.15.152:0/1253308144 learned_addr learned my addr 172.21.15.152:0/1253308144 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:56.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15effff640 1 --2- 172.21.15.152:0/1253308144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f066c480 0x55d5f0674170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15f526f640 1 --2- 172.21.15.152:0/1253308144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f0676d40 0x55d5f06746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15f4a6e640 1 -- 172.21.15.152:0/1253308144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f0676d40 msgr2=0x55d5f06746b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:56.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15f4a6e640 1 --2- 172.21.15.152:0/1253308144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f0676d40 0x55d5f06746b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15f4a6e640 1 -- 172.21.15.152:0/1253308144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f066c480 msgr2=0x55d5f0674170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:56.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15f4a6e640 1 --2- 172.21.15.152:0/1253308144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f066c480 0x55d5f0674170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15f4a6e640 1 -- 172.21.15.152:0/1253308144 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5f03426a0 con 0x55d5f066bb60 2024-09-11T23:01:56.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15effff640 1 --2- 172.21.15.152:0/1253308144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f066c480 0x55d5f0674170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:01:56.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15f4a6e640 1 --2- 172.21.15.152:0/1253308144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f066bb60 0x55d5f0673c30 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f15e4098490 tx=0x7f15e4066ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e888a62ea48d9599 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:56.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15cffff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:56.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15cffff640 1 -- 172.21.15.152:0/1253308144 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e409bac0 con 0x55d5f066bb60 2024-09-11T23:01:56.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15cffff640 1 -- 172.21.15.152:0/1253308144 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f15e4098130 con 0x55d5f066bb60 2024-09-11T23:01:56.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15f90ab740 1 -- 172.21.15.152:0/1253308144 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15dc040070 con 0x55d5f066bb60 2024-09-11T23:01:56.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15cffff640 1 -- 172.21.15.152:0/1253308144 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15e409fe70 con 0x55d5f066bb60 2024-09-11T23:01:56.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15f90ab740 10 client.17964 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:56.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15f90ab740 1 -- 172.21.15.152:0/1253308144 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d5f0685520 con 0x55d5f066bb60 2024-09-11T23:01:56.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.668+0000 7f15cffff640 1 -- 172.21.15.152:0/1253308144 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15dc040070 con 0x55d5f066bb60 2024-09-11T23:01:56.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.669+0000 7f15cffff640 1 -- 172.21.15.152:0/1253308144 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f15e40a42a0 con 0x55d5f066bb60 2024-09-11T23:01:56.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.669+0000 7f15f90ab740 10 client.17964 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:56.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.669+0000 7f15f90ab740 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:01:56.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.669+0000 7f15f90ab740 20 client.17964 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:56.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.669+0000 7f15f90ab740 1 --2- 172.21.15.152:0/1253308144 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d5f06869e0 0x55d5f0688e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.669+0000 7f15f90ab740 4 client.17964 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:01:56.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.669+0000 7f15f90ab740 1 -- 172.21.15.152:0/1253308144 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5f05de7f0 con 0x55d5f06869e0 2024-09-11T23:01:56.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.669+0000 7f15effff640 1 --2- 172.21.15.152:0/1253308144 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d5f06869e0 0x55d5f0688e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.669+0000 7f15cffff640 10 client.17964 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:56.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.669+0000 7f15effff640 1 --2- 172.21.15.152:0/1253308144 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d5f06869e0 0x55d5f0688e50 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeb712702f5d64c7 server_cookie=923d6e6f35471aa2 in_seq=0 out_seq=0 2024-09-11T23:01:56.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.670+0000 7f15cffff640 1 -- 172.21.15.152:0/1253308144 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5f05de7f0 con 0x55d5f06869e0 2024-09-11T23:01:56.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.670+0000 7f15cffff640 10 client.17964 handle_command_reply: tid=0 2024-09-11T23:01:56.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.670+0000 7f15f90ab740 1 client.17964 shutdown 2024-09-11T23:01:56.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.670+0000 7f15f90ab740 1 -- 172.21.15.152:0/1253308144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f066bb60 msgr2=0x55d5f0673c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:56.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.670+0000 7f15f90ab740 1 --2- 172.21.15.152:0/1253308144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f066bb60 0x55d5f0673c30 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f15e4098490 tx=0x7f15e4066ae0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.670+0000 7f15f90ab740 1 -- 172.21.15.152:0/1253308144 shutdown_connections 2024-09-11T23:01:56.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.670+0000 7f15f90ab740 1 --2- 172.21.15.152:0/1253308144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f0676d40 0x55d5f06746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.670+0000 7f15f90ab740 1 --2- 172.21.15.152:0/1253308144 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d5f06869e0 0x55d5f0688e50 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.670+0000 7f15f90ab740 1 --2- 172.21.15.152:0/1253308144 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f066c480 0x55d5f0674170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.670+0000 7f15f90ab740 1 --2- 172.21.15.152:0/1253308144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f066bb60 0x55d5f0673c30 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.670+0000 7f15cffff640 0 client.17964 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:56.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.670+0000 7f15f90ab740 1 -- 172.21.15.152:0/1253308144 >> 172.21.15.152:0/1253308144 conn(0x55d5f06680f0 msgr2=0x55d5f066ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:56.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.670+0000 7f15f90ab740 1 -- 172.21.15.152:0/1253308144 shutdown_connections 2024-09-11T23:01:56.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.670+0000 7f15f90ab740 1 -- 172.21.15.152:0/1253308144 wait complete. 2024-09-11T23:01:56.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.671+0000 7f15f90ab740 20 client.17964 trim_cache size 0 max 16384 2024-09-11T23:01:56.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.688+0000 7f15f90ab740 1 Processor -- start 2024-09-11T23:01:56.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.688+0000 7f15f90ab740 1 -- start start 2024-09-11T23:01:56.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.688+0000 7f15f90ab740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f066bb60 0x55d5f066dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.688+0000 7f15f90ab740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f066c480 0x55d5f068e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.688+0000 7f15f90ab740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f0676d40 0x55d5f068ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.688+0000 7f15f90ab740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f15f01ad750 con 0x55d5f0676d40 2024-09-11T23:01:56.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.688+0000 7f15f90ab740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f15f01af290 con 0x55d5f066bb60 2024-09-11T23:01:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.688+0000 7f15f90ab740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f15f01ade20 con 0x55d5f066c480 2024-09-11T23:01:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.688+0000 7f15f526f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f0676d40 0x55d5f068ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.688+0000 7f15f526f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f0676d40 0x55d5f068ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37126/0 (socket says 172.21.15.152:37126) 2024-09-11T23:01:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.688+0000 7f15f526f640 1 -- 172.21.15.152:0/2959886576 learned_addr learned my addr 172.21.15.152:0/2959886576 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.689+0000 7f15effff640 1 --2- 172.21.15.152:0/2959886576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f066c480 0x55d5f068e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.689+0000 7f15f4a6e640 1 --2- 172.21.15.152:0/2959886576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f066bb60 0x55d5f066dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.689+0000 7f15f526f640 1 -- 172.21.15.152:0/2959886576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f066c480 msgr2=0x55d5f068e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.689+0000 7f15f526f640 1 --2- 172.21.15.152:0/2959886576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f066c480 0x55d5f068e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.689+0000 7f15f526f640 1 -- 172.21.15.152:0/2959886576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f066bb60 msgr2=0x55d5f066dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.689+0000 7f15f526f640 1 --2- 172.21.15.152:0/2959886576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f066bb60 0x55d5f066dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.689+0000 7f15f526f640 1 -- 172.21.15.152:0/2959886576 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5f0667f20 con 0x55d5f0676d40 2024-09-11T23:01:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.689+0000 7f15f526f640 1 --2- 172.21.15.152:0/2959886576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f0676d40 0x55d5f068ec10 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009b70 tx=0x7f15e8009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba3821be8d8b091 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.689+0000 7f15da7fc640 1 -- 172.21.15.152:0/2959886576 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e802a070 con 0x55d5f0676d40 2024-09-11T23:01:56.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.689+0000 7f15da7fc640 1 -- 172.21.15.152:0/2959886576 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15e8019dd0 con 0x55d5f0676d40 2024-09-11T23:01:56.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.689+0000 7f15da7fc640 1 -- 172.21.15.152:0/2959886576 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8013040 con 0x55d5f0676d40 2024-09-11T23:01:56.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.690+0000 7f15f90ab740 1 -- 172.21.15.152:0/2959886576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f0676d40 msgr2=0x55d5f068ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:56.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.690+0000 7f15f90ab740 1 --2- 172.21.15.152:0/2959886576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f0676d40 0x55d5f068ec10 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009b70 tx=0x7f15e8009700 comp rx=0 tx=0).stop 2024-09-11T23:01:56.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.690+0000 7f15f90ab740 1 -- 172.21.15.152:0/2959886576 shutdown_connections 2024-09-11T23:01:56.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.690+0000 7f15f90ab740 1 --2- 172.21.15.152:0/2959886576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f0676d40 0x55d5f068ec10 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.690+0000 7f15f90ab740 1 --2- 172.21.15.152:0/2959886576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f066c480 0x55d5f068e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.690+0000 7f15f90ab740 1 --2- 172.21.15.152:0/2959886576 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f066bb60 0x55d5f066dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.690+0000 7f15f90ab740 1 -- 172.21.15.152:0/2959886576 >> 172.21.15.152:0/2959886576 conn(0x55d5f06680f0 msgr2=0x55d5f066d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:56.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.690+0000 7f15f90ab740 1 -- 172.21.15.152:0/2959886576 shutdown_connections 2024-09-11T23:01:56.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.690+0000 7f15f90ab740 1 -- 172.21.15.152:0/2959886576 wait complete. 2024-09-11T23:01:56.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.690+0000 7f15f90ab740 1 -- start start 2024-09-11T23:01:56.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.690+0000 7f15f90ab740 1 Processor -- start 2024-09-11T23:01:56.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f90ab740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f066bb60 0x55d5f067b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.702 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f90ab740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f066c480 0x55d5f067b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f90ab740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f0676d40 0x55d5f06f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f90ab740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f15f01aebc0 con 0x55d5f066bb60 2024-09-11T23:01:56.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f90ab740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f15f01ae4f0 con 0x55d5f0676d40 2024-09-11T23:01:56.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f90ab740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f15f01ab750 con 0x55d5f066c480 2024-09-11T23:01:56.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f4a6e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f066bb60 0x55d5f067b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f90ab740 10 client.0 _ll_register_callbacks cb 0x7f15e4097d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:56.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f4a6e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f066bb60 0x55d5f067b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37138/0 (socket says 172.21.15.152:37138) 2024-09-11T23:01:56.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f4a6e640 1 -- 172.21.15.152:0/779924150 learned_addr learned my addr 172.21.15.152:0/779924150 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:56.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15effff640 1 --2- 172.21.15.152:0/779924150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f066c480 0x55d5f067b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f526f640 1 --2- 172.21.15.152:0/779924150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f0676d40 0x55d5f06f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f4a6e640 1 -- 172.21.15.152:0/779924150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f066c480 msgr2=0x55d5f067b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:56.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f4a6e640 1 --2- 172.21.15.152:0/779924150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f066c480 0x55d5f067b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f4a6e640 1 -- 172.21.15.152:0/779924150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f0676d40 msgr2=0x55d5f06f4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:56.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f4a6e640 1 --2- 172.21.15.152:0/779924150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f0676d40 0x55d5f06f4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f4a6e640 1 -- 172.21.15.152:0/779924150 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5f06f50a0 con 0x55d5f066bb60 2024-09-11T23:01:56.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15f4a6e640 1 --2- 172.21.15.152:0/779924150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f066bb60 0x55d5f067b220 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f15e40cd2f0 tx=0x7f15e40cf220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906c78d2c13145ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:56.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15cffff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:56.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15cffff640 1 -- 172.21.15.152:0/779924150 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e409bac0 con 0x55d5f066bb60 2024-09-11T23:01:56.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.691+0000 7f15cffff640 1 -- 172.21.15.152:0/779924150 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f15e4098130 con 0x55d5f066bb60 2024-09-11T23:01:56.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.692+0000 7f15f90ab740 1 -- 172.21.15.152:0/779924150 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e8019dd0 con 0x55d5f066bb60 2024-09-11T23:01:56.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.692+0000 7f15cffff640 1 -- 172.21.15.152:0/779924150 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15e40a4730 con 0x55d5f066bb60 2024-09-11T23:01:56.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.692+0000 7f15f90ab740 10 client.17970 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:56.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.692+0000 7f15f90ab740 1 -- 172.21.15.152:0/779924150 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d5f06f63c0 con 0x55d5f066bb60 2024-09-11T23:01:56.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.692+0000 7f15cffff640 1 -- 172.21.15.152:0/779924150 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15e8019dd0 con 0x55d5f066bb60 2024-09-11T23:01:56.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.692+0000 7f15cffff640 1 -- 172.21.15.152:0/779924150 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f15e40662b0 con 0x55d5f066bb60 2024-09-11T23:01:56.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.692+0000 7f15f90ab740 10 client.17970 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:56.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.692+0000 7f15f90ab740 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:01:56.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.692+0000 7f15f90ab740 20 client.17970 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:56.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.692+0000 7f15f90ab740 1 --2- 172.21.15.152:0/779924150 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d5f06f6980 0x55d5f06f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:56.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.692+0000 7f15f90ab740 4 client.17970 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:01:56.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.692+0000 7f15f90ab740 1 -- 172.21.15.152:0/779924150 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d5f05d8d00 con 0x55d5f06f6980 2024-09-11T23:01:56.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.692+0000 7f15effff640 1 --2- 172.21.15.152:0/779924150 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d5f06f6980 0x55d5f06f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:56.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.693+0000 7f15cffff640 10 client.17970 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:56.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.693+0000 7f15effff640 1 --2- 172.21.15.152:0/779924150 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d5f06f6980 0x55d5f06f8da0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79aa04a5ab880f07 server_cookie=8ea0f92f1a0d726e in_seq=0 out_seq=0 2024-09-11T23:01:56.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.694+0000 7f15cffff640 1 -- 172.21.15.152:0/779924150 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d5f05d8d00 con 0x55d5f06f6980 2024-09-11T23:01:56.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.694+0000 7f15cffff640 10 client.17970 handle_command_reply: tid=0 2024-09-11T23:01:56.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.694+0000 7f15f90ab740 1 client.17970 shutdown 2024-09-11T23:01:56.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.694+0000 7f15f90ab740 1 -- 172.21.15.152:0/779924150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f066bb60 msgr2=0x55d5f067b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:56.705 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.694+0000 7f15f90ab740 1 --2- 172.21.15.152:0/779924150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f066bb60 0x55d5f067b220 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f15e40cd2f0 tx=0x7f15e40cf220 comp rx=0 tx=0).stop 2024-09-11T23:01:56.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.694+0000 7f15f90ab740 1 -- 172.21.15.152:0/779924150 shutdown_connections 2024-09-11T23:01:56.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.694+0000 7f15f90ab740 1 --2- 172.21.15.152:0/779924150 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55d5f06f6980 0x55d5f06f8da0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.694+0000 7f15f90ab740 1 --2- 172.21.15.152:0/779924150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55d5f0676d40 0x55d5f06f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.694+0000 7f15cffff640 0 client.17970 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:56.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.694+0000 7f15f90ab740 1 --2- 172.21.15.152:0/779924150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d5f066c480 0x55d5f067b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.694+0000 7f15f90ab740 1 --2- 172.21.15.152:0/779924150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d5f066bb60 0x55d5f067b220 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.694+0000 7f15f90ab740 1 -- 172.21.15.152:0/779924150 >> 172.21.15.152:0/779924150 conn(0x55d5f06680f0 msgr2=0x55d5f0674410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:56.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.694+0000 7f15f90ab740 1 -- 172.21.15.152:0/779924150 shutdown_connections 2024-09-11T23:01:56.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.694+0000 7f15f90ab740 1 -- 172.21.15.152:0/779924150 wait complete. 2024-09-11T23:01:56.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.694+0000 7f15f90ab740 20 client.17970 trim_cache size 0 max 16384 2024-09-11T23:01:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:01:56.707 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T23:01:56.707 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "a21c6891-9bf4-41bc-a988-38a0e08d648e", 2024-09-11T23:01:56.707 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T23:01:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:01:56.707 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.695+0000 7f15f6cd2640 1 -- 172.21.15.152:0/276791001 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f15e00a2cf0 msgr2=0x7f15e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:56.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.695+0000 7f15f6cd2640 1 --2- 172.21.15.152:0/276791001 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f15e00a2cf0 0x7f15e00a51b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f15e80070b0 tx=0x7f15e80093d0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.695+0000 7f15f6cd2640 1 -- 172.21.15.152:0/276791001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f0100e20 msgr2=0x7f15f01984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:56.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.695+0000 7f15f6cd2640 1 --2- 172.21.15.152:0/276791001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f0100e20 0x7f15f01984c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f15e400c920 tx=0x7f15e400cde0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.695+0000 7f15f6cd2640 1 -- 172.21.15.152:0/276791001 shutdown_connections 2024-09-11T23:01:56.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.695+0000 7f15f6cd2640 1 --2- 172.21.15.152:0/276791001 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f15e00a2cf0 0x7f15e00a51b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.695+0000 7f15f6cd2640 1 --2- 172.21.15.152:0/276791001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15f0102ef0 0x7f15f019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.695+0000 7f15f6cd2640 1 --2- 172.21.15.152:0/276791001 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f0102570 0x7f15f0198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.695+0000 7f15f6cd2640 1 --2- 172.21.15.152:0/276791001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15f0100e20 0x7f15f01984c0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:56.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.695+0000 7f15f6cd2640 1 -- 172.21.15.152:0/276791001 >> 172.21.15.152:0/276791001 conn(0x7f15f00fc7d0 msgr2=0x7f15f006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:56.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.695+0000 7f15f6cd2640 1 -- 172.21.15.152:0/276791001 shutdown_connections 2024-09-11T23:01:56.708 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:56.695+0000 7f15f6cd2640 1 -- 172.21.15.152:0/276791001 wait complete. 2024-09-11T23:01:56.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:56 smithi152 ceph-mon[31181]: pgmap v278: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:01:57.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:56 smithi191 ceph-mon[35211]: pgmap v278: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:01:57.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:56 smithi204 ceph-mon[35454]: pgmap v278: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:01:57.563 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:01:57.810 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:01:57.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:57 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T23:01:57.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:57 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T23:01:57.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:57 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T23:01:57.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:57 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T23:01:58.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:57 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T23:01:58.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:57 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T23:01:58.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:57 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T23:01:58.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:57 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T23:01:58.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:57 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T23:01:58.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:57 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T23:01:58.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:57 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T23:01:58.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:57 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T23:01:58.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 -- 172.21.15.152:0/1952562004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa044100a80 msgr2=0x7fa044100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:58.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 --2- 172.21.15.152:0/1952562004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa044100a80 0x7fa044100ee0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa030009930 tx=0x7fa03002a420 comp rx=0 tx=0).stop 2024-09-11T23:01:58.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 -- 172.21.15.152:0/1952562004 shutdown_connections 2024-09-11T23:01:58.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 --2- 172.21.15.152:0/1952562004 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa044101420 0x7fa04410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 --2- 172.21.15.152:0/1952562004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa044100a80 0x7fa044100ee0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 --2- 172.21.15.152:0/1952562004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa044106a80 0x7fa044106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 -- 172.21.15.152:0/1952562004 >> 172.21.15.152:0/1952562004 conn(0x7fa0440fc7a0 msgr2=0x7fa0440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:58.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 -- 172.21.15.152:0/1952562004 shutdown_connections 2024-09-11T23:01:58.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 -- 172.21.15.152:0/1952562004 wait complete. 2024-09-11T23:01:58.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 Processor -- start 2024-09-11T23:01:58.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 -- start start 2024-09-11T23:01:58.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa044100a80 0x7fa044199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa044101420 0x7fa044199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa044106a80 0x7fa0441a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa044110e80 con 0x7fa044106a80 2024-09-11T23:01:58.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa044110d00 con 0x7fa044100a80 2024-09-11T23:01:58.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.337+0000 7fa04b0f4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa044111000 con 0x7fa044101420 2024-09-11T23:01:58.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa049691640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa044106a80 0x7fa0441a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa049691640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa044106a80 0x7fa0441a42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37160/0 (socket says 172.21.15.152:37160) 2024-09-11T23:01:58.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa049691640 1 -- 172.21.15.152:0/1823133096 learned_addr learned my addr 172.21.15.152:0/1823133096 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:58.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa048e90640 1 --2- 172.21.15.152:0/1823133096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa044100a80 0x7fa044199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa043fff640 1 --2- 172.21.15.152:0/1823133096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa044101420 0x7fa044199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa049691640 1 -- 172.21.15.152:0/1823133096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa044101420 msgr2=0x7fa044199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:58.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa049691640 1 --2- 172.21.15.152:0/1823133096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa044101420 0x7fa044199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa049691640 1 -- 172.21.15.152:0/1823133096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa044100a80 msgr2=0x7fa044199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:58.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa049691640 1 --2- 172.21.15.152:0/1823133096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa044100a80 0x7fa044199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa049691640 1 -- 172.21.15.152:0/1823133096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0441a47f0 con 0x7fa044106a80 2024-09-11T23:01:58.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa043fff640 1 --2- 172.21.15.152:0/1823133096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa044101420 0x7fa044199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:01:58.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa049691640 1 --2- 172.21.15.152:0/1823133096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa044106a80 0x7fa0441a42b0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00cce0 tx=0x7fa03c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:58.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa041ffb640 1 -- 172.21.15.152:0/1823133096 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c00f450 con 0x7fa044106a80 2024-09-11T23:01:58.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa041ffb640 1 -- 172.21.15.152:0/1823133096 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa03c00fad0 con 0x7fa044106a80 2024-09-11T23:01:58.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa041ffb640 1 -- 172.21.15.152:0/1823133096 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c018c10 con 0x7fa044106a80 2024-09-11T23:01:58.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa04b0f4640 1 -- 172.21.15.152:0/1823133096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0441a49c0 con 0x7fa044106a80 2024-09-11T23:01:58.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.338+0000 7fa04b0f4640 1 -- 172.21.15.152:0/1823133096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0441a4eb0 con 0x7fa044106a80 2024-09-11T23:01:58.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.341+0000 7fa041ffb640 1 -- 172.21.15.152:0/1823133096 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa03c00f5f0 con 0x7fa044106a80 2024-09-11T23:01:58.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.341+0000 7fa04d4cd740 1 Processor -- start 2024-09-11T23:01:58.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa04d4cd740 1 -- start start 2024-09-11T23:01:58.358 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa04d4cd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc84a0 0x562b22bc8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa04d4cd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bc8dc0 0x562b22bd3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa04d4cd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bd5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa04d4cd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x562b22b2e2e0 con 0x562b22bd3680 2024-09-11T23:01:58.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa04d4cd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562b22ad5ce0 con 0x562b22bc84a0 2024-09-11T23:01:58.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa04d4cd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562b22b2a2a0 con 0x562b22bc8dc0 2024-09-11T23:01:58.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa041ffb640 1 --2- 172.21.15.152:0/1823133096 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa0340a2cf0 0x7fa0340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa043fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bd5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa043fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bd5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37170/0 (socket says 172.21.15.152:37170) 2024-09-11T23:01:58.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa043fff640 1 -- 172.21.15.152:0/616361697 learned_addr learned my addr 172.21.15.152:0/616361697 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:58.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa049691640 1 --2- 172.21.15.152:0/1823133096 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa0340a2cf0 0x7fa0340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa048e90640 1 --2- 172.21.15.152:0/616361697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bc8dc0 0x562b22bd3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa043fff640 1 --2- 172.21.15.152:0/616361697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc84a0 0x562b22bc8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa041ffb640 1 -- 172.21.15.152:0/1823133096 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7fa03c0cfbc0 con 0x7fa044106a80 2024-09-11T23:01:58.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa043fff640 1 -- 172.21.15.152:0/616361697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bc8dc0 msgr2=0x562b22bd3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:58.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa043fff640 1 --2- 172.21.15.152:0/616361697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bc8dc0 0x562b22bd3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa043fff640 1 -- 172.21.15.152:0/616361697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc84a0 msgr2=0x562b22bc8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:58.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa043fff640 1 --2- 172.21.15.152:0/616361697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc84a0 0x562b22bc8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.342+0000 7fa043fff640 1 -- 172.21.15.152:0/616361697 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b22aba440 con 0x562b22bd3680 2024-09-11T23:01:58.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.343+0000 7fa049691640 1 --2- 172.21.15.152:0/1823133096 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa0340a2cf0 0x7fa0340a51b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0a4840 tx=0x7fa03c06e0c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:58.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.343+0000 7fa043fff640 1 --2- 172.21.15.152:0/616361697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bd5a70 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fa0300057c0 tx=0x7fa030000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c673f18c3221d285 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:58.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.343+0000 7fa02effd640 1 -- 172.21.15.152:0/616361697 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030035040 con 0x562b22bd3680 2024-09-11T23:01:58.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.343+0000 7fa02effd640 1 -- 172.21.15.152:0/616361697 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0300035f0 con 0x562b22bd3680 2024-09-11T23:01:58.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.343+0000 7fa02effd640 1 -- 172.21.15.152:0/616361697 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03003c650 con 0x562b22bd3680 2024-09-11T23:01:58.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.344+0000 7fa04d4cd740 1 -- 172.21.15.152:0/616361697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 msgr2=0x562b22bd5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:58.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.344+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/616361697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bd5a70 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fa0300057c0 tx=0x7fa030000bc0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.344+0000 7fa04d4cd740 1 -- 172.21.15.152:0/616361697 shutdown_connections 2024-09-11T23:01:58.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.344+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/616361697 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bd5a70 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.344+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/616361697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bc8dc0 0x562b22bd3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.344+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/616361697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc84a0 0x562b22bc8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.344+0000 7fa04d4cd740 1 -- 172.21.15.152:0/616361697 >> 172.21.15.152:0/616361697 conn(0x562b22bc4a30 msgr2=0x562b22bc4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:58.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.344+0000 7fa04d4cd740 1 -- 172.21.15.152:0/616361697 shutdown_connections 2024-09-11T23:01:58.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.344+0000 7fa04d4cd740 1 -- 172.21.15.152:0/616361697 wait complete. 2024-09-11T23:01:58.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.344+0000 7fa04d4cd740 1 -- start start 2024-09-11T23:01:58.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.344+0000 7fa04d4cd740 1 Processor -- start 2024-09-11T23:01:58.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.344+0000 7fa04d4cd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc84a0 0x562b22bd3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa04d4cd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bc8dc0 0x562b22bd02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa04d4cd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bd0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa04d4cd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x562b22bd6250 con 0x562b22bd3680 2024-09-11T23:01:58.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa04d4cd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562b22ae9050 con 0x562b22bc84a0 2024-09-11T23:01:58.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa04d4cd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562b22bd63d0 con 0x562b22bc8dc0 2024-09-11T23:01:58.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa04d4cd740 10 client.0 _ll_register_callbacks cb 0x562b22b29160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:58.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa043fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bd0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa043fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bd0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37182/0 (socket says 172.21.15.152:37182) 2024-09-11T23:01:58.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa043fff640 1 -- 172.21.15.152:0/2350011163 learned_addr learned my addr 172.21.15.152:0/2350011163 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:58.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa048e90640 1 --2- 172.21.15.152:0/2350011163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bc8dc0 0x562b22bd02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa043fff640 1 --2- 172.21.15.152:0/2350011163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc84a0 0x562b22bd3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa043fff640 1 -- 172.21.15.152:0/2350011163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bc8dc0 msgr2=0x562b22bd02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:58.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa043fff640 1 --2- 172.21.15.152:0/2350011163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bc8dc0 0x562b22bd02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa043fff640 1 -- 172.21.15.152:0/2350011163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc84a0 msgr2=0x562b22bd3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:58.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa043fff640 1 --2- 172.21.15.152:0/2350011163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc84a0 0x562b22bd3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa043fff640 1 -- 172.21.15.152:0/2350011163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b2289f3c0 con 0x562b22bd3680 2024-09-11T23:01:58.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa043fff640 1 --2- 172.21.15.152:0/2350011163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bd0830 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fa030032fd0 tx=0x7fa03002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772430bf7569d84f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:58.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa023fff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:58.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa023fff640 1 -- 172.21.15.152:0/2350011163 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030035030 con 0x562b22bd3680 2024-09-11T23:01:58.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa023fff640 1 -- 172.21.15.152:0/2350011163 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7fa030004000 con 0x562b22bd3680 2024-09-11T23:01:58.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.345+0000 7fa04d4cd740 1 -- 172.21.15.152:0/2350011163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0300035f0 con 0x562b22bd3680 2024-09-11T23:01:58.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.346+0000 7fa023fff640 1 -- 172.21.15.152:0/2350011163 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa030003bb0 con 0x562b22bd3680 2024-09-11T23:01:58.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.346+0000 7fa04d4cd740 10 client.17982 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:58.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.346+0000 7fa04d4cd740 1 -- 172.21.15.152:0/2350011163 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562b22be1c70 con 0x562b22bd3680 2024-09-11T23:01:58.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.346+0000 7fa023fff640 1 -- 172.21.15.152:0/2350011163 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0300035f0 con 0x562b22bd3680 2024-09-11T23:01:58.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.346+0000 7fa023fff640 1 -- 172.21.15.152:0/2350011163 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa030054030 con 0x562b22bd3680 2024-09-11T23:01:58.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.346+0000 7fa04d4cd740 10 client.17982 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:58.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.346+0000 7fa04d4cd740 10 client.17982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:01:58.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.346+0000 7fa04d4cd740 20 client.17982 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:58.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.346+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/2350011163 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562b22be2fc0 0x562b22be53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.346+0000 7fa04d4cd740 4 client.17982 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:01:58.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.346+0000 7fa04d4cd740 1 -- 172.21.15.152:0/2350011163 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b22b3b850 con 0x562b22be2fc0 2024-09-11T23:01:58.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.346+0000 7fa048e90640 1 --2- 172.21.15.152:0/2350011163 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562b22be2fc0 0x562b22be53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.347+0000 7fa023fff640 10 client.17982 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:58.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.347+0000 7fa048e90640 1 --2- 172.21.15.152:0/2350011163 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562b22be2fc0 0x562b22be53b0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15281e1f2502e950 server_cookie=cc00f72f76c9ef9 in_seq=0 out_seq=0 2024-09-11T23:01:58.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.347+0000 7fa023fff640 1 -- 172.21.15.152:0/2350011163 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b22b3b850 con 0x562b22be2fc0 2024-09-11T23:01:58.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.347+0000 7fa023fff640 10 client.17982 handle_command_reply: tid=0 2024-09-11T23:01:58.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.347+0000 7fa04d4cd740 1 client.17982 shutdown 2024-09-11T23:01:58.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.347+0000 7fa04d4cd740 1 -- 172.21.15.152:0/2350011163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 msgr2=0x562b22bd0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:58.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.347+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/2350011163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bd0830 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fa030032fd0 tx=0x7fa03002a420 comp rx=0 tx=0).stop 2024-09-11T23:01:58.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.348+0000 7fa04d4cd740 1 -- 172.21.15.152:0/2350011163 shutdown_connections 2024-09-11T23:01:58.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.348+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/2350011163 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bd0830 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.348+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/2350011163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bc8dc0 0x562b22bd02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.348+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/2350011163 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562b22be2fc0 0x562b22be53b0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.348+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/2350011163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc84a0 0x562b22bd3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.348+0000 7fa04d4cd740 1 -- 172.21.15.152:0/2350011163 >> 172.21.15.152:0/2350011163 conn(0x562b22bc4a30 msgr2=0x562b22bca710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:58.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.348+0000 7fa023fff640 0 client.17982 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:58.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.348+0000 7fa04d4cd740 1 -- 172.21.15.152:0/2350011163 shutdown_connections 2024-09-11T23:01:58.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.348+0000 7fa04d4cd740 1 -- 172.21.15.152:0/2350011163 wait complete. 2024-09-11T23:01:58.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.348+0000 7fa04d4cd740 20 client.17982 trim_cache size 0 max 16384 2024-09-11T23:01:58.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa04d4cd740 1 Processor -- start 2024-09-11T23:01:58.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa04d4cd740 1 -- start start 2024-09-11T23:01:58.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa04d4cd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bc84a0 0x562b22be7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa04d4cd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc8dc0 0x562b22be8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa04d4cd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bcd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa04d4cd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa0441ad7c0 con 0x562b22bd3680 2024-09-11T23:01:58.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa04d4cd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa0441af300 con 0x562b22bc8dc0 2024-09-11T23:01:58.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa04d4cd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa0441ade90 con 0x562b22bc84a0 2024-09-11T23:01:58.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa043fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bcd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa043fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bcd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37190/0 (socket says 172.21.15.152:37190) 2024-09-11T23:01:58.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa043fff640 1 -- 172.21.15.152:0/1007876587 learned_addr learned my addr 172.21.15.152:0/1007876587 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:58.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa043fff640 1 --2- 172.21.15.152:0/1007876587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bc84a0 0x562b22be7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa048e90640 1 --2- 172.21.15.152:0/1007876587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc8dc0 0x562b22be8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa043fff640 1 -- 172.21.15.152:0/1007876587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bc84a0 msgr2=0x562b22be7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:58.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa043fff640 1 --2- 172.21.15.152:0/1007876587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bc84a0 0x562b22be7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa043fff640 1 -- 172.21.15.152:0/1007876587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc8dc0 msgr2=0x562b22be8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:58.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa043fff640 1 --2- 172.21.15.152:0/1007876587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc8dc0 0x562b22be8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.365+0000 7fa043fff640 1 -- 172.21.15.152:0/1007876587 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b22bc8bf0 con 0x562b22bd3680 2024-09-11T23:01:58.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.366+0000 7fa043fff640 1 --2- 172.21.15.152:0/1007876587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bcd510 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fa03004df10 tx=0x7fa030005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acd0619e963d121 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:58.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.366+0000 7fa02e7fc640 1 -- 172.21.15.152:0/1007876587 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030035040 con 0x562b22bd3680 2024-09-11T23:01:58.368 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.366+0000 7fa02e7fc640 1 -- 172.21.15.152:0/1007876587 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0300035f0 con 0x562b22bd3680 2024-09-11T23:01:58.369 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.366+0000 7fa02e7fc640 1 -- 172.21.15.152:0/1007876587 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03003cae0 con 0x562b22bd3680 2024-09-11T23:01:58.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.366+0000 7fa04d4cd740 1 -- 172.21.15.152:0/1007876587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 msgr2=0x562b22bcd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:58.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.366+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/1007876587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bcd510 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fa03004df10 tx=0x7fa030005fd0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.367+0000 7fa04d4cd740 1 -- 172.21.15.152:0/1007876587 shutdown_connections 2024-09-11T23:01:58.372 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.367+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/1007876587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bd3680 0x562b22bcd510 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.367+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/1007876587 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc8dc0 0x562b22be8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.367+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/1007876587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bc84a0 0x562b22be7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.367+0000 7fa04d4cd740 1 -- 172.21.15.152:0/1007876587 >> 172.21.15.152:0/1007876587 conn(0x562b22bc4a30 msgr2=0x562b22bcf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:58.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.367+0000 7fa04d4cd740 1 -- 172.21.15.152:0/1007876587 shutdown_connections 2024-09-11T23:01:58.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.367+0000 7fa04d4cd740 1 -- 172.21.15.152:0/1007876587 wait complete. 2024-09-11T23:01:58.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.367+0000 7fa04d4cd740 1 -- start start 2024-09-11T23:01:58.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.367+0000 7fa04d4cd740 1 Processor -- start 2024-09-11T23:01:58.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa04d4cd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bc84a0 0x562b22be85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa04d4cd740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc8dc0 0x562b22be5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa04d4cd740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bd3680 0x562b22be5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa04d4cd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa0441aec30 con 0x562b22bc84a0 2024-09-11T23:01:58.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa04d4cd740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa0441ae560 con 0x562b22bc8dc0 2024-09-11T23:01:58.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa04d4cd740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa0441ab7c0 con 0x562b22bd3680 2024-09-11T23:01:58.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa04d4cd740 10 client.0 _ll_register_callbacks cb 0x7fa030032cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:01:58.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa043fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bc84a0 0x562b22be85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa043fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bc84a0 0x562b22be85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:37196/0 (socket says 172.21.15.152:37196) 2024-09-11T23:01:58.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa043fff640 1 -- 172.21.15.152:0/3595941897 learned_addr learned my addr 172.21.15.152:0/3595941897 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:01:58.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa048e90640 1 --2- 172.21.15.152:0/3595941897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc8dc0 0x562b22be5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa043fff640 1 --2- 172.21.15.152:0/3595941897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bd3680 0x562b22be5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa043fff640 1 -- 172.21.15.152:0/3595941897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bd3680 msgr2=0x562b22be5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:58.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa043fff640 1 --2- 172.21.15.152:0/3595941897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bd3680 0x562b22be5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa043fff640 1 -- 172.21.15.152:0/3595941897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc8dc0 msgr2=0x562b22be5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:01:58.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa043fff640 1 --2- 172.21.15.152:0/3595941897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc8dc0 0x562b22be5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa043fff640 1 -- 172.21.15.152:0/3595941897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b22be64b0 con 0x562b22bc84a0 2024-09-11T23:01:58.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa043fff640 1 --2- 172.21.15.152:0/3595941897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bc84a0 0x562b22be85b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fa03003c7c0 tx=0x7fa03003c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace1c285ffcf8b42 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:01:58.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa023fff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:01:58.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa023fff640 1 -- 172.21.15.152:0/3595941897 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030066020 con 0x562b22bc84a0 2024-09-11T23:01:58.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa023fff640 1 -- 172.21.15.152:0/3595941897 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7fa03003dcb0 con 0x562b22bc84a0 2024-09-11T23:01:58.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.368+0000 7fa04d4cd740 1 -- 172.21.15.152:0/3595941897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0300035f0 con 0x562b22bc84a0 2024-09-11T23:01:58.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.369+0000 7fa023fff640 1 -- 172.21.15.152:0/3595941897 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa030035040 con 0x562b22bc84a0 2024-09-11T23:01:58.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.369+0000 7fa04d4cd740 10 client.17988 fetch_fsmap learned FSMap version 28 2024-09-11T23:01:58.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.369+0000 7fa04d4cd740 1 -- 172.21.15.152:0/3595941897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562b22c4d780 con 0x562b22bc84a0 2024-09-11T23:01:58.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.369+0000 7fa023fff640 1 -- 172.21.15.152:0/3595941897 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0300035f0 con 0x562b22bc84a0 2024-09-11T23:01:58.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.369+0000 7fa023fff640 1 -- 172.21.15.152:0/3595941897 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa03003a040 con 0x562b22bc84a0 2024-09-11T23:01:58.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.369+0000 7fa04d4cd740 10 client.17988 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:01:58.375 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.369+0000 7fa04d4cd740 10 client.17988 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:01:58.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.369+0000 7fa04d4cd740 20 client.17988 populate_metadata read hostname 'smithi152' 2024-09-11T23:01:58.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.369+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/3595941897 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562b22c4dce0 0x562b22c500d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:01:58.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.369+0000 7fa04d4cd740 4 client.17988 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:01:58.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.369+0000 7fa04d4cd740 1 -- 172.21.15.152:0/3595941897 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562b22b35d60 con 0x562b22c4dce0 2024-09-11T23:01:58.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.369+0000 7fa048e90640 1 --2- 172.21.15.152:0/3595941897 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562b22c4dce0 0x562b22c500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:01:58.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa023fff640 10 client.17988 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:58.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa048e90640 1 --2- 172.21.15.152:0/3595941897 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562b22c4dce0 0x562b22c500d0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806ae59bd267c41 server_cookie=fe618a4471f6dc5e in_seq=0 out_seq=0 2024-09-11T23:01:58.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa023fff640 1 -- 172.21.15.152:0/3595941897 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562b22b35d60 con 0x562b22c4dce0 2024-09-11T23:01:58.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa023fff640 10 client.17988 handle_command_reply: tid=0 2024-09-11T23:01:58.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa04d4cd740 1 client.17988 shutdown 2024-09-11T23:01:58.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa04d4cd740 1 -- 172.21.15.152:0/3595941897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bc84a0 msgr2=0x562b22be85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:58.376 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/3595941897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bc84a0 0x562b22be85b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fa03003c7c0 tx=0x7fa03003c4e0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa04d4cd740 1 -- 172.21.15.152:0/3595941897 shutdown_connections 2024-09-11T23:01:58.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/3595941897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b22bd3680 0x562b22be5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/3595941897 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x562b22c4dce0 0x562b22c500d0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/3595941897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562b22bc8dc0 0x562b22be5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa023fff640 0 client.17988 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:01:58.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa04d4cd740 1 --2- 172.21.15.152:0/3595941897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562b22bc84a0 0x562b22be85b0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa04d4cd740 1 -- 172.21.15.152:0/3595941897 >> 172.21.15.152:0/3595941897 conn(0x562b22bc4a30 msgr2=0x562b22bc3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:58.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa04d4cd740 1 -- 172.21.15.152:0/3595941897 shutdown_connections 2024-09-11T23:01:58.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa04d4cd740 1 -- 172.21.15.152:0/3595941897 wait complete. 2024-09-11T23:01:58.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.370+0000 7fa04d4cd740 20 client.17988 trim_cache size 0 max 16384 2024-09-11T23:01:58.377 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:01:58.378 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T23:01:58.378 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T23:01:58.378 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:01:58.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.371+0000 7fa04b0f4640 1 -- 172.21.15.152:0/1823133096 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa0340a2cf0 msgr2=0x7fa0340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:58.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.371+0000 7fa04b0f4640 1 --2- 172.21.15.152:0/1823133096 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa0340a2cf0 0x7fa0340a51b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0a4840 tx=0x7fa03c06e0c0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.371+0000 7fa04b0f4640 1 -- 172.21.15.152:0/1823133096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa044106a80 msgr2=0x7fa0441a42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:01:58.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.371+0000 7fa04b0f4640 1 --2- 172.21.15.152:0/1823133096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa044106a80 0x7fa0441a42b0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00cce0 tx=0x7fa03c00a620 comp rx=0 tx=0).stop 2024-09-11T23:01:58.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.371+0000 7fa04b0f4640 1 -- 172.21.15.152:0/1823133096 shutdown_connections 2024-09-11T23:01:58.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.371+0000 7fa04b0f4640 1 --2- 172.21.15.152:0/1823133096 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa0340a2cf0 0x7fa0340a51b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.371+0000 7fa04b0f4640 1 --2- 172.21.15.152:0/1823133096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa044106a80 0x7fa0441a42b0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.371+0000 7fa04b0f4640 1 --2- 172.21.15.152:0/1823133096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa044101420 0x7fa044199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.371+0000 7fa04b0f4640 1 --2- 172.21.15.152:0/1823133096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa044100a80 0x7fa044199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:01:58.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.371+0000 7fa04b0f4640 1 -- 172.21.15.152:0/1823133096 >> 172.21.15.152:0/1823133096 conn(0x7fa0440fc7a0 msgr2=0x7fa0440fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:01:58.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.371+0000 7fa04b0f4640 1 -- 172.21.15.152:0/1823133096 shutdown_connections 2024-09-11T23:01:58.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:01:58.371+0000 7fa04b0f4640 1 -- 172.21.15.152:0/1823133096 wait complete. 2024-09-11T23:01:58.971 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:58 smithi152 ceph-mon[31181]: pgmap v279: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:58.971 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:01:58 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T23:01:59.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:58 smithi191 ceph-mon[35211]: pgmap v279: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:59.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:01:58 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T23:01:59.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:58 smithi204 ceph-mon[35454]: pgmap v279: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:01:59.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:01:58 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T23:01:59.256 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:01:59.256 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:01:59.510 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:00.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.071+0000 7f6d37a2a640 1 -- 172.21.15.152:0/3751895833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d30102570 msgr2=0x7f6d301029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:00.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.071+0000 7f6d37a2a640 1 --2- 172.21.15.152:0/3751895833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d30102570 0x7f6d301029b0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f6d24009900 tx=0x7f6d2402a480 comp rx=0 tx=0).stop 2024-09-11T23:02:00.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.071+0000 7f6d37a2a640 1 -- 172.21.15.152:0/3751895833 shutdown_connections 2024-09-11T23:02:00.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.071+0000 7f6d37a2a640 1 --2- 172.21.15.152:0/3751895833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d30102ef0 0x7f6d3010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:00.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.071+0000 7f6d37a2a640 1 --2- 172.21.15.152:0/3751895833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d30102570 0x7f6d301029b0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:00.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.071+0000 7f6d37a2a640 1 --2- 172.21.15.152:0/3751895833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d30100e20 0x7f6d30101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:00.092 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.071+0000 7f6d37a2a640 1 -- 172.21.15.152:0/3751895833 >> 172.21.15.152:0/3751895833 conn(0x7f6d300fc7d0 msgr2=0x7f6d300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.071+0000 7f6d37a2a640 1 -- 172.21.15.152:0/3751895833 shutdown_connections 2024-09-11T23:02:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.071+0000 7f6d37a2a640 1 -- 172.21.15.152:0/3751895833 wait complete. 2024-09-11T23:02:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.071+0000 7f6d37a2a640 1 Processor -- start 2024-09-11T23:02:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.071+0000 7f6d37a2a640 1 -- start start 2024-09-11T23:02:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d37a2a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d30100e20 0x7f6d301984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d37a2a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d30102570 0x7f6d30198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d37a2a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d30102ef0 0x7f6d3019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d37a2a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f6d30110f80 con 0x7f6d30102570 2024-09-11T23:02:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d37a2a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f6d30110e00 con 0x7f6d30100e20 2024-09-11T23:02:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d37a2a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f6d30111100 con 0x7f6d30102ef0 2024-09-11T23:02:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d34fc5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d30102570 0x7f6d30198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:00.093 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d34fc5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d30102570 0x7f6d30198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52296/0 (socket says 172.21.15.152:52296) 2024-09-11T23:02:00.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d34fc5640 1 -- 172.21.15.152:0/311814189 learned_addr learned my addr 172.21.15.152:0/311814189 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:00.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d35fc7640 1 --2- 172.21.15.152:0/311814189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d30102ef0 0x7f6d3019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:00.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d357c6640 1 --2- 172.21.15.152:0/311814189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d30100e20 0x7f6d301984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:00.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d34fc5640 1 -- 172.21.15.152:0/311814189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d30102ef0 msgr2=0x7f6d3019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:00.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d34fc5640 1 --2- 172.21.15.152:0/311814189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d30102ef0 0x7f6d3019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:00.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d34fc5640 1 -- 172.21.15.152:0/311814189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d30100e20 msgr2=0x7f6d301984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:00.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d34fc5640 1 --2- 172.21.15.152:0/311814189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d30100e20 0x7f6d301984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:00.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d34fc5640 1 -- 172.21.15.152:0/311814189 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3019d4a0 con 0x7f6d30102570 2024-09-11T23:02:00.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d35fc7640 1 --2- 172.21.15.152:0/311814189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d30102ef0 0x7f6d3019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:00.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d34fc5640 1 --2- 172.21.15.152:0/311814189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d30102570 0x7f6d30198a30 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6d24006010 tx=0x7f6d24003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:00.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d2e7fc640 1 -- 172.21.15.152:0/311814189 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2403f070 con 0x7f6d30102570 2024-09-11T23:02:00.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d2e7fc640 1 -- 172.21.15.152:0/311814189 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d24031c30 con 0x7f6d30102570 2024-09-11T23:02:00.094 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d37a2a640 1 -- 172.21.15.152:0/311814189 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3019d730 con 0x7f6d30102570 2024-09-11T23:02:00.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.072+0000 7f6d2e7fc640 1 -- 172.21.15.152:0/311814189 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24035030 con 0x7f6d30102570 2024-09-11T23:02:00.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.073+0000 7f6d37a2a640 1 -- 172.21.15.152:0/311814189 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d301a5070 con 0x7f6d30102570 2024-09-11T23:02:00.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.073+0000 7f6d37a2a640 1 -- 172.21.15.152:0/311814189 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d30101200 con 0x7f6d30102570 2024-09-11T23:02:00.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.075+0000 7f6d2e7fc640 1 -- 172.21.15.152:0/311814189 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6d24042070 con 0x7f6d30102570 2024-09-11T23:02:00.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.075+0000 7f6d2e7fc640 1 --2- 172.21.15.152:0/311814189 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6d180a2d70 0x7f6d180a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:00.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.076+0000 7f6d357c6640 1 --2- 172.21.15.152:0/311814189 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6d180a2d70 0x7f6d180a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:00.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.076+0000 7f6d2e7fc640 1 -- 172.21.15.152:0/311814189 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f6d240f01a0 con 0x7f6d30102570 2024-09-11T23:02:00.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.076+0000 7f6d357c6640 1 --2- 172.21.15.152:0/311814189 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6d180a2d70 0x7f6d180a5230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6d280058c0 tx=0x7f6d2800a250 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:00.095 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.079+0000 7f6d2e7fc640 1 -- 172.21.15.152:0/311814189 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d240bce40 con 0x7f6d30102570 2024-09-11T23:02:00.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.281+0000 7f6d37a2a640 1 -- 172.21.15.152:0/311814189 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d30199850 con 0x7f6d30102570 2024-09-11T23:02:00.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.281+0000 7f6d2e7fc640 1 -- 172.21.15.152:0/311814189 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f6d30199850 con 0x7f6d30102570 2024-09-11T23:02:00.283 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T23:02:00.283 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T23:02:00.284 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T23:02:00.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.284+0000 7f6d37a2a640 1 -- 172.21.15.152:0/311814189 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6d180a2d70 msgr2=0x7f6d180a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:00.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.284+0000 7f6d37a2a640 1 --2- 172.21.15.152:0/311814189 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6d180a2d70 0x7f6d180a5230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6d280058c0 tx=0x7f6d2800a250 comp rx=0 tx=0).stop 2024-09-11T23:02:00.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.284+0000 7f6d37a2a640 1 -- 172.21.15.152:0/311814189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d30102570 msgr2=0x7f6d30198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:00.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.284+0000 7f6d37a2a640 1 --2- 172.21.15.152:0/311814189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d30102570 0x7f6d30198a30 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6d24006010 tx=0x7f6d24003c10 comp rx=0 tx=0).stop 2024-09-11T23:02:00.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.284+0000 7f6d37a2a640 1 -- 172.21.15.152:0/311814189 shutdown_connections 2024-09-11T23:02:00.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.284+0000 7f6d37a2a640 1 --2- 172.21.15.152:0/311814189 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f6d180a2d70 0x7f6d180a5230 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:00.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.284+0000 7f6d37a2a640 1 --2- 172.21.15.152:0/311814189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d30102ef0 0x7f6d3019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:00.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.284+0000 7f6d37a2a640 1 --2- 172.21.15.152:0/311814189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d30102570 0x7f6d30198a30 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:00.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.284+0000 7f6d37a2a640 1 --2- 172.21.15.152:0/311814189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d30100e20 0x7f6d301984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:00.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.284+0000 7f6d37a2a640 1 -- 172.21.15.152:0/311814189 >> 172.21.15.152:0/311814189 conn(0x7f6d300fc7d0 msgr2=0x7f6d30069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:00.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.284+0000 7f6d37a2a640 1 -- 172.21.15.152:0/311814189 shutdown_connections 2024-09-11T23:02:00.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:00.284+0000 7f6d37a2a640 1 -- 172.21.15.152:0/311814189 wait complete. 2024-09-11T23:02:01.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:00 smithi191 ceph-mon[35211]: pgmap v280: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 12 op/s 2024-09-11T23:02:01.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:00 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/311814189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:02:01.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:00 smithi204 ceph-mon[35454]: pgmap v280: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 12 op/s 2024-09-11T23:02:01.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:00 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/311814189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:02:01.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:00 smithi152 ceph-mon[31181]: pgmap v280: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 12 op/s 2024-09-11T23:02:01.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:00 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/311814189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:02:01.213 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:02:01.463 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:02.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:01.999+0000 7f78b745a640 1 -- 172.21.15.152:0/4211895206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78b0101df0 msgr2=0x7f78b01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:02.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:01.999+0000 7f78b745a640 1 --2- 172.21.15.152:0/4211895206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78b0101df0 0x7f78b01021d0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009900 tx=0x7f78ac02a410 comp rx=0 tx=0).stop 2024-09-11T23:02:02.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.000+0000 7f78b745a640 1 -- 172.21.15.152:0/4211895206 shutdown_connections 2024-09-11T23:02:02.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.000+0000 7f78b745a640 1 --2- 172.21.15.152:0/4211895206 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78b0106f20 0x7f78b010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.000+0000 7f78b745a640 1 --2- 172.21.15.152:0/4211895206 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78b01027a0 0x7f78b01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.000+0000 7f78b745a640 1 --2- 172.21.15.152:0/4211895206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78b0101df0 0x7f78b01021d0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.000+0000 7f78b745a640 1 -- 172.21.15.152:0/4211895206 >> 172.21.15.152:0/4211895206 conn(0x7f78b00fd8b0 msgr2=0x7f78b00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:02.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.000+0000 7f78b745a640 1 -- 172.21.15.152:0/4211895206 shutdown_connections 2024-09-11T23:02:02.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.000+0000 7f78b745a640 1 -- 172.21.15.152:0/4211895206 wait complete. 2024-09-11T23:02:02.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.000+0000 7f78b745a640 1 Processor -- start 2024-09-11T23:02:02.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.000+0000 7f78b745a640 1 -- start start 2024-09-11T23:02:02.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.000+0000 7f78b745a640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78b0101df0 0x7f78b0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.000+0000 7f78b745a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78b01027a0 0x7f78b01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.000+0000 7f78b745a640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78b0106f20 0x7f78b019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.000+0000 7f78b745a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f78b0111f40 con 0x7f78b01027a0 2024-09-11T23:02:02.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.000+0000 7f78b745a640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f78b0111dc0 con 0x7f78b0101df0 2024-09-11T23:02:02.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.000+0000 7f78b745a640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01120c0 con 0x7f78b0106f20 2024-09-11T23:02:02.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.001+0000 7f78b49f5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78b01027a0 0x7f78b01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.001+0000 7f78b49f5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78b01027a0 0x7f78b01989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52324/0 (socket says 172.21.15.152:52324) 2024-09-11T23:02:02.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.001+0000 7f78b49f5640 1 -- 172.21.15.152:0/1915788617 learned_addr learned my addr 172.21.15.152:0/1915788617 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:02.023 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.001+0000 7f78b51f6640 1 --2- 172.21.15.152:0/1915788617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78b0101df0 0x7f78b0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.001+0000 7f78b59f7640 1 --2- 172.21.15.152:0/1915788617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78b0106f20 0x7f78b019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.001+0000 7f78b49f5640 1 -- 172.21.15.152:0/1915788617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78b0106f20 msgr2=0x7f78b019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:02.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.001+0000 7f78b49f5640 1 --2- 172.21.15.152:0/1915788617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78b0106f20 0x7f78b019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.001+0000 7f78b49f5640 1 -- 172.21.15.152:0/1915788617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78b0101df0 msgr2=0x7f78b0198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:02.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.001+0000 7f78b49f5640 1 --2- 172.21.15.152:0/1915788617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78b0101df0 0x7f78b0198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.001+0000 7f78b49f5640 1 -- 172.21.15.152:0/1915788617 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b019d430 con 0x7f78b01027a0 2024-09-11T23:02:02.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.001+0000 7f78b49f5640 1 --2- 172.21.15.152:0/1915788617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78b01027a0 0x7f78b01989c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f78a000d950 tx=0x7f78a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:02.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.001+0000 7f78a67fc640 1 -- 172.21.15.152:0/1915788617 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0017070 con 0x7f78b01027a0 2024-09-11T23:02:02.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.001+0000 7f78b745a640 1 -- 172.21.15.152:0/1915788617 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b019d720 con 0x7f78b01027a0 2024-09-11T23:02:02.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.001+0000 7f78a67fc640 1 -- 172.21.15.152:0/1915788617 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78a0009c30 con 0x7f78b01027a0 2024-09-11T23:02:02.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.001+0000 7f78a67fc640 1 -- 172.21.15.152:0/1915788617 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a000f6b0 con 0x7f78b01027a0 2024-09-11T23:02:02.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.001+0000 7f78b745a640 1 -- 172.21.15.152:0/1915788617 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b01a5000 con 0x7f78b01027a0 2024-09-11T23:02:02.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.003+0000 7f78a67fc640 1 -- 172.21.15.152:0/1915788617 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f78a00109d0 con 0x7f78b01027a0 2024-09-11T23:02:02.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.004+0000 7f78b9833740 1 Processor -- start 2024-09-11T23:02:02.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.004+0000 7f78b9833740 1 -- start start 2024-09-11T23:02:02.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.004+0000 7f78b9833740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4cb40 0x55a7a1e4b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.004+0000 7f78b9833740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1e4ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.004+0000 7f78b9833740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 0x55a7a1e5cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.004+0000 7f78b9833740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a7a1db12f0 con 0x55a7a1e4cb40 2024-09-11T23:02:02.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.004+0000 7f78a67fc640 1 --2- 172.21.15.152:0/1915788617 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f789c0a2cf0 0x7f789c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.004+0000 7f78b9833740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a7a1d58cf0 con 0x55a7a1e4b5d0 2024-09-11T23:02:02.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.004+0000 7f78b9833740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a7a1dad2b0 con 0x55a7a1e5a780 2024-09-11T23:02:02.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.004+0000 7f78b59f7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4cb40 0x55a7a1e4b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.004+0000 7f78b59f7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4cb40 0x55a7a1e4b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52334/0 (socket says 172.21.15.152:52334) 2024-09-11T23:02:02.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.004+0000 7f78b59f7640 1 -- 172.21.15.152:0/982907825 learned_addr learned my addr 172.21.15.152:0/982907825 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:02.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.004+0000 7f78b59f7640 1 --2- 172.21.15.152:0/1915788617 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f789c0a2cf0 0x7f789c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.004+0000 7f78b49f5640 1 --2- 172.21.15.152:0/982907825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 0x55a7a1e5cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.005+0000 7f78a67fc640 1 -- 172.21.15.152:0/1915788617 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f78a00cf0a0 con 0x7f78b01027a0 2024-09-11T23:02:02.026 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.005+0000 7f78b51f6640 1 --2- 172.21.15.152:0/982907825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1e4ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.005+0000 7f78b59f7640 1 -- 172.21.15.152:0/982907825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 msgr2=0x55a7a1e5cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:02.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.005+0000 7f78b59f7640 1 --2- 172.21.15.152:0/982907825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 0x55a7a1e5cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.005+0000 7f78b59f7640 1 -- 172.21.15.152:0/982907825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e4b5d0 msgr2=0x55a7a1e4ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:02.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.005+0000 7f78b59f7640 1 --2- 172.21.15.152:0/982907825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1e4ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.005+0000 7f78b59f7640 1 -- 172.21.15.152:0/982907825 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7a1d3d3a0 con 0x55a7a1e4cb40 2024-09-11T23:02:02.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.005+0000 7f78b59f7640 1 --2- 172.21.15.152:0/982907825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4cb40 0x55a7a1e4b090 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f78a8013d20 tx=0x7f78a8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bc0f8bda53d8b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:02.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.005+0000 7f78b59f7640 1 --2- 172.21.15.152:0/1915788617 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f789c0a2cf0 0x7f789c0a51b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009fd0 tx=0x7f78a8015000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:02.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.005+0000 7f789b7fe640 1 -- 172.21.15.152:0/982907825 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8009af0 con 0x55a7a1e4cb40 2024-09-11T23:02:02.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.005+0000 7f789b7fe640 1 -- 172.21.15.152:0/982907825 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78a8013670 con 0x55a7a1e4cb40 2024-09-11T23:02:02.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.005+0000 7f789b7fe640 1 -- 172.21.15.152:0/982907825 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8005020 con 0x55a7a1e4cb40 2024-09-11T23:02:02.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.006+0000 7f78b9833740 1 -- 172.21.15.152:0/982907825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4cb40 msgr2=0x55a7a1e4b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:02.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.006+0000 7f78b9833740 1 --2- 172.21.15.152:0/982907825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4cb40 0x55a7a1e4b090 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f78a8013d20 tx=0x7f78a8010740 comp rx=0 tx=0).stop 2024-09-11T23:02:02.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.006+0000 7f78b9833740 1 -- 172.21.15.152:0/982907825 shutdown_connections 2024-09-11T23:02:02.027 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.006+0000 7f78b9833740 1 --2- 172.21.15.152:0/982907825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 0x55a7a1e5cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.006+0000 7f78b9833740 1 --2- 172.21.15.152:0/982907825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1e4ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.006+0000 7f78b9833740 1 --2- 172.21.15.152:0/982907825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4cb40 0x55a7a1e4b090 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.006+0000 7f78b9833740 1 -- 172.21.15.152:0/982907825 >> 172.21.15.152:0/982907825 conn(0x55a7a1e14310 msgr2=0x55a7a1e47c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:02.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.006+0000 7f78b9833740 1 -- 172.21.15.152:0/982907825 shutdown_connections 2024-09-11T23:02:02.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.006+0000 7f78b9833740 1 -- 172.21.15.152:0/982907825 wait complete. 2024-09-11T23:02:02.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.006+0000 7f78b9833740 1 -- start start 2024-09-11T23:02:02.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.006+0000 7f78b9833740 1 Processor -- start 2024-09-11T23:02:02.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b9833740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1e5a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b9833740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e4cb40 0x55a7a1e574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.029 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b9833740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 0x55a7a1e57be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b9833740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55a7a1e5d420 con 0x55a7a1e4b5d0 2024-09-11T23:02:02.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b9833740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55a7a1d6c060 con 0x55a7a1e4cb40 2024-09-11T23:02:02.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b9833740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55a7a1e5d5a0 con 0x55a7a1e5a780 2024-09-11T23:02:02.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b9833740 10 client.0 _ll_register_callbacks cb 0x55a7a1abf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:02.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b51f6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1e5a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b51f6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1e5a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52344/0 (socket says 172.21.15.152:52344) 2024-09-11T23:02:02.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b51f6640 1 -- 172.21.15.152:0/41055002 learned_addr learned my addr 172.21.15.152:0/41055002 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:02.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b59f7640 1 --2- 172.21.15.152:0/41055002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 0x55a7a1e57be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b49f5640 1 --2- 172.21.15.152:0/41055002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e4cb40 0x55a7a1e574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b51f6640 1 -- 172.21.15.152:0/41055002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 msgr2=0x55a7a1e57be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:02.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b51f6640 1 --2- 172.21.15.152:0/41055002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 0x55a7a1e57be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b51f6640 1 -- 172.21.15.152:0/41055002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e4cb40 msgr2=0x55a7a1e574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:02.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b51f6640 1 --2- 172.21.15.152:0/41055002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e4cb40 0x55a7a1e574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b51f6640 1 -- 172.21.15.152:0/41055002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7a1b223c0 con 0x55a7a1e4b5d0 2024-09-11T23:02:02.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b59f7640 1 --2- 172.21.15.152:0/41055002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 0x55a7a1e57be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:02.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f78b51f6640 1 --2- 172.21.15.152:0/41055002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1e5a1f0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f78ac000c00 tx=0x7f78ac031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf0d99a86fbb5c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:02.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.007+0000 7f7898ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:02.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.008+0000 7f7898ff9640 1 -- 172.21.15.152:0/41055002 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac035040 con 0x55a7a1e4b5d0 2024-09-11T23:02:02.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.008+0000 7f7898ff9640 1 -- 172.21.15.152:0/41055002 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f78ac004240 con 0x55a7a1e4b5d0 2024-09-11T23:02:02.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.008+0000 7f78b9833740 1 -- 172.21.15.152:0/41055002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78a8013670 con 0x55a7a1e4b5d0 2024-09-11T23:02:02.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.008+0000 7f7898ff9640 1 -- 172.21.15.152:0/41055002 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f78ac004440 con 0x55a7a1e4b5d0 2024-09-11T23:02:02.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.008+0000 7f78b9833740 10 client.18006 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:02.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.008+0000 7f78b9833740 1 -- 172.21.15.152:0/41055002 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7a1e68fa0 con 0x55a7a1e4b5d0 2024-09-11T23:02:02.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.008+0000 7f7898ff9640 1 -- 172.21.15.152:0/41055002 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78a8013670 con 0x55a7a1e4b5d0 2024-09-11T23:02:02.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.008+0000 7f7898ff9640 1 -- 172.21.15.152:0/41055002 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f78ac04d020 con 0x55a7a1e4b5d0 2024-09-11T23:02:02.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.008+0000 7f78b9833740 10 client.18006 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:02.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.008+0000 7f78b9833740 10 client.18006 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T23:02:02.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.008+0000 7f78b9833740 20 client.18006 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:02.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.008+0000 7f78b9833740 1 --2- 172.21.15.152:0/41055002 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55a7a1e6a3b0 0x55a7a1e6c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.008+0000 7f78b9833740 4 client.18006 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:02:02.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.008+0000 7f78b9833740 1 -- 172.21.15.152:0/41055002 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7a1dbe860 con 0x55a7a1e6a3b0 2024-09-11T23:02:02.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.008+0000 7f78b49f5640 1 --2- 172.21.15.152:0/41055002 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55a7a1e6a3b0 0x55a7a1e6c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.009+0000 7f78b49f5640 1 --2- 172.21.15.152:0/41055002 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55a7a1e6a3b0 0x55a7a1e6c7a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a53be087674ab35 server_cookie=f8e8b4db87d46b67 in_seq=0 out_seq=0 2024-09-11T23:02:02.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.009+0000 7f7898ff9640 10 client.18006 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T23:02:02.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.009+0000 7f7898ff9640 1 -- 172.21.15.152:0/41055002 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7a1dbe860 con 0x55a7a1e6a3b0 2024-09-11T23:02:02.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.009+0000 7f7898ff9640 10 client.18006 handle_command_reply: tid=0 2024-09-11T23:02:02.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.009+0000 7f78b9833740 1 client.18006 shutdown 2024-09-11T23:02:02.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.009+0000 7f78b9833740 1 -- 172.21.15.152:0/41055002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4b5d0 msgr2=0x55a7a1e5a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:02.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.009+0000 7f78b9833740 1 --2- 172.21.15.152:0/41055002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1e5a1f0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f78ac000c00 tx=0x7f78ac031920 comp rx=0 tx=0).stop 2024-09-11T23:02:02.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.009+0000 7f78b9833740 1 -- 172.21.15.152:0/41055002 shutdown_connections 2024-09-11T23:02:02.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.009+0000 7f78b9833740 1 --2- 172.21.15.152:0/41055002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 0x55a7a1e57be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.009+0000 7f78b9833740 1 --2- 172.21.15.152:0/41055002 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55a7a1e6a3b0 0x55a7a1e6c7a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.009+0000 7f78b9833740 1 --2- 172.21.15.152:0/41055002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e4cb40 0x55a7a1e574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.009+0000 7f78b9833740 1 --2- 172.21.15.152:0/41055002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1e5a1f0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.009+0000 7f78b9833740 1 -- 172.21.15.152:0/41055002 >> 172.21.15.152:0/41055002 conn(0x55a7a1e14310 msgr2=0x55a7a1e4d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:02.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.009+0000 7f7898ff9640 0 client.18006 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T23:02:02.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.010+0000 7f78b9833740 1 -- 172.21.15.152:0/41055002 shutdown_connections 2024-09-11T23:02:02.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.010+0000 7f78b9833740 1 -- 172.21.15.152:0/41055002 wait complete. 2024-09-11T23:02:02.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.010+0000 7f78b9833740 20 client.18006 trim_cache size 0 max 16384 2024-09-11T23:02:02.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b9833740 1 Processor -- start 2024-09-11T23:02:02.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b9833740 1 -- start start 2024-09-11T23:02:02.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b9833740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1e666c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b9833740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e4cb40 0x55a7a1e72050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b9833740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e5a780 0x55a7a1e72590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b9833740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01ab580 con 0x55a7a1e4b5d0 2024-09-11T23:02:02.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b9833740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01ad0c0 con 0x55a7a1e5a780 2024-09-11T23:02:02.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b9833740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01abc50 con 0x55a7a1e4cb40 2024-09-11T23:02:02.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b51f6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1e666c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b51f6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1e666c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52354/0 (socket says 172.21.15.152:52354) 2024-09-11T23:02:02.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b51f6640 1 -- 172.21.15.152:0/1079576734 learned_addr learned my addr 172.21.15.152:0/1079576734 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:02.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b49f5640 1 --2- 172.21.15.152:0/1079576734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e4cb40 0x55a7a1e72050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b59f7640 1 --2- 172.21.15.152:0/1079576734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e5a780 0x55a7a1e72590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b51f6640 1 -- 172.21.15.152:0/1079576734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e4cb40 msgr2=0x55a7a1e72050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:02.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b51f6640 1 --2- 172.21.15.152:0/1079576734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e4cb40 0x55a7a1e72050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b51f6640 1 -- 172.21.15.152:0/1079576734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e5a780 msgr2=0x55a7a1e72590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:02.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b51f6640 1 --2- 172.21.15.152:0/1079576734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e5a780 0x55a7a1e72590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b51f6640 1 -- 172.21.15.152:0/1079576734 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7a1e5cee0 con 0x55a7a1e4b5d0 2024-09-11T23:02:02.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.023+0000 7f78b49f5640 1 --2- 172.21.15.152:0/1079576734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e4cb40 0x55a7a1e72050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:02.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.024+0000 7f78b51f6640 1 --2- 172.21.15.152:0/1079576734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1e666c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f78ac005940 tx=0x7f78ac009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7c0e6e5741758f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:02.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.024+0000 7f789affd640 1 -- 172.21.15.152:0/1079576734 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac035070 con 0x55a7a1e4b5d0 2024-09-11T23:02:02.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.024+0000 7f789affd640 1 -- 172.21.15.152:0/1079576734 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78ac005510 con 0x55a7a1e4b5d0 2024-09-11T23:02:02.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.024+0000 7f789affd640 1 -- 172.21.15.152:0/1079576734 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac000e60 con 0x55a7a1e4b5d0 2024-09-11T23:02:02.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.024+0000 7f78b9833740 1 -- 172.21.15.152:0/1079576734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4b5d0 msgr2=0x55a7a1e666c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:02.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.024+0000 7f78b9833740 1 --2- 172.21.15.152:0/1079576734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1e666c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f78ac005940 tx=0x7f78ac009520 comp rx=0 tx=0).stop 2024-09-11T23:02:02.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.024+0000 7f78b9833740 1 -- 172.21.15.152:0/1079576734 shutdown_connections 2024-09-11T23:02:02.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.024+0000 7f78b9833740 1 --2- 172.21.15.152:0/1079576734 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e5a780 0x55a7a1e72590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.024+0000 7f78b9833740 1 --2- 172.21.15.152:0/1079576734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e4cb40 0x55a7a1e72050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.024+0000 7f78b9833740 1 --2- 172.21.15.152:0/1079576734 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1e666c0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.024+0000 7f78b9833740 1 -- 172.21.15.152:0/1079576734 >> 172.21.15.152:0/1079576734 conn(0x55a7a1e14310 msgr2=0x55a7a1e56590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:02.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.024+0000 7f78b9833740 1 -- 172.21.15.152:0/1079576734 shutdown_connections 2024-09-11T23:02:02.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.024+0000 7f78b9833740 1 -- 172.21.15.152:0/1079576734 wait complete. 2024-09-11T23:02:02.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.025+0000 7f78b9833740 1 -- start start 2024-09-11T23:02:02.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.025+0000 7f78b9833740 1 Processor -- start 2024-09-11T23:02:02.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.025+0000 7f78b9833740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1ecd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.025+0000 7f78b9833740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4cb40 0x55a7a1ecd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.025+0000 7f78b9833740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 0x55a7a1e5dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.025+0000 7f78b9833740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01ac9f0 con 0x55a7a1e4cb40 2024-09-11T23:02:02.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.025+0000 7f78b9833740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01ac320 con 0x55a7a1e4b5d0 2024-09-11T23:02:02.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.025+0000 7f78b9833740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01a9580 con 0x55a7a1e5a780 2024-09-11T23:02:02.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.025+0000 7f78b9833740 10 client.0 _ll_register_callbacks cb 0x7f78ac009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:02.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.025+0000 7f78b49f5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4cb40 0x55a7a1ecd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.025+0000 7f78b49f5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4cb40 0x55a7a1ecd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52360/0 (socket says 172.21.15.152:52360) 2024-09-11T23:02:02.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.025+0000 7f78b49f5640 1 -- 172.21.15.152:0/2995094918 learned_addr learned my addr 172.21.15.152:0/2995094918 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:02.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f78b51f6640 1 --2- 172.21.15.152:0/2995094918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1ecd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f78b59f7640 1 --2- 172.21.15.152:0/2995094918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 0x55a7a1e5dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f78b49f5640 1 -- 172.21.15.152:0/2995094918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 msgr2=0x55a7a1e5dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:02.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f78b49f5640 1 --2- 172.21.15.152:0/2995094918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 0x55a7a1e5dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f78b49f5640 1 -- 172.21.15.152:0/2995094918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e4b5d0 msgr2=0x55a7a1ecd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:02.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f78b49f5640 1 --2- 172.21.15.152:0/2995094918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1ecd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f78b49f5640 1 -- 172.21.15.152:0/2995094918 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7a1e5e590 con 0x55a7a1e4cb40 2024-09-11T23:02:02.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f78b59f7640 1 --2- 172.21.15.152:0/2995094918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 0x55a7a1e5dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:02.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f78b49f5640 1 --2- 172.21.15.152:0/2995094918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4cb40 0x55a7a1ecd870 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f78a0097a40 tx=0x7f78a0010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f47911489eec558 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:02.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f7898ff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:02.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f7898ff9640 1 -- 172.21.15.152:0/2995094918 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0020040 con 0x55a7a1e4cb40 2024-09-11T23:02:02.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f7898ff9640 1 -- 172.21.15.152:0/2995094918 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f78a00998e0 con 0x55a7a1e4cb40 2024-09-11T23:02:02.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f78b9833740 1 -- 172.21.15.152:0/2995094918 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78ac005510 con 0x55a7a1e4cb40 2024-09-11T23:02:02.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f7898ff9640 1 -- 172.21.15.152:0/2995094918 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f78a009bce0 con 0x55a7a1e4cb40 2024-09-11T23:02:02.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f78b9833740 10 client.18012 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:02.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f78b9833740 1 -- 172.21.15.152:0/2995094918 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a7a1ed4ad0 con 0x55a7a1e4cb40 2024-09-11T23:02:02.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.026+0000 7f7898ff9640 1 -- 172.21.15.152:0/2995094918 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78ac005510 con 0x55a7a1e4cb40 2024-09-11T23:02:02.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f7898ff9640 1 -- 172.21.15.152:0/2995094918 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f78a00cca90 con 0x55a7a1e4cb40 2024-09-11T23:02:02.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b9833740 10 client.18012 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:02.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b9833740 10 client.18012 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T23:02:02.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b9833740 20 client.18012 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:02.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b9833740 1 --2- 172.21.15.152:0/2995094918 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55a7a1ed5250 0x55a7a1ed7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:02.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b9833740 4 client.18012 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:02:02.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b9833740 1 -- 172.21.15.152:0/2995094918 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7a1db8d70 con 0x55a7a1ed5250 2024-09-11T23:02:02.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b51f6640 1 --2- 172.21.15.152:0/2995094918 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55a7a1ed5250 0x55a7a1ed7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:02.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f7898ff9640 10 client.18012 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T23:02:02.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b51f6640 1 --2- 172.21.15.152:0/2995094918 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55a7a1ed5250 0x55a7a1ed7670 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=67ac2a3b91625a49 server_cookie=ffb3332c9d00aa6e in_seq=0 out_seq=0 2024-09-11T23:02:02.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f7898ff9640 1 -- 172.21.15.152:0/2995094918 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7a1db8d70 con 0x55a7a1ed5250 2024-09-11T23:02:02.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f7898ff9640 10 client.18012 handle_command_reply: tid=0 2024-09-11T23:02:02.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b9833740 1 client.18012 shutdown 2024-09-11T23:02:02.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b9833740 1 -- 172.21.15.152:0/2995094918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4cb40 msgr2=0x55a7a1ecd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:02.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b9833740 1 --2- 172.21.15.152:0/2995094918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4cb40 0x55a7a1ecd870 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f78a0097a40 tx=0x7f78a0010d70 comp rx=0 tx=0).stop 2024-09-11T23:02:02.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b9833740 1 -- 172.21.15.152:0/2995094918 shutdown_connections 2024-09-11T23:02:02.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b9833740 1 --2- 172.21.15.152:0/2995094918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a7a1e5a780 0x55a7a1e5dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b9833740 1 --2- 172.21.15.152:0/2995094918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a7a1e4cb40 0x55a7a1ecd870 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b9833740 1 --2- 172.21.15.152:0/2995094918 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55a7a1ed5250 0x55a7a1ed7670 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b9833740 1 --2- 172.21.15.152:0/2995094918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a7a1e4b5d0 0x55a7a1ecd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f78b9833740 1 -- 172.21.15.152:0/2995094918 >> 172.21.15.152:0/2995094918 conn(0x55a7a1e14310 msgr2=0x55a7a1e46500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:02.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.027+0000 7f7898ff9640 0 client.18012 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T23:02:02.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.028+0000 7f78b9833740 1 -- 172.21.15.152:0/2995094918 shutdown_connections 2024-09-11T23:02:02.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.028+0000 7f78b9833740 1 -- 172.21.15.152:0/2995094918 wait complete. 2024-09-11T23:02:02.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.028+0000 7f78b9833740 20 client.18012 trim_cache size 0 max 16384 2024-09-11T23:02:02.045 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:02:02.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.028+0000 7f78b745a640 1 -- 172.21.15.152:0/1915788617 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f789c0a2cf0 msgr2=0x7f789c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:02.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.028+0000 7f78b745a640 1 --2- 172.21.15.152:0/1915788617 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f789c0a2cf0 0x7f789c0a51b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009fd0 tx=0x7f78a8015000 comp rx=0 tx=0).stop 2024-09-11T23:02:02.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.028+0000 7f78b745a640 1 -- 172.21.15.152:0/1915788617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78b01027a0 msgr2=0x7f78b01989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:02.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.028+0000 7f78b745a640 1 --2- 172.21.15.152:0/1915788617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78b01027a0 0x7f78b01989c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f78a000d950 tx=0x7f78a000de10 comp rx=0 tx=0).stop 2024-09-11T23:02:02.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.028+0000 7f78b745a640 1 -- 172.21.15.152:0/1915788617 shutdown_connections 2024-09-11T23:02:02.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.028+0000 7f78b745a640 1 --2- 172.21.15.152:0/1915788617 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f789c0a2cf0 0x7f789c0a51b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.028+0000 7f78b745a640 1 --2- 172.21.15.152:0/1915788617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78b0106f20 0x7f78b019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.028+0000 7f78b745a640 1 --2- 172.21.15.152:0/1915788617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78b01027a0 0x7f78b01989c0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.028+0000 7f78b745a640 1 --2- 172.21.15.152:0/1915788617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78b0101df0 0x7f78b0198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:02.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.028+0000 7f78b745a640 1 -- 172.21.15.152:0/1915788617 >> 172.21.15.152:0/1915788617 conn(0x7f78b00fd8b0 msgr2=0x7f78b00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:02.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.028+0000 7f78b745a640 1 -- 172.21.15.152:0/1915788617 shutdown_connections 2024-09-11T23:02:02.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:02.028+0000 7f78b745a640 1 -- 172.21.15.152:0/1915788617 wait complete. 2024-09-11T23:02:02.893 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:02 smithi152 ceph-mon[31181]: pgmap v281: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-09-11T23:02:02.925 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:02:03.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:02 smithi191 ceph-mon[35211]: pgmap v281: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-09-11T23:02:03.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:02 smithi204 ceph-mon[35454]: pgmap v281: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 9 op/s 2024-09-11T23:02:03.175 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:03.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.754+0000 7fb05b3c2640 1 -- 172.21.15.152:0/3161675140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0541036c0 msgr2=0x7fb05410bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:03.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.754+0000 7fb05b3c2640 1 --2- 172.21.15.152:0/3161675140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0541036c0 0x7fb05410bbb0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fb050009900 tx=0x7fb05002a480 comp rx=0 tx=0).stop 2024-09-11T23:02:03.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.754+0000 7fb05b3c2640 1 -- 172.21.15.152:0/3161675140 shutdown_connections 2024-09-11T23:02:03.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.754+0000 7fb05b3c2640 1 --2- 172.21.15.152:0/3161675140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb05410c180 0x7fb05410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.754+0000 7fb05b3c2640 1 --2- 172.21.15.152:0/3161675140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb0541036c0 0x7fb05410bbb0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.754+0000 7fb05b3c2640 1 --2- 172.21.15.152:0/3161675140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb054102da0 0x7fb054103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.754+0000 7fb05b3c2640 1 -- 172.21.15.152:0/3161675140 >> 172.21.15.152:0/3161675140 conn(0x7fb0540fc7a0 msgr2=0x7fb0540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:03.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.754+0000 7fb05b3c2640 1 -- 172.21.15.152:0/3161675140 shutdown_connections 2024-09-11T23:02:03.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.754+0000 7fb05b3c2640 1 -- 172.21.15.152:0/3161675140 wait complete. 2024-09-11T23:02:03.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05b3c2640 1 Processor -- start 2024-09-11T23:02:03.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05b3c2640 1 -- start start 2024-09-11T23:02:03.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05b3c2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb054102da0 0x7fb054198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05b3c2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0541036c0 0x7fb054198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05b3c2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb05410c180 0x7fb05419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05b3c2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb054111f60 con 0x7fb054102da0 2024-09-11T23:02:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05b3c2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb054111de0 con 0x7fb0541036c0 2024-09-11T23:02:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05b3c2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541120e0 con 0x7fb05410c180 2024-09-11T23:02:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05915e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb054102da0 0x7fb054198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05915e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb054102da0 0x7fb054198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52378/0 (socket says 172.21.15.152:52378) 2024-09-11T23:02:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05915e640 1 -- 172.21.15.152:0/1698410910 learned_addr learned my addr 172.21.15.152:0/1698410910 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05995f640 1 --2- 172.21.15.152:0/1698410910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb05410c180 0x7fb05419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05895d640 1 --2- 172.21.15.152:0/1698410910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0541036c0 0x7fb054198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05915e640 1 -- 172.21.15.152:0/1698410910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb05410c180 msgr2=0x7fb05419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05915e640 1 --2- 172.21.15.152:0/1698410910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb05410c180 0x7fb05419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05915e640 1 -- 172.21.15.152:0/1698410910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0541036c0 msgr2=0x7fb054198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05915e640 1 --2- 172.21.15.152:0/1698410910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0541036c0 0x7fb054198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.775 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05915e640 1 -- 172.21.15.152:0/1698410910 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05419d3e0 con 0x7fb054102da0 2024-09-11T23:02:03.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.755+0000 7fb05895d640 1 --2- 172.21.15.152:0/1698410910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0541036c0 0x7fb054198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:03.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.756+0000 7fb05915e640 1 --2- 172.21.15.152:0/1698410910 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb054102da0 0x7fb054198430 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb04400e9f0 tx=0x7fb04400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:03.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.756+0000 7fb04a7fc640 1 -- 172.21.15.152:0/1698410910 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04400cd80 con 0x7fb054102da0 2024-09-11T23:02:03.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.756+0000 7fb04a7fc640 1 -- 172.21.15.152:0/1698410910 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb044005500 con 0x7fb054102da0 2024-09-11T23:02:03.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.756+0000 7fb05b3c2640 1 -- 172.21.15.152:0/1698410910 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05419d6d0 con 0x7fb054102da0 2024-09-11T23:02:03.776 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.756+0000 7fb04a7fc640 1 -- 172.21.15.152:0/1698410910 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044010690 con 0x7fb054102da0 2024-09-11T23:02:03.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.756+0000 7fb05b3c2640 1 -- 172.21.15.152:0/1698410910 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0541a4fb0 con 0x7fb054102da0 2024-09-11T23:02:03.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.758+0000 7fb04a7fc640 1 -- 172.21.15.152:0/1698410910 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb0440046c0 con 0x7fb054102da0 2024-09-11T23:02:03.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.758+0000 7fb05d79b740 1 Processor -- start 2024-09-11T23:02:03.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05d79b740 1 -- start start 2024-09-11T23:02:03.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05d79b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572dcb40 0x5607572db090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05d79b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572db5d0 0x5607572dba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05d79b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572e6370 0x5607572e8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05d79b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5607572412f0 con 0x5607572e6370 2024-09-11T23:02:03.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05d79b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5607571e8cf0 con 0x5607572db5d0 2024-09-11T23:02:03.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05d79b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56075723d2b0 con 0x5607572dcb40 2024-09-11T23:02:03.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb04a7fc640 1 --2- 172.21.15.152:0/1698410910 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb0400a2cf0 0x7fb0400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05895d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572e6370 0x5607572e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05895d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572e6370 0x5607572e8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52386/0 (socket says 172.21.15.152:52386) 2024-09-11T23:02:03.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05895d640 1 -- 172.21.15.152:0/447131247 learned_addr learned my addr 172.21.15.152:0/447131247 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:03.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05995f640 1 --2- 172.21.15.152:0/447131247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572dcb40 0x5607572db090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05995f640 1 --2- 172.21.15.152:0/1698410910 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb0400a2cf0 0x7fb0400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb04a7fc640 1 -- 172.21.15.152:0/1698410910 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7fb0440cf990 con 0x7fb054102da0 2024-09-11T23:02:03.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05915e640 1 --2- 172.21.15.152:0/447131247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572db5d0 0x5607572dba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05895d640 1 -- 172.21.15.152:0/447131247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572dcb40 msgr2=0x5607572db090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:03.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05895d640 1 --2- 172.21.15.152:0/447131247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572dcb40 0x5607572db090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05895d640 1 -- 172.21.15.152:0/447131247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572db5d0 msgr2=0x5607572dba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:03.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05895d640 1 --2- 172.21.15.152:0/447131247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572db5d0 0x5607572dba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05895d640 1 -- 172.21.15.152:0/447131247 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607571cd3a0 con 0x5607572e6370 2024-09-11T23:02:03.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05915e640 1 --2- 172.21.15.152:0/447131247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572db5d0 0x5607572dba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:03.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05895d640 1 --2- 172.21.15.152:0/447131247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572e6370 0x5607572e8760 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb050031550 tx=0x7fb050032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126b611845b9ef21 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:03.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.759+0000 7fb05995f640 1 --2- 172.21.15.152:0/1698410910 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb0400a2cf0 0x7fb0400a51b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009270 tx=0x7fb04c009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:03.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.760+0000 7fb03f7fe640 1 -- 172.21.15.152:0/447131247 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05003f070 con 0x5607572e6370 2024-09-11T23:02:03.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.760+0000 7fb03f7fe640 1 -- 172.21.15.152:0/447131247 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb050032e20 con 0x5607572e6370 2024-09-11T23:02:03.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.760+0000 7fb03f7fe640 1 -- 172.21.15.152:0/447131247 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050035070 con 0x5607572e6370 2024-09-11T23:02:03.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.760+0000 7fb05d79b740 1 -- 172.21.15.152:0/447131247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572e6370 msgr2=0x5607572e8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:03.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.760+0000 7fb05d79b740 1 --2- 172.21.15.152:0/447131247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572e6370 0x5607572e8760 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb050031550 tx=0x7fb050032c30 comp rx=0 tx=0).stop 2024-09-11T23:02:03.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.760+0000 7fb05d79b740 1 -- 172.21.15.152:0/447131247 shutdown_connections 2024-09-11T23:02:03.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.761+0000 7fb05d79b740 1 --2- 172.21.15.152:0/447131247 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572e6370 0x5607572e8760 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.761+0000 7fb05d79b740 1 --2- 172.21.15.152:0/447131247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572db5d0 0x5607572dba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.761+0000 7fb05d79b740 1 --2- 172.21.15.152:0/447131247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572dcb40 0x5607572db090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.761+0000 7fb05d79b740 1 -- 172.21.15.152:0/447131247 >> 172.21.15.152:0/447131247 conn(0x5607572a4310 msgr2=0x5607572d7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:03.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.761+0000 7fb05d79b740 1 -- 172.21.15.152:0/447131247 shutdown_connections 2024-09-11T23:02:03.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.761+0000 7fb05d79b740 1 -- 172.21.15.152:0/447131247 wait complete. 2024-09-11T23:02:03.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.761+0000 7fb05d79b740 1 -- start start 2024-09-11T23:02:03.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.761+0000 7fb05d79b740 1 Processor -- start 2024-09-11T23:02:03.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.761+0000 7fb05d79b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572db5d0 0x5607572e5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.761+0000 7fb05d79b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572dcb40 0x5607572e2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.761+0000 7fb05d79b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572e6370 0x5607572e34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.761+0000 7fb05d79b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5607572e9010 con 0x5607572dcb40 2024-09-11T23:02:03.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.761+0000 7fb05d79b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5607571fc060 con 0x5607572db5d0 2024-09-11T23:02:03.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.761+0000 7fb05d79b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5607572e9190 con 0x5607572e6370 2024-09-11T23:02:03.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.761+0000 7fb05d79b740 10 client.0 _ll_register_callbacks cb 0x560756f4f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:03.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.761+0000 7fb05895d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572dcb40 0x5607572e2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.780 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb05895d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572dcb40 0x5607572e2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52392/0 (socket says 172.21.15.152:52392) 2024-09-11T23:02:03.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb05895d640 1 -- 172.21.15.152:0/2999709888 learned_addr learned my addr 172.21.15.152:0/2999709888 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:03.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb05915e640 1 --2- 172.21.15.152:0/2999709888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572db5d0 0x5607572e5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb05995f640 1 --2- 172.21.15.152:0/2999709888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572e6370 0x5607572e34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb05895d640 1 -- 172.21.15.152:0/2999709888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572e6370 msgr2=0x5607572e34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:03.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb05895d640 1 --2- 172.21.15.152:0/2999709888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572e6370 0x5607572e34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb05895d640 1 -- 172.21.15.152:0/2999709888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572db5d0 msgr2=0x5607572e5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:03.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb05895d640 1 --2- 172.21.15.152:0/2999709888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572db5d0 0x5607572e5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb05895d640 1 -- 172.21.15.152:0/2999709888 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560756fb23c0 con 0x5607572dcb40 2024-09-11T23:02:03.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb05895d640 1 --2- 172.21.15.152:0/2999709888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572dcb40 0x5607572e2f60 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fb050000c00 tx=0x7fb05002ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b319f44c0dac3ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:03.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb03cff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:03.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb03cff9640 1 -- 172.21.15.152:0/2999709888 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05003f070 con 0x5607572dcb40 2024-09-11T23:02:03.781 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb03cff9640 1 -- 172.21.15.152:0/2999709888 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7fb050004040 con 0x5607572dcb40 2024-09-11T23:02:03.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb05d79b740 1 -- 172.21.15.152:0/2999709888 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb050032e20 con 0x5607572dcb40 2024-09-11T23:02:03.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb03cff9640 1 -- 172.21.15.152:0/2999709888 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb050035040 con 0x5607572dcb40 2024-09-11T23:02:03.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb05d79b740 10 client.18024 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:03.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.762+0000 7fb05d79b740 1 -- 172.21.15.152:0/2999709888 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5607572f4970 con 0x5607572dcb40 2024-09-11T23:02:03.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.763+0000 7fb03cff9640 1 -- 172.21.15.152:0/2999709888 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb050032e20 con 0x5607572dcb40 2024-09-11T23:02:03.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.763+0000 7fb03cff9640 1 -- 172.21.15.152:0/2999709888 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb050054020 con 0x5607572dcb40 2024-09-11T23:02:03.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.763+0000 7fb05d79b740 10 client.18024 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:03.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.763+0000 7fb05d79b740 10 client.18024 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:02:03.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.763+0000 7fb05d79b740 20 client.18024 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:03.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.763+0000 7fb05d79b740 1 --2- 172.21.15.152:0/2999709888 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5607572f5cc0 0x5607572f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.763+0000 7fb05d79b740 4 client.18024 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:02:03.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.763+0000 7fb05d79b740 1 -- 172.21.15.152:0/2999709888 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56075724e860 con 0x5607572f5cc0 2024-09-11T23:02:03.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.763+0000 7fb05915e640 1 --2- 172.21.15.152:0/2999709888 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5607572f5cc0 0x5607572f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.763+0000 7fb03cff9640 10 client.18024 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T23:02:03.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.763+0000 7fb05915e640 1 --2- 172.21.15.152:0/2999709888 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5607572f5cc0 0x5607572f80b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b273040af053906a server_cookie=23d5d4f847cb1fbe in_seq=0 out_seq=0 2024-09-11T23:02:03.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.764+0000 7fb03cff9640 1 -- 172.21.15.152:0/2999709888 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56075724e860 con 0x5607572f5cc0 2024-09-11T23:02:03.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.764+0000 7fb03cff9640 10 client.18024 handle_command_reply: tid=0 2024-09-11T23:02:03.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.764+0000 7fb05d79b740 1 client.18024 shutdown 2024-09-11T23:02:03.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.764+0000 7fb05d79b740 1 -- 172.21.15.152:0/2999709888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572dcb40 msgr2=0x5607572e2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:03.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.764+0000 7fb05d79b740 1 --2- 172.21.15.152:0/2999709888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572dcb40 0x5607572e2f60 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fb050000c00 tx=0x7fb05002ac30 comp rx=0 tx=0).stop 2024-09-11T23:02:03.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.764+0000 7fb05d79b740 1 -- 172.21.15.152:0/2999709888 shutdown_connections 2024-09-11T23:02:03.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.764+0000 7fb05d79b740 1 --2- 172.21.15.152:0/2999709888 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x5607572f5cc0 0x5607572f80b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.783 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.764+0000 7fb05d79b740 1 --2- 172.21.15.152:0/2999709888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572e6370 0x5607572e34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.764+0000 7fb05d79b740 1 --2- 172.21.15.152:0/2999709888 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572dcb40 0x5607572e2f60 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.764+0000 7fb03cff9640 0 client.18024 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T23:02:03.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.764+0000 7fb05d79b740 1 --2- 172.21.15.152:0/2999709888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572db5d0 0x5607572e5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.764+0000 7fb05d79b740 1 -- 172.21.15.152:0/2999709888 >> 172.21.15.152:0/2999709888 conn(0x5607572a4310 msgr2=0x5607572dd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:03.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.764+0000 7fb05d79b740 1 -- 172.21.15.152:0/2999709888 shutdown_connections 2024-09-11T23:02:03.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.764+0000 7fb05d79b740 1 -- 172.21.15.152:0/2999709888 wait complete. 2024-09-11T23:02:03.784 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.765+0000 7fb05d79b740 20 client.18024 trim_cache size 0 max 16384 2024-09-11T23:02:03.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.777+0000 7fb05d79b740 1 Processor -- start 2024-09-11T23:02:03.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.777+0000 7fb05d79b740 1 -- start start 2024-09-11T23:02:03.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.777+0000 7fb05d79b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572db5d0 0x5607572f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.777+0000 7fb05d79b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572dcb40 0x5607572e1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb05d79b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572e6370 0x5607572e1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.789 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb05d79b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541ad860 con 0x5607572dcb40 2024-09-11T23:02:03.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb05d79b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541af3a0 con 0x5607572e6370 2024-09-11T23:02:03.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb05d79b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541adf30 con 0x5607572db5d0 2024-09-11T23:02:03.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb05895d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572dcb40 0x5607572e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb05895d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572dcb40 0x5607572e1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52404/0 (socket says 172.21.15.152:52404) 2024-09-11T23:02:03.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb05895d640 1 -- 172.21.15.152:0/3941475442 learned_addr learned my addr 172.21.15.152:0/3941475442 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:03.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb05995f640 1 --2- 172.21.15.152:0/3941475442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572e6370 0x5607572e1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb05895d640 1 -- 172.21.15.152:0/3941475442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572db5d0 msgr2=0x5607572f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:03.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb05895d640 1 --2- 172.21.15.152:0/3941475442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572db5d0 0x5607572f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb05895d640 1 -- 172.21.15.152:0/3941475442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572e6370 msgr2=0x5607572e1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:03.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb05895d640 1 --2- 172.21.15.152:0/3941475442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572e6370 0x5607572e1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb05895d640 1 -- 172.21.15.152:0/3941475442 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607572d74f0 con 0x5607572dcb40 2024-09-11T23:02:03.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb05995f640 1 --2- 172.21.15.152:0/3941475442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572e6370 0x5607572e1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:03.790 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb05895d640 1 --2- 172.21.15.152:0/3941475442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572dcb40 0x5607572e1830 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fb05002a4c0 tx=0x7fb050005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c16580ded86f949 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:03.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb03effd640 1 -- 172.21.15.152:0/3941475442 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb05003f070 con 0x5607572dcb40 2024-09-11T23:02:03.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb03effd640 1 -- 172.21.15.152:0/3941475442 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05005b070 con 0x5607572dcb40 2024-09-11T23:02:03.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.778+0000 7fb03effd640 1 -- 172.21.15.152:0/3941475442 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb050004fe0 con 0x5607572dcb40 2024-09-11T23:02:03.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.779+0000 7fb05d79b740 1 -- 172.21.15.152:0/3941475442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572dcb40 msgr2=0x5607572e1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:03.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.779+0000 7fb05d79b740 1 --2- 172.21.15.152:0/3941475442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572dcb40 0x5607572e1830 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fb05002a4c0 tx=0x7fb050005bd0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.779+0000 7fb05d79b740 1 -- 172.21.15.152:0/3941475442 shutdown_connections 2024-09-11T23:02:03.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.779+0000 7fb05d79b740 1 --2- 172.21.15.152:0/3941475442 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572e6370 0x5607572e1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.779+0000 7fb05d79b740 1 --2- 172.21.15.152:0/3941475442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572dcb40 0x5607572e1830 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.779+0000 7fb05d79b740 1 --2- 172.21.15.152:0/3941475442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572db5d0 0x5607572f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.779+0000 7fb05d79b740 1 -- 172.21.15.152:0/3941475442 >> 172.21.15.152:0/3941475442 conn(0x5607572a4310 msgr2=0x5607572de290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:03.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.779+0000 7fb05d79b740 1 -- 172.21.15.152:0/3941475442 shutdown_connections 2024-09-11T23:02:03.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.779+0000 7fb05d79b740 1 -- 172.21.15.152:0/3941475442 wait complete. 2024-09-11T23:02:03.791 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.779+0000 7fb05d79b740 1 -- start start 2024-09-11T23:02:03.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.779+0000 7fb05d79b740 1 Processor -- start 2024-09-11T23:02:03.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05d79b740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572db5d0 0x5607572ee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05d79b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572dcb40 0x5607572eed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05d79b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572e6370 0x5607572e9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05d79b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541aecd0 con 0x5607572e6370 2024-09-11T23:02:03.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05d79b740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541ae600 con 0x5607572db5d0 2024-09-11T23:02:03.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05d79b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541ab860 con 0x5607572dcb40 2024-09-11T23:02:03.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05d79b740 10 client.0 _ll_register_callbacks cb 0x7fb04c0126f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:03.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05995f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572e6370 0x5607572e9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05995f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572e6370 0x5607572e9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52412/0 (socket says 172.21.15.152:52412) 2024-09-11T23:02:03.792 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05995f640 1 -- 172.21.15.152:0/2644365038 learned_addr learned my addr 172.21.15.152:0/2644365038 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:03.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05895d640 1 --2- 172.21.15.152:0/2644365038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572dcb40 0x5607572eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05915e640 1 --2- 172.21.15.152:0/2644365038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572db5d0 0x5607572ee7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05995f640 1 -- 172.21.15.152:0/2644365038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572dcb40 msgr2=0x5607572eed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:03.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05995f640 1 --2- 172.21.15.152:0/2644365038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572dcb40 0x5607572eed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05995f640 1 -- 172.21.15.152:0/2644365038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572db5d0 msgr2=0x5607572ee7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:03.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05995f640 1 --2- 172.21.15.152:0/2644365038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572db5d0 0x5607572ee7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05995f640 1 -- 172.21.15.152:0/2644365038 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607572ea580 con 0x5607572e6370 2024-09-11T23:02:03.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.780+0000 7fb05895d640 1 --2- 172.21.15.152:0/2644365038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572dcb40 0x5607572eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:03.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.781+0000 7fb05995f640 1 --2- 172.21.15.152:0/2644365038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572e6370 0x5607572e9d50 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fb04c005fd0 tx=0x7fb04c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cef4adef07c72da server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:03.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.781+0000 7fb03cff9640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:03.793 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.781+0000 7fb03cff9640 1 -- 172.21.15.152:0/2644365038 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c010b40 con 0x5607572e6370 2024-09-11T23:02:03.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.781+0000 7fb03cff9640 1 -- 172.21.15.152:0/2644365038 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7fb04c01b070 con 0x5607572e6370 2024-09-11T23:02:03.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.781+0000 7fb05d79b740 1 -- 172.21.15.152:0/2644365038 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb05005b070 con 0x5607572e6370 2024-09-11T23:02:03.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.781+0000 7fb03cff9640 1 -- 172.21.15.152:0/2644365038 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb04c036050 con 0x5607572e6370 2024-09-11T23:02:03.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.781+0000 7fb05d79b740 10 client.18030 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:03.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.781+0000 7fb05d79b740 1 -- 172.21.15.152:0/2644365038 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5607573602e0 con 0x5607572e6370 2024-09-11T23:02:03.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.781+0000 7fb03cff9640 1 -- 172.21.15.152:0/2644365038 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05005b070 con 0x5607572e6370 2024-09-11T23:02:03.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.781+0000 7fb03cff9640 1 -- 172.21.15.152:0/2644365038 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb04c005440 con 0x5607572e6370 2024-09-11T23:02:03.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.781+0000 7fb05d79b740 10 client.18030 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:03.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.781+0000 7fb05d79b740 10 client.18030 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:02:03.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.781+0000 7fb05d79b740 20 client.18030 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:03.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.781+0000 7fb05d79b740 1 --2- 172.21.15.152:0/2644365038 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x560757360a80 0x560757362ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:03.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.781+0000 7fb05d79b740 4 client.18030 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:02:03.794 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.781+0000 7fb05d79b740 1 -- 172.21.15.152:0/2644365038 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560757248d70 con 0x560757360a80 2024-09-11T23:02:03.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.782+0000 7fb05915e640 1 --2- 172.21.15.152:0/2644365038 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x560757360a80 0x560757362ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:03.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.782+0000 7fb03cff9640 10 client.18030 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T23:02:03.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.782+0000 7fb05915e640 1 --2- 172.21.15.152:0/2644365038 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x560757360a80 0x560757362ea0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7dd2244ce753ae90 server_cookie=46b9ee6596be6ab in_seq=0 out_seq=0 2024-09-11T23:02:03.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.782+0000 7fb03cff9640 1 -- 172.21.15.152:0/2644365038 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560757248d70 con 0x560757360a80 2024-09-11T23:02:03.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.782+0000 7fb03cff9640 10 client.18030 handle_command_reply: tid=0 2024-09-11T23:02:03.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.782+0000 7fb05d79b740 1 client.18030 shutdown 2024-09-11T23:02:03.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.782+0000 7fb05d79b740 1 -- 172.21.15.152:0/2644365038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572e6370 msgr2=0x5607572e9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:03.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.782+0000 7fb05d79b740 1 --2- 172.21.15.152:0/2644365038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572e6370 0x5607572e9d50 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fb04c005fd0 tx=0x7fb04c004ab0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.782+0000 7fb05d79b740 1 -- 172.21.15.152:0/2644365038 shutdown_connections 2024-09-11T23:02:03.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.782+0000 7fb05d79b740 1 --2- 172.21.15.152:0/2644365038 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x560757360a80 0x560757362ea0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.782+0000 7fb05d79b740 1 --2- 172.21.15.152:0/2644365038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607572e6370 0x5607572e9d50 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.782+0000 7fb05d79b740 1 --2- 172.21.15.152:0/2644365038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5607572dcb40 0x5607572eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.782+0000 7fb05d79b740 1 --2- 172.21.15.152:0/2644365038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5607572db5d0 0x5607572ee7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.795 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.782+0000 7fb05d79b740 1 -- 172.21.15.152:0/2644365038 >> 172.21.15.152:0/2644365038 conn(0x5607572a4310 msgr2=0x5607572d6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:03.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.782+0000 7fb03cff9640 0 client.18030 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T23:02:03.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.783+0000 7fb05d79b740 1 -- 172.21.15.152:0/2644365038 shutdown_connections 2024-09-11T23:02:03.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.783+0000 7fb05d79b740 1 -- 172.21.15.152:0/2644365038 wait complete. 2024-09-11T23:02:03.796 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.783+0000 7fb05d79b740 20 client.18030 trim_cache size 0 max 16384 2024-09-11T23:02:03.796 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:02:03.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.783+0000 7fb05b3c2640 1 -- 172.21.15.152:0/1698410910 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb0400a2cf0 msgr2=0x7fb0400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:03.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.783+0000 7fb05b3c2640 1 --2- 172.21.15.152:0/1698410910 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb0400a2cf0 0x7fb0400a51b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009270 tx=0x7fb04c009500 comp rx=0 tx=0).stop 2024-09-11T23:02:03.797 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.783+0000 7fb05b3c2640 1 -- 172.21.15.152:0/1698410910 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb054102da0 msgr2=0x7fb054198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:03.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.783+0000 7fb05b3c2640 1 --2- 172.21.15.152:0/1698410910 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb054102da0 0x7fb054198430 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb04400e9f0 tx=0x7fb04400edf0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.783+0000 7fb05b3c2640 1 -- 172.21.15.152:0/1698410910 shutdown_connections 2024-09-11T23:02:03.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.783+0000 7fb05b3c2640 1 --2- 172.21.15.152:0/1698410910 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb0400a2cf0 0x7fb0400a51b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.783+0000 7fb05b3c2640 1 --2- 172.21.15.152:0/1698410910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb05410c180 0x7fb05419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.783+0000 7fb05b3c2640 1 --2- 172.21.15.152:0/1698410910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0541036c0 0x7fb054198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.798 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.783+0000 7fb05b3c2640 1 --2- 172.21.15.152:0/1698410910 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb054102da0 0x7fb054198430 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:03.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.783+0000 7fb05b3c2640 1 -- 172.21.15.152:0/1698410910 >> 172.21.15.152:0/1698410910 conn(0x7fb0540fc7a0 msgr2=0x7fb0540fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:03.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.783+0000 7fb05b3c2640 1 -- 172.21.15.152:0/1698410910 shutdown_connections 2024-09-11T23:02:03.799 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:03.783+0000 7fb05b3c2640 1 -- 172.21.15.152:0/1698410910 wait complete. 2024-09-11T23:02:04.645 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:02:04.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:04 smithi152 ceph-mon[31181]: pgmap v282: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T23:02:04.895 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:05.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:04 smithi191 ceph-mon[35211]: pgmap v282: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T23:02:05.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:04 smithi204 ceph-mon[35454]: pgmap v282: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T23:02:05.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.400+0000 7fd9866dd640 1 -- 172.21.15.152:0/234932863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd980102ef0 msgr2=0x7fd98010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:05.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.400+0000 7fd9866dd640 1 --2- 172.21.15.152:0/234932863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd980102ef0 0x7fd98010da40 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fd978009ca0 tx=0x7fd97802a950 comp rx=0 tx=0).stop 2024-09-11T23:02:05.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.400+0000 7fd9866dd640 1 -- 172.21.15.152:0/234932863 shutdown_connections 2024-09-11T23:02:05.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.400+0000 7fd9866dd640 1 --2- 172.21.15.152:0/234932863 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd980102ef0 0x7fd98010da40 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.400+0000 7fd9866dd640 1 --2- 172.21.15.152:0/234932863 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd980102570 0x7fd9801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.400+0000 7fd9866dd640 1 --2- 172.21.15.152:0/234932863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd980100e20 0x7fd980101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.437 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.400+0000 7fd9866dd640 1 -- 172.21.15.152:0/234932863 >> 172.21.15.152:0/234932863 conn(0x7fd9800fc7d0 msgr2=0x7fd9800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:05.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.400+0000 7fd9866dd640 1 -- 172.21.15.152:0/234932863 shutdown_connections 2024-09-11T23:02:05.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.400+0000 7fd9866dd640 1 -- 172.21.15.152:0/234932863 wait complete. 2024-09-11T23:02:05.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd9866dd640 1 Processor -- start 2024-09-11T23:02:05.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd9866dd640 1 -- start start 2024-09-11T23:02:05.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd9866dd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd980100e20 0x7fd980198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd9866dd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd980102570 0x7fd9801989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd9866dd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd980102ef0 0x7fd98019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd9866dd640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd980110f80 con 0x7fd980102570 2024-09-11T23:02:05.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd9866dd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd980110e00 con 0x7fd980100e20 2024-09-11T23:02:05.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd9866dd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd980111100 con 0x7fd980102ef0 2024-09-11T23:02:05.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd977fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd980102570 0x7fd9801989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd977fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd980102570 0x7fd9801989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52434/0 (socket says 172.21.15.152:52434) 2024-09-11T23:02:05.438 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd977fff640 1 -- 172.21.15.152:0/231251072 learned_addr learned my addr 172.21.15.152:0/231251072 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:05.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd97ffff640 1 --2- 172.21.15.152:0/231251072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd980100e20 0x7fd980198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd984c7a640 1 --2- 172.21.15.152:0/231251072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd980102ef0 0x7fd98019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd977fff640 1 -- 172.21.15.152:0/231251072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd980102ef0 msgr2=0x7fd98019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:05.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd977fff640 1 --2- 172.21.15.152:0/231251072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd980102ef0 0x7fd98019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd977fff640 1 -- 172.21.15.152:0/231251072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd980100e20 msgr2=0x7fd980198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:05.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd977fff640 1 --2- 172.21.15.152:0/231251072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd980100e20 0x7fd980198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd977fff640 1 -- 172.21.15.152:0/231251072 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98019d440 con 0x7fd980102570 2024-09-11T23:02:05.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd97ffff640 1 --2- 172.21.15.152:0/231251072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd980100e20 0x7fd980198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:05.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.401+0000 7fd984c7a640 1 --2- 172.21.15.152:0/231251072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd980102ef0 0x7fd98019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:05.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.402+0000 7fd977fff640 1 --2- 172.21.15.152:0/231251072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd980102570 0x7fd9801989d0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fd96800e9f0 tx=0x7fd96800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:05.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.402+0000 7fd97dffb640 1 -- 172.21.15.152:0/231251072 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96800cd80 con 0x7fd980102570 2024-09-11T23:02:05.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.402+0000 7fd97dffb640 1 -- 172.21.15.152:0/231251072 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd968010910 con 0x7fd980102570 2024-09-11T23:02:05.439 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.402+0000 7fd9866dd640 1 -- 172.21.15.152:0/231251072 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd98019d730 con 0x7fd980102570 2024-09-11T23:02:05.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.402+0000 7fd97dffb640 1 -- 172.21.15.152:0/231251072 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968018990 con 0x7fd980102570 2024-09-11T23:02:05.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.402+0000 7fd9866dd640 1 -- 172.21.15.152:0/231251072 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9801a5010 con 0x7fd980102570 2024-09-11T23:02:05.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.404+0000 7fd97dffb640 1 -- 172.21.15.152:0/231251072 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd968010ac0 con 0x7fd980102570 2024-09-11T23:02:05.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.404+0000 7fd988ab6740 1 Processor -- start 2024-09-11T23:02:05.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.404+0000 7fd988ab6740 1 -- start start 2024-09-11T23:02:05.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd988ab6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98957f50 0x559a989564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd988ab6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a989569e0 0x559a98956e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd988ab6740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a98961780 0x559a98963b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd988ab6740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x559a988b82f0 con 0x559a98961780 2024-09-11T23:02:05.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd988ab6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559a9885fcf0 con 0x559a989569e0 2024-09-11T23:02:05.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd988ab6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559a988b42b0 con 0x559a98957f50 2024-09-11T23:02:05.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd977fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a98961780 0x559a98963b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.440 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd977fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a98961780 0x559a98963b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52440/0 (socket says 172.21.15.152:52440) 2024-09-11T23:02:05.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd977fff640 1 -- 172.21.15.152:0/2041348530 learned_addr learned my addr 172.21.15.152:0/2041348530 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:05.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd984c7a640 1 --2- 172.21.15.152:0/2041348530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98957f50 0x559a989564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd97ffff640 1 --2- 172.21.15.152:0/2041348530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a989569e0 0x559a98956e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd97dffb640 1 --2- 172.21.15.152:0/231251072 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd96c0a2d70 0x7fd96c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd977fff640 1 -- 172.21.15.152:0/2041348530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98957f50 msgr2=0x559a989564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:05.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd977fff640 1 --2- 172.21.15.152:0/2041348530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98957f50 0x559a989564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd984c7a640 1 --2- 172.21.15.152:0/231251072 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd96c0a2d70 0x7fd96c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd977fff640 1 -- 172.21.15.152:0/2041348530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a989569e0 msgr2=0x559a98956e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:05.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd977fff640 1 --2- 172.21.15.152:0/2041348530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a989569e0 0x559a98956e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd977fff640 1 -- 172.21.15.152:0/2041348530 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a988443a0 con 0x559a98961780 2024-09-11T23:02:05.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd97ffff640 1 --2- 172.21.15.152:0/2041348530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a989569e0 0x559a98956e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:05.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd97dffb640 1 -- 172.21.15.152:0/231251072 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7fd9680cf800 con 0x7fd980102570 2024-09-11T23:02:05.441 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.405+0000 7fd977fff640 1 --2- 172.21.15.152:0/2041348530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a98961780 0x559a98963b70 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fd9680ccee0 tx=0x7fd968024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d577a2ef6cf9132f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:05.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.406+0000 7fd9767fc640 1 -- 172.21.15.152:0/2041348530 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9680d2e60 con 0x559a98961780 2024-09-11T23:02:05.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.406+0000 7fd9767fc640 1 -- 172.21.15.152:0/2041348530 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd96809c440 con 0x559a98961780 2024-09-11T23:02:05.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.406+0000 7fd984c7a640 1 --2- 172.21.15.152:0/231251072 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd96c0a2d70 0x7fd96c0a5230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd978030040 tx=0x7fd97802a950 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:05.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.406+0000 7fd9767fc640 1 -- 172.21.15.152:0/2041348530 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9680259d0 con 0x559a98961780 2024-09-11T23:02:05.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.406+0000 7fd988ab6740 1 -- 172.21.15.152:0/2041348530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a98961780 msgr2=0x559a98963b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:05.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.406+0000 7fd988ab6740 1 --2- 172.21.15.152:0/2041348530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a98961780 0x559a98963b70 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fd9680ccee0 tx=0x7fd968024000 comp rx=0 tx=0).stop 2024-09-11T23:02:05.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.406+0000 7fd988ab6740 1 -- 172.21.15.152:0/2041348530 shutdown_connections 2024-09-11T23:02:05.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.406+0000 7fd988ab6740 1 --2- 172.21.15.152:0/2041348530 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a98961780 0x559a98963b70 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.406+0000 7fd988ab6740 1 --2- 172.21.15.152:0/2041348530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a989569e0 0x559a98956e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.406+0000 7fd988ab6740 1 --2- 172.21.15.152:0/2041348530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98957f50 0x559a989564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.406+0000 7fd988ab6740 1 -- 172.21.15.152:0/2041348530 >> 172.21.15.152:0/2041348530 conn(0x559a9891b310 msgr2=0x559a98953070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:05.442 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.406+0000 7fd988ab6740 1 -- 172.21.15.152:0/2041348530 shutdown_connections 2024-09-11T23:02:05.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.407+0000 7fd988ab6740 1 -- 172.21.15.152:0/2041348530 wait complete. 2024-09-11T23:02:05.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.407+0000 7fd988ab6740 1 -- start start 2024-09-11T23:02:05.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.407+0000 7fd988ab6740 1 Processor -- start 2024-09-11T23:02:05.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.407+0000 7fd988ab6740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a989569e0 0x559a98961130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.407+0000 7fd988ab6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a98957f50 0x559a9895e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.407+0000 7fd988ab6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98961780 0x559a9895e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.407+0000 7fd988ab6740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x559a98964420 con 0x559a989569e0 2024-09-11T23:02:05.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.407+0000 7fd988ab6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x559a98873060 con 0x559a98957f50 2024-09-11T23:02:05.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.407+0000 7fd988ab6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x559a989645a0 con 0x559a98961780 2024-09-11T23:02:05.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.407+0000 7fd988ab6740 10 client.0 _ll_register_callbacks cb 0x559a985c68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:05.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd97ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a989569e0 0x559a98961130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd97ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a989569e0 0x559a98961130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52446/0 (socket says 172.21.15.152:52446) 2024-09-11T23:02:05.443 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd97ffff640 1 -- 172.21.15.152:0/2982699351 learned_addr learned my addr 172.21.15.152:0/2982699351 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:05.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd977fff640 1 --2- 172.21.15.152:0/2982699351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a98957f50 0x559a9895e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd97ffff640 1 -- 172.21.15.152:0/2982699351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98961780 msgr2=0x559a9895e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:05.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd97ffff640 1 --2- 172.21.15.152:0/2982699351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98961780 0x559a9895e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd97ffff640 1 -- 172.21.15.152:0/2982699351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a98957f50 msgr2=0x559a9895e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:05.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd97ffff640 1 --2- 172.21.15.152:0/2982699351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a98957f50 0x559a9895e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd97ffff640 1 -- 172.21.15.152:0/2982699351 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a986293c0 con 0x559a989569e0 2024-09-11T23:02:05.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd97ffff640 1 --2- 172.21.15.152:0/2982699351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a989569e0 0x559a98961130 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd97000bf80 tx=0x7fd970011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7f8ed138f6e81b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:05.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd95f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:05.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd95f7fe640 1 -- 172.21.15.152:0/2982699351 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970012710 con 0x559a989569e0 2024-09-11T23:02:05.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd95f7fe640 1 -- 172.21.15.152:0/2982699351 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7fd9700128b0 con 0x559a989569e0 2024-09-11T23:02:05.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd988ab6740 1 -- 172.21.15.152:0/2982699351 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd96809c440 con 0x559a989569e0 2024-09-11T23:02:05.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd95f7fe640 1 -- 172.21.15.152:0/2982699351 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd970012ab0 con 0x559a989569e0 2024-09-11T23:02:05.444 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd988ab6740 10 client.18042 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:05.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd988ab6740 1 -- 172.21.15.152:0/2982699351 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559a9896fed0 con 0x559a989569e0 2024-09-11T23:02:05.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.408+0000 7fd95f7fe640 1 -- 172.21.15.152:0/2982699351 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd96809c440 con 0x559a989569e0 2024-09-11T23:02:05.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.409+0000 7fd95f7fe640 1 -- 172.21.15.152:0/2982699351 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd970022070 con 0x559a989569e0 2024-09-11T23:02:05.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.409+0000 7fd988ab6740 10 client.18042 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:05.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.409+0000 7fd988ab6740 10 client.18042 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:02:05.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.409+0000 7fd988ab6740 20 client.18042 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:05.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.409+0000 7fd988ab6740 1 --2- 172.21.15.152:0/2982699351 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x559a98971390 0x559a98973850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.409+0000 7fd988ab6740 4 client.18042 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:02:05.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.409+0000 7fd988ab6740 1 -- 172.21.15.152:0/2982699351 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a988c5860 con 0x559a98971390 2024-09-11T23:02:05.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.409+0000 7fd977fff640 1 --2- 172.21.15.152:0/2982699351 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x559a98971390 0x559a98973850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.409+0000 7fd977fff640 1 --2- 172.21.15.152:0/2982699351 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x559a98971390 0x559a98973850 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1f8f2bfb631e3c8 server_cookie=4e2e83e7e6f32fdb in_seq=0 out_seq=0 2024-09-11T23:02:05.445 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.409+0000 7fd95f7fe640 10 client.18042 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T23:02:05.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.410+0000 7fd95f7fe640 1 -- 172.21.15.152:0/2982699351 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a988c5860 con 0x559a98971390 2024-09-11T23:02:05.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.410+0000 7fd95f7fe640 10 client.18042 handle_command_reply: tid=0 2024-09-11T23:02:05.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.410+0000 7fd988ab6740 1 client.18042 shutdown 2024-09-11T23:02:05.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.410+0000 7fd988ab6740 1 -- 172.21.15.152:0/2982699351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a989569e0 msgr2=0x559a98961130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:05.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.410+0000 7fd988ab6740 1 --2- 172.21.15.152:0/2982699351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a989569e0 0x559a98961130 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd97000bf80 tx=0x7fd970011870 comp rx=0 tx=0).stop 2024-09-11T23:02:05.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.410+0000 7fd988ab6740 1 -- 172.21.15.152:0/2982699351 shutdown_connections 2024-09-11T23:02:05.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.410+0000 7fd988ab6740 1 --2- 172.21.15.152:0/2982699351 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x559a98971390 0x559a98973850 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.410+0000 7fd988ab6740 1 --2- 172.21.15.152:0/2982699351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98961780 0x559a9895e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.410+0000 7fd988ab6740 1 --2- 172.21.15.152:0/2982699351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a98957f50 0x559a9895e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.410+0000 7fd988ab6740 1 --2- 172.21.15.152:0/2982699351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a989569e0 0x559a98961130 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.410+0000 7fd95f7fe640 0 client.18042 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T23:02:05.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.410+0000 7fd988ab6740 1 -- 172.21.15.152:0/2982699351 >> 172.21.15.152:0/2982699351 conn(0x559a9891b310 msgr2=0x559a98958d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:05.446 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.410+0000 7fd988ab6740 1 -- 172.21.15.152:0/2982699351 shutdown_connections 2024-09-11T23:02:05.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.410+0000 7fd988ab6740 1 -- 172.21.15.152:0/2982699351 wait complete. 2024-09-11T23:02:05.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.411+0000 7fd988ab6740 20 client.18042 trim_cache size 0 max 16384 2024-09-11T23:02:05.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.423+0000 7fd988ab6740 1 Processor -- start 2024-09-11T23:02:05.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.423+0000 7fd988ab6740 1 -- start start 2024-09-11T23:02:05.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd988ab6740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a989569e0 0x559a98953220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd988ab6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a98957f50 0x559a9896d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd988ab6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98961780 0x559a98979170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd988ab6740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd9801ab650 con 0x559a989569e0 2024-09-11T23:02:05.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd988ab6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd9801ad190 con 0x559a98957f50 2024-09-11T23:02:05.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd988ab6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd9801abd20 con 0x559a98961780 2024-09-11T23:02:05.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd97ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a989569e0 0x559a98953220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd97ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a989569e0 0x559a98953220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52452/0 (socket says 172.21.15.152:52452) 2024-09-11T23:02:05.447 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd97ffff640 1 -- 172.21.15.152:0/1839224268 learned_addr learned my addr 172.21.15.152:0/1839224268 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:05.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd984c7a640 1 --2- 172.21.15.152:0/1839224268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98961780 0x559a98979170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd977fff640 1 --2- 172.21.15.152:0/1839224268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a98957f50 0x559a9896d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd97ffff640 1 -- 172.21.15.152:0/1839224268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98961780 msgr2=0x559a98979170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:05.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd97ffff640 1 --2- 172.21.15.152:0/1839224268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98961780 0x559a98979170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd97ffff640 1 -- 172.21.15.152:0/1839224268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a98957f50 msgr2=0x559a9896d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:05.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd97ffff640 1 --2- 172.21.15.152:0/1839224268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a98957f50 0x559a9896d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd97ffff640 1 -- 172.21.15.152:0/1839224268 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a9895c950 con 0x559a989569e0 2024-09-11T23:02:05.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd97ffff640 1 --2- 172.21.15.152:0/1839224268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a989569e0 0x559a98953220 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd970005710 tx=0x7fd97000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25333647f5bfc3f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:05.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd975ffb640 1 -- 172.21.15.152:0/1839224268 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970031070 con 0x559a989569e0 2024-09-11T23:02:05.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd975ffb640 1 -- 172.21.15.152:0/1839224268 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd970012e50 con 0x559a989569e0 2024-09-11T23:02:05.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.424+0000 7fd975ffb640 1 -- 172.21.15.152:0/1839224268 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970028040 con 0x559a989569e0 2024-09-11T23:02:05.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.425+0000 7fd988ab6740 1 -- 172.21.15.152:0/1839224268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a989569e0 msgr2=0x559a98953220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:05.448 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.425+0000 7fd988ab6740 1 --2- 172.21.15.152:0/1839224268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a989569e0 0x559a98953220 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fd970005710 tx=0x7fd97000bfd0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.425+0000 7fd988ab6740 1 -- 172.21.15.152:0/1839224268 shutdown_connections 2024-09-11T23:02:05.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.425+0000 7fd988ab6740 1 --2- 172.21.15.152:0/1839224268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98961780 0x559a98979170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.425+0000 7fd988ab6740 1 --2- 172.21.15.152:0/1839224268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a98957f50 0x559a9896d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.425+0000 7fd988ab6740 1 --2- 172.21.15.152:0/1839224268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a989569e0 0x559a98953220 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.425+0000 7fd988ab6740 1 -- 172.21.15.152:0/1839224268 >> 172.21.15.152:0/1839224268 conn(0x559a9891b310 msgr2=0x559a98952b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:05.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.425+0000 7fd988ab6740 1 -- 172.21.15.152:0/1839224268 shutdown_connections 2024-09-11T23:02:05.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.425+0000 7fd988ab6740 1 -- 172.21.15.152:0/1839224268 wait complete. 2024-09-11T23:02:05.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.425+0000 7fd988ab6740 1 -- start start 2024-09-11T23:02:05.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.425+0000 7fd988ab6740 1 Processor -- start 2024-09-11T23:02:05.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd988ab6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a989569e0 0x559a98969830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd988ab6740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a98957f50 0x559a98969d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd988ab6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98961780 0x559a98966800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.449 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd988ab6740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd9801acac0 con 0x559a98957f50 2024-09-11T23:02:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd988ab6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd9801ac3f0 con 0x559a989569e0 2024-09-11T23:02:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd988ab6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd9801a9670 con 0x559a98961780 2024-09-11T23:02:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd988ab6740 10 client.0 _ll_register_callbacks cb 0x7fd97000b4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd977fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a98957f50 0x559a98969d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd977fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a98957f50 0x559a98969d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52458/0 (socket says 172.21.15.152:52458) 2024-09-11T23:02:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd977fff640 1 -- 172.21.15.152:0/650555260 learned_addr learned my addr 172.21.15.152:0/650555260 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd97ffff640 1 --2- 172.21.15.152:0/650555260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a989569e0 0x559a98969830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd984c7a640 1 --2- 172.21.15.152:0/650555260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98961780 0x559a98966800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd977fff640 1 -- 172.21.15.152:0/650555260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98961780 msgr2=0x559a98966800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd977fff640 1 --2- 172.21.15.152:0/650555260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98961780 0x559a98966800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd977fff640 1 -- 172.21.15.152:0/650555260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a989569e0 msgr2=0x559a98969830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:05.450 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd977fff640 1 --2- 172.21.15.152:0/650555260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a989569e0 0x559a98969830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd977fff640 1 -- 172.21.15.152:0/650555260 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a98966d40 con 0x559a98957f50 2024-09-11T23:02:05.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd977fff640 1 --2- 172.21.15.152:0/650555260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a98957f50 0x559a98969d70 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd9680981a0 tx=0x7fd968018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72a689285c8f11a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:05.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd95f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:05.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.426+0000 7fd95f7fe640 1 -- 172.21.15.152:0/650555260 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96809c440 con 0x559a98957f50 2024-09-11T23:02:05.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.427+0000 7fd95f7fe640 1 -- 172.21.15.152:0/650555260 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7fd96809a040 con 0x559a98957f50 2024-09-11T23:02:05.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.427+0000 7fd988ab6740 1 -- 172.21.15.152:0/650555260 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd970012e50 con 0x559a98957f50 2024-09-11T23:02:05.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.427+0000 7fd95f7fe640 1 -- 172.21.15.152:0/650555260 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd96809bd40 con 0x559a98957f50 2024-09-11T23:02:05.451 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.427+0000 7fd988ab6740 10 client.18048 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.427+0000 7fd988ab6740 1 -- 172.21.15.152:0/650555260 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559a989dc810 con 0x559a98957f50 2024-09-11T23:02:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.427+0000 7fd95f7fe640 1 -- 172.21.15.152:0/650555260 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd970012e50 con 0x559a98957f50 2024-09-11T23:02:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.427+0000 7fd95f7fe640 1 -- 172.21.15.152:0/650555260 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd9680983b0 con 0x559a98957f50 2024-09-11T23:02:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.427+0000 7fd988ab6740 10 client.18048 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.427+0000 7fd988ab6740 10 client.18048 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:02:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.427+0000 7fd988ab6740 20 client.18048 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.427+0000 7fd988ab6740 1 --2- 172.21.15.152:0/650555260 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x559a989dcbb0 0x559a989defa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.427+0000 7fd988ab6740 4 client.18048 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:02:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.427+0000 7fd988ab6740 1 -- 172.21.15.152:0/650555260 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a988bfd70 con 0x559a989dcbb0 2024-09-11T23:02:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.427+0000 7fd97ffff640 1 --2- 172.21.15.152:0/650555260 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x559a989dcbb0 0x559a989defa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd97ffff640 1 --2- 172.21.15.152:0/650555260 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x559a989dcbb0 0x559a989defa0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70c50b396db5670a server_cookie=4d681c5b269c194d in_seq=0 out_seq=0 2024-09-11T23:02:05.452 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd95f7fe640 10 client.18048 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T23:02:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd95f7fe640 1 -- 172.21.15.152:0/650555260 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a988bfd70 con 0x559a989dcbb0 2024-09-11T23:02:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd95f7fe640 10 client.18048 handle_command_reply: tid=0 2024-09-11T23:02:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd988ab6740 1 client.18048 shutdown 2024-09-11T23:02:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd988ab6740 1 -- 172.21.15.152:0/650555260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a98957f50 msgr2=0x559a98969d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd988ab6740 1 --2- 172.21.15.152:0/650555260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a98957f50 0x559a98969d70 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fd9680981a0 tx=0x7fd968018420 comp rx=0 tx=0).stop 2024-09-11T23:02:05.453 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:02:05.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd988ab6740 1 -- 172.21.15.152:0/650555260 shutdown_connections 2024-09-11T23:02:05.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd988ab6740 1 --2- 172.21.15.152:0/650555260 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x559a989dcbb0 0x559a989defa0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd988ab6740 1 --2- 172.21.15.152:0/650555260 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559a98961780 0x559a98966800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd988ab6740 1 --2- 172.21.15.152:0/650555260 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a98957f50 0x559a98969d70 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd95f7fe640 0 client.18048 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T23:02:05.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd988ab6740 1 --2- 172.21.15.152:0/650555260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x559a989569e0 0x559a98969830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd988ab6740 1 -- 172.21.15.152:0/650555260 >> 172.21.15.152:0/650555260 conn(0x559a9891b310 msgr2=0x559a98958520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:05.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd988ab6740 1 -- 172.21.15.152:0/650555260 shutdown_connections 2024-09-11T23:02:05.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd988ab6740 1 -- 172.21.15.152:0/650555260 wait complete. 2024-09-11T23:02:05.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.428+0000 7fd988ab6740 20 client.18048 trim_cache size 0 max 16384 2024-09-11T23:02:05.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.429+0000 7fd9866dd640 1 -- 172.21.15.152:0/231251072 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd96c0a2d70 msgr2=0x7fd96c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:05.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.429+0000 7fd9866dd640 1 --2- 172.21.15.152:0/231251072 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd96c0a2d70 0x7fd96c0a5230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd978030040 tx=0x7fd97802a950 comp rx=0 tx=0).stop 2024-09-11T23:02:05.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.429+0000 7fd9866dd640 1 -- 172.21.15.152:0/231251072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd980102570 msgr2=0x7fd9801989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.429+0000 7fd9866dd640 1 --2- 172.21.15.152:0/231251072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd980102570 0x7fd9801989d0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fd96800e9f0 tx=0x7fd96800eeb0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.429+0000 7fd9866dd640 1 -- 172.21.15.152:0/231251072 shutdown_connections 2024-09-11T23:02:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.429+0000 7fd9866dd640 1 --2- 172.21.15.152:0/231251072 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd96c0a2d70 0x7fd96c0a5230 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.429+0000 7fd9866dd640 1 --2- 172.21.15.152:0/231251072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd980102ef0 0x7fd98019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.429+0000 7fd9866dd640 1 --2- 172.21.15.152:0/231251072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd980102570 0x7fd9801989d0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.429+0000 7fd9866dd640 1 --2- 172.21.15.152:0/231251072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd980100e20 0x7fd980198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.429+0000 7fd9866dd640 1 -- 172.21.15.152:0/231251072 >> 172.21.15.152:0/231251072 conn(0x7fd9800fc7d0 msgr2=0x7fd98006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.429+0000 7fd9866dd640 1 -- 172.21.15.152:0/231251072 shutdown_connections 2024-09-11T23:02:05.455 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:05.429+0000 7fd9866dd640 1 -- 172.21.15.152:0/231251072 wait complete. 2024-09-11T23:02:06.355 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:02:06.605 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:06.956 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:06 smithi152 ceph-mon[31181]: pgmap v283: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-11T23:02:07.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:06 smithi191 ceph-mon[35211]: pgmap v283: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-11T23:02:07.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:06 smithi204 ceph-mon[35454]: pgmap v283: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-11T23:02:07.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.185+0000 7f38b7faf640 1 -- 172.21.15.152:0/3292749060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b0101de0 msgr2=0x7f38b01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:07.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.185+0000 7f38b7faf640 1 --2- 172.21.15.152:0/3292749060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b0101de0 0x7f38b01021c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009900 tx=0x7f38ac02a410 comp rx=0 tx=0).stop 2024-09-11T23:02:07.205 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.185+0000 7f38b7faf640 1 -- 172.21.15.152:0/3292749060 shutdown_connections 2024-09-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.185+0000 7f38b7faf640 1 --2- 172.21.15.152:0/3292749060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38b0106f10 0x7f38b010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.185+0000 7f38b7faf640 1 --2- 172.21.15.152:0/3292749060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38b0102790 0x7f38b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.185+0000 7f38b7faf640 1 --2- 172.21.15.152:0/3292749060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b0101de0 0x7f38b01021c0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.185+0000 7f38b7faf640 1 -- 172.21.15.152:0/3292749060 >> 172.21.15.152:0/3292749060 conn(0x7f38b00fd8c0 msgr2=0x7f38b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.186+0000 7f38b7faf640 1 -- 172.21.15.152:0/3292749060 shutdown_connections 2024-09-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.186+0000 7f38b7faf640 1 -- 172.21.15.152:0/3292749060 wait complete. 2024-09-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.186+0000 7f38b7faf640 1 Processor -- start 2024-09-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.186+0000 7f38b7faf640 1 -- start start 2024-09-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b7faf640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38b0101de0 0x7f38b01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b7faf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b0102790 0x7f38b0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b7faf640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38b0106f10 0x7f38b019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b7faf640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f38b0112010 con 0x7f38b0102790 2024-09-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b7faf640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f38b0111e90 con 0x7f38b0101de0 2024-09-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b7faf640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f38b0112190 con 0x7f38b0106f10 2024-09-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b554a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b0102790 0x7f38b0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b554a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b0102790 0x7f38b0198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52486/0 (socket says 172.21.15.152:52486) 2024-09-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b554a640 1 -- 172.21.15.152:0/642114027 learned_addr learned my addr 172.21.15.152:0/642114027 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b5d4b640 1 --2- 172.21.15.152:0/642114027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38b0101de0 0x7f38b01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b654c640 1 --2- 172.21.15.152:0/642114027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38b0106f10 0x7f38b019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b554a640 1 -- 172.21.15.152:0/642114027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38b0106f10 msgr2=0x7f38b019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b554a640 1 --2- 172.21.15.152:0/642114027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38b0106f10 0x7f38b019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b554a640 1 -- 172.21.15.152:0/642114027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38b0101de0 msgr2=0x7f38b01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b554a640 1 --2- 172.21.15.152:0/642114027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38b0101de0 0x7f38b01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b554a640 1 -- 172.21.15.152:0/642114027 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b019d560 con 0x7f38b0102790 2024-09-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b654c640 1 --2- 172.21.15.152:0/642114027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38b0106f10 0x7f38b019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b554a640 1 --2- 172.21.15.152:0/642114027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b0102790 0x7f38b0198af0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f38a000b4a0 tx=0x7f38a000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38a6ffd640 1 -- 172.21.15.152:0/642114027 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0014070 con 0x7f38b0102790 2024-09-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38a6ffd640 1 -- 172.21.15.152:0/642114027 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38a0010af0 con 0x7f38b0102790 2024-09-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38b7faf640 1 -- 172.21.15.152:0/642114027 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b019d850 con 0x7f38b0102790 2024-09-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.187+0000 7f38a6ffd640 1 -- 172.21.15.152:0/642114027 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0017d10 con 0x7f38b0102790 2024-09-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.188+0000 7f38b7faf640 1 -- 172.21.15.152:0/642114027 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b01a5130 con 0x7f38b0102790 2024-09-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.190+0000 7f38a6ffd640 1 -- 172.21.15.152:0/642114027 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f38a0017070 con 0x7f38b0102790 2024-09-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.190+0000 7f38ba388740 1 Processor -- start 2024-09-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.190+0000 7f38ba388740 1 -- start start 2024-09-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38ba388740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a5948b40 0x5610a5947090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38ba388740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a59475d0 0x5610a5947a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38ba388740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 0x5610a5954760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38ba388740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5610a58ad2f0 con 0x5610a59475d0 2024-09-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38ba388740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5610a5854cf0 con 0x5610a5952370 2024-09-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38ba388740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5610a58a92b0 con 0x5610a5948b40 2024-09-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38b5d4b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a59475d0 0x5610a5947a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38b5d4b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a59475d0 0x5610a5947a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52498/0 (socket says 172.21.15.152:52498) 2024-09-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38b5d4b640 1 -- 172.21.15.152:0/3791077213 learned_addr learned my addr 172.21.15.152:0/3791077213 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38b654c640 1 --2- 172.21.15.152:0/3791077213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a5948b40 0x5610a5947090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38b554a640 1 --2- 172.21.15.152:0/3791077213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 0x5610a5954760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38a6ffd640 1 --2- 172.21.15.152:0/642114027 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f389c0a2d70 0x7f389c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38b5d4b640 1 -- 172.21.15.152:0/3791077213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a5948b40 msgr2=0x5610a5947090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38b5d4b640 1 --2- 172.21.15.152:0/3791077213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a5948b40 0x5610a5947090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38b5d4b640 1 -- 172.21.15.152:0/3791077213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 msgr2=0x5610a5954760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38b5d4b640 1 --2- 172.21.15.152:0/3791077213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 0x5610a5954760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38b5d4b640 1 -- 172.21.15.152:0/3791077213 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a58393a0 con 0x5610a59475d0 2024-09-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38b554a640 1 --2- 172.21.15.152:0/3791077213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 0x5610a5954760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38b654c640 1 --2- 172.21.15.152:0/642114027 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f389c0a2d70 0x7f389c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.191+0000 7f38a6ffd640 1 -- 172.21.15.152:0/642114027 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f38a00ce670 con 0x7f38b0102790 2024-09-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.192+0000 7f38b5d4b640 1 --2- 172.21.15.152:0/3791077213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a59475d0 0x5610a5947a10 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009900 tx=0x7f38ac0316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546d69bf64beed0e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.192+0000 7f38b654c640 1 --2- 172.21.15.152:0/642114027 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f389c0a2d70 0x7f389c0a5230 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f38a800f4f0 tx=0x7f38a8004b50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.192+0000 7f389bfff640 1 -- 172.21.15.152:0/3791077213 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac040070 con 0x5610a59475d0 2024-09-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.192+0000 7f389bfff640 1 -- 172.21.15.152:0/3791077213 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38ac005020 con 0x5610a59475d0 2024-09-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.192+0000 7f389bfff640 1 -- 172.21.15.152:0/3791077213 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac035040 con 0x5610a59475d0 2024-09-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.192+0000 7f38ba388740 1 -- 172.21.15.152:0/3791077213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a59475d0 msgr2=0x5610a5947a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.192+0000 7f38ba388740 1 --2- 172.21.15.152:0/3791077213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a59475d0 0x5610a5947a10 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009900 tx=0x7f38ac0316e0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.192+0000 7f38ba388740 1 -- 172.21.15.152:0/3791077213 shutdown_connections 2024-09-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.193+0000 7f38ba388740 1 --2- 172.21.15.152:0/3791077213 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 0x5610a5954760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.193+0000 7f38ba388740 1 --2- 172.21.15.152:0/3791077213 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a59475d0 0x5610a5947a10 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.193+0000 7f38ba388740 1 --2- 172.21.15.152:0/3791077213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a5948b40 0x5610a5947090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.193+0000 7f38ba388740 1 -- 172.21.15.152:0/3791077213 >> 172.21.15.152:0/3791077213 conn(0x5610a5910310 msgr2=0x5610a5943c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.193+0000 7f38ba388740 1 -- 172.21.15.152:0/3791077213 shutdown_connections 2024-09-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.193+0000 7f38ba388740 1 -- 172.21.15.152:0/3791077213 wait complete. 2024-09-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.193+0000 7f38ba388740 1 -- start start 2024-09-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.193+0000 7f38ba388740 1 Processor -- start 2024-09-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.193+0000 7f38ba388740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a59475d0 0x5610a5951d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.193+0000 7f38ba388740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5948b40 0x5610a594efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.193+0000 7f38ba388740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 0x5610a594f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.193+0000 7f38ba388740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5610a5955010 con 0x5610a5948b40 2024-09-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.193+0000 7f38ba388740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5610a5868060 con 0x5610a5952370 2024-09-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.193+0000 7f38ba388740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5610a5955190 con 0x5610a59475d0 2024-09-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.193+0000 7f38ba388740 10 client.0 _ll_register_callbacks cb 0x5610a55bb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38b554a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5948b40 0x5610a594efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38b554a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5948b40 0x5610a594efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52510/0 (socket says 172.21.15.152:52510) 2024-09-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38b554a640 1 -- 172.21.15.152:0/1039173710 learned_addr learned my addr 172.21.15.152:0/1039173710 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38b5d4b640 1 --2- 172.21.15.152:0/1039173710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a59475d0 0x5610a5951d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38b654c640 1 --2- 172.21.15.152:0/1039173710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 0x5610a594f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38b554a640 1 -- 172.21.15.152:0/1039173710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a59475d0 msgr2=0x5610a5951d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38b554a640 1 --2- 172.21.15.152:0/1039173710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a59475d0 0x5610a5951d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38b554a640 1 -- 172.21.15.152:0/1039173710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 msgr2=0x5610a594f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38b554a640 1 --2- 172.21.15.152:0/1039173710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 0x5610a594f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38b554a640 1 -- 172.21.15.152:0/1039173710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a561e3c0 con 0x5610a5948b40 2024-09-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38b654c640 1 --2- 172.21.15.152:0/1039173710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 0x5610a594f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38b554a640 1 --2- 172.21.15.152:0/1039173710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5948b40 0x5610a594efb0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f38a000be60 tx=0x7f38a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88687732e3712f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38997fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38997fa640 1 -- 172.21.15.152:0/1039173710 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a001e050 con 0x5610a5948b40 2024-09-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38997fa640 1 -- 172.21.15.152:0/1039173710 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f38a00972d0 con 0x5610a5948b40 2024-09-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38ba388740 1 -- 172.21.15.152:0/1039173710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38ac005020 con 0x5610a5948b40 2024-09-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38997fa640 1 -- 172.21.15.152:0/1039173710 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f38a00b3e60 con 0x5610a5948b40 2024-09-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38ba388740 10 client.18060 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38ba388740 1 -- 172.21.15.152:0/1039173710 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610a5964eb0 con 0x5610a5948b40 2024-09-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.194+0000 7f38997fa640 1 -- 172.21.15.152:0/1039173710 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38a00b0020 con 0x5610a5948b40 2024-09-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.195+0000 7f38997fa640 1 -- 172.21.15.152:0/1039173710 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f38a00cc000 con 0x5610a5948b40 2024-09-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.195+0000 7f38ba388740 10 client.18060 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.195+0000 7f38ba388740 10 client.18060 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.195+0000 7f38ba388740 20 client.18060 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.195+0000 7f38ba388740 1 --2- 172.21.15.152:0/1039173710 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5610a59663c0 0x5610a59687b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.195+0000 7f38ba388740 4 client.18060 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.195+0000 7f38ba388740 1 -- 172.21.15.152:0/1039173710 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610a58ba860 con 0x5610a59663c0 2024-09-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.195+0000 7f38b5d4b640 1 --2- 172.21.15.152:0/1039173710 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5610a59663c0 0x5610a59687b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.195+0000 7f38b5d4b640 1 --2- 172.21.15.152:0/1039173710 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5610a59663c0 0x5610a59687b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d88edab6b33a4c86 server_cookie=de5153b4e15b45da in_seq=0 out_seq=0 2024-09-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.195+0000 7f38997fa640 10 client.18060 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.196+0000 7f38997fa640 1 -- 172.21.15.152:0/1039173710 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610a58ba860 con 0x5610a59663c0 2024-09-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.196+0000 7f38997fa640 10 client.18060 handle_command_reply: tid=0 2024-09-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.196+0000 7f38ba388740 1 client.18060 shutdown 2024-09-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.196+0000 7f38ba388740 1 -- 172.21.15.152:0/1039173710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5948b40 msgr2=0x5610a594efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.196+0000 7f38ba388740 1 --2- 172.21.15.152:0/1039173710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5948b40 0x5610a594efb0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f38a000be60 tx=0x7f38a00046c0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.196+0000 7f38ba388740 1 -- 172.21.15.152:0/1039173710 shutdown_connections 2024-09-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.196+0000 7f38ba388740 1 --2- 172.21.15.152:0/1039173710 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5610a59663c0 0x5610a59687b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.196+0000 7f38ba388740 1 --2- 172.21.15.152:0/1039173710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 0x5610a594f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.196+0000 7f38ba388740 1 --2- 172.21.15.152:0/1039173710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5948b40 0x5610a594efb0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.196+0000 7f38ba388740 1 --2- 172.21.15.152:0/1039173710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a59475d0 0x5610a5951d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.196+0000 7f38ba388740 1 -- 172.21.15.152:0/1039173710 >> 172.21.15.152:0/1039173710 conn(0x5610a5910310 msgr2=0x5610a5949960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.196+0000 7f38997fa640 0 client.18060 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.196+0000 7f38ba388740 1 -- 172.21.15.152:0/1039173710 shutdown_connections 2024-09-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.196+0000 7f38ba388740 1 -- 172.21.15.152:0/1039173710 wait complete. 2024-09-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.196+0000 7f38ba388740 20 client.18060 trim_cache size 0 max 16384 2024-09-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38ba388740 1 Processor -- start 2024-09-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38ba388740 1 -- start start 2024-09-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38ba388740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a59475d0 0x5610a5943e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38ba388740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5948b40 0x5610a5962740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38ba388740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 0x5610a596e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38ba388740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01ab760 con 0x5610a5948b40 2024-09-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38ba388740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01ad2a0 con 0x5610a5952370 2024-09-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38ba388740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01abe30 con 0x5610a59475d0 2024-09-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38b554a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5948b40 0x5610a5962740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38b554a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5948b40 0x5610a5962740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52516/0 (socket says 172.21.15.152:52516) 2024-09-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38b554a640 1 -- 172.21.15.152:0/3065588946 learned_addr learned my addr 172.21.15.152:0/3065588946 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38b5d4b640 1 --2- 172.21.15.152:0/3065588946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a59475d0 0x5610a5943e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38b554a640 1 -- 172.21.15.152:0/3065588946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a59475d0 msgr2=0x5610a5943e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38b554a640 1 --2- 172.21.15.152:0/3065588946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a59475d0 0x5610a5943e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38b554a640 1 -- 172.21.15.152:0/3065588946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 msgr2=0x5610a596e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38b654c640 1 --2- 172.21.15.152:0/3065588946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 0x5610a596e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38b554a640 1 --2- 172.21.15.152:0/3065588946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 0x5610a596e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38b554a640 1 -- 172.21.15.152:0/3065588946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a594a120 con 0x5610a5948b40 2024-09-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.210+0000 7f38b5d4b640 1 --2- 172.21.15.152:0/3065588946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a59475d0 0x5610a5943e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.211+0000 7f38b554a640 1 --2- 172.21.15.152:0/3065588946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5948b40 0x5610a5962740 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f38a00a33a0 tx=0x7f38a000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e31c22710bd6b14 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.211+0000 7f389b7fe640 1 -- 172.21.15.152:0/3065588946 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a001e070 con 0x5610a5948b40 2024-09-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.211+0000 7f389b7fe640 1 -- 172.21.15.152:0/3065588946 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38a00b0070 con 0x5610a5948b40 2024-09-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.211+0000 7f389b7fe640 1 -- 172.21.15.152:0/3065588946 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a00972d0 con 0x5610a5948b40 2024-09-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.212+0000 7f38ba388740 1 -- 172.21.15.152:0/3065588946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5948b40 msgr2=0x5610a5962740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.212+0000 7f38ba388740 1 --2- 172.21.15.152:0/3065588946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5948b40 0x5610a5962740 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f38a00a33a0 tx=0x7f38a000bf90 comp rx=0 tx=0).stop 2024-09-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.212+0000 7f38ba388740 1 -- 172.21.15.152:0/3065588946 shutdown_connections 2024-09-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.212+0000 7f38ba388740 1 --2- 172.21.15.152:0/3065588946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a5952370 0x5610a596e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.212+0000 7f38ba388740 1 --2- 172.21.15.152:0/3065588946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5948b40 0x5610a5962740 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.212+0000 7f38ba388740 1 --2- 172.21.15.152:0/3065588946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a59475d0 0x5610a5943e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.212+0000 7f38ba388740 1 -- 172.21.15.152:0/3065588946 >> 172.21.15.152:0/3065588946 conn(0x5610a5910310 msgr2=0x5610a5943700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.212+0000 7f38ba388740 1 -- 172.21.15.152:0/3065588946 shutdown_connections 2024-09-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.212+0000 7f38ba388740 1 -- 172.21.15.152:0/3065588946 wait complete. 2024-09-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.212+0000 7f38ba388740 1 -- start start 2024-09-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.212+0000 7f38ba388740 1 Processor -- start 2024-09-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.212+0000 7f38ba388740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a59475d0 0x5610a595a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.212+0000 7f38ba388740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a5948b40 0x5610a595adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38ba388740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5952370 0x5610a595b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38ba388740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01acbd0 con 0x5610a5952370 2024-09-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38ba388740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01ac500 con 0x5610a59475d0 2024-09-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38ba388740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f38b01a9760 con 0x5610a5948b40 2024-09-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38ba388740 10 client.0 _ll_register_callbacks cb 0x7f38a000b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38b654c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5952370 0x5610a595b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38b654c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5952370 0x5610a595b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:52518/0 (socket says 172.21.15.152:52518) 2024-09-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38b654c640 1 -- 172.21.15.152:0/3610752825 learned_addr learned my addr 172.21.15.152:0/3610752825 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38b554a640 1 --2- 172.21.15.152:0/3610752825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a5948b40 0x5610a595adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38b5d4b640 1 --2- 172.21.15.152:0/3610752825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a59475d0 0x5610a595a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38b654c640 1 -- 172.21.15.152:0/3610752825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a5948b40 msgr2=0x5610a595adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38b654c640 1 --2- 172.21.15.152:0/3610752825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a5948b40 0x5610a595adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38b654c640 1 -- 172.21.15.152:0/3610752825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a59475d0 msgr2=0x5610a595a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38b654c640 1 --2- 172.21.15.152:0/3610752825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a59475d0 0x5610a595a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38b654c640 1 -- 172.21.15.152:0/3610752825 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a59d0250 con 0x5610a5952370 2024-09-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38b654c640 1 --2- 172.21.15.152:0/3610752825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5952370 0x5610a595b300 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f38a8007f00 tx=0x7f38a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5457ebc5619811 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38997fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38997fa640 1 -- 172.21.15.152:0/3610752825 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a801ae60 con 0x5610a5952370 2024-09-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38997fa640 1 -- 172.21.15.152:0/3610752825 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8492+0+0 (secure 0 0 0) 0x7f38a80278d0 con 0x5610a5952370 2024-09-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.213+0000 7f38ba388740 1 -- 172.21.15.152:0/3610752825 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a00b0070 con 0x5610a5952370 2024-09-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.214+0000 7f38997fa640 1 -- 172.21.15.152:0/3610752825 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f38a8036050 con 0x5610a5952370 2024-09-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.214+0000 7f38ba388740 10 client.18066 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.214+0000 7f38ba388740 1 -- 172.21.15.152:0/3610752825 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5610a59554c0 con 0x5610a5952370 2024-09-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.214+0000 7f38997fa640 1 -- 172.21.15.152:0/3610752825 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38a00b0070 con 0x5610a5952370 2024-09-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.214+0000 7f38997fa640 1 -- 172.21.15.152:0/3610752825 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f38a801b3e0 con 0x5610a5952370 2024-09-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.214+0000 7f38ba388740 10 client.18066 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.214+0000 7f38ba388740 10 client.18066 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.214+0000 7f38ba388740 20 client.18066 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.214+0000 7f38ba388740 1 --2- 172.21.15.152:0/3610752825 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5610a5955c90 0x5610a59580b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.214+0000 7f38ba388740 4 client.18066 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.214+0000 7f38ba388740 1 -- 172.21.15.152:0/3610752825 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5610a58b4d70 con 0x5610a5955c90 2024-09-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.214+0000 7f38b5d4b640 1 --2- 172.21.15.152:0/3610752825 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5610a5955c90 0x5610a59580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38b5d4b640 1 --2- 172.21.15.152:0/3610752825 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5610a5955c90 0x5610a59580b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75b1d38e04aa9204 server_cookie=d922f3eba6efb570 in_seq=0 out_seq=0 2024-09-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38997fa640 10 client.18066 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38997fa640 1 -- 172.21.15.152:0/3610752825 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5610a58b4d70 con 0x5610a5955c90 2024-09-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38997fa640 10 client.18066 handle_command_reply: tid=0 2024-09-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38ba388740 1 client.18066 shutdown 2024-09-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38ba388740 1 -- 172.21.15.152:0/3610752825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5952370 msgr2=0x5610a595b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38ba388740 1 --2- 172.21.15.152:0/3610752825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5952370 0x5610a595b300 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f38a8007f00 tx=0x7f38a8004a40 comp rx=0 tx=0).stop 2024-09-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38ba388740 1 -- 172.21.15.152:0/3610752825 shutdown_connections 2024-09-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38ba388740 1 --2- 172.21.15.152:0/3610752825 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5610a5952370 0x5610a595b300 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38ba388740 1 --2- 172.21.15.152:0/3610752825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610a5948b40 0x5610a595adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38ba388740 1 --2- 172.21.15.152:0/3610752825 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5610a5955c90 0x5610a59580b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38ba388740 1 --2- 172.21.15.152:0/3610752825 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5610a59475d0 0x5610a595a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38997fa640 0 client.18066 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38ba388740 1 -- 172.21.15.152:0/3610752825 >> 172.21.15.152:0/3610752825 conn(0x5610a5910310 msgr2=0x5610a5950810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38ba388740 1 -- 172.21.15.152:0/3610752825 shutdown_connections 2024-09-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38ba388740 1 -- 172.21.15.152:0/3610752825 wait complete. 2024-09-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.215+0000 7f38ba388740 20 client.18066 trim_cache size 0 max 16384 2024-09-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.216+0000 7f38b7faf640 1 -- 172.21.15.152:0/642114027 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f389c0a2d70 msgr2=0x7f389c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.216+0000 7f38b7faf640 1 --2- 172.21.15.152:0/642114027 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f389c0a2d70 0x7f389c0a5230 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f38a800f4f0 tx=0x7f38a8004b50 comp rx=0 tx=0).stop 2024-09-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.216+0000 7f38b7faf640 1 -- 172.21.15.152:0/642114027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b0102790 msgr2=0x7f38b0198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.216+0000 7f38b7faf640 1 --2- 172.21.15.152:0/642114027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b0102790 0x7f38b0198af0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f38a000b4a0 tx=0x7f38a000b960 comp rx=0 tx=0).stop 2024-09-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.216+0000 7f38b7faf640 1 -- 172.21.15.152:0/642114027 shutdown_connections 2024-09-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.216+0000 7f38b7faf640 1 --2- 172.21.15.152:0/642114027 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f389c0a2d70 0x7f389c0a5230 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.216+0000 7f38b7faf640 1 --2- 172.21.15.152:0/642114027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38b0106f10 0x7f38b019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.216+0000 7f38b7faf640 1 --2- 172.21.15.152:0/642114027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b0102790 0x7f38b0198af0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.216+0000 7f38b7faf640 1 --2- 172.21.15.152:0/642114027 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38b0101de0 0x7f38b01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.216+0000 7f38b7faf640 1 -- 172.21.15.152:0/642114027 >> 172.21.15.152:0/642114027 conn(0x7f38b00fd8c0 msgr2=0x7f38b00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.216+0000 7f38b7faf640 1 -- 172.21.15.152:0/642114027 shutdown_connections 2024-09-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:07.216+0000 7f38b7faf640 1 -- 172.21.15.152:0/642114027 wait complete. 2024-09-11T23:02:07.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:07 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:02:08.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:07 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:02:08.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:07 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:02:08.048 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:02:08.312 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:08.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.813+0000 7f38aa491640 1 -- 172.21.15.152:0/3575520410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38a410c170 msgr2=0x7f38a410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:08.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.813+0000 7f38aa491640 1 --2- 172.21.15.152:0/3575520410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38a410c170 0x7f38a410e610 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f389c009ca0 tx=0x7f389c02a950 comp rx=0 tx=0).stop 2024-09-11T23:02:08.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.813+0000 7f38aa491640 1 -- 172.21.15.152:0/3575520410 shutdown_connections 2024-09-11T23:02:08.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.813+0000 7f38aa491640 1 --2- 172.21.15.152:0/3575520410 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38a410c170 0x7f38a410e610 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.813+0000 7f38aa491640 1 --2- 172.21.15.152:0/3575520410 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38a4103660 0x7f38a410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.813+0000 7f38aa491640 1 --2- 172.21.15.152:0/3575520410 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38a4102d40 0x7f38a4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.813+0000 7f38aa491640 1 -- 172.21.15.152:0/3575520410 >> 172.21.15.152:0/3575520410 conn(0x7f38a40fc7a0 msgr2=0x7f38a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:08.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.813+0000 7f38aa491640 1 -- 172.21.15.152:0/3575520410 shutdown_connections 2024-09-11T23:02:08.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.813+0000 7f38aa491640 1 -- 172.21.15.152:0/3575520410 wait complete. 2024-09-11T23:02:08.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.814+0000 7f38aa491640 1 Processor -- start 2024-09-11T23:02:08.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.814+0000 7f38aa491640 1 -- start start 2024-09-11T23:02:08.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.814+0000 7f38aa491640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38a4102d40 0x7f38a4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.814+0000 7f38aa491640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38a4103660 0x7f38a41988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.814+0000 7f38aa491640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38a410c170 0x7f38a419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.814+0000 7f38aa491640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f38a4111f70 con 0x7f38a4103660 2024-09-11T23:02:08.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.814+0000 7f38aa491640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f38a4111df0 con 0x7f38a4102d40 2024-09-11T23:02:08.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.814+0000 7f38aa491640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f38a41120f0 con 0x7f38a410c170 2024-09-11T23:02:08.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.814+0000 7f38a37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38a4103660 0x7f38a41988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.814+0000 7f38a37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38a4103660 0x7f38a41988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59378/0 (socket says 172.21.15.152:59378) 2024-09-11T23:02:08.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.814+0000 7f38a37fe640 1 -- 172.21.15.152:0/838691195 learned_addr learned my addr 172.21.15.152:0/838691195 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:08.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.814+0000 7f38a3fff640 1 --2- 172.21.15.152:0/838691195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38a4102d40 0x7f38a4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.814+0000 7f38a8a2e640 1 --2- 172.21.15.152:0/838691195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38a410c170 0x7f38a419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.815+0000 7f38a37fe640 1 -- 172.21.15.152:0/838691195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38a410c170 msgr2=0x7f38a419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:08.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.815+0000 7f38a37fe640 1 --2- 172.21.15.152:0/838691195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38a410c170 0x7f38a419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.815+0000 7f38a37fe640 1 -- 172.21.15.152:0/838691195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38a4102d40 msgr2=0x7f38a4198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:08.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.815+0000 7f38a37fe640 1 --2- 172.21.15.152:0/838691195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38a4102d40 0x7f38a4198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.815+0000 7f38a37fe640 1 -- 172.21.15.152:0/838691195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a419d340 con 0x7f38a4103660 2024-09-11T23:02:08.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.815+0000 7f38a3fff640 1 --2- 172.21.15.152:0/838691195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38a4102d40 0x7f38a4198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:08.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.815+0000 7f38a37fe640 1 --2- 172.21.15.152:0/838691195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38a4103660 0x7f38a41988d0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f389800c970 tx=0x7f389800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:08.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.815+0000 7f38a17fa640 1 -- 172.21.15.152:0/838691195 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898007bb0 con 0x7f38a4103660 2024-09-11T23:02:08.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.815+0000 7f38a17fa640 1 -- 172.21.15.152:0/838691195 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3898007d50 con 0x7f38a4103660 2024-09-11T23:02:08.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.815+0000 7f38aa491640 1 -- 172.21.15.152:0/838691195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a419d630 con 0x7f38a4103660 2024-09-11T23:02:08.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.815+0000 7f38a17fa640 1 -- 172.21.15.152:0/838691195 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898018820 con 0x7f38a4103660 2024-09-11T23:02:08.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.815+0000 7f38aa491640 1 -- 172.21.15.152:0/838691195 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a41a4f10 con 0x7f38a4103660 2024-09-11T23:02:08.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.817+0000 7f38a17fa640 1 -- 172.21.15.152:0/838691195 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f38980189c0 con 0x7f38a4103660 2024-09-11T23:02:08.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.817+0000 7f38ac86a740 1 Processor -- start 2024-09-11T23:02:08.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.817+0000 7f38ac86a740 1 -- start start 2024-09-11T23:02:08.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38ac86a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eaafb40 0x55ca8eaae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38ac86a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eaae5d0 0x55ca8eaaea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38ac86a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eab9370 0x55ca8eabb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38ac86a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ca8ea142f0 con 0x55ca8eab9370 2024-09-11T23:02:08.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38ac86a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ca8e9bbcf0 con 0x55ca8eaafb40 2024-09-11T23:02:08.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38ac86a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ca8ea102b0 con 0x55ca8eaae5d0 2024-09-11T23:02:08.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38a17fa640 1 --2- 172.21.15.152:0/838691195 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f388c0a2cf0 0x7f388c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38a8a2e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eaafb40 0x55ca8eaae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38a37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eab9370 0x55ca8eabb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38a3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eaae5d0 0x55ca8eaaea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38a3fff640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eaae5d0 0x55ca8eaaea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:38010/0 (socket says 172.21.15.152:38010) 2024-09-11T23:02:08.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38a8a2e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eaafb40 0x55ca8eaae090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.152:45190/0 (socket says 172.21.15.152:45190) 2024-09-11T23:02:08.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38a3fff640 1 -- 172.21.15.152:0/4012724674 learned_addr learned my addr 172.21.15.152:0/4012724674 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:08.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38a8a2e640 1 --2- 172.21.15.152:0/838691195 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f388c0a2cf0 0x7f388c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38a17fa640 1 -- 172.21.15.152:0/838691195 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f38980cf520 con 0x7f38a4103660 2024-09-11T23:02:08.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38a3fff640 1 -- 172.21.15.152:0/4012724674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eaafb40 msgr2=0x55ca8eaae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:08.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38a3fff640 1 --2- 172.21.15.152:0/4012724674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eaafb40 0x55ca8eaae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38a3fff640 1 -- 172.21.15.152:0/4012724674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eab9370 msgr2=0x55ca8eabb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:08.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38a3fff640 1 --2- 172.21.15.152:0/4012724674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eab9370 0x55ca8eabb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38a3fff640 1 -- 172.21.15.152:0/4012724674 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca8e9a03a0 con 0x55ca8eaae5d0 2024-09-11T23:02:08.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38a8a2e640 1 --2- 172.21.15.152:0/838691195 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f388c0a2cf0 0x7f388c0a51b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f389c009a60 tx=0x7f389c0095c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:08.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.818+0000 7f38a3fff640 1 --2- 172.21.15.152:0/4012724674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eaae5d0 0x55ca8eaaea10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f389400bd40 tx=0x7f389400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d8add3df95d0aa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:08.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.819+0000 7f388a7fc640 1 -- 172.21.15.152:0/4012724674 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894015070 con 0x55ca8eaae5d0 2024-09-11T23:02:08.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.819+0000 7f388a7fc640 1 -- 172.21.15.152:0/4012724674 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389400be30 con 0x55ca8eaae5d0 2024-09-11T23:02:08.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.819+0000 7f388a7fc640 1 -- 172.21.15.152:0/4012724674 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38940119a0 con 0x55ca8eaae5d0 2024-09-11T23:02:08.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.819+0000 7f38ac86a740 1 -- 172.21.15.152:0/4012724674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eaae5d0 msgr2=0x55ca8eaaea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:08.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.819+0000 7f38ac86a740 1 --2- 172.21.15.152:0/4012724674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eaae5d0 0x55ca8eaaea10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f389400bd40 tx=0x7f389400f730 comp rx=0 tx=0).stop 2024-09-11T23:02:08.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.819+0000 7f38ac86a740 1 -- 172.21.15.152:0/4012724674 shutdown_connections 2024-09-11T23:02:08.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.819+0000 7f38ac86a740 1 --2- 172.21.15.152:0/4012724674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eab9370 0x55ca8eabb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.819+0000 7f38ac86a740 1 --2- 172.21.15.152:0/4012724674 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eaae5d0 0x55ca8eaaea10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.819+0000 7f38ac86a740 1 --2- 172.21.15.152:0/4012724674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eaafb40 0x55ca8eaae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.819+0000 7f38ac86a740 1 -- 172.21.15.152:0/4012724674 >> 172.21.15.152:0/4012724674 conn(0x55ca8ea77310 msgr2=0x55ca8eaaac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:08.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.819+0000 7f38ac86a740 1 -- 172.21.15.152:0/4012724674 shutdown_connections 2024-09-11T23:02:08.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.819+0000 7f38ac86a740 1 -- 172.21.15.152:0/4012724674 wait complete. 2024-09-11T23:02:08.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.820+0000 7f38ac86a740 1 -- start start 2024-09-11T23:02:08.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.820+0000 7f38ac86a740 1 Processor -- start 2024-09-11T23:02:08.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.820+0000 7f38ac86a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaae5d0 0x55ca8eab8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.820+0000 7f38ac86a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eaafb40 0x55ca8eab6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.820+0000 7f38ac86a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eab9370 0x55ca8eab6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.820+0000 7f38ac86a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55ca8eabc070 con 0x55ca8eaae5d0 2024-09-11T23:02:08.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.820+0000 7f38ac86a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55ca8e9cf060 con 0x55ca8eaafb40 2024-09-11T23:02:08.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.820+0000 7f38ac86a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55ca8eabc1f0 con 0x55ca8eab9370 2024-09-11T23:02:08.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.820+0000 7f38ac86a740 10 client.0 _ll_register_callbacks cb 0x55ca8e7228c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:08.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.820+0000 7f38a3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaae5d0 0x55ca8eab8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.820+0000 7f38a3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaae5d0 0x55ca8eab8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59398/0 (socket says 172.21.15.152:59398) 2024-09-11T23:02:08.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.820+0000 7f38a3fff640 1 -- 172.21.15.152:0/1719037992 learned_addr learned my addr 172.21.15.152:0/1719037992 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:08.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38a37fe640 1 --2- 172.21.15.152:0/1719037992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eaafb40 0x55ca8eab6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38a8a2e640 1 --2- 172.21.15.152:0/1719037992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eab9370 0x55ca8eab6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38a3fff640 1 -- 172.21.15.152:0/1719037992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eab9370 msgr2=0x55ca8eab6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:08.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38a3fff640 1 --2- 172.21.15.152:0/1719037992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eab9370 0x55ca8eab6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38a3fff640 1 -- 172.21.15.152:0/1719037992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eaafb40 msgr2=0x55ca8eab6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:08.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38a3fff640 1 --2- 172.21.15.152:0/1719037992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eaafb40 0x55ca8eab6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38a3fff640 1 -- 172.21.15.152:0/1719037992 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca8e7853c0 con 0x55ca8eaae5d0 2024-09-11T23:02:08.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38a3fff640 1 --2- 172.21.15.152:0/1719037992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaae5d0 0x55ca8eab8d60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f38940116e0 tx=0x7f389400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6d96fbf5f3fc0d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:08.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38837fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:08.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38837fe640 1 -- 172.21.15.152:0/1719037992 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894028060 con 0x55ca8eaae5d0 2024-09-11T23:02:08.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38837fe640 1 -- 172.21.15.152:0/1719037992 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f3894005310 con 0x55ca8eaae5d0 2024-09-11T23:02:08.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38ac86a740 1 -- 172.21.15.152:0/1719037992 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f389400be30 con 0x55ca8eaae5d0 2024-09-11T23:02:08.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38837fe640 1 -- 172.21.15.152:0/1719037992 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3894015070 con 0x55ca8eaae5d0 2024-09-11T23:02:08.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38ac86a740 10 client.18078 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:08.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38ac86a740 1 -- 172.21.15.152:0/1719037992 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca8eac7b10 con 0x55ca8eaae5d0 2024-09-11T23:02:08.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38837fe640 1 -- 172.21.15.152:0/1719037992 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389400be30 con 0x55ca8eaae5d0 2024-09-11T23:02:08.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.821+0000 7f38837fe640 1 -- 172.21.15.152:0/1719037992 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f38940049f0 con 0x55ca8eaae5d0 2024-09-11T23:02:08.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.822+0000 7f38ac86a740 10 client.18078 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:08.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.822+0000 7f38ac86a740 10 client.18078 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:02:08.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.822+0000 7f38ac86a740 20 client.18078 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:08.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.822+0000 7f38ac86a740 1 --2- 172.21.15.152:0/1719037992 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ca8eac8f10 0x55ca8eacb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.822+0000 7f38ac86a740 4 client.18078 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:02:08.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.822+0000 7f38ac86a740 1 -- 172.21.15.152:0/1719037992 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca8ea21860 con 0x55ca8eac8f10 2024-09-11T23:02:08.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.822+0000 7f38a37fe640 1 --2- 172.21.15.152:0/1719037992 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ca8eac8f10 0x55ca8eacb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.822+0000 7f38837fe640 10 client.18078 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:08.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.822+0000 7f38a37fe640 1 --2- 172.21.15.152:0/1719037992 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ca8eac8f10 0x55ca8eacb300 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642d301514ac5412 server_cookie=c4b41f628823d43b in_seq=0 out_seq=0 2024-09-11T23:02:08.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.823+0000 7f38837fe640 1 -- 172.21.15.152:0/1719037992 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca8ea21860 con 0x55ca8eac8f10 2024-09-11T23:02:08.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.823+0000 7f38837fe640 10 client.18078 handle_command_reply: tid=0 2024-09-11T23:02:08.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.823+0000 7f38ac86a740 1 client.18078 shutdown 2024-09-11T23:02:08.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.823+0000 7f38ac86a740 1 -- 172.21.15.152:0/1719037992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaae5d0 msgr2=0x55ca8eab8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:08.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.823+0000 7f38ac86a740 1 --2- 172.21.15.152:0/1719037992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaae5d0 0x55ca8eab8d60 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f38940116e0 tx=0x7f389400bab0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.823+0000 7f38ac86a740 1 -- 172.21.15.152:0/1719037992 shutdown_connections 2024-09-11T23:02:08.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.823+0000 7f38ac86a740 1 --2- 172.21.15.152:0/1719037992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eab9370 0x55ca8eab6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.823+0000 7f38ac86a740 1 --2- 172.21.15.152:0/1719037992 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ca8eac8f10 0x55ca8eacb300 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.823+0000 7f38ac86a740 1 --2- 172.21.15.152:0/1719037992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eaafb40 0x55ca8eab6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.823+0000 7f38837fe640 0 client.18078 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:08.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.823+0000 7f38ac86a740 1 --2- 172.21.15.152:0/1719037992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaae5d0 0x55ca8eab8d60 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.823+0000 7f38ac86a740 1 -- 172.21.15.152:0/1719037992 >> 172.21.15.152:0/1719037992 conn(0x55ca8ea77310 msgr2=0x55ca8eab0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:08.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.823+0000 7f38ac86a740 1 -- 172.21.15.152:0/1719037992 shutdown_connections 2024-09-11T23:02:08.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.823+0000 7f38ac86a740 1 -- 172.21.15.152:0/1719037992 wait complete. 2024-09-11T23:02:08.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.823+0000 7f38ac86a740 20 client.18078 trim_cache size 0 max 16384 2024-09-11T23:02:08.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.836+0000 7f38ac86a740 1 Processor -- start 2024-09-11T23:02:08.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.836+0000 7f38ac86a740 1 -- start start 2024-09-11T23:02:08.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.836+0000 7f38ac86a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaae5d0 0x55ca8eab58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.836+0000 7f38ac86a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eaafb40 0x55ca8eab1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.836+0000 7f38ac86a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eab9370 0x55ca8eab1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.836+0000 7f38ac86a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f38a41ab660 con 0x55ca8eaae5d0 2024-09-11T23:02:08.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.836+0000 7f38ac86a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f38a41ad1a0 con 0x55ca8eaafb40 2024-09-11T23:02:08.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.836+0000 7f38ac86a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f38a41abd30 con 0x55ca8eab9370 2024-09-11T23:02:08.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.836+0000 7f38a3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaae5d0 0x55ca8eab58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.836+0000 7f38a3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaae5d0 0x55ca8eab58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59404/0 (socket says 172.21.15.152:59404) 2024-09-11T23:02:08.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.836+0000 7f38a3fff640 1 -- 172.21.15.152:0/546770236 learned_addr learned my addr 172.21.15.152:0/546770236 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:08.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.836+0000 7f38a37fe640 1 --2- 172.21.15.152:0/546770236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eaafb40 0x55ca8eab1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.837+0000 7f38a8a2e640 1 --2- 172.21.15.152:0/546770236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eab9370 0x55ca8eab1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.837+0000 7f38a3fff640 1 -- 172.21.15.152:0/546770236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eab9370 msgr2=0x55ca8eab1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:08.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.837+0000 7f38a3fff640 1 --2- 172.21.15.152:0/546770236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eab9370 0x55ca8eab1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.837+0000 7f38a3fff640 1 -- 172.21.15.152:0/546770236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eaafb40 msgr2=0x55ca8eab1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:08.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.837+0000 7f38a3fff640 1 --2- 172.21.15.152:0/546770236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eaafb40 0x55ca8eab1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.837+0000 7f38a3fff640 1 -- 172.21.15.152:0/546770236 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca8eab7100 con 0x55ca8eaae5d0 2024-09-11T23:02:08.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.837+0000 7f38a8a2e640 1 --2- 172.21.15.152:0/546770236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eab9370 0x55ca8eab1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:08.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.837+0000 7f38a3fff640 1 --2- 172.21.15.152:0/546770236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaae5d0 0x55ca8eab58d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f389400fe10 tx=0x7f389400b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2335cc1234fada9e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:08.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.837+0000 7f3889ffb640 1 -- 172.21.15.152:0/546770236 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894028040 con 0x55ca8eaae5d0 2024-09-11T23:02:08.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.837+0000 7f3889ffb640 1 -- 172.21.15.152:0/546770236 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3894011440 con 0x55ca8eaae5d0 2024-09-11T23:02:08.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.837+0000 7f3889ffb640 1 -- 172.21.15.152:0/546770236 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3894011910 con 0x55ca8eaae5d0 2024-09-11T23:02:08.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.837+0000 7f38ac86a740 1 -- 172.21.15.152:0/546770236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaae5d0 msgr2=0x55ca8eab58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:08.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.837+0000 7f38ac86a740 1 --2- 172.21.15.152:0/546770236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaae5d0 0x55ca8eab58d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f389400fe10 tx=0x7f389400b660 comp rx=0 tx=0).stop 2024-09-11T23:02:08.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.838+0000 7f38ac86a740 1 -- 172.21.15.152:0/546770236 shutdown_connections 2024-09-11T23:02:08.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.838+0000 7f38ac86a740 1 --2- 172.21.15.152:0/546770236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eab9370 0x55ca8eab1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.838+0000 7f38ac86a740 1 --2- 172.21.15.152:0/546770236 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eaafb40 0x55ca8eab1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.838+0000 7f38ac86a740 1 --2- 172.21.15.152:0/546770236 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaae5d0 0x55ca8eab58d0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.838+0000 7f38ac86a740 1 -- 172.21.15.152:0/546770236 >> 172.21.15.152:0/546770236 conn(0x55ca8ea77310 msgr2=0x55ca8eab5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:08.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.838+0000 7f38ac86a740 1 -- 172.21.15.152:0/546770236 shutdown_connections 2024-09-11T23:02:08.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.838+0000 7f38ac86a740 1 -- 172.21.15.152:0/546770236 wait complete. 2024-09-11T23:02:08.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.838+0000 7f38ac86a740 1 -- start start 2024-09-11T23:02:08.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.838+0000 7f38ac86a740 1 Processor -- start 2024-09-11T23:02:08.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.838+0000 7f38ac86a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eaae5d0 0x55ca8eacb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.838+0000 7f38ac86a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaafb40 0x55ca8eacbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.838+0000 7f38ac86a740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eab9370 0x55ca8eb31840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.838+0000 7f38ac86a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f38a41acad0 con 0x55ca8eaafb40 2024-09-11T23:02:08.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.838+0000 7f38ac86a740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f38a41ac400 con 0x55ca8eab9370 2024-09-11T23:02:08.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.838+0000 7f38ac86a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f38a41a9660 con 0x55ca8eaae5d0 2024-09-11T23:02:08.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.838+0000 7f38ac86a740 10 client.0 _ll_register_callbacks cb 0x7f389800fdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:08.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38a37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaafb40 0x55ca8eacbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38a37fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaafb40 0x55ca8eacbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59418/0 (socket says 172.21.15.152:59418) 2024-09-11T23:02:08.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38a37fe640 1 -- 172.21.15.152:0/3877931179 learned_addr learned my addr 172.21.15.152:0/3877931179 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:08.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38a8a2e640 1 --2- 172.21.15.152:0/3877931179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eab9370 0x55ca8eb31840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38a3fff640 1 --2- 172.21.15.152:0/3877931179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eaae5d0 0x55ca8eacb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38a37fe640 1 -- 172.21.15.152:0/3877931179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eaae5d0 msgr2=0x55ca8eacb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:08.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38a37fe640 1 --2- 172.21.15.152:0/3877931179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eaae5d0 0x55ca8eacb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38a37fe640 1 -- 172.21.15.152:0/3877931179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eab9370 msgr2=0x55ca8eb31840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38a37fe640 1 --2- 172.21.15.152:0/3877931179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eab9370 0x55ca8eb31840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38a37fe640 1 -- 172.21.15.152:0/3877931179 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca8eb31d80 con 0x55ca8eaafb40 2024-09-11T23:02:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38a37fe640 1 --2- 172.21.15.152:0/3877931179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaafb40 0x55ca8eacbde0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f38980ccd10 tx=0x7f3898097d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46bdd53696fbd46a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38837fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38837fe640 1 -- 172.21.15.152:0/3877931179 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389809ba30 con 0x55ca8eaafb40 2024-09-11T23:02:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38837fe640 1 -- 172.21.15.152:0/3877931179 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f3898096300 con 0x55ca8eaafb40 2024-09-11T23:02:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38ac86a740 1 -- 172.21.15.152:0/3877931179 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3894011440 con 0x55ca8eaafb40 2024-09-11T23:02:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38837fe640 1 -- 172.21.15.152:0/3877931179 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f389809fde0 con 0x55ca8eaafb40 2024-09-11T23:02:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38ac86a740 10 client.18084 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38ac86a740 1 -- 172.21.15.152:0/3877931179 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ca8eb32fb0 con 0x55ca8eaafb40 2024-09-11T23:02:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.839+0000 7f38837fe640 1 -- 172.21.15.152:0/3877931179 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3894011440 con 0x55ca8eaafb40 2024-09-11T23:02:08.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.840+0000 7f38837fe640 1 -- 172.21.15.152:0/3877931179 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f38980ccee0 con 0x55ca8eaafb40 2024-09-11T23:02:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.840+0000 7f38ac86a740 10 client.18084 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.840+0000 7f38ac86a740 10 client.18084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:02:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.840+0000 7f38ac86a740 20 client.18084 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.840+0000 7f38ac86a740 1 --2- 172.21.15.152:0/3877931179 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ca8eac8f10 0x55ca8eb356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.840+0000 7f38ac86a740 4 client.18084 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:02:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.840+0000 7f38ac86a740 1 -- 172.21.15.152:0/3877931179 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca8ea1bd70 con 0x55ca8eac8f10 2024-09-11T23:02:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.840+0000 7f38a3fff640 1 --2- 172.21.15.152:0/3877931179 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ca8eac8f10 0x55ca8eb356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.840+0000 7f38a3fff640 1 --2- 172.21.15.152:0/3877931179 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ca8eac8f10 0x55ca8eb356e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99e69f9a80589b91 server_cookie=5966036c0b3ea384 in_seq=0 out_seq=0 2024-09-11T23:02:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.840+0000 7f38837fe640 10 client.18084 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.840+0000 7f38837fe640 1 -- 172.21.15.152:0/3877931179 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca8ea1bd70 con 0x55ca8eac8f10 2024-09-11T23:02:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.840+0000 7f38837fe640 10 client.18084 handle_command_reply: tid=0 2024-09-11T23:02:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.840+0000 7f38ac86a740 1 client.18084 shutdown 2024-09-11T23:02:08.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38ac86a740 1 -- 172.21.15.152:0/3877931179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaafb40 msgr2=0x55ca8eacbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:08.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38ac86a740 1 --2- 172.21.15.152:0/3877931179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaafb40 0x55ca8eacbde0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f38980ccd10 tx=0x7f3898097d10 comp rx=0 tx=0).stop 2024-09-11T23:02:08.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38ac86a740 1 -- 172.21.15.152:0/3877931179 shutdown_connections 2024-09-11T23:02:08.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38ac86a740 1 --2- 172.21.15.152:0/3877931179 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55ca8eac8f10 0x55ca8eb356e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38ac86a740 1 --2- 172.21.15.152:0/3877931179 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ca8eab9370 0x55ca8eb31840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38ac86a740 1 --2- 172.21.15.152:0/3877931179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ca8eaafb40 0x55ca8eacbde0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38ac86a740 1 --2- 172.21.15.152:0/3877931179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca8eaae5d0 0x55ca8eacb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38837fe640 0 client.18084 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:08.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38ac86a740 1 -- 172.21.15.152:0/3877931179 >> 172.21.15.152:0/3877931179 conn(0x55ca8ea77310 msgr2=0x55ca8eab7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:08.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38ac86a740 1 -- 172.21.15.152:0/3877931179 shutdown_connections 2024-09-11T23:02:08.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38ac86a740 1 -- 172.21.15.152:0/3877931179 wait complete. 2024-09-11T23:02:08.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38ac86a740 20 client.18084 trim_cache size 0 max 16384 2024-09-11T23:02:08.858 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:02:08.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38aa491640 1 -- 172.21.15.152:0/838691195 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f388c0a2cf0 msgr2=0x7f388c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:08.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38aa491640 1 --2- 172.21.15.152:0/838691195 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f388c0a2cf0 0x7f388c0a51b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f389c009a60 tx=0x7f389c0095c0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38aa491640 1 -- 172.21.15.152:0/838691195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38a4103660 msgr2=0x7f38a41988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:08.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38aa491640 1 --2- 172.21.15.152:0/838691195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38a4103660 0x7f38a41988d0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f389800c970 tx=0x7f389800ce30 comp rx=0 tx=0).stop 2024-09-11T23:02:08.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38aa491640 1 -- 172.21.15.152:0/838691195 shutdown_connections 2024-09-11T23:02:08.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38aa491640 1 --2- 172.21.15.152:0/838691195 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f388c0a2cf0 0x7f388c0a51b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38aa491640 1 --2- 172.21.15.152:0/838691195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38a410c170 0x7f38a419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38aa491640 1 --2- 172.21.15.152:0/838691195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38a4103660 0x7f38a41988d0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38aa491640 1 --2- 172.21.15.152:0/838691195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f38a4102d40 0x7f38a4198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:08.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.841+0000 7f38aa491640 1 -- 172.21.15.152:0/838691195 >> 172.21.15.152:0/838691195 conn(0x7f38a40fc7a0 msgr2=0x7f38a40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:08.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.842+0000 7f38aa491640 1 -- 172.21.15.152:0/838691195 shutdown_connections 2024-09-11T23:02:08.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:08.842+0000 7f38aa491640 1 -- 172.21.15.152:0/838691195 wait complete. 2024-09-11T23:02:09.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:08 smithi191 ceph-mon[35211]: pgmap v284: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-11T23:02:09.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:08 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:02:09.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:08 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:02:09.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:08 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:02:09.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:08 smithi204 ceph-mon[35454]: pgmap v284: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-11T23:02:09.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:08 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:02:09.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:08 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:02:09.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:08 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:02:09.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:08 smithi152 ceph-mon[31181]: pgmap v284: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-11T23:02:09.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:08 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:02:09.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:08 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:02:09.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:08 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:02:10.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:09 smithi191 ceph-mon[35211]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T23:02:10.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:09 smithi204 ceph-mon[35454]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T23:02:10.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:09 smithi152 ceph-mon[31181]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T23:02:10.753 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:02:10.754 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:02:11.014 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:11.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:10 smithi191 ceph-mon[35211]: pgmap v286: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:02:11.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:10 smithi204 ceph-mon[35454]: pgmap v286: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:02:11.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:10 smithi152 ceph-mon[31181]: pgmap v286: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:02:11.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.551+0000 7f0c2ea0f640 1 -- 172.21.15.152:0/3533443960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c28103660 msgr2=0x7f0c2810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:11.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.551+0000 7f0c2ea0f640 1 --2- 172.21.15.152:0/3533443960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c28103660 0x7f0c2810bb50 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009900 tx=0x7f0c1c02a480 comp rx=0 tx=0).stop 2024-09-11T23:02:11.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.551+0000 7f0c2ea0f640 1 -- 172.21.15.152:0/3533443960 shutdown_connections 2024-09-11T23:02:11.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.551+0000 7f0c2ea0f640 1 --2- 172.21.15.152:0/3533443960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c2810c170 0x7f0c2810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.551+0000 7f0c2ea0f640 1 --2- 172.21.15.152:0/3533443960 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c28103660 0x7f0c2810bb50 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.551+0000 7f0c2ea0f640 1 --2- 172.21.15.152:0/3533443960 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c28102d40 0x7f0c28103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.551+0000 7f0c2ea0f640 1 -- 172.21.15.152:0/3533443960 >> 172.21.15.152:0/3533443960 conn(0x7f0c280fc7a0 msgr2=0x7f0c280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:11.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.551+0000 7f0c2ea0f640 1 -- 172.21.15.152:0/3533443960 shutdown_connections 2024-09-11T23:02:11.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.551+0000 7f0c2ea0f640 1 -- 172.21.15.152:0/3533443960 wait complete. 2024-09-11T23:02:11.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.552+0000 7f0c2ea0f640 1 Processor -- start 2024-09-11T23:02:11.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.552+0000 7f0c2ea0f640 1 -- start start 2024-09-11T23:02:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.552+0000 7f0c2ea0f640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c28102d40 0x7f0c28198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.552+0000 7f0c2ea0f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c28103660 0x7f0c28198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.552+0000 7f0c2ea0f640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c2810c170 0x7f0c2819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.552+0000 7f0c2ea0f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0c28111f70 con 0x7f0c28103660 2024-09-11T23:02:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.552+0000 7f0c2ea0f640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0c28111df0 con 0x7f0c28102d40 2024-09-11T23:02:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.552+0000 7f0c2ea0f640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281120f0 con 0x7f0c2810c170 2024-09-11T23:02:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.552+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c28103660 0x7f0c28198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.552+0000 7f0c277fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c28103660 0x7f0c28198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59438/0 (socket says 172.21.15.152:59438) 2024-09-11T23:02:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.552+0000 7f0c277fe640 1 -- 172.21.15.152:0/3311925637 learned_addr learned my addr 172.21.15.152:0/3311925637 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.552+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/3311925637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c2810c170 0x7f0c2819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.571 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.552+0000 7f0c27fff640 1 --2- 172.21.15.152:0/3311925637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c28102d40 0x7f0c28198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.553+0000 7f0c277fe640 1 -- 172.21.15.152:0/3311925637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c2810c170 msgr2=0x7f0c2819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.553+0000 7f0c277fe640 1 --2- 172.21.15.152:0/3311925637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c2810c170 0x7f0c2819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.553+0000 7f0c277fe640 1 -- 172.21.15.152:0/3311925637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c28102d40 msgr2=0x7f0c28198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.553+0000 7f0c277fe640 1 --2- 172.21.15.152:0/3311925637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c28102d40 0x7f0c28198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.553+0000 7f0c277fe640 1 -- 172.21.15.152:0/3311925637 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2819d3e0 con 0x7f0c28103660 2024-09-11T23:02:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.553+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/3311925637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c2810c170 0x7f0c2819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.553+0000 7f0c27fff640 1 --2- 172.21.15.152:0/3311925637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c28102d40 0x7f0c28198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.553+0000 7f0c277fe640 1 --2- 172.21.15.152:0/3311925637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c28103660 0x7f0c28198970 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c006010 tx=0x7f0c1c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.553+0000 7f0c257fa640 1 -- 172.21.15.152:0/3311925637 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c03f030 con 0x7f0c28103660 2024-09-11T23:02:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.553+0000 7f0c2ea0f640 1 -- 172.21.15.152:0/3311925637 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2819d670 con 0x7f0c28103660 2024-09-11T23:02:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.553+0000 7f0c257fa640 1 -- 172.21.15.152:0/3311925637 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c1c003ce0 con 0x7f0c28103660 2024-09-11T23:02:11.572 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.553+0000 7f0c257fa640 1 -- 172.21.15.152:0/3311925637 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c035070 con 0x7f0c28103660 2024-09-11T23:02:11.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.553+0000 7f0c2ea0f640 1 -- 172.21.15.152:0/3311925637 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c281a4fb0 con 0x7f0c28103660 2024-09-11T23:02:11.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.555+0000 7f0c257fa640 1 -- 172.21.15.152:0/3311925637 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0c1c042030 con 0x7f0c28103660 2024-09-11T23:02:11.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c30de8740 1 Processor -- start 2024-09-11T23:02:11.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c30de8740 1 -- start start 2024-09-11T23:02:11.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c30de8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fcead9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c30de8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceada480 0x555fceae4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c257fa640 1 --2- 172.21.15.152:0/3311925637 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c100a2cf0 0x7f0c100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c30de8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fceae4d40 0x555fceae7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.573 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c30de8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x555fcea3f280 con 0x555fcead9b60 2024-09-11T23:02:11.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c30de8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555fce9e6b10 con 0x555fceada480 2024-09-11T23:02:11.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c30de8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555fcea3b240 con 0x555fceae4d40 2024-09-11T23:02:11.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c2cfac640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fcead9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c277fe640 1 --2- 172.21.15.152:0/3311925637 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c100a2cf0 0x7f0c100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c2cfac640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fcead9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59442/0 (socket says 172.21.15.152:59442) 2024-09-11T23:02:11.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c2cfac640 1 -- 172.21.15.152:0/3990395880 learned_addr learned my addr 172.21.15.152:0/3990395880 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:11.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c257fa640 1 -- 172.21.15.152:0/3311925637 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f0c1c0efb50 con 0x7f0c28103660 2024-09-11T23:02:11.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c27fff640 1 --2- 172.21.15.152:0/3990395880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceada480 0x555fceae4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/3990395880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fceae4d40 0x555fceae7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c2cfac640 1 -- 172.21.15.152:0/3990395880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fceae4d40 msgr2=0x555fceae7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:11.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/3990395880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fceae4d40 0x555fceae7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c2cfac640 1 -- 172.21.15.152:0/3990395880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceada480 msgr2=0x555fceae4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:11.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.556+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/3990395880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceada480 0x555fceae4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.557+0000 7f0c2cfac640 1 -- 172.21.15.152:0/3990395880 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fce9f31f0 con 0x555fcead9b60 2024-09-11T23:02:11.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.557+0000 7f0c277fe640 1 --2- 172.21.15.152:0/3311925637 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c100a2cf0 0x7f0c100a51b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c000c00 tx=0x7f0c1c08bfe0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:11.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.557+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/3990395880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fcead9f40 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x555fcead7470 tx=0x7f0c200099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3488d629b5261f54 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:11.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.557+0000 7f0c0e7fc640 1 -- 172.21.15.152:0/3990395880 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20019030 con 0x555fcead9b60 2024-09-11T23:02:11.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.557+0000 7f0c0e7fc640 1 -- 172.21.15.152:0/3990395880 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c20013e50 con 0x555fcead9b60 2024-09-11T23:02:11.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.557+0000 7f0c0e7fc640 1 -- 172.21.15.152:0/3990395880 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20011bd0 con 0x555fcead9b60 2024-09-11T23:02:11.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 -- 172.21.15.152:0/3990395880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 msgr2=0x555fcead9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:11.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 --2- 172.21.15.152:0/3990395880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fcead9f40 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x555fcead7470 tx=0x7f0c200099c0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 -- 172.21.15.152:0/3990395880 shutdown_connections 2024-09-11T23:02:11.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 --2- 172.21.15.152:0/3990395880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fceae4d40 0x555fceae7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 --2- 172.21.15.152:0/3990395880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceada480 0x555fceae4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 --2- 172.21.15.152:0/3990395880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fcead9f40 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 -- 172.21.15.152:0/3990395880 >> 172.21.15.152:0/3990395880 conn(0x555fcead60f0 msgr2=0x555fcead64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:11.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 -- 172.21.15.152:0/3990395880 shutdown_connections 2024-09-11T23:02:11.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 -- 172.21.15.152:0/3990395880 wait complete. 2024-09-11T23:02:11.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 -- start start 2024-09-11T23:02:11.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 Processor -- start 2024-09-11T23:02:11.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fcead9b60 0x555fceae1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fceada480 0x555fceae2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceae4d40 0x555fceae2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x555fceae7910 con 0x555fceada480 2024-09-11T23:02:11.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x555fce9f9f10 con 0x555fceae4d40 2024-09-11T23:02:11.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.558+0000 7f0c30de8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x555fceae7a90 con 0x555fcead9b60 2024-09-11T23:02:11.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c30de8740 10 client.0 _ll_register_callbacks cb 0x555fce9c2a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:11.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fceada480 0x555fceae2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c27fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fceada480 0x555fceae2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59446/0 (socket says 172.21.15.152:59446) 2024-09-11T23:02:11.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c27fff640 1 -- 172.21.15.152:0/2751945755 learned_addr learned my addr 172.21.15.152:0/2751945755 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:11.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/2751945755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fcead9b60 0x555fceae1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/2751945755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceae4d40 0x555fceae2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c27fff640 1 -- 172.21.15.152:0/2751945755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fcead9b60 msgr2=0x555fceae1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:11.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c27fff640 1 --2- 172.21.15.152:0/2751945755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fcead9b60 0x555fceae1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c27fff640 1 -- 172.21.15.152:0/2751945755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceae4d40 msgr2=0x555fceae2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:11.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c27fff640 1 --2- 172.21.15.152:0/2751945755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceae4d40 0x555fceae2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c27fff640 1 -- 172.21.15.152:0/2751945755 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fce7b06a0 con 0x555fceada480 2024-09-11T23:02:11.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/2751945755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fcead9b60 0x555fceae1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:11.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c27fff640 1 --2- 172.21.15.152:0/2751945755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fceada480 0x555fceae2140 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800bf40 tx=0x7f0c180128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a05a93dcb3eb6cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:11.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c077fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:11.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c077fe640 1 -- 172.21.15.152:0/2751945755 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1801f060 con 0x555fceada480 2024-09-11T23:02:11.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c077fe640 1 -- 172.21.15.152:0/2751945755 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f0c18010960 con 0x555fceada480 2024-09-11T23:02:11.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c30de8740 1 -- 172.21.15.152:0/2751945755 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c20013e50 con 0x555fceada480 2024-09-11T23:02:11.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c077fe640 1 -- 172.21.15.152:0/2751945755 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c18010b60 con 0x555fceada480 2024-09-11T23:02:11.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c30de8740 10 client.18096 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:11.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.559+0000 7f0c30de8740 1 -- 172.21.15.152:0/2751945755 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555fceaf3550 con 0x555fceada480 2024-09-11T23:02:11.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.560+0000 7f0c077fe640 1 -- 172.21.15.152:0/2751945755 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c20013e50 con 0x555fceada480 2024-09-11T23:02:11.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.560+0000 7f0c077fe640 1 -- 172.21.15.152:0/2751945755 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0c18019030 con 0x555fceada480 2024-09-11T23:02:11.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.560+0000 7f0c30de8740 10 client.18096 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:11.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.560+0000 7f0c30de8740 10 client.18096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:02:11.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.560+0000 7f0c30de8740 20 client.18096 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:11.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.560+0000 7f0c30de8740 1 --2- 172.21.15.152:0/2751945755 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555fceaf4a60 0x555fceaf6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.560+0000 7f0c30de8740 4 client.18096 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:02:11.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.560+0000 7f0c30de8740 1 -- 172.21.15.152:0/2751945755 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fcea4c7f0 con 0x555fceaf4a60 2024-09-11T23:02:11.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.560+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/2751945755 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555fceaf4a60 0x555fceaf6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.561+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/2751945755 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555fceaf4a60 0x555fceaf6e80 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cd984f985efc0f server_cookie=48f59b6045825b2f in_seq=0 out_seq=0 2024-09-11T23:02:11.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.561+0000 7f0c077fe640 10 client.18096 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:11.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.561+0000 7f0c077fe640 1 -- 172.21.15.152:0/2751945755 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555fcea4c7f0 con 0x555fceaf4a60 2024-09-11T23:02:11.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.561+0000 7f0c077fe640 10 client.18096 handle_command_reply: tid=0 2024-09-11T23:02:11.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.561+0000 7f0c30de8740 1 client.18096 shutdown 2024-09-11T23:02:11.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.561+0000 7f0c30de8740 1 -- 172.21.15.152:0/2751945755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fceada480 msgr2=0x555fceae2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:11.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.561+0000 7f0c30de8740 1 --2- 172.21.15.152:0/2751945755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fceada480 0x555fceae2140 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800bf40 tx=0x7f0c180128e0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.562+0000 7f0c30de8740 1 -- 172.21.15.152:0/2751945755 shutdown_connections 2024-09-11T23:02:11.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.562+0000 7f0c30de8740 1 --2- 172.21.15.152:0/2751945755 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555fceaf4a60 0x555fceaf6e80 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.562+0000 7f0c30de8740 1 --2- 172.21.15.152:0/2751945755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceae4d40 0x555fceae2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.562+0000 7f0c30de8740 1 --2- 172.21.15.152:0/2751945755 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fceada480 0x555fceae2140 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.562+0000 7f0c30de8740 1 --2- 172.21.15.152:0/2751945755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fcead9b60 0x555fceae1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.562+0000 7f0c077fe640 0 client.18096 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:11.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.562+0000 7f0c30de8740 1 -- 172.21.15.152:0/2751945755 >> 172.21.15.152:0/2751945755 conn(0x555fcead60f0 msgr2=0x555fceadbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:11.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.562+0000 7f0c30de8740 1 -- 172.21.15.152:0/2751945755 shutdown_connections 2024-09-11T23:02:11.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.562+0000 7f0c30de8740 1 -- 172.21.15.152:0/2751945755 wait complete. 2024-09-11T23:02:11.582 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.562+0000 7f0c30de8740 20 client.18096 trim_cache size 0 max 16384 2024-09-11T23:02:11.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.579+0000 7f0c30de8740 1 Processor -- start 2024-09-11T23:02:11.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.579+0000 7f0c30de8740 1 -- start start 2024-09-11T23:02:11.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.579+0000 7f0c30de8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fceae0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.579+0000 7f0c30de8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceada480 0x555fceae24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.579+0000 7f0c30de8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fceae4d40 0x555fceaed0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.587 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.579+0000 7f0c30de8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281ad840 con 0x555fcead9b60 2024-09-11T23:02:11.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.579+0000 7f0c30de8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281af380 con 0x555fceada480 2024-09-11T23:02:11.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.579+0000 7f0c30de8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281adf10 con 0x555fceae4d40 2024-09-11T23:02:11.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.579+0000 7f0c2cfac640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fceae0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.579+0000 7f0c2cfac640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fceae0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59462/0 (socket says 172.21.15.152:59462) 2024-09-11T23:02:11.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.579+0000 7f0c2cfac640 1 -- 172.21.15.152:0/1295390074 learned_addr learned my addr 172.21.15.152:0/1295390074 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:11.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.580+0000 7f0c27fff640 1 --2- 172.21.15.152:0/1295390074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceada480 0x555fceae24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.580+0000 7f0c2cfac640 1 -- 172.21.15.152:0/1295390074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fceae4d40 msgr2=0x555fceaed0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:11.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.580+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/1295390074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fceae4d40 0x555fceaed0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.580+0000 7f0c2cfac640 1 -- 172.21.15.152:0/1295390074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceada480 msgr2=0x555fceae24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:11.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.580+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/1295390074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceada480 0x555fceae24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.580+0000 7f0c2cfac640 1 -- 172.21.15.152:0/1295390074 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fceae44d0 con 0x555fcead9b60 2024-09-11T23:02:11.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.580+0000 7f0c27fff640 1 --2- 172.21.15.152:0/1295390074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceada480 0x555fceae24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:11.588 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.580+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/1295390074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fceae0ab0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0c200133e0 tx=0x7f0c20009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674059a7467c116d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:11.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.580+0000 7f0c0dffb640 1 -- 172.21.15.152:0/1295390074 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20011cf0 con 0x555fcead9b60 2024-09-11T23:02:11.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.580+0000 7f0c0dffb640 1 -- 172.21.15.152:0/1295390074 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c20013e50 con 0x555fcead9b60 2024-09-11T23:02:11.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.580+0000 7f0c0dffb640 1 -- 172.21.15.152:0/1295390074 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20003e30 con 0x555fcead9b60 2024-09-11T23:02:11.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.581+0000 7f0c30de8740 1 -- 172.21.15.152:0/1295390074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 msgr2=0x555fceae0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:11.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.581+0000 7f0c30de8740 1 --2- 172.21.15.152:0/1295390074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fceae0ab0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0c200133e0 tx=0x7f0c20009f90 comp rx=0 tx=0).stop 2024-09-11T23:02:11.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.581+0000 7f0c30de8740 1 -- 172.21.15.152:0/1295390074 shutdown_connections 2024-09-11T23:02:11.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.581+0000 7f0c30de8740 1 --2- 172.21.15.152:0/1295390074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fceae4d40 0x555fceaed0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.581+0000 7f0c30de8740 1 --2- 172.21.15.152:0/1295390074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceada480 0x555fceae24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.581+0000 7f0c30de8740 1 --2- 172.21.15.152:0/1295390074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fceae0ab0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.581+0000 7f0c30de8740 1 -- 172.21.15.152:0/1295390074 >> 172.21.15.152:0/1295390074 conn(0x555fcead60f0 msgr2=0x555fceae03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:11.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.581+0000 7f0c30de8740 1 -- 172.21.15.152:0/1295390074 shutdown_connections 2024-09-11T23:02:11.589 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.581+0000 7f0c30de8740 1 -- 172.21.15.152:0/1295390074 wait complete. 2024-09-11T23:02:11.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.581+0000 7f0c30de8740 1 -- start start 2024-09-11T23:02:11.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.581+0000 7f0c30de8740 1 Processor -- start 2024-09-11T23:02:11.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c30de8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fceade050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c30de8740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceada480 0x555fceade590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c30de8740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fceae4d40 0x555fceadecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c30de8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281aecb0 con 0x555fcead9b60 2024-09-11T23:02:11.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c30de8740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281ae5e0 con 0x555fceada480 2024-09-11T23:02:11.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c30de8740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f0c281ab840 con 0x555fceae4d40 2024-09-11T23:02:11.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c30de8740 10 client.0 _ll_register_callbacks cb 0x7f0c1800bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:11.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c2cfac640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fceade050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c2cfac640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fceade050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59474/0 (socket says 172.21.15.152:59474) 2024-09-11T23:02:11.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c2cfac640 1 -- 172.21.15.152:0/3601254312 learned_addr learned my addr 172.21.15.152:0/3601254312 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:11.590 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c27fff640 1 --2- 172.21.15.152:0/3601254312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceada480 0x555fceade590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/3601254312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fceae4d40 0x555fceadecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c2cfac640 1 -- 172.21.15.152:0/3601254312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fceae4d40 msgr2=0x555fceadecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:11.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/3601254312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fceae4d40 0x555fceadecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c2cfac640 1 -- 172.21.15.152:0/3601254312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceada480 msgr2=0x555fceade590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:11.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/3601254312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceada480 0x555fceade590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c2cfac640 1 -- 172.21.15.152:0/3601254312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fceadf500 con 0x555fcead9b60 2024-09-11T23:02:11.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c2cfac640 1 --2- 172.21.15.152:0/3601254312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fceade050 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0c20005c00 tx=0x7f0c20013690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7134fe84d7915dd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:11.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c077fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:11.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c077fe640 1 -- 172.21.15.152:0/3601254312 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20033070 con 0x555fcead9b60 2024-09-11T23:02:11.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c077fe640 1 -- 172.21.15.152:0/3601254312 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f0c200049e0 con 0x555fcead9b60 2024-09-11T23:02:11.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.582+0000 7f0c30de8740 1 -- 172.21.15.152:0/3601254312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c20013e50 con 0x555fcead9b60 2024-09-11T23:02:11.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.583+0000 7f0c077fe640 1 -- 172.21.15.152:0/3601254312 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0c2003d050 con 0x555fcead9b60 2024-09-11T23:02:11.591 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.583+0000 7f0c30de8740 10 client.18102 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:11.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.583+0000 7f0c30de8740 1 -- 172.21.15.152:0/3601254312 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555fceb64090 con 0x555fcead9b60 2024-09-11T23:02:11.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.583+0000 7f0c077fe640 1 -- 172.21.15.152:0/3601254312 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c20013e50 con 0x555fcead9b60 2024-09-11T23:02:11.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.583+0000 7f0c077fe640 1 -- 172.21.15.152:0/3601254312 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0c20029070 con 0x555fcead9b60 2024-09-11T23:02:11.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.583+0000 7f0c30de8740 10 client.18102 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:11.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.583+0000 7f0c30de8740 10 client.18102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:02:11.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.583+0000 7f0c30de8740 20 client.18102 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:11.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.583+0000 7f0c30de8740 1 --2- 172.21.15.152:0/3601254312 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555fceb64410 0x555fceb66800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:11.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.583+0000 7f0c30de8740 4 client.18102 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:02:11.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.583+0000 7f0c30de8740 1 -- 172.21.15.152:0/3601254312 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555fcea46d00 con 0x555fceb64410 2024-09-11T23:02:11.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.583+0000 7f0c27fff640 1 --2- 172.21.15.152:0/3601254312 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555fceb64410 0x555fceb66800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:11.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.584+0000 7f0c27fff640 1 --2- 172.21.15.152:0/3601254312 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555fceb64410 0x555fceb66800 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d19eea1969be1f61 server_cookie=a6c57440bf935466 in_seq=0 out_seq=0 2024-09-11T23:02:11.592 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.584+0000 7f0c077fe640 10 client.18102 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:11.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.585+0000 7f0c077fe640 1 -- 172.21.15.152:0/3601254312 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555fcea46d00 con 0x555fceb64410 2024-09-11T23:02:11.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.585+0000 7f0c077fe640 10 client.18102 handle_command_reply: tid=0 2024-09-11T23:02:11.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.585+0000 7f0c30de8740 1 client.18102 shutdown 2024-09-11T23:02:11.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.585+0000 7f0c30de8740 1 -- 172.21.15.152:0/3601254312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 msgr2=0x555fceade050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:11.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.585+0000 7f0c30de8740 1 --2- 172.21.15.152:0/3601254312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fceade050 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0c20005c00 tx=0x7f0c20013690 comp rx=0 tx=0).stop 2024-09-11T23:02:11.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.585+0000 7f0c30de8740 1 -- 172.21.15.152:0/3601254312 shutdown_connections 2024-09-11T23:02:11.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.585+0000 7f0c30de8740 1 --2- 172.21.15.152:0/3601254312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555fceae4d40 0x555fceadecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.585+0000 7f0c30de8740 1 --2- 172.21.15.152:0/3601254312 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x555fceb64410 0x555fceb66800 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.585+0000 7f0c30de8740 1 --2- 172.21.15.152:0/3601254312 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x555fceada480 0x555fceade590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.585+0000 7f0c30de8740 1 --2- 172.21.15.152:0/3601254312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555fcead9b60 0x555fceade050 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.593 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.585+0000 7f0c30de8740 1 -- 172.21.15.152:0/3601254312 >> 172.21.15.152:0/3601254312 conn(0x555fcead60f0 msgr2=0x555fceadbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:11.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.585+0000 7f0c077fe640 0 client.18102 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:11.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.585+0000 7f0c30de8740 1 -- 172.21.15.152:0/3601254312 shutdown_connections 2024-09-11T23:02:11.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.585+0000 7f0c30de8740 1 -- 172.21.15.152:0/3601254312 wait complete. 2024-09-11T23:02:11.594 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.585+0000 7f0c30de8740 20 client.18102 trim_cache size 0 max 16384 2024-09-11T23:02:11.594 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:02:11.594 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T23:02:11.595 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "41ea0c41-919c-4a06-a177-e83cfcea3268", 2024-09-11T23:02:11.595 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T23:02:11.595 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:02:11.596 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.586+0000 7f0c2ea0f640 1 -- 172.21.15.152:0/3311925637 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c100a2cf0 msgr2=0x7f0c100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:11.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.586+0000 7f0c2ea0f640 1 --2- 172.21.15.152:0/3311925637 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c100a2cf0 0x7f0c100a51b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c000c00 tx=0x7f0c1c08bfe0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.586+0000 7f0c2ea0f640 1 -- 172.21.15.152:0/3311925637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c28103660 msgr2=0x7f0c28198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:11.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.586+0000 7f0c2ea0f640 1 --2- 172.21.15.152:0/3311925637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c28103660 0x7f0c28198970 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c006010 tx=0x7f0c1c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.586+0000 7f0c2ea0f640 1 -- 172.21.15.152:0/3311925637 shutdown_connections 2024-09-11T23:02:11.597 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.586+0000 7f0c2ea0f640 1 --2- 172.21.15.152:0/3311925637 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f0c100a2cf0 0x7f0c100a51b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.586+0000 7f0c2ea0f640 1 --2- 172.21.15.152:0/3311925637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c2810c170 0x7f0c2819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.586+0000 7f0c2ea0f640 1 --2- 172.21.15.152:0/3311925637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0c28103660 0x7f0c28198970 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.586+0000 7f0c2ea0f640 1 --2- 172.21.15.152:0/3311925637 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c28102d40 0x7f0c28198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:11.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.586+0000 7f0c2ea0f640 1 -- 172.21.15.152:0/3311925637 >> 172.21.15.152:0/3311925637 conn(0x7f0c280fc7a0 msgr2=0x7f0c280fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:11.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.586+0000 7f0c2ea0f640 1 -- 172.21.15.152:0/3311925637 shutdown_connections 2024-09-11T23:02:11.598 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:11.586+0000 7f0c2ea0f640 1 -- 172.21.15.152:0/3311925637 wait complete. 2024-09-11T23:02:12.573 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:02:12.821 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:12.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:12 smithi152 ceph-mon[31181]: pgmap v287: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:02:12.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:12 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T23:02:12.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:12 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T23:02:12.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:12 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T23:02:12.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:12 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T23:02:13.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:12 smithi191 ceph-mon[35211]: pgmap v287: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:02:13.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:12 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T23:02:13.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:12 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T23:02:13.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:12 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T23:02:13.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:12 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T23:02:13.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:12 smithi204 ceph-mon[35454]: pgmap v287: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:02:13.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:12 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T23:02:13.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:12 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T23:02:13.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:12 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T23:02:13.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:12 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T23:02:13.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.337+0000 7fc8a1d44640 1 -- 172.21.15.152:0/1788243573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c103660 msgr2=0x7fc89c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:13.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.337+0000 7fc8a1d44640 1 --2- 172.21.15.152:0/1788243573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c103660 0x7fc89c10bb50 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fc890009900 tx=0x7fc89002a480 comp rx=0 tx=0).stop 2024-09-11T23:02:13.373 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.338+0000 7fc8a1d44640 1 -- 172.21.15.152:0/1788243573 shutdown_connections 2024-09-11T23:02:13.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.338+0000 7fc8a1d44640 1 --2- 172.21.15.152:0/1788243573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc89c10c170 0x7fc89c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.338+0000 7fc8a1d44640 1 --2- 172.21.15.152:0/1788243573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c103660 0x7fc89c10bb50 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.338+0000 7fc8a1d44640 1 --2- 172.21.15.152:0/1788243573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc89c102d40 0x7fc89c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.338+0000 7fc8a1d44640 1 -- 172.21.15.152:0/1788243573 >> 172.21.15.152:0/1788243573 conn(0x7fc89c0fc7a0 msgr2=0x7fc89c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:13.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.338+0000 7fc8a1d44640 1 -- 172.21.15.152:0/1788243573 shutdown_connections 2024-09-11T23:02:13.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.338+0000 7fc8a1d44640 1 -- 172.21.15.152:0/1788243573 wait complete. 2024-09-11T23:02:13.374 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.338+0000 7fc8a1d44640 1 Processor -- start 2024-09-11T23:02:13.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.338+0000 7fc8a1d44640 1 -- start start 2024-09-11T23:02:13.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc8a1d44640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc89c102d40 0x7fc89c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc8a1d44640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c103660 0x7fc89c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc8a1d44640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc89c10c170 0x7fc89c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.377 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc8a1d44640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c111f70 con 0x7fc89c103660 2024-09-11T23:02:13.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc8a1d44640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c111df0 con 0x7fc89c102d40 2024-09-11T23:02:13.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc8a1d44640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1120f0 con 0x7fc89c10c170 2024-09-11T23:02:13.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc89affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c103660 0x7fc89c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc89affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c103660 0x7fc89c198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59478/0 (socket says 172.21.15.152:59478) 2024-09-11T23:02:13.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc89affd640 1 -- 172.21.15.152:0/3974187296 learned_addr learned my addr 172.21.15.152:0/3974187296 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:13.378 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc89bfff640 1 --2- 172.21.15.152:0/3974187296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc89c10c170 0x7fc89c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc89b7fe640 1 --2- 172.21.15.152:0/3974187296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc89c102d40 0x7fc89c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc89affd640 1 -- 172.21.15.152:0/3974187296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc89c10c170 msgr2=0x7fc89c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:13.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc89affd640 1 --2- 172.21.15.152:0/3974187296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc89c10c170 0x7fc89c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc89affd640 1 -- 172.21.15.152:0/3974187296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc89c102d40 msgr2=0x7fc89c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:13.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc89affd640 1 --2- 172.21.15.152:0/3974187296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc89c102d40 0x7fc89c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.379 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc89affd640 1 -- 172.21.15.152:0/3974187296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89c19d400 con 0x7fc89c103660 2024-09-11T23:02:13.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc89affd640 1 --2- 172.21.15.152:0/3974187296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c103660 0x7fc89c198990 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fc890006010 tx=0x7fc8900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:13.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc898ff9640 1 -- 172.21.15.152:0/3974187296 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89003f030 con 0x7fc89c103660 2024-09-11T23:02:13.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc898ff9640 1 -- 172.21.15.152:0/3974187296 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc890003ce0 con 0x7fc89c103660 2024-09-11T23:02:13.380 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc898ff9640 1 -- 172.21.15.152:0/3974187296 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890035070 con 0x7fc89c103660 2024-09-11T23:02:13.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.339+0000 7fc8a1d44640 1 -- 172.21.15.152:0/3974187296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89c19d690 con 0x7fc89c103660 2024-09-11T23:02:13.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.340+0000 7fc8a1d44640 1 -- 172.21.15.152:0/3974187296 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89c1a4fd0 con 0x7fc89c103660 2024-09-11T23:02:13.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.342+0000 7fc898ff9640 1 -- 172.21.15.152:0/3974187296 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc890042030 con 0x7fc89c103660 2024-09-11T23:02:13.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.342+0000 7fc8a411d740 1 Processor -- start 2024-09-11T23:02:13.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.342+0000 7fc8a411d740 1 -- start start 2024-09-11T23:02:13.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.342+0000 7fc8a411d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee7204a0 0x55b0ee720880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc8a411d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee720dc0 0x55b0ee72b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc8a411d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee72b680 0x55b0ee72da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc8a411d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55b0ee6862e0 con 0x55b0ee7204a0 2024-09-11T23:02:13.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc8a411d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b0ee62dce0 con 0x55b0ee72b680 2024-09-11T23:02:13.381 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc8a411d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b0ee6822a0 con 0x55b0ee720dc0 2024-09-11T23:02:13.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc898ff9640 1 --2- 172.21.15.152:0/3974187296 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8840a2cf0 0x7fc8840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc89bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee7204a0 0x55b0ee720880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc89bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee7204a0 0x55b0ee720880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59488/0 (socket says 172.21.15.152:59488) 2024-09-11T23:02:13.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc89bfff640 1 -- 172.21.15.152:0/3107029178 learned_addr learned my addr 172.21.15.152:0/3107029178 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:13.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc89bfff640 1 --2- 172.21.15.152:0/3974187296 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8840a2cf0 0x7fc8840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc89b7fe640 1 --2- 172.21.15.152:0/3107029178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee720dc0 0x55b0ee72b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc89affd640 1 --2- 172.21.15.152:0/3107029178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee72b680 0x55b0ee72da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc898ff9640 1 -- 172.21.15.152:0/3974187296 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7fc8900ef970 con 0x7fc89c103660 2024-09-11T23:02:13.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc89bfff640 1 -- 172.21.15.152:0/3107029178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee720dc0 msgr2=0x55b0ee72b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:13.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc89bfff640 1 --2- 172.21.15.152:0/3107029178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee720dc0 0x55b0ee72b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc89bfff640 1 -- 172.21.15.152:0/3107029178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee72b680 msgr2=0x55b0ee72da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:13.382 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc89bfff640 1 --2- 172.21.15.152:0/3107029178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee72b680 0x55b0ee72da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc89bfff640 1 -- 172.21.15.152:0/3107029178 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ee612440 con 0x55b0ee7204a0 2024-09-11T23:02:13.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc89bfff640 1 --2- 172.21.15.152:0/3107029178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee7204a0 0x55b0ee720880 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fc89400bd20 tx=0x7fc8940039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a210fff18be543bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:13.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc881ffb640 1 -- 172.21.15.152:0/3107029178 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89400b6e0 con 0x55b0ee7204a0 2024-09-11T23:02:13.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.343+0000 7fc881ffb640 1 -- 172.21.15.152:0/3107029178 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8940054b0 con 0x55b0ee7204a0 2024-09-11T23:02:13.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.344+0000 7fc881ffb640 1 -- 172.21.15.152:0/3107029178 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89401ed30 con 0x55b0ee7204a0 2024-09-11T23:02:13.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.344+0000 7fc8a411d740 1 -- 172.21.15.152:0/3107029178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee7204a0 msgr2=0x55b0ee720880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:13.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.344+0000 7fc89bfff640 1 --2- 172.21.15.152:0/3974187296 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8840a2cf0 0x7fc8840a51b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc894021000 tx=0x7fc894020000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:13.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.344+0000 7fc8a411d740 1 --2- 172.21.15.152:0/3107029178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee7204a0 0x55b0ee720880 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fc89400bd20 tx=0x7fc8940039c0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.344+0000 7fc8a411d740 1 -- 172.21.15.152:0/3107029178 shutdown_connections 2024-09-11T23:02:13.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.344+0000 7fc8a411d740 1 --2- 172.21.15.152:0/3107029178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee72b680 0x55b0ee72da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.344+0000 7fc8a411d740 1 --2- 172.21.15.152:0/3107029178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee720dc0 0x55b0ee72b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.344+0000 7fc8a411d740 1 --2- 172.21.15.152:0/3107029178 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee7204a0 0x55b0ee720880 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.383 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.344+0000 7fc8a411d740 1 -- 172.21.15.152:0/3107029178 >> 172.21.15.152:0/3107029178 conn(0x55b0ee71ca30 msgr2=0x55b0ee71ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:13.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.344+0000 7fc8a411d740 1 -- 172.21.15.152:0/3107029178 shutdown_connections 2024-09-11T23:02:13.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.344+0000 7fc8a411d740 1 -- 172.21.15.152:0/3107029178 wait complete. 2024-09-11T23:02:13.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc8a411d740 1 -- start start 2024-09-11T23:02:13.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc8a411d740 1 Processor -- start 2024-09-11T23:02:13.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc8a411d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee7204a0 0x55b0ee72afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc8a411d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee720dc0 0x55b0ee728270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc8a411d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee72b680 0x55b0ee7287b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc8a411d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55b0ee72e250 con 0x55b0ee720dc0 2024-09-11T23:02:13.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc8a411d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b0ee641050 con 0x55b0ee7204a0 2024-09-11T23:02:13.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc8a411d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b0ee72e3d0 con 0x55b0ee72b680 2024-09-11T23:02:13.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc8a411d740 10 client.0 _ll_register_callbacks cb 0x55b0ee681160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:13.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc89affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee720dc0 0x55b0ee728270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.384 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc89affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee720dc0 0x55b0ee728270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59502/0 (socket says 172.21.15.152:59502) 2024-09-11T23:02:13.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc89affd640 1 -- 172.21.15.152:0/3293348150 learned_addr learned my addr 172.21.15.152:0/3293348150 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:13.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc89b7fe640 1 --2- 172.21.15.152:0/3293348150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee7204a0 0x55b0ee72afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc89bfff640 1 --2- 172.21.15.152:0/3293348150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee72b680 0x55b0ee7287b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc89affd640 1 -- 172.21.15.152:0/3293348150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee72b680 msgr2=0x55b0ee7287b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:13.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc89affd640 1 --2- 172.21.15.152:0/3293348150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee72b680 0x55b0ee7287b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc89affd640 1 -- 172.21.15.152:0/3293348150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee7204a0 msgr2=0x55b0ee72afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:13.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc89affd640 1 --2- 172.21.15.152:0/3293348150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee7204a0 0x55b0ee72afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.345+0000 7fc89affd640 1 -- 172.21.15.152:0/3293348150 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ee3f73c0 con 0x55b0ee720dc0 2024-09-11T23:02:13.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.346+0000 7fc89affd640 1 --2- 172.21.15.152:0/3293348150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee720dc0 0x55b0ee728270 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fc8900efc10 tx=0x7fc89003b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c62a90c11085d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:13.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.346+0000 7fc87affd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:13.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.346+0000 7fc87affd640 1 -- 172.21.15.152:0/3293348150 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890049040 con 0x55b0ee720dc0 2024-09-11T23:02:13.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.346+0000 7fc87affd640 1 -- 172.21.15.152:0/3293348150 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7fc8900c46c0 con 0x55b0ee720dc0 2024-09-11T23:02:13.385 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.346+0000 7fc8a411d740 1 -- 172.21.15.152:0/3293348150 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8940054b0 con 0x55b0ee720dc0 2024-09-11T23:02:13.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.346+0000 7fc87affd640 1 -- 172.21.15.152:0/3293348150 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc8900bb8f0 con 0x55b0ee720dc0 2024-09-11T23:02:13.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.346+0000 7fc8a411d740 10 client.18114 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:13.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.346+0000 7fc8a411d740 1 -- 172.21.15.152:0/3293348150 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b0ee739c00 con 0x55b0ee720dc0 2024-09-11T23:02:13.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.346+0000 7fc87affd640 1 -- 172.21.15.152:0/3293348150 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8940054b0 con 0x55b0ee720dc0 2024-09-11T23:02:13.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.346+0000 7fc87affd640 1 -- 172.21.15.152:0/3293348150 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc89003bbb0 con 0x55b0ee720dc0 2024-09-11T23:02:13.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.346+0000 7fc8a411d740 10 client.18114 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:13.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.346+0000 7fc8a411d740 10 client.18114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:02:13.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.346+0000 7fc8a411d740 20 client.18114 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:13.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.346+0000 7fc8a411d740 1 --2- 172.21.15.152:0/3293348150 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b0ee73afa0 0x55b0ee73d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.346+0000 7fc8a411d740 4 client.18114 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:02:13.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.346+0000 7fc8a411d740 1 -- 172.21.15.152:0/3293348150 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0ee693850 con 0x55b0ee73afa0 2024-09-11T23:02:13.386 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.347+0000 7fc89b7fe640 1 --2- 172.21.15.152:0/3293348150 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b0ee73afa0 0x55b0ee73d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.347+0000 7fc89b7fe640 1 --2- 172.21.15.152:0/3293348150 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b0ee73afa0 0x55b0ee73d3d0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95757e19d380265e server_cookie=12787ac47da65ee6 in_seq=0 out_seq=0 2024-09-11T23:02:13.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.347+0000 7fc87affd640 10 client.18114 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:13.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.348+0000 7fc87affd640 1 -- 172.21.15.152:0/3293348150 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0ee693850 con 0x55b0ee73afa0 2024-09-11T23:02:13.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.348+0000 7fc87affd640 10 client.18114 handle_command_reply: tid=0 2024-09-11T23:02:13.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.348+0000 7fc8a411d740 1 client.18114 shutdown 2024-09-11T23:02:13.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.348+0000 7fc8a411d740 1 -- 172.21.15.152:0/3293348150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee720dc0 msgr2=0x55b0ee728270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:13.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.348+0000 7fc8a411d740 1 --2- 172.21.15.152:0/3293348150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee720dc0 0x55b0ee728270 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fc8900efc10 tx=0x7fc89003b510 comp rx=0 tx=0).stop 2024-09-11T23:02:13.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.348+0000 7fc8a411d740 1 -- 172.21.15.152:0/3293348150 shutdown_connections 2024-09-11T23:02:13.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.348+0000 7fc8a411d740 1 --2- 172.21.15.152:0/3293348150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee72b680 0x55b0ee7287b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.348+0000 7fc8a411d740 1 --2- 172.21.15.152:0/3293348150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee720dc0 0x55b0ee728270 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.348+0000 7fc8a411d740 1 --2- 172.21.15.152:0/3293348150 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b0ee73afa0 0x55b0ee73d3d0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.348+0000 7fc8a411d740 1 --2- 172.21.15.152:0/3293348150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee7204a0 0x55b0ee72afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.348+0000 7fc8a411d740 1 -- 172.21.15.152:0/3293348150 >> 172.21.15.152:0/3293348150 conn(0x55b0ee71ca30 msgr2=0x55b0ee722710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:13.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.348+0000 7fc87affd640 0 client.18114 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:13.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.348+0000 7fc8a411d740 1 -- 172.21.15.152:0/3293348150 shutdown_connections 2024-09-11T23:02:13.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.348+0000 7fc8a411d740 1 -- 172.21.15.152:0/3293348150 wait complete. 2024-09-11T23:02:13.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.348+0000 7fc8a411d740 20 client.18114 trim_cache size 0 max 16384 2024-09-11T23:02:13.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc8a411d740 1 Processor -- start 2024-09-11T23:02:13.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc8a411d740 1 -- start start 2024-09-11T23:02:13.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc8a411d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee7204a0 0x55b0ee7374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc8a411d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee720dc0 0x55b0ee742ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc8a411d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee72b680 0x55b0ee7431e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc8a411d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1ab670 con 0x55b0ee7204a0 2024-09-11T23:02:13.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc8a411d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1ad1b0 con 0x55b0ee720dc0 2024-09-11T23:02:13.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc8a411d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1abd40 con 0x55b0ee72b680 2024-09-11T23:02:13.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc89b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee7204a0 0x55b0ee7374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc89b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee7204a0 0x55b0ee7374b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59510/0 (socket says 172.21.15.152:59510) 2024-09-11T23:02:13.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc89b7fe640 1 -- 172.21.15.152:0/1389608291 learned_addr learned my addr 172.21.15.152:0/1389608291 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:13.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc89affd640 1 --2- 172.21.15.152:0/1389608291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee720dc0 0x55b0ee742ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc89b7fe640 1 -- 172.21.15.152:0/1389608291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee72b680 msgr2=0x55b0ee7431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:13.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc89b7fe640 1 --2- 172.21.15.152:0/1389608291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee72b680 0x55b0ee7431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc89b7fe640 1 -- 172.21.15.152:0/1389608291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee720dc0 msgr2=0x55b0ee742ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:13.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc89b7fe640 1 --2- 172.21.15.152:0/1389608291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee720dc0 0x55b0ee742ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc89b7fe640 1 -- 172.21.15.152:0/1389608291 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ee72ddb0 con 0x55b0ee7204a0 2024-09-11T23:02:13.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc89affd640 1 --2- 172.21.15.152:0/1389608291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee720dc0 0x55b0ee742ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:13.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc89b7fe640 1 --2- 172.21.15.152:0/1389608291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee7204a0 0x55b0ee7374b0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x55b0ee73dc40 tx=0x7fc88c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09a5a9368fb2003 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:13.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc8817fa640 1 -- 172.21.15.152:0/1389608291 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c01b3f0 con 0x55b0ee7204a0 2024-09-11T23:02:13.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc8817fa640 1 -- 172.21.15.152:0/1389608291 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc88c01ba70 con 0x55b0ee7204a0 2024-09-11T23:02:13.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.365+0000 7fc8817fa640 1 -- 172.21.15.152:0/1389608291 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c012d80 con 0x55b0ee7204a0 2024-09-11T23:02:13.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.366+0000 7fc8a411d740 1 -- 172.21.15.152:0/1389608291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee7204a0 msgr2=0x55b0ee7374b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:13.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.366+0000 7fc8a411d740 1 --2- 172.21.15.152:0/1389608291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee7204a0 0x55b0ee7374b0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x55b0ee73dc40 tx=0x7fc88c009680 comp rx=0 tx=0).stop 2024-09-11T23:02:13.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.366+0000 7fc8a411d740 1 -- 172.21.15.152:0/1389608291 shutdown_connections 2024-09-11T23:02:13.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.366+0000 7fc8a411d740 1 --2- 172.21.15.152:0/1389608291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee72b680 0x55b0ee7431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.366+0000 7fc8a411d740 1 --2- 172.21.15.152:0/1389608291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee720dc0 0x55b0ee742ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.366+0000 7fc8a411d740 1 --2- 172.21.15.152:0/1389608291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee7204a0 0x55b0ee7374b0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.366+0000 7fc8a411d740 1 -- 172.21.15.152:0/1389608291 >> 172.21.15.152:0/1389608291 conn(0x55b0ee71ca30 msgr2=0x55b0ee727360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:13.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.366+0000 7fc8a411d740 1 -- 172.21.15.152:0/1389608291 shutdown_connections 2024-09-11T23:02:13.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.366+0000 7fc8a411d740 1 -- 172.21.15.152:0/1389608291 wait complete. 2024-09-11T23:02:13.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.366+0000 7fc8a411d740 1 -- start start 2024-09-11T23:02:13.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc8a411d740 1 Processor -- start 2024-09-11T23:02:13.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc8a411d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee7204a0 0x55b0ee79cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc8a411d740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee720dc0 0x55b0ee79d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc8a411d740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee72b680 0x55b0ee730be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc8a411d740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1acae0 con 0x55b0ee720dc0 2024-09-11T23:02:13.392 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc8a411d740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1ac410 con 0x55b0ee72b680 2024-09-11T23:02:13.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc8a411d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1a9670 con 0x55b0ee7204a0 2024-09-11T23:02:13.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc8a411d740 10 client.0 _ll_register_callbacks cb 0x7fc8900b7bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:13.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc89affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee720dc0 0x55b0ee79d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc89affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee720dc0 0x55b0ee79d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59516/0 (socket says 172.21.15.152:59516) 2024-09-11T23:02:13.393 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc89affd640 1 -- 172.21.15.152:0/2007325653 learned_addr learned my addr 172.21.15.152:0/2007325653 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:13.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc89b7fe640 1 --2- 172.21.15.152:0/2007325653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee7204a0 0x55b0ee79cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc89bfff640 1 --2- 172.21.15.152:0/2007325653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee72b680 0x55b0ee730be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc89affd640 1 -- 172.21.15.152:0/2007325653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee7204a0 msgr2=0x55b0ee79cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:13.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc89affd640 1 --2- 172.21.15.152:0/2007325653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee7204a0 0x55b0ee79cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc89affd640 1 -- 172.21.15.152:0/2007325653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee72b680 msgr2=0x55b0ee730be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:13.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc89affd640 1 --2- 172.21.15.152:0/2007325653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee72b680 0x55b0ee730be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc89affd640 1 -- 172.21.15.152:0/2007325653 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ee731410 con 0x55b0ee720dc0 2024-09-11T23:02:13.394 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc89bfff640 1 --2- 172.21.15.152:0/2007325653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee72b680 0x55b0ee730be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:13.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc89affd640 1 --2- 172.21.15.152:0/2007325653 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee720dc0 0x55b0ee79d530 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fc8900b83f0 tx=0x7fc8900ed500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782560a75b456a67 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:13.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.367+0000 7fc87affd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:13.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.368+0000 7fc87affd640 1 -- 172.21.15.152:0/2007325653 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890049070 con 0x55b0ee720dc0 2024-09-11T23:02:13.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.368+0000 7fc87affd640 1 -- 172.21.15.152:0/2007325653 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7fc890003a10 con 0x55b0ee720dc0 2024-09-11T23:02:13.395 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.368+0000 7fc8a411d740 1 -- 172.21.15.152:0/2007325653 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc88c01ba70 con 0x55b0ee720dc0 2024-09-11T23:02:13.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.368+0000 7fc87affd640 1 -- 172.21.15.152:0/2007325653 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc8900bb700 con 0x55b0ee720dc0 2024-09-11T23:02:13.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.368+0000 7fc8a411d740 10 client.18120 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:13.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.368+0000 7fc8a411d740 1 -- 172.21.15.152:0/2007325653 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b0ee7a5730 con 0x55b0ee720dc0 2024-09-11T23:02:13.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.368+0000 7fc87affd640 1 -- 172.21.15.152:0/2007325653 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc88c01ba70 con 0x55b0ee720dc0 2024-09-11T23:02:13.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.368+0000 7fc87affd640 1 -- 172.21.15.152:0/2007325653 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc890031af0 con 0x55b0ee720dc0 2024-09-11T23:02:13.396 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.368+0000 7fc8a411d740 10 client.18120 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:13.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.368+0000 7fc8a411d740 10 client.18120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:02:13.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.368+0000 7fc8a411d740 20 client.18120 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:13.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.368+0000 7fc8a411d740 1 --2- 172.21.15.152:0/2007325653 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b0ee7a5d40 0x55b0ee7a8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:13.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.368+0000 7fc8a411d740 4 client.18120 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:02:13.397 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.368+0000 7fc8a411d740 1 -- 172.21.15.152:0/2007325653 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b0ee68dd60 con 0x55b0ee7a5d40 2024-09-11T23:02:13.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.369+0000 7fc89b7fe640 1 --2- 172.21.15.152:0/2007325653 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b0ee7a5d40 0x55b0ee7a8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:13.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.369+0000 7fc87affd640 10 client.18120 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:13.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.369+0000 7fc89b7fe640 1 --2- 172.21.15.152:0/2007325653 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b0ee7a5d40 0x55b0ee7a8160 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24395a5c51d1db0c server_cookie=8b644a3d1cfd0728 in_seq=0 out_seq=0 2024-09-11T23:02:13.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.369+0000 7fc87affd640 1 -- 172.21.15.152:0/2007325653 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b0ee68dd60 con 0x55b0ee7a5d40 2024-09-11T23:02:13.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.369+0000 7fc87affd640 10 client.18120 handle_command_reply: tid=0 2024-09-11T23:02:13.398 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.369+0000 7fc8a411d740 1 client.18120 shutdown 2024-09-11T23:02:13.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.369+0000 7fc8a411d740 1 -- 172.21.15.152:0/2007325653 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee720dc0 msgr2=0x55b0ee79d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:13.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.369+0000 7fc8a411d740 1 --2- 172.21.15.152:0/2007325653 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee720dc0 0x55b0ee79d530 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fc8900b83f0 tx=0x7fc8900ed500 comp rx=0 tx=0).stop 2024-09-11T23:02:13.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.369+0000 7fc8a411d740 1 -- 172.21.15.152:0/2007325653 shutdown_connections 2024-09-11T23:02:13.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.369+0000 7fc8a411d740 1 --2- 172.21.15.152:0/2007325653 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55b0ee7a5d40 0x55b0ee7a8160 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.369+0000 7fc8a411d740 1 --2- 172.21.15.152:0/2007325653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b0ee72b680 0x55b0ee730be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.369+0000 7fc8a411d740 1 --2- 172.21.15.152:0/2007325653 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b0ee720dc0 0x55b0ee79d530 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.369+0000 7fc87affd640 0 client.18120 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:13.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.369+0000 7fc8a411d740 1 --2- 172.21.15.152:0/2007325653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b0ee7204a0 0x55b0ee79cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.369+0000 7fc8a411d740 1 -- 172.21.15.152:0/2007325653 >> 172.21.15.152:0/2007325653 conn(0x55b0ee71ca30 msgr2=0x55b0ee71b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:13.399 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.369+0000 7fc8a411d740 1 -- 172.21.15.152:0/2007325653 shutdown_connections 2024-09-11T23:02:13.400 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.370+0000 7fc8a411d740 1 -- 172.21.15.152:0/2007325653 wait complete. 2024-09-11T23:02:13.400 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:02:13.400 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T23:02:13.400 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T23:02:13.400 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:02:13.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.370+0000 7fc8a411d740 20 client.18120 trim_cache size 0 max 16384 2024-09-11T23:02:13.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.370+0000 7fc8a1d44640 1 -- 172.21.15.152:0/3974187296 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8840a2cf0 msgr2=0x7fc8840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:13.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.370+0000 7fc8a1d44640 1 --2- 172.21.15.152:0/3974187296 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8840a2cf0 0x7fc8840a51b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc894021000 tx=0x7fc894020000 comp rx=0 tx=0).stop 2024-09-11T23:02:13.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.370+0000 7fc8a1d44640 1 -- 172.21.15.152:0/3974187296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c103660 msgr2=0x7fc89c198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:13.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.370+0000 7fc8a1d44640 1 --2- 172.21.15.152:0/3974187296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c103660 0x7fc89c198990 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fc890006010 tx=0x7fc8900042f0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.370+0000 7fc8a1d44640 1 -- 172.21.15.152:0/3974187296 shutdown_connections 2024-09-11T23:02:13.401 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.370+0000 7fc8a1d44640 1 --2- 172.21.15.152:0/3974187296 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc8840a2cf0 0x7fc8840a51b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.370+0000 7fc8a1d44640 1 --2- 172.21.15.152:0/3974187296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc89c10c170 0x7fc89c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.370+0000 7fc8a1d44640 1 --2- 172.21.15.152:0/3974187296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc89c103660 0x7fc89c198990 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.370+0000 7fc8a1d44640 1 --2- 172.21.15.152:0/3974187296 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc89c102d40 0x7fc89c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:13.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.370+0000 7fc8a1d44640 1 -- 172.21.15.152:0/3974187296 >> 172.21.15.152:0/3974187296 conn(0x7fc89c0fc7a0 msgr2=0x7fc89c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:13.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.370+0000 7fc8a1d44640 1 -- 172.21.15.152:0/3974187296 shutdown_connections 2024-09-11T23:02:13.402 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:13.370+0000 7fc8a1d44640 1 -- 172.21.15.152:0/3974187296 wait complete. 2024-09-11T23:02:14.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:13 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T23:02:14.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:13 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T23:02:14.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:13 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T23:02:14.187 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:02:14.187 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:02:14.438 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:14.850 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:14 smithi152 ceph-mon[31181]: pgmap v288: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:02:14.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.929+0000 7f91b7855640 1 -- 172.21.15.152:0/3506430551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b010c1a0 msgr2=0x7f91b010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:14.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.929+0000 7f91b7855640 1 --2- 172.21.15.152:0/3506430551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b010c1a0 0x7f91b010e610 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009900 tx=0x7f91a802a480 comp rx=0 tx=0).stop 2024-09-11T23:02:14.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.929+0000 7f91b7855640 1 -- 172.21.15.152:0/3506430551 shutdown_connections 2024-09-11T23:02:14.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.929+0000 7f91b7855640 1 --2- 172.21.15.152:0/3506430551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b010c1a0 0x7f91b010e610 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.929+0000 7f91b7855640 1 --2- 172.21.15.152:0/3506430551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91b0103720 0x7f91b010bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.929+0000 7f91b7855640 1 --2- 172.21.15.152:0/3506430551 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b0102e00 0x7f91b01031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.929+0000 7f91b7855640 1 -- 172.21.15.152:0/3506430551 >> 172.21.15.152:0/3506430551 conn(0x7f91b00fc7a0 msgr2=0x7f91b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.930+0000 7f91b7855640 1 -- 172.21.15.152:0/3506430551 shutdown_connections 2024-09-11T23:02:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.930+0000 7f91b7855640 1 -- 172.21.15.152:0/3506430551 wait complete. 2024-09-11T23:02:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.930+0000 7f91b7855640 1 Processor -- start 2024-09-11T23:02:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.930+0000 7f91b7855640 1 -- start start 2024-09-11T23:02:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b7855640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b0102e00 0x7f91b01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b7855640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b0103720 0x7f91b0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b7855640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91b010c1a0 0x7f91b019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b7855640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f91b0111f70 con 0x7f91b0102e00 2024-09-11T23:02:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b7855640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f91b0111df0 con 0x7f91b010c1a0 2024-09-11T23:02:14.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b7855640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f91b01120f0 con 0x7f91b0103720 2024-09-11T23:02:14.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b55f1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b0102e00 0x7f91b01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:14.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b55f1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b0102e00 0x7f91b01983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59526/0 (socket says 172.21.15.152:59526) 2024-09-11T23:02:14.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b55f1640 1 -- 172.21.15.152:0/2572729040 learned_addr learned my addr 172.21.15.152:0/2572729040 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:14.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b5df2640 1 --2- 172.21.15.152:0/2572729040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91b010c1a0 0x7f91b019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:14.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b4df0640 1 --2- 172.21.15.152:0/2572729040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b0103720 0x7f91b0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:14.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b55f1640 1 -- 172.21.15.152:0/2572729040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b0103720 msgr2=0x7f91b0198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:14.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b55f1640 1 --2- 172.21.15.152:0/2572729040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b0103720 0x7f91b0198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:14.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b55f1640 1 -- 172.21.15.152:0/2572729040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91b010c1a0 msgr2=0x7f91b019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:14.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b55f1640 1 --2- 172.21.15.152:0/2572729040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91b010c1a0 0x7f91b019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:14.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b55f1640 1 -- 172.21.15.152:0/2572729040 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b019d3a0 con 0x7f91b0102e00 2024-09-11T23:02:14.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b4df0640 1 --2- 172.21.15.152:0/2572729040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b0103720 0x7f91b0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:14.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b5df2640 1 --2- 172.21.15.152:0/2572729040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91b010c1a0 0x7f91b019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:14.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b55f1640 1 --2- 172.21.15.152:0/2572729040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b0102e00 0x7f91b01983f0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f91a000cce0 tx=0x7f91a000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:14.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91ae7fc640 1 -- 172.21.15.152:0/2572729040 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0004b20 con 0x7f91b0102e00 2024-09-11T23:02:14.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b7855640 1 -- 172.21.15.152:0/2572729040 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b019d690 con 0x7f91b0102e00 2024-09-11T23:02:14.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91ae7fc640 1 -- 172.21.15.152:0/2572729040 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91a0004cc0 con 0x7f91b0102e00 2024-09-11T23:02:14.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91ae7fc640 1 -- 172.21.15.152:0/2572729040 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a000f8c0 con 0x7f91b0102e00 2024-09-11T23:02:14.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.931+0000 7f91b7855640 1 -- 172.21.15.152:0/2572729040 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b01a4f70 con 0x7f91b0102e00 2024-09-11T23:02:14.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.932+0000 7f91b7855640 1 -- 172.21.15.152:0/2572729040 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b01031e0 con 0x7f91b0102e00 2024-09-11T23:02:14.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.933+0000 7f91ae7fc640 1 -- 172.21.15.152:0/2572729040 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f91a000fa60 con 0x7f91b0102e00 2024-09-11T23:02:14.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.934+0000 7f91ae7fc640 1 --2- 172.21.15.152:0/2572729040 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f91980a2cf0 0x7f91980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:14.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.934+0000 7f91b4df0640 1 --2- 172.21.15.152:0/2572729040 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f91980a2cf0 0x7f91980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:14.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.934+0000 7f91ae7fc640 1 -- 172.21.15.152:0/2572729040 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f91a00cf5e0 con 0x7f91b0102e00 2024-09-11T23:02:14.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.935+0000 7f91b4df0640 1 --2- 172.21.15.152:0/2572729040 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f91980a2cf0 0x7f91980a51b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f91b010d4e0 tx=0x7f91a4005d20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:14.963 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:14.938+0000 7f91ae7fc640 1 -- 172.21.15.152:0/2572729040 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f91a0014030 con 0x7f91b0102e00 2024-09-11T23:02:15.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:14 smithi191 ceph-mon[35211]: pgmap v288: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:02:15.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:14 smithi204 ceph-mon[35454]: pgmap v288: 97 pgs: 97 active+clean; 4.0 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-11T23:02:15.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:15.140+0000 7f91b7855640 1 -- 172.21.15.152:0/2572729040 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91b01a5170 con 0x7f91b0102e00 2024-09-11T23:02:15.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:15.141+0000 7f91ae7fc640 1 -- 172.21.15.152:0/2572729040 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f91a009c1f0 con 0x7f91b0102e00 2024-09-11T23:02:15.141 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T23:02:15.142 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T23:02:15.142 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T23:02:15.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:15.143+0000 7f91b7855640 1 -- 172.21.15.152:0/2572729040 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f91980a2cf0 msgr2=0x7f91980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:15.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:15.143+0000 7f91b7855640 1 --2- 172.21.15.152:0/2572729040 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f91980a2cf0 0x7f91980a51b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f91b010d4e0 tx=0x7f91a4005d20 comp rx=0 tx=0).stop 2024-09-11T23:02:15.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:15.143+0000 7f91b7855640 1 -- 172.21.15.152:0/2572729040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b0102e00 msgr2=0x7f91b01983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:15.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:15.143+0000 7f91b7855640 1 --2- 172.21.15.152:0/2572729040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b0102e00 0x7f91b01983f0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f91a000cce0 tx=0x7f91a000a620 comp rx=0 tx=0).stop 2024-09-11T23:02:15.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:15.143+0000 7f91b7855640 1 -- 172.21.15.152:0/2572729040 shutdown_connections 2024-09-11T23:02:15.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:15.143+0000 7f91b7855640 1 --2- 172.21.15.152:0/2572729040 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f91980a2cf0 0x7f91980a51b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:15.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:15.143+0000 7f91b7855640 1 --2- 172.21.15.152:0/2572729040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91b010c1a0 0x7f91b019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:15.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:15.143+0000 7f91b7855640 1 --2- 172.21.15.152:0/2572729040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91b0103720 0x7f91b0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:15.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:15.143+0000 7f91b7855640 1 --2- 172.21.15.152:0/2572729040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91b0102e00 0x7f91b01983f0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:15.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:15.143+0000 7f91b7855640 1 -- 172.21.15.152:0/2572729040 >> 172.21.15.152:0/2572729040 conn(0x7f91b00fc7a0 msgr2=0x7f91b00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:15.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:15.143+0000 7f91b7855640 1 -- 172.21.15.152:0/2572729040 shutdown_connections 2024-09-11T23:02:15.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:15.143+0000 7f91b7855640 1 -- 172.21.15.152:0/2572729040 wait complete. 2024-09-11T23:02:15.839 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:15 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2572729040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:02:15.869 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:02:16.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:15 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2572729040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:02:16.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:15 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2572729040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:02:16.115 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.598+0000 7f3532b9e640 1 -- 172.21.15.152:0/1487318490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f352c1008d0 msgr2=0x7f352c100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.599+0000 7f3532b9e640 1 --2- 172.21.15.152:0/1487318490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f352c1008d0 0x7f352c100d30 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f351c009930 tx=0x7f351c02a420 comp rx=0 tx=0).stop 2024-09-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.599+0000 7f3532b9e640 1 -- 172.21.15.152:0/1487318490 shutdown_connections 2024-09-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.599+0000 7f3532b9e640 1 --2- 172.21.15.152:0/1487318490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f352c101270 0x7f352c10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.599+0000 7f3532b9e640 1 --2- 172.21.15.152:0/1487318490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f352c1008d0 0x7f352c100d30 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.599+0000 7f3532b9e640 1 --2- 172.21.15.152:0/1487318490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f352c1068d0 0x7f352c106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.599+0000 7f3532b9e640 1 -- 172.21.15.152:0/1487318490 >> 172.21.15.152:0/1487318490 conn(0x7f352c0fc630 msgr2=0x7f352c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.599+0000 7f3532b9e640 1 -- 172.21.15.152:0/1487318490 shutdown_connections 2024-09-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.599+0000 7f3532b9e640 1 -- 172.21.15.152:0/1487318490 wait complete. 2024-09-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.599+0000 7f3532b9e640 1 Processor -- start 2024-09-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.599+0000 7f3532b9e640 1 -- start start 2024-09-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f3532b9e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f352c1008d0 0x7f352c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f3532b9e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f352c101270 0x7f352c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f3532b9e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f352c1068d0 0x7f352c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f3532b9e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f352c110cd0 con 0x7f352c1068d0 2024-09-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f3532b9e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f352c110b50 con 0x7f352c1008d0 2024-09-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f3532b9e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f352c110e50 con 0x7f352c101270 2024-09-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f353239d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f352c1068d0 0x7f352c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f353239d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f352c1068d0 0x7f352c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59540/0 (socket says 172.21.15.152:59540) 2024-09-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f353239d640 1 -- 172.21.15.152:0/3336852550 learned_addr learned my addr 172.21.15.152:0/3336852550 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f3531b9c640 1 --2- 172.21.15.152:0/3336852550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f352c1008d0 0x7f352c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f353139b640 1 --2- 172.21.15.152:0/3336852550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f352c101270 0x7f352c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f353239d640 1 -- 172.21.15.152:0/3336852550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f352c101270 msgr2=0x7f352c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f353239d640 1 --2- 172.21.15.152:0/3336852550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f352c101270 0x7f352c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f353239d640 1 -- 172.21.15.152:0/3336852550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f352c1008d0 msgr2=0x7f352c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f353239d640 1 --2- 172.21.15.152:0/3336852550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f352c1008d0 0x7f352c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f353239d640 1 -- 172.21.15.152:0/3336852550 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352c19d310 con 0x7f352c1068d0 2024-09-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f353239d640 1 --2- 172.21.15.152:0/3336852550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f352c1068d0 0x7f352c19cc30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f352400d750 tx=0x7f352400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f3522ffd640 1 -- 172.21.15.152:0/3336852550 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524017070 con 0x7f352c1068d0 2024-09-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f3532b9e640 1 -- 172.21.15.152:0/3336852550 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352c19d600 con 0x7f352c1068d0 2024-09-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f3522ffd640 1 -- 172.21.15.152:0/3336852550 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3524005500 con 0x7f352c1068d0 2024-09-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.600+0000 7f3522ffd640 1 -- 172.21.15.152:0/3336852550 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524004d90 con 0x7f352c1068d0 2024-09-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.601+0000 7f3532b9e640 1 -- 172.21.15.152:0/3336852550 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352c1a4ee0 con 0x7f352c1068d0 2024-09-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.603+0000 7f3522ffd640 1 -- 172.21.15.152:0/3336852550 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3524004f30 con 0x7f352c1068d0 2024-09-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.603+0000 7f35361d9740 1 Processor -- start 2024-09-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.603+0000 7f35361d9740 1 -- start start 2024-09-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.603+0000 7f3522ffd640 1 --2- 172.21.15.152:0/3336852550 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f35180a2cf0 0x7f35180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.603+0000 7f35361d9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c61270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.603+0000 7f35361d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c617b0 0x562be5c61bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.603+0000 7f35361d9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c70960 0x562be5c72d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f35361d9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x562be5bc72f0 con 0x562be5c62d20 2024-09-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f35361d9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562be5b6ecf0 con 0x562be5c70960 2024-09-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f35361d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562be5bc32b0 con 0x562be5c617b0 2024-09-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f353139b640 1 --2- 172.21.15.152:0/3336852550 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f35180a2cf0 0x7f35180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f3531b9c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c61270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f3531b9c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c61270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59546/0 (socket says 172.21.15.152:59546) 2024-09-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f3531b9c640 1 -- 172.21.15.152:0/463745158 learned_addr learned my addr 172.21.15.152:0/463745158 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f3522ffd640 1 -- 172.21.15.152:0/3336852550 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f35240cef50 con 0x7f352c1068d0 2024-09-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f353139b640 1 --2- 172.21.15.152:0/463745158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c617b0 0x562be5c61bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f353239d640 1 --2- 172.21.15.152:0/463745158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c70960 0x562be5c72d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f3531b9c640 1 -- 172.21.15.152:0/463745158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c617b0 msgr2=0x562be5c61bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f3531b9c640 1 --2- 172.21.15.152:0/463745158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c617b0 0x562be5c61bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f3531b9c640 1 -- 172.21.15.152:0/463745158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c70960 msgr2=0x562be5c72d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f3531b9c640 1 --2- 172.21.15.152:0/463745158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c70960 0x562be5c72d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f3531b9c640 1 -- 172.21.15.152:0/463745158 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562be5b533a0 con 0x562be5c62d20 2024-09-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f353139b640 1 --2- 172.21.15.152:0/3336852550 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f35180a2cf0 0x7f35180a51b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f351c000c00 tx=0x7f351c032f60 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f3531b9c640 1 --2- 172.21.15.152:0/463745158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c61270 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3528009fc0 tx=0x7f3528010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39ee902bd2fafe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f3517fff640 1 -- 172.21.15.152:0/463745158 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528003b70 con 0x562be5c62d20 2024-09-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f3517fff640 1 -- 172.21.15.152:0/463745158 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3528003d10 con 0x562be5c62d20 2024-09-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.604+0000 7f3517fff640 1 -- 172.21.15.152:0/463745158 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528011a80 con 0x562be5c62d20 2024-09-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.605+0000 7f35361d9740 1 -- 172.21.15.152:0/463745158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 msgr2=0x562be5c61270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.605+0000 7f35361d9740 1 --2- 172.21.15.152:0/463745158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c61270 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f3528009fc0 tx=0x7f3528010860 comp rx=0 tx=0).stop 2024-09-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.605+0000 7f35361d9740 1 -- 172.21.15.152:0/463745158 shutdown_connections 2024-09-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.605+0000 7f35361d9740 1 --2- 172.21.15.152:0/463745158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c70960 0x562be5c72d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.605+0000 7f35361d9740 1 --2- 172.21.15.152:0/463745158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c617b0 0x562be5c61bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.605+0000 7f35361d9740 1 --2- 172.21.15.152:0/463745158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c61270 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.605+0000 7f35361d9740 1 -- 172.21.15.152:0/463745158 >> 172.21.15.152:0/463745158 conn(0x562be5c2a310 msgr2=0x562be5c5de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.605+0000 7f35361d9740 1 -- 172.21.15.152:0/463745158 shutdown_connections 2024-09-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.605+0000 7f35361d9740 1 -- 172.21.15.152:0/463745158 wait complete. 2024-09-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f35361d9740 1 -- start start 2024-09-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f35361d9740 1 Processor -- start 2024-09-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f35361d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c617b0 0x562be5c70270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f35361d9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c6d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f35361d9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c70960 0x562be5c6dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f35361d9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x562be5c73660 con 0x562be5c62d20 2024-09-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f35361d9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x562be5b82060 con 0x562be5c70960 2024-09-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f35361d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x562be5c737e0 con 0x562be5c617b0 2024-09-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f35361d9740 10 client.0 _ll_register_callbacks cb 0x562be58d58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f353139b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c6d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f353139b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c6d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59548/0 (socket says 172.21.15.152:59548) 2024-09-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f353139b640 1 -- 172.21.15.152:0/4184376364 learned_addr learned my addr 172.21.15.152:0/4184376364 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f3531b9c640 1 --2- 172.21.15.152:0/4184376364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c617b0 0x562be5c70270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f353239d640 1 --2- 172.21.15.152:0/4184376364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c70960 0x562be5c6dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f353139b640 1 -- 172.21.15.152:0/4184376364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c617b0 msgr2=0x562be5c70270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f353139b640 1 --2- 172.21.15.152:0/4184376364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c617b0 0x562be5c70270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f353139b640 1 -- 172.21.15.152:0/4184376364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c70960 msgr2=0x562be5c6dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.606+0000 7f353139b640 1 --2- 172.21.15.152:0/4184376364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c70960 0x562be5c6dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f353139b640 1 -- 172.21.15.152:0/4184376364 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562be59383c0 con 0x562be5c62d20 2024-09-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f353139b640 1 --2- 172.21.15.152:0/4184376364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c6d570 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f351c009930 tx=0x7f351c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54518dad1b13d818 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f35157fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f35157fa640 1 -- 172.21.15.152:0/4184376364 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c04d070 con 0x562be5c62d20 2024-09-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f35157fa640 1 -- 172.21.15.152:0/4184376364 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f351c005510 con 0x562be5c62d20 2024-09-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f35361d9740 1 -- 172.21.15.152:0/4184376364 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3528003d10 con 0x562be5c62d20 2024-09-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f35157fa640 1 -- 172.21.15.152:0/4184376364 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f351c059050 con 0x562be5c62d20 2024-09-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f35361d9740 10 client.18138 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f35361d9740 1 -- 172.21.15.152:0/4184376364 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562be5c7f070 con 0x562be5c62d20 2024-09-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f35157fa640 1 -- 172.21.15.152:0/4184376364 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3528003d10 con 0x562be5c62d20 2024-09-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f35157fa640 1 -- 172.21.15.152:0/4184376364 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f351c003960 con 0x562be5c62d20 2024-09-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f35361d9740 10 client.18138 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f35361d9740 10 client.18138 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f35361d9740 20 client.18138 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f35361d9740 1 --2- 172.21.15.152:0/4184376364 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x562be5c80480 0x562be5c82900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f35361d9740 4 client.18138 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.607+0000 7f35361d9740 1 -- 172.21.15.152:0/4184376364 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562be5bd4860 con 0x562be5c80480 2024-09-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.608+0000 7f3531b9c640 1 --2- 172.21.15.152:0/4184376364 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x562be5c80480 0x562be5c82900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.608+0000 7f35157fa640 10 client.18138 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.608+0000 7f3531b9c640 1 --2- 172.21.15.152:0/4184376364 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x562be5c80480 0x562be5c82900 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c0221a727d675085 server_cookie=ef9b25ac2723afa1 in_seq=0 out_seq=0 2024-09-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.608+0000 7f35157fa640 1 -- 172.21.15.152:0/4184376364 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562be5bd4860 con 0x562be5c80480 2024-09-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.608+0000 7f35157fa640 10 client.18138 handle_command_reply: tid=0 2024-09-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.608+0000 7f35361d9740 1 client.18138 shutdown 2024-09-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.609+0000 7f35361d9740 1 -- 172.21.15.152:0/4184376364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 msgr2=0x562be5c6d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.609+0000 7f35361d9740 1 --2- 172.21.15.152:0/4184376364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c6d570 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f351c009930 tx=0x7f351c02a420 comp rx=0 tx=0).stop 2024-09-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.609+0000 7f35361d9740 1 -- 172.21.15.152:0/4184376364 shutdown_connections 2024-09-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.609+0000 7f35361d9740 1 --2- 172.21.15.152:0/4184376364 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x562be5c80480 0x562be5c82900 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.609+0000 7f35361d9740 1 --2- 172.21.15.152:0/4184376364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c70960 0x562be5c6dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.609+0000 7f35361d9740 1 --2- 172.21.15.152:0/4184376364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c6d570 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.609+0000 7f35157fa640 0 client.18138 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.609+0000 7f35361d9740 1 --2- 172.21.15.152:0/4184376364 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c617b0 0x562be5c70270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.609+0000 7f35361d9740 1 -- 172.21.15.152:0/4184376364 >> 172.21.15.152:0/4184376364 conn(0x562be5c2a310 msgr2=0x562be5c63b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.609+0000 7f35361d9740 1 -- 172.21.15.152:0/4184376364 shutdown_connections 2024-09-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.609+0000 7f35361d9740 1 -- 172.21.15.152:0/4184376364 wait complete. 2024-09-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.609+0000 7f35361d9740 20 client.18138 trim_cache size 0 max 16384 2024-09-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f35361d9740 1 Processor -- start 2024-09-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f35361d9740 1 -- start start 2024-09-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f35361d9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c617b0 0x562be5c7c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f35361d9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c6ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f35361d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c70960 0x562be5c88220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f35361d9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f352c1ab590 con 0x562be5c62d20 2024-09-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f35361d9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f352c1ad0d0 con 0x562be5c617b0 2024-09-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f35361d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f352c1abc60 con 0x562be5c70960 2024-09-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f353139b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c6ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f353139b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c6ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59556/0 (socket says 172.21.15.152:59556) 2024-09-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f353139b640 1 -- 172.21.15.152:0/2993255248 learned_addr learned my addr 172.21.15.152:0/2993255248 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f353239d640 1 --2- 172.21.15.152:0/2993255248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c70960 0x562be5c88220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f3531b9c640 1 --2- 172.21.15.152:0/2993255248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c617b0 0x562be5c7c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f353139b640 1 -- 172.21.15.152:0/2993255248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c70960 msgr2=0x562be5c88220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f353139b640 1 --2- 172.21.15.152:0/2993255248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c70960 0x562be5c88220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f353139b640 1 -- 172.21.15.152:0/2993255248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c617b0 msgr2=0x562be5c7c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f353139b640 1 --2- 172.21.15.152:0/2993255248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c617b0 0x562be5c7c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f353139b640 1 -- 172.21.15.152:0/2993255248 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562be5c64300 con 0x562be5c62d20 2024-09-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f3531b9c640 1 --2- 172.21.15.152:0/2993255248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c617b0 0x562be5c7c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f353139b640 1 --2- 172.21.15.152:0/2993255248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c6ce30 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f351c009900 tx=0x7f351c0316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aee11a29bd9ec95 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.622+0000 7f35177fe640 1 -- 172.21.15.152:0/2993255248 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c062070 con 0x562be5c62d20 2024-09-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.623+0000 7f35177fe640 1 -- 172.21.15.152:0/2993255248 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f351c003ad0 con 0x562be5c62d20 2024-09-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.623+0000 7f35177fe640 1 -- 172.21.15.152:0/2993255248 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c04d070 con 0x562be5c62d20 2024-09-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.623+0000 7f35361d9740 1 -- 172.21.15.152:0/2993255248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 msgr2=0x562be5c6ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.623+0000 7f35361d9740 1 --2- 172.21.15.152:0/2993255248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c6ce30 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f351c009900 tx=0x7f351c0316d0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.623+0000 7f35361d9740 1 -- 172.21.15.152:0/2993255248 shutdown_connections 2024-09-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.623+0000 7f35361d9740 1 --2- 172.21.15.152:0/2993255248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c70960 0x562be5c88220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.623+0000 7f35361d9740 1 --2- 172.21.15.152:0/2993255248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c62d20 0x562be5c6ce30 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.623+0000 7f35361d9740 1 --2- 172.21.15.152:0/2993255248 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c617b0 0x562be5c7c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.623+0000 7f35361d9740 1 -- 172.21.15.152:0/2993255248 >> 172.21.15.152:0/2993255248 conn(0x562be5c2a310 msgr2=0x562be5c6be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.623+0000 7f35361d9740 1 -- 172.21.15.152:0/2993255248 shutdown_connections 2024-09-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.623+0000 7f35361d9740 1 -- 172.21.15.152:0/2993255248 wait complete. 2024-09-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.623+0000 7f35361d9740 1 -- start start 2024-09-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f35361d9740 1 Processor -- start 2024-09-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f35361d9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c617b0 0x562be5c79230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f35361d9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c62d20 0x562be5c74560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f35361d9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c70960 0x562be5c74ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f35361d9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f352c1aca00 con 0x562be5c617b0 2024-09-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f35361d9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f352c1ac330 con 0x562be5c70960 2024-09-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f35361d9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f352c1a9590 con 0x562be5c62d20 2024-09-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f35361d9740 10 client.0 _ll_register_callbacks cb 0x7f35280093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f3531b9c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c617b0 0x562be5c79230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f3531b9c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c617b0 0x562be5c79230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59558/0 (socket says 172.21.15.152:59558) 2024-09-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f3531b9c640 1 -- 172.21.15.152:0/3314387938 learned_addr learned my addr 172.21.15.152:0/3314387938 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f353239d640 1 --2- 172.21.15.152:0/3314387938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c70960 0x562be5c74ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f353139b640 1 --2- 172.21.15.152:0/3314387938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c62d20 0x562be5c74560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f3531b9c640 1 -- 172.21.15.152:0/3314387938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c62d20 msgr2=0x562be5c74560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f3531b9c640 1 --2- 172.21.15.152:0/3314387938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c62d20 0x562be5c74560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f3531b9c640 1 -- 172.21.15.152:0/3314387938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c70960 msgr2=0x562be5c74ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f3531b9c640 1 --2- 172.21.15.152:0/3314387938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c70960 0x562be5c74ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f3531b9c640 1 -- 172.21.15.152:0/3314387938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562be5c752f0 con 0x562be5c617b0 2024-09-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f353139b640 1 --2- 172.21.15.152:0/3314387938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c62d20 0x562be5c74560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f353239d640 1 --2- 172.21.15.152:0/3314387938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c70960 0x562be5c74ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f3531b9c640 1 --2- 172.21.15.152:0/3314387938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c617b0 0x562be5c79230 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f3528006010 tx=0x7f3528011500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3209c2c0757b83c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.624+0000 7f35157fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.625+0000 7f35157fa640 1 -- 172.21.15.152:0/3314387938 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528004da0 con 0x562be5c617b0 2024-09-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.625+0000 7f35157fa640 1 -- 172.21.15.152:0/3314387938 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f3528011770 con 0x562be5c617b0 2024-09-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.625+0000 7f35361d9740 1 -- 172.21.15.152:0/3314387938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f351c003ad0 con 0x562be5c617b0 2024-09-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.625+0000 7f35157fa640 1 -- 172.21.15.152:0/3314387938 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3528038050 con 0x562be5c617b0 2024-09-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.625+0000 7f35361d9740 10 client.18144 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.625+0000 7f35361d9740 1 -- 172.21.15.152:0/3314387938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562be5ceb1f0 con 0x562be5c617b0 2024-09-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.625+0000 7f35157fa640 1 -- 172.21.15.152:0/3314387938 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f351c003ad0 con 0x562be5c617b0 2024-09-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.625+0000 7f35157fa640 1 -- 172.21.15.152:0/3314387938 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f35280055e0 con 0x562be5c617b0 2024-09-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.625+0000 7f35361d9740 10 client.18144 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.625+0000 7f35361d9740 10 client.18144 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.625+0000 7f35361d9740 20 client.18144 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.625+0000 7f35361d9740 1 --2- 172.21.15.152:0/3314387938 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x562be5ceb640 0x562be5ceda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.625+0000 7f35361d9740 4 client.18144 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.625+0000 7f35361d9740 1 -- 172.21.15.152:0/3314387938 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562be5bced70 con 0x562be5ceb640 2024-09-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.625+0000 7f353139b640 1 --2- 172.21.15.152:0/3314387938 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x562be5ceb640 0x562be5ceda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f35157fa640 10 client.18144 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f353139b640 1 --2- 172.21.15.152:0/3314387938 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x562be5ceb640 0x562be5ceda30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=abf32f37257a190 server_cookie=cec4e5140626d85d in_seq=0 out_seq=0 2024-09-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f35157fa640 1 -- 172.21.15.152:0/3314387938 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562be5bced70 con 0x562be5ceb640 2024-09-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f35157fa640 10 client.18144 handle_command_reply: tid=0 2024-09-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f35361d9740 1 client.18144 shutdown 2024-09-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f35361d9740 1 -- 172.21.15.152:0/3314387938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c617b0 msgr2=0x562be5c79230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f35361d9740 1 --2- 172.21.15.152:0/3314387938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c617b0 0x562be5c79230 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f3528006010 tx=0x7f3528011500 comp rx=0 tx=0).stop 2024-09-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f35361d9740 1 -- 172.21.15.152:0/3314387938 shutdown_connections 2024-09-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f35361d9740 1 --2- 172.21.15.152:0/3314387938 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x562be5ceb640 0x562be5ceda30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f35361d9740 1 --2- 172.21.15.152:0/3314387938 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x562be5c70960 0x562be5c74ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f35361d9740 1 --2- 172.21.15.152:0/3314387938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562be5c62d20 0x562be5c74560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f35361d9740 1 --2- 172.21.15.152:0/3314387938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x562be5c617b0 0x562be5c79230 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f35157fa640 0 client.18144 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f35361d9740 1 -- 172.21.15.152:0/3314387938 >> 172.21.15.152:0/3314387938 conn(0x562be5c2a310 msgr2=0x562be5c6edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f35361d9740 1 -- 172.21.15.152:0/3314387938 shutdown_connections 2024-09-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f35361d9740 1 -- 172.21.15.152:0/3314387938 wait complete. 2024-09-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.626+0000 7f35361d9740 20 client.18144 trim_cache size 0 max 16384 2024-09-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.627+0000 7f3532b9e640 1 -- 172.21.15.152:0/3336852550 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f35180a2cf0 msgr2=0x7f35180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.627+0000 7f3532b9e640 1 --2- 172.21.15.152:0/3336852550 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f35180a2cf0 0x7f35180a51b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f351c000c00 tx=0x7f351c032f60 comp rx=0 tx=0).stop 2024-09-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.627+0000 7f3532b9e640 1 -- 172.21.15.152:0/3336852550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f352c1068d0 msgr2=0x7f352c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.627+0000 7f3532b9e640 1 --2- 172.21.15.152:0/3336852550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f352c1068d0 0x7f352c19cc30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f352400d750 tx=0x7f352400dc10 comp rx=0 tx=0).stop 2024-09-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.627+0000 7f3532b9e640 1 -- 172.21.15.152:0/3336852550 shutdown_connections 2024-09-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.627+0000 7f3532b9e640 1 --2- 172.21.15.152:0/3336852550 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f35180a2cf0 0x7f35180a51b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.627+0000 7f3532b9e640 1 --2- 172.21.15.152:0/3336852550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f352c1068d0 0x7f352c19cc30 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.627+0000 7f3532b9e640 1 --2- 172.21.15.152:0/3336852550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f352c101270 0x7f352c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.627+0000 7f3532b9e640 1 --2- 172.21.15.152:0/3336852550 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f352c1008d0 0x7f352c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.627+0000 7f3532b9e640 1 -- 172.21.15.152:0/3336852550 >> 172.21.15.152:0/3336852550 conn(0x7f352c0fc630 msgr2=0x7f352c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.627+0000 7f3532b9e640 1 -- 172.21.15.152:0/3336852550 shutdown_connections 2024-09-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:16.627+0000 7f3532b9e640 1 -- 172.21.15.152:0/3336852550 wait complete. 2024-09-11T23:02:16.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:16 smithi152 ceph-mon[31181]: pgmap v289: 97 pgs: 97 active+clean; 4.0 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:02:17.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:16 smithi191 ceph-mon[35211]: pgmap v289: 97 pgs: 97 active+clean; 4.0 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:02:17.049 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:16 smithi204 ceph-mon[35454]: pgmap v289: 97 pgs: 97 active+clean; 4.0 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:02:17.444 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:02:17.689 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:18.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.259+0000 7fca0db76640 1 -- 172.21.15.152:0/2608862951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca08102790 msgr2=0x7fca081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:18.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.259+0000 7fca0db76640 1 --2- 172.21.15.152:0/2608862951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca08102790 0x7fca081067e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009900 tx=0x7fc9f002a410 comp rx=0 tx=0).stop 2024-09-11T23:02:18.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.259+0000 7fca0db76640 1 -- 172.21.15.152:0/2608862951 shutdown_connections 2024-09-11T23:02:18.277 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.259+0000 7fca0db76640 1 --2- 172.21.15.152:0/2608862951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca08106f10 0x7fca0810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.259+0000 7fca0db76640 1 --2- 172.21.15.152:0/2608862951 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca08102790 0x7fca081067e0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.259+0000 7fca0db76640 1 --2- 172.21.15.152:0/2608862951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca08101de0 0x7fca081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.259+0000 7fca0db76640 1 -- 172.21.15.152:0/2608862951 >> 172.21.15.152:0/2608862951 conn(0x7fca080fd8c0 msgr2=0x7fca080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:18.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.259+0000 7fca0db76640 1 -- 172.21.15.152:0/2608862951 shutdown_connections 2024-09-11T23:02:18.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.259+0000 7fca0db76640 1 -- 172.21.15.152:0/2608862951 wait complete. 2024-09-11T23:02:18.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.260+0000 7fca0db76640 1 Processor -- start 2024-09-11T23:02:18.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.260+0000 7fca0db76640 1 -- start start 2024-09-11T23:02:18.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.260+0000 7fca0db76640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca08101de0 0x7fca08198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.260+0000 7fca0db76640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca08102790 0x7fca08198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.260+0000 7fca0db76640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca08106f10 0x7fca0819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.260+0000 7fca0db76640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fca08111f70 con 0x7fca08106f10 2024-09-11T23:02:18.278 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.260+0000 7fca0db76640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fca08111df0 con 0x7fca08102790 2024-09-11T23:02:18.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.260+0000 7fca0db76640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fca081120f0 con 0x7fca08101de0 2024-09-11T23:02:18.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.260+0000 7fca07fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca08106f10 0x7fca0819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.260+0000 7fca07fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca08106f10 0x7fca0819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59582/0 (socket says 172.21.15.152:59582) 2024-09-11T23:02:18.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.260+0000 7fca07fff640 1 -- 172.21.15.152:0/311069185 learned_addr learned my addr 172.21.15.152:0/311069185 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:18.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.260+0000 7fca077fe640 1 --2- 172.21.15.152:0/311069185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca08101de0 0x7fca08198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.260+0000 7fc9fffff640 1 --2- 172.21.15.152:0/311069185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca08102790 0x7fca08198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.261+0000 7fca07fff640 1 -- 172.21.15.152:0/311069185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca08101de0 msgr2=0x7fca08198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:18.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.261+0000 7fca07fff640 1 --2- 172.21.15.152:0/311069185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca08101de0 0x7fca08198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.261+0000 7fca07fff640 1 -- 172.21.15.152:0/311069185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca08102790 msgr2=0x7fca08198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:18.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.261+0000 7fca07fff640 1 --2- 172.21.15.152:0/311069185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca08102790 0x7fca08198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.261+0000 7fca07fff640 1 -- 172.21.15.152:0/311069185 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0819d400 con 0x7fca08106f10 2024-09-11T23:02:18.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.261+0000 7fc9fffff640 1 --2- 172.21.15.152:0/311069185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca08102790 0x7fca08198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:18.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.261+0000 7fca07fff640 1 --2- 172.21.15.152:0/311069185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca08106f10 0x7fca0819cd20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fca0000cad0 tx=0x7fca0000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:18.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.261+0000 7fca057fa640 1 -- 172.21.15.152:0/311069185 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0000ad60 con 0x7fca08106f10 2024-09-11T23:02:18.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.261+0000 7fca0db76640 1 -- 172.21.15.152:0/311069185 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0819d6f0 con 0x7fca08106f10 2024-09-11T23:02:18.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.261+0000 7fca057fa640 1 -- 172.21.15.152:0/311069185 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca00005020 con 0x7fca08106f10 2024-09-11T23:02:18.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.261+0000 7fca057fa640 1 -- 172.21.15.152:0/311069185 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0000f6b0 con 0x7fca08106f10 2024-09-11T23:02:18.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.261+0000 7fca0db76640 1 -- 172.21.15.152:0/311069185 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca081a4fd0 con 0x7fca08106f10 2024-09-11T23:02:18.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.263+0000 7fca057fa640 1 -- 172.21.15.152:0/311069185 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fca00005560 con 0x7fca08106f10 2024-09-11T23:02:18.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.263+0000 7fca0ff4f740 1 Processor -- start 2024-09-11T23:02:18.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca0ff4f740 1 -- start start 2024-09-11T23:02:18.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca0ff4f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa8db40 0x558b1aa8c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca0ff4f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8c5d0 0x558b1aa8ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca0ff4f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa97370 0x558b1aa99760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.280 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca0ff4f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558b1a9f22f0 con 0x558b1aa8c5d0 2024-09-11T23:02:18.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca0ff4f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558b1a999cf0 con 0x558b1aa97370 2024-09-11T23:02:18.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca0ff4f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558b1a9ee2b0 con 0x558b1aa8db40 2024-09-11T23:02:18.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca057fa640 1 --2- 172.21.15.152:0/311069185 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9f40a2cf0 0x7fc9f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca077fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8c5d0 0x558b1aa8ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca077fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8c5d0 0x558b1aa8ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59588/0 (socket says 172.21.15.152:59588) 2024-09-11T23:02:18.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca077fe640 1 -- 172.21.15.152:0/4289012762 learned_addr learned my addr 172.21.15.152:0/4289012762 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:18.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fc9fffff640 1 --2- 172.21.15.152:0/4289012762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa8db40 0x558b1aa8c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca07fff640 1 --2- 172.21.15.152:0/311069185 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9f40a2cf0 0x7fc9f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fc9fffff640 1 --2- 172.21.15.152:0/4289012762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa97370 0x558b1aa99760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca077fe640 1 -- 172.21.15.152:0/4289012762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa8db40 msgr2=0x558b1aa8c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:18.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca057fa640 1 -- 172.21.15.152:0/311069185 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7fca000cf4c0 con 0x7fca08106f10 2024-09-11T23:02:18.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca077fe640 1 --2- 172.21.15.152:0/4289012762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa8db40 0x558b1aa8c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca077fe640 1 -- 172.21.15.152:0/4289012762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa97370 msgr2=0x558b1aa99760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:18.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca077fe640 1 --2- 172.21.15.152:0/4289012762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa97370 0x558b1aa99760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca077fe640 1 -- 172.21.15.152:0/4289012762 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b1a97e3a0 con 0x558b1aa8c5d0 2024-09-11T23:02:18.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fc9fffff640 1 --2- 172.21.15.152:0/4289012762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa97370 0x558b1aa99760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:18.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.264+0000 7fca077fe640 1 --2- 172.21.15.152:0/4289012762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8c5d0 0x558b1aa8ca10 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800ad00 tx=0x7fc9f800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce078bb76ab9a71 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:18.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.265+0000 7fca07fff640 1 --2- 172.21.15.152:0/311069185 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9f40a2cf0 0x7fc9f40a51b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca0000a620 tx=0x7fca00000f40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:18.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.265+0000 7fc9fdffb640 1 -- 172.21.15.152:0/4289012762 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f801d070 con 0x558b1aa8c5d0 2024-09-11T23:02:18.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.265+0000 7fc9fdffb640 1 -- 172.21.15.152:0/4289012762 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9f8018b20 con 0x558b1aa8c5d0 2024-09-11T23:02:18.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.265+0000 7fc9fdffb640 1 -- 172.21.15.152:0/4289012762 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8010bc0 con 0x558b1aa8c5d0 2024-09-11T23:02:18.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.265+0000 7fca0ff4f740 1 -- 172.21.15.152:0/4289012762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8c5d0 msgr2=0x558b1aa8ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:18.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.265+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/4289012762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8c5d0 0x558b1aa8ca10 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800ad00 tx=0x7fc9f800f790 comp rx=0 tx=0).stop 2024-09-11T23:02:18.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.265+0000 7fca0ff4f740 1 -- 172.21.15.152:0/4289012762 shutdown_connections 2024-09-11T23:02:18.282 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.265+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/4289012762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa97370 0x558b1aa99760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.265+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/4289012762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8c5d0 0x558b1aa8ca10 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.265+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/4289012762 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa8db40 0x558b1aa8c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.265+0000 7fca0ff4f740 1 -- 172.21.15.152:0/4289012762 >> 172.21.15.152:0/4289012762 conn(0x558b1aa55310 msgr2=0x558b1aa88c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:18.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.265+0000 7fca0ff4f740 1 -- 172.21.15.152:0/4289012762 shutdown_connections 2024-09-11T23:02:18.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.265+0000 7fca0ff4f740 1 -- 172.21.15.152:0/4289012762 wait complete. 2024-09-11T23:02:18.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.266+0000 7fca0ff4f740 1 -- start start 2024-09-11T23:02:18.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.266+0000 7fca0ff4f740 1 Processor -- start 2024-09-11T23:02:18.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.266+0000 7fca0ff4f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa8c5d0 0x558b1aa96d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.266+0000 7fca0ff4f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8db40 0x558b1aa94000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.266+0000 7fca0ff4f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa97370 0x558b1aa945d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.266+0000 7fca0ff4f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x558b1aa9a070 con 0x558b1aa8db40 2024-09-11T23:02:18.283 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.266+0000 7fca0ff4f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x558b1a9ad060 con 0x558b1aa8c5d0 2024-09-11T23:02:18.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.266+0000 7fca0ff4f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x558b1aa9a1f0 con 0x558b1aa97370 2024-09-11T23:02:18.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.266+0000 7fca0ff4f740 10 client.0 _ll_register_callbacks cb 0x558b1a7008c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:18.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.266+0000 7fca077fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8db40 0x558b1aa94000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.266+0000 7fca077fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8db40 0x558b1aa94000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59590/0 (socket says 172.21.15.152:59590) 2024-09-11T23:02:18.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.266+0000 7fca077fe640 1 -- 172.21.15.152:0/1803744981 learned_addr learned my addr 172.21.15.152:0/1803744981 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:18.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fc9fffff640 1 --2- 172.21.15.152:0/1803744981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa8c5d0 0x558b1aa96d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fc9fffff640 1 --2- 172.21.15.152:0/1803744981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa97370 0x558b1aa945d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fca077fe640 1 -- 172.21.15.152:0/1803744981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa97370 msgr2=0x558b1aa945d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:18.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fca077fe640 1 --2- 172.21.15.152:0/1803744981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa97370 0x558b1aa945d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fca077fe640 1 -- 172.21.15.152:0/1803744981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa8c5d0 msgr2=0x558b1aa96d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:18.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fca077fe640 1 --2- 172.21.15.152:0/1803744981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa8c5d0 0x558b1aa96d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.284 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fca077fe640 1 -- 172.21.15.152:0/1803744981 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b1a7633c0 con 0x558b1aa8db40 2024-09-11T23:02:18.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fc9fffff640 1 --2- 172.21.15.152:0/1803744981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa8c5d0 0x558b1aa96d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:18.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fca077fe640 1 --2- 172.21.15.152:0/1803744981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8db40 0x558b1aa94000 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8010470 tx=0x7fc9f8010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ee3538fc1b5be0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:18.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fc9e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:18.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fc9e6ffd640 1 -- 172.21.15.152:0/1803744981 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f801d040 con 0x558b1aa8db40 2024-09-11T23:02:18.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fc9e6ffd640 1 -- 172.21.15.152:0/1803744981 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7fc9f8018440 con 0x558b1aa8db40 2024-09-11T23:02:18.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fca0ff4f740 1 -- 172.21.15.152:0/1803744981 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9f8018b20 con 0x558b1aa8db40 2024-09-11T23:02:18.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fc9e6ffd640 1 -- 172.21.15.152:0/1803744981 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc9f8031050 con 0x558b1aa8db40 2024-09-11T23:02:18.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fca0ff4f740 10 client.18156 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:18.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fca0ff4f740 1 -- 172.21.15.152:0/1803744981 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b1aaa5a60 con 0x558b1aa8db40 2024-09-11T23:02:18.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.267+0000 7fc9e6ffd640 1 -- 172.21.15.152:0/1803744981 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9f8018b20 con 0x558b1aa8db40 2024-09-11T23:02:18.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.268+0000 7fc9e6ffd640 1 -- 172.21.15.152:0/1803744981 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc9f8020030 con 0x558b1aa8db40 2024-09-11T23:02:18.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.268+0000 7fca0ff4f740 10 client.18156 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:18.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.268+0000 7fca0ff4f740 10 client.18156 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:02:18.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.268+0000 7fca0ff4f740 20 client.18156 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:18.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.268+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/1803744981 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x558b1aaa6e60 0x558b1aaa9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.268+0000 7fca0ff4f740 4 client.18156 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:02:18.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.268+0000 7fca0ff4f740 1 -- 172.21.15.152:0/1803744981 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b1a9ff860 con 0x558b1aaa6e60 2024-09-11T23:02:18.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.268+0000 7fc9fffff640 1 --2- 172.21.15.152:0/1803744981 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x558b1aaa6e60 0x558b1aaa9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.268+0000 7fc9e6ffd640 10 client.18156 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T23:02:18.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.268+0000 7fc9fffff640 1 --2- 172.21.15.152:0/1803744981 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x558b1aaa6e60 0x558b1aaa9250 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6b96045c5d9a4ed7 server_cookie=18191e63d1d04f47 in_seq=0 out_seq=0 2024-09-11T23:02:18.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.269+0000 7fc9e6ffd640 1 -- 172.21.15.152:0/1803744981 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b1a9ff860 con 0x558b1aaa6e60 2024-09-11T23:02:18.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.269+0000 7fc9e6ffd640 10 client.18156 handle_command_reply: tid=0 2024-09-11T23:02:18.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.269+0000 7fca0ff4f740 1 client.18156 shutdown 2024-09-11T23:02:18.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.269+0000 7fca0ff4f740 1 -- 172.21.15.152:0/1803744981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8db40 msgr2=0x558b1aa94000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:18.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.269+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/1803744981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8db40 0x558b1aa94000 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8010470 tx=0x7fc9f8010bd0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.269+0000 7fca0ff4f740 1 -- 172.21.15.152:0/1803744981 shutdown_connections 2024-09-11T23:02:18.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.269+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/1803744981 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x558b1aaa6e60 0x558b1aaa9250 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.269+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/1803744981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa97370 0x558b1aa945d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.269+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/1803744981 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8db40 0x558b1aa94000 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.269+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/1803744981 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa8c5d0 0x558b1aa96d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.269+0000 7fc9e6ffd640 0 client.18156 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T23:02:18.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.269+0000 7fca0ff4f740 1 -- 172.21.15.152:0/1803744981 >> 172.21.15.152:0/1803744981 conn(0x558b1aa55310 msgr2=0x558b1aa8e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:18.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.269+0000 7fca0ff4f740 1 -- 172.21.15.152:0/1803744981 shutdown_connections 2024-09-11T23:02:18.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.269+0000 7fca0ff4f740 1 -- 172.21.15.152:0/1803744981 wait complete. 2024-09-11T23:02:18.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.270+0000 7fca0ff4f740 20 client.18156 trim_cache size 0 max 16384 2024-09-11T23:02:18.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.283+0000 7fca0ff4f740 1 Processor -- start 2024-09-11T23:02:18.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.283+0000 7fca0ff4f740 1 -- start start 2024-09-11T23:02:18.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.283+0000 7fca0ff4f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa8c5d0 0x558b1aa8f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.283+0000 7fca0ff4f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa8db40 0x558b1aa8f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.283+0000 7fca0ff4f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa97370 0x558b1aaae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.283+0000 7fca0ff4f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fca081ad8f0 con 0x558b1aa97370 2024-09-11T23:02:18.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.283+0000 7fca0ff4f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fca081af430 con 0x558b1aa8c5d0 2024-09-11T23:02:18.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.283+0000 7fca0ff4f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fca081adfc0 con 0x558b1aa8db40 2024-09-11T23:02:18.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.283+0000 7fc9fffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa97370 0x558b1aaae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.283+0000 7fc9fffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa97370 0x558b1aaae880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59606/0 (socket says 172.21.15.152:59606) 2024-09-11T23:02:18.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.283+0000 7fc9fffff640 1 -- 172.21.15.152:0/32144495 learned_addr learned my addr 172.21.15.152:0/32144495 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:18.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.283+0000 7fc9fffff640 1 --2- 172.21.15.152:0/32144495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa8c5d0 0x558b1aa8f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.283+0000 7fca077fe640 1 --2- 172.21.15.152:0/32144495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa8db40 0x558b1aa8f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.284+0000 7fc9fffff640 1 -- 172.21.15.152:0/32144495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa8db40 msgr2=0x558b1aa8f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:18.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.284+0000 7fc9fffff640 1 --2- 172.21.15.152:0/32144495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa8db40 0x558b1aa8f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.284+0000 7fc9fffff640 1 -- 172.21.15.152:0/32144495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa8c5d0 msgr2=0x558b1aa8f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:18.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.284+0000 7fc9fffff640 1 --2- 172.21.15.152:0/32144495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa8c5d0 0x558b1aa8f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.284+0000 7fc9fffff640 1 -- 172.21.15.152:0/32144495 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b1aa89960 con 0x558b1aa97370 2024-09-11T23:02:18.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.284+0000 7fc9fffff640 1 --2- 172.21.15.152:0/32144495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa97370 0x558b1aaae880 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0005940 tx=0x7fc9f002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c52c74e6bed23ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:18.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.284+0000 7fc9fd7fa640 1 -- 172.21.15.152:0/32144495 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0035040 con 0x558b1aa97370 2024-09-11T23:02:18.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.284+0000 7fc9fd7fa640 1 -- 172.21.15.152:0/32144495 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9f0031dc0 con 0x558b1aa97370 2024-09-11T23:02:18.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.284+0000 7fc9fd7fa640 1 -- 172.21.15.152:0/32144495 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0003e60 con 0x558b1aa97370 2024-09-11T23:02:18.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.285+0000 7fca0ff4f740 1 -- 172.21.15.152:0/32144495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa97370 msgr2=0x558b1aaae880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:18.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.285+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/32144495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa97370 0x558b1aaae880 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0005940 tx=0x7fc9f002a410 comp rx=0 tx=0).stop 2024-09-11T23:02:18.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.285+0000 7fca0ff4f740 1 -- 172.21.15.152:0/32144495 shutdown_connections 2024-09-11T23:02:18.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.285+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/32144495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa97370 0x558b1aaae880 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.285+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/32144495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa8db40 0x558b1aa8f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.285+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/32144495 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa8c5d0 0x558b1aa8f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.285+0000 7fca0ff4f740 1 -- 172.21.15.152:0/32144495 >> 172.21.15.152:0/32144495 conn(0x558b1aa55310 msgr2=0x558b1aa96080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:18.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.285+0000 7fca0ff4f740 1 -- 172.21.15.152:0/32144495 shutdown_connections 2024-09-11T23:02:18.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.285+0000 7fca0ff4f740 1 -- 172.21.15.152:0/32144495 wait complete. 2024-09-11T23:02:18.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.285+0000 7fca0ff4f740 1 -- start start 2024-09-11T23:02:18.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.285+0000 7fca0ff4f740 1 Processor -- start 2024-09-11T23:02:18.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fca0ff4f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8c5d0 0x558b1aa90f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fca0ff4f740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa8db40 0x558b1aa91440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fca0ff4f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa97370 0x558b1aa91980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fca0ff4f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fca081aed60 con 0x558b1aa8c5d0 2024-09-11T23:02:18.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fca0ff4f740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fca081ae690 con 0x558b1aa8db40 2024-09-11T23:02:18.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fca0ff4f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fca081ab8f0 con 0x558b1aa97370 2024-09-11T23:02:18.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fca0ff4f740 10 client.0 _ll_register_callbacks cb 0x7fc9f801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:18.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fc9fffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8c5d0 0x558b1aa90f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fc9fffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8c5d0 0x558b1aa90f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59622/0 (socket says 172.21.15.152:59622) 2024-09-11T23:02:18.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fc9fffff640 1 -- 172.21.15.152:0/4209577567 learned_addr learned my addr 172.21.15.152:0/4209577567 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:18.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fca077fe640 1 --2- 172.21.15.152:0/4209577567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa8db40 0x558b1aa91440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fc9fffff640 1 --2- 172.21.15.152:0/4209577567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa97370 0x558b1aa91980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fc9fffff640 1 -- 172.21.15.152:0/4209577567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa97370 msgr2=0x558b1aa91980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:18.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fc9fffff640 1 --2- 172.21.15.152:0/4209577567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa97370 0x558b1aa91980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fc9fffff640 1 -- 172.21.15.152:0/4209577567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa8db40 msgr2=0x558b1aa91440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:18.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fc9fffff640 1 --2- 172.21.15.152:0/4209577567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa8db40 0x558b1aa91440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fc9fffff640 1 -- 172.21.15.152:0/4209577567 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b1ab14c60 con 0x558b1aa8c5d0 2024-09-11T23:02:18.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fc9fffff640 1 --2- 172.21.15.152:0/4209577567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8c5d0 0x558b1aa90f00 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0004330 tx=0x7fc9f0005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10d2b5ccb029738 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:18.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fc9e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:18.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fc9e6ffd640 1 -- 172.21.15.152:0/4209577567 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0035070 con 0x558b1aa8c5d0 2024-09-11T23:02:18.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fc9e6ffd640 1 -- 172.21.15.152:0/4209577567 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7fc9f00037f0 con 0x558b1aa8c5d0 2024-09-11T23:02:18.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.286+0000 7fca0ff4f740 1 -- 172.21.15.152:0/4209577567 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9f0031dc0 con 0x558b1aa8c5d0 2024-09-11T23:02:18.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.287+0000 7fc9e6ffd640 1 -- 172.21.15.152:0/4209577567 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc9f00316e0 con 0x558b1aa8c5d0 2024-09-11T23:02:18.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.287+0000 7fca0ff4f740 10 client.18162 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:18.296 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.287+0000 7fca0ff4f740 1 -- 172.21.15.152:0/4209577567 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558b1ab15e40 con 0x558b1aa8c5d0 2024-09-11T23:02:18.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.287+0000 7fc9e6ffd640 1 -- 172.21.15.152:0/4209577567 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9f0031dc0 con 0x558b1aa8c5d0 2024-09-11T23:02:18.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.287+0000 7fc9e6ffd640 1 -- 172.21.15.152:0/4209577567 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc9f8019ce0 con 0x558b1aa8c5d0 2024-09-11T23:02:18.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.287+0000 7fca0ff4f740 10 client.18162 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:18.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.287+0000 7fca0ff4f740 10 client.18162 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:02:18.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.287+0000 7fca0ff4f740 20 client.18162 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:18.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.287+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/4209577567 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x558b1ab164a0 0x558b1ab18890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:18.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.287+0000 7fca0ff4f740 4 client.18162 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:02:18.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.287+0000 7fca0ff4f740 1 -- 172.21.15.152:0/4209577567 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b1a9f9d70 con 0x558b1ab164a0 2024-09-11T23:02:18.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.287+0000 7fca077fe640 1 --2- 172.21.15.152:0/4209577567 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x558b1ab164a0 0x558b1ab18890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:18.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fc9e6ffd640 10 client.18162 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T23:02:18.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fca077fe640 1 --2- 172.21.15.152:0/4209577567 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x558b1ab164a0 0x558b1ab18890 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3c322e9214572ee5 server_cookie=49c3346db6428269 in_seq=0 out_seq=0 2024-09-11T23:02:18.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fc9e6ffd640 1 -- 172.21.15.152:0/4209577567 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b1a9f9d70 con 0x558b1ab164a0 2024-09-11T23:02:18.297 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fc9e6ffd640 10 client.18162 handle_command_reply: tid=0 2024-09-11T23:02:18.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fca0ff4f740 1 client.18162 shutdown 2024-09-11T23:02:18.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fca0ff4f740 1 -- 172.21.15.152:0/4209577567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8c5d0 msgr2=0x558b1aa90f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:18.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/4209577567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8c5d0 0x558b1aa90f00 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0004330 tx=0x7fc9f0005e30 comp rx=0 tx=0).stop 2024-09-11T23:02:18.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fca0ff4f740 1 -- 172.21.15.152:0/4209577567 shutdown_connections 2024-09-11T23:02:18.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/4209577567 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x558b1ab164a0 0x558b1ab18890 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/4209577567 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558b1aa97370 0x558b1aa91980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/4209577567 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558b1aa8db40 0x558b1aa91440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fc9e6ffd640 0 client.18162 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T23:02:18.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fca0ff4f740 1 --2- 172.21.15.152:0/4209577567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558b1aa8c5d0 0x558b1aa90f00 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fca0ff4f740 1 -- 172.21.15.152:0/4209577567 >> 172.21.15.152:0/4209577567 conn(0x558b1aa55310 msgr2=0x558b1aa87310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:18.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fca0ff4f740 1 -- 172.21.15.152:0/4209577567 shutdown_connections 2024-09-11T23:02:18.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fca0ff4f740 1 -- 172.21.15.152:0/4209577567 wait complete. 2024-09-11T23:02:18.298 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.288+0000 7fca0ff4f740 20 client.18162 trim_cache size 0 max 16384 2024-09-11T23:02:18.299 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:02:18.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.289+0000 7fca0db76640 1 -- 172.21.15.152:0/311069185 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9f40a2cf0 msgr2=0x7fc9f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:18.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.289+0000 7fca0db76640 1 --2- 172.21.15.152:0/311069185 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9f40a2cf0 0x7fc9f40a51b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca0000a620 tx=0x7fca00000f40 comp rx=0 tx=0).stop 2024-09-11T23:02:18.299 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.289+0000 7fca0db76640 1 -- 172.21.15.152:0/311069185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca08106f10 msgr2=0x7fca0819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:18.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.289+0000 7fca0db76640 1 --2- 172.21.15.152:0/311069185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca08106f10 0x7fca0819cd20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fca0000cad0 tx=0x7fca0000ced0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.289+0000 7fca0db76640 1 -- 172.21.15.152:0/311069185 shutdown_connections 2024-09-11T23:02:18.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.289+0000 7fca0db76640 1 --2- 172.21.15.152:0/311069185 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9f40a2cf0 0x7fc9f40a51b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.289+0000 7fca0db76640 1 --2- 172.21.15.152:0/311069185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca08106f10 0x7fca0819cd20 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.289+0000 7fca0db76640 1 --2- 172.21.15.152:0/311069185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca08102790 0x7fca08198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.289+0000 7fca0db76640 1 --2- 172.21.15.152:0/311069185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca08101de0 0x7fca08198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:18.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.289+0000 7fca0db76640 1 -- 172.21.15.152:0/311069185 >> 172.21.15.152:0/311069185 conn(0x7fca080fd8c0 msgr2=0x7fca080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:18.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.289+0000 7fca0db76640 1 -- 172.21.15.152:0/311069185 shutdown_connections 2024-09-11T23:02:18.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:18.289+0000 7fca0db76640 1 -- 172.21.15.152:0/311069185 wait complete. 2024-09-11T23:02:19.016 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:18 smithi152 ceph-mon[31181]: pgmap v290: 97 pgs: 97 active+clean; 4.0 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:02:19.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:18 smithi191 ceph-mon[35211]: pgmap v290: 97 pgs: 97 active+clean; 4.0 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:02:19.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:18 smithi204 ceph-mon[35454]: pgmap v290: 97 pgs: 97 active+clean; 4.0 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:02:19.070 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:02:19.320 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:19.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.842+0000 7f78d17f3640 1 -- 172.21.15.152:0/3742241570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc102d40 msgr2=0x7f78cc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:19.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.842+0000 7f78d17f3640 1 --2- 172.21.15.152:0/3742241570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc102d40 0x7f78cc103120 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009900 tx=0x7f78b402a480 comp rx=0 tx=0).stop 2024-09-11T23:02:19.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.842+0000 7f78d17f3640 1 -- 172.21.15.152:0/3742241570 shutdown_connections 2024-09-11T23:02:19.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.842+0000 7f78d17f3640 1 --2- 172.21.15.152:0/3742241570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78cc10c170 0x7f78cc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.842+0000 7f78d17f3640 1 --2- 172.21.15.152:0/3742241570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78cc103660 0x7f78cc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.842+0000 7f78d17f3640 1 --2- 172.21.15.152:0/3742241570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc102d40 0x7f78cc103120 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.842+0000 7f78d17f3640 1 -- 172.21.15.152:0/3742241570 >> 172.21.15.152:0/3742241570 conn(0x7f78cc0fc7a0 msgr2=0x7f78cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:19.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.842+0000 7f78d17f3640 1 -- 172.21.15.152:0/3742241570 shutdown_connections 2024-09-11T23:02:19.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.842+0000 7f78d17f3640 1 -- 172.21.15.152:0/3742241570 wait complete. 2024-09-11T23:02:19.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.843+0000 7f78d17f3640 1 Processor -- start 2024-09-11T23:02:19.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.843+0000 7f78d17f3640 1 -- start start 2024-09-11T23:02:19.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.843+0000 7f78d17f3640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78cc102d40 0x7f78cc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.843+0000 7f78d17f3640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78cc103660 0x7f78cc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.843+0000 7f78d17f3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc10c170 0x7f78cc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.843+0000 7f78d17f3640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc111f70 con 0x7f78cc10c170 2024-09-11T23:02:19.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.843+0000 7f78d17f3640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc111df0 con 0x7f78cc102d40 2024-09-11T23:02:19.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.843+0000 7f78d17f3640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1120f0 con 0x7f78cc103660 2024-09-11T23:02:19.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.843+0000 7f78cb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc10c170 0x7f78cc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.843+0000 7f78cb7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc10c170 0x7f78cc19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40562/0 (socket says 172.21.15.152:40562) 2024-09-11T23:02:19.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.843+0000 7f78cb7fe640 1 -- 172.21.15.152:0/304538602 learned_addr learned my addr 172.21.15.152:0/304538602 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:19.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.843+0000 7f78caffd640 1 --2- 172.21.15.152:0/304538602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78cc102d40 0x7f78cc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.843+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/304538602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78cc103660 0x7f78cc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.844+0000 7f78cb7fe640 1 -- 172.21.15.152:0/304538602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78cc103660 msgr2=0x7f78cc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:19.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.844+0000 7f78cb7fe640 1 --2- 172.21.15.152:0/304538602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78cc103660 0x7f78cc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.844+0000 7f78cb7fe640 1 -- 172.21.15.152:0/304538602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78cc102d40 msgr2=0x7f78cc1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:19.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.844+0000 7f78cb7fe640 1 --2- 172.21.15.152:0/304538602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78cc102d40 0x7f78cc1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.844+0000 7f78cb7fe640 1 -- 172.21.15.152:0/304538602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc19d380 con 0x7f78cc10c170 2024-09-11T23:02:19.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.844+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/304538602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78cc103660 0x7f78cc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:19.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.844+0000 7f78cb7fe640 1 --2- 172.21.15.152:0/304538602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc10c170 0x7f78cc19cca0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f78c400eaa0 tx=0x7f78c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:19.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.844+0000 7f78c3fff640 1 -- 172.21.15.152:0/304538602 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c400cdb0 con 0x7f78cc10c170 2024-09-11T23:02:19.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.844+0000 7f78d17f3640 1 -- 172.21.15.152:0/304538602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78cc19d670 con 0x7f78cc10c170 2024-09-11T23:02:19.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.844+0000 7f78c3fff640 1 -- 172.21.15.152:0/304538602 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78c4005020 con 0x7f78cc10c170 2024-09-11T23:02:19.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.844+0000 7f78c3fff640 1 -- 172.21.15.152:0/304538602 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c40116b0 con 0x7f78cc10c170 2024-09-11T23:02:19.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.844+0000 7f78d17f3640 1 -- 172.21.15.152:0/304538602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78cc1a4f50 con 0x7f78cc10c170 2024-09-11T23:02:19.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.846+0000 7f78c3fff640 1 -- 172.21.15.152:0/304538602 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f78c4005560 con 0x7f78cc10c170 2024-09-11T23:02:19.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.846+0000 7f78d3bcc740 1 Processor -- start 2024-09-11T23:02:19.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78d3bcc740 1 -- start start 2024-09-11T23:02:19.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78d3bcc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c5b40 0x5638815c4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78d3bcc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815c45d0 0x5638815c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78d3bcc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815cf370 0x5638815d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78d3bcc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56388152a2f0 con 0x5638815c45d0 2024-09-11T23:02:19.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78d3bcc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5638814d1cf0 con 0x5638815c5b40 2024-09-11T23:02:19.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78d3bcc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5638815262b0 con 0x5638815cf370 2024-09-11T23:02:19.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78caffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815c45d0 0x5638815c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78caffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815c45d0 0x5638815c4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40572/0 (socket says 172.21.15.152:40572) 2024-09-11T23:02:19.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78caffd640 1 -- 172.21.15.152:0/2653515985 learned_addr learned my addr 172.21.15.152:0/2653515985 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:19.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78c3fff640 1 --2- 172.21.15.152:0/304538602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f78b80a2cf0 0x7f78b80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/2653515985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815cf370 0x5638815d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/2653515985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c5b40 0x5638815c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78caffd640 1 -- 172.21.15.152:0/2653515985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815cf370 msgr2=0x5638815d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:19.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78caffd640 1 --2- 172.21.15.152:0/2653515985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815cf370 0x5638815d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78caffd640 1 -- 172.21.15.152:0/2653515985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c5b40 msgr2=0x5638815c4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:19.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78caffd640 1 --2- 172.21.15.152:0/2653515985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c5b40 0x5638815c4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78caffd640 1 -- 172.21.15.152:0/2653515985 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638814b63a0 con 0x5638815c45d0 2024-09-11T23:02:19.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.847+0000 7f78cb7fe640 1 --2- 172.21.15.152:0/304538602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f78b80a2cf0 0x7f78b80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.848+0000 7f78c3fff640 1 -- 172.21.15.152:0/304538602 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f78c40cf650 con 0x7f78cc10c170 2024-09-11T23:02:19.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.848+0000 7f78caffd640 1 --2- 172.21.15.152:0/2653515985 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815c45d0 0x5638815c4a10 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009900 tx=0x7f78b4031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4251b234f9cfecf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:19.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.848+0000 7f78c17fa640 1 -- 172.21.15.152:0/2653515985 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4040070 con 0x5638815c45d0 2024-09-11T23:02:19.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.848+0000 7f78c17fa640 1 -- 172.21.15.152:0/2653515985 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b4005020 con 0x5638815c45d0 2024-09-11T23:02:19.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.848+0000 7f78c17fa640 1 -- 172.21.15.152:0/2653515985 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4035040 con 0x5638815c45d0 2024-09-11T23:02:19.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.849+0000 7f78d3bcc740 1 -- 172.21.15.152:0/2653515985 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815c45d0 msgr2=0x5638815c4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:19.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.849+0000 7f78cb7fe640 1 --2- 172.21.15.152:0/304538602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f78b80a2cf0 0x7f78b80a51b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f78c4098530 tx=0x7f78c4066790 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:19.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.849+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/2653515985 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815c45d0 0x5638815c4a10 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009900 tx=0x7f78b4031630 comp rx=0 tx=0).stop 2024-09-11T23:02:19.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.849+0000 7f78d3bcc740 1 -- 172.21.15.152:0/2653515985 shutdown_connections 2024-09-11T23:02:19.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.849+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/2653515985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815cf370 0x5638815d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.849+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/2653515985 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815c45d0 0x5638815c4a10 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.849+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/2653515985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c5b40 0x5638815c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.849+0000 7f78d3bcc740 1 -- 172.21.15.152:0/2653515985 >> 172.21.15.152:0/2653515985 conn(0x56388158d310 msgr2=0x5638815c0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:19.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.849+0000 7f78d3bcc740 1 -- 172.21.15.152:0/2653515985 shutdown_connections 2024-09-11T23:02:19.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.849+0000 7f78d3bcc740 1 -- 172.21.15.152:0/2653515985 wait complete. 2024-09-11T23:02:19.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.849+0000 7f78d3bcc740 1 -- start start 2024-09-11T23:02:19.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.849+0000 7f78d3bcc740 1 Processor -- start 2024-09-11T23:02:19.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78d3bcc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815c45d0 0x5638815cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78d3bcc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c5b40 0x5638815cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78d3bcc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815cf370 0x5638815cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78d3bcc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5638815d2010 con 0x5638815cf370 2024-09-11T23:02:19.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78d3bcc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5638814e5060 con 0x5638815c5b40 2024-09-11T23:02:19.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78d3bcc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5638815d2190 con 0x5638815c45d0 2024-09-11T23:02:19.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78d3bcc740 10 client.0 _ll_register_callbacks cb 0x5638812388c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:19.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78ca7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815cf370 0x5638815cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78ca7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815cf370 0x5638815cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40580/0 (socket says 172.21.15.152:40580) 2024-09-11T23:02:19.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78ca7fc640 1 -- 172.21.15.152:0/991098341 learned_addr learned my addr 172.21.15.152:0/991098341 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:19.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/991098341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815c45d0 0x5638815cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78caffd640 1 --2- 172.21.15.152:0/991098341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c5b40 0x5638815cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78ca7fc640 1 -- 172.21.15.152:0/991098341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815c45d0 msgr2=0x5638815cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:19.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/991098341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815c45d0 0x5638815cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78ca7fc640 1 -- 172.21.15.152:0/991098341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c5b40 msgr2=0x5638815cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:19.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/991098341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c5b40 0x5638815cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78ca7fc640 1 -- 172.21.15.152:0/991098341 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56388129b3c0 con 0x5638815cf370 2024-09-11T23:02:19.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/991098341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815cf370 0x5638815cc4f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f78bc012dc0 tx=0x7f78bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3addc81e818433ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:19.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:19.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78b27fc640 1 -- 172.21.15.152:0/991098341 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc010e70 con 0x5638815cf370 2024-09-11T23:02:19.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78b27fc640 1 -- 172.21.15.152:0/991098341 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f78bc005020 con 0x5638815cf370 2024-09-11T23:02:19.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78d3bcc740 1 -- 172.21.15.152:0/991098341 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78b4005020 con 0x5638815cf370 2024-09-11T23:02:19.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78b27fc640 1 -- 172.21.15.152:0/991098341 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f78bc005240 con 0x5638815cf370 2024-09-11T23:02:19.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.850+0000 7f78d3bcc740 10 client.18174 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:19.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.851+0000 7f78d3bcc740 1 -- 172.21.15.152:0/991098341 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638815dda80 con 0x5638815cf370 2024-09-11T23:02:19.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.851+0000 7f78b27fc640 1 -- 172.21.15.152:0/991098341 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78b4005020 con 0x5638815cf370 2024-09-11T23:02:19.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.851+0000 7f78b27fc640 1 -- 172.21.15.152:0/991098341 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f78bc01f440 con 0x5638815cf370 2024-09-11T23:02:19.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.851+0000 7f78d3bcc740 10 client.18174 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:19.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.851+0000 7f78d3bcc740 10 client.18174 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:02:19.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.851+0000 7f78d3bcc740 20 client.18174 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:19.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.851+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/991098341 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x5638815def40 0x5638815e1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.851+0000 7f78d3bcc740 4 client.18174 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:02:19.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.851+0000 7f78d3bcc740 1 -- 172.21.15.152:0/991098341 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563881537860 con 0x5638815def40 2024-09-11T23:02:19.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.851+0000 7f78caffd640 1 --2- 172.21.15.152:0/991098341 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x5638815def40 0x5638815e1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.852+0000 7f78b27fc640 10 client.18174 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T23:02:19.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.852+0000 7f78caffd640 1 --2- 172.21.15.152:0/991098341 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x5638815def40 0x5638815e1330 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1657d1c165f376e server_cookie=3146af796a3a9b77 in_seq=0 out_seq=0 2024-09-11T23:02:19.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.852+0000 7f78b27fc640 1 -- 172.21.15.152:0/991098341 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563881537860 con 0x5638815def40 2024-09-11T23:02:19.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.852+0000 7f78b27fc640 10 client.18174 handle_command_reply: tid=0 2024-09-11T23:02:19.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.852+0000 7f78d3bcc740 1 client.18174 shutdown 2024-09-11T23:02:19.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.852+0000 7f78d3bcc740 1 -- 172.21.15.152:0/991098341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815cf370 msgr2=0x5638815cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:19.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.852+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/991098341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815cf370 0x5638815cc4f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f78bc012dc0 tx=0x7f78bc004730 comp rx=0 tx=0).stop 2024-09-11T23:02:19.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.852+0000 7f78d3bcc740 1 -- 172.21.15.152:0/991098341 shutdown_connections 2024-09-11T23:02:19.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.852+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/991098341 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x5638815def40 0x5638815e1330 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.852+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/991098341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815cf370 0x5638815cc4f0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.852+0000 7f78b27fc640 0 client.18174 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T23:02:19.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.852+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/991098341 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c5b40 0x5638815cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.852+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/991098341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815c45d0 0x5638815cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.852+0000 7f78d3bcc740 1 -- 172.21.15.152:0/991098341 >> 172.21.15.152:0/991098341 conn(0x56388158d310 msgr2=0x5638815c6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:19.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.853+0000 7f78d3bcc740 1 -- 172.21.15.152:0/991098341 shutdown_connections 2024-09-11T23:02:19.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.853+0000 7f78d3bcc740 1 -- 172.21.15.152:0/991098341 wait complete. 2024-09-11T23:02:19.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.853+0000 7f78d3bcc740 20 client.18174 trim_cache size 0 max 16384 2024-09-11T23:02:19.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.865+0000 7f78d3bcc740 1 Processor -- start 2024-09-11T23:02:19.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78d3bcc740 1 -- start start 2024-09-11T23:02:19.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78d3bcc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815c45d0 0x5638815db2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78d3bcc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c5b40 0x5638815c0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78d3bcc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815cf370 0x5638815c0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78d3bcc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1ad8b0 con 0x5638815cf370 2024-09-11T23:02:19.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78d3bcc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1af3f0 con 0x5638815c5b40 2024-09-11T23:02:19.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78d3bcc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1adf80 con 0x5638815c45d0 2024-09-11T23:02:19.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78ca7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815cf370 0x5638815c0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78ca7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815cf370 0x5638815c0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40588/0 (socket says 172.21.15.152:40588) 2024-09-11T23:02:19.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78ca7fc640 1 -- 172.21.15.152:0/1502794892 learned_addr learned my addr 172.21.15.152:0/1502794892 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:19.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78caffd640 1 --2- 172.21.15.152:0/1502794892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c5b40 0x5638815c0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/1502794892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815c45d0 0x5638815db2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78ca7fc640 1 -- 172.21.15.152:0/1502794892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815c45d0 msgr2=0x5638815db2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:19.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/1502794892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815c45d0 0x5638815db2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78ca7fc640 1 -- 172.21.15.152:0/1502794892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c5b40 msgr2=0x5638815c0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:19.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/1502794892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c5b40 0x5638815c0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.866+0000 7f78ca7fc640 1 -- 172.21.15.152:0/1502794892 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638815e1910 con 0x5638815cf370 2024-09-11T23:02:19.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.867+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/1502794892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815cf370 0x5638815c0c40 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00d440 tx=0x7f78bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5672bba9b35ca43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:19.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.867+0000 7f78c0ff9640 1 -- 172.21.15.152:0/1502794892 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc02b040 con 0x5638815cf370 2024-09-11T23:02:19.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.867+0000 7f78c0ff9640 1 -- 172.21.15.152:0/1502794892 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78bc00db10 con 0x5638815cf370 2024-09-11T23:02:19.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.867+0000 7f78c0ff9640 1 -- 172.21.15.152:0/1502794892 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0053e0 con 0x5638815cf370 2024-09-11T23:02:19.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 -- 172.21.15.152:0/1502794892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815cf370 msgr2=0x5638815c0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:19.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/1502794892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815cf370 0x5638815c0c40 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f78bc00d440 tx=0x7f78bc0046c0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 -- 172.21.15.152:0/1502794892 shutdown_connections 2024-09-11T23:02:19.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/1502794892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815cf370 0x5638815c0c40 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/1502794892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c5b40 0x5638815c0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/1502794892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815c45d0 0x5638815db2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 -- 172.21.15.152:0/1502794892 >> 172.21.15.152:0/1502794892 conn(0x56388158d310 msgr2=0x5638815cbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:19.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 -- 172.21.15.152:0/1502794892 shutdown_connections 2024-09-11T23:02:19.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 -- 172.21.15.152:0/1502794892 wait complete. 2024-09-11T23:02:19.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 -- start start 2024-09-11T23:02:19.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 Processor -- start 2024-09-11T23:02:19.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c45d0 0x5638815d78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815c5b40 0x5638815c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815cf370 0x5638815c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1aed20 con 0x5638815c5b40 2024-09-11T23:02:19.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1ae650 con 0x5638815c45d0 2024-09-11T23:02:19.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc1ab8b0 con 0x5638815cf370 2024-09-11T23:02:19.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78d3bcc740 10 client.0 _ll_register_callbacks cb 0x7f78cc06b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:19.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.868+0000 7f78caffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815c5b40 0x5638815c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78caffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815c5b40 0x5638815c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40592/0 (socket says 172.21.15.152:40592) 2024-09-11T23:02:19.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78caffd640 1 -- 172.21.15.152:0/2686612892 learned_addr learned my addr 172.21.15.152:0/2686612892 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:19.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/2686612892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c45d0 0x5638815d78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/2686612892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815cf370 0x5638815c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78caffd640 1 -- 172.21.15.152:0/2686612892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815cf370 msgr2=0x5638815c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:19.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78caffd640 1 --2- 172.21.15.152:0/2686612892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815cf370 0x5638815c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78caffd640 1 -- 172.21.15.152:0/2686612892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c45d0 msgr2=0x5638815d78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:19.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78caffd640 1 --2- 172.21.15.152:0/2686612892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c45d0 0x5638815d78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78caffd640 1 -- 172.21.15.152:0/2686612892 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638815c94f0 con 0x5638815c5b40 2024-09-11T23:02:19.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/2686612892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815cf370 0x5638815c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:19.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78caffd640 1 --2- 172.21.15.152:0/2686612892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815c5b40 0x5638815c89f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f78b4031b20 tx=0x7f78b40317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c4395574111442 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:19.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:19.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78b27fc640 1 -- 172.21.15.152:0/2686612892 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4035040 con 0x5638815c5b40 2024-09-11T23:02:19.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78b27fc640 1 -- 172.21.15.152:0/2686612892 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f78b4004230 con 0x5638815c5b40 2024-09-11T23:02:19.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78d3bcc740 1 -- 172.21.15.152:0/2686612892 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78bc00db10 con 0x5638815c5b40 2024-09-11T23:02:19.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78b27fc640 1 -- 172.21.15.152:0/2686612892 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f78b405e050 con 0x5638815c5b40 2024-09-11T23:02:19.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78d3bcc740 10 client.18180 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:19.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.869+0000 7f78d3bcc740 1 -- 172.21.15.152:0/2686612892 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638816498c0 con 0x5638815c5b40 2024-09-11T23:02:19.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.870+0000 7f78b27fc640 1 -- 172.21.15.152:0/2686612892 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78bc00db10 con 0x5638815c5b40 2024-09-11T23:02:19.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.870+0000 7f78b27fc640 1 -- 172.21.15.152:0/2686612892 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f78b404e600 con 0x5638815c5b40 2024-09-11T23:02:19.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.870+0000 7f78d3bcc740 10 client.18180 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:19.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.870+0000 7f78d3bcc740 10 client.18180 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:02:19.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.870+0000 7f78d3bcc740 20 client.18180 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:19.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.870+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/2686612892 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x563881649c00 0x56388164bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:19.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.870+0000 7f78d3bcc740 4 client.18180 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:02:19.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.870+0000 7f78d3bcc740 1 -- 172.21.15.152:0/2686612892 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563881531d70 con 0x563881649c00 2024-09-11T23:02:19.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.870+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/2686612892 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x563881649c00 0x56388164bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:19.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78b27fc640 10 client.18180 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T23:02:19.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78ca7fc640 1 --2- 172.21.15.152:0/2686612892 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x563881649c00 0x56388164bff0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5488d2ef737ee50a server_cookie=89e460b7b986c51 in_seq=0 out_seq=0 2024-09-11T23:02:19.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78b27fc640 1 -- 172.21.15.152:0/2686612892 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563881531d70 con 0x563881649c00 2024-09-11T23:02:19.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78b27fc640 10 client.18180 handle_command_reply: tid=0 2024-09-11T23:02:19.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78d3bcc740 1 client.18180 shutdown 2024-09-11T23:02:19.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78d3bcc740 1 -- 172.21.15.152:0/2686612892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815c5b40 msgr2=0x5638815c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:19.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/2686612892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815c5b40 0x5638815c89f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f78b4031b20 tx=0x7f78b40317a0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78d3bcc740 1 -- 172.21.15.152:0/2686612892 shutdown_connections 2024-09-11T23:02:19.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/2686612892 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x563881649c00 0x56388164bff0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/2686612892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5638815cf370 0x5638815c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/2686612892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5638815c5b40 0x5638815c89f0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78b27fc640 0 client.18180 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T23:02:19.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78d3bcc740 1 --2- 172.21.15.152:0/2686612892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638815c45d0 0x5638815d78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78d3bcc740 1 -- 172.21.15.152:0/2686612892 >> 172.21.15.152:0/2686612892 conn(0x56388158d310 msgr2=0x5638815c0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:19.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78d3bcc740 1 -- 172.21.15.152:0/2686612892 shutdown_connections 2024-09-11T23:02:19.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78d3bcc740 1 -- 172.21.15.152:0/2686612892 wait complete. 2024-09-11T23:02:19.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.871+0000 7f78d3bcc740 20 client.18180 trim_cache size 0 max 16384 2024-09-11T23:02:19.888 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:02:19.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.872+0000 7f78d17f3640 1 -- 172.21.15.152:0/304538602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f78b80a2cf0 msgr2=0x7f78b80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:19.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.872+0000 7f78d17f3640 1 --2- 172.21.15.152:0/304538602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f78b80a2cf0 0x7f78b80a51b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f78c4098530 tx=0x7f78c4066790 comp rx=0 tx=0).stop 2024-09-11T23:02:19.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.872+0000 7f78d17f3640 1 -- 172.21.15.152:0/304538602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc10c170 msgr2=0x7f78cc19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:19.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.872+0000 7f78d17f3640 1 --2- 172.21.15.152:0/304538602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc10c170 0x7f78cc19cca0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f78c400eaa0 tx=0x7f78c400ef60 comp rx=0 tx=0).stop 2024-09-11T23:02:19.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.872+0000 7f78d17f3640 1 -- 172.21.15.152:0/304538602 shutdown_connections 2024-09-11T23:02:19.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.872+0000 7f78d17f3640 1 --2- 172.21.15.152:0/304538602 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f78b80a2cf0 0x7f78b80a51b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.872+0000 7f78d17f3640 1 --2- 172.21.15.152:0/304538602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f78cc10c170 0x7f78cc19cca0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.872+0000 7f78d17f3640 1 --2- 172.21.15.152:0/304538602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78cc103660 0x7f78cc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.872+0000 7f78d17f3640 1 --2- 172.21.15.152:0/304538602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78cc102d40 0x7f78cc1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:19.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.872+0000 7f78d17f3640 1 -- 172.21.15.152:0/304538602 >> 172.21.15.152:0/304538602 conn(0x7f78cc0fc7a0 msgr2=0x7f78cc0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:19.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.872+0000 7f78d17f3640 1 -- 172.21.15.152:0/304538602 shutdown_connections 2024-09-11T23:02:19.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:19.872+0000 7f78d17f3640 1 -- 172.21.15.152:0/304538602 wait complete. 2024-09-11T23:02:20.585 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:02:20.831 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:20.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:20 smithi152 ceph-mon[31181]: pgmap v291: 97 pgs: 97 active+clean; 4.0 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 194 B/s wr, 15 op/s 2024-09-11T23:02:20.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:20 smithi152 ceph-mon[31181]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T23:02:21.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:20 smithi191 ceph-mon[35211]: pgmap v291: 97 pgs: 97 active+clean; 4.0 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 194 B/s wr, 15 op/s 2024-09-11T23:02:21.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:20 smithi191 ceph-mon[35211]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T23:02:21.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:20 smithi204 ceph-mon[35454]: pgmap v291: 97 pgs: 97 active+clean; 4.0 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 194 B/s wr, 15 op/s 2024-09-11T23:02:21.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:20 smithi204 ceph-mon[35454]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T23:02:21.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.310+0000 7f4c77220640 1 -- 172.21.15.152:0/868103207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c70101420 msgr2=0x7f4c7010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:21.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.310+0000 7f4c77220640 1 --2- 172.21.15.152:0/868103207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c70101420 0x7f4c7010d950 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4c68009cd0 tx=0x7f4c6802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.310+0000 7f4c77220640 1 -- 172.21.15.152:0/868103207 shutdown_connections 2024-09-11T23:02:21.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.310+0000 7f4c77220640 1 --2- 172.21.15.152:0/868103207 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c70101420 0x7f4c7010d950 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.310+0000 7f4c77220640 1 --2- 172.21.15.152:0/868103207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c70100a80 0x7f4c70100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.310+0000 7f4c77220640 1 --2- 172.21.15.152:0/868103207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c70106a80 0x7f4c70106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.310+0000 7f4c77220640 1 -- 172.21.15.152:0/868103207 >> 172.21.15.152:0/868103207 conn(0x7f4c700fc7a0 msgr2=0x7f4c700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:21.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.310+0000 7f4c77220640 1 -- 172.21.15.152:0/868103207 shutdown_connections 2024-09-11T23:02:21.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.310+0000 7f4c77220640 1 -- 172.21.15.152:0/868103207 wait complete. 2024-09-11T23:02:21.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.310+0000 7f4c77220640 1 Processor -- start 2024-09-11T23:02:21.346 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.310+0000 7f4c77220640 1 -- start start 2024-09-11T23:02:21.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c77220640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c70100a80 0x7f4c70198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c77220640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c70101420 0x7f4c70198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c77220640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c70106a80 0x7f4c7019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c77220640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4c70110e80 con 0x7f4c70101420 2024-09-11T23:02:21.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c77220640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4c70110d00 con 0x7f4c70106a80 2024-09-11T23:02:21.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c77220640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4c70111000 con 0x7f4c70100a80 2024-09-11T23:02:21.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c67fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c70101420 0x7f4c70198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c67fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c70101420 0x7f4c70198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40598/0 (socket says 172.21.15.152:40598) 2024-09-11T23:02:21.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c67fff640 1 -- 172.21.15.152:0/1290438897 learned_addr learned my addr 172.21.15.152:0/1290438897 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:21.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c74fbc640 1 --2- 172.21.15.152:0/1290438897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c70100a80 0x7f4c70198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c757bd640 1 --2- 172.21.15.152:0/1290438897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c70106a80 0x7f4c7019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.347 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c67fff640 1 -- 172.21.15.152:0/1290438897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c70100a80 msgr2=0x7f4c70198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:21.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c67fff640 1 --2- 172.21.15.152:0/1290438897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c70100a80 0x7f4c70198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c67fff640 1 -- 172.21.15.152:0/1290438897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c70106a80 msgr2=0x7f4c7019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:21.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c67fff640 1 --2- 172.21.15.152:0/1290438897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c70106a80 0x7f4c7019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c67fff640 1 -- 172.21.15.152:0/1290438897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7019d3e0 con 0x7f4c70101420 2024-09-11T23:02:21.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c67fff640 1 --2- 172.21.15.152:0/1290438897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c70101420 0x7f4c70198970 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000d770 tx=0x7f4c6000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:21.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c65ffb640 1 -- 172.21.15.152:0/1290438897 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60017070 con 0x7f4c70101420 2024-09-11T23:02:21.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c65ffb640 1 -- 172.21.15.152:0/1290438897 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c60005500 con 0x7f4c70101420 2024-09-11T23:02:21.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c65ffb640 1 -- 172.21.15.152:0/1290438897 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60004e00 con 0x7f4c70101420 2024-09-11T23:02:21.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.311+0000 7f4c77220640 1 -- 172.21.15.152:0/1290438897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7019d6d0 con 0x7f4c70101420 2024-09-11T23:02:21.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.312+0000 7f4c77220640 1 -- 172.21.15.152:0/1290438897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c701a4fb0 con 0x7f4c70101420 2024-09-11T23:02:21.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.314+0000 7f4c65ffb640 1 -- 172.21.15.152:0/1290438897 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4c60004fa0 con 0x7f4c70101420 2024-09-11T23:02:21.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.314+0000 7f4c65ffb640 1 --2- 172.21.15.152:0/1290438897 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4c580a2d70 0x7f4c580a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.314+0000 7f4c795f9740 1 Processor -- start 2024-09-11T23:02:21.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.314+0000 7f4c795f9740 1 -- start start 2024-09-11T23:02:21.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.314+0000 7f4c74fbc640 1 --2- 172.21.15.152:0/1290438897 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4c580a2d70 0x7f4c580a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.314+0000 7f4c65ffb640 1 -- 172.21.15.152:0/1290438897 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f4c600ced50 con 0x7f4c70101420 2024-09-11T23:02:21.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.314+0000 7f4c795f9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167cf8b40 0x55e167cf7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.314+0000 7f4c795f9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf75d0 0x55e167cf7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c795f9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167d02370 0x55e167d04760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c795f9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55e167c5d2f0 con 0x55e167cf8b40 2024-09-11T23:02:21.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c795f9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e167c04cf0 con 0x55e167cf75d0 2024-09-11T23:02:21.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c795f9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e167c592b0 con 0x55e167d02370 2024-09-11T23:02:21.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c74fbc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167cf8b40 0x55e167cf7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c74fbc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167cf8b40 0x55e167cf7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40612/0 (socket says 172.21.15.152:40612) 2024-09-11T23:02:21.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c74fbc640 1 -- 172.21.15.152:0/3952305039 learned_addr learned my addr 172.21.15.152:0/3952305039 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:21.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c757bd640 1 --2- 172.21.15.152:0/3952305039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167d02370 0x55e167d04760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c67fff640 1 --2- 172.21.15.152:0/3952305039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf75d0 0x55e167cf7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c74fbc640 1 --2- 172.21.15.152:0/1290438897 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4c580a2d70 0x7f4c580a5230 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c012440 tx=0x7f4c6c012840 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:21.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c74fbc640 1 -- 172.21.15.152:0/3952305039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167d02370 msgr2=0x55e167d04760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:21.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c74fbc640 1 --2- 172.21.15.152:0/3952305039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167d02370 0x55e167d04760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c74fbc640 1 -- 172.21.15.152:0/3952305039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf75d0 msgr2=0x55e167cf7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:21.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c74fbc640 1 --2- 172.21.15.152:0/3952305039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf75d0 0x55e167cf7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c74fbc640 1 -- 172.21.15.152:0/3952305039 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e167be93a0 con 0x55e167cf8b40 2024-09-11T23:02:21.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c74fbc640 1 --2- 172.21.15.152:0/3952305039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167cf8b40 0x55e167cf7090 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00e6d0 tx=0x7f4c6c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141fe892253c467f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:21.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c56ffd640 1 -- 172.21.15.152:0/3952305039 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c025070 con 0x55e167cf8b40 2024-09-11T23:02:21.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c56ffd640 1 -- 172.21.15.152:0/3952305039 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6c01fbf0 con 0x55e167cf8b40 2024-09-11T23:02:21.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.315+0000 7f4c56ffd640 1 -- 172.21.15.152:0/3952305039 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c01ed50 con 0x55e167cf8b40 2024-09-11T23:02:21.350 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.316+0000 7f4c795f9740 1 -- 172.21.15.152:0/3952305039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167cf8b40 msgr2=0x55e167cf7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:21.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.316+0000 7f4c795f9740 1 --2- 172.21.15.152:0/3952305039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167cf8b40 0x55e167cf7090 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00e6d0 tx=0x7f4c6c01e000 comp rx=0 tx=0).stop 2024-09-11T23:02:21.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.316+0000 7f4c795f9740 1 -- 172.21.15.152:0/3952305039 shutdown_connections 2024-09-11T23:02:21.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.316+0000 7f4c795f9740 1 --2- 172.21.15.152:0/3952305039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167d02370 0x55e167d04760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.316+0000 7f4c795f9740 1 --2- 172.21.15.152:0/3952305039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf75d0 0x55e167cf7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.316+0000 7f4c795f9740 1 --2- 172.21.15.152:0/3952305039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167cf8b40 0x55e167cf7090 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.316+0000 7f4c795f9740 1 -- 172.21.15.152:0/3952305039 >> 172.21.15.152:0/3952305039 conn(0x55e167cc0310 msgr2=0x55e167cf3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:21.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.316+0000 7f4c795f9740 1 -- 172.21.15.152:0/3952305039 shutdown_connections 2024-09-11T23:02:21.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.316+0000 7f4c795f9740 1 -- 172.21.15.152:0/3952305039 wait complete. 2024-09-11T23:02:21.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.316+0000 7f4c795f9740 1 -- start start 2024-09-11T23:02:21.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c795f9740 1 Processor -- start 2024-09-11T23:02:21.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c795f9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf75d0 0x55e167d01d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c795f9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167cf8b40 0x55e167cfeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c795f9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167d02370 0x55e167cff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.351 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c795f9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55e167d05070 con 0x55e167cf8b40 2024-09-11T23:02:21.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c795f9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55e167c18060 con 0x55e167d02370 2024-09-11T23:02:21.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c795f9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55e167d051f0 con 0x55e167cf75d0 2024-09-11T23:02:21.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c795f9740 10 client.0 _ll_register_callbacks cb 0x55e16796b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:21.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c67fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167cf8b40 0x55e167cfeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c67fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167cf8b40 0x55e167cfeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40622/0 (socket says 172.21.15.152:40622) 2024-09-11T23:02:21.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c67fff640 1 -- 172.21.15.152:0/2924455672 learned_addr learned my addr 172.21.15.152:0/2924455672 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:21.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c757bd640 1 --2- 172.21.15.152:0/2924455672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167d02370 0x55e167cff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.352 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c74fbc640 1 --2- 172.21.15.152:0/2924455672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf75d0 0x55e167d01d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c67fff640 1 -- 172.21.15.152:0/2924455672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf75d0 msgr2=0x55e167d01d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:21.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c67fff640 1 --2- 172.21.15.152:0/2924455672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf75d0 0x55e167d01d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c67fff640 1 -- 172.21.15.152:0/2924455672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167d02370 msgr2=0x55e167cff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:21.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c67fff640 1 --2- 172.21.15.152:0/2924455672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167d02370 0x55e167cff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c67fff640 1 -- 172.21.15.152:0/2924455672 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1679ce3c0 con 0x55e167cf8b40 2024-09-11T23:02:21.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c74fbc640 1 --2- 172.21.15.152:0/2924455672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf75d0 0x55e167d01d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:21.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c67fff640 1 --2- 172.21.15.152:0/2924455672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167cf8b40 0x55e167cfeff0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000d740 tx=0x7f4c60004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696327d1340b29c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:21.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.317+0000 7f4c4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:21.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.318+0000 7f4c4ffff640 1 -- 172.21.15.152:0/2924455672 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c600b4060 con 0x55e167cf8b40 2024-09-11T23:02:21.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.318+0000 7f4c795f9740 1 -- 172.21.15.152:0/2924455672 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c6c01fbf0 con 0x55e167cf8b40 2024-09-11T23:02:21.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.318+0000 7f4c4ffff640 1 -- 172.21.15.152:0/2924455672 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f4c60009cc0 con 0x55e167cf8b40 2024-09-11T23:02:21.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.318+0000 7f4c4ffff640 1 -- 172.21.15.152:0/2924455672 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c60020040 con 0x55e167cf8b40 2024-09-11T23:02:21.353 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.318+0000 7f4c795f9740 10 client.18192 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:21.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.318+0000 7f4c795f9740 1 -- 172.21.15.152:0/2924455672 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e167d10b40 con 0x55e167cf8b40 2024-09-11T23:02:21.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.318+0000 7f4c4ffff640 1 -- 172.21.15.152:0/2924455672 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c6c01fbf0 con 0x55e167cf8b40 2024-09-11T23:02:21.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.318+0000 7f4c4ffff640 1 -- 172.21.15.152:0/2924455672 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c6009f170 con 0x55e167cf8b40 2024-09-11T23:02:21.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.318+0000 7f4c795f9740 10 client.18192 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:21.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.318+0000 7f4c795f9740 10 client.18192 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:02:21.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.318+0000 7f4c795f9740 20 client.18192 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:21.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.318+0000 7f4c795f9740 1 --2- 172.21.15.152:0/2924455672 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55e167d11f50 0x55e167d14360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.354 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.318+0000 7f4c795f9740 4 client.18192 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:02:21.354 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:02:21.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.318+0000 7f4c795f9740 1 -- 172.21.15.152:0/2924455672 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e167c6a860 con 0x55e167d11f50 2024-09-11T23:02:21.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.319+0000 7f4c74fbc640 1 --2- 172.21.15.152:0/2924455672 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55e167d11f50 0x55e167d14360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.319+0000 7f4c74fbc640 1 --2- 172.21.15.152:0/2924455672 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55e167d11f50 0x55e167d14360 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f3ce1666ef15f4b server_cookie=efdf7f67b4d0929a in_seq=0 out_seq=0 2024-09-11T23:02:21.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.319+0000 7f4c4ffff640 10 client.18192 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T23:02:21.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.320+0000 7f4c4ffff640 1 -- 172.21.15.152:0/2924455672 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e167c6a860 con 0x55e167d11f50 2024-09-11T23:02:21.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.320+0000 7f4c4ffff640 10 client.18192 handle_command_reply: tid=0 2024-09-11T23:02:21.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.320+0000 7f4c795f9740 1 client.18192 shutdown 2024-09-11T23:02:21.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.320+0000 7f4c795f9740 1 -- 172.21.15.152:0/2924455672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167cf8b40 msgr2=0x55e167cfeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:21.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.320+0000 7f4c795f9740 1 --2- 172.21.15.152:0/2924455672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167cf8b40 0x55e167cfeff0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000d740 tx=0x7f4c60004800 comp rx=0 tx=0).stop 2024-09-11T23:02:21.355 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.320+0000 7f4c795f9740 1 -- 172.21.15.152:0/2924455672 shutdown_connections 2024-09-11T23:02:21.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.320+0000 7f4c795f9740 1 --2- 172.21.15.152:0/2924455672 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55e167d11f50 0x55e167d14360 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.320+0000 7f4c795f9740 1 --2- 172.21.15.152:0/2924455672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167d02370 0x55e167cff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.320+0000 7f4c795f9740 1 --2- 172.21.15.152:0/2924455672 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167cf8b40 0x55e167cfeff0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.320+0000 7f4c795f9740 1 --2- 172.21.15.152:0/2924455672 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf75d0 0x55e167d01d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.320+0000 7f4c4ffff640 0 client.18192 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T23:02:21.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.320+0000 7f4c795f9740 1 -- 172.21.15.152:0/2924455672 >> 172.21.15.152:0/2924455672 conn(0x55e167cc0310 msgr2=0x55e167cf9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:21.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.320+0000 7f4c795f9740 1 -- 172.21.15.152:0/2924455672 shutdown_connections 2024-09-11T23:02:21.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.320+0000 7f4c795f9740 1 -- 172.21.15.152:0/2924455672 wait complete. 2024-09-11T23:02:21.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.320+0000 7f4c795f9740 20 client.18192 trim_cache size 0 max 16384 2024-09-11T23:02:21.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c795f9740 1 Processor -- start 2024-09-11T23:02:21.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c795f9740 1 -- start start 2024-09-11T23:02:21.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c795f9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf75d0 0x55e167cfa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.356 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c795f9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf8b40 0x55e167cfa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c795f9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167d02370 0x55e167d19c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c795f9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4c701ad7d0 con 0x55e167d02370 2024-09-11T23:02:21.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c795f9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4c701af310 con 0x55e167cf8b40 2024-09-11T23:02:21.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c795f9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4c701adea0 con 0x55e167cf75d0 2024-09-11T23:02:21.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c757bd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167d02370 0x55e167d19c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c757bd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167d02370 0x55e167d19c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40634/0 (socket says 172.21.15.152:40634) 2024-09-11T23:02:21.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c757bd640 1 -- 172.21.15.152:0/1520800476 learned_addr learned my addr 172.21.15.152:0/1520800476 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:21.357 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c74fbc640 1 --2- 172.21.15.152:0/1520800476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf75d0 0x55e167cfa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.359 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c757bd640 1 -- 172.21.15.152:0/1520800476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf75d0 msgr2=0x55e167cfa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:21.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c757bd640 1 --2- 172.21.15.152:0/1520800476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf75d0 0x55e167cfa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c757bd640 1 -- 172.21.15.152:0/1520800476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf8b40 msgr2=0x55e167cfa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:21.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c67fff640 1 --2- 172.21.15.152:0/1520800476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf8b40 0x55e167cfa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c757bd640 1 --2- 172.21.15.152:0/1520800476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf8b40 0x55e167cfa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c757bd640 1 -- 172.21.15.152:0/1520800476 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e167cf34f0 con 0x55e167d02370 2024-09-11T23:02:21.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c74fbc640 1 --2- 172.21.15.152:0/1520800476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf75d0 0x55e167cfa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:21.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.333+0000 7f4c67fff640 1 --2- 172.21.15.152:0/1520800476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf8b40 0x55e167cfa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:21.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.334+0000 7f4c757bd640 1 --2- 172.21.15.152:0/1520800476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167d02370 0x55e167d19c30 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f4c68009cd0 tx=0x7f4c68005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed726d943134b21 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:21.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.334+0000 7f4c567fc640 1 -- 172.21.15.152:0/1520800476 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c68031ac0 con 0x55e167d02370 2024-09-11T23:02:21.360 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.334+0000 7f4c567fc640 1 -- 172.21.15.152:0/1520800476 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c68036040 con 0x55e167d02370 2024-09-11T23:02:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.334+0000 7f4c567fc640 1 -- 172.21.15.152:0/1520800476 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c680047b0 con 0x55e167d02370 2024-09-11T23:02:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.334+0000 7f4c795f9740 1 -- 172.21.15.152:0/1520800476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167d02370 msgr2=0x55e167d19c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.334+0000 7f4c795f9740 1 --2- 172.21.15.152:0/1520800476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167d02370 0x55e167d19c30 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f4c68009cd0 tx=0x7f4c68005fa0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.334+0000 7f4c795f9740 1 -- 172.21.15.152:0/1520800476 shutdown_connections 2024-09-11T23:02:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.334+0000 7f4c795f9740 1 --2- 172.21.15.152:0/1520800476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167d02370 0x55e167d19c30 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.334+0000 7f4c795f9740 1 --2- 172.21.15.152:0/1520800476 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf8b40 0x55e167cfa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.334+0000 7f4c795f9740 1 --2- 172.21.15.152:0/1520800476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf75d0 0x55e167cfa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.334+0000 7f4c795f9740 1 -- 172.21.15.152:0/1520800476 >> 172.21.15.152:0/1520800476 conn(0x55e167cc0310 msgr2=0x55e167d01070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.334+0000 7f4c795f9740 1 -- 172.21.15.152:0/1520800476 shutdown_connections 2024-09-11T23:02:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.334+0000 7f4c795f9740 1 -- 172.21.15.152:0/1520800476 wait complete. 2024-09-11T23:02:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.335+0000 7f4c795f9740 1 -- start start 2024-09-11T23:02:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.335+0000 7f4c795f9740 1 Processor -- start 2024-09-11T23:02:21.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.335+0000 7f4c795f9740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf75d0 0x55e167d0a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.335+0000 7f4c795f9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf8b40 0x55e167d0a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.335+0000 7f4c795f9740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167d02370 0x55e167d07450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.335+0000 7f4c795f9740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4c701aec40 con 0x55e167d02370 2024-09-11T23:02:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.335+0000 7f4c795f9740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4c701ae570 con 0x55e167cf75d0 2024-09-11T23:02:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.335+0000 7f4c795f9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4c701ab7d0 con 0x55e167cf8b40 2024-09-11T23:02:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.335+0000 7f4c795f9740 10 client.0 _ll_register_callbacks cb 0x7f4c6009aca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.335+0000 7f4c757bd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167d02370 0x55e167d07450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.335+0000 7f4c757bd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167d02370 0x55e167d07450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40640/0 (socket says 172.21.15.152:40640) 2024-09-11T23:02:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.335+0000 7f4c757bd640 1 -- 172.21.15.152:0/704933132 learned_addr learned my addr 172.21.15.152:0/704933132 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:21.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.335+0000 7f4c74fbc640 1 --2- 172.21.15.152:0/704933132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf75d0 0x55e167d0a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c757bd640 1 -- 172.21.15.152:0/704933132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf8b40 msgr2=0x55e167d0a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c67fff640 1 --2- 172.21.15.152:0/704933132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf8b40 0x55e167d0a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c757bd640 1 --2- 172.21.15.152:0/704933132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf8b40 0x55e167d0a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c757bd640 1 -- 172.21.15.152:0/704933132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf75d0 msgr2=0x55e167d0a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c757bd640 1 --2- 172.21.15.152:0/704933132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf75d0 0x55e167d0a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c757bd640 1 -- 172.21.15.152:0/704933132 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e167d07990 con 0x55e167d02370 2024-09-11T23:02:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c74fbc640 1 --2- 172.21.15.152:0/704933132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf75d0 0x55e167d0a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c757bd640 1 --2- 172.21.15.152:0/704933132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167d02370 0x55e167d07450 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f4c68005e30 tx=0x7f4c68031b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0dd9aed26730ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c4ffff640 1 -- 172.21.15.152:0/704933132 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c68003d80 con 0x55e167d02370 2024-09-11T23:02:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c4ffff640 1 -- 172.21.15.152:0/704933132 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f4c68003f20 con 0x55e167d02370 2024-09-11T23:02:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c795f9740 1 -- 172.21.15.152:0/704933132 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c68036040 con 0x55e167d02370 2024-09-11T23:02:21.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c4ffff640 1 -- 172.21.15.152:0/704933132 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4c68062050 con 0x55e167d02370 2024-09-11T23:02:21.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c795f9740 10 client.18198 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:21.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c795f9740 1 -- 172.21.15.152:0/704933132 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e167d7d3c0 con 0x55e167d02370 2024-09-11T23:02:21.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c4ffff640 1 -- 172.21.15.152:0/704933132 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c68036040 con 0x55e167d02370 2024-09-11T23:02:21.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c4ffff640 1 -- 172.21.15.152:0/704933132 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4c68030070 con 0x55e167d02370 2024-09-11T23:02:21.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c795f9740 10 client.18198 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:21.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c795f9740 10 client.18198 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:02:21.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c795f9740 20 client.18198 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:21.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c795f9740 1 --2- 172.21.15.152:0/704933132 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55e167d7db00 0x55e167d7ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:21.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.336+0000 7f4c795f9740 4 client.18198 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:02:21.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.337+0000 7f4c795f9740 1 -- 172.21.15.152:0/704933132 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e167c64d70 con 0x55e167d7db00 2024-09-11T23:02:21.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.337+0000 7f4c74fbc640 1 --2- 172.21.15.152:0/704933132 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55e167d7db00 0x55e167d7ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:21.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.337+0000 7f4c74fbc640 1 --2- 172.21.15.152:0/704933132 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55e167d7db00 0x55e167d7ff20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22d18237c632b18e server_cookie=ea3b1463f086d34e in_seq=0 out_seq=0 2024-09-11T23:02:21.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.337+0000 7f4c4ffff640 10 client.18198 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T23:02:21.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.337+0000 7f4c4ffff640 1 -- 172.21.15.152:0/704933132 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e167c64d70 con 0x55e167d7db00 2024-09-11T23:02:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.337+0000 7f4c4ffff640 10 client.18198 handle_command_reply: tid=0 2024-09-11T23:02:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.337+0000 7f4c795f9740 1 client.18198 shutdown 2024-09-11T23:02:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.337+0000 7f4c795f9740 1 -- 172.21.15.152:0/704933132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167d02370 msgr2=0x55e167d07450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.337+0000 7f4c795f9740 1 --2- 172.21.15.152:0/704933132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167d02370 0x55e167d07450 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f4c68005e30 tx=0x7f4c68031b70 comp rx=0 tx=0).stop 2024-09-11T23:02:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.337+0000 7f4c795f9740 1 -- 172.21.15.152:0/704933132 shutdown_connections 2024-09-11T23:02:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.337+0000 7f4c795f9740 1 --2- 172.21.15.152:0/704933132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55e167d02370 0x55e167d07450 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.337+0000 7f4c795f9740 1 --2- 172.21.15.152:0/704933132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e167cf8b40 0x55e167d0a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.337+0000 7f4c795f9740 1 --2- 172.21.15.152:0/704933132 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x55e167d7db00 0x55e167d7ff20 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.337+0000 7f4c795f9740 1 --2- 172.21.15.152:0/704933132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e167cf75d0 0x55e167d0a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.337+0000 7f4c4ffff640 0 client.18198 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T23:02:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.337+0000 7f4c795f9740 1 -- 172.21.15.152:0/704933132 >> 172.21.15.152:0/704933132 conn(0x55e167cc0310 msgr2=0x55e167d00850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.338+0000 7f4c795f9740 1 -- 172.21.15.152:0/704933132 shutdown_connections 2024-09-11T23:02:21.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.338+0000 7f4c795f9740 1 -- 172.21.15.152:0/704933132 wait complete. 2024-09-11T23:02:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.338+0000 7f4c795f9740 20 client.18198 trim_cache size 0 max 16384 2024-09-11T23:02:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.338+0000 7f4c77220640 1 -- 172.21.15.152:0/1290438897 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4c580a2d70 msgr2=0x7f4c580a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.338+0000 7f4c77220640 1 --2- 172.21.15.152:0/1290438897 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4c580a2d70 0x7f4c580a5230 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c012440 tx=0x7f4c6c012840 comp rx=0 tx=0).stop 2024-09-11T23:02:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.338+0000 7f4c77220640 1 -- 172.21.15.152:0/1290438897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c70101420 msgr2=0x7f4c70198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.338+0000 7f4c77220640 1 --2- 172.21.15.152:0/1290438897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c70101420 0x7f4c70198970 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000d770 tx=0x7f4c6000dc30 comp rx=0 tx=0).stop 2024-09-11T23:02:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.338+0000 7f4c77220640 1 -- 172.21.15.152:0/1290438897 shutdown_connections 2024-09-11T23:02:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.338+0000 7f4c77220640 1 --2- 172.21.15.152:0/1290438897 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4c580a2d70 0x7f4c580a5230 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.338+0000 7f4c77220640 1 --2- 172.21.15.152:0/1290438897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4c70106a80 0x7f4c7019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.338+0000 7f4c77220640 1 --2- 172.21.15.152:0/1290438897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4c70101420 0x7f4c70198970 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.338+0000 7f4c77220640 1 --2- 172.21.15.152:0/1290438897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c70100a80 0x7f4c70198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.338+0000 7f4c77220640 1 -- 172.21.15.152:0/1290438897 >> 172.21.15.152:0/1290438897 conn(0x7f4c700fc7a0 msgr2=0x7f4c70104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.338+0000 7f4c77220640 1 -- 172.21.15.152:0/1290438897 shutdown_connections 2024-09-11T23:02:21.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:21.338+0000 7f4c77220640 1 -- 172.21.15.152:0/1290438897 wait complete. 2024-09-11T23:02:22.123 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:02:22.380 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:23.007 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:22 smithi152 ceph-mon[31181]: pgmap v293: 97 pgs: 97 active+clean; 4.0 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:02:23.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.002+0000 7f4e840c0640 1 -- 172.21.15.152:0/3314082130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e7c101de0 msgr2=0x7f4e7c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:23.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.002+0000 7f4e840c0640 1 --2- 172.21.15.152:0/3314082130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e7c101de0 0x7f4e7c1021c0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009900 tx=0x7f4e7802a410 comp rx=0 tx=0).stop 2024-09-11T23:02:23.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.002+0000 7f4e840c0640 1 -- 172.21.15.152:0/3314082130 shutdown_connections 2024-09-11T23:02:23.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.002+0000 7f4e840c0640 1 --2- 172.21.15.152:0/3314082130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e7c106f10 0x7f4e7c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.032 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.002+0000 7f4e840c0640 1 --2- 172.21.15.152:0/3314082130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e7c102790 0x7f4e7c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.002+0000 7f4e840c0640 1 --2- 172.21.15.152:0/3314082130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e7c101de0 0x7f4e7c1021c0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.002+0000 7f4e840c0640 1 -- 172.21.15.152:0/3314082130 >> 172.21.15.152:0/3314082130 conn(0x7f4e7c0fd8c0 msgr2=0x7f4e7c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:23.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.002+0000 7f4e840c0640 1 -- 172.21.15.152:0/3314082130 shutdown_connections 2024-09-11T23:02:23.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.002+0000 7f4e840c0640 1 -- 172.21.15.152:0/3314082130 wait complete. 2024-09-11T23:02:23.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.003+0000 7f4e840c0640 1 Processor -- start 2024-09-11T23:02:23.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.003+0000 7f4e840c0640 1 -- start start 2024-09-11T23:02:23.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.003+0000 7f4e840c0640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e7c101de0 0x7f4e7c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.003+0000 7f4e840c0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e7c102790 0x7f4e7c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.003+0000 7f4e840c0640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e7c106f10 0x7f4e7c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.003+0000 7f4e840c0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c111f30 con 0x7f4e7c102790 2024-09-11T23:02:23.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.003+0000 7f4e840c0640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c111db0 con 0x7f4e7c101de0 2024-09-11T23:02:23.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.003+0000 7f4e840c0640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1120b0 con 0x7f4e7c106f10 2024-09-11T23:02:23.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.003+0000 7f4e8165b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e7c102790 0x7f4e7c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.003+0000 7f4e8165b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e7c102790 0x7f4e7c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40648/0 (socket says 172.21.15.152:40648) 2024-09-11T23:02:23.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.003+0000 7f4e8165b640 1 -- 172.21.15.152:0/87998474 learned_addr learned my addr 172.21.15.152:0/87998474 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:23.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.003+0000 7f4e81e5c640 1 --2- 172.21.15.152:0/87998474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e7c101de0 0x7f4e7c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.004+0000 7f4e8165b640 1 -- 172.21.15.152:0/87998474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e7c106f10 msgr2=0x7f4e7c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:23.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.004+0000 7f4e8265d640 1 --2- 172.21.15.152:0/87998474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e7c106f10 0x7f4e7c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.004+0000 7f4e8165b640 1 --2- 172.21.15.152:0/87998474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e7c106f10 0x7f4e7c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.004+0000 7f4e8165b640 1 -- 172.21.15.152:0/87998474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e7c101de0 msgr2=0x7f4e7c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:23.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.004+0000 7f4e8165b640 1 --2- 172.21.15.152:0/87998474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e7c101de0 0x7f4e7c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.004+0000 7f4e8165b640 1 -- 172.21.15.152:0/87998474 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c19d3e0 con 0x7f4e7c102790 2024-09-11T23:02:23.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.004+0000 7f4e8265d640 1 --2- 172.21.15.152:0/87998474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e7c106f10 0x7f4e7c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:23.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.004+0000 7f4e8165b640 1 --2- 172.21.15.152:0/87998474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e7c102790 0x7f4e7c198970 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00d950 tx=0x7f4e6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:23.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.004+0000 7f4e72ffd640 1 -- 172.21.15.152:0/87998474 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c017030 con 0x7f4e7c102790 2024-09-11T23:02:23.034 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.004+0000 7f4e72ffd640 1 -- 172.21.15.152:0/87998474 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6c004020 con 0x7f4e7c102790 2024-09-11T23:02:23.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.004+0000 7f4e840c0640 1 -- 172.21.15.152:0/87998474 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7c19d6d0 con 0x7f4e7c102790 2024-09-11T23:02:23.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.004+0000 7f4e72ffd640 1 -- 172.21.15.152:0/87998474 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c0047a0 con 0x7f4e7c102790 2024-09-11T23:02:23.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.004+0000 7f4e840c0640 1 -- 172.21.15.152:0/87998474 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7c1a4fb0 con 0x7f4e7c102790 2024-09-11T23:02:23.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.006+0000 7f4e72ffd640 1 -- 172.21.15.152:0/87998474 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4e6c0041d0 con 0x7f4e7c102790 2024-09-11T23:02:23.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.007+0000 7f4e86499740 1 Processor -- start 2024-09-11T23:02:23.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.007+0000 7f4e86499740 1 -- start start 2024-09-11T23:02:23.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.007+0000 7f4e86499740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7a6b40 0x556bed7a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.007+0000 7f4e86499740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a55d0 0x556bed7a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.007+0000 7f4e86499740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7b0370 0x556bed7b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.007+0000 7f4e86499740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x556bed70b2f0 con 0x556bed7a55d0 2024-09-11T23:02:23.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.007+0000 7f4e86499740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556bed6b2cf0 con 0x556bed7b0370 2024-09-11T23:02:23.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.007+0000 7f4e86499740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556bed7072b0 con 0x556bed7a6b40 2024-09-11T23:02:23.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.007+0000 7f4e81e5c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a55d0 0x556bed7a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.035 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.007+0000 7f4e81e5c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a55d0 0x556bed7a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40662/0 (socket says 172.21.15.152:40662) 2024-09-11T23:02:23.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.007+0000 7f4e81e5c640 1 -- 172.21.15.152:0/2379694833 learned_addr learned my addr 172.21.15.152:0/2379694833 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:23.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.008+0000 7f4e8265d640 1 --2- 172.21.15.152:0/2379694833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7a6b40 0x556bed7a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.008+0000 7f4e8165b640 1 --2- 172.21.15.152:0/2379694833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7b0370 0x556bed7b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.008+0000 7f4e81e5c640 1 -- 172.21.15.152:0/2379694833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7a6b40 msgr2=0x556bed7a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:23.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.008+0000 7f4e81e5c640 1 --2- 172.21.15.152:0/2379694833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7a6b40 0x556bed7a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.008+0000 7f4e81e5c640 1 -- 172.21.15.152:0/2379694833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7b0370 msgr2=0x556bed7b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:23.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.008+0000 7f4e81e5c640 1 --2- 172.21.15.152:0/2379694833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7b0370 0x556bed7b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.008+0000 7f4e81e5c640 1 -- 172.21.15.152:0/2379694833 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bed6973a0 con 0x556bed7a55d0 2024-09-11T23:02:23.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.008+0000 7f4e72ffd640 1 --2- 172.21.15.152:0/87998474 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4e680a2d70 0x7f4e680a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.008+0000 7f4e8265d640 1 --2- 172.21.15.152:0/87998474 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4e680a2d70 0x7f4e680a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.008+0000 7f4e81e5c640 1 --2- 172.21.15.152:0/2379694833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a55d0 0x556bed7a5a10 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009900 tx=0x7f4e78031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e52bb8bb9e34ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:23.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.008+0000 7f4e72ffd640 1 -- 172.21.15.152:0/87998474 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f4e6c0ceda0 con 0x7f4e7c102790 2024-09-11T23:02:23.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.008+0000 7f4e67fff640 1 -- 172.21.15.152:0/2379694833 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78040070 con 0x556bed7a55d0 2024-09-11T23:02:23.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.008+0000 7f4e67fff640 1 -- 172.21.15.152:0/2379694833 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e78005020 con 0x556bed7a55d0 2024-09-11T23:02:23.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.008+0000 7f4e67fff640 1 -- 172.21.15.152:0/2379694833 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78035040 con 0x556bed7a55d0 2024-09-11T23:02:23.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.009+0000 7f4e86499740 1 -- 172.21.15.152:0/2379694833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a55d0 msgr2=0x556bed7a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:23.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.009+0000 7f4e8265d640 1 --2- 172.21.15.152:0/87998474 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4e680a2d70 0x7f4e680a5230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4e74005cc0 tx=0x7f4e740083f0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:23.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.009+0000 7f4e86499740 1 --2- 172.21.15.152:0/2379694833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a55d0 0x556bed7a5a10 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009900 tx=0x7f4e78031630 comp rx=0 tx=0).stop 2024-09-11T23:02:23.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.009+0000 7f4e86499740 1 -- 172.21.15.152:0/2379694833 shutdown_connections 2024-09-11T23:02:23.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.009+0000 7f4e86499740 1 --2- 172.21.15.152:0/2379694833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7b0370 0x556bed7b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.009+0000 7f4e86499740 1 --2- 172.21.15.152:0/2379694833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a55d0 0x556bed7a5a10 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.009+0000 7f4e86499740 1 --2- 172.21.15.152:0/2379694833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7a6b40 0x556bed7a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.009+0000 7f4e86499740 1 -- 172.21.15.152:0/2379694833 >> 172.21.15.152:0/2379694833 conn(0x556bed76e310 msgr2=0x556bed7a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:23.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.009+0000 7f4e86499740 1 -- 172.21.15.152:0/2379694833 shutdown_connections 2024-09-11T23:02:23.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.009+0000 7f4e86499740 1 -- 172.21.15.152:0/2379694833 wait complete. 2024-09-11T23:02:23.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.009+0000 7f4e86499740 1 -- start start 2024-09-11T23:02:23.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e86499740 1 Processor -- start 2024-09-11T23:02:23.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e86499740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7a55d0 0x556bed7afd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e86499740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a6b40 0x556bed7acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e86499740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7b0370 0x556bed7ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e86499740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x556bed7b3010 con 0x556bed7a6b40 2024-09-11T23:02:23.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e86499740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x556bed6c6060 con 0x556bed7b0370 2024-09-11T23:02:23.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e86499740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x556bed7b3190 con 0x556bed7a55d0 2024-09-11T23:02:23.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e86499740 10 client.0 _ll_register_callbacks cb 0x556bed4198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:23.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e8165b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a6b40 0x556bed7acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e8165b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a6b40 0x556bed7acfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40678/0 (socket says 172.21.15.152:40678) 2024-09-11T23:02:23.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e8165b640 1 -- 172.21.15.152:0/4004670382 learned_addr learned my addr 172.21.15.152:0/4004670382 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:23.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e8265d640 1 --2- 172.21.15.152:0/4004670382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7b0370 0x556bed7ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e81e5c640 1 --2- 172.21.15.152:0/4004670382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7a55d0 0x556bed7afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e8165b640 1 -- 172.21.15.152:0/4004670382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7a55d0 msgr2=0x556bed7afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:23.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e8165b640 1 --2- 172.21.15.152:0/4004670382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7a55d0 0x556bed7afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e8165b640 1 -- 172.21.15.152:0/4004670382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7b0370 msgr2=0x556bed7ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:23.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e8165b640 1 --2- 172.21.15.152:0/4004670382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7b0370 0x556bed7ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.010+0000 7f4e8165b640 1 -- 172.21.15.152:0/4004670382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bed47c3c0 con 0x556bed7a6b40 2024-09-11T23:02:23.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.011+0000 7f4e8165b640 1 --2- 172.21.15.152:0/4004670382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a6b40 0x556bed7acfb0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009fd0 tx=0x7f4e6c09b0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa7a21d0e6d7b58 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:23.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.011+0000 7f4e657fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:23.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.011+0000 7f4e657fa640 1 -- 172.21.15.152:0/4004670382 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c09b9e0 con 0x556bed7a6b40 2024-09-11T23:02:23.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.011+0000 7f4e657fa640 1 -- 172.21.15.152:0/4004670382 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f4e6c0bb0c0 con 0x556bed7a6b40 2024-09-11T23:02:23.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.011+0000 7f4e86499740 1 -- 172.21.15.152:0/4004670382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e78005020 con 0x556bed7a6b40 2024-09-11T23:02:23.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.011+0000 7f4e657fa640 1 -- 172.21.15.152:0/4004670382 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4e6c09b2e0 con 0x556bed7a6b40 2024-09-11T23:02:23.039 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.011+0000 7f4e86499740 10 client.18210 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:23.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.011+0000 7f4e86499740 1 -- 172.21.15.152:0/4004670382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556bed7bea40 con 0x556bed7a6b40 2024-09-11T23:02:23.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.011+0000 7f4e657fa640 1 -- 172.21.15.152:0/4004670382 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e78005020 con 0x556bed7a6b40 2024-09-11T23:02:23.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.011+0000 7f4e657fa640 1 -- 172.21.15.152:0/4004670382 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4e6c0b4120 con 0x556bed7a6b40 2024-09-11T23:02:23.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.011+0000 7f4e86499740 10 client.18210 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:23.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.011+0000 7f4e86499740 10 client.18210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:02:23.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.011+0000 7f4e86499740 20 client.18210 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:23.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.011+0000 7f4e86499740 1 --2- 172.21.15.152:0/4004670382 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556bed7bff00 0x556bed7c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.011+0000 7f4e86499740 4 client.18210 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:02:23.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.011+0000 7f4e86499740 1 -- 172.21.15.152:0/4004670382 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bed718860 con 0x556bed7bff00 2024-09-11T23:02:23.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.012+0000 7f4e81e5c640 1 --2- 172.21.15.152:0/4004670382 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556bed7bff00 0x556bed7c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.012+0000 7f4e81e5c640 1 --2- 172.21.15.152:0/4004670382 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556bed7bff00 0x556bed7c22f0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe93659083982489 server_cookie=b75503f57f9c8eb5 in_seq=0 out_seq=0 2024-09-11T23:02:23.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.012+0000 7f4e657fa640 10 client.18210 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:23.040 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.013+0000 7f4e657fa640 1 -- 172.21.15.152:0/4004670382 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556bed718860 con 0x556bed7bff00 2024-09-11T23:02:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.013+0000 7f4e657fa640 10 client.18210 handle_command_reply: tid=0 2024-09-11T23:02:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.013+0000 7f4e86499740 1 client.18210 shutdown 2024-09-11T23:02:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.013+0000 7f4e86499740 1 -- 172.21.15.152:0/4004670382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a6b40 msgr2=0x556bed7acfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.013+0000 7f4e86499740 1 --2- 172.21.15.152:0/4004670382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a6b40 0x556bed7acfb0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009fd0 tx=0x7f4e6c09b0b0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.013+0000 7f4e86499740 1 -- 172.21.15.152:0/4004670382 shutdown_connections 2024-09-11T23:02:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.013+0000 7f4e86499740 1 --2- 172.21.15.152:0/4004670382 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556bed7bff00 0x556bed7c22f0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.013+0000 7f4e86499740 1 --2- 172.21.15.152:0/4004670382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7b0370 0x556bed7ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.013+0000 7f4e86499740 1 --2- 172.21.15.152:0/4004670382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a6b40 0x556bed7acfb0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.013+0000 7f4e657fa640 0 client.18210 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.013+0000 7f4e86499740 1 --2- 172.21.15.152:0/4004670382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7a55d0 0x556bed7afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.013+0000 7f4e86499740 1 -- 172.21.15.152:0/4004670382 >> 172.21.15.152:0/4004670382 conn(0x556bed76e310 msgr2=0x556bed7a7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.013+0000 7f4e86499740 1 -- 172.21.15.152:0/4004670382 shutdown_connections 2024-09-11T23:02:23.041 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.013+0000 7f4e86499740 1 -- 172.21.15.152:0/4004670382 wait complete. 2024-09-11T23:02:23.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.013+0000 7f4e86499740 20 client.18210 trim_cache size 0 max 16384 2024-09-11T23:02:23.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e86499740 1 Processor -- start 2024-09-11T23:02:23.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e86499740 1 -- start start 2024-09-11T23:02:23.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e86499740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7a55d0 0x556bed7a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e86499740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a6b40 0x556bed7bc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e86499740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7b0370 0x556bed7c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e86499740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1ad760 con 0x556bed7a6b40 2024-09-11T23:02:23.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e86499740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1af2a0 con 0x556bed7a55d0 2024-09-11T23:02:23.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e86499740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1ade30 con 0x556bed7b0370 2024-09-11T23:02:23.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e8165b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a6b40 0x556bed7bc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e8165b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a6b40 0x556bed7bc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40688/0 (socket says 172.21.15.152:40688) 2024-09-11T23:02:23.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e8165b640 1 -- 172.21.15.152:0/4218427230 learned_addr learned my addr 172.21.15.152:0/4218427230 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:23.042 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e8265d640 1 --2- 172.21.15.152:0/4218427230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7b0370 0x556bed7c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e81e5c640 1 --2- 172.21.15.152:0/4218427230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7a55d0 0x556bed7a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e8165b640 1 -- 172.21.15.152:0/4218427230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7b0370 msgr2=0x556bed7c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:23.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e8165b640 1 --2- 172.21.15.152:0/4218427230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7b0370 0x556bed7c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e8165b640 1 -- 172.21.15.152:0/4218427230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7a55d0 msgr2=0x556bed7a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:23.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e8165b640 1 --2- 172.21.15.152:0/4218427230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7a55d0 0x556bed7a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e8165b640 1 -- 172.21.15.152:0/4218427230 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bed7a8120 con 0x556bed7a6b40 2024-09-11T23:02:23.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e8265d640 1 --2- 172.21.15.152:0/4218427230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7b0370 0x556bed7c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:23.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.026+0000 7f4e81e5c640 1 --2- 172.21.15.152:0/4218427230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7a55d0 0x556bed7a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:23.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.027+0000 7f4e8165b640 1 --2- 172.21.15.152:0/4218427230 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a6b40 0x556bed7bc2b0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c097740 tx=0x7f4e6c09b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8f3dee041b7e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:23.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.027+0000 7f4e677fe640 1 -- 172.21.15.152:0/4218427230 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c09b9e0 con 0x556bed7a6b40 2024-09-11T23:02:23.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.027+0000 7f4e677fe640 1 -- 172.21.15.152:0/4218427230 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6c095e50 con 0x556bed7a6b40 2024-09-11T23:02:23.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.027+0000 7f4e677fe640 1 -- 172.21.15.152:0/4218427230 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c0bb0c0 con 0x556bed7a6b40 2024-09-11T23:02:23.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.027+0000 7f4e86499740 1 -- 172.21.15.152:0/4218427230 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a6b40 msgr2=0x556bed7bc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:23.043 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.027+0000 7f4e86499740 1 --2- 172.21.15.152:0/4218427230 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a6b40 0x556bed7bc2b0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c097740 tx=0x7f4e6c09b1e0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.028+0000 7f4e86499740 1 -- 172.21.15.152:0/4218427230 shutdown_connections 2024-09-11T23:02:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.028+0000 7f4e86499740 1 --2- 172.21.15.152:0/4218427230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7b0370 0x556bed7c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.028+0000 7f4e86499740 1 --2- 172.21.15.152:0/4218427230 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a6b40 0x556bed7bc2b0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.028+0000 7f4e86499740 1 --2- 172.21.15.152:0/4218427230 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7a55d0 0x556bed7a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.028+0000 7f4e86499740 1 -- 172.21.15.152:0/4218427230 >> 172.21.15.152:0/4218427230 conn(0x556bed76e310 msgr2=0x556bed7a1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.028+0000 7f4e86499740 1 -- 172.21.15.152:0/4218427230 shutdown_connections 2024-09-11T23:02:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.028+0000 7f4e86499740 1 -- 172.21.15.152:0/4218427230 wait complete. 2024-09-11T23:02:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.028+0000 7f4e86499740 1 -- start start 2024-09-11T23:02:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.028+0000 7f4e86499740 1 Processor -- start 2024-09-11T23:02:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.028+0000 7f4e86499740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a55d0 0x556bed7b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.028+0000 7f4e86499740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7a6b40 0x556bed7b4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.028+0000 7f4e86499740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7b0370 0x556bed7b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.028+0000 7f4e86499740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1aebd0 con 0x556bed7a55d0 2024-09-11T23:02:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.028+0000 7f4e86499740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1ae500 con 0x556bed7a6b40 2024-09-11T23:02:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.028+0000 7f4e86499740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1ab760 con 0x556bed7b0370 2024-09-11T23:02:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.028+0000 7f4e86499740 10 client.0 _ll_register_callbacks cb 0x7f4e6c005130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e81e5c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a55d0 0x556bed7b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e81e5c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a55d0 0x556bed7b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40698/0 (socket says 172.21.15.152:40698) 2024-09-11T23:02:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e81e5c640 1 -- 172.21.15.152:0/2032317772 learned_addr learned my addr 172.21.15.152:0/2032317772 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e8265d640 1 --2- 172.21.15.152:0/2032317772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7b0370 0x556bed7b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e8165b640 1 --2- 172.21.15.152:0/2032317772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7a6b40 0x556bed7b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e81e5c640 1 -- 172.21.15.152:0/2032317772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7b0370 msgr2=0x556bed7b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e81e5c640 1 --2- 172.21.15.152:0/2032317772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7b0370 0x556bed7b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e81e5c640 1 -- 172.21.15.152:0/2032317772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7a6b40 msgr2=0x556bed7b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:23.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e81e5c640 1 --2- 172.21.15.152:0/2032317772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7a6b40 0x556bed7b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e81e5c640 1 -- 172.21.15.152:0/2032317772 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bed829d40 con 0x556bed7a55d0 2024-09-11T23:02:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e8165b640 1 --2- 172.21.15.152:0/2032317772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7a6b40 0x556bed7b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e8265d640 1 --2- 172.21.15.152:0/2032317772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7b0370 0x556bed7b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e81e5c640 1 --2- 172.21.15.152:0/2032317772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a55d0 0x556bed7b4410 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009850 tx=0x7f4e7802ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d1a0a409cf2212 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e657fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e657fa640 1 -- 172.21.15.152:0/2032317772 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78052020 con 0x556bed7a55d0 2024-09-11T23:02:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e86499740 1 -- 172.21.15.152:0/2032317772 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e6c095e50 con 0x556bed7a55d0 2024-09-11T23:02:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.029+0000 7f4e657fa640 1 -- 172.21.15.152:0/2032317772 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f4e78004040 con 0x556bed7a55d0 2024-09-11T23:02:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.030+0000 7f4e657fa640 1 -- 172.21.15.152:0/2032317772 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4e7805d050 con 0x556bed7a55d0 2024-09-11T23:02:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.030+0000 7f4e86499740 10 client.18216 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.030+0000 7f4e86499740 1 -- 172.21.15.152:0/2032317772 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556bed82af70 con 0x556bed7a55d0 2024-09-11T23:02:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.030+0000 7f4e657fa640 1 -- 172.21.15.152:0/2032317772 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6c095e50 con 0x556bed7a55d0 2024-09-11T23:02:23.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.030+0000 7f4e657fa640 1 -- 172.21.15.152:0/2032317772 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4e7804e3b0 con 0x556bed7a55d0 2024-09-11T23:02:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.030+0000 7f4e86499740 10 client.18216 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.030+0000 7f4e86499740 10 client.18216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:02:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.030+0000 7f4e86499740 20 client.18216 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.030+0000 7f4e86499740 1 --2- 172.21.15.152:0/2032317772 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556bed7bff00 0x556bed82d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.030+0000 7f4e86499740 4 client.18216 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:02:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.030+0000 7f4e86499740 1 -- 172.21.15.152:0/2032317772 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556bed712d70 con 0x556bed7bff00 2024-09-11T23:02:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.030+0000 7f4e8165b640 1 --2- 172.21.15.152:0/2032317772 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556bed7bff00 0x556bed82d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.031+0000 7f4e657fa640 10 client.18216 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.031+0000 7f4e8165b640 1 --2- 172.21.15.152:0/2032317772 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556bed7bff00 0x556bed82d210 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc0aca9260cb5a9 server_cookie=b79d569116ca0272 in_seq=0 out_seq=0 2024-09-11T23:02:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.031+0000 7f4e657fa640 1 -- 172.21.15.152:0/2032317772 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556bed712d70 con 0x556bed7bff00 2024-09-11T23:02:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.031+0000 7f4e657fa640 10 client.18216 handle_command_reply: tid=0 2024-09-11T23:02:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.031+0000 7f4e86499740 1 client.18216 shutdown 2024-09-11T23:02:23.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.031+0000 7f4e86499740 1 -- 172.21.15.152:0/2032317772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a55d0 msgr2=0x556bed7b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.031+0000 7f4e86499740 1 --2- 172.21.15.152:0/2032317772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a55d0 0x556bed7b4410 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009850 tx=0x7f4e7802ab10 comp rx=0 tx=0).stop 2024-09-11T23:02:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.031+0000 7f4e86499740 1 -- 172.21.15.152:0/2032317772 shutdown_connections 2024-09-11T23:02:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.031+0000 7f4e86499740 1 --2- 172.21.15.152:0/2032317772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556bed7b0370 0x556bed7b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.031+0000 7f4e86499740 1 --2- 172.21.15.152:0/2032317772 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556bed7bff00 0x556bed82d210 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.031+0000 7f4e86499740 1 --2- 172.21.15.152:0/2032317772 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x556bed7a6b40 0x556bed7b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.031+0000 7f4e86499740 1 --2- 172.21.15.152:0/2032317772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556bed7a55d0 0x556bed7b4410 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.031+0000 7f4e657fa640 0 client.18216 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.031+0000 7f4e86499740 1 -- 172.21.15.152:0/2032317772 >> 172.21.15.152:0/2032317772 conn(0x556bed76e310 msgr2=0x556bed7ae810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.031+0000 7f4e86499740 1 -- 172.21.15.152:0/2032317772 shutdown_connections 2024-09-11T23:02:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.031+0000 7f4e86499740 1 -- 172.21.15.152:0/2032317772 wait complete. 2024-09-11T23:02:23.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.032+0000 7f4e86499740 20 client.18216 trim_cache size 0 max 16384 2024-09-11T23:02:23.049 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:02:23.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.032+0000 7f4e840c0640 1 -- 172.21.15.152:0/87998474 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4e680a2d70 msgr2=0x7f4e680a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.032+0000 7f4e840c0640 1 --2- 172.21.15.152:0/87998474 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4e680a2d70 0x7f4e680a5230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4e74005cc0 tx=0x7f4e740083f0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.032+0000 7f4e840c0640 1 -- 172.21.15.152:0/87998474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e7c102790 msgr2=0x7f4e7c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.032+0000 7f4e840c0640 1 --2- 172.21.15.152:0/87998474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e7c102790 0x7f4e7c198970 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00d950 tx=0x7f4e6c00de10 comp rx=0 tx=0).stop 2024-09-11T23:02:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.032+0000 7f4e840c0640 1 -- 172.21.15.152:0/87998474 shutdown_connections 2024-09-11T23:02:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.032+0000 7f4e840c0640 1 --2- 172.21.15.152:0/87998474 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4e680a2d70 0x7f4e680a5230 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.032+0000 7f4e840c0640 1 --2- 172.21.15.152:0/87998474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e7c106f10 0x7f4e7c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.032+0000 7f4e840c0640 1 --2- 172.21.15.152:0/87998474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e7c102790 0x7f4e7c198970 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.032+0000 7f4e840c0640 1 --2- 172.21.15.152:0/87998474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e7c101de0 0x7f4e7c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.032+0000 7f4e840c0640 1 -- 172.21.15.152:0/87998474 >> 172.21.15.152:0/87998474 conn(0x7f4e7c0fd8c0 msgr2=0x7f4e7c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.032+0000 7f4e840c0640 1 -- 172.21.15.152:0/87998474 shutdown_connections 2024-09-11T23:02:23.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:23.032+0000 7f4e840c0640 1 -- 172.21.15.152:0/87998474 wait complete. 2024-09-11T23:02:23.051 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:22 smithi204 ceph-mon[35454]: pgmap v293: 97 pgs: 97 active+clean; 4.0 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:02:23.051 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:22 smithi191 ceph-mon[35211]: pgmap v293: 97 pgs: 97 active+clean; 4.0 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:02:24.738 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T23:02:24.739 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T23:02:25.000 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:25.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:24 smithi191 ceph-mon[35211]: pgmap v294: 97 pgs: 97 active+clean; 4.0 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:02:25.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:24 smithi204 ceph-mon[35454]: pgmap v294: 97 pgs: 97 active+clean; 4.0 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:02:25.120 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:24 smithi152 ceph-mon[31181]: pgmap v294: 97 pgs: 97 active+clean; 4.0 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-11T23:02:25.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.539+0000 7fd3fa21d640 1 -- 172.21.15.152:0/978128327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f4102790 msgr2=0x7fd3f41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:25.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.539+0000 7fd3fa21d640 1 --2- 172.21.15.152:0/978128327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f4102790 0x7fd3f41067e0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009900 tx=0x7fd3e802a410 comp rx=0 tx=0).stop 2024-09-11T23:02:25.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.539+0000 7fd3fa21d640 1 -- 172.21.15.152:0/978128327 shutdown_connections 2024-09-11T23:02:25.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.539+0000 7fd3fa21d640 1 --2- 172.21.15.152:0/978128327 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3f4106f10 0x7fd3f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.539+0000 7fd3fa21d640 1 --2- 172.21.15.152:0/978128327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f4102790 0x7fd3f41067e0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.539+0000 7fd3fa21d640 1 --2- 172.21.15.152:0/978128327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3f4101de0 0x7fd3f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.539+0000 7fd3fa21d640 1 -- 172.21.15.152:0/978128327 >> 172.21.15.152:0/978128327 conn(0x7fd3f40fd8c0 msgr2=0x7fd3f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:25.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.539+0000 7fd3fa21d640 1 -- 172.21.15.152:0/978128327 shutdown_connections 2024-09-11T23:02:25.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.539+0000 7fd3fa21d640 1 -- 172.21.15.152:0/978128327 wait complete. 2024-09-11T23:02:25.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.540+0000 7fd3fa21d640 1 Processor -- start 2024-09-11T23:02:25.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.540+0000 7fd3fa21d640 1 -- start start 2024-09-11T23:02:25.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.540+0000 7fd3fa21d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3f4101de0 0x7fd3f4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.540+0000 7fd3fa21d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3f4102790 0x7fd3f4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.540+0000 7fd3fa21d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f4106f10 0x7fd3f419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.540+0000 7fd3fa21d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4111f70 con 0x7fd3f4106f10 2024-09-11T23:02:25.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.540+0000 7fd3fa21d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4111df0 con 0x7fd3f4101de0 2024-09-11T23:02:25.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.540+0000 7fd3fa21d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41120f0 con 0x7fd3f4102790 2024-09-11T23:02:25.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.540+0000 7fd3f3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f4106f10 0x7fd3f419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.540+0000 7fd3f3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f4106f10 0x7fd3f419cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40720/0 (socket says 172.21.15.152:40720) 2024-09-11T23:02:25.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.540+0000 7fd3f3fff640 1 -- 172.21.15.152:0/867815005 learned_addr learned my addr 172.21.15.152:0/867815005 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:25.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.540+0000 7fd3f37fe640 1 --2- 172.21.15.152:0/867815005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3f4101de0 0x7fd3f4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.540+0000 7fd3f2ffd640 1 --2- 172.21.15.152:0/867815005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3f4102790 0x7fd3f4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.558 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.541+0000 7fd3f3fff640 1 -- 172.21.15.152:0/867815005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3f4102790 msgr2=0x7fd3f4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:25.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.541+0000 7fd3f3fff640 1 --2- 172.21.15.152:0/867815005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3f4102790 0x7fd3f4198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.541+0000 7fd3f3fff640 1 -- 172.21.15.152:0/867815005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3f4101de0 msgr2=0x7fd3f4198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:25.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.541+0000 7fd3f3fff640 1 --2- 172.21.15.152:0/867815005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3f4101de0 0x7fd3f4198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.541+0000 7fd3f3fff640 1 -- 172.21.15.152:0/867815005 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f419d4c0 con 0x7fd3f4106f10 2024-09-11T23:02:25.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.541+0000 7fd3f3fff640 1 --2- 172.21.15.152:0/867815005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f4106f10 0x7fd3f419cde0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00cad0 tx=0x7fd3ec00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:25.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.541+0000 7fd3f0ff9640 1 -- 172.21.15.152:0/867815005 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec00ae20 con 0x7fd3f4106f10 2024-09-11T23:02:25.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.541+0000 7fd3f0ff9640 1 -- 172.21.15.152:0/867815005 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ec005020 con 0x7fd3f4106f10 2024-09-11T23:02:25.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.541+0000 7fd3fa21d640 1 -- 172.21.15.152:0/867815005 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f419d7b0 con 0x7fd3f4106f10 2024-09-11T23:02:25.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.541+0000 7fd3f0ff9640 1 -- 172.21.15.152:0/867815005 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec00f6f0 con 0x7fd3f4106f10 2024-09-11T23:02:25.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.541+0000 7fd3fa21d640 1 -- 172.21.15.152:0/867815005 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f41a5090 con 0x7fd3f4106f10 2024-09-11T23:02:25.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.543+0000 7fd3f0ff9640 1 -- 172.21.15.152:0/867815005 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd3ec0051d0 con 0x7fd3f4106f10 2024-09-11T23:02:25.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.544+0000 7fd3fc5f6740 1 Processor -- start 2024-09-11T23:02:25.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.544+0000 7fd3f0ff9640 1 --2- 172.21.15.152:0/867815005 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd3dc0a2cf0 0x7fd3dc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.559 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.544+0000 7fd3fc5f6740 1 -- start start 2024-09-11T23:02:25.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.544+0000 7fd3f2ffd640 1 --2- 172.21.15.152:0/867815005 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd3dc0a2cf0 0x7fd3dc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.544+0000 7fd3fc5f6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 0x5563393cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.544+0000 7fd3fc5f6740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393ce480 0x5563393d8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.544+0000 7fd3f0ff9640 1 -- 172.21.15.152:0/867815005 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7fd3ec0cf510 con 0x7fd3f4106f10 2024-09-11T23:02:25.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.544+0000 7fd3fc5f6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393d8d40 0x5563393db130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.544+0000 7fd3fc5f6740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x556339333280 con 0x5563393ce480 2024-09-11T23:02:25.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.544+0000 7fd3fc5f6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5563392dab10 con 0x5563393d8d40 2024-09-11T23:02:25.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.544+0000 7fd3fc5f6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55633932f240 con 0x5563393cdb60 2024-09-11T23:02:25.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.544+0000 7fd3f2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393ce480 0x5563393d8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.544+0000 7fd3f2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393ce480 0x5563393d8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40732/0 (socket says 172.21.15.152:40732) 2024-09-11T23:02:25.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.544+0000 7fd3f2ffd640 1 -- 172.21.15.152:0/2818078917 learned_addr learned my addr 172.21.15.152:0/2818078917 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:25.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.544+0000 7fd3f3fff640 1 --2- 172.21.15.152:0/2818078917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393d8d40 0x5563393db130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.560 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.544+0000 7fd3f37fe640 1 --2- 172.21.15.152:0/2818078917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 0x5563393cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.545+0000 7fd3f2ffd640 1 --2- 172.21.15.152:0/867815005 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd3dc0a2cf0 0x7fd3dc0a51b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8005ab0 tx=0x7fd3e8031690 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:25.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.545+0000 7fd3f3fff640 1 -- 172.21.15.152:0/2818078917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 msgr2=0x5563393cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:25.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.545+0000 7fd3f3fff640 1 --2- 172.21.15.152:0/2818078917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 0x5563393cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.545+0000 7fd3f3fff640 1 -- 172.21.15.152:0/2818078917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393ce480 msgr2=0x5563393d8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:25.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.545+0000 7fd3f3fff640 1 --2- 172.21.15.152:0/2818078917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393ce480 0x5563393d8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.545+0000 7fd3f3fff640 1 -- 172.21.15.152:0/2818078917 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563392e71f0 con 0x5563393d8d40 2024-09-11T23:02:25.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.545+0000 7fd3f2ffd640 1 --2- 172.21.15.152:0/2818078917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393ce480 0x5563393d8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:02:25.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.545+0000 7fd3f3fff640 1 --2- 172.21.15.152:0/2818078917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393d8d40 0x5563393db130 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0ccc80 tx=0x7fd3ec09b6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c33daeb90040853c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:25.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.545+0000 7fd3d9ffb640 1 -- 172.21.15.152:0/2818078917 <== mon.2 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec01d040 con 0x5563393d8d40 2024-09-11T23:02:25.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.545+0000 7fd3d9ffb640 1 -- 172.21.15.152:0/2818078917 <== mon.2 v2:172.21.15.191:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ec005560 con 0x5563393d8d40 2024-09-11T23:02:25.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.545+0000 7fd3d9ffb640 1 -- 172.21.15.152:0/2818078917 <== mon.2 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec0cf2a0 con 0x5563393d8d40 2024-09-11T23:02:25.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.546+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/2818078917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393d8d40 msgr2=0x5563393db130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:25.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.546+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/2818078917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393d8d40 0x5563393db130 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0ccc80 tx=0x7fd3ec09b6c0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.546+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/2818078917 shutdown_connections 2024-09-11T23:02:25.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.546+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/2818078917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393d8d40 0x5563393db130 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.546+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/2818078917 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393ce480 0x5563393d8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.546+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/2818078917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 0x5563393cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.546+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/2818078917 >> 172.21.15.152:0/2818078917 conn(0x5563393ca0f0 msgr2=0x5563393ca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:25.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.546+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/2818078917 shutdown_connections 2024-09-11T23:02:25.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.546+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/2818078917 wait complete. 2024-09-11T23:02:25.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.546+0000 7fd3fc5f6740 1 -- start start 2024-09-11T23:02:25.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.546+0000 7fd3fc5f6740 1 Processor -- start 2024-09-11T23:02:25.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.546+0000 7fd3fc5f6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 0x5563393d5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.546+0000 7fd3fc5f6740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393ce480 0x5563393d6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3fc5f6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393d8d40 0x5563393d66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3fc5f6740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5563393db910 con 0x5563393ce480 2024-09-11T23:02:25.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3fc5f6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5563392edf10 con 0x5563393d8d40 2024-09-11T23:02:25.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3fc5f6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5563393dba90 con 0x5563393cdb60 2024-09-11T23:02:25.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3fc5f6740 10 client.0 _ll_register_callbacks cb 0x5563392b6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:25.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3f2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393ce480 0x5563393d6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3f2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393ce480 0x5563393d6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40734/0 (socket says 172.21.15.152:40734) 2024-09-11T23:02:25.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3f2ffd640 1 -- 172.21.15.152:0/3686661347 learned_addr learned my addr 172.21.15.152:0/3686661347 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:25.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3f3fff640 1 --2- 172.21.15.152:0/3686661347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393d8d40 0x5563393d66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3f37fe640 1 --2- 172.21.15.152:0/3686661347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 0x5563393d5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3f2ffd640 1 -- 172.21.15.152:0/3686661347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 msgr2=0x5563393d5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:25.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3f2ffd640 1 --2- 172.21.15.152:0/3686661347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 0x5563393d5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3f2ffd640 1 -- 172.21.15.152:0/3686661347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393d8d40 msgr2=0x5563393d66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:25.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3f2ffd640 1 --2- 172.21.15.152:0/3686661347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393d8d40 0x5563393d66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3f2ffd640 1 -- 172.21.15.152:0/3686661347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563390a46a0 con 0x5563393ce480 2024-09-11T23:02:25.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3f2ffd640 1 --2- 172.21.15.152:0/3686661347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393ce480 0x5563393d6190 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009230 tx=0x7fd3e8043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e272f47d485a5d8c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:25.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:25.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3d2ffd640 1 -- 172.21.15.152:0/3686661347 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8035050 con 0x5563393ce480 2024-09-11T23:02:25.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3d2ffd640 1 -- 172.21.15.152:0/3686661347 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7fd3e8005450 con 0x5563393ce480 2024-09-11T23:02:25.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/3686661347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3ec005560 con 0x5563393ce480 2024-09-11T23:02:25.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3d2ffd640 1 -- 172.21.15.152:0/3686661347 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd3e8005650 con 0x5563393ce480 2024-09-11T23:02:25.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3fc5f6740 10 client.18228 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:25.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.547+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/3686661347 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5563393e7540 con 0x5563393ce480 2024-09-11T23:02:25.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.548+0000 7fd3d2ffd640 1 -- 172.21.15.152:0/3686661347 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ec005560 con 0x5563393ce480 2024-09-11T23:02:25.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.548+0000 7fd3d2ffd640 1 -- 172.21.15.152:0/3686661347 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd3e8058020 con 0x5563393ce480 2024-09-11T23:02:25.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.548+0000 7fd3fc5f6740 10 client.18228 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:25.564 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.548+0000 7fd3fc5f6740 10 client.18228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:02:25.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.548+0000 7fd3fc5f6740 20 client.18228 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:25.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.548+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/3686661347 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5563393e8950 0x5563393ead80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.548+0000 7fd3fc5f6740 4 client.18228 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:02:25.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.548+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/3686661347 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563393407f0 con 0x5563393e8950 2024-09-11T23:02:25.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.548+0000 7fd3f37fe640 1 --2- 172.21.15.152:0/3686661347 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5563393e8950 0x5563393ead80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.549+0000 7fd3d2ffd640 10 client.18228 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:25.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.549+0000 7fd3f37fe640 1 --2- 172.21.15.152:0/3686661347 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5563393e8950 0x5563393ead80 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=374b4e8384f2b887 server_cookie=29cafc20f3b272b5 in_seq=0 out_seq=0 2024-09-11T23:02:25.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.549+0000 7fd3d2ffd640 1 -- 172.21.15.152:0/3686661347 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563393407f0 con 0x5563393e8950 2024-09-11T23:02:25.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.549+0000 7fd3d2ffd640 10 client.18228 handle_command_reply: tid=0 2024-09-11T23:02:25.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.549+0000 7fd3fc5f6740 1 client.18228 shutdown 2024-09-11T23:02:25.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.550+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/3686661347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393ce480 msgr2=0x5563393d6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:25.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.550+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/3686661347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393ce480 0x5563393d6190 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009230 tx=0x7fd3e8043000 comp rx=0 tx=0).stop 2024-09-11T23:02:25.565 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.550+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/3686661347 shutdown_connections 2024-09-11T23:02:25.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.550+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/3686661347 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5563393e8950 0x5563393ead80 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.550+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/3686661347 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393d8d40 0x5563393d66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.550+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/3686661347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393ce480 0x5563393d6190 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.550+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/3686661347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 0x5563393d5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.550+0000 7fd3d2ffd640 0 client.18228 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:25.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.550+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/3686661347 >> 172.21.15.152:0/3686661347 conn(0x5563393ca0f0 msgr2=0x5563393cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:25.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.550+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/3686661347 shutdown_connections 2024-09-11T23:02:25.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.550+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/3686661347 wait complete. 2024-09-11T23:02:25.566 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.550+0000 7fd3fc5f6740 20 client.18228 trim_cache size 0 max 16384 2024-09-11T23:02:25.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.567+0000 7fd3fc5f6740 1 Processor -- start 2024-09-11T23:02:25.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.567+0000 7fd3fc5f6740 1 -- start start 2024-09-11T23:02:25.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.567+0000 7fd3fc5f6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 0x5563393c9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.567+0000 7fd3fc5f6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393ce480 0x5563393f0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.567+0000 7fd3fc5f6740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393d8d40 0x5563393f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.567+0000 7fd3fc5f6740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41ad910 con 0x5563393d8d40 2024-09-11T23:02:25.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.567+0000 7fd3fc5f6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41af450 con 0x5563393ce480 2024-09-11T23:02:25.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.567+0000 7fd3fc5f6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41adfe0 con 0x5563393cdb60 2024-09-11T23:02:25.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.567+0000 7fd3f3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393d8d40 0x5563393f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.567+0000 7fd3f3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393d8d40 0x5563393f0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40744/0 (socket says 172.21.15.152:40744) 2024-09-11T23:02:25.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.567+0000 7fd3f3fff640 1 -- 172.21.15.152:0/2985599833 learned_addr learned my addr 172.21.15.152:0/2985599833 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:25.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.568+0000 7fd3f2ffd640 1 --2- 172.21.15.152:0/2985599833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393ce480 0x5563393f0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.568+0000 7fd3f37fe640 1 --2- 172.21.15.152:0/2985599833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 0x5563393c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.568+0000 7fd3f3fff640 1 -- 172.21.15.152:0/2985599833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 msgr2=0x5563393c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:25.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.568+0000 7fd3f3fff640 1 --2- 172.21.15.152:0/2985599833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 0x5563393c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.568+0000 7fd3f3fff640 1 -- 172.21.15.152:0/2985599833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393ce480 msgr2=0x5563393f0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:25.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.568+0000 7fd3f3fff640 1 --2- 172.21.15.152:0/2985599833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393ce480 0x5563393f0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.568+0000 7fd3f3fff640 1 -- 172.21.15.152:0/2985599833 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563393d4c00 con 0x5563393d8d40 2024-09-11T23:02:25.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.568+0000 7fd3f2ffd640 1 --2- 172.21.15.152:0/2985599833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393ce480 0x5563393f0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:25.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.568+0000 7fd3f3fff640 1 --2- 172.21.15.152:0/2985599833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393d8d40 0x5563393f0b90 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec09b9e0 tx=0x7fd3ec09bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2333fe52482c211c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:25.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.568+0000 7fd3d97fa640 1 -- 172.21.15.152:0/2985599833 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec01d040 con 0x5563393d8d40 2024-09-11T23:02:25.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.568+0000 7fd3d97fa640 1 -- 172.21.15.152:0/2985599833 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ec0ccf00 con 0x5563393d8d40 2024-09-11T23:02:25.570 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.568+0000 7fd3d97fa640 1 -- 172.21.15.152:0/2985599833 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec0977a0 con 0x5563393d8d40 2024-09-11T23:02:25.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.569+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/2985599833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393d8d40 msgr2=0x5563393f0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:25.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.569+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/2985599833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393d8d40 0x5563393f0b90 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec09b9e0 tx=0x7fd3ec09bbd0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.569+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/2985599833 shutdown_connections 2024-09-11T23:02:25.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.569+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/2985599833 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393d8d40 0x5563393f0b90 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.569+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/2985599833 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393ce480 0x5563393f0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.569+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/2985599833 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 0x5563393c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.574 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.569+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/2985599833 >> 172.21.15.152:0/2985599833 conn(0x5563393ca0f0 msgr2=0x5563393c8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:25.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.569+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/2985599833 shutdown_connections 2024-09-11T23:02:25.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.569+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/2985599833 wait complete. 2024-09-11T23:02:25.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.569+0000 7fd3fc5f6740 1 -- start start 2024-09-11T23:02:25.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.569+0000 7fd3fc5f6740 1 Processor -- start 2024-09-11T23:02:25.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3fc5f6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 0x5563393dce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3fc5f6740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393ce480 0x5563393dd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3fc5f6740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393d8d40 0x556339452620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3fc5f6740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41aed80 con 0x5563393d8d40 2024-09-11T23:02:25.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3fc5f6740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41ae6b0 con 0x5563393ce480 2024-09-11T23:02:25.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3fc5f6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41ab910 con 0x5563393cdb60 2024-09-11T23:02:25.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3fc5f6740 10 client.0 _ll_register_callbacks cb 0x7fd3e8005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:25.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3f3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393d8d40 0x556339452620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.575 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3f3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393d8d40 0x556339452620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40758/0 (socket says 172.21.15.152:40758) 2024-09-11T23:02:25.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3f3fff640 1 -- 172.21.15.152:0/3205304952 learned_addr learned my addr 172.21.15.152:0/3205304952 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:25.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3f37fe640 1 --2- 172.21.15.152:0/3205304952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 0x5563393dce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3f2ffd640 1 --2- 172.21.15.152:0/3205304952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393ce480 0x5563393dd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3f3fff640 1 -- 172.21.15.152:0/3205304952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 msgr2=0x5563393dce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:25.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3f3fff640 1 --2- 172.21.15.152:0/3205304952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 0x5563393dce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3f3fff640 1 -- 172.21.15.152:0/3205304952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393ce480 msgr2=0x5563393dd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:25.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3f3fff640 1 --2- 172.21.15.152:0/3205304952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393ce480 0x5563393dd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3f3fff640 1 -- 172.21.15.152:0/3205304952 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563393ddc30 con 0x5563393d8d40 2024-09-11T23:02:25.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3f3fff640 1 --2- 172.21.15.152:0/3205304952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393d8d40 0x556339452620 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec098420 tx=0x7fd3ec097db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b03d307c43da4c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:25.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:25.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3d2ffd640 1 -- 172.21.15.152:0/3205304952 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec01d030 con 0x5563393d8d40 2024-09-11T23:02:25.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/3205304952 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3ec0ccf00 con 0x5563393d8d40 2024-09-11T23:02:25.576 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.570+0000 7fd3d2ffd640 1 -- 172.21.15.152:0/3205304952 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7fd3ec0ba2b0 con 0x5563393d8d40 2024-09-11T23:02:25.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.571+0000 7fd3d2ffd640 1 -- 172.21.15.152:0/3205304952 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd3ec09b1c0 con 0x5563393d8d40 2024-09-11T23:02:25.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.571+0000 7fd3fc5f6740 10 client.18234 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:25.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.571+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/3205304952 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556339453cb0 con 0x5563393d8d40 2024-09-11T23:02:25.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.571+0000 7fd3d2ffd640 1 -- 172.21.15.152:0/3205304952 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ec0ccf00 con 0x5563393d8d40 2024-09-11T23:02:25.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.571+0000 7fd3d2ffd640 1 -- 172.21.15.152:0/3205304952 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd3ec09a910 con 0x5563393d8d40 2024-09-11T23:02:25.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.571+0000 7fd3fc5f6740 10 client.18234 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:25.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.571+0000 7fd3fc5f6740 10 client.18234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:02:25.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.571+0000 7fd3fc5f6740 20 client.18234 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:25.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.571+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/3205304952 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556339453cb0 0x5563394560a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:25.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.571+0000 7fd3fc5f6740 4 client.18234 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T23:02:25.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.571+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/3205304952 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55633933ad00 con 0x556339453cb0 2024-09-11T23:02:25.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.571+0000 7fd3f37fe640 1 --2- 172.21.15.152:0/3205304952 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556339453cb0 0x5563394560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:25.577 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.572+0000 7fd3d2ffd640 10 client.18234 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:25.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.572+0000 7fd3f37fe640 1 --2- 172.21.15.152:0/3205304952 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556339453cb0 0x5563394560a0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34e89bb9d2edeb56 server_cookie=f41668ce061f820e in_seq=0 out_seq=0 2024-09-11T23:02:25.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.573+0000 7fd3d2ffd640 1 -- 172.21.15.152:0/3205304952 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55633933ad00 con 0x556339453cb0 2024-09-11T23:02:25.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.573+0000 7fd3d2ffd640 10 client.18234 handle_command_reply: tid=0 2024-09-11T23:02:25.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.573+0000 7fd3fc5f6740 1 client.18234 shutdown 2024-09-11T23:02:25.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.573+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/3205304952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393d8d40 msgr2=0x556339452620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:25.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.573+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/3205304952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393d8d40 0x556339452620 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec098420 tx=0x7fd3ec097db0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.573+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/3205304952 shutdown_connections 2024-09-11T23:02:25.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.573+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/3205304952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5563393d8d40 0x556339452620 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.578 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.573+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/3205304952 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x556339453cb0 0x5563394560a0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.573+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/3205304952 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5563393ce480 0x5563393dd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.573+0000 7fd3fc5f6740 1 --2- 172.21.15.152:0/3205304952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5563393cdb60 0x5563393dce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.573+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/3205304952 >> 172.21.15.152:0/3205304952 conn(0x5563393ca0f0 msgr2=0x5563393d6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:25.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.573+0000 7fd3d2ffd640 0 client.18234 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:25.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.573+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/3205304952 shutdown_connections 2024-09-11T23:02:25.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.573+0000 7fd3fc5f6740 1 -- 172.21.15.152:0/3205304952 wait complete. 2024-09-11T23:02:25.579 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.573+0000 7fd3fc5f6740 20 client.18234 trim_cache size 0 max 16384 2024-09-11T23:02:25.579 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:02:25.579 INFO:teuthology.orchestra.run.smithi152.stdout: "return_code": 0, 2024-09-11T23:02:25.580 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_tag": "a0a9cbdd-4553-4c35-b538-f5cae16485ac", 2024-09-11T23:02:25.580 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": "asynchronous" 2024-09-11T23:02:25.580 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:02:25.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.574+0000 7fd3fa21d640 1 -- 172.21.15.152:0/867815005 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd3dc0a2cf0 msgr2=0x7fd3dc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:25.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.574+0000 7fd3fa21d640 1 --2- 172.21.15.152:0/867815005 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd3dc0a2cf0 0x7fd3dc0a51b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8005ab0 tx=0x7fd3e8031690 comp rx=0 tx=0).stop 2024-09-11T23:02:25.580 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.574+0000 7fd3fa21d640 1 -- 172.21.15.152:0/867815005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f4106f10 msgr2=0x7fd3f419cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:25.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.574+0000 7fd3fa21d640 1 --2- 172.21.15.152:0/867815005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f4106f10 0x7fd3f419cde0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00cad0 tx=0x7fd3ec00cf90 comp rx=0 tx=0).stop 2024-09-11T23:02:25.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.574+0000 7fd3fa21d640 1 -- 172.21.15.152:0/867815005 shutdown_connections 2024-09-11T23:02:25.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.574+0000 7fd3fa21d640 1 --2- 172.21.15.152:0/867815005 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fd3dc0a2cf0 0x7fd3dc0a51b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.574+0000 7fd3fa21d640 1 --2- 172.21.15.152:0/867815005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3f4106f10 0x7fd3f419cde0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.574+0000 7fd3fa21d640 1 --2- 172.21.15.152:0/867815005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3f4102790 0x7fd3f4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.574+0000 7fd3fa21d640 1 --2- 172.21.15.152:0/867815005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd3f4101de0 0x7fd3f4198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:25.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.574+0000 7fd3fa21d640 1 -- 172.21.15.152:0/867815005 >> 172.21.15.152:0/867815005 conn(0x7fd3f40fd8c0 msgr2=0x7fd3f40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:25.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.574+0000 7fd3fa21d640 1 -- 172.21.15.152:0/867815005 shutdown_connections 2024-09-11T23:02:25.581 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:25.574+0000 7fd3fa21d640 1 -- 172.21.15.152:0/867815005 wait complete. 2024-09-11T23:02:26.360 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:02:26.609 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:27.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:26 smithi191 ceph-mon[35211]: pgmap v295: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T23:02:27.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:26 smithi191 ceph-mon[35211]: scrub queued for path: / 2024-09-11T23:02:27.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:26 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T23:02:27.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:26 smithi191 ceph-mon[35211]: scrub summary: active paths [/] 2024-09-11T23:02:27.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:26 smithi191 ceph-mon[35211]: scrub summary: idle+waiting paths [/] 2024-09-11T23:02:27.041 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:26 smithi152 ceph-mon[31181]: pgmap v295: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T23:02:27.041 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:26 smithi152 ceph-mon[31181]: scrub queued for path: / 2024-09-11T23:02:27.041 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:26 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T23:02:27.041 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:26 smithi152 ceph-mon[31181]: scrub summary: active paths [/] 2024-09-11T23:02:27.041 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:26 smithi152 ceph-mon[31181]: scrub summary: idle+waiting paths [/] 2024-09-11T23:02:27.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:26 smithi204 ceph-mon[35454]: pgmap v295: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T23:02:27.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:26 smithi204 ceph-mon[35454]: scrub queued for path: / 2024-09-11T23:02:27.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:26 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T23:02:27.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:26 smithi204 ceph-mon[35454]: scrub summary: active paths [/] 2024-09-11T23:02:27.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:26 smithi204 ceph-mon[35454]: scrub summary: idle+waiting paths [/] 2024-09-11T23:02:27.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.122+0000 7fed4e98c640 1 -- 172.21.15.152:0/587260193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed48106f10 msgr2=0x7fed4810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:27.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.122+0000 7fed4e98c640 1 --2- 172.21.15.152:0/587260193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed48106f10 0x7fed4810ea90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fed40009ca0 tx=0x7fed4002a8c0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.122+0000 7fed4e98c640 1 -- 172.21.15.152:0/587260193 shutdown_connections 2024-09-11T23:02:27.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.122+0000 7fed4e98c640 1 --2- 172.21.15.152:0/587260193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed48106f10 0x7fed4810ea90 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.122+0000 7fed4e98c640 1 --2- 172.21.15.152:0/587260193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed48102790 0x7fed481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.122+0000 7fed4e98c640 1 --2- 172.21.15.152:0/587260193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed48101de0 0x7fed481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.122+0000 7fed4e98c640 1 -- 172.21.15.152:0/587260193 >> 172.21.15.152:0/587260193 conn(0x7fed480fd8c0 msgr2=0x7fed480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:27.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.122+0000 7fed4e98c640 1 -- 172.21.15.152:0/587260193 shutdown_connections 2024-09-11T23:02:27.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.122+0000 7fed4e98c640 1 -- 172.21.15.152:0/587260193 wait complete. 2024-09-11T23:02:27.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.123+0000 7fed4e98c640 1 Processor -- start 2024-09-11T23:02:27.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.123+0000 7fed4e98c640 1 -- start start 2024-09-11T23:02:27.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.123+0000 7fed4e98c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed48101de0 0x7fed480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.123+0000 7fed4e98c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed48102790 0x7fed48074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.123+0000 7fed4e98c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed48106f10 0x7fed48074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.123+0000 7fed4e98c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fed48111f70 con 0x7fed48106f10 2024-09-11T23:02:27.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.123+0000 7fed4e98c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fed48111df0 con 0x7fed48101de0 2024-09-11T23:02:27.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.123+0000 7fed4e98c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed481120f0 con 0x7fed48102790 2024-09-11T23:02:27.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.123+0000 7fed4cf29640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed48106f10 0x7fed48074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:27.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.123+0000 7fed4cf29640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed48106f10 0x7fed48074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40782/0 (socket says 172.21.15.152:40782) 2024-09-11T23:02:27.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.123+0000 7fed4cf29640 1 -- 172.21.15.152:0/3839763417 learned_addr learned my addr 172.21.15.152:0/3839763417 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:27.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.123+0000 7fed47fff640 1 --2- 172.21.15.152:0/3839763417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed48101de0 0x7fed480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:27.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.124+0000 7fed477fe640 1 --2- 172.21.15.152:0/3839763417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed48102790 0x7fed48074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.124+0000 7fed4cf29640 1 -- 172.21.15.152:0/3839763417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed48102790 msgr2=0x7fed48074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.124+0000 7fed4cf29640 1 --2- 172.21.15.152:0/3839763417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed48102790 0x7fed48074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.124+0000 7fed4cf29640 1 -- 172.21.15.152:0/3839763417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed48101de0 msgr2=0x7fed480744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.124+0000 7fed4cf29640 1 --2- 172.21.15.152:0/3839763417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed48101de0 0x7fed480744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.124+0000 7fed4cf29640 1 -- 172.21.15.152:0/3839763417 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed481a68b0 con 0x7fed48106f10 2024-09-11T23:02:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.124+0000 7fed4cf29640 1 --2- 172.21.15.152:0/3839763417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed48106f10 0x7fed48074f70 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fed40031a20 tx=0x7fed40031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.124+0000 7fed457fa640 1 -- 172.21.15.152:0/3839763417 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40003c30 con 0x7fed48106f10 2024-09-11T23:02:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.124+0000 7fed457fa640 1 -- 172.21.15.152:0/3839763417 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed40003dd0 con 0x7fed48106f10 2024-09-11T23:02:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.124+0000 7fed4e98c640 1 -- 172.21.15.152:0/3839763417 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed481a6b40 con 0x7fed48106f10 2024-09-11T23:02:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.124+0000 7fed457fa640 1 -- 172.21.15.152:0/3839763417 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4002fcf0 con 0x7fed48106f10 2024-09-11T23:02:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.124+0000 7fed4e98c640 1 -- 172.21.15.152:0/3839763417 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed481a7080 con 0x7fed48106f10 2024-09-11T23:02:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.127+0000 7fed457fa640 1 -- 172.21.15.152:0/3839763417 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fed40004050 con 0x7fed48106f10 2024-09-11T23:02:27.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.127+0000 7fed50d65740 1 Processor -- start 2024-09-11T23:02:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.127+0000 7fed50d65740 1 -- start start 2024-09-11T23:02:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.127+0000 7fed50d65740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56466516d4a0 0x56466516d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.127+0000 7fed50d65740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516ddc0 0x564665178140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.127+0000 7fed50d65740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564665178680 0x56466517aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.127+0000 7fed50d65740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5646650d32e0 con 0x56466516ddc0 2024-09-11T23:02:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.127+0000 7fed50d65740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56466507ace0 con 0x564665178680 2024-09-11T23:02:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.127+0000 7fed50d65740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5646650cf2a0 con 0x56466516d4a0 2024-09-11T23:02:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.127+0000 7fed47fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516ddc0 0x564665178140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.127+0000 7fed47fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516ddc0 0x564665178140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40784/0 (socket says 172.21.15.152:40784) 2024-09-11T23:02:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.127+0000 7fed47fff640 1 -- 172.21.15.152:0/3125754631 learned_addr learned my addr 172.21.15.152:0/3125754631 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.128+0000 7fed47fff640 1 -- 172.21.15.152:0/3125754631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56466516d4a0 msgr2=0x56466516d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.128+0000 7fed477fe640 1 --2- 172.21.15.152:0/3125754631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56466516d4a0 0x56466516d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:27.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.128+0000 7fed47fff640 1 --2- 172.21.15.152:0/3125754631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56466516d4a0 0x56466516d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.128+0000 7fed477fe640 1 --2- 172.21.15.152:0/3125754631 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564665178680 0x56466517aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.128+0000 7fed47fff640 1 -- 172.21.15.152:0/3125754631 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564665178680 msgr2=0x56466517aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.128+0000 7fed47fff640 1 --2- 172.21.15.152:0/3125754631 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564665178680 0x56466517aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.128+0000 7fed47fff640 1 -- 172.21.15.152:0/3125754631 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56466505f440 con 0x56466516ddc0 2024-09-11T23:02:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.128+0000 7fed477fe640 1 --2- 172.21.15.152:0/3125754631 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564665178680 0x56466517aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.128+0000 7fed457fa640 1 --2- 172.21.15.152:0/3839763417 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fed300a2cf0 0x7fed300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.128+0000 7fed47fff640 1 --2- 172.21.15.152:0/3125754631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516ddc0 0x564665178140 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fed3800ac20 tx=0x7fed3800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b8688f3ad1ba04 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.128+0000 7fed477fe640 1 --2- 172.21.15.152:0/3839763417 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fed300a2cf0 0x7fed300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.128+0000 7fed457fa640 1 -- 172.21.15.152:0/3839763417 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7fed400f0670 con 0x7fed48106f10 2024-09-11T23:02:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.128+0000 7fed2e7fc640 1 -- 172.21.15.152:0/3125754631 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3801d070 con 0x56466516ddc0 2024-09-11T23:02:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.128+0000 7fed2e7fc640 1 -- 172.21.15.152:0/3125754631 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed38018b20 con 0x56466516ddc0 2024-09-11T23:02:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.128+0000 7fed2e7fc640 1 -- 172.21.15.152:0/3125754631 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38010ba0 con 0x56466516ddc0 2024-09-11T23:02:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.129+0000 7fed50d65740 1 -- 172.21.15.152:0/3125754631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516ddc0 msgr2=0x564665178140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:27.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.129+0000 7fed477fe640 1 --2- 172.21.15.152:0/3839763417 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fed300a2cf0 0x7fed300a51b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x56466516adb0 tx=0x7fed3c0046c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.129+0000 7fed50d65740 1 --2- 172.21.15.152:0/3125754631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516ddc0 0x564665178140 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fed3800ac20 tx=0x7fed3800f790 comp rx=0 tx=0).stop 2024-09-11T23:02:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.129+0000 7fed50d65740 1 -- 172.21.15.152:0/3125754631 shutdown_connections 2024-09-11T23:02:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.129+0000 7fed50d65740 1 --2- 172.21.15.152:0/3125754631 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564665178680 0x56466517aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.129+0000 7fed50d65740 1 --2- 172.21.15.152:0/3125754631 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516ddc0 0x564665178140 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.129+0000 7fed50d65740 1 --2- 172.21.15.152:0/3125754631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56466516d4a0 0x56466516d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.129+0000 7fed50d65740 1 -- 172.21.15.152:0/3125754631 >> 172.21.15.152:0/3125754631 conn(0x564665169a30 msgr2=0x564665169e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.129+0000 7fed50d65740 1 -- 172.21.15.152:0/3125754631 shutdown_connections 2024-09-11T23:02:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.129+0000 7fed50d65740 1 -- 172.21.15.152:0/3125754631 wait complete. 2024-09-11T23:02:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.129+0000 7fed50d65740 1 -- start start 2024-09-11T23:02:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.129+0000 7fed50d65740 1 Processor -- start 2024-09-11T23:02:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed50d65740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516d4a0 0x564665177f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed50d65740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56466516ddc0 0x5646651751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed50d65740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564665178680 0x564665175720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed50d65740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x56466517b250 con 0x56466516d4a0 2024-09-11T23:02:27.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed50d65740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x56466508e050 con 0x56466516ddc0 2024-09-11T23:02:27.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed50d65740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x56466517b3d0 con 0x564665178680 2024-09-11T23:02:27.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed50d65740 10 client.0 _ll_register_callbacks cb 0x5646650ce160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:27.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed47fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516d4a0 0x564665177f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:27.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed47fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516d4a0 0x564665177f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40788/0 (socket says 172.21.15.152:40788) 2024-09-11T23:02:27.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed47fff640 1 -- 172.21.15.152:0/2226835128 learned_addr learned my addr 172.21.15.152:0/2226835128 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:27.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed477fe640 1 --2- 172.21.15.152:0/2226835128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56466516ddc0 0x5646651751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:27.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed4cf29640 1 --2- 172.21.15.152:0/2226835128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564665178680 0x564665175720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:27.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed47fff640 1 -- 172.21.15.152:0/2226835128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564665178680 msgr2=0x564665175720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:27.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed47fff640 1 --2- 172.21.15.152:0/2226835128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564665178680 0x564665175720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed47fff640 1 -- 172.21.15.152:0/2226835128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56466516ddc0 msgr2=0x5646651751e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:27.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed47fff640 1 --2- 172.21.15.152:0/2226835128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56466516ddc0 0x5646651751e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed47fff640 1 -- 172.21.15.152:0/2226835128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564664e443c0 con 0x56466516d4a0 2024-09-11T23:02:27.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed477fe640 1 --2- 172.21.15.152:0/2226835128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56466516ddc0 0x5646651751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:27.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed47fff640 1 --2- 172.21.15.152:0/2226835128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516d4a0 0x564665177f30 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fed38010700 tx=0x7fed380186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9ce5cbccf5366b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:27.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed277fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:27.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.130+0000 7fed277fe640 1 -- 172.21.15.152:0/2226835128 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3801d050 con 0x56466516d4a0 2024-09-11T23:02:27.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.131+0000 7fed277fe640 1 -- 172.21.15.152:0/2226835128 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7fed38010ae0 con 0x56466516d4a0 2024-09-11T23:02:27.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.131+0000 7fed50d65740 1 -- 172.21.15.152:0/2226835128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed38018b20 con 0x56466516d4a0 2024-09-11T23:02:27.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.131+0000 7fed277fe640 1 -- 172.21.15.152:0/2226835128 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fed38032050 con 0x56466516d4a0 2024-09-11T23:02:27.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.131+0000 7fed50d65740 10 client.18246 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:27.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.131+0000 7fed50d65740 1 -- 172.21.15.152:0/2226835128 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564665186b40 con 0x56466516d4a0 2024-09-11T23:02:27.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.131+0000 7fed277fe640 1 -- 172.21.15.152:0/2226835128 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed38018b20 con 0x56466516d4a0 2024-09-11T23:02:27.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.131+0000 7fed277fe640 1 -- 172.21.15.152:0/2226835128 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fed38004920 con 0x56466516d4a0 2024-09-11T23:02:27.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.131+0000 7fed50d65740 10 client.18246 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:27.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.131+0000 7fed50d65740 10 client.18246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:02:27.165 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.131+0000 7fed50d65740 20 client.18246 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:27.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.131+0000 7fed50d65740 1 --2- 172.21.15.152:0/2226835128 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x564665187ef0 0x56466518a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.131+0000 7fed50d65740 4 client.18246 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:02:27.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.131+0000 7fed50d65740 1 -- 172.21.15.152:0/2226835128 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646650e0850 con 0x564665187ef0 2024-09-11T23:02:27.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.131+0000 7fed477fe640 1 --2- 172.21.15.152:0/2226835128 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x564665187ef0 0x56466518a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:27.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.132+0000 7fed277fe640 10 client.18246 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:27.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.132+0000 7fed477fe640 1 --2- 172.21.15.152:0/2226835128 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x564665187ef0 0x56466518a2e0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6187eb5cbeedca6 server_cookie=6d0c8589edf5effe in_seq=0 out_seq=0 2024-09-11T23:02:27.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.132+0000 7fed277fe640 1 -- 172.21.15.152:0/2226835128 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646650e0850 con 0x564665187ef0 2024-09-11T23:02:27.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.132+0000 7fed277fe640 10 client.18246 handle_command_reply: tid=0 2024-09-11T23:02:27.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.132+0000 7fed50d65740 1 client.18246 shutdown 2024-09-11T23:02:27.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.133+0000 7fed50d65740 1 -- 172.21.15.152:0/2226835128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516d4a0 msgr2=0x564665177f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:27.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.133+0000 7fed50d65740 1 --2- 172.21.15.152:0/2226835128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516d4a0 0x564665177f30 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fed38010700 tx=0x7fed380186f0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.133+0000 7fed50d65740 1 -- 172.21.15.152:0/2226835128 shutdown_connections 2024-09-11T23:02:27.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.133+0000 7fed50d65740 1 --2- 172.21.15.152:0/2226835128 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564665178680 0x564665175720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.133+0000 7fed50d65740 1 --2- 172.21.15.152:0/2226835128 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x564665187ef0 0x56466518a2e0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.133+0000 7fed50d65740 1 --2- 172.21.15.152:0/2226835128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56466516ddc0 0x5646651751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.133+0000 7fed50d65740 1 --2- 172.21.15.152:0/2226835128 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516d4a0 0x564665177f30 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.133+0000 7fed277fe640 0 client.18246 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.133+0000 7fed50d65740 1 -- 172.21.15.152:0/2226835128 >> 172.21.15.152:0/2226835128 conn(0x564665169a30 msgr2=0x56466516f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.133+0000 7fed50d65740 1 -- 172.21.15.152:0/2226835128 shutdown_connections 2024-09-11T23:02:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.133+0000 7fed50d65740 1 -- 172.21.15.152:0/2226835128 wait complete. 2024-09-11T23:02:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.133+0000 7fed50d65740 20 client.18246 trim_cache size 0 max 16384 2024-09-11T23:02:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.150+0000 7fed50d65740 1 Processor -- start 2024-09-11T23:02:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.150+0000 7fed50d65740 1 -- start start 2024-09-11T23:02:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.150+0000 7fed50d65740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56466516d4a0 0x564665184510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.150+0000 7fed50d65740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516ddc0 0x564665170900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.150+0000 7fed50d65740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564665178680 0x564665168a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.150+0000 7fed50d65740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fed481afa40 con 0x56466516ddc0 2024-09-11T23:02:27.167 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.150+0000 7fed50d65740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fed481b1580 con 0x564665178680 2024-09-11T23:02:27.168 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.150+0000 7fed50d65740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed481b0110 con 0x56466516d4a0 2024-09-11T23:02:27.168 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:02:27.168 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T23:02:27.168 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": { 2024-09-11T23:02:27.168 INFO:teuthology.orchestra.run.smithi152.stdout: "a0a9cbdd-4553-4c35-b538-f5cae16485ac": { 2024-09-11T23:02:27.168 INFO:teuthology.orchestra.run.smithi152.stdout: "path": "/", 2024-09-11T23:02:27.168 INFO:teuthology.orchestra.run.smithi152.stdout: "tag": "a0a9cbdd-4553-4c35-b538-f5cae16485ac", 2024-09-11T23:02:27.168 INFO:teuthology.orchestra.run.smithi152.stdout: "options": "recursive,force" 2024-09-11T23:02:27.169 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:02:27.169 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:02:27.169 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:02:27.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.150+0000 7fed477fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516ddc0 0x564665170900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:27.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.150+0000 7fed477fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516ddc0 0x564665170900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40794/0 (socket says 172.21.15.152:40794) 2024-09-11T23:02:27.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.150+0000 7fed477fe640 1 -- 172.21.15.152:0/3842242411 learned_addr learned my addr 172.21.15.152:0/3842242411 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:27.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.150+0000 7fed477fe640 1 -- 172.21.15.152:0/3842242411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56466516d4a0 msgr2=0x564665184510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:27.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.150+0000 7fed477fe640 1 --2- 172.21.15.152:0/3842242411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56466516d4a0 0x564665184510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.169 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.150+0000 7fed477fe640 1 -- 172.21.15.152:0/3842242411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564665178680 msgr2=0x564665168a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.150+0000 7fed477fe640 1 --2- 172.21.15.152:0/3842242411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564665178680 0x564665168a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.150+0000 7fed477fe640 1 -- 172.21.15.152:0/3842242411 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564665169860 con 0x56466516ddc0 2024-09-11T23:02:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.151+0000 7fed477fe640 1 --2- 172.21.15.152:0/3842242411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516ddc0 0x564665170900 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fed48075f80 tx=0x7fed3c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b311b7f554449256 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.151+0000 7fed2dffb640 1 -- 172.21.15.152:0/3842242411 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c02d030 con 0x56466516ddc0 2024-09-11T23:02:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.151+0000 7fed2dffb640 1 -- 172.21.15.152:0/3842242411 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed3c0299e0 con 0x56466516ddc0 2024-09-11T23:02:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.151+0000 7fed2dffb640 1 -- 172.21.15.152:0/3842242411 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c01ad70 con 0x56466516ddc0 2024-09-11T23:02:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.151+0000 7fed50d65740 1 -- 172.21.15.152:0/3842242411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516ddc0 msgr2=0x564665170900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.151+0000 7fed50d65740 1 --2- 172.21.15.152:0/3842242411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516ddc0 0x564665170900 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fed48075f80 tx=0x7fed3c014000 comp rx=0 tx=0).stop 2024-09-11T23:02:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.151+0000 7fed50d65740 1 -- 172.21.15.152:0/3842242411 shutdown_connections 2024-09-11T23:02:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.151+0000 7fed50d65740 1 --2- 172.21.15.152:0/3842242411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564665178680 0x564665168a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.151+0000 7fed50d65740 1 --2- 172.21.15.152:0/3842242411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56466516ddc0 0x564665170900 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.151+0000 7fed50d65740 1 --2- 172.21.15.152:0/3842242411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56466516d4a0 0x564665184510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.170 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.151+0000 7fed50d65740 1 -- 172.21.15.152:0/3842242411 >> 172.21.15.152:0/3842242411 conn(0x564665169a30 msgr2=0x564665168350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.151+0000 7fed50d65740 1 -- 172.21.15.152:0/3842242411 shutdown_connections 2024-09-11T23:02:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.152+0000 7fed50d65740 1 -- 172.21.15.152:0/3842242411 wait complete. 2024-09-11T23:02:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.152+0000 7fed50d65740 1 -- start start 2024-09-11T23:02:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.152+0000 7fed50d65740 1 Processor -- start 2024-09-11T23:02:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.152+0000 7fed50d65740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56466516d4a0 0x564665180940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.152+0000 7fed50d65740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56466516ddc0 0x564665180e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.152+0000 7fed50d65740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564665178680 0x56466517bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.152+0000 7fed50d65740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fed481b0eb0 con 0x564665178680 2024-09-11T23:02:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.152+0000 7fed50d65740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fed481b07e0 con 0x56466516d4a0 2024-09-11T23:02:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.152+0000 7fed50d65740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fed481ada40 con 0x56466516ddc0 2024-09-11T23:02:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.152+0000 7fed50d65740 10 client.0 _ll_register_callbacks cb 0x7fed3801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.152+0000 7fed4cf29640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564665178680 0x56466517bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.152+0000 7fed4cf29640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564665178680 0x56466517bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:40806/0 (socket says 172.21.15.152:40806) 2024-09-11T23:02:27.171 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.152+0000 7fed4cf29640 1 -- 172.21.15.152:0/3539612082 learned_addr learned my addr 172.21.15.152:0/3539612082 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:27.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed47fff640 1 --2- 172.21.15.152:0/3539612082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56466516d4a0 0x564665180940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:27.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed477fe640 1 --2- 172.21.15.152:0/3539612082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56466516ddc0 0x564665180e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:27.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed4cf29640 1 -- 172.21.15.152:0/3539612082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56466516ddc0 msgr2=0x564665180e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:27.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed4cf29640 1 --2- 172.21.15.152:0/3539612082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56466516ddc0 0x564665180e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed4cf29640 1 -- 172.21.15.152:0/3539612082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56466516d4a0 msgr2=0x564665180940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:27.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed4cf29640 1 --2- 172.21.15.152:0/3539612082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56466516d4a0 0x564665180940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed4cf29640 1 -- 172.21.15.152:0/3539612082 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56466517c550 con 0x564665178680 2024-09-11T23:02:27.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed4cf29640 1 --2- 172.21.15.152:0/3539612082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564665178680 0x56466517bd20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fed400bca50 tx=0x7fed400bc770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4f4d86419bc257 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:27.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed277fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:27.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed277fe640 1 -- 172.21.15.152:0/3539612082 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed400bcd50 con 0x564665178680 2024-09-11T23:02:27.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed277fe640 1 -- 172.21.15.152:0/3539612082 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7fed400b93c0 con 0x564665178680 2024-09-11T23:02:27.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed50d65740 1 -- 172.21.15.152:0/3539612082 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed3c0299e0 con 0x564665178680 2024-09-11T23:02:27.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed277fe640 1 -- 172.21.15.152:0/3539612082 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fed400c1100 con 0x564665178680 2024-09-11T23:02:27.172 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed50d65740 10 client.18252 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed50d65740 1 -- 172.21.15.152:0/3539612082 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5646651f2560 con 0x564665178680 2024-09-11T23:02:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed277fe640 1 -- 172.21.15.152:0/3539612082 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed3c0299e0 con 0x564665178680 2024-09-11T23:02:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed277fe640 1 -- 172.21.15.152:0/3539612082 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fed400c54a0 con 0x564665178680 2024-09-11T23:02:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed50d65740 10 client.18252 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed50d65740 10 client.18252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:02:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed50d65740 20 client.18252 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed50d65740 1 --2- 172.21.15.152:0/3539612082 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5646651f2cb0 0x5646651f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed50d65740 4 client.18252 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:02:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.153+0000 7fed50d65740 1 -- 172.21.15.152:0/3539612082 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564665175ab0 con 0x5646651f2cb0 2024-09-11T23:02:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.154+0000 7fed47fff640 1 --2- 172.21.15.152:0/3539612082 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5646651f2cb0 0x5646651f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.154+0000 7fed47fff640 1 --2- 172.21.15.152:0/3539612082 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5646651f2cb0 0x5646651f50d0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2badd131124b3b44 server_cookie=efd0c2e6a6181241 in_seq=0 out_seq=0 2024-09-11T23:02:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.154+0000 7fed277fe640 10 client.18252 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:27.173 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.154+0000 7fed277fe640 1 -- 172.21.15.152:0/3539612082 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564665175ab0 con 0x5646651f2cb0 2024-09-11T23:02:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.154+0000 7fed277fe640 10 client.18252 handle_command_reply: tid=0 2024-09-11T23:02:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.154+0000 7fed50d65740 1 client.18252 shutdown 2024-09-11T23:02:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.154+0000 7fed50d65740 1 -- 172.21.15.152:0/3539612082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564665178680 msgr2=0x56466517bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed50d65740 1 --2- 172.21.15.152:0/3539612082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564665178680 0x56466517bd20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fed400bca50 tx=0x7fed400bc770 comp rx=0 tx=0).stop 2024-09-11T23:02:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed50d65740 1 -- 172.21.15.152:0/3539612082 shutdown_connections 2024-09-11T23:02:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed50d65740 1 --2- 172.21.15.152:0/3539612082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564665178680 0x56466517bd20 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed50d65740 1 --2- 172.21.15.152:0/3539612082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56466516ddc0 0x564665180e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed50d65740 1 --2- 172.21.15.152:0/3539612082 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x5646651f2cb0 0x5646651f50d0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed50d65740 1 --2- 172.21.15.152:0/3539612082 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x56466516d4a0 0x564665180940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed277fe640 0 client.18252 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed50d65740 1 -- 172.21.15.152:0/3539612082 >> 172.21.15.152:0/3539612082 conn(0x564665169a30 msgr2=0x564665173980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed50d65740 1 -- 172.21.15.152:0/3539612082 shutdown_connections 2024-09-11T23:02:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed50d65740 1 -- 172.21.15.152:0/3539612082 wait complete. 2024-09-11T23:02:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed50d65740 20 client.18252 trim_cache size 0 max 16384 2024-09-11T23:02:27.174 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed4e98c640 1 -- 172.21.15.152:0/3839763417 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fed300a2cf0 msgr2=0x7fed300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:27.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed4e98c640 1 --2- 172.21.15.152:0/3839763417 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fed300a2cf0 0x7fed300a51b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x56466516adb0 tx=0x7fed3c0046c0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed4e98c640 1 -- 172.21.15.152:0/3839763417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed48106f10 msgr2=0x7fed48074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:27.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed4e98c640 1 --2- 172.21.15.152:0/3839763417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed48106f10 0x7fed48074f70 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fed40031a20 tx=0x7fed40031a50 comp rx=0 tx=0).stop 2024-09-11T23:02:27.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed4e98c640 1 -- 172.21.15.152:0/3839763417 shutdown_connections 2024-09-11T23:02:27.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed4e98c640 1 --2- 172.21.15.152:0/3839763417 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fed300a2cf0 0x7fed300a51b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed4e98c640 1 --2- 172.21.15.152:0/3839763417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fed48106f10 0x7fed48074f70 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed4e98c640 1 --2- 172.21.15.152:0/3839763417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed48102790 0x7fed48074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed4e98c640 1 --2- 172.21.15.152:0/3839763417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed48101de0 0x7fed480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:27.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed4e98c640 1 -- 172.21.15.152:0/3839763417 >> 172.21.15.152:0/3839763417 conn(0x7fed480fd8c0 msgr2=0x7fed480feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:27.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed4e98c640 1 -- 172.21.15.152:0/3839763417 shutdown_connections 2024-09-11T23:02:27.175 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:27.155+0000 7fed4e98c640 1 -- 172.21.15.152:0/3839763417 wait complete. 2024-09-11T23:02:27.990 INFO:tasks.cephfs.filesystem:scrub status for tag:a0a9cbdd-4553-4c35-b538-f5cae16485ac - {'path': '/', 'tag': 'a0a9cbdd-4553-4c35-b538-f5cae16485ac', 'options': 'recursive,force'} 2024-09-11T23:02:29.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:28 smithi191 ceph-mon[35211]: pgmap v296: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T23:02:29.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:28 smithi191 ceph-mon[35211]: scrub summary: idle 2024-09-11T23:02:29.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:28 smithi204 ceph-mon[35454]: pgmap v296: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T23:02:29.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:28 smithi204 ceph-mon[35454]: scrub summary: idle 2024-09-11T23:02:29.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:28 smithi152 ceph-mon[31181]: pgmap v296: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T23:02:29.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:28 smithi152 ceph-mon[31181]: scrub summary: idle 2024-09-11T23:02:31.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:30 smithi191 ceph-mon[35211]: pgmap v297: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 11 op/s 2024-09-11T23:02:31.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:30 smithi204 ceph-mon[35454]: pgmap v297: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 11 op/s 2024-09-11T23:02:31.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:30 smithi152 ceph-mon[31181]: pgmap v297: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 11 op/s 2024-09-11T23:02:33.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:32 smithi191 ceph-mon[35211]: pgmap v298: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-11T23:02:33.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:32 smithi204 ceph-mon[35454]: pgmap v298: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-11T23:02:33.123 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:32 smithi152 ceph-mon[31181]: pgmap v298: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-11T23:02:35.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:34 smithi191 ceph-mon[35211]: pgmap v299: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T23:02:35.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:34 smithi204 ceph-mon[35454]: pgmap v299: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T23:02:35.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:34 smithi152 ceph-mon[31181]: pgmap v299: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T23:02:37.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:36 smithi191 ceph-mon[35211]: pgmap v300: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T23:02:37.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:36 smithi204 ceph-mon[35454]: pgmap v300: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T23:02:37.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:36 smithi152 ceph-mon[31181]: pgmap v300: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 13 op/s 2024-09-11T23:02:39.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:38 smithi204 ceph-mon[35454]: pgmap v301: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T23:02:39.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:38 smithi152 ceph-mon[31181]: pgmap v301: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T23:02:39.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:38 smithi191 ceph-mon[35211]: pgmap v301: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-09-11T23:02:41.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:40 smithi204 ceph-mon[35454]: pgmap v302: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 13 op/s 2024-09-11T23:02:41.089 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:40 smithi191 ceph-mon[35211]: pgmap v302: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 13 op/s 2024-09-11T23:02:41.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:40 smithi152 ceph-mon[31181]: pgmap v302: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 13 op/s 2024-09-11T23:02:43.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:42 smithi204 ceph-mon[35454]: pgmap v303: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-11T23:02:43.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:42 smithi152 ceph-mon[31181]: pgmap v303: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-11T23:02:43.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:42 smithi191 ceph-mon[35211]: pgmap v303: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-11T23:02:45.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:44 smithi204 ceph-mon[35454]: pgmap v304: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T23:02:45.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:44 smithi152 ceph-mon[31181]: pgmap v304: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T23:02:45.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:44 smithi191 ceph-mon[35211]: pgmap v304: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T23:02:47.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:46 smithi152 ceph-mon[31181]: pgmap v305: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:02:47.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:46 smithi191 ceph-mon[35211]: pgmap v305: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:02:47.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:46 smithi204 ceph-mon[35454]: pgmap v305: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:02:49.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:48 smithi152 ceph-mon[31181]: pgmap v306: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T23:02:49.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:48 smithi191 ceph-mon[35211]: pgmap v306: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T23:02:49.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:48 smithi204 ceph-mon[35454]: pgmap v306: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T23:02:51.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:50 smithi152 ceph-mon[31181]: pgmap v307: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-11T23:02:51.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:50 smithi191 ceph-mon[35211]: pgmap v307: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-11T23:02:51.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:50 smithi204 ceph-mon[35454]: pgmap v307: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-11T23:02:53.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:52 smithi152 ceph-mon[31181]: pgmap v308: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-11T23:02:53.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:52 smithi191 ceph-mon[35211]: pgmap v308: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-11T23:02:53.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:52 smithi204 ceph-mon[35454]: pgmap v308: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-11T23:02:55.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:54 smithi152 ceph-mon[31181]: pgmap v309: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:02:55.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:54 smithi191 ceph-mon[35211]: pgmap v309: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:02:55.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:54 smithi204 ceph-mon[35454]: pgmap v309: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:02:57.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:56 smithi152 ceph-mon[31181]: pgmap v310: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:02:57.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:56 smithi191 ceph-mon[35211]: pgmap v310: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:02:57.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:56 smithi204 ceph-mon[35454]: pgmap v310: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-11T23:02:57.992 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T23:02:58.251 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.846+0000 7f1c0a6ca640 1 -- 172.21.15.152:0/1500672499 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c04101de0 msgr2=0x7f1c041021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.846+0000 7f1c0a6ca640 1 --2- 172.21.15.152:0/1500672499 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c04101de0 0x7f1c041021c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009900 tx=0x7f1bf402a410 comp rx=0 tx=0).stop 2024-09-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.846+0000 7f1c0a6ca640 1 -- 172.21.15.152:0/1500672499 shutdown_connections 2024-09-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.846+0000 7f1c0a6ca640 1 --2- 172.21.15.152:0/1500672499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c04106f10 0x7f1c0410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.846+0000 7f1c0a6ca640 1 --2- 172.21.15.152:0/1500672499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c04102790 0x7f1c041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.846+0000 7f1c0a6ca640 1 --2- 172.21.15.152:0/1500672499 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c04101de0 0x7f1c041021c0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.846+0000 7f1c0a6ca640 1 -- 172.21.15.152:0/1500672499 >> 172.21.15.152:0/1500672499 conn(0x7f1c040fd8c0 msgr2=0x7f1c040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.846+0000 7f1c0a6ca640 1 -- 172.21.15.152:0/1500672499 shutdown_connections 2024-09-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.846+0000 7f1c0a6ca640 1 -- 172.21.15.152:0/1500672499 wait complete. 2024-09-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.847+0000 7f1c0a6ca640 1 Processor -- start 2024-09-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.847+0000 7f1c0a6ca640 1 -- start start 2024-09-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.847+0000 7f1c0a6ca640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c04101de0 0x7f1c040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.847+0000 7f1c0a6ca640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c04102790 0x7f1c04074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.847+0000 7f1c0a6ca640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c04106f10 0x7f1c04074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.847+0000 7f1c0a6ca640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04111f90 con 0x7f1c04101de0 2024-09-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.847+0000 7f1c0a6ca640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04111e10 con 0x7f1c04106f10 2024-09-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.847+0000 7f1c0a6ca640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1c04112110 con 0x7f1c04102790 2024-09-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.847+0000 7f1c03fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c04101de0 0x7f1c040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.847+0000 7f1c03fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c04101de0 0x7f1c040744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35498/0 (socket says 172.21.15.152:35498) 2024-09-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.848+0000 7f1c03fff640 1 -- 172.21.15.152:0/1250885065 learned_addr learned my addr 172.21.15.152:0/1250885065 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.848+0000 7f1c037fe640 1 --2- 172.21.15.152:0/1250885065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c04102790 0x7f1c04074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.848+0000 7f1c08c67640 1 --2- 172.21.15.152:0/1250885065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c04106f10 0x7f1c04074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.848+0000 7f1c03fff640 1 -- 172.21.15.152:0/1250885065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c04102790 msgr2=0x7f1c04074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.848+0000 7f1c03fff640 1 --2- 172.21.15.152:0/1250885065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c04102790 0x7f1c04074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.848+0000 7f1c03fff640 1 -- 172.21.15.152:0/1250885065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c04106f10 msgr2=0x7f1c04074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.848+0000 7f1c03fff640 1 --2- 172.21.15.152:0/1250885065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c04106f10 0x7f1c04074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.848+0000 7f1c03fff640 1 -- 172.21.15.152:0/1250885065 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c041a6940 con 0x7f1c04101de0 2024-09-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.848+0000 7f1c08c67640 1 --2- 172.21.15.152:0/1250885065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c04106f10 0x7f1c04074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.848+0000 7f1c03fff640 1 --2- 172.21.15.152:0/1250885065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c04101de0 0x7f1c040744f0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4006010 tx=0x7f1bf40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.848+0000 7f1c017fa640 1 -- 172.21.15.152:0/1250885065 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf403f030 con 0x7f1c04101de0 2024-09-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.848+0000 7f1c0a6ca640 1 -- 172.21.15.152:0/1250885065 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c041a6bd0 con 0x7f1c04101de0 2024-09-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.848+0000 7f1c017fa640 1 -- 172.21.15.152:0/1250885065 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf4003ce0 con 0x7f1c04101de0 2024-09-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.848+0000 7f1c017fa640 1 -- 172.21.15.152:0/1250885065 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4035070 con 0x7f1c04101de0 2024-09-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.848+0000 7f1c0a6ca640 1 -- 172.21.15.152:0/1250885065 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c041a70b0 con 0x7f1c04101de0 2024-09-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.850+0000 7f1c017fa640 1 -- 172.21.15.152:0/1250885065 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1bf4042030 con 0x7f1c04101de0 2024-09-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c0caa3740 1 Processor -- start 2024-09-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c0caa3740 1 -- start start 2024-09-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c0caa3740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c9004a0 0x55c17c900880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c0caa3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c900dc0 0x55c17c90b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c0caa3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c90b680 0x55c17c90da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c0caa3740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55c17c8662e0 con 0x55c17c9004a0 2024-09-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c0caa3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c17c80dce0 con 0x55c17c90b680 2024-09-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c0caa3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c17c8622a0 con 0x55c17c900dc0 2024-09-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c08c67640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c9004a0 0x55c17c900880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c08c67640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c9004a0 0x55c17c900880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35514/0 (socket says 172.21.15.152:35514) 2024-09-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c08c67640 1 -- 172.21.15.152:0/1984343338 learned_addr learned my addr 172.21.15.152:0/1984343338 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c03fff640 1 --2- 172.21.15.152:0/1984343338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c900dc0 0x55c17c90b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c037fe640 1 --2- 172.21.15.152:0/1984343338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c90b680 0x55c17c90da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c08c67640 1 -- 172.21.15.152:0/1984343338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c900dc0 msgr2=0x55c17c90b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c08c67640 1 --2- 172.21.15.152:0/1984343338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c900dc0 0x55c17c90b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c08c67640 1 -- 172.21.15.152:0/1984343338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c90b680 msgr2=0x55c17c90da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c08c67640 1 --2- 172.21.15.152:0/1984343338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c90b680 0x55c17c90da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c08c67640 1 -- 172.21.15.152:0/1984343338 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c17c7f2440 con 0x55c17c9004a0 2024-09-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c017fa640 1 --2- 172.21.15.152:0/1250885065 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1bec0a2cf0 0x7f1bec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.851+0000 7f1c03fff640 1 --2- 172.21.15.152:0/1984343338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c900dc0 0x55c17c90b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.852+0000 7f1c03fff640 1 --2- 172.21.15.152:0/1250885065 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1bec0a2cf0 0x7f1bec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.852+0000 7f1c08c67640 1 --2- 172.21.15.152:0/1984343338 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c9004a0 0x55c17c900880 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc007f30 tx=0x7f1bfc00c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66b4753c085b4a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.852+0000 7f1c017fa640 1 -- 172.21.15.152:0/1250885065 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f1bf40ef9f0 con 0x7f1c04101de0 2024-09-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.852+0000 7f1bea7fc640 1 -- 172.21.15.152:0/1984343338 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc014070 con 0x55c17c9004a0 2024-09-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.852+0000 7f1bea7fc640 1 -- 172.21.15.152:0/1984343338 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bfc00fd30 con 0x55c17c9004a0 2024-09-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.852+0000 7f1bea7fc640 1 -- 172.21.15.152:0/1984343338 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00ed00 con 0x55c17c9004a0 2024-09-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.853+0000 7f1c03fff640 1 --2- 172.21.15.152:0/1250885065 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1bec0a2cf0 0x7f1bec0a51b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1bf40ed210 tx=0x7f1bf402a410 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.853+0000 7f1c0caa3740 1 -- 172.21.15.152:0/1984343338 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c9004a0 msgr2=0x55c17c900880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.853+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/1984343338 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c9004a0 0x55c17c900880 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc007f30 tx=0x7f1bfc00c750 comp rx=0 tx=0).stop 2024-09-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.853+0000 7f1c0caa3740 1 -- 172.21.15.152:0/1984343338 shutdown_connections 2024-09-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.853+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/1984343338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c90b680 0x55c17c90da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.853+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/1984343338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c900dc0 0x55c17c90b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.853+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/1984343338 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c9004a0 0x55c17c900880 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.853+0000 7f1c0caa3740 1 -- 172.21.15.152:0/1984343338 >> 172.21.15.152:0/1984343338 conn(0x55c17c8fca30 msgr2=0x55c17c8fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.853+0000 7f1c0caa3740 1 -- 172.21.15.152:0/1984343338 shutdown_connections 2024-09-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.853+0000 7f1c0caa3740 1 -- 172.21.15.152:0/1984343338 wait complete. 2024-09-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.853+0000 7f1c0caa3740 1 -- start start 2024-09-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.853+0000 7f1c0caa3740 1 Processor -- start 2024-09-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.853+0000 7f1c0caa3740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c9004a0 0x55c17c90af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.853+0000 7f1c0caa3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c900dc0 0x55c17c908230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.853+0000 7f1c0caa3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c90b680 0x55c17c908770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c0caa3740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55c17c90e250 con 0x55c17c9004a0 2024-09-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c0caa3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c17c821050 con 0x55c17c90b680 2024-09-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c0caa3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c17c90e3d0 con 0x55c17c900dc0 2024-09-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c0caa3740 10 client.0 _ll_register_callbacks cb 0x55c17c861160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c08c67640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c9004a0 0x55c17c90af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c08c67640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c9004a0 0x55c17c90af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35526/0 (socket says 172.21.15.152:35526) 2024-09-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c08c67640 1 -- 172.21.15.152:0/3696733732 learned_addr learned my addr 172.21.15.152:0/3696733732 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c037fe640 1 --2- 172.21.15.152:0/3696733732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c900dc0 0x55c17c908230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c08c67640 1 -- 172.21.15.152:0/3696733732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c900dc0 msgr2=0x55c17c908230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c08c67640 1 --2- 172.21.15.152:0/3696733732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c900dc0 0x55c17c908230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c08c67640 1 -- 172.21.15.152:0/3696733732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c90b680 msgr2=0x55c17c908770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c08c67640 1 --2- 172.21.15.152:0/3696733732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c90b680 0x55c17c908770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c08c67640 1 -- 172.21.15.152:0/3696733732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c17c5d73c0 con 0x55c17c9004a0 2024-09-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c08c67640 1 --2- 172.21.15.152:0/3696733732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c9004a0 0x55c17c90af80 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc007f00 tx=0x7f1bfc007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241631eac9f61599 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1be37fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1be37fe640 1 -- 172.21.15.152:0/3696733732 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc028060 con 0x55c17c9004a0 2024-09-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1be37fe640 1 -- 172.21.15.152:0/3696733732 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f1bfc00f850 con 0x55c17c9004a0 2024-09-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c0caa3740 1 -- 172.21.15.152:0/3696733732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bfc00fd30 con 0x55c17c9004a0 2024-09-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1be37fe640 1 -- 172.21.15.152:0/3696733732 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1bfc014040 con 0x55c17c9004a0 2024-09-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c0caa3740 10 client.18264 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.854+0000 7f1c0caa3740 1 -- 172.21.15.152:0/3696733732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c17c919bc0 con 0x55c17c9004a0 2024-09-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.855+0000 7f1be37fe640 1 -- 172.21.15.152:0/3696733732 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bfc00fd30 con 0x55c17c9004a0 2024-09-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.855+0000 7f1be37fe640 1 -- 172.21.15.152:0/3696733732 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1bfc02b390 con 0x55c17c9004a0 2024-09-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.855+0000 7f1c0caa3740 10 client.18264 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.855+0000 7f1c0caa3740 10 client.18264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.855+0000 7f1c0caa3740 20 client.18264 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.855+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/3696733732 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c17c91b080 0x55c17c91d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.855+0000 7f1c0caa3740 4 client.18264 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.855+0000 7f1c0caa3740 1 -- 172.21.15.152:0/3696733732 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c17c873850 con 0x55c17c91b080 2024-09-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.855+0000 7f1c037fe640 1 --2- 172.21.15.152:0/3696733732 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c17c91b080 0x55c17c91d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.856+0000 7f1c037fe640 1 --2- 172.21.15.152:0/3696733732 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c17c91b080 0x55c17c91d470 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc4138ef6b0065bc server_cookie=96d47c875527bbed in_seq=0 out_seq=0 2024-09-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.856+0000 7f1be37fe640 10 client.18264 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.856+0000 7f1be37fe640 1 -- 172.21.15.152:0/3696733732 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c17c873850 con 0x55c17c91b080 2024-09-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.856+0000 7f1be37fe640 10 client.18264 handle_command_reply: tid=0 2024-09-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.856+0000 7f1c0caa3740 1 client.18264 shutdown 2024-09-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.856+0000 7f1c0caa3740 1 -- 172.21.15.152:0/3696733732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c9004a0 msgr2=0x55c17c90af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.856+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/3696733732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c9004a0 0x55c17c90af80 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc007f00 tx=0x7f1bfc007c70 comp rx=0 tx=0).stop 2024-09-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.856+0000 7f1c0caa3740 1 -- 172.21.15.152:0/3696733732 shutdown_connections 2024-09-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.856+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/3696733732 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c17c91b080 0x55c17c91d470 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.856+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/3696733732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c90b680 0x55c17c908770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.856+0000 7f1be37fe640 0 client.18264 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.857+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/3696733732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c900dc0 0x55c17c908230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.857+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/3696733732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c9004a0 0x55c17c90af80 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.857+0000 7f1c0caa3740 1 -- 172.21.15.152:0/3696733732 >> 172.21.15.152:0/3696733732 conn(0x55c17c8fca30 msgr2=0x55c17c8fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.857+0000 7f1c0caa3740 1 -- 172.21.15.152:0/3696733732 shutdown_connections 2024-09-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.857+0000 7f1c0caa3740 1 -- 172.21.15.152:0/3696733732 wait complete. 2024-09-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.857+0000 7f1c0caa3740 20 client.18264 trim_cache size 0 max 16384 2024-09-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c0caa3740 1 Processor -- start 2024-09-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c0caa3740 1 -- start start 2024-09-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c0caa3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c8fca30 0x55c17c9204a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c0caa3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c9004a0 0x55c17c9209e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c0caa3740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c900dc0 0x55c17c9054b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c0caa3740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041afa70 con 0x55c17c900dc0 2024-09-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c0caa3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041b15b0 con 0x55c17c9004a0 2024-09-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c0caa3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041b0140 con 0x55c17c8fca30 2024-09-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c08c67640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c900dc0 0x55c17c9054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c08c67640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c900dc0 0x55c17c9054b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35542/0 (socket says 172.21.15.152:35542) 2024-09-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c08c67640 1 -- 172.21.15.152:0/795195258 learned_addr learned my addr 172.21.15.152:0/795195258 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c08c67640 1 --2- 172.21.15.152:0/795195258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c8fca30 0x55c17c9204a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c037fe640 1 --2- 172.21.15.152:0/795195258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c9004a0 0x55c17c9209e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c08c67640 1 -- 172.21.15.152:0/795195258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c8fca30 msgr2=0x55c17c9204a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c08c67640 1 --2- 172.21.15.152:0/795195258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c8fca30 0x55c17c9204a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c08c67640 1 -- 172.21.15.152:0/795195258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c9004a0 msgr2=0x55c17c9209e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c08c67640 1 --2- 172.21.15.152:0/795195258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c9004a0 0x55c17c9209e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c08c67640 1 -- 172.21.15.152:0/795195258 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c17c8fdae0 con 0x55c17c900dc0 2024-09-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.874+0000 7f1c08c67640 1 --2- 172.21.15.152:0/795195258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c900dc0 0x55c17c9054b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc007840 tx=0x7f1bfc00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66555c5b42f23cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.875+0000 7f1be9ffb640 1 -- 172.21.15.152:0/795195258 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00cc50 con 0x55c17c900dc0 2024-09-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.875+0000 7f1be9ffb640 1 -- 172.21.15.152:0/795195258 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bfc02b390 con 0x55c17c900dc0 2024-09-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.875+0000 7f1be9ffb640 1 -- 172.21.15.152:0/795195258 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc017360 con 0x55c17c900dc0 2024-09-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.875+0000 7f1c0caa3740 1 -- 172.21.15.152:0/795195258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c900dc0 msgr2=0x55c17c9054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.875+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/795195258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c900dc0 0x55c17c9054b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc007840 tx=0x7f1bfc00ea10 comp rx=0 tx=0).stop 2024-09-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.875+0000 7f1c0caa3740 1 -- 172.21.15.152:0/795195258 shutdown_connections 2024-09-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.875+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/795195258 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c900dc0 0x55c17c9054b0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.875+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/795195258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c9004a0 0x55c17c9209e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.875+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/795195258 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c8fca30 0x55c17c9204a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.875+0000 7f1c0caa3740 1 -- 172.21.15.152:0/795195258 >> 172.21.15.152:0/795195258 conn(0x55c17c9088d0 msgr2=0x55c17c90a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.875+0000 7f1c0caa3740 1 -- 172.21.15.152:0/795195258 shutdown_connections 2024-09-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.875+0000 7f1c0caa3740 1 -- 172.21.15.152:0/795195258 wait complete. 2024-09-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.876+0000 7f1c0caa3740 1 -- start start 2024-09-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.876+0000 7f1c0caa3740 1 Processor -- start 2024-09-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.876+0000 7f1c0caa3740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c8fca30 0x55c17c979950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.876+0000 7f1c0caa3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c9004a0 0x55c17c979e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.876+0000 7f1c0caa3740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c900dc0 0x55c17c9849d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.876+0000 7f1c0caa3740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041b0ee0 con 0x55c17c900dc0 2024-09-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.876+0000 7f1c0caa3740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041b0810 con 0x55c17c8fca30 2024-09-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.876+0000 7f1c0caa3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1c041ada70 con 0x55c17c9004a0 2024-09-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.876+0000 7f1c0caa3740 10 client.0 _ll_register_callbacks cb 0x7f1bf80092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.876+0000 7f1c08c67640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c900dc0 0x55c17c9849d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.876+0000 7f1c08c67640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c900dc0 0x55c17c9849d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35550/0 (socket says 172.21.15.152:35550) 2024-09-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.876+0000 7f1c08c67640 1 -- 172.21.15.152:0/801971867 learned_addr learned my addr 172.21.15.152:0/801971867 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.876+0000 7f1c08c67640 1 --2- 172.21.15.152:0/801971867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c8fca30 0x55c17c979950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.876+0000 7f1c037fe640 1 --2- 172.21.15.152:0/801971867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c9004a0 0x55c17c979e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1c08c67640 1 -- 172.21.15.152:0/801971867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c9004a0 msgr2=0x55c17c979e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1c08c67640 1 --2- 172.21.15.152:0/801971867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c9004a0 0x55c17c979e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1c08c67640 1 -- 172.21.15.152:0/801971867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c8fca30 msgr2=0x55c17c979950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1c08c67640 1 --2- 172.21.15.152:0/801971867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c8fca30 0x55c17c979950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1c08c67640 1 -- 172.21.15.152:0/801971867 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c17c984f10 con 0x55c17c900dc0 2024-09-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1c037fe640 1 --2- 172.21.15.152:0/801971867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c9004a0 0x55c17c979e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1c08c67640 1 --2- 172.21.15.152:0/801971867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c900dc0 0x55c17c9849d0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc02b880 tx=0x7f1bfc02bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0497d9490bb2ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1be37fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1be37fe640 1 -- 172.21.15.152:0/801971867 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc028040 con 0x55c17c900dc0 2024-09-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1be37fe640 1 -- 172.21.15.152:0/801971867 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f1bfc02b540 con 0x55c17c900dc0 2024-09-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1c0caa3740 1 -- 172.21.15.152:0/801971867 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bfc02b390 con 0x55c17c900dc0 2024-09-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1be37fe640 1 -- 172.21.15.152:0/801971867 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1bfc00e670 con 0x55c17c900dc0 2024-09-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1c0caa3740 10 client.18270 fetch_fsmap learned FSMap version 28 2024-09-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1c0caa3740 1 -- 172.21.15.152:0/801971867 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c17c986090 con 0x55c17c900dc0 2024-09-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1be37fe640 1 -- 172.21.15.152:0/801971867 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bfc02b390 con 0x55c17c900dc0 2024-09-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1be37fe640 1 -- 172.21.15.152:0/801971867 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1bfc0387a0 con 0x55c17c900dc0 2024-09-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1c0caa3740 10 client.18270 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1c0caa3740 10 client.18270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1c0caa3740 20 client.18270 populate_metadata read hostname 'smithi152' 2024-09-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/801971867 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c17c90b680 0x55c17c9886a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1c0caa3740 4 client.18270 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.877+0000 7f1c0caa3740 1 -- 172.21.15.152:0/801971867 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c17c86dd60 con 0x55c17c90b680 2024-09-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.878+0000 7f1c037fe640 1 --2- 172.21.15.152:0/801971867 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c17c90b680 0x55c17c9886a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.878+0000 7f1c037fe640 1 --2- 172.21.15.152:0/801971867 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c17c90b680 0x55c17c9886a0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9de8e8a1d4609259 server_cookie=371fc2b1afd0194c in_seq=0 out_seq=0 2024-09-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.878+0000 7f1be37fe640 10 client.18270 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.878+0000 7f1be37fe640 1 -- 172.21.15.152:0/801971867 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c17c86dd60 con 0x55c17c90b680 2024-09-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.878+0000 7f1be37fe640 10 client.18270 handle_command_reply: tid=0 2024-09-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.878+0000 7f1c0caa3740 1 client.18270 shutdown 2024-09-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0caa3740 1 -- 172.21.15.152:0/801971867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c900dc0 msgr2=0x55c17c9849d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/801971867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c900dc0 0x55c17c9849d0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc02b880 tx=0x7f1bfc02bca0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0caa3740 1 -- 172.21.15.152:0/801971867 shutdown_connections 2024-09-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/801971867 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c17c900dc0 0x55c17c9849d0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/801971867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c17c9004a0 0x55c17c979e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/801971867 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c17c90b680 0x55c17c9886a0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0caa3740 1 --2- 172.21.15.152:0/801971867 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c17c8fca30 0x55c17c979950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0caa3740 1 -- 172.21.15.152:0/801971867 >> 172.21.15.152:0/801971867 conn(0x55c17c9088d0 msgr2=0x55c17c8fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1be37fe640 0 client.18270 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0caa3740 1 -- 172.21.15.152:0/801971867 shutdown_connections 2024-09-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0caa3740 1 -- 172.21.15.152:0/801971867 wait complete. 2024-09-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0caa3740 20 client.18270 trim_cache size 0 max 16384 2024-09-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: "status": "no active scrubs running", 2024-09-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: "scrubs": {} 2024-09-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0a6ca640 1 -- 172.21.15.152:0/1250885065 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1bec0a2cf0 msgr2=0x7f1bec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0a6ca640 1 --2- 172.21.15.152:0/1250885065 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1bec0a2cf0 0x7f1bec0a51b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1bf40ed210 tx=0x7f1bf402a410 comp rx=0 tx=0).stop 2024-09-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0a6ca640 1 -- 172.21.15.152:0/1250885065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c04101de0 msgr2=0x7f1c040744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0a6ca640 1 --2- 172.21.15.152:0/1250885065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c04101de0 0x7f1c040744f0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4006010 tx=0x7f1bf40042f0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0a6ca640 1 -- 172.21.15.152:0/1250885065 shutdown_connections 2024-09-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0a6ca640 1 --2- 172.21.15.152:0/1250885065 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1bec0a2cf0 0x7f1bec0a51b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0a6ca640 1 --2- 172.21.15.152:0/1250885065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1c04106f10 0x7f1c04074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0a6ca640 1 --2- 172.21.15.152:0/1250885065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c04102790 0x7f1c04074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0a6ca640 1 --2- 172.21.15.152:0/1250885065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1c04101de0 0x7f1c040744f0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0a6ca640 1 -- 172.21.15.152:0/1250885065 >> 172.21.15.152:0/1250885065 conn(0x7f1c040fd8c0 msgr2=0x7f1c040fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.879+0000 7f1c0a6ca640 1 -- 172.21.15.152:0/1250885065 shutdown_connections 2024-09-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:02:58.880+0000 7f1c0a6ca640 1 -- 172.21.15.152:0/1250885065 wait complete. 2024-09-11T23:02:59.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:02:58 smithi152 ceph-mon[31181]: pgmap v311: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T23:02:59.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:02:58 smithi191 ceph-mon[35211]: pgmap v311: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T23:02:59.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:02:58 smithi204 ceph-mon[35454]: pgmap v311: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T23:02:59.735 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T23:02:59.735 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:02:59.986 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:00.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.511+0000 7fbc4564c640 1 -- 172.21.15.152:0/2491615239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc40106f50 msgr2=0x7fbc4010eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:00.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.511+0000 7fbc4564c640 1 --2- 172.21.15.152:0/2491615239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc40106f50 0x7fbc4010eaf0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009930 tx=0x7fbc3802a420 comp rx=0 tx=0).stop 2024-09-11T23:03:00.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.511+0000 7fbc4564c640 1 -- 172.21.15.152:0/2491615239 shutdown_connections 2024-09-11T23:03:00.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.511+0000 7fbc4564c640 1 --2- 172.21.15.152:0/2491615239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc40106f50 0x7fbc4010eaf0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:00.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.511+0000 7fbc4564c640 1 --2- 172.21.15.152:0/2491615239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc40100d00 0x7fbc40106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:00.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.511+0000 7fbc4564c640 1 --2- 172.21.15.152:0/2491615239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc40100350 0x7fbc40100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:00.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.511+0000 7fbc4564c640 1 -- 172.21.15.152:0/2491615239 >> 172.21.15.152:0/2491615239 conn(0x7fbc40077210 msgr2=0x7fbc400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:00.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.511+0000 7fbc4564c640 1 -- 172.21.15.152:0/2491615239 shutdown_connections 2024-09-11T23:03:00.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.511+0000 7fbc4564c640 1 -- 172.21.15.152:0/2491615239 wait complete. 2024-09-11T23:03:00.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.511+0000 7fbc4564c640 1 Processor -- start 2024-09-11T23:03:00.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc4564c640 1 -- start start 2024-09-11T23:03:00.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc4564c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc40100350 0x7fbc401984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:00.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc4564c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc40100d00 0x7fbc40198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:00.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc4564c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc40106f50 0x7fbc4019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:00.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc4564c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40112000 con 0x7fbc40106f50 2024-09-11T23:03:00.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc4564c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40111e80 con 0x7fbc40100d00 2024-09-11T23:03:00.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc4564c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fbc40112180 con 0x7fbc40100350 2024-09-11T23:03:00.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc3f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc40106f50 0x7fbc4019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:00.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc3f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc40106f50 0x7fbc4019cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35566/0 (socket says 172.21.15.152:35566) 2024-09-11T23:03:00.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc3f7fe640 1 -- 172.21.15.152:0/1255382854 learned_addr learned my addr 172.21.15.152:0/1255382854 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:00.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc3effd640 1 --2- 172.21.15.152:0/1255382854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc40100350 0x7fbc401984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:00.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc3e7fc640 1 --2- 172.21.15.152:0/1255382854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc40100d00 0x7fbc40198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:00.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc3f7fe640 1 -- 172.21.15.152:0/1255382854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc40100350 msgr2=0x7fbc401984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:00.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc3f7fe640 1 --2- 172.21.15.152:0/1255382854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc40100350 0x7fbc401984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:00.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc3f7fe640 1 -- 172.21.15.152:0/1255382854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc40100d00 msgr2=0x7fbc40198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:00.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc3f7fe640 1 --2- 172.21.15.152:0/1255382854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc40100d00 0x7fbc40198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:00.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc3f7fe640 1 -- 172.21.15.152:0/1255382854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4019d4a0 con 0x7fbc40106f50 2024-09-11T23:03:00.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.512+0000 7fbc3f7fe640 1 --2- 172.21.15.152:0/1255382854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc40106f50 0x7fbc4019cdc0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fbc38005ab0 tx=0x7fbc380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:00.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.513+0000 7fbc27fff640 1 -- 172.21.15.152:0/1255382854 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3803f030 con 0x7fbc40106f50 2024-09-11T23:03:00.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.513+0000 7fbc4564c640 1 -- 172.21.15.152:0/1255382854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4019d730 con 0x7fbc40106f50 2024-09-11T23:03:00.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.513+0000 7fbc27fff640 1 -- 172.21.15.152:0/1255382854 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc38003ce0 con 0x7fbc40106f50 2024-09-11T23:03:00.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.513+0000 7fbc27fff640 1 -- 172.21.15.152:0/1255382854 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38035070 con 0x7fbc40106f50 2024-09-11T23:03:00.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.513+0000 7fbc4564c640 1 -- 172.21.15.152:0/1255382854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc401a5070 con 0x7fbc40106f50 2024-09-11T23:03:00.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.514+0000 7fbc4564c640 1 -- 172.21.15.152:0/1255382854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc40103410 con 0x7fbc40106f50 2024-09-11T23:03:00.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.515+0000 7fbc27fff640 1 -- 172.21.15.152:0/1255382854 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fbc38042030 con 0x7fbc40106f50 2024-09-11T23:03:00.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.516+0000 7fbc27fff640 1 --2- 172.21.15.152:0/1255382854 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbc280a2cf0 0x7fbc280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:00.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.516+0000 7fbc3effd640 1 --2- 172.21.15.152:0/1255382854 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbc280a2cf0 0x7fbc280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:00.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.516+0000 7fbc27fff640 1 -- 172.21.15.152:0/1255382854 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7fbc380efa80 con 0x7fbc40106f50 2024-09-11T23:03:00.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.516+0000 7fbc3effd640 1 --2- 172.21.15.152:0/1255382854 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbc280a2cf0 0x7fbc280a51b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbc30007350 tx=0x7fbc300072e0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:00.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.521+0000 7fbc27fff640 1 -- 172.21.15.152:0/1255382854 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc380bc6c0 con 0x7fbc40106f50 2024-09-11T23:03:00.736 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.735+0000 7fbc4564c640 1 -- 172.21.15.152:0/1255382854 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc4010a120 con 0x7fbc40106f50 2024-09-11T23:03:00.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.736+0000 7fbc27fff640 1 -- 172.21.15.152:0/1255382854 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7fbc380bbfc0 con 0x7fbc40106f50 2024-09-11T23:03:00.737 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T23:03:00.738 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T23:03:00.738 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T23:03:00.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.738+0000 7fbc4564c640 1 -- 172.21.15.152:0/1255382854 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbc280a2cf0 msgr2=0x7fbc280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:00.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.738+0000 7fbc4564c640 1 --2- 172.21.15.152:0/1255382854 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbc280a2cf0 0x7fbc280a51b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbc30007350 tx=0x7fbc300072e0 comp rx=0 tx=0).stop 2024-09-11T23:03:00.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.738+0000 7fbc4564c640 1 -- 172.21.15.152:0/1255382854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc40106f50 msgr2=0x7fbc4019cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:00.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.738+0000 7fbc4564c640 1 --2- 172.21.15.152:0/1255382854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc40106f50 0x7fbc4019cdc0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fbc38005ab0 tx=0x7fbc380042f0 comp rx=0 tx=0).stop 2024-09-11T23:03:00.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.738+0000 7fbc4564c640 1 -- 172.21.15.152:0/1255382854 shutdown_connections 2024-09-11T23:03:00.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.738+0000 7fbc4564c640 1 --2- 172.21.15.152:0/1255382854 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fbc280a2cf0 0x7fbc280a51b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:00.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.738+0000 7fbc4564c640 1 --2- 172.21.15.152:0/1255382854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc40106f50 0x7fbc4019cdc0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:00.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.738+0000 7fbc4564c640 1 --2- 172.21.15.152:0/1255382854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbc40100d00 0x7fbc40198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:00.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.738+0000 7fbc4564c640 1 --2- 172.21.15.152:0/1255382854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc40100350 0x7fbc401984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:00.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.738+0000 7fbc4564c640 1 -- 172.21.15.152:0/1255382854 >> 172.21.15.152:0/1255382854 conn(0x7fbc40077210 msgr2=0x7fbc400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:00.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.738+0000 7fbc4564c640 1 -- 172.21.15.152:0/1255382854 shutdown_connections 2024-09-11T23:03:00.740 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:00.738+0000 7fbc4564c640 1 -- 172.21.15.152:0/1255382854 wait complete. 2024-09-11T23:03:01.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:00 smithi152 ceph-mon[31181]: pgmap v312: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-11T23:03:01.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:00 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1255382854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:03:01.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:00 smithi191 ceph-mon[35211]: pgmap v312: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-11T23:03:01.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:00 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1255382854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:03:01.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:00 smithi204 ceph-mon[35454]: pgmap v312: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-11T23:03:01.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:00 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1255382854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:03:01.494 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T23:03:01.739 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:02.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.266+0000 7fb991cf2640 1 -- 172.21.15.152:0/2723117388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb98c103660 msgr2=0x7fb98c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:02.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.266+0000 7fb991cf2640 1 --2- 172.21.15.152:0/2723117388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb98c103660 0x7fb98c10bb50 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fb974009900 tx=0x7fb97402a480 comp rx=0 tx=0).stop 2024-09-11T23:03:02.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.267+0000 7fb991cf2640 1 -- 172.21.15.152:0/2723117388 shutdown_connections 2024-09-11T23:03:02.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.267+0000 7fb991cf2640 1 --2- 172.21.15.152:0/2723117388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb98c10c170 0x7fb98c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.267+0000 7fb991cf2640 1 --2- 172.21.15.152:0/2723117388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb98c103660 0x7fb98c10bb50 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.267+0000 7fb991cf2640 1 --2- 172.21.15.152:0/2723117388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb98c102d40 0x7fb98c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.267+0000 7fb991cf2640 1 -- 172.21.15.152:0/2723117388 >> 172.21.15.152:0/2723117388 conn(0x7fb98c0fc7a0 msgr2=0x7fb98c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:02.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.267+0000 7fb991cf2640 1 -- 172.21.15.152:0/2723117388 shutdown_connections 2024-09-11T23:03:02.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.267+0000 7fb991cf2640 1 -- 172.21.15.152:0/2723117388 wait complete. 2024-09-11T23:03:02.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.267+0000 7fb991cf2640 1 Processor -- start 2024-09-11T23:03:02.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.267+0000 7fb991cf2640 1 -- start start 2024-09-11T23:03:02.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb991cf2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb98c102d40 0x7fb98c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb991cf2640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb98c103660 0x7fb98c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb991cf2640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb98c10c170 0x7fb98c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb991cf2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c111f70 con 0x7fb98c102d40 2024-09-11T23:03:02.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb991cf2640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c111df0 con 0x7fb98c10c170 2024-09-11T23:03:02.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb991cf2640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c1120f0 con 0x7fb98c103660 2024-09-11T23:03:02.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb98b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb98c102d40 0x7fb98c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb98b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb98c102d40 0x7fb98c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35572/0 (socket says 172.21.15.152:35572) 2024-09-11T23:03:02.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb98b7fe640 1 -- 172.21.15.152:0/2293128883 learned_addr learned my addr 172.21.15.152:0/2293128883 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:02.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb98affd640 1 --2- 172.21.15.152:0/2293128883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb98c103660 0x7fb98c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb98bfff640 1 --2- 172.21.15.152:0/2293128883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb98c10c170 0x7fb98c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb98b7fe640 1 -- 172.21.15.152:0/2293128883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb98c103660 msgr2=0x7fb98c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:02.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb98b7fe640 1 --2- 172.21.15.152:0/2293128883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb98c103660 0x7fb98c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb98b7fe640 1 -- 172.21.15.152:0/2293128883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb98c10c170 msgr2=0x7fb98c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:02.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb98b7fe640 1 --2- 172.21.15.152:0/2293128883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb98c10c170 0x7fb98c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb98b7fe640 1 -- 172.21.15.152:0/2293128883 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c19d3e0 con 0x7fb98c102d40 2024-09-11T23:03:02.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb98bfff640 1 --2- 172.21.15.152:0/2293128883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb98c10c170 0x7fb98c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:03:02.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb98b7fe640 1 --2- 172.21.15.152:0/2293128883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb98c102d40 0x7fb98c198430 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00d950 tx=0x7fb97c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:02.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb988ff9640 1 -- 172.21.15.152:0/2293128883 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c017030 con 0x7fb98c102d40 2024-09-11T23:03:02.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.268+0000 7fb991cf2640 1 -- 172.21.15.152:0/2293128883 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98c19d6d0 con 0x7fb98c102d40 2024-09-11T23:03:02.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.269+0000 7fb991cf2640 1 -- 172.21.15.152:0/2293128883 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98c1a4fb0 con 0x7fb98c102d40 2024-09-11T23:03:02.287 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.269+0000 7fb988ff9640 1 -- 172.21.15.152:0/2293128883 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb97c004020 con 0x7fb98c102d40 2024-09-11T23:03:02.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.269+0000 7fb988ff9640 1 -- 172.21.15.152:0/2293128883 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c004dd0 con 0x7fb98c102d40 2024-09-11T23:03:02.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.272+0000 7fb988ff9640 1 -- 172.21.15.152:0/2293128883 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb97c005020 con 0x7fb98c102d40 2024-09-11T23:03:02.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.272+0000 7fb9940cb740 1 Processor -- start 2024-09-11T23:03:02.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.272+0000 7fb9940cb740 1 -- start start 2024-09-11T23:03:02.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.272+0000 7fb9940cb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d9b40 0x55b61d2d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.272+0000 7fb9940cb740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2d85d0 0x55b61d2d8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.272+0000 7fb9940cb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2e3370 0x55b61d2e5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.272+0000 7fb9940cb740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55b61d23e2f0 con 0x55b61d2d85d0 2024-09-11T23:03:02.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.272+0000 7fb9940cb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b61d1e5cf0 con 0x55b61d2d9b40 2024-09-11T23:03:02.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.272+0000 7fb9940cb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b61d23a2b0 con 0x55b61d2e3370 2024-09-11T23:03:02.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.272+0000 7fb98b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2d85d0 0x55b61d2d8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.272+0000 7fb98b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2d85d0 0x55b61d2d8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35576/0 (socket says 172.21.15.152:35576) 2024-09-11T23:03:02.288 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.272+0000 7fb98b7fe640 1 -- 172.21.15.152:0/3415917517 learned_addr learned my addr 172.21.15.152:0/3415917517 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:02.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.272+0000 7fb98bfff640 1 --2- 172.21.15.152:0/3415917517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d9b40 0x55b61d2d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.272+0000 7fb98affd640 1 --2- 172.21.15.152:0/3415917517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2e3370 0x55b61d2e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.273+0000 7fb988ff9640 1 --2- 172.21.15.152:0/2293128883 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9780a2d70 0x7fb9780a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.273+0000 7fb98bfff640 1 --2- 172.21.15.152:0/2293128883 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9780a2d70 0x7fb9780a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.273+0000 7fb988ff9640 1 -- 172.21.15.152:0/2293128883 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7fb97c0cee50 con 0x7fb98c102d40 2024-09-11T23:03:02.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.273+0000 7fb98b7fe640 1 -- 172.21.15.152:0/3415917517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2e3370 msgr2=0x55b61d2e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:02.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.273+0000 7fb98b7fe640 1 --2- 172.21.15.152:0/3415917517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2e3370 0x55b61d2e5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.273+0000 7fb98b7fe640 1 -- 172.21.15.152:0/3415917517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d9b40 msgr2=0x55b61d2d8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:02.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.273+0000 7fb98b7fe640 1 --2- 172.21.15.152:0/3415917517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d9b40 0x55b61d2d8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.273+0000 7fb98b7fe640 1 -- 172.21.15.152:0/3415917517 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b61d1ca3a0 con 0x55b61d2d85d0 2024-09-11T23:03:02.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.273+0000 7fb98affd640 1 --2- 172.21.15.152:0/3415917517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2e3370 0x55b61d2e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:03:02.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.273+0000 7fb98b7fe640 1 --2- 172.21.15.152:0/3415917517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2d85d0 0x55b61d2d8a10 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fb97c09b090 tx=0x7fb97c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e605b41568da8b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:02.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.273+0000 7fb98bfff640 1 --2- 172.21.15.152:0/2293128883 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9780a2d70 0x7fb9780a5230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb984005fd0 tx=0x7fb984004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:02.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.273+0000 7fb981ffb640 1 -- 172.21.15.152:0/3415917517 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c09ba90 con 0x55b61d2d85d0 2024-09-11T23:03:02.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.273+0000 7fb981ffb640 1 -- 172.21.15.152:0/3415917517 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb97c0d24b0 con 0x55b61d2d85d0 2024-09-11T23:03:02.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.273+0000 7fb981ffb640 1 -- 172.21.15.152:0/3415917517 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c0cc840 con 0x55b61d2d85d0 2024-09-11T23:03:02.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.274+0000 7fb9940cb740 1 -- 172.21.15.152:0/3415917517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2d85d0 msgr2=0x55b61d2d8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:02.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.274+0000 7fb9940cb740 1 --2- 172.21.15.152:0/3415917517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2d85d0 0x55b61d2d8a10 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fb97c09b090 tx=0x7fb97c000bc0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.274+0000 7fb9940cb740 1 -- 172.21.15.152:0/3415917517 shutdown_connections 2024-09-11T23:03:02.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.274+0000 7fb9940cb740 1 --2- 172.21.15.152:0/3415917517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2e3370 0x55b61d2e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.274+0000 7fb9940cb740 1 --2- 172.21.15.152:0/3415917517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2d85d0 0x55b61d2d8a10 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.274+0000 7fb9940cb740 1 --2- 172.21.15.152:0/3415917517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d9b40 0x55b61d2d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.274+0000 7fb9940cb740 1 -- 172.21.15.152:0/3415917517 >> 172.21.15.152:0/3415917517 conn(0x55b61d2a1310 msgr2=0x55b61d2d4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:02.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.274+0000 7fb9940cb740 1 -- 172.21.15.152:0/3415917517 shutdown_connections 2024-09-11T23:03:02.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.274+0000 7fb9940cb740 1 -- 172.21.15.152:0/3415917517 wait complete. 2024-09-11T23:03:02.290 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.275+0000 7fb9940cb740 1 -- start start 2024-09-11T23:03:02.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.275+0000 7fb9940cb740 1 Processor -- start 2024-09-11T23:03:02.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.275+0000 7fb9940cb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d85d0 0x55b61d2e2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.275+0000 7fb9940cb740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2d9b40 0x55b61d2e0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.275+0000 7fb9940cb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2e3370 0x55b61d2e0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.275+0000 7fb9940cb740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55b61d2e5fb0 con 0x55b61d2d9b40 2024-09-11T23:03:02.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.275+0000 7fb9940cb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b61d1f9060 con 0x55b61d2d85d0 2024-09-11T23:03:02.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.275+0000 7fb9940cb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b61d2e6130 con 0x55b61d2e3370 2024-09-11T23:03:02.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.275+0000 7fb9940cb740 10 client.0 _ll_register_callbacks cb 0x55b61cf4c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:03:02.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.275+0000 7fb98affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2d9b40 0x55b61d2e0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.275+0000 7fb98affd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2d9b40 0x55b61d2e0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35580/0 (socket says 172.21.15.152:35580) 2024-09-11T23:03:02.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.275+0000 7fb98affd640 1 -- 172.21.15.152:0/3105193466 learned_addr learned my addr 172.21.15.152:0/3105193466 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:02.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.275+0000 7fb98b7fe640 1 --2- 172.21.15.152:0/3105193466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d85d0 0x55b61d2e2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.291 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.276+0000 7fb98affd640 1 -- 172.21.15.152:0/3105193466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2e3370 msgr2=0x55b61d2e0610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T23:03:02.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.276+0000 7fb98affd640 1 --2- 172.21.15.152:0/3105193466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2e3370 0x55b61d2e0610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.276+0000 7fb98affd640 1 -- 172.21.15.152:0/3105193466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d85d0 msgr2=0x55b61d2e2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:02.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.276+0000 7fb98affd640 1 --2- 172.21.15.152:0/3105193466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d85d0 0x55b61d2e2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.276+0000 7fb98affd640 1 -- 172.21.15.152:0/3105193466 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b61cfaf3c0 con 0x55b61d2d9b40 2024-09-11T23:03:02.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.276+0000 7fb98b7fe640 1 --2- 172.21.15.152:0/3105193466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d85d0 0x55b61d2e2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:03:02.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.276+0000 7fb98affd640 1 --2- 172.21.15.152:0/3105193466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2d9b40 0x55b61d2e0020 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fb974009900 tx=0x7fb974031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b3bf545ede4963 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:02.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.276+0000 7fb96affd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:03:02.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.276+0000 7fb96affd640 1 -- 172.21.15.152:0/3105193466 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974035040 con 0x55b61d2d9b40 2024-09-11T23:03:02.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.276+0000 7fb96affd640 1 -- 172.21.15.152:0/3105193466 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7fb974004240 con 0x55b61d2d9b40 2024-09-11T23:03:02.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.276+0000 7fb9940cb740 1 -- 172.21.15.152:0/3105193466 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb97c0d24b0 con 0x55b61d2d9b40 2024-09-11T23:03:02.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.276+0000 7fb96affd640 1 -- 172.21.15.152:0/3105193466 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb974004440 con 0x55b61d2d9b40 2024-09-11T23:03:02.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.276+0000 7fb9940cb740 10 client.18288 fetch_fsmap learned FSMap version 28 2024-09-11T23:03:02.292 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.276+0000 7fb9940cb740 1 -- 172.21.15.152:0/3105193466 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b61d2f1ae0 con 0x55b61d2d9b40 2024-09-11T23:03:02.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.276+0000 7fb96affd640 1 -- 172.21.15.152:0/3105193466 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb97c0d24b0 con 0x55b61d2d9b40 2024-09-11T23:03:02.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.277+0000 7fb96affd640 1 -- 172.21.15.152:0/3105193466 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb9740035f0 con 0x55b61d2d9b40 2024-09-11T23:03:02.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.277+0000 7fb9940cb740 10 client.18288 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:03:02.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.277+0000 7fb9940cb740 10 client.18288 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T23:03:02.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.277+0000 7fb9940cb740 20 client.18288 populate_metadata read hostname 'smithi152' 2024-09-11T23:03:02.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.277+0000 7fb9940cb740 1 --2- 172.21.15.152:0/3105193466 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55b61d2f3040 0x55b61d2f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.277+0000 7fb9940cb740 4 client.18288 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:03:02.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.277+0000 7fb9940cb740 1 -- 172.21.15.152:0/3105193466 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b61d24b860 con 0x55b61d2f3040 2024-09-11T23:03:02.293 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.277+0000 7fb98b7fe640 1 --2- 172.21.15.152:0/3105193466 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55b61d2f3040 0x55b61d2f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.277+0000 7fb96affd640 10 client.18288 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T23:03:02.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.277+0000 7fb98b7fe640 1 --2- 172.21.15.152:0/3105193466 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55b61d2f3040 0x55b61d2f54a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4035580a5b79d959 server_cookie=cfee5eb515616518 in_seq=0 out_seq=0 2024-09-11T23:03:02.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.278+0000 7fb96affd640 1 -- 172.21.15.152:0/3105193466 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b61d24b860 con 0x55b61d2f3040 2024-09-11T23:03:02.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.278+0000 7fb96affd640 10 client.18288 handle_command_reply: tid=0 2024-09-11T23:03:02.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.278+0000 7fb9940cb740 1 client.18288 shutdown 2024-09-11T23:03:02.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.278+0000 7fb9940cb740 1 -- 172.21.15.152:0/3105193466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2d9b40 msgr2=0x55b61d2e0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:02.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.278+0000 7fb9940cb740 1 --2- 172.21.15.152:0/3105193466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2d9b40 0x55b61d2e0020 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fb974009900 tx=0x7fb974031850 comp rx=0 tx=0).stop 2024-09-11T23:03:02.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.278+0000 7fb9940cb740 1 -- 172.21.15.152:0/3105193466 shutdown_connections 2024-09-11T23:03:02.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.278+0000 7fb9940cb740 1 --2- 172.21.15.152:0/3105193466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2e3370 0x55b61d2e0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.278+0000 7fb9940cb740 1 --2- 172.21.15.152:0/3105193466 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2d9b40 0x55b61d2e0020 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.278+0000 7fb9940cb740 1 --2- 172.21.15.152:0/3105193466 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55b61d2f3040 0x55b61d2f54a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.278+0000 7fb9940cb740 1 --2- 172.21.15.152:0/3105193466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d85d0 0x55b61d2e2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.278+0000 7fb9940cb740 1 -- 172.21.15.152:0/3105193466 >> 172.21.15.152:0/3105193466 conn(0x55b61d2a1310 msgr2=0x55b61d2da990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:02.294 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.278+0000 7fb96affd640 0 client.18288 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T23:03:02.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.278+0000 7fb9940cb740 1 -- 172.21.15.152:0/3105193466 shutdown_connections 2024-09-11T23:03:02.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.278+0000 7fb9940cb740 1 -- 172.21.15.152:0/3105193466 wait complete. 2024-09-11T23:03:02.295 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.278+0000 7fb9940cb740 20 client.18288 trim_cache size 0 max 16384 2024-09-11T23:03:02.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.292+0000 7fb9940cb740 1 Processor -- start 2024-09-11T23:03:02.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.292+0000 7fb9940cb740 1 -- start start 2024-09-11T23:03:02.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.292+0000 7fb9940cb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2d85d0 0x55b61d2d4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.292+0000 7fb9940cb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d9b40 0x55b61d2ef2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.292+0000 7fb9940cb740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2e3370 0x55b61d2eb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.292+0000 7fb9940cb740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c00e6e0 con 0x55b61d2e3370 2024-09-11T23:03:02.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.292+0000 7fb9940cb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c010220 con 0x55b61d2d9b40 2024-09-11T23:03:02.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.292+0000 7fb9940cb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c00edb0 con 0x55b61d2d85d0 2024-09-11T23:03:02.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.292+0000 7fb98bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2e3370 0x55b61d2eb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.292+0000 7fb98bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2e3370 0x55b61d2eb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35594/0 (socket says 172.21.15.152:35594) 2024-09-11T23:03:02.300 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.292+0000 7fb98affd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d9b40 0x55b61d2ef2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.292+0000 7fb98bfff640 1 -- 172.21.15.152:0/44586331 learned_addr learned my addr 172.21.15.152:0/44586331 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:02.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.293+0000 7fb98b7fe640 1 --2- 172.21.15.152:0/44586331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2d85d0 0x55b61d2d4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.293+0000 7fb98bfff640 1 -- 172.21.15.152:0/44586331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2d85d0 msgr2=0x55b61d2d4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:02.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.293+0000 7fb98bfff640 1 --2- 172.21.15.152:0/44586331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2d85d0 0x55b61d2d4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.293+0000 7fb98bfff640 1 -- 172.21.15.152:0/44586331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d9b40 msgr2=0x55b61d2ef2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:02.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.293+0000 7fb98bfff640 1 --2- 172.21.15.152:0/44586331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d9b40 0x55b61d2ef2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.293+0000 7fb98bfff640 1 -- 172.21.15.152:0/44586331 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b61d2ef7f0 con 0x55b61d2e3370 2024-09-11T23:03:02.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.293+0000 7fb98affd640 1 --2- 172.21.15.152:0/44586331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d9b40 0x55b61d2ef2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:03:02.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.293+0000 7fb98bfff640 1 --2- 172.21.15.152:0/44586331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2e3370 0x55b61d2eb800 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fb984009800 tx=0x7fb984009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6e6aed0ea3e5f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:02.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.293+0000 7fb9817fa640 1 -- 172.21.15.152:0/44586331 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984026030 con 0x55b61d2e3370 2024-09-11T23:03:02.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.293+0000 7fb9817fa640 1 -- 172.21.15.152:0/44586331 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb984011e40 con 0x55b61d2e3370 2024-09-11T23:03:02.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.293+0000 7fb9817fa640 1 -- 172.21.15.152:0/44586331 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984022e20 con 0x55b61d2e3370 2024-09-11T23:03:02.301 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.294+0000 7fb9940cb740 1 -- 172.21.15.152:0/44586331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2e3370 msgr2=0x55b61d2eb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:02.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.294+0000 7fb9940cb740 1 --2- 172.21.15.152:0/44586331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2e3370 0x55b61d2eb800 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fb984009800 tx=0x7fb984009880 comp rx=0 tx=0).stop 2024-09-11T23:03:02.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.294+0000 7fb9940cb740 1 -- 172.21.15.152:0/44586331 shutdown_connections 2024-09-11T23:03:02.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.294+0000 7fb9940cb740 1 --2- 172.21.15.152:0/44586331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2e3370 0x55b61d2eb800 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.294+0000 7fb9940cb740 1 --2- 172.21.15.152:0/44586331 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d9b40 0x55b61d2ef2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.294+0000 7fb9940cb740 1 --2- 172.21.15.152:0/44586331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2d85d0 0x55b61d2d4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.294+0000 7fb9940cb740 1 -- 172.21.15.152:0/44586331 >> 172.21.15.152:0/44586331 conn(0x55b61d2a1310 msgr2=0x55b61d2d4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:02.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.294+0000 7fb9940cb740 1 -- 172.21.15.152:0/44586331 shutdown_connections 2024-09-11T23:03:02.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.294+0000 7fb9940cb740 1 -- 172.21.15.152:0/44586331 wait complete. 2024-09-11T23:03:02.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.294+0000 7fb9940cb740 1 -- start start 2024-09-11T23:03:02.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb9940cb740 1 Processor -- start 2024-09-11T23:03:02.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb9940cb740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d85d0 0x55b61d2ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb9940cb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2d9b40 0x55b61d2dc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb9940cb740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2e3370 0x55b61d2dcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.302 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb9940cb740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c00fb50 con 0x55b61d2e3370 2024-09-11T23:03:02.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb9940cb740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c00f480 con 0x55b61d2d85d0 2024-09-11T23:03:02.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb9940cb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c00c6e0 con 0x55b61d2d9b40 2024-09-11T23:03:02.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb9940cb740 10 client.0 _ll_register_callbacks cb 0x7fb974009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:03:02.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb98bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2e3370 0x55b61d2dcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb98b7fe640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d85d0 0x55b61d2ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb98bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2e3370 0x55b61d2dcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35606/0 (socket says 172.21.15.152:35606) 2024-09-11T23:03:02.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb98bfff640 1 -- 172.21.15.152:0/906828430 learned_addr learned my addr 172.21.15.152:0/906828430 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:02.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb98affd640 1 --2- 172.21.15.152:0/906828430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2d9b40 0x55b61d2dc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb98bfff640 1 -- 172.21.15.152:0/906828430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2d9b40 msgr2=0x55b61d2dc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:02.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb98bfff640 1 --2- 172.21.15.152:0/906828430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2d9b40 0x55b61d2dc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb98bfff640 1 -- 172.21.15.152:0/906828430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d85d0 msgr2=0x55b61d2ddc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:02.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb98bfff640 1 --2- 172.21.15.152:0/906828430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d85d0 0x55b61d2ddc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.303 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb98bfff640 1 -- 172.21.15.152:0/906828430 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b61d2dd540 con 0x55b61d2e3370 2024-09-11T23:03:02.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.295+0000 7fb98b7fe640 1 --2- 172.21.15.152:0/906828430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d85d0 0x55b61d2ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:03:02.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.296+0000 7fb98bfff640 1 --2- 172.21.15.152:0/906828430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2e3370 0x55b61d2dcd10 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fb984009850 tx=0x7fb984009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76664d38933b62 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:02.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.296+0000 7fb96affd640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:03:02.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.296+0000 7fb96affd640 1 -- 172.21.15.152:0/906828430 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb984026040 con 0x55b61d2e3370 2024-09-11T23:03:02.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.296+0000 7fb96affd640 1 -- 172.21.15.152:0/906828430 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7fb984014040 con 0x55b61d2e3370 2024-09-11T23:03:02.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.296+0000 7fb9940cb740 1 -- 172.21.15.152:0/906828430 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb984011e40 con 0x55b61d2e3370 2024-09-11T23:03:02.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.296+0000 7fb96affd640 1 -- 172.21.15.152:0/906828430 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb984014260 con 0x55b61d2e3370 2024-09-11T23:03:02.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.296+0000 7fb9940cb740 10 client.18294 fetch_fsmap learned FSMap version 28 2024-09-11T23:03:02.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.296+0000 7fb9940cb740 1 -- 172.21.15.152:0/906828430 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b61d361e80 con 0x55b61d2e3370 2024-09-11T23:03:02.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.296+0000 7fb96affd640 1 -- 172.21.15.152:0/906828430 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb984011e40 con 0x55b61d2e3370 2024-09-11T23:03:02.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.296+0000 7fb96affd640 1 -- 172.21.15.152:0/906828430 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb98403b020 con 0x55b61d2e3370 2024-09-11T23:03:02.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.296+0000 7fb9940cb740 10 client.18294 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:03:02.304 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.296+0000 7fb9940cb740 10 client.18294 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-09-11T23:03:02.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.296+0000 7fb9940cb740 20 client.18294 populate_metadata read hostname 'smithi152' 2024-09-11T23:03:02.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.296+0000 7fb9940cb740 1 --2- 172.21.15.152:0/906828430 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55b61d362540 0x55b61d364960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:02.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.296+0000 7fb9940cb740 4 client.18294 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:03:02.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.296+0000 7fb9940cb740 1 -- 172.21.15.152:0/906828430 --> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b61d245d70 con 0x55b61d362540 2024-09-11T23:03:02.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.297+0000 7fb98b7fe640 1 --2- 172.21.15.152:0/906828430 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55b61d362540 0x55b61d364960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:02.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.297+0000 7fb96affd640 10 client.18294 ms_handle_connect on v2:172.21.15.152:6840/2492281944 2024-09-11T23:03:02.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.297+0000 7fb98b7fe640 1 --2- 172.21.15.152:0/906828430 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55b61d362540 0x55b61d364960 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=20221345b9b6c6d9 server_cookie=fcc66a4319bf176f in_seq=0 out_seq=0 2024-09-11T23:03:02.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.297+0000 7fb96affd640 1 -- 172.21.15.152:0/906828430 <== mds.4 v2:172.21.15.152:6840/2492281944 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b61d245d70 con 0x55b61d362540 2024-09-11T23:03:02.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.297+0000 7fb96affd640 10 client.18294 handle_command_reply: tid=0 2024-09-11T23:03:02.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.297+0000 7fb9940cb740 1 client.18294 shutdown 2024-09-11T23:03:02.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.297+0000 7fb9940cb740 1 -- 172.21.15.152:0/906828430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2e3370 msgr2=0x55b61d2dcd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:02.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.297+0000 7fb9940cb740 1 --2- 172.21.15.152:0/906828430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2e3370 0x55b61d2dcd10 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fb984009850 tx=0x7fb984009280 comp rx=0 tx=0).stop 2024-09-11T23:03:02.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.297+0000 7fb9940cb740 1 -- 172.21.15.152:0/906828430 shutdown_connections 2024-09-11T23:03:02.305 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.297+0000 7fb9940cb740 1 --2- 172.21.15.152:0/906828430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b61d2e3370 0x55b61d2dcd10 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.297+0000 7fb9940cb740 1 --2- 172.21.15.152:0/906828430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b61d2d9b40 0x55b61d2dc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.297+0000 7fb9940cb740 1 --2- 172.21.15.152:0/906828430 >> [v2:172.21.15.152:6840/2492281944,v1:172.21.15.152:6841/2492281944] conn(0x55b61d362540 0x55b61d364960 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.297+0000 7fb9940cb740 1 --2- 172.21.15.152:0/906828430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b61d2d85d0 0x55b61d2ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.297+0000 7fb96affd640 0 client.18294 ms_handle_reset on v2:172.21.15.152:6840/2492281944 2024-09-11T23:03:02.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.297+0000 7fb9940cb740 1 -- 172.21.15.152:0/906828430 >> 172.21.15.152:0/906828430 conn(0x55b61d2a1310 msgr2=0x55b61d2e1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:02.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.298+0000 7fb9940cb740 1 -- 172.21.15.152:0/906828430 shutdown_connections 2024-09-11T23:03:02.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.298+0000 7fb9940cb740 1 -- 172.21.15.152:0/906828430 wait complete. 2024-09-11T23:03:02.306 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.298+0000 7fb9940cb740 20 client.18294 trim_cache size 0 max 16384 2024-09-11T23:03:02.306 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:03:02.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.298+0000 7fb991cf2640 1 -- 172.21.15.152:0/2293128883 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9780a2d70 msgr2=0x7fb9780a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:02.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.298+0000 7fb991cf2640 1 --2- 172.21.15.152:0/2293128883 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9780a2d70 0x7fb9780a5230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb984005fd0 tx=0x7fb984004ab0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.298+0000 7fb991cf2640 1 -- 172.21.15.152:0/2293128883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb98c102d40 msgr2=0x7fb98c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:02.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.298+0000 7fb991cf2640 1 --2- 172.21.15.152:0/2293128883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb98c102d40 0x7fb98c198430 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00d950 tx=0x7fb97c00de10 comp rx=0 tx=0).stop 2024-09-11T23:03:02.307 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.298+0000 7fb991cf2640 1 -- 172.21.15.152:0/2293128883 shutdown_connections 2024-09-11T23:03:02.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.298+0000 7fb991cf2640 1 --2- 172.21.15.152:0/2293128883 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9780a2d70 0x7fb9780a5230 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.298+0000 7fb991cf2640 1 --2- 172.21.15.152:0/2293128883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb98c10c170 0x7fb98c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.298+0000 7fb991cf2640 1 --2- 172.21.15.152:0/2293128883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb98c103660 0x7fb98c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.298+0000 7fb991cf2640 1 --2- 172.21.15.152:0/2293128883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb98c102d40 0x7fb98c198430 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:02.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.298+0000 7fb991cf2640 1 -- 172.21.15.152:0/2293128883 >> 172.21.15.152:0/2293128883 conn(0x7fb98c0fc7a0 msgr2=0x7fb98c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:02.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.298+0000 7fb991cf2640 1 -- 172.21.15.152:0/2293128883 shutdown_connections 2024-09-11T23:03:02.308 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:02.298+0000 7fb991cf2640 1 -- 172.21.15.152:0/2293128883 wait complete. 2024-09-11T23:03:03.095 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:02 smithi152 ceph-mon[31181]: pgmap v313: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-11T23:03:03.096 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T23:03:03.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:02 smithi191 ceph-mon[35211]: pgmap v313: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-11T23:03:03.298 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:02 smithi204 ceph-mon[35454]: pgmap v313: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-11T23:03:03.349 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:03.699 DEBUG:teuthology.orchestra.run.smithi152:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-11T23:03:03.805 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.802+0000 7f909a48d640 1 -- 172.21.15.152:0/2206272717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9094101370 msgr2=0x7f909410d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.802+0000 7f909a48d640 1 --2- 172.21.15.152:0/2206272717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9094101370 0x7f909410d8a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f908c009cd0 tx=0x7f908c02a8e0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.802+0000 7f909a48d640 1 -- 172.21.15.152:0/2206272717 shutdown_connections 2024-09-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.802+0000 7f909a48d640 1 --2- 172.21.15.152:0/2206272717 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9094101370 0x7f909410d8a0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.802+0000 7f909a48d640 1 --2- 172.21.15.152:0/2206272717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90941009d0 0x7f9094100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.802+0000 7f909a48d640 1 --2- 172.21.15.152:0/2206272717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90941069e0 0x7f9094106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.802+0000 7f909a48d640 1 -- 172.21.15.152:0/2206272717 >> 172.21.15.152:0/2206272717 conn(0x7f90940fc6f0 msgr2=0x7f90940feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.802+0000 7f909a48d640 1 -- 172.21.15.152:0/2206272717 shutdown_connections 2024-09-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.802+0000 7f909a48d640 1 -- 172.21.15.152:0/2206272717 wait complete. 2024-09-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f909a48d640 1 Processor -- start 2024-09-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f909a48d640 1 -- start start 2024-09-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f909a48d640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90941009d0 0x7f9094199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f909a48d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9094101370 0x7f9094199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f909a48d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90941069e0 0x7f90941a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f909a48d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f9094110e20 con 0x7f90941069e0 2024-09-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f909a48d640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f9094110ca0 con 0x7f9094101370 2024-09-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f909a48d640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f9094110fa0 con 0x7f90941009d0 2024-09-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f9098a2a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90941069e0 0x7f90941a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f9098a2a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90941069e0 0x7f90941a41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35620/0 (socket says 172.21.15.152:35620) 2024-09-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f9098a2a640 1 -- 172.21.15.152:0/466706634 learned_addr learned my addr 172.21.15.152:0/466706634 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f9093fff640 1 --2- 172.21.15.152:0/466706634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90941009d0 0x7f9094199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f908bfff640 1 --2- 172.21.15.152:0/466706634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9094101370 0x7f9094199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f9098a2a640 1 -- 172.21.15.152:0/466706634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90941009d0 msgr2=0x7f9094199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f9098a2a640 1 --2- 172.21.15.152:0/466706634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90941009d0 0x7f9094199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f9098a2a640 1 -- 172.21.15.152:0/466706634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9094101370 msgr2=0x7f9094199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f9098a2a640 1 --2- 172.21.15.152:0/466706634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9094101370 0x7f9094199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f9098a2a640 1 -- 172.21.15.152:0/466706634 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90941a4710 con 0x7f90941069e0 2024-09-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f9093fff640 1 --2- 172.21.15.152:0/466706634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90941009d0 0x7f9094199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.803+0000 7f908bfff640 1 --2- 172.21.15.152:0/466706634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9094101370 0x7f9094199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.804+0000 7f9098a2a640 1 --2- 172.21.15.152:0/466706634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90941069e0 0x7f90941a41d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f908c005cc0 tx=0x7f908c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.804+0000 7f9091ffb640 1 -- 172.21.15.152:0/466706634 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c004040 con 0x7f90941069e0 2024-09-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.804+0000 7f909a48d640 1 -- 172.21.15.152:0/466706634 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90941a48e0 con 0x7f90941069e0 2024-09-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.804+0000 7f9091ffb640 1 -- 172.21.15.152:0/466706634 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908c0046c0 con 0x7f90941069e0 2024-09-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.804+0000 7f9091ffb640 1 -- 172.21.15.152:0/466706634 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c046480 con 0x7f90941069e0 2024-09-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.804+0000 7f909a48d640 1 -- 172.21.15.152:0/466706634 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90941a4df0 con 0x7f90941069e0 2024-09-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.805+0000 7f909a48d640 1 -- 172.21.15.152:0/466706634 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9094100e30 con 0x7f90941069e0 2024-09-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.806+0000 7f9091ffb640 1 -- 172.21.15.152:0/466706634 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f908c0305a0 con 0x7f90941069e0 2024-09-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.807+0000 7f9091ffb640 1 --2- 172.21.15.152:0/466706634 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f90800a2da0 0x7f90800a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.807+0000 7f9093fff640 1 --2- 172.21.15.152:0/466706634 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f90800a2da0 0x7f90800a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.807+0000 7f9091ffb640 1 -- 172.21.15.152:0/466706634 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f908c02e030 con 0x7f90941069e0 2024-09-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.807+0000 7f9093fff640 1 --2- 172.21.15.152:0/466706634 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f90800a2da0 0x7f90800a5260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9084009ab0 tx=0x7f90840094b0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.814+0000 7f9091ffb640 1 -- 172.21.15.152:0/466706634 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f908c04b060 con 0x7f90941069e0 2024-09-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.840+0000 7f8b7c103640 1 -- 172.21.15.152:0/1354091517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b74102ef0 msgr2=0x7f8b7410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.840+0000 7f8b7c103640 1 --2- 172.21.15.152:0/1354091517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b74102ef0 0x7f8b7410da40 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c009ca0 tx=0x7f8b6c02a950 comp rx=0 tx=0).stop 2024-09-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.840+0000 7f8b7c103640 1 -- 172.21.15.152:0/1354091517 shutdown_connections 2024-09-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.840+0000 7f8b7c103640 1 --2- 172.21.15.152:0/1354091517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b74102ef0 0x7f8b7410da40 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.840+0000 7f8b7c103640 1 --2- 172.21.15.152:0/1354091517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b74102570 0x7f8b741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.840+0000 7f8b7c103640 1 --2- 172.21.15.152:0/1354091517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b74100e20 0x7f8b74101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.840+0000 7f8b7c103640 1 -- 172.21.15.152:0/1354091517 >> 172.21.15.152:0/1354091517 conn(0x7f8b740fc7d0 msgr2=0x7f8b740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.840+0000 7f8b7c103640 1 -- 172.21.15.152:0/1354091517 shutdown_connections 2024-09-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.840+0000 7f8b7c103640 1 -- 172.21.15.152:0/1354091517 wait complete. 2024-09-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.840+0000 7f8b7c103640 1 Processor -- start 2024-09-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.840+0000 7f8b7c103640 1 -- start start 2024-09-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b7c103640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b74100e20 0x7f8b740fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b7c103640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b74102570 0x7f8b740fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b7c103640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b74102ef0 0x7f8b740fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b7c103640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8b74110f80 con 0x7f8b74102ef0 2024-09-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b7c103640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8b74110e00 con 0x7f8b74100e20 2024-09-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b7c103640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8b74111100 con 0x7f8b74102570 2024-09-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b7a6a0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b74102ef0 0x7f8b740fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b7a6a0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b74102ef0 0x7f8b740fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35642/0 (socket says 172.21.15.152:35642) 2024-09-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b7a6a0640 1 -- 172.21.15.152:0/2718950857 learned_addr learned my addr 172.21.15.152:0/2718950857 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b79e9f640 1 --2- 172.21.15.152:0/2718950857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b74100e20 0x7f8b740fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b7969e640 1 --2- 172.21.15.152:0/2718950857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b74102570 0x7f8b740fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b7a6a0640 1 -- 172.21.15.152:0/2718950857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b74102570 msgr2=0x7f8b740fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b7a6a0640 1 --2- 172.21.15.152:0/2718950857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b74102570 0x7f8b740fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b7a6a0640 1 -- 172.21.15.152:0/2718950857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b74100e20 msgr2=0x7f8b740fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b7a6a0640 1 --2- 172.21.15.152:0/2718950857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b74100e20 0x7f8b740fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b7a6a0640 1 -- 172.21.15.152:0/2718950857 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b741a69a0 con 0x7f8b74102ef0 2024-09-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.841+0000 7f8b7a6a0640 1 --2- 172.21.15.152:0/2718950857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b74102ef0 0x7f8b740fd700 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c02ae70 tx=0x7f8b6c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.842+0000 7f8b6affd640 1 -- 172.21.15.152:0/2718950857 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c034040 con 0x7f8b74102ef0 2024-09-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.842+0000 7f8b6affd640 1 -- 172.21.15.152:0/2718950857 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b6c030920 con 0x7f8b74102ef0 2024-09-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.842+0000 7f8b7c103640 1 -- 172.21.15.152:0/2718950857 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b741a6c30 con 0x7f8b74102ef0 2024-09-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.842+0000 7f8b6affd640 1 -- 172.21.15.152:0/2718950857 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c045480 con 0x7f8b74102ef0 2024-09-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.842+0000 7f8b7c103640 1 -- 172.21.15.152:0/2718950857 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b741a7160 con 0x7f8b74102ef0 2024-09-11T23:03:03.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.844+0000 7f8b6affd640 1 -- 172.21.15.152:0/2718950857 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8b6c037020 con 0x7f8b74102ef0 2024-09-11T23:03:03.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.844+0000 7f8b7e4dc740 1 Processor -- start 2024-09-11T23:03:03.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.845+0000 7f8b7e4dc740 1 -- start start 2024-09-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.845+0000 7f8b7e4dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 0x561b0f726090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.845+0000 7f8b7e4dc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f7265d0 0x561b0f726a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.845+0000 7f8b7e4dc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f731370 0x561b0f733760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.845+0000 7f8b7e4dc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x561b0f68c2f0 con 0x561b0f731370 2024-09-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.845+0000 7f8b7e4dc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561b0f633cf0 con 0x561b0f7265d0 2024-09-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.845+0000 7f8b7e4dc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561b0f6882b0 con 0x561b0f727b40 2024-09-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.845+0000 7f8b7969e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f731370 0x561b0f733760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.845+0000 7f8b7969e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f731370 0x561b0f733760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35654/0 (socket says 172.21.15.152:35654) 2024-09-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.845+0000 7f8b7969e640 1 -- 172.21.15.152:0/2810251498 learned_addr learned my addr 172.21.15.152:0/2810251498 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.845+0000 7f8b6affd640 1 --2- 172.21.15.152:0/2718950857 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8b600a2d70 0x7f8b600a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.845+0000 7f8b7969e640 1 --2- 172.21.15.152:0/2810251498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 0x561b0f726090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.845+0000 7f8b79e9f640 1 --2- 172.21.15.152:0/2810251498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f7265d0 0x561b0f726a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.846+0000 7f8b6affd640 1 -- 172.21.15.152:0/2718950857 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f8b6c0f01b0 con 0x7f8b74102ef0 2024-09-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.846+0000 7f8b7a6a0640 1 --2- 172.21.15.152:0/2718950857 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8b600a2d70 0x7f8b600a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.846+0000 7f8b7969e640 1 -- 172.21.15.152:0/2810251498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 msgr2=0x561b0f726090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.846+0000 7f8b7969e640 1 --2- 172.21.15.152:0/2810251498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 0x561b0f726090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.846+0000 7f8b7969e640 1 -- 172.21.15.152:0/2810251498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f7265d0 msgr2=0x561b0f726a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.846+0000 7f8b7969e640 1 --2- 172.21.15.152:0/2810251498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f7265d0 0x561b0f726a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.846+0000 7f8b7969e640 1 -- 172.21.15.152:0/2810251498 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b0f6183a0 con 0x561b0f731370 2024-09-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.846+0000 7f8b79e9f640 1 --2- 172.21.15.152:0/2810251498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f7265d0 0x561b0f726a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.846+0000 7f8b7969e640 1 --2- 172.21.15.152:0/2810251498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f731370 0x561b0f733760 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f8b70012530 tx=0x7f8b7000b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdda4d9e370fa1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.846+0000 7f8b7a6a0640 1 --2- 172.21.15.152:0/2718950857 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8b600a2d70 0x7f8b600a5230 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c032000 tx=0x7f8b6c03c040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.846+0000 7f8b5ffff640 1 -- 172.21.15.152:0/2810251498 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70004220 con 0x561b0f731370 2024-09-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.846+0000 7f8b5ffff640 1 -- 172.21.15.152:0/2810251498 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b700043c0 con 0x561b0f731370 2024-09-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.846+0000 7f8b5ffff640 1 -- 172.21.15.152:0/2810251498 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b700109c0 con 0x561b0f731370 2024-09-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.847+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2810251498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f731370 msgr2=0x561b0f733760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.847+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/2810251498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f731370 0x561b0f733760 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f8b70012530 tx=0x7f8b7000b650 comp rx=0 tx=0).stop 2024-09-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.847+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2810251498 shutdown_connections 2024-09-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.847+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/2810251498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f731370 0x561b0f733760 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.847+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/2810251498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f7265d0 0x561b0f726a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.847+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/2810251498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 0x561b0f726090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.847+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2810251498 >> 172.21.15.152:0/2810251498 conn(0x561b0f6ef310 msgr2=0x561b0f722c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.847+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2810251498 shutdown_connections 2024-09-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.847+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2810251498 wait complete. 2024-09-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.848+0000 7f8b7e4dc740 1 -- start start 2024-09-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.848+0000 7f8b7e4dc740 1 Processor -- start 2024-09-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.848+0000 7f8b7e4dc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f730d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.848+0000 7f8b7e4dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 0x561b0f72dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.848+0000 7f8b7e4dc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f731370 0x561b0f72e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.848+0000 7f8b7e4dc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x561b0f734070 con 0x561b0f7265d0 2024-09-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.848+0000 7f8b7e4dc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x561b0f647060 con 0x561b0f731370 2024-09-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.848+0000 7f8b7e4dc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x561b0f7341f0 con 0x561b0f727b40 2024-09-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.848+0000 7f8b7969e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f730d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.848+0000 7f8b7e4dc740 10 client.0 _ll_register_callbacks cb 0x561b0f39a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.848+0000 7f8b7969e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f730d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35658/0 (socket says 172.21.15.152:35658) 2024-09-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.848+0000 7f8b7969e640 1 -- 172.21.15.152:0/2496300680 learned_addr learned my addr 172.21.15.152:0/2496300680 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.848+0000 7f8b79e9f640 1 --2- 172.21.15.152:0/2496300680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 0x561b0f72dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.849+0000 7f8b7969e640 1 --2- 172.21.15.152:0/2496300680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f731370 0x561b0f72e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.849+0000 7f8b7969e640 1 -- 172.21.15.152:0/2496300680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 msgr2=0x561b0f72dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.849+0000 7f8b7969e640 1 --2- 172.21.15.152:0/2496300680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 0x561b0f72dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.849+0000 7f8b7969e640 1 -- 172.21.15.152:0/2496300680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f731370 msgr2=0x561b0f72e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.849+0000 7f8b7969e640 1 --2- 172.21.15.152:0/2496300680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f731370 0x561b0f72e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.849+0000 7f8b7969e640 1 -- 172.21.15.152:0/2496300680 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b0f3fd3c0 con 0x561b0f7265d0 2024-09-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.849+0000 7f8b7969e640 1 --2- 172.21.15.152:0/2496300680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f730d30 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f8b70012e80 tx=0x7f8b70010680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea94ce7df4d50f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.849+0000 7f8b5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.849+0000 7f8b5d7fa640 1 -- 172.21.15.152:0/2496300680 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70022040 con 0x561b0f7265d0 2024-09-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.849+0000 7f8b5d7fa640 1 -- 172.21.15.152:0/2496300680 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f8b70003a00 con 0x561b0f7265d0 2024-09-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.849+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2496300680 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b700043c0 con 0x561b0f7265d0 2024-09-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.849+0000 7f8b5d7fa640 1 -- 172.21.15.152:0/2496300680 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8b7000bde0 con 0x561b0f7265d0 2024-09-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.849+0000 7f8b7e4dc740 10 client.18312 fetch_fsmap learned FSMap version 28 2024-09-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.849+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2496300680 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b0f73fb00 con 0x561b0f7265d0 2024-09-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.850+0000 7f8b5d7fa640 1 -- 172.21.15.152:0/2496300680 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b700043c0 con 0x561b0f7265d0 2024-09-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.850+0000 7f8b5d7fa640 1 -- 172.21.15.152:0/2496300680 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8b70032020 con 0x561b0f7265d0 2024-09-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.850+0000 7f8b7e4dc740 10 client.18312 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.850+0000 7f8b7e4dc740 10 client.18312 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.850+0000 7f8b7e4dc740 20 client.18312 populate_metadata read hostname 'smithi152' 2024-09-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.850+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/2496300680 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x561b0f740e50 0x561b0f743240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.850+0000 7f8b7e4dc740 4 client.18312 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.850+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2496300680 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b0f699860 con 0x561b0f740e50 2024-09-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.850+0000 7f8b79e9f640 1 --2- 172.21.15.152:0/2496300680 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x561b0f740e50 0x561b0f743240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.851+0000 7f8b79e9f640 1 --2- 172.21.15.152:0/2496300680 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x561b0f740e50 0x561b0f743240 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cae09c949510a104 server_cookie=31231c86abbfc814 in_seq=0 out_seq=0 2024-09-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.851+0000 7f8b5d7fa640 10 client.18312 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.851+0000 7f8b5d7fa640 1 -- 172.21.15.152:0/2496300680 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561b0f699860 con 0x561b0f740e50 2024-09-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.851+0000 7f8b5d7fa640 10 client.18312 handle_command_reply: tid=0 2024-09-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.851+0000 7f8b7e4dc740 1 client.18312 shutdown 2024-09-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.851+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2496300680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 msgr2=0x561b0f730d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.851+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/2496300680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f730d30 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f8b70012e80 tx=0x7f8b70010680 comp rx=0 tx=0).stop 2024-09-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.852+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2496300680 shutdown_connections 2024-09-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.852+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/2496300680 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x561b0f740e50 0x561b0f743240 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.852+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/2496300680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f731370 0x561b0f72e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.852+0000 7f8b5d7fa640 0 client.18312 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.852+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/2496300680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 0x561b0f72dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.852+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/2496300680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f730d30 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.852+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2496300680 >> 172.21.15.152:0/2496300680 conn(0x561b0f6ef310 msgr2=0x561b0f728960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.852+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2496300680 shutdown_connections 2024-09-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.852+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2496300680 wait complete. 2024-09-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.852+0000 7f8b7e4dc740 20 client.18312 trim_cache size 0 max 16384 2024-09-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.865+0000 7f8b7e4dc740 1 Processor -- start 2024-09-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.865+0000 7f8b7e4dc740 1 -- start start 2024-09-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.865+0000 7f8b7e4dc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f73d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.865+0000 7f8b7e4dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 0x561b0f72d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.865+0000 7f8b7e4dc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f731370 0x561b0f722680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.865+0000 7f8b7e4dc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741af9a0 con 0x561b0f7265d0 2024-09-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.865+0000 7f8b7e4dc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741b14e0 con 0x561b0f731370 2024-09-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.865+0000 7f8b7e4dc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741b0070 con 0x561b0f727b40 2024-09-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.865+0000 7f8b7969e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f73d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.865+0000 7f8b7969e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f73d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35672/0 (socket says 172.21.15.152:35672) 2024-09-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.865+0000 7f8b7969e640 1 -- 172.21.15.152:0/2257063984 learned_addr learned my addr 172.21.15.152:0/2257063984 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.865+0000 7f8b79e9f640 1 --2- 172.21.15.152:0/2257063984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 0x561b0f72d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.866+0000 7f8b7969e640 1 -- 172.21.15.152:0/2257063984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 msgr2=0x561b0f72d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.866+0000 7f8b7969e640 1 --2- 172.21.15.152:0/2257063984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 0x561b0f72d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.866+0000 7f8b7969e640 1 -- 172.21.15.152:0/2257063984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f731370 msgr2=0x561b0f722680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.866+0000 7f8b7969e640 1 --2- 172.21.15.152:0/2257063984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f731370 0x561b0f722680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.866+0000 7f8b7969e640 1 -- 172.21.15.152:0/2257063984 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b0f730950 con 0x561b0f7265d0 2024-09-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.866+0000 7f8b7969e640 1 --2- 172.21.15.152:0/2257063984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f73d340 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f8b70005d00 tx=0x7f8b70034440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e864f95ed8d13e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.866+0000 7f8b5f7fe640 1 -- 172.21.15.152:0/2257063984 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7000b8a0 con 0x561b0f7265d0 2024-09-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.866+0000 7f8b5f7fe640 1 -- 172.21.15.152:0/2257063984 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b70003ce0 con 0x561b0f7265d0 2024-09-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.866+0000 7f8b5f7fe640 1 -- 172.21.15.152:0/2257063984 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70034bd0 con 0x561b0f7265d0 2024-09-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.867+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2257063984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 msgr2=0x561b0f73d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.867+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/2257063984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f73d340 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f8b70005d00 tx=0x7f8b70034440 comp rx=0 tx=0).stop 2024-09-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.867+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2257063984 shutdown_connections 2024-09-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.867+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/2257063984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f731370 0x561b0f722680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.867+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/2257063984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 0x561b0f72d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.867+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/2257063984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f73d340 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.867+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2257063984 >> 172.21.15.152:0/2257063984 conn(0x561b0f6ef310 msgr2=0x561b0f729310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.867+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2257063984 shutdown_connections 2024-09-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.867+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/2257063984 wait complete. 2024-09-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.867+0000 7f8b7e4dc740 1 -- start start 2024-09-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.867+0000 7f8b7e4dc740 1 Processor -- start 2024-09-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b7e4dc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f72bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b7e4dc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 0x561b0f7394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b7e4dc740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f731370 0x561b0f739b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b7e4dc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741b0e10 con 0x561b0f7265d0 2024-09-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b7e4dc740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741b0740 con 0x561b0f731370 2024-09-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b7e4dc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f8b741ad9a0 con 0x561b0f727b40 2024-09-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b7e4dc740 10 client.0 _ll_register_callbacks cb 0x7f8b64004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b7969e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f72bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b7969e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f72bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35682/0 (socket says 172.21.15.152:35682) 2024-09-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b7969e640 1 -- 172.21.15.152:0/3293094828 learned_addr learned my addr 172.21.15.152:0/3293094828 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b79e9f640 1 --2- 172.21.15.152:0/3293094828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 0x561b0f7394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b7969e640 1 --2- 172.21.15.152:0/3293094828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f731370 0x561b0f739b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b7969e640 1 -- 172.21.15.152:0/3293094828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 msgr2=0x561b0f7394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b7969e640 1 --2- 172.21.15.152:0/3293094828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 0x561b0f7394b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b7969e640 1 -- 172.21.15.152:0/3293094828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f731370 msgr2=0x561b0f739b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b7969e640 1 --2- 172.21.15.152:0/3293094828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f731370 0x561b0f739b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.868+0000 7f8b7969e640 1 -- 172.21.15.152:0/3293094828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b0f72a270 con 0x561b0f7265d0 2024-09-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.869+0000 7f8b7969e640 1 --2- 172.21.15.152:0/3293094828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f72bc90 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f8b70000c00 tx=0x7f8b70005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f697f17708a21f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.869+0000 7f8b5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.869+0000 7f8b5d7fa640 1 -- 172.21.15.152:0/3293094828 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70034bd0 con 0x561b0f7265d0 2024-09-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.869+0000 7f8b5d7fa640 1 -- 172.21.15.152:0/3293094828 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f8b70034d70 con 0x561b0f7265d0 2024-09-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.869+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/3293094828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b70003ce0 con 0x561b0f7265d0 2024-09-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.869+0000 7f8b5d7fa640 1 -- 172.21.15.152:0/3293094828 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8b70003800 con 0x561b0f7265d0 2024-09-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.869+0000 7f8b7e4dc740 10 client.18318 fetch_fsmap learned FSMap version 28 2024-09-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.869+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/3293094828 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561b0f7aae10 con 0x561b0f7265d0 2024-09-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.869+0000 7f8b5d7fa640 1 -- 172.21.15.152:0/3293094828 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b70003ce0 con 0x561b0f7265d0 2024-09-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.869+0000 7f8b5d7fa640 1 -- 172.21.15.152:0/3293094828 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8b70048070 con 0x561b0f7265d0 2024-09-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.869+0000 7f8b7e4dc740 10 client.18318 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.869+0000 7f8b7e4dc740 10 client.18318 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.869+0000 7f8b7e4dc740 20 client.18318 populate_metadata read hostname 'smithi152' 2024-09-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.869+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/3293094828 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x561b0f7ab440 0x561b0f7ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.869+0000 7f8b7e4dc740 4 client.18318 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.869+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/3293094828 --> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b0f693d70 con 0x561b0f7ab440 2024-09-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.870+0000 7f8b79e9f640 1 --2- 172.21.15.152:0/3293094828 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x561b0f7ab440 0x561b0f7ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.870+0000 7f8b79e9f640 1 --2- 172.21.15.152:0/3293094828 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x561b0f7ab440 0x561b0f7ad860 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=59317537174868fc server_cookie=57eb90cbf5319fc0 in_seq=0 out_seq=0 2024-09-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.870+0000 7f8b5d7fa640 10 client.18318 ms_handle_connect on v2:172.21.15.204:6836/1863048373 2024-09-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.870+0000 7f8b5d7fa640 1 -- 172.21.15.152:0/3293094828 <== mds.3 v2:172.21.15.204:6836/1863048373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b0f693d70 con 0x561b0f7ab440 2024-09-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.870+0000 7f8b5d7fa640 10 client.18318 handle_command_reply: tid=0 2024-09-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.870+0000 7f8b7e4dc740 1 client.18318 shutdown 2024-09-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.870+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/3293094828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 msgr2=0x561b0f72bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.870+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/3293094828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f72bc90 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f8b70000c00 tx=0x7f8b70005fb0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.871+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/3293094828 shutdown_connections 2024-09-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.871+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/3293094828 >> [v2:172.21.15.204:6836/1863048373,v1:172.21.15.204:6837/1863048373] conn(0x561b0f7ab440 0x561b0f7ad860 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.871+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/3293094828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561b0f731370 0x561b0f739b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.871+0000 7f8b5d7fa640 0 client.18318 ms_handle_reset on v2:172.21.15.204:6836/1863048373 2024-09-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.871+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/3293094828 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561b0f727b40 0x561b0f7394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.871+0000 7f8b7e4dc740 1 --2- 172.21.15.152:0/3293094828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561b0f7265d0 0x561b0f72bc90 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.871+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/3293094828 >> 172.21.15.152:0/3293094828 conn(0x561b0f6ef310 msgr2=0x561b0f722680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.871+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/3293094828 shutdown_connections 2024-09-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.871+0000 7f8b7e4dc740 1 -- 172.21.15.152:0/3293094828 wait complete. 2024-09-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.871+0000 7f8b7e4dc740 20 client.18318 trim_cache size 0 max 16384 2024-09-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.871+0000 7f8b7c103640 1 -- 172.21.15.152:0/2718950857 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8b600a2d70 msgr2=0x7f8b600a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.871+0000 7f8b7c103640 1 --2- 172.21.15.152:0/2718950857 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8b600a2d70 0x7f8b600a5230 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c032000 tx=0x7f8b6c03c040 comp rx=0 tx=0).stop 2024-09-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.871+0000 7f8b7c103640 1 -- 172.21.15.152:0/2718950857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b74102ef0 msgr2=0x7f8b740fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.871+0000 7f8b7c103640 1 --2- 172.21.15.152:0/2718950857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b74102ef0 0x7f8b740fd700 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c02ae70 tx=0x7f8b6c02f5d0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.872+0000 7f8b7c103640 1 -- 172.21.15.152:0/2718950857 shutdown_connections 2024-09-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.872+0000 7f8b7c103640 1 --2- 172.21.15.152:0/2718950857 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f8b600a2d70 0x7f8b600a5230 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.872+0000 7f8b7c103640 1 --2- 172.21.15.152:0/2718950857 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8b74102ef0 0x7f8b740fd700 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.872+0000 7f8b7c103640 1 --2- 172.21.15.152:0/2718950857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b74102570 0x7f8b740fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.872+0000 7f8b7c103640 1 --2- 172.21.15.152:0/2718950857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b74100e20 0x7f8b740fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.872+0000 7f8b7c103640 1 -- 172.21.15.152:0/2718950857 >> 172.21.15.152:0/2718950857 conn(0x7f8b740fc7d0 msgr2=0x7f8b7406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.872+0000 7f8b7c103640 1 -- 172.21.15.152:0/2718950857 shutdown_connections 2024-09-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:03.872+0000 7f8b7c103640 1 -- 172.21.15.152:0/2718950857 wait complete. 2024-09-11T23:03:04.014 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.012+0000 7f909a48d640 1 -- 172.21.15.152:0/466706634 --> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f90940fd280 con 0x7f90800a2da0 2024-09-11T23:03:04.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.021+0000 7f9091ffb640 1 -- 172.21.15.152:0/466706634 <== mgr.15240 v2:172.21.15.152:6800/4130474708 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f90940fd280 con 0x7f90800a2da0 2024-09-11T23:03:04.022 INFO:teuthology.orchestra.run.smithi152.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-11T00:00:00", "created": "2024-09-11T22:53:01", "first": "2024-09-11T22:54:00", "last": "2024-09-11T23:03:00", "last_pruned": "2024-09-11T23:03:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-09-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.023+0000 7f909a48d640 1 -- 172.21.15.152:0/466706634 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f90800a2da0 msgr2=0x7f90800a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.023+0000 7f909a48d640 1 --2- 172.21.15.152:0/466706634 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f90800a2da0 0x7f90800a5260 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9084009ab0 tx=0x7f90840094b0 comp rx=0 tx=0).stop 2024-09-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.023+0000 7f909a48d640 1 -- 172.21.15.152:0/466706634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90941069e0 msgr2=0x7f90941a41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.023+0000 7f909a48d640 1 --2- 172.21.15.152:0/466706634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90941069e0 0x7f90941a41d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f908c005cc0 tx=0x7f908c003970 comp rx=0 tx=0).stop 2024-09-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.023+0000 7f909a48d640 1 -- 172.21.15.152:0/466706634 shutdown_connections 2024-09-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.023+0000 7f909a48d640 1 --2- 172.21.15.152:0/466706634 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f90800a2da0 0x7f90800a5260 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.023+0000 7f909a48d640 1 --2- 172.21.15.152:0/466706634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f90941069e0 0x7f90941a41d0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.023+0000 7f909a48d640 1 --2- 172.21.15.152:0/466706634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9094101370 0x7f9094199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.023+0000 7f909a48d640 1 --2- 172.21.15.152:0/466706634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90941009d0 0x7f9094199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.023+0000 7f909a48d640 1 -- 172.21.15.152:0/466706634 >> 172.21.15.152:0/466706634 conn(0x7f90940fc6f0 msgr2=0x7f90940fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.023+0000 7f909a48d640 1 -- 172.21.15.152:0/466706634 shutdown_connections 2024-09-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.023+0000 7f909a48d640 1 -- 172.21.15.152:0/466706634 wait complete. 2024-09-11T23:03:04.035 DEBUG:teuthology.orchestra.run.smithi152:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-11T23:03:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.137+0000 7f5e96a37640 1 -- 172.21.15.152:0/503218892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e90101350 msgr2=0x7f5e9010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.137+0000 7f5e96a37640 1 --2- 172.21.15.152:0/503218892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e90101350 0x7f5e9010d880 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009cd0 tx=0x7f5e8802a8e0 comp rx=0 tx=0).stop 2024-09-11T23:03:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.137+0000 7f5e96a37640 1 -- 172.21.15.152:0/503218892 shutdown_connections 2024-09-11T23:03:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.137+0000 7f5e96a37640 1 --2- 172.21.15.152:0/503218892 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e90101350 0x7f5e9010d880 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.137+0000 7f5e96a37640 1 --2- 172.21.15.152:0/503218892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e901009b0 0x7f5e90100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:04.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.137+0000 7f5e96a37640 1 --2- 172.21.15.152:0/503218892 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e901069c0 0x7f5e90106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:04.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.137+0000 7f5e96a37640 1 -- 172.21.15.152:0/503218892 >> 172.21.15.152:0/503218892 conn(0x7f5e90077990 msgr2=0x7f5e900fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:04.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.137+0000 7f5e96a37640 1 -- 172.21.15.152:0/503218892 shutdown_connections 2024-09-11T23:03:04.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.137+0000 7f5e96a37640 1 -- 172.21.15.152:0/503218892 wait complete. 2024-09-11T23:03:04.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.137+0000 7f5e96a37640 1 Processor -- start 2024-09-11T23:03:04.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.137+0000 7f5e96a37640 1 -- start start 2024-09-11T23:03:04.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e96a37640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e901009b0 0x7f5e90199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:04.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e96a37640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e90101350 0x7f5e901996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:04.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e96a37640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e901069c0 0x7f5e901a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:04.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e96a37640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90110e00 con 0x7f5e901009b0 2024-09-11T23:03:04.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e96a37640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90110c80 con 0x7f5e901069c0 2024-09-11T23:03:04.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e96a37640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f5e90110f80 con 0x7f5e90101350 2024-09-11T23:03:04.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e8ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e901009b0 0x7f5e90199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:04.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e8ffff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e901009b0 0x7f5e90199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35708/0 (socket says 172.21.15.152:35708) 2024-09-11T23:03:04.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e8ffff640 1 -- 172.21.15.152:0/3201158732 learned_addr learned my addr 172.21.15.152:0/3201158732 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:04.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e94fd4640 1 --2- 172.21.15.152:0/3201158732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e901069c0 0x7f5e901a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:04.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e8f7fe640 1 --2- 172.21.15.152:0/3201158732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e90101350 0x7f5e901996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:04.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e8ffff640 1 -- 172.21.15.152:0/3201158732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e90101350 msgr2=0x7f5e901996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:04.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e8ffff640 1 --2- 172.21.15.152:0/3201158732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e90101350 0x7f5e901996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:04.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e8ffff640 1 -- 172.21.15.152:0/3201158732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e901069c0 msgr2=0x7f5e901a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:04.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e8ffff640 1 --2- 172.21.15.152:0/3201158732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e901069c0 0x7f5e901a4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:04.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e8ffff640 1 -- 172.21.15.152:0/3201158732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e901a4740 con 0x7f5e901009b0 2024-09-11T23:03:04.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e8f7fe640 1 --2- 172.21.15.152:0/3201158732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e90101350 0x7f5e901996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:03:04.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e8ffff640 1 --2- 172.21.15.152:0/3201158732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e901009b0 0x7f5e90199180 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000d950 tx=0x7f5e8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:04.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.138+0000 7f5e8d7fa640 1 -- 172.21.15.152:0/3201158732 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80017070 con 0x7f5e901009b0 2024-09-11T23:03:04.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.139+0000 7f5e8d7fa640 1 -- 172.21.15.152:0/3201158732 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e80009c30 con 0x7f5e901009b0 2024-09-11T23:03:04.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.139+0000 7f5e96a37640 1 -- 172.21.15.152:0/3201158732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e901a4910 con 0x7f5e901009b0 2024-09-11T23:03:04.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.139+0000 7f5e8d7fa640 1 -- 172.21.15.152:0/3201158732 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8000f6b0 con 0x7f5e901009b0 2024-09-11T23:03:04.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.139+0000 7f5e96a37640 1 -- 172.21.15.152:0/3201158732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e901a4e30 con 0x7f5e901009b0 2024-09-11T23:03:04.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.140+0000 7f5e96a37640 1 -- 172.21.15.152:0/3201158732 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e90100e10 con 0x7f5e901009b0 2024-09-11T23:03:04.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.141+0000 7f5e8d7fa640 1 -- 172.21.15.152:0/3201158732 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5e800109d0 con 0x7f5e901009b0 2024-09-11T23:03:04.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.142+0000 7f5e8d7fa640 1 --2- 172.21.15.152:0/3201158732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f5e780a2da0 0x7f5e780a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:04.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.142+0000 7f5e8d7fa640 1 -- 172.21.15.152:0/3201158732 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f5e800cefc0 con 0x7f5e901009b0 2024-09-11T23:03:04.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.142+0000 7f5e8f7fe640 1 --2- 172.21.15.152:0/3201158732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f5e780a2da0 0x7f5e780a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:04.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.142+0000 7f5e8f7fe640 1 --2- 172.21.15.152:0/3201158732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f5e780a2da0 0x7f5e780a5260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5e84005fd0 tx=0x7f5e84005ea0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:04.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.145+0000 7f5e8d7fa640 1 -- 172.21.15.152:0/3201158732 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e80009de0 con 0x7f5e901009b0 2024-09-11T23:03:04.338 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.337+0000 7f5e96a37640 1 -- 172.21.15.152:0/3201158732 --> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5e90103eb0 con 0x7f5e780a2da0 2024-09-11T23:03:04.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.344+0000 7f5e8d7fa640 1 -- 172.21.15.152:0/3201158732 <== mgr.15240 v2:172.21.15.152:6800/4130474708 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5e90103eb0 con 0x7f5e780a2da0 2024-09-11T23:03:04.346 INFO:teuthology.orchestra.run.smithi152.stdout:/ 1m 6m3h 2024-09-11T23:03:04.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.346+0000 7f5e96a37640 1 -- 172.21.15.152:0/3201158732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f5e780a2da0 msgr2=0x7f5e780a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:04.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.346+0000 7f5e96a37640 1 --2- 172.21.15.152:0/3201158732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f5e780a2da0 0x7f5e780a5260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5e84005fd0 tx=0x7f5e84005ea0 comp rx=0 tx=0).stop 2024-09-11T23:03:04.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.346+0000 7f5e96a37640 1 -- 172.21.15.152:0/3201158732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e901009b0 msgr2=0x7f5e90199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:04.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.346+0000 7f5e96a37640 1 --2- 172.21.15.152:0/3201158732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e901009b0 0x7f5e90199180 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000d950 tx=0x7f5e8000de10 comp rx=0 tx=0).stop 2024-09-11T23:03:04.348 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.346+0000 7f5e96a37640 1 -- 172.21.15.152:0/3201158732 shutdown_connections 2024-09-11T23:03:04.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.347+0000 7f5e96a37640 1 --2- 172.21.15.152:0/3201158732 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f5e780a2da0 0x7f5e780a5260 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:04.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.347+0000 7f5e96a37640 1 --2- 172.21.15.152:0/3201158732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e901069c0 0x7f5e901a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:04.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.347+0000 7f5e96a37640 1 --2- 172.21.15.152:0/3201158732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e90101350 0x7f5e901996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:04.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.347+0000 7f5e96a37640 1 --2- 172.21.15.152:0/3201158732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e901009b0 0x7f5e90199180 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:04.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.347+0000 7f5e96a37640 1 -- 172.21.15.152:0/3201158732 >> 172.21.15.152:0/3201158732 conn(0x7f5e90077990 msgr2=0x7f5e901037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:04.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.347+0000 7f5e96a37640 1 -- 172.21.15.152:0/3201158732 shutdown_connections 2024-09-11T23:03:04.349 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:04.347+0000 7f5e96a37640 1 -- 172.21.15.152:0/3201158732 wait complete. 2024-09-11T23:03:04.359 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-11T23:03:04.371 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-11T23:03:04.726 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T23:03:04.972 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:05.093 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:04 smithi152 ceph-mon[31181]: pgmap v314: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T23:03:05.094 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:04 smithi152 ceph-mon[31181]: from='client.18300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:03:05.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:04 smithi191 ceph-mon[35211]: pgmap v314: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T23:03:05.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:04 smithi191 ceph-mon[35211]: from='client.18300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:03:05.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:04 smithi204 ceph-mon[35454]: pgmap v314: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-11T23:03:05.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:04 smithi204 ceph-mon[35454]: from='client.18300 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:03:05.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.507+0000 7fdeebb99640 1 -- 172.21.15.152:0/990780791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdee4106a80 msgr2=0x7fdee4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:05.528 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.507+0000 7fdeebb99640 1 --2- 172.21.15.152:0/990780791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdee4106a80 0x7fdee4106e60 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fded8009930 tx=0x7fded802a420 comp rx=0 tx=0).stop 2024-09-11T23:03:05.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.507+0000 7fdeebb99640 1 -- 172.21.15.152:0/990780791 shutdown_connections 2024-09-11T23:03:05.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.507+0000 7fdeebb99640 1 --2- 172.21.15.152:0/990780791 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee4101420 0x7fdee410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.507+0000 7fdeebb99640 1 --2- 172.21.15.152:0/990780791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee4100a80 0x7fdee4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.507+0000 7fdeebb99640 1 --2- 172.21.15.152:0/990780791 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdee4106a80 0x7fdee4106e60 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.507+0000 7fdeebb99640 1 -- 172.21.15.152:0/990780791 >> 172.21.15.152:0/990780791 conn(0x7fdee40fc7a0 msgr2=0x7fdee40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:05.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.507+0000 7fdeebb99640 1 -- 172.21.15.152:0/990780791 shutdown_connections 2024-09-11T23:03:05.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.507+0000 7fdeebb99640 1 -- 172.21.15.152:0/990780791 wait complete. 2024-09-11T23:03:05.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.507+0000 7fdeebb99640 1 Processor -- start 2024-09-11T23:03:05.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdeebb99640 1 -- start start 2024-09-11T23:03:05.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdeebb99640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee4100a80 0x7fdee41973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.529 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdeebb99640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdee4101420 0x7fdee41978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdeebb99640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee4106a80 0x7fdee419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdeebb99640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdee4110e80 con 0x7fdee4101420 2024-09-11T23:03:05.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdeebb99640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdee4110d00 con 0x7fdee4100a80 2024-09-11T23:03:05.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdeebb99640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdee4111000 con 0x7fdee4106a80 2024-09-11T23:03:05.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdee9134640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdee4101420 0x7fdee41978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdee9134640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdee4101420 0x7fdee41978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35724/0 (socket says 172.21.15.152:35724) 2024-09-11T23:03:05.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdee9134640 1 -- 172.21.15.152:0/4243727759 learned_addr learned my addr 172.21.15.152:0/4243727759 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:05.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdee9935640 1 --2- 172.21.15.152:0/4243727759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee4100a80 0x7fdee41973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdeea136640 1 --2- 172.21.15.152:0/4243727759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee4106a80 0x7fdee419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdee9134640 1 -- 172.21.15.152:0/4243727759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee4106a80 msgr2=0x7fdee419bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:05.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdee9134640 1 --2- 172.21.15.152:0/4243727759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee4106a80 0x7fdee419bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdee9134640 1 -- 172.21.15.152:0/4243727759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee4100a80 msgr2=0x7fdee41973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:05.530 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdee9134640 1 --2- 172.21.15.152:0/4243727759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee4100a80 0x7fdee41973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdee9134640 1 -- 172.21.15.152:0/4243727759 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee419c350 con 0x7fdee4101420 2024-09-11T23:03:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.508+0000 7fdee9134640 1 --2- 172.21.15.152:0/4243727759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdee4101420 0x7fdee41978e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fdee000e9f0 tx=0x7fdee000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.509+0000 7fded6ffd640 1 -- 172.21.15.152:0/4243727759 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee000cd80 con 0x7fdee4101420 2024-09-11T23:03:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.509+0000 7fded6ffd640 1 -- 172.21.15.152:0/4243727759 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee0005500 con 0x7fdee4101420 2024-09-11T23:03:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.509+0000 7fdeebb99640 1 -- 172.21.15.152:0/4243727759 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee419c640 con 0x7fdee4101420 2024-09-11T23:03:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.509+0000 7fded6ffd640 1 -- 172.21.15.152:0/4243727759 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0010690 con 0x7fdee4101420 2024-09-11T23:03:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.509+0000 7fdeebb99640 1 -- 172.21.15.152:0/4243727759 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee41a3f20 con 0x7fdee4101420 2024-09-11T23:03:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.511+0000 7fded6ffd640 1 -- 172.21.15.152:0/4243727759 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fdee00046c0 con 0x7fdee4101420 2024-09-11T23:03:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.512+0000 7fdeedf72740 1 Processor -- start 2024-09-11T23:03:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.512+0000 7fdeedf72740 1 -- start start 2024-09-11T23:03:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.512+0000 7fdeedf72740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c652090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.512+0000 7fdeedf72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c6525d0 0x55b39c652a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.512+0000 7fdeedf72740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c65f580 0x55b39c661970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.512+0000 7fdeedf72740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55b39c5b82f0 con 0x55b39c653b40 2024-09-11T23:03:05.531 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.512+0000 7fdeedf72740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b39c55fcf0 con 0x55b39c65f580 2024-09-11T23:03:05.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.512+0000 7fdeedf72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b39c5b42b0 con 0x55b39c6525d0 2024-09-11T23:03:05.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.512+0000 7fdeea136640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c652090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.512+0000 7fdeea136640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c652090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35732/0 (socket says 172.21.15.152:35732) 2024-09-11T23:03:05.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.512+0000 7fdeea136640 1 -- 172.21.15.152:0/1946997687 learned_addr learned my addr 172.21.15.152:0/1946997687 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:05.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.512+0000 7fdee9134640 1 --2- 172.21.15.152:0/1946997687 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c65f580 0x55b39c661970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.512+0000 7fded6ffd640 1 --2- 172.21.15.152:0/4243727759 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fded00a2cf0 0x7fded00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.512+0000 7fdee9935640 1 --2- 172.21.15.152:0/1946997687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c6525d0 0x55b39c652a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.513+0000 7fdeea136640 1 -- 172.21.15.152:0/1946997687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c6525d0 msgr2=0x55b39c652a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:05.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.513+0000 7fdeea136640 1 --2- 172.21.15.152:0/1946997687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c6525d0 0x55b39c652a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.513+0000 7fdeea136640 1 -- 172.21.15.152:0/1946997687 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c65f580 msgr2=0x55b39c661970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:05.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.513+0000 7fdeea136640 1 --2- 172.21.15.152:0/1946997687 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c65f580 0x55b39c661970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.532 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.513+0000 7fdeea136640 1 -- 172.21.15.152:0/1946997687 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39c5443a0 con 0x55b39c653b40 2024-09-11T23:03:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.513+0000 7fded6ffd640 1 -- 172.21.15.152:0/4243727759 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7fdee00cf810 con 0x7fdee4101420 2024-09-11T23:03:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.513+0000 7fdeea136640 1 --2- 172.21.15.152:0/4243727759 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fded00a2cf0 0x7fded00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.513+0000 7fdeea136640 1 --2- 172.21.15.152:0/1946997687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c652090 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0104a0 tx=0x7fdedc010960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b98d4ff8f2eab17 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.513+0000 7fdecffff640 1 -- 172.21.15.152:0/1946997687 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc022070 con 0x55b39c653b40 2024-09-11T23:03:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.513+0000 7fdecffff640 1 -- 172.21.15.152:0/1946997687 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdedc013c60 con 0x55b39c653b40 2024-09-11T23:03:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.513+0000 7fdecffff640 1 -- 172.21.15.152:0/1946997687 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc01b3c0 con 0x55b39c653b40 2024-09-11T23:03:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.514+0000 7fdeedf72740 1 -- 172.21.15.152:0/1946997687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 msgr2=0x55b39c652090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.514+0000 7fdeedf72740 1 --2- 172.21.15.152:0/1946997687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c652090 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0104a0 tx=0x7fdedc010960 comp rx=0 tx=0).stop 2024-09-11T23:03:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.514+0000 7fdeea136640 1 --2- 172.21.15.152:0/4243727759 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fded00a2cf0 0x7fded00a51b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdedc01f000 tx=0x7fdedc01c000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.514+0000 7fdeedf72740 1 -- 172.21.15.152:0/1946997687 shutdown_connections 2024-09-11T23:03:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.514+0000 7fdeedf72740 1 --2- 172.21.15.152:0/1946997687 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c65f580 0x55b39c661970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.514+0000 7fdeedf72740 1 --2- 172.21.15.152:0/1946997687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c6525d0 0x55b39c652a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.514+0000 7fdeedf72740 1 --2- 172.21.15.152:0/1946997687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c652090 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.533 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.514+0000 7fdeedf72740 1 -- 172.21.15.152:0/1946997687 >> 172.21.15.152:0/1946997687 conn(0x55b39c61b310 msgr2=0x55b39c64ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:05.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.514+0000 7fdeedf72740 1 -- 172.21.15.152:0/1946997687 shutdown_connections 2024-09-11T23:03:05.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.514+0000 7fdeedf72740 1 -- 172.21.15.152:0/1946997687 wait complete. 2024-09-11T23:03:05.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.514+0000 7fdeedf72740 1 -- start start 2024-09-11T23:03:05.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.514+0000 7fdeedf72740 1 Processor -- start 2024-09-11T23:03:05.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdeedf72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c6525d0 0x55b39c65ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdeedf72740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c65c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdeedf72740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c65f580 0x55b39c65c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdeedf72740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55b39c662280 con 0x55b39c653b40 2024-09-11T23:03:05.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdeedf72740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55b39c573060 con 0x55b39c65f580 2024-09-11T23:03:05.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdeedf72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55b39c662400 con 0x55b39c6525d0 2024-09-11T23:03:05.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdeedf72740 10 client.0 _ll_register_callbacks cb 0x55b39c2c68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:03:05.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdee9134640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c65c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.534 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdee9134640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c65c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35746/0 (socket says 172.21.15.152:35746) 2024-09-11T23:03:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdee9134640 1 -- 172.21.15.152:0/568094529 learned_addr learned my addr 172.21.15.152:0/568094529 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdeea136640 1 --2- 172.21.15.152:0/568094529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c65f580 0x55b39c65c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdee9935640 1 --2- 172.21.15.152:0/568094529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c6525d0 0x55b39c65ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdee9134640 1 -- 172.21.15.152:0/568094529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c6525d0 msgr2=0x55b39c65ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdee9134640 1 --2- 172.21.15.152:0/568094529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c6525d0 0x55b39c65ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdee9134640 1 -- 172.21.15.152:0/568094529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c65f580 msgr2=0x55b39c65c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdee9134640 1 --2- 172.21.15.152:0/568094529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c65f580 0x55b39c65c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdee9134640 1 -- 172.21.15.152:0/568094529 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39c3293c0 con 0x55b39c653b40 2024-09-11T23:03:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdee9935640 1 --2- 172.21.15.152:0/568094529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c6525d0 0x55b39c65ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:03:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdee9134640 1 --2- 172.21.15.152:0/568094529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c65c210 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fdee00981b0 tx=0x7fdee00d52b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f78e91db87bcb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdecd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:03:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.515+0000 7fdecd7fa640 1 -- 172.21.15.152:0/568094529 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee009b6b0 con 0x55b39c653b40 2024-09-11T23:03:05.535 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.516+0000 7fdecd7fa640 1 -- 172.21.15.152:0/568094529 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7fdee00a4730 con 0x55b39c653b40 2024-09-11T23:03:05.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.516+0000 7fdeedf72740 1 -- 172.21.15.152:0/568094529 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdedc013c60 con 0x55b39c653b40 2024-09-11T23:03:05.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.516+0000 7fdecd7fa640 1 -- 172.21.15.152:0/568094529 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdee009c450 con 0x55b39c653b40 2024-09-11T23:03:05.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.516+0000 7fdeedf72740 10 client.18336 fetch_fsmap learned FSMap version 28 2024-09-11T23:03:05.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.516+0000 7fdeedf72740 1 -- 172.21.15.152:0/568094529 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b39c66dd60 con 0x55b39c653b40 2024-09-11T23:03:05.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.516+0000 7fdecd7fa640 1 -- 172.21.15.152:0/568094529 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdedc013c60 con 0x55b39c653b40 2024-09-11T23:03:05.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.516+0000 7fdecd7fa640 1 -- 172.21.15.152:0/568094529 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdee009ac10 con 0x55b39c653b40 2024-09-11T23:03:05.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.516+0000 7fdeedf72740 10 client.18336 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:03:05.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.516+0000 7fdeedf72740 10 client.18336 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:03:05.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.516+0000 7fdeedf72740 20 client.18336 populate_metadata read hostname 'smithi152' 2024-09-11T23:03:05.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.516+0000 7fdeedf72740 1 --2- 172.21.15.152:0/568094529 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55b39c66f1c0 0x55b39c6715f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.516+0000 7fdeedf72740 4 client.18336 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:03:05.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.516+0000 7fdeedf72740 1 -- 172.21.15.152:0/568094529 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b39c5c5860 con 0x55b39c66f1c0 2024-09-11T23:03:05.536 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.516+0000 7fdee9935640 1 --2- 172.21.15.152:0/568094529 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55b39c66f1c0 0x55b39c6715f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.517+0000 7fdee9935640 1 --2- 172.21.15.152:0/568094529 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55b39c66f1c0 0x55b39c6715f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c51142584bf39011 server_cookie=3e50829eb61de22e in_seq=0 out_seq=0 2024-09-11T23:03:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.517+0000 7fdecd7fa640 10 client.18336 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T23:03:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.517+0000 7fdecd7fa640 1 -- 172.21.15.152:0/568094529 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b39c5c5860 con 0x55b39c66f1c0 2024-09-11T23:03:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.517+0000 7fdecd7fa640 10 client.18336 handle_command_reply: tid=0 2024-09-11T23:03:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.517+0000 7fdeedf72740 1 client.18336 shutdown 2024-09-11T23:03:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.518+0000 7fdeedf72740 1 -- 172.21.15.152:0/568094529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 msgr2=0x55b39c65c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.518+0000 7fdeedf72740 1 --2- 172.21.15.152:0/568094529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c65c210 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fdee00981b0 tx=0x7fdee00d52b0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.518+0000 7fdeedf72740 1 -- 172.21.15.152:0/568094529 shutdown_connections 2024-09-11T23:03:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.518+0000 7fdeedf72740 1 --2- 172.21.15.152:0/568094529 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55b39c66f1c0 0x55b39c6715f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.518+0000 7fdeedf72740 1 --2- 172.21.15.152:0/568094529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c65f580 0x55b39c65c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.518+0000 7fdeedf72740 1 --2- 172.21.15.152:0/568094529 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c65c210 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.518+0000 7fdeedf72740 1 --2- 172.21.15.152:0/568094529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c6525d0 0x55b39c65ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.518+0000 7fdeedf72740 1 -- 172.21.15.152:0/568094529 >> 172.21.15.152:0/568094529 conn(0x55b39c61b310 msgr2=0x55b39c654960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.518+0000 7fdecd7fa640 0 client.18336 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T23:03:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.518+0000 7fdeedf72740 1 -- 172.21.15.152:0/568094529 shutdown_connections 2024-09-11T23:03:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.518+0000 7fdeedf72740 1 -- 172.21.15.152:0/568094529 wait complete. 2024-09-11T23:03:05.538 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.518+0000 7fdeedf72740 20 client.18336 trim_cache size 0 max 16384 2024-09-11T23:03:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.531+0000 7fdeedf72740 1 Processor -- start 2024-09-11T23:03:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.531+0000 7fdeedf72740 1 -- start start 2024-09-11T23:03:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.531+0000 7fdeedf72740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c6525d0 0x55b39c66b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.531+0000 7fdeedf72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c653b40 0x55b39c676ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.531+0000 7fdeedf72740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c65f580 0x55b39c677400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.531+0000 7fdeedf72740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41ab5d0 con 0x55b39c65f580 2024-09-11T23:03:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.531+0000 7fdeedf72740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41ad110 con 0x55b39c6525d0 2024-09-11T23:03:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.531+0000 7fdeedf72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41abca0 con 0x55b39c653b40 2024-09-11T23:03:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.532+0000 7fdeea136640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c65f580 0x55b39c677400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.532+0000 7fdeea136640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c65f580 0x55b39c677400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35756/0 (socket says 172.21.15.152:35756) 2024-09-11T23:03:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.532+0000 7fdeea136640 1 -- 172.21.15.152:0/927132335 learned_addr learned my addr 172.21.15.152:0/927132335 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:05.543 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.532+0000 7fdee9935640 1 --2- 172.21.15.152:0/927132335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c6525d0 0x55b39c66b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.532+0000 7fdee9134640 1 --2- 172.21.15.152:0/927132335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c653b40 0x55b39c676ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.532+0000 7fdeea136640 1 -- 172.21.15.152:0/927132335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c653b40 msgr2=0x55b39c676ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.532+0000 7fdeea136640 1 --2- 172.21.15.152:0/927132335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c653b40 0x55b39c676ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.532+0000 7fdeea136640 1 -- 172.21.15.152:0/927132335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c6525d0 msgr2=0x55b39c66b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.532+0000 7fdeea136640 1 --2- 172.21.15.152:0/927132335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c6525d0 0x55b39c66b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.532+0000 7fdeea136640 1 -- 172.21.15.152:0/927132335 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39c655120 con 0x55b39c65f580 2024-09-11T23:03:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.532+0000 7fdeea136640 1 --2- 172.21.15.152:0/927132335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c65f580 0x55b39c677400 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fdedc005f10 tx=0x7fdedc015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa5c54927e324ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.532+0000 7fdecf7fe640 1 -- 172.21.15.152:0/927132335 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc035070 con 0x55b39c65f580 2024-09-11T23:03:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.532+0000 7fdecf7fe640 1 -- 172.21.15.152:0/927132335 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdedc009a00 con 0x55b39c65f580 2024-09-11T23:03:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.532+0000 7fdecf7fe640 1 -- 172.21.15.152:0/927132335 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdedc022040 con 0x55b39c65f580 2024-09-11T23:03:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.533+0000 7fdeedf72740 1 -- 172.21.15.152:0/927132335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c65f580 msgr2=0x55b39c677400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.533+0000 7fdeedf72740 1 --2- 172.21.15.152:0/927132335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c65f580 0x55b39c677400 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fdedc005f10 tx=0x7fdedc015040 comp rx=0 tx=0).stop 2024-09-11T23:03:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.533+0000 7fdeedf72740 1 -- 172.21.15.152:0/927132335 shutdown_connections 2024-09-11T23:03:05.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.533+0000 7fdeedf72740 1 --2- 172.21.15.152:0/927132335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c65f580 0x55b39c677400 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.533+0000 7fdeedf72740 1 --2- 172.21.15.152:0/927132335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c653b40 0x55b39c676ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.533+0000 7fdeedf72740 1 --2- 172.21.15.152:0/927132335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c6525d0 0x55b39c66b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.533+0000 7fdeedf72740 1 -- 172.21.15.152:0/927132335 >> 172.21.15.152:0/927132335 conn(0x55b39c61b310 msgr2=0x55b39c65aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.533+0000 7fdeedf72740 1 -- 172.21.15.152:0/927132335 shutdown_connections 2024-09-11T23:03:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.533+0000 7fdeedf72740 1 -- 172.21.15.152:0/927132335 wait complete. 2024-09-11T23:03:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.533+0000 7fdeedf72740 1 -- start start 2024-09-11T23:03:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.533+0000 7fdeedf72740 1 Processor -- start 2024-09-11T23:03:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdeedf72740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c6525d0 0x55b39c663490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdeedf72740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c6639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdeedf72740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c65f580 0x55b39c663f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdeedf72740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41aca40 con 0x55b39c653b40 2024-09-11T23:03:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdeedf72740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41ac370 con 0x55b39c65f580 2024-09-11T23:03:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdeedf72740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fdee41a95d0 con 0x55b39c6525d0 2024-09-11T23:03:05.545 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdeedf72740 10 client.0 _ll_register_callbacks cb 0x7fdee000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:03:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdee9134640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c6639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdee9134640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c6639d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35762/0 (socket says 172.21.15.152:35762) 2024-09-11T23:03:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdee9134640 1 -- 172.21.15.152:0/2992410963 learned_addr learned my addr 172.21.15.152:0/2992410963 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdeea136640 1 --2- 172.21.15.152:0/2992410963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c65f580 0x55b39c663f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdee9935640 1 --2- 172.21.15.152:0/2992410963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c6525d0 0x55b39c663490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdee9134640 1 -- 172.21.15.152:0/2992410963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c6525d0 msgr2=0x55b39c663490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdee9134640 1 --2- 172.21.15.152:0/2992410963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c6525d0 0x55b39c663490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdee9134640 1 -- 172.21.15.152:0/2992410963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c65f580 msgr2=0x55b39c663f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdee9134640 1 --2- 172.21.15.152:0/2992410963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c65f580 0x55b39c663f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.534+0000 7fdee9134640 1 -- 172.21.15.152:0/2992410963 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39c6d8f40 con 0x55b39c653b40 2024-09-11T23:03:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.535+0000 7fdee9134640 1 --2- 172.21.15.152:0/2992410963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c6639d0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fdee000c6a0 tx=0x7fdee0005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8acd7f6347d0fe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.535+0000 7fdecd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:03:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.535+0000 7fdecd7fa640 1 -- 172.21.15.152:0/2992410963 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee00a9060 con 0x55b39c653b40 2024-09-11T23:03:05.546 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.535+0000 7fdecd7fa640 1 -- 172.21.15.152:0/2992410963 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7fdee0097aa0 con 0x55b39c653b40 2024-09-11T23:03:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.535+0000 7fdeedf72740 1 -- 172.21.15.152:0/2992410963 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdedc009a00 con 0x55b39c653b40 2024-09-11T23:03:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.535+0000 7fdecd7fa640 1 -- 172.21.15.152:0/2992410963 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdee009b6b0 con 0x55b39c653b40 2024-09-11T23:03:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.535+0000 7fdeedf72740 10 client.18342 fetch_fsmap learned FSMap version 28 2024-09-11T23:03:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.535+0000 7fdeedf72740 1 -- 172.21.15.152:0/2992410963 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b39c6da220 con 0x55b39c653b40 2024-09-11T23:03:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.535+0000 7fdecd7fa640 1 -- 172.21.15.152:0/2992410963 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdedc009a00 con 0x55b39c653b40 2024-09-11T23:03:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.535+0000 7fdecd7fa640 1 -- 172.21.15.152:0/2992410963 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdee0097ca0 con 0x55b39c653b40 2024-09-11T23:03:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.535+0000 7fdeedf72740 10 client.18342 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:03:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.535+0000 7fdeedf72740 10 client.18342 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-11T23:03:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.535+0000 7fdeedf72740 20 client.18342 populate_metadata read hostname 'smithi152' 2024-09-11T23:03:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.535+0000 7fdeedf72740 1 --2- 172.21.15.152:0/2992410963 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55b39c6da940 0x55b39c6dcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.535+0000 7fdeedf72740 4 client.18342 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:03:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.535+0000 7fdeedf72740 1 -- 172.21.15.152:0/2992410963 --> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b39c5bfd70 con 0x55b39c6da940 2024-09-11T23:03:05.547 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.536+0000 7fdee9935640 1 --2- 172.21.15.152:0/2992410963 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55b39c6da940 0x55b39c6dcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:05.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.536+0000 7fdecd7fa640 10 client.18342 ms_handle_connect on v2:172.21.15.191:6836/314110375 2024-09-11T23:03:05.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.536+0000 7fdee9935640 1 --2- 172.21.15.152:0/2992410963 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55b39c6da940 0x55b39c6dcd60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e380485c681ef53 server_cookie=b97806d38832ff12 in_seq=0 out_seq=0 2024-09-11T23:03:05.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.536+0000 7fdecd7fa640 1 -- 172.21.15.152:0/2992410963 <== mds.2 v2:172.21.15.191:6836/314110375 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b39c5bfd70 con 0x55b39c6da940 2024-09-11T23:03:05.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.536+0000 7fdecd7fa640 10 client.18342 handle_command_reply: tid=0 2024-09-11T23:03:05.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.536+0000 7fdeedf72740 1 client.18342 shutdown 2024-09-11T23:03:05.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.536+0000 7fdeedf72740 1 -- 172.21.15.152:0/2992410963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 msgr2=0x55b39c6639d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:05.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.536+0000 7fdeedf72740 1 --2- 172.21.15.152:0/2992410963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c6639d0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fdee000c6a0 tx=0x7fdee0005b40 comp rx=0 tx=0).stop 2024-09-11T23:03:05.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeedf72740 1 -- 172.21.15.152:0/2992410963 shutdown_connections 2024-09-11T23:03:05.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeedf72740 1 --2- 172.21.15.152:0/2992410963 >> [v2:172.21.15.191:6836/314110375,v1:172.21.15.191:6837/314110375] conn(0x55b39c6da940 0x55b39c6dcd60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeedf72740 1 --2- 172.21.15.152:0/2992410963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55b39c65f580 0x55b39c663f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeedf72740 1 --2- 172.21.15.152:0/2992410963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b39c653b40 0x55b39c6639d0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdecd7fa640 0 client.18342 ms_handle_reset on v2:172.21.15.191:6836/314110375 2024-09-11T23:03:05.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeedf72740 1 --2- 172.21.15.152:0/2992410963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b39c6525d0 0x55b39c663490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeedf72740 1 -- 172.21.15.152:0/2992410963 >> 172.21.15.152:0/2992410963 conn(0x55b39c61b310 msgr2=0x55b39c65da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeedf72740 1 -- 172.21.15.152:0/2992410963 shutdown_connections 2024-09-11T23:03:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeedf72740 1 -- 172.21.15.152:0/2992410963 wait complete. 2024-09-11T23:03:05.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeedf72740 20 client.18342 trim_cache size 0 max 16384 2024-09-11T23:03:05.549 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:03:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeebb99640 1 -- 172.21.15.152:0/4243727759 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fded00a2cf0 msgr2=0x7fded00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeebb99640 1 --2- 172.21.15.152:0/4243727759 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fded00a2cf0 0x7fded00a51b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdedc01f000 tx=0x7fdedc01c000 comp rx=0 tx=0).stop 2024-09-11T23:03:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeebb99640 1 -- 172.21.15.152:0/4243727759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdee4101420 msgr2=0x7fdee41978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeebb99640 1 --2- 172.21.15.152:0/4243727759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdee4101420 0x7fdee41978e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fdee000e9f0 tx=0x7fdee000edf0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeebb99640 1 -- 172.21.15.152:0/4243727759 shutdown_connections 2024-09-11T23:03:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeebb99640 1 --2- 172.21.15.152:0/4243727759 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fded00a2cf0 0x7fded00a51b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeebb99640 1 --2- 172.21.15.152:0/4243727759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdee4106a80 0x7fdee419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeebb99640 1 --2- 172.21.15.152:0/4243727759 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdee4101420 0x7fdee41978e0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeebb99640 1 --2- 172.21.15.152:0/4243727759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee4100a80 0x7fdee41973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeebb99640 1 -- 172.21.15.152:0/4243727759 >> 172.21.15.152:0/4243727759 conn(0x7fdee40fc7a0 msgr2=0x7fdee40fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeebb99640 1 -- 172.21.15.152:0/4243727759 shutdown_connections 2024-09-11T23:03:05.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:05.537+0000 7fdeebb99640 1 -- 172.21.15.152:0/4243727759 wait complete. 2024-09-11T23:03:06.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:05 smithi152 ceph-mon[31181]: from='client.18324 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:03:06.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:05 smithi191 ceph-mon[35211]: from='client.18324 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:03:06.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:05 smithi204 ceph-mon[35454]: from='client.18324 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T23:03:06.329 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T23:03:06.578 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:07.029 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:06 smithi152 ceph-mon[31181]: pgmap v315: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-11T23:03:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.109+0000 7fb8c04bb640 1 -- 172.21.15.152:0/1533176663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b8100ca0 msgr2=0x7fb8b8106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.109+0000 7fb8c04bb640 1 --2- 172.21.15.152:0/1533176663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b8100ca0 0x7fb8b8106840 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009930 tx=0x7fb8b402a420 comp rx=0 tx=0).stop 2024-09-11T23:03:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.109+0000 7fb8c04bb640 1 -- 172.21.15.152:0/1533176663 shutdown_connections 2024-09-11T23:03:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.109+0000 7fb8c04bb640 1 --2- 172.21.15.152:0/1533176663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8b8106f50 0x7fb8b810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.139 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.109+0000 7fb8c04bb640 1 --2- 172.21.15.152:0/1533176663 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b8100ca0 0x7fb8b8106840 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.109+0000 7fb8c04bb640 1 --2- 172.21.15.152:0/1533176663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8b81002f0 0x7fb8b81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.109+0000 7fb8c04bb640 1 -- 172.21.15.152:0/1533176663 >> 172.21.15.152:0/1533176663 conn(0x7fb8b8077210 msgr2=0x7fb8b80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:07.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.109+0000 7fb8c04bb640 1 -- 172.21.15.152:0/1533176663 shutdown_connections 2024-09-11T23:03:07.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.109+0000 7fb8c04bb640 1 -- 172.21.15.152:0/1533176663 wait complete. 2024-09-11T23:03:07.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.110+0000 7fb8c04bb640 1 Processor -- start 2024-09-11T23:03:07.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.110+0000 7fb8c04bb640 1 -- start start 2024-09-11T23:03:07.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.110+0000 7fb8c04bb640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8b81002f0 0x7fb8b8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.110+0000 7fb8c04bb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b8100ca0 0x7fb8b8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.110+0000 7fb8c04bb640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8b8106f50 0x7fb8b819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.110+0000 7fb8c04bb640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8111fa0 con 0x7fb8b8100ca0 2024-09-11T23:03:07.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.110+0000 7fb8c04bb640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8111e20 con 0x7fb8b81002f0 2024-09-11T23:03:07.140 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.110+0000 7fb8c04bb640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8112120 con 0x7fb8b8106f50 2024-09-11T23:03:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.110+0000 7fb8bda56640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b8100ca0 0x7fb8b8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.110+0000 7fb8bda56640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b8100ca0 0x7fb8b8198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35790/0 (socket says 172.21.15.152:35790) 2024-09-11T23:03:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.110+0000 7fb8bda56640 1 -- 172.21.15.152:0/1920685641 learned_addr learned my addr 172.21.15.152:0/1920685641 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.110+0000 7fb8be257640 1 --2- 172.21.15.152:0/1920685641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8b81002f0 0x7fb8b8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.110+0000 7fb8bea58640 1 --2- 172.21.15.152:0/1920685641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8b8106f50 0x7fb8b819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.111+0000 7fb8bda56640 1 -- 172.21.15.152:0/1920685641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8b8106f50 msgr2=0x7fb8b819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.111+0000 7fb8bda56640 1 --2- 172.21.15.152:0/1920685641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8b8106f50 0x7fb8b819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.111+0000 7fb8bda56640 1 -- 172.21.15.152:0/1920685641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8b81002f0 msgr2=0x7fb8b8198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.111+0000 7fb8bda56640 1 --2- 172.21.15.152:0/1920685641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8b81002f0 0x7fb8b8198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.111+0000 7fb8bda56640 1 -- 172.21.15.152:0/1920685641 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b819d4d0 con 0x7fb8b8100ca0 2024-09-11T23:03:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.111+0000 7fb8bea58640 1 --2- 172.21.15.152:0/1920685641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8b8106f50 0x7fb8b819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:03:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.111+0000 7fb8bda56640 1 --2- 172.21.15.152:0/1920685641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b8100ca0 0x7fb8b8198a60 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4005940 tx=0x7fb8b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.111+0000 7fb8ab7fe640 1 -- 172.21.15.152:0/1920685641 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b403f030 con 0x7fb8b8100ca0 2024-09-11T23:03:07.141 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.111+0000 7fb8ab7fe640 1 -- 172.21.15.152:0/1920685641 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8b4003ce0 con 0x7fb8b8100ca0 2024-09-11T23:03:07.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.111+0000 7fb8c04bb640 1 -- 172.21.15.152:0/1920685641 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b819d760 con 0x7fb8b8100ca0 2024-09-11T23:03:07.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.111+0000 7fb8ab7fe640 1 -- 172.21.15.152:0/1920685641 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4035070 con 0x7fb8b8100ca0 2024-09-11T23:03:07.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.111+0000 7fb8c04bb640 1 -- 172.21.15.152:0/1920685641 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b81a50a0 con 0x7fb8b8100ca0 2024-09-11T23:03:07.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.113+0000 7fb8ab7fe640 1 -- 172.21.15.152:0/1920685641 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb8b4042030 con 0x7fb8b8100ca0 2024-09-11T23:03:07.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.113+0000 7fb8c2894740 1 Processor -- start 2024-09-11T23:03:07.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.114+0000 7fb8c2894740 1 -- start start 2024-09-11T23:03:07.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.114+0000 7fb8c2894740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a8140b40 0x5569a813f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.114+0000 7fb8c2894740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a813f5d0 0x5569a813fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.114+0000 7fb8c2894740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a814a370 0x5569a814c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.114+0000 7fb8c2894740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5569a80a52f0 con 0x5569a8140b40 2024-09-11T23:03:07.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.114+0000 7fb8c2894740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5569a804ccf0 con 0x5569a813f5d0 2024-09-11T23:03:07.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.114+0000 7fb8c2894740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5569a80a12b0 con 0x5569a814a370 2024-09-11T23:03:07.142 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.114+0000 7fb8ab7fe640 1 --2- 172.21.15.152:0/1920685641 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb8a40a2cf0 0x7fb8a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.114+0000 7fb8bea58640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a8140b40 0x5569a813f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.114+0000 7fb8bea58640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a8140b40 0x5569a813f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35800/0 (socket says 172.21.15.152:35800) 2024-09-11T23:03:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.114+0000 7fb8bea58640 1 -- 172.21.15.152:0/1996765988 learned_addr learned my addr 172.21.15.152:0/1996765988 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.114+0000 7fb8bea58640 1 --2- 172.21.15.152:0/1920685641 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb8a40a2cf0 0x7fb8a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.114+0000 7fb8ab7fe640 1 -- 172.21.15.152:0/1920685641 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7fb8b40ef950 con 0x7fb8b8100ca0 2024-09-11T23:03:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.114+0000 7fb8be257640 1 --2- 172.21.15.152:0/1996765988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a813f5d0 0x5569a813fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.114+0000 7fb8bda56640 1 --2- 172.21.15.152:0/1996765988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a814a370 0x5569a814c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.115+0000 7fb8bea58640 1 -- 172.21.15.152:0/1996765988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a814a370 msgr2=0x5569a814c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.115+0000 7fb8bea58640 1 --2- 172.21.15.152:0/1996765988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a814a370 0x5569a814c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.115+0000 7fb8bea58640 1 -- 172.21.15.152:0/1996765988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a813f5d0 msgr2=0x5569a813fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.115+0000 7fb8bea58640 1 --2- 172.21.15.152:0/1996765988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a813f5d0 0x5569a813fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.115+0000 7fb8bea58640 1 -- 172.21.15.152:0/1996765988 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569a80313a0 con 0x5569a8140b40 2024-09-11T23:03:07.143 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.115+0000 7fb8bea58640 1 --2- 172.21.15.152:0/1996765988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a8140b40 0x5569a813f090 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00092f0 tx=0x7fb8b0013cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949d028f4db7208 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:07.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.115+0000 7fb8a8ff9640 1 -- 172.21.15.152:0/1996765988 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b001d9f0 con 0x5569a8140b40 2024-09-11T23:03:07.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.115+0000 7fb8a8ff9640 1 -- 172.21.15.152:0/1996765988 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8b001db90 con 0x5569a8140b40 2024-09-11T23:03:07.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.115+0000 7fb8bea58640 1 --2- 172.21.15.152:0/1920685641 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb8a40a2cf0 0x7fb8a40a51b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0020000 tx=0x7fb8b001e000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:07.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.115+0000 7fb8a8ff9640 1 -- 172.21.15.152:0/1996765988 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b001cce0 con 0x5569a8140b40 2024-09-11T23:03:07.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.116+0000 7fb8c2894740 1 -- 172.21.15.152:0/1996765988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a8140b40 msgr2=0x5569a813f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:07.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.116+0000 7fb8c2894740 1 --2- 172.21.15.152:0/1996765988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a8140b40 0x5569a813f090 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00092f0 tx=0x7fb8b0013cc0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.116+0000 7fb8c2894740 1 -- 172.21.15.152:0/1996765988 shutdown_connections 2024-09-11T23:03:07.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.116+0000 7fb8c2894740 1 --2- 172.21.15.152:0/1996765988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a814a370 0x5569a814c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.144 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.116+0000 7fb8c2894740 1 --2- 172.21.15.152:0/1996765988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a813f5d0 0x5569a813fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.116+0000 7fb8c2894740 1 --2- 172.21.15.152:0/1996765988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a8140b40 0x5569a813f090 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.116+0000 7fb8c2894740 1 -- 172.21.15.152:0/1996765988 >> 172.21.15.152:0/1996765988 conn(0x5569a8108310 msgr2=0x5569a813bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:07.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.116+0000 7fb8c2894740 1 -- 172.21.15.152:0/1996765988 shutdown_connections 2024-09-11T23:03:07.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.116+0000 7fb8c2894740 1 -- 172.21.15.152:0/1996765988 wait complete. 2024-09-11T23:03:07.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.116+0000 7fb8c2894740 1 -- start start 2024-09-11T23:03:07.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.116+0000 7fb8c2894740 1 Processor -- start 2024-09-11T23:03:07.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.116+0000 7fb8c2894740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a813f5d0 0x5569a8149d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.116+0000 7fb8c2894740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a8140b40 0x5569a8147000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.116+0000 7fb8c2894740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a814a370 0x5569a81475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.145 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.116+0000 7fb8c2894740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x5569a814d070 con 0x5569a8140b40 2024-09-11T23:03:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8c2894740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x5569a8060060 con 0x5569a814a370 2024-09-11T23:03:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8c2894740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x5569a814d1f0 con 0x5569a813f5d0 2024-09-11T23:03:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8c2894740 10 client.0 _ll_register_callbacks cb 0x5569a7db38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:03:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8bda56640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a8140b40 0x5569a8147000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8bda56640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a8140b40 0x5569a8147000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35802/0 (socket says 172.21.15.152:35802) 2024-09-11T23:03:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8bda56640 1 -- 172.21.15.152:0/428666097 learned_addr learned my addr 172.21.15.152:0/428666097 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8bea58640 1 --2- 172.21.15.152:0/428666097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a814a370 0x5569a81475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8be257640 1 --2- 172.21.15.152:0/428666097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a813f5d0 0x5569a8149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8bda56640 1 -- 172.21.15.152:0/428666097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a813f5d0 msgr2=0x5569a8149d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8bda56640 1 --2- 172.21.15.152:0/428666097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a813f5d0 0x5569a8149d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8bda56640 1 -- 172.21.15.152:0/428666097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a814a370 msgr2=0x5569a81475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:07.146 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8bda56640 1 --2- 172.21.15.152:0/428666097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a814a370 0x5569a81475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8bda56640 1 -- 172.21.15.152:0/428666097 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569a7e163c0 con 0x5569a8140b40 2024-09-11T23:03:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8bda56640 1 --2- 172.21.15.152:0/428666097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a8140b40 0x5569a8147000 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fb8b40bbe20 tx=0x7fb8b40bbd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196324811256ebfb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:03:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8a1ffb640 1 -- 172.21.15.152:0/428666097 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4049050 con 0x5569a8140b40 2024-09-11T23:03:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8a1ffb640 1 -- 172.21.15.152:0/428666097 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7fb8b40ed340 con 0x5569a8140b40 2024-09-11T23:03:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8c2894740 1 -- 172.21.15.152:0/428666097 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8b001db90 con 0x5569a8140b40 2024-09-11T23:03:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8a1ffb640 1 -- 172.21.15.152:0/428666097 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb8b40044e0 con 0x5569a8140b40 2024-09-11T23:03:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8c2894740 10 client.18354 fetch_fsmap learned FSMap version 28 2024-09-11T23:03:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.117+0000 7fb8c2894740 1 -- 172.21.15.152:0/428666097 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5569a8158ae0 con 0x5569a8140b40 2024-09-11T23:03:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.118+0000 7fb8a1ffb640 1 -- 172.21.15.152:0/428666097 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8b001db90 con 0x5569a8140b40 2024-09-11T23:03:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.118+0000 7fb8a1ffb640 1 -- 172.21.15.152:0/428666097 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb8b4031bd0 con 0x5569a8140b40 2024-09-11T23:03:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.118+0000 7fb8c2894740 10 client.18354 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:03:07.147 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.118+0000 7fb8c2894740 10 client.18354 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:03:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.118+0000 7fb8c2894740 20 client.18354 populate_metadata read hostname 'smithi152' 2024-09-11T23:03:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.118+0000 7fb8c2894740 1 --2- 172.21.15.152:0/428666097 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5569a815a020 0x5569a815c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.118+0000 7fb8c2894740 4 client.18354 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:03:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.118+0000 7fb8c2894740 1 -- 172.21.15.152:0/428666097 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569a80b2860 con 0x5569a815a020 2024-09-11T23:03:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.118+0000 7fb8be257640 1 --2- 172.21.15.152:0/428666097 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5569a815a020 0x5569a815c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.118+0000 7fb8a1ffb640 10 client.18354 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T23:03:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.118+0000 7fb8be257640 1 --2- 172.21.15.152:0/428666097 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5569a815a020 0x5569a815c430 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc21ffbcc7b471cd server_cookie=d6321b7b0e8f4ec1 in_seq=0 out_seq=0 2024-09-11T23:03:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.119+0000 7fb8a1ffb640 1 -- 172.21.15.152:0/428666097 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5569a80b2860 con 0x5569a815a020 2024-09-11T23:03:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.119+0000 7fb8a1ffb640 10 client.18354 handle_command_reply: tid=0 2024-09-11T23:03:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.119+0000 7fb8c2894740 1 client.18354 shutdown 2024-09-11T23:03:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.119+0000 7fb8c2894740 1 -- 172.21.15.152:0/428666097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a8140b40 msgr2=0x5569a8147000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.119+0000 7fb8c2894740 1 --2- 172.21.15.152:0/428666097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a8140b40 0x5569a8147000 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fb8b40bbe20 tx=0x7fb8b40bbd20 comp rx=0 tx=0).stop 2024-09-11T23:03:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.119+0000 7fb8c2894740 1 -- 172.21.15.152:0/428666097 shutdown_connections 2024-09-11T23:03:07.148 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.119+0000 7fb8c2894740 1 --2- 172.21.15.152:0/428666097 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5569a815a020 0x5569a815c430 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.119+0000 7fb8c2894740 1 --2- 172.21.15.152:0/428666097 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a814a370 0x5569a81475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.119+0000 7fb8c2894740 1 --2- 172.21.15.152:0/428666097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a8140b40 0x5569a8147000 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.119+0000 7fb8a1ffb640 0 client.18354 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T23:03:07.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.119+0000 7fb8c2894740 1 --2- 172.21.15.152:0/428666097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a813f5d0 0x5569a8149d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.119+0000 7fb8c2894740 1 -- 172.21.15.152:0/428666097 >> 172.21.15.152:0/428666097 conn(0x5569a8108310 msgr2=0x5569a8141960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:07.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.119+0000 7fb8c2894740 1 -- 172.21.15.152:0/428666097 shutdown_connections 2024-09-11T23:03:07.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.119+0000 7fb8c2894740 1 -- 172.21.15.152:0/428666097 wait complete. 2024-09-11T23:03:07.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.119+0000 7fb8c2894740 20 client.18354 trim_cache size 0 max 16384 2024-09-11T23:03:07.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.132+0000 7fb8c2894740 1 Processor -- start 2024-09-11T23:03:07.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.132+0000 7fb8c2894740 1 -- start start 2024-09-11T23:03:07.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.132+0000 7fb8c2894740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a813f5d0 0x5569a8156310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.149 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.132+0000 7fb8c2894740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a8140b40 0x5569a8152860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.132+0000 7fb8c2894740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a814a370 0x5569a8152da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.132+0000 7fb8c2894740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81ad8c0 con 0x5569a813f5d0 2024-09-11T23:03:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.132+0000 7fb8c2894740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81af400 con 0x5569a8140b40 2024-09-11T23:03:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.132+0000 7fb8c2894740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81adf90 con 0x5569a814a370 2024-09-11T23:03:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.132+0000 7fb8be257640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a813f5d0 0x5569a8156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.132+0000 7fb8be257640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a813f5d0 0x5569a8156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35804/0 (socket says 172.21.15.152:35804) 2024-09-11T23:03:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.132+0000 7fb8be257640 1 -- 172.21.15.152:0/3397677869 learned_addr learned my addr 172.21.15.152:0/3397677869 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.133+0000 7fb8be257640 1 -- 172.21.15.152:0/3397677869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a814a370 msgr2=0x5569a8152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.133+0000 7fb8be257640 1 --2- 172.21.15.152:0/3397677869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a814a370 0x5569a8152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.133+0000 7fb8be257640 1 -- 172.21.15.152:0/3397677869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a8140b40 msgr2=0x5569a8152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.133+0000 7fb8be257640 1 --2- 172.21.15.152:0/3397677869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a8140b40 0x5569a8152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.133+0000 7fb8be257640 1 -- 172.21.15.152:0/3397677869 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569a815ca80 con 0x5569a813f5d0 2024-09-11T23:03:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.133+0000 7fb8be257640 1 --2- 172.21.15.152:0/3397677869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a813f5d0 0x5569a8156310 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x5569a815cc50 tx=0x7fb8ac00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91605d2455b3164 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:07.150 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.133+0000 7fb8a3fff640 1 -- 172.21.15.152:0/3397677869 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac026440 con 0x5569a813f5d0 2024-09-11T23:03:07.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.133+0000 7fb8a3fff640 1 -- 172.21.15.152:0/3397677869 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8ac026ac0 con 0x5569a813f5d0 2024-09-11T23:03:07.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.133+0000 7fb8a3fff640 1 -- 172.21.15.152:0/3397677869 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac0119d0 con 0x5569a813f5d0 2024-09-11T23:03:07.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.133+0000 7fb8c2894740 1 -- 172.21.15.152:0/3397677869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a813f5d0 msgr2=0x5569a8156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:07.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.133+0000 7fb8c2894740 1 --2- 172.21.15.152:0/3397677869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a813f5d0 0x5569a8156310 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x5569a815cc50 tx=0x7fb8ac00b390 comp rx=0 tx=0).stop 2024-09-11T23:03:07.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.133+0000 7fb8c2894740 1 -- 172.21.15.152:0/3397677869 shutdown_connections 2024-09-11T23:03:07.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8c2894740 1 --2- 172.21.15.152:0/3397677869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a814a370 0x5569a8152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8c2894740 1 --2- 172.21.15.152:0/3397677869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a8140b40 0x5569a8152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8c2894740 1 --2- 172.21.15.152:0/3397677869 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a813f5d0 0x5569a8156310 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8c2894740 1 -- 172.21.15.152:0/3397677869 >> 172.21.15.152:0/3397677869 conn(0x5569a8108310 msgr2=0x5569a8142310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:07.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8c2894740 1 -- 172.21.15.152:0/3397677869 shutdown_connections 2024-09-11T23:03:07.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8c2894740 1 -- 172.21.15.152:0/3397677869 wait complete. 2024-09-11T23:03:07.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8c2894740 1 -- start start 2024-09-11T23:03:07.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8c2894740 1 Processor -- start 2024-09-11T23:03:07.151 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8c2894740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a813f5d0 0x5569a81c1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8c2894740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a8140b40 0x5569a81c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8c2894740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a814a370 0x5569a81c0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8c2894740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81aed30 con 0x5569a814a370 2024-09-11T23:03:07.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8c2894740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81ae660 con 0x5569a8140b40 2024-09-11T23:03:07.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8c2894740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81ab8c0 con 0x5569a813f5d0 2024-09-11T23:03:07.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8c2894740 10 client.0 _ll_register_callbacks cb 0x7fb8b4009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:03:07.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8bea58640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a814a370 0x5569a81c0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:07.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8bea58640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a814a370 0x5569a81c0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35820/0 (socket says 172.21.15.152:35820) 2024-09-11T23:03:07.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8bea58640 1 -- 172.21.15.152:0/1259711905 learned_addr learned my addr 172.21.15.152:0/1259711905 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:07.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8bda56640 1 --2- 172.21.15.152:0/1259711905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a8140b40 0x5569a81c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:07.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.134+0000 7fb8be257640 1 --2- 172.21.15.152:0/1259711905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a813f5d0 0x5569a81c1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:07.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.135+0000 7fb8bea58640 1 -- 172.21.15.152:0/1259711905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a813f5d0 msgr2=0x5569a81c1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:07.152 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.135+0000 7fb8bea58640 1 --2- 172.21.15.152:0/1259711905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a813f5d0 0x5569a81c1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.135+0000 7fb8bea58640 1 -- 172.21.15.152:0/1259711905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a8140b40 msgr2=0x5569a81c24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.135+0000 7fb8bea58640 1 --2- 172.21.15.152:0/1259711905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a8140b40 0x5569a81c24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.135+0000 7fb8bea58640 1 -- 172.21.15.152:0/1259711905 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569a81c1380 con 0x5569a814a370 2024-09-11T23:03:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.135+0000 7fb8bea58640 1 --2- 172.21.15.152:0/1259711905 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a814a370 0x5569a81c0ce0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009fd0 tx=0x7fb8b00137a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00d22111c3d4db0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.135+0000 7fb8a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:03:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.135+0000 7fb8a1ffb640 1 -- 172.21.15.152:0/1259711905 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0013400 con 0x5569a814a370 2024-09-11T23:03:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.135+0000 7fb8a1ffb640 1 -- 172.21.15.152:0/1259711905 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7fb8b001c3c0 con 0x5569a814a370 2024-09-11T23:03:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.135+0000 7fb8c2894740 1 -- 172.21.15.152:0/1259711905 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8ac026ac0 con 0x5569a814a370 2024-09-11T23:03:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.135+0000 7fb8a1ffb640 1 -- 172.21.15.152:0/1259711905 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb8b001c5c0 con 0x5569a814a370 2024-09-11T23:03:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.135+0000 7fb8c2894740 10 client.18360 fetch_fsmap learned FSMap version 28 2024-09-11T23:03:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.135+0000 7fb8c2894740 1 -- 172.21.15.152:0/1259711905 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5569a81c9910 con 0x5569a814a370 2024-09-11T23:03:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.135+0000 7fb8a1ffb640 1 -- 172.21.15.152:0/1259711905 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8ac026ac0 con 0x5569a814a370 2024-09-11T23:03:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.135+0000 7fb8a1ffb640 1 -- 172.21.15.152:0/1259711905 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb8b0030070 con 0x5569a814a370 2024-09-11T23:03:07.153 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.135+0000 7fb8c2894740 10 client.18360 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:03:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.136+0000 7fb8c2894740 10 client.18360 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-11T23:03:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.136+0000 7fb8c2894740 20 client.18360 populate_metadata read hostname 'smithi152' 2024-09-11T23:03:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.136+0000 7fb8c2894740 1 --2- 172.21.15.152:0/1259711905 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5569a81c9d80 0x5569a81cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.136+0000 7fb8c2894740 4 client.18360 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:03:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.136+0000 7fb8c2894740 1 -- 172.21.15.152:0/1259711905 --> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569a80acd70 con 0x5569a81c9d80 2024-09-11T23:03:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.136+0000 7fb8be257640 1 --2- 172.21.15.152:0/1259711905 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5569a81c9d80 0x5569a81cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.136+0000 7fb8be257640 1 --2- 172.21.15.152:0/1259711905 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5569a81c9d80 0x5569a81cc1a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97c6ac9b89c3dead server_cookie=454c41008d4f5953 in_seq=0 out_seq=0 2024-09-11T23:03:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.136+0000 7fb8a1ffb640 10 client.18360 ms_handle_connect on v2:172.21.15.204:6838/3360312546 2024-09-11T23:03:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.136+0000 7fb8a1ffb640 1 -- 172.21.15.152:0/1259711905 <== mds.1 v2:172.21.15.204:6838/3360312546 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569a80acd70 con 0x5569a81c9d80 2024-09-11T23:03:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.136+0000 7fb8a1ffb640 10 client.18360 handle_command_reply: tid=0 2024-09-11T23:03:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.136+0000 7fb8c2894740 1 client.18360 shutdown 2024-09-11T23:03:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c2894740 1 -- 172.21.15.152:0/1259711905 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a814a370 msgr2=0x5569a81c0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c2894740 1 --2- 172.21.15.152:0/1259711905 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a814a370 0x5569a81c0ce0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009fd0 tx=0x7fb8b00137a0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.154 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c2894740 1 -- 172.21.15.152:0/1259711905 shutdown_connections 2024-09-11T23:03:07.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c2894740 1 --2- 172.21.15.152:0/1259711905 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569a814a370 0x5569a81c0ce0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c2894740 1 --2- 172.21.15.152:0/1259711905 >> [v2:172.21.15.204:6838/3360312546,v1:172.21.15.204:6839/3360312546] conn(0x5569a81c9d80 0x5569a81cc1a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c2894740 1 --2- 172.21.15.152:0/1259711905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5569a8140b40 0x5569a81c24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c2894740 1 --2- 172.21.15.152:0/1259711905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5569a813f5d0 0x5569a81c1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c2894740 1 -- 172.21.15.152:0/1259711905 >> 172.21.15.152:0/1259711905 conn(0x5569a8108310 msgr2=0x5569a813ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:07.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8a1ffb640 0 client.18360 ms_handle_reset on v2:172.21.15.204:6838/3360312546 2024-09-11T23:03:07.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c2894740 1 -- 172.21.15.152:0/1259711905 shutdown_connections 2024-09-11T23:03:07.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c2894740 1 -- 172.21.15.152:0/1259711905 wait complete. 2024-09-11T23:03:07.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c2894740 20 client.18360 trim_cache size 0 max 16384 2024-09-11T23:03:07.156 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:03:07.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c04bb640 1 -- 172.21.15.152:0/1920685641 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb8a40a2cf0 msgr2=0x7fb8a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:07.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c04bb640 1 --2- 172.21.15.152:0/1920685641 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb8a40a2cf0 0x7fb8a40a51b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0020000 tx=0x7fb8b001e000 comp rx=0 tx=0).stop 2024-09-11T23:03:07.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c04bb640 1 -- 172.21.15.152:0/1920685641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b8100ca0 msgr2=0x7fb8b8198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:07.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c04bb640 1 --2- 172.21.15.152:0/1920685641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b8100ca0 0x7fb8b8198a60 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4005940 tx=0x7fb8b40042f0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c04bb640 1 -- 172.21.15.152:0/1920685641 shutdown_connections 2024-09-11T23:03:07.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c04bb640 1 --2- 172.21.15.152:0/1920685641 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb8a40a2cf0 0x7fb8a40a51b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c04bb640 1 --2- 172.21.15.152:0/1920685641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8b8106f50 0x7fb8b819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c04bb640 1 --2- 172.21.15.152:0/1920685641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b8100ca0 0x7fb8b8198a60 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c04bb640 1 --2- 172.21.15.152:0/1920685641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb8b81002f0 0x7fb8b8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:07.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c04bb640 1 -- 172.21.15.152:0/1920685641 >> 172.21.15.152:0/1920685641 conn(0x7fb8b8077210 msgr2=0x7fb8b8105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:07.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c04bb640 1 -- 172.21.15.152:0/1920685641 shutdown_connections 2024-09-11T23:03:07.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:07.137+0000 7fb8c04bb640 1 -- 172.21.15.152:0/1920685641 wait complete. 2024-09-11T23:03:07.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:06 smithi191 ceph-mon[35211]: pgmap v315: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-11T23:03:07.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:06 smithi204 ceph-mon[35454]: pgmap v315: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-09-11T23:03:07.888 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T23:03:08.134 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:08.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.635+0000 7f1f6bba9640 1 -- 172.21.15.152:0/2728356456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f64101290 msgr2=0x7f1f64107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:08.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.635+0000 7f1f6bba9640 1 --2- 172.21.15.152:0/2728356456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f64101290 0x7f1f64107a50 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009930 tx=0x7f1f5402a420 comp rx=0 tx=0).stop 2024-09-11T23:03:08.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.636+0000 7f1f6bba9640 1 -- 172.21.15.152:0/2728356456 shutdown_connections 2024-09-11T23:03:08.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.636+0000 7f1f6bba9640 1 --2- 172.21.15.152:0/2728356456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f64108160 0x7f1f6410eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.636+0000 7f1f6bba9640 1 --2- 172.21.15.152:0/2728356456 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f64101290 0x7f1f64107a50 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.636+0000 7f1f6bba9640 1 --2- 172.21.15.152:0/2728356456 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f641008e0 0x7f1f64100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.636+0000 7f1f6bba9640 1 -- 172.21.15.152:0/2728356456 >> 172.21.15.152:0/2728356456 conn(0x7f1f64077210 msgr2=0x7f1f64077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:08.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.636+0000 7f1f6bba9640 1 -- 172.21.15.152:0/2728356456 shutdown_connections 2024-09-11T23:03:08.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.636+0000 7f1f6bba9640 1 -- 172.21.15.152:0/2728356456 wait complete. 2024-09-11T23:03:08.681 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.636+0000 7f1f6bba9640 1 Processor -- start 2024-09-11T23:03:08.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.636+0000 7f1f6bba9640 1 -- start start 2024-09-11T23:03:08.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.636+0000 7f1f6bba9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f641008e0 0x7f1f641984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.636+0000 7f1f6bba9640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f64101290 0x7f1f64198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.636+0000 7f1f6bba9640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f64108160 0x7f1f6419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.636+0000 7f1f6bba9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1f64111fb0 con 0x7f1f641008e0 2024-09-11T23:03:08.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.636+0000 7f1f6bba9640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1f64111e30 con 0x7f1f64101290 2024-09-11T23:03:08.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.636+0000 7f1f6bba9640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f64112130 con 0x7f1f64108160 2024-09-11T23:03:08.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.637+0000 7f1f69945640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f641008e0 0x7f1f641984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.637+0000 7f1f69945640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f641008e0 0x7f1f641984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60140/0 (socket says 172.21.15.152:60140) 2024-09-11T23:03:08.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.637+0000 7f1f69945640 1 -- 172.21.15.152:0/950746363 learned_addr learned my addr 172.21.15.152:0/950746363 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:08.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.637+0000 7f1f6a146640 1 --2- 172.21.15.152:0/950746363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f64108160 0x7f1f6419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.637+0000 7f1f69144640 1 --2- 172.21.15.152:0/950746363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f64101290 0x7f1f64198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.682 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.637+0000 7f1f69945640 1 -- 172.21.15.152:0/950746363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f64108160 msgr2=0x7f1f6419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:08.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.637+0000 7f1f69945640 1 --2- 172.21.15.152:0/950746363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f64108160 0x7f1f6419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.637+0000 7f1f69945640 1 -- 172.21.15.152:0/950746363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f64101290 msgr2=0x7f1f64198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:08.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.637+0000 7f1f69945640 1 --2- 172.21.15.152:0/950746363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f64101290 0x7f1f64198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.637+0000 7f1f69945640 1 -- 172.21.15.152:0/950746363 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6419d480 con 0x7f1f641008e0 2024-09-11T23:03:08.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.637+0000 7f1f69945640 1 --2- 172.21.15.152:0/950746363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f641008e0 0x7f1f641984d0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000e9f0 tx=0x7f1f6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:08.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.637+0000 7f1f5affd640 1 -- 172.21.15.152:0/950746363 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6000cd80 con 0x7f1f641008e0 2024-09-11T23:03:08.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.637+0000 7f1f6bba9640 1 -- 172.21.15.152:0/950746363 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6419d770 con 0x7f1f641008e0 2024-09-11T23:03:08.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.637+0000 7f1f5affd640 1 -- 172.21.15.152:0/950746363 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f60010910 con 0x7f1f641008e0 2024-09-11T23:03:08.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.637+0000 7f1f5affd640 1 -- 172.21.15.152:0/950746363 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f600188f0 con 0x7f1f641008e0 2024-09-11T23:03:08.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.637+0000 7f1f6bba9640 1 -- 172.21.15.152:0/950746363 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f641a5050 con 0x7f1f641008e0 2024-09-11T23:03:08.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.639+0000 7f1f5affd640 1 -- 172.21.15.152:0/950746363 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1f60010ac0 con 0x7f1f641008e0 2024-09-11T23:03:08.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.640+0000 7f1f6df82740 1 Processor -- start 2024-09-11T23:03:08.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.640+0000 7f1f6df82740 1 -- start start 2024-09-11T23:03:08.683 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.640+0000 7f1f6df82740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403739b40 0x55c403738090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.640+0000 7f1f6df82740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4037385d0 0x55c403738a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.640+0000 7f1f6df82740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c403743370 0x55c403745760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.640+0000 7f1f6df82740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55c40369e2f0 con 0x55c403743370 2024-09-11T23:03:08.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.640+0000 7f1f6df82740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c403645cf0 con 0x55c403739b40 2024-09-11T23:03:08.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.640+0000 7f1f6df82740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c40369a2b0 con 0x55c4037385d0 2024-09-11T23:03:08.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.640+0000 7f1f5affd640 1 --2- 172.21.15.152:0/950746363 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1f500a2cf0 0x7f1f500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.640+0000 7f1f69144640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c403743370 0x55c403745760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.640+0000 7f1f69144640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c403743370 0x55c403745760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60142/0 (socket says 172.21.15.152:60142) 2024-09-11T23:03:08.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.640+0000 7f1f6a146640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403739b40 0x55c403738090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.640+0000 7f1f69144640 1 -- 172.21.15.152:0/387571382 learned_addr learned my addr 172.21.15.152:0/387571382 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:08.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.641+0000 7f1f69945640 1 --2- 172.21.15.152:0/387571382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4037385d0 0x55c403738a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.684 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.641+0000 7f1f6a146640 1 --2- 172.21.15.152:0/950746363 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1f500a2cf0 0x7f1f500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.641+0000 7f1f5affd640 1 -- 172.21.15.152:0/950746363 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f1f600cf7c0 con 0x7f1f641008e0 2024-09-11T23:03:08.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.641+0000 7f1f69144640 1 -- 172.21.15.152:0/387571382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4037385d0 msgr2=0x55c403738a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:08.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.641+0000 7f1f69144640 1 --2- 172.21.15.152:0/387571382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4037385d0 0x55c403738a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.641+0000 7f1f69144640 1 -- 172.21.15.152:0/387571382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403739b40 msgr2=0x55c403738090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:08.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.641+0000 7f1f69144640 1 --2- 172.21.15.152:0/387571382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403739b40 0x55c403738090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.641+0000 7f1f69144640 1 -- 172.21.15.152:0/387571382 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c40362a3a0 con 0x55c403743370 2024-09-11T23:03:08.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.641+0000 7f1f6a146640 1 --2- 172.21.15.152:0/387571382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403739b40 0x55c403738090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:03:08.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.641+0000 7f1f6a146640 1 --2- 172.21.15.152:0/950746363 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1f500a2cf0 0x7f1f500a51b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c005fd0 tx=0x7f1f5c009500 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:08.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.641+0000 7f1f69144640 1 --2- 172.21.15.152:0/387571382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c403743370 0x55c403745760 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009900 tx=0x7f1f54005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7768dba92ead90 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:08.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.641+0000 7f1f4ffff640 1 -- 172.21.15.152:0/387571382 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54040070 con 0x55c403743370 2024-09-11T23:03:08.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.641+0000 7f1f4ffff640 1 -- 172.21.15.152:0/387571382 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f54004030 con 0x55c403743370 2024-09-11T23:03:08.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.641+0000 7f1f4ffff640 1 -- 172.21.15.152:0/387571382 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54035040 con 0x55c403743370 2024-09-11T23:03:08.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.642+0000 7f1f6df82740 1 -- 172.21.15.152:0/387571382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c403743370 msgr2=0x55c403745760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:08.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.642+0000 7f1f6df82740 1 --2- 172.21.15.152:0/387571382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c403743370 0x55c403745760 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f1f54009900 tx=0x7f1f54005d50 comp rx=0 tx=0).stop 2024-09-11T23:03:08.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.642+0000 7f1f6df82740 1 -- 172.21.15.152:0/387571382 shutdown_connections 2024-09-11T23:03:08.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.642+0000 7f1f6df82740 1 --2- 172.21.15.152:0/387571382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c403743370 0x55c403745760 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.642+0000 7f1f6df82740 1 --2- 172.21.15.152:0/387571382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4037385d0 0x55c403738a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.642+0000 7f1f6df82740 1 --2- 172.21.15.152:0/387571382 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403739b40 0x55c403738090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.642+0000 7f1f6df82740 1 -- 172.21.15.152:0/387571382 >> 172.21.15.152:0/387571382 conn(0x55c403701310 msgr2=0x55c403734c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:08.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.642+0000 7f1f6df82740 1 -- 172.21.15.152:0/387571382 shutdown_connections 2024-09-11T23:03:08.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.642+0000 7f1f6df82740 1 -- 172.21.15.152:0/387571382 wait complete. 2024-09-11T23:03:08.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.642+0000 7f1f6df82740 1 -- start start 2024-09-11T23:03:08.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.642+0000 7f1f6df82740 1 Processor -- start 2024-09-11T23:03:08.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f6df82740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4037385d0 0x55c403742d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f6df82740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c403739b40 0x55c403740010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f6df82740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403743370 0x55c403740600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f6df82740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x55c403746010 con 0x55c4037385d0 2024-09-11T23:03:08.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f6df82740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x55c403659060 con 0x55c403743370 2024-09-11T23:03:08.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f6df82740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x55c403746190 con 0x55c403739b40 2024-09-11T23:03:08.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f6df82740 10 client.0 _ll_register_callbacks cb 0x55c4033ac8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:03:08.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f69945640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4037385d0 0x55c403742d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f69945640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4037385d0 0x55c403742d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60144/0 (socket says 172.21.15.152:60144) 2024-09-11T23:03:08.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f69945640 1 -- 172.21.15.152:0/363432414 learned_addr learned my addr 172.21.15.152:0/363432414 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:08.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f69144640 1 --2- 172.21.15.152:0/363432414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c403739b40 0x55c403740010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f6a146640 1 --2- 172.21.15.152:0/363432414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403743370 0x55c403740600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f69945640 1 -- 172.21.15.152:0/363432414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c403739b40 msgr2=0x55c403740010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:08.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f69945640 1 --2- 172.21.15.152:0/363432414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c403739b40 0x55c403740010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f69945640 1 -- 172.21.15.152:0/363432414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403743370 msgr2=0x55c403740600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:08.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f69945640 1 --2- 172.21.15.152:0/363432414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403743370 0x55c403740600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f69945640 1 -- 172.21.15.152:0/363432414 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c40340f3c0 con 0x55c4037385d0 2024-09-11T23:03:08.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f69144640 1 --2- 172.21.15.152:0/363432414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c403739b40 0x55c403740010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:03:08.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f69945640 1 --2- 172.21.15.152:0/363432414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4037385d0 0x55c403742d60 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f1f600097b0 tx=0x7f1f6006ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0455ff0742b613 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:08.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.643+0000 7f1f4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:03:08.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.644+0000 7f1f4d7fa640 1 -- 172.21.15.152:0/363432414 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6009b400 con 0x55c4037385d0 2024-09-11T23:03:08.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.644+0000 7f1f4d7fa640 1 -- 172.21.15.152:0/363432414 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f1f60010430 con 0x55c4037385d0 2024-09-11T23:03:08.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.644+0000 7f1f6df82740 1 -- 172.21.15.152:0/363432414 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f54004030 con 0x55c4037385d0 2024-09-11T23:03:08.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.644+0000 7f1f4d7fa640 1 -- 172.21.15.152:0/363432414 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1f600a4970 con 0x55c4037385d0 2024-09-11T23:03:08.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.644+0000 7f1f6df82740 10 client.18372 fetch_fsmap learned FSMap version 28 2024-09-11T23:03:08.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.644+0000 7f1f6df82740 1 -- 172.21.15.152:0/363432414 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c403751b00 con 0x55c4037385d0 2024-09-11T23:03:08.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.644+0000 7f1f4d7fa640 1 -- 172.21.15.152:0/363432414 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f54004030 con 0x55c4037385d0 2024-09-11T23:03:08.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.644+0000 7f1f4d7fa640 1 -- 172.21.15.152:0/363432414 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1f60086040 con 0x55c4037385d0 2024-09-11T23:03:08.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.644+0000 7f1f6df82740 10 client.18372 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:03:08.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.644+0000 7f1f6df82740 10 client.18372 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:03:08.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.644+0000 7f1f6df82740 20 client.18372 populate_metadata read hostname 'smithi152' 2024-09-11T23:03:08.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.644+0000 7f1f6df82740 1 --2- 172.21.15.152:0/363432414 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c403752fc0 0x55c4037553b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.644+0000 7f1f6df82740 4 client.18372 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T23:03:08.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.644+0000 7f1f6df82740 1 -- 172.21.15.152:0/363432414 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4036ab860 con 0x55c403752fc0 2024-09-11T23:03:08.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.644+0000 7f1f69144640 1 --2- 172.21.15.152:0/363432414 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c403752fc0 0x55c4037553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.645+0000 7f1f4d7fa640 10 client.18372 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:03:08.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.645+0000 7f1f69144640 1 --2- 172.21.15.152:0/363432414 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c403752fc0 0x55c4037553b0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9cd15a4a75e7c79 server_cookie=485839978efa7d19 in_seq=0 out_seq=0 2024-09-11T23:03:08.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.645+0000 7f1f4d7fa640 1 -- 172.21.15.152:0/363432414 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4036ab860 con 0x55c403752fc0 2024-09-11T23:03:08.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.645+0000 7f1f4d7fa640 10 client.18372 handle_command_reply: tid=0 2024-09-11T23:03:08.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.645+0000 7f1f6df82740 1 client.18372 shutdown 2024-09-11T23:03:08.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.646+0000 7f1f6df82740 1 -- 172.21.15.152:0/363432414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4037385d0 msgr2=0x55c403742d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:08.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.646+0000 7f1f6df82740 1 --2- 172.21.15.152:0/363432414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4037385d0 0x55c403742d60 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f1f600097b0 tx=0x7f1f6006ada0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.646+0000 7f1f6df82740 1 -- 172.21.15.152:0/363432414 shutdown_connections 2024-09-11T23:03:08.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.646+0000 7f1f6df82740 1 --2- 172.21.15.152:0/363432414 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c403752fc0 0x55c4037553b0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.646+0000 7f1f6df82740 1 --2- 172.21.15.152:0/363432414 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403743370 0x55c403740600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.646+0000 7f1f6df82740 1 --2- 172.21.15.152:0/363432414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c403739b40 0x55c403740010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.646+0000 7f1f6df82740 1 --2- 172.21.15.152:0/363432414 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4037385d0 0x55c403742d60 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.646+0000 7f1f4d7fa640 0 client.18372 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:03:08.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.646+0000 7f1f6df82740 1 -- 172.21.15.152:0/363432414 >> 172.21.15.152:0/363432414 conn(0x55c403701310 msgr2=0x55c40373a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:08.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.646+0000 7f1f6df82740 1 -- 172.21.15.152:0/363432414 shutdown_connections 2024-09-11T23:03:08.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.646+0000 7f1f6df82740 1 -- 172.21.15.152:0/363432414 wait complete. 2024-09-11T23:03:08.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.646+0000 7f1f6df82740 20 client.18372 trim_cache size 0 max 16384 2024-09-11T23:03:08.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.658+0000 7f1f6df82740 1 Processor -- start 2024-09-11T23:03:08.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f6df82740 1 -- start start 2024-09-11T23:03:08.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f6df82740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4037385d0 0x55c403733bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f6df82740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403739b40 0x55c40373b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f6df82740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c403743370 0x55c40373b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f6df82740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641ad7b0 con 0x55c403743370 2024-09-11T23:03:08.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f6df82740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641af2f0 con 0x55c403739b40 2024-09-11T23:03:08.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f6df82740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641ade80 con 0x55c4037385d0 2024-09-11T23:03:08.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f6a146640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c403743370 0x55c40373b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f6a146640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c403743370 0x55c40373b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60154/0 (socket says 172.21.15.152:60154) 2024-09-11T23:03:08.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f6a146640 1 -- 172.21.15.152:0/1022504602 learned_addr learned my addr 172.21.15.152:0/1022504602 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:08.691 INFO:teuthology.orchestra.run.smithi152.stdout:[] 2024-09-11T23:03:08.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f69945640 1 --2- 172.21.15.152:0/1022504602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4037385d0 0x55c403733bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f69144640 1 --2- 172.21.15.152:0/1022504602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403739b40 0x55c40373b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f6a146640 1 -- 172.21.15.152:0/1022504602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4037385d0 msgr2=0x55c403733bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:08.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f6a146640 1 --2- 172.21.15.152:0/1022504602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4037385d0 0x55c403733bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f6a146640 1 -- 172.21.15.152:0/1022504602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403739b40 msgr2=0x55c40373b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:08.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f6a146640 1 --2- 172.21.15.152:0/1022504602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403739b40 0x55c40373b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f6a146640 1 -- 172.21.15.152:0/1022504602 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c403741100 con 0x55c403743370 2024-09-11T23:03:08.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f6a146640 1 --2- 172.21.15.152:0/1022504602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c403743370 0x55c40373b660 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0114f0 tx=0x7f1f5c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96825aedb63878ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:08.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.659+0000 7f1f4f7fe640 1 -- 172.21.15.152:0/1022504602 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c011e60 con 0x55c403743370 2024-09-11T23:03:08.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.660+0000 7f1f4f7fe640 1 -- 172.21.15.152:0/1022504602 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f5c01f990 con 0x55c403743370 2024-09-11T23:03:08.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.660+0000 7f1f4f7fe640 1 -- 172.21.15.152:0/1022504602 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0055a0 con 0x55c403743370 2024-09-11T23:03:08.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.660+0000 7f1f6df82740 1 -- 172.21.15.152:0/1022504602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c403743370 msgr2=0x55c40373b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:08.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.660+0000 7f1f6df82740 1 --2- 172.21.15.152:0/1022504602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c403743370 0x55c40373b660 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0114f0 tx=0x7f1f5c018000 comp rx=0 tx=0).stop 2024-09-11T23:03:08.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.660+0000 7f1f6df82740 1 -- 172.21.15.152:0/1022504602 shutdown_connections 2024-09-11T23:03:08.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.660+0000 7f1f6df82740 1 --2- 172.21.15.152:0/1022504602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c403743370 0x55c40373b660 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.660+0000 7f1f6df82740 1 --2- 172.21.15.152:0/1022504602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403739b40 0x55c40373b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.660+0000 7f1f6df82740 1 --2- 172.21.15.152:0/1022504602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c4037385d0 0x55c403733bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.660+0000 7f1f6df82740 1 -- 172.21.15.152:0/1022504602 >> 172.21.15.152:0/1022504602 conn(0x55c403701310 msgr2=0x55c4037334e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:08.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.660+0000 7f1f6df82740 1 -- 172.21.15.152:0/1022504602 shutdown_connections 2024-09-11T23:03:08.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.660+0000 7f1f6df82740 1 -- 172.21.15.152:0/1022504602 wait complete. 2024-09-11T23:03:08.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.660+0000 7f1f6df82740 1 -- start start 2024-09-11T23:03:08.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f6df82740 1 Processor -- start 2024-09-11T23:03:08.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f6df82740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4037385d0 0x55c40374bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f6df82740 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403739b40 0x55c403746f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f6df82740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c403743370 0x55c4037474d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f6df82740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641aec20 con 0x55c4037385d0 2024-09-11T23:03:08.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f6df82740 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641ae550 con 0x55c403739b40 2024-09-11T23:03:08.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f6df82740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641ab7b0 con 0x55c403743370 2024-09-11T23:03:08.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f6df82740 10 client.0 _ll_register_callbacks cb 0x7f1f6000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T23:03:08.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f69945640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4037385d0 0x55c40374bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f69945640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4037385d0 0x55c40374bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60162/0 (socket says 172.21.15.152:60162) 2024-09-11T23:03:08.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f69945640 1 -- 172.21.15.152:0/1469009804 learned_addr learned my addr 172.21.15.152:0/1469009804 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:08.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f6a146640 1 --2- 172.21.15.152:0/1469009804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c403743370 0x55c4037474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f69144640 1 --2- 172.21.15.152:0/1469009804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403739b40 0x55c403746f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f69945640 1 -- 172.21.15.152:0/1469009804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c403743370 msgr2=0x55c4037474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:08.694 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f69945640 1 --2- 172.21.15.152:0/1469009804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c403743370 0x55c4037474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f69945640 1 -- 172.21.15.152:0/1469009804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403739b40 msgr2=0x55c403746f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T23:03:08.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f69945640 1 --2- 172.21.15.152:0/1469009804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403739b40 0x55c403746f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f69945640 1 -- 172.21.15.152:0/1469009804 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c403747d00 con 0x55c4037385d0 2024-09-11T23:03:08.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f6a146640 1 --2- 172.21.15.152:0/1469009804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c403743370 0x55c4037474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:03:08.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f69945640 1 --2- 172.21.15.152:0/1469009804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4037385d0 0x55c40374bc20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f1f600cd530 tx=0x7f1f60097d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f374318c54f503f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:08.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.661+0000 7f1f4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-09-11T23:03:08.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.662+0000 7f1f4d7fa640 1 -- 172.21.15.152:0/1469009804 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f600b8020 con 0x55c4037385d0 2024-09-11T23:03:08.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.662+0000 7f1f4d7fa640 1 -- 172.21.15.152:0/1469009804 <== mon.0 v2:172.21.15.152:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8492+0+0 (secure 0 0 0) 0x7f1f60018af0 con 0x55c4037385d0 2024-09-11T23:03:08.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.662+0000 7f1f6df82740 1 -- 172.21.15.152:0/1469009804 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f5c01f990 con 0x55c4037385d0 2024-09-11T23:03:08.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.662+0000 7f1f4d7fa640 1 -- 172.21.15.152:0/1469009804 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1f6009b8f0 con 0x55c4037385d0 2024-09-11T23:03:08.695 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.662+0000 7f1f6df82740 10 client.18378 fetch_fsmap learned FSMap version 28 2024-09-11T23:03:08.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.662+0000 7f1f6df82740 1 -- 172.21.15.152:0/1469009804 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c4037bdc50 con 0x55c4037385d0 2024-09-11T23:03:08.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.662+0000 7f1f4d7fa640 1 -- 172.21.15.152:0/1469009804 <== mon.0 v2:172.21.15.152:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f5c01f990 con 0x55c4037385d0 2024-09-11T23:03:08.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.662+0000 7f1f4d7fa640 1 -- 172.21.15.152:0/1469009804 <== mon.0 v2:172.21.15.152:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1f60068020 con 0x55c4037385d0 2024-09-11T23:03:08.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.662+0000 7f1f6df82740 10 client.18378 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T23:03:08.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.662+0000 7f1f6df82740 10 client.18378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T23:03:08.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.662+0000 7f1f6df82740 20 client.18378 populate_metadata read hostname 'smithi152' 2024-09-11T23:03:08.696 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.662+0000 7f1f6df82740 1 --2- 172.21.15.152:0/1469009804 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c4037be370 0x55c4037c0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:08.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.662+0000 7f1f6df82740 4 client.18378 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T23:03:08.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.662+0000 7f1f6df82740 1 -- 172.21.15.152:0/1469009804 --> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c4036a5d70 con 0x55c4037be370 2024-09-11T23:03:08.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.662+0000 7f1f69144640 1 --2- 172.21.15.152:0/1469009804 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c4037be370 0x55c4037c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:08.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f4d7fa640 10 client.18378 ms_handle_connect on v2:172.21.15.191:6838/2588730805 2024-09-11T23:03:08.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f69144640 1 --2- 172.21.15.152:0/1469009804 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c4037be370 0x55c4037c0760 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa77aa4322bb9511 server_cookie=8ce070201243254c in_seq=0 out_seq=0 2024-09-11T23:03:08.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f4d7fa640 1 -- 172.21.15.152:0/1469009804 <== mds.0 v2:172.21.15.191:6838/2588730805 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c4036a5d70 con 0x55c4037be370 2024-09-11T23:03:08.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f4d7fa640 10 client.18378 handle_command_reply: tid=0 2024-09-11T23:03:08.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f6df82740 1 client.18378 shutdown 2024-09-11T23:03:08.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f6df82740 1 -- 172.21.15.152:0/1469009804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4037385d0 msgr2=0x55c40374bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:08.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f6df82740 1 --2- 172.21.15.152:0/1469009804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4037385d0 0x55c40374bc20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f1f600cd530 tx=0x7f1f60097d30 comp rx=0 tx=0).stop 2024-09-11T23:03:08.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f6df82740 1 -- 172.21.15.152:0/1469009804 shutdown_connections 2024-09-11T23:03:08.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f6df82740 1 --2- 172.21.15.152:0/1469009804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c403743370 0x55c4037474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f6df82740 1 --2- 172.21.15.152:0/1469009804 >> [v2:172.21.15.191:6838/2588730805,v1:172.21.15.191:6839/2588730805] conn(0x55c4037be370 0x55c4037c0760 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f6df82740 1 --2- 172.21.15.152:0/1469009804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55c403739b40 0x55c403746f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f6df82740 1 --2- 172.21.15.152:0/1469009804 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55c4037385d0 0x55c40374bc20 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f4d7fa640 0 client.18378 ms_handle_reset on v2:172.21.15.191:6838/2588730805 2024-09-11T23:03:08.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f6df82740 1 -- 172.21.15.152:0/1469009804 >> 172.21.15.152:0/1469009804 conn(0x55c403701310 msgr2=0x55c40374e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:08.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f6df82740 1 -- 172.21.15.152:0/1469009804 shutdown_connections 2024-09-11T23:03:08.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f6df82740 1 -- 172.21.15.152:0/1469009804 wait complete. 2024-09-11T23:03:08.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.663+0000 7f1f6df82740 20 client.18378 trim_cache size 0 max 16384 2024-09-11T23:03:08.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.664+0000 7f1f6bba9640 1 -- 172.21.15.152:0/950746363 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1f500a2cf0 msgr2=0x7f1f500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:08.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.664+0000 7f1f6bba9640 1 --2- 172.21.15.152:0/950746363 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1f500a2cf0 0x7f1f500a51b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c005fd0 tx=0x7f1f5c009500 comp rx=0 tx=0).stop 2024-09-11T23:03:08.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.664+0000 7f1f6bba9640 1 -- 172.21.15.152:0/950746363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f641008e0 msgr2=0x7f1f641984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:08.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.664+0000 7f1f6bba9640 1 --2- 172.21.15.152:0/950746363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f641008e0 0x7f1f641984d0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000e9f0 tx=0x7f1f6000eeb0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.664+0000 7f1f6bba9640 1 -- 172.21.15.152:0/950746363 shutdown_connections 2024-09-11T23:03:08.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.664+0000 7f1f6bba9640 1 --2- 172.21.15.152:0/950746363 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1f500a2cf0 0x7f1f500a51b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.664+0000 7f1f6bba9640 1 --2- 172.21.15.152:0/950746363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f64108160 0x7f1f6419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.664+0000 7f1f6bba9640 1 --2- 172.21.15.152:0/950746363 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f64101290 0x7f1f64198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.664+0000 7f1f6bba9640 1 --2- 172.21.15.152:0/950746363 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1f641008e0 0x7f1f641984d0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:08.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.664+0000 7f1f6bba9640 1 -- 172.21.15.152:0/950746363 >> 172.21.15.152:0/950746363 conn(0x7f1f64077210 msgr2=0x7f1f64105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:08.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.664+0000 7f1f6bba9640 1 -- 172.21.15.152:0/950746363 shutdown_connections 2024-09-11T23:03:08.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:08.664+0000 7f1f6bba9640 1 -- 172.21.15.152:0/950746363 wait complete. 2024-09-11T23:03:09.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:08 smithi152 ceph-mon[31181]: pgmap v316: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-11T23:03:09.122 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:08 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:03:09.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:08 smithi204 ceph-mon[35454]: pgmap v316: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-11T23:03:09.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:08 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:03:09.179 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:08 smithi191 ceph-mon[35211]: pgmap v316: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-09-11T23:03:09.179 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:08 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T23:03:09.414 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-11T23:03:09.415 INFO:tasks.fwd_scrub:done joining 2024-09-11T23:03:09.415 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-11T23:03:09.427 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T23:03:09.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.543+0000 7fca085a6640 1 -- 172.21.15.152:0/864006280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca0010c0c0 msgr2=0x7fca0010e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:09.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.543+0000 7fca085a6640 1 --2- 172.21.15.152:0/864006280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca0010c0c0 0x7fca0010e530 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8009ca0 tx=0x7fc9f802a950 comp rx=0 tx=0).stop 2024-09-11T23:03:09.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.543+0000 7fca085a6640 1 -- 172.21.15.152:0/864006280 shutdown_connections 2024-09-11T23:03:09.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.543+0000 7fca085a6640 1 --2- 172.21.15.152:0/864006280 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca0010c0c0 0x7fca0010e530 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:09.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.543+0000 7fca085a6640 1 --2- 172.21.15.152:0/864006280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca00103610 0x7fca0010baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:09.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.543+0000 7fca085a6640 1 --2- 172.21.15.152:0/864006280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca00102cf0 0x7fca001030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:09.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.543+0000 7fca085a6640 1 -- 172.21.15.152:0/864006280 >> 172.21.15.152:0/864006280 conn(0x7fca00077a10 msgr2=0x7fca000fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:09.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.543+0000 7fca085a6640 1 -- 172.21.15.152:0/864006280 shutdown_connections 2024-09-11T23:03:09.548 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.543+0000 7fca085a6640 1 -- 172.21.15.152:0/864006280 wait complete. 2024-09-11T23:03:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.544+0000 7fca085a6640 1 Processor -- start 2024-09-11T23:03:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.544+0000 7fca085a6640 1 -- start start 2024-09-11T23:03:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.544+0000 7fca085a6640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca00102cf0 0x7fca00198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.544+0000 7fca085a6640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca00103610 0x7fca001988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.544+0000 7fca085a6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca0010c0c0 0x7fca0019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.544+0000 7fca085a6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fca00111e70 con 0x7fca0010c0c0 2024-09-11T23:03:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.544+0000 7fca085a6640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fca00111cf0 con 0x7fca00102cf0 2024-09-11T23:03:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.544+0000 7fca085a6640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fca00111ff0 con 0x7fca00103610 2024-09-11T23:03:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.544+0000 7fca06b43640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca0010c0c0 0x7fca0019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.544+0000 7fca06342640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca00102cf0 0x7fca00198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.544+0000 7fca06b43640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca0010c0c0 0x7fca0019cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60172/0 (socket says 172.21.15.152:60172) 2024-09-11T23:03:09.549 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.544+0000 7fca06b43640 1 -- 172.21.15.152:0/619967454 learned_addr learned my addr 172.21.15.152:0/619967454 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.544+0000 7fca05b41640 1 --2- 172.21.15.152:0/619967454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca00103610 0x7fca001988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.545+0000 7fca06b43640 1 -- 172.21.15.152:0/619967454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca00103610 msgr2=0x7fca001988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.545+0000 7fca06b43640 1 --2- 172.21.15.152:0/619967454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca00103610 0x7fca001988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.545+0000 7fca06b43640 1 -- 172.21.15.152:0/619967454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca00102cf0 msgr2=0x7fca00198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.545+0000 7fca06b43640 1 --2- 172.21.15.152:0/619967454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca00102cf0 0x7fca00198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.545+0000 7fca06b43640 1 -- 172.21.15.152:0/619967454 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0019d370 con 0x7fca0010c0c0 2024-09-11T23:03:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.545+0000 7fca06b43640 1 --2- 172.21.15.152:0/619967454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca0010c0c0 0x7fca0019cc40 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8004af0 tx=0x7fc9f802f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.545+0000 7fc9f77fe640 1 -- 172.21.15.152:0/619967454 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8034040 con 0x7fca0010c0c0 2024-09-11T23:03:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.545+0000 7fc9f77fe640 1 -- 172.21.15.152:0/619967454 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9f8030920 con 0x7fca0010c0c0 2024-09-11T23:03:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.545+0000 7fca085a6640 1 -- 172.21.15.152:0/619967454 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0019d600 con 0x7fca0010c0c0 2024-09-11T23:03:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.545+0000 7fc9f77fe640 1 -- 172.21.15.152:0/619967454 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8004670 con 0x7fca0010c0c0 2024-09-11T23:03:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.545+0000 7fca085a6640 1 -- 172.21.15.152:0/619967454 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca001a4ef0 con 0x7fca0010c0c0 2024-09-11T23:03:09.550 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.546+0000 7fca085a6640 1 -- 172.21.15.152:0/619967454 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca001030d0 con 0x7fca0010c0c0 2024-09-11T23:03:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.547+0000 7fc9f77fe640 1 -- 172.21.15.152:0/619967454 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc9f8037020 con 0x7fca0010c0c0 2024-09-11T23:03:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.548+0000 7fc9f77fe640 1 --2- 172.21.15.152:0/619967454 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9ec0a2d00 0x7fc9ec0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.548+0000 7fca06342640 1 --2- 172.21.15.152:0/619967454 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9ec0a2d00 0x7fc9ec0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:09.551 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.548+0000 7fc9f77fe640 1 -- 172.21.15.152:0/619967454 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8536+0+0 (secure 0 0 0) 0x7fc9f80f0020 con 0x7fca0010c0c0 2024-09-11T23:03:09.552 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.548+0000 7fca06342640 1 --2- 172.21.15.152:0/619967454 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9ec0a2d00 0x7fc9ec0a51c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009bd0 tx=0x7fc9f0005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:09.553 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.552+0000 7fc9f77fe640 1 -- 172.21.15.152:0/619967454 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc9f80bcd40 con 0x7fca0010c0c0 2024-09-11T23:03:09.722 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.720+0000 7fca085a6640 1 -- 172.21.15.152:0/619967454 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca00107170 con 0x7fca0010c0c0 2024-09-11T23:03:09.724 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.723+0000 7fc9f77fe640 1 -- 172.21.15.152:0/619967454 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+23251 (secure 0 0 0) 0x7fc9f80b8030 con 0x7fca0010c0c0 2024-09-11T23:03:09.724 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T23:03:09.725 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":91,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","created":"2024-09-11T22:43:00.379317+0000","modified":"2024-09-11T23:03:08.836429+0000","last_up_change":"2024-09-11T22:50:01.043181+0000","last_in_change":"2024-09-11T22:49:38.073870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:46:10.984707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:50:50.833547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:50:53.047604+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":91,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3cd917b5-77f6-4a4a-8fd0-4ebe3034feac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6803","nonce":1270728007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6805","nonce":1270728007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6809","nonce":1270728007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6807","nonce":1270728007}]},"public_addr":"172.21.15.152:6803/1270728007","cluster_addr":"172.21.15.152:6805/1270728007","heartbeat_back_addr":"172.21.15.152:6809/1270728007","heartbeat_front_addr":"172.21.15.152:6807/1270728007","state":["exists","up"]},{"osd":1,"uuid":"ae589bc4-77b1-4522-9591-b0ce2a5432dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6801","nonce":354467155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6803","nonce":354467155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6807","nonce":354467155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6805","nonce":354467155}]},"public_addr":"172.21.15.191:6801/354467155","cluster_addr":"172.21.15.191:6803/354467155","heartbeat_back_addr":"172.21.15.191:6807/354467155","heartbeat_front_addr":"172.21.15.191:6805/354467155","state":["exists","up"]},{"osd":2,"uuid":"b35ffd0c-d084-43dd-a947-15132d78d59c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6801","nonce":3848396211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6803","nonce":3848396211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6807","nonce":3848396211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6805","nonce":3848396211}]},"public_addr":"172.21.15.204:6801/3848396211","cluster_addr":"172.21.15.204:6803/3848396211","heartbeat_back_addr":"172.21.15.204:6807/3848396211","heartbeat_front_addr":"172.21.15.204:6805/3848396211","state":["exists","up"]},{"osd":3,"uuid":"d66ba7e8-4ab7-449e-9460-2f44ffdf987f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6811","nonce":3351257181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6813","nonce":3351257181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6817","nonce":3351257181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6815","nonce":3351257181}]},"public_addr":"172.21.15.152:6811/3351257181","cluster_addr":"172.21.15.152:6813/3351257181","heartbeat_back_addr":"172.21.15.152:6817/3351257181","heartbeat_front_addr":"172.21.15.152:6815/3351257181","state":["exists","up"]},{"osd":4,"uuid":"e1b56468-6cc7-4d29-988e-9d281118ee36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6809","nonce":2967539563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6811","nonce":2967539563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6815","nonce":2967539563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6813","nonce":2967539563}]},"public_addr":"172.21.15.191:6809/2967539563","cluster_addr":"172.21.15.191:6811/2967539563","heartbeat_back_addr":"172.21.15.191:6815/2967539563","heartbeat_front_addr":"172.21.15.191:6813/2967539563","state":["exists","up"]},{"osd":5,"uuid":"89dabd3e-77e3-4294-9648-b112ef694e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6809","nonce":4283562158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6811","nonce":4283562158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6815","nonce":4283562158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6813","nonce":4283562158}]},"public_addr":"172.21.15.204:6809/4283562158","cluster_addr":"172.21.15.204:6811/4283562158","heartbeat_back_addr":"172.21.15.204:6815/4283562158","heartbeat_front_addr":"172.21.15.204:6813/4283562158","state":["exists","up"]},{"osd":6,"uuid":"46a0fded-b668-4aef-a68f-613cccc83483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6819","nonce":2484234042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6821","nonce":2484234042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6825","nonce":2484234042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6823","nonce":2484234042}]},"public_addr":"172.21.15.152:6819/2484234042","cluster_addr":"172.21.15.152:6821/2484234042","heartbeat_back_addr":"172.21.15.152:6825/2484234042","heartbeat_front_addr":"172.21.15.152:6823/2484234042","state":["exists","up"]},{"osd":7,"uuid":"e674bfe9-e2db-44a3-b74b-a21f198747c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6817","nonce":269608859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6819","nonce":269608859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6823","nonce":269608859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6821","nonce":269608859}]},"public_addr":"172.21.15.191:6817/269608859","cluster_addr":"172.21.15.191:6819/269608859","heartbeat_back_addr":"172.21.15.191:6823/269608859","heartbeat_front_addr":"172.21.15.191:6821/269608859","state":["exists","up"]},{"osd":8,"uuid":"0418bfc9-d6b6-488f-b4c7-0bc6291bb624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6817","nonce":1756190090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6819","nonce":1756190090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6823","nonce":1756190090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6821","nonce":1756190090}]},"public_addr":"172.21.15.204:6817/1756190090","cluster_addr":"172.21.15.204:6819/1756190090","heartbeat_back_addr":"172.21.15.204:6823/1756190090","heartbeat_front_addr":"172.21.15.204:6821/1756190090","state":["exists","up"]},{"osd":9,"uuid":"751a6964-798f-4240-a2ba-7fa9f0c72657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6827","nonce":1301191419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6829","nonce":1301191419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6833","nonce":1301191419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6831","nonce":1301191419}]},"public_addr":"172.21.15.152:6827/1301191419","cluster_addr":"172.21.15.152:6829/1301191419","heartbeat_back_addr":"172.21.15.152:6833/1301191419","heartbeat_front_addr":"172.21.15.152:6831/1301191419","state":["exists","up"]},{"osd":10,"uuid":"90e7c6ba-127c-4b33-b3c6-cedf72128451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6825","nonce":3695294503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6827","nonce":3695294503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6831","nonce":3695294503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6829","nonce":3695294503}]},"public_addr":"172.21.15.191:6825/3695294503","cluster_addr":"172.21.15.191:6827/3695294503","heartbeat_back_addr":"172.21.15.191:6831/3695294503","heartbeat_front_addr":"172.21.15.191:6829/3695294503","state":["exists","up"]},{"osd":11,"uuid":"c76a93b1-a045-477b-98d0-58a2e9d44a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6825","nonce":1709433236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6827","nonce":1709433236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6831","nonce":1709433236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6829","nonce":1709433236}]},"public_addr":"172.21.15.204:6825/1709433236","cluster_addr":"172.21.15.204:6827/1709433236","heartbeat_back_addr":"172.21.15.204:6831/1709433236","heartbeat_front_addr":"172.21.15.204:6829/1709433236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:17.259744+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:42.230389+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:07.172433+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:32.517060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:58.900615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:24.379803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:50.058928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:15.967299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:42.025873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:08.225448+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:33.910531+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:59.466423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:0/2259758812":"2024-09-12T22:52:53.067551+0000","172.21.15.152:0/3900732888":"2024-09-12T22:52:53.067551+0000","172.21.15.152:0/2557435316":"2024-09-12T22:52:53.067551+0000","172.21.15.152:0/3960601922":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6800/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6801/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/4286398432":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2270507277":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2452559014":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/3071755865":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6800/3438861948":"2024-09-12T22:52:53.067551+0000","172.21.15.152:6800/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/1721817189":"2024-09-12T22:52:53.067551+0000","172.21.15.152:0/1542993850":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/3438861948":"2024-09-12T22:52:53.067551+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:03:09.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.725+0000 7fca085a6640 1 -- 172.21.15.152:0/619967454 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9ec0a2d00 msgr2=0x7fc9ec0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:09.726 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.725+0000 7fca085a6640 1 --2- 172.21.15.152:0/619967454 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9ec0a2d00 0x7fc9ec0a51c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009bd0 tx=0x7fc9f0005f90 comp rx=0 tx=0).stop 2024-09-11T23:03:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.725+0000 7fca085a6640 1 -- 172.21.15.152:0/619967454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca0010c0c0 msgr2=0x7fca0019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.725+0000 7fca085a6640 1 --2- 172.21.15.152:0/619967454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca0010c0c0 0x7fca0019cc40 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8004af0 tx=0x7fc9f802f3a0 comp rx=0 tx=0).stop 2024-09-11T23:03:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.725+0000 7fca085a6640 1 -- 172.21.15.152:0/619967454 shutdown_connections 2024-09-11T23:03:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.725+0000 7fca085a6640 1 --2- 172.21.15.152:0/619967454 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fc9ec0a2d00 0x7fc9ec0a51c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.725+0000 7fca085a6640 1 --2- 172.21.15.152:0/619967454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca0010c0c0 0x7fca0019cc40 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.725+0000 7fca085a6640 1 --2- 172.21.15.152:0/619967454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fca00103610 0x7fca001988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.725+0000 7fca085a6640 1 --2- 172.21.15.152:0/619967454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca00102cf0 0x7fca00198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.725+0000 7fca085a6640 1 -- 172.21.15.152:0/619967454 >> 172.21.15.152:0/619967454 conn(0x7fca00077a10 msgr2=0x7fca000fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:09.727 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.725+0000 7fca085a6640 1 -- 172.21.15.152:0/619967454 shutdown_connections 2024-09-11T23:03:09.728 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.725+0000 7fca085a6640 1 -- 172.21.15.152:0/619967454 wait complete. 2024-09-11T23:03:09.738 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T22:46:10.984707+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T22:50:50.833547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T22:50:53.047604+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 91, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T23:03:09.738 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T23:03:09.852 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.851+0000 7fa40d20e640 1 -- 172.21.15.152:0/3427450991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4081069a0 msgr2=0x7fa408106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:09.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.851+0000 7fa40d20e640 1 --2- 172.21.15.152:0/3427450991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4081069a0 0x7fa408106d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009860 tx=0x7fa3f802a340 comp rx=0 tx=0).stop 2024-09-11T23:03:09.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.851+0000 7fa40d20e640 1 -- 172.21.15.152:0/3427450991 shutdown_connections 2024-09-11T23:03:09.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.851+0000 7fa40d20e640 1 --2- 172.21.15.152:0/3427450991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa408101330 0x7fa40810d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:09.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.851+0000 7fa40d20e640 1 --2- 172.21.15.152:0/3427450991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa408100990 0x7fa408100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:09.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.851+0000 7fa40d20e640 1 --2- 172.21.15.152:0/3427450991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4081069a0 0x7fa408106d80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:09.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.851+0000 7fa40d20e640 1 -- 172.21.15.152:0/3427450991 >> 172.21.15.152:0/3427450991 conn(0x7fa4080fc6d0 msgr2=0x7fa4080feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:09.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.851+0000 7fa40d20e640 1 -- 172.21.15.152:0/3427450991 shutdown_connections 2024-09-11T23:03:09.853 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.851+0000 7fa40d20e640 1 -- 172.21.15.152:0/3427450991 wait complete. 2024-09-11T23:03:09.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.852+0000 7fa40d20e640 1 Processor -- start 2024-09-11T23:03:09.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.852+0000 7fa40d20e640 1 -- start start 2024-09-11T23:03:09.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.852+0000 7fa40d20e640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa408100990 0x7fa408198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:09.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.852+0000 7fa40d20e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa408101330 0x7fa408198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:09.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.852+0000 7fa40d20e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4081069a0 0x7fa40819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:09.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.852+0000 7fa40d20e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fa408110db0 con 0x7fa408101330 2024-09-11T23:03:09.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.852+0000 7fa40d20e640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fa408110c30 con 0x7fa408100990 2024-09-11T23:03:09.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.852+0000 7fa40d20e640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fa408110f30 con 0x7fa4081069a0 2024-09-11T23:03:09.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.852+0000 7fa40659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa408101330 0x7fa408198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:09.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.852+0000 7fa40759e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4081069a0 0x7fa40819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:09.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.852+0000 7fa40659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa408101330 0x7fa408198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60190/0 (socket says 172.21.15.152:60190) 2024-09-11T23:03:09.856 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.852+0000 7fa40759e640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4081069a0 0x7fa40819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.152:51450/0 (socket says 172.21.15.152:51450) 2024-09-11T23:03:09.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.852+0000 7fa406d9d640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa408100990 0x7fa408198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:09.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.852+0000 7fa40659c640 1 -- 172.21.15.152:0/2207959153 learned_addr learned my addr 172.21.15.152:0/2207959153 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:09.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.853+0000 7fa40659c640 1 -- 172.21.15.152:0/2207959153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4081069a0 msgr2=0x7fa40819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:09.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.853+0000 7fa40659c640 1 --2- 172.21.15.152:0/2207959153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4081069a0 0x7fa40819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:09.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.853+0000 7fa40659c640 1 -- 172.21.15.152:0/2207959153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa408100990 msgr2=0x7fa408198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:09.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.853+0000 7fa40659c640 1 --2- 172.21.15.152:0/2207959153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa408100990 0x7fa408198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:09.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.853+0000 7fa40659c640 1 -- 172.21.15.152:0/2207959153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40819d400 con 0x7fa408101330 2024-09-11T23:03:09.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.853+0000 7fa406d9d640 1 --2- 172.21.15.152:0/2207959153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa408100990 0x7fa408198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:03:09.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.853+0000 7fa40759e640 1 --2- 172.21.15.152:0/2207959153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4081069a0 0x7fa40819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:03:09.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.853+0000 7fa40659c640 1 --2- 172.21.15.152:0/2207959153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa408101330 0x7fa408198950 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e9c0 tx=0x7fa3fc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:09.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.853+0000 7fa3effff640 1 -- 172.21.15.152:0/2207959153 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc00cd80 con 0x7fa408101330 2024-09-11T23:03:09.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.853+0000 7fa40d20e640 1 -- 172.21.15.152:0/2207959153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40819d6f0 con 0x7fa408101330 2024-09-11T23:03:09.857 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.853+0000 7fa3effff640 1 -- 172.21.15.152:0/2207959153 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3fc0035f0 con 0x7fa408101330 2024-09-11T23:03:09.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.853+0000 7fa3effff640 1 -- 172.21.15.152:0/2207959153 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc010670 con 0x7fa408101330 2024-09-11T23:03:09.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.853+0000 7fa40d20e640 1 -- 172.21.15.152:0/2207959153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4081a4f80 con 0x7fa408101330 2024-09-11T23:03:09.858 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.854+0000 7fa40d20e640 1 -- 172.21.15.152:0/2207959153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa408100df0 con 0x7fa408101330 2024-09-11T23:03:09.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.855+0000 7fa3effff640 1 -- 172.21.15.152:0/2207959153 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa3fc0037a0 con 0x7fa408101330 2024-09-11T23:03:09.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.856+0000 7fa3effff640 1 --2- 172.21.15.152:0/2207959153 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa3f00a2d00 0x7fa3f00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:09.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.856+0000 7fa406d9d640 1 --2- 172.21.15.152:0/2207959153 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa3f00a2d00 0x7fa3f00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:09.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.856+0000 7fa3effff640 1 -- 172.21.15.152:0/2207959153 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8536+0+0 (secure 0 0 0) 0x7fa3fc0cf690 con 0x7fa408101330 2024-09-11T23:03:09.859 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.856+0000 7fa406d9d640 1 --2- 172.21.15.152:0/2207959153 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa3f00a2d00 0x7fa3f00a51c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa3f802a930 tx=0x7fa3f80056d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:09.861 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:09.860+0000 7fa3effff640 1 -- 172.21.15.152:0/2207959153 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3fc09c2a0 con 0x7fa408101330 2024-09-11T23:03:10.014 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:09 smithi152 ceph-mon[31181]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T23:03:10.015 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:09 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/619967454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:03:10.015 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:09 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:10.015 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:09 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:10.015 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:09 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:10.015 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:09 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:10.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.032+0000 7fa40d20e640 1 -- 172.21.15.152:0/2207959153 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa4081a5270 con 0x7fa408101330 2024-09-11T23:03:10.033 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.032+0000 7fa3effff640 1 -- 172.21.15.152:0/2207959153 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fa3fc09bba0 con 0x7fa408101330 2024-09-11T23:03:10.034 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 1 2024-09-11T23:03:10.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.034+0000 7fa40d20e640 1 -- 172.21.15.152:0/2207959153 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa3f00a2d00 msgr2=0x7fa3f00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:10.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.034+0000 7fa40d20e640 1 --2- 172.21.15.152:0/2207959153 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa3f00a2d00 0x7fa3f00a51c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa3f802a930 tx=0x7fa3f80056d0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.034+0000 7fa40d20e640 1 -- 172.21.15.152:0/2207959153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa408101330 msgr2=0x7fa408198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:10.036 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.034+0000 7fa40d20e640 1 --2- 172.21.15.152:0/2207959153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa408101330 0x7fa408198950 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e9c0 tx=0x7fa3fc00edc0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.034+0000 7fa40d20e640 1 -- 172.21.15.152:0/2207959153 shutdown_connections 2024-09-11T23:03:10.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.034+0000 7fa40d20e640 1 --2- 172.21.15.152:0/2207959153 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fa3f00a2d00 0x7fa3f00a51c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.034+0000 7fa40d20e640 1 --2- 172.21.15.152:0/2207959153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4081069a0 0x7fa40819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.034+0000 7fa40d20e640 1 --2- 172.21.15.152:0/2207959153 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa408101330 0x7fa408198950 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.034+0000 7fa40d20e640 1 --2- 172.21.15.152:0/2207959153 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa408100990 0x7fa408198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.037 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.034+0000 7fa40d20e640 1 -- 172.21.15.152:0/2207959153 >> 172.21.15.152:0/2207959153 conn(0x7fa4080fc6d0 msgr2=0x7fa4081047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:10.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.034+0000 7fa40d20e640 1 -- 172.21.15.152:0/2207959153 shutdown_connections 2024-09-11T23:03:10.038 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.034+0000 7fa40d20e640 1 -- 172.21.15.152:0/2207959153 wait complete. 2024-09-11T23:03:10.047 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T23:03:10.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.151+0000 7fe5ac842640 1 -- 172.21.15.152:0/3924878392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5a8102cf0 msgr2=0x7fe5a81030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:10.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.151+0000 7fe5ac842640 1 --2- 172.21.15.152:0/3924878392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5a8102cf0 0x7fe5a81030d0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fe598009900 tx=0x7fe59802a480 comp rx=0 tx=0).stop 2024-09-11T23:03:10.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.151+0000 7fe5ac842640 1 -- 172.21.15.152:0/3924878392 shutdown_connections 2024-09-11T23:03:10.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.151+0000 7fe5ac842640 1 --2- 172.21.15.152:0/3924878392 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5a810c0c0 0x7fe5a810e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.151+0000 7fe5ac842640 1 --2- 172.21.15.152:0/3924878392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5a8103610 0x7fe5a810baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.155 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.151+0000 7fe5ac842640 1 --2- 172.21.15.152:0/3924878392 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5a8102cf0 0x7fe5a81030d0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.151+0000 7fe5ac842640 1 -- 172.21.15.152:0/3924878392 >> 172.21.15.152:0/3924878392 conn(0x7fe5a80779d0 msgr2=0x7fe5a80fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:10.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.151+0000 7fe5ac842640 1 -- 172.21.15.152:0/3924878392 shutdown_connections 2024-09-11T23:03:10.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.151+0000 7fe5ac842640 1 -- 172.21.15.152:0/3924878392 wait complete. 2024-09-11T23:03:10.156 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5ac842640 1 Processor -- start 2024-09-11T23:03:10.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5ac842640 1 -- start start 2024-09-11T23:03:10.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5ac842640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5a8102cf0 0x7fe5a81984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:10.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5ac842640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5a8103610 0x7fe5a81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:10.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5ac842640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5a810c0c0 0x7fe5a819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:10.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5ac842640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8111e90 con 0x7fe5a810c0c0 2024-09-11T23:03:10.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5ac842640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8111d10 con 0x7fe5a8103610 2024-09-11T23:03:10.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5ac842640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8112010 con 0x7fe5a8102cf0 2024-09-11T23:03:10.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5a6d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5a810c0c0 0x7fe5a819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:10.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5a6d9d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5a810c0c0 0x7fe5a819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60222/0 (socket says 172.21.15.152:60222) 2024-09-11T23:03:10.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5a6d9d640 1 -- 172.21.15.152:0/3499118641 learned_addr learned my addr 172.21.15.152:0/3499118641 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:10.157 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5a5d9b640 1 --2- 172.21.15.152:0/3499118641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5a8103610 0x7fe5a81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:10.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5a659c640 1 --2- 172.21.15.152:0/3499118641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5a8102cf0 0x7fe5a81984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:10.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5a6d9d640 1 -- 172.21.15.152:0/3499118641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5a8102cf0 msgr2=0x7fe5a81984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:10.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5a6d9d640 1 --2- 172.21.15.152:0/3499118641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5a8102cf0 0x7fe5a81984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5a6d9d640 1 -- 172.21.15.152:0/3499118641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5a8103610 msgr2=0x7fe5a81989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:10.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5a6d9d640 1 --2- 172.21.15.152:0/3499118641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5a8103610 0x7fe5a81989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.152+0000 7fe5a6d9d640 1 -- 172.21.15.152:0/3499118641 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a819d490 con 0x7fe5a810c0c0 2024-09-11T23:03:10.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.153+0000 7fe5a6d9d640 1 --2- 172.21.15.152:0/3499118641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5a810c0c0 0x7fe5a819cd60 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000efd0 tx=0x7fe5a000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:10.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.153+0000 7fe59f7fe640 1 -- 172.21.15.152:0/3499118641 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0003b70 con 0x7fe5a810c0c0 2024-09-11T23:03:10.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.153+0000 7fe59f7fe640 1 -- 172.21.15.152:0/3499118641 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5a0003d10 con 0x7fe5a810c0c0 2024-09-11T23:03:10.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.153+0000 7fe5ac842640 1 -- 172.21.15.152:0/3499118641 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a819d780 con 0x7fe5a810c0c0 2024-09-11T23:03:10.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.153+0000 7fe59f7fe640 1 -- 172.21.15.152:0/3499118641 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a00109f0 con 0x7fe5a810c0c0 2024-09-11T23:03:10.158 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.153+0000 7fe5ac842640 1 -- 172.21.15.152:0/3499118641 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a81a5010 con 0x7fe5a810c0c0 2024-09-11T23:03:10.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.154+0000 7fe5ac842640 1 -- 172.21.15.152:0/3499118641 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a81030d0 con 0x7fe5a810c0c0 2024-09-11T23:03:10.159 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.156+0000 7fe59f7fe640 1 -- 172.21.15.152:0/3499118641 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe5a0010b90 con 0x7fe5a810c0c0 2024-09-11T23:03:10.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.157+0000 7fe59f7fe640 1 --2- 172.21.15.152:0/3499118641 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5940a2d80 0x7fe5940a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:10.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.157+0000 7fe5a659c640 1 --2- 172.21.15.152:0/3499118641 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5940a2d80 0x7fe5940a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:10.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.157+0000 7fe59f7fe640 1 -- 172.21.15.152:0/3499118641 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8536+0+0 (secure 0 0 0) 0x7fe5a00cfbc0 con 0x7fe5a810c0c0 2024-09-11T23:03:10.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.158+0000 7fe5a659c640 1 --2- 172.21.15.152:0/3499118641 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5940a2d80 0x7fe5940a5240 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe598005ab0 tx=0x7fe598005a20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:10.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.160+0000 7fe59f7fe640 1 -- 172.21.15.152:0/3499118641 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe5a009c7d0 con 0x7fe5a810c0c0 2024-09-11T23:03:10.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:09 smithi191 ceph-mon[35211]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T23:03:10.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:09 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/619967454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:03:10.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:09 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:10.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:09 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:10.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:09 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:10.287 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:09 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:10.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:09 smithi204 ceph-mon[35454]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T23:03:10.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:09 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/619967454' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:03:10.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:09 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:10.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:09 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:10.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:09 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:10.296 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:09 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:10.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.361+0000 7fe5ac842640 1 -- 172.21.15.152:0/3499118641 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe5a802d7c0 con 0x7fe5a810c0c0 2024-09-11T23:03:10.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.362+0000 7fe59f7fe640 1 -- 172.21.15.152:0/3499118641 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7fe5a009c0d0 con 0x7fe5a810c0c0 2024-09-11T23:03:10.363 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 32 2024-09-11T23:03:10.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.364+0000 7fe5ac842640 1 -- 172.21.15.152:0/3499118641 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5940a2d80 msgr2=0x7fe5940a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:10.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.364+0000 7fe5ac842640 1 --2- 172.21.15.152:0/3499118641 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5940a2d80 0x7fe5940a5240 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe598005ab0 tx=0x7fe598005a20 comp rx=0 tx=0).stop 2024-09-11T23:03:10.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.364+0000 7fe5ac842640 1 -- 172.21.15.152:0/3499118641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5a810c0c0 msgr2=0x7fe5a819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:10.365 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.364+0000 7fe5ac842640 1 --2- 172.21.15.152:0/3499118641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5a810c0c0 0x7fe5a819cd60 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000efd0 tx=0x7fe5a000c6a0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.364+0000 7fe5ac842640 1 -- 172.21.15.152:0/3499118641 shutdown_connections 2024-09-11T23:03:10.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.364+0000 7fe5ac842640 1 --2- 172.21.15.152:0/3499118641 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fe5940a2d80 0x7fe5940a5240 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.364+0000 7fe5ac842640 1 --2- 172.21.15.152:0/3499118641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe5a810c0c0 0x7fe5a819cd60 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.364+0000 7fe5ac842640 1 --2- 172.21.15.152:0/3499118641 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5a8103610 0x7fe5a81989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.364+0000 7fe5ac842640 1 --2- 172.21.15.152:0/3499118641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5a8102cf0 0x7fe5a81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.364+0000 7fe5ac842640 1 -- 172.21.15.152:0/3499118641 >> 172.21.15.152:0/3499118641 conn(0x7fe5a80779d0 msgr2=0x7fe5a80fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:10.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.364+0000 7fe5ac842640 1 -- 172.21.15.152:0/3499118641 shutdown_connections 2024-09-11T23:03:10.366 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.364+0000 7fe5ac842640 1 -- 172.21.15.152:0/3499118641 wait complete. 2024-09-11T23:03:10.379 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T23:03:10.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.489+0000 7f1d8b1c4640 1 -- 172.21.15.152:0/4175678493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84101db0 msgr2=0x7f1d841056c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:10.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.489+0000 7f1d8b1c4640 1 --2- 172.21.15.152:0/4175678493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84101db0 0x7f1d841056c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009900 tx=0x7f1d7002a410 comp rx=0 tx=0).stop 2024-09-11T23:03:10.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.489+0000 7f1d8b1c4640 1 -- 172.21.15.152:0/4175678493 shutdown_connections 2024-09-11T23:03:10.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.489+0000 7f1d8b1c4640 1 --2- 172.21.15.152:0/4175678493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d84105df0 0x7f1d8410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.489+0000 7f1d8b1c4640 1 --2- 172.21.15.152:0/4175678493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84101db0 0x7f1d841056c0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.489+0000 7f1d8b1c4640 1 --2- 172.21.15.152:0/4175678493 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d84101400 0x7f1d841017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.489+0000 7f1d8b1c4640 1 -- 172.21.15.152:0/4175678493 >> 172.21.15.152:0/4175678493 conn(0x7f1d84077170 msgr2=0x7f1d840fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:10.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.489+0000 7f1d8b1c4640 1 -- 172.21.15.152:0/4175678493 shutdown_connections 2024-09-11T23:03:10.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.489+0000 7f1d8b1c4640 1 -- 172.21.15.152:0/4175678493 wait complete. 2024-09-11T23:03:10.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.490+0000 7f1d8b1c4640 1 Processor -- start 2024-09-11T23:03:10.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.490+0000 7f1d8b1c4640 1 -- start start 2024-09-11T23:03:10.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.490+0000 7f1d8b1c4640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d84101400 0x7f1d841993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:10.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.490+0000 7f1d8b1c4640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d84105df0 0x7f1d841998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:10.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.490+0000 7f1d8b1c4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84199e20 0x7f1d841a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:10.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.490+0000 7f1d8b1c4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f1d84110e50 con 0x7f1d84199e20 2024-09-11T23:03:10.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.490+0000 7f1d8b1c4640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f1d84110cd0 con 0x7f1d84101400 2024-09-11T23:03:10.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.490+0000 7f1d8b1c4640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f1d84110fd0 con 0x7f1d84105df0 2024-09-11T23:03:10.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.490+0000 7f1d89761640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84199e20 0x7f1d841a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:10.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.490+0000 7f1d89761640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84199e20 0x7f1d841a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60244/0 (socket says 172.21.15.152:60244) 2024-09-11T23:03:10.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.490+0000 7f1d89761640 1 -- 172.21.15.152:0/88676517 learned_addr learned my addr 172.21.15.152:0/88676517 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:10.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.491+0000 7f1d88f60640 1 --2- 172.21.15.152:0/88676517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d84101400 0x7f1d841993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:10.494 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.491+0000 7f1d83fff640 1 --2- 172.21.15.152:0/88676517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d84105df0 0x7f1d841998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:10.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.491+0000 7f1d89761640 1 -- 172.21.15.152:0/88676517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d84105df0 msgr2=0x7f1d841998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:10.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.491+0000 7f1d89761640 1 --2- 172.21.15.152:0/88676517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d84105df0 0x7f1d841998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.491+0000 7f1d89761640 1 -- 172.21.15.152:0/88676517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d84101400 msgr2=0x7f1d841993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:10.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.491+0000 7f1d89761640 1 --2- 172.21.15.152:0/88676517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d84101400 0x7f1d841993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.491+0000 7f1d89761640 1 -- 172.21.15.152:0/88676517 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d841a4b70 con 0x7f1d84199e20 2024-09-11T23:03:10.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.491+0000 7f1d89761640 1 --2- 172.21.15.152:0/88676517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84199e20 0x7f1d841a4420 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00da40 tx=0x7f1d7c00df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:10.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.491+0000 7f1d81ffb640 1 -- 172.21.15.152:0/88676517 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c017070 con 0x7f1d84199e20 2024-09-11T23:03:10.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.491+0000 7f1d81ffb640 1 -- 172.21.15.152:0/88676517 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d7c003e50 con 0x7f1d84199e20 2024-09-11T23:03:10.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.491+0000 7f1d8b1c4640 1 -- 172.21.15.152:0/88676517 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d841a4e60 con 0x7f1d84199e20 2024-09-11T23:03:10.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.491+0000 7f1d81ffb640 1 -- 172.21.15.152:0/88676517 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c010460 con 0x7f1d84199e20 2024-09-11T23:03:10.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.491+0000 7f1d8b1c4640 1 -- 172.21.15.152:0/88676517 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d841a5330 con 0x7f1d84199e20 2024-09-11T23:03:10.495 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.492+0000 7f1d8b1c4640 1 -- 172.21.15.152:0/88676517 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d84100670 con 0x7f1d84199e20 2024-09-11T23:03:10.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.493+0000 7f1d81ffb640 1 -- 172.21.15.152:0/88676517 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1d7c003970 con 0x7f1d84199e20 2024-09-11T23:03:10.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.495+0000 7f1d81ffb640 1 --2- 172.21.15.152:0/88676517 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1d740a2d30 0x7f1d740a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:10.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.495+0000 7f1d88f60640 1 --2- 172.21.15.152:0/88676517 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1d740a2d30 0x7f1d740a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:10.496 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.495+0000 7f1d81ffb640 1 -- 172.21.15.152:0/88676517 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8536+0+0 (secure 0 0 0) 0x7f1d7c0cefe0 con 0x7f1d84199e20 2024-09-11T23:03:10.497 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.495+0000 7f1d88f60640 1 --2- 172.21.15.152:0/88676517 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1d740a2d30 0x7f1d740a51f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800bbe0 tx=0x7f1d7800b390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:10.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.498+0000 7f1d81ffb640 1 -- 172.21.15.152:0/88676517 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d7c09bb90 con 0x7f1d84199e20 2024-09-11T23:03:10.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.708+0000 7f1d8b1c4640 1 -- 172.21.15.152:0/88676517 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1d8402d980 con 0x7f1d84199e20 2024-09-11T23:03:10.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.709+0000 7f1d81ffb640 1 -- 172.21.15.152:0/88676517 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f1d7c09b490 con 0x7f1d84199e20 2024-09-11T23:03:10.710 INFO:teuthology.orchestra.run.smithi152.stdout:pg_num: 64 2024-09-11T23:03:10.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.711+0000 7f1d8b1c4640 1 -- 172.21.15.152:0/88676517 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1d740a2d30 msgr2=0x7f1d740a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:10.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.711+0000 7f1d8b1c4640 1 --2- 172.21.15.152:0/88676517 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1d740a2d30 0x7f1d740a51f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800bbe0 tx=0x7f1d7800b390 comp rx=0 tx=0).stop 2024-09-11T23:03:10.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.711+0000 7f1d8b1c4640 1 -- 172.21.15.152:0/88676517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84199e20 msgr2=0x7f1d841a4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:10.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.711+0000 7f1d8b1c4640 1 --2- 172.21.15.152:0/88676517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84199e20 0x7f1d841a4420 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00da40 tx=0x7f1d7c00df00 comp rx=0 tx=0).stop 2024-09-11T23:03:10.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.711+0000 7f1d8b1c4640 1 -- 172.21.15.152:0/88676517 shutdown_connections 2024-09-11T23:03:10.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.711+0000 7f1d8b1c4640 1 --2- 172.21.15.152:0/88676517 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f1d740a2d30 0x7f1d740a51f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.711+0000 7f1d8b1c4640 1 --2- 172.21.15.152:0/88676517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d84199e20 0x7f1d841a4420 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.713 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.711+0000 7f1d8b1c4640 1 --2- 172.21.15.152:0/88676517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d84105df0 0x7f1d841998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.711+0000 7f1d8b1c4640 1 --2- 172.21.15.152:0/88676517 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d84101400 0x7f1d841993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.711+0000 7f1d8b1c4640 1 -- 172.21.15.152:0/88676517 >> 172.21.15.152:0/88676517 conn(0x7f1d84077170 msgr2=0x7f1d841040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:10.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.711+0000 7f1d8b1c4640 1 -- 172.21.15.152:0/88676517 shutdown_connections 2024-09-11T23:03:10.714 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.711+0000 7f1d8b1c4640 1 -- 172.21.15.152:0/88676517 wait complete. 2024-09-11T23:03:10.726 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-11T23:03:10.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.827+0000 7f46d944c640 1 -- 172.21.15.152:0/2684187232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46d4106990 msgr2=0x7f46d4106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:10.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.827+0000 7f46d944c640 1 --2- 172.21.15.152:0/2684187232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46d4106990 0x7f46d4106d70 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009930 tx=0x7f46c802a420 comp rx=0 tx=0).stop 2024-09-11T23:03:10.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.827+0000 7f46d944c640 1 -- 172.21.15.152:0/2684187232 shutdown_connections 2024-09-11T23:03:10.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.827+0000 7f46d944c640 1 --2- 172.21.15.152:0/2684187232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46d4101320 0x7f46d410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.827+0000 7f46d944c640 1 --2- 172.21.15.152:0/2684187232 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46d4100980 0x7f46d4100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.827+0000 7f46d944c640 1 --2- 172.21.15.152:0/2684187232 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46d4106990 0x7f46d4106d70 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.827+0000 7f46d944c640 1 -- 172.21.15.152:0/2684187232 >> 172.21.15.152:0/2684187232 conn(0x7f46d4077a10 msgr2=0x7f46d40fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:10.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.827+0000 7f46d944c640 1 -- 172.21.15.152:0/2684187232 shutdown_connections 2024-09-11T23:03:10.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.827+0000 7f46d944c640 1 -- 172.21.15.152:0/2684187232 wait complete. 2024-09-11T23:03:10.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.828+0000 7f46d944c640 1 Processor -- start 2024-09-11T23:03:10.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.828+0000 7f46d944c640 1 -- start start 2024-09-11T23:03:10.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.828+0000 7f46d944c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46d4100980 0x7f46d4196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:10.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.828+0000 7f46d944c640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46d4101320 0x7f46d41967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:10.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.828+0000 7f46d944c640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46d4106990 0x7f46d419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:10.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.828+0000 7f46d944c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f46d410fcd0 con 0x7f46d4100980 2024-09-11T23:03:10.837 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.828+0000 7f46d944c640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f46d410fb50 con 0x7f46d4106990 2024-09-11T23:03:10.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.828+0000 7f46d944c640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f46d410fe50 con 0x7f46d4101320 2024-09-11T23:03:10.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.828+0000 7f46d2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46d4100980 0x7f46d4196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:10.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.828+0000 7f46d2ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46d4100980 0x7f46d4196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:60262/0 (socket says 172.21.15.152:60262) 2024-09-11T23:03:10.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.828+0000 7f46d2ffd640 1 -- 172.21.15.152:0/340565991 learned_addr learned my addr 172.21.15.152:0/340565991 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:10.838 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.829+0000 7f46d2ffd640 1 -- 172.21.15.152:0/340565991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46d4101320 msgr2=0x7f46d41967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:10.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.829+0000 7f46d37fe640 1 --2- 172.21.15.152:0/340565991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46d4106990 0x7f46d419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:10.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.829+0000 7f46d27fc640 1 --2- 172.21.15.152:0/340565991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46d4101320 0x7f46d41967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:10.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.829+0000 7f46d2ffd640 1 --2- 172.21.15.152:0/340565991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46d4101320 0x7f46d41967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.829+0000 7f46d2ffd640 1 -- 172.21.15.152:0/340565991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46d4106990 msgr2=0x7f46d419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:10.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.829+0000 7f46d2ffd640 1 --2- 172.21.15.152:0/340565991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46d4106990 0x7f46d419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:10.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.829+0000 7f46d2ffd640 1 -- 172.21.15.152:0/340565991 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d419b270 con 0x7f46d4100980 2024-09-11T23:03:10.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.829+0000 7f46d37fe640 1 --2- 172.21.15.152:0/340565991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46d4106990 0x7f46d419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:03:10.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.829+0000 7f46d2ffd640 1 --2- 172.21.15.152:0/340565991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46d4100980 0x7f46d4196280 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009230 tx=0x7f46c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:10.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.829+0000 7f46bbfff640 1 -- 172.21.15.152:0/340565991 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c803f030 con 0x7f46d4100980 2024-09-11T23:03:10.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.829+0000 7f46bbfff640 1 -- 172.21.15.152:0/340565991 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c8003ad0 con 0x7f46d4100980 2024-09-11T23:03:10.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.829+0000 7f46d944c640 1 -- 172.21.15.152:0/340565991 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d419b500 con 0x7f46d4100980 2024-09-11T23:03:10.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.829+0000 7f46bbfff640 1 -- 172.21.15.152:0/340565991 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8035070 con 0x7f46d4100980 2024-09-11T23:03:10.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.829+0000 7f46d944c640 1 -- 172.21.15.152:0/340565991 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d41a2df0 con 0x7f46d4100980 2024-09-11T23:03:10.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.830+0000 7f46d944c640 1 -- 172.21.15.152:0/340565991 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d4100de0 con 0x7f46d4100980 2024-09-11T23:03:10.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.832+0000 7f46bbfff640 1 -- 172.21.15.152:0/340565991 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f46c8042030 con 0x7f46d4100980 2024-09-11T23:03:10.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.832+0000 7f46bbfff640 1 --2- 172.21.15.152:0/340565991 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f46bc0a2d00 0x7f46bc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:10.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.832+0000 7f46d27fc640 1 --2- 172.21.15.152:0/340565991 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f46bc0a2d00 0x7f46bc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:10.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.832+0000 7f46bbfff640 1 -- 172.21.15.152:0/340565991 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8536+0+0 (secure 0 0 0) 0x7f46c80eff70 con 0x7f46d4100980 2024-09-11T23:03:10.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.833+0000 7f46d27fc640 1 --2- 172.21.15.152:0/340565991 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f46bc0a2d00 0x7f46bc0a51c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f46d4100040 tx=0x7f46c4005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:10.842 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:10.839+0000 7f46bbfff640 1 -- 172.21.15.152:0/340565991 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46c80bcb80 con 0x7f46d4100980 2024-09-11T23:03:11.066 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:11.065+0000 7f46d944c640 1 -- 172.21.15.152:0/340565991 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f46d4108ef0 con 0x7f46d4100980 2024-09-11T23:03:11.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:11.072+0000 7f46bbfff640 1 -- 172.21.15.152:0/340565991 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740912 (secure 0 0 0) 0x7f46c80bc480 con 0x7f46d4100980 2024-09-11T23:03:11.083 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T23:03:11.083 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:03:11.083 INFO:teuthology.orchestra.run.smithi152.stdout: "epoch": 25, 2024-09-11T23:03:11.083 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.083 INFO:teuthology.orchestra.run.smithi152.stdout: "active_gid": 15240, 2024-09-11T23:03:11.083 INFO:teuthology.orchestra.run.smithi152.stdout: "active_name": "x", 2024-09-11T23:03:11.083 INFO:teuthology.orchestra.run.smithi152.stdout: "active_addrs": { 2024-09-11T23:03:11.083 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-09-11T23:03:11.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.084 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-09-11T23:03:11.084 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:6800", 2024-09-11T23:03:11.084 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 4130474708 2024-09-11T23:03:11.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.084 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v1", 2024-09-11T23:03:11.084 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:6801", 2024-09-11T23:03:11.084 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 4130474708 2024-09-11T23:03:11.084 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.084 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-11T23:03:11.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.084 INFO:teuthology.orchestra.run.smithi152.stdout: "active_addr": "172.21.15.152:6801/4130474708", 2024-09-11T23:03:11.084 INFO:teuthology.orchestra.run.smithi152.stdout: "active_change": "2024-09-11T22:52:53.067856+0000", 2024-09-11T23:03:11.085 INFO:teuthology.orchestra.run.smithi152.stdout: "active_mgr_features": 4540701547738038271, 2024-09-11T23:03:11.085 INFO:teuthology.orchestra.run.smithi152.stdout: "available": true, 2024-09-11T23:03:11.085 INFO:teuthology.orchestra.run.smithi152.stdout: "standbys": [ 2024-09-11T23:03:11.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.085 INFO:teuthology.orchestra.run.smithi152.stdout: "gid": 15237, 2024-09-11T23:03:11.085 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "z", 2024-09-11T23:03:11.085 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr_features": 4540701547738038271, 2024-09-11T23:03:11.085 INFO:teuthology.orchestra.run.smithi152.stdout: "available_modules": [ 2024-09-11T23:03:11.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.085 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "alerts", 2024-09-11T23:03:11.085 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.085 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.085 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.086 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-09-11T23:03:11.086 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-09-11T23:03:11.086 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.086 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.086 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.086 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-09-11T23:03:11.086 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.086 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.086 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.086 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T23:03:11.086 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.086 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.086 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.087 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.087 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.087 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.087 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.087 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.087 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.087 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.087 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.087 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.087 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.087 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.087 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.087 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.088 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.088 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.088 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.088 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.088 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.088 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.088 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.088 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.088 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.088 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.088 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.088 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.088 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.089 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.089 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.089 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.089 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.089 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.089 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.089 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.089 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.089 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.089 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.089 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.089 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.089 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.090 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.090 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.090 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.090 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.090 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.090 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.090 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.090 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.090 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.090 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.090 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.090 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.090 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.090 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.091 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.091 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.091 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.091 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.091 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.091 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.091 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.091 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.091 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.091 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.091 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.091 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.091 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.092 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.092 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_destination": { 2024-09-11T23:03:11.092 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_destination", 2024-09-11T23:03:11.092 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.092 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.092 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.092 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.092 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.092 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.092 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.092 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Email address to send alerts to", 2024-09-11T23:03:11.092 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_from_name": { 2024-09-11T23:03:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_from_name", 2024-09-11T23:03:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "Ceph", 2024-09-11T23:03:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Email From: name", 2024-09-11T23:03:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_host": { 2024-09-11T23:03:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_host", 2024-09-11T23:03:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SMTP server", 2024-09-11T23:03:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_password": { 2024-09-11T23:03:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_password", 2024-09-11T23:03:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Password to authenticate with", 2024-09-11T23:03:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_port": { 2024-09-11T23:03:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_port", 2024-09-11T23:03:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "465", 2024-09-11T23:03:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SMTP port", 2024-09-11T23:03:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_sender": { 2024-09-11T23:03:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_sender", 2024-09-11T23:03:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SMTP envelope sender", 2024-09-11T23:03:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_ssl": { 2024-09-11T23:03:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_ssl", 2024-09-11T23:03:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T23:03:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_user": { 2024-09-11T23:03:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_user", 2024-09-11T23:03:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "User to authenticate as", 2024-09-11T23:03:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "balancer", 2024-09-11T23:03:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "active": { 2024-09-11T23:03:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "active", 2024-09-11T23:03:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T23:03:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "begin_time": { 2024-09-11T23:03:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "begin_time", 2024-09-11T23:03:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0000", 2024-09-11T23:03:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T23:03:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:03:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "begin_weekday": { 2024-09-11T23:03:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "begin_weekday", 2024-09-11T23:03:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-09-11T23:03:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-09-11T23:03:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "0", 2024-09-11T23:03:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "6", 2024-09-11T23:03:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T23:03:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:03:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "crush_compat_max_iterations": { 2024-09-11T23:03:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crush_compat_max_iterations", 2024-09-11T23:03:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-09-11T23:03:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "25", 2024-09-11T23:03:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-09-11T23:03:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "250", 2024-09-11T23:03:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T23:03:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "crush_compat_metrics": { 2024-09-11T23:03:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crush_compat_metrics", 2024-09-11T23:03:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T23:03:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T23:03:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T23:03:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "crush_compat_step": { 2024-09-11T23:03:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crush_compat_step", 2024-09-11T23:03:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0.5", 2024-09-11T23:03:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "0.001", 2024-09-11T23:03:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "0.999", 2024-09-11T23:03:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "aggressiveness of optimization", 2024-09-11T23:03:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T23:03:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "end_time": { 2024-09-11T23:03:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "end_time", 2024-09-11T23:03:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "2359", 2024-09-11T23:03:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T23:03:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:03:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "end_weekday": { 2024-09-11T23:03:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "end_weekday", 2024-09-11T23:03:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-09-11T23:03:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-09-11T23:03:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "0", 2024-09-11T23:03:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "6", 2024-09-11T23:03:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T23:03:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:03:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "min_score": { 2024-09-11T23:03:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "min_score", 2024-09-11T23:03:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-09-11T23:03:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T23:03:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": { 2024-09-11T23:03:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mode", 2024-09-11T23:03:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "upmap", 2024-09-11T23:03:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "crush-compat", 2024-09-11T23:03:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "none", 2024-09-11T23:03:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "read", 2024-09-11T23:03:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap", 2024-09-11T23:03:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap-read" 2024-09-11T23:03:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Balancer mode", 2024-09-11T23:03:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "pool_ids": { 2024-09-11T23:03:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pool_ids", 2024-09-11T23:03:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T23:03:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-09-11T23:03:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-09-11T23:03:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-09-11T23:03:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T23:03:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap_max_deviation": { 2024-09-11T23:03:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "upmap_max_deviation", 2024-09-11T23:03:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-09-11T23:03:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-09-11T23:03:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T23:03:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T23:03:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap_max_optimizations": { 2024-09-11T23:03:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "upmap_max_optimizations", 2024-09-11T23:03:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-09-11T23:03:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T23:03:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephadm", 2024-09-11T23:03:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "agent_down_multiplier": { 2024-09-11T23:03:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "agent_down_multiplier", 2024-09-11T23:03:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "3.0", 2024-09-11T23:03:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T23:03:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "agent_refresh_rate": { 2024-09-11T23:03:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "agent_refresh_rate", 2024-09-11T23:03:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "20", 2024-09-11T23:03:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T23:03:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "agent_starting_port": { 2024-09-11T23:03:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "agent_starting_port", 2024-09-11T23:03:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "4721", 2024-09-11T23:03:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T23:03:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "allow_ptrace": { 2024-09-11T23:03:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "allow_ptrace", 2024-09-11T23:03:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T23:03:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T23:03:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.127 INFO:teuthology.orchestra.run.smithi152.stdout: "autotune_interval": { 2024-09-11T23:03:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "autotune_interval", 2024-09-11T23:03:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-09-11T23:03:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T23:03:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "autotune_memory_target_ratio": { 2024-09-11T23:03:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T23:03:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0.7", 2024-09-11T23:03:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T23:03:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "cephadm_log_destination": { 2024-09-11T23:03:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephadm_log_destination", 2024-09-11T23:03:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "file", 2024-09-11T23:03:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "file,syslog", 2024-09-11T23:03:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "syslog" 2024-09-11T23:03:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T23:03:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "cgroups_split": { 2024-09-11T23:03:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cgroups_split", 2024-09-11T23:03:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T23:03:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "config_checks_enabled": { 2024-09-11T23:03:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "config_checks_enabled", 2024-09-11T23:03:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T23:03:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "config_dashboard": { 2024-09-11T23:03:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "config_dashboard", 2024-09-11T23:03:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T23:03:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_alertmanager": { 2024-09-11T23:03:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_alertmanager", 2024-09-11T23:03:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T23:03:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-09-11T23:03:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_base": { 2024-09-11T23:03:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_base", 2024-09-11T23:03:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T23:03:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Container image name, without the tag", 2024-09-11T23:03:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_elasticsearch": { 2024-09-11T23:03:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_elasticsearch", 2024-09-11T23:03:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T23:03:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "elasticsearch container image", 2024-09-11T23:03:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_grafana": { 2024-09-11T23:03:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_grafana", 2024-09-11T23:03:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T23:03:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-09-11T23:03:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_haproxy": { 2024-09-11T23:03:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_haproxy", 2024-09-11T23:03:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T23:03:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "HAproxy container image", 2024-09-11T23:03:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_jaeger_agent": { 2024-09-11T23:03:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_jaeger_agent", 2024-09-11T23:03:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T23:03:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Jaeger agent container image", 2024-09-11T23:03:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_jaeger_collector": { 2024-09-11T23:03:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_jaeger_collector", 2024-09-11T23:03:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T23:03:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Jaeger collector container image", 2024-09-11T23:03:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_jaeger_query": { 2024-09-11T23:03:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_jaeger_query", 2024-09-11T23:03:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Jaeger query container image", 2024-09-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_keepalived": { 2024-09-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_keepalived", 2024-09-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T23:03:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Keepalived container image", 2024-09-11T23:03:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_loki": { 2024-09-11T23:03:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_loki", 2024-09-11T23:03:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T23:03:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Loki container image", 2024-09-11T23:03:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_node_exporter": { 2024-09-11T23:03:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_node_exporter", 2024-09-11T23:03:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T23:03:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-09-11T23:03:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_nvmeof": { 2024-09-11T23:03:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_nvmeof", 2024-09-11T23:03:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T23:03:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Nvme-of container image", 2024-09-11T23:03:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_prometheus": { 2024-09-11T23:03:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_prometheus", 2024-09-11T23:03:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T23:03:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-09-11T23:03:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_promtail": { 2024-09-11T23:03:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_promtail", 2024-09-11T23:03:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T23:03:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Promtail container image", 2024-09-11T23:03:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_samba": { 2024-09-11T23:03:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_samba", 2024-09-11T23:03:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T23:03:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Samba/SMB container image", 2024-09-11T23:03:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_snmp_gateway": { 2024-09-11T23:03:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_snmp_gateway", 2024-09-11T23:03:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T23:03:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SNMP Gateway container image", 2024-09-11T23:03:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "container_init": { 2024-09-11T23:03:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_init", 2024-09-11T23:03:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T23:03:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "daemon_cache_timeout": { 2024-09-11T23:03:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "daemon_cache_timeout", 2024-09-11T23:03:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-09-11T23:03:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T23:03:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "default_cephadm_command_timeout": { 2024-09-11T23:03:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T23:03:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "900", 2024-09-11T23:03:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T23:03:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "default_registry": { 2024-09-11T23:03:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "default_registry", 2024-09-11T23:03:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io", 2024-09-11T23:03:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T23:03:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "device_cache_timeout": { 2024-09-11T23:03:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "device_cache_timeout", 2024-09-11T23:03:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "1800", 2024-09-11T23:03:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "seconds to cache device inventory", 2024-09-11T23:03:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "device_enhanced_scan": { 2024-09-11T23:03:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "device_enhanced_scan", 2024-09-11T23:03:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T23:03:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "facts_cache_timeout": { 2024-09-11T23:03:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "facts_cache_timeout", 2024-09-11T23:03:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-09-11T23:03:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "seconds to cache host facts data", 2024-09-11T23:03:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "grafana_dashboards_path": { 2024-09-11T23:03:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "grafana_dashboards_path", 2024-09-11T23:03:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T23:03:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T23:03:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "host_check_interval": { 2024-09-11T23:03:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "host_check_interval", 2024-09-11T23:03:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-09-11T23:03:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to perform a host check", 2024-09-11T23:03:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "hw_monitoring": { 2024-09-11T23:03:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "hw_monitoring", 2024-09-11T23:03:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T23:03:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "inventory_list_all": { 2024-09-11T23:03:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "inventory_list_all", 2024-09-11T23:03:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T23:03:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "log_refresh_metadata": { 2024-09-11T23:03:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_refresh_metadata", 2024-09-11T23:03:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T23:03:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T23:03:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T23:03:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T23:03:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T23:03:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T23:03:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T23:03:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "*", 2024-09-11T23:03:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T23:03:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "max_count_per_host": { 2024-09-11T23:03:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_count_per_host", 2024-09-11T23:03:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "max number of daemons per service per host", 2024-09-11T23:03:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "max_osd_draining_count": { 2024-09-11T23:03:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_osd_draining_count", 2024-09-11T23:03:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T23:03:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "migration_current": { 2024-09-11T23:03:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "migration_current", 2024-09-11T23:03:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "internal - do not modify", 2024-09-11T23:03:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": { 2024-09-11T23:03:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mode", 2024-09-11T23:03:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "root", 2024-09-11T23:03:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "cephadm-package", 2024-09-11T23:03:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "root" 2024-09-11T23:03:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T23:03:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "oob_default_addr": { 2024-09-11T23:03:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "oob_default_addr", 2024-09-11T23:03:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "169.254.1.1", 2024-09-11T23:03:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T23:03:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "prometheus_alerts_path": { 2024-09-11T23:03:11.177 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "prometheus_alerts_path", 2024-09-11T23:03:11.177 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.177 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.177 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.177 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T23:03:11.177 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.177 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.177 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.177 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T23:03:11.177 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.177 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.177 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.177 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.177 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_insecure": { 2024-09-11T23:03:11.178 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_insecure", 2024-09-11T23:03:11.178 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.178 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.178 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.178 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.178 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.178 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.178 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.178 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T23:03:11.178 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.178 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.178 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.179 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.179 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_password": { 2024-09-11T23:03:11.179 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_password", 2024-09-11T23:03:11.179 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.179 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.179 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.179 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.179 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.179 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.179 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.179 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T23:03:11.179 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.179 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.180 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.180 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.180 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_url": { 2024-09-11T23:03:11.180 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_url", 2024-09-11T23:03:11.180 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.180 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.180 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.180 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.180 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.180 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.180 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.180 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T23:03:11.180 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_username": { 2024-09-11T23:03:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_username", 2024-09-11T23:03:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T23:03:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.182 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.182 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.182 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.182 INFO:teuthology.orchestra.run.smithi152.stdout: "secure_monitoring_stack": { 2024-09-11T23:03:11.182 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "secure_monitoring_stack", 2024-09-11T23:03:11.182 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.182 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.182 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.182 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.182 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.182 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.182 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.182 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T23:03:11.183 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.183 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.183 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.183 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.183 INFO:teuthology.orchestra.run.smithi152.stdout: "service_discovery_port": { 2024-09-11T23:03:11.183 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "service_discovery_port", 2024-09-11T23:03:11.183 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.183 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.183 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.183 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8765", 2024-09-11T23:03:11.183 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.183 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.183 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.183 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "cephadm service discovery port", 2024-09-11T23:03:11.184 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.184 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.184 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.184 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.184 INFO:teuthology.orchestra.run.smithi152.stdout: "ssh_config_file": { 2024-09-11T23:03:11.184 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssh_config_file", 2024-09-11T23:03:11.184 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.184 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.184 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.184 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.184 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.184 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.184 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.184 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T23:03:11.185 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.185 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.185 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.185 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.185 INFO:teuthology.orchestra.run.smithi152.stdout: "use_agent": { 2024-09-11T23:03:11.185 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "use_agent", 2024-09-11T23:03:11.185 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.185 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.185 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.185 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.185 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.185 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.185 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.185 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T23:03:11.186 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.186 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.186 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.186 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.186 INFO:teuthology.orchestra.run.smithi152.stdout: "use_repo_digest": { 2024-09-11T23:03:11.186 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "use_repo_digest", 2024-09-11T23:03:11.186 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.186 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.186 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.186 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.186 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.186 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.186 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.186 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T23:03:11.187 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.187 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.187 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.187 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.187 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_on_failed_host_check": { 2024-09-11T23:03:11.187 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_on_failed_host_check", 2024-09-11T23:03:11.187 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.187 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.187 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.187 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.187 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.187 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.187 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.187 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T23:03:11.188 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.188 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.188 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.188 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.188 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_on_stray_daemons": { 2024-09-11T23:03:11.188 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_on_stray_daemons", 2024-09-11T23:03:11.188 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.188 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.188 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.188 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.188 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.188 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.188 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.189 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T23:03:11.189 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.189 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.189 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.189 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.189 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_on_stray_hosts": { 2024-09-11T23:03:11.189 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_on_stray_hosts", 2024-09-11T23:03:11.189 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.189 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.189 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.189 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.189 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.189 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.189 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.190 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T23:03:11.190 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.190 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.190 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.190 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.190 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.190 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.190 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.190 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crash", 2024-09-11T23:03:11.190 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.190 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.190 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.190 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.191 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.191 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.191 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.191 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.191 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.191 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.191 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.191 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.191 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.191 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.191 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.191 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.191 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.192 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.192 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.192 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.192 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.192 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.192 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.192 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.192 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.192 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.192 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.192 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.192 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.192 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.192 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.193 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.193 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.193 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.193 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.193 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.193 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.193 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.193 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.193 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.193 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.193 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.193 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.193 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.193 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.194 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.194 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.194 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.194 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.194 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.194 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.194 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.194 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.194 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.194 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.194 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.194 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.194 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.195 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.195 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.195 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.195 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.195 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.195 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.195 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.195 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.195 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.195 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.195 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.195 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.195 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.195 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.196 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.196 INFO:teuthology.orchestra.run.smithi152.stdout: "retain_interval": { 2024-09-11T23:03:11.196 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "retain_interval", 2024-09-11T23:03:11.196 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.196 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.196 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.196 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "31536000", 2024-09-11T23:03:11.196 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.196 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.196 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.196 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T23:03:11.196 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.196 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.196 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.197 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.197 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_recent_interval": { 2024-09-11T23:03:11.197 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_recent_interval", 2024-09-11T23:03:11.197 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.197 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.197 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.197 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "1209600", 2024-09-11T23:03:11.197 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.197 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.197 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.197 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T23:03:11.197 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.197 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.198 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.198 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.198 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.198 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.198 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.198 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "dashboard", 2024-09-11T23:03:11.198 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.198 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.198 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.198 INFO:teuthology.orchestra.run.smithi152.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T23:03:11.198 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T23:03:11.198 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.198 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.198 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.199 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:11.199 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.199 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.199 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.199 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.199 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.199 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.199 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.199 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.199 INFO:teuthology.orchestra.run.smithi152.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T23:03:11.199 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T23:03:11.199 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.199 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.199 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.200 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.200 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.200 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.200 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.200 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.200 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.200 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.200 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.200 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.200 INFO:teuthology.orchestra.run.smithi152.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T23:03:11.200 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T23:03:11.200 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.200 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.200 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.201 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.201 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.201 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.201 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.201 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.201 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.201 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.201 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.201 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.201 INFO:teuthology.orchestra.run.smithi152.stdout: "AUDIT_API_ENABLED": { 2024-09-11T23:03:11.201 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T23:03:11.201 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.201 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.201 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.202 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.202 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.202 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.202 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.202 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.202 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.202 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.202 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.202 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.202 INFO:teuthology.orchestra.run.smithi152.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T23:03:11.202 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T23:03:11.202 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.202 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.202 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.203 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.203 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.203 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.203 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.203 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.203 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.203 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.203 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.203 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.203 INFO:teuthology.orchestra.run.smithi152.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T23:03:11.203 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T23:03:11.203 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.203 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.203 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.204 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.204 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.204 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.204 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.204 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.204 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.204 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.204 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.204 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.204 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T23:03:11.204 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T23:03:11.204 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.205 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.205 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.205 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.205 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.205 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.205 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.205 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.205 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.205 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.205 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.205 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.205 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T23:03:11.205 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T23:03:11.206 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.206 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.206 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.206 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.206 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.206 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.206 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.206 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.206 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.206 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.206 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.206 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.206 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T23:03:11.206 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T23:03:11.207 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.207 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.207 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.207 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.207 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.207 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.207 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.207 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.207 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.207 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.207 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.207 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.207 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T23:03:11.207 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T23:03:11.208 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.208 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.208 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.208 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.208 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.208 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.208 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.208 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.208 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.208 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.208 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.208 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.208 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T23:03:11.209 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T23:03:11.209 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.209 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.209 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.209 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.209 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.209 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.209 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.209 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.209 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.209 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.209 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.209 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.209 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T23:03:11.210 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T23:03:11.210 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.210 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.210 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.210 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.210 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.210 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.210 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.210 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.210 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.210 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.210 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.210 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.210 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T23:03:11.211 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T23:03:11.211 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.211 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.211 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.211 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.211 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.211 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.211 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.211 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.211 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.211 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.211 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.211 INFO:teuthology.orchestra.run.smithi152.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T23:03:11.212 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T23:03:11.212 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.212 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.212 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.212 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.212 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.212 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.212 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.212 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.212 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.212 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.212 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.212 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.213 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T23:03:11.213 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T23:03:11.213 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.213 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.213 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.213 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "admin", 2024-09-11T23:03:11.213 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.213 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.213 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.213 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.213 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.213 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.213 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.213 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.214 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T23:03:11.214 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T23:03:11.214 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.214 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.214 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.214 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.214 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.214 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.214 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.214 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.214 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.214 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.214 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.214 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.215 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_URL": { 2024-09-11T23:03:11.215 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_URL", 2024-09-11T23:03:11.215 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.215 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.215 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.215 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.215 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.215 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.215 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.215 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.215 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.215 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.215 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.215 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.216 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T23:03:11.216 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T23:03:11.216 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.216 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.216 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.216 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "admin", 2024-09-11T23:03:11.216 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.216 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.216 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.216 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.216 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.216 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.216 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.216 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T23:03:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T23:03:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.217 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T23:03:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T23:03:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.218 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.219 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.219 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.219 INFO:teuthology.orchestra.run.smithi152.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T23:03:11.219 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T23:03:11.219 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.219 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.219 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.219 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.219 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.219 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.219 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.219 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.219 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.220 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.220 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.220 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.220 INFO:teuthology.orchestra.run.smithi152.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T23:03:11.220 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T23:03:11.220 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.220 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.220 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.220 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.220 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.220 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.220 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.220 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T23:03:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T23:03:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.221 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.222 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.222 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.222 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.222 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.222 INFO:teuthology.orchestra.run.smithi152.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T23:03:11.222 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T23:03:11.222 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.222 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.222 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.222 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.222 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.222 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.222 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.223 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.223 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.223 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.223 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.223 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.223 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T23:03:11.223 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T23:03:11.223 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.223 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.223 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.223 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.223 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.223 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.223 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.224 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.224 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.224 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.224 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.224 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.224 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T23:03:11.224 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T23:03:11.224 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.224 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.224 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.224 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.224 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.224 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.225 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.225 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.225 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.225 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.225 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.225 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.225 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T23:03:11.225 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T23:03:11.225 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.225 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.225 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.225 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.225 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.225 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.226 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.226 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.226 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.226 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.226 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.226 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.226 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T23:03:11.226 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T23:03:11.226 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.226 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.226 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.226 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.226 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.226 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.227 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.227 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.227 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.227 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.227 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.227 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.227 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T23:03:11.227 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T23:03:11.227 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.227 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.227 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.227 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.227 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.227 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.228 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.228 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.228 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.228 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.228 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.228 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.228 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T23:03:11.228 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T23:03:11.228 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.228 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.228 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.228 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.228 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.228 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.229 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.229 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.229 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.229 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.229 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.229 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.229 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T23:03:11.229 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T23:03:11.229 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.229 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.229 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.229 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.229 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.229 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.230 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.230 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.230 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.230 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.230 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.230 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.230 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T23:03:11.230 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T23:03:11.230 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.230 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.230 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.230 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.230 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.231 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.231 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.231 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.231 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.231 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.231 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.231 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.231 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T23:03:11.231 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T23:03:11.231 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.231 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.231 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.232 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T23:03:11.232 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.232 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.232 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.232 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.232 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.232 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.232 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.232 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.232 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T23:03:11.232 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T23:03:11.232 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.232 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.233 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.233 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:11.233 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.233 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.233 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.233 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.233 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.233 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.233 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.233 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.233 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T23:03:11.233 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T23:03:11.233 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.234 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.234 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.234 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8", 2024-09-11T23:03:11.234 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.234 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.234 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.234 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.234 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.234 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.234 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.234 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.234 INFO:teuthology.orchestra.run.smithi152.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T23:03:11.234 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T23:03:11.234 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.235 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.235 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.235 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "45", 2024-09-11T23:03:11.235 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.235 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.235 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.235 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.235 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.235 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.235 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.235 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.235 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T23:03:11.235 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T23:03:11.235 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.236 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.236 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.236 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.236 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.236 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.236 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.236 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.236 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.236 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.236 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.236 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.236 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T23:03:11.236 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T23:03:11.236 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.237 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.237 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.237 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "admin", 2024-09-11T23:03:11.237 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.237 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.237 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.237 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.237 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.237 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.237 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.237 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.237 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T23:03:11.237 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T23:03:11.237 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T23:03:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T23:03:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T23:03:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T23:03:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-09-11T23:03:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T23:03:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T23:03:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T23:03:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T23:03:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-09-11T23:03:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: "cross_origin_url": { 2024-09-11T23:03:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cross_origin_url", 2024-09-11T23:03:11.243 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.243 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.243 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.243 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.243 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.243 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.243 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.243 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.243 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.243 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.243 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.243 INFO:teuthology.orchestra.run.smithi152.stdout: "crt_file": { 2024-09-11T23:03:11.243 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crt_file", 2024-09-11T23:03:11.244 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.244 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.244 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.244 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.244 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.244 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.244 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.244 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.244 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.244 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.244 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.244 INFO:teuthology.orchestra.run.smithi152.stdout: "debug": { 2024-09-11T23:03:11.244 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "debug", 2024-09-11T23:03:11.245 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.245 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.245 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.245 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.245 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.245 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.245 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.245 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Enable/disable debug options", 2024-09-11T23:03:11.245 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.245 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.245 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.246 INFO:teuthology.orchestra.run.smithi152.stdout: "jwt_token_ttl": { 2024-09-11T23:03:11.246 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "jwt_token_ttl", 2024-09-11T23:03:11.246 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.246 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.246 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.246 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "28800", 2024-09-11T23:03:11.246 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.246 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.246 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.246 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.246 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.246 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.246 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.247 INFO:teuthology.orchestra.run.smithi152.stdout: "key_file": { 2024-09-11T23:03:11.247 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "key_file", 2024-09-11T23:03:11.247 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.247 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.247 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.247 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.247 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.247 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.247 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.247 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.247 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.247 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.247 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.248 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.248 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.248 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.248 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.248 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.248 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.248 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.248 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.248 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.248 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.248 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.248 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.248 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.249 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.249 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.249 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.249 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.249 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.249 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.249 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.249 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.249 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.249 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.249 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.249 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.249 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.250 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.250 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.250 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.250 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.250 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.250 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.250 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.250 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.250 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.250 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.250 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.250 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.250 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.251 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.251 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.251 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.251 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.251 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.251 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.251 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.251 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.251 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.251 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.251 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.251 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.251 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.251 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.252 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.252 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.252 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.252 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.252 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.252 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.252 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.252 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.252 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.252 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.252 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.252 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.252 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.252 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.253 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.253 INFO:teuthology.orchestra.run.smithi152.stdout: "motd": { 2024-09-11T23:03:11.253 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "motd", 2024-09-11T23:03:11.253 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.253 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.253 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.253 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.253 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.253 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.253 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.253 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "The message of the day", 2024-09-11T23:03:11.253 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.253 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.253 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.254 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.254 INFO:teuthology.orchestra.run.smithi152.stdout: "redirect_resolve_ip_addr": { 2024-09-11T23:03:11.254 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T23:03:11.254 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.254 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.254 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.254 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.254 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.254 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.254 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.254 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.254 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.254 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.254 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.255 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.255 INFO:teuthology.orchestra.run.smithi152.stdout: "server_addr": { 2024-09-11T23:03:11.255 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_addr", 2024-09-11T23:03:11.255 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.255 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.255 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.255 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "::", 2024-09-11T23:03:11.255 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.255 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.255 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.255 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.255 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.255 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.255 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.256 INFO:teuthology.orchestra.run.smithi152.stdout: "server_port": { 2024-09-11T23:03:11.256 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_port", 2024-09-11T23:03:11.256 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.256 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.256 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.256 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8080", 2024-09-11T23:03:11.256 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.256 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.256 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.256 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.256 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.256 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.256 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.257 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.257 INFO:teuthology.orchestra.run.smithi152.stdout: "ssl": { 2024-09-11T23:03:11.257 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssl", 2024-09-11T23:03:11.257 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.257 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.257 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.257 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.257 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.257 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.257 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.257 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.257 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.257 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.257 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.258 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.258 INFO:teuthology.orchestra.run.smithi152.stdout: "ssl_server_port": { 2024-09-11T23:03:11.258 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssl_server_port", 2024-09-11T23:03:11.258 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.258 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.258 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.258 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8443", 2024-09-11T23:03:11.258 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.258 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.258 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.258 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.258 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.259 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.259 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.259 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.259 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_behaviour": { 2024-09-11T23:03:11.259 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_behaviour", 2024-09-11T23:03:11.259 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.259 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.259 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.259 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "redirect", 2024-09-11T23:03:11.259 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.260 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.260 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.260 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.260 INFO:teuthology.orchestra.run.smithi152.stdout: "redirect" 2024-09-11T23:03:11.260 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.260 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.260 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.260 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.260 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.260 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_error_status_code": { 2024-09-11T23:03:11.260 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_error_status_code", 2024-09-11T23:03:11.261 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.261 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.261 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.261 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "500", 2024-09-11T23:03:11.261 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "400", 2024-09-11T23:03:11.261 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "599", 2024-09-11T23:03:11.261 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.261 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.261 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.261 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.261 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.261 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.261 INFO:teuthology.orchestra.run.smithi152.stdout: "url_prefix": { 2024-09-11T23:03:11.261 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "url_prefix", 2024-09-11T23:03:11.262 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.262 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.262 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.262 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.262 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.262 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.262 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.262 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.262 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.262 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.262 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.262 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.262 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.263 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.263 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.263 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "devicehealth", 2024-09-11T23:03:11.263 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.263 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.263 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.263 INFO:teuthology.orchestra.run.smithi152.stdout: "enable_monitoring": { 2024-09-11T23:03:11.263 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enable_monitoring", 2024-09-11T23:03:11.263 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.263 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.264 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.264 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.264 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.264 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.264 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.264 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "monitor device health metrics", 2024-09-11T23:03:11.264 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.264 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.264 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.264 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.264 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.264 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.264 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.265 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.265 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.265 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.265 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.265 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.265 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.265 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.265 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.265 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.265 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.265 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.265 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.265 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.265 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.266 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.266 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.266 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.266 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.266 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.266 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.266 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.266 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.266 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.266 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.266 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.266 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.266 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.267 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.267 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.267 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.267 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.267 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.267 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.267 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.267 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.267 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.267 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.267 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.267 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.267 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.267 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.268 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.268 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.268 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.268 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.268 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.268 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.268 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.268 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.268 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.268 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.268 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.268 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.269 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.269 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.269 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.269 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.269 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.269 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.269 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.269 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.269 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.269 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.269 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.269 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.270 INFO:teuthology.orchestra.run.smithi152.stdout: "mark_out_threshold": { 2024-09-11T23:03:11.270 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mark_out_threshold", 2024-09-11T23:03:11.270 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.270 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.270 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.270 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "2419200", 2024-09-11T23:03:11.270 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.270 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.270 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.270 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T23:03:11.270 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.270 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.270 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.270 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.271 INFO:teuthology.orchestra.run.smithi152.stdout: "pool_name": { 2024-09-11T23:03:11.271 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pool_name", 2024-09-11T23:03:11.271 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.271 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.271 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.271 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "device_health_metrics", 2024-09-11T23:03:11.271 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.271 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.271 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.271 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T23:03:11.271 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.271 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.271 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.272 INFO:teuthology.orchestra.run.smithi152.stdout: "retention_period": { 2024-09-11T23:03:11.272 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "retention_period", 2024-09-11T23:03:11.272 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.272 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.272 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.272 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "15552000", 2024-09-11T23:03:11.272 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.272 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.272 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.272 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how long to retain device health metrics", 2024-09-11T23:03:11.272 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.272 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.272 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.273 INFO:teuthology.orchestra.run.smithi152.stdout: "scrape_frequency": { 2024-09-11T23:03:11.273 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "scrape_frequency", 2024-09-11T23:03:11.273 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.273 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.273 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.273 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "86400", 2024-09-11T23:03:11.273 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.273 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.273 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.273 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T23:03:11.273 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.274 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.274 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.274 INFO:teuthology.orchestra.run.smithi152.stdout: "self_heal": { 2024-09-11T23:03:11.274 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "self_heal", 2024-09-11T23:03:11.274 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.274 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.274 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.274 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.274 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.274 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.274 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.274 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T23:03:11.274 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.275 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.275 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.275 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-09-11T23:03:11.275 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-09-11T23:03:11.275 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.275 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.275 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.275 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-09-11T23:03:11.275 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.275 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.275 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.275 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T23:03:11.275 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.276 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.276 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.276 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_threshold": { 2024-09-11T23:03:11.276 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_threshold", 2024-09-11T23:03:11.276 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.276 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.276 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.276 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "7257600", 2024-09-11T23:03:11.276 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.276 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.276 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.277 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T23:03:11.277 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.277 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.277 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.277 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.277 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.277 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.277 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.277 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "diskprediction_local", 2024-09-11T23:03:11.277 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.277 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.277 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.277 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.278 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.278 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.278 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.278 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.278 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.278 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.278 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.278 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.278 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.278 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.278 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.278 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.278 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.278 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.279 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.279 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.279 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.279 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.279 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.279 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.279 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.279 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.279 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.279 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.279 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.279 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.279 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.280 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.280 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.280 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.280 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.280 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.280 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.280 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.280 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.280 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.280 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.280 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.280 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.281 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.281 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.281 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.281 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.281 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.281 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.281 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.281 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.281 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.281 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.281 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.281 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.281 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.282 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.282 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.282 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.282 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.282 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.282 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.282 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.282 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.282 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.282 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.282 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.282 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.282 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.283 INFO:teuthology.orchestra.run.smithi152.stdout: "predict_interval": { 2024-09-11T23:03:11.283 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "predict_interval", 2024-09-11T23:03:11.283 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.283 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.283 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.283 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "86400", 2024-09-11T23:03:11.283 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.283 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.283 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.283 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.283 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.283 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.283 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.284 INFO:teuthology.orchestra.run.smithi152.stdout: "predictor_model": { 2024-09-11T23:03:11.284 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "predictor_model", 2024-09-11T23:03:11.284 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.284 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.284 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.284 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "prophetstor", 2024-09-11T23:03:11.284 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.284 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.284 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.284 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.284 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.284 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.284 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.285 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-09-11T23:03:11.285 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-09-11T23:03:11.285 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.285 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.285 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.285 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-09-11T23:03:11.285 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.285 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.285 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.285 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.285 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.285 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.285 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.286 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.286 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.286 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "influx", 2024-09-11T23:03:11.286 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": false, 2024-09-11T23:03:11.286 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "influxdb python module not found", 2024-09-11T23:03:11.286 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.286 INFO:teuthology.orchestra.run.smithi152.stdout: "batch_size": { 2024-09-11T23:03:11.286 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "batch_size", 2024-09-11T23:03:11.286 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.286 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.286 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.286 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5000", 2024-09-11T23:03:11.287 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.287 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.287 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.287 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T23:03:11.287 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.287 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.287 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.287 INFO:teuthology.orchestra.run.smithi152.stdout: "database": { 2024-09-11T23:03:11.287 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "database", 2024-09-11T23:03:11.287 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.287 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.287 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.288 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "ceph", 2024-09-11T23:03:11.288 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.288 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.288 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.288 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T23:03:11.288 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.288 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.288 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.288 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": { 2024-09-11T23:03:11.288 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "hostname", 2024-09-11T23:03:11.288 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.288 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.289 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.289 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.289 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.289 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.289 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.289 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "InfluxDB server hostname", 2024-09-11T23:03:11.289 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.289 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.289 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.289 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-09-11T23:03:11.289 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-09-11T23:03:11.289 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.289 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.290 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.290 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "30", 2024-09-11T23:03:11.290 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "5", 2024-09-11T23:03:11.290 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.290 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.290 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T23:03:11.290 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.290 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.290 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.290 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.290 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.290 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.290 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.291 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.291 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.291 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.291 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.291 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.291 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.291 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.291 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.291 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.291 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.291 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.291 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.291 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.291 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.292 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.292 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.292 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.292 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.292 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.292 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.292 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.292 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.292 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.292 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.292 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.292 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.293 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.293 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.293 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.293 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.293 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.293 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.293 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.293 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.293 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.293 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.293 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.293 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.293 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.294 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.294 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.294 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.294 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.294 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.294 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.294 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.294 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.294 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.294 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.294 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.294 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.294 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.294 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.295 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.295 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.295 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.295 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.295 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.295 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.295 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.295 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.295 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.295 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.295 INFO:teuthology.orchestra.run.smithi152.stdout: "password": { 2024-09-11T23:03:11.295 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "password", 2024-09-11T23:03:11.295 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.296 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.296 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.296 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.296 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.296 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.296 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.296 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "password of InfluxDB server user", 2024-09-11T23:03:11.296 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.296 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.296 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.296 INFO:teuthology.orchestra.run.smithi152.stdout: "port": { 2024-09-11T23:03:11.296 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "port", 2024-09-11T23:03:11.297 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.297 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.297 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.297 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8086", 2024-09-11T23:03:11.297 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.297 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.297 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.297 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "InfluxDB server port", 2024-09-11T23:03:11.297 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.297 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.297 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.297 INFO:teuthology.orchestra.run.smithi152.stdout: "ssl": { 2024-09-11T23:03:11.297 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssl", 2024-09-11T23:03:11.298 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.298 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.298 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.298 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "false", 2024-09-11T23:03:11.298 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.298 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.298 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.298 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:03:11.298 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.298 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.298 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.298 INFO:teuthology.orchestra.run.smithi152.stdout: "threads": { 2024-09-11T23:03:11.299 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "threads", 2024-09-11T23:03:11.299 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.299 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.299 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.299 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-09-11T23:03:11.299 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-09-11T23:03:11.299 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "32", 2024-09-11T23:03:11.299 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.299 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T23:03:11.299 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.299 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.299 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.299 INFO:teuthology.orchestra.run.smithi152.stdout: "username": { 2024-09-11T23:03:11.300 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "username", 2024-09-11T23:03:11.300 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.300 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.300 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.300 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.300 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.300 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.300 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.300 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "username of InfluxDB server user", 2024-09-11T23:03:11.300 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.300 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.300 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.301 INFO:teuthology.orchestra.run.smithi152.stdout: "verify_ssl": { 2024-09-11T23:03:11.301 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "verify_ssl", 2024-09-11T23:03:11.301 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.301 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.301 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.301 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "true", 2024-09-11T23:03:11.301 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.301 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.301 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.301 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:03:11.301 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.301 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.302 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.302 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.302 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.302 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "insights", 2024-09-11T23:03:11.302 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.302 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.302 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.302 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.302 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.302 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.302 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.302 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.303 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.303 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.303 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.303 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.303 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.303 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.303 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.303 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.303 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.303 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.303 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.303 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.303 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.303 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.304 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.304 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.304 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.304 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.304 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.304 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.304 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.304 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.304 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.304 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.304 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.304 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.305 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.305 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.305 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.305 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.305 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.305 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.305 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.305 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.305 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.305 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.305 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.305 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.305 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.306 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.306 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.306 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.306 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.306 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.306 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.306 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.306 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.306 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.306 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.306 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.306 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.306 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.307 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.307 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.307 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.307 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.307 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.307 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.307 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.307 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.307 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.307 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.307 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.307 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "iostat", 2024-09-11T23:03:11.308 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.308 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.308 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.308 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.308 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.308 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.308 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.308 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.308 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.308 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.308 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.308 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.308 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.309 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.309 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.309 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.309 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.309 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.309 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.309 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.309 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.309 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.309 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.309 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.309 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.310 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.310 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.310 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.310 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.310 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.310 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.310 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.310 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.310 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.310 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.310 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.310 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.310 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.311 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.311 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.311 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.311 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.311 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.311 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.311 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.311 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.311 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.311 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.311 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.311 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.311 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.311 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.312 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.312 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.312 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.312 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.312 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.312 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.312 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.312 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.312 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.312 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.312 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.312 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.313 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.313 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.313 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.313 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.313 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.313 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.313 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.313 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "k8sevents", 2024-09-11T23:03:11.313 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.313 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.313 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.313 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_event_retention_days": { 2024-09-11T23:03:11.314 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ceph_event_retention_days", 2024-09-11T23:03:11.314 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.314 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.314 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.314 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "7", 2024-09-11T23:03:11.314 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.314 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.314 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.314 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T23:03:11.314 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.314 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.314 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.314 INFO:teuthology.orchestra.run.smithi152.stdout: "config_check_secs": { 2024-09-11T23:03:11.315 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "config_check_secs", 2024-09-11T23:03:11.315 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.315 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.315 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.315 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:11.315 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "10", 2024-09-11T23:03:11.315 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.315 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.315 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T23:03:11.315 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.315 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.315 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.316 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.316 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.316 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.316 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.316 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.316 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.316 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.316 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.316 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.316 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.316 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.316 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.316 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.317 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.317 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.317 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.317 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.317 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.317 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.317 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.317 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.317 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.317 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.317 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.317 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.317 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.318 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.319 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.319 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.319 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.319 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.319 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.319 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.319 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.319 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.319 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.319 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.319 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.319 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.319 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.320 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.320 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.320 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.320 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.320 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.320 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.320 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.320 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.320 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.320 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.320 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.320 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.320 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.321 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.321 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.321 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.321 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.321 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "localpool", 2024-09-11T23:03:11.321 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.321 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.321 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.321 INFO:teuthology.orchestra.run.smithi152.stdout: "failure_domain": { 2024-09-11T23:03:11.321 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "failure_domain", 2024-09-11T23:03:11.321 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.321 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.321 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.322 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "host", 2024-09-11T23:03:11.322 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.322 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.322 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.322 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "failure domain for any created local pool", 2024-09-11T23:03:11.322 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T23:03:11.322 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.322 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.322 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.322 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.322 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.322 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.322 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.322 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.323 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.323 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.323 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.323 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.323 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.323 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.323 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.323 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.323 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.323 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.323 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.323 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.323 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.323 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.324 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.324 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.324 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.324 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.324 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.324 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.324 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.324 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.324 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.324 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.324 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.324 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.324 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.324 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.325 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.325 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.325 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.325 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.325 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.325 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.325 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.325 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.325 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.325 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.325 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.325 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.325 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.326 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.326 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.326 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.326 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.326 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.326 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.326 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.326 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.326 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.326 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.326 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.326 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.326 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.327 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.327 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.327 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.327 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.327 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.327 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.327 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.327 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.327 INFO:teuthology.orchestra.run.smithi152.stdout: "min_size": { 2024-09-11T23:03:11.327 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "min_size", 2024-09-11T23:03:11.327 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.327 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.327 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.327 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.328 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.328 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.328 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.328 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "default min_size for any created local pool", 2024-09-11T23:03:11.328 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T23:03:11.328 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.328 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.328 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.328 INFO:teuthology.orchestra.run.smithi152.stdout: "num_rep": { 2024-09-11T23:03:11.328 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "num_rep", 2024-09-11T23:03:11.328 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.328 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.328 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.329 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "3", 2024-09-11T23:03:11.329 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.329 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.329 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.329 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "default replica count for any created local pool", 2024-09-11T23:03:11.329 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.329 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.329 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.329 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.329 INFO:teuthology.orchestra.run.smithi152.stdout: "pg_num": { 2024-09-11T23:03:11.329 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pg_num", 2024-09-11T23:03:11.329 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.330 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.330 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.330 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "128", 2024-09-11T23:03:11.330 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.330 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.330 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.330 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T23:03:11.330 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.330 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.330 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.330 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.330 INFO:teuthology.orchestra.run.smithi152.stdout: "prefix": { 2024-09-11T23:03:11.330 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "prefix", 2024-09-11T23:03:11.330 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.331 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.331 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.331 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.331 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.331 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.331 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.331 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "name prefix for any created local pool", 2024-09-11T23:03:11.331 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.331 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.331 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.331 INFO:teuthology.orchestra.run.smithi152.stdout: "subtree": { 2024-09-11T23:03:11.331 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "subtree", 2024-09-11T23:03:11.331 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.332 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.332 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.332 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "rack", 2024-09-11T23:03:11.332 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.332 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.332 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.332 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T23:03:11.332 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T23:03:11.332 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.332 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.332 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.332 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.332 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.333 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.333 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mds_autoscaler", 2024-09-11T23:03:11.333 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.333 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.333 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.333 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.333 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.333 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.333 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.333 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.333 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.333 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.333 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.333 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.334 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.334 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.334 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.334 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.334 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.334 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.334 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.334 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.334 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.334 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.334 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.334 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.334 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.334 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.335 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.335 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.335 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.335 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.335 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.335 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.335 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.335 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.335 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.335 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.335 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.335 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.335 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.336 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.336 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.336 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.336 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.336 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.336 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.336 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.336 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.336 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.336 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.336 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.336 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.336 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.336 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.337 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.337 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.337 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.337 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.337 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.337 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.337 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.337 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.337 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.337 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.337 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.337 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.337 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.337 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.338 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.338 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.338 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.338 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.338 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.338 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.338 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.338 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.338 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.338 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mirroring", 2024-09-11T23:03:11.338 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.338 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.338 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.338 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.339 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.339 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.339 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.339 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.339 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.339 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.339 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.339 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.339 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.339 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.339 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.339 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.339 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.339 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.340 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.340 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.340 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.340 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.340 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.340 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.340 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.340 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.340 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.340 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.340 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.340 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.340 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.340 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.341 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.341 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.341 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.341 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.341 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.341 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.341 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.341 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.341 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.341 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.341 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.341 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.341 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.341 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.342 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.342 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.342 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.342 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.342 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.342 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.342 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.342 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.342 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.342 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.342 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.342 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.342 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.342 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.343 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.343 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.343 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.343 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.343 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.343 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.343 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.343 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.343 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.343 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.343 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.343 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.343 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.344 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.344 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.344 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.344 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "nfs", 2024-09-11T23:03:11.344 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.344 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.344 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.344 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.344 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.344 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.344 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.344 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.344 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.345 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.345 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.345 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.345 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.345 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.345 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.345 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.345 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.345 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.345 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.345 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.345 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.345 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.345 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.346 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.346 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.346 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.346 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.346 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.346 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.346 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.346 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.346 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.346 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.346 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.346 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.346 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.347 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.347 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.347 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.347 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.347 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.347 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.347 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.347 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.347 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.347 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.347 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.347 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.347 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.347 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.348 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.348 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.348 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.348 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.348 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.348 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.348 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.348 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.348 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.348 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.348 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.348 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.348 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.349 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.349 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.349 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.349 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.349 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.349 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.349 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.349 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.349 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.349 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.349 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.349 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.349 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.349 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "orchestrator", 2024-09-11T23:03:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "fail_fs": { 2024-09-11T23:03:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "fail_fs", 2024-09-11T23:03:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T23:03:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.351 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.351 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.351 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.351 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.351 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.351 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.351 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.351 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.351 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.351 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.351 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.351 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.351 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.351 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.353 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.353 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.353 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.353 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.353 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.353 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.353 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.353 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.353 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.353 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.353 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.353 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.353 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.353 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.354 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.354 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.354 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.354 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.354 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.354 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.354 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.354 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.354 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.354 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.354 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.354 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.354 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.354 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.355 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.355 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.355 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.355 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.355 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.355 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.355 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.355 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.355 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.355 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.355 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.355 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.355 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.355 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.356 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.356 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.356 INFO:teuthology.orchestra.run.smithi152.stdout: "orchestrator": { 2024-09-11T23:03:11.356 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "orchestrator", 2024-09-11T23:03:11.356 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.356 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.356 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.356 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.356 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.356 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.356 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.356 INFO:teuthology.orchestra.run.smithi152.stdout: "cephadm", 2024-09-11T23:03:11.356 INFO:teuthology.orchestra.run.smithi152.stdout: "rook", 2024-09-11T23:03:11.357 INFO:teuthology.orchestra.run.smithi152.stdout: "test_orchestrator" 2024-09-11T23:03:11.357 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.357 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Orchestrator backend", 2024-09-11T23:03:11.357 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.357 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.357 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.357 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.357 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.357 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.357 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.357 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "osd_perf_query", 2024-09-11T23:03:11.357 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.358 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.358 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.358 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.358 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.358 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.358 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.358 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.358 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.358 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.358 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.358 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.358 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.358 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.359 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.359 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.359 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.359 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.359 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.359 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.359 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.359 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.359 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.359 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.359 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.359 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.359 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.359 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.360 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.360 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.360 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.360 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.360 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.360 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.360 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.360 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.361 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.361 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.361 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.361 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.361 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.361 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.361 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.361 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.361 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.361 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.361 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.361 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.361 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.362 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.362 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.362 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.362 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.362 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.362 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.362 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.362 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.362 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.362 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.362 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.362 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.363 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.363 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.363 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.363 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.363 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.363 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.363 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.363 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.363 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.363 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.363 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.363 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.363 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.363 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "osd_support", 2024-09-11T23:03:11.364 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.364 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.364 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.364 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.364 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.364 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.364 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.364 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.364 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.364 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.364 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.364 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.364 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.365 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.365 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.365 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.365 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.365 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.365 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.365 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.365 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.365 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.365 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.365 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.365 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.365 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.365 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.366 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.366 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.366 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.366 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.366 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.366 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.366 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.366 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.366 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.366 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.366 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.366 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.366 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.366 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.368 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.368 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.368 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.368 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.368 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.368 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.368 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.368 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.368 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.368 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.368 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.368 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.368 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.369 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.369 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.369 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.369 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.369 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.369 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.369 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pg_autoscaler", 2024-09-11T23:03:11.369 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.369 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.369 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.369 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.369 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.370 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.370 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.370 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.370 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.370 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.370 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.370 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.370 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.370 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.370 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.370 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.370 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.370 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.371 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.371 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.371 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.371 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.371 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.371 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.371 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.371 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.371 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.371 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.371 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.371 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.371 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.371 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.372 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.372 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.372 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.372 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.372 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.372 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.372 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.372 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.372 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.372 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.372 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.373 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.373 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.373 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.373 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.373 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.373 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.373 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.373 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.373 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.373 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.373 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.373 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.373 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.374 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.374 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.374 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.374 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.374 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.374 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.374 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.374 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.374 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.374 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.374 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.374 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.374 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.375 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-09-11T23:03:11.375 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-09-11T23:03:11.375 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.375 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.375 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.375 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-09-11T23:03:11.375 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.375 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.375 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.375 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.375 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.375 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.375 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.376 INFO:teuthology.orchestra.run.smithi152.stdout: "threshold": { 2024-09-11T23:03:11.376 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "threshold", 2024-09-11T23:03:11.376 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:11.376 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.376 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.376 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "3.0", 2024-09-11T23:03:11.376 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1.0", 2024-09-11T23:03:11.376 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.376 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.376 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "scaling threshold", 2024-09-11T23:03:11.376 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T23:03:11.376 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.376 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.377 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.377 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.377 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "progress", 2024-09-11T23:03:11.377 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.377 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.377 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.377 INFO:teuthology.orchestra.run.smithi152.stdout: "allow_pg_recovery_event": { 2024-09-11T23:03:11.377 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "allow_pg_recovery_event", 2024-09-11T23:03:11.377 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.377 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.377 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.378 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.378 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.378 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.378 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.378 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T23:03:11.378 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.378 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.378 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.378 INFO:teuthology.orchestra.run.smithi152.stdout: "enabled": { 2024-09-11T23:03:11.378 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enabled", 2024-09-11T23:03:11.378 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.378 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.378 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.379 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.379 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.379 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.379 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.379 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.379 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.379 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.379 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.379 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.379 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.379 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.379 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.379 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.379 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.380 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.380 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.380 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.380 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.380 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.380 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.380 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.380 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.380 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.380 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.380 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.380 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.380 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.381 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.381 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.381 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.381 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.381 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.381 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.381 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.381 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.381 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.381 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.381 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.381 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.381 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.382 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.382 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.382 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.382 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.382 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.382 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.382 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.382 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.382 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.382 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.382 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.382 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.382 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.383 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.383 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.383 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.383 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.383 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.383 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.383 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.383 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.383 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.383 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.383 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.383 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.383 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.384 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.384 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.384 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.384 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.384 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.384 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.384 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.384 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.384 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.384 INFO:teuthology.orchestra.run.smithi152.stdout: "max_completed_events": { 2024-09-11T23:03:11.384 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_completed_events", 2024-09-11T23:03:11.384 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.385 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.385 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.385 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "50", 2024-09-11T23:03:11.385 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.385 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.385 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.385 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "number of past completed events to remember", 2024-09-11T23:03:11.385 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.385 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.385 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.385 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-09-11T23:03:11.385 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-09-11T23:03:11.385 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.386 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.386 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.386 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-09-11T23:03:11.386 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.386 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.386 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.386 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how long the module is going to sleep", 2024-09-11T23:03:11.386 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.386 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.386 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.386 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.386 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.386 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.387 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.387 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "prometheus", 2024-09-11T23:03:11.387 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.387 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.387 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.387 INFO:teuthology.orchestra.run.smithi152.stdout: "cache": { 2024-09-11T23:03:11.387 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cache", 2024-09-11T23:03:11.387 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.387 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.387 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.387 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.387 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.387 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.387 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.388 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.388 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.388 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.388 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.388 INFO:teuthology.orchestra.run.smithi152.stdout: "exclude_perf_counters": { 2024-09-11T23:03:11.388 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "exclude_perf_counters", 2024-09-11T23:03:11.388 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.388 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.388 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.388 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.388 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.388 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.389 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.389 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T23:03:11.389 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T23:03:11.389 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.389 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.389 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.389 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.389 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.389 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.389 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.389 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.389 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.389 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.390 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.390 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.390 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.390 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.390 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.390 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.390 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.390 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.390 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.390 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.390 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.390 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.390 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.390 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.391 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.391 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.391 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.391 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.391 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.391 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.391 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.391 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.391 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.391 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.391 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.391 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.392 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.392 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.392 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.392 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.392 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.392 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.392 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.392 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.392 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.392 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.392 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.392 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.392 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.393 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.393 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.393 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.393 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.393 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.393 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.393 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.393 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.393 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.393 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.393 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.393 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.394 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.394 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.394 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.394 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.394 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.394 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.394 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.394 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.394 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_stats_pools": { 2024-09-11T23:03:11.394 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_stats_pools", 2024-09-11T23:03:11.394 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.394 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.394 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.395 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.395 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.395 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.395 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.395 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.395 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.395 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.395 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.395 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T23:03:11.395 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T23:03:11.395 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.395 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.395 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.396 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "300", 2024-09-11T23:03:11.396 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.396 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.396 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.396 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.396 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.396 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.396 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.396 INFO:teuthology.orchestra.run.smithi152.stdout: "scrape_interval": { 2024-09-11T23:03:11.396 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "scrape_interval", 2024-09-11T23:03:11.396 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:11.396 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.397 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.397 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "15.0", 2024-09-11T23:03:11.397 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.397 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.397 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.397 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.397 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.397 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.397 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.397 INFO:teuthology.orchestra.run.smithi152.stdout: "server_addr": { 2024-09-11T23:03:11.397 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_addr", 2024-09-11T23:03:11.397 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.397 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.398 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.398 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "::", 2024-09-11T23:03:11.398 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.398 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.398 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.398 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T23:03:11.398 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.398 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.398 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.398 INFO:teuthology.orchestra.run.smithi152.stdout: "server_port": { 2024-09-11T23:03:11.398 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_port", 2024-09-11T23:03:11.398 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.398 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.399 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.399 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "9283", 2024-09-11T23:03:11.399 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.399 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.399 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.399 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T23:03:11.399 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.399 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.399 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.399 INFO:teuthology.orchestra.run.smithi152.stdout: "stale_cache_strategy": { 2024-09-11T23:03:11.399 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "stale_cache_strategy", 2024-09-11T23:03:11.399 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.400 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.400 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.400 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "log", 2024-09-11T23:03:11.400 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.400 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.400 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.400 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.400 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.400 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.400 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.400 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_behaviour": { 2024-09-11T23:03:11.400 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_behaviour", 2024-09-11T23:03:11.401 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.401 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.401 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.401 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "default", 2024-09-11T23:03:11.401 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.401 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.401 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.401 INFO:teuthology.orchestra.run.smithi152.stdout: "default", 2024-09-11T23:03:11.401 INFO:teuthology.orchestra.run.smithi152.stdout: "error" 2024-09-11T23:03:11.401 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.401 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.401 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.401 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.402 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.402 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_error_status_code": { 2024-09-11T23:03:11.402 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_error_status_code", 2024-09-11T23:03:11.402 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.402 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.402 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.402 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "500", 2024-09-11T23:03:11.402 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "400", 2024-09-11T23:03:11.402 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "599", 2024-09-11T23:03:11.402 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.402 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.402 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.402 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.403 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.403 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.403 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.403 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_support", 2024-09-11T23:03:11.403 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.403 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.403 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.403 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.403 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.403 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.403 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.403 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.404 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.404 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.404 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.404 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.404 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.404 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.404 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.404 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.404 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.404 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.404 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.404 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.404 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.405 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.405 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.405 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.405 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.405 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.405 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.405 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.405 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.405 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.405 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.405 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.405 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.405 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.406 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.406 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.406 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.406 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.406 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.406 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.406 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.406 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.406 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.406 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.406 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.406 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.406 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.407 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.407 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.407 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.407 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.407 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.407 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.407 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.407 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.407 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.407 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.407 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.407 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.407 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.408 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.408 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.408 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.408 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.408 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.408 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.408 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.408 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.408 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.408 INFO:teuthology.orchestra.run.smithi152.stdout: "max_concurrent_snap_create": { 2024-09-11T23:03:11.408 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_concurrent_snap_create", 2024-09-11T23:03:11.408 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.409 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.409 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.409 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:11.409 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.409 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.409 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.409 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.409 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.409 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.409 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.409 INFO:teuthology.orchestra.run.smithi152.stdout: "mirror_snapshot_schedule": { 2024-09-11T23:03:11.409 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T23:03:11.409 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.410 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.410 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.410 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.410 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.410 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.410 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.410 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.410 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.410 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.410 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.410 INFO:teuthology.orchestra.run.smithi152.stdout: "trash_purge_schedule": { 2024-09-11T23:03:11.410 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "trash_purge_schedule", 2024-09-11T23:03:11.410 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.411 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.411 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.411 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.411 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.411 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.411 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.411 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.411 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.411 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.411 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.411 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.411 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.411 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "restful", 2024-09-11T23:03:11.412 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.412 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.412 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.412 INFO:teuthology.orchestra.run.smithi152.stdout: "enable_auth": { 2024-09-11T23:03:11.412 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enable_auth", 2024-09-11T23:03:11.412 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.412 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.412 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.412 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.412 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.412 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.412 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.412 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.413 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.413 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.413 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.413 INFO:teuthology.orchestra.run.smithi152.stdout: "key_file": { 2024-09-11T23:03:11.413 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "key_file", 2024-09-11T23:03:11.413 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.413 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.413 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.413 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.413 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.413 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.413 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.413 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.414 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.414 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.414 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.414 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.414 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.414 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.414 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.414 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.414 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.415 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.415 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.415 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.415 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.415 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.415 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.415 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.415 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.415 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.415 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.415 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.415 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.415 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.416 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.416 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.416 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.416 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.416 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.416 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.416 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.416 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.416 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.416 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.416 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.416 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.416 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.417 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.417 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.417 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.417 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.417 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.417 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.417 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.417 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.417 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.417 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.417 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.417 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.418 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.418 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.418 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.418 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.418 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.418 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.418 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.418 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.418 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.418 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.418 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.418 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.418 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.419 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.419 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.419 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.419 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.419 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.419 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.419 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.419 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.419 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.419 INFO:teuthology.orchestra.run.smithi152.stdout: "server_addr": { 2024-09-11T23:03:11.419 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_addr", 2024-09-11T23:03:11.419 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.419 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.420 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.420 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.420 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.420 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.420 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.420 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.420 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.420 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.420 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.420 INFO:teuthology.orchestra.run.smithi152.stdout: "server_port": { 2024-09-11T23:03:11.420 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_port", 2024-09-11T23:03:11.420 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.420 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.421 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.421 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.421 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.421 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.421 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.421 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.421 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.421 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.421 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.421 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.421 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.421 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rgw", 2024-09-11T23:03:11.422 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.422 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.422 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.422 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.422 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.422 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.422 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.422 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.422 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.422 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.422 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.422 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.422 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.422 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.423 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.423 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.423 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.423 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.423 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.423 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.423 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.423 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.423 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.423 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.423 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.423 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.423 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.424 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.424 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.424 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.424 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.424 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.424 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.424 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.424 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.424 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.424 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.424 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.424 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.424 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.425 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.425 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.425 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.425 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.425 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.425 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.425 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.425 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.425 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.425 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.425 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.425 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.425 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.426 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.426 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.426 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.426 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.426 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.426 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.426 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.426 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.426 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.426 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.426 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.426 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.426 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.427 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.427 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.427 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.427 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.427 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.427 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rook", 2024-09-11T23:03:11.427 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.427 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.427 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.427 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.427 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.427 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.428 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.428 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.428 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.428 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.428 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.428 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.428 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.428 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.428 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.428 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.428 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.428 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.429 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.429 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.429 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.429 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.429 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.429 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.429 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.429 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.429 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.429 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.429 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.429 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.430 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.430 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.430 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.430 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.430 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.430 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.430 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.430 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.430 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.430 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.430 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.430 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.430 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.431 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.431 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.431 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.431 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.431 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.431 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.431 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.431 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.431 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.431 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.431 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.431 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.431 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.432 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.432 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.432 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.432 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.432 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.432 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.432 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.432 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.432 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.432 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.432 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.432 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.432 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.433 INFO:teuthology.orchestra.run.smithi152.stdout: "storage_class": { 2024-09-11T23:03:11.433 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "storage_class", 2024-09-11T23:03:11.433 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.433 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.433 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.433 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "local", 2024-09-11T23:03:11.433 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.433 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.433 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.433 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T23:03:11.433 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.433 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.434 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.434 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.434 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.434 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "selftest", 2024-09-11T23:03:11.434 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.434 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.434 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.434 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.434 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.434 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.434 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.434 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.435 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.435 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.435 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.435 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.435 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.435 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.435 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.435 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.435 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.435 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.435 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.435 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.435 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.435 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.436 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.436 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.436 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.436 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.436 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.436 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.436 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.436 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.436 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.436 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.436 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.436 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.436 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.437 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.437 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.437 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.437 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.437 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.437 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.437 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.437 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.437 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.437 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.437 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.437 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.438 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.438 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.438 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.438 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.438 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.438 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.438 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.438 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.438 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.438 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.438 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.438 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.438 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.439 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.439 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.439 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.439 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.439 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.439 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.439 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.439 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.439 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.439 INFO:teuthology.orchestra.run.smithi152.stdout: "roption1": { 2024-09-11T23:03:11.439 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "roption1", 2024-09-11T23:03:11.439 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.439 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.440 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.440 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.440 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.440 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.440 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.440 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.440 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.440 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.440 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.440 INFO:teuthology.orchestra.run.smithi152.stdout: "roption2": { 2024-09-11T23:03:11.440 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "roption2", 2024-09-11T23:03:11.440 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.440 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.441 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.441 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "xyz", 2024-09-11T23:03:11.441 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.441 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.441 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.441 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.441 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.441 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.441 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.441 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption1": { 2024-09-11T23:03:11.441 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption1", 2024-09-11T23:03:11.441 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.442 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.442 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.442 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.442 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.442 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.442 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.442 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.442 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.442 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.442 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.442 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption2": { 2024-09-11T23:03:11.442 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption2", 2024-09-11T23:03:11.442 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.443 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.443 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.443 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.443 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.443 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.443 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.443 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.443 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.443 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.443 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.443 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption3": { 2024-09-11T23:03:11.443 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption3", 2024-09-11T23:03:11.444 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:11.444 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.444 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.444 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.444 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.444 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.444 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.444 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.444 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.444 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.444 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.444 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption4": { 2024-09-11T23:03:11.445 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption4", 2024-09-11T23:03:11.445 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.445 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.445 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.445 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.445 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.445 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.445 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.445 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.445 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.445 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.445 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.446 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption5": { 2024-09-11T23:03:11.446 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption5", 2024-09-11T23:03:11.446 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.446 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.446 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.446 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.446 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.446 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.446 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.446 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.446 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.446 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.446 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.447 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption6": { 2024-09-11T23:03:11.447 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption6", 2024-09-11T23:03:11.447 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.447 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.447 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.447 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.447 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.447 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.447 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.447 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.447 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.447 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.447 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.448 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption7": { 2024-09-11T23:03:11.448 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption7", 2024-09-11T23:03:11.448 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.448 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.448 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.448 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.448 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-09-11T23:03:11.448 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "42", 2024-09-11T23:03:11.448 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.448 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.448 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.448 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.448 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.449 INFO:teuthology.orchestra.run.smithi152.stdout: "testkey": { 2024-09-11T23:03:11.449 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "testkey", 2024-09-11T23:03:11.449 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.449 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.449 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.449 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.449 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.449 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.449 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.449 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.449 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.449 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.450 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.450 INFO:teuthology.orchestra.run.smithi152.stdout: "testlkey": { 2024-09-11T23:03:11.450 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "testlkey", 2024-09-11T23:03:11.450 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.450 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.450 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.450 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.450 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.450 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.450 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.450 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.450 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.450 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.451 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.451 INFO:teuthology.orchestra.run.smithi152.stdout: "testnewline": { 2024-09-11T23:03:11.451 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "testnewline", 2024-09-11T23:03:11.451 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.451 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.451 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.451 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.451 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.451 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.451 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.451 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.451 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.451 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.452 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.452 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.452 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.452 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snap_schedule", 2024-09-11T23:03:11.452 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.452 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.452 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.452 INFO:teuthology.orchestra.run.smithi152.stdout: "allow_m_granularity": { 2024-09-11T23:03:11.452 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "allow_m_granularity", 2024-09-11T23:03:11.452 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.452 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.452 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.453 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.453 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.453 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.453 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.453 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T23:03:11.453 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.453 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.453 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.453 INFO:teuthology.orchestra.run.smithi152.stdout: "dump_on_update": { 2024-09-11T23:03:11.453 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "dump_on_update", 2024-09-11T23:03:11.453 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.453 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.453 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.454 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.454 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.454 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.454 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.454 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "dump database to debug log on update", 2024-09-11T23:03:11.454 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.454 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.454 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.454 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.454 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.454 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.454 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.454 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.455 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.455 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.455 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.455 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.455 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.455 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.455 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.455 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.455 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.455 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.455 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.455 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.455 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.456 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.456 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.456 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.456 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.456 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.456 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.456 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.456 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.456 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.456 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.456 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.456 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.456 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.457 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.457 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.457 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.457 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.457 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.457 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.457 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.457 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.457 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.457 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.457 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.458 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.458 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.458 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.458 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.458 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.458 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.458 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.458 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.458 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.458 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.458 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.458 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.458 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.459 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.459 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.459 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.459 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.459 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.459 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.459 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.459 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.459 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.459 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.459 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.459 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.459 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.460 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "stats", 2024-09-11T23:03:11.460 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.460 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.460 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.460 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.460 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.460 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.460 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.461 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.461 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.461 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.461 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.461 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.461 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.461 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.461 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.461 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.461 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.461 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.461 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.461 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.462 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.462 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.462 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.462 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.462 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.462 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.462 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.462 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.462 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.462 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.462 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.462 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.462 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.463 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.463 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.463 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.463 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.463 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.463 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.463 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.463 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.463 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.463 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.463 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.463 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.464 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.464 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.464 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.464 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.464 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.464 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.464 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.464 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.464 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.464 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.464 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.464 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.464 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.465 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.465 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.465 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.465 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.465 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.465 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.465 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.465 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.465 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.465 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.465 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.465 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.465 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.466 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "status", 2024-09-11T23:03:11.466 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.466 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.466 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.466 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.466 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.466 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.466 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.466 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.466 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.466 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.466 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.466 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.467 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.467 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.467 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.467 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.467 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.467 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.467 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.467 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.467 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.467 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.467 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.467 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.467 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.468 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.468 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.468 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.468 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.468 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.468 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.468 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.468 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.468 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.468 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.468 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.468 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.469 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.469 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.469 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.469 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.469 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.469 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.469 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.469 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.469 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.469 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.469 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.469 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.469 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.469 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.470 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.470 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.470 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.470 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.470 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.470 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.470 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.470 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.470 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.470 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.470 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.470 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.470 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.471 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.471 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.471 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.471 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.471 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.471 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.471 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.471 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "telegraf", 2024-09-11T23:03:11.471 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.471 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.471 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.471 INFO:teuthology.orchestra.run.smithi152.stdout: "address": { 2024-09-11T23:03:11.472 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "address", 2024-09-11T23:03:11.472 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.472 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.472 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.472 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T23:03:11.472 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.472 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.472 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.472 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.472 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.472 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.472 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.473 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-09-11T23:03:11.473 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-09-11T23:03:11.473 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.473 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.473 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.473 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "15", 2024-09-11T23:03:11.473 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.473 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.473 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.473 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.473 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.473 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.474 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.474 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.474 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.474 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.474 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.474 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.474 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.474 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.474 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.474 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.474 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.474 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.474 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.475 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.475 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.475 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.475 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.475 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.475 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.475 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.475 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.475 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.475 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.475 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.475 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.476 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.476 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.476 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.476 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.476 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.476 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.476 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.476 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.476 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.476 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.476 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.476 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.477 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.477 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.477 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.477 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.477 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.477 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.477 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.477 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.477 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.477 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.477 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.477 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.477 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.477 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.478 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.478 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.478 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.478 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.478 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.478 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.478 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.478 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.478 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.478 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.478 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.478 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.478 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.479 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.479 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.479 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.479 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.479 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.479 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "telemetry", 2024-09-11T23:03:11.479 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.479 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.479 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.479 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_basic": { 2024-09-11T23:03:11.479 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_basic", 2024-09-11T23:03:11.479 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.480 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.480 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.480 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.480 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.480 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.480 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.480 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T23:03:11.480 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.480 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.480 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.480 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_crash": { 2024-09-11T23:03:11.480 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_crash", 2024-09-11T23:03:11.481 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.481 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.481 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.481 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.481 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.481 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.481 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.481 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T23:03:11.481 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.481 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.481 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.481 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_device": { 2024-09-11T23:03:11.482 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_device", 2024-09-11T23:03:11.482 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.482 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.482 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.482 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.482 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.482 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.482 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.482 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T23:03:11.482 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.482 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.482 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.482 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_ident": { 2024-09-11T23:03:11.483 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_ident", 2024-09-11T23:03:11.483 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.483 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.483 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.483 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.483 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.483 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.483 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.483 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T23:03:11.483 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.483 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.483 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.483 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_perf": { 2024-09-11T23:03:11.484 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_perf", 2024-09-11T23:03:11.484 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.484 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.484 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.484 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.484 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.484 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.484 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.484 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T23:03:11.484 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.484 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.484 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.484 INFO:teuthology.orchestra.run.smithi152.stdout: "contact": { 2024-09-11T23:03:11.485 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "contact", 2024-09-11T23:03:11.485 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.485 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.485 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.485 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.485 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.485 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.485 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.485 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.485 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.485 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.485 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.486 INFO:teuthology.orchestra.run.smithi152.stdout: "description": { 2024-09-11T23:03:11.486 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "description", 2024-09-11T23:03:11.486 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.486 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.486 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.486 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.486 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.486 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.486 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.486 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.486 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.486 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.486 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.487 INFO:teuthology.orchestra.run.smithi152.stdout: "device_url": { 2024-09-11T23:03:11.487 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "device_url", 2024-09-11T23:03:11.487 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.487 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.487 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.487 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T23:03:11.487 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.487 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.487 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.487 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.487 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.487 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.488 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.488 INFO:teuthology.orchestra.run.smithi152.stdout: "enabled": { 2024-09-11T23:03:11.488 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enabled", 2024-09-11T23:03:11.488 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.488 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.488 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.488 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.488 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.488 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.488 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.488 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.488 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.488 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.489 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.489 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-09-11T23:03:11.489 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-09-11T23:03:11.489 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.489 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.489 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.489 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "24", 2024-09-11T23:03:11.489 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "8", 2024-09-11T23:03:11.489 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.489 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.489 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.489 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.490 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.490 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.490 INFO:teuthology.orchestra.run.smithi152.stdout: "last_opt_revision": { 2024-09-11T23:03:11.490 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "last_opt_revision", 2024-09-11T23:03:11.490 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.490 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.490 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.490 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "1", 2024-09-11T23:03:11.490 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.490 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.490 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.490 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.490 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.491 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.491 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.491 INFO:teuthology.orchestra.run.smithi152.stdout: "leaderboard": { 2024-09-11T23:03:11.491 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "leaderboard", 2024-09-11T23:03:11.491 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.491 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.491 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.491 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.491 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.491 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.491 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.491 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.491 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.492 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.492 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.492 INFO:teuthology.orchestra.run.smithi152.stdout: "leaderboard_description": { 2024-09-11T23:03:11.492 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "leaderboard_description", 2024-09-11T23:03:11.492 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.492 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.492 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.492 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.492 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.492 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.492 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.492 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.493 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.493 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.493 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.493 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.493 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.493 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.493 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.493 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.493 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.493 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.493 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.493 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.493 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.494 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.494 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.494 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.494 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.494 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.494 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.494 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.494 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.494 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.494 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.494 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.494 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.494 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.495 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.495 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.495 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.495 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.495 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.495 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.495 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.495 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.495 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.495 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.495 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.495 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.495 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.496 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.496 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.496 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.496 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.496 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.496 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.496 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.496 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.496 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.496 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.496 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.496 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.496 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.497 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.497 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.497 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.497 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.497 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.497 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.497 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.497 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.497 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.497 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.497 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.497 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.497 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.498 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.498 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.498 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.498 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.498 INFO:teuthology.orchestra.run.smithi152.stdout: "organization": { 2024-09-11T23:03:11.498 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "organization", 2024-09-11T23:03:11.498 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.498 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.498 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.498 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.498 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.498 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.498 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.498 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.499 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.499 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.499 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.499 INFO:teuthology.orchestra.run.smithi152.stdout: "proxy": { 2024-09-11T23:03:11.499 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "proxy", 2024-09-11T23:03:11.499 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.499 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.499 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.499 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.499 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.499 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.499 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.500 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.500 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.500 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.500 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.500 INFO:teuthology.orchestra.run.smithi152.stdout: "url": { 2024-09-11T23:03:11.500 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "url", 2024-09-11T23:03:11.500 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.500 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.500 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.500 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T23:03:11.500 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.500 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.501 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.501 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.501 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.501 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.501 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.501 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.501 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.501 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "test_orchestrator", 2024-09-11T23:03:11.501 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.501 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.501 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.501 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.502 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.502 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.502 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.502 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.502 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.502 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.502 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.502 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.502 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.502 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.502 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.502 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.502 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.502 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.503 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.503 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.503 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.503 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.503 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.503 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.503 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.503 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.503 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.503 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.503 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.503 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.504 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.504 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.504 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.504 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.504 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.504 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.504 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.504 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.504 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.504 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.504 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.504 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.504 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.505 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.505 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.505 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.505 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.505 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.505 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.505 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.505 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.505 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.505 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.505 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.505 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.505 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.506 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.506 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.506 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.506 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.506 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.506 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.506 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.506 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.506 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.506 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.506 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.506 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.506 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.507 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.507 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.507 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "volumes", 2024-09-11T23:03:11.507 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.507 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.507 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.507 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.507 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.507 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.507 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.507 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.507 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.507 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.508 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.508 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.508 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.508 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.508 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.508 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.508 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.508 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.508 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.508 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.508 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.508 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.508 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.509 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.509 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.509 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.509 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.509 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.509 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.509 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.509 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.509 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.509 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.509 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.509 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.509 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.510 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.510 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.510 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.510 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.510 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.510 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.510 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.510 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.510 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.510 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.510 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.510 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.510 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.511 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.511 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.511 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.511 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.511 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.511 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.511 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.511 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.511 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.511 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.511 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.511 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.511 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.512 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.512 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.512 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.512 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.512 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.512 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.512 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.512 INFO:teuthology.orchestra.run.smithi152.stdout: "max_concurrent_clones": { 2024-09-11T23:03:11.512 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_concurrent_clones", 2024-09-11T23:03:11.512 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.512 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.512 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.513 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "4", 2024-09-11T23:03:11.513 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.513 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.513 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.513 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T23:03:11.513 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.513 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.513 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.513 INFO:teuthology.orchestra.run.smithi152.stdout: "periodic_async_work": { 2024-09-11T23:03:11.513 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "periodic_async_work", 2024-09-11T23:03:11.513 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.513 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.513 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.514 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.514 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.514 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.514 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.514 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Periodically check for async work", 2024-09-11T23:03:11.514 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.514 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.514 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.514 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot_clone_delay": { 2024-09-11T23:03:11.514 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snapshot_clone_delay", 2024-09-11T23:03:11.514 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.514 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.515 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.515 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-09-11T23:03:11.515 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.515 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.515 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.515 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T23:03:11.515 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.515 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.515 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.515 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot_clone_no_wait": { 2024-09-11T23:03:11.515 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T23:03:11.515 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.516 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.516 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.516 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.516 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.516 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.516 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.516 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T23:03:11.516 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.516 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.516 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.516 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.516 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.517 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix", 2024-09-11T23:03:11.517 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.517 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.517 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.517 INFO:teuthology.orchestra.run.smithi152.stdout: "discovery_interval": { 2024-09-11T23:03:11.517 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "discovery_interval", 2024-09-11T23:03:11.517 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-09-11T23:03:11.517 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.517 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.517 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "100", 2024-09-11T23:03:11.517 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.517 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.517 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.518 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.518 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.518 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.518 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.518 INFO:teuthology.orchestra.run.smithi152.stdout: "identifier": { 2024-09-11T23:03:11.518 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "identifier", 2024-09-11T23:03:11.519 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.519 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.520 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.520 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.520 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.520 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.520 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.520 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.520 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.520 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.520 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.520 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-09-11T23:03:11.520 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-09-11T23:03:11.520 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.521 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.521 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.521 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-09-11T23:03:11.521 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.521 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.521 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.521 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.521 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.521 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.521 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.521 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.521 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.521 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.522 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.522 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.522 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.522 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.522 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.522 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.522 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.522 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.522 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.522 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.522 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.522 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.522 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.522 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.523 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.523 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.523 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.523 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.523 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.523 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.523 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.523 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.523 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.523 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.523 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.523 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.523 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.524 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.524 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.524 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.524 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.524 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.524 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.524 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.524 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.524 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.524 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.524 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.524 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.525 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.525 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.525 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.525 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.525 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.525 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.525 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.525 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.525 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.525 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.525 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.525 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.525 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.526 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.526 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.526 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.526 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.526 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.526 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.526 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.526 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.526 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.526 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.526 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.526 INFO:teuthology.orchestra.run.smithi152.stdout: "zabbix_host": { 2024-09-11T23:03:11.526 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix_host", 2024-09-11T23:03:11.526 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.527 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.527 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.527 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.527 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.527 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.527 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.527 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.527 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.527 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.527 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.527 INFO:teuthology.orchestra.run.smithi152.stdout: "zabbix_port": { 2024-09-11T23:03:11.527 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix_port", 2024-09-11T23:03:11.527 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.528 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.528 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.528 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10051", 2024-09-11T23:03:11.528 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.528 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.528 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.528 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.528 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.528 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.528 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.528 INFO:teuthology.orchestra.run.smithi152.stdout: "zabbix_sender": { 2024-09-11T23:03:11.528 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix_sender", 2024-09-11T23:03:11.529 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.529 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.529 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.529 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T23:03:11.529 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.529 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.529 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.529 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.529 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.529 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.529 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.529 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.529 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.530 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.530 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-11T23:03:11.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.530 INFO:teuthology.orchestra.run.smithi152.stdout: "gid": 24770, 2024-09-11T23:03:11.530 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "y", 2024-09-11T23:03:11.530 INFO:teuthology.orchestra.run.smithi152.stdout: "mgr_features": 4540701547738038271, 2024-09-11T23:03:11.530 INFO:teuthology.orchestra.run.smithi152.stdout: "available_modules": [ 2024-09-11T23:03:11.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.530 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "alerts", 2024-09-11T23:03:11.530 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.530 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.530 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.531 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-09-11T23:03:11.531 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-09-11T23:03:11.531 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.531 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.531 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.531 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-09-11T23:03:11.531 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.531 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.531 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.531 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T23:03:11.531 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.531 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.531 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.532 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.532 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.532 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.532 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.532 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.532 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.532 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.532 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.532 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.532 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.532 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.532 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.532 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.533 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.533 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.533 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.533 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.533 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.533 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.533 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.533 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.533 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.533 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.533 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.533 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.533 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.534 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.534 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.534 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.534 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.534 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.534 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.534 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.534 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.534 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.534 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.534 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.534 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.535 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.535 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.535 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.535 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.535 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.535 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.535 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.535 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.535 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.535 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.535 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.535 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.535 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.535 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.536 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.536 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.536 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.536 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.536 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.536 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.536 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.536 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.536 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.536 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.536 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.536 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.536 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.537 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.537 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_destination": { 2024-09-11T23:03:11.537 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_destination", 2024-09-11T23:03:11.537 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.537 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.537 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.537 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.537 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.537 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.537 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.537 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Email address to send alerts to", 2024-09-11T23:03:11.537 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.537 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.538 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.538 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_from_name": { 2024-09-11T23:03:11.538 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_from_name", 2024-09-11T23:03:11.538 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.538 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.538 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.538 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "Ceph", 2024-09-11T23:03:11.538 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.538 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.538 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.538 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Email From: name", 2024-09-11T23:03:11.538 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.538 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.539 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.539 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_host": { 2024-09-11T23:03:11.539 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_host", 2024-09-11T23:03:11.539 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.539 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.539 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.539 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.539 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.539 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.539 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.539 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SMTP server", 2024-09-11T23:03:11.539 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.539 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.540 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.540 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_password": { 2024-09-11T23:03:11.540 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_password", 2024-09-11T23:03:11.540 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.540 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.540 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.540 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.540 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.540 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.540 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.540 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Password to authenticate with", 2024-09-11T23:03:11.540 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.541 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.541 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.541 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_port": { 2024-09-11T23:03:11.541 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_port", 2024-09-11T23:03:11.541 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.541 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.541 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.541 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "465", 2024-09-11T23:03:11.541 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.541 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.541 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.541 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SMTP port", 2024-09-11T23:03:11.541 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.542 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.542 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.542 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_sender": { 2024-09-11T23:03:11.542 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_sender", 2024-09-11T23:03:11.542 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.542 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.542 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.542 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.542 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.542 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.542 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.542 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SMTP envelope sender", 2024-09-11T23:03:11.543 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.543 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.543 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.543 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_ssl": { 2024-09-11T23:03:11.543 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_ssl", 2024-09-11T23:03:11.543 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.543 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.543 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.543 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.543 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.543 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.543 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.543 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T23:03:11.544 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.544 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.544 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.544 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_user": { 2024-09-11T23:03:11.544 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_user", 2024-09-11T23:03:11.544 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.544 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.544 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.544 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.544 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.544 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.545 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.545 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "User to authenticate as", 2024-09-11T23:03:11.545 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.545 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.545 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.545 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.545 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.545 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "balancer", 2024-09-11T23:03:11.545 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.545 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.545 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.545 INFO:teuthology.orchestra.run.smithi152.stdout: "active": { 2024-09-11T23:03:11.546 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "active", 2024-09-11T23:03:11.546 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.546 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.546 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.546 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.546 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.546 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.546 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.546 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T23:03:11.546 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.546 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.546 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.546 INFO:teuthology.orchestra.run.smithi152.stdout: "begin_time": { 2024-09-11T23:03:11.547 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "begin_time", 2024-09-11T23:03:11.547 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.547 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.547 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.547 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0000", 2024-09-11T23:03:11.547 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.547 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.547 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.547 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T23:03:11.547 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:03:11.547 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.547 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.548 INFO:teuthology.orchestra.run.smithi152.stdout: "begin_weekday": { 2024-09-11T23:03:11.548 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "begin_weekday", 2024-09-11T23:03:11.548 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-09-11T23:03:11.548 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.548 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.548 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-09-11T23:03:11.548 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "0", 2024-09-11T23:03:11.548 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "6", 2024-09-11T23:03:11.548 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.548 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T23:03:11.548 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:03:11.548 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.548 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.549 INFO:teuthology.orchestra.run.smithi152.stdout: "crush_compat_max_iterations": { 2024-09-11T23:03:11.549 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crush_compat_max_iterations", 2024-09-11T23:03:11.549 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-09-11T23:03:11.549 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.549 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.549 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "25", 2024-09-11T23:03:11.549 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-09-11T23:03:11.549 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "250", 2024-09-11T23:03:11.549 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.549 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T23:03:11.549 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.549 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.549 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.550 INFO:teuthology.orchestra.run.smithi152.stdout: "crush_compat_metrics": { 2024-09-11T23:03:11.550 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crush_compat_metrics", 2024-09-11T23:03:11.550 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.550 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.550 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.550 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T23:03:11.550 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.550 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.550 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.550 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T23:03:11.550 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T23:03:11.550 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.550 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.551 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.551 INFO:teuthology.orchestra.run.smithi152.stdout: "crush_compat_step": { 2024-09-11T23:03:11.551 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crush_compat_step", 2024-09-11T23:03:11.551 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:11.551 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.551 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.551 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0.5", 2024-09-11T23:03:11.551 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "0.001", 2024-09-11T23:03:11.551 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "0.999", 2024-09-11T23:03:11.551 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.551 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "aggressiveness of optimization", 2024-09-11T23:03:11.551 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T23:03:11.551 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.551 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.552 INFO:teuthology.orchestra.run.smithi152.stdout: "end_time": { 2024-09-11T23:03:11.552 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "end_time", 2024-09-11T23:03:11.552 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.552 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.552 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.552 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "2359", 2024-09-11T23:03:11.552 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.552 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.552 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.552 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T23:03:11.552 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:03:11.552 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.552 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.553 INFO:teuthology.orchestra.run.smithi152.stdout: "end_weekday": { 2024-09-11T23:03:11.553 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "end_weekday", 2024-09-11T23:03:11.553 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-09-11T23:03:11.553 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.553 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.553 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-09-11T23:03:11.553 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "0", 2024-09-11T23:03:11.553 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "6", 2024-09-11T23:03:11.553 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.553 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T23:03:11.553 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:03:11.553 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.553 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.555 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.555 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.555 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.555 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.555 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.555 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.555 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.555 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.555 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.555 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.555 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.555 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.555 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.556 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.556 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.556 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.556 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.556 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.556 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.556 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.556 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.556 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.556 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.556 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.556 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.557 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.557 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.557 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.557 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.557 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.557 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.557 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.557 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.557 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.557 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.557 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.557 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.557 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.557 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.558 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.558 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.558 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.558 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.558 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.558 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.558 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.558 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.558 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.558 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.558 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.558 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.559 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.559 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.559 INFO:teuthology.orchestra.run.smithi152.stdout: "min_score": { 2024-09-11T23:03:11.559 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "min_score", 2024-09-11T23:03:11.559 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:11.559 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.559 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.559 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-09-11T23:03:11.559 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.559 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.559 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.559 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T23:03:11.559 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.560 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.560 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.560 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.560 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": { 2024-09-11T23:03:11.560 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mode", 2024-09-11T23:03:11.560 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.560 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.560 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.560 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "upmap", 2024-09-11T23:03:11.560 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.561 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.561 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.561 INFO:teuthology.orchestra.run.smithi152.stdout: "crush-compat", 2024-09-11T23:03:11.561 INFO:teuthology.orchestra.run.smithi152.stdout: "none", 2024-09-11T23:03:11.561 INFO:teuthology.orchestra.run.smithi152.stdout: "read", 2024-09-11T23:03:11.561 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap", 2024-09-11T23:03:11.561 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap-read" 2024-09-11T23:03:11.561 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.561 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Balancer mode", 2024-09-11T23:03:11.561 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.561 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.561 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.562 INFO:teuthology.orchestra.run.smithi152.stdout: "pool_ids": { 2024-09-11T23:03:11.562 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pool_ids", 2024-09-11T23:03:11.562 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.562 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.562 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.562 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.562 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.562 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.562 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.562 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T23:03:11.562 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.562 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.562 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.563 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-09-11T23:03:11.563 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-09-11T23:03:11.563 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.563 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.563 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.563 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-09-11T23:03:11.563 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.563 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.563 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.563 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T23:03:11.563 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.563 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.564 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.564 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap_max_deviation": { 2024-09-11T23:03:11.564 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "upmap_max_deviation", 2024-09-11T23:03:11.564 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.564 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.564 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.564 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-09-11T23:03:11.564 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-09-11T23:03:11.564 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.564 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.564 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T23:03:11.564 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T23:03:11.564 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.565 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.565 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap_max_optimizations": { 2024-09-11T23:03:11.565 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "upmap_max_optimizations", 2024-09-11T23:03:11.565 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-09-11T23:03:11.565 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.565 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.565 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:11.565 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.565 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.565 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.565 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T23:03:11.565 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.565 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.566 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.566 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.566 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.566 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephadm", 2024-09-11T23:03:11.566 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.566 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.566 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.566 INFO:teuthology.orchestra.run.smithi152.stdout: "agent_down_multiplier": { 2024-09-11T23:03:11.566 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "agent_down_multiplier", 2024-09-11T23:03:11.566 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:11.566 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.567 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.567 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "3.0", 2024-09-11T23:03:11.567 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.567 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.567 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.567 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T23:03:11.567 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.567 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.567 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.567 INFO:teuthology.orchestra.run.smithi152.stdout: "agent_refresh_rate": { 2024-09-11T23:03:11.567 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "agent_refresh_rate", 2024-09-11T23:03:11.567 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.567 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.568 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.568 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "20", 2024-09-11T23:03:11.568 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.568 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.568 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.568 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T23:03:11.568 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.568 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.568 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.568 INFO:teuthology.orchestra.run.smithi152.stdout: "agent_starting_port": { 2024-09-11T23:03:11.568 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "agent_starting_port", 2024-09-11T23:03:11.568 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.568 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.569 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.569 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "4721", 2024-09-11T23:03:11.569 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.569 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.569 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.569 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T23:03:11.569 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.569 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.569 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.569 INFO:teuthology.orchestra.run.smithi152.stdout: "allow_ptrace": { 2024-09-11T23:03:11.569 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "allow_ptrace", 2024-09-11T23:03:11.569 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.570 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.570 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.570 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.570 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.570 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.570 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.570 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T23:03:11.570 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T23:03:11.570 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.570 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.570 INFO:teuthology.orchestra.run.smithi152.stdout: "autotune_interval": { 2024-09-11T23:03:11.570 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "autotune_interval", 2024-09-11T23:03:11.571 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.571 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.571 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.571 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-09-11T23:03:11.571 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.571 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.571 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.571 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T23:03:11.571 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.571 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.571 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.571 INFO:teuthology.orchestra.run.smithi152.stdout: "autotune_memory_target_ratio": { 2024-09-11T23:03:11.572 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T23:03:11.572 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:11.572 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.572 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.572 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0.7", 2024-09-11T23:03:11.572 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.572 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.572 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.572 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T23:03:11.572 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.572 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.572 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.572 INFO:teuthology.orchestra.run.smithi152.stdout: "cephadm_log_destination": { 2024-09-11T23:03:11.573 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephadm_log_destination", 2024-09-11T23:03:11.573 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.573 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.573 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.573 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.573 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.573 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.573 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.573 INFO:teuthology.orchestra.run.smithi152.stdout: "file", 2024-09-11T23:03:11.573 INFO:teuthology.orchestra.run.smithi152.stdout: "file,syslog", 2024-09-11T23:03:11.573 INFO:teuthology.orchestra.run.smithi152.stdout: "syslog" 2024-09-11T23:03:11.573 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.573 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T23:03:11.574 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.574 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.574 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.574 INFO:teuthology.orchestra.run.smithi152.stdout: "cgroups_split": { 2024-09-11T23:03:11.574 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cgroups_split", 2024-09-11T23:03:11.574 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.574 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.574 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.574 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.574 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.574 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.574 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.574 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T23:03:11.575 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.575 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.575 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.575 INFO:teuthology.orchestra.run.smithi152.stdout: "config_checks_enabled": { 2024-09-11T23:03:11.575 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "config_checks_enabled", 2024-09-11T23:03:11.575 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.575 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.575 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.575 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.575 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.575 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.575 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.576 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T23:03:11.576 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.576 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.576 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.576 INFO:teuthology.orchestra.run.smithi152.stdout: "config_dashboard": { 2024-09-11T23:03:11.576 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "config_dashboard", 2024-09-11T23:03:11.576 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.576 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.576 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.576 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.576 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.576 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.576 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.577 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T23:03:11.577 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.577 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.577 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.577 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_alertmanager": { 2024-09-11T23:03:11.577 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_alertmanager", 2024-09-11T23:03:11.577 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.577 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.577 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.577 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T23:03:11.577 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.577 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.578 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.578 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-09-11T23:03:11.578 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.578 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.578 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.578 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_base": { 2024-09-11T23:03:11.578 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_base", 2024-09-11T23:03:11.578 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.578 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.578 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.578 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T23:03:11.578 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.579 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.579 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.579 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Container image name, without the tag", 2024-09-11T23:03:11.579 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.579 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.579 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.579 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_elasticsearch": { 2024-09-11T23:03:11.579 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_elasticsearch", 2024-09-11T23:03:11.579 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.579 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.579 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.579 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T23:03:11.579 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.580 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.580 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.580 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "elasticsearch container image", 2024-09-11T23:03:11.580 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.580 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.580 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.580 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_grafana": { 2024-09-11T23:03:11.580 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_grafana", 2024-09-11T23:03:11.580 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.580 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.580 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.580 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T23:03:11.580 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.581 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.581 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.581 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-09-11T23:03:11.581 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.581 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.581 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.581 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_haproxy": { 2024-09-11T23:03:11.581 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_haproxy", 2024-09-11T23:03:11.581 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.581 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.581 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.581 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T23:03:11.582 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.582 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.582 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.582 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "HAproxy container image", 2024-09-11T23:03:11.582 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.582 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.582 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.582 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_jaeger_agent": { 2024-09-11T23:03:11.582 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_jaeger_agent", 2024-09-11T23:03:11.582 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.582 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.582 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.583 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T23:03:11.583 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.583 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.583 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.583 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Jaeger agent container image", 2024-09-11T23:03:11.583 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.583 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.583 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.583 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_jaeger_collector": { 2024-09-11T23:03:11.583 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_jaeger_collector", 2024-09-11T23:03:11.583 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.583 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.583 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Jaeger collector container image", 2024-09-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_jaeger_query": { 2024-09-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_jaeger_query", 2024-09-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.585 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.585 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T23:03:11.585 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.585 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.585 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.585 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Jaeger query container image", 2024-09-11T23:03:11.585 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.585 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.585 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.585 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_keepalived": { 2024-09-11T23:03:11.585 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_keepalived", 2024-09-11T23:03:11.585 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.585 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.586 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.586 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T23:03:11.586 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.586 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.586 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.586 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Keepalived container image", 2024-09-11T23:03:11.586 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.586 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.586 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.586 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_loki": { 2024-09-11T23:03:11.586 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_loki", 2024-09-11T23:03:11.586 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.586 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.587 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.587 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T23:03:11.587 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.587 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.587 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.587 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Loki container image", 2024-09-11T23:03:11.587 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.587 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.587 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.587 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_node_exporter": { 2024-09-11T23:03:11.587 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_node_exporter", 2024-09-11T23:03:11.587 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.588 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.588 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.588 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T23:03:11.588 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.588 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.588 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.588 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-09-11T23:03:11.588 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.588 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.588 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.588 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_nvmeof": { 2024-09-11T23:03:11.588 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_nvmeof", 2024-09-11T23:03:11.588 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.589 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.589 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.589 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T23:03:11.589 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.589 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.589 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.589 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Nvme-of container image", 2024-09-11T23:03:11.589 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.589 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.589 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.589 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_prometheus": { 2024-09-11T23:03:11.589 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_prometheus", 2024-09-11T23:03:11.589 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.590 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.590 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.590 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T23:03:11.590 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.590 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.590 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.590 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-09-11T23:03:11.590 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.590 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.590 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.590 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_promtail": { 2024-09-11T23:03:11.590 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_promtail", 2024-09-11T23:03:11.591 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.591 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.591 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.591 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T23:03:11.591 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.591 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.591 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.591 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Promtail container image", 2024-09-11T23:03:11.591 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.591 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.591 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.591 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_samba": { 2024-09-11T23:03:11.591 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_samba", 2024-09-11T23:03:11.592 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.592 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.592 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.592 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T23:03:11.592 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.592 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.592 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.592 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Samba/SMB container image", 2024-09-11T23:03:11.592 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.592 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.592 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.592 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.592 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_snmp_gateway": { 2024-09-11T23:03:11.592 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_snmp_gateway", 2024-09-11T23:03:11.593 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.593 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.593 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.593 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T23:03:11.593 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.593 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.593 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.593 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SNMP Gateway container image", 2024-09-11T23:03:11.593 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.593 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.593 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.593 INFO:teuthology.orchestra.run.smithi152.stdout: "container_init": { 2024-09-11T23:03:11.593 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_init", 2024-09-11T23:03:11.594 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.594 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.594 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.594 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.594 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.594 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.594 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.594 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T23:03:11.594 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.594 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.594 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.594 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.594 INFO:teuthology.orchestra.run.smithi152.stdout: "daemon_cache_timeout": { 2024-09-11T23:03:11.594 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "daemon_cache_timeout", 2024-09-11T23:03:11.595 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.595 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.595 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.595 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-09-11T23:03:11.595 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.595 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.595 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.595 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T23:03:11.595 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.595 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.595 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.595 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.596 INFO:teuthology.orchestra.run.smithi152.stdout: "default_cephadm_command_timeout": { 2024-09-11T23:03:11.596 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T23:03:11.596 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.596 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.596 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.596 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "900", 2024-09-11T23:03:11.596 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.596 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.596 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.596 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T23:03:11.596 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.596 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.596 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.596 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.597 INFO:teuthology.orchestra.run.smithi152.stdout: "default_registry": { 2024-09-11T23:03:11.597 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "default_registry", 2024-09-11T23:03:11.597 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.597 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.597 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.597 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io", 2024-09-11T23:03:11.597 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.597 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.597 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.597 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T23:03:11.597 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.597 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.597 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.598 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.598 INFO:teuthology.orchestra.run.smithi152.stdout: "device_cache_timeout": { 2024-09-11T23:03:11.598 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "device_cache_timeout", 2024-09-11T23:03:11.598 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.598 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.598 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.598 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "1800", 2024-09-11T23:03:11.598 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.598 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.598 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.598 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "seconds to cache device inventory", 2024-09-11T23:03:11.598 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.598 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.598 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.599 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.599 INFO:teuthology.orchestra.run.smithi152.stdout: "device_enhanced_scan": { 2024-09-11T23:03:11.599 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "device_enhanced_scan", 2024-09-11T23:03:11.599 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.599 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.599 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.599 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.599 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.599 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.599 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.599 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T23:03:11.599 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.599 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: "facts_cache_timeout": { 2024-09-11T23:03:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "facts_cache_timeout", 2024-09-11T23:03:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-09-11T23:03:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "seconds to cache host facts data", 2024-09-11T23:03:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.601 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.601 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.601 INFO:teuthology.orchestra.run.smithi152.stdout: "grafana_dashboards_path": { 2024-09-11T23:03:11.601 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "grafana_dashboards_path", 2024-09-11T23:03:11.601 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.601 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.601 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.601 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T23:03:11.601 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.601 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.601 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.601 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T23:03:11.601 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.601 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.602 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.602 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.602 INFO:teuthology.orchestra.run.smithi152.stdout: "host_check_interval": { 2024-09-11T23:03:11.602 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "host_check_interval", 2024-09-11T23:03:11.602 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.602 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.602 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.602 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-09-11T23:03:11.602 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.602 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.602 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.602 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to perform a host check", 2024-09-11T23:03:11.602 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.602 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.603 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.603 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.603 INFO:teuthology.orchestra.run.smithi152.stdout: "hw_monitoring": { 2024-09-11T23:03:11.603 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "hw_monitoring", 2024-09-11T23:03:11.603 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.603 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.603 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.603 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.603 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.603 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.603 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.603 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T23:03:11.603 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.603 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.604 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.604 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.604 INFO:teuthology.orchestra.run.smithi152.stdout: "inventory_list_all": { 2024-09-11T23:03:11.604 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "inventory_list_all", 2024-09-11T23:03:11.604 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.604 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.604 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.604 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.604 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.604 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.604 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.604 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T23:03:11.604 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.604 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.605 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.605 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.605 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.605 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.605 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.605 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.605 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.605 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.605 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.605 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.605 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.605 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.605 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.606 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.606 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.606 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.606 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.606 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.606 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.606 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.606 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.606 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.606 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.606 INFO:teuthology.orchestra.run.smithi152.stdout: "log_refresh_metadata": { 2024-09-11T23:03:11.606 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_refresh_metadata", 2024-09-11T23:03:11.606 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.606 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.607 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.607 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.607 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.607 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.607 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.607 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T23:03:11.607 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.607 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.607 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.607 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.607 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.607 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.607 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.608 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.608 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.608 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.608 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.608 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.608 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.608 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T23:03:11.608 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.608 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.608 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.608 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.608 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.608 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.609 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.609 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.609 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.609 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.609 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.609 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.609 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.609 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.609 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.609 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.609 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.609 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.609 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.609 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.610 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.610 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.610 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.610 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.610 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.610 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.610 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.610 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.610 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.610 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.610 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.610 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.610 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.611 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.611 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.611 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.611 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.611 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.611 INFO:teuthology.orchestra.run.smithi152.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T23:03:11.611 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T23:03:11.611 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.611 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.611 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.611 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.611 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.611 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.612 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.612 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T23:03:11.612 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.612 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.612 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.612 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.612 INFO:teuthology.orchestra.run.smithi152.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T23:03:11.612 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T23:03:11.612 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.612 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.612 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.612 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "*", 2024-09-11T23:03:11.612 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.612 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.613 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.613 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T23:03:11.613 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.613 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.613 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.613 INFO:teuthology.orchestra.run.smithi152.stdout: "max_count_per_host": { 2024-09-11T23:03:11.613 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_count_per_host", 2024-09-11T23:03:11.613 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.613 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.613 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.613 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:11.613 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.614 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.614 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.614 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "max number of daemons per service per host", 2024-09-11T23:03:11.614 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.614 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.614 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.614 INFO:teuthology.orchestra.run.smithi152.stdout: "max_osd_draining_count": { 2024-09-11T23:03:11.614 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_osd_draining_count", 2024-09-11T23:03:11.614 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.614 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.614 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.614 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:11.614 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.615 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.615 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.615 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T23:03:11.615 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.615 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.615 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.615 INFO:teuthology.orchestra.run.smithi152.stdout: "migration_current": { 2024-09-11T23:03:11.615 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "migration_current", 2024-09-11T23:03:11.615 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.615 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.615 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.615 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.615 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.616 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.616 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.616 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "internal - do not modify", 2024-09-11T23:03:11.616 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.616 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.616 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.616 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.616 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": { 2024-09-11T23:03:11.616 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mode", 2024-09-11T23:03:11.616 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.616 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.616 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.616 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "root", 2024-09-11T23:03:11.616 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.617 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.617 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.617 INFO:teuthology.orchestra.run.smithi152.stdout: "cephadm-package", 2024-09-11T23:03:11.617 INFO:teuthology.orchestra.run.smithi152.stdout: "root" 2024-09-11T23:03:11.617 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.617 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T23:03:11.617 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.617 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.617 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.617 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.617 INFO:teuthology.orchestra.run.smithi152.stdout: "oob_default_addr": { 2024-09-11T23:03:11.617 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "oob_default_addr", 2024-09-11T23:03:11.617 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.617 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.618 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.618 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "169.254.1.1", 2024-09-11T23:03:11.618 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.618 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.618 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.618 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T23:03:11.618 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.618 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.618 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.618 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.618 INFO:teuthology.orchestra.run.smithi152.stdout: "prometheus_alerts_path": { 2024-09-11T23:03:11.618 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "prometheus_alerts_path", 2024-09-11T23:03:11.618 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.618 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.619 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.619 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T23:03:11.619 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.619 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.619 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.619 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T23:03:11.619 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.619 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.619 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.619 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.619 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_insecure": { 2024-09-11T23:03:11.619 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_insecure", 2024-09-11T23:03:11.619 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.619 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.620 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.620 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.620 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.620 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.620 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.620 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T23:03:11.620 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.620 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.620 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.620 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.620 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_password": { 2024-09-11T23:03:11.620 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_password", 2024-09-11T23:03:11.621 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.621 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.621 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.621 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.621 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.621 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.621 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.621 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T23:03:11.621 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.621 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.621 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.621 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.621 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_url": { 2024-09-11T23:03:11.622 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_url", 2024-09-11T23:03:11.622 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.622 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.622 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.622 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.622 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.622 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.622 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.622 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T23:03:11.622 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.622 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.622 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.622 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_username": { 2024-09-11T23:03:11.623 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_username", 2024-09-11T23:03:11.623 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.623 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.623 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.623 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.623 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.623 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.623 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.623 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T23:03:11.623 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.623 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.623 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.623 INFO:teuthology.orchestra.run.smithi152.stdout: "secure_monitoring_stack": { 2024-09-11T23:03:11.624 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "secure_monitoring_stack", 2024-09-11T23:03:11.624 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.624 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.624 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.624 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.624 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.624 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.624 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.624 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T23:03:11.624 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.624 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.624 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.625 INFO:teuthology.orchestra.run.smithi152.stdout: "service_discovery_port": { 2024-09-11T23:03:11.625 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "service_discovery_port", 2024-09-11T23:03:11.625 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.625 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.625 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.625 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8765", 2024-09-11T23:03:11.625 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.625 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.625 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.625 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "cephadm service discovery port", 2024-09-11T23:03:11.625 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.625 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.625 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.626 INFO:teuthology.orchestra.run.smithi152.stdout: "ssh_config_file": { 2024-09-11T23:03:11.626 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssh_config_file", 2024-09-11T23:03:11.626 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.626 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.626 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.626 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.626 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.626 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.626 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.626 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T23:03:11.626 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.626 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.626 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.627 INFO:teuthology.orchestra.run.smithi152.stdout: "use_agent": { 2024-09-11T23:03:11.627 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "use_agent", 2024-09-11T23:03:11.627 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.627 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.627 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.627 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.627 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.627 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.627 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.627 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T23:03:11.627 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.627 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.627 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.628 INFO:teuthology.orchestra.run.smithi152.stdout: "use_repo_digest": { 2024-09-11T23:03:11.628 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "use_repo_digest", 2024-09-11T23:03:11.628 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.628 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.628 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.628 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.628 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.628 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.628 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.628 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T23:03:11.628 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.628 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.629 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.629 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_on_failed_host_check": { 2024-09-11T23:03:11.629 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_on_failed_host_check", 2024-09-11T23:03:11.629 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.629 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.629 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.629 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.629 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.629 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.629 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.629 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T23:03:11.629 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.629 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.630 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.630 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_on_stray_daemons": { 2024-09-11T23:03:11.630 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_on_stray_daemons", 2024-09-11T23:03:11.630 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.630 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.630 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.630 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.630 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.630 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.630 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.630 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T23:03:11.630 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.630 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.631 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.631 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_on_stray_hosts": { 2024-09-11T23:03:11.631 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_on_stray_hosts", 2024-09-11T23:03:11.631 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.631 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.631 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.631 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.631 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.631 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.631 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.631 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T23:03:11.631 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.631 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.632 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.632 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.632 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.632 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crash", 2024-09-11T23:03:11.632 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.632 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.632 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.632 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.632 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.632 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.632 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.632 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.633 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.633 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.633 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.633 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.633 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.633 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.633 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.633 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.633 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.633 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.633 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.633 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.634 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.634 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.634 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.634 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.634 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.634 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.634 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.634 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.634 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.634 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.634 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.634 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.634 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.635 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.635 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.635 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.635 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.635 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.635 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.635 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.635 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.635 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.635 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.635 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.635 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.635 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.636 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.636 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.636 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.636 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.636 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.636 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.636 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.636 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.636 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.636 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.636 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.636 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.637 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.637 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.637 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.637 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.637 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.637 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.637 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.637 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.637 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.637 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.637 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.637 INFO:teuthology.orchestra.run.smithi152.stdout: "retain_interval": { 2024-09-11T23:03:11.637 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "retain_interval", 2024-09-11T23:03:11.638 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.638 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.638 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.638 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "31536000", 2024-09-11T23:03:11.638 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.638 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.638 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.638 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T23:03:11.638 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.638 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.638 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.638 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_recent_interval": { 2024-09-11T23:03:11.639 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_recent_interval", 2024-09-11T23:03:11.639 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.639 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.639 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.639 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "1209600", 2024-09-11T23:03:11.639 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.639 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.639 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.639 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T23:03:11.639 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.639 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.639 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.639 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.639 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.640 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "dashboard", 2024-09-11T23:03:11.640 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.640 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.640 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.640 INFO:teuthology.orchestra.run.smithi152.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T23:03:11.640 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T23:03:11.640 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.640 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.640 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.640 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:11.640 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.640 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.641 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.641 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.641 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.641 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.641 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.641 INFO:teuthology.orchestra.run.smithi152.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T23:03:11.641 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T23:03:11.641 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.641 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.641 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.641 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.641 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T23:03:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T23:03:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.643 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.643 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.643 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.643 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.643 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.643 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.643 INFO:teuthology.orchestra.run.smithi152.stdout: "AUDIT_API_ENABLED": { 2024-09-11T23:03:11.643 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T23:03:11.643 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.643 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.643 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.643 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.643 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.644 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.644 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.644 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.644 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.644 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.644 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.644 INFO:teuthology.orchestra.run.smithi152.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T23:03:11.644 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T23:03:11.644 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.644 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.644 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.644 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.645 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.645 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.645 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.645 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.645 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.645 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.645 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.645 INFO:teuthology.orchestra.run.smithi152.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T23:03:11.645 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T23:03:11.645 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.645 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.645 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.645 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.646 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.646 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.646 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.646 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.646 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.646 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.646 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.646 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T23:03:11.646 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T23:03:11.646 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.646 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.646 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.646 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.647 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.647 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.647 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.647 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.647 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.647 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.647 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.647 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T23:03:11.647 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T23:03:11.647 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.647 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.648 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.648 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.648 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.648 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.648 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.648 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.648 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.648 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.648 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.648 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T23:03:11.648 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T23:03:11.648 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.648 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.649 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.649 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.649 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.649 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.649 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.649 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.649 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.649 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.649 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.649 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.649 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T23:03:11.649 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T23:03:11.649 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.650 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.650 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.650 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.650 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.650 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.650 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.650 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.650 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.650 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.650 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.650 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T23:03:11.650 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T23:03:11.650 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.651 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.651 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.651 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.651 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.651 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.651 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.651 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.651 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.651 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.651 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.651 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T23:03:11.651 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T23:03:11.651 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.652 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.652 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.652 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.652 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.652 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.652 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.652 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.652 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.652 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.652 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.652 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T23:03:11.652 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T23:03:11.653 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.653 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.653 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.653 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.653 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.653 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.653 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.653 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.653 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.653 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.653 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.653 INFO:teuthology.orchestra.run.smithi152.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T23:03:11.653 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T23:03:11.654 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.654 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.654 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.654 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.654 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.654 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.654 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.654 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.654 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.654 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.654 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.654 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T23:03:11.654 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T23:03:11.655 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.655 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.655 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.655 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "admin", 2024-09-11T23:03:11.655 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.655 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.655 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.655 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.655 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.655 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.655 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.655 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T23:03:11.655 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T23:03:11.656 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.656 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.656 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.656 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.656 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.656 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.656 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.656 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.656 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.656 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.656 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.656 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_URL": { 2024-09-11T23:03:11.656 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_URL", 2024-09-11T23:03:11.657 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.657 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.657 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.657 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.657 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.657 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.657 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.657 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.657 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.657 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.657 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.657 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T23:03:11.658 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T23:03:11.658 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.658 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.658 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.658 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "admin", 2024-09-11T23:03:11.658 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.658 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.658 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.658 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.658 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.658 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.658 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.658 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T23:03:11.658 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T23:03:11.659 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.659 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.659 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.659 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.659 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.659 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.659 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.659 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.659 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.659 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.659 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.659 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T23:03:11.659 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T23:03:11.660 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.660 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.660 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.660 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.660 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.660 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.660 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.660 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.660 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.660 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.660 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.661 INFO:teuthology.orchestra.run.smithi152.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T23:03:11.661 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T23:03:11.661 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.661 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.661 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.661 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.661 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.661 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.661 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.662 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.662 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.662 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.662 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.662 INFO:teuthology.orchestra.run.smithi152.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T23:03:11.662 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T23:03:11.662 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.662 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.662 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.662 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.662 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.662 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.662 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.663 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.663 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.663 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.663 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.663 INFO:teuthology.orchestra.run.smithi152.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T23:03:11.663 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T23:03:11.663 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.663 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.663 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.663 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.663 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.663 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.663 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.664 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.664 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.664 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.664 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.664 INFO:teuthology.orchestra.run.smithi152.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T23:03:11.664 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T23:03:11.664 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.664 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.664 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.664 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.664 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.664 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.665 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.665 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.665 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.665 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.665 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.665 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T23:03:11.665 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T23:03:11.665 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.665 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.665 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.665 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.665 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.666 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.666 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.666 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.666 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.666 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.666 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.666 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T23:03:11.666 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T23:03:11.666 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.666 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.666 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.666 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.666 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.667 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.667 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.667 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.667 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.667 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.667 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.667 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T23:03:11.667 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T23:03:11.667 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.667 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.667 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.667 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.667 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.668 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.668 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.668 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.668 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.668 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.668 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.668 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T23:03:11.668 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T23:03:11.668 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.668 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.668 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.668 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.669 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.669 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.669 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.669 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.669 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.669 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.669 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.669 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T23:03:11.669 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T23:03:11.669 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.669 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.669 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.669 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.670 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.670 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.670 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.670 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.670 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.670 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.670 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.670 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T23:03:11.670 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T23:03:11.670 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.670 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.670 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.670 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.671 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.671 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.671 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.671 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.671 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.671 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.671 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.671 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T23:03:11.671 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T23:03:11.671 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.671 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.671 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.671 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.672 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.672 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.672 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.672 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.672 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.672 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.672 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.672 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T23:03:11.672 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T23:03:11.672 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.672 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.672 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.673 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.673 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.673 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.673 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.673 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.673 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.673 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.673 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.673 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T23:03:11.673 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T23:03:11.673 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.673 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.674 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.674 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T23:03:11.674 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.674 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.674 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.674 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.674 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.674 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.674 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.674 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T23:03:11.674 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T23:03:11.675 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.675 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.675 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.675 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:11.675 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.675 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.675 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.675 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.675 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.675 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.675 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.675 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T23:03:11.675 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T23:03:11.676 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.676 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.676 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.676 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8", 2024-09-11T23:03:11.676 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.676 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.676 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.676 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.676 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.676 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.676 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.676 INFO:teuthology.orchestra.run.smithi152.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T23:03:11.677 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T23:03:11.677 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.677 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.677 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.677 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "45", 2024-09-11T23:03:11.677 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.677 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.677 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.677 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.677 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.677 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.677 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.677 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T23:03:11.678 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T23:03:11.678 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.678 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.678 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.678 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.678 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.678 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.678 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.678 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.678 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.678 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.678 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.679 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T23:03:11.679 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T23:03:11.679 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.679 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.679 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.679 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "admin", 2024-09-11T23:03:11.679 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.679 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.679 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.679 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.679 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.679 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.679 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.680 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T23:03:11.680 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T23:03:11.680 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.680 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.680 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.680 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.680 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.680 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.680 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.680 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.680 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.680 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.680 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.681 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T23:03:11.681 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T23:03:11.681 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.681 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.681 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.681 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.681 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.681 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.681 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.681 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.681 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.681 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.681 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.682 INFO:teuthology.orchestra.run.smithi152.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T23:03:11.682 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T23:03:11.682 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.682 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.682 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.682 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-09-11T23:03:11.682 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.682 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.682 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.682 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.682 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.682 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.682 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.683 INFO:teuthology.orchestra.run.smithi152.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T23:03:11.683 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T23:03:11.683 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.683 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.683 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.683 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:11.683 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.683 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.683 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.683 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.683 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.683 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.683 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.684 INFO:teuthology.orchestra.run.smithi152.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T23:03:11.684 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T23:03:11.684 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.684 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.684 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.684 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-09-11T23:03:11.684 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.684 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.684 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.684 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.684 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.684 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.684 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.685 INFO:teuthology.orchestra.run.smithi152.stdout: "cross_origin_url": { 2024-09-11T23:03:11.685 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cross_origin_url", 2024-09-11T23:03:11.685 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.685 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.685 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.685 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.685 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.685 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.685 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.685 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.685 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.685 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.685 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.686 INFO:teuthology.orchestra.run.smithi152.stdout: "crt_file": { 2024-09-11T23:03:11.686 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crt_file", 2024-09-11T23:03:11.686 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.686 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.686 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.686 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.686 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.686 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.686 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.686 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.686 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.686 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.686 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.687 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.687 INFO:teuthology.orchestra.run.smithi152.stdout: "debug": { 2024-09-11T23:03:11.687 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "debug", 2024-09-11T23:03:11.687 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.687 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.687 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.687 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.687 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.687 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.687 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.687 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Enable/disable debug options", 2024-09-11T23:03:11.687 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.687 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.687 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.688 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.688 INFO:teuthology.orchestra.run.smithi152.stdout: "jwt_token_ttl": { 2024-09-11T23:03:11.688 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "jwt_token_ttl", 2024-09-11T23:03:11.688 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.688 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.688 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.688 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "28800", 2024-09-11T23:03:11.688 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.688 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.688 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.688 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.688 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.689 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.689 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.689 INFO:teuthology.orchestra.run.smithi152.stdout: "key_file": { 2024-09-11T23:03:11.689 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "key_file", 2024-09-11T23:03:11.689 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.689 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.689 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.689 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.689 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.689 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.689 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.689 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.689 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.690 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.690 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.690 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.690 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.690 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.690 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.690 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.690 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.690 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.690 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.690 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.690 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.691 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.691 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.691 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.691 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.691 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.691 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.691 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.691 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.691 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.691 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.691 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.691 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.691 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.692 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.692 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.692 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.692 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.692 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.692 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.692 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.692 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.692 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.692 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.692 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.692 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.693 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.693 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.693 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.693 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.693 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.693 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.693 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.693 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.693 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.693 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.693 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.693 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.693 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.693 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.694 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.694 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.694 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.694 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.694 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.694 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.694 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.694 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.694 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.694 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.694 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.694 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.694 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.695 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.695 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.695 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.695 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.695 INFO:teuthology.orchestra.run.smithi152.stdout: "motd": { 2024-09-11T23:03:11.695 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "motd", 2024-09-11T23:03:11.695 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.695 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.695 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.695 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.695 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.695 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.695 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.696 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "The message of the day", 2024-09-11T23:03:11.696 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.696 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.696 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.696 INFO:teuthology.orchestra.run.smithi152.stdout: "redirect_resolve_ip_addr": { 2024-09-11T23:03:11.696 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T23:03:11.696 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.696 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.696 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.696 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.696 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.696 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.697 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.697 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.697 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.697 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.697 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.697 INFO:teuthology.orchestra.run.smithi152.stdout: "server_addr": { 2024-09-11T23:03:11.697 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_addr", 2024-09-11T23:03:11.697 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.697 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.697 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.697 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "::", 2024-09-11T23:03:11.697 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.697 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.698 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.698 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.698 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.698 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.698 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.698 INFO:teuthology.orchestra.run.smithi152.stdout: "server_port": { 2024-09-11T23:03:11.698 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_port", 2024-09-11T23:03:11.698 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.698 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.698 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.698 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8080", 2024-09-11T23:03:11.698 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.698 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.698 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.699 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.699 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.699 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.699 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.699 INFO:teuthology.orchestra.run.smithi152.stdout: "ssl": { 2024-09-11T23:03:11.699 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssl", 2024-09-11T23:03:11.699 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.699 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.699 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.699 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.699 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.699 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.699 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.700 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.700 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.700 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.700 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.700 INFO:teuthology.orchestra.run.smithi152.stdout: "ssl_server_port": { 2024-09-11T23:03:11.700 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssl_server_port", 2024-09-11T23:03:11.700 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.700 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.700 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.700 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8443", 2024-09-11T23:03:11.700 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.700 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.701 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.701 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.701 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.701 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.701 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.701 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.701 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_behaviour": { 2024-09-11T23:03:11.701 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_behaviour", 2024-09-11T23:03:11.701 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.701 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.701 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.701 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "redirect", 2024-09-11T23:03:11.701 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.701 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.702 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.702 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.702 INFO:teuthology.orchestra.run.smithi152.stdout: "redirect" 2024-09-11T23:03:11.702 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.702 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.702 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.702 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.702 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.702 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.702 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_error_status_code": { 2024-09-11T23:03:11.702 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_error_status_code", 2024-09-11T23:03:11.702 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.702 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.703 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.703 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "500", 2024-09-11T23:03:11.703 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "400", 2024-09-11T23:03:11.703 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "599", 2024-09-11T23:03:11.703 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.703 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.703 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.703 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.703 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.703 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.703 INFO:teuthology.orchestra.run.smithi152.stdout: "url_prefix": { 2024-09-11T23:03:11.703 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "url_prefix", 2024-09-11T23:03:11.703 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.703 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.704 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.704 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.704 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.704 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.704 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.704 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.704 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.704 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.704 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.704 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.704 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.705 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "devicehealth", 2024-09-11T23:03:11.705 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.705 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.705 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.705 INFO:teuthology.orchestra.run.smithi152.stdout: "enable_monitoring": { 2024-09-11T23:03:11.705 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enable_monitoring", 2024-09-11T23:03:11.705 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.705 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.705 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.705 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.705 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.705 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.705 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.705 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "monitor device health metrics", 2024-09-11T23:03:11.706 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.706 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.706 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.706 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.706 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.706 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.706 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.706 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.706 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.706 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.706 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.706 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.707 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.707 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.707 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.707 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.707 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.707 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.707 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.707 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.707 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.707 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.707 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.707 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.707 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.708 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.708 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.708 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.708 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.708 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.708 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.708 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.708 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.708 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.708 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.708 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.708 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.709 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.709 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.709 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.709 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.709 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.709 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.709 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.709 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.709 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.709 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.709 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.709 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.709 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.709 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.710 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.710 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.710 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.710 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.710 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.710 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.710 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.710 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.710 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.710 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.710 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.710 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.710 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.711 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.711 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.711 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.711 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.711 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.711 INFO:teuthology.orchestra.run.smithi152.stdout: "mark_out_threshold": { 2024-09-11T23:03:11.711 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mark_out_threshold", 2024-09-11T23:03:11.711 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.711 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.711 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.711 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "2419200", 2024-09-11T23:03:11.711 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.711 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.712 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.712 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T23:03:11.712 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.712 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.712 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.712 INFO:teuthology.orchestra.run.smithi152.stdout: "pool_name": { 2024-09-11T23:03:11.712 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pool_name", 2024-09-11T23:03:11.712 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.712 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.712 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.712 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "device_health_metrics", 2024-09-11T23:03:11.712 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.712 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.713 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.713 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T23:03:11.713 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.713 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.713 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.713 INFO:teuthology.orchestra.run.smithi152.stdout: "retention_period": { 2024-09-11T23:03:11.713 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "retention_period", 2024-09-11T23:03:11.713 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.713 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.713 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.713 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "15552000", 2024-09-11T23:03:11.713 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.714 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.714 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.714 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how long to retain device health metrics", 2024-09-11T23:03:11.714 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.714 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.714 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.714 INFO:teuthology.orchestra.run.smithi152.stdout: "scrape_frequency": { 2024-09-11T23:03:11.714 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "scrape_frequency", 2024-09-11T23:03:11.714 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.714 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.714 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.714 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "86400", 2024-09-11T23:03:11.714 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.715 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.715 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.715 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T23:03:11.715 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.715 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.715 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.715 INFO:teuthology.orchestra.run.smithi152.stdout: "self_heal": { 2024-09-11T23:03:11.715 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "self_heal", 2024-09-11T23:03:11.715 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.715 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.715 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.715 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.716 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.716 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.716 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.716 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T23:03:11.716 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.716 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.716 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.716 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-09-11T23:03:11.716 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-09-11T23:03:11.716 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.716 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.716 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.717 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-09-11T23:03:11.717 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.717 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.717 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.717 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T23:03:11.717 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.717 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.717 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.717 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_threshold": { 2024-09-11T23:03:11.717 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_threshold", 2024-09-11T23:03:11.717 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.717 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.717 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.718 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "7257600", 2024-09-11T23:03:11.718 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.718 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.718 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.718 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T23:03:11.718 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.718 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.718 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.718 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.718 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.718 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "diskprediction_local", 2024-09-11T23:03:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.720 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.720 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.720 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.720 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.720 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.720 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.720 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.720 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.720 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.720 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.720 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.720 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.721 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.721 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.721 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.721 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.721 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.721 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.721 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.721 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.721 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.721 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.721 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.721 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.721 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.722 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.723 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.723 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.723 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.723 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.723 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.723 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.723 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.723 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.723 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.723 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.723 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.723 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.724 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.724 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.724 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.724 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.724 INFO:teuthology.orchestra.run.smithi152.stdout: "predict_interval": { 2024-09-11T23:03:11.724 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "predict_interval", 2024-09-11T23:03:11.724 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.724 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.724 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.724 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "86400", 2024-09-11T23:03:11.724 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.724 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.724 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.725 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.725 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.725 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.725 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.725 INFO:teuthology.orchestra.run.smithi152.stdout: "predictor_model": { 2024-09-11T23:03:11.725 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "predictor_model", 2024-09-11T23:03:11.725 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.725 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.725 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.725 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "prophetstor", 2024-09-11T23:03:11.725 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.725 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.726 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.726 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.726 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.726 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.726 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.726 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-09-11T23:03:11.726 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-09-11T23:03:11.726 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.726 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.726 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.726 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-09-11T23:03:11.726 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.726 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.726 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.727 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.727 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.727 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.727 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.727 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.727 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.727 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "influx", 2024-09-11T23:03:11.727 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": false, 2024-09-11T23:03:11.727 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "influxdb python module not found", 2024-09-11T23:03:11.727 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.727 INFO:teuthology.orchestra.run.smithi152.stdout: "batch_size": { 2024-09-11T23:03:11.727 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "batch_size", 2024-09-11T23:03:11.728 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.728 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.728 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.728 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5000", 2024-09-11T23:03:11.728 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.728 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.728 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.728 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T23:03:11.728 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.728 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.728 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.728 INFO:teuthology.orchestra.run.smithi152.stdout: "database": { 2024-09-11T23:03:11.729 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "database", 2024-09-11T23:03:11.729 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.729 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.729 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.729 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "ceph", 2024-09-11T23:03:11.729 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.729 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.729 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.729 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T23:03:11.729 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.729 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.729 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.730 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": { 2024-09-11T23:03:11.730 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "hostname", 2024-09-11T23:03:11.730 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.730 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.730 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.730 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.730 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.730 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.730 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.730 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "InfluxDB server hostname", 2024-09-11T23:03:11.730 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.730 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.730 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.731 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-09-11T23:03:11.731 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-09-11T23:03:11.731 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.731 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.731 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.731 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "30", 2024-09-11T23:03:11.731 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "5", 2024-09-11T23:03:11.731 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.731 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.731 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T23:03:11.731 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.731 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.731 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.732 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.732 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.732 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.732 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.732 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.732 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.732 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.732 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.732 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.732 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.732 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.732 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.733 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.733 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.733 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.733 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.733 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.733 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.733 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.733 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.733 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.733 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.733 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.733 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.734 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.734 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.734 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.734 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.734 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.734 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.734 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.734 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.734 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.734 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.734 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.734 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.734 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.735 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.735 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.735 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.735 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.735 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.735 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.735 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.735 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.735 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.735 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.735 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.735 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.735 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.735 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.736 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.736 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.736 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.736 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.736 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.736 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.736 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.736 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.736 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.736 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.736 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.736 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.737 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.737 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.737 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.737 INFO:teuthology.orchestra.run.smithi152.stdout: "password": { 2024-09-11T23:03:11.737 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "password", 2024-09-11T23:03:11.737 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.737 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.737 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.737 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.737 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.737 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.737 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.737 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "password of InfluxDB server user", 2024-09-11T23:03:11.738 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.738 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.738 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.738 INFO:teuthology.orchestra.run.smithi152.stdout: "port": { 2024-09-11T23:03:11.738 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "port", 2024-09-11T23:03:11.738 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.738 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.738 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.738 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8086", 2024-09-11T23:03:11.738 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.738 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.738 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.738 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "InfluxDB server port", 2024-09-11T23:03:11.739 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.739 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.739 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.739 INFO:teuthology.orchestra.run.smithi152.stdout: "ssl": { 2024-09-11T23:03:11.739 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssl", 2024-09-11T23:03:11.739 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.739 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.739 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.739 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "false", 2024-09-11T23:03:11.739 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.739 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.739 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.739 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:03:11.740 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.740 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.740 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.740 INFO:teuthology.orchestra.run.smithi152.stdout: "threads": { 2024-09-11T23:03:11.740 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "threads", 2024-09-11T23:03:11.740 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.740 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.740 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.740 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-09-11T23:03:11.740 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-09-11T23:03:11.740 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "32", 2024-09-11T23:03:11.740 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.740 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T23:03:11.741 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.741 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.741 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.741 INFO:teuthology.orchestra.run.smithi152.stdout: "username": { 2024-09-11T23:03:11.741 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "username", 2024-09-11T23:03:11.741 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.741 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.741 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.741 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.741 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.741 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.741 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.741 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "username of InfluxDB server user", 2024-09-11T23:03:11.742 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.742 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.742 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.742 INFO:teuthology.orchestra.run.smithi152.stdout: "verify_ssl": { 2024-09-11T23:03:11.742 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "verify_ssl", 2024-09-11T23:03:11.742 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.742 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.742 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.742 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "true", 2024-09-11T23:03:11.744 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.744 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.744 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.744 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:03:11.744 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.744 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.744 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.745 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.745 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.745 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "insights", 2024-09-11T23:03:11.745 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.745 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.745 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.745 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.745 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.745 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.745 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.745 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.745 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.746 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.746 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.746 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.746 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.746 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.746 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.746 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.746 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.746 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.746 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.746 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.746 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.746 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.747 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.747 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.747 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.747 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.747 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.747 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.747 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.747 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.747 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.747 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.747 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.747 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.747 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.748 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.748 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.748 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.748 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.748 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.748 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.748 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.748 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.748 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.748 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.748 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.748 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.748 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.749 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.749 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.749 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.749 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.749 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.749 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.749 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.749 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.749 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.749 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.749 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.749 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.750 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.750 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.750 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.750 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.750 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.750 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.750 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.750 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.750 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.750 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.750 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.750 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "iostat", 2024-09-11T23:03:11.751 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.751 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.751 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.751 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.751 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.751 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.751 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.751 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.751 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.751 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.751 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.751 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.751 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.751 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.752 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.752 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.752 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.752 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.752 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.752 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.752 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.752 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.752 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.752 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.752 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.752 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.753 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.753 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.753 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.753 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.753 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.753 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.753 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.753 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.753 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.753 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.753 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.753 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.753 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.754 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.754 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.754 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.754 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.754 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.754 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.754 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.754 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.754 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.754 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.754 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.754 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.754 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.754 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.755 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.755 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.755 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.755 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.755 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.755 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.755 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.755 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.755 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.755 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.755 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.755 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.755 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.756 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.756 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.756 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.756 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.756 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.756 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "k8sevents", 2024-09-11T23:03:11.756 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.756 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.756 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.756 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_event_retention_days": { 2024-09-11T23:03:11.756 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ceph_event_retention_days", 2024-09-11T23:03:11.756 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.757 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.757 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.757 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "7", 2024-09-11T23:03:11.757 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.757 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.757 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.757 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T23:03:11.757 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.757 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.757 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.757 INFO:teuthology.orchestra.run.smithi152.stdout: "config_check_secs": { 2024-09-11T23:03:11.757 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "config_check_secs", 2024-09-11T23:03:11.758 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.758 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.758 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.758 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:11.758 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "10", 2024-09-11T23:03:11.758 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.758 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.758 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T23:03:11.758 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.758 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.758 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.758 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.758 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.759 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.759 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.759 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.759 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.759 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.759 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.759 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.759 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.759 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.759 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.759 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.759 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.759 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.760 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.760 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.760 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.760 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.760 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.760 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.760 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.760 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.760 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.760 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.760 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.760 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.761 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.761 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.761 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.761 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.761 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.761 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.761 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.761 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.762 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.762 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.762 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.762 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.762 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.762 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.762 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.762 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.762 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.762 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.762 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.762 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.762 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.763 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.763 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.763 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.763 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.763 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.763 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.763 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.763 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.763 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.763 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.763 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.763 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.763 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.764 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.764 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.764 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.764 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.764 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.764 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.764 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.764 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "localpool", 2024-09-11T23:03:11.764 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.764 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.764 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.765 INFO:teuthology.orchestra.run.smithi152.stdout: "failure_domain": { 2024-09-11T23:03:11.765 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "failure_domain", 2024-09-11T23:03:11.765 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.765 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.765 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.765 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "host", 2024-09-11T23:03:11.765 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.765 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.765 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.765 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "failure domain for any created local pool", 2024-09-11T23:03:11.765 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T23:03:11.765 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.765 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.766 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.766 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.766 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.766 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.766 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.766 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.766 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.766 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.766 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.766 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.766 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.766 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.766 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.766 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.767 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.767 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.767 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.767 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.767 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.767 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.767 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.767 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.767 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.767 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.767 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.767 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.768 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.768 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.768 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.768 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.768 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.768 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.768 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.768 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.768 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.768 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.768 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.768 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.768 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.769 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.769 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.769 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.769 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.769 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.769 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.769 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.769 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.769 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.769 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.769 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.769 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.769 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.769 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.770 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.770 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.770 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.770 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.770 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.770 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.770 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.770 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.770 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.770 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.770 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.770 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.770 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.771 INFO:teuthology.orchestra.run.smithi152.stdout: "min_size": { 2024-09-11T23:03:11.771 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "min_size", 2024-09-11T23:03:11.771 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.771 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.771 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.771 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.771 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.771 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.771 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.771 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "default min_size for any created local pool", 2024-09-11T23:03:11.771 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T23:03:11.771 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.771 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.772 INFO:teuthology.orchestra.run.smithi152.stdout: "num_rep": { 2024-09-11T23:03:11.772 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "num_rep", 2024-09-11T23:03:11.772 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.772 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.772 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.772 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "3", 2024-09-11T23:03:11.772 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.772 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.772 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.772 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "default replica count for any created local pool", 2024-09-11T23:03:11.772 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.772 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.773 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.773 INFO:teuthology.orchestra.run.smithi152.stdout: "pg_num": { 2024-09-11T23:03:11.773 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pg_num", 2024-09-11T23:03:11.773 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.773 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.773 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.773 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "128", 2024-09-11T23:03:11.773 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.773 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.774 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.774 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T23:03:11.774 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.774 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.774 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.774 INFO:teuthology.orchestra.run.smithi152.stdout: "prefix": { 2024-09-11T23:03:11.774 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "prefix", 2024-09-11T23:03:11.774 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.774 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.774 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.774 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.774 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.775 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.775 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.775 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "name prefix for any created local pool", 2024-09-11T23:03:11.775 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.775 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.775 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.775 INFO:teuthology.orchestra.run.smithi152.stdout: "subtree": { 2024-09-11T23:03:11.775 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "subtree", 2024-09-11T23:03:11.775 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.775 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.775 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.775 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "rack", 2024-09-11T23:03:11.776 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.776 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.776 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.776 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T23:03:11.776 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T23:03:11.776 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.776 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.776 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.776 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.776 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mds_autoscaler", 2024-09-11T23:03:11.776 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mirroring", 2024-09-11T23:03:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "nfs", 2024-09-11T23:03:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "orchestrator", 2024-09-11T23:03:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "fail_fs": { 2024-09-11T23:03:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "fail_fs", 2024-09-11T23:03:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T23:03:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "orchestrator": { 2024-09-11T23:03:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "orchestrator", 2024-09-11T23:03:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "cephadm", 2024-09-11T23:03:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "rook", 2024-09-11T23:03:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "test_orchestrator" 2024-09-11T23:03:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Orchestrator backend", 2024-09-11T23:03:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "osd_perf_query", 2024-09-11T23:03:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "osd_support", 2024-09-11T23:03:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pg_autoscaler", 2024-09-11T23:03:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-09-11T23:03:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-09-11T23:03:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-09-11T23:03:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "threshold": { 2024-09-11T23:03:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "threshold", 2024-09-11T23:03:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "3.0", 2024-09-11T23:03:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1.0", 2024-09-11T23:03:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "scaling threshold", 2024-09-11T23:03:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T23:03:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "progress", 2024-09-11T23:03:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "allow_pg_recovery_event": { 2024-09-11T23:03:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "allow_pg_recovery_event", 2024-09-11T23:03:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T23:03:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "enabled": { 2024-09-11T23:03:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enabled", 2024-09-11T23:03:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "max_completed_events": { 2024-09-11T23:03:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_completed_events", 2024-09-11T23:03:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "50", 2024-09-11T23:03:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "number of past completed events to remember", 2024-09-11T23:03:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-09-11T23:03:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-09-11T23:03:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-09-11T23:03:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how long the module is going to sleep", 2024-09-11T23:03:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "prometheus", 2024-09-11T23:03:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "cache": { 2024-09-11T23:03:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cache", 2024-09-11T23:03:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "exclude_perf_counters": { 2024-09-11T23:03:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "exclude_perf_counters", 2024-09-11T23:03:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T23:03:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T23:03:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_stats_pools": { 2024-09-11T23:03:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_stats_pools", 2024-09-11T23:03:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T23:03:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T23:03:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "300", 2024-09-11T23:03:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "scrape_interval": { 2024-09-11T23:03:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "scrape_interval", 2024-09-11T23:03:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "15.0", 2024-09-11T23:03:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "server_addr": { 2024-09-11T23:03:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_addr", 2024-09-11T23:03:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "::", 2024-09-11T23:03:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T23:03:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "server_port": { 2024-09-11T23:03:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_port", 2024-09-11T23:03:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "9283", 2024-09-11T23:03:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T23:03:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "stale_cache_strategy": { 2024-09-11T23:03:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "stale_cache_strategy", 2024-09-11T23:03:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "log", 2024-09-11T23:03:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_behaviour": { 2024-09-11T23:03:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_behaviour", 2024-09-11T23:03:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "default", 2024-09-11T23:03:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "default", 2024-09-11T23:03:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "error" 2024-09-11T23:03:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_error_status_code": { 2024-09-11T23:03:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_error_status_code", 2024-09-11T23:03:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "500", 2024-09-11T23:03:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "400", 2024-09-11T23:03:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "599", 2024-09-11T23:03:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_support", 2024-09-11T23:03:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "max_concurrent_snap_create": { 2024-09-11T23:03:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_concurrent_snap_create", 2024-09-11T23:03:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "mirror_snapshot_schedule": { 2024-09-11T23:03:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T23:03:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "trash_purge_schedule": { 2024-09-11T23:03:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "trash_purge_schedule", 2024-09-11T23:03:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "restful", 2024-09-11T23:03:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "enable_auth": { 2024-09-11T23:03:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enable_auth", 2024-09-11T23:03:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: "key_file": { 2024-09-11T23:03:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "key_file", 2024-09-11T23:03:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.858 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.858 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.858 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.858 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.858 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.858 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.858 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.858 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.858 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.858 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.858 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.858 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.859 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.859 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.859 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.859 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.859 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.859 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.859 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.859 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.859 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.859 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.859 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.859 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.860 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.860 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.860 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.860 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.860 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.860 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.860 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.860 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.860 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.860 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.860 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.860 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.861 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.861 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.861 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.861 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.861 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.861 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.861 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.861 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.861 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.861 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.861 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.862 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.862 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.862 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.862 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.862 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.862 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.862 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.862 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.862 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.862 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.862 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "server_addr": { 2024-09-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_addr", 2024-09-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "server_port": { 2024-09-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_port", 2024-09-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rgw", 2024-09-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rook", 2024-09-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "storage_class": { 2024-09-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "storage_class", 2024-09-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "local", 2024-09-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "selftest", 2024-09-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "roption1": { 2024-09-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "roption1", 2024-09-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "roption2": { 2024-09-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "roption2", 2024-09-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "xyz", 2024-09-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption1": { 2024-09-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption1", 2024-09-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption2": { 2024-09-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption2", 2024-09-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption3": { 2024-09-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption3", 2024-09-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption4": { 2024-09-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption4", 2024-09-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption5": { 2024-09-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption5", 2024-09-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption6": { 2024-09-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption6", 2024-09-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption7": { 2024-09-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption7", 2024-09-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-09-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "42", 2024-09-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "testkey": { 2024-09-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "testkey", 2024-09-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "testlkey": { 2024-09-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "testlkey", 2024-09-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "testnewline": { 2024-09-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "testnewline", 2024-09-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snap_schedule", 2024-09-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "allow_m_granularity": { 2024-09-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "allow_m_granularity", 2024-09-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "dump_on_update": { 2024-09-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "dump_on_update", 2024-09-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "dump database to debug log on update", 2024-09-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "stats", 2024-09-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "status", 2024-09-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "telegraf", 2024-09-11T23:03:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "address": { 2024-09-11T23:03:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "address", 2024-09-11T23:03:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T23:03:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-09-11T23:03:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-09-11T23:03:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "15", 2024-09-11T23:03:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "telemetry", 2024-09-11T23:03:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_basic": { 2024-09-11T23:03:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_basic", 2024-09-11T23:03:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T23:03:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_crash": { 2024-09-11T23:03:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_crash", 2024-09-11T23:03:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T23:03:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_device": { 2024-09-11T23:03:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_device", 2024-09-11T23:03:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T23:03:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_ident": { 2024-09-11T23:03:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_ident", 2024-09-11T23:03:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T23:03:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_perf": { 2024-09-11T23:03:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_perf", 2024-09-11T23:03:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T23:03:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "contact": { 2024-09-11T23:03:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "contact", 2024-09-11T23:03:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "description": { 2024-09-11T23:03:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "description", 2024-09-11T23:03:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "device_url": { 2024-09-11T23:03:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "device_url", 2024-09-11T23:03:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T23:03:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "enabled": { 2024-09-11T23:03:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enabled", 2024-09-11T23:03:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-09-11T23:03:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-09-11T23:03:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "24", 2024-09-11T23:03:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "8", 2024-09-11T23:03:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "last_opt_revision": { 2024-09-11T23:03:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "last_opt_revision", 2024-09-11T23:03:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "1", 2024-09-11T23:03:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "leaderboard": { 2024-09-11T23:03:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "leaderboard", 2024-09-11T23:03:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "leaderboard_description": { 2024-09-11T23:03:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "leaderboard_description", 2024-09-11T23:03:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "organization": { 2024-09-11T23:03:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "organization", 2024-09-11T23:03:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "proxy": { 2024-09-11T23:03:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "proxy", 2024-09-11T23:03:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: "url": { 2024-09-11T23:03:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "url", 2024-09-11T23:03:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T23:03:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.944 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.944 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.944 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.944 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.944 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.944 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.944 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.944 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "test_orchestrator", 2024-09-11T23:03:11.944 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.944 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.945 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.945 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.945 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.945 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.945 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.945 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.945 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.945 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.945 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.945 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.945 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.945 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.945 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.945 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.946 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.946 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.946 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.946 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.946 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.946 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.946 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.948 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.948 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.948 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.948 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.948 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.948 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.948 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.948 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.948 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.948 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.948 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.948 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.948 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.949 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.949 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.949 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.949 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.949 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.949 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.949 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.949 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.949 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.949 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.949 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.949 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.949 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.950 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.950 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.950 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.950 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.950 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.950 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.950 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.950 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.950 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.950 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.950 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.950 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.950 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.951 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.951 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.951 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.951 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.951 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.951 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.951 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.951 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.951 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.951 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.951 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "volumes", 2024-09-11T23:03:11.951 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.952 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.952 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.952 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.952 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.952 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.952 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.952 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.952 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.952 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.952 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.952 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.952 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.952 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.952 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.953 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.953 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.953 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.953 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.953 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.953 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.953 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.953 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.953 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.953 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.953 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.953 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.953 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.954 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.954 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.954 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.954 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.954 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.954 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.954 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.954 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.954 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.954 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.954 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.954 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.954 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.955 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.955 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.955 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.955 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.955 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.955 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.955 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.955 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.955 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.955 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.955 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.955 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.955 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.955 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.956 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.956 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.956 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.956 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.956 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.956 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.956 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.956 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.956 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.956 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.956 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.956 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.956 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.956 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.957 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.957 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.957 INFO:teuthology.orchestra.run.smithi152.stdout: "max_concurrent_clones": { 2024-09-11T23:03:11.957 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_concurrent_clones", 2024-09-11T23:03:11.957 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.957 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.957 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.957 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "4", 2024-09-11T23:03:11.957 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.957 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.957 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.957 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T23:03:11.957 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.957 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.958 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.958 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.958 INFO:teuthology.orchestra.run.smithi152.stdout: "periodic_async_work": { 2024-09-11T23:03:11.958 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "periodic_async_work", 2024-09-11T23:03:11.958 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.958 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.958 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.958 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.958 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.958 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.958 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.958 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Periodically check for async work", 2024-09-11T23:03:11.958 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.958 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.959 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.959 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot_clone_delay": { 2024-09-11T23:03:11.959 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snapshot_clone_delay", 2024-09-11T23:03:11.959 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.959 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.959 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.959 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-09-11T23:03:11.959 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.959 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.959 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.959 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T23:03:11.959 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.960 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.960 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.960 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot_clone_no_wait": { 2024-09-11T23:03:11.960 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T23:03:11.960 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.960 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.960 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.960 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.960 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.960 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.960 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.960 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T23:03:11.960 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.961 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.961 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.961 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.961 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.961 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.961 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.961 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix", 2024-09-11T23:03:11.961 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.961 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.961 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.961 INFO:teuthology.orchestra.run.smithi152.stdout: "discovery_interval": { 2024-09-11T23:03:11.961 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "discovery_interval", 2024-09-11T23:03:11.961 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-09-11T23:03:11.961 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.962 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.962 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "100", 2024-09-11T23:03:11.962 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.962 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.962 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.962 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.962 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.962 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.962 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.962 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.962 INFO:teuthology.orchestra.run.smithi152.stdout: "identifier": { 2024-09-11T23:03:11.962 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "identifier", 2024-09-11T23:03:11.962 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.963 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.963 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.963 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.963 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.963 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.963 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.963 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.963 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.963 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.963 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.963 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.963 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-09-11T23:03:11.963 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-09-11T23:03:11.963 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.964 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.964 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.964 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-09-11T23:03:11.964 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.964 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.964 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.964 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.964 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.964 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.964 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.964 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.964 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.964 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.965 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.965 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.965 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.965 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.965 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.965 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.965 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.965 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.965 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.965 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.965 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.965 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.965 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.965 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.966 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.966 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.966 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.966 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.966 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.966 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.966 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.966 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.966 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.966 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.966 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.966 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.966 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.967 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.967 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.967 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.967 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.967 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.967 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.967 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.967 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.967 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.967 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.967 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.967 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.967 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.967 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.968 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.968 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.968 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.968 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.968 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.968 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.968 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.968 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.968 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.968 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.968 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.968 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.968 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.969 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.969 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.969 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.969 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.969 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.969 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.969 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.969 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.969 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.969 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.969 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.969 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.969 INFO:teuthology.orchestra.run.smithi152.stdout: "zabbix_host": { 2024-09-11T23:03:11.969 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix_host", 2024-09-11T23:03:11.970 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.970 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.970 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.970 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.970 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.970 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.970 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.970 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.970 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.970 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.970 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.970 INFO:teuthology.orchestra.run.smithi152.stdout: "zabbix_port": { 2024-09-11T23:03:11.970 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix_port", 2024-09-11T23:03:11.971 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.971 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.971 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.971 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10051", 2024-09-11T23:03:11.971 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.971 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.971 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.971 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.971 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.971 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.971 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.971 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.971 INFO:teuthology.orchestra.run.smithi152.stdout: "zabbix_sender": { 2024-09-11T23:03:11.971 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix_sender", 2024-09-11T23:03:11.971 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.972 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.972 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:11.972 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T23:03:11.972 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.972 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.972 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.972 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.972 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.972 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.972 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.972 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.972 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.972 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.972 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-11T23:03:11.973 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.973 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.973 INFO:teuthology.orchestra.run.smithi152.stdout: "modules": [ 2024-09-11T23:03:11.973 INFO:teuthology.orchestra.run.smithi152.stdout: "cephadm", 2024-09-11T23:03:11.973 INFO:teuthology.orchestra.run.smithi152.stdout: "dashboard", 2024-09-11T23:03:11.973 INFO:teuthology.orchestra.run.smithi152.stdout: "iostat", 2024-09-11T23:03:11.973 INFO:teuthology.orchestra.run.smithi152.stdout: "nfs", 2024-09-11T23:03:11.973 INFO:teuthology.orchestra.run.smithi152.stdout: "restful", 2024-09-11T23:03:11.973 INFO:teuthology.orchestra.run.smithi152.stdout: "snap_schedule" 2024-09-11T23:03:11.973 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.973 INFO:teuthology.orchestra.run.smithi152.stdout: "available_modules": [ 2024-09-11T23:03:11.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.973 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "alerts", 2024-09-11T23:03:11.973 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.974 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.974 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.974 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-09-11T23:03:11.974 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-09-11T23:03:11.974 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:11.974 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.974 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.974 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-09-11T23:03:11.974 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.974 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.974 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.975 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T23:03:11.975 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.975 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.975 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.975 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.975 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.975 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.975 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.975 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.975 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.975 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.975 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.975 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.976 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.976 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.976 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.976 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.976 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.976 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.976 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.976 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.976 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.976 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.976 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.976 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.977 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.977 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.977 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.977 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.977 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.977 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.977 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.977 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.977 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.977 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.977 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.977 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.977 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.978 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.978 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.978 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.978 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.978 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:11.978 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.978 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.978 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.978 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.978 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.978 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.978 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.978 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.978 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.979 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.979 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.979 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.979 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.979 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.979 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:11.979 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:11.979 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.979 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.979 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.979 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.979 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.980 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.980 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.980 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.980 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.980 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.980 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.980 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_destination": { 2024-09-11T23:03:11.980 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_destination", 2024-09-11T23:03:11.980 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.980 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.980 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.980 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.980 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.981 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.981 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.981 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Email address to send alerts to", 2024-09-11T23:03:11.981 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.981 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.981 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.981 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_from_name": { 2024-09-11T23:03:11.981 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_from_name", 2024-09-11T23:03:11.981 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.981 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.981 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.981 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "Ceph", 2024-09-11T23:03:11.981 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.982 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.982 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.982 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Email From: name", 2024-09-11T23:03:11.982 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.982 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.982 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.982 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_host": { 2024-09-11T23:03:11.982 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_host", 2024-09-11T23:03:11.982 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.982 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.982 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.982 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.982 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.983 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.983 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.983 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SMTP server", 2024-09-11T23:03:11.983 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.983 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.983 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.983 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_password": { 2024-09-11T23:03:11.983 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_password", 2024-09-11T23:03:11.983 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.983 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.983 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.983 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.983 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.984 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.984 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.984 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Password to authenticate with", 2024-09-11T23:03:11.984 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.984 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.984 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.984 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_port": { 2024-09-11T23:03:11.984 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_port", 2024-09-11T23:03:11.984 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:11.984 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.984 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.984 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "465", 2024-09-11T23:03:11.984 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.985 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.985 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.985 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SMTP port", 2024-09-11T23:03:11.985 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.985 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.985 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.985 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_sender": { 2024-09-11T23:03:11.985 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_sender", 2024-09-11T23:03:11.985 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.985 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.985 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.985 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.985 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.986 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.986 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.986 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SMTP envelope sender", 2024-09-11T23:03:11.986 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.986 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.986 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.986 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_ssl": { 2024-09-11T23:03:11.986 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_ssl", 2024-09-11T23:03:11.986 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.986 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.986 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.986 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.986 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.987 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.987 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.987 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T23:03:11.987 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.987 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.987 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.987 INFO:teuthology.orchestra.run.smithi152.stdout: "smtp_user": { 2024-09-11T23:03:11.987 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "smtp_user", 2024-09-11T23:03:11.987 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.987 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.987 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.987 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.987 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.988 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.988 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.988 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "User to authenticate as", 2024-09-11T23:03:11.988 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.988 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.988 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.988 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.988 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:11.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:11.988 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "balancer", 2024-09-11T23:03:11.988 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:11.988 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:11.988 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:11.989 INFO:teuthology.orchestra.run.smithi152.stdout: "active": { 2024-09-11T23:03:11.989 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "active", 2024-09-11T23:03:11.989 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.989 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.989 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.989 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:11.989 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.989 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.989 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.989 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T23:03:11.989 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.989 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.989 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.990 INFO:teuthology.orchestra.run.smithi152.stdout: "begin_time": { 2024-09-11T23:03:11.990 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "begin_time", 2024-09-11T23:03:11.990 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.990 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.990 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.990 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0000", 2024-09-11T23:03:11.990 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.990 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.990 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.990 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T23:03:11.990 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:03:11.990 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.990 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.991 INFO:teuthology.orchestra.run.smithi152.stdout: "begin_weekday": { 2024-09-11T23:03:11.991 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "begin_weekday", 2024-09-11T23:03:11.991 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-09-11T23:03:11.991 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.991 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.991 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-09-11T23:03:11.991 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "0", 2024-09-11T23:03:11.991 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "6", 2024-09-11T23:03:11.991 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.991 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T23:03:11.991 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:03:11.991 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.991 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.992 INFO:teuthology.orchestra.run.smithi152.stdout: "crush_compat_max_iterations": { 2024-09-11T23:03:11.992 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crush_compat_max_iterations", 2024-09-11T23:03:11.992 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-09-11T23:03:11.992 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.992 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.992 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "25", 2024-09-11T23:03:11.992 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-09-11T23:03:11.992 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "250", 2024-09-11T23:03:11.992 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.992 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T23:03:11.992 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.992 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.993 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.993 INFO:teuthology.orchestra.run.smithi152.stdout: "crush_compat_metrics": { 2024-09-11T23:03:11.993 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crush_compat_metrics", 2024-09-11T23:03:11.993 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.993 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.993 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.993 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T23:03:11.993 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.993 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.993 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.993 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T23:03:11.993 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T23:03:11.994 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.994 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.994 INFO:teuthology.orchestra.run.smithi152.stdout: "crush_compat_step": { 2024-09-11T23:03:11.994 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crush_compat_step", 2024-09-11T23:03:11.994 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:11.994 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.994 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.994 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0.5", 2024-09-11T23:03:11.994 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "0.001", 2024-09-11T23:03:11.994 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "0.999", 2024-09-11T23:03:11.994 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.994 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "aggressiveness of optimization", 2024-09-11T23:03:11.994 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T23:03:11.995 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.995 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.995 INFO:teuthology.orchestra.run.smithi152.stdout: "end_time": { 2024-09-11T23:03:11.995 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "end_time", 2024-09-11T23:03:11.995 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.995 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.995 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.995 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "2359", 2024-09-11T23:03:11.995 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.995 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.995 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.995 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T23:03:11.995 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T23:03:11.996 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.996 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.996 INFO:teuthology.orchestra.run.smithi152.stdout: "end_weekday": { 2024-09-11T23:03:11.996 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "end_weekday", 2024-09-11T23:03:11.996 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-09-11T23:03:11.996 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.996 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.996 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-09-11T23:03:11.996 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "0", 2024-09-11T23:03:11.996 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "6", 2024-09-11T23:03:11.996 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.996 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T23:03:11.997 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T23:03:11.997 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.997 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.997 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:11.997 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:11.997 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:11.997 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.997 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.997 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:11.997 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.997 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.997 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:11.997 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:11.998 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:11.998 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:11.998 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:11.998 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:11.998 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:11.998 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:11.998 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.998 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.998 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.998 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.998 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:11.998 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:11.998 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:11.999 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:11.999 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:11.999 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:11.999 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:11.999 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:11.999 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:11.999 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:11.999 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:11.999 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:11.999 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:11.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:11.999 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:11.999 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:11.999 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.000 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.000 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.000 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.000 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.000 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.000 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.000 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.000 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.000 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.000 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.000 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.000 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.000 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.000 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.001 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.001 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.001 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.001 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.001 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.001 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.001 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.001 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.001 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.001 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.001 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.001 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.001 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.002 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.002 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.002 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.002 INFO:teuthology.orchestra.run.smithi152.stdout: "min_score": { 2024-09-11T23:03:12.002 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "min_score", 2024-09-11T23:03:12.002 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:12.002 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.002 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.002 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-09-11T23:03:12.002 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.002 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.002 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.002 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T23:03:12.003 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.003 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.003 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.003 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": { 2024-09-11T23:03:12.003 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mode", 2024-09-11T23:03:12.003 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.003 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.003 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.003 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "upmap", 2024-09-11T23:03:12.003 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.003 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.003 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.003 INFO:teuthology.orchestra.run.smithi152.stdout: "crush-compat", 2024-09-11T23:03:12.004 INFO:teuthology.orchestra.run.smithi152.stdout: "none", 2024-09-11T23:03:12.004 INFO:teuthology.orchestra.run.smithi152.stdout: "read", 2024-09-11T23:03:12.004 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap", 2024-09-11T23:03:12.004 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap-read" 2024-09-11T23:03:12.004 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.004 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Balancer mode", 2024-09-11T23:03:12.004 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.004 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.004 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.004 INFO:teuthology.orchestra.run.smithi152.stdout: "pool_ids": { 2024-09-11T23:03:12.004 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pool_ids", 2024-09-11T23:03:12.004 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.004 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.005 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.005 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.005 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.005 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.005 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.005 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T23:03:12.005 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.005 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.005 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.005 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-09-11T23:03:12.005 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-09-11T23:03:12.005 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.005 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.006 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.006 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-09-11T23:03:12.006 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.006 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.006 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.006 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T23:03:12.006 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.006 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.006 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.006 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap_max_deviation": { 2024-09-11T23:03:12.006 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "upmap_max_deviation", 2024-09-11T23:03:12.006 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.007 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.007 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.007 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-09-11T23:03:12.007 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-09-11T23:03:12.007 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.007 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.007 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T23:03:12.007 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T23:03:12.007 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.007 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.007 INFO:teuthology.orchestra.run.smithi152.stdout: "upmap_max_optimizations": { 2024-09-11T23:03:12.007 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "upmap_max_optimizations", 2024-09-11T23:03:12.007 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-09-11T23:03:12.008 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.008 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.008 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:12.008 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.008 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.008 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.008 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T23:03:12.008 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.008 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.008 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.008 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.008 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.009 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephadm", 2024-09-11T23:03:12.009 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.009 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.009 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.009 INFO:teuthology.orchestra.run.smithi152.stdout: "agent_down_multiplier": { 2024-09-11T23:03:12.009 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "agent_down_multiplier", 2024-09-11T23:03:12.009 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:12.009 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.009 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.009 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "3.0", 2024-09-11T23:03:12.009 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.009 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.009 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.010 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T23:03:12.010 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.010 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.010 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.010 INFO:teuthology.orchestra.run.smithi152.stdout: "agent_refresh_rate": { 2024-09-11T23:03:12.010 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "agent_refresh_rate", 2024-09-11T23:03:12.010 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.010 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.010 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.010 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "20", 2024-09-11T23:03:12.010 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.010 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.011 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.011 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T23:03:12.011 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.011 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.011 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.011 INFO:teuthology.orchestra.run.smithi152.stdout: "agent_starting_port": { 2024-09-11T23:03:12.011 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "agent_starting_port", 2024-09-11T23:03:12.011 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.011 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.011 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.011 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "4721", 2024-09-11T23:03:12.011 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.011 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.012 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.012 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T23:03:12.012 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.012 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.012 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.012 INFO:teuthology.orchestra.run.smithi152.stdout: "allow_ptrace": { 2024-09-11T23:03:12.012 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "allow_ptrace", 2024-09-11T23:03:12.012 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.012 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.012 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.012 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.012 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.013 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.013 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.013 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T23:03:12.013 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T23:03:12.013 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.013 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.013 INFO:teuthology.orchestra.run.smithi152.stdout: "autotune_interval": { 2024-09-11T23:03:12.013 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "autotune_interval", 2024-09-11T23:03:12.013 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.013 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.013 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.013 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-09-11T23:03:12.013 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.014 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.014 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.014 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T23:03:12.014 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.014 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.014 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.014 INFO:teuthology.orchestra.run.smithi152.stdout: "autotune_memory_target_ratio": { 2024-09-11T23:03:12.014 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T23:03:12.014 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:12.014 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.014 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.014 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0.7", 2024-09-11T23:03:12.014 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.015 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.015 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.015 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T23:03:12.015 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.015 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.015 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.015 INFO:teuthology.orchestra.run.smithi152.stdout: "cephadm_log_destination": { 2024-09-11T23:03:12.015 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cephadm_log_destination", 2024-09-11T23:03:12.015 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.015 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.015 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.015 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.015 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.016 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.016 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.016 INFO:teuthology.orchestra.run.smithi152.stdout: "file", 2024-09-11T23:03:12.016 INFO:teuthology.orchestra.run.smithi152.stdout: "file,syslog", 2024-09-11T23:03:12.016 INFO:teuthology.orchestra.run.smithi152.stdout: "syslog" 2024-09-11T23:03:12.016 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.016 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T23:03:12.016 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.016 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.016 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.016 INFO:teuthology.orchestra.run.smithi152.stdout: "cgroups_split": { 2024-09-11T23:03:12.016 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cgroups_split", 2024-09-11T23:03:12.016 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.017 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.017 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.017 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.017 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.017 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.017 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.017 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T23:03:12.017 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.017 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.017 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.017 INFO:teuthology.orchestra.run.smithi152.stdout: "config_checks_enabled": { 2024-09-11T23:03:12.017 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "config_checks_enabled", 2024-09-11T23:03:12.017 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.018 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.018 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.018 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.018 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.018 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.018 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.018 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T23:03:12.018 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.018 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.018 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.018 INFO:teuthology.orchestra.run.smithi152.stdout: "config_dashboard": { 2024-09-11T23:03:12.018 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "config_dashboard", 2024-09-11T23:03:12.018 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.019 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.019 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.019 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.019 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.019 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.019 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.019 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T23:03:12.019 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.019 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.019 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.019 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_alertmanager": { 2024-09-11T23:03:12.019 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_alertmanager", 2024-09-11T23:03:12.019 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.020 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.020 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.020 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T23:03:12.020 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.020 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.020 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.020 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-09-11T23:03:12.020 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.020 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.020 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.020 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_base": { 2024-09-11T23:03:12.020 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_base", 2024-09-11T23:03:12.021 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.021 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.021 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.021 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T23:03:12.021 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.021 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.021 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.021 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Container image name, without the tag", 2024-09-11T23:03:12.021 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.021 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.021 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.021 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_elasticsearch": { 2024-09-11T23:03:12.022 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_elasticsearch", 2024-09-11T23:03:12.022 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.022 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.022 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.022 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T23:03:12.022 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.022 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.022 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.022 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "elasticsearch container image", 2024-09-11T23:03:12.022 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.022 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.022 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.023 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_grafana": { 2024-09-11T23:03:12.023 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_grafana", 2024-09-11T23:03:12.023 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.023 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.023 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.023 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T23:03:12.023 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.023 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.023 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.023 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-09-11T23:03:12.023 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.023 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.023 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.024 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_haproxy": { 2024-09-11T23:03:12.024 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_haproxy", 2024-09-11T23:03:12.024 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.024 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.024 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.024 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T23:03:12.024 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.024 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.024 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.024 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "HAproxy container image", 2024-09-11T23:03:12.024 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.024 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.024 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.025 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_jaeger_agent": { 2024-09-11T23:03:12.025 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_jaeger_agent", 2024-09-11T23:03:12.025 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.025 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.025 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.025 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T23:03:12.025 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.025 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.025 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.025 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Jaeger agent container image", 2024-09-11T23:03:12.025 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.025 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.025 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.026 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_jaeger_collector": { 2024-09-11T23:03:12.026 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_jaeger_collector", 2024-09-11T23:03:12.026 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.026 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.026 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.026 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T23:03:12.026 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.026 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.026 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.026 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Jaeger collector container image", 2024-09-11T23:03:12.026 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.026 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.027 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.027 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_jaeger_query": { 2024-09-11T23:03:12.027 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_jaeger_query", 2024-09-11T23:03:12.027 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.027 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.027 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.027 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T23:03:12.027 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.027 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.027 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.027 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Jaeger query container image", 2024-09-11T23:03:12.027 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.027 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.028 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.028 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_keepalived": { 2024-09-11T23:03:12.028 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_keepalived", 2024-09-11T23:03:12.028 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.028 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.028 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.028 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T23:03:12.028 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.028 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.028 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.028 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Keepalived container image", 2024-09-11T23:03:12.028 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.029 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.029 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.029 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_loki": { 2024-09-11T23:03:12.029 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_loki", 2024-09-11T23:03:12.029 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.029 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.029 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.029 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T23:03:12.029 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.029 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.029 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.029 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Loki container image", 2024-09-11T23:03:12.029 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.029 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.030 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.030 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_node_exporter": { 2024-09-11T23:03:12.030 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_node_exporter", 2024-09-11T23:03:12.030 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.030 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.030 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.030 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T23:03:12.030 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.030 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.030 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.030 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-09-11T23:03:12.030 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_nvmeof": { 2024-09-11T23:03:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_nvmeof", 2024-09-11T23:03:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T23:03:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Nvme-of container image", 2024-09-11T23:03:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.032 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.032 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_prometheus": { 2024-09-11T23:03:12.032 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_prometheus", 2024-09-11T23:03:12.032 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.032 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.032 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.032 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T23:03:12.032 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.032 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.032 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.032 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Prometheus container image", 2024-09-11T23:03:12.032 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.032 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.033 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.033 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_promtail": { 2024-09-11T23:03:12.033 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_promtail", 2024-09-11T23:03:12.033 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.033 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.033 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.033 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T23:03:12.033 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.033 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.033 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.033 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Promtail container image", 2024-09-11T23:03:12.033 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.034 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.034 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.034 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_samba": { 2024-09-11T23:03:12.034 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_samba", 2024-09-11T23:03:12.034 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.034 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.034 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.034 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T23:03:12.034 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.034 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.034 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.034 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Samba/SMB container image", 2024-09-11T23:03:12.034 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.034 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.035 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.035 INFO:teuthology.orchestra.run.smithi152.stdout: "container_image_snmp_gateway": { 2024-09-11T23:03:12.035 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_image_snmp_gateway", 2024-09-11T23:03:12.035 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.035 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.035 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.035 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T23:03:12.035 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.035 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.035 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.035 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "SNMP Gateway container image", 2024-09-11T23:03:12.035 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.035 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.036 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.036 INFO:teuthology.orchestra.run.smithi152.stdout: "container_init": { 2024-09-11T23:03:12.036 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "container_init", 2024-09-11T23:03:12.036 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.036 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.036 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.036 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.036 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.036 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.036 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.036 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T23:03:12.037 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.037 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.037 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.037 INFO:teuthology.orchestra.run.smithi152.stdout: "daemon_cache_timeout": { 2024-09-11T23:03:12.037 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "daemon_cache_timeout", 2024-09-11T23:03:12.037 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.037 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.037 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.037 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-09-11T23:03:12.037 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.037 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.037 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.037 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T23:03:12.038 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.038 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.038 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.038 INFO:teuthology.orchestra.run.smithi152.stdout: "default_cephadm_command_timeout": { 2024-09-11T23:03:12.038 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T23:03:12.038 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.038 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.038 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.038 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "900", 2024-09-11T23:03:12.038 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.038 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.038 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.038 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T23:03:12.039 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.039 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.039 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.039 INFO:teuthology.orchestra.run.smithi152.stdout: "default_registry": { 2024-09-11T23:03:12.039 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "default_registry", 2024-09-11T23:03:12.039 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.039 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.039 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.039 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "docker.io", 2024-09-11T23:03:12.039 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.039 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.039 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.039 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T23:03:12.040 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.040 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.040 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.040 INFO:teuthology.orchestra.run.smithi152.stdout: "device_cache_timeout": { 2024-09-11T23:03:12.040 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "device_cache_timeout", 2024-09-11T23:03:12.040 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.040 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.040 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.040 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "1800", 2024-09-11T23:03:12.040 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.040 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.040 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.041 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "seconds to cache device inventory", 2024-09-11T23:03:12.041 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.041 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.041 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.041 INFO:teuthology.orchestra.run.smithi152.stdout: "device_enhanced_scan": { 2024-09-11T23:03:12.041 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "device_enhanced_scan", 2024-09-11T23:03:12.041 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.041 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.041 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.041 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.041 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.041 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.041 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.042 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T23:03:12.042 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.042 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.042 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.042 INFO:teuthology.orchestra.run.smithi152.stdout: "facts_cache_timeout": { 2024-09-11T23:03:12.042 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "facts_cache_timeout", 2024-09-11T23:03:12.042 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.042 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.042 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.042 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-09-11T23:03:12.042 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.042 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.042 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.043 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "seconds to cache host facts data", 2024-09-11T23:03:12.043 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.043 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.043 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.043 INFO:teuthology.orchestra.run.smithi152.stdout: "grafana_dashboards_path": { 2024-09-11T23:03:12.043 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "grafana_dashboards_path", 2024-09-11T23:03:12.043 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.043 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.043 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.043 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T23:03:12.043 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.043 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.043 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.044 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T23:03:12.044 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.044 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.044 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.044 INFO:teuthology.orchestra.run.smithi152.stdout: "host_check_interval": { 2024-09-11T23:03:12.044 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "host_check_interval", 2024-09-11T23:03:12.044 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.044 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.044 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.044 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-09-11T23:03:12.044 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.044 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.044 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.045 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to perform a host check", 2024-09-11T23:03:12.045 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.045 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.045 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.045 INFO:teuthology.orchestra.run.smithi152.stdout: "hw_monitoring": { 2024-09-11T23:03:12.045 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "hw_monitoring", 2024-09-11T23:03:12.045 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.045 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.045 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.045 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.045 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.045 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.046 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.046 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T23:03:12.046 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.046 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.046 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.046 INFO:teuthology.orchestra.run.smithi152.stdout: "inventory_list_all": { 2024-09-11T23:03:12.046 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "inventory_list_all", 2024-09-11T23:03:12.046 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.046 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.046 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.046 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.046 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.046 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.047 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.047 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T23:03:12.047 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.047 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.047 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.047 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.047 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.047 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.047 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.047 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.047 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.047 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.047 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.048 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.048 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.048 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.048 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.048 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.048 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.048 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.048 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.048 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.048 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.048 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.048 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.048 INFO:teuthology.orchestra.run.smithi152.stdout: "log_refresh_metadata": { 2024-09-11T23:03:12.049 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_refresh_metadata", 2024-09-11T23:03:12.049 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.049 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.049 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.049 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.049 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.049 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.049 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.049 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T23:03:12.049 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.049 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.049 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.049 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.050 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.050 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.050 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.050 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.050 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.050 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.050 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.050 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.050 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T23:03:12.050 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.050 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.050 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.050 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.051 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.051 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.051 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.051 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.051 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.051 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.051 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.051 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.051 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.051 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.051 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.051 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.051 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.052 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.052 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.052 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.052 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.052 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.052 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.052 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.052 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.052 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.052 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.052 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.052 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.053 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.053 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.053 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.053 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.053 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.053 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.053 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.053 INFO:teuthology.orchestra.run.smithi152.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T23:03:12.053 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T23:03:12.053 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.053 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.053 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.053 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.054 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.054 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.054 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.054 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T23:03:12.054 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.054 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.054 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.054 INFO:teuthology.orchestra.run.smithi152.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T23:03:12.054 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T23:03:12.054 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.054 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.054 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.054 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "*", 2024-09-11T23:03:12.055 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.055 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.055 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.055 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T23:03:12.055 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.055 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.055 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.055 INFO:teuthology.orchestra.run.smithi152.stdout: "max_count_per_host": { 2024-09-11T23:03:12.055 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_count_per_host", 2024-09-11T23:03:12.055 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.055 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.055 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.055 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:12.056 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.056 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.056 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.056 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "max number of daemons per service per host", 2024-09-11T23:03:12.056 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.056 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.056 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.056 INFO:teuthology.orchestra.run.smithi152.stdout: "max_osd_draining_count": { 2024-09-11T23:03:12.056 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_osd_draining_count", 2024-09-11T23:03:12.056 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.056 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.056 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.057 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:12.057 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.057 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.057 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.057 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T23:03:12.057 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.057 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.057 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.057 INFO:teuthology.orchestra.run.smithi152.stdout: "migration_current": { 2024-09-11T23:03:12.057 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "migration_current", 2024-09-11T23:03:12.057 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.057 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.058 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.058 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.058 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.058 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.058 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.058 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "internal - do not modify", 2024-09-11T23:03:12.058 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.058 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.058 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.058 INFO:teuthology.orchestra.run.smithi152.stdout: "mode": { 2024-09-11T23:03:12.058 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mode", 2024-09-11T23:03:12.058 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.058 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.059 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.059 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "root", 2024-09-11T23:03:12.059 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.059 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.059 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.059 INFO:teuthology.orchestra.run.smithi152.stdout: "cephadm-package", 2024-09-11T23:03:12.059 INFO:teuthology.orchestra.run.smithi152.stdout: "root" 2024-09-11T23:03:12.059 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.059 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T23:03:12.059 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.059 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.059 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.059 INFO:teuthology.orchestra.run.smithi152.stdout: "oob_default_addr": { 2024-09-11T23:03:12.059 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "oob_default_addr", 2024-09-11T23:03:12.060 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.060 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.060 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.060 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "169.254.1.1", 2024-09-11T23:03:12.060 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.060 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.060 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.060 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T23:03:12.060 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.060 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.060 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.060 INFO:teuthology.orchestra.run.smithi152.stdout: "prometheus_alerts_path": { 2024-09-11T23:03:12.061 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "prometheus_alerts_path", 2024-09-11T23:03:12.061 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.061 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.061 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.061 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T23:03:12.061 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.061 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.061 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.061 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T23:03:12.061 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.061 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.061 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.061 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_insecure": { 2024-09-11T23:03:12.062 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_insecure", 2024-09-11T23:03:12.062 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.062 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.062 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.062 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.062 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.062 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.062 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.062 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T23:03:12.062 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.062 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.062 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.062 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_password": { 2024-09-11T23:03:12.063 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_password", 2024-09-11T23:03:12.063 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.063 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.063 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.063 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.063 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.063 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.063 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.063 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T23:03:12.063 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.063 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.063 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.063 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_url": { 2024-09-11T23:03:12.064 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_url", 2024-09-11T23:03:12.064 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.064 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.064 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.064 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.064 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.064 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.064 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.064 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T23:03:12.064 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.064 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.064 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.064 INFO:teuthology.orchestra.run.smithi152.stdout: "registry_username": { 2024-09-11T23:03:12.065 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "registry_username", 2024-09-11T23:03:12.065 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.065 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.065 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.065 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.065 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.065 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.065 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.065 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T23:03:12.065 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.065 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.065 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.065 INFO:teuthology.orchestra.run.smithi152.stdout: "secure_monitoring_stack": { 2024-09-11T23:03:12.066 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "secure_monitoring_stack", 2024-09-11T23:03:12.066 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.066 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.066 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.066 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.066 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.066 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.066 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.066 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T23:03:12.066 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.066 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.066 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.067 INFO:teuthology.orchestra.run.smithi152.stdout: "service_discovery_port": { 2024-09-11T23:03:12.067 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "service_discovery_port", 2024-09-11T23:03:12.067 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.067 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.067 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.067 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8765", 2024-09-11T23:03:12.067 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.067 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.067 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.067 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "cephadm service discovery port", 2024-09-11T23:03:12.067 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.067 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.067 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.068 INFO:teuthology.orchestra.run.smithi152.stdout: "ssh_config_file": { 2024-09-11T23:03:12.068 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssh_config_file", 2024-09-11T23:03:12.068 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.068 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.068 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.068 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.068 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.068 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.068 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.068 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T23:03:12.068 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.068 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.068 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.069 INFO:teuthology.orchestra.run.smithi152.stdout: "use_agent": { 2024-09-11T23:03:12.069 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "use_agent", 2024-09-11T23:03:12.069 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.069 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.069 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.069 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.069 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.069 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.069 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.069 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T23:03:12.069 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.069 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.069 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.070 INFO:teuthology.orchestra.run.smithi152.stdout: "use_repo_digest": { 2024-09-11T23:03:12.070 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "use_repo_digest", 2024-09-11T23:03:12.070 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.070 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.070 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.070 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.070 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.070 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.070 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.070 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T23:03:12.070 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.070 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.070 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.071 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_on_failed_host_check": { 2024-09-11T23:03:12.071 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_on_failed_host_check", 2024-09-11T23:03:12.071 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.071 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.071 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.071 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.071 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.071 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.071 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.071 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T23:03:12.071 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.071 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.071 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.072 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.072 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_on_stray_daemons": { 2024-09-11T23:03:12.072 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_on_stray_daemons", 2024-09-11T23:03:12.072 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.072 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.072 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.072 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.072 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.072 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.072 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.072 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T23:03:12.072 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.072 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.073 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.073 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_on_stray_hosts": { 2024-09-11T23:03:12.073 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_on_stray_hosts", 2024-09-11T23:03:12.073 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.073 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.073 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.073 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.073 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.073 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.073 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.073 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T23:03:12.073 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.073 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.074 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.074 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.074 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.074 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crash", 2024-09-11T23:03:12.074 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.074 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.074 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.074 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.074 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.074 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.074 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.074 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.075 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.075 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.075 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.075 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.075 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.075 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.075 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.075 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.075 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.075 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.075 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.075 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.075 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.075 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.076 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.076 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.076 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.076 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.076 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.076 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.076 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.076 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.076 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.076 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.076 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.076 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.077 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.077 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.077 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.077 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.077 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.077 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.077 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.077 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.077 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.077 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.077 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.077 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.077 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.078 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.078 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.078 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.078 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.078 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.078 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.078 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.078 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.078 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.078 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.078 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.078 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.078 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.079 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.079 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.079 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.079 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.079 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.079 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.079 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.079 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.079 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.079 INFO:teuthology.orchestra.run.smithi152.stdout: "retain_interval": { 2024-09-11T23:03:12.079 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "retain_interval", 2024-09-11T23:03:12.079 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.079 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.080 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.080 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "31536000", 2024-09-11T23:03:12.080 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.080 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.080 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.080 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T23:03:12.080 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.080 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.080 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.080 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_recent_interval": { 2024-09-11T23:03:12.080 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_recent_interval", 2024-09-11T23:03:12.080 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.080 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.081 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.081 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "1209600", 2024-09-11T23:03:12.081 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.081 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.081 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.081 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T23:03:12.081 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.081 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.081 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.081 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.081 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.081 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "dashboard", 2024-09-11T23:03:12.082 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.082 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.082 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.082 INFO:teuthology.orchestra.run.smithi152.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T23:03:12.082 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T23:03:12.082 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.082 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.082 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.082 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:12.082 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.082 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.082 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.082 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.083 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.083 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.083 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.083 INFO:teuthology.orchestra.run.smithi152.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T23:03:12.083 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T23:03:12.083 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.083 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.083 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.083 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.083 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.083 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.083 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.083 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.084 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.084 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.084 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.084 INFO:teuthology.orchestra.run.smithi152.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T23:03:12.084 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T23:03:12.084 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.084 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.084 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.084 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.084 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.084 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.085 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.085 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.085 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.085 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.085 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.085 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.085 INFO:teuthology.orchestra.run.smithi152.stdout: "AUDIT_API_ENABLED": { 2024-09-11T23:03:12.085 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T23:03:12.085 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.085 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.085 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.085 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.085 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.085 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.086 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.086 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.086 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.086 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.086 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.086 INFO:teuthology.orchestra.run.smithi152.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T23:03:12.086 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T23:03:12.086 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.086 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.086 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.086 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.086 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.086 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.087 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.087 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.087 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.087 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.087 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.087 INFO:teuthology.orchestra.run.smithi152.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T23:03:12.087 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T23:03:12.087 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.087 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.087 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.087 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.087 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.088 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.088 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.088 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.088 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.088 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.088 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.088 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T23:03:12.088 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T23:03:12.088 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.088 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.088 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.088 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.088 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.089 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.089 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.089 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.089 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.089 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.089 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.089 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.089 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T23:03:12.089 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T23:03:12.089 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.089 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.089 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.089 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.089 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.090 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.090 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.090 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.090 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.090 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.090 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.090 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T23:03:12.090 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T23:03:12.090 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.090 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.090 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.090 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.090 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.091 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.091 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.091 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.091 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.091 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.091 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.091 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T23:03:12.091 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T23:03:12.091 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.091 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.091 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.091 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.091 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.092 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.092 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.092 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.092 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.092 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.092 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.092 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T23:03:12.092 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T23:03:12.092 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.092 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.092 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.092 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T23:03:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T23:03:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.093 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T23:03:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T23:03:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.094 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.095 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.095 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.095 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.095 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.095 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.095 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.095 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.095 INFO:teuthology.orchestra.run.smithi152.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T23:03:12.095 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T23:03:12.095 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.095 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.095 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.095 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.096 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.096 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.096 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.096 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.096 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.096 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.096 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.096 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T23:03:12.096 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T23:03:12.096 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.096 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.096 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.096 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "admin", 2024-09-11T23:03:12.097 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.097 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.097 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.097 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.097 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.097 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.097 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.097 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T23:03:12.097 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T23:03:12.097 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.097 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.097 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.097 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.098 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.098 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.098 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.098 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.098 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.098 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.098 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.098 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_URL": { 2024-09-11T23:03:12.098 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_URL", 2024-09-11T23:03:12.098 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.098 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.098 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.099 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.099 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.099 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.099 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.099 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.099 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.099 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.099 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.099 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T23:03:12.099 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T23:03:12.099 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.099 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.099 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.100 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "admin", 2024-09-11T23:03:12.100 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.100 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.100 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.100 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.100 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.100 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.100 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.100 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.100 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T23:03:12.100 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T23:03:12.100 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.100 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.101 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.101 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.101 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.101 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.101 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.101 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.101 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.101 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.101 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.101 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.101 INFO:teuthology.orchestra.run.smithi152.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T23:03:12.101 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T23:03:12.101 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.101 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.102 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.102 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.102 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.102 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.102 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.102 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.102 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.102 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.102 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.102 INFO:teuthology.orchestra.run.smithi152.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T23:03:12.102 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T23:03:12.102 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.103 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.103 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.103 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.103 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.103 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.103 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.103 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.103 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.103 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.103 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.103 INFO:teuthology.orchestra.run.smithi152.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T23:03:12.103 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T23:03:12.103 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.104 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.104 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.104 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.104 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.104 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.104 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.104 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.104 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.104 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.104 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.104 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.104 INFO:teuthology.orchestra.run.smithi152.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T23:03:12.104 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T23:03:12.104 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.105 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.105 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.105 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.105 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.105 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.105 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.105 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.105 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.105 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.105 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.105 INFO:teuthology.orchestra.run.smithi152.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T23:03:12.105 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T23:03:12.105 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.106 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.106 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.106 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.106 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.106 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.106 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.106 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.106 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.106 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.106 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.106 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.106 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T23:03:12.106 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T23:03:12.106 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.107 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.107 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.107 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.107 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.107 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.107 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.107 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.107 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.107 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.107 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.107 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T23:03:12.107 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T23:03:12.107 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.108 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.108 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.108 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.108 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.108 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.108 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.108 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.108 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.108 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.108 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.108 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T23:03:12.108 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T23:03:12.108 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.109 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.109 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.109 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.109 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.109 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.109 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.109 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.109 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.109 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.109 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.109 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T23:03:12.109 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T23:03:12.109 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.110 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.110 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.110 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.110 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.110 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.110 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.110 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.110 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.110 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.110 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.110 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T23:03:12.110 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T23:03:12.111 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.111 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.111 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.111 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.111 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.111 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.111 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.111 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.111 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.111 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.111 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.111 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.111 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T23:03:12.111 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T23:03:12.111 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.112 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.112 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.112 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.112 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.112 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.112 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.112 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.112 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.112 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.112 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.112 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T23:03:12.112 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T23:03:12.113 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.113 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.113 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.113 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.113 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.113 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.113 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.113 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.113 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.113 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.113 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.113 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T23:03:12.113 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T23:03:12.114 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.114 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.114 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.114 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.114 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.114 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.114 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.114 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.114 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.114 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.114 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.114 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T23:03:12.114 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T23:03:12.115 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.115 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.115 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.115 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T23:03:12.115 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.115 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.115 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.115 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.115 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.115 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.115 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.115 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T23:03:12.115 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T23:03:12.116 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.116 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.116 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.116 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:12.116 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.116 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.116 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.116 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.116 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.116 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.116 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.116 INFO:teuthology.orchestra.run.smithi152.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T23:03:12.117 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T23:03:12.117 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.117 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.117 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.117 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8", 2024-09-11T23:03:12.117 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.117 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.117 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.117 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.117 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.117 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.117 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.117 INFO:teuthology.orchestra.run.smithi152.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T23:03:12.118 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T23:03:12.118 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.118 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.118 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.118 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "45", 2024-09-11T23:03:12.118 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.118 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.118 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.118 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.118 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.118 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.118 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.118 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T23:03:12.119 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T23:03:12.119 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.119 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.119 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.119 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.119 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.119 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.119 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.119 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.119 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.119 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.119 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.119 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T23:03:12.120 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T23:03:12.120 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.120 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.120 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.120 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "admin", 2024-09-11T23:03:12.120 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.120 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.120 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.120 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.120 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.120 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.120 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.121 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T23:03:12.121 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T23:03:12.121 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.121 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.121 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.121 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.121 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.121 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.121 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.121 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.121 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.121 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.121 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.122 INFO:teuthology.orchestra.run.smithi152.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T23:03:12.122 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T23:03:12.122 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.122 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.122 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.122 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.122 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.122 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.122 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.122 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.122 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.122 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.122 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.123 INFO:teuthology.orchestra.run.smithi152.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T23:03:12.123 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T23:03:12.123 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.123 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.123 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.123 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-09-11T23:03:12.123 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.123 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.123 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.123 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.123 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.123 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.123 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.124 INFO:teuthology.orchestra.run.smithi152.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T23:03:12.124 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T23:03:12.124 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.124 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.124 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.124 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:12.124 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.124 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.124 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.124 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.124 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.124 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.124 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.125 INFO:teuthology.orchestra.run.smithi152.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T23:03:12.125 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T23:03:12.125 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.125 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.125 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.125 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-09-11T23:03:12.125 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.125 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.125 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.125 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.125 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.125 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.125 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.126 INFO:teuthology.orchestra.run.smithi152.stdout: "cross_origin_url": { 2024-09-11T23:03:12.126 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cross_origin_url", 2024-09-11T23:03:12.126 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.126 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.126 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.126 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.126 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.126 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.126 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.126 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.126 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.126 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.126 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.127 INFO:teuthology.orchestra.run.smithi152.stdout: "crt_file": { 2024-09-11T23:03:12.127 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "crt_file", 2024-09-11T23:03:12.127 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.127 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.127 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.127 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.127 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.127 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.127 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.127 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.127 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.127 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.127 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.127 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.128 INFO:teuthology.orchestra.run.smithi152.stdout: "debug": { 2024-09-11T23:03:12.128 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "debug", 2024-09-11T23:03:12.128 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.128 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.128 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.128 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.128 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.128 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.128 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.128 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Enable/disable debug options", 2024-09-11T23:03:12.128 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.128 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.128 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.128 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.129 INFO:teuthology.orchestra.run.smithi152.stdout: "jwt_token_ttl": { 2024-09-11T23:03:12.129 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "jwt_token_ttl", 2024-09-11T23:03:12.129 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.129 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.129 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.129 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "28800", 2024-09-11T23:03:12.129 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.129 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.129 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.129 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.129 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.129 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.129 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.129 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.130 INFO:teuthology.orchestra.run.smithi152.stdout: "key_file": { 2024-09-11T23:03:12.130 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "key_file", 2024-09-11T23:03:12.130 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.130 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.130 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.130 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.130 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.130 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.130 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.130 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.130 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.130 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.130 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.131 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.131 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.131 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.131 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.131 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.131 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.131 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.131 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.131 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.131 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.131 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.131 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.131 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.131 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.132 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.132 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.132 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.132 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.132 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.132 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.132 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.132 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.132 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.132 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.132 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.132 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.132 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.133 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.133 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.133 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.133 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.133 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.133 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.133 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.133 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.133 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.133 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.133 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.133 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.133 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.133 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.134 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.134 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.134 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.134 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.134 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.134 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.134 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.134 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.134 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.134 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.134 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.134 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.134 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.134 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.135 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.135 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.135 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.135 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.135 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.135 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.135 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.135 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.135 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.135 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.135 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.135 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.135 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.135 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.136 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.136 INFO:teuthology.orchestra.run.smithi152.stdout: "motd": { 2024-09-11T23:03:12.136 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "motd", 2024-09-11T23:03:12.136 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.136 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.136 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.136 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.136 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.136 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.136 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.136 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "The message of the day", 2024-09-11T23:03:12.136 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.136 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.137 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.137 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.137 INFO:teuthology.orchestra.run.smithi152.stdout: "redirect_resolve_ip_addr": { 2024-09-11T23:03:12.137 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T23:03:12.137 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.137 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.137 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.137 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.137 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.137 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.137 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.137 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.137 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.137 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.138 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.138 INFO:teuthology.orchestra.run.smithi152.stdout: "server_addr": { 2024-09-11T23:03:12.138 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_addr", 2024-09-11T23:03:12.138 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.138 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.138 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.138 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "::", 2024-09-11T23:03:12.138 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.138 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.138 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.138 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.138 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.138 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.139 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.139 INFO:teuthology.orchestra.run.smithi152.stdout: "server_port": { 2024-09-11T23:03:12.139 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_port", 2024-09-11T23:03:12.139 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.139 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.139 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.139 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8080", 2024-09-11T23:03:12.139 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.139 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.139 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.139 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.139 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.139 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.140 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.140 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.140 INFO:teuthology.orchestra.run.smithi152.stdout: "ssl": { 2024-09-11T23:03:12.140 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssl", 2024-09-11T23:03:12.140 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.140 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.140 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.140 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.140 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.140 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.140 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.140 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.140 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.141 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.141 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.141 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.141 INFO:teuthology.orchestra.run.smithi152.stdout: "ssl_server_port": { 2024-09-11T23:03:12.141 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssl_server_port", 2024-09-11T23:03:12.141 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.141 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.141 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.141 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8443", 2024-09-11T23:03:12.141 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.141 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.141 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.141 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.141 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.142 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.142 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.142 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.142 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_behaviour": { 2024-09-11T23:03:12.142 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_behaviour", 2024-09-11T23:03:12.142 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.142 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.142 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.142 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "redirect", 2024-09-11T23:03:12.142 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.142 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.142 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.142 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.142 INFO:teuthology.orchestra.run.smithi152.stdout: "redirect" 2024-09-11T23:03:12.143 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.143 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.143 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.143 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.143 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.143 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_error_status_code": { 2024-09-11T23:03:12.143 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_error_status_code", 2024-09-11T23:03:12.143 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.143 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.143 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.143 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "500", 2024-09-11T23:03:12.143 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "400", 2024-09-11T23:03:12.143 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "599", 2024-09-11T23:03:12.144 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.144 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.144 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.144 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.144 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.144 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.144 INFO:teuthology.orchestra.run.smithi152.stdout: "url_prefix": { 2024-09-11T23:03:12.144 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "url_prefix", 2024-09-11T23:03:12.144 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.144 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.144 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.144 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.144 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.144 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "devicehealth", 2024-09-11T23:03:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.145 INFO:teuthology.orchestra.run.smithi152.stdout: "enable_monitoring": { 2024-09-11T23:03:12.146 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enable_monitoring", 2024-09-11T23:03:12.146 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.146 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.146 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.146 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.146 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.146 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.146 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.146 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "monitor device health metrics", 2024-09-11T23:03:12.146 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.146 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.146 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.146 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.146 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.147 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.147 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.147 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.147 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.147 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.147 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.147 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.147 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.147 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.147 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.147 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.147 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.147 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.147 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.148 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.148 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.148 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.148 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.148 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.148 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.148 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.148 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.148 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.148 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.148 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.148 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.149 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.149 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.149 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.149 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.149 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.149 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.149 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.149 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.149 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.149 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.149 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.149 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.150 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.150 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.150 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.150 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.150 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.150 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.150 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.150 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.150 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.150 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.152 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.152 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.152 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.152 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.152 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.152 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: "mark_out_threshold": { 2024-09-11T23:03:12.153 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mark_out_threshold", 2024-09-11T23:03:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "2419200", 2024-09-11T23:03:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T23:03:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.154 INFO:teuthology.orchestra.run.smithi152.stdout: "pool_name": { 2024-09-11T23:03:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pool_name", 2024-09-11T23:03:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "device_health_metrics", 2024-09-11T23:03:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T23:03:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.155 INFO:teuthology.orchestra.run.smithi152.stdout: "retention_period": { 2024-09-11T23:03:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "retention_period", 2024-09-11T23:03:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "15552000", 2024-09-11T23:03:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how long to retain device health metrics", 2024-09-11T23:03:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: "scrape_frequency": { 2024-09-11T23:03:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "scrape_frequency", 2024-09-11T23:03:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "86400", 2024-09-11T23:03:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T23:03:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: "self_heal": { 2024-09-11T23:03:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "self_heal", 2024-09-11T23:03:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T23:03:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-09-11T23:03:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-09-11T23:03:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-09-11T23:03:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T23:03:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: "warn_threshold": { 2024-09-11T23:03:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "warn_threshold", 2024-09-11T23:03:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "7257600", 2024-09-11T23:03:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T23:03:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.160 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "diskprediction_local", 2024-09-11T23:03:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.161 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.162 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.163 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.164 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.164 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.164 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.164 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.164 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.164 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.164 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.164 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.164 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.164 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.164 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.164 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.164 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.164 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.165 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: "predict_interval": { 2024-09-11T23:03:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "predict_interval", 2024-09-11T23:03:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.166 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "86400", 2024-09-11T23:03:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: "predictor_model": { 2024-09-11T23:03:12.167 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "predictor_model", 2024-09-11T23:03:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "prophetstor", 2024-09-11T23:03:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-09-11T23:03:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-09-11T23:03:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "600", 2024-09-11T23:03:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "influx", 2024-09-11T23:03:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": false, 2024-09-11T23:03:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "influxdb python module not found", 2024-09-11T23:03:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: "batch_size": { 2024-09-11T23:03:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "batch_size", 2024-09-11T23:03:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5000", 2024-09-11T23:03:12.170 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T23:03:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: "database": { 2024-09-11T23:03:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "database", 2024-09-11T23:03:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "ceph", 2024-09-11T23:03:12.171 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T23:03:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: "hostname": { 2024-09-11T23:03:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "hostname", 2024-09-11T23:03:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.172 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "InfluxDB server hostname", 2024-09-11T23:03:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-09-11T23:03:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-09-11T23:03:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.173 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "30", 2024-09-11T23:03:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "5", 2024-09-11T23:03:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T23:03:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.174 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.175 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.176 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.177 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.178 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.178 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.178 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.178 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.178 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.178 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.178 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.178 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.178 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.178 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.178 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.178 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.178 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.178 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.179 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.179 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.179 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.179 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.179 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.179 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.179 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.179 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.179 INFO:teuthology.orchestra.run.smithi152.stdout: "password": { 2024-09-11T23:03:12.179 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "password", 2024-09-11T23:03:12.179 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.179 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.179 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.179 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.180 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.180 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.180 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.180 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "password of InfluxDB server user", 2024-09-11T23:03:12.180 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.180 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.180 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.180 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.180 INFO:teuthology.orchestra.run.smithi152.stdout: "port": { 2024-09-11T23:03:12.180 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "port", 2024-09-11T23:03:12.180 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.180 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.180 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.180 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "8086", 2024-09-11T23:03:12.181 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.181 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.181 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.181 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "InfluxDB server port", 2024-09-11T23:03:12.181 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.181 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.181 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.181 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.181 INFO:teuthology.orchestra.run.smithi152.stdout: "ssl": { 2024-09-11T23:03:12.181 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ssl", 2024-09-11T23:03:12.181 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.181 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.181 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.181 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "false", 2024-09-11T23:03:12.182 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.182 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.182 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.182 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:03:12.182 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.182 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.182 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.182 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.182 INFO:teuthology.orchestra.run.smithi152.stdout: "threads": { 2024-09-11T23:03:12.182 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "threads", 2024-09-11T23:03:12.182 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.182 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.182 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.182 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-09-11T23:03:12.183 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-09-11T23:03:12.183 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "32", 2024-09-11T23:03:12.183 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.183 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T23:03:12.183 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.183 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.183 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.183 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.183 INFO:teuthology.orchestra.run.smithi152.stdout: "username": { 2024-09-11T23:03:12.183 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "username", 2024-09-11T23:03:12.183 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.183 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.183 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.183 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.184 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.184 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.184 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.184 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "username of InfluxDB server user", 2024-09-11T23:03:12.184 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.184 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.184 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.184 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.184 INFO:teuthology.orchestra.run.smithi152.stdout: "verify_ssl": { 2024-09-11T23:03:12.184 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "verify_ssl", 2024-09-11T23:03:12.184 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.184 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.184 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.185 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "true", 2024-09-11T23:03:12.185 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.185 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.185 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.185 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T23:03:12.185 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.185 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.185 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.185 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.185 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.185 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.185 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.185 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "insights", 2024-09-11T23:03:12.186 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.186 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.186 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.186 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.186 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.186 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.186 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.186 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.186 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.186 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.186 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.186 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.186 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.186 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.187 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.187 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.187 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.187 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.187 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.187 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.187 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.187 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.187 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.187 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.187 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.187 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.187 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.187 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.188 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.188 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.188 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.188 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.188 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.188 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.188 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.188 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.188 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.188 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.188 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.188 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.188 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.189 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.189 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.189 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.189 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.189 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.189 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.189 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.189 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.189 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.189 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.189 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.189 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.189 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.189 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.190 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.190 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.190 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.190 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.190 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.190 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.190 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.190 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.190 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.190 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.190 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.190 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.190 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.190 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.191 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.191 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.191 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.191 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.191 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.191 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.191 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.191 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "iostat", 2024-09-11T23:03:12.191 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.191 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.191 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.191 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.191 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.191 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.192 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.192 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.192 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.192 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.192 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.192 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.192 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.192 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.192 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.192 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.192 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.192 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.192 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.193 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.193 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.193 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.193 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.193 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.193 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.193 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.193 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.193 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.193 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.193 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.193 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.193 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.193 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.194 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.194 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.194 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.194 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.194 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.194 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.194 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.194 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.194 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.194 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.194 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.194 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.194 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.194 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.195 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.195 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.195 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.195 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.195 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.195 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.195 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.195 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.195 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.195 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.195 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.195 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.195 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.195 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.196 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.196 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.196 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.196 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.196 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.196 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.196 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.196 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.196 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.196 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.196 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.196 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.196 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.196 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.197 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.197 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "k8sevents", 2024-09-11T23:03:12.197 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.197 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.197 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.197 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_event_retention_days": { 2024-09-11T23:03:12.197 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "ceph_event_retention_days", 2024-09-11T23:03:12.197 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.197 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.197 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.197 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "7", 2024-09-11T23:03:12.197 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.197 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.197 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.197 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T23:03:12.198 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.198 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.198 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.198 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.198 INFO:teuthology.orchestra.run.smithi152.stdout: "config_check_secs": { 2024-09-11T23:03:12.198 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "config_check_secs", 2024-09-11T23:03:12.198 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.198 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.198 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.198 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:12.198 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "10", 2024-09-11T23:03:12.198 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.198 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.199 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T23:03:12.199 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.199 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.199 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.199 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.199 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.199 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.199 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.199 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.199 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.199 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.199 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.199 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.199 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.199 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.200 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.200 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.200 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.200 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.200 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.200 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.200 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.200 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.200 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.200 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.200 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.200 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.200 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.201 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.201 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.201 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.201 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.201 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.201 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.201 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.201 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.201 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.201 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.201 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.201 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.201 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.202 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.202 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.202 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.202 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.202 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.202 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.202 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.202 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.202 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.202 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.202 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.202 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.202 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.202 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.203 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.203 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.203 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.203 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.203 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.203 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.203 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.203 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.203 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.203 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.203 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.203 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.203 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.203 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.204 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.204 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.204 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.204 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.204 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.204 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.204 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.204 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.204 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.204 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "localpool", 2024-09-11T23:03:12.204 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.204 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.204 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.204 INFO:teuthology.orchestra.run.smithi152.stdout: "failure_domain": { 2024-09-11T23:03:12.205 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "failure_domain", 2024-09-11T23:03:12.205 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.205 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.205 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.205 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "host", 2024-09-11T23:03:12.205 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.205 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.205 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.205 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "failure domain for any created local pool", 2024-09-11T23:03:12.205 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T23:03:12.205 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.205 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.205 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.206 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.206 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.206 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.206 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.206 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.206 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.206 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.206 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.206 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.206 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.206 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.206 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.206 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.206 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.207 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.207 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.207 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.207 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.207 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.207 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.207 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.207 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.207 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.207 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.207 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.207 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.207 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.207 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.208 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.208 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.208 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.208 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.208 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.208 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.208 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.208 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.208 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.208 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.208 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.208 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.208 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.209 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.209 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.209 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.209 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.209 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.209 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.209 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.209 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.209 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.209 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.209 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.209 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.209 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.209 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.210 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.210 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.210 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.210 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.210 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.210 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.210 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.210 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.210 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.210 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.210 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.210 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.210 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.210 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.211 INFO:teuthology.orchestra.run.smithi152.stdout: "min_size": { 2024-09-11T23:03:12.211 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "min_size", 2024-09-11T23:03:12.211 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.211 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.211 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.211 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.211 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.211 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.211 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.211 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "default min_size for any created local pool", 2024-09-11T23:03:12.211 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T23:03:12.211 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.211 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.212 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.212 INFO:teuthology.orchestra.run.smithi152.stdout: "num_rep": { 2024-09-11T23:03:12.212 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "num_rep", 2024-09-11T23:03:12.212 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.212 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.212 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.212 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "3", 2024-09-11T23:03:12.212 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.212 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.212 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.212 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "default replica count for any created local pool", 2024-09-11T23:03:12.212 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.212 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.213 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.213 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.213 INFO:teuthology.orchestra.run.smithi152.stdout: "pg_num": { 2024-09-11T23:03:12.213 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pg_num", 2024-09-11T23:03:12.213 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.213 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.213 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.213 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "128", 2024-09-11T23:03:12.213 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.213 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.213 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.213 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T23:03:12.213 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.213 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.214 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.214 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.214 INFO:teuthology.orchestra.run.smithi152.stdout: "prefix": { 2024-09-11T23:03:12.214 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "prefix", 2024-09-11T23:03:12.214 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.214 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.214 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.214 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.214 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.214 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.214 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.214 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "name prefix for any created local pool", 2024-09-11T23:03:12.214 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.214 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.215 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.215 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.215 INFO:teuthology.orchestra.run.smithi152.stdout: "subtree": { 2024-09-11T23:03:12.215 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "subtree", 2024-09-11T23:03:12.215 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.215 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.215 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.215 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "rack", 2024-09-11T23:03:12.215 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.215 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.215 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.215 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T23:03:12.215 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T23:03:12.215 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.216 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.216 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.216 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.216 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.216 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.216 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mds_autoscaler", 2024-09-11T23:03:12.216 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.216 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.216 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.216 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.216 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.216 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.216 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.216 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.217 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.217 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.217 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.217 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.217 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.217 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.217 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.217 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.217 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.217 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.217 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.217 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.217 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.218 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.218 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.218 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.218 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.218 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.218 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.218 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.218 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.218 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.218 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.218 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.218 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.218 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.218 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.219 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.219 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.219 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.219 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.219 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.219 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.219 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.219 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.219 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.219 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.219 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.219 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.219 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.220 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.220 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.220 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.220 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.220 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.220 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.220 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.220 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.220 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.220 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.220 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.220 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.220 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.220 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.221 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.221 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.221 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.221 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.221 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.221 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.221 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.221 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.221 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.221 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.221 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.221 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.221 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.222 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.222 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mirroring", 2024-09-11T23:03:12.222 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.222 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.222 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.222 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.222 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.222 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.222 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.222 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.222 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.222 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.222 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.222 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.223 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.223 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.223 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.223 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.223 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.223 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.223 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.223 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.223 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.223 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.223 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.223 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.223 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.223 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.224 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.224 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.224 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.224 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.224 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.224 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.224 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.224 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.224 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.224 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.224 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.224 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.224 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.225 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.225 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.225 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.225 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.225 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.225 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.225 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.225 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.225 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.225 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.225 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.225 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.225 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.225 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.226 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.226 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.226 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.226 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.226 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.226 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.226 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.226 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.226 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.226 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.226 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.226 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.226 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.226 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.226 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.227 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.227 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.227 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.227 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.227 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.227 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.227 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.227 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.227 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "nfs", 2024-09-11T23:03:12.227 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.227 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.227 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.227 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.227 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.228 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.228 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.228 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.228 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.228 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.228 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.228 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.228 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.228 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.228 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.228 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.228 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.228 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.229 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.229 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.229 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.229 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.229 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.229 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.229 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.229 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.229 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.229 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.229 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.229 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.229 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.229 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.230 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.230 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.230 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.230 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.230 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.230 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.230 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.230 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.230 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.230 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.230 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.230 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.230 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.230 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.231 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.231 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.231 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.231 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.231 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.231 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.231 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.231 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.231 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.231 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.231 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.231 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.231 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.231 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.231 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.232 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.232 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.232 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.232 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.232 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.232 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.232 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.232 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.232 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.232 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.232 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.232 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.232 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.233 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.233 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.233 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "orchestrator", 2024-09-11T23:03:12.233 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.233 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.233 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.233 INFO:teuthology.orchestra.run.smithi152.stdout: "fail_fs": { 2024-09-11T23:03:12.233 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "fail_fs", 2024-09-11T23:03:12.233 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.233 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.233 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.233 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.233 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.233 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.234 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.234 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T23:03:12.234 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.234 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.234 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.234 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.234 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.234 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.234 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.234 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.234 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.234 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.234 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.234 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.235 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.235 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.235 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.235 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.235 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.235 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.235 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.235 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.235 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.235 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.235 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.235 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.235 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.235 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.236 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.236 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.236 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.236 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.236 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.236 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.236 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.236 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.236 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.236 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.236 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.236 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.236 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.237 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.237 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.237 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.237 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.237 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.237 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.237 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.237 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.237 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.237 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.237 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.237 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.237 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.237 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.238 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.238 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.238 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.238 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.238 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.238 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.238 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.238 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.238 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.238 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.238 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.238 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.238 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.239 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.239 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.239 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.239 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.239 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.239 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.239 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.239 INFO:teuthology.orchestra.run.smithi152.stdout: "orchestrator": { 2024-09-11T23:03:12.239 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "orchestrator", 2024-09-11T23:03:12.239 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.239 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.239 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.239 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.240 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.240 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.240 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.240 INFO:teuthology.orchestra.run.smithi152.stdout: "cephadm", 2024-09-11T23:03:12.240 INFO:teuthology.orchestra.run.smithi152.stdout: "rook", 2024-09-11T23:03:12.240 INFO:teuthology.orchestra.run.smithi152.stdout: "test_orchestrator" 2024-09-11T23:03:12.240 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.240 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Orchestrator backend", 2024-09-11T23:03:12.240 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.240 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.240 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.240 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.240 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.241 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "osd_perf_query", 2024-09-11T23:03:12.241 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.241 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.241 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.241 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.241 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.241 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.241 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.241 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.241 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.241 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.241 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.241 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.242 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.242 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.242 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.242 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.242 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.242 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.242 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.242 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.242 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.242 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.242 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.242 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.242 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.243 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.243 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.243 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.243 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.243 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.243 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.243 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.243 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.243 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.243 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.243 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.243 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.243 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.244 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.244 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.244 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.244 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.244 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.244 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.244 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.244 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.244 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.244 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.244 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.244 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.244 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.244 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.245 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.245 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.245 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.245 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.245 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.245 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.245 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.245 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.245 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.245 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.245 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.245 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.245 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.246 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.246 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.246 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.246 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.246 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.246 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.246 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.246 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "osd_support", 2024-09-11T23:03:12.246 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.246 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.246 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.246 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.246 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.247 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.247 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.247 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.247 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.247 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.247 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.247 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.247 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.247 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.247 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.247 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.247 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.247 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.247 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.248 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.248 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.248 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.248 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.248 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.248 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.248 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.248 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.248 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.248 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.248 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.248 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.248 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.249 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.249 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.249 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.249 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.249 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.249 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.249 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.249 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.249 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.249 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.249 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.249 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.249 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.250 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.250 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.250 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.250 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.250 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.250 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.250 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.250 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.250 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.250 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.250 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.250 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.250 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.251 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.251 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.251 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.251 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.251 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.251 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.251 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.251 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.251 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.251 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.251 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.251 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.251 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.251 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.252 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.252 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "pg_autoscaler", 2024-09-11T23:03:12.252 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.252 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.252 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.252 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.252 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.252 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.252 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.252 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.252 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.252 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.252 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.252 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.253 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.253 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.253 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.253 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.253 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.253 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.253 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.253 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.253 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.253 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.253 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.253 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.253 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.254 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.254 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.254 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.254 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.254 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.254 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.254 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.254 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.254 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.254 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.254 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.254 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.254 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.254 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.255 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.255 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.255 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.255 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.255 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.255 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.255 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.255 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.255 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.255 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.255 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.255 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.255 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.255 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.256 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.256 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.256 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.256 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.256 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.256 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.256 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.256 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.256 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.256 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.256 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.256 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.256 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.257 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.257 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.257 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.257 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.257 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.257 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.257 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-09-11T23:03:12.257 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-09-11T23:03:12.257 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.257 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.257 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.257 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-09-11T23:03:12.257 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.258 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.258 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.258 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.258 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.258 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.258 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.258 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.258 INFO:teuthology.orchestra.run.smithi152.stdout: "threshold": { 2024-09-11T23:03:12.258 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "threshold", 2024-09-11T23:03:12.258 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:12.258 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.258 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.258 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "3.0", 2024-09-11T23:03:12.258 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1.0", 2024-09-11T23:03:12.259 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.259 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.259 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "scaling threshold", 2024-09-11T23:03:12.259 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T23:03:12.259 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.259 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.259 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.259 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.259 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.259 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.259 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "progress", 2024-09-11T23:03:12.259 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.259 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.259 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.260 INFO:teuthology.orchestra.run.smithi152.stdout: "allow_pg_recovery_event": { 2024-09-11T23:03:12.260 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "allow_pg_recovery_event", 2024-09-11T23:03:12.260 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.260 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.260 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.260 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.260 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.260 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.260 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.260 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T23:03:12.260 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.260 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.260 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.261 INFO:teuthology.orchestra.run.smithi152.stdout: "enabled": { 2024-09-11T23:03:12.261 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enabled", 2024-09-11T23:03:12.261 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.261 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.261 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.261 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.261 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.261 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.261 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.261 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.261 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.261 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.261 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.261 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.262 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.262 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.262 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.262 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.262 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.262 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.262 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.262 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.262 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.262 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.262 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.262 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.262 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.262 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.263 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.263 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.263 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.263 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.263 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.263 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.263 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.263 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.263 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.263 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.263 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.263 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.263 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.263 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.264 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.264 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.264 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.264 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.264 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.264 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.264 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.264 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.264 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.264 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.264 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.264 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.264 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.265 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.265 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.265 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.265 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.265 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.265 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.265 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.265 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.265 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.265 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.265 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.265 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.265 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.265 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.265 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.266 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.266 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.266 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.266 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.266 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.266 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.266 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.266 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.266 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.266 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.266 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.266 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.266 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.266 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.267 INFO:teuthology.orchestra.run.smithi152.stdout: "max_completed_events": { 2024-09-11T23:03:12.267 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_completed_events", 2024-09-11T23:03:12.267 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.267 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.267 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.267 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "50", 2024-09-11T23:03:12.267 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.267 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.267 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.267 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "number of past completed events to remember", 2024-09-11T23:03:12.267 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.267 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.267 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.268 INFO:teuthology.orchestra.run.smithi152.stdout: "sleep_interval": { 2024-09-11T23:03:12.268 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "sleep_interval", 2024-09-11T23:03:12.268 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.268 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.268 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.268 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "5", 2024-09-11T23:03:12.268 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.268 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.268 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.268 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "how long the module is going to sleep", 2024-09-11T23:03:12.268 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.268 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.268 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.269 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.269 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.269 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.269 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "prometheus", 2024-09-11T23:03:12.269 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.269 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.269 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.269 INFO:teuthology.orchestra.run.smithi152.stdout: "cache": { 2024-09-11T23:03:12.269 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "cache", 2024-09-11T23:03:12.269 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.269 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.269 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.270 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.270 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.270 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.270 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.270 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.270 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.270 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.270 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.270 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.270 INFO:teuthology.orchestra.run.smithi152.stdout: "exclude_perf_counters": { 2024-09-11T23:03:12.270 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "exclude_perf_counters", 2024-09-11T23:03:12.270 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.270 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.270 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.271 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.271 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.271 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.271 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.271 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T23:03:12.271 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T23:03:12.271 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.271 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.271 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.271 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.271 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.271 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.272 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.272 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.272 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.272 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.272 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.272 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.272 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.272 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.272 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.272 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.272 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.272 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.272 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.272 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.273 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.273 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.273 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.273 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.273 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.273 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.273 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.273 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.273 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.273 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.273 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.273 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.274 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.274 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.274 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.274 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.274 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.274 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.274 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.274 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.274 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.274 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.274 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.274 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.274 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.275 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.275 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.275 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.275 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.275 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.275 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.275 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.275 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.275 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.275 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.275 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.275 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.276 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.276 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.276 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.276 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.276 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.276 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.276 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.276 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.276 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.276 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.276 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.276 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_stats_pools": { 2024-09-11T23:03:12.276 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_stats_pools", 2024-09-11T23:03:12.277 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.277 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.277 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.277 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.277 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.277 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.277 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.277 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.277 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.277 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.277 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.277 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.277 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T23:03:12.277 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T23:03:12.278 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.278 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.278 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.278 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "300", 2024-09-11T23:03:12.278 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.278 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.278 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.278 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.278 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.278 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.278 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.278 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.278 INFO:teuthology.orchestra.run.smithi152.stdout: "scrape_interval": { 2024-09-11T23:03:12.278 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "scrape_interval", 2024-09-11T23:03:12.279 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:12.279 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.279 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.279 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "15.0", 2024-09-11T23:03:12.279 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.279 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.279 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.279 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.279 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.279 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.279 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.279 INFO:teuthology.orchestra.run.smithi152.stdout: "server_addr": { 2024-09-11T23:03:12.279 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_addr", 2024-09-11T23:03:12.280 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.280 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.280 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.280 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "::", 2024-09-11T23:03:12.280 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.280 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.280 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.280 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T23:03:12.280 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.280 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.280 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.280 INFO:teuthology.orchestra.run.smithi152.stdout: "server_port": { 2024-09-11T23:03:12.281 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_port", 2024-09-11T23:03:12.281 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.281 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.281 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.281 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "9283", 2024-09-11T23:03:12.281 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.281 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.281 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.281 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T23:03:12.281 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.281 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.281 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.281 INFO:teuthology.orchestra.run.smithi152.stdout: "stale_cache_strategy": { 2024-09-11T23:03:12.282 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "stale_cache_strategy", 2024-09-11T23:03:12.282 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.282 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.282 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.282 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "log", 2024-09-11T23:03:12.282 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.282 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.282 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.282 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.282 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.282 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.282 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.282 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_behaviour": { 2024-09-11T23:03:12.283 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_behaviour", 2024-09-11T23:03:12.283 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.283 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.283 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.283 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "default", 2024-09-11T23:03:12.283 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.283 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.283 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.283 INFO:teuthology.orchestra.run.smithi152.stdout: "default", 2024-09-11T23:03:12.283 INFO:teuthology.orchestra.run.smithi152.stdout: "error" 2024-09-11T23:03:12.283 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.283 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.283 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.283 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.284 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.284 INFO:teuthology.orchestra.run.smithi152.stdout: "standby_error_status_code": { 2024-09-11T23:03:12.284 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "standby_error_status_code", 2024-09-11T23:03:12.284 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.284 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.284 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.284 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "500", 2024-09-11T23:03:12.284 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "400", 2024-09-11T23:03:12.284 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "599", 2024-09-11T23:03:12.284 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.284 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.284 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.284 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.285 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.285 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.285 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.285 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_support", 2024-09-11T23:03:12.285 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.285 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.285 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.285 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.285 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.285 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.285 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.286 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.286 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.286 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.286 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.286 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.286 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.286 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.286 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.286 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.286 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.286 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.286 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.286 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.287 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.287 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.287 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.287 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.287 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.287 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.287 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.287 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.287 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.287 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.287 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.287 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.287 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.288 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.288 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.288 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.288 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.288 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.288 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.288 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.288 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.288 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.288 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.288 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.288 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.289 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.289 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.289 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.289 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.289 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.289 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.289 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.289 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.289 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.289 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.289 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.289 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.289 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.290 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.290 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.290 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.290 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.290 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.290 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.290 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.290 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.290 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.290 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.290 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.290 INFO:teuthology.orchestra.run.smithi152.stdout: "max_concurrent_snap_create": { 2024-09-11T23:03:12.290 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_concurrent_snap_create", 2024-09-11T23:03:12.291 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.291 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.291 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.291 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10", 2024-09-11T23:03:12.291 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.291 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.291 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.291 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.291 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.291 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.291 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.291 INFO:teuthology.orchestra.run.smithi152.stdout: "mirror_snapshot_schedule": { 2024-09-11T23:03:12.291 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T23:03:12.292 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.292 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.292 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.292 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.292 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.292 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.292 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.292 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.292 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.292 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.292 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.292 INFO:teuthology.orchestra.run.smithi152.stdout: "trash_purge_schedule": { 2024-09-11T23:03:12.293 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "trash_purge_schedule", 2024-09-11T23:03:12.293 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.293 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.293 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.293 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.293 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.293 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.293 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.293 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.293 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.293 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.293 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.293 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.293 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.294 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.294 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "restful", 2024-09-11T23:03:12.294 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.294 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.294 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.294 INFO:teuthology.orchestra.run.smithi152.stdout: "enable_auth": { 2024-09-11T23:03:12.294 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enable_auth", 2024-09-11T23:03:12.294 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.294 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.294 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.294 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.294 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.294 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.295 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.295 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.295 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.295 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.295 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.295 INFO:teuthology.orchestra.run.smithi152.stdout: "key_file": { 2024-09-11T23:03:12.295 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "key_file", 2024-09-11T23:03:12.295 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.295 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.295 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.295 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.295 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.295 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.296 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.296 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.296 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.296 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.296 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.296 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.296 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.296 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.296 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.296 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.296 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.296 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.296 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.297 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.297 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.297 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.297 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.297 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.297 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.297 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.297 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.297 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.297 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.297 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.297 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.297 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.299 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.299 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.299 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.299 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.299 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.299 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.299 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.299 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.299 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.299 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.299 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.299 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.299 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.299 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.300 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.300 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.300 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.300 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.300 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.300 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.300 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.300 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.300 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.300 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.300 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.300 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.301 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.301 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.301 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.301 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.301 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.301 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.301 INFO:teuthology.orchestra.run.smithi152.stdout: "server_addr": { 2024-09-11T23:03:12.301 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_addr", 2024-09-11T23:03:12.301 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.301 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.301 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.301 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.301 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.302 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.302 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.302 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.302 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.302 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.302 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.302 INFO:teuthology.orchestra.run.smithi152.stdout: "server_port": { 2024-09-11T23:03:12.302 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "server_port", 2024-09-11T23:03:12.302 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.302 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.302 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.302 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.302 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.303 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.303 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.303 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.303 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.303 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.303 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.303 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.303 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.303 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rgw", 2024-09-11T23:03:12.303 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.303 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.303 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.304 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.304 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.304 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.304 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.304 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.304 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.304 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.304 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.304 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.304 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.304 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.304 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.304 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.305 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.305 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.305 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.305 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.305 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.305 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.305 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.305 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.305 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.305 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.305 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.305 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.307 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.307 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.307 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.307 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.307 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.307 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.307 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.307 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.307 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.307 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.307 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.307 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.307 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.307 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.308 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.308 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.308 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.308 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.308 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.308 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.308 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.308 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.308 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.308 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.308 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.308 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.309 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.309 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.309 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.309 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.309 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rook", 2024-09-11T23:03:12.309 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.309 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.309 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.309 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.309 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.309 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.309 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.310 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.310 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.310 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.310 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.310 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.310 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.310 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.310 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.310 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.310 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.310 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.310 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.310 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.310 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.311 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.311 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.311 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.311 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.311 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.311 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.311 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.311 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.311 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.311 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.311 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.311 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.312 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.312 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.312 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.312 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.312 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.312 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.312 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.312 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.312 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.312 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.312 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.312 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.312 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.313 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.313 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.313 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.313 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.313 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.313 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.313 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.313 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.313 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.313 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.313 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.313 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.313 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.314 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.314 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.314 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.314 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.314 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.314 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.314 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.314 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.314 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.314 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.314 INFO:teuthology.orchestra.run.smithi152.stdout: "storage_class": { 2024-09-11T23:03:12.314 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "storage_class", 2024-09-11T23:03:12.314 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.315 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.315 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.315 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "local", 2024-09-11T23:03:12.315 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.315 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.315 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.315 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T23:03:12.315 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.315 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.315 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.315 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.315 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.316 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "selftest", 2024-09-11T23:03:12.316 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.316 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.316 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.316 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.316 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.316 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.316 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.316 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.316 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.316 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.316 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.316 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.316 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.317 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.317 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.317 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.317 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.317 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.317 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.317 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.317 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.317 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.317 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.317 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.317 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.317 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.318 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.318 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.318 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.318 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.318 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.318 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.318 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.318 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.318 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.318 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.318 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.318 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.318 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.318 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.319 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.319 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.319 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.319 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.319 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.319 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.319 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.319 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.319 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.319 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.319 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.319 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.319 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.319 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.320 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.320 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.320 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.320 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.320 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.320 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.320 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.320 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.320 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.320 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.320 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.320 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.320 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.321 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.321 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.321 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.321 INFO:teuthology.orchestra.run.smithi152.stdout: "roption1": { 2024-09-11T23:03:12.321 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "roption1", 2024-09-11T23:03:12.321 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.321 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.321 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.321 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.321 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.321 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.321 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.322 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.322 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.322 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.322 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.322 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.322 INFO:teuthology.orchestra.run.smithi152.stdout: "roption2": { 2024-09-11T23:03:12.322 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "roption2", 2024-09-11T23:03:12.322 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.322 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.322 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.322 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "xyz", 2024-09-11T23:03:12.322 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.322 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.322 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.323 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.323 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.323 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.323 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.323 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.323 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption1": { 2024-09-11T23:03:12.323 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption1", 2024-09-11T23:03:12.323 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.323 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.323 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.323 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.323 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.323 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.324 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.324 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.324 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.324 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.324 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.324 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.324 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption2": { 2024-09-11T23:03:12.324 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption2", 2024-09-11T23:03:12.324 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.324 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.324 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.324 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.324 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.325 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.325 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.325 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.325 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.325 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.325 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.325 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption3": { 2024-09-11T23:03:12.325 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption3", 2024-09-11T23:03:12.325 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "float", 2024-09-11T23:03:12.325 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.325 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.325 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.325 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.326 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.326 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.326 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.326 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.326 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.326 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.326 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption4": { 2024-09-11T23:03:12.326 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption4", 2024-09-11T23:03:12.326 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.326 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.326 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.326 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.326 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.327 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.327 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.327 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.327 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.327 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.327 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.327 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption5": { 2024-09-11T23:03:12.327 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption5", 2024-09-11T23:03:12.327 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.327 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.327 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.327 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.327 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.328 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.328 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.328 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.328 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.328 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.328 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.328 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.328 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption6": { 2024-09-11T23:03:12.328 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption6", 2024-09-11T23:03:12.328 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.328 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.328 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.328 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.329 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.329 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.329 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.329 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.329 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.329 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.329 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.329 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.329 INFO:teuthology.orchestra.run.smithi152.stdout: "rwoption7": { 2024-09-11T23:03:12.329 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rwoption7", 2024-09-11T23:03:12.329 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.329 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.329 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.329 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.330 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "1", 2024-09-11T23:03:12.330 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "42", 2024-09-11T23:03:12.330 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.330 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.330 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.330 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.330 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.330 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.330 INFO:teuthology.orchestra.run.smithi152.stdout: "testkey": { 2024-09-11T23:03:12.330 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "testkey", 2024-09-11T23:03:12.330 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.330 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.330 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.330 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.331 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.331 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.331 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.331 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.331 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.331 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.331 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.331 INFO:teuthology.orchestra.run.smithi152.stdout: "testlkey": { 2024-09-11T23:03:12.331 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "testlkey", 2024-09-11T23:03:12.331 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.331 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.331 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.331 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.332 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.332 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.332 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.332 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.332 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.332 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.332 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.332 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.332 INFO:teuthology.orchestra.run.smithi152.stdout: "testnewline": { 2024-09-11T23:03:12.332 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "testnewline", 2024-09-11T23:03:12.332 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.332 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.332 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.332 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.333 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.333 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.333 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.333 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.333 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.333 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.333 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.333 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.333 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.333 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.333 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.333 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snap_schedule", 2024-09-11T23:03:12.333 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.334 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.334 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.334 INFO:teuthology.orchestra.run.smithi152.stdout: "allow_m_granularity": { 2024-09-11T23:03:12.334 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "allow_m_granularity", 2024-09-11T23:03:12.334 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.334 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.334 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.334 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.334 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.334 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.334 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.334 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T23:03:12.334 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.334 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.335 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.335 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.335 INFO:teuthology.orchestra.run.smithi152.stdout: "dump_on_update": { 2024-09-11T23:03:12.335 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "dump_on_update", 2024-09-11T23:03:12.335 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.335 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.335 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.335 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.335 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.335 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.335 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.335 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "dump database to debug log on update", 2024-09-11T23:03:12.335 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.335 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.336 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.336 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.336 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.336 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.336 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.336 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.336 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.336 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.336 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.336 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.336 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.336 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.336 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.336 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.337 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.337 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.337 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.337 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.337 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.337 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.337 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.337 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.337 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.337 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.337 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.337 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.337 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.337 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.338 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.338 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.338 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.338 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.338 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.338 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.338 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.338 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.338 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.338 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.338 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.338 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.338 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.338 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.339 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.339 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.339 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.339 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.339 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.339 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.339 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.339 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.339 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.339 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.339 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.339 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.339 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.340 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.340 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.340 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.340 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.340 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.340 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.340 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.340 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.340 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.340 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.340 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.340 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.340 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.340 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "stats", 2024-09-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "status", 2024-09-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "telegraf", 2024-09-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi152.stdout: "address": { 2024-09-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "address", 2024-09-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-09-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-09-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "15", 2024-09-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "telemetry", 2024-09-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_basic": { 2024-09-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_basic", 2024-09-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_crash": { 2024-09-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_crash", 2024-09-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_device": { 2024-09-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_device", 2024-09-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_ident": { 2024-09-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_ident", 2024-09-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi152.stdout: "channel_perf": { 2024-09-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "channel_perf", 2024-09-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi152.stdout: "contact": { 2024-09-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "contact", 2024-09-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi152.stdout: "description": { 2024-09-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "description", 2024-09-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi152.stdout: "device_url": { 2024-09-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "device_url", 2024-09-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi152.stdout: "enabled": { 2024-09-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "enabled", 2024-09-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-09-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-09-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "24", 2024-09-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "8", 2024-09-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi152.stdout: "last_opt_revision": { 2024-09-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "last_opt_revision", 2024-09-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "1", 2024-09-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi152.stdout: "leaderboard": { 2024-09-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "leaderboard", 2024-09-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "leaderboard_description": { 2024-09-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "leaderboard_description", 2024-09-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi152.stdout: "organization": { 2024-09-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "organization", 2024-09-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi152.stdout: "proxy": { 2024-09-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "proxy", 2024-09-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi152.stdout: "url": { 2024-09-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "url", 2024-09-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "test_orchestrator", 2024-09-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "volumes", 2024-09-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.391 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.391 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.391 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.391 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.391 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.391 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.391 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.391 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.391 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.391 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.391 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.391 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.391 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.391 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.392 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.392 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.392 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.392 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.392 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.392 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.392 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.392 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.392 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.392 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.392 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.392 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi152.stdout: "max_concurrent_clones": { 2024-09-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "max_concurrent_clones", 2024-09-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "4", 2024-09-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.394 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.394 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T23:03:12.394 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.394 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.394 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.394 INFO:teuthology.orchestra.run.smithi152.stdout: "periodic_async_work": { 2024-09-11T23:03:12.394 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "periodic_async_work", 2024-09-11T23:03:12.394 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.394 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.394 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.394 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.394 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.395 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.395 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.395 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Periodically check for async work", 2024-09-11T23:03:12.395 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.395 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.395 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.395 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot_clone_delay": { 2024-09-11T23:03:12.395 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snapshot_clone_delay", 2024-09-11T23:03:12.395 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.395 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.395 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.395 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "0", 2024-09-11T23:03:12.395 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.396 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.396 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.396 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T23:03:12.396 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.396 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.396 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.396 INFO:teuthology.orchestra.run.smithi152.stdout: "snapshot_clone_no_wait": { 2024-09-11T23:03:12.396 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T23:03:12.396 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.396 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.396 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.396 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "True", 2024-09-11T23:03:12.396 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.397 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.397 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.397 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T23:03:12.397 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.397 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.397 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.397 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.397 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.397 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix", 2024-09-11T23:03:12.397 INFO:teuthology.orchestra.run.smithi152.stdout: "can_run": true, 2024-09-11T23:03:12.397 INFO:teuthology.orchestra.run.smithi152.stdout: "error_string": "", 2024-09-11T23:03:12.398 INFO:teuthology.orchestra.run.smithi152.stdout: "module_options": { 2024-09-11T23:03:12.398 INFO:teuthology.orchestra.run.smithi152.stdout: "discovery_interval": { 2024-09-11T23:03:12.398 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "discovery_interval", 2024-09-11T23:03:12.398 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "uint", 2024-09-11T23:03:12.398 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.398 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.398 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "100", 2024-09-11T23:03:12.398 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.398 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.398 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.398 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.398 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.398 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.398 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.399 INFO:teuthology.orchestra.run.smithi152.stdout: "identifier": { 2024-09-11T23:03:12.399 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "identifier", 2024-09-11T23:03:12.399 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.399 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.399 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.399 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.399 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.399 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.399 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.399 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.399 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.399 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.399 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.400 INFO:teuthology.orchestra.run.smithi152.stdout: "interval": { 2024-09-11T23:03:12.400 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "interval", 2024-09-11T23:03:12.400 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "secs", 2024-09-11T23:03:12.400 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.400 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.400 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "60", 2024-09-11T23:03:12.400 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.400 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.400 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.400 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.400 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.400 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.400 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.401 INFO:teuthology.orchestra.run.smithi152.stdout: "log_level": { 2024-09-11T23:03:12.401 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_level", 2024-09-11T23:03:12.401 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.401 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.401 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.401 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.401 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.401 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.401 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.401 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.401 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.401 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.401 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.402 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.402 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.402 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.402 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.402 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.402 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.402 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.402 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster": { 2024-09-11T23:03:12.402 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster", 2024-09-11T23:03:12.402 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.402 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.402 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.402 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.403 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.403 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.403 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.403 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.403 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.403 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.403 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.403 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_cluster_level": { 2024-09-11T23:03:12.403 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_cluster_level", 2024-09-11T23:03:12.403 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.403 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.403 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.403 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "info", 2024-09-11T23:03:12.404 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.404 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.404 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [ 2024-09-11T23:03:12.404 INFO:teuthology.orchestra.run.smithi152.stdout: "", 2024-09-11T23:03:12.404 INFO:teuthology.orchestra.run.smithi152.stdout: "critical", 2024-09-11T23:03:12.404 INFO:teuthology.orchestra.run.smithi152.stdout: "debug", 2024-09-11T23:03:12.404 INFO:teuthology.orchestra.run.smithi152.stdout: "error", 2024-09-11T23:03:12.404 INFO:teuthology.orchestra.run.smithi152.stdout: "info", 2024-09-11T23:03:12.404 INFO:teuthology.orchestra.run.smithi152.stdout: "warning" 2024-09-11T23:03:12.404 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.404 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.404 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.404 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.404 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.405 INFO:teuthology.orchestra.run.smithi152.stdout: "log_to_file": { 2024-09-11T23:03:12.405 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "log_to_file", 2024-09-11T23:03:12.405 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "bool", 2024-09-11T23:03:12.405 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.405 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 1, 2024-09-11T23:03:12.405 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "False", 2024-09-11T23:03:12.405 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.405 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.405 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.405 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.405 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.405 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.405 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.406 INFO:teuthology.orchestra.run.smithi152.stdout: "zabbix_host": { 2024-09-11T23:03:12.406 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix_host", 2024-09-11T23:03:12.406 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.406 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.406 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.406 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "", 2024-09-11T23:03:12.406 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.406 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.406 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.406 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.406 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.406 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.406 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.407 INFO:teuthology.orchestra.run.smithi152.stdout: "zabbix_port": { 2024-09-11T23:03:12.407 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix_port", 2024-09-11T23:03:12.407 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "int", 2024-09-11T23:03:12.407 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.407 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.407 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "10051", 2024-09-11T23:03:12.407 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.407 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.407 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.407 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.407 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.407 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.407 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.408 INFO:teuthology.orchestra.run.smithi152.stdout: "zabbix_sender": { 2024-09-11T23:03:12.408 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "zabbix_sender", 2024-09-11T23:03:12.408 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "str", 2024-09-11T23:03:12.408 INFO:teuthology.orchestra.run.smithi152.stdout: "level": "advanced", 2024-09-11T23:03:12.408 INFO:teuthology.orchestra.run.smithi152.stdout: "flags": 0, 2024-09-11T23:03:12.408 INFO:teuthology.orchestra.run.smithi152.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T23:03:12.408 INFO:teuthology.orchestra.run.smithi152.stdout: "min": "", 2024-09-11T23:03:12.408 INFO:teuthology.orchestra.run.smithi152.stdout: "max": "", 2024-09-11T23:03:12.408 INFO:teuthology.orchestra.run.smithi152.stdout: "enum_allowed": [], 2024-09-11T23:03:12.408 INFO:teuthology.orchestra.run.smithi152.stdout: "desc": "", 2024-09-11T23:03:12.408 INFO:teuthology.orchestra.run.smithi152.stdout: "long_desc": "", 2024-09-11T23:03:12.408 INFO:teuthology.orchestra.run.smithi152.stdout: "tags": [], 2024-09-11T23:03:12.409 INFO:teuthology.orchestra.run.smithi152.stdout: "see_also": [] 2024-09-11T23:03:12.409 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.409 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.409 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.409 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.409 INFO:teuthology.orchestra.run.smithi152.stdout: "services": { 2024-09-11T23:03:12.409 INFO:teuthology.orchestra.run.smithi152.stdout: "dashboard": "https://172.21.15.152:8443/" 2024-09-11T23:03:12.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.409 INFO:teuthology.orchestra.run.smithi152.stdout: "always_on_modules": { 2024-09-11T23:03:12.409 INFO:teuthology.orchestra.run.smithi152.stdout: "octopus": [ 2024-09-11T23:03:12.409 INFO:teuthology.orchestra.run.smithi152.stdout: "balancer", 2024-09-11T23:03:12.409 INFO:teuthology.orchestra.run.smithi152.stdout: "crash", 2024-09-11T23:03:12.409 INFO:teuthology.orchestra.run.smithi152.stdout: "devicehealth", 2024-09-11T23:03:12.409 INFO:teuthology.orchestra.run.smithi152.stdout: "orchestrator", 2024-09-11T23:03:12.410 INFO:teuthology.orchestra.run.smithi152.stdout: "pg_autoscaler", 2024-09-11T23:03:12.410 INFO:teuthology.orchestra.run.smithi152.stdout: "progress", 2024-09-11T23:03:12.410 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_support", 2024-09-11T23:03:12.410 INFO:teuthology.orchestra.run.smithi152.stdout: "status", 2024-09-11T23:03:12.410 INFO:teuthology.orchestra.run.smithi152.stdout: "telemetry", 2024-09-11T23:03:12.410 INFO:teuthology.orchestra.run.smithi152.stdout: "volumes" 2024-09-11T23:03:12.410 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.410 INFO:teuthology.orchestra.run.smithi152.stdout: "pacific": [ 2024-09-11T23:03:12.410 INFO:teuthology.orchestra.run.smithi152.stdout: "balancer", 2024-09-11T23:03:12.410 INFO:teuthology.orchestra.run.smithi152.stdout: "crash", 2024-09-11T23:03:12.410 INFO:teuthology.orchestra.run.smithi152.stdout: "devicehealth", 2024-09-11T23:03:12.410 INFO:teuthology.orchestra.run.smithi152.stdout: "orchestrator", 2024-09-11T23:03:12.410 INFO:teuthology.orchestra.run.smithi152.stdout: "pg_autoscaler", 2024-09-11T23:03:12.410 INFO:teuthology.orchestra.run.smithi152.stdout: "progress", 2024-09-11T23:03:12.411 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_support", 2024-09-11T23:03:12.411 INFO:teuthology.orchestra.run.smithi152.stdout: "status", 2024-09-11T23:03:12.411 INFO:teuthology.orchestra.run.smithi152.stdout: "telemetry", 2024-09-11T23:03:12.411 INFO:teuthology.orchestra.run.smithi152.stdout: "volumes" 2024-09-11T23:03:12.411 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.411 INFO:teuthology.orchestra.run.smithi152.stdout: "quincy": [ 2024-09-11T23:03:12.411 INFO:teuthology.orchestra.run.smithi152.stdout: "balancer", 2024-09-11T23:03:12.411 INFO:teuthology.orchestra.run.smithi152.stdout: "crash", 2024-09-11T23:03:12.411 INFO:teuthology.orchestra.run.smithi152.stdout: "devicehealth", 2024-09-11T23:03:12.411 INFO:teuthology.orchestra.run.smithi152.stdout: "orchestrator", 2024-09-11T23:03:12.411 INFO:teuthology.orchestra.run.smithi152.stdout: "pg_autoscaler", 2024-09-11T23:03:12.411 INFO:teuthology.orchestra.run.smithi152.stdout: "progress", 2024-09-11T23:03:12.411 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_support", 2024-09-11T23:03:12.411 INFO:teuthology.orchestra.run.smithi152.stdout: "status", 2024-09-11T23:03:12.412 INFO:teuthology.orchestra.run.smithi152.stdout: "telemetry", 2024-09-11T23:03:12.412 INFO:teuthology.orchestra.run.smithi152.stdout: "volumes" 2024-09-11T23:03:12.412 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.412 INFO:teuthology.orchestra.run.smithi152.stdout: "reef": [ 2024-09-11T23:03:12.412 INFO:teuthology.orchestra.run.smithi152.stdout: "balancer", 2024-09-11T23:03:12.412 INFO:teuthology.orchestra.run.smithi152.stdout: "crash", 2024-09-11T23:03:12.412 INFO:teuthology.orchestra.run.smithi152.stdout: "devicehealth", 2024-09-11T23:03:12.412 INFO:teuthology.orchestra.run.smithi152.stdout: "orchestrator", 2024-09-11T23:03:12.412 INFO:teuthology.orchestra.run.smithi152.stdout: "pg_autoscaler", 2024-09-11T23:03:12.412 INFO:teuthology.orchestra.run.smithi152.stdout: "progress", 2024-09-11T23:03:12.412 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_support", 2024-09-11T23:03:12.412 INFO:teuthology.orchestra.run.smithi152.stdout: "status", 2024-09-11T23:03:12.412 INFO:teuthology.orchestra.run.smithi152.stdout: "telemetry", 2024-09-11T23:03:12.412 INFO:teuthology.orchestra.run.smithi152.stdout: "volumes" 2024-09-11T23:03:12.413 INFO:teuthology.orchestra.run.smithi152.stdout: ], 2024-09-11T23:03:12.413 INFO:teuthology.orchestra.run.smithi152.stdout: "squid": [ 2024-09-11T23:03:12.413 INFO:teuthology.orchestra.run.smithi152.stdout: "balancer", 2024-09-11T23:03:12.413 INFO:teuthology.orchestra.run.smithi152.stdout: "crash", 2024-09-11T23:03:12.413 INFO:teuthology.orchestra.run.smithi152.stdout: "devicehealth", 2024-09-11T23:03:12.413 INFO:teuthology.orchestra.run.smithi152.stdout: "orchestrator", 2024-09-11T23:03:12.413 INFO:teuthology.orchestra.run.smithi152.stdout: "pg_autoscaler", 2024-09-11T23:03:12.413 INFO:teuthology.orchestra.run.smithi152.stdout: "progress", 2024-09-11T23:03:12.413 INFO:teuthology.orchestra.run.smithi152.stdout: "rbd_support", 2024-09-11T23:03:12.413 INFO:teuthology.orchestra.run.smithi152.stdout: "status", 2024-09-11T23:03:12.413 INFO:teuthology.orchestra.run.smithi152.stdout: "telemetry", 2024-09-11T23:03:12.413 INFO:teuthology.orchestra.run.smithi152.stdout: "volumes" 2024-09-11T23:03:12.413 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-11T23:03:12.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.414 INFO:teuthology.orchestra.run.smithi152.stdout: "last_failure_osd_epoch": 84, 2024-09-11T23:03:12.414 INFO:teuthology.orchestra.run.smithi152.stdout: "active_clients": [ 2024-09-11T23:03:12.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.414 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "devicehealth", 2024-09-11T23:03:12.414 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-09-11T23:03:12.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.414 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-09-11T23:03:12.414 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-09-11T23:03:12.414 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 2689228366 2024-09-11T23:03:12.414 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.414 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-11T23:03:12.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.414 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "libcephsqlite", 2024-09-11T23:03:12.415 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-09-11T23:03:12.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.415 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-09-11T23:03:12.415 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-09-11T23:03:12.415 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 3607176118 2024-09-11T23:03:12.415 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.415 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-11T23:03:12.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.415 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "rbd_support", 2024-09-11T23:03:12.415 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-09-11T23:03:12.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.415 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-09-11T23:03:12.415 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-09-11T23:03:12.416 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 2387764275 2024-09-11T23:03:12.416 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.416 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-11T23:03:12.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.416 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snap_schedule", 2024-09-11T23:03:12.416 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-09-11T23:03:12.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.416 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-09-11T23:03:12.416 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-09-11T23:03:12.416 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 834794573 2024-09-11T23:03:12.416 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.416 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-11T23:03:12.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.417 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "snap_schedule", 2024-09-11T23:03:12.417 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-09-11T23:03:12.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.417 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-09-11T23:03:12.417 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-09-11T23:03:12.417 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 1635861054 2024-09-11T23:03:12.417 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.417 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-11T23:03:12.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:12.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.417 INFO:teuthology.orchestra.run.smithi152.stdout: "name": "volumes", 2024-09-11T23:03:12.417 INFO:teuthology.orchestra.run.smithi152.stdout: "addrvec": [ 2024-09-11T23:03:12.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2024-09-11T23:03:12.418 INFO:teuthology.orchestra.run.smithi152.stdout: "type": "v2", 2024-09-11T23:03:12.418 INFO:teuthology.orchestra.run.smithi152.stdout: "addr": "172.21.15.152:0", 2024-09-11T23:03:12.418 INFO:teuthology.orchestra.run.smithi152.stdout: "nonce": 2921414613 2024-09-11T23:03:12.418 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.418 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-11T23:03:12.418 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:12.418 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-09-11T23:03:12.418 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:03:12.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:11.078+0000 7f46d944c640 1 -- 172.21.15.152:0/340565991 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f46bc0a2d00 msgr2=0x7f46bc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:12.419 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:11.078+0000 7f46d944c640 1 --2- 172.21.15.152:0/340565991 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f46bc0a2d00 0x7f46bc0a51c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f46d4100040 tx=0x7f46c4005f90 comp rx=0 tx=0).stop 2024-09-11T23:03:12.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:11.078+0000 7f46d944c640 1 -- 172.21.15.152:0/340565991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46d4100980 msgr2=0x7f46d4196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:12.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:11.078+0000 7f46d944c640 1 --2- 172.21.15.152:0/340565991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46d4100980 0x7f46d4196280 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009230 tx=0x7f46c8004270 comp rx=0 tx=0).stop 2024-09-11T23:03:12.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:11.078+0000 7f46d944c640 1 -- 172.21.15.152:0/340565991 shutdown_connections 2024-09-11T23:03:12.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:11.078+0000 7f46d944c640 1 --2- 172.21.15.152:0/340565991 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f46bc0a2d00 0x7f46bc0a51c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:12.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:11.078+0000 7f46d944c640 1 --2- 172.21.15.152:0/340565991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46d4106990 0x7f46d419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:12.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:11.078+0000 7f46d944c640 1 --2- 172.21.15.152:0/340565991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46d4101320 0x7f46d41967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:12.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:11.078+0000 7f46d944c640 1 --2- 172.21.15.152:0/340565991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46d4100980 0x7f46d4196280 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:12.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:11.078+0000 7f46d944c640 1 -- 172.21.15.152:0/340565991 >> 172.21.15.152:0/340565991 conn(0x7f46d4077a10 msgr2=0x7f46d41039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:12.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:11.078+0000 7f46d944c640 1 -- 172.21.15.152:0/340565991 shutdown_connections 2024-09-11T23:03:12.420 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:11.078+0000 7f46d944c640 1 -- 172.21.15.152:0/340565991 wait complete. 2024-09-11T23:03:12.421 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:10 smithi152 ceph-mon[31181]: pgmap v318: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-09-11T23:03:12.421 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:10 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2207959153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:03:12.421 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:10 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.421 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:10 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.421 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:10 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.421 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:10 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.421 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:10 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.421 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:10 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.421 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:10 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.421 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:10 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.421 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:10 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3499118641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:03:12.422 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:10 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:12.422 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:10 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:12.422 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:10 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/88676517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:03:12.422 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:10 smithi204 ceph-mon[35454]: pgmap v318: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-09-11T23:03:12.422 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:10 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2207959153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:03:12.422 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:10 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.422 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:10 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.422 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:10 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.422 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:10 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.422 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:10 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.422 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:10 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.422 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:10 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.423 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:10 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.423 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:10 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3499118641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:03:12.423 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:10 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:12.423 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:10 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:12.423 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:10 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/88676517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:03:12.423 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:11 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-11T23:03:12.423 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:11 smithi204 ceph-mon[35454]: Unable to set osd_memory_target on smithi204 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-11T23:03:12.423 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:11 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi191 to 176.0M 2024-09-11T23:03:12.423 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:11 smithi204 ceph-mon[35454]: Unable to set osd_memory_target on smithi191 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-11T23:03:12.423 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:11 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.423 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:11 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/340565991' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T23:03:12.423 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:11 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.423 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:11 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.423 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:11 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.424 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:11 smithi204 ceph-mon[35454]: Adjusting osd_memory_target on smithi152 to 176.0M 2024-09-11T23:03:12.424 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:11 smithi204 ceph-mon[35454]: Unable to set osd_memory_target on smithi152 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-11T23:03:12.424 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:11 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:03:12.424 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:11 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:03:12.424 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:11 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:12.424 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:10 smithi191 ceph-mon[35211]: pgmap v318: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-09-11T23:03:12.424 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:10 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2207959153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T23:03:12.424 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:10 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.424 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:10 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.424 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:10 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.424 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:10 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.424 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:10 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.424 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:10 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.425 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:10 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.425 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:10 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.425 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:10 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3499118641' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T23:03:12.425 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:10 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:12.425 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:10 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:12.425 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:10 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/88676517' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T23:03:12.425 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:11 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-11T23:03:12.425 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:11 smithi191 ceph-mon[35211]: Unable to set osd_memory_target on smithi204 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-11T23:03:12.425 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:11 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi191 to 176.0M 2024-09-11T23:03:12.425 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:11 smithi191 ceph-mon[35211]: Unable to set osd_memory_target on smithi191 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-11T23:03:12.425 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:11 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.425 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:11 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/340565991' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T23:03:12.425 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:11 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.426 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:11 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.426 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:11 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.426 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:11 smithi191 ceph-mon[35211]: Adjusting osd_memory_target on smithi152 to 176.0M 2024-09-11T23:03:12.426 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:11 smithi191 ceph-mon[35211]: Unable to set osd_memory_target on smithi152 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-11T23:03:12.426 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:11 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:03:12.426 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:11 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:03:12.426 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:11 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:12.432 DEBUG:tasks.check_counter:Getting stats from a 2024-09-11T23:03:12.433 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-11T23:03:12.433 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:11 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi204 to 176.0M 2024-09-11T23:03:12.433 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:11 smithi152 ceph-mon[31181]: Unable to set osd_memory_target on smithi204 to 184563097: error parsing value: Value '184563097' is below minimum 939524096 2024-09-11T23:03:12.433 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:11 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi191 to 176.0M 2024-09-11T23:03:12.433 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:11 smithi152 ceph-mon[31181]: Unable to set osd_memory_target on smithi191 to 184596070: error parsing value: Value '184596070' is below minimum 939524096 2024-09-11T23:03:12.433 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:11 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.434 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:11 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/340565991' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T23:03:12.434 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:11 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.434 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:11 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.434 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:11 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T23:03:12.434 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:11 smithi152 ceph-mon[31181]: Adjusting osd_memory_target on smithi152 to 176.0M 2024-09-11T23:03:12.434 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:11 smithi152 ceph-mon[31181]: Unable to set osd_memory_target on smithi152 to 184580300: error parsing value: Value '184580300' is below minimum 939524096 2024-09-11T23:03:12.434 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:11 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T23:03:12.434 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:11 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T23:03:12.434 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:11 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' 2024-09-11T23:03:12.692 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:13.221 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:03:13.222 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:03:13.222 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 405, 2024-09-11T23:03:13.222 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 671, 2024-09-11T23:03:13.222 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 115933, 2024-09-11T23:03:13.222 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 508205, 2024-09-11T23:03:13.222 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 11, 2024-09-11T23:03:13.222 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 2, 2024-09-11T23:03:13.222 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.063949125, 2024-09-11T23:03:13.222 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.027796800, 2024-09-11T23:03:13.222 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.028721605, 2024-09-11T23:03:13.222 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.003554640, 2024-09-11T23:03:13.222 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:13.223 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 670, 2024-09-11T23:03:13.223 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.047046342, 2024-09-11T23:03:13.223 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000070218 2024-09-11T23:03:13.223 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.223 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:13.223 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 908, 2024-09-11T23:03:13.223 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000225631, 2024-09-11T23:03:13.223 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000248 2024-09-11T23:03:13.223 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.223 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-11T23:03:13.223 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 310944 2024-09-11T23:03:13.223 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.223 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:03:13.223 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 333, 2024-09-11T23:03:13.224 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 334, 2024-09-11T23:03:13.224 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 1511437, 2024-09-11T23:03:13.224 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 216237, 2024-09-11T23:03:13.224 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 13, 2024-09-11T23:03:13.224 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-09-11T23:03:13.224 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.054973260, 2024-09-11T23:03:13.224 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.022270130, 2024-09-11T23:03:13.224 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.045806006, 2024-09-11T23:03:13.224 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000202125, 2024-09-11T23:03:13.224 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:13.224 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 333, 2024-09-11T23:03:13.224 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.025847313, 2024-09-11T23:03:13.224 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000077619 2024-09-11T23:03:13.225 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.225 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:13.225 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 121, 2024-09-11T23:03:13.225 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000070891, 2024-09-11T23:03:13.225 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000585 2024-09-11T23:03:13.225 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.225 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 1486640, 2024-09-11T23:03:13.225 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 132736 2024-09-11T23:03:13.225 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.225 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:03:13.225 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 148, 2024-09-11T23:03:13.225 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 177, 2024-09-11T23:03:13.225 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 49886, 2024-09-11T23:03:13.226 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 611984, 2024-09-11T23:03:13.226 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 45, 2024-09-11T23:03:13.226 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 7, 2024-09-11T23:03:13.226 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.024556900, 2024-09-11T23:03:13.226 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.005098820, 2024-09-11T23:03:13.226 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.017863841, 2024-09-11T23:03:13.226 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000233260, 2024-09-11T23:03:13.226 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:13.226 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 174, 2024-09-11T23:03:13.226 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.037160709, 2024-09-11T23:03:13.226 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000213567 2024-09-11T23:03:13.226 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.226 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:13.226 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 219, 2024-09-11T23:03:13.227 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000049737, 2024-09-11T23:03:13.227 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000227 2024-09-11T23:03:13.227 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.227 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T23:03:13.227 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 125184 2024-09-11T23:03:13.227 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.227 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-09-11T23:03:13.227 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-09-11T23:03:13.227 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-09-11T23:03:13.227 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.227 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-09-11T23:03:13.227 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-09-11T23:03:13.227 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-09-11T23:03:13.227 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 44, 2024-09-11T23:03:13.228 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.001386580, 2024-09-11T23:03:13.228 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000031513 2024-09-11T23:03:13.228 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:13.228 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.228 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-09-11T23:03:13.228 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-09-11T23:03:13.228 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-09-11T23:03:13.228 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2, 2024-09-11T23:03:13.228 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000008572, 2024-09-11T23:03:13.228 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000004286 2024-09-11T23:03:13.228 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:13.228 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.228 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-09-11T23:03:13.229 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 1, 2024-09-11T23:03:13.229 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 0, 2024-09-11T23:03:13.229 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-09-11T23:03:13.229 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.229 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.229 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.229 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.229 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-09-11T23:03:13.229 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 1, 2024-09-11T23:03:13.229 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-09-11T23:03:13.229 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-09-11T23:03:13.229 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 11, 2024-09-11T23:03:13.229 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 0, 2024-09-11T23:03:13.229 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 0, 2024-09-11T23:03:13.230 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 10, 2024-09-11T23:03:13.230 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 0, 2024-09-11T23:03:13.230 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 10, 2024-09-11T23:03:13.230 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2024-09-11T23:03:13.230 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 10, 2024-09-11T23:03:13.230 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 0, 2024-09-11T23:03:13.230 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 0, 2024-09-11T23:03:13.230 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 0, 2024-09-11T23:03:13.230 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 1, 2024-09-11T23:03:13.230 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 6, 2024-09-11T23:03:13.230 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 6, 2024-09-11T23:03:13.230 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 0, 2024-09-11T23:03:13.230 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 0, 2024-09-11T23:03:13.230 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:03:13.231 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-09-11T23:03:13.231 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 0, 2024-09-11T23:03:13.231 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 0, 2024-09-11T23:03:13.231 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-09-11T23:03:13.231 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2024-09-11T23:03:13.231 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2024-09-11T23:03:13.231 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2024-09-11T23:03:13.231 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2024-09-11T23:03:13.231 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-09-11T23:03:13.231 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:03:13.231 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 1, 2024-09-11T23:03:13.231 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 4, 2024-09-11T23:03:13.231 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 234, 2024-09-11T23:03:13.231 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 6, 2024-09-11T23:03:13.232 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:03:13.232 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-09-11T23:03:13.232 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:03:13.232 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:03:13.232 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:03:13.232 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:03:13.232 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:03:13.232 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 0, 2024-09-11T23:03:13.232 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:03:13.232 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:03:13.232 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:03:13.232 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:03:13.232 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:03:13.233 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:03:13.233 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 0, 2024-09-11T23:03:13.233 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:03:13.233 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 0, 2024-09-11T23:03:13.233 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 0 2024-09-11T23:03:13.233 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.233 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-09-11T23:03:13.233 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 0, 2024-09-11T23:03:13.233 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 0, 2024-09-11T23:03:13.233 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 0, 2024-09-11T23:03:13.233 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 1, 2024-09-11T23:03:13.233 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 0, 2024-09-11T23:03:13.233 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-09-11T23:03:13.233 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-09-11T23:03:13.234 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:03:13.234 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 0, 2024-09-11T23:03:13.234 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 0, 2024-09-11T23:03:13.234 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-09-11T23:03:13.234 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-09-11T23:03:13.234 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-09-11T23:03:13.234 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:03:13.234 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:03:13.234 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-09-11T23:03:13.234 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-09-11T23:03:13.234 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_path": 3, 2024-09-11T23:03:13.234 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_inode": 3, 2024-09-11T23:03:13.234 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:03:13.234 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2024-09-11T23:03:13.234 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-09-11T23:03:13.235 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:03:13.235 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-09-11T23:03:13.235 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-09-11T23:03:13.235 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.235 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-09-11T23:03:13.235 INFO:teuthology.orchestra.run.smithi152.stdout: "evlrg": 0, 2024-09-11T23:03:13.235 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 3, 2024-09-11T23:03:13.235 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 0, 2024-09-11T23:03:13.235 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 0, 2024-09-11T23:03:13.235 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 3, 2024-09-11T23:03:13.235 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-09-11T23:03:13.235 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 0, 2024-09-11T23:03:13.235 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 2, 2024-09-11T23:03:13.235 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 0, 2024-09-11T23:03:13.236 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 0, 2024-09-11T23:03:13.236 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 2, 2024-09-11T23:03:13.236 INFO:teuthology.orchestra.run.smithi152.stdout: "segmjr": 2, 2024-09-11T23:03:13.236 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-09-11T23:03:13.236 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 0, 2024-09-11T23:03:13.236 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 4194304, 2024-09-11T23:03:13.236 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 4195200, 2024-09-11T23:03:13.236 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 0, 2024-09-11T23:03:13.236 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-09-11T23:03:13.236 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2, 2024-09-11T23:03:13.236 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.002905683, 2024-09-11T23:03:13.236 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.001452841 2024-09-11T23:03:13.236 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.237 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 0 2024-09-11T23:03:13.237 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.237 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-09-11T23:03:13.237 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 13, 2024-09-11T23:03:13.237 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 13, 2024-09-11T23:03:13.237 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 0, 2024-09-11T23:03:13.237 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 11, 2024-09-11T23:03:13.237 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 12, 2024-09-11T23:03:13.237 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 1, 2024-09-11T23:03:13.237 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 10, 2024-09-11T23:03:13.237 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 10, 2024-09-11T23:03:13.237 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 0, 2024-09-11T23:03:13.237 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 0, 2024-09-11T23:03:13.237 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 0, 2024-09-11T23:03:13.238 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 0, 2024-09-11T23:03:13.238 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 50232, 2024-09-11T23:03:13.238 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 223520 2024-09-11T23:03:13.238 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.238 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-09-11T23:03:13.238 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 1, 2024-09-11T23:03:13.238 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 0, 2024-09-11T23:03:13.238 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 1, 2024-09-11T23:03:13.238 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 9, 2024-09-11T23:03:13.238 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 0, 2024-09-11T23:03:13.238 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-09-11T23:03:13.238 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.238 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.238 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.239 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-09-11T23:03:13.239 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.239 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.239 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.239 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-09-11T23:03:13.239 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.239 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.239 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.239 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-09-11T23:03:13.239 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.239 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.239 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.240 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-09-11T23:03:13.240 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.240 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.240 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.240 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-09-11T23:03:13.240 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.240 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.240 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.240 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-09-11T23:03:13.240 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.241 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.241 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.241 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-09-11T23:03:13.241 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.241 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.241 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.241 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-09-11T23:03:13.241 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.241 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.241 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.241 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-09-11T23:03:13.242 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.242 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.242 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.242 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-09-11T23:03:13.242 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.242 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.242 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.242 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-09-11T23:03:13.242 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.243 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.243 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.243 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-09-11T23:03:13.243 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.243 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.244 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.244 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-09-11T23:03:13.244 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.244 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.244 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.244 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-09-11T23:03:13.244 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.244 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.244 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.244 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-09-11T23:03:13.245 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.245 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.245 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.245 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-09-11T23:03:13.245 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.245 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.245 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.245 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-09-11T23:03:13.245 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.245 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.245 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.246 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-09-11T23:03:13.246 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.246 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.246 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.246 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-09-11T23:03:13.246 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.246 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.246 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.247 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-09-11T23:03:13.247 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.247 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.247 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.247 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-09-11T23:03:13.247 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.247 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.247 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.247 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-09-11T23:03:13.247 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.247 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.247 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.248 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-09-11T23:03:13.248 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.248 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.248 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.248 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-09-11T23:03:13.248 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.248 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.248 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.248 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-09-11T23:03:13.248 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.249 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.249 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.249 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-09-11T23:03:13.249 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.249 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.249 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.249 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-09-11T23:03:13.249 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.249 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.249 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.249 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-09-11T23:03:13.250 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.250 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.250 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.250 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:03:13.250 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:03:13.250 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-09-11T23:03:13.250 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.250 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.250 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.250 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:13.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.250 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-09-11T23:03:13.251 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 1, 2024-09-11T23:03:13.251 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 1, 2024-09-11T23:03:13.251 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 0, 2024-09-11T23:03:13.251 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 1, 2024-09-11T23:03:13.251 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-09-11T23:03:13.251 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 0, 2024-09-11T23:03:13.251 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 0, 2024-09-11T23:03:13.251 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 594, 2024-09-11T23:03:13.251 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-09-11T23:03:13.251 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.251 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-09-11T23:03:13.251 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:03:13.251 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-09-11T23:03:13.251 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:03:13.252 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:03:13.252 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:03:13.252 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:03:13.252 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:03:13.252 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:03:13.252 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:03:13.252 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:03:13.252 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:03:13.252 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:03:13.252 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:03:13.252 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:03:13.252 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:03:13.252 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-09-11T23:03:13.252 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:03:13.253 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-09-11T23:03:13.253 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:03:13.253 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:03:13.253 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:03:13.253 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:03:13.253 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:03:13.253 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:03:13.253 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:03:13.253 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-09-11T23:03:13.253 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:03:13.253 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:03:13.253 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:03:13.253 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-09-11T23:03:13.253 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-09-11T23:03:13.254 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-09-11T23:03:13.254 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:03:13.254 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:03:13.254 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:03:13.254 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:03:13.254 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 270611, 2024-09-11T23:03:13.254 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 93, 2024-09-11T23:03:13.254 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:03:13.254 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-09-11T23:03:13.254 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-09-11T23:03:13.254 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-09-11T23:03:13.254 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:03:13.254 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-09-11T23:03:13.254 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:03:13.255 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-09-11T23:03:13.255 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 8160, 2024-09-11T23:03:13.255 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 181, 2024-09-11T23:03:13.255 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:03:13.255 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:03:13.255 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-09-11T23:03:13.255 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-09-11T23:03:13.255 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 50872, 2024-09-11T23:03:13.255 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 198, 2024-09-11T23:03:13.255 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-09-11T23:03:13.255 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-09-11T23:03:13.255 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-09-11T23:03:13.256 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-09-11T23:03:13.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.256 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-09-11T23:03:13.256 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-09-11T23:03:13.256 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-09-11T23:03:13.256 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 37, 2024-09-11T23:03:13.256 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 11108, 2024-09-11T23:03:13.256 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 33, 2024-09-11T23:03:13.256 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 37, 2024-09-11T23:03:13.256 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-09-11T23:03:13.256 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 37, 2024-09-11T23:03:13.256 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.205997437, 2024-09-11T23:03:13.256 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.005567498 2024-09-11T23:03:13.257 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.257 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-09-11T23:03:13.257 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-09-11T23:03:13.257 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 37, 2024-09-11T23:03:13.257 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 59 2024-09-11T23:03:13.257 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.257 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 37, 2024-09-11T23:03:13.257 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 0, 2024-09-11T23:03:13.257 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 37, 2024-09-11T23:03:13.257 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-09-11T23:03:13.257 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-09-11T23:03:13.257 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 0, 2024-09-11T23:03:13.257 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 10, 2024-09-11T23:03:13.257 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 0, 2024-09-11T23:03:13.258 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 2, 2024-09-11T23:03:13.258 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 6, 2024-09-11T23:03:13.258 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-09-11T23:03:13.258 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-09-11T23:03:13.258 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-09-11T23:03:13.258 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 0, 2024-09-11T23:03:13.258 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 6, 2024-09-11T23:03:13.258 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-09-11T23:03:13.258 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-09-11T23:03:13.258 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-09-11T23:03:13.258 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 0, 2024-09-11T23:03:13.258 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 20, 2024-09-11T23:03:13.258 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:03:13.258 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-09-11T23:03:13.259 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:03:13.259 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-09-11T23:03:13.259 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-09-11T23:03:13.259 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-09-11T23:03:13.259 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:03:13.259 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-09-11T23:03:13.259 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:03:13.259 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-09-11T23:03:13.259 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-09-11T23:03:13.259 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-09-11T23:03:13.259 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-09-11T23:03:13.259 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-09-11T23:03:13.259 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-09-11T23:03:13.259 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-09-11T23:03:13.259 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-09-11T23:03:13.260 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-09-11T23:03:13.260 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-09-11T23:03:13.260 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-09-11T23:03:13.260 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-09-11T23:03:13.260 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-09-11T23:03:13.260 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-09-11T23:03:13.260 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-09-11T23:03:13.260 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-09-11T23:03:13.260 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-09-11T23:03:13.260 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 91, 2024-09-11T23:03:13.260 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-09-11T23:03:13.260 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 10, 2024-09-11T23:03:13.260 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 9, 2024-09-11T23:03:13.261 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 9, 2024-09-11T23:03:13.261 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 0, 2024-09-11T23:03:13.261 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-09-11T23:03:13.261 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 14, 2024-09-11T23:03:13.261 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 0, 2024-09-11T23:03:13.261 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 1 2024-09-11T23:03:13.261 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.261 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-09-11T23:03:13.261 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 1, 2024-09-11T23:03:13.261 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:03:13.261 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-09-11T23:03:13.261 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-09-11T23:03:13.261 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.261 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-09-11T23:03:13.262 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 4 2024-09-11T23:03:13.262 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.262 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-09-11T23:03:13.262 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-09-11T23:03:13.262 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:03:13.262 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-09-11T23:03:13.262 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 0, 2024-09-11T23:03:13.262 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed_ops": 0, 2024-09-11T23:03:13.262 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 0, 2024-09-11T23:03:13.262 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-09-11T23:03:13.262 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.263 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:03:13.263 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-09-11T23:03:13.263 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-09-11T23:03:13.263 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-09-11T23:03:13.263 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 882, 2024-09-11T23:03:13.263 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 1581630, 2024-09-11T23:03:13.263 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:13.263 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 882, 2024-09-11T23:03:13.263 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-09-11T23:03:13.263 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-09-11T23:03:13.263 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 882, 2024-09-11T23:03:13.263 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 1581630, 2024-09-11T23:03:13.263 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-09-11T23:03:13.264 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.264 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.264 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.264 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:13.264 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.264 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-09-11T23:03:13.264 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-09-11T23:03:13.265 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-09-11T23:03:13.265 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-09-11T23:03:13.265 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-09-11T23:03:13.265 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-09-11T23:03:13.265 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:13.265 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:13.265 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 37, 2024-09-11T23:03:13.265 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 11108, 2024-09-11T23:03:13.266 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 37, 2024-09-11T23:03:13.266 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 11108, 2024-09-11T23:03:13.266 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-09-11T23:03:13.266 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.266 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.266 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.266 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:13.266 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.266 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-09-11T23:03:13.267 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-09-11T23:03:13.267 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-09-11T23:03:13.267 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-09-11T23:03:13.267 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-09-11T23:03:13.267 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-09-11T23:03:13.267 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:13.268 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:13.268 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 37, 2024-09-11T23:03:13.268 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 37, 2024-09-11T23:03:13.268 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 37, 2024-09-11T23:03:13.268 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 37, 2024-09-11T23:03:13.269 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-09-11T23:03:13.269 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.269 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.269 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.269 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:13.270 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.270 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:03:13.270 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-09-11T23:03:13.270 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-09-11T23:03:13.271 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-09-11T23:03:13.271 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-09-11T23:03:13.271 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-09-11T23:03:13.271 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:13.272 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:13.272 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-09-11T23:03:13.272 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-09-11T23:03:13.272 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-09-11T23:03:13.272 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-09-11T23:03:13.272 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-09-11T23:03:13.272 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.273 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.273 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.273 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:13.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:13.274 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x562d78dce960": { 2024-09-11T23:03:13.274 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-09-11T23:03:13.274 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-09-11T23:03:13.274 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-09-11T23:03:13.274 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 3, 2024-09-11T23:03:13.274 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 896, 2024-09-11T23:03:13.274 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:13.275 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 3, 2024-09-11T23:03:13.275 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-09-11T23:03:13.275 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-09-11T23:03:13.275 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 2, 2024-09-11T23:03:13.275 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 896, 2024-09-11T23:03:13.275 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-09-11T23:03:13.276 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:13.276 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:13.276 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:13.276 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:13.276 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:13.276 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:03:13.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:13 smithi152 ceph-mon[31181]: pgmap v319: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-09-11T23:03:13.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:13 smithi191 ceph-mon[35211]: pgmap v319: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-09-11T23:03:13.717 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:13 smithi204 ceph-mon[35454]: pgmap v319: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-09-11T23:03:14.030 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-11T23:03:14.030 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-11T23:03:14.030 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-09-11T23:03:14.030 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:03:14.030 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:03:14.030 DEBUG:tasks.check_counter:Getting stats from d 2024-09-11T23:03:14.030 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-11T23:03:14.282 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:14.837 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:03:14.838 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:03:14.838 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 222, 2024-09-11T23:03:14.838 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 210, 2024-09-11T23:03:14.838 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 1444944, 2024-09-11T23:03:14.838 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 128352, 2024-09-11T23:03:14.838 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 4, 2024-09-11T23:03:14.838 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 0, 2024-09-11T23:03:14.838 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.043768395, 2024-09-11T23:03:14.838 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.015201600, 2024-09-11T23:03:14.838 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.039325696, 2024-09-11T23:03:14.838 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:03:14.838 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:14.838 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 210, 2024-09-11T23:03:14.839 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.016182289, 2024-09-11T23:03:14.839 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000077058 2024-09-11T23:03:14.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:14.839 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:14.839 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:14.839 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:14.839 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:14.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:14.839 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 1444944, 2024-09-11T23:03:14.839 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 128352 2024-09-11T23:03:14.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:14.839 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:03:14.839 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 2, 2024-09-11T23:03:14.839 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 160, 2024-09-11T23:03:14.840 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 320, 2024-09-11T23:03:14.840 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 185296, 2024-09-11T23:03:14.840 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 5, 2024-09-11T23:03:14.840 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 3, 2024-09-11T23:03:14.840 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.014153409, 2024-09-11T23:03:14.840 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.010088335, 2024-09-11T23:03:14.840 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.001747682, 2024-09-11T23:03:14.840 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:03:14.840 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:14.840 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 157, 2024-09-11T23:03:14.840 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.012866172, 2024-09-11T23:03:14.840 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000081950 2024-09-11T23:03:14.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:14.841 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:14.841 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:14.841 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:14.841 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:14.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:14.841 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:03:14.841 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 185296 2024-09-11T23:03:14.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:14.841 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:03:14.841 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 0, 2024-09-11T23:03:14.841 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 2, 2024-09-11T23:03:14.841 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 0, 2024-09-11T23:03:14.841 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 0, 2024-09-11T23:03:14.841 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 2, 2024-09-11T23:03:14.842 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 1, 2024-09-11T23:03:14.842 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.000314650, 2024-09-11T23:03:14.842 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T23:03:14.842 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.000249218, 2024-09-11T23:03:14.842 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:03:14.842 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:14.842 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:14.842 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:14.842 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:14.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:14.842 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:14.842 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:14.842 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:14.842 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:14.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:14.843 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:03:14.843 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:03:14.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:14.843 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-09-11T23:03:14.843 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 0, 2024-09-11T23:03:14.843 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-09-11T23:03:14.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:14.843 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-09-11T23:03:14.843 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:03:14.843 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-09-11T23:03:14.843 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:03:14.843 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:03:14.843 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:03:14.844 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:03:14.844 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:03:14.844 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:03:14.844 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:03:14.844 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:03:14.844 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:03:14.844 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:03:14.844 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:03:14.844 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:03:14.844 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:03:14.844 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-09-11T23:03:14.844 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:03:14.844 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-09-11T23:03:14.845 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:03:14.845 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:03:14.845 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:03:14.845 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:03:14.845 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:03:14.845 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:03:14.845 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:03:14.845 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-09-11T23:03:14.845 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:03:14.845 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:03:14.845 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:03:14.845 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-09-11T23:03:14.845 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-09-11T23:03:14.845 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-09-11T23:03:14.846 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:03:14.846 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:03:14.846 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:03:14.846 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:03:14.846 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 148995, 2024-09-11T23:03:14.846 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 31, 2024-09-11T23:03:14.846 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:03:14.846 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-09-11T23:03:14.846 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-09-11T23:03:14.846 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-09-11T23:03:14.846 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:03:14.846 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-09-11T23:03:14.846 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:03:14.846 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-09-11T23:03:14.847 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 0, 2024-09-11T23:03:14.847 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 0, 2024-09-11T23:03:14.847 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:03:14.847 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:03:14.847 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-09-11T23:03:14.847 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-09-11T23:03:14.847 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 928, 2024-09-11T23:03:14.847 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 2, 2024-09-11T23:03:14.847 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-09-11T23:03:14.847 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-09-11T23:03:14.847 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-09-11T23:03:14.847 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-09-11T23:03:14.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:14.847 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:03:14.848 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-09-11T23:03:14.848 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-09-11T23:03:14.848 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-09-11T23:03:14.848 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 220, 2024-09-11T23:03:14.848 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 1411224, 2024-09-11T23:03:14.848 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:14.848 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 220, 2024-09-11T23:03:14.848 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-09-11T23:03:14.848 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-09-11T23:03:14.848 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 220, 2024-09-11T23:03:14.848 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 1411224, 2024-09-11T23:03:14.848 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-09-11T23:03:14.848 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:14.849 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:14.849 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:14.849 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:14.849 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:14.849 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:03:15.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:15 smithi152 ceph-mon[31181]: pgmap v320: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-09-11T23:03:15.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:15 smithi191 ceph-mon[35211]: pgmap v320: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-09-11T23:03:15.546 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-11T23:03:15.546 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-11T23:03:15.546 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-11T23:03:15.546 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-11T23:03:15.546 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-11T23:03:15.546 DEBUG:tasks.check_counter:Getting stats from g 2024-09-11T23:03:15.546 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-11T23:03:15.547 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:15 smithi204 ceph-mon[35454]: pgmap v320: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 53 KiB/s wr, 33 op/s 2024-09-11T23:03:15.792 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:16.355 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:03:16.355 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:03:16.355 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 964, 2024-09-11T23:03:16.355 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 951, 2024-09-11T23:03:16.355 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 1717824, 2024-09-11T23:03:16.355 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 360524, 2024-09-11T23:03:16.356 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 5, 2024-09-11T23:03:16.356 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 1, 2024-09-11T23:03:16.356 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.121832058, 2024-09-11T23:03:16.356 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.049669149, 2024-09-11T23:03:16.356 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.055182685, 2024-09-11T23:03:16.356 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.020956394, 2024-09-11T23:03:16.356 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:16.356 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 951, 2024-09-11T23:03:16.356 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.055916336, 2024-09-11T23:03:16.356 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000058797 2024-09-11T23:03:16.357 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.357 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:16.357 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.357 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.357 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.357 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.357 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 1472544, 2024-09-11T23:03:16.357 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 128384 2024-09-11T23:03:16.357 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.357 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:03:16.357 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 1335, 2024-09-11T23:03:16.358 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 1490, 2024-09-11T23:03:16.358 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 340878, 2024-09-11T23:03:16.358 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 863215, 2024-09-11T23:03:16.358 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 7, 2024-09-11T23:03:16.358 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 5, 2024-09-11T23:03:16.358 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.095759506, 2024-09-11T23:03:16.358 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.031620986, 2024-09-11T23:03:16.358 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.024560124, 2024-09-11T23:03:16.358 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.031583384, 2024-09-11T23:03:16.358 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:16.358 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1487, 2024-09-11T23:03:16.359 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.055308429, 2024-09-11T23:03:16.359 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000037194 2024-09-11T23:03:16.359 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.359 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:16.359 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.359 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.359 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.359 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.359 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:03:16.359 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 439264 2024-09-11T23:03:16.360 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.360 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:03:16.360 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 4, 2024-09-11T23:03:16.360 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 6, 2024-09-11T23:03:16.360 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 1629, 2024-09-11T23:03:16.360 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 1284, 2024-09-11T23:03:16.360 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 5, 2024-09-11T23:03:16.360 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-09-11T23:03:16.360 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.001206031, 2024-09-11T23:03:16.360 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.000055641, 2024-09-11T23:03:16.360 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.000986694, 2024-09-11T23:03:16.361 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000061730, 2024-09-11T23:03:16.361 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:16.361 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 4, 2024-09-11T23:03:16.361 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.002045972, 2024-09-11T23:03:16.361 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000511493 2024-09-11T23:03:16.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.361 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:16.361 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.361 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.361 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.361 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:03:16.362 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:03:16.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.362 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-09-11T23:03:16.362 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-09-11T23:03:16.362 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-09-11T23:03:16.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.362 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-09-11T23:03:16.362 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-09-11T23:03:16.362 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-09-11T23:03:16.362 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 3500, 2024-09-11T23:03:16.363 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.082390524, 2024-09-11T23:03:16.363 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000023540 2024-09-11T23:03:16.363 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:16.363 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.363 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-09-11T23:03:16.363 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-09-11T23:03:16.363 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-09-11T23:03:16.363 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.363 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.363 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.363 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:16.364 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.364 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-09-11T23:03:16.364 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 0, 2024-09-11T23:03:16.364 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 0, 2024-09-11T23:03:16.364 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-09-11T23:03:16.364 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.365 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.365 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.365 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.365 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-09-11T23:03:16.365 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 0, 2024-09-11T23:03:16.365 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-09-11T23:03:16.365 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-09-11T23:03:16.365 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 0, 2024-09-11T23:03:16.365 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 0, 2024-09-11T23:03:16.365 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 0, 2024-09-11T23:03:16.365 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 0, 2024-09-11T23:03:16.365 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 0, 2024-09-11T23:03:16.366 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 0, 2024-09-11T23:03:16.366 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2024-09-11T23:03:16.366 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 0, 2024-09-11T23:03:16.366 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 0, 2024-09-11T23:03:16.366 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 0, 2024-09-11T23:03:16.366 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 0, 2024-09-11T23:03:16.366 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 1, 2024-09-11T23:03:16.366 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 0, 2024-09-11T23:03:16.366 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 0, 2024-09-11T23:03:16.366 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 0, 2024-09-11T23:03:16.366 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 0, 2024-09-11T23:03:16.366 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:03:16.366 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-09-11T23:03:16.366 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 0, 2024-09-11T23:03:16.367 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 0, 2024-09-11T23:03:16.367 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-09-11T23:03:16.367 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2024-09-11T23:03:16.367 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2024-09-11T23:03:16.367 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2024-09-11T23:03:16.367 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2024-09-11T23:03:16.367 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-09-11T23:03:16.367 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:03:16.367 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2024-09-11T23:03:16.367 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 0, 2024-09-11T23:03:16.367 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 0, 2024-09-11T23:03:16.367 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 0, 2024-09-11T23:03:16.367 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:03:16.367 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-09-11T23:03:16.368 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:03:16.368 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:03:16.368 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:03:16.368 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:03:16.368 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:03:16.368 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 0, 2024-09-11T23:03:16.368 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:03:16.368 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:03:16.368 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:03:16.368 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:03:16.368 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:03:16.368 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:03:16.368 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 0, 2024-09-11T23:03:16.369 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:03:16.369 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 0, 2024-09-11T23:03:16.369 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 0 2024-09-11T23:03:16.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.369 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-09-11T23:03:16.369 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 0, 2024-09-11T23:03:16.369 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 0, 2024-09-11T23:03:16.369 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 0, 2024-09-11T23:03:16.369 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 0, 2024-09-11T23:03:16.369 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 0, 2024-09-11T23:03:16.370 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-09-11T23:03:16.370 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-09-11T23:03:16.370 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:03:16.370 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 0, 2024-09-11T23:03:16.370 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 0, 2024-09-11T23:03:16.370 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-09-11T23:03:16.370 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-09-11T23:03:16.370 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-09-11T23:03:16.370 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:03:16.370 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:03:16.370 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-09-11T23:03:16.370 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-09-11T23:03:16.370 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:03:16.371 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:03:16.371 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:03:16.371 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2024-09-11T23:03:16.371 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-09-11T23:03:16.371 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:03:16.371 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-09-11T23:03:16.371 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-09-11T23:03:16.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.371 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-09-11T23:03:16.371 INFO:teuthology.orchestra.run.smithi152.stdout: "evlrg": 0, 2024-09-11T23:03:16.371 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 0, 2024-09-11T23:03:16.371 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 0, 2024-09-11T23:03:16.371 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 0, 2024-09-11T23:03:16.372 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 3, 2024-09-11T23:03:16.372 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-09-11T23:03:16.372 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 0, 2024-09-11T23:03:16.372 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 0, 2024-09-11T23:03:16.372 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 0, 2024-09-11T23:03:16.372 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 0, 2024-09-11T23:03:16.372 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 2, 2024-09-11T23:03:16.372 INFO:teuthology.orchestra.run.smithi152.stdout: "segmjr": 2, 2024-09-11T23:03:16.372 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-09-11T23:03:16.372 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 0, 2024-09-11T23:03:16.372 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 4194304, 2024-09-11T23:03:16.372 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 4195200, 2024-09-11T23:03:16.372 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 4194892, 2024-09-11T23:03:16.372 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-09-11T23:03:16.373 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.373 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.373 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.373 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 3 2024-09-11T23:03:16.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.373 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-09-11T23:03:16.373 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 3, 2024-09-11T23:03:16.373 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 3, 2024-09-11T23:03:16.373 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 0, 2024-09-11T23:03:16.373 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 1, 2024-09-11T23:03:16.373 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 1, 2024-09-11T23:03:16.373 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 0, 2024-09-11T23:03:16.374 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 0, 2024-09-11T23:03:16.374 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 0, 2024-09-11T23:03:16.374 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 0, 2024-09-11T23:03:16.374 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 0, 2024-09-11T23:03:16.374 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 0, 2024-09-11T23:03:16.374 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 0, 2024-09-11T23:03:16.374 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 50996, 2024-09-11T23:03:16.374 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 215328 2024-09-11T23:03:16.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.374 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-09-11T23:03:16.374 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 0, 2024-09-11T23:03:16.374 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 0, 2024-09-11T23:03:16.374 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 0, 2024-09-11T23:03:16.375 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 0, 2024-09-11T23:03:16.375 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 0, 2024-09-11T23:03:16.375 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-09-11T23:03:16.375 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.375 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.375 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.375 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-09-11T23:03:16.375 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.375 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.375 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.375 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-09-11T23:03:16.375 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.376 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.376 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.376 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-09-11T23:03:16.376 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.376 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.376 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.376 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-09-11T23:03:16.376 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.376 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.376 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.377 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-09-11T23:03:16.377 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.377 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.377 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.377 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-09-11T23:03:16.377 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.377 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.377 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.377 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-09-11T23:03:16.377 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.377 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.378 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.378 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-09-11T23:03:16.378 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.378 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.378 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.378 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-09-11T23:03:16.378 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.378 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.378 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.378 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-09-11T23:03:16.378 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.379 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.379 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.379 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-09-11T23:03:16.379 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.379 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.379 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.379 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-09-11T23:03:16.379 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.379 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.379 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.379 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-09-11T23:03:16.380 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.380 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.380 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.380 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-09-11T23:03:16.380 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.380 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.380 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.380 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-09-11T23:03:16.380 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.380 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.381 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.381 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.381 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-09-11T23:03:16.381 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.381 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.381 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.381 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.381 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-09-11T23:03:16.381 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.381 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.381 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.382 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-09-11T23:03:16.382 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.382 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.382 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.382 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-09-11T23:03:16.382 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.382 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.382 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.382 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-09-11T23:03:16.382 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.383 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.383 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.383 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-09-11T23:03:16.383 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.383 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.383 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.383 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-09-11T23:03:16.383 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.383 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.383 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.383 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-09-11T23:03:16.384 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.384 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.384 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.384 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-09-11T23:03:16.384 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.384 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.384 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.384 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-09-11T23:03:16.384 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.384 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.384 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.385 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-09-11T23:03:16.385 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.385 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.385 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.385 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-09-11T23:03:16.385 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.385 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.385 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.386 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.386 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-09-11T23:03:16.386 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.386 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.386 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.386 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.386 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:03:16.386 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:03:16.386 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-09-11T23:03:16.386 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.386 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.386 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.386 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:16.387 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.387 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-09-11T23:03:16.387 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 1, 2024-09-11T23:03:16.387 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 1, 2024-09-11T23:03:16.387 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 0, 2024-09-11T23:03:16.387 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 1, 2024-09-11T23:03:16.387 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-09-11T23:03:16.387 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 0, 2024-09-11T23:03:16.387 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 0, 2024-09-11T23:03:16.387 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 1686, 2024-09-11T23:03:16.387 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-09-11T23:03:16.387 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.387 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-09-11T23:03:16.387 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:03:16.388 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-09-11T23:03:16.388 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:03:16.388 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:03:16.388 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:03:16.388 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:03:16.388 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:03:16.388 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:03:16.388 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:03:16.388 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:03:16.388 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:03:16.388 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:03:16.389 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:03:16.389 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:03:16.389 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:03:16.389 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-09-11T23:03:16.389 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:03:16.389 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-09-11T23:03:16.389 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:03:16.389 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:03:16.389 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:03:16.389 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:03:16.389 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:03:16.390 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:03:16.390 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:03:16.390 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-09-11T23:03:16.390 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:03:16.390 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:03:16.390 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:03:16.390 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-09-11T23:03:16.390 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-09-11T23:03:16.390 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-09-11T23:03:16.390 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:03:16.390 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:03:16.390 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:03:16.390 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:03:16.390 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 197513, 2024-09-11T23:03:16.391 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 56, 2024-09-11T23:03:16.391 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:03:16.391 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-09-11T23:03:16.391 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-09-11T23:03:16.391 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-09-11T23:03:16.391 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:03:16.391 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-09-11T23:03:16.391 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:03:16.391 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-09-11T23:03:16.391 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 8160, 2024-09-11T23:03:16.391 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 181, 2024-09-11T23:03:16.391 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:03:16.391 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:03:16.392 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-09-11T23:03:16.392 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-09-11T23:03:16.392 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 8512, 2024-09-11T23:03:16.392 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 13, 2024-09-11T23:03:16.392 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-09-11T23:03:16.392 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-09-11T23:03:16.392 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-09-11T23:03:16.392 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-09-11T23:03:16.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.393 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-09-11T23:03:16.393 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-09-11T23:03:16.393 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-09-11T23:03:16.393 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 2067, 2024-09-11T23:03:16.393 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 21, 2024-09-11T23:03:16.393 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 3, 2024-09-11T23:03:16.393 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 2067, 2024-09-11T23:03:16.393 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-09-11T23:03:16.393 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2067, 2024-09-11T23:03:16.393 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 1.157985936, 2024-09-11T23:03:16.393 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000560225 2024-09-11T23:03:16.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.393 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-09-11T23:03:16.394 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-09-11T23:03:16.394 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2067, 2024-09-11T23:03:16.394 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 2068 2024-09-11T23:03:16.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.394 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 2067, 2024-09-11T23:03:16.394 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 2067, 2024-09-11T23:03:16.394 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 0, 2024-09-11T23:03:16.394 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-09-11T23:03:16.394 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-09-11T23:03:16.394 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 1330, 2024-09-11T23:03:16.394 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 0, 2024-09-11T23:03:16.394 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 735, 2024-09-11T23:03:16.394 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 0, 2024-09-11T23:03:16.394 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 0, 2024-09-11T23:03:16.395 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-09-11T23:03:16.395 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-09-11T23:03:16.395 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-09-11T23:03:16.395 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 0, 2024-09-11T23:03:16.395 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 0, 2024-09-11T23:03:16.395 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-09-11T23:03:16.395 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-09-11T23:03:16.395 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-09-11T23:03:16.395 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 1, 2024-09-11T23:03:16.395 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 0, 2024-09-11T23:03:16.395 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:03:16.395 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-09-11T23:03:16.396 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:03:16.396 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-09-11T23:03:16.396 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-09-11T23:03:16.396 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-09-11T23:03:16.396 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:03:16.396 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-09-11T23:03:16.396 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:03:16.396 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-09-11T23:03:16.396 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-09-11T23:03:16.396 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-09-11T23:03:16.396 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-09-11T23:03:16.396 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-09-11T23:03:16.396 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-09-11T23:03:16.397 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-09-11T23:03:16.397 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-09-11T23:03:16.397 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-09-11T23:03:16.397 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-09-11T23:03:16.397 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-09-11T23:03:16.397 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-09-11T23:03:16.397 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-09-11T23:03:16.397 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-09-11T23:03:16.397 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-09-11T23:03:16.397 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-09-11T23:03:16.397 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-09-11T23:03:16.397 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 91, 2024-09-11T23:03:16.397 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-09-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 10, 2024-09-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 6, 2024-09-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 6, 2024-09-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 0, 2024-09-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-09-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 0, 2024-09-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 2, 2024-09-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 0 2024-09-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-09-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 0, 2024-09-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-09-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-09-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-09-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 0 2024-09-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-09-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-09-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-09-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 0, 2024-09-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed_ops": 0, 2024-09-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 0, 2024-09-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-09-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-09-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-09-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-09-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 2299, 2024-09-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 1838639, 2024-09-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 2299, 2024-09-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-09-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-09-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 2299, 2024-09-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 1838639, 2024-09-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-09-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-09-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-09-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-09-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-09-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-09-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-09-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 2067, 2024-09-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 901, 2024-09-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 2067, 2024-09-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 901, 2024-09-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-09-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-09-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-09-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-09-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-09-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-09-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-09-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 2067, 2024-09-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 2067, 2024-09-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 2067, 2024-09-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 2067, 2024-09-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-09-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-09-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-09-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-09-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-09-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-09-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-09-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-09-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-09-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-09-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-09-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x55e7f2b80b40": { 2024-09-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-09-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-09-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-09-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-09-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-09-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-09-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-09-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-09-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-09-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-09-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:03:17.221 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-11T23:03:17.222 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-11T23:03:17.222 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-11T23:03:17.222 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:03:17.222 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:03:17.222 DEBUG:tasks.check_counter:Getting stats from j 2024-09-11T23:03:17.222 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-11T23:03:17.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:17 smithi152 ceph-mon[31181]: pgmap v321: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:03:17.473 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:17.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:17 smithi191 ceph-mon[35211]: pgmap v321: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:03:17.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:17 smithi204 ceph-mon[35454]: pgmap v321: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:03:17.998 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-09-11T23:03:17.999 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:03:17.999 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 0, 2024-09-11T23:03:17.999 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 2, 2024-09-11T23:03:17.999 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 0, 2024-09-11T23:03:17.999 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 0, 2024-09-11T23:03:17.999 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 4, 2024-09-11T23:03:17.999 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 0, 2024-09-11T23:03:17.999 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.001217004, 2024-09-11T23:03:17.999 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T23:03:17.999 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.000273383, 2024-09-11T23:03:17.999 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:03:17.999 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:18.000 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:18.000 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:18.000 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:18.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:18.000 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:18.000 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:18.000 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:18.000 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:18.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:18.000 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:03:18.000 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:03:18.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:18.001 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:03:18.001 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 220, 2024-09-11T23:03:18.001 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 210, 2024-09-11T23:03:18.001 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 1442496, 2024-09-11T23:03:18.001 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 129184, 2024-09-11T23:03:18.001 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 2, 2024-09-11T23:03:18.001 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 1, 2024-09-11T23:03:18.001 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.038586787, 2024-09-11T23:03:18.001 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.014489424, 2024-09-11T23:03:18.001 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.036000076, 2024-09-11T23:03:18.001 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:03:18.001 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:18.002 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 209, 2024-09-11T23:03:18.002 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.012050922, 2024-09-11T23:03:18.002 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000057659 2024-09-11T23:03:18.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:18.002 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:18.002 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:18.002 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:18.002 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:18.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:18.002 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 1442496, 2024-09-11T23:03:18.002 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 129184 2024-09-11T23:03:18.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:18.002 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:03:18.002 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 6, 2024-09-11T23:03:18.003 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 163, 2024-09-11T23:03:18.003 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 3312, 2024-09-11T23:03:18.003 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 185920, 2024-09-11T23:03:18.003 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 5, 2024-09-11T23:03:18.003 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 3, 2024-09-11T23:03:18.003 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.015557064, 2024-09-11T23:03:18.003 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.009252334, 2024-09-11T23:03:18.003 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.004111172, 2024-09-11T23:03:18.003 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T23:03:18.003 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:18.003 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 161, 2024-09-11T23:03:18.003 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.015566593, 2024-09-11T23:03:18.003 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000096686 2024-09-11T23:03:18.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:18.004 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:18.004 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:18.004 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:18.004 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:18.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:18.004 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-11T23:03:18.004 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 185920 2024-09-11T23:03:18.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:18.004 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-09-11T23:03:18.004 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 0, 2024-09-11T23:03:18.004 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-09-11T23:03:18.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:18.004 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-09-11T23:03:18.005 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:03:18.005 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-09-11T23:03:18.005 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:03:18.005 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:03:18.005 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:03:18.005 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:03:18.005 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:03:18.005 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:03:18.005 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:03:18.005 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:03:18.005 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:03:18.005 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:03:18.005 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:03:18.005 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:03:18.006 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:03:18.006 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-09-11T23:03:18.006 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:03:18.006 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-09-11T23:03:18.006 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:03:18.006 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:03:18.006 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:03:18.006 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:03:18.006 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:03:18.006 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:03:18.006 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:03:18.006 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-09-11T23:03:18.006 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:03:18.006 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:03:18.007 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:03:18.007 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-09-11T23:03:18.007 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-09-11T23:03:18.007 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-09-11T23:03:18.007 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:03:18.007 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:03:18.007 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:03:18.007 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:03:18.007 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 148977, 2024-09-11T23:03:18.007 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 30, 2024-09-11T23:03:18.007 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:03:18.007 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-09-11T23:03:18.007 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-09-11T23:03:18.007 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-09-11T23:03:18.008 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:03:18.008 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-09-11T23:03:18.008 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:03:18.008 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-09-11T23:03:18.008 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 0, 2024-09-11T23:03:18.008 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 0, 2024-09-11T23:03:18.008 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:03:18.008 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:03:18.008 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-09-11T23:03:18.008 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-09-11T23:03:18.008 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 928, 2024-09-11T23:03:18.008 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 2, 2024-09-11T23:03:18.008 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-09-11T23:03:18.008 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-09-11T23:03:18.009 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-09-11T23:03:18.009 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-09-11T23:03:18.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-09-11T23:03:18.009 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:03:18.009 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-09-11T23:03:18.009 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-09-11T23:03:18.009 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-09-11T23:03:18.009 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 222, 2024-09-11T23:03:18.009 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 1411484, 2024-09-11T23:03:18.009 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:18.009 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 222, 2024-09-11T23:03:18.009 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-09-11T23:03:18.009 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-09-11T23:03:18.009 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 222, 2024-09-11T23:03:18.010 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 1411484, 2024-09-11T23:03:18.010 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-09-11T23:03:18.010 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-09-11T23:03:18.010 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-09-11T23:03:18.010 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-09-11T23:03:18.010 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:18.010 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-09-11T23:03:18.010 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-09-11T23:03:18.768 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-11T23:03:18.768 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-11T23:03:18.768 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-11T23:03:18.768 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-11T23:03:18.768 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-11T23:03:18.768 DEBUG:tasks.check_counter:Getting stats from b 2024-09-11T23:03:18.768 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-11T23:03:19.027 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:03:19.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:19 smithi152 ceph-mon[31181]: pgmap v322: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:03:19.458 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:19 smithi191 ceph-mon[35211]: pgmap v322: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:03:19.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:19 smithi204 ceph-mon[35454]: pgmap v322: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-11T23:03:19.717 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-11T23:03:19.718 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:03:19.718 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 6206, 2024-09-11T23:03:19.718 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 6043, 2024-09-11T23:03:19.718 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 1684986, 2024-09-11T23:03:19.718 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 10673986, 2024-09-11T23:03:19.718 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 18, 2024-09-11T23:03:19.718 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 7, 2024-09-11T23:03:19.718 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.525680833, 2024-09-11T23:03:19.718 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.176526511, 2024-09-11T23:03:19.718 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.146617617, 2024-09-11T23:03:19.718 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.159432266, 2024-09-11T23:03:19.718 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:19.719 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 6035, 2024-09-11T23:03:19.719 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.639864721, 2024-09-11T23:03:19.719 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000106025 2024-09-11T23:03:19.719 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.719 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:19.719 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1189, 2024-09-11T23:03:19.719 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000285984, 2024-09-11T23:03:19.719 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000240 2024-09-11T23:03:19.719 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.719 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 97040, 2024-09-11T23:03:19.719 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 347808 2024-09-11T23:03:19.719 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.719 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:03:19.720 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 10105, 2024-09-11T23:03:19.720 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 10379, 2024-09-11T23:03:19.720 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 3305057, 2024-09-11T23:03:19.720 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 6908656, 2024-09-11T23:03:19.720 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 37, 2024-09-11T23:03:19.720 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 7, 2024-09-11T23:03:19.720 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.764032144, 2024-09-11T23:03:19.720 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.256584891, 2024-09-11T23:03:19.720 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.469021566, 2024-09-11T23:03:19.720 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.164956875, 2024-09-11T23:03:19.720 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:19.720 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 10380, 2024-09-11T23:03:19.721 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 1.107621389, 2024-09-11T23:03:19.721 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000106707 2024-09-11T23:03:19.721 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.721 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:19.721 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 11596, 2024-09-11T23:03:19.721 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.003212104, 2024-09-11T23:03:19.721 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000277 2024-09-11T23:03:19.721 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.721 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 295680, 2024-09-11T23:03:19.721 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 215008 2024-09-11T23:03:19.721 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.721 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:03:19.721 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 2835, 2024-09-11T23:03:19.722 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 2456, 2024-09-11T23:03:19.722 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 2251567, 2024-09-11T23:03:19.722 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 2568532, 2024-09-11T23:03:19.722 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 150, 2024-09-11T23:03:19.722 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 9, 2024-09-11T23:03:19.722 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.276599334, 2024-09-11T23:03:19.722 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.082465472, 2024-09-11T23:03:19.722 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.181289845, 2024-09-11T23:03:19.722 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.035403488, 2024-09-11T23:03:19.722 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:19.722 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2458, 2024-09-11T23:03:19.722 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.158630689, 2024-09-11T23:03:19.722 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000064536 2024-09-11T23:03:19.723 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.723 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:19.723 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2115, 2024-09-11T23:03:19.723 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000423144, 2024-09-11T23:03:19.723 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000200 2024-09-11T23:03:19.723 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.723 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 1559152, 2024-09-11T23:03:19.723 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 155808 2024-09-11T23:03:19.723 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.723 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-09-11T23:03:19.723 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-09-11T23:03:19.723 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-09-11T23:03:19.723 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.723 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-09-11T23:03:19.724 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-09-11T23:03:19.724 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-09-11T23:03:19.724 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 6212, 2024-09-11T23:03:19.724 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 2.319951097, 2024-09-11T23:03:19.724 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000373462 2024-09-11T23:03:19.724 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:19.724 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.724 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-09-11T23:03:19.724 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-09-11T23:03:19.724 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-09-11T23:03:19.724 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 4251, 2024-09-11T23:03:19.724 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.123057134, 2024-09-11T23:03:19.724 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000028947 2024-09-11T23:03:19.725 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:19.725 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.725 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-09-11T23:03:19.725 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 160, 2024-09-11T23:03:19.725 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 160, 2024-09-11T23:03:19.725 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-09-11T23:03:19.725 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 160, 2024-09-11T23:03:19.725 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 21.327606528, 2024-09-11T23:03:19.725 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.133297540 2024-09-11T23:03:19.725 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.725 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-09-11T23:03:19.725 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 0, 2024-09-11T23:03:19.725 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-09-11T23:03:19.726 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-09-11T23:03:19.726 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 82, 2024-09-11T23:03:19.726 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 0, 2024-09-11T23:03:19.726 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-09-11T23:03:19.726 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 36, 2024-09-11T23:03:19.726 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 10, 2024-09-11T23:03:19.726 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 26, 2024-09-11T23:03:19.726 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 0, 2024-09-11T23:03:19.726 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 21, 2024-09-11T23:03:19.726 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 0, 2024-09-11T23:03:19.726 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 3, 2024-09-11T23:03:19.726 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 4, 2024-09-11T23:03:19.726 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 2, 2024-09-11T23:03:19.727 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 221, 2024-09-11T23:03:19.727 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 211, 2024-09-11T23:03:19.727 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 0, 2024-09-11T23:03:19.727 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 0, 2024-09-11T23:03:19.727 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:03:19.727 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-09-11T23:03:19.727 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-09-11T23:03:19.727 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-09-11T23:03:19.727 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-09-11T23:03:19.727 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 0, 2024-09-11T23:03:19.727 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 0, 2024-09-11T23:03:19.727 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 0, 2024-09-11T23:03:19.727 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 0, 2024-09-11T23:03:19.728 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-09-11T23:03:19.728 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 194, 2024-09-11T23:03:19.728 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-09-11T23:03:19.728 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 4, 2024-09-11T23:03:19.728 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 234, 2024-09-11T23:03:19.728 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 6, 2024-09-11T23:03:19.728 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 194, 2024-09-11T23:03:19.728 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-09-11T23:03:19.728 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:03:19.728 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:03:19.729 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 124, 2024-09-11T23:03:19.729 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 17, 2024-09-11T23:03:19.729 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 107, 2024-09-11T23:03:19.729 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 70, 2024-09-11T23:03:19.729 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:03:19.729 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 24, 2024-09-11T23:03:19.729 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 1523, 2024-09-11T23:03:19.729 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:03:19.729 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:03:19.729 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:03:19.729 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 4569, 2024-09-11T23:03:19.730 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 4540, 2024-09-11T23:03:19.730 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 7, 2024-09-11T23:03:19.730 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 26 2024-09-11T23:03:19.730 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.730 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-09-11T23:03:19.730 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-09-11T23:03:19.730 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-09-11T23:03:19.730 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-09-11T23:03:19.730 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 0, 2024-09-11T23:03:19.730 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 8, 2024-09-11T23:03:19.730 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 2, 2024-09-11T23:03:19.731 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-09-11T23:03:19.731 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:03:19.731 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 5, 2024-09-11T23:03:19.731 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 3, 2024-09-11T23:03:19.731 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-09-11T23:03:19.731 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-09-11T23:03:19.731 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-09-11T23:03:19.731 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:03:19.731 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:03:19.731 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-09-11T23:03:19.731 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-09-11T23:03:19.731 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_quiesce_path": 3, 2024-09-11T23:03:19.731 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_quiesce_inode": 39, 2024-09-11T23:03:19.732 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 17, 2024-09-11T23:03:19.732 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 0, 2024-09-11T23:03:19.732 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-09-11T23:03:19.732 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:03:19.732 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-09-11T23:03:19.732 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-09-11T23:03:19.732 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.732 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-09-11T23:03:19.732 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-09-11T23:03:19.732 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 4709, 2024-09-11T23:03:19.732 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 4652, 2024-09-11T23:03:19.732 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 4652, 2024-09-11T23:03:19.732 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 57, 2024-09-11T23:03:19.733 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-09-11T23:03:19.733 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 1, 2024-09-11T23:03:19.733 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 19, 2024-09-11T23:03:19.733 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 18, 2024-09-11T23:03:19.733 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 18, 2024-09-11T23:03:19.733 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 1, 2024-09-11T23:03:19.733 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 1, 2024-09-11T23:03:19.733 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-09-11T23:03:19.733 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 1, 2024-09-11T23:03:19.733 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 15394124, 2024-09-11T23:03:19.734 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 15462265, 2024-09-11T23:03:19.734 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 0, 2024-09-11T23:03:19.734 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-09-11T23:03:19.734 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 4678, 2024-09-11T23:03:19.734 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 88.691683405, 2024-09-11T23:03:19.734 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.018959316 2024-09-11T23:03:19.734 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.734 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 0 2024-09-11T23:03:19.734 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.734 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-09-11T23:03:19.734 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 26, 2024-09-11T23:03:19.734 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 46, 2024-09-11T23:03:19.734 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 20, 2024-09-11T23:03:19.735 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 20, 2024-09-11T23:03:19.735 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 20, 2024-09-11T23:03:19.735 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 0, 2024-09-11T23:03:19.735 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 36, 2024-09-11T23:03:19.735 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 39, 2024-09-11T23:03:19.735 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 3, 2024-09-11T23:03:19.735 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 4, 2024-09-11T23:03:19.735 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 22, 2024-09-11T23:03:19.735 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 18, 2024-09-11T23:03:19.736 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 58472, 2024-09-11T23:03:19.736 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 231712 2024-09-11T23:03:19.736 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.736 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-09-11T23:03:19.736 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 205, 2024-09-11T23:03:19.736 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 1, 2024-09-11T23:03:19.736 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 160, 2024-09-11T23:03:19.736 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 34, 2024-09-11T23:03:19.736 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 2, 2024-09-11T23:03:19.736 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-09-11T23:03:19.737 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 7, 2024-09-11T23:03:19.737 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.006780117, 2024-09-11T23:03:19.737 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000968588 2024-09-11T23:03:19.737 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.737 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-09-11T23:03:19.737 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 97, 2024-09-11T23:03:19.737 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.017162195, 2024-09-11T23:03:19.737 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000176929 2024-09-11T23:03:19.737 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.737 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-09-11T23:03:19.737 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.737 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.737 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.738 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.738 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-09-11T23:03:19.738 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.738 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.738 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.738 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.738 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-09-11T23:03:19.738 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 4, 2024-09-11T23:03:19.738 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.001545838, 2024-09-11T23:03:19.738 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000386459 2024-09-11T23:03:19.738 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.738 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-09-11T23:03:19.738 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.739 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.739 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.739 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.739 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-09-11T23:03:19.739 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2, 2024-09-11T23:03:19.739 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000233156, 2024-09-11T23:03:19.739 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000116578 2024-09-11T23:03:19.739 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.739 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-09-11T23:03:19.739 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.739 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.739 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.739 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.740 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-09-11T23:03:19.740 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.740 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.740 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.740 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.740 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-09-11T23:03:19.740 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.740 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.740 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.740 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.740 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-09-11T23:03:19.740 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 10, 2024-09-11T23:03:19.740 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.001525248, 2024-09-11T23:03:19.741 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000152524 2024-09-11T23:03:19.741 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.741 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-09-11T23:03:19.741 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 8, 2024-09-11T23:03:19.741 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.011690262, 2024-09-11T23:03:19.742 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.001461282 2024-09-11T23:03:19.742 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.742 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-09-11T23:03:19.742 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.742 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.742 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.742 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.742 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-09-11T23:03:19.742 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 10, 2024-09-11T23:03:19.742 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.058565216, 2024-09-11T23:03:19.742 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.005856521 2024-09-11T23:03:19.742 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.743 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-09-11T23:03:19.743 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2, 2024-09-11T23:03:19.743 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000832915, 2024-09-11T23:03:19.743 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000416457 2024-09-11T23:03:19.743 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.743 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-09-11T23:03:19.743 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1, 2024-09-11T23:03:19.743 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.003868475, 2024-09-11T23:03:19.743 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.003868475 2024-09-11T23:03:19.743 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.743 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-09-11T23:03:19.743 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2, 2024-09-11T23:03:19.743 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.001933218, 2024-09-11T23:03:19.743 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000966609 2024-09-11T23:03:19.744 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.744 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-09-11T23:03:19.744 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.744 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.744 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.744 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.744 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-09-11T23:03:19.744 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.744 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.744 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.744 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.744 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-09-11T23:03:19.744 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2, 2024-09-11T23:03:19.745 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 21.179701899, 2024-09-11T23:03:19.745 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 10.589850949 2024-09-11T23:03:19.745 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.745 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-09-11T23:03:19.745 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 4, 2024-09-11T23:03:19.745 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.019415742, 2024-09-11T23:03:19.745 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.004853935 2024-09-11T23:03:19.745 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.745 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-09-11T23:03:19.745 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.745 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.745 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.745 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.746 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-09-11T23:03:19.746 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1, 2024-09-11T23:03:19.746 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000774054, 2024-09-11T23:03:19.746 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000774054 2024-09-11T23:03:19.746 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.746 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-09-11T23:03:19.746 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.746 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.746 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.746 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.746 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-09-11T23:03:19.746 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.746 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.746 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.747 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.747 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-09-11T23:03:19.747 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.747 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.747 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.747 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.747 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-09-11T23:03:19.747 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 7, 2024-09-11T23:03:19.747 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.017324300, 2024-09-11T23:03:19.747 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.002474900 2024-09-11T23:03:19.747 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.747 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-09-11T23:03:19.747 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.748 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.748 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.748 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.748 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-09-11T23:03:19.748 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 3, 2024-09-11T23:03:19.748 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.006253893, 2024-09-11T23:03:19.748 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.002084631 2024-09-11T23:03:19.748 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.748 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:03:19.748 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:03:19.748 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-09-11T23:03:19.748 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.748 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.749 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.749 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:19.749 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.749 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-09-11T23:03:19.749 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 2, 2024-09-11T23:03:19.749 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 5, 2024-09-11T23:03:19.749 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 3, 2024-09-11T23:03:19.749 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 2, 2024-09-11T23:03:19.749 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-09-11T23:03:19.749 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 27, 2024-09-11T23:03:19.749 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 13, 2024-09-11T23:03:19.749 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 592, 2024-09-11T23:03:19.750 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-09-11T23:03:19.750 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.750 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-09-11T23:03:19.750 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:03:19.750 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-09-11T23:03:19.750 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:03:19.750 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:03:19.750 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:03:19.750 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:03:19.750 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:03:19.750 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:03:19.750 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:03:19.750 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:03:19.751 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:03:19.751 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:03:19.751 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:03:19.751 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:03:19.751 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:03:19.751 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-09-11T23:03:19.751 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:03:19.751 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-09-11T23:03:19.751 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:03:19.751 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:03:19.751 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:03:19.751 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:03:19.751 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:03:19.751 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:03:19.752 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:03:19.752 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-09-11T23:03:19.752 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:03:19.752 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:03:19.752 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:03:19.752 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-09-11T23:03:19.752 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-09-11T23:03:19.752 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-09-11T23:03:19.752 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:03:19.752 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:03:19.752 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:03:19.752 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:03:19.752 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 319543, 2024-09-11T23:03:19.753 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 120, 2024-09-11T23:03:19.753 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:03:19.753 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 0, 2024-09-11T23:03:19.753 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-09-11T23:03:19.753 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-09-11T23:03:19.753 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:03:19.753 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-09-11T23:03:19.753 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:03:19.753 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-09-11T23:03:19.753 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 8160, 2024-09-11T23:03:19.753 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 181, 2024-09-11T23:03:19.753 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:03:19.753 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:03:19.754 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-09-11T23:03:19.754 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-09-11T23:03:19.754 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 129692, 2024-09-11T23:03:19.754 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 1054, 2024-09-11T23:03:19.754 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-09-11T23:03:19.754 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-09-11T23:03:19.754 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-09-11T23:03:19.754 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-09-11T23:03:19.754 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.754 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-09-11T23:03:19.754 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-09-11T23:03:19.754 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-09-11T23:03:19.754 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 10279, 2024-09-11T23:03:19.755 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 11401509, 2024-09-11T23:03:19.755 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 37, 2024-09-11T23:03:19.755 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 10279, 2024-09-11T23:03:19.755 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-09-11T23:03:19.755 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 10279, 2024-09-11T23:03:19.755 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 129.030756838, 2024-09-11T23:03:19.755 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.012552851 2024-09-11T23:03:19.755 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.755 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-09-11T23:03:19.755 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-09-11T23:03:19.755 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 10279, 2024-09-11T23:03:19.755 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 10515 2024-09-11T23:03:19.755 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.756 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 10279, 2024-09-11T23:03:19.756 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 231, 2024-09-11T23:03:19.756 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 10048, 2024-09-11T23:03:19.756 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-09-11T23:03:19.756 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-09-11T23:03:19.756 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 62, 2024-09-11T23:03:19.756 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 32, 2024-09-11T23:03:19.756 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 20, 2024-09-11T23:03:19.756 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 4681, 2024-09-11T23:03:19.756 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 68, 2024-09-11T23:03:19.756 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-09-11T23:03:19.756 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-09-11T23:03:19.756 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-09-11T23:03:19.757 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 0, 2024-09-11T23:03:19.757 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 5126, 2024-09-11T23:03:19.757 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-09-11T23:03:19.757 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-09-11T23:03:19.757 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-09-11T23:03:19.757 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 211, 2024-09-11T23:03:19.757 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 64, 2024-09-11T23:03:19.757 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:03:19.757 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-09-11T23:03:19.757 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:03:19.757 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-09-11T23:03:19.757 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-09-11T23:03:19.757 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-09-11T23:03:19.757 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:03:19.758 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-09-11T23:03:19.758 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:03:19.758 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-09-11T23:03:19.758 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-09-11T23:03:19.758 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-09-11T23:03:19.758 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-09-11T23:03:19.758 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-09-11T23:03:19.758 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-09-11T23:03:19.758 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-09-11T23:03:19.759 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-09-11T23:03:19.759 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-09-11T23:03:19.759 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-09-11T23:03:19.759 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-09-11T23:03:19.759 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-09-11T23:03:19.759 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-09-11T23:03:19.759 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-09-11T23:03:19.759 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-09-11T23:03:19.759 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-09-11T23:03:19.759 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-09-11T23:03:19.759 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 91, 2024-09-11T23:03:19.760 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-09-11T23:03:19.760 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 10, 2024-09-11T23:03:19.760 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 12, 2024-09-11T23:03:19.760 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 12, 2024-09-11T23:03:19.760 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 0, 2024-09-11T23:03:19.760 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-09-11T23:03:19.760 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 231, 2024-09-11T23:03:19.760 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 0, 2024-09-11T23:03:19.760 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 20 2024-09-11T23:03:19.760 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.760 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-09-11T23:03:19.760 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 1, 2024-09-11T23:03:19.761 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:03:19.761 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-09-11T23:03:19.761 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-09-11T23:03:19.761 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.761 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-09-11T23:03:19.761 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 28 2024-09-11T23:03:19.761 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.761 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-09-11T23:03:19.761 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-09-11T23:03:19.761 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 3584, 2024-09-11T23:03:19.761 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-09-11T23:03:19.761 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 1, 2024-09-11T23:03:19.762 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 4610, 2024-09-11T23:03:19.762 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 3, 2024-09-11T23:03:19.762 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-09-11T23:03:19.762 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.762 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:03:19.763 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:19.763 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-09-11T23:03:19.763 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:19.763 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 19142, 2024-09-11T23:03:19.763 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 5455106, 2024-09-11T23:03:19.763 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:19.763 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 19142, 2024-09-11T23:03:19.763 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-11T23:03:19.763 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-11T23:03:19.764 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 19142, 2024-09-11T23:03:19.764 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 5455106, 2024-09-11T23:03:19.764 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:19.764 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.764 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.764 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.764 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:19.764 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.764 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-09-11T23:03:19.764 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:19.764 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-09-11T23:03:19.764 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:19.765 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-11T23:03:19.765 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-11T23:03:19.765 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:19.765 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:19.765 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 10279, 2024-09-11T23:03:19.765 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 11401820, 2024-09-11T23:03:19.765 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 10279, 2024-09-11T23:03:19.765 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 11401820, 2024-09-11T23:03:19.765 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:19.766 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.766 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.766 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.766 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:19.766 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.766 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-09-11T23:03:19.766 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:19.766 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-09-11T23:03:19.766 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:19.766 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-11T23:03:19.766 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-11T23:03:19.766 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:19.766 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:19.767 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 10279, 2024-09-11T23:03:19.767 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 10279, 2024-09-11T23:03:19.767 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 10279, 2024-09-11T23:03:19.767 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 10279, 2024-09-11T23:03:19.767 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:19.767 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.767 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.767 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.767 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:19.767 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.767 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:03:19.767 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:19.767 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-09-11T23:03:19.768 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:19.768 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 3, 2024-09-11T23:03:19.768 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 311, 2024-09-11T23:03:19.768 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:19.768 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 3, 2024-09-11T23:03:19.768 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-11T23:03:19.768 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-11T23:03:19.768 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 3, 2024-09-11T23:03:19.768 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 311, 2024-09-11T23:03:19.768 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:19.768 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.768 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.768 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.769 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:19.769 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:19.769 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x564e52a6e960": { 2024-09-11T23:03:19.769 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:19.769 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-09-11T23:03:19.769 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:19.769 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 4709, 2024-09-11T23:03:19.769 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 11267961, 2024-09-11T23:03:19.769 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:19.769 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 4709, 2024-09-11T23:03:19.769 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-11T23:03:19.769 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-11T23:03:19.770 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 4678, 2024-09-11T23:03:19.770 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 11267961, 2024-09-11T23:03:19.770 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:19.770 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:19.770 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:19.770 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:19.770 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:19.770 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:19.770 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-11T23:03:20.729 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-11T23:03:20.729 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-11T23:03:20.729 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-11T23:03:20.729 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-09-11T23:03:20.729 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-09-11T23:03:20.729 DEBUG:tasks.check_counter:Getting stats from e 2024-09-11T23:03:20.729 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-11T23:03:20.989 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:03:21.263 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:21 smithi191 ceph-mon[35211]: pgmap v323: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 196 B/s wr, 15 op/s 2024-09-11T23:03:21.264 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:21 smithi191 ceph-mon[35211]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T23:03:21.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:21 smithi152 ceph-mon[31181]: pgmap v323: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 196 B/s wr, 15 op/s 2024-09-11T23:03:21.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:21 smithi152 ceph-mon[31181]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T23:03:21.517 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-11T23:03:21.517 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:03:21.517 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 2135, 2024-09-11T23:03:21.517 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 2137, 2024-09-11T23:03:21.518 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 606039, 2024-09-11T23:03:21.518 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 678937, 2024-09-11T23:03:21.518 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 6, 2024-09-11T23:03:21.518 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 2, 2024-09-11T23:03:21.518 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.156236949, 2024-09-11T23:03:21.518 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.050857936, 2024-09-11T23:03:21.518 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.040258683, 2024-09-11T23:03:21.518 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.053039344, 2024-09-11T23:03:21.518 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:21.518 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2135, 2024-09-11T23:03:21.518 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.058986444, 2024-09-11T23:03:21.518 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000027628 2024-09-11T23:03:21.519 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.519 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:21.519 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.519 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.519 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.519 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.519 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:03:21.519 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:03:21.519 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.519 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:03:21.519 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 245, 2024-09-11T23:03:21.519 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 229, 2024-09-11T23:03:21.519 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 1492748, 2024-09-11T23:03:21.519 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 132458, 2024-09-11T23:03:21.520 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 3, 2024-09-11T23:03:21.520 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 2, 2024-09-11T23:03:21.520 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.044802642, 2024-09-11T23:03:21.520 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.015189296, 2024-09-11T23:03:21.520 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.042848951, 2024-09-11T23:03:21.520 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.000064694, 2024-09-11T23:03:21.520 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:21.520 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 229, 2024-09-11T23:03:21.520 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.017295226, 2024-09-11T23:03:21.520 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000075525 2024-09-11T23:03:21.520 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.520 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:21.520 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.521 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.521 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.521 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.521 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 1492192, 2024-09-11T23:03:21.521 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 131760 2024-09-11T23:03:21.521 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.521 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:03:21.521 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 3, 2024-09-11T23:03:21.521 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 161, 2024-09-11T23:03:21.522 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 588, 2024-09-11T23:03:21.522 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 440110, 2024-09-11T23:03:21.522 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 6, 2024-09-11T23:03:21.522 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 4, 2024-09-11T23:03:21.522 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.010523268, 2024-09-11T23:03:21.522 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.006667511, 2024-09-11T23:03:21.522 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.002056366, 2024-09-11T23:03:21.522 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.000016918, 2024-09-11T23:03:21.522 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:21.522 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 158, 2024-09-11T23:03:21.522 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.013171467, 2024-09-11T23:03:21.522 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000083363 2024-09-11T23:03:21.522 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.522 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:21.523 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.523 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.523 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.523 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.523 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:03:21.523 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 439792 2024-09-11T23:03:21.523 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.523 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-09-11T23:03:21.523 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-09-11T23:03:21.523 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-09-11T23:03:21.523 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.523 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-09-11T23:03:21.523 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-09-11T23:03:21.523 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-09-11T23:03:21.524 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 3609, 2024-09-11T23:03:21.524 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.079129850, 2024-09-11T23:03:21.524 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000021925 2024-09-11T23:03:21.524 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:21.524 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.524 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-09-11T23:03:21.524 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-09-11T23:03:21.524 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-09-11T23:03:21.524 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.524 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.524 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.524 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:21.524 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.525 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-09-11T23:03:21.525 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 0, 2024-09-11T23:03:21.525 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 0, 2024-09-11T23:03:21.525 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-09-11T23:03:21.525 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.525 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.525 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.525 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.525 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-09-11T23:03:21.525 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 0, 2024-09-11T23:03:21.525 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-09-11T23:03:21.525 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-09-11T23:03:21.525 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 0, 2024-09-11T23:03:21.526 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 0, 2024-09-11T23:03:21.526 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-09-11T23:03:21.526 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 0, 2024-09-11T23:03:21.526 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 0, 2024-09-11T23:03:21.526 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 0, 2024-09-11T23:03:21.526 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 0, 2024-09-11T23:03:21.526 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 0, 2024-09-11T23:03:21.526 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 0, 2024-09-11T23:03:21.526 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 0, 2024-09-11T23:03:21.526 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 0, 2024-09-11T23:03:21.526 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 1, 2024-09-11T23:03:21.526 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 0, 2024-09-11T23:03:21.526 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 0, 2024-09-11T23:03:21.526 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 0, 2024-09-11T23:03:21.527 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 0, 2024-09-11T23:03:21.527 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:03:21.527 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-09-11T23:03:21.527 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-09-11T23:03:21.527 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-09-11T23:03:21.527 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-09-11T23:03:21.527 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 0, 2024-09-11T23:03:21.527 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 0, 2024-09-11T23:03:21.527 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 0, 2024-09-11T23:03:21.527 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 0, 2024-09-11T23:03:21.527 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-09-11T23:03:21.527 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:03:21.527 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-09-11T23:03:21.527 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 0, 2024-09-11T23:03:21.528 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 0, 2024-09-11T23:03:21.528 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 0, 2024-09-11T23:03:21.528 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:03:21.528 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-09-11T23:03:21.528 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:03:21.528 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:03:21.528 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:03:21.528 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:03:21.528 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:03:21.528 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 0, 2024-09-11T23:03:21.528 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:03:21.528 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:03:21.528 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:03:21.528 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:03:21.529 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:03:21.529 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:03:21.529 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 0, 2024-09-11T23:03:21.529 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:03:21.529 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 0, 2024-09-11T23:03:21.529 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 0 2024-09-11T23:03:21.529 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.529 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-09-11T23:03:21.529 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-09-11T23:03:21.529 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-09-11T23:03:21.529 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-09-11T23:03:21.529 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 0, 2024-09-11T23:03:21.529 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 0, 2024-09-11T23:03:21.529 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-09-11T23:03:21.530 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-09-11T23:03:21.530 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:03:21.530 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 0, 2024-09-11T23:03:21.530 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 0, 2024-09-11T23:03:21.530 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-09-11T23:03:21.530 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-09-11T23:03:21.530 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-09-11T23:03:21.530 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:03:21.530 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:03:21.530 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-09-11T23:03:21.530 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-09-11T23:03:21.530 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:03:21.530 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:03:21.530 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:03:21.531 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 0, 2024-09-11T23:03:21.531 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-09-11T23:03:21.531 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:03:21.531 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-09-11T23:03:21.531 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-09-11T23:03:21.531 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.531 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-09-11T23:03:21.531 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-09-11T23:03:21.531 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 0, 2024-09-11T23:03:21.531 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-09-11T23:03:21.531 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-09-11T23:03:21.531 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 8, 2024-09-11T23:03:21.531 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-09-11T23:03:21.531 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-09-11T23:03:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 0, 2024-09-11T23:03:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-09-11T23:03:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-09-11T23:03:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 2, 2024-09-11T23:03:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 2, 2024-09-11T23:03:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-09-11T23:03:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-09-11T23:03:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 4194304, 2024-09-11T23:03:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 4196494, 2024-09-11T23:03:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 4196353, 2024-09-11T23:03:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-09-11T23:03:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.532 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.533 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.533 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 8 2024-09-11T23:03:21.533 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.533 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-09-11T23:03:21.533 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 3, 2024-09-11T23:03:21.533 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 3, 2024-09-11T23:03:21.533 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 0, 2024-09-11T23:03:21.533 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 1, 2024-09-11T23:03:21.533 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 1, 2024-09-11T23:03:21.533 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 0, 2024-09-11T23:03:21.533 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 0, 2024-09-11T23:03:21.533 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 0, 2024-09-11T23:03:21.533 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 0, 2024-09-11T23:03:21.533 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 0, 2024-09-11T23:03:21.534 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 0, 2024-09-11T23:03:21.534 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 0, 2024-09-11T23:03:21.534 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 49020, 2024-09-11T23:03:21.534 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 215328 2024-09-11T23:03:21.534 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.534 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-09-11T23:03:21.534 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 0, 2024-09-11T23:03:21.534 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 0, 2024-09-11T23:03:21.534 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 0, 2024-09-11T23:03:21.534 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 0, 2024-09-11T23:03:21.534 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 0, 2024-09-11T23:03:21.534 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-09-11T23:03:21.534 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.534 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.535 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.535 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.535 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-09-11T23:03:21.535 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.535 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.535 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.535 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.535 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-09-11T23:03:21.535 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.535 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.535 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.535 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.535 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-09-11T23:03:21.535 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.536 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.536 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.536 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.536 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-09-11T23:03:21.536 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.536 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.536 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.536 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.536 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-09-11T23:03:21.536 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.536 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.536 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.536 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.537 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-09-11T23:03:21.537 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.537 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.537 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.537 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.537 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-09-11T23:03:21.537 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.537 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.537 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.537 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.537 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-09-11T23:03:21.537 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.537 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.537 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.538 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.538 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-09-11T23:03:21.538 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.538 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.538 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.538 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.538 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-09-11T23:03:21.538 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.538 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.538 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.538 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.538 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-09-11T23:03:21.538 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.539 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.539 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.539 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.539 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-09-11T23:03:21.539 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.539 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.540 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.540 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.540 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-09-11T23:03:21.540 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.540 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.540 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.540 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.540 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-09-11T23:03:21.540 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.540 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.540 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.540 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.540 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-09-11T23:03:21.540 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.541 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.541 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.541 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.541 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-09-11T23:03:21.541 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.541 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.541 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.541 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.541 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-09-11T23:03:21.541 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.541 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.541 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.541 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.544 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-09-11T23:03:21.544 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.544 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.544 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.545 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.545 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-09-11T23:03:21.545 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.545 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.545 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.545 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.545 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-09-11T23:03:21.545 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.545 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.545 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.545 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.545 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-09-11T23:03:21.545 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.545 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.546 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.546 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.546 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-09-11T23:03:21.546 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.546 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.546 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.546 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.546 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-09-11T23:03:21.546 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.546 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.546 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.546 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.546 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-09-11T23:03:21.546 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.547 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.547 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.547 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.547 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-09-11T23:03:21.547 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.547 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.547 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.547 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.547 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-09-11T23:03:21.547 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.547 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.547 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.547 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.548 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-09-11T23:03:21.548 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.548 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.548 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.548 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.548 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-09-11T23:03:21.548 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.548 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.548 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.548 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.548 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:03:21.548 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:03:21.548 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-09-11T23:03:21.548 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.549 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.549 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.549 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:21.549 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.549 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-09-11T23:03:21.549 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 0, 2024-09-11T23:03:21.549 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 3, 2024-09-11T23:03:21.549 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 3, 2024-09-11T23:03:21.549 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 2, 2024-09-11T23:03:21.549 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-09-11T23:03:21.549 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 0, 2024-09-11T23:03:21.549 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 0, 2024-09-11T23:03:21.549 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 1416, 2024-09-11T23:03:21.549 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-09-11T23:03:21.550 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.550 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-09-11T23:03:21.550 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:03:21.550 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-09-11T23:03:21.550 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:03:21.550 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:03:21.550 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:03:21.550 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:03:21.550 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:03:21.550 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:03:21.550 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:03:21.550 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:03:21.551 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:03:21.551 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:03:21.551 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:03:21.551 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:03:21.551 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:03:21.551 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-09-11T23:03:21.551 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:03:21.551 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-09-11T23:03:21.551 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:03:21.551 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:03:21.551 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:03:21.551 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:03:21.551 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:03:21.551 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:03:21.551 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:03:21.552 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-09-11T23:03:21.552 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:03:21.552 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:03:21.552 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:03:21.552 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-09-11T23:03:21.552 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-09-11T23:03:21.553 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-09-11T23:03:21.553 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:03:21.553 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:03:21.553 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:03:21.553 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:03:21.553 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 181370, 2024-09-11T23:03:21.553 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 49, 2024-09-11T23:03:21.553 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:03:21.553 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 0, 2024-09-11T23:03:21.553 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-09-11T23:03:21.554 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-09-11T23:03:21.554 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:03:21.554 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-09-11T23:03:21.554 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:03:21.554 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-09-11T23:03:21.554 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 8160, 2024-09-11T23:03:21.554 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 181, 2024-09-11T23:03:21.554 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:03:21.554 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:03:21.554 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-09-11T23:03:21.554 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-09-11T23:03:21.554 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 8512, 2024-09-11T23:03:21.554 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 13, 2024-09-11T23:03:21.555 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-09-11T23:03:21.555 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-09-11T23:03:21.555 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-09-11T23:03:21.555 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-09-11T23:03:21.555 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.555 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-09-11T23:03:21.555 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-09-11T23:03:21.555 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-09-11T23:03:21.555 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 2138, 2024-09-11T23:03:21.555 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 21, 2024-09-11T23:03:21.555 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 3, 2024-09-11T23:03:21.555 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 2138, 2024-09-11T23:03:21.555 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-09-11T23:03:21.555 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2138, 2024-09-11T23:03:21.556 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 1.259978089, 2024-09-11T23:03:21.556 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000589325 2024-09-11T23:03:21.556 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.556 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-09-11T23:03:21.556 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-09-11T23:03:21.556 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2138, 2024-09-11T23:03:21.556 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 2139 2024-09-11T23:03:21.556 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.556 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 2138, 2024-09-11T23:03:21.556 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 2138, 2024-09-11T23:03:21.556 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 0, 2024-09-11T23:03:21.556 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-09-11T23:03:21.557 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-09-11T23:03:21.557 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 1392, 2024-09-11T23:03:21.557 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 0, 2024-09-11T23:03:21.557 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 744, 2024-09-11T23:03:21.557 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 0, 2024-09-11T23:03:21.557 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 0, 2024-09-11T23:03:21.557 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-09-11T23:03:21.557 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-09-11T23:03:21.557 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-09-11T23:03:21.557 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 0, 2024-09-11T23:03:21.557 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 0, 2024-09-11T23:03:21.557 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-09-11T23:03:21.557 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-09-11T23:03:21.558 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-09-11T23:03:21.558 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 1, 2024-09-11T23:03:21.558 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 0, 2024-09-11T23:03:21.558 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:03:21.558 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-09-11T23:03:21.558 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:03:21.558 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-09-11T23:03:21.558 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-09-11T23:03:21.558 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-09-11T23:03:21.558 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:03:21.558 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-09-11T23:03:21.558 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:03:21.558 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-09-11T23:03:21.558 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-09-11T23:03:21.559 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-09-11T23:03:21.559 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-09-11T23:03:21.559 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-09-11T23:03:21.559 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-09-11T23:03:21.559 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-09-11T23:03:21.559 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-09-11T23:03:21.559 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-09-11T23:03:21.559 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-09-11T23:03:21.559 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-09-11T23:03:21.559 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-09-11T23:03:21.559 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-09-11T23:03:21.559 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-09-11T23:03:21.559 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-09-11T23:03:21.559 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-09-11T23:03:21.560 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-09-11T23:03:21.560 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 92, 2024-09-11T23:03:21.560 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-09-11T23:03:21.560 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 11, 2024-09-11T23:03:21.560 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 4, 2024-09-11T23:03:21.560 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 4, 2024-09-11T23:03:21.560 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 0, 2024-09-11T23:03:21.560 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-09-11T23:03:21.560 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 0, 2024-09-11T23:03:21.560 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 2, 2024-09-11T23:03:21.560 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 0 2024-09-11T23:03:21.560 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.560 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-09-11T23:03:21.560 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 0, 2024-09-11T23:03:21.561 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:03:21.561 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-09-11T23:03:21.561 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-09-11T23:03:21.561 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.561 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-09-11T23:03:21.561 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 0 2024-09-11T23:03:21.561 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.561 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-09-11T23:03:21.561 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-09-11T23:03:21.561 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:03:21.561 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-09-11T23:03:21.561 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 0, 2024-09-11T23:03:21.561 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 0, 2024-09-11T23:03:21.562 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 0, 2024-09-11T23:03:21.562 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-09-11T23:03:21.562 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.562 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:03:21.562 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:21.562 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-09-11T23:03:21.562 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:21.562 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 2379, 2024-09-11T23:03:21.562 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 1870054, 2024-09-11T23:03:21.562 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:21.562 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 2379, 2024-09-11T23:03:21.562 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-11T23:03:21.562 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-11T23:03:21.563 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2379, 2024-09-11T23:03:21.563 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 1870054, 2024-09-11T23:03:21.563 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:21.563 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.563 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.563 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.563 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:21.563 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.563 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-09-11T23:03:21.563 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:21.563 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-09-11T23:03:21.563 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:21.563 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-11T23:03:21.564 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-11T23:03:21.564 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:21.564 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:21.564 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 2138, 2024-09-11T23:03:21.564 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 2195, 2024-09-11T23:03:21.564 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2138, 2024-09-11T23:03:21.564 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 2195, 2024-09-11T23:03:21.564 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:21.564 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.564 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.564 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.564 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:21.564 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.565 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-09-11T23:03:21.565 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:21.565 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-09-11T23:03:21.565 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:21.565 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-11T23:03:21.565 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-11T23:03:21.565 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:21.565 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:21.565 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 2138, 2024-09-11T23:03:21.565 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 2138, 2024-09-11T23:03:21.565 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2138, 2024-09-11T23:03:21.565 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 2138, 2024-09-11T23:03:21.565 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:21.566 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.566 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.566 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.566 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:21.566 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.566 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:03:21.566 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:21.566 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-09-11T23:03:21.566 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:21.566 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-11T23:03:21.566 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-11T23:03:21.566 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:21.566 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:21.567 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-11T23:03:21.567 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-11T23:03:21.567 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-09-11T23:03:21.567 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-09-11T23:03:21.567 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:21.567 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.567 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.567 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.567 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:21.567 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:21.567 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x55b0608ecb40": { 2024-09-11T23:03:21.568 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:21.568 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-09-11T23:03:21.568 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:21.568 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-11T23:03:21.568 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-11T23:03:21.568 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:21.568 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:21.568 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-11T23:03:21.568 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-11T23:03:21.568 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-09-11T23:03:21.568 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-09-11T23:03:21.569 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:21.569 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:21.569 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:21.569 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:21.569 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:21.569 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:21.569 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-11T23:03:21.569 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:21 smithi204 ceph-mon[35454]: pgmap v323: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 196 B/s wr, 15 op/s 2024-09-11T23:03:21.570 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:21 smithi204 ceph-mon[35454]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T23:03:22.394 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-11T23:03:22.394 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-11T23:03:22.394 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-11T23:03:22.394 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:03:22.394 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:03:22.395 DEBUG:tasks.check_counter:Getting stats from h 2024-09-11T23:03:22.395 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-11T23:03:22.642 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:03:23.213 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-11T23:03:23.214 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:03:23.214 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 113, 2024-09-11T23:03:23.214 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 116, 2024-09-11T23:03:23.214 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 32963, 2024-09-11T23:03:23.214 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 473158, 2024-09-11T23:03:23.214 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 47, 2024-09-11T23:03:23.214 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 5, 2024-09-11T23:03:23.214 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.023683541, 2024-09-11T23:03:23.214 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.003189324, 2024-09-11T23:03:23.214 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.015750654, 2024-09-11T23:03:23.214 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.000488462, 2024-09-11T23:03:23.214 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:23.215 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 114, 2024-09-11T23:03:23.215 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.016929550, 2024-09-11T23:03:23.215 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000148504 2024-09-11T23:03:23.215 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.215 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:23.215 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 133, 2024-09-11T23:03:23.215 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000022243, 2024-09-11T23:03:23.216 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000167 2024-09-11T23:03:23.216 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.216 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-11T23:03:23.216 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 976 2024-09-11T23:03:23.216 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.216 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:03:23.216 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 341, 2024-09-11T23:03:23.216 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 448, 2024-09-11T23:03:23.217 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 1530165, 2024-09-11T23:03:23.217 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 483480, 2024-09-11T23:03:23.217 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 8, 2024-09-11T23:03:23.217 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 5, 2024-09-11T23:03:23.217 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.056816990, 2024-09-11T23:03:23.217 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.022174889, 2024-09-11T23:03:23.217 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.047767590, 2024-09-11T23:03:23.217 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.000408694, 2024-09-11T23:03:23.217 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:23.217 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 448, 2024-09-11T23:03:23.217 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.055443301, 2024-09-11T23:03:23.217 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000123757 2024-09-11T23:03:23.218 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.218 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:23.218 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 86, 2024-09-11T23:03:23.218 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000017702, 2024-09-11T23:03:23.218 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000205 2024-09-11T23:03:23.218 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.218 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 1505280, 2024-09-11T23:03:23.218 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 450000 2024-09-11T23:03:23.218 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.218 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:03:23.218 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 496, 2024-09-11T23:03:23.218 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 694, 2024-09-11T23:03:23.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 142076, 2024-09-11T23:03:23.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 435782, 2024-09-11T23:03:23.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 15, 2024-09-11T23:03:23.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 5, 2024-09-11T23:03:23.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.072768325, 2024-09-11T23:03:23.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.033763098, 2024-09-11T23:03:23.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.034527975, 2024-09-11T23:03:23.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.003688829, 2024-09-11T23:03:23.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:23.219 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 691, 2024-09-11T23:03:23.219 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.036280599, 2024-09-11T23:03:23.219 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000052504 2024-09-11T23:03:23.219 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.219 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:23.220 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 1123, 2024-09-11T23:03:23.220 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000272523, 2024-09-11T23:03:23.220 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000242 2024-09-11T23:03:23.220 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.220 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 6352, 2024-09-11T23:03:23.220 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 142144 2024-09-11T23:03:23.220 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.220 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-09-11T23:03:23.220 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-09-11T23:03:23.220 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-09-11T23:03:23.220 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.221 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-09-11T23:03:23.221 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-09-11T23:03:23.221 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-09-11T23:03:23.221 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 44, 2024-09-11T23:03:23.221 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.001478808, 2024-09-11T23:03:23.221 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000033609 2024-09-11T23:03:23.221 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:23.221 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.221 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-09-11T23:03:23.221 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-09-11T23:03:23.221 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-09-11T23:03:23.221 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2, 2024-09-11T23:03:23.221 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000017783, 2024-09-11T23:03:23.222 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000008891 2024-09-11T23:03:23.222 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:23.222 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.222 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-09-11T23:03:23.222 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 8, 2024-09-11T23:03:23.222 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 6, 2024-09-11T23:03:23.222 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-09-11T23:03:23.222 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 6, 2024-09-11T23:03:23.222 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.002412770, 2024-09-11T23:03:23.222 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000402128 2024-09-11T23:03:23.222 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.222 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-09-11T23:03:23.222 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 2, 2024-09-11T23:03:23.223 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-09-11T23:03:23.223 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-09-11T23:03:23.223 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 11, 2024-09-11T23:03:23.223 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 0, 2024-09-11T23:03:23.223 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-09-11T23:03:23.223 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 11, 2024-09-11T23:03:23.223 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 0, 2024-09-11T23:03:23.223 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 11, 2024-09-11T23:03:23.223 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 0, 2024-09-11T23:03:23.223 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 10, 2024-09-11T23:03:23.223 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 1, 2024-09-11T23:03:23.223 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 1, 2024-09-11T23:03:23.223 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 1, 2024-09-11T23:03:23.223 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 2, 2024-09-11T23:03:23.224 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 21, 2024-09-11T23:03:23.224 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 16, 2024-09-11T23:03:23.224 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 0, 2024-09-11T23:03:23.224 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 2, 2024-09-11T23:03:23.224 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:03:23.224 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-09-11T23:03:23.224 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-09-11T23:03:23.224 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-09-11T23:03:23.224 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-09-11T23:03:23.224 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 0, 2024-09-11T23:03:23.224 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 0, 2024-09-11T23:03:23.224 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 0, 2024-09-11T23:03:23.224 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 0, 2024-09-11T23:03:23.224 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-09-11T23:03:23.225 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:03:23.225 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-09-11T23:03:23.225 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 4, 2024-09-11T23:03:23.225 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 234, 2024-09-11T23:03:23.225 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 6, 2024-09-11T23:03:23.225 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:03:23.225 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-09-11T23:03:23.225 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:03:23.226 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:03:23.226 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:03:23.226 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:03:23.226 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:03:23.226 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 0, 2024-09-11T23:03:23.226 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:03:23.226 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 3, 2024-09-11T23:03:23.226 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:03:23.226 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:03:23.226 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:03:23.226 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:03:23.227 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 3, 2024-09-11T23:03:23.227 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:03:23.227 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 0, 2024-09-11T23:03:23.227 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 0 2024-09-11T23:03:23.227 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.227 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-09-11T23:03:23.227 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-09-11T23:03:23.227 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-09-11T23:03:23.227 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-09-11T23:03:23.227 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 3, 2024-09-11T23:03:23.227 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 0, 2024-09-11T23:03:23.227 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-09-11T23:03:23.227 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-09-11T23:03:23.228 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:03:23.228 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 0, 2024-09-11T23:03:23.228 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 0, 2024-09-11T23:03:23.228 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-09-11T23:03:23.228 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-09-11T23:03:23.228 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-09-11T23:03:23.228 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:03:23.228 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:03:23.228 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-09-11T23:03:23.229 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-09-11T23:03:23.229 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_quiesce_path": 3, 2024-09-11T23:03:23.229 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_quiesce_inode": 6, 2024-09-11T23:03:23.229 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:03:23.229 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 0, 2024-09-11T23:03:23.229 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-09-11T23:03:23.229 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:03:23.229 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-09-11T23:03:23.229 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-09-11T23:03:23.229 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.229 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-09-11T23:03:23.229 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-09-11T23:03:23.229 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 7, 2024-09-11T23:03:23.230 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-09-11T23:03:23.230 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-09-11T23:03:23.230 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 7, 2024-09-11T23:03:23.230 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-09-11T23:03:23.230 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-09-11T23:03:23.230 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 2, 2024-09-11T23:03:23.230 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-09-11T23:03:23.230 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-09-11T23:03:23.230 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 2, 2024-09-11T23:03:23.230 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 2, 2024-09-11T23:03:23.230 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-09-11T23:03:23.230 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-09-11T23:03:23.230 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 4194304, 2024-09-11T23:03:23.231 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 4196353, 2024-09-11T23:03:23.231 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 0, 2024-09-11T23:03:23.231 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-09-11T23:03:23.231 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 6, 2024-09-11T23:03:23.231 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.010033507, 2024-09-11T23:03:23.231 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.001672251 2024-09-11T23:03:23.231 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.231 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 0 2024-09-11T23:03:23.231 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.231 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-09-11T23:03:23.231 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 14, 2024-09-11T23:03:23.231 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 14, 2024-09-11T23:03:23.231 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 0, 2024-09-11T23:03:23.231 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 13, 2024-09-11T23:03:23.232 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 14, 2024-09-11T23:03:23.232 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 1, 2024-09-11T23:03:23.232 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 11, 2024-09-11T23:03:23.232 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 12, 2024-09-11T23:03:23.232 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 1, 2024-09-11T23:03:23.232 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 1, 2024-09-11T23:03:23.232 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 3, 2024-09-11T23:03:23.232 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 2, 2024-09-11T23:03:23.232 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 50896, 2024-09-11T23:03:23.232 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 223520 2024-09-11T23:03:23.232 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.232 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-09-11T23:03:23.232 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 13, 2024-09-11T23:03:23.233 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 0, 2024-09-11T23:03:23.233 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 8, 2024-09-11T23:03:23.233 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 16, 2024-09-11T23:03:23.233 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 1, 2024-09-11T23:03:23.233 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-09-11T23:03:23.233 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.233 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.233 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.233 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.233 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-09-11T23:03:23.233 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 4, 2024-09-11T23:03:23.233 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.001699960, 2024-09-11T23:03:23.234 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000424990 2024-09-11T23:03:23.234 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.234 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-09-11T23:03:23.234 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.234 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.234 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.234 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.234 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-09-11T23:03:23.234 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.234 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.234 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.234 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.234 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-09-11T23:03:23.235 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2, 2024-09-11T23:03:23.235 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000712810, 2024-09-11T23:03:23.235 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000356405 2024-09-11T23:03:23.235 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.235 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-09-11T23:03:23.235 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.235 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.235 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.235 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.235 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-09-11T23:03:23.235 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.236 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.236 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.236 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.236 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-09-11T23:03:23.236 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.236 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.236 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.236 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.237 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-09-11T23:03:23.237 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.237 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.237 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.237 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.237 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-09-11T23:03:23.237 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.237 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.237 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.237 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.237 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-09-11T23:03:23.238 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.238 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.238 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.238 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.238 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-09-11T23:03:23.238 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.239 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.239 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.239 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.239 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-09-11T23:03:23.239 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.239 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.239 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.239 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.239 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-09-11T23:03:23.240 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.240 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.240 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.240 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.240 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-09-11T23:03:23.240 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.240 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.240 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.240 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.241 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-09-11T23:03:23.241 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.241 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.241 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.241 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.241 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-09-11T23:03:23.241 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.241 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.241 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.241 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.241 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-09-11T23:03:23.242 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.242 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.242 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.242 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.242 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-09-11T23:03:23.242 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.242 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.242 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.242 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.242 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-09-11T23:03:23.242 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.243 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.243 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.243 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.243 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-09-11T23:03:23.243 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.243 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.243 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.243 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.243 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-09-11T23:03:23.243 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.244 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.244 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.244 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.244 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-09-11T23:03:23.244 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.244 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.244 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.244 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.244 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-09-11T23:03:23.244 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.244 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.244 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.245 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.245 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-09-11T23:03:23.245 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.245 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.245 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.245 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.245 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-09-11T23:03:23.245 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.245 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.245 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.245 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.245 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-09-11T23:03:23.246 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.246 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.246 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.246 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.246 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-09-11T23:03:23.246 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.246 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.246 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.246 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.246 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-09-11T23:03:23.247 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.247 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.247 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.247 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.247 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:03:23.247 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:03:23.247 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-09-11T23:03:23.247 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.247 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.247 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.247 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:23.248 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.248 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-09-11T23:03:23.248 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 1, 2024-09-11T23:03:23.248 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 3, 2024-09-11T23:03:23.248 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 2, 2024-09-11T23:03:23.248 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 1, 2024-09-11T23:03:23.248 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-09-11T23:03:23.248 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 4, 2024-09-11T23:03:23.248 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 1, 2024-09-11T23:03:23.248 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 560, 2024-09-11T23:03:23.248 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-09-11T23:03:23.248 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.249 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-09-11T23:03:23.249 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:03:23.249 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-09-11T23:03:23.249 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:03:23.249 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:03:23.249 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:03:23.249 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:03:23.249 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:03:23.249 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:03:23.249 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:03:23.249 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:03:23.249 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:03:23.249 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:03:23.250 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:03:23.250 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:03:23.250 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:03:23.250 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-09-11T23:03:23.250 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:03:23.250 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-09-11T23:03:23.250 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:03:23.250 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:03:23.250 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:03:23.250 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:03:23.251 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:03:23.251 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:03:23.251 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:03:23.251 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-09-11T23:03:23.251 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:03:23.251 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:03:23.251 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:03:23.251 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-09-11T23:03:23.251 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-09-11T23:03:23.252 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-09-11T23:03:23.252 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:03:23.252 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:03:23.252 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:03:23.252 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:03:23.252 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 264391, 2024-09-11T23:03:23.252 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 103, 2024-09-11T23:03:23.252 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 88, 2024-09-11T23:03:23.253 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 1, 2024-09-11T23:03:23.253 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-09-11T23:03:23.253 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-09-11T23:03:23.253 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:03:23.253 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-09-11T23:03:23.253 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:03:23.253 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-09-11T23:03:23.253 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 8040, 2024-09-11T23:03:23.253 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 178, 2024-09-11T23:03:23.254 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:03:23.254 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:03:23.254 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-09-11T23:03:23.254 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-09-11T23:03:23.254 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 56928, 2024-09-11T23:03:23.254 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 228, 2024-09-11T23:03:23.254 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-09-11T23:03:23.255 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-09-11T23:03:23.255 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-09-11T23:03:23.255 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-09-11T23:03:23.255 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.255 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-09-11T23:03:23.255 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-09-11T23:03:23.255 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-09-11T23:03:23.255 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 45, 2024-09-11T23:03:23.256 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 12586, 2024-09-11T23:03:23.256 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 33, 2024-09-11T23:03:23.256 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 45, 2024-09-11T23:03:23.256 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-09-11T23:03:23.256 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 45, 2024-09-11T23:03:23.256 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.186996705, 2024-09-11T23:03:23.256 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.004155482 2024-09-11T23:03:23.256 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.257 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-09-11T23:03:23.257 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-09-11T23:03:23.257 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 45, 2024-09-11T23:03:23.257 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 69 2024-09-11T23:03:23.257 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.257 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 45, 2024-09-11T23:03:23.257 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 0, 2024-09-11T23:03:23.257 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 45, 2024-09-11T23:03:23.257 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-09-11T23:03:23.258 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-09-11T23:03:23.258 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 0, 2024-09-11T23:03:23.258 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 10, 2024-09-11T23:03:23.258 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 0, 2024-09-11T23:03:23.258 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 6, 2024-09-11T23:03:23.258 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 8, 2024-09-11T23:03:23.258 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-09-11T23:03:23.258 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-09-11T23:03:23.259 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-09-11T23:03:23.259 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 0, 2024-09-11T23:03:23.259 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 6, 2024-09-11T23:03:23.259 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-09-11T23:03:23.259 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-09-11T23:03:23.259 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-09-11T23:03:23.259 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 0, 2024-09-11T23:03:23.259 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 20, 2024-09-11T23:03:23.260 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:03:23.260 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-09-11T23:03:23.260 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:03:23.260 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-09-11T23:03:23.260 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-09-11T23:03:23.260 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-09-11T23:03:23.260 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:03:23.260 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-09-11T23:03:23.261 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:03:23.261 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-09-11T23:03:23.261 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-09-11T23:03:23.261 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-09-11T23:03:23.261 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-09-11T23:03:23.261 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-09-11T23:03:23.261 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-09-11T23:03:23.261 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-09-11T23:03:23.261 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-09-11T23:03:23.262 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-09-11T23:03:23.262 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-09-11T23:03:23.262 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-09-11T23:03:23.262 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-09-11T23:03:23.262 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-09-11T23:03:23.262 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-09-11T23:03:23.262 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-09-11T23:03:23.262 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-09-11T23:03:23.263 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-09-11T23:03:23.263 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 92, 2024-09-11T23:03:23.263 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-09-11T23:03:23.263 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 11, 2024-09-11T23:03:23.263 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 8, 2024-09-11T23:03:23.263 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 8, 2024-09-11T23:03:23.263 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 0, 2024-09-11T23:03:23.263 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-09-11T23:03:23.264 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 17, 2024-09-11T23:03:23.264 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 0, 2024-09-11T23:03:23.264 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 2 2024-09-11T23:03:23.264 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.264 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-09-11T23:03:23.264 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 1, 2024-09-11T23:03:23.264 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:03:23.264 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-09-11T23:03:23.265 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-09-11T23:03:23.265 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.265 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-09-11T23:03:23.265 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 6 2024-09-11T23:03:23.265 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.265 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-09-11T23:03:23.265 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-09-11T23:03:23.265 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:03:23.265 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-09-11T23:03:23.266 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 0, 2024-09-11T23:03:23.266 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 0, 2024-09-11T23:03:23.266 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 0, 2024-09-11T23:03:23.266 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-09-11T23:03:23.266 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.266 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:03:23.266 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:23.266 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-09-11T23:03:23.266 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:23.266 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 946, 2024-09-11T23:03:23.266 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 1602190, 2024-09-11T23:03:23.266 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:23.266 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 946, 2024-09-11T23:03:23.266 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-11T23:03:23.267 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-11T23:03:23.267 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 946, 2024-09-11T23:03:23.267 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 1602190, 2024-09-11T23:03:23.267 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:23.267 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.267 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.267 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.267 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:23.267 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.267 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-09-11T23:03:23.267 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:23.267 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-09-11T23:03:23.267 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:23.267 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-11T23:03:23.268 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-11T23:03:23.268 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:23.268 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:23.268 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 45, 2024-09-11T23:03:23.268 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 12586, 2024-09-11T23:03:23.268 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 45, 2024-09-11T23:03:23.268 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 12586, 2024-09-11T23:03:23.268 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:23.268 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.268 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.268 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.268 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:23.268 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.269 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-09-11T23:03:23.269 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:23.269 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-09-11T23:03:23.269 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:23.269 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-11T23:03:23.269 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-11T23:03:23.269 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:23.269 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:23.269 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 45, 2024-09-11T23:03:23.269 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 45, 2024-09-11T23:03:23.269 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 45, 2024-09-11T23:03:23.269 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 45, 2024-09-11T23:03:23.270 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:23.270 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.270 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.270 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.270 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:23.270 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.270 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:03:23.270 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:23.270 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-09-11T23:03:23.271 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:23.271 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-11T23:03:23.271 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-11T23:03:23.271 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:23.271 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:23.271 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-11T23:03:23.271 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-11T23:03:23.271 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-09-11T23:03:23.271 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-09-11T23:03:23.271 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:23.271 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.272 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.272 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.272 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:23.272 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:23.272 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x563d4c114960": { 2024-09-11T23:03:23.272 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:23.272 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-09-11T23:03:23.272 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:23.272 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 7, 2024-09-11T23:03:23.272 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 2049, 2024-09-11T23:03:23.273 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:23.273 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 7, 2024-09-11T23:03:23.273 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-11T23:03:23.273 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-11T23:03:23.273 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 6, 2024-09-11T23:03:23.273 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 2049, 2024-09-11T23:03:23.273 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:23.273 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:23.273 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:23.274 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:23.274 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:23.274 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:23.274 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-11T23:03:23.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:23 smithi152 ceph-mon[31181]: pgmap v325: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-11T23:03:23.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:23 smithi191 ceph-mon[35211]: pgmap v325: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-11T23:03:23.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:23 smithi204 ceph-mon[35454]: pgmap v325: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-11T23:03:24.114 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-11T23:03:24.115 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-11T23:03:24.115 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-11T23:03:24.115 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:03:24.115 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:03:24.115 DEBUG:tasks.check_counter:Getting stats from k 2024-09-11T23:03:24.115 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-11T23:03:24.361 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T23:03:24.874 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-09-11T23:03:24.874 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:03:24.874 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 981, 2024-09-11T23:03:24.874 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 967, 2024-09-11T23:03:24.874 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 1738320, 2024-09-11T23:03:24.874 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 368234, 2024-09-11T23:03:24.874 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 5, 2024-09-11T23:03:24.874 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 1, 2024-09-11T23:03:24.874 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.130044428, 2024-09-11T23:03:24.875 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.051570170, 2024-09-11T23:03:24.875 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.059316062, 2024-09-11T23:03:24.875 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.023067015, 2024-09-11T23:03:24.875 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:24.875 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 966, 2024-09-11T23:03:24.875 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.058294777, 2024-09-11T23:03:24.875 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000060346 2024-09-11T23:03:24.875 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.875 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:24.875 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.875 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.875 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.875 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.876 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 1490016, 2024-09-11T23:03:24.876 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 133232 2024-09-11T23:03:24.876 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.876 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:03:24.876 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 664, 2024-09-11T23:03:24.876 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 825, 2024-09-11T23:03:24.876 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 163824, 2024-09-11T23:03:24.876 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 652520, 2024-09-11T23:03:24.876 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 7, 2024-09-11T23:03:24.876 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 5, 2024-09-11T23:03:24.876 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.069544039, 2024-09-11T23:03:24.876 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.022809131, 2024-09-11T23:03:24.876 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.016142369, 2024-09-11T23:03:24.876 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.023718608, 2024-09-11T23:03:24.877 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:24.877 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 822, 2024-09-11T23:03:24.877 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.036845151, 2024-09-11T23:03:24.877 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000044823 2024-09-11T23:03:24.877 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.877 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:24.877 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.877 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.877 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.877 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.877 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T23:03:24.877 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 441936 2024-09-11T23:03:24.878 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.878 INFO:teuthology.orchestra.run.smithi191.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:03:24.878 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_messages": 751, 2024-09-11T23:03:24.878 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages": 751, 2024-09-11T23:03:24.878 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_bytes": 200684, 2024-09-11T23:03:24.878 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_bytes": 237995, 2024-09-11T23:03:24.878 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_created_connections": 4, 2024-09-11T23:03:24.878 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_active_connections": 3, 2024-09-11T23:03:24.878 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_total_time": 0.060715264, 2024-09-11T23:03:24.878 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_send_time": 0.018944737, 2024-09-11T23:03:24.878 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_recv_time": 0.018501594, 2024-09-11T23:03:24.878 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_running_fast_dispatch_time": 0.017745167, 2024-09-11T23:03:24.878 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:24.879 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 750, 2024-09-11T23:03:24.879 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.026697736, 2024-09-11T23:03:24.879 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000035596 2024-09-11T23:03:24.879 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.879 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:24.879 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.879 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.879 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.879 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.879 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-11T23:03:24.879 INFO:teuthology.orchestra.run.smithi191.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T23:03:24.879 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.880 INFO:teuthology.orchestra.run.smithi191.stdout: "cct": { 2024-09-11T23:03:24.880 INFO:teuthology.orchestra.run.smithi191.stdout: "total_workers": 1, 2024-09-11T23:03:24.880 INFO:teuthology.orchestra.run.smithi191.stdout: "unhealthy_workers": 0 2024-09-11T23:03:24.880 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.880 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-MDSRank": { 2024-09-11T23:03:24.880 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-09-11T23:03:24.880 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-09-11T23:03:24.880 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 3513, 2024-09-11T23:03:24.880 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.096165151, 2024-09-11T23:03:24.880 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000027374 2024-09-11T23:03:24.880 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:24.881 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.881 INFO:teuthology.orchestra.run.smithi191.stdout: "finisher-PurgeQueue": { 2024-09-11T23:03:24.881 INFO:teuthology.orchestra.run.smithi191.stdout: "queue_len": 0, 2024-09-11T23:03:24.881 INFO:teuthology.orchestra.run.smithi191.stdout: "complete_latency": { 2024-09-11T23:03:24.881 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.881 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.881 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.881 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:24.881 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.881 INFO:teuthology.orchestra.run.smithi191.stdout: "mds": { 2024-09-11T23:03:24.881 INFO:teuthology.orchestra.run.smithi191.stdout: "request": 0, 2024-09-11T23:03:24.881 INFO:teuthology.orchestra.run.smithi191.stdout: "reply": 0, 2024-09-11T23:03:24.881 INFO:teuthology.orchestra.run.smithi191.stdout: "reply_latency": { 2024-09-11T23:03:24.882 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.882 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.882 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.882 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.882 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_reply": 0, 2024-09-11T23:03:24.882 INFO:teuthology.orchestra.run.smithi191.stdout: "forward": 0, 2024-09-11T23:03:24.882 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_complete": 0, 2024-09-11T23:03:24.882 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_fetch_keys": 0, 2024-09-11T23:03:24.882 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_commit": 0, 2024-09-11T23:03:24.882 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_split": 0, 2024-09-11T23:03:24.882 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_merge": 0, 2024-09-11T23:03:24.882 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes": 0, 2024-09-11T23:03:24.882 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_top": 0, 2024-09-11T23:03:24.883 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_bottom": 0, 2024-09-11T23:03:24.883 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pin_tail": 0, 2024-09-11T23:03:24.883 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_pinned": 0, 2024-09-11T23:03:24.883 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_expired": 0, 2024-09-11T23:03:24.883 INFO:teuthology.orchestra.run.smithi191.stdout: "inodes_with_caps": 0, 2024-09-11T23:03:24.883 INFO:teuthology.orchestra.run.smithi191.stdout: "caps": 0, 2024-09-11T23:03:24.883 INFO:teuthology.orchestra.run.smithi191.stdout: "subtrees": 1, 2024-09-11T23:03:24.883 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse": 0, 2024-09-11T23:03:24.883 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_hit": 0, 2024-09-11T23:03:24.883 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_forward": 0, 2024-09-11T23:03:24.883 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_discover": 0, 2024-09-11T23:03:24.883 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:03:24.883 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_remote_ino": 0, 2024-09-11T23:03:24.883 INFO:teuthology.orchestra.run.smithi191.stdout: "traverse_lock": 0, 2024-09-11T23:03:24.884 INFO:teuthology.orchestra.run.smithi191.stdout: "load_cent": 0, 2024-09-11T23:03:24.884 INFO:teuthology.orchestra.run.smithi191.stdout: "q": 0, 2024-09-11T23:03:24.884 INFO:teuthology.orchestra.run.smithi191.stdout: "exported": 0, 2024-09-11T23:03:24.884 INFO:teuthology.orchestra.run.smithi191.stdout: "exported_inodes": 0, 2024-09-11T23:03:24.884 INFO:teuthology.orchestra.run.smithi191.stdout: "imported": 0, 2024-09-11T23:03:24.884 INFO:teuthology.orchestra.run.smithi191.stdout: "imported_inodes": 0, 2024-09-11T23:03:24.884 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_dir_fetch": 0, 2024-09-11T23:03:24.884 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:03:24.884 INFO:teuthology.orchestra.run.smithi191.stdout: "openino_peer_discover": 0, 2024-09-11T23:03:24.884 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rfiles": 0, 2024-09-11T23:03:24.885 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rbytes": 0, 2024-09-11T23:03:24.885 INFO:teuthology.orchestra.run.smithi191.stdout: "root_rsnaps": 0, 2024-09-11T23:03:24.885 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:03:24.885 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_set_tag": 0, 2024-09-11T23:03:24.885 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:03:24.885 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:03:24.885 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:03:24.885 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:03:24.885 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:03:24.885 INFO:teuthology.orchestra.run.smithi191.stdout: "scrub_file_inodes": 0, 2024-09-11T23:03:24.885 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:03:24.885 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:03:24.885 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:03:24.886 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:03:24.886 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:03:24.886 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:03:24.886 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps": 0, 2024-09-11T23:03:24.886 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:03:24.886 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_cap_release": 0, 2024-09-11T23:03:24.886 INFO:teuthology.orchestra.run.smithi191.stdout: "process_request_cap_release": 0 2024-09-11T23:03:24.886 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.886 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_cache": { 2024-09-11T23:03:24.886 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update": 0, 2024-09-11T23:03:24.886 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_update_receipt": 0, 2024-09-11T23:03:24.886 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_try_discover": 0, 2024-09-11T23:03:24.886 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_send_discover": 0, 2024-09-11T23:03:24.887 INFO:teuthology.orchestra.run.smithi191.stdout: "dir_handle_discover": 0, 2024-09-11T23:03:24.887 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays": 0, 2024-09-11T23:03:24.887 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_delayed": 0, 2024-09-11T23:03:24.887 INFO:teuthology.orchestra.run.smithi191.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:03:24.887 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_created": 0, 2024-09-11T23:03:24.887 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_enqueued": 0, 2024-09-11T23:03:24.887 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_reintegrated": 0, 2024-09-11T23:03:24.887 INFO:teuthology.orchestra.run.smithi191.stdout: "strays_migrated": 0, 2024-09-11T23:03:24.887 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_processing": 0, 2024-09-11T23:03:24.887 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:03:24.887 INFO:teuthology.orchestra.run.smithi191.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:03:24.887 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_started": 0, 2024-09-11T23:03:24.887 INFO:teuthology.orchestra.run.smithi191.stdout: "recovery_completed": 0, 2024-09-11T23:03:24.888 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:03:24.888 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:03:24.888 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:03:24.888 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_exportdir": 0, 2024-09-11T23:03:24.888 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_flush": 0, 2024-09-11T23:03:24.888 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:03:24.888 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_fragstats": 0, 2024-09-11T23:03:24.888 INFO:teuthology.orchestra.run.smithi191.stdout: "ireq_inodestats": 0 2024-09-11T23:03:24.888 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.888 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_log": { 2024-09-11T23:03:24.888 INFO:teuthology.orchestra.run.smithi191.stdout: "evlrg": 0, 2024-09-11T23:03:24.888 INFO:teuthology.orchestra.run.smithi191.stdout: "evadd": 0, 2024-09-11T23:03:24.888 INFO:teuthology.orchestra.run.smithi191.stdout: "evex": 0, 2024-09-11T23:03:24.889 INFO:teuthology.orchestra.run.smithi191.stdout: "evtrm": 0, 2024-09-11T23:03:24.889 INFO:teuthology.orchestra.run.smithi191.stdout: "ev": 7, 2024-09-11T23:03:24.889 INFO:teuthology.orchestra.run.smithi191.stdout: "evexg": 0, 2024-09-11T23:03:24.889 INFO:teuthology.orchestra.run.smithi191.stdout: "evexd": 0, 2024-09-11T23:03:24.889 INFO:teuthology.orchestra.run.smithi191.stdout: "segadd": 0, 2024-09-11T23:03:24.889 INFO:teuthology.orchestra.run.smithi191.stdout: "segex": 0, 2024-09-11T23:03:24.889 INFO:teuthology.orchestra.run.smithi191.stdout: "segtrm": 0, 2024-09-11T23:03:24.889 INFO:teuthology.orchestra.run.smithi191.stdout: "seg": 2, 2024-09-11T23:03:24.889 INFO:teuthology.orchestra.run.smithi191.stdout: "segmjr": 2, 2024-09-11T23:03:24.889 INFO:teuthology.orchestra.run.smithi191.stdout: "segexg": 0, 2024-09-11T23:03:24.889 INFO:teuthology.orchestra.run.smithi191.stdout: "segexd": 0, 2024-09-11T23:03:24.889 INFO:teuthology.orchestra.run.smithi191.stdout: "expos": 4194304, 2024-09-11T23:03:24.889 INFO:teuthology.orchestra.run.smithi191.stdout: "wrpos": 4196353, 2024-09-11T23:03:24.889 INFO:teuthology.orchestra.run.smithi191.stdout: "rdpos": 4196212, 2024-09-11T23:03:24.890 INFO:teuthology.orchestra.run.smithi191.stdout: "jlat": { 2024-09-11T23:03:24.890 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.890 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.890 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.890 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.890 INFO:teuthology.orchestra.run.smithi191.stdout: "replayed": 7 2024-09-11T23:03:24.890 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.890 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_mem": { 2024-09-11T23:03:24.890 INFO:teuthology.orchestra.run.smithi191.stdout: "ino": 3, 2024-09-11T23:03:24.890 INFO:teuthology.orchestra.run.smithi191.stdout: "ino+": 3, 2024-09-11T23:03:24.891 INFO:teuthology.orchestra.run.smithi191.stdout: "ino-": 0, 2024-09-11T23:03:24.891 INFO:teuthology.orchestra.run.smithi191.stdout: "dir": 1, 2024-09-11T23:03:24.891 INFO:teuthology.orchestra.run.smithi191.stdout: "dir+": 1, 2024-09-11T23:03:24.891 INFO:teuthology.orchestra.run.smithi191.stdout: "dir-": 0, 2024-09-11T23:03:24.891 INFO:teuthology.orchestra.run.smithi191.stdout: "dn": 0, 2024-09-11T23:03:24.891 INFO:teuthology.orchestra.run.smithi191.stdout: "dn+": 0, 2024-09-11T23:03:24.891 INFO:teuthology.orchestra.run.smithi191.stdout: "dn-": 0, 2024-09-11T23:03:24.891 INFO:teuthology.orchestra.run.smithi191.stdout: "cap": 0, 2024-09-11T23:03:24.891 INFO:teuthology.orchestra.run.smithi191.stdout: "cap+": 0, 2024-09-11T23:03:24.891 INFO:teuthology.orchestra.run.smithi191.stdout: "cap-": 0, 2024-09-11T23:03:24.891 INFO:teuthology.orchestra.run.smithi191.stdout: "rss": 48596, 2024-09-11T23:03:24.891 INFO:teuthology.orchestra.run.smithi191.stdout: "heap": 215328 2024-09-11T23:03:24.891 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.892 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_server": { 2024-09-11T23:03:24.892 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_client_request": 0, 2024-09-11T23:03:24.892 INFO:teuthology.orchestra.run.smithi191.stdout: "dispatch_server_request": 0, 2024-09-11T23:03:24.892 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_request": 0, 2024-09-11T23:03:24.892 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_client_session": 0, 2024-09-11T23:03:24.892 INFO:teuthology.orchestra.run.smithi191.stdout: "handle_peer_request": 0, 2024-09-11T23:03:24.892 INFO:teuthology.orchestra.run.smithi191.stdout: "req_create_latency": { 2024-09-11T23:03:24.892 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.892 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.892 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.892 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.892 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getattr_latency": { 2024-09-11T23:03:24.892 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.893 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.893 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.893 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.893 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getfilelock_latency": { 2024-09-11T23:03:24.893 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.893 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.893 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.893 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.893 INFO:teuthology.orchestra.run.smithi191.stdout: "req_link_latency": { 2024-09-11T23:03:24.893 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.893 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.893 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.893 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.894 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookup_latency": { 2024-09-11T23:03:24.894 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.894 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.894 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.894 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.894 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookuphash_latency": { 2024-09-11T23:03:24.894 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.894 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.894 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.894 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.894 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupino_latency": { 2024-09-11T23:03:24.894 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.894 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.895 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.895 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.895 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupname_latency": { 2024-09-11T23:03:24.895 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.895 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.895 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.895 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.895 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupparent_latency": { 2024-09-11T23:03:24.895 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.895 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.895 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.895 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.895 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lookupsnap_latency": { 2024-09-11T23:03:24.896 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.896 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.896 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.896 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.896 INFO:teuthology.orchestra.run.smithi191.stdout: "req_lssnap_latency": { 2024-09-11T23:03:24.896 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.896 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.896 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.896 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.896 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mkdir_latency": { 2024-09-11T23:03:24.896 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.896 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.896 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.897 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.897 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mknod_latency": { 2024-09-11T23:03:24.897 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.897 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.897 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.897 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.897 INFO:teuthology.orchestra.run.smithi191.stdout: "req_mksnap_latency": { 2024-09-11T23:03:24.897 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.897 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.897 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.897 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.897 INFO:teuthology.orchestra.run.smithi191.stdout: "req_open_latency": { 2024-09-11T23:03:24.897 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.898 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.898 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.898 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.898 INFO:teuthology.orchestra.run.smithi191.stdout: "req_readdir_latency": { 2024-09-11T23:03:24.898 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.898 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.898 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.898 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.898 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rename_latency": { 2024-09-11T23:03:24.898 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.898 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.898 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.898 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.899 INFO:teuthology.orchestra.run.smithi191.stdout: "req_renamesnap_latency": { 2024-09-11T23:03:24.899 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.899 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.899 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.899 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.899 INFO:teuthology.orchestra.run.smithi191.stdout: "req_snapdiff_latency": { 2024-09-11T23:03:24.899 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.899 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.899 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.899 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.899 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmdir_latency": { 2024-09-11T23:03:24.899 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.899 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.900 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.900 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.900 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmsnap_latency": { 2024-09-11T23:03:24.900 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.900 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.900 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.900 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.900 INFO:teuthology.orchestra.run.smithi191.stdout: "req_rmxattr_latency": { 2024-09-11T23:03:24.900 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.900 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.900 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.900 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.900 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setattr_latency": { 2024-09-11T23:03:24.901 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.901 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.901 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.901 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.901 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setdirlayout_latency": { 2024-09-11T23:03:24.901 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.901 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.901 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.901 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.901 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setfilelock_latency": { 2024-09-11T23:03:24.901 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.901 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.901 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.902 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.902 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setlayout_latency": { 2024-09-11T23:03:24.902 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.902 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.902 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.902 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.902 INFO:teuthology.orchestra.run.smithi191.stdout: "req_setxattr_latency": { 2024-09-11T23:03:24.902 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.902 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.902 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.902 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.902 INFO:teuthology.orchestra.run.smithi191.stdout: "req_symlink_latency": { 2024-09-11T23:03:24.902 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.903 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.903 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.903 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.903 INFO:teuthology.orchestra.run.smithi191.stdout: "req_unlink_latency": { 2024-09-11T23:03:24.903 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.903 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.903 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.903 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.903 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:03:24.903 INFO:teuthology.orchestra.run.smithi191.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:03:24.903 INFO:teuthology.orchestra.run.smithi191.stdout: "req_getvxattr_latency": { 2024-09-11T23:03:24.903 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.903 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.903 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.904 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:24.904 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.904 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_sessions": { 2024-09-11T23:03:24.904 INFO:teuthology.orchestra.run.smithi191.stdout: "session_count": 1, 2024-09-11T23:03:24.904 INFO:teuthology.orchestra.run.smithi191.stdout: "session_add": 3, 2024-09-11T23:03:24.904 INFO:teuthology.orchestra.run.smithi191.stdout: "session_remove": 2, 2024-09-11T23:03:24.904 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_open": 2, 2024-09-11T23:03:24.904 INFO:teuthology.orchestra.run.smithi191.stdout: "sessions_stale": 0, 2024-09-11T23:03:24.904 INFO:teuthology.orchestra.run.smithi191.stdout: "total_load": 0, 2024-09-11T23:03:24.904 INFO:teuthology.orchestra.run.smithi191.stdout: "average_load": 0, 2024-09-11T23:03:24.904 INFO:teuthology.orchestra.run.smithi191.stdout: "avg_session_uptime": 3284, 2024-09-11T23:03:24.904 INFO:teuthology.orchestra.run.smithi191.stdout: "mdthresh_evicted": 0 2024-09-11T23:03:24.905 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.905 INFO:teuthology.orchestra.run.smithi191.stdout: "mempool": { 2024-09-11T23:03:24.905 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:03:24.905 INFO:teuthology.orchestra.run.smithi191.stdout: "bloom_filter_items": 0, 2024-09-11T23:03:24.905 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:03:24.905 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:03:24.905 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:03:24.905 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:03:24.905 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:03:24.905 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:03:24.905 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:03:24.905 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:03:24.905 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:03:24.905 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:03:24.906 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:03:24.906 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:03:24.906 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:03:24.906 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_extent_items": 0, 2024-09-11T23:03:24.906 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:03:24.906 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_blob_items": 0, 2024-09-11T23:03:24.906 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:03:24.906 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:03:24.906 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:03:24.906 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:03:24.906 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:03:24.906 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:03:24.906 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:03:24.907 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_txc_items": 0, 2024-09-11T23:03:24.907 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:03:24.907 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:03:24.907 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:03:24.907 INFO:teuthology.orchestra.run.smithi191.stdout: "bluestore_writing_items": 0, 2024-09-11T23:03:24.907 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_bytes": 0, 2024-09-11T23:03:24.907 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_items": 0, 2024-09-11T23:03:24.907 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:03:24.907 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:03:24.907 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:03:24.907 INFO:teuthology.orchestra.run.smithi191.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:03:24.907 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_bytes": 189383, 2024-09-11T23:03:24.907 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_anon_items": 50, 2024-09-11T23:03:24.907 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:03:24.908 INFO:teuthology.orchestra.run.smithi191.stdout: "buffer_meta_items": 0, 2024-09-11T23:03:24.908 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_bytes": 0, 2024-09-11T23:03:24.908 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_items": 0, 2024-09-11T23:03:24.908 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:03:24.908 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_mapbl_items": 0, 2024-09-11T23:03:24.908 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:03:24.908 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_pglog_items": 0, 2024-09-11T23:03:24.908 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_bytes": 8040, 2024-09-11T23:03:24.908 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_items": 178, 2024-09-11T23:03:24.908 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:03:24.908 INFO:teuthology.orchestra.run.smithi191.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:03:24.908 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_bytes": 0, 2024-09-11T23:03:24.908 INFO:teuthology.orchestra.run.smithi191.stdout: "pgmap_items": 0, 2024-09-11T23:03:24.909 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_bytes": 8512, 2024-09-11T23:03:24.909 INFO:teuthology.orchestra.run.smithi191.stdout: "mds_co_items": 13, 2024-09-11T23:03:24.909 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_bytes": 0, 2024-09-11T23:03:24.909 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_1_items": 0, 2024-09-11T23:03:24.909 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_bytes": 0, 2024-09-11T23:03:24.909 INFO:teuthology.orchestra.run.smithi191.stdout: "unittest_2_items": 0 2024-09-11T23:03:24.909 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.909 INFO:teuthology.orchestra.run.smithi191.stdout: "objecter": { 2024-09-11T23:03:24.909 INFO:teuthology.orchestra.run.smithi191.stdout: "op_active": 0, 2024-09-11T23:03:24.909 INFO:teuthology.orchestra.run.smithi191.stdout: "op_laggy": 0, 2024-09-11T23:03:24.909 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send": 2148, 2024-09-11T23:03:24.909 INFO:teuthology.orchestra.run.smithi191.stdout: "op_send_bytes": 21, 2024-09-11T23:03:24.909 INFO:teuthology.orchestra.run.smithi191.stdout: "op_resend": 3, 2024-09-11T23:03:24.910 INFO:teuthology.orchestra.run.smithi191.stdout: "op_reply": 2148, 2024-09-11T23:03:24.910 INFO:teuthology.orchestra.run.smithi191.stdout: "op_latency": { 2024-09-11T23:03:24.910 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2148, 2024-09-11T23:03:24.910 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 1.198979154, 2024-09-11T23:03:24.910 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000558183 2024-09-11T23:03:24.910 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.910 INFO:teuthology.orchestra.run.smithi191.stdout: "op_inflight": 0, 2024-09-11T23:03:24.910 INFO:teuthology.orchestra.run.smithi191.stdout: "oplen_avg": { 2024-09-11T23:03:24.910 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 2148, 2024-09-11T23:03:24.910 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 2149 2024-09-11T23:03:24.910 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.910 INFO:teuthology.orchestra.run.smithi191.stdout: "op": 2148, 2024-09-11T23:03:24.910 INFO:teuthology.orchestra.run.smithi191.stdout: "op_r": 2148, 2024-09-11T23:03:24.910 INFO:teuthology.orchestra.run.smithi191.stdout: "op_w": 0, 2024-09-11T23:03:24.911 INFO:teuthology.orchestra.run.smithi191.stdout: "op_rmw": 0, 2024-09-11T23:03:24.911 INFO:teuthology.orchestra.run.smithi191.stdout: "op_pg": 0, 2024-09-11T23:03:24.911 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_stat": 1398, 2024-09-11T23:03:24.911 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_create": 0, 2024-09-11T23:03:24.911 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_read": 748, 2024-09-11T23:03:24.911 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_write": 0, 2024-09-11T23:03:24.911 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writefull": 0, 2024-09-11T23:03:24.911 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_writesame": 0, 2024-09-11T23:03:24.911 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_append": 0, 2024-09-11T23:03:24.911 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_zero": 0, 2024-09-11T23:03:24.911 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_truncate": 0, 2024-09-11T23:03:24.911 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_delete": 0, 2024-09-11T23:03:24.911 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_mapext": 0, 2024-09-11T23:03:24.912 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_sparse_read": 0, 2024-09-11T23:03:24.912 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_clonerange": 0, 2024-09-11T23:03:24.912 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_getxattr": 1, 2024-09-11T23:03:24.912 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_setxattr": 0, 2024-09-11T23:03:24.912 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:03:24.913 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_rmxattr": 0, 2024-09-11T23:03:24.913 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:03:24.913 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_call": 0, 2024-09-11T23:03:24.913 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_watch": 0, 2024-09-11T23:03:24.913 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_notify": 0, 2024-09-11T23:03:24.913 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:03:24.913 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls": 0, 2024-09-11T23:03:24.914 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:03:24.914 INFO:teuthology.orchestra.run.smithi191.stdout: "osdop_other": 0, 2024-09-11T23:03:24.914 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_active": 0, 2024-09-11T23:03:24.914 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_send": 0, 2024-09-11T23:03:24.914 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_resend": 0, 2024-09-11T23:03:24.914 INFO:teuthology.orchestra.run.smithi191.stdout: "linger_ping": 0, 2024-09-11T23:03:24.914 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_active": 0, 2024-09-11T23:03:24.914 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_send": 0, 2024-09-11T23:03:24.914 INFO:teuthology.orchestra.run.smithi191.stdout: "poolop_resend": 0, 2024-09-11T23:03:24.914 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_active": 0, 2024-09-11T23:03:24.914 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_send": 0, 2024-09-11T23:03:24.914 INFO:teuthology.orchestra.run.smithi191.stdout: "poolstat_resend": 0, 2024-09-11T23:03:24.915 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_active": 0, 2024-09-11T23:03:24.915 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_send": 0, 2024-09-11T23:03:24.915 INFO:teuthology.orchestra.run.smithi191.stdout: "statfs_resend": 0, 2024-09-11T23:03:24.915 INFO:teuthology.orchestra.run.smithi191.stdout: "command_active": 0, 2024-09-11T23:03:24.915 INFO:teuthology.orchestra.run.smithi191.stdout: "command_send": 0, 2024-09-11T23:03:24.915 INFO:teuthology.orchestra.run.smithi191.stdout: "command_resend": 0, 2024-09-11T23:03:24.915 INFO:teuthology.orchestra.run.smithi191.stdout: "map_epoch": 92, 2024-09-11T23:03:24.915 INFO:teuthology.orchestra.run.smithi191.stdout: "map_full": 0, 2024-09-11T23:03:24.915 INFO:teuthology.orchestra.run.smithi191.stdout: "map_inc": 11, 2024-09-11T23:03:24.915 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_sessions": 5, 2024-09-11T23:03:24.915 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_open": 5, 2024-09-11T23:03:24.915 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_session_close": 0, 2024-09-11T23:03:24.916 INFO:teuthology.orchestra.run.smithi191.stdout: "osd_laggy": 0, 2024-09-11T23:03:24.916 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_wr": 0, 2024-09-11T23:03:24.916 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_rd": 2, 2024-09-11T23:03:24.916 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_del": 0 2024-09-11T23:03:24.916 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.916 INFO:teuthology.orchestra.run.smithi191.stdout: "oft": { 2024-09-11T23:03:24.916 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_objs": 0, 2024-09-11T23:03:24.916 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:03:24.916 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_updates": 0, 2024-09-11T23:03:24.916 INFO:teuthology.orchestra.run.smithi191.stdout: "omap_total_removes": 0 2024-09-11T23:03:24.917 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.917 INFO:teuthology.orchestra.run.smithi191.stdout: "osd-slow-ops": { 2024-09-11T23:03:24.917 INFO:teuthology.orchestra.run.smithi191.stdout: "slow_ops_count": 0 2024-09-11T23:03:24.917 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.917 INFO:teuthology.orchestra.run.smithi191.stdout: "purge_queue": { 2024-09-11T23:03:24.917 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops": 0, 2024-09-11T23:03:24.917 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:03:24.917 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing": 0, 2024-09-11T23:03:24.917 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executing_high_water": 0, 2024-09-11T23:03:24.917 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed_ops": 0, 2024-09-11T23:03:24.917 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_executed": 0, 2024-09-11T23:03:24.917 INFO:teuthology.orchestra.run.smithi191.stdout: "pq_item_in_journal": 0 2024-09-11T23:03:24.917 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.917 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:03:24.918 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:24.918 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-09-11T23:03:24.918 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:24.918 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 2392, 2024-09-11T23:03:24.918 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 1872181, 2024-09-11T23:03:24.918 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:24.918 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 2392, 2024-09-11T23:03:24.918 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-11T23:03:24.918 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-11T23:03:24.918 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2392, 2024-09-11T23:03:24.918 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 1872181, 2024-09-11T23:03:24.918 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:24.918 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.918 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.919 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.919 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:24.919 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.919 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_bytes": { 2024-09-11T23:03:24.919 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:24.919 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 104857600, 2024-09-11T23:03:24.919 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:24.919 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-11T23:03:24.919 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-11T23:03:24.919 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:24.919 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:24.919 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 2148, 2024-09-11T23:03:24.919 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 2054, 2024-09-11T23:03:24.919 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2148, 2024-09-11T23:03:24.920 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 2054, 2024-09-11T23:03:24.920 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:24.920 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.920 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.920 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.920 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:24.920 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.920 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-objecter_ops": { 2024-09-11T23:03:24.920 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:24.920 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 1024, 2024-09-11T23:03:24.920 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:24.920 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-11T23:03:24.920 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-11T23:03:24.921 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:24.921 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:24.921 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 2148, 2024-09-11T23:03:24.921 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 2148, 2024-09-11T23:03:24.921 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 2148, 2024-09-11T23:03:24.921 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 2148, 2024-09-11T23:03:24.921 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:24.921 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.921 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.921 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.921 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:24.921 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.921 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:03:24.921 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:24.922 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-09-11T23:03:24.922 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:24.922 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-11T23:03:24.922 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-11T23:03:24.922 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:24.922 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:24.922 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-11T23:03:24.922 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-11T23:03:24.922 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-09-11T23:03:24.922 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-09-11T23:03:24.922 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:24.922 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.922 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.922 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.923 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:24.923 INFO:teuthology.orchestra.run.smithi191.stdout: }, 2024-09-11T23:03:24.923 INFO:teuthology.orchestra.run.smithi191.stdout: "throttle-write_buf_throttle-0x562c7b61ab40": { 2024-09-11T23:03:24.923 INFO:teuthology.orchestra.run.smithi191.stdout: "val": 0, 2024-09-11T23:03:24.923 INFO:teuthology.orchestra.run.smithi191.stdout: "max": 3758096384, 2024-09-11T23:03:24.923 INFO:teuthology.orchestra.run.smithi191.stdout: "get_started": 0, 2024-09-11T23:03:24.923 INFO:teuthology.orchestra.run.smithi191.stdout: "get": 0, 2024-09-11T23:03:24.923 INFO:teuthology.orchestra.run.smithi191.stdout: "get_sum": 0, 2024-09-11T23:03:24.923 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:24.923 INFO:teuthology.orchestra.run.smithi191.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:24.923 INFO:teuthology.orchestra.run.smithi191.stdout: "take": 0, 2024-09-11T23:03:24.923 INFO:teuthology.orchestra.run.smithi191.stdout: "take_sum": 0, 2024-09-11T23:03:24.923 INFO:teuthology.orchestra.run.smithi191.stdout: "put": 0, 2024-09-11T23:03:24.923 INFO:teuthology.orchestra.run.smithi191.stdout: "put_sum": 0, 2024-09-11T23:03:24.924 INFO:teuthology.orchestra.run.smithi191.stdout: "wait": { 2024-09-11T23:03:24.924 INFO:teuthology.orchestra.run.smithi191.stdout: "avgcount": 0, 2024-09-11T23:03:24.924 INFO:teuthology.orchestra.run.smithi191.stdout: "sum": 0.000000000, 2024-09-11T23:03:24.924 INFO:teuthology.orchestra.run.smithi191.stdout: "avgtime": 0.000000000 2024-09-11T23:03:24.924 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:24.924 INFO:teuthology.orchestra.run.smithi191.stdout: } 2024-09-11T23:03:24.924 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-09-11T23:03:25.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:25 smithi152 ceph-mon[31181]: pgmap v326: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-11T23:03:25.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:25 smithi191 ceph-mon[35211]: pgmap v326: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-11T23:03:25.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:25 smithi204 ceph-mon[35454]: pgmap v326: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-11T23:03:25.565 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-11T23:03:25.565 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-11T23:03:25.566 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-11T23:03:25.566 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:03:25.566 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:03:25.566 DEBUG:tasks.check_counter:Getting stats from c 2024-09-11T23:03:25.566 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-11T23:03:25.823 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 86, 2024-09-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 217, 2024-09-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 29988, 2024-09-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 376538, 2024-09-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 14, 2024-09-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-09-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.020956157, 2024-09-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.008601644, 2024-09-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.006841542, 2024-09-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000670951, 2024-09-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 213, 2024-09-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.055939831, 2024-09-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000262628 2024-09-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 98, 2024-09-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000019374, 2024-09-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000197 2024-09-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1744, 2024-09-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 317312 2024-09-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 345, 2024-09-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 344, 2024-09-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1523431, 2024-09-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 241204, 2024-09-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 14, 2024-09-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 6, 2024-09-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.053224675, 2024-09-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.018548922, 2024-09-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.049627701, 2024-09-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000272366, 2024-09-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 343, 2024-09-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.033977619, 2024-09-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000099060 2024-09-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 133, 2024-09-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000031013, 2024-09-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000233 2024-09-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1498768, 2024-09-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 141264 2024-09-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 528, 2024-09-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 682, 2024-09-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 151673, 2024-09-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 771608, 2024-09-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 42, 2024-09-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 6, 2024-09-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.076855215, 2024-09-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.026530248, 2024-09-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.044292381, 2024-09-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.004669212, 2024-09-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 682, 2024-09-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.041851200, 2024-09-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000061365 2024-09-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1112, 2024-09-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000311857, 2024-09-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000280 2024-09-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 5792, 2024-09-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 129664 2024-09-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-09-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-09-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-09-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-09-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 55, 2024-09-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.001821628, 2024-09-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000033120 2024-09-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-09-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-09-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000013125, 2024-09-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000006562 2024-09-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-09-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 5, 2024-09-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 4, 2024-09-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-09-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 4, 2024-09-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000858163, 2024-09-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000214540 2024-09-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-09-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 1, 2024-09-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-09-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-09-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 11, 2024-09-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-09-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-09-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 11, 2024-09-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-09-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 11, 2024-09-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-09-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 10, 2024-09-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 1, 2024-09-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-09-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-09-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 2, 2024-09-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 17, 2024-09-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 12, 2024-09-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-09-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 2, 2024-09-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-09-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-09-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-09-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-09-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-09-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-09-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-09-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-09-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-09-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-09-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 4, 2024-09-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 234, 2024-09-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 6, 2024-09-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-09-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-09-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 2, 2024-09-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 2, 2024-09-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-09-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 1 2024-09-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-09-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-09-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-09-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-09-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 3, 2024-09-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-09-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-09-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-09-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-09-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-09-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-09-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-09-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-09-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-09-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-09-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 3, 2024-09-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 6, 2024-09-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-09-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-09-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-09-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-09-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-09-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-09-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 7, 2024-09-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-09-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-09-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 7, 2024-09-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-09-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-09-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 2, 2024-09-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-09-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-09-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-09-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-09-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-09-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-09-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-09-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 4196353, 2024-09-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 0, 2024-09-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-09-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 6, 2024-09-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.009864436, 2024-09-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001644072 2024-09-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 0 2024-09-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-09-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 14, 2024-09-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 14, 2024-09-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 0, 2024-09-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 13, 2024-09-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 14, 2024-09-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 1, 2024-09-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 11, 2024-09-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 12, 2024-09-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 1, 2024-09-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-09-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 3, 2024-09-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 3, 2024-09-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 50060, 2024-09-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 223520 2024-09-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-09-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 7, 2024-09-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-09-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 5, 2024-09-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 16, 2024-09-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-09-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-09-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-09-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-09-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000308086, 2024-09-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000154043 2024-09-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-09-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-09-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-09-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-09-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000550077, 2024-09-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000275038 2024-09-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-09-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-09-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-09-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-09-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-09-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-09-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-09-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-09-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-09-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-09-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-09-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-09-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-09-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-09-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-09-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-09-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-09-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-09-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-09-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-09-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-09-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-09-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-09-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-09-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-09-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-09-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-09-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 3, 2024-09-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 2, 2024-09-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 1, 2024-09-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-09-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 3, 2024-09-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 3, 2024-09-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 562, 2024-09-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-09-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-09-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-09-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-09-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-09-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-09-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-09-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-09-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-09-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 256007, 2024-09-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 98, 2024-09-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-09-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-09-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-09-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-09-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-09-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8040, 2024-09-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 178, 2024-09-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-09-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-09-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 56544, 2024-09-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 224, 2024-09-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-09-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-09-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-09-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-09-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-09-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-09-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-09-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 46, 2024-09-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 12676, 2024-09-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 33, 2024-09-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 46, 2024-09-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-09-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 46, 2024-09-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.216997241, 2024-09-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.004717331 2024-09-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-09-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-09-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 46, 2024-09-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 70 2024-09-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 46, 2024-09-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 0, 2024-09-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 46, 2024-09-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-09-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-09-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 0, 2024-09-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 10, 2024-09-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 0, 2024-09-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 6, 2024-09-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 9, 2024-09-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-09-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-09-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-09-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-09-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 6, 2024-09-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-09-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-09-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-09-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 0, 2024-09-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 20, 2024-09-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-09-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-09-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-09-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-09-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-09-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-09-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-09-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-09-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-09-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-09-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-09-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-09-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-09-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-09-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-09-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-09-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-09-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-09-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-09-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-09-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-09-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-09-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 92, 2024-09-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-09-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 11, 2024-09-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 8, 2024-09-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 8, 2024-09-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-09-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-09-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 17, 2024-09-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 0, 2024-09-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 2 2024-09-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-09-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 1, 2024-09-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-09-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-09-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-09-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 6 2024-09-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-09-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-09-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-09-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-09-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-09-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-09-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-09-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 955, 2024-09-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1601520, 2024-09-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 955, 2024-09-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 955, 2024-09-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1601520, 2024-09-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-09-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 46, 2024-09-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 12676, 2024-09-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 46, 2024-09-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 12676, 2024-09-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-09-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-09-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 46, 2024-09-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 46, 2024-09-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 46, 2024-09-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 46, 2024-09-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55c27b4b0960": { 2024-09-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 7, 2024-09-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 2049, 2024-09-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 7, 2024-09-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 6, 2024-09-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2049, 2024-09-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:26.411 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-11T23:03:27.179 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-11T23:03:27.179 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-11T23:03:27.179 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-11T23:03:27.179 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:03:27.179 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:03:27.179 DEBUG:tasks.check_counter:Getting stats from f 2024-09-11T23:03:27.180 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-11T23:03:27.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:27 smithi204 ceph-mon[35454]: pgmap v327: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:03:27.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:27 smithi152 ceph-mon[31181]: pgmap v327: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:03:27.433 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:03:27.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:27 smithi191 ceph-mon[35211]: pgmap v327: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:03:27.981 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-11T23:03:27.981 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:03:27.981 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1263, 2024-09-11T23:03:27.981 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1249, 2024-09-11T23:03:27.981 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 6006416, 2024-09-11T23:03:27.981 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 457112, 2024-09-11T23:03:27.982 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 5, 2024-09-11T23:03:27.982 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 1, 2024-09-11T23:03:27.982 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.162404726, 2024-09-11T23:03:27.982 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.049456375, 2024-09-11T23:03:27.982 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.078507309, 2024-09-11T23:03:27.982 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.037590856, 2024-09-11T23:03:27.982 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:27.983 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1248, 2024-09-11T23:03:27.983 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.067780591, 2024-09-11T23:03:27.983 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000054311 2024-09-11T23:03:27.983 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:27.983 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:27.983 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:27.983 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:27.983 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:27.984 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:27.984 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1489632, 2024-09-11T23:03:27.984 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 132112 2024-09-11T23:03:27.984 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:27.984 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:03:27.984 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1045, 2024-09-11T23:03:27.984 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1202, 2024-09-11T23:03:27.985 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 3146803, 2024-09-11T23:03:27.985 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 774116, 2024-09-11T23:03:27.985 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 7, 2024-09-11T23:03:27.985 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-09-11T23:03:27.985 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.089735333, 2024-09-11T23:03:27.985 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.034176770, 2024-09-11T23:03:27.985 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.022677614, 2024-09-11T23:03:27.985 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.025057929, 2024-09-11T23:03:27.986 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:27.986 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1200, 2024-09-11T23:03:27.986 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.068621511, 2024-09-11T23:03:27.986 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000057184 2024-09-11T23:03:27.986 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:27.986 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:27.986 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:27.987 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:27.987 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:27.987 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:27.987 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-11T23:03:27.987 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 443648 2024-09-11T23:03:27.987 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:27.987 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:03:27.987 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 256, 2024-09-11T23:03:27.987 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 258, 2024-09-11T23:03:27.987 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 4257674, 2024-09-11T23:03:27.987 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 81415, 2024-09-11T23:03:27.988 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-09-11T23:03:27.988 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-09-11T23:03:27.988 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.027876481, 2024-09-11T23:03:27.988 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.008874487, 2024-09-11T23:03:27.988 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.008010918, 2024-09-11T23:03:27.988 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.008293806, 2024-09-11T23:03:27.988 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:27.988 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 256, 2024-09-11T23:03:27.988 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.012138436, 2024-09-11T23:03:27.988 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000047415 2024-09-11T23:03:27.988 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:27.988 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:27.989 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:27.989 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:27.989 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:27.989 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:27.989 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:03:27.989 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:03:27.989 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:27.989 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-09-11T23:03:27.989 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-09-11T23:03:27.990 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-09-11T23:03:27.990 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:27.990 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-09-11T23:03:27.990 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-11T23:03:27.990 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-11T23:03:27.991 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3863, 2024-09-11T23:03:27.991 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.113542663, 2024-09-11T23:03:27.991 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000029392 2024-09-11T23:03:27.991 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:27.991 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:27.992 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-09-11T23:03:27.992 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-11T23:03:27.992 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-11T23:03:27.992 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:27.992 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:27.993 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:27.993 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:27.993 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:27.993 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-09-11T23:03:27.993 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-09-11T23:03:27.993 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-09-11T23:03:27.993 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-09-11T23:03:27.994 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:27.994 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:27.994 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:27.994 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:27.994 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-09-11T23:03:27.994 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-09-11T23:03:27.994 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-09-11T23:03:27.995 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-09-11T23:03:27.995 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-09-11T23:03:27.995 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-09-11T23:03:27.995 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-09-11T23:03:27.995 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 29, 2024-09-11T23:03:27.995 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 10, 2024-09-11T23:03:27.996 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 19, 2024-09-11T23:03:27.996 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-09-11T23:03:27.996 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 15, 2024-09-11T23:03:27.996 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-09-11T23:03:27.996 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-09-11T23:03:27.996 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-09-11T23:03:27.997 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 2, 2024-09-11T23:03:27.997 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-09-11T23:03:27.997 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-09-11T23:03:27.997 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-09-11T23:03:27.997 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-09-11T23:03:27.997 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:03:27.997 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-09-11T23:03:27.998 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-09-11T23:03:27.998 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-09-11T23:03:27.998 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-09-11T23:03:27.998 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-09-11T23:03:27.998 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-09-11T23:03:27.998 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-09-11T23:03:27.998 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-09-11T23:03:27.998 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-09-11T23:03:27.999 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:03:27.999 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-09-11T23:03:27.999 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 4, 2024-09-11T23:03:27.999 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 234, 2024-09-11T23:03:27.999 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 6, 2024-09-11T23:03:27.999 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:03:28.000 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-09-11T23:03:28.000 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:03:28.000 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:03:28.000 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:03:28.000 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:03:28.001 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:03:28.001 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-09-11T23:03:28.001 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:03:28.001 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:03:28.001 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:03:28.001 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:03:28.002 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:03:28.002 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:03:28.002 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-09-11T23:03:28.002 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:03:28.002 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-09-11T23:03:28.002 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-09-11T23:03:28.003 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.003 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-09-11T23:03:28.003 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-09-11T23:03:28.003 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-09-11T23:03:28.003 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-09-11T23:03:28.004 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-09-11T23:03:28.004 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-09-11T23:03:28.004 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-09-11T23:03:28.004 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-09-11T23:03:28.004 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:03:28.004 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-09-11T23:03:28.005 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-09-11T23:03:28.005 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-09-11T23:03:28.005 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-09-11T23:03:28.005 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-09-11T23:03:28.005 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:03:28.005 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:03:28.006 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-09-11T23:03:28.006 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-09-11T23:03:28.006 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:03:28.006 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:03:28.006 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:03:28.007 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-09-11T23:03:28.007 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-09-11T23:03:28.007 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:03:28.007 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-09-11T23:03:28.007 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-09-11T23:03:28.007 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.008 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-09-11T23:03:28.008 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-09-11T23:03:28.008 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-09-11T23:03:28.008 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-09-11T23:03:28.008 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-09-11T23:03:28.009 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 4709, 2024-09-11T23:03:28.009 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-09-11T23:03:28.009 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-09-11T23:03:28.009 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-09-11T23:03:28.009 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-09-11T23:03:28.009 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-09-11T23:03:28.010 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-09-11T23:03:28.010 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 15, 2024-09-11T23:03:28.010 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-09-11T23:03:28.010 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-09-11T23:03:28.010 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 15394124, 2024-09-11T23:03:28.011 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 15462265, 2024-09-11T23:03:28.011 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 15462153, 2024-09-11T23:03:28.011 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-09-11T23:03:28.011 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.011 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.011 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.012 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.012 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 4709 2024-09-11T23:03:28.012 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.012 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-09-11T23:03:28.013 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 20, 2024-09-11T23:03:28.013 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 23, 2024-09-11T23:03:28.013 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 3, 2024-09-11T23:03:28.013 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 14, 2024-09-11T23:03:28.013 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 14, 2024-09-11T23:03:28.014 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 0, 2024-09-11T23:03:28.014 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 29, 2024-09-11T23:03:28.014 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 29, 2024-09-11T23:03:28.014 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 0, 2024-09-11T23:03:28.014 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-09-11T23:03:28.014 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-09-11T23:03:28.015 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-09-11T23:03:28.015 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 53892, 2024-09-11T23:03:28.015 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 215328 2024-09-11T23:03:28.015 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.015 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-09-11T23:03:28.015 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-09-11T23:03:28.016 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-09-11T23:03:28.016 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-09-11T23:03:28.016 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-09-11T23:03:28.016 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-09-11T23:03:28.016 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-09-11T23:03:28.017 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.017 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.017 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.017 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.017 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-09-11T23:03:28.017 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.018 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.018 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.018 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.018 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-09-11T23:03:28.018 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.018 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.019 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.019 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.019 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-09-11T23:03:28.019 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.019 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.019 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.020 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.020 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-09-11T23:03:28.020 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.020 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.020 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.020 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.021 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-09-11T23:03:28.021 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.021 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.021 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.021 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.021 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-09-11T23:03:28.022 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.022 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.022 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.022 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.023 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-09-11T23:03:28.023 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.023 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.023 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.023 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.024 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-09-11T23:03:28.024 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.024 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.024 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.024 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.025 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-09-11T23:03:28.025 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.025 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.025 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.025 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.025 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-09-11T23:03:28.025 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.026 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.026 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.026 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.026 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-09-11T23:03:28.026 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.027 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.027 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.027 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.027 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-09-11T23:03:28.027 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.027 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.028 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.028 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.028 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-09-11T23:03:28.028 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.028 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.028 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.029 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.029 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-09-11T23:03:28.029 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.029 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.029 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.030 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.030 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-09-11T23:03:28.030 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.030 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.030 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.030 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.031 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-09-11T23:03:28.031 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.031 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.031 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.031 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.032 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-09-11T23:03:28.032 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.032 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.032 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.032 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.032 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-09-11T23:03:28.033 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.033 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.033 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.033 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.033 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-09-11T23:03:28.033 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.034 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.034 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.034 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.034 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-09-11T23:03:28.034 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.035 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.035 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.035 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.035 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-09-11T23:03:28.035 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.035 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.036 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.036 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.036 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-09-11T23:03:28.036 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.036 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.037 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.037 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.037 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-09-11T23:03:28.037 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.037 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.037 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.037 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.037 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-09-11T23:03:28.037 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.038 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.038 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.038 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.038 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-09-11T23:03:28.038 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.038 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.038 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.038 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.039 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-09-11T23:03:28.039 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.039 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.039 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.039 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.039 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-09-11T23:03:28.039 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.039 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.040 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.040 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.040 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-09-11T23:03:28.040 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.040 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.040 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.041 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.041 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:03:28.041 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:03:28.041 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-09-11T23:03:28.041 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.041 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.041 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.042 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:28.042 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.042 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-09-11T23:03:28.042 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 2, 2024-09-11T23:03:28.042 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 5, 2024-09-11T23:03:28.042 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 3, 2024-09-11T23:03:28.042 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 4, 2024-09-11T23:03:28.042 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-09-11T23:03:28.043 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-09-11T23:03:28.043 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-09-11T23:03:28.043 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 3266, 2024-09-11T23:03:28.043 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-09-11T23:03:28.043 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.043 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-09-11T23:03:28.043 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:03:28.043 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-09-11T23:03:28.043 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:03:28.043 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:03:28.043 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:03:28.043 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:03:28.044 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:03:28.044 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:03:28.044 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:03:28.044 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:03:28.044 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:03:28.044 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:03:28.044 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:03:28.044 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:03:28.044 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:03:28.044 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-09-11T23:03:28.045 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:03:28.045 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-09-11T23:03:28.045 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:03:28.045 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:03:28.045 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:03:28.045 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:03:28.045 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:03:28.046 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:03:28.046 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:03:28.046 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-09-11T23:03:28.046 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:03:28.046 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:03:28.046 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:03:28.047 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-09-11T23:03:28.047 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-09-11T23:03:28.047 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-09-11T23:03:28.047 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:03:28.047 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:03:28.047 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:03:28.047 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:03:28.047 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 189652, 2024-09-11T23:03:28.048 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 58, 2024-09-11T23:03:28.048 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:03:28.048 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-09-11T23:03:28.048 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-09-11T23:03:28.048 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-09-11T23:03:28.048 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:03:28.048 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-09-11T23:03:28.048 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:03:28.048 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-09-11T23:03:28.048 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8040, 2024-09-11T23:03:28.048 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 178, 2024-09-11T23:03:28.048 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:03:28.048 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:03:28.049 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-09-11T23:03:28.049 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-09-11T23:03:28.049 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 99867, 2024-09-11T23:03:28.049 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 648, 2024-09-11T23:03:28.049 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-09-11T23:03:28.049 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-09-11T23:03:28.049 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-09-11T23:03:28.049 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-09-11T23:03:28.049 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.049 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-09-11T23:03:28.050 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-09-11T23:03:28.050 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-09-11T23:03:28.050 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2317, 2024-09-11T23:03:28.050 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 26, 2024-09-11T23:03:28.050 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 4, 2024-09-11T23:03:28.050 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2317, 2024-09-11T23:03:28.050 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-09-11T23:03:28.050 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2317, 2024-09-11T23:03:28.050 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 1.613979404, 2024-09-11T23:03:28.050 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000696581 2024-09-11T23:03:28.050 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.051 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-09-11T23:03:28.051 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-09-11T23:03:28.051 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2317, 2024-09-11T23:03:28.051 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2318 2024-09-11T23:03:28.051 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.052 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2317, 2024-09-11T23:03:28.052 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2317, 2024-09-11T23:03:28.052 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-09-11T23:03:28.052 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-09-11T23:03:28.052 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-09-11T23:03:28.052 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1474, 2024-09-11T23:03:28.053 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-09-11T23:03:28.053 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 840, 2024-09-11T23:03:28.053 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-09-11T23:03:28.053 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-09-11T23:03:28.053 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-09-11T23:03:28.053 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-09-11T23:03:28.053 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-09-11T23:03:28.053 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-09-11T23:03:28.053 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-09-11T23:03:28.054 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-09-11T23:03:28.054 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-09-11T23:03:28.054 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-09-11T23:03:28.054 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 2, 2024-09-11T23:03:28.054 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-09-11T23:03:28.054 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:03:28.054 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-09-11T23:03:28.054 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:03:28.054 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-09-11T23:03:28.055 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-09-11T23:03:28.055 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-09-11T23:03:28.055 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:03:28.055 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-09-11T23:03:28.055 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:03:28.055 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-09-11T23:03:28.055 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-09-11T23:03:28.055 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-09-11T23:03:28.055 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-09-11T23:03:28.055 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-09-11T23:03:28.055 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-09-11T23:03:28.055 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-09-11T23:03:28.056 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-09-11T23:03:28.056 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-09-11T23:03:28.056 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-09-11T23:03:28.056 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-09-11T23:03:28.056 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-09-11T23:03:28.056 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-09-11T23:03:28.056 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-09-11T23:03:28.056 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-09-11T23:03:28.056 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-09-11T23:03:28.056 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-09-11T23:03:28.056 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 92, 2024-09-11T23:03:28.056 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-09-11T23:03:28.056 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 11, 2024-09-11T23:03:28.056 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 5, 2024-09-11T23:03:28.057 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 5, 2024-09-11T23:03:28.057 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-09-11T23:03:28.057 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-09-11T23:03:28.057 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-09-11T23:03:28.057 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-09-11T23:03:28.057 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-09-11T23:03:28.057 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.057 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-09-11T23:03:28.057 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-09-11T23:03:28.057 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:03:28.057 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-09-11T23:03:28.057 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-09-11T23:03:28.057 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.057 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-09-11T23:03:28.058 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-09-11T23:03:28.058 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.058 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-09-11T23:03:28.058 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-09-11T23:03:28.058 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:03:28.058 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-09-11T23:03:28.058 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-09-11T23:03:28.058 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-09-11T23:03:28.058 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-09-11T23:03:28.058 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-09-11T23:03:28.058 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.058 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:03:28.058 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:28.058 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-11T23:03:28.059 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:28.059 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2560, 2024-09-11T23:03:28.059 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 13165184, 2024-09-11T23:03:28.059 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:28.059 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2560, 2024-09-11T23:03:28.059 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-11T23:03:28.059 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-11T23:03:28.059 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2560, 2024-09-11T23:03:28.059 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 13165184, 2024-09-11T23:03:28.059 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:28.059 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.059 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.059 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.059 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:28.060 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.060 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-09-11T23:03:28.060 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:28.060 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-11T23:03:28.060 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:28.060 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-11T23:03:28.060 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-11T23:03:28.060 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:28.060 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:28.060 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2317, 2024-09-11T23:03:28.060 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 11267971, 2024-09-11T23:03:28.060 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2317, 2024-09-11T23:03:28.060 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 11267971, 2024-09-11T23:03:28.061 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:28.061 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.061 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.061 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.061 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:28.061 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.061 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-09-11T23:03:28.061 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:28.061 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-09-11T23:03:28.061 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:28.061 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-11T23:03:28.061 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-11T23:03:28.061 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:28.061 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:28.062 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2317, 2024-09-11T23:03:28.062 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2317, 2024-09-11T23:03:28.062 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2317, 2024-09-11T23:03:28.062 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2317, 2024-09-11T23:03:28.062 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:28.062 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.062 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.062 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.062 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:28.062 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.062 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:03:28.062 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:28.062 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-11T23:03:28.062 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:28.063 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-11T23:03:28.063 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-11T23:03:28.063 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:28.063 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:28.063 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-11T23:03:28.063 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-11T23:03:28.063 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-11T23:03:28.063 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-11T23:03:28.063 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:28.063 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.063 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.063 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.063 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:28.064 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:28.064 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55a1bc18eb40": { 2024-09-11T23:03:28.064 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:28.064 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-11T23:03:28.064 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:28.064 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-11T23:03:28.064 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-11T23:03:28.064 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:28.064 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:28.064 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-11T23:03:28.064 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-11T23:03:28.064 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-11T23:03:28.064 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-11T23:03:28.065 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:28.065 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:28.065 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:28.065 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:28.065 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:28.065 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:28.065 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-11T23:03:28.781 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-11T23:03:28.781 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-11T23:03:28.781 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-11T23:03:28.781 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:03:28.782 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:03:28.782 DEBUG:tasks.check_counter:Getting stats from i 2024-09-11T23:03:28.782 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-11T23:03:29.029 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:03:29.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:29 smithi152 ceph-mon[31181]: pgmap v328: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:03:29.491 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:29 smithi204 ceph-mon[35454]: pgmap v328: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:03:29.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:29 smithi191 ceph-mon[35211]: pgmap v328: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:03:29.593 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-11T23:03:29.593 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:03:29.593 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 476, 2024-09-11T23:03:29.593 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 624, 2024-09-11T23:03:29.593 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 136786, 2024-09-11T23:03:29.593 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 216479, 2024-09-11T23:03:29.593 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 14, 2024-09-11T23:03:29.593 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-09-11T23:03:29.593 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.068323455, 2024-09-11T23:03:29.594 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.023968596, 2024-09-11T23:03:29.594 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.034030006, 2024-09-11T23:03:29.594 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.005455598, 2024-09-11T23:03:29.594 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:29.594 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 623, 2024-09-11T23:03:29.594 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.046327650, 2024-09-11T23:03:29.594 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000074362 2024-09-11T23:03:29.594 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.594 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:29.594 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1020, 2024-09-11T23:03:29.594 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000273571, 2024-09-11T23:03:29.594 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000268 2024-09-11T23:03:29.595 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.595 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-11T23:03:29.595 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T23:03:29.595 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.595 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:03:29.595 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 370, 2024-09-11T23:03:29.595 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 352, 2024-09-11T23:03:29.595 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1535641, 2024-09-11T23:03:29.595 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 577162, 2024-09-11T23:03:29.595 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 43, 2024-09-11T23:03:29.595 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 9, 2024-09-11T23:03:29.595 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.063026604, 2024-09-11T23:03:29.595 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.019254670, 2024-09-11T23:03:29.595 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.056084539, 2024-09-11T23:03:29.596 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000375236, 2024-09-11T23:03:29.596 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:29.596 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 351, 2024-09-11T23:03:29.596 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.033580273, 2024-09-11T23:03:29.596 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000095670 2024-09-11T23:03:29.596 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.596 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:29.596 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 142, 2024-09-11T23:03:29.596 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000026146, 2024-09-11T23:03:29.596 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000184 2024-09-11T23:03:29.596 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.596 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1507424, 2024-09-11T23:03:29.597 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 137024 2024-09-11T23:03:29.597 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.597 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:03:29.597 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 65, 2024-09-11T23:03:29.597 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 224, 2024-09-11T23:03:29.597 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 24272, 2024-09-11T23:03:29.597 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 499880, 2024-09-11T23:03:29.597 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 13, 2024-09-11T23:03:29.597 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 5, 2024-09-11T23:03:29.597 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.017554884, 2024-09-11T23:03:29.597 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.008391368, 2024-09-11T23:03:29.597 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.007718584, 2024-09-11T23:03:29.597 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000231950, 2024-09-11T23:03:29.597 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:29.598 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 221, 2024-09-11T23:03:29.598 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.031577363, 2024-09-11T23:03:29.598 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000142883 2024-09-11T23:03:29.598 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.598 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:29.598 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 106, 2024-09-11T23:03:29.598 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000043223, 2024-09-11T23:03:29.598 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000407 2024-09-11T23:03:29.598 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.598 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-09-11T23:03:29.598 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 445616 2024-09-11T23:03:29.598 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.598 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-09-11T23:03:29.599 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-09-11T23:03:29.599 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-09-11T23:03:29.599 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.599 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-09-11T23:03:29.599 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-11T23:03:29.599 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-11T23:03:29.599 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 50, 2024-09-11T23:03:29.599 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.001890781, 2024-09-11T23:03:29.599 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000037815 2024-09-11T23:03:29.599 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:29.599 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.599 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-09-11T23:03:29.599 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-11T23:03:29.599 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-11T23:03:29.600 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2, 2024-09-11T23:03:29.600 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000007632, 2024-09-11T23:03:29.600 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000003816 2024-09-11T23:03:29.600 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:29.600 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.600 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-09-11T23:03:29.600 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-09-11T23:03:29.600 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-09-11T23:03:29.600 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-09-11T23:03:29.600 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.600 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.600 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.600 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.601 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-09-11T23:03:29.601 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-09-11T23:03:29.601 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-09-11T23:03:29.601 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-09-11T23:03:29.601 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 11, 2024-09-11T23:03:29.601 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-09-11T23:03:29.601 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-09-11T23:03:29.601 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 10, 2024-09-11T23:03:29.601 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-09-11T23:03:29.601 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 10, 2024-09-11T23:03:29.602 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-09-11T23:03:29.602 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 10, 2024-09-11T23:03:29.602 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-09-11T23:03:29.602 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-09-11T23:03:29.602 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-09-11T23:03:29.602 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 1, 2024-09-11T23:03:29.602 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 6, 2024-09-11T23:03:29.602 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 6, 2024-09-11T23:03:29.602 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-09-11T23:03:29.602 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-09-11T23:03:29.602 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:03:29.602 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-09-11T23:03:29.602 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-09-11T23:03:29.603 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-09-11T23:03:29.603 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-09-11T23:03:29.603 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-09-11T23:03:29.603 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-09-11T23:03:29.603 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-09-11T23:03:29.603 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-09-11T23:03:29.603 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-09-11T23:03:29.603 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:03:29.603 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-09-11T23:03:29.603 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 4, 2024-09-11T23:03:29.603 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 234, 2024-09-11T23:03:29.603 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 6, 2024-09-11T23:03:29.603 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:03:29.603 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-09-11T23:03:29.604 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:03:29.604 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:03:29.604 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:03:29.604 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:03:29.604 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:03:29.604 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-09-11T23:03:29.604 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:03:29.604 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:03:29.604 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:03:29.604 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:03:29.604 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:03:29.604 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:03:29.605 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-09-11T23:03:29.605 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:03:29.605 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-09-11T23:03:29.605 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-09-11T23:03:29.605 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.605 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-09-11T23:03:29.605 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-09-11T23:03:29.605 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-09-11T23:03:29.605 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-09-11T23:03:29.605 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 1, 2024-09-11T23:03:29.605 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-09-11T23:03:29.605 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-09-11T23:03:29.605 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-09-11T23:03:29.606 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:03:29.606 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-09-11T23:03:29.606 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-09-11T23:03:29.606 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-09-11T23:03:29.606 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-09-11T23:03:29.606 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-09-11T23:03:29.606 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:03:29.606 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:03:29.606 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-09-11T23:03:29.606 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-09-11T23:03:29.606 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 3, 2024-09-11T23:03:29.606 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 3, 2024-09-11T23:03:29.606 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:03:29.607 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-09-11T23:03:29.607 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-09-11T23:03:29.607 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:03:29.607 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-09-11T23:03:29.607 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-09-11T23:03:29.607 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.607 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-09-11T23:03:29.607 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-09-11T23:03:29.607 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 8, 2024-09-11T23:03:29.607 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-09-11T23:03:29.607 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-09-11T23:03:29.607 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 8, 2024-09-11T23:03:29.607 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-09-11T23:03:29.607 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-09-11T23:03:29.608 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 2, 2024-09-11T23:03:29.608 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-09-11T23:03:29.608 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-09-11T23:03:29.608 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-09-11T23:03:29.608 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-09-11T23:03:29.608 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-09-11T23:03:29.608 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-09-11T23:03:29.608 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-09-11T23:03:29.608 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 4196494, 2024-09-11T23:03:29.608 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 0, 2024-09-11T23:03:29.608 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-09-11T23:03:29.608 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 7, 2024-09-11T23:03:29.608 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.010998594, 2024-09-11T23:03:29.609 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001571227 2024-09-11T23:03:29.609 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.609 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 0 2024-09-11T23:03:29.609 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.609 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-09-11T23:03:29.609 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 13, 2024-09-11T23:03:29.609 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 13, 2024-09-11T23:03:29.609 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 0, 2024-09-11T23:03:29.609 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 11, 2024-09-11T23:03:29.609 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 12, 2024-09-11T23:03:29.609 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 1, 2024-09-11T23:03:29.609 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 10, 2024-09-11T23:03:29.609 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 10, 2024-09-11T23:03:29.610 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 0, 2024-09-11T23:03:29.610 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-09-11T23:03:29.610 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-09-11T23:03:29.610 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-09-11T23:03:29.610 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 53060, 2024-09-11T23:03:29.610 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 223520 2024-09-11T23:03:29.610 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.610 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-09-11T23:03:29.610 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-09-11T23:03:29.610 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-09-11T23:03:29.610 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-09-11T23:03:29.610 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 12, 2024-09-11T23:03:29.610 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-09-11T23:03:29.610 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-09-11T23:03:29.611 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.611 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.611 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.611 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.611 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-09-11T23:03:29.611 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.611 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.611 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.611 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.611 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-09-11T23:03:29.611 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.611 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.611 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.612 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.612 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-09-11T23:03:29.612 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.612 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.612 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.612 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.612 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-09-11T23:03:29.612 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.612 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.612 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.612 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.612 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-09-11T23:03:29.612 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.613 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.613 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.613 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.613 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-09-11T23:03:29.613 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.613 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.613 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.613 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.613 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-09-11T23:03:29.613 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.613 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.613 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.613 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.613 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-09-11T23:03:29.614 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.614 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.614 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.614 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.614 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-09-11T23:03:29.614 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.614 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.614 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.614 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.614 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-09-11T23:03:29.614 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.614 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.614 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.614 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.615 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-09-11T23:03:29.615 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.615 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.615 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.615 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.616 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-09-11T23:03:29.616 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.616 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.616 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.616 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.616 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-09-11T23:03:29.616 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.616 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.616 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.616 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.616 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-09-11T23:03:29.616 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.616 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.617 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.617 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.617 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-09-11T23:03:29.617 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.617 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.617 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.617 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.617 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-09-11T23:03:29.617 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.617 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.617 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.617 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.617 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-09-11T23:03:29.618 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.618 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.618 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.618 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.618 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-09-11T23:03:29.618 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.618 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.618 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.618 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.618 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-09-11T23:03:29.618 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.618 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.618 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.618 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.619 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-09-11T23:03:29.619 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.619 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.619 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.619 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.619 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-09-11T23:03:29.619 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.619 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.619 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.619 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.619 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-09-11T23:03:29.619 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.619 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.620 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.620 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.620 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-09-11T23:03:29.620 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.620 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.620 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.620 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.620 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-09-11T23:03:29.620 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.620 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.620 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.620 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.620 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-09-11T23:03:29.621 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.621 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.621 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.621 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.621 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-09-11T23:03:29.621 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.621 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.621 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.621 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.621 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-09-11T23:03:29.621 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.621 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.621 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.621 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.622 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-09-11T23:03:29.622 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.622 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.622 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.622 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.622 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:03:29.622 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:03:29.622 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-09-11T23:03:29.622 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.622 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.622 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.622 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:29.622 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.623 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-09-11T23:03:29.623 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 0, 2024-09-11T23:03:29.623 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 3, 2024-09-11T23:03:29.623 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 3, 2024-09-11T23:03:29.623 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 0, 2024-09-11T23:03:29.623 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-09-11T23:03:29.623 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-09-11T23:03:29.623 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-09-11T23:03:29.623 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 85, 2024-09-11T23:03:29.623 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-09-11T23:03:29.623 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.623 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-09-11T23:03:29.623 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:03:29.623 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-09-11T23:03:29.624 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:03:29.624 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:03:29.624 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:03:29.624 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:03:29.624 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:03:29.624 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:03:29.624 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:03:29.624 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:03:29.624 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:03:29.624 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:03:29.624 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:03:29.624 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:03:29.624 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:03:29.625 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-09-11T23:03:29.625 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:03:29.625 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-09-11T23:03:29.625 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:03:29.625 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:03:29.625 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:03:29.625 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:03:29.625 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:03:29.625 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:03:29.625 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:03:29.625 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-09-11T23:03:29.625 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:03:29.626 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:03:29.626 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:03:29.626 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-09-11T23:03:29.626 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-09-11T23:03:29.626 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-09-11T23:03:29.626 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:03:29.626 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:03:29.626 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:03:29.626 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:03:29.626 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 238519, 2024-09-11T23:03:29.626 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 83, 2024-09-11T23:03:29.626 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:03:29.626 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-09-11T23:03:29.626 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-09-11T23:03:29.627 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-09-11T23:03:29.627 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:03:29.627 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-09-11T23:03:29.627 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:03:29.627 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-09-11T23:03:29.627 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8040, 2024-09-11T23:03:29.627 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 178, 2024-09-11T23:03:29.627 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:03:29.627 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:03:29.627 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-09-11T23:03:29.627 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-09-11T23:03:29.627 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 50872, 2024-09-11T23:03:29.627 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 198, 2024-09-11T23:03:29.627 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-09-11T23:03:29.628 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-09-11T23:03:29.628 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-09-11T23:03:29.628 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-09-11T23:03:29.628 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.628 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-09-11T23:03:29.628 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-09-11T23:03:29.628 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-09-11T23:03:29.628 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 44, 2024-09-11T23:03:29.628 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 12582, 2024-09-11T23:03:29.628 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 28, 2024-09-11T23:03:29.628 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 44, 2024-09-11T23:03:29.628 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-09-11T23:03:29.628 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 44, 2024-09-11T23:03:29.629 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.218997204, 2024-09-11T23:03:29.629 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.004977209 2024-09-11T23:03:29.629 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.629 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-09-11T23:03:29.629 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-09-11T23:03:29.629 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 44, 2024-09-11T23:03:29.629 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 66 2024-09-11T23:03:29.629 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.629 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 44, 2024-09-11T23:03:29.629 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 0, 2024-09-11T23:03:29.629 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 44, 2024-09-11T23:03:29.629 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-09-11T23:03:29.629 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-09-11T23:03:29.629 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 0, 2024-09-11T23:03:29.630 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 10, 2024-09-11T23:03:29.630 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 0, 2024-09-11T23:03:29.630 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 7, 2024-09-11T23:03:29.630 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 8, 2024-09-11T23:03:29.630 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-09-11T23:03:29.630 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-09-11T23:03:29.630 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-09-11T23:03:29.630 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-09-11T23:03:29.630 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 6, 2024-09-11T23:03:29.630 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-09-11T23:03:29.630 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-09-11T23:03:29.630 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-09-11T23:03:29.630 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 0, 2024-09-11T23:03:29.630 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 20, 2024-09-11T23:03:29.631 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:03:29.631 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-09-11T23:03:29.631 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:03:29.631 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-09-11T23:03:29.631 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-09-11T23:03:29.631 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-09-11T23:03:29.631 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:03:29.631 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-09-11T23:03:29.631 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:03:29.631 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-09-11T23:03:29.631 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-09-11T23:03:29.631 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-09-11T23:03:29.631 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-09-11T23:03:29.631 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-09-11T23:03:29.632 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-09-11T23:03:29.632 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-09-11T23:03:29.632 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-09-11T23:03:29.632 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-09-11T23:03:29.632 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-09-11T23:03:29.632 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-09-11T23:03:29.632 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-09-11T23:03:29.632 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-09-11T23:03:29.632 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-09-11T23:03:29.632 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-09-11T23:03:29.632 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-09-11T23:03:29.632 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-09-11T23:03:29.632 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 92, 2024-09-11T23:03:29.633 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-09-11T23:03:29.633 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 11, 2024-09-11T23:03:29.633 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 9, 2024-09-11T23:03:29.633 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 9, 2024-09-11T23:03:29.633 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-09-11T23:03:29.633 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-09-11T23:03:29.633 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 14, 2024-09-11T23:03:29.633 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 0, 2024-09-11T23:03:29.633 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 1 2024-09-11T23:03:29.633 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.633 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-09-11T23:03:29.633 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 1, 2024-09-11T23:03:29.633 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:03:29.633 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-09-11T23:03:29.634 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-09-11T23:03:29.634 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.634 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-09-11T23:03:29.634 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 4 2024-09-11T23:03:29.634 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.634 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-09-11T23:03:29.634 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-09-11T23:03:29.634 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:03:29.634 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-09-11T23:03:29.635 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-09-11T23:03:29.635 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-09-11T23:03:29.635 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-09-11T23:03:29.635 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-09-11T23:03:29.635 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.635 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:03:29.635 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:29.635 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-11T23:03:29.635 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:29.635 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 907, 2024-09-11T23:03:29.635 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1597973, 2024-09-11T23:03:29.635 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:29.635 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 907, 2024-09-11T23:03:29.635 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-11T23:03:29.636 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-11T23:03:29.636 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 907, 2024-09-11T23:03:29.636 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1597973, 2024-09-11T23:03:29.636 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:29.636 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.636 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.636 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.636 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:29.636 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.636 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-09-11T23:03:29.636 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:29.636 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-11T23:03:29.636 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:29.637 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-11T23:03:29.637 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-11T23:03:29.637 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:29.637 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:29.637 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 44, 2024-09-11T23:03:29.637 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 12582, 2024-09-11T23:03:29.637 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 44, 2024-09-11T23:03:29.637 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 12582, 2024-09-11T23:03:29.637 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:29.637 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.637 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.637 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.637 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:29.637 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.638 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-09-11T23:03:29.638 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:29.638 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-09-11T23:03:29.638 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:29.638 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-11T23:03:29.638 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-11T23:03:29.638 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:29.638 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:29.638 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 44, 2024-09-11T23:03:29.638 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 44, 2024-09-11T23:03:29.638 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 44, 2024-09-11T23:03:29.638 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 44, 2024-09-11T23:03:29.638 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:29.638 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.639 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.639 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.639 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:29.639 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.639 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:03:29.639 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:29.639 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-11T23:03:29.639 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:29.639 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-11T23:03:29.639 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-11T23:03:29.639 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:29.639 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:29.639 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-11T23:03:29.639 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-11T23:03:29.640 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-11T23:03:29.640 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-11T23:03:29.640 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:29.640 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.640 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.640 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.640 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:29.640 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:29.640 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55d13957e960": { 2024-09-11T23:03:29.640 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:29.640 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-11T23:03:29.640 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:29.640 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 8, 2024-09-11T23:03:29.641 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 2190, 2024-09-11T23:03:29.641 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:29.641 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 8, 2024-09-11T23:03:29.641 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-11T23:03:29.641 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-11T23:03:29.642 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 7, 2024-09-11T23:03:29.642 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2190, 2024-09-11T23:03:29.642 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:29.642 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:29.642 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:29.642 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:29.642 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:29.642 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:29.642 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-11T23:03:30.399 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-11T23:03:30.400 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-11T23:03:30.400 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-11T23:03:30.400 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:03:30.400 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:03:30.400 DEBUG:tasks.check_counter:Getting stats from l 2024-09-11T23:03:30.400 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-11T23:03:30.647 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-09-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 750, 2024-09-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 752, 2024-09-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 199124, 2024-09-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 238564, 2024-09-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-09-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 2, 2024-09-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.079379012, 2024-09-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.023988281, 2024-09-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.023570410, 2024-09-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.023825668, 2024-09-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 750, 2024-09-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.041289435, 2024-09-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000055052 2024-09-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 248, 2024-09-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 362, 2024-09-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 1491506, 2024-09-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 453457, 2024-09-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 4, 2024-09-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 3, 2024-09-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.051414332, 2024-09-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.021556033, 2024-09-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.041890549, 2024-09-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.000065106, 2024-09-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 361, 2024-09-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.027744631, 2024-09-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000076854 2024-09-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 1490992, 2024-09-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 452816 2024-09-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_messages": 1383, 2024-09-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages": 1416, 2024-09-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_bytes": 409010, 2024-09-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_bytes": 566508, 2024-09-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_created_connections": 6, 2024-09-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_active_connections": 4, 2024-09-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_total_time": 0.155836211, 2024-09-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_send_time": 0.045156800, 2024-09-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_recv_time": 0.041600717, 2024-09-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_running_fast_dispatch_time": 0.054749733, 2024-09-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 1414, 2024-09-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.065389440, 2024-09-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000046244 2024-09-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_handle_ack_lat": { 2024-09-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi204.stdout: "msgr_send_encrypted_bytes": 128304 2024-09-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "cct": { 2024-09-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "total_workers": 1, 2024-09-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "unhealthy_workers": 0 2024-09-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-MDSRank": { 2024-09-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 3575, 2024-09-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.110996306, 2024-09-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000031047 2024-09-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "finisher-PurgeQueue": { 2024-09-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "queue_len": 0, 2024-09-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "complete_latency": { 2024-09-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "mds": { 2024-09-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "request": 0, 2024-09-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi204.stdout: "reply": 0, 2024-09-11T23:03:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: "reply_latency": { 2024-09-11T23:03:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.151 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_reply": 0, 2024-09-11T23:03:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "forward": 0, 2024-09-11T23:03:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_complete": 0, 2024-09-11T23:03:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_fetch_keys": 0, 2024-09-11T23:03:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_commit": 0, 2024-09-11T23:03:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_split": 0, 2024-09-11T23:03:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_merge": 0, 2024-09-11T23:03:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes": 0, 2024-09-11T23:03:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_top": 0, 2024-09-11T23:03:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_bottom": 0, 2024-09-11T23:03:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pin_tail": 0, 2024-09-11T23:03:31.152 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_pinned": 0, 2024-09-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_expired": 0, 2024-09-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "inodes_with_caps": 0, 2024-09-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "caps": 0, 2024-09-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "subtrees": 1, 2024-09-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse": 0, 2024-09-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_hit": 0, 2024-09-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_forward": 0, 2024-09-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_discover": 0, 2024-09-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_dir_fetch": 0, 2024-09-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_remote_ino": 0, 2024-09-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "traverse_lock": 0, 2024-09-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "load_cent": 0, 2024-09-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "q": 0, 2024-09-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi204.stdout: "exported": 0, 2024-09-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "exported_inodes": 0, 2024-09-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "imported": 0, 2024-09-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "imported_inodes": 0, 2024-09-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_dir_fetch": 0, 2024-09-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_backtrace_fetch": 0, 2024-09-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "openino_peer_discover": 0, 2024-09-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rfiles": 0, 2024-09-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rbytes": 0, 2024-09-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "root_rsnaps": 0, 2024-09-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_set_tag": 0, 2024-09-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_inotable_repaired": 0, 2024-09-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_inodes": 0, 2024-09-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "scrub_file_inodes": 0, 2024-09-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_inode_file_caps": 0, 2024-09-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_grant": 0, 2024-09-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps": 0, 2024-09-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_caps_dirty": 0, 2024-09-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_cap_release": 0, 2024-09-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "process_request_cap_release": 0 2024-09-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_cache": { 2024-09-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update": 0, 2024-09-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_update_receipt": 0, 2024-09-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_try_discover": 0, 2024-09-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_send_discover": 0, 2024-09-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "dir_handle_discover": 0, 2024-09-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays": 0, 2024-09-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_delayed": 0, 2024-09-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "num_strays_enqueuing": 0, 2024-09-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_created": 0, 2024-09-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_enqueued": 0, 2024-09-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_reintegrated": 0, 2024-09-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "strays_migrated": 0, 2024-09-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_processing": 0, 2024-09-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_enqueued": 0, 2024-09-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "num_recovering_prioritized": 0, 2024-09-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_started": 0, 2024-09-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "recovery_completed": 0, 2024-09-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_path": 0, 2024-09-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_quiesce_inode": 0, 2024-09-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_exportdir": 0, 2024-09-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_flush": 0, 2024-09-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragmentdir": 0, 2024-09-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_fragstats": 0, 2024-09-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "ireq_inodestats": 0 2024-09-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_log": { 2024-09-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "evlrg": 0, 2024-09-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "evadd": 0, 2024-09-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "evex": 0, 2024-09-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "evtrm": 0, 2024-09-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "ev": 7, 2024-09-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "evexg": 0, 2024-09-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "evexd": 0, 2024-09-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "segadd": 0, 2024-09-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi204.stdout: "segex": 0, 2024-09-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "segtrm": 0, 2024-09-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "seg": 2, 2024-09-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "segmjr": 2, 2024-09-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "segexg": 0, 2024-09-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "segexd": 0, 2024-09-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "expos": 4194304, 2024-09-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "wrpos": 4196353, 2024-09-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "rdpos": 4196212, 2024-09-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "jlat": { 2024-09-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "replayed": 7 2024-09-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_mem": { 2024-09-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "ino": 3, 2024-09-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "ino+": 3, 2024-09-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "ino-": 0, 2024-09-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "dir": 1, 2024-09-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "dir+": 1, 2024-09-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "dir-": 0, 2024-09-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "dn": 0, 2024-09-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "dn+": 0, 2024-09-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "dn-": 0, 2024-09-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "cap": 0, 2024-09-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi204.stdout: "cap+": 0, 2024-09-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "cap-": 0, 2024-09-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "rss": 48284, 2024-09-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "heap": 215328 2024-09-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_server": { 2024-09-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_client_request": 0, 2024-09-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "dispatch_server_request": 0, 2024-09-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_request": 0, 2024-09-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_client_session": 0, 2024-09-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "handle_peer_request": 0, 2024-09-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "req_create_latency": { 2024-09-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getattr_latency": { 2024-09-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getfilelock_latency": { 2024-09-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "req_link_latency": { 2024-09-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookup_latency": { 2024-09-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookuphash_latency": { 2024-09-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupino_latency": { 2024-09-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupname_latency": { 2024-09-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupparent_latency": { 2024-09-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lookupsnap_latency": { 2024-09-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "req_lssnap_latency": { 2024-09-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mkdir_latency": { 2024-09-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mknod_latency": { 2024-09-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "req_mksnap_latency": { 2024-09-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "req_open_latency": { 2024-09-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "req_readdir_latency": { 2024-09-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rename_latency": { 2024-09-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi204.stdout: "req_renamesnap_latency": { 2024-09-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "req_snapdiff_latency": { 2024-09-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmdir_latency": { 2024-09-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmsnap_latency": { 2024-09-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "req_rmxattr_latency": { 2024-09-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setattr_latency": { 2024-09-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setdirlayout_latency": { 2024-09-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setfilelock_latency": { 2024-09-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setlayout_latency": { 2024-09-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi204.stdout: "req_setxattr_latency": { 2024-09-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi204.stdout: "req_symlink_latency": { 2024-09-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi204.stdout: "req_unlink_latency": { 2024-09-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_revoke_eviction": 0, 2024-09-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi204.stdout: "cap_acquisition_throttle": 0, 2024-09-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi204.stdout: "req_getvxattr_latency": { 2024-09-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_sessions": { 2024-09-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi204.stdout: "session_count": 1, 2024-09-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi204.stdout: "session_add": 3, 2024-09-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi204.stdout: "session_remove": 2, 2024-09-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_open": 3, 2024-09-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi204.stdout: "sessions_stale": 0, 2024-09-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi204.stdout: "total_load": 0, 2024-09-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi204.stdout: "average_load": 0, 2024-09-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi204.stdout: "avg_session_uptime": 4588, 2024-09-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi204.stdout: "mdthresh_evicted": 0 2024-09-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi204.stdout: "mempool": { 2024-09-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_bytes": 0, 2024-09-11T23:03:31.174 INFO:teuthology.orchestra.run.smithi204.stdout: "bloom_filter_items": 0, 2024-09-11T23:03:31.174 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T23:03:31.174 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_alloc_items": 0, 2024-09-11T23:03:31.174 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T23:03:31.174 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_data_items": 0, 2024-09-11T23:03:31.174 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T23:03:31.174 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T23:03:31.174 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T23:03:31.174 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T23:03:31.174 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T23:03:31.174 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_other_items": 0, 2024-09-11T23:03:31.174 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T23:03:31.174 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T23:03:31.174 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_bytes": 0, 2024-09-11T23:03:31.175 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_extent_items": 0, 2024-09-11T23:03:31.175 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_bytes": 0, 2024-09-11T23:03:31.175 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_blob_items": 0, 2024-09-11T23:03:31.175 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T23:03:31.175 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T23:03:31.175 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T23:03:31.175 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T23:03:31.175 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T23:03:31.175 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_fsck_items": 0, 2024-09-11T23:03:31.175 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_bytes": 0, 2024-09-11T23:03:31.175 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_txc_items": 0, 2024-09-11T23:03:31.175 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T23:03:31.175 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T23:03:31.176 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_bytes": 0, 2024-09-11T23:03:31.176 INFO:teuthology.orchestra.run.smithi204.stdout: "bluestore_writing_items": 0, 2024-09-11T23:03:31.176 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_bytes": 0, 2024-09-11T23:03:31.176 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_items": 0, 2024-09-11T23:03:31.176 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T23:03:31.176 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_reader_items": 0, 2024-09-11T23:03:31.176 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T23:03:31.176 INFO:teuthology.orchestra.run.smithi204.stdout: "bluefs_file_writer_items": 0, 2024-09-11T23:03:31.176 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_bytes": 189463, 2024-09-11T23:03:31.176 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_anon_items": 52, 2024-09-11T23:03:31.176 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_bytes": 0, 2024-09-11T23:03:31.176 INFO:teuthology.orchestra.run.smithi204.stdout: "buffer_meta_items": 0, 2024-09-11T23:03:31.176 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_bytes": 0, 2024-09-11T23:03:31.176 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_items": 0, 2024-09-11T23:03:31.177 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_bytes": 0, 2024-09-11T23:03:31.177 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_mapbl_items": 0, 2024-09-11T23:03:31.177 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_bytes": 0, 2024-09-11T23:03:31.177 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_pglog_items": 0, 2024-09-11T23:03:31.177 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_bytes": 8040, 2024-09-11T23:03:31.177 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_items": 178, 2024-09-11T23:03:31.177 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T23:03:31.177 INFO:teuthology.orchestra.run.smithi204.stdout: "osdmap_mapping_items": 0, 2024-09-11T23:03:31.177 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_bytes": 0, 2024-09-11T23:03:31.177 INFO:teuthology.orchestra.run.smithi204.stdout: "pgmap_items": 0, 2024-09-11T23:03:31.177 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_bytes": 8512, 2024-09-11T23:03:31.177 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_co_items": 13, 2024-09-11T23:03:31.177 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_bytes": 0, 2024-09-11T23:03:31.177 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_1_items": 0, 2024-09-11T23:03:31.178 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_bytes": 0, 2024-09-11T23:03:31.178 INFO:teuthology.orchestra.run.smithi204.stdout: "unittest_2_items": 0 2024-09-11T23:03:31.178 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.178 INFO:teuthology.orchestra.run.smithi204.stdout: "objecter": { 2024-09-11T23:03:31.178 INFO:teuthology.orchestra.run.smithi204.stdout: "op_active": 0, 2024-09-11T23:03:31.178 INFO:teuthology.orchestra.run.smithi204.stdout: "op_laggy": 0, 2024-09-11T23:03:31.178 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send": 2130, 2024-09-11T23:03:31.178 INFO:teuthology.orchestra.run.smithi204.stdout: "op_send_bytes": 21, 2024-09-11T23:03:31.178 INFO:teuthology.orchestra.run.smithi204.stdout: "op_resend": 3, 2024-09-11T23:03:31.178 INFO:teuthology.orchestra.run.smithi204.stdout: "op_reply": 2130, 2024-09-11T23:03:31.180 INFO:teuthology.orchestra.run.smithi204.stdout: "op_latency": { 2024-09-11T23:03:31.180 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2130, 2024-09-11T23:03:31.180 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 3.349957219, 2024-09-11T23:03:31.180 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.001572749 2024-09-11T23:03:31.180 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.180 INFO:teuthology.orchestra.run.smithi204.stdout: "op_inflight": 0, 2024-09-11T23:03:31.180 INFO:teuthology.orchestra.run.smithi204.stdout: "oplen_avg": { 2024-09-11T23:03:31.180 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 2130, 2024-09-11T23:03:31.180 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 2131 2024-09-11T23:03:31.180 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.180 INFO:teuthology.orchestra.run.smithi204.stdout: "op": 2130, 2024-09-11T23:03:31.180 INFO:teuthology.orchestra.run.smithi204.stdout: "op_r": 2130, 2024-09-11T23:03:31.181 INFO:teuthology.orchestra.run.smithi204.stdout: "op_w": 0, 2024-09-11T23:03:31.181 INFO:teuthology.orchestra.run.smithi204.stdout: "op_rmw": 0, 2024-09-11T23:03:31.181 INFO:teuthology.orchestra.run.smithi204.stdout: "op_pg": 0, 2024-09-11T23:03:31.181 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_stat": 1378, 2024-09-11T23:03:31.181 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_create": 0, 2024-09-11T23:03:31.181 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_read": 750, 2024-09-11T23:03:31.181 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_write": 0, 2024-09-11T23:03:31.181 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writefull": 0, 2024-09-11T23:03:31.181 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_writesame": 0, 2024-09-11T23:03:31.181 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_append": 0, 2024-09-11T23:03:31.181 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_zero": 0, 2024-09-11T23:03:31.181 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_truncate": 0, 2024-09-11T23:03:31.181 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_delete": 0, 2024-09-11T23:03:31.181 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_mapext": 0, 2024-09-11T23:03:31.182 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_sparse_read": 0, 2024-09-11T23:03:31.182 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_clonerange": 0, 2024-09-11T23:03:31.182 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_getxattr": 1, 2024-09-11T23:03:31.182 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_setxattr": 0, 2024-09-11T23:03:31.182 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_cmpxattr": 0, 2024-09-11T23:03:31.182 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_rmxattr": 0, 2024-09-11T23:03:31.182 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_resetxattrs": 0, 2024-09-11T23:03:31.182 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_call": 0, 2024-09-11T23:03:31.182 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_watch": 0, 2024-09-11T23:03:31.182 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_notify": 0, 2024-09-11T23:03:31.182 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T23:03:31.182 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls": 0, 2024-09-11T23:03:31.182 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_pgls_filter": 0, 2024-09-11T23:03:31.182 INFO:teuthology.orchestra.run.smithi204.stdout: "osdop_other": 0, 2024-09-11T23:03:31.183 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_active": 0, 2024-09-11T23:03:31.183 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_send": 0, 2024-09-11T23:03:31.183 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_resend": 0, 2024-09-11T23:03:31.183 INFO:teuthology.orchestra.run.smithi204.stdout: "linger_ping": 0, 2024-09-11T23:03:31.183 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_active": 0, 2024-09-11T23:03:31.183 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_send": 0, 2024-09-11T23:03:31.183 INFO:teuthology.orchestra.run.smithi204.stdout: "poolop_resend": 0, 2024-09-11T23:03:31.183 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_active": 0, 2024-09-11T23:03:31.183 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_send": 0, 2024-09-11T23:03:31.183 INFO:teuthology.orchestra.run.smithi204.stdout: "poolstat_resend": 0, 2024-09-11T23:03:31.183 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_active": 0, 2024-09-11T23:03:31.183 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_send": 0, 2024-09-11T23:03:31.183 INFO:teuthology.orchestra.run.smithi204.stdout: "statfs_resend": 0, 2024-09-11T23:03:31.183 INFO:teuthology.orchestra.run.smithi204.stdout: "command_active": 0, 2024-09-11T23:03:31.183 INFO:teuthology.orchestra.run.smithi204.stdout: "command_send": 0, 2024-09-11T23:03:31.184 INFO:teuthology.orchestra.run.smithi204.stdout: "command_resend": 0, 2024-09-11T23:03:31.184 INFO:teuthology.orchestra.run.smithi204.stdout: "map_epoch": 92, 2024-09-11T23:03:31.184 INFO:teuthology.orchestra.run.smithi204.stdout: "map_full": 0, 2024-09-11T23:03:31.184 INFO:teuthology.orchestra.run.smithi204.stdout: "map_inc": 11, 2024-09-11T23:03:31.184 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_sessions": 5, 2024-09-11T23:03:31.184 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_open": 5, 2024-09-11T23:03:31.184 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_session_close": 0, 2024-09-11T23:03:31.184 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_laggy": 0, 2024-09-11T23:03:31.184 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_wr": 0, 2024-09-11T23:03:31.184 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_rd": 2, 2024-09-11T23:03:31.184 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_del": 0 2024-09-11T23:03:31.184 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.185 INFO:teuthology.orchestra.run.smithi204.stdout: "oft": { 2024-09-11T23:03:31.185 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_objs": 0, 2024-09-11T23:03:31.185 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_kv_pairs": 0, 2024-09-11T23:03:31.185 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_updates": 0, 2024-09-11T23:03:31.185 INFO:teuthology.orchestra.run.smithi204.stdout: "omap_total_removes": 0 2024-09-11T23:03:31.185 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.185 INFO:teuthology.orchestra.run.smithi204.stdout: "osd-slow-ops": { 2024-09-11T23:03:31.185 INFO:teuthology.orchestra.run.smithi204.stdout: "slow_ops_count": 0 2024-09-11T23:03:31.185 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.185 INFO:teuthology.orchestra.run.smithi204.stdout: "purge_queue": { 2024-09-11T23:03:31.185 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops": 0, 2024-09-11T23:03:31.185 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T23:03:31.185 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing": 0, 2024-09-11T23:03:31.185 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executing_high_water": 0, 2024-09-11T23:03:31.186 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed_ops": 0, 2024-09-11T23:03:31.186 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_executed": 0, 2024-09-11T23:03:31.186 INFO:teuthology.orchestra.run.smithi204.stdout: "pq_item_in_journal": 0 2024-09-11T23:03:31.186 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.186 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T23:03:31.186 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:31.186 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-11T23:03:31.186 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:31.186 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 2377, 2024-09-11T23:03:31.186 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 1870187, 2024-09-11T23:03:31.186 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:31.186 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 2377, 2024-09-11T23:03:31.186 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-11T23:03:31.186 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-11T23:03:31.187 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2377, 2024-09-11T23:03:31.187 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 1870187, 2024-09-11T23:03:31.187 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:31.187 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.187 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.187 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.187 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:31.187 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.187 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_bytes": { 2024-09-11T23:03:31.187 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:31.187 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 104857600, 2024-09-11T23:03:31.187 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:31.187 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-11T23:03:31.187 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-11T23:03:31.188 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:31.188 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:31.188 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2130, 2024-09-11T23:03:31.188 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2054, 2024-09-11T23:03:31.188 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2130, 2024-09-11T23:03:31.188 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2054, 2024-09-11T23:03:31.188 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:31.188 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.188 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.188 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.188 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:31.188 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.188 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-objecter_ops": { 2024-09-11T23:03:31.189 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:31.189 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 1024, 2024-09-11T23:03:31.189 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:31.189 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-11T23:03:31.189 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-11T23:03:31.189 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:31.189 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:31.189 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 2130, 2024-09-11T23:03:31.189 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 2130, 2024-09-11T23:03:31.189 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 2130, 2024-09-11T23:03:31.189 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 2130, 2024-09-11T23:03:31.189 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:31.189 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.189 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.190 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.190 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:31.190 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.190 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle": { 2024-09-11T23:03:31.190 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:31.190 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-11T23:03:31.190 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:31.190 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-11T23:03:31.190 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-11T23:03:31.190 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:31.190 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:31.190 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-11T23:03:31.190 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-11T23:03:31.190 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-11T23:03:31.191 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-11T23:03:31.191 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:31.191 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.191 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.191 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.191 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:31.191 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-09-11T23:03:31.191 INFO:teuthology.orchestra.run.smithi204.stdout: "throttle-write_buf_throttle-0x55faa9866b40": { 2024-09-11T23:03:31.191 INFO:teuthology.orchestra.run.smithi204.stdout: "val": 0, 2024-09-11T23:03:31.191 INFO:teuthology.orchestra.run.smithi204.stdout: "max": 3758096384, 2024-09-11T23:03:31.191 INFO:teuthology.orchestra.run.smithi204.stdout: "get_started": 0, 2024-09-11T23:03:31.191 INFO:teuthology.orchestra.run.smithi204.stdout: "get": 0, 2024-09-11T23:03:31.191 INFO:teuthology.orchestra.run.smithi204.stdout: "get_sum": 0, 2024-09-11T23:03:31.191 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_fail": 0, 2024-09-11T23:03:31.192 INFO:teuthology.orchestra.run.smithi204.stdout: "get_or_fail_success": 0, 2024-09-11T23:03:31.192 INFO:teuthology.orchestra.run.smithi204.stdout: "take": 0, 2024-09-11T23:03:31.192 INFO:teuthology.orchestra.run.smithi204.stdout: "take_sum": 0, 2024-09-11T23:03:31.192 INFO:teuthology.orchestra.run.smithi204.stdout: "put": 0, 2024-09-11T23:03:31.192 INFO:teuthology.orchestra.run.smithi204.stdout: "put_sum": 0, 2024-09-11T23:03:31.192 INFO:teuthology.orchestra.run.smithi204.stdout: "wait": { 2024-09-11T23:03:31.192 INFO:teuthology.orchestra.run.smithi204.stdout: "avgcount": 0, 2024-09-11T23:03:31.192 INFO:teuthology.orchestra.run.smithi204.stdout: "sum": 0.000000000, 2024-09-11T23:03:31.192 INFO:teuthology.orchestra.run.smithi204.stdout: "avgtime": 0.000000000 2024-09-11T23:03:31.192 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:31.192 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-09-11T23:03:31.192 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-09-11T23:03:31.295 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:31 smithi204 ceph-mon[35454]: pgmap v329: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:03:31.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:31 smithi152 ceph-mon[31181]: pgmap v329: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:03:31.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:31 smithi191 ceph-mon[35211]: pgmap v329: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-11T23:03:31.934 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-11T23:03:31.934 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-11T23:03:31.934 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-11T23:03:31.934 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-11T23:03:31.934 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T23:03:31.934 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-11T23:03:31.954 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-11T23:03:31.954 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T23:03:31.954 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T23:03:31.976 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-09-11T23:03:31.976 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T23:03:32.036 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T23:03:32.338 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:32.895 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.838+0000 7f5823395640 1 -- 172.21.15.152:0/2234898939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f581c106f10 msgr2=0x7f581c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:32.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.838+0000 7f5823395640 1 --2- 172.21.15.152:0/2234898939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f581c106f10 0x7f581c10ea90 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5814009ca0 tx=0x7f581402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:03:32.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.838+0000 7f5823395640 1 -- 172.21.15.152:0/2234898939 shutdown_connections 2024-09-11T23:03:32.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.838+0000 7f5823395640 1 --2- 172.21.15.152:0/2234898939 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f581c106f10 0x7f581c10ea90 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:32.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.838+0000 7f5823395640 1 --2- 172.21.15.152:0/2234898939 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f581c102790 0x7f581c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:32.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.838+0000 7f5823395640 1 --2- 172.21.15.152:0/2234898939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f581c101de0 0x7f581c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:32.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.839+0000 7f5823395640 1 -- 172.21.15.152:0/2234898939 >> 172.21.15.152:0/2234898939 conn(0x7f581c0fd8c0 msgr2=0x7f581c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:32.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.839+0000 7f5823395640 1 -- 172.21.15.152:0/2234898939 shutdown_connections 2024-09-11T23:03:32.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.839+0000 7f5823395640 1 -- 172.21.15.152:0/2234898939 wait complete. 2024-09-11T23:03:32.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.839+0000 7f5823395640 1 Processor -- start 2024-09-11T23:03:32.896 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.839+0000 7f5823395640 1 -- start start 2024-09-11T23:03:32.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.839+0000 7f5823395640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f581c101de0 0x7f581c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:32.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.839+0000 7f5823395640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f581c102790 0x7f581c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:32.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.839+0000 7f5823395640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f581c106f10 0x7f581c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:32.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.839+0000 7f5823395640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f581c112010 con 0x7f581c106f10 2024-09-11T23:03:32.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.839+0000 7f5823395640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f581c111e90 con 0x7f581c102790 2024-09-11T23:03:32.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.839+0000 7f5823395640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f581c112190 con 0x7f581c101de0 2024-09-11T23:03:32.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f5821932640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f581c106f10 0x7f581c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:32.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f5821932640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f581c106f10 0x7f581c19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:34020/0 (socket says 172.21.15.152:34020) 2024-09-11T23:03:32.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f5821932640 1 -- 172.21.15.152:0/3360327375 learned_addr learned my addr 172.21.15.152:0/3360327375 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:32.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f5820930640 1 --2- 172.21.15.152:0/3360327375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f581c102790 0x7f581c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:32.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f5821131640 1 --2- 172.21.15.152:0/3360327375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f581c101de0 0x7f581c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:32.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f5821932640 1 -- 172.21.15.152:0/3360327375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f581c101de0 msgr2=0x7f581c198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:32.897 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f5821932640 1 --2- 172.21.15.152:0/3360327375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f581c101de0 0x7f581c198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:32.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f5821932640 1 -- 172.21.15.152:0/3360327375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f581c102790 msgr2=0x7f581c198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:32.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f5821932640 1 --2- 172.21.15.152:0/3360327375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f581c102790 0x7f581c198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:32.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f5821932640 1 -- 172.21.15.152:0/3360327375 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581c19d520 con 0x7f581c106f10 2024-09-11T23:03:32.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f5820930640 1 --2- 172.21.15.152:0/3360327375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f581c102790 0x7f581c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:03:32.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f5821131640 1 --2- 172.21.15.152:0/3360327375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f581c101de0 0x7f581c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:03:32.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f5821932640 1 --2- 172.21.15.152:0/3360327375 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f581c106f10 0x7f581c19ce40 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f5814031a60 tx=0x7f5814031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:32.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f58127fc640 1 -- 172.21.15.152:0/3360327375 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814003c30 con 0x7f581c106f10 2024-09-11T23:03:32.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f5823395640 1 -- 172.21.15.152:0/3360327375 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581c19d7b0 con 0x7f581c106f10 2024-09-11T23:03:32.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f58127fc640 1 -- 172.21.15.152:0/3360327375 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5814003dd0 con 0x7f581c106f10 2024-09-11T23:03:32.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f58127fc640 1 -- 172.21.15.152:0/3360327375 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58140047e0 con 0x7f581c106f10 2024-09-11T23:03:32.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.840+0000 7f5823395640 1 -- 172.21.15.152:0/3360327375 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f581c1a50f0 con 0x7f581c106f10 2024-09-11T23:03:32.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.841+0000 7f5823395640 1 -- 172.21.15.152:0/3360327375 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f581c102240 con 0x7f581c106f10 2024-09-11T23:03:32.898 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.843+0000 7f58127fc640 1 -- 172.21.15.152:0/3360327375 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f581402f790 con 0x7f581c106f10 2024-09-11T23:03:32.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.843+0000 7f58127fc640 1 --2- 172.21.15.152:0/3360327375 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58080a2cf0 0x7f58080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:32.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.843+0000 7f5821131640 1 --2- 172.21.15.152:0/3360327375 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58080a2cf0 0x7f58080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:32.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.843+0000 7f58127fc640 1 -- 172.21.15.152:0/3360327375 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8492+0+0 (secure 0 0 0) 0x7f58140f0800 con 0x7f581c106f10 2024-09-11T23:03:32.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.844+0000 7f5821131640 1 --2- 172.21.15.152:0/3360327375 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58080a2cf0 0x7f58080a51b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f580c009930 tx=0x7f580c008040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:32.899 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:32.848+0000 7f58127fc640 1 -- 172.21.15.152:0/3360327375 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58140bd550 con 0x7f581c106f10 2024-09-11T23:03:33.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:33.027+0000 7f5823395640 1 -- 172.21.15.152:0/3360327375 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f581c10a150 con 0x7f581c106f10 2024-09-11T23:03:33.028 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:33.028+0000 7f58127fc640 1 -- 172.21.15.152:0/3360327375 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v92) ==== 81+0+962 (secure 0 0 0) 0x7f58140bce50 con 0x7f581c106f10 2024-09-11T23:03:33.028 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152:0/2259758812 2024-09-12T22:52:53.067551+0000 2024-09-11T23:03:33.028 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152:0/3900732888 2024-09-12T22:52:53.067551+0000 2024-09-11T23:03:33.029 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152:0/2557435316 2024-09-12T22:52:53.067551+0000 2024-09-11T23:03:33.029 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152:0/3960601922 2024-09-12T22:43:31.970432+0000 2024-09-11T23:03:33.029 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152:6800/2200357575 2024-09-12T22:43:31.970432+0000 2024-09-11T23:03:33.029 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152:6801/2200357575 2024-09-12T22:43:31.970432+0000 2024-09-11T23:03:33.029 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152:0/4286398432 2024-09-12T22:43:31.970432+0000 2024-09-11T23:03:33.029 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152:0/2270507277 2024-09-12T22:43:31.970432+0000 2024-09-11T23:03:33.029 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152:0/2452559014 2024-09-12T22:43:54.679375+0000 2024-09-11T23:03:33.029 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152:0/3071755865 2024-09-12T22:43:54.679375+0000 2024-09-11T23:03:33.029 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152:6800/3438861948 2024-09-12T22:52:53.067551+0000 2024-09-11T23:03:33.029 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152:6800/1868213162 2024-09-12T22:43:54.679375+0000 2024-09-11T23:03:33.029 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152:6801/1868213162 2024-09-12T22:43:54.679375+0000 2024-09-11T23:03:33.029 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152:0/1721817189 2024-09-12T22:52:53.067551+0000 2024-09-11T23:03:33.030 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152:0/1542993850 2024-09-12T22:43:54.679375+0000 2024-09-11T23:03:33.030 INFO:teuthology.orchestra.run.smithi152.stdout:172.21.15.152:6801/3438861948 2024-09-12T22:52:53.067551+0000 2024-09-11T23:03:33.030 INFO:teuthology.orchestra.run.smithi152.stderr:listed 16 entries 2024-09-11T23:03:33.030 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:33.030+0000 7f5823395640 1 -- 172.21.15.152:0/3360327375 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58080a2cf0 msgr2=0x7f58080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:33.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:33.030+0000 7f5823395640 1 --2- 172.21.15.152:0/3360327375 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58080a2cf0 0x7f58080a51b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f580c009930 tx=0x7f580c008040 comp rx=0 tx=0).stop 2024-09-11T23:03:33.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:33.030+0000 7f5823395640 1 -- 172.21.15.152:0/3360327375 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f581c106f10 msgr2=0x7f581c19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:33.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:33.030+0000 7f5823395640 1 --2- 172.21.15.152:0/3360327375 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f581c106f10 0x7f581c19ce40 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f5814031a60 tx=0x7f5814031a90 comp rx=0 tx=0).stop 2024-09-11T23:03:33.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:33.030+0000 7f5823395640 1 -- 172.21.15.152:0/3360327375 shutdown_connections 2024-09-11T23:03:33.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:33.030+0000 7f5823395640 1 --2- 172.21.15.152:0/3360327375 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58080a2cf0 0x7f58080a51b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:33.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:33.030+0000 7f5823395640 1 --2- 172.21.15.152:0/3360327375 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f581c106f10 0x7f581c19ce40 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:33.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:33.030+0000 7f5823395640 1 --2- 172.21.15.152:0/3360327375 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f581c102790 0x7f581c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:33.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:33.030+0000 7f5823395640 1 --2- 172.21.15.152:0/3360327375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f581c101de0 0x7f581c198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:33.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:33.030+0000 7f5823395640 1 -- 172.21.15.152:0/3360327375 >> 172.21.15.152:0/3360327375 conn(0x7f581c0fd8c0 msgr2=0x7f581c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:33.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:33.030+0000 7f5823395640 1 -- 172.21.15.152:0/3360327375 shutdown_connections 2024-09-11T23:03:33.031 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:33.030+0000 7f5823395640 1 -- 172.21.15.152:0/3360327375 wait complete. 2024-09-11T23:03:33.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:33 smithi152 ceph-mon[31181]: pgmap v330: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:03:33.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:33 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3360327375' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T23:03:33.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:33 smithi191 ceph-mon[35211]: pgmap v330: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:03:33.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:33 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3360327375' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T23:03:33.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:33 smithi204 ceph-mon[35454]: pgmap v330: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T23:03:33.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:33 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3360327375' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T23:03:33.848 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T23:03:33.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:03:33.849 DEBUG:teuthology.orchestra.run.smithi152:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T23:03:33.891 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi152.front.sepia.ceph.com 2024-09-11T23:03:33.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:03:33.891 DEBUG:teuthology.orchestra.run.smithi152:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T23:03:33.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:03:33.955 DEBUG:teuthology.orchestra.run.smithi152:> ip netns list 2024-09-11T23:03:34.012 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T23:03:34.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:03:34.012 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T23:03:34.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T23:03:34.079 DEBUG:teuthology.orchestra.run.smithi152:> sudo ip link delete ceph-brx 2024-09-11T23:03:34.179 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T23:03:34.190 INFO:tasks.cephadm:Teardown begin 2024-09-11T23:03:34.190 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T23:03:34.190 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:03:34.372 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:34 smithi152 ceph-mon[31181]: pgmap v331: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:03:34.444 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:34.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:34 smithi191 ceph-mon[35211]: pgmap v331: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:03:34.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:34 smithi204 ceph-mon[35454]: pgmap v331: 97 pgs: 97 active+clean; 4.0 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T23:03:35.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:34.999+0000 7f4a42b04640 1 -- 172.21.15.152:0/2531383227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a3c1036e0 msgr2=0x7f4a3c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:35.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:34.999+0000 7f4a42b04640 1 --2- 172.21.15.152:0/2531383227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a3c1036e0 0x7f4a3c107b90 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009930 tx=0x7f4a2c02a420 comp rx=0 tx=0).stop 2024-09-11T23:03:35.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:34.999+0000 7f4a42b04640 1 -- 172.21.15.152:0/2531383227 shutdown_connections 2024-09-11T23:03:35.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:34.999+0000 7f4a42b04640 1 --2- 172.21.15.152:0/2531383227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a3c1082a0 0x7f4a3c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:35.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:34.999+0000 7f4a42b04640 1 --2- 172.21.15.152:0/2531383227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a3c1036e0 0x7f4a3c107b90 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:35.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:34.999+0000 7f4a42b04640 1 --2- 172.21.15.152:0/2531383227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a3c102d30 0x7f4a3c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:35.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:34.999+0000 7f4a42b04640 1 -- 172.21.15.152:0/2531383227 >> 172.21.15.152:0/2531383227 conn(0x7f4a3c0fc7d0 msgr2=0x7f4a3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:35.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:34.999+0000 7f4a42b04640 1 -- 172.21.15.152:0/2531383227 shutdown_connections 2024-09-11T23:03:35.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:34.999+0000 7f4a42b04640 1 -- 172.21.15.152:0/2531383227 wait complete. 2024-09-11T23:03:35.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a42b04640 1 Processor -- start 2024-09-11T23:03:35.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a42b04640 1 -- start start 2024-09-11T23:03:35.019 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a42b04640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a3c102d30 0x7f4a3c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:35.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a42b04640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a3c1036e0 0x7f4a3c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:35.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a42b04640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a3c1082a0 0x7f4a3c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:35.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a42b04640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c1120a0 con 0x7f4a3c1036e0 2024-09-11T23:03:35.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a42b04640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c111f20 con 0x7f4a3c102d30 2024-09-11T23:03:35.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a42b04640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c112220 con 0x7f4a3c1082a0 2024-09-11T23:03:35.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a3bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a3c1036e0 0x7f4a3c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:35.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a3bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a3c1036e0 0x7f4a3c198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:34042/0 (socket says 172.21.15.152:34042) 2024-09-11T23:03:35.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a3bfff640 1 -- 172.21.15.152:0/2797754592 learned_addr learned my addr 172.21.15.152:0/2797754592 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:35.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a408a0640 1 --2- 172.21.15.152:0/2797754592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a3c102d30 0x7f4a3c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:35.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a410a1640 1 --2- 172.21.15.152:0/2797754592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a3c1082a0 0x7f4a3c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:35.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a3bfff640 1 -- 172.21.15.152:0/2797754592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a3c1082a0 msgr2=0x7f4a3c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:35.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a3bfff640 1 --2- 172.21.15.152:0/2797754592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a3c1082a0 0x7f4a3c19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:35.020 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a3bfff640 1 -- 172.21.15.152:0/2797754592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a3c102d30 msgr2=0x7f4a3c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:35.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a3bfff640 1 --2- 172.21.15.152:0/2797754592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a3c102d30 0x7f4a3c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:35.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a3bfff640 1 -- 172.21.15.152:0/2797754592 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c19d4d0 con 0x7f4a3c1036e0 2024-09-11T23:03:35.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.000+0000 7f4a3bfff640 1 --2- 172.21.15.152:0/2797754592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a3c1036e0 0x7f4a3c198a60 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c005940 tx=0x7f4a2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:35.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.001+0000 7f4a39ffb640 1 -- 172.21.15.152:0/2797754592 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c03f030 con 0x7f4a3c1036e0 2024-09-11T23:03:35.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.001+0000 7f4a39ffb640 1 -- 172.21.15.152:0/2797754592 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a2c003ce0 con 0x7f4a3c1036e0 2024-09-11T23:03:35.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.001+0000 7f4a39ffb640 1 -- 172.21.15.152:0/2797754592 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c035070 con 0x7f4a3c1036e0 2024-09-11T23:03:35.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.001+0000 7f4a42b04640 1 -- 172.21.15.152:0/2797754592 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3c19d760 con 0x7f4a3c1036e0 2024-09-11T23:03:35.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.001+0000 7f4a42b04640 1 -- 172.21.15.152:0/2797754592 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a3c1a50a0 con 0x7f4a3c1036e0 2024-09-11T23:03:35.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.002+0000 7f4a42b04640 1 -- 172.21.15.152:0/2797754592 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a3c06a850 con 0x7f4a3c1036e0 2024-09-11T23:03:35.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.003+0000 7f4a39ffb640 1 -- 172.21.15.152:0/2797754592 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4a2c042030 con 0x7f4a3c1036e0 2024-09-11T23:03:35.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.004+0000 7f4a39ffb640 1 --2- 172.21.15.152:0/2797754592 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4a280a2cf0 0x7f4a280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:35.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.004+0000 7f4a39ffb640 1 -- 172.21.15.152:0/2797754592 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8492+0+0 (secure 0 0 0) 0x7f4a2c0efaa0 con 0x7f4a3c1036e0 2024-09-11T23:03:35.021 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.004+0000 7f4a408a0640 1 --2- 172.21.15.152:0/2797754592 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4a280a2cf0 0x7f4a280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:35.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.004+0000 7f4a408a0640 1 --2- 172.21.15.152:0/2797754592 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4a280a2cf0 0x7f4a280a51b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4a24007ec0 tx=0x7f4a240075b0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:35.022 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.007+0000 7f4a39ffb640 1 -- 172.21.15.152:0/2797754592 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a2c0bc090 con 0x7f4a3c1036e0 2024-09-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.205+0000 7f4a42b04640 1 -- 172.21.15.152:0/2797754592 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a3c10a2b0 con 0x7f4a3c1036e0 2024-09-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.206+0000 7f4a39ffb640 1 -- 172.21.15.152:0/2797754592 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f4a2c0c0390 con 0x7f4a3c1036e0 2024-09-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T23:03:35.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.208+0000 7f4a42b04640 1 -- 172.21.15.152:0/2797754592 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4a280a2cf0 msgr2=0x7f4a280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.208+0000 7f4a42b04640 1 --2- 172.21.15.152:0/2797754592 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4a280a2cf0 0x7f4a280a51b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4a24007ec0 tx=0x7f4a240075b0 comp rx=0 tx=0).stop 2024-09-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.208+0000 7f4a42b04640 1 -- 172.21.15.152:0/2797754592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a3c1036e0 msgr2=0x7f4a3c198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.208+0000 7f4a42b04640 1 --2- 172.21.15.152:0/2797754592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a3c1036e0 0x7f4a3c198a60 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c005940 tx=0x7f4a2c0042f0 comp rx=0 tx=0).stop 2024-09-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.208+0000 7f4a42b04640 1 -- 172.21.15.152:0/2797754592 shutdown_connections 2024-09-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.208+0000 7f4a42b04640 1 --2- 172.21.15.152:0/2797754592 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4a280a2cf0 0x7f4a280a51b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.208+0000 7f4a42b04640 1 --2- 172.21.15.152:0/2797754592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a3c1082a0 0x7f4a3c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.208+0000 7f4a42b04640 1 --2- 172.21.15.152:0/2797754592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a3c1036e0 0x7f4a3c198a60 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.208+0000 7f4a42b04640 1 --2- 172.21.15.152:0/2797754592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a3c102d30 0x7f4a3c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.208+0000 7f4a42b04640 1 -- 172.21.15.152:0/2797754592 >> 172.21.15.152:0/2797754592 conn(0x7f4a3c0fc7d0 msgr2=0x7f4a3c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.208+0000 7f4a42b04640 1 -- 172.21.15.152:0/2797754592 shutdown_connections 2024-09-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:35.208+0000 7f4a42b04640 1 -- 172.21.15.152:0/2797754592 wait complete. 2024-09-11T23:03:35.522 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:35 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2797754592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:03:35.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:35 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2797754592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:03:35.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:35 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2797754592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:03:36.002 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T23:03:36.250 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:36.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:36 smithi191 ceph-mon[35211]: pgmap v332: 97 pgs: 97 active+clean; 4.0 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-11T23:03:36.544 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:36 smithi152 ceph-mon[31181]: pgmap v332: 97 pgs: 97 active+clean; 4.0 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-11T23:03:36.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:36 smithi204 ceph-mon[35454]: pgmap v332: 97 pgs: 97 active+clean; 4.0 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-11T23:03:36.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.790+0000 7f7949938640 1 -- 172.21.15.152:0/3395899152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7944102570 msgr2=0x7f79441029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:36.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.790+0000 7f7949938640 1 --2- 172.21.15.152:0/3395899152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7944102570 0x7f79441029b0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f792c009900 tx=0x7f792c02a480 comp rx=0 tx=0).stop 2024-09-11T23:03:36.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.790+0000 7f7949938640 1 -- 172.21.15.152:0/3395899152 shutdown_connections 2024-09-11T23:03:36.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.790+0000 7f7949938640 1 --2- 172.21.15.152:0/3395899152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944102ef0 0x7f794410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:36.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.790+0000 7f7949938640 1 --2- 172.21.15.152:0/3395899152 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7944102570 0x7f79441029b0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:36.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.790+0000 7f7949938640 1 --2- 172.21.15.152:0/3395899152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7944100e20 0x7f7944101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:36.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.790+0000 7f7949938640 1 -- 172.21.15.152:0/3395899152 >> 172.21.15.152:0/3395899152 conn(0x7f79440fc7d0 msgr2=0x7f79440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:36.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.790+0000 7f7949938640 1 -- 172.21.15.152:0/3395899152 shutdown_connections 2024-09-11T23:03:36.848 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.790+0000 7f7949938640 1 -- 172.21.15.152:0/3395899152 wait complete. 2024-09-11T23:03:36.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f7949938640 1 Processor -- start 2024-09-11T23:03:36.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f7949938640 1 -- start start 2024-09-11T23:03:36.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f7949938640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944100e20 0x7f7944199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:36.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f7949938640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7944102570 0x7f7944199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:36.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f7949938640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7944102ef0 0x7f79441a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:36.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f7949938640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f7944110f80 con 0x7f7944102570 2024-09-11T23:03:36.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f7949938640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f7944110e00 con 0x7f7944102ef0 2024-09-11T23:03:36.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f7949938640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f7944111100 con 0x7f7944100e20 2024-09-11T23:03:36.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f79427fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7944102570 0x7f7944199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:36.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f79427fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7944102570 0x7f7944199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:34060/0 (socket says 172.21.15.152:34060) 2024-09-11T23:03:36.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f79427fc640 1 -- 172.21.15.152:0/1239402096 learned_addr learned my addr 172.21.15.152:0/1239402096 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:36.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f79437fe640 1 --2- 172.21.15.152:0/1239402096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7944102ef0 0x7f79441a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:36.849 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f7942ffd640 1 --2- 172.21.15.152:0/1239402096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944100e20 0x7f7944199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:36.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f79427fc640 1 -- 172.21.15.152:0/1239402096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944100e20 msgr2=0x7f7944199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:36.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f79427fc640 1 --2- 172.21.15.152:0/1239402096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944100e20 0x7f7944199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:36.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f79427fc640 1 -- 172.21.15.152:0/1239402096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7944102ef0 msgr2=0x7f79441a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:36.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f79427fc640 1 --2- 172.21.15.152:0/1239402096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7944102ef0 0x7f79441a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:36.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.791+0000 7f79427fc640 1 -- 172.21.15.152:0/1239402096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79441a4910 con 0x7f7944102570 2024-09-11T23:03:36.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.792+0000 7f7942ffd640 1 --2- 172.21.15.152:0/1239402096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944100e20 0x7f7944199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:03:36.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.792+0000 7f79427fc640 1 --2- 172.21.15.152:0/1239402096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7944102570 0x7f7944199890 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f792c006010 tx=0x7f792c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:36.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.792+0000 7f7948936640 1 -- 172.21.15.152:0/1239402096 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c03f070 con 0x7f7944102570 2024-09-11T23:03:36.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.792+0000 7f7948936640 1 -- 172.21.15.152:0/1239402096 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f792c031c30 con 0x7f7944102570 2024-09-11T23:03:36.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.792+0000 7f7949938640 1 -- 172.21.15.152:0/1239402096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79441a4ae0 con 0x7f7944102570 2024-09-11T23:03:36.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.792+0000 7f7948936640 1 -- 172.21.15.152:0/1239402096 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c035030 con 0x7f7944102570 2024-09-11T23:03:36.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.792+0000 7f7949938640 1 -- 172.21.15.152:0/1239402096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79441a4f90 con 0x7f7944102570 2024-09-11T23:03:36.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.793+0000 7f7949938640 1 -- 172.21.15.152:0/1239402096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7944101200 con 0x7f7944102570 2024-09-11T23:03:36.850 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.795+0000 7f7948936640 1 -- 172.21.15.152:0/1239402096 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f792c042070 con 0x7f7944102570 2024-09-11T23:03:36.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.795+0000 7f7948936640 1 --2- 172.21.15.152:0/1239402096 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f79300a2d70 0x7f79300a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:36.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.796+0000 7f7942ffd640 1 --2- 172.21.15.152:0/1239402096 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f79300a2d70 0x7f79300a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:36.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.796+0000 7f7948936640 1 -- 172.21.15.152:0/1239402096 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8492+0+0 (secure 0 0 0) 0x7f792c0f0080 con 0x7f7944102570 2024-09-11T23:03:36.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.796+0000 7f7942ffd640 1 --2- 172.21.15.152:0/1239402096 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f79300a2d70 0x7f79300a5230 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f793c0058c0 tx=0x7f793c005850 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:36.851 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:36.799+0000 7f7948936640 1 -- 172.21.15.152:0/1239402096 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f792c031de0 con 0x7f7944102570 2024-09-11T23:03:37.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:37.003+0000 7f7949938640 1 -- 172.21.15.152:0/1239402096 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7944109090 con 0x7f7944102570 2024-09-11T23:03:37.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:37.003+0000 7f7948936640 1 -- 172.21.15.152:0/1239402096 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11988 (secure 0 0 0) 0x7f792c0bcd80 con 0x7f7944102570 2024-09-11T23:03:37.004 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T23:03:37.004 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":28,"btime":"2024-09-11T22:52:52:095124+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1098962542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1098962542},{"type":"v1","addr":"172.21.15.152:6835","nonce":1098962542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2615590856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2615590856},{"type":"v1","addr":"172.21.15.152:6837","nonce":2615590856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:50:55.047332+0000","modified":"2024-09-11T22:52:52.044945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24455,"mds_1":24442,"mds_2":24434,"mds_3":24430,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6839/97179606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":97179606},{"type":"v1","addr":"172.21.15.152:6839","nonce":97179606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.152:6841/2492281944","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6840","nonce":2492281944},{"type":"v1","addr":"172.21.15.152:6841","nonce":2492281944}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24377":{"gid":24377,"name":"k","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6833/2321157769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2321157769},{"type":"v1","addr":"172.21.15.191:6833","nonce":2321157769}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24397":{"gid":24397,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/2200342090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2200342090},{"type":"v1","addr":"172.21.15.204:6833","nonce":2200342090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.191:6835/2471607007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6834","nonce":2471607007},{"type":"v1","addr":"172.21.15.191:6835","nonce":2471607007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/763950562","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":763950562},{"type":"v1","addr":"172.21.15.204:6835","nonce":763950562}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.204:6837/1863048373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":1863048373},{"type":"v1","addr":"172.21.15.204:6837","nonce":1863048373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.191:6837/314110375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6836","nonce":314110375},{"type":"v1","addr":"172.21.15.191:6837","nonce":314110375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.204:6839/3360312546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":3360312546},{"type":"v1","addr":"172.21.15.204:6839","nonce":3360312546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.191:6839/2588730805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6838","nonce":2588730805},{"type":"v1","addr":"172.21.15.191:6839","nonce":2588730805}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24455,"qdb_cluster":[14628,24430,24434,24442,24455]},"id":1}]} 2024-09-11T23:03:37.004 INFO:teuthology.orchestra.run.smithi152.stderr:dumped fsmap epoch 28 2024-09-11T23:03:37.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:37.006+0000 7f7949938640 1 -- 172.21.15.152:0/1239402096 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f79300a2d70 msgr2=0x7f79300a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:37.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:37.006+0000 7f7949938640 1 --2- 172.21.15.152:0/1239402096 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f79300a2d70 0x7f79300a5230 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f793c0058c0 tx=0x7f793c005850 comp rx=0 tx=0).stop 2024-09-11T23:03:37.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:37.006+0000 7f7949938640 1 -- 172.21.15.152:0/1239402096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7944102570 msgr2=0x7f7944199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:37.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:37.006+0000 7f7949938640 1 --2- 172.21.15.152:0/1239402096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7944102570 0x7f7944199890 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f792c006010 tx=0x7f792c003c10 comp rx=0 tx=0).stop 2024-09-11T23:03:37.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:37.006+0000 7f7949938640 1 -- 172.21.15.152:0/1239402096 shutdown_connections 2024-09-11T23:03:37.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:37.006+0000 7f7949938640 1 --2- 172.21.15.152:0/1239402096 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f79300a2d70 0x7f79300a5230 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:37.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:37.006+0000 7f7949938640 1 --2- 172.21.15.152:0/1239402096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7944102ef0 0x7f79441a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:37.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:37.006+0000 7f7949938640 1 --2- 172.21.15.152:0/1239402096 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7944102570 0x7f7944199890 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:37.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:37.006+0000 7f7949938640 1 --2- 172.21.15.152:0/1239402096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7944100e20 0x7f7944199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:37.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:37.006+0000 7f7949938640 1 -- 172.21.15.152:0/1239402096 >> 172.21.15.152:0/1239402096 conn(0x7f79440fc7d0 msgr2=0x7f79440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:37.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:37.006+0000 7f7949938640 1 -- 172.21.15.152:0/1239402096 shutdown_connections 2024-09-11T23:03:37.007 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:37.006+0000 7f7949938640 1 -- 172.21.15.152:0/1239402096 wait complete. 2024-09-11T23:03:37.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:37 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/1239402096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:03:37.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:37 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/1239402096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:03:37.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:37 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/1239402096' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T23:03:37.910 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T23:03:38.158 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:38.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:38 smithi191 ceph-mon[35211]: pgmap v333: 97 pgs: 97 active+clean; 4.0 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-11T23:03:38.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:38 smithi204 ceph-mon[35454]: pgmap v333: 97 pgs: 97 active+clean; 4.0 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-11T23:03:38.578 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:38 smithi152 ceph-mon[31181]: pgmap v333: 97 pgs: 97 active+clean; 4.0 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 8 op/s 2024-09-11T23:03:38.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.657+0000 7fcf35a57640 1 -- 172.21.15.152:0/1890192650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf30100a80 msgr2=0x7fcf30100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:38.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.657+0000 7fcf35a57640 1 --2- 172.21.15.152:0/1890192650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf30100a80 0x7fcf30100ee0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009930 tx=0x7fcf1802a420 comp rx=0 tx=0).stop 2024-09-11T23:03:38.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.657+0000 7fcf35a57640 1 -- 172.21.15.152:0/1890192650 shutdown_connections 2024-09-11T23:03:38.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.657+0000 7fcf35a57640 1 --2- 172.21.15.152:0/1890192650 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf30101420 0x7fcf3010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:38.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.657+0000 7fcf35a57640 1 --2- 172.21.15.152:0/1890192650 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf30100a80 0x7fcf30100ee0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:38.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.657+0000 7fcf35a57640 1 --2- 172.21.15.152:0/1890192650 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf30106a80 0x7fcf30106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:38.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.657+0000 7fcf35a57640 1 -- 172.21.15.152:0/1890192650 >> 172.21.15.152:0/1890192650 conn(0x7fcf300fc7a0 msgr2=0x7fcf300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:38.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.657+0000 7fcf35a57640 1 -- 172.21.15.152:0/1890192650 shutdown_connections 2024-09-11T23:03:38.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.657+0000 7fcf35a57640 1 -- 172.21.15.152:0/1890192650 wait complete. 2024-09-11T23:03:38.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.658+0000 7fcf35a57640 1 Processor -- start 2024-09-11T23:03:38.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.658+0000 7fcf35a57640 1 -- start start 2024-09-11T23:03:38.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.658+0000 7fcf35a57640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf30100a80 0x7fcf3019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:38.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.658+0000 7fcf35a57640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf30101420 0x7fcf3019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:38.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.658+0000 7fcf35a57640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf30106a80 0x7fcf3019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:38.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.658+0000 7fcf35a57640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fcf30110e80 con 0x7fcf30101420 2024-09-11T23:03:38.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.658+0000 7fcf35a57640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fcf30110d00 con 0x7fcf30106a80 2024-09-11T23:03:38.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.658+0000 7fcf35a57640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fcf30111000 con 0x7fcf30100a80 2024-09-11T23:03:38.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.658+0000 7fcf2e7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf30101420 0x7fcf3019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:38.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.658+0000 7fcf2e7fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf30101420 0x7fcf3019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:45998/0 (socket says 172.21.15.152:45998) 2024-09-11T23:03:38.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.659+0000 7fcf2e7fc640 1 -- 172.21.15.152:0/101184508 learned_addr learned my addr 172.21.15.152:0/101184508 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:38.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.659+0000 7fcf2f7fe640 1 --2- 172.21.15.152:0/101184508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf30106a80 0x7fcf3019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:38.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.659+0000 7fcf2effd640 1 --2- 172.21.15.152:0/101184508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf30100a80 0x7fcf3019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:38.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.659+0000 7fcf2e7fc640 1 -- 172.21.15.152:0/101184508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf30100a80 msgr2=0x7fcf3019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:38.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.659+0000 7fcf2e7fc640 1 --2- 172.21.15.152:0/101184508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf30100a80 0x7fcf3019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:38.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.659+0000 7fcf2e7fc640 1 -- 172.21.15.152:0/101184508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf30106a80 msgr2=0x7fcf3019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:38.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.659+0000 7fcf2e7fc640 1 --2- 172.21.15.152:0/101184508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf30106a80 0x7fcf3019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:38.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.659+0000 7fcf2e7fc640 1 -- 172.21.15.152:0/101184508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3019f4f0 con 0x7fcf30101420 2024-09-11T23:03:38.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.659+0000 7fcf2effd640 1 --2- 172.21.15.152:0/101184508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf30100a80 0x7fcf3019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:03:38.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.659+0000 7fcf2f7fe640 1 --2- 172.21.15.152:0/101184508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf30106a80 0x7fcf3019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:03:38.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.659+0000 7fcf2e7fc640 1 --2- 172.21.15.152:0/101184508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf30101420 0x7fcf3019aa80 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009a50 tx=0x7fcf18004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:38.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.659+0000 7fcf34a55640 1 -- 172.21.15.152:0/101184508 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1803f030 con 0x7fcf30101420 2024-09-11T23:03:38.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.659+0000 7fcf34a55640 1 -- 172.21.15.152:0/101184508 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf18003ce0 con 0x7fcf30101420 2024-09-11T23:03:38.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.659+0000 7fcf35a57640 1 -- 172.21.15.152:0/101184508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3019f780 con 0x7fcf30101420 2024-09-11T23:03:38.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.659+0000 7fcf34a55640 1 -- 172.21.15.152:0/101184508 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18035070 con 0x7fcf30101420 2024-09-11T23:03:38.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.659+0000 7fcf35a57640 1 -- 172.21.15.152:0/101184508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf301a70c0 con 0x7fcf30101420 2024-09-11T23:03:38.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.660+0000 7fcf35a57640 1 -- 172.21.15.152:0/101184508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf301021c0 con 0x7fcf30101420 2024-09-11T23:03:38.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.661+0000 7fcf34a55640 1 -- 172.21.15.152:0/101184508 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fcf18042030 con 0x7fcf30101420 2024-09-11T23:03:38.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.662+0000 7fcf34a55640 1 --2- 172.21.15.152:0/101184508 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcf1c0a2cf0 0x7fcf1c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:38.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.662+0000 7fcf2effd640 1 --2- 172.21.15.152:0/101184508 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcf1c0a2cf0 0x7fcf1c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:38.690 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.662+0000 7fcf34a55640 1 -- 172.21.15.152:0/101184508 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8492+0+0 (secure 0 0 0) 0x7fcf180efae0 con 0x7fcf30101420 2024-09-11T23:03:38.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.663+0000 7fcf2effd640 1 --2- 172.21.15.152:0/101184508 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcf1c0a2cf0 0x7fcf1c0a51b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000baa0 tx=0x7fcf20005d20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:38.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.666+0000 7fcf34a55640 1 -- 172.21.15.152:0/101184508 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf180bc6c0 con 0x7fcf30101420 2024-09-11T23:03:38.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.831+0000 7fcf35a57640 1 -- 172.21.15.152:0/101184508 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf301a7290 con 0x7fcf30101420 2024-09-11T23:03:38.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.833+0000 7fcf34a55640 1 -- 172.21.15.152:0/101184508 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+23184 (secure 0 0 0) 0x7fcf180bbfc0 con 0x7fcf30101420 2024-09-11T23:03:38.833 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-09-11T23:03:38.833 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":92,"fsid":"0e4e82bc-708f-11ef-bcea-c7b262605968","created":"2024-09-11T22:43:00.379317+0000","modified":"2024-09-11T23:03:20.298338+0000","last_up_change":"2024-09-11T22:50:01.043181+0000","last_in_change":"2024-09-11T22:49:38.073870+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:46:10.984707+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:50:50.833547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:50:53.047604+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":91,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3cd917b5-77f6-4a4a-8fd0-4ebe3034feac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6803","nonce":1270728007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6805","nonce":1270728007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6809","nonce":1270728007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1270728007},{"type":"v1","addr":"172.21.15.152:6807","nonce":1270728007}]},"public_addr":"172.21.15.152:6803/1270728007","cluster_addr":"172.21.15.152:6805/1270728007","heartbeat_back_addr":"172.21.15.152:6809/1270728007","heartbeat_front_addr":"172.21.15.152:6807/1270728007","state":["exists","up"]},{"osd":1,"uuid":"ae589bc4-77b1-4522-9591-b0ce2a5432dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6801","nonce":354467155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6803","nonce":354467155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6807","nonce":354467155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":354467155},{"type":"v1","addr":"172.21.15.191:6805","nonce":354467155}]},"public_addr":"172.21.15.191:6801/354467155","cluster_addr":"172.21.15.191:6803/354467155","heartbeat_back_addr":"172.21.15.191:6807/354467155","heartbeat_front_addr":"172.21.15.191:6805/354467155","state":["exists","up"]},{"osd":2,"uuid":"b35ffd0c-d084-43dd-a947-15132d78d59c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6801","nonce":3848396211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6803","nonce":3848396211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6807","nonce":3848396211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":3848396211},{"type":"v1","addr":"172.21.15.204:6805","nonce":3848396211}]},"public_addr":"172.21.15.204:6801/3848396211","cluster_addr":"172.21.15.204:6803/3848396211","heartbeat_back_addr":"172.21.15.204:6807/3848396211","heartbeat_front_addr":"172.21.15.204:6805/3848396211","state":["exists","up"]},{"osd":3,"uuid":"d66ba7e8-4ab7-449e-9460-2f44ffdf987f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6811","nonce":3351257181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6813","nonce":3351257181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6817","nonce":3351257181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":3351257181},{"type":"v1","addr":"172.21.15.152:6815","nonce":3351257181}]},"public_addr":"172.21.15.152:6811/3351257181","cluster_addr":"172.21.15.152:6813/3351257181","heartbeat_back_addr":"172.21.15.152:6817/3351257181","heartbeat_front_addr":"172.21.15.152:6815/3351257181","state":["exists","up"]},{"osd":4,"uuid":"e1b56468-6cc7-4d29-988e-9d281118ee36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6809","nonce":2967539563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6811","nonce":2967539563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6815","nonce":2967539563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":2967539563},{"type":"v1","addr":"172.21.15.191:6813","nonce":2967539563}]},"public_addr":"172.21.15.191:6809/2967539563","cluster_addr":"172.21.15.191:6811/2967539563","heartbeat_back_addr":"172.21.15.191:6815/2967539563","heartbeat_front_addr":"172.21.15.191:6813/2967539563","state":["exists","up"]},{"osd":5,"uuid":"89dabd3e-77e3-4294-9648-b112ef694e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6809","nonce":4283562158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6811","nonce":4283562158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6815","nonce":4283562158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":4283562158},{"type":"v1","addr":"172.21.15.204:6813","nonce":4283562158}]},"public_addr":"172.21.15.204:6809/4283562158","cluster_addr":"172.21.15.204:6811/4283562158","heartbeat_back_addr":"172.21.15.204:6815/4283562158","heartbeat_front_addr":"172.21.15.204:6813/4283562158","state":["exists","up"]},{"osd":6,"uuid":"46a0fded-b668-4aef-a68f-613cccc83483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6819","nonce":2484234042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6821","nonce":2484234042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6825","nonce":2484234042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":2484234042},{"type":"v1","addr":"172.21.15.152:6823","nonce":2484234042}]},"public_addr":"172.21.15.152:6819/2484234042","cluster_addr":"172.21.15.152:6821/2484234042","heartbeat_back_addr":"172.21.15.152:6825/2484234042","heartbeat_front_addr":"172.21.15.152:6823/2484234042","state":["exists","up"]},{"osd":7,"uuid":"e674bfe9-e2db-44a3-b74b-a21f198747c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6817","nonce":269608859}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6819","nonce":269608859}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6823","nonce":269608859}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":269608859},{"type":"v1","addr":"172.21.15.191:6821","nonce":269608859}]},"public_addr":"172.21.15.191:6817/269608859","cluster_addr":"172.21.15.191:6819/269608859","heartbeat_back_addr":"172.21.15.191:6823/269608859","heartbeat_front_addr":"172.21.15.191:6821/269608859","state":["exists","up"]},{"osd":8,"uuid":"0418bfc9-d6b6-488f-b4c7-0bc6291bb624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6817","nonce":1756190090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6819","nonce":1756190090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6823","nonce":1756190090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1756190090},{"type":"v1","addr":"172.21.15.204:6821","nonce":1756190090}]},"public_addr":"172.21.15.204:6817/1756190090","cluster_addr":"172.21.15.204:6819/1756190090","heartbeat_back_addr":"172.21.15.204:6823/1756190090","heartbeat_front_addr":"172.21.15.204:6821/1756190090","state":["exists","up"]},{"osd":9,"uuid":"751a6964-798f-4240-a2ba-7fa9f0c72657","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6827","nonce":1301191419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6829","nonce":1301191419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6833","nonce":1301191419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":1301191419},{"type":"v1","addr":"172.21.15.152:6831","nonce":1301191419}]},"public_addr":"172.21.15.152:6827/1301191419","cluster_addr":"172.21.15.152:6829/1301191419","heartbeat_back_addr":"172.21.15.152:6833/1301191419","heartbeat_front_addr":"172.21.15.152:6831/1301191419","state":["exists","up"]},{"osd":10,"uuid":"90e7c6ba-127c-4b33-b3c6-cedf72128451","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6825","nonce":3695294503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6827","nonce":3695294503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6831","nonce":3695294503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":3695294503},{"type":"v1","addr":"172.21.15.191:6829","nonce":3695294503}]},"public_addr":"172.21.15.191:6825/3695294503","cluster_addr":"172.21.15.191:6827/3695294503","heartbeat_back_addr":"172.21.15.191:6831/3695294503","heartbeat_front_addr":"172.21.15.191:6829/3695294503","state":["exists","up"]},{"osd":11,"uuid":"c76a93b1-a045-477b-98d0-58a2e9d44a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6825","nonce":1709433236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6827","nonce":1709433236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6831","nonce":1709433236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":1709433236},{"type":"v1","addr":"172.21.15.204:6829","nonce":1709433236}]},"public_addr":"172.21.15.204:6825/1709433236","cluster_addr":"172.21.15.204:6827/1709433236","heartbeat_back_addr":"172.21.15.204:6831/1709433236","heartbeat_front_addr":"172.21.15.204:6829/1709433236","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:17.259744+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:45:42.230389+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:07.172433+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:32.517060+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:46:58.900615+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:24.379803+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:47:50.058928+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:15.967299+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:48:42.025873+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:08.225448+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:33.910531+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:49:59.466423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6},{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.152:0/2259758812":"2024-09-12T22:52:53.067551+0000","172.21.15.152:0/3900732888":"2024-09-12T22:52:53.067551+0000","172.21.15.152:0/2557435316":"2024-09-12T22:52:53.067551+0000","172.21.15.152:0/3960601922":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6800/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:6801/2200357575":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/4286398432":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2270507277":"2024-09-12T22:43:31.970432+0000","172.21.15.152:0/2452559014":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/3071755865":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6800/3438861948":"2024-09-12T22:52:53.067551+0000","172.21.15.152:6800/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/1868213162":"2024-09-12T22:43:54.679375+0000","172.21.15.152:0/1721817189":"2024-09-12T22:52:53.067551+0000","172.21.15.152:0/1542993850":"2024-09-12T22:43:54.679375+0000","172.21.15.152:6801/3438861948":"2024-09-12T22:52:53.067551+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T23:03:38.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.835+0000 7fcf35a57640 1 -- 172.21.15.152:0/101184508 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcf1c0a2cf0 msgr2=0x7fcf1c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:38.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.835+0000 7fcf35a57640 1 --2- 172.21.15.152:0/101184508 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcf1c0a2cf0 0x7fcf1c0a51b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000baa0 tx=0x7fcf20005d20 comp rx=0 tx=0).stop 2024-09-11T23:03:38.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.835+0000 7fcf35a57640 1 -- 172.21.15.152:0/101184508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf30101420 msgr2=0x7fcf3019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:38.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.835+0000 7fcf35a57640 1 --2- 172.21.15.152:0/101184508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf30101420 0x7fcf3019aa80 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009a50 tx=0x7fcf18004270 comp rx=0 tx=0).stop 2024-09-11T23:03:38.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.835+0000 7fcf35a57640 1 -- 172.21.15.152:0/101184508 shutdown_connections 2024-09-11T23:03:38.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.835+0000 7fcf35a57640 1 --2- 172.21.15.152:0/101184508 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fcf1c0a2cf0 0x7fcf1c0a51b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:38.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.835+0000 7fcf35a57640 1 --2- 172.21.15.152:0/101184508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf30106a80 0x7fcf3019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:38.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.835+0000 7fcf35a57640 1 --2- 172.21.15.152:0/101184508 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcf30101420 0x7fcf3019aa80 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:38.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.835+0000 7fcf35a57640 1 --2- 172.21.15.152:0/101184508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf30100a80 0x7fcf3019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:38.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.835+0000 7fcf35a57640 1 -- 172.21.15.152:0/101184508 >> 172.21.15.152:0/101184508 conn(0x7fcf300fc7a0 msgr2=0x7fcf3006a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:38.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.835+0000 7fcf35a57640 1 -- 172.21.15.152:0/101184508 shutdown_connections 2024-09-11T23:03:38.836 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:38.835+0000 7fcf35a57640 1 -- 172.21.15.152:0/101184508 wait complete. 2024-09-11T23:03:39.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:39 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/101184508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:03:39.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:39 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/101184508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:03:39.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:39 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/101184508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T23:03:39.666 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-11T23:03:39.916 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:40.347 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:40 smithi152 ceph-mon[31181]: pgmap v334: 97 pgs: 97 active+clean; 4.0 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-11T23:03:40.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.429+0000 7f4125374640 1 -- 172.21.15.152:0/4015235613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4120101df0 msgr2=0x7f41201021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:40.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.429+0000 7f4125374640 1 --2- 172.21.15.152:0/4015235613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4120101df0 0x7f41201021d0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4110009900 tx=0x7f411002a410 comp rx=0 tx=0).stop 2024-09-11T23:03:40.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.429+0000 7f4125374640 1 -- 172.21.15.152:0/4015235613 shutdown_connections 2024-09-11T23:03:40.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.429+0000 7f4125374640 1 --2- 172.21.15.152:0/4015235613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4120106f20 0x7f412010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:40.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.429+0000 7f4125374640 1 --2- 172.21.15.152:0/4015235613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41201027a0 0x7f41201067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:40.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.429+0000 7f4125374640 1 --2- 172.21.15.152:0/4015235613 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4120101df0 0x7f41201021d0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:40.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.429+0000 7f4125374640 1 -- 172.21.15.152:0/4015235613 >> 172.21.15.152:0/4015235613 conn(0x7f41200fd8b0 msgr2=0x7f41200ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:40.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.429+0000 7f4125374640 1 -- 172.21.15.152:0/4015235613 shutdown_connections 2024-09-11T23:03:40.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.429+0000 7f4125374640 1 -- 172.21.15.152:0/4015235613 wait complete. 2024-09-11T23:03:40.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f4125374640 1 Processor -- start 2024-09-11T23:03:40.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f4125374640 1 -- start start 2024-09-11T23:03:40.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f4125374640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4120101df0 0x7f4120198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:40.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f4125374640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41201027a0 0x7f41201989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:40.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f4125374640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4120106f20 0x7f412019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:40.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f4125374640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4120111f80 con 0x7f4120106f20 2024-09-11T23:03:40.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f4125374640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4120111e00 con 0x7f41201027a0 2024-09-11T23:03:40.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f4125374640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4120112100 con 0x7f4120101df0 2024-09-11T23:03:40.457 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f411f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4120106f20 0x7f412019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:40.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f411f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4120106f20 0x7f412019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46018/0 (socket says 172.21.15.152:46018) 2024-09-11T23:03:40.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f411f7fe640 1 -- 172.21.15.152:0/3207875699 learned_addr learned my addr 172.21.15.152:0/3207875699 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:40.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f411effd640 1 --2- 172.21.15.152:0/3207875699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4120101df0 0x7f4120198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:40.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f411e7fc640 1 --2- 172.21.15.152:0/3207875699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41201027a0 0x7f41201989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:40.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f411f7fe640 1 -- 172.21.15.152:0/3207875699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4120101df0 msgr2=0x7f4120198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:40.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f411f7fe640 1 --2- 172.21.15.152:0/3207875699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4120101df0 0x7f4120198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:40.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f411f7fe640 1 -- 172.21.15.152:0/3207875699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41201027a0 msgr2=0x7f41201989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:40.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.430+0000 7f411f7fe640 1 --2- 172.21.15.152:0/3207875699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41201027a0 0x7f41201989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:40.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.431+0000 7f411f7fe640 1 -- 172.21.15.152:0/3207875699 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412019d410 con 0x7f4120106f20 2024-09-11T23:03:40.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.431+0000 7f411e7fc640 1 --2- 172.21.15.152:0/3207875699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41201027a0 0x7f41201989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T23:03:40.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.431+0000 7f411f7fe640 1 --2- 172.21.15.152:0/3207875699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4120106f20 0x7f412019cd30 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f411800eb10 tx=0x7f411800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:40.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.431+0000 7f4117fff640 1 -- 172.21.15.152:0/3207875699 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118004b20 con 0x7f4120106f20 2024-09-11T23:03:40.458 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.431+0000 7f4125374640 1 -- 172.21.15.152:0/3207875699 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412019d700 con 0x7f4120106f20 2024-09-11T23:03:40.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.431+0000 7f4117fff640 1 -- 172.21.15.152:0/3207875699 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4118004cc0 con 0x7f4120106f20 2024-09-11T23:03:40.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.431+0000 7f4117fff640 1 -- 172.21.15.152:0/3207875699 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118010760 con 0x7f4120106f20 2024-09-11T23:03:40.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.431+0000 7f4125374640 1 -- 172.21.15.152:0/3207875699 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41201a4fe0 con 0x7f4120106f20 2024-09-11T23:03:40.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.432+0000 7f4125374640 1 -- 172.21.15.152:0/3207875699 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41201021d0 con 0x7f4120106f20 2024-09-11T23:03:40.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.433+0000 7f4117fff640 1 -- 172.21.15.152:0/3207875699 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4118005020 con 0x7f4120106f20 2024-09-11T23:03:40.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.434+0000 7f4117fff640 1 --2- 172.21.15.152:0/3207875699 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f410c0a2cf0 0x7f410c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:40.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.434+0000 7f411effd640 1 --2- 172.21.15.152:0/3207875699 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f410c0a2cf0 0x7f410c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:40.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.434+0000 7f4117fff640 1 -- 172.21.15.152:0/3207875699 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8492+0+0 (secure 0 0 0) 0x7f41180cfb20 con 0x7f4120106f20 2024-09-11T23:03:40.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.434+0000 7f411effd640 1 --2- 172.21.15.152:0/3207875699 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f410c0a2cf0 0x7f410c0a51b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4110009230 tx=0x7f41100058d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:40.459 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.438+0000 7f4117fff640 1 -- 172.21.15.152:0/3207875699 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f411809c760 con 0x7f4120106f20 2024-09-11T23:03:40.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-mon[35211]: pgmap v334: 97 pgs: 97 active+clean; 4.0 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-11T23:03:40.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-mon[35454]: pgmap v334: 97 pgs: 97 active+clean; 4.0 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-11T23:03:40.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.639+0000 7f4125374640 1 -- 172.21.15.152:0/3207875699 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4120109b70 con 0x7f4120106f20 2024-09-11T23:03:40.642 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.642+0000 7f4117fff640 1 -- 172.21.15.152:0/3207875699 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f411809c060 con 0x7f4120106f20 2024-09-11T23:03:40.642 INFO:teuthology.orchestra.run.smithi152.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T23:03:40.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.644+0000 7f4125374640 1 -- 172.21.15.152:0/3207875699 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f410c0a2cf0 msgr2=0x7f410c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:40.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.644+0000 7f4125374640 1 --2- 172.21.15.152:0/3207875699 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f410c0a2cf0 0x7f410c0a51b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4110009230 tx=0x7f41100058d0 comp rx=0 tx=0).stop 2024-09-11T23:03:40.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.644+0000 7f4125374640 1 -- 172.21.15.152:0/3207875699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4120106f20 msgr2=0x7f412019cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:40.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.644+0000 7f4125374640 1 --2- 172.21.15.152:0/3207875699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4120106f20 0x7f412019cd30 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f411800eb10 tx=0x7f411800c2d0 comp rx=0 tx=0).stop 2024-09-11T23:03:40.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.645+0000 7f4125374640 1 -- 172.21.15.152:0/3207875699 shutdown_connections 2024-09-11T23:03:40.645 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.645+0000 7f4125374640 1 --2- 172.21.15.152:0/3207875699 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f410c0a2cf0 0x7f410c0a51b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:40.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.645+0000 7f4125374640 1 --2- 172.21.15.152:0/3207875699 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4120106f20 0x7f412019cd30 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:40.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.645+0000 7f4125374640 1 --2- 172.21.15.152:0/3207875699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41201027a0 0x7f41201989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:40.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.645+0000 7f4125374640 1 --2- 172.21.15.152:0/3207875699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4120101df0 0x7f4120198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:40.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.645+0000 7f4125374640 1 -- 172.21.15.152:0/3207875699 >> 172.21.15.152:0/3207875699 conn(0x7f41200fd8b0 msgr2=0x7f41200feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:40.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.645+0000 7f4125374640 1 -- 172.21.15.152:0/3207875699 shutdown_connections 2024-09-11T23:03:40.646 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:40.645+0000 7f4125374640 1 -- 172.21.15.152:0/3207875699 wait complete. 2024-09-11T23:03:40.959 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-f[65501]: ignoring --setuser ceph since I am not root 2024-09-11T23:03:40.959 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-f[65501]: ignoring --setgroup ceph since I am not root 2024-09-11T23:03:40.959 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-mds[65505]: -- 172.21.15.204:0/886980707 <== mon.0 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5606c9f61c20 con 0x5606c9f3f000 2024-09-11T23:03:40.959 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-f[65501]: starting mds.f at 2024-09-11T23:03:40.959 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-i[66104]: ignoring --setuser ceph since I am not root 2024-09-11T23:03:40.959 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-i[66104]: ignoring --setgroup ceph since I am not root 2024-09-11T23:03:40.959 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-mds[66108]: -- 172.21.15.204:0/2490766194 <== mon.0 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5650ddacb860 con 0x5650ddaa9400 2024-09-11T23:03:40.960 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-i[66104]: starting mds.i at 2024-09-11T23:03:40.960 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-c[65804]: ignoring --setuser ceph since I am not root 2024-09-11T23:03:40.960 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-c[65804]: ignoring --setgroup ceph since I am not root 2024-09-11T23:03:40.960 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-mds[65808]: -- 172.21.15.204:0/1012057845 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a9eacb860 con 0x557a9eaa9000 2024-09-11T23:03:40.960 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-c[65804]: starting mds.c at 2024-09-11T23:03:40.960 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-l[65197]: ignoring --setuser ceph since I am not root 2024-09-11T23:03:40.960 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-l[65197]: ignoring --setgroup ceph since I am not root 2024-09-11T23:03:40.961 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-mds[65201]: -- 172.21.15.204:0/3787190279 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5591514a1860 con 0x55915147e800 2024-09-11T23:03:40.961 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 11 23:03:40 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-l[65197]: starting mds.l at 2024-09-11T23:03:41.027 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:40 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-a[31177]: 2024-09-11T23:03:40.640+0000 7fe3aeb24640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T23:03:41.028 INFO:journalctl@ceph.mds.a.smithi152.stdout:Sep 11 23:03:40 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-a[70022]: ignoring --setuser ceph since I am not root 2024-09-11T23:03:41.028 INFO:journalctl@ceph.mds.a.smithi152.stdout:Sep 11 23:03:40 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-a[70022]: ignoring --setgroup ceph since I am not root 2024-09-11T23:03:41.028 INFO:journalctl@ceph.mds.a.smithi152.stdout:Sep 11 23:03:40 smithi152 ceph-mds[70027]: -- 172.21.15.152:0/3367417936 <== mon.0 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a52f415860 con 0x55a52f3f3000 2024-09-11T23:03:41.028 INFO:journalctl@ceph.mds.a.smithi152.stdout:Sep 11 23:03:40 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-a[70022]: starting mds.a at 2024-09-11T23:03:41.028 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 23:03:40 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-g[69612]: ignoring --setuser ceph since I am not root 2024-09-11T23:03:41.029 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 23:03:40 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-g[69612]: ignoring --setgroup ceph since I am not root 2024-09-11T23:03:41.029 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 23:03:40 smithi152 ceph-mds[69616]: -- 172.21.15.152:0/236118009 <== mon.0 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5590c5a7dc20 con 0x5590c5a5b400 2024-09-11T23:03:41.029 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 23:03:40 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-g[69612]: starting mds.g at 2024-09-11T23:03:41.037 INFO:journalctl@ceph.mds.b.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-b[65317]: ignoring --setuser ceph since I am not root 2024-09-11T23:03:41.037 INFO:journalctl@ceph.mds.b.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-b[65317]: ignoring --setgroup ceph since I am not root 2024-09-11T23:03:41.037 INFO:journalctl@ceph.mds.b.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-mds[65321]: -- 172.21.15.191:0/4138882828 <== mon.2 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fcf7759860 con 0x55fcf7737400 2024-09-11T23:03:41.038 INFO:journalctl@ceph.mds.b.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-b[65317]: starting mds.b at 2024-09-11T23:03:41.038 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-e[64705]: ignoring --setuser ceph since I am not root 2024-09-11T23:03:41.038 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-e[64705]: ignoring --setgroup ceph since I am not root 2024-09-11T23:03:41.038 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-mds[64709]: -- 172.21.15.191:0/2224224820 <== mon.2 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5583c7e43860 con 0x5583c7e21400 2024-09-11T23:03:41.038 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-e[64705]: starting mds.e at 2024-09-11T23:03:41.039 INFO:journalctl@ceph.mds.h.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-h[65006]: ignoring --setuser ceph since I am not root 2024-09-11T23:03:41.039 INFO:journalctl@ceph.mds.h.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-h[65006]: ignoring --setgroup ceph since I am not root 2024-09-11T23:03:41.039 INFO:journalctl@ceph.mds.h.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-mds[65010]: -- 172.21.15.191:0/940054283 <== mon.0 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5626d12b7860 con 0x5626d1294800 2024-09-11T23:03:41.039 INFO:journalctl@ceph.mds.h.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-h[65006]: starting mds.h at 2024-09-11T23:03:41.039 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-k[64393]: ignoring --setuser ceph since I am not root 2024-09-11T23:03:41.039 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-k[64393]: ignoring --setgroup ceph since I am not root 2024-09-11T23:03:41.039 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-mds[64398]: -- 172.21.15.191:0/1296834845 <== mon.2 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cb55107c20 con 0x55cb550e5400 2024-09-11T23:03:41.039 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 23:03:40 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-k[64393]: starting mds.k at 2024-09-11T23:03:41.493 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T23:03:41.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:41 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3207875699' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T23:03:41.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:41 smithi191 ceph-mon[35211]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T23:03:41.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:41 smithi191 ceph-mon[35211]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T23:03:41.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:41 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/3207875699' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T23:03:41.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:41 smithi191 ceph-mon[35211]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T23:03:41.537 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:41 smithi191 ceph-mon[35211]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T23:03:41.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:41 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3207875699' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T23:03:41.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:41 smithi204 ceph-mon[35454]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T23:03:41.545 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:41 smithi204 ceph-mon[35454]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T23:03:41.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:41 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/3207875699' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T23:03:41.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:41 smithi204 ceph-mon[35454]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T23:03:41.546 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:41 smithi204 ceph-mon[35454]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T23:03:41.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:41 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3207875699' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T23:03:41.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:41 smithi152 ceph-mon[31181]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T23:03:41.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:41 smithi152 ceph-mon[31181]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T23:03:41.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:41 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/3207875699' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T23:03:41.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:41 smithi152 ceph-mon[31181]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T23:03:41.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:41 smithi152 ceph-mon[31181]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T23:03:41.747 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:42.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.301+0000 7f4fdfa90640 1 -- 172.21.15.152:0/1163045325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fd8100e20 msgr2=0x7f4fd8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:42.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.301+0000 7f4fdfa90640 1 --2- 172.21.15.152:0/1163045325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fd8100e20 0x7f4fd8101200 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009900 tx=0x7f4fd402a480 comp rx=0 tx=0).stop 2024-09-11T23:03:42.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.301+0000 7f4fdfa90640 1 -- 172.21.15.152:0/1163045325 shutdown_connections 2024-09-11T23:03:42.319 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.301+0000 7f4fdfa90640 1 --2- 172.21.15.152:0/1163045325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fd8102ef0 0x7f4fd810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:42.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.301+0000 7f4fdfa90640 1 --2- 172.21.15.152:0/1163045325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fd8102570 0x7f4fd81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:42.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.301+0000 7f4fdfa90640 1 --2- 172.21.15.152:0/1163045325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fd8100e20 0x7f4fd8101200 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:42.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.301+0000 7f4fdfa90640 1 -- 172.21.15.152:0/1163045325 >> 172.21.15.152:0/1163045325 conn(0x7f4fd80fc7d0 msgr2=0x7f4fd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:42.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.302+0000 7f4fdfa90640 1 -- 172.21.15.152:0/1163045325 shutdown_connections 2024-09-11T23:03:42.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.302+0000 7f4fdfa90640 1 -- 172.21.15.152:0/1163045325 wait complete. 2024-09-11T23:03:42.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.302+0000 7f4fdfa90640 1 Processor -- start 2024-09-11T23:03:42.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.302+0000 7f4fdfa90640 1 -- start start 2024-09-11T23:03:42.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.302+0000 7f4fdfa90640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fd8100e20 0x7f4fd8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:42.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.302+0000 7f4fdfa90640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fd8102570 0x7f4fd8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:42.320 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.302+0000 7f4fdfa90640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fd8102ef0 0x7f4fd819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.302+0000 7f4fdfa90640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8110f80 con 0x7f4fd8100e20 2024-09-11T23:03:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.302+0000 7f4fdfa90640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8110e00 con 0x7f4fd8102ef0 2024-09-11T23:03:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.302+0000 7f4fdfa90640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd8111100 con 0x7f4fd8102570 2024-09-11T23:03:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fdd82c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fd8100e20 0x7f4fd8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fdd82c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fd8100e20 0x7f4fd8198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46062/0 (socket says 172.21.15.152:46062) 2024-09-11T23:03:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fdd82c640 1 -- 172.21.15.152:0/2555446058 learned_addr learned my addr 172.21.15.152:0/2555446058 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fde02d640 1 --2- 172.21.15.152:0/2555446058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fd8102ef0 0x7f4fd819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:42.321 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fdd02b640 1 --2- 172.21.15.152:0/2555446058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fd8102570 0x7f4fd8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fdd82c640 1 -- 172.21.15.152:0/2555446058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fd8102570 msgr2=0x7f4fd8198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fdd82c640 1 --2- 172.21.15.152:0/2555446058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fd8102570 0x7f4fd8198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fdd82c640 1 -- 172.21.15.152:0/2555446058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fd8102ef0 msgr2=0x7f4fd819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fdd82c640 1 --2- 172.21.15.152:0/2555446058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fd8102ef0 0x7f4fd819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fdd82c640 1 -- 172.21.15.152:0/2555446058 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd819d4b0 con 0x7f4fd8100e20 2024-09-11T23:03:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fde02d640 1 --2- 172.21.15.152:0/2555446058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fd8102ef0 0x7f4fd819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T23:03:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fdd82c640 1 --2- 172.21.15.152:0/2555446058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fd8100e20 0x7f4fd8198500 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4006010 tx=0x7f4fd4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fceffd640 1 -- 172.21.15.152:0/2555446058 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd403f070 con 0x7f4fd8100e20 2024-09-11T23:03:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fdfa90640 1 -- 172.21.15.152:0/2555446058 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd819d740 con 0x7f4fd8100e20 2024-09-11T23:03:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fceffd640 1 -- 172.21.15.152:0/2555446058 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fd4031c30 con 0x7f4fd8100e20 2024-09-11T23:03:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fceffd640 1 -- 172.21.15.152:0/2555446058 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4035030 con 0x7f4fd8100e20 2024-09-11T23:03:42.322 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.303+0000 7f4fdfa90640 1 -- 172.21.15.152:0/2555446058 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd81a5080 con 0x7f4fd8100e20 2024-09-11T23:03:42.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.304+0000 7f4fdfa90640 1 -- 172.21.15.152:0/2555446058 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd8101200 con 0x7f4fd8100e20 2024-09-11T23:03:42.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.305+0000 7f4fceffd640 1 -- 172.21.15.152:0/2555446058 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4fd4042070 con 0x7f4fd8100e20 2024-09-11T23:03:42.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.306+0000 7f4fceffd640 1 --2- 172.21.15.152:0/2555446058 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4fc40a2d70 0x7f4fc40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:42.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.306+0000 7f4fdd02b640 1 --2- 172.21.15.152:0/2555446058 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4fc40a2d70 0x7f4fc40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:42.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.306+0000 7f4fceffd640 1 -- 172.21.15.152:0/2555446058 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8878+0+0 (secure 0 0 0) 0x7f4fd40f01d0 con 0x7f4fd8100e20 2024-09-11T23:03:42.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.306+0000 7f4fdd02b640 1 --2- 172.21.15.152:0/2555446058 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4fc40a2d70 0x7f4fc40a5230 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80fff40 tx=0x7f4fc8005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:42.323 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.309+0000 7f4fceffd640 1 -- 172.21.15.152:0/2555446058 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4fd4031de0 con 0x7f4fd8100e20 2024-09-11T23:03:42.511 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.511+0000 7f4fdfa90640 1 -- 172.21.15.152:0/2555446058 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4fd81a52b0 con 0x7f4fd8100e20 2024-09-11T23:03:42.650 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.650+0000 7f4fceffd640 1 -- 172.21.15.152:0/2555446058 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f4fd40b8070 con 0x7f4fd8100e20 2024-09-11T23:03:42.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.652+0000 7f4fdfa90640 1 -- 172.21.15.152:0/2555446058 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4fc40a2d70 msgr2=0x7f4fc40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:42.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.652+0000 7f4fdfa90640 1 --2- 172.21.15.152:0/2555446058 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4fc40a2d70 0x7f4fc40a5230 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80fff40 tx=0x7f4fc8005f90 comp rx=0 tx=0).stop 2024-09-11T23:03:42.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.652+0000 7f4fdfa90640 1 -- 172.21.15.152:0/2555446058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fd8100e20 msgr2=0x7f4fd8198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:42.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.652+0000 7f4fdfa90640 1 --2- 172.21.15.152:0/2555446058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fd8100e20 0x7f4fd8198500 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4006010 tx=0x7f4fd4003c10 comp rx=0 tx=0).stop 2024-09-11T23:03:42.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.652+0000 7f4fdfa90640 1 -- 172.21.15.152:0/2555446058 shutdown_connections 2024-09-11T23:03:42.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.652+0000 7f4fdfa90640 1 --2- 172.21.15.152:0/2555446058 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f4fc40a2d70 0x7f4fc40a5230 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:42.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.652+0000 7f4fdfa90640 1 --2- 172.21.15.152:0/2555446058 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fd8102ef0 0x7f4fd819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:42.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.652+0000 7f4fdfa90640 1 --2- 172.21.15.152:0/2555446058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fd8102570 0x7f4fd8198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:42.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.652+0000 7f4fdfa90640 1 --2- 172.21.15.152:0/2555446058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4fd8100e20 0x7f4fd8198500 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:42.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.652+0000 7f4fdfa90640 1 -- 172.21.15.152:0/2555446058 >> 172.21.15.152:0/2555446058 conn(0x7f4fd80fc7d0 msgr2=0x7f4fd806a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:42.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.652+0000 7f4fdfa90640 1 -- 172.21.15.152:0/2555446058 shutdown_connections 2024-09-11T23:03:42.653 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:42.652+0000 7f4fdfa90640 1 -- 172.21.15.152:0/2555446058 wait complete. 2024-09-11T23:03:42.662 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: pgmap v336: 97 pgs: 97 active+clean; 4.0 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T23:03:42.662 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.152:6838/2640970698,v1:172.21.15.152:6839/2640970698] up:boot 2024-09-11T23:03:42.662 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.152:6840/65539782,v1:172.21.15.152:6841/65539782] up:boot 2024-09-11T23:03:42.663 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.191:6832/3626552574,v1:172.21.15.191:6833/3626552574] up:boot 2024-09-11T23:03:42.663 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.191:6834/1832671169,v1:172.21.15.191:6835/1832671169] up:boot 2024-09-11T23:03:42.663 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.204:6832/2567932618,v1:172.21.15.204:6833/2567932618] up:boot 2024-09-11T23:03:42.663 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.204:6834/3307179269,v1:172.21.15.204:6835/3307179269] up:boot 2024-09-11T23:03:42.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: pgmap v336: 97 pgs: 97 active+clean; 4.0 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T23:03:42.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.152:6838/2640970698,v1:172.21.15.152:6839/2640970698] up:boot 2024-09-11T23:03:42.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.152:6840/65539782,v1:172.21.15.152:6841/65539782] up:boot 2024-09-11T23:03:42.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.191:6832/3626552574,v1:172.21.15.191:6833/3626552574] up:boot 2024-09-11T23:03:42.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.191:6834/1832671169,v1:172.21.15.191:6835/1832671169] up:boot 2024-09-11T23:03:42.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.204:6832/2567932618,v1:172.21.15.204:6833/2567932618] up:boot 2024-09-11T23:03:42.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.204:6834/3307179269,v1:172.21.15.204:6835/3307179269] up:boot 2024-09-11T23:03:42.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.191:6836/1038252121,v1:172.21.15.191:6838/1038252121] up:boot 2024-09-11T23:03:42.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.191:6837/668933565,v1:172.21.15.191:6839/668933565] up:boot 2024-09-11T23:03:42.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.204:6836/4150764206,v1:172.21.15.204:6837/4150764206] up:boot 2024-09-11T23:03:42.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: mds.? [v2:172.21.15.204:6838/2824024438,v1:172.21.15.204:6839/2824024438] up:boot 2024-09-11T23:03:42.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T23:03:42.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:03:42.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:03:42.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:03:42.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:03:42.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:03:42.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:03:42.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:03:42.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:03:42.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:03:42.874 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:03:42.875 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:42 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2555446058' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:03:43.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.191:6836/1038252121,v1:172.21.15.191:6838/1038252121] up:boot 2024-09-11T23:03:43.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.191:6837/668933565,v1:172.21.15.191:6839/668933565] up:boot 2024-09-11T23:03:43.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.204:6836/4150764206,v1:172.21.15.204:6837/4150764206] up:boot 2024-09-11T23:03:43.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: mds.? [v2:172.21.15.204:6838/2824024438,v1:172.21.15.204:6839/2824024438] up:boot 2024-09-11T23:03:43.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T23:03:43.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:03:43.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:03:43.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:03:43.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:03:43.038 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:03:43.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:03:43.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:03:43.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:03:43.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:03:43.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:03:43.039 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:42 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2555446058' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:03:43.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: pgmap v336: 97 pgs: 97 active+clean; 4.0 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T23:03:43.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.152:6838/2640970698,v1:172.21.15.152:6839/2640970698] up:boot 2024-09-11T23:03:43.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.152:6840/65539782,v1:172.21.15.152:6841/65539782] up:boot 2024-09-11T23:03:43.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.191:6832/3626552574,v1:172.21.15.191:6833/3626552574] up:boot 2024-09-11T23:03:43.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.191:6834/1832671169,v1:172.21.15.191:6835/1832671169] up:boot 2024-09-11T23:03:43.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.204:6832/2567932618,v1:172.21.15.204:6833/2567932618] up:boot 2024-09-11T23:03:43.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.204:6834/3307179269,v1:172.21.15.204:6835/3307179269] up:boot 2024-09-11T23:03:43.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.191:6836/1038252121,v1:172.21.15.191:6838/1038252121] up:boot 2024-09-11T23:03:43.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.191:6837/668933565,v1:172.21.15.191:6839/668933565] up:boot 2024-09-11T23:03:43.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.204:6836/4150764206,v1:172.21.15.204:6837/4150764206] up:boot 2024-09-11T23:03:43.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: mds.? [v2:172.21.15.204:6838/2824024438,v1:172.21.15.204:6839/2824024438] up:boot 2024-09-11T23:03:43.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T23:03:43.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T23:03:43.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T23:03:43.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T23:03:43.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T23:03:43.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T23:03:43.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T23:03:43.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T23:03:43.047 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T23:03:43.048 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T23:03:43.048 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: from='mgr.15240 172.21.15.152:0/4081199278' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T23:03:43.048 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:42 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2555446058' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T23:03:43.579 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T23:03:43.832 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:43.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:43 smithi152 ceph-mon[31181]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T23:03:43.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:43 smithi152 ceph-mon[31181]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T23:03:43.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:43 smithi152 ceph-mon[31181]: Cluster is now healthy 2024-09-11T23:03:43.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:43 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/2555446058' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:03:43.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:43 smithi152 ceph-mon[31181]: fsmap 12 up:standby 2024-09-11T23:03:44.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:43 smithi191 ceph-mon[35211]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T23:03:44.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:43 smithi191 ceph-mon[35211]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T23:03:44.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:43 smithi191 ceph-mon[35211]: Cluster is now healthy 2024-09-11T23:03:44.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:43 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/2555446058' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:03:44.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:43 smithi191 ceph-mon[35211]: fsmap 12 up:standby 2024-09-11T23:03:44.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:43 smithi204 ceph-mon[35454]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T23:03:44.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:43 smithi204 ceph-mon[35454]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T23:03:44.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:43 smithi204 ceph-mon[35454]: Cluster is now healthy 2024-09-11T23:03:44.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:43 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/2555446058' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T23:03:44.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:43 smithi204 ceph-mon[35454]: fsmap 12 up:standby 2024-09-11T23:03:44.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.343+0000 7f58b05cd640 1 -- 172.21.15.152:0/3193810282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a8103760 msgr2=0x7f58a8107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:44.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.343+0000 7f58b05cd640 1 --2- 172.21.15.152:0/3193810282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a8103760 0x7f58a8107ba0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f589c009930 tx=0x7f589c02a420 comp rx=0 tx=0).stop 2024-09-11T23:03:44.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.344+0000 7f58b05cd640 1 -- 172.21.15.152:0/3193810282 shutdown_connections 2024-09-11T23:03:44.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.344+0000 7f58b05cd640 1 --2- 172.21.15.152:0/3193810282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a81082e0 0x7f58a810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:44.361 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.344+0000 7f58b05cd640 1 --2- 172.21.15.152:0/3193810282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a8103760 0x7f58a8107ba0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:44.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.344+0000 7f58b05cd640 1 --2- 172.21.15.152:0/3193810282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58a8102db0 0x7f58a8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:44.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.344+0000 7f58b05cd640 1 -- 172.21.15.152:0/3193810282 >> 172.21.15.152:0/3193810282 conn(0x7f58a80fc7d0 msgr2=0x7f58a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:44.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.344+0000 7f58b05cd640 1 -- 172.21.15.152:0/3193810282 shutdown_connections 2024-09-11T23:03:44.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.344+0000 7f58b05cd640 1 -- 172.21.15.152:0/3193810282 wait complete. 2024-09-11T23:03:44.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.344+0000 7f58b05cd640 1 Processor -- start 2024-09-11T23:03:44.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.344+0000 7f58b05cd640 1 -- start start 2024-09-11T23:03:44.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58b05cd640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58a8102db0 0x7f58a81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:44.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58b05cd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a8103760 0x7f58a8198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:44.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58b05cd640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a81082e0 0x7f58a819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:44.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58b05cd640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8112070 con 0x7f58a8103760 2024-09-11T23:03:44.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58b05cd640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7f58a8111ef0 con 0x7f58a8102db0 2024-09-11T23:03:44.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58b05cd640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7f58a81121f0 con 0x7f58a81082e0 2024-09-11T23:03:44.362 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58adb68640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a8103760 0x7f58a8198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:44.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58adb68640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a8103760 0x7f58a8198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46080/0 (socket says 172.21.15.152:46080) 2024-09-11T23:03:44.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58adb68640 1 -- 172.21.15.152:0/4066483549 learned_addr learned my addr 172.21.15.152:0/4066483549 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:44.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58ae369640 1 --2- 172.21.15.152:0/4066483549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58a8102db0 0x7f58a81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:44.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58aeb6a640 1 --2- 172.21.15.152:0/4066483549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a81082e0 0x7f58a819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:44.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58adb68640 1 -- 172.21.15.152:0/4066483549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a81082e0 msgr2=0x7f58a819cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:44.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58adb68640 1 --2- 172.21.15.152:0/4066483549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a81082e0 0x7f58a819cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:44.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58adb68640 1 -- 172.21.15.152:0/4066483549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58a8102db0 msgr2=0x7f58a81985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:44.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58adb68640 1 --2- 172.21.15.152:0/4066483549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58a8102db0 0x7f58a81985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:44.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58adb68640 1 -- 172.21.15.152:0/4066483549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a819d5a0 con 0x7f58a8103760 2024-09-11T23:03:44.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58aeb6a640 1 --2- 172.21.15.152:0/4066483549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a81082e0 0x7f58a819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T23:03:44.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58adb68640 1 --2- 172.21.15.152:0/4066483549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a8103760 0x7f58a8198b30 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f589c009230 tx=0x7f589c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:44.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58a77fe640 1 -- 172.21.15.152:0/4066483549 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c03f070 con 0x7f58a8103760 2024-09-11T23:03:44.363 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58a77fe640 1 -- 172.21.15.152:0/4066483549 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f589c031e30 con 0x7f58a8103760 2024-09-11T23:03:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58b05cd640 1 -- 172.21.15.152:0/4066483549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a819d830 con 0x7f58a8103760 2024-09-11T23:03:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58a77fe640 1 -- 172.21.15.152:0/4066483549 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c035030 con 0x7f58a8103760 2024-09-11T23:03:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.345+0000 7f58b05cd640 1 -- 172.21.15.152:0/4066483549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a81a5170 con 0x7f58a8103760 2024-09-11T23:03:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.346+0000 7f58b05cd640 1 -- 172.21.15.152:0/4066483549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a8103190 con 0x7f58a8103760 2024-09-11T23:03:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.348+0000 7f58a77fe640 1 -- 172.21.15.152:0/4066483549 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f589c042070 con 0x7f58a8103760 2024-09-11T23:03:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.348+0000 7f58a77fe640 1 --2- 172.21.15.152:0/4066483549 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58900a2d70 0x7f58900a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.348+0000 7f58ae369640 1 --2- 172.21.15.152:0/4066483549 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58900a2d70 0x7f58900a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.348+0000 7f58a77fe640 1 -- 172.21.15.152:0/4066483549 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8878+0+0 (secure 0 0 0) 0x7f589c0f0350 con 0x7f58a8103760 2024-09-11T23:03:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.349+0000 7f58ae369640 1 --2- 172.21.15.152:0/4066483549 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58900a2d70 0x7f58900a5230 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5898005c40 tx=0x7f589800a590 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:44.364 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.351+0000 7f58a77fe640 1 -- 172.21.15.152:0/4066483549 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f589c0bce10 con 0x7f58a8103760 2024-09-11T23:03:44.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.517+0000 7f58b05cd640 1 -- 172.21.15.152:0/4066483549 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f58a810a2c0 con 0x7f58a8103760 2024-09-11T23:03:45.071 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:44 smithi204 ceph-mon[35454]: pgmap v337: 97 pgs: 97 active+clean; 4.0 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T23:03:45.071 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:44 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/4066483549' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:03:45.071 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:44 smithi191 ceph-mon[35211]: pgmap v337: 97 pgs: 97 active+clean; 4.0 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T23:03:45.071 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:44 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/4066483549' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.674+0000 7f58a77fe640 1 -- 172.21.15.152:0/4066483549 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) ==== 181+0+0 (secure 0 0 0) 0x7f589c0bc710 con 0x7f58a8103760 2024-09-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_metadata' removed 2024-09-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.676+0000 7f58b05cd640 1 -- 172.21.15.152:0/4066483549 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58900a2d70 msgr2=0x7f58900a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.676+0000 7f58b05cd640 1 --2- 172.21.15.152:0/4066483549 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58900a2d70 0x7f58900a5230 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5898005c40 tx=0x7f589800a590 comp rx=0 tx=0).stop 2024-09-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.676+0000 7f58b05cd640 1 -- 172.21.15.152:0/4066483549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a8103760 msgr2=0x7f58a8198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.676+0000 7f58b05cd640 1 --2- 172.21.15.152:0/4066483549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a8103760 0x7f58a8198b30 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f589c009230 tx=0x7f589c003c10 comp rx=0 tx=0).stop 2024-09-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.677+0000 7f58b05cd640 1 -- 172.21.15.152:0/4066483549 shutdown_connections 2024-09-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.677+0000 7f58b05cd640 1 --2- 172.21.15.152:0/4066483549 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7f58900a2d70 0x7f58900a5230 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.677+0000 7f58b05cd640 1 --2- 172.21.15.152:0/4066483549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58a81082e0 0x7f58a819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.677+0000 7f58b05cd640 1 --2- 172.21.15.152:0/4066483549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f58a8103760 0x7f58a8198b30 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.677+0000 7f58b05cd640 1 --2- 172.21.15.152:0/4066483549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f58a8102db0 0x7f58a81985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.677+0000 7f58b05cd640 1 -- 172.21.15.152:0/4066483549 >> 172.21.15.152:0/4066483549 conn(0x7f58a80fc7d0 msgr2=0x7f58a8105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.677+0000 7f58b05cd640 1 -- 172.21.15.152:0/4066483549 shutdown_connections 2024-09-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:44.677+0000 7f58b05cd640 1 -- 172.21.15.152:0/4066483549 wait complete. 2024-09-11T23:03:45.073 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:44 smithi152 ceph-mon[31181]: pgmap v337: 97 pgs: 97 active+clean; 4.0 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T23:03:45.073 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:44 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4066483549' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:03:45.623 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T23:03:45.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:45 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/4066483549' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:03:45.873 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:45 smithi152 ceph-mon[31181]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T23:03:45.875 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T23:03:46.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:45 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/4066483549' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:03:46.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:45 smithi191 ceph-mon[35211]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T23:03:46.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:45 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/4066483549' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:03:46.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:45 smithi204 ceph-mon[35454]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T23:03:46.387 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.370+0000 7fb981892640 1 -- 172.21.15.152:0/1900133806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb97c105580 msgr2=0x7fb97c10d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:46.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.370+0000 7fb981892640 1 --2- 172.21.15.152:0/1900133806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb97c105580 0x7fb97c10d0e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb974009ca0 tx=0x7fb97402a8c0 comp rx=0 tx=0).stop 2024-09-11T23:03:46.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.370+0000 7fb981892640 1 -- 172.21.15.152:0/1900133806 shutdown_connections 2024-09-11T23:03:46.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.370+0000 7fb981892640 1 --2- 172.21.15.152:0/1900133806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb97c105580 0x7fb97c10d0e0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:46.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.370+0000 7fb981892640 1 --2- 172.21.15.152:0/1900133806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb97c101080 0x7fb97c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:46.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.370+0000 7fb981892640 1 --2- 172.21.15.152:0/1900133806 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb97c1006d0 0x7fb97c100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:46.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.370+0000 7fb981892640 1 -- 172.21.15.152:0/1900133806 >> 172.21.15.152:0/1900133806 conn(0x7fb97c0fba90 msgr2=0x7fb97c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:46.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.370+0000 7fb981892640 1 -- 172.21.15.152:0/1900133806 shutdown_connections 2024-09-11T23:03:46.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.370+0000 7fb981892640 1 -- 172.21.15.152:0/1900133806 wait complete. 2024-09-11T23:03:46.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb981892640 1 Processor -- start 2024-09-11T23:03:46.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb981892640 1 -- start start 2024-09-11T23:03:46.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb981892640 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb97c1006d0 0x7fb97c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:46.388 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb981892640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb97c101080 0x7fb97c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:46.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb981892640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb97c105580 0x7fb97c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:46.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb981892640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c110570 con 0x7fb97c105580 2024-09-11T23:03:46.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb981892640 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1103f0 con 0x7fb97c1006d0 2024-09-11T23:03:46.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb981892640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1106f0 con 0x7fb97c101080 2024-09-11T23:03:46.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb97b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb97c105580 0x7fb97c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:46.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb97b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb97c105580 0x7fb97c19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:46102/0 (socket says 172.21.15.152:46102) 2024-09-11T23:03:46.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb97b7fe640 1 -- 172.21.15.152:0/826021469 learned_addr learned my addr 172.21.15.152:0/826021469 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-09-11T23:03:46.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb97affd640 1 --2- 172.21.15.152:0/826021469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb97c1006d0 0x7fb97c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:46.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb973fff640 1 --2- 172.21.15.152:0/826021469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb97c101080 0x7fb97c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:46.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb97b7fe640 1 -- 172.21.15.152:0/826021469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb97c101080 msgr2=0x7fb97c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:46.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb97b7fe640 1 --2- 172.21.15.152:0/826021469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb97c101080 0x7fb97c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:46.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb97b7fe640 1 -- 172.21.15.152:0/826021469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb97c1006d0 msgr2=0x7fb97c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:46.389 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb97b7fe640 1 --2- 172.21.15.152:0/826021469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb97c1006d0 0x7fb97c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:46.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb97b7fe640 1 -- 172.21.15.152:0/826021469 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c19d320 con 0x7fb97c105580 2024-09-11T23:03:46.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.371+0000 7fb97b7fe640 1 --2- 172.21.15.152:0/826021469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb97c105580 0x7fb97c19cc40 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb974031a60 tx=0x7fb974031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:46.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.372+0000 7fb980890640 1 -- 172.21.15.152:0/826021469 <== mon.0 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974003c30 con 0x7fb97c105580 2024-09-11T23:03:46.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.372+0000 7fb980890640 1 -- 172.21.15.152:0/826021469 <== mon.0 v2:172.21.15.152:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb974003dd0 con 0x7fb97c105580 2024-09-11T23:03:46.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.372+0000 7fb981892640 1 -- 172.21.15.152:0/826021469 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97c19d5b0 con 0x7fb97c105580 2024-09-11T23:03:46.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.372+0000 7fb980890640 1 -- 172.21.15.152:0/826021469 <== mon.0 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9740047e0 con 0x7fb97c105580 2024-09-11T23:03:46.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.372+0000 7fb981892640 1 -- 172.21.15.152:0/826021469 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97c1a4ef0 con 0x7fb97c105580 2024-09-11T23:03:46.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.373+0000 7fb981892640 1 -- 172.21.15.152:0/826021469 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb97c100ab0 con 0x7fb97c105580 2024-09-11T23:03:46.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.374+0000 7fb980890640 1 -- 172.21.15.152:0/826021469 <== mon.0 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb97402f790 con 0x7fb97c105580 2024-09-11T23:03:46.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.375+0000 7fb980890640 1 --2- 172.21.15.152:0/826021469 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9680a2cf0 0x7fb9680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T23:03:46.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.375+0000 7fb97affd640 1 --2- 172.21.15.152:0/826021469 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9680a2cf0 0x7fb9680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T23:03:46.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.375+0000 7fb980890640 1 -- 172.21.15.152:0/826021469 <== mon.0 v2:172.21.15.152:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8382+0+0 (secure 0 0 0) 0x7fb9740f0690 con 0x7fb97c105580 2024-09-11T23:03:46.390 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.375+0000 7fb97affd640 1 --2- 172.21.15.152:0/826021469 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9680a2cf0 0x7fb9680a51b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0058c0 tx=0x7fb96c00a2d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T23:03:46.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.378+0000 7fb980890640 1 -- 172.21.15.152:0/826021469 <== mon.0 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9740bd570 con 0x7fb97c105580 2024-09-11T23:03:46.544 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.543+0000 7fb981892640 1 -- 172.21.15.152:0/826021469 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb97c1a50c0 con 0x7fb97c105580 2024-09-11T23:03:46.689 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.688+0000 7fb980890640 1 -- 172.21.15.152:0/826021469 <== mon.0 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) ==== 169+0+0 (secure 0 0 0) 0x7fb9740bce70 con 0x7fb97c105580 2024-09-11T23:03:46.689 INFO:teuthology.orchestra.run.smithi152.stderr:pool 'cephfs_data' removed 2024-09-11T23:03:46.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.691+0000 7fb981892640 1 -- 172.21.15.152:0/826021469 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9680a2cf0 msgr2=0x7fb9680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:46.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.691+0000 7fb981892640 1 --2- 172.21.15.152:0/826021469 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9680a2cf0 0x7fb9680a51b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0058c0 tx=0x7fb96c00a2d0 comp rx=0 tx=0).stop 2024-09-11T23:03:46.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.691+0000 7fb981892640 1 -- 172.21.15.152:0/826021469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb97c105580 msgr2=0x7fb97c19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T23:03:46.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.691+0000 7fb981892640 1 --2- 172.21.15.152:0/826021469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb97c105580 0x7fb97c19cc40 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb974031a60 tx=0x7fb974031a90 comp rx=0 tx=0).stop 2024-09-11T23:03:46.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.691+0000 7fb981892640 1 -- 172.21.15.152:0/826021469 shutdown_connections 2024-09-11T23:03:46.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.691+0000 7fb981892640 1 --2- 172.21.15.152:0/826021469 >> [v2:172.21.15.152:6800/4130474708,v1:172.21.15.152:6801/4130474708] conn(0x7fb9680a2cf0 0x7fb9680a51b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:46.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.691+0000 7fb981892640 1 --2- 172.21.15.152:0/826021469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb97c105580 0x7fb97c19cc40 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:46.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.691+0000 7fb981892640 1 --2- 172.21.15.152:0/826021469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb97c101080 0x7fb97c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:46.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.691+0000 7fb981892640 1 --2- 172.21.15.152:0/826021469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb97c1006d0 0x7fb97c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T23:03:46.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.691+0000 7fb981892640 1 -- 172.21.15.152:0/826021469 >> 172.21.15.152:0/826021469 conn(0x7fb97c0fba90 msgr2=0x7fb97c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T23:03:46.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.692+0000 7fb981892640 1 -- 172.21.15.152:0/826021469 shutdown_connections 2024-09-11T23:03:46.693 INFO:teuthology.orchestra.run.smithi152.stderr:2024-09-11T23:03:46.692+0000 7fb981892640 1 -- 172.21.15.152:0/826021469 wait complete. 2024-09-11T23:03:46.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:46 smithi152 ceph-mon[31181]: pgmap v339: 65 pgs: 65 active+clean; 577 KiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:03:46.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:46 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/826021469' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:03:47.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:46 smithi191 ceph-mon[35211]: pgmap v339: 65 pgs: 65 active+clean; 577 KiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:03:47.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:46 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/826021469' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:03:47.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:46 smithi204 ceph-mon[35454]: pgmap v339: 65 pgs: 65 active+clean; 577 KiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T23:03:47.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:46 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/826021469' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T23:03:47.590 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:03:47.618 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:03:47.652 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:03:47.687 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T23:03:47.687 DEBUG:teuthology.orchestra.run.smithi152:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T23:03:47.703 DEBUG:teuthology.orchestra.run.smithi191:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T23:03:47.721 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T23:03:47.745 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T23:03:47.745 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T23:03:47.745 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mon.a 2024-09-11T23:03:47.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:47 smithi152 ceph-mon[31181]: from='client.? 172.21.15.152:0/826021469' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:03:47.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:47 smithi152 ceph-mon[31181]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T23:03:47.872 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:47 smithi152 systemd[1]: Stopping Ceph mon.a for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:03:48.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:47 smithi191 ceph-mon[35211]: from='client.? 172.21.15.152:0/826021469' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:03:48.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:47 smithi191 ceph-mon[35211]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T23:03:48.045 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:47 smithi204 ceph-mon[35454]: from='client.? 172.21.15.152:0/826021469' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T23:03:48.046 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:47 smithi204 ceph-mon[35454]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T23:03:48.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:48 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-a[31177]: 2024-09-11T23:03:48.147+0000 7fe3b432f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:03:48.622 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:48 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-a[31177]: 2024-09-11T23:03:48.147+0000 7fe3b432f640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T23:03:48.623 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:48 smithi152 podman[111440]: 2024-09-11 23:03:48.371987783 +0000 UTC m=+0.444622889 container died 316177fd5605dab938c8f1eb25560984f1f6f171aed29416c26038ec7f66c86d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:03:48.951 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:48 smithi152 podman[111440]: 2024-09-11 23:03:48.931300146 +0000 UTC m=+1.003935250 container cleanup 316177fd5605dab938c8f1eb25560984f1f6f171aed29416c26038ec7f66c86d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-09-11T23:03:48.951 INFO:journalctl@ceph.mon.a.smithi152.stdout:Sep 11 23:03:48 smithi152 bash[111440]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-a 2024-09-11T23:03:49.420 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T23:03:49.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:03:49.464 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T23:03:49.464 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-11T23:03:49.464 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mon.b 2024-09-11T23:03:49.787 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:49 smithi191 systemd[1]: Stopping Ceph mon.b for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:03:50.075 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:49 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-b[35207]: 2024-09-11T23:03:49.809+0000 7f6565969640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:03:50.075 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:49 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-b[35207]: 2024-09-11T23:03:49.809+0000 7f6565969640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T23:03:50.382 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:50 smithi191 podman[70542]: 2024-09-11 23:03:50.075320843 +0000 UTC m=+0.419492431 container died ee3efee9ccad0523c9130c7d4dcc25653e273d855fc50e6b80324a84d6695a40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T23:03:50.767 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:50 smithi191 podman[70542]: 2024-09-11 23:03:50.490464007 +0000 UTC m=+0.834635594 container cleanup ee3efee9ccad0523c9130c7d4dcc25653e273d855fc50e6b80324a84d6695a40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T23:03:50.767 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:50 smithi191 bash[70542]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-b 2024-09-11T23:03:51.037 INFO:journalctl@ceph.mon.b.smithi191.stdout:Sep 11 23:03:50 smithi191 podman[70554]: 2024-09-11 23:03:50.767143396 +0000 UTC m=+0.689379886 container remove ee3efee9ccad0523c9130c7d4dcc25653e273d855fc50e6b80324a84d6695a40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:03:51.062 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T23:03:51.118 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:03:51.118 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-11T23:03:51.118 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-11T23:03:51.118 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mon.c 2024-09-11T23:03:51.453 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:51 smithi204 systemd[1]: Stopping Ceph mon.c for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:03:51.784 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-c[35450]: 2024-09-11T23:03:51.452+0000 7f21f5c12640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:03:51.784 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-c[35450]: 2024-09-11T23:03:51.452+0000 7f21f5c12640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T23:03:51.784 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:51 smithi204 podman[71528]: 2024-09-11 23:03:51.596122358 +0000 UTC m=+0.269095537 container died eebdd83eb9213a8888f246b20ec44fbb73f1184e2120cd3884bc807079224d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:03:52.035 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:52 smithi204 podman[71528]: 2024-09-11 23:03:52.020440258 +0000 UTC m=+0.693413495 container cleanup eebdd83eb9213a8888f246b20ec44fbb73f1184e2120cd3884bc807079224d67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-11T23:03:52.035 INFO:journalctl@ceph.mon.c.smithi204.stdout:Sep 11 23:03:52 smithi204 bash[71528]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mon-c 2024-09-11T23:03:52.445 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T23:03:52.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:03:52.491 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-11T23:03:52.491 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T23:03:52.491 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mgr.x 2024-09-11T23:03:52.812 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 23:03:52 smithi152 systemd[1]: Stopping Ceph mgr.x for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:03:53.122 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 23:03:52 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T23:03:52.811+0000 7f5f70141640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:03:53.122 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 23:03:52 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x[31407]: 2024-09-11T23:03:52.811+0000 7f5f70141640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-11T23:03:53.122 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 23:03:52 smithi152 podman[111561]: 2024-09-11 23:03:52.843177156 +0000 UTC m=+0.153004459 container died 4c789e59e2f40e1ae10d44d44f69ea04a13e05589bafdd6bd523f85cebcfe6e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-11T23:03:53.622 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 23:03:53 smithi152 podman[111561]: 2024-09-11 23:03:53.395855073 +0000 UTC m=+0.705682415 container cleanup 4c789e59e2f40e1ae10d44d44f69ea04a13e05589bafdd6bd523f85cebcfe6e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T23:03:53.622 INFO:journalctl@ceph.mgr.x.smithi152.stdout:Sep 11 23:03:53 smithi152 bash[111561]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-x 2024-09-11T23:03:53.984 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T23:03:54.019 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:03:54.019 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T23:03:54.019 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T23:03:54.020 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mgr.y 2024-09-11T23:03:54.433 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 23:03:54 smithi191 systemd[1]: Stopping Ceph mgr.y for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:03:54.738 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 23:03:54 smithi191 podman[70663]: 2024-09-11 23:03:54.433208823 +0000 UTC m=+0.216186150 container died d4400e87ae3bf26dee913ec779b7c9bd0fb1293617f19c0d3e1fa966aa153907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-11T23:03:55.037 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 23:03:54 smithi191 podman[70663]: 2024-09-11 23:03:54.845794844 +0000 UTC m=+0.628772172 container cleanup d4400e87ae3bf26dee913ec779b7c9bd0fb1293617f19c0d3e1fa966aa153907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-11T23:03:55.037 INFO:journalctl@ceph.mgr.y.smithi191.stdout:Sep 11 23:03:54 smithi191 bash[70663]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-y 2024-09-11T23:03:55.327 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T23:03:55.361 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:03:55.361 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T23:03:55.361 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-11T23:03:55.362 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mgr.z 2024-09-11T23:03:55.704 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 23:03:55 smithi204 systemd[1]: Stopping Ceph mgr.z for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:03:56.026 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 23:03:55 smithi204 podman[71651]: 2024-09-11 23:03:55.704801213 +0000 UTC m=+0.145631900 container died 88afd8e36c63c6f556ee7456dbf9e7c54b82de1c55ea6aeae0f91842a6a6fd85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-11T23:03:56.296 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 23:03:56 smithi204 podman[71651]: 2024-09-11 23:03:56.125951715 +0000 UTC m=+0.566782401 container cleanup 88afd8e36c63c6f556ee7456dbf9e7c54b82de1c55ea6aeae0f91842a6a6fd85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-11T23:03:56.296 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Sep 11 23:03:56 smithi204 bash[71651]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mgr-z 2024-09-11T23:03:56.576 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-11T23:03:56.610 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:03:56.610 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-11T23:03:56.610 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T23:03:56.611 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.0 2024-09-11T23:03:56.917 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 23:03:56 smithi152 systemd[1]: Stopping Ceph osd.0 for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:03:57.372 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 23:03:56 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-0[42707]: 2024-09-11T23:03:56.917+0000 7f76b56be640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:03:57.373 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 23:03:56 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-0[42707]: 2024-09-11T23:03:56.917+0000 7f76b56be640 -1 osd.0 95 *** Got signal Terminated *** 2024-09-11T23:03:57.373 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 23:03:56 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-0[42707]: 2024-09-11T23:03:56.917+0000 7f76b56be640 -1 osd.0 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:04:02.602 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 23:04:02 smithi152 podman[111682]: 2024-09-11 23:04:02.290118784 +0000 UTC m=+5.494422455 container died adc0ccdef486bafe8709056168a2c7f477588dff453faa35b0dd4112dd7f702a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:04:02.872 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 23:04:02 smithi152 podman[111682]: 2024-09-11 23:04:02.80947387 +0000 UTC m=+6.013777529 container cleanup adc0ccdef486bafe8709056168a2c7f477588dff453faa35b0dd4112dd7f702a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T23:04:02.872 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 23:04:02 smithi152 bash[111682]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-0 2024-09-11T23:04:03.372 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 23:04:03 smithi152 podman[111696]: 2024-09-11 23:04:03.027848868 +0000 UTC m=+0.738733879 container remove adc0ccdef486bafe8709056168a2c7f477588dff453faa35b0dd4112dd7f702a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-11T23:04:03.684 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 23:04:03 smithi152 podman[111789]: 2024-09-11 23:04:03.33426367 +0000 UTC m=+0.016825717 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:04:03.684 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 23:04:03 smithi152 podman[111789]: 2024-09-11 23:04:03.436916072 +0000 UTC m=+0.119478121 container create e940096819d830081f969f37026da380652ceb40e5e7347ba4a5c2ce0a38e097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-11T23:04:03.684 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 23:04:03 smithi152 podman[111789]: 2024-09-11 23:04:03.681199025 +0000 UTC m=+0.363761071 container init e940096819d830081f969f37026da380652ceb40e5e7347ba4a5c2ce0a38e097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T23:04:03.685 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 23:04:03 smithi152 podman[111789]: 2024-09-11 23:04:03.684721546 +0000 UTC m=+0.367283602 container start e940096819d830081f969f37026da380652ceb40e5e7347ba4a5c2ce0a38e097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T23:04:04.122 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 23:04:03 smithi152 podman[111789]: 2024-09-11 23:04:03.789946068 +0000 UTC m=+0.472508117 container attach e940096819d830081f969f37026da380652ceb40e5e7347ba4a5c2ce0a38e097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:04:04.123 INFO:journalctl@ceph.osd.0.smithi152.stdout:Sep 11 23:04:03 smithi152 podman[111789]: 2024-09-11 23:04:03.872511723 +0000 UTC m=+0.555073774 container died e940096819d830081f969f37026da380652ceb40e5e7347ba4a5c2ce0a38e097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T23:04:04.673 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T23:04:04.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:04:04.714 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T23:04:04.714 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T23:04:04.714 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.1 2024-09-11T23:04:05.034 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 23:04:04 smithi191 systemd[1]: Stopping Ceph osd.1 for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:04:05.287 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 23:04:05 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-1[39672]: 2024-09-11T23:04:05.033+0000 7fb628e4e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:04:05.287 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 23:04:05 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-1[39672]: 2024-09-11T23:04:05.033+0000 7fb628e4e640 -1 osd.1 95 *** Got signal Terminated *** 2024-09-11T23:04:05.287 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 23:04:05 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-1[39672]: 2024-09-11T23:04:05.033+0000 7fb628e4e640 -1 osd.1 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:04:10.718 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 23:04:10 smithi191 podman[70789]: 2024-09-11 23:04:10.407947815 +0000 UTC m=+5.506463431 container died 1aaee7eebc3d33d566e9e31e14a66e039f5a318e925d79f8ed2dcc5e8beae626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-1, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T23:04:11.037 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 23:04:10 smithi191 podman[70789]: 2024-09-11 23:04:10.817022279 +0000 UTC m=+5.915537897 container cleanup 1aaee7eebc3d33d566e9e31e14a66e039f5a318e925d79f8ed2dcc5e8beae626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-11T23:04:11.037 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 23:04:10 smithi191 bash[70789]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-1 2024-09-11T23:04:11.486 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 23:04:11 smithi191 podman[70803]: 2024-09-11 23:04:11.071041922 +0000 UTC m=+0.664337939 container remove 1aaee7eebc3d33d566e9e31e14a66e039f5a318e925d79f8ed2dcc5e8beae626 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-11T23:04:11.780 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 23:04:11 smithi191 podman[70895]: 2024-09-11 23:04:11.389204713 +0000 UTC m=+0.015987373 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:04:11.780 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 23:04:11 smithi191 podman[70895]: 2024-09-11 23:04:11.511062582 +0000 UTC m=+0.137845281 container create e18507f2a9d16b04a44f39912c65f6a5f3ffd92985b74745f2db80f91c983183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T23:04:12.037 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 23:04:11 smithi191 podman[70895]: 2024-09-11 23:04:11.780471077 +0000 UTC m=+0.407253742 container init e18507f2a9d16b04a44f39912c65f6a5f3ffd92985b74745f2db80f91c983183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T23:04:12.037 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 23:04:11 smithi191 podman[70895]: 2024-09-11 23:04:11.783862779 +0000 UTC m=+0.410645434 container start e18507f2a9d16b04a44f39912c65f6a5f3ffd92985b74745f2db80f91c983183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T23:04:12.037 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 23:04:11 smithi191 podman[70895]: 2024-09-11 23:04:11.88883756 +0000 UTC m=+0.515620228 container attach e18507f2a9d16b04a44f39912c65f6a5f3ffd92985b74745f2db80f91c983183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-11T23:04:12.037 INFO:journalctl@ceph.osd.1.smithi191.stdout:Sep 11 23:04:11 smithi191 podman[70895]: 2024-09-11 23:04:11.976975272 +0000 UTC m=+0.603757934 container died e18507f2a9d16b04a44f39912c65f6a5f3ffd92985b74745f2db80f91c983183 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-11T23:04:12.674 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T23:04:12.718 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:04:12.718 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T23:04:12.718 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T23:04:12.718 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.2 2024-09-11T23:04:13.045 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 23:04:12 smithi204 systemd[1]: Stopping Ceph osd.2 for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:04:13.546 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 23:04:13 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-2[40411]: 2024-09-11T23:04:13.155+0000 7f5ffe87f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:04:13.546 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 23:04:13 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-2[40411]: 2024-09-11T23:04:13.155+0000 7f5ffe87f640 -1 osd.2 95 *** Got signal Terminated *** 2024-09-11T23:04:13.546 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 23:04:13 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-2[40411]: 2024-09-11T23:04:13.155+0000 7f5ffe87f640 -1 osd.2 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:04:18.949 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 23:04:18 smithi204 podman[71777]: 2024-09-11 23:04:18.630948894 +0000 UTC m=+5.710210417 container died 24e3b0ff1398a72c96cae0064a59ca757fc4b768e15fb1e6ac03a97b214f1d6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T23:04:19.295 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 23:04:19 smithi204 podman[71777]: 2024-09-11 23:04:19.048147814 +0000 UTC m=+6.127409337 container cleanup 24e3b0ff1398a72c96cae0064a59ca757fc4b768e15fb1e6ac03a97b214f1d6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T23:04:19.296 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 23:04:19 smithi204 bash[71777]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-2 2024-09-11T23:04:19.721 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 23:04:19 smithi204 podman[71791]: 2024-09-11 23:04:19.308096831 +0000 UTC m=+0.670862532 container remove 24e3b0ff1398a72c96cae0064a59ca757fc4b768e15fb1e6ac03a97b214f1d6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T23:04:19.986 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 23:04:19 smithi204 podman[71883]: 2024-09-11 23:04:19.623907126 +0000 UTC m=+0.015973824 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:04:19.986 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 23:04:19 smithi204 podman[71883]: 2024-09-11 23:04:19.742510607 +0000 UTC m=+0.134577293 container create 61b4cde8acdf5729c983c194bee2988b5a05bfeb53346b31655f0a48dd0a9f1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:04:19.987 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 23:04:19 smithi204 podman[71883]: 2024-09-11 23:04:19.986987713 +0000 UTC m=+0.379054403 container init 61b4cde8acdf5729c983c194bee2988b5a05bfeb53346b31655f0a48dd0a9f1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T23:04:20.296 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 23:04:19 smithi204 podman[71883]: 2024-09-11 23:04:19.990349919 +0000 UTC m=+0.382416605 container start 61b4cde8acdf5729c983c194bee2988b5a05bfeb53346b31655f0a48dd0a9f1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=squid-b53b6d3, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T23:04:20.296 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 23:04:20 smithi204 podman[71883]: 2024-09-11 23:04:20.103828066 +0000 UTC m=+0.495894780 container attach 61b4cde8acdf5729c983c194bee2988b5a05bfeb53346b31655f0a48dd0a9f1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T23:04:20.297 INFO:journalctl@ceph.osd.2.smithi204.stdout:Sep 11 23:04:20 smithi204 podman[71883]: 2024-09-11 23:04:20.183361129 +0000 UTC m=+0.575427826 container died 61b4cde8acdf5729c983c194bee2988b5a05bfeb53346b31655f0a48dd0a9f1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-11T23:04:20.822 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T23:04:20.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:04:20.863 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T23:04:20.863 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T23:04:20.863 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.3 2024-09-11T23:04:21.179 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:20 smithi152 systemd[1]: Stopping Ceph osd.3 for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:04:21.622 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:21 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3[49580]: 2024-09-11T23:04:21.178+0000 7ff0e8102640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:04:21.622 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:21 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3[49580]: 2024-09-11T23:04:21.178+0000 7ff0e8102640 -1 osd.3 95 *** Got signal Terminated *** 2024-09-11T23:04:21.622 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:21 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3[49580]: 2024-09-11T23:04:21.178+0000 7ff0e8102640 -1 osd.3 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:04:23.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:22 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:22.756+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:23.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:22 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:22.968+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:24.064 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:23 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:23.722+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:24.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:24 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:24.014+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:24.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:24 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:24.111+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:24.372 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:24 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3[49580]: 2024-09-11T23:04:24.062+0000 7ff0e4718640 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.609541+0000 front 2024-09-11T23:03:59.609502+0000 (oldest deadline 2024-09-11T23:04:23.109635+0000) 2024-09-11T23:04:25.022 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:24 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:24.738+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:25.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:25 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:25.027+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:25.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:25 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:25.096+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:25.372 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:25 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3[49580]: 2024-09-11T23:04:25.021+0000 7ff0e4718640 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.609541+0000 front 2024-09-11T23:03:59.609502+0000 (oldest deadline 2024-09-11T23:04:23.109635+0000) 2024-09-11T23:04:26.032 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:25 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:25.751+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:26.285 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:26 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3[49580]: 2024-09-11T23:04:26.031+0000 7ff0e4718640 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.609541+0000 front 2024-09-11T23:03:59.609502+0000 (oldest deadline 2024-09-11T23:04:23.109635+0000) 2024-09-11T23:04:26.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:26 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:26.011+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:26.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:26 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:26.115+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:26.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:26 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:26.161+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:26.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:26 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:26.283+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:27.021 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:26 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:26.801+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:27.021 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:26 smithi152 podman[111892]: 2024-09-11 23:04:26.723306668 +0000 UTC m=+5.665893882 container died b4ea50a87771df27b3f8bc7cc66124734133362aef455ddf8ffb7b73d8211386 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True) 2024-09-11T23:04:27.289 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:27 smithi152 podman[111892]: 2024-09-11 23:04:27.120715395 +0000 UTC m=+6.063302609 container cleanup b4ea50a87771df27b3f8bc7cc66124734133362aef455ddf8ffb7b73d8211386 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-11T23:04:27.289 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:27 smithi152 bash[111892]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3 2024-09-11T23:04:27.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:27 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:27.139+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:27.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:27 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:27.007+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:27.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:27 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:27.127+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:27.622 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:27 smithi152 podman[111908]: 2024-09-11 23:04:27.388806368 +0000 UTC m=+0.663927379 container remove b4ea50a87771df27b3f8bc7cc66124734133362aef455ddf8ffb7b73d8211386 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T23:04:27.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:27 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:27.287+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:28.008 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:27 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:27.797+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:28.008 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:27 smithi152 podman[111998]: 2024-09-11 23:04:27.636463885 +0000 UTC m=+0.016250411 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:04:28.008 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:27 smithi152 podman[111998]: 2024-09-11 23:04:27.756116618 +0000 UTC m=+0.135903147 container create 901839fd8177234dcb4b6ef0c138e19d0e6aeda42fedce9dc919ca87327cbe66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-11T23:04:28.009 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:28 smithi152 podman[111998]: 2024-09-11 23:04:28.008451846 +0000 UTC m=+0.388238382 container init 901839fd8177234dcb4b6ef0c138e19d0e6aeda42fedce9dc919ca87327cbe66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:04:28.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:27 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:27.963+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:28.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:28 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:28.143+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:28.373 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:28 smithi152 podman[111998]: 2024-09-11 23:04:28.011959166 +0000 UTC m=+0.391745696 container start 901839fd8177234dcb4b6ef0c138e19d0e6aeda42fedce9dc919ca87327cbe66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:04:28.373 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:28 smithi152 podman[111998]: 2024-09-11 23:04:28.091978352 +0000 UTC m=+0.471764875 container attach 901839fd8177234dcb4b6ef0c138e19d0e6aeda42fedce9dc919ca87327cbe66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-09-11T23:04:28.374 INFO:journalctl@ceph.osd.3.smithi152.stdout:Sep 11 23:04:28 smithi152 podman[111998]: 2024-09-11 23:04:28.200430928 +0000 UTC m=+0.580217460 container died 901839fd8177234dcb4b6ef0c138e19d0e6aeda42fedce9dc919ca87327cbe66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T23:04:28.374 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:28 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:28.238+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:28.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:28 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:28.099+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:28.954 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:28 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:28.759+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:28.972 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T23:04:29.013 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:04:29.013 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T23:04:29.013 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T23:04:29.013 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.4 2024-09-11T23:04:29.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:29 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:29.149+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:29.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:29 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:29.005+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:29.328 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:29 smithi191 systemd[1]: Stopping Ceph osd.4 for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:04:29.328 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:29 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:29.075+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:29.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:29 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:29.228+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:29.786 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:29 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4[46568]: 2024-09-11T23:04:29.328+0000 7fc0cc58d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:04:29.787 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:29 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4[46568]: 2024-09-11T23:04:29.328+0000 7fc0cc58d640 -1 osd.4 95 *** Got signal Terminated *** 2024-09-11T23:04:29.787 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:29 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4[46568]: 2024-09-11T23:04:29.328+0000 7fc0cc58d640 -1 osd.4 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:04:30.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:29 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:29.771+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:30.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:30 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:30.044+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:30.545 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:30 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:30.051+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:30.546 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:30 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:30.122+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:30.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:30 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:30.259+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:31.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:30 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:30.764+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:31.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:30 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:30.764+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:31.295 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:31 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:31.157+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:31.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:31 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:31.022+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:31.431 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:31 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:31.087+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:31.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:31 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:31.264+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:31.789 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:31 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4[46568]: 2024-09-11T23:04:31.431+0000 7fc0c4b9b640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.000690+0000 front 2024-09-11T23:04:09.000666+0000 (oldest deadline 2024-09-11T23:04:31.300785+0000) 2024-09-11T23:04:32.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:31 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:31.734+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:32.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:31 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:31.734+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:32.295 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:32 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:32.004+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:32.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:32 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:32.114+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:32.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:32 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:32.145+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:32.434 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:32 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:32.088+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:32.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:32 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:32.315+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:32.787 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:32 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4[46568]: 2024-09-11T23:04:32.434+0000 7fc0c4b9b640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.000690+0000 front 2024-09-11T23:04:09.000666+0000 (oldest deadline 2024-09-11T23:04:31.300785+0000) 2024-09-11T23:04:33.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:32 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:32.758+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:33.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:32 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:32.758+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:33.295 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:32 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:32.992+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:33.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:32 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:32.992+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.642715+0000 front 2024-09-11T23:04:08.642644+0000 (oldest deadline 2024-09-11T23:04:32.142746+0000) 2024-09-11T23:04:33.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:33 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:33.194+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:33.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:33 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:33.077+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:33.338 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:33 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:33.069+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:33.338 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:33 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:33.069+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:33.625 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:33.313+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:33.787 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:33 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4[46568]: 2024-09-11T23:04:33.449+0000 7fc0c4b9b640 -1 osd.4 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.000690+0000 front 2024-09-11T23:04:09.000666+0000 (oldest deadline 2024-09-11T23:04:31.300785+0000) 2024-09-11T23:04:33.787 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:33 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:33.337+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:34.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:33.751+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:34.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:33.751+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:34.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:34 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:34.001+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:34.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:34 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:34.001+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.642715+0000 front 2024-09-11T23:04:08.642644+0000 (oldest deadline 2024-09-11T23:04:32.142746+0000) 2024-09-11T23:04:34.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:34 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:34.175+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:34.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:34 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:34.122+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:34.353 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:34 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:34.046+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:34.353 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:34 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:34.046+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:34.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:34 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:34.264+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:34.688 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:34 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:34.352+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:35.006 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:34 smithi191 podman[71011]: 2024-09-11 23:04:34.688892882 +0000 UTC m=+5.483225037 container died fb4ee4c5c8c2baddc24394d21024d10cdf3b4187b48009cabc2b56bb271d5bd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4, ceph=True, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T23:04:35.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:34 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:34.775+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:35.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:34 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:34.775+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:35.287 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:35 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:35.094+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:35.287 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:35 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:35.094+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:35.287 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:35 smithi191 podman[71011]: 2024-09-11 23:04:35.111158818 +0000 UTC m=+5.905490972 container cleanup fb4ee4c5c8c2baddc24394d21024d10cdf3b4187b48009cabc2b56bb271d5bd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T23:04:35.287 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:35 smithi191 bash[71011]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4 2024-09-11T23:04:35.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:34 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:34.977+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:35.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:34 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:34.977+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.642715+0000 front 2024-09-11T23:04:08.642644+0000 (oldest deadline 2024-09-11T23:04:32.142746+0000) 2024-09-11T23:04:35.297 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:35 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:35.128+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:35.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:35 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:35.117+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:35.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:35 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:35.117+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:35.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:35 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:35.234+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:35.780 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:35 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:35.355+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:35.781 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:35 smithi191 podman[71028]: 2024-09-11 23:04:35.371446672 +0000 UTC m=+0.684443900 container remove fb4ee4c5c8c2baddc24394d21024d10cdf3b4187b48009cabc2b56bb271d5bd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:04:36.037 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:35 smithi191 podman[71117]: 2024-09-11 23:04:35.683410732 +0000 UTC m=+0.016016098 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:04:36.037 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:35 smithi191 podman[71117]: 2024-09-11 23:04:35.797211052 +0000 UTC m=+0.129816404 container create 02a33ed07c0256376b165aed6a26c36887f41de96011fe50d021e3f471be3f9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T23:04:36.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:35 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:35.765+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:36.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:35 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:35.765+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:36.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:36 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:36.120+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:36.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:35 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:35.976+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:36.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:35 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:35.976+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.642715+0000 front 2024-09-11T23:04:08.642644+0000 (oldest deadline 2024-09-11T23:04:32.142746+0000) 2024-09-11T23:04:36.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:36 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:36.069+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:36.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:36 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:36.069+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:36.324 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:36 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:36.098+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:36.324 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:36 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:36.099+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:36.324 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:36 smithi191 podman[71117]: 2024-09-11 23:04:36.068190507 +0000 UTC m=+0.400795860 container init 02a33ed07c0256376b165aed6a26c36887f41de96011fe50d021e3f471be3f9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T23:04:36.324 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:36 smithi191 podman[71117]: 2024-09-11 23:04:36.07141804 +0000 UTC m=+0.404023412 container start 02a33ed07c0256376b165aed6a26c36887f41de96011fe50d021e3f471be3f9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-11T23:04:36.324 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:36 smithi191 podman[71117]: 2024-09-11 23:04:36.175171345 +0000 UTC m=+0.507776712 container attach 02a33ed07c0256376b165aed6a26c36887f41de96011fe50d021e3f471be3f9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-11T23:04:36.324 INFO:journalctl@ceph.osd.4.smithi191.stdout:Sep 11 23:04:36 smithi191 podman[71117]: 2024-09-11 23:04:36.256158738 +0000 UTC m=+0.588764101 container died 02a33ed07c0256376b165aed6a26c36887f41de96011fe50d021e3f471be3f9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-11T23:04:36.611 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:36 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:36.323+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:36.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:36.199+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:36.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:36.199+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:06.764912+0000 front 2024-09-11T23:04:06.764926+0000 (oldest deadline 2024-09-11T23:04:36.165105+0000) 2024-09-11T23:04:36.958 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T23:04:37.000 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:04:37.000 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T23:04:37.000 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T23:04:37.000 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.5 2024-09-11T23:04:37.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:36.778+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:37.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:36 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:36.778+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:37.287 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:37 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:37.097+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:37.287 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:37 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:37.097+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:37.294 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:37 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:37.091+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:37.295 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:36 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:36.956+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:37.295 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:36 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:36.956+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.642715+0000 front 2024-09-11T23:04:08.642644+0000 (oldest deadline 2024-09-11T23:04:32.142746+0000) 2024-09-11T23:04:37.295 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:37 smithi204 systemd[1]: Stopping Ceph osd.5 for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:04:37.295 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:37 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:37.055+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:37.295 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:37 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:37.055+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:37.545 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:37 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:37.294+0000 7fa7f02e7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:04:37.546 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:37 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:37.294+0000 7fa7f02e7640 -1 osd.5 95 *** Got signal Terminated *** 2024-09-11T23:04:37.546 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:37 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:37.294+0000 7fa7f02e7640 -1 osd.5 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:04:37.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:37 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:37.193+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:37.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:37 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:37.193+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:06.764912+0000 front 2024-09-11T23:04:06.764926+0000 (oldest deadline 2024-09-11T23:04:36.165105+0000) 2024-09-11T23:04:37.787 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:37 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:37.341+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:38.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:37 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:37.779+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:38.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:37 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:37.779+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:38.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:38 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:38.093+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:38.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:37 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:37.979+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:38.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:37 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:37.979+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.642715+0000 front 2024-09-11T23:04:08.642644+0000 (oldest deadline 2024-09-11T23:04:32.142746+0000) 2024-09-11T23:04:38.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:38 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:38.018+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:38.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:38 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:38.018+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:38.353 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:38 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:38.085+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:38.353 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:38 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:38.085+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:38.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:38 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:38.216+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:38.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:38 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:38.216+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:06.764912+0000 front 2024-09-11T23:04:06.764926+0000 (oldest deadline 2024-09-11T23:04:36.165105+0000) 2024-09-11T23:04:38.786 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:38 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:38.352+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:39.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:38 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:38.753+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:39.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:38 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:38.753+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:39.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:39 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:39.138+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:39.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:38 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:38.948+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:39.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:38 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:38.948+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.642715+0000 front 2024-09-11T23:04:08.642644+0000 (oldest deadline 2024-09-11T23:04:32.142746+0000) 2024-09-11T23:04:39.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:39 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:39.044+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:39.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:39 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:39.044+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:39.320 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:39 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:39.066+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:39.320 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:39 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:39.066+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:39.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:39 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:39.180+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:39.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:39 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:39.180+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:06.764912+0000 front 2024-09-11T23:04:06.764926+0000 (oldest deadline 2024-09-11T23:04:36.165105+0000) 2024-09-11T23:04:39.787 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:39 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:39.319+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:40.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:39 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:39.791+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:40.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:39 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:39.791+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:40.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:39 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:39.791+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:40.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:40 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:40.126+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:40.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:40 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:40.039+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:40.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:40 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:40.039+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:40.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:39 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:39.986+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:40.297 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:39 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:39.986+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.642715+0000 front 2024-09-11T23:04:08.642644+0000 (oldest deadline 2024-09-11T23:04:32.142746+0000) 2024-09-11T23:04:40.332 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:40 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:40.069+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:40.332 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:40 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:40.069+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:40.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:40 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:40.193+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:40.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:40 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:40.193+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:06.764912+0000 front 2024-09-11T23:04:06.764926+0000 (oldest deadline 2024-09-11T23:04:36.165105+0000) 2024-09-11T23:04:40.786 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:40 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:40.332+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:41.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:40 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:40.815+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:41.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:40 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:40.815+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:41.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:40 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:40.815+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:41.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:41 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:41.085+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:41.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:41 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:41.085+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:41.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:41 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:41.019+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:41.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:41 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:41.019+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.642715+0000 front 2024-09-11T23:04:08.642644+0000 (oldest deadline 2024-09-11T23:04:32.142746+0000) 2024-09-11T23:04:41.296 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:41 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:41.019+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:15.643551+0000 front 2024-09-11T23:04:15.643526+0000 (oldest deadline 2024-09-11T23:04:40.844293+0000) 2024-09-11T23:04:41.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:41 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:41.085+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:41.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:41 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:41.085+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:41.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:41 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:41.085+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:41.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:41 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:41.337+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:41.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:41 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:41.117+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:41.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:41 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:41.117+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:41.623 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:41 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:41.185+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:41.623 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:41 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:41.185+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:06.764912+0000 front 2024-09-11T23:04:06.764926+0000 (oldest deadline 2024-09-11T23:04:36.165105+0000) 2024-09-11T23:04:42.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:41 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:41.772+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:42.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:41 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:41.772+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:42.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:41 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:41.772+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:42.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:42 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:42.310+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:42.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:42 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:42.310+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:42.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:42 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:42.081+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:42.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:42 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:42.081+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:42.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:42 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:42.072+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:42.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:42 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:42.072+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:42.546 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:42 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:42.049+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:59.841725+0000 front 2024-09-11T23:03:59.841744+0000 (oldest deadline 2024-09-11T23:04:22.741650+0000) 2024-09-11T23:04:42.547 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:42 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:42.049+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.642715+0000 front 2024-09-11T23:04:08.642644+0000 (oldest deadline 2024-09-11T23:04:32.142746+0000) 2024-09-11T23:04:42.547 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:42 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5[47027]: 2024-09-11T23:04:42.049+0000 7fa7e88f5640 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:15.643551+0000 front 2024-09-11T23:04:15.643526+0000 (oldest deadline 2024-09-11T23:04:40.844293+0000) 2024-09-11T23:04:42.547 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:42 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:42.088+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:42.547 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:42 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:42.088+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:42.547 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:42 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:42.088+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:42.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:42 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:42.205+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:42.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:42 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:42.205+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:06.764912+0000 front 2024-09-11T23:04:06.764926+0000 (oldest deadline 2024-09-11T23:04:36.165105+0000) 2024-09-11T23:04:43.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:42 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:42.807+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:43.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:42 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:42.807+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:43.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:42 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:42.807+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:43.179 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:42 smithi204 podman[71987]: 2024-09-11 23:04:42.869143971 +0000 UTC m=+5.671563420 container died 0d2c7069792c981e01414158af482998c553a873e83e24a20a594f6bc3777e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True) 2024-09-11T23:04:43.179 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:43 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:43.061+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:43.179 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:43 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:43.061+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:43.179 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:43 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:43.079+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:43.180 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:43 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:43.079+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:43.180 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:43 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:43.079+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:43.512 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:43 smithi204 podman[71987]: 2024-09-11 23:04:43.294786781 +0000 UTC m=+6.097206229 container cleanup 0d2c7069792c981e01414158af482998c553a873e83e24a20a594f6bc3777e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-11T23:04:43.513 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:43 smithi204 bash[71987]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5 2024-09-11T23:04:43.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:43 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:43.263+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:43.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:43 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:43.263+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:43.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:43 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:43.100+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:43.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:43 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:43.100+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:43.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:43 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:43.100+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:43.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:43 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:43.255+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:43.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:43 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:43.255+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:06.764912+0000 front 2024-09-11T23:04:06.764926+0000 (oldest deadline 2024-09-11T23:04:36.165105+0000) 2024-09-11T23:04:43.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:43 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:43.255+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.166658+0000 front 2024-09-11T23:04:17.166743+0000 (oldest deadline 2024-09-11T23:04:42.466435+0000) 2024-09-11T23:04:43.795 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:43 smithi204 podman[72006]: 2024-09-11 23:04:43.513035499 +0000 UTC m=+0.642426072 container remove 0d2c7069792c981e01414158af482998c553a873e83e24a20a594f6bc3777e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:04:44.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:43 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:43.835+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:44.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:43 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:43.835+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:44.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:43 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:43.835+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:44.233 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:43 smithi204 podman[72097]: 2024-09-11 23:04:43.857519045 +0000 UTC m=+0.015873213 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:04:44.233 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:43 smithi204 podman[72097]: 2024-09-11 23:04:43.972331286 +0000 UTC m=+0.130685435 container create 43e1e582de9d0556d62b9226d8c0b87b4d67194d01e761e1e1cec62abd37fa8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-11T23:04:44.233 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:44 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:44.014+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:44.233 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:44 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:44.014+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:44.233 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:44 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:44.070+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:44.234 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:44 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:44.070+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:44.234 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:44 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:44.070+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:44.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:44 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:44.250+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:44.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:44 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:44.250+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:44.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:44 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:44.094+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:44.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:44 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:44.094+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:44.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:44 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:44.094+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:44.546 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:44 smithi204 podman[72097]: 2024-09-11 23:04:44.233271368 +0000 UTC m=+0.391625521 container init 43e1e582de9d0556d62b9226d8c0b87b4d67194d01e761e1e1cec62abd37fa8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T23:04:44.546 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:44 smithi204 podman[72097]: 2024-09-11 23:04:44.236645137 +0000 UTC m=+0.394999287 container start 43e1e582de9d0556d62b9226d8c0b87b4d67194d01e761e1e1cec62abd37fa8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T23:04:44.546 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:44 smithi204 podman[72097]: 2024-09-11 23:04:44.341720984 +0000 UTC m=+0.500075148 container attach 43e1e582de9d0556d62b9226d8c0b87b4d67194d01e761e1e1cec62abd37fa8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-11T23:04:44.546 INFO:journalctl@ceph.osd.5.smithi204.stdout:Sep 11 23:04:44 smithi204 podman[72097]: 2024-09-11 23:04:44.438217679 +0000 UTC m=+0.596571828 container died 43e1e582de9d0556d62b9226d8c0b87b4d67194d01e761e1e1cec62abd37fa8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T23:04:44.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:44 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:44.297+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:44.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:44 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:44.297+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:06.764912+0000 front 2024-09-11T23:04:06.764926+0000 (oldest deadline 2024-09-11T23:04:36.165105+0000) 2024-09-11T23:04:44.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:44 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:44.297+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.166658+0000 front 2024-09-11T23:04:17.166743+0000 (oldest deadline 2024-09-11T23:04:42.466435+0000) 2024-09-11T23:04:45.045 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:45 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:45.016+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:45.046 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:45 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:45.016+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:45.108 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T23:04:45.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:04:45.147 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T23:04:45.148 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T23:04:45.148 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.6 2024-09-11T23:04:45.175 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:44 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:44.876+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:45.175 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:44 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:44.876+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:45.175 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:44 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:44.876+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:45.473 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:45 smithi152 systemd[1]: Stopping Ceph osd.6 for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:04:45.473 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:45 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:45.341+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:45.473 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:45 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:45.341+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:06.764912+0000 front 2024-09-11T23:04:06.764926+0000 (oldest deadline 2024-09-11T23:04:36.165105+0000) 2024-09-11T23:04:45.473 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:45 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:45.341+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.166658+0000 front 2024-09-11T23:04:17.166743+0000 (oldest deadline 2024-09-11T23:04:42.466435+0000) 2024-09-11T23:04:45.474 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:45 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:45.471+0000 7fdf2afc0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:04:45.474 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:45 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:45.471+0000 7fdf2afc0640 -1 osd.6 95 *** Got signal Terminated *** 2024-09-11T23:04:45.474 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:45 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:45.471+0000 7fdf2afc0640 -1 osd.6 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:04:45.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:45 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:45.094+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:45.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:45 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:45.094+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:45.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:45 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:45.094+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:45.538 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:45 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:45.213+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:45.538 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:45 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:45.213+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:45.546 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:45 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:45.065+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:45.546 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:45 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:45.065+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:45.546 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:45 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:45.065+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:46.294 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:45 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:45.888+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:46.295 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:45 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:45.888+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:46.295 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:45 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:45.888+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:46.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:45 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:45.993+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:46.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:45 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:45.993+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:46.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:46 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:46.037+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:46.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:46 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:46.037+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:46.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:46 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:46.037+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:46.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:46 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:46.250+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:46.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:46 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:46.250+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:46.538 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:46 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:46.124+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:46.538 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:46 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:46.124+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:46.538 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:46 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:46.124+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:46.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:46 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:46.294+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:46.623 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:46 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:46.294+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:06.764912+0000 front 2024-09-11T23:04:06.764926+0000 (oldest deadline 2024-09-11T23:04:36.165105+0000) 2024-09-11T23:04:46.623 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:46 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:46.294+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.166658+0000 front 2024-09-11T23:04:17.166743+0000 (oldest deadline 2024-09-11T23:04:42.466435+0000) 2024-09-11T23:04:47.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:46 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:46.854+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:47.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:46 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:46.854+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:47.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:46 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:46.854+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:47.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:47 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:47.041+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:47.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:47 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:47.041+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:47.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:47 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:47.049+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:47.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:47 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:47.049+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:47.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:47 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:47.049+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:47.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:47 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:47.209+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:47.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:47 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:47.209+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:47.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:47 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:47.162+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:47.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:47 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:47.162+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:47.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:47 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:47.162+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:47.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:47 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:47.262+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:47.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:47 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:47.262+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:06.764912+0000 front 2024-09-11T23:04:06.764926+0000 (oldest deadline 2024-09-11T23:04:36.165105+0000) 2024-09-11T23:04:47.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:47 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:47.262+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.166658+0000 front 2024-09-11T23:04:17.166743+0000 (oldest deadline 2024-09-11T23:04:42.466435+0000) 2024-09-11T23:04:47.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:47 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:47.262+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.167348+0000 front 2024-09-11T23:04:24.167442+0000 (oldest deadline 2024-09-11T23:04:47.067396+0000) 2024-09-11T23:04:48.264 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:47 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:47.878+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:48.264 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:47 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:47.878+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:48.264 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:47 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:47.878+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:48.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:48 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:48.203+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:48.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:48 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:48.203+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:48.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:48 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:48.203+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:21.855324+0000 front 2024-09-11T23:04:21.855346+0000 (oldest deadline 2024-09-11T23:04:47.755459+0000) 2024-09-11T23:04:48.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:48 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:48.133+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:48.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:48 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:48.133+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:48.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:48 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:48.133+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:48.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:48 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:48.133+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:04:48.545 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:48 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:48.071+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:48.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:48 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:48.071+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:48.546 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:48 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:48.075+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:48.546 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:48 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:48.076+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:48.546 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:48 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:48.076+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:48.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:48 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:48.263+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:48.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:48 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:48.263+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:06.764912+0000 front 2024-09-11T23:04:06.764926+0000 (oldest deadline 2024-09-11T23:04:36.165105+0000) 2024-09-11T23:04:48.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:48 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:48.263+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.166658+0000 front 2024-09-11T23:04:17.166743+0000 (oldest deadline 2024-09-11T23:04:42.466435+0000) 2024-09-11T23:04:48.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:48 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:48.263+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.167348+0000 front 2024-09-11T23:04:24.167442+0000 (oldest deadline 2024-09-11T23:04:47.067396+0000) 2024-09-11T23:04:49.287 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:48 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:48.914+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:49.287 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:48 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:48.914+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:49.287 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:48 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:48.914+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:49.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:49 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:49.190+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:49.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:49 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:49.190+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:49.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:49 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:49.190+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:21.855324+0000 front 2024-09-11T23:04:21.855346+0000 (oldest deadline 2024-09-11T23:04:47.755459+0000) 2024-09-11T23:04:49.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:49 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:49.095+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:49.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:49 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:49.095+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:49.538 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:49 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:49.095+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:49.538 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:49 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:49.095+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:04:49.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:49 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:49.083+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:49.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:49 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:49.083+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:49.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:49 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:49.083+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:04:49.546 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:49 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:49.053+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:49.546 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:49 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:49.053+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:49.546 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:49 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:49.053+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:49.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:49 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:49.286+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:49.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:49 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:49.286+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:06.764912+0000 front 2024-09-11T23:04:06.764926+0000 (oldest deadline 2024-09-11T23:04:36.165105+0000) 2024-09-11T23:04:49.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:49 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:49.286+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.166658+0000 front 2024-09-11T23:04:17.166743+0000 (oldest deadline 2024-09-11T23:04:42.466435+0000) 2024-09-11T23:04:49.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:49 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:49.286+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.167348+0000 front 2024-09-11T23:04:24.167442+0000 (oldest deadline 2024-09-11T23:04:47.067396+0000) 2024-09-11T23:04:50.288 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:49 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:49.895+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:50.288 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:49 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:49.895+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:50.288 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:49 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:49.895+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:50.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:50 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:50.149+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:50.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:50 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:50.149+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:50.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:50 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:50.149+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:21.855324+0000 front 2024-09-11T23:04:21.855346+0000 (oldest deadline 2024-09-11T23:04:47.755459+0000) 2024-09-11T23:04:50.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:50 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:50.046+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:50.537 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:50 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:50.046+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:50.538 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:50 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:50.046+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:50.538 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:50 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:50.046+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:04:50.545 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:50 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:50.078+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:50.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:50 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:50.078+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:50.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:50 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:50.078+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:04:50.546 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:50 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:50.075+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:50.546 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:50 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:50.075+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:50.546 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:50 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:50.075+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:50.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:50.287+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.563593+0000 front 2024-09-11T23:04:01.563387+0000 (oldest deadline 2024-09-11T23:04:26.263543+0000) 2024-09-11T23:04:50.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:50.287+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:06.764912+0000 front 2024-09-11T23:04:06.764926+0000 (oldest deadline 2024-09-11T23:04:36.165105+0000) 2024-09-11T23:04:50.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:50.287+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.166658+0000 front 2024-09-11T23:04:17.166743+0000 (oldest deadline 2024-09-11T23:04:42.466435+0000) 2024-09-11T23:04:50.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6[56480]: 2024-09-11T23:04:50.287+0000 7fdf275d6640 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.167348+0000 front 2024-09-11T23:04:24.167442+0000 (oldest deadline 2024-09-11T23:04:47.067396+0000) 2024-09-11T23:04:51.122 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:50 smithi152 podman[112101]: 2024-09-11 23:04:50.894436281 +0000 UTC m=+5.540679044 container died 1dfe0787865630d1f49318576a3d034b9a5c35b2440b5badd6b33bef2bd44dce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-11T23:04:51.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:50.859+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:51.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:50.859+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:51.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:50 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:50.859+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:51.287 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:51.156+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:51.287 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:51.156+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:51.287 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:51.156+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:21.855324+0000 front 2024-09-11T23:04:21.855346+0000 (oldest deadline 2024-09-11T23:04:47.755459+0000) 2024-09-11T23:04:51.287 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:51.012+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:51.287 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:51.012+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:51.287 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:51.012+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:51.288 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:51 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:51.012+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:04:51.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:51.104+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:51.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:51.104+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:51.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:51.104+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:04:51.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:51.038+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:51.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:51.038+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:51.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:51 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:51.038+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:51.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:51 smithi152 podman[112101]: 2024-09-11 23:04:51.406088691 +0000 UTC m=+6.052331450 container cleanup 1dfe0787865630d1f49318576a3d034b9a5c35b2440b5badd6b33bef2bd44dce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:04:51.622 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:51 smithi152 bash[112101]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6 2024-09-11T23:04:52.080 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:51 smithi152 podman[112116]: 2024-09-11 23:04:51.665885998 +0000 UTC m=+0.769415336 container remove 1dfe0787865630d1f49318576a3d034b9a5c35b2440b5badd6b33bef2bd44dce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T23:04:52.221 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:51 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:51.877+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:52.221 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:51 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:51.877+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:52.221 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:51 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:51.877+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:52.221 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:51 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:51.877+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:04:52.271 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:52 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:52.201+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:52.271 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:52 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:52.201+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:52.271 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:52 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:52.201+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:21.855324+0000 front 2024-09-11T23:04:21.855346+0000 (oldest deadline 2024-09-11T23:04:47.755459+0000) 2024-09-11T23:04:52.271 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:52 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:52.002+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:52.271 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:52 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:52.002+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:52.271 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:52 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:52.002+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:52.272 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:52 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:52.002+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:04:52.295 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:52 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:52.106+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:52.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:52 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:52.106+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:52.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:52 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:52.106+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:04:52.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:52 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:52.010+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:52.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:52 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:52.010+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:52.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:52 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:52.010+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:52.372 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:52 smithi152 podman[112202]: 2024-09-11 23:04:51.983661191 +0000 UTC m=+0.016333467 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:04:52.373 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:52 smithi152 podman[112202]: 2024-09-11 23:04:52.120094912 +0000 UTC m=+0.152767231 container create 2473127f0d9255631e031f59b6b89adfc027c42403dd6087c3e92417067cd1ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T23:04:52.872 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:52 smithi152 podman[112202]: 2024-09-11 23:04:52.402886305 +0000 UTC m=+0.435558591 container init 2473127f0d9255631e031f59b6b89adfc027c42403dd6087c3e92417067cd1ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-11T23:04:52.873 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:52 smithi152 podman[112202]: 2024-09-11 23:04:52.406510223 +0000 UTC m=+0.439182503 container start 2473127f0d9255631e031f59b6b89adfc027c42403dd6087c3e92417067cd1ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:04:52.873 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:52 smithi152 podman[112202]: 2024-09-11 23:04:52.502636382 +0000 UTC m=+0.535308666 container attach 2473127f0d9255631e031f59b6b89adfc027c42403dd6087c3e92417067cd1ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-11T23:04:52.873 INFO:journalctl@ceph.osd.6.smithi152.stdout:Sep 11 23:04:52 smithi152 podman[112202]: 2024-09-11 23:04:52.603069081 +0000 UTC m=+0.635741356 container died 2473127f0d9255631e031f59b6b89adfc027c42403dd6087c3e92417067cd1ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:04:53.181 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:52 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:52.953+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:53.182 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:52 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:52.953+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:53.182 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:52 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:52.953+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:53.182 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:52 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:52.953+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:04:53.289 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:52 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:52.906+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:53.290 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:52 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:52.906+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:53.290 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:52 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:52.906+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:53.290 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:52 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:52.906+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:04:53.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:53 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:53.071+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:53.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:53 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:53.071+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:53.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:53 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:53.071+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:04:53.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:52 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:52.985+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:53.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:52 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:52.985+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:53.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:52 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:52.985+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:53.309 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T23:04:53.391 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:04:53.391 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T23:04:53.391 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T23:04:53.391 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.7 2024-09-11T23:04:53.440 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:53.181+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:53.440 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:53.181+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:53.440 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:53.181+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:21.855324+0000 front 2024-09-11T23:04:21.855346+0000 (oldest deadline 2024-09-11T23:04:47.755459+0000) 2024-09-11T23:04:53.714 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:53 smithi191 systemd[1]: Stopping Ceph osd.7 for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:04:53.977 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:53.714+0000 7ff729fd0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:04:53.977 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:53.714+0000 7ff729fd0640 -1 osd.7 95 *** Got signal Terminated *** 2024-09-11T23:04:53.977 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:53.714+0000 7ff729fd0640 -1 osd.7 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:04:54.287 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:54 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:54.140+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:54.287 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:54 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:54.140+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:54.287 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:54 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:54.140+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:21.855324+0000 front 2024-09-11T23:04:21.855346+0000 (oldest deadline 2024-09-11T23:04:47.755459+0000) 2024-09-11T23:04:54.287 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:53.976+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:54.287 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:53.976+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:54.287 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:53.976+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:54.288 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:53.976+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:04:54.295 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:54 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:54.100+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:54.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:54 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:54.100+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:54.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:54 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:54.100+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:04:54.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:53 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:53.960+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:54.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:53 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:53.960+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:54.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:53 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:53.960+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:54.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:53.891+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:54.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:53.891+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:54.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:53.891+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:54.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:53 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:53.891+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:04:55.287 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:55 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:55.179+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:55.287 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:55 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:55.179+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:55.288 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:55 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:55.179+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:21.855324+0000 front 2024-09-11T23:04:21.855346+0000 (oldest deadline 2024-09-11T23:04:47.755459+0000) 2024-09-11T23:04:55.288 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:54 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:54.950+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:55.289 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:54 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:54.950+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:55.289 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:54 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:54.950+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:55.289 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:54 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:54.950+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:04:55.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:55 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:55.095+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:55.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:55 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:55.095+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:55.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:55 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:55.095+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:04:55.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:54 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:54.912+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:55.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:54 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:54.912+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:55.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:54 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:54.912+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:55.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:54.890+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:55.373 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:54.890+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:55.373 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:54.890+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:55.373 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:54 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:54.890+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:04:56.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:55.869+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:56.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:55.869+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:56.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:55.869+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:56.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:55 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:55.869+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:04:56.287 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:56.144+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:56.287 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:56.144+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:56.287 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:56.144+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:21.855324+0000 front 2024-09-11T23:04:21.855346+0000 (oldest deadline 2024-09-11T23:04:47.755459+0000) 2024-09-11T23:04:56.287 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:55 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:55.921+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:56.287 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:55 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:55.921+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:56.287 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:55 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:55.921+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:56.287 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:55 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:55.921+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:04:56.288 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:55 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:55.921+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:04:56.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:56 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:56.115+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:56.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:56 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:56.115+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:56.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:56 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:56.115+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:04:56.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:55 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:55.877+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:56.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:55 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:55.877+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:56.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:55 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:55.877+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:57.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:56 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:56.821+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:57.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:56 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:56.821+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:57.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:56 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:56.821+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:57.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:56 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:56.821+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:04:57.160 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:56 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:56.902+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:57.160 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:56 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:56.902+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:57.160 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:56 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:56.902+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:57.165 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:56.885+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:57.165 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:56.885+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:57.166 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:56.885+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:57.166 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:56.885+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:04:57.166 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:56 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:56.885+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:04:57.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:57 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:57.166+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:57.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:57 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:57.166+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:57.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:57 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:57.166+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:21.855324+0000 front 2024-09-11T23:04:21.855346+0000 (oldest deadline 2024-09-11T23:04:47.755459+0000) 2024-09-11T23:04:57.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:57 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:57.166+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:34.057417+0000 front 2024-09-11T23:04:34.057466+0000 (oldest deadline 2024-09-11T23:04:56.357479+0000) 2024-09-11T23:04:57.547 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:57 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:57.160+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:57.547 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:57 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:57.160+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:57.547 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:57 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:57.160+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:04:58.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:57 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:57.840+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:58.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:57 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:57.840+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:58.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:57 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:57.840+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:58.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:57 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:57.840+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:04:58.132 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:57 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:57.837+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:58.132 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:57 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:57.837+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:58.132 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:57 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:57.837+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:58.132 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:57 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:57.837+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:04:58.132 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:57 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:57.837+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:04:58.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:58 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:58.173+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:58.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:58 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:58.173+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:58.296 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:58 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:58.173+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:04:58.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:57 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:57.933+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:58.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:57 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:57.933+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:58.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:57 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:57.933+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:58.297 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:57 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:57.933+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:04:58.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:58 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:58.132+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:08.353526+0000 front 2024-09-11T23:04:08.353645+0000 (oldest deadline 2024-09-11T23:04:33.053706+0000) 2024-09-11T23:04:58.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:58 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:58.132+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.154787+0000 front 2024-09-11T23:04:17.154912+0000 (oldest deadline 2024-09-11T23:04:41.854757+0000) 2024-09-11T23:04:58.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:58 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:58.132+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:21.855324+0000 front 2024-09-11T23:04:21.855346+0000 (oldest deadline 2024-09-11T23:04:47.755459+0000) 2024-09-11T23:04:58.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:58 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7[53374]: 2024-09-11T23:04:58.132+0000 7ff7225de640 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:34.057417+0000 front 2024-09-11T23:04:34.057466+0000 (oldest deadline 2024-09-11T23:04:56.357479+0000) 2024-09-11T23:04:59.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:58 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:58.834+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:04:59.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:58 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:58.834+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:04:59.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:58 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:58.834+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:04:59.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:58 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:58.834+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:04:59.187 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:58 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:58.925+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:04:59.188 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:58 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:58.925+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:04:59.188 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:58 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:58.925+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:04:59.188 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:58 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:58.925+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:04:59.272 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:58 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:58.798+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:59.272 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:58 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:58.798+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:59.272 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:58 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:58.798+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:59.273 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:58 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:58.798+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:04:59.273 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:58 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:58.798+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:04:59.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:59 smithi191 podman[71231]: 2024-09-11 23:04:59.272574289 +0000 UTC m=+5.684544163 container died 438878d3e765631c3a8176bf76761e2c11a0e105d9bc629454c7a33c49232e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-11T23:04:59.545 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:59 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:59.188+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:04:59.545 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:59 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:59.188+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:04:59.545 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:04:59 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:04:59.188+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:04:59.956 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:59 smithi191 podman[71231]: 2024-09-11 23:04:59.652222739 +0000 UTC m=+6.064192612 container cleanup 438878d3e765631c3a8176bf76761e2c11a0e105d9bc629454c7a33c49232e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T23:04:59.956 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:59 smithi191 bash[71231]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7 2024-09-11T23:04:59.957 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:59 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:59.752+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:04:59.957 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:59 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:59.752+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:04:59.957 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:59 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:59.752+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:04:59.957 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:59 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:59.752+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:04:59.957 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:04:59 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:04:59.752+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:00.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:59 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:59.836+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:05:00.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:59 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:59.836+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:05:00.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:59 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:59.836+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:05:00.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:04:59 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:04:59.836+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:05:00.193 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:59 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:59.906+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:00.193 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:59 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:59.906+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:00.193 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:59 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:59.906+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:00.193 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:04:59 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:04:59.906+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:00.269 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:04:59 smithi191 podman[71246]: 2024-09-11 23:04:59.956822591 +0000 UTC m=+0.678602451 container remove 438878d3e765631c3a8176bf76761e2c11a0e105d9bc629454c7a33c49232e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-11T23:05:00.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:05:00 smithi191 podman[71335]: 2024-09-11 23:05:00.172265358 +0000 UTC m=+0.016123572 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:05:00.537 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:05:00 smithi191 podman[71335]: 2024-09-11 23:05:00.2913046 +0000 UTC m=+0.135162802 container create db949b417201fe05967aa8ac4cc33589d74ee7c3cbd8108ada46fe43ba1f3363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-11T23:05:00.545 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:00 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:00.193+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:05:00.545 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:00 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:00.193+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:05:00.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:00 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:00.193+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:05:00.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:00 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:00.193+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:33.990845+0000 front 2024-09-11T23:04:33.990787+0000 (oldest deadline 2024-09-11T23:04:59.890725+0000) 2024-09-11T23:05:00.851 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:05:00 smithi191 podman[71335]: 2024-09-11 23:05:00.660732217 +0000 UTC m=+0.504590438 container init db949b417201fe05967aa8ac4cc33589d74ee7c3cbd8108ada46fe43ba1f3363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:05:00.851 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:05:00 smithi191 podman[71335]: 2024-09-11 23:05:00.664399203 +0000 UTC m=+0.508257407 container start db949b417201fe05967aa8ac4cc33589d74ee7c3cbd8108ada46fe43ba1f3363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:05:00.851 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:05:00 smithi191 podman[71335]: 2024-09-11 23:05:00.735555726 +0000 UTC m=+0.579413932 container attach db949b417201fe05967aa8ac4cc33589d74ee7c3cbd8108ada46fe43ba1f3363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:05:00.851 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:00 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:00.743+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:00.852 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:00 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:00.743+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:00.852 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:00 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:00.743+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:00.852 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:00 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:00.743+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:00.852 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:00 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:00.743+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:01.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:00 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:00.811+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:05:01.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:00 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:00.811+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:05:01.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:00 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:00.811+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:05:01.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:00 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:00.811+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:05:01.197 INFO:journalctl@ceph.osd.7.smithi191.stdout:Sep 11 23:05:00 smithi191 podman[71335]: 2024-09-11 23:05:00.851934903 +0000 UTC m=+0.695793113 container died db949b417201fe05967aa8ac4cc33589d74ee7c3cbd8108ada46fe43ba1f3363 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T23:05:01.228 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:00 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:00.868+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:01.228 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:00 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:00.868+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:01.228 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:00 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:00.868+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:01.228 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:00 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:00.868+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:01.531 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T23:05:01.545 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:01 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:01.228+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:05:01.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:01 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:01.228+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:05:01.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:01 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:01.228+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:05:01.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:01 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:01.228+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:33.990845+0000 front 2024-09-11T23:04:33.990787+0000 (oldest deadline 2024-09-11T23:04:59.890725+0000) 2024-09-11T23:05:01.570 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:05:01.570 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T23:05:01.571 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-11T23:05:01.571 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.8 2024-09-11T23:05:01.780 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:01 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:01.736+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:01.780 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:01 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:01.736+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:01.780 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:01 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:01.736+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:01.780 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:01 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:01.736+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:01.780 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:01 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:01.736+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:01.907 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:01 smithi204 systemd[1]: Stopping Ceph osd.8 for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:05:01.907 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:01 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:01.906+0000 7eff96445640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:05:01.908 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:01 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:01.820+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:01.908 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:01 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:01.820+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:01.908 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:01 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:01.820+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:01.908 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:01 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:01.820+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:02.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:01 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:01.843+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:05:02.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:01 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:01.843+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:05:02.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:01 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:01.843+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:05:02.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:01 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:01.843+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:05:02.250 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:01 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:01.906+0000 7eff96445640 -1 osd.8 95 *** Got signal Terminated *** 2024-09-11T23:05:02.251 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:01 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:01.906+0000 7eff96445640 -1 osd.8 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:05:02.549 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:02 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:02.250+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:05:02.549 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:02 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:02.250+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:05:02.549 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:02 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:02.250+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:05:02.549 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:02 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:02.250+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:33.990845+0000 front 2024-09-11T23:04:33.990787+0000 (oldest deadline 2024-09-11T23:04:59.890725+0000) 2024-09-11T23:05:03.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:02 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:02.701+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:03.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:02 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:02.701+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:03.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:02 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:02.701+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:03.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:02 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:02.701+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:03.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:02 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:02.701+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:03.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:02 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:02.845+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:05:03.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:02 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:02.845+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:05:03.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:02 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:02.845+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:05:03.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:02 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:02.845+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:05:03.223 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:02 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:02.799+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:03.223 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:02 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:02.799+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:03.223 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:02 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:02.799+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:03.223 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:02 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:02.799+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:03.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:03 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:03.223+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:05:03.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:03 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:03.223+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:05:03.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:03 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:03.223+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:05:03.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:03 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:03.223+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:33.990845+0000 front 2024-09-11T23:04:33.990787+0000 (oldest deadline 2024-09-11T23:04:59.890725+0000) 2024-09-11T23:05:04.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:03 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:03.691+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:04.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:03 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:03.691+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:04.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:03 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:03.691+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:04.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:03 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:03.691+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:04.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:03 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:03.691+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:04.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:03 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:03.768+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:04.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:03 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:03.768+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:04.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:03 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:03.768+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:04.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:03 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:03.768+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:04.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:03 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:03.880+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:05:04.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:03 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:03.880+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:05:04.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:03 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:03.880+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:05:04.373 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:03 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:03.880+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:05:04.373 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:03 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:03.880+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.706992+0000 front 2024-09-11T23:04:39.707037+0000 (oldest deadline 2024-09-11T23:05:03.207103+0000) 2024-09-11T23:05:04.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:04 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:04.216+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:05:04.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:04 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:04.216+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:05:04.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:04 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:04.216+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:05:04.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:04 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:04.216+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:33.990845+0000 front 2024-09-11T23:04:33.990787+0000 (oldest deadline 2024-09-11T23:04:59.890725+0000) 2024-09-11T23:05:04.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:04 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:04.216+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.891186+0000 front 2024-09-11T23:04:39.891206+0000 (oldest deadline 2024-09-11T23:05:03.991283+0000) 2024-09-11T23:05:05.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:04 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:04.700+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:05.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:04 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:04.700+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:05.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:04 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:04.700+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:05.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:04 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:04.700+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:05.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:04 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:04.700+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:05.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:04 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:04.738+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:05.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:04 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:04.738+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:05.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:04 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:04.738+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:05.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:04 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:04.738+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:05.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:04 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:04.738+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:05.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:04 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:04.876+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:05:05.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:04 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:04.876+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:05:05.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:04 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:04.876+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:05:05.373 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:04 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:04.876+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:05:05.373 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:04 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:04.876+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.706992+0000 front 2024-09-11T23:04:39.707037+0000 (oldest deadline 2024-09-11T23:05:03.207103+0000) 2024-09-11T23:05:05.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:05 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:05.220+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:05:05.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:05 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:05.220+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:05:05.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:05 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:05.220+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:05:05.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:05 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:05.220+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:33.990845+0000 front 2024-09-11T23:04:33.990787+0000 (oldest deadline 2024-09-11T23:04:59.890725+0000) 2024-09-11T23:05:05.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:05 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:05.220+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.891186+0000 front 2024-09-11T23:04:39.891206+0000 (oldest deadline 2024-09-11T23:05:03.991283+0000) 2024-09-11T23:05:06.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:05 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:05.652+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:06.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:05 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:05.652+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:06.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:05 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:05.652+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:06.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:05 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:05.652+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:06.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:05 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:05.652+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:06.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:05 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:05.652+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:06.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:05 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:05.784+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:06.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:05 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:05.784+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:06.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:05 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:05.784+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:06.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:05 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:05.784+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:06.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:05 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:05.784+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:06.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:05 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:05.850+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:05:06.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:05 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:05.850+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:05:06.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:05 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:05.850+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:05:06.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:05 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:05.850+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:05:06.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:05 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:05.850+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.706992+0000 front 2024-09-11T23:04:39.707037+0000 (oldest deadline 2024-09-11T23:05:03.207103+0000) 2024-09-11T23:05:06.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:06 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:06.210+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:07.886038+0000 front 2024-09-11T23:04:07.885914+0000 (oldest deadline 2024-09-11T23:04:31.985980+0000) 2024-09-11T23:05:06.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:06 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:06.210+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.287058+0000 front 2024-09-11T23:04:17.287078+0000 (oldest deadline 2024-09-11T23:04:40.187183+0000) 2024-09-11T23:05:06.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:06 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:06.210+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:22.388934+0000 front 2024-09-11T23:04:22.388803+0000 (oldest deadline 2024-09-11T23:04:48.789193+0000) 2024-09-11T23:05:06.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:06 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:06.210+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:33.990845+0000 front 2024-09-11T23:04:33.990787+0000 (oldest deadline 2024-09-11T23:04:59.890725+0000) 2024-09-11T23:05:06.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:06 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8[53985]: 2024-09-11T23:05:06.210+0000 7eff92a5b640 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.891186+0000 front 2024-09-11T23:04:39.891206+0000 (oldest deadline 2024-09-11T23:05:03.991283+0000) 2024-09-11T23:05:07.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:06 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:06.647+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:07.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:06 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:06.647+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:07.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:06 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:06.647+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:07.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:06 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:06.647+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:07.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:06 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:06.647+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:07.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:06 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:06.647+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:07.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:06 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:06.855+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:05:07.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:06 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:06.855+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:05:07.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:06 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:06.855+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:05:07.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:06 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:06.855+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:05:07.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:06 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:06.855+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.706992+0000 front 2024-09-11T23:04:39.707037+0000 (oldest deadline 2024-09-11T23:05:03.207103+0000) 2024-09-11T23:05:07.243 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:06 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:06.817+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:07.243 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:06 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:06.817+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:07.243 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:06 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:06.817+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:07.243 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:06 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:06.817+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:07.243 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:06 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:06.817+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:07.533 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:07 smithi204 podman[72199]: 2024-09-11 23:05:07.24303779 +0000 UTC m=+5.475306254 container died 8be7291c9836e6bad9dd4c0e541a02d3ef4d02f7c54278891abb2ae2f718702f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T23:05:07.796 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:07 smithi204 podman[72199]: 2024-09-11 23:05:07.632286605 +0000 UTC m=+5.864555081 container cleanup 8be7291c9836e6bad9dd4c0e541a02d3ef4d02f7c54278891abb2ae2f718702f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T23:05:07.796 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:07 smithi204 bash[72199]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8 2024-09-11T23:05:08.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:07 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:07.655+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:08.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:07 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:07.655+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:08.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:07 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:07.655+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:08.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:07 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:07.655+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:08.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:07 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:07.655+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:08.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:07 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:07.655+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:08.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:07 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:07.852+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:05:08.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:07 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:07.852+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:05:08.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:07 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:07.852+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:05:08.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:07 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:07.852+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:05:08.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:07 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:07.852+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.706992+0000 front 2024-09-11T23:04:39.707037+0000 (oldest deadline 2024-09-11T23:05:03.207103+0000) 2024-09-11T23:05:08.242 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:07 smithi204 podman[72216]: 2024-09-11 23:05:07.900527211 +0000 UTC m=+0.650644152 container remove 8be7291c9836e6bad9dd4c0e541a02d3ef4d02f7c54278891abb2ae2f718702f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True) 2024-09-11T23:05:08.242 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:07 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:07.840+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:08.243 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:07 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:07.840+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:08.243 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:07 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:07.840+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:08.243 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:07 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:07.840+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:08.243 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:07 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:07.840+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:08.545 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:08 smithi204 podman[72308]: 2024-09-11 23:05:08.145016674 +0000 UTC m=+0.015938612 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:05:08.546 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:08 smithi204 podman[72308]: 2024-09-11 23:05:08.301942216 +0000 UTC m=+0.172864154 container create 5e5b98369e80a60d6180c514a57dc64f6a230051c9e8b4136f148c94c70fe796 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:05:08.804 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:08 smithi204 podman[72308]: 2024-09-11 23:05:08.571041346 +0000 UTC m=+0.441963286 container init 5e5b98369e80a60d6180c514a57dc64f6a230051c9e8b4136f148c94c70fe796 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T23:05:08.805 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:08 smithi204 podman[72308]: 2024-09-11 23:05:08.574412881 +0000 UTC m=+0.445334818 container start 5e5b98369e80a60d6180c514a57dc64f6a230051c9e8b4136f148c94c70fe796 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8-deactivate, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-11T23:05:08.805 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:08 smithi204 podman[72308]: 2024-09-11 23:05:08.63784042 +0000 UTC m=+0.508762366 container attach 5e5b98369e80a60d6180c514a57dc64f6a230051c9e8b4136f148c94c70fe796 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-11T23:05:08.805 INFO:journalctl@ceph.osd.8.smithi204.stdout:Sep 11 23:05:08 smithi204 podman[72308]: 2024-09-11 23:05:08.771050119 +0000 UTC m=+0.641972069 container died 5e5b98369e80a60d6180c514a57dc64f6a230051c9e8b4136f148c94c70fe796 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T23:05:09.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:08 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:08.628+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:09.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:08 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:08.628+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:09.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:08 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:08.628+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:09.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:08 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:08.628+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:09.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:08 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:08.628+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:09.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:08 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:08.628+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:09.082 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:08 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:08.802+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:09.082 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:08 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:08.802+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:09.082 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:08 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:08.802+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:09.082 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:08 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:08.802+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:09.082 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:08 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:08.802+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:09.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:08 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:08.858+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:05:09.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:08 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:08.858+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:05:09.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:08 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:08.858+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:05:09.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:08 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:08.858+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:05:09.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:08 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:08.858+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.706992+0000 front 2024-09-11T23:04:39.707037+0000 (oldest deadline 2024-09-11T23:05:03.207103+0000) 2024-09-11T23:05:09.412 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.8.service' 2024-09-11T23:05:09.450 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:05:09.450 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-11T23:05:09.450 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-11T23:05:09.450 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.9 2024-09-11T23:05:09.857 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:09 smithi152 systemd[1]: Stopping Ceph osd.9 for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:05:10.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:09 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:09.654+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:10.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:09 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:09.654+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:10.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:09 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:09.654+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:10.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:09 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:09.654+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:10.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:09 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:09.654+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:10.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:09 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:09.654+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:10.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:09 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:09.766+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:10.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:09 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:09.766+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:10.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:09 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:09.766+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:10.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:09 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:09.766+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:10.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:09 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:09.766+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:10.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:09 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:09.856+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:05:10.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:09 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:09.856+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:05:10.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:09 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:09.856+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:05:10.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:09 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:09.856+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:05:10.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:09 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:09.856+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.706992+0000 front 2024-09-11T23:04:39.707037+0000 (oldest deadline 2024-09-11T23:05:03.207103+0000) 2024-09-11T23:05:10.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:09 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:09.878+0000 7f49048d9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:05:10.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:09 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:09.878+0000 7f49048d9640 -1 osd.9 95 *** Got signal Terminated *** 2024-09-11T23:05:10.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:09 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:09.878+0000 7f49048d9640 -1 osd.9 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:05:11.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:10 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:10.696+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:11.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:10 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:10.696+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:11.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:10 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:10.696+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:11.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:10 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:10.696+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:11.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:10 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:10.696+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:11.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:10 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:10.696+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:11.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:10 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:10.717+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:11.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:10 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:10.717+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:11.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:10 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:10.717+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:11.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:10 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:10.717+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:11.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:10 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:10.717+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:11.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:10 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:10.882+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:05:11.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:10 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:10.882+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:05:11.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:10 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:10.882+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:05:11.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:10 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:10.882+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:05:11.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:10 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:10.882+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.706992+0000 front 2024-09-11T23:04:39.707037+0000 (oldest deadline 2024-09-11T23:05:03.207103+0000) 2024-09-11T23:05:11.938 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:11 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:11.663+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:11.938 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:11 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:11.663+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:11.938 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:11 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:11.663+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:11.939 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:11 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:11.663+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:11.939 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:11 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:11.663+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:11.939 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:11 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:11.663+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:12.045 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:11 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:11.671+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:12.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:11 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:11.671+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:12.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:11 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:11.671+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:12.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:11 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:11.671+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:12.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:11 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:11.671+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:12.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:11 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:11.864+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:05:12.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:11 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:11.864+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:05:12.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:11 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:11.864+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:05:12.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:11 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:11.864+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:05:12.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:11 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:11.864+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.706992+0000 front 2024-09-11T23:04:39.707037+0000 (oldest deadline 2024-09-11T23:05:03.207103+0000) 2024-09-11T23:05:13.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:12 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:12.686+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:13.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:12 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:12.686+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:13.038 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:12 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:12.686+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:13.038 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:12 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:12.686+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:13.038 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:12 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:12.686+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:13.038 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:12 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:12.686+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:13.038 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:12 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:12.686+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:49.539173+0000 front 2024-09-11T23:04:49.539118+0000 (oldest deadline 2024-09-11T23:05:12.439229+0000) 2024-09-11T23:05:13.045 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:12 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:12.668+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:13.045 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:12 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:12.668+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:13.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:12 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:12.668+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:13.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:12 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:12.668+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:13.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:12 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:12.668+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:13.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:12 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:12.891+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:05:13.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:12 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:12.891+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:05:13.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:12 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:12.891+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:05:13.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:12 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:12.891+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:05:13.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:12 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:12.891+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.706992+0000 front 2024-09-11T23:04:39.707037+0000 (oldest deadline 2024-09-11T23:05:03.207103+0000) 2024-09-11T23:05:13.947 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:13 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:13.653+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:13.947 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:13 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:13.653+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:13.948 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:13 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:13.653+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:13.948 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:13 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:13.653+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:13.948 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:13 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:13.653+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:13.948 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:13 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:13.653+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:14.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:13 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:13.644+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:14.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:13 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:13.645+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:14.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:13 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:13.645+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:14.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:13 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:13.645+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:14.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:13 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:13.645+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:14.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:13 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:13.645+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:14.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:13 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:13.645+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:49.539173+0000 front 2024-09-11T23:04:49.539118+0000 (oldest deadline 2024-09-11T23:05:12.439229+0000) 2024-09-11T23:05:14.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:13 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:13.920+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:05:14.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:13 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:13.920+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:05:14.372 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:13 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:13.920+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:05:14.373 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:13 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:13.920+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:05:14.373 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:13 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:13.920+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.706992+0000 front 2024-09-11T23:04:39.707037+0000 (oldest deadline 2024-09-11T23:05:03.207103+0000) 2024-09-11T23:05:15.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:14 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:14.634+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:15.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:14 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:14.634+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:15.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:14 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:14.634+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:15.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:14 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:14.634+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:15.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:14 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:14.634+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:15.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:14 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:14.634+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:15.038 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:14 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:14.634+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:49.539173+0000 front 2024-09-11T23:04:49.539118+0000 (oldest deadline 2024-09-11T23:05:12.439229+0000) 2024-09-11T23:05:15.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:14 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:14.623+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:15.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:14 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:14.623+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:15.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:14 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:14.623+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:15.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:14 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:14.623+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:15.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:14 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:14.623+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:15.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:14 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:14.623+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:15.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:14 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:14.871+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:03:58.300720+0000 front 2024-09-11T23:03:58.300922+0000 (oldest deadline 2024-09-11T23:04:22.401079+0000) 2024-09-11T23:05:15.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:14 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:14.871+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.702101+0000 front 2024-09-11T23:04:04.702174+0000 (oldest deadline 2024-09-11T23:04:30.601930+0000) 2024-09-11T23:05:15.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:14 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:14.871+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.703109+0000 front 2024-09-11T23:04:14.703087+0000 (oldest deadline 2024-09-11T23:04:38.803017+0000) 2024-09-11T23:05:15.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:14 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:14.871+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:23.305257+0000 front 2024-09-11T23:04:23.305123+0000 (oldest deadline 2024-09-11T23:04:50.905705+0000) 2024-09-11T23:05:15.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:14 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:14.871+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.706992+0000 front 2024-09-11T23:04:39.707037+0000 (oldest deadline 2024-09-11T23:05:03.207103+0000) 2024-09-11T23:05:15.123 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:14 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9[63299]: 2024-09-11T23:05:14.871+0000 7f48fcee7640 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:48.909004+0000 front 2024-09-11T23:04:48.909037+0000 (oldest deadline 2024-09-11T23:05:14.709552+0000) 2024-09-11T23:05:15.692 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:15 smithi152 podman[112305]: 2024-09-11 23:05:15.397890992 +0000 UTC m=+5.752926176 container died 64e13f18f496455ddcf832c5b8001cdc67bc071d2916bf4c46dee2c506f91c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:05:16.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:15 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:15.609+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:16.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:15 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:15.609+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:16.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:15 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:15.609+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:16.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:15 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:15.609+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:16.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:15 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:15.609+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:16.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:15 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:15.609+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:16.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:15 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:15.609+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:49.539173+0000 front 2024-09-11T23:04:49.539118+0000 (oldest deadline 2024-09-11T23:05:12.439229+0000) 2024-09-11T23:05:16.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:15 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:15.608+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:16.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:15 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:15.608+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:16.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:15 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:15.608+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:16.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:15 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:15.608+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:16.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:15 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:15.608+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:16.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:15 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:15.608+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:16.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:15 smithi152 podman[112305]: 2024-09-11 23:05:15.866195008 +0000 UTC m=+6.221230227 container cleanup 64e13f18f496455ddcf832c5b8001cdc67bc071d2916bf4c46dee2c506f91c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-11T23:05:16.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:15 smithi152 bash[112305]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9 2024-09-11T23:05:16.543 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:16 smithi152 podman[112322]: 2024-09-11 23:05:16.184225257 +0000 UTC m=+0.787991396 container remove 64e13f18f496455ddcf832c5b8001cdc67bc071d2916bf4c46dee2c506f91c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T23:05:16.804 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:16 smithi152 podman[112410]: 2024-09-11 23:05:16.446101395 +0000 UTC m=+0.015649739 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:05:16.804 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:16 smithi152 podman[112410]: 2024-09-11 23:05:16.576422011 +0000 UTC m=+0.145970354 container create 0b318e61b45df93c3b6601500847ba084e565b1e0198109ee5ac8ef583b0abf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:05:16.804 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:16 smithi152 podman[112410]: 2024-09-11 23:05:16.804010581 +0000 UTC m=+0.373558939 container init 0b318e61b45df93c3b6601500847ba084e565b1e0198109ee5ac8ef583b0abf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2) 2024-09-11T23:05:17.040 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:16 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:16.593+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:17.040 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:16 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:16.593+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:17.040 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:16 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:16.593+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:17.041 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:16 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:16.593+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:17.041 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:16 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:16.593+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:17.041 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:16 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:16.593+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:17.041 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:16 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:16.593+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:49.539173+0000 front 2024-09-11T23:04:49.539118+0000 (oldest deadline 2024-09-11T23:05:12.439229+0000) 2024-09-11T23:05:17.045 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:16 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:16.603+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:17.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:16 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:16.603+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:17.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:16 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:16.603+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:17.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:16 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:16.603+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:17.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:16 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:16.603+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:17.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:16 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:16.603+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:17.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:16 smithi152 podman[112410]: 2024-09-11 23:05:16.807584641 +0000 UTC m=+0.377132990 container start 0b318e61b45df93c3b6601500847ba084e565b1e0198109ee5ac8ef583b0abf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-11T23:05:17.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:16 smithi152 podman[112410]: 2024-09-11 23:05:16.895390097 +0000 UTC m=+0.464938441 container attach 0b318e61b45df93c3b6601500847ba084e565b1e0198109ee5ac8ef583b0abf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-11T23:05:17.122 INFO:journalctl@ceph.osd.9.smithi152.stdout:Sep 11 23:05:16 smithi152 podman[112410]: 2024-09-11 23:05:16.99654274 +0000 UTC m=+0.566091088 container died 0b318e61b45df93c3b6601500847ba084e565b1e0198109ee5ac8ef583b0abf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-11T23:05:17.650 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.9.service' 2024-09-11T23:05:17.689 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:05:17.689 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-11T23:05:17.690 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-11T23:05:17.690 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.10 2024-09-11T23:05:18.017 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:17 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:17.551+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:18.017 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:17 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:17.551+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:18.017 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:17 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:17.551+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:18.017 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:17 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:17.551+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:18.017 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:17 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:17.551+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:18.017 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:17 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:17.551+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:18.017 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:17 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:17.551+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:49.539173+0000 front 2024-09-11T23:04:49.539118+0000 (oldest deadline 2024-09-11T23:05:12.439229+0000) 2024-09-11T23:05:18.017 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:17 smithi191 systemd[1]: Stopping Ceph osd.10 for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:05:18.018 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:18 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:18.015+0000 7fb2cbf71640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:05:18.018 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:18 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:18.015+0000 7fb2cbf71640 -1 osd.10 95 *** Got signal Terminated *** 2024-09-11T23:05:18.018 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:18 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:18.015+0000 7fb2cbf71640 -1 osd.10 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:05:18.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:17 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:17.623+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:18.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:17 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:17.623+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:18.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:17 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:17.623+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:18.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:17 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:17.623+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:18.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:17 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:17.623+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:18.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:17 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:17.623+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:18.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:18 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:18.523+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:18.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:18 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:18.523+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:18.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:18 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:18.523+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:18.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:18 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:18.523+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:18.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:18 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:18.523+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:18.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:18 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:18.523+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:18.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:18 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:18.523+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:49.539173+0000 front 2024-09-11T23:04:49.539118+0000 (oldest deadline 2024-09-11T23:05:12.439229+0000) 2024-09-11T23:05:19.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:18 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:18.653+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:19.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:18 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:18.653+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:19.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:18 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:18.653+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:19.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:18 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:18.653+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:19.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:18 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:18.653+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:19.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:18 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:18.653+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:19.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:19 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:19.532+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:19.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:19 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:19.532+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:19.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:19 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:19.532+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:19.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:19 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:19.532+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:19.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:19 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:19.532+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:19.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:19 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:19.532+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:19.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:19 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:19.532+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:49.539173+0000 front 2024-09-11T23:04:49.539118+0000 (oldest deadline 2024-09-11T23:05:12.439229+0000) 2024-09-11T23:05:20.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:19 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:19.608+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:20.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:19 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:19.608+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:20.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:19 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:19.608+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:20.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:19 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:19.608+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:20.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:19 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:19.608+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:20.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:19 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:19.608+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:20.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:20 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:20.484+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:20.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:20 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:20.484+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:20.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:20 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:20.484+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:20.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:20 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:20.484+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:20.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:20 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:20.484+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:20.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:20 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:20.484+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:20.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:20 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:20.484+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:49.539173+0000 front 2024-09-11T23:04:49.539118+0000 (oldest deadline 2024-09-11T23:05:12.439229+0000) 2024-09-11T23:05:21.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:20 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:20.622+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:21.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:20 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:20.622+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:21.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:20 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:20.622+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:21.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:20 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:20.622+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:21.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:20 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:20.622+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:21.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:20 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:20.622+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:21.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:21 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:21.527+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:21.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:21 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:21.527+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:21.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:21 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:21.527+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:21.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:21 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:21.527+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:21.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:21 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:21.527+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:21.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:21 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:21.527+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:21.788 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:21 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:21.527+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:49.539173+0000 front 2024-09-11T23:04:49.539118+0000 (oldest deadline 2024-09-11T23:05:12.439229+0000) 2024-09-11T23:05:22.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:21 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:21.636+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:22.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:21 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:21.636+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:22.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:21 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:21.636+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:22.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:21 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:21.636+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:22.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:21 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:21.636+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:22.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:21 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:21.636+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:22.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:22 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:22.498+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.430670+0000 front 2024-09-11T23:04:01.430707+0000 (oldest deadline 2024-09-11T23:04:26.130450+0000) 2024-09-11T23:05:22.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:22 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:22.498+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:09.532074+0000 front 2024-09-11T23:04:09.532101+0000 (oldest deadline 2024-09-11T23:04:32.432263+0000) 2024-09-11T23:05:22.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:22 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:22.498+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:17.133245+0000 front 2024-09-11T23:04:17.133217+0000 (oldest deadline 2024-09-11T23:04:42.433276+0000) 2024-09-11T23:05:22.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:22 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:22.498+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6814 osd.3 since back 2024-09-11T23:04:24.734243+0000 front 2024-09-11T23:04:24.734402+0000 (oldest deadline 2024-09-11T23:04:47.634131+0000) 2024-09-11T23:05:22.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:22 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:22.498+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.935082+0000 front 2024-09-11T23:04:32.935118+0000 (oldest deadline 2024-09-11T23:04:55.735685+0000) 2024-09-11T23:05:22.788 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:22 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:22.498+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:39.836663+0000 front 2024-09-11T23:04:39.836640+0000 (oldest deadline 2024-09-11T23:05:05.036987+0000) 2024-09-11T23:05:22.788 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:22 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10[59957]: 2024-09-11T23:05:22.498+0000 7fb2c8587640 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:49.539173+0000 front 2024-09-11T23:04:49.539118+0000 (oldest deadline 2024-09-11T23:05:12.439229+0000) 2024-09-11T23:05:23.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:22 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:22.591+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:23.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:22 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:22.591+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:23.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:22 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:22.591+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:23.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:22 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:22.591+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:23.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:22 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:22.591+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:23.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:22 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:22.591+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:23.757 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:23 smithi191 podman[71450]: 2024-09-11 23:05:23.489297569 +0000 UTC m=+5.604641273 container died d9909006d101da2d7d7df758e3bc439ebde83a36aaf815853bc6787c0f967906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-11T23:05:24.000 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:23 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:23.601+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:24.001 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:23 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:23.601+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:24.001 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:23 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:23.601+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:24.001 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:23 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:23.601+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:24.001 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:23 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:23.601+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:24.001 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:23 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:23.601+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:24.001 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:23 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:23.601+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6820 osd.7 since back 2024-09-11T23:04:58.648863+0000 front 2024-09-11T23:04:58.648886+0000 (oldest deadline 2024-09-11T23:05:23.348779+0000) 2024-09-11T23:05:24.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:23 smithi191 podman[71450]: 2024-09-11 23:05:23.832036202 +0000 UTC m=+5.947379884 container cleanup d9909006d101da2d7d7df758e3bc439ebde83a36aaf815853bc6787c0f967906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-11T23:05:24.037 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:23 smithi191 bash[71450]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10 2024-09-11T23:05:24.492 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:24 smithi191 podman[71464]: 2024-09-11 23:05:24.141626597 +0000 UTC m=+0.653031991 container remove d9909006d101da2d7d7df758e3bc439ebde83a36aaf815853bc6787c0f967906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T23:05:24.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:24 smithi191 podman[71550]: 2024-09-11 23:05:24.395261481 +0000 UTC m=+0.015379863 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:05:24.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:24 smithi191 podman[71550]: 2024-09-11 23:05:24.500544635 +0000 UTC m=+0.120663006 container create 0028519df2fc791fc2ff4be268b43686596a1bd1f6c5e986d38c96bebe10cb4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-11T23:05:24.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:24 smithi191 podman[71550]: 2024-09-11 23:05:24.736254322 +0000 UTC m=+0.356372694 container init 0028519df2fc791fc2ff4be268b43686596a1bd1f6c5e986d38c96bebe10cb4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:05:24.787 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:24 smithi191 podman[71550]: 2024-09-11 23:05:24.739572556 +0000 UTC m=+0.359690936 container start 0028519df2fc791fc2ff4be268b43686596a1bd1f6c5e986d38c96bebe10cb4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T23:05:25.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:24 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:24.616+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:25.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:24 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:24.616+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:25.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:24 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:24.616+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:25.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:24 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:24.616+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:25.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:24 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:24.616+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:25.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:24 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:24.616+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:25.046 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:24 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:24.616+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6820 osd.7 since back 2024-09-11T23:04:58.648863+0000 front 2024-09-11T23:04:58.648886+0000 (oldest deadline 2024-09-11T23:05:23.348779+0000) 2024-09-11T23:05:25.222 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:24 smithi191 podman[71550]: 2024-09-11 23:05:24.81950551 +0000 UTC m=+0.439623887 container attach 0028519df2fc791fc2ff4be268b43686596a1bd1f6c5e986d38c96bebe10cb4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:05:25.222 INFO:journalctl@ceph.osd.10.smithi191.stdout:Sep 11 23:05:24 smithi191 podman[71550]: 2024-09-11 23:05:24.93346723 +0000 UTC m=+0.553585608 container died 0028519df2fc791fc2ff4be268b43686596a1bd1f6c5e986d38c96bebe10cb4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T23:05:25.531 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.10.service' 2024-09-11T23:05:25.569 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:05:25.569 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-11T23:05:25.569 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-11T23:05:25.569 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.11 2024-09-11T23:05:25.870 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:25 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:25.578+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:25.870 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:25 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:25.578+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:25.870 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:25 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:25.578+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:25.870 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:25 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:25.578+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:25.870 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:25 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:25.578+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:25.870 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:25 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:25.578+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:25.870 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:25 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:25.578+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6820 osd.7 since back 2024-09-11T23:04:58.648863+0000 front 2024-09-11T23:04:58.648886+0000 (oldest deadline 2024-09-11T23:05:23.348779+0000) 2024-09-11T23:05:25.870 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:25 smithi204 systemd[1]: Stopping Ceph osd.11 for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:05:25.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:25 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:25.869+0000 7f95b9395640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:05:26.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:25 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:25.869+0000 7f95b9395640 -1 osd.11 95 *** Got signal Terminated *** 2024-09-11T23:05:26.296 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:25 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:25.869+0000 7f95b9395640 -1 osd.11 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T23:05:26.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:26 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:26.544+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:26.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:26 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:26.544+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:26.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:26 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:26.544+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:26.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:26 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:26.544+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:26.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:26 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:26.544+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:26.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:26 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:26.544+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:26.797 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:26 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:26.544+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6820 osd.7 since back 2024-09-11T23:04:58.648863+0000 front 2024-09-11T23:04:58.648886+0000 (oldest deadline 2024-09-11T23:05:23.348779+0000) 2024-09-11T23:05:27.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:27 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:27.530+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:27.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:27 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:27.530+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:27.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:27 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:27.530+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:27.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:27 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:27.530+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:27.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:27 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:27.530+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:27.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:27 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:27.530+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:27.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:27 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:27.530+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6820 osd.7 since back 2024-09-11T23:04:58.648863+0000 front 2024-09-11T23:04:58.648886+0000 (oldest deadline 2024-09-11T23:05:23.348779+0000) 2024-09-11T23:05:28.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:28 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:28.498+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:28.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:28 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:28.498+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:28.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:28 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:28.498+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:28.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:28 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:28.498+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:28.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:28 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:28.498+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:28.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:28 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:28.498+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:28.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:28 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:28.498+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6820 osd.7 since back 2024-09-11T23:04:58.648863+0000 front 2024-09-11T23:04:58.648886+0000 (oldest deadline 2024-09-11T23:05:23.348779+0000) 2024-09-11T23:05:29.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:29 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:29.482+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:29.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:29 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:29.482+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:29.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:29 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:29.482+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:29.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:29 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:29.482+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:29.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:29 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:29.482+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:29.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:29 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:29.482+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:29.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:29 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:29.482+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6820 osd.7 since back 2024-09-11T23:04:58.648863+0000 front 2024-09-11T23:04:58.648886+0000 (oldest deadline 2024-09-11T23:05:23.348779+0000) 2024-09-11T23:05:29.797 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:29 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:29.482+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-09-11T23:05:03.349118+0000 front 2024-09-11T23:05:03.349188+0000 (oldest deadline 2024-09-11T23:05:29.249162+0000) 2024-09-11T23:05:30.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:30 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:30.439+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6806 osd.0 since back 2024-09-11T23:04:01.440267+0000 front 2024-09-11T23:04:01.440289+0000 (oldest deadline 2024-09-11T23:04:23.740329+0000) 2024-09-11T23:05:30.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:30 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:30.439+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6804 osd.1 since back 2024-09-11T23:04:04.841459+0000 front 2024-09-11T23:04:04.841297+0000 (oldest deadline 2024-09-11T23:04:34.841979+0000) 2024-09-11T23:05:30.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:30 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:30.439+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-09-11T23:04:14.842423+0000 front 2024-09-11T23:04:14.842459+0000 (oldest deadline 2024-09-11T23:04:40.742628+0000) 2024-09-11T23:05:30.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:30 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:30.439+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6812 osd.4 since back 2024-09-11T23:04:32.845886+0000 front 2024-09-11T23:04:32.845798+0000 (oldest deadline 2024-09-11T23:04:56.945863+0000) 2024-09-11T23:05:30.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:30 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:30.439+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-09-11T23:04:38.046973+0000 front 2024-09-11T23:04:38.046904+0000 (oldest deadline 2024-09-11T23:05:03.946987+0000) 2024-09-11T23:05:30.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:30 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:30.439+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.152:6822 osd.6 since back 2024-09-11T23:04:46.848124+0000 front 2024-09-11T23:04:46.848194+0000 (oldest deadline 2024-09-11T23:05:12.747948+0000) 2024-09-11T23:05:30.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:30 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:30.439+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.191:6820 osd.7 since back 2024-09-11T23:04:58.648863+0000 front 2024-09-11T23:04:58.648886+0000 (oldest deadline 2024-09-11T23:05:23.348779+0000) 2024-09-11T23:05:30.797 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:30 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11[60795]: 2024-09-11T23:05:30.439+0000 7f95b19a3640 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-09-11T23:05:03.349118+0000 front 2024-09-11T23:05:03.349188+0000 (oldest deadline 2024-09-11T23:05:29.249162+0000) 2024-09-11T23:05:31.545 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:31 smithi204 podman[72420]: 2024-09-11 23:05:31.282543995 +0000 UTC m=+5.514622384 container died e9a4d88e1eb109d6b33c8277f0d3b06e9e495388e3d01e3a8288e4502b79311c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-11T23:05:31.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:31 smithi204 podman[72420]: 2024-09-11 23:05:31.63571792 +0000 UTC m=+5.867796310 container cleanup e9a4d88e1eb109d6b33c8277f0d3b06e9e495388e3d01e3a8288e4502b79311c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:05:31.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:31 smithi204 bash[72420]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11 2024-09-11T23:05:32.256 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:31 smithi204 podman[72435]: 2024-09-11 23:05:31.895264492 +0000 UTC m=+0.610289481 container remove e9a4d88e1eb109d6b33c8277f0d3b06e9e495388e3d01e3a8288e4502b79311c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True) 2024-09-11T23:05:32.523 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:32 smithi204 podman[72526]: 2024-09-11 23:05:32.158743812 +0000 UTC m=+0.015556452 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T23:05:32.523 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:32 smithi204 podman[72526]: 2024-09-11 23:05:32.271032971 +0000 UTC m=+0.127845600 container create a9d229992f06e59cea26b32e17eb81fd11c2ac29e7fcfbb8fd897150c35a3f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:05:32.523 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:32 smithi204 podman[72526]: 2024-09-11 23:05:32.523556164 +0000 UTC m=+0.380368792 container init a9d229992f06e59cea26b32e17eb81fd11c2ac29e7fcfbb8fd897150c35a3f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T23:05:32.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:32 smithi204 podman[72526]: 2024-09-11 23:05:32.526840823 +0000 UTC m=+0.383653453 container start a9d229992f06e59cea26b32e17eb81fd11c2ac29e7fcfbb8fd897150c35a3f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-11T23:05:32.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:32 smithi204 podman[72526]: 2024-09-11 23:05:32.606888748 +0000 UTC m=+0.463701385 container attach a9d229992f06e59cea26b32e17eb81fd11c2ac29e7fcfbb8fd897150c35a3f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T23:05:32.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Sep 11 23:05:32 smithi204 podman[72526]: 2024-09-11 23:05:32.711299844 +0000 UTC m=+0.568112490 container died a9d229992f06e59cea26b32e17eb81fd11c2ac29e7fcfbb8fd897150c35a3f0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T23:05:33.275 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@osd.11.service' 2024-09-11T23:05:33.314 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:05:33.314 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-11T23:05:33.315 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-11T23:05:33.315 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.a 2024-09-11T23:05:33.617 INFO:journalctl@ceph.mds.a.smithi152.stdout:Sep 11 23:05:33 smithi152 systemd[1]: Stopping Ceph mds.a for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:05:33.872 INFO:journalctl@ceph.mds.a.smithi152.stdout:Sep 11 23:05:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-a[70022]: 2024-09-11T23:05:33.616+0000 7f7edff42640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:05:33.872 INFO:journalctl@ceph.mds.a.smithi152.stdout:Sep 11 23:05:33 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-a[70022]: 2024-09-11T23:05:33.616+0000 7f7edff42640 -1 mds.a *** got signal Terminated *** 2024-09-11T23:05:37.008 INFO:journalctl@ceph.mds.a.smithi152.stdout:Sep 11 23:05:36 smithi152 podman[112513]: 2024-09-11 23:05:36.706790259 +0000 UTC m=+3.200739905 container died 6c6fb2d53bc1286507d9283af1af80cbb830cd9dbac18dc250e4b80eb65ac8af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-11T23:05:37.308 INFO:journalctl@ceph.mds.a.smithi152.stdout:Sep 11 23:05:37 smithi152 podman[112513]: 2024-09-11 23:05:37.107569766 +0000 UTC m=+3.601519412 container cleanup 6c6fb2d53bc1286507d9283af1af80cbb830cd9dbac18dc250e4b80eb65ac8af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-11T23:05:37.308 INFO:journalctl@ceph.mds.a.smithi152.stdout:Sep 11 23:05:37 smithi152 bash[112513]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-a 2024-09-11T23:05:37.543 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.a.service' 2024-09-11T23:05:37.565 INFO:journalctl@ceph.mds.a.smithi152.stdout:Sep 11 23:05:37 smithi152 podman[112525]: 2024-09-11 23:05:37.308712001 +0000 UTC m=+0.603228002 container remove 6c6fb2d53bc1286507d9283af1af80cbb830cd9dbac18dc250e4b80eb65ac8af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T23:05:37.566 INFO:journalctl@ceph.mds.a.smithi152.stdout:Sep 11 23:05:37 smithi152 systemd[1]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.a.service: Deactivated successfully. 2024-09-11T23:05:37.566 INFO:journalctl@ceph.mds.a.smithi152.stdout:Sep 11 23:05:37 smithi152 systemd[1]: Stopped Ceph mds.a for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T23:05:37.566 INFO:journalctl@ceph.mds.a.smithi152.stdout:Sep 11 23:05:37 smithi152 systemd[1]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.a.service: Consumed 1.852s CPU time. 2024-09-11T23:05:37.578 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:05:37.579 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-11T23:05:37.579 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-11T23:05:37.579 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.d 2024-09-11T23:05:37.872 INFO:journalctl@ceph.mds.d.smithi152.stdout:Sep 11 23:05:37 smithi152 systemd[1]: Stopping Ceph mds.d for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:05:38.372 INFO:journalctl@ceph.mds.d.smithi152.stdout:Sep 11 23:05:37 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-d[68842]: 2024-09-11T23:05:37.911+0000 7fe2a25b4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:05:38.372 INFO:journalctl@ceph.mds.d.smithi152.stdout:Sep 11 23:05:37 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-d[68842]: 2024-09-11T23:05:37.911+0000 7fe2a25b4640 -1 mds.d *** got signal Terminated *** 2024-09-11T23:05:40.811 INFO:journalctl@ceph.mds.d.smithi152.stdout:Sep 11 23:05:40 smithi152 podman[112634]: 2024-09-11 23:05:40.555802082 +0000 UTC m=+2.758383591 container died f8beebbb7bf485156b398ebbe61ae863f3635fcf1b886435124becc4af18679d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-d, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T23:05:41.095 INFO:journalctl@ceph.mds.d.smithi152.stdout:Sep 11 23:05:40 smithi152 podman[112634]: 2024-09-11 23:05:40.894228378 +0000 UTC m=+3.096809878 container cleanup f8beebbb7bf485156b398ebbe61ae863f3635fcf1b886435124becc4af18679d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T23:05:41.095 INFO:journalctl@ceph.mds.d.smithi152.stdout:Sep 11 23:05:40 smithi152 bash[112634]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-d 2024-09-11T23:05:41.316 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.d.service' 2024-09-11T23:05:41.348 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:05:41.348 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-11T23:05:41.349 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-11T23:05:41.349 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.g 2024-09-11T23:05:41.622 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 23:05:41 smithi152 systemd[1]: Stopping Ceph mds.g for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:05:42.122 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 23:05:41 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-g[69612]: 2024-09-11T23:05:41.655+0000 7f8b0e1f3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:05:42.122 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 23:05:41 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-g[69612]: 2024-09-11T23:05:41.655+0000 7f8b0e1f3640 -1 mds.g *** got signal Terminated *** 2024-09-11T23:05:44.997 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 23:05:44 smithi152 podman[112752]: 2024-09-11 23:05:44.704860534 +0000 UTC m=+3.143880250 container died 208d85f65604d98645c29e22d739bbf8b07363a197071857d0fa05a5c4e1bf63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T23:05:45.372 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 23:05:45 smithi152 podman[112752]: 2024-09-11 23:05:45.189341937 +0000 UTC m=+3.628361651 container cleanup 208d85f65604d98645c29e22d739bbf8b07363a197071857d0fa05a5c4e1bf63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-11T23:05:45.372 INFO:journalctl@ceph.mds.g.smithi152.stdout:Sep 11 23:05:45 smithi152 bash[112752]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-g 2024-09-11T23:05:45.612 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.g.service' 2024-09-11T23:05:45.644 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:05:45.645 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-11T23:05:45.645 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-11T23:05:45.645 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.j 2024-09-11T23:05:46.122 INFO:journalctl@ceph.mds.j.smithi152.stdout:Sep 11 23:05:45 smithi152 systemd[1]: Stopping Ceph mds.j for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:05:46.122 INFO:journalctl@ceph.mds.j.smithi152.stdout:Sep 11 23:05:45 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-j[68452]: 2024-09-11T23:05:45.975+0000 7ff47d6d3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:05:46.122 INFO:journalctl@ceph.mds.j.smithi152.stdout:Sep 11 23:05:45 smithi152 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-j[68452]: 2024-09-11T23:05:45.975+0000 7ff47d6d3640 -1 mds.j *** got signal Terminated *** 2024-09-11T23:05:49.428 INFO:journalctl@ceph.mds.j.smithi152.stdout:Sep 11 23:05:49 smithi152 podman[112874]: 2024-09-11 23:05:49.136165262 +0000 UTC m=+3.263825164 container died 829e5f06a64d2a51f3cc45318bcbfefc973f0c773f1ef16965192671a0786b8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T23:05:49.844 INFO:journalctl@ceph.mds.j.smithi152.stdout:Sep 11 23:05:49 smithi152 podman[112874]: 2024-09-11 23:05:49.601608929 +0000 UTC m=+3.729268830 container cleanup 829e5f06a64d2a51f3cc45318bcbfefc973f0c773f1ef16965192671a0786b8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-j, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-11T23:05:49.844 INFO:journalctl@ceph.mds.j.smithi152.stdout:Sep 11 23:05:49 smithi152 bash[112874]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-j 2024-09-11T23:05:49.970 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.j.service' 2024-09-11T23:05:50.002 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:05:50.002 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-11T23:05:50.003 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-11T23:05:50.003 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.b 2024-09-11T23:05:50.287 INFO:journalctl@ceph.mds.b.smithi191.stdout:Sep 11 23:05:50 smithi191 systemd[1]: Stopping Ceph mds.b for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:05:50.597 INFO:journalctl@ceph.mds.b.smithi191.stdout:Sep 11 23:05:50 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-b[65317]: 2024-09-11T23:05:50.322+0000 7fc8d9907640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:05:50.597 INFO:journalctl@ceph.mds.b.smithi191.stdout:Sep 11 23:05:50 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-b[65317]: 2024-09-11T23:05:50.322+0000 7fc8d9907640 -1 mds.b *** got signal Terminated *** 2024-09-11T23:05:52.989 INFO:journalctl@ceph.mds.b.smithi191.stdout:Sep 11 23:05:52 smithi191 podman[71652]: 2024-09-11 23:05:52.70614384 +0000 UTC m=+2.496471797 container died d511cfc17a7eba62d45b7c122d9b3690e1457e8cef38aced4b403b2bee784ba5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T23:05:53.287 INFO:journalctl@ceph.mds.b.smithi191.stdout:Sep 11 23:05:53 smithi191 podman[71652]: 2024-09-11 23:05:53.097038078 +0000 UTC m=+2.887366025 container cleanup d511cfc17a7eba62d45b7c122d9b3690e1457e8cef38aced4b403b2bee784ba5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-11T23:05:53.287 INFO:journalctl@ceph.mds.b.smithi191.stdout:Sep 11 23:05:53 smithi191 bash[71652]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-b 2024-09-11T23:05:53.462 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.b.service' 2024-09-11T23:05:53.498 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:05:53.499 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-11T23:05:53.499 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-11T23:05:53.499 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.e 2024-09-11T23:05:53.786 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 23:05:53 smithi191 systemd[1]: Stopping Ceph mds.e for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:05:54.286 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 23:05:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-e[64705]: 2024-09-11T23:05:53.835+0000 7f338a4bb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:05:54.553 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 23:05:53 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-e[64705]: 2024-09-11T23:05:53.835+0000 7f338a4bb640 -1 mds.e *** got signal Terminated *** 2024-09-11T23:05:56.992 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 23:05:56 smithi191 podman[71785]: 2024-09-11 23:05:56.719952659 +0000 UTC m=+3.006085300 container died 4f2b9716349a5467093a94829eaa05189479205786dcf3f6360f7b04df45a94c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T23:05:57.287 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 23:05:57 smithi191 podman[71785]: 2024-09-11 23:05:57.075407986 +0000 UTC m=+3.361540627 container cleanup 4f2b9716349a5467093a94829eaa05189479205786dcf3f6360f7b04df45a94c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-11T23:05:57.287 INFO:journalctl@ceph.mds.e.smithi191.stdout:Sep 11 23:05:57 smithi191 bash[71785]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-e 2024-09-11T23:05:57.513 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.e.service' 2024-09-11T23:05:57.546 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:05:57.546 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-11T23:05:57.546 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-11T23:05:57.547 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.h 2024-09-11T23:05:57.870 INFO:journalctl@ceph.mds.h.smithi191.stdout:Sep 11 23:05:57 smithi191 systemd[1]: Stopping Ceph mds.h for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:05:58.287 INFO:journalctl@ceph.mds.h.smithi191.stdout:Sep 11 23:05:57 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-h[65006]: 2024-09-11T23:05:57.870+0000 7f569cfbb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:05:58.287 INFO:journalctl@ceph.mds.h.smithi191.stdout:Sep 11 23:05:57 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-h[65006]: 2024-09-11T23:05:57.870+0000 7f569cfbb640 -1 mds.h *** got signal Terminated *** 2024-09-11T23:06:01.004 INFO:journalctl@ceph.mds.h.smithi191.stdout:Sep 11 23:06:00 smithi191 podman[71906]: 2024-09-11 23:06:00.707377922 +0000 UTC m=+2.947536902 container died 348b5c3bc7079dba28992d5944647ed11b2c667490c04e977cbfbb6ec8593c98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:06:01.287 INFO:journalctl@ceph.mds.h.smithi191.stdout:Sep 11 23:06:01 smithi191 podman[71906]: 2024-09-11 23:06:01.086973362 +0000 UTC m=+3.327132341 container cleanup 348b5c3bc7079dba28992d5944647ed11b2c667490c04e977cbfbb6ec8593c98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-h, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:06:01.287 INFO:journalctl@ceph.mds.h.smithi191.stdout:Sep 11 23:06:01 smithi191 bash[71906]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-h 2024-09-11T23:06:01.517 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.h.service' 2024-09-11T23:06:01.549 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:06:01.549 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-11T23:06:01.550 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-11T23:06:01.550 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.k 2024-09-11T23:06:01.957 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 23:06:01 smithi191 systemd[1]: Stopping Ceph mds.k for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:06:02.286 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 23:06:01 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-k[64393]: 2024-09-11T23:06:01.956+0000 7faac3bc8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:06:02.287 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 23:06:01 smithi191 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-k[64393]: 2024-09-11T23:06:01.956+0000 7faac3bc8640 -1 mds.k *** got signal Terminated *** 2024-09-11T23:06:04.991 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 23:06:04 smithi191 podman[72027]: 2024-09-11 23:06:04.719612049 +0000 UTC m=+2.944912517 container died c100af8ff245fc7ca2e650cfef1586a8d9a65eeaf1aebbb9c5f758902de2587c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-k, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-11T23:06:05.287 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 23:06:05 smithi191 podman[72027]: 2024-09-11 23:06:05.074006215 +0000 UTC m=+3.299306665 container cleanup c100af8ff245fc7ca2e650cfef1586a8d9a65eeaf1aebbb9c5f758902de2587c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-k, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:06:05.287 INFO:journalctl@ceph.mds.k.smithi191.stdout:Sep 11 23:06:05 smithi191 bash[72027]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-k 2024-09-11T23:06:05.496 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.k.service' 2024-09-11T23:06:05.527 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:06:05.528 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-11T23:06:05.528 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-11T23:06:05.528 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.c 2024-09-11T23:06:05.919 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 23:06:05 smithi204 systemd[1]: Stopping Ceph mds.c for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:06:06.181 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 23:06:05 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-c[65804]: 2024-09-11T23:06:05.918+0000 7fc63861d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:06:06.181 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 23:06:05 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-c[65804]: 2024-09-11T23:06:05.918+0000 7fc63861d640 -1 mds.c *** got signal Terminated *** 2024-09-11T23:06:09.046 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 23:06:08 smithi204 podman[72629]: 2024-09-11 23:06:08.716026974 +0000 UTC m=+2.996364671 container died 4e76b0e1763970001f661a6c7d07b59903d78880bd40ecd99c33f7f5225aef06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T23:06:09.427 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 23:06:09 smithi204 podman[72629]: 2024-09-11 23:06:09.193400483 +0000 UTC m=+3.473738170 container cleanup 4e76b0e1763970001f661a6c7d07b59903d78880bd40ecd99c33f7f5225aef06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-11T23:06:09.428 INFO:journalctl@ceph.mds.c.smithi204.stdout:Sep 11 23:06:09 smithi204 bash[72629]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-c 2024-09-11T23:06:09.562 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.c.service' 2024-09-11T23:06:09.598 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:06:09.599 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-11T23:06:09.599 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-11T23:06:09.600 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.f 2024-09-11T23:06:09.796 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 23:06:09 smithi204 systemd[1]: Stopping Ceph mds.f for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:06:10.296 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 23:06:09 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-f[65501]: 2024-09-11T23:06:09.914+0000 7f0212c84640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:06:10.296 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 23:06:09 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-f[65501]: 2024-09-11T23:06:09.914+0000 7f0212c84640 -1 mds.f *** got signal Terminated *** 2024-09-11T23:06:12.998 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 23:06:12 smithi204 podman[72766]: 2024-09-11 23:06:12.724780927 +0000 UTC m=+2.906626519 container died 57177cca0c8d4135367c3ab1fce8e7591499c9ea6103cf44b69233724656e236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-f, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T23:06:13.295 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 23:06:13 smithi204 podman[72766]: 2024-09-11 23:06:13.105759359 +0000 UTC m=+3.287604946 container cleanup 57177cca0c8d4135367c3ab1fce8e7591499c9ea6103cf44b69233724656e236 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:06:13.296 INFO:journalctl@ceph.mds.f.smithi204.stdout:Sep 11 23:06:13 smithi204 bash[72766]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-f 2024-09-11T23:06:13.530 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.f.service' 2024-09-11T23:06:13.564 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:06:13.564 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-11T23:06:13.564 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-11T23:06:13.565 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.i 2024-09-11T23:06:14.046 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 23:06:13 smithi204 systemd[1]: Stopping Ceph mds.i for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:06:14.046 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 23:06:13 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-i[66104]: 2024-09-11T23:06:13.884+0000 7f06f5abc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:06:14.046 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 23:06:13 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-i[66104]: 2024-09-11T23:06:13.884+0000 7f06f5abc640 -1 mds.i *** got signal Terminated *** 2024-09-11T23:06:17.046 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 23:06:16 smithi204 podman[72887]: 2024-09-11 23:06:16.710345424 +0000 UTC m=+2.930219686 container died fd6b87b2557e00e6a8f3d1833ab21091c9fe89ff0121bebe31465b67cecab0d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-i, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T23:06:17.346 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 23:06:17 smithi204 podman[72887]: 2024-09-11 23:06:17.144941814 +0000 UTC m=+3.364816064 container cleanup fd6b87b2557e00e6a8f3d1833ab21091c9fe89ff0121bebe31465b67cecab0d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-i, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T23:06:17.346 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 23:06:17 smithi204 bash[72887]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-i 2024-09-11T23:06:17.574 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.i.service' 2024-09-11T23:06:17.597 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 23:06:17 smithi204 podman[72900]: 2024-09-11 23:06:17.346316319 +0000 UTC m=+0.636108012 container remove fd6b87b2557e00e6a8f3d1833ab21091c9fe89ff0121bebe31465b67cecab0d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T23:06:17.597 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 23:06:17 smithi204 systemd[1]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.i.service: Deactivated successfully. 2024-09-11T23:06:17.597 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 23:06:17 smithi204 systemd[1]: Stopped Ceph mds.i for 0e4e82bc-708f-11ef-bcea-c7b262605968. 2024-09-11T23:06:17.597 INFO:journalctl@ceph.mds.i.smithi204.stdout:Sep 11 23:06:17 smithi204 systemd[1]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.i.service: Consumed 1.892s CPU time. 2024-09-11T23:06:17.607 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:06:17.608 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-11T23:06:17.608 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-11T23:06:17.608 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.l 2024-09-11T23:06:17.907 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 11 23:06:17 smithi204 systemd[1]: Stopping Ceph mds.l for 0e4e82bc-708f-11ef-bcea-c7b262605968... 2024-09-11T23:06:18.296 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 11 23:06:17 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-l[65197]: 2024-09-11T23:06:17.907+0000 7f25d71cf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T23:06:18.296 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 11 23:06:17 smithi204 ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-l[65197]: 2024-09-11T23:06:17.907+0000 7f25d71cf640 -1 mds.l *** got signal Terminated *** 2024-09-11T23:06:20.983 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 11 23:06:20 smithi204 podman[73009]: 2024-09-11 23:06:20.723282076 +0000 UTC m=+2.898554215 container died 122efa3fbe8b6996d48cec63ce8db75da1d8e96def75f564dee282214ce14531 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-l, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T23:06:21.295 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 11 23:06:21 smithi204 podman[73009]: 2024-09-11 23:06:21.082132662 +0000 UTC m=+3.257404797 container cleanup 122efa3fbe8b6996d48cec63ce8db75da1d8e96def75f564dee282214ce14531 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-l, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-11T23:06:21.296 INFO:journalctl@ceph.mds.l.smithi204.stdout:Sep 11 23:06:21 smithi204 bash[73009]: ceph-0e4e82bc-708f-11ef-bcea-c7b262605968-mds-l 2024-09-11T23:06:21.471 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0e4e82bc-708f-11ef-bcea-c7b262605968@mds.l.service' 2024-09-11T23:06:21.503 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T23:06:21.503 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-11T23:06:21.504 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T23:06:21.711 INFO:teuthology.orchestra.run.smithi152.stdout:Deleting cluster with fsid: 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T23:06:24.219 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T23:06:24.420 INFO:teuthology.orchestra.run.smithi191.stdout:Deleting cluster with fsid: 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T23:06:26.871 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T23:06:27.072 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T23:06:29.434 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:06:29.468 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:06:29.501 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T23:06:29.528 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T23:06:29.531 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901430/remote/smithi152/crash 2024-09-11T23:06:29.531 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/crash -- . 2024-09-11T23:06:29.573 INFO:teuthology.orchestra.run.smithi152.stderr:tar: /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T23:06:29.573 INFO:teuthology.orchestra.run.smithi152.stderr:tar: Error is not recoverable: exiting now 2024-09-11T23:06:29.575 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901430/remote/smithi191/crash 2024-09-11T23:06:29.576 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/crash -- . 2024-09-11T23:06:29.614 INFO:teuthology.orchestra.run.smithi191.stderr:tar: /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T23:06:29.614 INFO:teuthology.orchestra.run.smithi191.stderr:tar: Error is not recoverable: exiting now 2024-09-11T23:06:29.616 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901430/remote/smithi204/crash 2024-09-11T23:06:29.617 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/crash -- . 2024-09-11T23:06:29.652 INFO:teuthology.orchestra.run.smithi204.stderr:tar: /var/lib/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T23:06:29.652 INFO:teuthology.orchestra.run.smithi204.stderr:tar: Error is not recoverable: exiting now 2024-09-11T23:06:29.653 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T23:06:29.654 DEBUG:teuthology.orchestra.run.smithi152:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-11T23:06:29.691 INFO:tasks.cephadm:Compressing logs... 2024-09-11T23:06:29.691 DEBUG:teuthology.orchestra.run.smithi152:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T23:06:29.734 DEBUG:teuthology.orchestra.run.smithi191:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T23:06:29.737 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T23:06:29.762 INFO:teuthology.orchestra.run.smithi152.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T23:06:29.762 INFO:teuthology.orchestra.run.smithi152.stderr:: No such file or directory 2024-09-11T23:06:29.762 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-11T23:06:29.762 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose --/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T23:06:29.762 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T23:06:29.772 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-11T23:06:29.772 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T23:06:29.772 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.audit.log: /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T23:06:29.772 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-11T23:06:29.772 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mon.a.log: 84.6%gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T23:06:29.773 INFO:teuthology.orchestra.run.smithi152.stderr: -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T23:06:29.773 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.log 2024-09-11T23:06:29.773 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.g.log: /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.6.log: 90.4% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T23:06:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T23:06:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:: No such file or directory 2024-09-11T23:06:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T23:06:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-11T23:06:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T23:06:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T23:06:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-11T23:06:29.784 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T23:06:29.785 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip 82.0% -5 --verbose -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.cephadm.log.gz -- 2024-09-11T23:06:29.785 INFO:teuthology.orchestra.run.smithi204.stderr: /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T23:06:29.785 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.log 2024-09-11T23:06:29.785 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-11T23:06:29.785 INFO:teuthology.orchestra.run.smithi204.stderr: 90.5% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T23:06:29.785 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.log: 87.2% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T23:06:29.785 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-11T23:06:29.785 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-11T23:06:29.785 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.l.log: /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.8.log: 91.0% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T23:06:29.785 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-11T23:06:29.789 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mgr.z.log: 91.9% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-11T23:06:29.790 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-11T23:06:29.791 INFO:teuthology.orchestra.run.smithi152.stderr: 90.9% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T23:06:29.791 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-11T23:06:29.791 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T23:06:29.791 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.d.log: 87.2% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T23:06:29.791 INFO:teuthology.orchestra.run.smithi152.stderr: 94.6% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-11T23:06:29.791 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-11T23:06:29.791 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T23:06:29.791 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.j.log: 94.7% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-11T23:06:29.791 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T23:06:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-11T23:06:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-11T23:06:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T23:06:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T23:06:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.10.log: /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T23:06:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-11T23:06:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-11T23:06:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T23:06:29.795 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T23:06:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T23:06:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.audit.log: 90.5% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T23:06:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T23:06:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.cephadm.log: 82.1% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T23:06:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.log 2024-09-11T23:06:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mgr.y.log: 91.0% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T23:06:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-11T23:06:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.log: 91.9% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T23:06:29.796 INFO:teuthology.orchestra.run.smithi191.stderr: 87.2%gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T23:06:29.796 INFO:teuthology.orchestra.run.smithi191.stderr: -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T23:06:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-11T23:06:29.796 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T23:06:29.800 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T23:06:29.812 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T23:06:29.819 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T23:06:29.825 INFO:teuthology.orchestra.run.smithi152.stderr: 92.5% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-11T23:06:29.825 INFO:teuthology.orchestra.run.smithi152.stderr: 94.5% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-11T23:06:29.826 INFO:teuthology.orchestra.run.smithi204.stderr: 92.7% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-11T23:06:29.829 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T23:06:29.832 INFO:teuthology.orchestra.run.smithi191.stderr: 94.5% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-11T23:06:29.845 INFO:teuthology.orchestra.run.smithi204.stderr: 94.4% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-11T23:06:29.854 INFO:teuthology.orchestra.run.smithi191.stderr: 92.6% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-11T23:06:29.855 INFO:teuthology.orchestra.run.smithi204.stderr: 92.4% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-11T23:06:29.879 INFO:teuthology.orchestra.run.smithi191.stderr: 94.5% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-11T23:06:30.013 INFO:teuthology.orchestra.run.smithi204.stderr: 96.4% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-11T23:06:30.119 INFO:teuthology.orchestra.run.smithi204.stderr: 92.0% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T23:06:30.140 INFO:teuthology.orchestra.run.smithi152.stderr: 90.1% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T23:06:30.208 INFO:teuthology.orchestra.run.smithi191.stderr: 91.8% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T23:06:30.611 INFO:teuthology.orchestra.run.smithi152.stderr: 90.5% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T23:06:32.447 INFO:teuthology.orchestra.run.smithi191.stderr: 94.9% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-11T23:06:33.938 INFO:teuthology.orchestra.run.smithi152.stderr: 93.5% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-11T23:06:34.732 INFO:teuthology.orchestra.run.smithi204.stderr: 93.3% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-11T23:06:35.152 INFO:teuthology.orchestra.run.smithi152.stderr: 93.5% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T23:06:35.171 INFO:teuthology.orchestra.run.smithi204.stderr: 93.7% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-11T23:06:35.683 INFO:teuthology.orchestra.run.smithi152.stderr: 93.5% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T23:06:35.798 INFO:teuthology.orchestra.run.smithi191.stderr: 93.3% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T23:06:35.863 INFO:teuthology.orchestra.run.smithi191.stderr: 93.5% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T23:06:35.868 INFO:teuthology.orchestra.run.smithi191.stderr: 93.2% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-11T23:06:35.872 INFO:teuthology.orchestra.run.smithi152.stderr: 93.5% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T23:06:35.874 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-09-11T23:06:35.874 INFO:teuthology.orchestra.run.smithi152.stderr:real 0m6.127s 2024-09-11T23:06:35.874 INFO:teuthology.orchestra.run.smithi152.stderr:user 0m22.059s 2024-09-11T23:06:35.874 INFO:teuthology.orchestra.run.smithi152.stderr:sys 0m0.876s 2024-09-11T23:06:36.327 INFO:teuthology.orchestra.run.smithi191.stderr: 93.7% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T23:06:36.329 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-09-11T23:06:36.329 INFO:teuthology.orchestra.run.smithi191.stderr:real 0m6.578s 2024-09-11T23:06:36.330 INFO:teuthology.orchestra.run.smithi191.stderr:user 0m27.102s 2024-09-11T23:06:36.330 INFO:teuthology.orchestra.run.smithi191.stderr:sys 0m0.993s 2024-09-11T23:06:36.366 INFO:teuthology.orchestra.run.smithi204.stderr: 93.2% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T23:06:36.433 INFO:teuthology.orchestra.run.smithi204.stderr: 93.3% -- replaced with /var/log/ceph/0e4e82bc-708f-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T23:06:36.435 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-09-11T23:06:36.435 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m6.683s 2024-09-11T23:06:36.435 INFO:teuthology.orchestra.run.smithi204.stderr:user 0m23.523s 2024-09-11T23:06:36.435 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m0.879s 2024-09-11T23:06:36.435 INFO:tasks.cephadm:Archiving logs... 2024-09-11T23:06:36.436 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901430/remote/smithi152/log 2024-09-11T23:06:36.437 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T23:06:39.698 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901430/remote/smithi191/log 2024-09-11T23:06:39.699 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T23:06:43.567 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901430/remote/smithi204/log 2024-09-11T23:06:43.568 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T23:06:47.163 INFO:tasks.cephadm:Removing cluster... 2024-09-11T23:06:47.163 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 --force 2024-09-11T23:06:47.364 INFO:teuthology.orchestra.run.smithi152.stdout:Deleting cluster with fsid: 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T23:06:47.652 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 --force 2024-09-11T23:06:47.854 INFO:teuthology.orchestra.run.smithi191.stdout:Deleting cluster with fsid: 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T23:06:48.145 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0e4e82bc-708f-11ef-bcea-c7b262605968 --force 2024-09-11T23:06:48.338 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: 0e4e82bc-708f-11ef-bcea-c7b262605968 2024-09-11T23:06:48.627 INFO:tasks.cephadm:Removing cephadm ... 2024-09-11T23:06:48.627 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T23:06:48.647 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T23:06:48.663 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T23:06:48.679 INFO:tasks.cephadm:Teardown complete 2024-09-11T23:06:48.679 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T23:06:48.691 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T23:06:48.691 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T23:06:48.694 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T23:06:48.706 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T23:06:48.841 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T23:06:48.851 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T23:06:48.852 DEBUG:teuthology.orchestra.run.smithi152:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:06:48.854 DEBUG:teuthology.orchestra.run.smithi191:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:06:48.858 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:06:48.869 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:06:48.871 INFO:teuthology.orchestra.run.smithi191.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:06:48.872 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:06:48.874 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:06:48.874 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-09-11T23:06:48.874 INFO:teuthology.orchestra.run.smithi152.stdout:^* hv01.front.sepia.ceph.com 3 6 377 41 +1307us[+1305us] +/- 47ms 2024-09-11T23:06:48.874 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 38 -1252us[-1252us] +/- 85ms 2024-09-11T23:06:48.874 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -1303us[-1303us] +/- 91ms 2024-09-11T23:06:48.874 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:06:48.876 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:06:48.876 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-09-11T23:06:48.876 INFO:teuthology.orchestra.run.smithi191.stdout:^* hv01.front.sepia.ceph.com 3 6 377 39 +1240us[+1251us] +/- 47ms 2024-09-11T23:06:48.876 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 -1298us[-1298us] +/- 85ms 2024-09-11T23:06:48.876 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -1347us[-1347us] +/- 91ms 2024-09-11T23:06:48.876 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:06:48.877 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:06:48.878 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-09-11T23:06:48.878 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv01.front.sepia.ceph.com 3 6 377 39 +1267us[+1268us] +/- 47ms 2024-09-11T23:06:48.878 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 41 -1289us[-1288us] +/- 85ms 2024-09-11T23:06:48.878 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 -1354us[-1353us] +/- 91ms 2024-09-11T23:06:48.878 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:06:48.878 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T23:06:48.902 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T23:06:48.902 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T23:06:48.912 DEBUG:teuthology.orchestra.run.smithi152:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T23:06:49.000 DEBUG:teuthology.orchestra.run.smithi191:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T23:06:49.070 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T23:06:49.139 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:06:49.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:06:49.170 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:06:49.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:06:49.199 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:06:49.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:06:49.226 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T23:06:49.238 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T23:06:49.252 INFO:teuthology.task.internal:Duration was 1843.586464 seconds 2024-09-11T23:06:49.252 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T23:06:49.265 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T23:06:49.265 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T23:06:49.268 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T23:06:49.270 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T23:06:49.304 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:06:49.306 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:06:49.309 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:06:49.827 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T23:06:49.827 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi152.front.sepia.ceph.com 2024-09-11T23:06:49.827 DEBUG:teuthology.orchestra.run.smithi152:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T23:06:49.848 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi191.front.sepia.ceph.com 2024-09-11T23:06:49.849 DEBUG:teuthology.orchestra.run.smithi191:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T23:06:49.893 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-09-11T23:06:49.893 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T23:06:49.924 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T23:06:49.924 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T23:06:49.927 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T23:06:49.935 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T23:06:49.992 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T23:06:49.992 DEBUG:teuthology.orchestra.run.smithi152:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T23:06:49.995 DEBUG:teuthology.orchestra.run.smithi191:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T23:06:50.002 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T23:06:51.307 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T23:06:51.320 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T23:06:51.320 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T23:06:51.345 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T23:06:51.375 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T23:06:51.406 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T23:06:51.419 DEBUG:teuthology.orchestra.run.smithi152:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:06:51.422 DEBUG:teuthology.orchestra.run.smithi191:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:06:51.425 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:06:51.443 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = core 2024-09-11T23:06:51.452 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = core 2024-09-11T23:06:51.473 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-09-11T23:06:51.680 DEBUG:teuthology.orchestra.run.smithi152:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:06:51.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:06:51.695 DEBUG:teuthology.orchestra.run.smithi191:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:06:51.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:06:51.713 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:06:51.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:06:51.738 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T23:06:51.751 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T23:06:51.752 DEBUG:teuthology.misc:Transferring archived files from smithi152:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901430/remote/smithi152 2024-09-11T23:06:51.752 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T23:06:51.815 DEBUG:teuthology.misc:Transferring archived files from smithi191:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901430/remote/smithi191 2024-09-11T23:06:51.815 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T23:06:51.856 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901430/remote/smithi204 2024-09-11T23:06:51.857 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T23:06:51.898 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T23:06:51.898 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T23:06:51.900 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T23:06:51.903 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T23:06:51.950 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T23:06:51.964 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T23:06:51.964 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T23:06:51.982 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T23:06:51.982 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T23:06:51.985 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T23:06:51.987 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T23:06:51.997 INFO:teuthology.orchestra.run.smithi152.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 23:06 /home/ubuntu/cephtest 2024-09-11T23:06:52.003 INFO:teuthology.orchestra.run.smithi191.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 23:06 /home/ubuntu/cephtest 2024-09-11T23:06:52.007 INFO:teuthology.orchestra.run.smithi204.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 23:06 /home/ubuntu/cephtest 2024-09-11T23:06:52.008 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T23:06:52.021 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T23:06:52.049 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} duration: 1843.5864639282227 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-11T23:06:52.049 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T23:06:52.156 INFO:teuthology.run:pass